Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs
Authors: Xudong Li
Mengdan Zhang
Peixian Chen
Xiawu Zheng
Yan Zhang
Jingyuan Zheng
Yunhang Shen
Ke Li
Chaoyou Fu
Xing Sun
Rongrong Ji
Venue: NeurIPS 2025 poster ()
Keywords: Multi-modal Large Language Models (MLLMs)
Direct Preference Optimization
Hallucination Mitigation
Primary area: applications

Abstract: Multi-modal Large Language Models (MLLMs) excel at single-image tasks but struggle with multi-image understanding due to cross-modal misalignment, leading to hallucinations (context omission, conflation, and misinterpretation). Existing methods using Direct Preference Optimization (DPO) constrain optimization to a solitary image reference within the input sequence, neglecting holistic context modeling. To address this, we propose Context-to-Cue Direct Preference Optimization (CcDPO), a multi-level preference optimization framework that enhances per-image perception in multi-image settings by zooming into visual clues—from sequential context to local details. Our approach features two sequentially dependent components: (i) Context-Level Optimization: By introducing low-cost sequence preference pairs, we optimize the model to distinguish between complete and disrupted multi-image contexts, thereby correcting cognitive biases in MLLMs’  multi-image understanding. (ii) Needle-Level Optimization: By integrating region-specific visual prompts with multimodal preference supervision, we direct the model’s attention to critical visual details, effectively suppressing perceptual biases toward fine-grained visual information. To support scalable optimization, we also construct MultiScope-42k, an automatically generated multi-image dataset with hierarchical preference pairs. Experiments show that CcDPO significantly reduces hallucinations and yields consistent performance gains across general single- and multi-image tasks. Codes are available at https://github.com/LXDxmu/CcDPO.

Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs
Xudong Li Jingyuan Zheng!
Mengdan Zh Peixian Chen
Xiawu Zheng' Xing Sun
Yan Zhang
Yunhang Shen Ke Li
Chaoyou Ful
Rongrong Ji
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China
2 Tencent Youtu Lab Nanjing University
(Ixd761050753, zhangmengdanra) @ gmail.com, (zhengxiawu, mij | @ @xmu.edu.cn
Abstract
Multi-modal Large Language Models (MLLMs) excel at single-image tasks but struggle with multi-image understanding due to cross-modal misalignment, leading to hallucinations (context omission, conflation, and misinterpretation) Existing methods using Direct Preference Optimization (DPO) constrain optimization solitary image reference within the input sequence, neglecting holistic con- text modeling. To address this, we propose Context-to-Cue Direct Preference Optimization (CcDPO), multi-level preference optimization framework that en- hances per-image perception in multi image settings by zooming into visual clues from sequential context to local details. Our approach features two sequentially dependent components: (i) Context-Leve Optimization: By introducing low-coss sequence preference pairs, optimize the model to distinguish between complete and disrupted multi-image contexts, thereby correcting cognitive biases in MI.MMs multi-image understanding. (ii) Needle-Level Optimization: By integrating region- specific visual prompts with multimodal preference supervision, we direct the model's attention to critical visual details, effectively suppressing perceptual biases toward fine-grained visual information. To support scalable optimization, we also construct MultiScope-42k, an automatically generated multi-image dataset with hierarchical preference pairs. Experiments show that CcDPO significantly reduces hallucinations and yields consistent performance gains across general single- and multi-image tasks. Codes are available at https://github.com woom///Dxmu//cJDDa
Introduction
Simultancously understanding multiple images remains fundamental yet underexplored challenge
for Multi-modal Large Language Models (MLLMs) [1, 2, 3, 4]. Despite MLLMs excelling in
single-image tasks like visual question answering (VQA) [5, 6, 7]. code generation [8, 9], and
storytelling [10, 11], and open-source models such as LLaVA [12], BLIP-2 [13], and Intern VL [14] showing competitive results on benchmarks including VQAv2 [15], OKVQA [16]. and MMMU [17], their capabilities in multi-image contexts are notably constrained. These models frequently struggle with tasks demanding cross-image comparison, spatial reasoning, or temporal alignment [18], often resulting in hallucinations like context omission, conflation, and misinterpretation of local details. These deficiencies ultimately compromise model reliability. The root cause lies in the weak cross- modal alignment within MLLMs, which frequently fails to integrate visual and textual information
Equual Contribution.
t Conreppondigg Author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Previous MIA-DPO method
(<) Oversiew of Proposed Hierarchical CcDPO (Ours)
(d) Performance Comparison
Singl Tinks
Nulis-e Tasks
AP
C2UPO (Dorn)
Image
Inieage
Descc Eatielan
Tortaflation (3g (1g) Mars
A/t
Conten
(b)
Comtort Cinission
Contest Confiation
Dotall Misketeerprcteriion
MAIL
which image move af offener Fooo cluncar
Hon man an Para images)
Wast fam onl at pividdi cuaccyy timager
charld fever, The oduree polr
peage 24r1 pine with 2g 21-00
MUSED
7000
Serwen The inage shone ill anidiirs
Peeitemi Thanfoy the ball Lalk
Avediimmn The dan aapeass verly inage
20.8
Figure I: (a) Prior multi-image DPO (e.g., MIA-DPO) is constrained by its reliance on predefined image references and text-only preferences, limiting holistic context modeling. (b) These limitations commonly lead to failures such as Context Omission (ignoring relevant images). Context Conflation (misattributing content across images), and Detail Misinterpretation (misrepresenting fine-grained visual cues). (c) CcDPO addresses these issues by hierarchically enhancing MLLMs' visual percep- contexts to specific fine-grained details. (d) Benchmark comparisons demonstrate CcDPO's improved reasoning capabilities on both multi-image and single-image tasks.
tion, from overall
coherently and comprehensively. This limitation becomes particularly pronounced in multi-image settings, where accurate reasoning requires both: (i) Precise interpretation of intra-image regional details, and (ii) establishing meaningful inter-image connections through contextual integration.
To overcome these limitations, instruction tuning with multi -mage supervision has been adopted in recent models such as Flamingo [19], IDEFICS [20], and Emu2 [21]. However, these approaches rely on large-scale annotated data, which is costly to construct due to the complexity of modeling inter- image relationships. As lightweight alternative, Direct Preference Optimizution (DPO) [22] has emerged as promising training paradigm, aligning model outputs with human preferences through pairwise supervision without requiring large-scale labeled data, significantly reducing reliance on costly annotations. Recent work has extended DPO to multimodal tasks [23, 24, 25, 26], with MIA- DPO [27] specifically pioneering its application to multi-image scenarios. As shown in Fig. (a), MIA-DPO conditions responses on a specific image through explicit query references (e.g.. "In Image what is...?"), helping the model associate questions with the correct visual input in context.
While this anchoring strategy mitigates referential ambiguity in MILMs, the absence of explicit contextual modeling in its architectural design fundamentally limits comprehensive cross-image integration. This constraint further exacerbated by the high vulnerability of inter-image attention mechanisms to multi- interference [28, 29, 30]. Consequently, without explicit image references, such approaches often demonstrate limited capacity in autonomously capturing sequential visual context and fine-grained details, giving rise to multiple forms of multi multi-image hallucinations: Context Omission: The model selectively ignores subsets of input images, generating responses based on incomplete sequences (e.g., ignoring Image 3; Fig. ((b), left). Context Conflation: The model erroneously attributes visual elements across images (e.g.. describing a cat from Image 3 as appearing in Image I; Fig. 1(b), middle). Detail Misinterpretation: Critical visual details in certain image are either missed or misinterpreted (e.g., without explicit image-specific instructions, the model fails to recognize the scissors in Image 2 and erroneously detects silver blades in Image 1; Fig. 1(b), right).
To address these challenges, we propose Context-to-Cuut Direct Preference Optimization (CcDPO), a two-level preference optimization framework that enhances MLLMs' capability to accurately perceive visual information across hierarchical levels-frre sequential multi-image contexts to individual fine-gra fine-grained details (as shown in Fig. I(c)). Specifically, it consists of two levels of alignment:
(1) Context-Level Optimization: We formulate structured multi- multi-image captioning as a proxy task to enforce global contextual alignment. By explicitly decomposing model responses into per-image descriptions (e.g.. "For Image I: <caption 1>", "For Image 2: <caption 2>"), we compel the model to accurately attend to each image within the sequence. This structured formulation ensures contextual completeness while mitigating inter-image interference. To further address Context Omission and Context Contlation, introduce two perturbation techniques-sequee truncation and content

---

swapping into the captioning preference optimization process. By training the model to distinguish coherent contexts from disrupted ones, we promote holistic reasoning across the entire imput sequence.
(2) Needle-Level Optimization: To address Detail Misinterpretation, we propose a fine-grained preference learning strategy that sharpens the model's sensitivity to critical visual cues. Our approach integrates
visual prompts into the preference data and employs DPO training to bias the model toward descriptions aligned with highlighted regions. This enhances the model's ability to detect, attend to, and describe salient visual elements across multiple images. Furthermore, inspired by [31], we incorporate vision contrastive preference supervision by constructing image pairs with varying alignment to reference descriptions. This encourages the model to refine its preference judgments on mne-grained visual cues within each image under contextual settings.
To support these two-level optimization objectives, we introduce MultiScope-42k, a scalable multi- image preference dataset. 'hee dataset comprises high-quality chosen responses--Syttesiinee by rejected responses gener-
splicing tegether accurate image- and region-level ated through targeted perturbations both contextual and local detail levels. This pipeline converts abundant single-image data into complex multi-image training signals, providing scalable solution data scarcity. Our main contributions are summarized as follows:
We pioneer the investigation of cognitive bias in multi-image comprehension for MLLMs, categorizing three prevalent hallucination types. To address these challenges, we propose Context-to to- Cue Direct Preference Optimization (CcDPO), an innovative two-level prefer- ence optimization framework that enhances per-image perception in multi-image settings by analyzing visual clues from sequential context to local details.
We design low-cost Context-Level Optimization mechanism, incorporating structured multi-image captioning preferences and targeted perturbation techniques to ensure MLLMs' comprehensive and consistent global context understanding. Complementarilyy, we develop a Needle-Level Optimization mechanism that enhances fine-grained visual acuity through the integration of region-focussed visual prompts and vision contrastive preference signals.
We construct MultiScope-42k, a large-scale, high-quality dataset for two-level multi-image preference learning. The fully automatic generation pipeline is cost-effective and scalable across diverse data sources. After direct preference optimization on this dataset, our method significantly reduces hallucinations and achieves superior performance on multi- image tasks.
2 Related Work
Multi-modal Large Language Models. Recent advances in MLLMs [12, 32, 33, 34] have combined powerful large language models (LLMs) with visual encoders via lightweight connectors, achieving impressive performance across dialogue [35], visual question answering (VQA) [5], and image captioning tasks [36]. These models are typically trained on image-text pairs with instruction tuning, yielding strong single-image understanding. However, they remain prone to hallucinations [37, 38, 39], especially in multi-image scenarios where accurate reasoning requires modeling not only individual images but also their cross-image relationships. Recent studies aim to advance multi-image understanding by incorporating image-text interleaved data [40, 41] during model training. This approach helps develop capabilities such as image comparison [42, 43], cross-image association [44, 45], and temporal reasoning [46, 47]. Nevertheless, instruction tuning with such data remains costly due to the need for complex, fine-graine mne-grained annotations an issue exacerbated in multi multi-image settings.
Direct Preference Optimization. Reinforcement Learning from Human Feedback (RLHF) [48, 39, 3] aligns L.MSs with human preferences by training reward model to maximize the gap between cho- sen and rejected responses. As a more efficient alternative, Direct Preference Optimization (DPO) [22] bypasses reward modeling and directly optimizes on preference pairs. Recent work investigates DPO's generalization and stability across tasks [26, 49, 50, 51], with multimodal extensions [31, 52, 53, 26] to reduce hallucinations and enhance vision-language grounding single-image settings. However, current language- based DPO methods often neglect visual details. To address this, vision contrastive DPO approaches either disrupt images [23, 52] or highlight key visual tokens [54, 55], enhancing preference learning but focusing mainly on single-image tasks. MIA-DPO [27] pioneered DPO to multi-image settings by anchoring prompts to specific images, achieving promising results on relevant benchmarks. However, its reliance on predefined references limits holistic context modeling and autonomous cross-image reasoning. contrast, we propose CcDPO, which explicitly models global context and fine-graine ane-inrined visual cues to enhance multi-image reasoning.
3

---

Answer Directly
(b) Deseription Generation Reasoning on Descriptions
(O) Performance Comparison
Quinkic Kring nage saage Zandivage La
Qemiliec Froeic disuritton staga
Producew: The aljes laagy
aadpaepr whill ouly le belieird wile liwil:
Plr
fon loal?
Multi-modal I.LM
Multi-modal L.M
Multi-modal I.M
MCCU
Q4
Drenal
Alrndy Prokiken Nicelças K. bagge
Capsian ealic Emage Bat avedden Dotkl
Caphan moneeee 1-
Dundox Aroig page aagg
plasi lnage carra
Expe Iilerpe
Figure 2: (a) Baseline: Direct inference without context as a condition. (b) Two-stage approach: Gen- erating image captions, then reasoning over them. (c) Performance: Accurate caption understanding as context substantially improves VQA accuracy, with noisy captions also proving beneficial. This highlights deficient intrinsic captioning in MLLMs as key bottleneck, motivating its enhancement.
Visual Prompting for MLLMs. Visual prompting has been widely used in vision models [56, 58). Manually annotated points, boxes, or masks-often encoded by a separate prompt encoder-can guide the model to adjust segmentation granularity or select specific instances. More recently, MLLMs have shown the ability to interpret visual prompts directly embedded in the image without additional prompt encoders [59, 60]. Unlike prior work, method actively integrates visual prompts into preference data and uses DPO training to encourage the model to prefer descriptions aligned with prompted regions, which enhances the model's sensitivity to visually grounded information.
3 Exploring Cognitive Bias in Multi-Image Comprehension of MLLMs
In this section, we investigate the cognitive biases of MLLMs in multi-image comprehension, identifying that deficient internal multi-image captioning ability is a core bottleneck for complex reasoning. Then, we introduce systematic evaluation methodology to diagnose and quantify three fundamental multi-image hallucination types. Our results confirm that even state-of- offtth--art models exhibit severe performance degradation and elevated hallucination rates as input images increase.
Limited Contextual Awareness. We identify a fundamental limitation in current MLLMs: their impaired ability to perceive and integrate partial context information for coherent understanding severely degrades performance. To investigate this limitation, we systematically examine how context quality affects multi-image understanding through controlled experiments with LLaVA-OV-7B [32]. As illustrated in Fig. 2, we evaluate the model's reasoning capability under three distinct context conditions: (1) Accurate Context: The model received the images along with accurate, descriptive captions. (2) Noisy Context: It received the images along with flawed, machine-gent -ennrreee captions. (3) Baseline (No Explicit Context): It received only the images, forcing it to rely entirely on its own internal understanding. As shown in Fig. (c), while accurate captions helped slightly more than noisy ones, the most dramatic finding was massive 7.37-point accuracy drop when the model performed inference directly from the images, without any context augmentation. This gap reveals the model's heavy reliance external caption aids, confirming that its intrinsic captioning capability is a core bottleneck for multi-image reasoning and motivating our direct efforts to enhance this skill.
Multi-image Captioning Re-evaluation. We design a caption generation task as a proxy to system- atically evaluate MILMs' multi-image understanding, identifying three fundamental hallucination types- Context Omission, Context Conflation, and Detail Misinterpretation that critically degrade model performance. To enable this evaluation, we construct Context-AMBER-III by systemat- ically concatenating images from the single-image AMBER dataset [37] into sequences of two types: short-context sequences with images and long-context sequences with 8 images. Each input is paired with the prompt: "Please sequentially describe each of the images shown above. Use the following format: For Image *: (description). Thhe expected out- put format is: "For Image 1: (caption 1) For Image 2: (caption 2) ..... For Image N: (caption N) ", ensuring comprehensive coverage of all input images. This controlled for- mat allows precise identification of hallucination behaviors during multi-image comprehension.
We assess caption quality through four complementary metries: (a) CHAIR [36] measures object hal- lucination severity, (b) Response-level Hallucination Rate (Hal) [37] quantifies incorrect descriptions, (c) Cognition- Hallucination (Cog) [37] detects reasoning errors, and (d) Sequence Coverage Rate (SCover) evaluates caption completeness across image sequences. Notably, for each input sequence, we compute hallucination scores (a), (b), and (c) for each image independently, then average these to get the sequence-level hallucination scores.

---

Table I: Hallucination and preference alignment results. We report metrics constructed multi-image AMBER benchmark. Lower scores indicate better performance for CHAIR, Hal, and Cog, while higher is better for SCover. CcDPO achieves consistent improvements under both 4-image and 8-image settings, effectively reducing hallucinations in contextual multi-image understanding.
Context-AMBER (4 Images)
Context-AMBER (8 Images)
Models
Parameter
CHAIR
SCovert
Hall
Cogl
CHAIR
SCover
Hal
Cogl
ILaVA-OV
7B
10.2
74.0%
31.8
2.6
50.6
10.3%
69.1
6.5
MII-DPO [27]
7B
8,9
83.9%
29.8
2.1
28.2
36.7%
45.0
3.8
CcDPO (Ours)
7B
3.7
100.0%
15.3
1.2
15.3
83.3%
27.5
2.1
A
+6.5
+26.0
+16.5
+1.4
+35.3
+73.0
+41.6
+4.4
Results. As shown in Table 1, even strong models like LLaVA-OV-7B exhibit significant halluci- nation rate increases in multi-image scenarios. For example, when input images grow from 4 to 8, Detail Misinterpretation hallucinations become severe-the the CHAIR score jumps from 10.2 to 50.6, indicating sharp decline in grounding accuracy. The sharp drop in the SCover score from 74.0% % to 10.3% % also reveals context omission issues within multi-image settings. Analysis of failure cases (Fig. 1) reveals that Context Conflation commonly occurs, critically degrading model performance.
4 CcDPO: Context-to-Cue Direct Preference Optimization
As discussed in Sec. 3, three fundamental hallucinations (context omission, context conflation, and de- tail misinterpretation) manifest as performance degradations in MLLMs' multi-image understanding. To address these challenges, we propose Context-to- Cue Direct Preference Optimization (CcDPO), a hierarchical preference alignment framework that refines MILMs at two levels (as shown in Fig. 3):
Loveext-Level Optimization: By contrasting complete and disrupted multi-image captions using language-based preference optimization, we enhance MLLMs' contextual understanding by ensuring comprehensive integration of all relevant visual information across image sequences.
Needle-Level Optimization: A hybrid preference optimization method integrates two complemen- tary objectives: (1) Contrasts captions that either align with mismatch visually prompted regions using danguage-based preference optimization, and (2) Discriminates between images semantically matching contradicting given captions using vision contrastive preference optimization. This dual approach trains the model to make preference judgments grounded in fine-grained visual details.
To support such hierarchical DPO, we construct MultiScope-42k, large-scale preference dataset with automatically generated positive and perturbed response pairs at both levels. See Appendix A.
4.1 Context-Level DPO with Language-Bas Based Preference Optimization
We propose low-cost DPO mechanism for MLLMs that leverages multi-image captioning as proxy task to enforce coherent multi-image understanding, thereby addressing context hallucinations (omission, conflation). Specifically, we reformulate the response generation task as a structured, per-image captioning problem. Each image in sequence is described in an explicit format, generating the preferred response y, as a coherent sequence of captions that reflect the content of each image:
y. {For Image 1: (caption 1), For Image 2:..., For Image N: (caption N).}
This encourages the model to practice selective attention and attribution- generation. During training, we construct preference pairs where the positive sample is coherent, full-sequence description y and the negative sample y is obtained from following two perturbation strategies:
simulates context omission by either removing captions entirely from one or more images (complete content omission) or replacing detailed captions with shorter versions (partial content omission), generating rejected responses yi and yhhon Complete omission disrupts sequence continuity, while partial omission results in sparse or incomplete sequence information:
Sequence Truncation:
y = {For Image 1: (captio 1), For Image 3: (caption 3), ...)
y {For Image 1: short caption 1 ) For Image 2: short caption ),, ...)
5

---

(o) Caption Pool
(b) Context-Level Prefenence Construction
Lnugue gre- teip
Question Mcrinap--inso
Reyected yrunts For Seage Suthrbngg
Oniissin
Contect
Irege
nomec hnsttuluud ueu lon Far Taage Teage chheere fame Heeds Cogtiin 2>.
Context Omiision
dish lovee -Captinn 21. For Inage A,1
Conficttin
Ieep
Level
Caption Pool
Context Level DPO
Context L LLaVA-OV
Needle Level DPO
Details LLavA-Ov
LLaVA-OV
(c) Needle-Level Preference Pair Construction
Regian Coprice mhevica
Question ansagge severe Rever prodiic
regies lmago
redding on the right of
Images
30
Beginn Caption 20
Region- Level Caption Pool
thall pland cosn 1o di
3 Thine -Gaghii 1.
age 5.0 fomertee -Caption
ligitiall ai iitiiiigge.... c10i :Cpption Rt.
dest chanes CCaptio Sv.
Figure 3: Overview of CcDPO. (a) Caption pools are built from LLaVA-23K [61], MDVP [63]. and MVC [49] for image- and region-level supervision. (b) Context-Level DPO aligns model outputs with complete, coherent image sequences and penalizes omissions, conflation, and misalignments. (c) Needle-Level DPO incorporates visual prompts enhance local detail understanding. chosen responses describe marked regions accurately, while rejected are drawn from mismatched regions. Both language-birn basse and vision-contrastive preferences are used to sharpen fine-grained perception.
Content Swapping: simulates context conflation by swapping the captions between different images within the sequence, thereby generating the rejected response yı- e.g.,
y {For Image 1: (captioo 2), For Image 2: (caption 1), ...J
Given an instruction and image sequence Uws we define the context-level DPO objective as:
To
Une: x))
Uwi 2r) ) Uwr .:)
LDPO, - looog (081lggg Blog
log
y/ E
{ul
trunc
Vi
swap)
(re (ww,
Teer(yi
(1)
where Te is the target model and Tie is a frozen reference model. This objective reinforces the preference (æ, Vw: y/o) > (æt, tw. Y1) for globally coherent responses over disrupted ones. We use LLaVA-23K [61] and COCO [62] as our detailed and brief context caption pool, respectively. The visualization cases for three different types of context preferences are presented in Figs. 6, 7, and 8.
4.2 Needle-Level DPO with Hybrid Visual-Language Optimization
Even when global context preserved, MLLMS often fail to identify or attend salient visual elements (e.g., missing objects, actions, or attributes). This leads to detail misinterpretation. which global response- baseed supervision alone cannot resolve. To address this, we introduce needle-level optimization, fine-graine ane-grained hybrid preference mechanism employing visual prompts and image-level perturbations sharpen the model's focus on often-overlooked local visual cues.
Language-based Preference Optimization (TDPO). This stage leverages region-specific visual prompts (bbox and point) guide the model's preference judgments. The visual prompt construction involves three key steps: (1) collecting source data with point or bounding box annotations, where points are defined as [x. y] coordinates and bounding boxes [x1, II, x2, y2] corner coordinates: (2) converting these annotations into visual prompts (e.g., red points or green rectangles) overlaid onto original images: (3) enhancing region salience by adding textual labels (e.g.. "REF") adjacent to visual markers, as illustrated in Fig. 3. These visual prompts explicitly define the target regions for description. To train the model interpret these specified areas accurately, our DPO method learns from preference pairs that penalize misinterpretations. These pairs are constructed follows:
Chosen Responses (y")) ) We integrate region-level visual prompts (e.g., bounding boxes, key- points) into images a to highlight a target region r, yielding v,.- The model is trained to prefer the accurate description y of this specific region, directing its attention to critical visual elements:
y.. [For the marked of Image 2: (caption r2)..., For the marked area of Image N: (caption rx)-]
of Image 1: (caption rı), For the marked

---

2) Rejected Responses (yf"); Descriptions of regions r' that are non-overlapping with r within the same image serve as rejections. By learning to identify and reject such region- inaccuracies, the model's pereeption of fine-grand -haiggai image details more precise. yi is formally defined as:
vi = [For the marked area of Image 1: (caption rí), For the marked area of Image 2: (caption re),.., For the marked area of Image N: (caption rxy)-
Given an instruction æ and the image sequence Vr. these pairs (x, Vr V > (ær, Vr, vf inform the language-based DPO objective (Eq. 1). We utilize MDVP [63] for the region-level caption pool.
Vision Contrastive Preference Optimization (VDPO). Inspired by [49, 31J. this stage further hones the model's visual discrimination. It trains the model by contrasting a single description yo against two image inputs: Vw. which correctly aligns with yo (often focusing on specific visual region), and 8n which is misaligned or visually contradicts yvu The objective combines two components:
1) Focusing on Relevant Visuals (Frocu)): This rewards prioritizing details in the correctly aligned image Uw when generating Yw. countering MILMS' tendency to neglect visual content.
(o Un, 4)
To xr)
Foccus ( (Dw, yw) = - log (31 log
B1 log
(he (w: 2)
Teer (Vhe
xx)
2) Rejecting Contradictory Visuals (CReject): This penalizes assigning high probability to you when conditioned on contradictory image vz.
To x) - Bo log To To(w (2, (ye x=)
Recjor (UI, Vie) = log 82 log
(3)
Teer(yn U1, xr)
The combined vision contrastive DPO loss is LDro,, Yw: 02) = Frocus (Uw: yw) + RRsjec (2%, Dw)- This objective sharpens the model's 's ability distinguish fine-grained visual cues by rewarding focus on relevant details and penalizing attention to misleading content. We use the MVC [49] dataset as a
region-level visual counterfactual caption pool. Visualization cases are shown in Figs. 10, and 11.
5
Experiments
5.1 Experimental Settings and Evaluation Benchmarks
Baselines. We apply CcDPO to two different 7B-size MLLMs: Qwen2-VL [33] and LLaVA-OV [32].
Due to differences in base models, preference data, and alignment strategies, direct comparisons with other LLMs are not possible. However, we provide the results for reference: LLaVA-1.5 [64],
VL2-88 [65], Mantis-Idefics [44], mPLUG-OW13 [66], Idefics2-8B [20], and Emu2-Chat [21].
Implementation Details. Our model undergoes a stree-stage sequential training process to better understand multi-image preferences at both broad (context) and detailed (needle) levels. Stage 1 focuses on context-level alignment, where we fine-tune Qwen2-VL-7B and LLaVA-OV-7B for one epoch with learning rates of 5 x 10-6 and 5 x 10-5, respectively, using Eq. 1. Stage applies needle-level language-based DPO using Eq. to improve sensitivity to fine-grained visual cues with the same learning rate of x 10-5, We conduct Stage and Stage 2 by using LoRA adaptation [67] with rank r = 128 for efficiency. Stage 3 performs vision contrastive DPO via full- palammeettter tuming for one epoch with learning rate of x 10 using Eq. 2, strengthening the model's ability to distinguish preferred visual content. Following the setup in [27], we set the temperature parameter 3= 31 = 32- = 0.1 and the negative log-likelihood (NLL) loss coefficient 7 = 0.1. All training is conducted on eight GPUs, each equipped with 90GB of memory.
Evaluation Benchmarks. We employ seven multi-image MIRB [68], BLINK [69]. Mantis-Eval [44], NLVR2 [70]. Q-Bench2 [71], and MIBench [72]-to holistically evaluate multi-image reasoning across four key dimensions: co-reference alignment, fine-grained comparison, contextual reasoning, and temporal understanding. Complementing these, eight representative single-image benchmarks assess specific multimodal capabilities: (1) Aca- demic/Scientific Reasoning: MMMU [73], MMStar [74], ScienceQA [75], (2) Diagram Understand- ing: A12D [76), (3) Robustness against hallucinations: POPE [77], HallBench [38], (4) General Multimodal Abilities: MMBench [78]. (5) Text Recognition: OCRBench [79]. This comprehensive
[42].
evaluation suite demonstrates our method's 's strengths in both holistic understanding and -neignd
visual grounding across single-imag and
general tasks.
7

---

Table 2: Main results general multi-image benchmarks. We compare proposed method, CcDPO, with existing multi-image DPO approaches across seven multi-image benchmarks. Our
method consistently enhances the performance of both LLaVA-OV and Qwen2-VL.
Models
Parameter
MuirBench
MIRB
BLINK
Mantis
NLVR2
MIBench Q- Q-Bench2 Average
GPT-4o [80]
62.3
$3.0
60.1
62.7
88.8
71.8
74.5
67.6
LLaVA-v1.5 [64]
7B
19.9
28.4
37.1
41.9
52.1
40.9
53.9
39.2
Idefics2 [20]
8B
26.1
33.0
45.2
48.9
86,9
29.7
57.0
46,6
Mantis-Idefics2 [44]
8B
44.5
41.8
49.1
57.1
89.7
44.3
75.3
57.4
mPLUG-OW13 [66]
8B
39.6
50.3
63.1
90.8
54.5
59.6
Emu2-Chat [21]
37B
33.6
27.2
36.2
37.8
58.2
39.7
65.3
42.6
IntemVLL-98 [65]
48.7
50.0
50.6
60.3
85.56
52.9
58.0
LLaVA-OV [32]
7B
42,5
47.3
51.1
60.4
89.4
73.6
73.8
62.5
SFT
7B
45.4
48.9
53.4
64.9
89.0
71.9
75.7
64.1
MIA-DPO [27]
7B
41.4
48.0
53.7
60.3
88.2
67.8
74.0
61.9
CcDPO (Ours)
48.6
51.4
55,9
69.6
91.2
75.2
77.6
67.1
+6.1
+4.1
+4.8
+9.2
+1.8
+1.6
+3.8
+4.6
Qwen2-VL [33]
7B
40.5
59.5
53.4
65.9
84.8
68.9
74.5
63.9
SFT
7B
43.1
59.8
54.7
64.9
85.2
69.4
74.1
64.5
MIA-DPO [27]
7B
40.1
61.4
54.5
69.1
84.5
66.7
75.6
64.5
CcDPO (Ours)
7B
44.8
60.7
56.5
69.1
86.4
71.9
77.0
66.6
+4.3
+1.2
+3.1
+3.2
+1,6
+3.0
+2.5
+2.7
Table 3: Main results on single-image benchmarks. We compare our CcDPO with existing DPO-based approaches across seven single-image benchmarks. Our CcDPO not only improves performance multi-image settings but also preserves strong capabilities on Langl-image tasks.
Models
Parameter
MMStar
SQA
MMMU
POPE
HallBench
MMB
OCR
AI2D
Avg.
LLaVA-v1.6 [81]
7B
37.6
87.5
35,8
70.3
51.6
69.8
53.7
67.0
59.1
Qwen-VL- VL-Chat [82]
34.5
68.8
35,9
74.9
39.2
61.8
48.8
63.0
53.3
Idefics2 [20]
8B
49.5
88.7
43,0
86.2
75.7
72.3
69.2
OpenFlamingo [83]
9B
36.9
44.8
52.6
38.4
32.4
14.9
31.7
35.9
InstructB1.IP [84]
13B
32.7
54.1
86.1
45.3
38.3
27.6
40.6
46.3
Emu2-Chat [21]
37B
40.7
68.2
36.3
88.0
63.4
43.6
49.7
55.7
ILaVA-OV [32]
58.7
92.1
47.7
86.1
52.9
81.8
47.3
81.6
68.5
SFT
57.8
91.5
47.1
88.4
57.2
81.5
50.2
81.6
69.4
MIA-DPO [27]
57.4
92.4
45.1
87.9
55.4
80.9
52.1
81.5
69.0
CcDPO (Ours)
59.5
92.6
45.7
86.6
58.4
81.9
51.0
82.1
69.7
Qwen2-VL. [33]
57.8
84.1
85.9
66.9
81.2
85.6
78.9
73.8
55.0
82.7
87.7
66.7
84.8
78.5
73.3
MIA-DPO [27]
58.2
84.0
88.4
62.7
85.1
78.9
73.3
CeDPO (Ours)
58.7
82.8
87.1
68.8
83.5
79.7
74.1
5.2 Main Results
Results on General Multi-Image Tasks. As shown in Table 2, we evaluate CcDPO across diverse multi-image benchmarks that span wide range of reasoning skills. CcDPO consistently outperforms both the SFT baseline and other DPO-based methods on all datasets, with a notable gain of +4.8 points on BLINK, which focuses on multi-view and spatial reasoning. On the large-scale MuirBench wheaset-where each sample contains average of 4.3 images and up to 9 images- CcDPO achieves largest improvement of +6.1 points, demonstrating its strength modeling complex multi-image dependencies such fine-grained perception, sequential cues, and holistic context. In contrast, MIA-DPO underperforms on MuirBench, highlighting its global context limitations. Consistent gains across LLaVA-OV and Qwen2-VL variants validate approach's 's generality and efffcctiveness.
Results on General Single-Image Tasks. While previous works [27, 44] indicate that multi-image training can degrade single-iman imagle-image understanding, our CcDPO, in contrast, generally yields perfor- mance gains on most single-image datasets shown Table averaging +1.2 points for LLa LLaVA-OV and +0.3 for Qwen2-Vi VL. Visually-driven tasks like HallBench exhibit the largest improvements, up +5.5 points under CcDPO. Conversely, for tasks with relatively low reliance on visual information, exemplified by ScienceQA [75], our method showed no notable gains, and performance slightly declined. These results highlight CcDPO's robustness: not only excels in multi-image scenarios but also preserves, and often enhances, Lingle-image capabilities. We attribute this success to

---

Table 4: Performance on the image retrieval task from needle-in-a-baystacc MM-NIAH [85]. We compare our proposed method, CcDPO, with DPO-based baselines acToss K-24KK contexts, where the number of images ranges from a few to over a hundred. CcDPO consistently outperforms prior methods, demonstrating its strength in capturing fine-gran fine-grained details in ultra-long image sequences.
Models
Parameter
1K
2K
4K
8K
12K
16K
24K
Average
LLVVA-OV-7B [32]
7B
89.2
88.1
82.3
71.2
65.0
60.9
45.0
71.7
SFT
7B
92.0
93.9
88.2
80.7
74.8
69.4
49.9
78.4
MTAA-DP [27]
7B
93.9
94.6
90.5
85.1
75.5
68.8
59.4
81.1
+ +CDDPO (Ours)
7B
95.3
96.9
91.4
89.6
78.8
74.5
64.7
84.5
A
+6.3
+8.8
+9.1
+18.4
+13.8
+13.6
+19.7
+12.8
Table 5: Ablation study of two-level CcDPO on MIBench and MIRB tasks requiring perception,
comparison, and reasoning across multiple images. Detailed task descriptions in the Appendix B.
MIBench Benchmark
MIRB Benchmark
Models
GC
TR
LR
FVR
TRI
VTK
TVK
Know.
Reas.
Perc.
M-Hop
LLaVA-OV-7B [32]
87.7
85.9
72.6
74.5
96.5
77.5
42.7
67.1
70.0
44.0
50.0
12.0
-Context-Level
87.4
88.6
69.5
76.0
97.9
76.7
43.4
68.8
75.0
44.0
52.0
15.0
e Needle-Level
88.8
90.4
70.3
76.0
98.2
77.6
52.6
69.8
72.0
48.0
55.0
18.0
A
+1.1
+4.5
-2.3
+1.5
+1.7
+0.1
+9.9
+2.7
+2.0
+4,0
+5.0
+6,0
SFT
87.9
88.2
71.0
75.5
87.8
77.4
38.5
68.3
75.0
43.0
53.0
9.0
+ MIA-DPO [27]
85.8
87.2
63.4
67.9
94.9
67.6
42.0
59.6
73.0
50.0
44,0
11.0
preference data design, which employs structured, per-image descriptions, thereby fostering precise understanding of individual images even within multi-image contexts.
5.3 Ablation Studies
Comparison with SFT Trained on MultiScope-42K, As shown in Table 2, Table 3, Table 4, CcDPO outperforms SFT across all benchmarks, achieving +3.0 multi-image tasks, +0.6 single-image tests, and a significant +6.1 gain on MM-NIAH. By integrating negative samples into DPO, CcDPO enhances discrimination between accurate and hallucinated outputs, improving fine-grained detail recognition and long-range dependency modeling while maintaining single-image performance. This demonstrates both robust generalization and superior contextual understanding, with negative sample integration proving essential to its performance advantages.
Superior Context Scaling for GrainGr Detail Capture. As shown in Table 4, CcDPO signifi- cantly outperforms prior DPO-based methods and SFT baselines on the challenging MM-NIAH [85] image retrieval task. Remarkably, CcDPO achieves consistent improvements across all tested context lengths (2KK,,,) with its largest gain (+19.7 points) occurring at the maxi- mum 24K context length compared to the baseline model. This demonstrates CcDPO's 's exceptional ability to: (1) Scale effectively to ultra-long image sequences, and (2) Capture fine- graiggained visual details critical for discriminating subtle differences in large image collections. The +12.8-point average improvement underscores its robustness in handling large-scale multi-image contexts. The hierarchical preference optimization in CcDPO enables precise, context-aware understanding an essential capability for processing extensive visual information.
Effectiveness of Context-Leww Optimization. The Context-Level DPO enhances alignment between language responses and the holistic visual context across image sequences. As shown in Table 6, this yields significant improvements Understanding (+5.1), Image-Text Matching (+9.8), Similarity Matching (+16.4). These gains reflect the module's ' ability to: (1) Capture semantic relationships across images, and (2) Maintain coherent multi-image descriptions through consistent attribute attribution. Notably, Scene Understanding and Retrieval tasks also benefit from improved global alignment, confirming that such optimization effectively reduces context omission and conflation errors in complex visual sequences.
sub-tasks requiring global reasoning, including: Diagram
Effectiveness of Needle-Level Optimization. The Needle-Level DPO improves the model's 's ability to capture fine-grained visual cues by contrasting localized content. As shown in Table and Table 6, this is especially effective for tasks requiring detailed comparisons across images. In MIBench, the VTK where the model must link information across image cells-shows a large gain of +9.9, demonstrating that our visual preference signals help focus factual visual details. Similarly,

---

Table 6: Ablation study of CcDPO on the MuirBench dataset across all dab-satasets, demonstrating
significant performance gains on most subsets. The symbol @ stands for method superposition.
Action
Similarity
Cartoon
Counting
Diagram
Difference
Goographic
Ordering
Sccne
Grounding
Retrieval
1-T
Models
Overall
LLaVA-OV.7B [32]
42.5
35.9
33.1
35.8
24.7
55.0
30.0
46.0
46.9
20.3
72.0
29.7
47.6
Context-Level
44.8
36.6
30.3
37.1
35.5
57.0
37.9
41.0
48.3
15.6
65.1
28.3
48.3
Neadle-L evell-DPPe
47.8
42.3
48.0
34.6
38.0
58.3
40.3
41.0
55.4
14.1
68.9
30.1
45.9
0 Needle-Lovel-DDO
48.6
44.5
49.5
33.3
39.3
60.1
39.1
41.0
56.7
13.0
70.4
28.6
46.6
+6.1
+8.6
+16.4
+14,6
+9.1
-1.0
Qwen2-VL-7B [33]
40.5
40.8
46.4
41.0
39.7
41.9
34.4
25.0
54.3
65.0
28.5
20.5
Context-Level
42.1
41.0
45.0
42.3
39.8
46.2
37,9
22.0
56.9
14.1
65.1
27.4
21.0
Needle-Level-TDDR
42.3
39.6
43.9
41.0
39.7
46.5
39.1
22.0
56.3
15.7
66.7
26.2
22,9
Needie-Leee--DDe
44.8
43.9
46.9
38.5
39.9
53.8
37,9
20.0
61.4
18.8
69.9
23.8
22.9
+4.3
+3.1
+0.5
-2.5
+0.2
+11.9
+3.5
-5.0
+7.1
+9.5
+4.9
-4.7
+2.4
Table 7: Ablation on component and training order.
Table 8: Ablation on visual prompt types.
Configuration
MuirBench Mantis BLINK Q-Bench2
Bhox Seg. Point Circle BLINK Q-Bench2
Baseline
42.5
60.4
51.1
73.8
55.1
76.0
Context-Level Only
44.4
69.0
55.3
76.4
v
56.2
76.6
Needle-Leve Only
46.3
65.0
54.3
75.5
V
56.8
76.3
Needle Context
46.5
66.3
54.1
77.8
54.5
77.2
Context > Needle
48.6
69.6
55.9
77.6
55.9
76.8
in MuirBench, we observe strong gains Action Understanding (+8.6), Counting (+14.6) and Difference Spotting (+9.1), all of which depend on localized perception. These results indicate that Needle-Level DPO significantly boosts the model's perceptual grounding and resistance to detail-level hallucinations, complementing the context-level DPO for better multi-image understanding.
Synergistic Two-Level Optimization in CcDPO. As evidenced in Table 7, performing context-level DPO prior to needle-level DPO consistently yields optimal benchmark performance, underscoring a hierarchical interdependence between the two stages. This workflow mirrors human general reasoning for multi-image problems: one first understands the global context, then zooms in to focus on specific details as needed for the task. Notably, inverting the training sequence or isolating either stage results in suboptimal outcomes, further validating the necessity of our two-stage sequential optimization.
Sensitivity Analysis of Visual Prompt. As shown in Table 8, our ablation studies reveal minimal performance variation (#1 point) across visual prompt types (bounding boxes, circles, segmentation masks) during the Ncedle-Level Language-bass based Preference Optimization stage, underscoring our method's robustness. Notably, we observe benchmark -specfffc patterns: mne-grained prompts (c.g.. segmentation masks) excel in spatial reasoning tasks BLINK by capturing precise local details, while coarse-graines prompts (e-g., bounding boxes) perform better in image-level comparisons Q- Q-Bench2 through efficient spatial guidance. Consequently, we adopt bounding box + keypoint as the final format, balancing accuracy and annotation practicality optimally.
Limitations. While CcDPO is primarily designed for general multi-image reasoning, it does not explicitly model temporal dependencies, which may limit its performance on video-like inputs. How- ever, our framework is readily extensible to such video data by incorporating video clip descriptions (as demonstrated in Table 13). Similarly, although OCR supervision is limited in our current dataset, CcDPO can be naturally enhanced with targeted text-centric preference data in future work.
Conclusion
This work introduces CcDPO, a two-level preference optimization method for enhancing multi-image understanding in MLLMs. By decoupling learning into context-level and needle-level stages, CcDPO addresses key hallucination issues including context omission, conflation, and detail misinterpretation. The context-level module promotes holistic sequence comprehension via structured caption prefer- ences, while the needle-level module strengthens fine-graines perception through visual prompts and contrastive supervision. To support optimization, we construct MultiScope-42k, a large-scale dataset with automatically generated multi-level preference pairs. Experiments across seven multi-image benchmarks show that CcDPO achieves consistent improvements over SFT and prior DPO variants, confirming its effectiveness in aligning MLLMs with both global and local visual content.
10

---

Acknowledgments
This work was supported by the National Science Fund for Distinguished Young Scholars N0.66025603), the National Natural Science Foundation of China (No. 62576299, No. 625B2156, No. U21B2037, No. U22B2051, No. U23A20383, No. U21A20472, No. 62176222, No. 62176223, No. 62176226, No. 62072386, No. 62072387, No. 62072389, No. 62002305, and No. 62272401), the Natural Science Foundation of Fujian Province of China (No. 2021J06003, No. 2022I06001), and the Fundamental Research Funds for the Central Universities.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08874, 2023.
[2] Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, et al. Seed1. 5-vl technical report. arXiv preprint arXiv:2505.07062, 2025.
[3] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Iims via reinforcement learning. arXiv preprini arXiv:2501.12945, 2025.
[4] Qiong Wu, Wenhao Lin, Yiyi Zhou, Weihao Ye, Zhanpeng Zen, Xiaoshuai Sun, and Rongrong Ji. Accelerating multimodal large language models via dynamic visual-token exit and the empirical findings. arXiv preprint arXiv:2411.19625 2024.
[5] Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In Proceedings of the IEEE international conference computer vision, pages 2425-2433, 2015.
[6] Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Flevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6904 1913, 2017.
[7] Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song- Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507-2521, 2022.
[8] Liangyu Chen, Bo Li, Sheng Shen, Jingkang Yang, Chunyuan Li, Kurt Keutzer, Trevor Darrell, and Ziwei Liu. Large language models visual reasoning coordinators. Advances in Neural Information Processing Systems, 36:70115-70140, 2023.
[9] Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. Chameleon: Plug-and-play compositional reasoning with large language models. Advances in Neural Information Processing Systems, 36:43447-43478. 2023.
[10] Neil Cohn, Ryan Taylor, and Kaitlin Pederson. A picture is worth more words over time: Multimodality and narrative structure across eight decades of american superhero comics. Multimodal Communication, 6(1):19-37, 2017.
[11] Bohao Li, Yuying Ge. Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. Seed-bench: Benchmarking multimodal large language models. In Proceedings of the IFEF/CVF Conference on Computer Vision and Pattern Recognition, pages 13299-13308, 2024.
[12] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction wning. Proceedings of the IEFF/VV Conference on Computer Vision and Pattern Recognition, pages 06296-26306 2024.

---

[13] Junnan Li. Dongxu Li, Silvio Savarese, and Steven Hoi. 2lip-2: Bootstrapping language inguage-image pre- pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730-19742. PMLR, 2023.
[14] Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang. Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual- -ouall-inguisttc tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 24185-24198, 2024.
[15] Yash Goyal, Tejas Khot, Douglas Summers-Star Dhruv Batra, and Devi Parikh. Making the in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6904 6913, 2017.
[16] Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/eyf conference on computer vision and pattern recognition, pages 3195-3204, 2019.
[17] Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 9556-9567, 2024.
[18) Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang. Jilan Xu, Guo Chen, Ping Luo, et al. Mybench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22220,, 2024.
[19] Jean-Baptiste Alayrac. Jeff Donahue, Pauline Luc, Antoine Miech, Lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716-23736 2022.
[20] Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models? Advances in Neural Information Processing Systems, 37:87874 87907, 2024.
[21] Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Generative multimodal models are in-context learners. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 14399,14409, 2024.
[22] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728-53741, 2023.
[23] Yuxi Xie, Guanzhen Li, Xiao Xu, and Min- Yen Kan. V-dpo: large vision language models via vision-guided direct preference optimization. arXiv preprint 2r.0v:2411.022122 2024.
Mitigating hallucination in
[24] Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. Aligning modali- ties in vision large language models via preference fine-tuning. arXiv preprint 11111 2024.
[25] Haojian Huang, Haodong Chen, Shengqiong Wu, Meng Luo, Jinlan Fu, Xinya Du, Hanwang Zhang, and Hao Fei. Vistadpo: Video hierarchical temporall direct preference optimiza- tion for large video models. arXiv preprint 2rXiv:2504.13122 2025.
[26] Mengxi Zhang, Wenhao Wu, Yu Lu, Yuxin Song, Kang Rong, Huanjin Yao, Jianbo Zhao, Fanglong Liu, Haocheng Feng, Jingdong Wang, et al. Automated nulti- preference for mllms. Advances in Neural Information Processing Systems, 37:26171-26199 2024.
12

---

[27] Ziyu Liu, Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Haodong Duan, Conghui He, Yuanjun Xiong, Dahua Lin, and Jiaqi Wang. Mia-dpo: Multi-image augmented direct preference optimization for large vision-language models. 2025.
[28] Xinyu Tian, Shu Zou, Zhaoyuan Yang, and Jing Zhang. Identifying and mitigating position bias of multi- imate vision- lsionnlanguage models. arXiv preprint arXiv:2503. 13792, 2025.
[29] Guanghao Zhang, Tao Zhong, Yan Xia, Zhelun Yu, Haoyuan Li, Wanggui He, Fangxun Shu, Mushui Liu, Dong She, Yi Wang, et al. Cmmcot: Enhancing complex multi-image comprehension via multi-modal chain-of-thoughe and memory augmentation. arXiv preprint arXiv:2503.05255, 2025.
[30] Zhongwei Wan, Ziang Wu, Che Liu, Jinfa Huang, Zhihong Zhu, Peng Jin, Longyue Wang, and Li Yuan. Look-m: Look-once optimization in kv cache for efficient multimodal long-context inference. arXiv preprint arXiv:2406. 18139, 2024.
[3]] Jinlan Fu, Shenzhen Huangfu, Hao Fei, Xiaoyu Shen, Bryan Hooi, Xipeng Qiu, and See-Kiong Ng. Chip: Cross-modal hierarchical direct preference optimization for multimodal Ilms. arXiv preprint arXiv:2501. 2025.
[32] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint 2xXi::2008.03326. 2024.
[33] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model's l's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024.
[34] Xudong Li, Runze Hu, Jingyuan Zheng. Yan Zhang, Shengchuan Zhang, Xiawu Zheng, Ke Li, Yunhang Shen, Yutao Liu, Pingyang Dai, et al. Integrating global context contrast and local sensitivity for blind image quality assessment. In Forty-first International Conference on Machine Learning, 2024.
[35] Ziyu Liu, Tao Chu, Yuhang Zang, Xilin Wei, Xiaoyi Dong, Pan Zhang, Zijian Liang, Yuanjun Xiong, Yu Qiao, Dahua Lin, et al. Mmdu: A multi-turn multi-image dialog understanding benchmark and instruction-tuning dataset for Ivims. arXiv preprint arXiv:2406. /183,, 2024.
[36] Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. arXiv preprint arXiv:1809.021555 2018.
[37] Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Zhang, and Jitao Sang. An 11m-free multi multi-dimensional benchmark for milms hallucination evaluation. CoRR, 2023.
[38] Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 14375-14385, 2024.
[39] Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang- Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multimodal models with factually augmented rlhf. arXiv preprint 23XX::2399.152.2 2023.
[40] Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang. Siddharth Karamcheti, Alexander Rush, Douwe Kiela, et al. Obelies: An open web-scale filtered dataset of interleaved image-text documents. Advances in Neural Information Processing Systems, 36:71683-71702. 2023.
[41] Wanrong Zhu, Jack Hessel, Anas Awadalla, Samir Yitzhak Gadre, Jesse Dodge, Alex Fang, Youngjae Yu, Ludwig Schmidt, William Yang Wang, and Yejin Choi. Multimodal c4: An open, billion-scale corpus of images interleaved with text. Advances in Neural Information Processing Systems, 36:8958-800 8974, 2023.
13

---

[42] Fei Wang, Xingyu Fu, James Y Huang. Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, et al. Muirbench: A comprehensive benchmark for robust multi-image understanding. arXiv preprint arXiv:2406.0944
2024.
[43] Mchran Kazemi, Nishanth Dikkala, Ankit Anand, Petar Devic, Ishita Dasgupta, Fangyu Liu, Bahare Fatemi, Pranjal Awasthi, Sreenivas Gollapudi, Dee Guo, et al. Remi: A dataset for reasoning with multiple images. Advances Neural Information Processing Systems, 37:60088- 60109, 2024.
[44] Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi-image instruction tuning. arXiv preprint arXiv:2405.01483, 2024.
[45] Shuo Chen, Zhen Han, Bailan He, Mark Buckley, Philip Torr, Volker Tresp, and Jindong Gu. Understanding and improving in-context learning on vision-language models. arXiv preprint w.Xiv:2311118021 2023.
[46] Fanqing Meng. Jin Wang, Chuanhao Li, Quanfeng Lu, Hao Tian, Jiaqi Liao, Xizhou Zhu, Jifeng Dai, Yu Qiao, Ping Luo, al. Mmiu: Multimodal nulti-image understanding for evaluating large vision-language models. arXiv preprint arXiv:2408. 02718, 2024.
[47] Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang. Yi Liu, Zun Wang. Jilan Xu, Guo Chen, Ping Luo, et al. Mybench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 22195-22206, 2024.
[48] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
[49] Shengguang Wu, Fan-Yun Sun, Kaiyue Wen, and Nick Haber. Symmetrical visual contrastive optimization: Aligning vision-language models with minimal contrastive images. arXiv preprint arXiv:2502.13928. 2025.
[50] Weiyun Wang. Zhe Chen, Wenhai Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu, Yu Qiao, et al. Enhancing the reasoning ability of multimodal large language models via mixed preference optimization. arXiv preprint arXiv:241 10442, 2024.
[51] Liping Yuan, Jiawei Wang. Haomiao Sun, Yuchen Zhang, and Yuan Lin. Tarsier2: Advanc- ing large vision-language models from detailed video description to comprehensive video understanding, arXiv preprint arXiv:2501.07888 2025.
[52] Songtao Jiang, Yan Zhang, Ruizhe Chen, Yeying Jin, and Zuozhu Liu. Modality-fai preference optimization for trustworthy milm alignment. arXiv preprint arXiv:2410.15334 2024.
[53] Ruohong Zhang, Liangke Gui, Zhiqing Sun, Yihao Feng, Keyang Xu, Yuanhan Zhang, Di Fu, Chunyuan Li, Alexander Hauptmann, Yonatan Bisk, et al. Direct preference optimization of video large multimodal models from language model reward. arXiv preprint arXiv:2404.01258, 2024.
[54] Chenhang Cui, An Zhang, Yiyang Zhou, Zhaorun Chen, Gelei Deng, Huaxiu Yao, and Tat-Seng Chua. Fine-grained verifiers: Preference modeling next-token prediction in vision-language alignment. arXiv preprint arXiv:2410.1 1414444...4. 2024.
[55] Jihao Gu, Yingyao Wang, Meng Cao, Pi Bu, Jun Song, Yancheng He, Shilong Li, and Bo Zheng. Token preference optimization with self-calibrated visual-anchored rewards for hallucination mitigation. arXiv preprini arXiv:2412. 14487, 2024.
[56] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yer Lo, et al. Segment anything. In Proceedings of the IEFE/CVF international conference on computer vision, pages 4015-4026. 2023.
14

---

[57] Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.000 007144 2024.
[58] Xudong Li, Zihao Huang, Yan Zhang, Yunhang Shen, Ke Li, Xiawu Zheng, Liujuan Cao, and Rongrong Ji. Few-shot image quality assessment via adaptation vision-language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10442- 10452, 2025.
[59] Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi. What does clip know about a red circle? visual prompt engineering for vlms. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11987-11997, 2023.
[60] Yuan Yao, Ao Zhang, Zhengyan Zhang. Zhiyuan Liu, Tat-Seng Chua, and Maosong Sun. Cpt: Colorful prompt tning for pre-trained vision-language models. Al Open, 5:30-38, 2024.
[61] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jac Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023.
[62] Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitick. Microsoft coco: Common objects in context. In Computer vision-ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part 13, pages 740-755. Springer, 2014.
[63] Weifeng Lin, Xinyu Wei, Ruichuan An, Peng Gao, Bocheng Zou, Yulin Luo, Siyuan Huang, Shanghang Zhang, and Hongsheng Li. enable milms to comprehend what you want. arXiv preprint arXhv:2403.20271, 2024.
Leveraging visual prompts to
[64] Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. -lava-next-interleave Tackling nulti-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.0789) 2024.
[65] Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling, arXiv preprint arXiv:241..05271, 2024.
[66] Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. mplug-ow13: Towards long image-sequence understanding in multi-modal large language models. arXiv preprint arXiv:2408. 04840, 2024.
[67] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang. Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR. 1(2):3, 2022.
[68] Bingchen Zhao, Yongshuo Zong, Letian Zhang, and Timothy Hospedales. Benchmarking multi- mulli-imaae understanding in vision and language models: Perception, knowledge, reasoning. and multi-hop reasoming. arXiv preprint arXiv:2406.122
2024.
[69] Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang. Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. In European Conference on Computer Vision, pages 166-166. Springer, 2024.
[70] Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, and Yoav Artzi A corpus for reasoning about natural language grounded in photographs. arXiv preprint arXiv: 10t::://1.000 1814,00 2018.
[71] Zicheng Zhang, Haoning Wu, Erli Zhang, Guangtao Zhai, and Weisi Lin. Q-bench: A benchmark for multi-modal foundation models on low-level vision from single images to pairs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[72] Haowei Liu, Xi Zhang, Haiyang Xu, Yaya Shi, Chaoya Jiang, Ming Yan, Ji Zhang, Fei Huang, Chunfeng Yuan, Bing Li, et al. Mibench: Evaluating multimodal large language models over multiple images. arXiv preprint 20X::2407..1272.. 2024.
15

---

[73] Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, al. Mmmu: massive multi-disciplim multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 9556-955655 2024.
[74] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, al. Are the right way for evaluating large vision- -aiiuull models? arXiv preprint arXiv:2403.20330, 2024.
[75] Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507-2521, 2022.
[76] Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. In Computer Vision-ECCY 2016: 14th European Conference, Amsterlam, The Netherlands, October 11-14, 2016, Proceedings, Part IV 14, pages 235 251. Springer, 2016.
[77] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision lasgualanguuae models. arXiv preprint arXiv:2305. 10355, 2023.
[78] Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang. Wangbo Zhao, Yike Yuan, Jiaqi Wang. Conghui He, Ziwei Liu, al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216-233. Springer, 2024.
[79] Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: on the hidden mystery of ocr in large multimodal models. Science China Information Sciences, 67(1)): 2024.
[80] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
[81] Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejum Ma, and Chunyuan Li. Llate-aext-ntterlaaa:: Tackling multi- multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07888 2024.
[82] Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308. 12966, 1(2):3, 2023.
[83] Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. Openflamingo: An open- source framework for training large autoregressive vision-language models. arXiv preprint arXiv:2308.01390 2023.
[84] Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023.
[85] Weiyun Wang, Shuibo Zhang, Yiming Ren, Yuchen Duan, Tiantong Li, Shuo Liu, Mengkang Hu, Zhe Chen, Kaipeng Zhang, Lewei Lu, et al. Needle in a multimodal haystack. Advances in Neural Information Processing Systems, 37:20540-20565 2024.
[86] Jianrui Zhang, Mu Cai, Tengyang Xie, and Yong Jae Lee. Countereurate: Enhaneing physical and semantic visio- linguliing compositional reasoning via counterfactual examples. arXiv preprint arXiv:2402.132 2024.
[87] Junzhuo Liu, Xuzheng Yang, Weiwei Li, and Peng Wang. Finecops-ref: A new dataset and task for fine-grained compositional referring expression comprehension. arXiv preprint arXiv:2409.14770 2024.
16

---

Technical Appendices
In this appendix, we provide additional materials to support more comprchensive understanding of our proposed method and dataset. In Sec. A, we detail the low-cost construction pipeline of MultiScope-42k and conduct comparative data analysis with MIA-DPO, including token length distributions, word cloud statistics, and supervision source breakdown. We also clarify the image source overlap between training data and benchmarks ensure fair evaluation. In Sec. B, summarize all benchmarks used in evaluation, including seven multi-image and eight single-image benchmarks. In Sec. C, provide additional experimental results, including ablation studies training data volume and supervision granularity to assess their impact on model performance. In Sec. D, we present qualitative observations and visualizations of preference pairs.
A
MultiScope-42k: A Context-to-Cue Captioning DPO Dataset
A.1 Low-cost Question-Answer Pair Construction
Constructing high-quality instruction-resppnse preference pairs for multi-image learning traditionally requires extensive manual annotation, especially when capturing subtle context dynamics or region- level semanties. To address this bottleneck, we design low-cost, fully automated pipeline for question-answer pair construction, enabling efficient and scalable data generation with broad coverage and controlled distributional properties.
Automated Caption Pool Sampling. We first leverage existing vision-language MaaasetssLLaaiee 23K [61], MDVP [63], and MVC [49]-10 construct diverse caption pool, containing both image- level and region-level descriptions. By decoupling question construction from caption generation, we able to sample visual contexts and their aligned captions independently, facilitating large-scale composition of input-output examples.
Structured QA Formatting. Given sampled image image sequence, construct templated instructions (e.g.. "Describe the following images" or "Please describe the marked area in each image") to form queries. For the corresponding answers, we structured formats that encourage compositional reasoning and grounding, such as:
[ For Image 1: <caption 1>, For Image 2: <caption 2>, ...]
For the marked area of Image X: <caption X>, ...]
This approach allows flexible variation in image number, visual scope, and response granular- ity sy-puportiing both context- and region-level supervision.
To generate preference pairs without additional labeling, we apply lightweight perturbation strategies to the answer side only. Specifically:
Controlled Perturbation for Preference Learning.
* (1) Truncation and swapping: simulate omissions and misalignments in context-level answers.
* (2) Region mismatches: in needle-level samples simulate detail hallucination.
These perturbations require human involvement yet introduce controlled emrors mirroring real- world MLLM failure modes, enabling scalable preference pair generation with low cost.
Efficient Coverage of Diverse Visual Distributions. With stratified control over image domains, scene compositions, and region attributes via automated sampling, we construct MultiScope-422 a large-scale corpus covering diverse multi-image tasks. Its diversity in visual layout and semantic granularity delivers robust preference supervision across image types and reasoning levels.
Overall, our pair construction strategy eliminates the need for dense manual annotation while producing rich and challenging preference data scale scalc-offering a practical solution for instruction tuning in multi-image multimodal models.
A.2 Data Analysis
To better understand the characteristics of our preference supervision, conduct comparative analysis of MultiScope-42k and MIA-DPO from both lexical and structural perspectives.
Dataset Composition Overview. Summary statistics of the proposed MultiScope-42k are presented in Table 9, which divided into three core subsets by supervision granularity and image source, totaling 41.8k preference pairs (27.3k context-level, 10.8k needle-level TDPO, and 3,7 3.7k needle-level
17

---

Table 9: Summary statistics MultiScope-42k by supervision level and image source, including total
pairs, number of images per instance, and average token lengths of chosen and rejected responses.
Level
Image Source
Total
Images Range
Avg. Chosen Len.
Avg. Rejected Len.
Context-l.avel
COCO-2014
27.3k
12, 5]
285.24
165.45
Needle-L.vel--DDR
COCO-2017
10.8k
12, 4]
173.35
173.97
Needle-Level-VDPO
Flickr30k
3.7k
[2. 4]
66.09
65,98
Chosen MultSScope-22
Chosen MIA-DPO
Rejested MultiScape-422
Rejected MIA-DPO
10000
3000
Count
2000
5000
2000
6000
5000
1932
2000
200
450
40
20
500
200
633
Deo 150
Token Length
Token Length
Token Length
Taken Length
Figure 4: Token length distributions of chosen and rejected responses in our MultiScope--2k and MIA-DPO [27]. MultiScope-42K exhibits significantly longer and more diverse answers, while MIA-DPO responses remain short and concentrated, indicating a simpler response pattern.
Chosen MultiScape-42
Chosen MIA-DPO
Rejected MultiScope 42K
Rejected MIA-DPO
image
right is
seen
FAREDAR
de
Oppop
whiter
man
ma
b
üe
woman
WO D Fagar
e
otherss dea
group
marked
aeggenrii Fight "ide
mar
ked point
area
table
staandingg
miit hor reft t
appeal
side
table
man
neldrng
marked area
son.......
holding i
mage
Ima
tabl scene
proup
might bi scene gitt
Water
backg grrgnng een Cbar
table
white enrand
per person
Figure 5: The dataset word cloud comparison between our MultiScope-42k and MIA-DPO [27].
VDPO) to cover supervision needs of varying granularities in multi-image scenarios. The 27.3k context-level pairs incorporate upproximately 7k multi-image QA data from Mantis [44] to enhance the model's iustruction-follooing capability. The negative samples are generated by the model based on responses to images with Gaussian noise added, inducing the model to produce answers that are unfaithful to the original images. Regarding the number of images per instance, all subsets contain 2 to images (up to 5 for context-level, 4 for needle-level), aligning with typical multi-image learning settings and supporting models in modeling multi-image sequence contexts.
Token Length Distributions. Fig. 4 presents the token length histograms of both chosen and rejected responses. MultiScope-42k responses are significantly longer on average and display wider spread, with many responses exceeding 400 tokens. This reflects the dataset's 's multi-stage, image-wise captioning format and compositional design. In contrast, MIA-DPO responses short and concentrated, with the majority under 30 tokens. This further suggests that MultiScope-42k provides richer and more diverse supervision signals, especially for multi-image reasoning.
Linguistic Focus via Word Cloud. We visualize the answer sets of both datasets using word clouds. As shown in Fig. 5, MultiScope-42k responses prominently feature structured and spatially grounded expressions such as "image," " "marked area," " "left," " "right," " "foreground," " "scene", indicating a strong alignment with multi-image, region-specific prompts. In contrast, MIA-DPO emphasizes atomic visual concepts (e.g., "man," " "table," "dog." " "red"), which are well-suited for single-image tasks but lack explicit inter-image reference or structural composition.
A.3 Dataset Independence Statement
To ensure fair and unbiased evaluation, we verify that the benchmarks used for testing do not overlap with the data sources involved in model training. We employ three types of annotated data for
two-level DPO training.
(1) Image-level captions are sourced from L.LAVA-23K [61], which is
derived from the COCO-2014 dataset. (2) Region-level captions come from the MDVP dataset [63],
based on COCO-2017. (3) Visual contrastive preference pairs are taken from MVC [49], whose image sources include CounterCurate [86] and FineCops-Ref Ref [87].
18

---

We conducted thorough review of all evaluation benchmarks for potential overlap with these training sources (COCO-2014 COCO-2017, Flickr30k). While most multi-image benchmarks appear be independently constructed, we conservatively flag the following for partial uncertain overlap:
MIRB [68] explicitly uses COCO images (e.g., for counting), and includes content from ImageNet-R, OpenFoodFact, Bitton et al., and arXiv. MIBench [72] comprises multiple public datasets, including VrR-VG, which inherits COCO images via Visual Genome. MUIRBench [42] integrates existing (e.g., SeedBench, IconQA), derived (e.g., NLVR2, MMBench), and newly collected datasets. While its new data COCO free, MMBench and IconQA are known to include COCO images.
summary of benchmark image sources is provided below:
MUIRBench combines new and repurposed datasets; some derived components include COCO. MIRB includes COCO, ImageNet-R, and other web-sourced content. BLINK uses synthetic and controlled real-world images. Mantis-Fval draws from web queries and manual composition, with no COCO usage. NLVR2 uses Flickr images and is COCCOindependern Q-Bench2 is based on IQA datasets like KonIQ- 10k and BID, unrelated COCO or Flickr30k. MIBench includes VrR-VG, which partially overlaps with COCO.
While a few benchmarks partially overlap with COCO or Flickr30k, evaluation faimess remains uncompromised for two reasons. First, our primary baseline MIA-DPO is also trained on LLaVA-233 (COCO-based), ensuring all methods are compared on equal footing. Second, our DPO training focuses on image-caption preference alignment, not question answering. These points validate the integrity and faimess of our evaluation protocol, despite partial dataset overlaps.
B Benchmark Sources
B.1 More Details on the Construction of Context-AMBER-IK
We design a caption generation task as a proxy to systematically evaluate MLLMs' multi-image understanding, identifying three fundamental hallucination types types-Contedd Context Omission, Context Confiation, and Detail Misinterpretation -that critically degrade model performance. To en- able this evaluation, we construct Context-AMBER-IK by systematically concatenating im- ages from the single-image AMBER dataset [37] into sequences of two types: short-context se- quences with images and long-context sequences with 8 images. Each input is paired with the prompt: "Please sequentially describe each of the images shown above. Use the format: For Image :description>.."
To evaluate Detail Misinterpretation, we employ three established metrics: (a) CHAIR [36]; (b) Response-level hallucination rate (Hal); and (c) Cognition-based hallucination (Cog). Each generated caption is matched to its corresponding image using regular expressions to extract per- image descriptions. We then compute hallucination scores for each image separately and average these scores across the entire image sequence. To assess Context Omission and Context Conflation, we introduce two rule-based penalization strategies:
(1) For context omission, if fewer than N captions are generated (e.g., the response ends prema- turely at {For Image N-1: <caption N-1>}), we explicitly pad the output to {For Image N: <caption N-1>}) to maintain a consistent structure. Missing or incomplete captions are heavily penalized. Additionally, we introduce the (d) Sequence Coverage Rate (SCover), a metric that evaluates caption completeness across image sequences and quantifies the degree of context omission.
(2) For context conflation, include the keyword "sequentially" in the prompt to encourage models to describe images in order. If the model cannot clearly associate each description with its cor- responding image (c.g.. For Image 1 1:<caption1>, For Image 3: 3<<aapion2>>> For Image 2: <<natioon>>). the resulting hallucination scores will be higher for out-of-order descriptions compared to their correct GTs, the mismatch leads to larger discrepancy.
B.2 Multi-Image Benchmarks
We employ seven multi-image Benchmarks-MMURRReendd MUIRBench [42], MIRB [68], BLINK [69], Mantis- Eval [44], NLVR2 [70], Q-Bench2 [71]. and MIBench [72]-to holistically evaluate multi-image
19

---

reasoning across four key dimensions: co-reference alignment, fine-grained comparison, contextual reasoning, and temporal understanding.
MUIRBench [42] is a comprehensive benchmark specifically designed to evaluate the robustness of multimodal large language models (MILMs)) in multi-image understanding scenarios. It comprises 2,600 multiplechhoice questions and 1 11,264 images, averaging 4.3 images per instance. The bench- mark covers 12 distinct multi-image understanding tasks tasks-including action understanding, diagram reasoning, geographic comprehension, and visual setrieval-spanning 10 diverse multi-image relation types such as temporal, narrative, and scene- mene-mulliview relations. To ensure both comprehensive- ness and robustness, MUIRBench adopts a pairwise design: each standard (answerable) question is paired with an unanswerable variant with minimal semantic perturbations. This enables fine-grained assessments of both reasoning capability and abstention behavior.
MIRB [68] is comprehensive benchmark designed evaluate vision-language models (VLMs) on four distinct aspects of multi-image understanding: perception, visual world knowledge, reasoning, and multi-hop reasoning. It comprises 925 multi-image questions across these categories, averaging 3.78 images per question, with some tasks requiring up to 42 images for complex reasoning. Unlike prior benchmarks that reuse video frames, MIRB independently sources images from real-world domains, such as code snippets, sightseeing scenes, food ingredient lists, and arXiv papers, ensuring diverse and challenging visual contexts. The benchmark includes wide array of tasks: image jigsaw reconstruction, object counting, attribute matching (Perception): food label comparison and geographic recognition (Knowledge); visual analogy. code understanding, 3D scene analysis (Rea- soning); and synthetic logic chains and citation lookups (Multi-Hop). Each question is formulated to necessitate reasoning across multiple images rather than from a single image.
MIBench [72] is large-scale benchmark designed to comprehensively evaluate the fine-grained multi-image understanding abilities of multimodal large language models (MLLMs). It categorizes multi-image inputs into three representative scenarios-Multi-lnnee Knowledge-Seeking (MKS), and Multimodal In-Context Learning
Instruction (MII), Multimodal
total of 13 distinct tasks and 13,000 annotated samples. In the MII setting, the model must perform perception, comparison, and reasoning over multiple images across five tasks: general comparison (GC), subtle difference (SD), visual referring (VR), temporal reasoning (TR),, and logical reasoning (LR). The MKS scenario evaluates the model's ability to extract and align information from interleaved image- text knowledge sources through four tasks: fine- mme-grained visual recognition (FVR), text-rich image VQA (TRI), rision-linked textual knowledge (VTK), and text-linked visual knowledge (TVK). Finally, the MIC setting assesses multimodal in-context learning across four tasks, including close- ended and open-ended VQA, hallucination mitigation, and demo-based task learning.
BLINK [69] tests rapid visual cognition through perceptual similarity, forensic analysis, and spa- tiotemporal matching. It includes highty--cotrooled multi-image tasks such as depth estimation, object matching, and outlier detection, with an emphasis on speed and perceptual accuracy.
Mantis-Eval [44] introduces 217 multi- tasks curated for conceptual inference, including abstract reasoning over physical quantities such as number, size, and weight. It combines both multiple-choice and open-ended questions, drawing from web-sourced image sets manually organized into logical visual groupings.
NLVR2 [70] (Natural Language Visual Reasoning) assesses model's ability to verify textual hypotheses against a pair of images. Each sample requires binary classification (True/False) over whether the provided statement is consistent with both images, making it a canonical test for visual entailment and compositional reasoning.
Q-Bench2 [71] is diagnostic benchmark tailored for evaluating visual quality perception and comparative assessment across image sets. It challenges models to identify subtle visual artifacts, distortions, or improvements between similar images. Our evaluation is based on the Q-Bench2-AI dev subset, which emphasizes multi-image multiple-choice assessments for perceptual judgment.
B.3 Single-Image Benchmarks
We test the model on eight representative single-image benchmarks assess specific multimodal capabil-
ities: (1) Academic/Scientific Reasoning: MMMU [73], MMStar [74], ScienceQA [75], (2) Diagram Understanding: A12D [76]. (3) Robustness against hallucinations: POPE [77], HallBench [38], (4)
General Multimodal Abilities: MMBench [78], (5) Text Recognition: OCRBench [79]. The results
20

---

Table 10: Benchmark Sources. We have included detailed information for all the multi-image and
single-image benchmarks tested in the paper in the table.
Setting
Models
Evaluation Metric
Number
Source
MUIRBench [42]
Multiple Choice
2,600
MUIRBench
MIRB [68]
Multiple Choice
925
MIRB
Multi-Image Benchmark
MIBench [72]
Multiple Choice
13,000
MIBench
BLINK [69]
Multiple Choice
3,807
BLINK
NLVR2 [70]
Multiple Choice
6,967
NLVR2
Q-Bench2 [71]
Multiple Choice
1,000
Q-Bench2
Mantis-Eval [44]
Multiple Choice
217
Mantis-Eval
MMStar [74]
Multiple Choice
1,500
MMStar
MMMU [73]
Multiple Choice
1,050
MMMU
Sci-QA [75]
Multiple Choice
4,241
ScienceQA
Single-Image Benchmark
POPE [77]
Yes/No
9,000
POPE
HallBench [38]
Yes/No
951
HallusionBench
MMB [78]
Multiple Choice
1,164
MMBench
OCR [79]
VQA
1,000
OCRBench
AT2D [76]
Multiple Choice
3,090
A12D
on this diverse set benchmarks demonstrate the effectiveness the proposed method, particularly in multi-image settings, confirming significant performance improvements.
MMMU [73] (Massive Multimodal Multitask Understanding) includes over 10k university-Iewe questions from 30+ disciplines such as physics, medicine, and art. It requires detailed reasoning over image-text inputs and is designed evaluate advanced academic-level understanding.
MMStar [74] is comprehensive diagnostic benchmark covering various sub-tasks such as OCR, VQA, and caption grounding, offering structured and hierarchical annotations across domains like natural science, medicine, and design.
ScienceQA [75] contains over 21k science questions aligned with elementary and middle school curricula, involving images such as diagrams and charts. tests the model's 's capability to perform science-related visual reasoning in multimodal format.
AI2D [76] (Allen Institute Diagram) features manually annotated science diagrams with associated multiple-choice questions. It focuses on assessing the model's 's understanding of labeled structures and their functional roles within the image.
POPE [77] (Position and Object-level Prompt Evaluation) is designed to test a model's resistance to hallucinations. It uses minimally perturbed prompts to identify failure cases in positional grounding and object identification, highlighting model robustness.
HallBench [38] provides structured framework measure hallucination frequency and grounding quality by comparing model outputs with annotated ground truths. It supports fine-grained scoring across categories such as incorrect object mentions or unsupported claims.
MMBench [78] a general-purpose evaluation benchmark comprising questions across 11 modalities including VQA, captioning, OCR, and commonsense reasoning. It uses GPT-4-based grading to ensure high-fidelity evaluation of answer correctness.
OCRBench [79] specifically targets the model's capability to recognize and reason about text in the visual domain, covering a range of document layouts, fonts, and multilingual content with both exact-match and reasoning-bree based questions.
C More Experiments
Data Scale Alignment with MIA-DPO. To assess the impact of training size and ensure a fair comparison with MIA-DPO, we conduct an ablation in Tab. 11 using similar total number of preference pairs. Specifically, we randomly sample 13.6k from our 27.3k Context-Level pairs and combine them with the fixed 14.5k Needle-Leve data, resulting in 28.1 training set compaaaaael to MIA-DPO's 28.9k. Notably, under this matched training data size, CcDPO still outperforms
21

---

Table 11: Ablation training data volume. To match MIA-DPO's training data size, we down- sample our Context-Leve data to 13.6k while keeping needle-level data fixed. The results reveal a trade-off between modeling global context and capturing fine-grained details: while reduced Context- Level data leads to performance drops on most multi-image tasks, benchmarks like BLINK and foouseh on localized perception- from higher proportion of needle-level data.
Models
Data Size
MuirBench
MIRB BLINK Mantis NLVR2
MIBench Q-Bench2 Average
LLaVA-OV [32]
42.5
47.3
51.1
60.4
89.4
73.6
73.8
62.5
MIA-DPO [27]
9K
41.4
48.0
53.7
60.3
88.2
67.8
74.0
61.9
+ CcDPO (Ours)
28.1K IK
46.7
51.2
56.5
69.1
90.7
72.1
79.3
66.5
+ CcDPO (Ours)
41.8K
48.6
51.4
55.9
69.6
91.2
75.2
77.6
67.1
+6.1
+4.1
+4.8
+9.2
+1.8
+1.6
+3.8
+4.6
Table 13: Performance on VideoMME Benchmark. CcDPO-Video improves baseline performance across core video tasks, verifying cross-domain generalizability.
Model
Overall
Perception
OCR Problems
Temporal
Spatial
Attribute
Action
Object
VL-VA-OV [32]
53.7
0.491
0,556 0.574
0.685
0.505
0.559
0.568
+ C&DPO- Video (Ours)
54.4
0.509
0.698
0.527
0.582
0.547
Model
Reasoning Spatial
Information Synopsis
Counting
Temporal
Action
Object
LLaVA-OV [32]
0.354
0,384
0.714
0.523
0,518
0.675
+ CcDPO- Video (Ours)
0.377
0.401
0.732
0.519
0.511
0,663
MIA-DPO across all benchmarks, demonstrating the effectiveness of our structured, dual-level supervision. In particular, the reduced-context setting yields better performance on BLINK and Q-Bench2, suggesting that relatively higher proportion of needle-level data may benefit fine-grained perceptual tasks. On the other hand, performance on context-heavy benchmarks drops slightly, likely due to weaker global context modeling, Overall, training with the full 42k preference set (28. Ik Context-Level 14.5k Needle-Level) leads to the best average performance. These results highlight the advantage of high-quality, large-scale supervision, while also revealing a trade-off between contextual alignment and perceptual precision.
Ablation of Training Strategies. We investigate whether our multi-stage training strategy-fins first training on Context-Level data, followed by Needle-Level supervision-is more effective than single-stage approach that mixes both types of data from the beginning. As shown in Tab. 12, the multi-stage strategy consistently outperforms the mixed-data alternative across benchmarks. We attribute this improvement to the se- quential learning structure. In the first stage, the model learns to capture global context and image-level coherence through structured, requence-aligned supervision. Once this foundation established, the sec- ond stage refines the model's 's ability to at- tend to local, fine-grained visual cues via region-specific preference optimization. In contrast, the one-stage strategy may dilute the model's focus by simultancously exposing it to competing global and local objectives, making optimization less efficient. These results suggest that decoupling context modeling and fine-graing grounding into separate stages can better guide the model toward hierarchical visual reasoning.
Table 12: training outperforms one-stage mixed training.
Ablation on training strategy. Multi-stage
Strategy
MuirBench BLINK Mantis
LLaVA-OV
42.5
51.1
60.4
One-Stage Multi-Stage 48.5
46.2
55.1
65.4
55.9
69.6
Framework Extension to Video Data. To explore our framework's potential for sequen- tial visual data beyond multi-images, we extend its core idea of stitching individual im- ages into multi-image sequences to video scenarios, adopting the structured input format: (video clip,) (caption,) (video clip,) (captiong). We directly adapt our Sequence Truncation and Content Swapping strategies for negative sample generation, avoiding costly manual re-labeling or rejection sampling. Following this pipeline, we construct a 7k-scale video preference dataset and perform DPO training the VLVVA-OV baseline (denoted as CeDPO-Video)
22

---

As shown in Table 13, CcDPO-Vider achieves a 0.7-point overall gain over LLVVA-OV on VideoMME, with measurable improvements temporal perception (+0.018), action recognition (+0.022), and object recognition (+0.023). These exploratory results highlight two promising strengths of our approach: (1) Video clip stitching- an extension of our multi-image sequence construction paradigm-is computationally economical, obviating the need for manual re-annotation or heavy computational overhead associated with existing methods; (2) Its perturbation-base sampling is transferable to sequential visual data: positive samples use semantically ordered video clips, while negatives are generated via scrambling or truncation. This trial thus demonstrates feasible pathway to extend multi- aulti-image DPO frameworks to video understanding tasks.
D More Observations
As illustrated in Fig. 6, 8, 7, 10, and 11, we present additional qualitative examples of the constructed preference pairs used in our CcDPO training. These examples cover range of perturbation types across both Context-Level and Needle-Level supervision, each designed to target specific failure modes in multi-image reasoning.
Fig, shows a context-level complete content omission scenario, where the rejected response omits part of the image sequence. This simulates context omission enror, encouraging the model to favor complete and coherent descriptions across multiple images.
Fig. 7 demonstrates context-level partial content omission scenario, where the rejected response uses overly brief or generic captions. Such examples train the model prefer rich, fine-grained visual descriptions that capture meaningful details.
of caption swapping, where descriptions mismatched with image indices. This context conflation perturbation helps the model learn to correctly associate content with its corresponding image.
Fig. 8 presents
Fig. 9 and Fig. 10 move to the Needle-Level and show a case where region-specinn (bbox or point) captions mismatched. This encourages the model to distinguish between accurate and inaccurate local grounding within a given image.
Fig. 11 illustrates the use of contrastive image inputs for the same caption, where one image visually aligns with the text while the other introduces a subtle contradiction. This visual preference pair sharpens the model's ability to resolve ambiguous references through visual evidence alone.
Together, these examples highlight how different types of preference signals are systematically constructed to improve the model's ability to reason over sequences, resolve visual references, and discriminate between subtle semantic mismatches. By explicitly encoding both global and local reasoning objectives into the supervision data, CcDPO promotes more robust and interpretable multi-image alignment.
23

---

Question: What this photo about? iimutttiiiiieeeiiiee--m-mmmm-em
For image 1: The image depicts two horses pulling a carriage on a street. Both horses beautifully decorated, and one them appears to be wearing chain mail hats. They tethered to the carriage and working in unison to tronsport it through the street. Thene are several people in the scene, all around the horses and carriage, Some of the people are walking along the sidewalk, while others may be getting the ride side of the scene, indicating controlled intersection. \n
Chosen:
off the carriage. stop sign is located
For image The shows holding tennis racket in his hand, with the ball positioned above him, preparing for the hit. Several spectators surround court area, attentively watching man performance. \n'nOther items be around the court, such suitcase located the right side and backpack towards the left. Many of the spectators are focused on the tennis player, eagerly anticipating the start the game.
tennis ready
the tennis ball. He
The image shows a group of people wearing wet suits at a beach, carrying surfboards under their arms while the water.
For image 3:
surfers are farther in the water, while person can be seen walking towards the water with their board, \n\nIn the background, multiple airplones are flying in the sky, and small boat noticeable near the middle of the scene, The bedyboarders
having a great time enjoying their presence of clouds contributing to the beach atmosphere. \n
activities, with the
crowded beach on people enjoying their time by the ocean, some standing or walking around, while others have set colarful umbrellas and chairs to relax on the beach. \nn in'nnlayy large umbrellas are spread along the shoreline the their vibrant colors. A variety of chairs can be found in the area, some near the umbrellas and others scattered across the beach. \n'nA few surfboards handbags ore also visible, indicating that people are likely engaging water activities and have brought personal belongings for their day at the beach.
image The image
day with blue sky. There numerous
providing shade for beachgoens, and brightening up the scene with
Rejected: For image The image depicts two horses pulling a carriage on a street. Both horses are beautifully decorated, and one of them appears to be wearing chain mail hats. They are tethered to the carriage and working in unison to transport through the street. \nAn scene, all around the horses and carriage. Some the people are walking along the sidewalk, while others may be getting on or off the carriage. A stop sign is located the ride side of the scene, indicating controlled intersection.
several people in the
For image 4: The image is of a crowded beach on clear day with blue sky. There numerous people enjoying their time by the ocean, some standing walking around, while others have set up colorful umbrellas and chairs relox on the beach. \n in'nMany large umbrellas are spread along the shoreline near the water, providing shade for beachgoers, and brightening their vibrant colors. variety of chairs can be found in the area, some near the umbrellas and others scattered across the beach. An'AnA few surfboards and handbags are also visible, indicating that people are likely engaging water activities and have brought personal belongings for their day at the beach.
the scene with
Figure 6: Example of Context-Level preference pair with truncation perturbation (Context Omission).
24

---

Question: What is this photo about? about?<image>> image > image> <mmage>
For image 1: The image features large group of people gathered at sandy beach, flying various kites in the cloudy sky. There are more than ten people visible in the scene, dispersed throughout the beach area. The kites come assorted shapes and sizes, with some resembling squid and octopus wind streamers soaring above the crowd. 'h'nThere are also multiple chairs placed in the scene, accommodating those who want to take break from kite flying or simply enjoy the spectacle. Additionally, few handbags lying the beach, close their respective owners.
Chosen:
For image 2: In the image, woman is standing outside near curb with her luggage. She has her hand on her hip, looking confident perhaps waiting for someone. Her luggage consists of suitcase next to her and a handbag placed closer to her. The scene appears to be near an airport or a drop-off offp off/pick- up area, with several cars parked or driving nearby. bench is located on the far right of the scene, commonly found such locations. Additionally, there are multiple other people various sized handbags suitcases spotted the area, indicating that this is a bustling and busy place for travelers. An
The image features man inside convenience store restaurant, in the process of placing hot dog a bun. He surrounded by an assortment of hot dogs in various positions, indicating the convenience store or restaurant has wide selection of hot dogs for customers to choose from. The seems focused preparing his chosen hot dog for consumption. An
For image 3:
For image 4: A woman is standing on stage, addressing her audience with microphone. She is wearing white and red outfit, and there is a large black and white picture in the background. The woman appears be giving lecture or making presentation to her students or diverse audience. There are several people scattered throughout the image, indicating sizeable gathering listening to the woman's 's lecture. In the vicinity, various cars are parked outside truck
the venue, some with bit of distance between them. There is parked further out near some other vehicles. Additionally, there a clock visible high up on a wall.".
Rejected: For image 1: Crowds of people are at the beach with squid and octopus wind streamers above.
For image 2: A woman standing at an airport next to luggage.
For image 3: The man is at the convenience store selecting hotdog.
For image The woman is speaking her audience on stage.
Figure 7: Example of Context-Level preference pair with caption shortening perturbation (Context Omission).
25

---

Question: What this photo cbout?>imnge--<imgg>>>>
Chosen: For image 1: The image showcases bathroom illuminated by stained glass window, with white bathtub located next to stone wall. The room features stone walls and floors, giving the bathroom rustic feel. AeelAA the bathroom, sink is situated along the wall, while wooden chair is placed in the foreground. Several bottles and cups can also be found near the sink, likely containing bathroom essentials, such as shampoo, soap, and other toiletries. An
The living room is elegantly arranged with old-fashioned, antique furniture and features exquisite details the woodwork. There an ornate couch situated on the left side of the room, and several antique style chairs are placed around a dining table. The parlor features an oriental rug, adding an extra layer of charm the room. The living various decorations such as multiple vases and potted plant, which accentuate the antique atmosphere.
For image 2:
also adorned with
In the image, group of people, including both and women, is gathered a park flying kites. Among them, two people are prominently seen flying a red kite in the field, possibly running to keep it airborne. The audience of onlookers is spread out in the park, watching and enjoying the kite-flying activities. An\nSome chairs relax. Additionally, there are few traffic lights visible, indicating that the park is likely situated near a road or busy area.
For image 3:
placed around the park for people to sit and
Rejected: For image The image showcases bathroom illuminated by a stained glass window, with white bathtub located next to stone wall. The room features stone walls and floors, giving the bathroom rustic feel. in'nIn the bathroom, sink situated along the wall, while wooden chair is placed the foreground. Several bottles and cups can also be found near the sink, likely containing bathroom essentials, such as shampoo, soap, and other toiletries. \n
For image 2: In the image, a group of people, including both men and women, gathered in park flying kites. Among them, two people are prominently seen flying red kite in the field, possibly running to keep it airborne. The audience of onlookers is spread out in the park, watching and enjoying the kite-flying activities. in'nnome chairs are placed around the park for people to and relax. Additionally, there are a few traffic lights visible, indicating that the park is likely situated near road or busy area.
For image 3: The living room elegantly arranged with fashiooiddd, antique furniture and features exquisite details in the woodwork. There is an ornate couch situated on the left side of the room, and several antique style chairs are placed around dining table. The parlor features extra layer of charm to the room. \n\nThe living room also adorned with various decorations such as multiple vases and potted plant, which accentuate the ontique atmosphere.
oriental rug, adding
Figure Example of Context- Level preference pair with swapping perturbation (Context Conflation).
26

---

Question:
<image><imag
Please provide detailed description of the marked areas each image. The marked areas are indicated by green rectangle with a 'REF' label around it.".
Chosen: For the marked areas of image 1: large commercial-grade stove crawls across a significant section of the kitchen, sparking with arrays of burners. Numerous knobs dot the front, lending look of complexity and functionality to the appliance. \n
For the marked areas of image 2: On the far right of the image, there's $ notably round, possibly teardrop-shaped, vase. This vase seems to be the bulkiest among all, being referred to as the 'fat one'. It is situated in front of some objects which appear to be red, An
For the marked areas of image 3: An arm wearing a purple sleeve is seen in the top right corner of the image, with hand gripping a fork poised to cut into some food. \n
For the marked areas of image 4: A tall, lush potted plant can be observed amidst a setting of a white building. This plant, which could be described bush large shrub, is positioned between green and red signs. It appears as a significant green element between these signs, contributing to the natural allure of the scene.
Rejected: For the marked areas of image 1: The right part of stainless- steel oven door is visible in the corner the image. It almost seems cut-off from the frame and includes sturdy handle, only partially seen due to positioning. \n
For the marked areas image 2: the center of the image, there's 's blue vase, distinctively sandwiched between two other vases. Its location in the very center of the image makes it a reference point, much like '12 noon' on clock. \n
For the marked areas of image 3: woman a grey shirt is seen at the top of the image with her left hand gracefully placed sandwich. She garners further attention due to the shiny ring her finger, indicating her fidelity to her husband.
For the marked areas of image 4: On the bottom left, there's 's noticeably yellowish-green bush, planted in pot. It's positioned interestingly, partially covering a blue sign situated next a sign indicating 'Moulin Hotel'. The position and color of this bush add an element of intrigue and a splash green to the urban scene.
Figure 9: Example of Needle-Level preference pair with bbox region mismatches perturbation (Detail Misinterpret).
27

---

Question: simmage><mmage <mage>> >mmee>>>mmeeeeeee provide detailed description of the marked point in each image. The marked points indicated by a hollow circle with 'REF' label positioned above the circle.
Chosen: For the marked point of image 1: The region shows segment of baseball field, focusing on the batter's area by home plate. Visible are players in uniforms engaged in the game, with one in stance to bat and catcher and umpire position behind him. The dirt-covered ground, home plate, and the players' equipment, like the batter's helmet, are prominent. \n
This region shows commercial airplane tail with distinctive circular branding. The aircraft is stationary the tarmac, and the tail appears prominently against a cloudy sky.
For the marked point of image 2:
For the marked point of image 3: This is image of zebra positioned in profile, grazing the ground. It full-grown adult with characteristic black and white striped fur. The mark placed near the upper region the zebra's s body, close to the base of its neck. This particular stripe pattern unique to each individual zebra, similar to human fingerprint. The immediate area around the mark shows the zebra's mane, which consists of short, erect fur that follows along the neck's curve. \n
For the marked point of image 4: A notepad likely containing handwritten notes or drawings, surrounded by creative writing utensils.
Rejected: For the marked point of image 1: This is display of large, horizontal electronic advertisement board located the edge of the stands baseball stadium, showing a commercial message or sponsorship. The board appears to be illuminated and is part of the stadium's 's infrastructure, meant for the audience's viewing. An
For the marked point of image 2: Focus on the wingtip of an airplane, showing the curved end and navigation lights. The wing is against backdrop of the sky ranging from pale to darker shades, possibly indicating cloudy weather.
For the marked point of image 3: This is an image zebra positioned in profile, grazing on the ground. It is full-grown adult with characteristic black and white striped fur. The mark is placed near the upper region of the zebra's body, close to the base each individual zebra, similar to human fingerprint. The immediate area around the mark shows the zebra's mane, which consists of short, erect fur that follows along the neck's curve. An
neck. This particular stripe pattern unique
For the marked point of image 4: A small, possibly decorative bowl that may be utilized for holding various small items or as decor piece itself.
Example of Needle-Level preference pair with point region mismatches perturbation Figure 10:
(Detail Misinterpret).
28

---

Chosen
Images
Rejected
Images
Question: <image> <image> image> image> \n For image 1, what can you tell me about the colors of the child's clothing while they' re opening the gift, and can you describe the scene around them?\n For image 2, how is the thinner woman positioned in relation to the person in the black shirt in the image?\n \n For image 3, what can you tell me about the position of the elderly Asian man with white hair in relation to the Chinese characters in the image? Is he the left the right?\n For image 4, how many people are there the image, and how many costumes can you see?
Chosen: For image 1, The child is dressed in red and gray while. opening a gray and white striped gift, sitting on man't lap who wearing blue shirt, and together they are positioned in front of window and small Christmas tree. \n
For image 2,
The thinner woman is positioned to the right of the person the black shirt. An
For image 3, The elderly Asian man with white hair is to the right of the Chinese characters.Vv \n
For image 4, There are people and 6 costumes.
Figure 11: Example of Needle-Level preference pair with image contrastive perturbation (Detail Misinterpret).
29

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: In the introduction, clearly outline the key challenges in multi multi-image under- standing and present CcDPO as a two-level solution that directly addresses these issues. de- scribe core secchds--structuree captioning, perturbation- based training, and region-focused prompts- and introduce the MultiScope-42k dataset to support them. These contributions align closely with the claims, accurately reflecting the scope of my work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
We discuss that CcDPO is designed for general multi-image reasoning and does not explicitly model temporally correlated inputs like vidcos, which may limit its performance in such scenarios. Additionally, due limited OCR-focused data our training set, the model may underperform on text-heavy tasks requiring fine-gra ine-grained text recognition.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
30

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide detailed implementation settings, including model architectures, training stages, learning rates, loss formulations (Eq.1, Eq.2), optimization strategies (LoRA and full-parameter tuning), and hyperparameters (e.g., temperature (3, loss weight 7). I also clearly state the number and type of GPUs used. The evaluation protocol is well- specified, listing all benchmarks and their assessed capabilities. These details are sufficient reproduce the main experimental results and validate the core claims, even if code and data are not directly released.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
31

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We have provided open GitHub code and sufficient reproduction instructions.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
public/guide SoodeSubmissionoool...
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips.coom wwwpppppccggppiiieeeeeeeeoooeele httpgguddd//CodSuumiii
CCodeSubmii ss ionPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We specify the training procedure, learning rates, optimizer settings (LoRA and full fine-tuning), key hyperparameters (8, 7), and hardware setup. Evaluation is conducted standard benchmarks.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
32

---

Answer: [Yes]
Justification: We report key results with sufficient experimental comparisons to support our main claims. While we do provide detailed error bars in all tables or figures, the results are consistent across benchmarks and training setups, and variability was assessed during model development ensure robustness.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (c.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We specify the number and type of GPUs used (8 GPUs with 90GB memory each).
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: //tpar/////uips/// http:p///nerrpp..cppppliilccc del ines?
Answer: [Yes]
Justification: All authors have reviewed and confirmed that the research conducted in the paper conforms, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
33

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our work improves the reliability of multi-modal large language models in multi-image reasoning tasks by reducing hallucinations and enhancing visual grounding. This has positive societal impact in applications requiring accurate visual understanding. such as education, medical imaging, and scientific analysis. However, more powerful image understanding capabilities may also raise risks related to surveillance, privacy invasion, or misuse in generating misleading content. We leave a deeper discussion these impacts to future work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation. generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our work focuses on reducing hallucinations and improving grounding in
multi-image understanding tasks, which poses relatively low risk of misuse. As such, do not introduce high-risk components such as large-scale generative models or scraped dataset.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
34

---

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All benchmark datasets and compared methods are properly cited with appro- priate references, and their licenses and use terms respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcoee. com datasete has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce new dataset, MultiScope-42k, which is automatically generated and used to support our two-level preference optimization framework. The dataset construc- tion process is thoroughly documented in the paper, including how chosen and rejected pairs are created at both the context and needle levels. We provide clear descriptions of the generation pipeline, perturbation strategies, and supervision signals, ensuring transparency and reproducibility.
Guidelines:
The answer NA means that the paper does release assets. Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper relies solely on existing publicly available benchmark datasets for both training and evaluation. It does not involve any crowdsoureing, user studies, or research with human subjects.
35

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects were directly involved in this research as we used existing datasets.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMS if is an important, original, or non-standard component of the core methods in this research? Note that if the L.M is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method in this research builds on existing multi-modal large language models (e.g., Qwen2-VI., L.LaVA-VV backbones but does not introduce L.LM architectures or non-standard modifications. Our contribution lies in the training framework themselves.
(CcDPO) and dataset construction, not in the development adaptation of the LLMs
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: //neuriis.... cc/Conferencess//20 for what should or should not be described.
36


Reviews:
[{"review_id": "Kfn9GqX61i", "reviewer": "Reviewer_3cFy", "summary": "This paper introduces a method for mitigating bias and hallucination in multi-image question answering (QA) with multimodal LLMs (MLLMs). The authors start by identifying the potential root causes of these errors: 1) context omission, 2) context conflation, and 3) misinterpretation. The authors constructed a preference optimization dataset to guide the MLLMs to avoid these issues. The authors also introduce the needle level preference optimization, which adds visual prompts to further guide the model towards preventing errors.  \n\nThe proposed method is applied to open-sourced foundation LLMs and tested on multi-image QA benchmarks. Improvement in response quality is observed. The authors also provided ablation studies to justify the effectiveness of the proposed method over SFT and previous baselines.", "questions": "Using DPO to guide the model towards better spatial and sequential understanding seems to be an interesting direction. I am wondering in terms of spatial reasoning, whether the format of the added visual prompt has an impact on the quality. In the current text it is roughly described as \"bounding boxes and key points\".  Can the authors give more details on how the visual prompts are injected and what are the impact of different types of visual prompts?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "+ The problem of multi-image MLLMs is both important and interesting. It is great that the authors identify existing methods' problems with detailed analysis and empirical results. The analysis is technically sound to me. \n+ Using multi-image captioning as a proxy to gauge the context understanding of multi-image MLLMs is an interesting take. \n+ The proposed method directly addresses the issues identified with the existing methods. It uses light-weight DPO approaches so practitioners can easily adopt it. \n+ Improvements in benchmark results are observed, and clear ablation studies are presented.\n\nWeaknesses\n- Details of how to construct visual prompts are missing.", "was_revised": true, "final_justification": "The authors have addressed my remaining questions in the feedback. I am keeping my recommendation to accept.", "rebuttal": "> **W1, Q1(1)**: More details on how to inject visual prompt.\n\nApologies for any confusion caused. To clarify, the visual prompt construction process involves three key steps:\n- **Data Collection:** We start by gathering source data with point or bounding box annotations. Points are defined as [x, y], where x and y represent the coordinates of the point. Bounding boxes are defined as [x1, y1, x2, y2], where [x1, y1] and [x2, y2] correspond to the top-left and bottom-right corner coordinates, respectively.\n- **Prompt Injection:** Based on the collected text-based annotation coordinates, we convert them into visual prompts **(such as red points and or green rectangles for bounding boxes)** and overlay them onto the original image. This creates clear markings that guide the model's attention.\n- **Emphasizing Important Regions**: To further highlight the marked areas, we **add textual labels (e.g., “REF”) near the visual prompts**. Additionally, we incorporate corresponding textual prompts to emphasize the marked regions, such as: “Please provide a detailed description of the marked areas in each image. The marked areas are indicated by a green rectangle with a 'REF' label around them.\n\nMore examples of visual prompts can be found in **Figures 1 and 7 of the supplementary material.**\n\n> **Q1(2):**: What are the effects of different types of visual cues on spatial reasoning?\n\nThank you for this insightful question. To investigate how different types of visual prompts affect spatial reasoning, we conducted additional ablation studies using two additional visual prompt formats: **circle** and **segmentation mask**. The **circle** prompt is generated as an inscribed ellipse within the bounding box (bbox), while the **segmentation mask** is derived directly from annotations in the MDVP dataset. Below are the detailed results:\n\n| bbox | segmentation | point | circle |Mantis | BLINK | Q-Bench2 | Average |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:-----:|\n| | | | | 60.4 | 51.1 | 73.8 |61.8  |\n| ✔ | | ✔ | | 70.5 | 55.9 | 76.8 |67.7  |\n| ✔ | | | ✔ | 70.0 | 54.5 | **77.2** | 67.2  |\n| | ✔ | ✔ | | **71.0** | **56.8** | 76.3 | **68.0**  |\n| | ✔ | | ✔ | 69.1 | 55.1 | 76.0 | 66.7  |\n| | | ✔ | ✔ | 67.7 | 56.2 | 76.6 | 66.8  |\n\nFrom the results, it is evident that different visual prompts have a relatively minor impact overall, with performance variations mostly within 1 point. This highlights the robustness of our method, as it adapts well to various visual prompt formats. However, certain trends are noticeable across specific benchmarks:\n\n- **Fine-grained prompts** such as **segmentation masks** and **key points** are particularly beneficial for benchmarks focused on spatial reasoning, such as **Mantis** and **BLINK**. These prompts encourage models to focus on finer-grained areas, enabling more precise spatial descriptions.\n\n- **Coarse-grained prompts**, like **bounding boxes (bbox)** and **circles**, perform better in benchmarks that emphasize broader image-level comparisons, such as **Q-Bench2**. These prompts provide sufficient spatial guidance without overloading the model with excessive detail.\n\nBased on these findings, we chose the combination of **bounding boxes** and **key points** as the our final visual prompt format due to the following advantages:\n- **Balanced Spatial Representation**: This combination effectively captures global spatial relationships while incorporating local feature details, achieving the best overall spatial reasoning performance across benchmarks.\n- **Simplicity and Efficiency**: Bounding boxes and key points annotations are more simple to obtain, making data preparation significantly more cost-effective compared to segmentation masks, which may not always be available."}, {"review_id": "VCPc6Rwd4b", "reviewer": "Reviewer_ut67", "summary": "This paper introduces CcDPO (Context-to-Cue Direct Preference Optimization) as a hierarchical preference optimization framework for improving multi-image understanding problems of Multimodal Large Language Models (MLLMs). The authors identify three prevalent types of hallucinations—context omission, context conflation, and detail misinterpretation—and propose a two-level optimization strategy to address them: (i) Context-Level Optimization improves holistic sequence comprehension via structured per-image captions and perturbation-based contrastive training; (ii) Needle-Level Optimization enhances fine-grained perception using region-level visual prompts and vision-language contrastive objectives. The proposed method is trained on a newly constructed dataset, MultiScope-42k, and achieves improvements across multiple multi-image benchmarks (e.g., MUIRBench, MIBench, MIRB). \n\nHowever, the authors do not include any ablation or comparative experiments where CcDPO is trained using existing public datasets (e.g., LLaVA-23K, MDVP, MVC) to isolate the contribution of the optimization method itself versus the data quality and scale. Also, I do not find any supplementary materials or documentation that would help reproduce the reported results using the released code. The provided README appears to be identical to that of LLaVA-NeXT: Open Large Multimodal Models, without any specific instructions for training or evaluating CcDPO.", "questions": "1. Could the authors clarify whether the released codebase includes any implementation of CcDPO beyond what is already available in LLaVA-NeXT? A readme of CcDPO is preferred in response.\n\n2. Could the authors provide an ablation study where CcDPO is trained without MultiScope-42k, for example using only existing datasets such as LLaVA-23K or MDVP? Also, the baselines trained with MultiScope-42k is also necessary for verification. \n\n3. The two optimization components—context-level and needle-level—appear to be trained independently and applied in parallel. Could the authors clarify whether there is any interaction or dependency between them during training or inference?\n\nThe major limitations are stated in the weakness part. \nI encourage the authors to respond the ablation study of MultiScope-42k, as well as the readme of codes for reproduction.", "limitations": "Yes", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 1, "originality": 1, "strengths_and_weaknesses": "Pros:\n1. The paper addresses a critical challenge in multi-image understanding. The authors clearly articulate the importance of this issue and provide a thoughtful taxonomy of error types, which helps ground the methodological design.\n\n2. The authors present a carefully constructed dataset (MultiScope-42k) tailored for hierarchical preference optimization, which effectively supports both context-level and fine-grained training objectives.\n\n3. The proposed two-level optimization framework (CcDPO) is methodologically sound and conceptually intuitive. \n\nCons: \n1. The released code lacks any specific README or documentation related to CcDPO. It appears to be identical to the LLaVA-NeXT repository, with no visible modifications or guidance for reproducing the results. This makes the authors’ claim of code release less convincing and significantly limits reproducibility.\n\n2. There is no ablation study evaluating CcDPO without the proposed MultiScope-42k dataset. As a result, it is unclear whether the performance gains stem from the optimization method itself or simply from the quality and scale of the new dataset. This is unfair for the baseline models that cannot access the MultiScope-42k datasets. \n\n3. The two proposed optimization stages—context-level and needle-level—are designed independently and operate in parallel. There is no clear logical or learning dependency between them, which weakens the hierarchical narrative implied in the method’s name and motivation.", "was_revised": true, "final_justification": "I recommend **rejecting this paper**. From the initial submission through the rebuttal, the authors have consistently attempted to blur the line between the performance gains contributed by their proposed algorithm (CcDPO) and those arising from the introduction of a new dataset (MultiScope-42k).\n\nFigure 1 is a particularly **misleading visualization**. At first glance, it appears to suggest that CcDPO yields significant performance improvements. However, upon closer examination of the provided tables, it becomes clear that the majority of the gains are attributable to the dataset rather than the algorithm itself.\n\nMoreover, the authors did not provide a proper README for their code, further hindering reproducibility and transparency. Taken together, these issues point to a systematic attempt to obscure the true source of improvements, which raises concerns about the credibility of the empirical findings.\n\nLastly, **rejecting this paper is necessary to uphold the value of genuine algorithmic innovation**. If the community starts to accept papers that primarily introduce new datasets but mask them as algorithmic contributions, it sets a dangerous precedent. It risks favoring researchers with the resources to construct larger datasets, while **penalizing those focused on advancing algorithm design — even if their improvements are more principled but less dramatic in performance**. This trend would ultimately undermine progress in core methodological research.", "rebuttal": "> ### **W1,Q1(1)**: Does the released code include any new CcDPO implementations not already available in LLaVA-NeXT?\n\nIndeed, we made the following modifications:\n1. **Data Processing**: Added `llava/train/train_vdpo.py` and `llava/train/train_svco_triples.py` to process inputs containing both text- and image-level positive and negative samples.\n2. **Model Training**: Added `trl/trainer/vdpo_trainer.py` and `trl/trainer/vdpo_trainer_svco_triples.py` to implement the Hybrid Visual-Language Optimization formulas (1), (2), and (3) as described in our paper.\n\n> ### **W1,Q1(2)**: The provided code lacks a README for CcDPO.\n \nWe apologize for the initial lack of documentation due to the submission deadline. The code is accurate, and we have now provided a detailed guide below to help reproduce results.\n#### **Data**\n- Training Data: In this work, we use the constructed MultiScope Dataset with 42k training samples as our training dataset.​\n- Evaluation Data: The evaluation data can be downloaded from project homepage. The evaluation datasets utilized in our work are listed below: MuirBench; MIRB; BLINK; Mantis; NLVR2; MIBench; Q-Bench2\n\n#### **Install**\n\n1. Clone this repository and navigate to the source folder:\n> cd LLaVA-NeXT\n2. Build Environment (key package versions: transformers=4.45.2，accelerate=1.4.0, deepspeed=0.15.4，peft=0.14.0)\n> pip install -e .\n\n#### **LLaVA-OV CcDPO Training**\nConfigure the training dataset ”DATA_PATH“ and image folder \"image_folder\" and the checkpoint name “OUTPUT_DIR“.\n- Stage1: set ”DATA_PATH“ to Context-Level DPO dataset.\n> bash scripts/train/vdpo_ov7b.sh\n- Stage2: set ”DATA_PATH“ to Needle-Level Language-based DPO dataset.\n> bash scripts/train/vdpo_ov7b.sh\n- Stage3: set ”DATA_PATH“ to Needle-Level Vision-based DPO dataset.\n> bash scripts/train/svco_ov7b.sh\n\n#### **Evaluation**\nThe trained models are evaluated on MuirBench, BLINK and MIBench benchmarks using VLMEvalKit tools.\n> cd VLMEvalKit\n> \n> CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 torchrun --nproc-per-node=8 --master_port 3951 run.py --data BLINK --model llava_onevision_qwen2_7b_ov --verbose --work-dir xxx\n\nThe trained models are evaluated on Mantis, NLVR2, and Q-Bench2 benchmarks using Mantis's code.\n\n> cd Mantis-main/mantis/benchmark\n> \n> CUDA_VISIBLE_DEVICES=0 torchrun --nproc-per-node=1 --master_port 1921 eval.py --dataset_path \"mantis_eval\" --dataset_name \"mantis_eval\" --model_name llava-ov --results_dir xxxx --overwrite True\n\n---\n\n> ### **W2,Q2**: There is no ablation study to disentangle the performance gains from the CcDPO method versus the new MultiScope-42k dataset.\n\nThank you for your insightful comments. To address your concern, we conducted three sets of experiments to verify **Method Effectiveness**, **Dataset Effectiveness**, and **Dataset Scale Evaluation**, with the following conclusions:\n- We trained CcDPO using existing public single-image or multi-image input DPO datasets, and **our CcDPO still achieved better performance compared to the baseline** SFT method. (Table 1)\n- We compared the effectiveness of DPO across different datasets and found that **single-image QA with multi-image inputs is not optimal** for addressing multi-image hallucinations—multi-image QA works better. In this scenario, **captioning is the lowest-cost method**. (Table 1)\n- When matched in size with the compared datasets, our MultiScope-42k still performs better, and **performance improves as the size increases.** (Table 2)\n   \nWe will now analyze each issue in detail:\n> ### **W2,Q2(1)**: Method Effectiveness：there is no ablation study evaluating CcDPO without the proposed MultiScope-42k dataset  (e.g., LLaVA-23K, MDVP).\n\nTo clarify, the suggested LLaVA-23k and MDVP datasets are primarily SFT datasets and are not directly applicable to DPO training. Therefore, we selected two existing DPO datasets to train CcDPO, including the multi-image input single-image QA dataset MIA-DPO, the **single-image input single-image QA** dataset POVID 17k[1] + RLHF-V 6k[2], and our **multi-image input multi-image QA** dataset MultiScope. We compared CcDPO with the baseline SFT method, as shown in the table below:\n\n| Description of Data | Method|Training Dataset| MuirBench | Mantis | BLINK | Q-Bench2 |\n|-|-|-|--|--|--|-|\n|    | LLaVA-OV | -   | 42.5  | 60.4   | 51.1  | 73.8     |\n|**MIA-DPO-28k**: multi-images input, single image QA| LLaVA-OV + SFT  | MIA-DPO-28k | 42.6  | 59.0   | 52.3  | 74.0     |\n| | LLaVA-OV + CcDPO | MIA-DPO-28k | **42.9**  | **62.7**| **53.2** | **74.8** |\n| **POVID-17k** + **RLHF-V-6k**: single-image input, single image QA | LLaVA-OV + SFT     | POVID-17k + RLHF-V-6k         | 43.3      | 59.4   | 51.7  | 73.5     |\n|    | LLaVA-OV + CcDPO| POVID-17k + RLHF-V-6k | **44.7**  | **61.8**| **54.1** | **75.6** |\n| **MultiScope**: multi-images input, multi-image QA | LLaVA-OV + SFT| MultiScope-42k| 45.4      | 64.9   | 53.4  | 75.7     |\n| | LLaVA-OV + CcDPO        | MultiScope-42k  | **48.6**  | **69.6**| **55.9** | **77.6** |\n\nAcross all datasets, CcDPO consistently outperforms the baseline method, showcasing its ability to leverage contrastive information from preference data. Its effectiveness in multi-image tasks, however, is limited with MIA-DPO-28k or POVID-17k + RLHF-V-6k—**likely because these datasets lack tasks explicitly involving multiple images in final answers.** In contrast, the low-cost, automatically constructed MultiScope dataset includes multi-image tasks that better utilize CcDPO’s capabilities, yielding its best performance.\n\n[1] RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback\n\n[2] Aligning modalities in vision large language models via preference fine-tuning.\n> ### **W2,Q2(2)**: Dataset Effectiveness: Do performance gains stem solely from the new dataset’s quality?\n\n- To clarify, **performance gains stem from both the data and the CcDPO method**. As the ablove table shows, while MultiScope outperforms other public datasets for multi-image tasks, integrating our CcDPO method yields a maximum average performance **improvement of ~3.1%**. This is mainly attributed to our core idea—**using multi-image descriptions as an alternative approach to understanding context**.\nOur dual-level optimization forces the model to describe each image accurately from coarse to fine, cutting multi-image hallucinations by up to **26%** (Table 2 of main paper) and boosting its ability to grasp multi-image context.\n- In contrast, CcDPO cannot fully exert its effect with other datasets, as their QA tasks only require leveraging local information within image sequences.\n\n> ### **W2,Q2(3)**:  Do performance gains stem solely from the new dataset’s scale?\n\nWe analyze the impact of training size by performing an ablation study using the same total number of preference pairs as MIA-DPO **(more data filtering details and analysis are in Supplementary Section C)**. The results are shown below.  \n\n| Dataset |Size|MuirBench|MIRB| BLINK | Mantis | NLVR2 | MIBench | Q-Bench2 | Average |\n|--|-|--|--|--|---|--|--|--|--|\n| LLaVA-OV| - | 42.5| 47.3  | 51.1  | 60.4   | 89.4  | 73.6 | 73.8   | 62.5    |\n| + MIA-DPO | 28.9K  |41.4| 48.0  | 53.7  | 60.3   | 88.2  | 67.8    | 74.0     | 61.9    |\n| + CcDPO | 28.1K | 46.7 | 51.2| **56.5** | 69.1 | **90.7** | 72.1 | **79.3** | 66.5 |\n| + CcDPO | 41.8K| **48.6** | **51.4** | 55.9 | **69.6** | **91.2** | **75.2** | 77.6 | **67.1** |\n| Δ   | - | +6.1 | +4.1  | +4.8  | +9.2   | +1.8  | +1.6    | +3.8     | +4.6    |\n\nResults show that, at the matched training size, our CcDPO consistently outperforms MIA-DPO across benchmarks, highlighting the effectiveness of its structured dual-level supervision. Training with the full preference set delivers the best average performance, demonstrating the benefits of high-quality, large-scale supervision. This also reveals a trade-off between contextual alignment and perceptual precision.\n\n---\n\n> ### **W3,Q3**: The two proposed optimization stages—context-level and needle-level—are designed independently and operate in parallel.\n\nThank you for your thoughtful comments. We would like to clarify that the two optimization stages—context-level and needle-level DPO—are **sequentially designed** and **mutually dependent**, reflecting a coarse-to-fine cognitive process. Specifically:\n- **Sequential Design:** The model trained with context-level DPO serves as **the initialization** for needle-level DPO training, establishing a **sequential workflow.** \n  - In the context-level stage, the model learns to integrate global information, addressing foundational errors like **context omission or conflation.** \n  - Following this, needle-level DPO focuses on **refining detail misinterpretation** by enhancing perceptual precision for critical regions. This workflow mirrors human general reasoning for multi-image problems: one first understands the global context, then zooms in to focus on specific details as needed for the task.\n- **Stage Dependency:** To explore interactions and dependencies between context-level and needle-level DPO, we conducted ablation experiments with varying training stage orders and mixed training approaches. Results are provided below：\n\n| Configuration | MuirBench | Mantis | BLINK | Q-Bench2 |\n| :- | :-: | :-: | :-: | :-: |\n| Baseline | 42.5 | 60.4 | 51.1 | 73.8 |\n| One-Stage Training | 46.2 | 65.4 | 55.1 | - |\n| Context-Level Only | 44.4 | 69.0 | 55.3 | 76.4 |\n| Needle-Level Only | 46.3 | 65.0 | 54.3 | 75.5 |\n| Needle-Level -> Context-Level | 46.5 | 66.3 | 54.1 | **77.8** |\n| Context-Level -> Needle-Level | **48.6** | **69.6** | **55.9** | 77.6 |\n\nResults demonstrate that training context-level DPO before needle-level DPO consistently yields optimal performance across benchmarks, emphasizing a hierarchical and interdependent relationship. Skills learned in the context-level stage (e.g., coherent global reasoning) establish a strong foundation for the needle-level stage to achieve fine-grained perceptual precision."}, {"review_id": "r6hSiKx95k", "reviewer": "Reviewer_wac8", "summary": "This paper proposes Context-to-Cue Direct Preference Optimization (CcDPO), a hierarchical framework to improve multi-image understanding in Multi-modal Large Language Models (MLLMs). The method tackles three common hallucination issues - context omission, context conflation, and detail misinterpretation - by introducing two levels of preference alignment: (1) Context-Level Optimization, which promotes accurate and complete multi-image comprehension through structured per-image captions and perturbation-based training; and (2) Needle-Level Optimization, which enhances fine-grained perception using region-level visual prompts and contrastive supervision. To support training, the authors construct MultiScope-42k, a large-scale dataset with automatically generated multi-level preference pairs. Experiments on a wide range of multi-image and single-image benchmarks demonstrate that CcDPO consistently improves performance and reduces hallucination compared to prior methods like MIA-DPO.", "questions": "The paper presents an impressive contribution with the CcDPO framework, supported by a robust dataset (MultiScope-42k) and thorough evaluations. My only minor concern is in Table 1, where the bolded result for Qwen2-VL on the MIRB task does not correspond to the highest score, which may confuse readers. Please clarify if this is intentional or correct the bolding, and consider verifying other tables for consistency. This small revision would further polish an already strong submission.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe paper presents a well-motivated and clearly described approach to improving multi-image understanding in MLLMs, addressing common hallucination issues. The proposed CcDPO framework is conceptually sound, combining structured context-level and fine-grained needle-level preference optimization. The methodology is supported by a newly constructed dataset, MultiScope-42k, which enables scalable training. The empirical results are thorough, covering a wide range of benchmarks, and show consistent improvements over strong baselines. The writing is clear and the paper is easy to follow.\n\nWeaknesses:\nThe overall design is solid, but the method mainly combines existing techniques such as DPO and visual prompting in a hierarchical way, rather than introducing fundamentally new mechanisms. In addition, as the authors acknowledge, the framework does not explicitly handle temporal reasoning, which may limit its applicability to video-related tasks.", "was_revised": true, "final_justification": "The authors have addressed all of my comments. In addition to correcting the formatting error in Table 1, they provided a detailed comparison between CcDPO and prior approaches, clarified its practical advantages, and extended their experiments to the video domain, demonstrating generalizability. These updates reinforce the paper’s originality and impact. I maintain my rating of 5.", "rebuttal": "> ### **W1(1)**: The framework's novelty lies in its hierarchical application and combination of existing mechanisms, which results in somewhat limited contributions.\n\n1. Firstly, let us outline the differences between our proposed approach, CcDPO, and MIA-DPO:\n- One key difference between CcDPO and MIA-DPO lies in the design of **question prompts**. \n  - **MIA-DPO solely associate each question with a specific image, which causes MIA-DPO to only consider a subset**  (instead of the entire set) of images to infer the final answer. This design limits the model's effectiveness when handling questions that require information from all images to arrive at the correct answer.\n  - CcDPO uses multi-image captioning prompts (two-stage: coarse-grained context → fine-grained \"needle\" analysis) to enhance **full-image context understanding** with minimal construction cost (Table 1). \n  - In addition, visual prompts in the needle stage **force the model to rely on the marked information within the images**, enhancing its ability to interpret fine details accurately while reducing the likelihood of hallucinations.\n\n- Another key difference between CcDPO and MIA-DPO is the **construction cost** of preference pairs. \n  - MIA-DPO requires pre-filtering rejected responses based on base model predictions, which introduces computational overhead and noise due to potential discrepancies between attention values and final answers. \n  - CcDPO leverages **structured captioning for efficient negative sample generation (via swapping/truncating captions)**, simulating key multi-image issues (Context Omission, Conflation, Detail Misinterpretation) at low cost.\n2. Next, let us outline the differences between our CcDPO and prior vision-contrastive work  [49, 32]:\n  - Prior methods align global image semantics but **miss critical instance-level details**; CcDPO uses visual prompts to **anchor regions/objects**, boosting fine-grained tasks (Counting, Attribute Understanding).\n  - While prior vision-contrastive learning methods primarily focus on single-image tasks, our work first extends this concept into the **multi-image understanding domain**.   \n  \n3. Finally, let's summarize the contributions of the paper:\n- We created a caption generation task to evaluate MLLMs' multi-image understanding, **identifying a key limitation: the inability to integrate partial context** (Figure 2 and Table 2 of main paper), which hinders complex reasoning. We also pinpoint three hallucination types: **Context Omission, Context Conflation, and Detail Misinterpretation.**\n- To address this, we propose a cost-effective two-level preference optimization framework that improves multi-image perception by leveraging sequential context and fine-grained details, **achieving up to a +4.6 point performance boost** (Table 1 in our main paper).  \n- We also introduce MultiScope-42k, a large-scale, auto-generated DPO dataset that **structures image annotations into sequences** with rich insights. This reduces data collection costs and, using a coarse-to-fine pairing strategy, delivers better generalization than SFT on multi-image tasks.  (refer to the responses of Reviewer ut67.)\n\n> ### **W1(2)**: The applicability of the proposed framework to video-related tasks.\n\nThank you for your valuable feedback. As acknowledged in the limitations section, we did not specifically construct preference pairs for video data. However, our approach can be easily extended to video data using a structured format, such as **<video clip 1> <caption 1> <video clip 2> <caption 2>**, while incorporating the proposed Sequence Truncation and Content Swapping strategies for generating negative samples. This allows video DPO datasets to be created at minimal cost.\n\nTo further support this, we constructed a 7k video dataset based on the described methodology and performed DPO training. The table below illustrates the effectiveness of our framework in VideoMME tasks:\n\n| Model         | Overall | Temporal Perception | Spatial Perception | Attribute Perception | Action Recognition | Object Recognition | OCR Problems | Counting Problem | Temporal Reasoning | Spatial Reasoning | Action Reasoning | Object Reasoning | Information Synopsis |\n|:-------------:|:-------:|:-------------------:|:------------------:|:--------------------:|:------------------:|:------------------:|:------------:|:----------------:|:------------------:|:-----------------:|:----------------:|:----------------:|:---------------------:|\n| LlaVA-OV      | 53.7    | 0.491               | 0.556              | 0.685               | 0.505              | 0.559              | **0.568**        | 0.354            | 0.384              | 0.714             | **0.523**            | **0.518**            | **0.675**                  |\n| +CcDPO-Video          | **54.4**    | **0.509**               | **0.574**              | **0.698**               | **0.527**              | **0.582**              | 0.547        | **0.377**            | **0.401**              | **0.732**             | 0.519            | 0.511            | 0.663                  |\n\nIt is worth noting that even an improvement of 1 point in video understanding tasks is significant—for instance, prior video DPO work [1,2] achieves improvements in the range of 0.4–1.2 points. While our CcDPO method is not specifically tailored for video understanding, **its use of sequence descriptions as a proxy shows potential in enhancing sequence comprehension.** Moreover, the proposed negative sample **perturbation strategies are generalizable**, making the construction process more cost-effective by reducing reliance on model re-labeling or rejection sampling strategies.\n\n---\nReferences\n\n[1] VideoPASTA : 7K Preference Pairs That Matter for Video-LLM Alignment\n\n[2] Direct preference optimization of video large multimodal models from language model reward.\n\n> ### **Q1**: Bold errors of Table 1.\n\nApologies for my mistake. We have corrected this error and have also verified the other tables to ensure consistency."}, {"review_id": "cRCm3mSfsg", "reviewer": "Reviewer_iyTW", "summary": "This paper aims to address common hallucinations in MLLMs when processing multi-image inputs, specifically context omission, conflation, and detail misinterpretation. To this end, the authors propose a hierarchical preference optimization framework named CCDPO. This framework is divided into two levels: the Context-Level, which optimizes the model's understanding of the global sequence through perturbation strategies like sequence truncation and content swapping , and the Needle-Level, which enhances the model's perception of local, key details through a combination of visual prompts and vision-contrastive learning. Additionally, the authors constructed an automatically generated preference dataset, MultiScope-42k, to support the training of this framework. Experimental results indicate that this method achieves performance gains across multiple multi-image and single-image benchmarks.", "questions": "See weaknesses.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper provides a valuable and clear categorization of hallucinations in multi-image understanding, identifying them as context omission, context conflation, and detail misinterpretation. This classification offers distinct targets and a focused direction for the methodological design that follows.\n2. The model is evaluated extensively on a wide range of multi-image and single-image benchmarks, with performance compared against relevant baselines like SFT and MIA-DPO. Additionally, the paper includes detailed ablation studies to validate the effectiveness of the framework's different components.\n\nWeaknesses:\n1. The paper's core contribution lies in its novel combination of existing techniques rather than a fundamental methodological breakthrough. DPO and its applications in multimodal contexts are previously established, with the paper itself citing MIA-DPO as a baseline for multi-image tasks. The vision-contrastive learning at the needle-level also explicitly draws from prior work. Therefore, the work presents itself more as a sophisticated engineering and integration effort, with its methodological novelty being somewhat limited.\n\n\n2. The ablation study does not analyze the individual effects of the Needle-Level and Context-Level optimizations. A more direct comparison could have provided better insight into the contribution of each method to downstream task performance.\n\n3. In the ablation study tables, shorthand terms like Needle-Levi-TDPO and Needle-Levi-VDPO are used without being clearly defined in the main text of the paper. This lack of explicit definition can cause confusion for the reader.", "was_revised": true, "final_justification": "The authors have solved most of my concerns.", "rebuttal": "> ### **W1**: The framework's novelty lies in its hierarchical application and combination of existing mechanisms, rather than in the creation of new fundamental components.\n\n1. Firstly, let us outline the differences between our proposed approach, CcDPO, and MIA-DPO:\n- One key difference between CcDPO and MIA-DPO lies in the design of **question prompts**. \n  - **MIA-DPO solely associate each question with a specific image, which causes MIA-DPO to only consider a subset**  (instead of the entire set) of images to infer the final answer. This design limits the model's effectiveness when handling questions that require information from all images to arrive at the correct answer.\n  - CcDPO uses multi-image captioning prompts (two-stage: coarse-grained context → fine-grained \"needle\" analysis) to enhance **full-image context understanding** with minimal construction cost (Table 1). \n  - In addition, visual prompts in the needle stage **force the model to rely on the marked information within the images**, enhancing its ability to interpret fine details accurately while reducing the likelihood of hallucinations.\n\n- Another key difference between CcDPO and MIA-DPO is the **construction cost** of preference pairs. \n  - MIA-DPO requires pre-filtering rejected responses based on base model predictions, which introduces computational overhead and noise due to potential discrepancies between attention values and final answers. \n  - CcDPO leverages **structured captioning for efficient negative sample generation (via swapping/truncating captions)**, simulating key multi-image issues (Context Omission, Conflation, Detail Misinterpretation) at low cost.\n2. Next, let us outline the differences between our CcDPO and prior vision-contrastive work  [49, 32]:\n  - Prior methods align global image semantics but **miss critical instance-level details**; CcDPO uses visual prompts to **anchor regions/objects**, boosting fine-grained tasks (Counting, Attribute Understanding).\n  - While prior vision-contrastive learning methods primarily focus on single-image tasks, our work first extends this concept into the **multi-image understanding domain**.   \n  \n3. Finally, let's summarize the contributions of the paper:\n- We created a caption generation task to evaluate MLLMs' multi-image understanding, **identifying a key limitation: the inability to integrate partial context** (Figure 2 and Table 2 of main paper), which hinders complex reasoning. We also pinpoint three hallucination types: **Context Omission, Context Conflation, and Detail Misinterpretation.**\n- To address this, we propose a cost-effective two-level preference optimization framework that improves multi-image perception by leveraging sequential context and fine-grained details, **achieving up to a +4.6 point performance boost** (Table 1 in our main paper).  \n- We also introduce MultiScope-42k, a large-scale, auto-generated DPO dataset that **structures image annotations into sequences** with rich insights. This reduces data collection costs and, using a coarse-to-fine pairing strategy, delivers better generalization than SFT on multi-image tasks.  (refer to the responses of Reviewer ut67.)\n  \n> ### **W2**: The ablation study does not analyze the individual effects of the Needle-Level and Context-Level optimizations.\n\nThank you for your valuable feedback. To address your concerns, we conducted additional analyses to evaluate the individual effects of Needle-Level and Context-Level optimizations. \n- The benchmark results below demonstrate that both optimizations independently improve performance, with Needle-Level excelling at fine-grained detail-oriented tasks (e.g., **MuirBench +3.8%**), and Context-Level performing better in holistic context-reasoning tasks (e.g., **Mantis +6.6%, Q-Bench2 +2.6%**). When combined, they achieve the best overall results, significantly outperforming the baseline (e.g., **MuirBench +6.1%, Mantis +9.2%, BLINK +4.8%, Q-Bench2 +3.8%**). \n  \n| Context-Level | Needle-Level | **MuirBench** | **Mantis** | **BLINK** | **Q-Bench2** |\n|---------------|--------------|----------------|------------|-----------|--------------|\n| ×| ×| 42.5 | 60.4 | 51.1 | 73.8 |\n| ✔| × | 44.4 | 67.0 | 54.8 | 76.4 |\n| ×| ✔ | 46.3 | 65.0  | 54.3| 75.5 |\n| ✔| ✔ | **48.6** | **69.6** | **55.9** | **77.6** |\n- The table below provides additional insights into the effects on MuirBench, showing that Needle-Level optimization improves tasks requiring fine-grained perception, such as **Attribute Understanding (+14.4%) and Counting (+6.9%)**, while Context-Level optimization enhances tasks involving contextual reasoning, such as **Diagram Understanding (+2.5%) and Difference Spotting (+9.1%)**. These results highlight that the two optimizations are complementary, and their integration effectively resolves challenges in multi-image detail recognition and context reasoning, achieving state-of-the-art performance.\n\n| Model | Overall | Action Understanding | Attribute Understanding| Cartoon Understanding | Counting | Diagram Understanding | Difference Spotting | Geographic Understanding | Image-Text Matching | Ordering | Scene Understanding | Visual Grounding | Visual Retrieval |\n| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| LlaVA-OV | 42.5 | 35.9 | 33.1 | **35.8** | 24.7 | 55.0 | 30.0 | **46.0** | 46.9 | **20.3** | 72.0 | 29.7 | 47.6 |\n| Needle-Level | 46.3 | **39.0** | **47.5** | 32.1 | **31.6** | 54.8 | 35.0 | 43.0 | **55.0** | 17.2 | **74.7** | **33.3** | **47.8** |\n| Context-Level | 44.4 | 37.8 | 39.8 | 33.3 | 29.5 | **57.5** | **39.1** | 39.0 | 50.4 | 14.1 | 64.0 | 27.4 | 44.2 |\n\n> ### **W3**: The shorthand terms like Needle-Levi-TDPO and Needle-Levi-VDPO are used without being clearly defined in the main text of the paper. \n\nApologies for any confusion caused.\n\n**Needle-Level Language-based Preference Optimization (Needle-Levl-TDPO)** focuses on refining the model's preferences based on textual information. It uses paired textual descriptions ($y_{chosen}, y_{rejected}$) to guide the model's attention to specific regions within an image ($x$). The optimization process ensures that the model learns to associate descriptions with the highlighted regions, favoring preferred regions as follows:  \n$$ (x, y_{\\text{chosen}}) \\succ (x, y_{\\text{rejected}}) $$  \n\n**Needle-Level Vision Contrastive Preference Optimization (Needle-Levl-VDPO)** forces the model to make preference judgments based on visual information. It works by constructing pairs of images ($x_{chosen}, x_{rejected}$) with preferences controlled as variables, guiding the model to distinguish and prioritize chosen visual regions. This process is expressed as:  \n$$ (x_{\\text{chosen}}, y) \\succ (x_{\\text{rejected}}, y) $$  \n\nWe will ensure these terms are clearly and thoroughly defined in the main text of the revised version to improve accessibility and understanding for all readers, including non-technical audiences."}]

Author rebuttal:

Meta-review: This paper introduces Context-to-Cue Direct Preference Optimization (CcDPO), a hierarchical framework designed to reduce hallucinations in multi-image question answering. The method first operates at a "context level," using perturbation-based training to improve the model's overall understanding of the image sequence, preventing errors like context omission and conflation. It then applies a "needle level" optimization, which uses region-specific visual prompts to enhance the perception of fine-grained details and prevent misinterpretation. Trained on a newly constructed preference dataset, MultiScope-42k, the CcDPO method demonstrates consistent performance gains and reduces hallucinations across several multi-image benchmarks.

The paper received mixed reviews, with ratings of 2 and 5s. While reviewers generally acknowledged the novelty of the proposed CcDPO, the primary focus was on the reproducibility and significance of CuDPO. The authors actively engaged in the rebuttal phase, providing explanations for the provided source code and presenting additional numerical evidence to demonstrate the contribution of the CuDPO algorithm and the newly constructed MultiScope-42k dataset. Although reviewers may still have some reservations regarding the conclusions drawn from the data, the overall feedback remains positive. We recommend that the authors consider making further clarifications and incorporating the numerical data and discussions into the next version of the paper.
Decision: Accept (poster)