LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models
Authors: Ruilin Yao
Bo Zhang
Jirui Huang
Xinwei Long
Yifang Zhang
Tianyu Zou
Shili Xiong
Yi Rong
Yufei Wu
Shichao Su
Yifan Xu
Wenxi Zeng
Zhaoyu Yang
Guoyou Li
Shilan Zhang
Zichan Li
Yaxiong Chen
Shengwu Xiong
Peng Xu
Jiajun Zhang
Bowen Zhou
David A. Clifton
Luc Van Gool
Venue: ICLR 2026 Poster (2026)
Keywords: MLLMs
Multimodal reasoning
Synergistic effects
Primary area: datasets and benchmarks

Abstract: Multimodal Large Language Models (MLLMs) have achieved significant advances in integrating visual and linguistic information, yet their ability to reason about complex and real-world scenarios remains limited.
Existing benchmarks are usually constructed in a task-oriented manner, without a guarantee that different task samples come from the same data distribution. Therefore, they often fall short in evaluating the synergistic effects of lower-level perceptual capabilities on higher-order reasoning. To lift this limitation, we contribute Lens, a multi-level evaluation benchmark of multimodal reasoning with with 3.4K contemporary images and 60K+ human-authored questions covering eight tasks and 12 daily scenarios, forming three progressive task tiers, i.e., perception, understanding, and reasoning. One feature is that each image is equipped with rich annotations for all tasks. Thus, this data set intrinsically supports evaluating MLLMs to handle image-invariable prompts, from basic perception to compositional reasoning. In addition, our images have been   collected manually from social media, with $53$% published after Jan. 2025. We evaluate 15+ frontier MLLMs such as  Qwen2.5-VL,  InternVL3,  GPT-4o  and two reasoning models  QVQ-Max and Kimi-VL. Most models were released in 2025, and none of them achieve an accuracy beyond $60$% in the reasoning tasks. Furthermore, we propose the Self-Driven Multi-Expert Collaborative Framework (SMEC), a framework designed for MLLMs that simulates a panel of experts discussing and exchanging viewpoints via self-generated role-specific prompts. The experimental results confirm the existence of synergistic effects in a hierarchical task structure, where low-level tasks facilitate the reasoning of MLLMs on more complex, high-level tasks. Statistical analysis and ablation studies further demonstrate the comprehensiveness of our dataset and the superiority of our methodology. Project page:  https://github.com/Lens4MLLMs/lens. We conducted the ICCV 2025 MARS2 Multimodal Reasoning Challenge on Lens. https://mars2workshop.github.io/iccv2025/

Published as a conference paper ICLR 2026
LENS: MULTI-LEVEL EVALUATION OF MULTIMODAL REASONING WITH LARGE LANGUAGE MODELS
Ruilin Yao 1.3 Bo Zhang', Jirui Huang Xinwei XinweiLonn'', Yifang Zhang', Tianyu Zou', Shili Xiong', Yi Rong', Yufei Wu', Shichao Su', Yifan Xu Wenxi Zeng', Zhaoyu Yang', Guoyou Li", Shilan Zhang'. Zichan Li', Yaxiong Chen', Shengwu Xiong'; Peng Xu X Jiajum Zhang
Bowen Zhou 2.4 David A. Clifton", Luc Van Gool
Wuhan University of Technology 2 Tsinghua University
Institute of Automation, Chinese Academy of Sciences Shanghai Al Lab University of Oxford INSAIT, Sofia Un. St Kliment Ohridski x longsw@whut. edu. en, peng xuê s inghuaa edu.cn cn
ABSTRACT
Multimodal Large Language Models (MLLMs) have achieved significant advances in integrating visual and linguistic information, yet their ability to reason about complex and real-world scenarios remains limited. Existing benchmarks are usually constructed in task-oriented manner, without a guarantee that different task samples come from the same data distribution. Therefore, they often fall short in evaluating the synergistic effects of lower-level perceptual capabilities we contribute Lens, a multi-level evaluation benchmark of multimodal reasoning with 3.4K contemporary images and 60K+
higher-order reasoning. To lift this limitation,
questions covering eight tasks and 12 daily scenarios, forming three progressive task tiers, i.e., perception, understanding, and reasoning. One feature is that each image is equipped with rich annotations for all tasks. Thus, this data set intrinsically supports evaluating MILMs to handle image- invariable prompts, from basic perception compositional reasoning. In addition, our images have been collected manually from social media, with 53% published after Jan. 2025. We evaluate 15+ frontier MLLMs such as Qwen2.5-VL. InternVL3, GPT-4o and two reasoning models QVQ-Max and Kimi-VLe Most models were released in 2025, and none of them achieve an accuracy beyond 60% in the reasoning tasks. Furthermore, we propose the Self-Driven Multi-Expert Collaborative Framework (SMEC), framework designed for MLLMs that simulates a panel of experts discussing and exchanging viewpoints via self-generated role-specifie prompts. The experimental results confirm the existence of synergistic effects in hicrarchical task structure, where low-level tasks facilitate the reasoning of MILMMs on more complex, high-level tasks. Statistical analysis and ablation studies further demonstrate the comprehensiveness of our dataset and the superiority of our methodology. Project page: htttps://g www.:om//aaaeaa Lens 4MLLMs / /lens We conducted the ICCV 2025 MARS2 Multimodal Reasoning Challenge on Lens ht https: ///paa/ewwrceehoo....ttttti.e/ee
"Corresponding authors.

---

Published as a conference paper ICLR 2026
1 INTRODUCTION
Multimodal Large Language Models (MLLMs) have emerged as a rapidly advancing field in artificial intelligence, demonstrating substantial improvements in visual content recognition and multimodal reasoning (Zhu et al. 2025: Bai et al., 2025; Wu et al. 2024 Team et al. 2024 2025a). Despite their promising capabilities, MLLMs continue to face significant challenges interpreting complex and real-world visual environments that are inherently dynamic, diverse, and grounded in physicality. However, existing benchmarks remain limited in their ability to evaluate multi-level reasoning.
Early evaluations were largely based on classical computer vision tasks (Everingham et al. 2010 Lin et al.] 2014; Yu et al. 2016) and their integration with natural language. The real-world knowledge was often superficial, resulting in weak alignment between visual input and linguistic output. Secondly, these benchmarks are typically constructed under closed-wort assumptions, lacking the inter-task consistency needed to assess reasoning across modalities (Fu et al. 2024]] Li et al.] (2024a). As result, the absence of quantitative multi-level evaluation hinders meaningful comparison across MLLMs.
More recent benchmarks have begun to shift toward open-world evaluation and multimodal reasoning tasks
Described Object Counting
(Wu & Xie 2024| Zang al. (025). While this repre- sents progress, current benchmarks do not adequately assess the nuanced performance necessary evaluate MI.LMs progression towards human-like intelligence in real-world settings. They require largely primary visual comprehension and fall short of measuring higher-order reasoning and spatial understanding Yuue et al., 2024
Object Detection
Perception
Object Existence Determination
LENS
Relation Extraction
Visual Grounding
Liu et al.) 2024c). Furthermore, data distributions of- len differed between tasks, so that high performance perceptual tasks did not necessarily translate into strong inference capabilities in more complex integrated mul- timodal tasks (Fan et al. 2025). As a result, they ignore the synergistic effect of the combinations of lower-order perceptual abilities on higher-order reasoning and are hard to provide a fine-grained assessment.
Understanding
Region-wise OCR
Spatial Relationship Comprehension
Reasoning
Scene Knowledge Inference
Figure 1: Illustration of the task split in Lens.
In this study, we propose hierarchical and comprehensive evaluation framework Lens specifically designed to assess the multimodal capabilities in real-world scenarios. Our benchmark focuses on both isolated tasks and the integration of perception, understanding, and reasoning- core tiers essential for intelligent multimodal systems. As shown in Figure ! Lens encompasses eight tasks, systematically organized into three hierarchical tiers with eight subtasks, and comprises 3.4K real-world photographs and 60K+ human- authored questions, in 12 diverse scenarios innriiii-nnnnuiingg streets, stations, schools, homes, and more, which can be roughly divided into three themes: "Home", "Education", and "City", and we visualize the high-frequency words under different themes in Figure 2 53% of the images are from 2025 and more than 80% of the images are from after September 2024, ensuring the content reflects contemporary environments.
For task design, Lens adopts an open-set configuration, allowing queries to be posed in natural language and grounded in authentic photographic content. This design enables evaluation of model performance complex, ambiguous, and information-rie settings, better aligning with real-time human demands. Moreover, benchmark introduces multi-level tasks, which are unified by shared visual contexts, making Lens well-suited for assessing the synergistic effects of lower-level perceptual abilities (e.g., object detection, localization) on higher-order reasoning tasks. To succeed in Lens, models must jointly process multimodal input, recall domain knowledge, and conduct multi-step reasoning to arrive valid conclusions. Our experimental results
2

---

Published as a conference paper ICLR 2026
confirm that current state-of-the-an MLLMs still struggle with these reasoning- tasks, revealing a significant gap between perception and functional understanding.
To bridge this gap, we propose the Self-Driven Multi- Multi-Expert Collaborative Framework (SMEC), novel reasoning framework that leverages the MLLM itself as a set of specialized experts instantiated through
self-generated prompts. Unlike tool-calling approaches (Wang et al., 2025, Gao et al. 2025 Liu et al.
2025b Zhang et al.) 2024b) that rely external modules, SMEC treats the base MILM as versatile reasoning engine: simulates diverse expert perspectives (e.g., spatial analyst, text interpreter, commonsense reasoner) via role-specific prompts and composes their insights into coherent final answers. This collaborative mechanism encourages the model to extract, expand, and integrate rich, task-relevant information. Our experiments demonstrate that SMEC significantly boosts performance validating its potential as a general-purpose, language-nate native method for enhancing multimodal reasoning. We take data privacy, copyright compliance, and platform terms of service seriously. Rigorous collection, filtering, and documentation procedures were implemented and detailed in Section 2. and 6 and Appendix A.4, To our knowledge, we make the following contributions:
reasoning tasks within Lens,
Realistic and Up-to-Date Evaluation. By leveraging a newly collected set of high-resolution, naturalistic images, our benchmark evaluates the latest multimodal reasoning models in settings that closely reflect real-world complexity.
Multi-Leevel Evaluation. It supports fine-grained and interpretable evaluation across three core dimensions--percepphhe understanding, and -rroniniig comprehensive view of
model's s multimodal competence.
Synergistic Capability Evaluation. Unlike existing benchmarks that often assess tasks in isolation, our framework emphasizes the synergistic effects of lower-level perceptual abilities on higher-order reasoning tasks. The experimental results also confirm that low-level tasks facilitate the reasoning of MLLMs on more complex, high-level tasks (e.g.. Scene Knowledge Inference).
Towards Generalizable Intelligence. By capturing both perceptual and reasoning performance in integrated tasks, our benchmark helps identify the gaps between current model capabilities and the requirements of human- aalnned reasoning systems and measure the shortcomings of current models.
Self-driven Reasoning Enhancement. We introduce SMEC, self-driven multi-expert collaborative framework that simulates specialized experts within a single MLLM through self-generated prompts. Unlike tool-calling approaches, SMEC enables modular, multi-perspectivy reasoning natively, leading to significant gains on complex reasoning tasks.
Comparison with existing benchmarks. Compared with existing multimodal benchmarks (Liu et al. 2024c Yue et al., 2024: Li et al. 2024b), Lens provides more contemporary, diverse, and densely annotated visual content. Our benchmark constructed from contemporary social-medin images, ensuring strong timeliness and significantly reducing the risk of contamination from pre-training corpora. In contrast to task saekispec datasets (Liu et al., 2024d| a Wei et al., 2024). our benchmark provides rich, multi-task annotations with the same visual content, across perception, understanding, and reasoning, enabling controlled analysis of cross-tash task synergies within a unified distribution. Additionally, Lens offers the detailed thought process in real-world reasoning tasks for potential future research. Appendix A.1 further discusses related work.
2
LENS DATASET AND BENCHMARK
2.1 DATA COLLECTION
The image data collection in our benchmark focuses on real-world scenes to ensure diversity, representa- tiveness, and practicality for visual perception, understanding, and reasoning tasks. To this end, we first defined a set of common real-life scenarios that are highly relevant to typical human visual experiences. The selection principle was that each visual scene should contain distinguishable and representative semantic
3

---

Published as a conference paper ICLR 2026
sunny
inneersection
reading
doina
beterponneedrokee
a
port
din
ng
nall
living
Toom
Living room
Kitchen
Classroom
2+3
A
oay
1oshz dta
Time.
airplans
classroom
tudy
shop
SOT
room
kitchen
Space
Bathroom
RE
Playground
season
daytime
aaee
booksaee
clothess
aarrad
cooking
style
sollet
school
safety city
table
student
daytime
Street Airport
apeeding weather dsmmate
chair
teacher
domestic
Scoot
iit tatterril bathroom From
Scenarios
City
Education
Home
Figure Three core themes, "Education", "City", and "Home", along with their word clouds of the scenario distributions by name size.
Table I: Comparison with other recently released multimodal benchmarks.
Interleaved
Beachmarks
Venue
Reasocing
Image Text
Image
V*
2024
C4
SA 1B
Kinllev
2023
C4PR
CVPR'24
ImageNet Russakwwkk et
LATON-5B
Schulmare
2022
HalsQuest
Ogen imggs
Kklin 2020
AS-V2
Winr
BOCIV'34
CoCo
ai 201K
MMDen
DOCV'24
Interne mingge
IIC RefLaCo
20531
NeurlPS'24
V
Multiple existing dalasets
Visaal Cal
STD 01
2023
NewiPPS124
Multiple exittig datnsets
MC-Rendh
2074
24Xi
X
Multigle evicine ditaects Intemet
CODE
2024
UCV
V
X
Flick22lkk series Young et 2014 19 Vummer et 2015
ChatteBox
AAAI22
Visual Tresionn Kiatinn
Tens
V
Collect manually from social media 53% published later than Jan. 2025
"AL." Auabure "Cat" 7 Coun: 'Loc" Lecalitation "Rel", Relation
content. For example, street scenes are usually populated with cars, pedestrians, and storefronts, while indoor environments like classrooms often involve students, teachers, and educational materials. To avoid regional or cultural bias and ensure a broad distribution of content, we collected images from multiple social media platforms, including X (formerly Twitter), Instagram, Weibo, and RedNote. These platforms were chosen due their global user bases and diverse content coverage across regions and lifestyles. During the collection process, we strictly complied with the copyright and licensing regulations of cach platform, ensuring that data was collected only from publicly accessible posts and that no images were downloaded from sources explicitly prohibiting data reuse or redistribution. Moreover, to facilitate the evaluation of multiple subtasks within the same image (e.g., detection, OCR, scene knowledge inference), we curated images that exhibit rich semantic content while maintaining scene clarity. Complex or ambiguous images were manually filtered out avoid introducing noise that could hinder benchmarking or evaluation consistency. Please note that we manually collect these data that are completely open to the Internet and have complied with the developer agreement of the relevant platform (e.g., Developer Policy of X[aand Meta D. ensuring non-commercial use, erasing geographic information, user personal information, etc. from the original data. For further details, please refer to the appendix A.4
2.2
TASK DESIGN AND ANNOTATION PROCESS
To construct comprehensive and diverse benchmark, recruited over 50 undergraduate and graduate students (including authors) as human annotators to assist in the process of question collection and task annotation and paid the corresponding salary. These annotators were carefully trained ensure high
4

---

Published as a conference paper ICLR 2026
8. Scene Knowledge Inference
7. Spatial Relationship Comprehension
Spatial Information
From perspective of the is located in the left front it.
When driving on the for left lone, heading to Googi Airport need to change to the far right
Abona
O
S
Bus Airplane
Knowledge Injection
Reasoning
6. Region- wise OCR
Road Sign
Object Detection
Gaogi Airport
(81.3,696.5.933222
2. Object Counting
Progression
5. Visual Grounding
Understanding
Perception
Obj
Num.
LENS
The blue and white airplane the top of the picture:
Road Sign
[256, 6,152. 8,959 .,, 366,66666
Airglane
3. Object Existence Determination There are trucks, buses and
traveling front and behind.
4. Relation Extraction
The are treveling in opposite directions.
Figure Lens consists of eight tasks at three levels. Perception tasks focus on recognizing object attribute and counting. Understanding tasks emphasizes localization and inter-object relationships with textual information. Reasoning tasks demand the use of external knowledge beyond the visual input and involve multi-step, complex reasoning processes to arrive at the correct answer.
annotation quality and consistency. As shown in Figure the generated questions were divided into three major categories: Perception, Understanding, and Reasoning. For Perception and Understanding, they primarily target the model's ability to perceive visual objects and align them accurately with natural language descriptions. They emphasize fine-grained visual grounding and object recognition rather than abstract reasoning. At last, reasoning-basel questions aim to evaluate the model's 's ability understand user intent and reason based on external knowledge, commonsense, physical laws, or background information beyond the purely visual content of the image. Based these assessment dimensions, we compare Lens with related multimodal benchmarks in Table ann formulate our challenging open-ended, language-drive tasks: Object Counting (OC), Object Detection (OD), Object Existence Determination (OE), Relation Extraction (RE), Visual Grounding (VG), Region-wise OCR (OCR), Spatial Relationship Comprehension (SRC), and Scene Knowledge Inference (SKI). We provide a more detailed introduction of these tasks in Appendix A.2
2.3 DATA ANALYSIS
We aim to construct dataset that is not only comprehensive and dynamic but also emphasizes reasoning capabilitiis. In the following analysis, we demonstrate the strengths of our benchmark in terms of diversity of images and annotations. The quantitative results are visualized in Figure 4
First, our benchmark incorporates scene-award content and real-time data. As shown in Figure more than 50% of the images in our dataset were collected in 2025, and approximately 70% were collected in November 2024 and beyond, which avoids potential data leakage. Many images reflect dynamic scenes (e.g., crowded streets, interactive environments) captured different times and locations, aligning with real-world scenarios.
Second, in our dataset, the coverage of a wide range of object categories, scene types, and bounding box annotations further supports diverse downstream tasks from detection to high-level semantic inference and
5

---

Published as conference paper ICLR 2026
Sumpli Filed
0000
1000
0000
After 2005
Frecuancy
0000
124
City
Alpart
1000
Resturent
Trand Liee
26'
1e.12
2
2484
202 2456
Scenic apst
2004 0024.12
(b) Distributionn of image collection lime.
(6) The distribution the image sizes the datasct.
Home
Livng sooes
Sancon Mili
27
0:35
14243
Badrrom
Borore 2004
Procuagy
2.30
12.000
11.19
Plyground
11,000
90.00
18:99
Clawroore
8:2
4.20
(3) Alluvial dingram am of soene and collection time distribution images.
(d) Size distribution of bounding box annotations.
(e) Number of Q&AA entries for different casks.
Figure Statistical analysis our dataset. We visualize the temporal distribution of the images for different scenarios, size distribution of images and bounding box annotations, and number of QA entries for different tasks, demonstrating the timeliness and diversity of our data.
interleaved image-text understanding. As illustrated Figure 4 (c), the high resolution the images in dataset makes it challenging for fine- mne-grained understanding of the model and supports evaluation across varying input sizes. Additionally, as shown in Figure (d), the various objects are labeled with different sizes of bounding boxes meet the needs of multi-scale object detection and region-wise OCR evaluation.
Furthermore, beyond perception, our dataset facilitates reasoning-orienter research by supporting tasks that require: Spatial reasoning (e.g., understanding object layouts and geometric relationships). Relational inference (e.g., extracting interactions between objects). Commonsense knowledge application (e.g.. inferring the feasibility of behavior or scene functionalities). Cross-modal alignment (e.g., grounding free-form language to specific visual content). We also analyze the question- answerr pairs distribution of these tasks and Figure (e) (e) shows that over 60% of the questions in the dataset go beyond simple recognition, explicitly encouraging models to reason about the scene, context, and user intent. Please refer to the Appendix A.5 for more low-level visual analysis.
3 SELF-DRIVEN MULTI-EXPERS COLLABORATIVE FRAMEWORK
We propose a Self-Driven Multi-Expert Collaborative Framework to tackle complex visual reasoning tasks that require diverse domain expertise and multi-level inference. Built on an instruction-tunee MLLM, our framework dynamically assembles set of self-generater experts, each embodying distinct reasoning perspective, as shown in Figure 5
Expert Generation via Prompted Role Construction. Given query q, the base model 0 first produces coarse initial answer ag. To enrich the reasoning space, a Meta Generation Prompt Pg is used to itcratively generate expert role deseriptions d' simulating specialized agents (e.g., geospatial analyst, cultural analyst). Each valid description yields a new expert response a,, which is added to the answer set A. This loop continues until either diversity criterion is violated maximum number of iterations N, is reached.
6

---

Published as a conference paper ICLR 2026
What the address the picture Please anewer North Amerion, South America or unable to determine.
Stage 2: and initial answers.
Build an expert teom based on the questions
Stage 3: Expert screening model retains the most relible experts.
Given the dignff cont overlop between Scone understonding and Georpatia andlysis expert, greater suhtbiiity getomopia onclysie geolocation related experts urderstording boan excluted from concidriitinn
as
Region perception
understonding expert
expert
Stage Original model outputs
arswer.
Unkown It not passible determine the foilening reosons:
Puf
Remove the understanding expert.
1. The mage shows the ieterior of a trein staton, and
construction
Cultural andyysis expert
Geospetiol analysis
scndening
cnough gesgrophic geogrophic
ceteriie
lecation.
2. Althouge "Janaica" (Janaica) on isend country loceted in North Americn, the syebel may be ony part the station' represent geogrophic
Stage 4: Integrate obtain the Final answer.
Original model
Final Answer
North Americe, This conclusion rebustly supported by the cellective expertise the Gezapatall analyas expart, reg on perception expert, astural analyst. who indicaters consittntt with North particulorly indeperdenll plaming tranaportation
Figure 5: SMEC is a step-wise reasoning framework for answering complex visual questions using the Multi-Expert Collaboration. Starting from an initial model response, SMEC constructs a specialized expert team, to re-evaluate and refine the answer. Through expert screening and integration, unreliable and duplicated experts are filtered out, and a consensus-bes modular, expert-driven collaboration in visual reasoning tasks.
final answer is produced, demonstrating the advantages of
Prompt Adaptation and Redundancy Filtering. To avoid degenerate expert generation, the framework dynamically updates Pg when semantically redundant descriptions emerge. This adaptation encourages exploration of novel expert roles. Implicit expert screening is performed by discarding repetitive or low- information descriptions, ensuring a concise yet diverse expert team with minimal computational overhead.
Consensus-Driven Answer Integration. The final stage aggregates the expert responses via Collaboration Prompt Pe. prompting / to synthesize unified answer Ofinal through deliberative reasoning. This mimics human expert panels that reconcile differing views to reach a robust consensus.
We detail formal description of the process in Appendix AA.1 In this way, our framework instantiates modular experts purely via prompt-based self-conditioning. Unlike fixed-rule multi-agent systems and tool-calling methods, our framework leverages the generative flexibility MLLM to dynamically instantiate and evolve its behaviors, without requiring external task task-specifie supervision.
4 EXPERIMENTS
4.1 EVALUATION MODELS
To illustrate the difficulty of our benchmark and evaluate the latest advances in current research, we evaluate various MIL.Ms belonging to three major categories: Closed-source generalist MILMMs, such as GPT-4o (Achiam et al. 2023) and 5emini2.5 Pro (Team et al. 2024). Open-source generalist MLLMs like Qwen2.5-
VL (Bai al. 2025). Deepseek-ViL VV2 (Wu al. 2024). Gemma3 (Team et al., 2025a Intern (Zhu et al.
2025). Multimodal reasoning models QvQ-Max, Kimi-VL-thinking (Team et al. 2025b) and GL.M4.11
Thinking (Hong et al. 2025), focusing on advanced reasoning capabilities. The release dates of these models are distributed from Dec. 2024 to Apr. 2025.
4.2 EVALUATION STRATEGY
To ensure a fair and efficient assessment of model performance across our benchmark, we adopt two evaluation strategies for main results. For perception and understanding tasks, models were evaluated based on their
7

---

Published as conference paper ICLR 2026
Table 2: Comparison state-of-the-art methods on Lens. We evaluate object detection (OD) performance
using APso (Lin et al. 2014), visual grounding (VG) performance with ACC@0.5 (Xiao al. 2024 and use
accuracy for other tasks. Task abbreviations follow the definitions provided Section 2.2 "MoR 1B/3B" denotes 3B Mixture of Experts model with 1B parameters activated. "N/A" denotes the official documentation
does not confirm that the model is applicable for the task. Best performing models are shaded in red.
Model
Perception
Understanding
Reasoning
Methods
size
OC
OD
VG
OCR
SRC SKI
MLLM (closed source)
GPT-4o
54.32
N/A
85.09
72.77
N/A
42.86
51.14
55.20
Gemini2.5-Pro
60.18
47.40
86.59
76.52
25.61
61.95
56.20
59.31
Open source
Reepseck-VL2--in
MoE IB/3B
56.22
21.12
72.11
58.73
16.09
44.01
38.97
45.12
Deepseek-VIL VL2
MoE 4.5B/27B
61.41
46.08
77.68
69.18
42.47
48.76
44.58
49.50
Gemma3
4B
38.85
N/A
71.88
62.98
N/A
27.03
39.53
45.18
Gemma3
12B
44.65
N/A
73.21
62.78
N/A
33.98
43.33
48.56
In3ernVL3
2B
55.81
18.39
71.96
64.49
15.22
45.51
40.56
48.59
InternVL3
9B
55.63
25.79
77.49
67.18
18.18
48.79
44.69
51.32
InternVL3
38B
62.78
43.44
81.60
71.37
24.98
51.72
47.18
51.85
InternVI.3
78B
61.38
47.44
84.87
74.93
27.24
54.21
49.39
55.17
Qwen2.5-VL
3B
58.76
35.16
74.01
66.52
39.44
52.43
40.33
46.50
Qwen2.5-VL
7B
58.35
37.75
83.75
71.58
40.11
61.65
46.28
48.87
Qwen2.5-VI.
32B
62.25
39.93
83.60
74.57
41.15
65.64
51.66
51.54
Qwen2.5-VL.
72B
59.75
43.48
85.67
75.98
44.98
68.51
53.65
54.79
Reasoning model
QVQ-Max
72B
49.95
N/A
85.37
74.01
N/A
58.67
50.80
58.86
K Kimi-VL-thinking
MoE 2.8B/16B
46.87
N/A
72.77
48.16
N/A
30.21
29.40
36.44
direct outputs without additional inference-time computations. For complex reasoning tasks, which require deeper multi-step inference, allow models to generate multiple candidate responses per question and the final prediction is then selected via majority voting (Liu et al. 2025a). For qualitative judgment, we follow prior work (Wang et al. 2023) and employ large language model (e.g. GLM4-flash (GLM et al. 2024) as an automatic evaluator. The LLM is prompted to produce multiple pieces of evaluation evidence for calibration, comparing the model- generrate responses against human-annoti annotated answers, aiming to offer a consistent framework for evaluating model performance across diverse tasks.
4.3 EVALUATION RESULTS
We evaluate a suite of state-of-the-ar Multimodal Large Language Models on our benchmark, which spans three tiers and eight tasks. Results, shown in Table 2 reveal insights into model scaling, inter-task dependencies, and capability gaps in current MI.LMs.
Model Scaling and General Trends. We observe consistent performance gain with increased model size both closed- and open-source models. For example, Qwen2.5- VL improves steadily from 3B to 72B, achieving top performance reasoning tasks. InternVL3 shows similar gains in OD, rising from 18.39%
8

---

Published as conference paper ICLR 2026
8
8.4
877
E11
6.0
8
8.7
8:00
(b) Scatter plots between ecn perception.
Lincar regression analyses un Pensoning tasks
undersianding, and nessening
8:94
8:30
8.70
3:90
6:35
MCC notur ineeta Mout integesian Eadisieed
sc: Postare impertuce Regrosse Canfirnnts
(o) Perception, understanding. and reasoning correlation matrix
(d) The feature importance analysis of regression coefficients
Figure 6: Statistical analysis of model accuracy and synergies between different tasks.
(2B) to 47.44% (78B), though performance saturates at higher scales. These trends confirm that scaling remains a key driver for multimodal reasoning, albeit with diminishing returns in some subtasks.
Perception: Foundation for Higher Cognition. Perception-level tasks form the backbone of visual reasoning. Closed-source models like Gemini2.5 and GPT-4o excel OE (86.59% and 85.09%, respectively), although OD support is lacking. Among open-source models, Deepseek- VL2 and Qwen2.5-VL-72B deliver competitive OD and OC performance. Notably, models with stronger perception capabilities tend to exhibit superior reasoning performance, highlighting the foundational role of low-level visual understanding.
Progress and Bottlenecks. Understanding tasks assess models' ability interpret struc- tured visual semantics with textual information. Gemini2.5-PP leads in RE (76.52%) and OCR (61. (61.95%) showcasing robust relational and textual grounding. However, VG remains bottleneck even for large-scale models like InternVL3-78B (27.24%) and Qwen2.5-VL-72B (44.98%), suggesting persistent challenges in fine-grained spatial-semantic alignment.
Understanding:
Reasoning: High-Level Generalization. Reasoning tasks are the most demanding. Closed-source models such GPT-4o and Gemini2.5- achieve strong results (51.14%/56.200) on SRC and 55 55.20% on SKI). Among open-source models, Qwen2.5-VI. 2iee2255-VL-728 VL-72B leads, while the sesoninn-specializedd QVQ-Max approaches closed-soure souure-- performance (58.86% on SKT) despite lacking OD and VG capabilities. This suggests that explicit reasoning models can partially compensate for perceptual limitations, likely relying on test-time scaling rather than grounded perception.
4.4 SYNERGISTIC EFFECTS ANALYSIS
To analyze the cross-task performance patterns different models, we perform a statistical analysis the synergies between different tasks and visualized the results as in Figurel6 We compute the Pearson correlation coefficients between Perception and Understanding tasks and observe notable interdependencies. OC and RE exhibit strong positive correlation of 0.73, while OE and OCR show similarly significant correlation of
9

---

Published as a conference paper ICLR 2026
0.67. These results indicate that effective performance in perception directly contributes understanding, which in turn underpins downstream reasoning. Scatter plot visualizations further confirm these links, OCR, in particular, correlates strongly with both SRC and SKI, underscoring its central role in enabling semantic reasoning. Linear regression analyses reinforce these findings: OE and OCR are strong predictors of SRC, while OC and RE significantly influence SKI, highlighting how object-level detection and relational reasoning jointly support high-level inference. Finally, we apply second-order polynomial regression and the feature importance analysis of regression coefficients reveals sask-specific contributions. These insights collectively demonstrate the layered structure of visual reasoning pipelines, where perception and understanding stages must be well-aligned to support robust inference. For further analysis, please refer to the appendix A.7
4.5 EFFECTIVENESS OF SMEC
To further evaluate the effectiveness of our proposed SMEC framework, we conduct experiments on the Scene Knowledge Inference (SKI) task using both sampled subset of 3,500 the full test set (Table 3. Compared to the direct in- ference baseline, SMEC consistently improves per- formance across both model scales and different iteration depths. For Qwen2.5VL-7B accuracy in- creases from 39.80% to 43.24% as the number of iterations N₂ grows from to 3, demonstrating the benefits of multi-step expert collaboration. Notably, SMEC also outperforms Self-Refine (Madaan et al., (023) and Majority voting (Chen et al. (0224b) under
Table 3: Accuracy comparison with different settings.
Methods
Model
Iterations
Performance
ansswer pairs and
Direct
Owen2.5V1-
39,80
Majority voting Self-Refine
Qwen2.5VL-
40.66
Qwen2.5VL-76
40.51 (40.71)
SMEC
Qwea2.5VL-27
1
41.35 (+11.5)
SMEC
Qwen2.5VL-77
2
42.97 (43.17)
SMBC
Qwen2.5VI-
43.24 (+3.44)
Direct
Qwen2.5VLL322b
49.17
SMBC
Qwen2.5VL.
3
52.44 (+3.27)
Direct (Full data) SMEC (Full data)
Qwen2.5VL
51.54
Qwen2.5VL-32b
54.66 (+3.12)
the same setting. A similar trend appears at larger scales. With Qwen2.5VL-32B, SMEC improves accuracy from 49.17% to 52.44% with three iterations, confirming that the benefits of iterative expert collaboration scale with model capacity. Importantly, when evaluated on the full test set rather than the 3,500-sample subset, SMEC continues to provide consistent improvements (+3.12%) This indicates that SMEC's gains are not distribution-specific and remain stable under more comprehensive evaluation conditions.
5 CONCLUSION
We contribute Lens, a multi-level benchmark designed to evaluate Multimodal Large Language Models (MLLMs) across perception, understanding, and reasoning. Unlike prior benchmarks, Lens aligns all tasks the same set of realistic, contemporary images, enabling fine- fnn-grainedd analysis of how low-level visual capabilities support higher-order reasoning. The evaluation of recent MLLMs further reveals a consistent performance gap in reasoning tasks, highlighting the limitations of current models in integrating perception and cognition. To address this, we proposed SMEC, self-driven multi-expert collaborative framework that prompts the MLLM to simulate a panel of specialized agents. Together, Lens and SMEC offer a new paradigm for evaluating and enhancing reasoning intelligence in MILMs,, paving the way more robust, human-aligned multimodal intelligence.
6 ETHICS STATEMENT
Data Collection and Privacy. All images in the Lens dataset were collected manually from publicly available posts social media platforms. We respect and strictly complied with the developer agreements and copyright regulations of these platforms. No private or restricted data was accessed, and all collection adhered to academic research policies. This dataset is intended strictly for non-commercial academic research
10

---

Published as a conference paper ICLR 2026
purposes. If any content in this project is found to raise concerns related privacy, copyright, or legal compliance, please contact us at yaoruilin @ whut.edu.cn. We will promptly review the request and are willing to remove, modify, or withdraw the sensitive data or materials.
Human Annotation. The dataset was constructed with the assistance of more than 50 undergraduate and graduate student annotators, who were trained to ensure annotation quality and consistency. All annota- tors were fairly compensated for their work. The annotation process involved only task-related labeling (e.g. bounding boxes, question-answer generation) and did not involve collection of personal or sensitive information about the annotators.
Bias, Fairness, and Representation. Although images were sourced from global platforms to ensure diversity of cultural and regional content, dataset bias may still exist due to platform-level demographic imbalances and uneven scenario representation. We acknowledge that these limitations may influence model evaluation results, and encourage future work to further broaden geographic and cultural coverage.
Research Integrity and Transparency. This study follows established standards of research integrity. We provide detailed dataset descriptions, evaluation protocols, and implementation details. We have conflicts of interest or external sponsorship that might bias the study.
7 REPRODUCIBILITY STATEMENT
We have taken extensive measures to ensure the reproducibility of results. A complete description of the Lens dataset, including data collection principles, filtering steps, and annotation procedures, is provided in Section 2 and Appendix. Details of the evaluation tasks, metrics, and benchmark comparisons reported in Section 4, with additional analyses in Appendix. The implementation of our proposed Self-Driven Multi- Expert Collaborative Framework (SMEC), including the expert construction, screening, and integration process, is fully described in Section 3 and formalized in Appendix A.9. Hyperparameter settings, model configurations, and ablation experiments are included in the appendix to facilitate replication of results. Together, these resources ensure that both dataset construction and methodological contributions can be faithfully reproduced by the research community.
ACKNOWLEDGEMENT
This work is supported by the National Key Research and Development Program of China No.20227/D0160604 and NSFC No.62306162.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08777 087744 2023.
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for few-shot learning. Advances in neural information processing systems, 35:23716-23753 2022.
Alisson Azzolini, Junjie Bai, Hannah Brandon, Jiaxin Cao, Prithvijit Chattopadhyay, Huayu Chen, Jinju Chu, Yin Cui, Jenna Diamond, Yifan Ding, et al. Cosmos-rcasonl: From physical common sense to embodied reasoning. arXiv preprint arXiv:2503. 15558, 2025.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint 2502: 2025.
11

---

Published as a conference paper ICLR 2026
Holger Caesar, Jasper Uijlings, and Vittorio Ferrari. Coco-stuff: Thing and stuff classes in context. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1209-1218, 2018.
Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Börje F Karlsson, Jie Fu, and Yemin Shi. Autoagents: A framework for automatic agent generation. arXiv preprint arXiv:2309. 17288, 2023a.
Jiangji Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang. Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, et al. From persona to personalization: survey on role-playing language agents. arXiv preprint ar arxiv:2404. 18231, 2024a.
Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multimodal l1m's 's referential dialogue magic. arXiv preprint arXiv:2306. 15/95, 2023b.
Lingjiao Chen, Jared Quincy Davis, Boris Hanin, Peter Bailis, lon Stoica, Matei A Zaharia, and James Y Zou, Are more IIM calls all you need? towards the scaling properties of compound systems. Advances Neural Information Processing Systems, 37:45767-44599 2024b.
Wei Chen, Long Chen, and Yu Wu. An efficient and effective transformer decoder- based framework for multi-task visual grounding. In European Conference on Computer Vision, pp. 125 2-5--11, 2024c.
Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In The Twelfth International Conference on Learning Representations, 2023c.
Zebang Cheng, Zhi-Qi Cheng, Jun-Yan He, Kai Wang, Yuxiang Lin, Zheng Lian, Xiaojiang Peng, and
Alexander Hauptmann. Emotion-llama: Multimodal emotion recognition and reasoning with instruction tuning. Advances in Neural Information Processing Systems, 37:1 10805-1 2024.
Ming Dai, Lingfeng Yang, Yihao Xu, Zhenhua Feng, and Wankou Yang. Simvg: A simple framework for visual grounding with decoupled multi-modal fusion. Advances in neural information processing systems, 17:121670-121698 2024.
Jiajun Deng, Zhengyuan Yang, Tianlang Chen, Wengang Zhou, and Houqiang Li. Transvg: grounding with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1769-1779 2021.
End-to-end visual
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning. arXiv preprint arXiv:2301.00234 2022.
Mark Everingham, Lue Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88:303-338, 2010.
David Fan, Shenghang Tong, Jiachen Zhu, Koustuv Sinha, Zhuang Liu, Xinlei Chen, Michael Rabbat, Nicolas Ballas, Yann LeCun, Amir Bar, et al. Scaling language-free free visual representation learning. arXiv preprint arXiv:2504.01017 2025.
Chaoyou Fu, Yi-Fan Zhang, Shukang Yin, Bo Li, Xinyu Fang. Sirui Zhao, Haodong Duan, Xing Sun, Ziwei Liu, Liang Wang, et al. Mme-survey: A comprehensive survey on evaluation of multimodal Ilms. arXiv preprint arXiv:241 1529,, 2024.
Zhi Gao, Bofci Zhang, Pengxiang Li. Xiaojian Ma, Tao Yuan, Yue Fan, Yuwei Wu, Yunde Jia, Song- Chun Zhu, and Qing Li. Multi-modal agent tuning: Building VLM-driven agent for efficient tool usage. In The Thirteenth International Conference on Learning Representations, 2025. URL https //openreview. net/ httt:/orumm.d-000mm..a0.
12

---

Published as a conference paper ICLR 2026
Team GLM, Aohan Zeng. Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406. 12793, 2024.
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang, Xiao Bi, et al. Deepscek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
14953-14962, 2023.
Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al. Metagpt: Meta programming for multi-agent aaget collaborative framework. In The Twelfth International Conference on Learning Representations, 2023.
Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, et al. Glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:2507.01006, 2025.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025.
Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, Defu Lian, Yasheng Wang, Ruim- ing Tang, and Enhong Chen. Understanding the planning of Ilm agents: A survey, arXiv preprint 2rXiv:2402.02716. 2024.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Alek- sander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card, arXiv preprint arXiv:2412.1672) 2024.
Yifan Jiang, Kexuan Sun, Zhivar Sourati, Kian Ahrabian, Kaixin Ma, Filip Ilievski, Jay Pujara, et al. Marvel: Multidimensional abstraction and reasoning through visual evaluation and learning. Advances in Neural Information Processing Systems, 37:46567-46592, 2024.
Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr. A diagmostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2901-2910, 2017.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4015-4026, 2023.
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123:32-73, 2017.
Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Tusee, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. International journal of computer vision, 128(7):1956-1981 2020.
Lin Li, Guikun Chen, Hanrong Shi, Jun Xiao, and Long Chen. A survey on multimodal benchmarks: the era of large ai models. arXiv preprint 4409 18142, 2024a.
13

---

Published as a conference paper ICLR 2026
Manling Li, Shiyu Zhao, Qineng Wang, Kangrui Wang, Yu Zhou, Sanjana Srivastava, Cem Gokmen, Tony Lee, Erran Li Li, Ruohan Zhang. et al. Embodied agent interface: Benchmarking llms for embodied decision making. Advances in Neural Information Processing Systems, 1111100028-000550 2024b.
Zhuowan Li, Xingrui Wang, Elias Stengel-Eskin, Adam Kortylewski, Wufei Ma, Benjamin Van Durme, and Alan Yuille. Super-clevr: A virtual benchmark to diagnose domain robustness visual reasoning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 14963-14977 2023.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision-ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part 13, pp. 740-755. Springer, 2014.
Chen Cecilia Liu, Fajri Koto, Timothy Baldwin, and Iryna Gurevych, Are multilingual Ilms culturally-divese reasoners? an investigation into multicultural proverbs and sayings. arXiv preprint arXiv:2309.0859), 2023a.
Fan Liu, Wenshuo Chao, Naiqiang Tan, and Hao Liu. Bag of tricks for inference-time computation of Ilm reasoning, arXiv preprint arXiv:2502.07199, 2025a.
Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning. Transactions of the Association for Computational Linguistics, 11:635665,, 2023b.
Junzhuo Liu, Xuzheng Yang, Weiwei Li, and Peng Wang. Finecops-ref: A new dataset and task for fine- grained compositional referring expression comprehension. arXiv preprint arXiv:2409. 14750, 2024a.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, pp- 55-55. Springer, 2024b.
Weiwen Liu, Xu Huang, Xingshan Zeng, xinlong hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, Zezhong WANG, Yuxian Wang. Wu Ning, Yutai Hou, Bin Wang, Chuhan Wu, Wang Xinzhi, Yong Liu, Yasheng Wang, Duyu Tang, Dandan Tu, Lifeng Shang, Xin Jiang, Ruiming Tang, Defu Lian, Qun Liu, and Enhong Chen. ToolACE: Winning the points LLM function calling. In The Thirteenth International Conference Learning Representations, 2025b. URL https://
openreview. net. wwt.w/iwwwweeeetnurnneeee /tr.:mmmm111BBBBB66e0
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pp. 216-233. Springer, 2024c.
Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu- Xu-Cheng Yin, Cheng Liu, Lianwen Jin, and Xiang Bai. Ocrbench: on the hidden mystery of oer in large multimodal models. Science China Information Sciences, 67(12)220010, 2024d.
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiagi Wang. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.01785 2025c.
Chuofan Ma, Yi Jiang, Jiannan Wu, Zehuan Yuan, and Xiaojuan Qi. Groma: Localized visual tokenization for grounding multimodal large language models. In European Conference on Computer Vision, pp. 417-435. Springer, 2024.
14

---

Published as a conference paper at ICLR 2026
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534 46594, 2023.
Wujian Peng, Sicheng Xie, Zuyao You, Shiyi Lan, and Zuxuan Wu. Synthesize diagnose and optimize: Towards fine-grained vision-language understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 13279-13288, 2024.
Bryan Plummer, Liwei Wang. Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentent models. In Proceedings of the IEEE international conference on computer vision, pp. 2641-2649, 2015.
Pavan Kartheek Rachabatuni, Filippo Principi, Paolo Mazzanti, and Marco Bertini. Context-aware chatbot using milms for cultural heritage. In Proceedings of the 15th ACM Multimedia Systems Conference, pp. 459-463, 2024.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211 2522 2015.
Gabriel Sarch, Lawrence Jang, Michael Tarr, William W Cohen, Kenneth Marino, and Katerina Fragkiadaki.
Vlm agents generate their own memories: Distilling experience into embodied programs of thought. Advances in Neural Information Processing Systems, 37:75942-75985, 2024.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural information processing systems, 35:25278-2522 2022.
Hao Shao, Shengju Qian, Han Xiao, Guanglu Song. Zhuofan Zong, Letian Wang. Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. Advances in Neural Information Processing Systems,
2024.
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-rl: A stable and generalizable rl-style large vision-language model. arXiv preprint arXiv:2504.07615, 2025a.
Haozhan Shen, Kangjia Zhao, Tiancheng Zhao, Ruochen Xu, Zilun Zhang, Mingwei Zhu, and Jianwei Yin. Zoomeye: Enhaneing multimodal 11ms with human-like zooming capabilities through tree-based image exploration. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 6613-6629, 2025b.
Charlic Snell, Jachoon Lee, Kelvin Xu, and Aviral Kumar. Scaling 11m test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408. 2.X0::2408.03314 2024.
Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 2024.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503. 19786, 2025a.
Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, et al. Kimi-yl technical report. arXiv preprint 27444 2025b.
15

---

Published as a conference paper ICLR 2026
Yunjie Tian, Tianren Ma, Lingxi Xie, and Qixiang Ye. Chatterbox: Multimodal referring and grounding with soain-of-questions Proceedings of the AAAI Conference on Artificial Intelligence, 39(7):7401-7409 Apr. 2025. doi: 10.166//aaii.v333 v39....29979999 URL https:://jss..... aaaii..s..aa.................... aaai. arg/index. php/ AAAI/ article/view/33 32796
Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal 1mss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9568-9578, 2024.
Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. arXiv preprint arXiv:2305. /7926, 2023.
Renxi Wang, Xudong Han, Lei Ji, Shu Wang, Timothy Baldwin, and Haonan Li. Toolgen: Unified tool retrieval and calling via gencration. In The Thirteenth International Conference on Learning Representations, 2025. URL
net wtt.:/ooooooon......... XLMAMmowaY
Weiyun Wang, Yiming Ren, Haowen Luo, Tiantong Li, Chenxiang Yan, Zhe Chen, Wenhai Wang, Qingyun Li, Lewei Lu, Xizhou Zhu, al. The all-seeing project v2: Towards general relation comprehension of the open world. In European Conference on Computer Vision, pp. 471 471-490. Springer, 2024a.
Zhecan Wang, Garrett Bingham, Adams Wei Yu, Quoe V Le, Thang Luong, and Golnaz Ghiasi. Haloquest: A visual hallucination dataset for advancing multimodal reasoning. In European Conference on Computer Vision, pp. 288- 304. Springer, 2024b.
Fangyun Wei, Jinjing Zhao, Kun Yan, Hongyang Zhang, and Chang Xu. A large-scale benchmark for referring expression comprehension in the Imm era. Advances in Neural Information Processing Systems, 37:69566-69587 2024.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thougg prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24883 2022.
Penghao Wu and Saining Xie. V Guided visual search as a core mechanism in multimodal 1lms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13084-13094, 2024.
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma. Chengyue Wu, Bingxuan Wang, et al. Deepseek-v12: Mixture-of-expers vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412.10302,, 2024.
Linhui Xiao, Xiaoshan Yang, Fang Peng, Ming Yan, Yaowei Wang, and Changsheng Xu. Clip-vg: Self-paced curriculum adapting of clip for visual grounding. IEEE Transactions on Multimedia, 26:4334 4347, 2023.
Linhui Xiao, Xiaoshan Yang, Xiangyuan Lan, Yaowei Wang, and Changsheng Xu. Towards visual grounding: A survey. arXiv preprint arXiv:2412.20200 2024.
Yunqiu Xu, Linchao Zhu, and Yi Yang. Mc-bench: A benchmark for multi-context visual grounding in the era of milms. arXiv preprint arXiv:2410.12332 2024.
Li Yang, Yan Xu, Chunfeng Yuan, Wei Liu, Bing Li, and Weiming Hu. Improving visual grounding with visual-linguistic verification and iterative reasoning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 1499-9508 2022.
16

---

Published as a conference paper ICLR 2026
Ruilin Yao, Shengwu Xiong, Yichen Zhao, and Yi Rong. Visual grounding with multi-modal conditional adaptation. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 3877-3886 2024.
Ruilin Yao, Yi Rong, Tianyu Zou, Bo Zhang. Jian Li, Shengwu Xiong, and Shili Xiong. Map: Parameter- efficient tuning for referring expression comprehension via multi-modal adaptive positional encoding. In 2025.
Proceedings of the 33rd ACM International Conference Multimedia, pp.
Haoxuan You, Haotian Zhang, Zhe Gan, Xianzhi Du, Bowen Zhang, Zirui Wang, Liangliang Cao, Shih-Fu Chang, and Yinfei Yang. Ferret: Refer and ground anything any anywhere at any granularity. arXiv preprint 2023.
Haoxuan You, Haotian Zhang. Zhe Gan, Xianzhi Du, Bowen Zhang, Zirui Wang, Liangliang Cao, Shih-Fu Chang, and Yinfei Yang. Ferret: Refer and ground anything anywhere any granularity. In The Twelfth International Conference on Learning Representations, 2024. URL https: http:///pppeevvvveeem ew. net./
Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the association for computational linguistics, 2:67-78, 2014.
Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expressions. In European Conference on Computer Vision, pp. 69-85. Springer, 2016.
Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Yongliang Shen, Kan Ren, Dongsheng Li, and Deqing Yang. Easytool: Enhancing Ilm-based agents with concise tool instruction. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume Long Papers), pp. 951-972, 2025.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9556-9566 2024.
Yuhang Zang, Wei Li, Jun Han, Kaiyang Zhou, and Chen Change Loy. Contextual object detection with multimodal large language models. International Journal of Computer Vision, 133(2):825 843, 2025.
Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, and Filip Ilievski. Milms know where to look: Training- free perception of small visual details with multimodal 1lms. arXiv preprint arXiv:2502.17422, 2025a.
Wanyue Zhang, Yibin Huang. Yangbin Xu, JingJing Huang, Helu Zhi, Shuo Ren, Wang Xu, and Jiajun Zhang. Why do milms struggle with spatial understanding? systematic analysis from data to architecture. arXiv preprint arXivv2509.02359, 2025b.
Yikai Zhang, Siyu Yuan, Caiyu Hu, Kyle Richardson, Yanghua Xiao, and Jiangjie Chen. Timearena: Shaping efficient multitasking language agents in time-aware simulation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3894-3916 2024a.
Zeyu Zhang, Quanyu Dai, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey on the memory mechanism of large language model-based agents. ACM Transactions on Information Systems, 43(6):1-47 2025c.
17

---

Published as conference paper ICLR 2026
Zhehao Zhang, Ryan Rossi, Tong Yu, Franck Dernoncourt, Ruiyi Zhang, Jiuxiang Gu, Sungchul Kim, Xiang Chen, Zichao Wang, and Nedim Lipka. Vipact: Visual-perception enhancement via specialized vlm agent collaboration and tool-use. arXiv preprint arXiv:2410. 2024b.
Zicheng Zhang, Haoning Wu, Erli Zhang. Guangtao Zhai, and Weisi Lin. Q-bench: A benchmark for multi-modal foundation models low-level vision from single images to pairs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024c.
Chuyang Zhao, YuXin Song, Junru Chen, Kang Rong, Haocheng Feng, Gang Zhang, Shufan Ji, Jingdong Wang, Errui Ding. and Yifan Sun. Octopus: A multi-modal Ilm with parallel recognition and sequential understanding. Advances in Neural Information Processing Systems, 37:90009-90029, 2024.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504. /0479, 2025.
Zhuofan Zong, Bingqi Ma, Dazhong Shen, Guanglu Song, Hao Shao, Dongzhi Jiang, Hongsheng Li. and Yu Liu. Mova: Adapting mixture vision experts to multimodal context. Advances in Neural Information Processing Systems, 57:103305-103333 2024.
18

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 RELATED WORK
A.1.1 BENCHMARKS FOR VISUAL CAPABILITY OF MLLMS
The capability of Visual Perception, Understanding and Reasoning is a foundational aspect understanding benchmarks, which involves the ability recognize and localize multiple objects, interpret various visual elements with complex emotional or implicit cues and summarize visual information for feedback and decision making (Li al. 2024b). Specifically, Perception in MILLMs involves the classification, detection of basic visual objects (e.g., dog, cat) and attributes (e.g., color, lighting). These low-level perceptual capabilities are crucial for various applications, including recognition systems (Zhao et al., 2024) and visual quality enhancement (Zhagg et al. 2024c). Understanding represents a sophisticated level of image understanding that focuses on the detailed and nuanced aspects of visual content. It includes recognizing and interpreting the visual- lisual-inguuistic concepts, such as text recognition (OCRBench (Liu et al., 2024d)), Visual Grounding (RefCOCO (Yu et al., 2016), FineCops-Ret (Liu et al.) (024a), HC-RefLoCo (Wei al. (024g) and Referring Expression Generation (Visual Genome) (Krishna al. 2017), which refers to the model's ability to accurately link visual elements with corresponding textual descriptions. Although tasks at this level begin to involve visual and textual alignment, they still do not require reasoning external knowledge. For higher-order capability, reasoning in MLLMs involves advanced event understanding and deep meaning extraction from multimodal data. These capabilities include interpreting and responding to complex emotional cues across multiple modalities (Cheng et al.) 2024), deriving subtle implicit meanings from visual and contextual information (Liu et 20023a), and range of other competencies, including knowledge acquisition, language generation, spatial awareness, and cultural context integration (Rachabatuni et al. 2024).
A.1.2 REASONING CAPABILITY OF MLLMS
MLLMs have demonstrated remarkable reasoning capabilities, largely facilitated by test-time scaling (Dong ct al. [022:] Wei et al. (2022), which allows feeding prompted samples and context. This capability has been further enhanced by chain-of-thowe of-thoutgh (CoT)) prompting (Wei et al. 2022), which enables L.MSs to generate coherent intermediate reasoning steps toward the final answer. Previous studies have shown that LLMSs benefit from manually written demonstrations as well as zero-shot prompting outputs. However, due to the domain gap between various modalities, the current reasoning capability of MLLMs in the complex real-world environment is still limited. To address this limitation, researchers have focused on enhancing the reasoning capability of MLLMs in both the training and prompting paradigms. Flamingo (Alayrac al. 2022 bridges the gap between these two modalities by pre- pre-training on interleaved visual and textual data. Some other works, such as Shikra (Chen et al., 2023b) and Ferret (You al. 2023), leverage visual grounding data (Xiao) let al.]| 2004 Yao et al., 2024) to achieve fine-grained vision-language alignment. Furthermore, recent studies have also demonstrated that augmenting computing resources during the testing phase (test-time scaling) can enhance the reasoning capabilities of LLMs (Jacch et al. 2024). More specifically, Prompt-based Reasoning Meta-Systems (PRMS) can be employed to guide LLMs evaluating and filtering intermediate "thinking" processes (Snell et al. 2024 This encourages the generation of more sophisticated reasoning steps during testing, ultimately leading to improved reasoning accuracy. Beyond that, some methods employ the external knowledge to focus on important visual details, like V* (Wu & Xie 2024). Marvel (Jiang et al. 2024). and ICAL (Sarch et al. 2024). collecting a series of visual reasoning steps as training data. More recently, with the emergence of DeepSeek-RI (Guo al. 2025) demonstrating strong potential in LLM reasoning, research efforts have begun to explore reasoning-cente centric models and R RI-style reinforcement learning strategies for understanding complex visual scenes and tasks. These studies (Huang et al., 2025 Shen al. 2025a Liu jet al. (225c) particularly emphasize the long-chain reasoning capabilities within MILMMs, aiming to enhance their performance in handling intricate visual-linguisti reasoning challenges.
19

---

Published as a conference paper ICLR 2026
A.1.3 LLM-BASED AGENTIC REASONING
The rapid progress of large language models (Achiam et al. 2023: Bai et al. 2025) has sparked significant interest in building autonomous agents capable of solving complex, multi-step reasoning tasks. Leveraging the strong chain-of-thought (Co')) abilities of modern LLMs (Wei et al. 2022), these systems typically decompose a complex problem into sequence of structured subtasks, invoke intermediate deliberation, and integrate the resulting insights produce a final answer (Gupta & Kembhavi [023] Chen et al.] [2023a)
Recent developments in LLM-based autonomous agents highlight the importance of planning (Huang et al.
2024; Zhang al. 2024a). tool usage (Yuan et al., 2025), memory (Zhang et al.] (025c), and persona (Chen et al. 2024a). In parallel, multi-agent frameworks such as MetaGIT (Hong et al. 2023). AgentVerse (Chen et al., 2023c), etc., demonstrate strong performance by orchestrating multiple interacting agents. often instantiated as distinct roles with specialized responsibilities. Despite their success, these systems rely heavily on manually designed personas, fixed role hierarchies, or hand-crafted coordination rules, which limits their flexibility and generalization across tasks and domains. Moreover, the dependence on external scaffolding or pre-specified agent behaviors often restricts the model's ability to adaptively adjust its internal reasoning pathway. To address these limitations, SMEC introduces a self-driven agentic reasoning mechanism that automatically generates diverse experts, elicits their reasoning, filters redundant experts, and synthesizes their perspectives into final consensus, all within the model's own language-naln native inference loop.
A.2 SPECIFIC DEFINATION OF DIFFERENT TASKS
Object Counting (OC): Estimating the number of object instances described by a free-form expression, often under complex conditions like occlusion, scale variation, or clutter.
Object Detection (OD): Localizing objects within an image by generating bounding boxes paired with corresponding class labels. In order to better match the real-life scenarios and practical applications, we construct more than 500 fine-grained object categories based on natural language.
Object Existence Determination (OE): Determining whether particular object, which described by a detailed expression, exists in the image without requiring spatial localization.
Relation Extraction (RE): Identifying semantic relationships (e.g., "holding", "next to". "wearing") between pairs of objects facilitate structured scene understanding. And we added questions about the objects that do not exist in the images to evaluate model's s ability to suppress hallucinations.
Visual Grounding (VG):
Localizing an image region that corresponds to a natural language expression, linking linguistic references to fine-gran fine-grained visual content.
Region-wise OCR (OCR): Recognizing and transcribing text within a region, which specified by coordinates or description, facilitating fine-grained interleaved image-text understanding.
Spatial Relationship Comprehension (SRC): Understanding geometric relationships (e.g., "above" and "to the left front to"") between objects within diverse 3D views, supporting visual-spatia reasoning. Compared to some rudimentary or synthetic spatial understanding datasets (Johnson et al. 2017 7, Li et al., [023 Liu et al., 2023b), our data is more realistic in emphasizing spatial location understanding under real-world scenarios as well as 2D images acquired by cameras or cell phones.
Scene Knowledge Inference (SKI): Inferring high-level semantic and functional information about the scene or making decision based on the visual contents, incorporating context, commonsense knowledge, and visual cues beyond explicit visual entities. Compared to the regular visual reasoning dataset, Lens additionally distinguish between "thought paths" and "final answers", differentiated by the < think > token, aiming to provide finer- fnnrneg information for potential test-time scaling tests and R -tyyle reinforcement learning.
20

---

Published as conference paper ICLR 2026
Object Detection á Visual Grounding
theelett pirte 1254 57734) (31. 101) Object decriottoo The the avertr
decription
conter"
Object decrronce oot lected betweer SOME"
90 (PAAT) 307000
1o porte 1236 623 142 7311
VD poens 1905 2002 foet 30 430
poltte 1269 695855
Object daciitioc Maaniidee oom
diject 1227 400 pa2200 Serlog
706.4g
Object Counting
Object Existence Determination
Relation Extraction
seas in this pictare)
One
C How mayy whire cure pottone
white wits sen pickee 3t' on
mearie skited
O. 2t than deyrr disk eupputevr
the pethore
spright
Arsser porco thepicaen.
Yea.
Pue de perune Mosse
Hore
doryen mectter
FIVE GUYS
At Tau The phrase the everles
The Fihrr srove
hoddng bovuhhppdd object neor har appeances Fiee bureer unitie mosovennle
aher
Region-wise OCR
Spotial Relationship Comprehension
rndig
Frowee rectiva
Prose the bockted ceneeeiiiee
mectly
right, dinecty left sirethy got
ing ner the
"6 offiee devae''
chea ractanatio
too Ohi lorg face
any from
drrettin the
Cormer Spicill
ranh abe
dees fere diaulty
fodt
luck and right.
left receed stop
what
drich giftt
Foant
worning this picture
Scene Knowledge Inference
If want is Dh Ardeon light for trus dooldd she
in wiic tee Sicheon Horen
Aldi the
not chimpiing
other
shaue micce til lii ai iilll"
iopiree un thes contuct
shuingl the soand
snreeed of dhue of aeee pettiee hes let feters
Thardfor based the ciucly
pur better juis iteemet
badket
Therdtore, e the tonal cont ow in stooreed moet potrroas and bother
A. pittes takee
Figure 7: Lens covers a wide range of images and annotations, from fine- mne-grained recognition and spatial localization complex reasoning over extended thought processes. Notably, each image annotated with labels corresponding to all subtasks concurrenlly, enabling comprehensive evaluation.
21

---

Published as a conference paper ICLR 2026
A.3 QUALITY CONTROL PROCESS
In addition to the annotations, diversified measures were taken to enrich the content of data samples and ensure their quality. Specifically, we implemented a multi-faceted quality control process. Beyond a two-step data cleaning protocol, we also enriched each image with supplementary metadata to facilitate traceability and contextual analysis.
First, manually performed two-stage data cleaning process. In the initial stage, reviewed and eliminated suspected duplicate images. The second stage involved distributing the problems among co- authors for meticulous format and typo checking, ensuring all annotations adhered to standardized format.
To further validate the quality and consistency of our annotations, we performed an additional two-step verification process. This included both manual and machine-assisted assisted checks. The entire dataset cross- verified by both an independent team of annotators and the Qwen2.5-Ve VL 72B open-source model. For machine validation, we input the original image, question, and answer into the MLLM. Cases flagged as invalid by the model were isolated for manual re-evaluation by a separate team of annotators. For object detection and visual grounding tasks, we directly visualized the annotations on the images, enabling human evaluators to assess the validity of the bounding boxes.
Additionally, we enriched each image with supplementary metadata. We included a pseudonymized Annotator ID to allow for aunotator-specifie quality tracking while preserving privacy. The Time of Online Publication and Scene Category were also labeled to facilitate temporal studies, filter outdated content, and organize the dataset by scene. Finally, ambiguous images that consistently resulted in low inter-annotator agreement were manually filtered out to ensure high-quality final dataset. Some cleaned representative examples are visualized in Figure 7
A.4 DATA PRIVACY PROTECTION AND COPYRIGHT STATEMENT
Our protocol for handling potentially sensitive information was a multi-stage process designed to be as thorough as possible:
Automated Pre-screening: As an initial step, we used automated tools (e.g., face detection models], docTR perform preliminary scan of the collected images. This
configured to flag images with high probability of containing human faces or dense blocks of text that might constitute personally identifiable information.
Every image, including those not flagged by the automated scan, was then subjected to a thorough manual review by team of over 20 trained human annotators. Annotators received specific training and a detailed guide on identifying a wide range of sensitive data, including but not limited to: Visible and recognizable faces; Full names, usernames, or contact information; License plates, street addresses, or other specific location markers; Private documents or screens displaying personal data.
Comprehensive Manual Review:
Sensitive Information Exclusion: Images containing sensitive personal information were either excluded or processed to blur or mask sensitive regions, to mitigate privacy risks. Based on the manual review, if an image contained sensitive information, one of two actions was taken as mentioned in the paper:
Processing: If the sensitive information was incidental the image's main content, we applied irreversible blurring or masking to the specific region.
Exclusion: If the sensitive information was central the image and could not be adequately anonymized without destroying the scene's context, the image was entirely excluded from the final dataset.
https//gttub..co//miineeddddd
22

---

Published as conference paper ICLR 2026
Station
Scanic spot
Restaurent
96
0/5
0.4
94
24
033
0.3
0.3
92
022
031
0.1
Playground
Living
Library
Kitchen
0.5
0.5
33
235
0.4
94
24
0.3
83
0.3
02
0.
61
Classroom
Bedroom
Belhrrom
Airport
85
0.5
0.3
02
02
0.2
5.1
0.1
Light
Contrast
Colorfulness
Blur
51
Figure 8: The normalized probability distributions of low-level attributes from different scenes. Scenes with flat peaks show more diversity, while those with sharp peaks have similar features.
To ensure consistency and quality, a final audit was conducted by a subset of the paper's authors. This team reviewed random sample of the approved images and 100% of the processed (blurred/masked) images to verify that our privacy protocol was correctly and consistently applied.
Final Verification:
This multi-stage, human-centric approach ensures that the images in the Lens dataset comply with platform policies and respect individual privacy. The Lens dataset is constructed solely for non-commercial academic research purposes and does not transfer copyright ownership of the original images. By accessing or using the Lens dataset, users agree to comply with all applicable copyright laws, platform policies, and research-uss restrictions.
A.5 LOW-LEVEL FEATURE ANALYSIS OF IMAGES FROM DIFFERENT SCENES
We counted five low-level visual attributes, including lighting, contrast, color, blur, and spatial information (SI), to assess the statistical difference between different scenes. As shown in Figure & the normalized probability density curves of low-level visual attributes across different scenes are consistent with human perceptual preferences. Scenes with regulated lighting conditions (e.g., classrooms, airports, and stations) demonstrate sharp peaks near x s 0 in the illumination curves (density > 0.5), indicating constrained variations in brightness. contrast, domestic environments (e.g., living rooms, bedrooms, and kitchens) display broader illumination distributions. suggesting more diverse and adaptive light sources. Furthermore, functional scenes such as bedrooms, bathrooms, and kitchens exhibit sharp, concentrated peaks color distributions (peak density 0.5), implying greater structural regularity in specific visual attributes.
A.6 FINE-GRAINED EVALUATION OF VG
We conducted a fine- rraigra evaluation of a diverse set of models on the Visual Grounding (VG) task You al. (2024): Bai et al. (2025) ; Yao et al. (2025) Zhu et al. (2025); Xiao al. 2023), categorizing them
23

---

Published as conference paper ICLR 2026
into two groups: traditional predictive multimodal models and generative multimodal large language models (MLLMs). The results, summarized in Table 4 reveal several key insights into the current state of visual grounding capabilities.
A.6.1 COMPARISON OF MODEL CATEGORIES
The results clearly indicate significant performance gap between the two model categories. The top- performing MILMs,, specifically Qwen2.5-V -LL VL-7B and Qwen2.5-VL-3225 demonstrate superior performance across all metrics, with accuracy of 46.94% and 48.47% at ToU respectively. This performance is substantially higher than the top predictive model, G-DINO, which achieves 37.05% at the same metric. This finding suggests that the generative and in-context learning capabilities of modern MLLMs provide a substantial advantage the complex VG task, enabling them to better understand nuanced linguistic instructions and ground them accurately in the visual space.
A.6.2 PERFORMANCE DIFFERENT SCALES
A multi-scale analysis, measured by accuracy on small (ACC.), medium (ACC and large (ACC)) objects, provides more granular view of each model's strengths and weaknesses. Both traditional and generative models exhibit a similar trend: performance consistently improves with the size the target object. This is a common challenge in visual grounding and object detection, as localizing and grounding small objects remains difficult.
Traditional Models: Among the traditional models, G-DINO demonstrates more balanced performance across scales, achieving 24.87% on small objects and 52.32% on large objects. In contrast, models like VLTVG and SimVG struggle significantly with small objects, with accuracies of 0.00% and 0.01 respectively, but show strong performance on large objects (29.70% and 45.20%).
While MILMs also struggle with small objects. their performance is notably better than most traditional models. Qwen2.5-VL-333 and Qwen2.5-V2 5-LL-7B achieve high accuracy medium and large objects, with their ACC,, and ACC, scores reaching 55.04% and 61.48% (for (wen2.5-VIL-32B) and 54. 12% and 60.24% (for Qwen2.5-VL-777 attributed their powerful visual backbones and advanced language understanding capabilities, which help them better contextualize the target within the scene.
Generative MLLMs:
respectively. The strong performance on larger objects may be
A.6.3 IMPACT OF MODEL ARCHITECTURE AND SIZE
Our results also highlight the importance model architecture and size. The Qwen2.5-VI. family of models, with its impressive performance, benefits from powerful visual encoder (FE-ViT) and sophisticated Qwen2.5 language backbone. Similarly, the Intern series shows a clear scaling effect, where performance most metrics improves as the model size increases from 2B 14B. The performance of the 38B variant is slightly lower than the 14B variant due to its different visual backbone. This trend, consistent with findings in large language models, suggests that scaling up both visual and linguistic components is promising direction for future research in visual grounding.
A.7 MORE SYNERGISTIC EFFECTS ANALYSIS
We further notice that Figure 6 (a) shows OC-OCR correlation (0.77) >> OC-OE (0.46). This contradicts intuition- object counting (OC) should align more naturally with existence checks (OE) than OCR. We attribute this two factors. First, the Object Existence (OE) task is simple binary classification: either object is present or it is not. It requires a model to make a broad, scene-level assessment. In contrast, Object Counting (OC) is more demanding task that requires fine-grained localization of individual objects, followed
24

---

Published as conference paper ICLR 2026
Visual
Linguistic Backbone
Accuracy @ IoU
Scale-wise Accuracy
Method
Backbone
80.5
30.66
00..7
690.8
30.9
ACC,
ACCm
ACCi
Methods based ou predictive multimodal models:
TransVG Deng et
2021
RN101
BERT-B
7.57
6.29
4.40
1.69
0.01
2.01
23.64
VLTVG Lang el
2022
RN101
BERT-B
11.04
9.60
7.75
5.33
1.99
0.00
2.80
29.70
MMCA
et ml.
2024
RNIO1
BERT-B
10.92
9.45
7.90
5.64
2.22
0.03
2.79
29.31
CLIP-VG
CLIF-B
CLIP-B
7.57
6.29
1.69
0.01
2.01
23.64
EEVG Chen al
80240
VIT-B/16
BERT-B
9.27
5.78
2.51
0.48
0.05
0.01
0.98
12
SimVO Der
2024
BETT-3
BETT-3
16.46
13.90
11.12
7.44
2.70
0.01
3,10
45.20
G-DINO Lau
2024b
Swin L
BERT-B
37.05
33.92
29.20
22.57
11.36
24.87
37.54
52.32
Methods based generative multimodal large language models:
Groma-7B
Ma al. 2024
DINOv2-L Multi-expert
Vicuna
33.59
29.95
25.
18.73
8.52
11.58
33,91
58.59
Mova-7B Zone 21 di 2004
Vicuna
20.44
13.10
5.98
1.09
0.13
5.06
15.97
40.36
Ferret 7B
20243
CLIP-L.
Vicuna
23.26
18.97
13.95
7.61
1.84
1.85
19.49
54.64
Ferre- ( of ai. 2024
CLIP-L
Vicuna
24.20
19.81
14.41
8.12
2.05
2.26
20.42
56.31
InternVL33
InternViT-O.3B
Qwen2.5
5.10
2.85
3.46
19.34
Zhu VLtemVL33-BB
2025
Intern VTT-0.3B
Qwen2.5
17.54
13.23
8.89
4.94
1.60
3.23
15,36
35.21
InternVL3-145 7Zhu et 20225
InteraViT-O.33 0.BB
Qwen2.5
29.53
23.98
17.25
10.05
3.00
4.58
27.80
57.07
3.. 17hu
0125
Intern InternViT.6BB
Qwen2.5
27.85
21.42
15.00
8.23
2.37
4.91
25.81
53.56
VLM-RI-3B
(Shen el m.R
4025g
FE-VIT
Qwen2.5
23.79
19.91
15.65
10.53
4.31
8.15
22.84
40.94
Qwen2.5
et
Qwen2.5
37.92
29.33
Qwea2.5-V1 VL-72.5-VL-7B
2015
FE-VIT
Qwen2.5
46.94
39.39
29.94
18.38
6.26
28.87
54.12
60.24
Qwen2.5 32-335
[Bai al
2022
FE
Qwen2.5
48.47
40.66
30.78
19.15
6.63
30.93
55.04
61.48
Table
Multi-scale evaluation results.
by an enumeration step. A model can be highly proficient at binary existence check without possessing the precise localization and counting skills needed for the OC task. This fundamental difference in cognitive demand limits the correlation between the two. Second, the high correlation between Object Counting (OC) and OCR not coincidental. Both tasks rely on a critical shared capability: fine-grained localization. To perform well on the OC task, model must accurately identify and localize each instance of an object to count it. Similarly, to perform region-wise OCR, the model must first precisely locate the bounding box of the text before reading it. The strong correlation suggests that the ability to perform precise object localization is a dominant factor in model's success on both tasks, thus strengthening their relationship despite their different end goals.
To further test the synergy between different tasks, we conducted a experiment with (ive22.5VVL-77 on a sampled subset of Lens. Specifically, when testing the SKI task, we fed the VQA duestion-answer pairs of other tasks into the model as context along with the question, and asked it to return the answer (refer to Appendix A. for the prompt template ps. where provide an example based the OCR task). The test results are shown in Table and reveal several noteworthy patterns. First, incorporating OCR information yields substantial performance gain (from 39.80% to 41.36%), indicating that understanding scene text helps the model solve some reasoning tasks. Second, although some tasks-Such as OE and OC-exhibit limited or even negative effects when introduced individually, their combination with OCR consistently boosts performance. This may indicate that auxiliary perceptual signals, while insufficient on their own, can enhance reasoning when mediated through textual understanding. Furthermore, the observed synergistic effects resonate with the design philosophy of our proposed Self-Driven Multi-Expert Collaborative (SMEC) framework. We argue that complex multimodal reasoning cannot be achieved by isolated competencies alone, but requires a structured mechanism to coordinate heterogeneous sources of evidence.
A.8 ANALYSIS OF INPUT RESOLUTION
We conducted a detailed analysis to understand the impact of varying input resolutions on model performance. The results, summarized in Table reveal several key insights.
25

---

Published as conference paper ICLR 2026
OCR RE OE OC
Performance
39.80
41.36
38.03
39.23
39.52
40.72
41.45
40.73
41.93
40.79
41.90
Table Testing the synergistic effects of different tasks on Scene Knowledge Inference (SKI).
Visual Backbone
Linguistic Backbone
Accuracy @ Input Resolution
640 x 640
960 960
1280 x 1280
1600 x 1600
1ntem Zhu et al.
2025
Inter
Qwen2.5
40.97
41.53
41.90
40.60
Vntemm11.3999
V.T-0.3
Internl.M3-88
46.65
In.em 14B
Zhuu
2025
Intem V
Qwen2.5
50.28
51.15
51.58
51.17
IntemV1133 38B
(Zhu
A1255
Intern ViT-6B
Qwen2.5
50.88
50.97
49.98
$1.08
Dwen2.5-VL-33-
FE-VIT
Qwen2.5
40.08
40.48
40.58
Qwen2.5 Qw
VL
[5ar el
2125
FE- VIT
Qwen2.5
46.52
47.41
48.13
48.11
Qwen2.5-VL-32BB
2025
FE-VIT
Qwen2.5
53.72
54.37
54.10
54.09
GLM-4.
[Hong 2025
AlMy2-Huge
GLM-4-0414
42.35
42.86
43.28
43.73
V.Mii..V-Tiinnin---8 (Hong a.. 2025
AlMv2-Huge
GLM-4-0414
48.77
50.78
51.32
51.13
Table 6: Benchmark results across varying input resolutions.
General Trend (Performance Improves with Resolution): For most models, performance generally improves as the imput resolution increases. This trend is evident models such as Qwen2.5-VL-7B. which shows steady increase in accuracy from 46.52% at 640 640 to 48.13% 1280x1280 Similarly, GLM-4.1V Thinking-999 improves from 48.77% to 51.32% over the same range. This is expected, as higher resolutions provide more visual detail, which is particularly beneficial for complex visual grounding and reasoning tasks that require fine-grained perception.
The Point of Diminishing Returns: However, the results also suggest point of diminishing returns. For many models, the performance gain from increasing the resolution beyond 1280x 1280 is minimal, and in some cases, performance slightly decreases. For example, Intern VL3- 14B peaks 51.58% at 1280x 1280 and then slightly drops to 51.17% at 1600x1600. Similarly, D0wen2.5-V-775 performance plateaus at x280x 1280. This phenomenon could attributed to several factors, including the model's 's architecture, which may not be fully optimized to handle the extra high-resolution information, or the fact that the added detail does not contribute meaningfully to solving the task.
Interestingly, some models, like Intern VI.3-2B, show less sensitivity to resolution changes, with its performance remaining relatively stable across all resolutions. In contrast, models such as 1.MM44..VThinking--290 and Qwen2.5-VL-33.
Model-Specific Variations:
demonstrate more pronounced performance improvement with higher resolutions, indicating that their architectures are more capable leveraging the extra visual information. This suggests that the optimal input resolution is not one-size-fitss fits-all solution and depends heavily on the model's architecture and design.
A.9 QUALITATIVE ERROR ANALYSIS
To better illustrate common failure patterns and the underlying limitations of current Multimodal Large Language Models, conduct qualitative analysis of representative еπтог cases across different task levels. Following the structure of our benchmark, we group the visualizations into two categories: (1) VQA-style tasks and (2) Localization tasks, including detection and visual grounding, as shown in Figure
For Perception Understanding Tasks, they primarily require directly aligning visual content with textual queries. While modern MILMs achieve reasonably high accuracy, their errors frequently stem from low-level
26

---

Published as a conference paper ICLR 2026
Hou many peop ane ridiig Sicylles
Qu Hoe many le wene thenc the pictorn irckuding the Plezae sriwar with number.
Qi Hup thi imsg boun maranall
Wring Answer (Quend B-V1) Nr.
Right
@) In the distance por
saleed
bresktner His writy two basf huse and one aca roll nuch monky should tranaforr
of att than mamerss?
Phaass
Wrong Answer (Ooverd 6-VL))
(Queed
Ussed on the monu prices the image: the best
Wrong frum people riding blyylls picture
The inturne
The oach sccilion
longer than the seigth of
tone
Wrong Anower (Gwven 5-V131
and sne scelion rall is yoar 2 yuar 9.5
should traesfer
you
dirnance atonore to greateen rathore
Right Arawert
Right Answer:
Right then. about 2:3 meters.
Right 7.5
acciiin rol 15
The with clamaa to the subway door
third beoksielff From the left in the distorce
the dak
light absoured by contraction sinn A boy with hesophancs
Figure 9: Failure cases for various tasks. Incorrect predictions and labels are indicated by blue and red, respectively
perceptual limitations such small objects, occlusion, distant subjects, and sensitivity resolution. Several examples clearly illustrate this issue- for instance, miscounting the number of cyclists in a street scene, or failing detect objects like straw on desk road light partially obscured by signage. For more complex reasoning tasks, we observe two major classes of systematic errors.
The model often demonstrates sound logical reasoning but bases its inference on incorrect or incomplete visual extraction. For example, in price-computation task, although the model performs the arithmetic correctly, it misreads the menu price of the beef bun, leading to a wrong total (5.5 instead of 7.5 yuan). This reveals persistent bottleneck where high-level reasoning is constrained by low-level perception, especially OCR and fine-grained attribute recognition. Such cases motivate the need for agent- based or sxpabt-collaborraive pipelines, which can iteratively refine visual cues or invoke specialized perception experts. They also highlight the importance of dynamic zoom-in strategies for capturing critical
Correct reasoning, incorrect perception:
but small textual visual details. 2024
(Wu& & Xie 2024 Zhang et al. 2025a Shen et al. 2025b Shao et al.
Spatial and physical reasoning deficits: A second recurring failure mode involves questions requiring geometric or physical commonsense. Models frequently struggle with tasks that implicitly require depth understanding, object-scale priors, spatial metric reasoning. For instance, the model incorrectly concludes that two cyclists are more than meters apart by comparing them to the length of a blue SUV, even though the correct distance is only about 2-3 meters. Likewise, it fails to judge whether an image is mirrored due
27

---

Published as a conference paper ICLR 2026
misunderstandinn spatial layout cues. These issues echo recent findings showing that current MLLMS still
lack robust spatial grounding and physical commonsense. (Zhang et al. 2025b Azzolini et al. 2025)
A.10 A FORMAL DESCRIPTION OF SMEC
As shown in Algorithm I A key advantage of SMEC is that it does not depend on fixed, hand-crafted prompts. Instead, the prompts and expert descriptions are self-generated by the model based on the given visual input and question. During each iteration, the model adaptively refines generation process when redundancy or low-quality information is detected. This adaptive design means that SMEC not tied to a specific phrasing or a predefined set experts, but can flexibly adjust to different problems and question types. As a result, our method is more robust than approaches that rely heavily on manually designed prompts, since the "experts" in SMEC emerge dynamically from the model itself rather than being externlly imposed.
expert descriptions and updates the
Algorithm 1 Self-driven Multi-Expert Generation & Collaboration
Initialization: Based Instruction MILMM 0, Question q, Meta Generation Prompt
Pg. Inspection prompt p.. Collaboration Prompt Pe, Description Set D = 0. Maximum Answer Set A 0, Iterations No.
1: ag = (4)), A = dp
# Initial answer for question.
2: for = 1,2, ................................ ., .2 do
3: if = then
4:
d² 0(pg. q, Ao)
# Initial expert description.
5: else
6: d² 0(p D. Ad)
# New description based existing information.
7:
end if
8:
if Opi, D,dg) Retain then
9:
D DUd
# Checking process. # New answer from the expert perspective.
10:
Ot = 0(g, dz); AU
11: else
12:
Pg = 0(q. d' Pg. Pg. D) # Update generation prompt while repeat descriptions.
13:
end if
14:
end for
15:
afinal O(g. A.pss D)
Summarize the final answer.
A.11 HUMAN PREFERENCE
To ensure the verifiability of our evaluation, particularly for open-ended reasoning tasks, we employed a large language model (LLM) as an automatic grader. To mitigate the concern regarding potential LLM hallucinations or failure to detect nuanced mistakes, it is noted that the LLM grader (e.g., GLM4-flash) is used to compare the model-geeerrted responses against our pre-existing, human-annotater answers, ensuring that the ground truth remains anchored in high-quality human data. We also conducted a human preference analysis on a representative and complex task SKI with subset of the dataset, aiming to provide a gold standard against which to measure the reliability of judgement model. We recruited a separate team of ten expert annotators who were not involved the original data collection to manually evaluate the accuracy of the model's 's answers compared to the labeled answers, as shown in Table In this setup, Actual Positive (AP) and Actual Negative (AN) represent human judgments of correctness and incorrectness, respectively, while Test Positive and Test Negative indicate the LLM grader's corresponding evaluations.
The results, shown in Table demonstrate a strong alignment between human preference and LLM-based judgments across all evaluated models. For instance, in the case of Dwen2.5-VL's responses, when humans
28

---

Published as conference paper ICLR 2026
Cwwn225-VL-77
5---725-V-788
Gemin
InterniV13-88
InternVL3398
IndernVL3--8
2018
Qwinn2. Qwen1255M0-726 5--1-228
Quein2.5-V1-72 555--222
Qwwn2.5-V1-P2
ANY
SAN
InternMl3--000
Qwen2.5-V1-722
Gemini-2.55
Cemini-2.5-pro
0.87
Before 2624.09
0024009-1244.7
After 2025
Figure 10: Model accuracy on SKI task across temporal split.
labeled an answer correct (AP), the LLM grader also marked it as correct 97.13% of the time. Similarly, when humans judged an answer as incorrect (AN), the LLM grader agreed 96. 14% of the time. Comparable trends are observed for Intern VL3 and Gemini2.5-pra though with slightly larger gaps in negative cases. These findings suggest that the L.LM grader provides a highly reliable approximation of human judgment, especially for positive cases. Incorporating human validation thus not only confirms the feasibility of using LLMs as evaluators but also highlights their potential to scale evaluation consistently across large datasets while retaining strong alignment with expert human preference.
Qwen2.5-VL
Intern VL3
Gemini2.5-pro
Method
AP
AN
AP
AN
AP
AN
Test Positive
97.13%
2.87%
94.70%
5.30%
93.65%
6.35%
Test Negative
3.86%
96.14%
7.77%
92.23%
11.41%
88.59%
Table 7: Human evaluation for the models' responses.
A.12 TEMPORAL GENERALIZATIO PERFORMANCE
We visualize the accuracy the best few models on data from different time periods Figure 10 including Qwen2.5-VL (7B/72B), InternVL3 (9B/78B), and Gemini-2.5-PPr Notably, models released after late 2024, such as Intern VL3-78B and Qwen2.5-VL-72B, consistently outperform Gemini-2.5-pro contemporary data, although their overall performance lower than Gemini-2.5-prr The experimental results further supporting the observation that newer models tend generalize better new data distributions. This trend underscores the impact of scaling, instruction tuning, and exposure to temporally aligned data in enhancing multimodal reasoning performance.
A.13 MORE ON SMEC VS. BASELINE METHODS
To further highlight the advantage of proposed SMEC framework, we provide additional qualitative comparisons against baseline methods, including direct inference and ChatGPT-style single-pass reasoning. Figure showcases shoowcsssss representative challenging samples drawn from Lens, where baseline models tend produce either incomplete or overconfident predictions.
In these cases, ChatGPT and other baselines often failed for recurring reasons:
Over-Reliance on Surface Cues. Baselines typically produced answers anchored on the most salient visual elements, neglecting contextual or relational signals. For instance, when asked to infer spatial constraints
29

---

Published as a conference paper at ICLR 2026
Q: Is flee the 2nd flor??
Answer:
image denalie ondlyss the syege. your corrent facetion cindeed Floor Here's complite ansver odditionl explesotion
Analysls bey inforaation Image:
left:
5it flor Boko Fe reiesssnn (Depentuee Hall, this where passengers board thee pkres
Penylancees
and Flart Beks kotboi (Arrivl Halp This is ahere poier epere so otte deploiing
bore Some ( Miden C o Pie Lumpur Thet
Meden bijers (fiee Genden). these are creaa offer ng ccormmaaation, parking, and deing sprien. ahu sernce
Ian Plan Halling (hariin Andl)
Shani
Right
The sig or the right ckartly stahi Tuevel 2 and hay aroon point in direstons Thi ireediendy and soner that you the and Plaar
Flear, this ner of has trun mrnnpottition tranderrs
Ground Land dlonage
There ars yellow aigu on the around that any "E-HOELLENg" indistiin thar the ride-hal the woiting oraa Thene arear are usadly loathed lover floor (such pesteegers pacle the relovst
Thee shees seversl pedestrirss droging hoodes, farther cerf-rung ny that you nour the Arrial Hall os this where pestengers ofter
Expart's Addhiana Seformation:
(Tren Outyett
The exuert neached sama cone baned the digngge coriey that your cur mone location and flaar The expart andlyd clyse with vina continiing by thle
signs you pheto you
camently wnveel
5. Peal Carckeie
dinectory or the lett
"Loall 3. end sile fhor's is fuection is reatted the Anriel Hit
Specifioalyy you
dinagn clicrty shotee
Lend Deperture Hall
Revaed Arawor
Leail bue
ctatton, Faod gardes
Aseal the diynag the ye and the cammacting informrtie, your carvent licetton and Fleer The fullow by the specific horle for tive
And parple
stove
The alan an the right dinectly libeled "Land 3 and has crrze ehingg in sertan direction, farther conhimiig that you are on the 3nd floar.
intionss Lest floor the And
peoeetmmee There durvent inooiie the and floer
greael signs
lageng
chonscterehhee fieing heen
foor (Lvell 2/ Aras 2).
Phal anserr Vicoees())
betemaise
restrurent licated
Based in daa the image dctoileed expert analyta we can drav the fellwing concisaions
1. Key Analysic
Milayeii, Bagnel naat centoped care ca hure crane thise couttia, and the desor and onbrent festunal the lmage are highs yy canair ort with Malzytian cork ca hure derge Thes style peodor a Seuthest Asan contries ricuarlly by Meleydo end segaere pericslert
The
"KEl UAA "exit Maday,
Trdkeedks Aawrerr Maleyoio hes
velepentont cefés ond small remoumant citer. aaaund Fecture nuttrneet particulrrly featares lorge sindows diiutig anple returd light cracting bright confortbble ctreaphers. desion connec Southest Agan Addiardyy nattornel's Farenure (such es weeces table and chais with wright wan franc)) and or condiooon (ve imsuttedd) are con steel with Malaysid's
hueld
rasiine whene intarior othen
Paccbiitie::
Srcre: Allheugh brone die upealo Me andlle land area sparse pender en don't it sllw for more developed cafe culture than Makayis Therefeng, Brunc len prinary lorgugee lncona in, Haity Mally map be sooken some areas Jnceeoioo cafe caltune hends to more modem the
lish, iectwo whith dessn'
the
Aeabyia
daudd on endlyte by expert on Sourneat Agian cafe outorn and daice mones the remonnnn's is wooden tolks and thais, indunnnl styl pencont lights, and retra pesteccilge
Spoticl Losel Ateaspaane The Melaydo loyett, anple natural light
retoo pia perfectly compiement the dieats aftire Southezst
Ast, pariodarty Valoes
Direct Arsver (Iren chatget) Band on aord nEELUAA the out sie. thit rechrorett lacond either Irdesess, Brand, or Stegppree
lood nyicel ehorecterin Mellytioon colés. Compendon with Ofivr Southest Adic covetnes & sigeperioe cofee are geaore by more water serredeeed derign while The cefés way recopoeete ote Buddhust dasely
The
entoddet Heed Eastore
aleeents
Conchuees:
dased stave srbbss. thes reshuumnt mont lieey locoted Moleyiia The Make "NELUMM vey due and the nastament's iss decer and surroundigg are cbe
Maley and Indeverio leguages
jylly condetuer with Melaydian cord caltra.
Feal arsver Mebsee)Mebppa))
Figure 11: Comparison of our method and ChatGPT some difficult examples.
traffic rules in airport photo, ChatGPT tended to extrapolate directly from textual OCR cues, leading to plausible but incorrect answers.
Lack of Internal Deliberation. Without multi-perspectivn reasoning, baselines converged prematurely on a single hypothesis. This often caused brittle errors in scenarios requiring integration of textual, spatial, and commonsense evidence.
By contrast, SMEC decomposed the problem into multiple role-specific perspectives, such as a scene analyst, a spatial reasoner, and cultural or commonsense expert. These experts generated partially overlapping but complementary hypotheses, which were then screened for redundancy and synthesized into a consensus. In the traffic-sign example shown in Figure TT SMEC correctly filtered out spurious cues and converged on the right driving instruction, whereas ChatGPT remained uncertain or hallucinated unsupported details. Across difficult samples, two consistent patterns emerged:
30

---

Published as conference paper ICLR 2026
Error Correction through Redundancy Filtering. Even when some experts produced misleading interpre- tations, SMEC's screening stage effectively down-weighted unreliable reasoning chains, preventing error propagation.
Multi-Step Enrichment. Iterative expert collaboration yielded richer reasoning trajectories. allowing the framework to exploit synergies between visual grounding, OCR, and commonsense inference. This process systematically improved robustness to ambiguous noisy inputs.
Taken together, these results suggest that SMEC is not merely a test-time ensemble but a principled framework that encourages internal debate within single model. Unlike single- pass inference, SMEC operationalizes a language-native form of deliberation, enabling models to approximate the dynamics of human expert panels. As Figure IT iillustratss, this mechanism directly translates into more accurate and interpretable reasoning on complex multimodal questions.
A.14 PROMPTS FOR SYNERGY TESTING
Prompt P. with OCR question-answer pairs context
fquestion from SKI}
You can use the following facts to help you answer this question. Please note that they may not be relevant the question. Here are some factual questions and answers about this picture: {question from OCR): (answer from OCR)
A.15 PROMPTS OF SELF-DRIVEN MULTI-EXPERY COLLABORATIVE FRAMEWORK
Meta Generation Prompt Py
{guestion}
Here are your answers and those of some experts: {unswer)
Now you can create and work with multiple experts improve your answer. So, please describe in as much detail possible the different skills and focus you need from each expert.
We will provide each expert with the same information and queries. Each expert should have his or her specialization covering perception, understanding and reasoning, etc., you assign only one subtask to each expert to ensure a more refined answer. We will relay their responses to you in turn that you reorganize them into better answers. Please note that descriptions should be in the second person, e.g. You are XXX.
These the descriptions of the experts you have previously created for this task:
[description)
Therefore, do not create the same experts as above over and over again.
Now you can create a description for the new expert (please note that you can only describe one, not more than one the same time):
31

---

Published as conference paper ICLR 2026
Inspection Prompt Pe
{guestion}
We hired multiple experts to answer this question. Below is second person description of the experts we hired: (erisling descriplion)
We are now hiring a new expert to help better provide the information needed for the question as well respond to user queries. Here is second person description of the new expert: {description) Since there an additional cost to hiring new Expert, please evaluate the new Expert based on the following two criteria to decide whether or not retain them.
1. based on the new Expert's s description, determine they can effectively assist in answering the user's question provide the information needed for the question.
2. the new expert is not a duplicate of any existing expert.
The new expert must meet both of these criteria. either criterion met, they should be discarded. If retaining, please reply 'Retain'. If discarded, please reply: 'Discard'.
Collaboration Prompt Pe
{guestion) } These are you and some experts' answer: {answer) The description of the experts you invited are: [description]
Now, you can refine your answer based on the answer and additional information they provided better answer the question. Keep in mind that the experts' answer and additional information may not be correct, so decide carefully whether to accept his answer or stick to your original one.
Revised answer:
A.16 LIMITATIONS
While Lens offers broad task coverage and a unified evaluation setting, it currently focuses on static images and short-form reasoning. Real-world applications may require multimodal reasoning over temporal sequences or long-form narratives, which are beyond the scope of this version. Additionally, SMEC relies on prompt-based expert simulation, which, though flcxible, may introduce redundancy or sensitivity to prompt phrasing and inference.
A.17 THE USE LARGE LANGUAGE MODELS (LLMs)
We have not used Large Language Models (LLMs) for our paper writing.
32


Reviews:
[{"review_id": "NGhftqtoO6", "reviewer": "Reviewer_SK8n", "summary": "The paper presents LENS, a multi-level benchmark for evaluating multimodal large language models on perception, understanding, and reasoning tasks within a unified dataset. LENS contains over 60K human-written questions built on 3.4K real-world images, enabling analysis of interdependencies among visual reasoning levels. The authors further propose SMEC, a self-driven multi-expert collaboration framework that improves reasoning performance without external tools. Experiments on 15+ open and closed-source models reveal strong correlations between perception and reasoning performance and highlight persistent challenges in complex reasoning.", "questions": "1. Could the authors clarify the rationale for categorizing SRC as a reasoning task, given that its scope appears to overlap with understanding-level tasks?\n2. Would it be possible for the authors to include or discuss more detailed qualitative error analyses to better illustrate common failure patterns and model limitations across different task levels?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. LENS unifies eight well-defined tasks over shared images, allowing systematic analysis of inter-level dependencies.\n2. Ensures methodological transparency and ethical compliance through detailed dataset documentation and privacy handling.", "weaknesses": "1. The boundaries between perception, understanding, and reasoning tasks are not clearly defined, and some tasks (e.g. SRC as a reasoning task) overlap in scope.\n2. Lacks qualitative or fine-grained error analysis, which limits understanding of how and why models fail across different levels.\n3. Font size in Figure 6 should be improved for better readability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ndIHmnyEvb", "reviewer": "Reviewer_19vU", "summary": "This paper presents Lens, a large-scale, multi-level benchmark for evaluating multimodal reasoning in large vision-language models (MLLMs). \n\nUnlike prior task-specific datasets, Lens uses a unified image set - 3.4K high-resolution, non-commercially licensed social media image - paired with over 60K human-authored questions across 12 real-world scenarios. Each image supports eight tasks organized into three progressive tiers: perception, understanding, and reasoning, enabling a structured evaluation of how lower-level perceptual skills contribute to higher-level reasoning, while minimizing the effects of data distribution across task categories.\n\nComprehensive experiments on 15+ state-of-the-art MLLMs, including GPT-4o, Qwen2.5-VL, InternVL3, and reasoning MLLMs like QVQ-Max, and Kimi-VL, reveal strong interdependencies between low-level tasks, like perception and high level tasks like reasoning.\n\nLastly, the paper also introduces Self-Driven Multi-Expert Collaboration (SMEC), a novel framework in which an MLLM simulates a panel of specialized agents via self-generated, role-specific prompts. SMEC can dynamically refine or even redefine these roles as needed to address a given instruction without external supervision. The authors demonstrate that this approach outperforms established baselines such as majority voting, self-reflection, and direct prompting.", "questions": "Follow-up questions presented along with the weaknesses", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Hierarchical evaluation:\nThe benchmark’s three-tiered structure -  spanning perception, understanding, and reasoning - with a consistent data distribution across tasks, enables a more causal interpretation of how lower-level perceptual abilities influence higher-level reasoning (if any).\n\n2. Scale of human annotations:\nIn terms of human-authored data, the dataset is impressively large and aligns with the current scale of contemporary multimodal benchmarks.\n\n3. Up-to-date and well-curated dataset:\nThe dataset is well-defined and systematically structured, with 70% images sourced after November 2024 and 50% after January 2025. The images are high-resolution and reflect contemporary visual content, ensuring relevance to real-world scenarios beyond academic testbeds.\n\n4. Strong empirical analysis:\nOne of the paper’s strongest aspects lies in its thorough analysis and experiments - particularly the dataset exploration in Fig. 4, cross-task synthetic analysis in Fig. 6, and detailed discussions in Sections 2.3, 4.3, 4.4, 4.5.\n\n5. Interesting new SMEC framework:\nThe proposed Self-Driven Multi-Expert Collaboration (SMEC) is an interesting  idea where the same MLLM, prompted with diverse role-specific instructions, can collaboratively address complex reasoning tasks. It consistently outperforms direct prompting, self-reflection, and majority voting. This approach also raises an intriguing direction for future work on redundancy in self-reflection or voting setups - showing that structured role specialization may better harness the diverse knowledge and reasoning abilities embedded within a single MLLM?\n\n6. Clarity and presentation:\nThe paper is clearly written and easy to follow, with well-organized appendices that are rich in detail and effectively referenced throughout the main text.", "weaknesses": "Weaknesses and Questions\n\n1. Limited scenario diversity: While the dataset spans three main scenarios -  education, city, and home - it would benefit from broader coverage. For example, incorporating workplace, outdoor, or social-interaction settings could better capture real-world multimodal reasoning. Additionally, introducing samples grounded in logical, mathematical, or scientific domains, following the same hierarchical design principles, would strengthen the benchmark’s comprehensiveness.\n\n2. Number and organization of tasks per tier: The benchmark currently features a relatively limited number of tasks within each tier. It is unclear whether this was an intentional design choice (e.g., grouping subtasks to facilitate downstream synergy analysis between task tiers) or simply a byproduct of human annotation diversity.\n\na) Did the authors design subtasks with cross-tier synergy in mind?\nb) How do they envision scaling the benchmark to include additional domains (e.g., logic, mathematics, science)?\n\n3. Clarifications on SMEC framework:\nThe Self-Driven Multi-Expert Collaboration (SMEC) framework is conceptually strong but would benefit from further detail and analysis. Section 3 and the appendix provide helpful descriptions, yet several aspects remain underspecified:\n\na) How exactly is the diversity metric defined and measured?\nb)  Why did the authors choose 3500 samples subset, and how representative it is of the entire benchmark? Do results in Table 3 scale to the entire dataset?\nc) What are the types of roles the model tends to generate, and which roles contribute most to performance gains? \nd) are roles domain-centric  (e.g., geometry, culture, ethics) or task-centric roles (e.g., perception vs. reasoning)? Which is better and what scenario? \ne)Could the framework be extended to multi-model setups, where different MLLMs act as specialized experts?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hraUXFarpQ", "reviewer": "Reviewer_d4LY", "summary": "This paper introduces \"Lens,\" a new multi-level benchmark for evaluating multimodal large language models (MLLMs). The authors argue that existing benchmarks fail to assess how foundational perceptual skills (like object detection) contribute to higher-order reasoning. Lens addresses this by providing 3.4K contemporary images, each annotated for a hierarchy of tasks progressing from perception to understanding and reasoning. The paper evaluates over 15 recent MLLMs and finds that even top models struggle with the reasoning tasks, scoring below 60%. To address this, the authors also propose SMEC, a framework where an MLLM uses self-generated prompts to simulate a \"panel of experts\" to collaborate on complex reasoning, showing improved performance. A key finding is the confirmation of synergistic effects, where improving low-level perceptual tasks directly facilitates better performance on high-level reasoning.", "questions": "There is a formatting problem of the citations in the paper. To cite a paper, \\citep should be used instead of \\cite.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Hierarchical Benchmark Design。 The \"Lens\" benchmark introduces a valuable multi-level structure (Perception, Understanding, Reasoning) where all tasks are annotated on the same set of images. This is a strength as it uniquely enables the evaluation of synergistic effects。\n\n2. Contemporary and Relevant Dataset. The dataset is built from 3.4K contemporary images manually collected from social media, with a large portion (53%) published after January 2025. This freshness is crucial for fairly evaluating modern MLLMs and mitigating the risk of data contamination from older, widely used training sets.\n\n3. Constructive Contribution with SMEC. Beyond just identifying a problem, the paper proposes a solution with the \"Self-Driven Multi-Expert Collaborative Framework\" (SMEC). This framework offers an innovative, tool-free method for enhancing MLLM reasoning by simulating a panel of experts. The positive results from SMEC add significant value to the paper.", "weaknesses": "1. Limited to Static Images. As acknowledged by the authors, the benchmark is confined to static images. This scope does not capture the complexities of real-world multimodal reasoning, which often involves video, temporal sequences, audio, or long-form narrative understanding.\n\n2. Potential Impracticality of SMEC. The SMEC framework relies on an iterative, multi-step process of generating expert prompts and synthesizing answers. This implies a significant increase in computational overhead and latency at inference time, which could make the approach impractical for real-time applications. The paper demonstrates effectiveness but does not deeply analyze this efficiency trade-off.  For example, if using several small models, how about just using a larger powerful model? Under the same compute, which method is better?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fAvk4dbc4m", "reviewer": "Reviewer_4Zew", "summary": "This paper introduces Lens, a benchmark for evaluating MLLMs on multi-modal reasoning and SMEC, a framework for MLLMs to solve the tasks with self-generated expert opinions. The Lens consists of eight tasks under three main categories: perception, understanding and reasoning. Images (3.4K) of the benchmark are collected manually from variety social media sources. The results of the experiments on Lens show that low-level visual tasks can affect the higher-order reasoning process. When the models apply SMEC for Scene Knowledge Inference task, the accuracy of the models increases with more iterations.", "questions": "•\tIn which task is the interleaved image-text feature used? Figure 7 does not show the example. \n\n•\tThe second link on foot page opens Instagram webpage, not developer agreement. The reviewer didn’t understand the relation of the link and text in the paper. \n\n•\tDo the authors get legal permissions from social media platforms X, Instagram, Weibo and RedNote to use their data? Although some processes, like erasing facial information, have been done for privacy concerns, the images are still copyrighted. For example, the copyright regulations of RedNote state that “RedNote's trademarks, logos, and content are protected by intellectual property laws. You may not use our intellectual property without explicit permission.” According to these cases the authors should get permission from the platforms to use their data for research purposes. If permission has been granted for data use, including documentation of this in the Appendix, is recommended. \nRedNote: https://red-note.co/terms-of-service \n\n•\tIt would be valuable to report human performance on the Lens benchmark and the performance comparison between humans and models. \n\n•\tThe Related Work section does not clearly articulate how Lens differs from existing benchmarks, particularly in terms of visual and reasoning capabilities. It would be helpful to clarify what unique contributions or challenges Lens introduces that are not addressed by prior datasets.\n\nSuggestions:\n\n•\tThe organization of the paper could be improved. For instance, placing the 'Related Work' section in the Appendix makes it harder to assess the motivation and novelty, whereas some less critical elements like Figure 2 are included in the main text", "limitations": "", "rating": 2, "confidence": 2, "soundness": 2, "presentation": 1, "contribution": 1, "strengths": "•\tThe data is manually filtered and annotated by human verifiers.\n\n•\tThe process of SMEC is explained very detailly.  \n\n•\tSynergistic effects evaluation provides detailed examination of cross-tasks performance.", "weaknesses": "•\tThe overall writing quality of the paper could be improved. Some sections lack clarity, and certain sentences are difficult to understand.\n\n•\tWhile the paper highlights multi-level evaluation and integrated tasks as key contributions, these concepts are not sufficiently explained or exemplified in the paper.\n\n•\tThe paper does not clearly justify the need for introducing a new benchmark. Several existing benchmarks already include some of the tasks used in the paper. It remains unclear why extending existing benchmarks with additional tasks would not be sufficient to analyze cross-task interactions. \n\n•\tThe authors did not explain why the community needs a new benchmark. Other benchmarks have already included some tasks. By adding other task-related questions to the existing benchmarks and data, the same effect can be achieved. In that case, the evaluation of the synergistic effect does not require generating a new benchmark. \n\n•\tIt remains unclear how the proposed SMEC framework distinguishes itself from prior work in terms of novelty and contribution. The related work section does not address prior efforts involving expert-based or modular task-solving frameworks. \n\n•\tThe experimental validation of SMEC appears limited, as its performance is only evaluated on a single task (Scene Knowledge Inference). To validate its effectiveness, it would be valuable to test SMEC across other tasks and other datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Conditions for acceptance have been satisfied.
Decision: Accept (Poster)