HSSBench: Benchmarking Humanities and Social Sciences Ability for Multimodal Large Language Models
Authors: Zhaolu Kang
Junhao Gong
Jiaxu Yan
Wanke Xia
Yian Wang
Zhuo Cheng
Wenhao Cao
Ziwen Wang
ZhiYuan Feng
Huaxuan Ding
Siqi He
Shannan Yan
Xiaomin He
Junzhe Chen
Chaoya Jiang
Wei Ye
Kaidong Yu
Xuelong Li
Venue: ICLR 2026 Poster (2026)
Keywords: MLLMs
Benchmark
Dataset
Humanities and Social Sciences
Primary area: datasets and benchmarks

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated significant potential to advance a broad range of domains. However, current benchmarks for evaluating MLLMs primarily emphasize general knowledge and vertical step-by-step reasoning typical of STEM disciplines, while overlooking the distinct needs and potential of the Humanities and Social Sciences (HSS). Tasks in the HSS domain require more horizontal, interdisciplinary thinking and a deep integration of knowledge across related fields, which presents unique challenges for MLLMs, particularly in linking abstract concepts with corresponding visual representations. Addressing this gap, we present HSSBench, a dedicated benchmark designed to assess the capabilities of MLLMs on HSS tasks in multiple languages, including the six official languages of the United Nations. We also introduce a novel data generation pipeline tailored for HSS scenarios, in which multiple domain experts and automated agents collaborate to generate and iteratively refine each sample. HSSBench contains over 13,000 meticulously designed samples, covering six key categories. We benchmark more than 20 mainstream MLLMs on HSSBench and demonstrate that it poses significant challenges even for state-of-the-art models. We hope that this benchmark will inspire further research into enhancing the cross-disciplinary reasoning abilities of MLLMs, especially their capacity to internalize and connect knowledge across fields.

Published as a conference paper at ICLR 2026
HSSBENCH: BENCHMARKING HUMANITIES AND SOCIAL SCIENCES ABILITY FOR MULTIMODAL LARGE LANGUAGE MODELS
Wanke Xia", Yian Wang', Ziwen Wang", Huaxuan Ding². Zhuo Cheng Wenhao Cao Zhiyuan Feng". Siqi He?, Shannan Yan Junzhe Chen Xiaomin He Chaoya Jiang', Wei Yel.t, Kaidong Yu Xuelong Li
Zhaolu Kangl.2.< Junhao Jiaxu Jiaxu Yan
'National Engineering Research Center for Software Engineering, Peking University PInstitute of Artificial Intelligence, China Telecom (TeleAI)
STsinghua University *Chinese Academy of Sciences 6 University of British Columbia Renmin University of China zlkang25estu. pku. edu.cn
Gongragtt
Type
The gruft denn Bu oue camur sec woce
Khe Hed cydde ringg
Tedtal Noreac
Calorr
Art
X
The viliial hooy) pertos
X
We propose HSSBench, a lcrge-scale benchmark spanning six diverse categories and 45 types, comprising 13,152 samples collected the six official languages of the United Nations.
Figure 1:
ABSTRACT
Multimodal Large Language Models (MLLMs) have demonstrated significant potential to advance broad range of domains. However, current benchmarks for evaluating MLLMs primarily emphasize general knowledge and vertical step-by- step reasoning typical of STEM disciplines, while overlooking the distinet needs and potential of the Humanities and Social Sciences (HSS). Tasks in the HSS domain require more horizontal, interdisciplinary thinking and deep integration of knowledge across related fields, which presents unique challenges for MLLMs, particularly linking abstract concepts with corresponding visual representations. Addressing this gap, we present HSSBench, dedicated benchmark designed to assess the capabilities of MILMS on HSS tasks in multiple languages, including the six official languages of the United Nations. We also introduce novel data generation pipeline tailored for HSS scenarios, which multiple domain experts and automated agents collaborate to generate and iteratively refine each sample. HSSBench contains over 13,000 meticulously designed samples, covering six key categories. We benchmark more than 20 mainstream MILMs HSSBench and demonstrate that it poses significant challenges even for trate-oftthe- art models. We hope that this benchmark will inspire further research into enhancing the cross- disciplinary reasoning abilities of MLLMs, especially their capacity to internalize and connect knowledge across fields.
INTRODUCTION
Multimodal large language models (MLLMs) Achiam et al. (2023); Team et (2024) have demon- strated remarkable performance across a wide range of tasks, recently achieving or even exceeding
"Equal Contribution. 'Corresponding Authors. HHSBeench is publicly available https://gg http://gtttuuucmmm//a/////////// wcw.//haaall--//SSSend..

---

Published as a conference paper at ICLR 2026
The manuscript closely resembles Spencerian Script due
Sore an of of to conatane
to its graceful, flowing style and balanced letterforms without heavy omamentation or thick strokes typical of Copperiaate. Answer: B. Spencerian Script
From: Palmer's The Palmer Method of Bustness Writing (19011.)
Q: What is the font of this manuscript?
The font in the menuscript resembles Copperplate script. Copperplate known for its uniformity elegance, and legibility, with rounded letters and smooth, flowing lines. Therefore, the correct answer is: D. Copperplate
0: A. Business Penmanship v B. Spencerion Script
C. Omamental Penmanship D. Copperplate
Business Penmansbip recers to style handwrtting used professional coneeponnence, charactetzed by clanty, fommaity, and legibliity
Figure 2: An example of cross-modal knowledge transfer issues in MILMS within the HSS domain. They struggle to associate "Business Penmanship" font knowledge with relevant images or recognize fonts in image text.
human-level capabilities in many of them. As the performance of MILMS continues to improve, conducting a comprehensive evaluation of their capabilities has become increasingly essential. In recent times, numerous benchmarks for evaluating MLLMs have emerged Hendrycks et al. (2020); Yue et al. (2024); Liu et al. (2024b); Saikh et al. (2022); Zhang et al. (2024). These tasks are designed to assess the models' ability to jointly understand and reason across multiple modalities, such as images and text, from various perspectives.
Specifically, most multimodal benchmarks are designed either from a general perspective Liu et al. (2024b; 2023); Meng et al. (2024); Han et al. (2024); Qian et al. (2024) or with a focus scien- tific disciplines such as mathematics Wang et al. (2024a); Lu et al. (2024) science Li & Tajbakhsh (2023); Liang et al. (2024), and programming Song al. (2025). Unlike STEM fields that employ "vertical reasoning": a focused sequential process using logical deduction and experimental analysis to arrive at singular correct answers, the Humanities and Social Sciences (HSS) emphasize "hor- izontal reasoning." requiring connections across different contexts and generating multiple valid interpretations rather than single solutions. This fundamental difference stems from the inherent attributes of these disciplines: While STEM fields utilize relatively fixed symbolic systems with standardized reading sequences, HSS disciplines feature symbol systems deeply rooted in regional cultures with meanings that require custorical-cultury context interpretation. Furthermore, STEM knowledge can be iteratively developed through logical deduction and experimental analysis, whereas HSS knowledge verification relies on more complex pathways involving cross-referencing literature and expert consensus, with strong dependencies on real-world information.
Although some efforts have been made to explore aspects of the HSS, these attempts lack depth and do not provide a comprehensive and thoughtful examination of MLLMs within the context of these fields. In the study of HSS-related problems, achieving pross-modal knowledge transfer is crucial. Take the scenario illustrated in Figure 2 as an example: humans with basic knowledge can accurately infer "Business Penmanship" from the image content. When directly asked about knowledge points related to "Business Penmanship", the MLLM provides correct answers. However, when queried indirectly through image, the model fails to recognize the font features in the text, preventing from associating the relevant knowledge points with the image. This reveals a problem: most models struggle to establish meaningful mapping relationships between HSS-related images and the abstract concepts they represent. Although these models may recognize abstract concepts in isolation, they fail effectively connect these concepts with the corresponding visual content. From a long-term perspective, a model skilled at solving mathematical problems but unable to interpret historical contexts or understand ethical principles offers an incomplete and potentially harmful form of intelligence.
To address this challenge, we introduce HSSBench, an innovative and comprehensive multilingual benchmark specifically crafted to thoroughly assess the performance of MILMs in the HSS domain. Comprising around 13k carefully curated test items, HSSBench is structured into 45 types and covers 6 key categories within the field. HSSBench utilizes the visual question answering (VQA) format for this purpose, as shown in Figure 1. Given the involvement of numerous key domains, our work requires interdisciplinary collaboration. To this end, we have engaged experts from various fields to design the data framework and ensure quality control, thereby maximizing the representativeness and credibility of HSS-related issues. In addition domain experts contributing data, we leveraged the
2

---

Published as a conference paper at ICLR 2026
STEP-1: Sieve bonegrniottb
Chacklntt:
dependenzios
seratiive beaze Reeviitate genzmiie
Test
All Experts
Eragoo
Datasets
Saxee Nabaimiie
Datasets
on slrmed
Checklat:
Aucescy
AApprepria No antigusity
Opes
Win Image
Experts (other dareein))
to Stage
STAGE II
STAGE II
Summarizzrr
Examples
Rukes irt
Lopp Search A Par
Corsh TucT
Overview
Mascher
Experts Seneration Bear saar. informe))
TExT Datasots
Busked
Mex Proerie for
Oragecs KJ.
Genemttor
Final Data
Extracter
Segmant
Top-N Text
Figure 3: Our pipeline build HSSBench.
expertise of both specialists and MLLMs to develop VQA generation pipeline, which produced a portion of the high-quality data. Finally, through evaluations supporting English, Chinese, French, Russian, Spanish, and Arabic, HSSBench enables the assessment of the capabilities of MLLMs in addressing HSS challenges across wide range of linguistic contexts.
In this study, we evaluate the performance of HSSBench across range MILMs and find that presents significant challenge for these models, their accuracy often falls below 60%. We con- ducted several comparative experiments to analyze the performance of the models. Our contributions can be summarized follows:
First, we introduced HSSBench, a novel dataset specifically designed for the HSS domain, which encompasses 6 distinct categories and 45 major types HSS tasks.
Second, offered practical data construction method. It utilizes multi-agent framework tailored for the HSS domain, allowing batch generation high-quality, novel datasets.
Finally, conducted detailed evaluations of over 20 MILMs HSSBench across six languages, verifying that HSS tasks still pose significant challenges for MLLMs. This work establishes foundation for future MLLM research focusing on HSS and serves as benchmark for future studies in this field.
2 HSSBENCH
The worldview within the HSS is expansive and lacks unified definitions. Our dataset focuses on six high-interest categories within the HSS domain: geography, art, culture, social sciences, history, and economics. Textual data remain the primary medium for disseminating knowledge the HSS domain, while pictorial data, though less abundant and more challenging to collect, provide valuable complementary information. This presents significant challenges for experts in the construction of datasets. For this purpose, we designed and developed VQA Generation Pipeline (VGP) generate the dataset. After engaging experts for annotation, we designed data construction agent based on their annotation logic, enabling us generate a sufficient volume of data.
In this section, we will outline the details of the VGP. We have divided this process into three main stages: Dataset Preparation, Dataset Construction, and Validation. The three stages are summarized in Figure 3. 1) During the Dataset Preparation stage, both experts and networked agent participate in obtaining the raw materials necessary for data construction. Either source sufficient to provide the foundational content for the subsequent stage. 2) In the Dataset Construction stage, data can be and agents- cecetts-onnttuucte approach. 3) Within the
constructed using two methods: sxpert-construcet Validation stage, both experts and agent filtration work jointly to identify high-qualit data. The screened data passes through the second stage again until it meets the required standards. Data that do not conform to the construction logic are removed.
3

---

Published as a conference paper at ICLR 2026
2.1 DATASET PREPARATION, STAGE
At this stage, focus collecting data target disciplines, covering texts and illustrations. Through systematic organization and strict screening, we lay solid data foundation for the subsequent development of high-quality questions and answers.
Data Selection and Collection with Experts. During the data preparation for HSSBench, ensuring data diversity is crucial. For image and text selection, adopt combined approach involving experts and multi-agent framework.
For images, avoid data leakage, we initially encourage experts from various fields to their private images. When acquisition is difficult, the dataset also incorporates images drawn by experts and licensed ones from open-source communities obtained by both experts and the multi-agent framework. Experts and agents collaborate to screen image content, choosing those related to subject areas and targeted knowledge points for constructing high-quality questions.
Regarding text, experts may face challenges like unfamiliarity with domain-specifie knowledge, we recommend they acquire high-quality textual materials from multiple sources, such as academic resource repositories of universities and open-source communities. This enriches information sources and mitigates bias. In the early stages, domain experts obtain credible resources like textbooks, past papers, and digital course materials from various disciplines. Since these resources are proofread dur- ing compilation, they possess high reliability and dense knowledge content, making them trustworthy sources for constructing questions. Each expert reviews materials in their field, extracting relevant text passages and images, eliminating redundant information and standardizing the data format.
Networked Information-Aggreeati Agent. Inspired by the expert-led data collection process, we designed the Networked Infornation-AAgreeatti Agent to mimic their workflow. This agent broadens data sources by filtering Internet data while maintaining quality. For each discipline, it first compiles relevant keywords as knowledge point indices. Then, it retrieves online data, classifies it into text and images, and matches data against the keywords assess relevance. For text, it evaluates aspects like professionalism. uniqueness, logical structure, and the need for cross-validaties with images. If the text meets criteria, related images are extracted. Through this process, we obtain high-quality multimodal data at controlled cost. Finally, domain experts review the curated data to ensure professionalism and reliability.
2.2 DATASET CONSTRUCTION, STAGE II
At this stage, we developed multidisciplinary visual-question generation pipeline comprising two key phases: Experts Construction and Multi-Agent Construction. These phases are designed to ensure that the questions produced offer a comprehensive assessment of the model's effectiveness.
Dataset Construction with experts. At this stage, experts have two main responsibilities: revising existing questions and creating entirely new ones. 1) When revising existing questions, experts optimize multiple-choice items, encompassing both the original ones provided in the materials and those requiring revision after failing validation tests. Experts carefully review the relevant textual and visual content, adjusting the question stems and answer choices by integrating real-world knowledge. The goal is to strengthen the connection between the questions and the accompanying images while improving the plausibility of the incorrect options. 2) For creating new questions, experts generate items based on the given high-quality texts and images. These new questions are expected to demand identification of knowledge points from the images and complex reasoning based on real-world understanding to arrive at the correct answers. Following these principles, experts produce set of high-quality questions that fully utilize the provided multimodal materials to meet the demands of model performance evaluation.
Dataset Construction with Multi-Agent. Inspired by the expert annotation process and the high- quality questions it produces, designed multi-agent automated construction framework aimed at improving data generation efficiency and reducing human labor. This agent architecture comprises several roles, including summarizer, extractor, question generator, and image matcher.
4

---

Published as a conference paper at ICLR 2026
Initially, the summarizer and extractor independently analyze the document text produce prehensive summary and a set of high-quality text segments, respectively. The summary provides an overview of the key knowledge points within the document, while the extracted segments offer detailed information suitable for direct question formulation. LLM then scores these text segments based on information density, uniqueness, and logical coherence, selecting the top N segments according to their scores.
The question generator then formulates N questions by leveraging both the full summary and the selected text segments. It operates under detailed guidelines and is supported by multiple examples of human-authon authored questions, including stems, options, answers, and explanations, to ensure adherence to question design standards. Finally, the image matcher pairs images with questions by leveraging either direct image-question matching or matching based on image descriptions, depending on the available data. This Multi-Agent approach enables large-scale generation of high-quality question datasets efficiently and with minimal human intervention.
2.3 VALIDATION, STAGE III
This subsection outlines the stringent validation procedures implemented to ensure the quality and relevance of the data used in model evaluation. Two critical types validation are performed: Agent Validation and Expert Validation.
Agent Validation. This stage aims to eliminate duplicate questions and verify the strength of the correlation between the visual content and the corresponding text. First, the Validation Agent calculates the textual similarity between questions and filters out highly redundant ones based on predefined criteria to ensure the diversity of the dataset. Next, for all constructed data, it is necessary to ensure that I) without providing an image, the question cannot be correctly answered based solely on the text, and 2) without providing a question, the image alone cannot lead the correct answer. This requirement stems from our aim to scrutinize the model's 'ss capabilities from multimodal perspective. If a single modality, text or images, suffices to answer a question, compromises the quality of the dataset and its ability to comprehensively evaluate the performance of MLLMs. To meet these two requirements, the Validation Agent assesses the extent to which the question depends on the image. If the image is deemed unnecessary, the question is sent back to the Stage II for revision; it still fails to meet the requirements after multiple iterations, the question is discarded.
Expert Validation. This stage focuses expert evaluation of the data. For data annotated by experts, each entry must be 1) validated by other domain experts to confirm its appropriateness and lack of ambiguity, and 2) confirmed by all experts ensure that it is free from sensitive issues. In the case of data generated by models, a rigorous evaluation by experts specializing in data generation is required verify its accuracy and absence of ambiguity.
Furthermore, it requires the collective consent of all experts to confirm that the data are free of sensitive issues.
Figure 4: Overview of HSS- Bench.
2.4 IMPLEMENTATION DETAILS
In our VGP, we leverage GPT-4o and GPT-4.1 due to their out- standing capabilities. Throughout all stages of LLM, we employed the Chain-of-ThoughtCCC Thought(COT) prompting strategy. For more detailed information on VGP, please refer to Appendix A.
30.30%)
(30.75%)
2.5
DATA STATISTICS
di
HSSBench consists of 13,152 multiple-choice questions distributed across six major categories- Economy, Art, Culture, Social Sci- ences, History, and Geography and further divided into 45 specialized subtypes.
We require each question to be presented in a multiple-choite format with one correct answer and several distractors that are plausible but ultimately incorrect. Although some original questions in the HSS domain naturally have multiple correct answers, these multi-answer questions have been reformulated into single-answer questions for consistency. This was achieved by combining the
5

---

Published as a conference paper at ICLR 2026
multiple correct options into single choice. This approach preserves the original content and complexity of the questions while conforming to a uniform single-answer multiple-choice format.
It supports evaluation in six different languages. Our initial data came from multiple countries and languages, with questions originally created by domain experts in their source languages (examples in Appendix D). We then used LLM-basee translation models to translate the questions into five other languages. All translations were carefully reviewed and validated by bilingual experts to ensure linguistic accuracy and cultural appropriateness, maintaining semantic consistency while respecting cultural nuances.
The statistical data of HSSBench is illustrated in Figure 4, with detailed description of the dataset provided in Appendix B. Appendix B.1 reports human expert accuracy across all categories, Ap- pendix B.2 provides an in-depth taxonomy of categories and subtypes. Appendix B.3 also includes an analysis of the most frequent content words per category. Appendix B.4 details the composition of the dataset in terms of contributions of human experts versus automated agents, together with quality validation through model accuracy comparisons.
3 EXPERIMENT
3.1 EXPERIMENTAL SETUP
Models. For open- models, we selected Owen2.5-VL Bai et al. (2025), Qwen2-VL-72B Wang et al. (2024c), QVQ-72B-Preview Wang et al. (2024b), Deepseek-VI.2- Tiny Wu et al. (2024c), MiniCPM-o-2.6 Yao et al. (2024), mPLJUGOO(3322B775 Ye et al. (2024), Llava-onevision-77 Li al. (2024b), et al. (2025), -L...VL...--BBMP. Chen et al. (2024b), Phi-3.5- www.--Viiion-nntteeee Abdin et al. (2024), Janus-Pro Chen al. (2025), Llaval.5 Liu et al. (2024a). For closed-soures commercial models, we utilized GPT-4o/4. 1/4.1 1mini/4 I-nano OpenAl et al. (2024).
Li et al. (2024a), VL3-8B Zhu
Evaluation Settings. 1) We conducted evaluations using two types of prompts to examine the MILMMs performance under different prompting strategies. One approach involved prompting the model to directly output the answer without any intermediate reasoning, while the other required the model to generate COT before providing the final answer. 2) Additionally, question in six different language versions to investigate how the MLLMs performance varies across languages for the same question. Furthermore, we optimized the question formats by designing two types of prompts for each item: multiple-choice and open-ended questions. This allowed us to assess whether the model can correctly answer questions without any explicit hints. Detailed prompt settings are provided in Appendix C.8. 4) For the HSSBench evaluation, we employed two assessment methods and selected GPT-4o as a representative dosed-souuce model and Qwen2.5-7B as representative open-source model. A detailed comparative analysis of both models is provided in Appendix C.9.
tested each
3.2 RESULTS
Overall Results. Table presents the overall results of various open-source and closed-source MLLMs evaluated on HSSBench in the context of English language tests. Ct. and Dt. represent two types of prompts: COT prompts and direct response prompts. C. and O. denote question types: multiple-choice and open-ended questions. "Human" refers to the average final scores achieved by experts in various fields. Detailed experimental results for Dt. and are presented Appendix C.
All of our initial generated data consisted of multiple-choice questions. Since some questions cannot be reasonably converted to open-ended formats, changing all of them into open-ended questions would lead poor-quality data. This approach would be unfair for evaluating the model and would fail to accurately reflect differences in model performance. Therefore, we chose to modify only those questions that can still be answered meaningfully open-ended questions serve the evaluation data for open-en open-ended question performance.
The experimental results highlight the performance differences between the various models. Of the open-source models, Qwen2.5- "L-72BB-Instit delivers the highest performance, although it still falls short of surpassing closed-source models in open-ended question tests. The GPT-4.1 series

---

Published as a conference paper at ICLR 2026
Model
Goography
Economy
Culture
Sciences
History
C.C.
CcO.
C.C.
C.O.
CLO.
CtC.
Ct..,
Cc..
DrC.
CtC.
Randen
24.03
0.00
21.92
0.00
25.00
0.00
24.00
0.00
24.91
0.00
25.00
0.00
24.62
0.00
24.62
0.00
Human
94.14
93.06
92.99
94.44
93.84
95.53
93.83
93.83
Open-sounce
I.M (Sivile
FOR)
Qwen2.5.VL.
3-VI--3B-nnomm
35.25
11:74
28.56
16.36
34.61
1.85
59.46
11.50
36.23
11.07
54.55
3,62
29.01
9.94
34,99
9.33
(wen2.5-VL-7B--noii
40.09
21.00
41.19
31.31
30.19
4.63
19.00
40.62
19.67
11.31
37.88
15.21
38.19
17.89
Llava-onevision-7
32.05
7.04
32.25
11.68
31,02
1.35
36.03
3.00
27.07
4.51
32.92
3.17
36.20
5.75
31.56
5.20
Iamm2-llann-set--8
4.23
19.82
6.54
3.24
32,80
8.61
29.
31.20
27.95
LiteraVL.3-88
42.12
10.80
33:70
16.36
38.69
7.41
48.30
13.00
45.09
12.70
38.61
13.57
42.14
12.27
41.42
12.31
MisroV1...55B-MMPP
57.24
17.37
34,07
20.09
35.00
8.11
43.35
15:50
40.54
16.39
35.99
15.57
79.50
11.77
37.68
15.21
Mu..35-Visio-Jssrrrtt
25.55
26.80
13.55
3.20
20.31
7.29
28.81
35.89
10.32
26.04
6.96
Janus-Pro
29.11
8:45
22.54
6.07
41.00
6.02
34.65
12.00
29.75
10.25
33.22
7.60
30.03
8.49
31.66
8.41
Deepseck- V1-2-iiy
5.78
3.29
3.77
4,95
0.00
2.46
16.74
8.25
mPLUGG-OW15-28
25.83
4.69
24.63
1.35
31.83
3.69
29.99
3.00
25.96
4.51
29.43
4.07
28.73
4.02
27.71
3.57
mPL.IG-Ow(5527
50.60
7.04
33.15
9.91
13.73
2,30
36.40
7.00
27.64
4.10
25.60
2.30
33.01
6.68
27.52
6.23
MiniCPM---22.6
19.08
7.22
3.57
4.03
3,70
24.11
1.
12.75
4.03
7,38
2,68
10.74
3.36
7.38
5.37
10.74
4.03
10.74
6.04
8.06
4.03
9.96
4.25
Open source LLM
(Scole >- 108)
Owen2.5 VL. Instract
52.48
21:33
52.79
6.67
28,94
8.00
53.87
24.00
$7.03
26.67
39.20
3.33
48.38
15.89
50.75
15.00
Qwen2: VL-72B Instruct
50.74
52.55
32.65
8.62
16.36
56.22
Owen2.5 VL-728 Instrrct
$5.59
13:33
53.83
37.33
41.49
7.33
57.77
17.57
69.30
28.19
40.84
14.67
54.17
18.17
$1.87
19.73
QVQ-72B Preview
19.93
3.33
21.87
17.33
29:67
3.33
26.54
7.43
28.86
18.79
23.30
9.33
25,60
10.37
24.69
9.92
Closed
LLM
GPT-4o
46.58
22.07
35, 14
14.29
45.26
16:00
15.98
46
19.36
GPT-4.1
39.11
40.38
48.08
52.70
48.95
24.88
35.36
49.25
41.91
48.77
43.51
23.53
45.02
25.38
42.66
39.97
G-T--,I-mmii
47.67
34.27
58.27
49.10
48.26
20.74
45.05
36.68
47,84
36.89
43.58
23.53
45.75
24.32
48.03
33.59
GPT-4. I-nono
33.21
30.74
39.71
41.44
38.70
7.83
37.52
28.14
34.01
30.74
35.83
20.36
36.33
21.12
35.83
26.22
Table I: Scores (%) of MLLMs on HSSBench (EN-I). The highest and second highest scores are marked in blue and green, respectively.
achieves sfate-of-the-art performance in most tasks, with a particularly impressive accuracy of 39.97% in open questions, almost double that of other closed-source models. In contrast, some open-source models show considerably lower performance. We also observe that model performance varies by language. Multilingual evaluation results are provided in the Appendix C.2.
Model performance on different categories. Among the six categories evaluated on HSSBench (EN-I), the economic-related tasks consistently emerge as the most challenging for the models.
The average score of all models in this category is the lowest, indicating that addressing economic problems requires a deep understanding of various economic theories and the ability to apply them in complex reasoning. However, current open-source MLLMs still exhibit significant deficiencies in these aspects. In contrast, closed-source models perform exceptionally well in economically related tasks, and this advantage may stem from their exposure to large amounts of high-quality training data in the economic domain.
On the other hand, the Geography category appears to be the easiest task for the models, with the highest average scores observed across the board. This trend implies that geographic knowledge, which is often more factual and less abstract compared to other humanities and social sciences domains, is better captured by the training data and reasoning capabilities of the models.
Interestingly, in certain categories of multiple-choice tasks, such as Culture and Social Sciences, some open-source models outperform their flosed-source counterparts. For example, larger open-source models like Qwen2.5-VL-322 and V-LL7-2B demonstrate competitive or superior results compared to closed-source models like GPT-4o in these domains. This phenomenon may be partially attributed to the fact that most of the data experts invited are Chinese. Although the evaluation was conducted in English, it is possible that Qwen benefits from a training data advantage related Chinese content, which could have influenced its superior performance in these tasks. also indicates that the gap between open- soun-s and closed-source models is narrowing in specific HSS tasks.
3.3 QUALITATIVE ANALYSIS
Comparison between Direct Answer and COT Prompting. Table 1 shows subtle differences in MLLMs' performance HSSBench when using direct answer versus COT prompting. Notably, COT does not always help; some models perform better with direct answers, indicating that longer reasoning can mislead them.
7

---

Published as a conference paper at ICLR 2026
Specifically, COT prompts exacerbate hallucination issues in certain models, where reasoning flaws in textual analysis and misinterpretation of visual inputs lead to the generation of incorrect backgroumd knowledge during step-by-stey analysis. This causes reasoning to deviate from the correct answer. For example, when tackling geographic questions, models often struggle to accurately interpret spatial elements such as location markers and contour lines-visual features primarily composed of points and lines rines-resulting in analytical outcomes that contradict the actual image content.
Even when notable errors do not occur in intermediate reasoning steps, the final summarization phase can suffer due to excessive information generation, which exceeds the model's ability to effectively weigh the importance of answer options, leading to prediction mistakes. This highlights the importance of integrating reasoning steps cohesively. Further detailed analysis in Appendix D reveals that many models fail to internalize visual knowledge during the divergent thinking process of HSS tasks.
Comparison between Multiple-choices and Open-ended. We reformulated some questions as open-ended to test models without answer options. Results reveal that HSS tasks remain very challenging: only a few models exceed 15% accuracy. This matches expectations, as even experts find these questions difficult without options or background cues.
Answer choices provide prior knowledge that narrows the answer space. Without them, models' reasoning becomes highly divergent, often drifting far from correct answers. This suggests models mainly rely on shallow visual features (e.g., size, type, motion) but miss deeper symbolic information like cultural context or spatiotemporal cues.
These findings highlight two key limitations: insufficient real-world knowledge and weak integration of visual and textual modalities. The performance drop with COT further reflects models' struggles to internalize and retrieve complex knowledge. Improving model capabilities on HSS tasks remains a critical challenge.
The role of accurate answers. To test if models truly understand questions rather than exploiting options, we added confusing choices like "None of the above" while keeping the correct answer unchanged, sampling 150 questions per category (results Table 11, Appendix C.3).
Lower-performing models' accuracy dropped noticeably. Analysis revealed two patterns: some models were misled to select the confusing option; others, though not choosing it, showed incoherent reasoning, failing to filter out wrong choices and sometimes excluding the correct answer altogether. This indicates models struggle to assess the relative credibility of options and make reliable judgments, exposing their vulnerability to distractors and raising concerns about robustness.
Visual Information Extraction. To investigate the impact of visual information loss on the perfor- mance of MLLMs, we designed two complementary experiments. In the first experiment, we used GPT-4.1 to generate detailed textual descriptions of the images. In the second experiment, we invited domain experts to produce comprehensive and precise annotations for each image (results in Table 12, Appendix C.4).
Most models' accuracy dropped when images were replaced by GPT-generated texts, showing that direct visual input contains critical details lost in conversion. However, with expert annotations, accuracy improved noticeably despite no image access. Some models even surpassed previous performance ceilings in Culture and Social Sciences, as expert texts helped focus on crucial visual cues and better link to domain knowledge.
These results confirm earlier observations that current MILMS have inherent limitations in retrieving and understanding visual information fully. They underscore the importance of improving models' abilities to extract and integrate key visual features for HSS tasks.
Comparative Analysis of HSSBench with Related Benchmarks. We compared HSSBench with other benchmarks that include some HSS data, such as CMMMU, MME, and MMMU, as well as with STEM benchmarks (details in Appendix C.5 and C.6).
Across HSS benchmarks, relative model performance trends are consistent, but HSSBench yields lower accuracy, indicating it is more challenging and better captures HSS complexity. Compared
8

---

Published as a conference paper at ICLR 2026
to STEM benchmarks, models perform much better STEM tasks, highlighting the unique diffi- culties of HSS domains that require nuanced cultural and social understanding. These comparisons demonstrate HSSBench's value in driving progress underexplored HSS challenges.
Evaluation of Retrieval-Augmeented Generation on HSSBench. We tested Retrieval -urieeene- Generation (RAG) by integrating external knowledge from Wikipedia and HSS documents with several smaller MILMs under direct and COT prompting (results in Tables 15 and 16, Appendix C.7).
Contrary to expectations, RAG did not consistently improve accuracy. Some modest gains appeared in specific domains or models, but overall performance was often similar or worse than direct prompting alone. This suggests that general retrieval corpora and simple integration methods are insufficient for the nuanced knowledge HSS tasks demand.
The limited effectiveness of RAG in supporting MLLMs for complex multi-hop HSS tasks stems from misalignment between general-purpee purpose corpora and HSS knowledge, as well as deficiencies in current retrieval and integration methods. Moreover, MLLMs struggle to internalize and transfer newly retrieved domain knowledge. These challenges highlight the need for specialized HSS retrieval resources, RAG techniques tailored HSS, and enhanced mechanisms for MLLMs to acquire, transfer, and apply HSS-specific knowledge.
RELATED WORK
In recent years, significant progress has been made in the development of multimodal benchmarks and methodologies. Numerous datasets have emerged that assess models from various perspectives, which can be broadly categorized into Generation Benchmarks Liu et al. (2024b; 2023); Meng et al. (2024); Han et al. (2024); Qian et al. (2024); Xu et al. (2023); Yin et al. (2023); Zeng et al. (2023); Wu et al. (2024a); Luo et al. (2024); Feng et al. (2025); Wang et al. (2025e), Reasoning Benchmarks Wang et al. (2024a); Lu et al. (2024); Li & Tajbakhsh (2023); Liang et al. (2024); Song et al. (2025), and Application Benchmarks Fan et al. (2022); Rawles et al. (2023); Chen et al. (2023); Sermanet et al. (2023); Chen et al. (2024a); Koh al. (2024); You et al. (2024); Gao et al. (2025); Kang et al. (2026); Zheng et al. (2026); He et al. (2026). Generation benchmarks cover a wide range of tasks, often including not only reasoning challenges but also content related the humanities and social sciences. Their main objective is to provide a comprehensive assessment of the performance of MLLMS in various dimensions. contrast, reasoning benchmarks are designed primarily around mathematical and scientific problems, seeking to rigorously assess the capabilities of MLLM in logical and analytical reasoning. Among the various evaluation formats, multiple-chotl choice datasets are the most prevalent, owing to their simplicity in evaluation and ease of comparison Zeng et al. (2023).
The proliferation of these data sets has significantly accelerated research progress, serving both as training resources and as tools for assessing the multifaceted competencies of MILMS Wang ct al. (2025f); Liu et al. (2025b); Luo et al. (2026); Guan ct al. (2026); Fu et al. (2026); Wu ct al. (2025; 2024b); Feng et al. (2026); Li et al. (2026; 2025dze); Hu et al.; Zhang et al. (2025c;baa); Ding et al. (2025); Li et al. (2025c); Wang et al. (2026); Ma et al. (2026); Liu et al. (2025a); Bi et al. (2026); Wang et al. (2025d;;;;;a); Liang et al. (2025); Ma et al. (2025a;b); Li et al. (2025a); Yu et al. (2025); Feng et al. (2024; 2023); Li et al. (2021; 2024c; 2025b). Advances in data set construction are particularly exciting, ranging from human-annotated high-quality datasets Romero et al. (2024) to those generated through pipelines built in LLM Chandrasegaran et al. (2024). Despite these achievements, several critical limitations remain. 1) Most multimodal datasets cover wide range of categories, but suffer from issues such as limited data sources, relatively simple questions, and insufficient image information. 2) Many large-scale datasets are collected through web crawling without thorough manual annotation and verification, which can introduce biases in evaluation results.
3) Current reasoning datasets predominantly focus on STEM tasks, relying heavily on scientific and mathematical data to assess reasoning capabilities. In contrast, the benchmark we propose is created through interactions among expert agents, targeting the HSS domain. In contrast, the benchmark we propose focuses on the HSS domain, emphasizing tasks that analyze and understand the abstract concepts embedded in images. This approach fills a notable gap in the field and lays the foundation for advancing model performance in this underexplored area.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
We present HSSBench, a novel benchmark dataset constructed through a multi-agent pipeline involving experts from diverse fields. HSSBench is designed rigorously evaluate the true mastery of tasks by models within the HSS domain. The dataset comprises six categories, each derived from raw data collected from repositories in the six official languages of the United Nations and subsequently processed to generate task-specific data.
We then carried comprehensive benchmarking of various MILMMs using HSSBench. Our results reveal that HSSBench poses significant challenges to all tested models, which exhibit poor perfor- mance on reasoning tasks in the HSS domain. In particular, the accuracy of the model decreases substantially when answer choices are not provided as prompts. We hope that releasing HSSBench will encourage the Al community to place greater emphasis on reasoning over non-STEM data, thereby ad advancing research on MLLMs from this important perspective.
REFERENCES
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matthew Dixon, Ronen Eldan, Victor Fragoso, Jianfeng Gao, Mei Gao, Min Gao, Amit Garg, Allie Del Giorno, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Wenxiang Hu, Jamie Huynh, Dan Iter, Sam Ade Jacobs, Mojan Javaheripi, Xin Jin, Nikos Karampatziakis, Piero Kauffmann, Mahoud Khademi, Dongwoo Kim, Young Jin Kim, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Yunsheng Li, Chen Liang, Lars Liden, Xihui Lin, Zeqi Lin, Ce Liu, Liyuan Liu, Mengchen Liu, Weishung Liu, Xiaodong Liu, Chong Luo, Piyush Madan, Ali Mahmoudzadeh, David Majercak, Matt Mazzola, Caio César Teodoro Mendes, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez-Beckee Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilae, Liliang Ren, Gustavo de Rosa, Corby Rosset, Sambudha Roy. Olatunji Ruwase, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah. Ning Shang, Hiteshi Sharma, Yelong Shen. Swadheen Shukla, Xia Song. Masahiro Tanaka, Andrea Tupini, Praneetha Vaddamanu, Chunyu Wang, Guanhua Wang. Lijuan Wang, Shuohang Wang, Xin Wang, Yu Wang. Rachel Ward, Wen Wen, Philipp Witte, Haiping Wu, Xiaoxia Wu, Michael Wyatt, Bin Xiao, Can Xu, Jiahang Xu, Weijian Xu, Jilong Xue. Sonali Yadav, Fan Yang, Jianwei Yang. Yifan Yang. Ziyi Yang. Donghan Yu, Lu Yuan, Chenruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yue Zhang, Yunan Zhang, and Xiren Zhou. Phi-3 technical report: highly capable language model locally on your phone, 2024. URL wwt.hhtsssstaiiiiie wtw..br///////../.....pg// 14219.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint 23..:x::333087777 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint ar arXiv:2502. 13923,, 2025.
Jinhe Bi, Aniri, Yifan Wang, Dangi Yan, Wenke Huang, Zengjie Jin, Xiaowen Ma, Sikuan Yan, Artur Hecker, Mang Ye, Xun Xiao, Hinrich Schuetze, Volker Tresp, and Yunpu Ma. Prism: Self-pruning intrinsic selection method for training-free multimodal data selection, 2026. URL /ttp:://aaxvvvv.o http:://aaxvv.orr//a/222222 12119.
Keshigeyan Chandrasegaran, Agrim Gupta, Lea M. Hadzic, Taran Kota, Jimming He, Cristóbal Eyzaguirre, Zane Durante, Manling Li, Jiajun Wu, and Li Fei-Fei. Hourvideo: 1-hour video- language understanding, 2024. URL mittps://arxxt wtwaaaiiw....abbbaeeeeaaeeeaaaae org/abs/2411. 04998.
Liang Chen, Yichi Zhang, Shuhuai Ren, Haozhe Zhao, Zefan Cai. Yuchi Wang, Peiyi Wang, Tianyu Liu, and Baobao Chang. Towards end-to-end embodied decision making via multi-modal large
10

---

Published as a conference paper at ICLR 2026
language model: Explorations with gpt4-vision and beyond, 2023. URL https//:://ax abs/2310 0111100001111
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling, 2025. URL htttps://axivv.ooo http///xxiv..og/bbbba//// www.://gg//bba/...ooooo.oo.sss// 2501 17811.
Yi Chen, Yuying Ge, Yixiao Ge, Mingyu Ding, Bohao Li, Rui Wang, Ruifeng Xu, Ying Shan, and Xihui Liu, Egoplan-bent :ooplan-bench: bench: Benchmarking multimodal large language models for human-level planning, 2024a. URL https:///ariiii http///awwwi..rr///////32/2 06722.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shengiong Ye, Hao Tian, Zhaoyang Liu, al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024b.
Yang Ding, Yizhen Zhang, Xin Lai, Ruihang Chu, and Yujuu Yang. Videozoomer: Reinforcement- learned temporal focusing for long video reasoning. arXiv preprint arXiv:2512.223115 2025.
Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang. Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar. Minedojo: Building open-ended embodied agents with internet-scale knowledge, 2022. URL https://aa/iii www..org//bbs.... www. 088533.
Chen Feng, Minghe Shen, Ananth Balashankar, Carsten Gerner-Beuerle, and Miguel RD Rodrigues. Noisy but valid: Robust statistical evaluation of LLMs with imperfect judges. In The Fourteenth International Conference on Learning Representations, 2026. URL. https:://pperreeeee forum?i dorum??d=hhhxrrea..u
Yujie Feng, Zexin Lu, Bo Liu, Liming Zhan, and Xiao-Ming Wu. Towards lIm-driven dialogue state tracking. arXiv preprint arXiv:2310.14970 2023.
Yujie Feng, Xu Chu, Yongxin Xu, Guangyuan Shi, Bo Liu, and Xiao-Ming Wu. Tasl: Continual dialog state tracking via task skill localization and consolidation. arXiv preprint arXiv:2408.0983 2024.
Zhiyuan Feng, Zhaolu Kang, Qijie Wang, Zhiying Du, Jiongrui Yan, Shubin Shi, Chengbo Yuan, Huizhi Liang, Yu Deng, Qixiu Li, al. Seeing across views: Benchmarking spatial reasoning of vision-language models in robotic scenes. arXiv preprint arXiv:2510. 19400, 2025.
Rong Fu, Muge Qi, Chunlei Meng. Shuo Yin, Kun Liu, Zhaolu Kang, and Simon Fong. Advsyngnn: Siructure-adaptiic graph neural nets via adversarial synthesis and self-corrective propagation. arXiv preprint urXiv:2602.1 17071, 2026.
Jian Gao, Richeng Xuan, Zhaolu Kang, Dingshi Liao, Wenxin Huang, Zongmou Huang, Yangdi Xu, Bowen Qin, Zheqi He, Xi Yang, et al. Laobench: A large-scale multidimensional lao benchmark for large language models. arXiv preprint arXiv:2511. 113344 2025.
Shuhao Guan, Moule Lin, Cheng Xu, Jinman Zhao, and Derek Greene. Teaching VLMS to admit uncertainty in OCR from lossy visual inputs. In The Fourteenth International Conference on Learning Representations, 2026. URL https:ttp//.. : http:///ppeeeeeeeeeeeeeeee wtt.eeeeeweooommmdddijiiiiiiaie
Tianyang Han, Qing Lian, Rui Pan, Renjie Pi, Jipeng Zhang, Shizhe Diao, Yong Lin, and Tong Zhang. The instinctive bias: Spurious images lead to illusion in milms, 2024. URL https: //arxiv. oww./vvoor/aaeo wwwp//202. 03757.
Yingjie He, Zhaolu Kang, Kehan Jiang, Qianyuan Zhang, Jiachen Qian, Chunlei Meng, Yujie Feng. Yuan Wang, Jiabao Dou, Aming Wu, et al. How order-sensitive are llms? orderprobe for deterministic structural reconstruction. arXiv preprint arXiv:2601. 00666, 2026.
Dan Hendrycks, Collin Burss, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint 00t::2009.033000 2020.
11

---

Published as a conference paper at ICLR 2026
Yuxiao Hu, Qian Li, Dongxiao Zhang, Jinyue Yan, and Yuntian Chen. Context-alignment Activating and enhancing Iims capabilities in time series. In The Thirteenth International Conference on Learning Representations.
Zhaolu Kang, Junhao Gong, Wenqing Hu, Shuo Yin, Kehan Jiang, Zhicheng Fang, Yingjie He, Chunlei Meng, Rong Fu, Dongyang Chen, et al. Quanteval: A benchmark for financial quantitative tasks in large language models. arXiv preprint wrXiv:2601.0868) 2026.
Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks, 2024. URL https://arxly: wwwpaaawwwvvv.rb/////aa. org/abs/2411. 13649.
Li, Kaichen Zhang, Hao Zhang, Dong Guo, Renrui Zhang, Feng Li, Yuanhan Zhang, Ziwei Liu, and Chunyuan Li. Llava-next: Stronger Ilms supercharge multi- modal capabilities in the wild, May 2024a. URL https://llaaavvl.get 2024-05-10-
io/blog/
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer, 2024b. URL
03326.
Junxian Li, Di Zhang, Xunzhi Wang. Zeying Hao, Jingdi Lei, Qian Tan, Cai Zhou, Wei Liu, Yaotian Yang, Xinrui Xiong, et Chemvlm: Exploring the power multimodal large language models in chemistry area. In Proceedings of the AAAI Conference on Arlificial Intelligence, volume 39, pp. 415-423, 2025a.
Shengzhi Li and Nima Tajbakhsh. Scigraphqa: large-scale synthetic multi-turn question-answering answering dataset for scientific graphs, 2023. URL ht tps: http///axxvv.../aabaaa/ 2308. 2ttbb///3/000339
Shuaibo Li, Shibiao Xu, Wei Ma, and Qiu Zong. Image manipulation localization using attentional cross-domain enn features. IEEE Transactions on Neural Networks and Learning Systems, 34(9): 5614-5628, 2021.
Shuaibo Li, Wei Ma, Jianwei Guo, Shibiao Xu, Benchong Li, and Xiaopeng Zhang. Unionformer: Unified-learning transformer with multi-view representation for image manipulation detection and localization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12523-12533 2024c.
Shuaibo Li, Zhaohu Xing, Hongqiu Wang, Pengfei Hao, Xingyu Li, Zekai Liu, and Lei Zhu. Toward medical deepfake detection: A comprehensive dataset and novel method. In International Conference on Medical Image Computing and Computer-Assiste Intervention, pp. 626-63 637. Springer, 2025b.
Yanshu Li, JianJiang Yang, Ziteng Yang, Bozheng Li, Hongyang He, Zhengtao Yao, Ligong Han, Yingjie Victor Chen, Songlin Fei, Dongfang Liu, et Cama: Enhancing multimodal in-context learning with context-aware modulated attention. arXiv preprint arXiv:2505. 7097, 2025c.
Yuqi Li, Kai Li, Xin Yin, Zhifei Yang, Junhao Dong, Zeyu Dong, Chuanguang Yang, Yingli Tian, and Yao Lu. Sepprune: Structured pruning for efficient deep speech separation. arXiv preprint arXiv:2505.12074 2025d.
Yuqi Li, Chuanguang Yang, Hansheng Zeng, Zeyu Dong, Zhulin An, Yongjun Xu, Yingli Tian, and Hao Wu. Frequency-aligned knowledge distillation for lightweight spatiotemporal forecasting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7262-7272, 2025e.
Yuqi Li, Siwei Meng, Chuanguang Yang, Weilun Feng, Junming Liu, Zhulin An, Yikai Wang, and Yingli Tian. A comprehensive survey of interaction techniques in 3d scene generation. Authorea Preprints, 2026.
Wengi Liang, Gan Sun, Yao He, Jiahua Dong, Suyan Dai. Ivan Laptev, Salman Khan, and Yang Cong. Pixelvla: Advancing pixel-level understanding in vision- Mision--angugee-aee action model. arXiv preprint 1rX15:2511.01571, 2025.
12

---

Published as a conference paper at ICLR 2026
Zhenwen Liang, Kehan Guo, Gang Liu, Taicheng Guo, Yujun Zhou, Tianyu Yang. Jiajun Jiao, Renjie Pi, Jipeng Zhang, and Xiangliang Zhang. Scemqu: A scientific college entrance level multimodal question answering benchmark, 2024. URL https://arxiv.orgg/ htt.a//////eeeeeee 240.205122......
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. URL mttps://arxivv or wwwoorgg/ss..o.oo..o 23tgg//aa////...0// 08485.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee, Improved baselines with visual instruction tuning, 2024a. URL https: http:/i////xvvv.rrrr/ ab3/2310 037444.
Shicheng Liu, Siyuan Xu, Wenjie Qiu, Hangfan Zhang, and Minghui Zhu. Explainable reinforcement learning from human feedback to improve language model alignment. Advances in Neural Information Processing Systems, 38, 2025a.
Xinzhang Liu, Chao Wang, Zhihao Yang, Zhuo Jiang, Xuncheng Zhao, Haoran Wang, Lei Li, Dongdong He. Luobin Liu, Kaizhe Yuan, et al. Training report of telechat3-moc. arXiv preprint arXiv:2512.24157, 2025b.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pp. 216-233. Springer, 2024b.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai- Reii Chang. Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts, 2024. URL https: org/ abs/ 2210 02255.
Fuwen Luo, Chi Chen, Zihao Wan, Zhaolu Kang, Qidong Yan, Yinjjie Li, Xiaolong Wang, Siyu Wang, Ziyue Wang, Xiaoyue Mi, Peng Li, Ning Ma, Maosong Sun, and Yang Liu. CODIS: Benchmarking context-dependent visual comprehension for multimodal large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 10639-10659, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/ www.//////044/ 2024. acl- 573/.
2024.acl-long.577 URL https: hatps////aaaaaaloo/o/oo hwwphhhooogooooog////eeee
Meng Luo, Bobo Li, Shanqing Xu, Shize Zhang, Qiuchan Chen, Menglu Han, Wenhao Chen, Yanxiang Huang, Hao Fei, Mong-Li Lee, and Wynne Hsu. Unveiling the cognitive compass: Theory-of-mine mind-guided multimodal emotion reasoning, 2026. URL https://arxxx org/abs/ 2602. 00971.
Hongxu Ma, Kai Tian, Tao Zhang, Xuefeng Zhang, Han Zhou, Chunjie Chen, Han Li, Jihong Guan, and Shuigeng Zhou. Generative regression based watch time prediction for short-video recommendation. arXiv preprint arXiv:24/2.20211 2025a.
Hongxu Ma. Guanshuo Wang, Fufu Yu, Qiong Jia, and Shouhong Ding. Ms-detr: Towards effective video moment retrieval and highlight detection by joint motion-semantic learning. Proceedings of the 33rd ACM International Conference on Multimedia, pp.
2025b.
Weijian Ma, Shizhao Sun, Tianyu Yu, Ruiyu Wang. Tat-Seng Chua, and Jiang Bian. Thinking with blueprints: Assisting lasion-language models in spatial reasoning via structured object 01984.
representation, 2026. URL
Ziyang Meng, Yu Dai, Zezheng Gong, Shaoxiong Guo, Minglong Tang, and Tongquan Wei. Vga: Vision gui assistant minimizing hallucinations through image-centric fine-tuning, 2024. URL 14056.
https://arxiv. http:://aaaxiv.ooraa////////////
OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Madry, Alex Baker- Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Conneau, Ali Kamali, Allan Jabri, Allison Moyer, Allison Tam, Amadou Crookes, Amin Tootoochian, Amin Tootoonchian, Ananya Kumar, Andrea Vallone, Andrej Karpathy, Andrew Braunstein, Andrew Cann, Andrew Codispoti, Andrew Galu, Andrew Kondrich, Andrew Tulloch, Andrey Mishchenko, Angela Back, Angela Jiang, Antoine Pelisse, Antonia Woodford, Anuj Gosalia,
13

---

Published as a conference paper at ICLR 2026
Arka Dhar, Ashley Pantuliano, Avi Nayak, Avital Oliver, Barret Zoph, Behrooz Ghorbani, Ben Leimberger, Ben Rossen, Ben Sokolowsky, Ben Wang. Benjamin Zweig, Beth Hoover, Blake Samic, Bob McGrew, Bobby Spero, Bogo Giertler, Bowen Cheng, Brad Lightcap, Brandon Walkin, Brendan Quinn, Brian Guarraci, Brian Hsu, Bright Kellogg, Brydon Eastman, Camillo Lugaresi, Carroll Wainwright, Cary Bassin, Cary Hudson, Casey Chu, Chad Nelson, Chak Li, Chan Jun Shern, Channing Conger, Charlotte Barette, Chelsea Voss, Chen Ding, Cheng Lu, Chong Zhang, Chris Beaumont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine McLeavey. Christopher Hesse, Claudia Fischer, Clemens Winter, Coley Czameekki Colin Jarvis, Colin Wei, Constantin Koumouzelis, Dane Sherburn, Daniel Kappler, Daniel Levin. Daniel Levy, David Carr, David Farhi, David Mely, David Robinson, David Sasaki, Denny Jin, Dev Valladares, Dimitris Tsipras, Doug Li, Duc Phong Nguyen, Duncan Findlay, Edede Oiwoh, Edmund Wong. Ehsan Asdar, Elizabeth Proehl, Elizabeth Yang. Eric Antonow, Fric Kramer, Eric Peterson, Eric Sigler, Eric Wallace, Eugene Brevdo, Evan Mays, Farzad Khorasani, Felipe Petroski Such, Filippo Raso, Francis Zhang, Fred von Lohmann, Freddie Sulit, Gabriel Goh, Gene Oden, Geoff Salmon, Giulio Starace, Greg Brockman, Hadi Salman, Haiming Bao, Haitang Hu, Hannah Wong, Haoyu Wang, Heather Schmidt, Heather Whitney, Heewoo Jun, Hendrik Kirchner, Henrique Ponde de Oliveira Pinto, Hongyu Ren, Huiwen Chang, Hyung Won Chung, Ian Kivlichan, Ian O' O'Connell, Ian O'Connell, Ian Osband, Ian Silber, Ian Sohl, Ibrahim Okuyucu, Ikai Lan, Ilya Kostrikov, 1lya Sutskever. Ingmar Kanitscheider, Ishaan Guirajani. Jacob Coxon, Jacob Menick, Jakub Pachocki, James Aung, James Betker, James Crooks, James Lennon, Jamie Kiros, Jan Leike, Jane Park, Jason Kwon, Jason Phang, Jason Teplitz, Jason Wei, Jason Wolfe, Jay Chen, Jeff Harris, Jenia Varavva, Jessica Gan Lee, Jessica Shieh, Ji Lin, Jiahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joanne Jang, Joaquin Quinonero Candela, Joe Beutler, Joe Landers, Joel Parish, Johannes Heidecke, John Schulman, Jonathan Lachman, Jonathan McKay, Jonathan Uesato. Jonathan Ward, Jong Wook Kim, Joost Huizinga, Jordan Sitkin, Jos Kraaijeveld, Josh Gross, Josh Kaplan, Josh Snyder, Joshua Achiam, Joy Jiao, Joyce Lee, Juntang Zhuang, Justyn Harriman, Kai Fricke, Kai Hayashi, Karan Singhal, Katy Shi, Kavin Karthik, Kayla Wood, Kendra Rimbach, Kenny Hsu, Kenny Nguyen, Keren Gu- Gu-Lemberg, Kevin Button, Kevin Liu, Kiel Howe, Krithika Muthukumar, Kyle Luther, Lama Ahmad, Larry Kai, Lauren Itow, Lauren Workman, Leher Pathak, Leo Chen, Li Jing. Lia Guy, Liam Fedus, Liang Zhou, Lien Mamitsuka, Lilian Weng, Lindsay McCallum, Lindsey Held, Long Ouyang. Louis Feuvrier, Lu Zhang. Lukas Kondraciuk, Lukasz Kaiser, Luke Hewitt, Luke Metz, Lyric Doshi, Mada Aflak, Maddie Simens, Madelaine Boyd, Madeleine Thompson, Marat Dukhan, Mark Chen, Mark Gray, Mark Hudnall, Marvin Zhang, Marwan Aljubeh, Mateusz Litwin, Matthew Zeng, Max Johnson, Maya Shetty, Mayank Gupta, Meghan Shah, Mehmet Yatbaz, Meng Jia Yang, Mengchao Zhong, Mia Glaese, Mianna Chen, Michael Janner, Michael Lampe, Michael Petrov, Michael Wu, Michele Wang, Michelle Fradin, Michelle Pokrass, Miguel Castro, Miguel Oom Temudo de Castro, Mikhail Pavlov, Miles Brundage, Miles Wang. Minal Khan, Mira Murati, Mo Bavarian, Molly Lin. Murat Yesildal, Nacho Soto, Natalia Gimelshein, Natalie Cone, Natalie Staudacher, Natalie Summers, Natan LaFontaine, Neil Chowdhury, Nick Ryder, Nick Stathas, Nick Turley, Nik Tezak, Niko Felix, Nithanth Kudige, Nitish Keskar, Noah Deutsch, Noel Bundick, Nora Puckett, Ofir Nachum, Ola Okelola, Oleg Boiko, Oleg Murk, Oliver Jaffe, Olivia Watkins, Olivier Godement, Owen Campbell-Moore, Patrick Chao, Paul McMillan, Pavel Belov, Peng Su, Peter Bak, Peter Bakkum, Peter Deng, Peter Dolan, Peter Hoeschele, Peter Welinder, Phil Tillet, Philip Pronin, Philippe Tillet. Prafulla Dhariwal, Qiming Yuan, Rachel Dias, Rachel Lim, Rahul Arora, Rajan Troll, Randall Lin, Rapha Gontijo Lopes, Raul Puri, Reah Miyara, Reimar Leike, Renaud Gaubert, Reza Zamani. Ricky Wang. Rob Donnelly, Rob Honsby, Rocky Smith, Rohan Sahai, Rohit Ramchandani, Romain Huet, Rory Carmichael, Rowan Zellers, Roy Chen, Ruby Chen, Ruslan Nigmatullin, Ryan Cheu, Saachi Jain, Sam Altman, Sam Schoenholz, Sam Toizer, Samuel Miscrendino, Sandhini Agarwal, Sara Culver, Scott Ethersmith, Scott Gray, Sean Grove, Sean Metzger, Shamez Hermani, Shantanu Jain, Shengjia Zhao, Sherwin Wu, Shino Jomoto, Shirong Wu, Shuaiqi, Xia, Sonia Phene, Spencer Papay, Srinivas Narayanan, Steve Coffey, Steve Lee, Stewart Hall, Suchir Balají, Tal Broda, Tal Stramer, Tao Xu, Tarun Gogineni, Taya Christianson, Ted Sanders, Tejal Patwardhan, Thomas Cunninghman, Thomas Degry, Thomas Dimson, Thomas Raoux, Thomas Shadwell, Tianhao Zheng, Todd Underwood, Todor Markoy, Toki Sherbakov, Tom Rubin, Tom Stasi, Tomer Kaftan, Tristan Heywood, Troy Peterson, Tyce Walters, Tyna Eloundou, Valerie Qi, Veit Moeller, Vinnie Monaco, Vishal Kuo, Vlad Fomenko, Wayne Chang, Weiyi Zheng, Wenda Zhou, Wesam Manassra, Will Sheu, Wojeiech Zaremba, Yash Patil, Yilei Qian, Yongjik Kim, Youlong Cheng, Yu Zhang, Yuchen He, Yuchen Zhang, Yujia Jin, Yunxing Dai, and Yury Malkov. Gpt-40 system card, 2024.
14

---

Published as a conference paper at ICLR 2026
URL
Yusu Qian, Haotian Zhang, Yinfei Yang, and Zhe Gan. How easy is fool your multimodal Ilms? an empirical analysis on deceptive prompts, 2024, URL https://arxiv.ct www..rggg/.............. abs/2402. 13220.
Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap. Android in the wild: A large-scale dataset for android device control, 2023. URL https:///arivv org.bb/////////////////// 2307. 10088.
David Romero, Chenyang Lyu, Haryo Akharianto Wihowo, Teresa Lynn, Injy Hamed, Aditya Nanda Kishore, Aishik Mandal, Alina Dragonetti, Artem Abzaliev, Atnafu Lambebo Tonja, Bontu Fufa Balcha, Chenxi Whitehouse, Christian Salamea, Dan John Velasco, David Ifeoluwa Adelani, David Le Meur, Emilio Villa-Cueva, Fajri Koto, Fauzan Farooqui, Frederico Belcavello, Gan- zorig Batnasan, Gisela Vallejo, Grainne Caulfield, Guido Ivetta, Haiyue Song. Henok Biadglign Ademtew, Hernán Maina, Holy Lovenia, Israel Abebe Azime, Jan Christian Blaise Cruz. Jay Gala, Jiahui Geng, Jesus-German Ortiz-Barajas, Jinheon Baek, Jocelyn Dunstan, Laura Alonso Alemany, Kumaranage Ravindu Yasas Nagasinghe. Luciana Benotti, Luis Fernando D' Haro, Marcelo Virid- iano, Marcos Estecha-Gariin Garitaaiiaaiaiiiitaa, Maria Camila Buitrago Cabrera, Mario Rodríguez- Cantelar, Mélanic Jouitteau, Mihail Mihaylov, Mohamed Fazli Mohamed Imam, Muhammad Farid Adi- lazuarda, Munkhjargal Gochoo, Munkh- Erdene Otgonbold, Naome Etori, Olivier Niyomugisha, Paula Mónica Silva, Pranjal Chitale, Raj Dabre, Rendi Chevi, Ruochen Zhang, Ryandito Diandaru, Samuel Cahyawijaya, Santiago Góngora, Soyeong Jeong, Sukannya Purkayastha, Tatsuki Kurib- ayashi, Teresa Clifford, Thanmay Jayakumar, Tiago Timponi Torrent, Toqeer Ehsan, Vladimir Araujo, Yova Kementchedjhieva. Zara Burzo, Zheng Wei Lim, Zheng Xin Yong, Oana Ignat. Joan Nwatu, Rada Mihalcea, Thamar Solorio, and Alham Fikri Aji. Cvqa: Culturally-diverse multilin- gual visual question answering benchmark, 2024. URL https: //arxiv. or abs 2406. 05967.
Tanik Saikh, Tirthankar Ghosal, Amish Mittal, Asif Ekbal, and Pushpak Bhattacharyya. Sciencequ: A novel resource for question answering on scholarly articles. International Journal on Digital Libraries, 23(3):289-300 2022.
Pierre Sermanet, Tianli Ding, Jeffrey Zhao, Fei Xia, Debidatta Dwibedi, Keerthana Gopalakrishnan. Christine Chan, Gabriel Dulac-Amola Sharath Maddineni, Nikhil J Joshi, Pete Florence, Wei Han, Robert Baruch, Yao Lu, Suvir Mirchandani, Peng Xu, Pannag Sanketi, Karol Hausman, Izhak Shafran, Brian lchter, and Yuan Cao. Robovqa: Multimodal long-horizon reasoning for robotics, 2023. URL https: /arxivv orggabs//3// 00899.
Xiaoshuai Song, Muxi Diao, Guanting Dong, Zhengyang Wang, Yujia Fu, Runqi Qiao, Zhexu Wang, Dayuan Fu, Huangxuan Wu, Bin Liang, Weihao Zeng, Yejie Wang, Zhuoma GongQue, Jianing Yu, Qiuna Tan, and Weiran Xu. CS-bench: A comprehensive benchmark for large language models towards computer science mastery. In The Thirteenth International Conference on Learning Representations, 2025. URL
Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.055.20 2024.
Haozhe Wang, Chao Qu. Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl-rethinker: Incentivizing self-reflection of vision- languaee models with reinforcement learning. arXiv preprint arXiv:25044088837, 2025a.
Haozhe Wang. Haoran Que, Qixin Xu, Minghao Liu, Wangchunshu Zhou, Jiazhan Feng, Wanjun Zhong, Wei Ye, Tong Yang, Wenhao Huang, et al. deverse-engineeree reasoning for open-ended generation. arXiv preprint 2rXiv:2509.0616/ 2025b.
Haozhe Wang, Alex Su. Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: In- centivizing pixel-space reasoning with -uriosity-driven reinforcement learning. arXiv preprint (.6.:25005.19666 2025c.
Haozhe Wang, Qixin Xu, Che Liu, Junhong Wu, Fangzhen Lin, and Wenhu Chen. Emergent hierarchical reasoning in llms through reinforcement learning. arXiv preprint 2oXiv:2509003444 2025d.
15

---

Published as a conference paper at ICLR 2026
Junze Wang, Lei Fan, Dezheng Zhang, Weipeng Jing, Donglin Di, Yang Song, Sidong Liu, and Cong Cong. Visual prompt-agnostic evolution. arXiv preprint arXiv:2601.20233 2026.
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset, 2024a. URL httpt://arxxii.. .rrg/ abs/2402. 14804.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision lsioonlanguage model's perception of the world at any resolution. arXiv preprint arXiv:2409. 1219,, 2024b.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution, 2024c. URL https: www.oogg//////.. awwpg//.b//44..... 12191.
Xiaolong Wang, Zhaolu Kang, Wangyuxuan Zhai, Xinyue Lou, Yunghwei Lai, Ziyue Wang, Yawen Wang, Kaiyu Huang, Yile Wang, Peng Li, et al. Mucar: Benchmarking multilingual cross-modal ambiguity resolution for multimodal large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 15037-15059, 2025e.
Zihan Wang, Xinzhang Liu, Yitong Yao, Chao Wang, Yu Zhao, Zhihao Yang. Wenmin Deng, Kaipeng Jia, Jiaxin Peng. Yuyao Huang, et al. Technical report of telechat2, telechat2. and t1. arXiv preprint arXiv:2507. 18013, 202.5f.
Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Chunyi Li, Wenxiu Sun, Qiong Yan, Guangtao Zhai, and Weisi Lin. Q-bench: benchmark for general- purpose foundation models on low-level vision, 2024a. URL https://arxivv http:///aaaivvoo/a////////////// 14181.
Xiangyu Wu, Qing Yuan Jiang, Yang Yang, Yi-Feng Wu, Qing-Guo Chen, and Jianfeng Lu. Tai++: Text image for multi-label image classification by co-learning transferable prompt. In I/CAI, 2024b.
Xiangyu Wu, Feng Yu, Yang Yang, Qing-Guo Chen, and Jianfeng Lu. Multi-label test-time adaptation with bound entropy minimization. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. URL https: wtwprrrriieeenettoouuunneeeeee
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang. Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang, Liang Zhao, Yisong Wang, and Chong Ruan. Deepseek-v12: Mixture-of-experts vision-language models for advanced multimodal understanding, 2024c. URL https://arxiv. org/abs/2411 2412. 10302.
Peng Xu, Wenqi Shao, Kaipeng Zhang, Peng Gao, Shuo Liu, Meng Lei, Fanqing Meng. Siyuan Huang, Yu Qiao, and Ping Luo. Lylm-ehub: A comprehensive evaluation benchmark for large vision-language models, 2023. URL httpp://arxiiss orggabs///////oooooo/ooooeeessss /bbs/236.. 09265.
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao Zhihui He, et al. Minicpm-v: A gpt-4v level milm on your phone. arXiv preprint arXiv:2408. 01 800, 2024.
Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Zhang, Fei Huang, and Jingren Zhou. mplug-ow13: Towards long image-sequence understanding in multi-modal large language models, 2024. URL
Zhenfei Yin, Jiong Wang, Jianjian Cao, Zhelun Shi, Dingning Liu, Mukai Li, Lu Sheng, Lei Bai, Xiaoshui Huang, Zhiyong Wang, Jing Shao, and Wanli Ouyang. Lamm: Language-assisted mudti-modal instruction-tuning dataset, framework, and benchmark, 2023. URL org/abs/23066 06687.
16

---

Published as a conference paper at ICLR 2026
Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Sweamgin, Jeffrey Nichols, Yinfei Yang. and Zhe Gan. Ferret-ui: Grounded mobile ui understanding with multimodal 1lms, 2024. URL https://arxiv. orgg/bs////4 05719.
Xinlei Yu, Chengming Xu, Guibin Zhang, Zhangquan Chen, Yudong Zhang, Yongbo He, Peng-Tao Jiang, Jiangning Zhang, Xiaobin Hu, and Shuicheng Yan. Vismem: Latent vision memory unlocks potential of vasion-language models. arXiv preprint arXiv:2511.11007 2025.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng. Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang. Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal under- standing and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 9556-95677 2024.
Yan Zeng, Hanbo Zhang, Jiani Zheng, Jiangnan Xia, Guogiang Wei, Yang Wei, Yuchen Zhang, and Tao Kong. What matters in training gpt4-style language model with multimodal inputs?, 2023. URL pittps://ariiv www.:org/////////eeoooo
abs/ /2307 02469.
Ge Zhang, Xinrun Du, Bei Chen, Yiming Liang, Tongxu Luo, Tianyu Zheng, Kang Zhu, Yuyang Cheng, Chunpu Xu, Shuyue Guo, et al. Cmmmu: A chinese massive multi-discipline multimodal understanding benchmark. arXiv preprint arXiv:2401. 11944, 2024.
Shuoshuo Zhang, Zijian Li, Yizhen Zhang, Jingjing Fu, Lei Song, Jiang Bian, Jun Zhang, Yujuu Yang. and Rui Wang. Pixelcraft: A multi- agent system for high-fidelity visual reasoning structured images. arXiv preprint arXiv:2509.25) 188,, 2025a.
Shuoshuo Zhang, Yizhen Zhang, Jingjing Fu, Lei Song, Jiang Bian, Yujiu Yang, and Rui Wang. See less, see right: Bi-directional perceptual shaping for multimodal reasoning. arXiv preprint arXiv:2512.22120. 2025b.
Yizhen Zhang, Yang Ding, Shuoshuo Zhang, Xinchen Zhang, Haoling Li, Zhong-zhi Li, Peijie Wang, Jie Wu, Lei Ji, Yelong Shen, et al. Perl: cermutation-chhancee reinforcement learning for interleaved vision-language reasoning. arXiv preprint arXiv:2506.14907 2025c.
Leqi Zheng, Jiajun Zhang, Canzhi Chen, Chaokun Wang, Hongwei Li, Yuying Li, Yaoxin Mao, Shannan Yan, Zixin Song, Zhiyuan Feng, et al. What should cite? a rag benchmark for academic citation prediction. arXiv preprint wrXiv:2601.149M. 2026.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang. Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang. Nianchen Deng, Songze Li, Yinan He. Tan Jiang. Jiapeng Luo, Yi Wang, Conghui He. Botian Shi, Xingcheng Zhang, Wenqi Shao, Junjun He, Yingtong Xiong, Wenwen Qu, Peng Sun, Penglong Jiao, Han Lv, Lijun Wu, Kaipeng Zhang, Huipeng Deng, Jiaye Ge. Kai Chen, Limin Wang, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, and Wenhai Wang. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models, 2025. URL https:/ arrxtixxx xiv. org/ httg//b///555 10479.
17

---

Published as a conference paper at ICLR 2026
A MORE VGP DETAILS
A.1 ANNOTATION PLATFORM
Seleet bata Fidds
Sova Reralt
Irage
Quesiiin Coliora
Question:
Rusenters free
shattess booch
iewead middy
Are much Texs dellUtuting then shetion
C. Oltan lake specitic enter nal casss Typically
aages
tay my
Concot Krowon E: 154
mure Recion
Prentcu
more shotness el breth
Loading tem.
Figure 5: The interface for data construction and validation, allowing experts to visual interface to assist their work.
We simple visual interface as our annotation platform. For question input, experts can upload or write questions in the form. The interface is shown in Figure 5. During the validation process, experts can see all the data submitted by other experts. They can select entries view detailed previews of the entries.
A.2 INFORMATION ABOUT EXPERTS
Name
Languages
Professional Background
Expert
Chinese, English
Computer Science, Art, Economy
Expert 2
Chinese, Japanese, English
Computer Science, Culture Computer Science, Geography Economy, History Language and Linguistics, History, Culture Language and Linguistics, Social Science
Expert 3
English, Chinese
Expert 4
Chinese, English
Expert 5
Chinese, English, Japanese
Expert 6
Chinese, English, Russian
Expert 7
Chinese, English, Arabic
Language and Linguistics, Social Science Economy, Social Science Geography
Expert 8
Chinese, English, Arabic Chinese, English Chinese, English, French
Expert 9
Expert 10
Economy, Art Art, Culture Computer Science, Geography
Expert 11
English, Chinese, Japanese Chinese, English
Expert 12
Expert 13
Chinese, English
Geography, History Computer Science, Geography
Expert 14
Chinese, English, Spanish
Table Information about the experts involved in dataset construction.
The team of experts who contributed to the creation of HSSBench comes from variety of cultural and linguistic backgrounds, as well as interdisciplinary academic fields spanning the humanities and social sciences. This diversity is reflected in their linguistic fluency, professional expertise, and
18

---

Published as a conference paper at ICLR 2026
international academic experiences. Table 2 summarizes the profiles of the experts involved in the dataset construction.
Many of the experts have international academic experiences and interdisciplinary training across various humanities and social science domains. While individual cultural perspectives may naturally influence the emphasis framing of certain questions, we consider this diversity a strength rather than limitation. enriches the dataset by incorporating multiple viewpoints and insights, which is essential for benchmark designed to be multilingual and cross-cultural.
To mitigate potential cultural bias, we carefully curated the source materials provided to the experts during the initial stage of dataset construction. These reference materials originate from multiple countries and languages, ensuring that the constructed questions and answers reflect widely accepted, global knowledge rather than culturally subjective viewpoints. Furthermore, during the data validation and filtering stages, we rigorously excluded content that could be culturally biased or inconsistent with universal values.
In summary, our approach balances the preservation of valuable cultural diversity with the need to maintain fairness and universality in the dataset. This careful design ensures that HSSBench serves as robust and inclusive benchmark for evaluating multilingual large language models (MILMss) across different languages and cultural contexts.
B DATASET DETAILS
This section provides detailed information about benchmark designed evaluate MILMs' visual comprehension abilities through multiple-choice questions. The benchmark spans six major categories: Economy, Art, Culture, Social sciences, History, and Geography, each containing various specialized subtypes.
B.1 HUMAN PERFORMANCE
To establish a performance baseline, we asked three relevant experts in each domain to spend significant amount of time answering the entire dataset. Table summarizes their performance across the six categories.
Category
Expert
Expert 2
Expert
Overall
Economy
96.02
91.70
94.70
94.14 14
Art
93.12
92.61
93.44
93.06
Culture
92.50
90.83
95.64
92.99
Social sciences
94.49
91.77
97.07
94.44
History
95.91
91.86
93.75
93,84
Geography
94.99
92.78
95.81
94.53
Average
94.84
91.76
95.40
93.83
Table 3: Scores (%) of Experts.
Human experts demonstrated high proficiency across all categories, with overall accuracy ranging from 92.99% to 94.53%. The highest individual performance was observed in Social sciences by Expert (97.07%), while the lowest was in Culture by Expert 2 (90.83%) This high level of human performance establishes a challenging benchmark for evaluating large language models.
B.2 THE DETAILS OF TYPES
Our dataset is organized into six major categories with various subtypes in each category. Table presents the detailed breakdown of the dataset structure and the count of questions in each type. We allow each data entry to have multiple types because the intersection knowledge across disciplines essential.
19

---

Published as a conference paper at ICLR 2026
The dataset exhibits varying distributions across categories, with Geography containing the largest number of questions and Economy the smallest. Within categories, there are also significant variations in subtype representation, reflecting the natural distribution of content within these domains.
B.3 MOST-FREQUENT WORDS IN THE QUESTIONS
We analyzed the most frequent content words in the questions in all categories to understand the linguistic characteristics and tasksur dataset. Figure 6 shows the word clouds for the most frequent words in HSSBench per category, excluding common stop words. The word frequency analysis reveals distinct patterns across categories:
1) Visual observation terms dominate in Art and Culture categories ("observe", "picture,, "shown", "scene"), indicating focus on visual analysis tasks.
2) Economy questions frequently use technical terms ("price", "firm", "cost", "demand", "market", "marginal"), reflecting domain-specific concepts.
3) Geography questions heavily employ spatial and diverse visual comprehension capabilities ("dia-
gram", "map". "area", "distribution"), " emphasizing spatial reasoning.
Tosserving
Ten JutAE
charatterstics details
paeo scene si
curves marginal
sceeario
déschptions
observe
output ghreat
refer
Cost
Blus.
details
observe
picture
traditional
suppose
quanity
producing
firm Tesphute
demang
matches picture
Char Csstis
Inferi
cultur
infer
descrip
descriptions
matches
performance
point sue
Hprgduction.
conkumer
Htttt
table market
figur e
price
diagraci
labou
asoume
Art
(b) Culture
(c) Economy
piture
distribution
ogserve
figure
population
comple
based
lete
area
Imagge
read
Resorru
coorrect
China
D
boserve figure
based
reflects
main
comic
economic
Pita
deter
chert
demand
development
eemine
based figüre e
price
madiets
answer
com
determine
determinne
supaly
reffects
sobserve
map
diag
ram
Di diagram Image
duggenn
Dictur
correct
river
lesation
chart
Ahanoe
(d) Geography
History
(f) Social sciences
Figure 6: Word Cloud in HSSBench per category.
B.4 COMPOSITION AND QUALITY COMPARISON BETWEEN HUMAN AND AUTOMATED AGENTS
We provide detailed breakdown of the data sources across the six domains in HSSBench, as shown in Table 5.
To further validate the quality of data generated by both human experts and automated agents, we evaluated the performance of GPT4.I-miní on the respective subsets. The accuracy results are summarized in Table 6.
The comparable accuracy results indicate that data generated by automated agents, after undergoing multi-round expert verification, maintain quality level consistent with that of human expert con- tributions. These clarifications and statisties have been incorporated into the revised manuscript to enhance transparency regarding dataset construction and quality assurance.
C EXPERIMENTAL DETAILS
C.I DETAILS OF RESULTS
Table provides additional details the experimental results presented in Table 1. The table below reports the overall accuracy of the models under the DIRECT prompt and presents a detailed breakdown of the performance for each category under the same prompt.
20

---

Published as a conference paper at ICLR 2026
Category
Type
Count
Microeconomics
1,193
Macroeconomics
163
Labor Economics
105
Economy
Environmental Economics
63
International Trade
63
Resource Economics
44
International Finance
32
Folk Art
1,051
Eastern Art
1,041
Western Art
672
Art
Spiritual Art
630
Applied Art
452
Entertainment Art
298
Elite Art
303
Regional Culture
1,790
Material Culture
1,382
Intangible Culture
1,218
Culture
Indigenous Culture
733
Institutional Culture
266
World Culture
228
Sociology
950
Political Science
550
Psychology
339
Anthropology
190
Economics
147
Education
135
Social sciences
Philosophy
127
Law
99
Ethics
61
Journalism
24
History
18
Asian History
1,586
World Modern History
885
World Ancient History
754
European History
602
History
World Contemporary History
346
American History
295
African History
147
Oceanian History
82
Physical Geography
2,361
Regional Geography
1,694
Descriptive Geography
1,682
Geography
Explanatory Geography
1,348
Human Geography
1,319
Predictive Geography
75
Table 4: Dataset Taxonomy and Question Distribution.
21

---

Published as a conference paper at ICLR 2026
Category
Human Expert
Automated Agent
Total Samples
Art
984
1,204
2,188
Geography
2,671
891
3,562
History
1,946
550
2,496
Economy
974
459
1,433
Social Science
487
947
1,434
Culture
652
1,387
2,039
Table 5: Distribution of samples constructed by human experts and automated agents across different domains in HSSBench.
Category
Human Expert Ace.
Automated Agent Acc.
Art
42.32
45.43
Geography
48.82
46.52
History
45.59
49.12
Economy
58.87
57.67
Social Science
43.15
46.95
Culture
49.11
47.41
Table 6: Model accuracy (%) of GPT4.1 1-mni on human-expert eeaaan---eeettt-eeaeee generated and automated agent- generated subsets across different domains.
Despwek VLD
Despwet Thy
191-43
Hisse
Hitto
(a) Effect under multiple-choice question setting
(b) Effect under opeeeended question setting
Figure 7: Partial MLLMs' results under CoT prompt settings.
22

---

Published as a conference paper at ICLR 2026
Model
Goography
Reonomies
Cilture
Social Sciences
History
Art
D:C.
DrD
DeC.
D.O
DrC.
DrD
DcC.
Dx.
D.C.
Dx.0
D.C.
Dx.0
DrC.
Tr
C.C.
CLD
Random
24.93
0.00
21.92
0.00
25.00
0.00
24.90
0.00
24.91
0.00
25,00
0.00
24,62
0.00
24.62
0.00
Human
93.83
Open-wvrrcy LLM (Scole <. 108)
20
11.27
29:44
17.76
27.83
5.089
35 22
10,00
26.91
8.61
26.75
7.24
29.01
9.54
34-99
9.33
Qwen2.5-V1-77
19.25
14.75
8.14
$7.68
17.89
Liava unevision 7b
37.46
6.10
31.20
10.75
38.64
1.39
60.95
7.00
31.65
4.51
37.33
4.96
36.20
5.73
31.56
5.20
29 98
6.10
21.81
5.61
35.09
6.48
35.35
6,001
29.45
9.02
34.46
5.43
31.20
6.50
27:93
5.81
ImsereV113-88
88
13.62
17.97
48.90 03.04
14.34
42.14
12.31
IntemVL.2.5 3D-M0--- MPO
38.08
14,08
34.73
16.36
39.13
6.94
45.69
10.50
40.82
11.07
38.27
11.76
39.30
11.77
37.68
15.21
Phi-3.5 Vii.io-Viison-Bell Lnstrncc
29:84
10.33
29.44
11.21
27.83
10.65
35.22
7.00
26.91
11.48
26.75
10.86
29.01
10,32
26.04
6.36
Junus Pro
27.57
8.92
7,01
42.38
31.26
9.00
12.30
32.33
7.24
30,03
3.49
31.66
Deepseck-VV VL.2-iiy
5.16
17.38
3.15
40.17
13:46
30.33
2.50
29.97
4.51
35.19
4.52
29.86
8.23
5.09
mPL.10G-DW18-28
25:27
4.23
30.48
2,70
32.56
2.50
34:17
4.50
26.88
6.15
30.48
4.07
28.73
4.02
27.71
3.57
mPLUG-OOwB-77B
52.71
7.04
33.08
6.31
33.45
4.61
37.31
5.50
29.97
8.20
33.73
8.14
33.01
6.68
27:52
6.23
MiniCPM-a-2.6
3.48
1.67
3.90
14.16
2.06
3,70
5.75
3.05
4.72
2.27
3,022
8.70
3.70
5.71
24.11
5.71
1aval155
3.36
2.58
10:07
5.37
3.36
2,01
8.05
4,70
8.05
1.34
15.44
8.05
8.06
4.03
9.96
4.25
Open-worrce 1JAM (Scoll 1AR)
Qwee2.
16.00
39.38
39:87
53:09
57.52
24.67
10:00
48.38
15.89
50.75
15.00
Ques2
V-2
56.71
16,00
51.67
32.00
57.77
5.53
47-43
16.89
59.36
24.16
63.12
14.67
54.17
19.17
51.88
19.73
QVg-728-Previe
21.53
3.33
18.38
22.00
29.57
2,67
26.22
10.14
29.58
12.75
23.85
11.33
25.60
10.37
24.09
9.92
Quaa2-VI-728
$4.22
19.30
24.07
33.08
54.22
Closed yource LLM
GPT-4o
45.68
23.47
48.92
35.14
48.16
15.67
46.72
18.00
47.24
14.75
46.66
13.57
46.99
20.05
46.38
19.36
GPT-4.1
26.29
42.34
86
26.15
22.13
44.74
16.74
45.02
25.38
42. 66.66
(PT-4.1-mim
26.29
54.93
45.95
52.08
15.21
41 14
22.11
39.34
18.5
44.70
17.65
45.75
04.32
48
33.90
4.1 nano
33.18
20,66
31.96
36.04
45.61
14.75
38.42
13.58
32.69
20.49
36.47
18.55
36.33
21.12
35.83
26.22
Table 7: Scores (%) of MILMs on HSSBench (EN-II).
C.2 RESULTS DIFFERENT LANGUAGES
Table and Table 7 present the test results English-language data. To analyze the impact of different languages on the model, we also conducted large-scale experiments using Chinese-language data. The final experimental results are shown in Table 8 and Table 9 below.
Model
Geography
Eccnemies
Culture
Social Sciences
History
C.C.
CrO.
CLC.
C.O.
CVC.
C1.O.
C.C.
CLO.
CLC.
Cx.0.
CLC.
C..0.
DEC.
Dr.0.
C.CC.
CL.O.
Randoon
24.95
0.00
21.92
0.00
25.00
0:00
24.00
0.00
24.91
0.00
25.00
0.90
24.62
0.00
24-62
0,30
Humsn
93.06
93.84
Oper-source LLM (Scale < (08)
Qwen2.5VVL3B VL-38
40.49
10.00
28.89
19.51
36.30
0:95
42.43
9.50
43.91
11.89
53.85
4.98
35:45
11.79
35.45
9.65
Qwes2.5-V1-77
48.43
15.06
35.00
25.00
2.78
49.34
19.50
50.54
15.08
6.70
86.
14.34
Llava-mmevision-
39.68
4.69
30.80
10.38
32.96
1.85
39.46
7.00
32.97
4.51
35.03
2.71
37.89
4.98
35.62
5.13
Llamad Ulava next 8b
27.29
3.29
19.82
3.77
33.55
0:93
32.89
2.50
26.84
3.28
29.48
2.25
51.18
5.21
28.35
2.68
Intenn11.3.3 3-888
47.45
17:92
9.26
53.47
14.00)
18:44
10.43
44.81
44.92
IntemVL12. -SP
41.14
11.74
56
23.58
36.20
6.94
48.09
16.50
47.63
6.94
38.35
10.86
42.72
14.01
41.45
14.55
Phi visiion instract
22.29
5.10
25.64
4.25
30.16
2.31
23.55
1.00
23.35
4.92
23.30
1.81
34.08
6.74
25.35
3.45
Janus-Pro
28.10
8.49
44.16
6.48
34.79
3.50
14:34
14.48
32.81
10.72
mPI. 1.2-
27.20
29:43
0.90
33.74
2.50
29. 78
3.00
26.
26.76
2.87
20.43
4.52
33.25
1.90
28.
2.58
mPLUG OwL3 7B
35.01
5.63
34.23
3.15
26.39
1,84
39.26
2.50
26.08
2.46
34.23
5.43
34.22
4,71
31.77
3.49
Min-CYMMo-26
26.97
6.25
3.90
1.29
29.04
Open-suuree (Scale > (08)
Qwen2 5-V
58.45
19:33
49.86
7.33
40.90
5.33
56.49
22.67
64.33
24.67
41.62
14.67
$1.60
14.22
$2.86
15.67
Owen2 5 NL.
61.65
12.67
52.02
33.30
43.60
10.67
61.01
14.19
68.08
18.12
43.76
16.00
60.04
21.95
55.94
17.51
QVQ-728-Piivvie
19.82
8.00
21:10
18.00
50.80
3.53
25.41
6.76
27.85
10.74
25.90
6.67
22.95
9.14
24.82
8.92
Qwen VL-22-
31.28
19:30
58.25
24.07
21.25
5.66
27.79
12.96
25.00
32.08
27.65
18.87
59.11
17.79
25.55 59
18.82
Cloned-vource 1.TM
GPI-4.tmial
47.53
35.21
55.55
35.41
44.43
18.43
44.07
34.17
47.92
43.03
42,50
30.32
44.19
24.54
46.78
36.32
Table 8: Scores (%) of MILMs on HSSBench (ZH-I).
Figure& illustrates the performance of various models under four different prompt configurations.
Table 10 and Figure 9 shows the performance of the model datasets in different languages. Our dataset includes data organized in six languages. The table below presents the experimental results on stratified sample of 900 instances.
23

---

Published as a conference paper at ICLR 2026
Model
Ganguply
Eamomiix
Callarc
Social Scionss
Hiskey
DxC
Dr0g
DrCig
Dr0g
DrC.g
Dr0
Dx
DrOg
DrCi
D=0g
D:C.g
Dr0g
DxC55
Dr00
ClCgg
C1.gg
Random Human
24.95
0.00
21.92
0.00
25.00
0.00
24.00
0.00
0.00
0.00
24.62
0.00
24.00
94.14
93.06
92.99
94.44
90.24
95.53
93.53
99.85
Opea-mame 11M (Snale /0R)
SV1-15
99.51
12.21
24.36
13.21
90:94
7.87
40.01
14.50
18.16
11:68
40.01
11:26
55.45
11: 11:79
15.45
9.66
Qwen25 VL
52.52
17.84
33.97
20.75
27.50
4.63
51.13
15.00
52.13
11.48
51.17
9.55
43.12
13.17
41,85
14.24
75
42.20
6.57
30.31
7.55
36.59
1.30
42.93
5,00
35.61
4.10
56.12
5.43
57.89
4.96
35.62
5:13
1landd Mlats-nt--
4.69
19.97
6.60
38.50
5.00
34.37
6.33
31.18
InimV1 3-B8
201
4.000
1steraVL2.5 MPO
18.78
51.32
14:15
40.72
7.57
49.87
17.50
40.78
14.03
42.72
14.03
41.45
14.35
Phi vison instrett
29-92
6.57
27.49
7.08
63.28
7.87
32.89
4.50
29:31
0.56
80.08
7.69
34.08
6.74
25.35
3.45
Janus Pro
27.29
9.86
18.50
12:74
50.83
14.35
52.18
8.50
29.44
10.25
43.21
15.84
33.50
11.94
32.81
10.72
$1.25
0196
23:73
2.71
45.95
0.46
57.07
5,00
11:39
2.45
59.18
2.71
52.25
1.90
18.25
2.58
MPLUG Owt3 7B
35.85
7.51
35.05
6.79
35.70
1.38
40.59
3.00
28.37
4.10
35.05
0.79
34,22
4.71
31,77
3.49
MiniCP5h-o226
0.93
7.55
0.77
8.06
1.42
1,99
2.42
2.27
1.29
6.85
29.04
Open
LLM (Scale 100)
Owe92.5 32B
56.29
20.67
42.83
6.00
41.83
57.42
51.00
14.22
52.85
15.67
Owor2.5-VE-715
22.67
30.00
50.07
10.04
21.96
55.04
0V0-728 (00-22-BPie
6.67
17.90
20.00
29.72
4.00
34.34
13.49
28.13
22.95
9.14
24.82
Qwant-Vi-222
64.42
28.07
41.83
16.67
53.98
7.55
60.00
14.81
68.37
27.64
16.98
99.11
17 00
35.59
18,82
Clound-asoree EJ.M
GPT-4 Imin
45:93
26. 06
83.11
43.69
46:35
14:29
19.61
20,10
39.26
24.15
| 42,09
17.00
64.19
24.54
46.78
36.32
Table 9: Scores (%) of MLLMs on HSSBench (ZH-II).
400
DrC
50
Cr.
Scon
173
20
feee 3- 710 inrmec
Gwend 5-H 335 Inverss
oocen
narmyy sammtt-Saaat Sahamo
GP I 1a00
Un Cove-zhvvitan feeitaa
Ph Matnn mutrec
And Pio
tou
Gva 728 Prove
24
Oweet
Figure 8: Comparison of Model Performances Across Four Prompt Settings.
Model
Arihic
Chinese
English
French
Russian
Spanish
IntemVV3-88
35.20
49.37
12.38
39.37
38.70
39.21
Qwen2,5-VI 5--V--BB-Inssra
33.74
40.21
38.37
34.08
35.55
34,83
50.24
$11.24
Qwen2.5-VL-72BB
19.72
55.41
51.28
51.28
18.27
48.83
QVQ-728-Previee
33.98
36.89
39. 80
37.86
40.77
38.83
19rr-4.1-mim
41.33
44.89
41.67 6.
41.78
41.44
46.89
Average
40.82
46.93
42.87
43,47
Table 10: Contrast Scores (%) of MILMMs on HSSBench for six UN languages and six models.
Figure 9: Contrast Scores (%) of MILMs on HSSBench for six UN languages and six models.
24

---

Published as a conference paper at ICLR 2026
Geognaphe
Culure
Social
Histeey
Toual
CMC
CrC.Cenf.
C1C:
CCC.Conf.
CLC.
OuC.oof
OLC.
OxC.Conf
OxC.
CrU.Conf
C.CC
CIC:.conf.
C1C:
CrC:Coon
Owen2.5 VL. 7B
$8,00
48.67
12.12
31.69
29.33
25.67
39.86
54,00
55.33
05.33
35.51
36.05
40.21
37:12
30.7
46:00
41.35
36.01
45.35
-9.33
53.38
63.07
60100
56.00
19. 39.56
45.99
45.37
46.01
MisCPMo
27.51
21.47
39.35
34.67
49.33
50.67
35.81
29.73
35.25
26.83
27.33
23.33
36.34
31.22
0wea2.5-VL.33
50.53
49.85
46.10
40.90
41
30.99
59.61
$1.92
61.98
56.77
50.94
53.77
$1.85
47.74
Qvg 72B Previow
16.57
16.67
29.35
28.00
35.78
29.67
26.25
15.91
28.15
27.51
25.33
24.06
26.98
24.41
Qwm22
Lamini
54.67
42.00
Table 11: Contrast Scores (%) with confounding option of MLLMs on HSSBench.
C.3 MULTIPLE-CHOICE CONFOUNDING OPTION EXPERIMENT DETAILS
Table 11 and Figure 10 provides detailed presentation of the experimental results after adding confounding option. We sampled 900 data points. "Conf." indicates that, in addition to the given options, an extra option "None of the above answers is correct" waas added. The model's output performance was then compared under these conditions.
(a) Art
(b) Culture
(c) Economy
(d) Geography
(e) History
(I) Social sciences
Figure 10: Contrast Scores (%) with confounding option of MLLMs on HSSBench.
C.4 VISUAL INFORMATION EXTRACTION EXPERIMENT DETAILS
Table 12 and Figure 11 presents the extraction of image information into text (where "De." indicates a detailed description of the image generated directly by GPT-4.1, and "De-H." indicates a detailed explanation provided by domain expert for each image). The table below shows the detailed results of comparative experiments in which only the extracted image information, rather than the images themselves, was provided the model.
Modil
Goograpky
Calaze
Siaial Scircco
Histery
Eccerecy
Out:
C1L::
Dec::
De-1I.:
USC.
DcC:
De-ILC
CSC:
DtC
De-ELU
LIC
Dect:
De-IIC
Okt::
11--111:
Qvent V-VL-B
19:33
40,07
41.33
43.45
37.41
40.14
32607
30.00
35000
37.16
39:33
42.67
41.00
46.00
51.33
35.77
35.21
35,73
42.00
41.33
43.13
33.33
34.50
20.16
4138)
32.01
4100
41.30
41:33
05.33
45181
48.0
50381
40.45
13.32
35644
Quent 5-N1 ItB--tate
$6.00
41.17
13.00
35.33
14.30
4333
43.67
3100
47.31
08.67
4167
52.67
41m
52.67
63.67
36.57
$3.00
55.37
Q4Q-77B. Posiow
11.31
36.07
13:00
26.67
30.57
38.00
76667
77.35
75.00
5167
11.00
36/0
34.35
00.77
38.67
19.11
13.31
35,37
Qwen2 NL. Imoood
54,77
54,77
58.67
42.00
48.57
41.33
3467
32.67
43.67
47.33
5060
47.33
36,77
64.67
65.67
35.31
55.00
57.33
CISS-11
$2.00
16.00
03.33
42.57
32.77
4187
3400
4009
46.77
46.7
15.11
51.31
Table 12: Contrast Scores (%) about Visual Information Extraction of MLLMs HSSBench.
C.5 COMPARISON WITH RES-REE BENCHMARKS
We acknowledge that some existing benchmarks, such MME, include HSS-related test data, particularly within the Art domain. To better understand the relationship between these datasets and proposed HSSBench, conducted detailed comparative analysis focusing the overlapping Art category. The evaluation was performed under identical prompt settings, including both Direct and Chain-of-Thouge (CoT) prompting. The combined results presented in Table 13.
25

---

Published as a conference paper at ICLR 2026
(a) Art
(b) Culture
(c) Economy
(d) Geography
(c) History
(f) Social sciences
Figure 11: Contrast Scores (%) about Visual Information Extraction of MILMs on HSSBench.
Benchmark (% Accuracy)
Prompting
Model
CMMMU
MME
MMMU
HSSBench
Direct
CoT
Qwen2.5-VL-3B-immg 47.73
77.32
57.76
29.01
47.73
48.86
Owen2.5-VL--7BIInsse
43.18
71.21
61.21
37.88
43.18
46.59
Intern
65.91
85.28
68.10
42.14
65.91
55.68
llava-onevisioo--qwen-77
53.41
77.21
49.14
36.20
53.41
52.27
Table 13: Performance comparison the Art category across HSS #SS-related benchmarks under Direct and CoT prompting.
As shown, the relative performance trends across these benchmarks broadly consistent, indicating that the challenges within the Art domain are similar across datasets. Notably, HSSBench presents more challenging evaluation, reflected by generally lower accuracy scores, which underscores its value in pushing forward research on Humanities and Social Sciences tasks.
C.6 COMPARISON WITH STEM BENCHMARKS
While our primary focus is on Humanities and Social Sciences (HSS), we also recognize the im- portance of situating HSSBench within the broader landscape of STEM benchmarks. To this end, we provide comparative overview of model performance on several representative STEM bench- marks alongside HSSBench under the Chain-of- of-ouughtt (CoT) prompting setting. The results summarized in Table 14.
MMLU GPQA Diamond
SWE-bench MATH- AIME LiveCode OpenCompass-
Model
HSSBench
Pro
Verified
500
2024
Bench
Reasoning
Qwen2.5-VL. LL--2BB
71.2
50.2
51.87
GPT-4.1 mini
65.00
23.60
49.60
46.0
48.03
GPT-4o
79.80
66.90
35.80
54.8
46.88
GPT-4.1
81.80
66.30
54.6
92.80
48.10
40.50
54.0
42.66
InternVL3-8B
41.4
41.42
GPT-4.1 nano
50.30
29.40
34.2
35.83
Janus-Pro-77
19.1
31.66
Table 14: Model performance comparison on STEM benchmarks and HSSBench (CoT prompting).
Notes: All STEM benchmark and OpenCompass results are sourced from official publications or model release notes. A dash (-) indicates that the model did not publicly report results for that
26

---

Published as a conference paper at ICLR 2026
benchmark. Minor discrepancies may exist due testing variability but do not affect the overall trend.
This comparison reveals that while models generally achieve higher accuracy STEM benchmarks (c.g., MMLU Pro, MATH-500), their performance on HSSBench is comparatively lower. This gap highlights the unique challenges posed by HSS tasks and the necessity of dedicated benchmarks like HSSBench to drive progress in this domain.
C.7 REUREEEAL-AUGEENTED GENERATION EVALUATION
We conducted additional experiments integrating Retrieval-Augmennn Generation (RAG) with several smaller MLLMs. The retrieval database was constructed from a general knowledge corpus comprising Wikipedia and publicly available documents related to HSS. Model performance was evaluated under two prompting strategies: direct prompting and CoT prompting.
Table 15: Performance of models under direct prompting.
Social Science
Model
Geography
Art
Culture
History
Economy
Overall
Without RAG
Deee2.5-VL-7B-Ime
60.09
28,05
27.19
53.50
54.51
40.09
43,89
Qwen2.5-VL-3B-Inntte
55.40
33.48
44.24
54.50
50.82
35.59
45.56
VL3-8B
54.93
37.56
43.78
62.00
51.64
44.14
48.82
llava-onevision-qgee22--7
46.01
30.77
34.10
46.50
38.93
29.28
37.43
With RAG
Swen2.5-VL-7B--Issti
47.42
29.86
29.95
51.00
45.08
38.74
40.24
Qwen2.5-VL VL-3B-LLnttuut
49.77
33.03
35.48
48.50
43.44
33.78
40.55
InternVL3-8B
53.52
35.75
35.94
59.50
49.59
37.39
45.10
llavaooneviioonqqwen)
43.19
32.13
26.73
41.00
32.38
21.17
32.57
Table 16: Performance of models under COT prompting.
Social Science
Model
Geography
Art
Culture
History
Economy
Overall
Without RAG
Qwen2.5-V 1reen..5-VL-7--Instruu
55.40
25.79
27.65
51.00
47.54
39.64
41.08
Qwen2.5-VL-3B-Inntte Instrrcctt
51.17
30.32
35.48
49.00
46.31
32.43
40.70
LntemVVL--BB
53.52
34.39
40.55
57.00
51.64
38.74
45.86
llava-onevision--geee2-77
43.66
31.22
28.57
43.00
33.61
29.73
34.78
RAG
Dwen2.5-VVL-BB-nnste Owee22,5-VL-3B-Insstt
47.89
27.15
27.19
46.50
42.62
36.49
37.89
41.78
31.22
29.95
44.00
40.57
36.04
37.21
VntemBVL3-8B
57.75
30.77
34.10
55.50
50.41
39.19
44.50
llava-onevision-qwwen--777
42.25
29.86
28.11
42.50
31.56
23.42
32.73
The results indicate that, although RAG occasionally yields modest improvements in specific domains or models, does not consistently outperform direct prompting without retrieval augmentation. This suggests that augmenting MLLMs with general retrieval corpus and straightforward prompting strategies may be insufficient to fully exploit the complex and nuanced knowledge required for HSS tasks.
These findings highlight the challenges inherent in applying RAG to Humanities and Social Sciences benchmarks such HSSBench. We hypothesize that more specialized, domain-specit retrieval corpora, combined with advanced retrieval and integration techniques, are necessary to unlock the full potential of RAG in this context.
27

---

Published as a conference paper at ICLR 2026
C.8 PROMPT FOR MODEL INFERENCE
Table 17 details the configurations of the four prompts employed in our experiments, specifying the presence or absence of CoT prompt and indicating whether the questions were open-ended or multiple-choie.
Prompt for Model Inference
Question: [question]
w/ MC
Options: [options]
w/ CoT
Think step by step determine correct answer.
End your response with [[X]] where X is your final answer (A, B, C. D E).
Question: [question]
w/ MC
Options: [options]
CoT
Give the correct answer directly.
End your response with [[X]] where X is your final answer (A, B, C, Dr
Question: [question]
w/o MC
Think step by step to determine the correct answer.
w/ CoT
End your response with [[X]] where X is your final answer.
Question: [question]
w/o MC
Give the correct answer directly.
w/o CoT
End your response with [[X]] where X is your final answer.
Table 17: Prompt for model inference.
C.9
EVALUATION FOR MODEL'S OUTPUT
Table 18 presents the evaluation prompts employed to assess the accuracy the model's responses.
Prompt for Model Evaluation
You are an evaluation assistant.
Please determine whether the answers output by the model below correct.
Question: [question]
Options: [options]
Correct answer: [correct answer]
w/ MC
Model output content: [model output]
Please extract its final answer from the model output and determine whether it is consistent with the content of the correct answer.
the answer is correct, reply with "1". Otherwise, reply with "0".
Only reply with numbers and no superfluous content.
You are an evaluation assistant.
Please determine whether the answers output by the model below are correct.
Question: [question]
Correct answer: [correct answer]
Model output content: mmodel output]
w/o MC
Please extract its final answer from the model output and determine whether it is consistent with the content of the correct answer. Attention! If the meanings are roughly consistent, it can be regarded correct. It not necessary be strictly and completely consistent. One needs carefully consider whether the model's response is reasonable and correct. the answer is correct, reply with "1". Otherwise, reply with "0". Only reply with numbers and no superfluous content.
Table 18: Prompt for model evaluation.
In our evaluation process, employed both expert sampling and automated assessment using GPT models. To verify the consistency of the evaluation results produced by the models, we randomly sampled 2,000 data points from the overall experiment, including 1, ,,000 multiple-choice questions and
28

---

Published as a conference paper at ICLR 2026
1,000 open-ended questions. These samples were independently evaluated by three human experts, as well as by Qwen2.5-7B and GPT-4.1.
For both multiple-choice and open-ended questions, the overall agreement among the two models and the three experts, measured by Fleiss' Kappa, was 96.82% and 94.79%, respectively. The agreement between the models alone, measured by Cohen's Kappa, 97.32% for multiple-choice questions and 94.79% for open-ended questions.
These results demonstrate that the consistency between the three experts and the two models remains at a reliable level. This indicates that either evaluation method yields results that are largely consistent with those reported in the paper, thereby ensuring the reproducibility of our findings.
D MORE CASES
29

---

Published as a conference paper at ICLR 2026
Art Easemn Art
Qustire:
Flonc aaiber strie fazz arendl beuinteke sy tor odlane, Box der dia ls tee 1Mhove is the asage, and Antostar be coarhoal iasirary rxproved borchhe
Optiona:
Unensrrieed and Di Salene and convise
Species and sotaod
X
GPT-40 (with CoTy: X
stoki appor
consulee, exmiscel presure oppltel Bloughot The bouikeek procis,
wdl-Serned inikwingg coeful fkoglafll
Bruburrkk Styce The trabhtrics nothe: everty beld occuurdlyy They mumma balaned ntytu, puapoting
teea on dastr and arrense rater the enotonai iaimaity sparasiiy
Fraeitua Tenkery The curall eyta canway saw rremmen dudylie achor saans to ha exgossing a iood of incoaroeed parden
expesseed
[AA
GPT-4.1 (With COT):
the sookes spacheat Doacc crensl and praciion or fooder spaninatty)
they smarth jegpedt
voriaton prosure, resahing
liuuhhiiic Soyks - Iohe
Accihe curadr cillim card die stie? -b the thy sealy at faatasing!
Eawtioal Teahery Cortiees and eetralocs world dove cacfal peoiie, and axtor sinket Unistrabe lumese dhtares
pootily
Spacks
Dowingg
speing.
socise
Sup diseving minke
5-1 The sosker
medesids thick with vers varatice
prosure, rek miniing dynems trai eovernet The bratwerk liss centorial with dlce dart and cre purth in sinke.
dencder seadidndyy unitem shine naxral iaggoiing baime
imikeen
on The evenll does evady
mady,
rairaie dygittdd flow
Step Enocioal tealone The brackookk fans esterne lellaess in widdess Malrreseee and intreee spacias
"ppaiia perbeet
tigat
mistellaia MSabers
leaal,
talensed delheette sple. sts8 dlggn sol WDeeliess wnlineineeeeet (Al)
GPT-41:
GPT-411 mini (with COTJ:
rolatvely tiik eoticotts wirtiio prssinn with come shiles leaner and otees lightr indicring fewoes nuverett The stokks out
ovely smoots Bay lave certin moglams cerigi
Observng mtroding
cliely spord, coupped teodneek flowing syle sigocos
neal
Shep Excricnal Teakeny The lyarsk and beld sookes sugent as issose mesiinaa expesiie. The Boving axl scceellu
ixbletes sxmizeieed syle. The energy sookes consons prskes steggl vollo Jur carlio
Connkaiee Guees In thick, demas vickes Dovitg sille and maapeii boniwetk the mestinal codnny expessal by dhe ealan Temidnoniled an Dxenatiine and ainne ([R)
GPT-4.1 mini:
GPT-111 nano (with COT): B X
Forms:
aroko apporr to fuid dyranic, varying thidkneres and sanec ravernant
linddamkk Sote The ryfe
with upananone tec-finnstr
The umvza
everyy crntsiaa crmetadoac Instad, the creayy every and emetion -Tec in sorne of contriire qpaceaoppiinne
way eroksn image exgontnn andeed and open repoach
rmprosiis
expuunto, usservize emku sourt crsstinal eas that
Basad en andyau, aocr alges mee dastty wask option ii limmrranee and insinee [[8)]]
+ GPT-41 nano: NONE
X
Ihs sery, nus canh snages the sruke tomm and brudurrcke stlc drom the image pooridd
+ InterRVL25-MPOO (with COT); C X
Syoke Souzo Stapo and Contanaly Eseasee stoles arr conpeted Duidy et boder ist repteas, indisting eac Dioctice Sape. Cleck stokes connlcal dovelsedlty vas, talouing nooge sotraal.
Taklaoo
spplee
alnade
camay Movcess: Unilansiy Oheo if bunistokes an stifon er way signtinnlly inds wing oliens er denasis annte sotig nedasdaa spanassonn indioking preciten iceden.
Bosinicke Stye.
lidy defitanke UUnostrazed bire laely sloke wild dynanko exiening of initui Specisaoo Liedy if stekk expense and lase nee crend thar, indeaing spenio. Soleenity Mere loat tokks as strigttfomen and cancie. fisitt.
faa
inleaitg
contucivd Ther runaimi is sreke facines but marty consiter prosure which regpeis raakemiin then curenes. Thee develice exaely Lerteedd wit sight venitiees indiviing a aotlodia apprcick specing boters dirrciio acogalts
the
flod
exgenting
ixpeniedd
Belr C. Spenres and Finil Auneor [[]]]
InteraVL225MFFO A
30

---

Published as a conference paper at ICLR 2026
Art Castem Art
Quiltion:
Fiouc andoer lie stoki fezze and avand beuinroke syle to sed foc colana, loss sia 19d in axage domunan da contioail taalnnyy cxpravedd
Cartieas mediotio
Bi Unrrstriaed and ichess
Speccos and pitend
X
Di Sulexe and concise
InternVVL3 (with
V
Furns: -6h Row The croker raleivell
concenses saaly connolied Seros
evelly compliiee indrating, won lovel of undfal sanitkenizn To Row The mokkes doow bellnce beteee Mily and eabtlly "here adile vortatonn cospartiy anertivaii and cocma Ett Row: The tot appear roce riliad hes all incrsss
caiiing withee Serad consriiss.
Overil Sanduarea Siyle - The ayk floe eculi tu houma sngrod angrediattty fradeer ofter asocilae 8tt anssrrabiil sieils Theral rosilin sumpassi appeach attrrtce
fimellss Compuring
options the
spacoes
sutlcced endess Sumal salood ovepaliion Optice (Caxtras and restaaled) is dasie bacaus the unvice showr ditiberttoon couted oftier the uesorved not inrrety opsion This that emoeiiaal teadance apperted auttor thi saction
the now of colines repratezed
The varation brahemikn reficee balanoed vnrrlaay betwase taieeity and precsies This duilyy bealwart euggros that Du author aime Eamvey saase at consotled siore and theqgioull dalieeation: cnotocal sendass expratred by the aatho: soquudi beilgg statioe meticaaios, inclaased the dillharns crasiioaal mancs. [[A]]
ponerrfll snolee Sinngido
Janus-Pre (with COT); X
+ fanus-Proc C
Amnwm: C
+ Llova-nese nest (with COTT): X
The nut in fhe 46s colora. from the rik the 198 res dllles broderekke stfle the shes Meocceetedd by mak of and lold emskks The foe staios are csed wathe the mellir dhanitar whihe andls and lirge ooee. This sagpacs areeine
oproach
mrokes
femcatio essiing
clartty This lavel
insrtion
drsil is aesiovat with cunton and auticatoon onsetiiaal sedewcy The offi snciss for the langr charne alis adds Theaatere contect innter [PJ
kinght ieperzens Beitee which aodl intoyyonad at soceno
Llavzwnext:
Ulwa--mevision (with COT)
cotama, from be pony inbiza contenatios of deleas Innaurnikn srragg crastoaa uranes Thauster, des ansincea siddann supeniee by las aukari iassttillll tae rectoss of Buily uncornssed BREmN!)
1as stuke torms
truharikz sofo expconire eynanic inkiinng wnher
Benzs th srancc
Llowa-onevissio::
X ovicees from to lick wws are defioon cosmiiied with belenoo heoeers lealls rrosurs tnlis charazers veli-tmma ottiber aeberasaaaan pracision This nazzors aursati
Phi-335-Vislon (with COT):
The srikk feres
contioal kaxienc [[DIl: Solane consas
Phi-355 V-sttoo::
X
Qwen2,5-VL-B V-L-33b (with COT);
Feres.
sdaisdy unfenn
loften spoondi Thoo
dedicatios
trraburkke Styte the beatwreke whic would eigaos unvitaaeeed
vell-ceteed, indicaingg
has
indicatio af species nutard syyee wiech would imply
relaned and fink) agpeseds
Eavekool Teedins:: Grom the somete and wet ramer of ansioss at eupgets Our aefive was Bocord and pethage ovan casteee abeut rhdr watk Thi caald iealy sprsc autoorracco darie atucitiion which attno with option Candoas und axtiealoa Connlciing tuou pazzo. Ule anat liting doattiiie lor Dbc stndiand lasalcay standoad by aakie
Derionseed undiodiia ([AII
+ Ooeen25-VLL-Be
X
Qwen25 V-L-76 (with
appcir
Brrshamkk Style which oven
keslorokke enliere
churatam
3. Eratioal Tendaas:: The dynamic craiic trabarokk tisi vaika
"Tee
axcaates stoke Jeneiizadl inaul ancladiei Dezagtiill sacaveds seliin
Gues these desvvites . te fringg Aesuiptice the enotioal seelenc expersed antor thet Hign with caation and ruticaien apponacc Thoredos, the comnet ansoersc [[BI
Qwen25-VL: A
copplater orthowuts, siggriting ind medii appcridi
lirsharika The curstre mort indicanne halancad and ad Thene
reaalty waingg winch implcca anxuuma puzii
Oies fwne dovesvaiin ailens, etnriicnall tendnncy expenselly for outine scone be entiion and retiekkosee Dessone R
31

---

Published as a conference paper at ICLR 2026
Caltuee Calure
Queetio::
of skypailding site pieture
sacrid
Optiona: Trasscod tadkored
T. sedo waad naloggas
X
GPT-to (with COT): A
<
Ohsurvriis of dighalling chr Ta day is bing coestrited is nediioual rumeer, witl vicits woode plack Geening the Iell, The mndouesed regenos amooola weitd middoced by Bh yel bee end prosiaily the she.
Mexril ctrndtratoo weed sppan dimu skrsth,, seaabl skpbuiking. "radiional shiphakkingg -ptisz (luunced saloooof:
furahppaiiddg daa Bell deabiiity and soar motriree Cyeso i (Phee and coder eved) These ar wetrr wood and last coneauily sad for depbutdias opecals is tmppial iedoa. -pptte C'aaalooo cocann stall:: Thase ranaralle sdttilee coomrctin comeasity
Ellasaatic
commeoiyy asad
teglass
aushar ratertha vadiland expeclidigg
anae.
infeeeen Staekss
ceckeal mest mnnoeill the
ireeaced tnaknoood Fisil
Annwm DAA
+ GPT-4x
[IA!
+ GPT-411 (with COT): A
Stp Offerve the wwggggikinggggggg st/l and boatee. The hing Bill togiei coardl and. indented cotallly prooeee of peir
tres and leigh. aner cnstocetie.
Skap
audiiaad dephalding makemil tosicd regions tuxica regions, opacially in Seakhart Auis audakand
Aralya o loorood talcwodd York denaii hasherads conmucity regiical skppeeiiiing
iire is thir io nartil hevet -0 than oscar sweet Tho ane oxter soves mone cenmoo mniereeee regainc
and aos reluace mailee crdiniee C. fheekee caconus Bundooo cael conaruzioo lea sylialty bulte
eutagggg vaditivall mupical ihipbalding
shrone millagaay is duraaall
Suy Oueckaiee Eteas sopina velling traditocal ikiptaakkin ructbado anda zmantal An ball Bidd iumnnal notioool IAAll
+ GPT-A1:
[IAT
+ GPT-411 mini (with COT): B X
Suy Ofserve Becastonnees contrasii atyk. The loingg insilis trapical comiidiian indesedd by the pro of peks. luee and for bright. vanny wadaa- The semuecies are voecen ploni end born, lepoal of tradiiseal wooke aepheidding
saiible sepperadny
nonmetss mnd tradiional dinaa tnauical axoi- Few ooler acher woods mam cxesnce torpesto sepies end los meskal to vame cenditon... Barkos and cocnut shall are not matibbl for hall
stregis dumiiity
eaheppry stoong words
commeny
tomperue
Styp Marta wood tyes ingine vedincal prectios Coves the topica seting tedizeed déelwailing netiod.
trawwed end rodoooo the andt bilyy nstends caed be the Dieeciioo the caine matornl taed Incile contitation
dkg [AII
namo (with COT): D
x
inuge
alyltalding cecerration aatiininnnnn invalve curificur
woodd,, valile wooden cladte and scaftiiding
The eatting raggnic tuditinad metled d. Bully wrig locily aikle anard masiial.
Aruana optices, roet pocemo limerially and woods for muditioaal day tell oui raalopary, known for
tranweed buksoed are
dephalding
connent Sauhaas Ails Imazo speiity
mag suplors
Firs ancrafer wode 'ax tuabia for balls chauch they are aoud dlypaaiiiig ovnzes.
eocwett
the bull.
Crees the madainal aigleaitting concet and the apparseees of concrrction, the most dittee optos is D. cuk and mainegures whice am stank di canilaa lmpelartng, kall maditiona snggeekiiing Thanriis, the mxteral nost for shill)
[[D]]]
nano:
VI25-rAV...--MPP COT):
This
leasly ovallable mummall that communily foand rupiaal coaull
Vintle Masrilii The hal beity cenatrated with aug weecm plado and hare. These ao weedes sonfloldnng and parouadin sce inciado pain tres ratios tpcs of
nappeti creand
smand
skoteildin
bunkoe und sadla, which lighnrcati peniial and pupetic vezakl daee cemiccatin..
Obouraiizs of Sposile Matridds The plato and mnd madema und tpparto bs boot souns sagpeiing duyy us saluble for sadke siga moken makmaaa inerwod se kakwoed, which typanll
voodes shgpesildng.
Cives sovveiinn rudemes
ondiieal lond. and avorenmacia
hambau canmaa sidll
Barrdian dicoredd enaeeta Basbos saceea stlll [[l]]
+ InteraVL2S-MPO. C Baxkee and conoed soll (K3]
X
32

---

Published as a conference paper at ICLR 2026
Culrce Maneril Colmre
Questton:
Obxcve ds ccasa the kndiiand aapbeiidaa ale is pieturt and ister waacc rait zexiil via and cuuracciss algo bal.
Optiona:
Ineeccdd tncloood
Bi Plae oodar Di O. metorgity
Baxbee and coooae stell
X
1L3ernVL33 (with scedas
appart teditional slip cocaraatios The estadial ssid Eor sail consiaatioon in lcolaan claser lasad an dunil and avataailts Glese siitta wouder ptanis bears, tets solree th qpiion boonood and walnood Befl strce dasile wasds commoolly and akiluiliing especialls
Ma lighewright cen cestraatonn aaee but are lass uccince in toditiona sily tells salou yure speclfe neghonal certiate D. Duk and rulbeaary Knoom straph duakiiiiy these wouds communiyy
las furandiie
corpurad lapa C Barlhs
chall Those mnsariils could ueed
olgheakling, y thoaph thes typuually ioppaal aibergicall Connileiing
appoors dos thair denbiiliy trups and the nyw drp tne lively to conurantd pornroo is Sorthare Ads for twaaeas) fronsoed and taadwasd starn moie
cad
aprees [AAI
InternaL3::
the tmans spnomm dre to mmm caidy mooot oyeest for mnditiouaal bear conmettin The scathlding and mulsmale vitlance locally
dephaiding
asailable musorall
[[C]]
+ Jamus-Pros (with COT);
Aunver:
+ Janus-Pro;
+ Liera-nest (witth COT):
X
Rasel to trang, the cais cutellal asd the constration of the sligls bidt agouiss hankee and coceeat
al, wornodly teaitional dkyttiiling catton regioss Therston, somnol
Llava
Llava-onevision (with COTY:
wenden die under conuirction, The hull of made wasd conuitution supportd by weedon wathetingg The ramonding anaa sas paire trcc and clcur sky i inperall lrcatin Headd on the vuchi ralemas and indirmml cen mustee miched. man materdl eaed the smp's lacty communsy shipnnaliigg
maalinaal shiph sing
Larriian [A]
Llava-omevisske:
Phi-3.5-Vlston (with COT):
X
fat
bul), The saconeding esvincennt stdik
tndicaer truet
lsuatio The rude wasdin alank, which could inkmte statnaking. The plank unieem
which commeliy and
dffiren tips wood The prosenee of piles tres end be style cosatsition sageot but the skly being tell is regice where oct mus an commos which is crnstira with mmpical citeopical mns Condtarng the option prodlict A Irraased and unlowood ane ereny and drabl: woods tal they at opiially ausease with hagica sagees Bt the and codor weed are conanas is aaay oakan, bui they ure eot sceeg isseweed taloosed C Sardoo and conout skell buth suatide tha aix coceanay fooed tegii rngles and
traitioaal slylialding Ouk malogger are also stree wark bor ratmas vasill mutembb babatoog andaciable brskbpssihiggg
cormenty finnd oupical regans hurtoo cocoent Bared antirmmeral
the imags likily anrwer Burboo roconal
(G3)
Phi-3 V-Viioo::
X
Qwen2.5 VLL-- (with COT): A
hursad takkoved Thos
for ther Brabitty rouueece aworch arubii for
deptesiingg
Fee and mola wood Theie alu acnnsell usd shytei due fais stengh and res tet
C: Bandhoo and conamtt dull While hamber on use some apects daybuilting socama dull oyplatty prinary matmaal for dep batb.
Ouk und maboggay Thei ace spong and dustle werds, ofle usei unditioa alipballiigg
Cooe inng shows mediteal skylvalinn ukc oo mood soss de mas cormas deakle aututiis
teorrially Boot monnood and raaonood lapply masiied guslii in smppatiian tieeeses conciienne Iopucaa acking sagpowed para
ups oxminuctee kogiiel condiions Thondton,
image. kalowaedd rrigh mune commoniy anocaie ansse
wwweachiihhhhoo doite nekuol
Qwen2 5-V-33b
Senn2.5-VL-75 (wtth COT):
X
tranweed und satowadd: These lawon Balr cemiilie notranee wanati aline natabll for dlppaaliiig. communily foand boptci mraves whese
Elreoves they
locnod.
the onker weed Thrse counte awtwids and sigttiiiing big doe firi mnilibiity and good stregth weight odlon They se ohene use is ropical and tergura seginn.
leebbo and eocoeat del: Whiee hambee can to sed contracten, bustee, it tysicaly soct prinary matordl te chaphaldinng
Creanu sndl mightt und decerateo papsece but net
Oak and ralegary Thes alo darkle soods wel akbbuikiigg appcial dully odllee oiieaoos Rovove, Ohyy ore lass comman is bupic naker compusod
cader
Gavma En copical wriingg rugacied by pulza bon ty
immge appeare Sphechheot
lieey watwvils
pine
Qwen2.5-V 5-VL: D X
1he image shows atip under conutracion on bauch woodes planko and beans virttee The matomale appen be large serls whick are oypial chomiteriiio in of maaoggees boo coomoody used tradttonall slophaldingg due dieleity eranghh Thanctins ovrra anopur
33

---

Published as a conference paper at ICLR 2026
Quesiion:
fapee does domand demuad prodact prodeced ixcuace from untt
4::do
I1:42
C:
B:
x
GPT-4o (with COT):: NONE X
Sup Road ferada sad onvalooeeae Piice Quanity
Step Tit the pice vai and crin Thess page +Ac pipiiiic.
Buy Caraeesed cocoeee Srank a cats. Prie Quanity $2.x9 $18
Gaeeake scmmu
Quarity 51
chaage
pronaed.
reat tas valao. [[oeee of
muck exncet mswell
+ GPT-4o: NONE X
Trtd Texeme tuzto Aurur be pereis
muphs. Reronzz Quarity s
Shep Texd Besmoo at 18 At Euriss the a fiiii Do prpp). Truil Boccter hike Quality S) 518
stap Uherge tn Toui Revene Charge Teal Revens Trst Receme unth Total Roveres at onde Change in Tead
dlsge
graph optiocs
GPT-411 (with COT):
poiz catz
Riqanais pris
Cacalai sovena Hounto: 1K
300
Cacalis the chonge not nnteer Change TR Toaanrs SEREEI Eo STE
So the hoveno derman commue by viec sugaa iuruae mace fox bu unin. The versat sower is -ee
GPT-411 1. A
Poee the meti -A---E0000 AE0 DESIEE AUTRiZi SESSUESS
Change IOSI Fenonne
GPT-41 mini (with COT):
Mantly Durpoic uano Pore de gzads ei menily-y grice
revenoee
Benaxe
Calalloo uoveno (ounis: IIK 33 536-56)
Cacalaoo chinge rewneet Change (WrilE wwrsh Kaatssu
GPT-41 minic
Frooe proti ArStast yrre St 1arais the peino sraloooeaaaeee
namo COT):
appoomants
Caleziue Insi revera (11) qarrty
TE Irie Queerry
-13 Tore
10-05500
supua
Traeporadio: The rst resepaa Coumanu by when lncrasss ouqua froes ualss Flul aurver: [[AT]
+ GPT-411 nano: A
Al. laaii Ba baan the denand curec)- AU the stee SS. Total reverue (TK) is caledaaed is Price
Quaniity
Chang iovemaa
E.teeraVI..-MPPOO COT):
Calealto Ie rewaLe unio
Rovenar Price Quanity Total revenae
Cacalato de revoue Leka. Jead tevanec
Ditemmnee charge reverue
CChngg ge revense Hevemee
Elange
ramai arelaly: Check
cumnity valee decell
anmi showm chinge
gave optsen) A ustic There might be vecianss, bet itss axied price aquitioni Selins cxnecily Sor correct untepenaiin
EnteraVL22-MMPO: C
EnternVIL3 (with COT)
dexand oppreinnnccl
Reverae
9
Caleratt revoue bunio: Phine 10 ania fonm the demand whodis ippeeninnntll
Whhse Quadly
Deferenee deange
-Oergein
change frems betteead beret uni lover hutoonal bu antt.
V
Caralio menuue ucls Teal Revause Siats Fike Pradatt 4.19 36
Cacalie revoce Taad Bernceet el cati) Pioc vaits niain 31106136
caoge by
dis
(66)
34

---

Published as a conference paper at ICLR 2026
Economy
Quretion:
Faues Jecare prodac prodieed
Whut fechage thi
it Earmers Gves Cucch (Daai)
Optione:
Jemus-Pro (with COT):
Atswvr
Sky idetify price quanity el prodaces
Kniia price
Skop 2: Calecas toad revense for ends quantiy. Trad rewnee for iinza SSlack Stdl Total cevenso It
Iiaiis solant 380
Skpp 1:
change
Change rovone sovenso unto Siaal revente
Stop siace change saal revenae negative finss revmuee dencavs free 5 unl aiass
Sky 5: Tae gandina nko for change seeeae is outor axcmars from vatti cazs. The change menencii (AA
comvet
Jamus-Pro:
X
fins
For
(ydke)=
(eaashy) SIT For to suall ueverse 3Tipiied (opanitl 517. The change seal reverae $17
Llava nest (with COTEe D
X
Llava-aeste &
Llava-otswislon (with COT): A V
chang in unall roveroe for lis firm un inonace expee free 10 amies St. Elacpine anate is lnc peceeie fei ai price
quantty
cuantly demanded price of fecrmaee St. Uharkkeo, change
revorua
dermaci from 35,
Theeffrr, mnwer opsinn
Llava-oneviston: B
X
Phi-3.5-Vision (with CoT): B X
The priss Se ii St and for 55 Thaefors, the change rovers (583 CE
[[R]]
Phi-3,5-Visient A
Qwenn25-VL-3b (with COT):
X
Calesaa Rrvenss erm
Qquanity uett
Revenai Price
Revenso x3)
Quuenit
Tutd S3T
welodeed
Lark rachct ciledatione Ar Reveras 33
Tool Roveras
38 S23 The corsot caleniation shoud be: Change Total Rovenzz $50 327 dives optien providad,
Exer night be maraie polina skanoaal eptocs Heaneen havel be conaaatien donin
sooill
be Hi:83
Qwwn25-VL-9b: B
Qween2.5VLL-b (with COT):
Moniity
Nouutdy piicaal oegst.
bee dound
priceee
Seal rovenee Prkc unia Quafie et Rii $3
Quarity a1 555
Sky revenue salo
Sepp Chirge in seal rovesss Texa rrveese at aabs TesEimas Sacc - 554-5555550 555
Qwwn25-VL:
Revera Change Rovenae
Tell Ravence Price
$3
Change Hevera::
Hoveres
550-356m-36
35

---

Published as a conference paper at ICLR 2026
Geogriphy Hutuo Get
Quesioon
is ite store adte cookd, whe be mid is dee smis brcac dishting crudesial had tas?
Optione:
ansiiiertiee
CSD
beaing
Et Carolaa expession the
GPT-40 (wtth "Te
readd Lest dowloped
madd
peplasiin the ialleno inceperation stace und ohhe lane fnstace ipeza ommggreciic of sife
Key fistace of bhe medet The cestal ans (ibeld 117) the Cattal Barise Dame (CED). Seddon raduxe outrart thes the iillateed by sunepmatio atunse Gasy lass Reodccial waa candad var,
1151'y are
vorepeattion anstie scish parvica locets in jais pruntttes -Wolte lorinn tode lested akon honquratia tadting deanld anot while poice kaysing other doserta indatiial tours
diratiy suiterrial
ure intorticiaaiin:: While indavtial infucaee conieniial aue day da presinily ridi basiiaa Tiis poud of the mode, out mast Euans Shaspetaiio utanoo Thanpunaiie nako ure des primary Lave shaping the soden and nondinntll lent uac. Caeakr equcaais Ja cit thi daatho des consenrie nex madd, the onder modd. lik main flcrn dleaing reokintal hind sou makl inmogamation
Exdaning uptien-
[ID]
GPT-4.1 (with COT):
redd show ccnaal
which Sidyy Boinon Diecii (CMP).
the seckes
dirdte
cotking bra cenes.
The rod faridl anus (tarll ooee and 4t amszad in anttii in thas rariuutitin stalitiond the CRD. New, lett cowkler each optior The was inbertilaatin The rool does not scrisity ligtied industal lonas inftunce cn futrrs the
boadng The mrad foor poodde inferater aloes uxisccaerii oe poolesity nick your heuder D Trreputation americs The irsoperntion unare cat Drough the Ety and soem infermse the dags dirritation iuccianta moees which amagee abre thoe antriaa Cincalr expendee
CRD. prodlette
perfact
Gikan abona the muin dicaiing meddenia renegetation artiees srichh dhpe the deriirrtinn sccsrriillly belly malieetts anne Thorationn comee paosertc [i1l
+ GPT-4,1: D
Serte Model (hoyt Modii which empharize that resiential land use dreasee renrty Inepuptaiio malo or acenes Exflacel secten calondl outward from the CHD Blong Inceportition lno. attlaninng die medid [00]
erban modd shows
duttcciie modertl moau
Dome diciting rosoenaa
GPT-41 mini (with COT); D
medie shrwa crmaal lenate waich lalyy the Germa Thainus Dierie (Tin).
There didder
thioggh
Tax de maal ame (faely soss and amoiged wedge withes bakerin th the CRD alin inaepetatt (oems
cheoker suggers
sosapeaeaa than just
The modil coseaiii the Seater Madil of when land wiic ang nos Ohe rale of copperaaio mosus is dlaping rediarcia
(
GPT-41 minic
be Secter Madd (flest Meddl, which emphastioc talk inaupurationl meka fetamaciile stuping utben lan us ratiom. in the made nendental areae indavraal auncs and other tand avos etiend extaard nocies wedass men aver intoperation fras. Siven mass thane staaing ronaicntaal land aec
raaki down
nano (with COT):
Jarnd exdinted the putlor and luncs.
The namberel dnddedinaa seruin, with offorest
3, The soy feature appicrr to be the rossechal brea are amanged aroand the ocrker with extormm typo of soping poiriiyy eegrggated dutane or sccur The opteen ruggri vamoaa Uermo inctrnltuais uue LSD, poournsty not poer
innoperatio
consideiing tysad strutice eich modde concentoe rings vils reprovent ditteri lnsd was makuinng tives the CBl. Acedmnid land sre in saaily lected to speriic sttp. oftec influcteed by vnsepertatoo coues soct cocnenec tactars The made scome erotiatee be pateen readentti seem expanding nahward, puaility iellemeed by tranperation aneicc does' explaster mxtutrid apeciiiully of ridi to pour boaing could a thane but dagar dool't caplistly dhow sict-soeenns desecienn D. Itanpetttiin ankite and promincti
of the CDD aut dnvely indioned
thene sosuunty
digaan. wish moite caduiina salward nagpoiig inflenne rendatil placenani Carcakr Conndatics diggarih umpdaaic
inflenne santanna mai thane Boor pasi pasiperatze ataiten shape nosketal land asc Theraker. de bot ansonticc [[0]]
G nano: D
[10
+ InternVV12.5MM MPP (with COT): B
X
Iccerit 1hiee Parsan The daggnn does censuntic chiles wakkingg estear fro count poin, typical patur ofulue sueral dnuolqerar
opertion
medirtal mos,, Herikor
soroastnng liffrwe civkss
The cin of Cars middeetal sas appaers onsicent indlatin& exfom speel arwor. The levr diiie sree reprovei
cum aves posiily corerecia sabenid parpore on prosber uoidecbil segaees The cate dedee grets of
expoollin fecirdly cazgaeing
Optees sie tf caD Nor
indoniaa
modd
fweee
CRD i (nxetree ceant polaa eot coanding resiierol use C poaliily of to poor bassnng Not doon dicicety model keppaas auiteer oiteor dert distctioo by sodtt piecery wosierii
Tranpesation une Whiee made carid affawee petion ontwoal denda grod
day
Coneell
Cessidniing the
Destook oseisies Thas, ansoen
36

---

Published as a conference paper at ICLR 2026
Geography Haston Get
Quesioon
is ite sture alee mookd whe coold mid smis becac distting cradesial had tac!
Optione:
CSD
beniin
E: Carolaa expesiion the
InteraVL2.5-MPO: E
Clecalr Do InteraVi,3 (with COT)s
The urca iskariiitattee The numthrr indetratioad avas daza infiazee the detiibttie of reddariiaa jaecs diiedly bslasté aceat is uthn rodile usalty appear futter from dhe cover iekkeed raterr than dccating harceria lreatin disc
(Cenal Butioo Dutrid) Whils CRD ais reased the evcual atao city door rot oppiatty
upp
commarcil sanzi hei
primurly
e The procents rich to hocditg -Th The otte mode doee cus apper to indices any presinity nic and poer heartwiing board on deftool puston. dirrtiatin lasty arm for more equtatic sgroad or sertin atatagy nethr thin dincc sugropetior
Tranapetation antttx The innepertation motel anano, depicad coved bas,
the rigniiio
avettaation of nontenta sonse Rtedertal mees onten soloy rujer teroperatin oe sonts prrvice scras and cienectiity for modeats param slige well with
afte ace city byyrre where coreprmen boy docemneare ronionnta
Whit uchea modic otte cocolor eapasises the one equlatty eviden is the amsagmmee modetal nowe cuarbos and
Clecalir oquanss
Thxe
auserd
Blsad the lyyur trags, star rendertil surar duarty aquatned aloag swecte traeperntin ce lank optaa
anstien [DII
+ InternVVLS:
abes mede darn tee immar knkerdd bun orpaniation and warscllaben of the cby. bhe un of sedon and anos to delinaee di difevel aoer poina to acsckard apprach aftee planing where roskental aras as opecally sruatint awend insuporatio atciis, faclikiing consesing cusir indarmal aascs wring bub lven which wkk madeiaal and rabanrid beanch out Wise tranchng pallm concemats redall calanding acoon aggpis salana but qurtttutiiin ralvorki (polniiily madic ratlveed for roriuetal cppaniaas Thei milee ("mmsppmaaee arionia. mast enean aiggga with senuttent and vammmna mature madii
tens cistaing ro kenniaal land
Jamua-Pro Pro (with C0T)=
the
land
redd
slows verke segnnnts with cilieeet nanben, indooiing the importace madh and pelkwaye conniiing difieee and Ther erinne feslitre insemee and beosiellts, wach ere creae for raiientel land use. The optioss seds prosintl
beuinng, ont stodle expensioo city darelly
Janus-Proz
+ Llava-next (wtth COT):
Llava-next: X
Llava-cnevisive (with COT):
Llava-onevisten:
Phi-3.5 Vission (with COTYe
Phi-3 S-Viion
Qwen2 5- VL-3b (with COT):
A: The soae of iindutattniio tho option ruppoi the indirtual actirtice anc the prinury feese intarminn rosdental lund Bloeeves the imago don su wiek un incestial zana (acerios
occenrcial aciviries or reay influesee residrciaa patura aoaudd i. Hooeeo, this optice does drualy abbuss nanderiaal us Itdi.
The of CRD The seruel besters duaria (CRD) reyually
Tae poocicaly coc poer losatgg opiia inplec secioevenomi grddici moddetill soc.. Whsle dai plociigg koage
earrih ixdenstioo
patcre
muggos
Cioalor eopeniion queiffaly albres
descches
of undde infleenc
rosseriial land
other qnion.
Cocudatica
puakan QwenZ V-L-3h x
D: Iraapetiatin atance [[B]]
Qwwn2,5-VL-77 (with CoT);
The somo afindatiaaaiiio ankortid aones err tepcully lootedd ou for suskuts clio dar to fadi need for lege spores end prokirity to maaopetttio somes Fooowwee radel does nor dovse any distert soterial susse
de be CHD- The Conmol bounen Damne (Caa) srally the of's city and ofte exmonnied by nustemaa arme Eioweves the modil presik cnenga infermatoo aboat
CHD this prinery festor
The prosinbl poor boaing
dow nel proside myy
soii stutiflatio
This restiraaa
wlish
Chreutrr oppaaiio city- While wezz expand cecatar pakeer model starty depest
cxpanice. appars sadal expanding octwand fvom contel
Ctee th Ewaraaya plechhi biekeeee diaiiing mesiiiraal land tix model presanc tnaoppptaiii anarios ([P)))
+ Qwen2.5-VL: C X
offfecn levelo lalaeee
37

---

Published as a conference paper at ICLR 2026
Hasuay- Astan Hittoty
Qualiie:
plaze
Optints:
A: ArPreggaah
B: LLagram crepesrd
D: Bernety
(with COT?:
Uskkttasing the splice: Patapaph (A) Glaratas farnd dixell repovalii eidica at scnapo vinals.
conopp..
Setaal condenin plande deses (iklniing wead) tnd senari: dicecl (indiaing mwaiig). Bonewiay (Ds Charaker benseal bon-eker legsago creesss.. alice with chang is recsing.
Auatenny curpue stamer mage appear ts posena spperemniio mast armummiler necct visch slgar
ota pertraph.
Crevister: -Sire chaneee sisuly aepeceess olject beacc), oneeeedd duarete firatic Putegrah Eisil
prsee Bitroagh
+ GPT-4cc
[AII
+ GFT-4.1 (with COT): C
Patogrphi danacers thet viunlly revenble bo eijera they repeoset Fer eeronappls the chanceer dee "see" lodk liee imo. Las chancier the iaage dess cet ckats reseable supllc otyed thinkn
Pasugrpa (oynnn Al:
Itagrare (Detion
Mangraua sproem aberact idaa or concpte raher than coeciss atiize They are afte sinak "aap""
kariip sbannc cymbal
Phoo ecarti compond (0piie Cy Paoc seruri compondd an ile norocess compesee of parts yor saggess the
menoiigg pecartk congoaett, the caggeos the your aacaaaao (plenedo conpanmdt This the nnd conioon ceflod diskoct perti,
ioege aypert
sesuuls
Homrnng (oytion Ii: ssmoving eden
moxing bact mand wabhat dianging the
sen The is ke shax he churder's tm and mar about cagee rost muthed oreerraee Sommation Phooee
Oive the cargleccy poseess wan. comporain
[AI]
+ GPT-411 mind (With COT): C
P.Pagggp CCptoa At Patturipee are danckar tua veally marris the ebject they reproust For cuappi, the chaneer de "axe" boda tns. The chanser inaas dom cleess rounbll cxpte objeci crihing traggee
oaaple
The chaneteri
abemcc syabal
Pooso accuntic compoind (Opiin Ct Hore semmti componnes ur chancers ootpondd twe
raggeta the
monning (ecaatis oompasetii and be clan saaess the prosactttin (plencls ousponcei: Tha is lis and cenaso ea of facsaie Chinox. chandu ixage loso coald compoed two partu, vox
car thc
Bamowing refore
chirazee
haral soaad ctetarity wihhet changing
the fim The trave asee maog the fomradoo
Gives the of ch cosi thely a place weseeveeeeee vesaaii componnd sdea and gives somningg
ferseand Thenderr. seune snwe
GPT-41 mini
[AII)
+ GPT-41 nano (with COT): A
Tas quotion sitcabert th matted curacur formatie in the tot ohe picurz
picce ahones Cherms charator, which appear to he explittdd oeabiiees bemi
The orteenscre ore Phrogragl obars les tul ere Ralled deoeiigg of choers 3. ileggams: clonetees oepecien Moee
eecepex ciuccly Phon-araate carpecrd: Cheaaten formed by combiiing comperux tnlliainng promuddaioon and compuner tadiainng maming. -D. Beroning: adapiyy chundte frves other lanpaago serpt.
Clinae ctansion
though racheds bei mai common singgaa planooemanii camprunds. syllani posiilly pitepoplis Meegrati
appers
reporecttioo,
x
Interni22.5 MPOO (with COT): A
The dlerame in image i gitterrgh, wisch menas à vus formet drnenng poice monppeienn oomee. Tiks
Ba pamue ancaa Panignggy [[AI]
Internw12.5900-M00 5-B-MPPO (with COT): B X
kemgnam (350)
InternVL3 (with COT):
X
Pesugrss Patigapke ronbel dai roceble oblera somns They unally stmple und ansighieevurd ixpoostatios The chancer the piease does eut rocebll as dose er aces, bu saker condino darara hasna
Sieziler chancirs Movgra spraii Jurad
Whik Moggane
pidegrapl,
iehocrily quevir dape twarngs.
thanssectaris is compand (5' is compiet chenzier fiom messingl pen (cmanic radical
wiich iommeiaaai Jquane Kari comple Clines cumnarr faing this eataeey.
Bumwringg ng (by This rélm to sing durrde them aneber lmmpage vibboat apsitcnn danges in ferm. Wideou spocifc cresal chamseer oegnate from anather Imgugge bamwwing decmstio litinn
reosgaiigg
38

---

Published as a conference paper at ICLR 2026
Astan Elotoey
Quotive:
The muted of chaneter formation of pictas
Optines
B: Llngraze
C: Phanoeeranticc compoend
Bernntaa
InternVL3:
Panus-Pro (with OOT):
B
x
Janus-Pro: B x
Kannmill il The chanscer the image is kemnd usng adegaams vinca are li repoviet wonta conceets ettt,, meched durtstt prcographo, rmeges reperctting poptt,, phone compoends, which Llava-next (with
Llava-next:
x
Llava-oneviston (with COT):
x
Llawn-nnevisoon:
Phi-3.5-Vaion (with COT):
noctintiio
duretar
demad contining maring
Thaxfen
Co
Ph--3.5 V-VVlllo:
Dwwe2..-V1--3b (with COT);
Picoggapa - Thi refos deeades lei repront de edto ilk risully, Fox euceple the chanexr ''Al" (oel agporcao
percee
Lenngramn dtanctor repraate aberact coners miher ptyeal stjens Ior compie, the chamcke 'X"" rupresett
compontd tyoo
compontt coepoeen plontii
liking otee
de" comibeu fhom Bigiish "Numppaec"
Oiess churdte
idoegarm
moont Taurston,
cenni anseet
Qwen2. 5-VL-38c
(with COT):
Pitegrph-
welling sysen picters rxpromnt
inngr
appiurtii
Paear
cadiins smmuthi
acivtto
cotich indicta prszasttioo The clanater
abera
could pheame: dinen, clon
compener makiha mannng.
Bornsing This rofes adytiie clasulee lvee audtes wriing gyten. The daredte fr appeas be
rative Chissio chaneeer bermsding serellidly.
Giee the wabpis mas mane spicn mabedd chacaee Semmation
Phazooor andi compoand [[[|]
+ Qwen2,5-PP 5-V--7Be
Weneneek Stratars
that roumbaa
shape kodo Like lesus, whihh llgi wits sowning of chinite theer (illnng) wass "leeet Chiosse
Mounin und Reprosntaiion Pastarapi offe
acian dandly
Conmanine with Other Optsae Ieangaan: Thees ar chancses thtt cent mtse cnscept siea The chancerr
eatraae as denrt expessmaassn of stacel ttsm-mmmntt oorpeand: There ane chanetton that confene a somantic componell (ldieaiing the genmel renningg
BI decs not
plontts conpunent fedketing proseciatioe) The cleat phonntt coneeeas Bas aqanas from reroruaaaion Berastng mivve shanatior banoving nct apiiatt [[AI]
charesee dass
it adesian of chanserr anadier laguape: Taccomee
Oivea anlysi, chanates pkttoppph.
39

---

Published as a conference paper at ICLR 2026
Prychology
Quatien:
Whish fillwwing not dieners consgt for Spuan
Az same habooter orher
Wa inagee how appar people arondd
trarpon othen noetins.
devulop coccue.
GPT-4o (with COT):
TTodkreeelaa radislegcal introducd Curee Honze Cadles Thi coplaie how andvibaa dockip fuir cllfecnicpe lued as thiir perieptie of how afters dize them The youers insahes Biod
desic
mas alavare: inaiie bow argua aifers ittaman BI inarpe sourd itferrn Ct
Whik momete haharie
spucfs dienet docrthul Corlay's thears
Thas cernec
[[A
GPT-4o: A
[AD
+ GPT-41 (with COT):
rolution creept ilestmndd the which
linking heneliie
afficee mimon, poactly refleting diftrine peroptions or eiltaia
wo surcer Iolunaee of atos This aggais jastinigg wtha belovier The fauar saves and infontiaa unlace lisa autiotica stigtt winch tim
cenopp
"eskkryytan adi.
we liney othecs sectioon This boy looking glass with we kneryse how others nect to as fort our oif cansogt
develop adfoomgee The alimate the precem cendlping adfocmngtt our porppionn
and coban (Tecking wate? the eleaaets
how
the naston, and devdlogng ciffccrcupe Minscing other bebanie sur the coreapt. Ihaliation the cornict anveaat
GPT-1:1:
BAD
+ GPT-4. mini (with OOT):
1B riim the bouste of chenc? Fagaees inviting coaet besssdas The figcee Esoos a peisi looking bemeltii
miiress intictin diffccnt imaga hhraet which rolass mase sall paleppcoppiin ramoria otiors the rightt nnt be part
ameed Thicallg wiech ica lay par: of dendoping sclfeanccps.
"We treatie
cliskingg cnoulf imagizing othen
Cu inarped "a
about andermande hew supood as
Hugetryeap waceing figure abonce loaking hewdlf
which symbelous sellecticcton trnaminn
hew othre soe -merppaing othend ruactone-C in part the procus of feminn edfcreeae. - coocceptuu a auff-orniue
(D) sendt Mimrettz astariee
aiguter ether which disells wibtad procee
bf others
+ GPT-4.1 mini:
[IAI
GPT-4.1 nano (with COT): D
X
"Mo caree behuniie of cshens
Thi dlign witk the sies tut audividuls obseve and instate oten'' Intaniies es
redeaceeing bocoddos.
anagar
oppece people
comppnnd be Looking Olos SelD"
C. WWW othes' inesiian.""
lnkopening other' mnaciies abo be pooo,, aud inflee pevertins.
(WWe de dog exef ooougg?" Dovckping sell connet is wvcil eahcencol of is powporrdoooo is preceea upiion. camidering gazikas
preses selepeoept de preess. whike D Loviikn fxal uuloea: lhe cendepanci of ail souppt Stace gasiia udo in the cleual pat tor conapi diss de figas, the bot docoo lhe uae dal manz sanaze sake: tas u skyy fe preeos. Eareloas de chamatic sooupt gelock Seaecoon poscoot deokko atleermept umment
conrept. optins
dkosal
1nternVL2 MPO COT): A
Wa mime halasker sthers Tix
with initaing setlactng bduutios of echor. toor Eiceet
Wilaeso in bioe canaihes in yiue oelose eiffcreagt nightt tethersndity the stoal reprrraatid
We magits looe appar to paople arand The will the figre. rimor shw dfficen veniooe es We insepee other
The would modee indernasdiny othnx
which taftuace
perception tex't dieetty depicedd by riiiirs bet is sebadd to can atticercppe.
dendly a sibooooegee The dieccty relaed the riimoos esiing dificeti bestionn anautf inflenee and offfccscpps
Grves aubuaa: Optin Ricusss bikanioo
silfpaarptioo desngh mamoe. Opeion solans toughingg
appurrencan which mummi dias Opeius C inviless insepreting ruacionn which wint druatt viralined mimaas. Oydic
ralaes diseets davebosing selficecope throas Iepromaation.. Thinifun comnci answee mome apparcicce
+
InteraVL2 MPO: A
ralivv betariee [[AII
40

---

Published as a conference paper at ICLR 2026
Prychology
Quatien:
Whish ofte fillwwing not dieners consgt forn Spuanl
Az name habooter other Wa inagee how appar peogle arondd
irarpon othen nartons
devulop coccue.
EnteraVVL3 (with COT)s x
eitfrret extt. danding bar 1ha iLestraiia explasitt uoil smage and poreption of oenres. We numor the hewsii of otbrn. The coceept illarreed too not mimes defemee sntto, emptaiting pesoanl peroption and appearnne, the
dowe
slatation
minning beasttr
dffiren loda,
diflees catlie We ahear reutions. Tiie dea
not show asliale say atoseion with csain to ineeect auntiss We deciop a veloesscey - This is riceatt as gei skove citiennt cadia paully entng dffiren facze rrlal reloemeer. concept figas iaaa
+ IntwraVILS:
Tae slany ikoank the
pitte thmoagh ciiouss highigging lone intivalal posciie fomddo and whenn veunden. Junan the option presades: We samor dee icharie others Wi raland butbirrniene laaring!
apear pegi smend implied aeligerpptio, openaaly unnembaciing hes olhen mige now Dr Mo dockkp selfeenopt. Thes a dred oxkoore of peserinng sssell which the can the concppt clarreed by the mimes. Ion. cansd inaeer, sttees tniba figans mnro
fipnns imagin
foure
[AA
Janus-Pro (with COTT:
[anus-Proe
(with COT):
thaneshec rdaaes
The shoss
leeking odleccioee which vixaal
resvosciation acciooonees The othrr sprees as air decely salaed to peffonccee mumer dhe bahanor of other infea procre intaase lommra fam otars,
deas imae. iragine ber appar to pearl u. We donday adikarmttt ovenechiu duil indidda ather skrwebs bui a not spaifi skanett being dquiasd the maas.
comnites, vieck
theme
Llava-nestt
X
Llava-mmevision (with COT):
Liava- anevision
Phi- V--ssoon (with COT) X
Iau sanses muvee [JB Squre doe sut ructio de develpenee of sellecaccpp aas 2f ceneeci The other optiaa
(A. is, and are manisone deseer of oneeg.
Phi-3.5 Pal-3.5-Vislom:
Qwen2.5-VL. (with COTy:
option orggant
slips
people pamoon
oplin looking
(WWe seahoor
rmmliss undentaoiiny lonuppaing
othn.. Whlls onild that bebanter
infoned
dux dinatty
(Wte cendap utticenapee" the eptoo vulirs fumning an unvir undenttediig of cearlf based as varses caper ences peueptone. The image dicc depcrceeppiiitty thow the praccor hat can be inforet acpurt of exeual concep antinnannnoe Cives lo aralasa de opike hui leas. drcely rebital to lile venapt dlgpccid the lemas to. C: We incrped othas' reacoees? Tlantiin + Qwen25-VVL3B
X
Qwen2 VL-77 (wtth
copsiigg
seff odlecionn which align
lookking rncott mamorr connterin
Uption We inlepert othere saatirn figure diccsa dane inforatioon
tram ochene
only alane Do wenan looking lxnelll facniisss
Opdlas "We devilog a oo This lnclos furning un rueding of craud of beard suf-rctttion whiidh evidest Ba wanant' vasiua poase and capenision..
Gioma lio an cleaeni that prrsei figan
+ Qwen2.5-VL. 5-Vi: A
antppeoppion and salleraaa. 1he opiinn prevaled relie difirent apeca aclicennrpt dnvelgenmm: A. We the betusize etiens -t We people emand
themsarree
diftirent which
imegie hew wo
We incrgot otter meciiess Di Wi devilp seliceniept Gives Dene desr detcrptionn opekon dean ne sam be dirccy ralaad to procar docokelliz adf- Sqpicad the imags. Tesiad) Bic alazod with Iww.ZR wwwaa Kasting sacher adfscllatie. Thacding the ([A))
41

---

Published as a conference paper at ICLR 2026
Gengraphy- Regional Goo
Qustion:
"The
Stayy &o fomoeezkker Tar
Arabic
CCCallena Notl Aanin Makenala
Questive
Optiom:
A: iati y
Iesiral
1SAA
Co Tuntorba ware of Madao
Di Tintaedd waze
CPT-4o (with COT):
of Calfema::
locned boteem Baja Catferna
Denvrdty:
Calfinns
crmerdal Medic
comitived Masint Foroell
Optien Analyta Opiic & Maaiiaa waes ot Maiio" digs wila de anopsphaal as puttind milis ot the Gull of Ammuna" axcned bemss Caiiieria au slan wates Cytin (Tennlend waken Uniol States Ameriall isoance beaac Oui Catierta i ns ef lankey. comni annoerts lifice isinn wamer Meccn. [[AI)
Oalierni Optin "lacand wato
saaind muller causry's
+ GPT-4c: 4cc
[AII
+
GPT-411 (with COT): A
Swy Me Or lostio the Gall Calfeeni Do copp The Gulf of Calibenia loured botwes Oo Baja Callfrni Prokssel and unéstred Moiio
Unserene pulnical
uvent
Cations miedds wakiin tmikey
Uydeoaad srrazebyy brasd wass insma waunoo
bedwaad roke huailne which
the sentrial mucred Taoe couilird pat the sssestas smine the saaa Tenboril waurs acend
rentell wille fur bwrllle
sooesaate raas bur rihis Beoign venut Since Gutter Califonna conadree
Mesiioo
gulma of MEMinoo BAII
+ GPT-41: A
Meetty (with COT)
the
Bija
crnikree owentyl Tonbooi
maatcal fom Nantine under cosuuaatt widke Moncc'' beandanc, it docd inkand
inland Stap
Sasce the Gulf of Calibeetaa inconed Mesica
wutes of Manioo other reaticcd wriees llbesdion Oua Ciliioa cecesiieed wonee Misical [[AI)
+ GPT-411 mini:
COT):
Califania
fur enaiiil semtory Nedkee bondoad Mincia bott
anines vakii Meciaa chankes, consitoned Mf tnueral
senturtal oce Menios. Thersker. contoll
United Sinka. Bandd
dut Calfere oonslend
lesver A [Al]]
+
GPT-411 cano: A
Gutl Catileetta
tigs Catiiemia Penenaza rainted Mouou oumacnnd Mouarii
iioma watts eot epmins yourr lemoond wares Uniid Stals litesterr, cuncet cunchaso BAU
+ Intermol2,5-Bh--Ml 5-Bh-MPP (with COT): C X
mep betweee Mesko and the Bay Calffrni Pesisill County Labik: The ligigge Mecka asi Dats Sorte, and shile the Gut ofCattinu under rigic thrt durid by
Livetice Jenndvetie:: The Galf Caifenza ahited
thas te contricc 3. Cimaat end coapppty Canrec The dat Caltomus to egritier fatae Manos nemeng dong th
waten con. scpening the Bigs Calioria Pesinda froe the Mosicn mailaad Tomooraal Conrel Olemn dr prograpii
lration way It's proanted on map pan Meske's basdess typoille reaggnne as part Mecia's Sentorill
wales Tharadonn barad an augpaaty pontas: Anown Ao soans Menicc cradiaios reappeeedd ds pusitin
rup tradtional chaaficaiiss [I9I]
Interni1. Internei2.5-8b-Mee 5-8b-MP: C
X
03 Tenmurta MoMioo ([]]]
42

---

Published as a conference paper at ICLR 2026
Geography Regioni Goo
"The
Stayy cockkr Gall
Arabic
ofCaffrra in Nort Ametiza
bie agtiitt pf Musiss lol shac
Options:
iilexd of Modic
X
dut Niials
C lammaad water Masio
X
Ut tomonial water of
X
IntecnViL3-0b (with COT): D
x
Geepgpphial Lowades: He Guffed Caliéenia, the known Sea Certo, a lental bowear Bigi Cultioii regic Kocke and sefteateon Ohine Sans
Temtural Weare MMsaoe The aued waze slon Malk Caliieria Pesioada bakeg to Maiza, Maizar temteral vairs Laited Americ waiee clay the anaforeaton cear
cocad up cllo
appar
Colf
Fousdation of
UTS
wakes. des andyis kads condinde:: -Badd
descibalaza alggiing
nustal initono Mosie and Unated Bida The Ouf Cabifonii cleny dis notiaon teradd Unitd States
specfiilly Kasiifel seenkeid of United Stones Americl So. oellleoooed lisven Ri (0 Testlla lecles
billandd Staar aramencc [PP]]
TeternVL33-Bb: C
X
Gullar Calffres
Cawffraltiei Califieni, stich Tleeforx
knooe (2.00 CoTarrrtoi
Maatoo
Janus-Pro COT);
+ Jamus-Pro: x
+ Llava-nest (with COT):
Llava-nest:
X
Llava-onevisionn (with COTY::
x
Llava-onevistone
x
1hi-3.5-Vistom (with COT)s
x
Acoedita irernitinal las, dee Gul calfara canadired part commertal sason Miccas Thacling arani Ciailiiii Tommntal Mulkes
CI)
Phi-3.5-Visoon:
Qwen2.5-VL3b (with COT):
X
Undioonniing Callforr Culiberia
aay Galf
Unio) Mexicr.
Temiarid Wates Delaiioon Tesleria vileo the unand aoutry's oust falt esiods rautied siles from exudiee.
Opiine: Option viles Maind-
ixeree
Gulf
bearse tncril vates
eedened bodes wike such likes riven, accade hadies lik Out of Catfarna Optice (eentrri watn Mrond Tas inomrs hoare the Gadl Catikene aut consdeed part
Usitl Stans Optic
United
vemiveaa wates.
Coochnier Band on the anilsis, mox the optinn easerly dramiie foo sans
Caliorti Howeees
geogratena
splice burd
ootios would be: Noar abost Roveees ause gpoiiis sike cocilaiee beadiaa deaai angerphiot.n sollits apperpaate back an opliona mnald bo. Aanesai
5-vn225-VL-Bb
X
Qwen255VV 7 (with CTT): fimurtt Galfer Caltaraa
X
Caliiieeia
batooe Mecico and Unkad Sunae qucificlly the wan
Eauuina optans Dption Valand
Mustasi Thi resly dall califoria extraly withi
Mecicn which me soursi abes pocads was latie Tose Optie fnnemal vaam of Cartod Seate Amatoc.
woeld tipl' the car Calieeria enrnay withe
acummr Fytio E (ommrntaa
rilf Catferna
Mestas Oppee
limee fous America' soild
oue the fulffrr Catioons per of thraed Steae tembrral eaure shich in our hool on
Eaadados Hurot rup opetons craltert door ention Mitics indtoring
Qwen25-VL-7tt
uad baroiialal cuntenl
labad Wasre Thae are vatee dhs ar compleely surranded by land and are not conseced to the ligh star. The clui Califrna 18
paies beaase oenera Fastlc Stesah Staat Magallaa
Calfama
becass
body.
3. Tunmeriil Waken: Thore arz vaker the adean fram De havdin the baak to eaaiial miles The Gattet calieeria ralle
Kazn Mexso, locatad semiion Micio coandd ressical milas
43

---

Published as a conference paper at ICLR 2026
Art Spirital Art
The luagrege bose: Chiarse
Whut chanczrito down the firm De leas rowercir in de smedd?
Oswetion:
Oplisns:
fils 4 10 L /f 9.6677
Ac Vihrar vialts and growth
Bi wihesd araarm
M. 201
C: 2.30% 1r
phitopph
It The dookree winteer
Di of R-d
GP7-4o (with COT):
B x
3. Unteraassa the Lotee Tlowers The lome tower tn the utterd appiir so be is satorrd mat, wits droepeus pests and atrees The regpora lack vaatry and goont, indicatus tanatse desime.
conways
3. Pulloppiicll Undervvrs: Whale the urwerk mny have ethmraal quikty the faoms walmeed stne lotal Bovers wiuca gessasees inpernaeess and hee spdre lif.
Haac in thoe absur alaon chanzarrtio thoom in Sime Doneze amwarc and deseciee
[CC
GPT-4.1 (with COT): C
Sup Anayyo the rmage exnter steork chous lreu frecc depictal in truddtond etfa, with fheas dulica draice
dispet curpestice.
Skyy 2: Cuende: De spudslaz Bowen, Lotes Lowess sysinaie parto, upiiked swaictingg texts in Apaase deewe philsppkcal quilyy
They Requettly
piolorppical
fhateee optieu Whrue vaslty grotth flown
whew till of
masut
wilhord baic sanmen- The favere do ladk witered anternal Aorme churudaoo filed taxkm pinlavppy This natcoa the dele sondelan of alwokk D The dookiiie of wisee The atwech Jos tel Joadaate wiste: atooell leas Dovess da ahossloss Illed Eiitan pilloopls 1831
Exews. Baal aod
GPT-411
GPT-4:1 mini (with COT):
The lake floe is oten webdiiss paila celigoeerees and opiitit esaioting osperals is Easeet philwayly
The fur laus floees for sttaal appen deis sonndha abstas wiihh enrenkk seese ahheennnns
TTe cheellaeph
This regenes
palcopple
Vhore viilyy
feoh lins flowes
The sifered
aanees This woeld lngls dhe flwes deasligg dikn. -C A semo dobornaes fldd with Butten philoppyy Thi wit de opiital. phiemphial repovesation The drukato wreee inginn hanver coldees which intwerhas physpal
depocied
Therrion
GPT-4:1 mini:
GPT-4:1 name (with COT):
Wandd
wolhord
A ahmmaaen tiledd vith Eaam phzkoophy D.
dosiatice ware The stwerk maanr lona toeess waic in onin spntotic in tsemn ar gltbisplyy Line tiven trpucily gesbuitee munty exmmal awatocnng algghtmme chnn arecate wot sear chaeazae and tutm phtiesppical suzmmo wtiterd, whit in deats indlatie. Opton (A was of cheralso diked with Eanom phi phicocty! aligu siill with the snditionll anbboles lasa Dover Harken an, ranossnin party and spiraal iraucsnlans. Opdis deuilatos of aeaal tunc arwork, the charastotitz alown mast reasdd altavatteos fattern phicopty Theraten de chase [[C))
tradtssal arptarzing vibrasy- Oatan (Te siterdd
uslen
drad
Boocn haren
+ GP7-4.1 nane: [[c]]
IntemV9...5 P-2--PP.. (with COT):
Vibeas Viality Grooch This suggests a liveyy and kesfly appearece, with shadiodee and fall Mourn. The lita inage
dony dtrioled firms. The lees etteati fon clarly this
Withored
choeereieed whiiaed coles
3. Seao Miooeeess Fillo leaten Philoppyy This option reggors wore abovet and syreukk opressnatic, olles ausecrted the
plikeophzel iipalienne
caltuee
afie depinee sappe. beld
The Destaiin of Wane: This iodustes stall, hones apperaoce toood of winee with misinal deal ead caited oales The lass in not slowe la desekit whner sao Resed on en amage be kes opeers to syntudielly smalesd widl bol limm the
foos
spye
counry deeps iptittal piolisephioo seakiggh dallowaae
IntemVI2.5-MPOO
(hilospty [IC])
44

---

Published as a conference paper at ICLR 2026
Art Spirital Art
The luageege bose: Chinrse
Whut chanczrito down the firm De leas Fowerve de smedd?
Osaetion:
Oplisns:
fils 4 10 2.4. ffft-/88
Ac Vitrar vialts and growth
Bi wakerd araur
M. 122-
C: 2.30 1r
phicopph
It The dookrre witteer
D: 1F:00 R-d
IntemPL33-0 COT): x
Visai heperiee: The lasa flowes depine with extit olegene and fem, wiich sapeck zathis clatedians. Te aic Imall stotos spean colect poserfid listingt sesccal bengaliyy nefinld hoaey.
Ceriottaing with Options
vinset poods
depicie
There isfustionn
if dews or wilterng the lake flowes They bodk wheast and fill Optice C A sesse cieeiises Siled wits Elatess phinuply! The deionn slook,, the ofgine Bowee seme depitionalign well Esare phitoeply -prise
appiar
cocratiig
3. Fiza Consdkentio The lats floos the emead Klow digariyy with cedesiated dogiarr segpira over perensd elborll qualyy Besed on be realyio, east Oitay therette chareteeatt late dowes in arhook depiseed Optia ceaeer (C3
vens
phikouply
IntemVVL3-80z
Jamus-Pro (with CоT):
JamussPrr:
Llava-nest (with CUT}s C
Llava-nest: C
Llava-covviiion (with COT):
Easker, philocephe floess sniwk depiced war consgy sene pesiviiue of be lsta The Hick zá sirplicio of sxeposiies contshaio uvendl Iunquilli oeecepliton olles aucentte with taskrn plakoophy ilberiion. decomntt exsserss!)
Avame cioendaoo
Llava-oneviatin::
x
Phi-3. 5-Visic (with COT): C
smort depics lons fomure withrrng which anicativ aarens warzs flooce sheaet arvial)
5his3.5-Vissons
Qwwn2.5-VL-b COT):
atwook piibis loable
eign will wils de vizrat and lincl edlas ootaee wil losa fineza
Bi: vilkenal datc adaam- Tii opóza inglaa tha de loko dowco
deciin daves, viads doa nut saak
typed immarr cllako flwess
unatate with parto reiiices
inbual with phllsoppica signtiinnn wiidi ood les catas inkoecction laua onibuiio Bazm cakaro. tn. toclaten of weser Thee oplic mainka tha lites Bovon an in saic of dondaass shandesner, which doo wiech accullr symbhi punto
stign with th typical imagey
Hightihtin
poods diniiign has with commen symbhilin yoxall enpeceraaion hous Dovenn, Irnal Answert Qwen255VVL-Bt A
Qwen2 L- (with COT): C
Vilcor vixitty and grort Luta Soovis oftee aesclaed vistley and growth dou thai asscciation sr the ridy
test auilty
desghn being aibearyd onter This woll with image wiik dlowss vibeau and livel depiatioi
vitured
vathard lons fiveess leaeed,
beitty Kieeninn lohw
Boones, siuck wolld be race adiurtiv o'tpiigg w seacer ooher thas sotcess
Auma atterdaaas ass filed with Eastenn Houpls Late ur deep's rysiioic in Eartes (looplyy oepres dlag regpest
pulls edightccoee,
ccopustiin
pillasephss wiich soald
D: doodatee in wheee The image degien dasklo whtte suow The sea dewer are la fut Yoors which sypial for wiene
Owen2.5-VL-7B:
arcccaee with antelammment ralvvti Tacam phlloupby Is symbol anowe
neebenor Thanckee
45

---

Published as a conference paper at ICLR 2026
Social Seiemon Paychnlogy
Quedion:
language
pazze
Eaglish
Optione:
Fxcty inpelaring
Repretetaaivve bevioko
Secarii exceling
GPT-4o (with COT): B
Thechid dava is with and ceneriy dkanifing a "'Paaicc"
Laan tue 20ld sue cai and siov calk in "Deepte" inciaing fur child asagii the aad Beeppee to aretter arrall
the dars reviantio is.s. are tiny skedlppdi The beustor cummlt autmilation, where
inooppennc drteeneent Aramiliaionn
cogattive developmart Thes
nattailattee [[]]]
GPT-4o:
GPT-41 (with COT);
("aggle??
other adut child who
cating cat Deggait wel.
The wiling the on "Daaghd'" Haganh Bat chll apabiin kazns latel Disalil be mons Allienn axtad Dile
optsen:
ampreing
sage
secugaing 1ha docett it the acauis is. Astrelase:: in cogative dovelemaa (Paget, auncetases to praces el macnn ncn infermttes and troepamin wts outinn ougative sciema.
uppauntly indepedees
actanne maally
model
clild callng i'Dagach beanee pobatiino band has sxzediigg mombile ponttepe This
decaice and out lesaaa: kausin dellice. Senanic eaceding This actens oo procea of eacedin scarrry lapei tus har outticlar scaatgg applid conaai Tis shout murayy caonding mistbhling vocn
apolytu
+ GPT-4:1:
(with
The chidd
"Dougic"""
cxaliciing oncgerenniainn
hadlyy inpoaing mng lepeining as farm of kantig socuriss zuttiska saar the in and appurelly indenrdes the creeqate sduskr. The ssaly azsies anrsh and fir altchrer passi faore. doart
thnugh dig The tis the scmuto well E: Equeeeatti hearess: Thes mecal divice what people jage prbbaniii hard on how lutciin ceriod The aboet cooudryg aboal mudlicingg excreeeralaate Einan child appiping the courty scieem Teennce bell the and Bni clame cuampee mntriiition llarciens, the conrot antrece
ataatin lin the
(Magaee bith
much werahin sacrthi graztope.
thant dectimm-macce jupgmoci sncemnnnty
Aarreiateo. [iIl]]
GPT-41 mini:
[(B)]
nano COTY:
Erelly sepricing
snly borning
Assisilatio (8) inston anngeiig afemsiio ino cuiting sibeas Gazewnlll groesily cefes los
expeinsos are bncrpectted miktng hawwlldg.
Repooweative
people soge pobsilll ever lose resesills
Tolee
Secenii cooding (D) The inndvers conoling seatng inlemain whic color
Lodiis the pieese The edat beddng dhidd miling day "Dggi"" and beoid peting dos viich syoos dill pellge
slotert araommape dbett dogs which dlgn will eoupp of feeiler ('Dogiei's
beaisde Thandrr ean leyond heat
deseikee as use rpeneentivve heanatss where cVld Whiing lose reerd skonce dag "daggic" image dog
[[C
46

---

Published as a conference paper at ICLR 2026
Social Science Piychikggy
Qucetin::
language
Eaglish
Optinas:
Tacts inpetting
Repructttive horiaki
Br Somand: sncodin
InteraVL22-MPO (with COT): A
Freiyy sepresing: Tai eours whes crea tegin to towl inskiaate syjet pesias It dereetizzed by eals ckixkeed mustare maggaknng e livig letggs fiver ece lskag sam
kepe are rapaive pou min, exausuayy rederaeed ligañiioo paydiscogs
Repesesntive lexiiie. Tha meall alattot the orlo en tread singeis cateaaiee flnns, afex looking sysiemetic
cuss judgace.
penticaletyy noxeuting meninng cale engaliie devckpsauul vlace lning hergs oe cnanelll catepaiiol. Mas the saroo Aealls inpesiing HAM
cawaing.
pooccee
whee chid beiing
day Thes reflcia MSady upsaning?
InternVL3 (with COT):
Faeiy trepancio Thes nykals rodte ibuades
organion form in luppoepiia anudrment er eally dryictive perseftic
tmage
Auiriiaaoo The angiitice process inchbes inaggraing new leto intormation imo extrting ctheaa or Samwsrrks the platue dhose inseprnttg informatie secodia is pnooccpption. Sicingge inloeee
Reposseeativ reherr
actal poluuiitt people Buurrtt
centrity infornore peroption the cadig dog puticilidyy fildyyy or conneas in cating when night be ceceoual
Socsetic Toeading untor wooding puaddoo
teme ofth mentigg imoge sheeling kndoraaionis
og mypecing day fut Chopute tnbeaits with toe dig waic reprents tatibe ousers the jet and Bantly. opppoped to the seed seateal out eipans Nix wabitii deooctonn recocd conitad HCI
Seaweti Teondag
InternViS:
The piztrs itserma difirmue ractoas
faxill nurnhar, exuid pair difieennee ln farillarty
[[NI]
Janus-Pro (with COT): B
Janus-Prox B
Llavi-next (with COTY::
x
Llava-nextt A
Llava-mevision (with COT): A
x
Llava-cmwwssion: A
47

---

Published as a conference paper at ICLR 2026
Social Sciener Paychologr
Question:
source language
pisas
Optione:
Faily ingeixinn
Leaittle
Secarii eaodiig
Phi-3.5-Valon (with COT); x
odil bollan
Geegorasl
backyoondd
Stop Anabee nelanmnae teween derers.
ropettvety dogie the forpreed sume lexe ti he dogin the women's TTe of tedkgeasd in relded to blooll cinees
boldngg
Sep Ditomene wiki worape hai exyline val itaiinilip fetweer the dlmmes - Fails tepriving This rifes w citadee where vidut famms as incemset anvadesion hakeon stimaka and repuens- Andrilation: This roles in precss of inoeperiing con infrrmuse iato coaling mantal cocpeciio Reprositannc hoaiite The rokm to sectad abutte ba idlo ce stenstyyes gneralontiess stale judgoess. Sacadii mooding This edtes to Eo procns of enooling Liforratio ano mustingtt fornat
snover hoard andtpi... Dhe
[[All
Phi-3.5 Viston:
2wwn225-VL-BR (with COTy;
crivval pariod
White
doort qucitially bncluas
Austmllatiee This peuradl refee
trugriing infermatio
baow Kudee drautaro Ii dodaar
dladdo adate Da adlnnciaa betwors the dald fackky innge.
C. Reprenctative leailds Tai hosh esing fasilar maaple sadenitaal sonellag weo la for image (E30 allling ts dog "eegiic stich sugport the usng destar ward (dag) cacembe the sea indiatse seprosnaati bustene demanse excocing- the relors to preces comorring sewary reformazen into moreiggh ippeconations. Waie the child Horry procesing the visal end coolivy afferationn about the dug. the spectf crtiee of cilling x l'Deggle alge mere cheely eith the aepruueart haunalc
ls unabes Qwen2,5 5- VL-3bt
Qwen2 5- VL-Tb (with CUT): X
Impeisting the refer poycholgicca plenencne where organios ferna atuchmeni firs moxing ebies fare indin
the indding
fookiill Cornng stadennd buse le list tarvicg slied
Astretatiin: Thas is oogative pruces where ane adferaaiion is lncapertee las vairtin kneededge strutirs The imaaee doer not saoe any ace infirmatio beng autriinc aan ccaning knendndgs arrcares.
Regrosersttee Hearittic This costd stutrcal del trooues making jalgners hare in hew will the currer situtice
exniisss prasope moreyle menway. The lmage covas dhill innoecing with dog and set whicc soeld seee remueessati beartet
Semantic Encssteg: louning degasa pramtepe undomanding
preses cnoading maniing mamory. image
cay spotille somati
enoaiing
witedd.
postatpp
Qwen25.VL-7he
The pimm chons chid bakag neggh call ("Doggia"" by esaratio Hiserna the causayt of anstntaation
xppo now Drigandiiy to femmur styee use oeg! acaits exampli Tharsters
48

---

Published as a conference paper at ICLR 2026
Art Wonemm Art
Comtlen:
"The
hereed French
WLe Jeail is the paiding atiante rehisvedap?
Question:
Qual Joal lhe suggte ancreiati trize
Bi expenioo compuct in laad the Sawess fair
bosele
Bi peadieer estaue okills sumerr
Ac cambe var sn traede::
mén
CPT-4e (with COT):
The canao tracile Camas franee parrase imapery. can specart: peneonll
had White inficmes wuth referaner
intinaea mllczeedyy
Like Mac Bones lax lait (Opiea Cy. Flwee on syntolax leady ss resuzes but day ure mas anaral and de ovt deaul
epciilially
eugou mnlatenahi aaci
Given option zamio hmaall
tepraient penonal comraction somseee imperters [IAI]
+ GFT-4c:
+ GPT-411 (with COT):
bosilee Cuoess sagesiing litinas. B. expessier coupeet it be band Thi inlianee well spent sultsee aetiicallp The dancs a bar Flowen soadedie fer vaiion thins. expersee bousailly lowls, riglt officion sisee ool Cheader conden on atrrayy sorururs
pertals
veally lut dows Cieecly
Thoceee braede
warch commen sombel intenat reberenaepe mant
Crecada idevart doadl
boad shonpet Indicase oteterniig IAII!
GPT-41:
[AII
+ GPT-4.1 mind (with COT): A
Sup Eaarns
rciation
imaeder Camess paraai leved
and us conbelin penecal inurnae connectioi The oppestre compacs in tand whix indicato waith or datta,
duenn recomunty Fuaect sairacy.
the fovere her san Plaooes cur combula vatiaa
forver
Bubiin dounts dieaaly imply intimai idatamkee.
the Unulity stara dead revt chougy raggets arteun atrtaasll The capeo hovater (K) pessaail Poin
Faitains lived portle, eelllloong clsee satente lood. Therdfrr the segpers as intinate mlnseesttyy
[[A
GPT-41 minii
x
[IC]
+
GPT-4.1 cano (with COT): A V
The cames on hor tracde whit canco can be docoediic praas it doart rcceuurty indosic intimayy bo mare kabiia excover.
Tie epeestee wocpprt haad okgax cooport suggett atvnlie penuoal appaxees, bui down't dicelyy inpls on Phrven oua syntocis vaiser thip. incanding kosts, promuce docrch snongly aegent bnieeey.
releceedtp.
Mar dewes
Inscid
whish othan
Blasstion desil
alatoratsp
GPT-41 beasdde
Interni12.5 5- MPO (with COT):
conncy
The Nai deeums hain This conll cauld indica
antal daseht
The aleappany oder of - edor of ba doos sken cannt intimac addiissig..
Now thes option wifhiffy sores intiwwy in intioonipy Hoseenn optlas B the sumpaat and aveiieed stility
mare intrras pessral hume ottan hold agaitlano selationiiss pe whooggh in gencal man. Bhit for anabali Bal
coneet maver magdt ancepptted bith laa nestos sapamns compact
Thas eocorred staver [[8]
+ Interne12 5-Bb-MPO B
bast dent meleeing kieiy mocaatd dece and intente relaeendgo. The compaci perseal exali be
untiaterred and pountlly ot' moat eanus
osnel by jerwone standng (PK)
49

---

Published as a conference paper at ICLR 2026
Art Wonemm Art
Comtlen:
"The
hereed French
WLe Jeail is the paiding atiante rehissedap?
Question:
Joal koma suggte ancreiati trize
Bi expenioe compes in laad the Sawes fair
bosele
Bi peadieer estaue okills cralcer precement mén
Ac cambe var sn traede::
InternVIL9-8b (with COT):
The oxnao aer bmoolt (Cytko Al: In Aacorie paraeture canoo colld indeate wmilfs pessont iitee, but docwrt dieivilly pour inteate siseskpyy
The experatee compac hand Opption IlI Corpacts poss this ccttary holdrng cornetic
sagante pereoal (Oykka indariii rehiseatpp.
paitings
pandy
arstiing
Heaed mulys,,
ccads arreaty auppod instma relanankie. Hlencerr quonze neght ftran
unkcntazing le vanes (Osiian Al selened deaale enght passed sdarnesd that inkede individad tais and My Nin a ietiodial lnteeds pessaal aalll sealth Thas plasiite anseut aconding cottast maditical insercitiion iel LXI
IntarnVL.3-0;; A
Issarica ripect beseeee lic. The la periinal len,
rntimate mamert and waac implec a dioec relitseaaa.ppp lhs other optices do ast decele sagest tatimbi rcadienile (8S]
Janus-Pro: A
Llava-nert (with COTT:
Llavo next:
x
Llavt-onevision (with COT>:
x
oespal
Llavt-onevision:
x
Pis--35-Vislon (with COT):
prrcoa
special
Val-3.5-Viston:
Qwen2.5-VL.s 5- (with COT):
The braccit Clnnec ofle gechalto Hidito
posiiog rantinatio
The coveatte oxnpas her sund - While expeatre aoma ou mabiat wollss they den' secovartty vagacet intmaas
whitienasp wdcnn spallicclll cehioealip..
adgest mater paising.
inbavedl
The sivernano sake bur dess The saiee clading on sumetions anivate moad ponamiity, but it docrt divedy
commanaly anccand
fikilts Fini Araserti
Dowen25-ViL-Se
2wwen2.-VLL-7 (with COT):
The camen braedat Carmas offir sen Gilh pensad cumbeh parmatss of lead cappasting connictie
regly baloar malesooti
rxperdee
dhlegst wetey
indiaanecll sedi-iesod poaclty officent
The than flooes in bare Rewen vir ove tobiiilt acoeer qucinially indiures intioat saatiwallyy They migan deceratio ar syntocs
plewing proside
intinate
Ceoscliiie these and bale, Bo nosr glas deal
(essoaal
+ Qwen2.5-Ni S-ve-2.--VL-Tbe
The Lead paizkin dua suggett inceate day is camao trcales Assoet:
50

---

Published as a conference paper at ICLR 2026
Question:
Thhe source Isaguage for iose:
feee moute seanber "3.
Question:
Yiaamo repremato
Options
rpecas enscisssro spprc sntpot vole.
Lreck
the mak from teagere
the twve Vaningiaa
A: Bearesh vesceeu mys
B: Flyss sppraa rpees Sipessult Raeas Unwrrppe
Stade roste hrm Kiev Norggnee
GPT-4o (with COTY:
Obsuvvatis the zap tade mak matidd number 1597 comadd mujians Kiis, Niogared, and
Uldeeaardda syninee A The Orva sik Rood Thes perorlly cooentd (aku karope and paondd tlrugg Cornol
Ask. it des ast
the nats rocked US1 map dt mass fwon the Vanigioo the Grovier Ihi wos iajor made ware consccinn the (aan Saa (Wraggoon to Biportio Depp cooeea va nowe in hhippe The more bol Secilydis
Selgee
Khaser
prisurly
Tooe from Kie Ningres Wade Riro Ninggrod ler expeeced, and the oots parsed" 32 varead fattlr work Ciestoriogle aulk-rting boowle tudde caste
which
Coeckutee: The node easkal '37 cencoumma lo soar frees Iec Vacungaan la Chocks. Eial pooee (180)
GPT-4cc
[TR
+ GPT-41 (with COT):
murted rember. meeeer C'T appure thees
The reates cakked with 132 exsset the Eellwing les bortonn Fress the Belli Sea regin foom Neggoel Iebigil
dooe frougl Stndook
Tram donn
(hest Clesseemnn)
option are The Ureal Head was mote connecing
stmpp manty troupt Cental Arts like Unagee,
makerd
Hhlt Sex (Voumgannt
throagh Kee and comm io Maack Sea C. note from the Baages Kharar The weeld be mone lecateed the Vaips
and Capas lea repora D. line from to Ningored Th merr lrcalaed mose hawem two ciics Eaiter conastied Hallic
Black Sea though Kins fucitannn vade betwoen Norbhen haaue [[8)]
Thanaties the sarve
+ GPT-411 B
The rade made marked Bree trace with the ramber 3r cencppndde the zven bhe Yomagaan ia Liredaa
warch
exdiana
Ballio with Bronnine Empinr sucdis Daiggn ITRI
+ GPT-41 mind (with COT): B
Dheaazhe '3 uppars
The mosee
Fram Hames (hoogeeed amal dran throegh
Rine - Thees Kire ioeande da Baakk Sea.
The Epriapion A Gred Read Tha ves not inteerl of ofuedo sutee venerve Best hsie in Ease taiely (oough
spoifiully seaks
les.
Veongion
doen Black Sex C. The sede Belgas Klaen This wead rasr boaklet to Welge Caylen Sis hewees two ollos Rail but fhe may slows
region D. Taak yore fwon Kire to Nespona- Thi moee boaliee reak
Baks: Sea (Wenagg
ihen and Kaew taz Bline Sea Bymaramm (Grmkki
Thaniiin the samee aneaat in ([R
GPT-4,1 minii B
COT):
caoked fiom croder
this bestieee -Tne Kirr ruands the wuthose, thrugh kirr twarts antiverr Chevveno (Kmmen) These cane vorn coened Kie with diffecel regien, indlnting sgaileust tede reac orginiing peoking Groadh Kis Aroing dhe splees .A The Crea Sis Raal Tiis maie vade wase arocelin China to the Motismmnnn, not spuiitially sentared avond Kins. B. The note bun de Veargiana to the Groboo This vas taak mas conaecing do Namagies (Väonge) to dhe Bysaniar Sugies, pavits deragg Earim Eauos and the Black Sea. C. The mase foo the Ri gars the Klanes This in tess procioee the centest of Siee's made wores In Tryde note finn Kiew Nongonot: Thee uelkimosn made mase connacting Kiern Nongueed paring though rupson dusen map makile moti matied comenny Kire verson regees, mai spaceai Giver the map's Margeoo.
Leoking the morked with though Fectatar.
Chembhe thees Kiry tountc auth
trrugh
+ GPT-41 nanor X
[[A]]
+ Internet2 MPO (with COT): C
X
OyOias A. The Geea SRA Rond Thh Geo 5Th Roed is Hiseis tadk nosr fust conmoced Class with the Wapers wvlld, ofice traventia cunual Aca and the Makee Taii The tore mokedd is dis maner would tppiaall toas dhessh rapies mach
raaden day Chim, Carmi Aats and of Basox.
Oyylos The core foo the Varaaiies to Gooks This sure us avlle takk or odturl cuse betweee Siralianiee
repions (Veasguur) and
Madarmmus (rorrd. -tat world skary include jerreyy, avtally though the
Bymentis Eirere sealeng eaatal sigins
Option The
Khars geaffullly inseles coneston betwoes Bulgerie the Trpple Sulgare grop
Balgels,
ctser like Kirr Norgoood WWit docttis naamal conraction tiemiie Uarkas
Lidoos durd irvend
the
hewador evetmeenal comerienens ireiad
tra. Cher biagge Andyaic The map
made dhat
wilely acres Europe end reaces hii noot macked ovr lyderiaall esocittdd with Rhears gpat Ruvis Earaei Europet and ussedd soward repica sere to the nalurion caltarn opeer Caniddilig the purtiiss and diractionl thas axedii retic are insvative of caneeieas frann boseted Asortpdons of rade reaice betwees the Belgara and Khaams onfice das intmal Okraiaa tode rodio or pasts notheen aeolin polna. The peltenn and gograpiee spoto dovetlee de aect accenntly alga with bo consetre fox. the Balges Kivdes Tiendion Decornee aasoe (831
51

---

Published as a conference paper at ICLR 2026
Question:
Thhe source Isagaage for iose:
feee moute seanber "3.
Question:
Yiaamo repremato
Options
rpecss enscisssro sppre sntpot vole.
Lreck
the mak from teagere
the twve Vanongiaa
A: Bearesh vesceeu mys
B: Flyss spprra rpees Sipessult Raeas Unwrrppe
up
Stade roste hrm Kiev Norggnee
XC
GPT-4o (with COTY:
Obsuvvatis the zap tade mak matidd
number 1597 comadds sujians Kiie, Niogared, and
Uldeeaardda syninee A The Orva sik Rood The perorlly cooentd (aku karope and paond tlruuh Cornol
Ask. it des ast
the nats rocked US1 the map dt mes fwon the Vanigioo the Grovier Ihis wos imjor made ware consccinn the (aan Saa (Wraggion to the Biportio Depp (Coeea vi nowe as Dhippe The more bol Secilydis
Selgoe
Kharar
prisurly
Tooe from Kie Ning Wade Riro Ninggrod ler expeeced, and the oots parsed" 32 varead fattlr work Ciestoriogle aulk-rting boowle tudde caste
which
Coeckutee: The modi easkal '37 cencoumma lo soar frees Iec Vacungaan la Chocks. Eial sooee (180)
GPT-4cc
[TR
+ GPT-41 (with COT):
murted rember. Teeeerr C'T appure thees
The reates cakked with 132 exsset the Eellwing les bortonn Fress the Belli Sea regin foom Neggoel Iebagi
dooe frougl Stndeok
Tran donn
(hest Clesseemnn)
option are The Ureat Head The was mote connecing
stmpp manty troupt Cental Arts like Unagee,
makerd
Hhlt Sex (Voumgannt
throagh Kee and comm io Maack Sea C. nose from the Baages Kharar The weeld be mone lecateed the Vaips
and Capas lea repora D. line from to Ningorod Th merr lrcalaed mose hawem two ciics Eaiter conastted Hallic
Black Sea though Kins fucitannn vade betwoen Norbhen haaue [[8)]
Thanatie the sarve
+ GPT-411 B
The rade made marked Bree trnce with the ramber 3r cencppndde the zven bhc Yomagaan ia Liredaa
Ballio with Bronnine Empinr sucdis Daiign ITRI
+ GPT-41 mind (with COT): B
Dheaazhe '3 uppars
The mosee
Fram Hame (hoogeeed amal drnn throegh
Rine - Thees Kire soeande da Baakk Sea.
The Epriapion A Gred Read Tha ves not inteerl of ('tedo suter venerve Best hsie in Ease taiely (oough
spoifiully seaks
les.
Veongion
doen Black Sex C. The snge foes Belgas Klaen This wead rasr boaklet to Welge Caylen Sis heoees two ollos Rail but fhe may slews
region D. Trak yore fwon Kire to Nespond- Thi roee boakiee reak
Baks: Sea (Wanag
ihen and Kaew taz Bline Sea Bymaramm (Grmkki
Thaniiis the samee aneaat sn ([R
GPT-4,1 minii B
COTO:
caoked fiom with croder
this bestieee -Tne Kirr ruands the wuthose, thrugh kirr twarts antiverr Chevveno (Kmmen) These cane vorn coened Kie with diffecet regien, indlnting sgaileast tede rea orgininng pooking Groadh Kis Aroing dhe splees .A The Crea STs Raal Tiis maie yade wase arocelin China to the Motismmnnn, not spuiitially sentared avond Kins. B. The note bun de Veargiaaa to the Groroo This vas taak mas conaecing do Namagins (Väonge) to the Bysaniar Sugies, pavins derage Earim Eaoos and the Black Sea. C. The mase Soo the Ri gars the Klanes This in tess procioee the centest of Siee's made wores In Tryde note finn Kiew Nongonot: Thee uelkimosn made mare connacting Kiern Nongueed paring though repson dusen map makile motia matied comenny Kire varson regees, mai space
Leoking the morked with though Fectatar.
Chembhe thees Kiry tountc auts
trrugh
Giver the map's Margeoo.
+ GPT-41 nanor X
[[AI]
+ Internet2 MPO (with COT): C
X
Oytias A. The Geea STA Rond Thh Geo 5Th Roed is Hiseis tadk nosr fust connoced Class with the Wapers wvlld, ofice traventia cunual Aca and the Makee Taii The tores mokedd is dis maner would tppiaall toas dhessh rapies much raaden day Chim, Carmi fats and of Basox.
This sure us alles takk or odturl cuse betweee Siralianiee repions (Veasguur) and lart Madarmmus (rorrd. -tat wew world skary include jerreyy, avtally though the Bymentis Eimere sealeng eautal sigins
Oyylos The core foo the Veraaires to Gooks
Option The
Khars geaffullly inseles coneston betwoes Bulgerie the Trpple Sulgare grop
Balgels,
Ntrgood
Livoos durd lrvend
conraction tieniee Uarkas
ctter like Kirr Norgoood WWit docttis narmal
the
hewador evetmeenal comerienens ireiad
tra. Cher biagge Andyaic The map
made dhat
wilely acres Europe end reaces hii noot ere macked over lyderiaall esocittdd with Rhears gpat Ruvis Earaei Europet and ussedd soward repica sere to the nalurion caltarn opeer Caniddiligg the purtiiss and diractioal thas axedii retic are insvative of caneeieas frann boseted Asortpdane of rade reaice betwees the Belgara and Khaams onfice das intmal Okraiaa tode rodio or pasts notheen aeoiin pelna. The pettenn and gograpiee spoto dovetlee de aeco accenntly alga with bo coceetre fox. the Balges Kivdes Tiendion Decornee aapoer (831
52

---

Published as a conference paper at ICLR 2026
Qseetien:
lungaage for
sose
Question:
Yizamo repromac
Optinns
rp spece enscessoro sigre ontipt vole.
Lrecks
Gh from fragare
theneur tree Vennngana
A: Buavasi viscceut myy
Elyrs spurr spees
Stade rote brm Kiee Norggrod
X
Sigrzeull Rasas lbwarept
Intermat2.5 1-0rrwl2---BBMMP.
La mure dve Venegiaes the Grecka [[8]
InternVL3--b (with COT): D
X
Map Layor Landic
slastiia deteust
cach marked with ramber. Number '37
verson
cyeiow Auchiii A: The Over ST& Road: This rais through fao throeh Cresal Aca connectng Cliss to Mutiemmnn viick doo not prorieenty Et wits "2" -Ii The from the Veaapaane to Crabbe This frttriaa mas fves Easton Eange arvo de Klamtia Khagante so Bycazine Exppee raties for alvettacea monchan jeussys. - lhe code fox Oe Sulges to Klaseo This sigifee tavel jath inland to the Belgrr Klas wolld are liss procioens with
Kiso Kangaced This mone aocoes the Kioas
lmad, arassaapp marked
mmde from Myantorr Khamm: This ma dexetty
ippors
Tarrpo
Klawnss lustrsills
Vinngtin marcuuat
Draike
Cencoo)
fram Nimeguzz Gieetal Iaa. (08))
InternVL.9-bb:
Janus-Pro (w0th COTY: A
Janus-Pro:
Llava-nest (with COT):
Llavs-anevision (with CoTy:
Llava-onevistom:
Pht 2aa-3.5-Vistoo (with CUT): x
thows marked wits namber 7337 punss through sevend lecalion inclading Claptan Sea, th Btack Sea This ras lanven as the Cres Sikk Road, which vas nutuart trade musis conserte the 1ast and Wart Thai comoct unsnnticA
Vad-3.5-Vislon;
Qwen2.5-VL.3B (with COT):
wes maje
connetta
Mediermees
prizueily
The note free the Vanigion glans Greeke This veless to make taken by the Veragian Clasd, who Vikings who tradel wil the tiveniiee Empioo reao woald lave hes sigefienn hus specitially marked theer lines raag Bedgas
juas,
Trndo reslo fees Kies Morgored This refers tede raim th Raif people who traked with Slavee
Norgond, Las reak wis sigatimet enall be awaand axdiple irm depending an the ipeaffe auterrall gested and trde coue Strgeoo,, aey, Lacyy besed
opties
Thnelan
Teddeante Biin Bongerii
Qwen2.5-V..c 3-e
Qwen2.5-VV.. 5-W (with COT):
x
The Creat SEA Read This usa muler vade mete that connected Chiica vith dee Mediarranaee rutice. sot marked tii Aela Soal pass dereah ragians ahulad with nurther
me primantly by Eroogh
Scnsiravia The make from de Bajurr in Klanse The ras maried on the map and comsect i the muliomaa as lands the Khans tentiamas Boosoees in maked namber
Ryarzino tirpine maked
namber (551
akko marked
namber 7T2
Chien the audlydi, the PAcE mets rurbad with the rember 270 mar ona cverasting Kier Novegoned ITherttve,
comorr SuAeEii [[PI]
Qwen2.5-1y 5--L-Thc 1he taate
X
tEau map with namber iotn Coeat Mile
[AAI]
53

---

Published as a conference paper at ICLR 2026
Questien:
"The language issue: Spanish
The cane
contitatee by mitai exturry
Chilc' reverae hetwors
Qucetion:
History- World Modern History
zaktao wxrice resss conithadda
that mook place darig mersunec potso it comeet vate that committtic or vitmar
CHil care 1800 1924 ba ya grocess trriiias sorrtse curnee periedo receernddooooo as conaecy afrrur
le vaals tetves
a drunts
decatio
Salowing the We of Phetti 30
racienties canecurtel
diletion sumento in lis dos dlaa poeer oree la pattidas docies Iteyiviia Polhowarra Iill
petiads deciine and resvvary dacza Paclanantary
Clerr
ID kenaacy fer
daria the dockkeeent of is Pras
mndeado samares conttrre dacoes
Opcions:
decamoll Frirows Gewrr Mandial
Sola
C: Ords a
E: Sate II
Di and
GPT-lo (with COT): C
Saaencei deati in mose the two dwals Silowing te War the Pueifse The Wes Facffs mske is 1884, ex]
poph daves loeaee
Betion
1488
esgars
stateent iis cisee
Stalenscs It: (Thrtids cecias and neweyy daring the aoceled Facianerary Repabli c- The Patianaetary Repchix as Cadz the
Daneg persod gresh shane tbetasion sines cottiltionn Alke pakking 1895, thar ii
Paslancrare Hap
Stalenent is perces
Sevemmen III U'A sere the onetart homess daring the decilopeire of This World Waytt The Tea lovld Wor batd moes to The papt ahrap incrase in arooss commotaiionl from 1908 1415 afte them ic clorp declioe Evakninn..
thrughoor period AaaaeiiAaaennn
slik
ICTII
GPT-4.1 (with COT):
drsnc Inmunc donder Skizwing Wer P'bctcc Thhe Wer tuntte unced in The gragn
cacalas
So,
corret
tt Periolk of de monwery dailg, the ce-uate Pallareraa RquAtki? The Puttanenury Reyaliic Cli lattdd
te fron to Froes the gord besser 1855 and 1924, fo ahise or citlccia destuanet El Bidedlass foom 50% (19))) patuer dhews poriok durao
(999/19993 Dive
is
lill umikery cvstaa tummar daring devel apancal Fist World Wart" The Fer Woll Wer lnttd fress 1914
1915 The proph shows mormis from abeett 3015 1891 in 50% in 2813 warch danig the ver. Horeeses eller 1S1 thec merrae aut wentn inceaur threughou eater deedapaecl
shupp dedane to
doriy was thedd
het faly acaiitan.. Given propl mecous and (315, thm deciin. zaoomeet ill nct fally coneet.
Condedar: lillenent ie somon iine fally comaac Thardie decomnn arooens Sylee
+ GPT-411
A
toz demalo Sellwing War Puetie (105-1sed- The priph iaen shkpp
sheall
Shi in 1890 to reals 59% 1885 Th's wonepeeds to the two doades aft the War the Poeifa, curing whick Chile garnel conns of wirttttti kintiviia So, datumee la coreet
Il Pestod of dodaw and recerary dutna the aocalled Packancrary Repacii (185 The gopp shows Duetattion hewwees 1855 and 1905, with diciies areaal 1999 1965 incseias assnal 1810 1915 and deniae ogats altil 1913. matites the Pathumarray Rey fillc paned, chancterned polmol leaiitly and conconse flxmuutuss, So, exnemeet Il soreec
IiIi iendmny coname merase during the develeennnt
World War 11824-29985 The groph abone in anmrrae dedine othe Si, inavase conter broogheat antorllyyy
51%
period
GPT-4.1 minl (with COT).
dusti lcomela too deca des Stlveing Wir Pacifui The Wh of Pacific reded in The gray's afeeth
10 Periods declie reovery daring the se-oiled Paclanentoy Reputtis" The Pellamentrr Reyukii Chile losted reagaly fress 1891 1925 The pipe slaws Dochatices during thi peied: cndine from 1885 in 1908 fives 50% A950) inveseyy frees
50%6
dedine igai 1915)- wanche deserption of pereea
(ill 15A tankeey zowann incuaie durtey developmurs that World Wain World War lattd thee 1914 1918. The jraph ahome in incrass foor 191105t rediakl which aligs with Ex vust and mádds of WWL - Hoscee, afler 1915, So. incrasc noi aawans Einoghhet the antre pariod; puato than dastine.. comeet
Ouz traokum desline
cernect
comet
minis
dradii insoeri dcedles Sollwing
(1835-1833) prph kilkpp locoss cariig
semopend
Paliaseemn Repaziic (191 1925)
Dustatiina
1421,
1966 1955
R85 Soloood dediimmdte 1935 This nuncle
Parturrarr Rapait: panad, masemee comees
lili sendamy
mermue during the dovckgeeen Hise World Wer 11984-9989-- - The groph showe cemoott frem ofe 1918) inaoeee costoot throoghout the WW) perieet ontinity camnes.
loot 91% in (910 io 4886 in 1915 thin
palk and the rale Thars excment III Diszakos, BN CRITE wwsinsc are Lund (191)
54

---

Published as a conference paper at ICLR 2026
Qsaation:
"The ssurce language this Insue: Spants
The doow graph Koom the cun Rutoo rebure Lodwary
Question:
Hatrry Modern Hattary
hoavical proceser that wook glace caring the meriondd potldd is cato
grkino manies consihatinn
Chil cate 1500 1924 buac
procose leaietics sourikk durnti peracce muniimado, o conacae
hotk cenves
Tt costi incrns in decdde
was chooooidd
Salowiigg the of Pooils lacin
radendle aaradaatiol D diledoo namento on lus do skcule peaserree Paciiia petieda dusiie itepanis that annatanne lill
tte cilled Polsoratryy Royali IDl welleay Se wester koowee dutig the devekpuane of Fast
teakenn sumento consintic damaic dovamall
Opeines:
Guers Mandiaa.
S
B: Onte
X
B: Sule
C: Ouds lulll
C: Boto
Food
S
CPT-41 nano (with CDT): C
A incrss decales fallowing Padffc.
Pacifi 1883. The gooph lncreiing seall
rigitficn
aloat
1915 Dactatioo periad decio
daiing
Patlanacttry Repubtic Candarier a
lll vedeaay
burnas duting develguace the World Was World War was fooz, 1916 1913. Hie
gryp slvres incnas from woued 1900 polling on anoee cat Dann delliw Iffe 1915 Thensdeer daning to the summitatio iacrased inaally did not comtaas a ooerant memmor it parted and the decteeeeteee dectiod
Comcharce: to not comad bomase the traal was not ormaat ieevuee durin W.I. Final eucurnent iill surces.
cennct IIl anatict The option Indldde Lnd muchoc one Sola
mteem (00
+ GPT-4.1 1.1 nano: MONE X
The grach dlnv that de wettissiin of sulpetic indarry to Ckiic' iucess manmard sigtiiiounte da owo docades (ollovisa Pazdic cades
mukhig anoas and peneds deciie and recowery with
incvusing centinairs uldhogh concet
annd over periol. Thaxiing
+ Internw12.5 5-20 MPO (with COT): C <
B draile remouse be desiees fallowing the Wer of the Pacifee - The Shone ntbie iscraai the pereotaage connthetien of the sirats baberry in netveal mooesss purte pulking avand the 1oine Tharthos ris annmmne iecennt
io Farred of dedine and secewayy duraa the oscalal Fallerenary Renallic- The proph (uurra fuctattees and seracdactiaa cvetiiiaisn Sekooed rooounio apecifiall muticaale tn valy 1900 and watiopuat paroodl Thia, dis stalcnica alio ceuret.
mi A wedncy contur incrus datag devalgerez af tiee World The greh daces candiner spouard War (191119913) fact thee Jeckens Hastaatorn dunnng pence. therrtoo, this ststement
durin
Interro02.5-80MMPP Bb-MP:
Suiu 1, [IC)
+ InternVL3-6b (with COT): C
Escific book plaze nxtilaatiss of le stone indaary seticaal www Ietooan 1880 and 1995 dweely she Du aocclatie of of' 7wrts. This, this stau mesnnset iecenct mi) ponode of docine and mowery datigg be recaled Pattameeary Repehticc- - The pertod known as the (Putaaettary Repabkel Clil: lasdd inre. 11.851 11188 The grih cencnatrate tai there wew Bactaatonn with pensocs of dechane and nowwrr ove this deaterr Specifiall from 111455 crsadh penorttggs Stark op to park amanal (199)) and flor dectioal aftrradd Thicaligne be mentindd ponao, milore thi entorort
docdda islowizg Warnd the Pacife (1B80
11835 to L1888 Acending
yaoks Dore indedd inkkaile
It's tendency comaat tncome dutin devilopenct of Wa.
miroonee Accrding to the praiis inaomn lucked,
wey peai is procstage cantiiiiis Juing
pelking
gradual
InternViLS-eht
The pooph dhrnc deats incruss the betioo of vitute ixdurry catinal sooeaae fillvorng oftte Pacitt,, which in The pencenage aharaby betawc
poraad tncine recavery propt reeplly eonopeed
palted Repultic Denadenonn,
conmensly
your
peneoligge disilii, wilh
rhbraadi Tawr
Basdd the ovment rakmund Solo
Tiax Ieal mawee
55

---

Published as a conference paper at ICLR 2026
Quaatien:
"The sourca langeagg for this lasur: Spanish
The abow graph cooo the
Chibls nowerar hetooes 1882 1999. Barad it and hisurica
Hatury World Modera Elasory
untrice connituice
proceses the wook place durlag the
la takanns catert angmonidee
Clile ente 1000 1924 En bese ol
that consttution of strate
procras lealinices sourdoo
nutioal rovera was sac-haase chaisctrindd by a drunto Hecmase the decario
unkuat lay ingaries casonales se caraiiiaa par as dilodvo
persode decilis and
durie
vaxeaty dos poste sere
the oveillad hurteeraary Regebii.
1il sand may for incmas
couggteeiin durast liannda
dovelogrent World Hox
Repalliv Palastntans e. HDII usa
toaksnia insaustt conslate
Opciven::
Al Duts
At Sole
BI Sala
3 EI Site B2 III
Lood
R: Onty Mlad
Janus-Pro COT);
womtbutio shrne The bost proside woner obeat to viterkal priceses faing period
paph
fwom
Shep Monalty yoeeatt infrrmtioo The grch soows iggaiiint laccase the consthatice the eiletto inskrry lacing the ieflsty engatimmod portoc
Scllwing
oilid Peforeetory
try Doserato the cxnect aaoee had adensiiis The graph and trat tadbote taa the engiitution of situte backetry
or tieal rrvecaes was deeriiedien by dentt locoss is the tic decoces Sclining fae War ofe Paildi 10, gedin of feclaun eevneyy daving Palloonnary Repaoli (1), selleny censtox loceese during dewileprent
Pay Cloese oxtedt crover Booo fhe enalyss the woreet onswer ( Skle. Finil Atseew $000
Janus-Pro
Llava-nestt (with OOT): A
x
Lleve--nstt: D
X
Llovs-nneviioon (with COT):
D
Llava anevisio::
Phi- Vaisi.5-Vision (with COT):
x
Phi- Vhisin:-Vision:
Qwen2 5-VI-3b (with CDT):
éncale inllowing 1265. elgas
greph signfiier insellesdy fallowing
Poclie (1579 1885) vinc to Clales inkyeakeu and coliilazee of sirae indaiiy.
in Paisob oz deliao and reovers daing le alled Paliesatery Bepable The Palfaundacy Repabli is Calee rofas to persol
1850 Duing
ferr var indel peisa
combiaiia silnte tadairy
ureaal 1855.
EDD Amzieny fn ocncas iannor daing the develgeann in Fas Wall Was exprosstaa fee gracs
The Eas Wold Was (1914 1959) dosdly
prads saganto gannal incver is cnsnchulice sizae indancy condaxe
devekggnnnt
Wolld Wes
besed en thes websis Sikennts and he deas conad. stteeer Il kas disels suppetiad by prattal
Qwen2
Qwen2 5-VL-7B (with COT):
meshed
strrevich timbery Thryack Ararfiguts Buints Pers Thes liidyy immodiins
negatii ieguet on the enree indurry -ooaaee the gogt doose a dtap incraan the conmitttio the ceroas incuerry carring amuad 1885, whith raggene the tuterr rewerdd frteed
sure per
periode losalllty delins The
indutry, indkaaiin pariads decilis and sueevay Foriangl the aoard fillondd deciis than andtal yark awaud 1915 batiraa dicctial Agais
dacurtons
buston
Devellpponn
expherios.
1915 which with incearad der dering wur Baaadd wudyede-
Saaorert is cammer beaase the srihh dloss dradte Incvars is the ooniiiation of the rima indwary in the two decades Reaeeen ouruet bacans the praph sbowe notlad duckiee recocrry daiing the
sclviing War
trad doas
mww.aor pariod
Thanhin the saree ana C2 Es ([C))
+ Qwen2.5-ip 5-L C
Diitt Iteearc Pollenty th War the Zasthe (1879-1800 The Wir of the Phoitt ancidd 1800 The gresh iboox Egzitii In centinatinn ttimtt natural reveran varting smand which sligns with peetoear persod.
contnbetue strass wash pmodte docine and Repehiz
1534 1as grash Bacaataza
anvonal
1806 Sallondd by dedies this anothe nor mound 1915 beteer starp dne in cats Hile
World bagan 1916, The graph cocada intiwonng War Facffi.- Yahraat il corret because de grge daoss patiede of decins and recovery durnng the Pachanerayy Aepasiic - Siatorann 20 teparest bucaas grazs deas doow krrdency commant encrin danng
Teedmyy Conmant trevaue Donty the Devdemmer the World War
troughed
years franic
thi Staament
Wai
56


Reviews:
[{"review_id": "XcBojUK6zQ", "reviewer": "Reviewer_vNZD", "summary": "This paper introduces HSSBench, a large-scale benchmark designed to evaluate multimodal large language models (MLLMs) in humanities and social sciences (HSS) domains. The benchmark comprises 13,152 multiple-choice visual question-answering samples spanning six major categories (Economics, Art, Culture, Social Science, History, and Geography) with 45 professional subtypes, and supports six UN official languages. The authors propose a VQA Generation Pipeline (VGP) that combines domain expert curation with automated agent-based generation and validation. Experimental evaluation of over 20 state-of-the-art MLLMs demonstrates that HSSBench poses significant challenges, with most models achieving accuracy below 60%, substantially lower than their performance on existing benchmarks.", "questions": "**Q1:** Given that MMMU and CMMMU include dedicated HSS evaluation categories, could you clarify the specific incremental value that HSSBench provides? I would find it helpful to see a detailed comparison showing how the 45 professional subtypes enable evaluation insights that cannot be obtained from existing benchmarks' broader HSS categories.\n\n**Q2:** Could you provide more systematic evidence of data quality consistency? \n\n**Q3:** Regarding the VGP pipeline, could you elaborate on what aspects are specifically tailored to HSS domains compared to similar pipelines used in other domains? This would help clarify the domain-specific insights that emerged from your data construction experience.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**[+] Comprehensive and Challenging Benchmark:** The paper presents a valuable evaluation resource with strong empirical validation. The experimental results (particularly in Appendix C.5) convincingly demonstrate that HSSBench captures unique challenges in HSS domains. For instance, InternVL3-8B achieves 77.21% on MME and 68.10% on MMMU but only 42.14% on HSSBench-Art, showing that the benchmark effectively tests capabilities that existing benchmarks may not fully capture.\n\n**[+] Unprecedented Depth and Multilingual Coverage:** I appreciate the fine-grained categorization into 45 professional subtypes within HSS domains, which provides valuable diagnostic capability for understanding model strengths and weaknesses at a granular level. The support for six UN official languages is particularly commendable, as it enables cross-cultural and cross-linguistic evaluation that is often missing in existing benchmarks.\n\n**[+] Transparent Data Construction Process:** The paper provides good transparency about the data generation pipeline, including the proportion of expert-curated versus agent-generated samples and quality validation procedures. The multi-stage verification process involving both domain experts and automated agents demonstrates careful attention to data quality.\n\n**[+] Thorough Experimental Analysis:** I find the experimental design to be comprehensive, including evaluations across different prompting strategies (CoT vs. Direct), question formats (multiple-choice vs. open-ended), and ablation studies examining the impact of visual information, distractors, and RAG augmentation.", "weaknesses": "**[-] Positioning and Novelty Claims Could Be More Precise:** I observed that the paper states existing benchmarks have been \"overlooking\" HSS domains and positions HSSBench as \"addressing this gap.\" However, I note that established benchmarks like MMMU and CMMMU already include \"Humanities & Social Science\" and \"Art & Design\" as core evaluation categories. I believe the paper's true contribution lies in providing deeper granularity (45 subtypes vs. broader categories) and expanded multilingual support, rather than being the first to cover HSS. I would encourage the authors to reframe their contribution as deepening and extending HSS evaluation, which would more accurately represent the work's value while properly acknowledging prior efforts in this space.\n\n**[-] Methodological Contribution Needs Clearer Differentiation:** The VGP pipeline is presented as a key contribution, but I find that similar expert-AI collaborative pipelines have been employed in concurrent work (such as MicroVQA for biological domains). The \"expert curation → LLM-assisted generation → agent refinement → expert validation\" workflow appears to be an emerging best practice in the field rather than a novel methodological innovation. I suggest repositioning VGP as a successful adaptation of this emerging paradigm to HSS domains rather than as an independent methodological contribution.\n\n**[-] Data Quality Validation Could Be More Comprehensive:** While the paper describes quality assurance procedures, I would have appreciated more quantitative validation metrics. Specifically: (1) inter-annotator agreement measures (e.g., Fleiss' Kappa) for expert-curated samples, (2) more extensive comparison between expert-generated and agent-generated data quality beyond the single model shown in Table 6, and (3) consistency metrics for the multilingual versions to ensure equivalence across translations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QRC3A5I9VH", "reviewer": "Reviewer_6Fgp", "summary": "This paper introduces HSSBench, a new 13,000-sample multilingual VQA benchmark for evaluating MLLMs in the Humanities and Social Sciences (HSS). The authors argue current benchmarks focus on STEM-based \"vertical reasoning,\" while their new dataset, created via a novel expert-and-agent pipeline, targets more nuanced \"horizontal, interdisciplinary thinking\". Experiments show that even top MLLMs struggle with these tasks.", "questions": "1.  Could you elaborate on the tension between the single-correct-answer MCQ format and your definition of \"horizontal reasoning\" as supporting \"multiple valid interpretations\"?\n2.  Regarding the expert demographics, what specific validation steps were taken to ensure the data was free from a specific cultural perspective, especially in the Social Science and History categories?\n3.  Was the full 13,152-item benchmark evaluated on all six UN languages? If not, Table 10 (using a 900-item sample) seems to be the only multilingual evaluation for 4 of the 6 languages. Could you clarify?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* **Addresses a Clear Gap:** The paper convincingly argues for the need for a benchmark beyond STEM, focusing on the challenging and underserved HSS domain.\n* **High-Quality Data Pipeline:** The 3-stage VGP, combining domain experts and agents, is a robust methodology. The validation step to ensure true multimodality (checking text-image dependencies) is a key strength.\n* **Insightful Analysis:** The findings are valuable, particularly that CoT can *increase* hallucinations on HSS tasks  and that standard RAG provides no consistent benefit, highlighting the unique challenges of this domain.", "weaknesses": "* **Format Mismatch:** The paper's stated goal is to test \"horizontal reasoning\" (implying divergent thought and multiple interpretations), but the benchmark uses an MCQ format, which enforces a single correct \"vertical\" answer.\n* **Potential Cultural Bias:** The authors disclose that \"most of the data experts... are Chinese\". This poses a significant risk of cultural bias in a global benchmark, a limitation the authors concede may have skewed the results (e.g., Qwen outperforming GPT-4o in some categories).\n* **Limited Multilingual Evaluation:** While the dataset supports 6 languages, the main evaluation is English-only. Full Chinese evaluation is in the appendix, but the other 4 languages are only tested on a small 900-item sample, which doesn't fully substantiate the multilingual claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VwN3lqZNuo", "reviewer": "Reviewer_geww", "summary": "This paper introduces HSSBench, a new large-scale, multilingual benchmark designed to evaluate the capabilities of Multimodal Large Language Models (MLLMs) in the Humanities and Social Sciences (HSS). The authors argue that existing benchmarks predominantly focus on STEM or general knowledge, overlooking the unique \"horizontal\" reasoning—interdisciplinary, context-dependent, and abstract—required for HSS tasks. The benchmark comprises over 13,000 visual question-answering samples across six categories and 45 subtypes, available in the six official UN languages. A key contribution is a novel data generation pipeline that combines domain expert knowledge with a multi-agent automated system. The authors benchmark over 20 MLLMs, demonstrating that HSSBench poses a significant challenge even for state-of-the-art models, whose performance often falls below 60%.", "questions": "1. Could you elaborate further on the validation process used by bilingual experts for the translated questions (lines 268-269)? What was the rate of rejection or significant modification, and were there particular language pairs or HSS domains that proved more challenging to translate accurately?\n2. The expert demographics (Table 2) are heavily skewed towards Chinese speakers. Beyond the performance advantage for certain models, how did you ensure that the core concepts and visual materials selected are globally representative and not biased towards a specific cultural perspective? The process described in lines 818-822 is noted, but more detail would be helpful.\n3. The multi-agent construction pipeline (Section 2.2) is a key contribution. Could you provide more detail on the specific guidelines and examples provided to the question generator agent (lines 216-220)? The quality of this automated step seems critical to the overall quality of the benchmark.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The work compellingly argues for and addresses the lack of dedicated, in-depth benchmarks for HSS domains, which require different reasoning skills than typical STEM tasks (Section 1).\n- The paper proposes a sophisticated VQA Generation Pipeline (VGP) that leverages both domain experts and a multi-agent framework (Figure 3, Section 2). The multi-stage validation process (Section 2.3) ensures data quality and that questions are truly multimodal.\n- The paper provides useful qualitative analyses, such as the performance drop on open-ended questions (lines 388-390) and the inconsistent benefit of CoT prompting (lines 370-373), which reveal key limitations of current models.", "weaknesses": "-  The group of human experts appears to have a strong majority of Chinese speakers (Table 2, Appendix A.2). While the authors acknowledge this may benefit certain models like Qwen (lines 362-365) and describe mitigation efforts (lines 818-827), this demographic skew could introduce subtle cultural biases into the dataset's content and framing, despite best efforts.\n- The multilingual aspect of the benchmark is created by translating an original set of questions using LLMs, followed by expert validation (lines 268-269). This is a practical approach, but it may not capture the full cultural and linguistic nuance that would be present in questions originally authored in each target language.\n-  The evaluation of Retrieval-Augmented Generation (RAG) in Appendix C.7 shows that general-purpose retrieval is ineffective, which is an interesting but not entirely surprising finding. This section could be strengthened by discussing what a more effective, domain-specific RAG approach for HSS might entail.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Dmp6r4zhoF", "reviewer": "Reviewer_Tyr7", "summary": "This paper presents HSSBench, a dedicated benchmark designed to assess the capabilities of MLLMs on HSS tasks in multiple languages, including the six official languages of the United Nations. The paper also introduces a novel data generation pipeline tailored for HSS scenarios, in which multiple domain experts and automated agents collaborate to generate and iteratively refine each sample. HSSBench contains over 13,000 meticulously designed samples, covering six key categories.", "questions": "Please refer to the weaknesses", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The paper convincingly identifies and addresses the significant bias in current MLLM evaluations towards STEM and \"vertical reasoning,\" providing a much-needed tool to assess the \"horizontal reasoning\" required for HSS.\n2. With 13,152 samples across 6 major categories and 45 specialized subtypes, the dataset is substantial and covers the HSS domain broadly and systematically.\n3. The inclusion of six UN official languages significantly enhances the benchmark's utility for studying MLLM performance across diverse linguistic and potentially cultural contexts.", "weaknesses": "1. The automated portion of the VGP heavily relies on powerful, closed-source MLLMs (GPT-4o/4.1) for tasks like summarization, extraction, and question generation. While expert validation is used, this reliance on the very class of models being tested raises concerns about potential model-specific bias or \"leakage\" in the generated data subset.\n2. I encourage the authors to evaluate more closed-source models such as seedvl-1.6 and gemini", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)