GeoBench: Rethinking Multimodal Geometric Problem-Solving via Hierarchical Evaluation
Authors: Yuan Feng
Yue Yang
Xiaohan He
Jiatong Zhao
Jianlong Chen
Daocheng Fu
Qi Liu
Renqiu Xia
Bo Zhang
Junchi Yan
Venue: ICLR 2026 Poster (2026)
Keywords: geometric problem solving
benchmark
Primary area: datasets and benchmarks

Abstract: Geometric problem solving constitutes a critical branch of mathematical reasoning, requiring precise analysis of shapes and spatial relationships. Current evaluations of geometric reasoning in vision-language models (VLMs) face limitations, including the risk of test data contamination from textbook-based benchmarks, overemphasis on final answers over reasoning processes, and insufficient diagnostic granularity. To address these issues, we present GeoBench, a hierarchical benchmark featuring four reasoning levels in geometric problem-solving: Visual Perception, Goal-Oriented Planning, Rigorous Theorem Application, and Self-Reflective Backtracking. Through six formally verified tasks generated via TrustGeoGen, we systematically assess capabilities ranging from attribute extraction to logical error correction. Experiments reveal that while reasoning models like OpenAI-o3 outperform general MLLMs, performance declines significantly with increasing task complexity. Key findings demonstrate that sub-goal decomposition and irrelevant premise filtering critically influence final problem-solving accuracy, whereas Chain-of-Thought prompting unexpectedly degrades performance in some tasks. These findings establish GeoBench as a comprehensive benchmark while offering actionable guidelines for developing geometric problem-solving systems. Our benchmark and
code are released at https://github.com/FrontierX-Lab/GeoBench.

Published as a conference paper at ICLR 2026
GEOBENCH: RETHINKING MULTIMODAL GEOMETRIC PROBLEM-SOLVIN VIA HIERARCHICAL EVALUATION
Yuan Feng". 1+, Yue Yang Xiaohan He².3.,, .Jiatong Zhao', JJanlong Chen??+,
Daocheng 2 Qi Liu', Renqiu Xia _a Bo Zhang", Junchi Yan
1 Shanghai Jiao Tong University SShanghai Artificial Intelligence Laboratory
3 Fudan University 4 "Th Chinese University of Hong Kong, Shenzhen
* Equal Contribution, sa Corresponding Authors
ABSTRACT
Geometric problem solving constitutes critical branch of mathematical reasoning. requiring precise analysis of shapes and spatial relationships. Current evaluations geometric reasoning in vision-language models (VLMs) face limitations, including the risk of test data contamination from textbook- benchmarks, overemphasis final answers over reasoning processes, and insufficient diagnostic granularity. To address these issues, we present GeoBench, a hierarchical benchmark featuring four reasoning levels in geometric problem-solving: Visual Perception, Goal- Oriented Planning, Rigorous Theorem Application, and Self-Reflective Backtrack- ing. Through six formally verified tasks generated via TrustGeoGen, we systemati- cally assess capabilities ranging from attribute extraction to logical error correction. Experiments reveal that while reasoning models like OpenAI-o3 outperform gen- eral MLLMs, performance declines significantly with increasing task complexity. Key findings demonstrate that sub-goal decomposition and irrelevant premise filter- ing critically influence final problem-solving accuracy, whereas Chain-of-Thouur prompting unexpectedly degrades performance in some tasks. These findings estab- lish GeoBench as comprehensive benchmark while offering actionable guidelines for developing geometric problem-solving systems. Our benchmark and code are www.//o//oooooee Pront - Lerx-Lab/GeeBenchh released at https: /ttpt/gtttttt/g/g///////gt.thhgt com
INTRODUCTION
Geometric Problem Solving (GPS) requires the integration of spatial understanding, theorem appli- cation, and logical deduction analyze shapes, angles, and their relationships (Xia et al., 2024a). Tasks span from identifying basic properties of triangles to constructing complex multi-step proofs, processes that demand rigorous reasoning rather than rote memorization. While recent multi-mode large language models (MLLMs) (Zhang et al., 2024c; Gao et al., 2023) have achieved notable success on geometric tasks, with some even surpassing human performance on benchmarks like GeoQA (Chen et al., 2021), these results obscure critical flaws in evaluation practices that fail to rigorously assess models' holistic geometric reasoning capabilities.
Current benchmarks (Cao & Xiao, 2022; Lu et al., 2021; Zhang et al., 2025; Lu et al., 2024) predom- inantly rely on problems sourced from public textbooks, creating risks of test data contamination as models exploit memorized patterns rather than true reasoning. Furthermore, existing evaluations focus narrowly on final answers, neglecting the logical processes such as theorem chaining and proof generation that define geometric rigor. Most critically, the lack of diagnostic frameworks leaves unresolved whether failures stem from weak spatial perception, inefficient theorem retrieval, or limited error correction, stalling targeted advancements.
To address these challenges, we propose GeoBench, a hierarchical geometric reasoning benchmark containing 1, 021 samples to overeome the limitations of existing geometric reasoning evaluations.
"This work was in part supported by Scientific Research Innovation Capability Support Project for Young Faculty (U40) of the Ministry of Education of China, SRICSPYF-2V2025009

---

Published as a conference paper at ICLR 2026
(1) Preliminary Data Engine
(2) GeoBench Six Tasks Corvesponding to Four Levels Geometric Problem- Prabiem-Sohving
TrustSeoGen
Faccppiin
Smmrwaatt
Flock abeeme Tha Wiat satae sal Infermation
ramenc
medium liniden Nhe
[Traae]
Drrblen1
The porcllilognee ABCD aB-J6, 8C-20, monddstern, P midpo in of All and whot the nerta Df-DE?
Agr 40%%
EAT BC t0 DC (0) KE&
Tocki, Alrel fand Paecayption
strachund
prelien What
citurie qounetry
and ler (resien) Meic
sub god's
[Selution Verification]
serai fram the
GenBench Vinta Parception
Geal- Oriented plarning
acperary to
gller andi dlie
(k A Fore Roco coll
#
Oorr Co Phaorem
Seit Baflectvv BOcKTY soking
Test Paulty Freshh Lenckest
muthemen de probees
dirigg prictes
complete
god orionee restuding patt
problcm [prblinn) Whld rake
hosel' profless
coler they
preeiee
(E) Jir deromincs lnsor ingler
razsciing moge
131 kiscre
141 Niearf Mtap
from proviux
stches
canckytion
one mile
Fadty branch
Figure 1: Overview of GeoBench: (I) Generation of formally verified geometry problems with images, problems and reasoning graphs via TrustGeoGen (Fu al., 2025), followed by (2) systematic construction of hierarchical evaluation tasks through four reasoning capability levels.
Grounded in the van Hiele model of geometric thinking (Vojkuvkova, 2012), our framework strat- ifies reasoning into four hicrarchical levels: (1) Visual Perception, which focuses on extracting numerical and structural details from geometric diagrams; (2) Goal-Oriented Planning, requiring decomposing problems into sub-goals and strategic sequencing of steps; (3) Rigorous Theorem Ap- plication, demanding precise selection of theorems to ensure logical validity; and (4) Self-Refleetive Backtracking, identifying deviations in reasoning steps and iteratively correcting flawed logic.
Grounded in the proposed four-level evaluation framework, we first generate geometrie problems through TrustGeoGen (Fu et al., 2025) with all reasoning steps formally verified via symbolic proof systems. Subsequently, we construct 1, 021 validated samples across six geometric reasoning tasks based the TrustGeoGen-gentt gennrated problems and their associated reasoning graphs. As shown in Fig. 1, these tasks systematically operationalize the hierarchical levels: Numerical Perception ex- tracts quantitative attributes from diagrams, Structural Perception identifies geometric relationships, Irrelevant Premise Filtering removes distracting conditions, Sub-Goal Decomposition breaks problems into atomic steps, Theorem Selection filters inapplicable geometric theorems, and Faulty Branch Localization prevents derailment into unproductive reasoning paths. By evaluating models across this hierarchy, we dissect their geometric reasoning capabilities while correlating final-answer accuracy with task-specific competencies.
Our experiments evaluated the capabilities of existing general multimodal LLMs and reasoning mod- els on six geometry-relatee tasks, revealing varying performance levels. A general trend shows that models exhibit declining performance as task difficulty increases, with reasoning models, particularly OpenAl-o3 (OpenAI, 2025), showing the strongest overall capabilities. Furthermore, we investigated the correlation between these six tasks and final problem-solving performance. Experiments on the GeoBench dataset and the OOD GcoQA benchmark indicate that decomposing sub-goals and climinating irrelevant conditions play more critical roles in solving complex geometry problems, providing insights for enhancing geometric reasoning abilities. Additionally, examined the impact of Chain-of- -Thoughh (CoT) prompting on geometric task performance in general models. Contrary to common intuition, CoT does not universally improve performance across all tasks, and even reduces effectiveness in self-reflective backtracking tasks. These findings challenge conventional assumptions about CoT's applicability in geometric reasoning scenarios.
Our contributions are:
2

---

Published as a conference paper at ICLR 2026
Hierarchical Evaluation Framework: We propose GeoBench, novel benchmark that system- atically evaluates geometric reasoning across four progressive levels, enabling detailed diagnosis of model capabilities beyond answer-only assessments.
Key Reasoning Bottlenecks Identification: Our experiments reveal that sub-goal decompo- sition and premise filtering critically determine complex problem-solving success, providing actionable insights for improving geometric reasoning models.
Task-Specific Prompting Limitations: We observe that chain-of-thought prompting fails in faulty branch localization tasks, suggesting potential interference when prompts contain misleading reasoning steps, which may divert models from effective error correction.
2 RELATED WORK AND PRELIMINARIES
MLLMS for Geometric Problem-Solving Recent advances in Multimodal Large Language Models (MLLMs) (OpenAI, 2024a; 2025; Wu et al.. 2024; DeepMind, 2025; Bai et al., 2025) have opened new possibilities for geometric problem solving (GPS). While general MLLMS demonstrate strong performance in vision-language tasks (Li et al., 2023; Xia et al., 2023; 2024b; Yang et al., 2024), their application to GPS remains challenging due to visual-semantic discrepancies, insufficient reasoning capacity, and lack of self velf-verification mechanisms, etc. Some works address these gaps by data- driven: MAVIS (Zhang et al., 2024c) synthesizes 834K th-hug trajectories to enhance reasoning coherence, while G-LLaVA (Gao et al., 2023) leverages supermodel-guie annotation of 170K geometric solutions, both achieving superior performance over GPT-4o (OpenAI, 2024b) with compact architectures. GeoX (Xia et al., 2024a) pioneers visual- language alignment, enabling interpretable theorem verification through symbolic solvers. However, current works overemphasize final answer correctness, calling for developing structured reasoning capacities (i.e., self-reflective reasoning and theorem orchestration) to solve more advanced geometric problems.
Geometric Problem-Solvinn Benchmark. Existing benchmarks primarily collect geometry prob- lems from middle and high school textbooks (Chen et al., 2021; Cao & Xiao, 2022; Lu et al., 2021; Zhang et al., 2023; 2025; Lu et al., 2024; Zhang et al., 2024b). To mitigate annotation challenges and data scarcity, synthetic dataset construction methods have emerged. Geom Verse (Kazemi et al., 2023) enhances authentic problems through LLM-based augmentation, while other approaches (Fu al., 2025) generate synthetic data via formal language rules to prevent model-induced biases. However, current evaluations focus on final answer accuracy, neglecting deeper analysis of reasoning capabilities. Although GeomRel (Wang et al., 2025) assesses structural diagram comprehension and GeoSense (Xu et al., 2025) examines theorem application patterns, their narrow scopes fail to system- atically link subskills to overall groblem-solving efficacy. Our GeoBench addresses these limitations by evaluating four critical reasoning levels, as compared with existing benchmarks in Table 1.
Synthetic Geometry Problems Generation. TrustGeoGen (Fu et al., 2025) is scalable, rule- driven formal engine for generating synthetic geometry problems alongside verifiable solutions and structured reasoning pathways. Built on formal language framework, it integrates three core components: constructions (defining geometric elements and premises, e-g., "isosceles triangle e ABC" implies AB=AC), states (logical propositions about geometric relationships), and rules (deductive operations like triangle congruence or parallelism properties). Starting from predefined base scenes (geometric configurations with numerical data), the engine iteratively expands problems by applying constructions to derive initial premises, then progressively generates new states via rule-based inference, forming traceable reasoning graph from premises to reasoning objectives with corresponding visualizations in Fig. 1.
3 METHODOLOGY
This section presents the construction process of GeoBench. In Sec. 3.1, outline the preparatory work for this including the hierarchical evaluation framework, data distribution analysis, and problem difficulty assessment, are presented in Sec. 3.2, 3.3, and 3.4, respectively.
a complete and correct reasoning chain. Additional details,
3.1 INPUT DATA PREPARATION
For concrete approach, without loss of generality we use TrustGeoGen (Fu et al., 2025) our preliminary data engine to obtain geometric diagrams, problems, and their corresponding reasoning graphs through TrustGeoGen preliminary input. Specifically, TrustGeoGen initiates with random base scene comprising base premises. Through iterative construction augmentation, the engine constructs complex geometries by expanding the premises, where each newly introduced premise
3

---

Published as a conference paper at ICLR 2026
Table Comparison with Geometric Froblem-Solving benchmarks. "F.A." is abbreviated "Final Answer", explicitly emphasizing the verification of the problem 's final result. "V.P.", "G.P., "R.T.A.", and "S.B." denote "Visual Perception", Proal-Oriented Planning". "Rigorous Theorem Application", and "Self-Reflectiie Backtracking" respectively, denoting four-level capability evaluation in GPS.
Language
F.A.
S.B.
GeoQA (Chen et al., 2021)
754
EN&CH
Middle School
GeoQA+ (Cao Xino, 2022)
755
EN&CH
Middle School
Geometry33 3K (Ia et al., 2021)
601
EN
Middle School
/
X
X
X
PGPS9K (7hang et al., 2023)
1,000
EN
Middle School
X
X
X
GeoEval (Zhang et al., 2025)
2,000
EN&CH
Middle&High School
X
X
X
X
Goom (Kazemi et al., 2023)
2,000
EN
Synthetic
X
X
X
X
MathVista et 2024)
5,487
EN
Middle School
x
X
X
X
Verse (Zhang et 2024h)
2,612
High School
x
X
X
GeomRel (Wang et al., 2025)
2,629
Syntheric
X
X
X
GeoSense al., 2025)
1,789
EN&CH
Synthetic
X
x
X
GeoBench(ours)
1,021
EN
Synthetic
7
/
maintains topological consistency with existing elements validated by the compiler (Sicca al., 2024). The premise set can be defined as P = {Pp ). where P. defines geometric relationships (e.g., po.poonttt.... C are coll nea r) and p" specifies numerical parameters (e.g., AB=3). TrustGeoGen then leverages predefined set of geometric theorems to infer new states from premises, formulating a complete reasoning graph:
g = (P, R, ++)
P represents initial premises from the geometric constructions.
S denotes the state collection where each element E S corresponds to a derived conclusion in reasoning graph.
R is the set of deductive rules, with each E R defines the logic coherence among states.
C PS x S ((S,,r, S, (SU (), E R E 5). The notation S, formalizes the derivation of state s' by applying rule to states subset Sp..
Goal-oriented Reasoning Path. derive the reasoning path for target state s; (final solution proof state in geometric problems), TrustGeoGenn systematically constructs backward transitions in the complete reasoning graph 9 to get the goal-oriented reasoning path:
P = (s. 8) Vs E S,, 3e 8, i,i= n........... n...n,,..,11
where each triplet (S;-1, r. 8,) captures rule application r, generating state s; from antecedent states S, The process initiates from target state E S and iteratively traces upstream transitions through rule dependencies until every initial state participating & is rigorously encapsulated within the premise set P, formally satisfying Su C P where S. := S 1=
3.2 HIERARCHICAL EVALUATION FRAMEWORK
Inspired by the van Hiele model (Vojkuvkova, 2012), we categorize the geometry problem-solving capabilities into four hierarchical levels comprising six specific tasks. All data are derived from geometry problems, visual diagrams, and corresponding reasoning graphs generated by TrustGeoGen. For each task, we construct answer choices in a multiple-choice format with one correct option and three carefully designed distractors. The developed GeoBench provides comprehensive evaluation of MLLMs' geometric reasoning abilities through systematically designed cerestion-nnswee pairs.
3.2. LEVEL-1: VISUAL PERCEPTION
Numerical Perception. We focus on the MILMs' perception ability of numerical information.
Since the initial premise set of geometric problem is denoted as P {pF .P) }. we randomly select one condition from the numerical premise set p). the ground truth answer. To generate the other three wrong choices, we designed two types of modifications. numerical modification and label modification. Based
pf. numerical condition is created by randomly altering the value it defines, and label condition is modified by changing the points into one that does not exist in the geometric scene. For example, if premise AB=6, wrong numerical choice could be AB=4, and
4

---

Published as a conference paper at ICLR 2026
wrong label choice can be AY=9 (point Y does not appear in the image). Therefore, we query the MLLMs with the question: What numerical condition can you extract from this geometric image? By providing one correct choice and three wrong choices, can effectively assess the ability of MLLMs to capture numerical content in geometric images and their potential hallucinations.
Structural Perception. To generate the one correct choice, we refer to the geometric scene P = {p. P)) where P. represents geometric relationship conditions. We randomly select one premise P as the ground truth answer. To generate three wrong choices, we apply inverse negation to the other given relationship conditions. For example, if one P. states "D, E, F. are coll innerr', r", the incorrect choice would be "D, E, are not collinear". In this way, we can construct accurate data to evaluate models' ability discern geometric structural information. We evaluate the MLLMs' ability to perceive geometric relations in the image by giving it four choices and querying: What structural information can be extracted from this geometric image?
3.2.2 LEVEL-2: GOAL-ORIENTED PLANNING
Irrelevant Premise Filtering. We systematically investigate the capability of MLLMs to filter out redundant premises and extract semantically relevant information in geometric reasoning tasks. Formally, let the initial premise set of a geometric problem be denoted as P. with Sy S P representing the subset of premises actually employed in the goal-oriented reasoning path. The set of unused premises is consequently defined as P So. To ensure meaningful evaluation, we apply two selection criteria to filter out all problems: P ) So * 0 and |.S,) 2 3. The evaluation protocol consists of prompting MLLMs with the interrogative: Which premise condition is unnecessary for deriving the conclusion? ? The ground truth answer is configured as any element p E (P \ So), while the distractor set comprises three randomly selected premises {p p E So)
Sub-Goal Decomposition. We aim to investigate the sub-goal decomposition capability of MLLMs. common reasoning strategy involves backward chaining, starting from the target conclusion and identifying the necessary preconditions required to derive it. Formally, let 8 represent the target conclusion of a geometric problem, and rz E R denote the final inference rule applied to obtain 8,. The preceding state set Sr, satisfies the relation S,, 4 st. indicating that when the reasoning process reaches state set S,,, the conclusion 8, can be derived through a single application of rule 9t. Therefore, we prompt the MLLMs with the question: What conditions must be established to derive conclusion st. Given that P represents the set of premises, configure the correct option S,,, P (the necessary intermediate conditions excluding initial premises). Let Salt denote the complete set of intermediate states encountered during the geometric reasoning process. The incorrect options are systematically constructed AUB, where A (S., \ (P) and B C (Sau \ P S.
In geometric proof and problem-solving
3.2.3 LEVEL-3: RIGOROUS THEOREM APPLICATION
Theorem Selection. We aim to investigate whether MLLMSs can effectively identify and filter out unused theorem rules. First, we filter questions where the construction of the inference graph G involved at least three rules (i.e., r). For each filtered question, its corresponding inference process contains a used rules subset Rased = {r1, T2; .2,, [k] with where 2 3 and Rused C R. Subsequently, we randomly select three distinct rules {T [Twonng_1, 1, T Twrong_2: Twweagg3) } C Rused from Rused as incorrect choices, and randomly select an unused rule Teomect (i.e., Teotrect of Ruyed) from the multiple-choice question with
entire rule repository R as the correct choice. The final question four choices, asking to identify which rule was not used to solve this problem? The four choices are Twrong_ 1. Twwnng 2, Twanng_3, 3, and Feoorett, where Feorrea is the only correct answer.
3.2.4 LEVEL-4: SELF-REFLECTIVE BACKTRACKING
Faulty Branch Localization. Trace-back reasoning data is defined as a subgraph of the complete reasoning graph G = (P, S. R, >) characterized by identical upstream premises and intermediate states but diverging at the final state. As illustrated in Fig. 1, the correct path P (the goal-oriented reasoning path marked in Fig. 1) consists of the light-colored nodes that successfully reach the correct target state 8t. While the subset of light-colored nodes represents a wrong reasoning path Pwmnng although sharing partial upstream nodes with P. it ultimately leads to erroneous terminal state Swreng. We formally define the faulty reasoning branch as the set difference:
(3) For localization, we denote stepo. the initial divergence step where Pwrong first deviates from valid reasoning the correct choice. Similarly, we have also randomly sampled three reasoning steps
Pranit- := Pwrong P C Pwrong
5

---

Published as a conference paper at ICLR 2026
1.0 Geo
6.
Gaobench imagas
(GeuBanch)
Rco limit
Ge/06
1.000
2.009
04
0.000
CooD Cc sotution
8.0000
$1.0
1,00
2309
2000
2200
2,000
(a) I-SNE of image emheddinngs
(b) t-NE of solution emheddings.
(c) Length of solution tokens.
Figure 2: Data Distribution Comparison between GeoBench and GeoQA Chen al. (2021).
Table 2: Data amount distribution across six tasks in GenBench.
Numerical Perception
Structral Trrelevant Premise Fittering
Sub-Gnal Decomposition
Theorem Selection
Faulty Branch Localization
Total
Perception
168
135
200
171
from P Wroong to serve as incorrect options. The final task is to prompt the MLLMS to point out the initial erroneous reasoning step in given wrong solution path Pwrong for a geometric problem. Since trace-back reasoning serves as a highly effective approach, the faulty branch localization data provides a reliable inspector to evaluate whether models truly possess self-reflective abilities.
3.3 DATA DISTRIBUTION ANALYSIS
Details about GenBench. Based on the aforementioned methodology, GenBench incorporates six tasks corresponding four difficulty levels, comprising a total of 1, 021 question-answer pairs, which aligns with the typical scale of specialized geometric reasoning benchmarks shown in Table 1. The quantitative distribution of these tasks is presented in Table 2. Among the geometric problem-solving objectives, 285 require numerical solutions while the remaining 736 are proof-based. Through TrustGeoGen implementation, curated 40 base scenarios, covering most real-world geometrie problem scenarios. Additional details are provided in the Appendix C.
employed 76 geometric constructions, 42 deduction rules, and
Comparison with OOD Benchmark. To understand the distributional characteristies of GeoBench, we visualize datasets of images and solutions using t-SNE (Van der Maaten & Hinton, 2008) compared with GeoQA (Chen et al., 2021).
1) Image-Level. As shown Fig. 2a, the images embeddings form GeoBench and those from GeoQA form largely disjoint clusters, with our GeoBench being significantly wider spread. This indicates that GeoBench covers more diverse set of visual pattens, reflecting a broader and more comprehensive range of problem instances.
2) Solution-Level. We perform similar t-SNE(Van der Maaten & Hinton, 2008) analysis on solutions, as shown in Fig. 2b, the GeoBench and GeoQA show small overlap. Notably, the GeoBench solutions are more dispersed and span a larger region of the embedding space. Furthermore, we complement this geometric analysis with quantitative comparisons of solution token lengths between the benchmarks. As demonstrated in Fig. 2c, GcoQA solutions exhibit significantly shorter token sequences (maximum 189) compared GeoBench's 's solutions, which predominantly maintain thousand-token-scale complexity that underscores GeoBench's heightened solving complexity.
3.4 PROBLEMS DIFFICULTY ASSESSMENT
We evaluate geometric problems from GenBench alongside three real-world benchmarks (GeoQA, Geometry3K, and OlympiadBench- Experimental results are presented in Table 3, demonstrating that model performance on GenBench is slightly weaker than that on OlympiadBench-CC Geo (Olympiad-leve)) therefore empirically indicating that the complexity and difficulty of synthetic problems in GeoBench have surpassed the middle- school difficulty tiers and generally fall within the elite problem-solving tiers.
(Zhang et al., 2024a)) across multiple reasoning models.
EXPERIMENTS
In this section, we evaluate state-of-the-ar multimodal large models on GeoBench and investigate the following research questions through experimental results:
Here, solution refers to the solving proving process targeting geometric objectives, rather than the responses the six tasks in GeoBench

---

Published as a conference paper at ICLR 2026
Table 3: Evaluation results on real-world datasets.
GeoQA school level)
Geometry3K (mid-schooe school level)
OlympiadBench-Ger (Otympiad level)
GeoBench-solving (ours)
Qwen2-VI--hb
36.34%
24.46%
7. 7.14%
4.17%
GPT-40
42.31%
31.45%
13.39%
22.08%
Geminni-2.5-pro
79.58%
80.70%
75.00%
49.58%
Table 4: Performance of oate-of-ttheart multi-modal language models on 6 tasks, which are divided into four difficulty levels. "N.P.", "S.P.", "I.P.F... "S.D.", "T.S.", "F.B.L." " correspond to tasks Numerical Perception, Structural Perception, Irrelevant Premise Filtering, Sub-Goal Decomposition, Theorem Selection and Faulty Branch Localization respectively. A denotes accuracy gain versus random choice and blocks indicate performance above/below random guessing.
Levdl
Level
Level
Model
N.P.
S.P.
LPF
S.D.
RRI.
Randica Choices
24.64%
25 7848
26.159%
24.55 54.55%
25.57%
Human Results
100.00%
+75.36%
100.00%
-74.22%
77.50%
+51.35%
100.00%
+75,45%
$6.6776
-31,10%
52.94
+27.73%%
Gemeral
MILMss
LLaVA-15-70 15]
28.57%
+3.9315
25.19%%
40.59%%
24.50%
-1.65%
19.00%
-5.55%
26.90%
+1.231%
13 1. [15]
25.00%
+0.365 369
24.44%
-1 345%
30.0095
+3.85%
12.50%
+7.95%
26 509
+1.235%
LLaVA-OV [13]
58.93%
+34,29%
22.96/%
2.821%
26.50/
+0.355
33.00/
+3.15%
16.49%
9.08%%
Mobno [8]
75.57%
+53.9356
12.599
-13. 19%
32,00%
45.855%
31.000 11.00%
+6.45%
26.90%
+1.33 +1.334%
19.3546
-7.35%
Qwen2-VL-Th [36]
54.17%
+79 539
18.52%
-7 7.26%
27.00%
+0.85%
36 00%%
-11.45%
33.71%
1.8%%
17.60 19.69%
-8.02%
Owen2 VL-72b [26]
86.31%
29.63%%
60
+35.95% 9516
-11.54%
21.77%
-3.45%
87.50%
866
+16.45 455
54%
25.17%
Qwen2.5-VL.
85.71%
007%
GPT-40 120)
06.07%
22.96%
+17.85%
57%5
932.9515
35.09%
+9.52%%
23.81 81.
-1.90%
GPT-4V [18]
51.79%
15%
34.81%
35.50%
+9.35%
50%
+33.95%
38.01
112.445
27.89%
DeepSeek: V12 (30)
66.07%
+11.43%
25.19%
0.59%
27.00%
+0,85%
36.00%
+11.45%
23.98%
1.59%
22.45%
3.26%
Reasoning
MLLMS
QvQ-72/[23]
61.31%
+36667
27.41%
+3.63%
29.50%
+3.3555 35%
50.00%
+34.45
44.33%
+18. 765
18.57%
-7.34%
OpenAI-ol [19]
75.00%
+50.36%
65.19%
-39.41%
61.50%
35.35%
77 10/%
+50.5
53.22%
+27.65%
27.89%
+2.15%
OpenA1-o3 [23]
80.95%
+56.31%
74.81%
+49.133%
70.00%
+43.85%
91.00%
+60. 45%
54.39%
+29. M216
22.85%
-3.25%
Slmde--,7-somee
87.58%
+62.80 86
46.67%
+30.80 -30.895
52.50 52.50%
+76, 359
67. 50%
+42. 9/66
45.07%
-19.50 46
21 17.7
-3.94%
Gemini 2.5
80.95%
+56.31%
60.00%
+34.22%
74.00%
+17.85%
00%%
+62. 15%
45.00%
06%
18. 18.35%
-7.34%
DeepNock-k1
415.955
Agent
CodePlot [9]
7381%
+49.17%6
35.52%
+12.74%
30.50%
+4.35%
41.50%
+669996
19.30%
4.27%
15.059
-10,00%
How do current MLLMs perform in geometric reasoning across different hierarchical levels?
Which geometric reasoning abilities are more indicative of a multimodal model's 's problem- solving proficiency in geometry?
Can the evaluation results from GeoBench generalize to OOD geometric test sets?
Is Chain of Thought (CoT) truly effective for geometric reasoning tasks?
4.1 DATASETS, METRICS AND IMPLEMENTATION DETAILS
Datasets. As written in Sec. 3, we divide geometric reasoning capabilities into four levels and use TrustGeoGen to generate six specialized geometric tasks covering all levels. Each instance consists of a multiple-choice question, including natural language problem description, one correct option, and three incorrect options. Regarding answer validation, we confirm that our dedicated team of eight doctoral researchers and PhD candidates, all possessing strong mathematical competencies, performed comprehensive cross- verificatoi..
Metrics. For General MLLMs, we adjust the prompt to have models output option indices directly. For Reasoning MLLMs, which generate reasoning processes, we specify an output format in the prompt that requires placing the deduced option index within \boxed() } brackets. After extracting answers, we calculate the accuracy(ace) for each task and compare it with the baseline model (random choices) to derive relative performance difference(A).
Implementation Details. We conducted all inference evaluations of open-source models on 8 NVIDIA A100 (80G) GPUs, while performing evaluations of proprietary models on CPUs. For detailed parameter configuration, please refer to Appendix C. 4.2 HUMAN EVALUATION
We equally distributed the data to five human experts (each holding a Ph.D. from top 1% national university) for manual evaluation. The results, calculated as the arithmetic mean with the exclusion of the highest and lowest ratings, are presented in the Table 4. Human experts consistently achieved
VLa series models cannot be evaluated on the F.B.L. task due to input token length limitations. Since DeepSeek-RI is text-only model, it cannot he tested on I Level visual tasks. For all other tasks, visual conditions fully provided in textual descriptions.
7

---

Published as a conference paper at ICLR 2026
Table 5: Comparative performance state-of-the-ar multi-modal language models on 6 tasks in GeoBench, with final answer accuracy on GeoBench-sollinng, GeoQA and Geometry3K datasets.
Level 1
Level 2
Level
Level
GeoBench- solving
Model
N.P.
S.P.
L.F.
S.D.
I.S.
F.B.L.
GeoQA
GCometry3K
General MIL.Ms
LLaVa-1.5-7b [15]
28.57%
25. 19%
24.50%
19.00%
26. 80%
2.50%
17.11%
1.16%
LLaVa-1.5-11 [15]
25.00%
24.44%
30.00%
32.50%
26.80%
3.75%
18.57%
2.33%
Qwen2-VL. [26]
54.17%
18.52%
27.00%
36.00%
23.71%
17.69%
4.17%
36.34%
24.46%
Qwen2 72h [26]
86.31%
29.63%
31.00%
60.50%
37.11%
21.77%
10.83%
55.70%
34.44%
5 [2)
87.50%
18.52%
32.00%
41.00%
37.11%
25.17%
11.67%
54.64%
35.94 25.94%
Qwen2.5 72h (2)
85.71%
40.74%
38.50%
77. 77.00%
47.42%
26.53%
17.50%
67.90%
48.42%
GPT-4o [20]
66.67%
22.96%
44.00%
57.50%
35.09%
23.81 %
22.08
.55%
GPT-4v [29]
51.79%
34.81% %
35.50%
57.50%
38.01%
27.89%
19.17%
35,94%
20.63%
Reasoning
MLLMs
QvQ-72b (0Q0-22bug [22]
61.31%
27.41%
29.50%
59.00%
44.33%
18.37%
61.67%
45.09%
OpenAT-o1 [19]
75.00%
65. 19%
61.50%
77.00%
53.22%
27.89%
48.33%
75.83%
71.29%
OpenAl-o3 [21]
80.95%
74.81%
70.00%
91.00%
54.39%
22.45%
53.33%
83.33%
80,20%
Gemini-2.5-pro (6]
80.95%
60.00%
74.00%
87.00%
45.03%
18.37%
49.58%
79.58%
80.70%
Claude-3.7-Soman [1]
87,50%
46.67%
52,50%
67,50%
45.03%
21.77%
35.42%
49.73%
33.28%
higher accuracy than MLLMs across all tasks. This performance gap was particularly pronounced for the N.P. and S.P. tasks, which require image-grounding, and the F.B.L. task, which involves more complex reasoning.
4.3 PERFORMANCE ON ALL 4 LEVELS
In Table the results on the GeoBench reveal that MILMs possess a certain level of reasoning capability in solving geometric problems, with performance generally declining as the difficulty level increases. Among the existing open-source models, the evaluation results of the LLaVa-1.5 (Liu et al., 2024) series are comparable to random selection, indicating relatively weak geometric perception and reasoning abilities. The LLa LLaVa-OV-7b (Li et al., 2024) model only slightly outperforms others in Level 1 (Visual Perception) tests. Other models demonstrate strong performance in Level and Level 2 (Goal-Oriented Planning), with modest improvements over baseline models in Level 3 (Rigorous Theorem Application). Comparative analysis suggests that as model parameters increase and versions advance, genuine geometric reasoning capabilities also improve.
We included the Molmo (Deitke et al., 2024) model (an open-source vision-language model trained on the PixMo (Deitke et al., 2024) dataset, a collection of image-text pairs) in our testing, with the results presented in the Table 4. Based on the results presented in the table. it can be observed that the performance the Molmo (Deitke et al., 2024) model falls short of that achieved by proprietary commercial MLLMs models, indicating its relatively limited capability in geometric reasoning.
Proprietary commercial models demonstrate superior reasoning capabilities. For instance, OpenAI- o3 (OpenAI, 2025) achieves 91.00% accuracy the S.D. task and 54.39% accuracy on the T.S. task, while OpenAl-ol also attains a comparable accuracy of 53.22% on the T.S. task-significantly outperforming most other models which generally exhibit poor performance these tasks.
Across most tasks, reasoning MLLMs outperform general MLLMs. Top-performing models such as OpenAl-ol (OpenAI, 2024a), OpenAl-o3 (OpenAI, 2025), Claude-3.7 7-sonnet (Anthropic, 2024), and Gemini-2.55pro (DeepMind, 2025) surpass 70% of general models in N.P. task, exceed all general models S.P. and I.P.F. tasks, and outperform over 90% of general models in S.D. and T.S. tasks. All evaluated MLLMs exhibited severe limitations in F.B.L. performance. The highest accuracy achieved by OpenAl-ol (OpenAI, 2024a) reached merely 27.89% a statistically insignificant improvement compared to the 25.71 % baseline of random choices.
Finally, we also selected an agent called CodePlot (Duan al., 2025) for testing, which can use Python to generate plots during the reasoning process to assist in thinking. The experimental results show that its performance is somewhat inferior to that of commercial large models, particularly with significant disadvantage in T.S. and F.B.L. tasks.
We conducted repeated experiments to demonstrate the reliability of the results. For details, please refer to the Appendix E.
8

---

Published as a conference paper at ICLR 2026
Table Spearman correlation (p) among six tasks and final-answer evaluation.
N.P.
S.P.
L.P.F
S.D.
T.S.
F.B.I.
with Geolench-SSllving
0.40351
0.75657
0,97902
0.88772
0.82954
0.49561
with GeoQA
0.65859
0.67181
0.74945
0,93392
0.84896
0,50057
with Geometry3K
0,59780
0.64374
0.71429
0,88981
0.81932
0,38122
1.
1.0
al Ext
0.7
0.-
0.6
0.8
0.5
0.5
0.5
0.4
0.4
94
0mir2.-11 Qwerd.3-12
Gward.3-123
Figure 3: Performance comparison w/ & w/o Srain-of-Thougeee (CoT) across six tasks in GeoBench.
4.4 CORRELATION AMONG TASK-SPECIFIC CAPABILITIES AND FINAL-ANSWER
PERFORMANCE
To investigate which geometric reasoning abilities have a greater impact on the performance of MLLMs in solving geometric problems, detailed in Table 5, we validate the final answer on GeoBench-solvinng the preliminary set of 285 geometric problem-solving questions with numeric answer from GeoBench. For each task, we constructed a feature vector X; based on the test results across different MLLMs and computed the Spearman Correlation Coefficient (De Winter et al., 2016) between X, and the CroBennch-solvingg vector Y to analyze the influence of each task on GPS ability,
The Spearman Correlation Coefficient (p) is rank-based measure that indicates the direction and strength of the relationship between an independent variable X and a dependent variable Y. Its value ranges from [-1, 1]. where values closer to or -1 indicate stronger correlation. As observed in Table 6, the first six tasks exhibit a positive correlation with GeoBench-solvine results, meaning that stronger geometric reasoning abilities correspond to better problem-solving performance. Among them, task I.P.F., S.D., and T.S. show the strongest correlations, followed by S.P. and N.P., with F.R.L. having the weakest correlation. This finding confirms that Sub-Goal Decomposition, Irrelevant Premise Filtering, and Theorem Selection abilities significantly influence MLLMs' geometric problem-solving capabilities.
4.5 ANALYSIS ON
DATASET
To validate the aut-of-distribution (OOD) generalization ability, evaluated the geometric problem- solving performance of MILMs on GeoQA and Geometry3K (see Table 5) and further computed the Spearman correlation coefficients between each task and the GeoQA, Geometry3K evaluation results (in Table 6). The correlation analysis reveals that task S.D. and T.S. exhibit the strongest correlations, followed by I.P.F., then N.P. and S.P., with F.B.L. showing the weakest correlation. This trend remains consistent across GcoQA, Geometry 3K, and boooBench-solviggg collectively confirming the robust -ut-of-distributtin generalization capability intrinsic GeoBench. By leveraging our benchmark, we confirm that the improvement in MILMs' geometric reasoning abilities is genuine and not merely a result of overfitting the answer patterns GeoQA or Geometry3K.
4.6 APPLICABILITY CHAIN-OF-THOUGHT PROMPTING
To enhance the reasoning capabilities of General MLLMs, we explored the optimization of prompts using Thought (CoT) (Wei et al.. 2022) methodology. which yielded intriguing findings. Specifically, we conducted ablation studies on the Qwen series (Wang et al., 2024; Bai al., 2025) of large models as well as GPT-40 (OpenAI, 2024b), evaluating their performance across six tasks with both CoT-enhanced prompts and non-CoT prompts. In the CoT condition, models were prompted with the instruction "let's think step by step", which elicited detailed reasoning processes from the models. Conversely, in the non-CoT condition, the prompt " "only output the answer" n employed, restricting the models to provide answers without explicit reasoning steps. Additional details regarding the prompt construction and implementation are provided in Appendix D.

---

Published as a conference paper at ICLR 2026
Table 7: Performance on Perturbed Dataset.
Level
Level
Level
Level 4
N.P.
S.P.
I.P.F.
S.D.
T.S.
F.B.I.
Owen2-VLL-70 [26]
57.14%
23.70%
25.50%
38.00%
26.32%
22.45%
Qwen2-VILs Vi.72- [26]
80.95%
33.34%
28.005
56.50%
39.77%
19.73%
Qwen2.5-VL-22 [2]
79.17%
20.74%
28.50%
35.50%
35.09%
23.13%
Qwen2.5 VL. [2]
83.93%
40.00%
39.00%
76.00%
44.44%
24.49%
Table 8: Text-Only Experiments Results.
Level 1
Level 2
Level
Level 4
N.P.
S.P.
I.P.P.
S.D.
T.S.
F.B.I.
Qwen2-VL. [26] w/ text
33.93%
23.70%
29.50%
36.00%
23.71%
22.45%
Qwen2-VL-7b text-image
54.17%
18.52%
27.00%
36.00%
23.71%
17.69%
Qwen2-VL-72bb [26] w/ text
47.62%
19.26%
27.50%
59.50%
30.99%
21.09%
Qwen2-VL-72b [26] w/ lext+image
86,31
29.63%
31.00%
60.50%
37.11%
21.77%
Qwen2.5-V1--77 [2] w/ text
39.29%
19.26%
28 00%
40.00%
25%
24.49%
Qwen2.5- VL.-7h [2] w/ text+image
18.52%
41.00%
37.11%
25.17%
Qwen2.5-VL-722 [2]
51.19%
22.96%
39.00%
75.00%
43.27%
21.77%
Qwen2.5-VVL-700 [2] w/ text+image
85.71%
40.74%
38.50%
77.00%
47.42%
26.53%
The experimental results are visualized in the line chart presented in Fig. 3. Analysis of the trend lines reveals that the CoT approach demonstrates marginally superior performance compared to the non-CoTT method in tasks S.P., I.P.F, and S.D., while showing inconsistent results in N.P. and T.S. tasks. Notably, the results of F.B.L. task indicate a systematic underperformance method relative to its non-CoT counterpart, suggesting that CoT is ineffective for this particular task specifically, fails to enhance the models' Faulty Branch Localization capability.
the CoT
Upon closer examination, we hypothesize that this phenomenon may be attributed to the presence of erroneous reasoning chains in F.B.L. task prompts. These misleading cues could potentially interfere with the reasoning processes of general MILMs when handling complex geometric problem-solvin tasks. This observation suggests a nuanced perspective: While CoT can enhance model performance under certain conditions, its effectiveness for geometric reasoning appears contingent on information reliability. Specifically in complex geometric scenarios, our findings indicate that CoT methodology might not consistently improve performance across general MLLMs, revealing important limitations regarding its application in mathematically rigorous domains.
4.7 PERTURBED DATASET EXPERIMENTS
We applied random rotations of -0 to the images to create a perturbed test set and conducted corresponding experiments. The results are presented in the Table 7 below. Based on comparative analysis with the evaluation results of the Qwen (Wang et al., 2024; Bai et al., 2025) MILMs presented in Table 4 of the paper, we observe that the performance metrics remain remarkably consistent. This finding empirically demonstrates that minor image perturbations (within +200 rotation) do significantly affect the evaluation outcomes.
4.8 TEXT-ONLY EXPERIMENTS
We conducted text-only ablation experiments on MLLMs, with results presented in the Table 8. Ex- perimental findings indicate that most vision-language models exhibit inferior evaluation performance when processing text-only inputs compared to multimodal inputs incorporating both images and text.
5
CONCLUSION
Our work establishes GeoBench as diagnostic framework for rigorously evaluating geometric reasoning through four hierarchical capability levels. Experimental analyses on MILMS reveal two critical bottlenecks: Sub-Goal Decomposition and Irrelevant Premise Filtering significantly impact complex problem-solving success, while Chain-of-Thought prompting exhibits task-specific limita- tions, particularly impairing Self-Reflective Backtracking. These findings challenge the assumption of universal CoT efficacy in geometric reasoning and highlight the necessity of structured evaluation beyond fnnl-answer metrics. The GeoBench framework provides actionable pathways for advancing geometric reasoning through targeted capability enhancement. Future work should expand task diversity while rigorously refining structured evaluation protocols for mathematical precision.
10

---

Published as a conference paper at ICLR 2026
Ethics Statement. Our proposed GeoBench may ultimately find applications in the field of educa- tional technology, such as in intelligent tutoring systems. We are committed to advancing this work responsibly, with the objective of assisting human learning and fostering interest in mathematics, rather than replacing deep human reasoning and creative thinking. We will refrain from pursuing any applications that could potentially facilitate academic misconduct, such automated problem- solving for cheating purposes. Besides, we will avoid using language that might lead to public misunderstanding excessive hype (claims like "AI has surpassed human mathematical abilities"), Instead, we will accurately and objectively delineate the MLLMs' capabilities, emphasizing that its current performance primarily evaluated within specific, controlled synthetic environments.
Reproducibility Statement. The complete source code and the full evaluation dataset have been made publicly available at the anonymous link: https: : /gi thub. com/F ront eerX-Lab/ GeoBench. Furthermore, both the code and the data will be open-sourced in the future.
REFERENCES
Anthropic. The claude model family: Opus, sonnet, haiku. 2024.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint arXiv: 250. 2025.
Jie Cao and Jing Xiao. An augmented benchmark dataset for geometric question answering through dual parallel text encoding. In Proceedings of the 29th International Conference on Computational Linguistics, pp. 151 1-11-1200, 2022.
Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingho Liu, Eric Xing, and Liang Lin. Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning, In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 513-523, 2021.
Joost CF De Winter, Samuel D Gosling, and Jeff Potter. Comparing the pearson and spearman correlation coefficients across distributions and sample sizes: A tutorial using simulations and empirical data. Psychological methods, 21(3):273, 2016.
DeepMind. Gemini-225-pro. https: /ttpe:/ddeeemmddde hwap:pmiidd.ggglll...mommo googlle/technoooggess//
pro/, 2025.
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang. Jun-Mei Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang. Xiaoling Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bing-Li Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dong-Li Li Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li. H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, Jiong Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang. Minghua Zhang. M. Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang. Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan. Runji Wang.
Chen, Ruiqi Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shao-Kang Wu, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng. Wanjia Zhao, Wen Liu, Wenfeng Liang. Wenjun Gao, Wen-Xia Yu, Wentao Zhang, W. L. Xiao, Wei An. Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng. Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyu Jin, Xi-Cheng Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yi Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang
11

---

Published as a conference paper at ICLR 2026
Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yu-Jing Zou, Yujia He, Yunfan Xiong, Yu-Wei Luo, Yu mei You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanping Huang, Yao Li, Yi Zheng, Yuchen Zhu, Yunxiang Ma, Ying Tang, Yukun Zha, Yuting Yan, Zehui Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhen guo Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zi-An Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi. Yue Yang. Jae Sung Park, Moham- madreza Salchi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-an multimodal models. arXiv e-prints, pp. arXiv-2409, 2024.
Chengqi Duan, Kaiyue Sun, Rongyao Fang, Manyuan Zhang, Yan Feng, Ying Luo, Yufang Liu, Ke Wang, Peng Pei, Xunliang Cai, et al. Codeplot-cot: Mathematical visual reasoning by thinking with code-driven images. arXiv preprint arXiv:2510.11171 2025.
Daocheng Fu, Zijum Chen, Renqiu Xia, Qi Liu, Yuan Feng, Hongbin Zhou, Renrui Zhang, Shiyang Feng. Peng Gao. Junchi Yan, Botian Shi, Bo Zhang, and Yu Qiao. Trustgeogen: Scalable and formal-verified data engine for trustworthy multi-modal geometric problem solving. arXiv preprint ar0iv:2504. 15780, 2025.
Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, Yufei Wang, Lanqing Hong, Jianhua Han, Hang Xu, Zhenguo Li, al. G-llava: Solving geometric problem with multi-modal large language model. arXiv preprint arXiv:2312.113700 2023.
Mehran Kazemi, Hamidreza Alvari, Ankit Anand, Jialin Wu, Xi Chen, and Radu Soricut. Geomverse: systematic evaluation of large models for geometric reasoning. arXiv preprint arXhv:2312.12241 2023.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.083: 03326, 2024.
Mingsheng Li, Xin Chen, Chi Zhang. Sijin Chen, Hongyuan Zhu, Fukun Yin, Gang Yu, and Tao Chen. M3dbench: Let's instruct large models with multi-modal 3d prompts. arXiv preprint arXiv:2312.10763 2023.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning. arXiv preprint arXiv:2105.04165 2021.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li. Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang. Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations (ICIR), 2024.
OpenAl. Gpt-4v. http::://ppeaai.coo//neeeee
www.ww--ssttmaraaraeeeea 2023.
OpenAI. Openai-ol.
OpenAl. Hello gpt-4o. http://pe///ggaaaaaac/////////ee 2024b.
OpenAl. Openai-o3. http://pppppppppopaaacc///aaaaaa and 04minn
2025. Qwen. http://pwwe.eeen......//////////
Vladmir Siec. Tianxiang Xia. Mathis Fédérico, Philip John Gorinski, Simon Frieder, and Shangling Jui. Newclid: A user-frice beer-friendly replacement for alphageometry. arXiv preprint arXiv:2411.11938. 2024.
12

---

Published as a conference paper at ICLR 2026
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
I Vojkuvkova. The van hiele model of geometric thinking. WDS' Proceedings of Contributed Pupers, 1(1):72-75, 2012.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang. Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024.
Xiaofeng Wang, Yiming Wang, Wenhong Zhu, and Rui Wang. Do large language models truly understand geometric structures? International Conference on Learning Representations (ICLR), 2025.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, F. Xia, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.1 / 1903, 2022.
Penghao Wu and Saining Xie. V Guided visual search as core mechanism in multimodal 1lms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13084-13094, 2024.
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bing-Li Wang. Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang. Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yu mei You, Kaihong Dong. Xingkai Yu, Haowei Zhang, Liang Zhao, Yisong Wang, and Chong Ruan. Deepseek-v12 v12: Mixture-of-experts vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412.10302. 2024.
Renqiu Xia, Bo Zhang, Haoyang Peng, Ning Liao, Peng Ye, Botian Shi, Junchi Yan, and Yu Qiao. Structchart: Perception, structuring, reasoning for visual chart understanding. arXiv preprint MrXiv:2309.1 2023.
Renqiu Xia, Mingsheng Li, Hancheng Ye, Wenjie Wu, Hongbin Zhou, Jiakang Yuan, Tianshuo Peng. Xinyu Cai, Xiangchao Yan, Bin Wang, Conghui He, Botian Shi, Tao Chen, Jun-Wei Yan, and Bo Zhang. Geox: Geometric problem solving through unified formalized vision-language pre-training. In International Conference on Learning Representations (ICLR), 2024a.
Renqiu Xia, Bo Zhang, Hancheng Ye, Xiangehao Yan, Qi Liu, Hongbin Zhou, Zijun Chen, Min Dou, Botian Shi, Junchi Yan, al. Chartx & chartvlm: A versatile benchmark and foundation model for complicated chart reasoning. arXiv preprint 240. 2024b.
Liangyu Xu, Yingxiu Zhao, Jingyun Wang, Yingyao Wang. Bu Pi, Chen Wang, Mingliang Zhang, Jihao Gu, Xiang Li, Xiaoyong Zhu, Jun chao Song, and Bo Zheng. Geosense: Evaluating identification and application of geometric principles in multimodal reasoning. arXiv preprint 12%4572504.12597 2025.
Yue Yang, Shuibai Zhang, Kaipeng Zhang, Yi Bin, Yu Wang, Ping Luo, and Wenqi Shao. Dynamic multimodal evaluation with flexible complexity by vision-language bootstrapping. In International Conference on Learning Representations (ICLR), 2024.
Jiarui Zhang, Ollie Liu, Tianyu Yu, Jinyi Hu, and Willie Neiswanger. Euclid: Supercharging multimodal Ilms with synthetic high-fidelity visual descriptions. arXiv preprint arXiv:2412.08737, 2024a.
Jiaxin Zhang, Zhongzhi Li, Ming-Liang Zhang, Fei Yin, Cheng-Lin Liu, and Yashar Moshfeghi. Geoeval: Benchmark for evaluating umms and multi-modal models on geometry problem-solving. arXiv preprint 101144 2025.
Ming-Liang Zhang, Fei Yin, and Cheng-Lin Liu. A multi-modal neural geometric solver with textual clauses parsed from diagram. arXiv preprint arXiv:2302. 1097, 2023.
13

---

Published as a conference paper at ICLR 2026
Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai- Wei Chang, Peng Gao, and Hongsheng Li. Mathverse: Does your multi-modal Ilm truly the diagrams in visual math problems? In European Conference Computer Vision, 2024b.
Renrui Zhang, Xinyu Wei, Dongzii Jiang, Yichi Zhang, Ziyu Guo, Chengzhuo Tong, Jiaming Liu, Aojun Zhou, Bin Wei, Shanghang Zhang, et al. Mavis: Mathematical visual instruction tuning. arXiv preprint 44770800, 2024c.
14

---

Published as a conference paper at ICLR 2026
A
DECLARATION OF AI USE
We utilized large language models, including DeepSeck-V3. GPT-4, and Gemini, to assist with the writing process by:
Translating technical terms into idiomatic English.
Correcting grammatical errors and improving sentence fluency.
Polishing the overall wording.
We assure that ideas, methods, code implementations, experiments, analyses, and conclusions done by human researchers ourselves.
B LIMITATIONS AND BROADER IMPACT
B.1 FALIURE CASES STUDY
We conducted failure analysis of GeoBench across its six tasks to identify key limitations. In the N.P. and S.P. tasks, complex diagrams sometimes caused perceptual errors large models, such as misreading labeled line segments overlooking point collinearity (Fig. 4a, 4b). For I.P.F. models produced incorrect intermediate conclusions or misinterpreted problem descriptions (Fig. 4c). In S.D., they often mistook irrelevant premises necessary and introduced hallucinated conditions (Fig. 4d). During T.S., models drew erroneous conclusions from incomplete evidence (Fig. 4e). Notably, in F.B.L., MLLMs primarily detected factual errors (incorrect single-step deductions) rather than the intended directional errors (steps deviating from the goal) (Fig. 4f).
B.2 LIMITATIONS
In this work, propose GeoBench, novel benchmark that systematically evaluates geometric reasoning across four progressive levels. Currently, our geoBench only applicable to planar geometry, the utilized data engine TrustGeoGen and generation pipeline are designed specifically for 2D geometric problems. To extend the benchmark to 3D geometric structures, new spatial rules and formal definitions tailored to three-dimensional settings would need be introduced, marking promising direction for future research.
B.3 BROADER IMPACT
GeoBench aims to reshape the evaluation paradigm for geometric reasoning in large multimodal language models by shifting the focus from answer correctness to reasoning fidelity. This benchmark provides standardized, verifiable framework for diagnosing fine-grained reasoning failures, which may inform the design of educational technologies, tutoring systems, and trustworthy models in mathematics. Furthermore, different levels of geometric abstraction, offering insights for geometric reasoning improvement and mathematical curriculum design.
task decomposition reveals the intelligence demands required at
Importantly, we recognize existing score-centric benchmarks could risk overfitting models to senchmark-specific reasoning templates, thereby reinforeing narrow optimization rather than general geometric understanding. To mitigate this, we propose the multi-dimensional GeoBench, advocating that future applications should prioritize diversity, such image perception, problem formulations, and reasoning paths. Ultimately, GeoBench contributes toward building more interpretable and cognitively aligned models capable of solving geometry problems rigorously and systematically.
C EXPERIMENTAL DETAILS
C. PARAMETER CONFIGURATION
For all MILMs, we set the parameters as temperature 0.95, max tokens max_tokens_ 8192, and top_p 0. 7. For APi-based models (including OpenAl-ol/o3, GPT-40/4v, DeepSeek-RI, Claude-3.7 S.tude-3.7-somen 3.7-sonnet, and Gemini-2.5-pro)) we implemented a retry logic that allows up to 5 API reattempts in case of network connection enrors.
15

---

Published as a conference paper at ICLR 2026
Deseriptio::
Dosrigtin::
Hlass powetry prabtes leage. Mat
information
Please chsance
(2) Dh-2:
lesger
642 tiad
pec not PCT) collinerr
Predicion::
(1)
Prodicaoo::
spaiizss Ladt "ee are co and cation the
Tree trapectten the dragranc ralin
66
feteral
the doe that saccles the rigurs.
Herce the porness cbotce ta
(a) N.P.
(b) S.P.
Decrigtio::
Deuription:: loxed coctoad bay of the to aillling prantess are micessary reach
Based esta contrrctioo glean taa to lowing cone pasent frem seom peints C & C. Pite unteet
Abide mecassrr reach
FP
Prodection
The canc lastce have rocs is
Prodection:
C2 and (3 vieee fects (teats Sotttug It 11 are
20A0 2y/l1
La3et roedid hive cinerate
free thece fets testter lowe mistade--taeee ettairs a C. preat redaxder.
Thes perpandica artiy Inospendent Informacton: zEzOS! Erem condtrent lncalf and crene
(c) I.P.F.
(d) S.D.
Devrijatoo
[ood Sahio cast
are Theear. aroblent
was solving muchemst coal problan but made soliving
praal [..]]
(1) Gontral sngle iesorited seglo [-c]
(0 tisselles entangle mgal uneless Gorl
are
ar collinee and Iessoele
on do
Protiction:
Zacncales crtangle alone
4000.00
the anef that the theee peliess b. and
coed cearted aroag? (2) Steedan
the esaulite the base angtes the socceles weweeeeeteatnneeeeee
relatioes
tusic prasetiinn new cantee from po stall
Prodiclon:
The ou ascertien is Steadon Seeecce
de ctop of the rceaf readiees the spccill
secuase Issicelo
parallel Tires Incarancs they corectos" (option J.. (izoeett))
Strpess aleeady (Mloeeed))
(e) T.S.
( E.B.L.
Figure 4: Failure (some question descriptions and prediction content omitted for space limit).
C.2 EVALUATION PROMPT
In this section, we will provide detailed prompt templates along with examples. Fig. 5 illustrates the basic structure of the prompt template, which consists of the following components: 1) Problem NL; 2) Question; 3) Multiple-choice task description; 4) Options; and 5) Output instructions.
Here, Problem refers to the natural language description of the problem. The output instructions are further divided into two variants: with and without Chain-of-Thooee (CoT), Fig. 5a demonstrates prompt without CoT, while Fig. presents the prompt incorporating COT.
C.3 MODEL NAME
In this section, provide list of model names corresponding selected MLLMS that employ API calls for inference, as shown in Table 9,
C.4 ANSWER EXTRACTION
To compute the multiple-choice question accuracy(ace) of MLLMs, it is necessary to extract the model's predicted answers from its reasoning outputs for comparison with the ground truth.
16

---

Published as a conference paper at ICLR 2026
<Problen NL> <Question>
<Problen NL> <Qesstion>
"The following will present four nunerically ly labeTed options, including one correct option incorrect options.
"The following will present four culerically 1abeled options, including one correct option three incorrect options, Please verify the contents of options (1) to (4) in sequence to determine whether they requi rements, 1,e,, whether they are correct. Finally, output the serial number of the option.
verify the contents of options (2) to (4) in sequence determine whether they requitenents, i.e., whether they are correct. Finally, output the serial number of the correct option.
the problen's 's
the problen's
The options follows:
The options follows:
2
(4)
when outputting, only provide serial without the text following it. Represent the parentheses digit, e.g.,
Please think step by step, output reasoning process. and select the single correct answer. Represent the
serial number (1), (3), etc.
number parentheses (1-4) and enclose it in (boxed(]. Note that the correct answer must be single choice between answers,
note, only output answer, without thought process or additional information."
output and do not output 'None' as an answer. For exanple, ... (b)xed ... ...)bxxe(((4),, etc."
w/o CoT
(b) w/ CoT
Figure 5: Prompts Format
Table Model Names for MILMs employing API Calls
Model
Model Name
GPT-4o
gpt-4o gpt-4- 106-preview o1 01-2024-12-11
GPT-4V
OpenAl-ol
OpenAl-o3
03-2025-04-16
Claude-3.7
claude-377-oomee-22255000
Gemini-2.5-pp
gemini-2.55pro-ce
DeepSeek-RR R1
deepseek-ri
For General MILMS not employing the CoT approach, we instruct them to output only the answer choice following the prompt template in Sec. C.2. Empirical observations confirm that these models consistently comply by generating only a single option identifier. We therefore directly compare their outputs with the ground-truth labels for accuracy evaluation.
For Reasoning MILMs or General MLLMs utilizing the CoT method, the model outputs a complete reasoning process before generating the final answer. Therefore, first employ the large language model to extract the answer from the reasoning output, then match substrings in the form of (1), (2), etc., and compare them with the ground-truth labels to compute accuracy.
17

---

Published as a conference paper at ICLR 2026
Please observe this geometry problen em image. what numerical information can you obtain from the image?
D
(I) AB=1;
(2) AB=6;
(3) AB-2;
(4) BM=1
(a) Numerical Perception
Please observe this geometry probl em image. What structural information can obtain from the image?
17
C, are collinear;
(2) A, B, are not collinear;
B
(3) D, E, are collinear;
(4) A, B, are collinear
17
(b) Structural Perception
Consider the following geometric problem: geometric construction given as follows:
nothing, adds point with randon coordinates. From point A, builds B with an integer distance 13 From B. From C. D. the other two vertices of square with side AB. From non- co collinear points through D, A, B. From four points C, B, D, such that C teit nor A, and such that A and are distinct, builds F, G, sinul sanultaneously tangent to both the circle of center C through and the circle of center through D. and at through B, and G and I are the tangent points on the circle centered at A through D. From two points G, with different from G, builds J, K, the other two vertices DG. Based on this construction, given the following premises: AB DKIGJ; AD
points A, with different from B, builds
A, B, creates E the center of the circle
H.
of lines
the tangent points the circle centered
square with side
CF FG; OK ]K;
BC; G) JK; DG G];
AI 1 HI; CH HI; DC G); AD CD; AG FG; BC CH; DGUK; D1 ABICD; AE DE; BC CF; BC CD;
AG;
]K; AD OD;
DJ GK; AC a BD; AE BE: ADIBC; AC BD; AB 13/1. Based
these premise, derive the conclusion: Angle IBI equals angle JCI. which the following prenises conclusion?
necessary reach
(1) BC CD;
(2) AB CD;
(3) DK T JK;
(4) AI HI
(c) Irrelevant Premise Filtering
Figure 6: Visualization Examples
18

---

Published as a conference paper at ICLR 2026
Consider the following geametric problem: A geonetric construction is given as follows: From nothing, adds point with random coordinates. From point builds wi an integer distance 6 from A to B. From two points
E
with A different from builds C, D, the other two vertices of a square with side AB. From two distinct points C, builds E, another point the line CA. From three non-collined G,
points 8, C, D adds F, and BC,
the midpoints of sides CD, respectively. It also adds I, the center of the circle
going through G, and H. which is also the nine points circle of the triangle BCD. Based on this constructions to arrive the conclusion: Angle FDF equals angle IDI. which of the following premises are necessary to reach this conclusion?
(1) ADF = AGI ADF = A FGA
(2) ADF
FGA ADD
DGF AACF
DFG
(3) ADF ADGI
(4) ACF & ADF
ACF =
A DCI
ADF = A PGI ;
A CDT = AGGI
(a) Sub-Goal Decomposition
Consider the following geometric problen: Fron nothing, adds point with random coordinates. Fron point A, builds with an integer distance 3 from to B. Fron three non-collinear points B, and D, adds E, the other intersection of the circle of center A through and the circle of center through points E, A, В, creates the center of the circle through E, A, B, Solve that: Angle CAD equals angle EAF which rule was not used to solve this problen?
From non-col Tinear
(1) Are determines inseribed angles (tangent): This rule corresponds to r03 in the case the arc is detemsined by a tangent line, An inscribed angle determining that same are will he leg being
congruent the angle deternining that arc,
vertex direction of the bisector theores: a line through a vertex of a triangle cuts the opposite side into two segnents that are in proportion as the neighboring sides of the triangle, the Tine bisects the angle the vertex it cuts. Sisector is perpandi cular: This rule the locus
tangent line
(2) Risector theoren I:
rection
of r22. says to the two vertices perpendicular to AB
points that are equidistan segnent AB 1s straight line
(4) Isosceles triangle equal angles: The thearen says that the base angles of 1sosceles trfangle congruent.
(b) Theorem Selection
was solving rathesat problem len but made wistake during process, solving it.
B
which caused me to integer distance from distance 3 fron to Fron three different points B, o, buildd X and Y. the points of tangency of the two lines through tangent to the eirele of center O through B. Fron three non-collinear points X, Y, A, adds C, D, and F. the nidpoints sides center the circle goting through
pros Fron builds B. Fron point G. buillds ldd A with integer
XA, and W. respectivel tivyy.. also adds the none
and E, which
triangle circle through
but lds point construct Douutiion Fon, solve that: Fros seven points C, X, F, X, D, Y. builds V. point such that CV/FX = DY/PY. These are my reasoning steps: 80 ox 80 2/1 (latlo (Pythagoras Veri verification)es Pythagorean Theoren's presises X. A, D are sactstied 002. Pythagorcan Theores's a preetses on Pythagoras Chast to or Veri Fication) Pythagoraan Theorwn's presises on V. Pychaporean Theoren's presises on Y. A. D are satisfied cr57 Pythogoras theoren)ms en) AY 38/17 306. AX 38/17 AY 35/17 (Ratio Chas ing) Ax aY 007. (r23 Bisector is perpendiculane as in reasoning to wrong?
and
ing OX 2/1 001. A0 3/1 , OX 2/1 AX ox
0 are
2/2 AY oy (Pythaperas
satisfied oos.
ux 90
(Ratio Chasing uX or 038. AX AY
seens I've reached dead (e-g., could you tell which started
by
represents
(1) Stop 5:
(2) Step
(4) Stop
(c) Faulty Branch Localization
Figure 7: Visualization Examples
19

---

Published as a conference paper at ICLR 2026
Table 10: Geometric Rules (Part 1)
Rule Name
Rule Statement
Definition of circle
Four points A, B, C, D equidistant from a center O all lie on a same circle. (One side of the definition of circle.)
Are determines internal angles
Two angles with the vertices circle are congruent.
circle that determine the same AB on that
Congruent angles are in circle
P. Q are vertices congruent angles, and A and are the intersections of legs of the angles with vertices P and Q. there is a circle through A, B, P. and Q. Ares of the same length determine chords of the same length on the same circle.
Same are same chord Base of half triangle
The linc connecting the midpoints of two sides of triangle is parallel to the third side of the same triangle.
Thales Theorem
Two parallel lines AB and cut by two intersecting transverse lines AC and BD, will determine collection of proportional segments, The original statement of this rule did not have the non-degeneracy condition ncoll O A B hypothesis.
Bisector theorem
If line through vertex of triangle cuts the opposite side into segments that are proportion the neighboring sides the triangle, line bisects the angle the vertex cuts.
Bisector theorem II
The internal bisector of a vertex of a triangle divides the opposite side into two segments that are in proportion to the neighboring sides of the triangle.
Isosceles triangle equal angles
The base angles of an isosceles triangle congruent.
Equal base angles imply isosceles
the base angles a triangle are congruent, the triangle is isosceles.
Arc determines inscribed angles (tangent)
An inscribed angle determining that same are will be congruent to the angle determining that are with leg being the tangent line at the vertex of the
Same are giving tangent
two angles with vertices on a circle see the same arc, one vertex is also extremal point
arc, leg of angle through extremal point the tangent to the circle at that same point.
Central angle vs inscribed angle I
The central angle doubles the inscribed angle when both determine the same are in circle. It mentions bisects the chord hypotheis instead of halves the angle because midpoint of segment
predicate, while bisector of an angle not.
Central angle vs inscribed angle IT
central angle has the same measure as a given inscribed angle on circle, will cut the chord corresponding to that angle in half.
Hypotenuse is diameter
The hypothenuse of right triangle is diameter of its circumeircle, or that the midpoint of the hypothenuse is the circumcenter of the right triangle.
Diameter is hypotenuse
two points are the edges of diameter a circle. and at same time are vertices of an inscribed triangle, the triangle has right angle at third vertex.
Cyclic trapezoid
cyclic trapezoid is isosceles (refering specifically to the congruence of the angles base).
Bisector Construction
The perpendicular line through the midpoint of the segment is the perpendicular bisector the segment (the locus equidistant points to the vertices of the segment).
Bisector is perpendicular
The locus of the points that are equidistant to the two vertices of a segment AB straight line perpendicular to AB.
Cyclic kite
cyclic kite is always formed by two symmetric right triangles.
20

---

Published as a conference paper at ICLR 2026
Table 11: Geometric Rules (Part 2)
Rule Name
Rule Statement
Diagonals of parallelogram
If two segments intersect at their common midpoint, the vertices of the segments are the vertices of a parallelogram.
I
Diagonals of parallelogram
The diagonals of parallelogram meet at their common midpoint.
Thales theorem II
two points and D split legs of a triangle on the same ratio, line CD will be parallel to the base of the triangle.
Overlapping parallels
Two intersecting parallel lines are actually the same.
Midpoint is an eqratio AA Similarity of triangles (Direct)
Midpoints split segments in the same ratio (1:2).
similarity condition for pair of triangles: that of two pairs of congruent angles between the triangles (angle-angle similarity).
AA Similarity of triangles (Reverse)
A similarity condition for pair of triangles: that of two pairs of congruent angles between the triangles (angle-angl) angll similarity).
Thales thcorem IV
If three parallel lines are cut by two other lines, there is corresponding pair of proportional segments determined by the intersection points.
Recognize center cyclic (circle)
three points lie on a circle with a known center, and there a fourth point on that circle, the distance of the center of the circle to this fourth point is the same as to other points in circle.
Midpoint splits in two
This rule converts symbolic statement (M is the midpoint of AB) into an algebraic one (the ratio between AM and AB is 1:2).
Properties of similar triangles (Direct)
This rule goes from the pure statement that two triangles are similar to spilling out the correspouding statements about the proportion of the lengths of the sides and the equivalence of angles on both triangles.
Properties of similar triangles (Reverse)
This rule goes from the pure statement that two triangles similar to spilling out the correspouding statements about the proportion of the lengths of the sides and the equivalence of angles on both triangles.
Definition of midpoint
This rule was created to detect midpoints by their defining axiomatic properties. It solidifies midp a predicate.
Properties of midpoint (cong)
This rule extracts from the midp predicate the property that the midpoint is equidistant from the extremes of segment.
Propertics of midpoint (coll) Pythagoras theorem
This rule extracts symbolically from the midp predicate the property that the midpoint on the line of the segment.
If the proof state symbolically knows the three lengths of the sides of right triangle ABC, and they satisfy that the sum of the squares of the lengths of the legs is equal the square of the hypothenuse, it will add proper orthogonal relation the proof state.
Same chord same are
This rule gives conditions for inscribed angles on a circle defining chords of the same length to have the same measure.
Same chord same II
This rule gives conditions for inscribed angles on circle defining chords of the same length to have the same measure.
SSS Similarity (Direct)
Proportional sides with same orientation imply similarity.
SSS Similarity (Reverse)
Proportional sides with opposite orientation imply similarity.
SAS Similarity (Direct)
Proportional sides with congruent included angle (same orientation) imply similarity.
SAS Similarity (Reverse)
Proportional sides with congruent included angle (opposite orientation) imply similarity.
Ratio Chasing
Manipulating and analyzing ratios hetween different quantitics in a geometric setup to derive new relationships prove geometric properties.
Numerical Check
Verifying geometric properties by numerical means, typically checking if numerical values geometric entities satisfy expected relationships.
Angle Chasing
To pursue and derive angle relationships in geometric configurations, often using angle sum properties, parallel line angles, or circle angle properties.
Pythagoras Verification
The Pythagorean relationship (a2 + ba = c²) in right triangles, either confirming triangle is right-angled using known right angle derive side lengths.
21

---

Published as a conference paper at ICLR 2026
Table 12: Confidence-Imterraal(444
Level 1
Level
Level
Level
N.P.
S.P.
L.P.F.
S.D.
T.S.
F.B.I.
Owen2-VL-72 (Wang al., 2024)
55.1624.26
19.51+2.13
16.17=1.90
36.84+2.59
25.25+3.39
19.9624.88
Qwen2-Vi 72 (Wang et al., 2024)
84.33+4.28
28.1543.68
30.17+1.90
60.34a4.37
35.37+4.01
21.5444.26
Qwen2.5-VL-7b (Bai al., 2025)
86.9023.93
20.00+3.18
30.8442.59
40.6742.59
35.7642.90
24.49+2.93
Qwen2.5-VI-72th (Bai et al., 2025)
85.72+4.44
39.51=3.83
39.00s1.22 .24
76.00+2,48
45.8243.73
25.85+2.98
GPT-4o (OpenAl, 2024b)
68.6544.26
22.71=1.06
59.50=4.97
43.00+2.48
37.27=4.73
23.3640.98
Table 13: Soafddenc-Scorres
Level
Level
Level
Level 4
N.P.
S.P.
LPE.
S.D.
T.S.
E.I.L
Dwen2-VL-78 (Wing et al., 2024)
0.97
0.96
0.97
0.97
0.95
0.90
Qwen2-VL. -VL (Wang al., 2024)
0.97
0.97
0.92
Qwen2.5-VLa VL-7b (Bai ul., 2025)
0.98
0.94
0.97
0.97
0.97
0,95
Qwen2.5-WL-72b (Bai al., 2025)
0.98
0.96
0.99
0.99
0.95
GPT-4o (OpenAI, 2024b)
0.97
0.98
0.97
0.98
0.95
0.98
D DETAILS OF GEOBENCH EVALUATION SET
D.I VISUALIZATION EXAMPLE DIFFERENT TASK
In this section, as illustrated in Fig. 6 and Fig. 7, we provide comprehensive description of the four levels and six exemplary instances geometric problems, along with sample option descriptions, in greater detail.
D.2 RULES USED
This section presents the complete set of geometric rules utilized by TrustGeoGen to construct reasoning graphs and those applied in GeoBench Task 5 (Theorem Selection). All rules systemati- cally categorized and listed in Table 10 and Table 11. These rules define the foundational logic for geometric reasoning within benchmark framework.
E CONFIDENCE ASSESSMENT
We have conducted additional replicate experiments and incorporated metries such confidence intervals and confidence scores, with the results presented in the Table 12 and 13. Among these, the Confidence-Score serves numerical indicator of data variability (with higher values corresponding to lower variability). As can be observed from the table, the testing results generally exhibit low variability, indicating high level of reliability.
22


Reviews:
[{"review_id": "td9ckJaAgN", "reviewer": "Reviewer_FDHN", "summary": "This work presents a benchmark to evaluate multimodal geometric reasoning by VLMs/MLLMs. The hierarchical evaluation proposed offers a novel assessment framework that goes beyond answer accuracy metrics. Additionally, this helps identify the exact steps at which models fail in the solving process, providing actionable insights into improving model performance. Finally, this work identifies the failure of the common CoT approach when it comes to geometric reasoning.", "questions": "Questions:\n1. How does this benchmark ensure diversity in the questions generated?\n2. Can this generation and evaluation framework be extended to geometric problems in the 3D space?\n\nSuggestions:\n1. This work would benefit from further proofreading. Some errors that could be fixed:\n- 1.1 L161: TrustGenGen -> TrustGeoGen\n- 1.2 L250: Capitalizing the first letter of the section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The hierarchical framework is a useful diagnostic tool to provide actionable insights into pitfalls in the reasoning process.\n2. The benchmark presented in this work goes beyond data collation and adds a unique set of information for analysing model performance.", "weaknesses": "1, This work does not detail how the automatically generated tasks are verified for accuracy and legitimacy\n2. Likewise, there is no insight into how the automatically-generated problems are distributed in terms of logical and reasoning complexity. In addition to the empirical comparison against established benchmarks and their levels, the work could benefit from a deeper, and qualitative, analysis of the complexity and difficulty of the problems in this benchmark,", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sIDrSxm9aV", "reviewer": "Reviewer_aj3h", "summary": "Addressing three major limitations in current geometric reasoning evaluation—the risk of test data contamination, overemphasis on final answers, and insufficient diagnostic granularity—this paper proposes a more scientific and refined hierarchical benchmark called GeoBench. Through systematic evaluation, this benchmark not only reveals the capability limitations of existing AI models in handling complex geometric problems but, more importantly, precisely diagnoses the core factors affecting model performance.", "questions": "N/A", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. It identifies key limitations in geometric reasoning evaluation (e.g., data contamination, overemphasis on answers) and addresses them through GeoBench—a hierarchical benchmark that decomposes geometric reasoning into distinct stages\n\n2. The benchmark leverages the TrustGeoGen methodology to generate tasks verified for logical rigor, ensuring data novelty and mitigating contamination risks. This establishes a reliable foundation for equitable model evaluation.", "weaknesses": "--The evaluation framework lacks a necessary human verification step. Given the complexity of the dataset problems (as shown in Figure 4), establishing a performance baseline from human experts is crucial. Furthermore, the scope of evaluation should be expanded to include advanced mathematical reasoning agents—particularly those capable of using tools for exploration or constructing auxiliary lines—in order to assess the true capabilities of current models under problem-solving paradigms that closely resemble human approaches.\n\n--Regarding the reasoning graphs relied upon in the synthetic data generation process, key details—such as the quality of these graphs and the fidelity with which models adhere to them—are not sufficiently elaborated. This raises concerns about potential risks to synthetic data quality: if models do not strictly follow the intended reasoning logic, or if problems are constructed based on flawed reasoning graphs, the validity and reliability of the resulting problems may be called into question.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "z153xXwi7i", "reviewer": "Reviewer_sFVL", "summary": "The paper proposes GeoBench, a hierarchical benchmark to evaluate geometric reasoning in multimodal large language models. It introduces four levels: visual perception, goal-oriented planning, rigorous theorem application, and self-reflective backtracking which is implemented through six tasks derived from formally verified problems generated using TrustGeoGen. Across 1,021 examples, the authors show that reasoning-oriented models like OpenAI-o3 outperform general MLLMs, though performance decreases sharply with task complexity. The analysis highlights that sub-goal decomposition and irrelevant premise filtering are key determinants of geometric problem-solving accuracy and that Chain-of-Thought prompting is not universally beneficial.", "questions": "I do not have any particular concerns with the paper, except maybe including some more MLLMs such as Molmo, Math-LLaVA and MathPuma. The authors can show some preliminary results on these models", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "I liked the clarity in the paper's writing and the results are comprehensive, and have two strengths to highlight:\n\n- Hierarchical evaluation grounded in cognitive theory: The benchmark’s structure, inspired by the van Hiele model, allows precise diagnosis of reasoning abilities rather than measuring final-answer accuracy alone.\n\n- Comprehensive and formally verified dataset: Using TrustGeoGen ensures rigorous, contamination-free problem generation, making GeoBench a strong diagnostic tool for evaluating how MLLMs reason through geometric logic.", "weaknesses": "The benchmark relies on synthetic, clean diagrams and controlled premises. This limits assessment of robustness to real-world variability such as hand-drawn figures, scanned textbook noise, ambiguous markings, and imperfect annotations. Adding a real-diagram slice or perturbation suite would strengthen ecological validity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CRvl99WDJV", "reviewer": "Reviewer_YWp2", "summary": "The paper introduces GeoBench, a hierarchical benchmark for multimodal geometric problem solving. It defines four reasoning levels: Visual Perception, Goal-Oriented Planning, Rigorous Theorem Application, and Self-Reflective Backtracking, instantiated via six multiple-choice tasks built from problems generated and formally verified by TrustGeoGen. The dataset has 1021 items and is used to evaluate a range of general and reasoning MLLMs. Key empirical findings: (i) performance drops as task difficulty rises; (ii) sub-goal decomposition and irrelevant-premise filtering correlate most with final-answer accuracy; (iii) Chain-of-Thought can hurt faulty-branch localization; (iv) some OOD generalization is observed to GeoQA/Geometry3K.", "questions": "* What is difference between GeoBench and GeoBench-Solving, is the later a subset of the former one? \n\n* The results in Table 4 and Table 5 have a lot overlap, which may not necessary.\n\n* A proper reference for TrustGeoGen at line 155 is missing. \n\n* Important evaluation details such as temperature, max tokens, retry logic, and related decoding parameters are not fully reported.\n\n* In Section 3.2, how the distractors are designed? \n\n* In Table 4, the DeepSeek-R1 results is obtained via image textual descriptions. This may cause unfair comparison as models may benefit from accurate image textual descriptions, especially for geometric problem solving. How the other models perform when the image is replaced by the textual descriptions?\n\n* Some results in Table 4 seems unintuitive. For example, Qwen2.5-VL-7b performs pretty well on N.P. tasks but suddenly very poor on S.P. tasks, which all belong to Level 1 tasks. \n\n*  In Section 4.3, how the feature vectors are generated? I am a bit confused by this section. \n\n*  In Section 4.4, why the other two benchmarks are OOD? What is the point of studying other two benchmarks in your paper? \n\n*  In Section 4.5, it would help to include side-by-side qualitative examples with vs. without CoT, highlighting where CoT helps or hurts. Annotated failure cases would strengthen the argument.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- Hierarchical evaluation design. The four-level tasks with difficulty level increase is principled, and the task design reflects real-life use cases. \n\n- Using TrustGeoGen with verified reasoning graphs reduces label noise.\n\n- Comprehensive evaluations cover many popular general and reasoning MLLMs and CoT ablation highlighting task-specific limits.", "weaknesses": "* While TrustGeoGen with formal verification is a strength, the heavy reliance on synthetic problems may narrow the distribution. It remains unclear how faithfully the tasks reflect real contest or classroom diagrams.\n\n* Results are reported mostly as point accuracies; significance tests, confidence intervals, or bootstrap estimates are missing.\n\n* A random-choice baseline is trivial; a properly calibrated human baseline (e.g., novice vs. expert) is needed to contextualize difficulty and headroom.\n\n* Inaccurate or overstated claims in results discussion.\n\n  * The statement around lines 346–348 (e.g., *“... T.S. task where other models generally perform poorly.”*) is not accurate, as **o1** attains similar performance.\n  * The claim *“Reasoning MLLMs consistently outperform General MLLMs … confirming their superior reasoning capabilities …”* overgeneralizes; several non-reasoning models outperform some reasoning models on certain tasks. These statements should be revised to reflect the actual per-task and per-model variability.\n\n* Some benchmark details require further clarification (see the Questions section).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)