MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with Holistic Platform and Adaptive Hybrid Policy Optimization
Authors: Xiangyu Zhao
Junming Lin
Tianhao Liang
Yifan Zhou
Wenhao Chai
Yuzhe Gu
Weiyun Wang
Kai Chen
Gen Luo
Junchi Yan
Wenwei Zhang
Hua Yang
Haodong Duan
Xue Yang
Venue: ICLR 2026 Poster (2026)
Keywords: MLLM
Reasoning
Primary area: foundation or frontier models, including LLMs

Abstract: While current Multimodal Large Language Models (MLLMs) have demonstrated proficiency in reasoning tasks such as mathematics and logic,  their capacity for long-chain reflective reasoning, a prerequisite for solving complex real-world problems, remains largely underexplored.  In this work, we first conduct an extensive empirical investigation to evaluate this capability. Leveraging a carefully designed data synthesis engine, we construct MM-HELIX, a multimodal benchmark consisting 1,260 samples of 42 challenging synthetic tasks that require iterative thinking and backtracking.  Empirical results on this benchmark reveal that existing MLLMs exhibit significant performance deficits in long-chain reflective reasoning.  To address this limitation, we generate post-training data and further explore learning paradigms for exploiting such data.  We first develop the Step-Elicited Response Generation pipeline to create MM-HELIX-100K, a large-scale dataset of 100k high-quality, reflective reasoning traces for instruction-tuning stage. Given that standard Reinforcement Learning fails on complex tasks due to sparse reward signals and catastrophic forgetting after Supervised Fine-Tuning,  we propose Adaptive Hybrid Policy Optimization (AHPO), a novel training strategy that dynamically unifies offline supervision and online optimization into a single stage. This strategy enables the model to learn from expert data when rewards are sparse and conduct independent exploration once proficient.  When applied to the Qwen2.5-VL-7B baseline, our method achieves a +18.6\% accuracy improvement on MM-HELIX benchmark and demonstrates strong generalization with a +5.7\% average performance gain on general mathematic and logic tasks. Our work demonstrate that reflective reasoning in MLLMs can be effectively learned and generalized, paving the way for developing more capable MLLMs.

Published as a conference paper at ICLR 2026
MM-HELIX: BOOSTING MULTIMODAL LONG-CHAIN REFLECTIVE REASONING WITH HOLISTIC PLATFORM AND ADAPTIVE HYBRID POLICY OPTIMIZATION
Xiangyu Zhao Junming Lin Tianhao Liang" Yifan Zhou-1.2, Wenhao Chai²,
Yuzhe Gu², Weiyun Wang", Kai Chen?, Gen Luo², Wenwei Zhang?, Junchi Yan',
Hua Yang', Haodong Duan 203 Xue Yangl
'CISEE & SAIS & SCS & SAI, Shanghai Jiao Tong University, Shangha AI Laboratory,
Beijing University of Posts and Telecommunications, *Zhejiang University, PPrinceton University
"Equal contribution
*corresponding author
https: htmm//mmhellle wew.eixx.o ix. wiightthhoooooo.
Multi-Modal Reflective Tasks
Data Generation
Reflective Reasoning
Algorithms
Graphs
Generator
Veriler
Verification
MM-HELIX
Lat's borkuat coll JUE, with shows
Fliddx, PtPoicss, CrytteMath
Hemiltnnian Cycley Fath, Max Flowe Shortest Dist,
Hypothesis Larts suppow (6.1 the mine
Slip-elicited Response
Rule-Based CoT
Owen--3-2358
Hypothesis Testing
Abl So immpsssiblle
Puzzles
Ealero, Uitori, Kukunss, Futeshaks,
Games
Minnoweger,
Sieg Nodle C. can reach...
Step Initialize now
But wall is there any way Nu betber
by
think through step
the enton
Therefons, (3.1) canpot be
Nibbles, Süiding, Haroo,
Stop I'ound a path...
Backtracking
(5.3) mine,
Adaptive Hybrid Policy Optimization
Off-Policy Expert Trace
MM-HELIX Bench
Lowsaspter
Generalization
Qwe--225-V1-76
Dynamic Activation
To solve Minesweeper
wi rand to
Second Row; The lina cell
Third Row: The first cell
Top Rew: TTee first cells
Question
Questton
On-Pulicy
Response
Los
So, ancweer
Group
NO REFLECTION
Computatun
Figure I: Overview of proposed framework. Our framework comprises two core components: (1) MM-HELIX benchmark to evaluate the reflective capabilities of MLLM, and (2) AHPO method to boost reflection capability and transfer enhanced skills to general reasoning tasks.
ABSTRACT
While current Multimodal Large Language Models (MLLMs) have demonstrated proficiency in reasoning tasks such as mathematics and logic, their capacity for long-chain reflective reasoning, prerequisite for solving complex real-world problems, remains largely underexplored. In this work, we first conduct an ex- tensive empirical investigation to evaluate this capability. Leveraging carefully designed data synthesis engine, we construct MM-HELIX, a multimodal bench- mark consisting 1,260 samples of 42 challenging synthetic tasks that require iterative thinking and backtracking. Empirical results on this benchmark reveal that existing MLLMs exhibit significant performance deficits in long-chain flective reasoning. To address this limitation, we generate post-training data and further explore learning paradigms for exploiting such data. We first develop the Step-Elicited Response Generation pipeline to create MM-HELIX- -I0LXX-100K, a large- scale dataset of 100k high-quality, reflective reasoning traces for instruction stage. Given that standard Reinforcement Learning fails on complex tasks due to sparse reward signals and catastrophic forgetting after Supervised Fine-Tuning, we propose Adaptive Hybrid Policy Optimization (AHPO), novel training strategy that dynamically unifies offline supervision and online optimization into a single stage. This strategy enables the model to learn from expert data when rewards are sparse and conduct independent exploration once proficient. When applied to the VVV--2.5-VI--B baseline, our method achieves + 18.6% accuracy improve-

---

Published as a conference paper at ICLR 2026
ment on MM-HELIX benchmark and demonstrates strong generalization with +5.7% average performance gain on general mathematic and logic tasks. Our work demonstrate that reflective reasoning in MLLMs can be effectively learned and generalized, paving the way for developing more capable MLLMss
INTRODUCTION
Human cognition fundamentally characterized by the processes of reflection and backtracking. This iterative cycle of trial, error, and correction allows individuals to adapt to novel environments and progressively refine their decisions for greater accuracy. Inspired by this cognitive process, recent advancements in Large Language Models (LLMs) have integrated reflective and multi-step thinking strategies (Guo et al., 2025a), unlocking significant improvements in their reasoning abilities. Concurrently, Multimodal Large Language Models (MLLMs) have undergone rapid development, achieving impressive performance across spectrum of downstream tasks, from perception (e.g., recognition) to reasoning (e.g., mathematic). Despite these advances, a significant limitation persists in current MLLMs. The majority of these models are designed to generate outputs in a single, direct pass, lacking the intrinsic mechanisms for self-correction and iterative refinement. Consequently, their capacity for end-to-end multi-step reflective reasoning within rich multimodal contexts remains largely unexplored and underevaluated.
Existing research, e.g. Enigmata (Chen et al., 2025), VGRP-Bench (Ren et al., 2025), and Code2Logic (Tong et al., 2025), have primarily concentrated on text-only problems or puzzle-like like challenges that are often constrained to multiple-choice or fill-i-thheblank formats, thereby failing to adequately evaluate the end-to-end reflective reasoning capabilities of MLLMs. To address this critical research gap, we introduce MM-HELIX, comprehensive benchmark designed to evaluate the long-chain iterative reasoning capabilities of MLLMs. MM-HELIX contains 42 meticulously curated challeng- ing tasks from diverse online sources, categorized into four domains: Algorithum, Graph, Puzzle, and Game. Each task requires the model to perform careful visual observation, develop a deep understanding of complex rules, and generate extended chain-of-thougg that necessitates reflec- tion and backtracking. We constructed a versatile procedural generation pipeline to systematically generate samples. This pipeline features: (1) a rule-based code Generator programmatically creates multimodal questions with tunable parameters, spanning five hierarchical difficulty levels, which indicates very easy, easy, middle, hard, very hard. (2) a Solver module engineered to produce ground-truth solutions; and (3) a Verifier module to algorithmically validate answers for tasks with non-unique solutions. This Verifier also functions as a reward oracle our reinforcement learning environment. Following rigorous filtering process, our benchmark consists of 1,260 high-quality samples. Our comprehensive evaluation reveals that state-of-the-an MLLMs struggle significantly on MM-HELIX. For instance, even leading model like Qwen-2.5-VL-72B achieves mere 13.9% accuracy, underscoring a profound deficit in their reflective reasoning capabilities.
Based on the observation, wonder if we can boost the reflection of MLLMs within MM-HELIX and generalize to general reasoning tasks like mathematics and logic. We then propose the Step- Elicited Response Generation (SERG) pipeline, method for efficiently generating high-quality, reflective CoT traces by integrating rule- based, key-step knowledge. Leveraging SERG, we construct MM-HELIX- 100K, a large-scale dataset comprising 100k high-quality samples that span 42 tasks across a full spectrum of difficulty levels.
Our initial experiments reveal the limitations of standard training paradigms: instruction-tuning on MM-HELIX- 100K caused catastrophic forgetting, while on-policy reinforcement learning failed due to extreme reward sparsity since base model lacks the foundational ability to solve the tasks. To this end, we introduce Adaptive Hybrid Policy Optimization (AHPO), framework that dynamically integrates off- off-policy expert guidance with on-policy exploration. AHPO implements an explore- with-supervision strategy by dynamically modulating the off-policy loss via reward-based based gating mechanism. Specifically, when the rewards within a group are sparse, indicating the model is struggling, the off-policy expert data is integrated to guide the model toward correct trajectories. Conversely, once the model demonstrates proficiency and rewards become dense, the off off-policy loss is attenuated, encouraging the policy to explore and discover novel solutions. Training with AHPO on a combination of MM-HELIX 100K and a general mathematics RL dataset yielded substantial gains. The model not only demonstrated mastery on in-domain tasks, achieving + 18.6% accuracy
2

---

Published as a conference paper at ICLR 2026
improvement MM-HELIX, but also successfully generalized its enhanced reflective skills to general math and logic tasks, with a +5.7% average increase in accuracy. These results validate that our approach effectively cultivates reflective capabilities that are both robust and transferable.
Our contributions can be summarized as follows:
1. We introduce MM-HELIX. benchmark comprising 42 challenging multimodal tasks specifically designed to assess long-chain iterative, and reflective reasoning. Through systematically evaluation, reveal the critical deficiencies of state-of-the-an MLLMs in these complex reasoning domains.
We propose the ttep-Elicitet Response Generation (SERG) pipeline, a novel and efficient method for generating high-quality demonstration data. We leverage SERG to construct a large-scale dataset of 100k multimodal high-quality reflective Chain-of- Thought (CoT) traces.
3. We introduce Adaptive Hybrid Policy Optimization (AHPO), novel training algorithm that dynamically integrates off-policy expert data with on-policy exploration in single stage. This hybrid approach is specifically designed to overcome the challenges of sparse rewards, fostering the acquisition of complex reasoning skills that demonstrate substantial generalization.
RELATED WORK
Multimodal Large Language Models. In recent years, MLLMs have rapidly advanced in both general multimodal capabilities and specialized reasoning. Representative models such as Gemini 2.5 (Comanici et al., 2025), Qwen-2.5-VD. (Bai et al., 2025b), and InternVL3 (Wang et al., 2025) establish strong general multimodal capabilities. More recently, reasoning-oriented models such as GLM--45V- Thinking (Team et al., 2025b), Seed1.5-Vi (Seed et al., 2025), and
-T--AiB-hiinking (Team al., 2025a) explicitly emphasize structured thinking. Together, these works indicate a growing consensus that reasoning is the next frontier for MLLMs.
Exploration of Long-chain Reasoning, Chain-of-Thought prompting (CoT) (Wei et al., 2022) and Tree-of-Thoughe (ToT) (Yao et al., 2023) demonstrate the value of intermediate reasoning traces. Procedural generation has emerged as solution: Enigmata (Chen et al., 2025) creates logic puzzles, Code2Logic (Tong et al., 2025) synthesizes multimodal QA from game logic, and benchmarks such as VGRP-Bench (Ren et al., 2025) reveal persistent weaknesses algorithmic reasoning. However, these works have primarily concentrated on text-only problems or puzzle-like challenges that are often constrained to multiple-choice or all-in-the-blane formats.
Reinforcement Learning Method. On-policy algorithms, such PPO (Schulman et al., 2017), stabilize training by clipping updates, but this computationally expensive. Variants such as GRPO (Shao et al., 2024) improve stability through within group advantage, while DAPO (Yu et al., 2025a) dynamically adjusts policy optimization, and GSPO (Zheng et al., 2025) emphasizes gradient scaling for more efficient updates. Off Off-policy methods reduce training costs by reusing data. Besides, LUFFY (Yan et al., 2025) applies sequence-level optimization to exploit offline preference datasets. Those RI. methods all meet problems of inefficient training when facing hard tasks, thus, we propose AHPO to simplify training and enhance multimodal reasoning.
METHOD
3.1 MM-HELIX: BENCHMARKING MULTIMODAL REFLECTIVE END-TO-END REASONING
Recent advancements in Multimodal Large Language Models (MLLMs) (Comanici al., 2025; Bai et al., 2025b; Wang et al., 2025; Luo et al., 2025b;a) have demonstrated remarkable capabilities, yet significant limitation persists their capacity for complex, multi-step reflective reasoning. Existing benchmarks often focus on direct inference tasks, such as mathematical and logical problem-solving, overlooking the evaluation of long-chain visual reasoning processes in an end-to-end manner. To address this gap, we introduce MM-HELIX, a novel benchmark specifically designed to assess and challenge the limits multimodal reflective reasoning in MLLMs. The construction of this benchmark is guided by four principles: Multimodal, Long-Chain Reasoning, Reflection, and End-to-End. To instantiate these principles, we have curated 42 diverse tasks from public web resources and existing academic datasets, which organized into four categories: algorithms,
3

---

Published as a conference paper at ICLR 2026
Velueiee Pack
000001
Binalro
Campaile
Cakudika
Word Wordsearch
Eulero
Shingoki
Wordlladder
Snakk
Puzzles
Hirtori
Kakuro
Kukkrass
Numbiix
Sadoku
Nenggam
Fatosiki
Skysempers
Aquasium
Graphs
Ealeman Cycle
Ealerian Path
Hamiltonian Cycle
Hamilhomion
Shortest
Max Flow
Topologica
Graph Isomorphism
FEI DCEI CHAH
17
FECI
Alogrithms
CryptoMath
Trapping
Longest
Valleys
Subsequence
Histogram
Lavel
Levvel
Levvll'
16
12
Games
Level
Solooban
Nibbles
Maze
Harox
Minwsweeper Slulinguuzzee
Level
Lavel's
Overview of tasks in MM-HELIX benchmark. MM-HELIX contains 42 challenging tasks designed to evaluate long-chain reflective reasoning across five progressive levels difficulty.
Figure 2:
graphs, puzzles, and games, shown in Fig. 2. Each task necessitates that the model comprehend complex rules, recognize states within a visual context, and engage in sequential process of thought, reflection, and backtracking reach a solution, presenting substantial challenge current MLLMs.
To ensure the scalability, diversity, and controlled difficulty of our benchmark, we develop procedural generation framework. This framework architected around three core components: an Instance Generator, a deterministic Solver, and an automated Verifier. The Instance Generator produces problem instances based on task-specific rules and scalable parameters. Each generated instance comprises three elements: Question Description: A textual prompt outlining the task with its corresponding detailed rules. Visual Input: image presenting the initial problem scenario (e.g.. game board). Initial State: A structured data representation of the visual input to facilitate post- evaluation verification. An example shown Fig. 3; see Appx. A.7 for all cases.
For each generated instance, the Solver first an- alyzes the initial state using a rule-based algo- rithm determine the instance's solvability, If a solution is deemed to exist, the Solver pro- duces feasible solution to serve the ground truth. While tasks in the algorithm and graph cat- egories typically have unique solution, game and puzzle tasks often permit multiple valid so- lutions. To facilitate objective and accurate eval- uation, we construct an automated Verifier to assess model outputs. This component employs two distinct validation strategies based the complexity of the required response. For tasks with simple, discrete answers (e.g., boolean or numerical value), it performs direct exact- match comparison against the ground truth. For tasks requiring complex, multi-step solutions, the Verifier first standardizes the model's output
Nibblee
Image
Question
Geal: Findi
toeat apples
Game Rules
1. Caneral makke move around grdd
communds dup, down, right)
applls grid
an apple
longer one segasent
unile collide walls ihelf
Output Foumal Requirements:
Answeer should be oequenze of moves Example:
up right down laft up
Reference Answer
Category
Lavel
Initial State
Solver
up left un right right night nigh-down-d dowwdddoooo dosen.
Gome
Leooll
Statis PV2 (331 Applise
Figure 3: Example of Nibbles task (Level 5) in MM-HELIX benchmark. The snake must eat all apples the grid by executing sequence of moves, demanding long-term reflection.
4

---

Published as a conference paper at ICLR 2026
and then simulates the proposed sequence of actions from the Initial State, leveraging the problem's intrinsic rules to confirm the solution's 's validity.
A key feature of MM-HELIX is its hierarchical difficulty system, designed for the fine- Line-grained evaluation of model capabilities. We scale task difficulty by programmatically adjusting task-specific parameters within the generation framework, primarily by controlling the number of reasoning steps required for a correct solution. By modulating these parameters, we generate tasks across five distinct difficulty levels ranging from Level (very easy) Level (very hard), where both the problem's scale and reasoning complexity increase with each level. This tiered structure enables a precise identification of the performance degradation threshold for given model, thereby revealing the limitations of its reasoning capacity. The final evaluation set comprises 1 1,260 unique instances, corpus size selected to ensure statistical robustness while maintaining computational tractability. The dataset is balanced across both tasks and difficulty levels: for each of the 42 tasks, we generated 30 instances by sampling instances from each of the difficulty levels. This composition facilitates a reliable and granular assessment of model performance across a wide spectrum of complexity.
3.2 HM-HHEIX-100K- GUIDING MULTIMODAL REFLECTIVE REASONING
The capability for long-chain reflection is crucial in various advanced applications. However, our evaluation results on MM-HELIX reveal that even state-of-thh-an art MILMs,, such as Qwen-2.5- VL-72B, struggle significantly with these challenging reflective tasks, achieving an accuracy of only approximately 10%. This performance gap motivates our investigation into whether targeted instruction tuning can enhance this reflective capability and if such improvements can generalize to other complex reasoning domains, such as mathematics and logic.
To effectively train models for such complex, long-chain reasoning. a large-scale, high-quality dataset of reasoning trajectories is indispensable. To this end, we introduce MM-HELIX- 100K, meticulously curated dataset for instruction- tuning comprising 100k instances. The dataset spans 42 distinct tasks and incorporates high- quality responses with reflection.
Solver
Rule-Based CoT
Planning to move Current surroundings: up: empty, down: empty, left: empty, right: empty à MOVE: Player moves up (4. (repositioning)
Move I:
Answer
Rule-Cuided Scaffolding
Move Planning move left Current
surroundings up: wall, down: wall, left: box, right: empty- E PUSH: Moving pushes box from (1, 2) to (1. (GOAL!)
</>
MM-HELIX-COT
Generating high-quality Chain-of-Thought (CoT) trajectories at this scale presents formidable challenge. Conventional meth- ods. such prompting large model to generate reasoning steps from scratch in an unconstrained manner, are often inefficient and yield low-quality results. To overcome these challenges, develop hybrid and highly efficient data generation pipeline, which term Step-Elicited Response Generation (SERG), as the pipeline shown in Fig. 4. The process begins with our task-specific generators creating a base set of 150k problem instances. We first employ a programmatic, rule-based CoT constructor generate a deterministic, skeletal reasoning path by strategically embedding anchors critical intermediate states or calculations andooons---dddddd connecting them with template- based natural language descriptions. This initial step produces a logically sound but often mechanical and rigid reasoning trace, which is suboptimal for training nuanced language model.
The box at (4,6) is up and right from
To push it down
Hmm
To push box down, have be
MLMe- Enhancement
above it. So to push the box from (4.6) to (5,6), need to be (3,6)
Wait let's see the path. can go up to (4.5), then
up (3.5). Then right to (3.6)
After the first push, am (4.6), and the box is at I immedialely push it again - down (5.6). So can
Verifier
((v6), and I and up at (5.6).
Figure 4: Demonstration of our Step-Elicited Re- sponse Generation pipeline.
This rule-based trajectory then serves as a high-quality scaffold for a powerful model, in this work Qwen3-235B. We provide the model with the original question and the rule-based reasoning path, prompting it to refine this scaffold into a more natural, comprehensive, and human-like reasoning process that includes reflective steps. This enhancement phase enriches the dataset with linguistic diversity and more detailed explanations. To guarantee the final dataset's 's integrity, each generated trajectory is only accepted if its final answer passes the corresponding automated verifier. This stringent filtering mechanism is crucial for eliminating any errors introduced during the LLM enhancement phase and ensures the high fidelity of the training data.
5

---

Published as a conference paper at ICLR 2026
Question
Group Compulation
Loss
oc polcy
Policy Model
I Reward Group
Response
LOSSARPO
1.0, D.I, 14,, Demis
Your tack to solve the puzzle according the rales current
Expert Model
Off-Policy Expert Response
0.0, D.,, Sparse
{-1
Dynamic Activation
Loss Lappliy
Figure 5: Demonstration of Adaptive Hybrid Policy Optimization (AHPO). AHPO dynamically inte- grates off- off-policy expert guidance with on-policy exploration, leading to performance generalization.
3.3 AHPO: ADAPTIVE HYBRID ALGORITHM GENERALIZING REFLECTION
On- On-policy reinforcement learning algorithm, such GRPO, update its model exclusively from data generated by the current policy. complex task domains like MM-HELIX, the policy seldom generates successful trajectories, leading to severe reward sparsity that renders the training process inefficient and often ineffective (sce GRPO Fig. 6). common strategy to mitigate this is to initialize the policy via Supervised Fine- Fine-Tuning (SFT)
offline dataset of expert demonstrations. However, this methodology can induce a significant distributional shift, biasing the policy towards the SFT data distribution and constraining its ability to generalize and adapt during the subsequent RL phase.
To overcome these limitations, introduce Adaptive Hybrid Policy Optimization (AHPO), novel algorithm that integrates off-policy and on-policy learning into a unified training framework shown in Fig. 5. The cornerstone of method is adaptive mechanism that modulates the influence of offline expert data based on the policy's 's real-time performance. This allows the model to leverage expert guidance when needed and rely on its own exploration as it improves. We show the comparison of integrating off- off-policy loss to GRPO (called static-AHPO) in Fig. 6.
The AHPO objective function dynamically combines a standard off off-policy loss with an on-policy GRPO-style objective. The off- pooliy component is negative log-likeldhood loss on expert data y"
Iv'
Loft-policy (0)
log To |x. v2e).
(1)
|
The on-policy component is a clipped policy gradient objective:
N I
Con-policy (f) = ΣN
(I),
(2)
A, = R(Ti) mean (R(Ti)
Ti To (T), 1, 1,,,..., NF) To.sa (T), 1,2,...,NJ) N2)
std({R(TT))
where A; represents the estimated advantage for trajectory Tv. and (.) To(Ts,egg Ti,c)) Te (Ti,lq, Ti,cr) is the probability ratio for importance sampling. Follow- ing (Yu et al., 2025a), we omit the KL divergence term from the original GRPO formulation to reduce constraints policy exploration and decrease computational overhead; also remove the CLIP module for more efficient training.
AHPO unifies these objectives into single loss function, where the influence of the off- oefipolicy term is governed by an adaptive coefficient E:
CAHPO (0) = policy (8) + (on-cyyccy(()
Nor vf'
{ flog (Vi viree)
CLIP(ra((),,A,
(5)
Off policy objective
On On-policy objective

---

Published as a conference paper at ICLR 2026
0.5
0.6
Statie-AHPO
whw W
AHPO Statie-AlIPO
0.4
LUFFY
0.5
GRPO
0.4
Rewards
0.3
Rewards
0.2
0.1
0.0
80
120
160
200
1k
Steps
Steps
Figure Comparison of GRPO, LUFFY and Static-AHPO, Static-AHPO achieves best perfor- mance on challenging tasks.
Figure Comparison of Static-AHPO and АНPO. AHPO dynamically integrates expert data ensure robust training.
the activation coefficient is controlled by the following adaptive rule:
Non Σ T(R(T)) =1
f =
(6)
Here, I(() is the indicator function and R is predefined success rate threshold. This mechanism conditionally applies supervision from expert data: it provides dense guidance when the model's on-policy success rate is below the threshold R, preventing the agent from getting stuck or hacking in early training. Conversely, the policy improves and consistently achieves high rewards, the off-policy supervision is deactivated (g 0). This adaptive strategy ensures that expert guidance is present during the crucial initial stages of learning but fades through pure exploration. This prevents the model from merely memorizing the expert distribution and encourages the discovery more robust solutions.
allow the model to refine its policy
While prior work, such as LUFFY (Yan et al., 2025), has used expert data as positive examples in preference-batee RL framework, our empirical results demonstrate that this approach is less effective than our adaptive loss formulation for the complex tasks. Furthermore, the activation coefficient plays key role in making robust training. Although a static coefficient provides strong initial guidance, it creates persistent conflict between the off-polic expert distribution and the models's evolving on-policy distribution. This mismatch can destabilize training and even lead to performance degradation once the model has surpassed the proficiency of the expert data, shown in Fig. 7.
For our off-policy expert data, we utilize the high-quality CoT trajectories from the MM-HELIX 100K dataset. By dynamically balancing the exploitation of this expert data with on-policy exploration, AHPO effectively learns the reflective reasoning capabilities required by MM-HELIX benchmark and successfully generalizes these skills broader reasoning domains, leading significant performance enhancements in tasks involving mathematics and logic.
4
EXPERIMENT
4.1
EVALUATION RESULTS ON MM-HELIX
Our comprehensive evaluation of 23 leading MILMS on MM-HELIX benchmark, with full results detailed in Tab. I, reveals critical limitations in the reasoning capabilities of current models. The evaluation settings are detailed in Appx. A.3, The analysis yields three primary findings:
First, profound deficit exists in multimodal reflective reasoning. Even the most advanced proprictary model. GPT-5 can only achieve 58.1 1% accuracy, with no other model surpassing the 50% threshold. This performance gap S1-241B, reaches just 33.3% accuracy. The importance this targeted capability is also underscored by the fact that models capable of iterative reflection systematically outperform their non-reflective
more pronounced for open-source models; the leading contender, Intern-

---

Published as a conference paper at ICLR 2026
Table I: Evaluation results on MM-HELIX across both multimodal and text-only settings. These results underscore the ongoing difficulty MLLMs face with complex, long-chain reflective tasks. Thinking models with reflective reasoning capabilities generally achieve higher scores than those without. Furthermore, significant modality gap is observed where text-only inputs are superior.
Breakdown by Category
Overall
Model
Thinking
Algorithms
Graphs
Puzzles
Games
Txt
Ing
Lst
Img
Txt
ling
Txt
Img
Txr
Ing
Proprictary Models
GPT-5 (OpenAI, 2025b)
83.0
88.5
98.3
50.4
80.9
52.6
80.0
40.0
84.5
58.1
Sood-1,5-VL (Guo al., 20256)
89.3
55.6
33.3
66.9
04 mini (OpenAI, 2025c)
76.3
50.7
95.0
42.1
69.1
45.8
66.7
35.6
75.2
44.7
Gemini-2.55 Be5iin-2.5-Flashh (Comanici et 2025)
92.6
66.7
88.3
40.8
52.1
36.7
49.4
28.3
67.3
42.7
GPT-41 (OpenAI, 2025a)
61.9
44.4
73.8
35.0
13,9
8.9
43.3
25.1
GPT-4o (OpenAI, 2024)
x
33.7
18.9
44.6
25.4
10.2
4.2
10.6
6.7
21.8
11.7
Open-Source Models
mnern-S1-241B-A4 (Bai al. 2025a)
75.2
76.7
30.0
35.3
23.5
26.1
15.0
50.4
33.3
GLM-4.5V.106R-A 1-2--1hhniing (Team 2025b)
49.6
29.3
40.4
11.3
15.3
20.2
12.2
13.9
27.0
19.5
Kimi VL- A3B Thinking 2506 (Team et 2025a)
45.9
36.3
49.6
23.3
9.6
10.4
10.6
7.2
28.9
19.3
GLM-4.V-9B-Thiniiin (Team et al., 2025b)
38.1
30.7
50.4
29.2
11.6
7.4
5.0
6.1
23.7
16.3
Qwen-255VLL-7BB (Bai et 20256)
24.4
18.5
42.1
25.8
8.2
3.9
5.6
7.2
20.1
13.9
Qwen-2,5-VL-22 (Bai et al. 2025b)
X
22.2
15.2
46.3
22.5
8.1
4.7
5.6
6.7
20.6
12.3
QVQ-72B-Previeww (Team, 2024)
6.7
3.3
17.7
MiniCPM V-4,5-8B (Yu et 20255)
20.0
20.0
32.1
20.8
5.8
3.7
0.0
3.3
13,0
10.4
IntemVL33-78 (Zhu et al., 2025)
X 20.0
14.4
43.3
25.4
10.2
4.0
10.0
1.1
18.6
9.9
Intem
2025)
16.7
Llama-4-Scuut 10t-S/ooo---09---1-----0-0- (Meta, 2025)
24.1
16.3
40.8
21.3
4.4
4.2
2.2
1.7
15.2
9.7
Ovis2-34B (Lu et ol., 2024)
X
14.4
104
33.8
22.1
3.9
1.2
5.0
1.7
12.0
7.2
Gemma 3:27B (Team, 2025)
X
5.6
Qwen-255VVL-7B (Bai et al., 2025b)
5.6
5.9
25.4
17.9
0.4
0.4
0.6
1.1
8.0
6.3
InternVL.3 8B (Zhu et 2025)
X
8.1
5.9
28.8
16.7
1.6
0.7
1.1
1.1
4.9
Ovis2-8B 2024)
7.8
1.1
0.6
Ours
MM HEL.XEEIIXP 7B Thinking
32.2
34.8
27.5
19.2
16.3
25.3
16.1
16.7
21.8
24.99
counterparts. For instance, powerful model like Intern despite its strong performance general benchmarks, scores a mere 9.9%. This stark contrast validates that MM-HELIX successfully isolates and measures this critical reasoning skill.
Second, models excel at structured computation but falter in tasks requiring dynamic state tracking. Models demonstrated the highest proficiency on Algorithm tasks, which primarily involve mathemat- ical computation. Performance was moderate on Graph and Puzzle tasks, while the weakest results were observed in Game task. This trend suggests that while current MLLMS are adept at executing well-defined calculations, they lack robustness in adhering to complex instructions and performing the iterative state-tras track-trackig inherent strict rules.
Besides, significant modality gap still exists between text and vision. When problems were presented text-only format, performance improved dramatically. GPT-5's accuracy, for example, surged from 58.1 % on the multimodal tasks to 84.5% on their text-only equivalents. This significant performance drop highlights persistent gap between language and visual inputs.
4.2 MAIN RESULTS OF AHPO
We benchmark our proposed AHPO against comprehensive set of baselines( based on Qwen- 2.5VL.-BB, including pure RI. (GRPO), SFT, a sequential SFT+RL. pipeline, and an alternative hybrid algorithm LUFF, with training settings detailed in Appx. A.3. As shown in Tab. 2, the results strongly demonstrate the superiority of our method. On MM-HELIX, AHPO achieves the highest accuracy of 24.9 % among all methods, representing substantial +8..6% point improvement over the base model Qwen2.5-VL V-LBBB. Notably, this performance also exceeds that of significantly larger, state-of of ofee--o--hhearr the-art models such Qwen2.5-VL-722 and GLM-4.5V-106B More importantly, AHPO demonstrates remarkable generalization of its learned reflective reasoning capabilities. When trained on mix including the MMK12 RI. dataset which lacks explicit CoT traces, the model still
8

---

Published as a conference paper at ICLR 2026
Table 2: Comparison of AHPO and other training strategies, AHPO achieves significant improvement on MM-HELIX while also showing great performance transfer to general mathematics and logic tasks, indicating a robust enhancement of both specialized and generalized reasoning abilities.
In- Di-oomin
General Reasoning
Method
Type
MM-HELIX
Vision
M
VisicVista
WeMath
Average
Qwen2.5VL-7B
Baseline
6.3
25.2
40.5
45.6
34.5
36.5
+GRPO
On-policy
9.0(+2.7)
25.8
41.0
43.6
36.4
36.7(+0.2)
+SFT
Off-policy
23.8(+17.5)
21.7
33.0
38.7
26.2
(9.9(-6.6)
+SFT&GRPO
Sequential
23.3(+17.0)
25.9
39.1
45.9
35.7
36.7(+0.2)
+LUFFY
Hybrid
9.1(+2.8)
26.0
37.9
42.7
34.8
35.4(-1.1)
+AHPO (Ours)
Hybrid
24.9(+18.6)
26.6
47.5
53.5
41.1
42.2(+5.7)
Table 3: Comparison of CoT generation methods cost. Our hybrid approach significantly save the generation and make less redundancy.
Table 4: Efficiency of our dataset in SFT stage. Our method outperforms Rule-Based Co',, indi- cates great quality of our generation method.
(%)
Inf. (hrs) Avg. (tokens)
Method
Puzzle
Game
Algorithm
Graph
Overall
Model Rollout
25.00
~-111.9 1.96
7140.59
Rule-Based
19.3
11.1
22.6
19.6
18.9
SERG
99,80
-77.78
5500.53
Ours
23.5
16.7
32.6
20.4
23,8
learns to apply reflective inference on out-of-domain tasks. This is attributed to AHPO's explore- with-supervision mechanism, which fosters intrinsic reasoning skills rather than mere mimicry. Consequently, AHPO achieves an average performance gain +5.7% % points across general mathematics and logic tasks, validating its ability to transfer complex reasoning skills entirely new domains.
In contrast, the baseline methods reveal critical limitations. The RL-only approach (GRPO) shows negligible improvement on both task sets, failing to learn effectively from the sparse reward signals. While SFT significantly boosts in-domain performance, achieving comparable performance with AHPO, it induces catastrophic forgetting, leading to substantial performance degradation on the general reasoming tasks. Sequentially applying GRPO after SFT fails to recover this deficit, indicating that the reflective skills learned via fine-tuning do not effectively transfer to out-of-domain problems within this paradigm. LUFFY, which mitigates sparse rewards by substituting policy rollouts with expert data, shows minor gains but remains significantly less effective than AHPO in both performance and generalization. These findings underscore the superior efficacy and generalization capacity of AHPO's unified training strategy compared to sequential pipelines and other hybrid methods.
4.3 COMPARISON OF GENERATION PIPELINE
To validate the effectiveness of our SERG pipeline, we conducted a comparative analysis against two baselines. First, we compared SERG's efficiency and output quality against direct roll-outs from a powerful LLM (Qwen3-235B). For this, we prompted the model to generate reasoning trajectories for 1,000 samples without guidance. As detailed in Tab. 3. this unconstrained approach was computationally prohibitive, incurring substantial time costs and producing highly redundant responses. contrast, SERG demonstrated vastly superior performance, reducing the generation time by 90% while yielding significantly more concise and structured reasoning traces. econd, we evaluated the downstream utility of SERG-generated data. fine-tuned a model using 22k samples generated by SERG and compared its performance against model trained of data generated by a purely rule- rule-based method. As shown in Tab. 4, the model trained on SERG data outperformed the rule-based baseline 4.9%. This result confirms that SERG produces data of higher quality, leading to more effective downstream model training. Collectively. these experiments validate that SERG strikes optimal balance between generation efficiency and data quality.
equivalent amount

---

Published as a conference paper at ICLR 2026
Table Performance of each data component in training data. By utilizing AHPO with MM-HELIX dataset which has no overlap with the mathematical content MMK12, the model achieves much better performance on both in in-domain and general benchmark.
Method
MM-HELIX MathVision
Math Verse-V
LogicVista
WeMath
Qwen2.5V1.-78
6.3
25.2
40.5
45.6
34.5
+MMK12 Only
5.5
26.0
44.0
43.2
35.2
+MM-HELIX Only
24.4
23.2
41.9
48.3
36.5
+Mixed
24.9
26.6
47.5
53.5
41.1
4.4 ABLATION OF TRAINING DATA
We condcuct an ablation study of different data components within training data, results shown in Tab. 5. It should be noted that MMK12 is math-specific dataset primarily used for Reinforcement Learning. It contains only Question and Ground Truth answers, lacking the Chain-of- -f-ouuhtth trajectories necessary for our AHPO method's reflection mechanism. Training solely on MMK 12 thus functionally aligns with the GRPO baseline.
The results clearly demonstrate that our AHPO utilizing the mixed dataset significantly surpasses the GRPO baseline trained only on MMK12. This empirical evidence strongly indicates that the MM-HELIX dataset, via its high-quality reflective CoT, contributes the essential supervisory signal for AHPO effectively learn and generalize sophisticated reflection and reasoning capabilities beyond the scope achievable with standard RL and MMK12 alone.
CONCLUSION
In this work, we address the critical deficiency of MLLMs in long-chain reflective reasoning. We begin by introducing MM-HELIX, a benchmark that confirmed the profound limitations of current models. To solve this, we develop the HM-HELIX- -00K dataset to provide high-quality training data and proposed AHPO, method that unifies on- and off- off-policy learning effectively cultivate this skill. The resulting MM-HELIX- -- model achieved significant performance gains on both our in-domain benchmark and general reasoning tasks. Our findings establish that reflective reasoning transferable skill that can be instilled and generalized in MI.Mss.
ACKNOWLEDGMENT
This work was supported by the National Natural Science Foundation of China (62506229), Natural Science Foundation of Shanghai under 25ZR 1402268, Shanghai Yuan Innovation Foundation, and Scientific Research Innovation Capability Support Project for Young Faculty (U40) of the Ministry of Education of China (SRICSPYF-ZY20055119,
ETHICS STATEMENT
This research adheres to the ICLR ethical guidelines and upholds the principles of responsible research. We ensure that no personally identifiable, sensitive, harmful data were used. Our experiments were based on publicly available datasets and did not involve any human subjects or vulnerable groups. We have considered the potential societal impact of our methods, including the risk misuse, and believe that these contributions primarily advance scientific understanding and do not pose foreseeable harm.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We follow the reproducibility guidelines in the ICLR 2026 author guidelines. We will open source code, configuration files, and scripts to reproduce our results, including dataset construction, model training, and evaluation, on platforms such as GitHub and Huggingface.
REFERENCES
Lei Bai, Zhongrui Cai, Maosong Cao, Weihan Cao, Chiyu Chen, Haojiong Chen, Kai Chen, Pengcheng Chen, Ying Chen, Yongkang Chen, et al. Intern-sl: A scientific multimodal foundation model. arXiv preprint wrXiv:2508.15000 2025a.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint 13923, 2025b.
Jiangjie Chen, Qianyu He, Siyu Yuan, Aili Chen, Zhicheng Cai, Weinan Dai, Hongli Yu, Qiying Yu, Xuefeng Li, Jiaze Chen, al. Enigmata: Scaling logical reasoning in large language models with synthetic verifiable puzzles. arXiv preprint urXiv:2505. 19914, 2025.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint 2rXiv:2507.006611, 2025.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et Deepseek-rl: Incentivizing reasoning capability llms via reinforcement learning. arXiv preprinl arXiv:2501. 122498, 2025a.
Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi. Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, et al. Seedl. 5-vl technical report. arXiv preprint arXiv:2505.07062, 2025b.
Shiyin Lu, Yang Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Han-Jia Ye. Ovis: Structural embedding alignment for multimodal large language model. arXiv:2405.20790 2024.
Gen Luo, Wenhan Dou, Wenhao Li, Zhaokai Wang. Xue Yang. Changyao Tian, Hao Li, Weiyun Wang, Wenhai Wang, Xizhou Zhu, et al. Mono-internvl-I.. Towards cheaper and faster monolithic multimodal large language models. arXiv preprint arXiv:2507. /2566, 2025a.
Gen Luo, Xue Yang, Wenhan Dou, Zhaokai Wang, Jiawen Liu, Jifeng Dai, Yu Qiao, and Xizhou Zhu. Mono-internvl: Pushing the boundaries of monolithic multimodal large language models with endogenous visual pre- vre-training. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 24960 24971. 2025b.
Fanqing Meng. Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang. Junjun He, Kaipeng Zhang, al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. CoRR, 2025.
Meta. The lama 4 herd: The beginning of new era of natively multimodal ai innovation. 2025.
URL httpt:tp//// / / meta. www/booo/geooooooooooooooooooooo 11ama 4 mww.imoddll-ineelllle
OpenAl. Hello gpt-4o. 2024. URL https::/p///. http://oaaaa.cacae//aaeaaaaaaaaa
OpenAl. Introducing gpt-4.1 in the api. 2025a.
URL https:///opeaaiccom/iii
gpt-4-1/.
OpenAl. Introducing gpt-5.
2025b.
URL
https://ogenaii.co//nneee
Introducing-gpt-55
OpenAl. Introducing openai o3 and o4-mini. 2025c. URL URU.htppee//ppeaaa.cc///oooo
introducing-ol -00 -o and-o4-minit
11

---

Published as a conference paper at ICLR 2026
Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, et al. We-math: Does your large multimodal model achieve numan-like mathematical reasoning? arXiv preprint arx 2024.
Yufan Ren, Konstantinos Tertikas, Shalini Maiti, Junlin Han, Tong Zhang, Sabine Süsstrunk, and Filippos Kokkinos. Vgrp-bench: Visual grid reasoning puzzle benchmark for large vision-language models. arXiv preprint arXiv:2503.23000 2025.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint TrXXvv1707.06444 2017.
ByteDance Seed, Jiaze Chen, Tiantian Fan, Xin Liu, Lingjun Liu, Zhiqi Lin, Mingxuan Wang. Chengyi Wang, Xiangpeng Wei, Wenyuan Xu, et al. Seed1. 5-thinking: Advancing superb reasoning models with reinforcement learning. arXiv preprint arXiv:2504. 13914, 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemat- ical reasoning in open language models. arXiv preprint arXiv:2402.033000 2024.
Gemma Team. Gemma 3. 2025. URL https: /goo gle/ /eema 3Rpport.
Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin
Zhang, Chenzhuang Du, Chu Wei, et al. Kimi-vl technical report. arXiv preprint arXiv:2504.07491, 2025a.
Qwen Team. Qvq: To see the world with wisdom, December 2024. URL https::////seeeeee github.ion io/blog http://loo/vqqg2---pe// wtwg--7bbpreviee.
Team, Wenyi Hong. Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi. Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang, Yan Wang, Yean Cheng. Zehai He, Zhe Su, Zhen Yang. Ziyang Pan, Aohan Zeng. Baoxu Wang. Bin Chen, Boyan Shi, Changyu Pang. Chenhui Zhang, Da Yin, Fan Yang. Guoqing Chen, Jiazheng Xu, Jiale Zhu, Jiali Chen, Jing Chen, Jinhao Chen, Jinghao Lin, Jinjiang Wang. Junjie Chen, Leqi Lei, Letian Gong, Leyi Pan, Mingdao Liu, Mingde Xu, Mingzhi Zhang, Qinkai Zheng, Sheng Yang. Shi Zhong, Shiyu Huang. Shuyuan Zhao, Siyan Xue, Shangqin Tu, Shengbiao Meng. Tianshu Zhang, Tianwei Luo, Tianxiang Hao, Tianyu Tong, Wenkai Li. Wei Jia, Xiao Liu, Xiaohan Zhang, Xin Lyu, Xinyue Fan, Xuancheng Huang, Yanling Wang. Yadong Xue, Yanfeng Wang, Yanzi Wang, Yifan An. Yifan Du, Yiming Shi, Yiheng Huang. Yilin Niu, Yuan Wang, Yuanchang Yue, Yuchen Li, Yutao Zhang, Yuting Wang, Yu Wang, Yuxuan Zhang, Zhao Xue, Zhenyu Hou, Zhengxiao Du, Zihan Wang. Peng Zhang, Debing Liu, Bin Xu, Juanzi Li, Minlie Huang, Yuxiao Dong, and Jie Tang. Glm-4.5v and glm-4.1v-te Iv-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning, 2025b. URL https://///. / / /awxivviivvv./iii//a..//aaaavvvt org/ abs 22007 01006.
Jingqi Tong, Jixin Tang. Hangcheng Li, Yurong Mou, Ming Zhang, Jun Zhao Yanbo Wen, Fan Song, Jiahao Zhan, Yuyang Lu, et al. Code2logic: came-code-driven data synthesis for enhancing vims general reasoning. arXiv preprint arXiv:2505. 10055.13886 13886, 2025.
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095 951699,9999911 2024.
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Lingli Jing, Shenglong Yc, Jie Shao, et al. Internv13. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint a5Xiv:2508. 18265, 2025.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Ohain-of-thouugh prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 24837, 2022.
Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logicvista: Multimodal 11m logical reasoning benchmark in visual contexts. arXiv preprint arXiv:2407.04997 047733 2024.
Jianhao Yan, Yafu Li, Zican Hu, Zhi Wang, Ganqu Cui, Xiaoye Qu, Yu Cheng, and Yue Zhang. Learning reason under off- ooflpo guidance. arXiv preprint a5Xiv:2504 14945, 2025.
12

---

Published as a conference paper at ICLR 2026
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36: 111118899111 2023.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo. Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source IIm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025a.
Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang, Wenshuo Ma, Zhihui He, Tianchi Cai, Weize Chen, Yuxiang Huang. Yuanqian Zhao, Bokai Xu, Junbo Cui, Yingjing Xu, Liqing Ruan, Luoyuan Zhang, Hanyu Liu, Jingkun Tang, Hongyuan Liu, Qining Guo, Wenhao Hu, Bingxiang He, Jie Zhou, Jie Cai, Ji Qi, Zonghao Guo, Chi Chen, Guoyang Zeng, Yuxuan Li, Ganqu Cui, Ning Ding, Xu Han, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Minicpm-v 4.5: Cooking efficient milms via architecture, data, and training recipe, 2025b. URL https: / aariv www.wi...gaaaaaaeeee.. abs /2009. 18154.
Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo. Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, al. Mathverse: Does your multi-modal Ilm truly see the diagrams in visual math problems? In European Conference on Computer Vision, pp. 169-186. Springer, 2024.
Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. Group sequence policy optimization. arXiv preprint arXiv:2507.18071 2025.
Jinguo Zhu, Weiyun Wang. Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint 154,,99994,,499, 2025.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 THE USE OF LARGE LANGUAGE MODELS (LLMS)
LLM was not involved the development of the ideas for this article. As for the writing process, only used LLM to correct minor errors such grammatical errors. During the data construction process, we rewrote the reasoning process using LLM Qwen3-235B.
A.2 ERROR ANALYSIS
We have conducted qualitative examination of the model's incorrect outputs and categorized the primary failure modes into four distinct classes:
1. Perception Errors:
This occurs when the model fails to correctly interpret the raw visual state.
Example: Maze task, the model fails identify the presence of wall at specific coordinate, leading to invalid path planning proposals that traverse obstacles.
Hallucination
Observation Mismatch):
Here, the model generates conclusions that contradict its own intermediate observations, effectively forcing result to match prior expectation rather than the evidence.
The model explicitly identifies character sequence at the predicted coordinates "L, N, O, (Reasoning trace: "So letters are L, N, O, I... that's LION!"). Crucially, the model ignores the clear mismatch between the observed sequence ("LNOI") and the target word ("LION"), hallucinating a successful anagram solution despite explicit contradictory evidence.
Example:
3. Insufficient Backtracking:
When facing conflicting constraints, the model tends attempt local fixes rather than performing valid backtracking to reset earlier variables. This often results in the model settling for an invalid approximate solution.
Example: In 5x5 Kukurasu Puzzle, The model successfully satisfied all row sum constraints but failed the column sum constraints (e.g., the predicted Column sum was 9, whereas the target was 10). Although the model explicitly identified the conflict during final verification ("Required 10. Close, but not enough"), it failed to backtrack and revise the earlier row-level decisions that caused this downstream conflict.
4. Instruction Following Failures(Minor):
The model occasionally violates explicit hard constraints, particularly negative constraints provided in the system prompt.
Example: In Word Ladder Task, The prompt explicitly forbids adding deleting letters (only substitution is allowed). The model violated this constraint by transitioning from "cappy" (5 letters) to "clappy" (6 letters), disregarding the length invariance requirement.
A.3 EXPERIMENT SETTINGS
Evaluation Settings. All evaluations were conducted using the VLMEvalKit framework. For our primary benchmark, MM-HELIX benchmark, evaluation parameters were tailored to the model type. For models equipped with thinking mode, we set the maximum generation length to 32,768 tokens (or 16,384 for models with smaller context windows) and temperature of 0.6 to allow for diverse outputs. For models without thinking steps, the maximum length was set to 8, 8,192 tokens and the temperature created a text-only version of the problems by transcribing the multimodal inputs. To assess the generalization of reasoning skills, on mathematics and logic: Math Vision (Wang et al., 2024), Math Verse- VisionOnly (Zhang et al., 2024), Vigic
set to 0.0. To isolate the textual reasoning component of the tasks,
also
included a suite of challenging external benchmarks focusing
Vista (Xiao et al., 2024), and WeMath (Qiao et al., 2024).
In MMM-HEIXX benchmark, each evaluation instance is composed of a textual task description and a problem representation. The problem representation is rendered in either visual (image) textual
14

---

Published as a conference paper at ICLR 2026
modality. Crucially, ensure fair comparison, the underlying content is identical across modalities; we first generate a unified "initial state" for each problem and then render into the corresponding image or text format. To represent image-based tasks in textual format, employ structured symbolic representations tailored to the specific domain:
Algorithmic Tasks: We utilize standard list serialization (e.g., 12, 5,...) represent data structures.
Graph Tasks: We employ adjacency lists to describe connectivity. For example, 0: [1. 7]; 1: [0. 2]... indicates that Node 0 is connected Nodes and 7, while Node is connected to Nodes 0 and 2.
Puzzles and Games:
We use 2D array (matrix) representations to denote the grid state. For instance, a Sudoku puzzle is serialized
nested list: [[3, 0, 2, .--], [1, 7, 9, -.-. .... [9, 3, 8, ...].
Training Settings. The RL training stage was implemented using the VERL framework and the verifiers from MM-HELIX-- -Engine were integrated into VERL reward judger. We used global batch size of 128. During the RL stages, we generated response trajectories for each data sample. In AHPO, success rate threshold R is defined to 2. For SFT stage, we used 22k samples from MM-HELIX-IONE dataset. RL stage (for GRPO, LUFFY, and our AHPO). we created combined training set of approximately 37k samples by mixing data from our MM-HELIX-CES 100k and the general mathematics RL dataset MMK12 (Meng et al., 2025), which contains 15k multimodal QA pairs and no off-policy response. For hybrid algorithms like LUFFY and our proposed AHPO, the response from HI-HHELXX-CO--18 served as off-policy expert data. The MMK12 dataset contained no off-policy traces, compelling the model to learn via exploration in the mathematical domain.
In our experiments, we set the empirical value of N = 5. Our implemented loss function utilizes a sample-level averaging scheme without token-level weights. Consequently, the normalization factor Z effectively simplifies to N.
A.4 DETAILS OF COMPARISON OF COT GENERATION PIPELINE
We compares the length distribution of Rule- Based CoT, CoTs generated by model rollout, and those generated by Step-Elicited Response Generation (SERG). The average token count for Rule-Based CoT is 2,728.83, 83, for model roll- out it is 7,552. 17, and for SERG it is 5,715 5. The specific distribution is presented in Fig. 8.
Qwer3_2356 Tiken Districution Proportion Comparisse son
Due to the rigid, mechanical reasoning process of Rule-Based CoT and the inefficiency, redun- dancy inherent in model rollout, coupled with the lack supervision to ensure the correctness of the reasoning process, we propose the Step- Elicited Response Generation (SERG) pipeline. The CoTs generated by SERG address the short- comings of the first two approaches, providing more accurate and concise reasoning process with reduced redundancy and improved quality.
000
0.00
20009
Riee
Figure 8: Tokens distribution Rule-Based CoT and CoT's generated by model rollout, and SERG.
This distribution reveals that CoTs generated by model rollout exhibit greater redundancy, containing a higher proportion of irrelevant information. In contrast, SERG produces higher-quality CoTs with fewer extraneous details, benefiting from the structured logical reasoning steps of the Rule-Based COT. This highlights the superiority of our feip-Eiicited Response Generation pipeline.
15

---

Published as a conference paper at ICLR 2026
A.5 MM-HELIX-IOOM 100K STATISTICS
Table 6: Difficulty distribution tasks in MM-HELIX-100K.
Table 7: Statistics for question, answer, and Gnain-of-thoughy (cot) MM HEM-HELIX-OE HELIX-10OK.
Category
Count
Percentage
MM-HELIX-100K Average Tokens
Count
18,932
17.48%
Question
161.93
2
20.834
19.23%
Question Language
295.58
3
23.531
21.72%
Answer
45.52
108,362
4 22,280
20.55%
Rule-Basee CoT
2,643.51
5
19,038
17.56%
Final CoT
4,181.40
We perform statistical analysis of MM-HELIX-1100s The difficulty distribution of MM-HELIX- 100k is uniform, with detailed data presented in Tab. 6. Additionally, we conduct token length analysis for the questions, answers, and CoTs in MM-HELIX-I00k, with the results shown in Tab.
A.6 TASK DIFFICULTY SETTINGS
For each task, we divide the difficulty based on different parameters when constructing the initial state of the data. The following is typical example.
Difficulty Settings of Nibbles
Initial State
Map Size:
Num of Apples:
Level
Initial State
Map Size: *
Num of Apples: 2
Level
Initial State
Map Size: *
Num of Apples: 3
Level
16

---

Published as a conference paper at ICLR 2026
Initial State
Map Size: 9
Num of Apples: 4
Level
Initial State
Map Size: 10 10
Num of Apples: 5
Level
A.7 MM-HELIX BENCHMARK EXAMPLES
Aquarium
Question
Image
The grid is divided into multiple aquariums (regions). Your task to determine which cells are filled with water based on the following rules: Game Rules:
1. Eaah region must be filled uniform water level (from bottom up).
Water cannot float if cell is filled, the cell directly below it (if any, in same region) must also be filled.
3. The numbers outside the grid indicate how many cells are filled with water in each row and column. 4. Regions separated by thick black lines in the grid. Cells within the same region (enclosed by thick lines) must follow the same water level rule. Cells separated by thinner lines are still in the same region. Coordinate system:
2
Category: Puzzle
(x, y) where (0, 0) is the top-left cell. x increases to the right, y increases downward.
Difficulty: Level
Answer Format:
Please list all the cells that are filled with water in the format: [(x1, y)), (x2, ...]
Example: [(0, 4), (1, 4), (1, (2.
Reference Answer [(2, 1), (3, 1), (0. 2). (3, 2), (0, 3), (1, 3)]
17

---

Published as a conference paper at ICLR 2026
Kakuro
Question
Image
Your task is to solve the Kakuro puzzle from the given image by filling white cells with appropriate digits.
4
5
Game Rules:
1. The puzzle is grid where black cells contain clue numbers and white cells need be filled with digits
1-9.
8
In black cells, numbers below the diagonal are 'down' clues, and numbers above are 'right' clues. Each clue indicates the sum of consecutive white cells in that direction.
9
12
12
4. Digits in each run cannot repeat.
Coordinate System:
The grid coordinates start at (0,0) in the top-left corner.
Category: Puzzle
Rows increase downward and columns increase to the right.
Difficulty: Level
Oulput Format:
Provide your answer as space-separated list of coordinate-value pairs in the format: (row,column):value.
Example: (0,2):5 (0,7):7
Reference Answer
(1,0):4 (1,1):3 (1,3):6 (2,1):2 (2,2):6 (3,0)9 (3,3):3 (3,4):9 (4,1):6 (4,3):9
18

---

Published as a conference paper at ICLR 2026
Nibbles
Question
Image
You are puzzle solver focusing Snake puzzles.
Game Rules:
1. Control snake to move around the grid using directional commands (up, down, left, right).
2. The snake must eat all apples on the grid to win. When the snake eats an apple, grows longer by segment.
4. The snake cannot collide with walls or itself.
5. The snake moves one cell at a time in the chosen direction.
Inpul: An image showing the initial state with the snake and apples.
Category: Puzzle
Goal: Find sequence of directional moves to eat all apples without the snake colliding with walls or itself.
Difficulty: Level
Oulput Format Requirements:
Your answer should be a sequence of directional moves separated by spaces. Valid moves are: up, down, left, right.
Example: up right down left up.
Reference Answer
down down down down down left left left left left down left
19

---

Example: Published as a conference paper at ICLR 2026
Nonogram
Question
Image
Your task is to solve the Nonogram puzzle according the rules and current state below:
Game Rules:
"The numbers outside each row or column are clues.
Each number indicates a continuous block filled cells.
1
1
The order of the numbers matches the order of the blocks from left to right (for rows) top to bottom (for columns).
1
There must be least one empty cell between consecutive blocks in row or column.
Fill the grid so that all row and column clues are satisfied simultaneously.
Category: Puzzle
Symbols: X Filled Empty cell
Difficulty: Level 2
Output Format:
Output the solution as a text-based grid using 'X' and :.:
Each line represents a row in the solved grid.
No spaces between characters.
X..
X.X.
.....
X.X.
X.X..
Task:
Carefully analyze the given image of the Nonogram. Produce the complete solved grid according to the rules.
Reference Answer
X....
X.X
...X.
X..
20

---

Published as a conference paper at ICLR 2026
Numbrix
Question
Image
Your task is to solve the Numbrix puzzle based the following rules and the current state below:
17
15
11
Game Rules:
Numbrix is played on square grid, where some cells are already filled with numbers.
19
You must fill in the empty cells with numbers to create continuous path starting from up to the maximum number in the sequence, which is not necessarily equal to the total number of cells (n²). The numbers must be adjacent either horizontally vertically (not diagonally).
21
7
9
23
4. Each number can only be used once.
The path must form single continuous sequence where consecutive numbers are adjacent.
25
5
3
Not every empty cell needs to be filled. Depending on the puzzle configuration, some cells may remain empty.
Category: Puzzle
Difficulty: Level 2
Important Notes:
The highest number in the puzzle might be equal or less than the total number of grid cells (e.g.. n² smaller).
is your job to determine what the highest number is, based the filled numbers and the constraints of the puzzle.
Current Numbrix State:
The current state of the Numbrix puzzle is shown in the image below.
Output Formal Requirements:
The final answer should the completed grid with all numbers from to the correct highest number, aligned clearly in rows and columns.
Example answer format for a 5x5 grid:
1111109/2231
112113181144
1151141716/51
11119110/231241 11711 1111812112212511
Reference Answer
117116111121111 II
111111141111101
1201/1889
1221231611121
(251241514131
21

---

Published as a conference paper at ICLR 2026
Shingoki
Question
Image
You are given a Shingoki puzzle. This is a logic puzzle where you need draw single continuous loop grid.
Game Rules:
Draw exactly one continuous loop without crossings branches.
The loop must eventually return to its starting point.
3. White circles must be passed through in a straight line (no turning white circles).
4. Black circles must be turned upon (the path must change direction black circles).
5. Each circle has a number that represents the sum of the lengths of the two straight line segments extending from that circle.
Category: Puzzle
Coordinate system:
Difficulty: Level 4
(0,0) is the top-left corner.
Row numbers increase downward, column numbers
increase rightward.
The loop connects adjacent grid points (no diagonal connections).
Objective:
Find the single continuous loop that: Passes through all circles according to their type constraints.
Satisfies all circle value constraints (sum of line
segment lengths).
Forms closed loop without crossings or branches.
Output Format:
Represent your solution as a sequence of connected line segments.
Each segment connects two adjacent grid points: (rl.c1)-(2,c2)..
Adjacent points differ by exactly in either row or column (no diagonals).
List all segments separated by spaces in one continuous string-
The segments should form complete closed loop. Example format: (0,0)-(0,1) (0.1)-(1,1) (1,1)-(1,0) (1,0)-(0,0).
Reference Answer
(0,0)-(0,1) (0.1)-(0,2) (0,2)-(0,3) (0,3) (0,4)- (0,5) (0,5)-(1,5) (1,5)-(2,5) (2,5)-3,5) (3,5)-(4.5) (4.5)-(5.5) (5,5)-5,44 (5,4)-(5.3) (5.3)-(5.2) (5,2)- (5.1) (5,1)-(500 (5,0)-4.00 (4,0)-(30) (3,0)-2.0)
(2,0)-11,0) (1,0)-(0,0)
22

---

Published as a conference paper at ICLR 2026
SlidingPuzzle
Question
Image
Your task is to solve the 15-puzzle game according the rules and current state below:
1
2
3
4
Game Rules:
1. The puzzle is played
4x4 grid with 15 numbered tiles and one empty space.
5
6
11
7
2. You can only move tiles horizontally or vertically
into the empty space.
3. The goal is to arrange the tiles in numerical order with:
9
10
8
-First 1, 2, 3,
Second row: 5, 6, 7, 8
Third row: 10, 11, 12
13
14
15
12
Fourth row: 13, 14, 15, empty space.
Coordinate System:
Category: Puzzle
The grid positions are numbered from left to right and top to bottom.
Difficulty: Level 2
Columns (horizontal): numbered 1,2,3, from left
right.
Rows (vertical): numbered 1, 2, 3, 4 from top to
bottom.
Each position can be identified by its row and column (row, column).
Current Puzzle State: The initial state is represented in the image shown.
Output Format Requirements:
"up" means the tile below the empty space moves up into the empty space.
"down" means the tile above the empty space moves down into the empty space. "left" means the tile to the right of the empty space moves left into the empty space.
"right" means the tile to the left of the empty space moves right into the empty space.
Your final answer format should be given like: up down up left right.
Reference Answer
left down left up up
23

---

Published as a conference paper at ICLR 2026
Snake
Question
Image
Please examine the image carefully. The image
shows a Snake puzzle grid.
Rules:
Draw a single, non non- -nt--seeteeeting snake path from (start) to E (end).
The snake occupies some cells; cannot touch itself, even diagonally.
The numbers outside the grid indicate how many
snake cells appear in each row and column.
Provided Clues:
-Grdd size: 9x9
Row counts: 0, 0, 0, 0, 8.
Column counts: 0, 1. 1, 1. 1.5
Category: Puzzle
Refer to the image to solve the puzzle
Difficulty: Level
Output Format:
Return the snake path as a sequence of coordinates, e.g.: (r0,c0) (rl,cl)...
Reference Answer
(4.1) (4,2) (4,3) (4,4) (4.5) (4.6) (4.7) (4,8) (5,8) (6,8) (7,8) (8,8)
24

---

Published as a conference paper at ICLR 2026
Sokoban
Question
Image
Your task is to solve the Sokoban puzzle according to the rules and current state shown in the image:
Game Rules:
You are the player and can move up, down, left, or
right.
You can push boxes one space at time.
3. You cannot pull boxes.
4. Boxes can only pushed if there's an empty
space behind them.
The goal is to push all boxes onto target positions.
6. Walls cannot be moved through pushed.
Current Sokoban State: The current state of the Sokoban puzzle is in the image shown below.
Direction Definitions:
Category: Puzzle
"up": Move up
"down": Move down
Difficulty: Level
"left": Move left
"right": Move right
Output Formal Requirements:
Your final answer should be in the format of space-separated sequence of moves like: up right down left.
Reference Answer
left left down down left left up up right down down left down right up up up right right right
25

---

Published as a conference paper at ICLR 2026
Wordsearch
Question
Image
Your task is solve the wordsearch game according to the rules and current state below:
B
Task:
Z
P
You are given a word search puzzle. Your task is to find the listed word hidden in the grid and provide their exact locations in the specified format.
V 0 A
VE
FEUDRE R
TH
10 MOVDD
0
RM
GAAHT C
P
P
Game Rules:
1 ILTGNE
1. Words can be hidden horizontally, vertically, or diagonally.
2
1 AITOE G
KIPEKI
0
2. Words can read forwards or backwards.
PPH
3. Words always follow a straight line (no zigzag- ging).
4. Each word's ' location should be identified by:
The starting position (coordinate where the first letter appears)
Category: Puzzle
The direction in which the word extends
Difficulty: Level
Coordinate System:
TTe grid uses coordinates where (x, y) represents the position.
x-axis: Numbers from 1 to width, rumning horizon- tally from left to right.
y-axis: Numbers from 1 to height, running vertically from top to bottom.
Example: Position (3, 4) means column 3 from left, row 4 from top.
Direction Notation:
N: North (upward)
-S South (downward)
E: East (rightward)
W: West (leftward)
NE: Northeast (up and right)
NW: Northwest (up and left)
SE: Southeast (down and right)
SW: Southwest (down and left)
WordSearch State: The current state of the WordSearch is shown in the image given below.
Output Format Requirements:
Your final answer format should be given like: WORD DIRECTION
(x, y), where WORD word you found, DIRECTION is the direction in which the word extends, and (x, y) is the starting position of the word.
Reference Answer
ELEPHANT NE @ (5,14)
26

---

Published as a conference paper at ICLR 2026
Tapa
Question
Image
Please look at the displayed Tapa puzzle image. The numbers the cells are clues indicating the lengths of connected groups of black cells surrounding that clue.
1
Task:
Your task is to fill in the grid with black cells according to the following rules.
1
2
5
Game Rules:
1
4
1. All black cells must form single connected group**: This means that all the black cells on the grid must be connected in one continuous region, without any isolated black cells.
There cannot be any 2x2 block of black cells: 2x2 block of black cells is not allowed anywhere on grid. This means that no four black cells can form square.
Category: Puzzle
Difficulty: Level
Clue cells: Each number in a clue cell indicates the length of connected group of black cells surrounding that clue. The "surrounding" refers the 8 neighboring cells that are orthogonally and diagonally adjacent to the clue (i.e., the cells that are directly adjacent horizontally, vertically, or diagonally to the clue).
For example, clue "3" means that exactly three black cells must be placed among the 8 surrounding cells, and these three black cells must form a single connected group.
Each clue cell contains only single number representing connected group of black cells. 4. Grid size: The grid is a sizexsize matrix of cells. Each row and column will contain mix of black (B) and white (W) cells.
Coordinate System:
The grid uses coordinate system where (0.0) is the top-left corner, the first number represents the row (increasing downward), and the second number represents the column (increasing rightward).
Output Format:
List only coordinates cells that should be colored black
Use the format (row,column) for each coordinate
Separate multiple coordinates with commas
For example: (0,1), (1,2), (2.0), (2,1)
Reference Answer
(0.1), (0,2), (0,3), (1,3), (2.3), (3,2), (3,3)
27

---

Published as a conference paper at ICLR 2026
Maze
Question
Image
Your task is solve the maze game according the rules and current state below:
Game Rules:
The maze consists of grid of cells.
2. Walls are represented by bold black line between cells, not as cells themselves.
3. You can move horizontally vertically between adjacent cells there no wall between them.
4. You can only move through one cell at time in any direction.
5. The goal is to find a path from the start cell (Green
Circle) to the end cell (Red Cross).
Direction Definitions:
"upp": : Move to the cell above the current position.
"down": Move to the cell below the current
Category: Puzzle
position.
Difficulty: Level 5
"left": Move to the cell to the left of the current
position.
"right": Move to the cell to the right of the current position.
Current Maze State:
The maze is represented in the image shown below.
In this representation:
Green circle marks the start position.
Red cross marks the end position.
Output Formal Requirements:
Your final answer should be in the format like: right down left up.
Reference Answer
up left left up left down down right down right right right up up up left left up up right up left left down left up left down down left down down down down left left left down right down left down down right up right right up up right right down right up up left left
28

---

Published as a conference paper at ICLR 2026
Hanoi
Question
Image
Your task is to solve the hanoi game according the rules and current state below:
Game Rules:
1,The Tower of Hanoi consists of three pegs (numbered 1, 2, and 3) and (maybe 3) disks of different sizes (from to n).
2. Disks stacked on pegs with larger disks always below smaller ones.
3. Only one disk can be moved at time, from the
Category: Puzzle
top one peg to the top of another.
4. A larger disk cannot be placed on top of smaller disk.
Difficulty: Level
Current Hanoi State: The current state of the Tower of Hanoi is in the image shown below.
Goal State:
The goal is to move all disks to peg maintaining the size order (largest at bottom, smallest at top).
For 3 disks: Peg 1: []. Peg 2: []. Pog 3: [3, 2, 1].
In this representation:
Each peg is shown with its contents in array format. Numbers represent disk sizes (higher numbers
larger disks).
Disks are listed from bottom top (first element bottom disk, last element top disk).
Output Format Requirements:
Your final solution format should be given like: (x.y) (x.y) (x.y).., where is the disk number and y is the destination peg number.
Reference Answer (1, 3)
29

---

Published as a conference paper at ICLR 2026
Hitori
Question
Image
You are given image of Hitori puzzle.
Puzzle Rules:
1. In each row and each column, numbers in unshaded cells must be unique.
1
1
2
5
1
2. Shaded cells cannot be adjacent horizontally or vertically.
2
3
4
3
5
3. All unshaded cells must form a single connected region (connected orthogonally).
1
3
3
4
2
2
4
1
2
1
Coordinate System:
Coordinates must be in the format (row, column) (0,0) 0) refers to the top-left cell of the grid Indexing is zero-based
4
2
4
3
1
Output Format:
Category: Puzzle
Please return the set of shaded cell coordinates.
Example output:
Difficulty: Level
[(0, 1), (2, 3), (4, 2)}
Reference Answer [00, 4), (2, 1), (3, 4), (0, 0), (4, 2), (3, 0), (1, 3))
Futoshiki
Question
Image
Your task is to recognize the grid and inequality constraints from the image, solve the puzzle, and provide the answer in structured format:
Futoshiki Puzzle (4x4)
4
3
Game Rules:
1. The puzzle a NxN grid (e.g., 5x5).
2. Fill each cell with number from N.
3
3. Each number must appear exactly once in each
row and each column (no repetition).
4. Inequality symbols between cells (either '<' or
'>') must be satisfied:
3
A horizontal constraint (i.j) < (i.++) means the left
cell must be less than the right.
A
A vertical constraint (i.j) < (i+1.j) means the top
3
1
cell must be less than the bottom.
Answer format:
Category: Puzzle
Output the final solution 2D list of integers.
answer: [[rowl], [row2],
....
Difficulty: Level
[rowN]
Reference Answer
[[4. 2, 1, 3], [1, 3, 4, 2), (2, 1.3, 4), [3, 4. 1]]
30

---

Published as a conference paper at ICLR 2026
Eulero
Question
Image
Your task is solve the Eulero puzzle, based on the rules and the current puzzle state shown below.
Goal: Fill all empty cells such that the following
rules are satisfied:
A
Global Rules:
1. Each cell contains letter-number pair (like A Each letter appears exactly once every row and every column.
B
Each number appears exactly once every row and every column.
4. Each letter-number pair is unique across the entire grid (i.e., no duplicate pairs anywhere).
5. For an NxN grid, the letters used are the first N letters of the alphabet (A=1, B=2, .. up to the N-th letter), and the numbers used are from to N.
2
A
Current Puzzle State: The puzzle displayed the image below: 1. Some cells are pre-filled with letter-numbe pairs. Blank cells are empty and must filled in.
Category: Puzzle
Difficulty: Level
Output Format:
Each row should be represented as single line of letter-number pairs, separated by (without spaces). Each row must be line using An to separate them.
For example:
AIIB2IC3
B3IC1AA2
C2IA3IB1
Answer Format: Please provide the letter-number pairs in the format as shown in the example above.
Reference Answer
B3IA1IC2
A2IC3IBI
C1IB2IA3
31

---

Published as a conference paper at ICLR 2026
Bridges
Question
Image
Please look carefully at the image showing a Bridges puzzle (Hashiwokakero), In this puzzle, you need to connect all numbered "islands" using horizontal/ver- tical bridges.
Bridges Puzzl
Game Rules:
Each island displays a number indicating how many bridges must connect to it Bridges can only run horizontally or vertically between islands
3. Bridges cannot cross other bridges or islands
4. At most bridges connect any pair of islands
5. All islands must form a single connected network Coordinate system:
'The grid uses (x.y) coordinates starting from (0,0) the top-left corner X increases from left to right,
Y increases from top to bottom
Answer Format:
Category: Puzzle
Provide your solution with each bridge connection in the format: (x1.yy))+(2.,22xccommm
Difficulty: Level 4
For example: (0.4)-(2,4): (2.1)-(2.4): (2,4)-(4,4):
Reference Answer
(1,1)-(15):1
(1,1)-(4,1)ı
(4.1)-(4.2):
(4.1)-(7,1)
(4,2)-4455):2
(4.5)-44.6):
Campsite
Question
Image
Solve this Campsite puzzle by placing tents adjacent trees while adhering to the game rules.
0
Game Rules:
Each tent must be orthogonally adjacent least one tree (up, down, left, or right). No tents be adjacent each other, even diago- nally.
1
1
4
0
The number of tents in each row and column must match the given constraints.
Coordinate System:
2
Return the coordinates where tents should be placed as a list [row, column] pairs using 1-based indexing (e.g.. top-left is [1,1]).
0
0
0
2
0
1
Answer Format:
[[1, 3], [3, 1), [4, 3]]
Category: Puzzle
Reference Answer [[2, 5]. [3, 3], [5, 3]. [5, 6]]
Difficulty: Level
32

---

Published as a conference paper at ICLR 2026
Sudoku
Question
Image
Your task is solve the 9x9 Sudoku puzzle according to the rules and current state below:
Game Rules:
1. Fill the grid so that each row contains the digits
8
2
1-9 exactly once.
2. Each column must contain the digits 1-9 exactly once.
3. Each 3x3 subgrid (rows 1-3/4-6/7-9) columns
1-3/4-6/7-9) must contain the digits 1-9 exactly
once. 4. Do not alter any given digits from the puzzle. Coordinate System:
7
-The grid is indexed left-to-right and top-to-bottom..
Columns: numbered 1-9 from left to right. Rows: numbered 1-9 from top bottom.
Current Puzzle State:
The puzzle solve is shown in the grid below: Given digits in the grid are fixed and cannot be changed. represents empty cells.
Category: Puzzle
Difficulty: Level
Sudoku Grid:
2
5
3
9
0
4
8
1
07
1
3
2
0
6
4
9
1
7
8
3
2
8
0
0
5
4
3
6
8
0
8
5
2
0
2
0
5
4
0
6
7
4
9
2
8
2
0
Answer Format:
Please output the fully solved grid as 81 integers in row-major order, separated by single spaces. The output should look like:
5
1
4
8
3
9
9
7
5
8
4
8
3
9
6
7
1
3
9
5
1
2
7
8
6
6
9
5
2
7
2
8
6
4
3
9
5
3
5
7
4
2
4
6
3
8
5
7
2
3.
Reference Answer
2
5
3
9
6
4
3
2
6
6
4
9
1
7
8
3
2
8
9
2
5
4
3
6
8
9
8
5
2
3
2
6
5
7
4
6
7
4
9
2
8
7
2
8
33

---

Published as a conference paper at ICLR 2026
Binairo
Question
Image
Please examine the grid carefully. The grid shows
Binairo puzzle grid with Os and Is. Empty cells need be filled.
0
1
Rules:
Fill the grid with and Is
2. Each row and column must contain exactly 2 Os and
1
1
3. No three consecutive identical digits in any
column
1
0
4. All rows must be unique and all columns must be unique
Coordinate System:
Rows are numbered 1 to from top to bottom Columns numbered 1 to from left to right Solve the 4x4 Binairo puzzle shown in the image and provide your complete solution.
1
0
Category: Puzzle
Output Format:
Your answer must be formatted as grid of Os and Is separated by spaces, with rows separated by newlines. For example:
Difficulty: Level
01 1
1010
0101
1001
Reference Answer
0101
1001
1010
0110
34

---

Published as a conference paper at ICLR 2026
Minesweeper
Question
Image
Your task is to solve the Minesweeper puzzle according to the rules and the current state below:
Game Rules:
1. Minesweeper is played on a grid where some cells
1
1
contain hidden mines.
2. Numbers the grid represent how many mines are adjacent to that cell (including diagonally).
2
1
3. A cell with number means it has adjacent mines (this is represented as a blank cell).
2
2
1
1
4. The goal is identify the location of all mines without detonating any.
1
1
2
1
You can mark cell as containing a mine you're certain based on logical deduction.
1
1
1
Current Minesweeper State:
Category: Puzzle
The current state of the Minesweeper puzzle is shown in the image.
Difficulty: Level 2
Output Format Requirements:
Your final answer should list all mine locations using O-based coordinates in the format (row,col).
Example answer format: (0.))...7).(1)))11..2..)
Reference Answer (2, 0),3, 1),(4, 3)
24 Points
Image
Question
Use these numbers exactly once, and combine them with +, -, X, +, and parentheses make 24. Please provide your answer only numbers, operators, and parentheses. Example answer format: (9 3) 8 2.
24 POINTS PUZZLE
expression that includes
1
6
7
9
Reference Answer (1 7) (9 6)
Category: Algorithm
Difficulty: Level
35

---

Published as a conference paper at ICLR 2026
Best Time To Buy And Sell Stock
Image
Question
Given bar chart of stock prices over time and each bar's height is the price on that day. You can buy and sell as much as you want, but can only hold one stock a time. Calculate the maximum profit you can get from this transaction. If you cannot get any profit, answer Please provide your answer as an integer.
Reference Answer
Category: Algorithm
Difficulty: Level
Trapping Rain Water
Question
Image
Here a bunch of bars lined up side by side, where the width of each bar is 1 and consecutive bars adjacent with no gaps between them. Compute how much water can trap after raining. Please provide your answer as an integer.
Reference Answer
45
Category: Algorithm
Difficulty: Level
H Index
Image
Question
Here is bar chart showing how many times each of a researcher's papers was cited. Determine the researcher's ' h-index: the largest value h such that at least h papers have at least h citations each. Please provide your answer as integer.
Reference Answer
5
Category: Algorithm
Difficulty: Level
36

---

Published as a conference paper at ICLR 2026
Largest Rectangle In Histogram
Image
Question
Here a histogram made of bars where each unit wide and packed tightly together. What's the biggest rectangle you can fit entirely inside the histogram? Please provide your answer as integer.
Reference Answer
20
Category: Algorithm
Difficulty: Level
Longest Increasing Subsequence
Image
Question
Here row of bars each with some height. Pick subset of these bars where each one is strictly taller than the last and they appear order from left to right. What's the longest such sequence you can find? Please provide your answer as an integer.
Reference Answer
3
Category: Algorithm
Difficulty: Level
Container With Most Water
Question
Image
Given of vertical bars where consecutive bars are adjacent with no gaps between them. Pick any two bars and form the sides of water container, with the x-axis as the base. How much water can the biggest possible container hold? Please provide your answer an integer.
Reference Answer
18
Category: Algorithm
Difficulty: Level
37

---

Published as a conference paper at ICLR 2026
Count Hills And Valleys
Image
Question
Here is a terrain made of bars. Hill: A flat or raised area where the land right before it is lower, and the land right after it is lower too. Valley: A flat or dipped area where the land right before is higher, and the land right after is higher too. Neighboring hars with the same height count as part of the same hill/valley. Calculate the number of hills and valleys. Please provide your answer as an integer.
Reference Answer
3
Category: Algorithm
Difficulty: Level
Calcudoku
Image
Question
This is 3x3 Calcudoku puzzle. Each and col- umn must contain the numbers to 3 exactly once. The grid is divided into regions, each with get number and a specified operation. The num- bers within each region must be combined using the given operation to achieve the target number. Please solve the puzzle and provide the solution as a two- dimensional array. Example answer format: [[1, 2, 3, 4), [4, 3, 2, 1J. (2, 1, 4, 3], [3, 4, 1, 2]].
Calaudeuu Puzzle
Category: Puzzle
Reference Answer
Difficulty: Level
[[3, I, 2], [2, 3, 1J, [1, 2, 3]]
CryptoMath
Image
Question
Solve this CryptoMath puzzle, where each letter rep- resents unique digit (0-9). Different letters must correspond to different values, and no leading letter can be zero. Please provide your answer as a list of comma-separated "letter" =number pairs. Example answer format: ["A"=5, "B"=3,... "Z"=9].
BBE BDD
ACBE
Reference Answer
Category: Algorithm
["A"=1, "B"=6, "C"=2, "D"=0, "E"=5]
Difficulty: Level
38

---

Published as a conference paper at ICLR 2026
Kukurasu
Image
Question
This is 3x3 Kukurasu puzzle. You need fill the grid with black cells according to the following rules: 1. The of column positions (1 of black cells in each row must equal the number on the right. 2. The sum of row positions each column must equal the number at the bottom. Please solve the puzzle and provide the solution as two-dimensional array, using for white cells and I for blackcells. Example answer format: [[I, 1, 0j, [1, 0, 11. [0. O, 1]].
to 3) of black cells in
Category: Puzzle
Reference Answer 0[ O, 1J, [1, 1, 0]. [1, 0, O]]
Difficulty: Level
Wordladder
Image
Question
This is a Word Ladder puzzle. Transform the left word into right word by changing one letter at time, ensuring that each step forms a valid word. The rules follows 1. Change exactly 2. Each step must form a valid English word. Please provide the complete solution path from 'how' to 'met' as a list of strings. Example answer format: ["hug", "bug", "beg", "bet", "set").
Word Ladder Puzzle
HOW I
letter time.
MET
Reference Answer
Category: Puzzle
["how", "hot", "got", "get", "met"]
Difficulty: Level
Skyserapers
Image
Question
Arrange skyscrapers of heights 1-3 on this 3x3 grid. The rules are as follows: Each row and column must contain exactly one of each height (1 to 3). The numbers around the grid indicate how many skyscrapers visible when looking from that di- rection, with taller buildings obscuring shorter ones behind them. Please provide your answer as two- dimensional list. Example answer format: [[1, 2, 3, 4], [4, 3, 2, 1], [2, 1, 4, 3], [3, 4, 2]].
Skyscrapers Puzzle
1
3
X
X
X
3
X
X
X
2
X
X
X
Reference Answer [[3, 2, 1], [1, 2], [2. 3]]
2
Category: Puzzle
Difficulty: Level
39

---

Published as a conference paper at ICLR 2026
Eulerian Cycle
Image
Question
Given the undirected, connected graph below, deter- mine there an Eulerian cycle. If exists, output the cycle list (e.g., [0,1,233,0]). If not, output 'No'.
Reference Answer
[0. 7, 6, 3, 7, 5, 2, 0]
Category: Graph
Difficulty: Level
Eulerian Path
Image
Question
Given the undirected, connected graph below, deter- mine if there is an Eulerian path. If it exists, output the path as list (e.g., [0,1,2.3]). If not, output 'No'.
Reference Answer
[0, 5, 4, 3, 0, 1,
Category: Graph
Difficulty: Level
Graph Isomorphism
Image G1
Image G2
40

---

Published as a conference paper at ICLR 2026
Question
Given two connected undirected planar graphs GI and G2 shown below, determine if they are isomorphic by analyzing their planar structure. Answer with "Yes" or 'No'.
Reference Answer
Yes
Hamiltonian Cycle
Image
Question
Given the undirected, connected graph below, de- termine if there is Hamiltonian cycle. If it exists, output the cycle as a list (e.g., [0.1,2,3]). If not, out- put 'No'.
Reference Answer [0. 1, 5, 4, 2, 3]
Category: Graph
Difficulty: Level
Hamiltonian Path
Image
Question
Given undirected graph below, determine whether a Hamiltonian path starting from vertex 2 (marked in red) exists. If exists, output the path list (e.g.. [0,1,2,3]). If not, output 'No'.
Reference Answer [2. 0, 1,
Category: Graph
Difficulty: Level
41

---

Published as a conference paper at ICLR 2026
Max Flow
Image
Question
Below is layered directed acyclic graph (DAG) with capacities on each edge. Compute the maximum flow from node node 3. Answer with the maximum flow value (an integer).
Reference Answer
M
14
Category: Graph
Difficulty: Level
Shortest Distance
Image
Question
Given a weighted undirected graph below, what is the shortest distance from node 0 node 5? Answer with number (can integer or decimal).
Reference Answer
10
Category: Graph
Difficulty: Level
Topological Sort
Image
Question
Given the directed acyclic graph (DAG) shown in the image below, please list ONE possible valid topo- logical orders. Answer with a list of numbers. For example: [0, 1, 2, 3].
Reference Answer [4. 5, 2, 0, 3]
Category: Graph
Difficulty: Level
42

---

Published as a conference paper at ICLR 2026
A.8
EXAMPLES MM- HEL--EELIIII 100K
Nibbles
Image
Question
You are a puzzle solver focusing on Snake puzzles.
Game Rules: 1. Control snake to move around the grid using directional commands (up, down, left, right) The snake must eat all apples on the grid to win When the snake eats an apple, it grows longer by one segment 4. The snake cannot collide with walls or itself 5. The snake moves one cell time the chosen direction
Input: An image showing the initial state with the snake and apples
Goal: Find sequence of directional moves to eat all apples without the snake colliding with walls itself.
Output Format Requirements: Your answer should be a sequence of directional moves separated by spaces. Valid moves are: up, down, left, right Example: up right down left up
Rule-based CoT
I need solve this Snake puzzle carefully analyzing the image and applying strategic reasoning. Let me work through this step by step.
### Step 1: Understanding the Game Rules and Mechanics
Let me first clarify the fundamental rules of this Snake puzzle:
* **Core Movement Rules:** The snake moves one cell at time in four directions: up. down, left, right The snake cannot immediately reverse direction (e.g., if moving right, cannot move left next) Each move, the snake's head advances the next cell in the chosen direction
**Collision Rules:** The snake dies if it hits any wall (moves outside the grid boundaries). The snake dies if its head collides with any part of its own body These are the only two failure conditions
**Apple and Growth Mechanics: There are 3 apple(s) placed on the 8x8 grid When the snake's head reaches apple's position, the apple is consumed Upon eating apple, the snake's 's length increases by exactly segment The tail doesn't move for one turn when an apple is eaten (snake grows)
**Victory Condition: **rimary Goal:* Consume all 3 apple(s) without any collisions The puzzle solved when no apples remain on the board
**Strategic Implications: As the snake grows longer, navigation becomes more constrained
Path planning must account for future snake length Early moves must preserve space for later maneuvers
### Step 2: Careful Image Analysis and Initial State Reading
43

---

Published as a conference paper at ICLR 2026
Now I will read the image carefully to extract the precise initial game state:
**Grid Layout Analysis:** Grid dimensions: rows x 8 columns Total cells available: 64 Coordinate system: (row, col) with (0,0) top-left
** Visual Grid Representation: ** ...
...A... ..S.... ........
Legend: H Snake Head, S Snake Body, A Apple, = Empty Cell
**Snake Initial Configuration:
Snake body positions: [(4, 2). (3, 2)] Snake length:
segments Head position: (4, Current facing direction: down Tail position: (3,
** **Apple Locations Analysis:** Apple positions: [(2, 3), (6, 4), (5, 7)] Number of apples to
collect: 3 Apples sorted by Manhattan distance from head: [(2, 3), (6, 4). (5. 7)]
**Distance Analysis:*** Apple at (2, 3): 3 moves minimum (Manhattan distance) Apple at (6,
4): moves minimum (Manhattan distance) Apple at (5, 7): moves minimum (Manhattan distance)
*taae Reading Verification: Let me double-check my reading the initial state: /Grid size correctly identified as 8x8 / Snake head at (4, 2) facing down v Snake body has segments total v Found all apples on the grid /No overlapping positions between snake and apples ** **Spatial Constraints: Available empty cells: 59 Grid boundaries: rows [0, 71, columns [0,
7] Movement constraints: cannot reverse direction, cannot hit walls or self
### Step 3: Strategic Exploration and Detailed Reasoning Process
Now I'll develop comprehensive strategy to solve this puzzle through systematic exploration: **Initial Strategic Assessment:***
** "*Apple-by-Apple Strategic Analysis:**
" **Target 1: Apple (2, 3)** Manhattan distance: moves
Direct path analysis: Need to move -2 vertically, 1 horizontally Path considerations: Check if snake body blocks direct routes Evaluate alternative paths if direct route blocked # Consider space requirements for future growth Ensure tail clearance for return paths
**Target 2: Apple at (6, 4)** Manhattan distance: moves Direct path analysis: Need to move 2 vertically, 2 horizontally
**aarget 3: Apple at (5, 7)** Manhattan distance: moves Direct path analysis: Need to move 1 vertically, 5 horizontally
*Move-by-Move Strategic Reasoning:**
Let trace through the solution logic step by step:
**Vov I: right** Current head: (4, 2) + New head: (4, 3) Direction change: down - right No apple at (4, 3), snake maintains length 2 Safety check: position (4, 3) is within bounds and collision free
** **oove 2: up** Current head: (4, 3) - New head: (3, 3) Direction change: right - up No apple at (3, 3), snake maintains length 2 Safety check: position (3, 3) is within bounds and
collision-
**Move 3: up** Current head: (3, 3) + New head: (2, 3) Direction change: up up
V Apple consumed at (2, 3)! Snake grows to length 3 Safety check: position (2, 3) is within bounds and collision-free
**Move 4: right** Current head: (2, 3) + New head: (2, 4) Direction change: up - right No apple at (2, 4), snake maintains length Safety check: position (2, 4) is within bounds and collision-free
**Move down** * Current head: (2, 4) + New head: (3, 4) Direction change: right - down No apple at (3, 4). snake maintains length 3 Safety check: position (3, 4) is within bounds and collision-free
"*Move 6: down** ** Current head: (3, 4) -> New head: (4, 4) Direction change: down - down - No apple at (4, 4), snake maintains length 3 Safety check: position (4, 4) is within bounds and collision-free
**Move 7: d Current head: (4, 4) - New head: (5, 4) Direction change: down - down No apple at (5, 4), snake maintains length 3 Safety check: position (5, 4) is within bounds and collision-free
**Move 8: down** Current head: (5, 4) New head: (6, 4) Direction change: down
down V Apple consumed at (6, 4)! Snake grows to length 4 Safety check: position (6, 4) is within bounds and collision-free (continuing for 4 more moves)
44

---

Published as a conference paper at ICLR 2026
**Strategic Heuristies Applied:** **Nearest-First Strategy:*s Prioritize closest apples to minimize travel distance #*Space Preservation:** Avoid moves that create dead-ends or trap the snake **Growth Planning:** ** Account for snake length increase when planning paths Boundary Awareness: ** Use grid edges strategically to guide movement **Tail Management: Ensure the tail has clearance for complex maneuvers Alternative Path Consideration:
direct path to nearest apple is blocked, consider perimeter approach Evaluate trade-offs between immediate apple collection vs. strategic positioning Plan escape routes before entering confined spaces
** Risk *ssessment:** Monitor available space as snake grows longer Avoid creating situations where the snake blocks its own path to remaining apples Maintain flexibility for direction changes throughout the solution
### Step 4: Solution Validation and Comprehensive Reflection
Now I'll thoroughly validate the solution and reflect on the reasoning process: "*Solution Completeness Verification:** / VAll apples successfully collected / Solution contains 12 moves v Snake length progression: 2 - (growth 3)
**Collision Safety Verification: Let verify each move is safe: v All moves verified safe
no wall or self-collisions detected Final snake length: (expected: 5) / Remaining apples: (expected: 0)
**Strategic Reflection:** The solution demonstrates effective application of nearest-firs.
strategy Space management was crucial for avoiding self-trapping scenarios Growth planning ensured the snake maintained maneuverability throughout
** Algorithm Efficiency Analysis:** Solution efficiency: moves for 3 apples Average moves per apple: 4.0 Path optimality: Solution balances directness with safety constraints ** **iinal Answer Confidence:*** Based on thorough analysis and verification, the solution is: *iight up up right down down down down right up right right*
This solution successfully guides the snake to collect all apples while avoiding all collision risks. The strategic approach demonstrates systematic planning and careful execution of the Snake puzzle mechanics.
MM-HELLX--100
<think> Alright, let's dive into this Snake puzzle. I'm looking at the grid and trying to get a feel for the layout not just the positions, but the "space", the flow, how the snake might move through as it grows. This isn't 't just about getting from point A to B; it's about doing so in a way that doesn't I paint the snake into a corner later.
First, let's orient ourselves. The grid is 8x8, and from what can see, the snake starts with its head (4,2), and there's 's a body segment just above at (3,2). So it's pointing downward. There are three apples: one up at (2,3), another down at (6,4), and one way over to the right at (5,7). Hmm. The closest apple (2,3) - just two rows up and one column to the right from the head. That feels like natural first target. But I can't just rush up the body is directly above the head, so if go up immediately. I'd be moving into where the body just was. Wait, no the snake moves forward, and the tail follows. So long I don't try to reverse direction, should be fine. But right now, the snake facing down. If go uup*, that's a 180-degree turn which isn't allowed. So can't 't go up first. That's key constraint.
So, options from (4,2): can't 't go down (would hit the wall? No - row 4 to is fine), but down the current direction. Wait that would be reversing.
*can* continue down, go left or right. But up, because
Wait- actually, the rule you can't reverse into your own body. Since the body is (3,2), above the head, moving up would immediately collide with the body. So yes, up is invalid. So first move must be either right, left, or down.
Left from (4,2) is (4,1) - empty. Right is (4.3) - also empty. Down is (5,2) also clear. So three options.
Now, the apple at (2,3) is the closest, but can't go straight up. So I need to approach it from different angle, or create space to maneuver.
Maybe I can go right first, then up. That might work. Let's try that in my mind.
Start: head at (4.2), body at (3,2).
Move 1: **ighh > head to (4,3). Body shifts now body is at (4.2), tail at (3,2) but now moves to (4,2)? Wait no
when the head moves, the entire snake shifts forward: the head
45

---

Published as a conference paper at ICLR 2026
moves to (4.3), the previous head (4.2) becomes the new body segment, and the old body (3,2) disappears? No - actually, in Snake, the tail moves forward too unless you cat an apple. correction: in standard Snake, when you move, the head advances, and the tail retracts 80 the body follows the path. Only when you eat an apple does the tail *not* retract, so the snake grows.
So after moving right: head at (4.3), body at (4,2), tail at (3,2)? No - the body should be the previous head position, and the tail moves up.
Let's track it properly:
Initial: head (4.2), body (3,2) so the snake occupies (3,2) and (4,2), with head at bottom.
Move *#right**: head goes to (4,3). The body follows: the segment that was at (4,2) is now the body, and the tail that was at (3,2) moves to (4,2). So now snake is: head (4,3), body (4,2), tail (4,2)? No two segments: head and one body. After move, positions are: (4.3) and (4,2). The old body (3,2) is gone - moved down to (4,2)? No.
Actually, in two-segment snake, when the head moves, the body takes the head's old position, and the tail (which is the body) leaves its old position.
So: start - positions: (3,2) and (4,2), with (4,2) as head.
After **right**: head moves to (4,3). The body moves to where the head was: (4,2). So now snake is at (4,2) and (4,3), head at (4,3), body at (4,2). The old tail (3,2) is no longer part of the snake.
Yes. that makes sense.
Now, from (4,3), can I go * To (3,3). Is that safe? (3.3) is empty no apple, no wall. And I'm not reversing
was going right, now up ap- that's a 90-degr turn, allowed. So yes. Move 2: **up** + head to (3,3), body to (4,3), tail to (4.2)? No: body was at (4.2), now moves to (4.3)? Let's track:
After move I: head (4,3), body (4.2)
Move 2: head to (3,3), body takes (4.3), so now snake is (4,2), (4.3), (3,3)? No - only two segments. After move, positions: head (3,3), body (4.3). The old body (4.2) is now gone the snake shifted.
Wait no: when the head moves (3,3), the body moves the head's previous position (4.3), and the tail (which was at (4,2)) is left behind. So yes, snake is now at (4.3) and (3,3). Now, from (3,3), can go **p*** to (2,3) and there's 's an apple! And I'm not reversing was going up, continue up - that's fine. Move 3: **up** ** to (2.3) - eat the apple! Snake grows. So now, when the head moves to (2,3), the body follows to (3,3), and the tail (which was at (4.3)) moves to (3,3)? No - because it grew, the tail doesn't retract. So the body segments are: new head (2,3), then (3,3), then (4.3). So now length is 3.
Great one apple down. Now, where to next? The remaining apples are at (6,4) and (5.7). (6,4) is closer from (2.3) to (6,4) is 4 down, right - Manhattan distance 5. (5,7) is farther.
But I need to get down there. From (2.3), I' 'm facing up, but I just moved up. can go left, right, or down- - not up (would be reversing if were going up, but I just came from down? Wait my last move was **up** I' m facing up. So can go left, right, or up but not down (reverse).
But want to go down eventually. So need to turn left or right first. Right to (2.4) That's 's empty. Let's try **right** Move 4: **gght to (2,4). Now facing right. Body: (2,3), (3,3). Tail still at (4.3)? Let's see: after eating, snake was at (2,3), (3,3), (4.3). After moving head to (2,4), body follows: (2.3) - (2.4) No - head moves to (2,4), the first body segment moves to (2,3), the second moves to (3,3), and the tail (4.3) moves to (3,3)? No -o-wo with growth, when you move #after* eating, the tail *does* move - only during the eat move does the tail not retract.
Clarify: when you eat an apple, that move causes the snake to grow so the tail does *not* move that turn. But on subsequent moves, the tail moves normally.
So after move 3 (eating), snake occupies: (2,3), (3,3), (4,3) three segments.
Now move 4: **right** - head to (2,4). The body shifts: (2,3) + (2,4y? No the head moves to (2.4), the segment that was (2,3) moves where the head was but the head was at (2,3)? No - head was at (2,3), moves to (2,4). So the first body segment (at (3,3)) moves to (2,3), and the tail (at (4.3)) moves to (3.3). So now snake (2,4), (2.3), (3,3).
46

---

Published as a conference paper at ICLR 2026
Yes.
Now, want to go down toward (6,4). From (2,4), can go **down** that's allowed, since
I'm facing right, down is a turn.
Move 5: ** to (3,4). Is (3,4) safe? Yes, empty. Snake now: head (3,4), body (2,4), (2,3)?
No after move: head (3,4), first body (2,4), second body (2,3)? No the body segments
shift: (2,4) - (3,4)2 No.
Head at (2,4), moves (3,4). The segment that was at (2.3) moves (2,4), and the one
(3.3) moves to (2,3). So now: (3,4), (2,4), (2,3).
Wait - the snake was at (2.4), (2.3), (3,3). After moving head to (3,4), the body follows: (2,4) becomes the new body position, but it moves to where the head was (2,4)? I'm getting tangled.
Standard rule: each segment moves to the position of the one ahead of it.
So: Head moves from (2,4) (3,4) The first body segment (at (2,3)) moves to (2,4) The second body segment (at (3,3)) moves to (2,3)
So snake now occupies: (3,4), (2,4), (2,3)
(3,3) is now empty.
Now, continue * **down** to (4.4)
Move 6: ** **own*** head to (4,4). Then body: (3,4) (4,4)? No head (3,4) - (4,4), body
(2,4) - (3,4), tail (2.3) - (2,4). So snake: (4,4), (3,4), (2,4)
Still safe.
Move 7: down + to (5,4). Body: (4.4) - (5,4)? Head (4,4) + (5,4), body (3,4) - (4,4),
tail (2,4) + (3,4). So (5,4), (4.4), (3.4)
Move 8: **down + to (6,4). And there's an apple! Eat it. Snake grows to length 4. So now, after eating: head at (6,4), and the body follows: previous head (5,4) becomes body, (4,4) becomes next, (3,4) becomes tail but since it grew, the tail doesn't 't retract. So segments: (6,4), (5.4), (4,4), (3,4) Now, apple left at (5,7). From (6,4), need to go right - but I'm facing down (last move was down), so can go left, right, or down - not up. (5,7) is row 5, column 7. I'm at row 6, column 4. So need to go up and right three. But can't go up directly- Move 9: **right"* * to (6,5). Now facing right. Snake: head (6,5), body (6,4), (5,4), (4,4) tail at (4,4)? Let's track: after eating, snake at (6,4), (5,4), (4,4), (3,4). Move right: head to (6,5), body to (6,4), then (5,4), then (4,4) tail moves to (5,4)? No each segment shifts forward: (6,4) - (6,5) No.
that would be reversing. So need to turn right first.
Head was (6,4), moves to (6.5). The first body segment (5,4) moves to (6,4), the next (4.4)
moves to (5,4), the tail (3,4) moves to (4,4). So now: (6,5), (6,4), (5.4). (4,4) Good.
Move 10: right - to (6,6). Head (6,6), body (6,5), (6,4), (5,4)
Move 11: *ight - to (6,7). Head (6,7), body (6,6), (6,5), (6,4)
Now, need to go to (5.7). So **up** Am allowed? I'm facing right, up is a 90-degree turn- yes.
Move 12: **
(5,7). Apple! Eat it. Snake grows to length 5.
Now all apples are collected. Let double-check for collisions:
All moves stayed within 0-7 rows and columns. No move went into a wall. Did the snake ever hit itself? Let's see the path:
After move 3: snake at (2,3), (3,3), (4.3) fine.
After move 4: (2.4). (2,3), (3,3) no overlap.
After move 5: (3.4). (2,4), (2,3)
all distinct.
After move 6: (4,4), (3,4), (2,4)
After move 7: (5,4), (4,4), (3.4)
After move 8: eat at (6,4) + (6.4), (5,4), (4,4), (3.4)
After move 9: (6.5), (6,4), (5,4), (4,4) (6,4) adjacent, not overlapping.
After move 10: (6,6), (6,5), (6,4), (5,4)
After move 11: (6,7), (6,6), (6,5), (6.4)
After move 12: (5,7), (6,7), (6,6), (6,5), (6,4) - all good.
No self-collisions, walls, all apples eaten.
47

---

Published as a conference paper at ICLR 2026
The path feels natural: start by going right to avoid reversing, then up to get the first apple. Then snake down the right side, collecting the second apple on the way down. Then move right across the bottom, and finally up the last apple.
I could have tried to go for (5.7) first, but that would have been longer and riskier. This way, kept the snake compact early on, and used the open space on the right move freely. So the final answer is: right up up right down down down down right up right right </think> Final Answer: right up up right down down down down right up right right
48

---

Published as a conference paper at ICLR 2026
A.9 COMPLETE BREAKDOWN RESULTS
This section provides the full breakdown results over the four task types of MM-HELIX.
A.9.1 BREAKDOWN RESULTS ON ALGORITHMMS CATEGORY
Table 8: Breakdown results on the Algorithms category.
BuySell Container
Crypto
HIndex
Rect
LIS
Rain
Proprictury Models
(OpenAl, 2025b)
96.7
96.7
93.3
Seed-1.5.VL.. (Gu0 al,, 2025h)
100.0
80.0
83.3
60.0
86.7
83.3
73.3
73.1
70.0
04-mini (OpenAl, 2025e)
86.7
10.0
36.7
66.7
63.3
40.0
Gemini-2.5-Phaa (Comanici etal,, 2025)
96.7
43.3
66.7
56.7
83.3
76.7
56.7
70.0
50.0
GPT-4.1 (OpenAl, 2025a)
63.3
46.7
56.7
16.7
26.7
60.0
33.3
43.3
53.3
GPT-4o (OpenA1, 2024)
10.0
30.0
23.3
0.0
30.0
23.3
33.3
20.0
Open Source Models
Intem=81-2441-AA205 (Bai al., 2025s)
86.7
80.0
70.0
83.3
46.7
66.7
83.3
43.3
GLMM-455V100BBB 228-Thinking (Team et all, 2025b)
56.7
16.7
40.0
3.3
23.3
23.3
33.3
53.3
13,3
2025a)
GLMM-..1-9B--hhinnig (Team et al,, 20256)
76.7
10.0
43.3
13.3
20.0
30.0
16.7
30.0
36.7
Qwen-2.5-VI-7228 (Rai 2025b)
13.3
20.0
26.7
43.3
6.7
30.0
10.0
Qwea--25-VLL3328 (Bai etall, 2025b)
33.3
26.7
16.7
16.7
3.3
26.7
10.0
IntemWVL3-788 (Zhu 2025)
46.7
20.0
20.0
6.7
6.7
10.0
10.0
10.0
0.0
V1.-381
2025)
26.7
10--oat-199--11--- 109---977--661 (Meta, 2025)
66.7
30.0
3.3
10.0
0.0
6.7
3,3
20,0
6.7
MiniCPM-V.4.588P nl., 20256)
53.3
6.7
20.0
13.3
30.0
13.3
33.1
3.3
QVQ-72B Preview (Teum, 2024)
76.7
20.0
26.7
3.3
20.0
3.3
33.3
6.7
Ovis2-34B (Lu al, 2024)
23.3
0.0
3.3
6.7
20.0
13.3
26.7
0.0
(Team, 2025)
Qwen-2.5-VL. VL-L-7BB (Bai et al., 20256)
10.0
0.0
6.7
0.0
0.9
10.0
3.3
23.3
0.0
VL3-8B (2ho
2025)
Ovis2-8B (Lu ot al. 2024)
13.3
0.0
0.0
0.0
0.0
10.0
0.0
6.7
0.0
WM-HELIX--7B. ww-HHIJX--7-Thinniing
56.7
30.0
46.7
40.0
10.0
46.7
26.7
43.3
13.3
The abbreviations used in the table above explained in the following table:
Table 9; Abbreviation list of the keywords in the Algorithms category.
Abbreviation
Task
24
24 Points
BuySell
Best Time to Buy and Sell Stock
Container
Container With Most Water
Hills
Counting Hills and Valleys CryptoMath H-Index
Crypt
HIndex
Rect
Largest Rectangle Histogram
LIS
Longest Increasing Subsequence
Rain
Trapping Rain Water
49

---

Published as a conference paper at ICLR 2026
A.9.2 BREAKDOWN RESULTS ON THE GRAPHS CATEGORY
Table 10: Breakdown results the Graphs category.
Model
EulerCye
EalerPath
Graphise
HamilCye
HamilPath
MaxHlow
ShortDist
TopeSiort
Proprictory Models
GPT5 (OpenAl, 2025b)
35.3
33.5
53.3
40.0
50.0
80.0
13.3
Seed-1,5-VL (. 20250)
30.4
23.3
(OpenA) 2025g
(Comanici 2005)
(OjenAl, 20253)
GPT-4o (OpmA1,20044)
26.7
16.7
Source Models
Iitem=S11241B-A2888 (Bti 202563
16.7
26.7
500
16.7
23.3
50.0
56.7
GLM-44.VV (0-.5VV------- A 12BB-TTinni Thinking (Team et al, 20036)
0.0
10.0
6.7
10.0
20.0
30.0
13.3
0.0
Kimi VL- 168 A38 Thnninn 2506 (leam et 2025g]
16.7
20.3
46.7
16.7
26.7
40.3
20.0
0.0
M.M-4.11-9B-hhikkinng (Team ot al, 20250)
16.7
23.3
46.7
16.7
33.1
50.0
41.3
3.1
2wen-255-VL-72D (Bai 2025b)
16.7
23.3
56.7
10.0
20.0
43.3
36.7
Owan-25-Vg.3288
IntemV1.3-788
2005)
10.0
IntemVL.3 2005)
Llmm----onms1199....
(Mlet,, 2025)
MimiCPM-Y-4.588B all 20256)
23.3
20,0
167
26.7
QVQ-72B Proview (leam, 2024)
16.7
16.7
36.7
6.7
13.3
20,3
20.0
Ovis2-148 (1.2 cial, 2004)
16.7
23.3
53.3
23.3
16.7
23.3
13.3
6.7
Gemma-3-27B-I7 (Team, 2025)
16.7
26.7
33.3
16.7
23.3
36.7
3000
3.3
2wm-22--V1-77B- (Bai al, 20256)
10.0
23.3
53.3
0.0
133
23.3
20.0
IntemVL3 (2bu 2025)
Ovis2-88
MM-HFI 1K-78-Thinking
The abbreviations used in the table above are explained the following table:
Table Abbreviation list of the keywords the Graphs category.
Abbreviation
Task
EulerCyc
Eulerian Cycle
EulerPath
Eulerian Path
GraphIso
Graph Isomorphism
HamilCyc
Hamiltonian Cycle
HamilPath
Hamiltonian Path
MaxFlow
Max Flow
ShortDist
Shortest Distance (Weighted)
TopoSort
Topological Sort
50

---

Published as a conference paper at ICLR 2026
A.9.3 BREAKDOWN RESULTS ON THE PUZZLES CATEGORY
Table 12: Breakdown results the Puzzles category (Part
Model
Aqua
Bina
Brid
Calen
Camp
Fule
Futo
Hito
Kaku
Kukku
Proprictary Models
GPT-5 (OpenAL, 2025b)
33.3
30.0
53.3
33.3
83.3
Seed-1.5-4 1.L-- (Guo et al, 20256)
10.0
30.0
50.0
16.7
86.7
60.0
20.0
40.0
36.7
63.3
04-mini (OpenAI, 2025c)
26.7
13.3
73.3
23.3
53.3
50.0
30.0
43.3
43.3
76.7
Gemini-2.55 F.5-Flahh (Comanici et al, 2025)
3.3
20.0
60.0
3.3
46.7
46.7
16.7
63.3
36.7
40.0
GPT-4.1 (OpenAJ, 2025g)
3.3
0.0
46.7
13.3
13.3
33.3
10.0
40.0
16.7
60.0
GPT-4o (OpenAl, 2024)
0.0
3.3
16.7
20.0
10.0
16.7
Open-Source Models
fneem-S1-241--2235 (Bai et 2025a)
3.3
23.3
60.00
26.7
20.0
16.7
20.0
20.0
30.0
0.0
GLM-1.5V-166BB4
M-N--06B-AA2B-Thieei 12B-Thinking (Team al., 2025b)
13.3
30.0
13.3
6.7
60.0
6.7
6.7
30.0
0.0
33.3
Kimi-VL-IGB-A5B-I1 -
(Team et 2025a)
6.7
13.3
26.7
13.3
GI.M.4.IV.9B Thinking (Team et 2025b)
6.7
16.7
6.7
13.3
40,0
10.0
3.3
16.7
13.3
20.0
Qwen-2.5.VL-72BB (Bai al,, 2025b)
Qwea-2.5-VL-322B (Bai et al, 2025b)
3,3
0.0
10.0
3.3
6.7
3.3
16.7
0.0
6.7
13.3
Intem\L337888 (Zhu
0.0
26.7
3.3
3.3
6.7
3.3
IntemV1.3-388 (Zhu al., 2025)
3.3
3.3
16.7
20.0
D.0
3.3
13.3
10.0
6.7
10.0
1a0e4--SSouu-00-AAA77---- (Meta, 2025)
6.7
10.0
13.3
3.3
30.0
16.7
3.3
23.3
10.0
3,3
MiniCPM-V445-8B
2025b)
6.7
20.0
QVQ-720 PeQ-r2e-Previeew (Team, 2024)
10.0
13.3
6.7
6.7
6.7
6.7
16.7
10.0
13.3
0.0
Gemma-3-27B-II (Team, 2025)
6.7
3,3
10.0
3.3
0.0
0.0
6.7
13.3
10.0
3,3
Qwen-2.5-VL-78 (Bai 2025b)
13.3
0.0
0.0
10.0
6.7
IntemV113 (Zhu al., 2025)
6.7
0.0
3,3
16.7
10.0
0.0
10.0
3.3
6.7
0.0
Ovis2-8B 2024)
0.0
10.0
0.0
6.7
3.3
10.0
3,3
Owrs
MM-HELIX-78 78- 7e-iiik-n-
3.3
23.3
6.7
46.7
43.3
20.0
13.3
30.0
26.7
Table
Breakdown results on the Puzzles category (Part 2).
Model
Nomo
Num
Shin
Sky
Snak
Sude
Tapa
WI.d
WSeh
Proprictary
Models
GPT-S (OpenAl, 2025b)
26.7
86.7
80.0
53.3
10.0
26.7
50.0
36.7
100.0
Seed-1.5- VL (Guo et al., 2025b)
3.3
6.7
70.0
40.0
100.0
50.0
33.3
20.0
60.0
mini (OpenAI, 2025c)
43.3
30.0
100.0
Gemini-2,5 2.5-in--,5--lashh (Comanici al,, 2025)
0.0
40.0
40.0
40.0
83.3
40.0
36.7
10.0
70.0
GPT-4.1 (OpenAl, 2025a)
0.0
16.7
53.3
20.0
43.3
3.3
10.0
33.3
GPT-4o (OpenAl, 2024)
0.0
0.0
6.7
3.3
33,3
13.3
13.3
Open Source Models
Intern-S-22411 A28B (Bai et 2025a)
0.0
26.7
13.3
23.3
53.3
53.3
16.7
0.0
43.3
GLM-4.5V106B-AI -iiiiiikinng (Team 2025b)
0.0
6.7
6.7
50.0
Kimi-V VLL-16--A31 Aai.--------Ttiiiin........ Saaiing-2500 2506 (Team et al., 2025a)
0.0
10.0
3,3
6.7
$0.0
10.0
0.0
0.0
26.7
G..M4.VV.RR.TTinie.. (Team
2025h)
0.0
Qwen-2.5-VLL7223 (Bai et 2025b)
0.0
6.7
16.7
3.3
13.3
6.7
0.0
0.0
10.0
Qwen 2.5 VL.32B (Bai et al, 2025b)
0.0
6.7
3.3
3.3
16.7
0.0
0.0
3.3
V1137m (Zhu 2025)
0.0
0.0
6.7
3.3
3.3
0.0
IntemVLL3-3BB (Zhu al., 2025)
0.0
0.0
3.3
3.3
3.3
0.0
0.0
0.0
3.3
Llama-4-Scout- 1e---109------BA (Meta, 2025)
20.0
3.3
6.7
16.7
MiniCPM V-4,5 8B (Yu al, 20256)
0.0
0.0
0.0
0.0
6.7
0.0
0.0
0.0
16.7
QVQ-72B-Previsse (Team, 2024)
0.0
0.0
6.7
0.0
6.7
10.0
Ovis2-34B (Lu et, al., 2024)
0.0
0.0
0.0
0.0
3.3
3.3
0.0
0.0
Gemma-3-277B. (Team, 2025)
0.0
0.0
3,3
0.0
0.0
0.0
0.0
10.0
Qwen-2.5-VL-77 (Bai al,, 2025b)
0.0
6.7
3.3
Intern 8B (7hu et al., 2025)
0.0
0.0
0.0
3.3
0.0
0.0
6.7
0.0
3.3
Ovis2-8B (L. ol,, 2024)
0.0
0.0
0.0
6.7
Ourx
MM-HELIX-7B. -Bi-iiikinn
13.3
23.3
60.0
20.0
16.7
30.0
6.7
6.7
40.0
The abbreviations used in the table above are explained the following table:
51

---

Published as a conference paper at ICLR 2026
Table 14: Abbreviation list of the keywords in the Puzzles category.
Abbreviation
Task
Aqua
Aquarium
Bina
Binairo
Brid
Bridges
Calcu
Calcudoku
Camp
Campsite
Fule
Fulero
Futo
Futoshiki
Hito
Hitori
Kaku
Kakuro
Kuku
Kukurasu
Nono
Nonogram
Num
Numbrix
Shin
Shingoki
Sky
Skyscrapers
Snak
Snake
Sudo
Sudoku
Tapa
Tapa
WLad
Word Ladder
WSeh
Wordsearch
52

---

Published as a conference paper at ICLR 2026
A.9.4 BREAKDOWN RESULTS ON THE GAMES CATEGORY
Table 15: Breakdown results the Games category.
Model
Maze
Mine
Nib
Slide
Soko
Hanoi
Proprietary Models
GPT-5-2025-08- ( (OpenAl, 20255)
10.0
23.3
10.0
86.7
16.7
93.3
) (Guo al,, 2025h)
6.7
53.3
20.0
63.3
3.3
53.3
04-mini(16kk (OpenAl, 2025c)
26.7
10.0
66.7
13.3
90.0
Gemini-2.5-flash 16K) (Comanici et al., 2025)
0.0
50.0
13.3
46.7
3.3
56.7
GPT-4.1-2025-04-144 (0334442022-) (OpenAI, 2025a)
3.3
0.0
0.0
3.3
0.0
46.7
GPT-4o (OpenAl, 2024)
0.0
0.0
0.0
3.3
0.0
36.7
Open-Source Models
Beem-S1-241B-A28B et al., 2025a)
0.0
36.7
33.3
L.M-4.5V-106B-A11 Thinking (Team et al., 2025b)
0.0
16.7
3.3
10.0
3.3
50.0
(Team et al., 2025a)
0.0
3.3
0.0
3.3
0.0
10.0
G..M4..VV-BB-TTinkiing (Team et al., 2025b)
0.0
0.0
3.3
3.3
0.0
26.7
Qwen-2.5-VL-72B (Bai et al., 2025b)
0.0
20.0
0.0
36.7
3.3
26.7
Qwen-2.5-VLL-22B (Bai et al., 2025b)
16.7
33.3
IntemVL3378B (Zhu al., 2025)
0.0
3.3
6.7
16.7
IntemVL3-38B (Zhu et al., 2025)
0.0
3.3
3.3
10.0
6.7
13.3
Lamma--Scouu-119B-AA1 1BB116 (Meta, 2025)
0.0
3.3
0.0
10.0
0.0
33.3
MiniCPM-VV4.5-8B (Yu al., 2025b)
0.0
0.0
0.0
3.3
3.3
13.3
QVQ-72B-Previee (Team, 2024)
16.7
Ovis2-34B ct 2024)
0.0
3.3
6.7
-C2------77BITT (Team, 2025)
0.0
0.0
0.0
3.3
0.0
3.3
Qwen-2.5-VL-7B (Bai et al., 2025b)
0.0
0.0
0.0
0.0
0.0
3.3
InternVL3-SB (Zhu et al., 2025)
0.0
0.0
0.0
0.0
0.0
6.7
Ovis2-8B (Lu al., 2024)
0.0
0.0
0.0
3.3
3.3
Ours
MM-HELIX-P7 -B-LIX---B-hhaeee Thinking
3.3
16.7
23.3
26.7
3.3
26.7
The abbreviations used in the table above are explained the following table:
Table 16: Abbreviation list of the keywords in the Games category.
Abbreviation
Task
Maze Mine
Maze
Minesweeper
Nib
Nibbles
Slide
Sliding Puzzle
Soko
Sokoban
Hanoi
Tower of Hanoi
53


Reviews:
[{"review_id": "Ay8Pj0pESn", "reviewer": "Reviewer_3idK", "summary": "This paper addresses the deficiency of current Multimodal Large Language Models (MLLMs) in long-chain reflective reasoning. The authors make three main contributions: (1) MM-HELIX, a benchmark consisting of 1,260 samples across 42 challenging tasks designed to evaluate multimodal reflective reasoning capabilities; (2) MM-HELIX-100K, a large-scale dataset of 100k high-quality reflective Chain-of-Thought (CoT) traces generated via the Step-Elicited Response Generation (SERG) pipeline; and (3) Adaptive Hybrid Policy Optimization (AHPO), a novel training algorithm that dynamically integrates off-policy expert guidance with on-policy exploration. When applied to Qwen2.5-VL-7B, their method achieves +18.6% accuracy improvement on MM-HELIX and +5.7% average performance gain on general mathematics and logic tasks, demonstrating effective learning and generalization of reflective reasoning capabilities.", "questions": "- Can you ablate training on MM-HELIX-100K only vs. MMK12 only to understand each dataset's contribution?\n- Can AHPO work with smaller base models (e.g., 1B-3B parameters)?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Novel focus on end-to-end multimodal reflective reasoning, going beyond existing benchmarks that use multiple-choice formats\n- Rigorous benchmark construction with automated verification ensuring correctness\n- Well-structured paper with clear methodology sections\n- Practical improvements on both in-domain and out-of-domain tasks", "weaknesses": "- Missing analysis of under what conditions AHPO would fail or succeed\n- Limited error analysis showing what types of mistakes the model makes\n- Training mix includes both MM-HELIX-100K and MMK12; unclear how much each contributes\n- No analysis of diversity in generated CoT traces (might all be similar)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jnJBunfrEk", "reviewer": "Reviewer_cRVP", "summary": "This paper addresses the weakness of MLLMs in long-chain reflective reasoning. It introduces MM-HELIX, a benchmark of 42 complex multimodal tasks and 1260 samples across difficulty levels. The authors also build MM-HELIX-100K, a dataset of 100k verified reflective reasoning traces generated via the SERG pipeline. They also propose AHPO, a training algorithm that adaptively balances expert supervision and reinforcement learning to improve reasoning stability and exploration.", "questions": "See weaknesses", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. The paper presents a substantial amount of work, including both a benchmark and a dataset, as well as a proposed improvement method. It makes a meaningful contribution to research on MLLM reasoning tasks.\n2. Based on the proposed dataset and methodology, the paper achieves significant improvements across various visual reasoning benchmarks. The proposed AHPO effectively alleviates the issue of insufficient high-quality rollouts when the model faces challenging tasks. Compared to previous hybrid method LUFFY, AHPO demonstrates a considerable performance gain.", "weaknesses": "My main concern is that the paper lacks many important implementation details, including those related to the benchmark, dataset, and ablation studies of the proposed method. These missing parts may confuse readers and make the work difficult to reproduce.\n\nBenchmark:\n1. What are the text and image settings during evaluation, and how are image-based tasks converted into textual form?\n2. What are the criteria for difficulty categorization? The paper only briefly mentions this. Is the classification done manually?\n3. What are the implementation details of the automated verifier?\n\nDataset:\n1. The paper lacks detailed information on data construction, including prompts and heuristic rules.\n2. There is no statistical breakdown of the proportions of different task types within the dataset.\n\nExperiments and Analysis:\n1. In Section A.8.2, the improvement on Graph-type tasks is smaller than on other types, but there is no accompanying analysis.\n2. The base model used for training should be explicitly stated in the experimental setup, even if readers can infer it from Table 2.\n3. In Figures 6 and 7, why is only static-AHPO compared with other algorithms, while Figure 7 also includes a non-static version(AHPO). Does static-AHPO mean the activation coefficient = 0?\n4. In Appendix A.2 (Training Setting), why are only 22k samples used for SFT instead of the full dataset? Is there any comparison showing the relationship between data volume and performance, and how were these 22k samples selected?\n5. Why is math data included during the RL stage? Was the AHPO model in Table 2 also trained with math data? If so, do benchmarks such as MathVision and MathVerse fall into the in-domain category? If that’s the case, does the claimed generalization ability still hold? Also, is there any comparison between models trained with and without math data?\n6. [Minor] The paper lacks ablation studies on different success rate thresholds and activation coefficient.\n7. [Minor] Is there any comparison showing how the CoT length in MM-HELIX-100K affects reasoning performance?\n\nTypo:\nLine 305–306: “following()”\n\nOthers:\n[Minor] I suggest that the authors move Sections A.5–A.7 to the end of the appendix, so that readers don’t have to scroll extensively or risk overlooking the subsequent content. Alternatively, at the very least, an appendix table of contents should be added.\n\nIf the authors address most of my concerns, I would be willing to raise my score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u8TmguzK6F", "reviewer": "Reviewer_onUL", "summary": "The paper introduces MM-HELIX, a multimodal benchmark of 42 tasks (algorithms, graphs, puzzles, games) with 1,260 procedurally generated instances designed to test long-chain reflective reasoning in MLLMs. It further proposes a Step-Elicited Response Generation (SERG) pipeline to build MM-HELIX-100K, a dataset of ~100k verified reasoning traces, and a training algorithm Adaptive Hybrid Policy Optimization (AHPO) that adaptively mixes off-policy supervision with on-policy exploration via a reward-gated coefficient. On MM-HELIX, state-of-the-art models underperform (e.g., strong open-source models <35% acc.; large proprietary models ≤58.1% multimodal), while AHPO on Qwen2.5-VL-7B improves accuracy by +18.6 points and yields +5.7 points average gains on external math/logic benchmarks. The work argues that reflective reasoning in MLLMs is learnable and transferable.", "questions": "- I'd love to see some abaltion on the success rate threshold and how would that impact the perfomrance\n- For generation pipeline, Did you evaluate sample efficiency when using fewer than 22k traces? Is there a particular reason for the \"22k\" number? Would be interesting to see the scaling behavior of the generated samples (i.e., if 5k SERG vs. 5k rule-based, how would the performance comparison look like).", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Well-posed evaluation target. MM-HELIX isolates long-chain reflective reasoning with verifiable tasks, hierarchical difficulty, and balanced sampling (1,260 instances across 42 tasks), enabling controlled diagnosis\n- SERG uses rule-based scaffolds refined by a strong LLM, then filtered by verifiers to produce 100k reflective traces with far lower time/cost than naive rollouts and with better downstream utility\n- AHPO’s reward-gated mixing of off-policy expert loss and on-policy GRPO addresses reward sparsity and SFT-forgetting. The authors show the learning curves of AHPO and demonstrate consistent gains against different baseline algorithms (GRPO/LUFFY/SFT)", "weaknesses": "- Equation (6) describes ξ’s activation by a success-rate threshold \\hat{R}, but the exact estimator (win-dow size, grouping), threshold choice, and sensitivity are not fully ablated. Providing an ablation over \\hat{R} and these different parameters would be helpful to get a deeper understanding of the algorithm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BiIiCRbw4E", "reviewer": "Reviewer_EG1J", "summary": "This paper presents MM-HELIX, a novel multimodal benchmark designed to evaluate MLLMs on long-chain, reflective reasoning tasks that require iteration and backtracking. MM-HELIX consists of 42 diverse, programmatically generated tasks across four domains—algorithms, graphs, puzzles, and games—spanning five difficulty levels. Through evaluation on this benchmark, the authors identify key shortcomings in existing MLLMs and propose a training pipeline that features: 1. SERG for generating large-scale reflective chain-of-thought data, and 2. AHPO for integrating off-policy SFT and on-policy RL. Empirical results demonstrate significant improvements on both MM-HELIX and standard mathematical and logic benchmarks, highlighting strong generalizability.", "questions": "1. How are the difficulty levels in the benchmark determined? Is there a principled methodology or validation process?\n\n2. Performance Disparities Across Domains: Open-source models perform substantially worse on Games and Puzzles compared to Algorithms and Paths (Table 1 and Appendix). What are the failure patterns underlying these disparities? Are they primarily due to reasoning capability limitations, context length constraints, instruction-following difficulties, or other factors? A more fine-grained error analysis could provide valuable insights.\n\n3. In SERG, the initial step provides logically sound trajectories. However, beyond oracle trajectories that directly reach the correct solution, could trajectories that incorporate rethinking, reflection, or error handling be more valuable for developing the model's generalizable reasoning capabilities? At the second step, what specific prompts are used to transform rule-based trajectories into more natural, comprehensive reasoning processes with reflective steps? How is diversity in reasoning behaviors (mentioned above) controlled/encouraged during this refinement stage? \n\n4. Are on-policy and off-policy learning applied to the same data points or separate sets? How is the number of samples per training step determined? Are there ablation studies on key hyperparameters, such as the ratio of on-policy to off-policy samples or the success rate threshold?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "**1. Originality:** The work makes notable contributions through (1) the MM-HELIX benchmark targeting multimodal long reasoning, (2) the SERG pipeline combining rule-based anchors with LLM refinement for large-scale CoT data generation, and (3) AHPO's dynamic integration of off-policy SFT and on-policy RL to address reward sparsity and catastrophic forgetting.\n\n**2. Quality:** The technical execution is strong, with SERG demonstrating efficiency and quality, rigorous AHPO formulation (Section 3.3), and comprehensive experiments showing substantial improvements on MM-HELIX and generalization to math/logic benchmarks (Table 2). The ablations (Figures 6-7) effectively validate the design choices.\n\n**3. Clarity:** The paper is well-structured with clear problem formulation and effective visualizations (Figures 1-3) that illustrate the benchmark design and system architecture. Additionally, the motivation for training methods that elicit extended reasoning from open-source MLLMs is well established.", "weaknesses": "**1. Benchmark Synthetic Nature / Coverage:** Although MM-HELIX procedurally generates tasks and spans many categories, all tasks appear synthetic and rule-based (see Figure 2 and example in Figure 3). There is no clear discussion or experiment detailing how well the benchmark reflects the “real-world” complexity or ambiguity of practical reasoning. Similarly, the tasks' diversity outside logic/math settings is not addressed. This may limit the external validity of the results.\n\n**2. Limited Evaluation Metrics:** The evaluation in Table 1 and the Appendix relies almost exclusively on raw accuracy, with limited discussion of more nuanced aspects such as error typologies or qualitative analysis beyond completion rates. Given that the benchmark aims to probe \"reflective\" capabilities—a term used throughout the paper—a deeper analysis of response length, backtracking frequency, and self-correction behaviors would strengthen the evaluation and provide insights into how models engage in iterative reasoning.\n\n**3. Equation Clarity and Notational Issues:** Although the main equations in Section 3.3 are written out, some aspects (but not limited to) lack precision for reproducibility:\nEquation defining $N$ could be more clearly defined as number of roll-outs, and the use of $\\operatorname{CLIP}$ could be further detailed.\nThe loss unification assumes normalization factor $Z$, but this is undefined and never discussed again.\nThe citation in line 305 should be added.\nThe off-policy and on-policy learning is uncleared whether they are applied to the same training data or different data point.\n\n**4. Ablations Analysis Limited:**\nThere is also no ablation analysis on hyperparameters introduced in Equation 5, despite their potential importance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)