Visual Planning: Let's Think Only with Images
Authors: Yi Xu
Chengzu Li
Han Zhou
Xingchen Wan
Caiqi Zhang
Anna Korhonen
Ivan Vulić
Venue: ICLR 2026 Oral (2026)
Keywords: visual planning
Primary area: foundation or frontier models, including LLMs

Abstract: Recent advancements in Large Language Models (LLMs) and their multimodal extensions (MLLMs) have substantially enhanced machine reasoning across diverse tasks. However, these models predominantly rely on pure text as the medium for both expressing and structuring reasoning, even when visual information is present. In this work, we argue that language may not always be the most natural or effective modality for reasoning, particularly in tasks involving spatial and geometrical information. Motivated by this, we propose a new paradigm, Visual Planning, which enables planning through purely visual representations for these "vision-first'' tasks, as a supplementary channel to language-based reasoning. In this paradigm, planning is executed via sequences of images that encode step-by-step inference in the visual domain, akin to how humans sketch or visualize future actions. We introduce a novel reinforcement learning framework, Visual Planning via Reinforcement Learning (VPRL), empowered by GRPO for post-training large vision models, leading to substantial improvements in planning in a selection of representative visual navigation tasks, FrozenLake, Maze, and MiniBehavior. Our visual planning paradigm outperforms all other planning variants that conduct reasoning in the text-only space. Our results establish Visual Planning as a viable and promising supplement to language-based reasoning, opening new avenues for tasks that benefit from intuitive, image-based inference.

Published as a conference paper at ICLR 2026
VISUAL PLANNING: LET'S THINK ONLY WITH IMAGES
Yi Xu'
Chengzu Li
Han Zhou Xingchen Wan
Caiqi Zhang'
Anna Korhonen'
Ivan Vulié"
Language Technology Lab, University of Cambridge (yx465, c1917, h2416, cz391, a1k23, iv250) 3cam. ac. kingchenv@google..cmm
Google
ABSTRACT
Recent advancements in Large Language Models (LLMs) and their multimodal extensions (MLLMs) have substantially enhanced machine reasoning across diverse tasks. However, these models predominantly rely pure text the medium for both expressing and structuring reasoning, even when visual information is present. In this work, we argue that language may not always be the most natural or effective modality for reasoning, particularly geometrical information. Motivated by this, we propose new paradigm, Visual Plamning, which enables planning through purely visual representations for these "vision-firs"" tasks, supplementary channel to language- paradigm, planning is executed via sequences of images that encode step-by-step inference in the visual domain, akin to how humans sketch visualize future actions. We introduce novel reinforcement learning framework, Visual Planning via Reinforcement Learning (VPRL), empowered by GRPO for post-training large vision models, leading to substantial improvements in planning in a selection of representative visual navigation tasks, FROZENLAKE, MAZE, and MINIBEHAVIOR, Our visual planning paradigm outperforms all other planning variants that conduct reasoning the text-only space. Our results establish Visual Planning as viable and promising supplement to language- based reasoning. opening new avenues for tasks that benefit from intuitive, image- based inference. Code available at: ptttps://gite com www.///w.........ooo..oo..eo...e http/yyxxXvvviua www.niaaaaaaaaanna...a
tasks involving spatial and
reasoning. In this
INTRODUCTION
Large Language Models (LLMs) (Brown et al., 2020: Ouyang et al. 2022 Anil et al. 2023) have demonstrated strong capabilities in language understanding and generation, well as growing competence in complex reasoning, enabled by their chain-of-thought reasoning abilities (Wei et al.) 2022b). Building these advances, recent work extends LLMS to support multiple modalities, yielding so-called Multimodal Large Language Models (MLLMs) (Reid et al. 2024 Hurst et al.
2024): they incorporate visual embedded information at the input tackle broader spectrum of tasks, such as visual spatial reasoning (Liu et al. 2023 Li et al. 2024a) and navigation (Gu et al.
2022 Li et al. 2024b). However, despite their multimodal inputs, these methods perform reasoning
purely in the text format during inference, from captioning visual content (Hao et al. 2025) to
generating verbal rationales (Zhang et al. 2024b).
Building on this observation, argue that performing multimodal reasoning only in the text pathway may not always offer the most intuitive or effective strategy, particularly for tasks that depend heavily on visual information and/or are 'vision-first" by design. Indeed, recent results from multimodal
benchmarks (Roberts et al.) 2025 Li et al. 2024a Chen et al.) 2024 Cheng et al., 2025) offer
growing evidence that purely language-bait based reasoning falls short in certain domains, particularly those involving spatial, geometric, or physical dynamics (Zhang et al. 2025a Li et al.; (025a).. Such
reliance on grounding visual information into text before reasoning introduces a modality gap that hinders the model's ability to capture visual features and state transitions. This highlights a potential
"Equal contribution.

---

Published as a conference paper at ICLR 2026
Image
Direct Prompting
Input
Verbal Output
Multimodal Chain-of-Thought
Input
Verbal Thought
Verbal Thought
Verbal Output
Go straight to the crossing.
Then turn slightly left at the crossing.
To conclude: you should first go straight to
Visual Planning
Input
Visual Thought
Visual Thought
Visual Thought
Figure I: Comparison of reasoning paradigms. The traditional approaches (top and middle rows) generate verbose and inaccurate textual plan, while the Visual Planning paradigm (bottom row) predicts the next visual state directly, forming a pure image trajectory without language mediation.
shortcoming of current MLLMs: while they process image inputs, they do not naturally "think" in images. For instance, tasks such as planning route through a maze, designing the layout of room, or predicting the next state of mechanical system are often better served by visual representations, as verbal descriptions may be less effective and struggle accurately capture complex spatial reasoning relationships. These examples suggest a broader question, which we aim to tackle in this work: can models directly plan in non-verbal modalities, such as images, without being mediated by text?
Cognitive science also offers compelling motivation for this question (Moulton & Kosslyn 2009). Dual Coding Theory (Paivio 199T) proposes that human cognition operates through both verbal and nonverbal channels, each capable of independent representational and inferential processes. Recent work on MLLMs incorporates interleaved text and images as reasoning steps (Hu et al., [202]] Li 2025b Zhang et al., 2025b). However, they still remain fundamentally text-driven and rely on
et al. bose- visualizations as auxiliary information for reasoning traces, with reasoning still mainly embedded in verbal traces. For instance, Visual Sketchpad (Hu et al. (2024) employs external tools to generate sketches as visual aids, and MVoT (Li et al. 2025b) generates per-step visualizations from language-based actions but still reasons in text for decision-making As such, a truly visual-only reasoning paradigm that avoids any text-based reasoning proxies remains underexplored.
In this work, we propose new paradigm, Visual Plamning, where reasoning is structured as sequence of images, but without the mediation of language. To the best of our knowledge, this is the first attempt to investigate whether models can achieve planning purely through visual representations. Rather than generating textual rationales and answers, our approach produces step-by-step visualizations that encode planning or inference steps directly in images. As a pioneering exploration, it circumvents the modality mismatch that occurs when visual problems must be forced into explanations in verbal form, reinforces state transitions, and provides a new trackable interface for tasks like navigation (Li
et al. 2024a), and visual gooblem-solving (Hao et al. 2025 Zhang et al. 2026).
Specifically, we explore this paradigm using the Large Vision Model (LVM) (Bai et al. 2024) trained
exclusively on images and video frames with no textual data. This design choice removes potential confounders introduced by language- supervision and enables clean investigation of whether models can reason purely within the visual modality. Motivated by the success of reinforcement learning in acquiring reasoning capabilities within the language modality (Guo et al., 2025a) and its strong generalization performance (Chu et al. 2025), we propose Visual Planning via Reinforcement Learning (VPRL), a novel two-stage reinforcement learning framework empowered by GRPO (Shao) let al.] 2024) for visual planning. It involves a distinct initializing stage for encouraging the exploration of the policy model in the given environment, which is then followed by reinforcement learning with progress reward function.
2

---

Published as a conference paper at ICLR 2026
We validate the feasibility of our paradigms on brid-based navigation representative of spatial planning tasks, including MAZE (Ivanitskiy et al. (023), FROZENLAKE (Wu et al. 2024b), and MINIBEHAVIOR QJin et al. 2023), where one agent is requested to navigate target location successfully without violating environment constraints. Our experiments reveal that the visual planning paradigm substantially surpasses the traditional textual reasoning method by supervised fine-tuning (SFT), achieving 27% higher average exact-match rate. In addition to better performance, our novel method VPRL exhibits stronger generalization out-of-distribution scenarios than the SFT method in the visual planning paradigm (VPFT). To the best of our knowledge, we are the first to apply RL to image generation in the context of planning, with contributions as follows:
We propose new reasoning paradigm, Visual Planning, and validate the feasibility of visual reasoning without any use of text and language for reasoning.
We introduce VPRL, novel two-stage training framework that applies RL to achieve visual planning via sequential image generation.
We demonstrate empirically that VPRI. significantly outperforms the traditional textual reasoning paradigm and supervised baselines in visual spatial planning settings, achieving substantial gains in task performance and exhibiting improved generalization.
2
VISUAL PLANNING VIA REINFORCEMENT LEARNING
2.1 THE VISUAL PLANNING PARADIGM
The majority of prior visual reasoning benchmarks (Goyal et al. 2017 Akula et al. 2021 Yue et al. 2024) be and is typically tackled by grounding the visual information in the textual domain (Gurari et al. 2018 Peng et al. 2024 Zhang et al., 2024a), followed by a few steps of textual reasoning. However, once the visual content is mapped text (e.g., object names, attributes, or relations), the problem gets reduced to a language reasoning task, where the reasoning is carried out by the language model, even without reflecting any information from the visual modality.
Our visual planning paradigm is fundamentally different. It performs planning purely within the visual modality a holistic process, where the actions are not explicitly predicted but instead implicitly represented by transitions between visual states. We formally define visual planning as process of generating a sequence of intermediate images T =011.... 6n). where each ê; represents a visual state that together constitute a visual planing trajectory, given the input image vo- Specifically,
let Ty denote a generative vision model parameterized by 9. The visual planning trajectory Ý is generated autoregressively, where each intermediate visual state ú; is sampled conditioned the initial state and previously generated states:
6 ~ TT (vyluo, 02, ..., 14-1).
2.2 REINFORCEMENT LEARNING FOR LARGE VISION MODELS
Reinforcement learning (RL) has shown notable advantages in improving the generalization of autoregressive models by optimizing with sequence-level rewards beyond token-level supervision signals (Chu al. 2025). In autoregressive image generation, image is represented of visual tokens. Inspired by the success of RI. in language reasoning (Guo et al.] 2025a), we introduce an RL-based training framework for visual planning empowered by Group Relative Policy Optimization (GRPO) (Shao et al. (0024). It leverages the transitions between visual states to compute the reward signals while verifying the constraints from the environments. To enforce the policy model that generates valid actions with diverse exploration during the RL process, we then propose a novel two-stage reinforcement learning framework for visual planning. In Stage 1, we first apply supervised learning to initialize the policy model with random trajectories. Model's visual planning is then optimized by the RL training in Stage 2.
sequence
Stage 1: Policy Initialization. In this stage, initialize the model me by training it on random trajectories obtained by random walks in the environment. The goal here is to generate valid sequences of visual states and retain exploration capability in a 'simulated" environment. For training, each
3

---

Published as a conference paper at ICLR 2026
Palicy Update
Action
Move
Reward
a
(1)
Copt
Optimal
(H) {o 6+1/
) o
8OR
a
(2)
Cnap
Non- ootimall
Large Vision Model
03
Decode
P(us, (8)) (+1)
Parse
(G)
O'nv
Invalid
1/t
Compare Score
Optimal action
Image
valid actions
for GARD
Visual State Prediction
Image Pairs
Reference @ current state
An overview the proposed VPRL framework, illustrated with autoregressive large vision models for image generation in the context of visual navigation task. We train the visual policy model with GRPO, using the progress reward that encourages progressing actions and penalizes invalid actions, yielding goal-aligned visual planning.
Figure 2:
trajectory' T consists of a sequence of visual states (ro..... Un) From each trajectory, we extract - image pairs of the form (Uga: U++1), where Ugi represents the prefix sequence (to...., U:)- Given an input prefix uga. to prevent overfitting to the specific transition and encourage stochasticity. we randomly sample one candidate state 0:+1 from all possible valid next states the supervision target, and minimize the following loss function of visual planning via fine- tne-tuning (VPFT):
LVPFT (0) (B F(t1) log Te( (i++1 (gg))
Overall, the first stage serves warm-up for subsequent optimization, focusing producing visually coherent outputs and enhancing the generation quality.
Stage 2: Reinforcement Learning for Visual Planning. Building on Stage 1, where the model is initialized with random trajectories, acquires the effective exploration capability. This property essential for RL, it ensures coverage over all possible transitions and prevents collapse to suboptimal behaviors. Stage 2 then leverages this ability to simulate the outcomes of potential actions by generating the next visual state and guiding the model to effectively do the planning. During this stage, the RL algorithm provides feedback and rewards based on the correctness of the simulated actions, gradually enabling the model to learn effective visual planning.
Specifically, given an input prefix Ugi, the behavior model Toas samples group of G candidate responses
(6(1) ( ((()) The candidate response then scored using composite reward function r(u;, o(*)) ), which quantifies whether the generated visual state represents meaningful progress toward the goal state. The reward design and implementations are described in detail in the next paragraph.
Instead of relying on learned critic to estimate value functions which may introduce additional sources of uncertainty and complexity, GRPO provides more computationally efficient and inter- pretable training signals by computing relative advantages through comparisons within the group. In
this case, the relative advantage of each candidate A ,.()) mean (r(1) ((()) ,..,,...)} To guide the ( ,,() .(...
model toward producing responses with higher advantages, we update the policy by maximizing the following objective:
JVPRI. (0) E VgorD, (6 N
(3)
min (p(t) ((0), clip ( 1cc A(a1) -B DKL (Te (rrt)
No(0!0) (u51) is the importance sampling ratio. no (0(15) vg.)
where D is the prefix distribution and ((() =
4

---

Published as a conference paper at ICLR 2026
Reward Design. Unlike discrete actions text tokens, visual outputs sparse, nigh-dimensional, and not easily decomposable into interpretable units. In our visual planning framework, the challenge is even specific: whether the generated visual state correctly reflect the intended planning action. Consequently, our reward design emphasizes both adherence to environment constraints (validity of state transitions) and progress toward the goal.
Formally, let A denote the set of valid actions and & the set of invalid ones (e.g., violations of physical constraints hallucinated new entities in the environment). To interpret and evaluate the intended action that connects the current state V; to a generated candidate state a we introduce 1) the dynamics interpreter D V x V A & to parse the transition and 2) the progress estimator P: + N to quantify the progress.
The dynamics interpreter D evaluates the transitions E AUE for validity, which, by implementation, can be dynamics model (Qiu et al. 2025) or rule-based system to elicit actions from state pairs, neural model holistic validator that judges transitions without explicitly inferring actions. The progress estimator P(v) quantifies progress by estimating the remaining steps or effort required reach the goal from each visual state. By comparing the agent's current and predicted state, we partition the generated candidate states A U & into three disjoint subsets:
Aops {a A: P(O P(u.)}, Anopt {a E A: P() P(E)), ENsY = E.
Here, Aops corresponds to optimal actions that reduce the distance to the goal, Anopt captures non- optimal but still valid actions, and Env denotes invalid ones determined by the dynamics interpreter.
Based on this partition, we define the progress reward function r(v;, 0241):
Opopt Di;, ((+1) E Aopt] + Onoopt D(c, o2+1) E Anopt
+
Oine D(r,, 0/+1) E Eine],
(4)
optimal
non-optinal
invalid
where Copt, Cnopts Ciny are reward coefficients. In our experiments, we set Coopt = 1, Ckoopt = 0, and Oinv 5. thereby rewarding progressing actions, assigning zero non-progressing actions, and heavily penalizing invalid transitions.
3 EXPERIMENTS AND RESULTS
Tasks. To evaluate our proposed visual planning paradigm, we select representative tasks where planning can be expressed and executed entirely in the visual modality. We focus on tasks where state transitions are visually observable, distinguishing them from language-centric tasks like code generation (Lai et al., 2023) or traditional visual question answering. This design allows us to analyze planning behavior without relying textual rationales symbolic outputs. To compare visual planning with language- based reasoning, we experiment with visual navigation environments: FROZENLAKE (Wu et al. 2024b). MAZE (Ivanitskiy al. 2023). and MINIBEHAVIOR (Jin et al.) 2023). All of them can be solved in both modalities, which enables a direct parallel comparison of pros and cons between visual planning and language reasoning strategies.
FROZENLAKE: It is initially proposed by Wu et al. (0024)) and implemented with Gym (Brockman 2016). It simulates a grid-based frozen lake, where the agent is supposed to start from the designated position and find its way to the destination safely without falling into the 'holes'.
MAZE: Given initial image describing the maze layout, the model is supposed to go through the maze from the starting point (green point) to the destination (red flag).
MINIBEHAVIOR: The agent is first required to reach the printer from the starting point and pick it up. After that, the agent should go to the table and drop the printer. This task consists of additional actions, including 'pick' and 'drop'.
We construct synthetic datasets for the tasks with varying levels of complexity in patterns and environments. Details on data collection and implementation are provided in Appendix E.I
Models. To explore visual planning without any language influence as confounders and enables clean investigation, we select models trained exclusively visual data without any exposure
5

---

Published as a conference paper at ICLR 2026
Table 1: Performance of the closed- and open-source models FROZENLAKE, MAZE, and MINIBEHAVIOR, VPRL performs consistently the best (bold) across all tasks. denotes the post- trained model. A represents average performance across three tasks.
and represents images. The last column AvG, reports the
Model
Input
Output
FROZENLAKE
MAZE
MINIBEHAVIOR
AVG.
(8) (%)
(9)
(Sc
(S))
(5)
(%)
(9))
Closed-Source Model
Gemini 2.0 Flash
Direct
21.2
47.6
8.3
31.4
0.7
29.8
10.1
36.3
CoT
A
27.6
52.5
6.9
29.8
4.0
31.2
12.8
37.8
Gemini 2.5 Pro (think)
A
72.0
85.0
21.5
35.5
37.6
59.9
43.7
60.1
Open-Snurce Model
Qwen I.5tVL-Instrrutt-BB
Direct
A
1.2
15.0
0.6
14.5
0.3
9.8
0.7
13.1
CoT
A
8.2
29.1
2.3
15.2
0.5
14.7
3.7
19.7
SPT!
70.3
31.3
56.1
LVM-7B
VPFT' (ours)
75.4
79.5
39.0
64.0
33.8
52.2
56.
65.2
VVRL (ours)
74.5
77.6
75.8
textual data during pretraining. For visual planning, we use the Large Vision Model (LVM-7B) (Bai et al. 2024) as the backbone, which is only trained on image sequences and videos. train the model with 1) supervised fine-tuning over golden planning trajectory (VPFT) and 2) two-stage reinforcement learning (VPRL), resulting two system variants with visual planning. For RL training, start with rule-based parsing function as the dynamics interpreter to parse the image pairs to actions, and a heuristic progress estimator, with details enclosed in Appendix E.3
For baselines, to facilitate parallel comparison for language-based planning, adopt Qwen 2.5- VL-Instruct (Bai et al. 2025), on both inference-only (Direcfaad and CoTT and post-training settings (SFT and RL), trained on the same data the visual planner. We further evaluate multimodal reasoning performance of proprietary models with Gemini 2.0 Flash (Kampf & Brichtova (2025) and advanced thinking model Gemini 2.5 Pro (Gemini 2025). Full training details, model versions, and hyperparameters are provided in Appendix E.4
Evaluation Metries. We adopt two complementary evaluation metrics for the selected tasks. Let O (T(1), ( [MM) denote the set of all shortest optimal trajectories of length n, where each trajectory is T = (v) Un (mm)) and let 7 (0),.. i,) denote the predicted trajectory.
(m)
Exact Match (EM) is defined EM ma I I(0, -.()), evaluating whether 7 coincides with any (m) E O. EM requires the entire trajectory to be valid and of minimal length, and accepts all optimal solutions rather than single reference. Here, the equality i; m.fm)) refers to whether the two states can be reached from their respective previous states by applying the same action. This means that the comparison is made at the level of environment transitions rather than pixel-wise match between images. In other words, two states are treated the same if they represent the same underlying configuration, even when their pixel values are not identical.
Progress Rate (PR) is defined as PR maxme クラ=1 II I(θ2 = {mm) measur-
ing the ratio consecutive correct steps (valid forward moves) from the start that align with least one optimal trajectory. PR thus provides a softer signal than EM, capturing the model's ability make meaningful progress towards a full solution. The same state equality is applied in EM.
Textual planning falls short in both proprietary models and open-sourced tuning baselines. Table shows that proprietary models yield average EM below 50% and PR only marginally above 50% at best, underscoring the challenges these tasks pose for current models despite being intuitive for humans. the other hand, while task-specific training provides partial improvement, the overall
Direct denotes prediction without being instructed conduct intermediate reasoning.

---

Published as a conference paper at ICLR 2026
Optimal Action
Non Optimal
Lake
Invalid Action Move Through Wall
Non-Optimal Action
Maze
agent
Physical
MiniBehavior
Figure 3: Illustration of each task with generated visual planning traces from LVM, covering different types of actions (optimal, non-optimal and invalid). More cases can be found Appendix F.6
performance of fine-tuned textual planners remains unsatisfactory, through either directly generating planned actions (SFT in Table T or first captioning the image with different textual representations and then generating answers (Table 2). We also observe that, unlike the notable gains of RL in the pure language domain (Guo et al. 2025a). RL yields limited performance gains when applied to text-based planning with multimodal inputs. Table 2 shows that when using progress reward as in VPRI. or directly using the Progress Rate metric as the outcome reward, none of the variants surpasses the SFT baseline. We attribute the bottleneck of language-based planning with SFT and RL to the modality gap, which leads to inaccuracies in grounding visual information into text and thereby constrains performance. Further discussion is provided in Section
Visual planning achieves better performance than tex- tual baselines via RL. While supervised fine-tuning (VPFT) achieves performance comparable text-based SFT, it remains constrained by imitation and limited ex- posure diverse trajectories. By contrast, our two-stage reinforcement learning framework (VPRL) substantially improves the planning capability, achieving the strongest overall results. After Stage 2 optimization, the model at-
Table 2: Performance of text-based plan- ning variants on FROZENLAKE, See Ta- ble in Appendix F2 the full results.
Model
EM (%) PR (98)
Qwen 23-VL-Insttcct78 SFT
-iirect
68.6
84.4
-w/ Coondinates
74.4
82.7
-w/ASCU
73.1
83.4
tains near-perfect accuracy on FROZENLAKE (91.6% EM, 93.2% PR) and maintains strong performance more complex MAZE and MINIBEHAVIOR tasks, outperform- ing VPFT by over 20% average. As expected, the improvement is fully driven by outcome-based optimization in Stage 2, as Stage alone yields near-random behavior (Table i00 in Appendix [.6)). Unlike VPFT, which mainly fits the training distribution, VPRI. enables exploration of diverse actions and learning from their consequences through reward-driven updates, thereby capturing underlying planning rules and achieving stronger performance.
-GRPO
-w/ VPRL progress revard
54.4
69.9
reward
60.1
74,3
VPRL shows robustness with scaling complexity. The advantage RL also holds when we study the performance of different methods with respect to task difficulties, where larger grid usually relates to higher difficulties. In Figure 5 as the grid size increases from FROZENLAKE environment, Gemini 2.5 Pro's 's EM score drops sharply from 98.0% to 38.8%. In comparison, our visual planners not only maintain higher accuracy at all grid sizes but also exhibit much flatter performance curve. Similarly. VPRL demonstrates even greater stability than VPFT, with EM remaining at 97.6% on x 3 grids and still achieving 22.4% on x 6, indicating strong robustness. We observe similar trends in other tasks; see Appendix for other tasks.
to 6 x 6 in the

---

Published as a conference paper at ICLR 2026
Promptt You firs ansowr endiosed cAb cIANSAVER.
Gemini 2.5 Pro
Green 2.5 VL Instruct- SFT
-tibio Fome the picture a gri.. abel the and every ice hole, and plan Breadth Firt Search. t. ofter expanding every moceeble ced, the queue driins write now 2, of five holes blocle a upward rouses, (7000s words oniiteds iee-nsspet the contrrm the proot no va poth extes, vithink So the correct ruple un emply action fut. -ANEWEE--SMMEEE
loft up upright
Vioiation of Constrant
VPFT
VPRL
Figure 4: Visualization of a test example from FROZENLAKE comparing visual planning variants (VPFT and VPRL) with language-based reasoning variants.
DISCUSSIONS AND ANALYSIS
Error Analysis and Case Study. We conduct error analysis for language- planning and visual planning. We observe that textual planning systems with both SFT and RL are prone to errors when grounding visual inputs to verbalized descriptions during the inference process, with 25.7% of generated coordinate-based layout descriptions and 22.3% of generated ASCII-based representations being mismatched with ground-truth layouts. Qualitative analysis of response from textual RL baselines (Figure 9 in Appendix [22.I] and proprictary models (Figure 4 also reveal similar observations. Taken together, these results demonstrate inherent modality gap where language may not be the most accurate and effective representation for vision-first problem. For visual planning, Figure 3 presents visual planning traces generated by LVM across different tasks. We observe that the model occasionally takes non-optimal actions that deviate from the shortest path (FROZENLAKE example). Surprisingly, VPRL demonstrates the ability to take detours bypass the obstacles while still progressing towards the goal, whereas VPFT lacks this flexibility and gets stuck, as shown in Figure 4 Additional traces covering optimal, non-optimal, and invalid cases can be found in Appendix [.6 Beyond these in-domain analyses, we further evaluate generalization on larger unseen grids and perturbed image inputs, with results reported in Appendix F.4
Random policy initialization enables exploration. We ablate whether we could directly use VPFT as the policy model for GRPO training rather than intentionally initialize a model with random trajectories. We hypothesize that VPFT, trained via teacher-foreing, inherently limits exploration by repeatedly generating similar actions, resulting in identical rewards. In this case, it yields zero advantage, preventing policy updates and hindering effective learning. We empirically validate this hypothesis by comparing the exploration capabilities of VPFT with VPRL Stage 1 (Figure ()) We observe that VPFT's entropy rapidly declines throughout training, eventually approaching zero, indicating severe exploration limitations. Although earlier VPFT checkpoints exhibit higher entropy, they produce significantly more invalid actions. In contrast, VPRL Stage I demonstrates significantly higher entropy, closely approaching the entropy of the uniform random planner, while maintaining a lower invalid action ratio, justifying the necessity of Stage 1 random initialization for RL framework.
VPRL reduces invalid action failure. Another important benefit of VPRL lies in its effectiveness in reducing invalid actions. To quantify this, we analyze all failed trajectories and compute the proportion that contains at least one invalid action, as opposed to failures caused by non-optimal but valid plans. We refer to this as the mmalid-failure ratio. As shown in Table 6 VPFT exhibits a high ratio ranging from 61% to 78% over three tasks, while VPRL reduces this ratio by at least 24% in all cases, demonstrating that VPRL not only improves success rates, but also encourage the model to stay within valid action spaces during planning.
8

---

Published as a conference paper at ICLR 2026
VPRL (Stage
VPHL
1.0
(Tran)
Owen ISFTI
Gemasi (CUT)
VPFT [2 ep)
Average Entropy
0.5
0.8
Accuracy
0.6
VPFT (5 ep)
0.4
0.3
0.2
0.2
VVPFT (10
3
VPFT (30 ep)
Grid Size
Invalid Ratio (%)
Figure 5: Evaluation of model performance FROZENLAKE under varying levels of difficulty. As the environment complexity increases with larger grid sizes, language-basee reasoning meth- ods experience sharp decline in performance, whereas visual planning methods exhibit more gradual drop, demonstrating greater robustness.
Figure 6: Comparison of exploration capabilities between VPFT and VPRL Stage 1 on FROZEN- LAKE. VPRL Stage achieves significantly bet- ter exploration efficiency, balancing high entropy with a low invalid action ratio, whereas VPFT struggles with diminishing entropy and increased invalid actions over training.
RELATED WORK
MLLM Reasoning. Recent work has extended CoT prompting (Wei et al. 2022c) to MLLMs
through approaches such as grounding visual inputs into symbolic representations, such graphs or bounding boxes (Zhang et al., 2024b, Lei et al. 2024). Other approaches integrate tools to generate
visualizations during reasoning (Hu et al. 2024 Zhou et al. 2024). For example, o3 model (OpenAI
2025) incorporates visual rationales using tools such as cropping and zooming. MVoT (Li et al. 2025b) is also essentially form of tool use: instead of relying on external modules, it invokes itself to generate visualizations of textual reasoning, These methods primarily conduct reasoning in language, with visual components merely illustrating the textual rationale rather than serving as the medium of reasoning. In this work, we take a step further to explore whether multi-step planning can emerge purely within visual representations, enabling reasoning without relying on language all.
Reinforcement Learning for Visual Reasoning. Reinforcement learning has been applied across wide range vision-related tasks, especially given the rise of GRPO as in DeepSeek-RI (Guo et al.) 2025a Concurrently, in object detection, visual perception (Yu et al. rewarding high Intersection-over-UU
2025) is optimized though (IoU) scores between predicted and ground-truth bounding boxes (Shen et al.) 2025). For visual reasoning tasks such as Visual Question Answering (VQA), GRPO has been utilized to optimize the models for longer, more coherent, and logically grounded reasoning traces in textual responses (Liu et al.) 2025 Zhou et al. 2025 Zhhng et al., [2025c] Team ct al. 2025). More recently, similar methods have also been applied to image generation tasks for recursive refinement with textual instructions (Guo
2025b Wang al. 2025 Jiang al. (0255. These approaches focus on pixel-level fidelity and semantic alignment with text, whereas our work leverages RL for goal-oriented visual planning, optimizing multi-stey secision-making through visual state transitions without any reliance on language.
Action-condittonal Generative Models. Action-conditional generative models has focused on constructing latent representations of the world and predicting future observations conditioned on given actions (Ha & Schmidhuber 2018 Ball et al. 2025). These models learn transition dynamics and are central to model-based reinforcement learning, where they allow agents to simulate potential outcomes without interacting directly with the environment (Hafner et al. 2019). While effective for representation learning and short-horizon prediction, cction-conditional generative models do not perform planning and must therefore be coupled with an external planner. contrast, our approach constitutes a holistic planner that internalizes planning within the visual generative flow, which is more effective for visual tasks than traditional text-based planners that suffer from modality gap. It can also benefit from action- citiionnoniiiionall generative models by using predicted observations as inputs.

---

Published as a conference paper at ICLR 2026
6 CONCLUSION
In this work, present Visual Planning new paradigm for reasoning in visually oriented tasks, challenging the prevailing reliance on language as the primary medium for structured inference. By enabling models to operate entirely through visual state transitions without textual mediation, show that purely visual representations provide performance comparable to text- base-based planning in spatially grounded and dynamic tasks, establishing visual planning viable alternative. More importantly, our proposed two-stage reinforcement learning framework, VPRL, empowered by GRPO, further enhances the planning capabilities of large vision models. It obtains significant gains across three visual navigation tasks, achieving 27% EM improvements in task performance than language-basee planning and showing stronger generalization on out-of-distribution scenarios. These findings underscore the promise of visual planning powerful alternative to text- based approaches. We believe our work opens up a rich new direction for multimodal research, offering a foundation for building more intuitive, flexible, and powerful reasoning systems across wide range of domains.
ACKNOWLEDGEMENT
The work has been supported by the UK Research and Innovation (UKRI) Frontier Research Grant EP/Y031350/1 (the UK government's 's funding guarantee for ERC Advanced Grants) awarded to Anna Korhonen the University of Cambridge. The work has also been supported in part by Royal Society University Research Fellowship (no 221137; 2022-) awarded to Ivan Vulić, and by the UK EPSRC grant EP/T02450X/1.
REFERENCES
Arjun Akula, Soravit Changpinyo, Boqing Gong, Piyush Sharma, Song-Chun Zhu, and Radu Soricut. CrossVQA: Scalably generating benchmarks for systematically testing VQA generalization. In Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 2148 2168. Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 111886611222221 1/655///222...mmppmmmaa 2021. emnlp main main.1647
URL http:httts///cm http://aaaaatooooo.ooeoo org/
Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical re- port. arXiv preprint 20400 2023. URL
/ww.o/////.../////////////////// 48550/
2305. 10403
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint 139233, 2025.
Yutong Bai, Xinyang Geng, Karttikeya Mangalam, Amir Bar, Alan L. Yuille, Trevor Darrell, Jitendra Malik, and Alexei A. Efros. Sequential modeling enables scalable learning for large vision models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seatrle, WA, USA, June 16-22, 2024, pp. -28872, 2024. doi: 10. 111.11CVR52733.2224..00 https: doi htt.:/ii.ogggg/// 10. wtt.:11119///22222 /V/R///VR2523333 2024 02157.
URL
Philip J. Ball, Jakob Bauer, Frank Belletti, Bethanie Brownfield, Ariel Ephrat, Shlomi Fruchter, Agrim Gupta, Kristian Holsheimer, Aleksander Holynski, Jiri Hron, Christos Kaplanis. Marjorie Limont, Matt McGill, Yanko Oliveira, Jack Parker-Holder, Frank Perbet, Guy Scully, Jeremy Shar, Stephen Spencer, Omer Tov, Ruben Villegas, Emma Wang. Jessica Yung. Cip Baetu, Jordi Berbel, David Bridson, Jake Bruce, Gavin Buttimore, Sarah Chakera, Bilva Chandra, Paul Collins, Alex Cullum, Bogdan Damoc, Vibha Dasagi, Maxime Gazeau, Charles Gbadamosi. Woohyun Han, Ed Hirst, Ashyana Kachra, Lucie Kerley, Kristian Kjems, Eva Knoepfel, Vika Koriakin, Jessica Lo. Cong Lu, Zeb Mehring, Alex Moufarek, Henna Nandwani, Valeria Oliveira, Fabio Pardo, Jane Park, Andrew Pierson, Ben Poole, Helen Ran, Tim Salimans, Manuel Sanchez, Igor Saprykin, Amy Shen, Sailesh Sidhwani, Duncan Smith, Joe Stanton, Hamish Tomlinson, Dimple Vijaykumar, Luyu Wang, Piers Wingfield, Nat Wong, Keyang Xu, Christopher Yew, Nick Young, Vadim Zubov,
10

---

Published as a conference paper at ICLR 2026
Douglas Eck, Dumitru Erhan, Koray Kavukcuoglu, Demis Hassabis, Zoubin Gharamani, Raia Hadsell, Aäron van den Oord, Inbar Mosseri, Adrian Bolton, Satinder Singh, and Tim Rocktäschel. Genie 3: A new frontier for world models. 2025.
G Brockman. Openai gym. arXiv preprint ar 1rXxv16060001400 2016.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-Vone Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam Mc- Candlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few- shot learners. In Advances Neural Information Processing Systems 33: Annual Con- ference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL 645 49741 wwwbfaccbbcccccceceec A4a- html
ngs. neurips wttpiip...pppppppppee wctpppppr//22///aasesess
Qiguang Chen, Libo Qin, Jin Zhang, Zhi Chen, Xiao Xu, and Wanxiang Che. C A novel benchmark for multi-domain multi-step multi-modal chain-of-thought. In Lun-Wet Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 1199-2221, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.11865/11/2224.acll URL
org
o 20tp./2244...
long. 44
Zihui Cheng, Qiguang Chen, Jin Zhang, Hao Fei, Xiaocheng Feng, Wanxiang Che, Min Li, and Libo Qin. Comt: A novel benchmark for chain of multi-modee thought on large rision-languee models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 23678-23686,
2025.
Ethan Chem, Jiadi Su, Yan Ma, and Pengfei Liu. Anole: An open, autoregressive, native large multimodal models for interleaved image-text generation. arXiv preprint arXiv:2407.06133 2024.
Rohan Choudhury, Guanglei Zhu, Sihan Liu, Koichiro Niiuma,, Kris Kitani, and László Jeni. Don't look twice: Faster video transformers with run-length tokenization. Advances in Neural Information Processing Systems, 37:28127-28111 2024.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Sergey Levine, and Yi Ma. SFT memorizes, RL generalizes: A comparative study of foundation model post-training. In The Second Conference on Parsimony and Learning (Recent Spotlight Track), 2025. URL
net. https://cpenreve wttp:/eeeeeeeeteeeeaaaaa (tt/:/ormmmmdd3333
Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12873-12883, 2021.
Gemini.
Gemini 2.5:
Our most intelligent AI model.
March 2025.
URL
logy/oy///// ggogoooooolooooooooooooooooooooo leooogleedeemmmm nd/
ni www.-ooollle-e- ton- -areess--rrthh www-maarrhhhh--5 Accessed: 2025-05-09.
Yash Goyal, Tejas Khot, Douglas Summers- Stay, Dhruv Batra, and Devi Parikh. Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering. In Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Jing Gu, Eliana Stefani, Qi Wu, Jesse Thomason, and Xin Wang. Vision-and-language naviga- tion: A survey of tasks, methods, and future directions. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 7606-7623 Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. 1/2022..1112222l.l.11 https: //ccc///ccccc antho Logy. .rg/ 2022 acl acw.12..11111oon.... 524/]
URL
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseck-rl: Incentivizing reasoning capability in 1lms via reinforcement learning. arXiv preprint arXiv:2501. /2948, 2025a.
11

---

Published as a conference paper at ICLR 2026
Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Peng Gao, Hongsheng Li, and Pheng- Ann Heng. Can we generate images with cot? let's verify and reinforce image generation step by step. arXiv preprint 2rXiv:2501.33926, 2025b.
Danna Gurari, Qing Li, Abigale Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey Bigham. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1008-3617,, 2018.
David Ha and Jürgen Schmidhuber. World models. arXiv preprini arXiv: 10/22, 2(3), 2018.
Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In International conference on machine learning, pp. 2555-2565. PMLR, 2019.
Yunzhuo Hao, Jiawei Gu, Huichen Will Wang. Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. Can milms reason in multimodality? emma: An enhanced multimodal reasoning benchmark. arXiv preprint arXiv:2501. 005444, 2025.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth Interna- tional Conference on Learning Representations, ICLR 2022, Virtual Event. April 25-29, 2022. OpenReview.net, 2022. URL http::s:///peezzeiiiee http::/www.......o...oeoeeeeeeee
Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. Xiviv preprint arXiv:2406. 244,0030 2024.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Michael Igorevich Ivanitskiy, Rusheb Shah, Alex F Spies, Tilman Räuker, Dan Valentine, Can Rager, Lucia Quirke, Chris Mathwin, Guillaume Corlouer, Cecilia Diniz Behn, al. A configurable library for generating and manipulating maze datasets. arXiv preprint arXiv:2309. 10498, 2023.
Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan. Pheng-Ann Heng, and Hongsheng Li. T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot. arXiv preprint arXiv:2505.0070:3 2025.
Emily Jin, Jiaheng Hu, Zhuoyi Huang, Ruohan Zhang, Jiajun Wu, Li Fei-Fei, and Roberto Martín- Martín. Mini-BEHAN BEHAVIOR: A procedurally generated benchmark for long morghorizon decision- making in embodied AL. In NeurlPS 2023 Workshop Generalization in Planning, 2023. URL
htttps://oppnreeee ew net/ forum? did= 9pYaVh5)
Kat Kampf and Nicole Brichtova.
Experiment with gemini 2.0 flash native im- age generation, March 2025. URL http:////evvvloeereoooooooeeoooo com/ en/
experriment-with-geem 2aa-------aaa---at-e-ee-eeeeeee- 2025-04-27.
Accessed:
Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu. Ds-1000: A natural and reliable benchmark for data science code generation. In International Conference on Machine Learning, pp. 18319-18345. PMLR, 2023.
Xuanyu Lei, Zonghan Yang, Xinrui Chen, Peng Li, and Yang Liu. Scaffolding coordinates to promote Lision-language coordination large mudti-modal models. arXiv preprint 12005 2024.
Chengzu Li, Caiqi Zhang, Han Zhou, Nigel Collier, Anna Korhonen, and Ivan Vulić. TopWie:RS: Vision-language models top-view spatial reasoners. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natu- ral Language Processing, pp. 7786-1807, Miami, Florida, USA. November 2024a. Associa- tion for Computational Linguistics. doi: 10.1008... 10.11865//1/22/ 12202/. 106. URL https
7/aclantho ogy. ogy. 2024 emn pp ma n. 1067
12

---

Published as a conference paper at ICLR 2026
Chengzu Li, Chao Zhang, Simone Teufel, Rama Sanand Doddipatla, and Svetlana Stoyanchev. Semantic map-based generation of navigation instructions. In Nicoletta Calzolari, Min- Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue (eds.), Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and 2024), pp- 14628 14640, Torino, Italia, May 2024b. ELRA and ICCL.. URL httpss:/aeeaanthoooo..ooo/////// 200 1rec main. 1274/
Evaluation
Chengzu Li, Wenshan Wu, Huanyu Zhang, Qingtao Li, Zeyu Gao, Yan Xia, José Hernández-Oralllo Ivan Vulić, and Furu Wei. 1plus-bench: Demystifying multimodal IIm spatial reasoning with cognitive-inspired analysis. arXiv preprint arXiv:2508.20066 20068, 2025a.
Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, and Furu Wei. Imagine while reasoning in space: Multimodal visualization-oftthuuus arXiv preprint arXiv:2501.0754. 2025b.
Chengzu Li, Zanyi Wang, Jiaang Li, Yi Xu, Han Zhou, Huanyu Zhang, Ruichuan An, Dengyang Jiang, Zhaochong An, Ivan Vulić, et al. Thinking in frames: How visual context and test-time scaling empower video reasoning. arXiv preprint ar arXiv:2601.2102 2026.
Drew Linsley Junkyung Kim*, Alckh Ashok, and Thomas Serre. Recurrent neural circuits for contour detection. In International Conference Learning Representations, 2020. |
//openreview.wt net forum?
Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning. Transactions of the Association URL https
for Computational Linguisticc, 11:635665,, 2023, doi: 20.1162/tac_aa0000 a
//aclanthology wrg/ tacctalllae 377
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi
Wang. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.01788 2025.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. 7th International Conference on Learning Representations, ICLR 20/9, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
Samuel T. Moulton and Stephen M. Kosslyn. Imagining predictions: mental imagery as mental emulation. Philosophical Transactions of the Royal Society B: Biological Sciences, 364:1273 1280, 2009.
OpenAl. Introducing OpenAl o3 and 04-mini: Our smartest and most capable models date. April 2025. Utthhtpss: hwwpee/ii..co//neeee///oooooooio -nd 04 mini/ Accessed: 2025-05-16.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
Allan Paivio. Dual coding theory: Retrospect and current status. Canadian Journal of Psycholo- gy/Revue canadienne de psychologie, 45(3):255, 1991.
Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Qixiang Ye, and Furu Wei. Grounding multimodal large language models to the world. In The Twelfth International Conference on Learning Representations, 2024. URL
net/ ffrum?
id= [d=IImmxkfST
Yifu Qiu, Yftah Ziser, Anna Korhonen, Shay B Cohen, and Edoardo M Ponti. Bootstrapping world models from dynamics models in multimodal foundation models. arXiv preprint arXiv:2506.0600 2025.
Nikhila Ravi, Valentin Gabeur, Yuan-Tine Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint 24Xiv:2408.00714 007 14, 2024.
13

---

Published as a conference paper at ICLR 2026
Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy P. Lillicrap. Jean- Baptiste Alayrac, Radu Soricut, Angeliki Lazaridou. Orhan Firat, Julian Schrittwieser, loannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew M. Dai, Katie Millican, Ethan Dyer, Mia Glaese, Thibault Sottiaux, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, James Molloy, Jilin Chen, Michael Isard, Paul Barham, Tom Hennigan, Ross Mcllroy, Melvin Johnson, Johan Schalkwyk, Eli Collins, Eliza Rutherford, Erica Moreira, Kareem Ayoub, Megha Goel, Clemens Meyer, Gregory Thornton, Zhen Yang, Henryk Michalewski. Zaheer Abbas, Nathan Schucher, Ankesh Anand, Richard Ives, James Keeling, Karel Lenc, Salem Haykal, Siamak Shakeri, Pranav Shyam, Aakanksha Chowdhery, Roman Ring, Stephen Spencer, Eren Sezener, and et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. CoRR, htlps/240.0055000 2024, doi: 1044850/ARXI..4400.5555 URL https 7 2403. 05530
Jonathan Roberts, Mohammad Reza Taesiri, Ansh Sharma, Akash Gupta, Samuel Roberts, Ioana Croitoru, Simion-Vlas Vlad Bogolin, Jialu Tang, Florian Langer, Vyas Raina, et al. Zerobench: An impossible visual benchmark for contemporary large multimodal models. arXiv preprint arXiv:2502.09696, 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. ArXiv preprint, abs/2402.03300,, 2024. URLhttps
//arxiv.org/eee abs 0aaxxi...r//aaaa/22203000 2402.
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, al. Vlm-rl: A stable and generalizable rl-style large vision-language model. arXiv preprint arXiv:2504.0760 076/5, 2025.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with Ilms. arXiv preprint arXiv:2501. /2599, 2025.
Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quemtin Gallouédec. Trl: Transformer reinforcement learning. https: //g thub. www.coo///oooo www..cww..oh...... http:///wggggngcc/////oaaa fgface/tr) 2020.
Junke Wang. Zhi Tian, Xun Wang, Xinyu Zhang, Weilin Huang, Zuxuan Wu, and Yu-Gang Jiang. Simplear: Pushing the frontier of autoregressive visual generation through pretraining, sft, and rl. arXiv preprint arXiv:2504.1 / 1455, 2025.
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc Le. Finetuned language models are zero-shot learners. In International Conference on Learning Representations, 2022a. URL httph:ts////// //tp:/:////p//eeeeee/ net nwt/orrrummt
#d=gE2+GCozdqR
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi. Quoc V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022b. URL https: /ttp//pppeeiiiveeeoe//eeee net forum?id
ViOIMeSB J
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of- thoughh prompting clicits reasoning in large language models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 24824-24837. Curran Associates, Inc., 2022c.
Junfeng WU, Yi Jiang, Chuofan Ma, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, and Xiang Bai. Liquid: Language models are scalable multi-modal generators. arXiv preprint arXiv:2412.0433. 2024a.
Qiucheng Wu, Handong Zhao, Michael Saxon, Trung Bui, William Yang Wang. Yang Zhang, and Shiyu Chang. Vsp: Assessing the dual challenges of perception and reasoning in spatial planning tasks for vims, 2024b.
14

---

Published as a conference paper at ICLR 2026
En Yu, Kangheng Lin, Liang Zhao, Jisheng Yin, Yana Wei, Yuang Peng, Haoran Wei, Jianjian Sun, Chunrui Han, Zheng Ge, et al. Perception-rl: Pioneering perception policy with reinforcement learning. arXiv preprint arXtv:2504.07954, 2025.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal under- standing and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9556-95617 2024.
Huanyu Zhang, Chengzu Li, Wenshan Wu, Shaoguang Mao, Ivan Vulić, Zhang Zhang, Liang Wang, Tieniu Tan, Furu Wei, et al. call for new recipes to enhance spatial reasoning in milms. arXiv preprint arXiv22504.10037 2025a.
Huanyu Zhang, Wenshan Wu, Chengzu Li, Ning Shang, Yan Xia, Yangyu Huang, Yifan Zhang, Li Dong, Zhang Zhang, Liang Wang, et al. Latent sketchpad: Sketching visual thoughts to elicit multimodal reasoning in milms.. arXiv preprint 10.2:22500.24514 2025b.
Huanyu Zhang, Xuehai Bai, Chengzu Li, Chen Liang, Haochen Tian, Haodong Li, Ruichuan An, Yifan Zhang, Anna Korhonen, Zhang Zhang, et al. How well do models follow visual instructions? vibe: systematic benchmark for visual instruction-driven image editing. arXiv preprint wrXiv22602.01851, 2026.
Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024a
Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. RI-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arXiv:2503. 20..:.937... 2025c.
Zhuosheng Zhang, Aston Zhang, Mu Li, hai zhao, George Karypis, and Alex Smola. Multimodal Mfuin-of-though reasoning in language models. Transactions on Machine Learning Research, 2024b. ISSN 2835-8856.
Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1- zero's" aha moment" in visual reasoning on 2b non-sft model. arXiv preprint 1:Xiv:2503.05112, 2025.
Qiji Zhou, Ruochen Zhou, Zike Hu, Panzhong Lu, Siyang Gao, and Yue Zhang. Image-of-thougee prompting for visual reasoning refinement in multimodal large language models, 2024.
15

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
Large language models (LLMs) were used as general-purpos tools in this work. Specifically, LLMs assisted in polishing the writing improve clarity and readability.
B ETHICS STATEMENT
Our research adheres to rigorous ethical guidelines. We verified the licenses of all softwares and datasets we used in this study and ensured full compliance with their terms. Furthermore, we have thoroughly assessed the project and do not anticipate any additional potential risks.
C REPRODUCIBILITY STATEMENT
Appendix E.1 introduces the datasets details with statistics and processing procedure. Appendix E.2 introduces models used in paper, and Appendix [E.3 provides detailed information regarding reward implementation for VPRL method. All hyper-parameters and training details are listed in Appendix [E.4 for reproducibility. Appendix [E.5 introduces the licences for the data and models used. Prompting templates are shown in Appendix G All data and scripts will be released publicly upon acceptance to facilitate reproducibility.
D LIMITATIONS AND FUTURE WORK
In this work, focus exclusively Large Vision Model (LVM) to investigate visual planning capabilities by climinating language as a confounding factor for research purposes. As such, this choice constraints the model size 7B as the only available size of LVM, and excludes recently released native multimodal models capable of generating multimodal outputs (Chern et al. 2024 Wu et al. 2024a). However, argue that the visual planning paradigm can be extended broader multimodal generation models for use in more diverse tasks, combined with more modalities, as long they support image (visual) generation (Li al. 2026
Additionally, explicitly generating images introduces computational overhead during inference compared a textual response. However, we argue that language-based reasoning performs worse than visual planning and can be equally or even more time-consuming, especially for thinking models (Gemini 2025). In our demonstration, Gemini generated over 7,000 thinking tokens yet failed to provide the correct answer in the end. The computation overhead introduced by image generation can be alleviated through more compact image representations using fewer tokens (Choudhury et al., 2024), which we advocate for future research.
Another limitation in this work lies in the implementation of dynamics interpreter. For simplicity, we adopt the rule-based approach that compares pixel-wise features between the current state and the previous state (details in Appendix E.3). While effective in our controlled setup, broader task settings involving more complex visual structures are yet to be explored. Nevertheless, that the underlying reward formulation remains extensible, but the primary challenge lies in defining reliable progress signals as visual transitions become more complex. Such signals could be supported by either dynamic models that elicit actions from pairs of images (e.g. segmentation (Ravi et al. 2024) contour detection (Linsley* * et al. 2020)) holistic neural model (e.g. Gemini (Gemini 2025) or a learned reward model) that directly judges whether the transitions are valid without explicitly inferring actions. Alternatively, trajectory-level rollouts with final success feedback could be leveraged to identify actions that contribute to progress toward successful outcomes, eliminating the requirement for an explicit dynamics interpreter. We encourage future research to explore more robust and scalable designs for interpreting visual transitions to advance visual planning systems.
argue
Broader Impact. This work introduces a novel paradigm of visual planning, where agents reason and act entirely within the visual modality without reliance on textual intermediaries. By demonstrating that models can plan through sequences of images, this research opens new possibilities for the way human and AI system interacts, particularly in domains like robotics, navigation, and assistive technologies, where perception and decision decision-making are tightly coupled. As the first step toward planning grounded purely in visual representations, work lays the foundation for AI systems that
16

---

Published as a conference paper at ICLR 2026
integrate both verbal and non- verbal reasoning. We advocate for future research into more holistic multimodal thinking systems where interleaved text and image traces enable richer, more human- like reasoning, and emphasize the importance of strengthening the visual component in such traces for improved planning and cognition.
E IMPLEMENTATION DETAILS
E.1 DATASET
Task Action Space. FROZENLAKE and MAZE both involve four primitive navigation actions: up, down, left. and right. MINIBEHAVIOR includes a more complex action space with two additional operations: pick, drop.
Dataset preparation. For both FROZENLAKE and MAZE, we construct environments of grid sizes ranging from 3 x to 6 6. For each size, we sample 1250 environments, with 1000 used for training and 250 held out for testing (Table Each environment here guaranteed have a unique layout, and the agent is randomly initialized at grid from which the goal is reachable, forming the initial state uy. Due to the relatively limited diversity of environments layout in MINIBEHAVIOR, where the complexity arises primarily from the action space, sampling unique environments in small grid size becomes challenging. Therefore, we focus only on grid sizes x 7 and x 8, allowing duplicates in layout but varying agent spawn positions ensure sufficient data volume. To prevent data leakage, we split the dataset based on layout identity, ensuring no layout overlap between the training and test sets.
We next describe the dataset construction procedures corresponding to the training setups outlined in Section 3 with the number of samples per task summarized in Table
SFT Text (Baseline): For each environment, we sample an optimal trajectory consisting of a sequence of visual states (No...., Ux) the ground truth. Each transition between states is determined by an action, enabling us to derive a corresponding verbalized action sequence (ao..... 1n) The input to the model is formulated by concatenating textual prompt with an image representation of the initial state Uo. while the target output is the verbalized action sequence representing the optimal trajectory. We further ablate different variants of the baseline with various representations and tuning methods (SFT and RL) in Appendix F22 The detailed prompts for all variants are provided in Appendix(Gi
VPFT: We utilize the same set of optimal trajectories the language- reasoning baseline described above. In the visual scenario, each trajectory generates multiple input-target pairs by pairing the state at timestep & as the input with the subsequent state at timestep + 1 the target.
VPRL:
Stage 1: This dataset serves solely for format control training of the visual backbone. For each environment, we enumerate all possible trajectories from the initial state as un and generate corresponding input-target pairs. Duplicate pairs are filtered maintain balanced distribution.
Stage 2: To ensure faimess and comparability, this dataset uses the same input states VPFT.
VPFT":
We conduct an ablation study (indicated with ") where VPFT is also trained in two stages, mirroring the structure of VPRL. Stage follows the same procedure as VPRL Stage 1, focusing format supervision using enumerated visual inputs. Stage reuses the original VPFT training pipeline, learning from optimal trajectories. Experimental results and analysis see Appendix F.5.
Note: For both textual and visual planning setups, evaluation is performed using only the initial state Uo of each test environment as input.
Dataset Statistics. We evaluate the performance of different system variants in in-distribution and oft-of--dstribbi (OOD) settings. Table 3 shows the training data distribution over different grid sizes across three tasks. The numbers of training and testing samples for different system variants shown in Table 4 For OOD evaluation, the enlarged grid sizes are shown in Table 9 OOD evaluation data includes 250 samples for each task.
17

---

Published as a conference paper at ICLR 2026
Table 3: Distribution of training dataset by grid sizes for each task. Value indicates the number of environments.
FROZENLAKE
Grid Size
3
4
5
Train
1000
1000
1000
1000
Test
250
250
250
250
MAZE
Grid Size
3
4
5
Train
1000
1000
1000
1000
Test
250
250
250
250
MINIBEHAVIOR
Grid Size
8
Train
796
801
Test
204
199
Table 4: Number of training and test samples for each task and method. For visual planning. the numbers here are represented in image pairs, which correspond to the same number of trajectories for SFT in Text.
SFT Text
VPFT
VPRL
VPFT"
Task
Split
Stage
Stage 2
Stage
SFT
Train
4000
12806
170621
12806
170621
12806
FROZENLAKE
Test
1000
1000
N/A
1000
N/A
1000
Train
4000
14459
156682
14459
156682
14459
MAZE
Test
1000
1000
N/A
1000
N/A
1000
Train
1597
9174
90808
9174
90808
9174
MINIBEHAVIOR
Test
403
403
N/A
403
N/A
403
E.2 MODELS
Large Vision Model (LVM) (Bai al., 2024 autoregressive models for image generation, which is only pretrained with image sequences with no exposure to language data. The model uses tokenizer based on the VQGAN architecture (Esser et 202T), which extracts visual information from raw images and encodes it into 256 tokens from fixed codebook. The image is generated in an auto-regressive manner with discrete tokens, which are then fed into the image detokenizer. Although LVM supports multiple model sizes, only the 7B-pal (B-parameter version is publicly available; thus, use this variant in our experiments. For fair comparison, we use Qwen 2.5-VL-Instruct (Bai et al. with matching parameter size as language
2025
aasse based baseline.
E.3
REWARD IMPLEMENTATION
We adopt a rule-based state-action parsing function as the dynamics interpreter D and heuristic progress estimator P in VPRL. For the progress estimator, apply the Breadth First Search (BFS) to search for the optimal trajectories and calculate the progress at each position in the grid for each task, order obtain progress map covering all positions. The progress map are then used as reward signal to guide VPRI. training.
Specifically, for state-action parsing function, we parse the state and identify the difference between the current state and the previous state through pixel-wise feature extractor. We first convert both input and predicted states into a coordinate-basl based its size. Each region corresponds discrete coordinate in the environment. To reduce sensitivity to color and focus on structural differences, we convert all images to grayscale. We subsequently compute the intersection-over Union (IoU) between each coordinate in the predicted state and the coordinate in the input state that contains the player (imput coordinate). The coordinate in the predicted state with the highest loU is selected the predicted agent position. The action is
representation by dividing the image into grid
18

---

Published as a conference paper at ICLR 2026
Table 5: Hyper-parametery of training both textual and visual planners.
Hyper-Paramete
SFT Text
RL in Text
VPFT
VPRL
VPFT
Stage
Stage
Stage
SFT
Epochs
10
30
10
10
30
Learning Rate
le-5
le-5
1.5e-4
1.Se-4
Se-5
1.5e-4
1.5e-4
Train Batch Size
Group Size
N/A
8
N/A
N/A
10
N/A
N/A
Grad Accumulation
1
1
1
GPUs
8
8
8
8
then inferred by comparing the start and predicted positions according task-specifie movement rules, For example, in the MAZE environment, movement across walls is not allowed and would be considered invalid.
Notably, to detect the invalid transitions, such as the disappearance of agents, we also calculate the pixel-wise mean squared error (MSE) between corresponding coordinates measure local visual differences. If two coordinates exhibit significant MSE differences exceeding predefined threshold, we treat them as the potential source and destination of movement (agent disappears from one and appears in another). If only one such coordinate is found, we treat it as a disappearance event, indicating an invalid transition.
In MINIBEHAVIOR we extend this logic to identify pick and drop actions. A pick is detected when the IoU between the printer's location the input and predicted states falls below threshold, indicating that the printer has been removed. A drop is inferred when coordinate corresponding to the table region shows a large MSE increase, suggesting the printer has been placed there. Additional edge cases in these tasks omitted for brevity.
For reward computation, if the predicted action is valid, we compare the progress values from the heuristic progress estimator P between the input and predicted states. A reward of 1 is given if the predicted state shows greater progress toward the goal than the input state; otherwise, the reward is 0. Invalid actions are penalized with reward of -5.
Our method and reward modeling approach are readily generalizable to other visual tasks. With reference computer vision techniques such segmentation (Ravi et al. (0244) and contour detection (Linsley et al. 2020), the pixel-level analysis used in our framework can be easily extended to wide range of structured visual environments. Furthermore, our reward design broadly applicable to planning tasks in general. Since actions in most planning settings can naturally be categorized into one of three types (valid and helpful, valid but non- non-progressing. or invalid), our simple reward structure remains intuitive and effective across tasks.
E.4 TRAINING DETAILS
In addition to VPRL, we include several training system variants baselines that differ in supervision modalities (language vs. image) and optimization methods (SFT vs. RL), allowing us to compare language- based and rision-based planning while assessing the role of reinforcement learning.
Visual Planning via Fine-Tuning (VPFT), We propose Visual Planning via Fine-Tuning (VPFT) as simplified variant of our framework, which shares the same training architecture Stage in Section 2.2 but replaces random trajectories with optimal planning trajectories. For each environment, we representing the minimal-step path from the initial opt state spt = 00 to the goal. At each step, the model trained to predict the next state e given the prefix opt The objective is identical to Equation 2 with supervision from the optimal trajectory. u
sample a distinct trajectory (u. opt ...., 0
Supervised Fine-Tuning (SFT) in Text. In this baseline, planning is formulated in the language modality. Instead of generating an intermediate visual consequence of an action, the model produces textual description of the intended action sequence. Formally, given visual input state and textual prompt p. which represents the task description, the model trained to generate verbalized action sequence i (I,,..., :,.), where each token I Vis represents action. The input to
19

---

Published as a conference paper at ICLR 2026
Table 6: We compute the percentage of failed trajectories that caused by least invalid
action, rather than a suboptimal but valid action. Lower values indicate better action validity control.
Task
Invalid-Failure Ratio (%)
VPRL
VPFT
FROZENLAKE
36.9
60.6
MAZE
25.1
73.7
MINIBEIAVVIO
29.6
78.3
the model is the concatenation of the prompt tokens and the visual tokens, and the target is the corresponding action sequence. Following prior work on supervised fine-tuning (SFT) (Wei et al. 2022a) in autoregressive models, we minimize the cross-entropy loss for action prediction:
(OF (o,0) log (o bei v,p)
(5)
Beyond directly training on action labels, we further conduct an ablation on FROZENLAKE with textual variants that verbalize the input state before predicting the action sequence. In particular, explore two structured representations: Coordinate descriptions and ASCII grids. During training, the target sequence consists of the description tokens (encoding the environment layout in either coordinate or ASCII form) concatenated with the action labels that lead to the goal.
Reinforcement Learning (RL) in Text. We also extend RL textual planning in the FROZENLAKE enviromment as an ablation. We optimize the textual planner with Group Relative Policy Optimization (GRPO). The reward design combines fixed format reward, which enforces the correct output structure, with an outcome reward defined in two variants: (1) progress based reward identical to that used in VPRI., or (2) the Progress Rate (PR) metric directly used as the reward.
For all post-training experiments, we apply Low-Rank Adaptation (LoRA) (Hu et al. 2022) on both attention layers and feed-forward layers. The detailed hyper-paraman papper-prraeterss are shown Table Only the loss of the targets is calculated in instraction-tuningg manner (Wei et al. 2022a for SFT. The image tokenizer detokenizer are frozen during training. We use the Adam optimizer Loshchilov & Hutter 2019) for all training procedures.
When SET for textual planning and visual planning, we train the model for a maximum of 30 epochs. For VPRL, we first do stage random trajectories for 10 epochs for the purpose of exploration. then use GRPO to optimize the model for planning for another epochs for stage 2. We sample group of 10 candidate responses per prompt to compute the advantages accordingly. To encourage a balance between exploration and exploitation, we apply a KL divergence penalty with a coefficient 3= 0.001. For RL in the textual modality, we adopt the same 10 training epochs for fairness, with group size of 8. We use the TRL library for training (von Werra et al. 2020). We've conducted our experiments on the machine with 8 A.00 GPUs.
E.5 LICENSES
Model-wise, Large Vision Model and Qwen 2.5 VL are under the Apache-2.0 license. TRL is under the Apache-2.0 license. We collect the MAZE dataset with Python seripts. FROZENLAKE is collected from OpenAl Gym under the MIT License.
F RESULTS
F.1 VPRL TRAINING
The reward curves with standard deviation for all tasks shown in Figure The shaded regions indicate the standard deviation across groups. For better visualization, we apply Gaussian smoothing to both the reward values and their corresponding standard deviations.
20

---

Published as a conference paper at ICLR 2026
1.0
0.5
Progress Reward
0.0
-0.5
MinBehavicur
Naze
PresonLake
-1.0
4000
8000
12000
16000
Step
Figure 7: Reward curves with standard deviation for VPRL on FROZENLAKE, MAZE and MINIBEHAVIOR.
Table 7: Performance of text-based variants of Qwen-2.5-VV Dwwen2.55VLL-Instee 1.--VLL-nnstrue--1 and 7B on FROZENLAKE,
We report Exact Match (EM) and Progress Rate (PR) across all difficulty levels and their average.
EM (%)
PR (%)
Model
L3
L4
LS
L6
Avg.
L3
L4
LS
L6
Avg.
Qwen 1.5-V..-nntrrett3BB
SFT
Direct
87.2
72.0
48.8
28.0
59.0
89.4
84.3
71.3
60.1
76.3
w/ Coordinates
87.2
78.0
64.8
30.8
65.2
89.6
82.5
74.0
57.2
75.8
w/ ASCII
79.6
75.6
58.8
34.0
62.0
83.3
82.5
74.8
59.1
74.9
GRPO
w/ VPRL progress reward
69.2
52.8
41.2
26.0
47.3
73.6
72.2
66.2
55.0
66.8
w/ PR metric reward
70.8
60.0
41.6
23.6
49.0
75.5
76.1
65.7
56.0
68.4
Qwen D.5-VL-Instruut-BB
SFT
Direct
97.6
86.0
56.4
34.4
68.6
98.1
92.1
78.9
68.4
84.4
w/ Coordinates
93.2
88.0
74.8
41.6
74.4
94.1
89.7
81.5
65.5
82.7
ASCII
93.2
86.0
68.0
45.2
73.1
94.1
88.6
8
69.6
83.4
GRPO
VPRL progress reward
72.4
50.4
30.8
54.4
76.2
69.2
69.9
w/ PR metric reward
82.8
68.8
51.6
37.2
60.1
84.9
79.6
71.5
61.0
74.3
LVM-7B
VPFT (ours)
92.0
82.8
68.8
58.0
7.5.4
93.1
84.7
73.4
66.9
79.5
VPRL (ours)
97.6
95.6
90.8
82.4
91.66
98.4
96.0
93.0
85.6
93.2
F.2 TRAINED TEXTUAL BASELINES AND REWARD DESIGN
To strengthen the comparison with our visual planners, we train different text-based baselines beyond the direct action- sequence SFT model reported in the main paper. We are interested in: 1) whether different textual representation influences the performance of language-based reasoning, and 2) whether reinforcement learning can help improve the language-based planning performance with multimodal input.
Specifically, experiment with two alternative SFT variants that first describe the environment layout in different formats (coordinates and ASCII) before predicting the action sequence.
Trained SFT variants.
21

---

Published as a conference paper at ICLR 2026
Table 8: Exact Match performance of VPFT and VPFT" across different grid sizes in FROZENLAKE.
Exact Match (%)
Model
3x3
4x4
5x5
6x6
VPFT*
86.4
73.6
50.0
33.2
VPFT
92.0
82.8
68.8
58.0
Table Out-of-distributiin (OOD) performance on enlarged grids. Models are trained on smaller grids and evaluated the sizes indicated in parentheses.
FROZENLAKE (7x7)
MAZR (7x7)
MINTBEHAVIOR (9x9)
Model
EM (%)
PR (%)
EM (%)
PR (%)
EM (%)
PR (%)
VPFT
9.6
15.3
9.2
17.8
0.0
5.8
VPRL
20.4
31.2
10.0
21.6
0.4
14.7
The model is trained to first output a coordinate-based description of the grid environment (e.g., positions of the agent, goal, and obstacles), followed by the full action sequence.
SFT with Coordinates:
SFT with ASCII: The model is trained to output an ASCII-based description of the environment layout before producing the action sequence. Specifically, S denotes the starting position, the goal, H an ice hole, and F passable cell.
The example input-output formats for different text-based reasoning variants shown in Figure in Appendix F.2.
We experiment with both variants for -wwnn225--VL-nnstmee-- and 7B, training them with the same configurations as the original text SFT baseline. As shown in Table 7 the SFT variants with either coordinates or ASCII do not provide consistent significant improvements over the direct SFT baseline. Specifically, these variants with additional structural descriptions in either coordinates yield slight gains in EM, but exhibit lower compared to the direct SFT baseline. Moreover, both variants still fall short of VPRL, suggesting that enriching textual input alone is insufficient to bridge the gap between visual and text-based planning.
ASCII
RL-trained text baseline. We also explore the feasibility of applying RL improve the planning performance with multimodal input, given the success RL in the pure language planning domain (Guo et al., 2025a). We train RL-based text model using -ren-225-VLIInstruuce the GRPO algorithm, with output format shown in Figure
and 7B with
We adopt the same progress baased reward design as in VPRL for fair comparison, in addition to forpla reward that ensures reasoning is enclosed within <think> tag and the final answer within <answer> tag.
If the action is optimal (i.c., aligned with some optimal trajectory from the current state), it receives reward of 1.
If the action is valid but non-optimal, it receives 0.
If the action is invalid (c.g., colliding with walls or stepping into holes), it receives -5, and all subsequent actions are also treated as invalid.
To prevent reward hacking through excessively short rollouts, any sequence shorter than the length of the optimal trajectory padded with invalid actions for the remaining steps, which are considered "stay-in-place" moves, in other words, invalid transitions. Finally, to make rewards comparable across sequences of varying lengths, normalize the total reward by sequence length Σ r./T.
We train the text RL baselines for 10 epochs, consistent with VPRL. Using the VPRL progress reward described above, the model achieves 54.4% EM (Table 7). We suspect that this limited performance
22

---

due to the reward design not being sufficiently discriminative. In particular, trajectories that contain the same number of optimal and non-optimal actions receive identical rewards regardless of their order (e.g., starting with optimal actions and then switching to non-optimal ones yields the same return as the reverse). As a result, the model tends to first learn to produce valid actions in general, Published as a conference paper at ICLR 2026
Input Example
Task: Generate the shortest valid sequence of ac- tions that moves the elf from the starting position to the goal without stepping into any ice holes (see detailed prompts in Appendix G
SFT (w/ Coordinates)
After analysis, the layout of the given image is follows:
Starting position: (3, 3)
Goal: (2, 2)
Ice holes: (1, 1), (2, 3)
Passable cells: (1, 2), (1, 3), (2, (3, (3.
Based the above layout, the final answer is: <ANSWER>Iefc up< ANSWER>
SFT (w/ ASCII)
symbols used to represent the grid are: denotes the starting position, G the goal, H the ice hole, F passable cells.
HFH FGH
FFS
Based on the above layout, the final answer is: <ANSWER> up</ ANSWER>
GRPO
<think>
The elf is in the bottom right cell.
2. The goal is in the top center cell.
The top left cell has ice hole, the elf cannot go left there.
The elf needs to move one cell
the center cell.
Then the elf needs to move one cell up to the goal cell.
The sequence of actions Move one cell left the center cell. Move one cell up to the goal cell. </think>
<answer>left up</ answer>
Figure 8: Examples of model outputs under different trained textual variants: SFT (w/ Coordinate), SFT (w/ ASCII), and GRPO. Each variant follows a distinct format for representing the environment. Even for the relatively simple 3 3 example input, observe that the ASCII and GRPO models still produce erroneous layout descriptions.
and only later to distinguish optimal actions among them.
23

---

Published as a conference paper at ICLR 2026
Level
<think>
The the center cell.
2. The goal is the gift box in the top left corner.
3. The elf needs to move 1 cell left and cell up.
4. There are ice holes directly below and to the right, so they impassable. </think>
<answer>left up</answer>
Level
<think>
1. The elf is at the top left section of the grid.
2. The goal is the gift box in the bottom right section.
3. The elf needs move 3 cells down and cell right.
4. There are ice holes directly below and to the right, so the elf cannot move through them. </think>
<answer>down down down right</answers
Level
<think>
1. The elf the bottom left.
top right. 3. The elf needs to move 3 cells right and cells up.
The goal is the gift hox
4. There are ice holes directly above and to the right. so the elf cannot </think>
through them.
<answer>right right right up</answer>
Level
<think>
1. The elf the top right.
2. The goal the gift box the bottom left.
3. The needs to move 3 cells down and cell left.
4. There are ice holes directly below and to the right of the elf, so cannot use those paths. </think>
<answer>loft down down down</answen
Figure 9: Examples of responses from the textual GRPO baseline with PR metric as the reward on FROZENLAKE across different difficulty levels. Each box shows the input image and the correspond- ing model output. In all cases, the model produces incorrect layout descriptions, which in turn lead to incorrect predicted action sequences.
To address this issue, we further design an alternative reward function by directly adopting the Progress Rate (PR) metric from the main paper. This formulation encourages the model to focus on generating consecutive valid forward moves from the start, rather than separating the learning of validity and optimality. Under the same training conditions, this reward improves EM 60.1 10,, but the performance still lags behind the direct SFT baseline. As we discussed in Section 4 (error analysis paragraph), attribute the bottleneck of language- planning with RL. the modality
24

---

Published as a conference paper at ICLR 2026
1.0
Gemini (Tand)
(STn
Gemeni (C:T)
1.0
Demtni (Thin)
Owen
Demini (CoT)
0.8
0.8
Accuracy
Accuracy
0.6
0.4
0.4
0.2
0.2
0.0
3
5
7
Grid Size
Grid Size
Figure 10: Performance across different grid sizes, reflecting task difficulty. Left: MAZE. Right: MINIBEHAVIOR, Visual planners consistently maintain higher accuracy and exhibit flatter perfor- mance curves, indicating robustness to increasing complexity.
gap, which introduces inaceuracies in grounding visual information into text, causing exploration to proceed from misinterpreted states and thereby reducing the overall effectiveness of learning. By contrast, our visual planning paradigm avoids this modality gap by operating directly the visual domain, ensuring exploration within the correct state space via RL.
F.2.1 EXAMPLES OF TRAINED TEXTUAL VARIANTS
Outputs of different textual variants are illustrated in Figure including SFT with coordinate and ASCII representations, as well as GRPO with reasoning traces. Even for the relatively simple 3 x 3 input, and despite all variants producing the correct final predictions shown in the figure, we observe that the ASCII and GRPO models still generate erroneous layout descriptions: in the ASCII case, the passable cell at the top right is misclassified as an ice hole, while the GRPO case, the goal position incorrectly identified.
We also conduct further qualitative analysis of responses from the textual RL baseline trained with the PR metric as the reward (Figure B.. In all cases, the model produces incorrect layout descriptions, which in turn lead to incorrect predicted action sequences, highlighting the modality gap in grounding visual information into text.
F.3 PERFORMANCE WITH SCALING DIFFICULTIES
We evaluate the performance of different methods with respect to task difficulty in MINIBEHAVIOR and MAZE, as shown in Figure [0 Our visual planners consistently achieve higher accuracy across all grid sizes and exhibit notably flatter performance curves, indicating greater robustness to increasing environment complexity.
Interestingly, in MINIBEHAVIOR, we observe that the accuracy visual planners increases with grid size, which in contrast to the trend exhibited by textual planners. We hypothesize that this is due to the fixed layout components in this task, specifically, the presence of only table and a printer. This maintains consistent layout complexity across different grid sizes and allows knowledge acquired in smaller grids to generalize effectively to larger grids. This suggests that visual planning better captures and transfers structural patterns the environment.
F.4 DUSTOR-DDITRIBUUTIN PERFORMANCE
Figure [1 illustrates generated images from VPFT and VPRL on OOD scenarios across MAZE, FROZENLAKE, and MINIBEHAVIOR tasks. Notably, both models exhibit certain level of visual generalization to unseen configurations, such as larger grids with finer step granularity, despite not encountering them during training.
We subsequently quantitatively test generalization by evaluating the model on OOD environments with larger grid sizes. We find that SFT models perform poorly, while VPRL still demonstrates
25

---

Published as a conference paper at ICLR 2026
VPFT
VPRL
VPFT
VPRL
VPFT
VPRL
Figure 11: Qualitative comparison of visual planning outputs from VPFT (top) and VPRL (bottom) duito--distiibution (OOD) scenarios with unseen larger grid size across MAZE, FROZENLAKE, and MINIBEHAVIOR. Each example shows a failure case from VPFT contrasted with successful trajectory generated by VPRL under the same environment configuration.
certain level of visual planning capability, as shown in Table 9 VPRL. consistently outperforms VPFT in both Exact Match and Progress Rate, suggesting that it, to some degree, captures underlying planning strategies rather than merely memorizing training patterns.
Finally, analyze the robustness of VPRL by qualitatively testing its behavior under perturbed inputs. As shown in Figure 12 we mask portions of the input images with black gray patches to simulate partial occlusion of the environment. Remarkably, the model continues produce coherent planning traces within the masked environments, while preserving structural consistency with the visible input regions. This observation highlights the generalization capability of our visual planner, as it adapts incomplete visual information without deviating from the underlying environment layout.
F.5 ABLATION: THE ROLE OF STAGE 1
To better understand the role of Stage 1 in our two-stage framework, we conduct an ablation study isolating its impact. The primary purpose of Stage 1 but rather to initialize a policy with strong exploration capacity and valid output formats. To verify this, we reuse the original VPFT training pipeline, i.e., learning from optimal trajectories, but start from the Stage 1 checkpoint as VPFT* Surprisingly, this variant yields lower final performance on FROZENAKK compared standard VPFT. This result supports our hypothesis that Stage does not contribute to planning ability itself, but instead provides an exploration -irioattoon-fiendlyy initialization that facilitates effective reinforcement learning in Stage 2.
improve planning performance directly,
F.6
VISUAL. PLANNING RESULTS
VPRL Stage and Stage 2. Table 10 presents results for each stage of VPRL. After Stage 1, the model lears to generate plausible images but lacks goal-directed behavior, resulting in near-random performance across tasks. In Stage 2, reinforcement learning instills purposeful planning, enabling the model to align generations with the goal and outperform VPFT across all benchmarks.
26

---

Published as a conference paper at ICLR 2026
Figure 12: Qualitative analysis of VPRL under perturbed inputs (the first image of each trace). When parts of the input environment are masked (black/gray regions), VPRL maintains consistent planning traces aligned with the visible structure, demonstrating robustness incomplete visual information without deviating from the underlying environment layout.
Table 10: Performance comparison VPRL Stage 1 and Stage 2 across all three tasks.
FROZENLAKE
MAZE
MINIBEHAVIOR
Model
EM (%)
PR (%)
EM (%)
PR (%)
EM (%)
PR (%)
VPRI. Stage
11.1
27.2
9.6
22.7
0.5
14.2
VPRL Stage 2
91.6
93.2
74.5
77.6
75.8
83.8
Generated Visual Planning Traces for Illustration. Figure 13 shows the generated visual planning traces for FROZENLAKE, with Figure 14 for MAZE and Figure [5] for MINIBEHAVIOR Each visual trajectory begins with the initial state as the input (the first frame), followed by sequence of intermediate states generated by VPRL that form the predicted visual plan.
We include examples from three categories: (1) Optimal cases, where the model successfully generates the shortest valid path the goal: (2) Non-optimal cases, where the agent fails reach the goal within the optimal number of steps due to intermediate non-optimal actions; and (3) Invalid cases, in which the generated trajectory contains invalid actions that violate environment constraints, preventing task completion. Notably, as illustrated in Figure 3 we still observe occasional planning errors. While reinforcement learning significantly improves generalization compared supervised fine-tuning, it does not fully eliminate such failure cases.
27

---

Published as a conference paper at ICLR 2026
Correct Cases
1.
2.
3.
4.
5.
6.
Non-optimal Cases
1.
Invalid Cases
2.
Figure 13: Generated visual planning trajectories from VPRL on the FROZENLAKE test set. We illustrate three representative categories: optimal, oon-opli and invalid cases. In non-optimal examples, the model occasionally enters local loops but still has the chance to make progress toward the goal, see the first and third trajectories. invalid cases, despite a significant reduction in failure rate, VPRI. still exhibits errors such as disappearing agents, contradictory actions (e.g., simultaneous left and right), or unrealistic teleportation.
28

---

Published as a conference paper at ICLR 2026
Correct Cases
1.
2.
3.
4.
5.
Non-optimal Cases
2.
Invalid Cases
1.-
2.
Figure 14: Generated visual planning trajectories from VPRL the MAZE test set. We illustrate three representative categories: optimal, non-optimal, and invalid cases. In non-optimal examples, similar FROZENLAKE, the model occasionally enters redundant loops but still progresses toward the goal. Invalid cases include maze-specific errors, such as the agent erroneously traversing through walls, violating the structural constraints of the environment. Notably, we observe that the last invalid case, the agent is able to plan an optimal trajectory in subsequent steps.
29

---

Published as a conference paper at ICLR 2026
Correct Cases
1.
2.
3.
4.
Non-optimal Cases
1.
2.
Invalid Cases
1.
2.
Figure 15:
Generated visual planning trajectories from VPRL on the MINIBEHAVIOR test set.
30

---

Published as a conference paper at ICLR 2026
Original Images
Predicted Images
Reconstructed Images
Figure 16:
Qualitative comparison between original images (top), predicted images by the model (middle), and reconstructed images obtained by encoding and decoding the original inputs (bottom).
Table 11: Exact Match (EM) and Progress Rate (PR) on FROZENLAKE under VPRL when using ground-truth images versus self-generated images inputs during inference.
FM (%)
PR (%)
Model
L3
L4
L5
L6
Avg.
L3
L4
L5
L6
Avg.
VPRL
(w/ self-ge generated images)
97.6
95.6
90.8
82.4
91.6
98.4
96.0
93.0
85.6
93.2
(w/ ground-truth boundd-truth images)
98.4
95.2
93.2
81.6
92.1
98.5
95.8
94.1
85.3
93.4
F.7
IMAGE QUALITY ANALYSIS
can be observed that the intermediate images on FROZENLAKE generated by the visual planner in Figure&ccnttan noticeable artifacts, and we suspect that this noise arises from the limitation of the image tokenizer rather than from the model's image generation ability. To verify this, we include an additional analysis FROZENLAKE that illustrates how the tokenizer reconstructs images in our framework.
Linitations of the Image Tokenizer. Figure 16 confirms that the artifacts observed in predicted images originate from the tokenizer rather than from the prediction process itself. When encoding a ground-truth image into visual tokens and decoding it back, the reconstructed output shows similar artifacts inevitably introduced by the tokenizer to those in the model's predictions, which makes the reconstruction identical to the original image. At the same time, we observe that the intermediate images produced by the model are already comparable in quality to the reconstructed images. While our work focuses on planning rather than image generation quality, this observation indicates that the visual planner generates images that are sufficient for effective planning.
We consider this behavior to encouraged by the dynamics interpreter. During the training, the dynamics interpreter serves as an implicit format constraint. Any generated image that it cannot parse treated as an invalid transition and receives a penalty, enforcing the model to maintain semantic structure of the environment in its generated images.
Robustness to Intermediate Image. We subsequently conduct quantitative study to evaluate whether providing high-quality intermediate images at inference improves performance. Instead of feeding back the model's ' self-geeerated image at each step, we replace with the ground-true truth image rendered by the environment, which serves high-quality version.
31

---

Published as a conference paper at ICLR 2026
Table 12: Average inference token cost across FROZENLAKE, MAZE, and MINIBEHAVIOR, We also report the average of the task-level average costs. Higher values indicate higher computational cost.
Model
FROZENLAKE MAZE
MINIBEHAVIOR
Avg.
Closed-Source Models
Gemini 2.0 Flash
Direct
10.8
12.5
14.8
12.7
CoT
150.5
166.5
196.5
171.2
Gemini 2.5 Pro (think)
885.6
1030.2
1619.9
1178.6
Open-Source Models
Qwen I.Sttut-n
Direct
13.4
95,9
13.9
41.1
CoT
306.2
316,4
272.3
298.3
SFT
10.7
11.4
13.2
11.8
LVM-7B
VPFT (ours)
819.2
957.2
1471.2
1082.5
VPRL (ours)
819.2
957.2
1471.2
1082.5
Table 11 shows that the performance with and without high-quality images remains similar across all grid sizes. This shows that our visual planner robust visual noise and does not depend on perfectly rendered images to plan effectively.
F.8 COMPUTATIONAL COST ANALYSIS
To provide quantitative comparison of the computational cost between visual planning and traditional textual rea- soning, we further analyse the token usage of both the visual planner and the textual baselines during inference. We compute the average number of generated tokens for all models reported in Table across all tasks. In addition, we include more detailed breakdown of the token cost for the trained textual planner variants listed Table evaluated on
Table 13: Average inference token cost of trained textual planner variants FROZENLAKE.
Model
Token Cost
Qwen 2.5-VL-Instmect77B -SFT
-Direct
10.7
-w/ Coordinates
179.0
F FROZENLAKE.
-W/ASCI
84.3
GRPO
Table 12 and Table summarise the resulting inference token cost. As expected, visual planning introduces no- ticeable computational overhead due to repeated image generation. However, this additional cost remains afford-
VPRL progress reward
129,8
PR metric reward
74.9
able in practice when compared with textual Co'.. On average across the three tasks, the token cost of our visual planner is roughly times that of Qwen 2.5-VL.-nntrruct-BB with CoT and around 6 times that of Gemini 2.0 Flash with Co',, suggesting that our method still computationally feasible. We also observe that thinking models, such Gemini 2.5 Pro, produce the largest number of tokens among all tasks, indicating that visual planning is not always the most expensive option.
32

---

Published as a conference paper at ICLR 2026
G PROMPTING TEMPLATES
FROZENLAKE (Direct)
Task: Frozen Lake Shortest Path Planning
You are given an image of a grid-based environment. In this environment:
An elf marks the starting positicn.
A gift represents the goal.
Some cells contain ice holes that are impassable for the elf.
The elf can move in one of four directions only: "up", "down", "left", or "zight". Each move transitions the elf by one cell in the correspending absolute direction. Diagonal movement is not permitted.
Your
to analyze the image and generate the shertest valid sequence of actions that moves the elf from the starting position to the geal without stepping into any ice holes.
Provide your final answer enclosed between <ANSWER> and </ANSWER>, for example: rANh up up</ANSWER>.
FROZENLAKE (Coordinate & ASCII Representation)
Task: Frozen Lake Shortest Path Planning
You are given image of grid-based environment. In this environnent:
An elf marks the starting position.
gift represents the goal.
Some cells contain ice holes that are impassable for the elf.
The elf can move in one of four directions only: "up", "down", "left", or "right". Each move transitions the elf one cell in the corresponding absolute direction. Diagoral movement is not permitted.
Your task is to analyze the image and generate the shortest valid sequence of actions that moves the elf from the starting position to the goal without stepping into any ice holes.
Describe the layout of the envmentttt based on your analysis of the image, then provide your final answer enclosed between <ANSWER> and </ANSWER>, for example: <ANsNER>right up</ANSWER>.
FROZENLAKE (GRPO)
Task: Frozen Lake Shortest Path Planning
You are given image of grid-based ronment. this envi ronnent:
An elf marks the starting positicn.
A gift represents the goal.
Some cells contain ice holes that are impassable for the elf. of four directions only: "up", "down", "left", "right", Each move transitions the elf by one cell the corresponding absolute direction. Diagonal movement is not permitted.
The elf move in
Your task is to analyze the image and generate the shortest valid the elf from the starting position
sequence of actions that the goal without stepping into any ice holes.
Present your reasoning enclosed within <think> and </think> tags. For example:
<thinkoReasoning steps go here.</think>
Then, provide your final answer enclosed within <answer> and </answer> tags. For example: <answer>right up up</answer>
33

---

Published as a conference paper at ICLR 2026
MAZE
Task: Maze Shortest Path Planning
You are given image of
environment. In this environment:
green circle marks the starting position the agent.
A red flag marks the goal.
The agent can move in one of four cardinal directions only: "up", "down "left", or "right". Each move shifts the agent exactly one cell in that direction. Diagonal movement is permitted. The black maze walls are impassable. The agent cannet pass through any wall segment.
Your task is to analyse the image and produce the shortest valid sequence of actions that moves agent from its starting position to the goal without crossing any wall.
Provide final answer enclosed between <ANSWER> and </ANSWER>, for example: <ANswER>right up up</ANSWER>.
MINIBEHAVIOR
Task: Mini Wini-Behavior Installing the Printer
You are given an image of grid-based environment. In this environment: The rec triangle represents the agent.
The white icon represents the printer, which must be picked up by the agent.
brown tiles represent the table, where the printer must be placed.
The agent can take the follewing actions:
"up", "down", "left", "right": each action shifts the agent by exactly cell that direction. Diagonal movement
not parmitted.
"pick": pick up the printer
is in one of the four adjacent cells surzounding the agent. This action is invalid if there is no adjacent printer.
"drop": drep the printer ente the table if the agent is adjacent to
table cell. This action invalid there is no adjacent table.
Constraints:
The agent cannot move through the table tiles. The agent cannot move through the printer until it has been picked up. After picking it up, the agent may move through the cell that previously contained the printer.
Your task is to analyse the image and produce the shortest valid sequence of actions that allows the agent pick up the printer and then place it on the table.
Provide final answer enclosed between <ANSWER> and </ANSWER>, for example: rANsHER>riqht down right pick left dzop</ANSWER>.
34


Reviews:
[{"review_id": "2vZ6MpvGAG", "reviewer": "Reviewer_dg8a", "summary": "This paper focuses on visual planning tasks in MLLMs. It proposes a novel way to perform visual planning by generating images during the planning process. Specifically, given an input image, the model reasons about the next state after performing certain actions by generating the corresponding images. To achieve this, the VLM is first trained to generate next-state images, then trained through reinforcement learning to encourage actions toward the target state. Experiments show that the proposed methods outperform strong open-source and private models in various environments.", "questions": "Please check the weaknesses section:\n1. (for weakness 1) How important it is to generate high-quality images? Would current intermediate images quality good enough?\n2. (for weakness 2) If I understand correctly about EM measure, How to compute the exact match in the image space?\n3. (for weakness 3) Comparison with similar methods such as MVoT?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Generating images makes the reasoning occur in the visual space rather than the textual space. Thus, the proposed method has the potential for more direct and better reasoning performance.\n2. Through qualitative analysis of intermediate outputs, the paper shows that the proposed method can generate reasonable intermediate images, which is key for correct visual reasoning.\n3. The experiment result demonstrates the proposed method outperforms strong baselines including private MLLMs.", "weaknesses": "1. It can be observed that the intermediate images are not perfect (e.g., in Fig. 3, first row, the player and goal tokens have artifacts). Thus, it would be interesting if the paper could show performance when the model reasons over high-quality images. For example, each time the model generates a new image, the corresponding high-quality image (rendered by the engine rather than generated by the model itself) is fed into the model. Would this lead to better performance? If so, the performance gap could quantify the importance of generating high-quality (precise) intermediate images.\n2. If I understand correctly, $v$ stands for one of the intermediate images, as stated in L132–133. As such, how is it determined whether two images are an exact match in the EM metric in L300-L302? Or is it actually comparing in the action space (e.g., left/right) rather than the image space?\n3. Similar to this work, some recent studies also explore generating intermediate images during reasoning, such as [1]. It is good that the paper discusses these related works, but it should consider directly comparing with these baselines.\n\n[1] Imagine while Reasoning in Space: Multimodal Visualization-of-Thought", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FitTHFLxib", "reviewer": "Reviewer_2U49", "summary": "This paper introduces Visual Planning, a new paradigm that performs reasoning through visual representations rather than language. Using VPRL (Visual Planning via Reinforcement Learning), a GRPO-based post-training framework for large vision models, our approach generates image sequences that visualize step-by-step inference. Experiments on visual navigation tasks (FrozenLake, Maze, MiniBehavior) show that VPRL surpasses text-based reasoning, highlighting visual reasoning as a powerful complement to language reasoning for spatial and geometry-driven problems.", "questions": "All of my qeustions are listed in the weakness section. If my concerns are well addressed, I will raise my rating.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. New Paradigm: The paper introduces \"Visual Planning\" as a genuinely new paradigm for reasoning.\n\n2. Good Empirical Results: The proposed method, Visual Planning via Reinforcement Learning (VPRL), significantly outperforms a wide range of baselines.\n\n3. Methodological Robustness: The two-stage VPRL framework is well-designed and justified.", "weaknesses": "1. Reliance on an External Oracle for Rewards: A significant weakness in the method's detail is its reliance on non-learned, external modules to provide the reward signal. The VPRL framework depends on a \"dynamics interpreter\" and a \"progress estimator\". The appendix reveals this estimator is a Breadth First Search (BFS) algorithm —an oracle that has already solved the task and knows the optimal path from every state. The interpreter also uses rule-based pixel and IoU comparisons. This means the model isn't learning the environment's dynamics or the concept of progress; it's learning to generate images that satisfy an external oracle that already has the answers.\n\n2. Insufficient Justification for a \"Purely Visual\" Paradigm: This paper needs to justify why exploring a purely non-verbal, vision-only paradigm is a necessary or superior research direction. The authors' decision to \"eliminate language as a confounding factor\" is a research-scoping choice, but it is not a strong argument for the paradigm's utility.\n\n3. Limited Task Complexity and Scalability: The paradigm is validated only on simple, 2D, discrete grid-world environments (FROZENLAKE, MAZE, MINIBEHAVIOR). It is highly questionable if this approach can scale to complex, 3D, photorealistic, or continuous-state environments (e.g., robotics). In such settings, autoregressively generating a perfect, step-by-step sequence of future images is computationally expensive (a point the authors concede ) and the rule-based reward function (which relies on pixel comparison ) would be far too brittle to work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uKqxAF3ePi", "reviewer": "Reviewer_uu3P", "summary": "The paper takes visuals as a medium for expressing and structuring reasoning, which potentially learn spatial and geometrical information. To realize this, the author proposes Visual Planning, planning through purely visual representations for “vision-first” tasks. The proposed GRPO demonstrates improvement in planning.", "questions": "See Weakness.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The author investigates the potential of visual representation as a medium, which expands the research of LLMs to a broader area.\n\nThe presentation of the paper is great, with a clear statement and an appropriate graph.\n\nThe paper is the first attempt to investigate whether models can achieve planning purely through visual representations.", "weaknesses": "Can you provide any figures to clearly show the difference between language as a medium and visual as a medium in certain cases?\n\nIt will be better if we can discuss any advantages of visual as a medium in real CV tasks, such as visual grounding. And the proposed methods, whether they can be easily transferred to 3D?\n\nIf we finally want to get an MLLM, how do we add GRPO to the regular training receipt? When to align visuals with other modalities?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KSj99bTRe4", "reviewer": "Reviewer_Rhdx", "summary": "This study proposes a visual planning approach that performs task planning entirely through visual representations. In this paradigm, planning is carried out via a sequence of images, which encode the step-by-step reasoning process in the visual domain, similar to how humans make sketches or visualize future actions.", "questions": "1. Visual planning involves generating high-dimensional image sequences, which can be computationally more expensive than searching in low-dimensional text/action spaces as in language models. Please elaborate on how  it's computational efficiency and search space complexity compare to textual CoT methods?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "It performs planning purely within the visual modality as a holistic process, where the actions are not explicitly predicted but instead implicitly represented by transitions between visual states.", "weaknesses": "The experimental validation is restricted to discrete, low-dimensional grid-world navigation tasks, where the visual states are comparatively simple and straightforward to encode.\nVisual planning implicitly expresses actions by generating a sequence of visual states. Although this avoids modality switching, when planning fails, the absence of an explicit action sequence (such as textual CoT) makes the model’s decision process difficult to debug and understand.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)