Latent Visual Reasoning
Authors: Bangzheng Li
Ximeng Sun
Jiang Liu
Ze Wang
Jialian Wu
Xiaodong Yu
Emad Barsoum
Muhao Chen
Zicheng Liu
Venue: ICLR 2026 Poster (2026)
Keywords: multimodal large language models
vision Language Models
multimodal reasoning
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Multimodal Large Language Models (MLLMs) have achieved notable gains in various tasks by incorporating Chain-of-Thought (CoT) reasoning in language spaces. Recent work extends this direction by leveraging external tools for visual editing, thereby enhancing the visual signal along the reasoning trajectories. Nevertheless, these approaches remain fundamentally constrained: reasoning is still confined to the language space, with visual information treated as static preconditions. We introduce Latent Visual Reasoning (LVR), a new paradigm that enables autoregressive reasoning directly in the visual embedding space. A visual encoder first projects images into visual tokens within a joint semantic space shared with the language model. The language model is then trained to generate latent states that reconstruct key visual tokens critical for answering the query, constituting the process of latent visual reasoning. By interleaving LVR with standard text generation, our model achieves substantial gains on perception-intensive visual question answering tasks. In addition, we adapt the GRPO algorithm to conduct reinforcement learning on latent reasoning, further balancing LVR and textual generation.  We show that LVR substantially improves fine-grained visual understanding and perception, achieving 71.67\% on MMVP compared to 66.67\% with Qwen2.5-VL. Code base and model weights will be released later.

Published as a conference paper at ICLR 2026
LATENT VISUAL REASONING
Bangzheng Li Ximeng Sun", Jiang Liu?, Ze W Jialian Wu', Xiaodong Yu², Hao Chen", Emad Barsoum", Muhao Chen', Zicheng Liu² UUniversity of California, Davis *Advanced Micro Devices, Inc. xbzhli brahli@ucdavis.co
Websin i te
Code
Model
ABSTRACT
Multimodal large language models (MLLMs) have achieved notable gains in var- tous tasks by incorporating chain-of-thought (CoT) reasoning in language spaces. Recent work extends this direction by leveraging external tools for visual editing, thereby enhancing the visual signal along the reasoning trajectories. Nevertheless, these approaches remain fundamentally constrained: reasoning is still confined to the language space, with visual information treated as static preconditions. We introduce Latent Visual Reasoning (LVR), a new paradigm that enables autore- gressive reasoning directly in the visual embedding space. A visual encoder first projects images into visual tokens within joint semantic space shared with the language model. The language model is then trained to generate latent states that reconstruct key visual tokens critical for answering the query, constituting the process of latent visual reasoning. By interleaving LVR with standard text genera- tion, our model achieves substantial gains on perception-intensivee intensive visual question answering tasks. In addition, we adapt the GRPO algorithm to conduct reinforce- ment learning latent reasoning, further balancing LVR and textual generation. We show that LVR substantially improves fine-grained visual understanding and perception, achieving 7 71.67% MMVP compared to 66.67% with Qwen2.5-VL. The Code base and model weights will be released later.
INTRODUCTION
Multimodal large language models (MLLMs) (Li al. 2024b Bai al. 2025b Wang et al. 2025d) have shown remarkable capability in jointly understanding visual and textual content. By leverag- ing the generative capabilities of their backbone large language models (LLMs), MLLMs extend the expressiveness of visual encoders beyond simple perception tasks. This advancement has en- abled the integration of chain-of-thought (CoT) reasoning into MLLMs, allowing them to perform structured textual reasoning in response to complex multimodal queries. In this paradigm, the LLM decomposes a query into intermediate steps and resolves each step while conditioning on static vi- sual inputs. This approach, referred to as "Thinking about Images" by (Su et al.) (025d). has proven effective across diverse domains, including scientific visual question answering (Zhang al. 2023).
mathematics (Huang et al. 2025a). and visual grounding (Bai et al. 2025c).
Further research has expanded the multimodal reasoning workspace by enabling active editing of input images alongside textual reasoning trajectories. Such editing includes drawing auxiliary lines (Hu et al. 2024), zooming (Shao et al. 22224a] [Su et al. 2025a Surís et al. (023), shifting image styles (Liu et al. 2025a), highlighting sub-regions (Fu et al. 2025), and more. During intermediate CoT steps, methods in this paradigm either call external tools or generate programs to manipulate images, re-encode the edited outputs, and inject the new image tokens visual enhancements into subsequent textual reasoning. In this way, the salient visual information is actively incorporated throughout the reasoning process. These methods are commonly termed "Thinking with images."
Work done during internship AMD.

---

Published as a conference paper at ICLR 2026
<thinks hat (xl. yi, x2), left, white. middle, navy, right, </thinlo This image shows three golfers
<thinle image shows three men ... The question asks for hat, so I will look closer at their seadds...//tinnke
<llvr_starti start
What color is the hat of the golfer with sunglasses?
<llve mi__endl) Answer: Navy
Answer: White
Answer: White, Navy
(a) Thinking about Images (via CaT in Text Space)
(b) Thinking with images (via Image Crapping Tools)
(c) Latent Visual Reasoning (via Visual Semantics Reconstructin)
Figure 1: Conceptual illustration of LATENT VISUAL REASONING (LVR). We compare LVR with two paradigms: "Think about images," which performs multimodal reasoning entirely in text space, and "Think with images". which leverages external visual tools to highlight regions of interest (ROIs). In contrast, LVR leverages the LLM's ' latent space to reconstruct the semantics of ROIs, enabling seamless cross-modal reasoning.
Both "Thinking about Images" and "Thinking with Images" aim to address a core limitation in cur- rent MILMs: despite sophisticated visual encoders projecting visual information into text spaces, backbone LLMs often fail to capture the visual details most relevant to the text query. This short- coming arises from factors such as modality projection bias (Zhang et al.] 2024 Liu et al. (023) modality interference (Cai et al., 2025; Wang et al. 2025e; Pezeshkpour et al.) 2025 Deng al. 2025a), and cross- modaaity attention bias (Zhang et al. (025c). "Thinking about Images" addresses these issues by generating additional task-relevant text tokens in the context window, thereby in- creasing the likelihood of correct answers. However, excessive token generation may cause the textual context dominate, overshadowing essential visual inputs (Huang et al. 2024). In contrast, "Thinking with Images" leverages external tools to inject visual information during text generation, calibrating the alignment between generated and the original visual input. Yet these approaches often bypass the newly injected sub-images due to training data bias, remain constrained by the predefined operations of the tools (Su al.] 2025a). In short, both categories primarily refine text generation to improve cross-modality understanding, yet fundamental gap persists between visual inputs and text generation in producing the final answer.
A parallel line of research explores omni-modalle foundation models that accept both text and vi- sual inputs and generate outputs both modalities simultaneously (Team] 2024 Deng et al. 2025b) Xie et al.) 2025). Some subsequent works attempt to exploit image generation capabilities for mul- timodal reasoning, but effectiveness has so far been demonstrated only in specific downstream tasks such navigation and maze solving (Li et al. 2025 Xu et al. 2025). Moreover, it remains unclear whether visual inputs, once decoded and re-encoded by such models, can still faithfully preserve the original information. Rethinking the input of MILMs, where continuous visual tokens and discrete text tokens are embedded in a shared latent semantic space, we ask the following question:
If visual and textual tokens are embedded in a joint semantic space within an MLLM, why not reason over both jointly as well?
is both natural and efficient extend reasoning beyond discrete tokens include visual tokens that directly encode visual information. However, conventional LLMs are limited to operating on discrete tokens due their next-token prediction training objective. To address this, HHo et al. proposed passing last hidden states rather than text tokens, enabling more efficient expression of complex thoughts through latent reasoning. Building on this idea, we introduce LATENT VISUAL REASONING (LVR), a novel paradigm for multimodal reasoning (see Fig. [ig. Our approach involves a simple yet fundamental modification to the conventional Vision- Projector-oL....M. structure: the LLM is able to perform hybrid reasoning that alternates between LVR and standard text generation. In the LVR phase, the LLM leverages the last hidden state to approximate the question-relee relevant visual tokens within the visual inputs. During the text generation phase, the model predicts the next text token in sequence. Both phases operate in an auto-regressive manner.
We propose two-stage training pipeline for LVR. The first stage is supervised fine-tuning (SFT), which jointly optimizes LVR 's internal processes alongside next-token prediction for text genera- tion. The second stage applies reinforcement learning (RI.), allowing LVR self-evolve the latent
2

---

Published as a conference paper at ICLR 2026
reasoning process while receiving policy rewards from generated text, thereby encouraging more unified semantic space. Specifically, we adapt the GRPO algorithm (Shao et al.) (024c) to replay latent reasoning steps during policy gradient loss computation. In addition, GRPO leverages ver- ifiable rewards to evaluate roll-out responses, where the policy gradient loss is computed solely from the token distribution of the text generation component. Experimental results demonstrate that LVR achieves substantial improvements over state-of-the-ar MLLMs, particularly on perception- intensive and visual -eeiid-dependett understanding tasks.
In summary, our contributions are as follows:
We propose LATENT VISUAL REASONING, novel multimodal reasoning paradigm that unifies latent reasoning over visual inputs with text generation in the language space, enabling deeper integration of visual and textual signals throughout the model's reasoning process.
We introduce architectural innovations and training frameworks for stable and scalable training MLLMs with LVR. Our approach combines a reconstruction loss with next-token prediction for SFT and extends the GRPO algorithm to latent reasoning for reinforcement learning.
Through extensive evaluation, we demonstrate that LVR achieves strong performance across di- verse visual question answering benchmarks requiring fine- aine-grained visual understanding and per- ception. In addition, our comprehensive ablation studies and discussions explore alternative ar- chitectural designs and training objectives, providing insights to guide future research on this emerging paradigm.
2 RELATED WORKS
Think about Images. Many prior works have employed text-space chain-of-thought (CoT) rea- soning to enhance visual perception and multimodal mathematical reasoning. Early approaches focused on constructing SFT datasets (Xu et al. 2024; Shao et al. 2024a Wei et al. 2025b). aiming
for models to fully acquire such reasoming patterns during training. More recently, the field has shifted toward RL-based methods (Peng al.) 2025 Tan al. 2025 Meng al. 2025 Yang al. 2025a) with many discussions on data design (Liang et al.) 2025). loss design (Hong et al., 2025 and training stages (Wei al. 2025a Deng et al. 2025e Liu et al. 2025b Chen al. (025b))
Some studies explore specialized RL phase designs, while others mitigate visual hallucination by generating auxiliary captions (Xia et al. (0255) or randomly masking parts of the image (Wang al.) 2025g). Additional efforts guide models to focus on regions of interest (ROIs) by predicting points, bounding boxes, or descriptions (Jiang et al. 2025 Ni al. 2025 Yu al. 2025 Liu et al. 2025c), ensuring that answers are grounded in the correct visual evidence. Another line of work incorpo- rates verification or rewriting steps refine reasoning quality (Wang et al. 2025c [hhn et al. 2025a] Zhang al., 2025b; Chen et al., 2025a). Despite these advances, most methods still perform reason- ing in text space, which remains an indirect and inefficient representation
visual understanding.
Humans, by contrast, can reason about images naturally without translating them into text. Inspired by this observation, our work seeks to more closely mimic human visual reasoning by enabling models to understand and reason directly in the visual space.
Think with Images. Another recent line of research emphasizes augmenting multimodal models with external, predefined visual tools. Many approaches employ zoom-in cropping utilities (Su et al. 2025b Zhang et al.) (222d)) to locate ROIs relevant to a given question, while others integrate more advanced tools such as OCR engines, chart parsers, or even drawing interfaces (Huang al.) 2025b). To determine when and how to invoke these tools, early studies relied on supervised fine-
tuning (Wang al. 2025b] Zhang al. 2025a Chung al. (025). whereas more recent work adopts reinforcement learning to learn tool-using behaviors (Zhang et al., [202d]] [S et al. [025cc] Geng al. 2025 Wu et 2025a), enabling interleaved CoT reasoning and tool execution (Wu et al. 2025b: Zheng et al. 2025). Despite their success, these approaches remain constrained by the availability and design of external tools. Tool APIS can be difficult to extend, and updates or changes often require substantial training effort. Moreover, many fundamental operations, such as zooming, cropping, or OCR, can potentially be solved directly within modern MLLMMs without tool invocation. Motivated by these limitations, our work explores latent visual reasoning, approximating relevvtt visual tokens directly in the visual representation space rather than relying on explicit external tools.
3

---

Published as a conference paper at ICLR 2026
[Latent Visual Reasoning)
[Answer] Exploring its surroundings
We think visually before speak
aller end
Last Hidden State
LM Head
Multi-Modal LLM
Inpul Embeedding-..
[Image]
[Question]
</|vr startp
@llv_eedd
What does the tiger's body posture suggest about its behavtor
Image Embedding
LVR Emmbedding
Input
mood?
Embedding
Special Embedding
Figure 2: Training and inference pipeline of LVR. The overall framework closely follows a standard MLLM. Images are encoded into tokens by visual encoder and mapped into joint semantic space with text embed- dings. During the SFT stage, bounding boxes provided to identify query-relevant visual tokens, which supervise the last hidden states in the LVR process. Here, only the LLM's last hidden states passed forward for latent reasoning. optimized with mean squared error (MSE) loss. The LVR process wrapped with special tokens that indicate reasoning mode. Once all query-relevant visual tokens are consumed, the model exits LVR and resumes standard text generation with cross-entropy loss. During RL training, the model self-evolvee the LVR process learned in SFT, while only the text generation part is supervised, using adapted GRPO At inference, the model triggers LVR upon generating the special token, propagates hidden states to reconstruct visual semantics, and resumes text generation when a stopping criterion is met.
Latent Reasoning. In natural language processing, several studies have explored performing rea- soning in the latent space (e.g., the final layer outputs) rather than directly in the token space (Hao) al. 2024). Some work (Shen et al.] 2025b) leverages latent representations to approximate leven reasoning, investigating both fixed-length and variable-length latent reasoning strate- gies (Cheng & Van Durme 2024). However, latent spaces in NLP are often difficult to interpret, making supervision space reasoning to the visual domain, where the latent tokens are grounded in visual meaning. Specifically, we aim for these tokens to approximate question-relevant visual features, Recent efforts have also explored similar directions, using latent spaces to capture visual content, but many rely on auxiliary images to supervise latent tokens (Yang et al., 22025b; Bigverdi et al., (025). Such auxil- iary data introduces additional labeling and pairing costs, ultimately limiting scalability. In contrast, our approach requires no extra images and can be readily applied across diverse vision tasks after single training.
such representations challenging. Building on this idea, we extend latent-
LATENT VISUAL REASONING
In this section, we introduce LATENT VISUAL REASONING, new paradigm that enables MLLMS reason jointly over text and visual tokens. The overall inference pipeline is illustrated in Figure 2 At high level, LVR is trained to reconstruct visual semantics relevant to both the input image and the accompanying text query. These reconstructed semantics, which we term latent visual thoughts, then combined with the original inputs to guide the generation of textual responses.
We begin by providing an overview of the LVR architecture, which is built on the Qwen-2.5-VL series (Bai et al. 2025a) ($3.1). Next, we present two-stage training pipeline-combininn vised fine-tuning and reinforcement learning taaming-that jointly teaches the model to reconstruct visual semantics and generate text (.2)). Finally, we introduce decoding strategies that allow the model to flexibly alternate between LATENT VISUAL REASONING and standard text generation ($3 during the inference.
super-
4

---

Published as a conference paper at ICLR 2026
3.1 METHOD OVERVIEW
Fig. 2 iillsstrates the architecture of LVR, which largely follows the standard MILLM design. It con- sists of three key components: vision encoder vision(-). an LLM backbone (()), and a multimodal projector proj(-) that aligns the two modalities. Given an input image-question pair (X,, X,), the vision encoder transforms the image into visual features V vision (X.). ). In parallel, the textual question X, embedded into language features T by the LLM's embedding layers. Since visual and textual features reside in distinct latent spaces, the projector proj(-) maps the visual features into a representation aligned with the language model's latent space, denoted as VT - proj(VV...
In standard MILMs, both VT and T are passed into the L.L (()) However, the decoding process remains strictly text-centric: hidden states are computed auto-regressively and mapped, through the language modeling head, to discrete tokens in the vocabulary. Thus, while visual information can guide the reasoning process, the output space is still constrained to text tokens, limiting the model's ability to directly reason over visual semantics.
To overcome this limitation, we propose LVR, which extends the conventional text-only genera- tion paradigm by enabling interleaved LATENT VISUAL REASONING. When the special token < lir start| >> is generated, the model enters a latent reasoning mode, where it reconstructs visual semantics in the space of Vr better support answering the textual query. The hidden states produced in this mode are directly propagated as input embeddings to subsequent positions until stopping criterion is reached. At that point, the model generates the token [ivr_end] and resumes standard text generation. The hidden-state sequence produced during the LVR segment can be regarded as an analogue of human's 's "visual thinking," " where query-relevant visual semantics are mentally reconstructed to enhance the precision of reasoning and answering.
3.2 TWO-STAGE TRAINING PIPELINE
3.2. SUPERVISED FINE-TUNING.
We begin with supervised fine-tuning (SFT) to instill the basic pattern of latent visual reasoning. During this stage, we explicitly supervise the reasoning content by forcing the MLLM to use its latent space embeddings (i.e., last hidden states) reconstruct the ground-truth regions of interest for each image-text pair. Since the reconstruction loss directly dictates what the model must reason about, the reasoning process is constrained rather than free-form. We therefore characterize this stage feacher-forcing paradigm for latent visual reasoning: although restrictive, it allows the model to quickly acquire the fundamental ability to reason in the latent space.
Each SFT instance consists of an image-question pair, accompanied by pre-annotated bounding box of a region of interest (ROI) relevant to the query. For given image, the model's image processor first crops it into grid of visual patches. Based the ROI bounding box, LVR efficiently selects the corresponding patches and retrieves their indices I = {I1 Ia........ Ir. } from the sequence of flattened visual patches in O(1) time. During the forward pass, the image is encoded into a sequence of visual embeddings in the semantic space, followed by the embeddings of the text tokens. A subset of visual embeddings v {uI, U/ u,, VT, } is then gathered using the index list I. which-by design of the ViT encoder-directly corresponds to the visual patches within the ROI. These selected tokens are enclosed by the special tokens |1 |lvr_ster vrrsstar] and [lvr_end] >, thereby defining the LATENT VISUAL REASONING process. Finally, the remaining textual response appended to the sequence for generation.
We train our model with two joint learning objectives that explicitly couple latent visual reasoning with downstream text generation.
Visual Reconstruction Loss. During latent reasoning, the model predicts sequence of the final hidden states ) that are expected encode the underlying visual semantics. We enforce these hidden states to approximate the ground-truth visual embeddings {= (MSE) objective:
Tr via a mean squared error
T LINR
h [=l
vell2
(1)
T,
5

---

Published as a conference paper at ICLR 2026
Next-Token Prediction (NTP) Loss. For the subsequent language modeling phase, the model gen- erates the final response tokens (y: ) answer the visual question. We adopt the standard cross- entropy (CE) loss to maximize the likelihood of the ground-trute sequence:
1 Tv
CNTP-
Σ Ty t=1
log (o yea. h1:r,).
(2)
Joint Objective.
The overall training loss a weighted sum of the two components:
L= LNTP ALVR LLVR,
(3)
where ÅLVR is a hyperparameter to balance reconstruction and generation signals.
3.2.2 REINFORCEMENT LEARNING
We employ GRPO to further refine the interaction between LVR and standard text generation. In this stage, rewards are computed solely based on output accuracy and adherence to the required response format. Unlike supervised fine-tuning, no constraints are imposed on the intermediate outputs of LVR, allowing the model to freely explore the latent visual reasoning space. This laxation also removes the need for pre-annotated ROI bounding boxes, as GRPO training can be performed directly on image-text pairs. Moreover, GRPO implicitly encourages the generation of the < [lvr-stare > token, thereby increasing the likelihood activating the LVR process during response generation.
However, directly applying the standard GRPO algorithm to LVR is non-trivial. The key challenge lies in the mismatch between the policy-gradient loss, which is defined over token distributions, and the latent reasoning process, which lacks an explicit token distribution. To address this issue, we propose GRPOtatents variant that can be seamlessly applied any latenf-reasoning model.
JGRPO (0) Egg, oretoold
min (re(0) A., clip(r. 1 1 1+ A1)
(To( q, Foor( q, 1))
Here, the token-wiss importance ratio (0) for text token yi, is computed using teacher-forcing log-probability pass, where the latent reasoning hidden states are replayed from original rollout:
T4e (0) = Tee (yh, q, h. Alasent y1.ce)
(5)
To.la (He,s q, I, laattet V6,c()
During rollout record the last hidden states of LVR processes
Iattent
latent ( ..... 4,4
lateut]
To evaluate importance ratios, we perform a teacher-forcing forward pass under both my and Tiaaa
The recorded hidden states hilttet patched into the latent-reasoning positions of the model, thereby restoring the exact context preceding text generation and ensuring consistent conditional pe-oaabbbiliiees for the output sequence.
Rewards are derived solely from the text output y. We incorporate two reward types: a format reward, which equals if the response contains both < [lvr start/ > and < [lvr end] > tokens and 0 otherwise, and an accuracy reward, which equals 1 if the answer is correct and 0 otherwise. The format reward will encourage LVR process in the response while the accuracy reward indirectly supervises the latent reasoning process through its impact on text generation.
Finally, the group-normalized reward is defined as:
R = R(u) - mean mecan(R()).......... R(yG)) std(R(y)),.... R(yc))
A = R. (vt E
(6)

---

Published as a conference paper at ICLR 2026
3.3 DECODING STRATEGIES
Our initial results reveal that decoding in LVR can be challenging, as it is often unclear when the model should exit the LVR process. Specifically, when the next-token prediction introduces a < [Ivr_start] > token, the decoder switches into the LVR mode, passing the last hidden states instead of the language model head's predicted tokens. During this phase, the language model head continues producing token predictions, but it should ideally emit < [ivr end once an optimal length of latent reasoning has been reached, reconstructing all necessary visual semantics for the current task. In practice, however, the predicted tokens during latent reasoning are often unstable, motivating us to propose three decoding strategies:
Fixed Token: assigns a constant budget of reasoning steps. Once the budget is reached, the model immediately exits latent reasoning mode.
Latent End Token: introduces a trainable tensor in the hidden state space. When the last hidden state approaches this tensor, the decoder resumes text-token generation.
Mode Switching Loss: adds an auxiliary loss term during SFT that supervises the token distribu- tion predicted by the language model head in the latent reasoning phase. A BCE loss encourages the distribution of the final latent reasoning token toward < [1vr_end] >> (close to 1), while all intermediate tokens are pushed away from < [ivr_end > (close O). At inference time, the model exits latent reasoning mode once < [ivr_end] > is predicted.
Empirically, we find that Fixed Token achieves the best performance, while Mode Switching Loss fails to work as intended. A detailed analysis is provided in Section 4.5
EXPERIMENT
We adopt Qwen-2.5-VL. VL 3B and 7B as the backbone MILMs. For the visual encoder, we set the maximum resolution to 5120 x 28 x 28 pixels and the minimum to 128 x 28 28 pixels. both training stages, the visual encoder and multimodal projector are kept frozen, with only the LLM parameters updated. This design reflects the learning objective of LVR to unify the reasoning space under the hypothesis that an optimal modality projection can be achieved without additional tuning.
In the SFT stage, we use VISUAL COT (Shao et al, 2024b) as the primary training data, This large-scale VQA dataset contains 438k question- answer pairs, each annotated with bounding boxes that mark the critical regions needed to derive the answer. During training, the variable number of image tokens and LVR tokens results in imbalanced instance lengths. To address this, adopt the adaptive multimodal data-packing strategy from Chen et al. (2024). which supports dynamic batching: multiple shorter instances can be packed together, while longer ones are grouped in smaller numbers. On average, this yields an effective batch size of ~322 per device. The learning rate is set x 10 5 For the 7B variant, supervised fineituning requires roughly 40 hours to complete 2,500 steps on AMD MI250 GPU cluster.
For reinforcement learning, we implement a latent GRPO framework customized from the Hugging- Face TRL package, using ViRL (Wang et al. (025a) as the training data. The policy generates 8 responses per input, with a learning rate x 10-2, a fixed sampling temperature of o = 0.9, and a KI. coefficient of B = 0.04. This stage is applied only to the 3B variant, requiring about 20 hours to complete 1,500 steps. We do not exlend to the 7B variant due to limited computational resources.
4.1 EVALUATION BENCHMARKS
We evaluate LVR on the visual detail understanding task and a diverse set of vision-centric bench- marks. For visual detail understanding, we adopt V* Bench, which assesses MILMSs ability to perform fine- fine-grained visual detail search and relative spatial reasoning respectively on two subsets. We further employ MMVP (Tong et al.) 2024) to measure perception robustness under subtle image perturbations, providing rigorous test of LVR 's fine-grained reasoning capabilities.
Beyond these, we evaluate on Counting (object enumeration). JigSaw (image reconstruction from fragments). Relative Reflectance (pixel-level albedo comparison), and Spatial Relation (ob- ject-relation understanding within a scene). These tasks are drawn from BLINK (Fu et al. 2024), a benchmark of expert-annotated, perception-hean tasks designed for MLLMS.
7

---

Published as a conference paper at ICLR 2026
Method
V
V....
VR..P
MMVP
Counting
IQ Test
JigSaw
Relative Reflect
Spatial Relation
Close Source Models
GPT-4o
62.8
51.7
30.0
58.0
38.8
76.9
Gcmini2.5-Pro
79.2
ARGUS-X3
78.5
45.5
Open Models based on Qwen2.5.
VL-7B
Qwen2.5-VL
78.5
81.7
73.7
66.7
66.1
26.0
52.0
38.8
87.4
PAPO
36.1
25.2
52.6
54.3
66.7
29.3
52.0
39.6
88.8
Vision-R1
70.2
70.4
69.7
46.7
51.7
26.7
27.3
44.8
66.4
PrxelReasoner
80.1
81.7
77.6
67.0
66.7
25.3
52.7
42.5
88.1
SFT
79.1
82.6
73.7
65.7
67.5
26.7
45.3
33.6
88.8
LVR Steps)
81.2
84.4
76.3
72.0
69.2
28.7
52.7
42.5
89.5
Steps)
81.7
84.4
71.7
70.0
29.3
52.0
42.5
86.0
LVR (16 Steps)
80.6
81.7
79.0
71.7
70.8
27.3
52.7
41.8
87.4
Table 1: Experimental results vision-centric tasks. LVR outperforms both
"Think about Images"
and
"Think with Images"
highlighting the scalability of this paradigm for MLLMs.
Method
V
P.
MMVP
IQ Test
JigSaw
PAPO
31.94
22.61
31.33
46.67
LVR (4 8 16)
64.9
65.5
66.5
09.6
71.3
71.3
60.5
60.5
56.6
54.7
56.0
56.0
29.3
30.7
52.0 30.0
30.0
52.7
52.7
Rt. (4 14]8]16)
65.5
67.0
66.5
69.6
72.2
71.3
59.2
59.2
59.2
55.3
35.3
58.0
30.7
32,0
52.7
52.7
50.7
Table 2: RL results with 3B models. GRPOoateet further boosts performance, demonstrating the effective- ness adapting RL for latent reasoning and enabling self-evolution.
To ensure consistency and fairness, all evaluations follow the standardized metrics provided by LMMs-Eval (Li et al. 2024a).
4.2 BASELINES
We compare LVR against state-of-the-art MILM baselines, grouped into the following categories:
Thinking about Images. This category includes Vision-RI (Huang et al. 2025al and PAPO (Wang al. a 2025f), which use reinforcement learning with verifiable rewards equip MLLMs with chain- of-thought reasoning. Vision-RI follows "think before answer" trajectory, while PAPO adds an implicit perception loss for image-grounded descriptions.
Thinking with Images. This category includes PixelReasonee et 2025a) and Argus-X: X3 Man al., (025). PixelReasoner employs image-editing tools to iteratively enhance the input image during reasoning. In contrast, Argus-X3 detects regions of interest via bounding boxes, extracts the corresponding visual tokens, and reinjects them into the MLLM input to strengthen perception. This makes Argus-X3 strong reference point for comparison with LVR: whereas Argus- X3 depends external tools for feature extraction, LVR directly learns to reconstruct visual semantics.
To isolate the effect of training data, also include bascline trained with standard supervised fine-tuning on the same dataset as LVR, denoted as SFT. For completeness, we include each baseline model's 's system prompt when it is available in the documentation.
4.3
MAIN RESULTS
The main evaluation results are summarized in Table We report performance under the best decoding strategy, Fixed Token, with LVR steps set to [4, 8, 16]. Overall, LVR achieves state-of- the-art results across most benchmarks. Importantly, for fair comparison, all open- baselines are built on the same backbone MILMs as LVR, highlighting the effectivenes8 of latent reasoning over existing "Think about images" or "Think with images" approaches.
The largest gains are observed on the V and MMVP benchmarks, where LVR consistently achieves top performance across all step settings. In particular, it improves the base model by 2.7% on the V.A. subset, which measures visual detail search, and by 5.3% on the V subset, which evalu- ates relative spatial reasoning. On MMVP, which perturbs subtle image details to assess perception robustness, LVR demonstrates strong performance by reconstructing target visual semantics, accu- rately identifying differences, and thereby improving accuracy. Moreover, surpasses PixelRea- soner in both categories, despite PixelReasoner relying external tools crop image sub-regions.
8

---

Published as a conference paper at ICLR 2026
Method
V
V..A.
V....
MMVP
IQ-Test
JigSaw
LVR
81.7
84.4
77.6
71.7
29.3
52.0
LVR
Eude
39.8
32.2
51.3
19.0
6.7
13.3
LVR MLP
74.4
76.5
71.1
69.7
23.3
50.0
LVR Grdd
79.6
82.6
75.0
69.0
25.3
44.0
Table 3: Ablation studies on the 7B model show the standard approach performs best, indicating the LLL natively aligns visual and textual semanties without extra head. However, the unstable latent end token suggests need for future work on veriable-length reasoning.
This underscores that reconstructing visual semantics can be more effective than depending on external visual-editing tools (as in "Think with Images") for fine-grained visual understand- ing. In addition, both PAPO and Vision-RI exhibit degraded performance textual-space CoT MLLMs (i.e,, "Think about Images") may introduce cross-modal in- terference and weaken perception, whereas LVR avoids this issue by reasoning jointly across modalities.
V*, suggesting that
LVR also achieves leading results on Counting, IQ-Test, JigSaw, and Spatial Relation. In Counting, the model is required to enumerate specific objects in a scene; in IQ-Test, it solves geometry-busd -based puzzles; In JigSaw, it reassembles image crops; and in Spatial Relation, it answers questions about relative object positions. These results demonstrate that LATENT VISUAL REASONING effectively unifies object detection, visual-dependent logical reasoning, visual reconstruction, and spatial rela- tion understanding.
However, LVR does not achieve top performance on Relative Reflect. We attribute this gap distribution shift between training and evaluation data: such tasks require reasoning over multiple images, whereas LVR was trained exclusively on single-image data. We anticipate that incorporating cross-image data augmentation in future work will further strengthen LVR 's capability for multi- image reasoning.
4.4
RL RESULTS
We present the results of our proposed EmPOO Table 2 Reinforcement learning further enhances LVR performance beyond the SFT stage across multiple benchmarks. The superior results demonstrate that incorporating a format reward based on the < |lvr_stare > and < _ivr_en|_>> > tokens effectively encourages the model to perform LATENT VISUAL REASON- ING. In contrast, removing these trigger tokens from the reward function destabilizes training, re- sulting in purely textual responses and degraded performance.
4.5 ABLATION STUDIES
We examine variants in the architectural design and decoding strategies of LVR models.
LVR with heads. Analogous to the LM head in LLMs, which maps text hidden states logits, add a LVR head on top of the latent reasoning positions to transfer LLM hidden states into visual semantics. We evaluate two designs: (i) 2-layer MLP without intermediate up-casting, and (ii) a gated linear unit (GLU) with an intermediate dimension expanded to 3x the LM hidden size. The 7B results (Fig. 3 show that standard LVR consistently achieves the best performance across benchmarks. This expected, as the LVR process is directly supervised in the joint semantic space of text and vision, ensuring no semantic gap between the L.LM's last hidden states and those of LVR.
Unrestricted LVR decoding. As discussed $3.3 we explored two alternative decoding strategies, Mode Switching Loss and Latent End Token, for variable-length LVR processes. However, Mode Switching Loss failed to encode stopping conditions, collapsing to zero LVR steps. The perfor- mance of Latent End Token is reported in Fig. where we observe significant instability. Human evaluation indicates that this instability stems from unreliable distance measurements between LVR hidden states and the latent end token. Despite testing cosine similarity, LI, and L2 distances under varying thresholds, the model often failed to terminate properly and max out generation steps. We expect future work on improving stability for fully free-form LATENT VISUAL REASONING.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
In this paper, we presented LVR as novel multimodal reasoning paradigm that unifies latent rea- soning over visual tokens with standard text generation. By extending the Vision-ProjectorLLLMM structure and training with supervised fine-tuning plus GRPO-based reinforcement learning, LVR achieves stable hybrid reasoning. Experiments show substantial gains on perception-intent intensive benchmarks, demonstrating that reasoning jointly over latent visual and textual spaces offers promising direction for future multimodal reasoning.
6 ETHICS STATEMENT
Technological innovations in multimodal large language models present both opportunities and risks. The impact of our proposed latent reasoning framework, associated decoding strategies, and evaluation benchmarks depends heavily on data quality and intended use. Ethical deployment re- quires that all training data be sourced responsibly and in compliance with legal and ethical stan- dards, alongside safeguards for individual data rights. the absence of comprehensive regulation, responsibility lies with practitioners to ensure proper use. Biases in either supervised finetuning data or reinforcement learning rewards may propagate disparities, particularly affecting underrepre- sented groups and undermining fairness and generalizability. To mitigate these risks, we emphasize adherence to ethical principles in system design, including transparency in training data and model- ing choices, open-source releases to support accountability, and protective measures for vulnerable populations.
REPRODUCIBILITY STATEMENT
Our experiments utilized open-source models (Qwen-2.5-VL) and datasets (VISUAL COT, VIRL) for training. Training was conducted using open-source tools, including the Huggingface Trainer and DeepSpeed frameworks. To ensure full reproducibility, we will release our complete code base, model weights, and a corresponding Docker file, allowing for a complete replication of our setup.
REFERENCES
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025a.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint 13923,, 2025b.
Sule Bai, Mingxing Li, Yong Liu, Jing Tang, Haoji Zhang, Lei Sun, Xiangxiang Chu, and Yansong Tang. Univg-rl: Reasoning guided universal visual grounding with reinforcement learning. arXiv preprint arXiv:2505. 14231, 2025c.
Mahtab Bigverdi, Zelun Luo, Cheng-Yo Yu Hsich, Ethan Shen, Dongping Chen, Linda G Shapiro, and Ranjay Krishna. Perception tokens enhance visual reasoning in multimodal language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3836-3845, 2025.
Rui Cai, Bangzheng Li, Xiaofei Wen, Muhao Chen, and Zhe Zhao. Diagnosing and mitigating modality interference in multimodal large language models. arXiv preprint arXiv:2505, 19616, 2025.
Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training rl-like reasoning large vision- language models. arXiv preprint a504v:2504.1 |468, 2025a.
10

---

Published as a conference paper at ICLR 2026
Shuang Chen, Yue Guo, Zhaochen Su, Yafu Li, Yulun Wu, Jiacheng Chen, Jiayu Chen, Weijie Wang, Xiaoye Qu, and Yu Cheng. Advancing multimodal reasoning: From optimized cold start to staged reinforcement learning. arXiv preprint arXiv:2506.04220 2025b.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint urXiv:24/2.05271, 2024.
Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations. arXiv preprint 2rXiv:24/2.13171, 2024.
Jiwan Chung. Junhyeok Kim, Siyeol Kim, Jaeyoung Lee, Min Soo Kim, and Youngjae Yu. Don't look only once: Towards multimodal interactive reasoning with selective visual revisitation. arXiv preprint ar arXiv:2505. 18842, 2025.
Ailin Deng, Tri Cao, Zhirui Chen, and Bryan Hooi. Words vision: Do vision-language models have blind faith in text?, 2025a.
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Wei- hao Yu, Xiaonan Nie, Ziang Song, Guang Shi, and Haoqi Fan. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505. 14683, 2025b.
Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. Openvithinker: An early exploration to complex vision-language reasoning via iterative self-improvement. arXiv preprint arXiv:2503.1733 17352, 2025c.
Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. arXiv preprint 2rXiv22404.12390, 2024.
Xingyu Fu, Minqian Liu, Zhengyuan Yang, John Corring, Yijuan Lu, Jianwei Yang, Dan Roth, Dinei Florencio, and Cha Zhang. Refocus: Visual editing chain of thought for structured image understanding, arXiv preprint arXiv:2501.00 00552,, 2025.
Xinyu Geng, Peng Xia, Zhen Zhang, Xinyu Wang. Qiuchen Wang, Ruixue Ding, Chenxi Wang, Jialong Wu, Yida Zhao, Kuan Li, et Webwatcher: Breaking new frontiers of vision-language deep research agent. arXiv preprint arXiv:2508.05748. 2025.
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in continuous latent space. arXiv preprint arXiv:2412.0627 067669,,,66, 2024.
Minjie Hong, Zirun Guo, Yan Xia, Zehan Wang, Ziang Zhang, Tao Jin, and Zhou Zhao. Apo: Enhancing reasoning ability of milms via asymmetric policy optimization. arXiv preprint arXiv:2006.21655. 2025.
Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as visual chain of thought for multimodal language models. arXiv preprint arXiv:2406. 09000009943..20 2024.
Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection cetrospectionnalllcattoo allocation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13418-13427, 2024.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao. Zheyu Ye, Fei Zhao, Zhe Xu. Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models, 2025a.
Zeyi Huang, Yuyang Ji, Anirudh Sundara Rajan, Zefan Cai, Wen Xiao, Haohan Wang, Junjie Hu, and Yong Jae Lee. Visualtoolagent (vista): A reinforcement learning framework for visual tool selection. arXiv preprint wrxiiv2505.20288 2025b.

---

Published as a conference paper at ICLR 2026
Chaoya Jiang, Yongrui Heng, Wei Ye, Han Yang, Haiyang Xu, Ming Yan, Ji Zhang, Fei Huang, and Shikun Zhang. Vlm-r 3: Region recogmition, reasoning, and refinement for enhanced multimodal thain-o arXiv preprint arXiv:2505. 2025.
Bo Li, Peiyuan Zhang, Kaichen Zhang, Fanyi Pu, Xinrun Du, Yuhao Dong, Haotian Liu, Yuanhan Zhang, Ge Zhang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Accelerating the development of large multimoal models, 2024a. URL https: //gi /ttt://gggttbbttt/ge thub ctmo//.cooo/a/ooomooo.oooo. wwm/Evolloooocooooooooooooooooo Evol wivolving..Ms Lab/
Imms-eva
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang. Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprini 1xXiv2408.033326, 2024b.
Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulié, and Furu Wei. Imagine while reasoning in space: Multimodal visualizzaioon-o-fthuuuhh 2025. URL
https: /ttpxi///axiv..o http//bb//22..1.. 07542
Yiqing Liang, Jielin Qiu, Wenhao Ding, Zuxin Liu, James Tompkin, Mengdi Xu, Mengzhou Xia, Zhengzhong Tu, Laixi Shi, and Jiacheng Zhu. Modomodo: Multi-domain data mixtures for multimodal Ilm reinforcement learning. arXiv preprint arXiv:2505.24877 2025.
Dairu Liu, Ziyue Wang, Minyuan Ruan, Fuwen Luo, Chi Chen, Peng Li, and Yang Liu. Visual Ab- stract Thinking Empowers Multimodal Reasoning, May 2025a. URL http:://///// /ttt::://axxvv.ooo org/
abs/ abs/ Nas/2205.200 20164
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurlPS, 2023.
Zeyu Liu, Yuhang Liu, Guanghao Zhu, Congkai Xie, Zhen Li, Jianbo Yuan, Xinyao Wang, Qing Li, Shing-Chi Cheung, Shengyu Zhang, al. Infi-mmr: Currieulum-baas based unlocking multimodal reasoning via phased reinforcement learning in multimodal small language models. arXiv preprint arXiv:2505.2309 2025b.
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint urXiv:2503.0178. 2025c.
Yunze Man, De-An Huang, Guilin Liu, Shiwei Sheng, Shilong Liu, Liang- Yan Gui, Jan Kautz, Yu- Xiong Wang, and Zhiding Yu. Argus: Vision-centric reasoning with grounded chain-of-thonn of-thought. In CVPR, 2025.
Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. CoRR, 2025.
Minheng Ni, Zhengyuan Yang. Linjie Li, Chung-Ching Lin, Kevin Lin, Wangmeng Zuo, and Li- juan Wang. Point-rft: Improving multimodal reasoning with visually grounded reinforcement finetuning. arXiv preprini arXiv:2505.1 000222555.99000 19702, 2025.
Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, and Xu Yang. Lmm-rl: Empowering 3b Imms with strong reasoning abilities through two-stage rule-based rl. arXiv preprini 2503.05000 2025.
Pouya Pezeshkpour, Moin Aminnaseri, and Estevam Hruschka. Mixed signals: Decoding vims' reasoning and underlying bias in vision-language conflict, 2025.
Hao Shao, Shengju Qian, Han Xiao, Guanglu Song. Zhuofan Zong, Letian Wang, Yu Liu, and Hong- sheng Li. Visual cot: Unleashing 2024a.
thougtt reasoning in multi-modal language models,
Hao Shao, Shengju Qian, Han Xiao, Guanglu Song. Zhuofan Zong, Letian Wang, Yu Liu, and Hong- sheng Li. Visual cot: Unleashing chain-of- of-thought reasoning in modill language models, 2024b.
12

---

Published as a conference paper at ICLR 2026
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models, 2024c.
Chuming Shen, Wei Wei, Xiaoye Qu, and Yu Cheng. Satori-rl: Incentivizing multimodal reasoning with spatial grounding and verifiable rewards. arXiv preprint arXiv:2505. 19094, 2025a.
Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. Codi: Compressing chain-of ohtinnt into continuous space via self-distillation. arXiv preprint arXiv:2502.21074, 2025b.
Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: In- centivizing pixel-space reasoning with curiosity-driven reinforcement learning. arXiv preprint arXiv:2505. 2025a.
Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: In- centivizing pixel-space reasoning with curiosity-driven reinforcement learning, arXiv preprint arXiv:2505.15966 2025b.
Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang. Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, et al. Openthinking: Learning to think with images via visual tool reinforcement learning. arXiv preprint arXiv:2505.08617, 2025c.
Zhaochen Su, Peng Xia, Hangyu Guo, Zhenhua Liu, Yan Ma, Xiaoye Qu, Jiaqi Liu, Yanshu Li, Kaide Zeng, Zhengyuan Yang, et al. Thinking with images for multimodal reasoning: Founda- tions, methods, and future frontiers. arXiv preprint 2xXi:22506.23118 2025d.
Dídac Surís, Sachit Menon, and Carl Vondrick. Vipergpt: Visual inference via python execution for reasoning, arXiv preprint arXiv:2303.08128, 2023.
Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Reason-rft: Reinforcement fine-tuning for visual reasoning. arXiv preprint
20Xi::2503.20752, 2025.
Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09881. 2024. doi: 10448500arXxi.2445.0088. URL htttps://gtthhhie com/
Facebookresearchh httpe/aaaaaccaahaeeeeeaaaaaeaeee chameleom
Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms, 2024.
Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. VI- rethinker: Incentivizing self-refiection of vision-language models with reinforcement learning. arXiv preprint 2rXiv:2504.08s 008337, 2025a.
Jiacong Wang, Zijian Kang, Haochen Wang, Haiyong Jiang, Jiawen Li, Bohong Wu, Ya Wang, Jiao Ran, Xiao Liang. Chao Feng, et al. Vgr: Visual grounded reasoning. arXiv preprint arXiv:2506. 1991, 2025b.
Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, et al. Visualprm: An effective process reward model for multimodal reasoning. arXiv preprint 2xXiv:2503.1022) 2025c.
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internv13. 5: Advancing open- source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:25083 18265, 2025d.
Zhaochen Wang, Bryan Hooi, Yiwei Wang, Ming- Hsuan Yang, Zi Huang, and Yujun Cai. Text speaks louder than vision: ASCII art reveals textual biases in vision- languaeee models. In Second Conference on Language Modeling, 2025e. URL htlltttss://ppezeee// haEyNVTL.sh
net forum?id-
13

---

Published as a conference paper at ICLR 2026
Zhenhailong Wang, Xuchang Guo, Sofia Stoica, Haiyang Xu, Hongru Wang, Hyeonjeong Ha, Xiusi Chen. Yangyi Chen, Ming Yan, Fei Huang, et al. Perception-awan aware policy optimization for mul- timodal reasoning. arXiv preprint arXiv:2507.0644
2025f.
Zhenhailong Wang, Xuehang Guo, Sofia Stoica, Haiyang Xu, Hongru Wang, Hyeonjeong Ha, Xiusi Chen. Yangyi Chen, Ming Yan, Fei Huang, et al. Perception-aware policy optimization for mul- timodal reasoning. arXiv preprint arXiv: 254404507.06488 2025g.
Lai Wei, Yuting Li, Kaipeng Zheng, Chen Wang, Yue Wang, Linghe Kong, Lichao Sun, and Weiran Huang. Advancing multimodal reasoning via reinforcement learning with cold start. arXiv preprint ar arXiv:2505.22334 2025a.
Yana Wei, Liang Zhao, Kangheng Lin, En Yu, Yuang Peng, Runpei Dong, Jianjian Sun, Haoran Wei, Zheng Ge, Xiangyu Zhang, et al. Perception in reflection. arXiv preprint arXiv:250.007165. 2025b.
Junfei Wu, Jian Guan, Kaituo Feng, Qiang Liu, Shu Wu, Liang Wang, Wei Wu, and Tieniu Tan. Re- inforcing spatial reasoning in vision- language models with interwoven thinking and visual draw- ing. arXiv preprint arXiv:2506.0990 599556 2025a.
Mingyuan Wu, Jingcheng Yang, Jize Jiang, Meitang Li, Kaizhuo Yan, Hanchao Yu, Minjia Zhang, Chengxiang Zhai, and Klara Nahrstedt. Vtool-r1: Vlms learn to think with images via reinforce- ment learning on multimodal tool use. arXiv preprint arXiv:2505.19255 2025b.
Jiaer Xia, Yuhang Zang, Peng Gao, Yixuan Li, and Kaiyang Zhou. Visionary-rl: Mitigating short- cuts in visual reasoning with reinforcement learning, arXiv preprint arXiv:2505.14667, 2025.
Jinheng Xie, Zhenheng Yang, and Mike Zheng Shou. Show-o2: Improved native unified multimodal models. arXiv preprint arXiv:2506. 15544, 2025.
Guowei Xu, Peng Jin, Ziang Wu, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.106 2024.
Yi Xu, Chengzu Li, Han Zhou, Xingchen Wan, Caiqi Zhang, Anna Korhonen, and Ivan Vulić. Visual planning: Let's think only with images, 2025. URL htttps://arxi..ortt /bs/
2505. 11409
Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et R1-onevision: Advancing generalized multimodal rea- soning through cross-modal formalization. arXiv preprint arXiv:2503. 10615, 2025a.
Zeyuan Yang, Xueyang Yu, Delin Chen, Maohao Shen, and Chuang Gan. Machine mental imagery: Empower multimodal reasoning with latent visual tokens. 2025b. URL https://arxiv:
borg/abs/2e 2tce//bb/25066.... 2506. 17218
En Yu, Kangheng Lin, Liang Zhao, Jisheng Yin, Yana Wei. Yuang Peng, Haoran Wei, Jianjian Sun, Chunrui Han, Zheng Ge, et al. Perception-rl: Pioneering perception policy with reinforcement learning. arXiv preprint arXiv:2504.00995 2025.
Guanghao Zhang, Tao Zhong, Yan Xia, Zhelun Yu, Haoyuan Li, Wanggui Hc, Fangxun Shu, Mushui Liu, Dong She, Yi Wang, et al. Cmmcot: Enhancing complex multi-image comprehension via multi-modal chain-of-thought and memory augmentation. arXiv preprint arxiv:2503.05255, 2025a.
Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vI: Learning to reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arXiv:2503. 2025b.
Qizhe Zhang, Aosong Cheng, Ming Lu, Renrui Zhang, Zhiyong Zhuo Jiajun Cao, Shaobo Guo, Qi She, and Shanghang Zhang. Beyond text-visual attention: Exploiting visual cues for effective 2025c.
token pruning in vlms. arXiv preprint 2rX1v:24/2.01811
14

---

Published as a conference paper at ICLR 2026
Xintong Zhang, Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaowen Zhang, Yang Liu, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, et al. Chain-of -fccus: Adaptive visual search and zooming for multimodal reasoning via rl. arXiv preprint arXtv:2505.1543. 2025d.
Yuhui Zhang, Alyssa Unell, Xiaohan Wang, Dhruba Ghosh, Yuchang Su, Ludwig Schmidt, and Serena Yeung-Levy. Why are risuallygrrundee language models bad at image classification? Conference on Neural Information Processing Systems (NeurIPS), 2024.
Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.0002.. 2023.
Ziwei Zheng, Michael Yang. Jack Hong. Chenxiao Zhao, Guohai Xu, Le Yang. Chao Shen, and Xing Yu. Deepeyes: Incentivizing" thinking with images" via reinforcement learning. arXiv preprint arXiv:2505.14362, 2025.
A APPENDIX
A.1 USAGE OF LLMS
LLMs were used in this research project for coding assistance and writing support. Specifically, they were employed to generate helper functions, implement data loaders, proofread text, and suggest ETXX formatting.
15


Reviews:
[{"review_id": "eejFQ2rvOT", "reviewer": "Reviewer_T9HU", "summary": "The paper introduces Latent Visual Reasoning (LVR), a novel paradigm that enables multimodal large language models to perform reasoning directly within the visual embedding space, rather than relying solely on textual chain-of-thought reasoning. LVR allows the model to interleave latent visual reasoning with text generation, jointly processing both modalities in a shared semantic space. The authors propose a two-stage training pipeline combining supervised finetuning for visual reconstruction and reinforcement learning with an adapted GRPO algorithm to refine reasoning dynamics. Experimental results show that LVR significantly improves fine-grained visual perception, spatial understanding, and robustness on benchmarks such as MMVP and V*, outperforming previous “Think about” and “Think with” image approaches.", "questions": "N/A", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is very well-written and logically structured, making the core ideas easy to understand. The figures are clear and effectively illustrate the main concepts.\n\n2. The motivation is straightforward and reasonable.", "weaknesses": "1. The paper mostly extends the idea of latent reasoning from text-only LLMs to multimodal models. While this is a reasonable step forward, it feels more like a technical adaptation than a fundamentally new idea. The authors could do a better job explaining why reasoning in the visual latent space offers a distinct advantage rather than just being a straightforward extension of existing methods.\n\n2. The approach relies heavily on Visual CoT data and also requires identifying regions of interest (ROIs) for training, which makes it conceptually close to Visual CoT itself. However, there’s no head-to-head comparison with Visual CoT models in terms of both performance and inference cost. Without this, it’s hard to judge how much the proposed method actually improves over existing baselines.\n\n3. Because the reasoning happens in the latent space, it’s not clear what the model is actually doing during the latent visual reasoning phase. The paper doesn’t provide any qualitative examples or visualizations of these hidden processes, which makes the claimed reasoning mechanism somewhat opaque.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8mtEpVGGlC", "reviewer": "Reviewer_Zdot", "summary": "This paper proposes latent visual reasoning for MLLMs, which supervises an autoregressive process on latent states with reconstruction of ground-truth visual semantic embeddings, resulting better results than previous \"think about images\" and \"think with images\" methods. The training stages include SFT with next token prediction combined with reconstruction, and a GRPO stage compatible with the generated latent sequences. The paper further investigates different decoding strategies and concludes that the method works best with a fixed number of reasoning steps.", "questions": "1. How can the RL process improve the latent generation process? As I see in (5), the latents seem to be used as inputs with no gradients.\n2. I find the \"LVR with heads\" confusing in the ablation section. Is it just some additional mapping parameters? Why is the performance worse with more parameters involved?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The framework is novel by directly using MSE on hidden states and incorporating GRPO with modifications.\n2. The paper is well-organized and clearly presented.\n3. The experimental results are achieved by carefully controlling the conditions, making the performance advantage convincing.", "weaknesses": "Although the training stages incorporates RL, it can not avoid using critical region annotation, as GRPO yields only marginal improvement, while the MSE supervision on latents brings clear advantage over vanilla SFT. The latents are supervised only by the region semantics from the input image, which seems more like learning a visual grounding. Thus, the novelty advantage might be limited compared to other works proposing latent visual reasoning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3pY0EH4gbg", "reviewer": "Reviewer_9pGd", "summary": "The paper introduces Latent Visual Reasoning (LVR) — a new paradigm for multimodal reasoning that allows autoregressive reasoning directly in the visual embedding (latent) space, rather than relying solely on textual Chain-of-Thought (CoT) reasoning or external visual tools. Experiments on visual reasoning and perception-intensive benchmarks (e.g., V*, MMVP, BLINK) show significant improvements over both “Think about Images” (text-only CoT) and “Think with Images” (external tool-based) baselines. LVR achieves up to 71.7% on MMVP, outperforming Qwen2.5-VL and PixelReasoner.", "questions": "1. Can you provide qualitative examples (e.g., t-SNE plots or attention maps) showing what information the latent visual reasoning reconstructs?\n2. Could alternative stopping mechanisms (e.g., energy-based thresholds or learned confidence) improve variable-length reasoning stability?\n3. Does LVR introduce latency at inference time due to latent-space propagation?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The concept of latent visual reasoning is novel and well-motivated.\n2. The methodology is sound: clear architectural modifications, well-defined training objectives, and careful integration of latent-space reconstruction.\n3. Experiments are comprehensive across multiple benchmarks and baselines (Qwen2.5-VL, PAPO, Vision-R1, PixelReasoner).\n4. The work offers a scalable alternative to tool-based multimodal reasoning, reducing dependency on external APIs and handcrafted operations.", "weaknesses": "1. While conceptually strong, the paper provides little visualization or introspection of the latent reasoning process (e.g., what latent reconstructions “look like”). Future work could include latent trajectory visualizations or probing to show what is being “reasoned” in the latent space.\n2. The reliance on a fixed latent reasoning length may restrict flexibility and generalization to variable reasoning complexity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "k6pOAB8KA1", "reviewer": "Reviewer_eavF", "summary": "The paper presents Latent Visual Reasoning (LVR), a novel method that allows multimodal models to reason directly with visual embeddings. Unlike existing models that reason with text or use external tools, LVR operates autoregressively within the visual space itself. The model learns to reconstruct important visual information related to a query by generating a sequence of latent states. This capability is developed in a two-stage training process, which starts with supervised fine-tuning and is later refined using reinforcement learning based on a modified GRPO algorithm. Experiments confirm the strength of this approach, showing it surpasses other methods on demanding visual perception benchmarks like MMVP and V*.", "questions": "1. The current evaluation focuses almost exclusively on perception-based tasks. However, this leaves its performance on more complex visual reasoning tasks as an open question. Could the authors comment on the applicability of LVR to benchmarks like MathVista, which require more symbolic or multi-step reasoning? It is important to understand whether LVR's core mechanism of reconstructing visual tokens is sufficient for tasks that depend less on recognizing pixels and more on abstract logical deduction.\n\n2. If the model focuses on an incorrect Region of Interest in an early step, does it have any mechanism to detect and correct this error in subsequent steps?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The primary strength of this paper lies in its introduction of Latent Visual Reasoning (LVR), a fundamentally new and well-motivated paradigm for multimodal reasoning. By enabling the model to reason directly within the visual embedding space, it provides an elegant alternative to purely text-based or tool-dependent approaches.  Furthermore, the methodology is well-conceived, featuring a logical two-stage training pipeline and a clever technical adaptation of reinforcement learning (GRPO_latent) to handle reasoning in a continuous latent space.", "weaknesses": "While the method excels at visual perception, its applicability to complex visual reasoning tasks remains unclear. For instance, in geometry problems where auxiliary lines are crucial, the practical value of the model's generated visual tokens is questionable and requires further validation. The work would be significantly strengthened by evaluations on more challenging reasoning benchmarks, such as MathVista, and by including more in-depth visual analyses that clearly illustrate the impact of these visual tokens on the reasoning process and its results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hOXNUlxRPG", "reviewer": "Reviewer_w9J6", "summary": "The paper presents Latent Visual Reasoning (LVR), a method that enables multimodal large language models to reason directly within their latent visual embedding space. LVR introduces a mechanism for switching between normal text generation and a special latent reasoning mode, indicated by specific tokens. In this mode, the model manipulates internal visual representations relevant to the question before producing text outputs. The training process has two stages: supervised fine-tuning using bounding-box annotations to align latent states with image regions, and reinforcement learning that refines reasoning quality.\n\nExperiments are conducted on Qwen2.5-VL backbones at 3B and 7B scales. LVR shows the largest gains on fine-grained visual reasoning benchmarks such as MMVP, MMStar, and MM-Vet, compared to both “think-about” (text-only reasoning) and “think-with” (external visual reasoning) baselines. It performs particularly well on tasks requiring precise spatial, compositional, or attribute-based reasoning. The paper also introduces new decoding strategies for latent reasoning, comparing fixed-length and adaptive termination approaches. Training efficiency and decoding stability are analyzed through detailed ablations.", "questions": "See questions above", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Simple modification of standard MLLM pipelines without external tools.\n2. Adapts reinforcement learning to hidden-state spaces effectively.\n3. Uses fixed encoders and adaptive batching for efficient multi-step reasoning and conceptually extends beyond CoT into “machine mental imagery.”\n4. Open weights and code for reproducibility.", "weaknesses": "1. Gains are strongest on perception-heavy benchmarks, but less so on others (e.g., Relative Reflect).\nQuestion: Have the authors tested multi-image or compositional reasoning tasks? Would LVR generalize beyond single-image settings?\n\n2. The variable-length LVR decoding strategies (LatentEnd Token, Mode Switching Loss) perform inconsistently; only the fixed-token variant works reliably.\nQuestion: Why does the model fail to self-terminate properly? Could an adaptive controller or entropy-based stopping improve this?\n\n3. The paper compares against open models but omits newer ones like LLaVA-OneVision or InternVL 3.5, which are directly relevant.\nQuestion: How does LVR compare to these recent omni-modal reasoning models?\n\n4. Since LVR heavily relies on Visual-CoT training data, there’s risk it memorizes dataset patterns instead of general reasoning.\nQuestion: Have they tested on out-of-distribution datasets to measure transfer robustness?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)