MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning
Authors: Zheng Jiang
Heng Guo
Chengyu Fang
Changchen Xiao
Xinyang Hu
Lifeng Sun
Minfeng Xu
Venue: ICLR 2026 Poster (2026)
Keywords: Think with images
Medical visual reasoning
Medical VQA
Agentic reinforcement learning
Primary area: foundation or frontier models, including LLMs

Abstract: Medical Vision-Language Models (VLMs) hold immense promise for complex clinical tasks, but their reasoning capabilities are often constrained by text-only paradigms that fail to ground inferences in visual evidence. This limitation not only curtails performance on tasks requiring fine-grained visual analysis but also introduces risks of visual hallucination in safety-critical applications. Thus, we introduce MedVR, a novel reinforcement learning framework that enables annotation-free visual reasoning for medical VLMs. Its core innovation lies in two synergistic mechanisms: Entropy-guided Visual Regrounding (EVR) uses model uncertainty to direct exploration, while Consensus-based Credit Assignment (CCA) distills pseudo-supervision from rollout agreement. Without any human annotations for intermediate steps, MedVR achieves state-of-the-art performance on diverse public medical VQA benchmarks, significantly outperforming existing models. By learning to reason directly with visual evidence, MedVR promotes the robustness and transparency essential for accelerating the clinical deployment of medical AI.

Published as a conference paper at ICLR 2026
MEDVR: ANNOTATION-FREE MEDICAL VISUAL REA- SONING VIA AGENTIC REINFORCEMENT LEARNING
Zheng Jiang' Heng G Chengyu Fang 1.2
Changehen Xiao
Xinyang Hu
Lifeng Sun 1.4f
Minfeng Xu
Tsinghua University DAMO Academy, Alibaba Group Hupan Lab, 310023, Hangzhou, China
Key Laboratory of Pervasive Computing, Ministry of Education *Zhejiang University
jz24 @mails tsinghua. edu, en, chengyufang- t.uu com s sunifet singhua edu. (gh205191, eric. xmf) ) @alibaba -ina com com
ABSTRACT
Medical -angoneLanguage Models (VLMs) hold immense promise for complex clinical tasks, but their reasoning capabilities are often constrained by text-only paradigms that fail to ground inferences in visual evidence. This limitation not only curtails performance on tasks requiring fine-graines visual analysis but also introduces risks of visual hallucination safety-critical applications. Thus, we in- troduce MedVR, novel reinforcement learning framework that enables annotation- free visual reasoning for medical VLMs. Its core innovation lies two synergistic mechanisms: Entropy-guided Visual Regrounding (EVR) uses model uncertainty to direct exploration, while Consensus-based Credit Assignment (CCA) distills pseudo-supervision from rollout agreement. Without any human annotations for intermediate steps, MedVR achieves state-of-the-ee performance on diverse public medical VQA benchmarks, significantly outperforming existing models. By learn- ing to reason directly with visual evidence, MedVR promotes the robustness and transparency essential for accelerating the clinical deployment of medical AI.
INTRODUCTION
Medical Vision-Language Models (VLMs) are increasingly applied clinical tasks such as computer- aided diagnosis, report generation, and decision support (Chen et al., 2024b; Xu et al., 2025; Hartsock & Rasool, 2024). Achieving reliable performance in these tasks requires models to perform multi-step reasoning that combines fine-grained visual analysis with medical expertise. Recent advances have demonstrated that Reinforcement Learning with Verifiable Rewards (RLVR) can substantially bolster the reasoning capabilities of eeneral-purpose VLMs (Guo et al., 2025). Inspired by this success, there growing interest in adapting RLVR endow medical VLMs with more robust reasoning abilities.
However, the prevailing RLVR paradigm remains almost exclusively confined to the textual domain, imposing fundamental limitations at odds with the demands of clinical practice (Lai et al., 2025; Pan et al., 2025; Dou et al., 2025). First, many diagnostic challenges. including the localization of subtle lesions, comparison of tissue densities, interpretation of hemodynamic flow, and quantification of anatomical structures, necessitate level of fine-graine visual grounding that a text-only chain-of- thought cannot provide (Liu et al., 2025a). The absence of direct interaction with visual data during the reasoning process severely curtails the model's ability to exploit the rich information embedded in medical images. Second, this reliance on text-only reasoning renders models susceptible to visual hallucinations and language shortcuts, where answers are generated from textual priors while ignoring visual information (Huang et al., 2024). Such failures introduce unacceptable risks in safety-critical medical applications. In stark contrast, human clinicians dynamically interact with images, zooming. adjusting windowing, defining regions of interest, and measuring findings to formulate and validate hypotheses. This discrepancy motivates a central research question: Can we empower medical VLMs to perform complex reasoning by naturally interleaving textual deliberation with image manipulation tools, thereby substantiating key analytical steps with verifiable visual findings?
"Equal Contribution, CCoresponning Author.

---

Published as a conference paper at ICLR 2026
Qwoilew
Verifibbee Visual Ressoning vi Toplicke!
Perfermance on medical VQQ beechnurke
Caveitie CT image-eggeu
proweoool craon Deptiies?
<chiko To vouss whetherihii CTT slee saggeds onner. soca jo rhnaaly imppec the inage de focal maeso, aupkinso mnunst-fheojamm toians imgila consitaationn pleasal desommaliiss -iiike
Opions
OMETQA
TQURAD
p'aase" Tanage aores "aeperenae" "ubef"
"theke Lodkerg at the CT image of txe lasas there are mndippl
while speasard megile ber ctepes sortane mosrggoee halt lang
scheol Lodlo-
dekb. rocliaal bmuspre, dhae beggtt spuir ollen indiair
<ireeeer X
<tiinko
Lock of Rediad Damain crowledgs
trege argrex
infocice.
<dankk right lang daplayad There vonkii syges
mesees opocites sign
<thirk
crol, palto ("axre" "maagy poam, in tool"
"apparetta" MMaoe 95, 784,476
evelatt
sinne- The aasal chast CV scan in the kay waskox: to
pulnssary
moses
sheel
endilor
she Alher
-thikk bay digtteed
Mactoga
Widneal abiliy sloule reslve fae mazgrr, fio
atrecves laste arrrainee for pritary mallyrone Eikis div Annei alaanesiig the aea jaggono pestnoiall prososer of cance vittas-
muascs omaatiios infectioon sealyas of bangsorggess the CTT son showiing ao aboormalities -thakk
Std
Slumwer-X
Text only Recsonnng Hallaciatiion
Nat OOD
Figure 1: MedVR enables verifiable visual reasoning via tool-use. In contrast to the visual hallucinations prevalent in text-only reasoning (left), MedVR (middle) interleaves textual deliberation with active image manipulation to achieve precise visual grounding. This synergy results in superior aceuracy and robustness across multiple public medical VQA benchmarks (right).
Although frameworks enabling interleaved language and tool-based image operations have emerged in general computer vision (Zheng et al., 2025; Su et al., 2025), their direct translation to the medical domain is fraught with challenges. The primary obstacle lies in the critical need for domain adaptation. purepoe VLMs, trained vast corpora of internet images, lack the specialized knowledge to connect medical imagery with clinical concepts and fail to generate reliable zero-shot localizations for subtle pathological findings (Xu et al., 2025; Kalpélhé al., 2025). Consequently, they typically require extensive, fine-grained supervision of intermediate steps to learn meaningful visual grounding (Liu et al., 2025a). However, this requirement creates a paradox as the annotations needed for such intermediate supervision are notoriously scarce and expensive in medicine, rendering the training and validation of explicit visual reasoning process prohibitively difficult.
To bridge this gap, we introduce MedVR, an end-to-end reinforcement learning framework that, to our knowledge, the first to achieve annotation-free visual reasoning for medical VLMs. As shown in Figure 1, MedVR empowers VLM to seamlessly interleave textual chain-of-thought reasoning with the use of image-manipulation tool. cireumvents the need for supervisory labels by augmenting verifiable end-task rewards with two novel mechanisms that provide fine-grained, self-generated supervision. The first, Entropy-guided Visual Regrounding (EVR), leverages the model's intrinsic predictive uncertainty to guide exploration. By monitoring token-level entropy during inference, EVR identifies moments of high uncertainty, signaling ambiguity in grounding, and triggers targeted exploration from these states. This adaptive branching diversifies reasoning trajectories precisely where the model is least confident, encouraging it to "re-ground" its attention on the image before proceeding. The second mechanism, then synthesizes information from the ensemble of trajectories generated by EVR. By aggregating the grounding boxes produced across diverse rollouts, CCA distills a high-confidence consensus target. pseudo-label, providing a fine- aineggrained supervisory signal
munsensus-based Credit Assignment (CCA),
This consensus serves as a self-generaae for visual operations without any human annotation. In essence, EVR acts as a prior exploration strategy guided by uncertainty, while CCA provides posterior supervision distilled from collective agreement. This synergy creates an entirely unsupervised curriculum for visual reasoning, where exploration is focused on critical decision points and in-batch consensus provides actionable feedback.
We conducted rigorous and thorough evaluation to demonstrate the capabilities of MedVR. Our framework was benchmarked against a comprehensive suite of public medical Visual Question An- swering (VQA) datasets. MedVR consistently achieves state-of-the-at results across both in-domain and out-of- scenarios, thereby demonstrating its generalizability and superior reasoning capa- bilities. Furthermore, a series of ablation studies validate the necessity and substantial impact of both EVR and CCA. Our principal contributions are summarized as follows:
(1) We introduce MedVR, the first end-to-end reinforcement learning framework that integrates visual and textual reasoning for medical VLMs, eliminating the need for costly intermediate supervision.
2

---

Published as a conference paper at ICLR 2026
(2) We propose two novel, label-free mechanisms: Entropy-guided Visual Regrounding (EVR) for uncertainty-awarr exploration and Consensus-bause based Credit Assignment (CCA) for self-ggeerated supervision, which collectively create self- superviini curriculum for fine-grann fine-grained visual reasoning.
(3) We establish new slate-of-the-an results on multiple public medical VQA benchmarks, showcasing MedVR's broad applicability, robustness, and enhanced reasoning capabilities in the medical domain.
RELATED WORK
Medical Reasoning with Vision-Language Models. Vision-Language Models (VLMs) have shown remarkable potential in multimodal medical analysis by jointly reasoning over clinical texts and medical images (Hartsock & Rasool, 2024; Xu et al., 2025; Dai et al., 2025; Fang et al., 2026). Pioneering works, including LLa (Li et al., 2023b), Med-Flamingo (Moor et al., 2023), and HuatuoGPT- Vision (Chen et al., 2024a), have demonstrated the efficacy of adapting general-purpose curated image-text datasets (Hu et al.,
VLMs to the medical domain via supervised fine-tuning 2024; Zhang et al., 2023). More recent efforts such as Med-R1 (Lai et al., 2025) and MedVLM- RI (Pan al., 2025) have incorporated reinforcement learning to further improve the robustness and generalization of the models' reasoning capabilities. However, these advances predominantly focus on generating textual chains of thought, neglecting explicit visual interaction during the reasoning process. To the best of our knowledge, our work is the first to endow medical VLMs with explicit visual reasoning capabilities, which we define as the ability to agentically interleave textual deliberation with executable image-manipe manipulatinnn actions. While powerful general models like GPT-40 possess strong visual understanding, they do not exhibit this interactive, tool-augmented reasoning loop. MedVR models a process analogous to how clinicians progressively refine their analysis through active visual inspection. This design promotes more interpretable and verifiable decision-makin aligning the model's s behavior more closely with established clinical workflows.
General-Purpon Visual Reasoning. Concurrently, research in the general domain has explored augmenting VLMs with capabilities for active perception. Works such as DeepEyes (Zheng et al., 2025), Pixel-Reasoner (Su et al., 2025) and Chain-of- (Zhang et al., 2025b) enable iterative visual operations like zooming and region-of-interes selection to refine evidence gathering on natural images. However, their direct translation into the medical domain is impeded by the inherent com- plexity of clinical images, which exhibit subtle pathologies, speeefit artifacts, and diverse object scales (Xu et al., 2025). Furthermore, these methods typically presuppose the availability of fine-grained grounding annotations (e.g., bounding boxes) for cold start. This prerequisite is often infeasible in the medical field due to the high cost and specialized expertise required for annota- tion (Liu et al., 2025b). MedVR directly confronts this limitation. By obviating the need for any explicit grounding supervision, our framework not only alleviates the prohibitive annotation burden but also significantly advances the clinical viability of interactive medical VLMs.
METHODOLOGY
3.1 PRELIMINARIES
Training Objective. The training of MedVR is cast as policy optimization problem within reinforcement learning (RL) framework. The VLM acts as an agent governed by policy na. which optimized to maximize the expected cumulative reward obtained from the generated reasoning trajectories. The objective is formally defined as:
max E(q. (R(T) - 3Da ( || xee(-11,, DI)
Here, (Q, f) denotes a question-image pair from the dataset D, and T represents complete reasoning trajectory sampled from the policy Ta. R(T) is the terminal reward for the trajectory, and the KL- divergence term, scaled by 3, penalizes drastic deviations from the reference policy Tee-
Rollout Formulation. Departing from conventional models that generate monolithic textual output, our agent constructs a dynamic reasoning trajectory by meticulously interleaving textual deliberation with explicit visual operations. The agent's 's action space encompasses both the generation of tokens for its chain-of of-thuthht and the invocation of specialized tool commands. In this work, we focus on fundamental yet powerful visual operation: Zoom-ing When Zoom-i command, including
3

---

Published as a conference paper at ICLR 2026
Ieputs
MedVR: AnnoidiionFree Medical Visual Reasoning via Agentic Reinfarcement Learning
Ioputs
Entropy-guidee Regrounding
Consersusi-be Credit Assignment
Group
corornaliy by tuait bo
Medica
Policy Ophimization
atserved in the ngg?ling?
Vitual
Ermmyy Step
Trepedary End
Vieue Regoouccing
Indiibual Grounding Nip
Conneraue
Acaracy flevars
Roward
aisoimally can abserved lung?
Figure 2: Overview of MedVR. The framework employs EVR to explore visual actions based on the model's ' intrinsic uncertainty and CCA to create a consensus sasees reward from successful trajectories, enabling annotation-free training of medical visual reasoning.
its precise coordinates, is fully generated, external tool is triggered execute the corresponding action (i.e., cropping a specified image region). The newly acquired visual information then encoded into a set of special tokens and integrated back into the agent's 's context. This grounded visual evidence conditions all subsequent reasoning and action steps. This iterative process of deliberation, action, and observation continues until the agent produces final answer and terminates the rollout.
Reward Design. To guide the agent's 's learning in the absence of fine tne-grained, step-by-ster supervision, we employ composite terminal reward function R(T). This function is designed to provide holistic evaluation of the entire trajectory and is defined as:
R(T)= = Race (7) + Riormat (T) + (Rase (T) > 0) Ruol (T)
The total reward comprises three components: a primary accuracy reward (Race) based on the correctness of the final answer, minor format penalty (Roomat) for syntactically invalid malformed outputs; and a crucial tool-use reward (Rwat). The indicator function 1(.) means that the tool reward is conferred only if the trajectory successfully leads to correct answer. This conditional structure vital for incentivizing the model to discover meaningful causal relationships between its visual actions and successful outcomes, thereby discouraging spurious ineffective tool use.
3.2 ENTROPY-GUIDER VISUAL REGROUNDING
formidable challenge in training medical VLMs for intricate diagnostic tasks lies in guiding the agent perform efficient and effective visual exploration. Without explicit supervision, an agent may struggle to learn where to look, often resorting to random or repetitive visual actions that are computationally expensive and yield little information. To overcome this, we introduce Entropy- guided Visual Regrounding (EVR),, novel exploration strategy that leverages the model's 's intrinsic uncertainty as a self-supervisory signal dynamically guide the visual search process.
The fundamental premise of EVR is that moments of elevated predictive uncertainty during tool command generation correspond critical junctures where the model lacks sufficient confidence in its impending visual action. When generating Zoom-in command, increase in entropy during the specification of coordinate tokens may indicate that while the model has identified the necessity for visual inspection, it remains uncertain about the precise region-of-interest (ROI), Rather than committing prematurely to a single, potentially suboptimal action, EVR harnesses this dynamic uncertainty signal to trigger parallel exploration of multiple visual hypotheses. This mechanism provides an inherent, label-free form of supervision for the visual operation itself, compelling the model explore diverse search strategies precisely at moments of heightened predictive uncertainty.
4

---

Published as a conference paper at ICLR 2026
Uncertainty Evaluation. To operationalize this principle, we continuously monitor the token-level entropy of the policy Te as it parameterizes a visual action. The uncertainty at each generation step t is quantitatively measured by the entropy of the next-token probability distribution:
M Σ Pt. log.t.
H2
where Pt mo(-1S Softmax
(3)
T
Here, Siz, denotes the sequence of previously generated tokens, V is the model's vocabulary, 24 E RR M the pre-softmax logits, and I is the decoding temperature. At the start of generation, we compute a baseline entropy Hame as the mean entropy over the initial tokens, capturing the model's baseline uncertainty. Subsequently, during the generation of tool-relevant tokens, we continuously compute the rolling mean entropy Hood over the window. The key signal we track is the entropy increase relative the baseline: A/ooll = Hool Husee A significant positive A Hoot serves robust indicator of escalating spatial uncertainty, signifying diminishing confidence in selecting a specific ROI and signaling an opportune moment to trigger an exploratory action.
Adaptive Branching. This dynamic uncertainty signal governs our exploration strategy through an adaptive branching mechanism. For each input, the total rollout budget divided equally: half of the trajectories are generated initially to form base set, while the remaining half are reserved for targeted exploration. Here, we define the branch probability P Pouse + H where Poase is base sampling probability to ensure a minimum level of exploration and 7 is the entropy weight that controls the influence of the measured entropy increase on the branching decision (Dong et al., 2025). During the generation of the base trajectories, while tool's parameters are being decoded, EVR initiates a branching event stochastically with probability P. This process involves forking the current generative state and expending one unit from the discretionary budget initiate new, parallel reasoning path from this specific point of high uncertainty. Consequently. different branches can sample distinct coordinate values for the visual tool, effectively exploring multiple visual actions stemming from a shared reasoning context. This ensures that the exploration budget is strategically allocated to resolve the agent's spatial uncertainty at critical moments. The output of the EVR stage is heterogeneous ensemble of M trajectories, which collectively embody diverse set of visual search hypotheses rooted in the model's own predictive uncertainty. This targeted collection of visual explorations provides suitable foundation for our subsequent Consensus-basse Credit Assignment (CCA) mechanism, designed to distill a robust supervisory signal from the ensemble.
3.3 CONSENSUS-BASE CREDIT ASSIGNMENT
While EVR generates diverse exploratory trajectories, the fundamental challenge of credit assignment remains: How can we reward beneficial intermediate visual actions without ground-truth spatial annotations? A singular reward for the final answer is inherently insufficient as it fails to distinguish between coincidental successes and inferences derived from sound visual grounding. To address this, we introduce Consensus based Credit Assignment (CCA), novel mechanism that provides fine-grained, self- ceff-generated supervision for visual operations.
The core principle of CCA is harness the "wisdom of the crowd" by synthesizing information from the entire set of successful trajectories generated by EVR. The underlying assumption is that if multiple, diverse reasoning paths converge particular image region, then that specific region is highly likely to be causally relevant to the solution. By identifying this consensus, CCA distills high-confidence, dynamic pseudo-label for visual grounding, which in turn provides the basis for nuanced grounding reward.
correct answer while consistently inspecting
Consensus Generation. The CCA process begins by identifying the subset of successful trajectories. T-, from the ensemble produced by EVR. For each successful trajectory T, E T+, we define its visual footprint as the union of all bounding boxes from its invoked Zoom- i operations. This footprint is then rasterized into binary mask M, These individual masks are aggregated to form a consensus heatmap C, where the value at each pixel represents its inspection frequency across the successful set. This heatmap is subsequently binarized via a majority vote rule yield the final consensus mask M, which isolates regions inspected by a majority of successful trajectories:
M(,, v) = (C((,, 1TT+1/))) where C=
Σ M TET
(4)
5

---

Published as a conference paper at ICLR 2026
where 1(.) is the indicator function. The resulting mask M represents high-confidence hypothesis of the salient image region, derived entirely from the model's own successful explorations.
Credit Assignment. Utilizing the generated consensus mask M, we formulate the tool reward Rgol to explicitly encourage alignment with this collectively identified visual strategy. For each successful trajectory T E T we quantify its alignment with the consensus by computing the Intersection over Union (IoU) between its visual footprint M, and the consensus mask M. To create robust learning signal and mitigate noise inherent in pseudo-label generation, we employ threshold r/ on the IoU score. The final Rooo is defined as:
1.0, if IoU M) 0.5, otherwise
MM n nM [M, U M|
Reool (Tg)
where IoU
(5)
This tiered structure provides a baseline reward of 0.5 for achieving a correct outcome, while offering significant bonus of 0.5 for doing via visual strategy that is consistent with the collective wisdom of other successful paths. This formulation rewards trajectories not merely for their final accuracy, but more importantly, for the verifiability and consistency of their underlying visual process.
The synergy between EVR and CCA thus establishes a fully self-supervisory learning loop. EVR acts the priori explorer, dynamically generating diverse visual hypotheses by focusing on moments of model uncertainty. Subsequently, CCA serves as the a posteriori distiller, extracting the collective wisdom from successful explorations to forge a fine-grained, self-generated reward signal. This symbiotic process guides the agent towards developing robust and interpretable visual reasoning capabilities, entirely obviating the need for costly, human- cemaa--rovided intermediate supervision.
4
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Evaluation Benchmarks We evaluate the effectiveness of MedVR across six public medical VQA benchmarks. Following established conventions (Huang et al., 2025), we categorize these benchmarks into two types: (1) General-Domain Medical VQA: This category includes OmniMedVQA QAQ (Hu et al., 2024), PMC-VQA (Zhang et al., 2023), and MedXpertQA (Zuo et al., 2025) datasets. These benchmarks encompass a diverse range of modalities and medical scenarios, with all questions presented in multiple-choice format. (2) Modality-
Medical VQA: This category comprises benchmarks focusing on single imaging modality, including VQA-RAD (Lau et al., 2018) (radiology X-rays), SLAKE (Liu et al., 2021) (slit-lamp ophthalmology images), and VQA (He al., 2020) (pathology images). These benchmarks require open-ended, free-text answers.
Evaluation Protocol. For the general-domain multiple-choice tasks, we employ OmniMedVQA for training and in-domain testing, strictly adhering to the official data splits from Med-R1 (Lai et al., 2025). To assess out-of-domain (OOD) generalization, we conduct zero-shot evaluations on the test sets of PMC-VQA and MedXpertQA. For the modality-specific free-text tasks, the model is trained on a consolidated dataset of the VQA-RAD and SLAKE training splits, which constitute our in-domain benchmarks. We then evaluate its OOD generalization on VQt. Additional details regarding dataset statistics, evaluation metrics and baselines are provided Appendix F and A.
Implementation Details. We implement MedVR using the Qwen2.5-VI. VL--7B model as our backbone. The policy is optimized using the Group Relative Policy Optimization (GRPO) algorithm (Shao et al., 2024; Guo al., 2025) for 64 iterations a distributed setup of 32 H20 GPUs. During training, we use a batch size of 256 prompts. For each prompt, the agent generates 16 trajectories, with a maximum of 6 tool calls permitted. Key hyperparameters EVR, and 77 = 0.5 for CCA. In line with recent advancements in RLVR, we adopt the "clip-higher" mechanism (Yu et al., 2025) and ablate the KL-divergence and standard value normalization terms from the policy objective to stabilize training (Liu et al., 2025c).
set as follows: Pruse = 0.5, 0.5 for
4.2
MAIN RESULTS
As presented in Table 1, MedVR establishes a new state-of tf-the- (SOTA) across a comprehensive suite of medical VQA benchmarks, outperforming both general-purposs and domain-specifer vision- language models. The superiority MedVR can be critically analyzed from three pivotal perspectives: (1) Versatility Across Diverse Task Formats. MedVR demonstrates remarkable adaptability by

---

Published as conference paper at ICLR 2026
Table I: Comprehensive performance comparison on medical VQA benchmarks, divided into General- Domain (multiple-choice) and Modality-Specine (free-text) tasks. Out-of-domain (OOD) test sets are marked with The best and second-best results are highlighted in bold and underlined.
Settings
General- Domuin VQA (Mulli-choice)
Mlodality-Specilie VQA (Free-text)
Tasks
OMVQA PMC-VQA
MedXQA
Avg.
VQA-RAD
SLAKE
PatVVQA
Avg.
Perpral-Perpose VLMs
D (Bai et 2025a)
59.0
51.2
22.3
44.2
64.5
67.2
44.1
58.6
Qwen2.5 (Bai al., 2025a)
69.9
54.1
24.5
49.5
71.8
71.2
41.9
61.6
2024c)
IntemVLL3-BB (Zhu at. 2025)
81.4
53.8
22.1
52.4
65.4
72.8
48.6
62.3
IntemVL3-14B (2hu et al., 2025)
81.9
54.1
23.1
53.0
66.3
72.8
48.0
62.4
Medical Specific VLMs
Med-R1-2B et 2025)
47.4
21.1
MedVLM R1 2B (Pun
2025)
57.4
47.5
20.1
41.7
48.6
56.0
32.5
45.7
MedGemma-4B 4B (Sellergren et al, 2025)
70.5
49.9
15.4
45.3
72.5
76.4
48.8
65.9
LLaVA-Med (Li et ul., 2023a)
29.3
30.5
20.3
26.7
53.7
48.0
38.8
46.8
HustueGPT-N-78 (Chen
53.3
67.0
Lingshu-7B B (Xu al, 2025)
84.2
54.3
26.5
55.0
67.9
83.1
61.9
70.3
McdVR (Ours)
96.8
54.3
26.4
59.2
74.4
85.3
62.3
74.0
excelling in both nultiple-choice and free-text generation tasks. It not only achieves the highest score on the in-domain multiple-choic benchmark but also consistently outperforms all baselines on the more challenging free-text generation tasks. This consistent, top-tier performance across disparate answer formats indicates that the reasoning capabilities cultivated by MedVR are fundamental and generalizable, rather than being overfitted to specific question answering paradigm. This suggests our model learns to reason about visual content, not just to select the most probable option. (2) Exceptional Out-of- -it-of--Dmminn Generalization. A critical indicator of model's 's robustness is its ability to generalize to unseen data distributions. MedVR exhibits outstanding OOD performance, achieving SOTA or highly competitive results all designated OOD test sets. This strong generalization capability suggests that our annotation-fire training mechanisms effectively guide the model to develop core, transferable reasoning skill grounded in visual evidence, thereby mitigating the risk of overfitting to Pre-training. noteworthy achievement that MedVR's performance is not only SOTA but also rivals or exceeds that of models pre-trained substantially larger, domain-specific corpora. For example, MedVR consistently outperforms Lingshu-7B and the larger Intern VI.3-14 14B across average scores for both task categories. This outcome is significant, principled improvements in reasoning processes can be more impactful than simply scaling up pre-training data. It underscores the efficacy and efficiency of our approach, which achieves superior results by fostering a deeper, evidence-basee reasoning process rather than relying solely on the statistical patterns learned from massive datasets.
biases and priors. (3) Performance Rivaling Massive-Scale
demonstrates that targeted,
4.3 ABLATION STUDY
Main Components of MedVR. To dissect the contributions of cach component within MedVR, conduct systematic ablation study. Starting from textual RL baseline (Guo et al., 2025), we incrementally introduce our three core contributions (1) the Zoom-in visual tool, (2) the EVR exploration strategy, and (3) the CCA reward mechanism. The results presented in Table 2. First, we equip the agent with the Zoom-in tool without the guidance of EVR or CCA, which leads to slight per- formance drop on the OmniMedVQA and PMC-VQA. This outcome suggests that VLMs do not inherently possess the zero- shot capability to effectively utilize novel tools. Without a sophisticated reward sig-
Table 2: Ablation study for core components of MedVR.
Methods
Zoom
Components
EVR
CCA
OmmiMedVQA
Performance
VQA
MedKpereQA
Textual
94.50
nal exploration strategy, the tool can introduce unproductive search paths, disrupting the model's pre- pre-established reasoning processes. Next, we isolate the individual impacts of EVR and CCA by adding them separately to the Zoom-in baseline. Integrating EVR alone yields substantial improvements, with the most significant gains on OOD benchmarks. This highlights EVR's 'ss role in enhancing model robustness and generalization steering the agent resolve predictive uncertainty
7

---

Published as a conference paper at ICLR 2026
150
9677
Model Performannc
(a) Entropy Weight
(b) Reward Design of CCA
(c) Global Rollout Nums
Figure 3: Ablation study for key hyperparameters and reward design of MedVR.
through visual exploration. Conversely, adding only CCA provides the most substantial benefit on the in-domain OmniMedVQA dataset, confirming its effectiveness at distilling and reinforeing reliable visual grounding strategies. Finally, the full MedVR model achieves the best performance across all benchmarks. This result illuminates a powerful synergy: EVR acts as an eeertainty-awann explorer, proposing diverse visual hypotheses, while CCA functions as a meticulous self- seff-supervisor identifying and rewarding consensus-driven, high-quality reasoning paths. This symbiotic interplay establishes a virtuous cycle of exploration and credit assignment, which is instrumental in unlocking the model's capacity for complex visual reasoning and generalization.
Sensitivity to Entropy Weight in EVR. We analyze the sensitivity of the EVR module to its entropy weight hyperparameter, 7, which governs the influence predictive entropy on the tool-sampling process. As depicted in Figure 3(a), vary y and evaluate performance on OmniMedVVA. At 0, EVR defaults to a random sampling strategy, failing to leverage the model's s intrinsic uncertainty to guide exploration. As y increases, the model progressively prioritizes actions that reduce predictive entropy. leading to a monotonic performance improvement up to ? = 0.5. Beyond this point (7 > 0.5), performance begins to decline. We attribute this to an overly greedy strategy where an excessive reliance on the entropy signal stifles exploration diversity, preventing the model from discovering a broader range of effective reasoning paths. This analysis validates our hyperparameter choice and underscores the critical trade-off between exploitation and exploration managed by EVR.
Analysis of CCA Reward Design. We also dissect the reward design of CCA by comparing against three simplified variants, with results shown in Figure 3(b). The ablations are: (1) w/o Tool Reward, which removes any explicit reward for tool use; (2) Unconditional Reward, which provides a constant positive reward for any tool use, irrespective of final task accuracy; and (3) Default Reward, which grants a fixed reward only if the tool-using trajectory leads to a correct final answer. The results reveal a clear hierarchy in reward efficacy. The w/o Tool Reward setting fails to incentivize any visual exploration. The Unconditional Reward encourages tool use but fails to distinguish between beneficial and detrimental actions, as the reward is decoupled from task success. The Default Reward marks an improvement by linking tool rewards to final accuracy, but it treats all successful trajectories equally, lacking the granularity reward more precise visual grounding. In contrast, our CCA mechanism resolves this by assigning credit based on cross-trajectory consensus. This enables fine-grained supervision that rewards not only correct outcome but also the most reliable and replicable reasoning path to achieve it, proving crucial for the model's 's superior performance.
Scalability of the Overall Framework. To evaluate the scalability of the MedVR framework, we investigated the impact of global rollout numbers on model performance. The experiment was conducted on the OmniMedVQA dataset, with results presented in Figure 3(c). A clear positive correlation is observed: as the number of rollouts increases, the model's accuracy consistently improves. This trend demonstrates the scalability of our approach, as a larger rollout budget provides the CCA mechanism with richer sample of trajectories to distill more reliable pscudo-sus pscudo-supervision Furthermore, this consistent improvement highlights the framework's 's robustness and its ability to effectively leverage increased computational resources.
4.4 IN-DEPTH ANALYSIS OF EVR
The fundamental premise of EVR is that the model's predictive uncertainty during the parameter- ization of a visual tool-call serves as a reliable signal to guide exploration. Rather than triggering exploration randomly, EVR directs the agent to explore alternative visual actions precisely at moments of high uncertainty, thereby optimizing the search for relevant visual evidence. We conducted several analyses validate this hypothesis and demonstrate the multifaceted benefits of this approach.
8

---

Published as a conference paper at ICLR 2026
Entrpp
Aug. Pausap
Infod3
Training Stage
Tooiiing Steps
Figure Correlation between Figure 5: Average token entropy Figure 6: Per-step training time token entropy and mloU.
comparison with and w/o EVR. comparison with and w/o EVR.
Correlation between Entropy and Localization Quality. We first sought empirically validate our core assumption: that token-level entropy reliable proxy for the model's confidence in its visual grounding decision. To this end, analyzed all tool-call tokens generated by Qwen2.5-VL-7B on the GEMEX- Think (Liu et al., 2025a) dataset, which provides grounding box annotations for visual references. We computed the average generation entropy of these tokens across different intervals of Intersection over Union (loU) between the predicted and ground-trun bounding boxes.
As illustrated in Figure 4, we observe strong negative correlation: high-quality localizations (high mloU) consistently correspond to low-entropy generations, whereas high entropy is associated with inaccurate grounding. This finding confirms that entropy is valid indicator of grounding uncertainty, justifying its use to trigger exploration precisely when a visual action is likely to be inaccurate.
Impact Training Stability and Exploration. Beyond improving final performance, effective exploration strategy should also promote stable learning dynamics. To evaluate this, we monitored the average policy entropy throughout the training process for models with and without EVR. As shown in Figure 5, the baseline model's entropy rapidly decays, indicating premature convergence to suboptimal, coweexploration policy. contrast, EVR maintains a consistently higher and more stable level of entropy. This sustained exploration prevents the agent from getting stuck in local minima, facilitating the discovery more robust reasoning paths and leading to better performance gains.
Computational Efficiency of Tree-Search Exploration. In addition to improving performance and stability, EVR's adaptive tree-search mechanism offers significant computational advantages over naive exploration strategies. By forking new reasoning paths from shared context only at points high uncertainty, EVR avoids the redundant token generation inherent in sampling multiple independent trajectories. Consequently, the computational complexity is potentially reduced from Onn²) to (n n). As quantified in Figure 6, EVR significantly reduces the computational overhead compared to the independent sampling baseline. This efficiency renders sophisticated, targeted exploration computationally feasible.
4.5 GENERALIZATION ACROSS MODEL SCALES AND VARIANTS
To rigorously demonstrate that the contribution of our proposed framework independent of model scale specific variant, we conducted series of controlled ablation studies across multiple backbone models: Qwen2.5-VLL333 VL--3B, Qwen2.5-VL. VLL-7B, Qwen2.5-VL. For each backbone, we compared zero-shot baseline, a standard Textual RL approach, and our MedVR framework, using the exact same RL settings, rollout budget, and training corpora.
and Lingshu-7B on OmniMed VAA.
As shown in Table 3, MedVR consistently and significantly outperforms the Textual RL base- line across all model sizes. These results demonstrate that the ben- efits of MedVR are not contin- gent on model size or spe- cific variant, but stem from our annotation visual reasoning method itself. Notably, the gains are most substantial in OOD set- tings, suggesting that visual rea-
Table 3: Performance of MedVR across model scales and variants.
Model
Method
OmmiMedVQA
PMC-VQA
MedXpert(AA
Avg.
Zero-shot
45.5
Qwen2.5-V1-38
Textual
MedVR (Ours)
92.1
49.1
22.2
54.5
Zaro-shol
64.3
51.2
22.3
46.0
Qwen2.5-V1-7B
21,4
56.4
MedVR (Ours)
96.7
54.3
26.4
59.1
Zaro-shot
69.9
51.1
24.5
49.5
Qwen2.5-VL-32B
Textual RL
95.9
53.2
25.2
58.1
26.5
Zeto-shor
84.2
54.1
26.5
55.0
Lingshu-7B
Textual RL
95.7
53.7
25.1
58.3
MedVR (Ours)
96.0
26.5
61.1

---

Published as a conference paper at ICLR 2026
Question:
observed in this image?
A. Hone fracture
B. Torn ligament
C. Labral pathology D. Joint Inflammation
Answer:
<think The image appears an scon shoulder joint, given its Bony stricture soft rissue architecture: The high signal intonsity in the orea correspondngg to the joint labrum (the cirelc-like structure on the left side) suggesis pachological condition, which charstemssic of labral herr. The rppearance of the surounding structures
quality of image further suppect diagnosis lahaal pathologgy. Typically, this would not correspund to bons fracturee, tom ligument, or joint infiammation which would different mdiological fantures <ithinie>
ctool call- ("xame": "image 200ms in tool", "argaments" ('bbox 24" [23, 31, 129, 191], "labe"": "Joint protective structure") [] shool call--
<chnkk The soomed-in in section shover a circle-Iile arracture with abnorma! Righ signal inteuvityy which
Eytinally represenny parkologg: The intensity in this portion corresponds to the labnum in the shoulder,
consistem ubservation of loen or injured labrum. Given this, the answer clinical presentation visible on the MRI of the shoulder is C) Labeal pathology, <ithink>
fiis
"answer> <intswer
Question:
discase characterized outline the main features of this condition?
specifie ahnormality visible this image, can you
Monkeypox
Lyme discase
C. Mumps
Rubella
Answer:
image leston on the skin that appears to be a nodule with a cencal depression. which is common feature of monkgypor There
<tinks To determine disease,
to asalyze the image charaterrsttic
rubella, <ithink stool call> ("name": "imago goom in sool", "ssguments": ("bbox 24": (308, 506, 437, 659). "label": "lesion") HI <htol colli-
of the nush associated with Lyme discass,
<think- The zoomeed-ee view confiems the presence of nodule with central depression, which consistent with the clinical presenation monkeypox, There signs the bullseya rash Lyme discase, the parotid gland unlargement seen in mumpe, or the macukqupulaz rash seen in rubella. <thinkk canswer> A <lanswer>
Figure 7: Qualitative examples of MedVR's visual reasoning process.
soning enhances robustness to domain shift by grounding decisions in observable image features rather than relying on memorized patterns. The consistent improvements across diverse backbones, ranging from cineral-purpose Qwen models to dpemain-peciaalzzed Lingshu, highlight the universality of MedVR's 's design. The framework is not tied to a specific architecture but can be readily integrated into any VLM capable of autoregressive policy generation and visual tool invocation.
4.6 CASE STUDY
To provide qualitative insights into MedVR's reasoning process, we present representative studies in Figure 7. These examples showcase a dynamic, multi-step diagnostic workflow that mimics the iterative analysis of human clinical experts. As illustrated, the model typically initiates its analysis with a global overview of the image to formulate an initial hypothesis based on the user's query. Crucially, rather than prematurely arriving at conclusion, MedVR identifies regions of uncertainty or high clinical relevance that necessitate closer inspection. At these critical junctures, it proactively invokes the Zoon-in. tool for a targeted examination of the specific region of interest. The subsequent reasoning steps are then explicitly grounded in this newly acquired fine-grained visual evidence, leading to a final conclusion that is both accurate and visually verifiable. The visualizations confirm that MedVR's ability to localize and interpret clinically significant findings is highly precise.
This dynamic, iterative process stands in stark contrast to the static, single-pass analysis conven- tional VLMs. By actively interrogating the visual data to resolve ambiguity and verify hypotheses, MedVR mitigates the risk of overlooking subtle pathologies. Furthermore, this grounding in direct visual evidence reduces the model's reliance potentially unreliable textual priors, thereby curbing the tendency for hallucination. This capacity for self-correction and evidence-basee reasoning is fundamental enhancing the trustworthiness and reliability of Al in diagnostic applications.
CONCLUSION
This paper introduced MedVR, novel reinforcement learning framework designed to overcome the critical limitations of text-only reasoning in medical VLMs. By seamlessly interleaving textual deliberation with image-tool use, MedVR grounds its inferences in visual evidence without relying on costly intermediate annotations. Our comprehensive experiments demonstrate that MedVR achieves state-of-the-arr performance on diverse medical VQA benchmarks. The substantial gains over strong text-only baselines empirically confirm that learning to reason directly with visual data is essential for building robust and reliable medical AL.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Muhammad Awais, Tauqir Ahmed, Muhammad Aslam, Amjad Rehman, Faten S Alamri, Saeed Ali Bahaj. and Tanzila Saba. Mathvision: An accessible intelligent agent for visually impaired people to understand mathematical equations. IEEE Access, 2024.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.1333 13923,, 2025a.
Sule Bai, Mingxing Li, Yong Liu, Jing Tang, Haoji Zhang, Lei Sun, Xiangxiang Chu, and Yansong Tang. Univg-rl: Reasoning guided universal visual grounding with reinforcement learning. arXiv preprint ar arXiv:2505. 2025b.
Jierun Chen, Fangyun Wei, Jinjing Zhao, Sizhe Song, Bohuai Wu, Zhuoxuan Peng, S-H Gary Chan, and Hongyang Zhang. Revisiting referring expression comprehension evaluation the era of large multimodal models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 513-524, 2025.
Junying Chen, Chi Gui, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Chen, Xidong Wang, Zhenyang Cai, Ke Ji, Xiang Wan, al. Towards injecting medical visual knowledge into multimodal Iims at scale. In EMNLP, pp. 7346-7370, 2024a.
Junying Chen, Chi Gui, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji, et al. Huatuogpt-vision, towards injecting medical visual knowledge into multimodal Ilms at scale. arXiv preprint arXiv:2406. 19280, 2024b.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.00221, 2024c.
Noel Codella, Veronica Rotemberg, Philipp Tschandl, M Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, et al. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv preprint arXiv:1902. 0e0019992.033.,,. 2019.
Wei Dai, Peilin Chen, Chanakya Ekbote, and Paul Pu Liang. Qoq-med: Building multimodal clinical foundation models with domain- -wwrre grpo training. arXiv preprint arXiv:2506.0077 0071/, 2025.
Guanting Dong, Hangyu Mao, Kai Ma, Licheng Bao, Yifei Chen, Zhongyuan Wang, Zhongxia Chen, Jiazhen Du, Huiyang Wang, Fuzheng Zhang, et al. Agentic reinforced policy optimization. arXiv preprint arXiv:2507.19849 2025.
Chengfeng Dou, Chong Liu, Fan Yang. Fei Li, Jiyuan Jia, Mingyang Chen, Qiang Ju, Shuai Wang, Shunya Dang, Tianpeng Li, et al. Baichuan-m2: Scaling medical capability with large verifier system. arXiv preprint 0xXiv:2509.02208, 2025.
Chengyu Fang, Heng Guo, Zheng Jiang, Chunming He, Xiu Li, and Minfeng Xu. Photon: Speedup volume understanding with efficient multimodal large language models. In The Fourteenth International Conference on Learning Representations, 2026.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-rl incentivizes reasoning in llms through reinforce- ment learning. Nature, 645(808 1):63 -38, 2025.
Iryna Hartsock and Ghulam Rasool. Vision-language models for medical report generation and visual question answering: A review. Frontiers in artificial intelligence, 7: 7:1430984, 2024.
Xuchai Hc, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint arXiv:2003. 10286, 2020.
Yutao Hu, Tianbin Li, Quanfeng Lu, Wenqi Shao, Junjun He, Yu Qiao, and Ping Luo. Omnimedvqa: A new large-scale comprehensive evaluation benchmark for medical lylm. In CVPR, pp. 22170- 22183, 2024.
11

---

Published as a conference paper at ICLR 2026
Wen Huang, Hongbin Liu, Minxin Guo, and Neil Zhenqiang Gong. Visual hallucinations of multi- modal large language models. arXiv preprint (xXiv24422114683 2024.
Xiaoke Huang, Juncheng Wu, Hui Liu, Xianfeng Tang, and Yuyin Zhou, Medvlthinker: Simple baselines for multimodal medical reasoning. arXiv preprint arXiv:2508.02660 2025.
Haobin Jiang and Zongqing Lu. Visual grounding for object-level generalization in reinforcement learning. European Conference on Computer Vision, pp. 55-72. Springer, 2024.
Béria Chingnabé Kalpélbé, Angel Gabriel Adaambiik, and Wei Peng. Vision language models in medicine. arXiv preprint arXiv:2503.01865; 2025.
Yuxiang Lai, Jike Zhong, Ming Li, Shitian Zhao, and Xiaofeng Yang. Med-rl: Reinforcement learning for generalizable medical reasoning in vision-language models, arXiv preprint arXiv:2503.13999 2025.
Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman. A dataset of clinically generated visual questions and answers about radiology images. Scientific data, 5(1 5(1):1-10, 2018.
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-yee assistant for biomedicine in one day. NeurIPS, 11928541-285666 2023a.
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. In NeurlPS, volume 36, pp. 28541-2000 122564,, 2023b.
Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang. and Xiao-Ming Wu. Slake: A semantically- labeled showlelge-enhancee dataset for medical visual question answering. In 2021 IEEE 18th international symposium on biomedical imaging (ISBI), pp. 1650-1654. IEEE, 2021.
Bo Liu, Xiangyu Zhao, Along He, Yidi Chen, Huazhu Fu, and Xiao-Ming Wu. Gemex-thinkvg: Towards thinking with visual grounding in medical vqa via reinforcement learning. arXiv preprint arXiv:2506. 150.2999..7999 2025a.
Che Liu, Zheng Jiang, Chengyu Fang, Heng Guo, Yan-Jie Zhou, Jiaqi Qu, Le Lu, and Minfeng Xu. M3ret: Unleashing zero-shot multimodal medical image retrieval via self-supervision. arXiv preprint w:Xiv:2509.03300 2025b.
Zichen Liu, Changyu Chen, Wenjun Li. Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl -i-ero-likke training: A critical perspective. arXiv preprint arXiv:2503.20783 2025c.
Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chur Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning. arXiv preprint arXiv:2 105. 04655, 2021.
Pan Lu. Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li. Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Rei Chang. Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:23 10.0255, 2023.
Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovec, Cyril Zakka, Eduardo Pontes Reis, and Pranav Rajpurkar. Med- Med-iiami a multimodal medical few-shot learner. In Machine Learning for Health (ML4H), pp. 353-367. PMLR, 2023.
Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. Medvlm-rl: Incentivizing medical reasoning capability of vision- language models (vlms) via reinforcement learning. In MICCAI, 2025.
Shaohao Rui, Kaitao Chen, Weijie Ma, and Xiaosong Wang. Improving medical reasoning with curriculum-avare reinforcement learning. arXiv preprint arXiv:2505./92// 2025.
Gabriel Sarch, Snigdha Saha, Naitik Khandelwal, Ayush Jain, Michael Tarr, Aviral Kumar, and Katerina Fragkiadaki. Grounded reinforcement learning for visual reasoning. arXiv preprint arXiv:2505.23678. 2025.
12

---

Published as a conference paper at ICLR 2026
Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, et al. Medgemma technical report. arXiv preprint arXiv:2507.05201, 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, al. Deepseekmath: Pushing the limits of mathemat- ical reasoning in open language models. arXiv preprint arXiv.2402.03300, 2024.
Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: In- centivizing pixel-space reasoning with curiosity-driver reinforcement learning, arXiv preprint arXiv:2505.15966, 2025.
Jiacong Wang, Zijian Kang, Haochen Wang, Haiyong Jiang, Jiawen Li, Bohong Wu, Ya Wang, Jiao Ran, Xiao Liang, Chao Feng, et al. Vgr: Visual grounded reasoning. arXiv preprint 1xXiv:2506611991, 2025.
Xiaosong Wang, Yifan Peng. Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Summers. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classi- fication and localization of common thorax diseases. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2097-2106, 2017.
Lai Wei, Yuting Li, Chen Wang. Yue Wang, Linghe Kong, Weiran Huang, and Lichao Sun. Unsu- pervised post-training for multi-modal Ilm reasoning via grpo. arXiv preprint 2rXiv:2505.22453, 2025.
Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang, Chenghao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, et al. Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning, arXiv preprint arXiv:2506.0704 2025.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09335 2025.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476, 2025.
Qingyang Zhang, Haitao Wu, Changging Zhang, Peilin Zhao, and Yatao Bian. Right question is already half the answer: Fully unsupervised Ilm reasoning incentivization. arXiv preprint arXiv:2504.05812. 2025a.
Renrui Zhang, Dongzii Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal Ilm truly see the diagrams in visual math problems? European Conference on Computer Vision, pp. 169-186. Springer, 2024.
Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie. Pme-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint 2.Xi::3305.10415. 2023.
Xintong Zhang, Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaowen Zhang, Yang Liu, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, et al. Chain-of-focus, Adaptive visual search and zooming for multimodal reasoning via rl. arXiv preprint arXiv:2505. 2025b.
Xuandong Zhao, Zhewei Kang, Aosong Feng Sergey Levine, and Dawn Song. Learning to reason without external rewards. arXiv preprint urXiv:2505. 19590, 2025.
Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang. Chao Shen, and Xing Yu. Deepeyes: Incentivizing" thinking with images" via reinforcement learning. arXiv preprint ar0iv:2505. 14362, 2025.
13

---

Published as a conference paper at ICLR 2026
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.1047, 2025.
Yuxin Zuo, Shang Qu, Yifei Li, Zhangren Chen, Xuekai Zhu, Ermo Hua, Kaiyan Zhang, Ning Ding. and Bowen Zhou. Medxpertqa: Benchmarking expert-level medical reasoning and understanding. arXiv preprint arXiv:2501 18362, 2025.
14

---

Published as a conference paper at ICLR 2026
APPENDIX CONTENTS
Introduction
2
Related Work
3
Methodology
3.1
Preliminaries
3.2
Entropy-guided Visual Regrounding
3.3
Consensus-based Credit Assignment
Experiments
4.1
Experimental Setup
4.2
Main Results
4.3
Ablation Study
4.4
In-Depth Analysis of EVR
4.5
Generalization Across Model Scales and Variants
4.6
Case Study
10
5
Conclusion
10
Supplement Experimental Results
17
A.I Baseline Training Data Exposure and Fairness Evaluation
17
A.2 Quantitative Evaluation of Localization Quality
17
A.3 Efficacy of Consensus-Based Credit Assignment (CCA).
18
A.4 Domain-Agnostic Generalization to Non-Medical Tasks
18
A.5 More Visualization Results
19
A.6 Analysis of Learned Tool Usage Dynamics
20
The Algorithm Workflow of MedVR
20
C Supplement Related Work
20
C.1 Annotation-free Reinforcement Learning.
20
C.2 Visual-Grounded Reasoning
22
D Implementation Details
23
D.I Training Initialization
23
D.2 Tool Implementation
23
Evaluation Protocol and Metrics
24
Inference Settings
24
15

---

Published as a conference paper at ICLR 2026
E.1 Prompts
24
E.2 Inference Protocol and Parameters
25
F
Datasets
26
G Baselines
27
G.1 General-Purpose VLMs
27
G.2 Medical-Specifie VLMs
28
G.3 Fine-tuned VLMs
28
H Limitations and Future Works
29
I Ethics Statement
29
Reproducibility Statement
29
K The statements of using Large Language Models
30
16

---

Published as a conference paper at ICLR 2026
A SUPPLEMENT EXPERIMENTAL RESULTS
To further validate the robustness, fairness, and generalizability of MedVR. we conducted a series of extended experiments. These include rigorously controlled comparisons against baselines, evaluations on diverse task formats and domains, and quantitative assessments of the model's core capabilities.
A.1 BASELINE TRAINING DATA EXPOSURE AND FAIRNESS OF EVALUATION
A critical challenge in evaluating state-of-the-ee art vision-language models (VLMs) lies in ensuring fair and meaningful comparisons, particularly when benchmarking performance across models developed with heterogeneous training protocols, data sources, and architectural scales. Published baselines often differ significantly in their pretraining corpora, fine-tuning objectives, and exposure to evaluation benchmarks, which can introduce confounding variables that obscure the true contribution of methodological innovations.
To ensure transparency and provide a rigorous contextualization of MedVR's s performance, especially in out-of-domain (OOD) settings, we present a comprehensive summary of the training data and learning paradigms used by leading medical VLMs in Table 4. This analysis reveals crucial discrepancy: several strong baseline models were trained on datasets that substantially overlap with the benchmarks used for our OOD evaluation. For instance, Lingshu-7B and MedGemma-4B were both trained on large-scale proprictary corpus that explicitly includes subsets of the OOD test sets, giving them an inherent advantage in generalization evaluations. In stark contrast, MedVR is trained exclusively filtered subset of OmniMedVQA with no access to any data from other benchmarks during training. This strict data isolation ensures a genuine distt---ittiibution evaluation setup, making our model's competitive performance on these benchmarks strong testament to its intrinsic generalization capability.
The data presented in Table 4 thus serves to clarify prior ambiguities in our initial manuscript, where the superiority of MedVR over models like Lingshu-7B on OOD tasks might have been misinterpreted. We emphasize that MedVR does not outperform Lingshu-7B due to data advantage. but rather demonstrates comparable or superior performance despite significantly less and less overlapping training data, underscoring the effectiveness of our annotation free visual reasoning framework in inducing generalizable, data-efficient learning.
Table 4: Summary of training data, methodologies, and benchmark overlap for key baselines.
Model
Training Data
Training Methods
Key Benchmarks Involved in Training
Med-R1-2B
OmniMedVQA
RL RL Pretraining. SFT, RL.
OmniMedVQA
OmniMcdVQA, PMC-VQA, VQA-RAD, SLAKE, PathVQA
McdVLM-R1-2B
HuatuoGPT-Visian Eval
MedGemma-4B
Proprietary curated data (33M)
PMC-VQA, VQA-RAD, SLAKE and others
LLaVA-Med-7B
PMC-ISM
PMC-VQA
HuatuoGPT-v-7B
PubMedVision (1.3M)
Pretraining.SFT Pretraining, SFT. RL
PMC-VQA
PMC-VQA, MedXperIQA, VQA-RAD, SLAKE, PathVQA and others
Lingshu-7B
Proprietary curated data (12M)
MedVR (Ours)
OmniMedVQA (36K filtered)
RL
OmniMedVQA
A.2 QUANTITATIVE EVALUATION OF LOCALIZATION QUALITY
cornerstone of our work is the claim that MedVR learns to perform precise localization and robust reasoning without recourse to intermediate supervision. The trustworthiness of the entire framework hinges on the fidelity of these learned visual grounding steps. To provide direct, quantitative evidence for this claim, we conducted a rigorous evaluation of MedVR's localization capabilities on three diverse tasks that demand precise visual grounding:
1. General Medical VQA on the GEMEX -hink VG (Liu et al., 2025a) dataset, which requires locating anatomical structures or pathologies to answer questions.
17

---

Published as a conference paper at ICLR 2026
2. Medical Phrase Grounding the ChestX-ray8 (Wang et al., 2017) dataset, which tests the ability to map a textual phrase to specific image region.
3. Lesion Detection on the ISIC (Codella et al., 2019) dataset, a classic task requiring precise outlining of skin lesions.
For each task, we measured the mean Intersection over Union (mloU) between the bound- ing boxes generated by the model's Zoom-in tool and the ground-truth annotations. The results, summarized in Table 5, reveal a dra- matic improvement localization accuracy for MedVR compared to the zero-shot Qwen2.5VL- 7B backbone. The consistent, pegh-pperfrrmacce across these varied tasks demonstrates that the learned skill is not a narrow, task-specific arti- fact but generalizable capability. The low dard deviation in our results further indicates stable and reliable performance. This provides strong empirical validation that our annotation- free learning process successfully distills high- quality supervisory signal, effectively teaching the model to identify and focus on causally relevant clinical regions.
Table 5: Quantitative evaluation of localization quality. MedVR significantly improves the mean Intersection over Union (mloU) compared to the zero-shot backbone, demonstrating its ability to learn precise visual grounding without annotations.
Model
GEMEX-ThiakVG
ChestX-ray8
Qwen2. V-V1.7B
17.34 2.13
36.33 3.21
35.73 1.87
MedVR (Ours)
$9.62 a 1.73
54.29 1.11
69.12 1.35
A.3 EFFICACY OF CONSENSUS- CREDIT ASSIGNMENT (CCA)
critical question for any self-supervisory mechanism is whether it distills true, meaningful signal or merely amplifies noise from the model's own biases. To validate that our annotation-free CCA mechanism generates high-quality pseudo-superrision, we compared its performance against super- vised upper bound. We conducted an experiment on the GEMEX Think VG dataset, which provides ground-trute truth masks for relevant regions. In this oracle setting, the CCA consensus mask was replaced with the ground-truth mask, providing direct, perfect supervision for the Zoom-in tool reward. The results are presented in Table 6. Strikingly, the performance of our fully annotation-free ap- proach is highly comparable to that of its super- vised counterpart, with only marginal differ- ence in both final task accuracy and localization mloU. This result provides powerful evidence that CCA is highly effective at distilling high- quality reward signal from the collective agree- ment of successful rollouts. It demonstrates that the "wisdom of the crowd" principle holds, suc- cessfully guiding the agent toward semantically relevant regions without requiring any costly human annotations. This finding is central to the practical value of MedVR, as it confirms the viability of achieving robust visual grounding in a scalable, annotation-free manner.
Table 6: Comparison of MedVR with its super- vised counterpart on TEMMEX-ThnnVG..
Method
Accuracy (%)
mloU (%)
w/ Annotation
79.62
61.33
w/o Annotation
79.08
59.62
A.4 DOMAIN-AGNOSTN GENERALIZATION TO NON-MEDICAL TASKS
To investigate whether our proposed mechanisms, EVR and CCA, constitute fundamental improve- ments in visual reasoning or are merely specialized heuristics for the medical domain, we assessed their efficacy on general-oe -oominn benchmarks. This evaluation serves to test the hypothesis that our contributions are Nomain-agnosse and broadly applicable. We evaluated our method on two categories of general-domain tasks: (1) Visual Grounding. For this task, our model was trained on the training splits of the RefCOCO/+/g (Chen et al., 2025) datasets and evaluated on the correspond- ing validation splits; and (2) Multimodal Reasoning. For this task, the model was first trained on Geometry3k (Lu et al., 2021) and then evaluated on three challenging benchmarks: MathVist (Lu al., 2023), Math Verse (Zhang al., 2024), and Math Vision (Awais et al., 2024).
As detailed in Table 7a and Table 7b, our method consistently outperforms both the powerful Qwen2.5- VL-7B backbone and DeepEyes (Zheng et al., 2025), a strong contemporary model specialized in visual reasoning. The superior performance across these non-medical tasks provides conclusive evidence that EVR and CCA not narrow, domain-specific solutions. Instead, they represent
18

---

Published as a conference paper at ICLR 2026
Table Performance on visual grounding and multimodal reasoning benchmarks.
(a) Visual grounding benchmarks.
(b) Multimodal reasoning benchmarks.
Model
RECOCO
reCOCC+
ref0OCOg
Avg.
Model
Madh
MathVerse
MathVista
Avg.
Qwen2-5VL-7B SVL-7B
89.1
82.6
86.1
85.6
Owen2.5VL-78
24,9
43.8
66.3
44.3
DeepPyye
89.8
83.6
86.7
86.7
DeepEyes
26.6
47.3
70.1
47.9
EVR CCA
EVR CCA (Ours)
29.0
46.4
71,4
48,9
Question:
W hhtt content appears in this Image?
A.Skin tissue
B. Breast tissue
C. Bone tissue
D. Pancreatic tissue
Sthink The image shows a mammoggmm, which mdiogmaphic examination the breast tissue. accurate assessment, detailed visual inspection will be performed. <ithnn
<tool ("name". : "inage
tool", "arguments" ("boox 24" -318, 329, "label":
"Breast tissue") }} </tool callo
<think- The zoomerd-le
presence breast pypicul manongrrphii foutures. Resed manmogrum, image content elearly breast tiesne, <ithink>
Sanswert x/mmswrr
Question:
specific
depicted image lung?
A. Bronchitis
R. Parumonia
C. COVID-19
D. AcAte respiratory distress syndrome (ARDS)
Answer:
<think The image shows an X-ma the chest that uppears to have ground glass opacities, which are common findings in COVID-19, <thinks
calls ("name" "image
toal", "angumenss:: ("boxx 26" (351,
"label":
ground--llas opacities)) [] shool call>
<thinic
The zoomed-la image shoms the presence ground-laa opacities hilaterally particularly in the periphery the lungs. These findings consistent with the churcterissie imaging feutures of COVID-19, 199 <obbink>
<answer C Clansser>>
Figure 8: Additional Visualization Results of MedVR's Reasoning Process.
principled and generalizable approach to agentic visual reasoning. They effectively address the core challenges of exploration and credit assignment in tool-augmented RL setting, regardless of whether the visual input is chest X-ray or a complex geometric diagram. This confirms that MedVR's core contributions are of broad relevance to the wider field of multimodal AI.
A.5 MORE VISUALIZATION RESULTS
We provide an extended collection of qualitative results to further illustrate the robustness and versatil- ity of MedVR's visual reasoning capabilities. These examples, presented in Figure 8, supplement case studies in Section 7 of the main paper and chosen demonstrate the model's performance across wider range of medical modalities, question complexities, and clinical scenarios. The visualized cases have been selected to highlight several key strengths of our approach:
The examples span multiple imaging modalities, including chest X-rays, CT scans, and MRIs. This demonstrates that MedVR's learned reasoning strategy is not confined to single type of medical image but generalizes effectively, correctly identifying regions of interest regardless of whether it's skeletal structure in a radiograph or soft tissue in magnetic resonance image.
Cross-Modality Proficiency:
Interleaved Textual and Visual Reasoning: The provided cases compellingly demonstrate the model's capacity for a tight coupling of textual deliberation and visual action. This process not a linear, one-shot analysis but rather dynamic, interleaved reasoning chain. Initially, the model forms textual hypothesis based on the global image and the query. This internal textual monologue then guides the decision to execute a visual action by invoking the Zoom-in tool on region interest. Subsequently, the new, high-resolution visual information is fed back into the reasoning process, allowing the model to update its textual understanding. refine its hypothesis, even trigger further visual exploration.
Precise Localization and Grounding: Across all examples, consistent theme is the precision of MedVR's s localization. The bounding boxes generated by the Zoom- in tool accurately frame the causally relevant anatomical or pathological features. The final
19

---

Published as a conference paper at ICLR 2026
generated answer explicitly references the findings from this localized view, confirming that the model's 's conclusion grounded in direct visual evidence rather than spurious correlations textual priors. This is exemplified in the last case, where the model correctly identifies and describes a small, easily missed fracture.
Collectively, these additional visualizations reinforce the central thesis of our work: by equipping VLMs with mechanism for active visual exploration and robust self-supervisory signal for credit assignment, we can foster more generalizable and trustworthy form of medical reasoning. The dynamic, evidence-seeking behavior demonstrated here stands in stark contrast to the static analysis of conventional models and represents a critical step towards building more reliable medical Al for clinical applications.
A.6 ANALYSIS OF LEARNED TOOL USAGE DYNAMICS
To understand how the agent's 's behavior evolves during training, analyzed the dynamics of tool usage over time. We tracked the average number of Zoom-in in calls per trajectory throughout the training process, as depicted in Figure 9. The resulting curve reveals classic learning pattern.. Initially, the number of tool calls increases as the agent explores the action space and discovers the utility of the tool for improving task success. Fol- lowing this exploration phase, the number of calls gradually decreases and stabilizes at an efficient rate (approximately 1.05 calls per trajectory).
Avg. Tool Tool calls
This trend demonstrates that the agent does not learn to call the tool randomly or excessively. Instead, through reinforcement learning, it masters sophis- ticated and judicious policy for tool use. learns to apply the Zoom-in function strategically, invoking it primarily when the expected information gain out- weighs the operational cost. This convergence an efficient and effective policy underscores the agent's ability to learn not just how to use tool, but more importantly, when to use it to maximize its reasoning performance.
Training Steps
Figure Average number of tool calls per trajectory during training.
B THE ALGORITHM WORKFLOW OF MEDVR
In this section, we provide a detailed flowchart of the MedVR algorithm in Algorithm 1.
C SUPPLEMENT RELATED WORK
This section provides an expanded discussion of related work to further contextualize the contributions of MedVR. We differentiate our framework from prior research in two key areas: annotation- reinforcement learning and visual-grounded reasoning.
C.1 A ANNOTATION-FRER REINFORCEMENT LEARNING
Recent advancements in annotation freee reinforcement learning have demonstrated the potential of training language models using intrinsic rewards, obviating the need for human-annotated labels. Seminal works in this area include EMPO (Zhang al., 2025a), which optimizes for semantic consistency by minimizing prediction entropy in a latent space; INTUITOR (Zhao al., 2025), which leverages the model's 's self-certainty which employs a majority voting mechanism to generate pseudo-labels for reward calculation.
intrinsic reward signal: and MM-UPT (Wei al., 2025),
While MedVR draws inspiration from this paradigm of self-supervision, addresses fundamentally different set of challenges rooted in the unique demands of multimodal medical reasoning. Our contributions are distinguished from these prior works in several critical aspects:
20

---

Published as a conference paper at ICLR 2026
Algorithm The MedVR Training Framework
Input: Initial policy model R task dataset hyperparameters for GRPO; EVR hyperpa- rameters Paase, Y; CCA hyperparameter total rollouts per prompt M.
2: Initialize: Policy model wo <- T0
3: for iteration i 1,.... do
4: Freeze a reference policy Tref f ng.
5: for step = , do Sample a batch prompts Dj = ((Q}; ((=
6:
from D.
7:
b Trajectory Generation with Entropy-guided guided Visual Regrounding (EVR)
8:
Initialize an empty set of trajectories for each prompt: T₁+ 5 for = 1.....
9:
Phase 1: Base Trajectory Generation & Adaptive Branching Generate M/2 initial trajectories for each prompt by sampling from Tref-
10:
for all each generated trajectory do Compute baseline entropy Hbase at the start of the trajectory. if'a <tool_call> for Zoom- being generated then
12:
14:
Compute current tool-call entropy Htool-
Calculate entropy increase Aoot Hioo! Haase-
16:
Define branch probability Pbranch Prose + A
if random number (0, 1) < Prranch and exploration budget allows then Branch: Fork the current generative state and generate parallel trajec- tory from this point of high uncertainty, adding it to the corresponding Ty.
18:
19:
end if end end for
20:
21:
22:
Phase 2: Complete Remaining Trajectories
23:
For each prompt, generate additional trajectories independently from Teef until [T,] M. Reward Calculation with massensus-based Credit Assignment (CCA)
24:
for all prompt E , do
Identify the set of successful trajectories T,* (T T, Race > 0}.
27:
y then
28:
Generate visual footprint masks {M&} } for each Th E &T²
29:
Aggregate masks create heatmap C Σ3 Mr. Binarize heatmap to obtain consensus mask M, = (C > I7, 1/2).
30:
31:
for all trajectory тą TTy do
32:
Compute ToU (M,, M2).
33:
Assign consensus-aligned tool reward: Reoot (Tk)-
34:
end for end if
35:
36:
Compute total terminal reward R(T) for all r E Tj using Equation (2). end for
38:
b- Optimization Use the collected trajectories (T))²= and their associated rewards (R(T)) to compute
39:
advantages.
40:
Update the policy model Ty by maximizing the Group Relative Policy Optimization
(GRPO) objective.
41:
end for
42: end for
43: Output: The optimized policy model ny.
21

---

Published as a conference paper at ICLR 2026
Problem Formulation and Definition of "Annotation-Free The primary distinction lies in the definition of what is "annotation-free Prior works typically address tasks where the final answer is unavailable, optimizing for the internal consistency of the reasoning process itself. In contrast, our medical VQA setting provides ground-truth final answers. Our "annotation- challenge pertains to the intermediate reasoning steps. In clinical workflow, a verifiable diagnostic process is as crucial as the final conclusion. However, annotating these fine-grained visual grounding steps is prohibitively expensive and requires specialized expertise. Me is designed specifically to learn this verifiable process without any intermediate supervision, a problem distinct from that addressed by prior text-only methods.
The aforementioned methods focus almost exclusively on text-only reasoning and are primarily validated on tasks such as mathematics or com- monsense reasoning. Our work tackles multimodal medical VQA, a domain where general- purpose Vision-Languaae Models (VIMs) face significant domain and modality gaps. Applying text-based intrinsic reward mechanisms directly is non-trivial and often insuffi- cient for tasks that demand nuanced interpretation of fine-grained visual evidence, such as identifying subtle pathologies in medical images.
Research Domain and Modality:
Agentic Framework vs. Single-Pass Generation: Previous methods typically operate on single-pas paggle-passaa textual generation paradigm. MedVR, however, is built upon an agentic rein- forcement learning framework that supports multi-turn, interleaved visual-textual reasoning. This allows the model to actively probe the visual input, refine its hypotheses, and ground its conclusions in dynamic, iterative manner. To our knowledge, MedVR is the first work to develop an annotation free optimization strategy specifically tailored for an agentic RL context in complex multimodal setting.
Semantic and Interpretable Technical Approach: Although we leverage concepts like entropy and voting, our implementations are fundamentally different and tailored for mul- timodal interpretability. Instead of optimizing at the token level, which lacks semantic completeness, our Entropy-guided Visual Regrounding (EVR) operates on the semantically complete tool_cal level. Furthermore, our Consensus-basse based Credit Assignment (CCA) moves beyond simple textual voting; it distills a visual consensus mask from the grounding boxes generated across diverse rollouts, providing highly interpretable supervisory signal directly within the visual domain.
In summary, MedVR represents a principled adaptation and extension of annotation- learning, specifically engineered to solve the confluence of challenges inherent in building verifiable and trustworthy medical AI.
C.2 VISUAL-GROUNDED REASONING
The field of groundd reasoning has seen significant progress, with models like ViGoRI (Sarch al., 2025), UniVG-RI (Bai al., 2025b). COPL (Jiang Lu, 2024), and VGR (Wang al., 2025) demonstrating impressive capabilities in linking textual outputs to specific image regions. These methods have pushed the boundaries of how models can "see" and reason about the visual world.
Despite these advances, MedVR is distinguished by its unique objectives, data assumptions, and learning paradigm, which are specifically tailored to the constraints and requirements of the medical domain.
Research Objective: A Means, Not End: A key difference lies in the ultimate goal. Methods like UniVG-RI I and COPL primarily focus on visual grounding or localization the end-goal itself. In contrast, MedVR leverages the model's grounding ability as an intermediate step within a broader diagnostic reasoning chain. For MedVR, precise localization is not the final output but critical means provide verifiable and interpretable visual reasoning process that can aid clinical decision-making.
Data Dependency and the Annotation-Fres Imperative: The most critical distinction is the reliance supervision. Prior works like ViGoRL and VGR critically depend on large-scale, human annotated supervised tne-ttunig (SFT) datasets that provide fine- grained grounding information bootstrap the model's 's capabilities. This prerequisite is infeasible in
22

---

Published as a conference paper at ICLR 2026
the medical domain, where such annotations are prohibitively expensive, require specialized clinical expertise, and are largely unavailable at scale. MedVR directly confronts this fundamental bottleneck. To our knowledge, it is the first framework to explore and achieve robust visual-grounded reasoning for medical VQA in a completely annotation-free manner, making scalable and practical solution for real-world medical Al development.
Methodological Focus on the RL Process:
The focus of many prior works, such as ViGoRI. and UniVG-R1, is heavily weighted toward dataset construction and supervised fine-tuning. The reinforcement learning component, if present, is often not the central methodological innovation. MedVR, conversely, is the first work in this area to perform fine-grained modeling of the end-to-end agentic RL process for visual reasoning. Our novel mechanisms for uncertainty- guuded exploration (EVR) and consensus based reward shaping (CCA) represent deep and principled exploration of the RL rollout and credit assignment problem in this context.
Internal Refinement vs. External Supervision: The underlying motivation of prior works is to enhance a model's grounding ability by learning from external, high-quality annotated data. This paradigm aims to "inject" grounding skill through SFT. In contrast, MedVR is designed to leverage and refine the model's internal, pre-existing grounding capabilities through a self-supervisory loop within the RL process itself. teaches the model to improve its own innate skills, guided only by the final task reward and the consensus of its own successful explorations.
Learning Paradigm:
By obviating the need for intermediate annotations, MedVR offers new paradigm for training visually-gronnded grounded medical AI that is both highly effective and scalable in the face of real-world data scarcity.
D IMPLEMENTATION DETAILS
D.1 TRAINING INITIALIZATION
We clarify that our "annotation-fre" RL training does not use any supervised fine-tuning (SFT) or instruction-tuning on aumaa--annotted reasoning traces to teach the model its output format. The agent learns produce structured outputs with thoughts and tool calls directly through reinforcement learning. This is achieved by providing detailed system prompt (see Figure 4 in the main paper) to the Qwen2.5-VI-
base model. The strong instruction-followe model allow to adhere to the predefined JSON format for tool calls without requiring a separate SFT warm-up stage. The reasoning behaviors observed are therefore direct result of the RL agent learning to operate within the constraints defined by the system prompt, guided by the task and tool rewards.
capabilities of the backbone
D.2 TOOL IMPLEMENTATION
The Zoom- in tool is the primary mechanism through which the MedVR agent actively interrogates the visual input. Its implementation is designed be both effective and computationally efficient.
Zoom-in Tool Workflow. The workflow proceeds as follows:
1. Invocation: When the agent's policy generates complete Zoom-in command containing valid bounding box coordinates, an external tool is triggered.
2. Execution: This tool operates on the original, full-resolution input image to ensure maximum fidelity. A high-resolution patch corresponding the specified coordinates is cropped.
3. Visual Encoding: The cropped patch subsequently processed by the same pre-trained vision encoder used for the full image. This generates a new set of visual tokens representing fine- uie-ggrinned information from the selected region of interest.
4. Context Integration: These new tokens are encapsulated within special markers (ctool_resportc response> and </tool response>) and integrated back into the agent's context sequence.
23

---

Published as a conference paper at ICLR 2026
5. Conditioned Generation: The model's subsequent generation is thereby conditioned on both its prior reasoning chain and this new, targeted visual evidence. Crucially, as these observation tokens are part of the environment's 's response and not generated by the policy, they are masked out during the policy loss calculation.
Image Resolution. A distinction is made between training and inference resolutions to balance computational feasibility with performance. During training, to manage GPU memory constraints, input images with longest side exceeding 1024 pixels are resized to a maximum of 1024 pixels while maintaining their aspect ratio; smaller images retain their original dimensions. During inference, however, the model operates the original, full-resolution images to ensure that the Zoom-in in tool can access the highest level of detail possible for precise analysis.
Inference Latency. To assess the practical impact of tool on performance, profiled the average inference latency on the OmniMedVQA dataset using H20 GPUs. As detailed in Table 8, the computational overhead introduced by the Zoom- in tool is negligible, constituting only 1.5% of the total inference time. This low overhead is attributable to the fact that the primary bottleneck in inference is typically the sequential, autoregressive generation of text. In contrast, the vision encoder forward pass is highly parallelizable and comparatively fast. Furthermore, the latency from tool execution can be further minimized through standard optimization techniques such as Key- Value (KV) Caching and parallelized tool execution, making the MedVR framework highly suitable for practical applications.
Table 8: Inference latency analysis for MedVR on the OmniMedVQA dataset.
Model
Total Tokens
Total Latency
Extra Visual Tokens
Extra Latency
MedVR
169.23
15.21s
84.76
0.23s (-1.5%
D.3 EVALUATION PROTOCOL AND METRICS
For the general-domain multiple-choice tasks, we employ OmniMedVQA for training and in-domain testing, strictly adhering to the official data splits from Med-R1 (Lai et al., 2025). To assess out- of-domain (OOD) generalization, and MedXpertQA. For these benchmarks, we use accuracy as both the evaluation metric and the reinforcement learning reward signal. For the modality free-text tasks, the model is trained on a consolidated dataset of the VQA-RAD and SLAKE training splits, which constitute our in-domain benchmarks. We then evaluate its OOD generalization on VQA. Evaluating open-ended text generation necessitates more nuanced approach, particularly for designing the reward function. Following prior work (Rui et al., 2025), we formulate composite reward signal, Ropens that balances lexical overlap with semantic similarity. It integrates n-gram- based metrics (BLEU-1 and ROUGE-1) with semantic-basee metric (BERTScore), defined as:
conduct zero-shot evaluations the test sets of PMC-VQA
Ropen (T) A (BLEU, (T) + ROUGE, (T)) + (1 (1 A) BERTScore(T)
where À is balancing hyperparameter and we set À 0.8. For the final evaluation of open-ended tasks, we employ the Dwwen--235--AV (Yang et al., 2025) to assess the correctness of the generated answers.
E INFERENCE SETTINGS
To ensure the rigor, fairness, and reproducibility of our experimental results, we established a comprchensive and standardized inference protocol. This protocol governs all aspects of the evaluation process, from the initial prompt engineering to the technical parameters of the decoding process.
E. PROMPTS
The agent's behavior is guided by carefully designed set of prompts that define its operational context and task objectives. This process involves two key components: a System Prompt and a User Prompt.
24

---

Published as a conference paper at ICLR 2026
System Prompt
You Medical Assistant.
You may call functions
user query. You are provided signatures <tolls--siools>> tags:
"yype" "function..
"Iunction::
"name"" "mnage, z00m 'loof.
"desoription: ZZom on speciiic region of medical image by crasping it based on bounding box (boox) and object label. Use this to inspect potential abnormaltees, les lons specific anatomicaal siructures mentioned in the query."
"parameters":
"byw" "odject".
properties":
Type "array"
"ype:: "uumber"
"mirltems"
maxitems":
"deccrittinn: The bourding box of the reg for dessiled examiration, es [x1. y1, x2, yol, where (x1, y1) top-keft comner and (x2, y2) is the buttom-right corner."
Tabor:
Type": "wiring"
describtonn: brief medical label for the object bounding box.'
<hoolg>>
Instructions
your resporss,, you noed to first think about ressoning procass the mind and then conduct function cailing to get additional information neaded. The reasoning process and function calling are endosed withnn <himb-ithhike e and sizol Loal t-oooooo call- tags respectively. The results of the function calls will be givan back to you after execution, and you can continue call functions until you get the final answer for the users question. Finally, if you have the answer, anckosa within sirdswrrr-/nsseer bigs and not continus call functions.
For each function call>
json object
within stool, callo-ctool,
tags:
"eguumants: sarg--jon-chjeer>>
Figure 10: System Prompt used in MedVR.
The System Prompt, detailed in Figure 10, is meticulously crafted to elicit precise, clinically- grounded visual reasoning within a strictly defined output format. It serves as the agent's core instruc- tion set, defining the "rules of engagement" by specifying the available tools (i.e., the Zoom- in tool), their function signatures, and the required JSON format for invocation. Furthermore, it enforces structured reasoning process that mirrors clinical diagnostic logic, compelling the model to interleave textual deliberation (<th i in>> with tool-based actions before arriving at conclusion. This structure essential for generating transparent and verifiable reasoning traces.
In stark contrast, the User Prompt, shown in Figure 11, is intentionally minimalist. It presents question and without imposing any constraints or hints on the intermediate reasoning path. This simplicity ensures that the model's reasoning process is driven by its own learned policy and understanding of the task, rather than by explicit guidance the prompt, thereby enabling a more robust and unbiased evaluation of its autonomous capabilities.
E.2 INFERENCE PROTOCOL AND PARAMETERS
All inference experiments were conducted on a cluster of NVIDIA H20 GPUs, leveraging the vLLM inference engine for its tigh-throughput performance and efficient memory management via PagedAttenntion. The generation of each reasoning trajectory is an autoregressive process. We employed a nucleus sampling (top-p) decoding strategy to guide this process. The key parameters were set as follows: Temperature: T 0.0, Top-p: = 1.0. To ensure computational tractability and prevent degenerate, non-terminating reasoning loops, we imposed the following constraints:
Maximum Tool Calls:
maximum of tool calls were permitted per trajectory. This limit was determined to sufficient for the complexity of the evaluated tasks without encouraging inefficient repetitive actions.
25

---

Published as a conference paper at ICLR 2026
User Prompt
Question:
Answer with option's letter from the given choices.
For free-text VQA:
Open-ended:
Clase endde:dddd
Judgement:
Question:
0
Please answer the quesiton concisely.
Answer the question using single word phrase. Please output 'yes' or "no"(no extra output).
phoses grounding:
Please provide the bounding coordinates for the region described by the following senfence:
Output the result the format [x1. y1. x2, y?]
For lesion detection::
Please provide the bounding box coordinates of the lesion in the image, formatted as [x1, y2]
Figure 11: User Prompt for inference.
Maximum Token Length: A global maximum token limit of 4096 was set for each turn.
This serves safeguard against excessively long or runaway outputs.
Termination Signal:
The primary termination condition for trajectory is that tool call tokens are generated, which signals that the agent has concluded its reasoning process and is providing its final output.
F DATASETS
OmniMedVQA (Hu et al., 2024) We utilize the publicly available subset of the OmniMedVQA benchmark as our in-domain dataset for medical visual question answering (VQA). This dataset comprises total of 82,059 medical images and 88,996 image-question-answwe triplets, spanning eight major imaging modalities: CT (15,808,, MRI (31,877), X-Ray (7,916), Ultrasound (10,991), Dermoscopy (6,679), Fundus (5,398), OCT (4,646), and Microscopy (5,680). The questions systematically categorized into five distinct types: Anatomy Identification (16,448 instances). Disease Diagnosis (55,387), Lesion Grading (2,098). Modality Recognition (1,5565), and Other Biological Attributes (3,498), enabling comprehensive evaluation across diverse clinical reasoning tasks. To ensure consistency with prior work, we adopt the train-test split configuration used Med-R1, facilitating direct comparison with existing models. In particular, to enhance training efficacy and concentrate on more challenging examples, we implement a data filtering strategy on the training set: we leverage Swen2.5-VL-77. (Bai et al., 2025a) to generate 8 responses for each question and estimate the sample's ' difficulty based on the resulting accuracy. Samples where the model exhibits high confidence, defined accuracy of 2 7/8, are subsequently excluded. This filtering process yields a curated, more challenging training dataset of 36,497 samples.
PMC-VQA (Zhang al., 2023) PMC-VQA is large-scale medical visual question answering dataset constructed through an automated pipeline that extracts and aligns multimodal content from open- biomedical literature in the PubMed Central (PMC) archive. contains 227,000 question-and question-answer answer pairs associated with 149,000 unique medical images, covering wide range of imaging modalities, anatomical regions, and pathological conditions. The dataset was designed to support generative MedVQA, where models are required to produce free-form textual answers rather than selecting from predefined options, thereby better simulating real-world clinician AI interaction scenarios.
MedXpertQA (Zuo et al., 2025) MedXpertQA is an expert-level multimodal benchmark designed assess advanced medical reasoning and domain-specific knowledge in artificial intelligence systems.
26

---

Published as a conference paper at ICLR 2026
Comprising 4,460 carefully curated questions across 17 medical specialties and 11 body systems, MedXpertQA goes beyond conventional VQA datasets by incorporating complex clinical contexts, including patient histories, laboratory results, and multi-image analyses. Unlike datasets derived from automated captioning or simplified QA generation, MedXpertQA leverages authentic board- certification examination questions and undergoes rigorous synthesis and filtering procedures to prevent data leakage and ensure clinical validity.
SLAKE (Liu et al., 2021) SLAKEiS a large-scale bilingual Medical VQA dataset in English and Chinese, built upon multimodal radiology sragess-iccludngg CT. MRI, and X-ray, with rich semantic annotations, diverse QA pairs, and an integrated structured medical knowledge graph. It spans five anatomical regions: head, neck, chest, abdomen, and pelvic cavity, and supports both visual and knowledge-drives reasoning. The dataset aims to foster the development of medical Al capable of interpreting clinical images and providing accurate, context-aware answers to domain-specific questions.
VathVQA (He et al., 2020) Vath is specialized visual question answering (VQA) dataset designed for pathology, comprising 4,998 pathology images and 32.799 question-answer pairs. The images were sourced from two pathology textbooks and the PEIR Digital Library, and the QA pairs were generated via a semi-automated pipeline that extracts captions and transforms them into questions using natural language processing techniques, followed by manual verification to ensure accuracy. Unlike most medical VQA datasets that focus primarily on radiology (e.g.. X-ray, CT, MRI), VAA centers on histopathological and cytological slides, along with some clinical photographs of pathological conditions, thereby addressing distinct and underrepresented domain in medical AL As the first publicly available VQA benchmark dedicated to pathology. Path VQA plays pivotal role in advancing AI systems toward expert-level diagnostic capabilities in digital pathology
VQA-RAD (Lau al., 2018) VQA-RAD is a radiology-focusee medical visual question answering (Med-VQA) VQA) dataset designed for training and evaluating Al systems that interpret clinical images and answer related questions. It comprises 2,244 unique question-answer pairs associated with 314 radiology images, sourced from the open-access MedPix database. The dataset includes both open-ended and binary questions, all of which were manually crafted by a team clinicians to reflect authentic diagnostic inquiry patterns. Originally released with 2,248 question-answer pairs across 315 images, the dataset was later refined by removing four problematic entries three duplicate triplets in the training set and one triplet shared between training and test sets to ensure clean evaluation conditions. VQA-RAD serves as a foundational benchmark in Medical VQA research due to its clinical authenticity and balanced design.
G BASELINES
G.1 GENERAL-PURPORS VLMS
Qwen2.5-VL (Bai et al., 2025a) Qwen2.5-Ve VL extends the powerful Qwen2.5 language model by integrating sophisticated multimodal capabilities. Its vision encoder is significantly enhanced through techniques such as dynamic resolution training, window attention, SwiGLU, and RM-SNorm. This architecture enables Qwen2.5-VL 5L to achieve robust visual reasoning performance across diverse range of visual inputs, including charts, text documents, and complex layouts, making it strong baseline for general multimodal tasks.
Intern VL2.5 (Chen et al., 2024c) Intern VL2.5 is prominent cemeral-purposse VLM known for its strong performance in various multimodal benchmarks. As part of the Intern series, typically features large-scale vision encoder meticulously aligned with powerful large language model. This architecture allows to process and understand complex visual information alongside textual prompts, demonstrating impressive capabilities in visual question answering, image captioning, and visual-grounded dialogue, thus representing a cutting-edge general VLM baseline.
Intern VL3 (Zhu et al., 2025) Intern VL3 is decoder- based multimodal model that combines Qwen2.5-derivee language backbone with a newly pre-trained vision encoder, following the ViT-MLP- LLM architecture. It is trained natively text, images, and videos, enabling strong long-context
27

---

Published as a conference paper at ICLR 2026
understanding and effective tool-use reasoning across diverse domains such 3D vision and GUI agents.
G.2 MEDICAL-SPECIRE VLMS
Med-R1-2B (Lai al., 2025) Med-R1-2B is specialized reinforcement learning-enhaneee vision- language model tailored for diverse medical tasks. A key innovation of Med-R1 1 is its employment of Group Relative Policy Optimization (GRPO) to significantly improve generalization capabilities in critical medical applications such disease diagnosis and lesion grading. Notably, Med-R1-2B has demonstrated superior performance, surpassing larger general- purposs models like Qwen2-VL-72B by significant margin the OmniMedVOA dataset.
MedVLM-R1-2B (Pan et al., 2025) SodVLM-R1-22 stands as a highly specialized medical Vision-Language Model, meticulously engineered achieve advanced visual and linguistic com- prehension within the intricate domain of medical imaging and clinical documentation. It is also optimized through Group Relative Policy Optimization (GRPO). This sophisticated reinforcement learning approach enables MedVLM-RR 1-MM-11-0 -B to learn robust reasoning strategies, specifically enhanc- ing its ability to interpret complex medical imagery and generate medically pertinent and actionable responses.
MedGemma-4B (Sellergren et al., 2025) MedGemma-4B a medical Vision-Language Model built upon the Gemma 3 language model. It integrates specialized MedSigLIP encoder, which is pre-trained on a vast collection of de-identified medical images. The language components of MedGemma are further refined through training on extensive medical text corpora. Optimized for instruction tuning, MedGemma-45 is designed to support range of medical applications, including summarization of clinical reports and the generation of diagnosis explanations, thereby aiming to assist healthcare professional..
LLava-Med-7B (Li et al., 2023b) LLaVA-Med-7B is a medical adaptation of the popular LLaVA framework, instantiated with 7B parameter language model backbone. LLaVA models typically connect a pre-trained vision encoder to a large language model via a lightweight projection layer, and trained multimodal instruction-follovinn data. LLL VA- specializes in medical visual question answering and dialogue, having been fine-tuned on medical image-text datasets to imbue it with domain-specifet knowledge and reasoning capabilities crucial for clinical tasks.
HuatuoGPT-v-7B (Chen et al., 2024a) HuatuoGPT-v-I is multimodal extension of the Hu- atuoGPT series, originally designed as a Chinese medical instruction-tumed Large Language Model (LLM) based on LLaMA. While the provided reference specifically describes the textual LLM, HuatuoGPT-v-77B would integrate a vision component to extend its diagnostic consultation capa- bilities multimodal inputs. The original HuatuoGPT was fine-tuned using synthetic instructions generated by ChatGPT and authentic doctor-patient dialogues, enabling strong performance in symp- tom interpretation and treatment recommendation. HuatuoGPT-V-7B aims bring this domain expertise to medical image understanding.
Lingshu-7h (Xu et al., 2025) Lingshu-70 7B is a comprehensive medical Multimodal Large Language Model (MLLM) built upon the Qwen2.5-VL architecture. It features a robust integration of a vision encoder, a powerful LLM, and a projection module to facilitate multimodal understanding. Lingshu- 7B undergoes multi-stage training process, critically incorporating Reinforcement Learning (RL) with verifiable rewards. It leverages an extensive dataset comprising over 5 million multimodal and textual medical samples, enabling unified understanding across various medical imaging types. This sophisticated training paradigm aims to deliver high-performance medical reasoning.
G.3 FINE-TUNED VLMS
Textual RL To provide a clear benchmark for the efficacy of our interleaved iisual-textura reasoning, we implemented Textual RL baseline. This model serves direct counterpart to MedVR, designed to answer the question: "How much performance is gained by adding explicit visual interaction?" To this end, we ensured maximum controlled consistency between the two models. The Textual
28

---

Published as a conference paper at ICLR 2026
System Prompt for Textual RL
Assistant. asks a question, process in the mind and then provides the user with the enswer: The reasoning process and answer are enclosed within <think<<think> and "ansver-</answer respectively, <think reasoning
conversaiio between
assistant reasoning
here <thini <answer> here <lanswer>
Figure 12: System Prompt used in Textual RL.
Prompt
Question:
task:
Think through the question step by step. enclose your reasoning process <think>.. <hhhnkk> lags.
Then provide the correct single-letter choice (A. B, D....) Inside sanswer...slaseeee tags. these lags.
Figure 13: User Prompt used in Textual RL.
RL baseline was trained using the same optimization algorithm (GRPO), training dataset, and key hyperparameters (e.g., batch size) as MedVR. Advanced optimization strategies, such as the "clip-higher" mechanism, were also identically applied to both setups.
The fundamental divergence is in the model's action space and its corresponding guidance. We crafted prompts inspired by the text-only reasoning paradigm of Med-R1. as shown in Figure 12 and Figure 13. These prompts instruct the model formulate its reasoning as a purely textual chain-of-thought,, processing the image static input without any interactive capabilities. While the Textual RL model demonstrates strong performance in its own right, its reliance on textual deliberation alone proves be significant limitation, particularly on tasks requiring fine-grained visual analysis. Its performance, therefore, serves as a robust reference point that underscores the substantial improvements afforded by MedVR's ability to ground its reasoning in dynamic visual evidence.
H LIMITATIONS AND FUTURE WORKS
While MedVR demonstrates significant advances, we acknowledge several limitations that highlight promising directions for future research. First, the current framework relies solely on Zoom-in in tool. Future work should expand the action space to include more sophisticated clinical tools, such as measurement, contrast adjustment (windowing), and multi-image comparison, to address broader range of complex diagnostic tasks like tumor growth tracking. Second, our evaluation primarily focuses on final-answer accuracy, indirectly assessing the quality of visual grounding. A crucial next step is to conduct formal evaluations with clinical experts to score the plausibility and clinical relevance of the model's intermediate reasoning steps, providing more direct validation of our approach. Finally, the reinforcement learning process can be computationally intensive. Future research could investigate more sample-efficient methods, such as offline RL or curriculum learning, to improve training stability and scalability, facilitating the application of MedVR to even larger models and more diverse datasets.
I ETHICS STATEMENT
This research strictly adheres to ethical guidelines for the use of medical data. Our experiments were conducted exclusively publicly available medical Visual Question Answering (VQA) benchmarks. All data utilized in these datasets are fully anonymized to safeguard patient confidentiality. and their in research is compliant with their respective data usage agreements. No new patient data was collected for this study.
J REPRODUCIBILITY STATEMENT
We are committed to ensuring the reprodueibility our research. the main body of the paper and this appendix, have provided comprehensive details regarding our model architecture, training
29

---

Published as a conference paper at ICLR 2026
protocols, and hyperparameter settings. The implementation of core mechanisms, cntropy-guides Visual Regrounding (EVR) and Consensus- based Credit Assignment (CCA), is described in detail to facilitate replication. We plan to release the source code for MedVR and the necessary configuration files to the research community upon completion of our institution's ' internal review process, thereby allowing for full verification of our findings.
K
THE STATEMENTS OF USING LARGE LANGUAGE MODELS
During the preparation of this paper, utilized large language models (LLMs) writing aid. Their use was strictly confined to improving the clarity, grammar, and style of the manuscript. The core scientific contributions, including the conceptualization of the MedVR framework, the design of experiments, the analysis of results, and the final conclusions, are entirely the intellectual product of the authors. The LLMs did generate any scientific content influence the interpretation of our findings.
30


Reviews:
[{"review_id": "Ic5IaCyKrJ", "reviewer": "Reviewer_kz6o", "summary": "The paper presents an RL framework that interleaves text-based reasoning with image-manipulation tools to achieve annotation-free visual reasoning in medical VLMs. Two mechanisms drive the approach: (i) Entropy-guided Visual Regrounding branches rollouts at high token-entropy steps to explore multiple regions of interest; (ii) Consensus-based Credit Assignment aggregates successful rollouts into a consensus mask and rewards trajectories whose visual footprints overlap with that mask. Experiments on three datasets, MedVR reports competitive performance.", "questions": "Please refer to Weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written. \n2. The motivation is sound. \n3. The presented approach shows competitive performance in both in-domain and out-of-domain settings.", "weaknesses": "1. The paper’s comparison to prior work needs to be improved. The idea of Annotation-Free RL and Visual-Grounded Reasoning with RL has already been studied in previous works [1–7]\n\n2. The paper lacks experiments to support the claim that the proposed method can provide robust and transparent reasoning. For example, the paper claims precise localization, but there are no quantitative grounding metrics or a human reader study to validate this. A localization-quality metric should be used. Additionally, evaluations of hallucination and robustness should be provided to support the arguments.\n\n3. The fairness of the experiments should be clarified. It is unclear whether previous medical LLM baselines and medical RL baselines were tuned with similar rollout budgets, RL variants/settings, and training corpora. The inference settings should also be clarified to ensure a fair comparison.\n\n[1] Right question is already half the answer: Fully unsupervised LLM reasoning incentivization. arXiv, 2025.\n\n[2] Learning to Reason without External Rewards. arXiv, 2025.\n\n[3] Unsupervised Post-Training for Multi-Modal LLM Reasoning via GRPO. arXiv, 2025.\n\n[4] Grounded Reinforcement Learning for Visual Reasoning. arXiv, 2025.\n\n[5] UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning. arXiv, 2025.\n\n[6] Visual Grounding for Object-Level Generalization in Reinforcement Learning. ECCV, 2024.\n\n[7] VGR: Visual Grounded Reasoning. arXiv, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CPrPksQLJb", "reviewer": "Reviewer_iJaK", "summary": "This work aims to propose a framework to better ground the reasoning of vision-language models into the visual perception. Concretely, 2 modules are proposed: (1) a entropy-guided visual grounding module using summed token entropy based model uncertainty (2) a consensus-based credit assignment module using distilled pseudo supervision from rollout agreement (e.g., union of multiple proposed bounding boxes). A qualitative study is demonstrated and quantitative experimental results show superior performance in medical VQA tasks without using human annotations.", "questions": "[line 107] what clinical workflow is referenced here?\n\n[line 162-164] Some implementation details are not clear enough to me: After a cropped image is obtained, is the image up-sampled to recover the full resolution or do they stay with their original resolution? How are the visual tokens of this cropped area obtained? Any additional padding? How are these visual tokens inserted into the reasoning path? What’s the latency of doing this in inference time?\n\nWill this agent crop images multiple times at different scales in different areas in a “successful trajectory”? If yes, would these areas contradict with each other (e.g., no overlapping, indicating the model is looking at completely different areas)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The writing is easy to follow.\n- The designed uncertainty based spatial exploration strategy based on the sum of entropy is interesting.\n- The consensus module is carefully designed as a posterior refinement step.", "weaknesses": "- The usage of “evidence” in this paper is not consistent with what “evidence” means in clinical practice. In clinical setting, “evidence” means the best available external clinical evidence from systematic research, instead of from the current tested image [1]. \n\n[1] Sackett D L. Evidence-based medicine[C]//Seminars in perinatology. WB Saunders, 1997, 21(1): 3-5.\n\n-\tThe contribution on “grounding every inferential step in verifiable visual evidence” is overclaimed. Is “zoom-in” enough to ground “every inferential step”? Is the “zoom-in” area always correct? What does “verifiable visual evidence” mean compared to “visual evidence”?\n-\tThe image manipulation tool of the approach is limited only to “zoom-in”. A wide range of other image manipulation tools such as different filtering/denoising techniques are not explored.\n-\tThere is no experiment justifying whether the zoom-in area is actually meaningful or not.\n-\tIt’s unclear whether this annotation-free approach might perform better than using the annotation or not under a fair comparison’s setting. \n\n-\t[line 104] The claim “First to endow medical VLMs with explicit visual reasoning capabilities.” is somewhat vague. This approach offers some improvements in the VQA performance, but existing powerful large MLLM such as GPT4o can also be deployed for medical tasks and has visual reasoning capabilities. What “explicit” means is unclear neither.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YNsSpM7oAz", "reviewer": "Reviewer_dA77", "summary": "This paper presents MedVR, a reinforcement learning framework that enables annotation-free visual reasoning in medical vision-language models. It introduces two mechanisms: Entropy-guided Visual Regrounding (EVR), which uses model uncertainty to trigger targeted visual exploration, and Consensus-based Credit Assignment (CCA), which derives pseudo-supervision from agreement among successful reasoning trajectories. Without human annotations, MedVR achieves state-of-the-art performance on multiple medical VQA benchmarks, improving both accuracy and visual grounding in clinical reasoning tasks.", "questions": "- Can you provide evidence that entropy spikes truly indicate regions where zooming in is useful? For example, visualize entropy over time or correlate entropy with answer improvements.\n\n- How frequently and at what stages do entropy spikes occur during reasoning? A simple distribution or timeline would clarify how often zoom-ins are triggered.\n\n- The approach seems well-suited for localization, detection, or report generation—why not evaluate on those tasks?\n\n- Would EVR + CCA generalize to non-medical visual reasoning tasks, or is it intended primarily for medical AI applications?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Proposes an annotation-free visual reasoning framework for medical VLMs\n- The idea of zooming into medical images to guide reasoning is inspired by real clinical practice, making the approach both creative and intuitively grounded\n- Using entropy as a proxy for uncertainty to trigger visual exploration adds interpretability to the reasoning process\n- Demonstrates consistent state-of-the-art performance across multiple medical VQA benchmarks, supported by detailed ablations validating each component\n- Addresses a key limitation in medical AI—lack of fine-grained visual supervision—in a practical and scalable way", "weaknesses": "- The use of entropy as a trigger for zoom-in is interesting but under-justified. The paper could provide stronger evidence that high-entropy points truly correspond to moments where zooming in improves reasoning. Visualizations or quantitative correlations between entropy spikes and successful zoom-ins would make this claim more convincing.\n\n- The method appears particularly well-suited for localization, lesion detection, or report generation, where visual grounding and fine-grained inspection matter most. However, these tasks are not evaluated, limiting the evidence that the approach generalizes beyond multiple-choice VQA.\n\n- It seems that every entropy spike results in a zoom-in, but the paper does not show when or how often this happens. It would be helpful to analyze where in the reasoning chain these entropy peaks occur and whether those zooms actually lead to better answers.\n\n- Tool-based visual reasoning is becoming increasingly common (e.g., Chain-of-Focus, Pixel Reasoner, DeepEyes). This paper mainly introduces the use of a zoom-in tool for the medical domain rather than a generally novel method. Therefore, it can feel more like a heuristic engineering improvement than a conceptual breakthrough. While combining entropy-based exploration and consensus pseudo-labeling is clever, both ideas are extensions of known techniques rather than fundamentally new principles.\n\nTo make the contribution stronger, the authors could either (1) demonstrate that the approach works on broader visual reasoning tasks including on general domains, or (2) position it as a domain-specific contribution for a medical AI venue.\n\nOverall: The idea is interesting and well-motivated, especially for medical settings, but needs stronger justification, broader evaluation, and clearer evidence that entropy-driven zooming genuinely improves reasoning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dDuYotuoPN", "reviewer": "Reviewer_P4Jp", "summary": "This paper introduces an innovative approach for training medical vision-language models through an annotation-free reinforcement learning framework. The authors demonstrate that reinforcement learning can effectively supervise model behavior using only final ground-truth answers, while incorporating a consensus-driven self-supervision mechanism that leverages model convergence across multiple reasoning trajectories. This design enables the model to learn fine-grained visual reasoning without relying on costly manual annotations. I find the work highly original and thoughtfully executed—the proposed combination of uncertainty-guided exploration and consensus-based reward shaping represents a creative step forward in medical AI. Overall, this is a well-motivated, technically solid, and enjoyable paper to read.", "questions": "See weakness", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- The paper is clearly written and easy to follow, with strong organization and clear motivation.\n\n- It presents a technically rigorous method, combining reinforcement learning, uncertainty-based exploration, and self-supervised reward shaping in a coherent framework.\n\n- The approach offers a novel way to guide RL toward fine-grained visual regions, ensuring that the model’s reasoning is grounded in meaningful image details rather than text-only patterns.\n\n- Empirical results are strong and consistent across multiple benchmarks", "weaknesses": "- Reproducibility: Please release the code repository. For a complex RL-based framework like this, access to the full codebase and training scripts is essential for replication and community adoption.\n\n- Evaluation scope: The paper primarily evaluates on multiple-choice medical VQA datasets, which simplifies the scoring through accuracy. How would the proposed RL and credit-assignment framework adapt to open-ended or free-text medical reasoning questions where correctness is less binary?\n\n- Training initialization:\na) The paper claims annotation-free RL training, yet the model produces structured reasoning traces (<think>, <tool_call>, <answer>). Was a supervised fine-tuning (SFT) or instruction-tuning stage used to teach this output format before RL?\nb) If so, could the authors clarify what data or synthetic prompts were used for this warm-up, and whether this step contributes to the strong structured reasoning behaviors observed in Figure 3?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SEjWV2JDtJ", "reviewer": "Reviewer_9mPg", "summary": "MedVR addresses this by framing the VLM as an agent that can interleave text generation with a `Zoom-in` tool to actively inspect image regions. To train this agent without supervision, the paper proposes two novel, annotation-free mechanisms:\n1.  **Entropy-guided Visual Regrounding (EVR):** This mechanism monitors the model's predictive uncertainty (entropy) during the generation of tool commands. When uncertainty is high (e.g., the model is unsure *where* to zoom), EVR triggers exploratory branching to sample different visual actions.\n2.  **Consensus-based Credit Assignment (CCA):** This mechanism provides a reward signal for visual grounding. It identifies the set of all *successful* reasoning trajectories (those that led to a correct final answer) and generates a \"consensus mask\" based on the image regions they collectively inspected. Trajectories that align with this consensus receive an additional reward.\n\nThe authors claim that this self-supervising loop of exploration (EVR) and credit assignment (CCA) enables MedVR to achieve state-of-the-art performance on several medical VQA benchmarks, improving both accuracy and robustness.", "questions": "1.  What is the input image resolution used for the VLM? How does the `Zoom-in` tool work? Does it crop the original high-resolution image, or does it simply crop the low-resolution feature map? This detail is critical for understanding if the tool is performing a meaningful action.\n2.  Can you provide any quantitative evaluation of the backbone model's zero-shot detection or localization capabilities? How can we be sure that the `Zoom-in` actions, which are the foundation of your method, are targeting clinically relevant regions?\n3.  Why were no experiments conducted at other model scales (e.g., 3B or 34B)? How can readers be confident that this complex RL framework is scalable and not just tuned to one specific 7B backbone model?\n4.  Regarding the OOD baseline comparisons, were all baseline models trained *only* on the OmniMedVQA training split to ensure a fair, controlled comparison for generalization? Or were your training set has similar case with the OOD data?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.  The paper tackles a key problem in medical VLMs: the lack of verifiable visual grounding. The proposed \"annotation-free\" approach, which avoids the need for expensive, step-by-step human annotations, is highly significant.\n2.   The two core contributions, EVR and CCA, are original. Using intrinsic model uncertainty (entropy) to guide visual exploration (EVR) is an intelligent way to focus the agent's attention. Distilling a reward signal from the \"consensus\" of successful trajectories (CCA) is a clever solution to the credit assignment problem in the absence of ground truth.\n3.   The paper correctly identifies the limitations of text-only reasoning and frames the solution as an agentic VLM that can interact with the image, mimicking a human clinician's workflow (e.g., zooming).", "weaknesses": "1.  **Misrepresentation of Results:** The most severe weakness is the contradiction between the paper's text and its tables. The text in Section 4.2 claims to \"surpass\" a key baseline on PMC-VQA, but Table 1 shows a marginal improvement (54.31 for MedVR vs. 54.29 for the Lingshu). This misrepresentation of out-of-domain performance is unacceptable and destroys confidence in the paper's conclusions.\n2.  **Critical Omission of Image Resolution:** The paper's methodology is centered on a `Zoom-in` tool. This tool's function is entirely dependent on the input image resolution, which is never stated. It is impossible to know if the agent is (a) meaningfully cropping a high-resolution image or (b) trivially cropping an already-low-resolution 336x336 patch. This is a fundamental flaw that makes the core premise of the paper unverifiable.\n3.  **No Evaluation of Core Detection Capability:** The `Zoom-in` tool and the entire CCA reward mechanism are critically dependent on the model's ability to *correctly* localize objects of interest. The paper provides no evaluation of the model's baseline detection or localization performance. This is a crucial omission. If the model's inherent ability to draw bounding boxes is poor, then the EVR is exploring random locations and the CCA is finding a \"consensus\" among incorrect boxes, which would poison the reward signal. The paper must demonstrate that the agent's visual grounding capabilities are a reliable foundation for its reasoning.\n4.  **Lack of Model Scaling Analysis:** The experiments are confined to a single 7B model. There is no analysis of how this method performs at different scales (e.g., on a 3B model or a 30B+ model). This makes it unclear if the approach is a general one or a finicky result specific to the 7B backbone architecture.\n5.  **Marginal OOD Performance:** Even taking the results at face value (and ignoring the misrepresentation), the OOD performance is not a strong endorsement. The model fails to demonstrate a substantial improvement on PMC-VQA (even this is easier than MedXpertQA) and is effectively tied on MedXpertQA. Given that the paper's premise is a more robust, generalizable reasoning process, these OOD results are weak.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)