MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoning
Authors: Peng Xia
Jinglu Wang
Yibo Peng
Kaide Zeng
Zihan Dong
Xian Wu
Xiangru Tang
Hongtu Zhu
Yun Li
Linjun Zhang
Shujie LIU
Yan Lu
Huaxiu Yao
Venue: ICLR 2026 Poster (2026)
Keywords: med-vlm
multi-agent collaboration
multimodal medical reasoning
medical vqa
reinforcement learning
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Medical Large Vision-Language Models (Med-LVLMs) have shown strong potential in multimodal diagnostic tasks. However, existing single-agent models struggle to generalize across diverse medical specialties, limiting their performance. Recent efforts introduce multi-agent collaboration frameworks inspired by clinical workflows, where general practitioners (GPs) and specialists interact in a fixed sequence. Despite improvements, these static pipelines lack flexibility and adaptability in reasoning. To address this, we propose MMedAgent-RL, a reinforcement learning (RL)-based multi-agent framework that enables dynamic, optimized collaboration among medical agents. Specifically, we train two GP agents based on Qwen2.5-VL via RL: the triage doctor learns to assign patients to appropriate specialties, while the attending physician integrates the judgments from multi-specialists and its own knowledge to make final decisions. To address the inconsistency in specialist outputs, we introduce a curriculum learning (CL)-guided RL strategy with dynamic entropy regulation, progressively teaching the attending physician to balance between imitating specialists and correcting their mistakes. Experiments on five medical VQA benchmarks demonstrate that MMedAgent-RL outperforms both open-source and proprietary Med-LVLMs. Notably, it achieves an average performance gain of 23.6\% over strong baselines.

Published as a conference paper at ICLR 2026
MMEDAGENT-RL: OPTIMIZING MULTI-AGENT COL- LABORATION FOR MULTIMODAL MEDICAL REASON-
ING
Peng Xia Jinglu Wang"1, Yibo Peng2.3ix Kaide Zeng', Zihan Dong', Xian Wu, Xiangru Tang". Hongtu Zhu', Yun Li', Linjum Zhang', Shujie Liu", Yan Lu Huaxiu Yao 11 -UNC-Chapel Hill, Microsoftt Research, CMU, 'Rutgers University, Yale University (pxia, huaxiu) @cs. unc. unc.edu, {jinglu. wang, yanlu) @microsoft.com
ABSTRACT
Medical Large Vision-Language Models (Med-LVLMs) have shown strong po- tential in multimodal diagnostic tasks. However, existing single-agent models struggle to generalize across diverse medical specialties, limiting their perfor- mance. Recent efforts introduce multi-agent collaboration frameworks inspired by clinical workflows, where general practitioners (GPs) and specialists interact in fixed sequence. Despite improvements, these static pipelines lack flexibility and adaptability in reasoning. To address this, we propose MMedAgent-RL a reinforcement learning (RL)-based multi-agent framework that enables dynamic, optimized collaboration among medical agents. Specifically, we train two GP agents based on Qwen2.5-VL. via RL: the triage doctor learns to assign patients appropriate specialties, while the attending physician integrates the judgments from multi-specialists and its own knowledge to make final decisions. To address the inconsistency in specialist outputs, we introduce a curriculum learning (Cl.)-guided RI. strategy with dynamic entropy regulation, progressively teaching the attending physician to balance between imitating specialists and correcting their mistakes. Experiments on five medical VQA benchmarks demonstrate that MMedAgent outperforms both open-source and proprietary Med-LVLMs. Notably, it achieves an average performance gain of 23.6% over strong baselines.
INTRODUCTION
Large Vision-Language Models (LVLMs) are becoming increasingly proficient in visual understand- ing and reasoning (Liu et al. 2024a b Zhu et al., 2023 Bai et al. 2023] Chen et al. (024c). This advancement also making a significant impact in the biomedical domain, where Medical Large Vision-Language Models
LVLMss have demonstrated great potential in enabling intelligent diagnostic applications (Li al. 2023 Moor et al., 2023 Nath et al. 2024). However, as shown in Figure D (a) left, although a single Med-LVLM can be trained with large amount of data and show promise results to some extent, it is challenging for a single model to handle diagnostic expertise from different subfields (c.g., radiology, pathology, etc.).
Therefore, some recent works propose using multi-agent collaboration to solve medical tasks (Li al. 2024c Kim et al. 2024 Tang et al. 2024), where different models act specialists or general practitioners, collaborating and discussing to arrive at a final answer, improving overall performance compared to single agent. These works follow the steps of simulating hospital visit process and adopt a General Practitioner (GP) - Specialist - GP workflow. First, the general practitioner (i.e., the triage doctor) classifies the patient based the consultation questions and images and selects the appropriate department from several predefined specialties. Then, specialist doctors from the relevant departments provide their diagnoses. Finally, the general practitioner (i.e., attending physician) makes the final decision based on the images, consultation questions, and the diagnostic results from multiple specialists. However, as illustrated in Figure (a) middle, such workflows are inherently static. Such interaction pattern between agents is fixed and predetermined, which limits the system's capacity for flexible, optimized reasoning across multiple modalities.
Work done Microsoft Research. Equual Contribution. 1 'Corresoondigg Authors.

---

Published as a conference paper at ICLR 2026
To address this challenge. motivated by the success of Reinforcement Learning (RL)-driven
reasoning (Jaech et al. [202]] Guo et al.) 2025] Team et al. (025). as shown in Fig-
ure T (a) right, perform stage-wise traming on two GPs based Qwen2.5-VL (Bai
et al. ) via reinforcement learning, namely the triage doctor and the attending physician.
Specifically, first, for the
first GP. i.c. triage doc- tor, we utilize the image modality information pro- vided by the dataset itself to reinforce the triage doc- tor, such as pathology slides - Pathologist, ensuring that the triage doctor can accurately assign patients to the appropriate depart- ment. Then, we use power- ful proprietary models like GPT (OpenAl 2025)
Single Agent
Multi-Agent
Data
Agent
PUnVQA
OrmMedVVA
MedVVLTTiner-7 7B
GPT-40
LLaVA-l 6:34
MadAgartte
Ouss (MAAeckgart-FLL
(+)
(b)
Figure 1: Comparison of Med-Agent paradigms: single-agent - static workflows - dynamic collaboration. (a) Motivation: Single-agent models struggle with domain specialization, and prior multi-agent sys- tems rely on fixed workflows, limiting adaptability. We propose a trainable ceasoning-enhance multi-agent system via RL. (b) Perfor- mance: Our method is highly competitive across multiple benchmarks.
play the role of the spe- cialist doctors and generate initial judgments. Finally, the second GP, i.e., attend- ing physician, integrates do- main knowledge from mul-
tiple specialists and their own judgment to make the final decision. Here, during the process of the general practitioner integrating specialist doctor information, while specialist doctors provide valuable domain knowledge, their judgments not always perfectly accurate. These inconsistencies in specialist performance can introduce noise into model training, preventing the model from simply memorizing blindly replicating their outputs. Instead, the model must learn to generalize beyond potentially flawed expert judgments.
To address this, inspired by Curriculum Learning (CL) (Bengio al. 2009 Pentina et al. 2015
Deng et al. 2025), which enables models to be trained progressively on increasingly difficult tasks, implement an entropy-aware reinforcement learning approach based on CL, aiming to help the model gradually learn to leverage the knowledge of specialist doctors. The core principle is that exploitation dilemma when integrating specialist opinions: it must learn when to trust and adopt consensus and when to challenge it and search for novel diagnostic paths (Cui et al. 2025) [aan al.) (0255)). Specifically, we use the accuracy of specialist results as a flag to classify the training data by difficulty: specialist results that are completely correct are labeled as easy, partially correct as medium, and completely incorrect as hard. In this way, we design three-stage curriculum reinforcement learning process for optimizing the attending physician to handle diverse specialist results, where the model starts with policies to exploit reliable "easy" cases and gradually increases its policy entropy to explore and correct flawed judgments in "hard" cases.
the attending physician agent faces a dynamic explovation-exxllitaatio
The primary contribution of this paper is MM. RL-driven framework optimized for multi-agent collaboration in improving medical reasoning. Empirical results on five medical multimodal datasets, shows that the model performs exceptionally well not only on in-domain datasets but also on out-of-domain datasets, outperforming a series of both open-source and proprictary LVLMs, exceeding SET method by 23.6%.
2 PRELIMINARIES
In this section, we will provide a brief overview of LVLMs, multi-agent collaboration and GRPO.
Large Vision Language Models. LVLMs enhance LLMs by integrating visual input x, with textual input 2,, forming a joint input x = (æv, 21). They autoregressively predict the next token's 's distribution to generate textual response y.
Multi-Agent Collaboration. To support complex workflows, multi-agent frameworks coordinate specialized agents. Our setting simulates hospital visit: GP -> Specialist(s) GP. Each agent
2

---

Published as a conference paper at ICLR 2026
Question
Initial Departmental Triage
Specialist Consul tatsonnation
tssuu have
granulation
answer j8 B.
First
..A.
infarmatiory
Yes
cell infitrate?
No
Tragge Doctar
No...
Reward Funchion
RL Updaled
Format
Specialist Doctors
Accurasy Reward
Ressoning: ettickr Acmov
Curriculum Reinforcement Lesrning with Entropp Reguletion
Collaborative Decision-Moke
gramtiction dssze
chanaterriee by swsendl
festuees inclusing
Expert Options
DOG
vassess Abrous
3ssue loose matck....
bload
Entrocy
Entropy
Entrapy
Firel. A
The anrower B
Attendiig Physizian
Answer: KaTSMOTAA yos cheoveer
Hard
Medium
Easy
than
Figure 2: Overview of MMedAgent-RRR -eaiggentttt,,tt RL-driven mulli--gent framework designed for multimodal medical reasoning. It simulates the clinical loop of General Practitioner (GP) - Specialists - GP. First, MMedAgent-RL. optimizes the triage doctor (the first GP) to improve triage accuracy. Then, proprietary LVLMs are used as the specialist doctors for the assigned department. Finally, curriculum learning and RL are combined to progressively train the attending physician (the second GP), who integrates the specialist knowledge and makes robust decisions.
a: E A follows policy тe, (y 2), with multimodal input x = (I,, (I,, where V. is an image, Iy is a text instruction, and y is the output. GP agent: aGri specialists: {ase (3) ...., (K)) The workflow proceeds as follows: 1) Triage: AGP selects department via = arg max Tornor (h x). 2) Specialist:
Ssp produces response yd Toin ) Aggregation: sesend outputs ylinal Tomma (y Ha).
Group Relative Policy Optimization (GRPO). Group Relative Policy Optimization (GRPO) Guo et al. )025) is a reinforcement learning method that avoids training a critic by using intra-group relative rewards
optimize the policy. For each query x, the model samples G responses {v(") ..... (......,,,)) (G),, ), which are scored to get rewards {RI..... Rc). GRPO computes normalized advantages and updates the policy with a PPO-style clipped objective (Schulman et al.] 2017):
Tolyi 1r) (3/
(f) Eur
min (r.A,, clip(r., 1- -,,1+)))) (Fol (ue))
G
xx)
(2.1)
where A, = me-mn (R, 1. o)) e, 3 are hyperparams, and Told is a old policy model. GRPO enables scalable policy learning using only relative rewards, without critic.
sul R2 15-o)
3 METHODOLOGY
In this section, as illustrated in Figure 2 we will present MMedAgent-RL. a RL-driven multi-agent framework for multimodal medical reasoning by emulating structured clinical workflow. Our approach begins with the first General Practitioner (GP) leveraging the input to select the appropriate medical department for further consultation. To optimize the initial triage decision, employ GRPO to refine the triage doctor's 's capabilities. Then the case is referred to a panel of specialist doctors, each served by a proprictary LVLM specialized the identified department, analyze data and provide expert opinions. Finally, the process culminates with the second GP, acting as the attending physician, integrates all specialist opinions form final diagnosis. We will delve into each stage as follows:
3.1 INITIAL DEPARTMENTAL TRIAGE
In real-world medical treatment, particularly for complex diagnoses, the workflow fundamentally re- lies on a Multi- Siltii-Disiipinaryy Team (MDT) process EAbo-Hamad & Arisha 2013): moving from initial assessment and triage to expert consultation, and finally, synthesis by an attending physician. This collaborative paradigm has been validated in recent medical AI literature as an effective abstraction for simulating clinical decision-maker making. Previous works like Agent Hospital et al. 2024c) and
3

---

Published as a conference paper at ICLR 2026
MDAgents (Kim et al. 2024) have adopted similar role-based structures, employing LLMs for triage or dynamic collaboration. However, these methods typically rely on rigid or predefined assignment strategies and lack the ability to update the model based on new data.
iriage who acts as the To address this challenge, the first step is optimize the general practitioner aGP
triage doctor (i.e., policy Togre) ). Here, we use the image modality information provided by the dataset itself as ground truth labels y" train the triage model. For example, pathology slides - pathologist (e.g., Path VQA contains pathology slide images and is thus assigned to pathologists), ensuring that the triage model can accurately assign patients to the appropriate medical specialty.
Specifically, when prompting the triage doctor, we provide k candidate specialties. In our setup, l: is set to 7, including Pathologist, Radiologist, Surgeon, Oncologist, Endocrinologist, Ophthalmologist, and Dermatologist, which broadly cover the main departments involved in the data. Our aim is not only to improve triage accuracy, but also to strengthen the model's reasoning process, helping explain why particular triage recommendation is made. Thus, we use GRPO as the base RL. algorithm. At this stage, the reward function adopts rule-based format with rewards Rivema E {0, 0.5) and accuracy rewards Raccurcc E {0, 1]. A reward is given when the output format meets the required criteria and the chosen specialty is correct. This stage can optimize the triage doctor's performance. improving their ability to select the appropriate specialty d = arg max; Tegin (k x)), and lays foundation for subsequently acting as the corresponding specialist.
3.2
SPECIALISTS OFFER VALUABLE INSIGHTS
After obtaining the department from the triage doctor, following previous work using LLMs or LVLMs for medical diseussions (Li et al., 2024c Tang et al. [004]] Li et al. (2024a). we utilize several
powerful models as specialist doctors asp to provide relatively accurate preliminary judgments. This facilitates subsequent reference by the attending physician. In our setup, we use responses from specialists as references for each sample. We only require the specialist doctors to independently provide expert opinions Yd ~ "of (y x) within their specialty, without engaging in complex interactions. This ensures system efficiency and avoids majority voting that could overshadow minority opinions, leaving the final decision to the attending physician.
(d)
3.3 EVOLVING DECISIONS BY ATTENDING PHYSICIAN VIA ONGOING COLLABORATION
After getting the responses from the specialists, we then integrate their knowledge into the final general practitioner designed to support the final diagnostic decision. The final decision-making agent, namely the attending physician, plays the most crucial role throughout the diagnostic process, as they must synthesize diverse expert opinions and draw upon their own clinical expertise to arrive at final judgment. This poses significant challenges for the attending physician, the specialists' conclusions are not always fully reliable. Over-reliance on specialist input can lead to suboptimal outcomes. Secondly, different specialists may offer conflicting interpretations of the same case, creating misalignment issues. If the model is unable to reconcile its internal reasoning with external expert input, risks compounding errors. For instance, while majority voting may help mitigate the influence of less competent specialists, it can also suppress minority views, including potentially the only correct one. Such multi-agent collaboration can yield adverse effects when the model is not properly aligned with the nature and limitations of expert knowledge.
To address these challenges, we draw inspiration from curriculum learning (Bengio et al., 2009). which emphasizes the importance of organizing learning experiences in meaningful progression, i.e., from easier to harder tasks. Motivated by this principle, see in Alg. we propose the Curriculum- based Entropy-Aware Reinforcement Learning for Multi-Agent Collaboration (C-MARL). The core principle of this framework is that the attending physician agent faces a dynamic exploration- exploitation dilemma when integrating specialist opinions: it must learn when to trust and adopt consensus (exploit) and when to challenge it and search for novel diagnostic paths (explore). We posit that the policy entropy is the central mechanism for navigating this trade-off (Cui et al. (2025). Therefore, C-MARL employs a purpose- -esiined curriculum to actively shape the policy entropy of the attending physician, enabling it to adapt to specialist information of varying reliability.
4

---

Published as a conference paper at ICLR 2026
Algorithm 1: Curriculum- Based Multi-Agent Reinforcement Learning (C-MARL)
Input:
Dataset 2 (x 26 (
Jam1. policy model Ta. old policy Matd, group size G. specialist
responses w/d
Output: Rd.
Initialize Deasy Dmedium Drard as empty sets
foreach (Ie, 2t, (=") E D do
bs Use Specialists' Accuracy to Categorize the Dataset by Task Difficulty Calculate the accuracy of the specialist doctor +- Acc(ya, )²) i 1 then
Put ((t,, y²) into Pessy
0 < s < then
Put [1 ) into Dmedium
if's O then
10
Put ((Te, att) y into Drand
foreach 21, E (Deary Dmedhu, Phard) y in batch do
12
Unilize the RL. Algorithm for Optimization at Each Stage
13
Sample G rollouts [Minat Vinal
Vinol from Told, where Vimi <- To (y) (Ev, 2e), Me)
(G)
14
foreach rollout V/nnal do Calculate the outcome reward R (((mm)) Riprnaa (Himal) ) + Rescuracy (Winal)
15
Compute the groupwise advantage A, &- Re-meanl (RR, 19
16
ad(( 15-01
17 Calculate the entropy regularization term H, Σ)= Pc., logpes
Compute the loss in Equation 3.1 and update
Curriculum Design based on Specialist Reliability. Unlike previous curriculum learning ap- proaches that define difficulty based on problem formulation or data domains, we categorize tasks based on the accuracy of specialists' diagnoses ya Tole (ya | 2)), denoted by Acc(yd. y"). The dataset is divided into three levels: fully correct specialist results (8 = 1) are labeled as easy, partially correct results (0 < 1) are labeled as medium, and completely incorrect results (s 0) are labeled as hard. The datasets corresponding to the three levels are denoted as Deasy, ' Dmdiium: Dhard respectively, and D Deasy U Pamddium U Dhard- Based on these data of three categories, we design three-stage curriculum reinforcement learning process to optimize the attending physician's 's ability to handle different types specialist knowledge, such as when to accurately leverage specialist knowledge and when to rely on their own understanding to solve problems.
Reinforcement Learning with Dynamic Entropy Regulation. We utilize GRPO RL algorithm. For each query z, the attending physician Tguns generates group of responses
base
GG i=1 The reward R2 for each response is determined by a format reward Riormat E 0.50 and an accuracy reward Raceurcy E {0, 1}, and GRPO calculates the relative advantage A; (R,
(M
me (R,)) for the policy update. To achieve dynamic entropy control, we introduce an entropy
regularization term into the standard GRPO objective function:
V
JCMARI (0) = [or To H, (TTTgn))) H.
Σ
Pts log Paps
(3.1)
where
= = Rece, 2;; Softmax
Here, JGRPO is the PPO-clip loss term from GRPO in Eq. 2.1 V is the vocabulary size, E RV are the pre-softmax logits, and T is the decoding temperature. Critically, the entropy bonus coefficient 7 is not a fixed hyperparameter but is dynamically set based on the curriculum level s of the current sample. For s = 1, we set Yeasy Rs 0, as we do not need to explicitly reward exploration when the agent should be learning to exploit reliable specialist knowledge. For 0 < < 1, we use a moderate positive bonus Tmedium > 0 to encourage policy diversity and prevent the agent from becoming overconfident in the face of conflicting information. For O, we apply strong positive bonus "hand >> Tmediuu to aggressively incentivize exploration, compelling the model to break from the misleading specialist consensus.
5

---

Published as a conference paper at ICLR 2026
4 THEORETICAL ANALYSIS
In this section, we provide theoretical insights explaining why curriculum learning works better than the usual SGD when optimize the GRPO objective function [.]]]oon3]]]]] Specifically, begin by analyzing the effectiveness of the curriculum learning strategy for policy learning. Our goal is to determine the number of samples and iterations required to achieve specified error tolerance.
The curriculum learning procedure can be simplified as follows. We consider J batches of samples arranged in increasing order of difficulty. In setting (Section 3.2 the curriculum is composed of = 3 difficulty stages: easy. medium, and hard. Starting from 1. we sequentially train on each batch using Stochastic Gradient Descent (SGD) to obtain the policies "0,.x, (y/ x).j I....,J.
We aim to track the policy's 's trajectory and convergence throughout this process. To model the difficulty of the data from each batch, we make the following assumption and suppose the target parameter 8" f
Assumption 4.1. From the easiest to the hardest dataset, for j-th dataset, we assume
Dj = (2.. 9:)
24 p(x), ys Taz (y) m)}
Theorem 4.1 (Informal). Suppose Assumption 4.1 holds, logether with some regularity conditions B.6 in Appendix B). Take e L L2 ///.. For curriculum learning, order to
(Assumptions B.2
make the error [0 0 112 < 4s,, set n, satisfies Rn, (II) S e1/4, where Rag (1I) is the Rademacher complexity of policy class IT and uses the learning rate S /5/ p//.2, and set the total iteration as
2 100 8 1111
K=Ω
log
FIM
For regular SGD, in order to make the error 100g and uses the same T3, and take
0ig12
<
4es
set Drg to satisfy Ry, (TT) S 61/4 Li
L2 110
0gg 11
K = Ω
log
p177
With probability at least 1 - 1eexp
neati 2
-
DE exp
n,ei 2B
(J + 1)E1 we have
0 - 0-
1 - 1-
Here, 0g is the starting point; Oet and Org are the final outputs of curriculum learning and standard SGD, respectively; nj is the sample size of batch j: Brg is the total sample size for standard SGD; measures the distance between the true minimizer 6* and the population minimizer of standard SGD
Og. Bi, L1, Lz, (2--) and (.)()) are defined in Appendix B
Remark 4.1. This theorem provides formal justification for the convergence of curriculum learning. The total training time, K, depends the sum of logarithmic distances between the optimal policies of consecutive stages: EjD log |00 0 l12 An effective curriculum ensures these intermediate distances small, allowing the solution from each stage serve strong warm start for the next. CL thereby decomposes a single, challenging optimization problem, leaping directly from an initial Oī the final 0) into sequence of more tractable sub-problems, creating an efficient optimization path. In contrast, we establish lower bound for standard SGD which shows that, under these conditions, it fails to converge the optimal policy. Details are provided in Appendix B
EXPERIMENTS
In this section, we evaluate the performance of MMedAgent-RR. aiming to answer the following questions: (1) Can MMedAgent-RL. effectively improve model performance compared to other LVLMs and the Qwen2.5-VIL V---baed baselines? (2) How does MMedAgent-RL perform on out-of- distribution datasets? (3) Does each proposed component contribute to performance gains? (4) What the impact of choosing different models as specialist doctors? (5) Does MMedAgent-RR RL truly enhance the model's capabilities across various specialist configurations?

---

Published as a conference paper at ICLR 2026
Table 1: The results of the medical VQA benchmark. Here, MMMU denotes MMMU (Health Medicine track). The best results and second best results are highlighted in and blue respectively, Majority voting is used for the test-time scaling (TTS).
mn-Domain Datasets PuthVQA
Duttrf-DDstrrbuution Datasets
Model
VQA-RAD
SLAKE
Avg.
OmniMedVQA
MMMU-Med
Avg.
GPT-4o
61.0
75.5
69.4
68.6
68.5
69.7
Med-Flamingo
45.4
43.5
54.7
47.9
30.7
28.3
29.5
RadFM
50.6
34.6
28.2
27.0
LLaVA-Med-7B
51.4
48.6
56.8
52.3
44.1
36.9
40.5
Chat
32.7
Yi-VI--34B
53.0
58.9
47.3
53.1
51.5
41.5
46.5
LLaVA-<1.6-7B
52.6
$7.9
47.9
52.8
49.0
33.1
41.1
LLaVA-v1.6-133
55.8
58.9
51.9
55.5
48.0
39.3
43.7
LL.VA--11.634BB
58.6
67.3
59.1
61.6
58.7
48.8
53.8
54.2
59.4
54.1
55.9
44.6
38.2
41.4
HuatuoGiPI-Vssion-77
77.2
66.3
Qwen2.5-VL.c. VL
61.0
62.7
$7.6
60.4
60.1
54.5
$7.3
Qwen2.5.VL.P7
MedVLThinker-77
63.7
67.8
65.2
65.6
62.4
57.0
59.7
Multi-Agent Collaboration
MedAgents
65 6
67.9
63,2
65.6
55.8
49.7
52,6
MDAgents
66.8
68.2
65.4
66.8
58.2
52.3
AFlow
67.3
68.9
66.4
67.5
59.6
53.6
56.6
MMcdAgent-Ri (7B)
71.5
76.2 w26
72.3 128
73.3 1/M
73.3
71.9 -20%
-2.6
Test-Time Scaling
73.9 128
80.1
74.3
76.1
1100
1196
73.5
117
76.6 1889
5.1 EXPERIMENTAL SETUP
Implementation Details. We use Qwen2.5-VL Bai et al. 2025) as the base model. We design the clearly specifying the required output structure, which includes
prompt template shown in Table 4 using <think> and <answer> tags separately contain the reasoning process and the final answer. 128, with 8 rollouts generated for each
The rollout batch size and training batch size are both sample. The sampling temperature is set to 1.0 to encourage response diversity, and optimization is done with learning rate of 10-6 The KI. divergence coefficients are and 10- respectively for curriculum reinforcement learning, For the number of specialists, set ε 3. The details are shown in Appendix F22
x 10-3 10-3
Dataset Splitting and Difficulty Stratification. We adhere strictly to the official guidelines for partitioning the training, validation, and test sets. To facilitate our curriculum learning strategy, we further categorize samples into three difficulty levels (Easy, Medium, Hard) based on the consistency and accuracy of specialist responses. is important to note that this difficulty grading utilized exclusively during the training phase schedule the curriculum. During inference, the model generates responses without access to difficulty labels or ground truth information. The difficulty- based breakdown presented in our analysis is applied post-hoc to the test solely visualize model robustness against specialist noise, ensuring no data leakage.
Baseline Methods. We compare MMedAgent-RL. with methods under two different settings:
Single-agent setting: This includes series of state-of-the- LVLMs, encompassing both general LVLMs and Speaai- LVLMs. Specifically, we include comparisons of the LLaVA series (Liu
et al. 2024a). Yi-VI-334B (Young et al. 2024), Qwen-VI. VI. (Bai et al. 2025). LLaVA-Med (Li et al.
2023
MedFlamingo (Moor al. 2002), RadFM (Wu et al. 2023). MedVL.Thinker-778 Huang et al.
(2025 and GPT-4o (OpenAl 2024). 2) Multi-agent setting: This includes MedAgents (Tang ct al.
2024 MDAgents (Kim et al. 2024). AFlow (Zhang et al. 2024).
Data and Metrics. We train the three medical VQA datasets, i.e., VQA-RAD Lau et al. 201 2018), SLAKE (Liu et al. [021), Path VQA (He et al. 2020). Their test sets are considered the in in-domain select the health and medicine subset of MMMU (Yue et al.) 2024). and OmniMedVOA (Hu et al.) (0224)) as out-off-dstrribu datasets. All evaluation questions multiple-choiee, and accuracy is used the evaluation metric.
test sets. Additionally, following Chen al. (2024a).
7

---

Published as a conference paper at ICLR 2026
5.2 MAIN RESULTS
In this section, conduct comprehensive comparison the medical VQA task involving six datasets and various LVLMs as well as baseline methods based on Qwen2.5-VL.
Comparison with Baselines in In-Distribution Datasets. Table T shows the performance of various models across four medical VQA benchmarks. General LVLMM like LLaVA-vl 1.6.633B3 and GPT-4o exhibit consistently strong performance, outperforming earlier medical-specific models such as Med-Flamingo and RadFM. Notably, GPT-4o achieves the highest average score (68.6%) among all single-agent models, demonstrating its powerful generalization capabilities even in specialized medical domains. Interestingly. the multi-agent collaboration strategy further boosts performance. MMedAgent-RL. achieves the best overall average (73.3%), surpassing even the strongest single-agent models. This highlights the effectiveness of collaborative inference in leveraging the complementary strengths of different models. In addition, we utilized majority voting as the method for test- time scaling (Snell et al. (024), which further improved the model performance by 4.5%. This demonstrates that optimizing token entropy during the RL process is positive for efficient sampling. For different specialists, a varied spirit of exploration remains very important.
Performance Out-of Distiiiiiiiitiionn Datasets. We evaluate the performance of MMedAgent-RL across various out-of out-of-distribun disstrriibttinn (OOD) datasets. The results presented in Table which demonstrates the generalization of our approach in adapting to different OOD scenarios. These two OOD datasets cover multiple body parts and involve various medical image modalities. Through rein- forcement learning, outperforming the base model by 21% and the SFT method by 23.6%. Moreover, it surpasses the performance of multi-agent collaboration methods that cannot optimize models, i.c., MedAgents, MDAgents and AFlow, by 23%. 19% and 17%, highlighting the effectiveness of our approach in handling diverse and unseen data distributions.
demonstrates significant superiority across multiple modalities,
5.3 ANALYSIS
3'63
("HustuGPP
71.30
Owird.5VV
4P Owend VL MARL (Easy)
2'o3r1 1Owon2.55LL V-VL
3'GPT-bo
71.20
Avenge Performenn
GPT-40 Cwene C-WRL (Hard)
GPT-40 Owenn VL C-WRL (Medium)
2'CPT-1p
00.00
2
69.20
POPT-4exTHumoOPP "HuswoOPT
08:00
"Owon2.5vV V-VL
1'GPT40
("HustuoGPT
81:80
1'Qwon2.5VV
Avarge Peroomanc
Easy
Medium
Hard
Figure 3: Results of different settings of specialist doctors.
Figure 4: Results under different levels of deci- sion difficulty.
In this section, we conduct a detailed performance analysis at each step and explore how model type, numbers of specialist doctors, and varying levels of decision difficulty affect the results, better understand the performance gains achieved by MMedAgent-RL.
Ablation Studies. We conducted series of ablation experiments evaluate the impact of each component in MMedAgent-RL as shown in Table We can see that: (1) Reliable triage doctors are important. Accurately determining the department which a specialist doctor belongs helps the model call upon knowledge from their corresponding field of expertise to answer questions, improving the accuracy of specialist doctors' answers. A fine-tuned triage doctor significantly improves model performance compared to the original model, with an average performance increase of 3% across multiple datasstts. (2) Based on this, the mechanism of specialist doctor consultation is introduced, further helping the increase of 4.5% across multiple datasets. (3) Most importantly, the addition curriculum multi multi-agent reinforcement learning (C-MARL) enhanced the decision-maie making agent's 's understanding of specialist doctors' knowledge, achieving a significant performance improvement of 18.6%. This indicates that C-MARL can effectively solve the problem of overall misalignment between the model and external
agent fully utilize expert opinions, with average performance
8

---

Published as a conference paper at ICLR 2026
knwoledge. Specifically, each stage plays corresponding role and can understand the specialist doctors' knowledge according to the goals of different stages, achieving overall performance gains.
Analysis of Specialist Doctors. We ana- lyze the types and number of models play- ing the role of specialist doctors. Specifi- cally, as shown in Figure [ regarding the model types, the performance of the final pacision-making agent is closely related to the performance of the specialist doctors. Therefore, we used a series of models that performed well on multiple datasets, such as o3, GPT-4o, HuatuoGPT- Vision, and Qwen2.5-VL, as specialist doctors. Since the areas tasks that each model excels in are not completely consistent, the specialist doctor played by 03 ultimately performed the best. Its performance across various aspects was relatively balanced, enabling MMedAgent-RR RL to achieve the best performance. Refer to Appendix G.2] for details.
Table 2: Ablation results on ID and OOD datasets.
Model
OOD
VQA-RAD
SLAKE
OmniMedVQA
MMMU
MMeeAAgent RL
71.5
76.2
73.3
71.9
w/o Triage
66.3
69.9
66.2
59.3
w/o Specialists
65.8
67.8
64.4
wio C-MARI.
63.5
65.5
$7.9
50.2
Easy
64.7
69.3
68.2
58.0
Medium
69.4
76.9
70.8
Hard
71.5
76.2
73.3
71.9
Performance under Different Levels of Decision Difficulty. To analyze the robustness of the GP agent against noisy advice, we categorize test samples into three difficulty levels based on the consistency and correctness of specialist outputs: Easy (all specialists are correct), Medium (spe- cialists disagree). and Hard (specialists consistently provide incorrect advice, creating a misleading consensus). Figure iillustrate the performance of baselines versus MMedAgent-RR across these categories. While specialist noise-particularly in the 'Hard' baseline model's 's decision-making distinguish and selectively utilize specialist knowledge. Consequently, MMedAgenttRR achieves an overall performance improvement of 20%, with the most significant gains observed in overcoming misleading specialist hallucinations in the hard cases.
hampers the our C-MARL method enables the agent to gradually learn to
Case Study and Outlook on "Aha Moments". shown in Figure 5 MMedAgent-RL. demonstrates strong performance across multiple cases. It provides accurate answers within the <answer> tags and generates high-quality reasoning resembling that of human doctors: defining the disease, analyzing images, and checking consistency with the definition. It also evaluates specialists' outputs before reasoning out the correct answer. While lacking the "aha moment" observed in humans, this structured reasoning highlights the potential for more human-likk scientific Al systems.
6 RELATED WORK
Medical Vision-Language Models. The advancement of Vision-Language Models (VLMs) (Liu
et al. 2024a b Zhu et al. 2023 Bai et al. 2023
Chen et al.] 2024ch has catalyzed significant
progress medical applications (ia al. 2024a bi Chen et al. 2002a Zhu et al.] 2024), with
(Chen et al. 2024a), and VILAAM3 (Nath et al. 2024) demonstrating profound results in medical diagnostics. However, single models struggle to handle cross-domain expertise. Although multi-agent systems (Li et al. 2024c| Kim et al., 2024, Tang et al. 2024 Zhang et al.) 2024) have been proposed to combine diverse medical expertise, existing approaches typically use preset workflows that lack adaptive reasoning capabilities. Works such as MedAgentsBench (Tang et al.) 2025) and AI Hospital (Fan et al. 2025) have established frameworks for evaluating multi-agent interactions and diagnostic thinking. Furthermore, AgentClinic (Schmidgall et al.) (024] introduces multimodal benchmark to simulate clinical environments specifically for evaluating embodied agents. Additionally, most Med-VLMs are developed through supervised fine-tuning (SFT) on general VLMs using biomedical instruction data (Chen et al., [2024a] [i et al. [2023 Liu et al. 2023), which is limited by the scarcity of high-qualit, reasoning examples and often results in models that struggle with complex diagnostic reasoning across specialties.
large-scale models like LLaVA-Med (Li et al.) 2024b). CiaatuGGPTVisstie
Reinforcement Learning for Multimodal Reasoning. To address the limitations of static multi- agent systems and overcome the constraints of supervised fine-tuning, Reinforcement Learning (RL) offers promising alternative for optimizing medical reasoning. RL evolves from establishing foundational frameworks for learning from human preferences (Christiano et al. 2017: Ziegler et al. 2019) to developing sophisticated approaches like RLHF for instruction following (Ouyang et al.
2022) and self- colfecorrection (Kumar et al. 2024 Wu et al., 2025). Recent advances with DeepSeek-
RI (Guo et al. 2025) demonstrate that L.MS leverage RL to enhance reasoning capabilities in

---

Published as a conference paper at ICLR 2026
Question: does muscle show kidney, adult polycyatic kidney?
Question: does the histologic appearance dernonstrate small round, uniform nuclei and moderate cytoplasm?
Options: yes, B. no.
Answer: B. no.
Options: A. yes, B. no.
Answer:
Ours: To distermin muscle shows kidney or adult polycystic
kidney, we would typically look for specific characteristics such the presence renal tubules, glomerul,, and the typical appearance kidney
Ours: The histologic appearance described in the image shows small uniform
and moderate cytoplasm, which characteristic certain types of cancer cells, such as those found in breast or ovarian cancer. The pathologist' is suggestion of 'No, the nuclei vary size" is less likely betause the nuclei in the
microscope. The raciologist's report mentions "no," which indicates that this image is not related to adult polyoyatic kidney disease. The pathologlst's report mentions "Not", Given these points: it unikelly that muscle shows kidney canswersB. nodlarewers
clearly of uniform these characteristics. canswer>A, yesc/answer>>
which usually doesn't show
polycystic kidnnyy
Figure 5: Several case analyses. In the model responses, blue text represents the process of reasoning about relevant medical knowledge based on the question, and red text represents the analysis of the answer provided by the specialists.
complex tasks without supervision, showing exceptional performance in mathematics and coding challenges Yeo et al. 2025). This success has extended to multimodal reasoning (Meng et al. 2025
Shen et al. 2025| Wang et al.
2025a Liu et al. 2025b Geng et al. 2025 Zhang et al. 2025b Xia
et al. 2025 Liu ct al.) 2025a
Su ct al. (025), including the biomedical domain (Pan et al. 2025).
DoctorAgent-RL. (Feng et al.
2025) employs a multi-agent collaborative RL system specifically to optimize multi-turn clinical dialogue policies. Then Med-U1 (Zhang et al.) (2025a) utilizes large- scale RL to incentivize unified reasoning patterns in medical LLMs. However, prior RL-based approaches for multimodal reasoning have primarily focused optimizing a single model, leaving the potential of RL for enhancing multi-agent medical collaboration largely unexplored.
7 CONCLUSION
This work presents MMedAgent-RL, a novel RL framework for multi-agent collaboration in medical multimodal reasoning. The framework mimics a clinical "triag--and-referraa" system, using curriculum RL strategy to train primary model to intelligently handle noisy or confficting inputs from different "specialist" agents. Experiments demonstrate the method's strong performance across multiple medical visual question answering datasets, offering a promising new direction for building reasoning models that more closely emulate human diagnostic thinking.
Ethics Statement. All authors have read and comply with the ICLR Code of Ethics. This work does not involve human subjects or sensitive data, and we are unaware of any potential misuse, harm, or bias. No conflicts of interest or compromising sponsorships exist.
Reproducibility Statement Details of the proposed methodology, training procedure, hyperparame- ters, and evaluation metrics are provided in Section F We include complete algorithm pseudocode in Alg. T and full description of the datasets in Appendix D and Appendix
REFERENCES
Waleed Abo-Hamad and Amr Arisha. Simulation- based framework improve patient experience in an emergency department. European journal of operational research, 224(1):154-166, 2013.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang. Junyang Lin, Chang Zhou, and Jingren Zhou Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint 250.. 2025.
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine leurning, pp. 41 41-48, 2009.
Junying Chen, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji, Guangjun Yu, et al. Huatuogpt-vision. towards injecting medical visual knowledge into multimodal llms at scale. arXiv preprint arXiv:2406. 19280, 2024a.
10

---

Published as a conference paper at ICLR 2026
Zhe Chen, Weiyun Wang. Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05277, 2024b.
Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing. Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistie tasks. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 24185-24198, 2024c.
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017.
Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang. Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al. The entropy mechanism of reinforcement learning for reasoning language models, arXiv preprint urXiv:2505 22611, 2025.
Huilin Deng, Ding Zou, Rui Ma, Hongchen Luo, Yang Cao, and Yu Kang. Boosting the generalization and reasoning of vision language models with curriculum reinforcement learning. arXiv preprint arXiv:2503.07065, 2025.
Zhihao Fan, Lai Wei, Jialong Tang, Wei Chen, Wang Siyuan, Zhongyu Wei, and Fei Huang. Ai hospital: Benchmarking large language models in a multi-agent medical interaction simulator. In Proceedings of the 31 International Conference Computational Linguistics, pp. 10183-10213, 2025.
Yichun Feng, Jiawei Wang, Lu Zhou, Zhen Lei, and Yixue Li. Doctoragent-rl: A multi-agent collaborative reinforcement learning system for multi-turn clinical dialogue. arXiv preprint arXiv:2505.19630, 2025.
Xinyu Geng, Peng Xia, Zhen Zhang, Xinyu Wang, Qiuchen Wang, Ruixue Ding, Chenxi Wang, Jialong Wu, Yida Zhao Kuan Li, et al. Webwatcher: Breaking new frontier of vision-language deep research agent. arXiv preprint arX4v 2025.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in lims via reinforcement learning. arXiv preprint arXiv:2501.12945, 2025.
Xuehai He, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint arXiv:2003. 10286, 2020.
Jian Hu, Xibin Wu, Zilin Zhu, Weixun Wang, Dehao Zhang, Yu Cao, al. Openrlhf: An easy-to-use, scalable and high-peeformance rlhf framework. arXiv preprint arXiv:240S.11145 2024a.
Yutao Hu, Tianbin Li, Quanfeng Lu, Wenqi Shao, Junjun He. Yu Qiao, and Ping Luo. Omnimed- vqa: A new large-scale comprehensive evaluation benchmark for medical Ivlm. arXiv preprint arXiv:2402.00 0810009911111111 2024b.
Xiaoke Huang, Juncheng Wu, Hui Liu, Xianfeng Tang, and Yuyin Zhou. Medvithinker: Simple basclines for multimodal medical reasoning. arXiv preprint arXiv:2508.0200 2025.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camey,, et al. Openai ol system card. arXiv preprins arXiv:2412.16720, 2024.
Yubin Kim, Chanwoo Park, Hyewon Jeong, Yik Siu Chan, Xuhai Xu, Daniel McDuff, Hyeonhoon Lee, Marzyeh Ghassemi, Cynthia Breazeal, Hae Park, et al. Mdagents: An adaptive collaboration of Ilms for medical decision decision-making. making. Advances Neural Information Processing Systems, 37:
79410-79452, 2024.
Abhishek Kumar, Victor Zhuang, Rishabh Agarwal, Ying Su, John D Co-Reyes, Aravind Singh, Kshitij Baumli, Shariq Iqbal, Christopher Bishop. Rebecca Roelofs, et al. Training language models to self-correct via reinforcement learning. arXiv preprint 24X.::2409.12911 2024.
11

---

Published as a conference paper at ICLR 2026
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang. Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang. and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushmen A dataset of clinically generated visual questions and answers about radiology images. Scientific data, 5(1):1-10, 2018.
Yunwen Lei, Ting Hu, Guiying Li, and Ke Tang. Stochastic gradient descent for nonconvex learning without bounded gradient assumptions. IEEE transactions neural networks and learning systems, 31 31110)443444400 2019.
Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, al. Mmedagent: Learning to use medical tools with multi mollil agent. In Findings of the Association for Compulational Linguistics: EMNLP 2024, pp. 8745-8760, 2024a.
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Nau- mann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and vis-io assistant for biomedicine in one day. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36, 2024b.
Junkai Li, Yunghwei Lai, Weitao Li, Jingyi Ren, Meng Zhang, Xinhui Kang, Siyu Wang, Peng Li, Ya-Qin Zhang, Weizhi Ma, al. Agent hospital: A simulacrum of hospital with evolvable medical agents. arXiv preprint arXiv:2405. 200.0:270 2024c.
Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma. Yan Yang. and Xiao-Ming Wu. Slake: A semantically- labeled nnowledge-enhaneed enhanced dataset for medical visual question answering. In 2021 IEEE 18th International Symposium Biomedical Imaging (ISBI). pp. 1650-1654. IEEE, 2021.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16296-26306 2024a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024b.
Jiaqi Liu, Kaiwen Xiong, Peng Xia, Yiyang Zhou, Haonian Ji, Lu Feng, Siwei Han, Mingyu Ding, and Huaxiu Yao. AgentO-vl: Exploring self-evolving agent for tool-integrated vision-language reasoning. arXiv preprint 29Xiv:2511.19900, 2025a.
Junling Liu, Ziming Wang, Qichen Ye, Dading Chong, Peilin Zhou, and Yining Hua. Qilin- med-vl: Towards chinese large vision-language model for general healthcare. arXiv preprint arXiv:2310. 179566 2023.
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.0/78. 2025b.
Fanqing Meng, Lingxiao Du Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang. Junjun He, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning. arXiv preprint vxXiv:2503.03365 2025.
Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga Yash Dalmia, Jure Leskovec, Cyril Zakka, Eduardo Pontes Reis, and Pranav Rajpurkar. Med flaminnoo a multimodal medical few-shot learner. In Machine Learning for Health (MI4H), pp. 353-367. PMLR, 2023.
Vishwesh Nath, Wenqi Li, Dong Yang, Andriy Myronenko, Mingxin Zheng, Yao Lu, Zhijian Liu, Hongxu Yin, Yucheng Tang, Pengfei Guo, et al. Vila-m3: Enhancing vision- languaea models with medical expert knowledge. arXiv preprint arXiv:2411. 12915, 2024.
12

---

Published as a conference paper at ICLR 2026
OpenAL. Gpt-4o system card, 2024.
bttp:://gpenai.comm/nndee
gpt 110--40--ystenne system-card//////
OpenAl. Introducing o3 and 04-mini, 2025.
https://gpenai.comm/
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, and Alex Ray. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730-27744, 2022.
Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. Medvlm-rl: Incentivizing medical reasoning capability of vision- language models (vlms) via reinforcement learning, arXiv preprint arXiv:2502. 2025.
Anastasia Pentina, Viktoriaa Sharmanska, and Christoph Lampert. Curriculum learning of multiple tasks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. $492-5500, 2015.
Samuel Schmidgall, Rojin Ziaei, Carl Harris, Eduardo Reis, Jeffrey Jopling, and Michael Moor. Agentclinic: a multimodal agent benchmark to evaluate ai in simulated clinical environments. arXiv preprint arXiv:2405.07966 2024.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. ArXiv preprint, abs/1707.063477 2017. URL
abs Noo//107.0633 06347)
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, al. Vlm-rl: A stable and generalizable rl-style large vision-language model. arXiv preprint arXiv:2504.0700
2025.
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling IIm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.033114 2024.
Zhaochen Su, Peng Xia, Hangyu Guo, Zhenhua Liu, Yan Ma, Xiaoye Qu, Jiaqi Liu, Yanshu Li, Kaide Zeng, Zhengyuan Yang, et al. Thinking with images for multimodal reasoning: Foundations, methods, and future frontiers. arXiv preprint arXiv:2506.2391 2025.
Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, and Mark Gerstein. Medagents: Large language models as collaborators for zero-shot medical reasoning. Findings of the Association for Computational Linguistics ACI 2024, pp. 599-621, 2024.
Xiangru Tang, Daniel Shao, Jiwoong Sohn, Jiapeng Chen, Jiayi Zhang, Jinyu Xiang, Fang Wu, Yilun Zhao, Chenglin Wu, Wenqi Shi, et al. Medagentshench: Benchmarking thinking models and agent frameworks for complex medical reasoning. arXiv preprint arXiv:2503.07453 2025.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with Ilss. arXiv preprint arXiv:2501. 2025.
Martin J Wainwright. High-dimensional statistics: non-asymptotic viewpoint, volume 48. Cam- bridge university press, 2019.
Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vi-rethinker: Incentivizing self-reflection of rision-language models with reinforcement learning. arXiv preprint N08V::2504.088877, 2025a.
Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng. Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang. Zhenru Zhang, et al. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for Ilm reasoning. arXiv preprint arXiv:2506.0/ 0099999660999,.... 2025b.
13

---

Published as a conference paper at ICLR 2026
Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoe V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang. and Weidi Xie. Towards generalist foundation model for radiology. arXiv preprint arXiv:2308. 02463, 2023.
Fang Wu, Xu Huang, Weihao Xuan, Zhiwei Zhang, Yijia Xiao, Guancheng Wan, Xiaomin Li, Bing Hu, Peng Xia, Jure Leskovec, et al. Multiplayer nash preference optimization. arXiv preprint arXiv:2509.23311 2025.
Peng Xia, Ze Chen, Juanxi Tian, Yangrui Gong, Ruibo Hou, Yue Xu, Zhenbang Wu, Zhiyuan Fan, Yiyang Zhou, Kangyu Zhu, et al. Cares: A comprchensive benchmark of trustworthiness in medical vision language models. Advances in Neural Information Processing Systems, 37:1403344 140365, 2024a.
Peng Xia, Kangyu Zhu, Haoran Li, Tianze Wang, Weijia Shi, Sheng Wang, Linjun Zhang, James Zou, and Huaxiu Yao. Mmed-rag: Versatile multimodal rag system for medical vision language models. arXiv preprint arXhh:2410.13088 2024b.
Peng Xia, Kangyu Zhu, Haoran Li, Hongtu Zhu, Yun Li, Gang Li, Linjun Zhang, and Huaxiu Yao. Rule: Reliable multimodal rag for factuality in medical vision language models. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, pp. 1081-1093, 2024c.
Peng Xia, Kaide Zeng, Jiaqi Liu, Can Qin, Fang Wu, Yiyang Zhou, Caiming Xiong, and Huaxiu Yao. AgentO: Unleashing self-evolving agents from zero data via tool-integrated reasoning. arXiv preprint arXiv:2511.1600 2025.
Edward Yeo. Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in Ilms. arXiv preprint arXiv:2502.03337; 2025.
Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Guoyin Wang, Heng
Li, Jiangcheng Zhu, Jianqun Chen, et al. Yi: Open foundation models by 01. ai. arXiv preprint 24X.::2403.04652, 2024.
Xiang Yue. Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal under- standing and reasoning benehmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xiong- Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, et al. Aflow: Automating agentic workflow generation. In The Thirteenth International Conference on Learning Representattins, 2024.
Xiaotian Zhang, Yuan Wang, Zhaopeng Feng, Ruizhe Chen, Zhijie Zhou, Yan Zhang, Hongxia Xu, Jian Wu, and Zuozhu Liu. Med-ul: Incentivizing unified medical reasoning in Ilms via large-scale reinforcement learning. arXiv preprint arXiv:2506. 12307, 2025a.
Yifan Zhang, Liang Hu, Haofeng Sun, Peiyu Wang, Yichen Wei, Shukang Yin, Jiangbo Pei, Wei Shen, Peng Xia, Yi Peng, et al. Skywork-r v4: Toward agentic multimodal intelligence through interleaved thinking with images and deepresearch. arXiv preprint arXiv:2512.02395, 2025b.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing vision-language understanding with advanced large language models. arXiv preprint vrXiv:2344.10522 2023.
Kangyu Zhu, Peng Xia, Yun Li, Hongtu Zhu, Sheng Wang, and Huaxiu Yao. Mmedpo: Aligning medical laniunel models with clinical-aware multimodal preference optimization. arXiv preprint drXiv:2412.06141,, 2024.
Daniel Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:I909.000 2019.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
Large Language Model Usage
16
Omitted Theorems and Proofs
16
Notation
16
2
Some Useful Lemmas
16
B.3
Analysis for Curriculum Learning
16
Analysis for Standard SGD
21
Evaluated Models
22
Evaluated Datasets
22
Overview of the Baselines
23
Experimental Setup
23
Data Statistics
23
F2
Hyperparameter Settings
24
Prompi].
24
F4
Dataset Splitting and Difficulty Stratification
25
Additional Results
25
G.1
Comparison with Supervised Fine-Tune Tuning (SFT)
................................
25
G.2
Different Specialists
................................
25
Ablation Analysis
26
................................
G.3.1
Performance Triage Doctor
................................
26
G.3.2
KL Divergence Coefficient
26
G.4
Comparison with Test-Time Scaling Methods
26
G.5
Quantify the Performance Gain from Routing and Aggregation
27
G.6
Detailed Ablation on Triage Agent
27
G.7
Detailed Ablation on Progressively Adding Components
28
G.8
Comparison on Triage Agent with Different Settings
29
G.9
Comparison on GP Update Strategies
29
10
Framework Transferability
29
G.
Baselines with Trained Aggregator
30
Detailed Results
30
1
More Cases
31
15

---

Published as a conference paper at ICLR 2026
A LARGE LANGUAGE MODEL USAGE
All content in this article is entirely authored by the writers. The LLM (we use Gemini2.5-Pry was used solely for language refinement and stylistic polishing, without contributing to content generation. All LLM-refined passages were subsequently reviewed and revised by the authors.
B OMITTED THEOREMS AND PROOFS
B.1 NOTATION
We denote by (F) : Bal supjer 1 complexity of the function class F C [ X + IJ,, where 01..... are i.idd Rademacher variables taking values in {-1,+1} with probability 1/2 each and the outer expectation over the sample S. The Kullback-Leine
the (expected) Rademacher
(KL) divergence from a discrete distribution to discrete p(m)
distribution (defined over a common support 2') is given by DKL ip q = ExEx p(xx) log
((x)
under the assumption that whenever p(z) > O, one also has ((x) sequences {un} and [bn]. write On = o(b,.) if lim,, bb, 0, On = O(b,,) if an S Cb.. and a, = (2(b,) On 2 Cbn for all n and some positive C. The l2 norm of a vector x E Rdd 1/2
for all X. For two positive
defined ||0||| = (E4 ( ==?)
For measurable function f X - R the Loo norm is Il supzex [f(x), which equals suprex [f(x) when X' is finite.
B.2 SOME USEFUL LEMMAS
Lemma B.1 (Theorem 4.10 in Wainwright 2019 For any b-uniformly bounded class of functions
F. any positive integer 2 1, and any scalar ò O, we have
n222
P
sup FEF
(P,, P)]] < 2R (F) + (F) 2 1 - exp
2b2
A function class F is said to be b-uniformly bounded if 1l. Sbyf
To prove Theorem 4.1 we split the analysis into two parts: Theorem B.1 which establishes the upper bound for curriculum learning, and Theorem B.2 which establishes the lower bound for standard SGD. Combining these results yields Theorem
B.3 ANALYSIS FOR CURRICULUM LEARNING
In this section, we systematically develop the theoretical foundation for curriculum learning. As previously discussed, we reiterate here that the curriculum learning strategy effective for policy learning in the reinforcement learning (RL) setting. Our goal is to quantify the number of samples required to achieve target error level e.
Recall the distribution assumption: Assumption B.1. From the easiest to the hardest dataset, for j-th dataset, we assume
(o), w:) x4 p(x), Ii noz x))
Dj
Recall that the GRPO loss is given by
]
JoRro(0) (x,
t M
min (riAi,
+ c)A, - DxL (Tol mer)
To(ys x)
mold (36 xx)
(B.1)
And our loss function is defined as
Je-AAK (0) = E [auro(() + +o He (To H: Σ Pt.j logPt.. Pc.
(B.2)
where
Pe = no(- Ree, Softmax
2,
16

---

Published as a conference paper at ICLR 2026
When minimizing the objective, interpret the procedure performing SGD the batch loss. Specifically, define
0)) = - -ewp(()) s-nuy (ull)) JGRPO (0) + H2 (TTTom))))
It is natural to introduce the following self-consistency condition:
Assumption B.2 (Self-consistency). f is the minimizer of Lj (0).
Under this assumption, minimizing (0) allows us to recover the true policy. For the empirical version, the dataset at stage j is
Dj (1 x4 ~ p(a)), yi moz(U m
and the overall dataset is
D = D
j=1
The corresponding empirical approximation of the batch loss is
())
1 ni (x.v)ED,
Σ
[JeRPO()) + +o Ho He( (ogna))))
We denote its minimizer by 0y.
The overall training procedure then consists of successively minimizing (,(0) for = 1, l.....JJ where at each stage j perform Kj iterations of SGD).
We first apply classical learning theory techniques establish the convergence between (, and (f). After that, we analyze the behavior of the SGD iterations.
To proceed, we introduce two assumptions on the loss function class
F = (0) (, j= 1j=00..... I.....J]
Assumption B.3 (Boundedness). For any loss function f E FF llfll B Assumption B.4. For any loss function F, let 07 denote its minimizer If f(0) f(07) s Un. then
L 1 07 1 < (0) 1(07)-
Remark B. Here assume that, in neighborhood of its minimizer, the objective is locally convex. Our GRPO loss satisfics this assumption.
Proposition B.1. Suppose Assumptions B. B.4 hold. the j-th step, for any < 61 < U1, assume the sample size nj is sufficiently large that
Ru
4
where =I= (mo (r) E θ denotes the distribution space induced by the parameter space θ. Define the event
-110 -0:10-0;1110
E1
L
Then,
2(0ff)221exx 11 exp
n
2B
In other words, with high probability, the empirical minimizer 0, lies close to the population minimizer 07 in the parameter space, and the estimation error decreases as the sample size ny increases.
Remark B.2. Usually, for regular parameter space, Rug (TT) is o(1).
17

---

Published as a conference paper at ICLR 2026
Proof. We begin by decomposing the excess risk:
(,(0,) (,(0)) 2, 2,(,)) -1) -,()) C2
Since 8 minimizes (0), the middle term is non-positive. Hence,
- - 2,(03) s sup 2,(0) - L/
Applying Lemma B.1 we obtain the high-probability bound
2 1 - exp (- !ntt -nnzf] 2B3
((c,(0,)- - c5(0)) - 2Rn, (II) + E1
2
If m2 is chosen sufficiently large such that
2R,, (II S C
2
then with the same probability we have
-0,) s E1.
Moreover, if S U1 Assumption B.4 implies
L L1 0, 07 || c,(,,) (,(0)).
Combining the above, we obtain
100
0 -
5 = -P S I
2 exp
nyci 2
This establishes the claim.
To establish the iteration complexity stage j. impose standard conditions the loss function class F.
Assumption B.5 (Smoothness). Let L2 > O. For any loss function E F. the gradient of f is Lz-Lipschitz continuous. That is, for all 0,0 O,
V V
< L2
Assumption B.6 satisfies the PL inequality with parameter μ 0. Specifically, for all 0 E θ,
(PL) condition). For any loss function f E F, we assume that
f(0) f(07) < 1 [V
2p
where 07 = arg mingee f()) denotes the minimizer of f.
Based on Assumptions B.5 and
B.6
at.
2019
established the following result.
Lemma B.2. Suppose Assumptions B.5 and B.6 hold, and that V ((())) = O. If the step size satisfies
n/T.2, then
- s (1 ((0))
where f E F and 07 arg mingee (()).
Next proposition give the detail how many iteration do need in j step SGD
18

---

Published as a conference paper at ICLR 2026
Proposition B.2. Suppose Assumptions B.1 B.5 B.6 and B.4 hold. Fix stage j. Run SGD the empirical loss Lj with constant stepsize y /1/ 1.2 for Kj iterations, starting from 0
8 Let
Dj-1 11030 - -,,12
Define the event
2 = -1 {10,ss - -0112= 61 L1
Then for any <1 S U1, if
K₃ o 1 log
D ))
um
have
P(17f0)) 2 1 61.
Proof. Firstly, applying Lemma B.2 (with/ ʃ = Lij and 1/; = S 11/ we obtain for Kj 2 0.
E (0,,,,, - (0,, $ (1 kn) 2,
-
By Assumptions B.6 (PL) and B.5 (Lipschitz gradient), we have
T3 00,0 8,12 2p
--)0 2,((,) 2p (22(
Taking expectations over the randomness up stage (recall 0,0 - oj-1,K,- (K--1) get
,(c,(05k,) 2p
L
(1 um) K, 13 00,0 0, [1]]
L2 (1 un) Dj-1.
2μ
Choose K5 so that
T un) D S e
(1
2p
Using 1 um S. e- sufficient condition is
L Dj-1
K₁
log
pT
2/
By Markov's inequality, with probability at least - E1.
20 =
- (, 2,(0,) $E
occurs. On 12,4), Assumption B.4 yields
|0,k, - 0, S L -(0,,))2,(00) $
E1 L
which completes the proof.
Theorem B.I. Suppose Assumptions B. B.6 hold. Fix O Un. For each stage -....,? choose ny such that Ray (0) S 61/4, and run SGD with constant stepsize 1i S p// L.. sufficient
total number of iterations is
J-1
L3
100
01+1 11
K K=O
log
161
j=0
With this K, the final iterate satisfies n,e2 - J E1.
1 - 0512 < e with probability at least 1
Eje1 exp(
2B
19

---

Published as a conference paper at ICLR 2026
Proof. Recall the following events from Propositions B.1 and B.2
2 - { 03 83 S Ln
2/3 110,0s, - -8112=
9
L1
By Proposition B.I and the choice of njj
P(25)) 2 1 - exp
2B
By Proposition B.2 and the choice of K,.
IP (1)))) 2 1 e1.
Let
Ω = ò ( )
j=l
A union bound gives
J
P(L) 2 1
Σ
exp
n,,t 2
Jei.
j=1
Condition on Ω. From the end of stage j to the start of stage
+ 1 we have 02K, = 0,,+1,0 and thus
0 - 8j+1 1|/ - [0,k,
8 + |0j - 87 2
+ ||0] - 0jt1l|2 1100001 -8j+1 11a
EL + 100 01+1112
+ V la 3 a 1107 0f l12
L1
where we used 12.²) and Assumption B.4 to bound |0,K,, - 0.113 - S 61 [T. and 2 bound
<1/L1,
the two terms involving 0, and 0,41. Squaring both sides and using (a b)2 s 2a + 2h gives the displayed recursion
D3 10+100- - -ōzz1 s
18 L1
+ 21107 - 0,+1112
1100--0+1113
By Proposition B.2 choosing
K3 15=0 o 1 log
Lス 11310011-02113
pm
1c
ensures 22(²) holds and
|0,K, - 0, -
E1
L
Iterating this from 1 to J. we obtain at the final stage
100kK - 03112 S 1ocK. 0, 00112 +
1030 0j|12
S / V + V +
and hence
2 4el |0,,k 1. 03 0712 <
L
Finally, the total number of iterations is K = E/= Kj. and the total sample size is n E/=1 Tzj.
11
20

---

Published as a conference paper at ICLR 2026
B.4 ANALYSIS FOR STANDARD SGD
Without loss of generality. assume 7i =... n.J Drg. where feg is the total sample size, and suppose SGD directly the dataset D. Under this equal-allocation setting, the pooled data are drawn from the uniform mixture
Toz (U
moz (v x))
Define the mixed (population) and empirical losses by
(g) E Ex=p((x) yru (vla) JGRPO
Y Ho (T
(0)
Σ
JGRPO (0) To Jallel (T
Yrg (x.
Let 0% and fig denote the minimizers of Lig and Eig. respectively.
8 110 12
Theorem B.2. If Big satisfies Ryz (T)) S 1 and SGD uses the same stepsize 7,, take
L 10 0ggle
Kig =
log
p177
ME
Then, with probability at least exp
negci 211
we have
L18²
110r 8- 1
2
whenever
c S
16
Proof. Apply Proposition B.1 to Lugi with probability at least 1 exp (opp--(ngg55) (nn(=((neg))(2221))
|10g 0- L1
Next apply Proposition
B.2 to the standard (pooled) SGD (same y S m/L and Kig
O log
L3 |006 -0, with probability at least 61. E1.
10 - 0ra113 S
E1 Li
By union bound, both events hold simultaneously with probability at least
exp(
D
25
C1.
On this event, the triangle inequality yields
|10gg 0igllz ll2 S 110rg
Orgll2 1101g - 12 S
61 2\1
L
hence
4c1
110gg
0ig112 1
L
Finally,
4e1
101g 00112 2 100 00112 ||2 100 - 02el2 2 8
L
and el L108 then /41//1 s 6 6/2, which gives
10 H* 1|2 2 8 2
Remark B.3. Notice that when we take e < L187 /66 we will have
8
J0 05113 < 4e
10 0" 0*1
2
This directly yields Theorem 4.1
21

---

Published as a conference paper at ICLR 2026
C EVALUATED MODELS
We evaluate a series of state-of-the-arr LVLMs and Multi-agent. The single-agent models include
LLaVA (Liu al. 2024a), Yi-VL-34B (Young et al. ) Qwen-VL (Bai et al. 2025), LLaVA-
Med (Li et al. 2023), MedFlamingo (Moor et al. 2023). RadFM (Wu et al. 2023). HuatuoGPT- Vision (Chen al. 2024a) and GPT-4o (OpenAl 2024). The multi-agent frameworks include prior collaborative systems such as MedAgents (Tang et al. (024), MDAgents (Kim et al.) (2024) and
AFlow (Zhang et al. 2024), as well as our proposed MMedAgent-RR RL framework that introduces reinforcement learning for adaptive multi-agent reasoning.
GPT-4o (OpenAl 2024) is OpenAl's latest multimodal large model that supports text, image, and
audio inputs. It exhibits strong generalization across vision-language benchmarks and serves both as single-agent baseline and as specialist in our multi-agent settings.
Med-Flamingo (Moor al.) 2023) multimodal few-shot learner designed for the medical domain. Built upon OpenFlamingo, it is further pre-trained on biomedical image-text data from sci- entific literature. enables few-shot medical visual question answering with minimal supervision.
RadFM (Wu et al. 2023) a domain-specific foundation model tailored for radiology. It leverages large-scale radiology reports and domain-adaptive learning to improve zero-shot and few-shot performance on radiographic image understanding.
VA-Med (Li et al. 2023) extends LLaVA to the biomedical domain by fine-tuning with medical image-inttruction pairs. It enhances medical reasoning and answer generation with limited supervision using domain-speciffe visual-textual alignments.
Qwen2.5-VI. (Bai et al. 2025) is a versatile vision-language model developed by Alibaba. supports high-quality OCR, multi-turn dialogue, and reasoning over complex multimodal inputs. It is used both as a strong single-agent baseline and as the foundation of agents in our proposed framework.
Yi-VL-34B (Young et al. 2024) is a large-scale multimodal model from OLA With 34 billion parameters, offers high-capacity visual understanding and serves as a powerful open-source baseline across medical and general VQA tasks.
LLaVA (Liu et al. 2024b a) are general-punee purpose vision-language models trained via visual instruc- tion tuning. Evaluated in several sizes (7B, 13B, 34B), they serve as strong single-agent baselines in both in-domain and out-of-domain medical benchmarks.
(Chen et al.) 2024a) is a medical multimodal large language model (MLLM) trained on the curated PubMed Vision dataset. This dataset was created by using GPT-4V to denoise and reformat 1.3 million image-text pairs from PubMed, significantly improving data quality. As result, CaaauuoGPTViiiion demonstrates superior performance medical multimodal benchmarks compared to other open-source models.
TuattoGPT-Visson..
D EVALUATED DATASETS
We employ three established medical vision-language datasets: VQA-RAD (Lau al.) 2018). SLAKE (Liu al. 2021), and Path VQA (He et al. 2020). Furthermore, to evaluate of performance, we incorporate the health and medicine subset of MMMU Yue et al. 2024) along with OmniMedVQA (Hu et al. 2024b).
VQA-RAD (au et al. 2018 is a manually constructed dataset containing 315 radiology images with 3,515 question-answer pairs. The images are distributed across head, chest, and abdomen regions, and include both open-ended and binary "yes/no" questions. Each image is associated with multiple clinically relevant questions generated by medical professionals. The dataset aims to facilitate the development of visual question answering systems for the medical domain.
knowledge-enhanced dataset featuring 642 radiology images and over 14,000 question-answer pairs. It offers comprehensive annotations including masks for semantic segmentation and bounding boxes for object detection. SLAKE is bilingual (English and Chinese) and covers 12 diseases and 39 organs across various body parts. The dataset also incorporates medical knowledge graph with 5,232 medical knowledge triplets to support knowledge-basse reasoning.
SLAKE (Liu et al. 202I> a semantically-labee
22

---

Published as a conference paper at ICLR 2026
PathVQA (He et al., 2020) is certology-focussee dataset containing 32,799 open-ended ques- tions from 4,998 pathology images. The dataset was created using a semi-automane pipeline to extract images and captions from pathology textbooks and generate question-anse -answrr pairs using natural language processing. Path VQA aims to support the development of Al systems capable of answering clinical questions about pathology images, with each question manually checked for correctness.
MMMU (Yue et al. 2024) (Health & Medicine subset) is part of the Massive Multi-discipline Multimodal Understanding benchmark. This subset contains approximately 1,752 test questions across five disciplines: Basic Medical Science, Clinical Medicine, Diagnostics and Laboratory Medicine, Pharmacy, and Public Health. The questions require college-level subject knowledge and deliberate reasoning, challenging models to perform expert-level perception and reasoning tasks.
OmniMedVQA (Hu et al. 2024b) is comprehensive medical VQA benchmark collected from 73 different medical datasets, featuring images across 12 different modalities and covering more than 20 distinct anatomical regions. All images are sourced from authentic medical scenarios, ensuring alignment with real- aol-dorld applications. The benchmark provides a diverse evaluation platform for testing the capabilities of large vision-language models in medical image understanding and reasoning.
E OVERVIEW OF THE BASELINES
We evaluate MMedAgent-RL against two main multi-agent baselines, MedAgents (Tang et al. 2024). MDAgents (Kim al. 2024) and AFlow (Zhang al., 2024). These baselines represent
lfateoof-the-at approaches in medical visual question answering.
MedA gents (Tang et al., 2024) establishes zero-shot multi-agent collaboration framework that simulates real-world clinical workflows. The framework encompasses five critical steps: gathering domain experts, proposing individual analyses, summarizing analyses into report, iterating over discussions until consensus is reached, and making a final decision. Different agents are assigned specific medical roles and collaborate solve complex medical reasoning tasks. The framework relies on pre- are-trained large language models without additional fine-tuning, enabling natural Hialogue-based interactions between agents. MedAgents demonstrates how specialized medical knowledge from different domains can be integrated through structured agent collaboration, providing a strong baseline for multi-agent medical reasoning.
MDAgents (Kim al. 2024) advances multi-agent medical systems by introducing adaptive collaboration mechanisms. Unlike fixed collaboration patterns, MDAgents dynamically selects the most appropriate agent configuration and communication structure based on the specific medical task. This framework allows for more flexible interactions between general practitioners and specialist agents, optimizing the collaboration pattern for different types of medical queries. MDAgents incorporates mechanisms to resolve conflicts between different agent opinions and adapts the consultation workflow to match the complexity of the medical case, resulting more robust decision-making across diverse medical scenarios.
Fllow Zhang et al. 2024) is framework designed to automatically generate and optimize complex problem-solving workflows for LLMs. Instead of relying on single inference pass, these workflows enhance performance through structured procedures. We evaluate three distinct strategies as strong baselines: SelfiConsist Ensemble, which runs an agent multiple times and selects the most frequent answer to improve reliability; Multi-Agent Debate, which uses several agents to collaboratively propose, critique, and refine solutions; and Self- Riff-Refne, which employs a feedback loop for single agent iteratively critique and improve its own output.
F EXPERIMENTAL SETUP
DATA STATISTICS
The data used in this work is shown in Table 3 and involves five multimodal medical datasets: VQA-RAD, SLAKE, Path VQA, OnniMedVQA and MMMU (Health & Medicine track). Among them, three used as in-domain datasets, with their training sets employed for model training. The
23

---

Published as a conference paper at ICLR 2026
Table 3: The results of the medical VQA benchmark. Here, MMMU denotes MMMU (Health Medicine track) and the number of training and testing phase denotes the number of QA items for each phase.
Model
All
VQA-RAD
SLAKE
PathVQA
OmniMedVQA
MMMU
Train
12,176
940
1,681
9,555
Fasy
8,321
498
1,284
6,539
Medium
1,409
160
1,135
-Hadd
2,626
281
275
2,070
/
Test
15,153
251
416
3,362
11.124
150
remaining two are directly used as out-of-domain (OOD) testing datasets. The specific data volume for each dataset used each stage of Curriculum-Based Multi-Agent Reinforcement Learning is detailed in Table 3
F.2 HYPERPARAMETER SETTINGS
We use Qwen2.5-Ve VL Bai et al. 2025) as the base model. We design the prompt template using the format employed in MM-EUREKS (Meng al. 2025), clearly specifying the required output structure, which includes using <thi nk> and <answer> tags to separately contain the reasoning process and the final answer, with the being separated. The detailed prompt shown in Table For training hyperparameters, the rollout batch size and training batch size are both set to 128, with rollouts generated for each sample. The sampling temperature is set to 1.0 encourage response diversity, and optimization is done with a learning rate of 1 x 10-6, Additionally, for the three stages of curriculum reinforcement learning, the KL divergence coefficients are set to 10-3 and 1 x 10 respectively to stabilize training. The dynamic entropy coefficient % is set to 0.03 hard (s 0), 0.005 for medium (0 < 1), and 0.0001 for easy (s = 1) samples, respectively, to adapt the level of exploration based on curriculum difficulty. For the number of specialists, we set e 3. For the baseline implementation, i.e., MedAgents (Tang et al., 2024). MDDAgents (Kim) Qwen2.5-VL as the agent for decision making to ensure a fair comparison between multi-agent baselines and MMedAgent-RR
10
itt al. 2024) and AFlow (Zhang et al. 2024),
For the training
framework, adopt multimodal RL framework based OpenRLHF (Hu al., (024a). For the inference, we adopt the vLLM framework (Kwon et al. 2023). All training is conducted on NVIDIA Tesla A A100 80GB GPUs.
F.3 PROMPT
The prompt for the fine-tuning of base model is shown in Table In this prompt, provide the question options, the input image, and i expert answers. In the experiment, set to 3. The model needs to first generate the reasoning process within the <think> tag, and then provide the final answer within the <answer> tag.
Table 4: Prompt template used for reinforcement learning fine-tuning.
Prompt Template:
As the General Practitioner coordinating this case, review the specialist expertise to make final decision. Answer from <Specialiste Provide your final assessment. You need to first think about the reasoning process in the mind and then provide the user with the answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </ </answer> tags, respectively, i.e., <think> reasoning process here /think><<nsserr answer here </answer>, The answer must be chosen from the given options.
:SpecialistAnsste
<Quest on>>
24

---

Published as a conference paper at ICLR 2026
Table 5: The comparison with SFT on several medical VQA benchmarks.
In-Domain Datasets PathYQA
Cou--f-Diesstuttion Datasets
VQA-RAD
Arg.
OmaiMeddVQ4
MMMUU-Med
Arg.
75.5
Med-Flamingo
30.7
RadFM
50.6
34.6
38.7
41.3
28.2
27.0
27.6
Med-7-
51.4
48.6
56.8
52.3
44.1
36.9
40.5
Owen- Vh-Chat
47.0
56.0
55.1
52.7
48.3
32.7
40.5
Yi-VL-34B
1LaVA-v1.66
$7.9
47.9
$2.8
49.0
33.1
41
11499-v16--198
35.8
58.9
$1.9
55.5
48.0
39.3
43.7
L1aVA-v1.6-348
58.6
67.3
59.1
61.6
58.7
48,8
53.8
59.4
$4.1
44.6
38.2
HuatuoGPT-V/a
63.0
77.2
58.7
66.1
74.5
51.0
62.8
S-em2.5-VL-3BB
61,0
62.7
57.6
60.4
60.1
54.5
57.3
2wos2.5-V1-78
M
63.7
67.8
65.2
65.6
624
$7.0
59.7
Collaboration
MeaAggett
67.9
49.7
MTDAgents
66.8
688.2
65.4
66.8
58.2
52.3
55.1
APlow
67.3
68.9
66.4
67.5
59.6
53.6
56.6
GPT-4o Owen2.5 Swenn..5VVL- 7B
62.5
63.9
53.2
59.9
56.4
50.7
53.6
reasoning
Qwen2.5 SLT reasoning
62.0
59.4
MMeeAAgent-BR RL
21.5
76.2
72.3
73.3
73.3
71.9
72.6
F.4 DATASET SPLITTING AND DIFFICULTY STRATIFICATION
The partition of training, validation, and test sets strictly follows the official dataset guidelines. The difficulty grading required exclusively for the training phase to implement curriculum learning. Specifically, we classify samples into difficulty levels based on the consistency and accuracy of the specialists' responses. We emphasize that the inference process and the calculation of evaluation scores do not use this difficulty grading at all. The model generates responses without access to any difficulty labels or ground truth. The difficulty stratification on the test set is applied solely for the analytical breakdown shown in Figure 4 It serves only to visualize and analyze model performance across different complexity levels post-generation, and has no influence on the inference process itself. The difficulty grading for the test dataset follows the
training data.
G ADDITIONAL RESULTS
G.1 COMPARISON WITH SUPERVISED FINE-TUNING TUNING (SFT)
We compare method with Supervised Fine-Tuning (SFT) methods baseline. As shown in Table our proposed method, MMedAgent-RI. demonstrates significant performance advantage over SFT methods across all the medical VQA benchmarks. The superiority of our approach is even more pronounced in the more challenging out-of-distributiin datasets. This highlights our model's enhanced robustness and generalization capabilities. Overall, MMedAgent-RI consistently new state-of-te oftte-of-the-art,, with our base model achieving 73.3% average on in-domain tasks and 72.6% on -iitt---iittiittion tasks, already surpassing the SFT method. The results clearly indicate that our multi-agent, reinforcement learning based approach is more effective than traditional SFT techniques for complex medical VQA tasks.
G.2 DIFFERENT SPECIALISTS
The effectiveness different specialist compositions within our framework is detailed in Table The results unequivocally show that multi-agent collaboration substantially outperforms single-agent baselines. Our premier configuration, 3 3*CpenAI-o3 (OpenAl 2025). achieved a top average score of 73.0, far exceeding the best-performing baseline, GPT40 (68.8). Crucially, this perfor- mance gain stems from the synergistic integration of multiple experts. The analysis also under- scores the importance of specialist diversity. Heterogeneous teams combining different models (e.g., 2 2pO0eenI- HuattuuG at 71.3) proved highly effective, demonstrating that fusing complementary knowledge enhances diagnostic robustness. This confirms that framework core strength lies its ability to dynamically orchestrate collaboration among a diverse team of high-quality specialists achieve superior decision-making
25

---

Published as a conference paper at ICLR 2026
Table 6: The comparison with different specialists.
Model
VQA-RAD
SLAKE
PathVQA
OmniMedVQA
MMMU
Avg.
GPT-4o
61.0
75.5
69.4
68.5
69.7
68.8
Qwen2.5-VL-7B
61.8
64.7
60.5
60.8
56.6
60.9
HuatuoGiPT-7B
63.0
77.2
58.7
74.6
51.0
64.9
MMeeAgent-RL
3*OpenAl-o3
71.5
76.2
72.3
71.9
73.04
3*GPT-40
70.4
75.2
72.7
69.1
67.1
70.9
1GPPT-4
63.2
65.9
62.8
63.4
59.0
62.86
IHHuutoOPT-VVsioo--BB
63.4
67.2
60.8
64.9
54.1
62.1
1"Qwen2.5-VL-7B
62.4
61.5
61.5
60.7
D=GPT-4o+1=Qwen225--10
70.0
68.5
75.0
71.7
68.0
62.0
69.0
69.9
75.8
73.0
70.0
68.0
71.3
2
71.1
75.4
71.8
69.1
69.2
71.3
68.9
73.8
70.6
66.1
68.5
69.0
74.7
68.3
70.6
HHoauuoiPT-Visionn7
65.8
78.2
61.3
73.8
50.1
65.8
G.3 ABLATION ANALYSIS
G.3.1 PERFORMANCE TRIAGE DOCTOR
The accuracy of the triage doctors is shown in Table We used the data with definitive department labels the evaluation target. From the results, we can observe that triage is not as challenging as answering complex medical diagnostic questions. Instead, department classification resembles modality classification process. The original model already achieved accuracy of over 80%, and after our fine-tuning, the model's performance has reached a human-level standard on these datasets.
Table 7: The performance of triage doctor.
Model
VQA-RAD
SLAKE
Vat
Qwen2.5-VL-3B
95.62
92.16
77.53
Owen2.5-VI.-77
96.21
94.41
80.58
MMedAggnt-RR..
99.98
99.94
99.06
G.3.2 KL DIVERGENCE COEFFICIENT
We conduct ablation experiments on the KL divergence coefficient at each stage, and the results are shown in 6 We observe that in the first stage, as the KL divergence coefficient increases, the model's performance tends to stabilize. This indicates that when training with simple data, where the specialist doctor's answers are entirely correct, i.e., the model merely needs learn to imitate, In this case, an additional KL divergence loss is required to constrain the policy model's 's update steps, preventing it from changing too drastically; otherwise, it would become model that simply copies the specialist's answers. In the second stage, the optimal KI. divergence coefficient is slightly larger than in the first stage, suggesting that the model needs some autonomy to explore its own more apparent in the third stage, where the optimal KL divergence
direction. This becomes coefficient is significantly higher. This reasonable because, when the specialist doctor's answers entirely incorrect, it becomes very difficult for the model to generate accurate response. If the KL divergence loss is too large in this stage, the model cannot explore effectively to find the correct answer. Therefore, in conclusion, different KL divergence coefficients need to be set for each stage of curriculum reinforcement learning to ensure optimal model performance.
G.4 COMPARISON WITH TEST-TIME SCALING METHODS
We conducted additional experiments using test-time scaling techniques, specifically Majority Voting and Self- Self-Consistency (Wang al.)), on both W and GPT-4o. For the experimental setup. Majority Voting involved sampling N = 3 outputs and selecting the most frequent answer as the final prediction. Self-Consistency sampled set of diverse reasoning paths rather than relying
26

---

Published as a conference paper at ICLR 2026
Easy
Medium
hard
Exy
Jounee Aecusey
66-
<1 secrpooecoeeaii Foce
Ru.eergere codf
tel
Figure 6: Ablation of KL divergence coefficient.
the greedy path, subsequently selecting the most consistent answer by marginalizing the sampled paths. As shown in Table test-time scaling improved the performance of Qwen2.5-VLL-7B by 2.6% and GPT-4o by 1.6% across the five datasets. However, despite these improvements, performance gap remains compared to our multi-agent framework. Our method outperforms these enhanced basclines by 18%, demonstrating the necessity and effectiveness of our proposed triage and multi-expert pipeline.
Table 8: Performance comparison with test-time scaling baselines.
Method
VQA-RAD
SLAKE
PathVQA
OmniMedVQA
MMMti-Med
Overall
Qwen2.5-VL-77
61.8
60.5
60.8
60.9
Qwen2.5-V1. Majority Voting
63.7
61.5
62.5
62.0
Qwen2.5-VL-7B Self-consistency
63.8
65.3
62.4
64.5
59.1
63.0
GPT-4o
61.0
75.5
69.4
68.5
69.7
68.8
GPT-40 Majority Voting
62.2
75.9
70.3
69.4
70.3
69.6
GPT-4o Self-consistency
63.6
75.4
70.8
69.1
71.5
70.1
MMedAgent-RI.
71.5
76.2
72.3
73.3
71.9
73.0
G.5 QUANTIFY THE PERFORMANCE GAIN FROM ROUTING AND AGGREGATION
To strictly quantify the model's s ability to correct mistakes rather than simply route to a specialist, we conducted a targeted evaluation on hard samples where all three specialists provided incorrect answers. We selected 200 such samples from Path VQA and 200 from OmniMedVQA to evaluate the performance of the Base Model (with and without expert knowledge), the SFT baseline, and our MMedAgent-RL.. Since every specialist is incorrect, any routing mechanism would yield 0% accuracy. Therefore, the performance this subset is strietly attributable the model's 's ability to perform intrinsic correction. As shown Table our method achieves substantial improvements. For instance, on the OOD OmniMedVOA hard subset, MMedAgent-RL. improves accuracy to 23.0%. We attribute this capability to the following logic: The extremely low accuracy of the Base Model w/o expert knowledge (4.5% on PathVQA and 2.0% on OmniMedVQA) confirms that the model lacks the intrinsic parametric knowledge to solve these hard cases independently. Although the specialists' final answers were wrong, their reasoning processes likely contained partial truths or valid medical context. While the SFT baseline struggles to utilize this conflicting information (often hallucinating along with the experts), AMMeAgeet-RR RL has learned via RL to critically synthesize these valid reasoning fragments, correcting the final conclusion rather than simply aggregating the errors.
G.6 DETAILED ABLATION TRIAGE AGENT
In Table "w/o Triage" refers to using the original base model to perform the routing (triage) task directly, without specific fine-tuning for this role. Following your suggestions, we have expanded our comparison in Table 10 We introduced two new settings test the necessity of the framework: 1) Random Routing: Assigning queries to specialists randomly to isolate the benefit of the specialists themselves. 2) Single Model w/o Routing: Using the Qwen2.5-VL-7B base model directly, and enhancing with Majority Voting (diverse sampling) as requested.
As shown in Table TO incorporating routing mechanism leads to significant performance gains. No-
27

---

Published as a conference paper at ICLR 2026
Table 9: Correctness ratio (accuracy on hard samples where all specialists failed).
Setting / Model
PathVQA
OmniMedVQA
w/o expert knowledge
Base Model
4.5%
2.0%
w/ expert knowledge
Base Model
5.5%
3.5%
SFT MMedAgent-RL.
10.0%
7.5%
26.5%
23.0%
tably, even Random Routing generally outperforms the single model equipped with Majority Voting. Furthermore, our proposed MMedAgent- significantly outperforms all baselines, confirming that dedicated triage-and-expert pipeline provides advantages that cannot be achieved by simple diverse sampling or random assignment.
Comparison with SFT. The original "w/o Triage" ablation was insufficient to justify the specific choice of RL over simpler methods. To address this, we conducted a detailed comparison between our GRPO-optimized Triage Agent and standard Supervised Fini-Tuning (SFT) baselines. As shown in Table [1] we evaluated two SFT configurations: 1) SFT (Standard): Fine-tuned on direct Question-Departmin
pairs. 2) SFT (w/ Reasoning): Fine-tuned using reasoning traces distilled from Qwen2.5 2-.nn.55-V (iven2.5-VL-321 VL-32B to simulate more capable classifier. While SFT significantly improves performance over the base model, GRPO still outperforms the best SFT baseline across datasets. Crucially, this advantage is most pronounced on MMMU-Med, GRPO outperforms "SFT w/ Reasoning" by +4. % (71.9 vs. 67.8), and on Om- niMedV QA by +2.5% (73.3 70.8). This indicates that while SFT can achieve high accuracy on standard distributions, the reinforcement learning process (GRPO) enables the Triage Agent to generalize better complex, unseen scenarios by learning from the reward signal of the downstream reasoning success, rather than just mimicking static label.
Out-of-Distribuut
(OOD) datasets. For instance,
Table 10: Performance comparison with or without routing.
Method
VQA-RAD
SLAKE
PathVQA
OmniMedVQQ
MMMU-Med
Overall
w/ triage
MMeeAgent--R (acro-shot)
69.9
MMedAgent (random triage)
63.4
64.7
67.8
58.2
64.2
MMedAgent--R RL. (fine-tuned triage)
71.5
76.2
72.3
73.3
71.9
73.0
w/o triage
Dwen2.5-VL-7B
61.8
Majority Voting
63.7
65.4
61.5
62.5
57.0
62.0
Table 11: Performance comparison of Triage Agent training strategies.
Method
VQA-RAD
SLAKE
PathVQA
OmniMedVOA
MMMU-Med
Base model
66.3
69.9
67.2
66.2
59.3
SFT w/o reasoning process
69.4
75.0
70.5
70.1
66.7
SFT reasoning process
70.2
75.9
71.0
70.8
67.8
+ GRPO
71.5
76.2
72.3
73.3
71.9
G.7 DETAILED ABLATION ON PROGRESSIVELY ADDING COMPONENTS
To quantify the contribution each component, we have conducted a progressive evaluation as shown Regarding the order of ablation, we formulated the progression as Base - Multi-expert + Triage. From an architectural perspective, the Triage module depends on the existence a candidate pool of experts to perform routing. Thus, first introduce the Specialists (Multi-expert) to build the capability pool, and subsequently add the Triage module to manage and utilize these experts efficiently. Starting with the Qwen2.5-VI.C baseline, observed the following trends:
in Table 12
28

---

Published as a conference paper at ICLR 2026
Majority Voting: Provides marginal improvement, indicating that simple test-time scaling has limits.
Specialists: Integrating domain-specific experts (with base model as the router) yields further gains, surpassing the single model with voting.
Introducing the learned Triage module significantly improves the effective utiliza- tion of specialists.
Triage:
Curriculum RL: Finally, applying our Curriculum RL strategy provides the most substantial performance leap, demonstrating that optimizing the collaboration between the triage and specialist agents is critical for complex medical reasoning.
Table 12: Performance progressively adding components.
Method
VQA-RAD
SLAKE
PathVQA
OmniMedVQA
MMMU-Med
Qwen2.5-VL- 5L (Base)
61.8
64.7
60.5
60.8
56.6
+ Specialists (Base Model)
64.5
66.9
63.2
63.4
60.7
Triage
65.7
68.4
64.4
64.8
62.6
+ Curriculum
71.5
76.2
72.3
73.3
71.9
G.8 COMPARISON ON TRIAGE AGENT WITH DIFFERENT SETTINGS
As shown in Table 13 we quantitatively evaluated the impact of different training stages on the triage agent's 's performance. We compared the Base Model, SFT (trained on direct inesttionanssee pairs without reasoning), SFT with Reasoning (trained on reasoning traces distilled from Qwen2.5-VLL 32B), and final GRPO-optimized model. The results demonstrate that the triage agent trained with GRPO yields the highest performance. Notably, this improvement most significant on the two Duitt--Distribution (OOD) datasets, i.e., OmniMedVQA and MMMU-Med, confirming that the reasoning capabilities reinforced by GRPO are crucial for generalization.
Table 13: Performance with triage agent with different settings.
Method
VQA-RAD
SLAKE
PathVQA
OmniMedVQA
MMMU-Med
Overall
Basc model
66.3
69.9
67.2
66.2
59.3
65.8
SFT w/o reasoning process
69.4
75.0
70.5
70.1
66.7
70.3
SFT w/ reasoning process
70.2
75.9
71.0
70.8
67.8
71.1
GRPO
71.5
76.2
72.3
73.3
71.9
73.0
G.9 COMPARISON ON GP UPDATE STRATEGIES
In the paper, the two GP agents (the Triage Agent and the Attending Physician) are updated inde- pendently. This design choice was primarily made ensure training stability and decouple the training process from potential API failures or latency when querying the external Open specialists. Specifically,, our procedure is as follows: 1) Triage Optimization: We first optimize the Triage Agent using image-modaan modality QA pairs to ensure accurate department routing. 2) Data Preparation: We classify the training data based on these departments and invoke the OpenAI API (acting as specialists) to generate expert knowledge offline. 3) Attending GP Training: Finally, we use these pre-generated expert trajectories to train the Attending GP (Qwen2.5-VL. via Reinforcement Learning.
also implemented an end-to-end online framework where both GPs are updated simultaneously. As shown in Table [4] the performance difference between the two settings is negligible. This con- firms that our decoupled training strategy is valid and yields results consistent with fully end-to-end approach while remaining more computationally efficient and stable.
G. FRAMEWORK TRANSFERABILIT
To demonstrate the transferability our framework, we conducted additional experiments using Intern VI.2.5- (Chen et al. 2024b) alternative base model. As shown in Table our
29

---

Published as a conference paper at ICLR 2026
Table 14: Performance comparison of GP update strategies.
Strategy
VQA-RAD
SLAKE
PathVQA
OmniMedVQA
MMMU-Med
Overall
Independent
71.5
76.2
72.3
73.3
71.9
73.0
Simultancous 71.3
76.5
72.4
73.6
71.6
73.1
method yields consistent and significant improvements across all datasets, regardless of the backbone architecture. OmniMedVQAA QA,, Intern VL.bbaed agent achieved remarkable score 82.4%, surpassing the performance of the Qwen-based version. Even datasets where the base Intern VL model struggled (c.g., PathVQA, where the base score was only 42.3%), our framework provided massive performance boost of +26. 1% (reaching 68.4%). These results confirm that our pipeline is model-agnostic and can effectively enhance the reasoning capabilities of diverse multimodal LLMs.
Table 15: Assessment of framework transferability.
Model
VQA-RAD
SLAKE
PathVQA
OmniMedVQA
MMMU-Med
Qwen2.5-VV V---BB
61.8
64.7
60.5
60.8
56.6
MMedAgent-- (Qwen2.5-VL.77B)
71.5
76.2
72.3
73.3
71.9
IntemV1.2.588B
58.6
68.6
42.3
76.5
51.4
MMMeAgent-RLL (InternVL2.5-88)
70.2
78.9
68.4
82.4
64.7
G.11 BASELINES WITH TRAINED AGGREGATOR
We implemented two new baselines where GPT-4o first samples N = 3 diverse outputs per query (T 1.0), and a Qwen2.5- VL-7B model then trained on the training set (same data used) to act as an aggregator that selects synthesizes the final answer from these candidates. We developed both an SFT Aggregator (via Supervised Fine-Tuning) and a GRPO Aggregator (via Group Relative Policy Optimization) to ensure robust comparison. As shown in Table 16] MMedAgent-RR RL still maintains a significant performance lead. We observed that directly training an aggregator yields limited gains, particularly when facing inconsistent candidate answers, i.e., challenge that directly motivated our proposed curriculum learning guided RL strategy. This confirms that our method's effectiveness stems from the process-level collaboration of specialized agents, which captures domain- specific nuances that cannot be replicated by simply aggregating generalist outputs.
searning-guided
Table 16: Performance comparison with GPT-4o+an aggregator (based on Qwen2.5-VL 7B) baselines.
Model
VQA-RAD
SLAKE
PathVQA
OmniMedVQA
MMMU-Med
Overall
Owen2.5 VL-7B (Base) GPT-4o
61.8
64.7
60.5
60.8
56.6
60.9
69.4
GPT-4o=Qwen2.5 V---L-BB (SFT Fine-tuned)
65.8
69.4
62.9
60.3
65.3
64.7
4PT-4n-Qwen2.5 V1.-BB (GRPO Fine tuned)
67.3
70.8
63.8
61.8
64.6
65.7
MMcdAgent-RL
71.5
76.2
72.3
73.3
71.9
73,0
G.12 DETAILED RESULTS
Traditional Medical Imaging Evaluation. Table 17 presents the accuracy of various models across five major medical imaging modalities in the OmniMedVQA benchmark. Our model (MMedAgent- RL) demonstrates strong generalization across all categories, achieving an average accuracy of 73.3%, significantly outperforming previous -tate-of-the-arr models including L.LVA-v1.6-- 11..1....3B (58.7%) and Dwen2.55VVL-77 (60.8%). Specifically. our method achieves 76% on microscopy images, indicating robust capability in processing fine-grained, high-resolution visual data typical of pathology slides. On MRI and CT modalities, MMedA gent-RL reaches 72% and 65%, respectively, outperforming strong baselines such as LLaVA-v1.C 116.-4BBBBBBB and Vi-VL-344 by wide margin. These results show that model captures both structural and soft-tissus anatomical details effectively. In X-R X-Ray, our method maintains competitive performance (78.8%) compared to high- performin models like CaaauoPPTVission-ae (80.3%) while achieving the highest accuracy Ultrasound (75%) among
30

---

Published as a conference paper at ICLR 2026
all models, demonstrating robustness in handling noisy, low-contrast imaging modalities.
MMMU Health & Medicine Track. In Table 18 model again establishes new performance standards, achieving 71.9% overall accuracy on the MMMU Health & Medicine test set. Compared existing large models such -wwn2.5-VL-7B (56.6%) and HuatuoGPT- Vision-- (51.0%), AMeeAggen-RLeemonsrraae clear advantages. Notably, our model excels across all five sub- domains: scoring 75% in Basic Medical Science (BMS), 78% in Clinical Medicine (CM), 65% in Diagnostics and Laboratory Medicine (DLM), 70% in Pharmacy (P), and 71.5% in Public Health (PH). These results reflect well-rounded capability across both foundational scientific understanding and applied clinical knowledge. In particular, performance in CM and shows substantial improvement over single-agent baselines, suggesting that our model benefits from enhanced reasoning and domain transfer. Taken together, these results confirm the effectiveness of our approach in both imaging- based and knowledge- based medical VQA settings, and highlight the potential our method as comprehensive solution for multimodal medical understanding.
Table 17: The accuracy of OmniMedVQA within different modalities (excluding FP, OCT, and Dermatology). CT: Computed Tomography, MRI: Magnetic Resonance Imaging, Mic: Microscopy Images, X-Ray: X-ray, US: Ultrasound.
Model
CT
MRI
Mic
X-Ray
US
Avg.
Med-Flamingo
34,6
27.5
28.1
30.1
33.2
30.7
RadFM
33.3
22.0
28.0
31.5
26.1
28.2
LLaVA-Med-7B
25.3
35.9
44.0
31.7
83.7
44.1
51.5
43.9
49.5
63.1
33.5
48.3
Yi-VI-334B
39.8
51.4
61.4
64.2
40.5
51.5
LLaVA-v1.6-7B
40.1
54,8
48.8
53.3
47.9
49,0
LLaVA-v1.6-13BB
40.0
47.4
50.5
59.6
42.6
48.0
LLaVA-v1.6-34B
50.6
60.9
62.8
74.7
58.7
1aVA-v1.5-LLLMA3-BB
33.0
53.8
48.4
56.6
31.2
44.6
HuatuoGPT-Vision-778
65.6
72.7
77.5
80.3
76.7
74.6
Qwen2.5-VL-3B
60.5
64.2
66.6
68.9
40.4
60.1
Qwen2.5-VL-7B
62.0
68.3
70.7
68.9
34.3
60.8
Multi-Agent Collaboration
MedAgents
55.0
57.2
58.6
55.8
MDAgents
58.1
60.5
61.7
60.2
50.6
58.2
AFlow
59.5
62.0
63.2
61.7
51.6
59.6
MMedAgent-RII -e.eeet-tR-R (7B)
64.6
71.7
76.0
78.8
75.4
73.3
G.13
MORE CASES
To further demonstrate the robustness and versatility of our proposed model in multimodal medical applications, present additional representative cases in Figure 7 Figure Figure annd Figure These examples encompass various clinical scenarios and imaging modalities, providing compre- hensive illustration of the model's ability effectively integrate and interpret diverse types of medical data.
31

---

Published as a conference paper at ICLR 2026
Table 18: Results the test for the MMMU Health & Medicine track. The Health & Medicine track is divided into five categories: BMS for Basic Medical Science, CM for Clinical Medicine, DLM for Diagnostics and Laboratory Medicine, P for Pharmacy, and PH for Public Health, Results are obtained by submitting to the official website.
Model
BMS
CM
DLM
PH
MMMU Health & Medicine
Mld-Flamingo
29.6
28.1
24.8
25.3
31.2
28.3
RadFM
27.5
26.8
25.8
24.7
29.1
27.0
LLaVA-Med-7B
39.9
39.1
34.6
37.4
34.0
36.9
Owen-VL-Chat
36.5
31.7
32.7
28.4
34.6
32.7
Yi-VL-34B
49.4
48.9
43.2
40.5
32.0
41.5
LLaVA-v1.6-7B
40.5
36.9
32.1
32.3
26.9
33.1
LLaVA-v11.6-13B
53.6
46.7
33.3
22.2
40.0
39.3
LLaVA-v1.6-34B
56.4
56.0
46.9
46.7
41.7
48.8
HuatuoGPT-VisionnTTB
60.7
63.3
36.7
51.0
Qwen2.5-VL VL.3B
67.8
53.3
43.3
55.6
53.3
54.5
Qwen2.5-VL-7B
67.9
56.7
36.7
66.7
56.7
56.6
Multi-Agent Collaboration
MedAgents
60.0
55.0
45.0
50.0
52.9
52.6
MDAgents
62.5
57.5
50.0
55.0
50.5
55.1
MMedAgent-RRL (7B)
75.0
78.0
65.0
70.0
71.5
71.9
Question:
What can be observed in this image?
Options:
A. Shin splints, B. Tennis elbow, C. Plantar fascia pathology, D.
Meniscal tear
Answer: C. Plantar fascla pathology.
The image provided is CT of foot, specifically showing the plantar fascia. The CT scan typically shows bone structures and soft tissue, and in this case, clearly displays the plantar fascia, which common site for plantar fasciitis or other plantar fascia-related pathologies. <answer>C. Plantar fascla pathology</answer>
Ours:
The figure below <image shows stable structure a protein that is be usad for each part of this problem. The dashed line divides the structure into three regions labeled structure and how do you reach that conclusion? Options: A. True, B. False.
Question:
and in the figure. Does this protein have quaternary
Answer: False
The figure shows single protein structure with no indication of multiple protein subunits or different domains interacting. In proteins, quaternar structure refers to the arrangement of two or more polypeptide chains that form a complex. Since image only displays one continuous chain without any visible subunit separation, does not represent quaternary structure. <answer>B, False</answer>
Ours:
Figure 7: Case 1 (CT sean of foot): The model identifies plantar fascia pathology from a CT image, requiring anatomical knowledge of soft tissue structures in the foot and recognition of characteristic changes consistent with plantar fasciitis. Case 2 (Protein structure diagram): A structural biology reasoning task based on protein diagram, where the absence of distinct subunit boundaries leads to the conclusion that the protein does not exhibit quaternary structure showcasing visual-structural reasoning in molecular biology.
32

---

Published as a conference paper at ICLR 2026
Question:
What part is shown ultrasound image?
Options: Testicle, B. Large intestine, C. pancreas, Liver Answer: C. pancreas.
Ours:
The ultrasound image shows structure that appears to be solid mass or lesion, which is consistent with C. Plantar fascia pathology. This is typically seen in imaging of the foot and ankle region.
<answer>G</answer>
The vestibular system can stimulated irritated by an infection the inner ear or by turor (arrow) pressing on vestibulocochlear nerve. Which the following would NOT occur?
Question:
Options: Decreased auditory acuity. Vertigo. Tinnitis. D. Loss facial sensation.
Answer: D. Loss facial sensation
MRI scan of the brain, not of the inner Therefore, it does not show any information related to the vestibular system or its components such as the vestibulocochlear nerve. The question asks about the vestibular system, which is not depicted in this image. Since the image does not contain relevant information for the question, we cannot determine what would or would not occur based on this image alone. <answer>D. Loss of facial sensation.c/answer:
Ours: The image provided
vestibular system.
Figure 8: Case (Ultrasound image): The model interprets an ultrasound to identify the pancreas, integrating grayscale image features with organ shape and texture- example of soft tissue recog- nition using sonography. Case (Brain MRI with question about vestibular system): This task highlights the model's ability identify mismatches between the image modality (brain MRI) and the clinical question (vestibular system), and to reason that facial sensation loss is not explainable from the given scan.
Question: Is polysplenia present?
Options: A. yes, B. no
Answer: A. yes.
The image provided multiple organs appear abnormally positioned or fused together, which characteristic of polysplenia. Polysplenia is condition where there than two spleens, often accompanied other anomalies such as heart defects and other organ malformations. <answer=A, yes</answer>
Ours:
Question: Does malignant adenoma show bizarre strap and fusiform cells? Options: yes, B. Answer: B, no
The image provided is histological slide showing pattern consistent with plantar fascia pathology. There is no indication of malignant adenoma bizarre strap and fusiform cells. Therefore, the correct answer is B. B<aasser>
Ours:
Figure 9: Case 5 (Abdominal CT image polysplenia diagnosis): The task involves recognizing abnormal organ positioning indicative of polysplenia. This case highlights anatomical reasoning through CT imaging, requiring recognition of fused or duplicated spleens and an understanding of associated syndromic features. Case 6 (Histopathology slide adenoma diagnosis): A histological image is used to assess the presence of malignant features. The model correctly distinguishes normal plantar fascia morphology from pathological adenoma patterns, demonstrating reasoning in pathology image interpretation.
33

---

Published as a conference paper at ICLR 2026
Question: Five elements make up 98% the mass of the human body. These elements are oxygen (61%) 9), carbon (23%), hydrogen (10.0%), nitrogen (2.6%), and calcium (1.4%).. Compare these data with those in the pie graph below, which shows five most abundant elements by mass in Earth's crust, oceans, and atmosphere. <image are abundant both the human body Earth's crust, oceans,
1>,
and atmosphere?
Options: iron, oxygen, Aluminum, silicon, B. Aluminum, silicon, iron, calcium, C. oxygen, silicon. iron, calcium
Answer: oxygen, silicon, iron, calcium.
Ours:
The pie chart shows that the five most abundant elements by mass in Earth's crust, oceans, and atmosphere are oxygen (49.8%), silicon (26.0%), aluminum (7.6 (7.6%), iron (4.8%), and calcium (3.4%). Comparing these with the five most abundant elements in the human body (oxygen, carbon, hydrogen, nitrogen, and calcium), we can see that oxygen is present in both sets of data. <answer>C. oxygen, silicon, iron, calcium</answer>
Figure 10: Case 7 (Pie chart comparison elemental composition); This case blends image data (a pie chart of Earth's element composition) with textual biochemical data (composition of the human body). The reasoning required crosses domains, comparing datasets to identify overlapping elements, exemplifying multimodal cross-referencing and synthesis.
34


Reviews:
[{"review_id": "diK9Fu0Sb2", "reviewer": "Reviewer_NzMH", "summary": "This paper proposes MMedAgent-RL, a reinforcement learning framework for multi-agent medical reasoning. The system simulates clinical workflows with a triage doctor routing cases to specialists (proprietary LVLMs), then an attending physician trained via curriculum RL to aggregate specialist opinions. The key innovation is a curriculum learning strategy with dynamic entropy regulation that progressively teaches the model to handle specialist outputs of varying reliability. Experiments on 5 medical VQA benchmarks show significant gains over baselines.", "questions": "1. In \"w/o Triage\" (Table 2), what exactly happens? Random specialist? All specialists? Please clarify and consider: single model with diverse sampling might achieve similar diversity without routing.\n\n2. Can you provide results progressively adding components (base → +triage → +multi-expert → +curriculum RL) to quantify each contribution?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Framing multi-agent medical reasoning as a curriculum RL problem with dynamic entropy control is well-motivated by the reality of imperfect expert judgments.\n\n2. Strong empirical results, 23.6% average gain over baselines and excellent OOD generalization (72.6% on MMMU/OmniMedVQA) demonstrate effectiveness.\n\n3. The three-stage curriculum (easy/medium/hard based on specialist accuracy) with corresponding entropy coefficients (0.0001/0.005/0.03) is principled and clearly explained.", "weaknesses": "1. Missing critical baselines: No comparison with simpler alternatives that could  possible achieve similar results, eg. single GPT-4o or Qwen2.5-VL sampling N diverse outputs using different prompts or high temperatures → majority voting or trained aggregator. These would test if the complex triage+multi-expert pipeline is necessary.\n\n2. The paper claims the attending physician learns to \"correct specialist mistakes,\" but provides no quantitative evidence on hard cases where all specialists fail. A fairer and more rigorous evaluation is needed to determine how much of the performance gain is due to routing and how much is due to aggregation. \n\n3. Best performance requires OpenAI llms, but medical data cannot be sent to external APIs in privacy-critical environments. Where is the evaluation with deployable open-source specialists ?\n\nI will reconsider my rating if the author addresses these questions well.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DyGw2qX4m1", "reviewer": "Reviewer_noLG", "summary": "This paper introduces MMedAgent-RL, a reinforcement learning (RL)-based multi-agent framework that overcomes the rigidity of existing collaboration systems by enabling dynamic and optimized cooperation among medical agents. The framework mimics a clinical \"triage-and-referral\" system, utilizing a curriculum RL strategy with dynamic entropy regulation to train a primary model to intelligently integrate and resolve noisy or conflicting inputs from various specialist agents.", "questions": "Please refer to Weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper is well-written, logically clear, and easy to follow.\n2. The theoretical derivations are fairly sound.\n3. Extensive experiments demonstrate the superiority of the proposed MMedAgent-RL.", "weaknesses": "1. The middle part of Figure 1(a) does not reflect the practical workflow of Multi-Agent collaboration; it seems to lack representation of the General Practitioner, which leads to ambiguity.\n2. Section 3.1 mentions optimizing the triage doctor using GRPO, so it would be worthwhile to discuss the triage doctor's capability (quantitatively) as well as its reasoning process.\n3. The underlying mechanism for the entropy regularization term in Equation 3.1 needs to be explained, and the rationale behind the choice and range of values for $\\gamma_s$ should also be elaborated.\n4. Figure 3 mentions the selection of three 'o3' models as specialist doctors? Does this mean each specialty uses three 'o3's? If so, there seems to be no differentiation between the specialties. Why was 'o3' chosen over a specially designed medical MLLM?\n5. The experimental setup in Figure 4 is not clearly described, and more details should be provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ygwAWgRO7G", "reviewer": "Reviewer_dwrJ", "summary": "This paper develops a RL-based framework, namely MMedAgent-RL, for optimizing multi-agent collaboration in medical VLM reasoning. The proposed framework is well-motivated and empirically strong, with evaluations on both in-domain and out-of-distribution datasets.", "questions": "The authors use Qwen2.5-VL as the base model. Is this framework transferable to other base models?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "(1) This framework develops a machine that can adjust collaboration policies based on task difficulty. The integration of C-MARL for entropy control is theoretically motivated.\n(2) The evaluation and experimental design are comprehensive. The proposed framework shows good performance in 5 public datasets, including both in-domain and out-of-distribution datasets.", "weaknesses": "(1) The framework is inspired by the 'triage–specialist–attending'. The authors need to find more evidence to demonstrate that this aligns with the real hospitalization process. Within different sections in a hospital, the workflow may differ.\n(2) This work lacks the involvement of human experts. \n(3) Some of the technical details are missing. For example, are the first GP and the second GP updated simultaneously?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YJxzQ5zLQ5", "reviewer": "Reviewer_A9FJ", "summary": "This paper proposes MMedAgent-RL, a multi-agent framework based on RL designed to optimize collaboration for multimodal medical diagnostic reasoning. The framework trains a triage doctor agent to assign cases and an attending physician agent, which uses a curriculum learning strategy, to integrate opinions from various specialist agents and make a final decision.", "questions": "Please see the weaknesses above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposed curriculum-based RL strategy to train an attending agent to evaluate and integrate potentially noisy or conflicting advice from specialists is a novel and well-motivated approach to improving the robustness of multi-agent systems.", "weaknesses": "My major comments are:\n\n1. The specialist agents that provide the core domain knowledge are powerful proprietary models like GPT-4o and o3. This makes the framework's performance heavily dependent on external, closed-source models. The experiments do not clearly show how the system performs when only open-source models are used as specialists.\n2.  The paper claims a significant performance gain over an SFT method. However, this SFT baseline is not clearly defined in the main text. It is uncertain if this is a model fine-tuned on ground-truth answers, on specialist responses, or another configuration. \n3. The paper states the triage doctor is optimized using GRPO. However, the ablation study in Table 2 only presents a \"w/o Triage\" condition, which removes the step entirely. It fails to compare the RL-optimized triage agent against a simpler, non-RL baseline (e.g., a standard SFT-trained classifier). Given the near-perfect triage accuracy reported, the necessity of using RL for this component is not well-justified.\n4.  The curriculum learning strategy splits training data based on specialist accuracy (easy, medium, hard). The paper then presents an analysis of performance on test data that is also split by difficulty. The method for splitting this test data is not explained. If it is based on the specialists' accuracy on the test set, this constitutes data leakage, as it uses information about the test answers to perform the analysis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)