MedAgentGym: A Scalable Agentic Training Environment for Code-Centric Reasoning in Biomedical Data Science
Authors: Ran Xu
Yuchen Zhuang
Yishan Zhong
Yue Yu
Zifeng Wang
Xiangru Tang
Hang Wu
May Dongmei Wang
Peifeng Ruan
Donghan Yang
Tao Wang
Guanghua Xiao
Xin Liu
Carl Yang
Yang Xie
Wenqi Shi
Venue: ICLR 2026 Oral (2026)
Keywords: Medical Reasoning
LLM Agent
Code Generation
Primary area: datasets and benchmarks
TL;DR: MedAgentGym is a scalable and interactive training environment designed to enhance coding-based biomedical reasoning capabilities in LLM agents.

Abstract: We introduce MedAgentGym, a scalable and interactive training environment designed to enhance coding-based biomedical reasoning capabilities in large language model (LLM) agents. MedAgentGym comprises 72,413 task instances across 129 categories derived from 12 authentic real-world biomedical scenarios. Tasks are encapsulated within executable sandbox environments, each featuring detailed task specifications, interactive feedback mechanisms, verifiable ground truth annotations, and scalable training trajectory generation. Extensive benchmarking of 29 LLMs reveals substantial performance disparities in biomedical data science between commercial and open-source LLMs. Leveraging efficient multi-threaded and multi-turn trajectory sampling in MedAgentGym, Med-Copilot achieves performance gains of +43.02% and +45.28% from offline and online reinforcement learning, respectively, demonstrating MedAgentGym as an effective training ground while establishing itself as a cost-effective, privacy-preserving alternative competitive with proprietary LLMs (gpt-4o). By offering a unified execution environment with a comprehensive benchmark and accessible, extensible training resources, MedAgentGym delivers an integrated platform to develop LLM-based coding assistants for advanced biomedical data science.

Published as a conference paper at ICLR 2026
MEDAGENTGYM: A SCALABLE AGENTIC TRAINING ENVIRONMENT FOR CODE-CENTRIC REASONING IN BIOMEDICAL DATA SCIENCE
Ran Xu'; Yuchen Zhuang"; Yishan Zhong?, Yue Yu', Zifeng Wang', Xiangru Tang", Hang W May D. Wang", Peifeng Ruan Donghan Yang", Tao Wang',
Guanghua Xiao², Xin Liu", Carl Yang', Yang Xie Wenqi Shi
Emory University' Georgia Institute Technology² University of Illinois Urbana-Champaign" Yale University* UT Southwestern Medical Center University of Washington
MedAgentGym: https: //gi thub. cwm/wshi// wttp://hhhhhsaaaem 83/MedggnttGymm MedAgentGym:
ABSTRACT
We introduce MedAgentGym, a scalable and interactive training environment de- signed to enhance coding-based biomedical reasoning capabilities large language model (LLM) agents. MedAgentGym comprises 72, 413 task instances across 129 categories derived from 12 authentic real-world biomedical scenarios. Tasks are encapsulated within executable sandbox enviromments, each featuring detailed task specifications, interactive feedback mechanisms, verifiable ground truth an- notations, and scalable training trajectory generation. Extensive benchmarking of 29 LLMS reveals substantial performance disparities in biomedical data science between commercial and open-source LLMs. Leveraging efficient multi-hhreaded and multi-turn trajectory sampling in MedAgentGymm Med-Copildt Copilot achieves per- formance gains of -2% and +-4.28% from offline and online reinforcement learning. respectively, demonstrating MedAgentGym an effective training ground while establishing itself as a cost-effective, privacy-preserving alternative competi- tive with proprietary LLMs (g)t-40). By offering a unified execution environment with comprehensive benchmark and accessible, extensible training resources, MedAgentGym delivers an integrated platform to develop LLM-baser coding assis- tants for advanced biomedical data science.
Avgi 56.75
Bio SWE
Medical Calculation
opt-o4-mini
q-i-4.1-néni
30t
APi-bared Commerdall
C.B
spt-4o-nini
Clinical Notes
Clinical DB Quarry
Med 14p
Analysis
Med Coplict 78
Oura
Ditill Loma 708
Owen3
Ulmm-3 T08-
BHR Predictive
Modeling
Biostats
Qwen3-148
Instrct
OSS LLMs
Owen3-80
Dats Analysis
Uama3311-5
Instruct
Wed-Coolten 20 (bum)
g of (3625-56-1))
Dwwr2.5-7Binatrp ManurGFTel-78
Avgg: 25.78
Med-Coolte 348 10umi
037-4 1-min 2035 04-14)
Owerida
Oworn2 trstrct
MedAgentGym Overall Score
(a) Biomedical coding capabilities with MedAgent
(b) Overall score of MedAgentGym
Figure I: Overview of (a) task-specific and (b) overall leaderboard evaluation in MedAgentGym. The results show the (a) performance variations across biomedical data science tasks and (b) large gaps between proprietary and open-source (OSS) LLMs, highlighting the need for continued development of
affordable LLM agents, especially for complex code-based biomedical reason- ing tasks such as biomedical software engineering and predictive modeling.
Equul contribution. *Correspondence (Yang. Xie, Wenqi. Shi wwUTSouthweetern.nmmm edu.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
The exponential growth of healthcare data has fundamentally transformed modern biomedical re- search, intensifying the need for integration of advanced computational methods with medical domain expertise (Wornow et al., 2023b; Liu et al., 2025b). Biomedical researchers routinely face data science challenges that demand both medical data analysis knowledge and programming proficiency, such as querying large-scale databases, conducting statistical analyses, processing genomic sequences, and building predictive models from electronic health records (EHRs) (Nimmolrat et al., 2021; Lee al., 2022; Wornow et al., 2023a). While recent advances in large language models (LLMs) have demonstrated significant capabilities in advanced reasoning (OpenAI, 2025b; Guo et al., 2025), including code generation (DeepMind. 2025) and scientific discovery (Swanson et al., 2024; Team et al., 2025; Yuan et al., 2025). it remains challenging to translate real-works biomedical data science requirements into executable computational solutions (Wang al., 2024b; 2025d).
Developing effective biomedical coding agents poses unique challenges beyond nowledge-intensivgg medical reasoning (Wang al., 2025b;c) and general-purposs code generation (Zheng al., 2024; Jing et al., 2025). Within biomedical research and clinical practice, direct deployment of proprietary LLMs remains infeasible due to strict privacy requirements and prohibitive operational costs (Meskó & Topol, 2023; Shi et al., 2024a), whereas OSS LLMs exhibit substantial deficiencies in biomedical coding capabilities (Figure 1). Mitigating this performance disparity calls for addressing two infras- tructure gaps: (1) comprehensive, code-centric biomedical reasoning benchmarks to diagnose agent limitations and support rigorous, reproducible evaluation; and (2) specialized, interactive training environments to develop the complex reasoning and robust coding capabilities required for real-world biomedical data science.
In this study, we introduce MedAgentGym, a scalable and agentic training environment designed to systematically enhance the coding-centric reasoning capabilities of LLM agents for biomedical data science workflows. Grounded in diverse real-world biomedical scenarios, MedAgentGym provides:
Comprehensive suite of code-centric biomedical reasoning tasks. MedAgentGym encompasses 72,413 biomedical coding-centre instances across 129 categories grounded in 12 real-world biomedical scenarios'. We standardize rich collection of biomedical data science tasks as executable problems with verifiable ground truth, spanning structured medical information retrieval, numerical clinical reasoning, bioinformatics, and machine learning (ML) modeling. Tasks incorporate diverse data modalities, including EHR tables, clinical notes, genomics, drugs, and biological sequences, which require medical sppeif- reasoning capabilities.
Scalable and interactive training infrastructure. MedAgentGym provides optimized, user- friendly environment to accelerate agent training. Each instance is encapsulated within executable, isolated, and reproducible Docker environments with pre-install dependencies, supporting multi- threading, parallel execution, and sequential sampling. MedAgentGym ensures efficient trajectory collection and facilitate large-scale automated evaluation compatible with diverse agent scaffolds.
Extensive benchmarking and effective agent training for biomedical data science. Through an extensive benchmark of 29 proprietary and open-source LLMs, we identify critical deficiencies in biomedical data analysis and predictive modeling. MedAgentGym effectively strengthens agentic training: Med-Copi 1o achieves gains of +43.02% and +45.28% through offline and online reinforcement learning (RL), respectively, and performs comparably to gpt-4o on both in- and out-of-distribution tasks. We publicly release MedAgentGym and Med-Copilot, together with high-quality training trajectories and the outcome verifier, to support reproducible benchmarking and continued development of LLM coding agents in biomedical data science.
2 RELATED WORKS
Coding-Centrie Reasoning in Biomedical Data Science. Most existing medical benchmarks primarily evaluate LLMs on knowledge-intensive narrative reasoning (Jin et al., 2019; Pal et al., 2022; Tsatsaronis al., 2015). Although several efforts target isolated biomedical algorithmic tasks (Tang et al., 2024a; HAI Stanford, 2025; Wang et al., 2024b) or simulate portions of clinical
We emphasize that MedAgent mainly focuses on computational code generation for biomedical reason- ing, rather than traditional medical coding systems (Soroush et al., 2024) such as ICD-9 or ICD-10.
2

---

Published as a conference paper at ICLR 2026
Table I: Summary of related biomedical reasoning and coding datasets with task details and execution environments. MyyAgentGym is among the first publicly available training environments for LLM agents in biomedical data science, uniquely integrating executable environments, interactive feedback, and task-isolated run-time facilities for coding-barc based reasoning. "DB", "DA", "Bioinfo", and "ML" denote "database", "data analytics", "bioinformaties" and "machine learning" respectively.
Environment Facility
Coding
Bioanto
Execution
Ineeaction
Isolztion
Truning
Truin
MedMCQA (Pul 2012)
X
X
X
3K
4.14K
MadQA (Jin st, 2021)
X
X
X
X
1.21K
X
PuhMalQQA (Jin et al, 2019)
X
X
X
X
X
450
500
X
BioASQ (Tsatsuronis 2015) 2015)
x
X
X
X
X
MalA Bench (Tang et 2015)
x
X
X
X
X
862
X
MIRAGE (Ging 2024))
x
X
K
K
7,6KK
X
Health (Amer ot 2025)
X
X
X
X
X
SK
X
FHRSQL (1.00 2022)
V
X
X
X
X
I5.5K
1
X
MedCaleBench (Khimdekar 2024)
X
X
X
X
10.1K
LOSK
X
MedAgemmBench (Jang et 20256)
X
X
300
BioCuder (Ting al,, 2024g)
124K
X
BieDS (Wang al., 2024b)
128
EHKSHOT (Womew 2023a)
X
X
X
X
X
MedkgentGyn (Ours)
59.2
13.2K
6.7K
workflows (Schmidgall et al., 2024; Li et al., 2024c;b), they do not capture complete set of tasks in the full end-to-end lifecycle biomedical data science, from data extraction (Lee et al., 2022; Ryu al., 2024) model development (Wornow al., 2023a; Wang al., 2020b). Complementing these benchmarks, MedAgentGym emphasizes computation- and coding-intensive tasks that require LLM agents retrieve, transform, analyze, and compute biomedical data while generating and executing code with pre-installed biomedical libraries and dependencies to produce verifiable solutions.
Scalable and Interactive Training Environment for Biomedical Coding Agents. Agentic RL (Guo et al., 2025; Schulman et al., 2017; Shao et al., 2024b) shifts LLM post-training from passive sequence generation to autonomous agents operating in complex, dynamic settings, including medical reasoning (Xia et al., 2025; Jiang et al., 2025a; Chen et al., 2024; Lan et al., 2025; Wu et al., 2025a; Wang et al., 2025a). Within such a framework, agents interact iteratively with their environment, receiving observations and executing actions, while the environment returns reward signals and state updates (Wang et al., 2025e; Chezelles et al., 2024; Shao et al., 2024a; Nathani et al., 2025), However, most biomedical reasoning and data science benchmarks (Table 1) single-pass evaluations without executable environments or agent-level interaction signals (Zhu et al., 2025; Arora et al., 2025; Wu et al., 2025b). In contrast, MedAgentGym uniquely provides executable and interactive biomedical coding environment covering comprehensive range of tasks. It also supports efficient multi-turn trajectory sampling through multi-threaded rollouts, thus enabling scalable and systematic improvement via agentic fine-tuning beyond prompting (Shi et al., 2024b; Huang et al., 2025a).
3 MEDAGENTGYM: A SCALABLE AND INTERACTIVE LLM AGENT TRAINING ENVIRONMENT FOR CODE -ED-RCCC BIOMEDICAL REASONING
3.1 PROBLEM FORMULATION
We formulate coding-basee reasoning structured problem-solving task: given problem descrip- tion æ E X, the goal is to generate a code snippet e that produces an output y E y. Each instance (xr, y) is paired with ground truth output y², and the correctness is verified using & C'y + (0, 1}. where & = I[ = )² Existing biomedical reasoning datasets typically provide only question-answer pairs (x., 5/") without code solutions only include single predefined code solution per task. To address this, MedAgentGym enables scalable generation and sampling of multiple coding trajectories (()), c(1) (kk) with corresponding exccutions ()) ,y ... (() through parallel execution of LLM agents. Each trajectory is either single-turn multi-turn, depending task complexity and user requirements. Crucially, MedAgentGym captures both positive trajectories (c(0) = y²} that succeed and negative trajectories (1(() + y²} including error messages learning signals.
3.2
DATA CONSTRUCTION: FROM INDIVIDUAL DATASETS TO UNIFIED BENCHMARK
Task and Data Identification, MedAgentGym focuses on verifiable biomedical data science tasks that benefit from code-based solutions (i.e., code-centric biomedical reasoning). Clinically, we prioritize
3

---

Published as a conference paper at ICLR 2026
Table 2: Dataset statistics for MedAgentGym and its lightweight subset for leaderboard evaluation. "For open-ended tasks without explicit ground truth (e.g., ML coding in EHRSHOT and MIMIC- Extract), we follow standard RL settings by using the same dataset for training and evaluation.
Data Sourees
Tack Instunces (all)
Tasks (leader-board)
Type
Patients
#Table
#Elemenis
Category
4Train
4Train
#Tesi
Training and fntersal Volidation [Nn-Disriiution)
MIMIC-UI (Jeesson et al., 2016)
Tabular
<IK
17
1.4
9,318
1,122
10,480
552
581
1.133
eCUU (Pollant atal. 2018)
Tobular
<1K
10
1.5M
9
6,213
611
6,824
559
610
1,169
TREQS (Wing et oll, 2020s>
Tabular
100
5
2.5M
4
8,988
996
9,984
897
995
1,892
MooCaleBesch (Khandekar
20124
Texr
IK
55
10,053
1,347
11,100
1,005
1,046
2,051
ModAAgenBBncchh (liang et 20256)
Tabular
100
700K
10
433
109
542
239
.99
298
HloCldder
2034a)
Text
981
156
EHRSEBOT
2023x)
Tabular
1.2M
BioUSS (Wang 2024b)
50
Nod (Internal)
7.3M
113
36/036
4,106
40,142
4,283
3
7,794
Rateraal Volidation Dea--fff-Dtttiaaiionn 4 only the aert vet for exteraat evaluntion; training data remoiins anceasible
EHR ScySQL (Ryu et al,, 2024)
Tabular
<IK
17
1.4M
4
18,950
7,913
26,863
1,000
500
1,500
EHRCon (Kwon al., 2024)
Tear
46K
13
3,229
976
4,205
1,000
500
1,500
MIMIC-Extraet (Wang 20206)
Tabular
35K
35K
3
N-PowarAl (Ruin 2025)
960
MedAgentSym (Extermal)
1.4M
23,142
9.132
32,271
2,963
1.243
Ovenll
MedAgentGyn
146K
80
7.4M
129
59,175
13,238
72,413
7,243
4,754
11,997
tasks originating from real-world healtheare settings and validated by a multidisciplinary panel of healthcare experts. For example, MedAgentGym involves MIMIC-III and eICU in EHRSQL. (Lee al., 2022) collected from 222 hospital staff members and annotated by human programmers. Computationally, we integrate diverse coding tasks, ranging from structured medical information retrieval to open-ended biomedical research, ensuring comprehensive coverage and task diversity.
Verifiable Instances Preparation. To standardize tasks across various sources, each instance in MedAgentGym is structured with: (1) a problem description, (2) verifiable ground-truthh outputs, and (3) optional data resources (e.g.. EHRs). Additionally, standardized system and user prompts designed to initiate the problem-solving process (see appendix G). MedAgentGym is highly flexible, easily accommodating new tasks that include clear descriptions and verifiable ground-truth outputs. For coding-centric tasks that provide only reference code implementations (e.g., BioCoder (Tang al., 2024a)), we validate task correctness based on the execution output of these reference solutions, generating definitive output signatures. This transformation is necessary because multiple valid code implementations may yield identical execution results, making the execution outcome outcome-rather than the code itself-a a more reliable and consistent verification signal. For tasks involving additional data resources (e.g., EHRSQL (Lee et al., 2022)), we include metadata on data access and sources. Detailed task overview and task-specific preparation are documented in appendix C.
Data Statistics. MedAgentGym is a unified training environment built upon large-scale, high-quality dataset comprising approximately 72,000 task instances across 129 categories from 12 real- world biomedical scenarios. Notably, with MedAgentGym, we collect large-scale agent trajectories to support coding agent development (section 5). To ensure reproducible and robust evaluation, we define clear train/test splits, separate internal and external validation sets, and perform n- n-gram (n = 10) string match to eliminate the data contamination issue. Table provides statisties for MedAgentGym. To accommodate diverse research needs, we offer two versions of MedAgentGym: (1) a comprehensive, full-scale dataset for extensive exploration and detailed analysis, and (2) balanced, lightweight subset for efficient leaderboard training and evaluation.
3.3 CODING ENVIRONMENT: FROM STATIC BENCHMARK TO INTERACTIVE INTERFACE
Isolated and Executable Sandbox Environment. To ensure robust and reproducible coding- based biomedical reasoning, MedAgentGym provides isolated executable coding environments (i.e. sandbox) through Docker containers tailored to each task (Figure 2). These containers come pre- installed with all required dependencies, including specialized biomedical packages (e.g.. AlignIO in BioCoder (Tang et al., 2024a)), facilitating reliable task execution. To address critical data safety concerns, each Docker environment guarantees: (1) environmental integrity, where isolation prevents contamination or data corruption potentially caused by LLM--ennrated code, preserving both the computational environment and the underlying data systems (Yang al., 2024b); (2)
4

---

Published as a conference paper at ICLR 2026
Bio-Medical Metadata
7.4M Data
Agentic Traj.
MedAgentGym
Sandbox
EHR Data
Genomics Data
Sampling
LLM-friendly commands
Terminal
Clinical Notes
Drug ...
Request Info
Evaluste Code
Filesysten
Bio-Medical Data
Tasks
Agont
Edit File
Tiehag Code
Features
Science Tasks
72K Instances
LLM-friendly environment feedbacks
Labels
BioMed SWE
Predictive Medeling
Data Info
Code Results
Retadata
Trajectories
Unit-Test
Bio Statisties
BooMed Analysis...
Error Mags
Interact History
BioMed
Question
Reequest Info
Data Info
Coding
Errors
Debug
Resnlts
Reward
Figure 2: Overview of MydAgentGGm. MedAgentGym contains a comprehensive suite of coding-centric biomedical data science tasks with an interactive execution environment for LLM agents.
medical data security, where secure containerization enforces compliance with medical data usage policies, safeguarding sensitive patient information. Additionally, MedAgentGym supports extensive flexibility for integrating new tasks, where users can define customized Docker environments through configuration files. If certain packages are not initially available, a terminal tool allows LLM agents to dynamically install the required dependencies within their isolated environments.
Interactive Feedback. MedAgentGym incorporates interactive feedback mechanisms, effectively bridging LLMs with coding interpreters: (1) robust parsing: To begin, the output generated by LLMs is formatted in structured JSON, facilitating straightforward parsing and code execution. In cases of execution errors, iterative ISON regeneration is employed to maximize successful code execution rates. (2) debugging and error grounding: Compile- time and runtime error messages systematically translated into a unified natural language format, making them more accessible to LLMs and significantly improving debugging efficiency and interpretability.
Efficient Trajectory Collection. Each task in MedAgentGym is packaged in a reproducible Docker image with built-in support for multi-threadie parallel execution, and sequential sampling. Specifi- cally, we integrate two widely used multi- tittiihhradiig backend engines, Ray² and Joblib', to accelerate trajectory sampling. This infrastructure ensures efficient and scalable trajectory collection, supporting both extensive experimentation and systematic evaluation across multiple scenarios.
Plug-and-Play. A key strength of MedAgentGym lies in its flexible and modular architecture, which readily supports the integration of new biomedical coding tasks. This inherent extensibility enables MedAgentGym to continually adapt to evolving advancements in biomedical sciences and artificial intelligence methodologies. Additionally, its trajectory sampling approach allows the straightforward transformation of traditional, non-executable biomedical reasoning tasks into coding-based scenarios with verifiable outputs, significantly broadening the scope and complexity of tasks that can be system- atically evaluated. Moreover, users can define custom Docker environments through configuration files, and, if specific software packages initially absent, a built-in terminal tool facilitates dynamic installation within each isolated execution environment, further improving MedAgentGym in runtime adaptability and user-friendliness.
4 EVALUATING LLMS AS MEDICAL CODING AGENTS WITH MEDAGENTGYM
4.1 EXPERIMENTS SETUP
Agent Scaffolds. Following CodeAct (Wang et al., 2024a), we establish a default agent scaffold for systematically evaluating coding- based biomedical reasoning. Interactions within MedAgentGym are modeled as a Partially Observable Markov Decision Process (POMDP), focusing on sampled biomedical data science tasks p E P. At each timestep I, the agent observes o, E O and samples an action d++1 E AA from the current policy based on interaction history. We define four primary
action types: (a) request_inmeee innfesttttiooottttet retrieve relevant data from sources such as EHRs; (b) terminal: manage dependencies or local files within isolated Docker environments. (c)
execute code generated by LLMs through an integrated interpreter; and (d) debugging translate code execution errors into natural language explanations enriched with detailed error information for LLM comprehension.
5

---

Published as a conference paper at ICLR 2026
Table 3: Test set results (zero-shot) of LLMs on MedAgentGym. Bold indicates the best result at each
scale. # and V denote coding LLMs and medical reasoning LLMs, respectively.
Duttaseds (-)
MIMIC SR
eCCU SR
TREQS
MedCal..
MedAgent.
BioCuder SR
BioDS, SR
EHRSHOOT Are
Avg. Score
Baselines LJ/ Metrics
SR
SR
SR
API-bared Proprietory L.Mel; We only consider Microedt Azure
OpenAT API
services doe enddeniiated hesilih data agreementt
art-4o-mini (8-444 (Huns et al, 2024)
35.97
16.57
18.39
73.01
40.38
30.12
57.35
7.84
37,47
ast-4 (2824-08-3) 05) (Hasst 2024)
43.04
13.97
54,23
30.12
gpt-4.1-min Immini (2875-44-14
(OpanA), 2025a)
62.79
84.30
54.23
47.
apt-1.1
(2825-94-14) (OpenAl,
52.95
apt-of-mini (2825-66-10) (OpeA)) 2025h)
76.45
minder-inni (2925-86-15) (Chen 11,2200 2021)
67. 30
64.75
82.40
43.78
58.06
OSS (Bow Stzel: < 108 parameterss
Qwen3-1 76 (Qwes, 20155]
20.12
10.62
15.00
-6.24
16.95
15.38
6.12
1.87
16.55
Qwen3-48 (Uwen, 20253)
27
30.77
28.85
52.80
15.25
19.16
20.43
23.85
27.29
gema-3344--tt (Gommm 2025)
27. 36
29. 10
24.52
42.
18,64
17.05
8.16
4.37
21.57
"medgena-t0--1 (Google, 2025)
15:51
13.11
14.85
41.89
17.62
26.74
17.82
1.33
18.61
(wen225-78-Irie 5-78-Irstruct (Ying et st, 2004g)
13.08
15.57
12.76
25.91
30.36
21.79
10.20
5.42
17.45
L -8----Instt (Dubey et 2024)
16.67
25.00
19.17
27.53
16.95
18.39
9.19
2.36
16.97
loven2. 1--coder-B--Isttee
st
9.32
10.56
24.62
10.60
17.24
10.55
14.65
"HustuodPT-ol-D (Clinn
4.99
25.21
"#1-78-22K (Tlsngg 20256)
12.00
Qwen3-8B (Uwen, 2025a)
29.08
37.37
211.4
20.51
-t--srral----nnsrutt--222- (Ministral, 2025)
16.70
18.92
23.72
12.24
"MidResson-38 (Wo 2025g)
9.12
9.51
9.15
21
31.42
17:42
3.88
CSded-Coder-4B Reeessoing (Seed et al., 2025)
42.51
45.74
39.50
35.18
28.83
23.72
20.43
22.89
32.35
OSS (Large Strrl: 10-308 puromsiezz
Qwan3-148 148 (Qwen, 2025a)
31. $1.50
31:97
30.05
61.38
22.03
22.
26.53
26.77
31.60
Qwen2, 5- 1--140-nnstruc (Yang et al., 202(a)
17:21
14.07
16.43
27:40
35.59
29.49
16.33
-1455
20.12
Rl-bii------Dt::111-- 14 (Gun et 2025)
35.12
38.52
32.95
48.09
32.20
21.29
24.49
11.39
30.51
fowen2. 5-Coder-1168-8 48-Instruct (Hul 2024)
41.82
44.26
33.75
26.28
32,89
"Balchuan-N) 148-----rt (Wang 2025a)
4.50
17.42
(SS5 Siorio pommmoeeess
(Qwen,
32.67
18.57
Qven2. 50t.5---2----nneee (Yang et al,, 2024a)
QxQ-328 (Qwes, 20256)
62: 31
47.46
42.31
14.20
51.50
DeepSeek-R1 R11----t:11----e----- (Gun
2025)
62:18
53.36
26.53
31.17
47.05
Buichhnn-M2-372 (Don al. 2025)
20:83
23.61
34.93
30.02
25.42
25.00
12.94
22,89
Llang-3, 3-g--tttttt (Dubey et et.l. 2024) (Gun et al., 2025)
39.93
25.08
24.95
84.99
39,00
27.35
24.89
29.93
37.04
DeepScek-R1 1ist::0ttllllaassss
64.59
64.92
36.95
76.55 96
28.81
32.05
42.86
33.42
50.07
HHutuntiP--l-72 QQen2.55 5-728) (Chen dl. 2024)
27.19
29.55 34
29.65
52.01
28.83
31.41
26.53
16.87
30.29
Tasks and Datasets. Building upon MedAgentGym, we train and evaluate Med-Copilot 7,794
coding-based biomedical reasoning tasks across 8 datasets: (1) MIMIC-III (Johnson et al., 2016) and (2) eICU (Pollard al., 2018) from EHRSQL (Lee et al., 2022), (3) TREQS (Wang al., 2020a), (4) MedCalcBench (Khandekar et al., 2024), (5) MedAgentBencc (Jiang et al., 2025b), (6) BioCoder (Tang et al., 2024a), (7) EHRSHOT (Wornow al., 2023a), and (8) BioDSBench (Wang et al., 2024b). Moreover, we conduct experiments for out-of-distribuutin evaluation on 4,203 tasks from the following datasets: (9) EHR HHR-Se (Ryu et al., 2024), (10) EHRCon (Kwon al., 2024), (11) MIMIC-Extract (Wang et al., 2020b), and (12) N-PowerAl (Ruan et al., 2025). Note that we do not consider knowledge-intensive medical question- question-answering tasks (Jin et al., 2019; Pal et al., 2022; Jin al., 2021), as they are orthogonal to coding-aides -iiid reasoning. We include detailed task and dataset information in appendix C.
Baselines. We extensively benchmark the following state-of-the-e art LLMs on MedAgentGym: (i) API-based proprietary LLMs, including gpt-4o-mini (Hurst et al., 2024), gpt-40 (Hurst al., 2024), gpt-4.1-mini (OpenAI, 2025a), gpt-4.1 (OpenAI, 2025a), gpt-o4-mini (OpenAI, 2025b), and codex-mini (Chen et al., 2021); ); (ii) OSS I.MMs, including gemma-3 (Gemma, 2025), Qwen3 (Qwen, 2025a), Qwen2.5 (Yang et al., 2024a), Llama-3 (Dubey et al., 2024), Minis- tral (Ministral, 2025), and DeepSeek-RI (Guo et al., 2025); (iii) coding LLMs, including codex- mini (Chen et al., 2021), Swwen25--Coder-7--Imme Seed-Coder-See Coder- Reasoonin (Seed et al., 2025): and (iv) medical reasoning LLMs or medical domain- specific LLMs, including medgemma- (Qwen2. 5- 5-7B-Instruct) and MimauooPPT-o--2200 (Qwen2.5-72B) (Chen et al., 2024), m m1-7B-23K (Qwen2. 5-7B-Instrucct (Huang et al., 2025b), MedReason-8B (Llama-3, Inst-unst (Wu al., 2025a), Baichuan-Mh 10---11111 I1----nnsttu ( (Wang et al., 2025a), and Baichuan-M2-322 (Dou et al., 2025). Additional model details are available in appendix D.
and -14B-Instruct (Hui al., 2024), and
(gemma-3-4b-pt) (Google, 2025), HuatuoGPT-ol--7
We adopt success rate (SR) the primary evaluation metric. For database, data science, and bioinformaties tasks with explicit ground truths, we compare LLM-gennrated code execution outputs with reference solutions using exact match. For open-ended ML tasks in clinical decision support, we measure performance using accuracy (Acc) across test cases. See appendix for implementation details and F.1 for additional evaluation code quality and efficiency.
Evaluation Metries.

---

Published as a conference paper at ICLR 2026
Action
{ r))
Action
((,ar))
T
Trajectories
π
k
Gym
Agent
Gym
Agent
Gym
Agent
7k++1
Obs
Trajectory
πI
Obs o.
Obs Reward r
Reward r
Sampling
Reward
Inference
Rollouts
Me+1
(a) Offline Reinforcement Learning w/ MedAgentGym
(b) Online RL w/ MedAgentGym
Figure 3: Comparison of (a) offline and (b) online RL paradigms within MedAgentGym.
4.2 RESULTS: BENCHMARKING LLMs AND REASONING MODELS WITH MEDAGENTGYM
Table benchmarks the tarte-rrt L.Mss on MedAgent We summarize key observations from our zero-shot leaderboard evaluation follows: o Significant Performance Gap Between Commercial API-based and OSS LLMs. This evident performance gap highlights the critical need for continued development of lightweight OSS LLMs that match commercial performance while addressing real-world privacy and cost constraints. o Task-Specific Performance Variations between Structured and Open-ended Medical Tasks. LLMs consistently perform better on structured tasks (e.g., database queries, medical calculations) compared to open-ended tasks requiring advanced coding and reasoning (e-g-, data analysis, MI prediction). o Suboptimal Outcomes in Dedicated Coding and Medical Domain-Speciffe LLMs. Both coding and medical reasoning LLMs deliver limited improvement or even decline over base models, revealing that poding-based biomedical reasoning represents a unique capability not adequately captured by specialization in either coding or medical reasoning. Surprisingly, medical reasoning models (regardless of model sizes) consistently underperform relative to their base models except for Knowledge-intensiv tasks (e.g., MedCal cBench, BioCoder). showing that fine-tuning in medical QA may reduce generalization and cnatructton-folloviiii ability. These findings highlight the need to jointly improve coding skills and medical reasoning, rather than treating them as separate objectives.
5 TRAINING LLM AGENTS FOR CODE-CENTRIC BIOMEDICAL REASONING
In this section, we leverage MedAgentGym to systematically enhance lightweight OSS LLMs as proficient coding agents (Med-Copilot) for biomedical reasoning. We first explore two-stage agentic fine-tuning framework (section 5) followed by a detailed analysis of model scaling behaviors (section 5.2). We then introduce ceff-improvement to further boost agent performance (section 5.3) and conduct additional analysis on model generalization, ablation, and error patterns (section 5.4).
5.1 RL FINE-TUNING WITH TRAJECTORY SAMPLING
Training Setup. We select Qwen-2. 2.nen-2.5-Immee 5-Instructt-8 and -14B (Yang et al., 2024a) as our backbones. To enable effective evaluation within MedAgentGym, utilize consistent CodeAct-style scaffold, allowing LLM agents to iteratively reason and refine biomedical code through interactive environment feedback. Detailed training setups, including hyperparameters, are provided in appendix E.
Trajectory Sampling. MedAgentGym facilitates efficient parallel trajectory sampling using ray and joblib backends. Specifically, we roll out (1) 2,137 successful trajectories using gpt-4. a temperature of 0 to warm the fine-tuning for smaller OSS models. Each successful trajectory contains 9.25 turns between the LLM and the code interpreter average. In addition to 2,137 positive trajectories for supervised fine-tuning (SFT), we prepare additional trajectory pairs for RL such as direct preference optimization (DPO), including (2) 1,646 off-policy preference pairs sampled from gpt-4. 1-mini, and (3) 2,939 on-policy preference pairs For both types, we use the initial prompt interactions as shared context and contrast successful final codes against intermediate erroncous attempts. In addition, also performed quantitative analysis 250+ trajectories (randomly sampled over 10% our trajectory collection) and confirmed that the vast majority of successful solutions followed logically sound path, with cases of 'correct answer from flawed code' being exceptionally rare (<1 %). We release all 6K trajectories above to accelerate coding agent development. See appendix C.6 for detailed trajectories composition.
1-mini 1--inn with
Two-Stage Fine-Tuning, We benchmark two policy improvement methods: (1) SFT directly mimics high-reward trajectories consisting exclusively of successful outcomes, whereas (2) offline online
7

---

Published as a conference paper at ICLR 2026
Table 4: Med-Copilot performance MedAgentGym finetuned with sampled trajectories.
Dattaset (-))
MIMIC-III
eiCU SR
TREQS
MedCale.
MedAgent.
BloCoder
BioDS. SR
RHRSHOT
Avg.
Buse (U1/ Metries
SR
SR
SR
SR
SR
Acc
Score
Qwen2 1aon2.6-7--Insruuee
21.79
*SFT
57.83
61.48
72.66
89.06
28.35
55. 55,10
15.62
53.87
+DPO
64.13
66.91
72.02
90.06
52.54
34,62
69.39
29.55
39.90
(+13.002
-PPO
66.10
67.25
73.88
74,52
51.35
32.71
65.47
52.40
57.96
(+41,07)
.GRPO
68.21
68.73
70.50
92.33
58.87
37.40
71.11
33.18
62.17
(145522)
Owen2 5-155-11I-nneee 449-Inst
17.21
14.07
16.43
27.40
35.59
29.49
16.35
4.45
20.12
+SFT
62
52.54
89.
34.58
(+41.80)
+DPO
66.37
+PPO
67.55
68.53
78.32
94.86
53.22
45.83
91.35
56.79
69.56
(+19.444 (9.44)
+GRPO
68.78
69.34
76.84
95.81
S7.41
49.32
94.78
39.05
71.42
(+:31.30)
Inference Time Scaling
Training-Time Time Scaling
50
Success Rate (56)
40-
Success Rate (9%)
38
35
40
43.7%
300
25
Pass Best@k
917.0
20
6.11
14
60
80
100
Number of Agent Rollouts
Training Used
Figure 4: Scalable improvements of LLM agents in MedAgentGym. For inference-time scaling, we employ T 0 for the initial rollout and T = 0.6 for the rest. For train-time scaling, we T 0.
RL optimizes the policy by favoring selected responses over rejected ones (Figure 3). We further consider a two-stage fine-tuning, initially warming up with SFT and subsequently refining with RL.
Offline RL (DPO). Table compares several post-training methods, revealing that sim- ple SFT over successful trajectories significantly boosts performance on structured coding tasks, demonstrating its effectiveness in capturing structured coding patterns. Besides, DPO is particularly beneficial for optimizing open- open-ended task performance.
Results:
Results: Online RL (PPO and GRPO). We further consider online RL methods, including Prox- imal Policy Optimization (PPO) (Schulman et al., 2017) and Group Relative Policy Optimization (GRPO) (Shao al., 2024b), to enable Med-Copilot to actively explore tasks and dynamically generate higher-quality training data through interaction. The evaluation module of Med-Copi is employed to provide two reward signals: a correctness reward and a format reward, the latter indicating whether the generated output contains code blocks. As shown in Table 4, GRPO achieve markedly stronger performance, suggesting enhanced generalization capabilities in diverse biomedical scenarios compared with offline RL.
5.2 SCALING LLM AGENT IMPROVEMENTS WITH MEDAGENTGYM
Verifier Training Setup. In addition to directly training coding agents, MedAgentGym facilitates the development of an nutcome-supervised reward model (ORM) to evaluate generated solutions effectively. Inspired by prior work (Cobbe al., 2021; Pan al., 2025), formalize the verifier task as predicting the probability that a given trajectory successfully solves a coding task. Formally, we represent a trajectory as an interleaved sequence 7 [01, a1 O2, 02, ... ,On, an,, [0, 1], where each observation comprises elements such as task descriptions, code execution results, and error feedback. We fine-tune a Qwen2. IB-Instt model as a verifier with binary predictions 'YES' (ly) or 'NO' (l,), from which we compute success probability: = cxp(ly)) (exp(l,) + +xxp(I,))
Verifier Training Data. We construct the verifier training dataset by combining two sets of trajectories originally sampled for agent training: (1) off-policy Irajectories, consisting of 2,742 samples from 10t-4.1-mint and (2) on-policy trajectories, comprising 2,939 samples generated by the agent. Combining both and off off-policy trajectories,
ensure balanced dataset of successful and unsuccessful trajectories, filtering to within a maximum context length of 32k tokens.
Results: Inference and Training-Time Scaling, We introduce two additional evaluation metrics: (1) Pass @ K: the fraction tasks solved by at least one trajectory from sampled attempts; and (2) @est@A K: the fraction of accurately selects successful trajectories that actually solves the task from a set of candidate generations. Figure 4 (left) illustrates the performance scaling with increasing trajectory sampling. Pass @K significantly improves from 17.0% at K to 45.0% at 16, while
8

---

Published as a conference paper at ICLR 2026
60
Base
RS+ RS+SFT+DRO+DFO
gpt-4.1-minl
50
gpt-4.1-mini wio debug
50.39%
Avg Score (N)
(90)
2.53%
40
15 12.22%
1.47%
30
40
20
50.39%
20
enzr
10
BloInto+ML
Othere
Figure 5: Self-Improvement
Figure 6: Effect of Debug
Figure Error Types
Best@K K shows steady advancement from 17.0% to 41.7%. The relatively small gap between metrics indicates that our trained verifier effectively identifies successful trajectories, unleashing its potential as a reward model for integration into advanced online RI. frameworks. Figure (right) examines agent performance as function of increased training data volumes in SFT. We observe consistent performance improvements with greater training data availability, suggesting additional computational resources dedicated to sampling further trajectories are likely to yield continued performance gains.
5.3 MODEL PERFORMANCE SCALING WITH SELF-IMPROVEMENT
leef-emprovetnnt Training Setup. Beyond expert sepert-generated trajectories from gpt-4. 1-mini, we also explore self- improvemeet by refining the model on its own outputs (Qwen2. 1iven2.5-7BB8 5-7B-Insstruct) We first apply rejection-sampling SFT: starting from Qwen2. 5-7BBIInstruct we collect 1,000 successful trajectories and perform filtered behavior cloning on this set. We subsequently apply DPO (section 5.1) using on-policy preference pairs generated by the rejection sampling SFT checkpoint. Specifically, we sample eight rollouts per task, score them via the verifier in section 5.2, and form 4,298 pairs by contrasting the highest-scoring correct and lowest-scoring incorrect trajectories. Following Pang al. (2024), we repeat this data collection and policy update cycle (iDPO) for further refinement, resampling trajectories and reconstructing preference pairs after each DPO update. In contrast to standard DPO, which performs a single offline preference- based update on a fixed dataset, iDPO alternates between on-policy data collection and DPO optimization that the policy and training data co-evolve (i.e., self-impprvemmnnt)
Rejection Sampling (RS) and iDPO. Figure 5 illustrates consistent performance gains across one SFT stage and two subsequent DPO stages. However, we observe diminishing returns over successive iterations. Initially, rejection sampling SFT significantly boosts performance by effectively capturing successful coding patterns. Subsequent DPO stages show smaller incremental improve- ments, reflecting the model's diminishing exploration space as tackles increasingly challenging tasks, ultimately converging toward an approximate Nash equilibrium.
Results:
5.4 GENERALIZATION, ABLATION, AND ERROR ANALYSIS
Results: External Evaluation. Table 5 summarizes external eval- uation results on MedAgentGym. The external suites were inten- tionally chosen to stress different code-centric skills (e.g., sequen- tial SQL, hybrid text-table consis- tency, EHR time series, bio- statistical power analysis), which induces markedly different agent trajectories (Figure 10(b)) and nat- urally yield lower absolute scores across all models, including pro- prietary baselines. Med-Copilot with SFT and DPO modestly improve performance on open- ended, reasoning- intensive tasks (e.g., MIMIC- Extract) However, improvements remain limited, in-
Table 5: External test set results on MedAgentGym.
Boon (.3 (Mtrio(--))
SRt
SB
Score:
APF-Aaee Propetenry M2 (for eytorenrry
rpt 4a mins rtal., 2024)
ppt-4o
1-eled (OpenAL 2025g)
25.00
43.20
apt-4, 4, (OnesAl, 2025g
78.20
63.00
10:41
33.53
5..25
apt oeraini (OnesAl, 2025b)
180.0
$1.00
16:58
36.15
$3.94
LEAM
Owend-1 (Qwer, 3005a)
$1.00
12.30
1.91
14.22
16:86
Ower (Qwom 307913
41.80
76.30
1.4
19.90
33.73
2-15 (Qwen. 20254)
52,00
31.40
6.52
20.12
27.59
Red Capslat (ST1, 780
42.80
1.05
10.45
30:01
Nod-Coolltt 0o33ttt:0000 (DP0, 7B0
43.00
25.00
2.14
14.52
30:84
Hod-Cooslot (PPO, 78)
45.00
26. 40
4.30
17.09
22.87
Dopilat
23.59
35.71
Ower2 Z-Coder-140 (Mal
25.95
32.55
Ower2: 3- Tnee (Ybin al. 3000g
06:00
39 30
0.51
$1.57
31:43
Red-Cosilat IDPO, 148)
42.20
40.00
2.75
25.80
27.91
Coollot bt
46.00
40:00
2.19
17.00
75.19
Gnor2-128 (Qwes, 20254)
64.30
54.00
12.17
31.25
42.16

---

Published as a conference paper at ICLR 2026
dicating challenges in generalizing across specialized biomedical contexts. In particular, incorporating online RL optimization techniques, especially GRPO (Shao et al., 2024b), can effectively improve performance unseen, out-of-distribution tasks. Specifically, Med- Bedd-Copilot-14B (GRPO) achieves 47.02% on the external suite, a + +19. gain over its backbone (27.92%) 29.... This significant gain on unseen distributions verifies that MedAgent&ym instills transferable biomedical coding proficiency rather than memorizing training trajectories.
Effect of Interactive Coding. Figure 6 shows that removing debugging capabilities significantly decreases model performance across all tasks. Interactive coding mechanism in MedAgentGym substantially contributes to successful coding-based medical reasoning by enabling the model to effectively interpret and rectify execution errors.
Error Analysis. Figure 7 summarizes common error types encountered by the strongest evaluated LLM, gpt-4.1. 1. Loop-related issues dominate, accounting for 50.39% of errors, where agents repeatedly execute the same action in the final turns, indicating difficulty in adapting or exploring alternative strategies. This highlights the need to promote effective exploration and enhance robustness in solving complex biomedical reasoning tasks. Additional experimental results, including cost analysis, case studies, and human studies, are available in appendix F.
6 CONCLUSION
We present MedAgentGym, an executable, privacy-presse prereccirresrvingg and extensible training environment for scaling code-based biomedical reasoning in LLM agents. With 72K task instances across 129 categories, MedAgentGym enables comprehensive benchmarking of 29 proprictary and OSS LLMs for biomedical data science within modular, decoupled architecture that supports flexibility and extensibility. Med-Copilot further demonstrates that systematic training and trajectory sampling with MedAgentGym improve coding proficiency for biomedical data science tasks. MedAgentGym has the potential to accelerate progress from structured medical information retrieval tasks toward more open-ended computational research questions in clinical research and biomedical discovery.
ETHICS STATEMENT
We confirm that all authors read and will adhere to the ICLR Code of Ethics. This study uses only publicly available or credentialed deidentified datasets (e.g.. MIMIC-III and eICU) under their licenses or data use agreements. We do not redistribute data that require credentialed access; instead, we provide scripts obtain and prepare such data. Licensing and access requirements for all datasets and associated code bases are summarized in Table 6, and privacy practices are detailed in appendix A.3. particular, we followed the PhysioNet Credentialed Health Data Use Agreement for MIMIC-III and eICU and did not transfer any confidential patient data to third-party services. When using Microsoft Azure OpenAl services, we opted out of human review and followed the PhysioNet guidelines for responsible use.
ACKNOWLEDGMENTS
RX and CY were partially supported by the internal funds and GPU servers provided by the Computer Science Department of Emory University, Emory Global Diabetes Center of the Woodruff Sciences Center, the US National Science Foundation under Award Numbers 2442172, 23 2312502, 2319449, and the US National Institutes of Health under Award Numbers K25DK13591, RFINS 39325, RO1DK 143456 and U18DP006922. GX was partially supported by the the US National Institutes of Health under Award Numbers
BUGGI154473 UOICA249245. and CPRIT Awards under RP230330 and RP250561. YX was partially supported by the the US National Institutes of Health under Award Numbers U01AI169298, R35GM1 36375, and CPRIT RP240521. WS was partially supported by the internal funds and GPU servers provided by the Texas Advanced Computing Center (TACC), NVIDIA Academic Grant Program, Google Gemini Academic Program Award, and Thinking Machines Lab Tinker Research Grant.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Rahul K. Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quinonero-Candela. Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, Johannes Heidecke, and Karan Singhal. Hcalthbench: Evaluating large language models towards improved human health. OpenA/ Blog, 2025.
Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, et al. Mle-bench: Evaluating machine learning agents on machine learning engineering. arXiv preprint arXiv:2410.07092 2024.
Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang. Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. Huatuogpt-ol, towards medical complex reasoning with llms. arXiv preprint arXiv:2412.18923 2024.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint (ttt:21007.037744 2021.
De Chezelles, Thibault Le Sellier, Maxime Gasse, Alexandre Lacoste, Alexandre Drouin, Massimo Caccia, Léo Boisvert, Megh Thakkar, Tom Marty, Rim Assouel, et al. The browsergym ecosystem for web agent research. arXiv preprint arXiv:2412.05461 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Refichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. 14168, 2021.
Google DeepMind. Alphaevolve: A coding agent for scientific and algorithmic discovery. Google DeepMind Blog, 2025.
Chengfeng Dou, Chong Liu, Fan Yang, Fei Li, Jiyuan Jia, Mingyang Chen, Qiang Ju, Shuai Wang, Shunya Dang, Tianpeng Li, al. Baichuan-m2: Scaling medical capability with large verifier system. arXiv preprint urXiv:2509.022000 2025.
Alexandre Drouin, Maxime Gasse, Massimo Caccia, Issam H. Laradji, Manuel Del Verme, Tom Marty, David Vazquez, Nicolas Chapados, and Alexandre Lacoste. Workarena: How capable are web agents at solving common knowledge work tasks? In Forty-first International Conference on Machine Learning. 2024. URL pettps://openreviiee www.eee..ee/fooooe net//
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 2024.
Shanghua Gao, Richard Zhu, Zhenglun Kong, Ayush Noori, Xiaorui Su, Curtis Ginder, Theodoros Tsiligkaridis, and Marinka Zitnik. Txagent: An ai agent for therapeutic reasoning across universe of tools. arXiv preprint arXiv:2503.10970 2025.
Gemma. Gemma technical report. arXiv preprint arXiv:2503. 2025.
Google. Medgemma hugging face. wtw.erreeeeeeeeeeee0ee4400000000
https://huggingfaace co/ co/collecttoo htt.cooollecioooooogllleeee 2025. Accessed: [2025-05-20].
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprins arXiv:2501. 2025.
Siyuan Guo, Cheng Deng, Ying Wen, Hechang Chen, Chang, and Jun Wang. DS-agent: Automated data science by empowering large language models with case-based reasoning. In Forty-first International Conference on Machine Learning, 2024. URL hwwph: httttp:://opereeviewneet///// net.feruuwntffooum id=LfJgeBNCFI.
HAI@ Stanford. plications.
Holistic evaluation of large language models for medical ap- Blog Post, 2025. URL https://hait
11

---

Published as a conference paper at ICLR 2026
Kexin Huang, Serena Zhang. Hanchen Wang, Yuanhao Qu, Yingzhou Lu, Yusuf Roohani, Ryan Li, Lin Qiu, Gavin Li, Junze Zhang, et al. Biommi: A general-purpose biomedical ai agent. biorxiv, 2025a.
Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. Mlagentbench: Evaluating language agents on machine learning experimentation. arXiv preprint arXiv:2310.03302, 2023.
Xiaoke Huang. Juncheng Wu, Hui Liu, Xianfeng Tang, and Yuyin Zhou. ml: Unleash the tential of test-time scaling for medical reasoning with large language models. arXiv preprint NrXiv:2504.00866 2025b.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang. Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409. 12/86, 2024.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024.
Shuyang Jiang, Yusheng Liao, Zhe Chen, Ya Zhang, Yanfeng Wang, and Yu Wang. Meds Towards medical small language models with self-evolved slow thinking, 2025a. URL https: org/abs/2501. 12051.
Yixing Jiang, Kameron C Black, Gloria Geng, Danny Park, Andrew Y Ng, and Jonathan H Chen. Medagentbench: Dataset for benchmarking Ilms agents in medical applications. arXiv preprint 1.Xi::2501.1664,, 2025b.
Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024. URL /ttttrr//operreview.. net/ forum?id=vTFayNNMM6
Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421, 2021.
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. PubMedQA: A dataset for biomedical research question answering. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing
pp. 2567-2577, Hong Kong, China, November 2019, Association for Computational Linguistics. doi: 10. 10.18653/v1/D19- 1259. URL https: //aclanthology... www/ 1259/.
Qiao Jin, Zhizheng Wang, Yifan Yang. Qingqing Zhu, Donald Wright, Thomas Huang, W John Wilbur, Zhe He, Andrew Taylor, Qingyu Chen, et al. Agentmd: Empowering language agents for risk prediction with large-scale clinical tool learning. arXiv preprint arXiv:2402. 13225, 2024.
Ruofan Jin, Zaixi Zhang, Mengdi Wang, and Le Cong. Stella: Self-evolving Ilm agent for biomedical research. arXiv preprint arXiv:2507.02004 2025.
Liqiang Jing, Zhehui Huang, Xiaoyang Wang, Wenlin Yao, Wenhao Yu, Kaixin Ma, Hongming Zhang, Xinya Du, and Dong Yu. DSBench: How far are data science agents from becoming data science experts? In The Thirteenth International Conference on Learning Representations, 2025. wwweeeewwwwwweeeeeeeeeeeeeee nett/forum?id-DssSPPRRR
URL
Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger Mark. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1 3(1):1-9, 2016.
Hao Kang and Chenyan Xiong. Researcharena: Benchmarking llms' ability to collect and organize information as research agents. arXiv preprint arXiv:2406. 10291, 2024.
12

---

Published as a conference paper at ICLR 2026
Nikhil Khandekar, Qiao Jin, Guangzhi Xiong. Soren Dunn, Serina Applebaum, Zain Anwar, Maame Sarfo-Gyamfi, Conrad Safranek, Abid Anwar, Andrew Zhang, et al. Medcalc-benche Evaluating large language models for medical calculations. Advances in Neural Information Processing Systems, 1184300-84755 2024.
Yubin Kim, Chanwoo Park, Hyewon Jeong, Yik Siu Chan, Xuhai Xu, Daniel McDuff, Hyeonhoon Lee, Marzyeh Ghassemi, Cynthia Breazeal, and Hae Won Park. MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making. In Advances in Neural Information Processing Systems, 2024.
Yeonsu Kwon, Jiho Kim, Gyubok Lee, Seongsu Bae, Daeun Kyung, Wonchul Cha, Tom Pollard, ALISTAIR JOHNSON, and Edward Choi. EHRCon: Dataset for checking consistency between unstructured notes and structured tables in electronic health records. In The Thirly-eighl Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https: //openreiie. net/ wit.ummmdddo0oocooooo
Yuxiang Lai, Jike Zhong, Ming Li, Shitian Zhao, and Xiaofeng Yang. Med-rl: Reinforcement learning for generalizable medical reasoning in vision-language models, arXiv preprint arXiv:2503. 13399, 2025.
Wuyang Lan, Wenzheng Wang. Changwei Ji, Guoxing Yang. Yongbo Zhang, Xiaohong Liu, Song Wu, and Guangyu Wang. Clinicalgpt-rl: Pushing reasoning capabilliy of generalist disease diagnosis with large language model. arXiv preprint arXiv:2504. 09420,941111 2025.
Gyubok Lee, Hyeonji Hwang, Seongsu Bae, Yeonsu Kwon, Woncheol Shin, Seongjun Yang, Minjoon Seo, Jong-Yeur Kim, and Edward Choi. Ehrsqi: A practical text-to-sql benchmark for electronic health records. Advances in Neural Information Processing Systems, 35:15589-15601, 2022.
Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, et al. Mmedagent: Learning to use medical tools with multi-modal agent. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 8745-8760, 2024a.
Junkai Li, Siyu Wang, Meng Zhang, Weitao Li, Yunghwei Lai, Xinhui Kang, Weizhi Ma, and Yang Liu. Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents. arXiv preprint arXiv:2405.02957 2024b.
Shuyue Stella Li, Jimin Mun, Faeze Brahman, Jonathan Ilgen, Yulia Tsvetkov, and Maarten Sap. Aligning Ilms to ask good questions case study in clinical reasoning. arXiv preprint arXiv:2502. 14860. 2025.
Stella Li, Vidhisha Balachandran, Shangbin Feng. Jonathan Ilgen, Emma Pierson, Pang Wei Koh, and Yulia Tsvetkov. Mediq: Question-asking Ilms and a benchmark for reliable interactive clinical reasoning. Advances in Neural Information Processing Systems, 37:28858- -28888, 2024c.
Yusheng Liao, Shuyang Jiang, Yanfeng Wang, and Yu Wang. Reflectool: Towards reflection-awe aware tool-augmented clinical agents. arXiv preprint arXiv:2410.17657 2024.
Valentin Liévin. Christoffer Egeberg Hother, Andreas Geert Motzfeldt, and Ole Winther. Can large language models reason about medical questions? Patterns, 5(3), 2024.
Che Liu, Haozhe Wang, Jiazhen Pan, Zhongwei Wan, Yong Dai, Fangzhen Lin, Wenjia Bai, Daniel Rueckert, and Rossella Arcucci. Beyond distillation: Pushing the limits of medical Ilm reasoning with minimalist rule-based rl. arXiv preprint arXiv:2505. 17952, 2025a.
Tianyu Liu, Simeng Han, Xiao Luo, Hanchen Wang. Pan Lu, Biqing Zhu, Yuge Wang. Keyi Li, Jiapeng Chen, Rihao Qu, et al. Towards artificial intelligence research assistant for expert-involved learning. arXiv preprint 2xXx::2555004633 2025b.
Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agenthench: Evaluating llms agents. arXiv preprint 2rXiv:2308.03688, 2023.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 171.0007 2017.
13

---

Published as a conference paper at ICLR 2026
Bertalan Meskó and Eric J Topol. The imperative for regulatory oversight of large language models (or generative ai) in healthcare. NPJ Digital Medicine, 6(1):120, 2023.
Jiacheng Miao, Joe R Davis, Jonathan K Pritchard, and James Zou. PaperZagent: Reimagining research papers as interactive and reliable ai agents. arXiv preprint arXiv:2509.06917, 2025.
Ministral. Un ministral, des ministraux. Ministral Blog, 2025. URL.https//miitraaa ai/nwws/ ministraux.
Ludovico Mitchener, Jon M Laurent, Benjamin Tenmann, Siddharth Narayanan, Geemi Wellawatte, Andrew White, Lorenzo Sani, and Samuel G Rodriques. Bixbench: comprehensive benchmark for llm-based agents in computational biology. arXiv preprint arXiv:2503. 00096, 2025.
Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M Krumholz, Jure Leskovec, Eric Topol, and Pranav Rajpurkar. Foundation models for generalist medical artificial intelligence. Nature, 616(7956):259-265, 2023.
Deepak Nathani, Lovish Madaan, Nicholas Roberts, Nikolay Bashlykov, Ajay Menon, Vincent Moens, Amar Budhiraja, Despoina Magka, Vladislav Vorotilov, Gaurav Chaurasia, et al. Migym: A new framework and benchmark for advancing ai research agents. arXiv preprint 25X142250..14499 2025.
Acrapol Nimmolrat, Krongkarn Sutham, and Orawit Thinnukool. Patient triage system for supporting the operation of dispatch centres and rescue teams. BMC medical informatics and decision making, 21:1-16, 2021.
OpenAI. Introducing gpt-4.1 in the api. OpenAl Blog, 2025a. URL https:///ppeaai.ccm/iimmm gpt-4-1/.
OpenAl. Openai o3 and o4-mini system card. OpenAl Blog, 2025b. URL httpt:/:://ppeea..coo -aitee/i-ooo-iii---- www.miiisstemeaarrnneeee card-
Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmeqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on health, inference, and learning, pp. 248 260. PMLR, 2022.
Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. Training software engineering agents and verifiers with SWE-gym. In ICLR 2025 Third Workshop Deep Learning for Code, 2025. URL
Richard Yuanzhe Pang, Weizhe Yuan, He He, Kyunghyun Cho, Sainbayar Sukhbaatar, and Jason Weston. Iterative reasoning preference optimization. Advances in Neural Information Processing 2024.
Systems,
Tom J Pollard, Alistair EW Johnson, Jesse D Raffa, Leo A Celi, Roger G Mark, and Omar Badawi. The eicu collaborative research database, a freely available multi-center database for critical care research. Scientific data, 5(1):1-13, 2018.
Cheng Qian, Emre Can Acikgoz, Hongru Wang, Xiusi Chen, Avirup Sil. Dilek Hakkani-TH -ürr, Gokhan Tur, and Heng Ji. Smart: Self-aware agent for tool overuse mitigation. arXiv preprint arXiv:2502. / 1435, 2025.
Jiahao Qiu, Xuan Qi, Tongcheng Zhang, Xinzhe Juan, Jiacheng Guo, Yifu Lu, Yimin Wang, Zixin Yao, Qihan Ren, Xun Jiang, Xing Zhou, Dongrui Liu, Ling Yang. Yue Wu, Kaixuan Huang, Shilong Liu, Hongru Wang, and Mengdi Wang. Alita: Generalist agent enabling scalable agentic reasoning with minimal predefinition and maximal self-evolution, 2025. URL https://arxiv.. org/abs/2505 2255.2222505..226.. 20286.
Qwen. Qwen3: Think deeper, act faster. Qwen Blog, 2025a. URL httpps:///wenll. http:/wwwwl..g.t...... githu..... io/ blog/qwen3/.
Qwen. Qwq-32b: Embracing the power of reinforcement learning, March 2025b. URL https: //qwenlm. gi thub. io/blog/qwqq32bb//
14

---

Published as a conference paper at ICLR 2026
Peifeng Ruan, Ismael Villanueva- Jialiang Liu, Donghan M Yang. Qinbo Zhou, Guanghua Xiao, and Yang Xie. N-power ai: A specialized agent framework for automated sample size and power analysis in clinical trial design. bioRxiv, pp. 2025-022 02, 2025.
Jachee Ryu, Seonhee Cho, Gyubok Lee, and Edward Choi. Ehr-seqsql: sequential text-to-say to! dataset for interactively exploring electronic health records. In Findings of the Association for Computational Linguistics ACI. 2024, pp. 16388-16407, 2024.
Samuel Schmidgall and Michael Moor. Agentrxiv: Towards collaborative autonomous research.
arXiv preprint arXiv:2503. 18102, 2025.
Samuel Schmidgall, Rojin Ziaci, Carl Harris, Eduardo Reis, Jeffrey Jopling, and Michael Moor. Agentelinic: a multimodal agent benchmark to evaluate ai in simulated clinical environments. arXiv preprint wxXh:2240500766. 2024.
Samuel Schmidgall, Yusheng Su, Ze Wang. Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using Ilm agents research assistants, 2025.
URL https:tpp:///t.. //txxvavvvv///////////e////aaav/ www. 2500 04227.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
ByteDance Seed, Yuyu Zhang, Jing Su, Yifan Sun, Chenguang Xi, Xia Xiao, Shen Zheng, Anxiang Zhang, Kaibo Liu, Daoguang Zan, et al. Seed-coder: Let the code model curate data for itself. arXiv preprint 24552420 2025.
Yijia Shao, Vinay Samuel, Yucheng Jiang, John Yang, and Diyi Yang. Collaborative gym: A frame- work for enabling and evaluating human-agent collaboration. arXiv preprint arXiv:2412.15701, 2024a.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.33300 2024b.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pp. 1279-1297, 2025.
Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Haotian Sun, Hang Wu, Carl Yang, and May D Waan. Medadapter: Efficient test-time adaptation of large language models towards medical reasoning. In Proceedings of the Conference Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, volume 2024, pp. 22294, 2024a.
Wenqi Shi, Ran Xu, Yuchen Zhuang. Yue Yu, Jieyu Zhang, Hang Wu, Yuanda Zhu, Joyce C. Ho, Carl Yang, and May Dongmei Wang. EHRAgent: Code empowers large language models for shot complex tabular reasoning electronic health records. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, pp. 22315-22339, 2024b.
Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. Nature, (20 2023.
Ali Soroush, Benjamin S Glicksberg, Eyal Zimlichman, Yiftach Barash, Robert Freeman, Alexan- der W Chamnyy Girish N Nadkarni, and Eyal Klang. Large language models are poor medical coders--benchmarking of medical code querying. NEJM AI, (5):AIdbp2300040, 2024.
Zafir Stojanovski. Oliver Stanley, Joe Sharratt, Richard Jones, Abdulhakeem Adefioye, Jean Kaddour, and Andreas Kopf. Reasoning gym: Reasoning environments for reinforcement learning with verifiable rewards, 2025. URL www.hhtts////iiiio wttp://iwwi....//////./.... 24760.
Kyle Swanson, Wesley WU, Nash L Bulaong, John E Pak, and James Zou. The virtual lab: Ai agents design sars-cov-2 nanobodies with experimental validation. bioRxiv, pp. 2024-11, 2024.
15

---

Published as a conference paper at ICLR 2026
Xiangru Tang, Yuliang Liu, Zefan Cai, Yanjun Shao, Junjie Lu,, Yichi Zhang, Zexuan Deng, Helan Hu, Kaikai An, Ruijun Huang, et al. MI-bench: Evaluating large language models and agents for machine learning tasks repository-level code. arXiv preprint arXiv:2311.0968 2023.
Xiangru Tang, Bill Qian, Rick Gao, Jiakang Chen, Xinyun Chen, and Mark B Gerstein. Biocoder: benchmark for bioinformatics code generation with large language models. Bioinformatics, 40, 2024a.
Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, and Mark Gerstein. Medagents: Large language models as collaborators for zero-shot medical reasoning. Findings of the Association for Computational Linguistics ACL 2024, pp. 599-621, 2024b.
Xiangru Tang, Daniel Shao, Jiwoong Sohn, Jiapeng Chen, Jiayi Zhang, Jinyu Xiang, Fang Wu, Yilun Zhao, Chenglin Wu, Wenqi Shi, et al. Medagentsbench: Benchmarking thinking models and agent frameworks for complex medical reasoning. arXiv preprint arXiv:2503.07459 2025.
Soroosh Tayebi Arasteh, Tianyu Han, Mahshad Lotfiiia, Christiane Kuhl, Jakob Nikolas Kather, Daniel Truhn, and Sven Nebelung. Large language models streamline automated machine learning for clinical studies. Nature Communications, 15(1) 2024.
NovelSeek Team, Bo Zhang, Shiyang Feng. Xiangchao Yan, Jiakang Yuan, Zhiyin Yu, Xiaohan He, Songtao Huang, Shaowei Hou, Zheng Nie, et al. Novelseck: When agent becomes the scientist- building closed-loop system from hypothesis to verification. arXiv preprint 2505 16938, 2025.
Rahul Thapa, Qingyang Wu, Kevin Wu, Harrison Zhang, Angela Zhang, Eric Wu, Haotian Ye, Suhana Bedi, Nevin Aresh, Joseph Boen, et al. Disentangling reasoning and knowledge in medical large language models. arXiv preprint arXiv:2505. 462, 2025.
George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopou- los, et al. An overview of the bioasq large-scale biomedical semantic indexing and question answering competition. BMC bioinformatics, 16: 16:1-28, 2015.
Bingning Wang, Haizhou Zhao, Huozhi Zhou, Liang Song, Mingyu Xu, Wei Cheng, Xiangrong Zeng, Yupeng Zhang, Yuqi Huo, Zecheng Wang, et al. Maichuan-mml Pushing the medical capability of large language models. arXiv preprint DrXiv:2502.12677 2025a.
Ping Wang, Tian Shi, and Chandan K Reddy. Text-to-sql generation for question answering on electronic medical records. In Proceedings of The Web Conference 2020, pp. 350-361, 2020a.
Shirly Wang, Matthew B. A. McDermott, Geeticka Chauhan, Marzyeh Ghassemi. Michael C. Hughes, and Tristan Naumann. Mimic-extract: data extraction, preprocessing, and representation pipeline for mimic-ii. In Proceedings of the ACM Conference on Health, Inference, and Learning, CHIL '20, 222-235, New York, NY, USA, 2020b. Association for Computing Machinery. ISBN 978 178145337442.. doi: (0.1145/3368555.338444 3384469.
URL https: / /ttp://oiioooo org/10. / 145/3368555.
Wenxuan Wang. Zizhan Ma, Meidan Ding, Shiyi Zheng, Shengyuan Liu, Jie Liu, Jiaming Ji, Wenting Chen, Xiang Li, Linlin Shen, et al. Medical reasoning in the era of Ilms: A systematic review of 2025b.
enhancement techniques and applications. arXiv preprint arXiv:2508.000
Wenxuan Wang, Zizhan Ma, Zheng Wang. Chenghan Wu, Jiaming Ji, Wenting Chen, Xiang Li, and Yixuan Yuan. A survey of llmebased agents in medicine: How far are we from baymax? arXiv preprint a1X1:25522.12211, 2025c.
Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better Ilm agents. In Forty-first International Conference on Machine Learning, 2024a.
Zifeng Wang, Benjamin Danek, Ziwei Yang, Zheng Chen, and Jimeng Sun. Can large language models replace data scientists in clinical research? arXiv preprint 11X11.2410.21591, 2024b.
16

---

Published as a conference paper at ICLR 2026
Zifeng Wang, Benjamin Danek, and Jimeng Sun. Biodsa-1k: Benchmarking data science agents for biomedical research. arXiv preprint arXiv:2505. 2025d.
Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, et al. Ragen: Understanding self-evolution in Ilm agents via multi-turn reinforcement learning, arXiv preprint arXiv:2504.20073 2025e.
Michael Wornow, Rahul Thapa, Ethan Steinberg, Jason Fries, and Nigam Shah. Ehrshot: An ehr benchmark for few-shot evaluation of foundation models. Advances in Neural Information Processing Systems, 36:671 25-67137, 2023a.
Michael Wornow, Yizhe Xu, Rahul Thapa, Birju Patel, Ethan Steinberg, Scott Fleming, Michael A Pfeffer, Jason Fries, and Nigam H Shah. The shaky foundations of clinical foundation models: A survey of large language models and foundation models for emrs. ArXiv preprint, abs/2303.12961. 2023b. URL https: /tttps://ariiv org/ 23033. 12961.
Juncheng Wu, Wenlong Deng, Xingxuan Li, Sheng Liu, Taomian Mi, Yifan Peng, Ziyang Xu, Yi Liu, Hyumjin Cho, Chang-In Choi, et al. Medreason: Eliciting factual medical reasoning steps in Ilms via knowledge graphs. arXiv preprini XXXv:2504.00993 2025a.
Kevin Wu, Eric Wu, Rahul Thapa, Kevin Wei, Angela Zhang, Arvind Suresh, Jacqueline Tao, Min Woo Sun, Alejandro Lozano, and James Zou. Medcasereasoning: Evaluating and learning diagnostic reasoning from clinical case reports. arXiv preprint urXiv:2505. 11733, 2025b.
Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong. Honglin Guo, Junzhe Wang, Dingwen Yang, Chenyang Liao, Xin Guo, Wei He, et al. Agentgym: Evolving large language model-based agents across diverse environments. arXiv preprint arXiv:2406.0417 2024.
Peng Xia, Jinglu Wang, Yibo Peng, Kaide Zeng, Xian Wu, Xiangru Tang, Hongtu Zhu, Yun Li, Shujie Liu, Yan Lu, et al. Mmedagent-rl: Optimizing multi-agent collaboration for multimodal medical reasoning, arXiv preprint 120Kv:::0000005555 2025.
Guangzhi Xiong, Qiao Jin, Zhiyong Lu, and Aidong Zhang. Benchmarking retrieval-augmennee generation for medicine. In Findings of the Association for Computational Linguistics ACL 2024, pp. 6233-6251, 2024.
Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint 1rX1v:2412.15115. 2024a.
John Yang. Carlos Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering. Advances in Neural Information Processing Systems, 37:50528-50652 2024b.
John Yang. Kilian Leret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. Swe-smith: Scaling data for software engineering agents, 2025. URL
Ailing Yu, Lan Yao. Jingnan Liu, Zhe Chen, Jiajun Yin, Yuan Wang, Xinhao Liao, Zhiling Ye, Ji Li. Yun Yue, et al. Medrescacher-rl: Expert-level medical deep researcher via a knowledge-informed trajectory synthesis framework. arXiv preprint arXiv:2508. /4880, 2025.
Jiakang Yuan, Xiangchao Yan, Botian Shi, Tao Chen, Wanli Ouyang, Bo Zhang, Lei Bai, Yu Qiao, and Bowen Zhou. Dolphin: Closed-loop open-ended auto-research through thinking, practice, and feedback. arXiv preprint arXiv:2501. 03916, 2025.
Dan Zhang, Sining Zhoubian, Min Cai, Fengzu Li. Lekang Yang, Wei Wang, Tianjiao Dong, Ziniu Hu, Jie Tang, and Yisong Yue. Datascibench: An 1lm agent benchmark for data science. arXiv preprint arXiv:2502. 02.5502..3899) 13897, 2025a.
Sheng Zhang, Qianchu Liu, Guanghui Qin, Tristan Naumann, and Hoifung Poon. Med-rlvr: Emerging medical reasoning from 3b base model via reinforcement learning. arXiv preprint arXiv:2502. 19655, 2025b.
17

---

Published as a conference paper at ICLR 2026
Yuge Zhang, Qiyang Jiang, XingyuHan XingyuHan, Nan Chen, Yuqing Yang. and Kan Ren. Bench- marking data science agents. In Proceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5677-5700, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 111166/////222442aoooogggg.. httoo://aaclantooooo...o/////// http://oogg..gg//...alllllla acl-long.-1111-.. 308/.
URL
Wanjia Zhao, Mert Yuksekgonul, Shirley Wu, and James Zou. Sirius: Self self-improving multi agent systems via bootstrapped reasoning. arXiv preprint arXiv:2502.04780, 2025.
Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue. Opencodeinterpreeer:: Integrating code generation with execution and refinement. arXiv preprint arXiv:2402.14633 2024.
Weihai Zhi, Jiayan Guo, and Shangyang Li. Medgr: Breaking the data barrier for medical reasoning via generative reward learning, arXiv preprini arXiv:2508, 20549, 2025.
Yinghao Zhu, Ziyi He, Haoran Hu, Xiaochen Zheng, Xichen Zhang, Zixiang Wang, Junyi Gao, Liantao Ma, and Lequan Yu. Medagentboard: Benchmarking multi-agent collaboration with conventional methods for diverse medical tasks. arXiv preprint arXiv:2505. 12371, 2025.
Yuxin Zuo, Shang Qu, Yifei Li, Zhangren Chen, Xuckai Zhu, Ermo Hua, Kaiyan Zhang, Ning Ding, and Bowen Zhou. Medxpertqa: Benchmarking expert-level medical reasoning and understanding. arXiv preprint arXiv:2501.18662, 2025.
18

---

Published as a conference paper at ICLR 2026
A LIMITATIONS AND BROADER IMPACTS
A.1 LIMITATIONS
Resource Limitations. Although MedAgentGym demonstrates strong empirical performance im- provement in a wide range of coding-aided biomedical reasoning tasks, several limitations remain. Firstly, MedAgentGym requires substantial computational resources for trajectory sampling, model fine-tuning, and iterative self-imppovemment procedures. Although we achieve significant improve- ments with relatively lightweight OSS LLMs, further scaling and advanced RL methods require increased computing infrastructures, limiting accessibility for resource-constrained research groups. Secondly, current dataset size and trajectory collection are primarily constrained by computational budget rather than data availability, potentially limiting the full exploration of model scaling behavior. Thirdly, MedAgentGym primarily supports text and structured data modalities. Future extensions will incorporate multimodal biomedical data (e.g., medical imaging, EEG, audio or video signals), en- abling a richer and more comprehensive evaluation of multi-modee reasoning capabiliitess Achieving effective multi-modal integration, however, presents significant challenges in data collection, curation, and standardized evaluation frameworks. Lastly, because of the substantial computational cost of API-based LLMs, we restrict ourselves to large-scale, execution-verifee single-run evaluation under a fixed scaffold. We encourage future work to conduct more extensive uncertainty analyses and multi-run evaluations where resources allow.
Data Decontamination. We acknowledge the possibility of data contamination in the pre-training of proprietary LLMs. To mitigate this, we have taken several steps: (1) Restricied access and credentialing: We constructed many datasets from protected data that cannot be used for proprietary LLMs training (e.g.. MIMIC, EHRSHOT, EHR-SeqSQL. EHRCon, MIMIC-Extract or very recent sources where possible (e.g., Bentgenn BioDSBench). (2) Newly curated samples: For N-PowerAI, manually curated samples rather than public repository, effectively creating private evaluation set. (3) Rigorous N-Gram decontamination: We performed n-gram overlap checks eliminate direct contamination between our training and test splits.
A.2 BROADER IMPACTS
Potential Positive Societal Impacts. MedAgentGym can significantly enhance the development of accessible, affordable, and privacy-preserving AI tools for clinical decision-making. Improved coding-based biomedical reasoning capabilities in open-source I.I.M agents (e.g., Med-Copi lot) have the potential to democratize access to advanced computational healthcare assistance, benefiting clinicians, researchers, and healthcare systems globally, particularly in mnource-llmitte settings. The plug-and-play architecture also allows continuous adaptation to new medical knowledge and practices, fostering sustainable and community-drivee innovation in healthcare technology.
Potential Negative Socictal Impacts. Despite the benefits, the introduction and widespread de- ployment of sophisticated computational frameworks like MedAgentGym may unintentionally widen existing healthcare inequities. Institutions with limited computational resources (including both Mi- crosoft Azure API service and nigh-performance computing clusters) or inadequate data infrastructure may struggle to access or fully benefit from these technological advancements, potentially exacerbat- ing disparities in healthcare capabilities across regions or socioeconomic groups. Moreover, reliance on publicly available datasets may perpetuate existing biases due to uneven data representation, potentially disadvantaging underrepresented patient populations and rare disease conditions.
Implications for Low-Resource Settings. For institutions constrained to <10B OSS models, we recommend the following practices: (1) Prioritize agentic fine-tuning over zero-shot prompt- ing. Zero-shot performance of small OSS models is modest (e.g., Qwen3-8B at 30%), but agentic fine-tuning in MedAgentGym substantially closes this gap. In particular, GRPO lifts a 7B backbone from 16.89% to 62. 67%, approaching the performance of commercial APIs (e.g., gpt-4. 1-iin at 61.72%) This suggests that small models are viable for deployment they are post-trained, rather than used purely in zero-shot mode. (2) Exploit released models and trajectories to re- duce compute. To lower the barrier to entry. we release Med-Copilot-7B/114 as ready-to-use, fine-tuned models that achieve state-of-the art performance among OSS baselines, well >6K high-quality trajectories (successful and preference pairs). These resources allow practitioners to (i) adopt Med-Copilo lot directly, or (ii) perform lightweight SFT/DPO on top of their own backbones
19

---

Published as a conference paper at ICLR 2026
without incurring the full cost of trajectory sampling and filtering. (3) Leverage verifier-gated inference rather than larger models. We recommend integrating the outcome verifier at deployment time. By sampling a small number of candidate solutions and selecting with the verifier (Best @ K), practitioners can substantially improve reliability relative to single-shot decoding, effectively trading modest additional inference compute for large accuracy gains, without resorting to much larger or proprietary models. (4) Specialize to the target biomedical data science task type. The aggregate scores in Table reflect performance over 129 heterogeneous categories across 8 datasets, whereas real deployments often focus on narrower workloads (e.g., clinical SQL, risk calculations, or bioinfor- matics pipelines). Fine-tuning a small model on the subset of MedAgentGGy that matches the intended application typically yields higher task-speciff performance than the overall average suggests, and is more camputee-effciee than aiming for a fully generalist agent.
A.3 PRIVACY STATEMENTS
Table 6: Data Access and License Information of 12 datasets in MedAgentGym. "Custom" represents additional dataset- or task-specific license and data access requirements (e.g., DUA or credentials).
Dutuset
Data License
Duta Access
Code License
Code Access
Training Internal Volidation (Ie- (e-Dissribution)
MIMIC-II (Johnson et al., 2016; Lee et al., 2022)
Custom
MIMIC-III on PhysioNet
CC-8Y-400
MIMIC-III on EHRSQL
elCU (Pollard et al. 2018; Lec et ol., 2022)
Custom
eICU on PhysioNet
CC-BY-400
eICUU on EHRSQII
TREQS (Wang et al., 2020a)
Custom
MIMIC-II on PhysioNet
MIT
TREQS GitHuu
MedCaleBench (Khandekar al., 2024)
CC-BY.SA 4.0
MedCaleBench
Public
MedCaleBesch GitHub
Med'Agent Rench (Jing et al., 20258)
MIT
Med Bench (PHIR Server)
MIT
Bennh on GitHob
BinCader (Tang et al., 2024g)
CC-BY-4.0
BioC on Huggingface
NIA
BioCnder on GitHah
BieDSBench (Wang et al., 2024h)
MIT
BinDSBench
MIT
BiaD& on GitHab
EHRSHOT (Woroow ol., 2023g)
Custom
EHRShot (Standoord)
Apache
EHRSHOT on Gitbab
External Validation (Dirag--Daatiaiiiin
EHH-SSq8QLL (Ryu et al., 2024)
Custom
MIMIC-III on PhysioNet
N/A
HHR-SoySQL on GitHub
FHR-Con (Kwon et al., 2024)
Custom
MIMIC-III on PhysioNet
MIT
EHR -on on GilHub
MIMIC-Extnit (Wang al., 2020b)
Custom
MIMIC-III PhysioNet
MIT
MIMIC-Exttet GitHub
N.PowerAl (Ruan al., 2025)
N-Power Supp.
N/A
N-Power Al Webgage
Data Privacy and Licensing. We carefully curated MedAgentGym with strict adherence ethical standards, using publicly available datasets or datasets with appropriate privacy protections and anonymizations. Table lists the access requirements for the 12 datasets in MedAgen and the code base for data processing or task implementation. We explicitly designed isolated Docker environments to ensure data privacy and security. Nevertheless, ethical usage of our methods and models in clinical settings requires rigorous validation, transparency in limitations, and close collaboration with healthcare professionals. We encourage responsible deployment, emphasizing human oversight, continuous evaluation, and clear communication of model capabilities and uncertainties to mitigate ethical and practical risks.
LLM Usage Statement. In compliance with the PhysioNet Credentialed Health Data Use Agreement (version 1.5.0)", we strictly prohibit transferring confidential patient data (e.g.. MIMIC-III and eICU) third-party party entities, including external online services and APIs. To responsibly utilize the Azure OpenAl Service, we adhere closely to PhysioNet's guidelines on responsible GPT usage". Specifically, we have opted out of the human review process by completing the Azure OpenAl Additional Use Case Form', thereby ensuring no third-party entity accesses or processes sensitive patient information. We consistently monitor our data handling practices and strictly adhere to applicable guidelines and privacy regulations, maintaining the highest ethical standards in our research and operations.
to third-
B ADDITIONAL RELATED WORKS
Medical Reasoning Models. Recent advancements have substantially improved biomedical reasoning capabilities of LLMs through RL (Huang et al., 2025b; Lai et al., 2025; Zhang et al., 2025b; Jiang al., 2025a; Wu al., 2025a; Chen et al., 2024; Lan et al., 2025; Wang et al., 2025a; Li et al., 2025; Zhang et al., 2025b; Miao et al., 2025; Jin et al., 2025; Yu et al., 2025; Zhi et al., 2025; Liu
license-
20

---

Published as a conference paper at ICLR 2026
et al., 2025a). For example, M1 (Huang et al., 2025b) improves by distilling knowledge from the reasoning traces generated by DeepSeek-RI R1 (Guo et al., 2025). MedS3 (Jiang et al., 2025a) employs Monte Carlo Tree Search (MCTS) to generate rule-verifiable reasoning trajectories and employs process-reward models to select optimal reasoning paths during inference. Similarly, HuatuoGPT- ol (Chen et al., 2024) and ClinicalGPT-RI (Lan al., 2025) integrate domain-specific verifiers to guide RL fine-tuning processes for improved clinical reasoning. Extending beyond language modeling, Med-RI (Lai al., 2025) and MedXpertQA (Zuo al., 2025) adapt RL methodologies to vision-language models, effectively addressing medical visual question answering tasks. Despite these developments, current medical reasoning models predominantly target natural language-bar reasoning, with limited attention given to coding-intensive scenarios common in biomedical research and clinical practice.
Medical Reasoning Benchmarks. Most existing medical reasoning benchmarks focus primarily on evaluating LLM performance through closed-form medical QA tasks (Pal et al., 2022; Jin et al., 2021; 2019; Tsatsaronis et al., 2015; Tang al., 2025; Xiong al., 2024; Arora et al., 2025). In addition, AgentClinic (Schmidgall et al., 2024) further evaluates diagnosis prediction within simulated clinical scenarios, while MedHELM (HAI @ Stanford, 2025) provides comprehensive evaluations in various medical NLP tasks. Despite these extensive benchmarking efforts, existing benchmarks including recent concurrent works such MedAgentBoard (Zhu al., 2025), HealthBench (Arora et al., 2025), and MedCaseReasonine
(Wu et al., 2025b) typically focus on evaluation scenarios, with limited emphasis on dedicated training environments aimed systematically improving medical reasoning capabilities (Thapa et al., 2025), especially within coding- iodinn-intensive and interactive medical scenarios.
Medical Agents (Coding). Recent advances have demon- strated that LLMs exhibit strong capabilities in medical rea- soning and planning leveraging extensive biomedical knowl- edge (Singhal et al., 2023; Moor et al., 2023; Liévin et al., 2024), fueling increased interest in developing ..I.M-baeed autonomous agents tailored specifically for medical tasks (Jin ct al., 2024; Gao et al., 2025; Li et al., 2024a; Liao et al., 2024; Tang et al., 2024b; Kim et al., 2024). In particular, LLM-based agents have shown promise in specialized computational tasks, including querying EHR databases (Shi et al., 2024b), perform- ing bio-statistical calculations (Ruan et al., 2025), and conduct- ing bioinformatics analyses (Tang et al., 2024a; Wang al., 2024b; Tayebi Arasteh et al., 2024). As shown in Figure 8, integrating coding capabilities into LLM-based agents fur- ther enhances performance on tasks traditionally approached through natural language reasoning (e.g., MIMIC-III, eICU (Lee al., 2022)), as well as numerical and rule-based medical reasoning (e.g., MedCaleBench (Khandekar et al., 2024)). However, existing coding-based medical agents rely primarily on prompt engineering without systematic improvement, limiting their robustness and scalability when addressing complex and diverse coding tasks in real- world biomedical scenarios. In contrast, MedAgentGyn specifically targets reasoning-intensio coding tasks by introducing unified, scalable, and interactive training environment that systematically improves the coding-basee based medical reasoning capabilities of LLM agents.
Narrative Reasoning
0.6
Success Rake (SR)
Computational Reasoning
0.4
0.2
0.0
MIMIC-II
eICu
MedCalcBench
Figure Coding empowers com- putational medical reasoning (w/ gpt-4-turbo).
Medical Agent Training Environments. To advance medical agents with narrative reasoning, AgentClinic (Schmidgall al., 2024) and AgentHospital (Li et al., 2024b) simulate hospital workflows focused on diagnostic tasks, while MediQ (Li et al., 2024c) offers interactive simulations designed for medical information retrieval. Beyond medicine, specialized environments have emerged for systematically evaluating and improving LLM agents across diverse tasks (Zhao et al., 2025; Wang et al., 2025e), such as software engineering (Pan et al., 2025; Yang et al., 2024b; 2025). reasoning (Stojanovski al., 2025), web browsing (Drouin al., 2024). agent planning and collaboration (Xi et al., 2024; Shao et al., 2024a), data science (Guo et al., 2024; Jing et al., 2025; Zhang et al., 2025a; 2024), machine learning engineering (Nathani et al., 2025; Huang et al., 2023; Chan et al., 2024; Tang et al., 2023), automated research (Kang & Xiong, 2024; Schmidgall & Moor, 2025; Schmidgall et al., 2025), and scientific discovery (Team et al., 2025; Yuan et al., 2025). Inspired by these interactive training frameworks, MedAgentGym uniquely targets real-world biomedical scenarios, aiming to rigorously benchmark and systematically enhance coding- based biomedical reasoning capabilities of LLM agents. Unlike general coding agent benchmarks that primarily target software
21

---

Published as a conference paper at ICLR 2026
engineering tasks (Jimenez al., 2024; Yang et al., 2025), MedAgentGym emphasizes biomedical coding reasoning, requiring integration of clinical knowledge and domain--peccfic data formats (e.g.. EHRs, lab reports, biological sequences) within executable environments.
C TASK AND DATA DETAILS
C.I OVERVIEW
We refer task coding-based biomedical reasoning when LLM agents write and code whose execution yields a verifiable outcome biomedical data science. This definition allows us to objectively verify the results while preserving the steps that agents actually take, allowing for training and analysis at the trajectory level.
Biomedical Application Category. MedAgentGym spans multiple biomedical subdomains, including Database queries (DB, including MIMIC-III, eICU, TREQS, EHR- SeqSQL, and EHRCon), Data Analytics (DA, including MedCaleBench and MedAgentBench), Bioinformatics (Bioinfo, including BioCoder, BioDSBench, N-PowerAI), and Machine Learning (ML, including EHRSHOT and MIMIC-Extrach
Polnt of Care
9.9%
Rewsoning
Blostwt Analysis
2.2%
48.3%
Showledge
Figure 9 illustrates the diverse task distribution within MedAgentGym. Consider clinician identifying patients at risk for sepsis from EHR data, a task requiring not only understanding of sepsis criteria but also SQL queries extract relevant laboratory values, temporal logic to track patient trajectories, and statistical methods to validate find- ings. Similarly, researchers analyzing multi-omics data must integrate biological knowledge with bioinformaties
11.
Outcome
4.2%
Data Integrity
Figure 9: Diversity analysis.
algorithms and computational pipelines. These scenarios exemplify the core challenge of biomedical data science: operationalizing medical expertise through executable code, where domain knowledge alone proves insufficient without corresponding computational implementation.
In- cattof-Distribution. We further catego- rize tasks in MedAgentGym into in- and out-of- distribution, facilitating a rigorous evaluation of model generalization and adaptability. To highlight intrinsic differences between these dis- tributions, Figure 10(b) shows the distribution of sampled code trajectories. The resulting vi- sualization demonstrates significant divergence in trajectory complexity, interaction frequency, and required code refinement steps between in- distribution and out-of-distributiin tasks, under- scoring the challenges posed by novel biomed- ical reasoning contexts.
Gructure Ended
Disributiion
Cosine Similamt
Similarity
(a) Intra- Cate-Cy
(b) Inter- fier--Dstrrbuttin
Figure 10: Similarity analysis
Computational Task Category. Structured tasks primarily include database query scenarios, such as those from MIMIC-III, eICU, TREQS, EHR-SeqSQL, EHRCon, and MedCalcBench (rule- or equation-based) which require precise formulation of executable queries against structured EHR data. Open-ended tasks include biomedical data analysis and medical coding scenarios drawn from datasets such as MedAgentBench, BioCoder, BioDSBench, EHRSHOT, MIMIC-Extrace and N-PowerAI, demanding nuanced and flexible code generation for complex analysis, statistical reasoning, or clinical decision- decision-making making.
Specifically. evaluate LLMs across eight biomedical coding domains: (1) clinical database query- ing (MIMIC-III, eICU, TREQS, EHRsegSQL). (2) clinical note analysis (EHRcon), (3) medical computation (MedCalcBench), (4) health information technology gMedAgentBench), (5) biomedi-
22

---

Published as a conference paper at ICLR 2026
cal software engineering (Biocoder), (6) biomedical data analysis (BioDSBench), (7) biostatistics (NPowerAD), and (8) ML-based predictive modeling (EHRSHOT, MIMIC-Extract).
C.2 TRAINING AND INTERNAL TESTING (N-DDITRIIBTTON) DATASET DETAILS
EHRSQL: MIMIC-III and eICU. EHRSQL (Lee et al., 2022) comprises text-to-SCC Eext-to-SL tasks that leverage electronic health records from MIMIC-III (Johnson et al., 2016) and eICU (Pollard et al., 2018). They evaluate the ability of LLMs (and agents) translate clinical questions posed by healthcare professionals into executable SQL queries. This includes handling complex queries involving temporal logic and conditional abstention.
TREQS. TREQS (Wang et al., 2020a) is text-to-SQL benchmark tailored specifically to clinical question answering using the MIMIC-III dataset. emphasizes generating accurate SQL queries from template- based natural language questions against a simplified schema comprising five core tables, with an emphasis on large result-set handling.
Mecneehennch MedCaleBench (Khandekar et al., 2024) provides structured evaluation of clinical calculation capabilities LLMss. Each instance poses a patient-specific clinical scenario requiring precise medical calculations such as clinical scores medication dosages, accompanied by expert-curated stepwise solutions for validation.
Med Mednnnennn MedAgentBench (Jiang et al., 2025b) is a simulated EHR environment designed to evaluate LLM-driven clinical workflows. It features realistic patient scenarios across ten task categories, requiring agents perform clinical reasoning, EHR querying via FHIR interfaces, and clinical decision support.
BioCoder. BioCoder (Tang et al., 2024a) assesses the capability of LLMs to generate accurate bioinformatics code solutions. It comprises practical coding challenges derived from authentic bioinformatics software, requiring the generation and verification of functionally correct Python methods.
BioDSBench. BioDSBench (Wang et al., 2024b) evaluates LLM proficiency in biomedical data science coding tasks, involving the generation of Python or R code to replicate analytical workflows derived from actual biomedical research studies. Tasks span statistical analyses, data manipulations, and visualization routines.
EHRSHOT. EHRSHOT (Wornow et al., 2023a) benchmarks LLMs few-shot clinical prediction tasks leveraging real-world, longitudinal, deidentified EHR data. It focuses on rapid adaptation to tasks such risk prediction and forecasting clinical outcomes given limited labeled examples.
C.3 EXTERNAL EVALUATION (OUT-OF-DISTRIBUTT)
DATASET DETAILS
EHR-SeqSQL. EHR- SeqSQL (Ryu et al., 2024) extends text-to-SQL evaluation to sequential, multi- turn interactions, emulating realistic clinical dialogues. Tasks require maintaining context across multiple SQL queries, assessing LLM capability in handling compositional and contextual reasoning.
EHRCon. EHRCon (Kwon et al., 2024) involves assessing clinical note consistency with structured EHR records, focusing on identifying discrepancies. It serves as a verification task requiring precise alignment between unstructured clinical text and corresponding database entries.
MIMIC-Extract. MIMIC-Extract Wang et al., 2020b) provides structured, preprocessed time- series patient data derived from the MIMIC-III dataset, used in clinical predictive modeling such as mortality risk or intervention prediction, enabling standardized assessments of time-series reasoning capabilities.
N-PowerAL N-PowerAI (Ruan al., 2025) evaluates LLM capabilities in performing statistical sample-size and power analyses for clinical trial design. requires multi-step statistical reasoning and the generation of precise numeric results corresponding various clinical scenarios.
C.4 TRAIN- TEST SET SPLIT
For datasets that provide predefined training, validation, and test splits, we combine the training and validation subsets into single unified training set and retain the original test subset exclusively for
23

---

Published as a conference paper at ICLR 2026
evaluation. In cases where datasets lack predefined splits, we randomly allocate 50% of the instances to training, assigning the remaining 50% to the test set. For tasks containing more than 1000 samples in both training and test sets, create lighter subset through downsampling to support efficient leaderboord--he hased training and evaluation. Specifically, we leverage task-specific metadata to perform uniform sampling within each fine-grained category, thereby maintaining diversity, ensuring balanced representation, and preserving the original data distribution.
C.5 DATA PRE-PROCESSING DETAILS
Rather than simple concatenation of existing benchmarks, MedAgentGym transforms and unifies heterogeneous biomedical datasets into single executable, Docker-isolated environment with standardized JSON I/O, natural-language error grounding, and ieccutionverifiin tasks, together with multi-threaded trajectory sampling and an outeome-superrised verifier to support agentic RL training. Dataset-specific transformations are detailed as follows:
C.5.1 STRUCTURED TASKS
For database querying related datasets, including MIMIC-III, eICU, TREQS, and SHR-Se each task instance is structured into JSON format comprising: (1) the contextual description and the corresponding natural-language query, (2) the ground-truth SQL. query, and (3) the resulting answer from the database execution. Instances yielding null results upon SQL execution, indicating the absence valid answer, are excluded from the dataset.
For EHRCon, we organize the data into structured databases that link patient records through hospital admission IDs, complemented by a separate database containing associated clinical notes. Each task is formulated as a JSON object consisting of: (I) admission ID, (2) relevant medical terminology. (3) count of detected inconsistencies, and (4) a binary indicator denoting the presence or absence of inconsistencies.
For MedCaleBench, each instance initially consists a patient note, a specific medical calculation query, ground-truth answer, and a detailed step-by-step solution. To accurately evaluate the coding capabilities of LLM agents without direct guidance, we remove all intermediate calculation hints, presenting only the patient note and the calculation query for model inference.
For N-PowerAI, statistical analysis tasks are augmented through attribute substitution. Specifically, each original instance is expanded 100-fold by systematically replacing an attribute with a randomly chosen equivalent from a predefined valid range, preserving the integrity and interpretability of the statistical context. Each augmented instance includes recalculated values for sample size (N) and statistical power, stored systematically within ISON-formatter records.
C.5.2 OPEN-ENDED TASKS
MeenAgenBeench instances require LLM agents to follow natural-language instructions to perform tasks within a FHIR-compliant interactive medical environment. We retain original instructions, solutions, and Medical Record Numbers (MRNs). To derive verifiable evaluation signals, we execute the provided ground-trutt -trth on the server-side environment to obtain authoritative reference answers.
BioCoder tasks require implementing biostatistics algorithms or addressing scientific programming challenges. Each instance comprises problem description, context-specife code, test cases, and expected outputs. While evaluation datasets already contain all necessary components, training instances initially lack context-specific code and test cases. To address this gap, we employ the o3-mini model to auto-generate relevant context code and corresponding test cases based on provided ground-trune -ruth functions. Generated functions undergo rigorous validation via a code interpreter, retaining only verified, error-free instances. Additionally, exclusively utilize the Python- based subset of BioCoder, deferring the JavaScript subset for subsequent integration.
BioDSBench instances involve biomedical data analysis tasks derived from real-world datasets. Features are systematically organized into directories by task, with each task's 's description and reference Python implementation captured within JSON structures.
For datasets dedicated to predictive model development (e.g., EHRSHOT and MIMIC-Extracte initial features are provided in pre-processed form but necessitate additional table joining, filtering,
24

---

Published as a conference paper at ICLR 2026
and integration to produce final training inputs. While labels accompany these tasks, explicit reference Python implementations are not provided, as evaluation metrics directly measure the accuracy of model predictions predefined test subsets. Distinct subsets of training, validation, and testing data and labels are explicitly maintained and separately utilized for both training and evaluation phases.
C.6 SAMPLED TRAJECTORY DETAILS
Table details the proportion of action types (sec- tion 4.1) in trajectories. Structured tasks pre- dominantly involve data retrieval (over 50%) from databases or resources, complemented by coding and debugging steps. In contrast, open-ended tasks re- quire significant coding and debugging efforts due to diverse question types, often necessitating terminal in- teractions to install specialized biomedical packages. Although MedAgentGGm contains extensive training data and allows repeated sampling, the current tra- jectory count primarily reflects computational bud- get constraints. Specifically, Figure 4 (right) demon- strates consistent performance improvements with increasing training data volume, indicating that ex- panded trajectory sampling through additional com- putational resources would yield further gains.
Table Trajectory Composition (%).
Actions (->)
request info
terminal
debug
MIMICIII eICU
71.07
0
28.84
0.08
TREQS
64.27
0
35.54
0.19
MedCale.
0
74.91
25.09
Structured
51.88
0
41.61
6.52
MedAgent. RioCoder
0.29
96.11
3.60
BioDS, EHRSHOT
6.30
87.60
0.43
59.43
40.14
Open-ended
1.76
85.79
12.46
MedkgentGyn
32.71
0.14
57.11
10.04
D BASELINE DETAILS
We include additional details of the coding and medical domain-specific LLMs:
Owen2:5-Coder-Instrue (Hui et al., 2024) is derived from the Qwen2.5 series and further fine-tuned explicitly on large-scale coding datasets and coding-specific instruction sets. This targeted training substantially enhances their capabilities in code generation, debugging, and programmatic reasoning, outperforming seneral-purposs. models of similar scale on coding tasks.
(Seed et al., 2025) is an 8B- aB-parameter open-source coding LLM optimized for code generation, leveraging LongCChain-of-Thoggee (LongCoT) reinforcement learning improve multi-step code reasoning.
medgemma-44-it (gemma-3-4b-pp) (Google, 2025) is medical-domain variant based on gemma architecture and fine-tuned specifically on medical QA and instruction datasets, which provide strong capabilities for medical reasoning and question answering.
HuatuoGPT-o1-77 (Qwen2. 1--7--Instruch (Chen et al., 2024), built on the Qwen2.5-7B architecture, is extensively fine-tuned in clinical reasoning datasets via PPO with verifier-based rewards to enhance complex reasoning capabilities. Specifically, it incorporates medical-specific verifier model that guides the generation of complex reasoning trajectories. HuatuoGPT-ol-7B excels in medical reasoning tasks by explicitly generating intermediate reasoning steps that facilitate iterative refinement and introspective evaluation. We also evaluate HuatwoGPT-ol-7288 (Qwen2. 5-72B) to provide a more equitable and rigorous comparison with large-scale LLMSs.
m1-7B-23K (Qwen2.5-74) 5-7B-Instrruct (Huang et al., 2025b) is fine-tuned on approximately 23,000 rigorously curated medical QA examples, significantly enhancing its domain-specific knowledge and reasoning capabilities.
MedReason-SBB (Llama-3. 1--8--Instructt (Wu et al., 2025a) is fine-tuned for medical questions-answering and clinical reasoning tasks. Its training emphasizes the generation of step-by-step rationales, enabling robust performance on medical reasoning and diagnostic tasks.
-at--laaan---------------- (Wang et al., 2025a) is a 14B medical LLM pre- are-trained from scratch approximately 20 trillion tokens of medical domain-specifee content and high-qualing general text. integrates specialized modeling across over 20 medical specialties with advanced architectural modifications enhancing context understanding and long-sequence reasoning.
25

---

Published as a conference paper at ICLR 2026
Baichuan-M2-32B (Dou al., 2025) is 32B medical LLM mre-trained from seratch on large- scale medical corpora and high-quality general text, with architectural and training adaptations for multi-specialty clinical reasoning and long-context understanding. We use it as representative large medical-domain baseline.
E IMPLEMENTATION DETAILS
Evaluation Metrics. Following existing agent benchmarks (Liu et al., 2023), we adopt success rate (SR) the primary evaluation metric. For database, data science, and bioinformatics tasks with explicit ground truths, we compare LL.M-generteed code execution outputs with reference solutions using exact match. For open-ended MI. tasks in clinical decision support, we measure performance using accuracy (Acc) across provided test cases. Note that these code generation tasks inherently have infinite solution spaces, unlike traditional classification problems with bounded solution spaces (e.g., even random guessing can yield around 50% accuracy in binary classification). The overall score is computed by averaging performance across tasks in test sets of MedAgentGyyn (leaderboard), providing a comprehensive evaluation of coding- based biomedical reasoning capabilities within MedAgentGym.
Experimental Setup Details. We limit interactions to a maximum of 15 turns session, providing agents full access to interaction histories and constraining runtime to 120 seconds per session. Input tokens are capped at 32, 768, with output limited to 8, 192 tokens per round. We use Python 3.10 as the primary language for agent-code execution due to its modular design and suitability for biomedical computations. To enable interactive feedback (section 3.3), we employ rule-based parser converting LLM outputs to JSON, facilitating seamless code execution, and utilize gpt-4. 1-mini to translate execution eπrors into grounded explanations. We configure all baseline LLMs following established best practices for reprodueibility. Specifically, instruction-following LLMS are configured with temperature of zero, while reasoning models a temperature of 0.6. For all experiments with Qwen-3 series, we switch to thinking mode for optimal performance under complex reasoning scenarios (e.g., logic, math, and coding).
SFT. For SFT experiments, smaller models (up to 8B parameters) are trained using eight NVIDIA A100 GPUs, whereas the 14B- -ar-paraeteer model is trained on eight NVIDIA H200 GPUs. We utilize the Adam W optimizer (Loshchilov & Hutter, 2017) with learning rate of le 4. The training batch size is set to 8, and the maximum input token length per batch is configured to 40,000 tokens.
DPO. DPO experiments are conducted using the same hardware configurations as SFT experiments. We employ the AdamW optimizer with a reduced learning rate of 5e 6. Training utilizes a batch size of 64 and KL-divereence coefficient (3) of 0.1 to regulate the divergence from the initial policy,
PPO & GRPO. PPO and GRPO experiments are conducted using the same hardware configurations as SFT experiments. All online RL experiments are conducted using VeRL framework (Sheng et al., 2025). We integrate the VeRL package and dependencies inside the Med-Copi docker image to enable communication between the reward functions and the evaluation module. PPO and GRPO training is performed with a batch size of 128 and a lcarning rate 1 x 10-5, The temperature parameter during model rollout is consistently set to 0.6. Throughout training, the coefficient for the KL divergence regularization term is fixed at / = 1 x 10-3
F ADDITIONAL EXPERIMENTAL RESULTS
F.1
CODE QUALITY AND EFFICIENCY
For comprehensive evaluation, we further report additional evaluation metrics on code quality and efficiency, including (1) number of turns for interaction effectiveness, (2) cyclomatic complexity for code complexity, (3) maintainability index for code readability, and (4) line-of-code (loc) and (5) logical line-of-code (Iloc) for code efficiency (Table 8). Comparing different tasks (take gpt-4. for example), we observe that machine learning tasks such as EHRSHOT involve significantly higher complexity and longer code. Comparing different models (averaged across datasets), observe that advanced dlseddsouuree models generate more complex and longer code; after training, Med-Copilot produces structurally efficient and more maintainable code compared backhone models.
26

---

Published as conference paper at ICLR 2026
Table 8: Additional evaluation on code quality and efficiency for in- and out-of-distribution tasks.
Dataenta (-))
MIXIC.
&BCU
TREES
MatCClic
MedAgaal.
BieCidarr
BidD.
EHIISBOO
II Arg.
EaHR-Con
NIMIC-Eatrat
Nponer-Al 0K10 Arg.
(2025
Rume
20:65
00:33
22.06
8:71
19:31
16:97
05.14
65
56:63
88:30 5b
01.04
68. k0
03.67
56.34
85.5G
98.17
00:65
68.83
$6.03
1987
4.17
24.32
0/4
12:43
120:20
23.75
Aoeoe/ (3025-0--4)
19.00
16:35
21185
18.05
18:31
23.42
12200
12.40
cxceploity
0.00
D04
0.01
3.50
7.30
0.35
19:85
3.88
001
0.01
19.58
4.22
14.07
0.35
23337
17.08
23 16
773.90
32.58
20.30
12:88
334.35
5.05
7.22
318
17.5
2.50
30.75
15.40
119.58
23.05
13.07
3.8
115.0
4.1
28.35
Okeen? 5-76 Aecwie
Home
082
4.7
0.01
0.02
96.54
14.10
08.59
63.65
10.30
81.00
03.96
55.89
68.75
01.38
08.33
04.08
98.00
16:22
9.25
4.30
15
20:34
15.46
13.79
58158
22.5
16.75
0.25
26.05
Aamo
30.34
17.50
14.30
1.55
16.34
23.00
15.00
20.33
19.06
28.76
30.87
18.76
26.32
31.25
uuuploiit
0.01
001
0.91
3.90
000
5.8
064
18.55
3.45
001
0.86
18.42
000
3.1
21.38
19.58
12.00
25.42
53.00
34.70
13.06
141.0
39.30
32.14
14.38
12007
8.71
40.3
Ibc
9:00
5.5
17.00
26.36
1780
933
95.93
33.66
16.88
7.45
33045
F.2 ABLATION STUDY: EFFECT OF PRE-DEFINED TOOLSET
W/ Taot Set
Figure 11 compares the performance of GPT-4-based agents on the MIMIC-III dataset with and without predefined toolsets integrated into our agent scaffold. This illustrates our agent scaffold's ability to flexibly accommodate external tools. Interestingly, despite providing set of predefined tools, including functions for database loading, data filtering, value retrieval, arithmetic calculations, date com- putations, and SQL execution (see additional details of toolset Shi et al. (2024b)), observe a surprising decline in agent performance. It suggests that the LLM agent inherently generates more flexible and contextually appropriate code when uneneumbered by predefined function constraints, aligning with the observations reported by (Qian et al., 2025; Qiu et al., 2025).
0.3
Success
0.2
MIMIC-II
Figure 11: Ef- fect of toolset.
F.3
COST ANALYSIS
Table summarizes input and output token statistics for various API-based proprietary LLMs evaluated on datasets within MedAgentGym. Notably, the input and output token lengths per query vary significantly across models and tasks. Among these models, gpt-4. 1-mini achieves rela- tively low average input and moderate output token counts, which implies more efficient token utilization during in- ference compared to larger variants such as got-4o and spt-o4-mini. Conversely, o4--o4- incurs higher average input costs. Figure 12 presents the API cost per 100 tasks. Overall, smaller GPT variants (e.g., gpt-4. 1-mini and gpt-4o-mini) offer superior token-efficiency, translating into lower computational and API costs without substantial compromise in performance, demonstrating their effectiveness cost-efficient solutions for large-scale biomedical reasoning applications.
input output
100 baske
2.5
2.0
1.5
1.0
0.5
0.0
gpt-4o-mini 9pt-4o gpt-4.1-mini gpt-41
64-mini
Figure 12: Cost information.
F.4 STRUCTURED AND OPEN-ENDED TASKS
+5.8877%
SPT
60
+22L
4202.36
(a) Structured Tasks
(b) Open-Ended Tasks
Overall
Figure 13: Med-Copilot SFT performance MedAgentGGm across various backbone LLMss.
27

---

Published as a conference paper at ICLR 2026
Table 9: Statistics of input and output tokens per question for API-based commercial LLMs.
Datasets (-)
MIMIC.
eCCU
TREQS
MedCale.
MedAgent.
BioCoder
BioDS.
EHRSHOT
Avg.
Jeput
not ninnin (Husst ot 2024)
3430,83
1947.72
1689.71
651,92
9501 86
5166.50
5068.85
5986.20
4180.45
gpt gpt-4o (Hurst
2024)
3122.02
5133.71
21077.12
6000.75
2pt-4.1 1inini (OpenAL, 2025a)
1869.37
1691.45
1430.15
834.73
8087.50 550
2621.79
7369.35
4466.07
3546630
spt-4.1 (OpenAA, 2025a)) 2025a)
3730.90
2979.57
1754.15
759.64
7912.81
2728.24
3035.45
2092. 14
3124.12
xpt-oé-nini (OpenAl. 20256)
2005.11
1688.73
1534.84
1306.49
7586.32
2193.82
50768.08
2858.79
8742.77
Ougut
not-4o-ninl (Hurst ct 2024)
1206.00
714.72
918.45
379.25 28
4206:73
4170.56
1479.87
10484.53
2945 012
got (Hurst 2024)
840.16
852.41
696.61
537.00
4144.91
7278.49
9127.11
3287.23
got-4. I-mini (OpenAL, 2025a)
952.68
991.78
2892.98
3328.07
13008 73
23276.67
4328.93
gpt-4.1 (OpenAl, 2025a)
771.91
781.86
751.88 88
787.45
2051.20
2846.58
1627.78
5163.57
1848.003
oo nini (OpenAl, 20256)
1586.65
1392.11
893.76
2407.87
1718.22
3144.74
1952.88
8063.71
26-17.49
Figure 13 shows substantial performance gains from SFT across four OSS backbone LLMS of varying
sizes. Simple SFT on successful trajectories markedly hoosts performance on structured coding tasks, indicating its effectiveness in capturing structured coding patterns. DPO, in contrast, particularly effective for optimizing performance on open-ended tasks.
F.5 ABLATION STUDY: EFFECT OF WARM-UP STAGE
Table 10: Effect of SFT stage in two-stage finetuning framework.
Datasets (-+)
MIMIC-III
eICU SR
TREQS SR
MedCalc. SR
MedAgent. SR
BioCoder SR
BioDS. SR
EHRSHOT Acc
Arg.
Base (1)/ Motrics
SR
Secre
Qwen2. I--tt-nntttttt
13.08
15.57
12.76
25.91
30.36
21.79
10.20
5.42
16,89
+DPO
(+28.15)
+DPO
64.13
66.91
72.02
90.06
$2.54
34.62
69.39
29.55
59.90
(+43.02)
Qwen2
17.21
14:07
16.43
27.40
35.59
29.49
16.33
4.45
20.12 12
+DPO w/a SPT
57.49
59.18
70.45
71.32
47.46
42.95
91.84
41.33
60.25
(+40. 13)
76.08
(+46.25)
Table 10 shows the effect of the initial SFT stage during agentic RL finetuming. Although DPO alone slightly underperforms compared to SFT, combining an initial SFT warm-up with subsequent DPO further improves overall results by leveraging their complementary strengths.
F.6
CASE STUDY
To illustrate the practical utility of interactive coding mechanism, conduct detailed case study involving typical bioinformatics coding task in Figure 14. Specifically, the task requires writing Python function @eddd__xchnge_____ rxns) that modifies biochemical reaction graphs by integrating exchange reactions. Initially, the LLM agent-generated solution encountered an attribute error, mistakenly invoking non-existent text_ type method Graph object. Upon receiving explicit debugging feedback, the LLM agent effectively identified and corrected the mistake by utilizing the standalone text_type function rather than incorrectly calling as method of the graph instance. This case highlights the capability of debugging in MedAgentGym environment to provide targeted, actionable debugging feedback, enabling iterative code refinement and significantly enhancing agent- generated solutions for complex biomedical programming tasks. Case studies with code patterns in Figures 15 to 17 further illustrate how baseline models frequently produce syntactically valid code but incorrect solution in biomedical tasks, from hardcoding biological parameters to misapplying medical formulas, while our fine- tuned models demonstrate accurate implementation of domain-specinf constraints and current clinical standards.
F.7 DIFFICULTY ANALYSIS ON EXTERNAL EVALUATION SET
MedAgentGym includes four challenging unseen out-of-distribution medical coding tasks as external validation sets in section 5.4. For example, the original MIMIC-Extrace task in MedAgentGym intentionally utilizes unprocessed data challenging. nut-of-distribution scenario designed specifically to assess model capabilities in feature engineering and data preprocessing. To illustrate the difference clearly, additionally evaluate fully pre-processed version of MIMIC-Extract.
28

---

Published as a conference paper at ICLR 2026
hiomedicaa expert in writing bioinformatics Your objective is wrile a pylhon function to solve the given question.
anvover questions accondnggy.
Please only waite the function, do not indlud any other text.
Please write Python function with the following signature: exchange, nanalg ron, reaction, style dict)
def add exchange rxns(o, rxn.id, reaction, style.flux.dde Flux.dict): ACTIVE_COLOR for in reuction. cocction.orpoundsy type(c)
g.nodes diet:
Tmll
Validate Code
Traceback fmost recent call last?:
exchaange_ssn
node plest [lype(c)
AttributeError 'Graph' object has no attribute "text_ type'
Python debuggging expert.
to debug code.
has provided following code enron:
[code] (enron: nsg)
Debug
The enron occurs becouse you're altempting to call text, type nethod on the Graph object Mitch doesn'tt exist, Strply use the standalone text. type function instead:
add wwwhhngeeennaeeeec
man_id, reaction, style_flux.co
ACTIVE COLOR
reaction. compounds: Assuning text. type
reacction.coooound:::
function, not
Validate Code
node_1d texe__)
Figure 14: Case study of gpt-4. 4pt-4.11ming 1-mini on BioCoder.
Question: You biomedical expert. Write Python script that constructs 'CNrrray' object with chromosomes 'IT,, 'X, 'Y, 2, ', "Y', 33,", uses "loidy=2", and calls the aassolute eeppect" function to compute expected copy numbers for both female and male samples.
Qwen-2.5- -In-Inst
DPO-14B
def absolute_ extpbuteeexppc((cnnr, ploidy, fs_sal) oc (p..ful(lle((earr))) float(ploidy))
def absolute_ absolute_eppect((() (eppectccnaarr, plotdy, 1s_ fe expected, coppos for chron in cnarr.chromosome:
x_chron cnarn._chr, chr_x_iabel
y s_chron cnarr._ch.. chr_y_label
cnarr._chr:
if is_sample_ferran ferrlee::
if is_sample s_.aanl_femalees
Hardcodes fenale
FIK:
ploidy fenale
ec| chromosone
chanr.ccrrmmoooeee == x_chron] =
expected_copiee wwwpcc...ppend((a.......
LeLeaarcchromossee
y_chron]
else:
else:
expected supacted_copies..o copies.append(()
ec ec[cnarr.chree chrromsereee
ccroon]
elif chron smarr__chr_y label: Labely_labell expected coplcc..pppppooooocco append(e if
ec[cnarr. chronosone ciannrchhromosoeee return ec
== y_chron]
is_sample_fent fenale else
expected cwppccppppppppeceeee appeiddlopp np. apraayyyxxeectee copies)
The expected
[_crro]]
wrongly assumes females always
have two X chromosomes, which only holds for diploid samples (ploidy 2). In higher-ploidy cases (e.g.. tetraploid tumors), females should have more copies (e-g, four).
The Correction: The revised code scales chromosome copies with overall ploidy: åf is sample sample_femlees females
expected capies. append(ploidy) This ensures expected copy number matches the biological reality in cases like whole-genome duplication.
Figure 15: Domain-specific code generation error in a biomedical task from BioCoder (Tang et al., 2024a). The task requires implementing a Python function compute chromosome copy numbers based on ploidy. The baseline model (Qwen-2.5- 2..---5---B...... 4B-Instruc,, left) incorrectly hardcodes the female chromosome count to 2, failing to account for non-diploid scenarios such as tetraploid tumor cells. Our DPO-trained model (DPO- 14B, right) correctly implements dynamic scaling of X chromosome copy numbers proportional to the ploidy parameter, demonstrating improved understanding domain- specific biological constraints.
As demonstrated in Table 11, providing structured data significantly improves model performance, highlighting the distinct difficulty posed by raw data.
To further demonstrate the generalization of Med-Copilot, include additional evaluation BixBench (Mitchener et al., 2025), a bioinformatics coding dataset comprising over 50 real-world scenarios of practical biological data analysis with nearly 300 associated open-answer questions. It is
29

---

Published as a conference paper at ICLR 2026
Question: You have simple metabolic network represented by ReactionDatabase, which holds list of reaction IDs and a stoichiometric matrix (mapping each (Compound, reaction id) to its stoichiometric coefficient). Using the provided function check reaction consistency, identify any mass-imbalanced reactions by minimizing the L1 norm of the mass residuals, and also compute a valid integer mass assignment (21) for each compound. Test this minimal example where reaction RI converts compound A to
The main task is to write Python function with the following signature: def check wwwaaeeeanconeeeee consaistenccydaaaaaee,, solver, exchange, checked, zeromass, weights)
DPO-148
MedCopilot-148 (GRPO)
chack ponectoooeenitteeiiiaaaae scleer, sctarpessti,, checoed- sett),
cer deecc iatpiinneerttttaa//aaaaaaa lasttaas,, salver, eac ascugennett), chockeds
weiptt-()):
r**Dwck Uncesistent widerss
residnals
5 Raturt reaction iterable, and corpart deraale. The reaction Je states rectier lsh raxs recionals. Tw carpornt Itemale ylalds
Recam rection iteratte, and coppand steruble, The reacties Atenabiz yoalds reaction ite mas meidals, coppoid steraole plalds
9. kath concent agseee miss at least wy and the masies
tick comport is assigedd a moss lesst cet, the masses are be belancee wurs the ateicttentt ze notrokk En ceauiten, each reactase tea ressdel mas sxcluded in bolanse equetians, The Li-err residaal
resinual 19
exlonced saie the stauct pecoric satris Ir adsttion, exch routtin hop
33
includer maxe coutionns Li-nor
12
saailly checked therfoore
rusidast food
meaally checked therefore
14
14
15
rob www.rrroor. (mm))
15. prik calver.cmeat, graslent]
keystt))
prct. www.oonccpccccaaoooo cowpuuusp, lavera))
19
prob. ranopecoleeaaaes reacticns, lovers:
11
prct on derazaee ractiion, lovemb)
24 prtt www.uucaaaaaaaacooo dew.tara reaction))
eijectio Exanttraaessaa
dbjective cnnerspett
TEr reaction BETANIITTOOTCOOO
oojective in zEROECtio 201
cuset (detchern reac: 3870
costiietatesss reacties!
prob and, 15nerr www.ratrtssia treccttrrrrttt .41 inmacttonndd for restiie Id
prch art. Lireer www.seaarmmmtinnnnnnn....... reltmactee, Le) for reaction,
-axtnectioo__dd
preb.arr_ www.rrrcnntraineeerraasaco
rectimL avtakase (eac)1an))
reactieRiE in catause. rracticss
rastelacce froactioniis reaction. catatese, resctiess
(d0 Eneersaed reaction.id
Invalid Objective Function:
The incorrect code attempts to set the optimization objective using Python list, which is not a valid mathematical expression for the solver. The correct code properly constructs LinearExpr object, which correctly represents the mathematical function to be minimized.
Improper Initialization: The incorrect code initializes the mass balance equations with integer The correct approach initialize them with empty LinearExpr() objects, ensuring type consistency and making subsequent mathematical operations clear and bug-free.
Figure 16: Qualitative comparison of code generation for complex optimization task from BioDS- Bench (Wang et al., 2024b). The task requires implementing a linear program to verify mass conservation in metabolic networks. The baseline model (DPÓ-14B, left) generates syntactically plausible but semantically incorrect code with two critical errors: (1) defining the optimization objective using a Python list rather than the required LinearExpr object. and (2) initializing mass balance equationn with integer instead of LinearExpr(). In contrast, Hed-Copilot-144 (GRPO, right) correctly employs the LinearExpr class for both objective function construction and mass balance initialization, producing executable code that accurately models the metabolic constraints.
Table 11: Difficulty analysis on external sets for model generalization.
Model
MIMIC-Extract (raw)
MIMIC-Extract (processed)
BixBench
gpt-4,1-min
5.62
23.47
26.01
gpt-4.1
10.41
28.9 99
32.09
Qwven-2.5-7B-Instree.
1.34
17.06
18.92
Med-Copilot (7B)
2.14
25.88
28.72
Qwen-2.5.14BB- Inn-srruttt
4.51
18.5
20.61
Med-Copilot (14B)
2.75
28.66
29.39
designed to measure the ability of --adeddd agents to explore biological datasets, perform long, multi-step analytical trajectories, and interpret the nuanced results of those analyses. Exceptional performance in BixBench demonstrates the robustness of Med-Copilot and its ability to generalize beyond the specific domain of medical coding to broader scientific analytical tasks.
30

---

Published as a conference paper at ICLR 2026
Question: <Paatient Information
Using the 2021 CKD-EPI Creatinine equation, what is the patient's Glomerular Filtration Rate (GFR) terms mL//min/1.33 Answer with decimal number without unit and with relative precision 0.0001.
Qwen-2.5- 35-148-Innstuct
MedCopilot-14B
def celculate_gfr(cti gff(creatinine, age, sex):
def calculate, @fr(creatinine, age, sex):
Patient fron the sce -loat(creatinnie))
float(creatindne)
k, alpha 0.8, 8.8 if sex.lower() 'female';
age gee_val_ int(age)
gfr - 175 (ser -1.154) (age_val -8.283)
0.7
if sex.lover() 'fenale";
alpha -8.241
gfr gfr 6.742
elif sex, lower() =s 'nale":
return
k 8.9
alpha -0.382
ser ser_over__ scr
nin term (nin(scr_ serer_ove__e k, 1.0)) alpha
nax, nax_term term (nax(scr over__ 1.0)) -1.
age 0.9938 xx age_val
gfr 142 nin tor_ nax torm ago ago_tern tern
return gfr
Flawed Solution: Incorrectly implements the older and accurate MDRD (Modification Diet Renal Disease) equation. This formula uses a single, continuous calculation.
Correct Solution: Properly implements the required 2021 CKD-EPI (Chronic Kidney Disease Epidemiology Collaboration) equation. This more modem accurate formula uses complex, conditional logic, changing the calculation based the patient's threshold.
whether their creatinine level is above below specific
Figure 17: Domain-specific complexity medical code generation from MedCalcBench (Khandekar al., 2024). The task requires implementing the 2021 CKD-EPI equation for Glomerular Filtration Rate (GFR) calculation. The baseline model (05een2..5144, left) incorrectly generates a flawed implementation of the outdated MDRD formula instead of the requested 2021 standard. In contrast, Med-Copilot-lee 14B (right) accurately implements the complex conditional logic specified the 2021 CKD-EPI guidelines, demonstrating precise adherence to current medical standards.
Table 12: Human evaluation on structured and open open-ended tasks from MedAgentGym.
Dataset (L)
Attempt
Correct
SR
Total Time (min)
Avg Time (min)
Structured
MIMIC-II (Johnson
2016; al., 2022)
eICU (Pollard et ul., 2018; Lee ul., 2022)
8
5
63%
63
7,88
TREQS (Wang et al., 2020s)
10
70%
39
3,90
EHR-ScqSOL (Ryu
2024)
10
80%
6.70
MedCaleBench (Khandekar al., 2024)
7
71%
57
8.14
N-PowerAI (Ruan et al., 2025)
86%
13.7
Structured Task (Total)
S2
39
75%
396
7.62
Open-ended
MedA BentBech (Jiang
2025h)
6
100%
89
14,833
EHRCon (Kwon et al., 2024)
6
17%
241
40.17
BioDSBench (Wang al., 2024b)
3
0%
65.00
BioCoder (Tang et al., 2024a)
8
2
25%
142
17.75
EHRSHOT (Wornow et al,, 2023a)
5
89%.
185
37.00
MIMIC-Extrade (Wang al., 2020h)
94%
Open-ended Task (Total)
31
45%
1067
34.419
F.8 HUMAN STUDY
To systematically compare coding styles and performance differences between human programmers and automated agents, we conducted a human evaluation involving 83 tasks randomly selected from the test subsets of the datasets included in MedAgentGym. This evaluation set comprises 52 struc- tured and 31 open-ended biomedical coding tasks. The human participants biomedical engineers and research scientists with over six years of experience in computational biology, relational database querying. WTTTP-basel interactions, and machine learning development. The human evaluation study was conducted under the approval the Institutional Review Board (IRB). Participants voluntarily contributed to the evaluation and did not receive monetary compensation.
Table summarizes the results of human evaluation study conducted to establish reference per- formance benchmarks across representative structured and open-ended biomedical reasoning tasks
31

---

Published as a conference paper at ICLR 2026
from the MedAgentGym benchmark. Human experts completed selected instances from each dataset, documenting the number of attempts, correctly solved instances, overall SR, total time spent, and average time per task (in minutes). Results indicate that, on average, the human subject required approximately 4.5 times longer to solve open-ended tasks relative to structured tasks, while achieving 40% lower success rate, reflecting the increased complexity and cognitive load associated with open-ended biomedical reasoning scenarios. Given that neither current LLMs nor human experts achieve perfect accuracy, we recommend deploying these models strictly in copilot role rather than as fully autonomous agents. The interactive features (e.g., debugging. execution-bare based verification) of MedAgentGym also supports such human oversight, enabling human experts to validate code execution outputs rather than manually writing code from scratch.
F.9
EFFECT OF REWARD DESIGN
Table 13: Effect reward design.
Datasets (-+)
MIMIC
eCCU
TREQS
MedCale.
MedAgent.
BloCoder
BioDS.
EHRSHOT
Avg.
GRPO
68.21
68.73
70.50
92.33
55.87
37.40
71.11
33.18
62.17
GRPO w/ penalty
63.68
65.08
64.82
84.13
$2.54
33.33
65.31
26.97
56.98
Motivated by the predominance of loop-related failures in our error analysis (Figure 7), we experi- mented with an additional shaping term, assigning a penalty term (Fpenalty -0.3) whenever the agent produced highly repetitive code blocks (cosine similarity 0.9 between consecutive genera- tions). As shown in Table 13, this heuristic consistently degraded performance across all datasets, reducing the average score from 62.17% 56.98%. Manual inspection of trajectories indicates that the penalty suppresses benign self-debugging and iterative refinement (e.g., small edits to a prior code block), causing the agent terminate early or switch strategies prematurely rather than repairing its own code. We therefore treat such shaping hooks (e.g., penalties on repeated actions) as optional, implementation-leve choices for users who desire more granular rewards, and not as prerequisites for the gains reported in main results.
32

---

Published as a conference paper at ICLR 2026
G PROMPT DETAILS
G.1 MIMIC-I11 PROMPTS
We include prompt details for MIMIC-III tasks as follows:
MIMIC-III Prompt
You are a biomedical expert in handling EHR data and answer questions. Your objective is to solve coding problem with given EHR data, with the goal of finally give concrete answer the question. several tables:
Assume you have knowledge (1) Tables are linked by identifiers which usually have the suffix 'ID For example, SUBJECT_ID refers to a unique patient, HADM_ID refers to unique admission to the hospital, and ICUSTAY_ID refers unique admission to an intensive (2) Charted events such as notes, laboratory tests, are stored in a series of 'events tables. For example the outputevents table contains all measurements related to output for given patient, while the labevents table contains laboratory test dictionary tables and provide
unit.
fluid balance
(3) Tables prefixed with 'd_' definitions for identifiers. For example, every row of chartevents is associated with a single ITEMID which represents the concept measured, but it does not contain the actual name of the measurement. By joining chartevents and d_items possible identify the concept represented by given ITEMID. (4) For the databases, four of them are used to define and track patient stays: admissions, patients, icustays, and transfers. Another four tables are dictionaries for cross-referencing codes against their respective definitions: diicd__iagnosss, d_icd_procedurrs, d_items,
ITEMID, it is
d_labitems.
For different tables, they contain the following information:
(1) ADMISSIONS. csv: ROW_ID, SUBJECT_ID. HADM_ID, ADMITTIME, DISCHTIME, ADMISSION_TYPS ADMISSION__LOATION DISCHARGE INSURANCE, LANGUAGE, MARITAL_STATUS, ETHNICITY, AGE
(2) CHARTEVENTS.COM csV: ROW_ID, SUBJECT_ID, HADM_ID, ICUSTAY_ID, ITEMID, CHARTTIME, VALUENUM, VALUEUOM
(3) COST. csv: ROW_ID. SUBJECT_ID, HADM_ID, EVENT_ EVENT_ID, CHARGETIME, COST
(4) D_ICD_DE SEODDDIAANOSSS... csV: ROW_ID, (CDE_CODE, TITLE, LONG LONG_TITLE TITLE
(5) BIIID_PROCEDURES csv: ROW_ID, ICD9_CODE, SHORT_TITLE, LONG_
(6) D_ITEMS. CSV: ROW_ID ID. ITEMID. LABEL LINKSTO
(7) D_LABITEMS CSV: ROW_ID, ITEMID LABEL
(8) DIAGNOSES ICD. CSV: ROW SUBJECT HADM ICD9_CODE
(9) ICUSTAYS CSV: ROW_ID, SUBJECT ID, HADM_ID, ICUSTAY_ ID.
LAST_CAREUNIT HIRDI_W LAST_ WARDID, INTIME (10) INPUTEVENTS CVV
ROW_ID, SUBJECT_ID. HADM_ID, ICUSTAY_ID,
CHARTTIME, ITEMID, AMOUNT
(11) LABEVENTS. CSV: ROW_ID, SUBJECT_ID, HADM_ID, ITEMID, CHARTTIME,
VALUENUM, VALUEUOM
(12) MICROBIOLOGYEVENNTS
RROW_ID SUBJECT_ID. HADM_ID, CHARTTIME.
SPEC_ TYPE DESC, ORG_NAME NAME
(13) OUTPUTEVENTS. CSV: ID, SUBJECT_ID, HADM_ID, ICUSTAY_ID,
CHARTTIME, ITEMID, VALUE
(14) PATIENTS. CSV: ROW_ID, SUBJECT_ID GENDER, DOB DOD
(15) PRESCRIPTIONS, CSV: RDW_ID. SUBJECT_ID HADM_ID, STARTDATE, ENDDATE DRUG, DOSE_VAL_RY DOSE UNI LNIT_RR RX, ROUTE
DOSE_UNIT_E
(16) PROCEDURES csv: ROW_ID, SUBJECT_ID, ID, ICDE_ CHARTTIME
(17) TRANSFERS. CSV: ROW_ID, SUBJECT_ID. HADM_ID, ICUSTAY_ID, EVENTTYPE, CAREUNIT, WARDID INTIME, OUTTIME
All the tabls saved in the data directory 0.
33

---

Published as a conference paper at ICLR 2026
G.2 EICU PROMPTS
We include prompt details for eICU tasks follows:
eICU Prompt Main
You biomedical expert in handling EHR data and answer questions. solve coding problem with given EHR data, with the goal of finally give a concrete answer to the question. Assume you have knowledge of several tables:
Your objective
(1) Tables are linked
identifiers whose name usually ends 'ID'. For example, PATIENTUNITSTAVID refers a unique patient, LABID refers a unique lab test, ALLERGYID refers to unique incidence of allergy occurence.
(2) Four tables are related to measurements. First, the lab table contains laboratory measurements of chemicals such chloride albumin. Secondly,
intake and output (intakeoutput) table records all fluid-related measurements such as administered normal saline (ns) and urination. Thirdly, the microlab table records measurements of culture of microorganisms. Fourth, the vitalperiod table describes the patients' vitals during their stay.
(3) The remaining tables (allergy, cost, diagnosis, medication, patient and treatment) contain other critical information, and the table names are self self-explanatory.
(EHR_tables}
eICU Prompt- Table Information
For different tables, they contain the following information:
(1) allergy. csv: csv: ALLERGYID, PATIENTUNITSTAYID, DRUGNAME, ALLERGYNAME, ALLERGYTIME
(2) cost. csv: COSTID, UNIQUEPID, PATIENTHEAL THIASSEETAAT EVENTTYPE, EVENTID, CHARGETIME, COST
(3) diagnosis. csv: DIAGNOSISID,
PATIENTUNITSTAYID, ICD9CODE, DIAGNOSISNAME, DIAGNOSISTIME
(4) intakeoutput.com csv: INTAKEOUTPUTID, PATIENTUNITSTAVID, CELLPATH,
CELLLABEL, CELLVALUENUMERIC, INTAKEOUTPUTTIME
(5) lab. LABID, PATIENTUNITSTAYID, LABNAME, LABRESULT,
LABRESULTTIME
(6) medication. CSV: MEDICATIONID, PATIENTUNITSTAYID DRUGNAME, DOSAGE,
ROUTEADMIN, DRUGSTARTTIME, DRUGSTOPTIME
(7) microlab.
MICROLABID, PATIENTUNITSTAYD. CUL CULTURESITE ORGANISM, CULTURETAKENTIME
(8) patient. csV: PATIENTUNITSTAYID. AATEENTHEALTHSYSTEMSTTTTTT
GENDER,
AGE ETHNICITY, HOSPITALID, WARDID ADMISSIONHEIGHT.
NOSPITALADMITSOUREE DOSPITALDISCHARGESTTTU ADMISSIONWEIGHT,
DISCHARGEWEIGHT, UNIQUEPID, HOSPITALADMITTIME, UNITADMITTIME,
UNITDISCHARGETIME HOSPITALDISSHARRETIIE
(9) treatment. TREATMENTID, PATIENTUNITSTAYID, TREATMENTNAME,
TREATMENTTIME
(10) vitalperiod. VITALPERIODICID. PATIENTUNITSTAYID. TEMPERATURE,
SA02, HEARTRATE, RESPIRATION, SYSTEMICSYSTOLIC, SYSTEMICOIASTOLTC. SYSTEMICMEAN, OBSERVATIONTIME
All the tabls are saved in the data directory (data_directory).
G.3 TREQS PROMPTS
We include prompt details for TREQS tasks as follows:
34

---

Published as a conference paper at ICLR 2026
TREQS Prompt
You are an biomedical expert in handling EHR data and answer questions accordingly.
Your objective
solve coding problem with given EHR data, with the goal of finally give a concrete answer to the question.
Assume you have knowledge of several tables:
(1) Tables are linked
identifiers which usually have the suffix 'ID '. For example, SUBJECT_ID refers to unique patient. HADM_ID refers to unique admission to the hospital, and ICUSTAY_ID refers to a unique admission to an intensive care unit.
(2) All tables contain SUBJECT_ID (patient identifier) and HADM_ID ( hospital admission identifier).
(3) The table names are self-explanatory.
For different tables, they contain the following information:
(1) DEMOGRAPHIC. csv: SUBJECT_ID. HADM_ID, NAME, STATUSS, AGE, DOB, GENDER, LANGUAGE RELIGION, ADMISSION_ DAYS_STAY, INSURANCE, ETHNICITY, FLAG, ADMISSION_LOCATION DISCHARGE_LOCATTONN DIAGNOSIS, DOD, DOB_YEAR, DOD_ YEAR, ADMITTIME, DISCHTIME, ADMITYEAR
(2) DIAGNOSES. csv: SUBJECT_ID, HADM_ID, ICD9_CODE, SHORT_TITLE,
LONG_TITLE
(3)
L SUBJECT_ID, HADM_ID, ITEMID, CHARTTIME, FLAG, VALUE_UNIT,
LABEL, FLUID, CATEGORY
(4) PRESCRIPTIONS.
SUBJECT_ID. HADM_ID, ICUSTAY_ID, DRUG_TYPE, DRUG FORMULARY_DRUGG ROUTE, DRUG_DOSE
(5) PROCEDURES, cSV: SUBJECT_ID, HADM_ID, ICD9_CODE, SHORT_TITLE,
LONG_TITLE
All the tabls saved in the data directory (data_direcctor)).
G.4
MEDCALCBENCH PROMPTS
We include prompt details for MedCalcBench tasks as follows:
MedCalcBench Prompt
You work in hospital, and common task your work is to calculate some biological values of your patients.
To do this, you need to identify from clinical notes what information
is relevant, before using your clinical knowledge to calculate.
And then write Python code to calculate the value.
the code, please use the variable 'answer' to store the answer of the code.
In the main function, please print the final answer of the code without any other text.
G.5 MEDAGENTBENCH PROMPTS
We include prompt details for MedAgentBench tasks as follows:
MedAgentBench Prompt Part
You are an expert using FHIR functions assist medical
professionals.
In FHIR, there are few common HTTP GET or POST requests to interact with the server. The descriptions of requests are listed here: wwr.rfuuntiooddecrpptiiiii
35

---

Published as a conference paper at ICLR 2026
MedAgentBench Prompt Part II
You are given question and set of possible functions. Based the question, you will need write python code achieve the purpose.
1. Write a python script to invoke a GET function of the FHIR server, aaarraanammeeeeprrra
MUST put it in the format ofNGGET ur1?param_namel=
2. Write python script server, you MUST put it the format ofPnOST unourr payload data in JSON format]
invoke POST function of the FHIR
3. If you have got answers for all the questions and finished all the requested tasks, you MUST format of {answer_format) (make sure the list JSON loadable ))
the final answers in the
You SHOULD NOT include any other text in the response. Please write the python code and the variable 'answer' to store the answer the code.
Question: {question]\n. The FHIR server base URL is (fhir_api_base). Do not directly write the GET and POST requests.
MedAgentBench Prompt - Answer Format
answer ("GET": ["60" "S2874899"J, "POST": /hir/Observation" "payload]}
8088/
The answers to the questions are listed in "GET" instead of the get commands, while the post url and payload listed in "POST"
G.6
BIOCODER PROMPTS
We include prompt details for Biocoder tasks follows:
Biocoder Prompt
You are an biomedical expert in writing bioinformatics code and answer questions accordingly. Your objective
write python function solve the given Please only write the function, do not include any other text.
question.
Please write Python function with the following signature: (signature}
G.7 BIODSBENCH PROMPTS
We include prompt details for BioDSBench tasks as follows:
BioDSBench Prompt
You biomedical expert in writing bioinformatics code and questions accordingly.
Your objective is to write a python code to solve the given question.
Please only write the code, do not include any other text. All the required data are stored in the directory: {dataset_path)
36

---

for Published as a conference paper at ICLR 2026
G.8 EHRSHOT PROMPTS
We include prompt details for EHRShot tasks as follows:
BioDSBench Prompt Main
You are an biomedical expert writing machine learning code to solve EHR-relevane tasks.
Your objective is to solve machine learning task based on the given
data, with the goal of maximizing the performance of the model in limited steps.
You must use Machine Learning /Depp Learning methods to solve the problem, the score of random guess or without any ML/DL methods will be canclled finally.
You are likely to train models according specific task requirements. You have access to GPU several CPUs for training DL/ML models. Use CUDA and PyTorch for faster training if needed.
Code requirements:
Read all data files from (ota_dir=(data_drr)
Save all the predictions given by the model to file named iegiitioon-(tas__naaaa directory.
in the './cache/ tethee/ehrhhot.ee (model))m
Don 't add, delete, modify any files in data_dir
Use "print"
output information in the feedback
No plotting or visualization is allowed
Code should be self-contained and not rely on any variables or
state outside
Code must be completely runnable, otherwise it will be considered
as failed
Optimize your Model/Parameterr// /Data DData Processing/Algorii continuous improvement
The prediction file should be file with the following format, where the prediction should be predicted labels instead predicted probabilities: patient_id, prediction 115967096, 8192
(feature_information)) (label_information
BioDSBench Prompt Feature Information
The corresponding features are stored in the following directories: {feature teaturreddeectory.trrii.. training features for the task ceatturr_ddrecttry.valll. validation features for the task (feature_ ciwpuuredddrrettttestttte test features for the task Each of the feature files is a dictionary, containing the following
keys:
data_matrix: the feature vectors of the visits, where each is embedded vector, representing a single visit of patient patient_ids: the identifiers of the patients, where each row is a visit and the corresponding patient id
labeling_time: the time of the visit, where each row is visit and the corresponding time
37

---

Published as a conference paper at ICLR 2026
BioDSBench Prompt Label Information
The corresponding labels are stored in the following directories: (label_dirrcctory.ttran::: training labels for the task (label_cdreectory.val]] validation labels (label_directtrr_teet]] test labels for the task Each of the label files contain the following columns: patient_id: the identifier of the patient value: the label value of the patient the (task_name) task label_type: the type of the label, which can be "categorical boolean', etc.
task
prediction_time: only the features before this time can be used to predict the label, used in data processing stage
G.9 CHR-SEOSQU PROMPTS
include prompt details for EHR-SeqSQL tasks as follows:
EHR-SeqSQL Prompt Part
You are an biomedical expert in handling EHR data and answer questions accordingly.
Your objective is solve coding problem with given EHR data, with the goal of finally give concrete several tables:
the question.
Assume you have knowledge (1) Tables are linked by identifiers which usually have the suffix 'ID '. For example, SUBJECT_ID refers to a unique patient, HADM_ID refers to unique admission to the hospital, and ICUSTAY_ID refers to a unique admission to an intensive care unit.
(2) Charted events such as notes, laboratory tests, and fluid balance are stored in a series of 'events' tables. For example the outputevents table contains all measurements related to output for given patient, while the labevents table contains laboratory test results for patient.
(3) Tables prefixed with 'd_' are dictionary tables and provide definition for identifiers. For example, every row of chartevents is associated with single ITEMID which represents the concept measured, but it does not contain measurement. By joining chartevents and d_items on ITEMID, it is possible to identify the concept represented by given ITEMID. (4) For the databases, four of them are used to define and track patient stays: admissions. patients, icustays, and transfers. Another four tables are dictionaries for cross-referenningg codes against their respective definitions: d_icd_diaggoses, d_icd_proceduress d_items, and d_labitems.. The remaining tables, including chartevents. cost, inputevents.cv, labevents, microbiologyevent. outputevents, prescriptions, procedures_ice iidd contain data associated with patient care. such as physiological measurements, caregiver observations. and billing information.
actual name the
For different tables, they contain the following information: (1) ADMISSIONS, csv: ROW_ID, SUBJECT_ID. HADM_ID ADMITTIME, DISCHTIME, COMISSION_TYPE LOMAATOONNLOCAATON, DISCHARGE_LOCATIONN INSURANCE, LANGUAGE, MARITAL_STATUS ETHNICITY, AGE
(2) CHARTEVENTS. csv: ROW_ID, SUBJECT_ID, HADM_ID, ICUSTAY_ID ITEMID, CHARTTIME, VALUENUM, VALUEUOM
38

---

Published as a conference paper at ICLR 2026
EHR-SeqSQL Prompt Part II
(3) COST csv: ROW_ID, SUBJECT_IO, ID, HADM_ID, EVENT_TYPE, EVENT_ID,
CHARGETIME, COST
(4) BIC_LDIANOOSSS csv: ROW_ID. ICD9_CODE, SHORT_TITLE, LONG LONG_TITLE TITLE
(5) D_ICD PROCEDURES csv: ROW ID. ICD9_CODE, SHORT_TITLE, LONG_TITLE
(6) D_ITEMS csv: ROW ITEMID, LABEL, LINKSTO
(7) D_LABITEMS csv: ROW_ID, ITEMID, LABEL
(8) DIAGNOSES_ ICD csv: ROW_ID, SUBJECT_ID, HADM_ID, ICD9_CODE,
CHARTTIME
(9) ICUSTAYS CSV: ROW ID. SUBJECT ID, HADM_ID, ICUSTAY_ ID FIRST_CAREUNIT, LAST_CAREUNIT FIRST_MARDID LAST_WARDID WARDID, INTIME, OUTTIME
(10) INPUTEVENTS_CV. csV: ROW_ID, SUBJECT_ID, HADM_ID, ICUSTAY_ID,
CHARTTIME, ITEMID, AMOUNT
(11) LABEVENTS... csv: ROW_ID, LUBJECT_ID, HADM_ID, ITEMID, CHARTTIME,
VALUENUM, VALUEUOM
(12) MICROBTOLOGYEVENTS
RROW_ID, SUBJECT_ HADM_ID, CHARTTIME,
CWEC_TYPEDDESC. ORG_NAME
(13) OUTPUTEVENTS.COM CSV: ROW_ID, LUBJECT_ID, ID, ICUSTAY_ID,
CHARTTIME, ITEMID, VALUE
(14) PATIENTS ROW_ID, SUBJECT_ID, GENDER, DOB, DOD
(15) PRESCRIPTIONS. CSV: ROW_ID, SUBJECT_IN HADM_ID, STARTDATE, ENDDATE DRUG, DOL__VAL_RX DOSE UNIT_RX ROUTE
(16) PROCEDURES.CG. csV: ROW_ID, SUBJECT_ID, HADM_ID, CODE_ CHARTTIME
(17) TRANSFERS... csV: ROW_ID. SUBJECT_II ID, HADM_ID, ICUSTAY_ID, EVENTTYPE, CAREUNIT, WARDID, INTIME, OUTTIME
Al1 the tabls are saved in the data directory (data_drrecttrr)..
G.10 EHRCON PROMPTS
We include prompt details for EHRCon tasks follows:
EHRCon Prompt Part
You are biomedical expert in handling EHR data and answer questions accordingly.
Your objective is to solve a coding problem with given EHR data, with the goal of finally give a concrete answer to the question.
Assume you have knowledge of several tables: linked
(1) Tables For example, SUBJECT_ID refers to a unique patient, HADM_ID refers to a unique admission to the hospital, and ICUSTAY_ID refers to a unique admission to an intensive care unit.
identifiers which usually have the suffix 'ID
(2) Charted events such as notes, laboratory tests, and fluid balance are stored in series of 'events" tables. For example the outputevents table contains all measurements related to output for a given patient, while the labevents table contains laboratory test results for patient.
(3) Tables prefixed with 'd_' definitions for identifiers. For example, every row of chartevents is associated with a single ITEMID which represents the concept measured, but it does not contain the actual name of the measurement. By joining chartevents and d_items on ITEMID, it is possible identify the concept represented by given ITEMID.
dictionary tables and provide
39

---

Published as a conference paper at ICLR 2026
EHRCon Prompt Part
(4) For the databases, four of them are used to define and track patient stays: admissions, patients, icustays, and transfers. Another four tables are dictionaries for cross- codes against their respective definitions: d_icd_diagnoses d_icd_proceduurss d_items, and d_labitems. The remaining tables, including chartevents, cost, inputevents.cv. labevents, microbiologyevents. outputevents, prescriptions, procedures_icd contain data associated with patient care, such as physiological measurements, caregiver observations, and billing information.
For different tables, they contain the following information: (1) ADMISSIONS, csv: ROW_ID, SUBJECT_ID, HADM_ID, ADMITTIME, DISCHTIME, ADMISSION_TYPE ADMISSION_
DISCHAREE_LOCATION, INSURANCE, SAIITALSTAATU, ETHNICITY, AGE (2) CHARTEVENTS. csv: ROW_ID, SUBJECT_ID, HADM_ID, ICUSTAY_ID, ITEMID, CHARTTIME, VALUENUM, VALUEUOM
LANGUAGE,
(3) COST csV: ROW_ID, SUBJECT_ID, HADM_ID, EVPE EVENT_ID, CHARGETIME, COST
(4) D_ICD_ [CC__DIANOOSS CSV: IDW_ID, ICD9_CODE SHORT_TITLE, LONG_ TITLE
(5) D_ICD_ LIICD_PROEEDURES
ROW_ID, ICD9_CODE, SHORT_TITLE, LONG_TITLE
(6) D_ITEMS. csv: ROW_ID ITEMID, LABEL LINKSTO
(7) D_LABITEMS C CSV: ROW_ID, ITEMID, LABEL
(8) DIAGNOSES_ csv: ROW_ID, SUBJECT_ID. HADM_ID, ICD9_CODE.
CHARTTIME
(9) ICUSTAYS ROW_ID, SUBJECT_ID, HADM_ID, ICUSTAY_IL
LAST_CAREUNIT, FIRST_WARDID, LAST_WARDID, INTIME,
OUTTIME
(10) INPUTEVENTS_CY csv: ROW_ID, SUBJECT_ID, HADM_ID, ICUSTAY_ID,
CHARTTIME, ITEMID AMOUNT
(11) LABEVENTS. csv: ROW_ID, SUBJECT_ID, HADM_ID, ITEMID, CHARTTIME,
VALUENUM, VALUEUOM
(12) MICROBI0LLGGEVEENTS csv: RROW_ID ID, SUBJECT_ID, ID, ID, CHARTTIME,
SPEC BECC_TYPE_DEESSS NAME
(13) OUTPUTEVENTS
ROW_ID, SUBJECT HADM_ID. ID, ICUSTAY_ID,
CHARTTIME, ITEMID, VALUE
(14) PATIENTS. CSV: ROW_ID, SUBJECT_ID, GENDER, DOB DOD
(15) PRESCRIPTIONS CSV: ROW ID. SUBJECT ID. HADM_ID, STARTDATE, ENDDATE
DRUG, DOL__VAL_RX, DOSE_UNIT_RX ROUTE
(16) PROCEDURES. cSV: :::: ROW_ID, SUBJECT_ID HADM_ID, ICD9_CODE, CHARTTIME
(17) TRANSFERS cSV: ROW_ID, SUBJECT_ID, HADM_ID, ICUSTAY_ID. EVENTTYPE, CAREUNIT, WARDID, INTIME, OUTTIME
All the tables saved in the db file at (db_location).
In addition, you have access to a csv containing the clinical notes with the matching subject ids and hospital admission ids: ROW_ID, SUBJECT_ID HADM_ID, CHARTDATE, CHARTTIME, STORETIME, CATEGORY, DESCRIPTION, CGID, ISERROR, TEXT, ADMITTIME
This clinical note esv is at (note_csv).
G.11 MIMIC-EXTRACT PROMPTS
We include prompt details for MIMIC-EXTRACT tasks follows:
40

---

Published as a conference paper at ICLR 2026
MIMIC-EXTRACT Prompt PART
You are an biomedical expert in writing machine learning code to solve reR-vele tasks.
Your objective
solve machine learning task based on given data, with the goal of maximizing the performance of the model in limited steps.
You must use Machine Learning/Deep Learning methods to solve the problem, the score of random guess without any ML/DL methods
will canceled finally.
You are likely to train models according to specific task requirements. You have access to GPU and several CPUs for training DL/ML models. Use CUDA and PyTorch for faster training if needed.
Code requirements:
Read all data files from
Save all the predictions given the model to file named preiictions(ttss__aam)).. directory.
in the ///cache/ehrshot..//
Don t add, delete, or modify any files in data_dir
Use "print" to output information in the feedback
No plotting
visualization is allowed
Code should
self-contained and not rely on any variables or state outside
Code must be completely runnable, otherwise it will be considered
failed
Optimize your Model/Parrmeters/Dota Processing Algor for continuous improvement
The prediction file should be a csv file with the following format, where the prediction should be predicted labels instead of predicted probabilities:
You have the data splits based on hospital admission ids. You are asked to use longitudinal EHR data within each admission instance to predict two types of tasks:
(1) Classification associated with the entire duration of admission: mortality inside hospital, mortality inside ICU, length of stay beyond 3 days, length of stay beyond 7 days. All 4 are binary classification tasks using lab features only.
For the first task, the output esv should have two columns: subject_id, prediction 9923,
(2) Classification associated with hourly measurements: intervention of vasopressor ICU, and intervention ventilator in ICU, Use the past 6 hours of lab measurements and static demographics (matching patient id) to predict the intervention statuses during the 4- hour period after 6 hours.
For the second task, the output esv should have three colums instead: subject_id, window_idx, prediction 140, 4,
The corresponding features are stored the following directories: (feature feaature.drrettoryyttainn training features for the task validation features for the task [featuree_drectorry_est))) test features for the task
41

---

Published as a conference paper at ICLR 2026
MIMIC-EXTRACT Prompt PART II
Each of the feature files is pickled pandas dataframe:
subject_id: the unique ID of the subject
hadm_id: the unique icustay_id: the unique ID of the ICU session hours_in: the number of hours since hospital admission. Counting from
the hospital admission
The rest of the columns
organized in groups of three, where the outer level specifies the type measurements (e.g. alanine aminotransferase and ph urine), and the inner level lists the count, mean and std of the measurements, respectively The table has been imputed.
(feature_iinormatiion)
(label_informatiee
MIMIC-EXTRACT Prompt - Lab Feature
The corresponding features are stored in the following directories: (feature_directoryy.crann training features for the task
(Featuree_drectory_vaaa] validation features for the task
dreature_directory_cesss Each of the feature files a pickled pandas dataframe:
test features for the task
subject_id: the unique ID of the subject
hadm_id: the unique ID of the hospital admission icustay_id: the unique ID of the ICU session
hours_in: the number of hours since hospital admission. Counting from
The rest of the columns are organized in groups of three, where the outer level specifies the type of measurements (e.g. alanine aminotransferase and ph urine), and the inner level lists the count, mean and std of the measurements, respectively The table has been imputed.
MIMIC-EXTRACT Prompt Static Feature
The carresponding features are stored in the following directories:
demographic training features for the task (feature_directoryyttee demographic validation features the task [feature__drecttry.val))
demographic test features for the task Each of the feature files is a pickled pandas dataframe:
subject_id: the unique ID of the subject
hadm_id: the unique
of the hospital admission
icustay_id: the unique ID of the ICU session
intime: the total number of hours in the associated admission
gender_F and gender_M: one-hot boolean columns for gender
Age 1.0, Age 2.0, Age 3.0. Age 4.0: one-hot boolean columns for ages groups 10-30, 30-50, 50-70, and >70, respectively
EEthiiity columns: one-hot boolean columns for ethnicity ( American Indian, Asian, Black, Hispano, Other, White)
First care columns: one-hot boolean columns for first admitted unit (CCU, CSRU, MICU, SICU, TSICU)
42

---

Published as a conference paper at ICLR 2026
MIMIC-EXTRACT Prompt Mor Los Label
The corresponding labels are stored in the following directories: [label_diircctory.ttaan:: training labels for the task (label_ctrecttory.va)): validation labels [Label_directorr_testt]] test labels for the task Each of the label csv files contain the following columns: subject_id: the unique ID of the subject
task
hadm_id: the unique mort_icu or mort_hosp or los_3 or los_7: the boolean label for whether the patient died in the ICU, died in hospital, the length of stay exceeding 3 days, and LOS exceeding 7 days, respectively
of the hospital admission
label_type: the type of the label, which can be 'categorical '/' boolean', etc.
MIMIC-EXTRACT Prompt - Ventilator Vasopressor Label
The corresponding labels are stored in the following directories: (Labe__directoryyttraan::: training labels for the task (1abel_ddrecttrr.vva]]) validation labels for the task [labe_ddrrectory.tess}]. test labels for the task Each of the label csv files contain the following columns:
subject_id: the unique ID of the subject 6 __hour_window_j patient is admitted to hospital.
the 6 hour predicted window counted since the
intervention.categryy
of the four scenarios: Label 1 CONTROL No intervention throughout the prediction window. Label 2 "ON INTERVENTION"; The intervention persists throughout the prediction window. Label 3 "ONSET": Intervention starts within the prediction window. Label "WEAN": Intervention ends within the prediction window.
tabel_typer type of the label, which can be 'categorical boolean etc.
G.12 N-POWERAI PROMPTS
We include prompt details for NPowerAl tasks follows:
NPowerAl Prompt
You scientist conducting biomedical research and constantly facing statistical problems. Sometimes, you need to find the minimum sample size to achieve a specific power. In other times, would like to know the statistical power given population size.
43


Reviews:
[{"review_id": "Ykoz9j3ga8", "reviewer": "Reviewer_7VJa", "summary": "MedAgentGym introduces an interactive training environment for code-centric biomedical reasoning. There are numerous scenarios, with different LLMs evaluated in the environment showing gaps between especially closed vs open models. They also introduce Med-copilot trained on the env which is very strong.", "questions": "- Can you assess some trajectories for sound reasoning vs just only correctness\n\n- Can we know that none of the LLMs have already trained on the benchmark datasets? Maybe need some private data or new data\n\n- To the OOD drop please can you dig in and understand why?\n\n- Please can you add variance not just mean to understand overlap of models", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. Solving an important problem & provides a comprehensive benchmark on real-world medical tasks. \n\n2. Very rigorous evals across many tasks over many models \n\n3. Strong Performance Gains of Med-Copilot on the env", "weaknesses": "- Only execution evals, no assessment of intermediate reasoning and steps which is vital in medicine. Where the trajectory matters as much as the solution\n\n- Big OOD drops unexplained on external dataset (more validation and digging). Maybe things are just overfit?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WL6GxDUxq3", "reviewer": "Reviewer_cegQ", "summary": "The paper presents a released training environment MedAgentGym for the purposes of benchmarking and training LLMs for the use of biomedical data science coding tasks. Benchmarking against many propriatary and (varying sized) open-source LLMs demonstrates the state of the field in this task. Training demonstrates impressive gains in task performance of OS LLMs, comparable to gpt-4o. The authors conduct extensive benchmarking and experiments to demonstrate the utility of their training environment.", "questions": "* Table 3 demonstrates results of LLMs on MedAgentGym. Some \"best avg. scores\" (for a given LLM size) are relatively quite low. For example, the OSS <10B has Qwen3-8B at a success rate of 30.83. Given that some practitioners may only have the compute for such models, are you able to give reccomendations (complementing the writing in §4.2) regarding this? For example, what do you deem a sufficiently good performance on benchmarking for real-world deployment of a LLM?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is written to a good standard and certainly looks publication-ready\n* The consideration of open-source LLMs for the papers setting of biomedical data science is important, as a large amount of data will be under stringent data privacy rules. Many alternative similar papers operating in this field do not consider this\n* Good contribution over prior literature, encapsulating the majority of tasks that I believe would be applicable for biomedical data science\n* Extensive benchmarking of many existing open-source and proprietary LLMs gives important information regarding the capabilities of such models\n* Thorough experiments for model Med-Copilot. Results look promising", "weaknesses": "* I have a feeling that the the title and naming given to the training environment is slightly overstepping and too generelized. Perhaps 'BioMedAgentGym' is more suitable. \n* Since the paper is releasing a training environment for the practical real-world use of biomedical data science, I would like to see some discussion on the implications of this and reccomendations to users (please see questions below) \n* I cannot see many weakensses, though I am not familiar with the field of biomedical research nor such benchmarking papers", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8BsOiY9SWO", "reviewer": "Reviewer_xZ94", "summary": "This paper presents MedAgentGym, an extensible agentic environment and benchmark designed to systematically advance code-centric medical reasoning in LLM-based agents. It encapsulates 72,413 executable medical data science tasks spanning 129 categories derived from 12 scenarios, featuring isolated Docker-based sandboxes, ground-truth verifiers, multi-turn feedback, and scalable trajectory collection. Through comprehensive empirical evaluation, MedAgentGym is used to benchmark 29 LLMs, highlighting persistent deficits for medical code generation, and demonstrating substantial improvements via agentic reinforcement learning fine-tuning.", "questions": "1. Can the authors clarify how different error types (see Figure 7) are incorporated into the RL reward signal? Are there distinct penalties for, say, 'stuck in the loop' vs. compile/runtime/IO errors? How sensitive is final agent performance to this reward model?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. MedAgentGym aggregates an exceptionally broad and diverse set of medical code-centric tasks.\n2. The environment is built around reproducible, interactive Docker sandboxes, allowing code execution, error handling, debugging, and dynamic dependency install, addressing reproducibility and privacy.\n3. Med-Copilot exhibit strong improvements over baselines, with RL strategies yielding notable boosts, and ablation studies clarifying contributions.", "weaknesses": "1. MedAgentGym is constructed by integrating 12 existing datasets. Although it provides a division between training and test sets, the model is exposed to the task types and data patterns from these datasets during training. Therefore, its strong performance on the internal test set may partially result from memorization of specific task patterns or overfitting, rather than genuinely acquiring a universal biomedical code reasoning capability.\n2. While integrating these components into a large-scale, biomedical-oriented environment represents a significant engineering contribution, the core technical concepts underlying the environment—such as the use of Docker sandboxes, provision of interactive debugging feedback, and trajectory collection for reinforcement learning—have precedents in the AI agent domain.\n3. The medical-specific models evaluated in the paper, such as HuatuoGPT-o1-7B and MedReason-8B, are relatively small, with only 7B/8B parameters. Attributing their suboptimal performance solely to the limitations of medical specialization, while overlooking the substantial differences in model size, is logically flawed. A more equitable comparison would be to assess a large-scale, medically optimized model against a general-purpose large model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YINP35HRtY", "reviewer": "Reviewer_Keuw", "summary": "This paper introduced MedAgentGym, a training environment for coding-based biomedical agents. It consists of three folds of contributions: (1) MedAgentGym involves 72,413 task instances across 129 categories derived from 12 biomedical scenarios. (2) This training platform allows for efficient deployment and scalable evaluation, benchmarking 29 LLMs. (3) The training data collected from MedAgentBench leads to the powerful Med-Copilot-7B/14B, which produce comparable results as the much larger proprietary LLMs.", "questions": "Q1: In Line 358, how exactly do you prepare the online pairs for DPO? Isn't DPO an offline algorithm?  \nQ2: What do you mean by \"accurately selects successful trajectories\" in Line 417? Can you explain the difference between Pass@K and Best@K? Are they metrics for the agent or for the verifier? And why does a small gap between the two metrics indicate that \"the verifier can effectively identify successful trajectories\"?  \nQ3: What do you mean by \"repeat this DPO step iteratively\" in Line 443, and what do you mean by \"DPO using eight new rollouts per task\"? Can you explain the setup of iDPO? And why do you need \"eight new rollouts per task\" in addition to the 4,298 pairs?  \nQ4: Can you compare the results from the self-improvement in Section 5.3 and the results using the setup in Section 5.1?  \nQ5: How exactly is Figure 10 computed? Based on what features did you calculate the cosine similarity, and what does the \"in/out-of-distribution\" in \"inter-distribution\" mean? How did you determine if a task should belong to in- or out-of-distribution? What's the average number of turns and other statistics of those tasks?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1: The preparation of MedAgentGym requires a significant amount of effort, and is a non-trivial contribution to the open-source community.  \nS2: The setup in MedAgentGym is comprehensive, and the evaluation of existing LLMs on coding-based medical reasoning tasks is thorough.  \nS3: Med-Copilot-7B and -14B are also very useful open-source models for medical reasoning tasks, and their training and testing setups are solid.", "weaknesses": "W1: How are the tasks studied in this paper fundamentally different from the general-purpose coding tasks? To what extent is the medical knowledge essential here? If an agent excels in general-purpose coding tasks, does it still perform well here? How do the rankings differ?  \nW2: Similarly, the related work lacks a discussion of existing general-purpose coding benchmarks.  \nW3: The data construction step in Section 3.2 is unclear, and particularly, it does not show the difference between the contributed benchmark and the constituent datasets from existing work. Based on Table 2, is MedAgentGym simply an ensemble of all the existing benchmark datasets? Is this paper overclaimed?  \nW4: Can you provide some example instances to illustrate MedAgentGym qualitatively?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)