AnesSuite: A Comprehensive Benchmark and Dataset Suite for Anesthesiology Reasoning in LLMs
Authors: Xiang Feng
Wentao Jiang
Zengmao Wang
Yong Luo
Pingbo Xu
Baosheng Yu
Hua Jin
Jing Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Large language model
Reasoning
Anesthesiology
Medicine
Primary area: datasets and benchmarks

Abstract: The application of large language models (LLMs) in the medical field has garnered significant attention, yet their reasoning capabilities in more specialized domains like anesthesiology remain underexplored. To bridge this gap, we introduce AnesSuite, the first comprehensive dataset suite specifically designed for anesthesiology reasoning in LLMs. The suite features AnesBench, an evaluation benchmark tailored to assess anesthesiology-related reasoning across three levels: factual retrieval (System 1), hybrid reasoning (System 1.x), and complex decision-making (System 2).  Alongside this benchmark, the suite includes three training datasets that provide an infrastructure for continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning with verifiable rewards (RLVR). Leveraging this suite, we develop Morpheus, the first baseline model collection for anesthesiology reasoning.  Despite undergoing limited training with SFT and group relative policy optimization (GRPO), Morpheus not only achieves substantial improvements in anesthesiology that rival larger-scale models, but also demonstrates enhanced reasoning capabilities across general medical and broad-domain benchmarks. Furthermore, through comprehensive evaluations and experiments, we analyze the key factors influencing anesthesiology reasoning performance, including model characteristics, training strategies and training data. Both AnesSuite and Morpheus will be open-sourced at https://github.com/MiliLab/AnesSuite.

Published as a conference paper at ICLR 2026
ANESSUITE: A COMPREHENSIVE BENCHMARK AND DATASET SUITE FOR ANESTHESIOLOGY REASONING IN LLMS
Xiang Feng*
Wentao Jiang!
Zengmao Wang!
Yong Luo
Pingbo 2
Baosheng Yu
Hua Jin
Jing Zhang'
School of Computer Science, National Engineering Research Center for Multimedia Software and Hubei Key Laboratory Multimedia and Network Communication Engineering,
Wuhan University, China
Department of Anesthesiology. Zhejiang Cancer Hospital, China
Institute of Medicine, Chinese Academy of Sciences, Hangzhou, Zhejiang, China LLee Kong Chian School of Medicine, Nanyang Technological University, Singapore
*Centre of AI in Medicine, Nanyang Technological University, Singapore Department of Anesthesiology. First People's Hospital of Yunnan Province, China "Kunming University of Science and Technology, China (Fengxiang.cs, jiang.wentao, wangzengmao, luoyang) edu en xupb0zjcc. org. cn, I jingzhang. cvegmai com
yu@ntu, sg, jinhuakmö I 63. com
ABSTRACT
The application of large language models (LLMM)) the medical field has gar- nered significant attention, yet their reasoning capabilities in more specialized domains like anesthesiology remain underexplored. To bridge this gap, we in- troduce AnesSuite, the first comprehensive dataset suite specifically designed for anesthesiology reasoning in LLMs. The suite features AnesBench, an evalua- tion benchmark tailored to assess anesthe reasoning across three levels: factual retrieval (System 1), hybrid reasoning (System 1.x), and com- plex decision-making (System 2). Alongside this benchmark, the suite includes three training datasets that provide an infrastructure for continued pre- pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning with verifiable rewards (RLVR). Leveraging this suite, we develop Morpheus, the first base- line model collection for anesthesiology reasoning. Despite undergoing limited training with SFT and group relative policy optimization (GRPO), Morpheus not only achieves substantial improvements anesthesiology that rival larger- scale models, but also demonstrates enhanced reasoning capabilities across gen- eral medical and broad-domain benchmarks. Furthermore, through comprehen- sive evaluations and experiments, analyze the key factors influencing anesthe- siology reasoning performance, including model characteristics, training strate- gies and training data. Both AnesSuite and Morpheus will be open-sourced cww/ 1 httpaa//c/coiiiaaa/eeeeettttttt
INTRODUCTION
The success of LLMs has fundamentally reshaped the domain of medical artificial intelligence and catalyzed the creation of specialized medical models (Liu al., 2024b). However, the reasoning ability of LLMs within medicine and its subdisciplines still need significant improvement, partic- ularly when compared with general reasoning tasks such as mathematics and programming (Weng et al., 2023).
"Equal Contribution
1 "ooresponniig Author

---

Published as a conference paper at ICLR 2026
AnesR1
Baseline Model
AnesSuite
Marphous 78/L48/328
Verifiblee Question
GAPO
Wendeentss The biltyal cxetiion
AreSCorpur 2:40- desumarts
AnesR1
Aneay 201- QA pare
renfidble
Evoluation Analysis
AnesQA
Model charactenistics
risk Lo tooic
Ai an the teom bet snrthhed io deconee
teat Lat
The kay doiee in iesasss and Be araccadio oue Saredd slans,
System1
System1x
AnesCorpus
Cantaed
Blxie patceat peber por porting
System2 2
aholkabl mntice mccthott thch contrnad
durigg pusticsing seguiiigio di any contortttee
Training strategies:
med yoirr
Anes
Figure I: Overview of AnesSuite. AnesSuite is composed of four components: AnesBench, cross- lingual structured benchmark; AnesCorpus, a collection of anesthesiology documents; AnesQA, creestion-answerre dataset derived from domain literature; and AnesRI, dataset featuring verifi- able anesthesiology questions with chain-of-thoughe annotations. Leveraging this suite, we devel- oped Morpheus, the first collection of reasoning LLMS for anesthesiology. Subsequent ablation and experiments identified key factors influencing reasoning performance in this specialized domain.
Anesthesiology, as a highly specialized discipline, requires extensive medical expertise and high- stakes clinical decision-making In clinical practice, anesthesiology encompasses the simultaneous management of diverse and interdependent physiological systems- function, cardiovascular stability, electrolyte balance and sedation levels, as well as perioperative physiological fluctuations. This multifaceted responsibility means that anesthetic decision-making requires not only the factual recall of System thinking but also the complex, context-aware judg- ment and higher-order reasoning of System 2 (Kahneman, 2011). Existing medical benchmarks and models often overlook the unique characteristies of anesthesiology (Panagoulias et al., 2024; Olatunji et al., 2024; Abacha et al., 2019; Liu et al., 2024c). Others focus mainly on assessing factual recall (Li et al., 2023; Grzybowski et al., 2024; Wang et al., 2024a; Zhou et al., 2025).
airway and respiratory
To address this gap, we introduce the first comprehensive suite of benchmark and datasets, Anes- Suite, dedicated to anesthesiology reasoning. This suite includes AnesBench, a bilingual and structured benchmark comprising 7,972 questions designed to evaluate both factual retrieval and clinical decision- -aaiing in anesthesiology, which accompanied by a full range of training re- sources: AnesCorpus,
extensive collection of over 2.4 million documents for continued pre- training; AnesQA, a SFT (Ouyang et al., 2022) dataset with over 20,713 question-answer (QA) pairs; AnesR1, dataset with over 10,287 instances of verifiable anesthesiology questions with thain-of-thought annotations, which can be used for SFT (Ouyang et al., 2022) or RLVR (Lambert al., 2024).
Leveraging AnesSuite, we propose Morpheus, the first collection of reasoning LLMs specifically de- signed for anesthesiology. Initialized from the Qwen2.5-7B, Qwen2.5-14B and Qwen2.5-32B (Yang et al., 2024), it was trained on AnesR1 using SFT and GRPO (Shao et al., 2024). Despite limited training. Morpheus consistently enhances reasoning performance across domain-specific, general medical, and general-domain benchmarks, rivaling the performance of larger-scale models.
Moreover, using AnesBench, we evaluate a range of leading LLMMs, including proprietary mod- els such as GPT-4o (OpenAI, 2024a), Claude-3.7-Somae (Anthropic, 2025), Gemini-2.5-Flas and Gemini-2.5-Pa (Comanici et al., 2025). We also conducted extensive ablation and analytical ex- periments using the training datasets in AnesSuite. Our analysis yields several key insights: (1) Model performance benefits from larger model scales, whereas System 2 tasks exhibit compar- atively smaller gains. Both systems experience diminishing marginal returns, with performance improvements decreasing as model scale increases. (2) The length of the Ch Thought (Wei et al., 2022, CoT) is critical factor in enhancing performance, particularly for System 2 tasks, it
2

---

Published as a conference paper at ICLR 2026
enables more structured, step-by-step reasoning. (3) Language transferability remains significant factor limiting the performance of multilingual models. (4) Model's multilingual amesthesiology reasoning capabiliitiss are highly sensitive to the training data of CPT phase, necessitating careful corpus management. (5) Data from the general medical domain serves as a valuable complement to meesthesiology-speciif data, enhancing overall reasoning performance.
Our contributions are threefold: (1) Dataset Suite Construction: We introduce AnesSuite, the first comprehensive dataset suite for anesthesiology reasoning. It includes AnesBench, structured, cross-lingual benchmark for evaluating factual retrieval and clinical decision-making, along with three training datasets supporting CPT. SFT, and RLVR. (2) Baseline Model: We present Morpheus, the first collection of reasoning LLMs for amesthesiology. With SFT and GRPO training on the AnesR1 I dataset, Morpheus yields consistent reasoning enhancements across domain-specific, gen- eral medical, and general-domain benchmarks, underscoring the efficacy of data from reasoning- intensive disciplines like anesthesiology. (3) Comprehensive Evaluation and Analysis: Through systematic evaluations and extensive ablation studies, we provide an in-depth analysis of factors influencing anesthesiology reasoning, such as model characteristics, training strategies and training data. Overall, we committed to leveraging ysis collectively provide valuable insights for developing LLMs with more enhanced anesthesiology reasoning capabilities. We will publicly release AnesSuite, Morpheus, and all associated resources.
dataset suite, baseline models, and in-depth anal-
2 RELATED WORK
2.1 REASONING LLMS
Reasoning refers to solving problems involve complex, multi-step processes with intermediate steps (Li al., 2025b). The emergence of reasoning LLMs such as OpenAl o1/o3 (OpenAI, 2024c; 2025) and Deepseek RI (Guo et al., 2025) has spurred interest in enhancing LLM's reasoning abil- ity. Recent research indicates that LLMs tend generate longer outputs when solving mathematical or reasoning problems (Xiang et al., 2025). Model performance benefits from scaling the length of the reasoning process (Tang et al., 2025; Huang et al., 2025). Models that exhibit strong exploratory tendencies or incorporate intrinsic verification mechanisms in their outputs often demonstrate en- hanced reasoning capabilities (Xiang et al., 2025; Guo et al., 2025). Moreover, different model scales have been demonstrated to exhibit significant differences in training dynamics and reasoning characteristics (OpenAI, 2024c; 2025; Zhao et al., 2024). On the other hand, training and inference paradigms, such as reinforcement learning fine-tuning (Luong et al., 2024; Guo et al., 2025; Team et al., 2025b, RFT), self-improvement methods (Zelikman et al., 2022; Gulcehre et al., 2023; Singh al., 2024), and structured search techniques exemplified by MCTS (Bi al., 2025; Qiu et al., 2024; Li et al., 2025a), have shown significant development potential.
2.2 ANESTIESIA-RELATEE BENCIMARRS AND LLMS
Anesthesiology. due to its unique interdisciplinary nature, is often implicitly classified under surgery, dentistry, similar categories (Panagoulias et al., 2024; Olatunji al., 2024; Abacha al., 2019), even in benchmarks that categorize based on different departments. Some benchmarks explicitly treat anesthesiology as separate category (Grzybowski et al., 2024; Wang et al., 2024a), while some studies employ existing anesthesiology-reee 2022: Shay al., 2023; Kinoshita al., 2024: Patnaik & Hoffmann, 2023) (e.g., exam questions from American Society of Ancsthesiologists. ASA and Japanese Society of Anesthesiologists, JSA) to evaluate I.MMs. However, the overall scale of these evaluations for anesthesiology remains very limited. Chinese Anesthesiology Benchmark (Zhou et al., 2025, CAB) is the first to focus primarily on anesthesiology. Nevertheless, CAB's 's primary emphasis on factual retrieval questions results in insufficient exploration of the unique challenges in anesthetic reasoning and decision- makiin.. Addi- tionally, its restriction to the Chinese language further limits its applicability and impact. Moreover, Hypnos and some works (Wang et al., 2024c; Guthrie et al., 2024) attempts to anesthesiology- related corpora for supervised fine-tuning to improve question- guestion-answering abilities, but they have not significantly enhanced the reasoning capabilities.
questions (Khan et al., 2024; Angel et al.,
3

---

Published as a conference paper at ICLR 2026
A 100
8 193
Dataset
Data type
Size
AnesBench
MCQs
4.4k (EN) 3.5k (CH)
Proportonn
Pagprtia
63
AnesCorpus
Plain Text
1.8M (EN) 0.6M (CH)
43-
AnesQA
QA pairs
20k (EN)
AnesR1
MCQs CUT
3.2k (EN) + 7k (CHI)
Englsb
Englis AneeRI
AnesBerch
Spenul Syaznta Spocud
Table 1: Overview of datasets AnesSuite.
Figure 2: Distribution of cognitive de- mand levels in AnesBench and AnesRI.
ANESSUITE DATASET
We propose AnesSuite, the first comprchensive dataset suite specifically designed for reasoning and decision-making in anesthesiology. This suite includes AnesBench, a lingull structured benchmark. In addition, AnesSuite includes three training datasets, AnesCorpus, AnesQA, and AnesRI, which collectively support a range of fine-tuning and application in anesthesiology.
3.1 DATASETS OVERVIEW
AnesBench is bilingual benchmark featuring parallel English and Chinese subsets, containing 4,418 and 3,554 multiple-choice questions (MCQs), respectively. Its core design is three-level of cognitive demand: System 1, focused on factual recall; System 2, involving complex reasoning; and System 1.x, a hybrid category merging recall with elementary reasoning (Saha et al., 2025), Criti- cally, shown in Fig. 2, questions demanding higher-order cognitive skills (System 1.x and System 2) constitute substantial 20-30% the total items. This hierarchical structure and emphasis on complex tasks, can provide nuanced methodology for evaluating the advanced anesthesiological reasoning capabilities of L.Ms..
Complementing the benchmark are three training datasets designed support model development. AnesCorpus is a large-scale text collection for continued pre-training, comprising over 1.8 million English and 0.6 million Chinese documents pertinent to anesthesiology. AnesQA contains 20,713 English QA pairs, where each question is annotated with one of five question type labels to enable targeted fine-tuning, as presented in Fig. 3. AnesR1 comprises a total of 10,287 items, distributed across Chinese and English subsets. Each item consists of verifiable MCQ accompanied by de- tailed CoT reasoning process. Like AnesBench, also employs the three-level cognitive demand framework. The inclusion of detailed CoT reasoning is a defining feature of A which is re- flected its data structure. As illustrated in Fig. 4, this leads to substantially longer answers in AnesRI I compared to the concise answers in AnesQA, even though question lengths remain com- parable across AnesBench, AnesQA, and AnesRI. Collectively, these training datasets provide a foundational infrastructure that supports the entire workflow, from CPT and SFT to RLVR.
3.2 DATASETS CURATION
AnesBench was constructed from authoritative sources, including American Board of Anesthesiol- ogy (ABA) examination materials, standardized textbooks, and validated online assessment tools. After collection, each item was annotated for its cognitive demand level using the DeepSeek- RI (Guo et al., 2025). This annotation was performed via prompting strategy that incorpo- rated comprehensive guidelines and few-shot examples (the complete prompt is included in the randomly chosen for manual review.
Appendix A.2). For quality assurance, 60% of the items Subsequently, we conducted data contamination (whether the questions have been seen in each model training) analysis using specialized algorithm (Ni et al., 2025) for MCQs to verify that data

---

Published as a conference paper at ICLR 2026
contamination minimal. The detailed results of the manual verification and the data leakage analysis are presented in the Appendix A.I.
To create AnesCorpus, sourced data from two large-scale web datasets, namely Fineweb! and Chinese Fineweb". We employed a trequency-based approach to filter this extensive collection. Specifically, curated list of keywords pertinent anesthesia and pain management and cate- gorized them into two groups (the complete list is available in the Appendix A.4). A document was included in AnesCorpus only if it met two criteria:
(1) least one keyword from the first group appeared once per 4,000 characters, and (2) the document contained at least one keyword from the second group. To prevent contamination from this pre-training corpus to the AnesBench benchmark, we implemented two-stage decontamination process. The initial stage used rapid n-gram-based screening to identify documents with potential content overlap. In the second stage, these flagged documents were subjected to more granular analysis using Longest Common Substring (LCS). In line with previous studies, such as HuatuoGiT--ol (Chen al., 2024) and Med-PaL.M (Sing- hal et al., 2025), documents with LCS exceeding 64 characters when compared question in AnesBench were removed. A detailed description of this decontamination protocol provided in the Appendix A.4.
The construction of AnesQA began with sourcing amesthesiology--elttee research papers from PubMed 9. These papers underwent an extensive cleaning process process-including content removal, format standardization, and relevance filtering-to yield high-quality, domain-specifie text. We then employed LLM--based automated pipeline generate question-answer pairs. This pipeline in- volved segmenting the cleaned papers into text segments based on length and section structure. For each segment, contained question. Subsequently, Sreen2.5-72B-Insttiee (Yang et al., 2024) was employed to select the most informative and relevant questions and generate their corresponding answers. The complete prompts used in this process are detailed in the Appendix A.5. While this dual-model approach can partially mitigate the inherent biases of LLMs, the generated content may still con- tain formatting errors or hallucinations. To address this, we conducted further manual verification stage: we inspected sample of the LLM outputs to identify common errors, then applied filters to remove all QA pairs exhibiting these issue.
(Grattafiori et al., 2024) was utilized generate a self-
The construction of AnesRI followed procedure similar to that of AnesBench, starting with the collection of anesthesiology multiple-choice questions from distinct set of authoritative sources. During data deduplication, we implemented a process mirroring the second stage of the AnesCorpus decontamination: any question with LCS greater than 64 characters when compared AnesBench was flagged for manual inspection. Questions deemed highly similar were subsequently eliminated. For each validated question, reasoning trajectories were generated using DeepSeek- RI (Guo et al., 2025). A rejection sampling protocol was then enforced, accepting only those trajec-
item in
Sceeario baord 30%
Daniyy
Dataset
Max
Mean
AnesBench-EN
9
4.3
Kaowaader laaod 29%
AnesBench-CH
7
5.0
AnesQA
Obatsn (Dicss Segio
AnesR 1-eSR--EN
5
4.5
Decisóo naking 24%
AnesR1-CI
8
4.9
Danity
Inferasce lesed 15%
Open coded 7%
Table 2: Number of choices in AnesBench and AnesR1.
Figure 3: Distribution of Figure 4: Data length distribu- AnesQA question type.
tion for AnesBench, AnesQA, and AnesRI.
FFineweb: http://.........ooooooo/eeeeeeen
"Chinese Fineweh: hww.ohhwwww..igggoooooooooooooio PPubMed: https://pbmed..net nchi www..mm...........go......
5

---

Published as a conference paper at ICLR 2026
Table 3: Main Evaluation Results on AnesBench. The highest and second-highest accuracies in each column are highlighted in bold and underlined, respectively. Complete results are in Ap- pendix B.3.
ChesBench-Englin Sysl Sysl.x Sys2 Total
AnesBench-Chines
Model
Avg.
Sysl Sysl.x Sys2 Total
Medical Specific LLMs
BioMistral 7B Chat
0.43 0.30
0.32 0.39
0.24
0.25
0.16
0.24
0.31
HuatuoGPT-ol 7B
0.56 0.45
0.38 0.52
0.73
0.59
0.55
0.71
0.61
HuatuoGPT-oo 8B
0.58 0.46
0.39 0.53
0.58
0,46
0.46
0.56
0.54
HuatuoGPT-ol 70B
0.70
0.58
0,48 0.65
0.71
0.57
0.49
0.68
0.66
HuatuoGPT-ol 72B
0.71
0.61 0.48 0.67
0.79
0.67
0.61
0.76
0.71
Open-Source LLMs
Qwen3-0.6B
0.39 0.31 0.26 0.36
0.34
0.29
0.22 0.35
0.31
Qwen3-1.77
0.48
0.37
0.30 0.44
0.53
0.39
0.34 0.50
0.50
Qwen3-4B
0.60
0.46
0.34 0.54
0.64
0.66
0.50 0.37
0.58
Qwen3-8B
0.65
0.50
0.40 0.60
0.72
0.44
0.39 0.70
0.65
Qwen3-14BB
0.70 0.57
0.45 0.65
0.79
0.63
0.52 0.76
0.70
Qwea3-32B
0.72 0.64
0.48 0.68
0.81
0.64
0.57 0.78
0.70
Qwen3-30B-A3B
0.73 0.60
0,48 0.68
0.77
0.62
0.50 0.74
0,61
Qwen3-235B-A22B
0.78 0.67
0.57 0.74
0.85 0.72 0.58 0.81
0.77
Llama-4-Scout-17
0.77 0.66 0.55 0.72
0.80 0.66
0.55 0.77
0.74
0.83 0.73 0.64 0.79
0.86
0.72 0.59 0.83
0.81
GPT-OSS-20B
0.70 0.60 0.52 0.66
0.71
0.58
0,49 0.68
0.67
GPT-OSS-120B
0.80 0.72 0.62 0.76
0.80
0.64
0.54 0.76
0.76
DeepSeek-V3
0.77 0.69 0.55 0.73
0.87
0.75
0.60 0.84
0.78
DeepSeek-RI
0.78
0.70 0.82
0.77
0.61
0.82
Proprietary
LLMs
GPT-4o
0.81 0.72 0.59 0.77
0.79
0.64
0.52
0.76
0.76
Gemini-22.5-Faash
0.84
0.76
0.68 0.81
0.84
0.72
0.59
0.81
0.81
Gemini-2.5-PP
0.82
0.77 0.86
0.75
0.60
0.85
Claude-3.7-Somme
0.80 0.73
0.63 0.77
0.82
0.65
0.55
0.78
0.77
tories that culminated in the correct answer. If three sampling attempts failed produce correct solution, source question was excluded from the final AnesR dataset.
EVALUATION AND BASELINE MODEL
4.1 EVALUATION
Benchmarked LLMs. We conducted comprehensive zero-shot evaluation on over 50 LLMs from diverse sets. The evaluation encompassed four leading proprictary models, namely GPT- 4o (OpenAI, 2024a). Gemini-2.5-Flass. Gemini-2.5-Per (Comanici et al., 2025), and Claude-3.7- Sonnet (Anthropic, 2025). The selection of open-source models included both domain-specific LLMs, such as HuatuoGPT-ol (Chen al., 2024) and BioMistral (Labrak et al., 2024), well as prominent general-purpor LLMs like DeepSeck-RI/V3 (Guo et al., 2025; Liu et al., 2024a), Qwen3 (Team, 2025), and Llama4 (Meta, 2025). To facilitate more granular analysis, models of varying parameter scales from each open-source family. While the main text presents selection of results, the complete list of models and their corresponding evaluation outcomes are detailed in the Appendix B.
included
Supplementary Evaluation To further refine our assessment, we introduced supplementary open-ended subset, which focused on open-ended questions in anesthesiology, to capture and an- alyze the more nuanced reasoning capabilities of LLMs within the anesthesiology domain. Addi- tionally, conducted comparative analysis using the MCQ subset from CAB (Zhou et al., 2025) against AnesBench. The detailed methodologies and results for these supplementary evaluation are available in the Appendix G.

---

Published as a conference paper at ICLR 2026
Table 4: Evaluation Results Morpheus on AnesBench.
Stage
AnesBench-Englin
AnesBench- Chinese
Model
Avg.
SFT GRPO
Sys1
Sysl.x
Sys2
Total
Sys1
Sys1.x
Sys2
Total
Qwen2.5-7B-Instnee
0.56
0.44
0.36
0.51
0.69
0.55
0.55
0.66
0.59
Morpheus-785 7B (SFT only)
x 0.59
0,45
0.35
0.54
0.58
0.42
0.47
0.56
0,54
Morpheus-7B
/
0.61
0.48
0.40
0.56
0.73
0.55
0.50
0.70
0.63
Owen2.5-144B-Issiie
0.61
0.52
0.41
0.57
0.75
0.63
0.53
0.72
0.64
Morpheus- -4B (SFT only)
x
0.65
0.51
0.38
0.60
0.57
0.51
0.45
0.55
0.57
Morpheus-LP 14B
/
0.68
0.55
0.47
0.63
0.78
0.62
0.53
0.75
0.69
Qwen2.5- 3-22--BB2 Instructtt
0.65
0.55
0.44
0.61
0.79
0.65
0.54
0.76
0.68
Morpheus-33 32B (SFT only)
x
0.72
0.60
0.52
0.67
0.67
0.53
0.47
0.64
0.65
Morpheus-32B
/
0.72
0.61
0.54
0.68
0.80
0.66
0.53
0.77
0.72
Qwen2.5-722 Instruct
0.69
0.58
0.49
0.65
0.79
0.66
0.52
0.76
0.70
Implementation details. The metric reported is Accuracy on multiple-choice questions. To ensure reproducibility, all models were evaluated with the temperature set and maximum output length set to 2048 tokens. We employed Zero-Shot CoT prompting strategy to encourage structured, step-by-steg reasoning (Wei al., 2022). detailed description of the experimental setting and the prompt is provided in Appendix B.2.
Main Results. Tab. 3 presents the main evaluation results on the AnesBench. We observe that complex anesthesiology reasoning remains significant challenge for contemporary LLMs. On AnesBench, most open-source models achieve scores below 0.5 on System questions. Even pro- prictary models, such as GPT-4o and 0.6. Furthermore, consistent trend across most models is that performance System 1.x and System tasks is markedly lower than on System tasks. This discrepancy suggests that the prin- cipal challenge for current LLMs is not simple deficit in anesthesiological knowledge, but rather an underdeveloped ability to apply this knowledge to complex reasoning problems. A major factor contributing to these lower scores in complex reasoning tasks is the prevalence of logical hallucina- generation process. To better understand this phenomenon, we provide
only achieve accuracies of around 0.5 to
tions during the step-by- detailed evaluation of reasoning hallucinations, such as Non-Sequiturs and extrapptraoo,,,i in Appendix G.3. Additionally, medical specific L.MSs do not demonstrate significant advantage over recent general-purposs reasoning models in our evaluation. This finding underscores the specialized nature of anesthesiological reasoning. distinguishing it from general medical domain.
4.2 BASELINE MODEL
We developed baseline models by fine-tuning Qwen2.5-7B, Qwen2.5-14B and Qwen2.5-32B (Yang et al., 2024) on the AnesRI. Both models underwent SFT followed by GRPO and are desig- nated Morpheus-7B, Morpheus-14B and Morpheus-32B, respectively. In alignment with established paradigms, the SFT stage serves as cold-start initializing for the subsequent GRPO training. For comparative purposes, we also present these intermediate models, which obtained after the limited SFT steps. Complete training details are provided in Appendix C.2. In addition, the results of AnesRI 1 on more models can be found in Tab. 11.
The evaluation results for Morpheus are presented in Tab. 4. A key observation is that, after GRPO training, the model achieves consistent improvements across various systems, demonstrating the ef- fectiveness of the GRPO algorithm in enhancing reasoning capabilities. Notably, following SFT and GRPO training on AnesR1, Morpheus-7BB Morpheus- 14B and Morpheus-33 reach performance levels comparable to Qwen2.5- Instruct 3.ve2..-32BB.. Insttrutt and Qwen2.5-72B Dreen..5-2B-Instteeee Instruct, spectively. This highlights the efficacy of AnesRI in cultivating complex reasoning within the spe- cialized domain of anesthesiology. Furthermore, shown in Tab 18 (Appendix G.1). Morpheus

---

Published as a conference paper at ICLR 2026
hibits enhanced reasoning capabilities across wide range of general medical and general-domain benchmarks (e.g., MMLU, MedQA). However, on specific subsets of these benchmarks, the im- provements exhibited by Morpheus are less pronounced. This may be associated with the correla- tion between the generalization of reasoning capabilities acquired through reinforcement learning and the extent of data exposure during the pre-training phase, discussed by (Cheng al., 2025). Furthermore, considering that AnesRI focuses exclusively on anesthesiology. domain typically underrepresented in standard LLM training corpora, these findings may offer valuable insights into another pertinent question: whether reasoning skills acquired via reinforcement learning in sparsely exposed domains can effectively transfer to more general domains.
5 ABLATION AND ANALYSIS
In this section, we present a series of ablation studies and analyses to provide insights for devel- oping advanced LLMs for anesthesiology reasoning. Our investigation examines two key aspects: model characteristics, training methodologies and data. Furthermore, the majority of the insights and claims presented in this section have undergone statistical significance testing. The full results provided in Appendix F.
5.1 MODEL CHARACTERISTIC
Performance Across Model Scale. Models are arranged in increasing order of their accuracy on the AnesBench- English subset. As depicted in Fig. 5, there is strong positive correlation be- tween model performance and scale, which exhibits diminishing marginal returns; larger increases in model scale correspond to progressively smaller performance gains. Furthermore, the fitted-line slopes for Systeml and Systeml.x are relatively close. In contrast, the slope for System2 sig- nificantly lower than Systeml.x, indicating that performance gains from increasing model size are markedly smaller for System2.
Fnnglsh
Systcn
In Engfish In Ehnone Aftrrmatsion
Benchnerk Aasiltac-Cugaa Ce
Sysze 1x
Symce:
lind Systee
Fimd Syaun
70
Rodic Rosic "geog Sing 00
Accuracy
0.6
50-
40-
30
Model
Owro25
Liuma11.
Ma GLA44
Gomoad
Model Index
Figure 5: Impact of Model Scale. Colors denote model scale (model index) and problem type (scat- ter points). Models are sorted by overall score in ascending order.
Figure Multilingual Assessment
Translated Benchmarks. Background colors denote base models: purple for Qwen2.5-7B, blue for 1ama--311-BB. gray for others.
Performance Across Language. To evaluate model performance across different linguistic set- tings, we performed a cross-translation of the AnesBench-Fnggee English and AnesBench-Chinee subsets using GPT-4o (OpenAI, 2024a). Specifically, the English subset was translated into Chinese, and the Chinese subset into English. This process resulted in each subset having versions in both languages, enabling direct cross-lingual comparisons. The prompts and details are detailed in Appendix C.I.
As shown in Fig. 6, Qwen2.5-7B based and other models exhibit minimal disparity between English and Chinese performance. In contrast, Llama-3.1 3..1-88-basee models, despite claiming multilingual support (Grattafiori et al., 2024), perform notably worse in Chinese. These results indicate that lan- guage transferability remains a key factor influencing multilingual model performance. According to the theory proposed in (Chen et al., 2025), once LLM reaches stable language learning stage,
8

---

Published as a conference paper at ICLR 2026
different languages form independent knowledge systems rather than relying on translation. At this stage, a lack of domain-speciffe knowledge within the ainuage-speccife knowledge system can lead to substantial performance disparities across languages. Therefore, recommend supplementing bilingual domain-specific knowledge during the different training stage to mitigate cross-linguistic performance gaps.
Performance Across Output Length By comparing output length and accuracy across three lev- els of questions AnesBench-Englise we discovered: (1) Models with longer CoT reasoning pro- cesses tend to exhibit superior response performance. As illustrated in the right subplot of Fig. 7, for System2 questions, models achieve higher scores with longer outputs. (2) However, this trend not pronounced in System1 and Systeml.x .x, which do not require extensive reasoning processes. As shown in the left and middle subplots of Fig. 7, the scores of models are almost solely correlated with the size of the model.
Systeml
Systeml.x 1.x
System2
0.75
0.65
0.60
0.60
0.55
Aocuracy
Accuracy
0.55
Accuracy
0.60
0.50
0.45
0.55
0.45
Oven
0.40
0.33
0.40
greae)
ThataGrT
Lianc)
0.35
0.45
5.19
5.42
5.40
5.25
5.50
5.75
6.00
6.25
6.50
5.50
5.75
6.00
6.25
6,50
6.75
5.75
6.00
6.25
6.50
6.75
Output Length (Tokens, Log Scale)
Output Length (Tokens, Log Scale)
Output Length (Tokens, Log Scale)
Figure 7: Impact of Output Length Shapes and colors denote model families and scales.
5.2 TRAINING STRATEGY AND DATA
This section focuses on two key research questions: (1) the effect of CPT training strategies on model performance, and (2) the impact of training data from different domains on the model's 's performance.
To investigate these questions, we subjected models to SFT for an equivalent number of steps using two distinct datasets: AnesQA, representing the wresthesiology-specfff domain, and the bilingual Medical-ol (Chen et al., 2024), representing the general medical domain. To isolate the impact of CPT, this process was applied to models both with and without prior continued pre-training on AnesCorpus. Notably, since AnesQA lacks native Chinese data, translated the dataset using GPT-4o-mini (OpenAI, 2024b). This step was crucial to align its format with the setting of Medical- ol and to ensure the validity of evaluations AnesBench-Ching and translation are provided in Appendix C.3 and Appendix C.I. The results, presented in Tab. 5, yield two key insights.
Further details on the training
Table 5: Effectiveness of Training Strategies and Data The Qwen2.5-7B-Basee BBT model is trained on our AnesCorpus, with Qwen2.5-7B-Ban serving as the foundation model.
SFT Data
AnesBench
Model
AnesQA
Medical-ol
English
Chinese
49.3
64.9
Qwen2.5-7B-Base
X
49.1
63.0
/
49.7
65.9
X
49.7
50.7
@wen2.5-7B-Ban 1---7-BBase-CT
X
/
50.7
59.4
51.2
60.0

---

Published as a conference paper at ICLR 2026
Complementarity of AnesQA and Medical-ol. The results underscore the complementary ture of AnesQA and Medical-ol for fine-tuning. As demonstrated, integrating AnesQA with Medical-ol leads to varying degrees of performance improvement across models. This finding suggests that, although Medical-ol is not exclusively focused on anesthesiology. its inclusion of general medical knowledge is advantageous for enhancing the model's anesthesiological capabili- ties, thereby complementing the specialized content provided by AnesQA, This synergy highlights that, even within highly specialized domains such as anesthesiology, the incorporation of broader medical knowledge remains essential.
Impact of Continued Pre-training. was found that continued pre-training with AnesCorpus has a dichotomous effect on multilingual performance. While it enhances performance on the AnesBench-Englin raising the maximum accuracy from 49.7% on the base model to 51.2% ter CPT, it concurrently degrades performance on the AnesBench-Chinese This demonstrates that domain-specific pre-training does not uniformly guarantee improvements across all languages. We hypothesize that this disparity stems from the model's inability to effectively transfer knowledge across languages; the gains English proficiency are not successfully propagated to the Chinese counterpart, potentially inducing catastrophic forgetting. Consequently, we recommend that the lin- guistic distribution within CPT corpora be carefully managed to avoid compromising the model's multilingual versatility.
LIMITATIONS
While AnesSuite contributes anesthesiology reasoning with LLMs, several limitations remain. First, the System 2 questions are constructed Second, future iterations of AnesSuite should consider incorporating additional modalities, to better reflect real-world clinical environments.
from abstract scenarios rather than real clinical cases.
7 CONCLUSION
This paper introduces AnesSuite, the first comprehensive suite dedicated to anesthesiology reason- ing. AnesSuite features a structured evaluation benchmark and three training datasetss, establishing a complete infrastructure for key development stages: CPT. SFT, and RLVR. We also developed Mor- pheus, the first strong baseline model collection for this domain. Trained exclusively on the AnesRI, these models demonstrate consistent performance gains across amesthesiology, general medical, and general-domal benchmarks. Furthermore, extensive ablation studies and analyses yield several key insights for improving these specialized reasoning capabilities. We hope this work will serve cornerstone for developing LLMs with enhanced anesthesiology reasoning capabilities.
ACKNOWLEDGEMENTS
This work was supported in part by the New Generation Artificial Intelligence-Nationn Science and Technology Major Project (Grant No. 20252D0123602), the National Natural Science Foundation of China (Grant No. U23A203 18 and 62276195). the Science and Technology Major Project of Hubei Province (Grant No. 2024BAB046 and 2025BCB026), the Foundation for Innovative Re- search Groups of Hubei Province (Grant No. 2024AFA017), the Medical Health Science and Tech- nology Project of Zhejiang Province (Grant No. WKJ-ZJ-26004) and the Major Science and Tech- nology Special Plan Project, Department of Science and Technology of Yumnan Province (Grant No. 202502A5080002) This work was also supported by WHU-Kingsen Joint Lab. The numerical cal- culations in this paper have been done the supercomputing system in the Supercomputing Center of Wuhan University.
We would like to thank Professor Bo Du for his helpful discussions throughout this work. Fur- thermore, we extend our gratitude to Tao Huang and Zhiwei Wang for their assistance in the data collection process.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Asma Ben Abacha, Yassine Mrabet, Mark Sharp, Travis R Goodwin, Sonya E Shooshan, and Dina Gemner-Fushmann Bridging the gap between consumers' medication questions and trusted an- swers. In MEDINFO, 2019.
Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, al. Phi-4 techni- cal report. arXiv preprint arXiv:2412.099h0 2024.
Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss- 1ecooss-100b & gpt-oss-20b model card. arXiv preprint arXiv:2508. 10925, 2025.
lñigo Alonso, Maite Oronoz, and Rodrigo Agerri. Medexpqa: Multilingual benchmarking of large language models for medical question answering. Artificial intelligence in medicine, 2024.
Mirana C Angel, Joseph B Rinehart, Maxime P Canneson, and Pierre Baldi. Clinical knowledge and reasoning abilities ai large language models in anesthesiology: comparative study on the american board of anesthesiology examination. Anesthesia & Analgesia, 139(2):00-2133, 2022.
Malaikannan Sankarasubbu Ankit Pal. Openbiollms: Advancing open-source large lan-
guage models for healthcare and life sciences. https:/ htnps/:///gggingfaeeee co. co/aatttya////////////o/aaaaaaaa CoenBioLLM-LLamae 3 2024.
Anthropic. Claude 3.7 sonnet: Frontier reasoning made practical, September 2025. URL https: //www. www.aaanthhcc.co.. www.//wwwwww...eecooooetotooeooe claude 3- sonnet.
Baichuan. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309. 10305, 2023.
URL https::tp//.o.....p. / /tar//iivvorrrr wrgg 9200 10305.
Zhenni Bi, Kai Han, Chuanjian Liu, Yehui Tang, and Yunhe Wang. Forest-of-thoughe Scaling test-time compute for enhancing Ilm reasoning. In ICML. 2025.
Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. Internlm2 technical report. arXiv preprint vxXiv:2403.1229) 2024.
Jiawei Chen, Wentao Chen, Jing Su, Jingjing Xu, Hongyu Lin, Mengjie Ren, Yaojie Lu, Xianpei Han, and Le Sun. The rise and down of babel tower: Investigating the evolution process of multilingual code large language model. In /CLR, 2025.
Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. Huatuogpt-ol, towards medical complex reasoning with 1lms. arXiv preprint arXiv:2412.18925 2024.
Zhoujun Cheng, Shibo Hao, Tianyang Liu, Fan Zhou, Yutao Xie, Feng Yao, Yuexin Bian, Yonghao Zhuang, Nilabjo Dey, Yuheng Zha, et al. Revisiting reinforcement learning for Ilm reasoning from cross-domain perspective. In NeurIPS, 2025.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.06261. 2025..
ContactDoctor.
high perffrmerfoe biomedical language http://hhinnnncceooooooooooooooe
Bio-medical: A
model.
2024.
Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng. Hanlin Zhao, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406. /2793, 2024.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv: 178.2 2024.

---

Published as a conference paper at ICLR 2026
Lukasz Grzybowski, Jakub Pokrywka, Michat Ciesiótka, Jeremi Kaczmarek, and Marek Kubis. Polish medical exams: A new dataset for cross-lingual medical knowledge transfer assessment. arXiv preprint arXiv:2412.00559, 2024.
Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, et al. Reinforced self-training (rest) for language modeling. arXiv preprini urXiv:2308. 08998, 2023.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-rl incentivizes reasoning in Ilms through reinforce- ment learning. Nature, 645(808 1):388))333333000 638, 2025.
Estefania Guthrie, Dominique Levy, and Gabriel Del Carmen. The operating and anesthetic ref- erence assistant (oara): A fine-tuned large language model for resident teaching. The American Journal of Surgery, 234:28-34: 34, 2024.
Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv. Yikai Zhang, jiayi lei, Yao Fu, Maosong Sun, and Junxian He. C-eval: A multi- level multi-discipline chinese evaluation suite for foundation models. In NeurIPS, 2023.
Zhongzhen Huang, Gui Geng, Shengyi Hua, Zhen Huang, Haoyang Zou, Shaoting Zhang, Pengfei Liu, and Xiaofan Zhang. 01 replication journey-part 3: Inference-time scaling for medical rea- soning. arXiv preprint arXiv:2501.066 06645, 2025.
Wentao Jiang, Xiang Feng. Zengmao Wang. Yong Luo, Pingbo Xu, Zhe Chen, Bo Du, and Jing Zhang. Rex-rag: Reasoning exploration with policy correction retrieval-augmented generation. arXiv preprint arwiv:2508.08111111 2025.
Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What dis- ease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 2021.
Daniel Kahneman. Fast and slow thinking. Allen Lane and Penguin Books, New York, 2011.
Adnan A Khan, Rayaan Yunus, Mahad Sohail, Taha A Rehman, Shirin Saeed, Yifan Bu, Cullen D Jackson, Aidan Sharkey, Feroze Mahmood, and Robina Matyal. Artificial intelligence for anes- thesiology board-style examination questions: Role of large language models. Journal of Car- diothorucic and Vascular Anesthesia, 38(5): 125511 2024.
Michiko Kinoshita, Mizuki Komasaka, and Katsuya Tanaka. Chatgpt's performance on jsa-certified anesthesiologist exam. Journal of Anesthesia, 38(2):282-283, 2024.
Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pictre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. Biomistral: A collection of open-source pretrained large language models for medical domains. arXiv preprint arXiv:2402. 10373, 2024.
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brah- man, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, al. Tulu 3: Pushing frontiers in open language model post-training, arXiv preprint arXiv:2411. 1.11244 2024.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented gener- ation for knowledge-intensinn nlp tasks. NewrIPS, 2020.
Binbin Li, Tianxin Meng, Xiaoming Shi, Jie Zhai, and Tong Ruan. Meddm: Llm-exccutable clinical guidance tree for clinical decision-making, arXiv preprint arXiv:2312.02441, 2023.
Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. Cmmlu: Mcasuring massive multitask language understanding in chinese. In ACL, 2024.
Jierui Li, Hung Le, Yingbo Zhou, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Codetree: Agent-guided tree search for code generation with large language models. In NAACI, 2025a.
12

---

Published as a conference paper at ICLR 2026
Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. From system to system 2: A survey of reasoning large language models. arXiv preprint wrXiv22502.74419 2025b.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In ICLR, 2023,
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024a.
Lei Liu, Xiaoyan Yang. Junchi Lei, Xiaoyang Liu, Yue Shen, Zhiqiang Zhang, Peng Wei, Jinjie Gu, Zhixuan Chu, Zhan Qin, al. A survey on medical large language models: Technology, application, trustworthiness, and future directions. arXiv preprint arXiv:2406.02. 00.006003711 12, 2024b.
Mianxin Liu, Weiguo Hu, Jinru Ding, Jie Xu, Xiaoyang Li, Lifeng Zhu, Zhian Bai, Xiaoming Shi, Benyou Wang, Haitao Song, et al. Medhench: A comprehensive, standardized, and reliable benchmarking system for evaluating chinese medical large language models. Big Data Mining and Analytics, 7(4): 2024c.
Trung Quoe Luong, Xinbo Zhang, Zhamming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Rea- soning with reinforced fine-tuning In ACL, 2024.
Meta.
of natively multi- URL https:/ //ai meta. ccm/blog/ 1lama- momultimodal-inee liigeee/.
The Ilama herd: The beginning of
modal ai innovation, April 2025.
Henning Müller and Devrim Unay. Retrieval from and understanding of large-scale multi-modal medical datasets: review. TMM, 2017.
Shiwen Ni, Xiangtao Kong, Chengming Li, Xiping Hu, Ruifeng Xu, Jia Zhu, and Min Yang. Train- ing on the benchmark not all you need. In AAAI, 2025.
Tobi Olatunji, Charles Nimo, Abraham Owodunni, Tassallah Abdullahi, Emmanuel Ayodele, Mard- hiyah Sanni, Chinemelu Aka, Folafunmi Omofoye, Foutse Yuehgoh, Timothy Faniran, et al. Afrimed-qa: A pan-african, multi-specialty, medical question-answering benchmark dataset. arXiv preprint arXiv:2411. 2024.
OpenAl. Hello gpt-4o, May 2024a. URL
OpenAl. gence, July
Gpt-4o 2024b.
mini:
advancing cost-efficient intelli-
URL
https:/ wttppeaai.commnnee.....p com/index/
gpt-44-mini-advann hww.-oomii--ttttttooooooooooeedd
OpenAl. Learning to reason with lims, September 2024c.
URL https: https::/openat..com
OpenAI. Openai o3-mini, January 2025.
URL https://opent: http://oopeaa..com///eoea wwm. ccom//ndex//////c
openai-o3-mini/
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kel- ton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In NeurlPS, 2022.
Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference health, inference, and learning. PMLR, 2022.
Dimitrios P Panagoulias, Persephone Papatheodosion, Anastasios P Palamidas, Mattheos Sanou- dos, Evridiki Tsoureli-Nikita, Maria Virvou, and George A Tsihrintzis. Cognet-md, evalua- tion framework and dataset for large language model benchmarks in the medical domain. arXiv preprint ar arXiv:2405. 10893, 2024.
13

---

Published as a conference paper at ICLR 2026
Sourav S Patnaik and Ulrike Hoffmann. Comparison of chatgpt vs. bard to amesthesia-related queries. medRxiv, pp. 2023-06, 2023.
Jiahao Qiu, Yifu Lu, Yifan Zeng. Jiacheng Guo, Jiayi Geng, Huazheng Wang, Kaixuan Huang, Yue Wu, and Mengdi Wang. Treebon: Enhancing inference- alignment with speculative tree- search and best-of-n sampling. arXiv preprint 2tt1::2410.6633333 2024.
Qwen. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https: / //qwenlmm g tub.. 1o/ zh/blog blgg/ /tw////w////0 32b/.
Swamaddepp Saha, Archiki Prasad, Justin Chih- Yao Chen, Peter Hase, Elias Stengel-Eske and Mohit Bansal. System-1. Learning balance fast and slow planning with language models. In ICLR, 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang. Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.
Denys Shay, Bhawesh Kumar, David Bellamy, Anil Palepu, Mark Dershwitz, Jens M Walz, Max- imilian S Schaefer, and Andrew Beam. Assessment of chatgpt success with specialty medical knowledge using anacsthesiology board examination practice questions. BJA: British Journal of Angesthesia, 131(2)=e3, 2023.
Xiaobo Shen, Yunpeng Tang, Yuhui Zheng, Yun-Hao Yuan, and Quan-Sen Sun. Unsupervised multiview distributed hashing for large-scale retrieval. TCSVT, 2022.
Xiaobo Shen, Yinfan Chen, Weiwei Liu, Yuhui Zheng, Quan-Sen Sun, and Shirui Pan. Graph convolutional multi-label hashing for cross-modal retrieval. TNNLS, 2024a.
Xiaobo Shen, Wei Wu, Xiaxin Wang, and Yuhui Zheng. Multiple riemannian kernel hashing for large-scale image set classification and retrieval. TIP, 2024b.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rihf framework. EuroSys, 2025.
Avi Singh, John D Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter Liu, James Harrison, Jaehoon Lee, Kelvin Xu, et al. Beyond human data: Scaling self-training for problem-solving with language models. Transactions on Machine Learning Research, 2024. ISSN 2835-8856.
Karan Singhal, Tao Tu, Juraj Goutweis Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, et al. Toward expert-level medical question answering with large language models. Nature Medicine, 31(3):943-950, 2025.
Zeyu Tang, Zhenhao Chen, Loka Li, Xiangchen Song, Yunlong Deng, Yifan Shen, Guangyi Chen, Peter Spirtes, and Kun Zhang. Reflection-window decoding: Text generation with selective re- finement. In ICML, 2025.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhu- patiraju. Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé. et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118 2024.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint a50iv:2503. 19786, 2025a.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen. Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with Ilms. arXiv preprint arXiv:2501. 12599, 2025h.
Qwen Team. Qwen3, April 2025. URL https: htttt::::///wweeeeee http:///wwllmmmmiiieee wttpobb...obbbog///eeeeeeeee
14

---

Published as a conference paper at ICLR 2026
Guoxin Wang, Minyu Gao, Shuai Yang, Ya Zhang, Lizhi He, Liang Huang, Hanlin Xiao, Yexuan Zhang, Wanyue Li, Lu Chen, et al. Citrus: Leveraging expert cognitive pathways in medical language model for advanced medical decision support. arXiv preprint 11.Xi::2502.182774, 2025.
Xidong Wang, Guiming Chen, Song Dingjie, Zhang Zhiyi, Zhihong Chen, Qingying Xiao, Junying Chen. Feng Jiang, Jianquan Li, Xiang Wan, et al. Cmb: A comprehensive medical benchmark in chinese. In NAACL, 2024a.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain thought reasoning language models. In ICLR, 2023a.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In ICLR, 2023b.
Yubo Wang, Xueguang Ma, Ge Zhang. Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et Mmlu-pro: more robust and challenging multi-task language understanding benchmark. NeurlPS, 2024b.
Zhonghai Wang, Jie Jiang, Yibing Zhan, Bohao Zhou, Yanhong Li, Chong Zhang, Liang Ding, Hua Jin, Jun Peng, Xu Lin, et al. Towards training a chinese large language model for anesthesiology. arXiv preprint 2403:22403.2244, 2024c.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of of-thought prompting clicits reasoning in large language models. In NeurIPS, 2022.
Yixuan Weng, Bin Li. Fei Xia, Minjun Zhu, Bin Sun, Shizhu He, Kang Liu, and Jun Zhao. Large language models need holistically thought in medical conversational qa. arXiv preprint arXiv:2305.0544 05010 2023.
Violet Xiang, Charlie Snell, Kanishk Gandhi, Alon Albalak, Anikait Singh, Chase Blagden, Duy Phung, Rafael Rafailov, Nathan Lile, Dakota Mahan, al. Towards system reasoning Ilms: Learning how to think with meta chain-of-thougee arXiv preprint arXiv:2501.0m 04682,, 2025.
An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang. Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115, 2024.
Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Guoyin Wang, Heng Li, Jiangcheng Zhu, Jianqun Chen, et al. Yi: Open foundation models by 01. ai. arXiv preprint arXiv:2403.04652, 2024.
Hongzhou Yu, Tianhao Cheng. Ying Cheng, and Rui Feng. Finemedlm-ol: Enhancing the med- ical reasoning ability of Ilm from supervised fine-tuning to test-time training. arXiv preprint arXiv:2501.09213. 2025.
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. In NeurIPS, 2022.
Kaiyan Zhang, Sihang Zeng, Ermo Hua, Ning Ding, Zhang-Ree Ren Chen, Zhiyuan Ma, Haoxin Li, Ganqu Cui, Biqing Qi, Xuekai Zhu, Xingtai Lx, Jin-Fang Hu, Zhiyuan Liu, and Bowen Zhou. Building specialized generalists in biomedicine. In NewrlPS, 2024.
Ultramedical:
Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. Marco-ol: Towards open reasoning models for open-ended solutions. arXiv preprint arXiv:2411.14400 2024.
15

---

Published as a conference paper at ICLR 2026
Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Livia Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, lon Stoica, Joseph E Gonzalez, et al. Sglang: Efficient execution of structured language model programs. In NeurIPS, 2024a.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhna YeYanhan, and Zheyan Luo. Llamafac- tory: Unified efficient fine-tuning of 100+ language models. In ACL. 2024b.
Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. Agieval: human-centric benchmark for evaluating foundation models. In NAACL, 2024.
Bohao Zhou, Yibing Zhan, Zhonghai Wang, Yanhong Li, Chong Zhang, Baosheng Yu, Liang Ding, Hua Jin, Weifeng Liu, Xiongbin Wang, et al. Benchmarking medical llms on anesthesiology. A comprehensive dataset in chinese. In TETCI, IEEE, 2025.
16

---

Published as a conference paper at ICLR 2026
Appendix
CONTENTS
Datasets Curation Details
18
A.1 Data Leakage Analysis
18
A.2 Details AnesBench and AnesR Annotation
18
A.3 Details of Manual Inspection in AnesBench.
19
A.4 Details of AnesCorpus Curation
20
A.5 Details of AnesQA Curation
20
B Evaluation Details
23
B.1 List of Evaluation Models
23
B.2 Experimental Environment for Evaluation
23
B.3 Complete Evaluation Results
23
Training Experiment Details
25
C.I Experiment Details of Using AnesQA for SFT
25
C.2 Experiment Details for Baseline Model
25
C.3 Experiment Details for Ablation Study
27
AnesCorpus Visualization
27
E Case Study
27
F Significance Test
33
F.I Significance Test Model Characteristic
33
Significance Test Training Methodology
34
Supplementary Evaluation
35
G.I Evaluation on Out of Distribution Benchmarks
35
G.2 Self-Consistency in Test Time Scaling
36
G.3 Hallucination Evaluation
38
G.4 Evaluation on the Open-ended Subset
39
G.5 Evaluation CAB
42
Datasheet for AnesSuite
43
Usage of LLM
47
17

---

Published as a conference paper at ICLR 2026
A DATASETS CURATION DETAILS
A.1 DATA LEAKAGE ANALYSIS
Given the increasing volume pre-training data utilized by large language models, coupled with the fact that the sources and specific acquisition methods are often regarded core secrets and rarely disclosed, pertinent issue arises: whether the questions in our benchmark have been inad- vertently leaked into the pre-training data of various models. We have employed method specif- ically designed for multiple-choice questions to detect potential data leakage (Ni et al., 2025). For given multiple-choice instance I (Zstem: 01, o2...., on), where seem represents the question stem and 01, .2...., On are the options, we generate all possible permutations of the options, re- sulting in {01, 22 Fnd For each permutation, the sequence is decomposed into tokens as x (4, .,,,..; (m)) We then compute the confidence score for each sequence using the following method:
m--1
COnfLLM(
II (L)M()s(Z)
If the original order zo has the highest confidence score among all permutations, we consider that the multiple-choice instance may have experienced data leakage. We randomly select 500 questions from AnesBench, excluding those with excessive options. Under the assumption that the LLM has no prior knowledge and calculates probabilities randomly, this method yields a potential leakage pro- portion of 0.04. As shown in Fig. 8, most models exhibit a potential data leakage proportion around 0.10, with the exception of the Qwen and Yi model families, which are slightly higher (consistent with the evaluation results of MMLU U(Wag et al., 2023a), CMMLU(Li et al., 2024), C-Eval(Huang et al., 2023), and CMB( Wang et al., 2024a) in the original text (Ni et al., 2025)). Overall, the low potential leakage proportions of most models on AnesBench validate the quality of our work.
Proportion of Potential Data Leakage Across Different Model Families
Normal Condition
0.20
Family
CLM
Qwe
GPT
60.15
(pentratLM
0.10
0.00
Figure 8: Proportion of potential data leakage in the model. Each below the x-axis denotes a model family.
A.2 DETAILS OF ANESBENCH AND ANESRI ANNOTATION
We annotated AnesBench and AnesRI using detailed few-shot prompts with the inference model DeepSeek-R1. The specific prompts are as follows.
18

---

Published as a conference paper at ICLR 2026
Annotation Prompt
SYSTEM PROMPT
You are a anesthesiologist and helpful personal assistant, 7 proficient in categorizing questiens strictly according to instructiens.
USER PROMPT
Problem description:
[Insert the question stem and all options here]
Answer:
[Issert the correct answer here]
Classification Categories:
1. Knowledge-Based: Involves recalling fundamental facts, definitions, or procedures. This type of question primarily
4 tests factual menory without requiring application or in- O depth reasening. For example: "What is the standard dose of Propofol for adult patient undergeing general anesthesia?"
2. Application-Bascd: Involves using basic knowledge or cencepts in straightforward situations. The question requires applying known information to solve problems or make decisions, but does not require complex thinking multiple-step reasoning. example: "If patient has history of allergies local anesthetics, which alternative drug should be considerea?"
3. Beasoning-Based: Requires logical thinking, analysis, and the of problem-solving skills. This type of question challenges the ability to evaluate complex situations, reason through different factors, and make informed decisions. For example: "Given the patient's s medical history and lab results determine the most appropriate anesthetic technique for the upcoming surgery."
Instructions for Classification:
Please categorize the problem by selecting the most appropriate category that aligns with the type thinking and approach
requi red address the question. Consider factors such as
the level of complexity, the type of cognitive processing
needed, and whether the problem invel straightforward
recall, application, or in-depth reasoning. Your response categorized S\boxed(ANSWERI) where ANSWER one the numbers 1, 2, or 3.
should conclude with: "So, the problem.
A.3 DETAILS MANUAL INSPECTION IN ANESBENCH
General Review To ensure the data quality of AnesBench, we selected approximately 60% of the questions for manual inspection. Specifically, the types errors we identified and their proportions among the inspected questions are shown in Tab 6. The proportions of the above three types of erros are all very low, which can basically guarantee the data quality of AnesBench. All the problematic questions have checked above has been excluded from AnesBench.
Table 6: Summary of Manual Inspection on AnesBench
Issue Type
Count
Percentage
Format Error
0.02%
Content Mistake
5
0.1%
Missing Key Information
11
0.22%
19

---

Published as a conference paper at ICLR 2026
A.4 DETAILS OF ANESCORPUS CURATION
To construct AnesCorpus, we first retrieved anesthesia-related documents from FineWeb and Chi- nese Web based manually selected prompts. Then, through two-stage data cleaning process on AnesBench, we finally obtained AnesCorpus. The two sets of keywords we used in the first stage are as follows. The first set of keywords is highly relevant anesthesiology. The second closely related to perioperative management in amesthesiology, which helps not only focus on theoretical knowledge of anesthesiology but also obtain more content related clinical periopera- tive management.
Group
Keyword
Keywords1
Ka ME AAM HA .anasthe,analge ,sdatiion ,nerve block
Keywords2
F*
surgery.surgical.operraa operative
Table 7: Keywords used for AnesCorpus Construction
Decontamination protocol We employed a two-stage decontamination protocol to eliminate po- tential data contamination. The first stage consisted of a preliminary screening based on n-gram overlap. Specifically, we constructed an inverted index of n-grams from the AnesBench evaluation dataset, mapping each n-gram fragment to the set of questions in which it appears. We then pro- cessed the initial corpus by querying each document's n-grams against this index to compute the number of shared n-gram fragments with AnesBench. Documents exceeding predefined threshold were flagged as potentially contaminated.
The second stage implemented more stringent filtering procedure, following the methodology of HuatuoGPT-ol (Chen et al., 2024). In this phase, any document exhibiting a LCS greater than 64 characters with any item in AnesBench with this protocol, we set the character-level n-gram size to 35 and the overlap threshold to 9. During the first stage, we identified 359,598 suspicious candidates. In the second stage, we further screened and detected a total of 3,240 contaminated documents that exhibited an overlap with AnesBench greater than 64 characters.
definitively excluded from the corpus. In accordance
A.5 DETAILS OF ANESQA CURATION
AnesQA was constructed using multi-stage, LLM-based pipeline to transform existing anesthe- siology literature into question-answer pairs. The generation process began by segmenting source documents into smaller chunks based on length and section structure. For each chunk, LLaMA3.3- Ins-ruttt (Grattafiori al., 2024) was prompted
generate self-contained questions. Subse- quently, distinct model, Qwen2.5-72B- 1een2.5-72B-Insiiie Innttucttt (Yang et al., 2024), was employed to filter these questions, selecting only the most informative and domain-relevant ones answer. This two-model approach was implemented to mitigate potential self self-preferenc bias.
To address the risk of model-induced hallucinations, we instituted several quality assurance proto- cols, including model volsiaaaii and manual quality assessments on sampled subset. The value of AnesQA is further substantiated by its demonstrated ability to enhance the reasoning per- formance of L.Mss, as detailed in the main text. This finding serves an indirect validation of the dataset's ss quality and its utility for domain adaptation. The specific prompts used for the generation and answering stages provided below.
AnesQA Generation Prompt
SYSTEM PROMPT
Based the fol following text, generate series of closely related questions. Ensure that the questions are phrased in the
language as the text and cover the fellowing types:
Knowledge-based questions: Assessing fundamental knowledge and key concepts.
20

---

Open-ended questions: Encouraging in-depth thinking discussion. Ensure that the questions are varied in nature, particularly the 4 scenario questions, which shculd reflect broad range Published as a conference paper at ICLR 2026
Scenario-based questions: Designing practical application scenarios + based on the text content.
Decision-making questions: Proposing questions that require judgment or selection under specific conditions.
Inference-based questions: Asking for logical reasoning based on 4 the text information.
4 of potential real-world situations related to the text's context.
The diversity of the questions should cover different angles and + contexts, enhancing the complexity and richness. Additionally, retain the full names of abbreviations in the questions.
Ensure that the questions are closely related to anesthesia.
Example Output:
Knowledge based questions: What the key phases of general anesthesia, and what their purpose?
Scenarie-based questions: Imagine pediatric patient undergoing
9 surgery. How would the anesthetic approach di ffer from that of an elderly patient?
makini questions: If patient has a history of adverse S reactions to anesthesia, what considerations should guide the selection of anesthetic agents?
Inference-based questions: Why might patient-speciflc facters such 7 as weight and medical history significantly influence
4 anesthesia management?
Open-ended questions: What the potential challenges in balancing safety and efficacy in anesthesia for high-risk patients?
USER PROMPT
The provided text is follows:
[Insert the text here for generating related questions]
AnesQA Filtering and answering Prompt
SYSTEM PROMPT
Based on the set of questions generated from the given text, select
3 to 5 question that the most related to anesthesia and for it.
provide clear, well-structured
Ensure that each question and answer highly relevant the field of anesthesia. The responses should address the core 4 principles, practices, and challenger in anesthesia, and should not diverge into unrelated topics.
Ensure the following:
Knowledge-based questions: Choose question that asks for a + fundamental concept or core knowledge and provide a precise,
+ informative answer that fully explains the concept without referring back to the original text.
21

---

Published as a conference paper at ICLR 2026
Scenario-based questions: Select a question that invoives a practical, real-world application and provide a thoughtful 4 response, addressing all aspects of the scenario described and providing complete solution explanation.
Decision-making questions: Pick a question that requires judgment or decision-making and offer a reascned, evidence-basee answer. Ensure your response fully justifies the choice and 9 addresses the relevant factors invelved in making the decision.
Inference-based questions: Choose question that requires logical
4 reasoning or drawing conclusions, and provide a well-
9 explained answer that walks through the reasoning process and
3 clearly supports the conclusion, without relying information cutside the answer itself.
Open questions: Pick question that encourages in-depth
4 thinking and discussion, and provide a comprehensive answer
9 that explores multiple perspectives and offers complete exploration of the topic, fully addressing all aspects of the question.
Each answer should be complete and self-contained, without indirect
9 references to the original text. For each question, explicitly state which question has been selected, and then provide detailed, comprehensive answer.
For each questien, follow this structure:
Selected Question:** [State the exact question chosen] +electtd Question Type:** [State the question type, e.g.r Knowledge- Scenario-based, becisfon-making, Inference- based, Open-ended]
**AnSwer:** [Provide the complete answer without referring back to 9 the original text.
USER PROMPT
The provices ded set of questions is fal follows: [insert the set of generated quest ons here]
To ensure data quality and integrity, we performed a comprchensive technical review the AnesQA dataset. Initially, manual inspection of 100 randomly sampled QA pairs revealed specific format- ting error. Based on this finding, we developed a regular expression to systematically sean the entire dataset for all instances of this and similar errors. This automated process identified 119 problem- atic QA pairs, which were then expunged from the dataset. The resulting refined AnesQA dataset, consisting of over 20,000 QA pairs, is well-suited for its designated purpose.
22

---

Published as a conference paper at ICLR 2026
Model Group
Models
Qwen
Qwen3-0.6B, Qwen3-1.7B,
Qwen3-4B,
Qwen3-8B,
Qwen3-
14B, Qwen3-30B-A3B, Qwen3-32B, Qwen3-235B-A228 (Team,
(w25),QQQ-32B-PP (Qwen, 2025)
Qwen2.5- 14B-Itruutt Qwen2.5-33B-Instruuc,
Qwen2.5-7BB-mmstruuc (Yang et al., 2024)
DeepSeek
DeppSeek-RI-Dissill---Waaann
DepsSekk-R--Disiill---ennn DeenSeek-R--Distill--eeee Deepseek-RI (Guo et al., 2025),
DeepsSeck-V2 V3 (Liu et al., 2024a)
Gemma
Gemma-3-lb-it Gemma-3-4b-ir, Gemma-3-12b-it, Gemma-3-27b-
it (Team et al., 2025a) Gemma-2-9b-it, Gemma-2-27b-it (Team et al., 2024)
Llama
Metaa-Llama-388BIInniie
Llama-3.3-70B:
Instruct (Grattafiori et al., 2024), Llama-4-Scou--778 17B-1666 .llma-4-
Maverick-17B-I22E (Meta, 2025)
HuatuoGPT
HuatuoGPT-ol--B:
HuatuoGPT-ol-8B.
HuamoGPT-o1--00B,
HuatuoGPT-o1-722B (Chen et al., 2024)
Other Chinese Models
chatglm3-6b, glm-4-9b-char (GLM et al., 2024), internim-88b. instruct (Cai et al., 2024), Baichuan2-7B-Char Baichuan2-138: Chat (Baichuan, 2023), Yi-1.5-34B-Chat (Young et al., 2024)
Other Medical Models
BioMistral-7B (Labrak et al., 2024), FineMedL.M, FineMedl (Yu et al., 2025), Citrus1.0-Ii 10-1lama--7000 (Wang et al., 2025), Llama3- OMMMBBi Llama--OpenBBoLLMM-000 (Ankit Pal, 2024), 1aa---3--7O-UUUraMeieeee 2024), Bio-Medical-Llama-
(Zhang et al., (ContactDoctorr 2024)
Proprietary Models
GPT-4o (OpenAl, 2024a), Gemini-2.5-Flaan Gemini-2.5-Pir (Co- manici et al., 2025), Claude-3.7-Sonmee (Anthropic, 2025)
Other General Models
Phi-4 (Abdin al., 2024)
Table 8: List of Evaluation Models
B EVALUATION DETAILS
B.1 LIST OF EVALUATION MODELS
In addition to the results in Tab 3 in the main text, we have evaluated the anesthesiology reason- ing performance of more than 50 LLMs using AnesBench-English in total. The complete list of evaluated models shown in Tab. 8.
B.2 EXPERIMENTAL ENVIRONMENT FOR EVALUATION
The evaluation experiments were primarily conducted on a server equipped with eight NVIDIA A800 GPUs. The sglang framework (Zheng et al., 2024a) was employed for inference. To ensure deterministic and reproducible outputs, the sampling temperature was set to 0, with the maximum number of output tokens limited to 2048. We adopted the Zero-Shot COT methodology for prompt- ing (Wei et al., 2022). For proprietary and larger models (e.g., Llama-4-Maverick- 17B-112E,, in- ference performed via their respective APIs.
B.3 COMPLETE EVALUATION RESULTS
The complete evaluation results on AnesBench-Englist are shown in Tab. 9.
23

---

Published as a conference paper at ICLR 2026
Table 9: Evaluation Results on AnesBench-
Model
Sys1
Sysl.x
Sys2
Total
Qwen3-0.6B
0.39
0.31
0.26
0.36
Qwen3-1.7B
0,48
0.37
0.30
0.44
Qwen3-4B
0.60
0.46
0.34
0.54
Qwen2.5-7BBinntruct
0.56
0.44
0.36
0.52
Qwen3-8B
0.65
0.50
0.40
0.60
Qwen2.5-14B-Instee
0.61
0.52
0.41
0.57
Qwen3-14h
0.70
0.57
0.45
0.65
Qwen3-30B-Ar
0.73
0.60
0.48
0.68
Qwen3-32B
0.72
0.64
0.48
0.68
QwQ-32B-Previee
0.69
0.58
0.44
0.64
Bwen2.5-32B-Instruee
0.65
0.55
0.44
0.61
Qwen2.5-72B-Instruue
0.72
0.60
0.48
0.67
Owen3-235B-AV-
0.78
0.67
0.57
0.74
DeepSeck-RI--iisill-Oee
.BB
0.32
0.28
0.27
0.31
DeepSeek-
DeepSeek-RI--issill--ewesse
0.40
0.34
0.28
0.38
DeepSSek<RR--issill--veee
0.64
0.51
0.40
0.59
CrepSeek-RI-Distill--Weee
0.67
0.56
0.45
0.63
DeepSSek-RR--Diiill---mmm----
0.77
0.68
0.56
0.73
DeepSeek-V3
0.77
0.69
0.55
0.73
DeepSeek-RI
0.85
0.78
0.70
0.82
Gemma-3-1b-
0.33
0.26
0.21
0.30
Gemma-3-4b-it
0.46
0.36
0.35
0.42
Semma-2-9bi1
0.53
0.40
0.36
0.49
Gemma-3-12b-it
0.56
0.46
0.36
0.52
Gemma-2-27b-it
0.60
0.43
0.36
0.54
Gemma-3-27b-it
0.63
0.52
0.40
0.58
Meta-Llama---SD-Insttuu
0,54
0.42
0.39
0.50
Lama-3.1-8B-Inssee
0.58
0.45
0.36
0.53
0.63
0.47
0.41
0.57
Jlama-333-70B-IIntrnctt
0.74
0.63
0.51
0.70
0.73
0.60
0.47
0.68
Jamas-OpennBoLLLMM
0.68
0.55
0.44
0.63
Jama-4-Scouu-11B-166
0.77
0.66
0.55
0.72
Jama-4-Maverick1177-PD
0.83
0.73
0.64
0.79
HuatooGPT--1-77B
0.56
0.45
0.38
0.52
HuatooGPT-o1-8B
0.58
0.46
0.39
0.53
HuatuoGPT-ol-70B
0.70
0.58
0.48
0.65
HuatuoGPT-o11-228
0.71
0.61
0.48
0.67
chatglm3-6b
0.37
0.28
0.25
0.34
glm-4-9b-chat
0.48
0.36
0.36
0.44
intemmlm3-Sb-instruee
0.60
0.43
0.40
0.54
Baichuan2-7B-Chat
0.39
0.31
0.30
0.37
Baichuan2-13BB-Chatt
0.42
0.31
0.34
0.39
Yi-1.5-34B-C
0.54
0.44
0.35
0.50
BioMistral-7B
0.43
0.30
0.32
0.39
FineMedLM
0.40
0.35
0.27
0.38
FineMedI.M-ol
0.43
0.34
0.26
0.39
Citrus 1a-mus1..0lam--000
70B
0.71
0.60
0.52
0.67
Jaam3--pennBiolL.MM.a
0.44
0.35
0.30
0.41
Bio-Mediiall-Lamm---88
0.53
0.41
0.38
0.49
GPT-40
0.81
0.72
0.59
0.77
2.5ini-2.5-Flash
0.84
0.76
0.68
0.81
Gemini-2.5-Pro
0.89
0.82
0.77
0.86
Claude-3.7-Sood
0.80
0.73
0.63
0.77
phi-4
0.69
0.57
0.41
0.64
24

---

Published as a conference paper at ICLR 2026
C TRAINING EXPERIMENT DETAILS
C.I. EXPERIMENT DETAILS OF USING ANESQA FOR SFT
In the ablation study presented in Section 5.2, utilized the AnesQA dataset domain-specifer data to evaluate its effectiveness in enhancing the anesthesiology reasoning capabilities of LLMs. As AnesQA consists exclusively of English question-answer pairs, we translated it into Chinese using GPTT4o-mini (OpenAI, 2024b). This step was taken to ensure the reliability of our results on the AnesBench- Chineee by mitigating the influence of language distribution and to align with the bilingual configuration of the Medical-ol dataset. The specific prompt used for this translation process is provided below.
Prompt used for Translating AnesQA from English to Chinese
SYSTEM PROMPT
You are professional translater with expertise in medical and anesthesia-related terminology.
Your task to accurately translate anesthesia-related QA pairs
4 from English to Chinese,
ensuring that professional terms are translated precisely and
4 consistently.
Maintain the original meaning, tone, and level of technical detail
while ensuring readability for a Chinese speakinn medical 4 audience.
Do not add or omit any information.
USER PROMPT
Please translate the following amesthesia QA pair from English to Chinese,
ensuring that all medical terms, abbreviations, and technical 4 expressions are translated precisely and appropriately for 4 professional audience.
Maintain clarity, coherence, and fidelity to the original meaning.
Input (English QA Pair):
[Insert the English QA Pair here]
C.2 EXPERIMENT DETAILS FOR BASELINE MODEL
The training of the baseline model comprises two primary stages: SFT and GRPO. All training experiments were conducted server with 8 NVIDIA A800 (80G) GPUs. For the SFT stage, employed the Llamafactory framework (Zheng et al., 2024b). The key training hyperparameters are detailed in Tab. 10.
Table 10: Key Training Hyperparameters of SFT
Category
Hyperparameter
Value
Cutoff Length
4096
Training Setup
Num Train Steps
100
Learning Rate
1.0 10
Optimization
Warmup Ratio
0.175
LR Scheduler
cosine
Per-device Train Batch Size
16
Batch
Gradient Accumulation Steps
In the second stage, we employed the Verl framework (Sheng et al., 2025) for model training using GRPO. The primary training parameters were configured follows: training batch size of 512 and
25

---

Published as a conference paper at ICLR 2026
mini-batch size of 256 for each gradient update. The maximum response length was constrained to 2048 tokens. For each prompt, the group size for rollout was set to 5. The training process was conducted for maximum of 4 epochs, with learning rate of 1 x 10 and learning rate warmup ratio 0.3. Under this configuration, the Morpheus-7h5 7B Morpheus-32B and Morpheus- 14B models were trained for 50, 50 and 40 steps, respectively. For the Morpheus-32B, due to the constraints of computing resources, we adopted the parameter-efficient fine-tuning method of LoRA. Among them, the rank of matrix factorization was
128, and the scaling factor a
32.
In our specific training process, we exclusively employed an outcome-based binary reward mech- anism: a reward of is assigned if and only if the model's final output matches the ground truth; otherwise, the reward is O. To accurately extract the final answer from the model's output, we uti- lized robust regular expressions demonstrate the stability of the training process, we conducted three independent training runs for the Morpheus-7BB model. As illustrated by the reward curves Fig 9 and Fig 10, the training process exhibits significant stability across both the 7B and 14B models.
strictly match patterns such as "Answer is" or "Answer:". To
for Morpheus-77
of GRPO Morpheus-144
1.
Mean Standard Deviation
33-
33-
X1-
Rorpent
0.-
35.
Training Step
ning
Figure 9: Reward curve of Morpheus7B
Figure 10: Reward curve of Morpheus 14B
Furthermore, we trained multiple models using the AnesRI dataset with the number of training steps extended to 200. As shown in Tab 11, the results demonstrate significant performance improvements across both the Chinese and English subsets of AnesBench, validating the high quality and rich knowledge encapsulated within A Notably, as shown in Tab. 4, the Qwen model exhibited decline in the Chinese subset following SFT, which may be attributed an alignment tax resulting from an insufficient number of SFT steps intended for domain alignment.
Model
AnesBench-Engee English
AnesBench-Chinge
Avg.
1aama-3.1-8B-Insttee Llama-3.-SBB(Aesk))
0.54
0.48
0.51
0.60
0.51
0.55
Gemma2-22 ---naa---BBaane Insstruct
0.40
0.34
0.37
Gemma2-2B 2B(nnesR1)
0.45
0.41
0.43
Gemma3 4B- 4B-Iruct G 4B(AnesR1)
0.32
0.40
0.36
0.36
0.43
0.39
Table 11: Additional Training Experiments with AnesR1
Finally, we conducted supplementary experiments using the InternLM--8B-Innttn (Cai et al., 2024) model. The training dataset employed was AnesR1, consistent with the setup used to train the Morpheus model the main text. The number of training steps for SFT and GRPO was set to 100 and 60. respectively. As illustrated in Tab. 12. following SFT and GRPO training AnesR1, the model demonstrated sigmificant improvements across all three cognitive dimensions, thereby indicating the strong adaptability of dataset across diverse foundation models.
26

---

Published as a conference paper at ICLR 2026
Model
Systeml
System1.x
System2
Overall
InternLM3-83 M----BB-nnnttuttt
0.59
0.45
0.39
0.54
InternI ---8 (SFT + RL.
0.62
0.50
0.43
0.58
Table 12: Results of fine-tuned InternLM3 on AnesBench-English
C.3 EXPERIMENT DETAILS FOR ABLATION STUDY
Hardware and Software All experiments were conducted on 8 NVIDIA A800 (80G) GPUs, interconnected via NVLink. We utilized Llama-Factory as the training infrastructure. Performance was further enhanced by leveraging BF16 mixed- precisson,, FlashAttention-22 2, and the LIGER kernel.
Continued Pre-training (CPT) The CPT stage involved full-parameter fine-tuning of the Qwen2.5-7B base model. We set the maximum sequence length to 2048 tokens. The model was trained for 3 epochs using DeepSpeed with ZeRO Stage I configuration. Key hyperparameters included an effective batch size of 64 (a per-device batch size of 8 with gradient accumulation steps), peak learning rate of 1.0 10-4 decayed with cosine annealing schedule, and linear warmup phase over the initial 5% of training steps.
Supervised Fine-tuning (SFT) Starting from the CPT model, we performed full-parameter su- pervised fine-tuning. The maximum sequence length increased to 4096 tokens. This stage for 5 epochs, employing a more memory-efficient DeepSpeed ZeRO Stage configuration. The hyperparameters were adjusted to effective batch size of 32 per-device batch size of 8 with accumulation steps) and a reduced peak learning rate of 1.0 x 10-5, also with a cosine schedule. The warmup period was extended 10% of the total steps, and gradient clipping with a maximum norm of 5,0 was applied to ensure stability.
D ANESCORPUS VISUALIZATION
As illustrated in Fig. 11, our constructed anesthesia-related corpus encompasses a diverse range of medical terminologies, both in English and Chinese The English word cloud highlights key terms such "general anesthesia", "local anesthetic", "surgical procedure", and "pain manage- ment", reflecting a strong focus on clinical applications, patient care, and perioperative consider- ations. Additionally, frequent mentions of terms like "nerve block", "blood pressure", and "side effect" indicate a broad coverage of anesthesia techniques and associated physiological impacts. Similarly, the Chinese word cloud presents comparable distribution, with dominant terms such as "(uugerr"""""" "i (treatment)", dural and treateent-related aspects. The presence of terms related medical interventions, such as "/i/ (intubation)" and "2 (medication)" exhibits emphasis on clinical procedures and pharmacological considerations.
(anestheiia) indicating strong alignment with proce-
pattent &
denital impllant
Masestii Wri kg Side effect
Saaie Ban pain management
local anesthetic
T
plestic sur surgzon
blood pressure
Sper alting Feaiti
plashic Sursery
general anesthesfa
optter
surgical procedure
Werdom mn tenth
nerve block
Figure 11: Word clouds of CPT corpus.
E CASE STUDY
To highlight the challenges inherent in specialized medical reasoning, we conducted a compar- ative case study featuring Gemini-2.5-Pro, DWwen..5-14B-Instuuu and Morpheus-144 14BB. The cases
27

---

Published as a conference paper at ICLR 2026
presented are those in which Morpheus-144 14B provided the correct answer while other two models did not. The objective of this analysis is not to assert the overall superiority of Morpheus- -anae Rather, it is to illustrate that for highly specialized discipline such as amesthesiology. professional knowledge and general reasoning capabilities are insufficient. Subtle distinctions in clinical sce- narios lead to fundamentally different outcomes, underscoring the need for more advanced and specialized reasoning.
For example, in Case Gemini-2.5-Pro demonstrates clear reasoning and accurately recognizes that the y-opioid receptor is blocked. However, it fails to acknowledge that naltrexone exhibits affinity not only for the μ receptor but also for the к and opioid receptors. As a result, the combined use of naltrexone and nalbuphine cannot achieve an analgesic effect. Consequently, option "-"onnii naltrexone with small doses of nalbuphine every 4 hours as needed", incorrect.
In Case 2, the reasoning of Gemini-2.5-Prr was almost correct; however, it failed to recognize that its rationale is valid only in the context of severe hypotension. According the case description, the clinician's 's primary concern should be pump failure and hypoperfusion rather than hypotension. As a result, the model's recommendation was premature in the specific clinical scenario.
The following are the specific contents of the two cases.
Case 1: Postoperative Pain Management
Question: A 37-year-old man is brought to the operating room for repair of a broken mandible sus- tained in a motor vehicle accident. No other injuries are significant. The patient has been in treatment for alcohol abuse and takes disulfiram and naltrexone. Which of the following would be the best technique for management this patient's 's postoperative pain?
A. Continue naltrexone with round-the-clock low-dose methadone B. Continue naltrexone with small doses of morphine every 4 hours as needed C. Continue naltrexone with small doses of nalbuphine every 4 hours as needed D. Discontinue naltrexone and treat pain with morphine needed
Correct Answer: D
Gemini-2.5-Pro (Incorrect): This is multiple-choiee question about postoperative pain management in a patient on naltrexone, Let's break down the options step-by-step.
**Patient Profile:** 37-year-old male undergoing repair of a broken mandible. This is expected to be painful procedure. History of alcohol abuse. Current medications: Disulfiram and Naltrexone. competitive opioid antagonist, primarily the mu-opioid receptor. is used to prevent relapse in alcohol and opioid use disorders. By block- ing mu-receptors, it will render typical opioid agonists (like morphine, fentanyl, oxycodone) ineffec- tive at standard doses. **Mopphine & Methadone:** These are pure mu-opioid receptor agonists. Their analgesic effects are mediated through the mu-receptor. **Nalbuphine:** A mixed agonist- antagonist opioid.
**eyy Pharmacological Concepts:** *** Naltrexoone:
an agonist at the kappa-opiois receptor (which provides analgesia) and an antagonist at the mu-opioid receptor.
"* **Analysis of Options:**
**A. Continue naltrexone with round-the-cleck low-dose methadone:** This is incorrect. Methadone is mu-opioid agonist. Naltrexone is mu-opioid antagonist. Administering methadone to a patient on naltrexone will result in lack analgesic effect because the mu-receptors are blocked by naltrexone.
#*B, Continue naltrexone with small doses of morphine every 4 hours as needed;* This is incorrect for the same reason option A. Morphine is a pure mu-opioid agonist. Its analgesic effect will be blocked by naltrexone. Overcoming this blockade would require dangerously high doses morphine, which is not a safe or predictable approach.
.CC. Continue naltrexone with small doses of nalbuphine every 4 hours as neoded: This is the most suitable option. Naltrexone primarily blocks the mu-opioid receptors. Nalbuphine provides analgesia through its agonist activity the kappa-opioid receptors. Since naltrexone does not significantly block kappa receptors, the analgesic effect nalbuphine should be preserved. This allows for effective pain management while continuing the naltrexone therapy, which is important for preventing alcohol relapse.
=*D. Discontinue naltrexone and treat pain with morphine as needed;** This is not the best approach. Naltrexone has long duration of action (its active metabolite has half-life life of hours). To allow morphine to effective, naltrexone would need to be discontinued for at least 72 hours prior to surgery. This is often not fcasible in an acute setting like a trauma case. Furthermore, discontinuing naltrexone, especially during stressful period like surgery, significantly increases the patient's risk of relapsing to alcohol use. The goal should be manage pain while maintaining the patient's substance abuse treatment possible.
28

---

Published as a conference paper at ICLR 2026
**Conclusion:** The best strategy among the choices is to use analgesic that bypasses the mu- receptor blockade. Nalbuphine, with its kappa-agonist activity, fits this requirement. This approach allows for pain control without interrupting the patient's 's crucial naltrexone therapy for alcohol abuse. **Final Asswer: The final answer #*)()**
Morpheus-144 (Correct); First, the patient is 37-ycar-old man with broken mandible from a motor vehicle accident. He's in treatment for alcohol abuse and is taking disulfiram and naltrexone. I need to manage his postoperative pain.
Disulfiram is used to treat alcoholism by causing unpleasant reaction when alcohol is consumed, so he shouldn't 't drink. Naltrexone an opioid antagonist that blocks the effects of opioids, used prevent relapse by reducing the rewarding effects alcohol and opioids.
The key point is that naltrexone blocks opioid receptors, so give him opioids for pain, they won't work because naltrexone is antagonizing the receptors. That means I can't 't use standard opioid anal- gesics like morphine.
Now, looking at the options:
A. Continue naltrexone with round-the-clock low- methadone
B. Continue naltrexone with small doses of morphine every 4 hours as needed
C. Continue naltrexone with small doses of nalbuphine every 4 hours needed
D. Discontinue naltrexone and treat pain with morphine as needed
Option B says to continue naltrexone and use morphine. But since naltrexone blocks opioid receptors, morphine won't 't be effective. That seems counterproductive.
Option C mentions nalbuphine. Nalbuphine is an opioid agonist- agonist-antagonet antagonist. It has partial agonist activ- ity at kappa receptors and antagomist activity at mu receptors. Since naltrexone is already blocking mu receptors, nalbuphine might work well hecause it's an antagonist at receptors too. Nalbuphine could actually precipitate withdrawal opioid-dependent patients, but here it 's for alcohol abuse, not opioid dependence. Still, with naltrexone blocking, it might not be effective.
Option A has methadone. Methadone is full opioid agonist, but it's long-acting. If naltrexone is blocking, methadone might work. Also, methadone used for maintenance opioid addiction, but here it's for pain, and with naltrexone, could be ineffective.
Option says to discontinue naltrexone and treat with morphine. That makes sense because if stop naltrexone, the opioid receptors are free, and morphine can work. But need to consider the context. He's in treatment for alcohol abuse, and naltrexone is part of that. Discontinuing it might compromise his treatment for alcoholism.
The question about postoperative pain management. The surgery is for broken mandible, which can be painful, but it's not life-threatening. need to balance pain control with the patient's 's ongoing treatment.
Disulfiram is also mentioned, but doesn't directly affect opioid management; it's for alcohol.
Naltrexone is key here. It's an opioid antagonist, so any opioid give will be blocked.
Therefore, can't 't use opioids while he's on naltrexone. need an alternative pain management strategy. Option C has nalbuphine. Nalbuphine is an opioid agonist-antagonist It has kappa agonist and mu antagonist properties. Since naltrexone is blocking receptors, nalbuphine might still have some kappa agonist effects, but could also precipitate withdrawal not be effective.
In practice, for patients on naltrexone, we avoid opioids and use non-opioid analgesics. But let's see the options again. Option C says "small doses of nalbuphine every 4 hours as needed." Nalbuphine is sometimes used in such cases, but it's not ideal because of the antagonist properties. Perhaps I should look for non-opioid option, but it's not listed.
Option D: discontinue naltrexone and use morphine. That seems straightforward, but it might not be ethical or practical to discontinue naltrexone just for pain management.
The patient has no other significant injuries, so pain is manageable. I recall that in anesthesia, for patients on naltrexone, we often discontinue it hefore surgery and restart after, non-opioid analgesics. But here, the options don't include non-opioids. They all involve some form opioid or naltrexone continuation.
Option A has methadone, which is an opioid, so it won't work with naltrexone. Option B has morphine, which won't 't work.
Option C has nalbuphine, which might have some efficacy but is not reliable. Option D: discontinue naltrexone and use morphine. This might be the best choice because it allows effective pain control.
But discontinuing naltrexone could be problematic. Naltrexone is used to prevent relapse, stopping it might increase the risk of alcohol relapse.
The surgery for a broken mandible, which might not directly involve alcohol, but pain could be a stressor.
Perhaps the best approach is to discontinue naltrexone temporarily for the duration of the pain, and then restart it.
29

---

Published as a conference paper at ICLR 2026
In many guidelines. for patients naltrexone, it's recommended hold the medication before elective surgery and restart after the acute pain period. Since this is a motor vehicle accident, it might not be elective, but the question is about postoperative management.
The question says "repair of a broken mandible," so it's likely planned surgery, but the injury was from accident.
I think the most suitable answer is D. discontinue naltrexone and treat with morphine as needed. But let's see why the others are wrong.
A: Methadone won't I work with naltrexone.
B: Morphine won't work.
work well due antagonist properties, and could cause issues. D: Discontinue naltrexone and use morphine allows effective pain control.
C: Nalbuphine- - might
Also, disulfiram is not relevant here.
So. I think D is the best choice.
But is it ethical to discontinue naltrexone? The question doesn't 't specify, so have to go with the most effective pain management.
Perhaps in some contexts, we use non-opioid analgesics, but they're not options.
Option C has nalbuphine, which is an opioid, but it's not pure agonist. Nalbuphine is sometimes used in patients naltrexone because it has kappa agonism, but it can cause dysphoria other issues.
think D is the standard approach.
Final decision: D.
The patient is on naltrexone, opioid antagonist used for alcohol abuse treatment, which blocks opi- oid receptors. This means standard opioid analgesics like morphine (options B and D) methadone (option A) will ineffective due to receptor blockade. Nalbuphine (option C) an opioid agonist- antagonist with partial kappa agonist and mu antagonist properties; however, in the presence of nal- trexone, it may not provide reliable analgesia and could precipitate withdrawal or cause dysphoria. Option D involves discontinuing naltrexone temporarily for the postoperative period, allowing effec- tive morphine for pain control, and then restarting naltrexone after the acute pain resolves. This approach balances pain management with the patient's 's ongoing alcohol treatment, as the mandible repair is not life-threatening and pain can be managed with opioids without compromising long-term naltrexone therapy.
Thus, the best technique is to discontinuc naltrexone and treat pain with morphine needed. ** Answer: D**
Swen-2.5-14B-Instruu (Incorrect); To determine the best technique managing postoperative pain in this patient, we need to consider the pharmacological interactions and the patient's medical history.
1. **Disulfiram and "a The patient taking both disulfiram and naltrexone for alcohol abuse. Disulfiram is an alcohol deterrent that causes unpleasant reactions if alcohol is consumed. Naltrexone is an opioid antagonist used to reduce cravings for alcohol and opioids.
2. **Opioid Managemen Since naltrexone is an opioid antagonist, blocks the effects of opioids. Therefore, using traditional opioids like morphine would he less effective because naltrexone would reduce their analgesic effects.
#* Alternative *nalgesics**: Given that naltrexone being used, we need consider alternative analgesics that are not opioids or that have a different mechanism of action. Nalbuphine is a kappa- opioid receptor agonist and mu-opioid receptor antagonist, which means it can provide pain relicf without being fully blocked by naltrexone. Let's analyze cach option:
**Option ***; Continuing naltrexone with low-dose methadone. Methadone is an opioid and would he significantly reduced in effectiveness by naltrexone. **Option B*#; Continuing naltrexone with small doses of morphine. Morphine is also opioid and would be reduced in effectiveness by nal- trexone. **Option C*.; Continuing naltrexone with small doses of nalbuphine. Nalbuphine is less likely to be fully blocked by naltrexone continuing naltrexone and treating pain with morphine. Discontinuing naltrexone might not be ideal as is part of the patient's 's treatment for alcohol abuse. Considering the above points, the best option would be blocked by naltrexone.
its mixed mechanism of action. **Option D**; Dis-
analgesic that is less likely to be fully
**Final Answer: C. Continue naltrexone with small doses of nalbuphine every 4 hours as needed
30

---

Published as a conference paper at ICLR 2026
Case 2: Cardiogenic Shock
Question: A 70-ycar-old man presents to the hospital with chest pain and dyspnea. ECG shows 2-mm ST elevations in V2-V4. His vital signs are as follows: HR 85, BP 100/55, SpO2 95% room air. Laboratory test results are remarkable for troponin 3.5 ng/mL. and lactate 6 mmml/L. Emergent cardiac catheterization is planned. What other treatment should be initiated at this time?
A. Dopamine B. 1 L crystalloid bolus C. Dobutamine D. Norepinephrine E. No other intervention necessary besides immediate cardiac catheterization
Correct Answer: C
Gemini-2.5-Prr (Incorrect): This is step-by-step analysis of the question and options.
1. ** A***nllyze the Patient's Clinical Presentation:** a +*Diagnosis: +* The patient presents with classic symptoms an acute myocardial infarction (chest pain, dyspnea). The ECG confirms an ST-segment elevation myocardial infarction (STEMI) in the anterior leads (V2-V4), which typically involves the left anterior descending (LAD) artery. The elevated troponin level further confirms myocardial necro- sis. ** Hemodyynamics " The patient is hypotensive with a blood pressure of 100/55 mmHg. The mean arterial pressure (MAP) is approximately
mmHg 52 + 100y3]. While a MAP of 70 might seem borderline acceptable, the systolic pressure is low, and more importantly, there evidence of end-organ hypoperfusion. * **Evidence of Shock: The elevated lactate level of 6 mmol/I. is a critical finding. It indicates significant tissue hypoperfusion and anaerobic metaholism, which is the hallmark of shock. # **yythhesis:** The combination of a large anterior STEMI (a common cause of pump failure), hypotension, and elevated lactate establishes the diagnosis of cardiogenic shock. The primary problem is the heart's inability to pump effectively, leading to inadequate cardiac output and systemic hypoperfusion.
2. "*Evaluate the Treatment Options:** ** A. Dopamine:** Dopamine is vasopressor and inotrope. However, clinical trials (c.g., the SOAP II trial) have shown that patients with cardiogenic shock, dopamine is associated with a higher incidence of arrhythmias and potentially higher mortality com- pared to norepinephrine. Therefore, it is no longer considered the first-line agent for cardiogenic shock. #*B. L. crystalloid bolus:** In cardiogenic shock, the primary problem pump failure, not volume depletion. The left ventricle is already failing and likely has a high filling pressure (left ventricular end-diastolic pressure). Administering a large fluid bolus would further increase this pressure, poten- tially worsening pulmonary edema (as suggested by the patient's 's dyspnea) without improving cardiac output, according to the Frank-Starling mechanism in a failing heart. Fluid administration should be done with extreme caution, if at all, in cardiogenic shock. #+C. Dobutamine: * Dobutamine is pri- mary inotrope (increases contractility) with some vasodilatory effects (due to heta-2 agonism). While can improve cardiac output, its vasodilatory properties can worsen hypotension, which is danger- ous in a patient who is already hypotensive. It is often used after blood pressure has been stabilized with a vasopressor or in patients with cardiogenic shock who are not significantly hypotensive. Starting dobutamine as the initial agent in this patient would be inappropriate. #*D. Norepinephrine:#** Nore- pinephrine the first-line vasopressor recommended by current guidelines (c.g., European Society of Cardiology, American Heart Association) for the management of cardiogenic shock. It is a potent alpha-l agonist, which causes peripheral vasoconstriction and increases systemic vascular resistance (SVR). thereby raising blood pressure. It also has modest beta-1 agonist effects, which can provide some inotropic support without causing the significant tachycardia or arrhythmogenicity associated with dopamine. By increasing diastolic blood pressure, also improves coronary perfusion pressure, which is crucial in the setting of an MI. Initiating a norepinephrine infusion is the most appropriate step to stabilize the patient's 's blood pressure and improve organ perfusion while awaiting definitive revascu- larization. # #*E. No other intervention necessary besides immediate cardiac catheterization:** While emergent cardiac cathetcrization the definitive treatment address the underlying cause of the shock (the occluded coronary artery), the patient is hemodynamically unstable. Delaying hemodynamic sup- port until arrival in the cathcterization lab is unsafe and increases the risk of cardiac arrest and further end-organ damage. The patient must be stabilized pharnacologically a bridge to revascularization. #CConclusion:** The patient is in cardiogenic shock secondary an anterior STEMI. The immediate priority, alongside preparing for emergent revascularization, to correct the hypotension and improve tissue perfusion. Norepinephrine is the recommended first-line vasopressor for this purpose.
**Final Answer: The final answer is *****
Morpheus-14B (Correct): First, the patient is a 70-year-old man with chest pain and dyspnea. The ECG shows ST elevations in V2-V4, which suggests anterior STEMI (ST-elevation myocardial infarction). His vital signs are: HR 85 (which is normal). BP 100/55 (hypotensive, as diastolic is low), SpO2 95% on room air (good oxygenation). Lab results: troponin 3.5 ng/mL (elevated, indicating myocardial injury), lactate 6 mmol/L (high, indicating tissue hypoperfusion or shock). He's scheduled for emergent cardiac catheterization, which is appropriate for STEMI. Now, the question is: what other treatment should be initiated this time? The options are:
31

---

Published as a conference paper at ICLR 2026
A. Dopamine
B. L. crystalloid bolus
C. Dobutamine
D. Norepinephrine
E. No other intervention necessary besides immediate cardiac catheterization
I need to think about the patient's condition. He has hypotension (BP 100/55), which is concerning. Hypotension in STEMI patient can indicate cardiogenic shock or low perfusion.
Lactate is elevated at 6 mmol/L. which is high. Normal lactate is less than 2 mmol/L, so this suggests significant tissue hypoperfusion, likely due to the heart attack.
In STEMI, especially with hypotension and elevated lactate, need to address the shock state. The goal is to improve perfusion and support the heart. Let's evaluate each option:
A. Dopamine: Dopamine is a vasopressor that can increase heart rate and contractility at higher doses. It's used in shock states, but it's not first-line for cardiogenic shock from STEMI. Dopamine can have arrytthogennc effects and isn't preferred in acute coronary syndromes.
B. crystalloid bolus: This is fluid resuscitation. In hypotension. fluids can be given to increase preload and cardiac output. However, in cardiogenic shock from STEMI, fluid boluses are generally avoided because they can worsen pulmonary edema or increase left ventricular end-diastolic pressure. The patient has dyspnea, which might indicate heart failure or pulmonary issues, fluids could be harmful.
C. Dobutamine: Dobutamine is a beta-agonist inotropic agent that increases cardiac contractility and heart rate. It's often used in cardiogenic shock to support cardiac output. In STEMI with hypotension, dobutamine can be beneficial to improve perfusion while waiting for reperfusion.
D. Norepinephrine: Norepinephrine is a potent vasopressor that increases systemic vascular resistance and blood pressure. It's commonly used septic shock or other distributive shocks, in cardiogenic shock, can he used if other agents fail. However, might not be the first choice hecause it can cause vasnconstriction, which might increase afterload and reduce coronary perfusion.
E. No other intervention necessary: But the patient is hypotensive and has elevated lactate, indicating shock. Immediate catheterization is planned, but we need to stabilize him before or during transport. The patient is going for emergent catheterization, so interventions should support him until reperfusion. In STEMI management guidelines, for patients with cardiogenic shock, inotropes like dobutamine are recommended to support hemodynamics while PCI is performed.
Also, lactate level of 6 mmol/L is high, suggesting severe shock. We need to address this urgently. Option crystalloid bolus, not idcal because could worsen the situation. Option A, dopamine, is less preferred due to potential side effects.
Option D, norepinephrine, might he used if dobutamine isn't it sufficient, but it's not first-line for car- diogenic shock from STEMI. The most appropriate choice is C, dobutamine, because it directly supports cardiac contractility without increasing afterload excessively.
In some protocols, norepinephrine is used, but dobutamine is more specific for inotropy in heart failure. The patient has dyspnea, which could he due to pulmonary edema, so inotropes might help by increas- ing cardiac output and reducing congestion.
Also, hypotension needs correction.
So. think C. Dobutamine is the best choice. But let's confirm the BP: 100/55. Systolic is 100, which is low for a 70-year-old, and diastolic is 55, which is very low. This indicates poor perfusion. In STEMI with hypotension, dobutamine is often initiated support the heart.
Moreover, lactate high, so we need improve perfusion. Therefore, the answer should be C. Dobutamine.
The patient presents with ST-elevation myncardial infarction (STEMT) confirmed by ECG, hypotension (BP 100/55), elevated troponin (3.5 ng/ml.), .), and significantly elevated lactate (6 mmol/L.) J), indicating cardiogenic shock. The emergent cardiac catheterization is appropriate, but additional interventions are needed to support hemodynamics and improve tissue perfusion before reperfusion.
**Option A (Dopamine)**;; Dopamine is a vasopressor that can increase heart rate and contractility at higher doses but is not first-line for cardiogenic shock in STEMI due to potential trrhythmogenic effects and lack of strong evidence for benefit in this setting. *ptpioo B (1 L crystalloid bolus)**, Fluid resuscitation is generally avoided in cardiogenic shock from STEMI because can worsen pul- monary edema or increase left ventricular end and-diastolic pressure, exacerbating dyspnca and heart fail- ure. - *OOption C (Dobutamine)**: Dobutamine is a cota-agonist inotropic agent that increases cardiac contractility and heart rate, improving cardiac output and perfusion without significantly increasing af- terload. It is recommended in guidelines for cardiogenic shock in STEMI to support hemodynamics while awaiting reperfusion. **Option D (Norepinephrine)+* Norepinephrine potent vasopressor that increases systemic vascular resistance and blood pressure but can reduce coronary perfusion by
32

---

Published as a conference paper at ICLR 2026
increasing afterload. is typically reserved for refractory shock when inotropes like dobutamine insufficient. **Option E (No other intervention necessary)*** The patient's 's hypotension and elevated lactate indicate shock, requiring immediate intervention to prevent organ damage. Delaying treatment could worsen outcomes.
The most suitable choice is **C. Dobutamine**, as it directly addresses the cardiogenic shock by enhancing cardiac contractility, supporting perfusion, and reducing the risk of complications during transport catheterization.
Qwen-2.5-144 2a--2..5-1--Insruue (Incorrect): Let's analyze the patient's condition step by step:
1. **Clinical Presentation The patient is a 70-year-old male with chest pain and dyspnea, which are classic symptoms of possible acute coronary syndrome (ACS), specifically STEMI (ST-elevation myocardial infarction) given the ECG findings of 2-mm ST elevations in V2-V4.
**Vital Signs**; HR 85, BP 100/55, SpO2 95% on room air. blood pressure notably low (hypotension). which is concerning and suggests possible cardiogenic shock.
3. #**aabrrtorry Results Troponin 3.5 ng/mL (elevated, indicating myocardial damage) and lactate 6 mmol/l. (elevated, indicating tissue hypoperfusion and possible shock).
**Management** Given the patient's 's presentation, emergent cardiac catheterization is appropriate.
However, the patient's hypotension and elevated lactate suggest the need for immediate hemodynamic support.
Now, let's evaluate each option:
A. **Dopamine**; Dopamine be used in shock states, but it has dose-dependent effect. At low doses, can increase renal and splanchnic blood flow, but at higher doses, can increase myocardial oxygen demand and potentially worsen ischemia. is not the first-line agent for hypotension in this setting.
B. **1 L Crystalloid Bolus**: This is reasonable initial step in hypotensive patients to assess for volume responsiveness. However, given the patient's low blood pressure and elevated lactate, likely that patient in cardiogenic shock rather than hypovolemic shock, making this less likely to he the primary intervention.
C. ** *Dobuaamine This is beta-1 selective inotrope that increases cardiac output by increasing myocardial contractility. It is often used in cardiogenic shock to improve cardiac output and perfusion. However, it does not address the hypotension directly.
D. *NNorppinphrrin=** This is a potent alpha-1 agonist that increases systemic vascular resistance and can be used to treat hypotension in cardiogenic shock. It is the most appropriate choice for imme- diate hemodynamic support this patient.
E. No other intervention necessary besides immediate cardiac catheterization"*: Given the patient's hypotension and elevated lactate, immediate hemodynamic support necessary to stabilize the patient before cardiac catheterization.
Considering the patient's hemodynamic instability and the need for immediate support, the most ap- propriate initial treatment is:
**D. Norepinephrine***
This choice addresses patient's hypotension and provides the necessary hemodynamic support while preparing for emergent cardiac catheterization.
F SIGNIFICANCE TEST
To rigorously validate the claims presented in Sec. 5.1, we conducted series of statistical tests to confirm the validity and significance of our insights. This section presents the results of these tests and discusses additional findings uncovered during the analysis.
Methodology By default, all comparisons evaluated using bootstrap resampling. We report the difference in means (A), the 90% confidence interval (CI, corresponding to a 0.1), and the one- two-sided p-value.
F.I SIGNIFICANCE TEST ON MODEL CHARACTERISTIC
Performance Across Model Scale Our analysis confirmed the relationship between model seale and performance as described in Sec. 5.1. The claim of a "strong positive correlation between model performance and scale" was strongly supported by Spearman correlation tests across all systems. As detailed in Tab. 13, the Spearman's 's rank correlation coefficient (p) exceeds 0.8 for all three systems,
33

---

Published as a conference paper at ICLR 2026
while the corresponding p values approach zero. significant positive association.
These results indicate strong and statistically
Table 13: Statistical Test for Model Scale
Table 14: Statistical Test for Marginal Return
System
Ps
p-value
System
82
p-value
Systeml 0.859
1.49e-15
Systeml
-2.79
0.006
Systeml.x
0.861
1.07e-15
Systemlx
-2.607
0.007
System2 0.818
3.
System2
-1.920
0.008
The hypothesis of diminishing marginal returns was also supported. We employed a quadratic or- dinary least squares (OLS) model to examine the relationship between model size and performance score. For this analysis, the alternative hypothesis posits that the coefficient of the quadratic term, 32. less than zero. The results of one-sided tests confirmed statistically significant negative co- efficient for this quadratic term across all evaluated systems (all < 0.025), indicating the presence of diminishing returns.
To evaluate the claim that "the slope for System2 is significantly lower than Systeml.x, xx indicating that performance gains from increasing model size are markedly smaller for System2," we employed a linear model. The model is defined as:
y = 3o + f1 + B2 group + fa(x x group) +
(2)
In this model, the group variable is binary indicator, specifying whether the data point originated from Systeml.x xx or System2. We conducted a one-sided bootstrap hypothesis test with the alternative hypothesis that B3 is greater than zero
Our tests revealed this holds true when comparing System 2 to System 1.x (p=0.0009). In con- trast, the difference between System 2 and System (factual recall) was statistically significant (5=0,1538), suggesting that the scaling advantag diminishes most sharply when moving from hy- brid reasoning to complex decision-makingg
Performance Across Language Statistical analysis corroborates the cross-lingual observations detailed in Section 5.1. To test the hypothesis that English accuracy exceeds Chinese accuracy (H1 ACCEN > AcсcH). we performed bootstrap tests on the AnesBench Chinese-English mutual translation benchmark. The results strongly indicate that models based on Llama-3.1 perform notably worse in Chinese. For all evaluated models in this series, this performance disparity was sta- tistically significant (p < 0.05), highlighting that language transferability remains a key challenge.
Table 15: Statistical Test for Performance Across Language
Model
p-value
-lama-3.1-8B-Instiii 1atta-3..--B---ttaaa FineMedL 9
9e-5
9e
Performance Across Output Length Regarding the impact of reasoning length, we tested the claim that "Models with longer CoT reasoning processes tend to exhibit superior response perfor- mance." in Section 5.1. A Spearman correlation test provided weak evidence for this relationship in the context of System 2 tasks (p=0.4917, p=0.0741). No significant correlation found for System or System 1.x tasks, reinforcing the idea that extended reasoning is primarily beneficial for problems that demand complex, multi-step cognitive processes.
F.2 SIGNIFICANCE TEST ON TRAINING METHODOLOGGY
Impact of Continuous Pre-Training We commenced our investigation by examining the claim from Sec. 5.2 concerning the dichotomous effect of CPT. This involved comparative anal-
34

---

Published as a conference paper at ICLR 2026
ysis of the top-performing models trained with and without the CPT methodology. Specifi- cally, we formulated distinct alternative hypotheses (H1) for the English and Chinese bench- marks. For AnesBench-English, the hypothesis H1 AnesQA Medical-ol (with CPT) > AnesQA + Medical-ol (without CPT), whereas for the AnesBench-Chinese, it was H1 : AnesQA Medical-ol (without CPT) AnesQA Medical-ol (with CPT).
To test these hypotheses, we employed a one-sided bootstrap test. The results, as presented in Table 16, indicate that the p-values for both tests are below the 0.05 significance level. Consequently, both alternative hypotheses are accepted. This finding substantiates the initial claim: CPT improves the upper bound of the model's performance on AnesBench -nggish,,,,, while degrades performance on the CeasBench-Chiness
Table 16: Statistical Test for the Impact of Continuous Pre- rrearrainng
Subset
A
p-value
90 % CI
AnesBench-Enge Englishh
0.0149
0.0413
(0.0009, 0.0287)
AnesBench-Chinee 0.0592
0.000
(0.0438, 0.0746)
Complementarity of AnesQA and Medical-ol To assess the complementarity between AnesQA and Medical-ol, we conducted four sets of tests under both CPT-trained and -on-Cae-- condi- tions. These experiments evaluated whether combining AnesQA and Medical-ol could yield better performance than using either individually. As shown in Tab. 17, a statistically significant im- provement was observed only when both models were combined under the CPT-trained setting and evaluated on AnesBench- Englihh.. Accordingly, we used the phrase "leads to varying degrees of per- formance improvement" in the main text to reflect this outcome. Although the performance gains from the combination are not universally significant, consistent improvements are observed regard- less of CPT training and across both CresBBncch-Eggleeee and AnesBencch-hhing benchmarks. This consistency suggests that combining the two models generally yields incremental benefits.
Table 17: Overall Complementarity of Supervised Fine-Tuning Datasets
Comparison
p-value
90% CI
Without Continuous Pre-Training AnesQA Medical-ol > AnesQA
0.0041 0.0059
0.2794
(-0.0079, 0.0149)
AnesQA + Medical-ol > Medical-ol
0.2216
(-0.0070, 0.0185)
With Continuous Pre- Pre-Trainin
AnesQA Medical-ol > AnesQA
0.0154
0.0237
(0.0029, 0.0278)
AnesQA Medical-ol > Medical-ol
0.0056
0.2284
(-0.0068, 0.0181)
G SUPPLEMENTARY EVALUATION
G.1 EVALUATION ON OUT OF DISTRIBUTION BENCHMARKS
To further assess the generalization capability of the model trained on anesthesiology reasoning data, we evaluated Morpheus across three general medical benchmarks (MedQA, MedMCQA, and MedExpQA; Jin al., 2021: Pal al., 2022: Alonso et al., 2024) and three general domain bench- marks (MMLU, MMI.UPro, and AGleval; Wang et al., 2023a; 2024b; Zhong et al., 2024). Con- sistent with our previous experiments, we employed the same framework used AnesBench to extract answers for multiple-choic questions. Notably, for AGIeval, we restricted the evaluation to its 18 MCQ tasks, whereas comprehensive evaluations were conducted for all other benchmarks. The medical subset of MMLU includes the tasks anatomy, clinical knowledge, college. biology, college. medicine, higo.school.. www.ol..oooollloooooo biology, medical genetics, nutrition, professional medicine, and vi- rology.
The 7B/14B/32B models, fine-tuned via SFT and GRPO on anesthesia reasoning data, consistently outperform the Qwent25-Instruet baselines across all three out-of-distributin general
35

---

Published as a conference paper at ICLR 2026
Model
AGlEval MMLU
MMLU-Pro MedExpQA
MedMCQA
MedQA
Qwen2.5-7B-Insse Instrucct
0.649
0.709
0.455
0,526
0.532
0.654
Morpheus-7B
0.663
0.786
0.580
0.582
0.593
0.720
Swen2.5511B-Inssruue
0.701
0.801
0.627
0.642
0.593
0.735
Morpheus- 14B
0.747
0.843
0.667
0.752
0.661
0.798
Qwen2.5-32Br
0.739
0.828
0.672
0.746
0.628
0,780
Morpheus-32B
0.766
0.873
0.717
0.798
0.685
0.814
Qwen2.5-72B-Inste Instructt
0.742
0.852
0.703
0.790
0.674
0.812
Table 18: Evaluation Result on OOD Benchmarks
Model
English
French
Italian
Spanish
Bren2.5-7B-Instruee
0.600
0.512
0,512
0.480
Morpheus-7B
0.632
0.608
0.464
0.624
Qwen2.5-14B-Instrun
0.712
0.576
0.632
0.648
Morpheus-14BB 14B
0.736
0.728
0.776
0.768
Qwen2.5-32B-Im Inn-ructt
0.784
0.768
0.712
0.720
Morpheus-32B
0.784
0.808
0,808
0.792
Owwen.55-7B-Innttrue
0.800
0.776
0.776
0.808
Table 19: Evaluation Result MedExpQA
medical benchmarks. In certain settings, they even surpass the performance of 72B-scale mod- els, indicating that our training strategy primarily enhances general medical reasoning capabilities rather than merely causing overfitting the source task. Regarding language transferability, Mor- pheus generally outperforms the corresponding Qwen2.5-Instruct models across four languages on the MedExpQA benchmark. The improvements are particularly significant in Italian and Spanish, demonstrating strong cross- loos-lingual reasoning transfer. A similar trend is observed in MedQA, where performance gains are notably substantial in Traditional Chinese and English contexts.
In terms of -omaii generalization, Morpheus demonstrates strong performance on bench- marks such as AGIEval, MMLU, and MMLU-Pro The performance improvements observed across the majority of tasks indicate that the reasoning gains derived from our training data extend beyond the medical domain to general-purpose tasks.
Since we performed no training or fine-tuning domains other than anesthesiology. no new domain-specific knowledge was introduced. Consequently, these performance gains can be tributed to enhanced reasoning capabilities, allowing the model to more effectively utilize its internal knowledge base. While prior works (Cheng et al., 2025) have focused on multi-domain reasoning generalization, findings offer insights into generalization derived from highly specific, smaller source domain.
G.2 SELF-CONSISTENCY IN TEST TIME SCALING
We utilized AnesBench to evaluate the efficacy of test-time scaling techniques, such Self- Consistency (Wang et al., 2023b), for enhanc- ing reasoning in anesthesiology. Specifically, we employed the Morpheus-70 7B as the policy model for sampling. As illustrated in Fig. 12, the application of Self-Consistency markedly surpasses the performance of direct inference. Furthermore, the model's performance consis- tently improves with an increased number of samples. These findings underscore that test- time scaling is an effective method for unlocking the full potential of LLMs in complex anesthesi- ology reasoning tasks.
0.78
085
0.55
Dest Berteeee
0.55
0.58
0.3
0.00
0.59
Sample 2000
Serp Stxe
Figure 12: Test time scaling of Ceff-Consistency.
36

---

Published as a conference paper at ICLR 2026
Model
Mainland
Taiwan
US
Swen2.5-7B-Instree
0.797
0.660
0.505
Morpheus-7B
0.795
0.709
0.656
Qwen2.5-
10102.5--14BInstrruu
0.816
0.745
0.643
Morpheus-14B
0.835
0.815
0.746
Brren..5-32B-Insteee
0,853
0,804
0,682
Morpheus-32B
0.831
0.817
0.794
Qwen2.5-72B-nnstreee Instruct
0.861
0.833
0.742
Table 20: Evaluation Result on MedQA
Model
AQuA-RAT Gaokao
JEC
LSAT LogiQA
SAT
Swen2.5-7B-Instruue
0.622
0.740
0.563 0.476
0.528
0.722
Morpheus-7B
0.721
0.737
0.575 0.489
0.530
0.766
Qwen2.5-14B-Insrrun
0.752
0.767
0.601 0.533
0.621
0.798
Morpheus-144 14B
0.772
0.817
0.670 0.585
0,648
0.830
Qwen2.5-33BBInnsrruct
0.744
0.817
0.663 0.576
0.652
0.804
Morpheus-32B
0.787
0,820
0.696 0.628
0.705
0.843
Qwen2.5-72B -nnttruutt
0.752
0.812
0.676 0.577
0.663
0.815
Table 21: Evaluation Result on AGleval
Model
medical
College
HchooS
Other
Professional
Bren2.557B-Instruuee
0.728
0.595
0.773
0.703
0.581
Morpheus-7B
0.782
0.734
0.859
0.773
0.673
Qwen2.5-14B-Inns,.
0.797
0.724
0.864
0.796
0.693
Morpheus-14B
0.842
0.791
0.905
0.834
0.741
Owen2.5-32B-Inste Instruct
0.827
0,801
0,896
0,812
0.732
Morpheus-32B
0.862
0.875
0,922
0.862
0.789
Qwen2.5-77BB-Instrue
0.857
0.810
0.912
0.837
0.771
Table 22: Evaluation Result on MMLU
Task
Q-7B-It M-7B Q-14B-It M-14B
Q-32B-It M-32B
Q-72B-It
Health
0.502
0.630
0.643
0.703
0.686
0.737
0.698
Biology
0.695
0.738
0.787
0.827
0.829
0.861
0.849
Business
0.459
0.649
0.710
0.693
0.734
0.771
0.769
Chemistry
0.413
0.572
0.657
0.627
0.691
0.730
0.736
Computer Sci
0.400
0.637
0.649
0.749
0.698
0.773
0.756
Economics
0.556
0.679
0.719
0.778
0.751
0.820
0,780
Engineering
0.366
0.339
0.490
0.396
0.550
0.474
0.542
History
0.417
0.533
0.541
0.625
0.583
0.640
0.619
Law
0.299
0.326
0.341
0.445
0.406
0.501
0.465
Math
0.486
0.759
0.778
0.810
0.799
0.869
0.826
Other
0.432
0.556
0.590
0.672
0.648
0.703
0.695
Philosophy
0.361
0.453
0.529
0.629
0.581
0.657
0.611
Physics
0.400
0.590
0.657
0.642
0.719
0.728
0.739
Psychology
0.589
0.654
0.691
0.734
0.727
0.773
0.756
Table 23: Evaluation Result on MMLU-Pro
37

---

Published as a conference paper at ICLR 2026
Table 24: Analysis results of the model's reasoning hallucinations on the AnesBench-Engliss sys- tem2 subset. The numbers in the cells respectively represent the quantity of various types of reason- ing sentences in the sampled subset and their proportion of the total reasoning sentences.
Model
Valid
NS
CR
OE
MC
AnesBench-English System2
Morpheus-78 Qwen2.5-7B-Ine -nttttrrctt
953 (76.2%)
206 (16.5%)
13 (1.0%)
66 (5.3%)
12 (1.0%)
181 (68.8%)
55 (20.9%)
1 (0.4%)
18 (6.8%)
8 (3.0%)
Morpheus- 14B
1119 (81.4%)
191 (13.9%)
5 (0.4%)
51 (3.7%)
(0.6%)
Qwen2.5-14B3 1-B-5114---nntrcutt Instruct
254 (77.0%)
51 (15.5%)
4 (1.2%)
18 (5.5%)
3 (0.9%)
Morpheus-32BB
1179 (83.6%)
151 (10.7%)
4 (0.3%)
62 (4.4%)
14 (1.0%)
Qwen2.5-322 5-BBB -2B-Insttuct
233 (78.2%)
43 (14.4%)
1 (0.3%)
15 (5.0%)
(2.0%)
Qwenn..5-7BB-Instruc
230 (74.9%)
52 (16.9%)
5 (1.6%)
13 (4.2%)
7 (2.3%)
Notes: NS: Non-Sequiur, CR: Causal Reversal, OE: Over-extrapolation, MC: Medical Contraindication.
G.3 HALLUCINATION EVALUATION
Beyond the accuracy on MCQs, ensuring the accuracy of each reasoning step generated by the model is of greater importance when translating research findings into practical clinical applications. Therefore, we performed a detailed hallucination evaluation on the outputs the several models presented in Tab. 4, utilizing LLM-as-Judge paradigm.
Specifically, our analytical pipeline comprises several steps. First, we sampled questions from System 2 of the English subsets of AnesBench. For these questions, employed spaCy 4 to seg- ment each model's response into independent sentences. Subsequently, the context and each individ- ual sentence were fed into GPT-OSS- 120B (Agarwal et al., 2025) for coreference resolution, while simultaneously filtering for complete sentences that contained explicit reasoning behaviors. Fi- nally, GPT-OSS- 122- together with an evaluation prompt, used to assess each sentence for the presence of four common caasoning-hallucinatio patterns during inference: Non-Sequitur, Causal Reversal, Medical Contraindication, and Over-extrapolation.
As shown in Tab. 24, the results indicate that among these hallucination types, Non-Sequitur and Over-extrapolation are the most prevalent. This suggests that models are highly susceptible to bro- ken logical chains or drawing conclusions based on insufficient evidence, highlighting the need for future research to employ methods such as process reward model (Lightman et al., 2023) and serreval-auumented generation (Lewis et al., 2020; Jiang et al., 2025) to enhance reasoning coher- ence and mitigate issues of insufficient evidence. Moreover, scaling these retrieval systems to ac- commodate large and multimodal clinical data will be increasingly critical (Müller & Unay, 2017; Shen et al., 2022: 2024b;a).
Comparing the models, larger parameter scale generally correlates with stronger reasoning valid- ity in most cases. direct comparison between the Morpheus and Qwen2.5 models reveals that Morpheus tends to generate more extensive reasoning steps. Notably. Morpheus demonstrated consistent improvement in reasoning validity over the baseline Qwen2.5, thereby substantiating the effectiveness of our training data and methodology. Furthermore, across various parameter scales, the Morpheus models consistently maintain the MC (high-risk hallucination) error rate at or be- low 1. 1.0%, which likewise demonstrates the enhanced safety profile of the models trained with our resources.
Prompt used for Hallucination Evaluation
SYSTEM PROMPT
sspaCy: https://spcy.iio/
38

---

Published as a conference paper at ICLR 2026
You are an expert Clinical Logic Judge. Your task is evaluate
4 the logical scundness of a single reasoning step generated by 4 medical Al.
You are NOT verifying isolated facts; you are judging the causal link and logical deduction within the sentence. Does the stated premise or fact logically justify the clinical 9 conclusion or recomeendation
EVALUATION CRITERIA:
Evaluate the internal logic of the Target_Bentence for the following fatal flaws:
Non-Sequitur: The conclusion does not logically follow from the
7 premise (e.g., "Patient has headache, therefore amputate 4 the leg").
Causal Reversal: Mixing up cause and effect.
Medical Contraindication: Recommending treatment that is
S logically contradicted by the premise stated in the very same
9 sentence (e.g., "Given the severe bleeding, administer heparin").
over-extrapolation. Jumping
definitive, extreme conclusion 4 without sufficient logical bridging.
# EVALUATION CATEGORIES:
1. "Valid": The reasoning is medically sound, coherent, and the conclusion logically follows the premises provided in the sentence.
2. "Flawed": The reasoning contains a logical fallacy, 4 contradiction, or medically dangerous 1eap in logic.
INSTRUCTIONS:
Analyze the "iaaget_Sentence"" Output ONLY a valid object with the following structure:
"<Must be exactly one of: Valid, Flawed>", "flaw "flaw_type" "<I= Flawed, briefly
the issue (e.g., Nen 4 Sequitur, contradiction) Valid, output null>", "rationale": "Briefly explain why the logical bridge helds up or 4 breaks down."
EXAMPLES:
Target_Sentence "Because the patient allergic to penicillin, we 4 should prescribe amoxicillin."
Output: "LLogic_judgment"
"Flawed", "flaw_type"; "Contradiction", "rationale": "Amoxicillin is penicillin-ciass antibiotic, so the premise se of allergy directly contradi cts the treatment 4 recommendaticn."!
Barget_Sentence: "Given the elevated troponin levels and chest pain an ECG should be performed to evaluate for myocardial infaretion."
Output: ("logic_judmment" "Valid", "flaw_type") null, "rationale";
9 "Elevated tropenin and chest pain are classic signs of cardiac injury, making an ECG a legical next diagnostic step
G.4 EVALUATION ON THE OPEN-ENDED SUBSET
To address the limitations of MCQs and further validate our findings, we augmented our evaluation with open-ended questions. A subset of System 2 benchmark items converted into open- ended format.
To construct our experimental dataset, manually curated specific subset from System2 questions of AnesBench- Engglihhh We identified and selected 64 items that were coption-indepeddeee ing they could be answered based solely the information provided in the question stem, without reliance on the accompanying options. Subsequently, we employed the Gemini-2.5-Pro (Comanici al., 2025) to transform this curated subset of questions into open-ended format. This conver-
mean-
39

---

Published as a conference paper at ICLR 2026
sion guided by specific prompt designed to minimize unnecessary alterations to the original questions.
To ensure robust and objective scoring, LLM-as-judge to evaluate performance on open-ended questions. To implement LLM-as-judge, we engineered detailed prompt that instructed the evaluator models (GPT-4.1 and Gemini-2.5-Pro)) to adopt the persona of an expert medical evaluator. To standardize the assessment, the prompt included detailed 10-point scoring rubric, mapping score ranges specific quality levels. We repeated each evaluation three times. The final scores for each response were then averaged across both evaluator models ensure robust and reliable assessment. The prompts used for LLM-as-Judges and question rewriting are provided as below.
automated metrics such BLEU and Fl along with
The results presented in Tab. 25 reveal strong correlation between the LLM-based scores (GPT and Gemini) and the original MCQ accuracy on AnesBench, which validates the use of MCQs as reliable proxy for complex reasoning. were consistently low (mostly below 0.25), underscoring their inadequacy for assessing the nuanced, multi-step logic required in medical decision-making. Therefore, we prioritize MCQs our bench- mark as more robust and standardized method for evaluating reasoning performance.
contrast, automated metries like BLEU and scores
Prompt used for LLM-as-judge
SYSTEM PROMPT
You are an expert medical evaluator tasked with scoring responses to medical questions. Your job is to evaluate how well
model's respense matches the target scenarios.
for medical
Evaluation Criteria:
1. Accuracy: Does the response provide the correct medical
information that aligns with the target answer?
2. Completeness: Does the response address all key aspects the 4 question?
3. Clinical Reasoning: Is the madical reasoning sound and
+ appropriate?
4. Clarity: the response clear and well sell-structured?
Scoring Scale: 0-10 points
10: Ferfect match with target, excellent reasoning, complete and
( accurate
8-9: Very good match with target, miner gaps slight inacouracies
6-7: Good match with target, some important information icn missing
() or unclear
4-5: Partial match with target, significant gaps inaccuracies
2-3: Poor match with target, majer errors missing key information
0-1: Completely incorrect or irrelevant response
Instructions
1. First, provide your reasoning and justification for the score
2. Compare the model's response with the target answer
3. Identify strengths and weaknesses in the response
4. Then provide your final score in the exact format: \"Score: X
4 /10'"
Input Format:
Question: [The medical question/scenaric]
Target Answer: [The correct answer]
Modol Response: [The model's respense to evaluate]
Please evaluate the response thoroughly and provide your sccre.
40

---

block returned; nothing Published as a conference paper at ICLR 2026
Prompt used for Question Rewriting
You are converter that rewrites multiple-choicee questions (MCQs)
into open-ended, directly answerable questions (open QA) with the smallest possible changes to the original stem.
GOAL
Keep the background/stent stem text virtually unchanged (only tiny grammar/clarity fixes if absolutely necessary).
Replace MCQ-style query ("Which of the following... options
A-E, etc.) with direct question that can be answered 9 solely from the stem-without referring to any options.
Do NOT add new facts that aren't already in the stem. De NOT include answer choices mention "eptions."
Preserve technical terms, numbers, units, time courses, and
4 qualifiers (e.g., "best next step," "initial management,"
4 inntrrindicattdd "КXCEPт/N01" -> phrase as "which [thing] is contraindicated/mm true in this scenario?" but still as direct, standalene question). Use the same language as the input sten.
Do NOT introduce or roduce "most" qualifiers (e.g., avoid 4 most likely").
ALLOWED MICRO-EDITS
Remove phrases like "of the follcwing, " "choose, "sclect,"
4 letters (A/B/C...), and references to options.
Minimal rewording of the final question clause to be a direct 4 inquiry (e.g., "What medication shculd be administered next ?", "What is the diagnosis?", "Which intervention should be performed next?")
If the stem relies on a test or tigure, you may add at most one short bricging clause drawn from the sten. (e.g., "Based on this finding, ....) -n- new information. OUTPUT FORMAT (STRICT)
Return EXACTLY one block:
YoUR SINGLE, OPEN-ENDED QUESTION HERE</refined>
No explanations, no option lists, reasoning, no extra tags text outside <refined>... </refined>. One-shot example:
Input (MCQ with known answer provided for your understanding-do 4 not print it):
patient is receiving blood preduct transfusions after undergeing ng aortic valve replacement. Despite the transfusions, the surgeons still note that their field is very "cezy." TEG is performed, which reveals decreased alpha angle. Which blood product should be administered next?
Correct answer (šor reference only): Cryoprecipitate Output (note the minimal edits and the required tags):
<refined>A patient is receiving blood product transfusicns
after undergoing aortic valve replacement. Despite the transfusicns, the surgical field remains very oozy. TEG shows a decreased alpha angle. Based on this finding, what bloed product should be administered next?</refinedh VALIDATION CHECKLIST
Background preserved with only minimal edits.
[ ] Final question is direct and ansverable from the stem alone,
4 with no reference to options.
[ ] No new clinical facts added; all original data retained.
[ ] No "most" qualifiers introcuced. crefinedb..<<refffiee
Exactly
else.
THE QUESTION I NEED YOU TO CONVERT IS: (question)
CORRECT ANSWER (FOR REFERENCE ONLY) (correct_answer)
41

---

Published as a conference paper at ICLR 2026
Table 25: Evaluation Results of Open-Ended QA vs. AnesBench (MCQ)
Model
LLM-based Scores
Automated Metries
AnesBench
GPT Score
Gemini Score
BLEU
Fl
Acc (%)
DeepSeek-RI
8.22
8.76
0.037
0.230
82
Gemini-2.5-flas
7.88
8.53
0.026
0.196
81
DeepSeck-V3
7.85
8.34
0.037
0.230
73
GPT-40
7.77
7.65
0.029
0.205
77
Gemma-3-27b-i
6.55
6.09
0.000
0.000
58
Qwen3-30B-A3B
6.44
6.02
0.000
0.000
68
CrepSeek-R1-Disse
6.26
5.79
0.000
0.020
63
Qwen2.5-72B-Instrene
6.17
5.51
0.000
0.000
67
Qwen3-14B
6.10
5.45
0.000
0.068
65
Claude-3.7
5.11
5.35
0.020
0.165
77
G.S EVALUATION CAB
In addition to AnesBench, we evaluated the models supplementary resource of over 8,000 MCQs from the CAB (Zhou et al., 2025). The complete evaluation results are presented in Tab. 26. Although this benchmark effectively highlights performance discrepancies among different models, its lack of a structured design precludes a detailed analysis of model capabilities across various systems. Furthermore, the exclusive inclusion of Chinese questions restricts its applicability for multilingual assessment.
Table 26: Evaluation Results MCQs from CAB
Model
Acc.
Model
Acc.
Baichuan2-7B-CC
0.43
Baichuan2-13B-Char
0.48
Bio-Mediial-LLama---88
0.48
BioMistral-7B
0.26
chatglm3-6b
0.36
Citrus 10-1aa..00
0.71
deepseek-rl
0.87
DeepSeek-RI--Distil-Laam-----
0,64
PeepSeek-RI-Disillle 1.5B
0.26
DepSSek-RR--Disill--eennn
0.33
DeepSeck-R1-Distill-e
0.63
DeepSek-RR--Disilll-wv.a..
0.67
deepseek-v3
0.78
FineMedI
0.31
FineMedl M-o1
0.35
gemma-2-9b-it
0.52
gemma-2-221 2-27b-it
0.56
gemma-3-Ib-in
0.26
gemma-3-4b-it
0.43
gemma-3-12b-it
0.58
gemma-3-27b-it
0.65
glm-4-9b-chat
0.61
gpt-4o
0.78
HustuoGPT-ol1-7
0.70
HuatuoGGPT-o1-SBB
0.56
HuatwoGPTT:o1-708
0.70
HuatuoGPP-ol1-229
0.81
internlm3-Sb-innstnn
0.84
Jlama---70B--Utraaeediiall
0.71
0.53
Llama-3,1-813--Utraaaaaaee
0.54
Llama-3.3-708-Insttu
0.68
Jaama-4-coout///8-11-----tm
0.78
Llama3-OpenBioLLMM
0.25
Lamaa-OpennBooLLMM4770
0.64
Mcta-Llama-3-SB-insite
0.49
phi-4
0.57
Qwen2.5-7B-Innstrue
0.67
Owen22.5-14B-Instruee
0.73
Owen2.5-32B-Instrruct
0.76
Qwwen2.5-72B-Instrruet
0.81
Qwen3-0.6B
0.37
Qwen3-1.7B
0.53
Qwen3-4B
0.53
Qwen3-8B
0.65
Qwen3- 14B
0.76
0.73
Owen3-32B
0.80
Qwen3-235B-ADD
0.75
QwQ-32B
0.73
Yi-1.5-34B-CCI
0.65
42

---

Published as a conference paper at ICLR 2026
H DATASHEET FOR ANESSUITE
MOTIVATION
1. For what purpose was the dataset created? Was there a specific task in mind? Was there a specific gap that needed to be filled? Please provide description.
A1: The AnesSuite dataset was created to address critical gap in artificial intelligence research: the limited exploration of reasoning abilities in LLMs within the highly specialized field of anesthesiol- ogy. Its primary goal is to serve as the first comprehensive resource specifically designed to train and evaluate LLMs tasks relevant anesthesiology. AnesSuite includes the AnesBench benchmark, which assesses model performance across a wide range of reasoning challenges faallnnges-fro straightfor- ward factual recall to complex clinical maiiiinnmmaiin.... Additionally, it offers dedicated training datasets that support various strategies for model improvement, including continuous pretraining, supervised fine-tuning, and reinforcement learning.
2. Who created this dataset (e.g,, which team, research group) and on behalf of which entity (e.g., company, institution, organization)?
A2: This dataset is created by the authors of this paper.
3. Who funded the creation of the dataset? If there is an associated grant, please provide the name of the grantor and the grant name and number.
A3: N/A.
COMPOSITION
1. What do the instances that comprise the dataset represent (e.g., documents, photos, people, countries)? Are there multiple types of instances(e.g, movies, users, and ratings; people and interactions between them; nodes and edges)? Please provide description.
A1: The dataset suite's 's instances include four types: (a) MCQs that assess anesthesiology reasoning at three cognitive levels (System 1, 1.x, 2) in AnesBench; (b) plain-text anesthesiology documents for continued pre-training in AnesCorpus; (c) QA pairs in AnesQA; and (d) verifiable MCQs paired with detailed CoT reasoning in AnesRI.
2. How many instances are there in total (of each type, if appropriate)?
A2: Instance counts: AnesBench has 4k English MCQs and 3k Chinese MCQs; AnesCorpus tains 1.8 million English and 0.6 million Chinese documents; AnesQA has 20k English QA pairs; AnesRI totals 10k items (3.2k English + 7k Chinese) of MCQs with COT.
3. Does the dataset contain all possible instances or is it a sample (not necessarily random) of instances from a larger set? If the dataset is sample, then what is the larger set? Is the sample representative of the larger set (e.g., geographic coverage)? If so, please describe how this representativeness was velidfted/veriffeed. If it is not representative of the larger set, please describe why not (e.g., to cover more diverse range of instances, because instances were withheld or unavailable).
A3: AnesSuite is a sample drawn from a larger set of instances. AnesBench and AnesR1 are MCQs related to anesthesiology, but they do not imply full representativeness of all anesthesiology ques- tions. AnesCorpus is a keyword-filtered sample from large web corpora; it is therefore targeted subset. not a census of all anesthesiology texts. AnesQA questions were generated from cleaned PubMed papers via LLM pipeline with further filtering, which can't imply full representativeness of all ancsthesiology QAs.
4. What data does each instance consist of? "Raw" data (e.g., unprocessed text images)or features? In either case, please provide a description.
A4: Per-instance data: AnesBench items are MCQs annotated with degandd level (Sys- tem 1/1.x/2). AnesCorpus instances are plain-text documents relevant to anesthesia and pain man- agement. AnesQA instances consist of question and its answer, with each question typed into one
43

---

Published as a conference paper at ICLR 2026
of five categories. AnesR1 instances pair a verifiable MCQ with full CoT reasoning trace under the same three-level cognitive framework.
5. Is there label target associated with each instance? If so, please provide a description.
A5: Yes, MCQ datasets (AnesBench, AnesRI) include the correct choice as the target.
6. Is any information missing from individual instances? If so, please provide deseription, explaining why this information is missing (e.g., because was unavailable). This does not include intentionally removed information, but might include, e.g., redacted text.
A6: No.
7. Are relationships between individual instances made explicit (e.g., users' movie ratings, social network links)? If so, please describe how these relationships are made explicit.
A7: No.
Are there recommended data splits (e.g., training, development//vllddateee testing)? If so, please provide description of these splits, explaining the rationale behind them.
A8: Yes, as mentioned in the main text, we recommend using AnesBench for testing, and AnesCor- pus, AnesQA, and AnesR1 for training. There is no explicit validation set, and users split it from the training set by themselves.
9. Are there any errors, sources of noise, redundancies in the dataset? If so, please provide a description.
A9: Yes. since AnesSuite contains datasets from different sources, for example, AnesCorpus is derived from cleaned web data, but may still contain noise. The construction of AnesQA relies LLM pipeline, which may also contain hallucinations and noise. However, each part has corresponding dataset validation methods, as detailed in the relevant sections of the main text and appendices.
10. Is the dataset self-contained, or does it link to or otherwise rely on external resources (e.g., websites, tweets, other datasets)? If links to or relies on external resources, a) are there guarantees that they will exist, and remain constant, over time; b) are there official archival versions of the complete dataset (i.e., including the external resources as they existed at the time the dataset was created); c) are there any restrictions (e-g., licenses, fees) associated with any of the external resources that might apply to a future user? Please provide descriptions of all external resources and any restrictions associated with them, as well as links or other access points, as appropriate.
A10: Yes, AnesSuite is self-contained, and users can obtain it directly from the project homepage, HuggingFace, and other channels once the dataset is made public.
11. Does the dataset contain data that might be considered confidential (c.g., data that is protected by legal privilege or by doctorpatient confidentiality, data that includes the content of individuals non-public communications)? If so, please provide description.
A11: No. although AnesSuite contains abstract clinical scenarios, they are all virtualized and do not include any personal information.
12. Does the dataset contain data that, if viewed directly, might be offensive, insulting, threat- ening, or might otherwise cause anxiety? If so, please describe why.
A12: No.
COLLECTION PROCESS
1. How was the data associated with each instance acquired? Was the data directly observ- able (e.g., raw text, movie ratings), reported by subjects (e.g., survey responses), or indirectly inferred/derived from other data (e.g., part-of-speech tags, model-based guesses for age or lan- guage)? If data was reported by subjects or indirectly inferred/derived from other data, was the data validated/verffied? If so, please describe how.
44

---

Published as a conference paper at ICLR 2026
A1: Yes, in AnesSuite, except for AnesQA, all other data instances be acquired directly. AnesQA is derived from the LLM pipeline, and its verification process is detailed in the relevant sections of the main text and appendices.
2. What mechanisms or procedures were used to collect the data (e.g., hardware apparatus or sensor, manual human curation, software program, software API)? How were these mecha- nisms or procedures validated?
A2: The collection of AnesSuite mainly uses Python scripts. After obtaining the data, adopted both manual and automated verification methods, as detailed in the relevant sections of the main text and appendices.
3. If the dataset is a sample from a larger set, what was the sampling strategy (e.g., determin- istie, probabilistic with specific sampling probabilities)?
A3: The sampling of AnesSuite is a deterministic process, not based on specific sampling prob- abilities. For example, due the keyword filtering method, AnesCorpus is only sample of deterministic subset related to these keywords among all anesthesia-related texts.
Who was involved in the data collection process (e-g., students, crowdworkers, contractors) and how were they compensated (e.g,, how much were crowdworkers paid)?
A4: The data collection was primarily conducted by the lead authors of this paper. Additionally, Tao Huang and Zhiwei Wang from the School of Computer Science, Wuhan University, assisted in the data collection process.
5. Over what timeframe was the data collected? Does this timeframe match the creation timeframe of the data associated with the instances (e-g., recent crawl of old news articles)? If not, please describe the timeframe in which the data associated with the instances was created.
A5: The collection period of AnesSuite is from 2024 2025, which does not exactly match the creation time of the data.
PEFROOCRSINN/CCEETTINI/AAEAEI
1. Was any preprocesiin/ddaainggllbiiin of the data done (e.g,, discretization or bucketing, tokenization, part-of-speech tagging, SIFT feature extraction, removal of instances, processing of missing values)? If so, please provide description. If not, you may skip the remainder of the questions this section.
A1: Yes, labeling and cleaning performed on the data refer to the relevant sections in the main text and appendices.
2. Was the "raw" data saved in addition to the preproceseddcleenne//bbeeee data (e.g., to support unanticipated future uses)? so, please provide a link or other access point to the "raw" data.
A2: No.
3. Is the software used to preprocess/deann/abel the instances available? If so, please provide a link or other access point.
A3: All of our prepocesinngdeeenig//bbliing are done using Python scripts, not specific soft- ware.
USES
1. Has the dataset been used for any tasks already? If so, please provide a description.
A1: Yes, have used AnesBench from AnesSuite evaluate the reasoning abilities of current state-of-the--rt LLMs. Additionally, using its training set, we have developed Moupheus, the first collection of reasoning LLMs in anesthesiology.
there a repusitory that links to any or all papers or systems that use the dataset? If If, please provide link or other access point.
45

---

Published as a conference paper at ICLR 2026
A2: N/A.
What (other) tasks could the dataset be used for?
A3: AnesSuit contains three training sets, among which AnesRl and AnesQA also include instance labels. This design allows users to freely combine and train models according to their needs. Anes- Bench can also be used test other LLMs.
there anything about the composition of the dataset or the way it was collected and preproessed/cleeme//bbeeee that might impact future uses? For example, is there anything that a future user might need to know to avoid uses that could result in unfair treatment of individuals or groups (e.g,, stereotyping, quality of service issues) other undesirable harms (e.g., financial harms, legal risks) If so, please provide a description. Is there anything a future could do to mitigate these undesirable harms?
A4: No.
5. Are there tasks for which the dataset should not be used? If so, please provide description.
A5: N/A
DISTRIBUTION
Will the dataset be distributed to third parties outside of the entity (e.g., company, in- stitution, organization) on behalf of which the dataset was created? If so, please provide description.
A1: Yes, the dataset will be made available to the public after the paper is published.
How will the dataset will be distributed (e.g., tarball on website, API, GitHub)? Does the dataset have a digital object identifier (DOI)?
A2: AnesSuite will be distributed via GitHub and HuggingFace.
3. When will the dataset be distributed?
A3: The dataset will be distributed once the paper is accepted after peer review.
4. Will the dataset be distributed under a copyright or other intellectual property (IP) license, and/or under applicable terms of (ToU)? If so, please describe this license and/or ToU, and provide link or other access point to, or otherwise reproduce, any relevant licensing terms or ToU, as well as any fees associated with these restrictions.
A4: For the specific license, please refer to the project homepage and Huggingface after we open- source the AnesSuite.
5. Have any third parties imposed IP-hased or other restrictions on the data associated with the instances? If so, please describe these restrictions, and provide link or other access point to, otherwise reproduce, any relevant licensing terms, as well any fees associated with these restrictions.
AS: No.
Do any export controls or other regulatory restrictions apply to the dataset or to individual instances? If so, please describe these restrictions, and provide link other access point to, or otherwise reproduce, any supporting documentation.
A6: No.
MAINTENANCE
1. Who will be supportinghhsttingmmnnaaaingggg the dataset?
Al: The authors.
2. How can the owner/curatorr/maaager of the dataset be contacted (e.g,, email address)?
A2: Users contact the authors via the email address the first page of the main text.
46

---

Published as a conference paper at ICLR 2026
3. Is there erratum? If so, please provide link other access point.
A3: After the paper is published, we will make public the link pointing to the erratum.
4. Will the dataset be updated (e.g., to correct labeling errors, add new instances, delete in- stances)? If so, please describe how often, by whom, and how updates will be communicated to users (e.g., mailing list, GitHub)?
A4: Yes, dataset will be updated. We welcome feedback from AnesSuite users. The authors this paper will update the dataset the project homepage and GitHub from time to time based on the feedback received, including removing any errors identified.
5. Will older versions of the dataset continue be suppotee/hnstelnnanttanet If so, please describe how. If not, please describe how its obsolescence will be communicated to users.
A5: No, each time update the dataset, will mark the update time simultaneously, which will help users understand when each version of the dataset becomes obsolete.
6. If others want to extend/augment//uild on/contribute to the dataset, is there mechanism for them to do so? If so, please provide a description. Will these contributions be validat- ed/verified? If so, please describe how. If not, why not? Is there process for communicat- ing/distributing these contributions to other users? If so, please provide a description.
A6: N/A.
I USAGE OF LLM
Writing Assistance We LLMs assist with the proofreading and refinement of the manuscript, primarily for correcting grammar, optimizing sentence structure, and improving the clarity of the language. All content generated with AI assistance was strictly reviewed and revised by the authors to ensure its accuracy and academic integrity. We take full responsibility for the final content of the manuscript.
Research Applications Beyond assisting writing, LLMs are integral to our research. Specifically, we utilized LLMs in the process of dataset construction, evaluation, training and so on. The specific methodologies, implementation details, and the function of these applications in research are described in detail within the corresponding sections of this paper.
47


Reviews:
[{"review_id": "BhdPavv0XA", "reviewer": "Reviewer_rYyW", "summary": "The paper introduces AnesSuite, a suite aimed at assessing and improving anesthesiology reasoning in LLMs. It comprises:\n(1) AnesBench — a bilingual (EN/ZH) multiple‑choice benchmark (7,972 items) stratified by cognitive demand into System 1 (factual), System 1.x (hybrid), and System 2 (complex decision making). \n(2) AnesCorpus — ~2.4M anesthesia‑related documents (1.8M EN + 0.6M ZH) for continued pretraining (CPT), filtered from FineWeb/Chinese‑FineWeb with keyword rules and a two‑stage decontamination (n‑gram screen + LCS>64 removal) to reduce overlap with the benchmark.\n(3) AnesQA — 20,713 English QA pairs produced via a two‑model LLM pipeline (Llama‑3.3‑70B to generate questions; Qwen‑2.5‑72B to filter/answer), with manual spot‑checks and regex filtering to excise 119 flawed items.\n(4) AnesR1 — 10,287 verifiable MCQs (EN+ZH) with chain‑of‑thought (CoT) traces generated by DeepSeek‑R1 and kept via rejection sampling only when the final choice is correct. Using these resources, the authors train Morpheus‑7B/14B (Qwen‑2.5 inits) with SFT + GRPO; they report improvements over the instruction‑tuned bases and parity with larger baselines on AnesBench.", "questions": "1. For AnesR1, did clinicians evaluate a subset of DeepSeek‑R1 chains for medical correctness beyond the final answer? Any evidence that RL on these chains avoids entrenching subtle misconceptions?  \n\n2. What exact reward(s) were used in GRPO—answer correctness only, or also intermediate checks (e.g., vital‑sign logic)? How robust are results to reward misspecification?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- First-of-it-kind domain‑specific reasoning suite for anesthesiology that separates fast recall from complex decision‑making (System 1→2) and offers training corpora aligned with CPT/SFT/RLVR stages. Prior work either subsumed anesthesiology under broad medical categories or focused on factual recall; this suite targets decision‑making explicitly;\n\n- Bilingual design with explicit experiments on language transfer. The ablation shows CPT on English‑heavy corpora can help EN while hurting ZH, highlighting data‑language balance as a first‑order concern in medical domains;\n\n- Reasoning‑centric baselines (Morpheus) showing that limited RL with verifiable rewards (GRPO) plus CoT data can close the gap to much larger models on this domain.", "weaknesses": "1. External validity of System‑2 constructed from exam/textbook‑style abstractions rather than real perioperative trajectories; this may under‑represent key anesthetic complexities (e.g., dynamic vitals, ventilator settings, lab time‑series). \n\n2. Leakage analysis scope: the permutation‑confidence heuristic over 500 items is informative but indirect; it cannot rule out contamination in proprietary models, nor leakage via near‑duplicates beyond LCS>64.\n\n3. GRPO is described at a high level; the exact verifiable reward signal (e.g., answer‑only vs. additional structure) and stability/variance aren’t deeply analyzed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qkhsraIzGg", "reviewer": "Reviewer_utdx", "summary": "The paper introduces AnesSuite, a dataset and benchmark collection designed to test and improve LLM reasoning in anesthesiology. AnesBench provides evaluation tasks that cover factual knowledge and complex decision making in English and Chinese. The authors also release three training datasets that target continued pretraining, supervised fine tuning, and reinforcement learning with verifiable rewards. Using these resources, they build a baseline model named Morpheus and run a broad set of experiments against strong public models. The analysis examines how language transfer, training data choices, and training methods affect performance on anesthesiology reasoning.", "questions": "Please see the question about weakness above.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.  The study makes a clear and timely observation that evaluation should reflect realistic clinical settings, and it identifies contextual complexity as a common bottleneck for reasoning across tasks.\n\n2. The three-tier benchmark design and the linked training resources form a coherent pipeline, and the Morpheus baselines with ablations provide useful evidence on which choices lead to gains.", "weaknesses": "1. I appreciate the careful build of a bilingual, cognitively tiered benchmark and the broad empirical sweep, yet the main novelty lies in the dataset suite rather than a method or algorithm that is central to ICLR. The focus on anesthesiology is narrow for a broad ML venue and is also narrow even within medical or clinical tracks, which limits general use.\n\n2. The dataset and benchmark draw mainly on public exams and textbooks and are text only. They do not include images, multimodal clinical data, or real clinical or PHI data. Existing resources already support assessment of medical knowledge, such as MedQA, MedMCQA, and PubMedQA. Clinically grounded datasets that capture real decision-making and multimodal signals are more directly relevant to patient care. The contribution is limited to that, not a direct link to clinically actionable use.\n\n\n3. I am concerned about data quality. On AnesBench, SFT on the proposed dataset does not beat the Qwen2.5-Instruct baselines: Morpheus 7B with SFT only averages 54 percent compared with 59 percent for the 7B baseline, and Morpheus 14B with SFT only averages 57 percent compared with 64 percent for the 14B baseline. Most of the gains appear after the GRPO stage, to 63 percent and 69 percent, and that is modest. SFT slightly improves the English subset but drops the Chinese subset by a large margin, which points to issues with data quality, data balance, or objective alignment. Train and test sit in the same domain due to vertical fine-tuning. One would usually expect larger SFT gains in that case, so the results are underwhelming.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rzgtNuBJDP", "reviewer": "Reviewer_EdCA", "summary": "The paper introduces a comprehensive suite of artifacts for the medical domain of anesthesiology, including benchmarks, a specialized corpus, and a anesthesiology-aligned model. The authors detail their end-to-end pipeline, covering pre-training data collection, benchmark creation, and the continuous training of a base model on the resources. The proposed benchmark is used to benchmark current SoTA medical LLMs, as well as open-weight and frontier models.", "questions": "Please refer to the Weaknesses section above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- The authors have conducted a substantial amount of work, delivering all the necessary sources and artifacts for the medical domain of anesthesiology, a field that is currently underserved by existing resources.\n\n- The paper presents a solid methodology for assessing both contamination and originality across all proposed resources, ensuring the integrity of their contributions.", "weaknesses": "**[W1]** While the authors acknowledge that anesthesiology is underrepresented in current medical NLP resources and provide a setup that uses both reasoning and factual knowedge, they do not offer a clear, tangible motivation for how these resources will benefit medical-aligned LLMs overall (i.e. examples illustrating how these resources could be applied in real-world scenarios).\n\n\n**[W2]** The methodology for benchmark creation and human verification is insufficiently detailed. Although their hybrid approach (LLMs + human verification) is reasonable, the paper does not specify:\n- The number of human annotators involved per sample\n- The instructions or rubric used for verification\n- Annotator agreement rates\n- How quality is ensured for data not verified by humans\nThese missing details make it difficult to assess the reliability and validity of the benchmarks.\n\n**[W3]** The paper does not discuss how alignment in the anesthesiology domain affects performance on other downstream medical tasks (e.g., MedQA) or general tasks (e.g., MMLU). \n\n**[W4]** The pre-training and SFT data are likely to be used in continuous pretraining of medical-aligned models that need to perform well across multiple medical domains. However, the authors do not provide a realistic CPT scenario demonstrating how their data would be used in such a context. Even a hypothetical example would help clarify the potential impact and application of their resources.\n\n**[W5]** The use of anesthesiology-related documents from Fineweb for ontinuous pretraining involves replaying existing data rather than introducing new resources. As a result, this aspect of the contribution is less substantial compared to the creation of the main benchmark.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ww2hc5NWxN", "reviewer": "Reviewer_eXNB", "summary": "This paper introduces a comprehensive suite for analyzing, training, and evaluating LLMs’ capabilities in the domain of anesthesiology. The benchmark results demonstrate that current models, especially open-weight solutions may not perform perfectly on the benchmark. And both Qwen2.5 models and Morpheus would improve after being trained on the provided datasets.", "questions": "How would the model trained on the proposed dedicated data perform on other medical tasks or even general tasks?\n\nOverall, I appreciate the effort of constructing this suite for such a specific domain, it would potentially be a better paper with further clarifications and improvement. I would consider increasing my score once most of my concerns are addressed.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "1. The first suite for benchmarking, training, and analyzing failure patterns in the particular domain of anesthesiology\n2. Comprehensive analysis and vivid figures demonstrating model performance gaps and some insights on the benchmark", "weaknesses": "1. The benchmark is not challenging enough. Overall the best open-weight model could solve 82% of the problems (DeepSeek-R1) and even a smaller Qwen3-14B could solve most of them (70%). These observations indicate that these problems in the benchmark are not challenging enough, especially considering the form of MCQA. One solution may be to refactor the MCQA questions into the free-end generation task that asks the model to answer the question without providing options.\n2. Training data effectiveness is not properly demonstrated: table 4 presents models trained on the proposed training set. However, this type of training-testing paradigm falls back to the in-distribution evaluation. A better way to validate the training data would be mixing them with other post-processing data and test the model or test the trained model on other OOD medical tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)