Language Agents for Hypothesis-driven Clinical Decision Making with Reinforcement Learning
Authors: David Bani-Harouni
Chantal Pellegrini
Ege Özsoy
Nassir Navab
Matthias Keicher
Venue: ICLR 2026 Poster (2026)
Keywords: Clinical Decision Making
Large Language Models
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose LA-CDM, an uncertainty-aware language agent trained with supervised and reinforcement learning to iteratively refine diagnoses through targeted test selection.

Abstract: Clinical decision-making is a dynamic, interactive, and cyclic process where doctors have to repeatedly decide on which clinical action to perform and consider newly uncovered information for diagnosis and treatment. Large Language Models (LLMs) have the potential to support clinicians in this process, however, most applications of LLMs in clinical decision support suffer from one of two limitations: Either they assume the unrealistic scenario of immediate availability of all patient information and do not model the interactive and iterative investigation process, or they restrict themselves to the limited "out-of-the-box" capabilities of large pre-trained models without performing task-specific training. In contrast to this, we propose to model clinical decision-making for diagnosis with a hypothesis-driven uncertainty-aware language agent, LA-CDM, that converges towards a diagnosis via repeatedly requesting and interpreting relevant tests. Using a hybrid training paradigm combining supervised and reinforcement learning, we train LA-CDM with three objectives targeting critical aspects of clinical decision-making: accurate hypothesis generation, hypothesis uncertainty estimation, and efficient decision-making. We evaluate our methodology on MIMIC-CDM, a real-world dataset covering four abdominal diseases containing various clinical tests and show the benefit of explicitly training clinical decision-making for increasing diagnostic performance and efficiency. Our code is available at https://github.com/dharouni/LA-CDM.

Published as a conference paper at ICLR 2026
LANGUAGE AGENTS FOR HYPOTHESIS-DRIVEN CLIN- ICAL DECISION MAKING WITH REINFORCEMENT LEARNING
David Bani-Harouni
Chantal Pellegrini
Ege Özsoy
Nassir Navab
Matthias Keicher
Technical University of Munich
Munich Center for Machine Learning david. david.bani- bani-harouna @tum. de
ABSTRACT
Clinical Docision-making is a dynamic, interactive, and cyclic process where doe- tors have to repeatedly decide on which clinical action to perform and consider newly uncovered information for diagnosis and treatment. Large Language Mod- els (LLMs) have the potential to support clinicians in this process, however, most applications of LLMs in clinical decision support suffer from one of two limi- tations: Either they assume the unrealistic scenario of immediate availability all patient information and do not model the interactive and iterative investigation process, or they restrict themselves to the limited "out-of-the-box"" capabilities of large pre-trained models without performing task-specific training. In con- trast to this, we propose to model clinical decision-making for diagnosis with a hypothesis-driven uncertainty-awer aware language agent, LA-CDM, that converges towards diagnosis via repeatedly requesting and interpreting relevant tests. Us- ing a hybrid training paradigm combining supervised and reinforcement learn- ing, we train LA-CDM with three objectives targeting critical aspects of clinical decision-making: accurate hypothesis generation, hypothesis uncertainty estima- tion, and efficient decision-making. We evaluate our methodology MIMIC- CDM, a real-world dataset covering four abdominal diseases containing various clinical tests and show the benefit of explicitly training clinical decision-makinn making for increasing diagnostic performance and efficiency. Our code available at httpps:/giie com/ cha Laaanii/L--ii -DDC
INTRODUCTION
In modern healthcare systems, accurate and efficient diagnosis stands critical component in pa- tient management and treatment (Savioli et al., (022). To form a diagnosis, clinicians often engage in clinical decision-making through a dynamic, iterative process, called differential diagnosis. This process involves forming hypotheses about the patient and testing those hypotheses through request- ing and interpreting information from relevant available diagnostic tests (Sox al., (2024). At the beginning multiple diagnoses may be possible and there is still high uncertainty. Through diagnos- tic testing the uncertainty is minimized and the space of possible diseases reduced until a sufficient confidence is reached, a diagnosis can be given, and treatment can begin (Sox et al. 2024). It is espe- cially important in complex, high-stakes environments such as emergency departments, where often little is known about a patient at admission and fast and accurate diagnosis is paramount (Savioli et al. 2022).
Large Language Models (LLMss), with their ability to synthesize complex textual information, seem well-suited for aiding clinicians this task, as most medical information textual or can be repre- sented in text, e.g., clinical notes, imaging reports numerical laboratory results. This enables great flexibility and variety in medical modalities. In the medical domain LLMs have already shown great success in passing medical license exams (Singhal et al.) 2023 Gilson al. 2023) and diagnosing case challenges (Buckley et al.] 2023). The potential to revolutionize healthcare through accurate diagnostic capabilities is enormous, however, existing approaches often either (1) assume immedi-

---

Published as a conference paper at ICLR 2026
Patient State
Hypothesis Agent
Decision Agent
Patient State
Patient History: The patient 43-year-old woman. She presents with 4 days of intense, crampy, midepigastric pain...
Reasoning: The symptoms suggest gastrointestinal disorder,
Patient History: The patient 43-year...
Hypothesis: Pancreatitis
Ultrasound Report: ...Teere is mild gallbladder wall edema...
Confidence: 3
Request Ultrasound
Hypothesis Agent
Decision Agent
Hypothesis Agent
Decision Agent
Reasoning: white blood cell count would identify inffammation.
Reasoning: The findings are consistent with cholecystitis.
Hypothesis: Cholecystitis
Hypothesis: Cholecystitis
Confidence: 6 10
Request WBC
Confidence: 9 10
Diagnose Cholecystitis
Figure I: An illustrative process of clinical pacision-making performed by LA-CDM. At the begin- ning, only the patient history including symptoms and family history is known. In a cyclic process, the hypothesis agent forms an uncertainty-awar clinical action (request test or diagnose). If test is requested the results are added to the known patient information. The cycle repeats until a final diagnosis is given.
hypothesis and the decision agent decides on a
ate availability of all patient data (Buckley et al. 2023 McDuff et al. 2023 Chen et al. 2025
which rarely the case in practice, or (2) rely on the often limited "out-of-the-box" behavior of pre-trained LLMs without any task-specific fine-tuning to the complexities of diagnostic decision- making (Hager et al. 2024 Li ct al. 2024; Nori al. 2025). This mismatch between research and real-world clinccal decision- makiing himits the applicability of LLMs the clinical setting.
In this paper, we address the above limitations by modeling and training Language Agents for Clin- ical Decision Making (LA-CDM), tasked with iteratively reducing hypothesis uncertainty through repeated diagnostic testing. Inspired by cognitive research on human clinical decision- makiing (Sox) al. 2024), we design two-agent system replicating the two main cognitive tasks of clinicians involved in clinical decision-making. It consists of one LLM agent, the hypothesis agent, forming the most likely diagnosis hypothesis based on all available patient information and estimating its confidence in that hypothesis, and another agent, the decision agent, that evaluates the patient in- formation and the hypothesis agent's output to either provide diagnosis or request an additional To train this system, we propose novel training strategy with three
diagnostic test (Figure D.
distinct objectives that target the core principles of successful clinical decision-makinn (Sox al.) 2024):
1. Accurate hypothesis generation: Using supervised fine-tuning, the hypothesis agent is trained to form a correct hypothesis. Since information on the patient is only uncovered step-by-step, the agent has to make use of limited information from various data sources.
2. Hypothesis uncertainty estimation: Using reinforcement learning the hypothesis agent is trained to be well-calibrated in its verbalized uncertainty estimation. A well-calibrated model that c.g. 60% certain on a hypothesis will be correct in 60% of cases.
3. Efficient decision-making
Using reinforcement learning, the decision agent is trained to select the most informative next test and reach diagnosis when sufficiently confident, taking test costs into account. The model gets rewarded for final correct diagnosis, rein- forcing the testing pathway that led to that diagnosis.
Analogously to doctors graduating from medical school, LLMs have a strong medical knowledge foundation, but not trained on performing clinical decision-making Clinicians learn this skill through years experience, pointing towards experience-basee reinforcement learning as a prime paradigm for teaching clinical decision-making L.MMs. Further, since optimal testing pathways
2

---

Published as a conference paper at ICLR 2026
not known, supervised learning of clinical decision-male making is infeasible. The interplay of the three objectives results in the model learning which tests to request in order to increase the hypoth- esis confidence leading to accurate diagnosis at minimal testing cost. This guides the model to request those tests that are most informative in given situation. To the best of our knowledge, we propose the first method for explicitly training L.LSs for clinical decision-making.
We evaluate our method on MIMIC-CDM (Hager et al. 2024), a real-world dataset focused on four abdominal diseases that mirrors clinical workflows by modeling differential diagnosis through se- quential test requests. Its standardized test naming and inclusion of lab results, notes, and imaging reports make uniquely suited for training and evaluating LLM- diagnostic reasoning. Our ex- periments demonstrate the benefit of explicitly training clinical prcision-making Notably, training reduces the diagnostic cost, which has practical implications in reducing healthcare costs, diagnosis time, and patient discomfort (Savioli et al., (022). We show the benefit of our hypothesis-driven approach to clinical decision-making and demonstrate that the model adapts its testing procedure to the patient at hand, placing this work step towards patient-specific personalized differential diagnosis.
2 RELATED WORK
Reinforcement Learning for Clinical Decision Making Reinforcement learning has been ex- plored for cost-efficient clinical decision-making based on tabular data. Yu et al. (2023) train SM- DDPO, a model that iteratively requests laboratory tests optimizing diagnostic performance and cost-efficiency. Their method features an imputation model, estimating missing (and not yet re- quested) laboratory tests and a classification model predicting the diagnosis. A policy network trained with Q-learning predicts the next action, i.e. which test to request or which diagnosis to give. They show improvements in efficiency at a similar diagnostic performance compared to base- lines making use of all available information. However, as the method is only compatible with tabular data, it neglects many important medical modalities, like clinical notes or imaging reports, which are often crucial for diagnosis.
ED-Copilot (Sun et al. 2024) employs Language Model for encoding serialized patient laboratory values which is trained end-to-end with two MLPs, one predicting test requests, and one predicting the task of outcome severity. They train their method in two stages. First, they use supervised learning to teach the model to predict all tests in a pre-defined order. In a second training stage, they use reinforcement learning to finetune the model to reduce time cost of the testing regiment. They also report great reduction in testing time, while remaining similar in performance to baselines making use of all available information. While this method uses a language model as its encoding backbone, this language model is not directly requesting tests. Same as the previous method, they further only consider tabular laboratory values and therefore do not make use of valuable information in textual form.
Reflexion (Shinn al. 2023) addresses general LLM decision decision-making by introducing zero-shot reinforcement learning approach. Rather than fine-tuning the LLM with traditional reinforcement learning, the method allows the model to iteratively attempt the same task. After each trial, limited number of previous attempts and a numerical reward are added to the input context, guiding the next generation. While this is effective at general decision-making, this approach cannot be applied in the clinical context, where multiple diagnosis trials with the same patient and intermediate diagnosis correctness feedback are infeasible.
LLMs for Clinical Decision Making LLMs have so far only been used as zero-shot methods for clinical decision support. Hager al. (2024) place large "out-of-the-box" language models in an evaluation framework where they are tasked with interactively requesting diagnostic tests and diagnose patients. They show severe limitations of L.LMs for clinical decision-making and report worse diagnostic performance than clinicians. Vaid et al. (2024) approach clinical decision-making with a tool-using LLM. Through zero-shot prompting, they provide the LLM with a number of available tools, e.g., symptom tool for retrieving the sympioms, or imaging study tool for getting any imaging reports on the patient. They evaluate various proprietary LLMs, with GPT-4 (Achiam et al. 2023) showing the best performance. Liu et al. 2024) model LLM clinical decision-maaee as a multi-agent setting, where a doctor agent communicates with a patient agent who can detail his symptoms and technician agent who can perform laboratory imaging results. They compare
3

---

Published as a conference paper at ICLR 2026
different prompting techniques, like chain-of-thouuge (Wei et al. 2022) one-shot prompting and show that GPT-40 achieves the best performance. All these methods do not attempt to improve model performance by explicitly training clinical decision-makinn with LLMs.
PRELIMINARIES
3.1 REINFORCEMENT LEARNING ON LARGE LANGUAGE MODELS
Reinforcement learning has emerged as powerful framework for fine-tuning LLMs by aligning their behavior with desired objectives, especially when no clear ground truth for this behaviour ex- ists. Most notably, Reinforcement Learning with Human Feedback (RLHF) (Ouyang et al., 2022) was used to align LLM generations with human preferences. LA-CDM utilizes a direct reinforce- ment learning method based Group Relative Policy Optimization (GRPO) (Shao et al. 2024) without requiring human feedback.
Let X' be the space of textual inputs (prompts) and θ C _ be the parameter space of a LLM. We denote by Ih s) the stochastic policy the LLM, parameterized by 0, which specifies the prob- ability distribution over discrete vocabulary AA (the set of possible tokens) given state S. In the context of language modeling, a state s is typically a sequence of tokens (#1,..., (2)) consisting of prompt and previously generated tokens. An action is the next token to be generated, yielding the updated state 8/ (f1...., Tt, a). Every state transition gives rise to an reward the model is trained maximize.
3.2 CONFIDENCE CALIBRATION OF LARGE LANGUAGE MODELS
While LLMs have shown impressive capabilities in many language-related tasks, hallucinations and monfidently-presentee wrong answers are common and well-known problem (Kadavath et al.) 2022]. A well-calibrated model is able to express confidence that aligns with the epistemic proba- bility of correctness. This means that of all the answers which are presented with a confidence of 0 S & 1, the fraction correct answers is
In this work, train confidence calibration with reinforcement learning as proposed by Stangel et al. (2025). They model confidence calibration as a betting game, where the model bets on the correctness of its answer. If is correct with high confidence it receives large reward. However, if it is wrong with high confidence, the punishment becomes large. Analogously, if the answer is wrong the model receives the largest reward if it expresses low confidence. Concretely, they use the reward function
[lo((c),
(Vpred) True [log11 - cc), if J (Upred) is False,
R(ypred; c,
where Upred is the predicted answer, 0 < c < is the (scaled and clipped) confidence prediction, and J(-) is binary function evaluating the correctness of Vpred- The reward is then scaled be be- tween -1 and 1. They train the model using Proximal Policy Optimization (PPO) (Schulman et al.] 2017]. This training approach removes the need for an artificially constructed ground truth confi- dence dataset, as done by other confidence calibration methods (Azaria & Mitchell 2023) Kadavath ct al., (2022), and instead only requires a measure of answer correctness. The authors prove that an optimal policy under their reward design produces perfectly calibrated confidence expressions.
4 LANGUAGE AGENTS FOR CLINICAL DECISION MAKING
We propose LA-CDM consisting of two language agents, hypothesis agent and decision agent, trained with three different objectives. The hypothesis agent is trained in accurate hypothesis gen- eration through supervised fine-tuning and encertainty-wwarenee through reinforcement learning. The decision agent is trained in fecision-makin, using reinforcement learning. Both agents share the LLM weights, so training one agent also influences the other. In Figure 2 we show the full model. The two agents and the three training objectives will be explained in detail in this section.
4

---

Published as a conference paper at ICLR 2026
Observed Patient State
Update Patient State
Ground Truth
Diagnosis
Confidence Calibration th
Test Request
Hypothesis Agent
Diagnasis Hypothesis
Confidence
Decision Agent
Choose Action
Diagnosis
Supervized Hypothesis Trairing
Clinical Action Training
Ground Truth Diagnosis
Ground Truth Diagnosis
Figure 2: Overview of our method LA-CDM and its three training objectives. The hypothesis agent receives the current patient state and predicts hypothesis and confidence. The hypothesis genera- tion trained supervised, the confidence calibration using reinforcement learning. The hypothesis agent output and the current patient state are then provided to the decision agent that is trained to decide on an optimal clinical action (test request or diagnosis) using reinforcement learning.
4.1 MODELING CLINICAL DECISION MAKING
Clinical Decision Making Environment In order train model in decision-making it has to operate in a reinforcement learning environment to explore testing strategies and receive reward sig- nals. The model learns decision-makee making through interacting with this environment and diagnosing patients. Let each patient be described by a number of n test results [t,] as textual records of clin- ical notes, imaging reports and laboratory panels. Since patient information iteratively uncovered through the model's 's test requests fy, we define the currently observed patient state at time-step j as the set of all observed tests and write as Pj- The correct diagnosis for the patient is denoted by Strue.
As we simulate a clinical patient-doctor interaction, Po, the initial observed patient state, consists of the first clinical notes detailing symptoms, medical and family history. This information is always available to the model. The environment advances step-wise with each model action. the model requests additional test, the observed patient state is updated, and the results are made available to both the hypothesis agent and the cecisionnmaking agent for the next step. The simulation ends when the model provides diagnosis for the patient or if one of the two failure cases is reached: (1) the model exceeds the specified maximum number of generated tokens, or (2) the model violates its specified output format.
Hypothesis Agent Through its system prompt, the hypothesis agent is introduced to its task and provided with the possible diagnoses. At each time-step j of the environment, the hypothesis agent H is given the currently observed patient state p5 to predict the most likely diagnosis hy based on the limited available information, as well the confidence that prediction cy. therefore produces mapping
H. : PÚ - Ch..
The model generates this output in format of "Hypothesis: hy. Confidence: ey" The agent reports numerical confidence estimations on a scale of to 10, where 10 means absolute certainty of the correctness of the hypothesis, and 0 means absolute certainty that it is incorrect.
Decision Agent The decision agent D is the actor advancing the environment. It decides on which action to take at each time-step. Through its system prompt it is provided with its task, list of tests present in the dataset, and the possible patient diagnoses. Provided with the currently observed patient state P, and the hypothesis agent's hypothesis h, and confidence Cy it produces a decision on whether to request another diagnostic test tú and move on to the next time-step j + 1 or whether commit specific diagnosis Upred for the patient and end the episode. Formally, it produces
5

---

Published as a conference paper at ICLR 2026
mapping
D hp. + {
if further test requested if a diagnosis is given
Upred
Specifically, we employ the ReAct prompting technique (Yao et al. 2022). to prime the model to first produce a reasoning trace, following chain-of of thought principles (Wei et al., 2022). and then provide action and action imput (in our case, the specific test or diagnosis) in a structured format.
If further test was requested, the test results are appended to the conversation context as a user response to the LLMs generation. Since we can only work with retrospective data, where not every test result is present for every patient, we cannot always fulfill the model's request. In these cases, the user reply tells the model the requested test is unavailable and asks it to choose a different action. The implications of this will be discussed later on. Also if the model requests tests or provides diagnoses that are not on the list of possible tests or, respectively, diseases, the model is asked to choose a different action.
4.2 TRAINING CLINICAL DECISION MAKING
In our training objectives we follow the three main principles of clinical decision maasinn, as pro- posed by Sox et (2024): (1) accurate hypothesis generation, (2) hypothesis uncertainty estimation, and (3) efficient decision-making. The first two objectives are trained with respect to the generations of the hypothesis agent, the last objective is trained on the environment interactions of the decision agent. We follow a cyclic training approach, where each objective is trained individually for a spec- ified number of episodes after which the objective changes to the next one until the cycle repeats, resulting in much more stable training compared to optimizing all objectives simultaneously.
Training Hypothesis Generation A baseline of good clinical decision- making is a high accuracy in hypothesis generation. If the model knows the most likely candidate for the diagnosis, it can adapt its testing strategy quickly confirm reject this hypothesis. While the model interacts with the environment, the hypothesis agent is confronted with various patient states consisting of different subsets of test combinations. We collect all contexts shown to the model within all episodes of a patient batch, usually including multiple hypothesis generation steps per patient. To perform supervised fine-tuning, we create target sequences, consisting of the collected conversation contexts concatenated with the correct hypothesis generation Utruue We compute the cross-entropy loss for the sequences, ignoring the token at the position where the model should place its confidence score.
Since the available patient information is often limited, espe- cially at early stages of the diagnostic process, and the available data does not always clearly point at specific diagnosis, uncertainty is inherent to clinical decision-making timation of that uncertainty by the model can give an improved basis for decisions on when to stop the diagnostic process and produce a diagnosis. In this work, we train confidence calibration fol- lowing a method proposed by Stangel et al. (2025) as previously introduced in Section however instead of PPO, we use the GRPO algorithm (Shao et al. 2024). We define correctness measure (hb)) as equality between the predicted hypothesis hy and the ground truth diagnosis Ytrue-
Training Uncertainty-Awarene
An accurate intrinsic es-
Training Efficient Clinical Action Selection At the core of training clinical decision decision-making lies the training of clinical action selection. During interaction with the clinical decision-making envi- ronment the model can freely choose iteratively request any number of tests in any order. Given the vast number of possible diagnostic pathways, defining an optimal test sequence for each patient infeasible, we therefore do not have a ground truth on which tests to perform. To still be able to train clinical decision-making learn through trial-and-error which tests are useful in which situations. Equal to the confidence calibration training, we employ the GRPO algorithm (Shao et al.) 2024) for reinforcement learning training. Through interaction with the clinical decision-making environment the model freely re- quests different tests until it decides on a diagnosis. The requested tests are provided to the model by the environment whenever available. If not available the model is notified and asked to request different test. We design our diagnosis reward function Raing to present the model with fixed positive reward pos if the final diagnosis at the end of the diagnosis episode was correct, or a fixed negative reward Tneg if it is wrong. Additionally, we punish the model with reward Fanvalid if the
we propose to use reinforcement learning, where the model can

---

Published as a conference paper at ICLR 2026
model violates the specified format. Our reward function is thus:
Toos
if Vpred ytrue
Rdioy (apred))ree))))
Tneg
if ypred # ytrue
Tinealid
for out-of-format generations
Not just diagnosis accuracy but also diagnostic efficiency is an important factor in clinical decision making. Different tests have different costs for the healthcare system, e.g.. CT scan is much more expensive, than simple blood value. The model should therefore only request expensive tests, when necessary for diagnosis. To incentivize efficient behaviour, we add an additional reward function Reost that punishes test usage relative to their cost:
Reast (T)== = ===
GGT
where 7' is the list of all performed tests and c(tz) is the cost of test
The interaction of these three objectives enables the model to learn which tests to request to enhance hypothesis confidence, ultimately leading to a more accurate diagnosis while also taking test cost into account. This drives the model to prioritize tests that provide the most informative insights in a given situation.
EXPERIMENTAL SET-UP
Dataset and Pre- Pre-Processing We evaluate our method on the MIMIC-CDM dataset (Hager al., 2024), a curated subset MIMIC-IV (Johnson et al.] 2020) designed for modeling sequential clin- ical decision making. It contains 2,400 patients diagnosed with of four abdominal conditions: appendicitis, cholecystitis, diverticulitis, or pancreatitis. This focused setting on four pathologies reflects real clinical workflows, where physicians perform differential diagnosis with narrowed down space of possible diseases and request tests to distinguish between likely candidates. The dataset includes patient histories (symptoms, comorbidities, family histories) and physical exam notes for all patients. It also provides 5,959 textual imaging reports (CT, x-ray, ultrasound, MRI) lab results (blood, urine, microbiology), however, not every test result is reported for
and 143,191 191 every patient. If multiple values for specific test were recorded during the hospital stay of the patient only the first one was included in MIMIC-CM to simulate an early diagnosis after hospital admission. Crucially, the dataset includes comprehensive mappings of test names across patients, an essential feature for modeling test requests reliably. Without this normalization, a model could not query the same test across different cases due inconsistent naming in clinical documentation. To our knowledge, MIMIC-CDM is the only publicly available dataset that enables simulation this setting. For use-case, we construct the set of available tests as: physical examination, all imag- ing modalities, and the most common laboratory panels. These panels are collections of individual tests that are usually ordered together.
To evaluate model performance, we report class- wise accuracies along with their mean, as well as micro and macro Fl-scores. Additionally, we compute the Expected Calibration Error (ECE) to assess confidence calibration of our hypothesis agent. ECE measures the discrepancy between predicted confidence scores and actual accuracy. lower ECE indicates better model calibration, meaning the predicted probabilities align well with actual correctness frequencies.
Metrics
Baselines We compare our method to various zero-shot and trained baselines. In their work, Hager al. (2024) introduce both the MIMIC-CDM dataset and evaluate how various "out-of-the- pre-trained models perform when tasked with clinical decision-making. We compare with OASST, the best performing model from this evaluation, however, direct comparison is very difficult for multiple reasons. First, zero-shot method, the model was evaluated on the complete dataset, whereas we reserve some part of that dataset for training. Second, the clinical decision framework was constructed differently, as the OASST model is not provided with the possible diagnosis classes or available tests in its prompt. Additionally, as an approximate upper-bound for our method, we compare with Diven22557BB-Iniiie model (Yang al. 2024) identical to L.LM backbone,

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison of LA-CDM and baseline methods. We report class-wise aceu- racies and Fl-scores. The avg. test cost is the mean cost of the tests for each patients in the test *OASST results are taken from Hager al. (2024). evaluated on different test set in different framework. *SM-DDPO can only process tabular data. ZS zero-shot
Accuracy
FI-Score
Avg.
Method
Append.
Cholec.
Divert.
Pancr.
Mean
Micro
Macro
Test Cost
OASST*
82.0
48.0
45.5
44.1
54.9
SFT-all
98.4
89.8
95.8
87.5
92.8
93.6
92.9
$3792.79
SM-DDPO"
74.3
0.0
15.6
58.0
37.0
45,4
31,8
ReAct
90.2
79.7
66.7
62.9
74.9
79.1
74.8
$1480.32
LA-CDM (ZS)
73.5
55.1
72.0
57.5
64.5
65.3
64.5
$1521.73
LA-CDM
93.1
83.6
75.0
73.5
81.3
84.1
81.3
$1295.61
1.0
LA-CDM (zero-shol)
LA-CDM
250
0.8
200
0.6
150
100
0.2
50
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Figure 3: Left: Calibration curves before and after training LA-CDM. Right: Distribution of confi- dence estimations before and after training LA-CDM.
trained with supervised fine-tuning to predict the correct diagnosis. Instead of requesting tests simply receives all the available patient information directly which is not realistic diagnostic pro- cess. We refer to this method as SFT-all. We compare to three other adaptive test selection methods on MIMIC-CDM. SM-DDPO (Yu al. 2023) trains MLP using reinforcement learning for clini- cal decision making, however, the method is only able to process tabular data and can only request laboratory values. ReAct (Yao al. 2022) is zero-shot decision making method and LA-CDM (ZS) is the untrained version of our method.
6 RESULTS AND DISCUSSION
Our comparison with baselines is shown Table When compar- ing with OASST (Hager et al. 2024), LA-CDM shows improvement in accuracy for each class sulting in almost 30 percentage points difference when comparing the mean of all class accuracies. The performance shows the largest improvement for pathologies that less common compared with the majority class appendicitis. While the methods are not directly comparable as outlined in section [5] the high performance improvement clearly shows the advantage of training on the task of clinical makiing and not relying on inherent capabilities of large pre-trained models. This is further supported by the evaluation results of ReAct and of LA-CDM as zero-shot model. com- parison to our trained model, we see a great performance improvement achieved through training. Importantly, the untrained methods also diagnoses less efficiently than the trained version, acquiring, in the case of ReAct, almost $200 more test cost to form diagnosis. Also the individual perfor- mance of the hypothesis agent benefits from training LA-CDM. Through our hypothesis generation training. improve the ability of the model to form correct hypotheses from 75.7% to 81.9%. We equally show an improvement of uncertainty mnsertainty-awarrness awareness through training the confidence calibra- tion objective. The ECE decreases from 0.069 0.037. We visualise the calibration eurves and
Comparison with Baselines
8

---

Published as a conference paper at ICLR 2026
Table 2: Ablation of the inclusion of the test reward function.
Accuracy
FI-Score
Test Cost
Append.
Cholec.
Divert.
Paner.
Mean
Micro
Macro
Avg. Test Cost
93.5
81.4
82.1
72.3
82.3
84.3
82.4
$1427.85
93.1
83.6
75.0
73.5
81.3
84.1
81.3
$1295.61
Table 3: Ablation of the hypothesis-drivers approach. Experiments without test cost. HA hypoth- esis agent, DA = decision agent.
Agents
Accuracy
FI-Score
HA
DA
Append.
Cholee.
Divert.
Paner.
Mean
Micro
Macro
Avg. Test Cost
93.0
79.7
79.2
62.2
78.5
81.7
78.6
$1410.01 $1427.85
93.5
81.4
82.1
72.3
82.3
84.3
82.4
confidence distribution of the two models in Figure where the trained model shows better calibra- tion, especially at often predicted confidences. When comparing with SM-DDPO, the other trained baseline, we can see the benefit of being able to process textual input. SM-DDPO almost entirely fails it is limited to tabular data which is not sufficient in many diagnostic tasks like the one at hand, where imaging results are paramount. The SFT-all model serves as a rough upper bound, as leverages all available retrospective patient data, unrealistic setup for real-time clinical decision- making. Therefore, it cannot be used for direct patient interactions. Naturally, the performance is than tripled. The
very strong, however, as all available tests are used, the diagnostic cost is our approach. Moreover, we observe
substantial reduction test cost highlights the efficiency evidence of patient-adaptive testing strategies aligning with best practices: for suspected cholecys- titis, the model most frequently selects ultrasound (64.9% of cases), the gold-standard test (Hirota al. 2007); for appendicitis, prioritizes CT scans (85.1% of cases), consistent with diagnostic guidelines (Di Saverio et al. (2020). These results demonstrate that our method not only achieves high diagnostic accuracy but also optimizes resource usage clinically meaningful way. The reasoning traces generated by chain-of-thought prompting further enable the interpretation of the model's 's testing pathways. We report qualitative examples of the model's generations Appendix
B
Ablation Study In Table 2 observe that while the inclusion of the test cost reward function, performs similarly to training without test cost, the average test cost per patient is reduced signif- icantly. This shows how the model learns to become more efficient in its testing pathways while achieving similar diagnostic accuracy. More information on the choice of test costs is given in ap- pendix C Further, we evaluate the benefit of our hypothesis-driven approach Table 3 When removing the hypothesis agent from our methodology, the decision agent has to learn to request tests and diagnose without relying on the uncertainty- the hypothesis agent, explicitly trained for these objectives. The benefit of the hypothesis agent is demonstrated clearly by improvements all metrics.
hypothesis generation capabilities of
Limitations The data we are training on retrospective with different tests missing for differ- ent patients. Furthermore, the available tests are those tests that the clinicians involved in treating that patient performed. The model can only explore a limited spread of testing pathways. It can therefore only learn to become more efficient within the testing protocols performed by doctors, e.g by skipping unnecessary test. Our objective is therefore not to discover novel clinical strategies, but to optimize decision-mail making within realistic, sspert-demonttrated behavior. Simulation of un- available test data could open up a pathway for modeling a more holistic clinical decision makinng environment.

---

Published as a conference paper at ICLR 2026
7 CONCLUSION
In this work, we propose a novel approach to clinical decision-making using LLMs by explicitly modeling the iterative hypothesis refinement process that clinicians follow practice. Unlike previ- ous work that either assumes full access to patient data or relies on untrained, out-of-the-box LLM behavior, our method introduces a structured, two-agent system trained dynamically acquire and integrate diagnostic information. By leveraging reinforcement learning to optimize uncertainty esti- mation and decision-maie makinng we enable model that not only improves diagnostic accuracy but also enhances efficiency in medical testing. Our evaluation the MIMIC-CDM dataset demonstrates that our approach surpasses existing baselines, achieving more accurate diagnoses with fewer diag- nostic tests. This reduction in testing needs has direct benefits for real-world healthcare settings, including lower costs, faster diagnosis times, and reduced patient burden. Furthermore, our findings highlight the ability of the model adapt its testing strategy based patient-specific information, an essential step toward personalized Al-assisted healthcare.
ACKNOWLEDGEMENNT
The authors gratefully acknowledge the financial support by the Bavarian Ministry of Economic Affairs, Regional Development and Energy (StMWi) under project ThoraXAI (DIK-2302-0002), and the German Research Foundation (DFG, grant 469106422 NA 620/51-1).
REPRODUCIBILITY STATEMENT
In order to ensure reproducibility. we describe implementation details in Section and Appendix C Further, Appendix provides provides the exact prompts used in our experiments. Lastly, will publish our complete code containing preprocessing scripts, model code, training pipelines, and evaluation scripts, as stated the abstract.
ETHICS STATEMENT
The development of LA-CDM represents step forward in Al-driven clinical decision support, offering a structured and hypothesis-driven approach to diagnosis. By iteratively requesting and interpreting clinical tests, LA-CDM has the potential to improve diagnostic accuracy while opti- mizing resource utilization, ultimately enhancing patient outcomes. From an ethical perspective, the deployment of LA-CDMM necessitates rigorous safeguards to ensure patient safety, fairness, and transparency. As an Al system designed to support medical decision-making it is crucial that its recommendations remain interpretable and aligned with established medical guidelines. Moreover, biases present in training data must be carefully monitored to prevent disparities in diagnostic per- formance across different patient populations. In the broader societal context, LA-CDM has the potential to support clinicians in environments with high cognitive load, such emergency depart- ments and primary care settings, where time-sensitive decision-makee is critical. However, it is essential that such Al-driven tools positioned augmentative rather than substitutive, ensuring that they enhance rather than diminish the role of healthcare professionals. Additionally, accessibil- ity must be prioritized ensure equitable deployment across diverse healthcare systems, including under-resourced settings.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint WxXiv:2303.08777 2023.
Amos Azaria and Tom Mitchell. The internal state of an 11m knows when it's lying. arXiv preprint wxXiv:2304.13744 2023.
Thomas Buckley, James A Diao, Adam Rodman, and Arjun K Manrai. Accuracy of vision- language model challenging medical cases. arXiv preprint 2xXiv:2311.05500 2023.
10

---

Published as a conference paper at ICLR 2026
Zhen Chen, Zhihao Peng, Xusheng Liang, Cheng Wang, Peigan Liang, Linsheng Zeng, Minjie Ju, and Yixuan Yuan. Map: Evaluation and multi-agent enhancement of large language models for inpatient pathways. arXiv preprint arXiv:2503. 13205, 2025.
Salomone Di Saverio, Mauro Podda, Belinda De Simone, Marco Ceresoli, Goran Augustin, Alice Gori, Marja Boermeester, Massimo Sartelli, Federico Coccolini, Antonio Tarasconi, al. Diag- nosis and treatment of acute appendicitis: 2020 update of the wses jerusalem guidelines. World journal of emergency surgery, 15:1-42, 2020.
Aidan Gilson, Conrad W Safranek, Thomas Huang, Vimig Socrates, Ling Chi, Richard Andrew Taylor, David Chartash, et al. How does chatgpt perform on the united states medical licens- ing examination (usmle)? the implications of large language models for medical education and knowledge assessment. JMIR medical education, 911):45511 2023.
Paul Hager, Friederike Jungmann, Robbie Holland, Kunal Bhagat, Inga Hubrecht, Manuel Knauer, Jakob Vielhauer, Marcus Makowski, Rickmer Braren, Georgios Kaissis, et al. Evaluation and mit- igation of the limitations of large language models in clinical decision-making. Nature medicine, 30(9):2613-2622 2024.
Masahiko Hirota, Tadahiro Takada, Yoshifumi Kawarada, Yuji Nimura, Fumíhiko Miura, Koichi Hirata, Toshihiko Mayumi, Masahiro Yoshida, Steven Strasberg, Henry Pitt, et al. Diagnostic criteria and severity assessment of acute cholecystitis: Tokyo guidelines. Journal of hepato-
surgery, 14:78-82, 2007.
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022.
Albert Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bam- ford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088 2024.
Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Steven Horng, Leo Anthony Celi, and Roger Mark. Mimic-iv. PhysioNet. Available online at: https://pyysioee.. org/content/mimiciv// httpeiiiiiiiiii..///aeeeaaeeee acccesseed August 23, 2021), pp. 49-55, 2020.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Watfield-Dodke Nova DasSarma, Eli Tran-Johnson, et al. Language mod- els (mostly) know what they know. arXiv preprint arXiv:2207.0522. 2022.
Shuyue Stella Li. Vidhisha Balachandran, Shangbin Feng, Jonathan S Ilgen, Emma Pierson, Pang Wei Koh, and Yulia Tsvetkov. Mediq: Question-asking Ilms and a benchmark for reliable interactive clinical reasoning. In The Thihtt-eigtth Annual Conference on Neural Information Processing Systems, 2024.
Hongcheng Liu, Yusheng Liao, Siqv Ou, Yuhao Wang, Heyang Liu, Yanfeng Wang, and Yu Wang. Med-pme: Medical personalized multi-modal consultation with proactive tak-firtt-bbserve-ne paradigm. arXiv preprint Ntt::2408.089933 2024.
Daniel McDuff, Mike Schaekermann, Tao Tu, Anil Palepu, Amy Wang, Jake Garrison, Karan Sing- hal, Yash Sharma, Shekoofch Azizi, Kavita Kulkarni, et al. Towards accurate differential diagno- sis with large language models. arXiv preprint arXiv:2312.00164 2023.
Harsha Nori, Mayank Daswani, Christopher Kelly, Scott Lundberg, Marco Tulio Ribeiro, Mare Wil- son, Xiaoxuan Liu, Viknesh Sounderajah, Jonathan Carlson, Matthew P Lungren, et al. Sequential diagnosis with language models. arXiv preprint
2025.
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730 2022.
11

---

Published as a conference paper at ICLR 2026
Gabriele Savioli, Iride Francesca Ceresa, Nicole Gri, Gaia Bavestrello Piccini, Yaroslava Longhi- tano, Christian Zanza, Andrea Piccioni, Ciro Esposito, Giovanni Ricevuti, and Maria Antonietta Bressan. Emergeney department overcrowding: understanding the factors to find corresponding solutions. Journal of personalized medicine, 12(2):279, 2022.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint vrXiv:1707.06347 2017.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634-8652, 2023.
Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et Large language models encode clinical knowledge. Nature, 620(7972) 2023.
Harold C Sox, Michael C Higgins, Douglas K Owens, and Gillian Sanders Schmidler. Medical decision making. John Wiley & Sons, 2024.
Paul Stangel, David Bani-Harouni, Chantal Pellegrini, Ege Özsoy, Kamilia Zaripova, Matthias Ke- icher, and Nassir Navab. Rewarding doubt: A reinforcement learning approach to confidence calibration of large language models. arXiv preprint arXiv:2503.02623 2025.
Liwen Sun, Abhineet Agarwal, Aaron Korbllith, Bin Yu, and Chenyan Xiong. ED-copilot: Re-
duce emergency department wait time with language model diagnostic assistance. In Forty-first International Conference on Machine Learning, 2024.
Akhil Vaid, Joshua Lampert, Juhee Lee, Ashwin Sawant, Donald Apakama, Ankit Sakhuja, Ali Soroush, Sarah Bick, Ethan Abbott, Hernando Gomez, et al. Natural language programming in medicine: Administering evidence based clinical workflows with autonomous agents powered by generative large language models. arXiv preprint arXiv:240I.022. 102851, 2024.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi. Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 248377, 2022.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang. Jian Yang, Jianhong Tu, Jianwei Zhang. Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang. Peng Wang, Ru Peng, Rui Men, Ruize Gao. Runji Lin, Shijie Wang. Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zhihao Fan. Qwen2 technical report. arXiv preprint XXXx:24477.10677 2024.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03623 2022.
Zheng Yu, Yikuan Li, Joseph Chahn Kim, Kaixuan Huang, Yuan Luo, and Mengdi Wang. Deep reinforcement learning for cost-effective medical diagnosis. In The Eleventh International Con- ference on Learning Representations, 2023.
12

---

Published as a conference paper at ICLR 2026
APPENDIX
A PROMPTS
A.1 HYPOTHESIS AGENT PROMPT
You are a helpful medical expert. Your task is to create hypothesis for diagnosis of patient admitted to the emergency department complaining about abdominal symptoms. can hypothesise one the following four conditions:
appendicitis
cholecystitie
diverticulitis
pancreatitis
You have to base your hypothesis the information the patient that will be provided to you at the end. If you are uncertain, provide your best guess.
Additionally, you always have to provide your confidence as number from 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10, where 0 means very uncertain and means very certain.
#### Always use following format for your hypothesis:
Hypothesis: Here, you put your chosen condition as single word. Nothing else goes here.
Confidence: Here, you put your confidence as a number between and 10
####
Here the four master rules:
1. Only answer in the above format!
2. Always provide hypothesis, even if you are uncertain and always
provide a confidence.
3. Only give naming.
of the four possible conditions and their exact
4. Do not answer with anything other than the hypothesis and confidence specified in the format description.
Any violation of these rules, will have dire consequences for the patient. Any correct hypothesis will greatly rewarded.
####
Here is the available information the patient: (provideddtestt_eeullls
A.2 DECISION AGENT PROMPT
You interactive, helpful medical expert. Your task is to diagnose patients admitted to the emergency department complaining about abdominal symptoms into the following four conditions:
appendicitis cholecystitis
13

---

single test per Published as a conference paper at ICLR 2026
diverticulitis pancreatitis
To aid you in your diagnosis, you can request the results of the following tests for the patient:
Physical Examination CT
MRI
Radiograph
Ultrasound
Complete Blood Count
Basic Metabolic Panel
Comprehensive Metabolic Panel
Renal Function Panel
Liver Function Panel Urinalysis Electrolyte Panel
At the start you will be provided with some context on the patient.
At the beginning of every step you will be further provided with current hypothesis of the condition.
####
In your diagnosis process always the following format:
Thought: [Here, you should think about what do next.]
Action: [Here, you can either write "Diagnosis" if you are confident enough to provide diagnosis or "Test" if you want to request single test.]
your action, you should put the correct condition of the four provided above here. You MUST put one of the four conditions here. If you chose "Test", provide the test name from the list mentioned above. You can only request action. Use the exact naming of the test as given above. Do not add any specification, abbreviation Observation: [Here, you will receive the test results for the patient. Always write the keyword "Observation:" when requesting test.]
Action Input:
you chose "Diagnosis"
explanation to the test name.]
The current hypothesis is: [Here you will the current hypothesis]
(confidence [Here, you will get the confidence in the hypothesis on a scale of (very uncertain) to (very certain)])
Thought: [Here, you evaluate the test result given above and continue with your next thought and action]
Action: [You can request more tests with the action "Test".] Action Input: [...]
Observation: [...]
The current hypothesis is:
(confidence: ...)
14

---

Published as a conference paper at ICLR 2026
The current hypothesis is (confidence:
Thought: [Finally, when you are confident in a diagnosis you provide your reasoning for that diagnosis first.]
Action: [You can provide a diagnosis with the action "Diagnosis"]
Action Input: [Here, you provide the condition you choose from the four given above.]
####
If you requested a test, the user will provide you with test results if available. If the test is not available, do not request the same test again. you receive an unknown test or diagnosis error, try again and ensure that you use the exact wording given above.
####
Here are the six master rules:
1. Ask for exactly test per test action.
2. Only ask for one of the tests given in the list above.
3. If test returns "not available" under circumstance request the
same test again. Try some other test. Ask for no test twice.
4. Use the exact naming of the tests given in the list above without additional specifications of the region, test abbreviations, or explanations.
5. Be brief and concise in all your text.
6. Keep asking for tests until you are sufficiently confident but always provide a valid diagnosis in the end. Make sure to ask for enough tests to reach diagnosis.
Any violation of these rules, will have dire consequences for the patient. Any correct diagnosis will be greatly rewarded.
####
Here is some context on the patient: (patient.history
B QUALITATIVE EXAMPLES
We show exemplary LA-CDM decision-making procedures. In Figure and Figure 5 show two successful predictions. In Figure we show a failure case where the model gives diagnosis prematurely without providing reason, probably due to overly confident hypothesis.
C IMPLEMENTATION DETAILS
Dataset Pre-processing The initial patient history is shortened by processing them with Mixtral- 8x7B (Jiang et al., 2024) model, prompted to summarize most important aspects. Importantly, this summarization is performed without any auxiliary patient information. such disease labels. Also the imaging reports are shortened, wherever a separation into sections was available, by only keeping the findings section of the report and removing the remaining sections. The original dataset does not have a data split, since was intended for the evaluation of zero-shot models. We therefore split the data into training of 80%, and validation and test set of 10% each.
Training Details We use a PyTorch implementation of the Qwen-2.5-7B al. (2024) base L.M and fine-tune it using LoRA Hu et al. 2022) We train our method in
model Yang
15

---

Published as a conference paper at ICLR 2026
Patient History: The palient male with history open bypass surgery. recently experienced hematemesis after drinking vodka, with episodes of bloody vomiting on the previous two days, twice the day before, and once this morning. The palient also reported epigastric pain
tongue fasciculations upon Valium. His past medical history includes asthma, obstructive sleep apnea (OSA), dyslipidemia, hypertension, cholelithiasis, back pain, knee pain, laclose intolerance, and a history of right arm fracture. The patients family history includes his mother with diabetes, esthma, arthritis, obealty, and siblings obeelty.
emergency department,
resolved
Hypothesls Agent
Decision Agent
The patient's history of choleithiasis and the current hypothesla pencreatits should be confimed. Given abdominal pain and the recent aloohol intake, an ulfresound could deneficial to pancreas and gallbladder..
Reasoning:
Hypothesls:
Confidence: 9 10
Pancrestitis
Request
Ultrasound
severely poor penetration
body habitus,
evaluation
Main
portal vein appears patent. Common bile duct not visualizd. No asciies. BILE DUCTS: There no intrahepatic biliary
dilation. GALLBLADDER Gallbladdrr surgically abseni. PANCREAS: The pancreas not well visualized, largely obscured overlying bowel gas, SPLEEN. Normal echogenicity, meesuring 10.0 cm. KIDNEYS: Limited views of the kidneys show no hydronephrosis.
Hypothesis Agent
Decision Agent
Reasoning: Given poor visualization
ulrresoud, an MRI CT scan might provide beiler delinsation and help confirm rule pancresttis.
Hypothesls:
Confidence:
Pencreatitie
Request CT
CT: LOWER CHEST: [..]] ABDOMEN: HEPATOBILIARY: [...] PANCREAS: The pancreas is diffusely atrophic. In the left upper quadrant, dominant collection
the region of pancreatic tail superior the previously fat stranding (series 2: image 30), Mid stranding and thickening extends inferiorty along Gerota's fascia into the left hamipelvis. More inferior findings are likely chronic, however, upper quadrant findings may represent
there inflammation and significant
superinposed chronic infeemmation. underlying drainable collection identffie. Spienic vein not clearty delineated may be chronically occluded, an appearance unchanged from prior. SPLEEN: [..]ADRENALS; [...] URINARY: [..]] GASTRINNTESTINAL [..]]
Hypothesis Agent
Decision Agent
Reasoning:
The CT findings confirm
pancrrstili,, especially the diffusely atrophic pancrese with aignificant fal stranding and inflammation, The palient's history and clinical presentation support diagnosis.
Hypothesls:
Confidence:
Pancreatitis
Diagnose
Pancrealitis
A successful diagnosis performed with the evaluation of ultrasound and CT. The model decides to confirm an initial suspicion with an ultrasound, however, the results are not conclusive, prompting another imaging test.
Figure 4:
cyclic training regiment with each objective trained for 100 steps in the order of (1) clinical action training, (2) hypothesis generation training, and (3) confidence calibration training. All objectives optimized using the Adam optimizer with learning rate of le 5 and batch size of 2 single NVIDIA A40 GPU until convergence on the validation set, taking approximately 3 days.
Test Costs Specific cost data was taken from the 2025 standard charges table sourced under the CMS HHS price transparency rule from the Beth Israel Deaconess Medical Center, where the MIMIC-CDM dataset was collected. The specific test costs are given in Table For calculating the specific test punishment within the test cost reward function, all costs where normalized to sum to the reward for correct diagnosis, such that diagnosis pathway that requires all tests achieves total reward of zero. Thus, is always better to request more tests it helps the model to reach the correct diagnosis.
16

---

Published as a conference paper at ICLR 2026
History: patient person identfiid experiencing 4-day peraistent right-sided abdominal pain intensified, with no associated Gl symptoms or diarhea, unlike past experiences. Notably, they recently completed course of Their medical history
diagnosis Crohn's disease certain age, though they are not currently on any medication for having previously been on Asacol for extended period. Their physical and social histories are unremarkable, as is their familial medical history. The recent abdominal pain, accurring outside context of GI symptoms given patient's history Crohn's disease.
Z-pak
diarree,
Hypothesls Agent
Decislon Agent
Reasoning: Given the peraistent right-aided abdominal pain and the hypotheais apperoictts,
would be the informative test to confirm or rule out this diagnoosis, given the patient's hislory Crohn's disease.
Hypothesis:
Confidence:
Appendicitis
Request CT
visualized lung are unremerkable. The liver, gallbladder. spleen, pancress, and bilateral adrenal glands are within normel limit, Both kioneys enhance and symmetricalyy without evidence hydronephrosis The appendix appears fluid filled with mild mucosal hyperemia and transverse diameter approximately 12 mm with surrounding periappendiceal stranding. There mild thickening abdomen.
consolidation
and pericardum
calculi. There free free fluid within abdomen.
base of the cecum. There is free air within the within polvis. Retropentoneal mesenteric lymph nodes pathologic enlargement. The bladder, distal ureters, reclum, and sigmoid colon are unreraankable. The visualized osseous structures suspicious malignancy.
show lyšc
Decision
Hypothesis Agent
presence appendix with surroundling pertappendical stranding and mild mucosal hyperemia, which strongly supports diegnoais appendicilis. The patient's history of Crohn's 's disease does not rule appendicitis, and the imaging findings consislent with this condition.
Reasoning: scan
Hypothesis:
Confidence: 10 10
Appendicits
Diagnose
Appendictiss
Figure short example diagnosis. many cases, the model learns to correctly predict the condition very briefly with just CT.
Table 4: Test costs as derived from the standard charges table at the Beth Israel Deaconess Medical Center.
Test Name
Cost in USD
1306
MRI
4866
Radiograph
434
Ultrasound
1288
Complete Blood Count
71
Basic Metabolic Panel
298
Comprehensive Metabolic Panel
636
Renal Function Panel
394
Liver Function Panel
413
Urinalysis
50
Electrolyte Panel
134
D EXPERIMENTS ON COST SENSITIVITY
In order to evaluate cost sensitivity, we perform two experiments: (i) tion, and (ii) sensitivity large price deviations.
Inter-institutional cost varia-
17

---

Published as a conference paper at ICLR 2026
The patient, history GERD gallbladder problems, presents 2-day history severe, sharp, spreading abdominal pain, associated with nausea. The pain, rated high severity, may have been triggered by consumption of Nolably, patient experienced similar episode year ago lipase levels, ultrasound, and CT scan, revealed cholelithiasis and pancreatitis with perinephric fat necrosis. The patient was treated with antibiolics, fluids, and pain medicalion. Currently, the patient still experiencing abdominal pain and feels homibe.. patient's family history includes multiple family history related to the current episode are mentioned in the text.
History:
self-resolied.
diagnostic
including
had their gallbladder removed. other symptoms medical
Hypothesls Agent
Decision Agent
Ressoning: Given the history of cholelihissis and previous
pancreatliis, the current symploms are consistent wilh pancreatiliss However, the seventy of the pain and the lack of improvement suggest the
Hypothesis: Pancreatitis
Confidence: 9 10
Request Complete Blood Count
Complete Blood Count: Red Blood Cells: 5 mvul., White Blood Cells: 14.0 Kiul,, Hemoglobin: 16 g/dl, Hematocrit: 47.1 %, MCV: 90.0 MCH: pg.
KluL, Lymphacyte Basophils: Easinaphils:
Absolute Monocyle Count: 4.1 %. Neutrophils: 92.7 %, RDW-SD: 13.2 %
Hypothesis Agent
Decision Agent
Reasoning:
consistent
infiammation seen in pancreattiis However, to further confirm, we should look the lipass levefs.
Hypothesls: Pancreatitie
Confidence:
Request Lipase
Error: Unknown lest. Only request lests given above
Hypothesis Agent
Decision Agent
Reasoning: high supports suspicion pancreatiis, and requesting the lipase lavels would provide addtional diagnostic Let's procsed
Hypothesis:
Confidence:
Pancreatitis
Request Lipase
Error: Unnknwwn test. Only request tests as given above
Hypothesis Agent
Decision Agent
Reasoning: Since on the clinical presentation and laboratory findings. The high WBC count and the ongoing severe abdominsi psin suggest pancrestitie.
lipase levels, should mly
Hypothesis:
Confidence: 10
Pancreatitis
Diagnose Pancreatitis
X
Figure 6: An example of failed diagnosis cholecystitis. The model reaches a high confidence in wrong diagnosis too quickly and requests non-existent tests.
D.I INTER-INSTITUTIONAL VARIATION
To evaluate sensitivity to inter-institutional cost variation, create a new cost table that exhibits slight variations in pricing to our original cost table. This simulates a cost table from a different institution with different prices that, however, stay within similar range. We provide the changed cost table in Table 5 After retraining, we observe that both diagnostic performance and test cost efficiency remain largely unaffected. The results shown in Table 6
18

---

Published as a conference paper at ICLR 2026
Table 5: Test costs variations derived slight deviations from the standard charges table the
Beth Isracl Deaconess Medical Center.
Test Name
Cost in USD
Cost after variation in USD
CT
1306
1325
MRI
4866
4891
Radiograph
434
407
Ultrasound
1288
1274
Complete Blood Count
71
66
Basic Metabolic Panel
298
290
Comprehensive Metabolic Panel
636
626
Renal Function Panel
394
397
Liver Function Panel
413
418
Urinalysis
50
53
Electrolyte Panel
134
136
Table Performance comparison of LA-CDM with and without test cost variation. (CV) marks the model with trained with the test cost variation.
Accuracy
F1-Score
Avg.
Method
Append.
Cholec.
Divert.
Paner.
Mean
Micro
Macro
Test Cost
LA-CDM
93.1
83.6
75.0
73.5
81.3
84.1
81.3
$1295.61
LA-CDM (CV)
93.0
75.0
83.6
77.8
82.3
83.9
82.3
$1276.22
D.2 SENSITIVITY SPECIFIC PRICE CHANGES
In order to test model reactance to specific cost signals, we construct artificial test table where one test's cost, the complete blood count (CBC), has been increased to be very expensive. setting all requests of the CBC, lead to a test cost punishment added to the final reward of -00. This is significantly larger than all other test costs which are normalized to be in the range of 0 to 1. After training with this cost table, observe the number of CBC requests going down significantly compared to evaluating the model trained with the standard cost table. The prevalence is reduced from 19.2% of cases 2.5% of cases. The diagnostic performance is also reduced CBC can be an informative test in many situations. The Fl micro score decreases from 84.1 to 81.9 and the Fl macro score decreases from 81.3 to 77.8. This experiment clearly demonstrates the sensitivity to specific test cost distributions.
this
E USE OF LARGE LANGUAGE MODELS
We employed ChatGPT to enhance the clarity of the manuscript by focusing on grammar correc- tions, shortening overly complex sentences, and providing alternative wording suggestions. All outputs were manually reviewed before inclusion, and no new technical material, code, results, or figures were generated by the tool.
19


Reviews:
[{"review_id": "LcapR9wNp4", "reviewer": "Reviewer_sB6P", "summary": "This paper proposes LA-CDM, a two-agent system combining hypothesis generation and decision-making modules trained with hybrid supervised and reinforcement learning for iterative clinical diagnosis.", "questions": "See above", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper addresses an important gap in clinical AI systems by modeling the iterative nature of diagnostic reasoning.\n2. The paper provides thorough comparison against multiple baselines and includes cost-efficiency metrics.\n3. The paper attempts to integrate uncertainty estimation with sequential decision-making in a clinical context.\n4. The paper includes detailed prompts and implementation details that facilitate understanding of the approach.", "weaknesses": "1. The paper introduces separate hypothesis and decision agents without sufficiently justifying their architectural separation. Specifically:\n- The hypothesis agent generates diagnostic hypotheses and confidence scores, while the decision agent uses this output to select tests or final diagnoses. However, both agents inherently engage in diagnostic reasoning, and the decision agent could potentially internalize hypothesis generation and confidence estimation.\n- The confidence score produced by the hypothesis agent could be implicitly learned by the decision agent through its reasoning process, eliminating the need for explicit separation. This raises questions about whether the dual-agent design introduces unnecessary complexity.\n\n2. The reinforcement learning component is trained on retrospective clinical data, which poses significant off-policy learning challenges:\n- Historical data reflects actions taken by clinicians, which may not align with the optimal policy learned by the agent. This can lead to distributional shift and biased policy evaluation, as the agent may propose test sequences not represented in the data.\n- The paper mentions handling unavailable tests by prompting the agent to choose alternatives, but this does not address the core issue of learning from a fixed dataset with limited action coverage. Off-policy correction methods (e.g., importance sampling, conservative Q-learning) or imitation learning techniques are not discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AQgYLLg5hM", "reviewer": "Reviewer_j7xL", "summary": "This paper proposes **LA-CDM (Language Agents for Clinical Decision Making)** — a two-agent LLM system for **iterative, hypothesis-driven diagnosis**.  \nA **Hypothesis Agent** generates provisional diagnoses with calibrated confidence, while a **Decision Agent** chooses whether to request additional tests or finalize a diagnosis.  \nTraining uses a **hybrid pipeline**:\n1. **SFT** on labeled cases for hypothesis quality.\n2. **Reinforcement Learning (GRPO)** for calibrating confidence via a “betting” reward.\n3. **RL** for cost-aware decision optimization, balancing diagnostic accuracy and test cost.\n\nThe environment is based on **MIMIC-CDM**, including text notes, imaging summaries, and lab data for 2,400 cases across four abdominal conditions.  \nResults show LA-CDM achieves **higher accuracy and lower average testing cost** than ReAct or untrained baselines, with **better calibration (ECE ↓ from 0.069 to 0.037)**.", "questions": "1. How robust is LA-CDM when scaling to new diseases or hospital systems?  \n2. What is the impact of unavailable tests on exploration and policy stability?  \n3. Why does cyclic training outperform joint optimization—any gradient conflict analysis?  \n4. Does better hypothesis calibration improve decision timing and stop behavior?  \n5. How sensitive is the cost trade-off to regional price variations?  \n6. Can hard safety constraints be added to the RL objective?  \n7. Have clinicians evaluated qualitative reasoning traces for plausibility?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **Innovative design:** Two-agent division closely reflects clinical reasoning loops.  \n2. **Calibration reward:** Improves reliability of verbal confidence estimates.  \n3. **Cost-sensitive optimization:** Demonstrates efficiency improvements without sacrificing accuracy.  \n4. **Transparent methodology:** Prompts, cost tables, and training configs are public.  \n5. **Multimodal context:** Integrates notes, labs, and imaging text within a unified environment.  \n6. **Clear ablations:** Each module’s contribution is empirically validated.", "weaknesses": "1. **Limited dataset scope:** Only four conditions; no cross-domain or cross-hospital validation.  \n2. **Environment artifacts:** Some requested tests are unavailable, possibly biasing learning.  \n3. **Weak baselines:** Comparison methods are not fully aligned in modality or setting.  \n4. **Preprocessing bias:** Summarization of clinical notes may distort reasoning cues.  \n5. **Safety unaddressed:** No human-in-the-loop validation or fail-safe mechanism.  \n6. **Ablation granularity:** Lacks exploration of cyclic vs. joint training and reward sensitivity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lgHg30ZhZk", "reviewer": "Reviewer_epNq", "summary": "This paper introduces LA-CDM, a two-agent LLM framework for clinical diagnosis that mirrors clinicians’ workflow: a hypothesis agent proposes a diagnosis with an explicit confidence, and a decision agent either orders the next test or finalizes the diagnosis. The authors train three objectives in a cyclic schedule—(1) supervised hypothesis generation, (2) RL-based confidence calibration (a betting-style reward), and (3) RL for action selection with a test-cost penalty—so the system learns to raise hypothesis confidence while minimizing unnecessary testing. Evaluated on MIMIC-CDM (2,400 patients; four abdominal diseases; labs, notes, imaging reports), LA-CDM improves mean class accuracy to 81.3% with macro-F1 81.3 and reduces average test cost to USD1,295.61 versus a zero-shot ReAct baseline at USD1,480.32; an ablation shows the cost reward cuts cost from USD1,427.85 to USD1,295.61 with similar accuracy. Calibration also improves (ECE 0.069 → 0.037).", "questions": "1.How does LA-CDM generalize to other institutions/specialties or non-abdominal conditions? Any plans for external-site validation or leave-one-hospital evaluation?\n2.Does summarizing histories and trimming imaging reports change diagnostic difficulty or shift feature reliance? Please provide with/without pre-processing results.\n3.Do you quantify how often “test unavailable” occurs and its effect on learning/inference? Could synthetic imputation or simulation reduce bias from missing tests?\n4.How sensitive are policies to the cost table (e.g., ±x% scaling, switching to a different hospital’s charges)? Please report accuracy/cost under several cost schedules.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper's theme is based on real-world clinical problems. The design of the two agents reflects the actual diagnostic process: hypothesis formation → targeted testing → decision-making.\n2.The paper demonstrates a strong focus on cost. The R_cost term explicitly penalizes expensive tests (e.g., MRI USD4,866 vs CBC USD71), driving parsimonious testing.\n3.The paper has principles for handling uncertainty. Confidence calibration via RL (betting-style reward) trains the model to express calibrated confidences without ground-truth confidence labels.\n4.The paper aligns with the guidelines for patient adaptive behaviors.The trained model preferentially orders ultrasound for cholecystitis (64.9%) and CT for appendicitis (85.1%), matching clinical practice.", "weaknesses": "1.This study's evaluation is limited to four abdominal diseases based on a single retrospective dataset; it lacks broader disease coverage or external validation. It lacks generalizability.\n2.The data in the paper has limitations in retrospective constraints and test availability bias. Many requested tests are unavailable in logs, forcing the agent to try alternatives; this caps exploration to clinician-observed pathways and can bias learned policies.\n3.The paper preprocesses patient information.Summarization of histories and imaging reports (e.g., keeping only findings) could remove cues clinicians use, but the impact isn’t quantified.\n4.The paper did not analyze the sensitivity of the cost model.Test costs are tied to a single hospital’s 2025 charge schedule; robustness to alternative cost tables or scaling is not reported.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)