DiaLLM: EHR-Enhanced Clinical Conversational System for Clinical Test Recommendation and Diagnosis Prediction
Authors: Weijieying Ren, Tianxiang Zhao, Lei Wang, Tianchun Wang, Vasant G Honavar
Accepted at: acl25

DiaLLMs: EHR Enhanced Clinical Conversational System for Clinical Test Recommendation and Diagnosis Prediction

Weijieying Ren†, Tianxiang Zhao†, Lei Wang‡, Tianchun Wang†, Vasant Honavar†
† College of Information Sciences and Technology, Pennsylvania State University
‡ Salesforce AI Research
lemontreejying@gmail.com, tkz5084@psu.edu, vuh14@psu.edu *

Abstract

Recent advances in Large Language Models (LLMs) have led to remarkable progresses in medical consultation. However, existing medical LLMs overlook the essential role of Electronic Health Records (EHR) and focus primarily on diagnosis recommendation, limiting their clinical applicability. We propose DiaLLM, the first medical LLM that integrates heterogeneous EHR data into clinically grounded dialogues, enabling clinical test recommendation, result interpretation, and diagnosis prediction to better align with real-world medical practice. To construct clinically grounded dialogues from EHR, we design a Clinical Test Reference (CTR) strategy that maps each clinical code to its corresponding description and classifies test results as “normal” or “abnormal”. Additionally, DiaLLM employs a reinforcement learning framework for evidence acquisition and automated diagnosis. To handle the large action space, we introduce a reject sampling strategy to reduce redundancy and improve exploration efficiency. Furthermore, a confirmation reward and a class-sensitive diagnosis reward are designed to guide accurate diagnosis prediction. Extensive experimental results demonstrate that DiaLLM outperforms baselines in clinical test recommendation and diagnosis prediction. Our code is available at Github†.

1 Introduction

Rapid advancements in LLMs (Touvron et al., 2023; Wu et al., 2023) expanded opportunities to improve diagnostic assistance and patient interactions in the healthcare domain(Biswas, 2023; Li et al., 2023; Shah, 2024; Singhal et al., 2023), and clinical conversational systems (Wang et al., 2023a; Yang et al., 2024b) have emerged as a promising approach to enhance clinical reasoning and assist doctors with diagnosis. However, as shown in Table 1, existing studies primarily rely on synthetic dialogues generated from medical knowledge graph (Wang et al., 2023a) or open medical Question Answer (QA) (Labrak et al., 2024) and are difficult to work on real-world healthcare settings which use Electronic Health Records (EHR). Furthermore, the diagnosis workflow involves multiple critical sub-tasks, including the inquiry of clinical lab tests and the interpretation of their results before giving the diagnosis results, which are often neglected in current approaches (Zhou et al., 2023; Liu et al., 2024). Consequently, existing explorations in the clinical conversational system are still far from practical healthcare scenarios, primarily due to challenges in understanding EHR and adapting to real-world diagnosis workflows (Li et al., 2024).

How to learn from EHR data? EHR data is a comprehensive digital record encompassing a patient’s medical history, treatments, test results, and clinical decisions. As shown in Figure 1, EHR captures multiple clinical visits and can be structured as a dialogue for clinical conversational systems. However, its heterogeneity and domain-specific nature pose a fundamental challenge, limiting compatibility with existing LLMs (Li et al., 2024). Unlike general NLP tasks, EHR data includes nu-

<table>
  <tr>
    <th>Patient Name</th>
    <th>Gender</th>
    <th>Alert</th>
    <th colspan="2">Surfa-Drugs</th>
    <th rowspan="2">Results</th>
    <th rowspan="2">Units</th>
    <th rowspan="2">Photo</th>
  </tr>
  <tr>
    <td>Smith</td>
    <td></td>
    <td></td>
    <td>Phone</td>
    <td>Age</td>
  </tr>
  <tr>
    <td>365-5372-2186</td>
    <td></td>
    <td></td>
    <td></td>
    <td>27</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <th>Date</th>
    <th>Code System</th>
    <th>Code</th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <td>20190104</td>
    <td>LOINC</td>
    <td>48643-1</td>
    <td></td>
    <td></td>
    <td>84.34</td>
    <td>mL/min (1.73 m2)</td>
    <td></td>
  </tr>
  <tr>
    <td>20190104</td>
    <td>ICD-10</td>
    <td>R10.12</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>20190110</td>
    <td>LOINC</td>
    <td>48647-5</td>
    <td></td>
    <td></td>
    <td>90.86</td>
    <td>mL/min (1.73 m2)</td>
    <td></td>
  </tr>
  <tr>
    <td>20190110</td>
    <td>LOINC</td>
    <td>8462-4</td>
    <td></td>
    <td></td>
    <td>93</td>
    <td>mm[Hg]</td>
    <td></td>
  </tr>
  <tr>
    <td>20190115</td>
    <td>LOINC</td>
    <td>2160-0</td>
    <td></td>
    <td></td>
    <td>0.73</td>
    <td>mg/dl</td>
    <td></td>
  </tr>
  <tr>
    <td>20190115</td>
    <td>LOINC</td>
    <td>29463-7</td>
    <td></td>
    <td></td>
    <td>175</td>
    <td>lb_a</td>
    <td></td>
  </tr>
  <tr>
    <td>20190117</td>
    <td>ICD-10</td>
    <td>Q65.89</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>20190117</td>
    <td>ICD-10</td>
    <td>M87.00</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
</table>

Figure 1: Structured representation of EHR data, illustrating patient demographics, symptoms, clinical tests (e.g., lab test, vital signs), and physician decisions over time. Symptoms and diagnoses are encoded using ICD codes, while clinical tests follow the LOINC system.
<table>
  <tr>
    <th>Model</th>
    <th>Main Data Source</th>
    <th>Function</th>
    <th>Training Method</th>
    <th>Language</th>
  </tr>
  <tr>
    <td>ChatDoctor (Li et al., 2023)</td>
    <td>Medical Consultation Website</td>
    <td>Diagnosis Assistance</td>
    <td>SFT</td>
    <td>English</td>
  </tr>
  <tr>
    <td>DoctorGLM (Xiong et al., 2023)</td>
    <td>Physician-Patient Dialogues</td>
    <td>Diagnosis Assistance</td>
    <td>SFT</td>
    <td>Chinese</td>
  </tr>
  <tr>
    <td>BenTsao (Wang et al., 2023a)</td>
    <td>Medical knowledge Graphs</td>
    <td>Diagnosis Assistance</td>
    <td>SFT</td>
    <td>Chinese</td>
  </tr>
  <tr>
    <td>Zhongjing (Yang et al., 2024b)</td>
    <td>Proprietary data, Crawled data</td>
    <td>Diagnosis Assistance</td>
    <td>SFT + RLHF</td>
    <td>Chinese</td>
  </tr>
  <tr>
    <td>Biomistral (Labrak et al., 2024)</td>
    <td>Medical QA</td>
    <td>Diagnosis Assistance</td>
    <td>SFT</td>
    <td>English</td>
  </tr>
  <tr>
    <td>Meditron (Chen et al., 2024)</td>
    <td>PubMed articles</td>
    <td>Diagnosis Assistance</td>
    <td>SFT</td>
    <td>English</td>
  </tr>
  <tr>
    <td>DiaLLM</td>
    <td>EHR data</td>
    <td>Diagnosis Assistance, Test Results Analysis, Clinical Test Ordering</td>
    <td>PPO</td>
    <td>English</td>
  </tr>
</table>

Table 1: Comparison of Data Sources, Function, Training Method, and Language across Popular Medical Conversational Models.

merical values, categorical attributes from clinical tests, and domain-specific terminologies such as ICD codes and LOINC lab identifiers (Sui et al., 2024; Liang et al., 2024). These complexities require precise numerical reasoning and contextual understanding. While LLMs benefit from massive text-based data and are trained using next-token prediction (Zhao et al., 2023), they exhibit significant weaknesses in understanding numeric and specialized clinical knowledge (Sui et al., 2024).

Which Service should Clinical Conversational System Provide? The second challenge lies in the alignment between LLM-supported conversational system and clinical workflows. As shown in Table 1, existing works primarily focus on diagnosis assistance using "symptom-diagnosis" conversational data, which oversimplifies the diagnostic process and limits practical applicability. In a typical diagnostic scenario, a patient presents symptoms, prompting the clinician to iteratively gather information through inquiries and clinical lab tests. This process follows a cycle of 'evidence acquisition, results interpretation, and diagnosis confirmation', with the patient’s medical trajectory information promptly recorded in the EHR data. Simulating clinicians’ evidence acquisition and automated diagnosis process within LLMs still remains unexplored (Zhou et al., 2023).

Motivation of DiaLLM. In this work, we propose a novel conversational agent, named DiaLLM, which provides an EHR-grounded transformation pipeline and explicitly models clinicians’ reasoning processes for evidence acquisition and automated diagnosis.

Technically, the EHR-grounded transformation aims to convert EHR data into dialogues that are aligned with common-sense knowledge and interpretable to LLMs. It first converts the EHR data into a single or multi-turn dialogue dataset, structured according to the patient’s clinical visit timeline. Then, we translate the heterogeneous dialogue into clinically-grounded text by designing a Clinical Test Reference (CTR) strategy. The CTR facilitates (1) the translation of standardized clinical codes (such as ICD-9/10, LOINC) into clinically-grounded text; and (2) the interpretation of clinical test results, conditioned on the patient’s age and gender.

On top of transformed EHR data, DiaLLM models evidence acquisition and automated diagnosis with a reinforcement learning framework. A policy network selects clinical tests (i.e., take actions) or terminates the process to make a diagnosis. Upon termination, a supervised classification model is invoked for disease diagnosis. To handle the large action space of clinical tests, we introduce a novel rejection sampling strategy (Bardenet et al., 2014; Fan et al., 2023; Mandel et al., 2016) that pre-filters redundant or unnecessary tests, ensuring only clinically relevant ones are selected. Meanwhile, since patients can have multiple diagnoses and disease distributions exhibit a long-tail pattern across populations, we propose a new confirmation reward and a class-sensitive classification reward to enhance diagnosis prediction.

Evaluation. We propose a comprehensive evaluation framework for medical LLMs, assessing both single-turn and multi-turn consultations. Experimental results show that DiaLLM outperforms both general-purpose and medical-specific LLMs in clinical test selection and diagnosis prediction. Our ablation study reveals that integrating the EHR-grounded transformation pipeline and specialized reward modeling significantly improves clinical test comprehensiveness, result interpretation, and early diagnosis accuracy.

2 Related Works

Representation Learning for EHR data. The majority of existing studies enhance medical code rep-
representations by incorporating external relational information through medical ontologies (Choi et al., 2017; Panigutti et al., 2020) and knowledge graphs (Jiang et al., 2023b; Wang et al., 2023b). However, these established medical knowledge sources are restricted to specific diseases (Si et al., 2021). Recent works have attempted to derive clinical concept embeddings from large-scale medical text corpora (Ye et al., 2021), learning relational graphs through self-supervised learning (Yao et al., 2024), contrastive learning (Cai et al., 2022), or generating medical concepts directly using LLMs (Ma et al., 2024). However, these approaches often lack clinically grounded annotations, making the learned embeddings incompatible with LLM inputs.

Another line of research improves clinical prediction performance by learning from clinical test results. Existing works derive partition functions to learn from numerical and categorical clinical test results (La Cava et al., 2019), employing methods such as XGBoost (Chen and Guestrin, 2016), additive models (Hastie, 2017), piecewise linear functions (Montomoli et al., 2021), and logic-based rule learning (Ren et al., 2024). However, these approaches can be susceptible to biases introduced by patient population variations and may not align with established guidelines for interpreting test results (Ren et al., 2024).

Diagnosis-oriented Conversational System. Early research primarily focused on ICU-based temporal EHR data (Yoon et al., 2019; Fansi Tchango et al., 2022; Qin et al., 2024; He and Chen, 2022) or specific diagnosis categories (He and Chen, 2022; Fansi Tchango et al., 2022), and proposed to model evidence inquiry and diagnosis as a Markov decision process (MDP) (Tang et al., 2016). With the rise of foundation models, clinical conversational systems have been explored by tuning on various medical corpora, including clinical conversations collected from online medical consultation website (Li et al., 2023), symptom-diagnosis dialogues (Toma et al., 2023), medical question-answering pairs (Han et al., 2023), and knowledge graph-generated dialogues (Yang et al., 2024b; Wang et al., 2023a). However, these curated datasets deviate from real-world data distribution, lack essential clinical test support and do not interpret lab test results. These shortcomings limit their real-world applicability. For a comprehensive review, see (Zhou et al., 2023). In contrast, our approach constructs single-turn and multi-turn dialogue data leveraging real-world EHR data, and facilitates lab test requesting and diagnosis prediction.

3 Methodology

3.1 Diagnostic Conversational System Setup

Task Definition and Notations. DiaLLM encapsules the evidence acquisition and diagnosis automation with the following steps:

1. Initial Query: The patient provides demographic information d and symptoms s.

2. Clinical Test Recommendation: The LLM agent suggests initial clinical tests \( c_0 \).

3. Test Result Analysis and Follow-ups: Upon receiving the patient’s initial clinical test results \( v_0 \), the agent conducts a clinically grounded analysis and iteratively recommends additional follow-up tests \( c_{t>0} \) and interprets new results \( v_{t>0} \) to gather further evidence. This process continues until a conclusive diagnosis is achieved.

4. Diagnosis Prediction: The system ultimately predicts the diagnosis \( y \).

Problem Formulation. This process can be modeled as a Markov Decision Process (MDP) \( \mathcal{M}(\mathcal{S}, \mathcal{A}, \mathcal{R}, \gamma) \), where: \( \mathcal{S} = \mathcal{S}' \cup \{ s_\perp \} \) is the state space, with \( s_\perp \) as the terminal state. \( \mathcal{A} = \mathcal{A}' \cup \{ a_\perp \} \) is the action space, with \( a_\perp \) as the stop action. \( \gamma \) is the discount factor. Each dialogue eEach dialogue consists of at most \( T \) turns, where \( T \) represents the maximum number of visits recorded in the EHR data, consists of at most \( T \) turns, where \( T \) represents the maximum number of visits recorded in the EHR data. At turn \( t \), the state \( s_t \in \mathcal{S}' \) encodes socio-demographics, acquired evidence and dialogue history \( h_t \):

\[
s_t = \{ d, s, c_t, v_t, h_t \}.
\]

At turn \( t \), the agent selects an action \( a_t \), determining whether to request further tests or stop. The clinical test recommendation model \( \pi_\theta(a_t|s_t) \) governs test selection, while the diagnosis prediction model \( \pi_\phi(y|s_\perp) \) makes the final diagnosis.

Overview of DiaLLM. In this work, we present DiaLLM designed to enhance evidence acquisition, result interpretation, and the diagnostic workflow. It consists of two key components: (1) a transformation strategy that constructs dialogues from heterogeneous EHR data, aligning them to
Figure 2: DiaLLM operates in two stages: (I) Dialogue Data Construction, where EHR data is transformed into clinically grounded dialogue data, and (II) Reward Modeling, which optimizes clinical test selection and diagnosis prediction. In Stage I, DiaLLM converts clinical test codes and results into texts using our constructed Clinical Test Reference. In Stage II, we adopt a novel rejection sampling strategy to handle the large action space and incorporate two reward signals to facilitate diagnosis prediction learning.

common-sense texts for the ease of understanding by LLMs; and (2) a reinforcement learning framework to empower the base LLM with evidence acquisition and automated diagnosis capabilities, for which we design a reject sampling strategy and several reward signals to facilitate learning. Both components will be detailed later in this section.

3.2 Dialogue Data Construction from EHR.

We introduce an EHR transformation strategy to convert structured EHR into clinically grounded dialogues. This process begins by segmenting patient visit records into conversational episodes when clinical visit intervals are within one week. A major challenge lies in representing medical terminologies (e.g., codes and lab test identifiers) and interpreting lab results. While text-driven LLMs excel in natural language understanding and reasoning, they lack direct exposure to structured EHR data. To address this, we manually curate a Clinical Test Reference data to translate heterogeneous EHR data in a clinically meaningful manner.

The transformation comprises two components: (1) Clinical Code Translation that converts specialized medical terms, including symptom codes s_t, diagnosis codes y_t, and clinical test codes c_t into clinically grounded and common-sense language that is easily understandable by LLMs. An example of the clinical code translation data is provided in Table 6 in the Appendix. (2) Clinical Test Interpretation that transforms heterogeneous lab test results using domain knowledge, including grounded reference ranges conditioned on gender and age, and classifies test results as 'normal' or 'abnormal'. An example of the clinical test interpretation data is shown in Table 7 in the Appendix.

Details of Clinical Test Reference database. For clinical code translation, symptom and diagnosis codes are commonly recorded using the ICD system, such as ICD-9 and ICD-10. To standardize code descriptions, we build a comprehensive codebase that includes ICD-9, ICD-10, and a mapping from ICD-9 to ICD-10 for standardization. Clinical tests are generally coded using the LOINC system. We annotate 735 clinical test code descriptions (including both lab test and vital sign codes) 2 which cover most of diseases to ensure comprehensive coverage and standardization.

For clinical test interpretation, we annotate 262 commonly used clinical tests based on established

2LOINC Organization: https://loinc.org/
medical guidelines3. This resulted in 1,163 annotations4 categorized as follows: (1) Normal Ranges that define reference ranges considering age and gender influences. (2) Critical Values that identifies thresholds indicating life-threatening conditions requiring immediate intervention. (3) Demographic Variability that includes age- and gender-specific ranges to account for physiological differences. (4) Units of Measurement that ensures consistency across international and clinical standards.

Example. Given the EHR example: "Female, Age 27, LOINC 2160 0.48 mg/dL," the CTR translates it to: "Given the patient demographic information (Age: 27, gender: Female). These lab tests show normal results: Creatinine in Serum or Plasma".

3.3 Learning for Lab Test and Diagnosis.

On top of transformed EHR data, DiaLLM models lab test acquisition and automated diagnosis using a reinforcement learning framework. At each step, the model will select follow-up lab tests (i.e., take actions) or termination of this process to make a diagnosis. To facilitate learning, we adopt rejection sampling to reduce the complexity. During fine-tuning with PPO, both a confirmation-based reward and a Class-sensitive Diagnosis reward are utilized.

Rejection Sampling for Lab Test Selection. In the presence of cost pressure, the goal of the agent is to balance timely and accurate evidence acquisition with cost-effective feature selection. The vast number of potential lab tests leads to a large action space, making direct reinforcement learning (RL) inefficient. To mitigate this, we introduce a strategy based on rejection sampling (Bardenet et al., 2014; Mandel et al., 2016) which pre-filters redundant or unnecessary lab tests before execution, ensuring that only clinically relevant tests are selected.

At each decision step t, the actor model \( \pi(a_t|s_t) \) samples a candidate lab test \( c_t \) from the full action space. Instead of executing the sampled action immediately, we introduce a rejection sampling mechanism that determines whether the test should be conducted based on its informativeness, cost, and redundancy. The probability of accepting a test \( p_{accept}(c_t|s_t) \) is defined as:

\[
p_{accept}(c_t \mid s_t) = \frac{H(y \mid s_t) - H(y \mid s_t, v_t)}{\max_{c_t} H(y \mid s_t) - H(y \mid s_t, v_t)} \cdot 1[c_t \notin C_{prev}],
\]

where \( H(y \mid s_t, v_t) \) denotes the entropy of predicted diagnosis given the current sate \( s_t \) and value \( v_t \) of candidate test \( c_t \), capturing how much uncertainty is reduced by performing \( c_t \). The redundancy filter \( 1[c_t \notin C_{prev}] \) prevents reordering previously conducted tests.

Confirmation Reward. As the conversation progresses, the agent should systematically acquire more evidence to refine its belief in the correct diagnosis while minimizing uncertainty. Inspired by potential-based reward shaping (Hu et al., 2020), we formalize the confirmation reward \( R_{Co} \) as:

\[
R_{Co}(s_t, a_t; s_{t+1}) = 1_{s_{t+1} \neq s_\perp} \cdot (CE(\hat{y}_{t+1}, y) - CE(\hat{y}_t, y)),
\]

where \( 1_{s_{t+1} \neq s_\perp} \) is an indicator that the terminal state has not been reached yet.

Class-sensitive Diagnosis Reward. This reward is designed to provide feedback to the agent by evaluating the quality of its final predicted diagnosis with respect to the ground truth diagnosis \( y \) once the interaction process is completed. To directly optimize diagnosis prediction while addressing the issue of class imbalance (Puthiya Parambath et al., 2014; Elkan, 2001), we introduce a weighted classification reward:

\[
R_{CI}(s_T) = \sum_{y_i \in y} w_{CI}(y_i) \cdot \mathrm{CE}(\hat{y}_i, y_i),
\]

where \( w_{CI}(y_i) = \frac{1}{p(y_i)} \) assigns a higher weight to rare diagnoses based on the inverse of the class frequency. This reward adjusts the importance of correctly predicting minority classes, ensuring that the agent focuses on both high-risk and low-risk diagnoses effectively.

4 Experiments

4.1 Dataset and Task Description

To critically evaluate the performance of LLMs on EHR data, we design the dataset with two key considerations: (I) Unexposed: Ensuring that the dataset has not been previously used by most LLMs. (II) Predictive: Focusing on diseases that can be predicted solely from EHR data, reflecting
their clinical applicability. Detailed data statistical analysis is listed in Table 8 in Appendix A.2.1.

NHANES Dataset. Following (Kachuee et al., 2019), we construct a diabetes dataset from the public National Health and Nutrition Examination Survey (NHANES) 5. The NHANES dataset does not include clinical visit time information, and we construct a single-turn dialogue based on this data. The dataset includes 8,897 patients and 45 heterogeneous features, such as demographic data, lab results (e.g., total cholesterol, triglycerides), physical measurements (e.g., weight, height), and responses to questionnaires (e.g., smoking, alcohol consumption).

TriNetX Dataset. TriNetX6 is a global health research network providing access to large-scale de-identified patient EHR data. TriNetX includes records from 33,105 de-identified patients, spanning data from 1982 to 2023, collected across more than 100 community hospitals and 500 outpatient clinics. To satisfy goal (I) and (II), we extract data on metabolic, respiratory, and circulatory diseases, referred to as TriNetX-Metabolic, TriNetX-Respiratory and TriNetX-Circulatory, respectively.

4.2 Baselines and Evaluation Metrics
We compare DiaLLM with (1) pretrained foundation models7 including Mistralv0.3-7B Instruct (Jiang et al., 2023a), Llama3.1-8B Instruct (Dubey et al., 2024), Qwen2.5-7B Instruct (Yang et al., 2024a); and (2) medical-specialized models include BioMistral (Labrak et al., 2024), Meditron-7B (Chen et al., 2024), Meditron3-8B (Chen et al., 2023), Chatdoctor (Li et al., 2023). To evaluate zero-shot performance, we incorporate diagnosis labels in the prompt. For foundation models, we also test an alternative approach where we extract their generated embeddings and train an MLP for prediction.

We evaluate the prediction performance through two dimensions: (1) Coverage Ability. We use the recall@5 and F1 to measure the coverage ability. Since there are no lab test requests in single-turn dialogues, we evaluate lab test performance on the multi-turn dialogue data using recall@5. (2) Early-Prediction Ability. We utilize Mean Reciprocal Rank (MRR) to indicate the effectiveness of early diagnosis prediction. Details about evaluation metrics are shown in A.2.2 in Appendix.

4.3 Implementation Details
During implementation, we select Llama3.1-8B as the backbone foundation model, and append a mean-pooling layer and a two-layer MLP to it for lab test recommendation and diagnosis prediction. During tuning, \( r \) is set to 16 for the LoRA adapters (Hu et al., 2021). Batch size is set to 4 and learning rate is \( 1e - 4 \). Training epoch is set to 5. We randomly run each experiment twice and report the mean. We run experiments on four A100, and train:eval:test is set to 8:1:1.

4.4 Main Results and Analysis

Results on Single-turn Prediction. For single-turn diagnosis, additional lab test query is not needed. For space limitation, experimental results are presented in Table 2 and Table 4 in Appendix A.2.3. Several key observations can be made from the following dimensions: (1) Superiority of DiaLLM. It is obvious that DiaLLM consistently outperforms all baselines, demonstrating its effectiveness. This improvement is mainly due to the EHR transformation pipeline and rewarding models for PPO, which incorporate clinically grounded knowledge, thereby enhancing the model’s decision-making abilities. (2) Impact of Tuning. Comparing base models (e.g., Mistralv0.3-7B, Llama3.1-8B, Qwen2.5-7B) with their supervised fine-tuned counterparts, we observe substantial performance gains across all metrics, highlighting the importance of task-specific fine-tuning in improving decision-making in clinical dialogue tasks. (3) Performance of clinical-Specific Models. The performance of BioMistral and ChatDoctor is significantly lower than that of fine-tuned general-purpose models. This suggests that current biomedical LLMs are primarily trained for general clinical tasks, such as clinical QA, rather than being optimized for real medical data. (4) Comparison with Fine-Tuned Baselines. Among fine-tuned models, those on Llama3.1-8B and Qwen2.5-7B perform competitively. However, DiaLLM consistently outperforms them, indicating that its enhanced clinical grounding and policy optimization contribute to superior prediction.

Results on Multi-turn Prediction. For multi-turn dialogue data, models can query additional lab tests before giving the final prediction. As shown in Table 3, performance of DiaLLM significantly exceeds baseline models, similar to previous single-turn cases. The performance of general-

5https://www.cdc.gov/nchs/nhanes/
6https://trinetx.com/
7For simplicity, we use shortened names later.
<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="4">TriNetX-Metabolic</th>
    <th colspan="4">TriNetX-Respiratory</th>
    <th colspan="4">TriNetX-Circulatory</th>
  </tr>
  <tr>
    <th>Recall@5</th>
    <th>F1</th>
    <th>MRR</th>
    <th>Recall@5</th>
    <th>F1</th>
    <th>MRR</th>
    <th>Recall@5</th>
    <th>F1</th>
    <th>MRR</th>
  </tr>
  <tr>
    <td>Mistralv0.3-7B</td>
    <td>3.64</td>
    <td>11.89</td>
    <td>10.12</td>
    <td>4.54</td>
    <td>7.84</td>
    <td>12.09</td>
    <td>10.98</td>
    <td>8.67</td>
    <td>17.57</td>
  </tr>
  <tr>
    <td>Llama3.1-8B</td>
    <td>5.34</td>
    <td>9.28</td>
    <td>10.95</td>
    <td>7.49</td>
    <td>12.05</td>
    <td>15.48</td>
    <td>17.02</td>
    <td>10.89</td>
    <td>22.97</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td>8.58</td>
    <td>10.51</td>
    <td>12.11</td>
    <td>10.03</td>
    <td>10.64</td>
    <td>16.25</td>
    <td>5.84</td>
    <td>7.82</td>
    <td>9.68</td>
  </tr>
  <tr>
    <td>BioMistral</td>
    <td>39.67</td>
    <td>13.82</td>
    <td>26.10</td>
    <td>25.95</td>
    <td>15.50</td>
    <td>33.78</td>
    <td>20.01</td>
    <td>16.62</td>
    <td>31.08</td>
  </tr>
  <tr>
    <td>ChatDoctor</td>
    <td>19.16</td>
    <td>11.57</td>
    <td>20.82</td>
    <td>35.88</td>
    <td>9.71</td>
    <td>26.29</td>
    <td>10.03</td>
    <td>10.03</td>
    <td>18.34</td>
  </tr>
  <tr>
    <td>Meditron-7B</td>
    <td>6.84</td>
    <td>9.31</td>
    <td>11.66</td>
    <td>11.58</td>
    <td>10.34</td>
    <td>16.80</td>
    <td>15.07</td>
    <td>14.71</td>
    <td>18.88</td>
  </tr>
  <tr>
    <td>Meditron3-8B</td>
    <td>5.99</td>
    <td>6.46</td>
    <td>10.74</td>
    <td>11.98</td>
    <td>13.28</td>
    <td>15.42</td>
    <td>10.13</td>
    <td>16.04</td>
    <td>19.34</td>
  </tr>
  <tr>
    <td>Mistralv0.3-7B<sub>mlp</sub></td>
    <td>66.72</td>
    <td>50.92</td>
    <td>45.61</td>
    <td>67.71</td>
    <td>54.95</td>
    <td>45.64</td>
    <td>67.51</td>
    <td>55.95</td>
    <td>45.74</td>
  </tr>
  <tr>
    <td>Llama3.1-8B<sub>mlp</sub></td>
    <td>68.11</td>
    <td>55.01</td>
    <td>45.76</td>
    <td>67.66</td>
    <td>56.80</td>
    <td>45.77</td>
    <td>66.72</td>
    <td>56.23</td>
    <td>45.76</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B<sub>mlp</sub></td>
    <td>69.16</td>
    <td>52.95</td>
    <td>45.72</td>
    <td>67.66</td>
    <td>55.21</td>
    <td>45.71</td>
    <td>68.86</td>
    <td>55.18</td>
    <td>45.79</td>
  </tr>
  <tr>
    <td>DiaLLMs</td>
    <td><b>79.54</b></td>
    <td><b>76.21</b></td>
    <td><b>47.33</b></td>
    <td><b>78.19</b></td>
    <td><b>74.82</b></td>
    <td><b>48.18</b></td>
    <td><b>77.75</b></td>
    <td><b>74.36</b></td>
    <td><b>48.14</b></td>
  </tr>
</table>

Table 2: Performance Comparison of DiaLLM and Baselines on the Constructed Single-turn Dialogue.

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="4">TriNetX-Metabolic</th>
    <th colspan="4">TriNetX-Respiratory</th>
    <th colspan="4">TriNetX-Circulatory</th>
  </tr>
  <tr>
    <th>Diagnosis</th>
    <th>Lab Test</th>
    <th>Recall@5</th>
    <th>F1</th>
    <th>MRR</th>
    <th>Diagnosis</th>
    <th>Lab Test</th>
    <th>Recall@5</th>
    <th>F1</th>
    <th>MRR</th>
    <th>Diagnosis</th>
    <th>Lab Test</th>
    <th>Recall@5</th>
    <th>F1</th>
    <th>MRR</th>
    <th>Recall@5</th>
    <th>F1</th>
    <th>MRR</th>
    <th>Recall@5</th>
    <th>F1</th>
    <th>MRR</th>
  </tr>
  <tr>
    <td>Mistralv0.3-7B</td>
    <td>6.89</td>
    <td>8.07</td>
    <td>13.23</td>
    <td>0.58</td>
    <td>14.44</td>
    <td>16.02</td>
    <td>11.98</td>
    <td>1.96</td>
    <td>18.00</td>
    <td>16.93</td>
    <td>25.39</td>
    <td>0.21</td>
  </tr>
  <tr>
    <td>Llama3.1-8B</td>
    <td>12.44</td>
    <td>11.84</td>
    <td>14.79</td>
    <td>8.52</td>
    <td>18.22</td>
    <td>17.84</td>
    <td>26.08</td>
    <td>0.75</td>
    <td>12.72</td>
    <td>13.24</td>
    <td>21.24</td>
    <td>1.87</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td>7.56</td>
    <td>16.68</td>
    <td>17.86</td>
    <td>0.61</td>
    <td>5.56</td>
    <td>8.90</td>
    <td>15.03</td>
    <td>1.33</td>
    <td>8.22</td>
    <td>9.23</td>
    <td>15.23</td>
    <td>1.75</td>
  </tr>
  <tr>
    <td>Meditron-7B</td>
    <td>4.22</td>
    <td>8.31</td>
    <td>13.28</td>
    <td>1.21</td>
    <td>8.89</td>
    <td>13.04</td>
    <td>14.67</td>
    <td>0.79</td>
    <td>12.22</td>
    <td>12.43</td>
    <td>21.90</td>
    <td>1.29</td>
  </tr>
  <tr>
    <td>Meditron3-8B</td>
    <td>16.22</td>
    <td>21.25</td>
    <td>21.57</td>
    <td>2.90</td>
    <td>17.33</td>
    <td>15.38</td>
    <td>24.32</td>
    <td>1.37</td>
    <td>8.22</td>
    <td>15.25</td>
    <td>17.26</td>
    <td>1.91</td>
  </tr>
  <tr>
    <td>BioMistral</td>
    <td>9.37</td>
    <td>13.54</td>
    <td>16.62</td>
    <td>2.33</td>
    <td>12.67</td>
    <td>19.51</td>
    <td>22.98</td>
    <td>1.70</td>
    <td>4.44</td>
    <td>13.37</td>
    <td>10.56</td>
    <td>0.79</td>
  </tr>
  <tr>
    <td>ChatDoctor</td>
    <td>6.87</td>
    <td>9.71</td>
    <td>18.92</td>
    <td>1.77</td>
    <td>2.89</td>
    <td>9.66</td>
    <td>9.44</td>
    <td>0.95</td>
    <td>16.00</td>
    <td>19.22</td>
    <td>22.08</td>
    <td>1.18</td>
  </tr>
  <tr>
    <td>Mistralv0.3-7B<sub>mlp</sub></td>
    <td>57.90</td>
    <td>56.03</td>
    <td>37.42</td>
    <td>10.45</td>
    <td>59.76</td>
    <td>59.14</td>
    <td>38.90</td>
    <td>10.51</td>
    <td>59.56</td>
    <td>58.90</td>
    <td>39.93</td>
    <td>10.39</td>
  </tr>
  <tr>
    <td>Llama3.1-8B<sub>mlp</sub></td>
    <td>57.31</td>
    <td>55.92</td>
    <td>37.43</td>
    <td>10.64</td>
    <td>61.22</td>
    <td>59.58</td>
    <td>39.42</td>
    <td>10.97</td>
    <td>61.21</td>
    <td>59.47</td>
    <td>40.13</td>
    <td>10.16</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B<sub>mlp</sub></td>
    <td>58.01</td>
    <td>56.72</td>
    <td>38.37</td>
    <td>11.04</td>
    <td>61.91</td>
    <td>60.21</td>
    <td>39.72</td>
    <td>11.13</td>
    <td>60.89</td>
    <td>58.54</td>
    <td>38.46</td>
    <td>10.30</td>
  </tr>
  <tr>
    <td>DiaLLMs</td>
    <td><b>70.22</b></td>
    <td><b>73.59</b></td>
    <td><b>41.88</b></td>
    <td><b>11.43</b></td>
    <td><b>70.89</b></td>
    <td><b>75.12</b></td>
    <td><b>40.52</b></td>
    <td><b>12.31</b></td>
    <td><b>71.11</b></td>
    <td><b>75.39</b></td>
    <td><b>43.34</b></td>
    <td><b>11.55</b></td>
  </tr>
</table>

Table 3: Performance Comparison of DiaLLM with Baselines on the Constructed Multi-turn Dialogue.

used LLMs, e.g., Mistralv0.3-7B, Llama3.1-8B, Qwen2.5-7B still falls short in comparison to fine-tuned models and DiaLLM, indicating that they lack the clinical specificity necessary for multi-turn dialogues. Medical-specific LLMs like BioMistral, Meditron, and ChatDoctor underperform compared to general-purpose models such as Llama3.1-8B and Qwen2.5-7B. This is likely due to their training on open medical QA datasets or synthetic medical dialogues, which fail to capture the complexity of real-world clinical practice.

4.5 Ablation Study

To evaluate the effectiveness of different components in DiaLLM, we conduct an ablation study to quantify the contributions of the EHR transformation pipeline and the rewards in PPO. For space limitation, experimental results on single-turn and multi-turn dialogue data are shown in Figure 3, Figure 4 and Figure 5 in Appendix A.2.4.

4.5.1 Ablation Study on EHR Transformation

We assess the impact of different EHR transformation components by progressively removing them: (1) DiaLLM (w/o CT): Removes code translation (CT). (2) DiaLLM (w/o CTI): Removes clinical test interpretation (CTI). (3) DiaLLM (w/o Both): Removes all EHR transformation components.

These results highlight the importance of integrating clinical grounded knowledge into LLMs. DiaLLM (w/o Both) results in a noticeable performance drop in both single-turn and multi-turn dialogue data, confirming that the EHR transformation strategy enhances the model’s ability to understand EHR data and improves diagnostic predictions. Besides, DiaLLM (w/o CTI) exhibits lower performance than DiaLLM (w/o CT) across most datasets, highlighting the necessity of understanding clinical terminology and numerical values.
(a) TriNetX-Metabolic Dataset.    (b) TriNetX-Respiratory Dataset.    (c) TriNetX-Circulatory Dataset.

Figure 3: Ablation Study on Single-turn Dialogue Data.

(a) TriNetX-Metabolic Dataset.    (b) TriNetX-Respiratory Dataset.    (c) TriNetX-Circulatory Dataset.

Figure 4: Ablation Study on Multi-turn Dialogue Data.

4.5.2 Ablation Study on Reward Modeling
We analyze the influence of specific PPO mechanisms: (1) DiaLLM (w/o PPO-RJ): Removes reject sampling. (2) DiaLLM (w/o PPO-Conf): Removes confirmation reward. (3) DiaLLM (w/o PPO-Imb): Removes imbalanced reward. Reward modeling is primarily effective for multi-turn dialogue data, and experimental results are shown in Figure 4.

Each PPO component contributes to model performance, as elaborated in the experimental results. In the TriNetX-Respiratory multi-turn dialogue dataset, removing the imbalanced reward reduces the F1-score to 0.6954, while removing the confirmation reward lowers it to 0.7129. These declines highlight the importance of addressing class imbalance (PPO-Imb) and incorporating confidence-based rewards (PPO-Conf) to improve classification quality, balance predictions, and enhance decision reliability. Furthermore, the EHR transformation component has a greater impact on model performance than reward modeling in multi-turn dialogue data, showing the importance of aligning medical terms with common-sense knowledge.

4.6 Case Study
We qualitatively evaluate the diagnostic recommendations of various models using the same example dialogue, as presented in Table 9 in the Appendix A.2.5. The selected case presents a complex scenario requiring not only diagnosis prediction but also the integration of additional clinical test evidence. DiaLLM initially recommended tests for carbon dioxide, chloride, etc. Upon receiving the results, it identified abnormalities in carbon dioxide and chloride levels and suggested further tests for glucose and sodium. Based on these analysis, DiaLLMs can recommend relevant diagnoses.

In contrast, all baseline medical LLMs lacked the ability to inquire about appropriate clinical tests or effectively analyze test results, highlighting a significant gap in diagnostic reasoning. BioMistral has a narrow diagnostic scope, potentially missing relevant diagnoses. While Meditron-7B and 8B considered a broader range of conditions, their diagnostic results diverged from the ground truth. ChatDoctor’s diagnoses often lacked relevance to the patient’s symptoms and lab results, suggesting limited comprehension of the medical history. Among these, DiaLLM demonstrated the most comprehensive and accurate diagnostic approach, effectively inquiring, analyzing lab results, and adjusting recommendations, showcasing superior practical medical value.

5 Conclusion
In this work, we introduced DiaLLM, a clinical dialogue model that enhances medical conversational systems through a combination of clinical-grounded EHR transformation and PPO optimization. Experimental results consistently show that
DiaLLM outperforms existing models across a variety of metrics, demonstrating its superior prediction ability. By integrating domain-specific knowledge and a structured decision-making process, DiaLLM offers a significant improvement over both general-purpose and medical-specific LLMs in complex medical dialogues.

Limitations

Automation of CTR Database In this work, we recruited three medical students to annotate clinical test code descriptions and clinical value reference ranges based on medical guidelines and publicly available clinical resources (e.g., Mayo Clinic). Our constructed Clinical Test Reference data are crucial for diagnosis prediction using EHR data grounded in clinical knowledge. However, manual annotation is time-consuming and prone to variability. Future work will focus on automating this process to enhance efficiency and consistency.

Clinical Conversational System with Multimodal Data In this work, we focus on diseases that can be solely predicted through EHR data. Our approach focuses on EHR data and lab tests with numerical or textual features. With advances in multimodal learning, integrating imaging-based diagnostic tests, such as CT scans, could further enrich the model’s clinical reasoning capabilities. We also leave this extension for future exploration.

Ethic Statement

Data Collection This study utilizes publicly available EHR data and de-identified in-hospital records approved for research use. To safeguard patient privacy, we conducted a thorough review to ensure the dataset contains no sensitive information. We adhere to strict ethical standards in data handling and acknowledge the research community’s commitment to maintaining data integrity and privacy.

Compensation and Ethical Collaboration with Medical Professionals We collaborated with two senior medical graduates to identify diseases predictable solely from EHR data and to evaluate the constructed CTR. Each was compensated 200 RMB per hour, aligning with local salary standards to ensure fair remuneration for their expertise.

Concerning the Trustworthiness of DiaLLM While DiaLLM demonstrate strong diagnostic reasoning capabilities, its reliability in real-world clinical settings is not yet fully established. Potential risks include hallucinations, biases from training data, and inconsistencies in medical reasoning. To enhance trust, further research is needed to improve model transparency, robustness, and validation through expert evaluation. Developing rigorous verification mechanisms and integrating clinician oversight will be essential for safe deployment in medical practice.

References

Rémi Bardenet, Arnaud Doucet, and Chris Holmes. 2014. Towards scaling up markov chain monte carlo: an adaptive subsampling approach. In International conference on machine learning, pages 405–413. PMLR.

Som S Biswas. 2023. Role of chat gpt in public health. Annals of biomedical engineering, 51(5):868–869.

Derun Cai, Chenxi Sun, Moxian Song, Baofeng Zhang, Shenda Hong, and Hongyan Li. 2022. Hypergraph contrastive learning for electronic health records. In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), pages 127–135. SIAM.

Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794.

Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, et al. 2023. Meditron-70b: Scaling medical pretraining for large language models. arXiv preprint arXiv:2311.16079.

Zeming Chen, Angelika Romanou, Antoine Bonnet, Alejandro Hernández-Cano, Badr Alkhamissi, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, et al. 2024. Meditron: Open medical foundation models adapted for clinical practice.

Edward Choi, Mohammad Taha Bahadori, Le Song, Walter F Stewart, and Jimeng Sun. 2017. Gram: graph-based attention model for healthcare representation learning. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 787–795.

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.

Charles Elkan. 2001. The foundations of cost-sensitive learning. In International joint conference on artificial intelligence, volume 17, pages 973–978. Lawrence Erlbaum Associates Ltd.
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. 2023. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 36:79858–79885.

Arsene Fansi Tchango, Rishab Goel, Julien Martel, Zhi Wen, Gaetan Marceau Caron, and Joumana Ghosn. 2022. Towards trustworthy automatic diagnosis systems by emulating doctors’ reasoning with deep reinforcement learning. Advances in Neural Information Processing Systems, 35:24502–24515.

Tianyu Han, Lisa C Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexander Löser, Daniel Truhn, and Keno K Bressem. 2023. Medalpac—a open-source collection of medical conversational ai models and training data. arXiv preprint arXiv:2304.08247.

Trevor J Hastie. 2017. Generalized additive models. In Statistical models in S, pages 249–307. Routledge.

Weijie He and Ting Chen. 2022. Scalable online disease diagnosis via multi-model-fused actor-critic reinforcement learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4695–4703.

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.

Yuqing Hu, Weixun Wang, Hangtian Jia, Yixiang Wang, Yingfeng Chen, Jianye Hao, Feng Wu, and Changjie Fan. 2020. Learning to utilize shaping rewards: A new approach of reward shaping. Advances in Neural Information Processing Systems, 33:15931–15941.

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023a. Mistral 7b. arXiv preprint arXiv:2310.06825.

Pengcheng Jiang, Cao Xiao, Adam Cross, and Jimeng Sun. 2023b. Graphcare: Enhancing healthcare predictions with personalized knowledge graphs. arXiv preprint arXiv:2305.12788.

Mohammad Kachuee, Orpaz Goldstein, Kimmo Karkkainen, Sajad Darabi, and Majid Sarrafzadeh. 2019. Opportunistic learning: Budgeted cost-sensitive learning from data streams. arXiv preprint arXiv:1901.00243.

William La Cava, Christopher Bauer, Jason H Moore, and Sarah A Pendergrass. 2019. Interpretation of machine learning predictions for patient outcomes in electronic health records. In AMIA Annual Symposium Proceedings, volume 2019, page 572. American Medical Informatics Association.

Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. 2024. Biomistral: A collection of open-source pretrained large language models for medical domains. arXiv preprint arXiv:2402.10373.

Lingyao Li, Jiayan Zhou, Zhenxiang Gao, Wenyue Hua, Lizhou Fan, Huizi Yu, Loni Hagen, Yongfeng Zhang, Themistocles L Assimes, Libby Hemphill, et al. 2024. A scoping review of using large language models (llms) to investigate electronic health records (ehrs). arXiv preprint arXiv:2405.03066.

Yuxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. 2023. Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus, 15(6).

Yuxuan Liang, Haomin Wen, Yuqi Nie, Yushan Jiang, Ming Jin, Dongjin Song, Shurui Pan, and Qingsong Wen. 2024. Foundation models for time series analysis: A tutorial and survey. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pages 6555–6565.

Lei Liu, Xiaoyan Yang, Junchi Lei, Xiaoyang Liu, Yue Shen, Zhiqiang Zhang, Peng Wei, Jinjie Gu, Zhixuan Chu, Zhan Qin, et al. 2024. A survey on medical large language models: Technology, application, trustworthiness, and future directions. arXiv preprint arXiv:2406.03712.

Mingyu Derek Ma, Chenchen Ye, Yu Yan, Xiaoxuan Wang, Peipei Ping, Timothy Chang, and Wei Wang. 2024. Clibench: A multifaceted and multigranular evaluation of large language models for clinical decision making. arXiv preprint arXiv:2406.09923.

Travis Mandel, Yun-En Liu, Emma Brunskill, and Zoran Popović. 2016. Offline evaluation of online reinforcement learning algorithms. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30.

Jonathan Montomoli, Luca Romeo, Sara Moccia, Michele Bernardini, Lucia Migliorelli, Daniele Bernardini, Abele Donati, Andrea Carsetti, Maria Grazia Bocci, Pedro David Wendel Garcia, et al. 2021. Machine learning using the extreme gradient boosting (xgboost) algorithm predicts 5-day delta of sofa score at icu admission in covid-19 patients. Journal of Intensive Medicine, 1(02):110–116.

Cecilia Panigutti, Alan Perotti, and Dino Pedreschi. 2020. Doctor xai: an ontology-based approach to black-box sequential data classification explanations. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pages 629–639.

Shameem Puthiya Parambath, Nicolas Usunier, and Yves Grandvalet. 2014. Optimizing f-measures by cost-sensitive classification. Advances in neural information processing systems, 27.
Yuchao Qin, Mihaela van der Schaar, and Changhee Lee. 2024. Risk-averse active sensing for timely outcome prediction under cost pressure. Advances in Neural Information Processing Systems, 36.

Weijieying Ren, Xiaoting Li, Huiyuan Chen, Vineeth Rakesh, Zhuoyi Wang, Mahashweta Das, and Vasant G Honavar. 2024. Tablog: Test-time adaptation for tabular data using logic rules. In Forty-first International Conference on Machine Learning.

Savyasachi V Shah. 2024. Accuracy, consistency, and hallucination of large language models when analyzing unstructured clinical notes in electronic medical records. JAMA Network Open, 7(8):e2425953–e2425953.

Yuqi Si, Jingcheng Du, Zhao Li, Xiaoqian Jiang, Timothy Miller, Fei Wang, W Jim Zheng, and Kirk Roberts. 2021. Deep representation learning of patient data from electronic health records (ehr): A systematic review. Journal of biomedical informatics, 115:103671.

Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2023. Large language models encode clinical knowledge. Nature, 620(7972):172–180.

Yuan Sui, Mengyu Zhou, Mingjie Zhou, Shi Han, and Dongmei Zhang. 2024. Table meets llm: Can large language models understand structured table data? a benchmark and empirical study. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 645–654.

Kai-Fu Tang, Hao-Cheng Kao, Chun-Nan Chou, and Edward Y Chang. 2016. Inquire and diagnose: Neural symptom checking ensemble using deep reinforcement learning. In In Proceedings of NIPS Workshop on Deep Reinforcement Learning.

Augustin Toma, Patrick R Lawler, Jimmy Ba, Rahul G Krishnan, Barry B Rubin, and Bo Wang. 2023. Clinical camel: An open-source expert-level medical language model with dialogue-based knowledge encoding. CoRR.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.

Haochun Wang, Chi Liu, Nuwa Xi, Zewen Qiang, Sendong Zhao, Bing Qin, and Ting Liu. 2023a. Hutato: Tuning llama model with chinese medical knowledge. arXiv preprint arXiv:2304.06975.

Zifeng Wang, Zichen Wang, Balasubramaniam Srinivasan, Vassilis N Ioannidis, Huzeifa Rangwala, and Rishita Anubhai. 2023b. Biobridge: Bridging biomedical foundation models via knowledge graph. arXiv preprint arXiv:2310.03320.

Tianyu Wu, Shizhu He, Jingping Liu, Siqi Sun, Kang Liu, Qing-Long Han, and Yang Tang. 2023. A brief overview of chatgpt: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica, 10(5):1122–1136.

Honglin Xiong, Sheng Wang, Yitao Zhu, Zihao Zhao, Yuxiao Liu, Linlin Huang, Qian Wang, and Dinggang Shen. 2023. Doctorglm: Fine-tuning your chinese doctor is not a herculean task. arXiv preprint arXiv:2304.01097.

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024a. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115.

Songhua Yang, Hanjie Zhao, Senbin Zhu, Guangyu Zhou, Hongfei Xu, Yuxiang Jia, and Hongying Zan. 2024b. Zhongjing: Enhancing the chinese medical capabilities of large language model through expert feedback and real-world multi-turn dialogue. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19368–19376.

Hao-Ren Yao, Nairen Cao, Katina Russell, Der-Chen Chang, Ophir Frieder, and Jeremy T Fineman. 2024. Self-supervised representation learning on electronic health records with graph kernel infomax. ACM Transactions on Computing for Healthcare, 5(2):1–28.

Muchao Ye, Suhan Cui, Yaqing Wang, Junyu Luo, Cao Xiao, and Fenglong Ma. 2021. Medretriever: Target-driven interpretable health risk prediction via retrieving unstructured medical text. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 2414–2423.

Jinsung Yoon, James Jordon, and Mihaela Schaar. 2019. Asac: Active sensing using actor-critic models. In Machine Learning for Healthcare Conference, pages 451–473. PMLR.

Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223.

Hongjian Zhou, Fenglin Liu, Boyang Gu, Xinyu Zou, Jinfa Huang, Jinge Wu, Yiru Li, Sam S Chen, Peilin Zhou, Junling Liu, et al. 2023. A survey of large language models in medicine: Progress, application, and challenge. arXiv preprint arXiv:2311.05112.
A Appendix

<table>
  <tr>
    <th>Model</th>
    <th colspan="3">NHANES</th>
  </tr>
  <tr>
    <th></th>
    <th>Recall@1</th>
    <th>F1</th>
    <th>MRR</th>
  </tr>
  <tr>
    <td>Mistralv0.3-7B</td>
    <td>28.60</td>
    <td>40.48</td>
    <td>37.41</td>
  </tr>
  <tr>
    <td>Llama3.1-8B</td>
    <td>29.70</td>
    <td>45.69</td>
    <td>47.64</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td>28.97</td>
    <td>44.83</td>
    <td>44.38</td>
  </tr>
  <tr>
    <td>BioMistral</td>
    <td>36.19</td>
    <td>48.25</td>
    <td>43.13</td>
  </tr>
  <tr>
    <td>ChatDoctor</td>
    <td>39.83</td>
    <td>54.29</td>
    <td>46.72</td>
  </tr>
  <tr>
    <td>Meditron-7B</td>
    <td>36.82</td>
    <td>52.47</td>
    <td>47.77</td>
  </tr>
  <tr>
    <td>Meditron-3-8B</td>
    <td>37.69</td>
    <td>55.34</td>
    <td>49.78</td>
  </tr>
  <tr>
    <td>Mistralv0.3-7B<sub>mlp</sub></td>
    <td>91.98</td>
    <td>92.47</td>
    <td>97.99</td>
  </tr>
  <tr>
    <td>Llama3.1 8B<sub>mlp</sub></td>
    <td>92.15</td>
    <td>92.66</td>
    <td>98.06</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B<sub>mlp</sub></td>
    <td>92.07</td>
    <td>92.45</td>
    <td>98.02</td>
  </tr>
  <tr>
    <td>DiaLLMs</td>
    <td><b>95.94</b></td>
    <td><b>96.07</b></td>
    <td><b>98.97</b></td>
  </tr>
</table>

Table 4: Performance Comparison of DiaLLM and Baselines on NHANES Dataset.

A.1 Methodology

A.1.1 Illustration of EHR data.
We summarize main elements and corresponding examples in EHR data in Table 5. EHR data is heterogeneous and can be represented as a sequence of tables.

A.1.2 Dialogue Data Construction from EHR
The EHR transformation comprises two components (1) Clinical Code Translation and (2) Clinical Test Interpretation. Examples of our constructed Clinical Code Translation data is shown in Table 6 and examples of Clinical Test Interpretation data is shown in 7, respectively.

A.2 Experiments

A.2.1 Statistical Analysis of Datasets.
We exclude clinical tests with a frequency of less than 10 across the entire dataset. Additionally, we filter out tests for which no test results are available for each patient. Statistical analysis of datasets is shown in Table 8.

<table>
  <tr>
    <th></th>
    <th>TriNetX- Metabolic</th>
    <th>TriNetX- Respiratory</th>
    <th>TriNetX- Circulatory</th>
    <th>NHANES</th>
  </tr>
  <tr>
    <td>#Samples</td>
    <td>15844</td>
    <td>10068</td>
    <td>6747</td>
    <td>8897</td>
  </tr>
  <tr>
    <td>#Diagnosis</td>
    <td>49</td>
    <td>47</td>
    <td>49</td>
    <td>4</td>
  </tr>
  <tr>
    <td>#Clinical Test</td>
    <td>272</td>
    <td>301</td>
    <td>217</td>
    <td>45</td>
  </tr>
</table>

Table 8: Statistical Analysis of Datasets.

![Bar chart comparing performance metrics for DiaLLM and baselines](page_1012_120_377_246.png)

Figure 5: Ablation Study on NHANES Dataset.

A.2.2 Evaluation Metrics
The Mean Reciprocal Rank (MRR) is a metric used to evaluate systems that return a ranked list of answers to queries, focusing on the position of the first relevant answer. A higher MRR indicates that relevant items tend to appear higher in the ranked list of results. It is defined as the average of the reciprocal ranks of the first relevant answer for a set of queries. Formally, MRR is expressed as:

\[
\text{MRR} = \frac{1}{|Q|} \sum_{i=1}^{|Q|} \frac{1}{\text{rank}_i}
\]

(4)

A.2.3 Main Results on NHANES Dataset
Experimental results on the NHANES dataset is shown in Table 4. It can be observed that: (1) Medical-specialized models consistently outperform general foundation models; (2) When adapting foundation models with an MLP classifier trained on extracted embeddings, performance improves drastically; (3) DiaLLM significantly outperforms all baselines, demonstrating its superior ability in diagnosis prediction. These results highlight the effectiveness of DiaLLM’s integration of structured EHR data and reinforcement learning framework, enabling more accurate and clinically relevant predictions compared to both zero-shot LLMs and embedding-based MLP classifiers.

A.2.4 Ablation Study on NHANES Dataset
Experimental results on the NHANES dataset is shown in Figure 5. The NHANES dataset presents a simpler task compared to TriNetX data. The ablation study highlights the critical role of clinically grounded data transformation and rewarding modeling for improving diagnosis performance.

A.2.5 Case Study
The case study example is presented in Table 9.
<table>
  <tr>
    <th>EHR Data</th>
    <th>Description</th>
    <th>Event</th>
    <th>Visit Time</th>
  </tr>
  <tr>
    <td>Demographics</td>
    <td>General characteristics of patients</td>
    <td>Age, gender, ethnicity/race</td>
    <td></td>
  </tr>
  <tr>
    <td>Symptom</td>
    <td>Patient symptoms are cataloged in the ICD system under codes R00-R99.</td>
    <td>ICD-10-CM R10.12: Left upper quadrant pain</td>
    <td>2019-02-15</td>
  </tr>
  <tr>
    <td>Vital Signs</td>
    <td>Medical signs indicating the status of the body’s vital functions</td>
    <td>LOINC 8462-4 82 mm[Hg]<br>LOINC 29463-7 84 [lb_av]<br>LOINC 8480-6 124 mm[Hg]</td>
    <td>2019-02-15<br>2019-02-19<br>2019-02-22</td>
  </tr>
  <tr>
    <td>Laboratory Results</td>
    <td>Medical examination results, generate are organized in a code format</td>
    <td>LOINC 73578-7 8 mmol/L,<br>LOINC 6768-6 48 U/L,<br>LOINC 19261-7 Negative</td>
    <td>2019-02-15<br>2019-02-19<br>2019-02-22</td>
  </tr>
  <tr>
    <td>Diagnostic</td>
    <td>Codes representing diseases and related health problems (e.g., ICD-9 and ICD-10)</td>
    <td>ICD-10-CM Q65: Congenital deformities of hip,<br>ICD-10-CM M25.552: Pain in left hip,<br>ICD-10-CM R06.02: Shortness of breath</td>
    <td>2019-02-22</td>
  </tr>
</table>

Table 5: Description of key components and corresponding examples in EHR data.

<table>
  <tr>
    <th>LOINC Code</th>
    <th>Description</th>
  </tr>
  <tr>
    <td>100091-8</td>
    <td>Trypanosoma cruzi Ab [Units/volume] in Serum by Immunoassay</td>
  </tr>
  <tr>
    <td>100092-6</td>
    <td>Trypanosoma cruzi Ab bands panel - Serum by Immunoblot</td>
  </tr>
  <tr>
    <td>100093-4</td>
    <td>Trypanosoma cruzi 15-16kD IgG Ab [Presence] in Serum by Immunoblot</td>
  </tr>
  <tr>
    <td>100094-2</td>
    <td>Trypanosoma cruzi 21-22kD IgG Ab [Presence] in Serum by Immunoblot</td>
  </tr>
  <tr>
    <td>100095-9</td>
    <td>Trypanosoma cruzi 27-28kD IgG Ab [Presence] in Serum by Immunoblot</td>
  </tr>
  <tr>
    <td>100096-7</td>
    <td>Trypanosoma cruzi 42kD IgG Ab [Presence] in Serum by Immunoblot</td>
  </tr>
  <tr>
    <td>100097-5</td>
    <td>Trypanosoma cruzi 45-47kD IgG Ab [Presence] in Serum by Immunoblot</td>
  </tr>
  <tr>
    <td>100098-3</td>
    <td>Trypanosoma cruzi 120-200kD IgG Ab [Presence] in Serum by Immunoblot</td>
  </tr>
  <tr>
    <td>100099-1</td>
    <td>Trypanosoma cruzi 160kD IgG Ab [Presence] in Serum by Immunoblot</td>
  </tr>
  <tr>
    <td>1001-7</td>
    <td>DBG Ab [Presence] in Serum or Plasma from Donor</td>
  </tr>
  <tr>
    <td>10010-7</td>
    <td>R’ wave amplitude in lead AVF</td>
  </tr>
  <tr>
    <td>100100-7</td>
    <td>Fasciola sp IgG Ab [Presence] in Serum by Immunoassay</td>
  </tr>
  <tr>
    <td>100101-5</td>
    <td>Fasciola sp 8-9kD IgG Ab [Presence] in Serum by Immunoblot</td>
  </tr>
  <tr>
    <td>100102-3</td>
    <td>Fasciola sp 27-28kD IgG Ab [Presence] in Serum by Immunoblot</td>
  </tr>
</table>

Table 6: The Mapping between LOINC Codes and their Corresponding Clinical Descriptions.

<table>
  <tr>
    <th>Lab Code</th>
    <th>Test Name</th>
    <th>Age Range</th>
    <th>Gender</th>
    <th>Reference Range</th>
    <th>Unit</th>
    <th>Critical Value</th>
  </tr>
  <tr>
    <td>2823-3</td>
    <td>Potassium Serum</td>
    <td>1–18 years</td>
    <td>Any</td>
    <td>3.4–4.7</td>
    <td>mEq/L</td>
    <td>-</td>
  </tr>
  <tr>
    <td>2823-3</td>
    <td>Potassium Serum</td>
    <td>Any ≥18 years</td>
    <td>Any</td>
    <td>3.5–5.2</td>
    <td>mEq/L</td>
    <td>-</td>
  </tr>
  <tr>
    <td>17861-6</td>
    <td>Total Calcium</td>
    <td><1 year</td>
    <td>Any</td>
    <td>8.7–11.0</td>
    <td>mg/dL</td>
    <td>-</td>
  </tr>
  <tr>
    <td>17861-6</td>
    <td>Total Calcium</td>
    <td>1–17 years</td>
    <td>Any</td>
    <td>9.3–10.6</td>
    <td>mg/dL</td>
    <td>-</td>
  </tr>
  <tr>
    <td>17861-6</td>
    <td>Total Calcium</td>
    <td>18–59 years</td>
    <td>Any</td>
    <td>8.6–10.0</td>
    <td>mg/dL</td>
    <td>-</td>
  </tr>
  <tr>
    <td>17861-6</td>
    <td>Total Calcium</td>
    <td>≥60 years</td>
    <td>Any</td>
    <td>8.8–10.2</td>
    <td>mg/dL</td>
    <td>-</td>
  </tr>
  <tr>
    <td>33914-3</td>
    <td>eGFR (Estimated Glomerular Filtration Rate)</td>
    <td>Any</td>
    <td>Any</td>
    <td>>90</td>
    <td>mL/min/1.73m<sup>2</sup></td>
    <td>Mildly low: 60–89</td>
  </tr>
  <tr>
    <td>33914-3</td>
    <td>eGFR (Estimated Glomerular Filtration Rate)</td>
    <td>Any</td>
    <td>Any</td>
    <td>None</td>
    <td>mL/min/1.73m<sup>2</sup></td>
    <td>Moderate to severe kidney disease: 30–59</td>
  </tr>
  <tr>
    <td>33914-3</td>
    <td>eGFR (Estimated Glomerular Filtration Rate)</td>
    <td>Any</td>
    <td>Any</td>
    <td>None</td>
    <td>mL/min/1.73m<sup>2</sup></td>
    <td>Kidney failure: <15</td>
  </tr>
</table>

Table 7: Examples of the Manually Constructed Clinical Test Interpretation Data.
A Sub-piece of Initial EHR Data:
Demographic Information: age: 30, gender: female, race: white
Symptom Code : R10.2, R10.32
Lab Test Code : 2028-9, 2075-0, 2951-2, 3040-3, 33037-3, 48642-3, 777-3, 1742-6, 2028-9, 2075-0
Lab Results Values : 31.0, 105.0, 83.0, 139.0,7.0, 99.0, 283.0, 27.0,
Vital Sign Code : 8302-2, 8310-5, 8462-4, 8480-6, 9279-1
Vital Sign Values : 62.0, 98.0, 70.7, 119.7, 98.7
Diagnosis Codes: E03.9, K80.62, K81.0, N80.9, K80.20, K80.10

Prompts:
Given the patient demographic information: (age: 30, gender: female, race: white).
Patient shows symptom of : (1) Pelvic and perineal pain; (2) Right lower quadrant pain.
These lab tests show normal results : (1) Carbon dioxide, totalin Serum or Plasma, (2) Chloridein Serum or Plasma, (3) Glucosein Serum or Plasma, (4) Sodiunin Serum or Plasma, (5) Lipasein Serum or Plasma, (6) Glomerular filtration rate/1.73 sq M predicted among non-blacksin Serum or Plasma by Creatinine-based formula (MDRD).
These lab tests show abnormal results : (1) Plateletsin Blood by Automated count.
These vital sign show normal Results : (1) Temperature, (2) Blood Pressure (Systolic), (3) Blood Pressure (Diastolic), (4) Respiratory Rate,
These vital sign show abnormal values : (1) Heart Rate
What are your recommended diagnosis? If you need further information, what are your recommended clinical test for further decision making?

DiaLLM
I recommend you to take (1) Alanine aminotransferase in Serum or Plasma; (2) Carbon dioxide in Serum or Plasma, (3) Chloride in Serum or Plasma.
Prompts (User):
My Alanine aminotransferase in Serum or Plasma results is 27.0 \( U/L \), Carbon dioxide in Serum or Plasma is 83.0 \( mEq/L \), Chloride in Serum or Plasma is 139.0 \( mEq/L \).
DiaLLM:
These lab test shows normal results: (1) Alanine aminotransferase in Serum or Plasma results;
These lab test shows abnormal results: (1) Carbon dioxide in Serum or Plasma, (2) Chloride in Serum or Plasma. I further recommend you to take (1) Glucose in Serum or Plasma; (2) Sodium in Serum or Plasma;
Prompts (User):
My Glucose in Serum or Plasma is 7.0 \( mmol/L \), (2) Sodium in Serum or Plasma 99.0 \( mmol/L \).
DiaLLM:
These lab test shows abnormal results: (1) Glucose in Serum or Plasma; (2) Sodium in Serum or Plasma. I recommend the following possible diagnosis: (1) disorders of thyroid gland, (2) disorders of gallbladder, biliary tract and pancreas, (3) Noninflammatory disorders of female genital tract.

BioMistral
I recommend the following possible diagnosis: (1) Chlamydia trachomatis infection, (2) Pelvic inflammatory disease, (3) Pelvic pain

Meditron-7B
I recommend the following possible diagnosis: (1) Pelvic inflammatory disease, (2) Acute appendicitis, (3) Ovarian torsion, (4) Ectop

Meditron-8B
I recommend the following possible diagnosis: (1) Pelvic inflammatory disease, (2) Chlamydia trachomatis infection, (3) Pelvic inflammatory disease with chlamydia trach.

Chatdoctor
I recommend the following possible diagnosis: (1) pelvic inflammatory disease, (2) tuberculosis, (4) acute pancreat.

Table 9: Sample Responses for Case Study.

Reviewer paper summaries: This paper presents DiaLLM, a clinical dialogue system that integrates electronic health record (EHR) data to enhance clinical test recommendations, result interpretation, and diagnostic prediction. Unlike existing medical LLMs, DiaLLM employs an EHR data transformation strategy to enable the model to comprehend heterogeneous medical data and utilizes reinforcement learning to optimize diagnostic decision-making.
Summary of strengths: 1.DiaLLM leverages EHR data transformation to utilize real electronic health records rather than relying solely on medical knowledge graphs or synthetic dialogues, making it more aligned with real-world medical scenarios.

2.The paper employs reinforcement learning (PPO) for automated evidence retrieval and diagnostic prediction, incorporating techniques such as rejection sampling to reduce redundant tests and confirmation rewards to enhance decision accuracy, thereby improving the model's practicality.
Summary of weaknesses: 1.The EHR transformation strategy requires manual construction, which limits its applicability across different diseases. Additionally, the quality of manual annotations is a key factor affecting performance.

2.Compared to Llama3.1-8B (MLP), DiaLLM has more trained parameters. As described in lines 411 to 413 of the paper, Llama3.1-8B (MLP) only trains a fully connected layer, making this baseline comparison unfair.

3.The reward mechanism relies on training with a specific dataset. I am somewhat concerned about whether the model can accurately reject samples when faced with data that differs significantly from the training set.
Comments, suggestions and typos: No Suggestions
Overall assessment scores: 3
Metareview score: 3
Metareview: This paper presents a novel clinical dialogue system, DiaLLM, which integrates heterogeneous EHR data into multi-step conversations for test recommendation, interpretation, and diagnosis.
Summary of reasons to publish: DiaLLM introduces a novel framework that integrates real-world EHR data into clinical dialogues, enabling test recommendation and diagnosis prediction in a clinically grounded, multi-step manner. The work bridges a gap between medical LLMs and real-world healthcare practice, with strong empirical results.