Listening to Patients: Detecting and Mitigating Patient Misreport in Medical Dialogue System
Authors: Lang Qin, YAO ZHANG, Hongru Liang, Adam Jatowt, Zhenglu Yang
Accepted at: acl25

Listening to Patients: Detecting and Mitigating Patient Misreport in Medical Dialogue System

Lang Qin1,2 Yao Zhang*2 Hongru Liang3 Adam Jatowt4 and Zhenglu Yang1
1Key Laboratory of DISSec, College of Computer Science, Nankai University, Tianjin, China
2School of Statistics and Data Science, AAIS, Nankai University, Tianjin, China
3College of Computer Science, Sichuan University, Chengdu, China
4Department of Computer Science & Digital Science Center at the University of Innsbruck, Austria.
qinlang14@mail.nankai.edu.cn, yaozhang@nankai.edu.cn, lianghongru@scu.edu.cn,
Adam.Jatowt@uibk.ac.at, yangzl@nankai.edu.cn

Abstract

Medical Dialogue Systems (MDSs) have emerged as promising tools for automated healthcare support through patient-agent interactions. Previous efforts typically relied on an idealized assumption — patients can accurately report symptoms aligned with their actual health conditions. However, in reality, patients often misreport their symptoms, due to cognitive limitations, emotional factors, etc. Overlooking patient misreports can significantly compromise the diagnostic accuracy of MDSs. To address this critical issue, we emphasize the importance of enabling MDSs to “listen to patients” by tackling two key challenges: how to detect misreport and mitigate misreport effectively. In this work, we propose PaMis, a novel framework that can detect patient misreports based on calculating the structural entropy of the dialogue entity graph, and mitigate them through generating controlled clarifying questions. Our experimental results demonstrate that PaMis effectively enhances MDSs reliability by effectively addressing patient misreports during the medical response generation process.

1 Introduction

Medical Dialogue Systems (MDSs) aim to provide automated healthcare support through natural language interactions between patients and system agents (Li et al., 2021; Liu et al., 2022b; Xu et al., 2024). In this process, taking Figure 1 as an example, when a patient reports his/her symptom, e.g., feels dizzy (P1), the agent will process this narrative, and inquire about more information related to symptoms, e.g., vomiting (A1) and cold (A2). In order to simulate the questioning strategies employed by human doctors, researchers have made substantial progress in medical response generation. Specifically, Lin et al. (2019) proposed a symptom graph sturture to capture symptom-related information, Liu et al. (2021) developed a heterogeneous graph to integrate dialogue context understanding with entity reasoning, and Xu et al. (2023) focused on modeling the transitions of the medical entities and the doctor’s dialogue acts.

Despite extensive research efforts, existing studies predominantly operate under an idealized assumption — patients can accurately report symptoms aligned with their actual health conditions. In reality, patients cannot always be that professional and often misreport their symptoms due to cognitive limitations or emotional factors, etc (Berkman et al., 2011; Prior et al., 2011). A piece of strong evidence is that such misreports exist in approxi-

<table>
  <tr>
    <th colspan="2">A patient with myocardial infarction</th>
  </tr>
  <tr>
    <td>P1: Doctor, I feel dizzy when I get up in the morning.</td>
    <td>医生，我早上起来就觉得很晕。</td>
  </tr>
  <tr>
    <td>A1: Have you experienced any vomiting?</td>
    <td>您有呕吐现象吗？</td>
  </tr>
  <tr>
    <td>P2: Yes, and I also experienced a rapid heartbeat.</td>
    <td>有，我还经历了一阵心跳加速。</td>
  </tr>
  <tr>
    <td>A2: Have you noticed chest pain or tightness?</td>
    <td>您有感到胸部或胸闷吗？</td>
  </tr>
  <tr>
    <td>P3: No, but my abdomen hurts a bit. It’s mostly on the right side.</td>
    <td>没有，但我的腹部有点疼。主要是在右侧。</td>
  </tr>
  <tr>
    <th colspan="2">Patient Misreport Occur</th>
  </tr>
  <tr>
    <td>If the agent ignores the misreport</td>
    <td>A3: It could be stomach flu. Do you have diarrhea? 可能是肠胃型感冒，您有腹泻吗？</td>
  </tr>
  <tr>
    <td>A human doctor</td>
    <td>D1: Heart discomfort may radiate to other areas. Do you experience any discomfort in your left back? 有时心脏不舒服会放射到其他地方，您左背部有不适感吗？</td>
  </tr>
</table>

Figure 1: Example of patient misreport in patient (P)-agent (A) conversations and a response generated by the human doctor (D). When a patient with myocardial infarction misreports angina as abdominal pain, the doctor remains vigilant and asks more detailed questions to discern the patient’s actual symptoms. However, the agent can easily be influenced by the patient’s misreport and arbitrarily shift the focus to stomach flu.
mately 15–20% of real-world cases (Fleischer et al., 2015; Merckelbach et al., 2019). Through a preliminary analysis, we find another evidence that patient misreports occurred in 16.9% of the dialogues from the public corpus MedDG (Liu et al., 2022c). Overlooking patient misreports can significantly compromise the diagnostic accuracy of MDSs. As illustrated in Figure 1, a patient experiencing a myocardial infarction might inaccurately report angina as abdomen pain (P3). If the agent ignores the patient’s misreport, it will arbitrarily diagnose that the patient may have a stomach flu (A3). In contrast, the human doctor would remain vigilant and ask for more details about the location of pain (D1) to discern the patient’s actual symptoms.

Therefore, listening to patients and moving beyond idealized assumptions are critical for establishing reliable MDSs. To address patient misreports, we propose two key challenges that need to be tackled:

• Misreport Detection: How to detect misreport from multi-turn medical dialogues with the complex patient narratives and strong dependence on medical knowledge.
• Misreport Mitigation: How to generate targeted clarifying questions to help patients accurately report their symptoms without breaking the dialogue flow.

To address the above challenges, we propose PaMis, a framework for detecting and mitigating Patient Misreport for medical response generation. To ground the dialogue with medical knowledge, we equip PaMis with external medical knowledge from CMeKG (Byambasuren et al., 2019), MedDG (Liu et al., 2022c), and KaMed (Li et al., 2021). Specifically, given dialogue context, the PaMis(1) constructs an entity graph that models the relations between medical entities in patient narratives, (2) detects patient misreport based on calculating the structural entropy of the dialogue entity graph, and (3) mitigates patient misreport by generating controlled clarifying questions based on the detected misreport information. We conduct a comprehensive evaluation of PaMis on two medical dialogue datasets, MedDG and KaMed, comparing both state-of-the-art large language models (LLMs) and fine-tuned models through LLM-based metrics, N-gram metrics, misreport-aware metrics, and human evaluation. The results demonstrate that PaMis achieves superior performance in medical response generation and can effectively detect and mitigate misreports during doctor-patient interactions. We believe that PaMis can effectively enhance MDSs in capturing patients’ actual symptoms and establishing reliable healthcare support.

In conclusion, the key contributions of this research are outlined below.
• We emphasize the existence of patient misreporting of symptoms, which can affect the ability of MDSs to provide reliable healthcare support.
• We propose PaMis for detecting patient misreports based on calculating the structural entropy of the dialogue entity graph, and mitigating patient misreports by generating controlled clarifying questions.
• The comprehensive evaluation demonstrates that PaMis achieves superior performance in generating medical response and enhances MDSs to address patient misreports.

2 Related Work

MDSs aim to collect symptoms and automate diagnosis through conversation. MDSs first perform symptom collection during the dialogue with the patient (Liu et al., 2022c). Lin et al. (2019) utilized a symptom graph with global attention, and Xu et al. (2023) combined medical entity flows with dialogue action flows. The MDSs then perform an automatic Diagnosis based on the collected symptoms and develop agents that mimic physicians’ diagnostic behavior. Wei et al. (2018) applied DQN to extract symptoms, while Xu et al. (2019b) used knowledge graphs for automated diagnosis. Prior studies have addressed challenges in vague patient statements (Zhao et al., 2022; Xu et al., 2023), limited medical data (Tang et al., 2023; Lin et al., 2021; Hou et al., 2023), and physician behavior simulation (Li et al., 2021; Liu et al., 2022b). However, they have not adequately considered the issue of patient misreporting, and recent research continues to assume that patients can accurately report symptoms aligned with their actual health conditions (Li et al., 2024). Our study innovatively addresses patient misreporting by introducing a medical dialogue management framework that detects and mitigates misreports.

To address vague or incorrect user inputs, task-oriented dialogue systems often use proactive questioning to maximize information gain. These systems construct clarifying questions to maximize the expected information gain based on the princi-
Figure 2: An illustration of PaMis, using the dialogue in Figure 1 as an example. PaMis first constructs the entity graph, and then detects and mitigates the patient misreport based on the entity graph.

ple that “a good question is one whose expected answer is the most useful (Rao, 2017).” Aliannejadi et al. (2019) retrieved context-aware questions for information-seeking tasks; Xu et al. (2019a) resolved knowledge ambiguities through judge-then-generate methods; Feng et al. (2023) and Zhao and Dou (2024) handled user-specific needs via multi-turn dialogues, and Oshima et al. (2023) highlighted error detection in visual QA systems. Building on these, we propose PaMis, a framework for generating medical responses and enhancing MDSs to address patient misreports.

3 Methodology

3.1 Overview

We formulate the doctor-patient conversation as \( \mathcal{C} = \{(\mathcal{P}_i, \mathcal{D}_i)\}_{i=1}^T \), where \( \mathcal{P}_i \) denotes the patient’s statement and \( \mathcal{D}_i \) represents the doctor’s response. The primary objective of MDS is to develop a physician agent that generates an appropriate response \( \mathcal{D}_i \) based on the dialogue history \( \mathcal{C}_{1:i-1} \) and the patient’s current statement \( \mathcal{P}_i \).

The entities \( \{e_i\} \) appearing in the conversation \( \mathcal{C} \) can be organized into a graph \( G_e \) to represent the patient’s health condition and the doctor’s logic of inquiry. In addition to generating responses \( \mathcal{D}_i \), a MDS that addresses patient misreports must detect misreported entities \( e_m \) and ask clarifying questions to reduce inaccuracies. The system leverages both the dialogue content and the entity graph to maximize the probability \( P\left( \mathcal{D}_i \mid \mathcal{C}_{1:i-1}, \mathcal{P}_i, G_e, e_m \right) \).

3.2 Entity Graph Construction

We introduce a modeling approach that integrates a static knowledge graph \( G \) with a dynamic dialogue entity graph \( G_e \subseteq G \) to improve detection effectiveness. The graph \( G \) is constructed from the corpus, with nodes representing entities and edge weights representing co-occurrence relationships. Entity extraction follows previous works (Li et al., 2021; Xu et al., 2023), which extract entities from the corpus through text matching, referencing predefined medical entities from CMeKG (Byambasuren et al., 2019), MedDG (Liu et al., 2022c), and KaMed (Li et al., 2021) We use LLM to generate multiple synonymous expressions for entities, enhancing extraction accuracy and robustness. The likelihood of mentioning entity \( e_t \) at turn \( t \) is modeled as the conditional probability given the entities \( \{e_1, e_2, \ldots, e_{t-1}\} \) in the dialogue history:

\[
P\left(e_t \mid \mathcal{C}_{1:t-1}\right) \propto P\left(e_t \mid e_1, e_2, \ldots, e_{t-1}\right).
\]

Thus, we use the co-occurrence frequency in the corpus as the directed\footnote{Based on the sequence of entity occurrences in the corpus, it reflects the logic of doctors’ inquiries.} weight \( w_{ij} \) from \( e_i \) to \( e_j \):

\[
w_{ij} = \frac{\mathrm{freq}(e_i, e_j)}{\mathrm{freq}(e_i)} \propto P\left(e_j \mid e_i\right).
\]

The edge weights in graph \( G \) serve as the basis for ranking relevant entities during subsequent retrieval for response generation.

The dialogue history references or denies medical entities, which are extracted through medical slot-filling (Hu et al., 2023). The extracted entities and relationships \( r = (e_i, e_j, w_{ij}) \) must align with established background knowledge \( G \), rather than
transient associations. Entities identified in each utterance form a separate dialogue entity graph \( G_e \) for that utterance. We track changes in these graphs to generate a sequence of dynamic graphs, \( \{G_{e_1}, G_{e_2}, \ldots\} \). Misreport detection and classification are then performed using graph entropy theory.

3.3 Misreport Detection

Patient misreporting occurs when discrepancies arise between the health conditions described in the dialogue and the patient’s actual health conditions. Health conditions often follow inherent medical co-occurrence relationships (Liu et al., 2022a; Bhoi et al., 2023), which can be disrupted by misreports, leading to anomalies in the dialogue’s entity graph. We analyze misreport patterns using real-world data (Liu et al., 2022c; Li et al., 2021) and apply graph entropy to detect these misreports.

(1) Misreport Feature Classification

Patient misreports often lack clear linguistic indicators, but they manifest as anomalies on the entity graph due to inconsistencies with established medical knowledge. Based on observed patient behavior, we categorize misreports into three types.

• Introducing isolated entities (Figure 2(b)-(Is)). This type is typically initiated by patients because of anxiety or vague descriptions.
• Denying crucial entities (Figure 2(b)-(De)). This type involves deleting entities and transferring graphs to disconnect components, which occurs when patients deny entities in doctors’ inquiries.
• Presenting self-contradictions (Figure 2(b)-(Co)). This type involves entities appearing and disappearing without disrupting the graph’s structure, typically reflecting patients misreporting the duration of symptoms or medical history2.

(2) Graph Entropy-based Detection

Graph entropy characterizes the structural information of graphs. Given that misreporting disrupts graph structure, we use graph entropy for detection. Specifically, to address the potential disconnected features of entity graphs, we apply one-dimensional structural entropy (Li and Pan, 2016) to detect misreporting phenomena.

The definition of graph entropy aligns with Shannon’s information entropy (Shannon, 1953), aiming to represent structure information and complexity through the degree distribution of nodes. In the following formula, \( d_i \) denotes the degree of node \( i \), and \( \mathrm{Vol}(G) \) is the degree sum of \( G \):

\[
H(G) = - \sum_{i=1}^n \frac{d_i}{\mathrm{Vol}(G)} \log_2 \frac{d_i}{\mathrm{Vol}(G)}. \tag{3}
\]

For the entity graph \( G_e \), the one-dimensional structural entropy is calculated as the weighted average of the entropy of each connected component, defined as follows:

\[
H^1(G) = \frac{1}{\mathrm{Vol}(G)} \sum_{j=1}^L \mathrm{Vol}(G_j) \cdot H^1(G_j), \tag{4}
\]

where \( G_j \) represents a connected subgraph of \( G \), \( L \) represents the number of connected components, and the entropy is considered as 0 if has no edges.

An effective inquiry process should enhance the co-occurrence relationships between entities, thereby expanding information pathways and increasing graph entropy. As illustrated in Figure 2, misreports can disrupt this pattern. A single calculation of graph entropy can be used to detect and classify such misreports:

• (Is) occurs when the number of nodes increases while the graph entropy remains unchanged because an isolated node has zero entropy.
• (De) and (Co) reduce information pathways in the graph, leading to a decrease in entropy. When nodes are lost, the entropy lower bound for contradictions is higher than the upper bound for denials. This boundary helps detect and classify misreports, with the formulations provided below and detailed proofs available in Appendix A:

\[
H(G_{Co}) \geq - \sum_{i=1}^n \frac{d_i - 1}{2(n-1)} \log_2 \frac{d_i - 1}{2(n-1)}, \tag{5}
\]

\[
H(G_{De}) \leq - \left( \sum_{i=1}^{n-2} F(d_i) + F(d_{n-1} - 1) \right), \tag{6}
\]

\[
F(d) = \frac{d}{\mathrm{Vol}(G) - 4} \log_2 \frac{d}{\mathrm{Vol}(G) - 4}. \tag{7}
\]

In summary, the sequence of graph entropy values enables the straightforward detection and classification of misreports.

2Doctors may focus on recent symptoms for acute conditions or long-term medication history for chronic illnesses. Patients might provide contradictory information, such as first claiming “never had a stomach problem” and later mentioning “occasional stomach pain.”
3.4 Misreport Mitigation

The mitigation module generates clarifying questions based on detected patient misreports to obtain accurate information about patients’ health conditions and mitigate misreporting.

(1) Response Generation

Misreport mitigation can be perceived as an instance of response generation as it involves the agent asking questions. Therefore, we first introduce the standard process of PaMis in utilizing medical knowledge to generate responses. As illustrated in Figure 2, we establish a connection between the medical knowledge graph and real physician responses based on entities. Next, we identify the most valuable responses to serve as guiding information. A set of real responses \( \{S\} \) containing the entity \( e \) can be extracted from the corpus. We calculate the cosine similarity of word vectors \( v \) for the sentences in \( \{S\} \) and select the top-k sentences that exhibit the highest average similarity to others as the knowledge \( \{S_e\} \) related to entity \( e \):

\[
\text{avg\_sim}(e) = \frac{1}{|S|} \sum_{s_i, s_j \in S, i \neq j} \text{sim}\left(s_i, s_j\right),
\]
\[
\text{sim}\left(s_i, s_j\right) = \frac{v_i \cdot v_j}{||v_i|| \cdot ||v_j||}.
\]

The agent leverages the dialogue context and the dialogue entity graph to retrieve knowledge pairs \( \{(e, S_e)\} \) from the one-hop neighbors of existing nodes. The candidate ranking process uses the weights described in Section 3.2, where the edge weights to adjacent nodes serve as scores for neighboring entities. This step does not mean performing the entity prediction task in MDS but provides a preliminary enhancement to the proposed framework. This explicit approach gives the framework flexibility to incorporate generative models, including large language models.

Overall, the aim of response generation is to maximize the conditional probability \( P\left(D_t \mid C_{t:i-1}, P_t, \{(e, S_e)\}\right) \).

(2) Clarifying Process

Based on the approach described above, the clarifying process is triggered when misreports are detected. This process involves re-retrieving entity-response pairs \( \{(e, S_e)\} \) related to the misreported entity \( e_m \) as guiding information. The implications vary depending on the scenario:

• (\( Is \)) PaMis retrieves bridging nodes from the static graph \( G \) to complete the dialogue entity graph, aiming to generate the next inquiry that restores the graph to a connected state.
• (\( De \)) PaMis uses related knowledge of denied entities to generate clarifying questions, reconfirming the existence or absence of the relevant health condition.
• (\( Co \)) PaMis selects broader attribute entities (such as duration, location, etc.) to generate inquiries, obtaining supplementary information to assist in judgment.

In summary, related knowledge is integrated to augment clarifying question generations by retrieving information related to misreports.

<table>
  <tr>
    <th>Dataset</th>
    <th>MedDG</th>
    <th>KaMed</th>
  </tr>
  <tr>
    <th>Dialogues</th>
    <td>17,864</td>
    <td>63,754</td>
  </tr>
  <tr>
    <th>Avg. # of utterances</th>
    <td>19.85</td>
    <td>23.25</td>
  </tr>
  <tr>
    <th>Avg. # of entities</th>
    <td>12.11</td>
    <td>14.94</td>
  </tr>
</table>

Table 1: The statistics of datasets.

4 Experiments

We conduct experiments focusing on two core research questions:
• RQ1: In line with the unified setup used in related work. Does PaMis outperform current methods in overall performance for medical response generation?
• RQ2: Can PaMis effectively detect and mitigate misreports in doctor-patient interactions within an interactive experimental setting?

In addition, we conduct further analysis to comprehensively evaluate PaMis, including: 1) conducting the human evaluation of its performance, 2) conducting an ablation study to analyze component contributions, 3) evaluating its effectiveness across different misreporting types, and 4) performing a comprehensive case analysis.

4.1 Settings

Datasets We conducted experiments using the MedDG dataset (Liu et al., 2022c) and the KaMed dataset (Li et al., 2021). Medical entities mentioned in the dialogues were annotated by domain experts. Detailed dataset statistics are provided in Table 1. To enrich the medical background knowledge, we integrated multiple knowledge graphs from the datasets, focusing on the co-occurrence relationships between medical entities.

Baselines We selected two categories of base-
<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="5">MedDG</th>
    <th colspan="5">KaMed</th>
  </tr>
  <tr>
    <th>P</th><th>A</th><th>H</th><th>LQ</th><th>DV</th><th>Overall</th>
    <th>P</th><th>A</th><th>H</th><th>LQ</th><th>DV</th><th>Overall</th>
  </tr>
  <tr>
    <td>HuatuoGPT2</td>
    <td>3.46 ±0.03</td><td>3.99 ±0.02</td><td>3.32 ±0.03</td><td>4.27 ±0.07</td><td>3.08 ±0.03</td><td>18.11 ±0.08</td>
    <td>3.35 ±0.05</td><td>3.95 ±0.05</td><td>3.26 ±0.04</td><td>4.18 ±0.03</td><td>2.96 ±0.04</td><td>17.70 ±0.19</td>
  </tr>
  <tr>
    <td>DISC-MedLLM</td>
    <td>3.20 ±0.05</td><td>3.95 ±0.05</td><td>3.19 ±0.06</td><td>4.16 ±0.09</td><td>2.87 ±0.07</td><td>17.37 ±0.28</td>
    <td>3.12 ±0.01</td><td>3.93 ±0.02</td><td>3.10 ±0.01</td><td>4.03 ±0.05</td><td>2.75 ±0.04</td><td>16.93 ±0.08</td>
  </tr>
  <tr>
    <td>ChatGPT</td>
    <td>3.70 ±0.05</td><td>4.15 ±0.03</td><td>3.61 ±0.05</td><td>4.23 ±0.01</td><td>3.43 ±0.17</td><td>19.13 ±0.14</td>
    <td>3.72 ±0.03</td><td>4.18 ±0.04</td><td>3.61 ±0.05</td><td>4.19 ±0.00</td><td>3.44 ±0.04</td><td>19.14 ±0.14</td>
  </tr>
  <tr>
    <td>ChatGPT (w/ Gold Know.)</td>
    <td>3.90 ±0.02</td><td>4.23 ±0.02</td><td>3.75 ±0.06</td><td>4.28 ±0.02</td><td>3.70 ±0.04</td><td>19.87 ±0.12</td>
    <td>3.89 ±0.06</td><td>4.23 ±0.07</td><td>3.77 ±0.06</td><td>4.24 ±0.03</td><td>3.65 ±0.05</td><td>19.78 ±0.27</td>
  </tr>
  <tr>
    <td>GPT-4</td>
    <td>3.93 ±0.03</td><td>4.21 ±0.01</td><td>3.82 ±0.01</td><td>4.36 ±0.03</td><td>3.61 ±0.02</td><td>19.93 ±0.04</td>
    <td>3.98 ±0.03</td><td>4.27 ±0.03</td><td>3.89 ±0.05</td><td>4.37 ±0.02</td><td>3.65 ±0.04</td><td>20.16 ±0.14</td>
  </tr>
  <tr>
    <td>GPT-4 (w/ Gold Know.)</td>
    <td>3.96 ±0.02</td><td>4.23 ±0.02</td><td>3.87 ±0.03</td><td>4.37 ±0.02</td><td>3.67 ±0.03</td><td>20.10 ±0.09</td>
    <td>4.03 ±0.04</td><td>4.29 ±0.02</td><td>3.96 ±0.04</td><td>4.38 ±0.02</td><td>3.72 ±0.05</td><td>20.38 ±0.14</td>
  </tr>
  <tr>
    <td>PaMis</td>
    <td><b>4.00†</b> ±0.01</td><td><b>4.26†</b> ±0.02</td><td><b>3.93†</b> ±0.05</td><td><b>4.36†</b> ±0.01</td><td><b>3.78†</b> ±0.02</td><td><b>20.33†</b> ±0.03</td>
    <td><b>4.05†</b> ±0.02</td><td><b>4.32†</b> ±0.01</td><td><b>4.00†</b> ±0.02</td><td><b>4.37†</b> ±0.03</td><td><b>3.84†</b> ±0.01</td><td><b>20.58†</b> ±0.06</td>
  </tr>
</table>

Table 2: Evaluation results of responses generated from LLMs and PaMis. “Gold” indicates that the entities in the actual doctor’s responses are included in the input. † denotes statistically significant differences (p < 0.05).

<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="3">MedDG</th>
    <th colspan="3">KaMed</th>
  </tr>
  <tr>
    <th>P</th><th>FT</th><th>FI</th>
    <th>P</th><th>FT</th><th>FI</th>
  </tr>
  <tr>
    <td>GPT-3.5</td><td>48.73</td><td>50.14</td><td>48.36</td><td>25.55</td><td>28.15</td><td>24.49</td>
  </tr>
  <tr>
    <td>GPT-3.5 (few)</td><td>52.62</td><td>63.63</td><td>55.22</td><td>27.18</td><td>28.84</td><td>25.51</td>
  </tr>
  <tr>
    <td>GPT-4</td><td>53.33</td><td>72.95</td><td>59.28</td><td>28.03</td><td>34.09</td><td>27.82</td>
  </tr>
  <tr>
    <td>GPT-4 (few)</td><td>54.36</td><td>72.32</td><td>59.40</td><td>27.50</td><td>36.41</td><td>28.61</td>
  </tr>
  <tr>
    <td>PaMis</td><td><b>57.35</b></td><td><b>65.89</b></td><td><b>61.44</b></td><td><b>29.03</b></td><td><b>35.10</b></td><td><b>29.38</b></td>
  </tr>
</table>

Table 3: Evaluation results on misreport detection.

lines: LLMs with demonstrated medical capabilities and fine-tuned models. Detailed information about the baselines is provided in Appendix B.

(1) LLMs: DISC-MedLLM (Bao et al., 2023), HuatuoGPT2 (Chen et al., 2024), GPT-3.5 (Ouyang et al., 2022), and GPT-4 (OpenAI, 2024).

(2) Fine-tuned models: GPT-2 (Radford et al., 2019), VRBot (Li et al., 2021), DFMED (Xu et al., 2023), and EMULATION (Xu et al., 2024).

Metrics We employed three categories of evaluation metrics:

(1) LLM-based: We follow previous studies (Bao et al., 2023; Xu et al., 2024) that employ LLMs (e.g., GPT-4) as evaluators to assess MDS on the dimensions of Proactivity, Accuracy, Helpfulness, and Linguistic Quality. Additionally, we introduced a dimension called Diagnostic Vigilance3 to measure the model’s ability to detect and respond to misreporting phenomena.

(2) Misreport-aware metrics: To evaluate the effectiveness of misreport mitigation, we introduce two metrics: \( \Delta GE \), which quantifies the average change in graph entropy before and after interactions, and MR, the mitigation rate of misreports.

(3) N-gram-based: We follow prior works (Liu et al., 2022c; Xu et al., 2023) that utilize BLEU (Pa-

3DV: The doctor skillfully identifies inconsistencies or errors in the patient’s statement and asks clarifying questions to ensure an accurate diagnosis.

Figure 3: Evaluation results of interactive experiment under two misreport-aware metrics: \( \Delta GE \) and MR.

pineni et al., 2002), ROUGE (Lin, 2004), Distinct (Li et al., 2016), and Entity-F1 (Liu et al., 2022c) for the response generation task.

Implementation Details Building on previous studies (Chen et al., 2023; Bao et al., 2023), we construct prompts for generating responses and evaluations in comparison with LLMs. To compare with fine-tuned models, we employ a backbone model4 of similar scale to the baselines. The code will be released upon acceptance of this paper. Details of the prompts and additional implementation information are provided in Appendix C and D.

4.2 Overall Performance

RQ1: Performance w.r.t. Medical response generation In line with the unified setup used in related work, we conduct experiments on the MedDG and KaMed datasets with the response generation

4https://huggingface.co/fnlp/bart-base-chinese
Figure 4: The human evaluation results of PaMis vs. GPT-4 (w/ Gold Know.) on two datasets.

<table>
  <tr>
    <th>Method</th>
    <th>B-1</th>
    <th>B-4</th>
    <th>R-1</th>
    <th>R-2</th>
    <th>D-1</th>
    <th>D-2</th>
    <th>E-F</th>
  </tr>
  <tr>
    <th colspan="8">MedDG</th>
  </tr>
  <tr>
    <td>PaMis</td>
    <td><b>44.28</b></td>
    <td><b>24.88</b></td>
    <td><b>28.12</b></td>
    <td><b>13.80</b></td>
    <td>1.23</td>
    <td>11.58</td>
    <td><b>25.13</b></td>
  </tr>
  <tr>
    <td>w/o Detection</td>
    <td>37.06</td>
    <td>20.28</td>
    <td>27.05</td>
    <td>12.55</td>
    <td>1.36</td>
    <td>13.28</td>
    <td>24.02</td>
  </tr>
  <tr>
    <td>w/o Mitigation</td>
    <td>42.17</td>
    <td>23.06</td>
    <td>28.05</td>
    <td>13.55</td>
    <td>1.26</td>
    <td>12.34</td>
    <td>24.61</td>
  </tr>
  <tr>
    <td>w/o D&M</td>
    <td>35.83</td>
    <td>19.57</td>
    <td>26.34</td>
    <td>11.83</td>
    <td>1.29</td>
    <td>12.48</td>
    <td>24.09</td>
  </tr>
  <tr>
    <th colspan="8">KaMed</th>
  </tr>
  <tr>
    <td>PaMis</td>
    <td><b>41.02</b></td>
    <td><b>21.30</b></td>
    <td><b>28.42</b></td>
    <td>12.01</td>
    <td>1.30</td>
    <td>11.39</td>
    <td><b>28.18</b></td>
  </tr>
  <tr>
    <td>w/o Detection</td>
    <td>35.34</td>
    <td>18.36</td>
    <td>27.22</td>
    <td>11.95</td>
    <td>1.34</td>
    <td>12.70</td>
    <td>24.12</td>
  </tr>
  <tr>
    <td>w/o Mitigation</td>
    <td>39.01</td>
    <td>19.74</td>
    <td>28.35</td>
    <td>12.12</td>
    <td>1.31</td>
    <td>12.44</td>
    <td>24.89</td>
  </tr>
  <tr>
    <td>w/o D&M</td>
    <td>34.23</td>
    <td>17.99</td>
    <td>27.01</td>
    <td>11.37</td>
    <td>1.33</td>
    <td>12.36</td>
    <td>24.17</td>
  </tr>
</table>

Table 4: Ablation study. “D” and “M” in “w/o D&M” denote Detection and Mitigation modules respectively.

task, which involves the agent acting as a doctor and responding to the patient. Table 2 demonstrates that while PaMis is specifically designed to address patient misreports, it outperforms advanced LLMs on general metrics. We provided a robust baseline setting (i.e., w/ Gold Know.) for LLMs with medical capabilities. Incorporating gold knowledge improved GPT-4’s Overall score by 0.17 / 0.22. Introducing the misreporting mechanism and related entities in the input content via PaMis further surpassed it by 0.23 / 0.22. Given that GPT-4 served as the generation model, this suggests that the PaMis framework can enhance even highly advanced methods. Furthermore, the improvements are concentrated in areas beyond linguistic quality (i.e., LQ), suggesting that the enhancements arise from medical-related capabilities rather than language tricks.

RQ2: Performance w.r.t. Misreport Detection and Mitigation

(1) Detection We employ GPT-4 to annotate misreported entities, ensuring rigorous alignment with the misreport detection principle through prompts mirroring the generation phase (see Appendix D). As shown in Table 3, PaMis achieves superior F1 scores across both datasets, outperforming GPT-4 by 3.7% and 5.6%, respectively. The performance gap between datasets likely stems from KaMed’s broader scope of medical entities, which amplifies the challenge of context-specific entity linking. While LLMs exhibit higher recall (e.g., 72.95 on MedDG), their generalized output patterns risk over-inclusion of secondary entities. PaMis addresses this by precision-driven misreport detection, validating its capability to reduce noise while maintaining diagnostic relevance.

(2) Mitigation Mitigating misreports requires continuous dialogue to guide the patient in confirming or modifying the mentioned information. We conducted interactive experiments utilizing a simulator-agent format. Implementation details are provided in Appendix D.

The results of the interactive experiment are presented in Figure 3. It demonstrates that in both quantitative and qualitative analyses, the responses generated by models guided by PaMis exhibit the ability to mitigate misreporting. This is reflected in an increase in the entropy of the entity graph, representing the completion of co-occurrence relationships, which leads to a more reasonable collection of symptoms. Notably, after removing the knowledge related to misreported entities retrieved by PaMis, the results on MedDG perform worse than the baseline, suggesting that the naive retrieval method described in Section 3.4 is effective for mitigating misreporting. In contrast, the MR on KaMed slightly increased, possibly due to the longer average dialogue length in KaMed, which provides sufficient entities to be confirmed, thereby reducing the need for redundant retrieval.

Experimental results above indicate that PaMis has the potential to serve as an aid tool to alleviate the burden of doctor inquiries and mitigate subsequent risks of diagnostic mistakes.

<table>
  <tr>
    <th rowspan="2">Type</th>
    <th rowspan="2">Method</th>
    <th colspan="2">MedDG</th>
    <th colspan="2">KaMed</th>
  </tr>
  <tr>
    <th>DV</th>
    <th>All</th>
    <th>DV</th>
    <th>All</th>
  </tr>
  <tr>
    <td rowspan="3">Is</td>
    <td>GPT-4</td>
    <td>3.53</td>
    <td>19.68</td>
    <td>3.68</td>
    <td>20.19</td>
  </tr>
  <tr>
    <td>4 (w/G.)</td>
    <td>3.54</td>
    <td>19.70</td>
    <td>3.76</td>
    <td>20.50</td>
  </tr>
  <tr>
    <td>PaMis</td>
    <td><b>3.77</b></td>
    <td><b>20.25</b></td>
    <td><b>3.85</b></td>
    <td><b>20.59</b></td>
  </tr>
  <tr>
    <td rowspan="3">De</td>
    <td>GPT-4</td>
    <td>3.68</td>
    <td>20.10</td>
    <td>3.63</td>
    <td>20.16</td>
  </tr>
  <tr>
    <td>4 (w/G.)</td>
    <td>3.74</td>
    <td>20.22</td>
    <td>3.67</td>
    <td>20.25</td>
  </tr>
  <tr>
    <td>PaMis</td>
    <td><b>3.77</b></td>
    <td><b>20.33</b></td>
    <td><b>3.83</b></td>
    <td><b>20.55</b></td>
  </tr>
  <tr>
    <td rowspan="3">Co</td>
    <td>GPT-4</td>
    <td>3.56</td>
    <td>19.91</td>
    <td>3.46</td>
    <td>19.75</td>
  </tr>
  <tr>
    <td>4 (w/G.)</td>
    <td>3.70</td>
    <td>20.42</td>
    <td>3.58</td>
    <td>20.17</td>
  </tr>
  <tr>
    <td>PaMis</td>
    <td><b>3.79</b></td>
    <td><b>20.49</b></td>
    <td><b>3.75</b></td>
    <td><b>20.62</b></td>
  </tr>
</table>

Table 5: Results on different misreport types.

Human evaluation We selected GPT-4 (w/ Gold Know.) as a baseline for comparison with the proposed method and conducted human evaluations using sample-wise comparisons. We employed three doctors to assess the overall quality of the responses, with prior knowledge of the patient misreporting phenomenon. As shown in Figure 4, PaMis significantly outperforms GPT-4 in ranking and achieves a higher sample win rate. The higher performance on KaMed (58.2% win rate vs. 54.9% on MedDG) aligns with PaMis’s strength in handling
P: After drinking a bit of liquor, I woke up in the middle of the night to use the restroom and noticed bleeding. The blood was bright red, and my stomach hurt a little.

D: Hello, have you had any gastrointestinal issues in the past? Do you have hemorrhoids? How many times have you had bloody stools? Have you experienced abdominal pain, nausea, vomiting, fever, or any other discomfort?

P: I have a slight abdominal pain and a burning sensation. I’ve had discomfort from hemorrhoids before but never treated it. I only had bloody stool once, and I haven’t had a bowel movement since. No other discomfort.

D: Could you describe the location of your pain? If your gastrointestinal health has been good in the past, the most likely cause is an acute hemorrhoid flare-up, followed by acute damage to the intestinal mucosa.

P: The burning sensation is in the stomach area, but there’s almost no pain.

<table>
  <tr>
    <th>Gold</th>
    <td>D: Alright, it’s quite normal to experience a burning sensation in the stomach after drinking. How do you feel compared to yesterday?</td>
  </tr>
  <tr>
    <th>Misreported</th>
    <td>Heartburn: Isolated</td>
  </tr>
  <tr>
    <th>GPT-4</th>
    <td>D: Your condition could be caused by acute hemorrhoids or mild gastritis. I suggest monitoring it for now, drinking plenty of water, and avoiding spicy foods. If the bleeding persists or the symptoms worsen, you may want to consider seeking medical attention.</td>
  </tr>
  <tr>
    <th>PaMis</th>
    <td>D: Based on your description, the symptoms may be caused by gastritis or hemorrhoids. I recommend that you seek medical attention as soon as possible to determine the appropriate treatment plan. Have you experienced acid reflux recently?</td>
  </tr>
</table>

Table 6: Case study on the misreported entities and the responses generated by LLMs.

broader medical contexts, where its context-aware misreport detection reduces ambiguities in complex records. Further details and case studies are provided in Appendix E.

Ablation Study We conducted ablation experiments on the misreport-related modules to analyze their effects on response generation, as shown in Table 4. It observed a decrease in E-F scores after removing the misreport detection module and employing a naive prediction method based on entity co-occurrence relationships (i.e., “w/o Detection”). Furthermore, after removing the detection module, the Distinct-1/2 scores of the generated results on both datasets increased. This may be because the generated content was not constrained by specific entities. Removing the misreport mitigation module (i.e., “w/o Mitigation”) resulted in fluctuations in BLEU and ROUGE scores. An increase in R-2 can be observed on the KaMed dataset, which may be because the model no longer receives an excessive number of entities for constructing clarifying questions, resulting in outputs that more closely resemble the conversational style of doctors. Further ablation results are provided in Appendix G.

Performance w.r.t. Misreport Types The results shown in Table 5 validate PaMis’s capability to address three critical misreport categories: (1) Is: PaMis achieves 3.77 DV on MedDG (vs. GPT-4’s 3.53), demonstrating its dynamic grounding to detect misreports while preserving diagnostic intent. (2) De: PaMis’s entity linking ensures robustness, outperforming baselines in All scores across datasets. (3) Co: PaMis attains 20.62 All on KaMed (vs. GPT-4’s 19.75), highlighting its temporal consistency verification to resolve ambiguities. The consistent superiority of PaMis across all types—particularly in DV metrics demonstrates its efficiency in misreport mitigation and resilience to semantic instability.

Case Study As shown in Table 6, when the patient mentions “burning in the stomach”, given that the patient’s description may not be accurate, it could indicate either gastric mucosal damage or heartburn. Since there is no additional information supporting the heartburn symptom, it is necessary to ask the patient again about the presence of acid reflux to determine whether he is experiencing esophageal damage. Existing models often ignore the potential inaccuracy, failing to confirm the patient’s actual condition. PaMis retrieves related entities [Heartburn, Hemorrhoids, Gastritis] for response and probes the intermediate node “reflux”, which could link heartburn to gastritis.

5 Conclusion

We focus on the differences between real patients and the assumed ideal users in the field of medical dialogue systems. We defined the phenomenon that the content mentioned by the patient does not align with known health conditions as Patient Misreports. We then proposed PaMis to detect and mitigate patient misreports. Experimental results indicate the high effectiveness of PaMis in response generation. Interactive experiments further show its
effectiveness in mitigating misreports. We believe the proposed approach can serve as a diagnostic aid tool to alleviate the burden on doctors.

Limitations

This section discusses three limitations of this work. 1) Given the stringent reliability requirements in the medical field, PaMis has clearly defined functional constraints: it is not designed as an independent diagnostic tool, but rather serves as an auxiliary tool within medical dialogue systems to provide recommendations to doctors. The model’s role is limited to detecting potential patient misreports and raising suspicions, rather than directly adjudging the accuracy of patient statements. The performance upper bound is successfully identifying actual misreports and providing recommendations for clarification, while the lower bound is false suspicions, which may result in an increased number of dialogue turns between doctors and patients but do not ultimately affect the final diagnostic outcomes. 2) Secondly, PaMis’s misreport detection primarily relies on prior co-occurrence relations between symptoms and diseases and the temporal patterns of entity appearances in doctor-patient dialogues. PaMis currently lacks the capability to process more complex relationships (such as causality) and subtle contextual nuances that experienced doctors can identify. 3) Finally, PaMis’s effectiveness depends on the coverage of its background static medical knowledge graph. For symptoms or diseases not encountered during training, the framework reverts to a standard RAG model, maintaining medical response capability. However, PaMis demonstrates good adaptability, as it can function normally by incorporating new symptoms into the static knowledge graph without requiring retraining.

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (No.62306156), and in part by the Undergraduate Education and Teaching Reform Projects, Nankai University (NKJG2025047).

References

Mohammad Aliannejadi, Hamed Zamani, Fabio Crestani, and W. Bruce Croft. 2019. Asking clarifying questions in open-domain information-seeking conversations. In Proceedings of SIGIR, page 475–484.

Zhijie Bao, Wei Chen, Shengze Xiao, Kuang Ren, Jiaao Wu, Cheng Zhong, Jiajie Peng, Xuanjing Huang, and Zhongyu Wei. 2023. Disc-medllm: Bridging general large language models and real-world medical consultation. arXiv preprint arXiv:2308.14346.

Nancy D Berkman, Stacey L Sheridan, Katrina E Donahue, David J Halpern, and Karen Crotty. 2011. Low health literacy and health outcomes: an updated systematic review. Annals of internal medicine, 155(2):97–107.

Suman Bhoi, Mong Li Lee, Wynne Hsu, and Ngiap Chuan Tan. 2023. Refine: A fine-grained medication recommendation system using deep learning and personalized drug interaction modeling. In Proceedings of NeurIPS, volume 36, pages 24013–24024.

Odmaa Byambasuren, Yunfei Yang, Zhifang Sui, Damai Dai, Baobao Chang, Sujuan Li, and Hongying Zan. 2019. Preliminary study on the construction of chinese medical knowledge graph. Journal of Chinese Information Processing, 33(10):1–7.

Junying Chen, Xidong Wang, Ke Ji, Anningzhe Gao, Feng Jiang, Shunian Chen, Hongbo Zhang, Song Dingjie, Wenya Xie, Chuyi Kong, Jianquan Li, Xi-ang Wan, Haizhou Li, and Benyou Wang. 2024. HuatuoGPT-II, one-stage training for medical adaptation of LLMs. In Proceedings of the 1st Conference on Language Modeling.

Siyan Chen, Mengyue Wu, Kenny Q. Zhu, Kunyao Lan, Zhiling Zhang, and Lyuchun Cui. 2023. Llm-empowered chatbots for psychiatrist and patient simulation: Application and evaluation. arXiv preprint arXiv:2305.13614.

Yue Feng, Hossein A Rahmani, Aldo Lipani, and Emine Yilmaz. 2023. Towards asking clarification questions for information seeking on task-oriented dialogues. arXiv preprint arXiv:2305.13690.

Avi Fleischer, Alan D Mead, and Jialin Huang. 2015. Inattentive responding in mturk and other online samples. Industrial and Organizational Psychology, 8(2):196–202.

Zhenyu Hou, Yukuo Cen, Ziding Liu, Dongxue Wu, Baoyan Wang, Xuanhe Li, Lei Hong, and Jie Tang. 2023. Mtdrag: An effective multi-task framework for automatic diagnosis. In Proceedings of AAAI, pages 14241–14248.

Zefa Hu, Xiuyi Chen, Haoran Wu, Minglun Han, Ziyi Ni, Jing Shi, Shuang Xu, and Bo Xu. 2023. Matching-based term semantics pre-training for spoken patient query understanding. In Proceedings of ICASSP, pages 1–5.
Angsheng Li and Yicheng Pan. 2016. Structural information and dynamical complexity of networks. IEEE Transactions on Information Theory, 62(6):3290–3339.

Dongdong Li, Zhaochun Ren, Pengjie Ren, Zhumin Chen, Miao Fan, Jun Ma, and Maarten de Rijke. 2021. Semi-supervised variational reasoning for medical dialogue generation. In Proceedings of SIGIR, page 544–554.

Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of NAACL-HLT, pages 110–119.

Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan S. Ilgen, Emma Pierson, Pang Wei Koh, and Yulia Tsvetkov. 2024. Mediq: Question-asking llms and a benchmark for reliable interactive clinical reasoning. arXiv preprint arXiv:2406.00922.

Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81.

Shuai Lin, Pan Zhou, Xiaodan Liang, Jianheng Tang, Ruihui Zhao, Ziliang Chen, and Liang Lin. 2021. Graph-evolving meta-learning for low-resource medical dialogue generation. In Proceedings of AAAI, pages 13362–13370.

Xinzhu Lin, Xiahui He, Qin Chen, Huaixiao Tou, Zhongyu Wei, and Ting Chen. 2019. Enhancing dialogue symptom diagnosis with global attention and symptom graph. In Proceedings of EMNLP-IJCNLP, pages 5033–5042.

Fenglin Liu, Bang Yang, Chenyu You, Xian Wu, Shen Ge, Zhangdaihong Liu, Xu Sun, Yang Yang, and David Clifton. 2022a. Retrieve, reason, and refine: Generating accurate and faithful patient instructions. In Proceedings of NeurIPS, volume 35, pages 18864–18877.

Wenge Liu, Yi Cheng, Hao Wang, Jianheng Tang, Yafei Liu, Ruihui Zhao, Wenjie Li, Yefeng Zheng, and Xiaodan Liang. 2022b. “my nose is running.” “are you also coughing?”: Building a medical diagnosis agent with interpretable inquiry logics. In Proceedings of IJCAI, pages 4266–4272.

Wenge Liu, Jianheng Tang, Yi Cheng, Wenjie Li, Yefeng Zheng, and Xiaodan Liang. 2022c. Meddg: An entity-centric medical consultation dataset for entity-aware medical dialogue generation. In Proceedings of the Natural Language Processing and Chinese Computing, pages 447–459.

Wenge Liu, Jianheng Tang, Xiaodan Liang, and Qingling Cai. 2021. Heterogeneous graph reasoning for knowledge-grounded medical dialogue system. Neurocomputing, 442:260–268.

Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101.

Harald Merckelbach, Brechje Dandachi-FitzGerald, Daniel van Helvoort, Marko Jelicic, and Henry Otgaar. 2019. When patients overreport symptoms: More than just malingering. Current Directions in Psychological Science, 28(3):321–326.

OpenAI. 2024. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.

Ryosuke Oshima, Seitaro Shinagawa, Hideki Tsunashima, Qi Feng, and Shigee Morishima. 2023. Pointing out human answer mistakes in a goal-oriented visual dialogue. In Proceedings of ICCV Workshops, pages 4663–4668.

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155.

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of ACL, pages 311–318.

Lindsay Prior, Meirion R Evans, and Hayley Prout. 2011. Talking about colds and flu: the lay diagnosis of two common illnesses among older british people. Social Science & Medicine, 73(6):922–928.

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.

Sudha Rao. 2017. Are you asking the right questions? teaching machines to ask clarification questions. In Proceedings of ACL, Student Research Workshop, pages 30–35.

C. Shannon. 1953. The lattice theory of information. Transactions of the IRE Professional Group on Information Theory, 1(1):105–107.

Chen Tang, Hongbo Zhang, Tyler Loakman, Chenghua Lin, and Frank Guerin. 2023. Terminology-aware medical dialogue generation. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1–5.

Zhenduo Wang, Zhichao Xu, Qingyao Ai, and Vivek Srikumar. 2024. An in-depth investigation of user response simulation for conversational search. arXiv preprint arXiv:2304.07944.

Zhongyu Wei, Qianlong Liu, Baolin Peng, Huaixiao Tou, Ting Chen, Xuanjing Huang, Kam-fai Wong, and Xiangying Dai. 2018. Task-oriented dialogue system for automatic diagnosis. In Proceedings of ACL, pages 201–207.
Jingjing Xu, Yuechen Wang, Duyu Tang, Nan Duan, Pengcheng Yang, Qi Zeng, Ming Zhou, and Xu Sun. 2019a. Asking clarification questions in knowledge-based question answering. In Proceedings of EMNLP-IJCNLP, pages 1618–1629.

Kaishuai Xu, Yi Cheng, Wenjun Hou, Qiaoyu Tan, and Wenjie Li. 2024. Reasoning like a doctor: Improving medical dialogue systems via diagnostic reasoning process alignment. In Findings of ACL, pages 6796–6814.

Kaishuai Xu, Wenjun Hou, Yi Cheng, Jian Wang, and Wenjie Li. 2023. Medical dialogue generation via dual flow modeling. In Findings of ACL, pages 6771–6784.

Lin Xu, Qixian Zhou, Ke Gong, Xiaodan Liang, Jianheng Tang, and Liang Lin. 2019b. End-to-end knowledge-routed relational dialogue system for automatic diagnosis. In Proceedings of AAAI, pages 7346–7353.

Yu Zhao, Yunxin Li, Yuxiang Wu, Baotian Hu, Qingcai Chen, Xiaolong Wang, Yuxin Ding, and Min Zhang. 2022. Medical dialogue response generation with pivotal information recalling. In Proceedings of SIGKDD, page 4763–4771.

Ziliang Zhao and Zhicheng Dou. 2024. Generating multi-turn clarification for web information seeking. In Proceedings of WWW, page 1539–1548.
(a) The worst scenario of self-contradiction   (b) The best scenario of denial
Figure 5: Different scenarios after losing a node (using the example of 4 remaining nodes).

A Supplementary derivation of graph entropy

If there are \( n \) nodes remaining in the entity graph after a misreport, a self-contradiction involves at least \( n - 1 \) edges connecting these nodes. As illustrated in Figure 5, in the worst-case scenario, the missing node was connected to every node in the original graph. Therefore, the minimum graph entropy is given by:

\[
-\sum_{i=1}^{n} \frac{d_i - 1}{2(n-1)} \log_2 \frac{d_i - 1}{2(n-1)}.
\]

In the case of a denial, the best-case scenario occurs when the missing node in the original graph has only two edges. After the denial, the result is a connected graph with \( n - 1 \) nodes and one isolated node, where the degree of the connected graph is \( \mathrm{Vol}(G) - 4 \geq 2(n-1) \). Thus, the upper bound of entropy is:

\[
F(d) = \frac{d}{\mathrm{Vol}(G)-4} \log_2 \frac{d}{\mathrm{Vol}(G)-4},
\]
\[
-\left( \sum_{i=1}^{n-2} F(d_i) + F(d_{n-1} - 1) \right),
\]

Subtracting them yields a new equation that is evident when \( n = 2 \). For \( n \geq 2 \), the left-hand side is a monotonically increasing function with respect to \( n \), while the right-hand side remains constant, indicating that if the equation holds for \( n = 2 \), it will also hold for any \( n \geq 2 \). Consequently, the lower bound for contradictions surpasses the upper bound for denials:

\[
-\sum_{i=1}^{n} \frac{d_i - 1}{2(n-1)} \log_2 \frac{d_i - 1}{2(n-1)} + \sum_{i=1}^{n-2} F(d_i) > -F(d_{n-1} - 1).
\]

B Baselines

LLMs: (1) DISC-MedLLM (Bao et al., 2023), a trustworthy medical LLM adapted for multi-turn dialogues. (2) HuatuoGPT2 (Chen et al., 2024), an advanced medical LLM trained on Chinese medical corpus.

Fine-tuned models: (1) VRBot (Li et al., 2021), a medical dialogue generation model based on patient entity tracking and doctor entity learning; and (2) DFMed (Xu et al., 2023), this framework performs the generation by fine-tuning a backbone model with dual-flow modeling. (3) EMULATION (Xu et al., 2024), this framework relies on diagnostic reasoning analyses and aligns with clinician preferences through thought process modeling.

C Complementary Implementation Details

For comparisons with fine-tuned baselines on the response generation task, we follow the settings of previous works, allowing any doctor’s response to serve as the target sentence. We employ AdamW optimizer (Loshchilov and Hutter, 2017) with a weight decay of 0.02. Reproducibility is ensured by fixing random seeds. The proposed model is trained on an A100. Training is conducted with a learning rate of 1e-4 for 10 epochs, with no adjustment of other hyperparameters apart from setting the maximum generation length to 160 tokens.

(1) Time complexity. The graph constructing and entropy calculation has a time complexity of \( O(m+n) \), where \( m \) represents the number of edges and \( n \) the number of nodes. The graph maintaining is \( O(1) \). Since entropy requires only scalar computation, its computational complexity is negligible compared to classification or detection methods in modeling approaches.

(2) Space complexity. Due to the limited number of dialogue turns and entities (Table 1: 20 utterances and 12-15 entities), the system’s space complexity is minimal. To further reduce costs, PaMis only records changes in entities within the graph. The storage cost for most samples (including multiple graphs) is typically under 0.03 MB, as measured using networkx and pympler.asizeof for memory profiling.

D Prompts

Addressing concerns about the quality of dialogue simulators (Wang et al., 2024), we adopted the method proposed by Chen et al. (2023) to employ LLMs as simulated patients. The interactive experiment was initiated by selecting instances where misreports were detected within the dialogue. We then input the dialogue context into LLMs and re-
<table>
  <tr>
    <th>Annotation</th>
    <td>There is a doctor conducting a conversational consultation with a patient.<br><b>1.</b> Determine whether the symptoms or diseases mentioned in the conversation content are:<ul><li>Insufficient supporting evidence (requires doctor’s further inquiry)</li><li>Not in line with medical knowledge (requires doctor’s correction)</li><li>Patient self-contradiction (requires repeated confirmation)</li></ul><b>2.</b> Directly output the medical entities meeting these conditions.</td>
  </tr>
  <tr>
    <th>Detection</th>
    <td>There is a doctor conducting a conversational consultation with a patient.<br><b>1.</b> Analyze [Dialogue History] containing potentially incomplete, inaccurate, and inconsistent statements.<br><b>2.</b> Directly output medical entities matching the problematic conditions.</td>
  </tr>
</table>

Table 7: The prompts for annotation and detection tasks.

<table>
  <tr>
    <th>Doctor</th>
    <td>You are a doctor conducting a conversational consultation with a patient.<br><b>1.</b> Take the information from the [Dialogue History] into account, which may include incomplete, inaccurate, or inconsistent details in the patient’s statement.<br><b>2.</b> Reference the ‘Potentially Inaccurate Entity’ and ‘Related Entity’ in [Medical Knowledge] to provide accurate medical advice and help resolve uncoordinated issues.<br><b>3.</b> Respond in a way that is concise, approachable, and compassionate. Ask follow-up questions to gather more details and may also correct errors.<br><b>4.</b> Keep the conversation natural, focusing on one or two key points at a time to ensure the patient feels supported and informed.<br><b>5.</b> The response should be bite-sized and not give too much information at once, which is similar to what the doctor did in dialogue history.</td>
  </tr>
  <tr>
    <th>Patient</th>
    <td>You are a patient engaging in a conversational consultation with a doctor.<br><b>1.</b> Consider the [Dialogue History] and [Patient’s Symptoms] for context to continue the conversation naturally.<br><b>2.</b> Respond briefly, using one to two short sentences. Describe your symptoms in a vague and colloquial manner, avoiding professional terminology.<br><b>3.</b> Express emotions like worry, anxiety, or hesitation as appropriate to reflect your life experiences.<br><b>4.</b> Keep your responses natural and reflective of how a patient might realistically communicate their concerns.</td>
  </tr>
</table>

Table 8: The prompts for simulated patients and doctors.

quested that they respond as patients, guided by the generated doctor’s inquiries. We employed the graph entropy method to assess whether misreports within the dialogue were mitigated. Notably, although the ChatGPT series models are not specialized in the medical domain, they surpass most models in medical-related evaluations\footnote{paperswithcode.com/sota/question-answering-on-medqa-umlsle}. We build on the prompt settings provided by Chen et al. (2023) to simulate patient-doctor interactions, as illustrated in Table 8. Simulated patients are utilized to assess whether doctor responses can help patients clarify their actual health conditions, thereby mitigating misreports. The prompts above were constructed with the participation of medical professionals and demonstrated their effectiveness as simulations of patients and doctors.

E Human evaluation

We employed three medical students at the graduate level, compensating them at four times the local base hourly wage, to conduct the manual evaluations. Prior to the evaluation, they were fully informed that all samples were from publicly available datasets and did not involve any real patients. Furthermore, it was ensured that the evaluators were unaware of which model each response originated from. Positive and negative cases, along with their corresponding evaluations, can be found in Table 9 and Table 10.
P: After drinking a bit of liquor, I woke up in the middle of the night to use the restroom and noticed bleeding. The blood was bright red, and my stomach hurt a little.

D: Hello, have you had any gastrointestinal issues in the past? Do you have hemorrhoids? How many times have you had bloody stools? Have you experienced abdominal pain, nausea, vomiting, fever, or any other discomfort?

P: I have a slight abdominal pain and a burning sensation. I’ve had discomfort from hemorrhoids before but never treated it. I only had bloody stool once, and I haven’t had a bowel movement since. No other discomfort.

D: Could you describe the location of your pain? If your gastrointestinal health has been good in the past, the most likely cause is an acute hemorrhoid flare-up, followed by acute damage to the intestinal mucosa.

P: The burning sensation is in the stomach area, but there’s almost no pain.

<table>
  <tr>
    <th>Misreported</th>
    <td>Heartburn: Isolated</td>
  </tr>
  <tr>
    <th>GPT-4</th>
    <td>D: Your condition could be caused by acute hemorrhoids or mild gastritis. I suggest monitoring it for now, drinking plenty of water, and avoiding spicy foods. If the bleeding persists or the symptoms worsen, you may want to consider seeking medical attention.</td>
  </tr>
  <tr>
    <th>PaMis</th>
    <td>D: Based on your description, the symptoms may be caused by gastritis or hemorrhoids. I recommend that you seek medical attention as soon as possible to determine the appropriate treatment plan. Have you experienced acid reflux recently?</td>
  </tr>
  <tr>
    <th>Human Evaluation</th>
    <td>The patient’s condition may involve acute gastritis and hemorrhoids. The second response is more precise in its inquiry about the symptom details and recommends that the patient seek medical attention promptly, which aligns more closely with clinical guidelines.</td>
  </tr>
</table>

Table 9: Case study with human evaluations.

<table>
  <tr>
    <th>Context</th>
    <td>P: Prostatitis, I want to see a traditional Chinese medicine doctor (Male, 26 years old).<br>D: Hello! What symptoms do you have?<br>P: I often have a low fever, drink a lot of water but urinate less, and feel like toxin are not being expelled from my body.<br>D: Have you measured your temperature? What is the exact reading?<br>P: A little over 36 degrees. The main issues are joint heat and poor sleep.<br>D: Your symptoms may not be just prostatitis; further examination is needed to confirm.<br>P: Could it be that my prostate is blocked, leading to toxin buildup?</td>
  </tr>
  <tr>
    <th>PaMis</th>
    <td>D: Have you noticed any other unusual symptoms, such as a cold or fever?</td>
  </tr>
  <tr>
    <th>Ground-Truth</th>
    <td>D: It’s not understood that way. Prostatitis can cause prostate enlargement, but joint inflammation is likely caused by other inflammatory factors.</td>
  </tr>
  <tr>
    <th>Human Evaluation</th>
    <td>The second response better addresses the patient’s concerns and, to some extent, corrects and enhances the patient’s relevant knowledge.</td>
  </tr>
</table>

Table 10: Error case analysis with human evaluations.

F Error Case Analysis

As shown in Table 10, this error case demonstrates that due to semantic complexity, PaMis is unable to associate the toxin mentioned by the patient with an existing entity. As a result, PaMis fails to perform accurate misreport detection and instead focuses only on the patient’s self-reported but unverified fever symptoms when generating inquiries. In contrast, the doctor, familiar with the patient’s colloquial use of toxin, appropriately clarified the misunderstanding.

G Supplementary Experiments

Additional Baselines As illustrated in Table 11, we conducted additional experiments using the more recent DeepSeek-V3 as the baseline, and our method has maintained its superior performance.

Supplementary Ablation Study As illustrated in Table 12, PaMis outperforms the overall average on samples containing misreports, and the detection method is effective for all types of misreports, resulting in an improved F1 score when perform entity prediction task. The performance only declines on disease entities, possibly due to the tendency to select other types of entities to fill co-occurrence relationships rather than making direct diagnoses.

N-gram-based results For RQ2, The n-gram-based results presented in Table 13 illustrate that the proposed misreport mitigation module can remain competitive with the state-of-the-art approach in the response generation task of medical dialogue systems.

Specifically, when relevant knowledge from the entity graph is introduced into the input of the generation model, it tends to generate inquiries that comprehensively incorporate related entities. This
<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="6">MedDG</th>
    <th colspan="6">KaMed</th>
  </tr>
  <tr>
    <th>P</th><th>A</th><th>H</th><th>LQ</th><th>DV</th><th>Overall</th>
    <th>P</th><th>A</th><th>H</th><th>LQ</th><th>DV</th><th>Overall</th>
  </tr>
  <tr>
    <td>deepseek-v3</td>
    <td>3.68</td><td>4.21</td><td>3.54</td><td>4.45</td><td>3.39</td><td>19.28</td>
    <td>3.61</td><td>4.31</td><td>3.63</td><td>4.41</td><td>3.33</td><td>19.29</td>
  </tr>
  <tr>
    <td></td>
    <td>±0.03</td><td>±0.02</td><td>±0.02</td><td>±0.02</td><td>±0.03</td><td>±0.10</td>
    <td>±0.04</td><td>±0.03</td><td>±0.01</td><td>±0.01</td><td>±0.05</td><td>±0.06</td>
  </tr>
  <tr>
    <td>deepseek-v3 (w/ G.)</td>
    <td>3.91</td><td>4.30</td><td>3.77</td><td>4.52</td><td>3.65</td><td>20.15</td>
    <td>3.79</td><td>4.34</td><td>3.84</td><td>4.46</td><td>3.51</td><td>19.94</td>
  </tr>
  <tr>
    <td></td>
    <td>±0.01</td><td>±0.03</td><td>±0.02</td><td>±0.02</td><td>±0.02</td><td>±0.08</td>
    <td>±0.03</td><td>±0.01</td><td>±0.04</td><td>±0.01</td><td>±0.02</td><td>±0.02</td>
  </tr>
  <tr>
    <td>PaMis</td>
    <td>4.22</td><td>4.29</td><td>4.12</td><td>4.52</td><td>3.99</td><td>21.14</td>
    <td>4.20</td><td>4.33</td><td>4.10</td><td>4.49</td><td>4.00</td><td>21.12</td>
  </tr>
  <tr>
    <td></td>
    <td>±0.03</td><td>±0.03</td><td>±0.02</td><td>±0.02</td><td>±0.02</td><td>±0.08</td>
    <td>±0.01</td><td>±0.01</td><td>±0.02</td><td>±0.02</td><td>±0.00</td><td>±0.03</td>
  </tr>
</table>

Table 11: Evaluation results of responses generated from DeepSeek-V3 and PaMis. “Gold” indicates that the entities in the actual doctor’s responses are included in the input.

<table>
  <tr>
    <th rowspan="2"></th>
    <th colspan="2">P</th>
    <th colspan="2">R</th>
    <th colspan="2">F1</th>
    <th colspan="2">F1_A</th>
    <th colspan="2">F1_D</th>
    <th colspan="2">F1_E</th>
    <th colspan="2">F1_M</th>
    <th colspan="2">F1_S</th>
  </tr>
  <tr>
    <th>Is</th><th></th>
    <th>Is w/o Detection</th><th></th>
    <th>De</th><th></th>
    <th>De w/o Detection</th><th></th>
    <th>Co</th><th></th>
    <th>Co w/o Detection</th><th></th>
  </tr>
  <tr>
    <td>Is</td>
    <td>34.48</td><td>44.44</td>
    <td>30.32</td><td>41.05</td>
    <td>34.88</td><td>100.0</td>
    <td>50.00</td><td>0.0</td>
    <td>35.90</td><td>30.49</td>
  </tr>
  <tr>
    <td>Is w/o Detection</td>
    <td></td><td></td>
    <td></td><td></td>
    <td></td><td>100.0</td>
    <td>25.00</td><td>50.00</td>
    <td>31.97</td><td>29.64</td>
  </tr>
  <tr>
    <td>De</td>
    <td>43.78</td><td>47.07</td>
    <td>41.76</td><td>48.66</td>
    <td>45.37</td><td>57.32</td>
    <td>61.11</td><td>33.33</td>
    <td>40.41</td><td>36.24</td>
  </tr>
  <tr>
    <td>De w/o Detection</td>
    <td></td><td></td>
    <td></td><td></td>
    <td></td><td>54.37</td>
    <td>70.32</td><td>33.33</td>
    <td>32.31</td><td>40.85</td>
  </tr>
  <tr>
    <td>Co</td>
    <td>30.21</td><td>43.86</td>
    <td>16.99</td><td>37.27</td>
    <td>23.35</td><td>50.00</td>
    <td>42.86</td><td>100.0</td>
    <td>70.59</td><td>29.73</td>
  </tr>
  <tr>
    <td>Co w/o Detection</td>
    <td></td><td></td>
    <td></td><td></td>
    <td></td><td>45.45</td>
    <td>68.18</td><td>0.0</td>
    <td>50.00</td><td>16.00</td>
  </tr>
</table>

Table 12: Ablation results for entity prediction on samples containing different types of misreports. A-S denote attribute, disease, examination, medicine, and symptom.

<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="6">B-I</th>
    <th colspan="6">B-4</th>
    <th colspan="6">R-1</th>
    <th colspan="6">R-2</th>
    <th colspan="6">D-1</th>
    <th colspan="6">D-2</th>
    <th colspan="6">E-F</th>
  </tr>
  <tr>
    <th colspan="6">MedDG</th>
    <th colspan="6">KaMed</th>
  </tr>
  <tr>
    <td>GPT-2</td>
    <td>35.27</td><td>19.16</td><td>28.74</td><td>13.61</td><td>0.87</td><td>11.20</td><td>16.14</td>
    <td>33.76</td><td>17.82</td><td>26.80</td><td>10.56</td><td>-</td><td>-</td><td>17.26</td>
  </tr>
  <tr>
    <td>VRBot</td>
    <td>29.69</td><td>16.34</td><td>24.69</td><td>11.23</td><td>-</td><td>-</td><td>12.78</td>
    <td>30.04</td><td>16.36</td><td>18.71</td><td>7.28</td><td>1.61</td><td>9.30</td><td>12.08</td>
  </tr>
  <tr>
    <td>DFMED</td>
    <td>42.56</td><td>22.53</td><td>29.31</td><td>14.21</td><td>0.85</td><td>8.28</td><td>22.66</td>
    <td>40.50</td><td>20.92</td><td>28.33</td><td>11.73</td><td>-</td><td>-</td><td>22.31</td>
  </tr>
  <tr>
    <td>EMULATION</td>
    <td>42.35</td><td>22.76</td><td>30.91</td><td>15.17</td><td>-</td><td>-</td><td>24.03</td>
    <td>39.87</td><td>19.79</td><td>28.54</td><td>12.33</td><td>-</td><td>-</td><td>24.27</td>
  </tr>
  <tr>
    <td>PaMis</td>
    <td>44.28</td><td>24.88</td><td>28.12</td><td>13.80</td><td>1.23</td><td>11.58</td><td>25.13</td>
    <td>41.02</td><td>21.30</td><td>28.42</td><td>12.01</td><td>1.30</td><td>11.39</td><td>28.18</td>
  </tr>
</table>

Table 13: Evaluation results of responses generated from fine-tuned baselines and PaMis under the N-gram-based metrics. Baseline results are reported from Li et al. (2021) and Xu et al. (2024).

approach improves the E-F score but reduces the ROUGE score compared to ground-truth responses, as some doctors prefer shorter, more conversational questions.

Reviewer paper summaries: This paper investigates the issue of patient misreporting in medical dialogue systems  and proposes the PaMis framework to detect and mitigate such misreports. Experimental results demonstrate that this approach improves the quality of medical responses, enabling models like GPT-4 to more effectively detect and reduce misreporting, thereby enhancing the accuracy and reliability of medical consultations.
Summary of strengths: 1.PaMis  effectively detects potential misreports and mitigates them through intelligent inquiry strategies, improving diagnostic accuracy.

2.The results show that PaMis outperforms existing methods in medical response generation and enhances the capability of large models like GPT-4 in medical consultations.
Summary of weaknesses: 1.PaMis relies on medical entity maps and knowledge bases to detect false positives, but the quality and coverage of these knowledge bases may affect detection. If diseases not included in the knowledge base are encountered, the method may not accurately identify false positives, affecting its application in real scenarios.

2.The method involves entity graph construction, graph entropy computation and clarifying question generation with high computational cost. In practical applications, especially in real-time medical dialogue systems, this may lead to high inference latency and the authors should analyse the computational latency problem.

3.The article primarily relies on entity relationships and structural entropy to determine patient misreporting. I am curious whether a patient's vague description, such as "pain in the lower part of the chest," would be considered a misreport. After all, many patients may not be able to accurately name specific organs or describe referred pain.
Comments, suggestions and typos: No Suggestions.
Overall assessment scores: 3
Metareview score: 3.5
Metareview: This paper (which is a resubmission) uses entity graphs to detect cases when patients misreport problems to doctors, which is a huge problem in real-world healthcare (and doctors have complained about this to me).  The innovation is the application more than the tech, which is appropriate for a paper in NLP Applications track.
Summary of reasons to publish: I love the application, this would make a real difference to healthcare if it worked and was deployed.

Evaluation is not ideal but I was happy to see a human evaluation and some error analysis

Overall I strongly support this paper being published in some form.  It is an imprortant and innovative application, which is what the NLP Applications track is about