Reasoning Like a Doctor: Improving Medical Dialogue Systems via Diagnostic Reasoning Process Alignment (2024)

Abstract: Medical dialogue systems have attracted significant attention for their potential to act as medical assistants. Enabling these medical systems to emulate clinicians’ diagnostic reasoning process has been the long-standing research focus. Previous studies rudimentarily realized the simulation of clinicians’ diagnostic process by fine-tuning language models on high-quality dialogue datasets. Nonetheless, they overly focus on the outcomes of the clinician’s reasoning process while ignoring their internal thought processes and alignment with clinician preferences. Our work aims to build a medical dialogue system that aligns with clinicians’ diagnostic reasoning processes. We propose a novel framework, Emulation, designed to generate an appropriate response that relies on abductive and deductive diagnostic reasoning analyses and aligns with clinician preferences through thought process modeling. Experimental results on two datasets confirm the efficacy of Emulation. Crucially, our framework furnishes clear explanations for the generated responses, enhancing its transparency in medical consultations.

## introduction medical dialogue systems as a fundamental tool in facilitating effective healthcare consultations have garnered sustained attention in recent years @xcite . especially with the rise of large language models (llms), these systems have shown promising potential to act as virtual medical assistants that can aid clinicians in accurate and efficient diagnosis @xcite @xcite . in this research area, how to enable these medical systems to emulate clinicians' diagnostic reasoning process has been the long-standing research focus ever since hello, how long have you been experiencing this? are there any other symptoms of discomfort? i often feel uncomfortable in the lower left abdomen. it's not painful, and i've had an abdominal ultrasound which showed no issues. (female, 27 years old). it's been on and off for a few months. is your bowel movement normal? my stool is not well-formed and sometimes i have diarrhea. but i do have a bowel movement almost every day, which is quite regular. ## method in this section, we introduce a diagnostic reasoning framework, which analyzes patient conditions through abductive reasoning and deductive reasoning @xcite , and aligns responses with the clinician preferences. as shown in figure 2 , our framework includes three modules. the abductive reasoner ( §3.1) first generates potential diseases based on the new clinical findings of each turn. then, the deductive reasoner ( §3.2) comprehensively analyzes the relation between potential diseases and new clinical findings. finally, the thought alignment ( §3.3) module adjusts the disease priority that might be discussed in subsequent dialogues and performs thought process modeling to generate the next response. given the medical dialogue history u t at the t-th conversation turn, we first use a pre-trained llm to summarize new clinical findings from the recent utterances u d t-1 and u p t . the clinical findings are presented using the soap note @xcite , a documentation technique utilized by healthcare professionals for recording notes (e.g., chief complaint) in a patient's medical record. for example, we provide the llm with a prompt to "summarize the clinical findings from the recent conversation between doctor and patient, adhering to the structure of the soap note". each of the summarized clinical findings is in the phrase format and merged into a set s @xmath0, where m represents the number of clinical findings at the t-th conversation turn. ## thought process corpus in this section, we present the medical dialogue thought process corpus, mediator, where each dialogue turn is annotated with a chain-of-thought reasoning path. we utilize the powerful reasoning ability of gpt-4 and its impressive professional skills in the medical domain for automatic annotation. the annotated reasoning path is expected to reflect the thought process of clinicians who analyze the patient's condition and determine what to discuss. two datasets, meddg @xcite and @xcite , are annotated following the few-shot prompt in §3.3. as shown in patient: what's going on with my throat feeling itchy and making me want to gag? (female, 22 years old) doctor: how long have you been experiencing this? patient: about half an hour. and my temperature is just under 37 degrees. doctor: have you had pharyngitis before? patient: no, but i have chronic gastritis. doctor: so, this hasn't happened before? patient: only when i have a cold, i would feel itchy and cough. also, i tend to feel uncomfortable when the seasons change. other than that, no. thought processs: 1. the patient mentions symptoms of itchy throat and gagging during colds or when the seasons change. this indicates respiratory issues. 2. the doctor considers the possibility of an allergic reaction, as ... seasonal changes frequently introduce variations in allergens and irritants in the environment, which might explain the patient's symptoms. 3. the patient's symptoms are not frequent, but appear to be triggered by specific conditions or events. this further supports the allergies or environmental causes. 4. the patient mentions having chronic gastritis, but the relation to the current symptoms is unclear ... but usually does not cause an itchy throat. so, confirming allergy issues is a priority. therefore, the doctor responds, "have you done any cleaning today? or been exposed to dust mites?"... human assessments are carried out to evaluate the quality of the generated thought processes. we randomly select 100 samples from each dataset and ask three medical students who have undergone clinical internships to assess them. the evaluation employs three metrics: (1) knowledge: whether the knowledge used in the thought process is accurate; (2) consistency: whether the thought process is consistent with the dialogue history; (3) rationality: whether the thought process starts with premises and uses logical progression to derive responses.