Knowledge Grounded Conversational Symptom Detection with Graph Memory Networks (2020)

Abstract: In this work, we propose a novel goal-oriented dialog task, automatic symptom detection. We build a system that can interact with patients through dialog to detect and collect clinical symptoms automatically, which can save a doctor’s time interviewing the patient. Given a set of explicit symptoms provided by the patient to initiate a dialog for diagnosing, the system is trained to collect implicit symptoms by asking questions, in order to collect more information for making an accurate diagnosis. After getting the reply from the patient for each question, the system also decides whether current information is enough for a human doctor to make a diagnosis. To achieve this goal, we propose two neural models and a training pipeline for the multi-step reasoning task. We also build a knowledge graph as additional inputs to further improve model performance. Experiments show that our model significantly outperforms the baseline by 4%, discovering 67% of implicit symptoms on average with a limited number of questions.

## introduction in a typical clinical conversation between a patient and a doctor, the patient initiates the dialog by providing a number of explicit symptoms as a selfreport. based on this information, the doctor asks about other possible symptoms, in order to make an accurate diagnosis and suggest treatments. this is a multi-step reasoning process. at each step, the doctor choose a symptom to ask or concludes the diagnosis by considering the dialog history and possible diseases. with recent advances in deep reinforcement learning @xcite and task-oriented dialog systems @xcite , recent studies have proposed human-computer dialog systems for automatic diagnosis @xcite . the automatic diagnosis system applied a deep q network (dqn) to decide whether to continue the dialog by asking about a symptom or conclude the diagnosis by predicting a disease. @xcite proposed a knowledge-routed dqn that improves this process by considering relations among diseases and symptoms. the systems described above can achieve around 70% accuracy in making a diagnosis among 4 common diseases and detects a few implicit symptoms. however, the automatic diagnosis systems is far from being ready for clinical diagnosis, since there is still a gap in accuracy between the system and human doctors. furthermore, he current legislation system has to be amended such that liability can be clarified when the system mis-diagnoses. as a result, current machines are not ready to replace human doctors, but they can still detect symptoms automatically to assist doctors making decision more efficiently. in this work, we propose a system that automatically detects clinical symptoms of a patient through dialog, instead of diagnosing. with such symptom information collected, a doctor can be more knowledgeable before interviewing the patient. only those symptoms not covered by the dialog system should be asked during the interview. thus both the time and cost of the clinical interviews can be reduced. meanwhile, the diagnosis is still made by the doctor, which guarantees the accuracy and responsibility. the baby has a runny nose. agent: does the baby cough? user: yes, the baby is coughing. agent: does the baby have a fever? user: i am not sure. agent: it is upper respiratory infection ## data and task definition in this section, we formally define the automatic symptom detection task and describe the corpus used to train and evaluate the model. we first introduce the muzhi corpus @xcite , then describe the task based on the corpus. lastly, we describe the medical knowledge graph we annotated and the annotation method. ## methods in this section, we introduce the structure and pipeline of the proposed automatic symptom detection system, including dialog state representation, the neural models for predicting symptoms and dialog actions, the training strategy, and the evaluation metrics. ## experiments we train and evaluate our models on the muzhi corpus. the symptom predictor and the dialog action predictor are trained separately. using the same strategy of simulating the training set, we also generated test sets for symptom prediction and action prediction respectively using the test user goals with the same method. the generated test sets are used for evaluating the performances of our models on both unit tasks. after evaluating the models in with the unit tasks, we conduct conversational evaluations using the trained models and a user simulator. we evaluate the performance of the models by accounting the number of implicit and unrelated symptoms queried in the conversations. ## conclusion in this work, we propose a new task: detecting implicit symptoms of patient with an automatic dialog system. we construct the system with a dialog action prediction module and a symptom query module. we first implement and evaluate a baseline system based on multi-layer perceptrons (mlps). to improve the performance of the system, we annotate a medical-domain knowledge graph and propose the graph memory network (gmemnn) model. we systematically evaluate and compare both models with unit tasks and conversations. we also studied how the number of dialog turns effects the performance of the systems. experiments showed that both models can detect more than 60% implicit symptoms using limited turns of dialogs, which significantly outperformed the state-of-theart automatic diagnosis system. in future work, we will expand the knowledge graph and aim to assist human doctors by making the clinical interview process more efficient.