Learning to Generate Explanation from e-Hospital Services for Medical Suggestion (2022)

Abstract: Explaining the reasoning of neural models has attracted attention in recent years. Providing highly-accessible and comprehensible explanations in natural language is useful for humans to understand model’s prediction results. In this work, we present a pilot study to investigate explanation generation with a narrative and causal structure for the scenario of health consulting. Our model generates a medical suggestion regarding the patient’s concern and provides an explanation as the outline of the reasoning. To align the generated explanation with the suggestion, we propose a novel discourse-aware mechanism with multi-task learning. Experimental results show that our model achieves promising performances in both quantitative and human evaluation.

## introduction neural models have shown remarkable success in various tasks, however, simply offering the predictions may not satisfy the requirement of end-users. understanding how the decision has been reached by the model is essential in real-world applications. to provide a meaningful, human-comprehensible explanation, presenting it in natural language is a proper fashion. note that simply present the explanation as a shopping list or fragments of texthighlight is not an ideal way. humans prefer to read a text composed of a narrative structure, organized by the discourse relations elaborating the causality between the model input and output @xcite . in this work, we propose a novel health consultancy model which can provide medical suggestions accompanied with natural language explanations learned from medical specialists to help humans make decisions in their daily life. as many people are eager for help in addressing their health concerns, a model is necessitated to be capable of not only providing suggestions regarding their concerns, but also explaining the suggestions, alleviating their worries before visiting male / (20~29), 2021/02/25 / ask： i went to the er in april after having a fever, the initial diagnosis is urethritis. the doctor gave me antibiotics and antipyretics, however, my temperature didn't stepped down and i went to another hospital for examinations (a blood test and urine test). the results came back normal but the temperature kept running high. how come i am still having a fever? are there any further examinations i should take? oo hospital ooo m.d. / internal medicine department / 2021/02/25 / response： it seems that you have been feeling feverishness for over a month. to evaluate the possibility that the fever is not caused by infections and assess the need for further referrals, i suggest that you make an appointment with doctors of infectious disease department. figure 1 : a data instance from the health consultancy website the doctors. taking figure 1 as an example, given a question asked by a patient, the physician answers it by explaining what disease might cause the symptoms mentioned in the question and suggests which medical specialty the patient should seek. in this example, the response has a clear narrative structure, where the explanation and suggestion are denoted by the yellow and green highlights, respectively. first, the patient's concern is addressed (e.g., having a fever for over a month). then, before giving the suggestion, the physician explains the causality between the concerns and the suggestion. recently, explainable artificial intelligence (xai), which aims at explaining how the decision is reached by the machine learning model @xcite @xcite , has been gaining attentions in the research community, including works that provide textual explanations @xcite @xcite . generally, generating textual explanation can be regarded as a natural language generation task. typically, most works collect explanations by asking annotators to write free-text sentences. since human annotation is expensive, labor-intensive, and time-consuming, especially in domains where expertise is needed, an alternative way is to construct synthesized explanations by designing rules to exploit information from other datasets as explanations @xcite . although previous works demonstrate that introducing an auxiliary generation task to explain the prediction enables performance improvement, two main issues remain to be tackled: (1) the explanations are annotation artifacts @xcite since the annotators typically write vanilla and trivial description @xcite , resulting in a lack of linguistic variety @xcite ) that leads the model prone to overfit on annotator characteristics @xcite . (2) whether the model faithfully explains the suggestion is still an open question @xcite . producing an explanation just mimicking the way humans would say is impractical in fields involving high-stake scenarios. to address the aforementioned two issues, our work is based on real-world data. we collect 86,399 question answering (qa) pairs foot_0 from an online health consultancy website called taiwan e-hospital, foot_1 which allows users to ask questions regarding their health conditions and physicians will respond to their concerns. the linguistic diversity from multiple users is greater than free-text produced by a few crowd-workers, and the explanation within an answer is more natural than handcrafted annotation. then we propose pilot models to generate response consisting of explanation and suggestion according to the question. note that our approach can be easily generalized since it is language and domain independent. the contributions of our work are summarized as follows: ## dataset as mentioned in section 1, we construct our dataset by crawling approximately 86k qa pairs from tai-wan e-hospital website, where an answer is a freetext response written by the physician, containing the explanation and suggestion as figure 1 shows. an ideal instance would be a triple of (q, s, e), where q is the question asked by the patient, and s and e are the suggestion and the explanation responded by the physician. however, the crawled raw text often carries greeting terms, salutations, and personal information, such as names of the patients and doctors, which are noise for our task and should be pruned. to gather the desired (q, s, e), we propose a rule-based keyword matching method to extract text snippets that belong to the suggestion and explanation, defined as follows. the details of our method are shown as follows. step1: we define a set of keyphrases g that belongs to greeting terms or salutations by regular expressions. given a sequence of sentences @xmath0, x 2 , ..., x n ) in the response, the i-th sentence x i that contains a word @xmath1. afterwards, a sequence of sentences x ′ is obtained, where step2: we find that the sentences belonging to suggestions usually contain certain keywords such as "suggest", "recommend", and the name of the department to "seek". hence, to identify whether a sentence @xmath2, we manually collect a set of keywords k from several responses. then, the sentences in x ′ that contain a word @xmath3, and the remaining sentences are considered as the explanation. step3: in addition to preparing the (q, s, e) triples, we also construct binary labels (0/1) from s, denoting whether the patient should receive medical assistance based on his/her health condition. taking figure 1 as an example, it would be labeled as 1 since the doctor suggests the user to make an the processed result from each step, namely r 1 , r 2 , and r 3 , is utilized as references for our different proposed methods described in section 3, i.e., mt5 (3.1), mtl mt5 (3.2), and dmtl mt5 (3.3), respectively. the details of the dataset formats are shown in to validate the results of our regular expressions in steps 1, 2, and 3, we randomly sample 100 instances from r 1 , r 2 , and r 3 , respectively. and by checking the correctness of these instances with human evaluation described as follows, we can assess the quality of the regular expressions. for steps 1 and 2, an instance is considered incorrect if it contains sentences that should be filtered or missing sentences that should be retained. that is, the regular expression admits or filters the wrong sentences. otherwise, it is considered as correct. for step 3, the correctness of an instance is determined by checking if the binary label l is the same as whether the physician suggests the patient to seek medical attention or not. the results are measured by # correct instances 100 and presented in ## methodology in this section, we introduce our models for the task of suggestion and explanation generation. as the references of suggestions and explanations are collected by handcrafted rules without human annotation, the models are weakly supervised. given a question q, our goal is to learn a generator gen(•) to generate a textual response with a narrative structure consisting of a suggestion s and an explanation e. three gen(•) models are described as follows. ## experiments and discussions we conduct both quantitative and qualitative evaluations to compare the generated suggestions and explanations of our proposed methods. the dataset are randomly split into train, validation, and test sets by the ratio 8:1:1 (69,119, 8,640, 8,640), where every instance is a qa pair. and we adopt teacherforcing strategy @xcite with the cross-entropy loss as the objective function for optimizing all models, i.e., mt5, mtl mt5 and dmtl mt5. the results reported in this section are conducted on the test set. ## conclusion this paper proposes a discourse-aware generative model based on multi-task learning to generate narrative structured responses consisting of suggestions and explanations to the questions. experimental results show that our model with the discourse-aware mechanism outperforms baseline models on both quantitative and qualitative evaluations. however, based on the human evaluation results, there is still ample room for improvement on providing medical explanations. as the correctness of explanation is still relatively lower than our expectation. on the other hand, without integrating explicit medical knowledge, there exists potential risks of producing unfaithful results. in the future, we plan to incorporate external domain knowledge, e.g., medical knowledge base, into the model to generate enriched and faithful explanations that are not only relevant to suggestions, but also contain correct information.