Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain Text (2020)

Abstract: Machine reading comprehension (MRC) has achieved significant progress on the open domain in recent years, mainly due to large-scale pre-trained language models. However, it performs much worse in specific domains such as the medical field due to the lack of extensive training data and professional structural knowledge neglect. As an effort, we first collect a large scale medical multi-choice question dataset (more than 21k instances) for the National Licensed Pharmacist Examination in China. It is a challenging medical examination with a passing rate of less than 14.2% in 2018. Then we propose a novel reading comprehension model KMQA, which can fully exploit the structural medical knowledge (i.e., medical knowledge graph) and the reference medical plain text (i.e., text snippets retrieved from reference books). The experimental results indicate that the KMQA outperforms existing competitive models with a large margin and passes the exam with 61.8% accuracy rate on the test set.

## introduction with the advent of large scale datasets such as squad @xcite , race @xcite , and natural questions @xcite on the open domain, machine reading comprehension (mrc) has become a hot topic in the natural language processing field. in the past few years, the mrc has obtained substantial progress, and many recent models have surpassed the human performance on several datasets. the superiority of these models is mainly attributed to two significant aspects: 1) the powerful representations ability of large pre-trained language models (plms), which can cover or remember most of the language variations implicitly. ## related work medical question answering the medical domain poses a challenge to existing approaches since the questions may be more challenging to answer. bioasq @xcite is one of the most significant community efforts made for advancing biomedical question answering (qa) systems. seareader @xcite is proposed to answer questions in clinical medicine using documents extracted from publications in the medical domain. yue et al. (2020) conduct a thorough analysis of the emrqa dataset @xcite and explore the ability of qa systems to utilize clinical domain knowledge and to generalize to unseen questions. @xcite introduce pubmedqa where questions are derived based on article titles and can be answered with its respective abstracts. recently, pre-trained models have been introduced to medical domain @xcite @xcite . they are trained on unannotated biomedical texts such as pubmed abstracts and have been proven useful in biomedical question answering. in this paper, we focus on multiple choice problems in medical exams that are more difficult and diverse, which allows us to directly explore the capabilities of qa models to encode domain knowledge. knowledge enhanced methods kagnet @xcite represents external knowledge as a graph, and then uses graph convolution and lstm for inference. @xcite adopt the bertbased option comparison network (ocn) for answer prediction, and propose an attention mechanism to perform knowledge integration using relevant triples. @xcite propose a gnn-based inference model on conceptual network relationships and heterogeneous graphs of wikipedia sentences. bert-mk @xcite integrates fact triples in the kg, while realm @xcite augments language model pre-training algorithms with a learned textual knowledge retriever. unlike previous works, we incorporate external knowledge implicitly and explicitly. built upon pre-trained models, our work combines the strengths of both text and medical knowledge representations. ## method the medical mrc task in this paper is a multiplechoice problem with five answer candidates. it can be formalized as follows: given the question q and answer candidates {o i }, the goal is to select the most plausible correct answer ô from the candidates. kmqa utilizes textual evidence spans and incorporates knowledge graphs facts for medical multi-choice question answering. as shown in figure 1 , it consists of several modules: (a) the multi-level co-attention reader that computes context-aware representations for the question, options and retrieved snippets, and enables rich interactions among their representations. (b) the knowledge acquisition which extracts knowledge facts from kg given the question and options. (c) the injection layer that further incorporates knowledge facts into the reader, and (d) a prediction layer that outputs the final answer. and also, we utilize the relational structures of question-tooptions paths to further augment the performance of kmqa. ## conclusion in this work, we explore how to solve multi-choice reading comprehension tasks in the medical field based on the examination problems of licensed pharmacists, and propose a novel model kmqa. it explicitly combines knowledge and pre-trained models into a unified framework. moreover, kmqa implicitly takes advantage of factual information via learning from an intermediate task and also transfers structural knowledge to enhance entity representation. on the test set from the real world, the kmqa is the single model that outperforms the human pass line. in the future, we will explore how to apply our model to more domains, and enhance the interpretability of the reasoning path when the model answers questions.