Medical knowledge-enhanced prompt learning for diagnosis classification from clinical text (2023)

Abstract: Artificial intelligence based diagnosis systems have emerged as powerful tools to reform traditional medical care. Each clinician now wants to have his own intelligent diagnostic partner to expand the range of services he can provide. When reading a clinical note, experts make inferences with relevant knowledge. However, medical knowledge appears to be heterogeneous, including structured and unstructured knowledge. Existing approaches are incapable of uniforming them well. Besides, the descriptions of clinical findings in clinical notes, which are reasoned to diagnosis, vary a lot for different diseases or patients. To address these problems, we propose a Medical Knowledge-enhanced Prompt Learning (MedKPL) model for diagnosis classification. First, to overcome the heterogeneity of knowledge, given the knowledge relevant to diagnosis, MedKPL extracts and normalizes the relevant knowledge into a prompt sequence. Then, MedKPL integrates the knowledge prompt with the clinical note into a designed prompt for representation. Therefore, MedKPL can integrate medical knowledge into the models to enhance diagnosis and effectively transfer learned diagnosis capacity to unseen diseases using alternating relevant disease knowledge. The experimental results on two medical datasets show that our method can obtain better medical text classification results and can perform better in transfer and few-shot settings among datasets of different diseases.

## introduction clinical notes in electronic health records (ehrs) are the medical texts written by a physician to ad-dress the patient's medical history, chief complaints and examinations during a patient's visit. physicians can get the corresponding diagnosis through their expertise based on the patient's clinical notes. in the past decade, researchers have tried various methods for medical text classification tasks to assist doctors in their treatment. text classification models in the generic domain are developing most rapidly. traditional machine learning methods, such as naive bayesian (nb) @xcite , k-nearest neighbor (knn) @xcite , support vector machine (svm) @xcite , and random forest (rf) @xcite are first introduced to solve text classification tasks. for deep learning models, textcnn @xcite is widely used, where convolutional neural network (cnn) @xcite models are introduced to solving text classification problems. whereafter, pre-trained language models (plms), such as bert @xcite achieve state-of-the-art results on several natural language processing (nlp) tasks and thus has been widely used. however, these approaches are based on generic data and therefore ignore the high reliance on medical knowledge in medical text classification tasks. when applied directly to the medical field, these models often fail to achieve the same performance as in the generic field. to address the knowledge-dependent medical text classification tasks, researchers have proposed a number of medical text classification models that incorporate knowledge. @xcite map medical text to corresponding medical concepts and is the first to conduct feature engineering. @xcite use medical concept descriptions to improve distributed document representations. @xcite use external terminology resources to expand and represent the text with a combination of different methods. nevertheless, these models only learn the relationship between the text and the corresponding knowledge, without having a good generalization ability. therefore they tend to be less effective when transferring to the medical domains beyond the training data. in the medical field, there are rich sources of knowledge, such as expert knowledge @xcite , medical knowledge bases @xcite , medical knowledge graphs @xcite , medical information on the web, etc. these knowledge present a heterogeneous structure(such as triples, sqls and free texts, etc.) and cannot be well uniformed in the previous methods. differences among knowledge sources prevent these models from learning by using knowledge prompt from all sources and thus may have bias when dealing with real-world data. therefore, we hope to propose a model that is compatible with all the sources of medical knowledge. to solve the above problems in medical text classification, we propose a medical knowledgeenhanced prompt learing (medkpl) model that can uniform different knowledge sources. the contribution of this paper can be summarized as follows: 1) we design the medkpl model to uniform heterogeneous knowledge by transforming knowledge from different sources into free texts. experiments prove that structured and unstructured texts can be uniform in our model, and both yield good results. 2) we use the medkpl model to conduct medical text classification tasks on two chinese ehr datasets and obtain state-of-the-art classification results through knowledge incorporation. 3) we evaluate the medkpl model for few-shot learning among departments. the results show that our method can obtain good results in both zero-shot and few-shot scenarios, and can effectively transfer between departments that have low text similarity in a robust way. ## method the overall structure of our model is shown in figure 2. our model introduce disease @xmath0, where d is the disease set. the knowledge prompt can come from a variety of sources, e.g. expert knowledge, knowledge graphs, knowledge bases, online resources, etc. we use p(y|x i , k) to denote the probability of patient i getting disease y, where x i is the clinical notes for patient i, and k is the set of knowledge prompts used for knowledge incorporation. specifically, we decompose the process of knowledge incorporation into three stages. 1) extract medical knowledge of disease d from different knowledge sources and transform the knowledge into a uniform representation k d . 2) construct templates that incorporate knowledge prompts set k with clinical notes. we concatenate the collected medical knowledge prompts into natural text and generate the template based on the disease name d. 3) predict labels using mlm on the [mask] token in prompt template. it is also possible to integrate knowledge prompt and clinical notes at a deep level by using plm to represent knowledge prompt and clinical notes separately and aligning them using soft attention mechanisms to enhance the knowledge representation. we will then go over our model's methodology and its three stages of knowledge incorporation. ## conclusion in this paper, we propose a medkpl model and achieve state-of-the-art classification results on two medical ehr datasets. with the advantage of knowledge extraction and uniform process, our model can eliminate the difference among different sources and organize all knowledge into one representation style. the knowledge incorporation and soft attention mechanism between knowledge prompt and clinical notes enable the model to be more robust and achieve appreciable improvement on medical text classification tasks. the introduction of knowledge and prompt learning method exploits better few-shot and zero-shot transferability among departments.