A Novel Framework Based on Medical Concept Driven Attention for Explainable Medical Code Prediction via External Knowledge (2022)

Abstract: Medical code prediction from clinical notes aims at automatically associating medical codes with the clinical notes. Rare code problem, the medical codes with low occurrences, is prominent in medical code prediction. Recent studies employ deep neural networks and the external knowledge to tackle it. However, such approaches lack interpretability which is a vital issue in medical application. Moreover, due to the lengthy and noisy clinical notes, such approaches fail to achieve satisfactory results. Therefore, in this paper, we propose a novel framework based on medical concept driven attention to incorporate external knowledge for explainable medical code prediction. In specific, both the clinical notes and Wikipedia documents are aligned into topic space to extract medical concepts using topic modeling. Then, the medical concept-driven attention mechanism is applied to uncover the medical code related concepts which provide explanations for medical code prediction. Experimental results on the benchmark dataset show the superiority of the proposed framework over several state-of-the-art baselines.

## introduction medical codes, also known as icd codes, are organized by international classification of diseases (icd, recent versions are icd-9 and icd-10) taxonomies. each medical code corresponds to a disease, procedure or sign, and so on. medical codes can abstract away fine details of free-text clinical notes, which provide great convenience for analyzing clinical data directly @xcite . it is time consuming, costly and error-prone for manual medical coding due to the large menu of options (over 15,000 codes in icd-9) and the complex lengthy clinical notes @xcite . medical code prediction aims at automatically associating the relevant * corresponding author figure 1 : an example of a clinical note annotated with 3-digit icd-9 code "250" and the corresponding wikipedia document, where words in red are medical concept-indicative words which can be employed as evidences to infer medical codes. medical codes with the clinical notes. treating medical code prediction as a multi-label text classification problem, many machine learning based approaches have been proposed including @xcite and support vector machine based @xcite . with the success of deep learning, many researchers propose neural networks with attention mechanism @xcite @xcite to identify representative words in clinical notes and those with large weights serve as evidence for prediction. @xcite utilize convolutional neural networks with several fixed window sizes to capture various medical patterns, then identify representative ones through label attention mechanism. rare code problem, the medical codes with low occurrences, is prominent in medical code prediction. it was pointed out in @xcite that among 942 3-digit icd-9 codes occurring in the mimic-iii database (the largest publicly-available medical database), the least com-mon 437 codes account for only 1% of code occurrences. to tackle the rare code problem, @xcite leverage the code hierarchy and code cooccurrences information to aid predict rare codes. @xcite introduce a hierarchical joint learning architecture using the hierarchical relationships among codes to alleviate the rare code problem. @xcite incorporate the external wikipedia knowledge to enhance semantic information of the rare codes. the matching score between a clinical note and a medical code is calculated based on the code's related wikipedia document. however, most of the approaches mentioned above lack interpretability, which is vital for medical-related tasks. moreover, most of these approaches fail to achieve satisfactory results because of the noisy and lengthy clinical notes (containing an average of 1,596 words). to address these challenges, we propose to explore latent medical concepts (including signs, symptoms, treatments, etc.) related to diseases, hidden in the clinical notes and wikipedia knowledge. as shown in figure 1 , we can identify the informative medical concepts related to 'diabetes mellitus', including signs 'high blood sugar' and 'not enough insulin', typical symptoms 'frequent urination, increased thirst, increased hunger', and typical treatments 'insulin injection' and 'insulin sensitizer' based on the wikipedia document describing icd-9 code "250". obviously, medical concepts mentioned above in clinical notes provide the effective evidences to infer disease 'diabetes mellitus'. moreover, based on the extracted medical concepts, the lengthy and noisy clinical notes can be alleviated. therefore, in this paper, we propose a novel framework based on medical concept driven attention (mcda) to predict medical codes. specifically, both the clinical notes and wikipedia documents are fed as a whole corpus into the topic model to extract medical concepts. both the wikipedia documents and the clinical notes are represented as the distributions over the hidden topics (medical concepts) instead of the lengthy texts. then, the medical concept-driven attention mechanism is applied, consisting of note-specific and label-specific concept-driven attention. on the one hand, the note-specific concept-driven attentions capture the salient medical concepts hidden in a specific clinical note. on the other hand, the labelspecific concept-driven attentions focus on relevant medical concepts in a clinical note for each medical code. experimental results show that the proposed framework outperforms a number of state-of-theart models on a benchmark dataset. the main contributions of this paper are listed as follows: ## related work medical code prediction, also known as automatic icd coding, is a challenging and important task in the limelight of medical informatics community. many traditional machine learning methods have been proposed including @xcite and support vector machine based @xcite . fueled by deep learning, many researchers have proven the effectiveness of convolutional neural network (cnn) and long short-term memory (lstm) for medical code prediction. for example, baumel et al. (2018) apply hierarchical attention networks for predicting medical codes. @xcite propose a cnn with attention mechanism to capture relevant information in source text for each code. to find the specific evidence in the lengthy and noisy text for predicting accurately, researchers use cnn and variants (including multi-filter convolution, dilated convolution) with label attention mechanism to capture codes' relevant text patterns (i.e. n-grams) in clinical notes @xcite @xcite . @xcite focus on label-specific words in notes via lstm with customized label attention mechanism. to tackle the rare code problem, different kinds of knowledge, such as label structure, label co-occurrence statistics, label descriptions and wikipedia are employed. for example, the hierarchical tree structure of icd-9 ontology is firstly exploited by @xcite . @xcite employ graph convolutional network (gcn) to capture the hierarchical relationships among medical codes. @xcite construct a co-graph to incoporate code co-occurrence prior. instead of employing internal knowledge such label structure and label co-occurrence, some external knowledge are incorporated. regarding label descriptions, @xcite apply characteraware neural network to match medical codes and clinical notes. @xcite develop tree lstm to use label descriptions. @xcite train a teacher network with label descriptions and model the code co-occurrence through interactive shared attention. regarding wikipedia, @xcite propose knowledge source integration (ksi) framework to integrate wikipedia documents describing medical codes during training of any baseline models. compared with other external knowledge, wikipedia knowledge is more informative and accessible. regarding incorporating the wikipedia knowledge, the proposed approach is similar to ksi @xcite , but with the following significant differences: (1) we propose medical conceptdriven attention to find note-specific and labelspecific medical concepts in clinical notes as explainable evidences. while ksi simply calculates the matching score between a clinical note and a medical code's related wikipedia document, unable to locate evidences in the context of clinical notes; (2) we make the most of wikipedia knowledge through medical concepts, while ksi only considers the intersection of words in a clinical note and a wikipedia document when predicting the corresponding medical code. ## experiments in this section, we describe the datasets, evaluation metrics, baselines and implementation details, before discussing the experimental results. ## conclusions we have presented a novel framework based on medical concept driven attention for explainable medical code prediction from clinical notes. to the best of our knowledge, our work is the first attempt to uncover and explore latent medical concepts guided by the external knowledge while medical concept-indicative words serve as the evidences for explainable medical code prediction. experimental results show that mcda improves significantly several state-of-the-art models in most evaluation metrics on the benchmark dataset. in future, more wikipedia documents will be incorporated and other ways of incorporating will be explored to promote medical code prediction task.