H yper C ore: Hyperbolic and Co-graph Representation for Automatic ICD Coding (2020)

Abstract: The International Classification of Diseases (ICD) provides a standardized way for classifying diseases, which endows each disease with a unique code. ICD coding aims to assign proper ICD codes to a medical record. Since manual coding is very laborious and prone to errors, many methods have been proposed for the automatic ICD coding task. However, most of existing methods independently predict each code, ignoring two important characteristics: Code Hierarchy and Code Co-occurrence. In this paper, we propose a Hyperbolic and Co-graph Representation method (HyperCore) to address the above problem. Specifically, we propose a hyperbolic representation method to leverage the code hierarchy. Moreover, we propose a graph convolutional network to utilize the code co-occurrence. Experimental results on two widely used datasets demonstrate that our proposed model outperforms previous state-of-the-art methods.

## introduction the international classification of diseases (icd) is a healthcare classification system supported by the world health organization, which provides a unique code for each disease, symptom, sign and so on. icd codes have been widely used for analyzing clinical data and monitoring health issues @xcite . due to the importance of icd codes, icd coding -which assigns proper icd codes to a medical record -has drawn much attention. the task of icd coding is usually undertaken by professional coders according to doctors' diagnosis descriptions in the form of free texts. however, manual coding is very expensive, time-consuming and error-prone. the cost incurred by coding errors and the financial investment spent on improving coding quality are estimated to be $25 billion per year in the us @xcite . two main reasons can account for this. first, only the people who have medical expert knowledge and specialized icd coding skills can handle the task. however, it is hard to train such an eligible icd coder. second, it is difficult to correctly assign proper codes to the input document even for professional coders, because one document can be assigned multiple icd codes and the number of codes in the taxonomy of icd is large. for example, there are over 15,000 and 60,000 codes respectively in the ninth version (icd-9) and the tenth version (icd-10) of icd taxonomies. to reduce human labor and coding errors, many methods have been carefully designed for automatic icd coding @xcite . for example in figure 1 , given the clinical text of a patient, the icd coding model needs to automatically predict the corresponding icd codes. the automatic icd coding task can be modeled as a multi-label classification task since each clinical text is usually accompanied by multiple codes. most of the previous methods handle each code in isolation and convert the multi-label problem into a set of binary classification problems to predict whether each code of interest presents or not @xcite . though effective, they ignore two important characteristics: code hierarchy and code co-occurrence, which can be leveraged to improve coding accuracy. in the following, we will introduce the two characteristics and the reasons why they are critical for the automatic icd coding. code hierarchy: based on icd taxonomy, icd codes are organized under a tree-like hierarchical structure as shown in figure 2 , which indicates the parent-child and sibling relations between codes. in the hierarchical structure, the upper level nodes represent more generic disease categories and the lower level nodes represent more specific diseases. the code hierarchy can capture the mutual exclusion of some codes. if code x and y are both children of z (i.e., x and y are the siblings), it is unlikely to simultaneously assign x and y to a patient in general @xcite . for example in figure 2 , if code "464.00 (acute laryngitis without mention of obstruction)" is assigned to a patient, it is unlikely to assign the code "464.01 (acute laryngitis with obstruction)" to the patient at the same time. if automatic icd coding models ignore such a characteristic, they are prone to giving inconsistent predictions. thus, a challenging problem is how to model the code hierarchy and use it to capture the mutual exclusion of codes. code co-occurrence: since some diseases are concurrent or have a causal relationship with each other, their codes usually co-occur in the clinical text, such as "997.91 (hypertension)" and "429.9 (heart disease)". in this paper, we call such characteristic code co-occurrence which can capture the correlations of codes. the code co-occurrence can be utilized to correctly predict some codes which are difficult to predict by only using the clinical text itself. for example in figure 1 , the code of "acute respiratory failure" can be easily inferred via capturing apparent clues (i.e., the green bold words) from the text. although there are also a few clues to infer the code of "acidosis", they are very obscure, let alone predict the code of "acidosis" by only using these obscure clues. fortunately, there is a strong association between these two diseases: one of the main causes of "acidosis" is "acute respiratory failure". this prior knowledge can be captured via the fact that the codes of the two diseases usually co-occur in clinical texts. by considering the correlation, the automatic icd coding model can better exploit obscure clues to predict the code of "acidosis". therefore, another problem is how to leverage code co-occurrence for icd coding. in this paper, we propose a novel method termed as hyperbolic and co-graph representation method (hypercore) to address above problems. since the tree-likeness properties of the hyperbolic space make it more suitable for representing symbolic data with hierarchical structures than the euclidean space @xcite , we propose a hyperbolic representation learning method to learn the code hierarchy. meanwhile, the graph has been proved effective in modeling data correlation and the graph convolutional network (gcn) enables to efficiently learn node representation @xcite . thus, we devise a code co-occurrence graph (co-graph) for capturing code co-occurrence and exploit the gcn to learn the code representation in the co-graph. the contributions of this paper are threefold. firstly, to our best knowledge, this is the first work to propose a hyperbolic representation method to leverage the code hierarchy for automatic icd coding. secondly, this is also the first work to utilize a gcn to exploit code co-occurrence correlation for automatic icd coding. thirdly, experiments on two widely used automatic icd coding datasets show that our proposed model outperforms previous state-of-the-art methods. though effective, they ignore the code hierarchy and code co-occurrence. hyperbolic representation. hyperbolic space has been applied to modeling complex networks @xcite . recent research on representation learning demonstrates that the hyperbolic space is more suitable for representing symbolic data with hierarchical structures than the euclidean space @xcite @xcite . in the field of natural language processing (nlp), the hyperbolic representation has been successfully applied to question answering @xcite , machine translation @xcite and sentence representation @xcite . to our knowledge, this is the first work to apply hyperbolic representation method to the automatic icd coding task. graph convolutional networks. gcn (kipf and welling, 2016) is a powerful neural network, which operates on graph data. it yields substantial improvements over various nlp tasks such as semantic role labeling @xcite , multi-document summarization @xcite and machine translation @xcite . @xcite propose graph atten-tion networks (gat) to summarize neighborhood features by using masked self-attentional layers. we are the first to capture the code co-occurrence characteristic via the gcn for the automatic icd coding task. ## method we propose a hyperbolic and co-graph representation (hypercore) model for automatic icd coding. firstly, to capture the code hierarchy, we learn the code hyperbolic representations and measure the similarities between document and codes in the hyperbolic space. secondly, to exploit code cooccurrence, we exploit the gcn to learn code cooccurrence representations and use them as query vectors to obtain code-aware document representations. finally, the document-code similarity scores and code-aware document representations are then aggregated to predict the codes. figure 3 shows the overall architecture of our proposed model. ## conclusion in this paper, we propose a novel hyperbolic and cograph representation framework for the automatic icd coding task, which can jointly exploit code hierarchy and code co-occurrence. we exploit the hyperbolic representation learning method to leverage the code hierarchy in the hyperbolic space. moreover, we use the graph convolutional network to capture the co-occurrence correlation. experimental results on two widely used datasets indicate that our proposed model outperforms previous state-ofthe-art methods. we believe our method can also be applied to other tasks that need to exploit hierarchical label structure and label co-occurrence, such as fine-grained entity typing and hierarchical multi-label classification.