Clinical Note Owns its Hierarchy: Multi-Level Hypergraph Neural Networks for Patient-Level Representation Learning (2023)

Abstract: Leveraging knowledge from electronic health records (EHRs) to predict a patient’s condition is essential to the effective delivery of appropriate care. Clinical notes of patient EHRs contain valuable information from healthcare professionals, but have been underused due to their difficult contents and complex hierarchies. Recently, hypergraph-based methods have been proposed for document classifications. Directly adopting existing hypergraph methods on clinical notes cannot sufficiently utilize the hierarchy information of the patient, which can degrade clinical semantic information by (1) frequent neutral words and (2) hierarchies with imbalanced distribution. Thus, we propose a taxonomy-aware multi-level hypergraph neural network (TM-HGNN), where multi-level hypergraphs assemble useful neutral words with rare keywords via note and taxonomy level hyperedges to retain the clinical semantic information. The constructed patient hypergraphs are fed into hierarchical message passing layers for learning more balanced multi-level knowledge at the note and taxonomy levels. We validate the effectiveness of TM-HGNN by conducting extensive experiments with MIMIC-III dataset on benchmark in-hospital-mortality prediction.

## introduction with improvement in healthcare technologies, electronic health records (ehrs) are being used to monitor intensive care units (icus) in hospitals. since it is crucial to schedule appropriate treatments for patients in icus, there are many prognostic models that use ehrs to address related tasks, such as inhospital mortality prediction. ehrs consist of three types of data; structured, semi-structured, and unstructured. clinical notes, which are unstructured data, contain valuable comments or summary of the patient's condition written by medical professionals (doctors, nurses, etc.). however, compared to structured data, clinical notes have been underutilized in previous studies due to the difficult-to-understand contents and the complex hierarchies (figure 1(a) ). transformer-based @xcite methods like clinicalbert @xcite @xcite have been proposed to pretrain on large-scale corpus from similar domains, and fine-tune on the clinical notes through transfer learning. while transformer-based methods can effectively detect distant words compared to other sequence-based methods like convolutional neural networks @xcite and recurrent neural networks @xcite @xcite , there are still limitations of increasing computational complexity for long clinical notes (figure 2 ). recently, with the remarkable success of the graph neural networks (gnns) @xcite @xcite , graph-based document classification methods have been proposed @xcite that can capture long range word dependencies and can be adapted to documents with different and irregular lengths. some methods build word co-occurrence graphs by sliding fixed-size windows to model pairwise interactions between words @xcite @xcite . however, the density of the graph increases as the document becomes longer. besides, there are also some methods apply hypergraph for document classification @xcite , which can alleviate the high density of the document graphs and extract high-order structural information of the documents. adopting hypergraphs can reduce burden for managing long documents with irregular lengths, but additional issues remain when dealing with clinical notes: (1) neutral words deteriorate clinical semantic information. in long clinical notes, there are many frequently written neutral words (e.g. "rhythm") that do not directly represent the patient's condition. most of the previous methods treat all words equally at the learning stage, which may result in dominance of frequent neutral words, and negligence of rare keywords that are directly related to the patient's condition. meanwhile, the neutral word can occasionally augment information of rare keywords, depending on the intra-taxonomy context. taxonomy represents the category of the clinical notes, where implicit semantic meaning of the words can differ. for example, "rhythm" occurred with "fibrillation" in ecg taxonomy can represent serious cardiac disorder of a patient, but when "rhythm" is written with "benadryl" in nursing taxonomy, it can hardly represent the serious condition. therefore, assembling intra-taxonomy related words can leverage "useful" neutral words with rare keywords to jointly augment the clinical semantic information, which implies the necessity of introducing taxonomy-level hyperedges. ## results since the dataset has imbalanced class labels for in-hospital mortality as shown in ## conclusion in this paper, we propose a taxonomy-aware multilevel hypergraph neural networks, tm-hgnn, a novel approach for patient-level clinical note representation learning. we employ hypergraph-based approach and introduce multi-level hyperedges (note and taxonomy-level) to address long and complex information of clinical notes. tm-hgnn aims to extract high-order semantic information from the multi-level patient hypergraphs in hierarchical order, note-level and then taxonomy-level. clinical note representations can be effectively learned in an end-to-end manner with tm-hgnn, which is validated from extensive experiments.