Description-based Label Attention Classifier for Explainable ICD -9 Classification (2021)

Abstract: ICD-9 coding is a relevant clinical billing task, where unstructured texts with information about a patient’s diagnosis and treatments are annotated with multiple ICD-9 codes. Automated ICD-9 coding is an active research field, where CNN- and RNN-based model architectures represent the state-of-the-art approaches. In this work, we propose a description-based label attention classifier to improve the model explainability when dealing with noisy texts like clinical notes.

## introduction physicians are obliged to thoroughly document every patient encounter. structured and semistructured reports become more common, which contain comprehensive information about performed treatments, procedures and diagnoses. they are typically annotated with multiple billing codes, the international classification of diseases codes (icd-9 in the us, icd-10 in europe). annotating the noisy discharge summaries with icd-9 codes is not only manual and labor-intensive, but also error-prone, which has attracted much attention both from clinical and technical perspectives. to facilitate the clinical workflow, we propose a new approach with state-of-the-art annotation performance while providing an explanation for the proposed annotation. in this work, we propose a description-based label attention classifier (dlac). we show that it can be applied to different transformer-based encoders and provides explainable predictions on noisy texts. dlac learns icd-9 code embeddings by integrating the descriptions of the icd-9 codes and applies the embeddings to the respective text representations to obtain label-specific representations for each code classification. we evaluate different model architectures on the mimic-iii-50 dataset, a benchmark dataset for the task of icd-9 coding, in order to answer the following research questions (rqs). rq1: which transformer-based encoder is best suited for icd-9 coding? we evaluate and compare bert @xcite , hierarchical bert @xcite and longformer @xcite as transformer-based encoders and find that the longformer @xcite yields the best results for icd-9 coding. furthermore, we investigate: rq2: how does our proposed description-based label attention classifier perform on icd-9 coding? we compare dlac with a common logistic regression classifier (lrc) on top of the transformerbased encoder for icd-9 coding. while adding ex-plainability to the predictions, dlac outperforms the corresponding lrc by 1-4 %. since explainable predictions are crucial for noisy texts, like discharge summaries, we investigate: rq3: to which extent can the dlac provide explainable predictions for icd-9 codes? since the attention scores in dlac offer a way to explain the predictions with respect to different text segments, we analyze the top attention scores of dlac and demonstrate the utility as part of a graphical interface. ## methods as a multi-label text classification problem, each discharge summary is represented by a tokenized input sequence of words x i := [x 1 , . . . , x t i ] ∈ r l×t i , where l denotes the vector dimension and t i is the length of the i-th input sequence. the goal is to predict a binary vector y @xmath0, where m represents the set size of icd-9 codes. each element in the predicted vector y is of value 0 or 1. in the following, we denote scalars with lowercase letters like x, vectors with bold lowercase letters like x, and matrices with bold uppercase letters like x. ## experiments in contrast to dlac, lrc does not take the description embeddings d, into account. we set d @xmath1. we train all architectures using adam optimizer with a learning rate of α = 1.41 × 10 -5 and a global batch size of 64. furthermore, we use k-fold cross-validation with @xmath2. for regularization, we use dropout layers with a probability set to @xmath3, 1 and early stopping. we use the widely adopted micro-and macro averaged area under the roc curve (auc), f1 and precision@n as evaluation metrics to ensure comparability with other works. for p@n we choose @xmath4, which is 5, 77 for the mimic-iii-50 dataset. the implementation is made available to ensure the reproducibility of the work foot_0 . ## results and discussions ## conclusion transformer-based architectures show promising performance on the task of icd-9 coding. we find that the longformer encoder is the best suitable encoder architecture for processing long, noisy input sequences such as discharge summaries. we show that our proposed description-based label attention classifier (dlac) can be applied to various transformer-based encoders and the resulting model outperforms a common decoder architecture like logistic regressions by 1-4%. in addition, our proposed dlac is especially suitable for a practical use case when working with fuzzy, long texts such as the discharge summaries, where explainability for the predicted icd-9 codes is necessary.