ICDB ig B ird: A Contextual Embedding Model for ICD Code Classification (2022)

Abstract: The International Classification of Diseases (ICD) system is the international standard for classifying diseases and procedures during a healthcare encounter and is widely used for healthcare reporting and management purposes. Assigning correct codes for clinical procedures is important for clinical, operational and financial decision-making in healthcare. Contextual word embedding models have achieved state-of-the-art results in multiple NLP tasks. However, these models have yet to achieve state-of-the-art results in the ICD classification task since one of their main disadvantages is that they can only process documents that contain a small number of tokens which is rarely the case with real patient notes. In this paper, we introduce ICDBigBird a BigBird-based model which can integrate a Graph Convolutional Network (GCN), that takes advantage of the relations between ICD codes in order to create ‘enriched’ representations of their embeddings, with a BigBird contextual model that can process larger documents. Our experiments on a real-world clinical dataset demonstrate the effectiveness of our BigBird-based model on the ICD classification task as it outperforms the previous state-of-the-art models.

## introduction real-world data in healthcare refers to patient data routinely collected during clinic encounters such as visits and hospitalization. after each clinical visit, a set of codes representing diagnostic and procedural information are submitted to various regulatory agencies @xcite . the international classification of diseases (icd) system is the most widely used coding system, maintained by the world health organization @xcite . assigning the most appropriate codes is an important task in healthcare since erroneous icd codes could seriously affect the organization's ability to accurately measure the patient outcome @xcite . contextual word embedding models (such as elmo @xcite and bert @xcite ) have achieved state-of-art results in many nlp tasks. however, recent attempts of using contextual models on the icd classification task have failed to achieve state-of-the-art results @xcite ) mainly due to the fact that they can only process documents that contain a small number of tokens. advances such as the bigbird model @xcite allows contextual models to process long documents, thus reducing the risk of losing information from the original texts. in this paper, we present a novel model for the icd classification task. specifically: ## conclusion and future work we present the icdbigbird model, which is a novel contextual model for the icd coding task. icdbig-bird has the ability to integrate a graph embedding model that takes advantage of the relations between icd codes with a bigbird contextual model that can process larger documents. experiments on the mimic iii have shown that the icdbigbird model outperforms previous state-of-the-art models. as for future work, we plan to address the limitations of this study including (i) testing icdbigbird in other medical datasets to examine its generalizability, strengths and limitations, (ii) experimenting on the task of classifying the full icd code set and (iii) examining the performance of the model in datasets of other languages @xcite .