Divide and Conquer: An Extreme Multi-Label Classification Approach for Coding Diseases and Procedures in S panish (2022)

Abstract: Clinical coding is the task of transforming medical documents into structured codes following a standard ontology. Since these terminologies are composed of hundreds of codes, this problem can be considered an Extreme Multi-label Classification task. This paper proposes a novel neural network-based architecture for clinical coding. First, we take full advantage of the hierarchical nature of ontologies to create clusters based on semantic relations. Then, we use a Matcher module to assign the probability of documents belonging to each cluster. Finally, the Ranker calculates the probability of each code considering only the documents in the cluster. This division allows a fine-grained differentiation within the cluster, which cannot be addressed using a single classifier. In addition, since most of the previous work has focused on solving this task in English, we conducted our experiments on three clinical coding corpora in Spanish. The experimental results demonstrate the effectiveness of our model, achieving state-of-the-art results on two of the three datasets. Specifically, we outperformed previous models on two subtasks of the CodiEsp shared task: CodiEsp-D (diseases) and CodiEsp-P (procedures). Automatic coding can profoundly impact healthcare by structuring critical information written in free text in electronic health records.

## introduction the international classification of diseases (icd) is a medical glossary published by the world health organization, which establishes specific coding rules for healthcare procedures and diseases. mapping electronic health records into alphanumeric codes allows rapid summarization of information, which is necessary to calculate costs, support clinical decisions, and conduct detailed epidemiological studies. however, manual coding is timeconsuming, resource-intensive, and error-prone, even for specialists. for this reason, developing tools to support this task is precious. extreme multi-label classification is a subset of the multi-label classification task in which the objective is to learn feature architectures and classifiers that can automatically tag a data point with the most relevant labels from a huge label set @xcite . the term extreme is justified in this case since the space of possible labels is generally very large and can exceed the number of documents in a given corpus. clinical coding is an important natural language processing (nlp) task that seeks to automatically assign codes to medical documents following a standard terminology, such as the icd glossary. since each document can be labeled with more than one code from an extensive list, this task can be considered an extreme multi-label classification task @xcite . an example of a codiesp-d medical document is shown in figure 1 . despite the availability of clinical coding datasets in spanish, such as cantemist @xcite or codiesp @xcite , the size of these resources is not yet comparable to that for the english language. for example, codiesp has 1,000 clinical case reports, while the mimic-iii dataset @xcite has 52.726 discharge summaries. this scarcity of data forces models in other languages to have different architectures than those trained on english datasets. to fill this gap, we introduce a novel architecture for solving clinical coding on three spanish clinical datasets. this architecture is composed of two modules: the ranker and the matcher. the first module calculates the probability of a document belonging to a cluster, while the second performs the classification of documents into codes. each cluster is created previously by analyzing the ontology of each dataset. our experimental results show the effectiveness of our model, achieving state-ofthe-art performance on codiesp-d (diseases) and codiesp-p (procedures) according to the mean average precision (map) and f 1 score. ## related work in recent years, there has been a growing interest from the nlp research community in studying the clinical coding task. early work focuses on creating machine learning-based classifiers with heavy feature engineering @xcite . however, as mentioned in @xcite and @xcite , recent deep learning advancements have greatly improved clinical coding models' performance for all languages. regarding extreme multi-label architectures, there is one work that heavily inspired this work, which is x-transformers @xcite . it proposes creating a clusterization of labels using the distance between the label descriptions encoded using contextualized embeddings retrieved from transformers' language models. then they predict the clusters using a transformers classifier, and finally, they predict the labels over the subset of predicted clusters using one-vs-rest linear classifiers. the design of this architecture was thought to handle corpora much larger than the ones we have studied in this work, thus prioritizing time efficiency much more. one of the most popular datasets used for clinical coding for the spanish language is codiesp @xcite . most of the work proposed formulated the problem as text classifi-cation. in lópez-garcıa et al. (2020), they used a transformer-based model to classify the sentences of the documents. then, the whole document set of codes is the union of the sentence-level codes. other approaches focused on solving the problem as a named entity recognition (ner) task. in @xcite , they created a dictionary based on entity mentions and code definitions. then, they matched spans of documents with the code definitions in the dictionary using a tree-based algorithm. finally, other ensemble-based models combined text classification and ner tasks to solve the clinical coding problem. for example, @xcite implemented a model that used stringmatching encoders and one-vs-rest document classification. this model obtained the best results in the competition. another important task of clinical coding in spanish is cantemist @xcite , which aims to identify codes present in cancer diagnoses. this task had two winner systems obtaining the same map score. the first model proposed by @xcite used different transformer-based models to predict specific parts of a code. these models were ensembled using a novel voting system. the second winner was @xcite , who reused their approach proposed in codiesp but further pre-trained a language model with a private oncology corpus. recent work by lópez-garcía et al. (2021) outperformed previous models in codiesp and can-temist by a wide margin. first, they trained three multilingual language models using private oncology datasets and then fine-tuned these models for classifying documents into codes. to improve the performance of their models, they ensembled the results from five different instances of each trained transformer. ## model our proposed architecture comprises two main modules: the matcher and the ranker. the first module calculates the probability that a document belongs to some cluster, while the second one calculates the probability of codes in the document. the results of both modules are used to perform the final prediction of codes. this process is carried out by multiplying the probability of codes obtained from the ranker for each document with the code cluster probability obtained from the matcher module. we refer to this approach as the divide and emc code partitioning assigned cluster from a00 to b99 some diseases caused by infections and parasites from c00 to d49 ## results our base model achieves state-of-the-art results in both codiesp tasks, surpassing the best base model @xcite ) by 8% in codiesp diagnostics and by 6% in codiesp procedures. even in comparison with an ensemble of strong learners, which obtains a similar performance (clinical coding transformers -ensemble @xcite ), our base model surpasses their results by a small margin of 0.5% in codiesp diagnostics and 0.2% in codiesp procedures. their results correspond to 15 different runs of 3 strong learners, where each language model was trained with a private oncology corpus. unlike the mentioned work, we used only publicly available resources and a simpler architecture regarding computational cost. most notably, our ensemble-based version of 15 different instances of our model outperformed previous results in the codiesp tasks by a wide margin, outperforming state-of-the-art methods, including ensembles of strong learners, in codiesp-d and codiesp-p by 3.0% and 3.3%, respectively. we hypothesize that the high performance of our model is explained since the original text classification task is reduced to two subtasks, where the number of possible labels is smaller. first, the matcher module performs a text classification in which the number of labels equals the number of clusters. second, the ranker is trained only with documents belonging to a cluster, which allows for a fine-grained differentiation between similar codes. we believe that the incapability to obtain stateof-the-art for cantemist is because it is a simpler task than codiesp-d and codiesp-p. this is noticeable by looking at the performance of every model in each task. our architecture is built to thrive under challenging tasks where a straightforward fine-tuning of a transformer is not the best approach. nonetheless, our architecture is the third best evaluated for cantemist, considering that icb-uma @xcite and clinical coding transformers @xcite are from the same authors and used the same approach. therefore, we obtained competitive results compared to state-of-the-art models and surpassed the performance of most of the systems @xcite . ## module analysis in notably, the map and f 1 scores for the matcher are high in all experiments. this is required for the architecture to be competitive; otherwise, the error propagation leads to a low-quality final model. another interesting finding is that we can see no significant difference between the domain-specific language models (biomedical roberta and bio-clinical) across all our experiments. however, the general-domain language model we have tested (beto) has significantly lower performance on all tasks. finally, it is worth mentioning that the ensemble-based architecture significantly outperforms all base models at hand, at least in the map metric. according to the f 1 metric, it surpasses the models in the codiesp tasks and fails in the cantemist corpus. this adds room for improvement in how the class prediction is combined to calculate this paper proposes a novel model for clinical coding in spanish, outperforming previous results in two datasets; codiesp-d and codiesp-p. our method uses a divide and conquer approach that creates semantic groups of codes to build an architecture composed of two specialized modules: the matcher and the ranker. the clinical coding task is separated into two simpler tasks solved with the modules mentioned above. first, the matcher predicts the clusters of each document, and then the ranker predicts the codes of each document given a cluster. this divi-sion allows us to use state-of-the-art transformers to solve the task of cluster prediction and permits a fine-grained differentiation between similar codes in a cluster using xgboost. although our base model achieves better results than previous ensemble-based models, we included the results of an ensemble strategy to perform a fair comparison with previous work. our experimental results demonstrate that ensembling models yield better results than our base model. furthermore, our dac approach allows us to identify where future research can have a greater impact on improving accuracy. the results of each module indicate that there is more potential for improvement focusing on the ranker module. future directions include implementing and test-ing the divide and conquer model on other multilabel text classification corpora. first, we expect to test the dac architecture on clinical corpora in other languages, including languages with more resources, such as english. second, we expect to test the architecture on other extreme multi-label classification corpora. this poses a challenge since the number of labels we have processed thus far, although very vast, falls into the category of small extreme multi-label classification datasets @xcite . we expect to encounter issues with the training time required to process other large corpora, forcing us to modify the library to optimize the speed. in terms of improving the performance using this architecture, we identify opportunities to optimize the number of layers that we left fine-tuneable in the matcher module, given that we have seen research that shows that fine-tuning more layers provides better results @xcite . also, for the ranker, we know that xgboost can be trained with a ranking objective function, thus providing an alternative to the one-vs-rest approach. implementing the ranker using this approach would be faster to train and may provide similar or better results. in addition, we would like to improve data augmentation techniques by improving ner models. this can be achieved by using contextualized embeddings at the character level, which has been shown to improve the performance of models on various nlp tasks @xcite . finally, the dac architecture is a black box when defining which labels to assign for each document. recently, explainability features of the different architectures are gaining more relevance. it is paramount that the model's predictions are understood to help the user make appropriate choices @xcite . we expect to develop explainability to the labels predicted by providing textual queues of what features the model used to choose each label. the textual queues that support label assignment can be provided by the ranker leveraging the explainability features of tree ensembles @xcite , and the textual queues that support the cluster choice can be obtained using the attention weights of the transformer model @xcite .