Application of Deep Learning Methods to SNOMED CT Encoding of Clinical Texts: From Data Collection to Extreme Multi-Label Text-Based Classification (2021)

Abstract: Concept normalization of clinical texts to standard medical classifications and ontologies is a task with high importance for healthcare and medical research. We attempt to solve this problem through automatic SNOMED CT encoding, where SNOMED CT is one of the most widely used and comprehensive clinical term ontologies. Applying basic Deep Learning models, however, leads to undesirable results due to the unbalanced nature of the data and the extreme number of classes. We propose a classification procedure that features a multiple-step workflow consisting of label clustering, multi-cluster classification, and clusters-to-labels mapping. For multi-cluster classification, BioBERT is fine-tuned over our custom dataset. The clusters-to-labels mapping is carried out by a one-vs-all classifier (SVC) applied to every single cluster. We also present the steps for automatic dataset generation of textual descriptions annotated with SNOMED CT codes based on public data and linked open data. In order to cope with the problem that our dataset is highly unbalanced, some data augmentation methods are applied. The results from the conducted experiments show high accuracy and reliability of our approach for prediction of SNOMED CT codes relevant to a clinical text.

## introduction the task of automatic encoding of clinical text with standard medical classifications and ontologies is with high importance for healthcare organizations and medical research. truly, more than 80% of clinical documents are stored in free-text format. this paper presents a research effort in solving the problem of automatic encoding of textual description of medical diagnoses with one of the most widely used @xcite and comprehensive ontologies -the systematized nomenclature of medicine -clinical terms (snomed ct) 1 . one of the most important characteristics of snomed ct, which makes it significantly different from the rest of the standard medical classifications, is that it is based on compositional grammar foot_1 . another aspect of popularity and importance of snomed ct for health information is interoperability, that is discussed in @xcite . snomed ct is well known for being one of the most comprehensive medical ontologies, which makes the task of automatic encoding an extreme scale classification task with more than 360,000 medical codes. currently, this task has not been solved with sufficient accuracy for all possible classes. usually the developed solutions cover restricted terminology from 10 to a couple of thousands terms @xcite . the compositional nature of the snomed ct codes gives us the opportunity to address the problem either with classical approaches for classification tasks or with specific solutions that benefit from the compositional grammar's structure. as a product of our research, the developed service for automatic encoding with snomed ct codes will be used mainly for electronic health records (ehr) processing for patients with oncological diseases and certain rare diseases. most of these diseases are well known to have a huge number of related diseases. thus, our study will not be restricted only to the diseases of interest but will have a much broader scope. we propose an adaptation of the approach proposed by @xcite and demonstrate the entire process from training dataset construction to classification model design and training. ## related work the problem of automatic encoding of ehr with snomed ct codes was investigated by many researchers since the very beginning of the ontology development. different solutions cover broad range of snomed ct codes from 10 to a couple of thousands, usually the main obstacle for scalabily is the availability of sufficient volume of annotated training data. basaldella et al @xcite present cometa -manually annotated corpora by experts that contain 20k english biomedical entities encoded with snomed ct. the most popular approaches for automatic encoding, include hybrid methods combining regular expressions and vector space models @xcite with top precision 0.823 and mean avg precision 0.45 for 1239 medline citations. some approaches take in consideration compositional structure @xcite of snomed ct. recent research is based on deep learning techniques, and the most promising solutions are using transformers like bert @xcite . pattisapu et al @xcite apply word embeddings, graph embeddings and bert derivatives transformers and achieve the highest accuracy 0.83 for two benchmark datasets cadec and psytar. kraljevic et al @xcite propose medcat with macro f1: 0.841-0.860 across different clinical domains and tasks. medcat is based on word2vec embeddings, and there is also medcat bert version based on clinicalbert @xcite , and latter model shows a little bit lower performance than the former one. a recent systematic review @xcite shows that only few of the developed services for automatic encoding with snomed ct, are provided as open source -the clinical text analysis and knowledge extraction system (ctakes) @xcite and metamap @xcite . both of them are rule-based. ## data one of the key factors that plays a role in the automatic encoding of snomed ct codes is the data. in our project, we do not have annotated data which can be used to train the developed models. thus, we use certain public data and linked open data in order to automatically generate annotated corpora that can serve as a training dataset. ## conclusion and further work we demonstrated how can be generated annotated dataset with snomed ct codes. the proposed approach demonstrates high accuracy and scalability. in comparison with other state-of-the-art approaches the achieved accuracy for the proposed model is relatively high and more over for wider coverage of snomed ct. our further work includes training of multilingual bert to solve the multilingual problem. possible increase of the area under the curve (roc auc) scores can be achieved through grid search applied to the selection of the k-means clusters number, until finding the optimal number, based on the distribution of the data.