Multi-label Diagnosis Classification of S wedish Discharge Summaries – ICD -10 Code Assignment Using KB - BERT (2021)

Abstract: The International Classification of Diseases (ICD) is a system for systematically recording patients’ diagnoses. Clinicians or professional coders assign ICD codes to patients’ medical records to facilitate funding, research, and administration. In most health facilities, clinical coding is a manual, time-demanding task that is prone to errors. A tool that automatically assigns ICD codes to free-text clinical notes could save time and reduce erroneous coding. While many previous studies have focused on ICD coding, research on Swedish patient records is scarce. This study explored different approaches to pairing Swedish clinical notes with ICD codes. KB-BERT, a BERT model pre-trained on Swedish text, was compared to the traditional supervised learning models Support Vector Machines, Decision Trees, and K-nearest Neighbours used as the baseline. When considering ICD codes grouped into ten blocks, the KB-BERT was superior to the baseline models, obtaining an F1-micro of 0.80 and an F1-macro of 0.58. When considering the 263 full ICD codes, the KB-BERT was outperformed by all baseline models at an F1-micro and F1-macro of zero. Wilcoxon signed-rank tests showed that the performance differences between the KB-BERT and the baseline models were statistically significant.

## introduction there are both administrative and statistical purposes of icd coding. administrative to reimburse the clinical unit or hospital, but also to plan healthcare. the codes are assigned by both treating physicians and designated coders. the current version of the icd system, icd-10, contains tens of thousands of codes divided into 22 chapters @xcite . icd coding is time-consuming and error-prone, either missing the main diagnosis or displaying errors in the coding in up to 20 per cent of the patient records @xcite . therefore, it would be valuable to have a supporting tool to assist the physician or coder in choosing among the codes. in this article, swedish patient records in the medical speciality of gastrointestinal surgery and their already assigned icd-10 codes are used to perform supervised learning to predict icd-10 codes. more specifically, the part of the patient records that summarises the patient's care period at the time of the discharge, the discharge summaries, and their associated icd-10 codes are used. the assigned codes belong to the swedish version of the icd-10 system known as icd-10-se @xcite . the codes considered are both full icd codes at the highest level of granularity and the full codes grouped into ten blocks. the research question is how the deep learning language model kb-bert, compared to the traditional supervised learning models support vector machines, decision trees, and k-nearest neighbours performs in pairing discharge summaries with the correct icd codes. since 2007, icd classification has shifted away from rule-based techniques, and many recent studies use traditional supervised learning methods or deep learning approaches. examples of conventional models used in previous icd coding papers are support vector machines, decision trees, knearest neighbours, naïve bayes, and ensembles of these models. in @xcite , comparing these conventional models with multi-layer perceptrons resulted in decision trees and ad-aboost using decision trees being the superior classifiers at f-scores of approximately 0.9. @xcite compared traditional models with convolutional neural networks and concluded that the results for the convolutional neural networks were comparable with the results of support vector machines, but that support vector machines outperformed convolutional neural networks as the number of classes increased. the best-achieved accuracy score of support vector machines in @xcite was 0.75. for bulgarian, @xcite carried out icd classification using support vector machines. she used 6,200 and 1,300 electronic patient records for training and evaluation, respectively, obtaining a precision of 0.97, a recall of 0.74, and an f-score of 0.85. an increasingly popular deep learning approach to icd coding tasks is using the language model bert (bidirectional encoder representations from transformers) developed in 2018 by @xcite . bert was pre-trained on 3.3 billion words from two english corpora -the bookscorpus and the english wikipedia -making it an expert in understanding general english. however, bert has also been adapted to domain-specific language. for example, @xcite developed biobert -a bert model pre-trained on biomedical texts. when @xcite adopted biobert to perform icd coding, they reached an f 1 -micro score of 0.73. moreover, a bert model pre-trained on clinical text was developed by @xcite and used by @xcite for icd classification, achieving an f 1 -score of 0.75. since the original bert and many of its domainspecific adaptations are trained on english texts, bert has also been adapted to understand other languages. in 2020, the national library of sweden pre-trained a bert model on billions of swedish words, naming this model kb-bert, @xcite . @xcite showed that kb-bert outperformed the multilingual version of the bert model. as discussed in a review paper by @xcite , it is difficult to compare previous icd classification approaches since previous studies using the techniques apply them differently. previous studies use different label sets, evaluate the classifiers' performance differently, and use texts written in different languages. therefore, it is favourable to investigate the alternative icd coding methods in the context they will be used. moreover, while many studies explore rule-based methods, traditional supervised models, and deep learning approaches to solve icd coding tasks, few studies use swedish data. @xcite attempted automatic icd classification on swedish data using cooccurrences of words and icd codes. pairing clinical notes with semantically correlated icd codes resulted in the correct icd code being present in the top ten suggested icd codes in 20 per cent of the cases. when considering codes at a lower level of granularity, the correct icd codes were found in the top 10 suggested codes in 77 per cent of the cases. optimising the dimensionality improved these results by 18 percentage points @xcite . this study explores conventional supervised learning methods and the deep learning swedish model kb-bert for swedish icd classification. the f 1 -micro is used to evaluate the different approaches. the f 1 -macro is also presented. a discussion of the results follows, addressing the implications of the study.