Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Coding (2022)

Abstract: Automatic ICD coding is defined as assigning disease codes to electronic medical records (EMRs).Existing methods usually apply label attention with code representations to match related text snippets. Unlike these works that model the label with the code hierarchy or description, we argue that the code synonyms can provide more comprehensive knowledge based on the observation that the code expressions in EMRs vary from their descriptions in ICD. By aligning codes to concepts in UMLS, we collect synonyms of every code. Then, we propose a multiple synonyms matching network to leverage synonyms for better code representation learning, and finally help the code classification. Experiments on the MIMIC-III dataset show that our proposed method outperforms previous state-of-the-art methods.

## introduction international classification of diseases (icd) is a classification and terminology that provides diagnostic codes with descriptions for diseases 1 . the task of icd coding refers to assigning icd codes to electronic medical records (emrs) which is highly related to clinical tasks or systems including patient similarity learning @xcite , medical billing @xcite , and clinical decision support systems @xcite . traditionally, healthcare organizations have to employ specialized coders for this task, which is expensive, time-consuming, and error-prone. as a result, many methods have been proposed for automatic icd coding since the 1990s (de @xcite . recent methods treat this task as a multi-label classification problem @xcite @xcite , which learn deep representations of emrs with an rnn or cnn encoder and predict codes with a multi-label classifier. recent state-of-the-art methods propose label attention that uses the code representations as attention queries to extract the code-related representations foot_0 @xcite . following this idea, many works further propose using code hierarchical structures @xcite @xcite and descriptions @xcite for better label representations. in this work, we argue that the synonyms of codes can provide more comprehensive information. for example, the description of code 244.9 is "unspecified hypothyroidism" in icd. however, this code can be described in different forms in emrs such as "low t4" and "subthyroidism". fortunately, these different expressions can be found in the unified medical language system (bodenreider, 2004), a repository of biomedical vocabularies that contains various synonyms for all icd codes. therefore, we propose to leverage synonyms of codes to help the label representation learning and further benefit its matching to the emr texts. to model the synonym and its matching to emr text, we further propose a multiple synonyms matching network (msmn) foot_1 . specifically, we first apply a shared lstm to encode emr texts and each synonym. then, we propose a novel multi-synonyms attention mechanism inspired by the multi-head attention @xcite , which considers synonyms as attention queries to extract different code-related text snippets for codewise representations. finally, we propose using a biaffine-based similarity of code-wise text representations and code representations for classification. we conduct experiments on the mimic-iii dataset with two settings: full codes and top-50 codes. results show that our method performs better than previous state-of-the-art methods. ## approach consider free text s (usually discharge summaries) from emr with words {w i } n @xmath0. the task is to assign a binary label y @xmath1, 1} based on s. figure 1 shows an overview of our method. ## related work automatic icd coding is an important task in the medical nlp community. earlier works use machine learning methods for coding @xcite @xcite . with the development of neural networks, many recent works consider icd coding as a multilabel text classification task. they usually apply rnn or cnn to encode texts and use the label attention mechanism to extract and match the most relevant parts for classification. the label attention relies on the label representations as attention queries. li and yu (2020); vu et al. (2020) randomly initialize the label representations which ignore the code semantic information. cao et al. (2020) use the average of word embeddings as label representations to leverage the code semantic information. xie et al. (2019); cao et al. (2020) use gcn to fuse hierarchical structures of icd codes for label representations. compared with previous works, we use synonyms instead of a single description to represent the code, which can provide more comprehensive expressions of codes. biomedical entity linking is a related task to automatic icd coding. the task requires standardizing given terms to a pre-defined concept dictionary. there are two differences between biomedical en-tity linking and automatic icd coding: (1) they have different target concepts. icd coding map emrs to icd codes, while biomedical entity linking usually map terms to a larger dictionary like snomed-ct or umls. (2) they have different input formats. entity linking task has labeled entities in texts, while icd coding only provides texts. synonyms have also been used in biomedical entity linking @xcite . biosyn @xcite uses marginalization to sum the probabilities of all synonyms as the similarity between a term and a concept. however, we consider multi-synonyms attention to extracting different parts of clinical texts to interact with synonyms. ## conclusions in this paper, we propose msmn to leverage code synonyms from umls to improve the automatic icd coding. multi-synonyms attention is proposed for extracting different related text snippets for code-wise text representations. we also propose a biaffine transformation to calculate similarities among texts and codes for classification. experiments show that msmn outperforms previous methods with label attention and achieves state-ofthe-art results in the mimic-iii dataset. ablation studies show the effectiveness of multi-synonyms attention and biaffine-based similarity.