Medical Concept Normalization in User-Generated Texts by Learning Target Concept Embeddings (2020)

Abstract: Medical concept normalization helps in discovering standard concepts in free-form text i.e., maps health-related mentions to standard concepts in a clinical knowledge base. It is much beyond simple string matching and requires a deep semantic understanding of concept mentions. Recent research approach concept normalization as either text classification or text similarity. The main drawback in existing a) text classification approach is ignoring valuable target concepts information in learning input concept mention representation b) text similarity approach is the need to separately generate target concept embeddings which is time and resource consuming. Our proposed model overcomes these drawbacks by jointly learning the representations of input concept mention and target concepts. First, we learn input concept mention representation using RoBERTa. Second, we find cosine similarity between embeddings of input concept mention and all the target concepts. Here, embeddings of target concepts are randomly initialized and then updated during training. Finally, the target concept with maximum cosine similarity is assigned to the input concept mention. Our model surpasses all the existing methods across three standard datasets by improving accuracy up to 2.31%.

## background internet users use social media to voice their views and opinions. medical social media is a part of social media in which the focus is limited to health and related issues @xcite . user generated texts in medical social media include tweets, blog posts, reviews on drugs, health related question and answers in discussion forums. this rich source of data can be utilized in many health related applications to enhance the quality of services provided @xcite . medical concept normalization aims at discovering standard medical concepts in free-form text. in this task, health related mentions are mapped to standard concepts in a clinical knowledge base. for example, the concept mention 'hard to stay awake' is mapped to the standard concept 'drowsy'. the common public express their health related conditions in an informal way using layman terms while clinical knowledge base contains concepts expressed in scientific language. this variation (colloquial vs scientific) in the languages of common public and knowledge bases makes concept normalization an essential step in understanding user-generated texts. this task is much beyond simple string matching as the same concept can be expressed in a descriptive way using colloquial words or in multiple ways using aliases, acronyms, partial names and morphological variants. further, noisy nature of user-generated texts and the short length of health-related mentions make the task of concept normalization more challenging. research in medical concept normalization started with string matching techniques @xcite @xcite followed by machine learning techniques @xcite . the inability of these methods to consider semantics into account shifted research towards deep learning methods with embeddings as input @xcite @xcite . for example, @xcite and @xcite experimented with rnn on the top of domain specific embeddings. further, lack of large labeled datasets and necessity to train deep learning models like cnn or rnn from scratch (except embeddings) shifted research towards using pretrained language models like bert and roberta @xcite @xcite . @xcite experimented with bert based fine-tuned models while kalyan and sangeetha (2020a) provided a comprehensive evaluation of bert based general and domain specific models. the approach of @xcite is based on roberta @xcite and graph embedding based target concept vectors. the main drawbacks in existing work are : moreover, the quality of generated concept embeddings using graph embedding methods depends on the comprehensiveness of vocabulary. for example, meddra is less fine grained compared to snomed-ct @xcite . this requirement of comprehensive vocabulary limits the effectiveness of this approach. our model normalizes input concept mention by jointly learning the representations of input concept mention and target concepts. by learning the representations of target concepts along with input concept mention, our model a) exploits target concepts information unlike existing text classification approaches @xcite @xcite ) and b) eliminates the time and resource consuming process of separately generating target concept embeddings unlike existing text similarity approach @xcite . our key contributions are : ## analysis here, we discuss merits and demerits of our proposed method. ## conclusion in this work, we deal with medical concept normalization in user generated texts. our model overcomes the drawbacks in existing text classification and text similarity approaches by jointly learning the representations of input concept mention and target concepts. by learning target concept representations along with input concept mention representations, our approach a) exploits valuable target concepts information unlike existing text classification approaches and b) eliminates the need to separately generate target concept embeddings unlike existing text similarity approach. our model surpasses all the existing methods across three standard datasets by improving accuracy up to 2.31%. in future, we would like to explore other possible options to include target concept information which may further improve the results.