Medical Concept Mention Identification in Social Media Posts Using a Small Number of Sample References (2023)

Abstract: Identification of mentions of medical concepts in social media text can provide useful information for caseload prediction of diseases like Covid-19 and Measles. We propose a simple model for the automatic identification of the medical concept mentions in the social media text. We validate the effectiveness of the proposed model on Twitter, Reddit, and News/Media datasets.

## introduction caseload information of diseases like covid-19 and measles are likely reflected in social media posts in the form of mentions of relevant medical concepts. for example, increase in the mentions of medical concepts like fever, headache, cough, loss of smell etc. in social media text is a potential indication of increasing covid caseloads. therefore models which identify the mentions of medical concepts from social media text can provide useful features for the caseload prediction of such diseases. state-of-the-art natural language processing techniques mostly rely on huge pre-trained language models and such models can be utilized for identifying the mentions of medical concepts in social media texts. in this work, we propose a simple and effective model to automatically identify the presence of 24 selected medical concepts in social media text by using a small number of reference texts and a pre-trained language model. ## related works the basis for the medical concept mention identification method carried out in this work is the research on medical concept normalization. in the literature, the medical concept normalization problem is addressed by using different approaches. traditionally lexicon-based string-matching approaches and rule-based approaches are used for medical concept normalization. for example, @xcite used a knowledge-intensive approach for concept normalization which is based on symbolic language processing. @xcite approached the medical concept normalization problem by learning the similarity between mentions and concept names. @xcite approached this medical concept normalization as a phrase-based machine translation problem and they translated social media phrases into formal medical concepts. in another approach, @xcite used simple deep-learning-based models like cnn, and rnn with pre-trained lm and improved the performance of the medical concept normalization. @xcite further improved this performance by refining the dataset and leveraging the neural embeddings of health-related text. @xcite showed that language models can learn the semantics of medical concepts. they found that subword information is crucial for learning medical concept representation and global word co-occurance information is more useful for downstream tasks using these representations. this suggests the suitability of language models that have both subword information and global co-occurrence information for medical concept normalization. more recently kalyan and sangeetha (2020) used the transformer-based bert pre-trained language model for the medical concept normalization. in this method, they generated the embeddings of concepts and mentions by using the pre-trained roberta language model @xcite . they further enriched concept embeddings using synonym information by using a retrofitting method proposed by @xcite . then the relations between concepts and mentions are calculated by using cosine similarity between their embeddings. @xcite also proposed a similar and simple model for medical concept normalization by using pre-trained language model sapbert and cosine similarity. based on these approaches we formulated our new model to extract medical concept features from social media text. however, in our approach, instead of retrofitting the concept embedding by using synonyms, we use information from manually selected positive and negative samples along with synonyms information and propose a novel closed-form optimization formulation for generating concept representations. ## the proposed model our proposed model to automatically identify the mentions of a set of medical concepts from the social media text is inspired by the medical concept normalization model proposed by @xcite . the proposed model utilizes a small number of preselected positive and negative samples along with the name and synonyms of the medical concepts to learn an anchor vector representation (distributed representation) of each of these concepts. in order to learn the anchor vector of concepts we first generate distributed representations of all the selected positive and negative samples, name of the concept, and its synonyms. then we will learn an anchor vector for each concept (v c ) by solving the optimization with the following objectives: we formulated this multiobjective optimization problem as a single objective optimization by defining a single aggregate objective function by taking the weighted sum of these objectives. the final objective will be: where λ p , λ n , λ s are positive weights corresponding to each of three objectives and without loss of generality we can set the weight corresponding to the fourth objective (first term in the single aggregate objective) as 1. if we add a constraint that the v c is a unit vector we will get a nice closed-form solution for this optimization problem, which is: this closed-form solution enables us to learn the anchor vector representation of each of the concepts by calculating the exact solution for the proposed optimization problem with linear time complexity. the samples used to learn the proposed model are very small and therefore we can easily learn the anchor vector representations of concepts without much computational resources. once we learn the anchor vector representations of each of the concepts, we can easily calculate the components of these concepts in any of the social media texts by taking the cosine similarity between the distributed representation of the social media text and the anchor vector representation of the corresponding concept. ## experimentations medical concepts considered for this work are based on the covid-19 foot_0 and measles foot_1 symptoms mentioned by the world health organisation. we selected 24 key medical concepts related to symptoms of covid-19 or measles. for each of the selected 24 medical concepts, we manually identified a set of synonyms. we used two sources of information for this process, first we consulted the snomed ct browser 3 , and then we also manually reviewed the top webpages returned in response to a general web search using the medical concept as the keyword to identify potential synonyms. the number of synonyms identified ranged from a minimum of 6 synonyms for the concept cough to 41 synonyms for the concept loss of mobility foot_3 . then we manually collected 10 positive and 10 negative sample tweets for each of the 24 medical concepts. a tweet which contains the mention of a medical concept is selected as the positive sample for that concept. in order to select the negative samples we considered tweets which can be misinterpreted as a positive sample. for example, a tweet which contains the term 'pink-eye shadow' may be misinterpreted as being relevant to the medical concept conjunctivitis due to the relation to 'pink-eye'. but the term 'pink-eye shadow' is not related to the medical concept conjunctivitis and therefore explicitly providing the information that this social media text is not related to conjunctivitis will be helpful for the model. therefore we selected such misinterpretable samples as negative samples for all 24 medical concepts. if for a given medical concept we can't find 10 tweets that contain mentions to concepts that can be confused with the target concept then we select arbitrary samples which are not related to the corresponding medical concept as the remaining negative samples. we generated 768-dimensional distributed representations of concept name, synonyms, and it's manually selected 10 positive sample tweets and 10 negative sample tweets of each of the 24 medical concepts by using a pre-trained sentence bert language model (all-mpnet-base-v2) @xcite . then we learned the 768-dimensional anchor vector representations corresponding to each of these 24 concepts. we used this generated anchor vector representation for all our experiments on twitter, reddit, and news/media datasets. ## discussion the basis of the proposed model is for each medical concept that we wish to identify mentions of we learn an anchor vector (embedding). in our experiments, we used selected twitter samples to learn this anchor vector. one interesting question is how effective this model which is trained by using data from one social media on another social media data. we already evaluated the model on two other social media, reddit and news/media. in order to compare the performance of the model on twitter with the performance on reddit and news/media we generated box plots of these three datasets, see in fig. 5 . for each of these three datasets, the corresponding box plot shows the minimum, first quartile, median, third quartile, and maximum auc-cg scores across all medical concepts considered for the evaluation. from the box plots, we can see that the performance of the model on reddit is comparable with twitter for most of the concepts, but on news/media data we can see a performance drop. we note that compared to twitter samples the social media texts in news/media dataset are longer and therefore the model's anchor vectors trained using twitter samples may be less effective for samples from news/media. topical divergence between samples from twitter and news/media may also affect the performance of the model. in order to improve the performance we may need to include samples from news/media for training the model. ## conclusion we proposed a simple model to automatically identify the mentions of medical concepts in social media text by using a pre-trained language model and a small set of carefully selected samples. we validated the effectiveness of the proposed model on three social media sources twitter, reddit, and news/media particularly focusing on medical concepts related to covid-19 and measles.