CSECU - DSG @ SMM 4 H ’22: Transformer based Unified Approach for Classification of Changes in Medication Treatments in Tweets and W eb MD Reviews (2022)

Abstract: Medications play a vital role in medical treatment as medication non-adherence reduces clinical benefit, results in morbidity, and medication wastage. Self-declared changes in drug treatment and their reasons are automatically extracted from tweets and user reviews, helping to determine the effectiveness of drugs and improve treatment care. SMM4H 2022 Task 3 introduced a shared task focusing on the identification of non-persistent patients from tweets and WebMD reviews. In this paper, we present our participation in this task. We propose a neural approach that integrates the strengths of the transformer model, the Long Short-Term Memory (LSTM) model, and the fully connected layer into a unified architecture. Experimental results demonstrate the competitive performance of our system on test data with 61% F1-score on task 3a and 86% F1-score on task 3b. Our proposed neural approach ranked first in task 3b.

## introduction user-generated contents on social media and online health forums represent a wide variety of facts, experiences, and opinions on various health topics including personal health issues, reviews on medication, side effects, and informal questions on health concerns. shared information on social media or online health forums allows to detect users' self-declared changes in medication. self-declared changes include stopping a treatment, changing a dose, or forgetting to take the drugs, etc. in this work, we focus on classifying patients making changes to their medication treatments (i.e nonpersistent patients) as a part of our participation in the social media mining for health (smm4h) 2022 shared task 3 @xcite . this task includes two subtasks. we have to detect changes in medication treatments from tweets and webmd reviews in subtasks 3a and 3b, respectively. webmd * corresponding author is one of the top healthcare websites and an online publisher of news and information pertaining to drugs, human health and well-being. the rest of the paper describes our proposed framework, experimental details, and evaluation. ## proposed framework an overview of our proposed framework is shown in figure 1 . for a given text, we use the flair @xcite framework to extract document embedding features from the transformer model roberta. these features are carried out through two different capsules. one is a stacked lstm and the other is a simple linear layer. then, we concatenate the features of the lstm and linear architecture, which are then applied to a feedforward fully-connected output layer to procure predicted labels. roberta base roberta base stacked lstm stacked lstm predicted label predicted label fully connected linear layer fully connected linear layer feed-forward output layer feed-forward output layer feature fusion tweet/review tweet/review ## error analysis figure 2 shows the confusion matrix for subtasks 3a and 3b on the development set. we observe a relatively high proportion of misclassified nonpersistent tweets, while the system performs fairly well in detecting non-persistent reviews. further, we articulate some erroneous predictions of our system in ## conclusion in this paper, we introduce an approach to identify the self-declared drug-changing information by integrating roberta, lstm, and fully connected linear layers. in the future, we intend to explore biomedical-based transformers and model ensembling to distill better feature representation.