Innovators@ SMM 4 H ’22: An Ensembles Approach for Stance and Premise Classification of COVID -19 Health Mandates Tweets (2022)

Abstract: This paper presents our submission for the Shared Task-2 of classification of stance and premise in tweets about health mandates related to COVID-19 at the Social Media Mining for Health 2022. There have been a plethora of tweets about people expressing their opinions on the COVID-19 epidemic since it first emerged. The shared task emphasizes finding the level of cooperation within the mandates for their stance towards the health orders of the pandemic. Overall the shared subjects the participants to propose system’s that can efficiently perform 1) Stance Detection, which focuses on determining the author’s point of view in the text. 2) Premise Classification, which indicates whether or not the text has arguments. Through this paper we propose an orchestration of multiple transformer based encoders to derive the output for stance and premise classification. Our best model achieves a F1 score of 0.771 for Premise Classification and an aggregate macro-F1 score of 0.661 for Stance Detection. We have made our code public here

## introduction users actively share their opinions on numerous problems on social media. these concerns are now frequently linked to the covid-19 pandemic. users, for example, share their feelings about quarantines and wearing masks in public places. some comments are supported by arguments, while others are simply emotional claims. using twitter tweets, automated algorithms for assessing people's attitudes regarding covid-19 health orders can assist in determining the extent of cooperation with the mandates. in this paper, we try to provide a feasible solution towards the task of stance-detection @xcite . we are extracting arguments from covidrelated tweets. according to argumentation theory, an argument must include a claim containing a stance towards some topic or object, and at least one premise supporting this stance. so basically, we are finding whether the given tweet contains a stance (favour, against, or none) and also if it contains a premise (reason to support the stance). the first subtask of the shared task is to design a system that can determine the point of view (stance) of the text's author in relation to the given claim (e.g., wearing a face mask).given a tweet, there can be 3 different values for stance favor -positive stance against -negative stance neitherneutral/unclear/irrelevant stance the second subtask is to predict whether at least one premise or argument is mentioned in the text. a given tweet is considered to have a premise if it contains a statement that can be used as an argument in a discussion. there are 2 values for premise. 1 -tweet contains a premise (argument) 0 -tweet doesn't contain a premise (argument). ## system architecture the main task was divided into two subtasks. subtasks (a)stance detection and (b)premise classification, respectively. both subtasks have classification problems, with subtask (a) having 3 classes (favour, against, or none) and subtask (b) having 2 classes (1-tweet contains a premise , 2-tweet does not contain a premise). we try out five different transformer models: bert, pubmedbert, roberts, scibert, and specter. finally, we combine all five models' predictions with a voting classifier. n ensemble modeling, multiple diverse models are created to predict an outcome. in our case, we use a voting classifier to ensemble the predictions of all five models (bert, roberta, pubmed-bert, scibert and specter). as a result, the overall accuracy and f1 scores increased for stance detection and premise classification. ## analysis the overall dataset contains 6269 tweets, 3669 in training, 600 in validation and 2000 in testing. the tweets were further divided into three categories based on their claim, (i) face masks (ii) stay at home orders (iii) school closures. as previously indicated, our proposed system employs an ensemble voting classifier, which assigns distinct weights to each of the predictions from the 5 included models during inference and determines the output label appropriately. ## conclusions and future work we explored the use of a simple transformer based architecture for task 2 proposed by smm4h'22. the five different transformer models (bert, roberta, pubmed-bert, scibert, and specter) were trained separately on both subtasks, and their results were then combined to create a voting classifier. as expected, the overall accuracy and f1 scores of the voting ensemble classifier were better as compared to individual model prediction for both the aforementioned subtasks. however, there are still greater possibilities towards improving the overall accuracy. we intent to mine crucial medical information from the text and train a separate classifier on them in this manner we'll have a more precise and robust data to infer from. also, due to time constraint we were restricted towards training a monolingual model, however the given data also entail written text in languages other than english which are ignored by our proposed system. recently proposed language-models like bloom foot_0 can help the model in dealing with the multilingual data.