Classification of COVID 19 tweets using Machine Learning Approaches (2021)

Abstract: The reported work is a description of our participation in the “Classification of COVID19 tweets containing symptoms” shared task, organized by the “Social Media Mining for Health Applications (SMM4H)” workshop. The literature describes two machine learning approaches that were used to build a three class classification system, that categorizes tweets related to COVID19, into three classes, viz., self-reports, non-personal reports, and literature/news mentions. The steps for pre-processing tweets, feature extraction, and the development of the machine learning models, are described extensively in the documentation. Both the developed learning models, when evaluated by the organizers, garnered F1 scores of 0.93 and 0.92 respectively.

## introduction in order to identify personal tweets related to covid-19, it becomes necessary to distinguish them from tweets made by others related to this issue. classification of medical symptoms from posts related to covid-19 poses two major challenges: firstly, the amount of information available as news articles, scientific papers etc that describe various medical symptoms is huge @xcite . all this information makes it extremely difficult to spot significant user reported information. secondly, there are multiple users who report information which is not experienced by themselves but by other people they know or come across @xcite . this makes the task of identifying self reported information from the huge amount of discourse available very complex. the current shared task (task no. 6) 1 , namely "classification of covid19 tweets containing symptoms" provided participants with three classes 1 https://healthlanguageprocessing.org/smm4h-2021/task6/ viz. i. self-reports ii. non-personal reports, and iii. literature/news mentions. this task is a three way classification task. for developing the learning models, we used traditional machine learning (ml) and state-of-theart deep learning (dl) approaches @xcite @xcite . besides, extra features, like parts-of-speech (pos) tags as well as term frequency-inverse document frequency (tf-idf) was used, which enabled the developed models to learn the hidden classes better. upon evaluation, our developed models performed well and this was ratified by the fact that they garnered f1 scores of 0.93 (ml model) and 0.92 (dl model) respectively. the rest of the paper is organized as follows. section 2 describes the data and methodology that was used to develop both the models. this section describes the pre-processing steps, will talk about the extra features that were used, and will also narrate the learning models that were used to build our systems. following this, section 3 and 4 will chronicle the results and the concluding remarks respectively. ## methodology initially, the organizers provided us with 9,567 training data and 500 validation data. this labeled dataset consisted of three fields; tweet id, the actual tweet, and the respective label (self-reports, nonpersonal reports, and literature/news mentions). the training and validation data were later combined and it was pre-processed for further development. steps of pre-processing the tweets included the removal of extra characters to clean the data. the extra characters that were removed/cleaned included mentions, punctuation's and urls. additionally, words from hashtags were extracted and extra spaces were contracted. after the preprocessing steps, pos tags of individual words of every tweet were found out using the python pack-ages natural language toolkit foot_0 (nltk). pos tag features were used as they can help in determining authorship as people's use of words varies. on the other hand, it can easily differentiate between the same words, applied in different settings. e.g., "like" is a verb semantically charged with positive weight, as in "i like you", but it becomes neutral conjunction, as in "i am like you". for feeding the extra pos tag feature along with individual words, to the ml model, we concatenated them to form an extended input of the structure both these vectorized inputs and encoded outputs were fed to a multi-layer perceptron classifier (mlp), where alpha was kept at 1 and maximum iterations were kept at 1,000. for training the dl model, we took the words, pos tags, and tf-idf values as separate inputs passed them through their respective default tensorflow embedding layer, and concatenated the outputs. the output, from the concatenation layer, was then passed through two layers of bi-directional long-short term memory @xcite ) (lstms) and finally fed to a dense layer which mapped the tensors to the respective labels. other parameters of the model were as follows. optimizer was kept as "adam" and loss was kept as "sparsecategoricalcrossentropy". the batch size was kept as 128 and the number of epochs was fixed at 50. also, the early stopping mechanism, where the metric was fixed to validation loss, was applied to stop over-fitting. a depiction of the developed model is shown in figure 1 . both the ml and dl models were then deployed on the 500 validation data provided by the organizers and upon submission, garnered f1 scores of 0.98 and 0.97 respectively. other validation metrics are shown in ## conclusion the reported system paper presents two models developed using ml and dl approaches, that were trained to classify tweets related to covid19, into personal/non-personal mentions or standard literature. from the results, we can see that since the amount of training data was low, traditional ml