DSC - IIT ISM at WNUT -2020 Task 2: Detection of COVID -19 informative tweets using R o BERT a (2020)

Abstract: Social media such as Twitter is a hotspot of user-generated information. In this ongoing Covid-19 pandemic, there has been an abundance of data on social media which can be classified as informative and uninformative content. In this paper, we present our work to detect informative Covid-19 English tweets using RoBERTa model as a part of the W-NUT workshop 2020. We show the efficacy of our model on a public dataset with an F1-score of 0.89 on the validation dataset and 0.87 on the leaderboard.

## related work detection of useful-crisis-related content has been pivoting around twitter due to its interactive media via microtexts @xcite . continuous bag-of-words (cbow) based approach has been used for text classification @xcite . @xcite proposes the use of different user-based features representing messages and tweet propagation for classifying tweet credibility. some works proposed use of svm @xcite , logistic regression @xcite , random forest classifier @xcite and word embedding based method @xcite for classification of tweet contents. @xcite provided the use of unsupervised methods to cluster news topics from tweets. the capability of dependency learning and semantic information extraction enables us to learn complex decision boundaries. the evolution around capturing the semantic relationships between words lead to the widely used transformer @xcite architecture. such models have outperformed conventional methods over natural language processing (nlp) tasks. they have been widely used for various real-world applications such as language modeling @xcite , sarcasm detection @xcite , summarization @xcite , and other language tasks. ## dataset the training and validation data consists of 7000 and 1000 samples, respectively. the test dataset consists of 12000 tweets, out of which 2000 tweets were selected by the organizers for final evaluation. the actual labels of the test dataset was not revealed by the shared task, hence the accuracy metric is only reported for the validation dataset in this paper. the number of samples in each class is presented in the word count increased due to emojis' conversion to text and decreased due to the removal of non-ascii characters. an example of a preprocessed tweet for both the informative and uninformative class is shown in figure 2 . ## methods we have applied various conventional machine learning and transformer-based approaches. ## results the f1 scores on the validation dataset obtained for conventional methods and transformer-based methods are presented in it is observed that the tf-idf vectorizer gives better results when compared to bow in almost all the conventional approaches. among all the conventional approaches, mlp gives the best result. the transformer-based methods performed better compared to all the conventional approaches. bert and roberta showed competitive performance. however, roberta has shown better results compared to other transformers based methods. ## conclusion classifying twitter texts has been at the forefront of various nlp applications. here, in this paper we have worked on one such task of classifying a tweet as informative or uninformative in the context of covid-19. we have extensively compared the performance of various methods for this task. we applied conventional approaches and the latest state-of-the-art transformer-based methods. the results shows that the roberta gives superior result on this task. since twitter is primarily a microblogging media, short text classification using topic modeling @xcite , and topic-enhanced embedding-based approach. @xcite can also be useful for tweet classification. in future work, we wish to apply topic modeling to produce enhanced word embeddings for this task.