Siva at WNUT -2020 Task 2: Fine-tuning Transformer Neural Networks for Identification of Informative Covid-19 Tweets (2020)

Abstract: Social media witnessed vast amounts of misinformation being circulated every day during the Covid-19 pandemic so much so that the WHO Director-General termed the phenomenon as “infodemic.” The ill-effects of such misinformation are multifarious. Thus, identifying and eliminating the sources of misinformation becomes very crucial, especially when mass panic can be controlled only through the right information. However, manual identification is arduous, with such large amounts of data being generated every day. This shows the importance of automatic identification of misinformative posts on social media. WNUT-2020 Task 2 aims at building systems for automatic identification of informative tweets. In this paper, I discuss my approach to WNUT-2020 Task 2. I fine-tuned eleven variants of four transformer networks -BERT, RoBERTa, XLM-RoBERTa, ELECTRA, on top of two different preprocessing techniques to reap good results. My top submission achieved an F1-score of 85.3% in the final evaluation.

## introduction in today's highly connected world, social media assumes a vital role during pandemics like covid-19. social media platforms have been used positively by international health organizations and governments to disseminate information about the pandemic and precautions regrading the same. unfortunately, the world has also seen the misuse of social media for achieving cheap ends, severely damaging the physical and mental health of individuals, along with the increase in societal distrust. the false information is dangerous during a crisis when the mass panic can only be controlled with information @xcite . a study @xcite ) identifies that panic in social media regarding covid-19 spread faster than the covid-19 virus spread. the proportionately large volume of misinformation compared to credible information can be seen from the very less engagement generated by posts from who (in several thousands) as compared to false news(over 52 million) @xcite . the direct damage of misinformation is clearly evident. in iran, hundreds of people died after consuming methanol, which is propagated on social media as one of the cures foot_0 . there is even concern from authorities that even if a vaccine is discovered, anti-vaccination groups on social media may prevent people from getting vaccinated. false and conspiracy information about leading health organizations like who and leading scientists can quickly decrease the public trust on the correct agents, thus causing self-damage to individuals. the ill-effects of misinformation during pandemics can reach such extremes where they can even affect governmental policies. for example, @xcite reminds us how the rejection of aids by the south african government in the early 2000s cost more than 300,000 lives. in this context, proper identification and control of misleading information on social media assume a pivotal role. however, manual identification and control of the false posts is a herculean task. for example, a 900 percent increase of fact-checkers during covid-19 could not handle the flow of misinformation @xcite . so developing automated methods and techniques to identify misinformation is essential in the current context. the wnut-2020 task 2 take steps in this direction. recently, transformer neural architectures achieved state-of-the-art results in several nlp tasks, including text classification. in this work, i make use of eleven variants of four different transformer archi-tectures to reap better results on the identification of informative covid-19 tweets. the rest of the paper is organized as follows. section 2 discusses work related to the covid-19 misinformation spread. section 3 provides a brief description of the task and dataset. later, in section 4, i discuss the training procedure followed by my submission's internal and official results in section 5. finally, i conclude my work in section 6. the quantitative analysis, done on 673 tweets by @xcite shows that around 25% of tweets are misinformative, and 17% of the tweets have unverifiable information.the authors also report that misinformation rate is higher than informal individual accounts and that certain tags like "@2019 nconv" and "corona" are associated with higher misinformation than tags like "covid-19". @xcite reports that most of the misinformation spreads via retweets and that social bots are involved in amplifying and posting low-credibility information. furthermore, they analyze that the amount of misinformation on covid-19 is more than the total volume of new york times articles. there is no research in the automatic identification of misinformative posts on social media so far. ## brief description of task and dataset in this section, i present a brief description about the task and the dataset provided. interested readers can refer the task description paper(nguyen et al., 2020) for further details.