TATL at WNUT -2020 Task 2: A Transformer-based Baseline System for Identification of Informative COVID -19 E nglish Tweets (2020)

Abstract: As the COVID-19 outbreak continues to spread throughout the world, more and more information about the pandemic has been shared publicly on social media. For example, there are a huge number of COVID-19 English Tweets daily on Twitter. However, the majority of those Tweets are uninformative, and hence it is important to be able to automatically select only the informative ones for downstream applications. In this short paper, we present our participation in the W-NUT 2020 Shared Task 2: Identification of Informative COVID-19 English Tweets. Inspired by the recent advances in pretrained Transformer language models, we propose a simple yet effective baseline for the task. Despite its simplicity, our proposed approach shows very competitive results in the leaderboard as we ranked 8 over 56 teams participated in total.

## introduction the covid-19 pandemic has been spreading rapidly across the globe and has infected more than 20 millions men and women. as a result, more and more people have been sharing a wide variety of information related to covid-19 publicly on social media. for example, there are a huge number of covid-19 english tweets daily on twitter. however, the majority of those tweets are uninformative and do not contain useful information, therefore, systems which can automatically filter out uninformative tweets are needed by the community. tweets are generally different from traditional written-text such as wikipedia or news articles due to its short length and informal use of words and grammars (e.g abbreviations, hashtags, marker). these special characteristics of tweets may pose a challenge for many nlp techniques that focus solely on formally written texts. informative covid-19 english tweets @xcite . inspired by the recent success of transformer-based pre-trained language models in many nlp tasks @xcite @xcite @xcite , we propose a simple yet effective baseline for the task. despite its simplicity, our proposed approach shows very competitive results. in the following sections, we first describe the task definitions in section 2 and proposed methods in section 3. we then describe the experiments and their results in section 4. finally, in section 5, we conclude this work and discuss potential future research directions. ## task definitions the goal of shared task 2 is to identify whether a covid 19 english tweet is informative or not. such informative tweet provides information about recovered, suspected, confirmed and death cases as well as location and history of each case. the dataset introduced in this shared task consists of 10k covid 19 english tweets. dataset statistics can be found in ## conclusion in this paper, we introduce a simple but effective approach for identifying informative covid-19 english tweets. despite the simplicity of our approach, it achieves very competitive results in the leaderboard as we ranked 8 over 56 teams partici-pated in total. in future work, we will conduct thorough error analysis and apply visualization techniques to gain more understandings of our models @xcite . furthermore, we will also extend our approach to other languages. finally, we will investigate the use of advanced techniques such as transfer learning, few-shot learning, and self-training to improve the performance of our system further @xcite @xcite @xcite .