IDSOU at WNUT -2020 Task 2: Identification of Informative COVID -19 E nglish Tweets (2020)

Abstract: We introduce the IDSOU submission for the WNUT-2020 task 2: identification of informative COVID-19 English Tweets. Our system is an ensemble of pre-trained language models such as BERT. We ranked 16th in the F1 score.

## introduction the spread of the covid-19 is causing fear and panic to people around the world. to monitor the covid-19 outbreaks in real-time, sns analysis such as twitter is attracting much attention. although there are 4 million covid-19 english tweets posted daily on twitter @xcite , most of them are uninformative. against this background, wnut-2020 held a shared task 2 @xcite to automatically identify whether a covid-19 english tweet is informative or not. our system employs an ensemble approach based on pre-trained language models. such pretrained language models @xcite @xcite @xcite have achieved high performance in various text classification tasks @xcite . in addition, we employ domain-specific pre-trained language models @xcite @xcite to build models suitable for covid-19 and twitter domains. each model is optimized for three types of loss functions, cross-entropy, negative supervision @xcite , and dice similarity coefficient @xcite , which are useful for various text classification tasks. finally, we ensemble 48 classifiers based on 16 pre-trained language models and 3 loss functions with a random forest classifier @xcite ## wnut-2020 shared task 2 in the shared task @xcite , systems are required to classify whether a covid-19 english tweet is informative or not. such informative tweets provide information about recovered, suspected, confirmed and death cases as well as location or travel history of the cases. the 10,000 covid-19 english tweets foot_0 shown in the baseline system is based on fasttext @xcite . systems are evaluated by accuracy, precision, recall and f1 score, and are ranked by f1 score, which is the main metric. note that the latter three metrics are calculated for the informative class only. ## idsou system we first introduce each base model in section 3.1 and each loss function in section 3.2. we then introduce the ensemble model in section 3.3. finally, section 3.4 describes the implementation details. ## results regarding the loss function, xe loss showed stable performance. ns loss is effective for 6 out of 16 models and seems to be compatible with bert. ds loss achieved the best performance in combination with covid-twitter-bert, although overall performance is not high. ## conclusions we describe the idsou submission for the wnut-2020 task 2. our system is an ensemble model based on 16 pre-trained language models and 3 loss functions with a random forest classifier. in the official result, we ranked 16th out of 55 teams.