Cross-Lingual Disaster-related Multi-label Tweet Classification with Manifold Mixup (2020)

Abstract: Distinguishing informative and actionable messages from a social media platform like Twitter is critical for facilitating disaster management. For this purpose, we compile a multilingual dataset of over 130K samples for multi-label classification of disaster-related tweets. We present a masking-based loss function for partially labelled samples and demonstrate the effectiveness of Manifold Mixup in the text domain. Our main model is based on Multilingual BERT, which we further improve with Manifold Mixup. We show that our model generalizes to unseen disasters in the test set. Furthermore, we analyze the capability of our model for zero-shot generalization to new languages. Our code, dataset, and other resources are available on Github.

## introduction in times of disaster, affected individuals often turn to social media platforms, such as twitter or facebook, to express their feelings generated by a disaster, update friends and relatives on their status, request help or supplies, or report useful information to the disaster response teams. response organizations can use social media to increase situational awareness by providing information about disaster status, ongoing rescue operations, and disaster warnings @xcite . however, the low entry-barrier of social media platforms, where everybody can post their own "news" in realtime, leads to information overload, making it hard for users to find relevant and useful information @xcite . thus, it is crucial to filter out the non-informative messages, and to distinguish among different categories of informative messages to ensure that a message reaches its target users. in turn, this can help facilitate disaster response and increase situational awareness. towards this goal, in recent years, many works have focused on disaster-related tweet classification @xcite @xcite @xcite @xcite . however, most of these works have focused on the classification of english tweets only, with a few notable exceptions @xcite @xcite . we stress that there are a lot of disaster-prone non-englishspeaking countries, which could benefit from a multilingual classifier that can be used in real-time to identify useful information on social media. furthermore, there is a lack of a large scale standard multilingual disaster-related dataset for multi-label classification with diverse disaster types. against this background and needs, we make the following contributions: ## related work there are numerous prior works on disaster-related tweet classification. for example, @xcite focus on classifying and extracting actionable information from disaster-related tweets, assuming that sufficient labeled tweets from the ongoing disaster are available for model training. later, @xcite explore real-time classification of tweets from a target disaster using models trained on past disasters. @xcite introduce a convolutional neural network that performs robustly even on out-of-event data during inference. other works explore domainadaptation that uses labeled tweets from past disasters and unlabeled tweets from an ongoing disaster @xcite . @xcite take a few-shot learning approach, in which a disaster-specific model is trained using only a few (around 10) examples for disaster-related tweet detection. in contrast, we train a universal model on diverse disaster types for fine-grained classification and show that it performs remarkably well on unseen disaster types without further training (specifically, it achieves zero-shot generalization to unseen events). @xcite classify actionable tweets using elmo contextual word embeddings, whereas @xcite uses a monolingual bert-based model for disaster-related tweet classification. in contrast, we work with a multilingual model, which we compare with multiple baselines, and augment with manifold mixup. regarding cross-lingual approaches, dittrich and lucas (2014) present a real-time application tool for multilingual tweet classification and disaster detection. however, this tool requires a long training phase with tweets from specific areas for robust detection, and its multilingual classifier filters messages based on shallow matching of pre-selected keywords (and their translations). @xcite construct a multilingual model for tweet classification using multilingual wikipedia articles as knowledge repository. @xcite al. (2018) a few recent works @xcite also demonstrate the strong cross-lingual and zero-shot transfer capabilities of m-bert, but not in the disaster domain. ## aggregated dataset to prepare our large multilingual dataset, we aggregated several resources from crisisnlp, foot_1 together with two resources from crisislex. 3 specifically, we used resource #1 @xcite , resource #5 @xcite , and resource #7 @xcite from crisisnlp, and crisislext6 @xcite and crisislext26 @xcite from crisislex. the original classes in each resource, together with the mapping to the new classes included in our data set, can be seen in ## conclusion we present a way to aggregate prior disaster-related resources to compile a large scale tweet dataset for multi-label classification utilizing both multi-class classes and binary classes. we motivate the use of m-bert for disaster-related tweet classification and we demonstrate its strong performance on unseen disasters and languages. we also motivate the use of manifold mixup for further improvement. in the future, it would be interesting to explore weak suervision and other data augmentation techniques to improve models' robustness further.