Bridging the gap between supervised classification and unsupervised topic modelling for social-media assisted crisis management (2021)

Abstract: Social media such as Twitter provide valuable information to crisis managers and affected people during natural disasters. Machine learning can help structure and extract information from the large volume of messages shared during a crisis; however, the constantly evolving nature of crises makes effective domain adaptation essential. Supervised classification is limited by unchangeable class labels that may not be relevant to new events, and unsupervised topic modelling by insufficient prior knowledge. In this paper, we bridge the gap between the two and show that BERT embeddings finetuned on crisis-related tweet classification can effectively be used to adapt to a new crisis, discovering novel topics while preserving relevant classes from supervised training, and leveraging bidirectional self-attention to extract topic keywords. We create a dataset of tweets from a snowstorm to evaluate our method’s transferability to new crises, and find that it outperforms traditional topic models in both automatic, and human evaluations grounded in the needs of crisis managers. More broadly, our method can be used for textual domain adaptation where the latent classes are unknown but overlap with known classes from other domains.

## introduction 1.social media for crisis management as climate change increases the frequency of extreme weather events and the vulnerability of affected people, effective crisis management is becoming increasingly important for mitigating the negative effects of these crises @xcite . in the general crisis management literature, social media has been identified as a useful source of information for crisis managers to gauge reactions from and communicate with the public, increase situational awareness, and enable data-driven decision-making @xcite @xcite . ## methodology in this section we describe the dataset of snowstorm-related tweets we created to evaluate our model's ability to discover novel topics and transfer to unseen crisis events. we then outline the process by which our model learns to extract crisis-relevant embeddings from tweets, and clusters them into novel topics from which it then extracts interpretable keywords. ## evaluation in this section we describe the baselines, as well as the automatic and human evaluations used. ## conclusion this paper introduces a novel approach for extracting useful information from social media and assisting crisis managers. we propose a simple method that bridges the gap between supervised classification and unsupervised topic modelling to address the issue of domain adaptation to novel crises. our model (fte, finetuned tweet embeddings) incorporates crisis-related knowledge from supervised finetuning while also being able to generate salient topics for novel crises. to evaluate domain adaptation, we create a dataset of tweets from a crisis that significantly differs from existing crisis twitter datasets: the 2020 winter storm jacob. our paper also introduces human evaluation methods better aligned with downstream use cases of topic modelling in crisis management, emphasizing human-centered machine learning. in both these human evaluations and traditional automatic evaluations, our method outperforms existing topic modelling methods, consistently producing more coherent, interpretable and useful topics for crisis managers. interestingly, our annotators reported that several coherent topics seemed to be composed of related subtopics. in future work, the number of topics as a hyper-parameter could be explored to see if our approach captures these salient subtopics. our method, while simple, is not specific to crisis management and can be more generally used for textual domain adaptation problems where the latent classes are unknown but likely to overlap with known classes from other domains.