Cross-lingual Transfer Learning for COVID-19 Outbreak Alignment (2020)

Abstract: The spread of COVID-19 has become a significant and troubling aspect of society in 2020. With millions of cases reported across countries, new outbreaks have occurred and followed patterns of previously affected areas. Many disease detection models do not incorporate the wealth of social media data that can be utilized for modeling and predicting its spread. It is useful to ask, can we utilize this knowledge in one country to model the outbreak in another? To answer this, we propose the task of cross-lingual transfer learning for epidemiological alignment. Utilizing both macro and micro text features, we train on Italy’s early COVID-19 outbreak through Twitter and transfer to several other countries. Our experiments show strong results with up to 0.85 Spearman correlation in cross-country predictions.

## introduction during the covid-19 pandemic, society was brought to a standstill, affecting many aspects of our daily lives. with increased travel due to globalization, it is intuitive that countries have followed earlier affected regions in outbreaks and measures to contain to them @xcite . a unique form of information that can be used for modeling disease propagation comes from social media. this can provide researchers with access to unfiltered data with clues as to how the pandemic evolves. current research on the covid-19 outbreak concerning social media includes word frequency and sentiment analysis of tweets @xcite and studies on the spread of misinformation @xcite . social media has also been utilized for other disease predictions. several papers propose models to identify tweets in which the author or nearby person has the attributed disease @xcite @xcite . @xcite and @xcite utilize word frequencies to align tweets to disease rates. a shortcoming of the above models is they do not consider how one region's outbreak may relate to another. many of the proposed models also rely on lengthy keyword lists or syntactic features that may not generalize across languages. text embeddings from models such as multilingual bert (mbert) @xcite and laser @xcite ) can allow us to combine features and make connections across languages for semantic alignment. we present an analysis of twitter usage for crosslingual covid-19 outbreak alignment. we study the ability to correlate social media tweets across languages and countries in a pandemic scenario. based on this demonstration, researchers can study various cross-cultural reactions to the pandemic on social media. we aim to analyze how one country's tweets align with its own outbreak and if those same tweets can be used to predict the state of another country. this can allow us to determine how actions taken to contain the outbreak can transfer across countries with similar measures. we show that we can achieve strong results with crosslingual transfer learning. our contributions include: figure 1 : timeline of covid-19-related tweets, from covid-19 dataset @xcite , in various languages. the peaks are marked by events relating to each language's main country's initial outbreak. ## conclusion in this paper, we performed an analysis of crosslingual transfer learning with twitter data for covid-19 outbreak alignment using cross-lingual sentence embeddings and keyword frequencies. we showed that even with our limited sample sizes, we can utilize knowledge of countries with earlier outbreaks to correlate with cases in other countries. with larger sample sizes and when training on a variety of points during the outbreak, we can obtain stronger correlations to other countries. we hope our analysis can lead to future integration of social media in epidemiological prediction across countries, enhancing outbreak detection systems.