SPEED ++: A Multilingual Event Extraction Framework for Epidemic Prediction and Preparedness (2024)

Abstract: Social media is often the first place where communities discuss the latest societal trends. Prior works have utilized this platform to extract epidemic-related information (e.g. infections, preventive measures) to provide early warnings for epidemic prediction. However, these works only focused on English posts, while epidemics can occur anywhere in the world, and early discussions are often in the local, non-English languages. In this work, we introduce the first multilingual Event Extraction (EE) framework SPEED++ for extracting epidemic event information for any disease and language. To this end, we extend a previous epidemic ontology with 20 argument roles; and curate our multilingual EE dataset SPEED++ comprising 5.1K tweets in four languages for four diseases. Annotating data in every language is infeasible; thus we develop zero-shot cross-lingual cross-disease models (i.e., training only on English COVID data) utilizing multilingual pre-training and show their efficacy in extracting epidemic-related events for 65 diverse languages across different diseases. Experiments demonstrate that our framework can provide epidemic warnings for COVID-19 in its earliest stages in Dec 2019 (3 weeks before global discussions) from Chinese Weibo posts without any training in Chinese. Furthermore, we exploit our framework’s argument extraction capabilities to aggregate community epidemic discussions like symptoms and cure measures, aiding misinformation detection and public attention monitoring. Overall, we lay a strong foundation for multilingual epidemic preparedness.

## introduction timely epidemic-related information is vital for policymakers to issue warnings and implement control measures @xcite . social media being timely, publicly accessible, widely used, and high in volume @xcite , 2013; @xcite acts as a crucial information source. previous works @xcite have explored utilizing event extraction (ee) @xcite to extract epidemic events from social media posts for epidemic prediction. however, these works have focused only on english; while epidemics can originate anywhere worldwide and be discussed in various regional languages. in our work, we introduce speed++ (social platform based epidemic event detection + arguments + multilinguality), the first multilingual ee framework designed for epidemic preparedness. we advance english-based speed @xcite multilingually by developing and benchmarking zero-shot cross-lingual models capable of extracting epidemic information across many languages. while speed primarily identifies basic epidemic events, we develop enhanced mod-els capable of extracting detailed event-specific information (e.g., symptoms, control measures) by incorporating event argument extraction (eae). to integrate eae, we enrich the speed ontology with event-specific roles relevant to social media to create a rich ee ontology comprising 7 event types (e.g. infect, cure, prevent, etc.) and 20 argument roles (e.g. disease, symptoms, time, means, etc.). apart from english, we also annotated three other languages -spanish, hindi, and japaneseto benchmark multilingual ee models. leveraging the enriched ontology and expert annotations, we develop our speed++ dataset comprising 5.1k tweets and 4.6k event mentions across four different diseases (covid-19, monkeypox, zika, and dengue) in four languages. using speed++, we develop our zero-shot cross-lingual models by empowering the state-ofthe-art ee models like tagprime @xcite with multilingual pre-training and augmented training using pseudo-generated multilingual data from clap @xcite . these models are trained realistically on limited english covidspecific data. benchmarking on speed++ reveals how our trained models outperform various baselines by an average of 15-16% f1 points for unseen diseases across four different languages. to demonstrate the utility of our multilingual ee speed++ framework, we apply it to two epidemicrelated applications. first, we utilize the framework's multilingual capabilities for epidemic prediction by aggregating epidemic events across different languages. by incorporating tweet locations, we construct a global epidemic severity meter capable of providing epidemic warnings in 65 languages spanning 117 countries. applying our framework for covid-19 to chinese weibo posts, we successfully detected early epidemic warnings by dec 30, 2019 (figure 1 ) -three weeks before the global infection tracking even began. this multilingual epidemic prediction capability can significantly enhance our global preparedness for future epidemics. as another application, we repurpose our framework as an information aggregation system for community discussions about epidemics such as symptoms, cure measures, etc. leveraging the eae capability of our framework, we meticulously extract these event-specific details from millions of tweets across diseases and languages. similar arguments are then agglomeratively clustered to generate an aggregated ranked bulletin. we demonstrate that this bulletin can aid misinformation detection (e.g., my mother is tested positive for covid19 , i need to go in quarantine as well. ## background epidemic prediction is a classic epidemiological task that provides early warnings for future epidemics of any infectious disease @xcite . previous works @xcite have utilized keyword-based and simple classification-based methods for extracting epidemic mentions (detailed in § 6). speed @xcite was the first to explore event extraction (ee) for extracting epidemic-based events in english. in our work, we utilize event extraction but focus multilingually on a broader range of languages. the extracted events are aggregated over time and abnormal influxes are reported as early epidemic warnings. to our best knowledge, we are the first to develop a multilingual event extraction framework for epidemic prediction. ## dataset creation we focus on social media, specifically twitter as our main document source for studying four diseases -covid-19, monkeypox, zika, and dengue. speed @xcite focused only on event detection (ed) for english. since ed identifies events but does not provide any epidemic-related information, we improve speed by additionally incorporating event argument extraction (eae) to develop a complete ee dataset speed++. furthermore, we extend to three other languages used on social media -spanish, hindi, and japanese -to enhance the multilingual capability of our framework. we detail the data creation process below while figure 3 provides a high-level overview. ## zero-shot cross-lingual event extraction to validate the effectiveness of ee for epidemic events, we benchmark various ee models using speed++. given the infeasibility of procuring quality data in all languages for all diseases, we benchmark in a zero-shot cross-lingual crossdisease fashion i.e. we train models only on english covid data and evaluate on the rest. we provide the data split for our benchmarking in model covid mpox zika + dengue average hi jp es en en hi jp es ## applications to validate its practical utility for epidemic preparedness, we demonstrate our framework's use in two downstream applications: global epidemic prediction and epidemic information aggregation. for this, we train a multilingual tagprime model on the entire speed++ dataset. further details about these applications are provided below. ## related works event extraction datasets event extraction (ee) aims at detecting events (event detection) and extracting details about specific roles associated with each event (event argument extraction) from natural text. unlike document parsing @xcite , we utilize ee only at the sentence/tweet level in our work. overall, ee is a well-studied task with earliest works dating back to muc @xcite , ace @xcite and ere @xcite , but there have been various newer diverse datasets like maven @xcite , wikievents @xcite , docee @xcite , and geneva @xcite . while most of these datasets are only in english, some datasets like ace @xcite , ere @xcite , and mee @xcite provide ee data in ten languages for general-purpose events in the news and wikipedia domains. speed @xcite introduces data for epidemic-based events in social media but is limited to event detection and focuses on en-glish. overall, speed++ extends speed to four languages and event argument extraction. ## conclusion and future work in our work, we pioneer the creation of the first multilingual event extraction (ee) framework for application in epidemic prediction and preparedness. to this end, we create a multilingual ee benchmarking dataset speed++ comprising 5k tweets spanning four languages and four diseases. to realistically deploy our models, we develop zero-shot cross-lingual cross-disease models and demonstrate their capability to extract events for 65 languages spanning 117 countries. we prove the effectiveness of our model by providing early epidemic warnings for covid-19 from chinese weibo posts in a zero-shot manner. we also show evidence that our framework can be utilized as an information aggregation system aiding in misinformation detection and public attention monitoring. in conclusion, we provide a strong utility of multilingual ee for global epidemic preparedness.