Massively Multi-Lingual Event Understanding: Extraction, Visualization, and Search (2023)

Abstract: In this paper, we present ISI-Clear, a state-of-the-art, cross-lingual, zero-shot event extraction system and accompanying user interface for event visualization & search. Using only English training data, ISI-Clear makes global events available on-demand, processing user-supplied text in 100 languages ranging from Afrikaans to Yiddish. We provide multiple event-centric views of extracted events, including both a graphical representation and a document-level summary. We also integrate existing cross-lingual search algorithms with event extraction capabilities to provide cross-lingual event-centric search, allowing English-speaking users to search over events automatically extracted from a corpus of non-English documents, using either English natural language queries (e.g. “cholera outbreaks in Iran”) or structured queries (e.g. find all events of type Disease-Outbreak with agent “cholera” and location “Iran”).

## introduction understanding global events is critical to understanding the world around us-whether those events consist of pandemics, political unrest, natural disasters, or cyber attacks. the breadth of events of possible interest, the speed at which surrounding socio-political event contexts evolve, and the complexities involved in generating representative annotated data all contribute to this challenge. events are also intrinsically global: many downstream use cases for event extraction involve reporting not just in a few major languages but in a much broader context. the languages of interest for even a fixed task may still shift from day to day, e.g. when a disease emerges in an unexpected location. the isi-clear (cross-lingual event & argument retrieval) system meets these challenges by building state-of-the-art, languageagnostic event extraction models on top of massively multi-lingual language models. these event models require only english training data (not even bitext-no machine translation required) and can identify events and the relationships between them in at least a hundred different languages. unlike more typical benchmark tasks explored for zero-shot cross-lingual transfer-e.g. named entity detection or sentence similarity, as in @xcite )-event extraction is a complex, structured task involving a web of relationships between elements in text. isi-clear makes these global events available to users in two complementary ways. first, users can supply their own text in a language of their choice; the system analyzes this text in that native language and provides multiple event-centric views of the data in response. second, we provide an interface for cross-lingual event-centric search, allowing english-speaking users to search over events automatically extracted from a corpus of non-english documents. this interface allows for both natural language queries (e.g. statements by angela merkel about ukraine) or structured queries (event @xmath0, protest}, @xmath1, and builds upon our existing cross-lingual search capabilities, demonstrated in @xcite . the primary contributions of this effort are threefold: ## ontology & training data the isi-clear demo system is compatible with any event ontology that identifies a set of event types and argument roles. the system expects sentence-level english training data that identifies, for each event, one or more anchor spans and zero or more argument spans (with roles). for this demonstration, we use the "basic event" ontology and data developed for the iarpa better program (available at https://ir.nist. gov/better/ ). the ontology consists of 93 event types and a small set of argument roles (agent, patient, and related-event). in other settings, we have trained and tested the underlying system on the publicly available ace event ontology foot_1 , showing stateof-the-art zero-shot cross-lingual results in @xcite . we prefer the better ontology for this demo because of its broad topical coverage and its inclusion of event-event relations (in the form of related-event arguments). the isi-clear system is also designed to attach general-purpose when and where arguments to any event, regardless of ontology; see section 4.5. ## system components we present here the highlights of our technical approach, which relies on a collection of strong, language-agnostic models to perform all aspects of event extraction and the classification of relationships between events, as well as machine translation and foreign-to-english projection of event output (for display purposes). ## system evaluation & analysis we evaluate our system on a variety of languages and ontologies and compare where possible to existing baselines. following community practice, e.g. @xcite , we consider an anchor correct if its offsets and event type are correct, and we consider an argument correct if its offsets, event type, and role find a match in the ground truth. for event coreference (same-sentence only), we consider each anchor pair separately to produce an overall f-score. it is difficult to compare system accuracy across languages; a lower score in one language may reflect a real difference in performance across languages-or just that one set of documents is harder than another. still, we observe the following. first, performance on anchors seems most sensitive to language choice-for instance, we note that arabic and chinese anchor performance on ace differs by almost 10 points. for arguments, however, non-english performance is relatively consistent given a task-but varies more widely between tasks. second, we note that cross-lingual performance seems best on anchors, where it exceeds 80% of english performance for all but one condition. in contrast, argument performance varies more widely, with many conditions below 70% of english (though some as high as 89%). we also compare against existing published baselines where possible. there are relatively few published results on cross-lingual event anchor detection (and none that we could find on the task of for argument detection, much more published work exists, and we show in ## conclusion isi-clear provides a monolingual englishspeaking user with effective access to global events, both on-demand (extracting events from input of a user's choice) or as a set of indexed documents accessible via cross-lingual search. the system provides a variety of visualizations and modes for engaging with system results. we look forward to future work improving the quality of the underlying components and exploring additional capabilities to cross language barriers and expand access to information around the globe.