RESIN -11: Schema-guided Event Prediction for 11 Newsworthy Scenarios (2022)

Abstract: We introduce RESIN-11, a new schema-guided event extraction&prediction framework that can be applied to a large variety of newsworthy scenarios. The framework consists of two parts: (1) an open-domain end-to-end multimedia multilingual information extraction system with weak-supervision and zero-shot learningbased techniques. (2) schema matching and schema-guided event prediction based on our curated schema library. We build a demo website based on our dockerized system and schema library publicly available for installation ( https://github.com/RESIN-KAIROS/RESIN-11 ). We also include a video demonstrating the system.

## introduction if the evening news discusses a migration of people due to drought and the infrastructure cannot handle the population influx, what will happen next? while annotated datasets have fueled progress in machine intelligence, the knowledge required for event forecasting is vast and potentially ambiguous. for example, to predict that a rebellion is likely in the future, models need to integrate background events (drought), abstractions (strained infrastructure causes unrest), and event schemas (structure and duration of rebellion). existing link prediction @xcite @xcite or knowledge graph completion methods @xcite @xcite cannot meet this goal because the event instance graphs extracted from news are often incomplete and noisy. recent work @xcite proposes to leverage complex event 1 demo: http://18.221.187.153:11000 2 github: https://github.com/resin-kairos/resi n-11 3 video: https://screencast-o-matic.com/watch /c3nlhnvbeyg schemas (stereotypical evolution pattern of complex events) for event prediction. however, these methods are often limited to a few scenarios due to the lack of high-quality, open-domain information extraction systems to construct event instance graphs needed for schema induction. to tackle these challenges, in this paper we use the event schemas discovered at scale to guide the learning of predictive models. we first identify 11 newsworthy scenarios, and construct comprehensive hierarchical schemas through the combination of automatic schema induction and manual curation. then we develop an open-domain end-to-end multimedia multilingual information extraction system that can extract entities, relations, events, and temporal orders for all of these scenarios, based on a series of weak supervision based methods, including few-shot learning and lifelong learning. compared with previous event tracking systems @xcite that conduct graph matching on a linearized sequence, we propose a new schema matching algorithm that directly operates on graphs. we also proposed a event prediction model trained with self-supervision to predict possible missing events to form a coherent story. our contributions include: ## methodology overview figure 1 illustrates the overall architecture of our framework. it includes three parts: (1) schema library construction (section 2.1); on the ie side, we assume our input consists of multilingual multimedia document clusters about a specific scenario (e.g., disease outbreak). each document cluster contains documents about a specific complex event scenario (e.g., covid-19 pandemic). our textual ie pipeline takes documents and transcribed speech as input and extracts entity, relation and event mentions (section 2.2). in order to extend ie to open-domain, we have adopted weak supervision and zero-shot transfer learning techniques. then we perform cross-document cross-lingual entity and event coreference resolution, and link them to wikidata. the extracted events are then ordered by temporal relation extraction. our visual pipeline extracts events and arguments from visual signals (i.e., images and videos), and link the extracted knowledge elements to our extracted graph using cross-media event coreference resolution (section 2.3). finally, our system automatically selects the schema from a schema library that best matches the extracted ie graph and new events are predicted (section 2.4). ## schema and experiments the overall statistics of our schema library are presented in ## related work event schema induction and curation event schemas, or otherwise known as scripts, are structures that represent typical event progressions @xcite . prior to this work, there has been some effort in creating schema component benchmark metric score weakly-supervised ie trigger ace f1 63.3 argument ace f1 41.5 mention extraction en trigger ace+ere f1 64.1 argument ace+ere f1 49.7 relation ace+ere f1 49.5 es trigger ace+ere f1 63.4 argument ace+ere f1 46.0 relation ace+ere f1 46.6 document-level argument extraction ace f1 66.7 rams f1 48.6 coreference resolution en entity ontonotes conll 92.4 event ace conll 84.8 es entity semeval 2010 conll 67.6 event ere-es conll 81.0 wikidata qnode linking tackbp-2010 acc. 90.9 temporal ordering roberta matres f1 81.7 t5 matres-b acc. 89.6 visual event extraction m2e2 f1 52.7 cross-media event coreference video m2e2 f1 51.5 benchmark metric score schema matching ldc2022e03 recall 63.2 schema prediction wikievents f1 45.5 in addition to schema resources, there has also been work on automating the schema induction process, through the use of probabilistic graphical models @xcite @xcite and eventbased language models @xcite @xcite @xcite . we hope that our schema library can serve as a resource for the development of better automatic schema induction methods. weakly-supervised event extraction due to the high cost of annotating event instances, low resource event extraction has received much attention in recent years. there are a variety of settings explored, including zero-shot transfer learning @xcite , cross-lingual trans-fer @xcite , inducing event types @xcite , keyword-based supervision @xcite and few-shot learning @xcite @xcite @xcite . ## conclusions and future work we build an open-domain schema-guided event prediction system that is capable of extracting and predicting structured information regarding events from various scenarios. in the future, we plan to further improve both the extraction quality and portability to cover even more scenarios, and use the automatic zero-shot schema induction algorithm to iteratively extend our curated schemas. the hierarchy structure of our event schemas can also be further utilized to improve future event prediction.