RESIN : A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System (2021)

Abstract: We present a new information extraction system that can automatically construct temporal event graphs from a collection of news documents from multiple sources, multiple languages (English and Spanish for our experiment), and multiple data modalities (speech, text, image and video). The system advances state-of-the-art from two aspects: (1) extending from sentence-level event extraction to cross-document cross-lingual cross-media event extraction, coreference resolution and temporal event tracking; (2) using human curated event schema library to match and enhance the extraction output. We have made the dockerlized system publicly available for research purpose at GitHub, with a demo video.

## introduction event extraction and tracking technologies can help us understand real-world events described in the overwhelming amount of news data, and how they are inter-connected. these techniques have already been proven helpful in various application domains, including news analysis @xcite @xcite , aiding natural disaster relief efforts @xcite @xcite , financial analysis @xcite @xcite @xcite and healthcare monitoring @xcite @xcite . however, it's much more difficult to remember event-related information compared to entityrelated information. for example, most people in 1 github: https://github.com/resin-kairos/resi n-pipeline-public 2 video: http://blender.cs.illinois.edu/softwa re/resin/resin.mp4 the united states will be able to answer the question "which city is columbia university located in?", but very few people can give a complete answer to "who died from covid-19?". progress in natural language understanding and computer vision has helped automate some parts of event understanding but the current, first-generation, automated event understanding is overly simplistic since most methods focus on sentence-level sequence labeling for event extraction. existing methods for complex event understanding also lack of incorporating knowledge in the form of a repository of abstracted event schemas (complex event templates), understanding the progress of time via temporal event tracking, using background knowledge, and performing global inference and enhancement. to address these limitations, in this paper we will demonstrate a new end-to-end open-source dockerized research system to extract temporally ordered events from a collection of news documents from multiple sources, multiple languages (english and spanish for our experiment), and multiple data modalities (speech, text, image and video). our system consists of a pipeline of components that involve schema-guided entity, relation and complex event extraction, entity and event coreference resolution, temporal event tracking and cross-media entity and event grounding. event schemas encode knowledge of stereotypical structures of events and their connections. our end-to-end system has been dockerized and made publicly available for research purpose. ## conclusions and future work we demonstrate a state-of-the-art schema-guided cross-document cross-lingual cross-media information extraction and event tracking system. this system is made publicly available to enable users to effectively harness rich information from a variety of sources, languages and modalities. in the future, we plan to develop more advanced graph neural networks based method for schema matching and schema-guided event prediction.