The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction (2021)

Abstract: Event schemas encode knowledge of stereotypical structures of events and their connections. As events unfold, schemas are crucial to act as a scaffolding. Previous work on event schema induction focuses either on atomic events or linear temporal event sequences, ignoring the interplay between events via arguments and argument relations. We introduce a new concept of Temporal Complex Event Schema: a graph-based schema representation that encompasses events, arguments, temporal connections and argument relations. In addition, we propose a Temporal Event Graph Model that predicts event instances following the temporal complex event schema. To build and evaluate such schemas, we release a new schema learning corpus containing 6,399 documents accompanied with event graphs, and we have manually constructed gold-standard schemas. Intrinsic evaluations by schema matching and instance graph perplexity, prove the superior quality of our probabilistic graph schema library compared to linear representations. Extrinsic evaluation on schema-guided future event prediction further demonstrates the predictive power of our event graph model, significantly outperforming human schemas and baselines by more than 17.8% on HITS@1.

## introduction the current automated event understanding task has been overly simplified to be local and sequential. real world events, such as disease outbreaks and terrorist attacks, have multiple actors, complex timelines, intertwined relations and multiple possible outcomes. understanding such events requires knowledge in the form of a library of event schemas, capturing the progress of time, and performing global inference for event prediction. for example, regarding the 2019 protest in hong kong international airport, a typical question from analysts would be "how long will the flights being canceled?" this requires an event understanding system to match events to schema representations and reason about what might happen next. the airport protest schema would be triggered by "protest" and "flight cancellation", and evidence of protesters (e.g., the number of protesters, the instruments being used, etc) will suggest a ceo resignation event, or a flight rescheduling event, or continuous flight cancellation events with respective probabilities. a person learned to make bombs and bought materials as well as a vehicle. then the bomb was assembled to the vehicle, and then the attacker drove it to attack people. people can be hurt by the vehicle, or by the explosion of the bomb, or by the crash of the vehicle. comprehending such a news story requires following a timeline, identifying key events and tracking characters. we refer to such a "story" as a complex event, e.g., the kabul ambulance bombing event. its complexity comes from the inclusion of multiple atomic events (and their arguments), relations and temporal order. a complex event schema can be used to define the typical structure of a particular type of complex event, e.g., carbombing. this leads us to the new task that we address in this paper: temporal complex event schema induction. figure 1 shows an example schema about car-bombing with multiple temporal dependencies between events. namely, the occurrence of one event may depend on multiple events. for example, the assemble event happens after buying both the bomb materials and the vehicle. also, there may be multiple events following an event, such as the multiple consequences of the attack event in figure 1 . that is to say, "the future is not one-dimensional". our automatically induced probabilistic complex event schema can be used to forecast event abstractions into the future and thus provide a comprehensive understanding of evolving situations, events, and trends. for each type of complex event, we aim to induce a schema library that is probabilistic, temporally organized and semantically coherent. low level atomic event schemas are abundant, and can be part of multiple, sparsely occurring, higher-level schemas. we propose a temporal event graph model, an auto-regressive graph generation model, to reach this goal. given a currently extracted event graph, we generate the next event type node with its potential arguments, such as the arrest event in figure 2 , and then propagate edge-aware information following temporal orders. after that, we employ a copy mechanism to generate coreferential arguments, such as the detainee argument is the attacker of the previous attack event, and build relation edges for them, e.g., part whole relation between the place arguments. finally, temporal dependencies are determined with argument connections considered, such as the temporal edge showing that arrest is after attack. our generative model serves as both a schema library and a predictive model. specifically, we can probe the model to generate event graphs unconditionally to obtain a set of schemas. we can also pass partially instantiated graphs to the model and "grow" the graph either forward or backward in time to predict missing events, arguments or relations, both from the past and in the future. we propose a set of schema matching metrics to evaluate the induced schemas by comparing with human-created schemas and show the power of the probabilistic schema in the task of future event prediction as an extrinsic evaluation, to predict event types that are likely to happen next. we make the following novel contributions: ## problem formulation from a set of documents describing a complex event, we construct an instance graph g which contains event nodes e and entity nodes (argument nodes) v . there are three types of edges in this graph: (1) event-event edges e i , e l connecting events that have direct temporal relations; (2) evententity edges e i , a, v j connecting arguments to the event; and (3) entity-entity edges v j , r, v k indicating relations between entities. we can construct instance graphs by applying information extraction (ie) techniques on an input text corpus. in these graphs, the relation edges do not have directions but temporal edges between events are directional, going from the event before to the event after. for each complex event type, given a set of instance graphs g, the goal of schema induction is to generate a schema library s. in each schema graph s, the nodes are abstracted to the types of events and entities. figure 1 is an example of schema 2 for complex event type car-bombing. schema graphs can be regarded as a summary abstraction of instance graphs, capturing the reoccurring structures. ## related work the definition of a complex event schema separates us from related lines of work, namely schema induction and script learning. previous work on schema induction aims to characterize event triggers and participants of individual atomic events @xcite @xcite @xcite , ignoring inter-event relations. work on script learning, on the other hand, originally limited attention to event chains with a single protagonist @xcite @xcite @xcite and later extended to multiple participants @xcite @xcite . recent efforts rely on distributed representations encoded from the compositional nature of events @xcite @xcite @xcite , and language modeling @xcite @xcite . all of these methods still assume that events follow linear order in a single chain. they also overlook the relations between participants which are critical for understanding the complex event. however, we induce a comprehensive event graph schema, capturing both the temporal dependency and the multi-hop argument dependency across events. recent work on event graph schema induction @xcite only considers the connections between a pair of two events. similarly, their event prediction task is designed to automatically generate a missing event (e.g., a word sequence) given a single or a sequence of prerequisite events @xcite @xcite @xcite , or predict a pre-condition event given the current events @xcite . in contrast, we leverage the automatically discovered temporal event schema as guidance to forecast the future events. existing script annotations @xcite @xcite @xcite cannot support a comprehensive graph schema induction due to the missing of critical event graph structures, such as argument relations. furthermore, in real-world applications, complex event schemas are expected to be induced from large-scale historical data, which is not feasible to annotate manually. we propose a data-driven schema induction approach, and choose to use ie systems instead of using manual annotation, to induce schemas that are robust and can tolerate extraction errors. our work is also related to recent advances in modeling and generation of graphs @xcite @xcite @xcite @xcite @xcite @xcite . we are the first to perform graph generation on event graphs. ## conclusions and future work we propose a new task to induce temporal complex event schemas, which are capable of representing multiple temporal dependencies between events and their connected arguments. we induce such schemas by learning an event graph model, a deep auto-regressive model, from the automatically extracted instance graphs. experiments demonstrate the model's effectiveness on both intrinsic evaluation and the downstream task of schema-guided event prediction. these schemas can guide our understanding and ability to make predictions with respect to what might happen next, along with background knowledge including location-, and participant-specific and temporally ordered event information. in the future, we plan to extend our framework to hierarchical event schema induction, as well as event and argument instance prediction.