Event Schema Induction with Double Graph Autoencoders (2022)

Abstract: Event schema depicts the typical structure of complex events, serving as a scaffolding to effectively analyze, predict, and possibly intervene in the ongoing events. To induce event schemas from historical events, previous work uses an event-by-event scheme, ignoring the global structure of the entire schema graph. We propose a new event schema induction framework using double graph autoencoders, which captures the global dependencies among nodes in event graphs. Specifically, we first extract the event skeleton from an event graph and design a variational directed acyclic graph (DAG) autoencoder to learn its global structure. Then we further fill in the event arguments for the skeleton, and use another Graph Convolutional Network (GCN) based autoencoder to reconstruct entity-entity relations as well as to detect coreferential entities. By performing this two-stage induction decomposition, the model can avoid reconstructing the entire graph in one step, allowing it to focus on learning global structures between events. Experimental results on three event graph datasets demonstrate that our method achieves state-of-the-art performance and induces high-quality event schemas with global consistency.

## introduction event schema @xcite @xcite @xcite is induced from historical events to describe the common or stereotypical evolution pattern of complex events. figure 1b shows an example schema of complex event "ied-bombing" (improvised explosive device), where multiple events are inter-connected via temporal links (e.g., transport happens after as-semble) and arguments (e.g., the transporting en-tity is the weapon that is being assembled; the transporting destination is controlled by the assembler, i.e., the agent). it enables a descriptive analysis of inter-event structures, as well as the prediction of future events over temporal-based and argument-based structures. a number of methods have been proposed for learning event schemas from instance event graphs, called event schema induction, which fall into three categories. the set-based @xcite @xcite and sequence-based @xcite methods treat a complex event as a set or a linear sequence of atomic events, respectively. however, they fail to capture the multi-dimensional evolution of real-world complex events, i.e., multiple events may precede or follow one event. graph-based methods @xcite , instead, adopt graphs to formulate event schemas. a graph model is usually learned from instance event graphs through generating the schema event by event. although graphbased methods are theoretically superior to the first two categories, existing graph-based methods are limited to modeling only the local structure of event graphs, i.e., the first-order dependency of an event node with respect to its neighbors, while ignoring the high-order and global dependencies among atomic events in the entire graph. however, modeling the global structure of event graphs is crucial to event schema induction. the global structure enables the model to be aware of the position of each event node in the entire graph. it allows the model to better comprehend the role of a specific event in the complex event. for example, in figure 1b , there are three transport events in the schema, but they differ regarding the item being transported, i.e., bombs, the injured, or suspects. the global structure context enables the model to differentiate the position of the three transport events and predict their neighbor events precisely. moreover, when the model only has access to local the september 11 attacks were a series of four coordinated terrorist attacks. four commercial airliners traveling from the northeastern u.s. to california were hijacked mid-flight by 19 al-qaeda terrorists. each group had one hijacker who took over control of the aircraft. their explicit goal was to crash each plane into a prominent american building, causing mass casualties and partial or complete destruction of the targeted buildings. the attacks resulted in 2,977 fatalities, over 25,000 injuries, and substantial long-term health consequences. structure information during the generation process, the schema generated may only be consistent at the local level, but not at the global level. for example, the schema may keep repeating the sequence of assemble -transport -attack. from this perspective, the global structure information can be viewed as the supervision that guides the entire generation process to be globally consistent. to capture the global structure information of complex events, in this paper, we propose a new event schema induction approach using double graph autoencoders. graph autoencoder @xcite ) is known to be able to preserve the structural information of an entire graph in the embedding space. therefore, our key idea is to use graph autoencoders to capture the global dependency among nodes in event graphs. specifically, our model contains two graph autoencoders that are organized in a hierarchical manner: (1) to model the skeleton of an event graph, we design a high-level variational graph autoencoder for directed acyclic graphs (dags). event skeleton is a subgraph of an event graph, which consists of salient events and their temporal orders, representing the fundamental structure of event evolution. (2) with the event skeleton as the global context, we decorate entity nodes in the skeleton by introducing a low-level graph autoencoder based on graph convolutional network (gcn). it takes as input an expanded event skeleton and reconstructs the original event graph by adding coreferential entities and entity-entity relations. these two graph autoencoders decompose the process of event schema induction into two steps, thereby avoiding the need to reconstruct the entire graph directly, and improving the schema learning efficiently at both the high level (event schema skeleton) and the lower level (entity-entity relations). we conduct extensive experiments on three event graph datasets. the experimental results demonstrate that our proposed method achieves state-ofthe-art performance on event schema induction. additionally, we show in a case study that the event schema generated by our method is more reasonable and globally consistent. we make the following novel contributions: ## problem formulation our data resources come from news and wikipedia articles that describe a series of complex events. 𝒩(𝛍 , 𝚺 ) 𝒔 𝑮 instance graph 𝑮 event skeleton 𝑮 𝑺 reconstructed event skeleton 𝑮 𝑺 # expanded graph 𝑮 𝑬 expanded graph 𝑮 𝑬 # encoder decoder decoder encoder sample high-level variational graph autoencoder low-level gcn graph autoencoder reconstruction loss extract encode decode encode decode add arguments node embeddings adding entity-entity relations merging coreferential entities ## related work existing event schema induction methods can be classified into three categories: set-based schema, sequence-based schema, and graph-based schema. the set-based methods represent event triggers by a set without modeling their inter-relations @xcite @xcite @xcite @xcite @xcite , which can be regarded as atomic schema induction. in contrast, we aim to induce schemas for complex events involving multiple events. another line of work focuses on the sequencebased methods, which takes event-event relations into account, and orders event structures into sequences @xcite @xcite @xcite @xcite . instead of representing events as structures, some work treats events as natural language steps and induces schema knowledge through story ending prediction @xcite @xcite , machine reading comprehension @xcite , and schema goal-step prediction @xcite . instead of ignoring event structures or organizing events as simple sequences, we aim to capture the multi-dimensional evolution of events, as well as the structured connections. as a further step, researchers propose to use graphs to represent schemas @xcite @xcite , where each event can be followed by multiple alternative outcomes. @xcite introduce the concept of "complex event schema", a comprehensive graph schema consisting of both temporal orders and multi-hop argument relations, allowing time-, location-, and argument-based tracking of events. however, it adopts an auto-regressive graph generation model, which only models the first-order dependency of an event node with respect to its neighbors. in contrast, we propose to encode a global graph context via event skeleton generation using double auto-encoders. ## conclusion and future work in this work, we propose a novel event schema induction framework using double graph autoencoders, i.e., a high-level variational graph autoencoder to learn the event skeleton, followed by a low-level gcn graph autoencoder to reconstruct entity-entity relations. our autoencoders are able to preserve the global structural information of event graphs, thus capturing the multi-dimensional evolution of complex events, and providing global context to consolidate argument relations. experiments demonstrate that our method significantly outperforms baselines by generating high-quality and globally consistent event schemas. in the future, we aim to effectively induce schemas from graphs of different sizes, especially extremely large graphs. we also plan to make use of event hierarchies and induce hierarchical schemas with optimal event type granularity.