A Diffusion Model for Event Skeleton Generation (2023)

Abstract: Event skeleton generation, aiming to induce an event schema skeleton graph with abstracted event nodes and their temporal relations from a set of event instance graphs, is a critical step in the temporal complex event schema induction task. Existing methods effectively address this task from a graph generation perspective but suffer from noise-sensitive and error accumulation, e.g., the inability to correct errors while generating schema. We, therefore, propose a novel Diffusion Event Graph Model (DEGM) to address these issues. Our DEGM is the first workable diffusion model for event skeleton generation, where the embedding and rounding techniques with a custom edge-based loss are introduced to transform a discrete event graph into learnable latent representations. Furthermore, we propose a denoising training process to maintain the model’s robustness. Consequently, DEGM derives the final schema, where error correction is guaranteed by iteratively refining the latent representations during the schema generation process. Experimental results on three IED bombing datasets demonstrate that our DEGM achieves better results than other state-of-the-art baselines. Our code and data are available at https://github.com/zhufq00/EventSkeletonGeneration .

## introduction event schema induction is to identify common patterns and structures in event data, which can extract high-level representation of the events. current event schema induction tasks mainly focus on simple event schemas, e.g., templates @xcite and scripts @xcite . however, real-world events are usually more complex, which include multiple atomic events, entities, and their relations, which require more advanced schema figure 1: an illustrated example demonstrates the utilization of multiple instance graphs extracted from news articles depicting complex events to generate an event schema skeleton graph for the complex event type car bombing. the presented instance graph specifically represents the complex event known as the kabul ambulance bombing. a circle symbolizes an atomic event. recently, @xcite propose the temporal complex event schema induction task in order to understand these complex events. the task seeks to abstract a general evolution pattern for complex events from multiple event instance graphs. it is divided into two subtasks: event skeleton generation and entity-entity relation completion. the first task focuses on creating the event skeleton, i.e., representing each atomic event with its associated event type as an event node and exploring their temporal relations. the second one is to complete entities and entity links for the event skeleton. in this paper, we focus on event skeleton generation as it is a prerequisite yet formidable task in temporal complex event schema induction. figure 1 illustrates an example of instance graphs foot_0 and the corresponding abstracted schema. both include abstract event types, such as attack, and their temporal relations, like injure happening after attack. event skeleton generation requires a deep understanding of events and their multi-dimensional relations. previous methods employ autoregressive graph generation models to generate a schema, sequentially generating event nodes from the previous ones. for example, @xcite generate the event node with its potential arguments and propagates edge-aware information within the temporal orders. @xcite improves this approach by applying a graph convolutional network (gcn) to better capture structural information in instance graphs and adopting a similar autoregressive generation approach to generate event graphs. however, autoregressive generation methods for event skeleton generation result in errors accumulating over time, which may degrade the performance of the generation model. for instance, as shown in figure 1 , the model may mistakenly generate "explode" as "die", causing it to fail to generate subsequent events correctly. intuitively, as the number of event nodes increases, the error accumulation becomes more severe. this comes from two factors. the first one is error propagation in the autoregressive graph generation models because they are noisesensitive and strongly rely on the correctness of the generated node. if the model generates an incorrect node, it will lead to a cascading effect of errors in generating the schema. robustness is a serious issue in autoregressive methods. the second factor is the model's inability to correct errors in the generation procedure. hence, we need a model, which can correct the generated event-type nodes during generating. to this end, we propose a novel event graph generation model, dubbed diffusion event graph model (degm), to address these issues. to battle the model's robustness, we propose a diffusionbased method, inspired by the outstanding performance in recent research @xcite . by carefully selecting the amount of gaussian noise in the diffusion process, the model can remove adversarial perturbations, thereby increasing the model's robustness. however, there are still two challenges in applying this method directly to the event graph: (1) mapping the discrete graph structures and event types to a continuous space, and (2) finding a way to recover the event graph from the continuous space. we then develop the denoising stage, including converting the event graph into a sequence and applying an embedding technique to project it to the continuous space. additionally, we introduce a custom edge-based loss function to capture the missing structural information during the transformation. to tackle the second challenge, we develop a rounding technique to predict the event types based on their representation and a pre-trained classifier to predict the event edges. to address the second issue, we derive the final schema, which guarantees error correction, by iteratively refining the latent representation. we summarize our contributions as follows: ## method we propose diffusion event graph model (degm) to tackle the event skeleton generation task. our degm is capable of generating temporal event graphs from random noise. fig. 2 illustrates an overview of our degm. ## related work according to @xcite , event schema induction can be divided into three categories: (1) atomic event schema induction @xcite @xcite @xcite has focused on inducing an event template, called atomic event schema, for multiple similar atomic events. the template includes an abstracted event type and a set of entity roles shared by all atomic events, while ignoring the relations between events. (2) narrative event schema induction @xcite @xcite @xcite @xcite @xcite , in contrast, pays attention to the relations between events. in this task, schema is defined as a narrativeordered sequence of events, with each event including its entity roles. however, complex events in real-world scenarios often consists of multiple events and entities with innerwined relations. to under such complex events, @xcite incorporate graph structure into schema definition. however, they only consider the relations between two events and their entities. (3) temporal complex event induction task, recently, @xcite propose this task in which a schema consists of events, entities, the temporal relations between events, relations between entities, and relations between event and entity (i.e., argument). each event and entity is abstracted as an event type or entity type, and each event type contains multiple predefined arguments associated with entities. to address this issue, @xcite generates the schema event by event. each time an event is generated, the model links it to existing events, expands it with predefined arguments and entities, and links the entities to existing nodes. this approach leads to the entities' inability to perceive the events' position, resulting in entities cannot distinguish between events of the same type. therefore @xcite divide the task into two stages: event skeleton generation and entity-entity relation completion. in the first stage, they employ an autoregressive directed graph generation method @xcite to generate the schema skeleton, including events and their relations. in the second stage, they expand the schema skeleton with predefined arguments and entities and complete the remaining relations vis a link prediction method vgae @xcite . the above event graph induction methods suffer from error accumulation due to the limitations of the autoregressive schema generation paradigm. to address this issue, we propose degm which utilizes a denoising training process to enhance the model's robustness to errors and a schema generationt process to continuously correct the errors in the generated schema. ## conclusions we propose diffusion event graph model, the first workable diffusion model for event skeleton generation. a significant breakthrough is to convert the discrete nodes in event instance graphs into a continuous space via embedding and rounding techniques and a custom edge-based loss. the denoising training process improves model robustness. during the schema generation process, we iteratively correct the errors in the schema via latent representation refinement. experimental results on the three ied bombing datasets demonstrate that our approach achieves better results than other state-of-the-art baselines.