Toward Consistent and Informative Event-Event Temporal Relation Extraction (2023)

Abstract: Event-event temporal relation extraction aims to extract the temporal order between a pair of event mentions, which is usually used to construct temporal event graphs. However, event graphs generated by existing methods are usually globally inconsistent (event graphs containing cycles), semantically irrelevant (two unrelated events having temporal links), and context unaware (neglecting neighborhood information of an event node). In this paper, we propose a novel event-event temporal relation extraction method to address these limitations. Our model combines a pretrained language model and a graph neural network to output event embeddings, which captures the contextual information of event graphs. Moreover, to achieve global consistency and semantic relevance, (1) event temporal order should be in accordance with the norm of their embeddings, and (2) two events have temporal relation only if their embeddings are close enough. Experimental results on a real-world event dataset demonstrate that our method achieves state-of-the-art performance and generates high-quality event graphs.

## introduction event-event temporal relation extraction aims to extract the temporal order between a pair of event mentions in natural language text (i.e., an event is before or after another event), which is essential for constructing temporal event graphs. event-event temporal relation extraction enables researchers to understand the dynamics of complex events, and benefits a variety of downstream tasks, including event graph construction @xcite , future event prediction @xcite @xcite , question answering @xcite , and summarization @xcite . researchers have proposed many methods @xcite @xcite to tackle this challenging task. previous work usually formulates the problem as a pairwise classification task @xcite @xcite . however, they have three major issues when applied to constructing the temporal event graph: in this paper, we propose a new event-event temporal relation extraction approach that addresses the above limitations of existing methods. the goal of our approach is to learn event representations that are globally consistent, semantically relevant, and context-aware. as shown in figure 2 , given an input document as well as the entity mentions, we first use off-the-shelf information extraction tools @xcite to extract arguments of events. we then use a pretrained language model (plm, @xcite to encode events/arguments and get their plm-based embeddings. to allow events to be aware of their contextual information, we construct an initial event graph consisting of events/arguments as nodes and eventargument links as edges, then use a graph neural network @xcite to aggregate neighborhood information for each node iteratively and get their gnn-based embeddings. the plmbased and gnn-based embeddings are combined together as the final embeddings of events. moreover, to ensure that the learned event embeddings are globally consistent and semantically relevant, we hypothesize that the event embedding space should be geometrically meaningful, in which event embeddings satisfy the following two rules: (1) the temporal order of events is in accordance with the norm of event embeddings. specifically, if event a happens before event b, then the embedding norm of event a should be smaller than that of event b. (2) there exists a temporal link between two events if and only if their embeddings are close enough to each other in the event embedding space. specifically, if events a and b are connected by a temporal edge (either a happens before b or after b), then the distance between a's and b's embedding should be smaller than a predefined threshold, and vice versa. the first rule ensures that the constructed event graph is globally consistent, and the second rule ensures that there will be a temporal link between two events only if they are semantically relevant. we implement these two rules in our model by minimizing a corresponding margin-based loss w.r.t the model parameters, thus the whole model can be trained in an end-to-end fashion. we conduct experiments on the event story line dataset @xcite . the experimental results demonstrate that our proposed method achieves state-of-the-art performance on event-event temporal relation extraction. we also show that compared with baseline methods, event graphs generated by our method are globally consistent and semantically relevant. in summary, our contributions are as follows: ## our approach the overall framework of our proposed approach is shown in figure 2 . in this section, we will introduce each part of the framework in detail. ## related work event-event temporal relation extraction can be viewed as a classification task that predicts the relation type between two event mentions. in general, existing event-event temporal relation extraction methods can be classified into two categories: traditional rule-based methods and neural network based methods. the traditional rule-based methods apply linguistic rules to the features extracted from documents to predict the relation between a given event pair. for example, @xcite propose a sys-tem that uses a rule-based approach as baseline to determine temporal links and a machine learning classifier to filter out baseline candidates. @xcite design a sieve-based architecture caevo that applies a sequence of temporal relation classifiers to label event-event temporal relations. this supports a combination of both rule-based and machine learned classifiers. however, these rule-based methods require substantial manual design of rules, which greatly limits their usage in practice. moreover, rules are usually not comprehensive enough to capture the complex event-event temporal relations. another line of related work focuses on the neural network based methods, which extracts eventevent temporal relations via deep neural networks and pre-trained language models. for example, @xcite introduce a joint constrained learning framework that incorporates contextual features encoded with pre-trained language models and external knowledge from commonsense knowledge bases. @xcite adopt a stackpropagation framework to combine relative time prediction and event-event temporal relation classification. however, they do not consider global consistency and semantic relevance of the generated event graphs. ## conclusion and future work in this paper, we propose a globally consistent, semantically relevant, and context aware event-event relation extraction framework, which addresses the limitations of existing methods. our model uses a pretrained language model module and graph neural network module to jointly represent event graphs. in addition, we make the event embedding space geometrically meaningful by imposing two constraints on event embeddings: event temporal order should be in accordance with event embedding norm, and event temporal relations should only exist between events whose embeddings are close enough. experiments demonstrate that our method significantly outperforms baselines by generating accurate and globally consistent temporal event graphs. in the future, we aim to incorporate external background knowledge and commonsense knowledge into our framework. we also plan to make use of the generated temporal event graphs in downstream tasks, such as future event prediction and question answering.