DCT -Centered Temporal Relation Extraction (2022)

Abstract: Most previous work on temporal relation extraction only focused on extracting the temporal relations among events or suffered from the issue of different expressions of events, timexes and Document Creation Time (DCT). Moreover, DCT can act as a hub to semantically connect the other events and timexes in a document. Unfortunately, previous work cannot benefit from such critical information. To address the above issues, we propose a unified DCT-centered Temporal Relation Extraction model DTRE to identify the relations among events, timexes and DCT. Specifically, sentence-style DCT representation is introduced to address the first issue and unify event expressions, timexes and DCT. Then, a DCT-aware graph is applied to obtain their contextual structural representations. Furthermore, a DCT-anchoring multi-task learning framework is proposed to jointly predict three types of temporal relations in a batch. Finally, we apply a DCT-guided global inference to further enhance the global consistency among different relations. Experimental results on three datasets show that our DTRE outperforms several SOTA baselines on E-E, E-T and E-D significantly.

## introduction temporal relation extraction focuses on the occurrence order (tlink) of event mentions, time expressions (timexes) and document creation time (dct). most previous studies only focus on the event-centered tasks and consider three tlinks: event-event (e-e), event-timex (e-t), and event-dct (e-d). as a crucial component of relation extraction, temporal relation extraction can help many downstream nlp tasks, such as question answering @xcite , summarization @xcite and timeline construction @xcite . most previous studies @xcite only focus on the single e-e task, ignoring the other e-t and e-d tasks. the main barricade is the hardness of combining the e-t or e-d task with the e-e task due to their different expression forms. since most event mentions foot_0 are sentences or clauses, their rich information is helpful for a neural network model to identify the specific relation between two event mentions. however, timex and dct (especially dct) are only word-level or phrase-level tokens, and the information imbalance between events and timexes/dct will lead to the issue that it is difficult for a neural network model to extract their correct temporal relation by a unified model. as shown in figure 1 , identifying the temporal relation between the two long-distance event mentions e2 and e9 is challenging, even for humans. however, if we first recognize the is included and before links of (e2, dct) and (e9, dct), then the after link of (e2, e9) will be much easier to obtain. fortunately, identifying the temporal relation between event (or timex) and dct is relatively simple for humans and pre-trained language models. since event, timex and dct have different forms of expressions, most previous studies @xcite only focus on e-e or regarded e-e, e-t and e-d as three independent tasks, as we mentioned above. they often suffer from the issues of data scarcity and overfitting due to the small size of available datasets. @xcite propose a model to fuse e-e, e-t and e-d into a unified model by introducing a learnable parameter-based dct representation. however, they still suffer from two issues: 1) the different expressions of event, timex and dct harm the information interaction among different tasks; 2) they ignore the importance of dct to connect the events and timexes in a document. to address the above issues, we propose a unified dct-centered temporal relation extraction (dtre) model to discover the temporal relations among events, timexes and dct in a document. specifically, we first introduce a sentence-style dct representation to unify the expressive forms of event, timex, and dct. then, we generate a dct-aware graph to obtain their contextual structural representations. furthermore, we propose a dct-anchoring multi-task learning framework to jointly predict three temporal relations (i.e., e-e, e-t and e-d). finally, we provide a dct-guided global inference mechanism to benefit from the high accuracies of the e-d and t-d tasks. experimental results on three popular datasets tbd, tdd-man and tdd-auto show that our dtre outperforms several sota baselines on all three tasks e-e, e-t and e-d significantly. in summary, our contributions are as follows: ## related work timebank @xcite is an early temporal relation corpus and its extended version timebank-dense (tbd) (cassidy et al., 2014) adopts a dense annotation scheme in a slide window within adjacent sentences, in which there are mainly five types of tlinks: e-e, e-t, t-t, e-d and t-d. matres (ning et al., 2018) only contains e-e relations and simplifies the relation labels with a higher inter-annotator agreement. tddiscourse (naik et al., 2019) is a discourse-level temporal relation dataset based on tbd, which also focuses on e-e temporal ordering. early work on temporal relation extraction @xcite @xcite @xcite focused on various linguistic features, including part-of-speech (pos), lexical and morphological features, dependency parsing information, etc. recent work mainly focuses on the e-e task using neural networks. @xcite and @xcite show that graphbased neural networks can help relieve informative sparsity between long-distance event mentions in discourse-level temporal relation extraction. besides, a bunch of efforts focus on incorporating external resources to deal with the limited training resource, such as combining the pre-trained temporal-aware language model @xcite , collecting the distantly-supervised examples @xcite , applying the transfer learning methods to leverage complementary datasets @xcite . other methods seek to enhance global inference with structural constraints, i.e., relieving the transitive conflicts within temporal graphs @xcite . only a few studies consider all three tasks. early methods were mostly rule-based on event attributes @xcite , whose performance are deeply harmed by the vague relation, or simply transferred the neural architecture on e-e to e-d and e-t directly via input adjustment @xcite . motivated by the success of multi-task deep neural network @xcite that leverages different supervised learning tasks with the shared contextual embeddings, @xcite proposed a multi-category learning framework to joint e-e, e-t and e-d. specially, they introduce a learnable vector to represent dct as it does not explicitly occur in documents. ## dtre: dct-centered temporal relation extraction the temporal relations between event mentions are determined by their occurrence intervals (i.e., start and end points). however, in most real-world texts, events' intervals are rarely explicitly mentioned, and then external knowledge or common sense is required for temporal reasoning. one important clue is "had this event happened yet?" or "is this a future event?", i.e., the e-d task, which is easier and helpful for the e-e and e-t tasks. to fully exploit the dct representation and its bridge function to connect events and timexes, we propose a dct-centered temporal relation extraction model dtre to discover the temporal relations among events, timexes and dct. figure 2 illustrates the overview of our dtre framework. we first combine the sentence-style dct representation with the original document as the input of the pre-trained model (bert or roberta) to obtain the mention embeddings of the different types. then, we build a dct-aware graph dag for each document to obtain the contextual structural representations of events, timexes, dct, etc. furthermore, we conduct a dct-anchoring multi-task learning framework daml to jointly predict the three tasks of temporal relation extraction. finally, we introduce a dct-guided global inference mechanism dgi to our model according to the high accuracies of the e-d and t-d tasks. ## experimentation in this section, we first introduce the datasets and the experimental settings, and then report results on our proposed dtre and baselines. ## ablation study in this section, we conduct the ablation study of dtre (roberta-based) on tbd as examples. it is worth mentioning that bert-based dtre on tbd and tdd also show the similar results and we do not describe here for simplification. ## conclusion in this paper, we proposed a unified dct-centered temporal relation extraction model dtre to discover the relations among events, timexes and dct. specifically, we first introduce sentence-style dct to unify the expressions of event, timex and dct. then, we apply a dct-aware graph to obtain their contextual structural representations. furthermore, we propose a dct-anchoring multi-task framework to jointly predict three tasks of temporal relation extraction in a batch. finally, we provide a dct-guided global inference to further enhance the global consistency among different relations. experimental results on three popular datasets show that our dtre outperforms several sota baselines on e-e, e-t and e-d significantly. our future work will focus on discovering effective graph structure and inference mechanism for temporal relation extraction.