TIMERS : Document-level Temporal Relation Extraction (2021)

Abstract: We present TIMERS - a TIME, Rhetorical and Syntactic-aware model for document-level temporal relation classification in the English language. Our proposed method leverages rhetorical discourse features and temporal arguments from semantic role labels, in addition to traditional local syntactic features, trained through a Gated Relational-GCN. Extensive experiments show that the proposed model outperforms previous methods by 5-18% on the TDDiscourse, TimeBank-Dense, and MATRES datasets due to our discourse-level modeling.

## introduction temporal relation extraction (temprel) is a challenging task that involves determining the temporal order between two events in a text @xcite . understanding the temporal ordering of events in a document plays a key role in downstream tasks such as timeline creation @xcite , time-aware summarization @xcite , temporal question-answering @xcite , and temporal information extraction @xcite . prior work focuses on extracting temporal relations between event pairs (a.k.a., tlinks) present in the same sentence (intra-sentence tlinks) or adjacent sentences (inter-sentence tlinks), mostly ignoring document-level pairs (crossdocument tlinks) @xcite . past works have used rnn @xcite @xcite @xcite and transformer networks @xcite for encoding a few sentences or a short paragraph but do not capture longrange dependencies and multi-hop reasoning at the document-level. this shortcoming is shown in the tddiscourse dataset @xcite , which was designed to highlight global discourse-level challenges, e.g., multi-hop chain reasoning, future or hypothetical events, and reasoning requiring world knowledge. we propose timers -a time, rhetorical, and syntactic-aware model for document-level temporal relation extraction. timers uses discourse features in the form of connections from rhetorical structure theory (rst) parsers @xcite to leverage long-range inter-sentential relationships. it also extends existing contextual embeddings with structural and syntactic dependency parse connections. lastly, it uses timextimex relations, dct (document creation date)-timex relations, and temporal arguments obtained via sentence-level semantic role labeling. these rhetorical, syntactic, and temporal features are learned through a modified version of relational graph convolutional networks (r-gcn) with a gating mechanism (gr-gcn) @xcite , which learns highly relational data relationships in densely-connected graph networks. our main contribution is a document-level model that incorporates these three features to improve temporal relationship extraction. we obtain state-of-the-art performance across three datasets with 5-18% relative improvement, showing improvement for events that require chain reasoning, causal prerequisite links, and future events. ## methodology and q timexes @xmath0, • • • , t q }, where p, @xmath1. the creation date of the document is represented by timestamp t dct . we denote the source and target events by e s and e t , respec-+ @xmath2. [despite the military conflict with iraq,] 1 [oil prices have been falling,] 2 [because of a worldwide glut of oil] 3 [and recession.] 4 3. [oil prices] 5 [have come down] 6 [from the middle of october] 7 [as we see today.] 8 4. [that's built up a roll up] 9 [and that it will look to increase.] 10 t 0 t 1 e 1 t 2 e 2 t 3 e 3 t 4 e p t q e t v w w w h event node timex node sentence node word node target node source node edu node document node tdct g tg after after before after after before elaboration condition reason contrast label weighted unweighted directed undirected spanext dct-timex timex-timex rst feature srl temporal argument dependency parse bert encoder @xmath3, gr-gcn gr-gcn gr-gcn after before equal include vague ........... ........... ............... o t o t o edu o edu o edu o t o t source edu (1 x 256) target edu (1 x 256) source token (1 x 256) target token (1 x 256) context (1 x 768) o edu input document context encoder @xmath4, @xmath5, @xmath6, rhetoric syntactic time graph graph graph o ce o ce figure 2: timers learns rhetorical, syntactic, and temporal features through a gated relational-graph convolutional networks (gr-gcn). the output of gsg forms the input of gt g. the output corresponding to the source and target nodes learned by gt g (ot ) and gdg (oedu ) are concatenated with the output of the bert based context encoder (oce), which forms the final output hg that passes through the softmax layer to predict the temporal relation. to solve this task, our model (fig. 1 ) builds the timers-graph, which consists of a syntactic graph (sec.2.1), a time graph (sec. 2.2), and a rhetorical graph (sec.2.3). each graph is learned through gr-gcn to extract the embeddings used for temporal relation extraction (fig. 2 , sec.2.4). ## conclusion this work presents a neural architecture that utilizes local syntactic features, rhetorical discourse features, and temporal arguments in semantic role labels through a gated relational-gcn for document-level temporal relation extraction on tddiscourse, matres, and timebank-dense datasets. experiments show that timers shows substantial improvement for events that require chain reasoning and causal prerequisite links. future work will focus on exploring real-world scenarios in which the temporal extraction task suffers from absent or erroneous event and timex annotations. we believe our proposed methods can also be adapted for other languages as well by overcoming possible limitations such as dependency parsing, semantic parsing, timex normalization for the non-english corpora.