Modeling Document-level Temporal Structures for Building Temporal Dependency Graphs (2022)

Abstract: We propose to leverage news discourse profiling to model document-level temporal structures for building temporal dependency graphs. Our key observation is that the functional roles of sentences used for profiling news discourse signify different time frames relevant to a news story and can, therefore, help to recover the global temporal structure of a document. Our analyses and experiments with the widely used knowledge distillation technique show that discourse profiling effectively identifies distant inter-sentence event and (or) time expression pairs that are temporally related and otherwise difficult to locate.

## introduction grounding all events and time expressions to a reference timeline is fundamental to text understanding. recently, @xcite proposed a new task and dataset for building temporal dependency graph (tdg) foot_0 . tdg is based on the notion of narrative time and temporal anaphora, and references each timex to a timex or a meta node and each event to a timex and maybe an event. the reference timex of an event is either the smallest time (when identifiable) that encloses the event or the document creation time (dct). similarly, the reference event is selected such that it gives the most precise temporal interpretation for a child event. because each event and timex is referenced to only one timex (or additionally an event), identified temporal relations represent the most salient relations that can potentially be used to infer additional temporal relations through transitivity or commonsense reasoning @xcite . this makes identifying reference timex and reference event more challenging, especially when they are mentioned across sentences. human evaluations by @xcite also found that identifying the appropriate reference timex and reference event was the most challenging aspect of their annotation. in this work, we focus on improving crosssentence reference timex and event mentions identification by exploring discourse-level temporal cues. we choose the news discourse profiling structure (dp) @xcite . dp classifies sentences in a news document into one of eight content types, defined based on the functional role of a sentence in describing the main news story @xcite @xcite , and provides an event-based functional interpretation of sentences. the eight content types include main, consequence, previous event, current context, historical, anecdotal, evaluation and expectation. as shown in figure 1 , different content types induce different time frames relevant to a news story that can be beneficial for the global interpretation of temporal orders among event and timex mentions. for instance, mentions in historical sentences have a temporal adjacency with other mentions in historical sentences but are likely to be distant from mentions in other content types. similarly, mentions in previous event sentences may have a temporal adjacency with mentions from one of the previous event, main event or current-context sentences but are likely to be separated from mentions in any of the historical, expectation or consequence sentences. we first summarize the distributional association between the position of reference mentions and discourse content types in §2.3. then, we propose a knowledge distillation-based method to incorporate discourse knowledge into the tdg system. we experiment with the @xcite and roberta @xcite pre-trained languages models and find that the proposed knowledge distillation-based tdg system is effective in using discourse-level cues and achieves improved performance on identifying cross-sentence reference mentions while retaining performance on the intra-sentence mention pairs. ## empirical evaluations and results based on our observations in §2.3, we perform empirical evaluations to demonstrate the effectiveness of news discourse profiling for building tdg. ## related work most previous works @xcite @xcite @xcite @xcite @xcite @xcite @xcite treat temporal relation extraction as a pair-wise classification problem and most widely used datasets follow the same pair-wise schema for annotating temporal relations between event/ timex pairs @xcite @xcite @xcite . however, as discussed by @xcite ; @xcite ; @xcite , pairwise annotations as well as classification models suffer from quadratic complexity, partial annotations and inconsistent predictions. recently, @xcite proposed to build a dependency tree (tdt) structure to address the above three problems with pair-wise annotations and modeling and later extended that to temporal dependency graph @xcite . we use the most recent temporal dependency graph dataset that improves the expressiveness of previous tdt datasets @xcite and follow their neural ranking modeling approach. however, different from the previous work, we explore news discourse profiling to explicitly focus on improving the performance of a neural ranking model on cross-sentence event/ timex pairs. @xcite were the first to show the effectiveness of several discourse analysis frameworks, including rhetorical structure theory (rst) @xcite , pdtb-style discourse relations @xcite and topical text segmentation @xcite for temporal relation extraction. different from the above three discourse structures, discourse profiling is a functional (web-ber and joshi, 2012) structure and has global eventcentric interpretations. secondly, @xcite focused on classifying temporal relations between a given pair of temporally related events. in contrast, our goal is to identify the most salient reference for every event/ timex mention that determines its most precise location on the timeline. ## conclusion we have shown that news discourse profiling can be used to incorporate document-level temporal structures when building temporal dependency graphs. through analyses, we have shown the distributional association between discourse content types and positions of reference and child mentions. further, empirical evaluation using the knowledge distillation technique shows that discourse profiling is effective in identifying cross-sentence referencechild mention pairs. in the future, we will explore new linguistics structures and modeling techniques to incorporate document-level temporal structures for building tdg. joint conference on lexical and computational semantics (*sem 2019), pages 178-185, minneapolis, minnesota. association for computational linguistics.