Temporal Reasoning on Implicit Events from Distant Supervision (2021)

Abstract: We propose TRACIE, a novel temporal reasoning dataset that evaluates the degree to which systems understand implicit events—events that are not mentioned explicitly in natural language text but can be inferred from it. This introduces a new challenge in temporal reasoning research, where prior work has focused on explicitly mentioned events. Human readers can infer implicit events via commonsense reasoning, resulting in a more comprehensive understanding of the situation and, consequently, better reasoning about time. We find, however, that state-of-the-art models struggle when predicting temporal relationships between implicit and explicit events. To address this, we propose a neuro-symbolic temporal reasoning model, SymTime, which exploits distant supervision signals from large-scale text and uses temporal rules to combine start times and durations to infer end times. SymTime outperforms strong baseline systems on TRACIE by 5%, and by 11% in a zero prior knowledge training setting. Our approach also generalizes to other temporal reasoning tasks, as evidenced by a gain of 1%-9% on MATRES, an explicit event benchmark.

## introduction understanding temporal relations between events in narrative text is a crucial part of text understanding. when reading a story, a human can construct a latent timeline about events' start and end times, similar to the one shown in fig. 1 about an automobile accident. this timeline not only contains the placements of explicitly mentioned events (e.g., ride a bicycle), but also accounts for implicit events (e.g., farrah was distracted so she looked away). such a latent timeline explains the dynamics between events; for example, the possible chain of events between ride and recovered in this context farrah was driving home from school. a person was riding a bicycle in front of her. farrah looked away for a second. she didn't notice that he stopped. she tried to brake but it was too late. the person recovered soon. ## related work temporal reasoning has received much attention in the nlp community, and to date, there are many datasets that focus on temporal ordering @xcite @xcite @xcite @xcite , and other temporal knowledge @xcite . we focus here on modeling implicit events, which has received relatively little attention. multiple systems have been proposed as part of research into temporal ordering @xcite @xcite @xcite , duration prediction @xcite and other tasks. our decision to use a textual entailment style follows recent work on natural language inference @xcite @xcite , which tends to not focus on time (for recent work on temporal nli, see @xcite ). many have used distant supervision for temporal reasoning @xcite @xcite . comparatively, our work captures longer-range dependencies in narrative text (for related ideas, see @xcite ). we are inspired by structural predictions and constraints that combat the sparsity of temporal knowledge @xcite , as well as neural module networks @xcite and other decomposition-based approaches @xcite @xcite @xcite . in particular, we build neuralsymbolic transformer models that operationalize some of the classical interval-based computations used in earlier work on temporal reasoning @xcite (for related ideas, compare with @xcite ; @xcite ). ## the tracie dataset in this section, we introduce the tracie dataset. foot_2 ## pattern-based pre-training as argued in §3.2, we believe that it is more efficient to build a model that learns the prior knowledge needed for the task with distant signals and only subsequently learns the task definition through a small training set. this section describes how we collect the distant signals related to events' starttime comparisons and pre-train a novel temporallyaware transformer model called ptntime. while ptntime will be used for fine-tuning directly on tracie, it will also form the basis of a more general temporal reasoning model called symtime that we describe in §5. ## symbolic temporal reasoning model (symtime) to address the challenge of predicting event end times for which it is difficult to obtain high-quality direct or distant supervision, we introduce a new reasoning model called symtime in this section. this model makes end-time comparisons by symbolically combining start time distance and duration from separate predictions based on some of the components introduced in the previous section. different from @xcite and @xcite , our model does not rely on explicit annotations on timepoints, but only relative comparisons between them. ## experiments in this section, we detail our experimental setup ( §6.1-6.2) and report our main results ( §6.3-6.5). foot_8 ## conclusion we introduce a challenging dataset tracie, to evaluate systems' temporal understanding of implicit events. we propose a distant supervision process that improves language models' understanding of start times of both explicit and implicit events. we further combine this process with a distantly supervised model that estimates events' duration to compare event end times, under the explicit rule that end times are start times plus durations. we show that our model improves over tracie and matres, suggesting the effectiveness of highprecision pre-training and symbolic temporal reasoning. despite these advances, tracie continues to be a challenging task for future work on general temporal reasoning.