Incorporating Temporal Information in Entailment Graph Mining (2020)

Abstract: We present a novel method for injecting temporality into entailment graphs to address the problem of spurious entailments, which may arise from similar but temporally distinct events involving the same pair of entities. We focus on the sports domain in which the same pairs of teams play on different occasions, with different outcomes. We present an unsupervised model that aims to learn entailments such as win/lose → play, while avoiding the pitfall of learning non-entailments such as win ̸→ lose. We evaluate our model on a manually constructed dataset, showing that incorporating time intervals and applying a temporal window around them, are effective strategies.

## introduction recognising textual entailment and paraphrases is core to many downstream nlp applications such as question answering and semantic parsing. in the case of open-domain question answering over unstructured data, the answer to a question may not be explicitly stated in the text, but may be recovered via paraphrases and/or entailment rules. entailment graphs @xcite @xcite , in which nodes represent predicates and edges are entailment relations, have been proposed as a means to answer such questions. they can be mined using unsupervised methods applied over large collections of text, by keeping track of which entity pairs occur with which predicates. one common error made by these graphs, however, is that they assert spurious associations between similar but temporally distinct events that occur with the same entity pairs. for example, both the predicates beat and lost against will apply to sports team entity pairs such as (arsenal, man united). this is likely to mislead the current methods into incorrectly assigning an entailment relation between these two predicates. in this paper we extend the framework of @xcite to incorporate the temporal location of events, with the aim of mitigating these spurious entailments. temporal information can be used to disentangle these groups of highly correlated predicates, because although they will share entity pairs, they will never occur at the same time. for example, in figure 1 arsenal and man united played each other three times in 2019, with three different outcomes: win (beat), lose (lost against), tie (tied with). in previous methods, the context in which the predicates occur appears to be the same, because they only consider entity pairs as context. therefore they mistakenly take the examples in figure 1 as evidence of entailments or paraphrases between the three outcome predicates (win, lose, and tie), depending on the distributions found in the data. our method enriches this context to include time interval information, thereby filtering out combinations that are not temporally near each other. thus we hope to avoid learning that @xmath0, while still learning that @xmath1. as an initial test domain, we focus on the sports news genre, using extracted relations that involve two sports teams. we evaluate on a manually constructed dataset of 1,312 entailment pairs based on paraphrases of the predicates in the graph on the right hand side of figure 1 . our goal is to recover the structure of this graph in an unsupervised way, separating each of the highly correlated outcome predicates while predicting that they all entail play. arsenal-played and lost against-man united 1-3 (25/01/2019) arsenal-played and beat-man united 2-0 (10/03/2018) arsenal-played and tied with-man united 1-1 (30/09/2019) the contributions of this work are: 1) a model for incorporating relation-level time intervals into an entailment graph mining procedure, outperforming non-temporal models, and 2) a manually constructed evaluation dataset of sports domain predicates. to our knowledge this is the first attempt to incorporate temporal information for learning entailment graphs. ## conclusions we injected temporal information into the local entailment graph construction method of @xcite , with the goal of comparing only those events that are temporally near each other. this is achieved by filtering the counts of predicate p according to whether its events' time intervals overlap with the those of predicate q. we considered a range of new local similarity scores based on both temporally filtered counts and scaled pmi scores, which we evaluate on a semi-automatically constructed dataset, based on manually constructed paraphrase clusters. our temporal similarity measures outperform their non-temporal counterparts, including binc, the state-of-the-art measure for relation entailment. we also show that using a combination of time expressions recovered from the text and the document creation date performed better than using only one of these sources, and that adding a temporal window around the time intervals of the events is essential. the performance of the temporal similarity measures over the non-temporal measures is particularly strong at the low recall range when only time expressions from the text are used. this is especially promising as it suggests that there is much room for improvement in using more sophisticated temporal resolution systems and methods for linking times to events.