M us TQ : A Temporal Knowledge Graph Question Answering Dataset for Multi-Step Temporal Reasoning (2024)

Abstract: Question answering over temporal knowledge graphs (TKGQA) is an emerging topic, which has attracted increasing interest since it considers the dynamic knowledge in the world. Several datasets along with model developments are proposed in the TKGQA research field. However, existing studies generally focus on fact-centered reasoning, with limited attention to temporal reasoning. To tackle the intricate and comprehensive nature of temporal reasoning, we propose a new TKGQA dataset, MusTQ, which contains 666K multi-step temporal reasoning questions as well as a TKG. The multi-step temporal reasoning is established based on six basic temporal reasoning types derived from a well-established measure theory. Using MusTQ, we evaluate previous TKGQA methods and find that they typically fall short in multi-step temporal reasoning. Furthermore, we propose a TKGQA model, MusTKGQA, which enhances multi-step reasoning ability with entity-time attention mechanism and optimized temporal knowledge graph representation. Extensive experiments on MusTQ show that our model achieves state-of-the-art multi-step temporal reasoning performance.

## introduction given natural questions, question answering over temporal knowledge graphs (tkgqa) aims to provide the corresponding answers using a temporal knowledge graph (tkg) as the knowledge base. since many real-world facts will change over time and formulating questions with temporal constraint is a straightforward method to ensure factual validity, tkgqa has gradually attracted increasing research attention @xcite @xcite @xcite @xcite @xcite . compared with the traditional knowledge graph question answering (kgqa) that adopts static knowledge graphs as the knowledge base, tkgqa should not only consider the knowledge formed in triples but also emphasize its temporal information. therefore, reasoning on temporal information could naturally become a vital perspective for questions that involve temporal constraints. existing tkgqa datasets either query the information in tkg @xcite or make prediction in the future timestamps @xcite . in this paper, we mainly focus on the former type of tkgqa datasets. though great success has been achieved in previous work, existing tkgqa datasets typically focus on fact-centered reasoning questions, with less attention on temporal reasoning. specifically, they only involve singlestep temporal reasoning, ignoring the systematical exploration of temporal reasoning. among them, cronquestions @xcite constructs its questions with single-step temporal reasoning related to explicit time or entity. although mul-titq @xcite ) contains two-step temporal reasoning questions, its questions only involve fact-centered reasoning and lack systematic explorations on temporal reasoning. in this paper, we propose mustq (muti-step temporal reasoning questions), a new tkgqa dataset, which aims to enhance the temporal reasoning ability in tkgqa. in order to systematically study temporal reasoning, we start with the measure of time. according to the well-established theory for the scale of measurement @xcite , the measure of time determines the basic operations that can be applied to the temporal data. the timestamp information in tkg involves the following three operations: determination of i) equality; ii) greater or less; iii) equality of intervals or differences. the first operation maps the facts to their corresponding timestamps, while the second operation sorts facts according to their chronology. the final operation enables timestamp reasoning with the addition and subtraction operations. under the guidance of this theory, we classify temporal reasoning into temporal comparison reasoning and temporal numeric reasoning. the former uses operations i) and ii) to reason timestamps based on the comparison, while the latter uses operation iii) to reason timestamps based on the interval. further, we divide these two categories into six detailed reasoning types and construct the diverse multi-step temporal reasoning by their combination. finally, mustq contains ~666k temporal questions and an underlying tkg (with 125k entities and 326k facts) as its knowledge base. based on our mustq, we build and evaluate various baseline systems, including the pre-trained language models and the representative kgqa and tkgqa models. the experimental results demonstrate that all existing methods show their limited capabilities in multi-step temporal reasoning. furthermore, we propose a new tkgqa model, mustkgqa, which improves the multi-step temporal reasoning ability via carefully designed tkg representation learning and entity-time attention mechanism. in detail, to enhance the temporal reasoning, we optimize the tkg representation in time and fact perspectives with sequence alignment and fact alignment respectively. the sequence alignment enhances the reasoning between timestamps, while the fact alignment improves the awareness for time boundary of interval facts. then, we propose an entity-time attention mechanism to infer the temporal information of the question steply and finally reason the answer. extensive experiments on mustq show that our model achieves state-ofthe-art multi-step temporal reasoning performance. the results on the previous cronquestions dataset also show the strong ability of our model. our main contributions are concluded as follows: ## related work temporal questions over kg. several datasets are proposed for temporal question answering. tempquestions @xcite and timequestions @xcite are two datasets that collect temporal questions from kgqa datasets. however, they adopt traditional static knowledge graphs as their knowledge base @xcite . cronquestions @xcite is the first temporal question dataset over tkg @xcite , whose questions are classified into simple and complex questions based on the number of facts leveraged to answer the questions. additionally, multitq @xcite focuses on questions in multiple temporal granularities and contains two-step comparison temporal reasoning questions with icews @xcite . however, its questions are factcentered and lack temporal reasoning variety. the questions in the above datasets require models to query information within the provided kgs. fore-casttkgquestions @xcite is another tkgqa dataset that aims to predict facts in the future timestamps beyond the given tkg. we mainly focus on datasets that query information within tkg. different from existing datasets that generally focus on fact-centered reasoning, our mustq contains multi-step temporal reasoning questions constructed based on the systematical exploration of temporal reasoning. ## mustq construction mustq consists of multi-step temporal questions and a tkg as its knowledge base. we first introduce the tkg in mustq ( § 3.1). then we describe the systematic temporal reasoning categories, and the single-step reasoning types in each category ( § 3.2). next, we present the process of developing multi-step temporal reasoning questions ( § 3.3). finally, we show the data statistics of mustq ( § 3.4). ## steps how many award nominations has {head} received in total in the time span from {timea} to {timeb}? ## steps who took on the {ordinal} job as {tail} in {time} during the phase between {timea} and {timeb}? ## conclusion we propose the first multi-step temporal reasoning tkgqa dataset, mustq, which constructs multi-step reasoning under the guidance of a wellestablished measure theory. mustq is centered on temporal reasoning and is challenging for existing tkgqa models. further, we establish the mustkgqa model with entity-time attention mechanism and optimized tkg representation to tackle temporal reasoning in multi-steps. experimental results on mustq show that our model achieves state-of-the-art multi-step temporal reasoning performance. the results on the previous cronquestions dataset also show the strong ability of our model.