T ime R ^4 : Time-aware Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question Answering (2024)

Abstract: Temporal Knowledge Graph Question Answering (TKGQA) aims to answer temporal questions using knowledge in Temporal Knowledge Graphs (TKGs). Previous works employ pre-trained TKG embeddings or graph neural networks to incorporate the knowledge of TKGs. However, these methods fail to fully understand the complex semantic information of time constraints in questions.In contrast, Large Language Models (LLMs) have shown exceptional performance in knowledge graph reasoning, unifying both semantic understanding and structural reasoning. To further enhance LLMs’ temporal reasoning ability, this paper aims to integrate relevant temporal knowledge from TKGs into LLMs through a Time-aware Retrieve-Rewrite-Retrieve-Rerank framework, which we named TimeR ^4 .Specifically, to reduce temporal hallucination in LLMs, we propose a retrieve-rewrite module to rewrite questions using background knowledge stored in the TKGs, thereby acquiring explicit time constraints. Then, we implement a retrieve-rerank module aimed at retrieving semantically and temporally relevant facts from the TKGs and reranking them according to the temporal constraints.To achieve this, we fine-tune a retriever using the contrastive time-aware learning framework.Our approach achieves great improvements, with relative gains of 47.8% and 22.5% on two datasets, underscoring its effectiveness in boosting the temporal reasoning abilities of LLMs. Our code is available at https://github.com/qianxinying/TimeR4.

## introduction knowledge graph question answering (kgqa) aims to provide answers based on knowledge from knowledge graphs @xcite . however, many real-world questions include temporal constraints, such as "who is the president of the united states after obama?" to address this, some knowledge graphs store time-aware facts as quadruples (subject, predicate, object, timestamp), which are known as temporal knowledge graphs (tkgs). temporal knowledge graph question answering (tkgqa) focuses on obtaining answers using the knowledge in tkgs @xcite . recent works @xcite incorporate knowledge from tkgs by utilizing pre-trained tkg embeddings or graph neural networks (gnns). however, these methods fail to fully understand the complex semantic information of time constraints in questions @xcite . in contrast, large language models (llms) have demonstrated exceptional performance in knowledge graph reasoning @xcite and can unify semantic understanding and graph reasoning @xcite . to further enhance the temporal reasoning capabilities of llms, in this paper, we aim to integrate temporal knowledge into llms, thereby addressing complex and multi-granularity temporal questions. however, enhancing the temporal reasoning capabilities within llms remains several significant challenges: to address the above challenges, we propose timer 4 , a time-aware retrieve-rewrite-retrieve-rerank framework for the tkgqa task. specifically, to mitigate the issue of llms hallucinating when faced with implicit temporal questions, we employ the retrieve-rewrite strategy. we perform fact retrieval from the facts knowledge store (fks) to obtain relevant facts for implicit questions. subsequently, we rewrite these questions by replacing temporal facts with specific timestamps, ensuring that all questions contain explicit temporal information. for constructing fks, we fix the language model to obtain semantical fact embeddings. to simultaneously capture semantic similarity and temporal constraints, we employ the retrieve-rerank strategy. this involves conducting a time-aware retrieval from the temporal knowledge store (tks) and reranking the facts based on temporal constraints to refine the retrieval process. the tks is constructed by fine-tuning a language model with a contrastive time-aware retrieval strategy, which develops an encoder capable of capturing both semantic similarity and temporal constraints by constructing three types of negatives for each question. finally, we fine-tune open-source llms with the retrieved facts from tks, leveraging the temporal knowledge in tkgs to enhance the model's temporal reasoning capabilities. experiments on two datasets demonstrate that our strategy significantly enhances the temporal reasoning abilities of llms. overall, our work makes the following contributions: ## preliminaries temporal knowledge graph @xmath0, p, t , f} is a directed graph where vertices are a set of entities e. the edges are a set of @xmath1. the quadruple set @xmath2, p, o, t)} ⊆ e × p × e × t represents the temporal facts, where s and o are subject and object, p is predicate between s and o at timestamp t. tkgqa is a task to infer the correct answer to natural language question @xmath3, p, o, t) in the tkg, where the answer can be either an entity name or timestamp. algorithm 1: the training procedure of timer 4 after 2016-01-05, who was the first to visit iraq? (b) rewriting retrieved quadruples (danish ministry, visit, iraq, 2016-01-05) (evan bayh, visit, iraq, 2016-01-04) … (obama, visit, iraq, 2016-01-09) replace the fact with explicit timestamps response: ['jack straw'] ## conclusion in this work, we address two key challenges that llms face when handling temporal questions and introduce a time-aware retrieve-rewrite-retrievererank framework named timer 4 . to mitigate the issue of the temporal hallucination of llms, we utilize a retrieve-rewrite strategy to fetch relevant facts in fks and integrate specific timestamps into the questions. afterward, in order to retrieve facts that satisfy both time constraints and semantic similarity, we implement a retrievererank strategy. we perform time-aware retrieval from the tks and rerank them based on temporal constraints. finally, we fine-tune open-source llms, leveraging the knowledge in tkgs to enhance the temporal reasoning capabilities in llms. experiments on two datasets demonstrate that our framework significantly enhances the temporal reasoning abilities of llms.