Temporal Knowledge Question Answering via Abstract Reasoning Induction (2024)

Abstract: In this study, we address the challenge of enhancing temporal knowledge reasoning in Large Language Models (LLMs). LLMs often struggle with this task, leading to the generation of inaccurate or misleading responses. This issue mainly arises from their limited ability to handle evolving factual knowledge and complex temporal logic. To overcome these limitations, we propose Abstract Reasoning Induction (ARI) framework, which divides temporal reasoning into two distinct phases: Knowledge agnostic and Knowledge-based. This framework offers factual knowledge support to LLMs while minimizing the incorporation of extraneous noisy data. Concurrently, informed by the principles of constructivism, ARI provides LLMs the capability to engage in proactive, self-directed learning from both correct and incorrect historical reasoning samples. By teaching LLMs to actively construct knowledge and methods, it can significantly boosting their temporal reasoning abilities. Our approach achieves significant improvements, with relative gains of 29.7% and 9.27% on two temporal QA datasets, underscoring its efficacy in advancing temporal reasoning in LLMs. The code can be found at https: //github.com/czy1999/ARI-QA.

## method algorithm 1 abstract reasoning induction require: temporal knowledge graph k, question q, historical memory h q , abstract methodology instruction set m c ensure: answer to the question q ## conclusion and limitation this study, anchored in the principles of constructivism, critically examines the shortcomings of llms in addressing complex temporal reasoning challenges and proposes an innovative approach to augment their reasoning capabilities. through the integration of a knowledge adaptability framework and abstract methodological guidance, we have shown that llms can attain more precise and efficient reasoning in complex temporal scenarios, effectively overcoming their constraints in processing and interpreting time-sensitive knowledge. limitations. while ari demonstrates impressive results, it also presents several limitations for ongoing refinement. firstly, the efficacy of generating abstract guidance heavily relies on the capabilities of llms. smaller-scale llms may struggle to produce high-quality abstract guidance, thus potentially restricting their application. secondly, the ari framework depends on multi-step reasoning to arrive at final answers, a process moderately influenced by the llm's reasoning efficiency, which extends the duration of inference. finally, our method is primarily concentrated on complex temporal reasoning, with its effectiveness in other reasoning domains remaining to be examined. future research should aim to refine these methods to make them more adaptable to various models and problem domains, enhance the balance between reasoning efficiency and depth, and expand their scope to include a broader range of reasoning tasks.