C urious LLM : Elevating Multi-Document Question Answering with LLM -Enhanced Knowledge Graph Reasoning (2025)

Abstract: Large Language Models (LLMs) have achieved significant success in open-domain question answering. However, they continue to face challenges such as hallucinations and knowledge cutoffs. These issues can be mitigated through in-context learning by providing LLMs with relevant context before generating answers. Recent literature proposes Knowledge Graph Prompting (KGP) which integrates knowledge graphs with an LLM-based traversal agent to substantially enhance document retrieval quality. However, KGP requires costly fine-tuning with large datasets and remains prone to hallucination. In this paper, we propose CuriousLLM, an enhancement that integrates a curiosity-driven reasoning mechanism into an LLM agent. This mechanism enables the agent to generate relevant follow-up questions, thereby guiding the information retrieval process more efficiently.Central to our approach is the development of the new Follow-upQA dataset, which includes questions and supporting evidence as input, with follow-up questions serving as ground truths. These follow-up questions either inquire about what is still missing to fully answer the user’s query or use special tokens to signify that the retrieved evidence is sufficient. Our experiments show that CuriousLLM significantly boosts LLM performance in multi-document question answering (MD-QA), circumventing the substantial computational costs and latency from the original KGP framework.

## introduction large language models (llms) have demonstrated remarkable success in open-domain question answering. however, they continue to face challenges such as hallucinations and knowledge cutoffs @xcite @xcite . to address these issues, recent research has explored in-context learning with llm, using external knowledge sources such as retrieved documents or knowledge graphs (kg) to improve their accuracy and reasoning ability @xcite @xcite @xcite @xcite @xcite . among the popular approaches are rag @xcite and kaping @xcite , which provide context by retrieving relevant documents or kg triplets to support the llm reasoning process. although these methods have proven effective, few approaches have fully integrated llms into the retrieval process, leaving a gap in the ability to efficiently navigate and extract relevant information from vast knowledge sources. recently, @xcite propose knowledge graph prompting (kgp), which incorporates a fine-tuned llm agent into the kg traversal process. this agent predicts missing evidence based on the initial query and the retrieved documents. the predictions are then used to identify relevant passages from neighboring nodes in the kg through similarity ranking. with this prompt reformulation approach, kgp achieves state-of-the-art results in several benchmarks for factual consistency. however, during our experiments, we notice that the original kgp technique involves fine-tuning a t5 @xcite agent with a large corpus. despite this, the agent's performance remains limited due to hallucination: while correctly predicts missing evidence, it often uses irrelevant or erroneous keywords, which obscure the search for the actual piece of missing evidence, as shown in to address these challenges, we propose a novel framework by fine-tuning an llm agent to emulate the curious nature of a human researcher. instead of predicting missing evidence, our agent asks followup questions to more efficiently guide the search toward missing evidence. compared to the original approach, our curiousllm: 1) requires significantly fewer training samples for fine-tuning, 2) markedly improves md-qa performance, and 3) can terminate the search before exhausting the preset search budget, thereby reducing latency. our contributions are as follows. ## experiments to evaluate the multi-document question answering (md-qa) capabilities of our curiousllm agent, kgp-mistral, we adopt the experimental setup used by kgp-t5 @xcite . in the original kgp-t5 evaluation, four md-qa validation sets are used, each comprising 500 questions sampled from the following datasets: hotpotqa, 2wikimqa @xcite , iirc @xcite , and musique @xcite . these datasets include 270k, 120k, 470k, and 173k unique passages, respectively, which serve as supporting evidence or distracting passages. following our approach to kg construction, we treat each of these passages as a distinct node within the kg. for this evaluation, we limit the search scope to a 2-hop traversal, meaning that the system can retrieve and consider information from up to two edges away from the starting node in the graph. to ensure a fair comparison with kgp-t5, we standardize the retrieval parameters by selecting the top 30 passages based on a similarity search to generate the answers. ## conclusion our new curiousllm-enhanced kgp framework represents a significant advance in the field of md-qa by integrating an llm-guided prompt reformulation mechanism into the kg traversal process. by introducing the follow-upqa dataset and a curiosity-driven llm traversal agent, the system effectively addresses challenges like hallucination, inefficient retrieval, and the limitations of the original kgp framework. extensive experiments show that this approach enhances md-qa accuracy and efficiency while reducing computational overhead, making it a practical solution for real-world applications. this work paves the way for future research into optimizing llm-guided retrieval processes by offering a scalable, robust framework balancing performance and resource efficiency.