Graph Neural Network Enhanced Retrieval for Question Answering of Large Language Models (2025)

Abstract: Retrieval augmented generation has revolutionized large language model (LLM) outputs by providing factual supports. Nevertheless, it struggles to capture all the necessary knowledge for complex reasoning questions. Existing retrieval methods typically divide reference documents into passages, treating them in isolation. These passages, however, are often interrelated, such as passages that are contiguous or share the same keywords. Therefore, it is crucial to recognize such relatedness for enhancing the retrieval process. In this paper, we propose a novel retrieval method, called GNN-Ret, which leverages graph neural networks (GNNs) to enhance retrieval by exploiting the relatedness between passages. Specifically, we first construct a graph of passages by connecting passages that are structure-related or keyword-related. A graph neural network (GNN) is then leveraged to exploit the relationships between passages and improve the retrieval of supporting passages. Furthermore, we extend our method to handle multi-hop reasoning questions using a recurrent graph neural network (RGNN), named RGNN-Ret. At each step, RGNN-Ret integrates the graphs of passages from previous steps, thereby enhancing the retrieval of supporting passages. Extensive experiments on benchmark datasets demonstrate that GNN-Ret achieves higher accuracy for question answering with a single query of LLMs than strong baselines that require multiple queries, and RGNN-Ret further improves accuracy and achieves state-of-the-art performance, with up to 10.4 accuracy improvement on the 2WikiMQA dataset.

## introduction large language models (llms) continue to struggle with factual errors when encountering knowledge intensive questions @xcite @xcite . although retrieval-augmented llms @xcite have improved factuality and precision of question answering by including relevant passages, there remains a persistent challenge in accurately capturing all the supporting passages when encountering complex knowledge-intensive questions. this limitation can be attributed to the inherent information asymmetry in complex questions. in particular, the questions tend to consist of elaborated background details, leaving only a small portion dedicated to specific inquiries. as an example, in question: 'why did crime rise on mars after the mafia's arrival?', the majority part delves into the background (i.e., 'crime rise ...... mafia's arrival') with only a few words requesting the reason (i.e., 'why'), which consequently retrieves passages on details of crime rising instead of the reason for it. this phenomenon also frequently arises in multihop reasoning questions. considering the sample question 'where was the performer of song left & right (song) born?' in fig. 1 , it becomes apparent that while we can retrieve the knowledge that the performer of song left & right is d'angelo, his birthplace remains absent. previous works @xcite @xcite have attempted to address this issue by incorporating multi-hop reasoning or question rewriting into retrieval processes, which enables them to retrieve information based on prior reasoning outcomes. llms, however, often generate plausible reasons and incorrect rewriting questions without accurate prior domain knowledge of the given question @xcite , thus affecting the subsequent retrieval process. one reason why existing methods struggle to handle the information asymmetry is their tendency to consider passages in isolation @xcite and retrieve the supporting passages based mainly on semantic distances, making it difficult to retrieve all the supporting passages, especially those containing only few words for inquiry. however, the supporting passages of background and inquiry information are usually correlated. they can be structure-related when located in the same section or document, e.g., the happened event (i.e., crime rises on mars) and its corresponding reason are located in the same section but different passages. also, they can be keyword-related by sharing the same keywords or entities, e.g., the passages of background information (performer of song left & right) and inquiry information (birthplace of d'angelo) in fig. 1 share the same keyword: d'angelo. by considering the relatedness between passages, it is possible to retrieve all the supporting passages even when they have significant semantic differences from the query. in this work, we aim to enhance retrieval by taking the relatedness between passages into account. contributions. to establish the relatedness between passages for retrieval purposes, this study initially constructs a graph of passages by connecting individual passages based on both structural information and shared keywords, with each passage as a node in this graph. the key challenge lies in how to effectively leverage the passage of graphs to enhance retrieval coverage. graph neural networks (gnns) are neural networks tailored to analyze graph data and adeptly grasp the relationships between nodes and edges @xcite . thus, we propose to leverage a gnn to enhance the retrieval process by effectively capturing the relatedness between passages and name this method gnn-ret. the gnn facilitates the integration of semantic distances between related passages, thus enabling the semantic distances of passages containing background information to impact the retrieval of passages relevant to the inquiry. to address multi-hop reasoning questions, we further propose a retrieval method, named rgnn-ret, which leverages a recurrent graph neural network (rgnn) to enhance the retrieval process at each step by integrating the retrievals from previous steps through the interconnected graphs of passages. this boosts the retrieval coverage for supporting passages over steps even when llms generate incorrect reasons or subquestions for retrieval. through the experiments on four benchmark datasets, we demonstrate the effectiveness of gnns in enhancing retrieval of supporting passages and thus improving accuracy for qa. for example, as shown in fig. 2 , our proposed gnn-ret with a single query of llms significantly outperforms baselines in accuracy on 2wikimqa @xcite , including those methods that require multiple queries of llms. moreover, by extending gnn to multi-hop reasoning questions, our proposed rgnn-ret achieves state-of-the-art accuracy. to address this limitation, many approaches have been proposed to integrate the retrieval and reasoning processes to improve the retrieval of supporting passages @xcite @xcite @xcite @xcite . they first prompt llms to generate the next-step reason or subquestion and then use it to guide retrieval for qa. however, a challenge in these methodologies pertains to the potential generation of hallucinated reasons or erroneous subquestions by llms without accurate domain knowledge of this question, leading to degraded subsequent retrievals @xcite . graph-enhanced llm. as structured, explicit, and editable representations of knowledge, knowledge graphs (kgs) have been effectively used for boosting retrieval coverage @xcite and enhancing reasoning capabilities of llms @xcite @xcite @xcite @xcite @xcite . instead of naively combining kgs with llms, many works directly prompt llms to perform reasoning on kgs @xcite @xcite . they first identified the entities on the knowledge graph and then search reasoning paths accordingly @xcite @xcite . however, kg-based methods encounter a limitation that they can only handle questions that are effectively represented as kgs. they may struggle with questions that necessitate a comprehensive understand-ing of lengthy contextual information, such as the mentioned example: the cause of crime increasing. rather than kgs, a recent study @xcite attempts to construct a graph of passages and perform reasoning on this graph. it connects documents according to some specific words and performs reasoning on this graph for qa. in contrast, we utilize the graph of passages to enhance the retrieval process. another related work, kgp @xcite , performs retrieval on the graph and search related passages. however, this approach does not consider the integration between related passages and may not effectively enhance retrieval for the supporting passages with poor semantic similarity. ## graph neural network enhanced the qa system typically adopts a retrieve-andread routine @xcite : the retriever computes semantic distances between passages and the question in the embedding space, finds out relevant passages from the whole corpus, and then the reader (llm) generates the answer based on them. considering the information asymmetry of complex questions, directly using the initial semantic distances of passages for retrieval is difficult to retrieve all the supporting passages. therefore, we propose to process by taking the relatedness between passages into account. specifically, we construct a graph of passages (gops) by connecting related passages (section 3.1). to exploit the relatedness between passages to enhance retrieval, we utilize a gnn to enable integration between related passages (section 3.2). for multi-hop questions, we further leverage an rgnn to establish relationship between graphs over steps to enhance retrieval over answering processes (section 3.3). ## experiments in this section, we conduct experiments to demonstrate the effectiveness of gnn-ret and rgnn-ret, including 1) superiority of gnn-ret and rgnn-ret on various qa datasets ( ## conclusion in this paper, we proposed gnn-ret, an effective method to enhance retrieval for qa of llms by exploiting the inherent relatedness between passages on a graph of passages. by extending gnn-ret to multi-hop reasoning questions, we proposed rgnn-ret, which enhances the retrieval for subquestions through the interconnection between graphs of passages across steps. the experiments clearly demonstrated the superiority of both gnn-ret and rgnn-ret over baselines, highlighting the effectiveness of leveraging relatedness between passages to enhance retrieval. from these advantages, we believe that the incorporation of graph-based representations and the exploitation of passage relatedness can open up new avenues of research in the field of llms in understanding structural documents and answering complex questions.