GNN - RAG : Graph Neural Retrieval for Efficient Large Language Model Reasoning on Knowledge Graphs (2025)

Abstract: Retrieval-augmented generation (RAG) in Knowledge Graph Question Answering (KGQA) enhances the context of Large Language Models (LLMs) by incorporating information retrieved from the Knowledge Graph (KG). Most recent approaches rely on costly LLM calls to generate executable relation paths or traverse the KG, which is inefficient in complex KGQA tasks, such as those involving multi-hop or multi-entity questions. We introduce the GNN-RAG framework, which utilizes lightweight Graph Neural Networks (GNNs) for effective and efficient graph retrieval. The GNN learns to assign importance weights to nodes based on their relevance to the question, as well as the relevance of their neighboring nodes. This enables the framework to effectively handle context from deeper parts of the graph, improving retrieval performance. GNN-RAG retrieves the shortest paths connecting question entities to GNN answer candidates, providing this information as context for the LLM. Experimental results show that GNN-RAG achieves effective retrieval on two widely used KGQA benchmarks (WebQSP and CWQ), outperforming or matching GPT-4 performance with a 7B tuned LLM. Additionally, GNN-RAG excels on multi-hop and multi-entity questions outperforming LLM-based retrieval approaches by 8.9–15.5% points at answer F1. Furthermore, it surpasses long-context inference while using 9\times fewer KG tokens. The code is provided in https://github.com/cmavro/GNN-RAG .

## introduction large language models (llms) @xcite @xcite are the state-of-the-art models in many nlp tasks due to their remarkable ability to understand natural language. the power of llm comes from pretraining in a large corpus of textual data to obour gnn-rag outperforms existing kg-rag methods by 8.9-15.5% points at f1 (ref: knowledge graphs (kgs) @xcite store information in structured form that can be easily updated. kgs represent human-crafted factual knowledge in the form of triplets @xcite , e.g., <@xmath0, which collectively form a graph. in the case of kgs, the stored knowledge is updated by fact addition or removal. as kgs capture complex interactions between stored entities, e.g. multi-hop relations, they are widely used for knowledge-intensive tasks, such as question answering (qa) @xcite with retrieval-augmented generation (rag) @xcite . rag performance is highly dependent on the kg facts that are retrieved @xcite . the challenge in kgqa is that kgs store complex graph information, usually consisting of millions of facts, and retrieving the right information requires effective graph processing @xcite . retrieval methods that rely on off-the-shelf nlp retrievers @xcite or classical graph algorithms @xcite are lim-ited as retrieval is not tailored for kgqa. most recent methods rely on llms for semantic parsing @xcite or for traversing the kg @xcite . however, llms can be ineffiecient in complex kgqa tasks, such as those involving multi-hop or multi-entity questions. in such cases, leveraging context from distant nodes in the graph is crucial, and llms may struggle to process the context of the graph which expands exponentially at deeper levels @xcite . to address retrieval efficiency in complex kgqa, we present gnn-rag, a graph neural framework which utilizes context from deeper parts of the graph, such as information from distant nodes, during retrieval. gnn-rag relies on graph neural networks (gnns) @xcite , which are powerful graph representation learners, able to handle the complex graph information stored in the kg. gnn-rag consists of a graph neural phase, where the gnn learns to embed kg-specific semantics and to identify relevant nodes, given a question. in gnn-rag, we implement deep (multi-layer) gnn models to leverage graph context. the gnn learns to assign importance weights to nodes based on their relevance to the question, as well as the relevance of their neighboring nodes. this enables the framework to effectively handle context from deeper parts of the graph, improving retrieval performance. at retrieval, the gnn scores answer candidates for the given question, and the shortest paths that connect question entities and answer candidates are retrieved, which are verbalized and given as context to the llm. experimental results show gnn-rag's superiority over competing rag-based systems for kgqa by outperforming them by up to 15.5% points at complex kgqa performance (figure 1 ). furthermore, we show the effectiveness of gnn-rag in retrieval augmentation, while outperforming long-context retrieval using 9× fewer kg tokens. our contributions are summarized below: ## gnn-rag we present gnn-rag, a new graph neural retrieval method for kgqa that leverages gnns to improve retrieval performance when questions require complex graph information. gnn-rag's primary contribution lies in its use of gnns for efficient graph retrieval, distinguishing it from previous methods that rely on long-context retrieval based on text similarity or retrieval via llms. we provide the overall framework at inference time in figure 3 . first, the gnn processes a dense subgraph to utilize deep graph context and identify relevant answer nodes for a given question. second, the shortest paths in the kg that connect question entities and gnn-based answers are extracted to represent useful kg reasoning paths. the extracted paths are verbalized and given as context for llm reasoning via rag. in our gnn-rag framework, the gnn acts as a dense subgraph processor, while the llm leverages its natural language processing ability for downstream kgqa. ## conclusion we introduce gnn-rag, a novel graph neural method for enhancing rag in kgqa with gnns. our contributions are the following. (1) framework: gnn-rag tailors gnns for kg retrieval due to their ability to handle complex graph information. ( 2 ) effectiveness: gnn-rag achieves superior performance when multi-hop information is needed for kgqa. (3) efficiency: gnn-rag improves vanilla llms on kgqa performance without incurring additional llm calls as existing rag systems for kgqa require. in addition, gnn-rag outperforms long-context retrieval using 9× fewer kg tokens.