ODA: Observation-Driven Agent for integrating LLMs and Knowledge Graphs (2024)

Abstract: The integration of Large Language Models (LLMs) and knowledge graphs (KGs) has achieved remarkable success in various natural language processing tasks. However, existing methodologies that integrate LLMs and KGs often navigate the task-solving process solely based on the LLM’s analysis of the question, overlooking the rich cognitive potential inherent in the vast knowledge encapsulated in KGs. To address this, we introduce ObservationDriven Agent (ODA), a novel AI agent framework tailored for tasks involving KGs. ODA incorporates KG reasoning abilities via global observation, which enhances reasoning capabilities through a cyclical paradigm of observation, action, and reflection. Confronting the exponential explosion of knowledge during observation, we innovatively design a recursive observation mechanism. Subsequently, we integrate the observed knowledge into the action and reflection modules. Through extensive experiments, ODA demonstrates state-of-theart performance on several datasets, notably achieving accuracy improvements of 12.87% and 8.9%. Our code and data are available on https://github.com/lanjiuqing64/KGdata.

heading: 1 Introduction
text: Large language models (LLMs) (Touvron et al., 2023; Scao et al., 2022; Muennighoff et al., 2022; Brown et al., 2020) have exhibited extraordinary capabilities across a variety of natural language processing tasks. Despite their impressive accomplishments, LLMs often struggle to provide accurate responses to queries that necessitate specialized expertise beyond their pre-training content. In response to this limitation, a natural and promising approach involves the integration of external knowledge sources, such as knowledge graphs (KGs), to
*Equal contribution †Corresponding author
augment LLM reasoning abilities. KGs provide structured, explicit, and explainable knowledge representations, offering a synergistic method to overcome the intrinsic constraints of LLMs. The fusion of LLMs with KGs has garnered significant interest in recent research (Pan et al., 2024), underlying a vast array of applications (Zhang et al., 2023; Do et al., 2024; Sun et al., 2023b).
Existing methodologies for solving tasks that integrate KGs with LLMs can be categorized into two groups. The first one involves retrieving relevant triples from KGs in response to specific questions (Wang et al., 2023b; Luo et al., 2024; Jiang et al., 2023). The second part adopts an exploreexploit strategy, directing the knowledge utilization process within the graph according to the question (Sun et al., 2023b; Guo et al., 2023). However, both categories navigate the task-solving process by merely relying on the LLM’s analysis of the question, overlooking the rich cognitive potential inherent in the abundant knowledge encapsulated
1 7417
in KGs. KGs, which store a wealth of informative and symbolic facts, should deeply participate in the reasoning process together with LLM rather than being merely treated as a static repository of knowledge (Pan et al., 2024). As the example in the upper panel of Figure 1, LLM analyzes the question and navigates towards Narrative location relation of entity The Call of The Wild. However, this entity has many neighboring entities with that relation, leading LLM to incorrectly infer Canada as the answer. In contrast, the bottom panel demonstrates how KG provides key patterns that reveal both The Call of The Wild and White Fang share the location Yukon. If LLM could observe this information beforehand, it would precisely guide its reasoning process towards the correct answer (as shown in the bottom panel). Therefore, LLM should adopt an overall observation to incorporate the extensive knowledge and intricate patterns embedded within the KG. Achieving this objective presents two primary challenges: firstly, a global observation of the KG can result in an exponential growth in the number of triples. As shown in the upper panel of Figure 1, fully processing all 3-hop connections for The Call of the Wild is impractical. Secondly, the integration of such comprehensive observation into the existing reasoning paradigms of LLMs presents another challenge. How to combine the observation with the reasoning process of LLM matters for solving the tasks.
Motivated by this, we introduce a novel framework, the Observation-Driven Agent (ODA), aimed at sufficiently and autonomously integrating the capabilities of both LLM and KG. ODA serves as an AI Agent specifically designed for KG-centric tasks. ODA engages in a cyclical paradigm of observation, action, and reflection. Within ODA, we design a novel observation module to efficiently draw autonomous reasoning patterns of KG. Our observation module avoids the problem of exponential growth of triples via recursive progress. This approach ensures ODA integrating abilities of KG and LLM while mitigating the challenges associated with excessive data in KG, improving the efficiency and accuracy. Following the observation phase, ODA takes action by autonomously amalgamating insights derived from LLM inferences with the observed KG patterns. ODA can perform actions of three distinct types: Neighbor Exploration, Path Discovery, and Answering. Subsequently, ODA reflects on its internal state, considering both the outcomes of its actions and the
prior observations. This iterative process continues until ODA accomplishes the task at hand.
We conduct extensive experiments to testify to the effectiveness of ODA on four datasets: QALD10-en, T-REx, Zero-Shot RE and Creak. Notably, our approach achieved state-of-the-art (SOTA) performance compared to competitive baselines. Specifically, on QALD10-en and T-REx datasets, we observed remarkable accuracy improvements of 12.87% and 8.9%, respectively. We conclude the contributions as follows:
• We propose ODA, an AI Agent tailored for KG-centric tasks. ODA conducts observation to incorporate the reasoning ability of KG.
• We design action and reflection modules that integrate observation into LLM reasoning. This strategy leverages the autonomous reasoning of KG and LLM in synergy.
• We conduct experiments on four datasets and achieve SOTA performances.

heading: 2 Methods
text: In this work, we aim to solve tasks associated with KG. Let q represent a user question. The task T can be defined as generating an answer Y given a question q, task-relevant entities E = {e0, e1, ..., ek}, and a KG denoted as G. Formally, the task T can be expressed as:
T : (q, E), G → Y Employing an iterative approach, ODA tackles the challenges inherent in KG-centric tasks. In contrast to existing methods that couple LLMs and KGs and rely solely on analyzing the LLM’s query, ODA autonomously integrates observed knowledge from the KG into the entire reasoning process, resulting in more informed decisions. To achieve this objective, our ODA system, illustrated in Figure 2, primarily comprises three key modules for task resolution:
• Observation: This module efficiently observes and processes relevant knowledge from the KG environment. In each iteration i, it constructs an observation subgraph (denoted as Oi). By leveraging insights and patterns gleaned from the KG, this subgraph is autonomously incorporated into a reasoning LLM. This synergistic integration equips ODA with enhanced capabilities from both the LLM and KG, allowing it to tackle tasks more effectively.
2
• Action: Drawing upon both the observation subgraph Oi and ODA memory (denoted as M<i), the action module,represented by ai, strategically selects the most suitable action to execute on the KG, ensuring the accurate answering of the question.
• Reflection: Utilizing the observation subgraph Oi, the reflection module provides feedback by reflecting on the knowledge obtained from the action step. The reflected knowledge is then stored in memory Mi for the next iteration, facilitating continuous reasoning.
Through this iterative process, ODA dynamically updates its observation subgraph Oi and memory Mi at each iteration i. Each module is discussed in detail in the following sections.

heading: 2.1 Observation
text: The observation module is designed to inspect global KG knowledge and navigate the autonomous reasoning process with the KG environments. At each iteration i, it leverages task-relevant entities Ei and a question q to generate an observation subgraph Oi. This process can be formulated as:
Oi = Observation([Ei, q])
Initially, the task-relevant entities are populated with the entities embedded within the question q.
For KG-centric tasks, the observation incurs the problem of an explosive number of nodes. To address the scalability challenge during observation subgraph updates, we propose a D-turn observe
strategy, where D represents the maximum hop depth. Each turn has two steps: update and refine. The update step focuses on expanding the subgraph, while the refining step ensures its appropriate size without loss of important information.
For each entity e ∈ Ei, the observation module initializes the observation entities as a set Edo = {e} at hop depth d, where d represents the current search depth within the KG. The two-step, update and refine, iterates for each entity e until D is reached. The specific details are described as:
Algorithm 1 Observation Require: Question q, limit D, N , and P
Initialize task-relevant entities Ei with the entities in q for e ∈ Ei do
Set d = 0 Initialize observation entities Edo = {e} while d < D do
for entity ∈ Edo do Extract the neighboring triples end for for (r, t) ∈ triples do
Cosine similarity(q,r + t) end for Sort similarity scores of triples Append top N triples to Oi Extract top P% triples from top N Update Edo with t in top P% Increment d
end while end for
3
Update:
• For each entity e in Edo , neighboring triples are extracted from KG. Each triple takes the form [e, r, t], where r signifies the relation, and t denotes the tail entity.
• The similarity score between the question and the combined representation of r and t, is computed by measuring the cosine similarity of their embeddings*:
Cosine Similarity(vq,vr+t) = vq · vr+t
∥vq∥∥vr+t∥
• All triples associated with entities in E are collectively sorted in descending order based on their similarity scores.
• The Top-N triples are added to the observation subgraph Oi.
Refine: *We use the GPT text-embedding-ada-002 model from
OpenAI for encoding.
• The Top-N triples are further refined by retaining only the top P% with the highest similarity scores.
• The tail entities from the refined top P% triples are identified as the starting observation entities Edo for the next iteration.

heading: 2.2 Action
text: Harnessing the power of an LLM, the action module crafts strategic prompts to generate optimal actions. Based on its memory M<i, observation subgraph Oi, and historical actions a<i, the ODA selects the most accurate action ai.
ai = Action([Oi, a<i,M<i])
We propose three core actions designed to empower ODA :
• Neighbor Exploration: This action explores the KG neighborhood of task-relevant entities Ei and retrieves all neighboring triples. This helps build context and understand interconnectedness within the KG for ODA .
4
• Path Discovery: Given two entities in taskrelevant entities Ei, this action searches for all possible paths connecting them. Each path consists of interconnected triples, allowing the ODA to explore various connections and potentially uncover hidden relationships.
• Answer: This action responds to the question only if the required information is present in memory M<i.
Upon selecting an answer action, ODA halts the iterative loop of observation, action, and reflection. Leveraging the reliable knowledge within memory M<i, it can then directly formulate the answer to the question. Alternatively, if a Neighbor Exploration or Path Discovery action is selected, ODA strategically extracts relevant knowledge from the KG as a set of triples. These extracted triples are then fed into the subsequent reflection step for further processing. The prompt used here can be found in Table 7.

heading: 2.3 Reflection
text: The reflection module plays a crucial role in evaluating the triples generated from the action step and subsequently updating ODA memory Mi. Designed specifically for KG tasks, memory Mi adopts a subgraph format consisting of a network of paths that align with the inherent structure of KG, aimed at optimizing efficiency and relevance. By integrating the observation subgraph Oi and existing memory M<i autonomously, the reflection module provides invaluable feedback that guides future decision-making. This process can be formalized as:
Mi = Reflection([Oi, ai,M<i])
Given that memory Mi is structured as a network of paths, the reflection module navigates these paths to identify the first suitable one for integrating the reflected triple. This suitability arises from aligning the tail t of the last triple in the selected path with the entity e of the reflected triple. If a matching path is found, the reflected triple is appended. Otherwise, a new path is created based on the reflected triple. The maximum size of reflected triples is denoted as K.
Subsequently, the tail entities in the reflected triples are designated as the task-relevant entities for the next iteration. The specific prompt description used for the reflection module is provided in Table 8.
The observation, action, and reflection modules collaborate iteratively until either the Answer action is triggered or the maximum iteration limit is reached. Figure 3 shows how observation, action, and reflection work together.

heading: 3 Experiments

heading: 3.1 Dataset
text: To evaluate the performance of our ODA , we conduct experiments on four diverse KBQA datasets encompassing various task types: QALD10-en (Perevalov et al., 2022) for multi-hop reasoning, Creak (Onoe et al., 2021) for fack checking, and T-REx (Elsahar et al., 2018) and Zero-Shot RE (Petroni et al., 2021) for slot filling. Detailed specifications for each dataset are provided in Table 1. The Hits@1(Sun et al., 2019) accuracy with exact match is utilized as our evaluation metric.

heading: 3.2 Setup
text: We utilized the GPT-4 (OpenAI, 2023) model as the ODA via the OpenAI API. Throughout our experiments, we consistently configured the temperature value of GPT-4 to 0.4 and set the maximum token length to 500.
For the observation step, we tuned key parameters based on preliminary experiments. we set Pt to 10 and Nt to 50. Furthermore, the ODA loop was capped at a maximum of 8 iterations. Lastly, the maximum hop depth D is set to 3. As for the reflection module, we set the size of reflected triples K to 15.
To establish Wikidata KG database and retrieve information from it, we employed the simplewikidata-db* Python library. This library provides various scripts for downloading the Wikidata dump, organizing it into staging files, and executing distributed queries on the data within these staged files. Specifically, we deployed the Wikidata dump across five AWS EC2 instances, each consisting of a 768GB machine with 48 cores.
*https://github.com/neelguha/simple-wikidata-db
5
Considering that our ODA relies heavily on continuous interaction with the KG, we discovered that the real-time extraction of required Wikidata knowledge on AWS achieved an average completion time of 50 seconds per question-answer pair within QALD10-en dataset. However, as the KBQA dataset expanded, the cost of using the Wikidata database on AWS became prohibitively expensive. Consequently, to address the computational expenses involved, we devised a solution by generating an offline subgraph for each KBQA dataset. This offline subgraph captures all the triples within a 3-hop radius of the entities in each dataset, including the properties of both the entities and the relations involved. Notably, generating such a subgraph for the T-REx dataset, with its 4943 entities (as listed in Table 1), takes approximately 54 minutes and 42.834 seconds in practice.

heading: 3.3 Baseline Models
text: To comprehensively evaluate ODA effectiveness, we conduct a rigorous benchmark against several SOTA models across diverse categories. The comparison encompasses various models, starting with prompt-based approaches that do not utilize external knowledge. These include direct answering with GPT-3.5 and GPT-4, as well as the SelfConsistency (Wang et al., 2023c) and CoT (Sun et al., 2023b). On the other hand, Kownledgecombined models are considered, which incorporate fine-tuned techniques such as SPARQL-QA (Santana et al., 2022), RACo (Yu et al., 2022), RAG (Petroni et al., 2021) and Re2G (Glass et al., 2022). Additionally, there is ToG (Sun et al., 2023a) model, which integrates LLM with KG to bolster question-answering proficiency.

heading: 3.4 Main Result
text: Our ODA method outperforms existing methods, as shown in Table 2. On average, our method achieves an accuracy gain of up to 19.58% compared to direct answering with GPT-4, 19.28% compared to fine-tuned models, and 7.09% compared to TOG. These results demonstrate the efficiency and effectiveness of our method in comparison to other state-of-the-art methods.
Furthermore, our ODA significantly outperforms the prompt-based methods across various datasets, particularly showing an improvement of 65.50% and 23.77% on Zero-Shot REx and QALD10-en , respectively. These results underscore the importance of leveraging external knowledge
graphs for reasoning and completing the questionanswering task.
Compared to the fine-tuned method, our ODA method demonstrates superior performance. Specifically, our method achieves a performance gain of 21.27% for the QALD10-en dataset, 6.99% for the Creak dataset, and 50.56% for the Zero-Shot RE dataset. Notably, this interaction between the LLM and KG, as our method employs, proves more effective than data-driven fine-tuned techniques, despite requiring no explicit training.
Our ODA method exhibits significant performance gains over the ToG method across most datasets, with improvements of 12.87% (QALD10-en), 8.9% (T-REx), and 7% (Zero-Shot RE), despite both methods leveraging large language models and knowledge graphs. This performance disparity highlights the critical role of our observation module and the effectiveness of autonomously incorporating reasoning from KG. Specifically, our method demonstrates significantly stronger performance on the QALD10-en dataset, known for its multi-hop and complex reasoning requirements. This achievement underscores our ODA ability to exploit the rich knowledge and patterns within KG effectively, combining the autonomous reasoning strengths of both LLM and KG to tackle complex questions successfully.

heading: 4 Discussion
text: To better understand the key factors influencing our ODA, we conducted extensive analysis experiments. To conserve computational resources, we kept the previously mentioned datasets (QALD10-en, Creak, T-REx, and Zero-Shot RE) but randomly sampled 400 examples each for Creak, T-REx, and Zero-Shot RE.

heading: 4.1 Effect of Observation
text: To assess the efficacy of the observation module, we conducted comprehensive experiments with the model without observation. During the action step, ODA selects the action only based on the memory. Subsequently, the reflection step reflects on the triples outputted by the action and updates memory without the guide from observation.
A statistical comparison was performed to evaluate the performance of the ODA with and without observation across all datasets (see Table 3). The results show that the ODA with observation outperforms the ODA without observation, with an
6
average improvement of 3.14%. Specifically, for QALD10-en dataset, the ODA with observation outperforms the ODA without observation by 5.41%. Since QALD10-en involves multi-hop reasoning, the improved performance of the ODA with observation indicates that the observation module enhances the reasoning ability of the agent, enabling more accurate action selection and reflection.
We can further illustrate the benefits of the observation module with a practical case. In this scenario (see Table 6), question is Where are both The Call of the Wild and White Fang set, the most two famous works of Jack London?. Without observation, ODA generated the memory, such as (The Call of the Wild, narrative location, Canada), ultimately produced the wrong answer of Canada. However, with the observation module, the ODA correctly reasons the memory,such as (The Call of the Wild, narrative location, Yukon), (White Fang, narrative location, Yukon). As a result, the ODA with observation provides the correct answer, Yukon. This case exemplifies how the observation module improves the accuracy of action selection and reflection, consequently enhancing the reasoning ability of ODA .
By incorporating observation information, ODA reasoning power undergoes a dramatic leap, therefore generate an accurate answers. This boost stems from the synergistic interplay between the observation module, harnessing the KG’s autonomous reasoning capabilities, and LLM, which further amplifies those strengths.

heading: 4.2 Effect of Observation on Reflection
text: In this section, we discuss the impact of observation on reflection module. Three non-observation reflection methods were designed to verify whether observation can enhance the effectiveness of reflection. The similarity-based involves reflecting on the triples from action steps by calculating similarity. In this approach, triples are first sorted based on the similarity score between the r+ t and the question. The top-K triples are then selected and stored in memory for the next iteration. The random-based method randomly picks K triples from the action’s output and stores them in memory. Finally, the generated-fact method creates K natural language question-related facts for storage. All methods use a setting of K = 15.
Table 3 showcases our ODA dominance over all three non-observation methods. It achieved an average accuracy increase of 2.48% compared to the similarity-based method, 6.00% compared to the random-based method, and 3.66% compared to the generated-fact method.
In specific scenarios (see Table 5), when answering the question What is the capital of the prefecture Tokyo?, the generated-fact method resulted in problematic facts, such as Tokyo is the capital of Tokyo, and Tokyo is the capital of Japan. These were essentially hallucinations created by the LLM based on the given question, which misled the agent and resulted in incorrect answers. In contrast, the reflection of our ODA leveraging observation yielded factual knowledge, (Tokyo, instance of, prefecture of Japan), (Tokyo, capital,Japan) and
7
(Tokyo, capital, Shinjuku), consequently enabling the ODA to answer the question correctly.
The findings of Table 3 reveal that observation enables reflection module to generate more accurate memories, which translates to improved question-answering accuracy for ODA. This result underscores the value of both leveraging KG autonomous reasoning capabilities and fostering deep collaboration between KG and LLMs.

heading: 4.3 Performance across Different Backbone Models
text: To evaluate the effectiveness of ODA across various backbones, we analyzed its impact on performance in T-REx and QALD10-en datasets. We employed three backbones: GPT-3.5, GPT4 and DeepSeek-V2 (DeepSeek-AI et al., 2024). DeepSeek-V2 stands out as a powerful, economical, and efficient mixture-of-experts language model. Notably, DeepSeek-V2 surpasses the performance of LLaMA3 70B Instruct on standard benchmarks.
As evidenced by the Table 4, our ODA approach significantly outperformed the direct answering methods using GPT-3.5, GPT-4 and DeepSeek-V2. Notably, ODA demonstrated a remarkable 30.4% improvement in direct answering performance when utilizing GPT-3.5 model on QALD10-en dataset. This experiment suggests the generalizability of ODA across different LLMs.

heading: 5 Related Works
text: KG-enhanced LLM Knowledge Graphenhanced Language Models utilize two primary methodologies when tackling tasks that require integration with KGs. The first involves the extraction of relevant triples from KGs in response to posed questions. Wang et al. (2023b) prompt LLMs to generate explicit knowledge evidence structured as triples, while Jiang et al. (2023) develop specialized interfaces for gathering pertinent evidence from structured data, enabling LLMs to focus on reasoning tasks based on this information. Baek et al. (2023) retrieve facts related to the input question by assessing semantic similarities between the question and associated facts, then prepending these facts to the input. Meanwhile, Li et al. (2023) iteratively refine reasoning rationales by adapting knowledge from the KG. Wang et al. (2023a) dissect complex questions using predefined templates, retrieve entities from the KG, and generate answers accordingly. Luo et al. (2024) employs a planningretrieval-reasoning framework to generate relation paths grounded by KGs, thereby enhancing the reasoning capabilities of LLMs. Recently, graph retrieve augmented generation (GRAG) has been introduced to retrieve proper knowledge subgraphs rather than triplets (He et al., 2024; Hu et al., 2024; Mavromatis and Karypis, 2024). In GRAG approaches, subgraphs are first encoded into graph embeddings, and then retrieved based on their similarity to the query. Hu et al. (2024) proposes an additional step of filtering irrelevant entities within each retrieved subgraph.
The second approach employs an explore-exploit strategy that guides the knowledge utilization process within the graph. Sun et al. (2023b) perform an iterative beam search on the KG to identify the most promising reasoning pathways and report the outcomes. Guo et al. (2023) selectively accumulate supporting information from the KG through an iterative process that incorporates insights from the LLM to address the question. HyKGE (Jiang
8
et al.) first generates a hypothesis of the question by an LLM. Then it retrieves knowledge from the KG according to the entities of the hypothesis and answers the question based on the knowledge. To deal with incomplete KG, Xu et al. (2024) add a generation operation if some knowledge is missing.
Although these methods utilize the structural knowledge of KG, the searching or retrieving process is driven by the rationale of LLMs to the target question. Our method is the first to incorporate existing patterns in KG as a way to deeply bind the reasoning abilities of both LLM and KG via our novel observation mechanism.
AI Agent In the domain of AI agents, Yao et al. (2022) utilize LLMs to interleave the generation of reasoning traces with task-specific actions. Wu et al. (2023) propose an adaptable and conversational agent framework. This framework can operate in various modes, leveraging combinations of LLMs, human input, and auxiliary tools, resulting in a flexible and versatile system. Chen et al. (2023) focus on creating expert agents capable of solving complex tasks.

heading: 6 Conclusion
text: In this work, we design ODA framework for KGcentric tasks. In ODA, we introduce KG observation mechanism to autonomously combine the reasoning abilities of KG with LLM. We first propose the observation method to mitigate the problem of explosive number of triples in KG when tackling complex tasks. Then we fuse the observation into the action and reflection modules to further enhance the overall performance. We conduct extensive experiments, and the results clearly illustrate the effectiveness of our framework, highlighting its capability to enhance performance across four KBQA datasets, particularly in handling complicated questions.
Limitation
Given the diverse nature of KG-related tasks spanning multiple domains and requiring a broad range of capabilities, the implementation of a multi-agent system is promising to significantly enhance task performance. We leave the integration of our observation mechanism with multi-agent system to future work.

heading: A Case Study
text: To demonstrate ODA effectiveness, we highlight several representative cases from the QALD10-en dataset. For improved readability, we replaced entity IDs in Outputs with their corresponding names. We selected two examples to illustrate how ODA utilizes observation to correctly address the question in Table 5 and Table 6.
12
13

heading: B Prompt
text: This section presents the prompts for the action and reflection modules in tables 7 and 8.
14
15

heading: Future Work
text: We leave the integration of our observation mechanism with multi-agent system to future work