Hierarchical Planning for Complex Tasks with Knowledge Graph-RAG and Symbolic Verification
Authors: Flavio Petruzzellis
Cristina Cornelio
Pietro Lio
Venue: ICML 2025 poster (2025)
Keywords: Neuro-Symbolic
Hierarchical Planning
Robotics
Symbolic Verification
Knowledge Graph RAG
Primary area: applications->robotics
TL;DR: We propose a neurosymbolic method combining hierarchical planning, KG-RAG, and symbolic validation to enhance LLMs for complex task planning, ensuring correctness, failure detection, and offering a tool to evaluate LLM reasoning/compositional skills.

Abstract: Large Language Models (LLMs) have shown promise as robotic planners but often struggle with long-horizon and complex tasks, especially in specialized environments requiring external knowledge. While hierarchical planning and Retrieval-Augmented Generation (RAG) address some of these challenges, they remain insufficient on their own and a deeper integration is required for achieving more reliable systems. To this end, we propose a neuro-symbolic approach that enhances LLMs-based planners with Knowledge Graph-based RAG for hierarchical plan generation. This method decomposes complex tasks into manageable subtasks, further expanded into executable atomic action sequences. To ensure formal correctness and proper decomposition, we integrate a Symbolic Validator, which also functions as a failure detector by aligning expected and observed world states. Our evaluation against baseline methods demonstrates the consistent significant advantages of integrating hierarchical planning, symbolic verification, and RAG across tasks of varying complexity and different LLMs. Additionally, our experimental setup and novel metrics not only validate our approach for complex planning but also serve as a tool for assessing LLMs' reasoning and compositional capabilities. Code available at https://github.com/corneliocristina/HVR.

the faucet). This hierarchical Hierarchical Planning for Complex Tasks with Knowledge Graph-RAG and Symbolic Verification
Flavio Petruzzellis Cristina Cornelio Pietro Liò
Abstract
pot with water) and then breaking these down into a series of individual actions (e.g., picking up the pot, carrying to the sink, and turning process allows humans not only to plan the sequence of actions but also to ensure the plan is feasible before starting. Additionally, during execution, humans constantly monitor the task, identifying and addressing any issues that may arise (e.g., realizing the pot dirty or the faucet isn't working). For agents replicate this capability, they need manage actions varying levels of abstraction, know how to interact with objects in their environment, and detect and respond to planning or execution issues. These characteristics are particularly crucial for complex or long-horizon tasks.
Large Language Models (LLMs) have shown promise as robotic planners but often struggle with long-horizon and complex tasks, especially in specialized environments requiring external knowledge. While hierarchical planning and Retrieval-Augmented Generation (RAG) address some of these challenges, they remain insufficient on their own and a deeper integration is required for achieving more reliable systems. To this end, we propose a neuro-symbolic approach that en- hances LLMs-bbsed planners with Knowledge Graph-based RAG for hierarchical plan genera- tion. This method decomposes complex tasks into manageable subtasks, further expanded into executable atomic action sequences. To ensure formal correctness and proper decomposition, we integrate a Symbolic Validator, which also func- tions as a failure detector by aligning expected and observed world states. Our evaluation against baseline methods demonstrates the consistent sig- nificant advantages of integrating hierarchical planning, symbolic verification, and RAG across tasks of varying complexity and different LLMs. Additionally, our experimental setup and novel metrics not only validate our approach for com- plex planning but also serve as a tool for assess- ing LLMs' reasoning and compositional capa- bilities. Code available at https: // //t///gtttt github.
LLMs as planners. Recently, Large Language Models (LLMs) have emerged as promising tools for planning tasks in agentic systems (Huang et al., 2022). Their ability pro- natural language (NI.) and generate structured outputs makes them appealing for robotic task planning. Despite this potential, they continue to struggle with long-horizon tasks due to a lack of hierarchical reasoning and insuffi- cient knowledge of the specific properties of objects and environments. These limitations have driven the develop- ment of hierarchical planning, which decomposes complex tasks into multiple abstraction levels (Höller al., 2020). While this approach improves long-horizon robotic plan- ning, it still faces challenges in robustness and adaptability in complex enviromments. To address knowledge limitations, Retrieval-Augmennedd Generation (RAG) methods have been introduced, enabling systems to retrieve task-relevant infor- mation from external sources (Lewis al., 2020). Such approaches are particularly crucial in specialized settings, like healthcare (e.g., surgical robots), automated transporta- tion (e.g., spacecraft, autonomous vehicles), and domestic assistance (c.g., kitchen robots), where general common- sense knowledge falls short. However, while RAG improves access to relevant information, LLM-generated plans still lack the precision and reliability required for real-world robotics (Xie, 2020). LLMs generate plausible action se- quences relying on statistical correlations rather than logical inference, leading to inconsistencies and errors. To mitigate these issues, symbolic verification is essential for ensuring correctness before execution, reducing the risk of failures. Some methods attempt to translate NL instructions into for-
1. Introduction
Humans naturally tackle complex tasks, such as boiling water, by first conceptualizing high-level steps (e.g., filling
'Department of Mathematics, University of Padova, Padova, Italy 'Samsung AI, Cambridge, UK 'Computer Science Depart- ment, University of Cambridge, Cambridge, UK. Correspondence to: Cristina Comelio <c.comelio
samsung.com>. <acommlio
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
mal languages (Liu al., 2023a) integrate verifiers into planning pipelines (Xie, 2020), but they still face challenges in ensuring scalability and adapting to the complexities of real-world environments. Thus, while LLMs offer promis- ing capabilities for planning, their deep and effective inte- gration with hierarchical reasoning, retrieval, and symbolic verification remains an open challenge.
While newer, paid models may offer improved general per- formance, they would also benefit from our method, partic- ularly in complex tasks that require specialized knowledge.
Our results demonstrate that HVR significantly outperforms all baselines across both small and large LLMs, as well as tasks spanning different levels of complexity, from moderate to high. For smaller LLMs, RAG plays a crucial role in compensating for their limited reasoning capabilities, while for larger LLMs, hierarchical planning emerges the most impactful feature. The combination of these two approaches, coupled with symbolic verification, achieves the best overall performance. Our analysis reveals that -based planners still tend to generate unnecessarily long plans with extra steps, indicating areas for further improvement. Moreover, while performing very well on tasks with specific goal states, they struggle with more generic or open-ended objectives.
In this work, we introduce HVR, a method that enhances LLM planning capabilities through novel neuro-symbolic integration of Hierarchical planning, symbolic Verification and reasoning, and RAG methods over symbolic Knowledge Graphs. By decomposing complex tasks into manageable sub-tasks, our approach simplifies the planning process, while retrieving external information about the environment improves accuracy and reduces hallucinations. A Symbolic Validator further enhances reliability by simulating the plan in an "ideal world", enabling the identification and correc- tion of potential issues before execution. Additionally, it acts as a failure detector, monitoring discrepancies between the ideal and real-world states during execution, allowing the system to adapt and correct itself in real time (Cornelio & Diab, 2024; Liu et al., 2023b).
State of the Art. Recent research has explored the use of pre-trained LLMs for planning tasks in interactive environ- ments, leveraging their embedded knowledge to generate plans (Huang et al., 2022; Wu et al., 2023; Ahn et al., 2022; Yao et al., 2020), but these methods often face hallucina- tion and lack robust reasoning (Golovneva et al., 2022; Ahn et al., 2022). Hierarchical planning has been studied in com- bination with LLMs, such as HiP for symbolic planning combined with visual grounding (Ajay et al., 2024), and MLDT for multi-level decomposition (Wu et al., 2024). RAG systems (Lewis et al., 2020), which retrieve task- specific knowledge from sources like text (Guu et al., 2020), knowledge graphs (Edge al., 2024). or the web (Kim et al., 2024), have also proven effective, especially for long- horizon tasks (Lu et al., 2023). Recent works employ exter- nal verifiers to refine LLM planning, as seen in the LLM- Modulo framework (Kambhampati et al.), where tools like CLAIRIFY (Skreta et al., 2023) ensure syntactic correct- ness, and symbolic verifiers improve LLM performance by validating plans (Valmeekam al., 2023). Our work takes inspiration from these methods, integrating symbolic verifi- cation and hierarchical planning within an RAG-enhanced I.I.-based planner to address complex, long-horizon tasks. However, most tasks addressed by current state-of-the-arr models are very simple, such as rearranging blocks or re- trieving objects in an environment, and do not reach the complexity of even the simpler tasks in our evaluation pool. See Appendix A for a detailed discussion of otate-of-the-art approaches and their differences from our work.
Bridging Model-Based & Policy-Basede Approaches. Tra- ditionally, autonomous robotics has relied on two main approaches (Chatzilygerouddi et al., 2019): model-based systems, enabling an agent to simulate future states and rewards using an internal model of the environment; and policy-based systems, focusing on learning optimal action strategy based on the current state and the goal of the task. We propose hybrid method where an ontology is used both to model the environment and to refine an LLM-generated policy along with a Symbolic Validator, enhancing its adapt- ability and robustness, particularly in situations with limited cases. An additional advantage
data and of our method is the ability to create a reusable macro-action library. allowing agents to share and reuse plans, reducing the need to regenerate plans. These macro actions can be shared, further organized, and refined by clustering similar actions from different robots, extracting high-level common- alities across varied hardware and capabilities.
Major contributions and results summary. The major contribution of this work is HVR, a novel neuro-symbolic system that integrates high-level reasoning with hierarchi- cal planning, efficient information retrieval with RAG over symbolic knowledge graph, and formal verification to enhance planning accuracy and formal correctness. Addi- tionally, we introduce a diverse set of metrics and tasks designed for robotic simulators. These not only evaluate the effectiveness of planning systems like HVR, but also pro- vide a comprehensive framework for assessing the reasoning and compositional capabilities of LLMs. Importantly, we selected freely available LLMs to ensure reproducibility.
2. The HVR Method
The problem we address is generating a plan to make a robot execute a task in an environment &, based on a description of the task in natural language (NL) and an ontology O describing & Figure provides an overview of our method, while Figure 2 illustrates the agent's execution and feedback.
2

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
Macre-pa pl--plln
ma ma*
LLM
Action parser
Policy
me!
ma
Agent
Report
Matrics
Validator
PC/ES
Numerical
New Task NL)
AA bllock
Bel
Results ki
LLM
Action parser
Action mapger
Poriec
nely
EPV/MPV/AABV
action outcome
mei
Retrieved Context
Detection
+A block
Fallure
memorization
Symbolic Validator
LLM
KG-inifi
Agenit KG
RAG
Ontelogy
MA clustering abstraction
Figure 1. Overview of HRV: (1) Given a natural language task description, a pre- trainrd frozen LLM to generate macro-pia pacro-plan (policy p) which is expanded into an AA-block (policy x). Retrieval-augmentee augmented generation (RAG) method retrieves relevant context from the agent knowledge graph (initialized with the environment ontology) to support plan generation, while a Plan Validator detects and triggers the correction of potential emrors. (2) Once the plan finalized, the agent executes the atomic actions (AA) from each AA-block within the environment (see Figure 2), while recording the execution details in the knowledge graph. (3) After each action, Symbolic Validator verifies alignment between the "ideal plan" and actual environment state, potentially detecting failures. The system then reports the performance of the LLM-based planner using a set of novel metries.
2.1. Ontology & Dynamic Knowledge Graph
for retrieval, which are particularly suited for efficient query- ing and reasoning over symbolic data. A general KG-based RAG process starts with query formulated based on the task requirements. This query is then used to search within the KG, retrieving nodes (entities) and edges (relationships) relevant to the task. The result is a subgraph composed of these nodes and edges, which serves as the context for the LLM, that can be either directly included in the LLM prompt or mapped to an embedding space that the model can understand. In approach, utilizing KG-based RAG to enhance plan generation by exploiting both the retrieved and the LLM implicit knowl- edge. Specifically, we extract a subgraph g' (of the full KG 9), containing instances of the objects required for the plan, along with their corresponding types, properties, and state.
We adopted the approach in Cornelio & Diab, 2024, which utilizes symbolic knowledge support planning. This knowledge is provided an ontology O describing kitchen environment, robot actions, and human preferences. The ontology includes a taxonomy of abstract object classes (uppercase, e.g.. Apple) and defines relationships between them (e.g., Apple is subclass of Sliceable). During task ex- ecution, the ontology is combined with real-time instances (lowercase, e.g., apple-I), representing conerete objects, agents events. Together, the ontology O and these in- stances forms Knowledge Graph G, containing a set of relationships represented by either unary (indicating the class an entity) or binary (relations between entities) predicates. The Knowledge Graph G is initialized as copy of the ontology O. As the robot executes actions, it records each new event instance in G along with the multi-modal action outcome. Visual data is converted into triples repre- senting the scene-graph captured by the robot camera while auditory information is labeled and stored into triples.
use LLM as planner,
2.3. Hicrarchical Plan generation (H)
The input to our planning problem consists of NI. text including (1) the task definition, and (2) subgraph g' extracted from the knowledge graph G, containing relevant information for achieving the goal. The task definition can be provided in either a constructive form, which outlines instructions on how to solve the task (e.g.. "boil water in a pot and place it on a countertop"), or in a goal-oriented form, describing the desired end state (e.g., "I want a pot with boiling water on a countertop"). The final output of the system is sequence of atomic actions (AAs), which can be directly exccuted by the robot (c.g., put-on(pan, stove)). We refer to the complete set of all possible atomic actions the robot can execute within the environment as the action
2.2. Knowledge Graph RAG (R)
Retrieval-Augmenne Generation (RAG) (Lewis et al., 2020; Edge et al., 2024) is a well-established technique used in tasks like question answering, planning, or image genera- tion, especially when the task requires additional informa- tion from external sources (e-g., text or knowledge graphs) enhance generation. In our work, we specifically focus on Graph RAG methods, utilizing Knowledge Graphs (KGs)
3

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
Environment
8 l + Pas. where PM = (ma-, mam)) ) In our ap- proach, yp corresponds to the combination of pre-trained frozen LLM and an action parser: vot) = (arse (t)). The LLM associates the input text t with a transformed text t' ttrouuhh the mapping LLM : + t'. Subsequently, the action parser parse(-) processes the generated text t' and extracts each line as an action string. The generation of an AA-block is achieved using a tusk-conditioned policy n, which is a function that maps a macro action ma' to sequence P of atomic actions: т maa" + P. where P = {aaj, aog,..., .aak,) with aa) E A, and k; the length of the AA-block. Similar to the macro plan generation, N also employs a pre-trained frozen LLM combined with an action parser. However, it introduces an additional com- ponent, an action mapper μ. ((() = uuparse seuurrs((LL) (t)) which associates the transformed input text E' with se- quence of AAs, mapping it as μ t' + P. The action mapper identifies the corresponding action in the action set A that is closest in the embedding space to each action string (see Section 3). The policy π can also be used to generate the full E-plan directly from the task description, bypassing the generation of MAs.
() Event& Audio classification
E-plan
ACT: aia
3a-1
an-2
Robot
8ain
OBSERVE
Scene Graph
Figure 2. An agent exccutes an Expanded plan in a kitchen envi- ronment following its sequence of atomic actions aaj. For cach action, the robot interACTs with the environment and OBSERVES the resulting state. Visual observations are captured as a scene graph, representing object relative positions and states, while audi- tory feedback is processed through classification. This multi-modal outcome corresponds to the Agent's action outcome in Fig. 1.
Macro
Example: T1 - Serve wine
Actions
AA-block,
E-plan
mal:
Pick
na"; navigata_jaaa oobbbjbatlll
bottle
aa's ş bott-
ea; navigale_to, bb_oob/oottle1)
wine
sny pick 10/tene-bottle-1)
Pour wine
ma?:
s=²; navigate 1bjtoup-1]
AA-block,
any navigate__a__ohe o1
into the cup
an²z: pounpporee-1
aRg pounpbotte-1.cupp1)
2.4. Plan Verification and Correction (V)
We represent both AAs and MAS using a syntax based on the Planning Domain Definition Language (PDDL) (MeDer- mott et al., 1998), a formal language widely employed in automated planning systems that provides standardized framework for defining objects, actions, and their associated preconditions and effects. In PDDL, a task is defined in two main parts: (1) the domain, which describes the atomic actions (c.g.. upp toggle_on) along with their respective pre and postconditions; and (2) the problem, which outlines the initial state of the environment and the target goal. In our approach, the atomic actions in the domain correspond A, and consists of agent-specific actions with explicitly defined pre- and postconditions, as the agent's available capabilities are known and finite. On the other hand, MAs can cover an unlimited range of combinations of actions in A. Thus, it is impractical for domain experts to manually specify all the possible preconditions and effects that would determine the feasibility of each MA. To address this, we leverage the LLM to dynamically generate these conditions.
Figure 3. The task "Serve wine" is divided into multiple macro actions MAs (ma', ....., .....,,a""" ma"") such as "Pick up the bottle of wine" and "Pour wine into the cup". Each macro action is decomposed into AA-block with atomic actions AAs (oa),.. ...i....... ... aai,) ) such as navigate to obj. pick up, and powr. The final expanded plan (E-plan) is the concatenation of all the AAs from each AA-block.
set A, and this set is assumed be finite.
Since for complex tasks the sequence of atomic actions (AAs) can become quite lengthy, we decompose the task into a series of subtasks called macro actions (MAs) (ma?,... ma"".. Each macro action ma² is a high-level, concise NL description of the subtask goal and can be further expanded into sequence of AAS (aaf,...; oat,) )- We refer to each ordered sequence of K; AAs expanding macro action maa as an AA-block. These AA-blocks are then concatenated to form the final expanded plan (E-plan) (aaj,..., a.,),, where n = Em h; See Figure 3 for an example. It is important to notice that RAG is employed at each step both for generating MAs and AA-blocks, as requirements for different MAs might vary and the KG is dynamic: object states change over time, and new objects can be created/destroyed (e.g., through actions like slicing).
Heuristic- based Plan Verification and Correction. For each given macro plan, expanded plan, or AA-block, we initially employ a deterministic heuristic to adjust actions, such as inserting any missing navigation steps. This process leverages information defined in the PDDL. For instance, if the original sequence of actions is (pick-up, tomato-1), J, (put- on, tomato-1, counteriop-l), the heuristic would correct it to (navigate-to-olh tomato-1), 1), (pick-up, tomato-1). (navigate- to-obj, countertop-l) ), (put-on, tomato-1, countertop-l) to ensure the plan includes all necessary navigation steps.
The generation of a Macro Plan (M-plan) is performed using a condicondi policy yp which a function map- ping a NI. text t to a sequence of macro actions (MAs):
4

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
Symbolic Plan Verification. For given macro plan, ex- panded plan, or AA-block, utilize Symbolic Validator to assess its feasibility. The validator indicates whether the
Table 1. Tasks implemented in the experiments.
Name
steps
objects
plan is valid or, if not, identifies the step where the enror occurred specifying which particular condition was violated.
T1
Serve wine
8
2
T2
Make coffee
9
2
moderate complexity
T3
Fry egg in pan
13
2
VAL (Howey et al., 2004) is an example of a widely used verification tool for PDDL plans.
T4
Toast bread
15
3
T5
Warm water (in microwave)
16
3
TSbis
Warm water (generic)
16
LLM-based Plan Correction. Our system employs back- prompting with LLMs to correct plans using feedback from the Symbolic Validator, addressing two types of errors: (1)
T6
Cook potato slice (in microwave)
20
T7
Salad
26
high complexity
T8
Vogan sandwich
30
When an error is identified in macro plan, the LLM adjusts the pre and postconditions by adding, removing, or modify- ing them based on the feedback; (2) If the error is found in an expanded plan or AA-block, the L.LM corrects the sequence of AAs by modifying, adding, or removing actions. For macro plan corrections (1), the LLM receives input the task description, the plan generated by the policy y based on that description, the original list of pre- and post-conditions for each MA, and details on the violated condition. Addi- tionally, the LLM is provided with a list of relevant objects retrieved with RAG with their properties (e.g. pickupaal,, cookable), states (e.g., closed, open, picked uu)), and triples about object locations (e.g.. ( (nside,objecc2)) but no in-context examples of correction are supplied. The LLM then generates an updated list of pre- and post-conditions for each MA. For AA-block corrections (2), the LLM uses the error information from the Symbolic Validator, along with the MA description and the sequence of atomic ac- tions executed up to that point. The model also receives one in-context example of correction and produces an updated version of the AAoblock.
T9
Cook egg and potato slice
32
T10
Complex salad
33
7
TI1
Tomato-ogg on toust
38
10
T12
Complex plate
41
10
corresponding atomic steps. During task execution, both AAs and MAs are recorded in the agent's Knowledge Graph G. Once the task is successfully completed, the macro ac- tions are then transferred from G to the ontology O. These actions are then organized by clustering similar macro ac- tions, even if they have been executed by different robots with varying hardware (e.g,, single- or dual-arm robots) or capabilities (e.g., a robot that can pick up objects versus one that can only push objects). The clusters are refined using established techniques such as summarization or abstrac- tion to identifyy high-level commonalities. This process is depicted in Figure I with dotted lines.
3. Experiments Setup
3.1. A12Thor simulator and OnthoTTor Ontology.
Aligning observations and expected world state. At run- time, we employ the same PDDL description to verify if the expected environment state aligns with the actual state. This done by aligning symbolic information from the scene graph (encoding the visual data captured by the agent) with the expected state generated by executing the PDDL in planner. The Symbolic Validator thus acts as a failure detector: if a discrepancy is found between the "ideal world" triples and those in the scene graph, the action's 's expected pre- or post-conditions were not met in the environment, suggesting incorrect execution. Upon identifying a failure, established correction techniques (Cornelio & Diab, 2024: Liu et al., 2023b) can be applied.
A12Thor is a 3D simulator that provides a realistic virtual enviromment where agents can navigate and interact with objects in a home setting. Following previous studies (e.g.. Cornelio & Diab, 2024; Liu al., 2023b), we focused on the kitchen domain. We used A12Thor as state transition function T(s, au) = 8', which maps an environment state s to the next state s' through the application of an atomic action aa. Formally, A12Thor acts as T : SS x A + S where S denotes the state space of the environment E.
We employed OntoThor (Cornelio & Diab, 2024), which describes the A12Thor kitchen environment, as our ontology.
This ontology includes not only classes representing agents, physical objects, and their properties but also the framework to handle events, actions, and to capture audio-visual inputs from the robot sensors in the form of triples.
2.5. Macro Action Library and Knowledge Transfer
A key advantage of our method is the ability to build a reusable library of macro actions that can be accessed by the same agent for future tasks shared across multiple agents. This eliminates the need to regenerate plans from scratch, facilitating knowledge transfer and enabling agents to bene- fit from a collective "culture" of successful actions and their
3.2. Tasks
We chose to conduct our experiments on the tasks defined in (Cornelio & Diab, 2024) for their complexity, as most tasks tackled by sfate-of-the-an models are simple, such as rearranging blocks or retrieving objects, and do not match
5

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
the complexity required for our study. In addition, we in- troduced two new complex tasks, T11 and T12!, and we duplicated T5 as TSbis: in T5, the robot must warm wa- ter specifically using the microwave, whereas in T5bis, the robot can warm the water using any available method, such as on a stove, in a kettle, or in the microwave. Table pro- vides an overview of these tasks, organized by the number of steps in the ground truth plan. The tasks are divided into two categories: moderate complexity tasks with up to 20 steps, and high complexity tasks with more than 20 steps and involving more than three objects. The ground truth plans are minimal, containing only the essential steps required for completion without unnecessary actions.
actions. Both the prompt template for generating macro actions and the one for generating AA-blocks include an updated list of objects in the environment along with their properties. When KG-RAG is enabled, this list is limited to only task-relevant objects.
In our experiments we implemented the LLM-based planner using both Shi---minni<4k-nnttte 2024), a small-scale open-source LLM specialized on reasoning tasks, and gemini- 5-flash (Anil et al., 2023), larger closed-source model with stronger reasoning capabilities, also admitting a bigger context. We selected freely available LLMS to ensure reproducibility.
(Abdin et al.,
Action parser and action mapper. We use an action parser to extract both macro and atomic actions from the LLM output by first splitting lines and removing line numbers, if present. For macro actions, which consist of a predicate and one or two object arguments (e.g., poggle-on(micrwwee put-on(pan, stove), as described in Section 2.3), the parser extracts and separates these elements. Each component is then mapped to a set of predicates and available objects taking the closest one via cosine similarity. However, this initial step does not guarantee the action's 's feasibility. To ensure the action is valid, symbolically generate all possible predicate-object pairs/triples in the set of valid actions A and match them with the extracted pairs/triples based on semantic similarity of their sentence embeddings.
3.3. Implementation details.
Knowledge Graph RAG. To provide the LLM-planner with relevant, up-to-date information about the envi- ronment's state, we use a Knowledge Graph Retrieval- Augmented Generation (KG-RAG) approach. The same frozen LLM identifies subset of relevant objects from those currently available in the kitchen, stored in the KG. Using zero-shot prompt, we provide the LLM with natu- ral language task description and complete list of objects (a feasible step due to the manageable size of OntoThor). The LLM selects a subset of objects it deems necessary for the task, which are then mapped to their corresponding instances in the KG using cosine similarity over their em- beddings. We then symbolically query the KG to retrieve relevant information about the selected objects, including their current state (e.g., open, closed, cooked), inherent properties (receptacle, cookable, pickable) and technical capabilities (c.g., appliances being toggleable).
or
Symbolic Validator. To define the pre- and post-conditions in our system we use conjunctive (and) as well as disjunc- tive (or) and conditional (when) PDDL statements. We implemented an ad-hoc validator in python that is based on PDDL adapted to the specific A12Thor environment. As mentioned in Section 2.4, we manually defined the pre- and post-conditions of the atomic actions, as they are tied to the A12Thor simulator. To generate pre- and post-conditions for the macro actions we employ a one-shot prompt template in which the LLM is given the following elements: short description of what formal pre- and post-conditions are, and what kind of predicates can be used to express them (e.g.. predicates describing objects state, properties or physical locations, etc.); the goal-oriented task description; the plan of macro actions generated to accomplish the task; and a list of relevant objects (selected via KG-RAG, if enabled).
LLM acting as a planner. To generate macro actions, we employ a two-shot prompt template that incorporates the goal-oriented task description and the list of relevant objects with their properties, retrieved by the KG-RAG approach. A frozen LLM is fed with this prompt and outputs an or- dered list of macro actions in natural language. To generate AA-blocks for a given macro action (described in natural language), the LLM is first asked to generate a verbal de- scription of the steps needed to complete the task, followed by sequence of atomic actions for the robot execute. The prompt briefly outlines key environment informations, such as the robot's capabilities in the A12Thor simulator, its single-arm limitation, and how to interact with kitchen appliances, including one example for each. Additionally, the LLM provided with the history of executed atomic actions, grouped by macro action if hierarchical planning is enabled. To ensure consistent results, employ in-context learning, providing two examples of plans for simple macro
4. Experiments
The main results can be summarized as follows: (1) HVR significantly outperforms all baselines across both small and large LLMs and tasks of moderate-to-high complex- ity. (2) RAG is crucial for smaller LLMs, while hierarchical planning is more impactful for larger LLMs, as a longer con- text makes retrieval less useful. (3) Formally correct plans strongly correlate with correctly generated plans, indicat-
Tasss were renumbered from (Comello & Diab, 2024); see Appendix B.4 for details on the mapping.
6

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
Table 2. Summary of results using Phi3 and Gemini across the metries Plan Correctness (PC), Length Discrepancy (I.D), Expanded Plan Verification (EPV), Macro Plan Verification (MPV), and Atomic Action Block Verification (AABV), averaged over all 12 tasks. PC is also averaged separately for moderate and higl-xxop tasks. MPV and AABV are reported after correction (with pre-correction values in parentheses). Green indicates metrics for plan generation and execution, purple and gray (for negative values) represent plan minimality metries, and blue indicates metrics for symbolic verification and plan correction. All results are reported as percentages.
Plan Correctness
Length Discrepancy
Verification
all
moderate
high
min
max
avg.
abs. avg.
EPV
MPV
AABV
HVR (our)
59.66
$1.59
67.73
39.39
562.50
203.39
203.39
47.39
(47.03)
74.20
(27.41) 37.14
HV
18.91
16.34
21.48
34.15
626.09
202.01
202.01
39.69
(63.51) 87,86
(6.90) 25.24
HR
55.50
51.48
59.52
46.34
223.53
123.72
123.72
20.08
(17.91)
VR
20.04
31,08
9.01
-24.39
262.50
41.59
58.75
28.57
R
17.92
28.18
7.67
-48.48
387.50
41.11
62.31
11.58
I.LM
11.89
18.92
4.85
-30.43
116.67
22.61
36.68
21.11
IVVR (our)
94.19
100:00
88.39
33.33
337.50
109.13
109.13
88.
(100.00)
100.00
(16.33) 79.83
HV
85.27
93.06
77.48
21.74
158.82
67.25
67.25
100.00
(92.11) 100.00
(42.11) 100.00
Gemini
HR
49.01
68.43
29.60
-50.00
75.00
32.31
41,40
25.82
(29.91)
VR
28,50
40.10
16.89
-17.39
126.67
20.02
27.33
32.14
R
23.87
37.56
10.18
-27.78
37.50
-4.43
23.20
15.36
LLM
17.91
31.68
4.15
-41.18
37.50
-3.49
19.54
16.37
ing the effectiveness of symbolic verification in improving plan quality. (4) LLM-based planners produce unneces- sarily long plans, even with RAG, hierarchical planning, and verification; corrections improve accuracy at the cost of introducing additional steps. (5) Simulator execution of correct plans does not always reach the end goal, with an average success rate of 95%, highlighting limitations in current state-of-the-art simulators. (6) LLM-based planners excel in tasks with well-defined goal states but struggle with open-ended objectives and increased task complexity.
discrepancies (minimum and maximum values). Negative values indicate that the generated plan is shorter than the GT-ppann, while positive values indicate it is longer.
Symbolic Verification and Plan Correction. To assess the formal correctness and feasibility of the generated plans and measure the impact of symbolic corrections in improv- ing the accuracy, we defined the following three metrics: (4) Expanded Plan Verification (EPV) indicates the extent to which the expanded plan (full sequence of atomic ac- tions) has been successfully verified. It is calculated as the ratio of verified steps to the total number of steps in the generated plan. EPV is computed after all corrections have been applied. (5) Macro Plan Verification (MPV) mea- sures the extent to which the macro plan has been verified. It is calculated as the ratio of verified macro plan steps to the total number of macro steps. MPV is computed in two stages: before and after any symbolic macro plan cor- rections. (6) Atomic Action Block Verification (AABV)
4.1. Metrics and Baselines
We proposed six new metrics to evaluate the system's 's per- formance and the effectiveness of the integrated L.M..
Plan generation and execution. To assess the quality of both plan generation and its execution, we introduced two key metrics: (1) Plan Correctness (PC) identifies the fur- thest point in the expanded plan that aligns successfully with the ground truth (GT) plan. It is calculated as the ratio of correctly planned steps up to the first incorrect step, divided by the total number of steps in the GT-plan. (2) Execution Success (ES) measures how far a correctly generated plan can be executed in the environment without a failure occur- ring. It is defined as the ratio of steps executed correctly, up to the first failure, to the total number of steps in the GTTp This highlights the limitations of the simulator, as it sometimes fails to execute even entirely accurate plans.
evaluates the extent to which the macro plan has been ver- ified at the level of atomic action blocks. It is determined by dividing the number of verified atomic action blocks by the total number of macro actions. AABV is computed both before and after corrections to the expanded plan.
Baselines. We evaluated our system against five basclines: (1) HV: Utilizes hierarchical planning with symbolic veri- fication but without RAG (the complete KG is provided as 2 (2) HR: Implements hierarchical planning and RAG but omits symbolic verification. (3) VR: Incorporates both symbolic verification and RAG, but without hierarchi- cal planning. (4) R: Uses only RAG, without verification or hierarchical planning. (5) LLM: The LLM serves purely as planner, with the entire KG provided as context?
Plan Minimality, To assess the length of the generated plan in comparison to the minimal ground truth plan we defined (3) Length Discrepancy (LD) metric which measures the discrepancy in the number of steps between the GT-plan and the generated plan. Results are reported as the average of absolute differences, signed differences, and the range of
POlyy feasible in small environments with limited objects.
7

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
4.2. Results
methods are not only crucial for overcoming the inherent limitations of simulators but are also indispensable for man- aging execution errors in real-world robotics applications.
We calculated the results for all metrics across all 12 tasks for all the LLMs under consideration (see Section 3.3). A summary of the results for Phi3 and Gemini is presented in
Plan Minimality: We analyzed the results for the plan minimality metric (see Table 2, with details in Appendix Tables 14 and 8) and observed that smaller LLMs tend to generate significantly longer plans, with HVR producing plans that are 200% longer than the minimal ground truth plan. Larger LLMs, on the other hand, generate more effi- cient plans, with HVR resulting in only a 100% increase in length over the minimal plan. While the use of more power- ful LLMs improves plan efficiency, our analysis shows that LLM- planners, on average, still produce plans with redundant steps (see Appendix B.1 for examples). Inter- estingly, HVR increases plan length compared to the other methods, but also improves accuracy. This suggests that the corrections introduced during planning may come with the cost of adding unnecessary steps. Furthermore, in methods that do not employ hierarchical decomposition, we observed negative plan length values, indicating that the generated plans are shorter than the minimal ground truth plan. This issue is absent in methods that use hierarchical decomposi- tion, which suggests that hierarchical decomposition helps avoiding oversimplification or omission of essential steps.
Table 2. The complete results are provided in Appendix D and C, omitted here due to space constraints.
Plan generation and execution. The results for plan cor- rectness in Table (details in Appendix Tables 12 and 6) demonstrate that HVR significantly outperforms the other methods. For larger LLMss, HV performs better than HR, highlighting the importance of symbolic verification for more capable models. In this case, RAG appears to provide minimal benefit, likely because the larger context length of these models is sufficient to include all the necessary information. However, this may change in scenarios with larger environments containing many actions and objects, where context limitations could make RAG more valuable. Conversely, for smaller LLMs, HR outperforms HV, indi- cating that the RAG component is essential in this case. The combination of hierarchical decomposition, RAG, and symbolic verification, consistently outperforms all other configurations, showing that these three components com- plement each other effectively. Furthermore, the difference in performance gains suggest that larger LLMs are better equipped to leverage the advantages of hierarchical decom- position, symbolic verification, and RAG. However, smaller LLMs, when paired with HVR, still achieve reasonable per- formance. Another key observation is the difference in Plan Correctness between tasks of moderate and high complex- ity. Using Phi-3, the performance gap between these two categories relatively small (32.93% vs 27.74% averaged on all the systems), whereas for Gemini, the gap is much larger (61. 80% vs 38.22% averaged on all the systems) sug- gesting that larger LLMs are very efficient on easier tasks and struggle only on long-horizon complex tasks.
Symbolic Verification and Plan Correction. The results for the EPV metric (see Table 2, with details in Appendix Tables 15 and 9) demonstrate that incorporating the verifier improves both the performance and quality of the generated plans. When analyzing EPV alongside plan correctness, strong correlation emerges, indicating that more for- mally correct plan is also closer to the ground truth plan. Also examining the results with MPV and AABV metrics (see Table 2, with details in Appendix Tables 16 and 10 for MPV and Appendix Tables 17 and 11 for AABV), it is evi- dent that plan correction leads to significant improvements, highlighting its effectiveness. The improvement in MPV is most pronounced with Phi-3, as Gemini already produces macro plans that are largely formally correct, reflecting its ability to accurately generate pre- and post-conditions. In contrast, Phi-3 struggles more with this aspect. For atomic actions blocks, achieving formal correctness is generally more challenging. However, the results show that the cor- rection substantially enhances plan quality for both LLM models and across different methods (HV and HVR).
For the tasks that were planned correctly, we evaluated the execution success ES metric and found that, on average, the simulator was able to execute approximately 95% (details in Appendix Tables 13 and 7) of the planned actions. This highlights certain limitations of current simulators, such as A12Thor, which lack the robustness needed to reliably han- dle complex or long-horizon tasks. Notably, this percentage is slightly lower for tasks with high complexity compared to those with moderate complexity. A likely contributing factor to these failures is the noise in the plans generated by LLM-based planners, specifically the presence of superflu- ous actions, which strain the simulator and increase error rates. These execution failures underscore the critical need for robust failure detection and correction methods. Our approach, which also serves as an effective failure detector (similar to other methods like (Cornelio & Diab, 2024)), of- fers a mechanism identify and address such errors. These
Additional results. Further results, including the perfor- mance comparison between T5 and T5bis, are provided in Appendix B.5, showing that LLM-based planners ex- cel in tasks with well-defined goal states but struggle with open-ended objectives. Additional details, including the comparison with state-of-the-art systems, efficiency consid- erations, a discussion on HVR limitations and the full set of experimental results can be found in the Appendix.
8

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
Impact Statement
2024. doi: 10448550ARRXIV.224444442 URL https:
/ //doi.org/10..... 4005448550//550 2404. 14219
This paper aims to advance Machine Learning and Robotics by improving planning in complex, dynamic environments. Our HVR method integrates LLMs with hierarchical plan- ning, symbolic verification, and RAG enable robots to handle long-horizon tasks by dynamically adapting plans based on real-time conditions. Such approaches are partic- ularly crucial in specialized domains like healthcare (e.g., surgical robots), automated navigation and transportation (e.g., spacecraft or autonomous vehicles), and domestic as- sistance (e.g., kitchen robots), where general commonsense knowledge is insufficient.
Ahn, M., Brohan, A., Brown, N.. Chebotar, Y., Cortes, O., David, B., Finn, C.,, Fu, C.,, Gopalakrishnan, K., Hausman, K., et al. Do as can, not as say: Grounding language in robotic affordances. arXiv preprint MxXvv:200.001911, 2022.
Ajay, A., Han, S., Du, Y., Li, S., Gupta, A., Jaakkola, T., Tenenbaum, J., Kaelbling, L., Srivastava, A., and Agrawal, P. Compositional foundation models for hi- erarchical planning. Advances in Neural Information Processing Systems, 36, 2024.
By combining LLM flexibility with symbolic verification, HVR improves planning accuracy, reduces errors, and cre- ates reusable macro-action libraries that allows knowledge transfer across agents. This enhances scalability, collabora- tion, and reliability in autonomous systems, enabling them to perform tasks with minimal human intervention by recog- nizing and correcting failures to ensure safer environment for humans.
Anil, R., Borgeaud, S., Wu, Y., Alayrac, J., Yu, J., Sori- cut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., Silver, D., Petrov, S., Johnson, M., Antonoglou, I., Schritwieser, J., Glaese, A., Chen, J., Pitler, E.,, Lilli- crap, T. P., Lazaridou, A., Firat, O., Molloy, J., Isard, M., Barham, P. R., Hennigan, T., Lee, B., Viola, F,, Reynolds. M., Xu, Y., Doherty, R., Collins, E., Meyer, C., Ruther- ford, E., Moreira, E., Ayoub, K., Goel, M., Tucker, G., Piqueras, E., Krikun, M., Barr, I., Savinov, N., Danihelka, I., Roelofs, B., White, A., Andreassen, A., von Glehn, T., Yagati, L., Kazemi, M.. Gonzalez, L., Khalman, M., Sygnowski, J., and et al. Gemini: A family of highly capable multimodal models. CoRR, abs/2312. 1 1805, 2023. doi: 10.44555/ARRXII23122.118 / //dpi. 008.448555/xixx/ wtw..i...111.... 11805.
HVR is designed to tackle complex scenarios highly rel- evant to real-world applications. Our experiments, using freely available LLMs to ensure reproducibility, show that even newer, proprietary models could benefit from our ap- proach, particularly for tasks requiring specialized knowl- edge. These work hold significant societal potential for safety-critical industries, where trust, explainability, and ac- countability supported by our symbolic verification are essential.
URL https:
Carta, T., Romac, C.,, Wolf, T., Lamprier, S., Sigaud, O., and Oudeyer, P.-Y. Grounding large language models in interactive environments with online reinforcement learn- ing. In International Conference on Machine Learning, pp. 3676-3713 PMLR, 2023.
References
Abdin, M. Jacobs, S. A., Awan, A. A., Aneja, J., Awadal-
lah, A., Awadalla, H., Bach, N., Bahree, A., Bakhtiari, A., Behl, H. S., Benhaim, A., Bilenko, M., Bjorck, J., Bubeck, S., Cai, M., Mendes, C. C. T., Chen, W., Chaudhary, V., Chopra, P., Giorno, A. D., de Rosa, G., Dixon, M., EI- dan, R., Iter D., Garg, A., Goswami, A., Gunasekar, S., Haider, E.,, Hao, J., Hewett, R. J., Huynh, J., Javaheripi, M., Jin, X., Kauffmaan, P., Karampatziakis, N., Kim, D., Khademi, M., Kurilenko, L., Lee, J. R., Lee, Y. T., Li, Y., Liang, C., Liu, W., Lin, E., Lin, 7., Madan, P., Mitra, A., Modi, H., Nguyen, A., Norick, B., Patra, B., Perez- Becker, D., Portet, T., Pryzant, R., Qin, H., Radmilac, M., Rosset, C., Roy, S., Ruwase, O., Saarikivi, O., Saied, A., Salim, A., Santacroce, M., Shah, S., Shang, N., Sharma, H., Song, X., Tanaka, M., Wang, X., Ward, R., Wang, G., Witte, P., Wyatt, M., Xu, C., Xu, J., Yadav, S., Yang, F, Yang, Z., Yu, D., Zhang, C., Zhang, C., Zhang, J., Zhang, L. L., Zhang, Y., Zhang, Y., Zhang, Y., and Zhou, X. Phi-3 technical report: A highly capable language model locally your phone. CoRR, abs/2404.142115
Chatzilygeroudis, K., Vassiliades, V., Stulp, F., Calinon, S., and Mouret, J.-B. A survey on policy search algorithms for learning robot controllers in a handful of trials. IEEE Transactions on Robotics, 36(2):328-33 347, 2019.
Cornelio, C. and Diab, M. Recover: A neuro-symbolic framework for failure detection and recovery. In IROS 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2024.
Das, D. and Chernova, S. Semantic-based explainable ai: Leveraging semantic scene graphs and pairwise ranking to explain robot failures. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp- 3034 3041. IEEE, 2021.
Edge, D.,. Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., and Larson, J. From local to global: A graph query-focused
rag approach to query focused summarization. arXiv preprint arXiv:2404. 16/30, 2024.

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
Golovneva, O., Chen, M. P., Poff, S., Corredor, M., Zettlemoyer, L., Fazel-Zarandi, M., and Celikyilmaz, A. Roscoe: A suite of metrics for scoring step-by-step rea- soning. In The Eleventh International Conference on Learning Representations, 2022.
Kambhampati, S., Valmeekam, K., Guan, L., Verma, M., Stechly, K., Bhambri, S., Saldyt, L.. P., and Murthy, A. B. Position: Lims can't plan, but can help planning in Ilm- modulo frameworks. In Forty-first International Confer- ence on Machine Learning.
Goyal, A., Friesen, A., Banino, A., Weber, T., Ke, N. R., Badia, A. P., Guez. A., Mirza, M., Humphreys, P. C., Konyushova, K., et al. autreeeal-uggmented reinforce- ment learning. In International Conference on Machine Learning, pp.
Kannan, S. S., Venkatesh, V. L., and Min, B.-C. Smart- Ilm: Smart multi-agent robot task planning using large language models. In IROS 2024: IEEE/RSJ International Conference on Intelliget Robots and Systems, 2024.
PMLR, 2022.
Kim, M., Bursztyn, V., Koh, E., Guo, S., and Hwang, S.- web agent planning with
Guan, L.. Valmeekam, K. Sreedharan, S., and Kambham- pati, S. Leveraging pre-trained large language models to construct and utilize world models for model-based task planning. Advances in Neural Information Processing Systems, 36:79081-79094 2023.
w. Rada: Retrieval-augmente llms. In Findings of the Association for Computational Linguisties ACL 2024, pp. 1351 13511-,3525, 2024.
Lewis, P., Perez, E.,, Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t,, Rocktäschel, T., et al. Retrievallauummenee generation for knowledge- intensive nlp tasks. Advances in Neural Information Pro- cessing Systems, 33:9459-9474, 2020.
Guu, K., Lee, K., Tung, Z., Pasupat, P., and Chang, M. Retrieval augmented language model pre-training. In International conference machine learning, pp. 3929- 3938. PMLR, 2020.
Li, S., Puig, X., Paxton, C., Du, Y., Wang, C., Fan, L., Chen, T., Huang, D.-A., Akyürek, E., Anandkumar, A., et al. Pre-trained language models for interactive decision- making. Advances in Neural Information Processing Systems, 35:31199-31221 2022.
Höller, D., Behnke, G., Bercher, P., Biundo, S., Fiorino, H., Pellier, D., and Alford, R. Hddl: An extension to pddl for expressing hierarchical planning problems. In Pro- ceedings of the AAAI conference on artificial intelligence, volume 34, pp. 1883-9891, 2020.
Liang, J., Huang, W., Xia, F., Xu, P., Hausman, K., Ichter, B., Florence, P., and Zeng, A. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 9493 9500. IEEE, 2023.
Howey, R., Long, D., and Fox, M. Val: Automatic plan validation, continuous effects and mixed initiative plan- ning using pddl. In 16th IEEE International Conference on Tools with Artificial Intelligence, pp. 294-301. IEEE, 2004.
Lin, B. Y., Huang, C.,, Liu, Q.. Gu, W., Sommerer, S., and Ren, X. On grounded planning for embodied tasks with language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 13192-13200, 2023a.
Huang, W., Abbeel, P., Pathak, D., and Mordatch, I. Lan- guage models as zero-shot planners: Extracting ac- tionable knowledge for embodied agents. In Interna- tional Conference on Machine Learning, pp. 9118-9147. PMLR, 2022.
Lin, K., Agia, C., Migimatsu, T., Pavone, M., and Bohg, J. Text2motion: From natural language instructions to feasible plans. Autonomous Robots, 4745511345 2023b.
Hughes, N., Chang, Y., and Carlone, L. Hydra: A real-time spatial perception system for 3D scene graph construction and optimization. 2022.
Jain, A., Jermaine, C., and Unhelkar, V. Rag-modulo: Solv- ing sequential tasks using experience, critics, and lan- guage models. arXiv preprint arXiv:2409. 12294, 2024.
Liu, B., Jiang, Y., Zhang, X., Liu, Q., Zhang, S., Biswas, J., and Stone, P. Llm+ p: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304. /477, 2023a.
Jin, Y., Li, D., Yong, A., Shi, J., Hao, P., Sun, F., Zhang, J., and Fang, B. Robotgpt: Robot manipulation learning from chatgpt. IEEE Robotics and Automation Letters, 2024.
Liu, Z., Bahety, A., and Song, S. REFLECT: Summarizing robot experiences for failure explanation and correction. In 7th Annual Conference on Robot Learning, 2023b.
Kagaya, T., Yuan, J., Lou, Y., Karlekar, J., Pranata, S., Ki- nose, A., Oguri, K., Wick, F,, and You, Y. Rap: Retrieval- augmented planning with contextual memory for mul- timodal I1m agents. arXiv preprint arXiv:2402.03610, 2024.
Lu, Y., Feng, W., Zhu, W., Xu, W., Wang, X. E., Eckstein, M., and Wang, W. Y. Neuro-symbolic procedural plan- ning with commonsense prompting. In The Eleventh International Conference on Learning Representations,
10

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
2023. URL httpt:///ppeerreeeeeee. id=10c57x9KM54
Singh, I,, Blukis, V., Mousavian, A., Goyal, A., Xu, D.,
Tremblay, J., Fox, D., Thomason, J., and Garg. A. Prog- prompt: Generating situated robot task plans using large language models. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp- 11523-11530. IEEE, 2023.
McDermott, D., Ghallab, M., Howe, A., Knoblock, C., Ram, A., Veloso, M., Weld, D., and Wilkins, D. Pddl-the planning domain definition language. In Technical Report DCS TR-1165, Yale Center for Computational Vision and Control, 1998.
Skreta, M., Yoshikawa, N., Arellano-Rubcch, S., Ji, Z., Kristensen, L. B., Darvish, K., Aspuru-Guzik A., Shkurti, F., and Garg, A. Errors are useful prompts: Instruction guided task programming with verifier-assisted iterative prompting. arXiv preprint arXiv: 14100, 2023.
Pallagani, V., Muppasani, B., Murugesan, K., Rossi, F,, Horesh, L., Srivastava, B.. Fabiano, F., and Loreggia, A. Plansformer: Generating symbolic plans using transform- ers. arXiv preprint arXiv:22/2.08663 2022.
Song, C. H.,, Wu, J., Washington, C.,, Sadler, B. M., Chao, W.-L., and Su, Y. LIm-planner: Few-shot grounded plan- ning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2998 00088,009,0000 2023.
Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., and Wu, X. Unifying large language models and knowledge graphs: A roadmap. /EEE Transactions on Knowledge & Data Engineering, (01):1-20, 2024.
Rajvanshi, A., Sikka, K., Lin, X., Lee, B., Chiu, H.-P., and Velasquez, A. Saynav: Grounding large language models for dynamic planning to navigation in new environments. In Proceedings of the International Conference on Auto- mated Planning and Scheduling, volume 34, pp. 4644444, 2024.
Trinh, T. H., Wu, Y., Le, Q. V., He, H., and Luong, T. Solv- ing olympiad geometry without human demonstrations. Nature, (2747995):476-482 20244
Valmeekam, K., Marquez, M., Sreedharan, S., and Kamb- hampati, S. On the planning abilities of large language models-a critical investigation. Advances in Neural Infor- mation Processing Systems, 36:75993-76000 2023.
Raman, S. S., Cohen, V., Rosen, E., Idrees, I., Paulius, D., and Tellex, S. Planning with large language models via corrective re-prompting. In NeurlPS 2022 Foundation Models for Decision Making Workshop. 2022.
Vemprala, S., Bonatti, R., Bucker, A., and Kapoor, A. Chat- gpt for robotics: Design principles and model abilities. arXiv preprint arXiv:2306.1755 2023.
Rana, K., Haviland, J., Garg, S., Abou-Chakra, J., Reid, D., and Suenderhauf, N. Sayplan: Grounding large language models using 3d scene graphs for scalable task planning. CORR, 2023.
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E. H., Le, Q., and Zhou, D. Chain of thought prompt- ing elicits reasoning in large language models. CoRR, abs/2201.11903 2022.
Ren, A. Z., Dixit, A., Bodrova, A., Singh, S., Tu, S., Brown, N., Xu, P., Takayama, L., Xia, F., Varley, J., et al. Robots that ask for help: Uncertainty alignment for large lan- guage model planners. In 7th Annual Conference Robot Learning, 2023.
Wu, J., Antonova, R., Kan, A., Lepert, M., Zeng, A., Song, S., Bohg, J., Rusinkiewicz, S., and Funkhouser, T. Tidy- bot: Personalized robot assistance with large language models. Autonomous Robots, 2023.
Pareee,, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J., Ellenberg, J. S., Wang, P., Fawzi, O., et al. Mathematical discoveries from program search with large language models. Nature, 625 (7995):468-4755 2024.
Wu, Y.. Zhang, J.. Hu, N., Tang. L., Qi. G., Shao, J., Ren, J., and Song, W. Mldt: Multi-level decomposition for com- plex long-horizon robotic task planning with open-source large language model. arXiv preprint 2XXV::2403118760 2024.
Schick, T., Dwivedi-YYi Yu, J., Dessi, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., and Scialom, T. Toolformer: Language models can teach themselves to use tools. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Xie, Y. Translating natural language planning goals with large-language models. The International Journal of Robotics Research, 2019:1,
Xu, W., Wang, M., Zhou, W., and Li, H. P-rag: Progressive retrieval augmented generation for planning on embodied everyday task. arXiv preprint arXiv:2409.11273 2024.
Silver, T., Hariprasad, V., Shuttleworth, R. S., Kumar, N., Lozano Pérez, T., and Kaelbling, L. P. Pddl planning with pretrained large language models. In NeurIPS 2022 foundation models for decision making workshop, 2022.
Yao, S., Rao, R., Hausknecht, M., and Narasimhan, K. Keep calm and explore: Language models for action generation

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
in text-based games. In Empirical Methods in Natural Language Processing (EMNLIP. 2020.
Zhou, L., Schellaert, W., M..Martinzz-llumed, F., Moros- Davoll Y., Ferri, C., and Hernández-Orallo, J. Larger and more instructable language models become less reliable. Na- ture, pp. 1-8, 2024a.
Zhou, Z., Song, J., Yao, K.. Shu, Z., and Ma, Isr-Ilm: Iter- ative self-refined large language model for ong-horizon sequential task planning. In 2024 IEEE International Con- ference on Robotics and Automation (ICRA), pp. 2081- 2088. IEEE, 2024b.
Zhu, Y. Ou, Z., Mou, X., and Tang, Retrieval-augmentte embodied agents. Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 17985-17995, 2024a.
Zhu, Y., Qiao, S., Ou, Y., Deng, S., Zhang, N., Lyu, S., Shen. Y.. Liang. L.. Gu, J., and Chen, H. Knowagent: Chowledge-augmente planning for Ilm-based agents. arXiv preprint 243:0:203303101, 2024b.
12

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
A. Extended State of the Art Review
LLM as planners Recent studies have explored the use of pre-trained Large Language Models (LLMs) for planning and executing actions in interactive environments by leveraging the prior knowledge embedded in LLMs for plan generation (Wu et al., 2023; Wei et al., 2022; Ahn et al., 2022; Yao et al., 2020; Huang al., 2022: Schick et al., 2023). Typically, this process involves translating multi-modal observations into natural language, using the LLM to generate domain-specing actions or plans, and subsequently employing an agent to execute them. However, these methods are prone to hallucination (Golovneva et al., 2022; Pan et al., 2024) and often lack the ability for deep understanding and reasoning (Ahn et al., 2022). The rapid advancements and growing adoption of LLMs in recent years have facilitated their extensive application across diverse Al research domains, including robotics (Ren et al., 2023; Das & Chernova, 2021: Jin et al., 2024; Vemprala et al., 2023). LLMS can be directly employed for generating plans (I.in et al., 2023a; Xie, 2020; Huang et al., 2022; Carta et al., 2023; Song et al., 2023; Lin et al., 2023b). In addition to using off-the-shelf pre-trained LLMs, several studies have explored fine-tuning LLMs specifically for planning tasks (Valmeekam et al., 2023; Li et al., 2022; Pallagani al., 2022) However, these studies indicate that LLMs are more effective at goal translation than at actual planning, they often lack robust reasoning capabilities and are highly sensitive to prompt variations. Thus, LLMs have been utilized for translating natural language (NL) goals into symbolic languages. For instance, they have been employed to convert NL task descriptions into PDDL (Planning Domain Definition Language) representations (Guan et al., 2023; Liu et al., 2023a) allowing the problem to be solved by standard PDDL solver. Similarly, other studies (Liang et al., 2023; Singh et al., 2023) have translated NL instructions into Python-style code, making it easier to generate executable plans for robotic tasks. Finally, (Silver et al., 2022) studied how well pretrained large language models, with few-shot prompting, perform in the role of solvers for PDDL planning problems.
Hirearchical planning Hierarchical planning has recently garnered significant interest from the research community, as evidenced by the following studies: (Höller et al., 2020) introduce HDDL, extension of PDDL (Planning Domain Definition Language) that serves as standardized input language, enabling support for hierarchical planning tasks across various systems. HDDL aims to facilitate the comparison and integration of different hierarchical planning systems by providing a common feature set. (Ajay et al., 2024) introduce a model that integrates expert foundation models trained on language, vision, and action data to address long-horizon tasks. HiP employs an LLM for symbolic planning, a video diffusion model for visual grounding, and an inverse dynamics model to translate image trajectories into actionable steps, emphasizing iterative refinement across task planning, visual planning, and action planning levels. Unlike our symbolic approach, the hierarchy in HiP operates at the robot movement level rather than at the level of macro actions. (Wu et al., 2024) present MLDT, a method that tackles long-horizon task planning challenges by decomposing tasks into three levels: goal-level, leve-, and actio.-level. The approach uses three distinct prompting templates to generate plans at each level, with each stage conditioned on the output of the preceding layer. Finally, other works (Hughes et al., 2022; Rana et al., 2023) leverage the hicrarchical structures of the environment by utilizing 3D scene graphs for more effective task planning in robotic systems.
RAG systems for Robotics. Retrieval-Aummented Generation (RAG) (Lewis et al., 2020) systems enhance language model predictions by retrieving task-specific relevant knowledge from external sources such as text (Guu et al., 2020), knowledge graphs (Edge et al., 2024), or web (Kim al., 2024). RAG has become increasingly important for generative models, especially given recent findings that indicate L.MSs are becoming less reliable (Zhou et al., 2024a). (Goyal et al., 2022) explore retrieval in the context of deep reinforcement learning (RL) agents, though it does not employ LLMs, which limits its applicability and efficiency. Other studies emphasize the retrieval of past experiences to support planning processes. For instance, (Xu et al.. 2024) focus on retrieving plans related to similar tasks, demonstrating the value of retrieval in supporting planning processes. Similarly, (Jain et al., 2024) enhances LLMs as policies by retrieving relevant past interactions from a continuously expanding database, which stores both previous interactions and feedback from critics; (Zhu al., 2024a) utilize a policy retriever to extract robotic policies from a large-scale policy memory; and (Kagaya et al., 2024) employ retrieval of past experiences from a memory structure to inform agent planning. Examples of Graph-RAC RAGG methods for planning include the work of (Rana et al., 2023) which performs semantic search for task-relevant subgraphs from a condensed representation of full scene graph representing the environment, optimizing the planning process; and the work of (Zhu et al., 2024b) which introduces KNOWAGENT, an method that enhances LLM planning capabilities by incorporating explicit knowledge retrieved from an action knowledge base and applying a knowledgeable self-learnin strategy to guide action paths during planning. Finally, the most relevant work to ours is the work of (Lu et al., 2023) which utilizes a KG-RAG method for retrieving relevant nodes from a knowledge graph (KG) while mapping the actions to only
13

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
feasible ones. However, their approach lacks comprehensive verification of the entire plan and is likely to encounter the same challenges LLMSs when handling complex, nang-horizon tasks.
Symbolic verification and LLMs The position paper by (Kambhampati al.) argues that LLMs are not suitable as standalone planners plan verifiers, and instead proposes framework that combines LLMs with external verifiers, named LLM-Modull Framework, with particular focus PDDL planning tasks. Several implementations of this concept exist. For instance, FunSearch (Romera-Paredes et al., 2024) and Alpha-Geomety (Trinh et al., 2024) employs a alternate between a fine-tuned LLM that generates potential solutions and an symbolic evaluator that evaluate them. In reinforcement learning scenarios with simulators, the simulator itself serves as plan evaluator and critique mechanism within the LLM- Modulo framework. For instance, in the works of (Rajvanshi et al., 2024) and (Ahn et al., 2022) the simulator help to explicitly filter the action choices suggested by the LLM. Another relevant work is CLAIRIFY (Skreta et al., 2023), which combines iterative prompting with program verification ensure that the generated programs are syntactically correct and incorporate environmental constraints. However, CLAIRIFY's verifier only checks the syntax of instructions and the existence of hardware and reagents used (thereby avoiding hallucinations), without validating the overall correctness of the plan. Similarly, another work (Raman et al., 2022) identifies and corrects errors in LLM-generated plans through iterative re-prompting using the error information. However, this approach is limited to handling preconditions, and assumes that error information is provided (c.g., generated by a simulator) rather than detected independently. Most relevant for our approach is the work by (Valmeekam et al., 2023), which demonstrates that LLM performance improves significantly when backprompting with the VAL-verifier (Howey al., 2004). This highlights the effectiveness of integrating verification tools in conjunction with LLM-based planning.
B. Additional details
B.1. Examples of alternative plans vs the ground truth plans
As discussed in the paper, generated plans often tend to be longer and include unnecessary steps, which can affect execution success even when the plan is correct leading to the desired goal. This partly due to LLM-based planners generating redundant actions during task decomposition and error correction. For example, task T3, where the goal is want a fried egg in a plate the countertop", the plan generated by HVR includes several superfluous steps that do not contribute achieving the objective efficiently.
ALTERNATIVE PLAN (32 steps)
GROUD THRUTH PLAN (16 steps)
(navigate_to_ob)) 8gg-1)
(navigate_to_obj,, Egg-1)
(pick_up, kgg-1)
(pick_up, kgg-1) (crack_obj, kgg-1)
(navigate_tooob) 8gg-1)
(crack obj, Egg-1)
naviiate_to_doe to obj, Pan-1)
(navigate_]] to to__jj, Pan-1)
(put_in, in, EggCracked, Pan-1) (pick_up) Pan-1)
(nsvigate to obj, CounterTop-1)
(put_on, BggCracked-1, CounterTop-1)
(havigate_to_obl) StoveBurner-1)
(navigate_to__bb), Pan-1)
(put_on, Fan-1, StoveBurner-1)
10
(pick_up, Pan-1)
(toggle_on, StoveBurner-1)
11
(havigate_to_obj.. StoveBurner-1
(toggle_off, StoveBurner-1)
(put_on, Pan-1, StoveBurner-1)
(pick_up, EggCracked-1)
(navigate_to_obj, StoveRurner-1)
(navigate_to_obj,, Plate-1)
14 (toggle_on, StoveBurner-1)
(put_on, EggCracked-1, Plate-1)
15 anavigate_to_oby) EggCracked-1)
(pick_up, Flate-1)
(pick_up, EggCracked-1)
(bavigate_to_db)) CounterTop-1)
[havigate_to_obj,, Pan-1)
(put_on, Flate-1, Topnter
(put_in, KggCracked-1, Pan-1)
19
[navigate_to_ob)] StoveBurner-1)
20
(toggle_off) StoveBurner-1)
Slavigat_tt_oda Plate-1) (pick_up, Plate-1)
23
(navigate_too_obj, EggCracked-1)
24
[navigate_to_ob)]. CounterTop-1)
25
(put_on, Flate-1, CounterTop-1)
(navigate_to_obj,, EggCracked-1)
27 (pick_up, RggCracked-1)
28
(navigate_to_obj, Plate-1)
14

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
(put_on, BggCracked-1, Plate-1)
30 (navigate_tt_ob)) Plate-1)
31 (pick_up, Plate-1)
(navigate_ter obj, CounterTop-1)
(nut_on, Plate-1, CounterTop-1)
B.2. Generic tasks
Tasks T12 and T5bis are open-ended, causing their ground truth plans to vary across rums based on the LLM's intentions. The goal for T12 is "Set the table and serve vegan meal", while for TSbis, is "I want warm water in cup." In contrast, all other tasks have a single manually created ground truth plan, as their goals can be achieved in a unique way with a minimal number actions. Since T12 and T5bis allow multiple ways to accomplish their goals, we generated ground truth plan for each run, aligning it with the LLM's specific intentions.
Two examples of ground truth plans (aligned with their corresponding generative ones) for T5bis are:
PLAN 1 (18 steps)
PLAN-2 (17 steps)
to (Pot-1)
navigate_to_ohe (Cup-1)
pick_up (Pot-1)
pick_up (Cup-1)
navigate_to_cot Loattte_t___oo obj (SinkBasin-1)
navigate_to_objj (SikkBaiinin))))
put_inn(Cup_,,snnkBB.i
[oggle_oon
toggle_otff[auuctt=]
tosgle__off(Faucct-11
pick_up(Pot-1)
havigate__t_cbbhiMccrrvvvvvs
navigate_to_obji
(StoveRurner-1)
10 put_on [fn_oonnnotttttttteeebrrneeee
navigate_to__bb
11 toggle_ong (Stovekhobb))
pick_up(Cup-1)
toggle_off ootile_offfffooo (Stovekoobb))
navigate_to_okt (Microwave-1) [M
13 pick up (Fot-1)
put_iin (
14 navigate_to_cb)
close_obj (Mirrowav-e)
15 pour (Pot-1,
toggle_on (Microwve--l)
navigate_to_obe (CounterTop-1)
toggle_off (Microwave-1)
pn (Pot-1, CounterTop-1)
open_obj (Microwave-l)
18 navigate_to. _to_ebjtooo____ooo (Cup-1)
pick_up (Cup-1)
19 pick_up (Cup-1)
Defining the ground truth plans is essential for computing the metrics.
B.3. Additional Implementation Details
The code will be made available upon publication.
Here some additional details about the implementation for reproducibility:
The plan correctness (PC) metric is adjusted for plans that follow a partial order, where actions can be executed in parallel or different sequences. For example, when preparing a salad with lettuce and tomato, the tomato can be cut and added first, or the lettuce can be prepared first-both sequences achieve the same goal and are valid plan linearizations. To account for this, the metric is adapted take the maximum score across all possible linearizations.
In the prompt for macro action generation, provide the classes of the available object (e.g..Apple) inform the LLM of what is present in the environment. In contrast, the prompt for AA-blocks expansion we list specific object instances (e.g., apple-T).
When correcting an AA-block, for each step, the system attempts correction up to 2 times, where x, the lenght of the action block dynamically updated based on the current number of steps. For example, starting with steps (x = 5), if correction added a missing step, x increases to 6. Similarly, & adjusts whenever steps are removed. To prevent an infinite loop of corrections, each block has a static upper limit of 50 steps.
For macro actions, an initial attempt is made to correct pre- and post-conditions, based on the symbolic validator feedback, followed by further corrections after each AA-block execution. the end of AA-block execution,
15

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
failurres occur, the system reviews and refines the pre- and post-conditions of the macro action that generated it, incorporating feedback from the environment. This refinement allows for updating and saving better quality of macro action in the macro-action "culture" library.
B.4. Task renumbering
In Table 3 we report the task renumbering we used in comparison with RECOVER (Cornelio & Diab, 2024). Task "Boil water a pot" from (Cornelio & Diab, 2024) was used as a one-shot example in our LLM-based planner. Additionally, we introduced two new complex tasks, T11 and T12. Task T5 was duplicated T5bis, with the difference that in T5, the rohot must warm water specifically using the microwave, whereas in T5bis, the robot can warm the water using any available method, such as stove, in kettle, or in the microwave.
Table 3. Task numbering: work RECOVER (Corelio & Diab, 2024) ("used as I-shot example)
HVR ID
RECOVER ID
Name
T1
TI
Serve wine
T2
T2
Make coffee
Moderate
complexity
T3
Boil water in pot
T3
T4
Fry egg in a pan
T4
T5
Toast bread
T5
T6
Warm water (in microwave)
T5bis
Warm water (generic)
T6
T7
Cook potato slice (in microwave)
T7
T8
Simple salad
High complexity
T8
T10
Vegan sandwich
T9
T11
Cook egg and potato slice
T10
T12
Complex salad
TIL
Tomato-egg toast
T12
Complex plate
B.5. Additional results: T5 vs T5bis
Table 4 presents the results comparing T5 "Warm water in cup using the microwave" (specific goal) and TSbis "Warm water in cup" (general goal) across various metrics. The results show that tasks with more general objectives lead to worse performance due to increased ambiguity in goal interpretation due to fact that there multiple ways to accomplish it. This highlights the challenge LLM-based planners face in handling open-enue open-ended tasks.
Table Results comparing T5 and T5bis over the different metries Plan Correctness (PC), Length Discrepancy (I.D), Expanded Plan Verification (EPV), Macro Plan Verification (MPV) after correction, and Atomic Action Block Verification (AABV) after correction.
Gemini
Phi3
PC
LD
EPV
PC
LD
EPV
TSbis
T5
TSbis
T5
TSbis
T5
T5bis
T5
TSbis
T5
TSbis
T5
HVR (our)
88.89
100.00
100.00
223.53
96.88
34.55
23.53
29.41
368.75
335.29
22.67
6.76
HV
52.94
88.24
175.00
158.82
100.00
100.00
0.00
11.76
868.75
305.88
5.16
11.59
HR
29.41
29.41
37.50
5.88
12.20
13.16
27.78
268.75
223.53
2.56
6.67
VR
17.65
37.50
23.53
7.32
51.22
11.76
17.65
125.00
11.76
5.45
46.15
17.65
52.94
25.00
-17.65
7.69
20.59
11.76
11.76
0.00
-5.88
8.57
8.33
LLM
17.65
17.65
0.00
-41.18
8.57
10.00
0.00
11.76
162.50
5.88
32.79
23.68
Avg.
0.32
0.49
62.50
58.82
38.78
38.25
0.11
0.14
298.96
146.08
12.87
17.20
B.6. Library of Macro Actions
ONTOTHOR (Cornelio & Diab, 2024) contains the class Action to represent the agent's 's interactions with the environment, such as object manipulations and environmental observation. We further refined this structure by introducing two subclasser
16

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
Atomic Action, which aligns with the original Action class in ONTOTHOR, and newly defined Macro Action class, used represent higher-level operations like boil-water.
During task execution, instances of both Atomic Actions (AAs) and Macro Actions (MAs) are stored in the agent's s Knowledge Graph G. Each Macro Action instance is connected to its natural language description- generated by the LLM and including pre- and post-conditions modeled as triples, extending the scene-graph representation introduced by Cornelio Diab (2024). Additionally, each Macro Action instance linked its corresponding sequence of Atomic Actions using the hasAtomicActton predicate.
the hasDescription predicate. These pre- and post-conditions could also be
Thus, after task execution, the Knowledge Graph 9 contains set of macro actions along with their associated pre- and post-conditions. If macro action is successfully verified by the environment state, it is then transferred to the ontology O, For details on how this alignment between observations and the expected world state is verified after each macro action, refer to Section 2.4.
B.7. Efficiency Considerations
Figure 4 shows the average computational time for the different models across the 13 tasks using Gemini- 1.5-flash as LLM. As expected, approaches involving hierarchical decomposition require (approximately three times) more time than those without. However, as LLMs become faster, the overhead introduced by the HVR framework becomes increasingly negligible, while substantially improving plan correctness. For example, running the same tasks with Gemini-2.0-faa reduced the average execution time for HVR from 3285.32 seconds 681.51 seconds 5x improvement.
3.78
U Gemini 1.5 flash
Avg Execution Time (Ik seconds)
3.49
I| Gemini 2.0 flash
3.3
1.29
1.1
1.07
0.68
0.68
0.51
0.57
0.53
0.39
HVR
HV
HR
VR
R
LLM
Figure 4. Average exccution times (in seconds) per model using Gemini 1.5 and 2.0 flash
In contrast to this improvement, relying solely on an L.M as a planner demands substantially larger context window, which quickly becomes impractical as task environment complexity increases. HVR addresses this limitation through retrieval-augee -ugmented generation (RAG), enabling it to dynamically access only the relevant information from the knowledge graph. This design ensures stable processing times and scalability, even the environment grows, whereas LLM-only approaches face escalating computational demands and eventual context window exhaustion.
Figure shows the trade-off between plan correctness and execution time across the different methods. While HVR takes longer to compute plans, it consistently achieves the highest correctness. In contrast, simpler methods like LLM or R achieve faster execution but produce significantly less reliable plans, demonstrating the benefit of HVR's more structured approach.
B.S. Comparison with the State-of-the-Art
We do not provide direct experimental comparison with existing tfate-of-the-r methods, as we were unable to find prior work that targets complex, horgzhorizon tasks in a kitchen environment compatible with our setup.
17

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
100
HVR
HV
80
Plan Correctness (%)
60
HR
40
VR
R
20
LLM
1,000
1,500
2,000
2,500
3,000
3,500
Exccution Time (scconds)
Figure 5. Efficiency of Methods: Plan Correctness vs. Execution Time in seconds with Gemini 1.5 flash
However, we conducted comparison experiments with two relevant approaches: Smart-LLM and ProgPrompt (see details below). While we also considered several other methods, we were unable to include them in our evaluation due to compatibility issues (see below for more detailed information).
Smart-LLM (Kannan et al., 2024) uses large language models for planning in robotic systems, with focus multi-agent coordination. It handles task decomposition, coalition formation, and task allocation among heterogeneous robot teams based on their capabilities. The approach focuses on generating executable Python code tailored to the robot skills and environment, with an emphasis on parallel and distributed execution across agents.
This work is closely related ours, it uses the same simulator (A12Thor) for the experiments and addresses complex tasks (even if in multi-agent setting).
In the data finallllllaet _eetttttttttttte folder in their GIT repository there are 7 rooms of which only 3 are kitchen: FloorP an FloorPlants FlccrPlan21. The others are bedrooms and living rooms which are outside the scope of our ontology. There are a total of 15 tasks these rooms of which only 12 were feasible in our setup: tasks out of 3 for P1 oorP1 P1rPaan an tasks out for FloorPl 1 and tasks out of for FloorPlan2 This is due to a key implementation choice: in SLM the robot has full knowledge of all object locations, including hidden ones (e.g.. inside fridge or cupboard), whereas in system, consistently with prior work like RECOVER (Cornelio & Diab, 2024), assume only visible objects can be used interacting with hidden objects is thus treated as failure.
We evaluated HVR on these 12 tasks, and it successfully planned and executed all of them.
ProgPrompt (Singh et al., 2023) uses large language models for robotic task planning, with focus on generating structured, Python-like programs that include pre/post condition in the form of assertions and recovery steps. This helps ensure the plans are grounded and executable. The method relies careful prompt engineering, incorporating imports, object lists, and example functions to guide the model's output.
In the progprompt http://ttttttttttttttttt/ttaaaee /nseen// folder of their GIT repository, there are 10 available tasks (see Table 2in Singh et al. (2023)), 7 of which are kitchen related. We excluded the following tasks as they are either human-speciff or fall outside the scope of our ontology: brush teeth; watch rv; and eat chips on the sofa. Although these tasks were originally designed for the VirtualHome simulator, we reimplemented them in the A12Thor simulator. Table 5 provides details of this reimplementation.
Forr these additional experiments we used Gemini flash, since these experiemnts were done in subsequent moment.
18

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
Table 5. Propttompt tasks translated from VirtualHome simulator to A12Thor simulator.
VirtualHome Description
A12Thor description
A12Thor room
put salmon in the fridge
put egg in the fridge wash the plate
FloorPlan2
wash the plate
FloorPlan
bring coffeepot and cupcake to the coffee table
bring kettle and apple to the table
FloorPlan
microwave salmon
microwave potato turn off light throw away apple make toast
FloorPlan2
turn off light
FloorPlan
throw away apple
FloorPlan
make toast
FloorPlan
HVR able to plan and execute correctly all of the tasks. However, it's worth noting that all of these tasks are relatively simple and fall well below the complexity of those in our benchmark. Additionally, ProgPrompt was evaluated using GPT-3, and their performance would likely improve with more capable LLM.
Other works we considered. We also considered other LLLM-aseed systems such as Liu et al. (2023a) and Guan et al. (2023). However, these methods are built on different simulators and/or involve tasks that are not compatible with our setup (c.g., organizing blocks on a table). As a result, a direct comparison would require a substantial and non- non-trivial reimplementation, which falls outside the scope of this work.
Similarly, we could not compare with PDDL-based planning systems on our tasks as we found no existing works using planners expressive enough to support tasks. As described in Section 3.3 ( ('Symbolic Validator'), our system uses conjunctive (and), disjunctive (or), and conditional (when) PDDL statements. We therefore implemented a custom validator in Python, adapted specifically to the A12Thor kitchen environment.
We have considered the implementation of Zhou et al. (2024b), which involves solving simple abstract tasks in a simulated kitchen environment (e.g., organizing generic 'ingredient' objects in 'pot' objects by following a 'recipe"). ) In this work, the authors generate both a PDDL domain and a goal state starting from a task specification. While this can work for simple environment with few available actions, we argue it is generally not possible to generate PDDL domain specifications from high-level natural language task specifications such the ones use in our work. Thus, we did not run experimental comparisons with this method.
Finally, we reviewed the implementation of Xie (2020) which uses the ALFRED simulator, a close relative of A12Thor. However, the implementation is too limited for our use case and does not support key functionalities like creating object slices after slicing an object, handling liquids, or modeling interactions such as opening/closing appliances, which are necessary to create plans for the vast majority of our tasks. The actions modeled in the implementation are more coarse goasee-raiied than the ones we used in work: for example, instead of modeling the act of opening microwave, the system simply assumes that an object becomes hot when the robot is near the appliance, without simulating the intermediate steps. For these reasons, the system is not suitable for meaningful comparison.
B.9. Limitations and Future Work
The HVR framework integrates hicrarchical planning, knowledge graph retrieval, and symbolic validation to enhance LLM- based task planning, but it also presents several limitations that point to promising directions for future work. One key limitation is its reliance on a fixed ontology and action space, which constrains generalization to new environments or tasks. While updating to the ontology currently requires expert domain knowledge, the predefined action space could be extended to include novel actions automatically using LLM-based techniques-sine similar to how HVR currently generates macro actions along with their corresponding pre- and post-conditions. Prompt sensitivity is another concern common to LLM-based systems, although recent models show improved robustness to variations in prompt phrasing. Additionally, HVR's current use of natural language mediate interactions between the LLM and the knowledge graph may not be the most efficient; leveraging sub-symbolic or embedded representations could improve this integration. Another limitation is the current restriction to a linear structure for the generated plans. Supporting partial-ordes plans would not only allow for more flexible planning but also make it possible to execute different branches in parallel, which is especially valuable in multi-agent settings. Finally, in our method, error correction is done independently for each part of the plan and does not address interdependencies between errors at different planning levels. more connected correction strategy that reasons
19

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
and atomic actions could help improve overall correctness.
C. Full Results using Phi3
In Table we report the results for the metric Plan Correctness (PC) for HVR and all the baselines approaches using Phi3 as LLM-based planner. The results for each method are presented per task, along with the averages across all 12 tasks, as well as separately for tasks of moderate and high complexity.
Table Results for the Plan Correctness (PC) metric using Phi3 across all tasks are presented as percentages. The results are provided as overall averages and also separately averaged for moderate and high-complextn tasks.
Moderate completity tasks
High completity tasks
T2
T3
T4
T5
T6
T8
T10
T11
T12
arg.
avg. moderate
arg. high
HVR (our)
37.50
100.00
31.25
72.22
29.41
39.13
100.00
100.00
25.71
69:44
51.22
60.00
59.66
$1.59
67.73
HV
37.50
0,90
0.00
44.44
11.76
4.35
20.69
18.18
8.57
16.67
53.66
11.11
18.91
16.34
21.48
HR
37.50
100.00
31:25
83.33
17.65
99:13
82.76
84.85
17.14
69:44
48.78
54.17
55.50
51.48
59.52
31.08
R 25.00
1000000
12.50
11.11
11.76
8.70
13.79
6.06
5.71
5.56
4.88
10.00
17.92
28.18
7.67
30.65
32.93
28.38
In Table report the results for the metric Execution Success (ES) for HVR and all the baselines using Phi3 L.MM-based planner. The results for each method are presented per task that planned correctly (100% Plan Correctness).
Table 7. Results for the Execution Success (ES) metric using Phi3 across all 12 tasks are presented as percentages.
Moderate complexity tasks
High complexity tasks
TI
T2
T7
T8
T9
TI0
TII
T12
HVR (our)
100.00
86.20
100.00
HV
IIR
100.00
VR
100.00
LLM
In Table we report the results for the metric Length Discrepancy (LD) for HVR and all the baselines using Phi3 as LLM-based planner. The results for each method are presented per task, along with the average across 12 tasks, the absolute average, the min and max values.
Table Results for the Length Discrepancy (LD) metric using Phi3 across all 12 tasks presented percentages. The results include averages, absolute averages, and the minimum and maximum values across tasks.
Mederste complesity
High complesity
2Vg
abs. avg.
HVR (our)
100,00
222.22
362.30
206.07
335.29
250.52
55.02
29.39
20000
100.00
50.10
39.39
562.30
203.39
HIV
587.50 30
156.67
106.23
30.00
205.50
626.19
41.38
121.21
83.71
97.22
34.15
34.15
625.09
202.01
202 01
200.00
223.53
79.31
46.34
-2.14
41.59
387.
44
12.50
-11.11
.3888
4.35
-37.95
-45.8
20.00
-8.33
-4.8
-45.8
387.50
41.11
62.51
LLM
30,00
-2.220
62.50
116.7
5.83
30.43
34.48
39.39
17.14
-2.78
21,95
-3043
116.67
22.61
36.68
20

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
In Table report the results for the metric Expanded Plan Verification (EPV) for HVR and all the baselines using Phi3 as LLM-based planner. The results for each method presented per task, along with the average across all 12 tasks.
Table 9. Results for the Expanded Plan Verification (EPV) metric using Phi3 across all 12 tasks are presented percentages, with task averages also provided.
Moderate complexity tasks
High complexity tasks
TI
T2
T3
T4
T3
T6
T7
T8
T9
T10
T11
T12
avg.
IVVR (our)
93.75
6.76
21.95
9,43
56.06
100.00
15.22
100
11.76
53.13
44,44
56.16
47.39
HV
61.82
11.59
5.99
100.00
100.00
45.83
26.83
32.88
12.31
49.09
25.35
4.55
39.69
HR
55.56
6.67
16.25
8.96
20.90
2.94
8.33
45.65
8.53
27.88
6.52
32.80
20.08
VR
35.00
46.15
53.57
12.82
57.14
25.00
14.81
20.00
11.76
4.00
36.76
25.81
28.57
R 28.00
8.33
6.00
29.73
8.11
26.47
10.00
5.66
3,75
3.61
4.17
5,08
11.58
LLM
13.04
23.68
7.14
6.67
28.33
42.11
18.31
3.66
3.80
5.26
4.05
97.22
21.11
In Table 10 we report the results for the metric Macro Plan Verification (MPV) for all the methods that use symbolic validation (i.e., HVR and HV) using Phi3 LLM-based planner. The results for each method are presented per task, along with the average across all 12 tasks. They are shown both before correction (first part of the table) and after correction (second part of the table).
Table 10. Results for the Macro Plan Verification (MPV) metric using Phi3 across all 12 tasks presented percentages. The results are shown both before and after correction, with task averages also provided.
Moderate complexity tasks
High complexity tasks
T2
T6
T10
T11
T12
avg.
Before correction
HVR (our)
100.00
60.00
0.00
66.67
33.33
100:00
14.29
100.00
14.29
25.00
28.57
22.22
47.03
100.00
100.00
33.33
40.00
40.00
25.00
28.57
28.57
100.00
63.51
After correction
HVR (our)
100.00
60.00
50.00
66.67
44.44
100000
85.71
100.00
71.43
37.50
85.71
88.89
74.20
HV
100.00
100.00
100.00
100.00
80.00
60.00
100.00
28.57
100.00
87.86
In Table 11 report the results for the metric Atomic Action Block Verification (AABV) for all the methods that symbolic validation (i.e., HVR and HV) using Phi3 as LLM--bsed planner. The results for each method are presented per task, along with the average across all 12 tasks. They are shown both before correction (first part of the table) and after correction (second part of the table). For comparison purposes, we also included results for HR in the first part of the table.
Table Results for the Atomic Action Block Verification (AABV) metric using Phi3 across tasks are presented as percentages.
The results shown both hefore and after correction, with task averages also provided.
Moderate complexity tasks
High complexity tasks
T9
T10
T11
T12
avg.
Before correction
HVR (our)
50.00
0.00
0.00
66.67
0.00
14.29
0.00
100.00
0.00
25.00
28.57
44.44
27.41
HV
40.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
14.29
28.57
0.00
6.90
HR
50.00
0.00
0.00
14.29
0.00
14.29
0.00
71.43
0.00
0.00
28.57
36.36
17.91
After corrcction
HVR (our)
$0.00
100.00
16.67
66.67
0.00
14.29
0.00
100.00
0.00
25.00
28.57
44.44
37.14
HV
40.00
0.00
100.00
100.00
0.00
0.00
0.00
20.00
0.00
14.29
28.57
0.00
25.24
21

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
D. Full Results using Gemini
Table 12 we report the results for metric Plan Correctness (PC) for HVR and all the baselines approaches using Gemini as LL--daeed planner. The results for each method are presented per task, along with the averages across all 12 tasks, as well separately for tasks of moderate and high complexity.
Table 12. Results for the Plan Correctness (PC) metric using Gemini across all 12 tasks presented as percentages. The results provided as overall averages and also separately averaged for moderate and high complexxy tasks.
Mederate complexity tasks
High complex
T2
T3
T1
T5
17
T11
T12
arg:
ayg. moderate
avg. high
HVR (onr)
100100
100.00
100100
10013]
100100
100000
106.00
100100
57.14
[00100
73.17
100100
94.19
[00UR]
XX 39
100.00
ERR
50,00
100.00
100.00
83.33
29.41
47.83
100.00
0.00
5.71
69.44
2.44
0.00
19.01
68.43
29.60
33.33
50.00
100.00
12.50
5.56
52.94
4.35
13.79
3.03
25.71
2.78
2.44
13.33
23.87
37.56
10.18
LLM
30,00
100,00
12.50
5.56
17.65
4.35
13.79
3,03
2.86
2.78
2.44
0.00
17.91
31,68
4.15
49.79
61.80
37.78
In Table 13 we report the results for the metric Execution Success (ES) for HVR and all the baselines using Gemini as LLM-based planner. The results for each method are presented per task that was planned correctly (100% Plan Correctness).
Table 13. Results for the Exccution Success (ES) metric using Gemini across all tasks are presented as percentages.
Moderate complexity tasks
High complex tasks
TI
T2
T3
T4
T5
T6
T7
T8
T9
T10
T11
T12
HVR (our)
100.00
100.00
100.00
100.00
100.00
100.00
100000
100.00
88.89
100.00
HV
100.00
100.00
100.00
86.20
100.00
HR
100.00
100.00
86.20
VR
100.00
In Table 14 we report the results for the metric Length Discrepancy (LD) for HVR and all the baselines using Gemini as LLM-based planner. The results for each method presented per task, along with the average across all 12 tasks, the absolute average, the min and max values.
Table 14, Results for the Length Discrepancy (LD) metric using Gemini across all 12 tasks are presented percentages. The results include averages, absolute averages, and the minimum and maximum values across tasks.
Modirrtte complexity tisks
High complesity tokks
zvg
avg.
HYR (our)
332.50
33,33
10000
25.56
223.53
82.61
62.07
87.88
88.57
$0.56
48.78
33.33
337.50
109.13
109.13
HV
75.00
33.35
100.00
83.33
158.82
21.74
59.62
39.39
57.14
41.67
70.73
21.74
158.82
67.25
67.25
34.29
55.56
0.00
33.33
25.00
126.67
23.53
-17.39
24.14
0.00
-3.71
27.78
-17.07
-17.39
126.67
20,02
27.35
R
-2500
33.33
37.50
26.67
-17.65
-13.04
-24.14
-27.27
5.71
-2.78
-17.07
-27.78
37.50
-4.43
23.20
22

---

Hicrarchical Planning with Knowledge Graph-RAG and Symbolic Verification
In Table 15 report the results for the metric Expanded Plan Verification (EPV) for HVR and all the baselines using Gemini LLM--ased planner. The results for each method are presented per task, along with the average across all 12 tasks.
Table 15. Results for the Expanded Plan Verification (EPV) metric using Gemini across all 12 tasks presented as percentages. with
task averages also provided.
Moderate complexity tasks
High complexity tasks
T1
T2
T3
T4
avg.
HVR (our)
100.00
100.00
74.24
100.00
100.00
100.00
HV
100.00
100.00
100.00
100.00
100.00
100.00
100,00
100.00
100.00
100.00
100.00
100.00
100.00
HR
33.33
13.16
34.00
10.64
54.76
54.17
60.26
3,49
3.53
3.13
34.74
4.62
25.82
VR
42.11
51.22
48.89
12.82
53.85
50.00
13.24
13.04
14.08
35.90
10,59
40.00
32.14
R
29.41
20.59
6.52
12.20
8.11
54.17
12.96
5.00
12.00
3.85
4.62
14.89
15.36
LLM
29.41
10.00
6.82
12.20
8.82
54.17
12.96
4.48
4.29
8.14
5.13
40.00
16.37
In Table 16 we report the results for the metric Macro Plan Verification (MPV) for all the methods that use symbolic validation (i.e., HVR and HV) using Gemini as LLM-based planner. The results for each method are presented per task, along with the average across all 12 tasks. They are shown both before correction (first part of the table) and after correction (second part of the table).
Table 16. Results for the Macro Plan Verification (MPV) metric using Gemini across all 12 tasks are presented as percentages. The results are shown both before and after correction, with task averages also provided.
Moderate complexity tasks
High complexity tasks
T1
T3
T4
avg.
Before correction
HVR (our)
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
HV
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
5.26
92.11
After correction
HVR (our)
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
HV
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
In Table 17 report the results for the metric Atomic Action Block Verification (AABV) for all the methods that use symbolic validation (i.e., HVR and HV) using Gemini as LLM-based planner. The results for each method are presented per task, along with the average across all 12 tasks. They are shown both before correction (first part of the table) and after correction (second part of the table). For comparison purposes, we also included results for HR in the first part of the table.
Table 17. Results for the Atomic Action Block Verification (AABV) metric using Gemini across all 12 tasks are presented as percentages.
The results shown both before and after correction, with task averages also provided.
Moderate complexity tasks
High complexity tasks
T1
T2
T3
T4
T5
T6
T7
T8
T9
T10
T11
T12
Before correction
HVR (our)
0.00
100.00
11.11
0.00
12.50
25.00
0.00
11.11
9.09
7.14
0.00
20.00
16.33
HV
50.00
100.00
12.50
28.57
0.00
33,33
100.00
100.00
9.09
30.00
31.25
10.53
42.11
HR
0.00
75.00
12.50
85.71
16.67
33,33
85.71
0.00
0.00
50.00
0.00
0.00
29.91
After correction
HVR (our)
100.00
100.00
100.00
100.00
100.00
100.00
45.45
100.00
100.00
100.00
HV
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
23


Reviews:
[{"review_id": "I6pu8Pt0ea", "reviewer": "Reviewer_HvPs", "summary": "This paper proposes a LLM-based approach (HAR) to tackle long-horizon and complex robotic planning, which integrates hierarchical planning and Retrieval-Augmented Generation (RAG). Specifically, HAR leverages the LLM to decompose complex tasks into subtasks at different abstraction levels while integrates the RAG method to retrieve relevant context from the agent’s knowledge graph for plan generation. Then, HAR employs a Symbolic Validator to verify and correct t the generated plans. Experiments on multiple datasets of varying difficulty levels demonstrate the effectiveness of the HAR method across multiple LLMs.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback, which has greatly improved our paper. Below, we summarize the main concerns and detail the revisions and clarifications made to address them.\n\n**Q1: Missing details and experiments regarding the reusable library of macro actions**\n\nThe macro actions are stored in the ontology following the same approach used in Cornelio & Diab 2024. While we have not yet conducted experiments leveraging the macro action library to accelerate the planning process, we plan to explore this in the future. We added additional details in a new section in the appendix, summarized below:\n\nOntoThor contains the class Action representing agent-environment interactions. We refine this by introducing two subclasses: Atomic Action (AA), as the original Action class, and Macro Action (MA) representing higher-level operations like boil-water.\nDuring execution, AA and MA instances are stored in the agent’s Knowledge Graph. Each MA instance is linked to its NL description including pre- and post-conditions, via the hasDescription predicate. These conditions can also be modeled as triples, extending the scene-graph representation in OntoThor.\nEach MA instance is also linked to its sequence of AAs via the hasAtomicAction predicate. After task execution, the set of MAs, along with their associated conditions, are stored in the Knowledge Graph and, if validated by the environment (see section 2.4), also added to the ontology. \n\n**Q2: Missing References**\n\nThank you for the suggested references. We’ve added the missing citations and included SMART-LLM—previously unknown to us—in the revised manuscript, along with an experimental comparison (see Q3).\n\n**Q3: Missing comparison with State-of-the-art systems**\n\nWe did not include direct comparisons with existing methods, as none address complex, long-horizon kitchen tasks compatible with our setup. However, following reviewers’ suggestions, we considered additional works and included additional results/discussion in the Appendix of the revised paper, summarized below:\n* *SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models*\nThis work is closely related to ours, as it uses the same simulator and addresses complex tasks (but in a multi-agent setting). We considered only 15 tasks, excluding those not related to the kitchen domain, of which only 12 were feasible in our setup due to a key design difference: unlike SMART-LLM, which assumes full knowledge of all object locations (including hidden ones), our system—aligned with RECOVER —only considers visible objects (hidden objects are treated as failures). HVR with Gemini-2.0-flash successfully planned and executed all 12 tasks.\n* *ProgPrompt: Generating Situated Robot Task Plans using Large Language Models*\nIn this work there are 10 available tasks, 7 of which are kitchen related. We adapted the original implementation with the VirtualHome simulator to the AI2Thor simulator. Using Gemini-2.0-flash, HVR was able to plan and execute correctly all of the tasks. However, it's worth noting that these tasks are relatively simple compared to those in our benchmark. Additionally, ProgPrompt was evaluated using GPT-3, and their performance would likely improve with a more capable LLM.\n* *LLM+P: Empowering Large Language Models with Optimal Planning Proficiency* \nThese methods are built on different simulators and/or involve tasks that are not compatible with our setup (e.g., organizing blocks on a table). As a result, a direct comparison would require a substantial and non-trivial reimplementation, which falls outside the scope of this work.\n* *PDDL-based planning systems*\nWe could not compare with PDDL-based planning systems on our tasks as we found no existing works using planners expressive enough. As described in Section 3.3, our system uses conjunctive, disjunctive, and conditional PDDL statements. We therefore implemented a custom validator in Python, adapted specifically to the AI2Thor environment.\n* *ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning*\nHere, given specifications of simple abstract tasks in a simulated kitchen environment, the authors generate both a PDDL domain and a goal state. While this can work for a simple environment with few available actions, we argue it is generally not possible to generate PDDL domain specifications from high-level natural language task specifications such as those we use in our work. Thus, we did not run experimental comparisons with this method.\n* *Translating Natural Language to Planning Goals with Large-Language Models*\nThis work uses the ALFRED simulator, similar to AI2Thor. However, the implementation is too limited for our use case and does not support key functionalities like creating individual slices after slicing an object, or modeling interactions such as opening appliances—essential for our tasks— making it unsuitable for a meaningful comparison.", "claims_and_evidence": "Most claims are supported. However, in Section 2.5, the authors mention that one advantage of this method is its ability to build a reusable library of macro actions, but they do not provide experimental evidence to support its effectiveness.", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "No theoretical proof.", "experimental_designs_or_analyses": "Yes. This paper designs comprehensive and well-reasoned evaluation metrics to thoroughly assess the effectiveness of the proposed method. Additionally, through extensive ablation experiments, it validates the effectiveness of each component.", "relation_to_broader_scientific_literature": "This paper leverages the advantages of existing methods to propose a novel methodological framework.", "essential_references_not_discussed": "S. S. Kannan, V. L. N. Venkatesh and B. -C. Min, \"SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models,\" 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)\n\nZ. Zhou, J. Song, K. Yao, Z. Shu and L. Ma, \"ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning,\" 2024 IEEE International Conference on Robotics and Automation (ICRA)", "other_strengths_and_weaknesses": "Strengths：\n1. A novel planning framework that integrates hierarchical planning, RAG, and symbolic validation.\n2. Comprehensive and clear ablation experiments validate the effectiveness of each component. \n3. Designed effective and sufficient experimental evaluation metrics.\n\nWeaknesses：\n1. The sufficiency of the experiments has certain limitations. The paper conducts experiments on only one dataset and does not include comparative experiments with existing methods（smart-llm）.", "other_comments_or_suggestions": "Please refer to Other Strengths And Weaknesses", "questions_for_authors": "Please refer to Other Strengths And Weaknesses", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zKoj31JTUa", "reviewer": "Reviewer_ojF6", "summary": "This paper introduces HVR, a task planning method that integrates hierarchical planning, retrieval-augmented generation (RAG) over symbolic knowledge graphs, and formal verification to enhance the performance of large language models (LLMs) in complex task planning. The proposed method decomposes the language-described tasks into manageable macro actions and further into atomic actions, ensuring formal correctness through symbolic validation. Experimental results in the AI2Thor kitchen environment demonstrate that HVR outperforms baseline methods.\n\n## update after rebuttal\nIn settings where a precise world model exists, the world state is fully known, and the problem is guaranteed to be solvable, symbolic planners are already highly effective at solving planning problems. Therefore, a more meaningful and impactful research direction is to explore the use of LLMs for planning in scenarios where the world model is incomplete or unavailable. Extending HVR to operate under such conditions would significantly enhance its practical value and increase its potential for broader acceptance.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback, which has greatly improved our paper. Below, we summarize the main concerns and detail the revisions made to address them.\n\n**Q1: Why is it necessary to use an LLM for planning in this setting?**\n\nWhile prior works such as [1] and [2] show how natural language (NL) instructions can be translated into PDDL goals, they are limited to simple tasks and lack the flexibility needed for more complex scenarios. Moreover, symbolic planners tend to be brittle—minor changes in the environment or execution failures can break the entire plan. In contrast, HVR leverages the flexibility of LLMs throughout the whole planning process, addressing several key limitations of symbolic–only methods:\n* Partial plan execution: for particularly complex tasks, LLM-based models are able to produce at least some correct initial steps or an initial sub-task, while symbolic planners can only generate full plans and thus likely failing entirely on these type of tasks\n* LLMs enable dynamic adaptation during task execution, incorporating feedback from the environment to replan on the fly. Symbolic planners would need new mid-task NL instructions.\n* Thanks to LLMs, and even more thanks to the employment of retrieval-augmented generation (RAG), our approach achieves significantly faster planning, avoiding the exponential search space growth typical of classical planners.\n\nLastly, although HVR currently relies on a fixed world model, it can be extended to operate without one—just as we generate macro-actions and their pre/post-conditions dynamically.\n\n**Q2: Missing comparison with the state-of-the-art**\n\nWe did not include direct comparisons with existing methods, as none address complex, long-horizon kitchen tasks compatible with our setup. However, following reviewers’ suggestions, we considered additional works and included new results/discussions in the appendix of the revised paper. See the full response to *Review-Ac9Q*, summarized here:\n* *SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models* This work is closely related to ours, as it uses the same simulator and addresses complex tasks (but in a multi-agent setting). HVR with Gemini-2.0-flash successfully planned and executed all 12 tasks.\n* *ProgPrompt: Generating Situated Robot Task Plans using Large Language Models* We adapted the original implementation with the VirtualHome simulator to the AI2Thor simulator. HVR with Gemini-2.0-flash was able to plan and execute correctly all kitchen-related tasks.\n* *LLM+P: Empowering Large Language Models with Optimal Planning Proficiency* These methods are built on different simulators and/or involve tasks that are not compatible with our setup.\n* *PDDL-based planning systems* We could not compare with PDDL-based planning systems on our tasks as we found no existing works using planners expressive enough. \n* *Translating Natural Language to Planning Goals with Large-Language Models* This work uses the ALFRED simulator, similar to AI2Thor. However, the implementation is too limited for our use case and does not support key functionalities making it unsuitable for a meaningful comparison.\n\n**Q3: Missing discussion about HVR limitations**\n\nWe have included a discussion of our work’s limitations and future work in a new section in the appendix in the revised paper:\n\nThe HVR framework integrates hierarchical planning, knowledge graph retrieval, and symbolic validation to enhance LLM-based task planning, but it also presents several limitations that point to promising directions for future work. \nOne key limitation is its reliance on a fixed ontology and action space, which constrains generalization to new environments or tasks.  While updating to the ontology currently requires expert domain knowledge, the predefined action space could be extended to include novel actions automatically using LLM-based techniques—similar to how HVR currently generates macro actions along with their corresponding pre- and post-conditions. \nPrompt sensitivity is another concern common to LLM-based systems, although recent models show improved robustness to variations in prompt phrasing. Additionally, HVR's current use of natural language to mediate interactions between the LLM and the knowledge graph may not be the most efficient; leveraging sub-symbolic or embedded representations could improve this integration. \nAnother limitation is the current restriction to a linear structure for the generated plans. Supporting partial-order plans would not only allow for more flexible planning but also make it possible to execute different branches in parallel, which is especially valuable in multi-agent settings. \nFinally, in our method, error correction is done independently for each part of the plan and does not address interdependencies between errors at different planning levels. A more connected correction strategy that reasons across macro and atomic actions could help improve overall correctness.", "claims_and_evidence": "In this paper, the authors assume that a precise domain model is known (as stated in lines 194-198 on page 4) and that the environment state can be obtained through OntoThor. Given these assumptions, a classical planner (e.g., Fast Downward) should be able to generate a valid plan with the PDDL-style goal specification. Why is it necessary to use an LLM for planning in this setting? It seems that the LLM’s role could be limited to converting natural language task descriptions into PDDL goal specifications, as done in [1] and [2], after which classical planning would likely achieve a high success rate. The key advantage of using an LLM for planning is its ability to operate without a precisely defined domain model, which can be complex to construct. However, this paper still relies on an exact domain model, which raises questions about the necessity of employing an LLM for planning in this context.\n\nReferences:\n[1] Translating Natural Language to Planning Goals with Large-Language Models, ARXIV 2023.\n[2] LLM+P: Empowering Large Language Models with Optimal Planning Proficiency, ARXIV 2023.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "No, there is no theoretical claim.", "experimental_designs_or_analyses": "Yes. The design of the baselines serves as an ablation study of the complete method, allowing the effectiveness of the proposed components to be validated through experiments. \n\nHowever, it lacks a comparison with code-style task planning approaches, such as ProgPrompt [3].\n\nReference:\n[3] ProgPrompt: Generating Situated Robot Task Plans using Large Language Models, ICRA 2023.", "relation_to_broader_scientific_literature": "Hierarchical planning helps prevent oversimplification or the omission of essential steps in long-horizon tasks.", "essential_references_not_discussed": "See [1] and [2] mentioned in Claims And Evidence.", "other_strengths_and_weaknesses": "Strengths:\n1. The proposed method achieves good performance on various tasks in the AI2THOR environment.\n2. Several new metrics are valuable for evaluating the methods.\n\nWeaknesses:\n1. The paper lacks a discussion about the limitations of the proposed method.", "other_comments_or_suggestions": "No.", "questions_for_authors": "See the comments under Claims and Evidence, Experimental Designs or Analyses, and Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KONGEUk4HW", "reviewer": "Reviewer_9MAP", "summary": "The authors propose a neuro-symbolic approach that combines LLMs-based planners with Knowledge Graph-based RAG for hierarchical plan generation. It breaks down complex tasks into subtasks and then into executable atomic action sequences. A symbolic validator is integrated to ensure formal correctness, task decomposition, and to detect failures by comparing expected and observed world states.\n\n## update after rebuttal\nI will maintain my positive opinion.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback, which has greatly improved our paper. Below, we summarize the main concerns and detail the revisions and clarifications made to address them.\n\n**Q1: Missing comparison and discussion of efficiency**\n\nWe have included a study of the efficiency of our method including 2 new plots in a new section in the appendix \"Efficiency considerations\" in the revised paper:\n\nFigure A (*Average execution times (in seconds) per model using Gemini 1.5 flash*) shows the average computational time for the different models across the 13 tasks using Gemini-1.5-flash as LLM. As expected, approaches involving hierarchical decomposition  require (approximately three times) more time than those without. \nHowever, as LLMs become faster, the overhead introduced by the HVR framework becomes increasingly negligible, while substantially improving plan correctness.\nFor example, running the same tasks with Gemini-2.0-flash reduced the average execution time for HVR from 3285.32 seconds to 681.51 seconds—a 5x improvement.\n\nIn contrast to this improvement, relying solely on an LLM as a planner demands a substantially larger context window, which quickly becomes impractical as task or environment complexity increases. HVR addresses this limitation through retrieval-augmented generation (RAG), enabling it to dynamically access only the relevant information from the knowledge graph. This design ensures stable processing times and scalability, even as the environment grows, whereas LLM-only approaches face escalating computational demands and eventual context window exhaustion.\n\nFigure B (*Plan Correctness vs. Execution Time in seconds with Gemini 1.5 flash*) shows the trade-off between plan correctness and execution time across the different methods. \nWhile HVR takes longer to compute plans, it consistently achieves the highest correctness. In contrast, simpler methods like LLM or R achieve faster execution but produce significantly less reliable plans, demonstrating the benefit of HVR’s more structured approach.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "No theoretical claims.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "HVR enhances LLM planning capabilities through a novel neuro-symbolic integration of Hierarchical planning, symbolic Verification and reasoning, and RAG methods over symbolic Knowledge Graphs.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n1. The proposed framework is reasonable. It provides a new idea for  LLM planning through a novel neuro-symbolic integration.\n\n2. HVR shows obvious improvements in performance.\n\n3. The result analysis is in-depth, strongly supporting the viewpoints of the paper.\n\nWeaknesses:\n\nThe issue of efficiency has not been discussed, nor has it been compared in the experimental section. Conducting planning and retrieval augmentation at two granularities, as well as failure detection, constitutes a relatively complex process. It is unacceptable that there is no comparison of efficiency.", "other_comments_or_suggestions": "The comparison of efficiency and ablation experiments should be further improved.", "questions_for_authors": "No questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kUPFdFeZTc", "reviewer": "Reviewer_Ac9Q", "summary": "This paper introduces HVR, a neuro-symbolic approach that enhances LLM-based planning by integrating hierarchical planning, retrieval-augmented generation (RAG) over knowledge graphs, and symbolic verification. The proposed method tackles long-horizon and complex task planning by decomposing tasks into macro actions and atomic actions, refining them through symbolic verification to ensure feasibility before execution. The Knowledge Graph-RAG (KG-RAG) component provides structured knowledge retrieval, improving accuracy while reducing hallucinations. The Symbolic Validator plays a dual role: first, it simulates plans in an \"ideal world\" for verification before execution; second, it functions as a failure detector, aligning expected world states with real-time observations. The system also builds a macro-action library, enabling knowledge transfer across agents by storing reusable action sequences. HVR is evaluated in AI2Thor, a kitchen-based robotic simulator, across 12 tasks of varying complexity, demonstrating significant performance improvements over baselines. Results show that RAG is crucial for smaller LLMs, while hierarchical planning is more impactful for larger models. Symbolic verification consistently enhances plan correctness, but LLMs still tend to generate unnecessarily long plans. The study highlights that LLM-based planners perform well on goal-oriented tasks but struggle with open-ended objectives, emphasizing the need for better failure detection and plan optimization techniques.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback, which has greatly improved our paper. Below, we summarize the main concerns and detail the revisions and clarifications made to address them.\n\n**Q1: Missing comparison with the state-of-the-art**\n\nWe did not include direct comparisons with existing methods, as none address complex, long-horizon kitchen tasks compatible with our setup. However, following reviewers’ suggestions, we considered additional works and included additional results/discussion in the Appendix of the revised paper, summarized below:\n\n* *SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models* This work is closely related to ours, as it uses the same simulator and addresses complex tasks (but in a multi-agent setting). We considered only 15 tasks, excluding those not related to the kitchen domain, of which only 12 were feasible in our setup due to a key design difference: unlike SMART-LLM, which assumes full knowledge of all object locations (including hidden ones), our system—aligned with RECOVER —only considers visible objects (hidden objects are treated as failures). HVR with Gemini-2.0-flash successfully planned and executed all 12 tasks.\n* *ProgPrompt: Generating Situated Robot Task Plans using Large Language Models* In this work there are 10 available tasks, 7 of which are kitchen related. We adapted the original implementation with the VirtualHome simulator to the AI2Thor simulator. Using Gemini-2.0-flash, HVR was able to plan and execute correctly all of the tasks. However, it's worth noting that these tasks are relatively simple compared to those in our benchmark. Additionally, ProgPrompt was evaluated using GPT-3, and their performance would likely improve with a more capable LLM.\n* *LLM+P: Empowering Large Language Models with Optimal Planning Proficiency* These methods are built on different simulators and/or involve tasks that are not compatible with our setup (e.g., organizing blocks on a table). As a result, a direct comparison would require a substantial and non-trivial reimplementation, which falls outside the scope of this work.\n* *PDDL-based planning systems* We could not compare with PDDL-based planning systems on our tasks as we found no existing works using planners expressive enough. As described in Section 3.3, our system uses conjunctive, disjunctive, and conditional PDDL statements. We therefore implemented a custom validator in Python, adapted specifically to the AI2Thor environment.\n* *ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning* Here, given specifications of simple abstract tasks in a simulated kitchen environment, the authors generate both a PDDL domain and a goal state. While this can work for a simple environment with few available actions, we argue it is generally not possible to generate PDDL domain specifications from high-level natural language task specifications such as those we use in our work. Thus, we did not run experimental comparisons with this method.\n* *Translating Natural Language to Planning Goals with Large-Language Models* This work uses the ALFRED simulator, similar to AI2Thor. However, the implementation is too limited for our use case and does not support key functionalities like creating individual slices after slicing an object, or modeling interactions such as opening appliances—essential for our tasks— making it unsuitable for a meaningful comparison.\n\n**Q2: Evaluation of Error Detection/Recovery and System Robustness**\n\nWe thank the reviewer for the suggestion. The metrics EPV, MPV, and AABV (highlighted as blue metrics in the ‘Verification’ part of  in Table 2) are specifically designed to evaluate the role of symbolic verification and correction. These metrics are defined as follows:\n(4) Expanded Plan Verification (EPV) indicates the extent to which the expanded plan (full sequence of atomic ac- tions) has been successfully verified. It is calculated as the ratio of verified steps to the total number of steps in the generated plan. (5) Macro Plan Verification (MPV) measures the extent to which the macro plan has been verified. It is calculated as the ratio of verified macro plan steps to the total number of macro steps. (6) Atomic Action Block Verification (AABV) evaluates the extent to which the macro plan has been verified at the level of atomic action blocks. It is determined by dividing the number of verified atomic action blocks by the total number of macro actions. \n\nTo summarize, these metrics reflect how effectively the symbolic validator detects and helps recover from errors at different levels of the plan (expanded, macro, and atomic action blocks). Their strong correlation with Plan Correctness (PC) demonstrates that symbolic validation not only identifies errors but also leads to measurable improvements in plan quality. This provides insight into the system’s robustness and adaptability in recovering from planning errors.", "claims_and_evidence": "I think most of the claims are well supported, for example:\n1. HVR improves task success rates through hierarchical planning, KG-RAG retrieval, and symbolic verification\n- The paper evaluates HVR on AI2Thor, showing higher task success rates than baselines.\n- The ablation study confirms that hierarchical planning, RAG, and symbolic verification each contribute to performance gains.\n- Symbolic verification improves plan feasibility, and RAG reduces hallucinations, validating these components.\n2. Symbolic verification consistently enhances plan correctness\n- The Symbolic Validator verifies plans before execution, preventing physically impossible or illogical actions.\n- Experiments show that removing the validator leads to lower success rates, confirming its importance.\n- **Suggestion**: It would be better if the model compare with alternative approaches for consistency concern, e.g. LLM self-consistency, or self-checking before plan execution.", "methods_and_evaluation_criteria": "Mostly yes, for example the idea about hierarchical planning and symbolic verification are well-suited for long-horizon task and are applicable in real-world planning problems. The selected benchmark and metrics like success rate and execution efficiency are also crucial in real-world applications. \n\nSome suggestions:\n1. The work mainly focuses on comparing with the variation of itself, e.g. HR, HV. But comparing with classical planning methods such as PDDL planners in the experiments is also necessary.\n2. The renumbering task is good to show how the proposed method deals with specific operation order. It'd be better if you can also show and evaluate how the method can recover from errors. The current evaluation focuses on success rates, but some evaluation on how often the symbolic validator can detect errors and adjust the plan would be helpful.", "theoretical_claims": "The paper doesn't have formal mathematical proofs. The claims are mainly supported by evaluations. And I believe they are generally supported by those evidence.", "experimental_designs_or_analyses": "The selected benchmarks and metrics generally make sense to me. There are some suggestions I provide in the *Methods And Evaluation Criteria* section. Specifically:\n1. AI2Thor is a widely used interactive simulation environment that allows for testing LLM-based planning agents in real-world-inspired tasks. It supports long-horizon goal-oriented tasks, which are crucial for evaluating structured planning. Prior work on embodied AI (e.g., ALFRED, BabyAI) has used similar environments, ensuring comparability.\n2. The paper evaluates \n- Task success rate – Measures whether the agent achieves the goal state.\n- Plan efficiency – Measures action length and unnecessary steps.\nThese metrics are crucial to long-horizon planning, as it requires both correctness (success rate) and efficiency (fewer redundant steps).\nPrior AI planning work (e.g., Hierarchical Task Networks (HTNs), PDDL-based planners) has used similar evaluation criteria.\n3. The paper conducts ablation studies to test how much each component (hierarchical planning, KG-RAG, symbolic verification) contributes to performance. This isolates which modules are most crucial for performance improvements and helps clarify whether RAG and symbolic verification contribute independently or synergistically.\n\nSuggestions:\n1. Compare HVR to traditional symbolic planning systems (e.g., PDDL).\n2. Evaluate plan robustness and **error recovery** to measure adaptability.", "relation_to_broader_scientific_literature": "HVR builds upon prior work in hierarchical task planning, retrieval-augmented generation (RAG), symbolic reasoning, and LLM-based embodied AI. It extends classical AI planning methods by incorporating LLM-based reasoning with **symbolic verification** to ensure plan feasibility before execution. Compared to existing RAG-based AI agents like AutoGPT and ReAct, HVR leverages **structured knowledge retrieval (KG-RAG)** instead of unstructured document retrieval, reducing hallucinations in task planning. It also dynamically adjusts symbolic constraints using retrieved knowledge, enhancing planning accuracy in real-world tasks.\n\nIn the context of LLM-based task planning, HVR introduces a hierarchical macro-action library, similar to meta-learning frameworks and the options framework in reinforcement learning, to enable knowledge transfer across tasks. Compared to SayCan (Google DeepMind) and ALFRED (AI2Thor-based task planning models), HVR structures planning hierarchically and incorporates symbolic validation to improve long-horizon reasoning. However, the paper does not explicitly compare HVR to these models, making it unclear whether hierarchical planning, KG-RAG, and symbolic verification offer unique advantages over these prior methods. Including these comparisons would provide a clearer understanding of HVR’s contributions relative to existing AI planning systems.", "essential_references_not_discussed": "I think the discussion on related works is comprehensive.", "other_strengths_and_weaknesses": "For weaknesses just the ones mentioned in previous sections. \nFor strengths, HVR presents a novel integration of hierarchical planning, structured knowledge retrieval (KG-RAG), and symbolic verification to improve long-horizon task execution using LLMs. This neuro-symbolic approach is an important step toward more structured and interpretable AI planning, moving beyond purely end-to-end LLM reasoning. The evaluation in AI2Thor demonstrates real-world applicability for embodied AI and assistive task planning, while the macro-action library enables knowledge transfer across tasks, making HVR significant for autonomous systems.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "No questions other than the ones in previous sections.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper received four high-quality reviews (scores: 3, 3, 3, 1).

The three positive reviewers appreciated the extensive experiments and the innovative methodology, which effectively integrates hierarchical planning, knowledge graph-based retrieval augmentation (KG-RAG), and symbolic verification within a unified neuro-symbolic framework. The reviewers highlighted the paper's strong empirical results, clearly demonstrating improved task success rates and robustness compared to baseline methods.

The negative reviewer raised concerns regarding the necessity of employing large language models (LLMs) in scenarios where symbolic planners are traditionally sufficient due to precise domain models. In the rebuttal, the authors convincingly addressed this concern by clarifying the unique advantages of their approach. The authors emphasized that their use of LLMs enhances planning flexibility, supports partial plan execution, and allows dynamic adaptation during execution, effectively overcoming limitations inherent to purely symbolic methods.

Having thoroughly reviewed the paper, carefully considered all reviewer comments, and evaluated the authors' detailed rebuttal, the AC concurs with the positive reviewers. The innovative integration proposed, along with robust experimental validation, clearly justifies a clear acceptance.
Decision: Accept (poster)