NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domains
Authors: Wonje Choi
Jinwoo Park
Sanghyun Ahn
Daehee Lee
Honguk Woo
Venue: ICLR 2025 Poster (2025)
Keywords: Embodied AI
Neuro-symbolic AI
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
TL;DR: NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domains

Abstract: We explore neuro-symbolic approaches to generalize actionable knowledge, enabling embodied agents to tackle complex tasks more effectively in open-domain environments. A key challenge for embodied agents is the generalization of knowledge across diverse environments and situations, as limited experiences often confine them to their prior knowledge. To address this issue, we introduce a novel framework, NeSyC, a neuro-symbolic continual learner that emulates the hypothetico-deductive model by continually formulating and validating knowledge from limited experiences through the combined use of Large Language Models (LLMs) and symbolic tools. Specifically, we devise a contrastive generality improvement scheme within NeSyC, which iteratively generates hypotheses using LLMs and conducts contrastive validation via symbolic tools. This scheme reinforces the justification for admissible actions while minimizing the inference of inadmissible ones. Additionally, we incorporate a memory-based monitoring scheme that efficiently detects action errors and triggers the knowledge refinement process across domains. Experiments conducted on diverse embodied task benchmarks—including ALFWorld, VirtualHome, Minecraft, RLBench, and a real-world robotic scenario—demonstrate that NeSyC is highly effective in solving complex embodied tasks across a range of open-domain environments.

Published as a conference paper at ICLR 2025
NESYC: A NEURO-SYMBOLLE CONTINUAL LEARNER FOR COMPLEX EMBODIED TASKS IN OPEN DOMAINS
Wonje Jinwoo Park2,Sanghi Ahn', Daehee Lee', Honguk Woo 1.2
*Department of Computer Science and Engineering. Sungkyunkwan University Department of Artificial Intelligence, Sungkyunkwan University {wjchoi1995, pjw971022, shyuni5, dulgi7245, hwoo) @skku.edu
ABSTRACT
We explore neuro-symbolic approaches to generalize actionable knowledge, en- abling embodied agents to tackle complex tasks more effectively in open-domain environments. A key challenge for embodied agents is the generalization of knowledge across diverse environments and situations, as limited experiences of- ten confine them to their prior knowledge. To address this issue, introduce novel framework, NESYC, a neuro-symbolic continual learner that emulates the hypothetico-deductive model by continually formulating and validating knowl- edge from limited experiences through the combined use of Large Language Mod- els (LLMSs) and symbolic tools. Specifically, we devise a contrastive generality improvement scheme within NESYC, which iteratively generates hypotheses us- ing LLMs and conducts contrastive validation via symbolic tools. This scheme reinforces the justification for admissible actions while minimizing the inference of inadmissible ones. Additionally, we incorporate memory-based monitoring scheme that efficiently detects action errors and triggers the knowledge refine- ment process across domains. Experiments conducted on diverse embodied task benchmarks -iniiaaunnsiinuuiingg ALFWorld, VirtualHome, Minecraft, RLBench, and a real-world robotic scenario demonstrate that NESYC is highly effective in solv- ing complex embodied tasks across range of open-domain environments.
INTRODUCTION
Recent advances in neuro-symbolic systems -integrating Large Language Models (LLMs) with symbolic tools (Frederiksen 20088; Gebser et al. 2019)-have gained much attention for embodied task planning (Lin et al.] [0244] Liu et al., 2023). These systems decouple contextual understand- ing-such as observation and instruction translation- frrmm actionable knowledge including action preconditions and effects. Yet, these systems have not been thoroughly explored in open-domains, where the environment is not restricted to pre-defined tasks or knowledge and embodied agents must manage diverse scenarios. Conventional approaches rely on symbolic representations of expert-level actionable knowledge, which limits their applicability and effectiveness in real-world situations. The unpredictable and dynamic nature of open-domains often leads to incompleteness and inconsistency in knowledge, thus complicating the decision- decision-making process of embodied agents.
In neuro-symbolic systems, generalizing prior actionable knowledge in open-domain environments presents practical challenges: (1) inherent lack of fiexibility in symbolic systems apply knowledge to unfamiliar environments, (2) limited methods to bridge the gap between the prior knowledge and new environments, leading to repeated action errors in complex situations, and (3) mislabeling of action affordances or insufficient feedback, caused by the inability to retain labeled experiences, which hinders the agent's ability generalize knowledge and improve maaiii..mmaing.....
To address the challenges posed by adopting neuro-symbolic approaches in open- open-domains, we draw inspiration from the hypothetico-deduetie model (Smokler) 1966). This model emphasizes falsifica- tion through experiences and emulates the scientific inquiry process by continually forming hypothe-
"Equal contribution
Work done while visiting scholar CMU
1 Corresponding author

---

Published as a conference paper at ICLR 2025
Open Domain Env. Interaction
Experfence Sat
Case 1: Conventionll Neuro-symoolic Framework
Open-Gomatn Inderaction (oadeeed)
saudo cablie
Factal
Ots
Action
movi,
Rube su:
Jiowor
Cerree pick place side)
Pall
Actionable Knowledge gre-celledd)
Clear the center the desk.
relaxo
Esacution
Findi iy diject
Actiomable Knowedge
Case 2: Neuro-symbolk Continual Leamer (ours)
Open- Domann Ena. Interaction (cadlendi
jarsto-dbted smmalllteed
Fot, babl Deweesy
Generatty Improvement:
abor spaui so Dope lock
Oes Mandly
move, bo Dransr Comas pick, place topt
Actionatle Knozdedge Ccntinually geoenibad
bukking and (int, por porediae
move, tark Dranzt
more, to aas Dus Moady
Eoocutco
hain, milliioi pladd
Reasoning Modee
mone, hilllig
Nondy-
Exacudion
Fettal
Symbolic
Figure I: The concept of NESYC. In the leftmost part, domain shift leads the agent to fail by trying to grasp an oversized drawer's broad surface, which is infeasible. The remaining parts contrast two approaches: Case I treats LLMs and symbolic tools as separate functions for semantic parsing and logical reasoning, while Case 2 integrates them into a collaborative process, cnabling NESYC to generalize actionable knowledge and compute logically valid actions for open-domain environment.
ses, rigorously testing them against available observations, and iteratively revising them. Guided by this model, we explore knowledge generalization strategies that interleave inductive and deductive reasoning, aiming to generalize knowledge applicable in open doen-dom and enable embodied agents to adapt more effectively to unpredictable situations. We introduce novel framework, NESYC, neuro-symbolic continual learner that combines the strengths of LLMSs and symbolic tools to ef- fectively formulate and apply knowledge open-domains. In Figure I unlike conventional neuro- symbolic approaches relying on pre-defined actionable knowledge, NESYC uses accumulated ex- periences to generalize knowledge. This enables NESYC to execute actions effectively in the given environment while continually generalizing actionable knowledge to adapt across open- nee-ddoaiins.
Specifically, we devise two key components in NESYC. First, we employ a contrastive generality improvement scheme that iteratively generates hypotheses using LLMs and conducts contrastive validation through symbolic tools. This scheme reinforces the validity of admissible actions while minimizing the inference of inadmissible ones, combining the generalization capabilities of LLMs with the logical rigor of symbolic tools to generalize actionable knowledge. Second, we implement a memory- monitoring scheme that efficiently detects action errors and triggers the knowledge refinement process, continually expanding the agent's 's coverage of actionable knowledge.
To evaluate NESYC, we conduct experiments ALFWorld (Shridhar et al. (020c), Virtual- Home (Puig et al.] 2018). Minecraft in Silver & Chitnis (2020). RLBench (James et al.) 2020). and real-world robotic scenario, demonstrating its applicability in open domains. Compared the advanced baseline AutoGen (Wu et al.) 2023a). NESYC achieved task success rate improvements of 33.6% on ALFWorld, 43.9% on VirtualHome, 53.7% on Minecraft, and 52.6% on RLBench.
The contributions of our work are as follows: (1) We present the neuro-symbolic continual learner NESYC based on the hypothetico-deduutivv model to enable generalization of actionable knowl- edge in open-domain environments. (2) We devise two schemes tailored for actionable knowledge generalization in NESYC: contrastive generality improvement and memory-based monitoring. (3) We validate NESYC through experiments on diverse benchmarks and real-world scenarios, demon- strating its effectiveness and significant performance improvements in open-domaa environments.
2 BACKGROUND AND PROBLEM FORMULATION
2.1 INDUCTIVE LOGIC PROGRAMMING (ILP)
ILP Muggleton & De Raedt 1994) is a machine learning technique where the learned model is represented logic program, or hypothesis (i.e., a of rules), derived from combination of
2

---

Published as a conference paper at ICLR 2025
examples and background knowledge. A common setting in ILP is Learning from Interpretations (LFD, where each example is an interpretation represented as a set of facts (Cropper & Dumančid, Given program BK denoting the background knowledge, along with sets of positive exam- and negative examples E-, the goal is to find an optimal hypothesis H satisfying:
2022
ples E
Ve Et, is an interpretation of H U BK. {
(I)
Ve E e is not an interpretation of H U BK.
Here, BK functions similarly to features in traditional machine learning, but it more ex- pressive, as it can include relations and information associated with examples. During LFI, 0- subsumption (Sakama 200T) key determining whether hypothesis subsumes examples, check- ing if the hypothesis can be interpreted as the examples through variable substitution. Further details can be found in Cropper & Dumančić (2022). which offers an in-depth overview of ILP.
2.2 ANSWER SET PROGRAMMING (ASP)
ASP (Lifschitz 2019) is a declarative programming paradigm well-suited for solving complex com- binatorial problems like planning, particularly in non-monotonic domains where the dynamics and actions of embodied environments
alter future states. An ASP solver (Gebser et al.) (019) com- putes one or more answer sets, representing valid solutions by encoding problems as logic programs composed of rules in the following form:
: BI...., Bm, not Bm+l,.... not B,,
(2)
In this general form, a rule consists of a head (A) and a body (B) not B,), where each A and B, (1 S S n)) is an atom. The head represents the conclusion, and the body specifies the conditions, which include both positive conditions (Bi...., Bm). that must hold true, and negated conditions (not Bm+ not B.), where not denotes negation as failure (NAF). NAF assumes the negated conditions to be false unless evidence to the contrary is provided. A rule with empty body is a fact (e.g., A), while a rule with an empty head is a constraint, representing conditions that must not be satisfied. ASP is well-suited for evaluating the coverage of hypotheses in IIP by identifying which examples are satisfied (Law et al. 2020), and it also proves effective for planning in complex, dynamic environments (Cabalar al., 2019). These capabilities are essential our framework, enabling knowledge generalization through the interplay of induction and deduction.
2.3 PROBLEM FORMULATION
The open-domain embodied task planning problem is formulated as tuple (D, S, Here, D represents the domain space for open-domain environments, while S denotes the state space. Due to partial observability (Sutton & Barto 2018), the agent perceives observations or E S2 at each timestep, which provide partial information about state S. A is the action space. The function F maps a domaín d E D to its specific goal states and dynamics F(d) = {9a, Ta} (Hallak et al.) 2015). For given domain d, Ga S represents the goal states derived from the instruction set Ia. while Ta : S x A + S models how actions affect state transitions within the domain, defining the environment dynamics. In open-domain environments, the agent may not have full knowledge of Ta, making it essential to adapt to the environment. The objective of NESYC is formulated as:
n arg max E d~D
Σ SR (st, T(. ( ia))
oz,
where is ~ Z. corresponds to Ia E Ga, and SR S x A } indicates whether the agent successfully completes the task given current states. Policy π selects action a, based on observation of and instruction id (Yoo et al. 2024 Brohan et al.) 2203
Huang et al. 2024
3 NESYC: A SATRRO-SYMOLICC CONTINUAL LEARNER
3.1 OVERALL FRAMEWORK
We propose NESYC, neuro-symbolic continual learner designed to generalize actionable knowl- edge for embodied agents in open-domain environments. To effectively utilize the limited experi- ences of agents, this framework integrates the capabilities of L.LMs and symbolic tools. It maximizes
3

---

Published as a conference paper at ICLR 2025
Rule Reformulation
Contras tive Generality improvement
Start
(o,, 1) Success
Parser
Hypothests Generator
Saburpia
(e., c 0) Fall
Negatie
4 hyyp
Experience set?
Generate 8K
G Generalized Knowledge R
Formulate
Update the
( (n.lyu))
Frecondioon of Action
(H.
Validate
eetenis cE 1P seld my
Hypothesis Interpreter
actimp up/0, L) (abrill)) not opent, T;-
18 Vinterp
(ii) Rule Application
Admissibe
Action Plans
A.P
Inference @
aanningg
Semantic Parser
Instructinn
User
Observations st
Inadmissible
Memory-based monitoring
Action Executor
Cit Revs
Enor Handler
o
Continue Phase (U)
Working memory
Enor Handler OP
Pase
D EnI.
er7
Phose
Affordance
G
Re de-entering Phase
Figure 2: The structure of NESYC. NESYC iterates (i) Rule Reformulation and (ii) Rule Application phases. In (i), generalized knowledge R is reformulated via contrastive generality improvement. In (i), R is applied and continually adapted to the environment via memory-basee monitoring.
the consistencies of experiences with admissible actions via common-sense reasoning and minimizes contradictions from experiences with inadmissible actions via symbolic reasoning. To achieve this, our framework operates in two phases: (i) Rule Reformulation and (ii) Rule Application, as illus- trated in Figure 2 Each phase is enabled by contrastive generality improvement and memory- monitoring schemes, respectively, both of which are implemented via the interleaved collaboration of L.LMS and symbolic tools. the Rule Reformulation phase, NESYC employs contrastive gen- erality improvement scheme based ILP to formulate generalized knowledge from accumulated experiences. An LLM leverages common-sense reasoming to generate hypotheses, while a symbolic tool ensures their logical consistency through systematic validation. The iterative and reflective na- ture of this scheme allows the resulting knowledge to achieve both accuracy and broad applicabil- ity across domains. In the Rule Application phase, NESYC employs memory -based monitoring scheme based on ASP for embodied task planning, where experiences are collected and categorized based on admissible and inadmissible actions. The LLM offers contextual understanding of obser- vations, while the symbolic tool computes precise actions using the current actionable knowledge. If an action fallure detected during task execution based on observations, NESYC then re-enters the phase (i), triggering the knowledge refinement to better adapt to the environment.
3.2 RULE REFORMULATION
As shown in Figure NESYC reformulates generalized knowledge R, which represents causal rules for action preconditions and effects derived from the experience set T. To achieve this, NESYC employs the contrastive generality improvement scheme based on ILP. The experiences in T are translated into example set E, comprising positive examples E+ and negative examples E- based on action affordances. The generality improvement process is both iterative and reflective, driven by the hypothesis generator 9hyp and the hypothesis interpreter V inteep. which collaboratively refine the hypotheses. Through this process, the interpretability of the hypotheses H is progressively enhanced, reinforcing the logical justification and reasoning with respect E+ and E until R is obtained. Algorithm lists the rule reformulation phase of NESYC
Semantic parser. To extract ground rules (i.e., rules without variables) from the experience set T. the semantic parser 2.ems utilizes in-context learning with an LLM, following neuro-symbolic approaches (Olausson et al.) 2023 Pan et al., 2023). Focusing on action preconditions and effects, we prompt the LLM to translate trajectory a = (01, a1..... 0z) into ground rules that represent transition. The parser &sem and the example set are formulated as:
E = (0)) 4-11) (or, Q-11) E T where P + (1-1
(4)
Examples are partitioned into positive set E+ and negative F- based action affordance G-
4

---

Published as a conference paper at ICLR 2025
Hypothesis generator. To induce hypotheses H. that satisfy both positive and negative examples, we guide the LLM to extract background knowledge BK, which enhances context and facilitates the alignment of hypotheses with the examples. We then use BK to generate H. by employing structured prompt that incorporates the 0-subsumption technique from IIP, combined with a batch sampling strategy. The f-subsumption technique allows to determine if one clause is more general than another by finding a substitution 0 that makes one clause imply the other. To simplify this pro- cess, we leverage the LLMs' multi-step reasoning capabilities via a subsumption Mhain-of-Thought (CoT) prompt, denoted as lub. The hypothesis generator Pnyp is then defined as:
Payp (B, Hi-11 Laab: (%001) (Hg, BK) where B P E.
(5)
Here, B is batch of k randomized examples, and Hi is the hypotheses at batch iteration b. Feedback " from the previous interpretation step i-1, provided by the hypothesis interpreter Vinterp. guides the update of Ht" The labb explicitly derives B K, which serves as intermediate rationales chaining the to H. The generated BK is then reused by inter to validate Hi as input for symbolic tool.
Hypothesis interpreter. To validate the hypotheses H', we employ symbolic tool (i.c., ASP solver) to assess the interpretabilityy of each hypothesis H for including the positive examples E and excluding the negative examples E We define the hypothesis interpreter V Vinterp (E, H', BK) H>
(H, 4mb), where H is the optimized hypothesis. and Lab is a feedback for the hypothesis generator Payp The H and lab are determined by: "satisfy"
if i itermax feedback with HI(H) otherwise
4db
(6)
where Ā = arg max HEW HH( and the scoring function HI is defined as:
HI(H; E, BK) = fIPR(H, E+,BK)= E BBK.
(7)
If an interpretation step reaches its maximum, Á is accepted as generalized knowledge R. The generalizability of hypothesis across the entire E' is assessed using HI, following an approach similar to the contrastive learning objective (Oord et al. 2018). We define fVPR and fTRR as metric functions that evaluate the False Positive Rate (FPR) and True Positive Rate (TPR) for a given H, with respect to the E' and BK. In the formulation of fFPR and STPR, we prioritize examples from the current environment by assigning them higher weights than existing examples, thereby ensuring that knowledge improvement is aligned to the current environment. Details are provided in Appendix D
3.3
RULE APPLICATION
As shown in Figure 2 generalized knowledge R is used complete embodied tasks specified by the user instruction i. Specifically, NESYC employs a symbolic tool and a memory- based monitoring scheme, utilizing ASP for action planning. During task execution, the error handler Per manages in- teraction experiences from the environment via the action executor &exe. storing them in the working memory M. If an inadmissible action is detected, &ur triggers the refinement of R by re-entering the phase (i), where M is integrated into the experience set T. With the refined R, NESYC effectively adapts to unpredictable situations. Algorithm 2 lists the rule application phase.
Task planner. In computing action plans, a symbolic tool that takes (R, St, g) as input programs is used, following TTan et al. (2023); /Aeronautiuues al. 1998), where R is the generalized knowl- edge for action preconditions and effects, $; is a current state, and g is goal state. Since the current and goal state often not clearly specified the environment, the semantic parser &sem in Eq.44) translates the trajectory a; into programmatic form of 8t, and the instruction i into y. Based on these inputs, the task planner V plann computes action plans P to transition from St to g. utilizing R. Formally, the L plan is defined as Vl plan (R, st.9) + P.
Action executor. From action plans P deduced by task planner pplan an individual plan can be chosen by action executor Lee to perform relevant actions in the environment, starting from the current step t; i.e., Pexe : (P, Sts g) + a. Note that Texe performs action ae sending observation 0t4-1 from the environment along with the result of the action taken, to the error handler her-
Error handler. To maintain consistency between the predicted state changes and actual observa- tions, the error handler & monitors task execution using the memory- -aased retention of trajectory
5

---

Published as a conference paper at ICLR 2025
samples. Due to the dynamic nature of embodied environments, planning based V plan often falls short of task completion. Based on the execution results, Per measures action affordance o and rewrites the next observation 0++11 to provide more attentive representation of the environment. We define Per to trigger the refinement of generalized knowledge R based on action affordance ct.
Phase (ii), if = Phase (i), if =
Next phase-
where Per (0t, de, 0 H Ct. Or+1
(8)
When Q: 1, even though the action is successfully executed, the changes in the observations might invalidate the preconditions for the next action. To resolve this, 인er updates the current observation via Psem. and V plan re-plans accordingly. When 0, the action fails, necessitating the refinement of generalized knowledge R. Den appends all pairs of (0++1. c₂) to working memory M,, including those for C₁ = O. This robustly refines R by re-entering the phase (i) with the updated experience set TUTUM, continually improving the understanding on the environment.
Algorithm Rule Reformulation
Algorithm 2 Rule Application
Agent: Psm-- Pyyp- iiterrp
Agent: xsem: v plan Pexe. Per
Experience T
Experience set 'T, Generalized knowledge R Working memory M trajectory o +
Example set +-
Hypotheses H
0
Optimized hypothesis //
0
1: L 4- 0, (or, env.reset)
Generalized knowledge R t- (1 Subsumption CoT prompt Lub Feedback prompt 4im-
2: a + a.pppend(or)
3: for 1 S itermax do
4:
(st 19-11,4-1,, t d sem (o)
1: for all O.C do
5: g 1 (mmm())
2: + EU & (o)
6:
P + V plan R. St,
3: end for
de (P)
4: while = do do
8:
0+41 + enw enw.step(a;)
5: for all batch B E E do
9:
a. 0++1 + Den (o, ar, 0t+1)
6:
H. BKK +- Phyp (B, H. Lsub labb)
10:
a + w.woncat((aa................. 0++1))
7:
end for
11:
M < Maappend(
8:
H. Lab +- Vinterp E, BK)
12:
if c = 0 then
9:
if Tra is not "satisfy" then
13:
T TUM, H R
10: H + H- +- 0
14:
R Re-entering phase (i)
11: end if
15:
M + 0, < [0r+1]
12:
end while
16:
end if
13:
return R A
17: end for
4 EVALUATION
4.1 EXPERIMENT SETTING
Environments. For evaluation, utilize several embodied benchmarks such ALFWorld, Virtu- alHome, Minecraft, and RLBench. Additionally, we conduct experiments with real-world robot to demonstrate NESYC's effectiveness and applicability in real-world complex tasks. For open-domain evaluation, we use three environment settings, categorized by their level of dynamics, which result in significant state changes.
Statie setting, object states, goal conditions and action effects are consistent across episodes. In a Low Dynamic setting, object states change unpredictably within an episode, though goal conditions and action preconditions remain consistent. In High Dynamic set- ting, both object states, goal conditions, and even the preconditions of actions change unpredictably within an episode. For each task, we generate rephrased instructions using ChatGPT (Ouyang et al. 2022) based the templated instructions from each benchmark, similar to Szot et al. (2023).
Dataset. We utilize a few expert-level episodic experience data from environments with conditions identical to the Static setting. Each experience captures state transitions, including an initial observa- tion, action taken, execution result, and resulting next observation. Note that this experience dataset represents about 7% of the evaluation task episodes used in experiments.
Evaluation metrics. We use several evaluation metrics, consistent with prior works (Shridhar et al, 2020b c). SR (%) measures the percentage of tasks successfully completed, defined meeting all

---

Published as a conference paper at ICLR 2025
Table I: Performance comparison of open-domain embodied task planning for static and dy- namic environment configurations. Variations for each metric are reported with three seeds.
ALFWorld
Dynamic
High Dynumic
Methods
SR
GC
Step
SR
GC
Step
SR
GC
Step
LIM-planner
10.0+22.8
17.7+3.4
20.9+3.7
9.8+2.7
22.2+3.8
26.8-4.0
7.3-2.4
17.1+3.4
21.1=3.7
ReAct
35 84333
18. 18.2+-111
51.7+13
31.1+1.3
15.2==1.55
50.6=155
18.7+3.5
28. 28.1=1111
33.3=1.3
Reflexion
39.0+3377
63.5+44.4
67.0=44.5
37.4+4.4
64.8+4.33
70.6-4.1
21.1+44.4
41.5+4.3
43.6=4.2
AutoGen
58.5114.4
71.8-13.7
81.1:3.5
51.244.55
69.343.99
75.6 4.2
30.944.22
5..6:4.5
58.8-4.4 4.4
CLMASP
88.5+2.99
89.142.8
89.142.8
68.8444.4
63.3+4.44
71.8-4.1
23.8+33.9
36.5+444
45.8-4.5
NESYC
82.9433.4
83.5+33.
83.6+3.3
3.7
79.413.77
80.0-13.6
70.714.1
7..5
76.4
VirtualHome
Static
Low Dynamic
High Dynomic
Methods
SR
GC
Step
SR
GC
Step
SR
GC
Step
M..M-llnner
21.5+00.
31.240.4
33.0-9.7
20.5++0.6
32.7-0088
29.8-2.7
14.843..4
27.7+2.99
21.5-2.2
ReAct
40.0+55.0
61.9+4.5
448=0088
34.6+3.6
46.8+3.4
35.9=3.7
17.2+2.1
32.4+1.2
18.8=2.0
Reflexion
36.24-1.7
47.5+2.0
16.4+1.1
35.4++.9
36.5-1.8
15.5++0.9
20.7+1.6
16.4-1.1
AutoGen
44.312222
54.8.1.77
45.8.12.2
43.211.2
54.41144
44.9 1.2
18.9.10.9
33.0.1.7
20.61.0
CLMMSP
76.44-0.8
84.1-0.0
28.9++0.4
28.9-0.4
0.0-0.0
NESYC
82.310.4
87.4
84.2.10.6
79.611.9
85.811.1
80.8.199
77.5.1.3
84.1
79.011.2
Minecraft
Static
Las Dynamic
High Dynamic
Methods
SR
GC
Step
SR
GC
Step
SR
GC
Step
LLM plaaner
41.2.1.4
42.5.12.4
31.9.1.6
37.0
23.3.12.7
25.811.3
28.9..044
ReAci
31.41:6
38 38.31114
44.422.7
27 8.1
31661:77
00.3=110
21 1.:16
24.722.8
30.623366
47.211.3
49.011.6
30.0.12.7
21.1.1..6
24.0.1244
30.9-..0
AutoGin
51.14:4.2
52 23344
54.92288
33.7
36.6:22
38.2-25
25.63:3.1
28 28.6
32.7=122
CLMASP
91413.1
95.1+177
95.8+133
52
65.4
59.1=5.5
9:1311
50.9
52.8=2.8
NESYC
92.2116
91.320.9
95.321.0
91.111.6
93.211.4
94.11114
87.815.7
89.915.9
90.915.9
RLbench
Static
Low Dynamic
High Dynamic
Methods
SR
GC
Step
SR
GC
Step
SR
GC
Stop
LLM-plamner
16.7.15.7
21.3129
35.52119
16.74:29
21.8:1144
27.4=1.0
18.3:129
21.711.4
27.2=1.9
ReAct
23.3::2.9
25.8
38.321.7
21.742..8
23.3:114
30.8-1.6
18.99
20.02255
26.8=2.5
Reflexion
33.3++2.8
41.4+5.1
47.5=3.3
21.7+29
24442211
32.1-2.
23.3+22.8
23.3+228
29.6=2.8
AutoGen
13.328.6
51221:6
57.923.3
23 23.9929
28 28.622777
32.1=23
21.72
23.3=2.9
28.9=1.9
CLMASP
94.5+:42
95.8++2.8
96.0++2.7
0.0=0.0
6.0=0.88
25.-220
0.0=0.0
3.7=0.7
12.3=0.9
NESYC
85.5-42.77
88.5+00.9
91.9-0.7
81.5
84.8-14.5
88.743.6
79.0+6.6
81.8+7.0
86.2-66.
goal conditions. GC (%) measures the success rate of individual goal conditions. Step (%) measures the percentage of the action sequences that align with the ground-truth sequence from the start.
Baselines. We implement several baselines, categorized into three groups: i) an LLM-based plan- ning method LLM-planner (Song et al.) 2023). ii) Multi-agent frameworks including ReAct (Yao)
al. 2023), Reflexion (Shinn al. 2024). ) and AutoGen (Wu et al. 2023a). iii) neuro-symbolic
approaches such as ProgPrompt (Singh et al. 2023) and CLMASP (in et al. 2024).
NESYC implementation. For the symbolie tool, (version 5.7.1). The LLM mainly used is GPT-4o (version gpt-4o-2024-08- with temperature 0. For fair comparisons, the same LLM configuration applied across all baselines.
the ASP solver clingo (Lifsehitz 2019)
Detailed explanations of the experiment settings are in Appendix B
4.2 MAIN RESULTS
In Table D we evaluate the performance of embodied task planning in open-domain settings, com- paring each method's action plans based on their use of experiences, primarily through in-context learning. NESYC consistently outperforms the most competitive baseline, AutoGen, across all test settings (Static, Low Dynamic, and High Dynamic) and on evaluation metrics (SR, GC, and Step). Specifically, NESYC achieves an average improvement of 45.2% in SR, 38.7% in GC, and 38.4% in Step across the evaluated benchmarks.
In the Static setting, the conventional neuro-symbolic approach, CLMASP, outperforms NESYC due to its use of additional expert-level knowledge tailored to the given environment. However, NESYC achieves comparable performance by generalizing knowledge solely from the provided experience data. As the dynamicity of the environment increases, CLMASP, which lacks the abil- ity to reformulate its knowledge, exhibits noticeable decline in performance, even falling behind LLM-based approaches. contrast, NESYC remains robust across range of poen-domain settings, including both Low Dynamic and High Dynamic environments. By continually refining generalized

---

Published as a conference paper at ICLR 2025
knowledge from accumulated experiences, NESYC maintains consistent performance across the benchmarks that involve varying action types and environmental dynamics. In RLBench, specifi- cally, the focus is fine-grained physical control and interaction, constrained by factors such as actuator range, grip force, and balance. These precise physical constraints, which are critical for task success, pose significant challenges for L.MM-basd approaches and can lead to substantial performance drops even with subtle environmental changes for neuro-symbolic agents. In contrast, NESYC controls a robotic arm with precision and stability by continually refining its knowledge, resulting in robust performance across all open- open-domain settings.
4.3 ANALYSIS
Table 2: Performance evaluation on robustness to experience incompleteness. 'Logic Exp.' denotes the logic expression (i.e., Natural Language, Imperative Programming, or Declarative Program- ming). 'Refine" indicates if the logic is refined, with x for refinement and v for refinement.
ALF World
Complete Experience Set
Noisy Experieace Set
Imperfect Experience Set
Logic Exp.
Method
SR
GC
Refine
GC
Refine
SR
Refine
NL
Autlggnn
54.618.7 8.7
73.7
x
54.6.18.7
63.418.4
X
57.6.18.6
76.017.4
x
Imperative
ProgPrompt
72.7-7.8
98.5+2.1
x
48.5+8.77
61.1+8.5
48.5488.2
67.4+8,2
Declarative
CLMASP
NESYC
97.0+3.0
90.9-5.0
96.0+-3.4
98.0+2.55
x
90.9+5.0
69.7+8.0
91.9-4.7
78.8+7.1
X
84.9+6.2
54.648.7
89.945.33
68.2=8.1
x
Robustness to experience incompleteness. In Table 2 evaluate the robustness of NESYC upon varied experience quality. The experience sets are categorized by their incompleteness: Complete in- cludes sufficient actionable knowledge, Noisy contains mislabeled action affordances, and Imperfect omits some actionable knowledge. Although the baselines use expert knowledge in different forms, they similarly assume that provided knowledge is either noisy or imperfect. In the Complete case, with full expert knowledge, ProgPrompt and CLMASP perform best in GC and SR, respectively. However, NESYC not only achieves task planning performance comparable to CLMASP with com- plete experiences but also demonstrates robust performance with incomplete experiences through its knowledge generalization strategy.
LLM
SR
SR
GC
GC
H1
HI
Llama-3-8B
43.9
++ 40.2
43.9
- 40.2
0.344
++ 0.378
GPT-o-mini
41.9
> 78.7
44.7
79.3
0.634
0.697
Claude-3 -l-pue-3.--Opus
50.7
- 76.7
53.6
- 78.6
0.516
+ 0.567
Claude
51.4
78.4
54.2
+ 80.2
0.559
0.615
Llama-3-70B
58,8
85.1
60.4
86.6
0.709
0.762
Specifiitt
Precision
GPT-40
64.2
- 90.2
67.0
+ 90.5
0.752
+ 0.786
GPT4
69.6 + 89.2
73.3
+ 89.8
0.784
+ 0.804
Table 3: Contrastive generality improvement scheme evaluation on environments. SR, GC, and HI measure the performance of the generalized knowledge R the first interpretation step. SR, + GC and + HI report scores after iterative adjustment.
Accuracy
HI Score
Figure 3: HI score evaluation.
Impact by different LLMs. We examine the dependency of the contrastive generality improve- ment scheme on various LLMs in the phase (i). Table 3 specifies the impact of different LLMs on the planning performance, measured in SR, GC, and HI between each initial hypothesis (i.e., SR, GC, HI in the table) and its corresponding updated one (i.e., SR. GC and + HI) after the improvement process. As shown, updated hypotheses consistently achieve performance gains in planning for all tested LLMs with the exception of smaller Llama-3-8B These results indicate the robustness of our contrastive generality improvement across capable LLMs. However, smaller mod- els like Llama-3-8B reveal certain limitations in the scheme's 's effectiveness. In Figure 3 we further illustrate the consistency of these results, emphasizing the alignment between the HI score of the updated hypotheses and other traditional metrics.
Comparison on different dynamics predicates. Figure 4 shows differences in performance with respect to different dynamics predicates in Table 4 The performance based on the comparison between expert-level actionable knowledge, initial hypotheses, and updated hypotheses. In Figure the blue arrows indicate the improved performance of updated hypotheses from their respective ini- tial ones. As shown, the Attribute dynamics predicates, which pertain to relatively static features,
8

---

Published as a conference paper at ICLR 2025
100
Category
Num.
Predicates
90
grabbable, cuttable, can open, readable, puper, movable, pourable,
Autriutee
20 has switch, lus..pug. drinkable, body poltypprtt, surfaces, sittable, lieahle, person, hangable, clothes, eatable
60
closed. open. plugged out, plugged in. off. sitting. lying. clean, dirty
Status
50
obj.ontop. ontop, inside_room, obj iinide, inside, on.char, obj_next, to, next_to, between
40
Aitribute
Status
Spalahit
Spatiality
Figure 4: Fl score evaluation on predicate car- egories. Colors indicating LLMs are consistent Table 4: Dynamics Predicates. The predicates are with those used for different LLMs in Table
categorized based on their dynamics.
consistently achieve high F1 scores; conversely, Status and Spatiality dynamics predicates show variability due to their dynamic natures on physical states and spatial relations. While the improve- ment varies across different predicates and LLMs, predicates related to common-sense knowledge tend to show better improvement overall.
Table 5: Ablation on two reasoning components of NESYC. 'w/o. Luab" refers to replacing Sub with simple LLM prompt, and 'w/o. ASP sol.' indicates using LLM prompting for planning instead of the ASP solver. The symbol ' denotes replacement with another LLM prompting technique.
ALFWorld
Static
Low Dynamic
High Dynamic
Method
SR
GC
Step
SR
GC
Step
SR
GC
Step
NESYC
82.9133.4
83.5233.
83.6133.3
78.9
79.41:3.7
80.02:16
75.513.9
76.41:8
w/o. Saz
41.5-+-.4
45.7+44.5
51.7+44.5
39.0++4.4
42.1+44.5
48.4+4.5
36.6++4.3
43.3+4.5
49.3++4.5
ASP sol. + SymbCoT
71.514.1
89.812.7
92.311.4
60.23:4.4
80.013.6
85.313.2
27.6.14.0
48.01:455
58.914.4
ASP sol.
$7.6+3.
80.+228
58.5+44.4
77:2+3.8
81.5+3.5
24.4+3..
49.9+4.5
w/o, ASP sol.
48.82:4.5
77.7=3.8
81.723.5
3..82113
59.12-1.1
70.0::111
17.123. 1
115:244
51.41:15
ASP
25.2+3.9
67.9++4.5
06.3++4.3
9.8+2277
30.6+-4.2
41.2444.4
3.3+1:.6
19.8-336
27.3++4.0
Ablation study. In Table 5 we conduct an ablation study assessing the impact of reasoning com- ponents in NESYC, specifically the subsumption CoT prompt Sub and the ASP solver used for planning. The w/o. Lub. NESYC experiences an average 38.5% decrease in SR, underscoring the importance of structured prompting for generating background knowledge and hypotheses. The next three rows present the results when we replace the ASP solver in the task planner Vplan with different LLM-based reasoning methods: 'A.SP sol. - SymbCoT" employs LLM as symbolic reasoning tool, following the symbolic CoT (Xu et al. 2024), 'ASP sol. - CoT' refers to the use of standard CoT (Wei al. 2022), and 'w/o. ASP sol.' represents a naive prompting without ASP solver. Using an LLM in place of the ASP solver can simplify planning, often improving GC and Step performance for shorter sequences by reducing the strictness required by symbolic tools. However, they exhibit lower reliability in task completion, as indicated by reduced SR. Furthermore, replacing both Luh and the ASP solver with simple L.LM prompting, denoted as 'w/o. Lub & ASP sol.', , leads to significant degradation in task performance.
User Organize your desk and complete
Subticc 2: Organire desk
Enror Feadback
Subtik 2 Complete the Tower of Hanol
Reformulation
Intai Stop
Fiop
ASP program
S Success
program
Fail
for cyddrdreeut cesects that diab ahape!
son zyy sedetral caseere
essannipick, placeryindraal wtgee Grbal T1-5) not dttpptap00 Orip power.
cccusspie, ploce aijertt, Geipl, A. Gri praee
eocunipick placetcyivatiiat sngeett shappt prechion.
Arswer sel pick placelsylotticdl sijectt,, powert harsi palettl....
Figure 5: Real-world desk rearrangement tasks. Initially, NESYC does not include knowledge for picking up Hanoi blocks from experiences. After failures, NESYC refines to enhance grasping ca- pabilities, enabling the robot successfully complete the desk rearrangement task.

---

Published as a conference paper at ICLR 2025
Real-word scenario. In Figure 5 we illustrate the real-world experimental setup for demonstrating the practical applicability of NESYC. The task involves rearranging a desk, with scattered blocks of Hanoi Tower and other objects. Using the same experience set of the RLBench experiments, NESYC restructures actionable knowledge for the real-world rohot and refines the knowledge to handle unfamiliar objects, such the Hanoi blocks. The robot successfully completes the instruc- tion, highlighting the practical applicability of NESYC.
Table 6: Comparison of LLM feedback and Human feedback. In Binary case, the LLM solely cal- culates action affordances but does not rewrite the next observation. In Cause case, the Human and LLM,, respectively, calculate action affordances and rewrite the next observation. In Guidance case, the Human and LLM additionally incorporate corrected experiences.
Real-world
Static
Dynomic
Real-world
Static
Dynanic
Real-world
Statie
Dynamic
Binory
SR
GC
SR
GC
Cause
SR
GC
SR
GC
Guidance
SR
GC
SR
GC
LLM
22.2
59.2
111
42.4
LLM
Human
66.6
77.8
87.0
92.6
44.4
55.6
79.7
85.2
LLM
Human
88.9
55.6
98.1
81.4
66.7
33.3
94.3
72.1
In Table 6 we compare different types of feedback integrated into experiences via memory- based monitoring. In both the Binary and Cause cases, we observe that the error handler her effectively refines actionable knowledge, comparable to high-quality human feedback, by directly measuring action affordances and rewriting the next observations based the action taken. However, in the Guidance case, while accurate guidance from humans is effective, LLM guidance often adds errors in experience representations, resulting in performance degradation.
5 RELATED WORK
LLM--ased task planning approaches have opened new avenues for leveraging linguistic knowledge
to guide agent behaviors in embodied environments (Brohan et al., 2023;
Huang et al. 2023b Song
jct al.
2023 Driess et al., 2023: Zhao et al. 2024] Singh et al. 2023 Wu et al.
2023b Wang
al. 2023). Mcanwhile, nwwro- symbolic systems combine the capabilities of neural networks with
symbolic reasoning tools to enhance explainability, reliability, and flexibility (Olausson et al. 2023
Pan et al. 2023 Fang et al. 2024 Yang al. 2023 Ishay et al. (023). These systems often rely on
fully defined symbolic knowledge for embodied control (Lin et al.) 2024| Liu et al. 2023 Agarwal
al. 2024 Cornelio & Diab 2024), which constrains their applicability and effectiveness in open
domains. Recent advancements in LLM-based multi-agent frameworks have enhanced problem-
solving capabilities by fostering the collaborative interaction between agents. external tools, and
environments (Yao et al., 22044] Hao et al.) [023) Shinn et al., 2024 Yao et al.) 2023 Wu et al.)
2023ab. Further details on related work are Appendix
CONCLUSION
We presented the NESYC framework to enable effective embodied task planning in open domains by continually generalizing actionable knowledge from experiences. The framework adapts neuro- symbolic approaches via two schemes, contrastive generality improvement, and memory-based monitoring, which enable the interleaving of inductive and deductive knowledge refinement a continual learning manner. Experiments on ALFWorld, VirtualHome, Minecraft, RLBench, and real-world robotic scenarios demonstrate the robustness and applicability of NESYC across diverse open domains. In static settings, although using pre-defined expert knowledge involves trade-offs, our model still shows clear advantages over other LLM- and neuro-symbolic approaches.
Limitation and future work. As reported in Figure and Table 3 NESYC encounters difficulties when applied to smaller LLMs, such as Llama-3-88 performance improvements are rarely achieved due persistent rule conflicts and errors during the rule reformulation phase. We plan explore neuro-symbolic knowledge distillation for resource-efficient embodied control with smaller LLMss
Ethical concerns. LLMs operating in environments with hazardous tools (e.g.. knives and forks) can lead to unsafe outcomes if errors occur. Therefore, strict and transparent safety guidelines must be implemented to verify all outputs, and we committed providing them.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT), (RS-2022-1122044 (2022- 0-00043), Adaptive Personality for Intelligent Agents, RS- RS-2022-11221044 (2022-0-00045), Self- directed multi-modal Intelligence for solving unknown, open domain problems, RES-2025-02218 Accelerated Insight Reasoning via Continual Learning, and RS-2019-1119042) Artificial Intelli- gence Graduate School Program (Sungkyunkwan University), the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.
2023-00213118 ITTP- ITRC (Information Technology Research Center) grant funded by the Korea government (MIST) (IITP-205-44 RS---R--2000 ASS2022--00436633, 10%), LTTP-CC Creative Consilience Program grant funded by the Korea government (MSIT) (TTP-2025-RSS-20000110 10%), and by Samsung Electronics.
REFERENCES
Constructions Acronautiques, Adele Howe, Craig Knoblock, ISI Drew McDermott, Ashwin Ram, Manuela Veloso, Daniel Weld, David Wilkins Sri, Anthony Barrett, Dave Christianson, et al. Pddl- the planning domain definition language. Technical Report, Tech. Rep., 1998.
Sudhir Agarwal, Anu Sreepathy, David H Alonso, and Prarit Lamba. Llm+ reasoning+ planning for supporting incomplete user queries in presence apis. arXiv preprint urXiv:2405. 12433, 2024.
Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, Julian Ibarz, Alex Irpan, Eric Jang, Ryan Julian, et al. Do as can, not as say: Grounding language robotic affordances. In Proceedings of the 6th Conference on Robot Learning, pp. 287-318, 2023.
Pedro Cabalar, Manuel Rey, and Concepción Vidal. A complete planner for temporal answer set pro- gramming. In Progress Artificial Intelligence: 19th EPIA Conference on Artificial Intelligence, EPIA 2019, Vila Real, Portugal, September 3-6, 2019, Proceedings, Part 19, pp. 520-525. Springer, 2019.
Shaofei Cai, Zihao Wang, Xiaojian Ma, Anji Liu, and Yitao Liang. Open-world multi-task control through goal-aware representation learning and adaptive horizon prediction. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13744, 2023.
Guanqi Chen, Lei Yang, Ruixing Jia, Zhe Hu, Yizhou Chen, Wei Zhang, Wenping Wang, and
Jia Pan. Languagg--uummnttn symbolic planner for open-worke task planning. arXiv preprint arXiv:2407.09792, 2024.
David Coleman, Ioan Sucan, Sachin Chitta, and Nikolaus Correll. Reducing the barrier to entry of complex robotic software: moveit! case study. arXiv preprint 1rXiv:1404.3788 2014.
Cristina Cornelio and Mohammed Diab. Recover: A neuro-symbolic framework for failure detection and recovery. arXiv preprint arXiv:2404. 00756, 2024.
Marc-Alexandre Côté, Ákos Kádár, Xingdi Yuan, Ben Kybartas, Tavian Barnes, Emery Fine, James Moore, Ruo Yu Tao, Matthew Hausknecht, Layla El Asri, Mahmoud Adada, Wendy Tay, and Adam Trischler. Textworld: A learning environment for text-based games. CoRR, abs/1800 1532, 2018.
Andrew Cropper and Sebastijan Dumančić. Inductive logic programming at 30: new introduction. Journal of Artificial Intelligence Research, 74:765-888 850, 2022.
Danny Driess, Fei Xia. Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Mare Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. Palm-e: An embodied multimodal language model. In arXiv preprint arXiv:2303..m 033788, 2023.
Meng Fang, Shilong Deng, Yudi Zhang, Zijing Shi, Ling Chen, Mykola Pechenizkiy, and Jun Wang. Large language models are ncurosymbolic reasoners. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 17985-1 1799, 2024.

---

Published as a conference paper at ICLR 2025
Bruce Frederiksen. Applying expert system technology to code reuse with pyke. PyCon: Chicago, 2008.
Martin Gebser, Roland Kaminski, Benjamin Kaufmann, and Torsten Schaub. Multi-shot asp solving with clingo. Theory and Practice of Logic Programming, 19(1):27-82, 2019.
Assaf Hallak, Dotan Di Castro, and Shie Mannor. Contextual markov decision processes. arXiv 2015.
preprint arXiv:1502.000
Shibo Hao, Yi Gu, Haodi Ma. Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305. 14992, 2023.
Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. Voxposer: Composable 3d value maps for robotic manipulation with language models. arXiv preprint NKKV:2307.00973. 2023a.
Wenlong Huang, Fei Xia, Dhruv Shah, Danny Driess, Andy Zeng. Yao Lu, Pete Florence, Igor Mordatch, Sergey Levine, Karol Hausman, et al. Grounded decoding: Guiding text generation with grounded models for robot control. arXiv preprint wxXiv:2303.00885 2023b.
Wenlong Huang, Fei Xia, Dhruv Shah, Danny Driess, Andy Zeng, Yao Lu, Pete Florence, Igor Mordatch, Sergey Levine, Karol Hausman, al. Grounded decoding: Guiding text generation with grounded models for embodied agents. Advances in Neural Information Processing Systems, 36, 2024.
Adam Ishay, Zhun Yang, and Joohyung Lee. Leveraging large language models to generate answer set programs. arXiv preprint arXiv:2307.07699, 2023.
Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew Davison. Rlbench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters, 2020.
Byeonghwi Kim, Minhyuk Seo, and Jonghyun Choi. Online continual learning for interactive in- struction following agents. arXiv preprint :xXx:2403300558, 2024.
Mark Law, Alessandra Russo, and Krysia Broda. The ilasp system for inductive learning of answer programs. arXiv preprint 2:Xx:2005.00004 2020.
Dachee Lee, Minjong Yoo, Woo Kyung Kim, Wonje Choi, and Honguk Woo. Incremental learn- ing of retrievable skills for efficient continual task adaptation. Advances in Neural Information Processing Systems, 2025.
Vladimir Lifschitz. Answer sel programming, volume 3. Springer Heidelberg. 2019.
Xinrui Lin, Yangfan Wu, Huanyu Yang, Yu Zhang, Yanyong Zhang, and Jianmin Ji. Clmasp: Coupling large language models with answer set programming for robotic task planning. arXiv preprint 2.Xiv:2406603367 2024.
Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. Lim+ p: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304.11477, 2023.
Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling open-vocabulary object detection.
Advances in Neural Information Processing Systems, 36, 2024.
Stephen Muggleton and Luc De Raedt, Inductive logic programming: Theory and methods. The Journal of Logic Programming, 19:629-679, 1994.
Theo X Olausson, Alex Gu, Benjamin Lipkin, Cedegao E. Zhang, Armando Solar- ezama, Joshua Tenenbaum, and Roger Levy. Line: A neurosymbolic approach for logical reasoning by combin- ing language models with first-order logic provers. arXiv preprint arXiv:2310.15166 2023.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint 1807.00807..33 2018.
12

---

Published as a conference paper at ICLR 2025
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35:
27730-27744, 2022.
Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. Logic-Im: Empowering large language models with symbolic solvers for faithful logical reasoning. arXiv preprint arXiv:2305. /2295, 2023.
Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Torralba. Virtualhome: Simulating houschold activities via programs. In Proceedings of the 29th IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8494-85000 8502, 2018.
Chiaki Sakama. Nonmonotomic inductive logic programming. In Logic Programming and Nonmo- tonic Reasoning: 6th International Conference, LPNMR 2001 Vienna, Austria, September 17-19, 2001 Proceedings 6, pp. 62-80. Springer, 2001.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024.
Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 10740-10749, 2020a.
Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 10740-10749, 2020b.
Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning. arXiv preprint arXiv:2010.03766 2020c.
Tom Silver and Rohan Chitnis. Pddlgym: Gym environments from pddl problems. arXiv preprint 20Xi::2002.064433. 2020.
Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. Progprompt: Generating situated robot task plans using large language models. In Proceedings of the 40th IEEE International Conference on Roboties and Automation, pp. 11523- 1922--1530, 11530, 2023.
Howard Smokler. Aspects of scientific explanation and other essays in the philosophy of science, 1966.
Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. Lim-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings the 19th IEEE/CVF International Conference on Computer Vision, pp. 2998-3009, 2023.
Sanjana Srivastava, Chengshu Li, Michael Lingelbach, Roberto Martin-Martín, Fei Xia, Kent Elliott Vainio, Zheng Lian, Cem Gokmen, Shyamal Buch, Karen Liu, et al. Behavior: Benchmark for everyday household activities in virtual, interactive, and ecological environments. In Proceedings of the 5th Conference on Robot Learning, pp. 477-490, 2022.
Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. MIT press, 2018.
Andrew Szot, Max Schwarzer, Harsh Agrawal, Bogdan Mazoure, Rin Metcalf, Walter Talbott, Na- talie Mackraz, R Devon Hjelm, and Alexander T Toshev. Large language models as generalizable policies for embodied tasks. In The Twelfth International Conference on Learning Representa- fions, 2023.
13

---

Published as a conference paper at ICLR 2025
Son Cao Tran, Enrico Pontelli, Marcello Balduccini, and Torsten Schaub. Answer set planning: survey. Theory and Practice of Logic Programming, 23(1):226 2023.
Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma, and Yitao Liang. Describe, explain, plan and select: Interactive planning with Ilms enables open-world multi-task agents. In Proceedings of the 37th Advances in Neural Information Processing Systems, 2023.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24833 2022.
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Enabling next-gen Ilm applications via multi- agent conversation framework. arXiv preprint arXiv:2308.08855: 2023a.
Zhenyu Wu, Ziwei Wang, Xiuwei Xu, Jiwen Lu, and Haibin Yan. Embodied task planning with large language models. arXiv preprint arXiv:2307.01844 2023b.
Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. Travelplanner: benchmark for real-world planning with language agents. arXiv preprint arXiv:2402.01622, 2024.
Jundong Xu, Hao Fei, Liangming Pan, Qian Liu, Mong-Li Lee, and Wynne Hsu. Faithful logical reasoning via symbolic chain-of-thought. arXiv preprini arXiv:2405. 18357, 2024.
Zhun Yang, Adam Ishay, and Joohyung Lee. Coupling large language models with logic program- ming for robust and general reasoning from text. arXiv preprint arXiv:2307.077
2023.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, 2023.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2024.
Minjong Yoo, Jinwoo Jang, Wei-Jin Park, and Honguk Woo, Exploratory retrieval-augmennte plan- ning for continual embodied instruction following. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang- Huei Lee, Montse Gonzalez Are- nas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, et al. Language to rewards for robotic skill synthesis. arXiv preprint arXiv:2306.086. 2023.
Zirui Zhao, Wee Sun Lee, and David Hsu. Large language models commonsense knowledge for large-scale task planning. Advances in Neural Information Processing Systems, 36, 2024.
Kaizhi Zheng, Kaiwen Zhou, Jing Gu, Yue Fan, Jialu Wang, Zonglin Di, Xuehai He, and Xin Eric Wang. Jarvis: A neuro-symbolic commonsense reasoning framework for conversational embodied agents. arXiv preprint arXiv:2208. 13266, 2022.
14

---

Published as a conference paper at ICLR 2025
A RELATED WORK
Embodied Control with Large-Language Models. Integrating LLMs into embodied control sys- tems has opened new opportunities to leverage extensive linguistic knowledge to guide agent be- havior. LLMs offer flexible and generalizable control of embodied agents in realistic environments, including real-world scenarios (Brohan et al. 2023] Huang et al.) 2023b Song et al., 2023] Driess al. 2023 Zhao al. 2024 Singh et al. 2023 Wu et al. [202b)] Wang et al. (022)). Our work aims to achieve robust performance in LLM-base embodied control within open-domain environments
by integrating the strengths of LLM common-sens, reasoning and symbolic reasoning tools.
Neuro-Symbolic Approaches. Neuro-symbolic approaches combine the strengths of neural net- works with symbolic systems, enhancing explainability, generalizability, and flexibilityy Recently, systems augmented with LLMs have significantly advanced in solving traditional logic problems within Natural Language Processing domains (Olausson et al., [023]] Pan al., 2023 Fang al.)
2024 Yang et al., (2023 [shaa et al.) 2023). Research interest is growing in adapting neuro-symbolic approaches to embodied domains (Lin et al. 2024 Liu et al. 2023 Agarwal et al. 2024; Cornelio &
Diab 2024). While existing approaches depend on the complete provision of expert-level symbolic knowledge for embodied control, our work addresses scenarios where this knowledge is insufficient inapplicable due to the unpredictable nature of open open-domain environments.
Large Language Model-based multi-agent framework. LLM-based multi-agent frameworks have recently garnered significant attention, with many works improving their problem-solving abilities through collaboration among autonomous agents. Recent advancements in multi-agent architecture show a trend toward integrating diverse techniques, where agents interact with external tools and environments to improve planning, execution, and iteration (Yao et al. 2024 Hao et al. 2023 Shinn al. 2024, Yao al. 2023 Wu al. 2023a). Our work introduces novel multi auulti-agent framework that emulates the -eduuciivee model, with a focus on generalizing actionable knowledge dynamic environments.
B ENVIRONMENT SETTINGS
B. ALFWORLD
We use ALFWorld (Shridhar et al. 2020c), an advanced simulator that integrates the text-based interactive environment of TextWorld (Côté al. 2018) with the visual and physical interaction capabilities the ALFRED benchmark (Shridhar et al. 2020a). This integration bridges the gap between abstract reasoning and physical action. ALFWorld includes 108 different object types (e.g., bread) and 37 receptacle types (e.g.. plate) spread across 120 diverse indoor scenes (e.g., kitchen). The platform supports 3554 unique tasks, each crafted by combining these elements with one of six instruction types (e.g., pick & place), such as "Put a keychain in a plate and then put them on shelf." Details of the instructions and executable plans are provided in Table and visualizations of various indoor scenes and observations in ALFWorld are depicted in Figure
Fast Thrn vhil balding
Koycus door
Seipue is batias Delppine
Gote prttegon
(a) Example Pick & Place
(b) Example of Examine & in Light
Figure 6: Task examples set of ALFFWolld
15

---

Published as a conference paper at ICLR 2025
Table Instructions and executable plans in ALFWorld
Type
Example
Pick & Place
Apply spray bottle to toilet.
Pick Two & Place
Locate two glass bottles and place them the shelf.
Clcan Place
Wash mug and place in the coffee machine.
Instructions
Heat & Place
Refrigerate the heated tomato.
Cool & Place
Chill the wine bottle and place it on the dining table.
Examine & in Light
Examine the compact disc beneath the desk lamp.
Goto |Receptacle Object]
Goto countertop
Open [Receptacle Object]
Open garbagecan
Close [Receptacle Object]
Close garbagecan
Pickup [Object] [Receptacle Object]
Take cloth from countertop
Put [Object] |Receptacle Object]
Put plate in/on diningtable
Heat [Object] |Receptacle Object]
Heat mug with microwave
Cool [Object] |Receptacle Object]
Cool apple with fridge
Plans
Clcan [Ohject] [Receptacle Ohject]
Clean tomato with sinkhassin Slice tomato with knife
Slice [Ohject] [Instrument Object]
Examine [Ohject]
Examine cloth
Examine [Receptacle Object]
Examine countertop
End
Finish
B.2 VIRTUALHOME
We also utilize VirtualHome (Puig al. 2018), a simulation platform designed for modeling every- day household activities in a 3D environment. This platform aids in the training and evaluation of agents who understand and execute complex task sequences based on language instructions. Virtual- Home includes 188 different object types (e.g.. Fridge) across 7 unique environment IDs. It supports 2821 distinct tasks, each created by combining these elements with various instruction types, such "Throw away newspaper". Details of the executable actions provided in Table and visual- izations of various indoor scenes and observations are depicted Figure]
Enviromment ID
Observation
Environment ID
Obscrvation
(a) Example of environment & observation
(b) Example of environment & observation
Environment ID
Observation
(c) Example of enviromment & observation
Figure 7: Environment examples set of VirtualHome
16

---

Published as a conference paper at ICLR 2025
Table 8: Instructions and executable plans in VirtualHome
Type
Example
Pick Place
Locate an apple on the kitchen table.
Pick Place
Detect an apple and convey it onto the kitchen table. Can you place apple upon the kitchen table? for the apple, hold it,
Pick & Place
Undertake the endeavor
pick & Place
move position the apple on the kitchen table.
Pick Sit
Grab book and sit on the bed.
Instructions
Scour the room for the book, firmly grab it, seek the bed, and ease yourself onto the bed. Begin the mission to fetch the book, seize the book, move to the hed, and relax into seated position on the bed. Embark upon the quest to get the book, pick the book, find the bed. and calmly take seat on the bed.
Pick Sit
Pick & Sit
Pick Sit
TurnLeft
Turnleft
TurnRight
Turnright
StandUp
Standup
Walk [Object1]
Walk kitchen
Run [Object]]
Run kitchen
Walkforward
Walkforward
Walktowards [Objectl]
Walktowards kitchen
Sit [Objectl]
Sit chair
Grab [Object1]
Grab apple
Plans
Open [Object
Open fridge
Close [Object]]
Close fridge
SwitchOn [Objectl]
Switchon stove
SwitchOff [Objecti]
Switchoff stove
Drink [Objectl]
Drink waterglass
Touch [Object]]
Touch stove
LookAt [Objectl]
Lookat stove
Put [Objectl] [Object2]
Putback apple table
PutIn [Objectl] [Objec12]
Putin apple fridge
B.3 MINECRAFT
In this study, we also utilize the Minecraft environment from PDDLGym (Silver & Chitnis 2020) as a benchmark for testing task planning in complex outdoor scenarios. This environment is one of the seven classical planning domains written in PDDL that empirically tested our approach on, as mentioned in the main text. Minecraft provides a unique setting for evaluating planning algorithms in dynamic, spen-world context. Key Features:
Outdoor Environment Simulation: Unlike many indoor-based benchmarks. Minecraft simulates outdoor environment, presenting challenges
akin real-world scenar- ios. The environment includes fundamental materials such as wood, stone, and grass, with which agents must interact by moving, processing, and storing them.
Distinet Skill Requirements: Tasks in Minecraft demand different set of skills compared to indoor environments, including resource gathering, crafting, and construction. Addi- tionally, agents need to continuously assess the current state of their tasks and adapt their strategies as the environment changes, requiring dynamic task management skills. The en- vironment also provides sequentially complex task instructions that agents must interpret and execute. These instructions involve multiple steps that must be performed in a specific order.
These characteristics make the Minecraft environment particularly suitable for testing egocentric planning approaches. challenges agents to operate effectively in complex, dynamic world where the ability to adapt to changing circumstances, manage resources efficiently. and execute multi-step instructions is crucial. The open-world nature of Minecraft, coupled with the complexity of the provided commands, allows for the creation of diverse and challenging scenarios. This provides
17

---

Published as a conference paper at ICLR 2025
robust testbed for evaluating the flexibility and effectiveness of planning algorithms in environments that more closely resemble real-world outdoor settings and task complexities.
Details of the executable actions provided in Table 9 and visualizations of various grid worlds and observations are depicted Figure
Table 9: Instructions and executable plans in Minecraft
Type
Example
Move & Equip
Move location 2-4 and equip grass-O.
Collect & Move
Collect grass-1 and move location 1-1.
Craft Equip
Craft planks from new-1 and equip them.
Instructions
Move & Inventory
Move to location 0-3 and store new-0 in the inventory.
Equip & Inventory
Equip grass-2 and store log-3 in the inventory.
Craft & Inventory
Craft planks from new-O and store them in the inventory.
Recall [Object]
Recall log-1.
Move [Location]
Move 1oc-2-4
Plans
CraftPlank [Object] [Object]
Craftplank new-O log-1. Equip grass-O. Pick grass-l from lac-1-1.
Equip [Object]
Pick [Object] [Location]
(a) Example of "Get new-O item and go to loc-0-3."
(b) Example of "Equip grass-2 and get log-3."
Figure 8: Environment examples set of Minecraft
B.4 RLBENCH
We implement a tabletop manipulation environment using the RLBench (James al. 2020 with a Franka Emika Panda 7-DoF robotic arm. This setup integrates precise robotic control with a rich, interactive environment, bridging the gap between abstract reasoning and physical manipulation. Our environment consists of 12 diverse objects arranged within customizable workspace on wooden table. The objects include everyday items and structural elements: a charger, walls, steak and chicken, grill, wine bottle, cap, 2 windows (right and left), and handles for each window. The position each object is randomly assigned for each instance of the environment, creating unique configurations every time. The scene is illuminated by 3 directional lights ensure consistent visual input. For perception, we employ a stereo camera system and a monocular wrist camera, providing RGB, depth, and segmentation mask data for each frame. Additionally, the system
18

---

Published as a conference paper at ICLR 2025
retrieve robot proprioceptive data, including joint angles, velocities, torques, and end-effector pose. Additionally, we used VoxPoser (Huang et al.) 2023a) as a skill decoder to execute actions. By using this skill decoder, it became possible to execute open-set actions.
Details of the executable actions are provided in Table 10 and visualizations of various tabletop scenes and observations are depicted in Figure g
Table 10: Instructions and executable plans in RLBench
Type
Example
Pick & Place
Pick the chicken from the grill and place it in the goal place.
Pick & Move
Unplug the charger
Instructions
Pick & Rotate
Open wine bottle.
Pick & Rotate Move
Open left window through the handle.
Grasp [Object]
Grasp cube.
Rotate [Direction] [Degree]
Rotate clockwise 100 degrees.
Move [Direction] [Distance]
Move forward 10cm. Move to top of sphere.
Plans
Move To [Position] [Object]
Open Gripper
Open gripper.
Back To Default Pose
Back default pose.
(a) Example of "Unplug charger" with gripper view
(b) Example of "Open the left window through the handle" with overhead view
Figure 9; Environment examples set of RLBench
B.5 REAL-WORLD
Setup and Implementation. In our real-world experiments, we implemented tabletop manipula- tion system using a Franka Emika Research 7-DoF robotic arm. The environment was equipped with an RGB-D camera(Intel RealSense D435) mounted for a top-down view, providing point cloud and RGB information for object detection and coordinate estimation.
Environment Configuration. Our experimental setup consisted 10 common tabletop objects: three Hanoi tower blocks of varying sizes, three Hanoi tower poles, three paper cups, plastic cup, one cardboard box, and a trash can. To ensure robustness, object positions were randomly assigned for each experimental instance, creating unique environmental configurations.
Control System Implementation. Unlike the RLBench experimental setting where VoxPoser was adapted, our roal-world implementation utilized distinct methods for both perception and control:
19

---

Published as a conference paper at ICLR 2025
For perception, we integrated RGB-D camera with an object detection module (Minderer ct al. 2024) for accurate object coordinate identification:
top-view image captured using camera positioned above the table.
Thee object detection module (Minderer al., 2024) used identify the categories
and bounding box coordinates of one or more target objects from the top-view image, with their heights calculated using a depth camera.
The bounding box and height coordinates of the target object(s) are converted into the robot arm's s coordinate system.
For low-level control, we employed Movelt (Coleman et al. 2014), an open-source robot motion-planninn framework:
Model Predictive Control (MPC) was implemented for trajectory optimization, which
highly effective in handling complex environments and constraints while ensuring real-time execution performance Yu et al. (003)).
When one or more transformed target object coordinates are provided, they are input into predefined heuristic skill functions (e.g., 'move to B', "grasp A') to prioritize which target coordinates to approach first and assign waypoints for these target coor- dinates accordingly.
The inferred waypoints are fed into the motion planning algorithm (i.e., MPC) to compute the low-level action sequence.
The robot executed movements using pre-defined primitive skills based on the generated plans.
While RLBench experiments used VoxPoser for low-level control with parameterized actions (e.g., accioonforate(Cllockiss,,, 100), T)), our real-world implementation adapted this approach handle physical constraints. Although continuous force control for the gripper was not implemented, we developed discrete mapping system that translates physical parameters into appropriate grasping strategies. An example of this mapping is provided in Table 12] The detailed executable actions are provided in Table [T] and the environmental setup shown Figure I0 The actual robot arm
actions executed according to plans generated by NESYC can be observed in Figure 13
Table 11: Instructions and executable plans in real-world experiments
Type
Example
Pick N & Place N
Clean up the table.
Instructions
Pick N & Place N
Complete the Tower of Hanoi.
Grasp [Object]
Grasp log-1.
Move [Object] to [Location]
Move cup trashbin.
Plans
Open Gripper
Open gripper.
Back Default Pose
Back to default pose.
B.6 DATASETS
Table 12: Mapping of physical parameters to discrete semantic actions
Semantic values for parameter Grip Grab Region (Diameter Range)
precision
Center Point + 0-10%
focus
Center Point 10-20%
standard
Center Point + 20-30%
balance
Center Point + 30-40%
power
Center Point 40-50%
In the static setting. the episodic data used for each environment is as follows: ALFWorld used 10 episodic data, VirtualHome used 15 episodic data, Minecraft used 6 episodic data, and RLBench
20

---

Published as a conference paper at ICLR 2025
Figure 10: Environment examples set of real-world
used 5 episodic data. Episodic data comprises series of transitions, capturing the state, the action taken, the success or failure of the action, and the resulting next state. Additionally, episodic data expressed text using VLM LLM to represent visual observations sensor data obtained from the environment. Note that this dataset is only a small portion of the overall evaluation environments used in experiments.
For each task, we generate several rephrased instructions using ChatGPT, based on the templa- tized instructions provided by each benchmark, as detailed in Szot et al. (202)). These variations demonstrate how accurately NESYC can define goal conditions and execute tasks in open-doman scenarios, allowing to evaluate its performance ple, the original instructions such as "Clean some plates and put them in the fridge." "Examine the pillow with the desk lamp," and "Heat some tomatoes and put them in the fridge" are transformed into generated instructions like "Wash few plates and refrigerate them," "Inspect the pillow using the desk lamp," and "Refrigerate the heated tomatoes," respectively.
diverse range of instructions. For exam-
B.6.1 TASK-AGNOSTIC DATASET GENERATION PIPELINE
Our dataset generation follows three-step process create task-agnostic examples:
Step I: Task-agnostic Environment Information Collection. The first step consists of two main processes. Initially, we collect environment meta-information including lationships, and transition rules from the base environment. (Srivastava et al. 2022) Then, using this collected information context, leverage ChatGPT to generate new task-agnostic transition datasets.
layouts, object
Step State Set Example Generation. This step involves creating two types of examples: positive and negative. Positive examples represent scenarios where actions are affordable and transitions are valid, documenting successful state changes with satisfied preconditions and effects. Negative examples demonstrate unaffordable actions and invalid transitions, capturing constraint violations and realistic emror cases. This dual approach ensures comprehensive coverage both successful and failed interactions.
Step 3: Task-agnostic Episode Scenario Generation. The final step transforms the generated ex- amples into task-agnostic experience dataset, where each experience is encoded in natural lan- guage form, abstracting specific objects and actions into general categories while maintaining rich textual descriptions of environmental states and interactions. We provide detailed examples of this experience dataset in Section B.6.2
21

---

Published as a conference paper at ICLR 2025
B.6.2 EXAMPLE OF DATA STRUCTURE
"instruction"; "The robot needs to extract a fragile antique teacup from a high kitchen cabinet and place it safely on the dining table for inspection on.",
Sinitial_obseraatiie
"You're in traditional kitchen with oak cabinets. It's mid-noming, and sunlight streams through lace curtains, creating intricate shadow patterns. The high cabinet's glass door reveals several pieces of antique china. The valuable teacup in question sits the top shelf, partially hidden behind a larger serving plate. stepstool is visible near the refrigerator, and there's a slight layer of dust on some of the cabinet surfaces, suggesting those items aren't frequently accessed.",
"trajectory":
*0":
"observation": "The antique teacup is clearly visible through the glass cabinet door, sitting on the highest shelf approximately 7 feet from the floor. It's a delicate Bone China piece with hand-painted roses and gold trim, estimated be from the 1920s. The cup positioned behind a heavy ceramic serving plate, which partially blocks access. The cabinet's brass handle shows fingerprints from previous use, and the glass panes have smudges that distort the view slightly. The overhead lighting reflects off the cabinet's glass, creating glare spots that make it difficult to see all angles of the teacup. A thin film of dust is visible the shelf surface.",
"action": "pick_up stepstool from floor", "affordance": "true",
Thext_observatee
"Standing with the stepstool, you' re facing the cabinet. The stool's rubber feet rest on the recently waxed hardwood floor, which shows noticeable sheen. The teacup remains visible through the glass, but from this angle, you can now see that the serving plate in front appears to be leaning slightly against the cup. The cabinet's hinges look slightly loose, with one screw not fully tightened. The morning sunlight has shifted, causing stronger glare on the cabiret glass."
"1":(
B.6.3 IMPLEMENTATION CONSIDERATIONS
Our implementation emphasizes several key aspects ensure dataset quality:
Consistent use of structured templates across all generated examples
Regular validation of logical coherence and physical plausibility
Careful balance between specific detail and general applicability
Maintenance of realistic physical constraints and interaction patterns
B.7 PERFORMANCE DIFFERENCES IN OPEN- OPEN-DOMAN DOMAIN SETTINGS
The performance differences observed in the baseline methods between the existing work and our experiments, such as in ALFWorld. are primarily attributed to changes in the environmental config- uration made accommodate the open-domain setting.
22

---

Published as a conference paper at ICLR 2025
The environmental settings designed to reflect open-domain conditions, building upon existing benchmark configurations, which are inherently complex due to their dynamic and unpredictable nature. These settings involve: (1) expanded observations, including spatial relations and object physical states, along with diverse and rephrased instructions that go beyond simple templates, re- quiring agents process varied linguistic inputs, as demonstrated in [Zheng et ) Chen al.. (2024); and (2) frequent environmental changes, such as unpredictable state transitions and variable object affordances, necessitating robust reasoning and adaptability, similar to Yoo et al. (2024) Cai et al. 2023).
Specifically, for observations, we configured the environment to include a wide range of object rela- tions and states, extending beyond simple list of object types. This setup incorporates richer details using various predicates related to object states and relations (e.g., pickupable, sliceable, can open, dirty). For instructions, instead of relying solely on about ten types of template-based instructions (e.g., "Heat some tomato and put in the fridge.") from the original benchmarks, we utilized a di- verse set of paraphrased instructions (e.g.. "Refrigerate the heated tomato."), These extensions were designed to effectively capture and utilize the changes in environmental dynamics as inputs for the agent.
In terms environment dynamics, we categorized the settings into three levels based on their dy- namics, each affecting state changes:
For Static settings, the environment remains consistent across episodes, with stable object states, action preconditions and effects, and goal conditions.
For Low Dynamic settings, objects may change locations or conditions within episodes, but these changes require only minor adjustments to the agent's 's existing transition rules.
For High Dynamic settings, object states and attributes can change unpredictably within episodes, affecting goal conditions and action preconditions, significantly increasing complexity. The agent must continuously re-evaluate its plans and refine its knowledge to effectively handle frequent state shifts and varying affordances.
For this reason, baselines such as ReAct (Yao et al. 2023) and Reflexion (Shinn et al. 2024) show performance differences even in the static setting. Additionally, to ensure fair and meaningful com- parisons under these open- open-domain conditions, we made the following adjustments for the baselines, as demonstrated in Xie et al. (2024).
For ReAct and Reflexion, the original papers utilize task-specific demonstrations as few-shot input prompts. In our setting, however, we provide task-agnostic demonstrations as input prompts without specifying task information, which increases the complexity of grounding within the given domain.
To quantitatively validate this performance difference, we conduct ablation studies with ReAct fo- cusing on two key factors: expanded observations and task-specifie few-shot prompts, which can affect model performance. We randomly selected 50 tasks and used GPT-4o-mini the L.M for the experiments.
The results indicate that when we adjust our Static settings to replicate the original experimental settings used by ReAct-by reducing the complexity of observations and providing task-specific prompts-the performance aligns more closely with the results reported in the original paper. This demonstrates that the observed performance differences stem from the additional challenges intro- duced by our open-domain settings. Additionally, the 71% SR in Yao et al. (2023) corresponds to the average performance the best-performing task category. However, when considering the overall average performance across all task categories, the reported SR in the original paper is 57%. In our experimental results in [3] when comparing the average performance across all tasks, Re-
Act achieved approximately 59.2% in the reproduced original setting, which aligns closely with the score originally reported in Yao et (2023).
C BASELINES
We implement several baselines for comparison. These baselines represent diverse range of LLM- based approaches for task planning and execution in embodied environments, including methods that utilize dynamic replanning, self-reflection, grounding agents, and declarative programming. Each
23

---

Published as a conference paper at ICLR 2025
Table 13: Performance comparison of ReAct on variants of ALFWorld environmental settings.
Static
Configuration
SR
GC
Static setting (woo task-specific prompt & w. expended observations)
14.3
33.7
Original setting (w/ task-specific prompt & w/o expended observations)
59.2
64.8
baseline is adapted, where necessary, operate in open-domain setting, ensuring fair comparison with our proposed method.
C.I LLM-BASED PLANNING APPROACH
The hyperparameter settings for the LLM-based planning approach summarized in Table [4 For l'air comparison, incontext samples were provided through retrieval of the top k samples. For Alfworld and Minecraft k=3, while for VirtualHome k=5 and RLBench k=10.
LLM-planner (Song et al., 2023) utilizes an LLM for embodied task planning with dynamic re-planning. While originally designed use task mask-specific expert knowledge, our implementa- tion adapts it to an open-domain setting by providing task-agnostic experiences retrieved from general dataset, aligning with our proposed method for fair comparison.
Table 14: Hyperparamer settings for LLM
Hyperparameters
Value
LLM configuration
Model
200-4--002
Text generation configuration
Temperature
0.0
Top
Top
1.0
Maximum new tokens
256
C.2 LLM-BASED MULTI-AGENT FRAMEWORK
The hyperparameter settings for LLM- based multi-agent frameworks summarized in Table 14 Other settings are similar to those of the LLM-based planning approach. However, in the case of AutoGen, additional expert knowledge was provided through the grounding agent. For the remaining comparison groups in this approach, ReAct and Reflexion, this additional expert knowledge was not provided.
ReAct Yao et al. 2023). an LLM-based agent repeatedly performs reasoning and decision- making to solve a given task the environment. Upon receiving observations, the LLM alternates between generating thoughts (reasoning) and acts (actions), autonomously making decisions to complete the task. To ensure a fair comparison, we provided task-agnostic samples for retrieval, similar to our approach with LLM-planner (Song al. 2023). rather than using task-specific knowledge.
Reflexion (Shinn and acting, but uniquely incorporates a self-reflection mechanism that generates feedback by an- alyzing its long-term and short-term memory, using this introspective insight to iteratively refine its decision decision-making process.
2024), an LLM-based agent that, like ReAct, alternates between reasoning
AutoGen (Wu et al. 2023a), an LLM--bsed system that, similar to ReAct, employs reasoning and acting eycles, but distinctively features separate grounding agent that leverages both expert domain knowledge and implicit linguistic understanding to anchor the system's 's operations in the environment.
24

---

Published as a conference paper at ICLR 2025
C.3 CEURO-SYMBOLIC APPROACH
The hyperparameter settings for neuro-symbolic approaches are summarized in Table 14 Other settings are also similar to those using the LLM-based planning approach. CLMASP used an ASP solver, and in D.2 used the same settings as ours.
ProgPrompt (Singh et al. 2023) utilizes a predefined code-based system for planning, incorpo- rating human-crafted programmatic assertion syntax verify skill execution pre-conditions and respond to failures with predefined recovery rules; it requires expert knowledge in imperative pro- gram formats, which adds its specificity but potentially limits its accessibility. However, it lacks a dynamic replanning mechanism, limiting its adaptability to unforeseen scenarios.
CLMASP (Lin et al. 2024) is neuro-symbolic approach that integrates LLMs with ASP, uti- lizing ASP's non-monotonic logic programming to represent and reason based on the robot's action knowledge; it employs declarative programming, making it the most similar approach to our research. This combination of neural networks (LLMMs and symbolic reasoning (ASP) allows CLMASP leverage the strengths of both paradigms. However, CLMASP lacks mechanisms for dynamically generating and improving its programs, which limits its adaptability and self- improvement capabilities.
D NESYC
D.1 HI SCORING
HI a STPR (H, E+ BK) -
-1 (1 (1-a) o) SFPR (H, (H,E BBK)
where:
PPRR H, E+ BK) X TPT
TPM
+ +(1 -))
|ET
FFM
SVPR (H. ET, BK) = A FPT
FPM
+ +(1 (1-A)
E
|EM
Eq. 9 is rigorous and specific version of 7q. Here, À (0, 1] is hyperparameter that balances the importance between the existing experience set T and working memory M. A higher À value gives more weight the performance on T. while a lower value emphasizes MM. Additionally, E [0, 1] is a parameter that adjusts the relative importance of TPR (True Positive Rate) and FPR (False Positive Rate) in the hypothesis evaluation. higher a value places more emphasis on TPR, while a lower value gives more weight to minimizing FPR. For each experience set X' E MT.
we define:
TPx He E E : BK U HH
FPx I[f Ex e BK U HH
(10)
TNx Ie E Ex :e BK Uff)
FNx = He E Et ee BKUH)
where TPx represents the number of true positives (i.c.. positive examples in E that are models of BK UH,, Px is the number of false positives, TNN is the number of true negatives, and F FNx is the number of false negatives. The symbol F= denotes the satisfaction relation, indicating that an example e a model of BK U H.
This formulation allows for comprehensive evaluation of the hypothesis H, considering its perfor- mance on both the existing experience set and the current environment experience set while provid- ing flexibility in adjusting their relative importance through the parameter.
25

---

Published as a conference paper at ICLR 2025
D.2 HYPERPARAMTER SETTINGS
The configuration of the LLM follows the setup described in Table 14 For the heuristic solver in Answer Set Programming (ASP), we used clingo version 5.7.1. The clingo solver was with spe- cific control parameters. The parameter --opt-mode=optNN was used to specify that the optimization mode was set find all optimal models. The option -t '1' controlled the number of threads used dur- ing execution, which was set to I in this case. Lastly, the '-seed' parameter was employed to control randomness during solving, and seeds from 1 to were used to ensure reproducibility across differ- ent runs. For different LLMs, we include Llama-3 (meta-llama-3 3.0-8b, meta-llama-3.0 3.0-m00,, GPT-4 (gpt-4o-mini, gpt-4o), Claude
claude-3-opus),
D.3 COMPONENT DETAILS
D.3.1 EXAMPLES OF ILP
Consider the simple embodied agent scenario, where B:
sapll(apple). small(cup). smali(glass. vase). B lipht(apple). light(cup)- heavy(table)). heary(shelf). Tragile(glass vase). nol. rragieeg))) :- small(x)), light
and the examples E [E+ }:
i pickup(apple, table).
6 pickup(table, table).
E
E =
22 pickup(cup shelf).
E1
pickup(glass.vase, shelf),
Also assume the hypothesis space H::
hl pickup(z x, v) small(x), (eaaall(
H h2 pickupl xx, 8) :- small(s)) (igh(()) heaehyy))) h3 pickupp xz v) small((x) nigh
heavyls).
we need to find hypothesis H such that ef and e are models of H E B, while ej and eg are not. In this case, ej is not a model of h2 because there exists substitution 0 == (z/taalle, y //ttable} such that the body does hold, and thus the head is not valid. For the same reason, none of the examples is a model of hy. This indicates that the hypothesis H = (h2, h2 consists of both h2, h3.
D.3.2 IMPLEMENTATION GUIDELINES FOR ASP ACTION RULES
ASP rules for actions are primarily divided into two categories: Precondition rules that specify when actions are allowed, and Effect rules that define how actions change the world state.
Action Precondition Format:
ion (ActionNane) (Args), Time), Condl (Args, Time), not Cond2 (Args, Time)
Uses integrity constraints (:-) to specify invalid action conditions
When the body of constraint is satisfied, the action is forbidden
not operator indicates the precondition must be satisfied Multiple constraints can be defined for single action
Action Effect Format:
State (Args, Time) act (Act i enName (Args), Time)
Defines state changes resulting from actions
State be any predicate (e.g., holding/////cco.o......c..i..mpi 2, at/3)
Direct causal relationship between action and its effects
Can include additional conditions with multiple body literals
26

---

Published as a conference paper at ICLR 2025
Key Components:
action/2: Predicate representing actions with arguments and time
State predicates: Represent world states (e.g., holding/ at/3)
Condition predicates: State properties that must hold
Time variable: Usually denoted for temporal reasoning
Variables: Typically capitalized (e.g., O for object, for location)
Anonymous variable, used when specific value is not relevant
Each is treated as distinct, unique variable
Used when we don't need to reference the value later
Multiple in the same rule are independent variables
Example of Precondition Rules:
1. object Location Check:
(pick (0, 1), T), not at (0, L, T). Ensures object is at lecatien before pickup
2. Holding State Check:
:- action (pick_ up (O, J, T), holding T
8 Prevents pickup when already holding something
D.3.3 EXAMPLES OF ASP
Given scenario where an agent receives environmental observation information and an instruction to "pick up a fruit", here is an example an ASP program for this simple task.
1. #program base.
2. location (table). object (plate). object (fork) object (orange). hol (none)
3. (X) :- object (X)
4. holds
init (F).
5. init (on (fork init (cn (crance,, plate) init (an (plate, table))
6. goal (helding
7. #program step(t).
8. 0 { occurs (pickup (t, object (x), location(Y), X != 1.
9. occurs (pickup (x, t), holds (on (A, X) (t-1).
10. holds (helding (X) t) :- occurs (pickup (x, t),
11. holds (on (X, 2) :- holds (on (X, 2) t-1), not hxlding
12. #program check (t).
13. (u)ry(t), goal(F), not helds (F,t).
The planning problem defined in lines to 6. This code sets up scenario where three ob- jects-plate, fork, and orange-are are placed either on a table or on each other. The goal for the agent be holding the orange. The initial positions of the objects are specified using the init predi- cates. The action logic is defined in lines 7 to 11, which includes the preconditions and effects of the pickup action. The occurs predicate is used specify when actions happen, while the preconditions (line 9) restrict when the pickup action can be executed. The effects (line 10) update the state of the environment based on the actions taken. lines 12 and 13, the goal condition checked to ensure that the agent reaches the desired goal state.
For ALFWorld agents, we define symbolic actions by combining behaviors (e.g., "pick up"") with target objects (e.g., "apple") and receptacle objects (e.g., "sink"). A symbolic action can be repre- sented as pXckup(X, Y), with preconditions such as at(apple 2, diningtable 1), pickupable(apple 2), Object(apple 2), and receptaclefdiningtuue
1). The effects of this action would be hold(agent, apple 2), and at(apple 2, agent). For RLBench agents, we define skills based on the combination of prim- itive actions (e.g., "move forward". "rotate left". "gripper open"), target objects (e.g., "window"", "umbrella", "red block"), and action-specific parameters that quantify the magnitude of the action (e.g., "Degree: 55" for rotation, "Distance: for movement, or "Force: 2" for gripping).
27

---

Published as a conference paper at ICLR 2025
Consider the symbolic action pickup(bread, StoveBurner), which is intended to fulfill the user in- struction, "heat the bread and place it on the side table". If the object state of the bread has already changed to "heated" before the agent performs the pickup action, the agent should re-plan to go directly to the side table, bypassing the use of the microwave. Continuing with actions intended to heat the bread in this situation could lead inefficiencies ramifications.
incorrect plan due to unintended
D.3.4 EXAMPLES PROMPT
Prompt1 of Hypothesis Generator
You inductive logic programming agent using Answer Set Programming. Task: Proceed with the inductive logical programming process. Generate the Background knowledge based on learning from interpretation.
Role:
Learning from Interpretations (LFT) Concept:
1. Background Knowledge (B):
father(henry.bill) (lanbrralan,etsy) father(alan.bemny)
mother(beth.bil)) mother(ann, mother(alice,benny)
2. Positive Examples (E+):
el {carrier(alan), carrier(ann), carrier(betsyy)
c2 {carrier(benny), carrier(alan). carrier(alice))
3. Negative Example (E-):
e3 {carrier(henry), carrien(beth))
4. Hypothesis Space (H):
hl carrier(X):-
h2 carrier(X)=- mother (.x).aa)e)r
LFI Problem Definition:
Find hypothesis H such that el and e2 models of H U and e3 is not.
Predicates: {predicates}
Instructions:
1. Read the provided LFI descriptions.
2. Use only the predicates and parameter descriptions provided.
3. Generate the Background Knowledge that can use hypothesis about Positive/Negative Examples.
Positive/Negative Examples: (gesitive.negatiee_exampl
Make Background knowledge.
Prompt2 of Hypothesis Generator
You are an inductive logic programming agent using Answer Set Programming. Task: Proceed with the inductive logical programming process. Find hypothesis based learning from interpretation.
Role:
ASP Rule Formats:
Constraint Rules (also known as Integrity Constraints):
Learning from Interpretations (LFI) Concept:
1. Background Knowledge (B):
father(henry, bill). father(alan, benbrr(alan.benny).
mother(beth.biig mother(ann,betsy)) mother(alice,benny)
28

---

Published as a conference paper at ICLR 2025
2. Positive Examples (E+):
cl {carrier(alan), carrier(ann), carrier(betsy))
c2 {carrier(benny), carrier(alan), carrier(alice))
3. Negative Example (E-):
e3 {carrier(henry), carrien(beth))
4. Hypothesis Space (H):
hl carrier(X):-
h2 carrier(X):- mother(
LFI Problem Definition:
Find hypothesis H such that el and e2 models of H U B and e3 is not.
Predicates: {predicates}
Feedback: (inlepprete__feedback] Instructions:
1. Carefully read the provided ASP and LFI descriptions. Create rules in the Constraint Rules format.
2. Use only the predicates and parameter descriptions provided.
3. Apply O-subsumption for Rule ordering.
Think generalized rules to include positive examples and Background Knowledge for target predicates while excluding negative ones, ensuring that feedback is incorporated into the generalization process.
Generate the most general rules.
Positive/Negative Examples: (positive negative examples)
Background Knowledge: (background knowledge)
Target Action Predicate: aepeeaaactio_peddiccte
Make generalized rules.
ASP program example for the Alfworld from Rule Generalization
Constraint:
object cannot be picked up if it is not at the robot's s location.
action (pick_up L), T), not at (0, L, T)
Constraint: object cannot be picked up if the robot is not the object's location.
action (pick_up (0, L), T), not robot_at (I, T)
Constraint: An object cannot be picked if it is not openable and opened.
action (pick_upp)pp (O, L), T) not openable (I), is open (L, T
Constraint: An object cannot be picked if it is not being held
action (pick_up (O,, L), T), not holding (0,
29

---

Published as a conference paper at ICLR 2025
Positive/Negative Examples
Positive Examples:
iis.opened(bookcass, noboLatthookcaas_Joaatt 1), at(book, book- casc.location, 1), up(book, bookcase loccation, 2)}
{is opened(toollox, 3), at(tool, toolbox, 3), robot at(toolbox location,,, 3), ac- tion(pick, .uttool,, toolbox), 4)}
Negative Examples:
{not is @ 5), holding(shir,, 5), robot at(wardrobe, 5), tion(open(wardrobe)) 5), action(pick up(shirt, wardrobe), 6)}
{is opened(refrigerrtoor 7), holding(food, 7). robot at(kitchen, 7), ac-
tion(opendreffigeratto)) 7), action(pick up(food, refrigerator), 8)}
(pp(poo,
Predicates Examples
Basic Predicates:
Object Properties:
location(L).
openable(L)..
object(O)).
cleanable(O).
goal(T),
coolable(O).
step(T).
heatable(O).
sliceable(O).
Specitic Objects:
State Predicates:
is peppershaker(O),
at(O, L, T).
robot_a((,, T).
holding(O, T).
is is.pillow(O).
Furniture and Appliances:
is (Oe T).
bed(OO.))
Location Properties:
Countertop(O).
is .s_heeter(L.).
is.cooler(L).
is cleaner(L).
dresser(O).
Actions:
Kitchen Items:
action(go.to(L). T).
is-pot(O).
action(pick.up(O, L)., T).
is.winebottle(O).
action(close(O)... T)..
is spatula(O)..
30

---

Published as a conference paper at ICLR 2025
Background Knowledge Examples
Locations:
Openable:
location(bookcase location).
openable(bookcase).
location(dishwashee)...
openable(reffigerattr))
location(kitchen)
Cleanable:
Objects:
clcanable(plate).
object(book).
object(food).
E ADDITIONAL EXPERIMENTS
E.1 SIMULATED ENVIRONMENT ROLLOUT TRAJECTORIES
Figure illustrates the demonstration trajectories from the baselines and NESYC for the window manipulation task. LLM-pplnner in Figure [I(a) and ReAct in Figure [aibeailed because they did not consider the preceding action rotating the gripper forward to grasp the window handle and attempted grab the handle directly. Reflexion in Figure [Tc] successfully grabbed the window handle but failed to account for the preceding action of rotating the handle before pushing the win- dow. AutoGen in Figure [Idd) influenced by the environmental description "consider preceding actions for the main action" in the grounding prompt, successfully completed the task. However, there is tendency to perform inefficient planning due to the inability consider all preceding ac- tions. CLMASP in Figure and NESYC C in Figure [ achieved efficient planning by considering all preceding actions through symbolic execution.
E.2 ADDITIONAL EVALUATION ON PREDICATE CATEGORIES.
Figure 12 presents graphs that evaluate not only the Fl score, as shown in Figure 4 but also the Recall and Precision scores. The left side of the graph displays Recall scores, while the right side shows Precision scores. The color coding for different Large Language Models (LLMs) corresponds that used in Figure For the Status category, the Recall scores of each LLM demonstrated similar performance improvement trends to their F1 scores. In contrast, for the Spatiality category, the Precision scores showed trends similar to the F1 scores in terms of performance improvement. This additional analysis provides more comprehensive view of the models' performance across different evaluation metrics.
E.3 DETAILED SCORE OF GENERALIZATION LOOP EVALUATION ON EXPERIENCE SET.
Table 15 shows the raw performance data (%) of various models evaluated on their ability to gener- alize from experiences. This table provides the numerical values represented in Figure 3 Precision, Recall, and score measure the models' prediction accuracy, while Specificity and Accuracy as- sess their overall correctness. GPT-4 performs best overall, with the highest Precision. Recall, and Fl score, though its Specificity is moderate. Llama-3.0-8B shows the weakest performance, with low scores across the board.
31

---

Published as a conference paper at ICLR 2025
Image Observation
Map
Plan Warpoont Sequence
Collison Detection
Callision
Detection
buzile
staper
grop hands
lack (afind)
(a) ALM-planner
Image
3D Vocol Map & d'sypoint Sequacce
Collision Detection
Plan Sequencee
aet slndow handla
rotale left 90 dogrues
zaav Sorward 10
pipppr
back to defaell poss
(b) ReAct
Image Observation
Vocel Map &
W Waypoint Sequenic
Collision Detection
Plan Soquence
focwadd degrees
bundle
dosce
15cm fooward
badk defuult
(c) Reflexion
Image Observation
3D Vocel Mag & Waypoint Sequacce
Plam Sequence
intata forword 53 dagres
wildio Landle
mitte doobonse 96 degrees
ogmL papperr
mave Liam tonvad
(d) AutoGen
32

---

Published as a conference paper at ICLR 2025
Image Observation
Map
Warpoont Sequence
Success
Plan
graep landle
forwod
(c) CLMASP
Imagre Observation
3D Vetell Map & Waypotnt Sequunce
Suncess
Plan Sequence
motate foeward degrus
grepp window bundle mraate dledontoe 100 degrov mone ban left
move 1öen forward
(f) NESYC
Figure 11: Visualization of trajectories R.BBench.
60
50
50
40
Attribute
Status
Spatiality
40
Attribute
Status
Spatiality
Figure 12: Recall score and Precision score evaluation on predicate categories.
Table 15: Detailed score of Generalization loop evaluation on experience set Before A: After).
Model
Precision
Recall B
F1
Specificity B
Accuracy
HI Score
B
A
A
B
A
A
B
A
B
A
GPT-4
66.5
65.7
91.7 95.5
77.1
77.8
53.7
50.0
72.7
72.7
0.78 0.80
GPT-4o
62.7
63.6
91.7
95.5
74.5
76.3
45.5
45.5
68.6
70.5
0.75 0.79
GPT-4o-mini
61.9
62.1
75.2
81.8
67.9
70.6
53.7
54.5
64.5
65.9
0.63 0.70
Claud-3-Open
60.7 63.6
58.7
63.6
59.7
63.6
62.0
63.6
60.3
63.6
0.52 0.57
Claud-3.5 3.5-Sonnet
59.1 61.5
66.9
72.7
62.8
66.7
53.7
54.5
60.3
63.6
0.56 0.62
Llama-3.0-70B
64.3
64.6
83.5
90.9
72.7
75.5
53.7
50.0
68.6
70.5
0.71 0.76
Llama-3.0-8B
44.5 46.2
50.4 54.5
47.3
50.0
37.2
36.4
43.8
45.5
0.34 0.38
33

---

Published as a conference paper at ICLR 2025
E.4 DETAILED EVALUATION ON PREDICATE CATEGORIES.
Table 16 presents comprehensive scores (%) for different models across various predicate categories. This data is visually represented in Figure 4 and Figure 12 Analysis of predicate categories reveals diverse model performance across semantic domains. GPT-4 and GPT-4o consistently excel in after score, achieving near-perfect scores in Attributes and 97 77.4-100% accuracy in Status categories. Spa- tiality shows the highest inter-model variability, with Claude-3-Opus leading in precision. Most models demonstrate significant improvement in the 'After' condition, particularly in Attributes. Alama-3.00-00 and Blaude-3.5-Sonme show moderate performance, while GPT-44-mini, despite lower overall scores, exhibits substantial improvement.
Table 16: Detailed evaluation on predicate categories.
(a) Attribute Scores
Precision
Recall
Fl
Model
Before
After
Before
After
Before
After
GPT-4
86.1
100.0
80.9
100.0
83.0
100.0
GPT-4o
86.0
100.0
80.9
100.0
83.0
100.0
86.1
100.0
80.1
99.2
82.6
99.5
Claud-3.5-Sonee
86.0
99.9
80.1
99.2
82.5
99.4
Claud-3-Opus
85.2
99.1
80.1
99.2
82.2
99.1
GPT-4o-miní
84.1
84.4
83.6
83.6
83.8
84.0
(b) Status Scores
Precision
Recall
F1
Model
Before
After
Before
After
Before
After
GPT-4
94.6
97.8
94.3
97.4
94.3
97.5
GPT-4o
92.1
100.0
94.3
97.4
93.0
98.6
Llama-3.0-70B
74.7
80.6
66.2
69.3
69.7
74.1
Claud-3.5-
79.6
83.1
74.2
77.3
76.7
80.0
Claud-3-Oput
89.8
92.9
90.5
93.6
90.0
93.1
GPT-4o-mini
92.8
95.1
93.6
93.7
93.1
94.3
(c) Spatiality Scores
Precision
Recall
Model
Before
After
Before
After
Before
After
GPT-4
85.8
87.3
87.3
87.8
86.4
87.6
GPT-40
61.8
80.5
85.8
87.8
71.7
83.8
Llama-3.0-70B
77.5
80.4
84.5
87.8
80.6
83.7
Claud-3.5-Sonnet
82.4
83.5
83.1
84.3
82.3
83.4
Claud-3-Opus
94.5
97.2
84.7
87.7
88.5
91.4
GPT-4o-mini
45.2
74.7
72.4
75.5
55.5
75.0
E.5 DETAILED STEP OF REAL-WORLD EXPERIENCE
In our real-world experiment, we implemented the instruction "Clean up the desk and complete the Tower of Hanoi." The experimental steps are illustrated in Figure [3 The objects involved this task consisted of three disposable paper cups, three Tower of Hanoi disks of varying sizes, three Tower of Hanoi poles, and a trash bin. subtask 1 comprised 13 steps, while subtask 2 consisted of 29 steps. The entire task was completed in total of 32 steps.
34

---

Published as a conference paper at ICLR 2025
During the experiment, encountered challenge step 8 when attempting to grasp the blue block of the Tower of Hamoi. We found that the existing rules were insufficient to resolve this issue. To address this problem, utilized the NESYC framework add additional rules specifically for grasping the Tower of Hanoi blocks. The details of these additional rules and their implementation can be found in Figure
Subtask Organize desk
Step
Step
Step
Subtask 2: Complete the Tower of Hanoi
Step 16
tee
21
Step 26
Step
Figure 13: Example of Real-world task
E.6 ANALYSIS OF CONTINUAL, LEARNING AND KNOWLEDGE GENERALIZATION
For further clarification, we conducted additional experiments to demonstrate the generalizability of our framework's s actionable knowledge a continual learning scenario. These experiments evaluate how effectively the actionable knowledge acquired through continual learning has been refined to generalize to future unseen tasks, as demonstrated in Kim et al. (2024): Lee et (2025). The results highlight our framework's ability adapt and refine its knowledge throughout the continual learning
35

---

Published as a conference paper at ICLR 2025
phases, showing performance that supports zero-shot generalization in dynamic and open- domain environments.
Let first provide summary of the continual learning scenario and results, along with the Ta- ble [7 and Table [8] for reference. The scenario consisted of phases, requiring approximately 50 and 40 steps (based oracle performance) for task completion, respectively. NESYC continually generalized actionable knowledge by leveraging accumulated experiences.
At each phase, evaluated the framework's 's performance both current and previous tasks to assess how well it retained existing knowledge while adapting to new environments. Throughout these scenarios, measured the rate of change between successive phases (4), based on the number of rules, and assessed the similarity of the refined rules to predefined expert-level rules using the F! score.
Table 17: Performance of continual learning scenario in ALFWorld
Metric
Phase 1
Phase
Phase 3
Phase
Phase S
Phase
Phase 7
Phase 8
SR
1.0
1.0
1.0
1.0
1.0
1,0
1.0
1.0
GC
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
NESYC
(%)
150
0
200
67
40
0
0
0
F1
28.6
28.6
50.0
73.7
90.9
90.9
90.9
90.9
Table 18: Performance continual learning scenario in RLBench
Metric
Phase
Phase
Phase 3
Phase
Phase
Phase
Phase
Phase
SR
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
NESYC
(%)
100
250
0
100
F1
30.3
63.4
63.4
63.4
80.6
80.6
96.3
96.3
Additionally, shown in the results in the Table [9la and Table 20 we evaluated how the improved actionable knowledge during the continual learning scenarios each benchmark was refined to generalize effectively to future unseen tasks. For this evaluation, we randomly selected 50 unseen tasks in a static setting and tested the intermediate rules (R) generated at specific phases. The re- sults indicate that rules from earlier phases consistently expanded task coverage, demonstrating the framework's s ability to adapt and improve over time. This refinement process effectively leverages accumulated experiences to generalize existing rules.
Table 19; Performance actionable knowledge from each phase on unseen tasks ALFWorld
R from Phase
Phase 3
Phase 4
Phase 5
Phase 8
Method
SR
GC
SR
GC
SR
GC
SR
GC
SR
GC
NESYC
38.3
50.4
53.2
67.9
74.5
77.3
87.2
90.8
91.5
94.3
Table 20: Performance actionable knowledge from each phase on unseen tasks RLBench
R from Phase
Phase 2
Phase
Phase
Phase 8
Method
SR
GC
SR
GC
SR
GC
SR
GC
SR
GC
NESyC
23.8
24.3
43.9
45.6
67.9
71.4
93.3
95.2
93.9
96.8
36

---

Published as a conference paper at ICLR 2025
Table 21: Comparative performance analysis of different LLM models with initial and improved results
LLM
SR
SR
GC
+ GC
Llama-3.1-8B
44.3
+ 56.8
49.6
- 60.0
Llama-3-8B
43.9
- 40.2
43.9
- 40.2
GPT-4o-mini
41.9
+ 78.77
44.7
79.3
Claude-3.0-Opus
50.7
7 76.7
53.6
78.6
Claude-3.5-Sonet
51.4
- 78.4
54.2
- 80.2
Llama-3-70B
58.8
+ 85.1
60.4
86.6
GPT-4o
64.2
- 90.2
67.0
90.5
GPT-4
69.6
89.2
73.3
- 89.8
E.7 EXTENDED LLM PERFORMANCE ANALYSIS
We conducted experiments similar to those presented in Table 3 using Llama-3. 1.tmm--3.1-88 3.B and Llama- 3.2-3B, with results shown in Table 21 For Llama-3.1-8B, while it outperformed Llama-3-8B with SR improvement from 44.3% to 56.8% and GC improvement from 49.6% 60.0%, its performance remained below practically meaningful levels compared to other models. For L.ama-332-3BB issues with the semantic parsing module prevented from functioning correctly, making it challenging to obtain meaningful experimental results and thus excluding it from Table
Specifically, Llama-3.2-3B frequently introduced syntax errors during the semantic parsing pro- cess, such as incorrectly mapping predicates (e.g., using 'objectl' instead of the appropriate 'object' type parameter), creating non-sten standaandard predicates like 'inside_room' instead of the correct 'inside' predicate, and generating non non-exise existent predicates such as 'variable type'. These errors disrupted subsequent processing and affected overall performance.
37


Reviews:
[{"review_id": "VZYFFBtRtD", "reviewer": "Reviewer_B2XV", "summary": "This paper addresses the challenge of generalizing knowledge for embodied task planning from prior experiences to novel scenarios and domains using a neural-symbolic continual learning approach. The proposed approach, NeSyC (Neural-Symbolic Continual Learner?), represents general knowledge and experience (hypotheses and grounded instances of action preconditions and effects, respectively) using Answer Set Programming and relies on existing ASP solvers to find plans that satisfy goals according to the best current hypothesis. Given a set of experiences (the effect of actions taken given the precondition state), an LLM is used to parse these experiences into positive and negative examples based on the action affordances. Given these positive and negative examples, an LLM is used to generate a set of hypotheses by first extracting background knowledge from the examples then generating hypotheses that satisfy batches of the examples. The hypotheses are further curated via an LLM-based implementation of \\theta-subsumption to include the most general of the generated hypotheses. Given these hypotheses, a symbolic hypothesis interpreter validates each with respect to all of the examples, providing feedback and a score for each hypothesis. This process of hypothesis generation and interpretation repeats for a maximum number of iterations, resulting in a final, most general hypothesis to be used as the general knowledge during planning.\n\t\nGiven a natural language instruction of a task and the current history of execution, an LLM is used to generate both goal states associated with the instruction and the current state of the world. A symbolic task planner uses the general knowledge from the iterative hypothesis generation-interpretation phase to find plans that reach the goal state. During execution, an error handler identifies after each action whether it succeeded or failed (or whether the effects match the general knowledge) and updates a working memory accordingly. In the case of failure, the current working memory is appended to the initial set of experiences, and the system re-enters the hypothesis generation-interpretation phase to update the general knowledge used for planning.\n\nThe proposed approach is evaluated through extensive experimentation on multiple standard benchmarks and compared against existing state-of-the-art methods, as well as demonstrated on a real robot in a complex tabletop scenario.", "questions": "I recommend strong accept at this time because of the strengths listed above.\n\nHere are a few questions that I would appreciate having answered:\n- What is the motivation for batching the examples during the hypothesis generation step? I imagine that fewer examples makes it easier to generate a good hypothesis, and batches could promote hypothesis diversity. Did you notice any interesting relationship between batch size and performance? Did you try including all examples?\n- Why is the \"satisfy\" feedback only provided once the max iterations are met (equation 6)? Is there no early termination condition for hypotheses that satisfy the examples?\n- The approach does not reach 100% on any of the metrics on any benchmarks; what are some common failures?\n- Llama-3-8B performed quite poorly, as reported in Table 3. Future work is proposed to use model distillation. Have you tried more recent small models, like LLama-3.1-8B or LLama-3.2-3B?\n\t\n\t\nHere are a few minor grammatical errors to fix:\n- Line 421 \"in Figure 3\" -> \"In Figure 3\"\n- line 422 \"alignment between the H1 score\" H1 should be HI\n- line 514 \"which constraints\" -> \"which constrains\"\n\nA small part of the appendix seems to have been mistakenly erased. See line 1201, which seems to start in the middle of a sentence.", "limitations": "", "rating": 10, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- The problem being addressed is of high significance to the embodied intelligence community.\n- The paper is well written and organized; the method is clearly and thoroughly explained and illustrated.\n- The contributions are clear, novel, and significant.\n- The experimental evaluation is extensive and thorough. The approach is evaluated across multiple diverse benchmarks with increasing difficulty within each benchmark (increasing dynamics). The methods against which the approach is compared are, to the best of my knowledge, the most appropriate state-of-the-art methods.\n- The performance is a significant improvement over other methods for dynamic environments.\n- Several relevant ablations are included that help answer questions about the relative performance of different components; an appropriate suite of state-of-the-art LLMs are included, showing that the hypothesis generation-interpretation phase is very successful across LLMs, with the exception of Llama-3-8B.", "weaknesses": "- The proposed approach performs worse than CLMASP (another neuro-symbolic approach) in static environments for 3 of the 4 benchmarks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w8kvYuNUZ2", "reviewer": "Reviewer_2Gcc", "summary": "This paper introduces NeSyC, where a large language model is given access to ILP like framework, specifically Answer Set Programming (ASP), alongside an ASP solver like clingo, where the agent iteratively comes up with hypotheses, attempts to solve tasks, and builds up a knowledge base about the world, aided by a symbolic tool. The authors evaluate their approach on open-domain environments, and show performance improvements over previous results.", "questions": "- In Line 354, the authors mention temperature = 0, why was greedy sampling used?\n- What was the observation space for the various environments examined? Were RGB images provided, like was done in e.g., LLM-Planner?\n- Why are the ReAct performance numbers in the static case so different from the ReAct paper (71 SR reported in their paper for ALFWorld vs 35.8 SR). Same for Reflexion (>0.9 SR reported in their paper on ALFWorld vs 39.0 SR). Is this purely from the difference in the LM used?\n- How many trials were used for Reflexion? How many trials were used with the proposed method?\n- In Figure 12, the Tower of Hanoi example, why did the agent decide to first put the rings in pegs in the wrong order, only to finish it correctly later?\n- In Appendix B.5, do the authors still use VoxPoser for low-level robot control?\n- The robotics tasks seem to rely on VoxPoser for low-level control. The reason for the performance of the proposed approach is “constrained by factors such as actuator range, grip force, and balance”, but the plans in Table 11 don’t have any such parameters. How is NeSyC influencing things like grip force? Does the model come up with things like “fragile” in Appendix D.3.1?\n- What do some qualitative planned trajectories on a task from all the baselines, and the proposed method look like? Would be great to include those.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The proposed method seems novel, and the combination of neural agents and symbolic tools in open-domain settings is of interest to the community at large.\n- The method outperforms previous methods on open-domain environments.\n- The proposed method seems to be able to handle high-dynamic settings where the world changes in unpredictable ways.", "weaknesses": "- The experimental results seem to have different numbers from the baselines, particularly in a comparable environment like ALFWorld (see questions about Reflexion and ReAct). The baseline methods in their own existing work did not use some of the environments used in this work. The numbers seem to be different enough to ask the question on performance, though it is possible this is not a weakness and the difference can be explained.\n- There is no comparison to the proposed approach for RLBench on existing work on that environment like ARP+ or RVT.\n- The writing can be improved to be clearer, as there are several complicated modules at play. It would be good to include qualitative examples of tasks, plans, and trajectories (see questions).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yfYesOcxMS", "reviewer": "Reviewer_HoAB", "summary": "This submission proposes NeSyC, a neuro-symbolic continual learner for open-domain embodied tasks. It combines LLM with symbolic tools to improve the generalization capability with limited experience. Extensive experiments are conducted on four benchmarks and on a Franka robot in the real world. The proposed framework shows competitive performance compared to multiple existing methods.", "questions": "See above.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Good presentation.\n- Extensive evaluation on multiple benchmarks and on a real robot.", "weaknesses": "- The overall system seems very complicated. It is unclear which part contributes to the overall performance. More detailed analysis and ablations are necessary to shed light into the framework design.\n- Tasks are too easy. How dynamics are the environment is also unclear (in paper it only mentioned low dynamics and high dynamics). \n- Since it's a planning framework, we'd expect it can handle long-horizon tasks. Considered tasks are instead short-horizon.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u3MQphhrFa", "reviewer": "Reviewer_CpeJ", "summary": "This work introduces a continual learning framework for dynamically restructuring neural-symbolic knowledge into rules via a Rule Reformulation process. It employs a contrastive generality improvement scheme based on Learning from Interpretations (LFI), a form of Inductive Logic Programming (ILP). The learned rules are utilized for interaction during Rule Application, supported by a memory-based monitoring mechanism within the Answer Set Programming (ASP) framework. Feedback from the Rule Application phase collects new data and periodically triggers Rule Reformulation to further refine the logical rules.\n\nThe experiments feature a diverse set of baselines, robustness evaluations, and ablation studies on key components, such as backbone VLMs and prompts. The results highlight the framework’s potential to seamlessly integrate low-level execution with logical reasoning and natural language instructions.", "questions": "### Questions Regarding $itermax$ and Rule Learning ($R$)\n\n- I am curious about the role of the fixed **$itermax$** in learning the rules $R$. Does setting a fixed $itermax$ always guarantee a reasonable system with appropriate rules? Could you provide results for different $itermax$ values to show their impact on performance and the resulting rules?\n- Additionally, I am particularly interested in the notion of **convergence** within the NeSYC system. It might be valuable to explore how convergence is defined in your case. How sensitive is the system to environmental changes—would significant changes disrupt convergence? Also, is it possible for the rules $R$ to exhibit a **cyclical pattern** rather than converging to a stable state?\n- On the topic of **continual learning**, I noticed that your system appears to stop growing its rules $R$ at some point (**$itermax$**), with subsequent testing based on a fixed set of rules. This seems different from my understanding of continual learning as a life-long process where rules evolve continuously based on new data. Could you conduct experiments where NeSYC and baseline agents keep learning and testing in various settings without a fixed $itermax$? It would be interesting to compare their performance, especially in terms of how quickly each system improves its success rate across tasks.\n\n### Questions Regarding Experiments\n\n- I have some concerns about the \"Step\" evaluation in your experiments. Although this evaluation follows prior work, I worry about potential overfitting. Achieving tasks does not always require strict adherence to ground truth; instead, achieving key milestones or sub-goals should suffice. This is particularly true for manipulation tasks—for example, a robot’s trajectory does not need to exactly match the demonstration. In Table 1, I noticed that both NeSYC and other systems show high \"Step\" evaluation scores, indicating close similarity to demonstrations. This raises questions about these systems' generalization capabilities in new environments. Could you clarify how NeSYC ensures generalization beyond following exact demonstrations? Are there additional experiments or insights that address this concern?\n- Similarly, in the **Ablation Study** (Section 4.3), could you provide more insight into why simplified planning improves Goal Completion (GC) and \"Step\" evaluation? \n\n- For the **Robustness to Experience Incompleteness** experiment in Section 4.3, could you elaborate on what you mean by *Noisy* and *Imperfect* experiences? What preprocessing steps did you apply to the datasets or system to create these noisy or imperfect conditions?\n\n- In the **comparison of different dynamics predicates** (Section 4.3), could you clarify what your **F1 score** represents? How was the evaluation data collected to calculate the F1 score? What insights are you aiming to provide with the changes in F1 score, especially in relation to the characteristics of the predicates?\n\n- Lastly, I would appreciate more detailed observations from your **real-world experiments**. Much of the current description is qualitative. Were these real-world scenarios tested multiple times, and did you observe recurring patterns? It would also be helpful to quantify these observations with metrics, rather than relying solely on descriptive results.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The work provides a solid introduction to the foundational neural-symbolic knowledge required for its framework, including Learning from Interpretations (LFI) within Inductive Logic Programming (ILP) and Answer Set Programming (ASP), as part of the background preliminaries.\n\nThe explanations effectively convey the system's core ideas, offering essential details about key functions, such as the use of LFI in Rule Reformulation and the mechanisms for memory modification.\n\nComprehensive information about NeSYC is provided in the Appendix, along with examples of neural-symbolic designs, making it easier for readers to understand the underlying processes.\n\nThe experiments are extensive, covering diverse environments and settings, demonstrating NeSYC’s capability and robustness.", "weaknesses": "Some aspects of the method remain unclear. For example, what exactly is the role of $\\mathcal{I}_d$, and how does it relate to $\\mathcal{G}_d$ as introduced? A reorganization of Section 2.3 may be necessary, as it forms the foundation of the problem setup.\n\nAdditionally, while the authors seem intent on making NeSYC capable of handling a broad range of situations, many of the symbolic formulations explaining the method feel too abstract. This is not a major issue, but it would be helpful if the authors provided more concrete examples of how these symbols are implemented within the main body of the method section, rather than placing all such details in the appendix.\n\nThe left portion of Figure 1 is also confusing. It is not adequately addressed in the annotations, leaving the reader unsure of its significance.\n\nFinally, certain experimental settings could benefit from clearer descriptions. For example, in the Dataset section (Section 4.1), the format of the expert-level episodic experience data is ambiguous. Is the data presented in natural language, code, or another format? Providing compact and clear explanations of these settings would make it easier for others to understand the data processing steps necessary for reproducing a similar system.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)