NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domains

Authors: Wonje Choi, Jinwoo Park, Sanghyun Ahn, Daehee Lee, Honguk Woo
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords: Embodied AI, Neuro-symbolic AI

Abstract:
We explore neuro-symbolic approaches to generalize actionable knowledge, enabling embodied agents to tackle complex tasks more effectively in open-domain environments. A key challenge for embodied agents is the generalization of knowledge across diverse environments and situations, as limited experiences often confine them to their prior knowledge. To address this issue, we introduce a novel framework, NeSyC, a neuro-symbolic continual learner that emulates the hypothetico-deductive model by continually formulating and validating knowledge from limited experiences through the combined use of Large Language Models (LLMs) and symbolic tools. Specifically, we devise a contrastive generality improvement scheme within NeSyC, which iteratively generates hypotheses using LLMs and conducts contrastive validation via symbolic tools. This scheme reinforces the justification for admissible actions while minimizing the inference of inadmissible ones. Additionally, we incorporate a memory-based monitoring scheme that efficiently detects action errors and triggers the knowledge refinement process across domains. Experiments conducted on diverse embodied task benchmarks—including ALFWorld, VirtualHome, Minecraft, RLBench, and a real-world robotic scenario—demonstrate that NeSyC is highly effective in solving complex embodied tasks across a range of open-domain environments.

Decision: accept (poster)
Reviewer scores: 10, 5, 5, 8

Reviews and discussion:
<thread>
creation date:
1730295173008

thread_type:
official_review

<thread_object>
creator: reviewer_cpej

<content>
summary:
This work introduces a continual learning framework for dynamically restructuring neural-symbolic knowledge into rules via a Rule Reformulation process. It employs a contrastive generality improvement scheme based on Learning from Interpretations (LFI), a form of Inductive Logic Programming (ILP). The learned rules are utilized for interaction during Rule Application, supported by a memory-based monitoring mechanism within the Answer Set Programming (ASP) framework. Feedback from the Rule Application phase collects new data and periodically triggers Rule Reformulation to further refine the logical rules.

The experiments feature a diverse set of baselines, robustness evaluations, and ablation studies on key components, such as backbone VLMs and prompts. The results highlight the framework’s potential to seamlessly integrate low-level execution with logical reasoning and natural language instructions.

strengths:
The work provides a solid introduction to the foundational neural-symbolic knowledge required for its framework, including Learning from Interpretations (LFI) within Inductive Logic Programming (ILP) and Answer Set Programming (ASP), as part of the background preliminaries.

The explanations effectively convey the system's core ideas, offering essential details about key functions, such as the use of LFI in Rule Reformulation and the mechanisms for memory modification.

Comprehensive information about NeSYC is provided in the Appendix, along with examples of neural-symbolic designs, making it easier for readers to understand the underlying processes.

The experiments are extensive, covering diverse environments and settings, demonstrating NeSYC’s capability and robustness.

weaknesses:
Some aspects of the method remain unclear. For example, what exactly is the role of $\mathcal{I}_d$, and how does it relate to $\mathcal{G}_d$ as introduced? A reorganization of Section 2.3 may be necessary, as it forms the foundation of the problem setup.

Additionally, while the authors seem intent on making NeSYC capable of handling a broad range of situations, many of the symbolic formulations explaining the method feel too abstract. This is not a major issue, but it would be helpful if the authors provided more concrete examples of how these symbols are implemented within the main body of the method section, rather than placing all such details in the appendix.

The left portion of Figure 1 is also confusing. It is not adequately addressed in the annotations, leaving the reader unsure of its significance.

Finally, certain experimental settings could benefit from clearer descriptions. For example, in the Dataset section (Section 4.1), the format of the expert-level episodic experience data is ambiguous. Is the data presented in natural language, code, or another format? Providing compact and clear explanations of these settings would make it easier for others to understand the data processing steps necessary for reproducing a similar system.

questions:
### Questions Regarding $itermax$ and Rule Learning ($R$)

- I am curious about the role of the fixed **$itermax$** in learning the rules $R$. Does setting a fixed $itermax$ always guarantee a reasonable system with appropriate rules? Could you provide results for different $itermax$ values to show their impact on performance and the resulting rules?
- Additionally, I am particularly interested in the notion of **convergence** within the NeSYC system. It might be valuable to explore how convergence is defined in your case. How sensitive is the system to environmental changes—would significant changes disrupt convergence? Also, is it possible for the rules $R$ to exhibit a **cyclical pattern** rather than converging to a stable state?
- On the topic of **continual learning**, I noticed that your system appears to stop growing its rules $R$ at some point (**$itermax$**), with subsequent testing based on a fixed set of rules. This seems different from my understanding of continual learning as a life-long process where rules evolve continuously based on new data. Could you conduct experiments where NeSYC and baseline agents keep learning and testing in various settings without a fixed $itermax$? It would be interesting to compare their performance, especially in terms of how quickly each system improves its success rate across tasks.

### Questions Regarding Experiments

- I have some concerns about the "Step" evaluation in your experiments. Although this evaluation follows prior work, I worry about potential overfitting. Achieving tasks does not always require strict adherence to ground truth; instead, achieving key milestones or sub-goals should suffice. This is particularly true for manipulation tasks—for example, a robot’s trajectory does not need to exactly match the demonstration. In Table 1, I noticed that both NeSYC and other systems show high "Step" evaluation scores, indicating close similarity to demonstrations. This raises questions about these systems' generalization capabilities in new environments. Could you clarify how NeSYC ensures generalization beyond following exact demonstrations? Are there additional experiments or insights that address this concern?
- Similarly, in the **Ablation Study** (Section 4.3), could you provide more insight into why simplified planning improves Goal Completion (GC) and "Step" evaluation? 

- For the **Robustness to Experience Incompleteness** experiment in Section 4.3, could you elaborate on what you mean by *Noisy* and *Imperfect* experiences? What preprocessing steps did you apply to the datasets or system to create these noisy or imperfect conditions?

- In the **comparison of different dynamics predicates** (Section 4.3), could you clarify what your **F1 score** represents? How was the evaluation data collected to calculate the F1 score? What insights are you aiming to provide with the changes in F1 score, especially in relation to the characteristics of the predicates?

- Lastly, I would appreciate more detailed observations from your **real-world experiments**. Much of the current description is qualitative. Were these real-world scenarios tested multiple times, and did you observe recurring patterns? It would also be helpful to quantify these observations with metrics, rather than relying solely on descriptive results.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730617865749

thread_type:
official_review

<thread_object>
creator: reviewer_hoab

<content>
summary:
This submission proposes NeSyC, a neuro-symbolic continual learner for open-domain embodied tasks. It combines LLM with symbolic tools to improve the generalization capability with limited experience. Extensive experiments are conducted on four benchmarks and on a Franka robot in the real world. The proposed framework shows competitive performance compared to multiple existing methods.

strengths:
- Good presentation.
- Extensive evaluation on multiple benchmarks and on a real robot.

weaknesses:
- The overall system seems very complicated. It is unclear which part contributes to the overall performance. More detailed analysis and ablations are necessary to shed light into the framework design.
- Tasks are too easy. How dynamics are the environment is also unclear (in paper it only mentioned low dynamics and high dynamics). 
- Since it's a planning framework, we'd expect it can handle long-horizon tasks. Considered tasks are instead short-horizon.

questions:
See above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730708386897

thread_type:
official_review

<thread_object>
creator: reviewer_2gcc

<content>
summary:
This paper introduces NeSyC, where a large language model is given access to ILP like framework, specifically Answer Set Programming (ASP), alongside an ASP solver like clingo, where the agent iteratively comes up with hypotheses, attempts to solve tasks, and builds up a knowledge base about the world, aided by a symbolic tool. The authors evaluate their approach on open-domain environments, and show performance improvements over previous results.

strengths:
- The proposed method seems novel, and the combination of neural agents and symbolic tools in open-domain settings is of interest to the community at large.
- The method outperforms previous methods on open-domain environments.
- The proposed method seems to be able to handle high-dynamic settings where the world changes in unpredictable ways.

weaknesses:
- The experimental results seem to have different numbers from the baselines, particularly in a comparable environment like ALFWorld (see questions about Reflexion and ReAct). The baseline methods in their own existing work did not use some of the environments used in this work. The numbers seem to be different enough to ask the question on performance, though it is possible this is not a weakness and the difference can be explained.
- There is no comparison to the proposed approach for RLBench on existing work on that environment like ARP+ or RVT.
- The writing can be improved to be clearer, as there are several complicated modules at play. It would be good to include qualitative examples of tasks, plans, and trajectories (see questions).

questions:
- In Line 354, the authors mention temperature = 0, why was greedy sampling used?
- What was the observation space for the various environments examined? Were RGB images provided, like was done in e.g., LLM-Planner?
- Why are the ReAct performance numbers in the static case so different from the ReAct paper (71 SR reported in their paper for ALFWorld vs 35.8 SR). Same for Reflexion (>0.9 SR reported in their paper on ALFWorld vs 39.0 SR). Is this purely from the difference in the LM used?
- How many trials were used for Reflexion? How many trials were used with the proposed method?
- In Figure 12, the Tower of Hanoi example, why did the agent decide to first put the rings in pegs in the wrong order, only to finish it correctly later?
- In Appendix B.5, do the authors still use VoxPoser for low-level robot control?
- The robotics tasks seem to rely on VoxPoser for low-level control. The reason for the performance of the proposed approach is “constrained by factors such as actuator range, grip force, and balance”, but the plans in Table 11 don’t have any such parameters. How is NeSyC influencing things like grip force? Does the model come up with things like “fragile” in Appendix D.3.1?
- What do some qualitative planned trajectories on a task from all the baselines, and the proposed method look like? Would be great to include those.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730938405839

thread_type:
official_review

<thread_object>
creator: reviewer_b2xv

<content>
summary:
This paper addresses the challenge of generalizing knowledge for embodied task planning from prior experiences to novel scenarios and domains using a neural-symbolic continual learning approach. The proposed approach, NeSyC (Neural-Symbolic Continual Learner?), represents general knowledge and experience (hypotheses and grounded instances of action preconditions and effects, respectively) using Answer Set Programming and relies on existing ASP solvers to find plans that satisfy goals according to the best current hypothesis. Given a set of experiences (the effect of actions taken given the precondition state), an LLM is used to parse these experiences into positive and negative examples based on the action affordances. Given these positive and negative examples, an LLM is used to generate a set of hypotheses by first extracting background knowledge from the examples then generating hypotheses that satisfy batches of the examples. The hypotheses are further curated via an LLM-based implementation of \theta-subsumption to include the most general of the generated hypotheses. Given these hypotheses, a symbolic hypothesis interpreter validates each with respect to all of the examples, providing feedback and a score for each hypothesis. This process of hypothesis generation and interpretation repeats for a maximum number of iterations, resulting in a final, most general hypothesis to be used as the general knowledge during planning.
	
Given a natural language instruction of a task and the current history of execution, an LLM is used to generate both goal states associated with the instruction and the current state of the world. A symbolic task planner uses the general knowledge from the iterative hypothesis generation-interpretation phase to find plans that reach the goal state. During execution, an error handler identifies after each action whether it succeeded or failed (or whether the effects match the general knowledge) and updates a working memory accordingly. In the case of failure, the current working memory is appended to the initial set of experiences, and the system re-enters the hypothesis generation-interpretation phase to update the general knowledge used for planning.

The proposed approach is evaluated through extensive experimentation on multiple standard benchmarks and compared against existing state-of-the-art methods, as well as demonstrated on a real robot in a complex tabletop scenario.

strengths:
- The problem being addressed is of high significance to the embodied intelligence community.
- The paper is well written and organized; the method is clearly and thoroughly explained and illustrated.
- The contributions are clear, novel, and significant.
- The experimental evaluation is extensive and thorough. The approach is evaluated across multiple diverse benchmarks with increasing difficulty within each benchmark (increasing dynamics). The methods against which the approach is compared are, to the best of my knowledge, the most appropriate state-of-the-art methods.
- The performance is a significant improvement over other methods for dynamic environments.
- Several relevant ablations are included that help answer questions about the relative performance of different components; an appropriate suite of state-of-the-art LLMs are included, showing that the hypothesis generation-interpretation phase is very successful across LLMs, with the exception of Llama-3-8B.

weaknesses:
- The proposed approach performs worse than CLMASP (another neuro-symbolic approach) in static environments for 3 of the 4 benchmarks.

questions:
I recommend strong accept at this time because of the strengths listed above.

Here are a few questions that I would appreciate having answered:
- What is the motivation for batching the examples during the hypothesis generation step? I imagine that fewer examples makes it easier to generate a good hypothesis, and batches could promote hypothesis diversity. Did you notice any interesting relationship between batch size and performance? Did you try including all examples?
- Why is the "satisfy" feedback only provided once the max iterations are met (equation 6)? Is there no early termination condition for hypotheses that satisfy the examples?
- The approach does not reach 100% on any of the metrics on any benchmarks; what are some common failures?
- Llama-3-8B performed quite poorly, as reported in Table 3. Future work is proposed to use model distillation. Have you tried more recent small models, like LLama-3.1-8B or LLama-3.2-3B?
	
	
Here are a few minor grammatical errors to fix:
- Line 421 "in Figure 3" -> "In Figure 3"
- line 422 "alignment between the H1 score" H1 should be HI
- line 514 "which constraints" -> "which constrains"

A small part of the appendix seems to have been mistakenly erased. See line 1201, which seems to start in the middle of a sentence.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732042949355

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response to All Reviewers

comment:
We thank all reviewers for their constructive feedback. Based on the reviewers' comments and suggestions, we have made the following revisions to our paper.

Main Manuscript Revisions:

* Fixed typos and clarified notation (Reviewer B2XV):
    * Line 421: "in Figure 3" → "In Figure 3"
    * Line 422: "H1 score" → "HI score"
    * Line 514: "which constraints" → "which constrains"
* Clarified notation distinctions between sets ($\mathcal{I}_d$, $\mathcal{G}_d$) and elements ($i_d$, $g_d$) in Eq. (3) (Reviewer CpeJ)
* Enhanced caption in Figure 1 to better reflect the process details (Reviewer CpeJ, 2Gcc)
* Modified examples in General Knowledge $R$ in Figure 2 (Reviewer CpeJ)
* Revised Figure 1 and introduction section to better illustrate our neuro-symbolic framework (Reviewer 2Gcc)


Appendix Enhancements:

* Added a comprehensive analysis of experimental results in Appendix E.1 (Reviewer 2Gcc)
* Updated Appendix B.6 with detailed dataset descriptions (Reviewer CpeJ)
* Included examples and detailed setup for real-world scenarios in Appendix B.5 (Reviewer 2Gcc)
* Updated Appendix B.7 with performance differences in open-domain settings (Reviewers 2Gcc, HoAB)
* Added Figure 11 with sample trajectories from comparative models (Reviewer 2Gcc)
* Updated explanation of real-world object detection module in Appendix B.5 (Reviewer 2Gcc)

We have carefully addressed all major concerns raised by the reviewers and made detailed revisions. We hope these changes have enhanced the clarity and quality of our paper. We will continue to make updates as necessary throughout the discussion period.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732792409594

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response to All Reviewers

comment:
We sincerely appreciate the constructive feedback provided by all reviewers—Reviewer B2XV, Reviewer 2Gcc, Reviewer HoAB, and Reviewer CpeJ. In response to the reviewers' comments and feedback, as well as through our own efforts, we have finalized the revisions to our paper and uploaded the final version. We hope that these updates will be carefully considered by the reviewers and effectively address any remaining concerns. The detailed list of revisions follows below:


**Main Manuscript Revisions**
 * **Correction of Typos (Reviewer B2XV)**
    * Corrected 'in Figure 3' to 'In Figure 3' in Section 4.
    * Corrected 'H1 score' to 'HI score' in Section 4.
    * Corrected 'which constraints' to 'which constrains' in Section 5.
 * **Mathematical Clarity (Reviewer CpeJ)**
    * Clarified the distinction between sets ($\mathcal{I}_d, \mathcal{G}_d$) and elements ($i_d, g_d$) in Eq. (3) of Section 2.3.
    * Revised the notation for instructions and goal states in Section 3.3 to align with Eq. (3) in Section 2.3.
    * Revised the notation for instructions and goal states in Algorithm 2 to ensure consistency with both Eq. (3) in Section 2.3 and the description in Section 3.3.
 * **Figure Improvements (Reviewers 2Gcc, CpeJ)**
    * Simplified Figure 1 to clearly present the motivation and problem addressed by our framework.
    * Revised the caption of Figure 1 to clearly convey the concept and objective of our framework as discussed in Section 1.
    * Simplified the visual representation of Figure 2 to make it more intuitive and help readers better understand the key components of our framework.
 * **Readability Improvements (Reviewer 2Gcc)**
    * Refined Section 1 by reducing paraphrasing of key terms, enhancing consistency, and clarifying ambiguous expressions for improved clarity.
    * Added citations in Section 1 to support the argument that conventional neuro-symbolic approaches face challenges in open-domain environments.
    * Revised the text in Section 1 to emphasize the practical challenges of generalizing actionable knowledge in open-domain environments.
    * Revised terminology in Section 3 for precision and clarity, replacing 'general knowledge' with 'generalized knowledge' and 'most general hypothesis' with 'optimized hypothesis'.
    * Refined the explanation of the process for inducing and validating hypotheses in Section 3.2 to improve clarity.
    * Revised the explanation of components managing environment interactions and processes for handling experiences in Section 3.3 for better clarity.
    * Adjusted experiment descriptions in Section 4 to align with the revised terminology introduced in Section 3.
 * **Simplification of Framework Explanations (Reviewers 2Gcc, HoAB)**
    * Revised the framework motivation, technical overview, and contributions in Section 1 to ensure cohesive connections.
    * Revised and Expanded the explanation of ILP and ASP in Section 2 to highlight their relevance to our framework, ensuring smoother integration with Section 3.
    * Updated the descriptions of the Rule Reformulation and Rule Application phases in Section 3.1 to clearly articulate their objectives and techniques.

**Appendix Enhancements (Reviewers B2XV, 2Gcc, HoAB, CpeJ)**
* Enhanced Appendix B.5 with detailed real-world scenario examples and a comprehensive explanation of the object detection module.
* Updated Appendix B.6 with detailed dataset documentation, including data creation procedures, preprocessing methods, and illustrative examples of how the datasets were constructed and utilized in our experiments.
* Updated Appendix B.7 with an explanation of why baseline methods show lower performance in open-domain settings compared to their reported results in the original papers, clarifying the impact of environmental differences on performance metrics.
* Added rollout trajectories in Appendix E.1 to provide qualitative comparisons between baseline methods and our framework, demonstrating the effectiveness of our approach in handling sequential tasks and decision-making processes.
* Added experiments with smaller LLMs, such as LLama-3.1-8B and LLama-3.2-3B, in Appendix E.7, showing that their performance is still not practically meaningful.
* Added experiments on continual learning scenarios in ALFWorld and RLBench in Appendix E.6, showing robust task performance superior to Reflexion and stable generalization results with respect to rule convergence.

---

We understand that the discussion period is ongoing, and we welcome any responses, additional feedback, or questions regarding our response. We will continue to provide updates and actively participate as necessary throughout the discussion period. Once again, we sincerely thank all reviewers for their effort and dedication.
</content>
</thread_object>

</thread>
<thread>
creation date:
1740719973026

thread_type:
public_comment

<thread_object>
creator: ~wonje_choi2

<content>
title:
Minor Title Correction Request

comment:
Dear ACs and Reviewers,

We would like to begin by expressing our sincere gratitude for your positive evaluation of our paper.

We hope this message finds you well. We are writing regarding our paper titled:

"NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks in Open Domains.”

We noticed a small typographical error in the title: specifically, the word “in” should be capitalized. Would it be possible to update the title to the following?

"NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domains”

We fully understand that the title can only be changed at the request of the reviewers or the area chair, and that it must not significantly alter the scope of the paper. This proposed change is purely typographical and does not affect the paper’s content or scope in any way.

Thank you for your time and consideration. We look forward to hearing from you.

Sincerely,

The Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
**Summary**: This paper introduces NeSyC, a neuro-symbolic continual learner designed to enable embodied agents to generalize actionable knowledge across diverse open-domain tasks. Inspired by the hypothetico-deductive model, NeSyC employs a contrastive generality improvement scheme to iteratively generate and validate hypotheses using LLMs and symbolic tools. Additionally, it incorporates a memory-based monitoring scheme to detect action errors and trigger knowledge refinement. The framework is evaluated on benchmarks like ALFWorld, VirtualHome, Minecraft, RLBench, and a real-world robotic tabletop scenario. NeSyC demonstrates strong performance across static, low dynamic, and high dynamic environments, outperforming baselines like AutoGen, ReAct, and CLMASP in open-domain settings.

**Strengths**:
- High-importance problem and clear, novel, and substantial contributions: The reviewers agree that NeSyC presents an innovative neuro-symbolic approach that combines LLM-based hypothesis generation with symbolic validation. This iterative process enables the agent to continually refine its knowledge, which is particularly valuable in open-domain, dynamic environments.
- Strong empirical performance: NeSyC demonstrates significant improvements over state-of-the-art baselines, especially in environments with high dynamics where existing methods struggle. Reviewer `B2XV` highlights the thorough and extensive evaluation on multiple benchmarks, including both simulation and real-world tasks. In fact, the reviewer gives the paper a 10, which I've personally never seen before at ICLR.
- Comprehensive evaluation: The reviewers commend the extensive experiments across ALFWorld, VirtualHome, Minecraft, RLBench, and real-world tasks. Additionally, Reviewer `B2XV` notes that the ablation studies provide valuable insights into the contributions of different components.
- Solid writing: (with some exceptions, read below) Reviewers generally praise the writing and organization, and clarity, and exposition of the paper.

**Weaknesses**:
- Inconsistencies in baseline results: Reviewer `2Gcc` raises concerns about discrepancies in baseline performance for methods like ReAct and Reflexion in ALFWorld. The authors attribute these differences to variations in the LLMs and environments but could improve clarity around this point. Additionally, comparisons to other RLBench methods, such as ARP+ or RVT, are missing, although the authors explain (convincingly) why these methods are not directly comparable to NeSyC. This discussion should be included in the related work section of the paper.
- Performance in static environments: Reviewer `B2XV` points out that NeSyC underperforms compared to CLMASP in static environments. The authors clarify that this is due to CLMASP’s reliance on predefined expert knowledge, which is less applicable in dynamic tasks. While the explanation is reasonable, the comparison highlights a trade-off between generalization and static task optimization, which should be included in the discussion section of the paper.
- Additional discussion of failure cases: While NeSyC shows strong performance, Reviewer `B2XV` and Reviewer `2Gcc` suggest adding a more detailed analysis of common failure cases. The authors provide some discussion of errors related to LLM-based semantic parsing and symbolic representation, but further exploration of these limitations would strengthen the work.
- Writing clarity: Despite the overall solid writing, multiple reviewers point out things that could be clarified or that read as convoluted, particularly given the complexity of the proposed framework. The authors address this by refining several sections and figures during the rebuttal, but further qualitative examples of planned trajectories and error handling would enhance clarity.

**Recommendation**: The paper introduces an innovative neuro-symbolic approach for open-domain embodied tasks with clear technical contributions and strong experimental validation. While there are concerns about performance in static environments, baseline discrepancies, and writing clarity, the authors have addressed these points adequately. The reviewers agree on the significance of the work, particularly its robustness in dynamic and open-domain settings. Finally, the paper scored a 10, which is the only such high grade that I've ever seen as an AC. As such, I vote for Accept (spotlight).