NeSyPr: Neurosymbolic Proceduralization For Efficient Embodied Reasoning
Authors: Wonje Choi
Jooyoung Kim
Honguk Woo
Venue: NeurIPS 2025 poster ()
Keywords: Neurosymbolic
Large Language Model
Embodied Agent
Primary area: reinforcement_learning
TL;DR: NeSyPr: Neurosymbolic Proceduralization For Efficient Embodied Reasoning

Abstract: We address the challenge of adopting language models (LMs) for embodied tasks in dynamic environments, where online access to large-scale inference engines or symbolic planners is constrained due to latency, connectivity, and resource limitations. To this end, we present NeSyPr, a novel embodied reasoning framework that compiles knowledge via neurosymbolic proceduralization, thereby equipping LM-based agents with structured, adaptive, and timely reasoning capabilities. In NeSyPr, task-specific plans are first explicitly generated by a symbolic tool leveraging its declarative knowledge. These plans are then transformed into composable procedural representations  that encode the plans' implicit production rules, enabling the resulting composed procedures to be seamlessly integrated into the LM's inference process. This neurosymbolic proceduralization abstracts and generalizes multi-step symbolic structured path-finding and reasoning into single-step LM inference, akin to human knowledge compilation. It supports efficient test-time inference without relying on external symbolic guidance, making it well suited for deployment in latency-sensitive and resource-constrained physical systems. We evaluate NeSyPr on the embodied benchmarks PDDLGym, VirtualHome, and ALFWorld, demonstrating its efficient reasoning capabilities over large-scale reasoning models and a symbolic planner, while using more compact LMs.

NeSyPr: Neurosymbolic Proceduralization For Efficient Embodied Reasoning
Wonje Choi, Jooyoung Kim, Honguk Woo"
Department of Computer Science and Engineering, Sungkyunkwan University {wjchoi1995, onsaemiro, hwoo) edu
Abstract
We address the challenge of adopting language models (LMs) for embodied tasks in dynamic environments, where online access to large-scale inference engines or symbolic planners is constrained due latency, connectivity, and resource limita- tions. To this end, we present NESYPR, a novel embodied reasoning framework that compiles knowledge via neurosymbolic proceduralization, thereby equipping LM-based agents with structured, adaptive, and timely reasoning capabilities. In NESYPR, task-specific plans first explicitly generated by symbolic tool lever- aging its declarative knowledge. These plans are then transformed into composable procedural representations that encode the plans' implicit production rules, en- abling the resulting composed procedures to be seamlessly integrated into the LM's inference process. This neurosymbolic proceduralization abstracts and generalizes multi-step symbolic structured path-finding and reasoning into single-step LM inference, akin to human knowledge compilation. It supports efficient test-time inference without relying on external symbolic guidance, making it well suited for deployment in latency-sensitive and resource- conureeennn physical systems. We evaluate NESYPR on the embodied benchmarks PDDLGym, VirtualHome, and ALFWorld, demonstrating its efficient reasoning capabilities over large-scale reasoning models and symbolic planner, while using more compact LMs.
Introduction
Recent works such as Inner Monologue (T. SayCan (2). and LLM-Planner (3] have demonstrated the potential of large-scale language models (LMs) to control embodied agents on complex tasks in dynamic environments. Yet, the inherent limitations of autoregressive inference (e.g., shallow plan- ning, inefficient context reuse, and lack of structure) have led researchers to explore more structured reasoning approaches. Three primary directions have emerged: (i) agentic frameworks that support autonomous planning and multi-step reasoning 5 6 7 (ii) neurosymbolic methods that integrate LMS with external symbolic reasoning frameworks [8] 10 IT. and (iii) augmented LM approaches
that incorporate memory modules [12] [3] 14 15]. Despite these advances, existing approaches dynamic environments. Agentic
still face significant limitations, particularly in frameworks require iterative inference of large-scale models, leading to substantial computational overhead [16]. Neurosymbolic methods, while using predefined rules systematically search for ac- curate reasoning paths, often suffer from increased task-solving time and diminished effectiveness in dynamic settings unless the solver and its rule base continuously updated reflect environmental LM approaches typically expand the context window to encode more information, yet they rarely retain the procedural structure needed for complex embodied tasks.
changes [17]. ddemory-augment
These limitations emphasize the need for a reasoning framework tailored to LM-based embodied agents, capable of structured and adaptive decision-mann decision-making making under time and resource constraints and
Honguk Woo is the corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

without online symbolic assistance. To this end, we draw inspiration from the Adaptive Control of Thought (ACT) theory [18), which models skill acquisition as process of knowledge compila- tion-the conversion of declarative knowledge into procedural form via repeated practice. Declarative knowledge held in declarative memory as chunks that encode explicit facts such as propositions or problem states. By contrast, procedural knowledge consists of condition- action rules, stored in procedural memory and triggered automatically as cognitive procedures. A central mechanism in ACT, known proceduralization, hinges on the interplay of three memory systems. Declarative memory supplies the relevant chunk into working memory, which temporarily holds the current problem state during interaction with the environment. Through repeated use, the factual patterns gradually compiled into production rules stored in procedural memory. Once compiled, production rules fire whenever their conditions match the contents of working memory, enabling actions without further reference to declarative memory. By bypassing declarative retrieval, proceduralization reduces cognitive load and supports faster, more automatic, and less error-prone execution 19 20 21 22
Building on the ACT theory, we present NESYPR, novel embodied reasoning framework based on neurosymbolic proceduralization. It performs knowledge compilation by abstracting and generalizing multi-step symbolic aath-finding and reasoning into a single-step LM inference, thereby enabling LM- based agents to perform embodied reasoning efficiently, without relying on large-scale inference or online access to external symbolic tools. In NESYPR, task-specific plans are first explicitly generated by symbolic tool leveraging its declarative knowledge. These plans composable procedural representations that encode the plans' implicit production rules, enabling the resulting composed procedures to be scamlessly integrated into the LM's s inference process. This proceduralization proceeds in two phases: i) compositional NeSy procedure learning and ii) NeSy procedure contrastive planning. During training phase i), an agent learns to encode production rules into a vector-quantized procedural memory, in which the resulting vectors are structured to be composable for task-specific plan generation. At test phase ii), without access to symbolic tools, the agent continues adaptive reasoning by generating plans augmented with procedural memory, while contrastively reconstructing their internal representations based on environmental feedback.
then transformed into
We evaluate NESYPR on PDDLGym 23 VirtualHome [24), and ALFWorld (25.. where inputs
include observation, goal, and domain knowledge that specified symbolically. For structured reasoning, NESYPR achieves a 46.7% higher task success rate than DeepSeek-R1-
(26). a distilled 70B-scale reasoning model, while operating with a 70 times smaller LM (as shown in Table For adaptive reasoning, it attains a 62. 1% higher success rate on unseen tasks with dynamic conditions than the symbolic planner (27) (in Table 3). For timely reasoning, it reduces inference latency by more than 90.0% compared to BoT (28). a large-scale inference baseline, while achieving 36.0% improvement in task success rate (in Table 3). These results demonstrate that NESYPR endows LM-based agents with strong structured, adaptive, and timely reasoning capabilities.
Our contributions summarized as: (1) We present NESYPR, the neurosymbolic proceduralization- based reasoning framework inspired by ACT, which compiles multi-step symbolic reasoning into single-step LM inference, eliminating the need for online symbolic planners in embodied tasks. (2) We develop compositional NeSy procedure learning, which encodes production rules into a vector- quantized procedural memory whose vectors be compositionally combined to generate task- specific plans. (3) We implement NeSy procedure contrastive planning, which adaptively generates plans by contrastively reconstructing task-specific procedures from stored procedures labeled with environmental feedback. (4) We show the effectiveness and efficiency of NES NESYPR through extensive evaluations including 3 embodied benchmarks and 9 experimental scenarios, demonstrating its capabilities for structured, adaptive, and timely reasoning.
2
Related Work
Agentic Frameworks for Embodied Tasks. A growing body of research has explored how LMs can be utilized to plan actions physical or simulated environments [ 21 29 (1, Recent studies emphasize agentic frameworks that enable autonomous planning and multi-stpp reasoning, rather than relying on single-step predictions. Within this paradigm, methods such as ReAct
Reflexion (3), and others [6] 33
35 36) integrate Mhain-of-Thhugle (CoTT) 337 38 39 40
reasoning with environmental feedback. Although these frameworks demonstrate strong performance, they typically rely on iterative inference of large-scale models, leading to substantial computational

---

overhead. In contrast, our approach employs an LM equipped with specialized memory architecture, enabling robust reasoning without dependence on large-scale models or multi-step inference.
Neurosymbolic Methods for Embodied Task Reasoning. Neurosymbolic approaches integrate symbolic reasoning modules-suce
rule-based or logic programming tools [41
42
43
with
neural networks to achieve interpretable and verifiable reasoning. With the advent of LMs, these
hybrid systems have advanced logical reasoning in natural language tasks (44,
45
46 47
This line of research has also extended into embodied tasks 8
49 where delegating
task reasoning to symbolic tools yielded more reliable and optimized action plans. However, these
methods depend on handcrafted domain knowledge (e.g., action rules) and require continuous updates to remain effective in dynamic environments [17]. Moreover, solving time increases sharply with task complexity, limiting -aletimm Decision-mmking in complex settings. In contrast, our approach embeds procedural knowledge within the LM's s memory architecture, enabling efficient reasoning and end-to-end adaptation through environmental feedback, without online symbolic assistance.
Memory-augmented LMs for Long-term Generation. Recent studies [14
15
50 51
52
53
enhance LMs with external memory structures to better retain long-term context, often through recurrent memory updates within transformer architectures. Other approaches store intermediate attention states-Such as key-value pairs from relevant documents or histories-in external memory modules for retrieval during inference [12 13 54]. While these methods expand the model's context window to capture richer semantic information, only few studies (55] 56 57] explore memory architectures specialized for embodied tasks. In contrast, our approach introduces procedural memory architecture that encodes task-level procedural knowledge, enabling efficient reasoning and adaptive behavior in dynamic embodied environments.
3 Problem formulation
We consider embodied reasoning in dynamic settings, where agent engages with stream of tasks and must adapt to changing states and goals over time. Each task is defined as a tuple T T= (S.AA.P.)) .g). where s e S is the state, E A is the action, P S A + S is the transition function describing dynamics, g E G denotes the goal. Due to partial observability (58) the agent receives an observation or at each timestep t. Unlike conventional multitask settings [ 3]. the agent must solve a sequence of tasks T= (71, (2,, .TN J. over time, where both g and P may vary across tasks (59 60). Our objective develop an LM-based agent (LM policy) that solves tasks autonomously and continuously at test time with no online access to any symbolic tools, while internalizing procedural knowledge from symbolic guidance during training. Note that Eq. ( defines the ideal objective, which is approximated in practice by supervising the LM on planner-computed action sequences with symbolic inputs.
N
or, || Ttoal oc. 02.
FiM = argmnax
Σ E
SR(ss. (LM g)) -
(I)
FLM
Here, SR: S A {0, 1} indicates whether actions taken in current states st lead task success, and DKL measures the divergence between the LM--asedd policy TLM and the symbolic policy Toool derived from external tools in 42 27 61 Accordingly, aं. aims at maximizing task success while aligning its learned procedural behavior with the tool's 's declarative guidance.
4 NESYPR: Neurosymbolic Proceduralization
To equip agents with structured and adaptive reasoning for diverse embodied tasks, NESYPR learns to encode production rules and compose procedures, compressed representations derived from the declarative knowledge of symbolic tools. We refer to this end-to-ent learning and utilization process as newrosymbolic proceduralization. As illustrated in Figure | neurosymbolic proceduralization operates in two phases: i) a training phase, compositional NeSy procedure learning, where procedural knowledge is structured within procedural memory using plans generated by symbolic tool, and ii) a test phase, NeSy procedure contrastive planning, where the agent autonomously adapts to new tasks by contrastively reconstructing procedural representations, without relying on symbolic tools.
During phase i), the agent trains on offline data comprising symbolically defined problem instances (observations and goals) and associated domain knowledge (action rules). The declarative knowledge
3

---

Compositional NeSy Procedure Learning
ii) NeSy Procedure Contrastive Planning
Aponledge
Iintance
(aacii Dig
Cinsital stats
Soal Beoord) (ratl que Iyfire
Offtine dataset
Cigial
Excoct lig inride Sedd Good Bucacit
New Tisk
Everschangigg covinument
Sym Symbolic Solver
LM-based Agent
L.-based Agent
Imputs
Search Aleo.
Production rules
Procedural incosrry
EMM
Precedaral
Cost Eas.
Solving
Thmn
Anm
Then
...
Stato Vran.
Thai
Enivered Feeuuck
Loss
x
Optimal Plas
Predicted
Predicted
Procedure
Figure 1: The framework architecture of NESYPR
used by symbolic tools for problem-solving, such as search algorithms, state transitions, cost estima- tion, and goal evaluation, is internalized production rules. The agent composes these rules into task-solving procedures in procedural memory, which it then exploits to generate plans. In phase ii), using the procedural memory established during phase i) training. the agent performs structured reasoning without access to external symbolic tools (i.e., declarative memory). It further engages in adaptive reasoning by contrastively reconstructing procedures from prior ones labeled successes or failures via environmental feedback. The agent continually reinforces plans aligned with valid procedures while suppressing those associated with invalid ones. Accordingly. NESyPR enables LM-based agents to reason robustly across tasks and adapt efficiently to ever-changing environments.
4.1 Compositional NeSy Procedure Learning
As shown in Figure our learning method incorporates procedural memory that extends existing approaches [5 I4 across L layers to support structured reasoning. Working memory M extends the context window by accumulating symbolic inputs from the environment. The procedural memory then performs vector quantization (VQ) [62]. encoding production rules into discrete procedure-unit stored in procedure-book C, which are composed to generate plans.
H1, M Decokde (H).,, M å e2,, Es). e, E RD
(2)
At each layer / E {l,.... L}. the decoder block DecoderBlock; takes the previous hidden state Hi-1 and M as input. Each slot e; encodes environmental context in dimension D, with S defining memory capacity. Runtime procedure R integrated into DecoderBlock; contributes to refining Hi.
Memory-augmented module. M encodes the current envirommental state, using memory- augmented cross tross-attention adapted from [15]. To enable information exchange between the self- attention output Exelt Txx from Hi-1 and M, we apply a cross-attention.
Ework = softmax
QK DD
V.
= E Wo: MWK. . V MWv
(3)
Here, Wo, WK, Wy DxD learnable projection matrices. M is then updated via gating mechanism that merges the original memory with the cross-attended representation Ework-
M + Sup @ a (wook) + (1 gup) @ M, 9up (w(Ew)kk) Wup)
(4)
Here, is alignment operator ensuring dimensional consistency, denotes the sigmoid activation function, @ represents slot-wise multiplication. and Wup &g learnable projection matrix.
In procedural memory, R is obtained by applying VQ to M using a procedure-book C = (c1.02.-..4 ) where each procedure-unit cj Rdd is d-dimensional vector. Each slot e; M is partitioned into contiguous d-dimensional chunks to align with the procedure-uniss
e; = [e (1) Each e² is replaced with its nearest procedure-units ck. EC, selected by minimizing the Euclidean distance.
=
[D/d], e(") E Rd
(5)
[ckg i Ckgi ... :ct,]] Ck argmin Ile,") c7ll2
(6)
eyec

---

Trgurs
Furbodding
Expt
Urooo Atteriton
Decoder Bluck
booki
Pracefural Momory
Feed Fieward
Head Layer
Decoder Black
Contraeive Flaming
Procrdinn
book
Pce Contrastive Planning
Language Model
Memorg-Angmmented Medale
Figure 2: Vector-quantized procedural memory and contrastive planning
Concatenating the selected procedure-unie forms a composite procedure Aggregating all such procedures across memory slots yields runtime procedure: R= C2,, ,,s] R into the model reasoning process, we combine R with E enhancing Hi.
R SxD integrate
H FFN (Ework + Hout a(R)) Pout - a (Ework Wout), Wout E R DXD
Here, FFN is feed-forward submodule, Wout a learnable matrix, and Jout is gating matrix.
Learning objective. To train the procedure- C end-to-end for compositional procedures, we combine the task objective (e.g., LM fine-tuning) with VQ loss applied at each layer I.
cV = ||sg( (M ) R 1 + M| (1) -sg -sg(R )|?
(8)
Here, sg is the stop-gradien operator, ||E the Frobenius norm [63]. and is weighting coefficient that controls M to align with R. The overall training objective is defined as
L logg To(a o. M)] +
VQ
where & denotes the learnable parameters of the LM, and À balances the task-specific objective and
the VQ term. is updated using an exponential moving average (EMA) (64), providing compositional procedural representations to R for use in structured reasoning.
4.2 NeSy Procedure Contrastive Planning
To support adaptive reasoning test time without symbolic tools, introduce procedure-basee contrastive planning strategy that reconstructs composed procedures from feedhack-labeled stored procedures and contrastively generates plans aligned with dynamic environments.
Procedure reconstruction. We maintain two procedure banks: MM + for successful procedures and M for failures. For each c; R, before integration in Eq. 77) reconstruct two versions: positive procedure ct by matching against M+ and a negative procedure c, against M
3c EMM Asim(c,, ct) o { at c
3f EM Asim(c, c 2 (10) otherwise
otherwise
Here, sim denotes similarity function (c.g., cosine similarity), and v is a reconstruction threshold. We then reconstruct These are used to generate two versions of the hidden state H, within a single batch: one conditioned on R+ and the other on Both are then used contrastive decoding (65).
runtime procedures: R$ Icct .c. ......,....... c5] and R [c, .2- ,.... cs]
Contrastive planning. Following [66), we guide generation toward successful procedures by com- puting contrastive score for each token x, within adaptive plausibility (
Vhead (x {x, E p1ppp(e
(z: I M; M++ Z 1 maxp* (a' | M;M+))
(11)
where 00.1 controls the truncation threshold. For z; Vheaa (Z<x), obtain contrastive score by
S(x,) loggp" (x; T<i, M; logp (xs 22i M
(12)
5

---

where p and p denote token distributions conditioned on successful and failed procedures, respec- tively. The final next-token distribution then reshaped as follows.
softmax (S(x)) ExrEVinus pd (a' (ee)) if ay E Vhead (F
(2) x=L)
(13)
1pt (z; x<())
otherwise
This decoding process suppresses failure patterns and promotes plans aligned with the environment, enabling adaptive reasoning without symbolic guidance. Algorithm of NESYPR is in Appendix.
5 Evaluation
5.1 Experiment Setting
Environments. We evaluate NESYPR across diverse embodied benchmarks, including multiple do- mains from PDDLGym 23 (e.g., Minecraft, Rearrangement, GlibRearrangementt, VirtualHome [67].
and ALFWorld [25] To assess embodied reasoning performance in dynamic environments, de- scribed in Section 3 we configure each benchmark to support task sequences that require multi-step planning and continual adaptation. In PDDLGym, where symbolic planners are built-in (27 61]. obser- vations are provided in symbolic form. We construct 9 distinct task sequences by randomly composing tasks, ensuring consistent evaluation settings across all baselines. For VirtualHome and ALFWorld, we utilize symbolic observation interfaces provided by their respective open-source implementation. We evaluate under continual task settings with configurations, using distinct task sequences for each, following [68] During the evaluation, agents receive only binary feedback the task level (success failure), and no gradient updates allowed.
and environment inceemeetnn
Datasets. For training, we use a small set of problem instances paired with plans generated by symbolic planners (27]. In PDDLGym, the train sets include 29 instances for Minecraft, 20 for Rearrangement, and 40 for GlibRearrangemeett The test sets contain 389, 400, and 80 instances respectively, all disjoint from the train data. For VirtualHome and ALFWorld, the train sets consist of 77 and 549 instances, respectively. Each test set is split into seen and unseen sets. The seen set contains 1 12 and 1,509 instances respectively, and shares the same goal as the train set, but varies in object placement and inter-object relations. The unseen set contains 52 and 1,369 instances respectively, and introduces entirely tasks, not present in both the train and the seen sets. At test time, agents are given only the current observation and goal, with no access to symbolic tools.
Baselines. For comparison, organize the baselines into four categories: (i) Single-step planning, including ZSP (30). RAP [69], and LLM--llaner 33, generates action plans an inference step. (ii Agentic workflow, such as CoT (37. ToT (38), GoT (39,. ReAct [4, and Reflexion (5), per- forms multi-step reasoning through multiple LM calls. (iii) Memory-augmented LM, including LongMem [12 and LM2 [13. incorporates long-term memory into the LM attention mechanism. Optimus-2 2 (55) and DT-Mem (57) extend this approach for embodied agents. We also include BUT-
LER 70). parameter-efffiieet fine-tuning method. (iv) Proceduralization such as BoT [28] stores abstract reasoning templates in a meta-buffer and dynamically instantiates them to guide procedural LM reasoning. Furthermore, Large Reasoning Model (LRM) (71) integrates Co'T-style reasoning through reinforcement learning, with compact variants distilled from larger models. PlaSma (72) distills procedural knowledge from larger LMs into compact models. By default, we use LLaMA-3.2- (74 for VirtualHome and ALFWorld.
IB 73 for PDDLGym, and Qwen2.5-0.5B
Metries. We use four standard metrics, following 33 75 76). Cumulative Task Success Rate (CSR)
measures the percentage of tasks where all uob-goals are achieved. Cumulative Coal-Condt Success Rate (CGC) reports the fraction of individual sub-goals achieved across all tasks. Executabil- ity (Exe) assesses each selected action is feasible. Success rate weighted by Path Length (SPL) reflects both task success and path efficiency.
Further details of the experimental settings are provided the Appendix.
5.2 Main Result
Open-loop continual embodied tasks. To evaluate the generalization performance of NESYPR on open-loop continual task planning, we conduct experiments in Table jusing multiple domains from PDDLGym. In this setting, the agent generates complete action sequence without intermediate

---

Table I: Performance open-loop continual embodied tasks in PDDLGym. Metrics averaged over random seeds, with standard deviations to indicate consistency across runs. PARAMS denotes the total number of model parameters, with the ratio of trainable parameters in parentheses.
METHOD
PARAMS
CSR
CGC (f)
EXE (1)
SPL (f)
CSR
CGC (1)
EXE (f)
SPL. (1)
DOMAIN: MINECRAFT
ZSP
(0.0%)
76.4+5.5
81.744.8
100.0+0.0
0.8+0..1
16.4=1.5
35.9+22.0
100.0+0.1
0.1=0.0
RAP
1.7B (0.0%)
76.445.5
81.74:4.8
100.020.0
0.810.11
16.820.9
18.0:1.9
100.010.1
0.1:0.00
COT
1.78 (0.0%)
83.3#6.7
83.3+9.3
100.0+0.0
0.84:0.1
17.0=0.5
25.7+188
100.0#0.0
0.1=0.0
85.1+4.22
85.743.7
100.0+0.0
0.94-0.0
18.7+1.0
32.242.6
100,0+0.00
0,1+0.00
GOT
1.78 (0.0%)
89.1#5.11
94.6#7..7
100. 000.0=0.0
0.940.1
18.9=0.5
25 25.9+1.2
100.0:0.00
0.1=0.00
BUTLER
100.0+-0.0
100.0+0..0
100.0=0.0
1.0+0.0
51.4=199
742.6
99.7=0.3
0,4+0.0
LONGMEM
1.6B (24.6%)
100.04:0.0
100.0::0.0
100.0::0.0
1.400.0
33.3+33.7
56.144.1
99.8:0.11
0.5:0.00
LM2
1.38 (6.3%)
100.0+00.0
100.0+0.00
100.0+0.0
1.000.0
47.9=8.4
56.544.6 6
99.9=0.1
0.4+0.0
(6.39)
100.04-0.0
100.0+00.0
100.0+0.00
65.2::1.4
68.9422.4
100.0:+0.1
0.6
DOMAIN: REARRANGEMENT
ZSP
1.78 (0.0%)
94.2#3.88
97. 6
100.0:4 00000=00.0
0.9:00.0
15.3 2.
22 22.4#3..4
100.0:0..0
0.1=0.00
RAP
1.7B (0.0%)
94.243.8
97.611.2
100.010.0
0.910.0
18.3
29.0:12.6
100.010.00
0.1:0.0
COT
1.78 (0.0%)
95.0=44.5
5-2.5
100.0=0.0
1000.00
19.2=0.9
29.9+1.5
100.0+0.0
0.1
TOT
1.78 (0.0%)
95.0++6.3
1.2
100.0+0.0
1.0+0.1
20.2+11.
33.9+1.4
100.0+0.00
0.2+0.0
GOT
100.0::0.0
BUTLER
1.28 (0.6%)
100.0::011
100.0::0.0
100.0=0..0
1.000.0
56.5=2.0
69.842.9
100.0+0.00
0.5=0.0
LONGMEM
1.6B (24.6%)
100.04-0.0
100.0+-0.0
100.0+00.0
1.040.00
58.2+1.9
69.9+1.4
100.0+0.0
0.5+0.0
1.38 (6.3%)
100. 02:00::00
100.0+:00
100.0=0.0
1.040.0
57.4=4.0
69. 69.648.99
100.0+0.0
0.5=0011
NESYPR
1,38 (6.355)
100.0+:0.0
100.0+0.00
100.0+0.0
L0400.0
73.5+3.0
80.8+1.0
100.0+0.00
0.7+ 0.0
observations and receives only binary task feedback (success or failure) before proceeding to the next task. NESYPR outperforms the strongest baseline, LongMem, in the Minecraft and Rearrangement domains, achieving improvements of 13.6% in CSR, 11.7% in CGC, and 0.15 in SPL on the test set, thereby demonstrating superior structured and adaptive reasoning capabilities. Similar performance gains are observed in the Glibecarrangement domain as well. More experimental results are provided Appendix. The single-step planning baselines such ZSP and RAP, which rely on in-context serieval-auggente generation [77], show limited reasoning capacity on unseen tasks. The agentic workflow baselines such COT, ToT, and GoT, which use reasoning guiddnnce prompts crafted from the train set [16), perform slightly better, but remain far from achieving reliable task success. The memory-augmentee LMs such as LongMem and LM2 surpass the fine-tuning baseline BUTLER, but their average CSR on the test set is still 15.2% lower than that of NESYPR.
Closed-loop continual embodied tasks. To further evaluate the generalization performance of NESYPR alongside its adaptability in dynamic settings, we conduct experiments under a closed-loop continual task planning setup VirtualHome and ALFWorld. The test set divided into seen and unseen sets, enabling a detailed assessment of the agent's s structured and adaptive reasoning capabil- ities. Unlike open-loop settings, the agent selects actions sequentially in response to intermediate
Table 2: Performance on closed-loop continual embodied tasks in VirtualHome and ALFWorld.
METHOD
TRAIN
SEEN
UNSEEN
(7)
CGC (t)
SPI. (1)
(f)
CGC (1)
SPL (1)
(1)
CGC (?)
SPI. (t)
BENCHMARK: VIRTUALHOME
LLM PLANNER
61.0.1.5
66.4.1..
0.4
45.5
47.4
0.310.0
28.8.12.2
30.0.1..1
0.210.0
REACT
63.6=E1.1
70.0+0.6
0.4
54.7411.
56.44-088
0.440.0
32.743.5
34.3-33.6
0.340.0
REPLEXION
60.8=5.9
68.8=5.5
0.4=0.0
57.243.8
61.6+5.4
0.4+0.0
33.7+2..5
35.3=2.1
0.3+0.0
LONGMEM
80.5422.
86.242..5
0.8
63.34...6
68.3466.3
0.6:10.1
45.747.3
52.2-1.9
0.410.1
LM2
80.244.2
85.2=2.9
0.8=0.0
53.645.9
$7.645.2
0.5+0.1
38.8+4.44
43.3-4.4
0.3+0.0
DT-MEM
77.3=3.8
80.824.8
0.7-0.0
69.345.7
71.955.9
0.740.1
48.745.9
52.626.0
0.520.1
OPTIMUS-2
79.3
50.917.3
0.410.1
NESYPR
89.8
92.321.1
0.9
78.914.5
81.742.5
0.8:0.00
61.12..2
69.322.6
0.6:0.0
BENCHMARK: ALFWORLD
LLM-PLANNER
52,4-2.6
142.0
0,5-0.0
14.040.8
21.740.7
0.14:0.0
3,340.4
11.720.5
0.0+0..0
REACT
46.3=1.5
65.3=1.1
0.4=0.0
12.820.5
21.400.6
0.140.0
2.920.2
11.620.2
0.0=0.0
REFLEXION
44.310.7
64.120.7
0.4
13.010.5
21.610.8
0.1100.0
2.910.4
11.8:0.4
0.0:0.00
LONGMEM
50.143.1
58.612.4
0.5
48.610.6
57.310.7
0.5
45.2.1..8
54.510.8
0.510.0
LM2
63.8:1..5
71.521.4
0.6:0.0
42.612.4
46.912.0
0.410.0
38.742.1
45.242.5
0.420.0
DT. MEM
57.742.2
61.742.7
0.6
41.312.9
48.013..
0.4.0.00
38.011388
44.114.2
0.410.0
OPTIMUS-2
59.5+11.5
67.41.3
0.6=0.0
52.840.9
61.6+0.77
0.5440.0
140.7
58.740.6
0.540.00
NESYPR
69.6.1277
76.2.2.1
0.7
61.111.1
68.611.3
0.610..0
59.7:11.4
6..9
0.6:0.0

---

observations. In Table NESYPR outperforms the strongest baseline in VirtualHome, DT-Mem, with average improvements of 12.5% CSR and 11.5% CGC on the train set, 9.6% and 9.8% on the seen set, and 12.4% and 16.7% the unseen set, respectively. In ALFWorld, NESYPR outperforms the strongest baseline, Optimus-2, achieving average gains of 9.8% CSR and 8.8% in CGC on the train set, 8.3% and 7.0% on the seen set, and 10.6% and 9.2% on the unseen set, respectively. Notably, the unseen sets show greater performance gains than the seen sets. Combined with an average improvement of 0.12 in SPL, these results indicate that NESYPR performs effective symbolic reasoning. In Appendix, additional results for each incremental configuration are provided, along with complete baseline comparisons. Specifically, both LLM-Plannee and agentic workflows such as ReAct and Reflexion exhibit limited capability for symbolic reasoning across benchmarks. While Reflexion leverages past experiences via verbal feedback, it appears to lack the robust reasoning capabilities required in dynamic and complex tasks. Memory-augmented approaches for embodied agents, such DTT-Mem and Optimus-2, outperform other baselines. Yet, NESyPR achieves higher performance, surpassing both DTT-Mem and Optimus-2 by an average 11 11.2% in CSR and 1 11.6% in CGC, showing the effectiveness of neurosymbolic proceduralization.
5.3 Analysis and Ablation
Table 3: Analysis on proceduralization. LATENCY denotes the agent's planning time in seconds. TOKENS denote the total number of input and output tokens used.
TASK PERFORMANCE
REASONING LOAD
CSR (1)
CGC (f)
SPL
LATENCY (L)
IN TOKENS (J)
OUT TOKENS (J)
LLAMA3,1-8B
53.0+0.5
63.5+00.4
0.34-0.0
$9.5+1.9
8007 8007.9 103.9
1315.4428.1
BoT
LLAMA3.1-708
81.910.4
85.1
0.6::.00
75.1
7651.01127.7
794.1433.44
GPT4.1
92.140.3
93.64:0.2
0.740.0
22.242288
7986.1+11477
2202.2:119..2
DEEPSEEK RI
11,520.3
15.6100.3
0.11:0.0
111.013.33
3198.5:11.8
2187.6:16.,0
LRM
DERPSFFK-RI-70S
26.540.4
27.54:0.4
0,240.0
209.44492
3198.55+15.8
1679.3+87.5
78.920.4
80.810..
0.510.0
18.621.7
3214.6:11.77
2113.9:63.2
LLAMA3.2-1B
67.420.5
71.9:0.4
0.740.0
2.74:0.5
3221.8145.3
32.711466
PLASMA
LLAMA3.2-B8 3B
70.7:10.4
75.7100.
0.710..0
7.2.10.7
3247.7H11..2
29.5:1:55
80.5:40.5
89.24-2.3
LLAMA3.2-11
73.2:10.4
76.0110.4
0.710.00
1,2.0.3
3168.510.0
30.1::..3
NESYPR
LLAMA3,2-3B
83.612..0
88.812..0
0.810.0
3.54003
3169.520.0
43.6.5..3
LLAMA3.1-8B
89.0122.0
93.5111.8
0.9::0.0
5.240.7
3155.510.0
41.911..0
Analysis on proceduralization Table presents a comparative analysis of our neurosymbolic proce- duralization method to existing proceduralization methods, evaluated in terms of task performance and reasoning efficiency, with particular focus on enabling timely reasoning through single-step inference. To ensure fair comparison, we additionally include a unified setting in which all methods evaluated under identical inference conditions using the same LLaMA 3.I-8B backbone, high- lighted in gray background in the table. NESYPR achieves the lowest average plan generation latency, the highest task success rate, and the minimal input and output token usage. BoT and LRM exhibit latencies that are 54.3 and 105.8 seconds longer than NESYPR, respectively, along with 6, 125.9 and 2, 188.7 more total tokens consumed, due their reliance multi-step reasoning. PlaSma, which distills procedural knowledge from larger to smaller LMs, achieves competitive results with efficient inference, reaching 80.5% in CSR using an 8B LM. Yet, NESYPR outperforms it with a higher CSR of 83.6% while operating with only a 3B LM.
80
80
62.1 1%
NeSyPr
60
60
Symbolic Planner 48.5%
CSR
ReAct
40
40
20
NeSyPr
20
Symbolic Planner
0.0% Symbolic Planner model (b) Robustness to dynamic conditions
0
5
10 12
14 16 18 20 22 24 26 28 30
NeSyPr
ReAct
Time constraint (s)
(a) Comparison under varying time constraints
Figure 3: Comparison with symbolic planner

---

Comparison with online symbolic planner. Figure analyzes the impact of neurosymbolic pro- ceduralization on automated decision-makiin of NESYPR and a symbolic planner under strict time constraint, allowing 1% violation. For tasks where the symbolic planner takes over 10 seconds to find solution, NESYPR completes planning within 5-second constraint while achieving 50.6% in CSR. In comparison, the symbolic planner takes up to 22 seconds to reach similar performance. Figure b)) evaluates robustness on unseen tasks with dynamic conditions. While the symbolic planner fails when input information is incomplete, NESY PR maintains stable performance and even outperforms ReAct using GPT-4o (78)
agents. Figure compares the task success rate
Table Analysis on continual embodied task adaptation scenario. During continual task inference, the entire test set is periodically evaluated across 15 intermediate continual evaluation phases.
CONTINUAL EVALUNTION PHASE
MICHOO
MATRIC
7
8
9
10
13
14
15
SR
57.1
56.0
58.6
54.6
47.5
51.1
52.4
51.5
52.2
52.4
53.0
53.6
55.7
54.4
53.8
LONGMEM
FWT (1)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
BWT (T)
0.0
0.0
0.0
0.0
D.D
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
(t)
64.3
52.0
57.6
53.3
50.8
50.0
49.3
48.5
52.3
52.2
51.6
LM2
BWT
0.0
0.0
0.0
0.0
-2.5
2.2
-3.2
1.6
-3.0
1.5
1.4
1.2
1.2
1.2
1.1
1.1
-2.2
2.2
FR (1)
0.0
0.0
0.0
0.0
9.1
8.0
6.1
5.9
5.7
4.9
4.8
4.7
4.3
4.2
4.0
RR (*)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
2.4
0.0
2.4
0.0
SR (t)
64.3
64.0
62.1
63.6
65.0
66.7
61.9
60.6
60.9
61.0
61.5
61.9
63.6
62.2
61.3
NONY
FWT
<t)
7.1
4.0
3.4
3.0
3.3
2.2
1.6
4.5
1.5
1.4
1.2
1.2
1.2
1.1
2.2
4.3
3.2
FR (1)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
12.5
10.0
7.1
13.0
12.0
11.5
12.9
12.5
12.1
Analysis on continual task adaptation. Table presents continual adaptation results, highlighting NESYPR'S adaptive reasoning using metrics from continual learning (79). Forward Transfer (FWT) measures how newly acquired procedures improve performance on future tasks by comparing average per-task SR with overall CSR. Backward Transfer (BWT) compares current CSR with that obtained when re-evaluating earlier tasks using retained procedures. Forgetting Rate (FR) is the proportion of previously successful tasks that fail upon re-evaluation, while Recovery Rate (RR) is the proportion of previously failed tasks that later succeed. LongMem, which stores key-value states for retrieval, FWT and no degradation in BWT. In contrast, LM2, which implicitly
shows no improvement maintains a working memory to extend context, shows moderate improvement in FWT but fails to preserve BWT. By leveraging both valid and invalid procedures, NESYPR achieves superior performance in both FWT and BWT. Notably, its FR converges zero, and it attains about 12.0% RR, demonstrating effective adaptation without symbolic tools.
Task type
Task type 3
scone
scene
scene
scans
10 10011111111511119 Task type 2
1011 100122111110 01112221111111100222222200000000
Task type 4
5cone
04
scene
scene
02
2
1314 11 Code ld
10 11
0100001111111 5
101112131411 1617 18 23
Code ld
0.
Figure 4: Analysis on procedural memory interpretability
Analysis on procedural memory interpretation. Figure 4 shows heatmap of procedure-unit (in Eq. (6) usage across 4 task types and scenes. Task types and 2 share similar solution and exhibit consistent usage patterns across different scenes. In contrast, task types 3 and 4 involve object-specific actions (e.g.. picking up or turning on items), which are more sensitive to scene variations and lead to more divergent patterns.
Application with different LMs. Table reports the performance of NESYPR using three LM families across five model sizes [80
73 82 evaluated on the Minecraft domain. Across all LMs, NESYPR achieves an average CSR improvement of 10.0% over BUTLER. Performance generally improves with model size, although the degree of improvement varies across different LM families.
81

---

Table 5: Application with different LMs
Table 6: Ablation study of NESYPR
METHOD
CGC
METHOD
CSR (1)
CGC (1)
BUTLER
QWEN2-D.5B
41.3++1.0
54,7+3.3
NESYPR (FULL)
65.2+1.4
68.9+2.4
OWEN2-D.5B
51.711.7
NESYPR W/O EMA UPDATE OFC
63.0#3.1
64.8+3.7
BUTLER
LLAMA3.2-18
$1.42119
66.7#2.6
NESYPR
65.241.4
68.9#2.4
NESYPR
47.948.4
56.5+4.6
BUTLER
QWEN2,5-1.98
50.341.5
58.613.44
NESYPR M ONLY
63.2.11.3
65.1:2.4
NESYPR
QwEN2.5-1.58
16.7#1.3
19.0=1.5
ONLY
63.3111.4
66.4.2.5
BUTLER GEMMA2-28
1..
59.9=0.5
NESYPR W/O
59,911.4
62.942.4
NESYPR
GEMMA2-2B
66.434.1
73.8:0.5
NESYPR w/o CP, FOLLOW M
60.9+1.1
64.4+2.2
LLAMA3.2-3B
NESYPR
LLAMA3.2-38
74.9#2.4
77.1218
NESYPR w/o CP, FOLLOW M
59.7:11.4
6..2:1.3
ablation study that evaluates the contribution of each component NESYPR, using Minecraft. Using the procedure-book C without EMA updates (i.e., NESyPR w/o EMA UPDATE OF C) results in performance drop of 2.2% in CSR and 4.1 in CGC. When C removed entirely (i.e., NESYPR w/o C), the performance drops by 17.3% in CSR and 12.4% in CGC, demonstrating the importance of learned C. Models using only successful procedures M (i.e., NESYPR w/ MM ONLY) or only failure procedures M (i.e.. NESYPR w/ M ONLY)-where the original composed procedure is used as the counterpart in contrastive planning, perform 2.0% and 1.9% worse in CSR, respectively, compared to the full version. Disabling contrastive planning (i.e., NESyPR w/o CP) leads to further performance drop of 5.3% in CSR. Without CP, simply following either M or M (i.e., NESYPR w/o CP, FOLLOW M and NESYPR w/o CP, FOLLOW M leads to less reliable plan generation, with CSR reductions of 4.3% and 5.5%, respectively.
Ablation study. T
100
100
BUTLER
DT Mem
Ne5yPr
60
CSR
CSR
40
Procedure-book size
Procedure-unit size
Roconstruction Threshold
Low
Middle
High
16.75
33.3%
Train datasel size
50.0%
7%
83.3%
100.0%
(a) Hyperparameter sensitivity
(b) Data efficiency
Figure 5: Ablation study on procedural memory hyperparameters and learning data efficiency
Hyperparameter sensitivity and data efficiency. Figure 51a) shows that a small size of C' limits task coverage, while sufficiently large one enables consistently high performance. In contrast, increasing the size of each procedure-unit e reduces diversity in the composed procedures, leading to performance degradation. We also observe that setting low reconstruction threshold causes over- generalization by accepting low-siiilarity procedures, while a high leads under-gennt dider--eeeralizzttion by rejecting valid ones. Figure ()b) shows the data efficiency of each method by reporting task success rates across varying training dataset sizes. NES YPR consistently achieves higher CSR with less data, indicating superior learning efficiency compared to BUTLER and DTTMem.
6 Conclusion
We presented the NESYPR framework that employs neurosymbolic proceduralization inspired by ACT theory. It performs knowledge compilation by abstracting and generalizing multi-step symbolic auth-finding and reasoning into single-step inference within an LM. This enables LM-based agents to conduct embodied reasoning efficiently, without relying large-scale inference engines online access to symbolic tools. Experimental results on PDDLGGm, ALFWorld, and VirtualHome show that NESYPR enables structured, adaptive, and timely reasoning in dynamic embodied environments.
Limitation and future direction. As shown Table NESYPR's performance partially depends on the pretrained knowledge of the LM. To mitigate this, we plan to explore a joint learning strategy that combines knowledge distillation from larger LMs with neurosymbolic proceduralization. This approach has the potential to enhance generalization to more complex, real-world scenarios.
10

---

Acknowledgement
This work was supported by the Institute of Information & communications Technology Planning & Evaluation (ITTP) grant funded by the Korea government (MSIT), RRS-2022-I1220043 (2022- 0-00043), Adaptive Personality for Intelligent Agents, RS- RS-2022-1I221002 directed multi-modal Intelligence for solving unknown, open domain problems, 255-2025-0221881
0002-00-1045), Self-
Accelerated Insight Reasoning via Continual Learning, RS-2025-25544 2024-555,,,,,, AT Star Fellowship Support Program (Sungkyunkwan Univ.), and RS-2019-111904211 Artificial Intelligence Graduate School Program (Sungkyunkwan University)), the National Research Foundation of Korea (NRF)
grant funded by the Korea government (MSIT) (No. 24-2023-0021311 18), ITTP-ITRC (Information Technology Research Center) grant funded by the Korea government (MIST) (IITP-202) 2025-RR-2002- 00437633, 10%). IITP-ICT Creative Consilience Program grant funded by the Korea government (MSIT) 12-25-5--220--11 112000-122118000 821, 10%), and by Samsung Electronics Co., Ltd.
References
[1] Wenlong Huang al. "Inner monologue: Embodied reasoning through planning with language models". In: arXiv preprint arXiv:2207. SEC.:222070050088888 (2022).
[2] Anthony Brohan et al. "Do as can, not as say: Grounding language in robotic affordances".
In: Proceedings of the 6th Conference on Robot Learning. 2023.
[3] Chan Hee Song et al. "LLM-planner: Few-shot grounded planning for embodied agents with large language models". In: Proceedings of the 19th IEEE/CVF International Conference on Computer Vision. 2023.
[4] Shunyu Yao et al. "ReAct: Synergizing Reasoning and Acting in Language Models". In: The Eleventh International Conference on Learning Representations. 2023.
[5] Noah Shinn et al. "Reflexion: Language agents with verbal reinforcement learning". In: Advances in Neural Information Processing Systems (2024).
[6] Chen Liang et al. "Textualized Agent-Style Reasoning for Complex Tasks by Multiple Round LLM Generation". In: arXiv preprint arXivv2409. 12411 (2024).
[7] Junde Wu, Jiayuan Zhu, and Yuyuan Liu. "Agentic Reasoning: Reasoning LLMs with Tools for the Deep Research" In: arXiv preprint arXiv:2502.04644 (2025).
[8] Tom Silver et al. "Generalized planning in pddl domains with pretrained large language models". In: Proceedings of the AAAI conference on artificial intelligence. 2024.
[9] Xinrui Lin et al. "CLMASP: Coupling Large Language Models with Answer Set Programming for Robotic Task Planning". In: arXiv preprint arXiv:2406.03: 03367 (2024).
[10] Bo Liu et al. "LIm+ p: Empowering large language models with optimal planning proficiency".
In: arXiv preprint arXiv:2304. 1477 (2023).
[11] Sudhir Agarwal et al. "LLM+ Reasoning+ Planning for supporting incomplete user queries in presence APIs". In: arXiv preprint arXiv:2405. /2433 (2024).
[12] Weizhi Wang et al. "Augmenting language models with long-term memory". In: Advances in Neural Information Processing Systems (2023).
[13] Weijie Liu al. "Memlong: Memory-augmented retrieval for long text modeling". In: arXiv preprint arXiv:2408. 16967 (2024).
[14] Aydar Bulatov, Yury Kuratov, and Mikhail Burtsev. "Recurrnt memory transformer". In: Advances in Neural Information Processing Systems (2022).
[15] Jikun Kang et al. "LM2: Large Memory Models". In: arXiv preprint arXiv:2502.06049 (2025).
[16] Wonje Choi et al. "Embodied CoT Distillation From LLM To Off-the-shelf Agents". In: Proceedings of the s1 International Conference on Machine Learning- 2024.
[17] Wonje Choi et al. "NeSyC: A Neuro-symbolic Continual Learnr For Complex Embodied Tasks In Open Domains". In: arXiv preprint arXiv:2503.008: 000870 (2025).
[18]
John R Anderson. Cognitive science series. The architecture of cognition. 1983. URL: http
// //act-r- pay.pa...mm cmu. edu/)
[19] Flavio TP Oliveira and David Goodman. "Conscious and effortful or effortless and automatic: a practice/performance paradox in motor learning". In: Perceptual and motor skills (2004).
[20] John R Anderson. "Acquisition of cognitive skill." In: Psychological review (1982).

---

[21] Farnaz Tehranchi, Jacob David Oury, and Frank E Ritter. "Predicting learning and retention of a complex task using a cognitive architecture". In: Proceedings of the Anmual Meeting of the Cognitive Science Society. 2021.
[22] Frank E Ritter et al. "Declarative procedural tutors: A family of cognitive architecture- based tutors". In: Proceedings of the 22nd conference on behavior representation in modeling and simulation. 2013.
[23] Tom Silver and Rohan Chitnis. "PDDLGym: Gym environments from PDDL problems". In: arXiv preprint arXiv:2002.0644 (2020).
[24] Xavier Puig et al. "VirtualHome: Simulating household activities via programs". In: Pro- ceedings of the 29th IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018.
[25] Mohit Shridhar al. "Alfworld: Aligning text and embodied environments for interactive learning". In: arXiv preprini (XXv:20010.03768 (2020).
[26] Daya Guo et al. "Deepseek-r1: Incentivizing reasoning capability in ilms via reinforcement learning". In: arXiv preprint arXiv:2501. /2948 (2025).
[27] Malte Helmert. "The fast downward planning system". ". In: Journal of Artificial Intelligence
Research (2006).
[28] Ling Yang et al. "Buffer of thoughts: Thought-augmentte reasoning with large language models". In: Advances in Neural Information Processing Systems (2024).
[29] Zihao Wang al. "Describe, explain, plan and select: Interactive planning with LLMs enables open-world multi-task agents". In: Proceedings of the 37th Advances in Neural Information Processing Systems. 2023.
[30] Wenlong Huang et al. "Language models as zero-shot planners: Extracting actionable knowl- edge for embodied agents". In: Proceedings of the 39th International Conference on Machine Learning. 2022.
[31] Zhenyu Wu al. "Embodied task planning with large language models". In: arXiv preprint arXiv:2307.00 0888 (2023).
[32] Ishika Singh al. "ProgPrompt: Generating situated robot task plans using large language mod- els". In: Proceedings of the 40th IEEE International Conference on Robotics and Automation. 2023.
[33] Andy Zhou et al. "Language agent tree search unifies reasoning acting and planning in language models". In: arXiv preprint arXiv::3 1300440 (2023).
[34] Zirui Zhao, Wee Sun Lee, and David Hsu. 'Large language models as commonsense knowledge for large-scale task planning". In: Advances in Neural Information Processing Systems (2024).
[35] Bowen Jin et al. "Search-rl: Training 1lms to reason and leverage search engines with rein- forcement learning". In: arXiv preprini arXiv: 095/6 (2025).
[36] Michal Zawalski et al. "Robotic control via embodied chain-of offthought reasoning". In: arXiv preprint (.4i::2407.06699 (2024).
[37] Jason Wei et al. "Chain-of-thouge prompting elicits reasoning in large language models". In: Proceedings of the 36th Advances in Neural Information Processing Systems. 2022.
[38] Shunyu Yao et al. "Tree of thoughts: Deliberate problem solving with large language models".
In: Advances in Neural Information Processing Systems (2024).
[39] Maciej Besta et al. "Graph of thoughts: Solving elaborate problems with large language
models". In: Proceedings of the AAAI Conference Artificial Intelligence. 2024. [40] Yao Yao, Zuchao Li, and Hai Zhao. "Beyond maai--of-thoughe effective graph-of-thought reasoning in large language models". In: arXiv preprini arXiv:2305. 16582 (2023).
[41] Bruce Frederiksen. "Applying expert system technology to code reuse with pyke". In: PyCon: Chicago (2008).
[42] Martin Gebser et al. "Multi-shot ASP solving with clingo". In: Theory and Practice of Logic Programming (2019).
[43] Leonardo De Moura and Nikolaj Bjørner. "Z3: An efficient SMT solver". In: International conference on Tools and Algorithms for the Construction and Analysis of Systems. 2008.
[44] Theo X Olausson et al. "LINC: A neurosymbolic approach for logical reasoning by combining language models with first-order logic provers". In: arxiv preprint 1511 (2023).
12

---

[45] Liangming Pan et al. "Logic-Im: Empowering large language models with symbolic solvers for faithful logical reasoning". In: arXiv preprint arXiv:2305.12299 (2023).
[46] Meng Fang et al. "Large language models are neurosymbolic reasoners". In: Proceedings of the AAAI Conference on Artificial Intelligence. 2024.
[47] Zhun Yang, Adam Ishay, and Joohyung Lee. "Coupling large language models with logic
programming for robust and general reasoning from text". In: arXiv preprint arXiv:2307.07691 (2023).
[48]
Adam Ishay, Zhun Yang, and Joohyung Lee. "Leveraging large language models to generate answer set programs". In: arXiv preprint arXiv: 0769977999999 (2023).
[49] Cristina Cornelio and Mohammed Diab. "Recover: A Neuro-Symbolic Framework for Failure Detection and Recovery". In: arXiv preprint 2:Xx:22404000756 (2024).
[50] Zihang Dai et al. "Transformer-x xl: Attentive language models beyond fixed-length context".
In: arXiv preprint arXiv: 1901.1100000 02860 (2019).
[51] Zexuan Zhong, Tao Lei, and Danqi Chen. "Training language models with memory augmenta- tion". In: arXiv preprint arXiv: 12674 (2022).
[52] Ching-Yun Yun Ko al. "MeeReasoner: A Memory-augmented LLM Architecture for Multi-hop Reasoning". In: The First Workshop on System-2 Reasoning at Scale, NeurIPS'24 24. 2024.
[53] Ivan Rodkin et al. "Associative recurrent memory transformer". arXiv:2407.0m 00844 (2024).
In: arXiv preprint
[54] Yuhuai Wu et al. "Memorizing transformers". In: arXiv preprint arXiv:2003.08913 (2022).
[55] Zaijing Li et al. "Optimus-2: Multimodal minecraft agent with goal-obbeervaiion-actonn condi-
tioned policy". In: arXiv preprint arXiv:2502. 19902 (2025).
[56] Shihao Wang et al. "Omnidrive: A holistic Ilm-agent framework for autonomous driving with
3d perception, reasoning and planning". In: arXiv preprint arXiv:2405.0153: (2024).
[57] Jikun Kang et al. "Think before you act: Decision transformers with working memory". In:
arXiv preprint arXiv:2305. 16338 (2023).
[58] Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. MIT press,
2018
[59] David Abel et al. "A definition of continual reinforcement learning". In: Advances in Neural Information Processing Systems (2023).
[60] Daehee Lee et al. "Incremental learning of retrievable skills for efficient continual task adapta-
tion". In: Advances in Neural Information Processing Systems (2024).
[61]
Jörg Hoffmmann. "FF: The fast-forward planning system". In: Al magazine (2001).
[62] Aaron Van Den Oord, Oriol Vinyals, et al. "Neural discrete representation learning". In: Advances in neural information processing systems (2017).
[63] Oscar Skean et al. "FroSSL: Frobenius Norm Minimization for Efficient Multiview Self- Supervised Learning.. In: arXiv preprint arXiv:2310.02000 02900 (2023).
[64] Minyoung Huh et al. "Straightening out the trraihh-tthrouge estimator: Overcoming opti- mization challenges in vector quantized networks". In: International Conference on Machine Learning. 2023.
[65] Xiang Lisa Li et al. "Contrastive decoding: Open-ended text generation as optimization". In: arXiv preprint arXiv:2210. 15097 (2022).
[66]
Ariel Gera et al. "The benefits of bad advice: Autocontrastive decoding across model layers".
In: arXiv preprint arXiv:2305. 006880 (2023).
[67] Manling Li et al. "Embodied agent interface: Benchmarking Ilms for embodied decision making". In: Advances in Neural Information Processing Systems (2024).
[68] Byeonghwi Kim, Minhyuk Seo, and Jonghyun Choi. "Online Continual Learning for Interactive
Instruction Following Agents". In: ICLR. 2024.
[69] Tomoyuki Kagaya et al. 'Rap: eetrieval-augmenter planning with contextual memory for multimodal Ilm agents". In: arXiv preprint arXiv:2402.036100 (2024).
[70] Vincent Micheli and François Fleuret. "Language models are few-shot butlers".. In: arXiv preprint arXiv:21144 011.00447772 (2021).
[71] Ahmed Ki-Kishky et al. "Competitive programming with large reasoning models". In: arXiv preprint arXiv:2502.00 06807 (2025).
13

---

[72] Faeze Brahman et al. "PlaSma: Procedural Knowledge Models for LLaguage-basee based Planning
and Re-Planning". In: The Twelfth International Conference on Learning Representations. 2024.
[73] Meta. Llama 3.2 1B Language Model. 2024. URL: https: hhtgs://huginggfac... co meta.
huggingface
Bama/LLama-3.2211
[74] Qwen Team. Qwen2.5: A Party of Foundation Models. 2024. URL: httpp::////g/nnnn gitthu
Lo/blog/qwent .5/
[75] Mohit Shridhar et al. "ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks". In: The IEEE Conference Computer Vision and Pattern Recognition
(CVPR). 2020.
[76] Peter Anderson et al. "On evaluation of embodied navigation agents". In: arXiv preprint arXiv:1807.00 07.80777777777 (2018).
[77] Ori Ram et "In-context retrieval-augmenttr language models". In: Transactions of the Association for Computational Linguistics (2023).
[78] Aaron Hurst et al. "Gpt-4o system card". In: arXiv preprint arX1v:2410.21276 (2024).
[79]
Bo Liu et al. "Libero: Benchmarking knowledge transfer for lifelong robot learning". In: Advances in Neural Information Processing Systems (2023).
[80]
An Yang et al. Qwen2 Technicul Report. 2024. URL: https: /ttp/aaavvv/...//////.../.a.aa.. org/ abs 2407
10671
[81]
An Yang et al. Qwen2.5 Technical Report. 2025. URL: https :ttp:s:///xxxvo/// org/ abs 2412
15115
[82]
Gemma Team al. "Gemma Improving open language models practical size". In: arXiv preprint arXiv:2408, 00118 (2024).
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
The paper does not include theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Ys]]
Justification: We include implementation details in Appendix, and include source codes in supplementary materials.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification:
We include the source codes in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ctiipe.cc/pulli//uuiii htt.pp/bll.//giiiidodddeeeeetiii CodeeSubeeiieeeeeeeeee ssion ceiiionPoicyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper speciffy all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We specify all the training and test details in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We provide error bars for all experiments in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
17

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification:
We include information the computer resources in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps://me h/tppp/////urrps...// httpp//ps.....b......h..ccccaa.a
Answer:
[Yes]
Justification:
We conform the NeurIPS Code of Ethies.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
18

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite the original paper that produced the code package or dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, haaerssithcoode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
19

---

Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
20

---

Justification: We use LLM for the baselines and proposed method. Detailed implementations are described in Appendix.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy hattps://neuris... 200p:/e////oeeeeeeee/222//////// for what should or should be described.
21


Reviews:
[{"review_id": "ojkXXzhUjL", "reviewer": "Reviewer_sgZ5", "summary": "In my understanding, the paper proposes an approach for task planning by combining transformer-based models and symbolic solvers. First a transformer model is trained to generate actions that solve a given problem definition, supervised by the solution of the PDDL solver. The transformer is augmented by a procedure codebook, an explicit memory with latent codes that are used to generate the actions. Then, the pretrained LM samples a set of latent procedures and select the procedures that maximize the likelihood of the transformer model. The approach is tested in 3 different environments, modifying existing benchmarks to encode sequences of tasks. The proposed method improves over several LLM-based and symbolic planning based approaches, obtaining higher success rate and lower computation time.", "questions": "- Provide one example to ground the method. State what are the inputs of the LM, and how is M formed to generate the procedural memory R.\n- Better interpretability of what the codebook R is doing. Would it be possible to decode the R codes back into task predicates?\n- How does Equation 1 and 9 relate to each other?\n- How are the positive and negative procedures obtained in the contrastive planning stage?\n- How does the approach compare to works like DualFormer? (https://arxiv.org/abs/2410.09918)", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Very thorough experiments and strong results. The method is compared in 3 different benchmarks in with a large number of baselines. The method obtains significant improvements across the 3 benchmarks compared to both LM-based baselines as well as baselines using symbolic planners. Furthermore, the latent procedures can be reused across tasks, leading to the agent being able to adapt as goals change in the environment, compare to other baselines.\n- Ablations of the effect of different LLMS to generate the symbolic predicates, as well as the effect of the different components of the NeSYPR approach.\n\nWeaknesses:\n\nClarity: In the current state the paper is too difficult to follow to be a valuable contribution for a conference. I would strongly advice to remove abstract statements that do not provide information about the method (e.g. grounding parts of the method on human knowledge) and try to ground the framework in a concrete problem earlier on. The method is stated too abstractly and it is really difficult to follow. Below more details. \n\n- The paper uses a lot of concepts without clear meaning. Why is \"multi-step symbolic structured path-finding and reasoning into single-step LM inference, akin to human knowledge compilation\"? What does \"triggered automatically as cognitive procedures\" (l42) mean? The second paragraph in the introduction doesn't provide any clarity on how declarative vs procedural memory are used in this framework, a concrete example to ground the framework would help improve clarity. The introduction is too abstract and it is really hard to understand what the method is solving or why it enables faster LM inference. \n- The methods in line 64 to 72 are referenced without further context which makes it hard to understand why the stated results are relevant. \n- Equation 1 is introduced without any explanation or reasoning. Why do we want to minimize the KL divergence between the LM policy and the tool policy? Are the outputs of those policies in the same domain? \n- Figure 1 includes a lot of details that are barely described in the main text. \n- Overuse of notation. In equation 2, is the M in the input the same as the M at the output of the decoder block? Is E_self the same as H_{l-1} ? What is M for the first layer?\n- What makes a procedure positive or negative in Equation 10? How are the procedures R+ and R- constructed?\n\nMethod:\n- The method seems overly complex, and doesnt justify some of the decisions. Why learning a codebook end to end with the planning? The general approach is to learn these in two stages. Does this lead to codebook collapse? Did authors follow the more common approach? Why isn't there a reconstruction loss in the VQ loss? The loss in equation 9 does not match the equation 1, what is the relationship between both equations?\n- Is the LM based agent represented in Figure 2? What makes that model a language model? Is it initialized from a pretrained LM? From the text and figure it looks like a transformer.\n- How does the proposed method enable planning under dynamic environments, particularly in changing dynanmics and goals over time? What part of the model is enabling that?\n- If I understand correctly, the input to the NeSyPR model is a PDDL like definition of the current problem and the domain. Where does that come from? How is it extracted from an environment? Is an LLM used to generate that?\n- Why does the process in section 3.2  supress failure patterns? (L169).\n\nResults:\n- Beyond the quantaitive results, it is very hard to evaluate why the propsoed method is doing better. Showing some qualitative examples or understanding how the propsoed approach is performing neurosymbolic reasoning would be helpful.\n- What do the evaluation phases represent in Table 4?\n\n\n- Related work: Some form of related work should be included in the main paper, it is really hard to understand where the presented work fits in the literature. It seems like the paper fits into the second family of approaches (neurosymbolic methods that integrate LMs) but it is unclear what the main differences are.", "was_revised": true, "final_justification": "Based on the author's rebuttal as well as other reviewer's points, the contributions of the proposed method, and the justification of their approach are much clearer to me. I still think that clarity could be greatly improved, but I think that the paper merits acceptance, hence I change my score to borderline accept.", "rebuttal": "Dear Reviewer sgZ5,\n\nWe appreciate the reviewer's feedback. Below, we provide clarifications and responses to the suggestions.\n\n---\n### Clarity weakness ###\n**w1.** NeSyPr transforms multi-step symbolic reasoning into single-step neural inference, substantially reducing inference latency. This process mirrors human proceduralization as described in ACT [1], where repeated experience converts declarative knowledge into fast-access production rules.\n\nIn ACT, production rules (condition–action mappings) are automatically triggered when their conditions match the contents of working memory. Similarly, in NeSyPr, once the agent's working memory encodes the current observation and goal, relevant procedures are retrieved from memory based on similarity (e.g., cosine $\\geq$ 0.95), without requiring external prompting.\n\nNeSyPr separates declarative and procedural knowledge in both representation and execution. During training, declarative knowledge such as search algorithms, transitions, and goal evaluation is used to generate symbolic plans, which are then encoded into vector-quantized procedural memory. At test time, the LM bypasses symbolic reasoning and generates plans by attending to the procedural memory, enabling single-pass inference.\n\n[1] John R Anderson. Cognitive science series. The architecture of cognition. 1983.\n\n**w2.** Lines 64-72 of the Introduction summarize NeSyPr's empirical contributions, organized around the core challenges of structured, adaptive, and timely reasoning. Each cited baseline highlights a specific comparison point: DeepSeek-R1-Distill for structured reasoning, the symbolic planner for adaptive reasoning, and BoT for timely reasoning. In the final version, we plan to enrich this overview by incorporating the detailed baseline descriptions provided in Appendix C.1.4.\n\n**w3.** Eq. (1) includes an imitation learning objective [2], minimizing the KL divergence to align the LM policy with the symbolic planner. As both operate over the same symbolic action space, their outputs are directly comparable.\n\n[2] Wonje Choi et al. Embodied CoT Distillation From LLM To Off-the-shelf Agents. ICML 2024.\n\n**w4.** Figure 1 illustrates the two main phases of NeSyPr: i) Compositional NeSy Procedure Learning and ii) NeSy Procedure Contrastive Planning. The roles and workflows of each phase are outlined in Section 3 (Lines 105-120), and we plan to enhance this explanation by incorporating additional implementation details and algorithmic steps from Appendix B.\n\n**w5.** While we recognize that the notation is dense, we do not believe it constitutes overuse. As addressed in our response to Reviewer cUia's Weakness 4, we will include a notation table in the Appendix to enhance clarity and readability.\n\nIn Eq. (2), working memory $M$ is updated at each layer and passed forward, despite sharing the same symbol. Self-attention output $E_\\mathrm{self}$ is computed by applying Self-attention to the previous layer's hidden state $H_{l-1}$ and is not identical to $E_\\mathrm{self}$. For the first layer, $M$ is initialized using the output from the final layer of the previous inference step.\n\n**w6.** As described in Section 3.2 of the main paper and Algorithm 2 in Appendix B.1, the agent maintains two procedure banks: $\\mathcal{M}^+$ for successes and $\\mathcal{M}^-$ for failures. During inference, each $c_i \\in R$ is compared against both banks using a similarity threshold (e.g., $\\upsilon = 0.95$). If a match is found, $c_i$ is replaced; otherwise, it remains unchanged. This runtime produces $R^+$ and $R^-$ for contrastive planning. After each task, the current procedure is added to $\\mathcal{M}^+$ or $\\mathcal{M}^-$ based on binary feedback.\n\n---\n### Method weakness ###\n**w1.** The goal of our study is to enable the LM to generate action sequences guided by a symbolic planner via single-step inference. This requires end-to-end training of the inputs, memory module, LM, and output sequence [3,4]. In continual learning settings, as in Appendix C.2.4, the learning of the memory module and LM can be decoupled, allowing each to be trained incrementally across stages [5].\n\nTo prevent collapse of the procedure-book $\\mathcal{C}$, we apply stabilization techniques beyond the commitment loss in Eq. (8), including periodic reinitialization of the $\\mathcal{C}$ and careful tuning of EMA hyperparameters. As shown in Table 6, the 'w/o EMA update of $\\mathcal{C}$' variant reduces CSR and CGC by 2-4%, but avoids the collapse.\n\nUnlike conventional VQ-based methods such as VQ-VAE that rely on pixel-level reconstruction losses, our approach optimizes task performance via the behavior log-likelihood term in Eq. (9).\n\nEq. (1) defines the ideal policy objective: maximizing task success while minimizing KL divergence from the symbolic planner. In practice, we approximate this by supervising the LM using planner-computed action sequences and symbolic inputs.\n\n[3] Weizhi Wang et al. Augmenting language models with long-term memory. NeurIPS 2023.\n\n[4] Jikun Kang et al. LM2: Large Memory Models. arXiv 2025.\n\n[5] Jikun Kang et al. Think before you act: Decision transformers with working memory. ICML 2024.\n\n**w2.** Figure 2 shows the architecture of NeSyPr's LM-based agent, with components laid out from left to right. The 'Language Model' in Figure 2 includes a decoder-only Transformer with Decoder Blocks, Self-Attention, and Feed-Forward layers, generating a token sequence as the action plan. As described in Appendix B.2, NeSyPr builds on a pretrained LM and incorporates a memory-augmented module. Table 5 of the main paper demonstrates its compatibility across various pretrained LM families and model scales.\n\n**w3.** In a continual task stream, NeSyPr enables planning in dynamic environments by leveraging contrastive planning, which allows the agent to adaptively reconstruct procedures based on environmental feedback. Using stored experiences of prior successes and failures, the agent applies contrastive planning to guide generation toward reliable behaviors while suppressing failure patterns. As this process repeats, the procedure banks are continually updated, enabling retrieval and refinement of appropriate procedures within a single forward pass, even under changing conditions. This adaptive reasoning capability is demonstrated in Table 4 and Figure 3(b) of the main paper.\n\n**w4.** We directly use benchmarks that provide symbolic PDDL-style definitions, such as PDDLGym, VirtualHome, and ALFWorld, where the domain and problem descriptions are already structured for symbolic planners. No LLM is used to extract or generate these definitions. This design is noted in Section 4.1 of the main paper and detailed in Appendix C.1.1-C.1.3.\n\n**w5.** At each decoding step, NeSyPr computes token distributions conditioned on successful and failed procedures ($p^+$ and $p^-$) and derives a contrastive score by subtracting $p^-$. This suppresses failure-prone tokens and lowers the chance of repeating invalid plans.\n\n---\n### Results weakness ###\n**w1.** In Figure 4 of the main paper, we visualize procedure-book usage patterns as a heatmap to illustrate how different procedure-units are composed and reused across four task types. Tasks with similar logics activate overlapping units, while object-specific tasks use more diverse ones. We also plan to include side-by-side examples of successful and failed plans to help clarify the sources of performance gains.\n\n**w2.** Table 4 evaluates how the agent adapts over time in a continual task setting. We measure forward transfer (benefits to future tasks) and backward transfer (impact on past tasks) as indicators of adaptation. The agent solves tasks sequentially, and the full test set is periodically re-evaluated across 15 checkpoints to reflect accumulated procedural knowledge.\n\n**w3.** We acknowledge that the Related Work section in the main paper is a condensed summary of the full discussion in Appendix A, where prior work is grouped into three areas: (i) agentic frameworks, (ii) neurosymbolic methods, and (iii) memory-augmented LMs, and NeSyPr is positioned in contrast to each. In the final version, we plan to expand this section by incorporating key theoretical background, including ACT theory, into the main paper.\n\n---\n### Questions ###\n**q1.** The input includes observation, goal, and domain knowledge, all represented as PDDL strings (examples are in Appendix C.1). The working memory $M$ encodes the current state via memory-augmented cross-attention, and the runtime procedure $R$ is obtained by vector-quantizing $M$ using the procedure-book $\\mathcal{C}$.\n\n**q2.** $R$ is derived from the procedure-book $\\mathcal{C}$, which encodes implicit production rules as composable latent representations. While not symbolically interpretable, $R$ integrates seamlessly into LM inference. We provided behavioral analysis in Figure 4 of the main paper to offer insight into its role.\n\n**q3.** Please refer to our response to Method weakness w1.\n\n**q4.** Please refer to our response to Clarity weakness w6.\n\n**q5.** We thank the reviewer for suggesting this relevant baseline. Although DualFormer was published recently (ICLR 2025), it is a meaningful baseline in the context of proceduralization.\nNeSyPr is inspired by ACT, compiling declarative knowledge into procedural memory and adaptively reconstructing it for reasoning in embodied environments. In contrast, DualFormer follows a System-1/2 framework, learning from randomly sampled reasoning traces to train a single Transformer that dynamically adjusts its reasoning depth based on task difficulty.\nWe provide a comparison between DualFormer and NeSyPr in the Minecraft environment.\n\n|Method|Mode|Train CSR|Train CGC|Test CSR|Test CGC|Latency (s)|\n|-|-|-|-|-|-|-|\n|NeSyPr||100.0|100.0|65.2|68.9|4.7|\n|DualFormer|Fast|100.0|100.0|52.2|63.2|5.2|\n||Slow|100.0|100.0|55.0|65.8|6.2|\n||Auto|100.0|100.0|55.3|64.2|5.8|\n---\nWe hope our responses clarify the contributions of our work."}, {"review_id": "ukZ7QCG5Mv", "reviewer": "Reviewer_La85", "summary": "The paper presents NESYPR, a neurosymbolic proceduralization based reasoning framework inspired by ACT, which compiles multi-step symbolic reasoning into single-step LM inference. The paper claims this framework eliminates the need for online symbolic planners in embodied tasks. Additionally, the paper develops compositional NeSy procedure learning, which encodes production rules into a vector quantized procedural memory whose vectors can be compositionally combined to generate task specific plans. The paper also implements NeSy procedure contrastive planning, which adaptively generates plans by contrastively reconstructing task-specific procedures from stored procedures labeled with environmental feedback. The paper shows effectiveness and efficiency of NESYPR through extensive\n evaluations on 3 embodied benchmarks and 9 experimental scenarios.", "questions": "None", "limitations": "The proposed method is making a non-standard change to the LLM architecture which likely leads to catastrophic forgetting of pretrained knowledge of the LLM which limits applicability of this method to more general problems and domains.\n\nI think the paper itself is interesting and has a bunch of interesting experiments. I am willing to increase my rating if authors can clarify the questions I asked in weaknesses section and add additional ablations.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The paper is well written and easy to follow\n2. The proposed method outperforms strong baselines on all the benchmarks and tasks that authors consider for task planning in the paper which shows effectiveness of the method.\n3. Additionally, the method is also efficient at inference time and performs effectively under a time constraint compared to symbolic planners.\n4. The analysis in figure 4 also shows neat results where procedure codebooks across tasks show shared features\n5. The ablations provided in the paper are also thorough and valuable to show effectiveness of the method.\n6. Additionally, the analysis in figure 5 is also insightful. The finding that increasing codebook size leads to drop in composition performance is a interesting finding.\n\nWeaknesses:\n\n1. In Table 3 why does Deepseek 8B model (row 4) have a higher latency at almost 1/2 number of tokens compared to llama 3.1 8b (row 1)? This seems counterintuitive. Can authors explain why this is the case? Instead of latency isn’t it better to compare efficiency by FLOPs? Can author show similar analysis using FLOPs?\n2. One thing that is a bit unclear is the effectiveness of pretrained representations which come from using a pretrained language model when learning the procedural memory for neurosymbolic planning. I understand the motivation to equip LLMs with neurosymbolic planning ability with procedural memory but the current experiments do not ablate the effectiveness of using a pretrained LLM as initial starting weights vs using a LLM from scratch to learn such neurosymbolic planning. Currently, it is unclear whether the gains in performance are coming from pretrained representation + architecture change vs just the changes in architecture and training recipe. I would appreciate it if authors can run this experiment as I believe it will make the paper stronger.", "was_revised": true, "final_justification": "Thanks to the authors for addressing my concerns. The new experiment results presented in rebuttal address my concerns. I'd recommend authors to add this to the main paper. I have updated my rating to reflect the same.", "rebuttal": "Dear Reviewer La85,\n\nThank you for your thoughtful and constructive review. We appreciate the opportunity to address your questions in detail and provide additional analysis to clarify our contributions.\n\n---\n\n### **Weakness 1** ###\n\nWe appreciate the reviewer's observation.\nThe latency difference between BoT (LLaMA3.1-8B) and LRM (DeepSeek-R1-8B) in Table 3 of the main paper may seem counterintuitive, but it becomes clear when comparing the input and output token lengths, as shown in Appendix Table 11. For reference, the evaluation results from Appendix Table 11 are reproduced in the table below.\nBoT uses long prompts as input, leading to many input tokens but relatively short outputs.\nIn contrast, LRM starts with short prompts but generates a much larger number of output tokens due to its multi-step reasoning.\n\nIn decoder-only models, input tokens are processed in a single forward pass, while output tokens are generated sequentially via autoregressive decoding. As a result, output length becomes the dominant factor in inference latency.\nIn the case of LRM, although it uses shorter input prompts than BoT, it generates significantly more output tokens due to its multi-step reasoning strategy.\nAs a result, the higher number of generated tokens in LRM leads to substantially higher plan generation latency.\n\nTo provide a hardware-independent measure of efficiency, we agree that reporting the computational cost in terms of floating point operations (FLOPs) is more appropriate.\nAccordingly, we report the FLOPs for BoT, LRM, PlaSma, and NeSyPr using the LLaMA3.1-8B backbone, measured with the official PyTorch profiler on an NVIDIA A100 server.\nThe profiler was applied to plan generation runs using task inputs from the same benchmark as in Table 3 of the main paper. The results are summarized in the table below.\n\n|Method|CSR (%)|Latency (s)|In Tokens|Out Tokens|FLOPs|\n|-|-|-|-|-|-|\n|BoT|53.0|59.5|8007.9|1315.4|86.8+$\\alpha$ TFLOPs|\n|LRM|11.5|111.0|3198.5|2187.6|55.4 TFLOPs|\n|PlaSma|80.5|18.4|3371.0|122.4|236.9 TFLOPs|\n|NeSyPr|89.0|5.2|3155.5|41.9|98.3 TFLOPs|\n\nWhile NeSyPr incurs slightly higher FLOPs than simpler baselines such as LRM, this overhead stems from its memory-augmented module and contrastive planning mechanisms, which are designed to enhance robustness and correctness in reasoning.\nIt is important to note that in our implementation, the problem distillation step of BoT is performed by GPT-4.1 regardless of the base LM, as described in Appendix C.1.4. Consequently, this additional step is excluded from FLOPs measurement, and we denote it as \"+$\\alpha$\" to reflect the unaccounted cost—potentially underestimating the actual computational burden of BoT.\nIn contrast to prompt-based or distillation-based approaches, NeSyPr actively composes and validates procedures during inference, trading off minimal compute overhead for significantly higher planning accuracy and consistency.\n\nDespite this moderate FLOPs usage, NeSyPr achieves the best trade-off across success rate, latency, and token efficiency. This demonstrates that structured reasoning can remain performant without relying on excessively large models or token-intensive decoding.\n\nFurthermore, as reported in Appendix Table 12, NeSyPr maintains fast inference times (e.g., 5.0s on Jetson Orin) and stable performance even on embedded devices, confirming that the compute overhead remains practical for real-world deployment. The table below references the same experimental results from Appendix Table 12.\n\n|Device|Latency Min (s)|Latency Max (s)|Latency Avg (s)|Mem Min (GB)|Mem Max (GB)|Train SR|Train GC|Seen SR|Seen GC|Unseen SR|Unseen GC|\n|-|-|-|-|-|-|-|-|-|-|-|-|\n|RTX A6000|0.5|5.1|1.5|0.0|1.9|97.4|97.7|88.4|89.6|61.5|67.9|\n|RTX 4090|0.5|4.5|1.0| 0.0|1.9|96.1|98.7|88.4|89.7|61.5|68.5|\n|RTX 3090|0.5|4.8|1.2|0.0|1.9|92.2|94.6|86.6|87.8|61.5|68.5|\n|RTX 3050|0.6|7.7|1.6|0.0|1.9|94.8|96.9|84.8|86.5|61.5|67.9|\n|Jetson Orin|2.0|11.4|5.0|4.3|8.6|93.5|96.1|87.5|88.7|59.6|66.6|\n\nDue to the inherent overhead of profiling, we were unable to replicate the full-scale evaluation for all settings in Table 3 of the main paper, but we plan to extend this analysis with more comprehensive measurements in the final version.\n\n---\n\n### **Weakness 2** ###\n\nWe fully agree that disentangling the effects of pretrained representations and architectural design is essential to understanding the source of performance gains in NeSyPr.\n\nOur implementation adopts a parameter-efficient LoRA-based finetuning [1] setup, which preserves the pretrained weights and updates only lightweight adapter layers, as described in Appendix C.1.4. This design choice was made to address the low-data regime, where full finetuning or training from scratch may lead to instability or inefficiency. We clarify this rationale more explicitly in the revised version.\n\nAdditionally, to directly address the reviewer's concern, we conducted an ablation study comparing three LM training configurations: (1) training from scratch, (2) full finetuning, and (3) LoRA-based finetuning.\nEach configuration was evaluated both with and without the NeSyPr architecture, using the LLaMA‑3.2‑1B model on the PDDLGym benchmark. The results are presented below:\n\n|Domain|\\||LM-Scratch||\\|||LM-FullFinetune||\\|||LM-LoRA||\\|||NeSyPr-Scratch||\\|||NeSyPr-FullFinetune||\\|||NeSyPr-LoRA||\\|||\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n||\\||SR|GC|\\|||SR|GC|\\|||SR|GC|\\|||SR|GC|\\|||SR|GC|\\|||SR|GC|\\|||\n|Minecraft|\\||0.3|39.4|\\|||2.0|31.2|\\|||75.7|78.4|\\|||0.4|57.5|\\|||6.1|39.6|\\|||82.6|84.5|\\|||\n|Rearrangement|\\||2.6|29.3|\\|||5.0|44.3|\\|||78.3|84.9|\\|||3.0|45.6|\\|||20.0|49.8|\\|||86.8|90.4|\\|||\n|Glibrearrange|\\||0.0|39.5|\\|||1.2|49.0|\\|||73.3|78.6|\\|||2.4|40.4|\\|||10.3|55.4|\\|||79.4|86.2|\\|||\n\nAcross all training configurations, including the from-scratch setting, the addition of procedural memory consistently improves performance and highlights the effectiveness of structured memory-based reasoning.\nWith the architecture fixed, using pretrained weights improves NeSyPr's performance over training from scratch, highlighting the value of pretrained representations in proceduralization.\nGiven the same pretrained initialization, augmenting the model with procedural memory (i.e., transitioning from LM-LoRA to NeSyPr-LoRA), as well as preserving pretrained weights while adding procedural memory (i.e., transitioning from NeSyPr-FullFinetuning to NeSyPr-LoRA), leads to further performance gains. This suggests a synergistic effect between pretrained representations and procedural memory.\n\nWe believe this ablation further supports the complementary roles of pretrained representations and procedural memory in NeSyPr.\n\n[1] Hu, Edward J., et al. \"Lora: Low-rank adaptation of large language models.\" ICLR 2022.\n\n---\n\n### **Limitation** ###\n\nWhile we acknowledge that catastrophic forgetting is a well-known challenge in continual learning and LM finetuning, we clarify that our method introduces no parameter modification to the backbone LM. NeSyPr employs a parameter-efficient LoRA-based finetuning strategy that keeps the pretrained weights frozen, while augmenting the model with an external procedural memory module. This design ensures that the pretrained knowledge within the LM remains intact throughout both training and inference.\n\nEmpirically, we observe that the LLaMA-3.2‑1B backbone retains its pretrained capabilities, with no change in perplexity on WikiText-103 before and after training (16.4 ppl). Since the procedural memory operates as a complementary reasoning module that encodes symbolic abstractions of previously successful solutions, it guides planning without modifying the LM's internal parameters.\n\nAdditionally, our framework demonstrates robustness in both continual adaptation and continual learning scenarios.\nAs shown in Table 4 of the main paper, NeSyPr achieves strong performance in the continual adaptation scenario, maintaining stable success rates over 15 evaluation phases, with positive forward and backward transfer, zero forgetting, and consistent recovery.\nIn a separate continual learning scenario, Appendix Figure 5 shows that NeSyPr-Incremental, which gradually expands procedural memory over time, achieves steady improvement across 10 learning phases and eventually approaches the performance of an oracle trained on all tasks at once.\nTogether, these results confirm that our method not only avoids catastrophic forgetting of pretrained knowledge but also enables continual retention and reuse of experience in dynamic embodied environments.\n\n---\n\nWe have carefully addressed the concerns raised in the weaknesses section and included additional ablation studies as recommended. We greatly appreciate your thoughtful review and hope these updates provide a clearer understanding of our contributions."}, {"review_id": "w065bRRc7p", "reviewer": "Reviewer_cUia", "summary": "This work introduces a neuro-symbolic framework that distills compositional procedure knowledge from symbolic solvers to a language model. During inference, the system can avoid using symbolic solvers and leverage contrastive planning to generate solutions for unseen tasks. The authors also conduct exhaustive experiments and analysis to demonstrate the effectiveness of their method.", "questions": "- I see one potential weakness is the reliance on existing symbolic solvers during training time, which means the system can only solve problems with defined symbols/structure -> It is just making planning more efficient by learning. Have the authors considered distilling LM's knowledge back to the symbolic solver / guide the solver so it could potentially solve more tasks that previously had no existing domain knowledge? Doing this in a cycle to continually evolve the LM seems to be very interesting.\n- For PDDL representations, there exist domains where procedural knowledge (efficient imperative rules/feedforward programs if I understand correctly) does not exist, e.g., Sokoban. What is the procedural knowledge learned/computed there?", "limitations": "See questions.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength**\n- Encoding procedure rules into implicit LM knowledge is an interesting idea, and contrastive planning during inference is also intriguing to me.\n- The experimental supports are quite strong, which cover various benchmarks and baselines.\n- The problem addressed in this work is important to me, i.e., compositional generalization of neural networks.\n\n**Weaknesses**\nThe major weakness of this work is the writing issue. \n- The experimental section is 4 pages long, making the related works + methodology only 2.5 pages long (with figures). This makes the information in the paper largely biased, and readers can hardly understand the technical details of your algorithm. I'd recommend put some experimental content in the appendix and making more room for the method writing (~3 pages). Here are something I feel confused after trying to carefully read the method:\n- What is the input to the LM during training? Is it the plan from the symbolic engine/problem description? What is the specific state space? There should be a running example here.\n- How is the procedural knowledge/Rule computed, and what is its format? Is it based on LLM's summarization/ symbolic induction? Please add an example for understanding.\n- There are heavy notations across the paper; the authors are expected to provide a notation table.\n- Are the unseen tasks from the same domain but with different entity composition? Or are the unseen tasks from different domains but same types?", "was_revised": true, "final_justification": "The direction that this paper is exploring is interesting, and the clarifications have resolved my concerns. I would like to maintain my score.", "rebuttal": "Dear Reviewer cUia,\n\nThank you for taking the time to review our paper. Your thoughtful and constructive feedback has helped identify areas for improvement. Below, we provide detailed responses to the weaknesses and questions you raised.\n\n---\n\n### **Weakness 1** ###\n\nWe recognize that the current presentation of the methodology is somewhat compact and that a more detailed ​explanation would improve clarity.\n\nWhile Sections 2 and 3 in the main paper are relatively compact, we would like to point out that further technical details are included in the Appendix.\n\n- The **related work** is expanded to cover recent advances in agentic frameworks, neurosymbolic methods, and memory-augmented LMs, with comparisons that highlight differences in our approach, as detailed in Appendix A.\n\n- The core **algorithms** are described in detail, including the Compositional NeSy Procedure Learning and Contrastive Planning phases in Appendix B.1.\n\nSince the Appendix was submitted separately from the main paper, we were not able to include explicit cross-references. We will address this in the final version to help readers navigate the supplementary content more easily.\n\nBased on your comment and that of Reviewer sgZ5, we recognize that additional clarification on the ACT theory and its connection to our framework would improve clarity. We will briefly summarize the key components of ACT (e.g., procedural and declarative memory) and clarify how they are implemented in our framework through Compositional NeSy Procedure Learning and NeSy Procedure Contrastive Planning.\n\nWhile we focused on demonstrating the effectiveness of our framework, we will adjust the allocation to better explain the core components and technical details of our method.\n\n---\n\n### **Weakness 2** ###\n\nDuring the training phase (Compositional NeSy Procedure Learning), as described in Section 3 (lines 110–111), the input to the LM consists of:\n\n- Observation $o$: A symbolic description of the environment state, represented using predicates (e.g., `obj_next_to(plate kitchen_counter)`)\n\n- Goal $g$: A symbolic goal specification (e.g., `(and (open freezer) (obj_inside food_food freezer))`)\n\n- Domain Knowledge DK: Action rules with preconditions and effects that define the action space and dynamics, as shown in Appendix C.1\n\nThese inputs are processed by the LM together with working memory, as described in Eq. (3-4) and Appendix Algorithm 1. Representative input examples are shown in Appendix C.1.\nThe model is trained to generate action plans that are aligned with the symbolic planner's inputs and outputs.\n\nThe state space is defined as the set of all grounded PDDL states. Each state corresponds to a specific truth assignment over the instantiated predicates for the given set of objects in the environment. These symbolic facts represent the current environment configuration in each problem instance.\n\nTo illustrate this concretely, Appendix C.1.1-C.1.3 includes some examples for each benchmark:\n\n- PDDLGym: The state includes facts such as `(equipped new-2 agent)` and actions like `equip(log-2 agent)`.\n\n- VirtualHome: The state includes facts like `(holds_lh character tooth_paste)` and actions like `switch_on (character:character, computer:object)`.\n\n- ALFWorld: The state includes relational predicates like `(inreceptacle ?o1 ?r)` or `(iscool ?o)`, and actions such as `pickupobject(agent1:agent, loc_21:location, tomato_1:object, countertop_1:receptacle)`.\n\nAll inputs follow PDDL-style representations.\n\nWe will expand Appendix C.1 to include additional detailed examples with full symbolic observations, goals, and domain knowledge.\n\n---\n\n### **Weakness 3** ###\n\nThe procedural knowledge in our framework is not directly summarized by the LM nor handcrafted. Instead, it is learned by encoding the symbolic planner's output (e.g., action plans) into a vector-quantized procedural memory during training.\n\nSpecifically, the declarative knowledge used by symbolic tools, such as search algorithms, state transitions, cost estimation, and goal evaluation, is internalized as production rules via a memory-augmented module (Eq. (3-4)) and compressed into discrete procedure-units through vector quantization (Eq. (5-6)). These units form a composite vector, which is stored in a procedure memory (Figure 2).\n\nAs a result, procedural knowledge is represented as quantized latent vectors that implicitly encode condition-action patterns, rather than explicit symbolic rules. These vectors are compositional, reusable across tasks, and grounded in experience.\n\nInspired by ACT theory, these latent procedures can be seen as production rules that are probabilistically activated based on task-specific context. This structure allows our framework to acquire and reuse implicit condition-action mappings through accumulated interaction, closely resembling proceduralization in cognitive architectures.\n\nWhile these procedures are not symbolically interpretable, we provide an alternative perspective through behavioral analysis. In Figure 4 in the main paper, we visualize the usage frequency of procedure-units across different task types and scenes.\nThe results show that tasks with similar solution structure activate consistent subsets of the procedure-book, even across different environments. In contrast, task types involving object-specific interactions lead to more divergent usage patterns.\n\n---\n\n### **Weakness 4** ###\n\nWe agree that the use of formal symbols throughout the paper could benefit from a centralized summary for easier reference. We will include a notation table summarizing the key symbols and their meanings, as shown below:\n\n|**Symbol**|**Meaning**|**Section**|\n|-|-|-|\n| $\\pi_{\\text{LM}}$ | LM-based policy for task planning | §2, Eq. (1) |\n| $M$ | Working memory: vector slots for input context | §3.1, Eq. (2) |\n| $\\mathcal{C}$ | Procedure-book: codebook of discrete procedure-units | §3.1, Eq (5-6) |\n| $R$ | Runtime procedure: composition of quantized procedure-units | §3.1, Eq. (7) |\n|$\\cdots$\n\n---\n\n### **Weakness 5** ###\n\nIn all benchmarks, unseen tasks belong to the same domain but involve novel goal types or previously unseen combinations of objects and predicates, requiring generalization beyond the training distribution. As detailed in Appendix C.1.2 and C.1.3, in VirtualHome and ALFWorld, unseen tasks introduce entirely new task types, requiring the agent to adapt and generalize based on previously acquired procedural knowledge.\n\n---\n\n### **Question 1** ###\n\nOur framework indeed leverages symbolic planners during training to generate optimal plans for proceduralization. As such, it operates in domains where symbolic action rules and typed predicates are defined. However, once training is complete, the model no longer depends on symbolic tools. Instead, it performs adaptive reasoning based on procedural knowledge stored in memory, enabling the agent to reason efficiently without relying on large-scale inference or online access to symbolic tools.\n\nAs shown in Figure 3, our model even outperforms symbolic planners under dynamic and partially observable conditions. For example, under tight time constraints, NeSyPr achieves similar or higher success rates while planning significantly faster. In settings with incomplete input, symbolic planners often fail due to unmet preconditions, whereas NeSyPr generalizes effectively by retrieving relevant procedures from memory. These results suggest that proceduralization enables generalization beyond pre-defined domain knowledge by leveraging LM capabilities and grounding knowledge through task experience.\n\nAdditionally, we fully agree that enabling the LM to inform or refine symbolic tools in a closed-loop setup is a promising direction. This aligns with recent work, NeSyC [1], which explores continual interaction between symbolic tools and LLMs for constructing generalized symbolic knowledge.\n\nWe are planning a follow-up study in this direction. As an initial step, Appendix C.2.3 presents continual learning experiments demonstrating that NeSyPr remains effective across evolving task distributions. Building on this, we aim to extend the framework to continual online environments, where sub-optimal trajectories from the LM agent can be refined by symbolic tools via decomposition or relabeling, and the improved knowledge can be fed back to the agent for further generalization.\n\n[1] Choi, W. et al. \"NeSyC: A Neuro-symbolic Continual Learner for Complex Embodied Tasks In Open Domains\". ICLR 2025.\n\n---\n\n### **Question 2** ###\n\nSokoban represents one of the most challenging domains for proceduralization, as it requires long-horizon planning, precise spatial reasoning, and backtracking-based solution strategies, with few reusable subgoals or feedforward rules. Small variations in state often lead to completely different solution paths, making it hard to define general procedural abstractions in an interpretable form.\n\nWe additionally tested NeSyPr on Sokoban using benchmarks from [2] to assess its performance in procedurally challenging domains. Although the learned procedures are not explicitly interpretable, they capture structural patterns that support generalization across similar Sokoban instances without relying on subgoals or feedforward rules.\n\n|Sokoban|Success Rate|\n|-|-|\n|Train|100%|\n|Test|30.5%|\n\nWe note that Sokoban differs from our main focus on embodied domains in that it lacks modularity and procedural structure, making proceduralization particularly difficult. Adapting NeSyPr to such settings may require a distinct approach, and we appreciate the reviewer’s insight in identifying this as a valuable direction for future work.\n\n[2] Su, DiJia, et al. \"Dualformer: Controllable fast and slow thinking by learning with randomized reasoning traces\". ICLR 2025.\n\n---\nWe hope our responses have adequately addressed your concerns, and we would be happy to provide further clarification if needed. Thank you again for your valuable review."}, {"review_id": "ThFH6D9Jye", "reviewer": "Reviewer_fttA", "summary": "This paper introduces NESYPR, a framework that helps language models plan actions in embodied environments without relying on external symbolic planners. It learns from symbolic plans during training and compiles them into reusable procedures stored as vector memories. At test time, the model uses these memories—and feedback from the environment—to adapt and generate plans efficiently. The key idea is to turn multi-step symbolic reasoning into fast, single-step neural inference.", "questions": "1. Have you tried training NESYPR on one environment (e.g., PDDLGym) and testing on a structurally different one (e.g., ALFWorld)? This would provide stronger evidence for generalization beyond task-level variation.\n\n2. Given that the unseen tasks come from the same environment as the training tasks, have you considered using few-shot prompting baselines, such as GPT-4 with training examples as prompts? \n\n3. Could you provide more detailed ablations of the contrastive planning component? For instance, what happens when using only successful memory (M+) or changing how memories are retrieved?\n\n4. Do you have any insights into what the procedural memory vectors encode? Are they interpretable or reusable in a symbolic form?", "limitations": "While NESYPR shows strong performance within individual environments, its ability to generalize across environments with different symbolic structures remains unclear. The evaluation also lacks comparisons to prompt-based baselines using few-shot examples from training tasks, which could offer competitive alternatives. Additionally, the contrastive planning component, though promising, would benefit from a more thorough ablation to clarify its exact contribution.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nA major strength of the paper is that it avoids online symbolic planners during inference, yet still achieves strong performance. What's impressive is that NESYPR runs significantly faster than both symbolic planners and large-scale reasoning baselines like BoT or GPT-4, while using much smaller models (e.g., 3B vs. 70B), as demonstrated through latency and token comparisons.\n\nThe idea of combining contrastive reconstruction with symbolic plan generation is novel and well-motivated. It offers a promising direction for making planning more adaptive without relying on traditional solvers.\n\nThe authors run NESYPR across several popular embodied benchmarks with both seen and unseen tasks. The results are solid and consistent within each environment, suggesting the approach generalizes well to new tasks in familiar domains.\n\nWeaknesses\n\nWhile the model is evaluated on unseen tasks within the same environment, it is unclear how well it generalizes across environments with different object vocabularies, action schemas, or symbolic representations. A stronger demonstration of generalization would involve training on one environment and testing zero-shot on another.\n\nSince the unseen tasks come from the same environment as the training tasks, it would be natural to construct few-shot prompts using training examples and evaluate models like GPT-4 in an in-context setting. This kind of prompt-based reasoning is a competitive alternative to finetuning or memory-based methods, especially with today's large models. Without such a comparison, it's hard to tell how much of NESYPR's performance comes from its memory mechanism versus simply reusing examples in a smarter prompt.\n\nThe contrastive planning mechanism is a core contribution, but the ablation study on it is fairly limited. Table 2 shows results with and without contrastive decoding, but doesn't explore variants (e.g., what happens if only M+ is used, or if memory is retrieved differently). A deeper analysis would help isolate where the performance gains are truly coming from.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer fttA,\n\nThank you for your thoughtful and constructive review. We appreciate the opportunity to clarify our contributions and address your concerns in detail.\n\n---\n\n### **Weakness 1 and Question 1** ###\n\nWe appreciate the reviewer's suggestion to evaluate cross-environment generalization. While the unseen tasks originate from the same benchmark, we would like to argue that they already exhibit substantial variation in object configurations, goal predicates, and task structures as described in Appendix C.1.1-C.1.3. These instances introduce entirely novel task types that are not present in the training data, requiring the agent to recombine and reinterpret previously acquired procedural knowledge in a zero-shot manner.\n\nWe agree that generalization across environments (e.g., training on a domain in PDDLGym and testing on ALFWorld) is an important direction for future work.\nHowever, such transfer is non-trivial, as it involves differences in symbolic engine versions and representation formats as shown in Appendix C.1.1-C.1.3.\nAddressing these discrepancies would require additional symbolic alignment mechanisms, which are orthogonal to our primary focus on memory-based proceduralization within a unified neurosymbolic framework.\n\nHowever, in response to the reviewer's suggestion, we additionally conducted cross-domain evaluation within PDDLGym, where all domains share the same symbolic planner. This setting provides a controlled evaluation for cross-domain generalization without symbolic incompatibility.\n\n|Test Domain|CSR|CGC|\n|-|-|-|\n|Rearrangement (source)|84.9|95.7|\n|Minecraft (target)|20.7|25.4|\n|Glibrearrangement (target)|48.8|82.7|\n\nInterestingly, even within this aligned symbolic setting, transfer across domains (e.g., training on Rearrangement and testing on Minecraft) remains non-trivial.\nThis suggests that domain-specific reasoning patterns, such as variations in precondition/effect structures, goal dependencies, and procedural depth, introduce meaningful challenges for cross-domain adaptation.\nDespite the overall drop in performance, we observe that NeSyPr can still generate partially coherent plans in the target domain.\nFor instance, in the Minecraft domain, NeSyPr generates domain-specific actions such as `craftplank`, which were never seen during training on Rearrangement, as part of accurate action sequences.\n\nFigure 3 in the main paper illustrates that our evaluation already encompasses scenarios requiring generalization beyond pre-defined domain knowledge. NeSyPr even outperforms symbolic planners under dynamic and partially observable conditions. In settings with incomplete input, symbolic planners often fail due to unmet preconditions, whereas NeSyPr generalizes robustly by retrieving relevant procedures from memory. These results suggest that NeSyPr supports generalization beyond pre-defined domain knowledge by leveraging LM capabilities and grounding knowledge through task experience.\n\n---\n\n### **Weakness 2 and Question 2** ###\n\nWe fully agree that prompt-based reasoning approaches are strong alternatives to memory-augmented methods, especially when using powerful models such as GPT-4.\n\nAs stated in the Introduction, our goal is to perform embodied reasoning efficiently, without online access to large-scale inference or symbolic tools. While prompting large LMs like GPT-4 can yield strong performance in certain settings, these methods typically assume online access, long prompts, and high inference costs. By contrast, our focus is on achieving comparable effectiveness under resource constraints, which is critical for embodied agents operating in latency-sensitive and resource-constrained physical systems.\n\nTo evaluate the trade-off between effectiveness and efficiency, as raised by the reviewer, we compared NeSyPr with strong prompt-based methods leveraging large-scale LMs.\nIn Table 3 of the main paper, we compare NeSyPr with BoT using GPT-4.1 and LRM using o3-mini. Additionally, we evaluate LLM-Planner, a representative retrieval-augmented in-context learning approach, implemented with GPT-4.1.\n\n|Method|LM|CSR|\n|-|-|-|\n|LLM-Planner (3-shot)|GPT-4.1|81.5|\n|LLM-Planner (8-shot)|GPT-4.1|82.0|\n|BoT|GPT-4.1|92.1|\n|LRM|o3-mini|78.9|\n|NeSyPr|LLaMA3.1–8B|89.0|\n\nNeSyPr achieves performance comparable to or better than much larger prompt-based methods, without requiring extensive prompt engineering, long context windows, or costly inference. In addition, it operates with significantly lower reasoning overhead, including reduced token usage and latency, as detailed in Appendix Table 11.\n\nAdditionally, our evaluation includes ten prompt-based approaches:\n\n- Single-step prompting: ZSP, RAP, LLM-Planner\n- Multi-step reasoning: CoT, ToT, GoT\n- Agentic prompting: ReAct, Reflexion\n- Large model prompting: BoT (GPT-4.1), LRM (o3-mini)\n\nThese are reported in Tables 1, 2, and 3 of the main paper and described in detail in Appendix C.1.4.\n\nThe performance gap between NeSyPr and these baselines, especially under constrained resources, highlights the contribution of procedural memory beyond what can be achieved through prompt reuse alone.\n\n---\n\n### **Weakness 3 and Question 3** ###\n\nWe agree with the reviewer that a deeper ablation analysis of the contrastive planning mechanism is essential for understanding where performance gains originate. As reported in Table 6 of the main paper, we conducted seven ablation variants that directly address the reviewer's points, including configurations such as using only the successful procedure bank $\\mathcal{M}^+$ and modifying the memory retrieval strategy. The table below summarizes key results from Table 6.\n\n|Method Variant|CSR|CGC|\n|-|-|-|\n|NeSyPr (Full)|65.2|68.9|\n|NeSyPr w/ $\\mathcal{M}^+$ only|63.2|65.1|\n|NeSyPr w/ $\\mathcal{M}^-$ only|63.3|66.4|\n|NeSyPr w/o CP, follow $\\mathcal{M}^+$|60.9|64.4|\n|NeSyPr w/o CP, follow $\\mathcal{M}^-$|59.7|62.2|\n|NeSyPr w/o CP|59.9|62.9|\n\nIn particular:\n\n- **NeSyPr (Full)** uses the complete contrastive planning mechanism with both $\\mathcal{M}^+$ and $\\mathcal{M}^-$ and the learned procedure-book $\\mathcal{C}$.\n- **NeSyPr w/ $\\mathcal{M}^+$ only** uses only successful procedures during contrastive planning.\n- **NeSyPr w/ $\\mathcal{M}^-$ only** uses only failure procedures during contrastive planning.\n- **NeSyPr w/o CP, follow $\\mathcal{M}^+$** disables contrastive planning and directly follows the most similar procedure from $\\mathcal{M}^+$.\n- **NeSyPr w/o CP, follow $\\mathcal{M}^-$** disables contrastive planning and directly follows the most similar procedure from $\\mathcal{M}^-$.\n- **NeSyPr w/o CP** removes contrastive planning entirely.\n\nThese configurations, along with other ablations, directly assess the role of contrastive planning as well as the individual contributions of the successful and failure procedure banks.\n\nAs summarized in Table 6, contrastive planning yields a 5.3% improvement in CSR, and further removing contrastive planning or relying solely on one type of memory results in additional performance degradation. These findings highlight the importance of contrastive evaluation between positive and negative procedures for robust and adaptive plan generation.\n\n---\n\n### **Question 4** ###\n\nOur procedural memory is deliberately designed to encode abstract, reusable task knowledge in a latent vectorized form. Each vector in $\\mathcal{C}$ represents procedural abstractions such as precondition–effect dependencies, goal schemas, and operator compositions, in a format that is not directly interpretable but remains functionally composable during inference.\nDuring inference, the agent retrieves and recombines these vectors based on contextual similarity through contrastive planning, enabling the generation of valid symbolic action sequences without relying on the symbolic planner.\nThis design aligns with our ACT-inspired perspective, in which procedural knowledge is retained as implicit production rules that are probabilistically activated depending on the task context.\nAs a result, procedural knowledge is represented as quantized latent vectors that implicitly encode condition-action patterns, rather than as explicit symbolic rules. These vectors are compositional, reusable across tasks, and grounded in experience, closely resembling proceduralization in cognitive architectures.\n\nAlthough individual procedure vectors are not directly interpretable in symbolic form, our behavioral analysis suggests they encode meaningful task structure. As shown in Figure 4 of the main paper, tasks with similar solution strategies consistently activate similar subsets of the procedure-book, even across different environments. In contrast, tasks involving object-specific interactions show more divergent activation patterns, indicating that the procedure-book captures structure relevant to task generalization.\n\n---\n\n### **Limitation** ###\n\nWe kindly refer the reviewer to our previous responses for clarification on the following points:\n\n- As discussed in our first response, we conducted cross-domain transfer within PDDLGym under a controlled setup. While generalization across different environments is beyond the scope of our current focus, we agree it is a valuable direction for future work.\n\n- As clarified in the second response, we compare NeSyPr with strong prompt-based methods using GPT-4.1 and o3-mini. Despite using significantly smaller models, NeSyPr achieves comparable or better task performance while requiring no prompt engineering, shorter context windows, and significantly lower inference cost. These results highlight that our memory-based proceduralization offers a more efficient and scalable alternative to prompt-based reasoning with large LMs.\n\n- As clarified in the third response, we already provide a detailed ablation study covering seven variants to isolate the contributions of contrastive planning and memory components.\n\nWe believe these points have been sufficiently addressed in our responses."}]

Author rebuttal:

Meta-review: This paper presents NeSyPr, a neurosymbolic framework that compiles symbolic planning knowledge into procedural memory for efficient embodied reasoning without online symbolic planners. While reviewers initially raised concerns about clarity, computational claims, and generalization, the authors provided comprehensive rebuttals with additional experiments demonstrating genuine efficiency gains, cross-domain transfer capabilities and thorough baseline comparisons. Despite limitations in cross-environment generalization and procedural memory interpretability, the work addresses an important practical problem with a novel approach grounded in cognitive architecture theory, supported by comprehensive evaluation across multiple benchmarks. The substantial efficiency improvements for resource-constrained embodied systems, combined with the authors' thorough responses leading reviewers to favorable score updates, justify acceptance of this technically solid contribution to an emerging research direction.
Decision: Accept (poster)