Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling
Authors: Jinghan Li
Zhicheng Sun
Yadong MU
Venue: ICML 2025 poster (2025)
Keywords: Deep Equilibrium Models
Large Language Models
Robot Task Planning
Primary area: applications->robotics
TL;DR: An equilibrium model-based LLM planner capable of self-refining plans based on external and internal feedback.

Abstract: In the endeavor to make autonomous robots take actions, task planning is a major challenge that requires translating high-level task descriptions to long-horizon action sequences. Despite recent advances in language model agents, they remain prone to planning errors and limited in their ability to plan ahead. To address these limitations in robotic planning, we advocate a self-refining scheme that iteratively refines a draft plan until an equilibrium is reached. Remarkably, this process can be optimized end-to-end from an analytical perspective without the need to curate additional verifiers or reward models, allowing us to train self-refining planners in a simple supervised learning fashion. Meanwhile, a nested equilibrium sequence modeling procedure is devised for efficient closed-loop planning that incorporates useful feedback from the environment (or an internal world model). Our method is evaluated on the VirtualHome-Env benchmark, showing advanced performance with improved scaling w.r.t. inference-time computation. Code is available at https://github.com/anonymous-icml-2025/equilibrium-planner.

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Jinghan Li
Zhicheng Sun Yadong Mu
Abstract
poses high-level task into mid-level actions. While this problem requires long-horizon planning well consid- eration of environmental feedback, LLMs are often limited by: (1) imidirectional dependency: due to autoregressive generation, previous tokens cannot attend to future tokens, resulting in limited ability to plan ahead (Wu et al., 2024); (2) lack of error correction for existing outputs, unless with heavy system 2: (3) fixed forward process hindering the allocation of more inference computation to further improve planning performance. These inherent limitations of LLMs inhibit closed- loop annghorizon robotic planning.
In the endeavor to make autonomous robots take actions, task planning is a major challenge that requires translating high-level task descriptions to long-horizon action sequences. Despite recent advances in language model agents, they remain prone to planning errors and limited in their ability to plan ahead. To address these limitations in robotic planning, we advocate a self-refining scheme that iteratively refines a draft plan until an equilibrium reached. Remarkably, this process be optimized end-to-end from an analytical perspective without the need curate additional verifiers reward models, allowing us to train self-refining planners in simple supervised learning fashion. Meanwhile, a nested equilibrium sequence modeling procedure is devised for efficient closed-loop planning that incorporates useful feedback from the environment (or internal world model). Our method is evaluated the VirtualHome-Emi benchmark, showing advanced performance with improved scaling w.r.t. inference-tine timee computation. Code is available at https: / ///ithub.com// www.o////ninggllninnnmmm Sepullibriumpplanae
To address the above challenges LLM planners in closed- loop long-horizon planning, we advocate the approach of self-refinement (Welleck et al., 2023: Shinn et al., 2023; Kim al., 2023b; Madaan et al., 2023) that iteratively im- proves previously generated plan. The reasons behind are threefold: (1) bidirectional dependency: since the output is conditioned on a previous draft plan, it can attend to all tokens in the plan (from an old version), thus improving its ability to plan ahead; (2) internal error correction which allows implicit self-correction in forward pass without an explicit, heavy system (3) dynamic computation allo- cation by iterating through self-refine refineeefm process until convergence. However, such a self-refining strategy imposes significant training difficulties because it requires backprop- agation through infinite self-refining steps (Werbos, 1990). This may be seen as an extreme case that reflects some of the general challenges in teaching L.LMs to plan and reason. Existing solutions include curating process supervi- sion (Uesato al., 2022; Lightman al., 2024) applying reinforcement learning (Zelikman et al., 2024; Jaech et al., 2024; Kumar et al., 2025; Guo et al., 2025), but they are considerably more complex than supervised training and remain unproven in the absence of efficient verifiers.
1. Introduction
Recent advances in large language models (LLMs) have spurred tremendous progress in robotic planning (Huang et al., 2022; Li et al., 2022; Singh et al., 2023; Driess et al., 2023; Ahn et al., 2023; Huang et al., 2023; Zhao et al., 2023; Hu et al., 2024). Based on their extensive world knowledge, LLM agents seem close to autonomously performing robotic tasks, such as in household scenarios. However, growing evidence shows that existing LLM agents struggle with task planning (Kaelbling & Lozano-Pérez, 201 that decom-
This work proposes a simple supervised learning framework for planning via self-refinement. Specifically, we formu- late the self-refining process recursively refines the plan until the equilibrium point, as illustrated in Figure 1. While the forward process of this fixed-point problem could be solved efficiently using root- finding methods, more interestingly. its backpropagation can be skipped since its gradient explicated by the im- plicit function theorem (Krantz & Parks, 2002) as in deep equilibrium models (Bai et al., 2019; Geng & Kolter, 2023).
fixed-point problem that
'Peking Unviersity, China. Correspondence to: Yadong Mu mmydd@pk..ddu.c>>.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Should I change the plan? No
plan (feedback')
plan, (feedback)
Input, plan (feedback)
Input, plano (feedbacko)
(a) Equilibrium in planning
(b) Iterative planning until equilibrium is reached
Figure 1: Illustration the equilibrium point in planning. We view planning as a self-refinement process in which the ideal plan emerges as an equilibrium point, remaining unchanged by any refinement attempts even with newer information (e.g. feedback from the environment or a world model). This enables us tackle robotic planning from an optimization perspective around its equilibrium, bypassing the need for sophisticated reinforcement learning.
It is noted that the derived gradient term may be further sim- plified through Jacobian-free approximation (Fung et al., 2022) to facilitate training. These analytical techniques al- low end-to-end supervised training of the LLM planner to accomplish self-refinement without the need for additional verifiers or reward models in reinforcement learning-bai based counterparts, greatly enhancing simplicity and practicality. And after training, equilibrium model-based planner dynamically allocates more inference computation based on the number of iterations needed to solve the equilibrium, thereby achieving better planning performance.
ficiently incorporate closed-loop feedback into the equi- librium sequence modeling, reusing previous equilibrium solutions to alleviate inference computation. It is further implemented with a world model to improve practicality.
Our method is evaluated on the VirtualHome bench- mark (Puig et al., 2018; Liao al., 2019), demonstrating its advantageous performance with better scaling w.r.t. inference computation than tree-based alternatives.
2. Related Work
Another important cue for self-refinement in robotic tasks is closed-loop feedback from the environment. To efficiently incorporate environmental feedback, devise nested equilibrium sequenee modeling procedure consisting of in- ner and outer loops, where the inner loop iteratively refines a plan using previous feedback, while the outer loop up- dates the feedback by interacting with the environment. This enables closed-loop planning from even a few envi- ronmental interactions. Moreover, the nested equilibrium solving process is accelerated by reusing the previously de- rived equilibrium. We further implement the above design within LLM agent framework, seamlessly integrating the equilibrium model-based planner, an experience mem- ory buffer containing past plans and feedback, and a world model to estimate feedback in the absence of environmental interactions, thus allowing the planning system to operate effectively in closed-loop long-horizer robot task planning scenarios. The core contributions of our work are follows:
LLMs for Planning. LLMs demonstrate outstanding capa- bilitics in robotic planning (Silver et al., 2024; Zhang et al., 2023; Nayak et al., 2024; Wang et al., 2024). Scaling up inference computation improve LLMs' performance on planning and reasoning tasks has received increasing atten- tion (Brown et al., 2024; Snell et al., 2025; Wu et al., 2025; Jaech et al., 2024; Guo et al., 2025). Precedent techniques involving chain-of-thought (Wei et al., 2022; Zelikman et al., 2022; 2024), repeated sampling (Wang et al., 2023; Brown et al., 2024) and tree search (Yao et al., 2023a; Zhao et al., 2023) showed preliminary results with handcraft system 2. Alternatively, method called self-refinement (Welleck et al., 2023; Shinn et al., 2023; Kim et al., 2023b; Madaan et al., 2023) suggests recursively refining the existing LLM output an autonomous manner, but it relies heavily on prompting or sophisticated reinforcement learning. To fully exploit its potential, we propose an end-to-end optimization method for self-refinement via deep equilibrium models.
We present equilibrium sequence modeling, simple train- ing approach for self-refining L.LMs based on equilibrium models, allowing for end-to-end supervised learning with- out additional verifiers or reward models.
Deep Equilibrium Models (Bai et al., 2019) infinite- depth neural networks specified by fixed-point problems x¹ = fe(æ*), where fe is an equilibrium layer. While their inference can take infinite steps by the fixed-point iteration, their gradients are estimated using implicit differ- entiation (Krantz & Parks, 2002) without backpropagating
nested equilibrium solving process proposed to ef-
2

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
through all layers, thus enabling memory-efficient training. They have been extensively applied to tasks such as visual understanding (Bai et al., 2020; 2022) and image genera- tion (Pokle et al., 2022; Geng et al., 2023; Bai & Melas- Kyriazi, 2024). In this paper, we apply the fixed-point for- mulation of deep equilibrium models to the self-refinement process LLM planners, allowing for simple supervised training to refine themselves. More detailed introduction to deep equilibrium models is presented in Appendix A.
However, self-refinement via prompting (Shinn et al., 2023; Kim et al., 2023b; Madaan et al., 2023) has been found to be very limited by Huang et al. (2024). Alternative training-based methods require careful curation of train- ing sequences (Welleck et al., 2023; Havrilla et al., 2024) or reinforcement learning (Qu et al., 2024; Kumar et al., 2025) and are therefore difficult to implement, even for domains with efficient verifiers such as coding and math. Overall, they remain deficient for robotic planning compared to sys- tem 2-based alternatives, as shown in Hu et al. (2024).
3. Method
3.3. Seff-Refinement as An Equilibrium Model
We study the problem of robot task planning that aims to de- compose a high-level task description into long-horizon mid-
To address the training inefficiency of self-refinement ap- proaches, this section proposes equilibrium sequence model- ing, a simple supervised training scheme for teaching LLM planners to self-refine through the lens of deep equilibrium models (Bai et al., 2019; Geng & Kolter, 2023).
level action sequences (Kaelbling& Lozano-Pérez, 2011), In the following, we first introduce the closed-loop robotic planning problems (Section 3.1). Then we discuss the limi- tations of L.LM planners in self-refinement (Section 3.2) and address them with novel equilibrium sequence modeling scheme (Section 3.3). This framework is adapted to closed- loop feedback with efficient designs in Section 3.4. Finally, practical implementations are presented in Section 3.5.
Let us first consider simplified scenario of self-refinement without environmental feedback. namely that the context c is fixed, e.g. to predefined system message Then, the self-refinement process in Equation (1) reduces to a fixed fixed-point problem concerning only the plan 2,. Denote the initial plan by z0 Ø and the equilibrium plan, i.e. the endpoint, by x", then its trajectory can be expressed as:
3.1. Problem statement
We assume that the agent runs in a fully observed envi- ronment with coarse feedback. Given a high-level task described in natural language instruction In, and the cor- responding environment information Env, the agent is re- quired to decompose In into sequence of low-level ac- tions {a1. 02...on) } as an action plan to accomplish its sub- goals [9), -mm J. where an are semantic actions from the given action space and Im are goal-oriented conditions that are invisible to the agent. In terms of loop task planning, we allow the agent to interact with the environ- ment many times and receive feedback for adjustment.
(zo. c) - .... (Iz, c) + (x* .c.
(2)
Although its forward process is tractable with existing root- solving techniques, such as the classic fixed-point iteration or alternative numerical methods (Broyden, 1965; Ander- son, 1 1965), its training requires recurrent backpropagation through multiple self-refining steps (Werbos, 1990). This re- sults in an extremely inefficient and unstable computational process where end-to-end training fails.
Instead, we approach it directly from an analytical perspec- tive. Assuming access only to outcome supervision T.(,;)) v) on the plan, e.g. its distance to the ground truth plan y, then self-refinement is formulated as an optimization problem minimizing the loss function of the equilibrium plan:
3.2. Preliminaries on Self-Refinement
The prevailing LLMs are intrinsically limited in planning, as their unidirectional dependency results in limited capa- bility to plan ahead (Wu et al.. 2024), and the lack of error correction hinders closed-loop planning. These reasons call for alternative mechanisms to address robot task planning.
min 0
L(x", .1)
s.t.
x ====foo>"c)
Interestingly, the above optimization problem can be solved without backpropagating over the entire inference process. As the following theorem indicates, we can directly differ- entiate through its fixed point regardless of the solution path, with only a constant computational and memory cost.
Recently, Welleck et al. (2023); Shinn et al. (2023); Kim et al. (2023b); Madaan et al. (2023) proposed self- refinement, which uses an LLM fo to iteratively refine the previous L.LM output. This strategy naturally addresses the above limitations, since it introduces bidirectional to- ken dependency and a dynamic error correction mechanism. Formally, let 24 denote a draft plan and e denote context (e.g. environmental feedback), then planning may be viewed as a self-refinement process as follows:
Theorem 3.1. (Implicit Function Theorem (Bai et al., 2019;
Kruntz & Parks, 2002)) Assuming that (I- - Fe ible, then the loss gradient of Equation (3) w.r.f. 0 is given by:
afa is invert-
0L
aL
Ofe
0f
[
(4)
T+ = So (Fe, C2).
00
dr
Ox
00
3

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Tter
Iter I
Iter t
Tter t+]
Input
Input
Input
Input
Input
(feedbacks)
(feedbacki)
(feedback.
(feedback))
(feedback')
plano
plan
plan,
plan"
plan
Inputs
Output
plan
plan
plan *
plan
plan
(a) Inference: solving equilibrium via fixed-point iteration
(b) Training
Figure 2: Illustration of equilibrium sequence modeling with two alternating steps: (a) Prior to training, the model undergoes iterative inference reach an equilibrium plan. (b) Then, it is pushed away from the equilibrium towards the ground truth by supervised loss. This teaches the model to self-refine by mapping suboptimal equilibrium plan better plan.
Its proof is given in Appendix A.4. It is noteworthy that the inverse Jacobian term A (T 8fs )-1 within the above 2xz
point, which reduces overfitting compared to the vanilla supervised finetuning: (2) by guiding the LLM to map a sub- optimal plan x to better plan it teaches self-refinement via simple supervised loss, without requiring additional value functions or reward models (Welleck et al., 2023; Havrilla al., 2024; Qu et al., 2024; Kumar et al., 2025).
gradient is difficult to compute exactly, for which existing work often approximates through the damped fixed-point unrolling or the Neumann series (Geng et al., 2021b). For computational efficiency, we drop the inverse Jacobian term using A R / as in Fung al. (2022); Geng et al. (2021 Choe al. (2023), the latter work having been validated Transformer-based LLMs:
3.4. Equilibrium Models with Feedback
This section extends the derived equilibrium sequence mod- eling to more practical scenario where the environment may provide some closed-loop feedback, e.g. failure details, during plan execution. Such auxiliary information would be effective cue for planners further refine their plan.
ÒL aL 0. 00 Oz de
(5)
For a comprehensive introduction to equilibrium models and this approximation, please refer to Appendix A.
Equilibrium Sequence Modeling, Based on the simpli- fied gradient estimation, we reformulate its training into supervised learning problem. According the chain rule, the derived gradient is exactly the gradient of the following optimization problem associated with the equilibrium r":
To take into account environmental feedback, we consider an adaptive context e that influenced by the plan x; rather than fixed. Then, the previously considered equilibrium solving process of Equation (2) should be revised as an iterative process coupling the plan x; with the feedback co. starting from To Co Ø:
min L(fo(æ", ()).))
(6)
(mo, co) + .... - (Ir, Ce) + + (x (x",c").
This new formula represents new equilibrium sequence modeling scheme that can be implemented in two alter- nating steps: (1) In the first step, we solve the fixed-point problem by iterative inference of LLM based on the previ- ous output tokens F, until the new output tokens converges, yielding an equilibrium plan = (2) In the second step, the equilibrium x² is paired with the ground truth plan y training sequence, which is used as in the standard super- vised finetuning pipeline to teach the LLM to self-refine. The two-step procedure is illustrated in Figure 2.
After the modification, the existing derivations only hold when we neglect the derivatives related to c". Fortunately, this natural choice due to the non-differentiability of most feedbacks. Therefore, the equilibrium planner can be trained in similar supervised way as Equation (6) and Figure 2, and after training it would be able to self- refine based on the latest feedback just by forward passes. However, iteratively interacting with the environment to obtain feedback is costly and may not be recoverable. In response, we devise a nested equilibrium solving scheme for more efficient closed-log loop planning.
features two intuitive advantages: (1) instead of directly regressing the ground truth, it gently adjusts the equilibrium

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Algorithm 1 Inference of Equilibrium Planner
Require: planner Sa, environment world model Env, number of iterations N. Initialize start point zo and feedback co.
for i <- 0 to N or converged do
Solve inner equilibrium loop to obtain z;- Update next plan Tt+1 and feedback c++1 with Env.
> Equation (8)
b Equations (9) and (10)
Ensure: generated plan x².
X
Equilibrium memory
Environment
plan,
(feedback:)
Equilibrium planner
World model
Figure 3: Illustration of our proposed framework. incorporates (1) memory containing all equilibrium experiences during inference, (2) self-refining planner trained on equilibrium plans alongside the ground truth, and (3) world model trained experiences simulate environmental feedback. Three modules work synergistically for closed-loop planning. The planner interacts with the environment to output plan, with equilibrium sequence modeling and the equilibrium plans with feedback are stored in the equilibrium memory. During training, The planner is trained on these equilibrium plans together with the ground truth while the world model is trained on the past experiences stored in the equilibrium memory.
Nested Equilibrium Solving. Inspired by the introspection process in human daily life, we propose divide equilib- rium solving into nested loop process. The inner loop introspects on the existing plan and feedback and takes action, while the outer loop interacts with the environment to update the feedback. Formally, each inner loop is an equilibrium solving process with fixed feedback e;:
different inner loops. This is achieved by initializing the context of the next inner loop by concatenating the previous feedback and the latest feedback (paired with its plan):
C++1 (Z++1, Env(2++1)) || Ct>
(10)
where || denotes concatenation. The nested inference proce- dure with reuse of equilibrium is described in Algorithm
1 = fo(of,c)) ct)
3.5. Practical Implementation
2t = So(x;,
This section discusses the implementation of the proposed equilibrium planner. To enable effective training while inter- acting with the environment, the following two modules carefully devised complement the planner: experience memory that caches all equilibrium plans and their feedback during equilibrium solving, and world model to estimate the feedback in the absence of environmental interactions. The complete planning framework is illustrated Figure and the specific implementation details are explained below.
Thanks to this inner-loop introspection mechanism, our equi- librium model can be more efficient in closed-loop planning, outperforming with fewer environmental interactions.
Reusing Equilibrium Solution. Another efficiency bottle- neck is the equilibrium solving. Considering that its speed depends largely on the initial plan, it is unnecessary to restart from Ø every time. Therefore, we accelerate equilibrium solving by reusing the previously derived equilibrium plan as the starting point of the next iteration, similar Bai al. (2022); Bai & Helas-Kyriaz (2024):
Equilibrium Experience Memory. During the training process, equilibrium model interacts with the environ- ment only when the inner loop reaches the equilibrium point. This results in small number of cquilibrium points, which may not be sufficient for supervised training. To improve our training efficiency and stability, we opt to cache all previously obtained equilibrium points, along with their
Fe+1 x7.
(9)
which corresponds to the starting point 2f+1 of the inner loop. Similarly, history feedback could be reused across
5

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Table I: Performance on VirtualHome-Env without correction. Our planner achieves state-of-the-art performance in most evaluations. Note that the Exec. metrics are marked in gray because they are already high and can easily exceed 99% with simple automated rules (by truncating illegal output). See Table 8 in the appendix for full comparisons with Peee-elaneer.
Novel Scene and Task
Novel Scene
Novel Task
Exec.
SR
GCR
Exec.
SR
GCR
Exec.
SR
GCR
GPT-3.5 API:
Zero-shot Planner
16.49
1.07
1.52
ProgPrompt
35.04
12.54
19.99
Iterative-Plann Planner
44.54
27.04
33.25
Tree-PlanneTN-25
55.74
28.33
39.96
Tree-Plannery=50
49.01
28.14
35.84
Finetuned Ilama 3 8B:
Supervised
93.55
24.19
32.55
96.84
41.05
49.81
95.94
26.07
35.53
Tree-Plannery-25
95.16
38.71
63.18
96.08
51.58
69.45
95.50
40.38
63.75
Tree-Plannery-50
94.94
38.71
63.50
96.06
51.58
69.54
95.40
39.74
63.29
Ours
90.32
40.32
65.40
95.79
65.26
79.47
93.38
41.88
62.76
Table 2: Performance on VirtualHome Env with up to 10 corrections. Our planner consistently leads in SR and GCR performance. In particular, the comparison with SELF-REFINE confirms the effectiveness of our new training method.
Novel Scene and Task
Novel Scene
Novel Task
Exec.
SR
GCR
Exec.
SR
GCR
Exec.
SR
GCR
GPT-3.5
Local Replan
79.66
37.46
51.90
Global Replan
82.09
37.93
52.46
Tree-PlannerN-25
89.13
35.30
56.65
Plannnelnner
88.26
41.58
59.55
Finetuned Llama 3 8B:
SELF-REFINE
96.77
43.55
65.18
92.63
54.74
70.24
94.44
39.96
62.37
Pre-Plaanees-2s
95.16
41.94
56.49
96.08
55.79
68.82
95.50
42.09
57.83
Tree- Ple-PlannerN-so
94.94
43.55
58.91
96.06
58.95
70.00
95.40
43.38
59.79
Ours
91.94
56.45
76.63
97.89
77.89
87.07
92.31
54.91
74.18
environmental feedback, in an experience memory. There- after, these equilibrium points can be sampled repeatedly for versatile training purposes. For example, for the planner, we randomly sample a batch of equilibrium points at each training epoch, which are paired with the ground truth for su- pervised training. In particular, the most recent equílibrium points sampled more frequently to reduce distribution shift, Next, we describe another crucial component.
we implement the world model with an LLM and finetune it on the planner's equilibrium feedback over all iterations for better generalizability. And during inference, we alternate between using real and generated feedback at each iteration for a good balance between performance and efficiency.
4. Experiments
4.1. Experimental Settings
Internal Feedback from World Model. Due to inefficiency of interacting with the environment, we construct a world model (Ha Schmidhuber, 2018) to provide the neces- sary feedback in closed-loop planning. Our world model takes the environmental context, task instruction and current plan inputs and predicts some basic types of feedback. This definition is slightly simpler than the commonly used world model, which requires simulation of the environmen- tal states, and therefore may be easier to train. Concretely,
Benchmark. The VituuaHHome-Em benhmark (Puig et al., 2018; Liao et al., 2019) is adopted during the experiments. It contains 1360 long-horizon tasks with ground truth action sequence annotations (average length 10.8) and provides updated scene graphs after each action, allowing simulation of closed-loop feedback. To analyze the generalizability. divided the dataset into training set and three test subsets, including the novel scene set, the novel task set, and the
6

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Output log
Output leg
Output log
[NALK] <heme_officee
[ALKK] <home_office>
[MALK] <home_office"
[NALK] <ehair>
MALK <chair>
[NALK] <chair>
[FIND] <chair>
[INJ] <chair>
[FIND] <cheir>
[CRAB] <chair>
GRAB <chair>
[GRAB] <ahoir>
[NALK] sdining_rromm
MALK]
<dining_ rooms
[PUTBACK] <chair>
relat've pos:
fail, trackback
chome_effice>
<chal:> and <floor>
[ALK] <bedroca>
Wrong relative goer
<oharacter> <home_office>
<chai>>
[NALK] chome_off-cca
[NALK] _em
[NALK] sehair>
[NALR] <chair?
[PIND] <cheir>
[PIND] <ehair>
[АRАн] <chair>
[GRAB] <cheir>
[NALK] <dining_room
[KALK]
Same plan, Lask [ail
[PUTBACK] <chair> <élcor>
success
(a) SELF-REFINE
(b) Tree-Planner
(c) Ours
Figure 4: Visualization of our self-correction process compared the baselines SELF-REFINE and Tree-Planner. The task instruction is "Take comfortable chair and place it in the entrance hall".
novel scene and task set. More statistics and examples about VirtualHome-Eni can be found in Appendix B.1.
set (paired with the equilibrium points). The number of finetuning epochs is set to 6, and the learning rate is 0.0002. The world model is finetuned on all planner interactions for epochs using the same learning rate. A greedy LLM sampling strategy is used in later refinement steps until con- vergence. Moreover, we implement the KV cache to speed up inference. Further details are provided in Appendix B.3.
Metrics. We use executability (Exec.), success rate (SR),
goal conditions recall (GCR) following Hu al. (2024). Exec. evaluates whether the plan can be executed in given the environment, SR refers whether the goal is accom- plished, and GCR measures the proportion of goal condi- tions achieved. To examine closed-loop planning capabili- ties, we study two test settings, without error correction or with up to 10 corrections, the latter allowing self-correction based on environmental feedback. We also evaluate compu- tational efficiency by measuring TFLOPS at inference.
4.2. Main Results
The experimental results in the two planning setups, without correction with up to 10 corrections, summarized in Tables and 2. Overall, our method achieves the lead- ing performance on the majority of metrics. Specifically, the experimental results show that: (1) Even without error correction, our self-refining process still brings a significant improvement of 14% on SR in the novel scene subset, with other metrics superior comparable the previous leading method. (2) By incorporating environmental feedback, approach improves all metrics by more than 11 % and up to 19%, showing clear advantages. (3) Similar to the exist- ing finetuning-based methods, our generated plans exhibit a high exccutability over 90%, which can be improved to 99% by simply truncating illegal overlength outputs. These results clearly confirm the advantages of our approach.
Baselines. Our method is compared with Pree-Planner (Hu et al., 2024), SELF-REFING (Madaan et al., 2023), and supervised finetuned planner. They are all reproduced using finetuned Ilaaaa 3 8B (Dubey et al., 2024) (the original Llama cannot achieve meaningful SR due to format errors). In addition, we consider several baseline methods that call the GPT-3.5 API, including ProgPrompt (Singh et al., 2023), Zero-shot Planner (Huang et al., 2022) and two self-refining planners, Local Replan (Raman et al., 2022; Guo et al., 2023) and Global Replan (Shinn et al., 2023). Their results are for reference only. See Appendix B.2 for details.
Implementation Details. Our implementation is consis- tent with the baseline methods by finetuning from Ilama 3 8B (Dubey et al., 2024) on the VirtualHome- training
In particular, Table 2 compares the efficacy of training scheme, equilibrium sequence modeling, against alternative self-correction methods. It shows more than % improve-

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Table 3: Effectiveness of different types of feedback. They are measured under the constraint of up to 10 rounds of internal or external feedback. Our trained planner able to take into account various types of feedback to refine the plan.
Novel Scene and Task
Novel Scene
Novel Task
World model
Env.
Exec.
GCR
Exec.
SR
GCR
Exec.
GCR
88.71
33.87
59,98
96.79
49.47
66.60
93,80
34.62
59.06
83.87
51.61
75.13
96.84
75.79
85.79
92.31
56.62
75.53
90.32
40.32
65.40
95.79
65.26
79.47
93.38
41.88
62.76
91.94
56.45
76.63
97.89
77.89
87.07
92.31
54.91
74.18
Doet
Dut
3.0
Succos (50)
50
Troe-Planet FEFFRE
Steccss zate (Sg)
20
SELF REFINE
Tax-Harz
Success 201 (%)
Fixed point Seecttboss
Outs we
Inforence TFLOPS
Plan length
Coedbackttembon
Plan lengt
(a) Scaling of performance w.r.t (b) Comparison performance inference-ting time computation over different planning horizons
(a) Improving efficiency through (b) Dynamic compute allocation
reusing equilibrium solutions for more complex plans
Figure 5: Performance analysis vs. inference computation and plan length. Our method shows leading scaling w.r.t. in- ference computation and long-horizon planning capabilities. The inference computation is measured by TFLOPS, and we consider KV cache when computing inference TFLOPS.
Figure 6: Ablation study computational efficiency, Our method reuses equilibrium solutions and dynamically allo- cates inference compute to improve performance-efficien tradcoff. The latter is measured by the number of fixed-point iterations before convergence with mean and std.
4.4. Ablation Study
ment over the prompting-based self-refinement method SELF-REFINE (Madaan et al., 2023) and more than 12% improvement over the system 2-based Tree-Planner (Hu et al., 2024). The former stems from the effectiveness of our training objective, while the latter is due to the flexible self-correction mechanism without extensive manual design. Meanwhile, maintain simplistic supervised finetuning fashion similar to the compared methods, without intricate reinforcement learning. These validates the effectiveness of cquilibrium sequence modeling in robot task planning.
This section validates the effectiveness of our method in per- formance and efficiency. See Appendix for more results.
Effectiveness of various feedback. As can be observed in Table 3, incorporating external feedback from the environ- ment or internal feedback from the world model consistently improves performance. Even though the world model does not provide as much improvement as the real environment, it also increases performance by over 3%. In particular, the synergy of both types of feedback yields the highest per- formance most of the metrics, further confirming their effectiveness. In the following analysis, we will focus on method using only environmental feedback for simplicity.
4.3. Visualization
Figure 4 further compares our self ceff-correction process with the baseline methods. As can be seen, our method is bet- ter at incorporating environmental feedback to improve the plan, while the baselines fail by simply repeating the pre- vious plan, or by making only local adjustments that are insufficient. The rationale behind is that our method can flex- ibly take into account feedback through forward passes of LLMs, allowing arbitrary changes based on its knowledge. In contrast, the tree-based alternative requires backtracking in tree, which is costly and does not fully exploit the ver- balized knowledge in the feedback. More qualitative results are illustrated in Figures 10 to 14 the appendix.
Scaling of performance. Here, we follow Brown et al. (2024); Snell et al. (2025); Wu et al. (2025); Jaech et al. (2024); Guo et al. (2025) considering the scaling w.n.t. inference computation. The results Figure 5a show that our method achieves better off along with leading scaling w.r.t. inference computation. Thus, more inference budget can be allocated to improve its performance. Furthermore, in Figure 5b, we show that its performance advantage is largely due to better long-horizon plamning capabilities, achieving more than twice the success rate of baselines on extremely long plans (length>>20).
trade-
8

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Table 4: Convergence analysis. They summarize the number of iterations for LLMs to reach a fixed point over 10 runs on 60 random tasks with the equilibrium solving prompts. Each of them can reach convergence a few steps.
Table 5: Robustness to disturbed environment. During in- ference, random feedback is injected into the environmental feedback according to the noise ratio. Our model remains stable, demonstrating robustness to noisy feedback.
Mean
Std
Min
Max
Both Novel
Novel Scene
Novel Task
Original Llama-3-8B
6.70
2.12
3.22
14.98
Noise ratio
SR
GCR
SR
GCR
SR
GCR
Original Qwen-2.5-0.5B
4.80
2.13
2.69
9.43
0%
51.61
75.13
75.79
85.79
56.62
75.53
Original Qwen-2.5-7B
7.68
5.03
2.46
16.78
5%
51.61
74.30
75.79
85,40
54.27
73.89
Supervised Finetuned Llama
2.46
0.88
2.02
3.80
10%
53.23
73.43
74.74
85,84
53.85
73.09
Ours
3.02
1.61
2.32
4.07
20%
50.00
73.10
73.68
83.22
54,49
71.80
mental feedback or a world model for closed-loop planning, We implement the proposed approach on the VirtualHome- Env benchmark, and the experimental results suggest that it can dynamically allocate inference-time computation to achieve state-of-the-ar robot task planning performance.
Computational efficiency. Although our planner training is slower than baselines (36h vs. 12h) due to the equilib- rium solving process for synthesizing training pairs (=22hh,, exhibits a competitive inference efficiency. For exam- ple, our method takes 16h to evaluate, while Tree-Planner takes 24h. This can be attributed to our design of reusing equilibrium in nested equilibrium solving, As illustrates in Figure 6a, it accelerates the convergence, achieving better performance (>55%) with significantly fewer interactions. Furthermore, Figure 6b shows that our planner dynamically allocates compute for tasks of different complexity.
Impact Statement
This paper presents a supervised learning framework for im- proving the closed-loop long-horizon capabilities of LLM agents, with the potential to complement the prevailing rein- forcement learning- based or prompting- based frameworks. However, we note that any such improvement in planning capabilities should be treated with caution. For example, the LLM agent could autonomously create sub-goals that are threatening to humans, e.g., gaining more control. This calls for further research into LLM interpretability and safety.
Fixed-point convergence. Table 4 shows the number of iterations for an LLM to reach its fixed point, aggregated over 10 initial plan for each of 60 random tasks. All LLMs considered, including the original Llama and Qwen, the supervised finetuned Llama, and our model, can converge to a fixed point within a few iterations, regardless of the initial sequence. This is partly due to our greedy sampling strategy (described in Appendix B.3) that reduces the LLMs' randomness, after which they tend to repeat themselves (please see Figures 4a, 10a and 1 1la) and converge easily.
The work is supported by an internal grant of Peking University (2024JK28), a grant from China Tower Corporation Limited. We thank Xingjian Bai and Liyuan Wang for their helpful discussions.
Acknowledgemente
Robustness to noisy feedback. Table 5 shows the robust- ness of our model when the environment is disturbed by noise. The case where the model only receives environmen- tal feedback is considered. During inference, we randomly replace some of the feedback with incorrect feedback, such as incorrect feedback types or incorrect corresponding ob- jects and actions. As shown in the results, our model exhibits stable performance under small amounts of noise (110%), demonstrating its robustness in a disturbed environment.
References
Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al. Do can, not as I say: Grounding language in robotic affordances. In Conference on Robot Learning, pp. 287-318, 2023.
Anderson, D. G. Iterative procedures for nonlinear integral equations. Journal of the ACM, 12(4):547-560, 1965.
5. Conclusion
Athalye, A., Kumar, N., Silver. T., Liang, Y., Lozano-Péres T., and Kaelbling, L. P. Predicate invention from pixels via pretrained vision- languaae models. arXiv preprint 2rx1iv2501100026 2024.
This work proposes an equilibrium model-based LLM plan- ner that is capable of self-refining plans from external and internal feedback. Unlike existing self-refinement methods based on prompting or sophisticated reinforcement learning, our proposed equilibrium sequence modeling allows simple supervised training of the self-refining planners. Moreover, it also enables the planner to efficiently incorporate environ-
Bai, S., Kolter, Z., and Koltun, V. Deep equilibrium models. In Advances in Neural Information Processing Systems, pp. 690-701, 2019.

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Bai, S., Koltun, V., and Kolter, J. Z. Multiscale deep equi- librium models. In Advances in Neural Information Pro- cessing Systems, pp. 5238-5250, 2020.
Du, Y., Yang, S., Florence, P., Xia, F., Wahid, A., Sermanet, P., Yu, T., Abbeel, P., Tenenbaum, J. B., Kaelbling, L. P., et al. Video language planning. In International Confer- ence on Learning Representations, 2024.
Bai, S., Geng, Z., Savani, Y., and Kolter, Z. Deep equi- librium optical flow estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 620-630, 2022.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint 2407.21173. 2024.
Bai, X. and Melas-Kyriaze L. Fixed point diffusion models.
In Proceedings of the IEEF/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 9430-9440, 2024.
El Ghaoui, L.. Gu, F., Travacca, B., Askari, A., and Tsai, A. Implicit deep learning. SIAM Journal on Mathematics of Data Science, 3(3):930-958, 2021.
Banach, S. Sur les opérations dans les ensembles abstraits et leur application aux équations intégrales. Fundamenta Mathematicae, 3(1):133-133 1922.
Finn, C., Abbeel, P., and Levine, S. Model-agnostic meta- learning for fast adaptation of deep networks. Interna- tional Conference on Machine Learning, pp. 1126-1135, 2017.
Bolte. J., Pauwels, E., and Vaiter, S. One-step differentiation of iterative algorithms. In Advances in Neural Informa- tion Processing Systems, pp. 77089-77109 7003, 2023.
Fung, S. W., Heaton, H., Li, Q. McKenzie, D., Osher, S., and Yin, W. JFB: Jacobian free backpropagation for implicit networks. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6648-6656, 2022.
Brooks, T., Peebles, B., Holmes, C., DePue, W., Guo, Y., Jing, I., Schnurr, D., Taylor, J., Luhman, T., Luhman, E., et al. Video generation models as world simulators. https: http:///ppaaaaccocoo.. research/ video-generation wwweeerrrtioommmdeeeeeseee assdell-asswordd simulators, 2024.
Garima, Liu, F., Kale, S., and Sundararajan, M. Estimating training data influence by tracing gradient descent. In Advances in Neural Information Processing Systems, pp. 19920-19995 2020.
Brown, B.. Juravsky, J., Ehrlich, R., Clark, R., Le, Q. V.,
Ré, C., and Mirhoseini, A. Large language monkeys: Scaling inference compute with repeated sampling, arXiv preprint arXiv:240..21787 2024.
Geng, Z. and Kolter, J. Z. TorchDEQ: A library for deep equilibrium models. arXiv preprint arXiv:2310.18605 2023.
Broyden, C. G. A class of methods for solving nonlinear simultaneous equations. Mathematics of Computation, 19(92):577-593, 1965.
Geng, Z., Guo, M.-H., Chen, H., Li, X., Wei, K., and Lin, Z. Is attention better than matrix decomposition? In International Conference on Learning Representations, 2021a
Cai, T., Li, Y., Geng, Z., Peng. H., Lee, J. D., Chen, D., and Dao, T. Medusa: Simple L.M inference acceleration framework with multiple decoding heads. In Interna- tional Conference on Machine Learning, pp. 5209-5235, 2024.
Geng, Z., Zhang, X.-Y., Bai, S., Wang, Y., and Lin, Z. On training implicit models. In Advances in Neural Informa- tion Processing Systems, pp. 24247-24266 -4260, 2021b.
Geng, Z., Pokle, A., and Kolter, Z. One-step diffusion distillation via deep equilibrium models. In Advances in Neural Information Processing Systems, pp. 41914- 41931, 2023.
Chen, R. T., Rubanova, Y., Bettencourt, J., and Duvenaud, D. Neural ordinary differential equations. Advances in Neural Information Processing Systems, pp. 6572-6583, 2018.
Choe, S. K., Mehta, V., Ahn, H., Neiswanger, W., Xie, P., Strubell, E., and Xing, E. Making scalable meta learning practical. In Advances in Neural Information Processing Systems, pp. 26271-26290 2023.
Guan, L., Valmeekam, K., Sreedharan, S., and Kambham- pati, S. Leveraging pre-trained large language models to construct and utilize world models for model-based task planning. In Advances in Neural Information Processing Systems, pp. 79081-79094, 2023.
Driess, D., Xia, F., Sajjadi, M. S., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., al. PaLM-E: An embodied multimodal language model. In International Conference on Machine Learning, pp. 8469-8488, 2023.
Guo, D., Yang, D.,, Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang. P., Bi, X., et al. DeepSeek-Ri Incentivizing reasoning capability in LLMS via reinforce- ment learning. arXiv preprint arXiv:2501. 2025.
10

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Guo, Y., Wang, Y.J., Zha, L.., Jiang, Z., and Chen, J. DoReMi: Grounding language model by detecting and recovering from plan-execution misalignment. arXiv preprint mrXiv:2337.003229 2023.
the IEEE/CVF International Conference on Computer Vision, pp. 10936-10946, 2023a.
Kim, G., Baldi, P., and McAleer, S. Language models can
solve computer tasks. In Advances in Neural Information Processing Systems, pp. 39648 -36487, 2023b.
Ha, D. and Schmidhuber. J. World models. arXiv preprint arXiv: 1803. 10122, 2018.
Kim, T., Min, C., Kim, B., Kim, J., Jeung, W., and Choi, J. ReALFRED: An embodied instruction following bench- mark in photo-realistic environments. In European Con- ference on Computer Vision, 2024.
Havrilla, A., Raparthy, S. C., Nalmpantis, C., Dwivedi- Yu, J., Zhuravinsky,, M., Hambro, E.,, and Raileanu, R. GLoRe: When, where, and how to improve LLM reason- ing via global and local refinements. In International Con- ference Machine Learning, pp. 17719-17733, 2024.
Kou, S., Hu, L., He, Z., Deng, Z., and Zhang, H. CLLMs: Consistency large language models. In International Con- ference on Machine Learning, pp. 25426-25440, 2024.
Hu, M., MU, Y., Yu, C., Ding, M., Wu, S., Shao, W., Chen, Q., Wang, B., Qiao, Y., and Luo, Tree-planner: Efficient close-loop task planning with large language models. In International Conference on Learning Repre- sentations, 2024.
Krantz, S. G. and Parks, H. R. The Implicit Function The- orem: History, Theory, and Applications. Birkhäuser, 2002.
Kumar, A., Zhuang, V., Agarwal, R., Su, Y., Co-Reyes, J. D., Singh, A., Baumli, K., Iqbal, S., Bishop, C., Roelofs, R., Zhang, L. M., et al. Training language models to self-correct via reinforcement learning. In International Conference on Learning Representations, 2025.
Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, W., Song, X., and Zhou, D. Large language models cannot self-correct reasoning yet. In International Conference on Learning Representations, 2024.
Huang, W., Abbeel, P., Pathak, D., and Mordatch, I. Lan- guage models as zero-shot planners: Extracting action- able knowledge for embodied agents. In International Conference Machine Leurning, pp. 911 9118-9147, 2022.
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., et al. Inner monologue: Embodied reasoning through planning with language models. In Conference on Robot Learning, pp. 1769-1782, 2023.
Li, S., Puig, X., Paxton, C., Du, Y., Wang, C., Fan, L., Chen, T., Huang, D.-A., Akyürek, E., Anandkumar, A., et al. Pre-trained language models for interactive decision- making. In Advances in Neural Information Processing Systems, pp. 31199-31212, 2022.
Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., Beutel, A., Car- ney, A., et al. OpenAI ol system card, arXiv preprint 1IXi::2412.16720, 2024.
Liang, J., Huang, W., Xia, F,, Xu, P., Hausman, K., Ichter, B., Florence, P., and Zeng, A. Code as policies: Lan- guage model programs for embodied control. In IEEE International Conference on Robotics and Automation, pp. 9493 9500, 2023.
Kaelbling, L. and Lozano-Pérez, T. Hierarchical task and motion planning in the now. In IEEE International Conference Robotics and Automation, pp. 1470-1477, 2011.
Liao, Y.H.., Puig, X., Boben, M., Torralba, A., and Fidler, S. Synthesizing environment-aware activities via activity sketches. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6291 6299, 2019.
Kidger, P.. Morrill, J., Foster, J., and Lyons, Neural controlled differential equations for irregular time series. In Advances in Neural Information Processing Systems, pp. 6696-6707, 2020.
Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let's verify step by step. In International Conference on Learning Representations, 2024.
Kidger, P., Foster, J., Li, X., and Lyons, T.J. Neural SDEs as ddfnite-dimensional GANs. In International Conference on Machine Learning, pp. 5453-5463, 2021.
Liu, B., Jiang, Y., Zhang, X., Liu, Q., Zhang, S., Biswas, J., and Stone, P. LLM+P: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304. / /477, 2023.
Kim, B., Kim, J., Kim, Y., Min, C., and Choi, J. Context- aware planning and environment-aween aware memory for in- struction following embodied agents. Proceedings of

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Liu, H., Simonyan, K., and Yang. Y. DARTS: Differen- tiable architecture search. In International Conference Learning Representations, 2019.
models using 3d scene graphs for scalable robot task planning. In Conference on Robot Learning, pp. 23-72, 2023.
Luketina, J., Berglund, M., Greff, K., and Raiko, T. Scalable gradient-based tuning of continuous regularization hyper- parameters. In International Conference on Machine Learning, pp. 2952-2960, 2016.
Santilli, A., Severino, S., Postolache, E., Maiorca, V., Man- cusi, M., Marin, R., and Rodola, E. Accelerating trans- former inference for translation via parallel decoding. In Proceedings of the Anmual Meeting of the Association for Computational Linguistics, pp. 12336- 2023.
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, I.., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. SELF-REFINE: Iterative refinement with self- feedback. In Advances in Neural Information Processing 2023.
Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K. R., and Yao, S. Reflexion: Language agents with verbal rein- forcement learning. In Advances in Neural Information Processing Systems, pp. 8634-6636661 8652, 2023.
Systems, pp.
Shridhar, M., Thomason, J., Gordon, D., Bisk, Y., Han, W., Mottaghi, R., Zettlemoyer, L., and Fox, D. ALFRED: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVE Con- ference on Computer Vision and Pattern Recognition, pp. 10740-10749, 2020.
McDermott, D. M. The 1998 Al planning systems competi- tion. Al Magazine, 21 21(2)355-3,, 2000.
Nayak, S., Morrison Orozco, A., Have, M., Zhang, J., Thiru- malai, V., Chen, D., Kapoor, A., Robinson, E., Gopalakr- ishnan, K., Harrison, J., et al. Long-horizon planning for multi-agent robots in partially observable environments. Advances in Neural Information Processing Systems, 37: 67929-67967, 2024.
Silver, T., Dan, S.,, Srinivas, K., Tenenbaum, J. B.,, Kaelbling, L., and Katz, M. Generalized planning in pddl domains with pretrained large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 38, pp. 20256-20264, 2024.
Peebles, W. and Xie, S. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 4195-4205, 2023.
Singh, I., Blukis, V., Mousavian, A., Goyal, A., Xu, D., Tremblay, J., Fox, D.,, Thomason, J., and Garg. A. Prog- Prompt:
Generating situated robot task plans using large language models. In IEEE International Conference on Robotics and Automation, pp. 1 11523-1 11530-1530,, 2023.
Pokle, A., Geng, Z., and Kolter, Z. Deep equilibrium ap- proaches to diffusion models. In Advances in Neural Information Processing Systems, pp. 37975-37990, 2022.
Snell, C., Lee. J., Xu, K., and Kumar, A. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. In International Conference on Learning Representations, 2025.
Puig, X., Ra, K., Boben, M., Li, J., Wang, T., Fidler, S., and Torralba, A. VirtualHome: Simulating household activities via programs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 1494-85022 2018
Sun, H., Zhuang, Y., Kong, L., Dai, B., and Zhang, C. Ada- Planner: Adaptive planning from feedback with language models. In Advances in Neural Information Processing Systems, pp. 58202-58245, 2023.
Puig, X., Shu, T., Li, S., Wang, Z., Liao, Y.-H., Tenenbaum, B., Fidler, S., and Torralba, A. Watch-and-helpy chal- lenge for social perception and human-A Al collaboration. In International Conference on Learning Representations, 2021.
Uesato, J., Kushman, N., Kumar, R., Song, F., Siegel, N., Wang, L., Creswell, A., Irving, G., and Higgins, I. Solv- ing math word problems with process-and outcome- feedback. arXiv preprint urXiv:2211.14277 2022.
Qu, Y., Zhang, T., Garg, N., and Kumar, A. Recursive introspection: Teaching language model agents how to self-improve. In Advances in Neural Information Pro- cessing Systems, 2024.
Valmeekam, K., Marquez, M., Olmo, A., Sreedharan, S., and Kambhampati. S. PlanBench: An extensible bench- mark for evaluating large language models planning and reasoning about change. In Advances in Neural In- formation Processing Systems, pp.
Raman, S. S., Cohen, V., Rosen, F.,, Idrees, I., Paulius, D., and Tellex, S. Planning with large language models via corrective re-prompting. In Advances in Neural Informa- tion Processing Systems Workshops, 2022.
2023.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, E.,, and Polosukhin, I. Atten- tion is all you need. In Advances in Neural Information Processing Systems, pp. 6000 6010, 2017.
Rana, K., Haviland, J., Garg, S., Abou-Chakra, J., Reid, I., and Suenderhauf, N. SayPlan: Grounding large language
12

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Wang, J., He, G., and Kantaros, Y. Safe task planning for language-instructer multi-robot systems using conformal prediction. arXiv e-prints, pp. arXiv-2402, 2024.
Zelikman, E., Wu, Y., Mu, J., and Goodman, N. D. STaR: Self-taught reasoner bootstrapping reasoning with rea- soning. In Advances in Neural Information Processing Systems, pp. 15476-15488, 2022.
Wang, X., Wei, J., Schuurmans, D., Le, Q. V., Chi, E. H., Narang, S., Chowdhery, A., and Zhou, D. Self- consistency improves chain of thought reasoning in lan- guage models. In International Conference on Learning Representations, 2023.
Zelikman, E., Harik, G., Shao, Y., Jayasiri, V., Haber, N., and Goodman, N. D. Quiet-STaR: Language models can teach themselves think before speaking. In Conference on Language Modeling, 2024.
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F,, Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, pp-
Zhang, H., Du, W., Shan, J., Zhou, Q. Du, Y., Tenenbaum,
J. B., Shu, T., and Gan, C. Building cooperative embodied agents modularly with large language models. arXiv preprint arXiv:230.00288, 2023.
24824-24837, 2022.
Zhao, Z., Lee, W. S., and Hsu, D. Large language models as commonsense knowledge for large-scale task planning. In Advances in Neural Information Processing Systems, pp. 31967-31987, 2023.
Welleck, S., Lu, X., West, P., Brahman, F., Shen, T., Khashabi, D., and Choi, Y. Generating sequences by learning to self-correct. In International Conference on Learning Reprexentations, 2023.
Zhu, Y. Tremblay, J., Birchfield. S., and Zhu, Hierar- chical planning for long-horizon manipulation with geo- metric and symbolic scene graphs. In IEEE International Conference on Robotics and Automation, pp. 2021.
Werbos, P. J. Backpropagation through time: What it does and how to do it. Proceedings of the IEEE, 78(10):1550- 1560, 1990.
Wu, W., Morris, J. X., and Levine, L. Do language models plan ahead for future tokens? In Conference on Language Modeling, 2024.
Wu, Y.. Sun, Z., Li, S., Welleck, S., and Yang. Y. Inference scaling laws: An empirical analysis of compute-optinal inference for Ilm problem-solving. In International Con- ference Learning Representations, 2025.
Xie, J., Zhang, K., Chen, J., Zhu, T., Lou, R., Tian, Y., Xiao, Y., and Su, Y. TravelPlanner: A benchmark for real-world planning with language agents. In International Confer- ence on Machine Learning, pp. 54590-54613, 2024.
Yang, S., Du, Y., Ghasemipour, S. K. S., Tompson, J., Kael- bling, L. P., Schuurmans, D., and Abbeel, P. Learning interactive real-world simulators. In International Con- ference on Learning Representations, 2024a.
Yang, Z., Garrett, C., Fox, D., Lozano-Pérez T., and Kaelbling, I. P. Guiding long-horizon task and motion planning with vision language models. arXiv preprint arXiv:2410.02193. 2024b.
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, L., Cao, Y., and Narasimhan, K. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems, pp. 11809- 11822, 2023a.
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. R., and Cao, Y. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations, 2023b.
13

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
The appendices are organized follows. First, the background of deep equilibrium models is discussed in Appendix A. Then, we describe henchmarks in Appendix B.1, baselines in Appendix B.2, and our implementation details in Appendix B.3. Lastly, additional experimental results and limitation analysis are provided in Appendices C and D.
A. Background
A.1. Deep Equilibrium Models
Traditional neural networks are constructed by explicitly stacking layers f(i), which can be limited in their expressiveness due to the fixed number of layers and predetermined forward process. Instead, implicit models are defined by an underlying dynamic system be solved, such an ordinary differential equation (Chen al., 2018), controlled differential equation (Kidger et al., 2020), stochastic differential equation (Kidger et al., 2021) or fixed-point problem (Bai et al., 2019).
Deep equilibrium models, first introduced in Bai et al. (2019), are a representative class of implicit models characterized by fixed-point problems. Given an input с and a function fo (2x c) such as a Transformer block (Bai et al., 2019) or a Transformer (Geng et al., 2023), deep equilibrium models define infinite-level stacking of this function zi+1 = So (2i, e) with 0, 1, .............. L and L + oo by solving the solution x² to the following fixed-point equation defined by fe and c:
= fffoxx,c)
(11)
The forward pass of deep equilibrium models is root solving for the fixed-point problem. A common choice is the fixed-point iteration method, which starts from an initial guess zu and iteratively applies the transformation 2/+1 = f c) until convergence. A sufficient condition for its convergence is if So is contraction mapping w.r.t. x, namely its Lipschitz constant is less than one (Banach, 1922), which could be relaxed by the well well-posedness condition in El Ghaoui et al. (2021), More advanced root solvers include Broyden's 's method (Broyden, 1965) or Anderson acceleration (Anderson, 1965).
A.2. Training Deep Equilibrium Models
Unlike traditional neural networks, whose gradient requires backpropagation through time (Werbos, 1990) at high memory and computational cost, the gradient of deep equilibrium models is computed analytically without differentiating over its forward pass. Given an equilibrium point x fa (a", .c) and loss function L(x", y), the loss gradient w.r.t. the model parameters 0 is provided by the implicit function theorem (Krantz & Parks, 2002; Bai al., 2019) as follows:
0L 00
0L (1 Oxt
fo Ox
af 30
(12)
Its proof is given in Appendix A.4. Due the challenge of exactly computing the inverse Jacobian term A = (1 in the above gradient, existing work often approximate via the damped fixed-point unrolling the Neumann series (Geng et al., 2021b). Recently, Fung et al. (2022); Geng et al. (2021a) propose to approximate the inverse Jacobian term by A R I, the former proving it under strong theoretical assumptions. In practice, dropping the inverse Jacobian/Hessian has been used extensively in one-step gradiens (Bolte et al., 2023; Luketina et al., 2016; Finn et al., 2017; Liu et al., 2019; Garima et al., 2020) and shown to be effective on Transformer-bart LLMs (Choe et al., 2023).
0/a
De
A.3. Transformer-based Deep Equilibrium Models
Deep equilibrium models are initially proposed on Transformer architecture (Vaswani et al., 2017) for language modeling tasks (Bai et al., 2019). This seminal work considers a Transformer block as the basic unit fe in the equilibrium model. Then, Geng et al. (2021a) investigates improvements over the Transformer block by replacing self-attention with matrix decomposition. They also introduce one-step gradient based on the approximation of A se I for efficiency and stability, assuming that the Lipschitz condition apply to large number of matrix decomposition methods.
Recently, following the prevalence of Diffusion Transformers (Peebles & Xie, 2023), deep equilibrium models are extended to image generation tasks. Geng et al. (2023) propose generative equilibrium Transformers consisting of two modules, one using Transformer as the basic unit fo in the equilibrium model. Their method yields advanced one-step image generation results. Bai & Melas-Kyriaz (2024) replace most of the intermediate Transformer blocks with an equilibrium model, thus significantly reducing the number of parameters and memory usage for training and inference.
14

---

Figure visualizes few examples sampled from the VirtualHome- benchmark. In each example, the planner is placed in an environment that spans few indoor rooms and is given detailed description of the environment. The description is originally in the form of scene graph with objects as nodes and spatial relationships as edges, but for simplicity we present the planner with only the object nodes. The planner is then asked to generate a semantic action sequence based on short task description. For instance, after receiving an instruction "turn on .V..." ...,, the robot agent must first walk to the table and grab the remote control, and then point at the TV to turn it on. As seen, these planning tasks usually involve rather Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
A.4. Proof of Implicit Function Theorem
Theorem A.I. (Implicit Function Theorem (Bai et al., 2019; Krantz & Parks, 2002)) Let L M Rd R be differentiable loss function, and let fo R x R - R be differentiable function parameterized by 8 Ry. Consider the following optimization problem:
min
L(x",
0
(13)
s.t. x² fol ,c).
where ,y y R", and E IRP, (
01o Ox
is invertible, then the loss gradient w.r.l. 0 is given by:
ØL
0L
(I
050 0 Ox
(14)
00
Oxt
00
Proof of Theorem 3.1. 7. To derive the loss gradient w.r.t. 8, we begin by differentiating the equilibrium condition x² fo(x".c) with respect to 0. Applying the chain rule, have:
Ox x
0ʃ
0ʃ Ox
(15)
00
00
Or
a0
Given that (1-
is invertible, we can rearrange the above equation and solve for dz" Ox")
0le)
00
Ox
afa
Qfe
(16)
a0
Ox
00
The chain rule implies
0L 88
aL Ox" 8z
Substituting the expression for
we obtain:
88
88
0L
0L
0ʃe
0fa
1
(17)
00
Ox
Ox
00
B. Experimental Settings
We detail the benchmark in Appendix B.1, 1,1, the baselines in Appendix B.2, and our implementation details in Appendix B.3.
B.1. Benchmark
Environment. We adopt the robotic planning benchmark AitualHome-EEn (Liao et al., 2019) based on VirtualHome (Puig et al., 2018). It consists of a complex set of 292 planning tasks in 7 different indoor scenes, provided with 1360 mid-level action trajectories as ground truth annotations. These action trajectories are typically very long, with an average execution length of 10.8, highlighting its long-horizon characteristic. Moreover, the VirtualHome environment provides detailed feedback after performing each mid-level action, making
ideal testbed for closed-loop plamning.
complex scene setup, and the ground truth action sequences are quite long. We provide more detailed statistics in Figure 8.
Compared alternative embodied planning benchmarks, VirtualHome- Env features both long time horizons and closed-loop feedback. For example, ALFRED (Shridhar et al., 2020) and ReALFRED (Kim et al., 2024) are two common embodied instruction following benchmarks, but their plan lengths are relatively short and can be determined by a few templates, making them unsuitable for long-horizon planning. PlanBench (Valmeekam et al., 2023) and TravelPlanner (Xie et al., 2024) are recent benchmarks designed specifically for LLM planning, but they do not provide closed-loop feedback during execution, which is an essential element of robotic planning. Therefore, we adopt VirtualHome- during the experiments.
15

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Task description:
Dask description:
Task description:
Bring ne red cookbook
Bring dirty place to sink
Turn on TV with remote
Ground truth:
Ground truth:
Ground truth:
[NALK] chome_office>
[AAK]] <dining_roum
[KALK]
<home_office
[MAL]] <bookshelf>
[WALK] <table>
[NALK]
[FIND] <noval>
FIND) <table>
[FIND] <remate_ contrrll
[GRAB] <novel>
TURNTO] stable>
[GRAB] <remste_controi>
[NALK] <table>
FIND) <plate>
[FIND] <televisiars
[PUTRACK] <novel> <table>
GRAR <plate>
[TURNTC] <televiatons
sdining_room
[POINTAT] <televieion>
WALK] <sink>
[SRITCHON] <television
FIND]
(PUTOBJBACK] <remote_contrel>
FUTRACK] <plate> <sink>
(a)
(b)
Figure 7: Examples in VirtualHome-En (Puig et al., 2018; Liao et al., 2019). The planner is given detailed description of the environment (specifically, the objects within each rooms), a short task instruction, and is asked to output a sequence of mid-level actions associated with the correct objects.
Action. The VirtualHome environment (Puig et al., 2018) originally supported animating 12 atomic actions based on the Unity simulator, with the followup work VirtualHomee (Liao al., 2019) adding support for more actions using a graph simulator. It currently supports 40 atomic actions, in which 21 actions can be animated through Unity. Each action defined by an action name and some object arguments, and implemented by prewritten code executors. In our experiments, we full set of 40 actions included in the VirtualHome-Emn dataset, summarized as follows:
1. Actions without object association: SI.EE,, STANDUP, WAKEUP.
2. Actions associated with one object: WALK, FIND, GRAB, WASH, WIPE, PULL, PUSH, POUR, TURNTO, POINTAT, WATCH, TOUCH, OPEN, CLOSE, RUN, SIT, READ, PUTON, PUTOFF, DROP, LIE, SWITCHON, SWITCHOFF, DRINK, LOOKAT, TYPE, CUT, PUTOBJBACK, EAT, RINSE, PLUGIN, PLUGOUT, GREET, SCRUB, SQUEEZE.
3. Actions associated with two objects: PUTIN, PUTBACK.
Feedback. Because the environment includes graph simulator of the scene graph, can respond quickly actions, e.g. changing object attributes, and provide the updated scene graph each step. In experiments, curate several types of closed-loop feedback based on these scene graphs, simulating coarse feedback that may be received in real-world situations. Specifically, we consider the following four categories of environmental feedback associated with task failure:
1. Program format (eedback:"Yout output does not conform to the required format", indicating that the generated action sequence does not conform to the required format.
2. Invalid command feedback: "Your output has an invalid command: ...", indicating that the generated action sequence has illegal command line.
3. Execution feedback: "Your output is executed incorrectly in the environment.", indicating that the generated action
sequence cannot be executed in the environment.
4. Task completion feedback: "You have not completed this task. The following objects and corresponding states do not meet the goals: The following objects have wrong relative position: indicating that the generated action sequence cannot complete the task, with more details about the task failure.
16

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
140
500
120
Number of edges
480
100
Frequency
460
80
60
440
40
420
20
400
10
20
30
280
285
290
295
300
Plan length
Number of nodes
(a) Distribution of plan length
(b) Distribution of scene graph size
Figure 8: Detailed statistics of VirtualHome-Env (Puig et al., 2018; Liao et al., 2019). features (a) a large set of long- horizon plans with an average length of 10.8, and (b) complex scenes containing more than 280 objects and more than 400 valid relations. For clarity, we exclude the CLOSE and FACING relations, which are redundant for most planning tasks.
Table 6: Comparison of dataset protocols of VirtualHome (Puig al., 2018). Since previous works have released their dataset, we use the original VirtualHome-En dataset (Liao et al., 2019) and perform own partitioning.
Public
#Tasks
#Scenes
Task Content
Task Splits
Tree-Planner LLM-MCTS
35
4
Household
Novel scene and task
2000
4
Rcarrangement Novel seene and task, Novel scene, Novel task, Seen seene and task
Ours
1360
7
Household
Novel scene and task, Novel scene, Novel task
Dataset Split. We randomly divide the VirtualHome- dataset into training set and set in a 50:50 ratio. To analyze the generalizability of our method, mainly study the following three subsets of the test set: novel scene set, novel task set, and novel scene and task set. For instance, the novel scene set consists of seen planning tasks unseen scenes. Overall, the dataset contains 735 training trajectories, 468 trajectories within the novel task set, 95 trajectories within the novel scene set, 62 trajectories within the novel scene and task set. Our models are first trained on the training set for fixed number of epochs and then evaluated on the three test subsets above.
Note that there are two alternative dataset protocols for VirtualHome, represented by LLM-MCTS (Zhao et al., 2023) and Tree-Planne (Hu al., 2024), as summarized in Table 6. Specifically,, LLM-MCTS uses the indoor scenes of VirtualHome and synthesizes its dataset focusing object rearrangement tasks. evaluate for training-free methods. However, since neither works has released their detailed dataset, therefore we simply use the original VirtualHome-Eng dataset and partition it accordingly in our experiments.
uses subset of VirtualHome Eny to
B.2. Baselines
Our method is mainly compared with Tree-Planner (Hu al., 2024) and SELF-REFINE (Madaan et al., 2023), both reproduced using Llama 3 8B Instruct (Dubey et al.. 2024) in line with ours. The former traverses an action tree that is built by repeated plan sampling, while the latter relies on self-refinement. To reproduce them, we perform supervised finetuning of Llama 3 on the training split of VirtualHome-Em for the same number of epochs as our method, and then follow their original procedures for inference. For instance, Tree-Planner is reproduced with both settings E (25, 50) in action tree construction. The system prompts they use are similar to ours in Figure 9.
We also report the results summarized by Hu al. (2024) for reference. They additionally considered ProgPrompt (Singh et al., 2023), Zero-shot Planner (Huang et al., 2022) and two self-refinement planners, Local Replan (Raman et al., 2022; Guo et al., 2023) and Global Replan (Shinn et al., 2023). Since these baselines were implemented by calling the GPT-3.5 API instead of finetuning Llama 3, we report them in the novel scene and task track for relatively fair comparison. worth noting that they adopted a smaller subset of actions and feedback, and differed in the curation of partial observations of the environment. Therefore, their results are presented for reference only.
17

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
There are several robotic planning baselines that we have not compared due large environmental differences. For example, LLM-MCTS (Zhao al., 2023) is representative tree-search (Yao al., 2023a) based planner. It followed Watch-and-help (Puig et al., 2021) to generate dataset of simple embodied tasks (mostly object rearrangement tasks), while our work considers more complex set of planning tasks, see Table 6. Alternative planners based on symbolic scene graph (Zhu et al., 2021; Rana et al., 2023), code (Liang et al., 2023; Sun et al., 2023), or PDDL (McDermott, 2000; Liu et al., 2023; Guan et al., 2023) are less flexible and difficult to implement in our environment.
You need to act as a task plannez, who first deafs an initial sub-task sequence and then zefine it in the next Iav iterations.
when the
sut-task sequence
should output initial sub-task sequence.
Khan the the draft sub-task saquance not Noll, You should zatine based on the draft sub-task sequence.
If you have previously generated some action sequences and tried to exccute thom in the enviromentt, thoir reodback will bo provided Lo for reference.
Bach aub-task
following forn: [action_rana)) 2. [actior_name] <abjact 1> lobfact netion_nene] sobjest name (ob3ect id <oatest aeme 2> (object 21.
System Prompt
The job juct. 14) is used to tell the simulater which objectt the action should ast p0.
The cumber of arqunenta depends on the action type.
action
1, the available accion type 2, the available actiona ares WAL3, FIND, CRAB, HABH, WIFE, POLL, POSH, POUR, TURNTO, POINTAT, WARCH, TOUCH OEEN, CLOSE, ROE, EI2, READ, EUTON, PUTOEF, DRCP, 213, SWITCHON, SMITCHOFE, DRINK, 2O0KAS, EYPE, CUT, PUTOBJEACK, EAT, RINGE FLUGIN, PLOGOUT, GREZT, SCRUS, SQUREZR
ELESE, STANDUE, MAREUF
action type 3, the available actions
POTIN, EUTBACK
All action_nane of the sub-tasks must chesen from above acticas.
You should output Lhe aub Lask sequence in suscinet focm,
You muat output END after have outout the entire sub-task sequence.
Task
Grab some juice
Ies:ruct lona:
Task
fuidge,
juico
then glass, the juice into the olass.
Thare ase rooma, you are enbodied charactor wish ID 198 in bodroon with 139.
Tho objects in each roon in as rollows:
hene_offies
Boon ID:
dbject ID and name in this rooms
28 hanger
Roon name: dining_room
Env
Roon ID: 100
and this 116 coiling
2003 food__ood
Teadbacks
executions:
Action soquenze:
[WALK] <dining_raun> (1C0)
[MAL]] <oupboard> (152)
[FIE]] <cupboard> (132)
[088K] Souphoard> (132)
[FN[]] <oup> (1030)
[GR18] <oup> (1030)
[C1OSE: scupboard> (132)
[MAK]] <freoze>> (141)
[0PЕ5] <treezer> (142)
Feed- back Ca.
[FIN]] <juice> (1001)
[CRA6] <fuice> (1001)
[FCUR] <juice> (1001) krupó (1000)
[FU70BJBACK] <'uice> (1CD1)
[CLOSE] <freezer> (141)
[END]
Feadback:
You have not complsted this task.
The Following ub jects have ielative positions (100C, tup) and (128, table).
The draft
sequence:
[WAL]] <dining_raon> (1C0)
[MALK] <oupboard> (132)
[PIND] Kouphnard> (152)
[0P8K] <eupboard> (152)
[FI[]] <oup> (1030)
[GRA8] Sougo (1030)
[C105E] scupboard> (132)
Draft
[MALK] <freczar> 1141)
[0088] <Erewses> (141)
Plan
[FI[]] <juice> (1001)
[GRAN] <juice> (1601)
[[0U]] <juice> (1001)
(1000)
[ЕUT0B]BACK] stuice> (1CD1)
[CtneR] <freezer> (141)
[EXD]
Figure 9: Example of the prompt used by our equilibrium planner.
18

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
B.3. Implementation Details
Prompt. Our approach involves two LLMs, one LLM as the equilibrium planner and an additional LLM an optional world model. The planner's 's input is illustrated in Figure 9. As can be seen, it consists of five parts: system prompt, task definition, environment description, history feedback, and draft plan. Notably, the system prompt is modified from Hu et al. (2024), and the environment section describes the initial environment sorted by rooms, including the object names with their IDs within each room. For the optional world model, we adopt similar prompt, except that it receives more information about the initial environment, including edges in the scene graph that indicate spatial relations. This additional information helps the world model to better predict the environmental feedback given a generated plan.
Finetuning. Both our equilibrium planner and the world model are finetuned in a supervised manner. The equilibrium planner is finetuned for 6 iterations with learning rate of 0.0002. The training data is constructed adaptively using all previous equilibrium solutions. Specifically, an equilibrium memory is maintained that buffers all equilibrium solutions, including the newest ones. At each iteration, we curate the training data by weighted sampling from this memory (where the newest solutions are sampled more frequently) and then pairing them with the ground truths. To prevent overfitting to the history equilibrium, a decay ratio of 0.5 is used when sampling from the fixed points of previous iterations. Thereafter, we update the model parameters using gradient descent according to Equation (6) for one epoch per iteration.
For the world model, collect all interacting experiences between the planner and the environment, including plans and feedback, and finetune it for 5 epochs using the same learning rate of 0.0002. The world model is initialized from Llama 3 8B Instruct and supervised finetuned on all the planner's s environmental interactions. This procedure takes place after the equilibrium planner has completed training, so that all of its data can be leveraged at once. The finetuning data is constructed in a format similar to Figure 9 in the appendix, with a different order to predict feedback from the plan. Finetuning the world model takes about 30 hours due to its longer context (e.g. spatial relations).
Inference. The inference of our planner is described in Algorithm 1, which involves a nested equilibrium solving process. Given the environment and the task instruction, we initialize the draft plan Bi and the feedback co as null and iterate through nested loop. Each inner loop reuses the feedback from the outer loop to self-refine the draft plan, and after the inner loop converges, we update the feedback by interacting with the environment or world model, The ratio of environmental interactions to world model calls is currently set to 1:1, i.e., the planner alternates between using the environmental feedback and the world model at each loop. Note that is possible to reduce this ratio and the number of environmental interactions required if the planner has access to a more accurate world model. This process continues until it converges to an equilibrium point or reaches an upper bound on the outer loop, which we set to 10 to match Tree-Planner (Hu et al., 2024) but is rarely reached. Thus, the inference compute used is mostly determined by the convergence speed of the model itself.
Our LLM sampling strategy facilitates model convergence. Specifically, except that the first refinement step uses top-k sampling with k = 10 for higher diversity. Since most of the text prompt remains unchanged during equilibrium solving, we employ KV cache to accelerate inference, which can be further improved with parallel decoding techniques (Santilli et al., 2023; Cai et al., 2024; Kou et al., 2024).
greedy sampling to stabilize LLM outputs,
C. Additional Results
Effectiveness of our full method. We exemplify the self-correction trajectories of our full method in Figs. 10 and 11. Compared to SELF-REFING (Madaan et al., 2023) and Tree-Planner (Hu et al., 2024), our approach is more competent in revising a long plan through few forward passes without additional system 2. This is attributed to our efficient training scheme for teaching plamners to self-refine. We also compare different types of feedback utilized by our planner in Figure 12. As can be observed, internal feedback alone cannot enable successful replanning. but it can reduce environmental interactions prior to convergence. This confirms the effectiveness of both internal and external feedback in closed-loop planning.
Effectiveness of the inner loop. Table 8 illustrates that even without feedback, our method yields significant performance improvements. To better understand this, we visualize the self-refining traces of the inner loop in Figures 13 and 14. Specifically, Figure 13 shows the inference trace at each inner-loop iteration. Even though the planner only succeeded in later steps, there are consistent quality improvements in its output during the inner-loop introspection without any feedback. Figure 14 compares our inference trace to a prompting-base baaedd alternative, where our method shows to be more effective at steering the output toward a correct plan via the inner loop. The working mechanism of our inner loops was mentioned in the introduction, iee allowing for bidirectional dependency as well as scaling of inference-time compute.
19

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Table 8: Effectiveness of our method in the no-feedback setting, where it shows clear performance advantages.
Novel Scene and Task
Novel Scene
Novel Task
Exec.
SR
GCR
Exec.
SR
GCR
Exec.
SR
GCR
Supervised SELF-REFINE
93.55
24.19
32.55
96,84
41.05
49.81
95,94
26.07
35,53
72.58
32.26
52.29
74.74
44.21
62.25
65.38
30,98
51.80
Ours
88.71
33.87
59.98
96.79
49.47
66.60
93.80
34.62
59.06
Table 9: Comparison Tree-Planner with and without world model. Only our method shows a significant improvement.
Novel Scene and Task
Novel Scene
Novel Task
World model
Exec.
SR
GCR
Exec.
SR
GCR
Exec.
SR
GCR
P1eeePlanners-25
95.16
38.71
63.18
96.08
51.58
69.45
95.50
40.38
63.75
Tree-Planee Piee-Plannerr-25
96.25
38.71
58.71
98.81
51.58
63.94
96.66
38.46
57.40
Tree-Ple Pre-Pllaneer-s0
94.94
38.71
63.50
96.06
51.58
69.54
95.40
39.74
63.29
Tree- Pree-Plannern-so
96.16
37.10
57.53
98.22
54.74
69.64
96.80
39.32
58.84
Ours
88.71
33.87
59.98
96.79
49.47
66.60
93.80
34.62
59.06
Ours
90.32
40.32
65.40
95.79
65.26
79.47
93.38
41.88
62.76
Comparison with Tree-Planner. As shown in Table 9, our performance is inferior to Pree-Planeer in the no-feedback setting because of the difference in refining single plan with <100 iterations instead of generating 25 or 50 candidate plans (giving Tree-Planner a comparative advantage). However, in the other settings that allow feedback, our method outperforms Tree-Planner by incorporating feedback more flexibly. By taking into account feedback through forward passes of LLMs, our method allows arbitrary changes based on the LLMs' knowledge, and correcting multiple errors in parallel (Figure 10c). In contrast, tree-based alternatives require backtracking in tree, which is costly when correcting an early mistake and does not fully exploit the implicit knowledge in feedback. For example Figure 11, Tree-Planner ignored the earlier feedback and made the same mistake twice ([OPEN] (laptop)).
Generalization to environment without feedback. we test our pre-trained VirtualHome planner on ALFRED benchmark with updated prompts. For evaluation, we consider two planning metries: task classification ac-
Table Zero-shot evaluation on ALFRED. Our model demon- strates better generalization without retraining.
Task classification acc.
Action-object recall
curacy (across 7 task types in ALFRED) and recall of ground- truth actien-object pairs in the predicted plan. As shown in Table 7, our planner generalizes significantly better than the supervised trained planner.
SFT Ilama
1 1%
0.50%
Ours
54 %
27.08 %
D. Limitations
While our equilibrium sequence modeling improves the planning capability of LLMs, we identify the following failure scenarios during the experiments: (1) hallucination of the equilibrium planner and the world model in vanilla LLMs; (2) lack of awareness of history context such as previously grabbed objects. The latter can be resolved with the context module in Kim al. (2023a) reasoning techniques as in Yao et al. (2023b).
In a broader sense, our method may be limited in generalizing to new domains because it requires the ground truth and environmental feedback during training. These procedures with the equilibrium solving process results in lower training efficiency. Also, the current formulation only considers the explicit output plan without implicit reasoning steps. While it is possible to synergize planning and reasoning with their combined advances, we leave this future work due to feasibility constraints for small-scale experiments with large reasoning LLMs (Jaech et al., 2024; Guo al., 2025). Furthermore, our model has only text input and visual input, which limits its applicability in the real world. This be resolved by introducing video-based planners (Du et al., 2024), world models (Yang et al., 2024a; Brooks et al., 2024) and vision-language models( (Yang et al., 2024b; Athalye et al., 2024).
20

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Output log
Uctpuc leg
Output log
[NALK] <bedroom>
MATX] <badroom>
[NALK] <hedroom>
[NALK] <toy.>
[WAKK] <toyl>
[NAaK] <toyl>
[FIND] <toy.>
FIND) <tayl>
[FIND] <toyl>
[GRAB] <toy">
.GRBB <toy1>
[GRAB] <toyl>
<Loy2>
FIND <Loy2>
[KALK]
[GRA5] <toy2>
[GABB] <toy2>
[PUTBACK] <toyl> <table>
[FIND] <toy3>
faf trackback
[NALK] <toy2>
[GRAB] <toy3>
[FTND] <toy2>
[FIND] <desk>
[ATK] <dresser1>
[GRAB] <toy2>
[PCTRACK] <toy1> <desk>
[PPEN] sdresserls
[KAM]] <table>
[PUTBACK] <Loy2> <desk>
[PUTIN] <toyl> <dresserl>
[PUTDACK] <toy2> <table>
[PUTBACK] <toy3>
(CLOSE] <iresseri>
[KALK] <toy3>
Wrong relative gos: <t,yy> and <tiling_cabinet>
FIND <toy2>
[GAAB] <tay2>
[GRAR] <toy3>
[FIND] <tey]>
<tey2> and <filing_cabinet>
fail,
[WALK] <table>
<tcy3> and <Filing__abinet>
[ATK] <deak>
[PUTDACK] <toy3> <table>
[NALK] <bedrcom>
FUTBACK] <toyl> <deak>
Wrong relative pas:
[NALK] <toy.>
Task fail, traexback
<toy1> and <f1 <fillng__sbinnn
<Loy2> <filing_cabinet
[G8AS] <toy.>
<toy.>
[WALK] <dresser2>
<toy3> and <filing
[OPEN] <drasser2>
[FTN]] <toy2>
[NALK] <hedroom>
FUTTN] <toyl> <dresser2>
[GRAB] <toy2>
[MALK] <toyl>
[FIND] <toy3>
fail
[FIND] <teyl>
[ВRА]] <toy3>
[G8AB] <toyl>
[FTN]] <desk>
[NALK] <filing_eabinet>
[PUTBACK] <toy1>
[CPEN] <filing_oebleet>
[PUTBACK] <toy2> <desk>
[PUTBACK] <toyl>
[PUTBACK] <toy3>
<riling_cabtret>
[FIND] <bookehelD>
[FIND] <Loy2>
[PUTBACK] <teyl> <bookshelf>
[GRAB] <toy2>
[PUTBACK] <toy2> <beokshelf>>
[NALK] <filing__abinet>
[PUTRACK] <toy3> <hookshel
[PUTRACK] <toy2>
<tiling_oebiret>
Wreng relative gos: <tcyl> and tfiling_cabinet
[FTND] <toy3>
<tcy2>> silling_eabinet.
[GRAB] <toy3>
stoy3>> and siilinggeabbine
[KALK] <filing_ebbinet>
[PUTBACK] <toy]>
<filiing_ebireet> eabiret>
Same
task fail
[CLOSE] ccibiing_
(a) SELF-REFINE
(b) Tree-Planner
(c) Ours
Figure 10: Visualization of self-correction process in comparison with baselines. This example uses only environmental feedback, and include toy IDs in the presentation for clarity. The task instruction is "Pick up all the toys on the floor and put them in their correct storage bin or shelf"
21

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Output log
Dutput log
Output log
[NALK] shome_off!:cm
[ATX] shame_officez
[WAIR] shome_offion
[KALK] <laptop>
[ALK] <laptopa
[KALK] <laptop>
[FIND] <laptop>
[FIND] <lspoop>
[FIND] <laptop>
[GPEN] <laptop>
[OPEN] <laptop>
[SWITCHON] <laptop>
[FTNN] <mai">
Action fail, trackback
[TURNTC] <laptop>
[GRAB] smail>
[100KA7] <laptop>
[FTND] <desk>
[GRAB] <laptop>
[FTN]] <mail>
[CPEN] sdesks
.FIND] <chair>
[READ] <mail>
[PUTBACK] <_aptop> <desk>
[S11]
[TURNTC] <mail>
[PUТBACK] <mail> <desk>
[SITCHCN] <laprop>
[POLNTAT] <mail>
[FIND] <keyboard>
[IND] <mail>
[GRAB] <mail>
[TYPE] <keyboards
[RAB] <mail>
[FIND] <keyboard>
Wreng state:
FIND <keyboard>
RRA3 <mai
[TYPB] <keyboard>
<cherseter> ISIT:ING)
TYY2E] <keyboard>
Wrong state:
Wrong ve pos: scharacter> and <mail>
Scharacter> [STT*TNG]
fail, trackback
Prong relative
<character> and <cheir
FIND <daak>
<charecter> and <mail>
[PPEN] <deak>
<charseter> and <chair>
Some plan, tack fail
Action fail, trackback
[KAGK] <home_office>
[KALK] <laptop>
[WALK] <table>
[FIND] <laptop>
[OPEN] <laptop>
[SWITCHON] <laptap>
Action fail, trackback 2
[TURNTC] <laptop>
MATK]
[10OKAт] <laptop>
[OPEN] <laptopp
[FTN]] <chair>
[SIT] <chair>
Action fail, trackbsck
[PIND] <mail>
[TURNIO] <laptap>
[G8A8] <mail>
[00KKA]] <laptop>>
[REAn]
[WITCHON] <laprop>
[TURNTC] smail>
FIND <chair>
[POINTAT] <mail>
[STT]
[KRITE]
FIND <mall>
[SKITCHOPF] <laptop>
GRAB
Invalid command:
READ <mail>
[68ITE] <mail>
FIND <keyboard>
TYPE)
<keyboard>
[NATK] <home_afficen
[WALK] <laptop>
cak scecess
[PIND] <laptopa
[WWITCHON] <laptop>
(TURNTC) <laptop>
[10OKAT] <laptop>
[FIND] <chair>
[8TT] <chafr>>
[TURETC] <laptop>
[РOΣNТAТ] <laptop>
[TURNTC] <laptop>
[100RA7] <laptop>
[PIND] <mail>
[GRAR] <mail>
[READ] <mail>
[TYPB] <laptop>
Task success
(a) SELF-REFINE
(b) Tree-Planner
(c) Ours
Figure 11: Visualization our self-correction process in comparison with baselines. This example uses only environmental feedback. The task instruction is "Open email application, open new emails and respond accordingly". Our proposed method succeeds with fewer external feedback.
22

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Output log
Output log
Output log
[NALK] Deome_office"
[WALK] <hame_office>
[MALK] phome_office"
[NALK] <laptop>
MAIX] <laptop>
[NALK] <laptop>
[FIND] <laptop>
[+IND] <laptop>
[FIND] <laptop>
[SKITCHON) <laptopp>
[WITCHON] <laptop>
[SWITCHON] <laptop>
[TURNTO] <laptop>
TTURTTO] <laptop>
[TURNTC] <laptop>
[100KA]] <lapLop>
[LOOKA]] <laptops
[100KA7] <laplop>
[FIND] <mail>
.FIND] <mail>
[FIND] <mail>
[READ] <mail>
[READ]
[READ]
(TURNTC] <mafl>
TTRRNT]] <mafl>
[TURNTG] <mail>
[POINTAT] <sail>
[ОINTAA] <mall>
[POINTAT] <mail>
[GRAR] <mai
[RARR <mail>
[GRAR] <mat
[FIND] <keyboard>
.IND] <keyboard>
[PIND] <keyboard>
[TYPB] <keyboard>
(YPP)] <keyboard>
[TYPE] <keyboard>
World model feedback
Envizonmental feedback
World model feedback
Wreng stase:
Krong state:
Wrang state: Kehsracters (SITTING)
<character> (SITTING)
<chazacter> (SITTIN3)
<oomputer> :ON)
Krong relative pos:
<eomputer> (ON)
<eomputer> (FLUGCED_IM)
<character> and <mail>
<oomputer> (FLUGCED_
Wrong relat' ve pos:
<character> and <choi>>
Wrong relative pos:
Kcharacter>
<hame_office>
Kcharacter> <chair>
[NALK] <home_office>
[ALK] <laptop>
[NALK] <homs_office>
[NALK]
FIND) <paptop>
[NALK]
[FIND] <chair>
SNITCHON] <lapcop>
[FIND] <chair>
[SIT] <cheir>
TURNTO] <laptop>
[SIT] <chair>
[FIND] <oomputer>
[ooKAt]] <leptop>
[FIND] <computer>
[SKITCHON] <computer>
(FIND) <chair>
[SWITCHON] <computer>
[TURNTO] scomputer>
<chair>
[TURNTC] scomputer>
[10OKAT] <computer>
(FIND) <mail>
[10OKAт] <oomputer>
[FTND] <laptop>
GRAR <ma1
[FTNN] <laptop>
[TURNTO] <laptop>
RBAD] <mail>
[TURNTC] <laptop>
[10OKa]] <laptop>
TTRRIOO] <mail>
[1ooкaт] <laptop>
[FIND] <mail>
[FINNAAT] <mail>
[FIND] <mail>
[TURNTG] <mail>
WRITR] <mall>
[TURNTC] <mail>
[100KAT]
[SITCHOFFF <laptop>
[10OKAT] smail>
[CRAB] <mai">
Ezvizonmental feedback
[GRAB] <mail>
[READ] <mai_>
commond:
[READ] <mail>
[PIND] <keyboard>
[RITE] <mall>
(FIND) <keyboard>
[ΣxP=] <keyboard>
[1YPE] <keyboard>
[RALK] <hom__ofice>>
World model feedback
[WAKK] <laptop>
Invironmental feedback
Task success
FTND] <psptop>
Wrong relative gost
fail
SWITCHON] <lapcop>
<mail>
TURNIOO] <laptop>
[WALK] <home_office>
[OCKA]] <laptop>
[MALK] <laptop>
FIND) <chaii>>
[FTN] <laptop>
'S17] <chsir>
[SWITCHON] <laptop>
TTURNTO] <laptop>
[TURNTC] <laptop>
[OINTAT] <laptop>
[10ORAT] <laptop>
TURNTO] <laptop>
[PIND]
[LOOKA]] <laptap>
[S1T] <chair>
FIND)
[TURNTO] <laptop>
GRAR <mail>
[POTETAT] <laptop>
READ] <mail>
[NArCH] <laptop>
TYPF) <laptop>
[FTND] <mat
Eask aveceas
[TRRTTC] smail>
[100KAτ] <mail>
[GMAS] <mail>
[READ]
[TYPE] <laptop>
World model feedback
Task
Task success
(a) Ours w/o external foedback
(b) Ours w/o internal foodback
(c) Ours
Figure 12: Visualization our self-correction process with different types of feedback. The task instruction "Open email application, open new emails and respond accordingly".
23

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Output log
Output log
Output log
[NALK] <bethroom?
[ALK] <bathroom>
[NALK] <bathroom>
scleaning_solin fon>
inleening__m maaiingsee solouutioon
[PIND]
<cleaning_eolutioon
FIND
dleening_sclutee
[PIND]
ccleaning_eolutionn
[GRAB]
<cleaning_solution>
[GRAB]
<cleaniag_solution:
[FIND] <rag>
FIND
<sink>
[FIND] <eink>
[GRAR]
<rag>
FOUR
celeannngsoo f
[POUR] scleaning_sooutlle
[POUR] soleeniing_eouution
<aink>
<sink>
<reg>
[TURNIO] <sink>
[KIPE] <sink>
[PUTOBJSACK]
[LOOKA]]
<sink>
[PUTOBJSACK]
[FINU] <cleaning_eoluttion>
[TOUCH] <sink>
coluuion
1oop convergeace
(TURNTC] <sink>
[100KAT] <sink>
[ALK]] <bathroam>
Task success
[IOUCH] <aink>
MATX
seleaning_so on?
FIND
eeleaning_solutoon
Inner self-refinement
GRAB
scleaning_sclute
[NALK] <bathroem>
FIND
<sink>
[NALK] <cleaning__oolutfon
FOUR celeanig_aoolute lution
<cleaning_eoluuionn
<aink>
[GRAD]
<cleaning_eoluttonn
[AITT
[FIND]
<rag>
FIND <rag?
[POUR] <clesning_eolutioon>
GRAB
<rag>
[WPEE] <rag> <sink>
[PUTORJSACK]
FUUOBBJAAC]] <rag>
[FIND] <oleaniingeeolurtonn
SNITCHON] <faucet>
RINSB]
[SURNTO] <sink>
SWITCHCFF]] <faucet>
[100KA5] <sink>
loop convergence
[2OUCH] <sink>
Ervivormental feedback
Inner 1oop convergeace
Lrvalid command:
ronmentlt feadbask
MAIT)
Wrong relat ve pos:
Keleaning_solle ioso
(a) The inner loop
(b) The 2nd inner loop
(c) The 3rd inner loop
Figure 13: Visualization our self-correction process with inner-loop traces. This example uses only environmental feedback. The task instruction is "Spray some cleaner and wait minute then wipe it out", As can be seen, each inner loop improves planning quality: the first removes redundant [GRAB] action, while the second adds crucial [WIPE] action and other interesting details.
24

---

Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling
Output log
Uutpu: lcg
[KALX] <bathroom>
[MALX] <bothroom>
[KA13] selectric_shaver
MALK selectric_shave
relectrie_shever>
<electric_shave
[GRA3] <electric_chavep>
[GRAB] @leetric__hhver>
[FIND] <mirror>
[PIND] <mirior>
[TURNTO] <nirror>
[TURNT:] <mirror>
[LOCKAT] <nirror>
[LOOKAT] <mirror>
[FTND] <ehair>
[SNTTCHON] <electrio_dhaver>
[8I7] <chair>
[3UN] <pirror>
[РWITCHON] <electric_ehave>>
[TURNIO]
[*I <beard>
[LOCKAT] <mirror>
[TURNTO] <beards
[SNITCHOFF] #ilettricshaaeer
[РОЗNTАT] <beard>
[еUTOвJHАCK] <electris_thaves>
[FIND] <beards
inner self-refinement
[KTP3]
[ATX]
<bathroom>
Tnner loop convergence
[ALK
<electric_shaver
command:
<elestric_ehaverz
[ [ND] <beards
[RAB
<clestric_chaver>
FTND <mirzar>
TURNTO <mirror>
[LOOKAT] <mirror>
<electric_shavee
[SNITCHCEP]
Electric_shaver
loep convergence
Task success
(a) Prompt-based self-refinement
(b) Ours
Figure 14: Visualization of our self-correction process in comparison with prompting-based method without any feedback.
The task instruction is "Pick up razor and shave yourself". As can be seen, the prompting-base based model converges without any self-correction, while our model achieves success.
25


Reviews:
[{"review_id": "EJfY01PdmT", "reviewer": "Reviewer_nTaB", "summary": "This paper proposes an  equilibrium model-based planner for decomposing high-level tasks into mid-level action sequences in an iterative manner taking environment and world model feedback. Experiments on VirtualHome-Env benchmark demonstrates that the approach can improve over a few existing approaches.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate your valuable suggestions, and we would like to address your main concerns as follows:\n\n[W1] Connection to real-world robot control\n\n* Our work focuses on high-level planning that decomposes each task into mid-level actions, as a complementary direction to low-level control. Extending this to real-world robotics with low-level actions presents several key challenges: (1) effective tokenization of actions and visual inputs, requiring progress in vision-language action models, (2) handling latency from the equilibrium solving process. To address the latter, our method includes designs such as reusing equilibrium solutions to reduce overhead.\n\n[W2] Low success rate towards application\n\n* The lower success rate can be attributed to three main factors: (1) limited data in the VirtualHome-Env dataset, with only 1360 action sequence annotations, (2) relatively small model size (Llama 3 8B), which may constrain capability, (3) the absence of multimodal inputs that humans naturally rely on. We expect significant performance gains by increasing model size and multimodal data.\n\n[W3] Clarity of Figure 3 and Section 3.4\n\n* Figure 3 illustrates our training framework. At each step, the planner takes context $c_t$ as input and outputs a plan $x_t$. Only the subset of equilibrium plans interact with the environment and are stored in the equilibrium memory. During training, the planner receives feedback solely from the environment based on these equilibrium plans.\n\n* The training and inference pipelines are detailed separately in Appendix B.3. Both the planner and world model are trained using environmental feedback stored in the equilibrium memory. They then work together during inference, where the world model provides additional feedback to the planner.\n\n[W4] Use of world model and environmental feedback in Table 3\n\n* During training, we use only environmental feedback to train both the planner and the world model. At inference time, we evaluate the following four setups:\n  1. No feedback: Only inner-loop refinements without external feedback.\n  2. Environmental feedback: Up to 10 rounds, provided after each inner loop converges.\n  3. World model feedback: also up to 10 rounds, provided after inner loop convergence.\n  4. Both feedback: Alternating between environmental and world model feedback after each inner loop, within a total budget of 10 environmental interactions.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes, but it would be better if  the proposed approach could be grounded from high level tasks and mid-level actions on to level real robot executable primitive actions or even control.", "theoretical_claims": "Yes, the proofs in the appendix  from the literature.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "Planning with LLMs or multi-modality foundation models is an important topic to push the AI frontier with physical interaction. This work depicts one way to do so in an iteratively refinement manner and improve over a few existing planning algorithms/paradigms using LLMs as backbone.", "essential_references_not_discussed": "No, as I am aware of.", "other_strengths_and_weaknesses": "Strength:\n\t1. Introducing equilibrium sequence modeling for planning with LLMs.\n\t2. With iterative refinement planning paradigm, environment feedbacks and world model feedbacks can be taken into the planning process.\n\nWeakness:\n\t1. Haven't connect the approach to real-world robot control yet to fully verify the validity of the proposal.\n\t2. Successful rate is still low towards real application.", "other_comments_or_suggestions": "1. Figure 3, it would be better to clarify whether it is an inference framework, training framework or both. As it is now, the input-output relationship of the planner is not clear: no output. And how the output of the planner gets to the environment and world model, how the planner get feedback from them should be depict clearly.  Similarly,  for section 3.4 practical implementation, it would be better to specify the training and inference pipelines separately for clarification reason.\n\t2. In the description for table 3, please clarify or define how the world model and environment feedbacks enter the model during training and inference time separately for each combination.", "questions_for_authors": "As mentioned above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qOTNVgQGwf", "reviewer": "Reviewer_dmhf", "summary": "This paper introduces \"Equilibrium Sequence Modeling,\" a novel framework for robotic task planning using Large Language Models (LLMs). The authors propose reformulating plan refinement as a fixed-point problem solvable with deep equilibrium models, enabling a supervised training scheme for improving self-refinement. The fine-tuned model is then used in the proposed framework, iteratively refining the plan while incorporating predicted environmental feedback from a fine-tuned LLM world model. The authors demonstrate improved success rate over several baselines on novel tasks from the VirtualHome dataset.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the very constructive comments. Below, we first address the raised questions.\n\n[Q1] Advantage of combined feedback\n* The world model's feedback offers an alternative to environmental feedback when the interaction budget is limited. By combining both feedbacks, the planner can perform additional self-correction rounds within the same budget. This leads to faster convergence (see response to W1 below) and thus improved performance (see Table 3).\n\n[Q2] Lower executability\n* The lower executability of our method is due to format issues in overlength outputs. This is likely caused by complex prompts (including history plans and feedback) or limited memorization of ground truths. However, these errors are minor and can be fixed with simple post-processing, resulting in nearly 100% executable plans in the virtual environment.\n\n[Q3] Iterative training with new dataset\n* This is also how we train the model: after each iteration, we update the training dataset with the equilibrium solutions generated by the newly finetuned planner, and use this updated dataset for the next training iteration. As shown in the table below, the overall performance steadily improves over training iterations.\n  |#Iterations|Both|novel||Novel|scene||Novel|task|\n  |-|-|-|-|-|-|-|-|-|\n  ||SR|GCR||SR|GCR||SR|GCR|\n  |0|00.00|00.00||00.00|00.00||00.00|00.00|\n  |2|51.61|74.67||65.26|82.78||**60.68**|**78.40**|\n  |4|51.61|73.30||68.42|82.93||53.20|72.81|\n  |6|**51.61**|**75.13**||**75.79**|**85.79**||56.62|75.53|\n\n[Q4] Convergence analysis\n* We observe that self-refinement consistently converges across different initializations. The table below summarizes the number of iterations to convergence over 10 runs on 60 random tasks, where all tested LLMs (including two new Qwen models) converge to an equilibrium within a finite number of iterations. The intuition behind this is that LLMs tend to repeat themselves, leading to easier convergence.\n  |Method|Mean|Std|Min|Max|\n  |-|-|-|-|-|\n  |Original Llama-3-8B|6.70|2.12|3.22|14.98|\n  |Original Qwen-2.5-0.5B|4.80|2.13|2.69|9.43|\n  |Original Qwen-2.5-7B|7.68|5.03|2.46|16.78|\n  |SFT Llama-3-8B|2.46|0.88|2.02|3.80|\n  |Ours|3.02|1.61|2.32|4.07|\n\n[Q5] Dataset generation w.r.t. base model\n* Since self-refinement consistently converges in practice, the resulting equilibrium can be reliably paired with ground truth to form meaningful training data. This holds true for both Qwen and Llama base models.\n\n---\n\nWe also summarize other weaknesses mentioned in the review and provide our clarifications below.\n\n[W1] Claim of reducing computational cost\n* To clarify, we did not claim that the world model reduces computational cost. Instead, its goal is to reduce the number of environmental interactions, as mentioned in Section 3.4. To verify this claim, we compare the evolution of the success rate (SR) with and without the world model in the table below, which shows that the world model allows a similar or higher success rate even with fewer environmental interactions.\n  |#Interactions|World model|Both novel|Novel scene|Novel task|\n  |-|-|-|-|-|\n  |0|x|33.87|49.47|34.62|\n  ||$\\checkmark$|**37.09**|**65.26**|**40.17**|\n  |1|x|50.00|71.57|**53.41**|\n  ||$\\checkmark$|**53.22**|**75.78**|50.85|\n  |2|x|51.61|74.73|**55.12**|\n  ||$\\checkmark$|**56.45**|**76.84**|53.63|\n  |3|x|51.61|75.79|**56.62**|\n  ||$\\checkmark$|**56.45**|**77.89**|54.91|\n\n[W2] Robustness to environmental disturbances\n* We simulate environmental disturbances by randomly replacing some environmental feedback with wrong feedback during inference. As shown in the results below, our model exhibits stable performance under small amounts of disturbances ($\\le$10%), demonstrating its robustness.\n  |Disturbances|Both|novel||Novel|scene||Novel|task|\n  |-|-|-|-|-|-|-|-|-|\n  ||SR|GCR||SR|GCR||SR|GCR|\n  |0%|51.61|**75.13**||**75.79**|85.79||**56.62**|**75.53**|\n  |10%|**53.23**|73.43||74.74|**85.84**||53.85|73.09|\n  |20%|50.00|73.10||73.68|83.22||54.49|71.80|\n\n[W3] Missing references\n* Thank you for pointing out the related work on planning with VLMs and LLMs. We will update the draft to include their discussion.\n\n[W4] Description of task planning problem\n* Thanks for the valuable suggestion. We will add a brief description of the task planning problem at the beginning of Section 3 to improve clarity.\n\n[W5] Clarity of Figure 3\n* Figure 3 illustrates our planning framework during training, and thus includes training-specific components, e.g. the equilibrium memory you noted. The world model does not directly interact with the environment; instead, it is trained on past experiences stored in the equilibrium memory, as shown by the arrow pointing to it. The single arrow from the environment (and none from the world model) indicates that the planner receives feedback solely from the environment during training. We will revise the figure description to make this clearer.", "claims_and_evidence": "The paper claims that the proposed method has improved planning performance, better scaling w.r.t inference computation, and effectively incorporates closed-loop feedback. \n\nThe evidence for these claims is a set of experiments on the VirtualHome-Env benchmark. The claims of improved performance are supported by the presented results, showing improved success rates over baselines. The results show that the method improves how quickly self-refinement converges, and also how much performance improves with increasing test-time computation. This suggests that the proposed method shows promise for effectively using inference-time resources. \n\nTo support the claim that the method effectively incorporates closed-loop feedback, the authors have an ablation which shows that including feedback correction improves planning performance. \n\nThe claim that the world model significantly reduces computational cost is plausible, but no data or evidence is given for this claim. The results also show a decrease in executability, which needs further explanation.", "methods_and_evaluation_criteria": "Using deep equilibrium models to improve the self-refinement process seems to be a reasonable approach. In principle the approach could be applied to domains beyond robot task planning.\n\nW.r.t the claim of incorporating closed-loop feedback, one of the primary motivations for closed-loop feedback is to adapt to environmental disturbances. An experiment showing that incorporating closed-loop feedback better adapts to online disturbances would be much more compelling. \n\nTraining the model requires generating a dataset of plans by solving the equilibrium model, which is an expensive process. The quality of this original dataset also depends strongly on the quality of the original model. What if the initial self refinement fails to converge/improve and fails to produce an informative dataset? It would be informative to ablate the performance with different base models of different quality.", "theoretical_claims": "The paper does not present formal proofs for theoretical claims. The core theoretical concept is the formulation of plan refinement as a fixed-point problem. While this is a reasonable approach, there is no analysis of the convergence of the fixed-point problem.", "experimental_designs_or_analyses": "N/A, sufficiently covered above", "relation_to_broader_scientific_literature": "The paper relates to the broader scientific literature on LLM-based robotic planning and deep equilibrium models.. The use of deep equilibrium models for self-refinement is a novel contribution.", "essential_references_not_discussed": "Guiding long-horizon task and motion planning with vision language models, Z Yang, C Garrett, D Fox, T Lozano-Pérez, LP Kaelbling\n\nPredicate Invention from Pixels via Pretrained Vision-Language Models. Ashay Athalye, Nishanth Kumar, Tom Silver, Yichao Liang, Tomás Lozano-Pérez, Leslie Pack Kaelbling\n\nGeneralized Planning in PDDL Domains with Pretrained Large Language Models. Tom Silver, Soham Dan, Kavitha Srinivas, Joshua B. Tenenbaum, Leslie Pack Kaelbling, Michael Katz", "other_strengths_and_weaknesses": "The clarity of the paper could be improved:\n\n- A brief description of the task planning problem in section 3 would be useful for setting the paper up. \n\n- Figure 3 is not very clear to me at all. It contains all proposed components, i.e. the memory, world model, and planner. However it seems there should be a distinction between what is occurring at inference-time and what is occurring at training time. For example, from my understanding the memory is not used during inference-time but is used during training. In addition, what is the interaction between the world model and the environment during training in Figure 3? The meaning of the arrows to and from the world model / environment is not clear.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. Why does using a combination of feedback from the world model and environment result in higher performance than the environment only?\n2. The results show that the proposed method scores lower on executability than baselines. The reason given is because of illegal overlength inputs. Why is the proposed method more susceptible to this problem than the baselines?\n3. Did the authors consider iteratively training and re-collecting a new dataset using the new refinement model? I am curious about the potential performance ceiling.\n4. Did the authors come across any situations where the refinement method fails to converge?\n5. How does the quality of the base model affect the dataset generation, and what happens if the initial self-refinement fails to converge?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "inqWKH0Mkv", "reviewer": "Reviewer_Rz3G", "summary": "In this paper, a closed-loop long-horizon robot planning method based on equilibrium sequence modeling is proposed. By treating the planning as a self-optimizing fixed-point solution process, the implicit gradient of the deep equilibrium model is leveraged for end-to-end supervised training without the need for additional validators or reinforcement learning. From an analytical point of view, theoretical tools such as the implicit function theorem are used to optimize the training process and avoid complex backpropagation. In addition, the nested equilibrium mechanism is designed to dynamically allocate computing resources to optimize planning efficiency by combining environmental feedback and world model.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful and constructive feedback. Below are our responses to the raised concerns.\n\n[W1] Generalization to environment without feedback\n* There are two workarounds in the absense of environmental feedback: (1) Use zero-shot LLM feedback during training. Since LLMs have been shown effective as zero-shot reward models in reinforcement learning [1], they could similarly provide guidance in our training setup. (2) Pre-train in environments with feedback. We validate this by evaluating pre-trained VirtualHome planners on the new ALFRED environment, where our planner generalizes well to a completely new environment (see reply to Reviewer Bgnq's W4 for details).\n  |VirtualHome -> ALFRED|Task classification acc.|Action-object recall|\n  |-|-|-|\n  |SFT Llama|11%|0.50%|\n  |Ours|**54%**|**27.08%**|\n\n[W2] Robustness to noisy feedback\n* To evaluate our model under noisy feedback, we randomly replace some environmental feedback with incorrect feedback during inference. As shown in the results below, our model exhibits stable performance under small amounts of noise ($\\le$10%), demonstrating its robustness.\n  |Noise ratio|Both|novel||Novel|scene||Novel|task|\n  |-|-|-|-|-|-|-|-|-|\n  ||SR|GCR||SR|GCR||SR|GCR|\n  |0%|51.61|**75.13**||**75.79**|85.79||**56.62**|**75.53**|\n  |10%|**53.23**|73.43||74.74|**85.84**||53.85|73.09|\n  |20%|50.00|73.10||73.68|83.22||54.49|71.80|\n\n[W3] Hallucination of planner and world model\n* Hallucination is inherent to LLMs and affects both our planner and world model. However, a key strength of our planner is its ability to self-correct based on feedback. As shown in Figures 10-14 in the appendix, the planner may start with an incorrect plan but improves it using feedback from the environment or the world model, leading to significantly better performance than standard LLM planners (see Tables 1 and 2).\n* For the hallucination of the world model, we show in the reply to W2 that our method is robust to noisy feedback. Thus, even if the world model occasionally hallucinates, the feedback it provides still helps to improve performance (see Table 3).\n\n[W4] Lack of visual information\n* We acknowledge this limitation in Appendix D. While the current method is implemented on LLM planners focused on text input, it could be adapted to video-language planners [2] with visual capabilities. We will continue to investigate this line of planning in future work.\n\n[W5] Impact of Jacobian-free approximation\n* Theoretically, due to the complexity of transformers, the Jacobian-free approximation cannot guarantee convergence to the global optimal solution, which may limit performance. Nevertheless, such approximation is a necessary tradeoff for the computational feasibility of large language models, as demonstrated in [3].\n* Empirically, we confirm that the model can sometimes get stuck in local optimal equilibrium solutions that lead to task failure (see the respone to Q1 below). While there might be several reaons, we suspect that this is partly due to the limitations of the Jacobian-free approximation. Improving this approximation could lead to better performance.\n\n[Q1] Global optimality of equilibrium solution\n* To assess the optimality of equilibrium solutions, we evaluate 10 seeds on each of 60 random tasks. The results show that: (1) our planner can converge to 2-4 different equilibrium solutions for the same task, and the solutions are highly consistent in that they either all succeed (SR→100) or all fail (SR→0), (2) it doesn't always converge to global optimal solutions, as there are still failure cases. This could be due to the limitation of Jacobian-free approximation discussed above.\n  |Task subset|#Equilibrium solutions|Average SR|\n  |-|--|-|\n  |I|2.080|97.33|\n  |II|3.885|5.91|\n* In practice, we find that the equilibrium solving process always converges (see Table 5 in the appendix). This is largely due to the tendency of LLMs to repeat themselves under greedy sampling, which facilitates convergence.\n\n[Q2] Use of environmental and world model feedback\n* Our planner alternates between using environmental feedback and world model feedback at each outer-loop iteration, as illustrated in Figure 12c in the appendix. This allows for better performance within limited environmental interactions. We note that in practical applications, their frequencies could be adjusted according to the accuracy of the world model and the interaction budget.\n* Despite the fact that the world model feedback may be skewed, it still provides useful information for self-refinement, resulting in improved performance (see Table 3) and convergence speed (see reply to Reviewer dmhf's W1). This can be explained by the response to W2, where our method is shown to be robust to noisy feedback.\n\n---\n\n[1] Ma, et al. Eureka: Human-level reward design via coding large language models. ICLR 2024.\n\n[2] Du, et al. Video language planning. ICLR 2024.\n\n[3] Choe, et al. Making scalable meta learning practical. NeurIPS 2023.", "claims_and_evidence": "Yes. The claims made in the submission are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are well-suited for the problem of closed-loop long-horizon robotic planning. The equilibrium sequence modeling approach, nested equilibrium solving, and world model integration address key challenges in this domain. The use of the Virtual Home-Env benchmark and relevant evaluation metrics provides a robust framework for assessing the method's performance.", "theoretical_claims": "Yes, the theoretical claims and proofs in the paper are correct.", "experimental_designs_or_analyses": "The experimental designs and analyses in the paper are generally sound and well-structured.", "relation_to_broader_scientific_literature": "The key contributions of the paper are much related to the field of robotic planning.", "essential_references_not_discussed": "The paper has provided a comprehensive review of relevant literature.", "other_strengths_and_weaknesses": "Strengths: \n1. The paper presents a novel approach by combining the concepts of deep equilibrium models with iterative refinement for robotic planning. \n2. The application of the proposed method to the Virtual Home-Env benchmark demonstrates its potential for real-world robotic planning tasks. \n\nWeaknesses:\n1. In the training stage, it needs to rely on environmental feedback and real data, which may limit its generalization ability in scenarios where there is a lack of sufficient environmental interaction data. For example, in some real-world robotics tasks, obtaining large amounts of high-quality environmental feedback can be very difficult or costly.\n2. In practice, environmental feedback may be disturbed by noise or inaccurate information, does this affect the self-refinement ability and final performance of the model?\n3. The article mentions that models can have hallucinatory problems in some cases, such as the equilibrium planner and the world model that may generate plans or feedback that do not correspond to reality. Does this hallucinatory problem lead to planning failures or execution errors, especially in complex robotic tasks?\n4. The model currently only supports text input and lacks the ability to process visual information, which limits its applicability in real-world robot tasks. In many real-world scenarios, robots need to process both visual and textual information for accurate planning and decision-making.\n5. Although the paper theoretically proposes an optimization method based on equilibrium sequence modeling, in the actual implementation, the authors use approximation methods (such as ignoring the inverse Jacobian matrix) to simplify the training process. The authors should explain whether this approximation will cause the model to fail to achieve theoretically optimal performance in some complex tasks.", "other_comments_or_suggestions": "No.", "questions_for_authors": "1. In the process of iterative refinement, how to ensure that the final equilibrium point is the global optimal solution, not the local optimal solution? In theory, the equilibrium point (fixed point) satisfies the condition that the Lipschitz coefficient is less than 1, but how to deal with this problem in practical application to ensure that the sequence reaches convergence?\n\n2. How are the environmental feedback and the world model feedback in the article coordinated and selected in practical applications? In some cases, the feedback from the world model may be skewed from the real-world feedback, will this affect the model's ability to self-refine?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YyOWcphvP5", "reviewer": "Reviewer_Bgnq", "summary": "This paper introduces a novel approach for long-horizon robotic task planning using LLMs with Equilibrium Sequence Modeling. The core idea is to view planning as a self-refinement process converging to a fixed-point (equilibrium) plan that integrates feedback from the environment or a world model. Rather than relying on reinforcement learning or complex prompting strategies, the paper presents an end-to-end supervised training framework grounded in deep equilibrium models. Additionally, it introduces a nested equilibrium modeling loop to efficiently incorporate closed-loop feedback. The proposed method is evaluated on the VirtualHome-Env benchmark and demonstrates state-of-the-art performance compared to Tree-Planner and SELF-REFINE approaches, particularly in success rate (SR) and goal condition recall (GCR).", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer for providing valuable feedback. Below are our responses to the raised concerns.\n\n[W1] Existence and uniqueness of equilibrium points\n* We empirically confirm that equilibria always exist across different initializations and tasks (see Table 5 in the appendix). The intuition behind this is that LLMs tend to repeat themselves under greedy sampling, leading to easier convergence.\n* For the uniqueness of the equilibria, we run 10 seeds per task over 60 random tasks. As shown in the table below, our planner typically converges to 2-4 different equilibrium solutions for the same task. However, these solutions are highly consistent in that they either all succeed (SR$\\rightarrow$100) or all fail (SR$\\rightarrow$0), further confirming the stability of convergence.\n  |Task subset|#Equilibrium solutions|Average SR|\n  |-|--|-|\n  |I|2.080|97.33|\n  |II|3.885|5.91|\n\n[W2] Intuition behind learning to refine\n* Training the model to map the equilibrium plan $x^*$ to the ground truth $y$ is essentially mapping a suboptimal solution to a better solution, which involves self-refinement. Meanwhile, $x^*$ provides a good prior so that the training objective does not collapse to direct regression of the ground truth (as in standard supervised learning).\n* The generalization of self-refinement is facilitated by our training objective, which avoids directly regressing to the ground truth and therefore overfits less to the training data. As further evidence, in the response to W4 below, we validate the generalizability of our method by evaluating it on ALFRED without retraining.\n* The consistency during self-refinement (as shown in Figures 10-14) comes implicitly from the LLM's instruction-following capability and its tendency to repeat itself. We note that it is possible to incorporate explicit rules to generate structured output from LLMs, which may provide further performance improvements.\n\n[W3] Robustness to noisy feedback\n* Please see the response to Reviewer Rz3G's W2.\n\n[W4] Generalization to more benchmarks\n* Following your suggestion, we test pre-trained VirtualHome planners on ALFRED with updated prompts. For quick evaluation, we consider two planning metrics: task classification accuracy (across 7 task types) and recall of ground-truth action-object pairs in the predicted plan. As shown in the table below, our planner generalizes significantly better than the supervised trained planner.\n  |VirtualHome -> ALFRED|Task classification acc.|Action-object recall|\n  |-|-|-|\n  |SFT Llama|11%|0.50%|\n  |Ours|**54%**|**27.08%**|\n\n[W5] Missing references\n* Thank you for pointing out the recent papers on long-horizon planning. We will revise the draft to include their discussion.\n\n[W6] Distinction between \"plan convergence\" and \"task success\"\n* \"Plan convergence\" refers to the convergence of planner output, but this does not necessarily indicate \"task success\", which requires successful execution of the plan. See Figures 13a, 13b and 14a for failures after plan convergence.\n\n[W7] Visual explanations of baseline failure\n* Thank you for your suggestion. We provide more failure cases of the baselines in Figures 10 and 11 in the appendix, where they fail to correct a long plan efficiently. We will add more visualization explanations in the revision.\n\n[Q1] Sensitivity to initialization\n* To validate the performance stability w.r.t. initialization, we evaluate over 10 initial seeds for each of 60 random tasks. As shown in the results below, our method has a low standard deviation in SR and GCR and outperforms the strongest baseline Tree-Planner, confirming its stability. For convergence sensitivity w.r.t. initialization, see Table 5 in the appendix.\n  |Method|SR|GCR|\n  |-|-|-|\n  |Tree-Planner (N=50)|38.33|56.95|\n  |Ours|**43.50** $\\pm$ 5.21|**68.88** $\\pm$ 5.94|\n\n[Q2] Curriculum training\n* Following your suggestion, we perform curriculum training on tasks of increasing difficulty. Specifically, the model is trained for six iterations: the first four use subsets of task lengths below 5, 10, 15, and 20, and the last two use the full dataset. As shown in the table below, despite the reduced total data size (by 36%), the model achieves higher SR and comparable GCR in Both novel scenarios. This demonstrates the improved data efficiency of curriculum training.\n  |Curriculum training|Data size|Both|novel||Novel|scene||Novel|task|\n  |-|-|-|-|-|-|-|-|-|-|\n  |||SR|GCR||SR|GCR||SR|GCR|\n  |x|11469|51.61|**75.13**||**75.79**|**85.79**||**56.62**|**75.53**|\n  |$\\checkmark$|7411|**54.83**|73.73||69.47|82.93||52.99|71.31|\n\n[Q3] Generalization without retraining\n* Please see the response to W4.\n\n[Q4] Limitation of Jacobian-free approximation\n* Please see the reponse to Reviewer Rz3G's W5.\n\n[Q5] Practicality on real-world robots\n* Please see the response to Reviewer nTaB's W1.", "claims_and_evidence": "The central claims are:\n* Self-refinement can be formulated as an equilibrium sequence modeling problem.\n* Such formulation enables efficient supervised training without reinforcement learning or verifiers.\n* Nested equilibrium modeling improves closed-loop planning using minimal feedback.\n* The approach scales better w.r.t inference-time computation and long-horizon task complexity.\n\nThe evidence provided includes:\n* Theoretical derivation using implicit function theorem.\n* Detailed experimental benchmarks with comparative tables (Tables 1-3).\n* Clear ablations on feedback types, reuse strategies, and scalability (Figures 5 and 6).\n\nThe experiments are clear and the proposed method shows improvement over other approaches.", "methods_and_evaluation_criteria": "I actively follow the literature related to robotic task planning and LLM-based reasoning, though I am less familiar with the technical details of deep equilibrium models, which this paper builds upon. From this perspective, the author's core contribution—formulating plan refinement as a fixed-point problem and leveraging DEQ-style training—appears conceptually elegant and well-motivated. However, several aspects of the method raise important questions and concerns when assessed from a robotic planning standpoint.\n\nFirst, the overall structure of the proposed Equilibrium Sequence Modeling, where planning is treated as a self-refinement process that converges to an equilibrium, fits naturally with how one might think about iterative plan correction in robotics. The idea of conditioning future plan iterations on environmental feedback (closed-loop refinement) is also quite compelling. However, the paper does not offer much intuition about the theoretical behavior of these equilibria—whether they always exist, whether they are unique, and what properties ensure convergence in practical LLM-based settings. These aspects are critical to establishing confidence in the method, especially in robotics, where convergence and stability can directly affect task execution.\n\nSecond, while I appreciate the effort to avoid complex reinforcement learning pipelines or externally curated verifiers, I find the training procedure somewhat underexplained from a robotics planning perspective. The authors argue that supervised learning suffices by using the final equilibrium plan $x^*$ as the input and the ground truth plan $y$ as the target, but it is not entirely clear how this helps the model learn to 'refine' in the planning sense, rather than simply mapping one plan to another. In practical terms, what guarantees that the system learns generalizable plan improvement heuristics, rather than overfitting to a specific dataset structure (e.g., VirtualHome-Env)? Also, the method seems to lack explicit regularization mechanisms or consistency constraints across refinement iterations.\n\nFinally, from a planning systems perspective, I would have appreciated more clarity on how well this equilibrium-based method performs when feedback is noisy, delayed, or partially incorrect (scenarios that are quite common in real-world robotic environments). While the use of a world model to estimate internal feedback is a reasonable workaround, the model's robustness to imperfect feedback (or feedback that disagrees with reality) remains unclear.\n\nOverall I like this new perspective in the LLM-based planning space but would appreciate more clarification on these questions.", "theoretical_claims": "Yes, theoretical claims regarding differentiability through the fixed-point (via the implicit function theorem) are correct and consistent with standard literature (Bai et al., 2019; Krantz & Parks, 2002). \n\nI am not familiar with the Jacobian-free approximation (Fung et al., 2022) though and would urge the Area Chair to rely on the reviews of other reviewers on the correctness of the claim (Line 73).", "experimental_designs_or_analyses": "The experiments are well-designed. The authors isolate model contributions via ablation studies (feedback, compute). \n\nHowever, the evaluation is constrained to a single benchmark (VirtualHome-Env), and further diversity in tasks/environments would have improved generalizability assessment.", "relation_to_broader_scientific_literature": "This paper contributes meaningfully at the intersection of LLM-based planning and implicit model training. It extends prior self-refinement work (Welleck et al., 2023; Madaan et al., 2023) by providing a principled optimization framework rather than heuristic prompting. It also offers a scalable alternative to methods like Tree-of-Thoughts (Yao et al., 2023) and SELF-REFINE.", "essential_references_not_discussed": "Some of the recent papers that use LLMs for long-horizon planning in both centralized and decentralized fashion, that allow re-planning (and self-verification to some extent):\n\n1. Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B. Tenen- baum, Tianmin Shu, and Chuang Gan. Building cooperative embodied agents modularly with large language models, 2023.\n2. Nayak, S., Morrison Orozco, A., Have, M., Zhang, J., Thirumalai, V., Chen, D., ... & Balakrishnan, H. (2024). Long-horizon planning for multi-agent robots in partially observable environments. Advances in Neural Information Processing Systems, 37, 67929-67967. \n3. Jun Wang, Guocheng He, and Yiannis Kantaros. Safe Task Planning for Language- Instructed Multi-Robot Systems using Conformal Prediction. arXiv preprint arXiv:2402.15368, 2024.", "other_strengths_and_weaknesses": "Strengths:\n1. The paper is conceptually elegant: combining DEQ with planning.\n2. Removes reliance on RL while maintaining flexibility and robustness.\n3. Strong empirical evidence and reproducibility through open code (did not verify the reproducibility of the code by running it).\n\nWeaknesses:\nrefer to methods and evaluation section", "other_comments_or_suggestions": "* Suggest clarifying the distinction between \"plan convergence\" and \"task success\"—sometimes conflated in figures.\n* Consider adding visual explanations of failure cases across baselines (Figure 4 could be expanded).\n\n## Update after rebuttal\nI wasn't sure if the \"official comments\" were visible to the authors and hence am including them here:\n\nI appreciate the authors' rebuttal and addressing my concerns in the review. I am assuming that the authors will include some of these changes in the camera ready version (if accepted). I would like to keep my scores the same.", "questions_for_authors": "1. How sensitive is performance to the choice of initialization for fixed-point iteration? Would a poor initialization degrade convergence/stability?\n2. Have you considered curriculum training—starting from short tasks and scaling up plan length? Would it improve convergence and data efficiency?\n3. Can the model generalize to tasks beyond the VirtualHome-Env benchmark without retraining? For instance, can you reuse the same equilibrium modeling process in the ALFRED or RoboTHOR environment (were the prompts tailored heavily for the VirtualHome environment)?\n4. Out of curiosity, what are the limitations of Jacobian-free approximation in your experience? Have you quantified the impact of this approximation empirically?\n5. Do you anticipate your approach being practical on real-world robot hardware where interaction latencies and imperfect perception are constraints?\nRefer to more broader questions in the Methods and Evaluation criteria.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a novel framework, Equilibrium Sequence Modeling, applying deep equilibrium models to LLM-based robotic task planning for iterative self-refinement. The reviews are generally positive (ranging from weak accept to accept), recognizing the conceptual novelty and the empirical evidence demonstrating improved performance over baselines on the VirtualHome-Env benchmark. 

Reviewers highlighted the strengths of the principled approach to supervised training for refinement without RL and the effective integration of feedback. The authors provided a comprehensive rebuttal. They included additional experiments addressing robustness, generalization to a new environment (ALFRED), convergence analysis, and the benefits of curriculum training, along with clarifications on the methodology and results. These responses appear to have addressed the major points raised.

Given the novel contribution, the supportive experimental results, and the thorough author response addressing reviewer concerns, the paper is recommended for acceptance.
Decision: Accept (poster)