Continual Robot Learning via Language-Guided Skill Acquisition
Authors: Shuo Cheng
Zhaoyi Li
Kelin Yu
Danfei Xu
Venue: Accepted by TMLR ()

Abstract: To support daily human tasks, robots need to tackle complex, long-horizon tasks and continuously acquire new skills to handle new problems. Deep Reinforcement Learning (DRL) offers potential for learning fine-grained skills but relies heavily on human-defined rewards and faces challenges with long-horizon goals. Task and Motion Planning (TAMP) are adept at handling long-horizon tasks but often need tailored domain-specific skills, resulting in practical limitations and inefficiencies. To overcome these complementary limitations, we propose LG-SAIL (Language Models Guided Sequential, Adaptive, and Incremental Skill Learning), a framework that leverages Large Language Models (LLMs) to synergistically integrate TAMP and DRL for continuous skill learning in long-horizon tasks. Our framework achieves automatic task decomposition, operator creation, and dense reward generation for efficiently acquiring the desired skills. To facilitate new skill learning, our framework maintains a symbolic skill library and utilizes the existing model from semantic-related skills to warm start the training. LG-SAIL demonstrates superior performance compared to baselines across six challenging simulated task domains across two benchmarks. Furthermore, we demonstrate the ability to reuse learned skills to expedite learning in new task domains, and deploy the system on a physical robot platform. More results on website: https://sites.google.com/view/continuallearning.

Published in Transactions on Machine Learning Research (03/2026)
Continual Robot Learning via Lunduege-Guided Skill Acquisition
Shuo Cheng
School of Interactive Computing Georgia Institute of Technology
Zhaoyi Li
School Interactive Computing Georgia Institute of Technology
Kelin Yu
Department of Computer Science University of Moryland, College Park
Danfei Xu
School of Interactive Computing Georgia Institute of Technology
Reviewed on OpenReview: https: // openreview. net/ forum? id= o YYNNu
Abstract
To support daily human tasks, robots need to tackle complex, long-horizon tasks and con- linuously acquire new skills to handle new problems. Deep Reinforcement Learning (DRL) offers potential for learning fine-grad finegrained skills but relies heavily on human-defined rewards and faces challenges with long-horizon goals. Task and Motion Planning (TAMP) are adopt at handling long-horizon lasks but often need tailored domain-specific skills, resulting in practical limitations and inefficiencies. To overcome these complementary limitations, we propose LG-SAIL (Language Models Guided Sequential, Adaptive, and Incremental Skill Learning), framework that leverages Large Language Models (LLMs) to synergistically integrate TAMP and DRL for continuous skill learning in long-horizon tasks. Our frame- work achieves automatic task decomposition, operator creation, and dense reward generation for efficiently acquiring the desired skills. To facilitate new skill learning, our framework maintains a symbolic skill library and utilizes the existing model from semantic-related skills to warm start the training. LG-SAIL demonstrates superior performance compared to baselines across six challenging simulated task domains across two benchmarks. Further- more, demonstrate the ability to reuse learned skills to expedite learning in new task domains, and deploy the system on a physical robot platform. More results on website: http:://stttsggogggg....oo////es http:///...oooo////ooo//aaaaaaaa
1 Introduction
For robots to aid daily human lasks, they need to tackle intricate long-term challenges and adapt unfamiliar situations. While Deep Reinforcement Learning (DRL) techniques are promising for acquiring fino-grained manipulation skills (Levine et al. 2016 Gu et al., 2017), they require carefully designed reward functions and often fail to achieve long-terın goals in complex enviromments. Conversely, Task and Motion Planning (TAMP) methods (Garrett et al. 2021 are adept at addressing and adapting to long-term tasks due to their robust state and action abstracts. However, their dependence on expertise to construct the planning domain restricts their practical use in real-world scenarios.
FEqually Contributed

---

Published in Transactions on Machine Learning Research (03/2026)
In effort to overcome the constraints of both TAMP and skill learning. researchers have proposed the concept of learning skill policies within TAMP systems (Cheng & Xu, 2023: McDonald & Hadfield- Menell, 2022 Mandlekar et al. 2023b). While these approaches have shown promise in enabling skill acquisition for long-horizon tasks efficiently, they still rely on expert knowledge to define planning domains (c.g., symbolic operators with conditions and effects) and dense reward functions. These assumptions limit the scalability of skill learning, especially in real-world contexts where robots frequently confront novel challenges where the planning domains cannot be defined beforehand.
In this work, we introduce LG-SAIL (Fig. D continuous learning framework for robot manipulation that integrates large language models (LLMs) with TAMP and DRL. We observe that LLMs, trained on vast web data, excel at commonsense reasoning for task decomposition and skill creation, while TAMP's compact predicate- based state representations provide strong priors that guide and regularize LLM reasoning, improving efficiency and robustness. Leveraging TAMP's abstract states, we structure task decomposition and skill generation to reduce LLM hallucinations. We also define metric functions from TAMP predicates to enable LLMs to generate dense rewards for fast policy learning. To accelerate skill acquisition, trained policies are stored in skill library, and relevant skills are retrieved via semantic similarity of LLM encoder features to warm-start training. This creates virtuous cycle between planning and skill learning, supporting efficient continual robot learning.
To summarize, our key contributions include: 1) using LLMs for continual skill learning via automatic task decomposition, skill creation, and dense reward generation; 2) leveraging structured state represent ations to regularize LLM content generation; 3) improving skill acquisition by maintaining and reusing a skill library; 4) demonstrating through extensive evaluation six challenging simulated tasks from two benchmarks that our framework outperforms prior methods and supports real-world deployment, paving the way for lifelong robot learning in complex environments.
2
Related Work
2.1 LLMs for Reward Design
Designing effective reward functions has long been a challenge in reinforcement learning Sutton 2018
Recent studies have investigated using LLMs to generate reward functions (Yu et al., 2023 Xie et al. 2023
Zeng et al.] 2024 Ma et al.] 2023 2024). While these methods have demonstrated potential in learning complex, long-horizon lasks remains uncertain. Free-form LLM- based reward generation is prone to hallucinations, often producing syntax errors and irrelevant content. Moreover, approaches such Eureka (Ma et al. 2023) are time-intensive, requiring multiple rounds of policy training and evaluation to refine the reward functions for each individual skill, making them impractical for long-horizon task learning. Zeng et al. (Zeng et. al. 2024) propose a method that iteratively refines reward functions based on feedback from the task learning process. However, this trial-and-error approach can result in delayed convergence and fluctuating performance, hindering consistent improvements. We address these limitations by parameterizing object relationships with metric functions, using LLMs to compose them for efficient, automated dense reward generation.
short-horizon skills, their scalability
2.2
LLMs for Planning and Decision Making
In recent years, LLMs have been widely explored for robot planning (Huang et al. 2022b a Rana et al. 2023
Ahn et al. 2022a Liu et al.) 2023 Joublin et al., 2024 and feeision-making (Singh et al.) 2023 Liang al. 2022 Li et al. 2022 These studies demonstrate LLMs' ability to understand complex commands, create task plans, and translate natural language into executable actions. For instance, SayCan (Ahn et al. 2022a) uses LLMs to generate lask plans, grounding each skill with learned control policies, while ProgPrompl (Singh et al. 2023) prompts LLMs to generate executable programs that invoke function calls for robot tasks. Rescarchers have also explored using LLMs to solve PDDL planning problems (Silver et al. (024)), showing that LLMs can act as generalized planners by generating efficient programs for tasks within a domain. This finding informs our approach. Unlike prior methods that generate skill sequences or programs, we use LLMs'
2

---

Published in Transactions on Machine Learning Research (03/2026)
semantic knowledge to construct planning domains and guide skill learning in TAMP, enabling more efficient and robust handling of long-horizon tasks.
2.3 TAMP and Learning for TAMP
Task and Motion Planning (TAMP) (Kaellinn & Lozano-Péree 2013 Garrett et al. 2021 offers a robust framework for tackling long-horizon tasks in structured environment settings by decomposing complex plan- ning problems into sequence of symbolie-contmmuous subtasks, which simplifies the optimization for finding solutions. However, traditional TAMP systems require manually defined planning domains, which depend on extensive expert knowledge. To reduce this burden, recent works (Konidaris et al.) 2018 Silver et al. 2022: McDonald & Seadfield-Mennll 2022 Zhao et al. 2024) have integrated learning into TAMP, including efforts to learn low-level skill policies. Mandlekar et al. (Mandlekar et al. 2023b) proposed a TAMP-gated control mechanism that selectively transfers control between a human teleoperator and the robot, using the collected data to train skill policies and improve the TAMP system. LEAGUE (Cheng & Xu 2023 takes different approach by leveraging the symbolic interface of lask planners to guide RL-based skill learning, creating abstract state spaces that enable skill reuse and improve scalability for long-horize tasks. Despite these advances, task decomposition still relies on hand-crafted symbolic planners. Besides using symbolic planners for skill planning, researchers have also explored learned planners or heuristics to compose skills Nasiriany et al. (2022b); Zheng et al. (2025); Mishra et al. (2023), which typically rely on task-specific training data, collected either via fiil-and-erroo interaction or expert demonstrations, In this work, we utilize LLMs to integrate TAMP planning with skill learning, cnabling robots to tackle long-horizon tasks more efficiently with minimal human input.
3 Background and Problem Setting
Planning domain and skill representations. We focus on deterministic, fully observed tasks that can be described in PDDL Fox & Long) 2003), with object-centric states, continuous actions, and known transition function. Formally, an environment can be characterized by tuple (O, A, V. 9) Each object entity o E O within the environment (e.g., peg1), possesses a specific type À e A (such as peg) and a tuple of dim(A) (A)-dimmnnsio features containing object state information such as pose and size. The environment state x E X' is a mapping from object entities to features: x(o) E [idin(tspa(o)), Predicates v describe the relationships among objects in the environment. Each predicate &a (c.g., Holding (rodjnn((obecct:pee) characterized by tuple of object types (A1,-..:Ami Xm) and binary classifier that determines whether the relationship holds: Ge X' x Om (True, False), where each substitute entity o, O is restricted to have type X E A. Each binary classifier is constructed through evaluating set of metric functions Fp. = {fi) related to this predicate: cv A ANEr, [[e S <t), where each metric function fi (e-g., xy_ xy_dis(.,)) outputs a real number to quantify the relationships among the query objects (e.g., the distance between two object centers projected onto the xy-plane). Evaluating a predicate on the state by substituting corresponding object entities will result in a ground atom (e.g., Holding (peg1)). A task goal g E G is represented conjunction over a set of ground atoms (c.g., (n holel) A In (ppe2,,hole))) where a symbolic state zo can be obtained by evaluating a set of predicates V and dropping all negative ground atoms.
is
Each lifted symbolic operator W E Ô is defined by a tuple (Par, Pre, Eff Efff,, where Pre denotes the precondition of the operator, Efft and Eff are lifted atoms that describe the expected effects (changes in conditions) upon successful skill execution. Par is an ordered parameter list that defines all object types used in Pre, Efft, and Eff An example of Pick operator can be defined as: A ground skill operator z substitutes lifted atoms with object instances: W (w, & (Pre, Efff-Eff*i where 5 A + O. A symbolic task plan is a sequence of ground operators that, when executed successfully, leads to an environment
Pick Pick (Pobject)
PAR: [?object:peg]
PRE: (HandEupty O. OnTablalleeee (Fobjectt))))))
EFF-: (HandEnpty O OaTablle (?object)}
EFF (Holding(Fobjecss)
state that satisfies the task goal.
MDP. Learning the grounded low-level skills π for any symbolic operators w can be formulated as a Markov Decision Process (MDP) denoted by (X Rxx a), (xx (x, a), ((x())) 7), with continuous state space X.
3

---

Published in Transactions on Machine Learning Research (03/2026)
continuous action space A, reward function R, environment transition model T, distribution of initial states p(x!")), discount factor 7. The objective for RI. training is to maximize the expected total reward J of the policy xaaz) that the agent employs to interact with the environment:
Σ (R(aae))
(0) x00) (mm(((
(1)
Problem setting. Our setting inspired by prior work on operator invention for bi-level planning (Silver et al. 2023 Chitnis et al. 2022). However, we assume only small set of predefined predicates and metric functions. Given task goal 9. automate operator discovery for decomposing minghorizon tasks and learn primitive skill policies to achieve the resulting subgoals. Each lifted operator ß is associated with single skill policy shared across all its groundings during execution.
4
Method
Skill Creation Planning
Slill Execution
State Descriptions
Init: Onfpegl.table) On(peg2,table) A HandEmpty()
Pick [peg2]
Insert (peg), holes)
(PAlpeg2), EFP": (P2(peg2)....
(P1(peg1)... ...)
Goal: Infpeg1.hole1] A Infpeg2.hole2)
Info
Feedback curriculum
Instantlate
Reward Generation
Skill Learning
Skill Library
Extend
Predicates Functions
Pick reward:
On
Holding
In
dis: 0
Warm-start
oo) 0.7,
Pick
Place
Insert
dis
to ob
...
LLMs
Figure 1: Framework Overview. Our framework automates task decomposition, skill creation, and dense reward generation by creating virtuous cycle between planning and skill learning. Execution failures reveal skills needing improvement, while a skill library enables warm-starting new skills using semantic similarity.
We seek to achieve automatic skill abstraction and continual learning in long-horizon tasks by leveraging the strong priors in LLMs. Our primary contribution is integrated pipeline (Fig. [ that uses LLMs for task decomposition and planning (Sec. 4.1). and constructs dense rewards for skill learning (Sec. 4.2
We
describe how to accelerate learning of new skills in lifelong learning scenarios in Sec. 4.3
4.1 Task Decomposition and Skill Creation
To roduce the domain knowledge needed for designing valid skills in TAMP, we proposc leveraging the rich semanttic knowledge of LLMs decompose tasks into reusable, elementary skills. We use GPT-4 (Achiam) et al. 2023 in our experiments. To address the hallucination issue-where generated task plans may overlook constraints between adjacent skills and "hallucinate" impossible effects we incorporate structural information from TAMP system, including available predicates and the task goal description, as prompts to regularize the outputs. An example of operator generation is shown below:
Task: Generate preconditione and effects the "Pick" action by selecting from all available predicates. Ensure the output follows the format [Predicatel, Predicate2, .....
Inportant: Each predicate is preceded by crucial comments in the code these comments must be retained and considered carefully shile generating preconditions and effects.
Action: "Pick"
All predicates: ("Holding", "OnTable", "In", ...3
code those predicates are shown below:

---

Published in Transactions on Machine Learning Research (03/2026)
Plan verification. Although the availability of predicates simplifies the skill creation process, the gener- ated operators may occasionally contain irrelevant preconditions and effects. To enhance the accuracy of task decomposition, we validate the generated symbolic operators using the A algorithm to produce the symbolic task plan via deterministic search. To plan for a task goal 9. we first evaluate the predicates on the current environment state x, yielding the corresponding symbolic state zg. We then ground each lifted operator ã f2 by substituting object entities the environment in preconditions and effects, leading ground operators st- - (Pre, Eff .Eff*) that support operating with symbolic states. A ground operator is considered executable only when its preconditions satisfied: Pre S zg. The operators induce an abstract transition model ((x w) that allows planning in symbolic space:
z = T(zy, w) (x \ Eff " JUEff"
(2)
Compared to directly using LLMs for task plan generation, incorporating an A planner provides feedback by evaluating the proposed operators and verifying whether they are sufficient to reach the task goal. the search fails, unreachable operators are identified and usod guide L.MM-basod regencration, enabling the LLM to produce correct operators with fewer altempts.
4.2 Reward Generation and Skill Learning
Once the planning domain established, and symbolic skills can be planned for specific task goals, we can proceed to learn the corresponding low-level skill policies via deep reinforcement learning. Dense rewards are crucial for reinforcement learning, but human-crafted dense rewards require considerable effort. It presents challenges for long-horizon lasks with various skills and lifelong learning scenarios where agents are exposed to diverse new tasks. We therefore consider using LLMs for automating reward generation.
Reward construction with metric functions. LLM-generated reward functions often contain imprecise code or syntax errors et al. 2023 Ma et al., 2023 Yu et al. 2023). requiring iterations or human feedback. Our key insight is that predicate-ders structural regularization on the generated content. Each metric function f E F defines quantitative relationship between objects: f X' x O R. For example, dis dis_to_obl (robot, peg1) defines the distance between the robot's gripper and the peg1 object. Instead of generating code from scratch, we simplify the problem by using LLMs with extracted skills definition from structural skill operators to select metric functions and their corresponding weights to construct dense rewards. This strategy improves the success rate of generating usable reward functions by constraining the LLMs solution space, thereby reducing irrelevant or erroncous outputs. Additionally, the semantic information from metric functions aids the LLMs in composing reward functions that better align with task objectives.
metric functions in TAMP encode object relationships, providing
To better harness the LLMs' proficiency in code interpretation and task comprehension, for each skill operator w, provide the LLM with the source code of metric functions along with generated skill operators. In this work we use GPT-4 (Achiam et al. 2023 An examplar prompt for reward generation can be found below:
Task:
Design a reward function for training "Pick" skill using RL based on the template.
Skill: "Pick"
Objects: ["pag", ...] Metric functions:
...]
The template is defined as:
.en
Reward Template:
seevarddtempaate... [[revard score, hetric_function (inputs)) ?], ...]}
Guidelines to complete the revard template:
Select Metric Functions: Choose functions that help in learning skill (code provided belou).
Assign Reward Scores: Give each chosen function score (0.0 to 1.0), with higher scores indicating greater inportance.
.*n
The code of metric functions are shown below:
In this example, the metric function dis dis_to_ tostto_of obj with its corresponding code provides semantic information that helps the LLMs understand its purpose is move the gripper closer to object. Subsequently, the LLM
5

---

Published in Transactions on Machine Learning Research (03/2026)
generates dense rewards R² by selecting subset of relevant metric functions Fu- (fi f2......... f,) S F and assigning weights Vi_ = (11, u2,,... .,, en) to their normalized forms f reflecting their relative importance to the reward. The dense reward is thus defined as:
R_d (2
Σ - E. Ui ff()) E [0, Vi.
where O2 defines a group of object entities relevant to operator w
To ensure the learned policy achieves the desired outcomes specified by the skill operators, we incorporate sparse rewards derived from predicates. The agent receives maximal reward when all predicates that define the desired effect of the skill operator in are met:
{
if (Acst (x(0)) -Cy C (z!(0)
R (((00))
-
for wt Efft for 3/- E Eff
(4)
otherwise
This complements the dense rewards constructed by the LLM, promoting the development of more robust skills for long-horizon tasks and ensuring the expected outcomes met to seamlessly transition to the next skill. Finally, the skill reward is constructed as: (x((( max(R (zit)), (x(1)))
Skill learning with state abstraction. With the LLM constructed dense rewards, we can train a policy for each symbolic skill using RL. Since the precondition and effect of a ground operator & induce an effective abstraction of the environment, we can define a skill-relevant state space to prevent the learned policy from being influenced by task-irrelevant objects, thus enhancing its learning efficiency and generalization (Cheng (z(()) : : Ow). The training objective for RL is therefore formulated ) f
& Xu
E''Re(o!)) +
(5)
We use Soft Actor-Critic (SAC) (Haarnoja et al. 2018 to optimize the skill policy, where H is the entropy term. Despite the use of dense rewards, RL exploration can be inefficient for learning complex motions (Cheng & Xu 2023 Conversely, while motion planning from TAMP systems struggles with contact-rich manip- ulation, it is well-suited for handling free-space motions (Mandlekar et al. 2023b). Building this, we propose augmenting our policy with motion planner-based transition primitives. The key idea is to first use an off-the-shelf motion planner to approach the skill-relevant object (as specified by the skill operator) before initiating RL-based skill learning. For the motion planning target, we set the goal position 0.04 m above the object or placement position identified by the task planner. This strategy facilitates more efficient exploration while preserving the ability learn closod-loop, contact-rich manipulation skills.
4.3 Continual Skill Acquisition through Integrated Planning and Skill Learning
So far, we have described how to leverage the rich semantic knowledge from LLMs to guide task decomposi- tion, skill operators creation, and reward generation for continual skill learning. In this section, we describe how to learn skills within the context of task planning system. This integrated planning and learning approach ensures that the learned skills are compatible with the planner, while continuously expanding the system's capability to solve more tasks.
Task planning and skill execution. For any given task goal g, use A planner with the generated operators to scarch task plan, as described in Sec. 4.1 With the generated task plan, we sequentially invoke the corresponding skill n" to reach the subgoal that complies with the effects of each skill operator w in the plan. We roll out each skill until fulfills the effects of the operator or a maximum skill horizon И is reached. To verify whether the I-th skill exccuted successfully, we obtain the corresponding symbolic state 2y by parsing the ending environment state x.. Then, the execution is considered successful only when the environment state x conforms to the expected effects: T(x 1-1
1 wi) C any We track failed skills along with
6

---

Published in Transactions on Machine Learning Research (03/2026)
the corresponding initial simulator state s" to inform the learning curriculum, using this information to reset the environment for focused practice on those failed skills.
Automated Curriculum. To efficiently aequire the necessary skills for a multi-step task, we utilize the task planner as an automated curriculum, which enables progressive skill learning. The core idea is to use already proficient skills to achieve the preconditions of those skills which are still required further learning. At high level, we iteratively alternate between task planning and skill learning until convergence, We track skill failures during executions and apply strict scheduling criteria: skill is prioritized for further learning (Sec. 4.2 whenever it fails during rollouts. The algorithm is sketched in Alg, [ Notably, share the replay buffers across different skill instances (e.g., Pick (peg1) and Pick (peg2)) that correspond to the same lifted operator. This allows relevant trajectories to be reused, which enhances both learning efficiency and generalization.
Accelcrate lcarning with skill library. Efficient learning of complex tasks requires the continual adap- lation of acquired skills to new lasks, even across different domains. Our insight is that semantically similar skill operators often share underlying low-level behaviors. For instance, opening refrigerator and opening door may involve similar actions and interactions. As result, policy models from previously learned skills can serve as effective initializations, providing a warm start for learning new skills in novel situations.
Based on this, we propose to maintain a semantic skill library, where each element key represents the semantic embedding of the skill's symbolic definition, which is extracted by a pre-trained LLM encoder (specifically, OpenAI's -eaet--eeeennn---eeeee 3-lagge): 2 ((w) & @(Pre, Eff -Eff*)). offf"fff*)) The value corresponds to the neural network weights of the associated policy.
key-value pair. The
Algorithm SKILLACQUISITION
input: env (task environment), (symbolic goal), V (state predicates), Ω (operators by LLM), R (reward functions by LLM), (skill library), (LLM encoder)
start:
Or)!") (0))) < www.gge_sstttee
( (A)SE() O, T)
continuous state to symbolic state & get grounded operators found plan with length L
Q GROUND(O) (O, !2)
[ + SEARCH(I (0) 9.
while Not Conveeryed do
sev.set..... (t
[T] (4) .... L ( +- RETRIEVE( ....
warm-start initialization failed skills and simulator states
D t ROLLOUT(env, Q, [T(") ..... (DD
D then
break end if
for s, D do
env env.set_eet set_ssate(t))
((()) +- [T))
n (t+1) +- SAC(env m. (1) w R.))
RL training
+- IIU (#(w) m ((+1))
end for
t + t+1 end while return
To leverage the models stored in this library, for any new skill that needs to be acquired, we first extract its feature embedding z'. We then identify the most similar skill in the library using cosine similarity and use the corresponding weights to initialize the new skill, which provides a warm start for continual training. Each newly learned skill is subsequently added the library, expanding it speed up the learning process.
7

---

Published in Transactions on Machine Learning Research (03/2026)
5 Experiment
In this section, show that LG-SAIL autonomously learns long-horizon tasks and outperforms prior state- of-the-art methods. We demonstrate its ability to adapt skills to novel tasks and domains crucial for continual learning and validate our planning and reward designs via ablations. Finally, we deploy the system on a physical robot platform and present both quantitative and qualitative results in the real world.
5.1 Experimental Setup
We conduct experiments in six simulated domains. The domain "StackAtTarges" "StowHammer", "PegInHole", and "MakeCoffee" are from LEAGUE (Cheng & Xu 2023), which involve long-horizon rea- soning and contact-rich manipulation. The task setups, predicates, and our generated operators and task plan are shown in Fig. 2 To further evaluate the framework's 's ability to efficiently adapt learned skills to novel tasks, we include the LIBERO benchmark (Liu et al. 2024b), a benchmark in lifelong robot learning that features diverse objects and long-horizon tasks. We test our framework on both "LIBERO-Spatial" and "LIBERO-Object", The "LIBERO-Spatial" set comprises 10 tasks with varying scene configurations, where the robot must pick up target bowl and place it on the goal plate. The "LIBERO-Object" set includes 10 tasks involving different objects with diverse shapes, which requires the robot to grasp the target object and place it in basket. The setups and our generated operators are shown in Fig.
hammerO
hole1
cubeo
cabinet1
cabinett
cabinet
hole2
holder
lid
targeto
pod
hammerl
peg1
pegz
Predicates
HandEmpty, Holding, In, On, IsEmpty, IsTo, HasFreeSpace, InCabinet, IsCabinetClose, IsCabinetOpen, IsLidClose, IsLidOpen
Metric functions
Is grasped, dis to_ obj, xt_xxy___a dis, dis,, parallel dis, perpendicular dis, norm_ norm_angle angle
Training soel
(/hammer0,cabinettee
On(cube1,cube0) On(eube
(n/hane)r,,cabneet)) A
In(pod, A IsLidClose(lid) A
In(peg1,holet) In(peg2,hoel))
(SCabine(loose(cabieett)) A UCCainetCloseecabiiee
IsCabinetCiose((abbnnet)
Our generated aperators and task plan
Open(cabinet0), Pick(hammero),
Place(hammerO, cabbine)),, Close(cabineto), Open(cabinet1), Pick(hammer1), Place(hammer1,cc cabbiett))) Close(cabinet1)
Open(cabinet), Pick(pod), Insert(pod, holder), CloseLid(lid), Close(cabinet)
Pick(cubeO), saaget), Pick(cube1), Place(cube1,cubeo)
Pick(peg2), Insert(peg2, Pick(peg1), Insert(peg1.holei)
Figure LEAGUE Tasks (Cheng & Xu 2023). Illustration of task setups for "StackAtTarget"" "Stouffammer". "PegInHole",, and "MakeCoffee".
All environments built the MuJoCo engine Todorov al. 2012 We Franka robotic arm, controlled at 20Hz with operational space controller (OSC), providing degrees of freedom: end-effector position, yaw angle, and gripper position.
5.2 Quantitative Evaluation
To assess performance, we compare LG-SAIL against strong baselines for learning long-horizon tasks in "StackAtTarget", "StowHammer"", and "PegInHole":
RL (SAC) We utilize Soft Actor-Critic (SAC) (Haarnoja al. 2018 for vanilla RL baseline;
Curriculum RL (CRL) This baseline follows the same curriculum strategy as prior works (Sharma et al. 2021 Uchendu et al. 2023), starting training from near-success states and gradually shifting the reset. states back to the initial task states. We use the same model as RL (SAC) for fair comparison:

---

Published in Transactions on Machine Learning Research (03/2026)
object0
basket0
bowlo
plate0
Predicates
HandEmpty, Holding, In, On
Metric functions
is grasped, dis to obj, xy vy_dis, 2_dis
Training goal
On(bowi0,plate0) generated operators and plan
In(objectO.baskett)
Pick(bowlO), Place(bowlO,plate0)
Pick(objectO), Place(objectO,basket)
Figure 3: LIBERO (Liu et al. 2024b). Tasks sctup for "LIBERO-Object" and "LIBERO-Spatial""
Hierarchical RL (HRL) This bascline employs HRL designs (Dalal et al. 2021 Nasiriany al. 2022ab, where a high-lovel controller composcs paramoterized skill primitives. Our implementation builds MAPLE (Nasiriany et al. 2022b and uses the oracle task plan to identify larget objects for affordance definition.
LEAGUE (Cheng Xu 2023 This baseline reflects the recent trend of integrating TAMP with skill learning. We utilize it for this experiment due its superior performance on long- aag-horizon tasks.
To ensure fair comparison, we follow experiment settings in LEAGUE (Cheng & Xu 2023 and we adopt task progress evaluation metric, which is defined as the summed reward of all task stages and normalized to [0, 1). The results are presented in Fig.
StackArTarget
PeginHore
Stowl
Durs Fsea::
Ourt
0.8
LEAGUE
0.6
0.6
0.6
0.4
0.4
0.8
1.2
1.4
Emv Step
X30*
Emn Step
X10
Env Step
K10P
Figure 4: Baseline comparison. We compare our framework with other baselines across three task do- mains. The plot illustrates the average task progress during evaluation over the training phase, measured the sum of rewards for cach successfully exccuted skill in the task plan, normalized to 1. The shaded area represents the standard deviation for 5 random seeds.
Our framework efficiently and autonomously learns long-horizon manipulation tasks. By just providing predicates, metric functions, and task goals of different domains (shown in Fig. 2 our framework autonomously achieves task decomposition, skill creation, reward generation, and integrates TAMP with skill learning. Additionally, we found that methods incorporating skill abstraction and planning (i.e., LG-SAIL and LEAGUE (Cheng & Xu 2023)) significantly outperform other baselines, underscoring the importance of explicit skill rouse in multi-step tasks with repeating structures. Notably, LG-SAIL demonstrates slightly higher learning efficiency compared to LEAGUE (Cheng Xu 2023). potentially reflecting the advantages of -LL--geerated rewards over handcrafted ones, as supported by recent literature (Ma et al.) 2023 2024
5.3 Validating Skill Generalization and Adaptation
We design experiments evaluate the system's continual learning capability by constructing tasks that require varying levels of adaptation.

---

Published in Transactions on Machine Learning Research (03/2026)
Generalizing to new goals. In addition to training goals for "PegInHole" and "StowHammer"", directly evaluate our framework on new goals without further learning. For "StowHammer", the first test goal is to swap the hammer-cabinet mapping, while the second goal is to place hammer1 into cabinetO while keeping cabinet1 open. For "PegInHole"; the first test goal is to swap the pog-hole mapping, and the second goal is insert pegl into hole2. The results are presented in Tab.
Table 1: Task goal generalization. We evaluate RL (SAC) and LG-SAIL. novel task goals in the "StowHammerr" and "PegInHole" domains under zero-shot settings.
Domain
Methad
Training Goal
Test Goal
TTst Goal 2
(SAC)
0.15 0.02
0.07 + 0.02
0.09 0.01
"Stoullammer"
LG-SAIL
0.96 + 0.15
0.91 + 0.09
0.71 + 0.28
(SAC)
0.19
0.03
+ 0.01
0.10 0.01
"PegInHole"
LG-SAIL
0.92 1 0.11
0.59 + 0.14
0.93 1 0.06
We observe that LG-SAIL exhibits minimal performance drop when generalizing to new goals within the domain without additional Training, demonstrating strong compositional generalization capabilities.
Adapting to new objects and scenes. We aim to evaluate LG-SAIL's ability to adapt learned skills to novel situations using the LIBERO (Liu et al., 2024b) benchmark. The "LIBERO-Object" set requires the robot to pick up various target objects and place them into basket, testing its ability to adapt to new object shapes and poses. The "LIBERO-Spatial" set features diverse scene layouts where the robot must place a target bowl on plate, emphasizing genoralization to different object poses and scene configurations. We sequentially lest. LG-SAIL on lasks from both "LIBERO-Object" and SAIBERO-Spatial.
with results shown in Fig. 5 Interestingly, while LG-SAIL. takes some time to learn the initial task, as more skills are accumulated, the learning of subsequent tasks accelerates significantly, demonstrating its strong potential for continual and lifelong learning.
Average Tasks Progress for LIBERO Spatial
Task Progress
0.8
0.6
0.4
0.2
0.0
100000
200000
Steps
300000
400000
50000
Average Tasks Progress for LIBERO Object
1.0
Task Progress
0.8
0.6
0.4
0.0
5000D
100000
Steps
150000
200000
3151719
Figure 5: Continuous skill adaptation and learning. The learning process is shown for both "LIBERD-Object" and CEIBERO-Spation tasks.
5.4 Ablation Study
We evaluate various design choices for task planning and reward generation in LG-SAIL.
Task planning. We compare three design choices of using LLMs for skill generation and planning:
10

---

Published in Transactions on Machine Learning Research (03/2026)
MakeCoffee
Learning in new domains. Another advantage of LG-SAIL is its ability to reuse learned models from other domains to warm-start the learning of similar skills in new domains.
1.2
LG SAIL skil rousing
League- rusiin
LG-SAIL
Loague from scratch
To validate this, we evaluate LG-SAIL. on "MakeCoffee".
Task Progress
0.5
LG-SAIL retrieves skills Open, Close, Pick, and Place from "StowHammer" in the skill library using semantic embeddings and uses these models to initialize skill policies, leading to improved learning efficiency compared to training from scratch (Fig. 6). Compared to League, LG-SAIL can retrieve skills that are similar but not. identical by measuring semantic distances over symbolic operators such as warm-starting the novel skill Insert with Place from "Stowftammer" transfer that is difficult to specify manually. In addition, this task again demonstrates that LLM-generated rewards provide more informa- tive supervision than human-designed rewards.
0.4
0.8
Env Step
Figure 6: Skill adaptation in new do- mains. For "MakeCoffee", we compare learning from scratch and adapting skills learned in the "StowHammer" domain with both LG-SAIL and League.
A Planner (LLM+A")): This variant is adopted in LG-SAIL, where LLMs are used for skill gener- ation, and A search utilized to find task plan based on the generated operators. If no valid task plan is found, the skill gencration process is re-invoked to refine the operators.
LLM Planner (LLM)): Similar to prior works on LLM-based planning (Ding et al. 2023 Kannan et al. 2024), this variant uses LLMs for both skill operator generation and task planning. If the generated plan is not executable, the process is re-invoked to produce new plan.
LLM Planner w/o Replanning (LLM 1-shot): This variant skips task plan validation and invokes skill generation and task planning only once.
We evaluate "StowHammer" across four aspects: (1) success rate of generating skill operators with correct preconditions and effects for connecting other skills, (2) success rate of generating valid plans conditioned correctly generated skills, (3) overall task success rate per method (for LLM+A* and LLM, skill generation is retried until valid plan is found), and (4) average number of attempts. Each variant runs until 25 correct plans are generated.
Results are in Tab. 2 We found that both design choices with a replanning strategy achieved 100% success rate, while the LLM 1-shot variant only reached a 36% success rate. Additionally, the LLM + A approach guarantees 100% success rate if correct skills are generated, effectively reducing LLM-based planning hallucinations and minimizing the number of replanning attempts.
Table 2: Task Planning Comparison. We evaluate the performance of different designs on "StowHammer", The task planning success rate is calculated based on trials where the skills are correctly generated.
LLM+A"
LLM
LLM 1-shot
Skill Generation Success Rate (t)
61.0%
59.3%
52.0%
Tusk Plamning Success Rate (1)
100.0%
78.1%
69.2%
Full Design Success Rate (t)
100.0%
100.0%
36.0%
Average Attempls
2.16
N/A
Reward generation. We validate the impact of full metrie functions on reward generation by comparing our approach (Full MF) to an ablated variant W/OO MF) that provides only metric function's header to the LLM, omitting both the code implementation and descriptive comments. Both are tested on "StowHammer" over 25 trials, evaluating reward validity for each skill. Following Xie et al. (Xie et al. (023).. we classify errors into four types: (1) Class altribute misuse- incorrect object selection for metric functions, (2) Attribute
11

---

Published in Transactions on Machine Learning Research (03/2026)
hallucination referencing non-existent entities, (3) Syntaz/formal errors structural mistakes, and (4) Metric selection errors inappropriate function choices for rewards. Success is based on error-free reward generation, not its impact on policy optimization.
Table 3: Reward generation comparison. We compare reward generation success rates for different skills and the full task on "StowHammer".
Full Task
Pick
Place
Open
Close
W/O MF
20.0%
40.0%
36.0%
44.0%
48.0%
Full MF
92.0%
96.0%
96.0%
100.0%
100.0%
Tab. 3 presents the reward generation success rates for "StowHammer".. Our method achieves 92% overall success, with 96% for Pick and Place, and 100% for Open and Close. reliability for long-horizon tasks and potential for lifelong learning.
The low error rate underscores its
Class Attribute Misuse
Attribute Hallucination
Syntax/Formal Error
Metric Selection Errors Correct
Full MF
92%
WO MI
52%
48%
20%
0%
25%
50%
75%
Figure 7: Error breakdown. We show the error distribution of reward generation for "StowHammer".
Fig. illustrates the error distribution across reward generation strategies. In LG-SAIL, using improved prompts and detailed metric functions reduces attribute/ /ojject selection errors from 80% to 8%. This emphasizes that structured information, like quantificd object relationships, enhances LLMs' scene interpre- tation and reward accuracy.
5.5 Real World Results
We demonstrate the transfer of our simulation-traineed LG-SAIL system to two real-world tasks: "StackAtTarget", where the robot stacks two cubes in target region in specified order, and "SortBowl", which is the task in LIBERO Benchmark that involves placing bowl on target plate.
Our system employs Franka Emika robotic arm and Microsoft Azure Kinect camera for RGBD image capture. We use AprilTag (Olson 2011) to detect the 6D poses of relevant objects and perform state estimation to synchronize with the simulated environment. Skills generated by LG-SAIL in simulation are exccuted in the real world through open-loop control.
We conduct trials per lask, with key execution frames Fig. While occasional failures occur due occlusion-indueet induced pose estination errors and bowl slippage from insecure grasps, we achieve overall success rates of 0.8 and 1.0, highlighting the robustness of our system.
6 Conclusions and Limitations
We present LG-SAIL, a framework that intogrates LLM-guidod skill generation and lcarning with TAMP, enhancing automatic and continuous robot learning across various tasks. Our results show that LG-SAIL outperforms previous approaches by reducing human intervention and improving learning efficiency.
Despite strong performance on challenging long-horiae horizon tasks, we observe common failure modes, including object slippage from unsecured grasps and unexpected contact behaviors. These issues could be mitigated by incorporating failure context into reward generation and by improving robustness during execution. While LG-SAIL currently assumes accurate object states and hand-crafted predicates, recent advances in visual perception [Wen et al. (2024), object relation modeling (Yuan al. 2022). and predicate learning (Migimatsu)
& Bohg 2022 Silver al. 2023) offer promising paths toward real-world deployment.
12

---

Published in Transactions on Machine Learning Research (03/2026)
Goal: On(bowlo, plate0)
Goal: OnTarget(cube1,, targetO) A On(cube0, cube1)
Figure 8: Real-world results. We demonstrate the deployment of LG-SAIL for real-world tasks.
References
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, llge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. In arXiv, 2023.
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Kuang-Huei Lee, Sergey Levine, Yao Lu, Linda Lun, Carolina Parada, Peter Pastor, Jornell Quiambao, Kanishka Rao, Jarek Rettinghouse, Diego Reyes, Pierre Sermanct, Nicolas Sievers, Clayton Tan, Alexander Toshev, Vincent Vanhoucke, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Mengyuan Yan, and Andy Zeng. Do as can and not as say: Grounding language in robotic affordances. In Xrv, 2022a.
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as can, not i say; Grounding language in robotic affordances. ai preprial or Xiu:2224.00169 2022b.
Shuai Bai, Yuxuan Cai, Ruizhe Chen, Kegin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, al. Qwen3-vl technical report. arXiv preprint orXiv:2511.2166 2025.
Shuo Cheng and Danfei Xu. League: Guided skill learning and abstraction for long-horizon manipulation. RA-L, 2023.
Rohan Chitnis, Tom Silver, Joshua Tenenbaum, Tomas Lozano-Perez, and Leslie Pack Kaelbling. Learning neuro-symbolic relational transition models for bilevel planning. In IROS, 2022.
Murtaza Dalal, Decpak Pathak, and Ruslan Salakhutdinov. Accelerating robotic reinforcement lcarning via parameterized action primilives. In NewrIPS, 2021.
Yan Ding, Xiaohan Zhang, Chris Paxton, and Shiqi Zhang. Task and motion planning with large language models for object rearrangement. Tn IROS, 2023.
13

---

Published in Transactions on Machine Learning Research (03/2026)
Maria Fox and Derek Long. Pdd12. 1: JATR, 2003.
An extension to pddl for expressing temporal planning domains.
Caelan Reed Garrett, Rohan Chitnis, Rachel Holladay, Beomjoon Kim, Tom Silver, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. Integrated task and motion planning. Annu. Rev. Control Robot. Auton. Syst.. 2021.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llaaa 3 herd of models. arXiv preprint (xXix2240.211688 2024.
Shixiang Gu, Ethan Holly, Timothy Lillicrap, and Sergey Levine. Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In ICRA, 2017.
Thomas Haarnoja, Aurick Zhou, Picter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In ICML, 2018.
Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shol planners: Extracting actionable knowledge for embodied agents. In ICML, 2022a.
Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tomp- son, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arve 2022b.
Frank Joublin, Antonello Ceravola, Pavel Smirnov, Felix Ocker, Joerg Deigmoeller, Anna Belardinelli, Chao Wang, Stephan Hasler, Daniel Tanneberg, and Michael Gienger. Copal: corrective planning of robot actions with large language models. In ICRA, 2024.
Leslie Pack Kaelbling and Tomás Lozano-Pérez. Integrated task and motion planning in belief space. IJRR, 2013.
Shyam Sundar Kannan, Vishnunandan LN Venkatesh, and Byung- Cheol Min. Smart-llm: Smart multi-agent robot task planning using large language models. In TROS, 2024.
George Konidaris, Leslie Pack Kaelbling, and Tomas Lozano- Perez. From skills to symbols: Learning sym- bolic representations for abstract high-level planning. JAIR, 2018.
Sergey Levine, Chelsea Finn, Trevor Darrell, and Picter Abbeel. End-to-end training of deep visuomotor policies. JMLR, 2016.
Shuang Li, Xavier Puig, Chris Paxton, Yilun Du, Clinton Wang, Linxi Fan, Tao Chen, De-An Huang, Ekin Akyürek, Anima Anandkumar, et al. Pre-trained language models for interactive decision- meaiiinn-making... NewrIPS, 2022.
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodicd control. In arXiv, 2022.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. ar preprinl ar Xrx::224 19437, 2024a.
Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. Llm+ p: Empowering large language models with optimal planning proficiency. In arxiv, 2023.
Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Bench- marking knowledge transfer for lifelong robot learning. NewrIPS, 2024b.
Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. Xiv, 2023.
14

---

Published in Transactions on Machine Learning Research (03/2026)
Yecheng Jason Ma, William Liang, Hungju Wang, Sam Wang, Yuke Zhu, Linxi Fan, Osbert Bastani, and Dinesh Jayaraman. Dreureka: Language model guided sim-to-real transfer. In RSS, 2024.
Ajay Mandlekar, Caelan Garrett, Danfei Xu, and Dieter Fox. Human-in-the-loop task and motion planning for imitation learning. In 7th Annual Conference on Robot Learning, 2023a.
Ajay Mandlckar, Caclan Reed Garrett, Danfei Xu, and Dicter Fox. Human-in-the-lopp task and motion planning for imitation learning. In CoRL, 2023b.
Michael James McDonald and Dylan Hadlield-Menell. Guided imitation of task and motion planning, In CoRL, 2022.
Toki Migimatsu and Jeannette Bohg. Grounding predicates through actions. 2022 International Confer- ence on Robotics and Automation (ICRA), pp. 3498 3504. IEEE, 2022.
Utkarsh Aashu Mishra, Shangjie Xue, Yongxin Chen, and Danfei Xu. Generative skill chaining: Long- horizon skill planning with diffusion models. In Conference Robot Learning, pp. 2905-2925. PMLR, 2023.
Soroush Nasiriany, Huihan Liu, and Yuke Zhu. Augmenting reinforcement learning with behavior primitives for diverse manipulation tasks. In ICRA, 2022a.
Soroush Nasiriany, Huihan Liu, and Yuke Zhu. Augmenting reinforcement learning with behavior primitives for diverse manipulation tasks. In ICRA, 2022b.
Edwin Olson. Apriltag: A robust and flexible visual fiducial system. In ICRA, 2011.
Krishan Rana, Jesse Haviland, Sourav Garg, Jad Abou-Chakra, Ian Reid, and Niko Suenderbauf. Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning. In CoRLL, 2023.
Archit Sharma, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Autonomous reinforce- ment learning via subgoal curricula. In NeurIPS, 2021.
Tom Silver, Ashay Athalye, Joshua B Tenenbaum, Tomás Lozano-Pérez and Leslie Pack Kaelbling, Learning neuro-symbolic skills for bilevel planning. In CoRL, 2022.
Tom Silver, Rohan Chitnis, Nishanth Kumar, Willie McClinton, Tomás Lozano- Leslie Kaelbling, and Joshua B Tenenbaum. Predicate invention for bilevel planning. In AAAI, 2023.
Tom Silver, Soham Dan, Kavitha Srinivas, Joshua B Tenenbaum, Leslic Kaclbling, and Michael Katz. Generalized planning in pddl domains with pretrained large language models. In AAAI, 2024.
Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. Progprompt: Generating situated robot task plans using large language models. In ICRA, 2023.
Richard S Sutton. Reinforcement learning: An introduction. A Bradford Book, 2018
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: physics engine for model- control. In IROS, 2012.
Ikechukwu Uchendu, Ted Xiao, Yao Lu, Banghua Zhu, Mengyuan Yan, Joséphine Simon, Matthew Bennice, Chuyuan Fu, Cong Ma, Jiantao Jiao, et al. Jump-start reinforcement learning. In ICML, 2023.
Bowen Wen, Wei Yang, Jan Kautz, and Stan Birchfield. Foundationpose: Unified 6d pose estimation and tracking of novel objects. In CVPR, 2024.
Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, and Tao Yu. 2reward: Automated dense reward function generation for reinforcement learning. arXiv, 2023.
15

---

Published in Transactions on Machine Learning Research (03/2026)
Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang- Huei Lee, Montse Gonzalez Arenas, Hao- Tien Lewis Chiang, Tom Frez, Leonard Hasenclever, Jan Humplik, et al. Language to rewards for robotic skill synthesis. arxi 2023.
Wentao Yuan, Chris Paxton, Karthik Desingh, and Dieter Fox. Sornet: Spatial object-centric representations for sequential manipulation. In CoRL, 2022.
Yuwei Zeng, Yao Mu, and Lin Shao. Learning reward for robot skills using large language models via sclf-alignment. arXiv, 2024.
Zhigen Zhao, Shuo Cheng, Yan Ding, Ziyi Zhou, Shiqi Zhang, Danfei Xu, and Ye Zhao. A survey of oplimizalion-based task and motion planning: From classical to learning approaches. or arn, 2024.
Zexin Zheng, Jia-Feng Cai, Xiao-Ming Wu, Yi-Lin Wei, Yu-Ming Tang, Ancong Wu, and Wei-Shi Zheng, ima- nip: Skill-incremental learning for robotic manipulation. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 13890- 13900, 2025.
16

---

Published in Transactions on Machine Learning Research (03/2026)
A Metric Function Definitions
Table 4: Metric Functions. We list all metric functions available in the task environments and provide their definitions.
Metric Function
Definition
is is_grasped dis ____t_obb]
Indicates whether the object is currently being held by the gripper
Distance between gripper and query object
xy vy_dis z_dis parallel_de _diis__ss___
2D Euclidean distance between gripper and object on the XY plane
Vertical (7-axis) distance between gripper and object
2D Euclidean distance between two objects on the XY plane Vertical (Z-axis) distance between two objects
perpendicular _dis
norm_ane angle
Angle difference between two objects' orientations
Tab. summarizes the metric functions used our tasks, along with their corresponding definitions.
B Task Decomposition and Skill Operator Generation
B.1 Prompt Examples for Task Decomposition
We provide the complete prompt used for the "PegInHole" task as an example for generating required actions:
Task: Decompose the given task into a sequence of reusable parameterized skills that can achieve the specified goals from the start states. Output should in format: skilll (Pbjjcctbbct sxini2 Tobject_type2), ...]
Important:
Skills should type sppamaarateerized varisbles (e.g., ?peg, ?hole) rather specific
instances (e.g., pegt, holel)
The skill sequence should represent a general solution pattern that can be applied to any valid object instances
Skills should composable and reusable similar tasks
Environment:
(:init
(HandEmpty)
(OnTable pag1)
(OnTable peg2)
(IaClear holel)
(IaClear hole2)
goal
(and
(In pegt hole1) (In pag2 hole2)
Available Predicates:
[ ('Holding(Pobbectt (?object)", "OnTable (?object)*, "In (?peg, ?hole)", "IsPeg (?object)", "IsHole (? object)", "IaClear (?hole)", "HandEmpty ()*)
Output Requirements:
Generate minimal set of parameterized skills that represent the task deconposition Use only type variables (e.g., ?peg, Phole) in the skill paraneters, NOT specific instances Skills should capture the abstract manipulation pattern, independent how many times it needs to be repeated
Output ONLY the skill sequence array explanations, extra text Format: [Sk1111 (Ptypel), Skiyl2(?typpe, ?type2), ...]
The output actions:
(Ppck)? (nper, ?hole)]
17

---

Published in Transactions on Machine Learning Research (03/2026)
After task decomposition, the next step is generating symbolic operators for each action, and then using A* for planning.
B.2 Prompt Examples for Operator Generation
We provide the complete prompt used for the "PegInHole" task as an example:
Generate preconditions (PRE) effects (EFF)
"Pick" using only
the given of predicates
The output must strictly follow the exact format below, with no additionsl text or explanations:
Pick (?peg)
PAR: [?peg:
PRE: (<list of preconditione>)
EFF-: (<list of negative effects>) {<list of positive effects>)
EFF+:
Inportant:
All possible predicates
("Holding (?abject)", "OnTable "In(?peg, ?hole)", "Ioboeee( "IsClear (?hole)", "HandEmpty() ()
"IsPeg
2. The definitions of each predicate are provided belov. These predicates definitions used in PRE EFF
considered when deciding the predicate should
Do not invent new predicates.
3. All variable names must exactly match the placeholders shown in the format (e.g., Pobject
4. PRE is the set of conditions that must be true before the action can be executed.
5. EFF- is the set of conditions that will no longer hold after the action.
6. EFF+ the set of conditions that will becone true after the action. predicates separated
7. You
keep commas.
curly-brace
notation exactly
8. Only include logically correct predicates based on the given code comments and the meaning of "Pick".
Action:
"Pick"
All predicates: ("Holding(Pobjsct)) "Oobabl"("objact"" "In (?peg, ?hole)", "IsPog(Tobject
)", "IaHooe(fobject)", "IsClear (?hole)", "HandEmpty (}
Predicate definitions:
def Holting obj): obj):
The peg grasped by the gripper
return is_grasped(ob))
def In (staate peg, hole): the hole
The peg (statee peg, hole) parallel carallel_dissstate perpendicular_dis cos norm norm_ angle (state, peg. hole)
peg. hole)
return 0.05 and -0.05 t 0.05 and cos 0.95
(other predicates definitions here)
Task:
Cenerate the preconditions (PRE) and effects (EFF) for the action "Insert" using only the given set of predicates.
output nust strictly follow exact belox, with additional explanations:
(?peg, Insert(?peg,
Insert
Phole)
PAR: pep. thole:hole]
PRE: (<list of preconditions>)
EFF-: (<list of negative effects>) {<list of positive effects>)
EFF+:
Inportant:
1. All possible predicates are: ("Holding (tobjscttt ODTable((oobeec)) "In(?pe ?hole)",
"IsPeg (Pobfect)) "IsHole (?objee("" "IsClear (?hole)", "HandEmpty()" ()
2. The definitions These predicates definitions MUST be considered when deciding if the predicate should be used in PRE or EFF
each predicate provided belov.
not invent new predicates.
3. All variable ).
must exactly match the placeholders shown the format (e.g., Pobject
4. PRE is the set of conditions that must be true before the action can be executed. no longer hold
conditions
action.
18

---

Published in Transactions on Machine Learning Research (03/2026)
conditions that will becone after action. 7. You must keep the curly-brace O notation exactly shown, with predicates separated by commas.
EFF+
8. Only include logically correct predicates based on the given code comments the meaning of "Insert".
Action: "Insert"
predicates: ("Holding(Pebjeet)) "OnTabie("object)", *In (Ppg, ?hole)", "IsPeg(Pobjec" )" "Ialoole(abjeesz"
"IsClear (?hole)", (H
Predicate definitions:
def Holding obj):
The peg grasped by the gripper
return 1ggrrappe
def (statee,, peg, hole): The peg d srrallel_dissttatee t perpendicular, cos norn antaee (state, peg, hole)
the hole
peg. hole)
satpdiiccaaaddlseeeaeeeeee dis (state, (state, peg, hole)
return 0.05 and -0.05 <= <= 0.05 and 0.95
(other predicates definitions here)
For the Holding predicate, the classifier uses the metric function is -s_graseed to evaluate whether the query object obj is held by the robot. For the In predicate, the corresponding classifter parallel dis, perpendicular dis, and norm_angle to determine whether the peg object obj 1 is inserted into the hole object obj2.
B.3 Generated Skill Operators
Below are examples of some skill operators generated by our framework for LEAGUE tasks.
"PegInHole":
Pick Pick(?peg) (?peg)
PAR: [?pag:
PRE: (HandEmpty O,, (Xpee)
EFF (HandEnpty() (), OnTable (?peg))
EFF (bolding(?peg)
Insert (?peg, Thole)
PAR: pep, ?hole:hole]
(Holding (?peg) IsClear (?hole))
EFF (Holding IsClear (?hole))
EFF
(In(?peg, ?hole))
"StowHammer":
(Pobject)
PAR: [Robject:object]
PRE: (HandEmpty O. OnTable (?object)
EFF (HandEmpty O,, OnTable (?object))
EFP+: (Holding(Pobject)
Place (?object, ?cabinet)
PAR: [robject:object Feabinet:cabinet))
PRE: (Holding (?object), IsEmpty (?cabinet), IsCabinetOpen (?cabinet))
IsEmpty (?cabinet)) EFF (In Pcabinet), HandEnpty (}}
EFF (Holding
Open (?cabinet)
PAR: [Pcabinet:cabinet]
PRE: (HasFreeSpace()) Aaaabinnettioseeeecaiieeee HandEmpty ())
EFF (HasPreeSpace (),, IsCabinetClosed (?caiinet)
EFF*: (IsCabinet0pen (?cabinet))
Close (?cabinet)
PAR: (Pcabinet:cabsnett)
(IsCabinetOpen (?cabinet), HandEmpty ()}
19

---

Published in Transactions on Machine Learning Research (03/2026)
EFF (IsCabinetOpen (?cabinet)}
EFF (HasFreeSpace (), IsCabinetClosed (?cabinet))
C Reward Generation
C.1 Prompt Examples
As example, include the prompt used to generate skill rewards for the "PegInHole" task:
Task:
Generate reward function dictionary for training the "Pick" skill in RL.
Skill: Pick (?peg)
PAR: [?peg:
PRE: (HandEapty(), (), OnTable(?peg))
EFF-: (HandEmpty() O), (n
EFF+: (Holding(?peg))
Objects: ["peg"]
Available Normalized Metric Functions: ["die_to_obj", "orientation_diff" "parallel_dis", perpendicular_dier
Reward Template format:
<metrrc__fucttoo>> <revard,score>,
Inportant Rules:
1. Output must be exactly one JSON dictionary with keys selected only from the given Metric Functions.
The definitions
each normalized metric function provided below.
These netric function definitions MUST be considered when deciding whether metric function should be used in the composed revard and its correaponding reward score. not invent new metric functions. all revard represents full task success. The purpose of this reward dictionary is to provide smooth shaping signal, allowing the revard to gradually increase as the agent approaches success.
The
will usually be less than 1.0, since total reward 1.0
4. Each keys value must be float betueen 0.0 and 1.0 (inclusive).
5. No additional text, no explanation, no comments, no formatting other than the J80N
dictionary.
6. The chosen netric functions on the given function definitions.
logically contribute learning "Pick" skill baaed
7. The only alloved output format is:
"<metric__unctton>>" srorard_
not include any metric function not listed in the available set.
Absolutely text outside dictionary.
The code of normalized netric functions are shovn below:
def dis_to_ob] (self, state, obj): a Calculate the distance from the robot gripper to the object, normalized it to [o, 1] tar prr_obj_state state [obj]
grip_dist np.linalg norm (tar_ttttrrttttttttto ("riate""gip.diee"" r_reach (1 np. (10.0 grip_dist)) return r_reach
def parallel_dis (self, state, peg. hole):
# Calculate the revard for aligning the peg with the hole, normalized it to [o, 1] peg_state state [peg] hole_state state[hole] peg_pos ("g_sttee""pos"
hole_pos = "to_stttee""aaa"
hole_mat sole_sstate""po""
hole_mat.sh. shape (3, 3)
center hole_pos hole_mat mat array ([o.1, 0, ol)
hole_normal = hole_mat a up a[r 0, o]) # np- array (IO, o, 1))
(hole_pos pog_ a hole DEFAULT
hole_normsl nornal
t_tanh_mull
20

---

Published in Transactions on Machine Learning Research (03/2026)
up. tanh (t_tanb_mule mult ap.aba(t))
return t_rew
(other normalized metric functions definitions here)
Task: Generate reward fuaction dictionary for training the "Insert" skill in AL.
Skill:
Insert (?peg, ?hole)
PAR: pepe hhole:hole]
PRE: (Holding (?peg), IsClear (?hole)) IsClear (?hole)) {In(?peg, ?hole))
EFF-: (Holding(?pen)
EFF+:
Objects: ["peg", "hole"] Available Normalized Metric Functions: ["dis_to_obj", "orientation_diff" "parallel_diet" serpeniicula_dds""
Revard Template format:
Cnnttrie_fucttinn <reward_score>,
Inportant Rules:
Output
exactly one dictionary keys selected only given Metric
Functions.
2. The definitions of each normalized metric function are provided below.
These netric function definitions MUST be considered when deciding whether metric function should be used the соmроаed reward and corresponding
Do not invent new metric functions.
3. The sum of all reward scores will usually be less than 1.0, since a total reward of 1.0 task success. The purpose this reward dictionary agent approaches smooth shaping signal, allowing the revard gradually increase
represents
provide
success.
4.
Each keys value must be a float betveen 0.0 and 1.0 (inclusive). additional text, no explanation, dictionary.
comments, formatting
6. The chosen metric functions must logically contribute to learning the "Insert" skill based on the given function definitions.
The only alloved output format 1s:
[
"<mstric__uucttoo>>> <rorard_score>,
8. Do not include any metric function not listed in the available set.
9. Absolutely no text outside the dictionary.
The code normalized
functions shoun below: [obte[ob]]
def dis_to cos_to_objioo (self, state, obj);
Calculate distance
gripper object, nornalized [o,
tar_obj_state diinelg.non norm (tar_ettrrp [toj_sstee(ggripddaes grip_dist linalg
r_reach (1 np. t10. grip_dist)) return r_resch
def darallel.dis
state, peg, hole):
# Calculate the revard for aligning the peg with the hole, normalized [0, 1] peg_state state hole_state = state[hole]
peg_pos peg_state s
hole_pos hole_state ["oe__tatee"ppe"]
hole_mat hole_state("natt
hole_mat.shan = (3,
center hole_pos hole_mat array (IO.1, 0, o1)
hole_normal hole_mat up array 0, 0]) mp. array ([O, 0, 1))
(hole_pos pos_ a hole nornal
t_tanh_mult 4.0 DEFAULT BEALL_PEG_IN_HEEE BWW__HLL__COPPPP
t_rew - np. tanh tanh_ mult ab.
return t_rex
(other normalized metric functions definitions here)
21

---

Published in Transactions on Machine Learning Research (03/2026)
C.2
Generated Rewards
We show examples of the generated rewards below:
Pick Reward
Pick P Revard =
Reard
'Is_grasped' 1.0
Pick Sick_Dense__ _eeeeddewrdd_ee =
"dis_to, to___to____l____ obj": 0.4,
"xy_dis" 0.2, "z_dis": 0.2,
Insert Reward SeerrttSpprreeeeeeeee = { 'In': 1.0,
'Is_gaaspe'' -1.0
Insert
Sessert_Dense_Reee Reward
"parallei_dia"" 0.2,
"perpendicular 0.2,
"orientation_difff" 0.35
Stack Reward
Stack Sparse_ Reward { 'On': 1.0,
"Ia_graspe'': -1.0
=
"parallel_die" 0.35, "perpendiculardde 0.35
Place Revard
Place_ Sarrce_Spars__eeee Reward
'In' 1.0, "Iaggaaspe'' -1.0
Place Place_Dense_ Revae_ Reward
"parallel_dis" dis": 0.35,
"perpendicular_ dis": 0.35
D General Compatibility with Alternative LLMs
To assess the compatibility of our framework with other open-source language models, we evaluate four state-of-the-art foundation models GPT-5.2, DeepSeck-V3 Liu et al. (2024a), Qwen3-VT. VT.225BB Bai et al.) (2025), and Llama-3.3-70B3 Grattafiori et al. (2024) on isolated task planning and skill learning settings.
For task planning and operator generation, evaluate the framework across five aspects: (1) the success rate of generating skill operators with correct preconditions and effects, (2) plan possibility (the completeness guarantee that A will find a plan whenever one exists under the generated operators), (3) the overall task success rate per model, (4) the average number of attempts, and (5) the zero-shot success rate of reward generation utilizing our Full MF formulation. For operator generation (aspects 1-4), each variant runs until 100 correct plans are generated. For reward generation and skill learning (aspect 5), each skill evaluated over 25 independent trials.
Results for operator and reward generation shown in Tab. and Tab. 6 respectively. We observe that all evaluated models achieve highly competitive performance, demonstrating the broad compatibility of our framework. Regardless of the underlying architecture, the models successfully generate valid skills with high initial success rates (ranging from 79.37% to 98.04%) and require very few replanning attempts (between 1.02 and 1.26 on avcrage). Crucially, the A planner provides a formal guarantoe of the possibility of finding an optimal task plan if the required operators are generated, thereby ensuring 100% overall task success rate across all models via replanning strategy. Concurrently, in the reward generation phase,
22

---

Published in Transactions on Machine Learning Research (03/2026)
the framework demonstrates exceptional compatibility, maintaining consistently high zero-shot success rates (averaging above 97%) across all evaluated LLMs without any model-specific fine-tuning.
Furthermore, these results suggest a clear scaling trend in our framework: as foundational LLMs improve their reasoning, the initial skill generation success rate increases, which in turn reduces the number of replanning attempts needed. This makes our framework robust to future model upgrades as base models get stronger, our system's efficiency and performance improve accordingly.
Table 5: Comparisons on Alternative LLMs for Task Planning, We evaluate our LLM+A' frame- work across four stale-ofthhe-rl models. All models achieve a 100% end-to-end success rate via replanning, A guarantees plan possibility under the generated operators. Moreover, stronger base models require fewer attempts average, indicating favorable scaling behavior.
LLM+A* Framework
GPT-5.2
Llama-3.3-70B
Swen3-VL-2230
DeepSeek- V3
Skill Generation Success Rate (t)
79.37%
90.09%
92.59%
98.04%
Plan Possibility / Task Planning Success (t)
100.0%
100.0%
100.0%
100.0%
Full Design Success Rate (f)
100.0%
100.0%
100.0%
100.0%
Average Attempts (1)
1.26
1.11
1.08
1.02
Table 6: Comparisons on Alternative LLMs for Reward Generation. We demonstrate the general compatibility of our Full MF approach across different leading foundation models on the "StowHammer" task. Results reprosent the success rate over 25 trials.
Model
Average
Pick
Place
Open
Close
-lama-3.3-70B-Insttree
97.0%
96.0%
92.0%
100.0%
100.0%
DeepSeek-V3 V3
100.0%
100.0%
100.0%
100.0%
100.0%
Owen-3-235B-Instee Instruct
99.0%
100.0%
96.0%
100.0%
100.0%
GPT-5.2
98.0%
100.0%
100.0%
92.0%
100.0%
E Comparisons on LLM-Based Skill-Planning Variants
We compare different implementation choices for LLM-based skill skill-planning systems, including LLM+IL (Be- havior Cloning) and TAMP+IL variants that follows common design patterns Ahn et al. (002bb): Mandlekar
ct al. 2023a
The LLM+IL baseline utilizes our LLM-based task planner decompose lasks into symbolic skills, and
trains language-conditionee diffusion policies for each skill. Skill execution is composed using predicate- based termination and success chocking, conceptually similar to SayCan-style skill composition Ahn et al.)
The TAMP+IL baseline replaces our RL-trained skills with behavior-cloned policies, following HITL-TAMP- style designs Mandlekar et al. (2023a) in which TAMP provides structured guidance and imitation learning refines local control.
We train scparate langnage-conditioned IL policies for each skill. For example, for the pick skill, different object instances are distinguished via CLIP-encoded language tokens such "pick peg 1 Training data is collected by rolling out expert policies that successfully complete the task. For LLM+IL, we collect 100 trajectorics per skill; for TAMP+IL, we collect 50 trajectories per skill, following the training data format in HITL-TAMP.
Results are presented in Tab. 7 IL and RL perform comparably on simpler low-contact tasks such as "StackAtTarget". However, for contact-rich tasks like "PegInHole", RL significantly outperforms II-basod variants. Furthermore, in unseen task scenarios (e.g., Goal 2), IL methods suffer from limited generalization due restricted data coverage and the inability of language--onnditoone policies to extrapolate to unseen
23

---

Published in Transactions on Machine Learning Research (03/2026)
language tokens. In contrast, method (LG-SAIL) demonstrates substantially stronger generalization under distribution shift. Moreover, when combined with our automatic domain and reward construction, the RL pipeline enables end-to-end automatic learning and continual adaptation without human in the loop (i.e,, no iterative demonstration collection or manual reward tuning).
Table 7: Comparisons LLM-Based Skill-Planning Variants.
We show success rates of different
variants on three tasks.
Task
LG-SAIL
TAMP + IL
LLM + IL
"StackAtTarget"
1.00
0.95
0.80
"PegInHole"
0.95
0.70
0.50
"PegInHole" (Goal 2)
0.90
0.40
0.10
F More Simulation Results
We present snapshots of simulation rollouts in Fig. Fig. 10 and Fig. additional video results are available on our project website: https://sites gowgle.ooooo..oooooooooooo.oo... le.... com /vw//viwwww//////////// wwntimulleeaninggg Fig. 9 gillustrates "LIBERO-Spatial", where each row corresponds to distinct task instance with the target bowl and target plate randomly placed. Fig. 10 presents "LIBERO-Object", where cach row depicts a task instance with a different larget object; the goal in each case to grasp the target object and place it into the basket. Fig. [1 shows key frames from the execution of "StackAtTarget" "StowHammer"", "PegInHole" and "MakeCoffee".
24

---

Published in Transactions on Machine Learning Research (03/2026)
Figure 9: Task Executions. We visualize tasks "LIBERO- SSpttaa " domains.
25

---

Published in Transactions on Machine Learning Research (03/2026)
Figure 10: Task Executions. We visualize tasks in "LIBERO-Object" domains.
26

---

Published in Transactions on Machine Learning Research (03/2026)
Figure 11: Task Executions. We visualize the task execution process of "PegInHole", "StackAtTarget", "Stowiammeer", and "MakeCoffee", respectively.
27


Reviews:
[{"review_id": "gxvgRyYFRE", "reviewer": "Reviewer_3M37", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents LG-SAIL, a framework and approach for combining Task and Motion Planning (TAMP) with Deep Reinforcement Learning (DRL) for lifelong robot learning, and using large Language Models to bridge the gap between them. The paper’s main thrust is that both DRL and TAMP have their unique advantages and can complement one another. While this has been shown in some recent work, the paper then introduces LLMs as the third component, which can be used to generate dense rewards, automate task decomposition, and create skills. The pipeline presented in the paper is set up so that each component helps address shortcomings from others: expert domain knowledge needed in TAMP can be replaced by using LLM’s breadth of semantic knowledge, LLM hallucinations can be fixed by grounding using TAMP’s structural information, hand-crafted dense rewards for RL can then be omitted in favor of rewards generated from LLMs. The paper also includes the use of a skill library, which allows previously learned skills to be reused to warm-start new skills. The paper benchmarks LG-SAIL against previous methods, including LEAGUE, Soft-Actor Critic, Curriculum RL, and Hierarchical RL on three domains from the LEAGUE paper; shows generalization to new goals and environments by changing aspects of the LEAGUE domains and by using tasks from the LIBERO benchmark; and performs ablations on its use of LLMs for task planning and reward generation. Finally, the paper also demonstrates that its results hold in the real world by applying LG-SAIL to two real-world tasks. \n\nKey strengths of the paper include the description of its pipeline, the emphasis on how each component strengthens the overall approach, and the breadth of the experimental design and ablations. A primary weakness of the paper is the omission of certain tasks in some experiments, without clarification of why those choices were made.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper robustly and clearly makes its case for why integrating LLMs with DRL and TAMP  is needed, supported by motivation, evaluations, and a literature review, and adequately presents the various components of LG-SAIL.", "audience_yn": "Yes", "audience_explanation": "The paper's approach will be of interest to researchers and audience members working in the fields of robot learning, motion planning, Large Language Models (LLMs) for robotics, and reinforcement learning (RL).", "requested_changes": "1. The authors need to clarify why certain tasks are chosen for certain experiments. For example, in the baseline comparison(Section 5.2 and Figure 4), why is MakeCoffee the only task of the original four from LEAGUE that is omitted?\n\n2. Similarly, in the generalization tests on LIBERO-Object and LIBERO-SPATIAL, it would have been useful to compare LG-SAIL against LEAGUE at the very least. The authors should include this or provide sufficient justification for omitting them.", "broader_impact_concerns": "The paper has no Broader Impact statement section. No broader, unique impact concern comes to mind having gone through the paper.", "additional_comments": "On a minor note, the paper contains a few grammatical or formatting errors that need correction and would benefit from general proofreading by the authors. For example, in Figure 9, rows 5 and 6 show the same rollout.", "extra_scores": {}, "extra_text": {}}, {"review_id": "qLtYEwjZoM", "reviewer": "Reviewer_zS1c", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes LG-SAIL, a framework that integrates LLMs, Task and Motion Planning, and deep reinforcement learning to support continual skill acquisition for long-horizon robotic manipulation tasks. The core idea is to leverage LLMs for (i) automatic task decomposition, (ii) symbolic operator generation, and (iii) dense reward construction, while grounding and regularizing these outputs using structured predicate and metric-function representations from TAMP. Learned skills are stored in a semantic skill library and reused via embedding-based retrieval to accelerate learning in new tasks and domains.\n\nThe paper demonstrates the framework across multiple simulated benchmarks (LEAGUE tasks and LIBERO) and includes real-world deployment on a Franka arm. Empirically, LG-SAIL shows improved learning efficiency over strong baselines, better generalization to new goals, and effective skill reuse across tasks.\n\nStrengths include a well-motivated integration of LLMs with symbolic planning, a carefully designed reward-generation mechanism constrained by metric functions, and a relatively thorough experimental evaluation including ablations and real-world tests.\nWeaknesses mainly concern assumptions on predicate availability, reliance on proprietary LLMs, and limited analysis of failure modes and scalability beyond the tested domains.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The main claims of the paper are well supported by the presented empirical and methodological evidence. The authors clearly articulate the limitations of prior approaches that either rely on hand-crafted planning domains or use unconstrained LLM outputs, and the proposed design choices are justified through both conceptual arguments and experiments.\n\nThe quantitative comparisons against RL, curriculum RL, HRL, and LEAGUE demonstrate consistent gains in task progress across several long-horizon manipulation tasks. The ablation studies, particularly those isolating the role of A* verification in planning and structured metric functions in reward generation, directly support the claims about reduced hallucination and improved robustness. The continual learning experiments on LIBERO further substantiate the claim that the skill library enables faster adaptation over time.\n\nSome claims, especially regarding generality and real-world applicability, are supported by relatively narrow evidence. The real-world experiments are promising but limited in scope, and the reliance on pre-defined predicates means that “automation” still assumes a non-trivial amount of structured prior knowledge. These points do not invalidate the claims but slightly temper their generality.", "audience_yn": "Yes", "audience_explanation": "This work is likely to be of strong interest to a subset of the TMLR audience working at the intersection of reinforcement learning, robotics, planning, and language models. In particular, researchers interested in long-horizon robotic manipulation, continual learning, and neuro-symbolic methods will find the framework and empirical insights valuable.\n\nBeyond robotics, the paper also contributes to a broader methodological discussion on how to **constrain and ground LLM outputs** using structured representations, which is relevant to ongoing debates in machine learning about reliability, hallucination, and hybrid symbolic–neural systems. While the paper is application-driven, the design principles—LLM regularization via symbolic structure and metric-function interfaces—are general enough to inspire work in adjacent domains.", "requested_changes": "Below I distinguish **critical** changes from **non-critical but strengthening** suggestions.\n\n### Critical / Important\n1. **Clarify assumptions on predicate and metric availability.**  \n   The framework assumes a predefined set of predicates and metric functions. A clearer discussion is needed on how much domain engineering this requires in practice, and how sensitive performance is to predicate design quality.\n\n2. **Reduce reliance on proprietary LLMs or discuss alternatives.**  \n   The use of GPT-4 and OpenAI embeddings is understandable, but the paper should more explicitly discuss reproducibility and expected performance with open-source LLMs, or at least acknowledge potential variability.\n\n3. **Failure mode analysis.**  \n   While success rates are reported, a more systematic analysis of common failure cases (e.g., incorrect operator effects, reward misalignment, planner–policy mismatch) would strengthen the credibility of the approach.\n\n### Non-critical / Strengthening\n4. **Scalability discussion.**  \n   Some discussion or preliminary evidence on how the approach scales with larger predicate sets, longer plans, or more diverse objects would be valuable.\n\n5. **Broader comparison to recent LLM-planning hybrids.**  \n   The related work is solid, but a more explicit comparison of conceptual differences with recent LLM-as-planner or code-as-policy approaches could help position the contribution more sharply.\n\n6. **Clarify computational overhead.**  \n   Reporting approximate LLM query counts and their relative cost compared to RL training would help readers assess practicality.", "broader_impact_concerns": "The work does not raise significant ethical concerns beyond standard issues in robotics and autonomous systems. The primary broader impact considerations relate to deployment safety and reliability, especially when LLM-generated plans or rewards are used in real-world robotic systems. While the current experiments are controlled and limited, a brief discussion on safeguards, verification, or human oversight in more open environments would be appropriate. No major red flags are apparent.", "additional_comments": "Overall, this is a thoughtful and technically solid paper that tackles a genuinely hard problem in long-horizon and continual robot learning. While some aspects—particularly the assumptions on symbolic structure and LLM dependence—limit the scope of generality, the integration itself is non-trivial and well executed. With clearer discussion of assumptions and limitations, this work would make a meaningful contribution to the literature and fits well within TMLR’s emphasis on careful empirical and methodological advances.", "extra_scores": {}, "extra_text": {}}, {"review_id": "zQsFbmC1zg", "reviewer": "Reviewer_dEU1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper gives the hypothesis that Large Language Models can integrate TAMP and DRL for continuous skill learning in long-horizon tasks. It design a LLM-in-the-loop method that using LLM to do task decomposition, reward generation and new skill creation, which can address continuous skill learning without human expert specific knowledge.", "claims_and_evidence_yn": "Yes", "claims_explanation": "This paper proposes that previous DRL methods cannot solve long-horizon manipulation problems, and previous TAMP methods need human tailored skills. And it proposes a LLM-in-the-loop method LG-SAIL to solve these problems. And experiences also effectively verify the claim by comparing the method with some methods within DRL and TAMP domains.", "audience_yn": "Yes", "audience_explanation": "This paper is about robot learning, which is a popular and worth-exploring application in machine learning.", "requested_changes": "1. I think using LLM is not an efficient way. Giving time comparison with previous methods in DRL and TAMP are also needed.\n2. Recently, imitation learning is also popular in robot learning. I think performance comparison with previous imitation learning methods are also needed, for example, iManip [1].\n\n[1] Zheng Z, Cai J F, Wu X M, et al. imanip: Skill-incremental learning for robotic manipulation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2025: 13890-13900.", "broader_impact_concerns": "No Ethical Concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)