UniDomain: Pretraining a Unified PDDL Domain from Real-World Demonstrations for Generalizable Robot Task Planning
Authors: Haoming Ye
Yunxiao Xiao
Cewu Lu
Panpan Cai
Venue: NeurIPS 2025 poster ()
Keywords: Robotics
Task Planning
Symbolic Planning
Vision Language Models
Robotic Knowledge Graph
Primary area: reinforcement_learning
TL;DR: UniDomain pre- trains a PDDL domain from robot manipulation demonstrations and applies it for online robotic task planning.

Abstract: Robotic task planning in real-world environments requires reasoning over implicit constraints from language and vision. While LLMs and VLMs offer strong priors, they struggle with long-horizon structure and symbolic grounding. Existing meth-
ods that combine LLMs with symbolic planning often rely on handcrafted or narrow domains, limiting generalization. We propose UniDomain, a framework that pre-trains a PDDL domain from robot manipulation demonstrations and applies it for online robotic task planning. It extracts atomic domains from 12,393 manipulation videos to form a unified domain with 3137 operators, 2875 predicates, and 16481 causal edges. Given a target class of tasks, it retrieves relevant atomics from the unified domain and systematically fuses them into high-quality meta-domains for zero-shot planning. Experiments on diverse real-world tasks show that UniDomain solves complex, unseen tasks in a zero-shot manner, achieving up to 58% higher task success and 160% improvement in plan optimality over state-of-the-art LLM and LLM-PDDL baselines.

UniDomain: Pretraining a Unified PDDL Domain from Real-World Demonstrations for Generalizable Robot Task Planning
Haoming Yel Yunxiao X3 Cewu Lu Panpan Cai
Shanghai Jiao Tong University
Shhaggai Innovation Institute
Beijing University of Posts and Telecommunications
Equal contribution
f Corresponding author: cai jai_paaaan tu. edu. cn
Abstract
Robotic task planning in real-world environments requires reasoning over implicit constraints from language and vision. While LLMs and VLMs offer strong pri- ors, they struggle with long-horizon structure and symbolic grounding. Existing methods that combine LLMs with symbolic planning often rely on handcrafted narrow domains, limiting generalization. We propose UniDomain, a framework that pre-trains a PDDL domain from robot manipulation demonstrations and ap- plies it to online robotic task planning. It extracts atomic domains from 12,393 manipulation videos to form a unified domain with 3, 1,13 operators, 2,875 predi- cates, and 16,481 causal edges. Given target class of tasks, it retrieves relevant atomics from the unified domain and systematically fuses them into high-quality meta- meta-domains support compositional generalization in planning. Experiments on diverse real-world tasks show that UniDomain solves complex, unseen tasks a zero-shot manner, achieving up to 58% higher task success and 160% improvement in plan optimality over state-of-the-ar LLM and LLM-PDDL baselines,
Introduction
Robotic task planning in real-world environments requires reasoning over complex constraints that are often implicitly specified in natural language instructions and grounded in visual observations. For instance, the task "partition the stack into even and odd numbers, sorted in ascending order" implicitly encodes long-term dependencies involving unstacking, sorting, and placement. Similarly, "make cup of tea" entails sequence of preparatory steps such as opening the cabinet, finding the tea cup, and boiling the water. These tasks demand structured reasoning over action preconditions, temporal dependencies, and physical affordances, in order to ensure safety (e.g., avoiding spills), prevent irreversible states, and minimize human intervention.
However, these problems remain fundamentally challenging: instructions are open-ended, scenes are unstructured, and constraints are implicit. Recent approaches leverage the commonsense priors of Large Language Models (L..Ms) and Vision-I Language Models (VL.Ms) 2 to generalize across real-world tasks. Yet, despite their strengths in language and visual understanding. L.LMs and VLMSs often fail to model action preconditions and effects accurately, and struggle with generating coherent, long-horizon plans [3]. To improve reasoning, recent work 44 5 integrates LLMs with symbolic planning using the Planning Domain Definition Language (PDDL) 7 A common pipeline translates language instructions and scene images into structured PDDL domain and problem files, then invokes symbolic planner [8] produce plan. While LLMs can reliably generate PDDL problems when the domain is given 10), they struggle to construct realistic domain files
for planning tasks.
Thee code and demonstration video are available at: huttp::/rrooticcajtuu github. wwt.ib..i//uuui......e io/UniDomain//
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

hand__re
telb
on Jabile
clear
pick, from,
hard
pour into bone
remova lid
holdiigg
con table
tabko
pidd
houd
hot
IC, Copoper
wipe table
place iin pot hand,
Figure Visualization of our pre-trained unified domain, with 3,137 operator nodes (green) and 2,875 predicate nodes (purple).
Most models are only familiar with abstract domains and Rovers, and lack grounding in real-world robot interactions.
13 14]
such as BlocksWorld, Logistics,
To overcome this limitation, we propose leveraging large-scale demonstration datasets designed for training Aasion-Language-- Actioa (VLA) models [TS 16 17 (c.g.. DROID [8] ). These datasets
grounded in real robot executions. Visual information demonstrations captures the actual precondi- tions and effects of robot actions in diverse environments. Although each demonstration typically covers a single operation, the collection spans a broad spectrum of manipulation tasks, enabling compositional generalization. By learning atomic domains from 12,393 real-world demonstrations and merging them into structured unified domain, we construct a connected symbolic knowledge graph (Figure - 3,137 operators, 2,875 predicates, and 16,481 causal edges-to support long-term planning under complex constraints across diverse household tasks.
Particularly, introduce UniDomain, framework for pre-training a unified, general-purpor PDDL domain (referred to as the unified domain) from large-scale robot manipulation demonstrations, and applying it for online planning in unseen tasks. UniDomain comprises three stages: (1) Domain Pretraining. Given a demonstration dataset, each video is segmented into keyframes using an energy- based method. A VLM proposes initial atomic domain from the keyframes, which is refined via closed-loop verification with an LLM to ensure syntactic correctness, solvability, and commonsense alignment. The resulting of atomic domains constitutes large-scale unified domain. (2) Domain Fusion. For any targeted task class, a relevant subset of atomic domains is retrieved from the large- scale unified domain and systematically fused into high-quality meta-domain. The fusion merges functionally overlapping predicates and operators. yielding a compact yet expressive subgraph for generalizable task planning. (3) Online Planning. Given a specific task, UniDomain constructs grounded PDDL problem with the high-quality meta-domain and solves it using a PDDL planner to generate an optimal plan.
We analogize these stages to the pre-training, post-training, and inference phases of foundation models 19 20]. Domain pretraining builds comprehensive unified domain encoding general manipulation knowledge. Domain fusion constructs a specialized meta-domain with enhanced symbolic connectivity. Online planning applies the meta-domain in zero-shot fashion, solve unseen tasks without additional demonstrations or feedback.
We evaluate UniDomain on four real-world task domains unseen during training, comprising 100 long-horizon tasks with complex constraints. Results show that UniDomain significantly outperforms popular LLM-only planners (e.g., PPlecissPolicies (2I]. ReAct (2) and Kate-of-the-al hybrid LLM-PDDL baselines (e.g.. ISR-LLM [23], BoN-iVML [14). achieving up to 58% higher task success and 160% better plan optimality than the strongest baselines. Ablation studies confirm that performance gains stem from data-driven domain learning, closed-loop verification, hierarchical fusion for meta-domain construction, and task-relevant grounding during online planning.
In summary, the main contributions of this work include: (1) The first framework to pre-train unified PDDL domain for high-level robot task planning from large-scale, real-world demonstrations;

---

(2) A novel LLM-based domain fusion method for combining small, disconnected PDDL domains into a coherent and compact meta-domain thus supporting compositional generalization: and (3) A novel online task planner that applies the fused meta-domain to solve general, unseen tasks through VL.M-gruudde PDDL. planning.
2 Background and Related Work
2.1 PDDL Fundamentals
The Planning Domain Definition Language (PDDL) (7] formalizes classical planning problems as a tuple (D, P), where D is a domain and P is a problem instance. The domain is defined as D (O, P), consisting of a set of operators O and predicates P. Each predicate p P is Boolean-valued function over typed objects, representing properties or relations among entities. Each operaoor O defined by its preconditions pre(o) C P U UPP and effects eff(o) C PU-P, where -P denotes the set of negated predicates. These preconditions and effects describe the logical requirements and state transitions induced by executing o. The problem instance defined as So: $g), where B is the object set, Sp is the initial state given as a grounded conjunction
P = (g, of predicates, and 8g is a partially specified target state. planner searches for an action sequence A (a1, ....... ar), where each 0₂ is grounded instance of an operator from O, such that applying A to Sg results state satisfying $g. Given (D, P), off-th off-the-shte teetthe-shelffffe symbolic planners such as Fast Downward [ use heuristic search to compute valid plan that transitions the world from 8o to goal state. The symbolic structure of PDDL enables interpretable, verifiable, and constraint-aware planning in complex domains.
2.2 LLM-based Task Planning
Large Language Models (LLMs) 1 24
exhibit strong commonsense reasoning and structured generation, making them appealing for robotic task planning. Recent work translates free-form instructions into code-like plans (e.g.. Code-as-Policiess [21], ProgPrompt (5]), filters actions using
affordance and cost (SayCan 266] SayCanPay 27 ), or integrates feedback via closed-loop reasoning
(ReAct 222. Inner Monologue 288. Reflexion 29 Others incorporate search (LLM-MCTS (30 symbolic grounding often fail enforce implicit constraints and produce coherent long-horizon plans, motivating their integration with structured symbolic systems, such as PDDL.
31I), or multimodal inputs (ViLa (32). Yet, LLMs alone
2.3 Integration of PDDL and LLM Planning
Recent work has explored hybrid LLM-PDDL. approaches to task planning. One common paradigm fixes the PDDL domain and uses LLMs to generate problems. For example, LLM++ 4 Au- toGPT+P [9]. and ViLaIn [00 translate natural language into PDDL problems conditioned on predefined domains, then solve them using classical planners. Some frameworks [5] () combine this with adaptive planning. However, their reliance manually crafted domains requires significant effort from PDDL experts and restricts generalization across tasks and environments.
Several works attempted to construct PDDL domains directly from language instructions. ISR- LLM 23 generates a nomain- pair from language. NL2Plan
uses five stages of LLM-
based verification to iteratively construct PDDL domain. BoN-iVML 14 uses Best-of-N sampling followed by iterative refinement to generate a domain. The domains generated by these methods have limited quality due to restricted information. Other work attempts to iteratively construct PDDL domains through external feedback. InterPreT [1] and LLM-DM [ incorporate human-in-thee loop to iteratively refine operators. Ada [33] and LASP 34 use environment feedback to refine the domain. While effective in simulation, these methods are impractical for scalable deployment due to the cost, latency, and noise in external feedback. In contrast, method uses internal closed-loop validation with synthetic test problems, which automates domain learning without human input.
There also exist a few works that learn PDDL domains from visual demonstration. BLADE
35
learns a domain from a robot manipulation trajectory with a given set of actions. pix2pred
36
extracts operators from visual demonstrations given a predefined set of predicates. Diehl et al. 37
and Huang et 3S automate the generation of a robotic planning domain from single or a few repeated demonstrations in simulation. These works typically aim to generate narrow domain
3

---

(a) Keyframe Extraction
(b) Atomic Domain Generation
(c) Domain Fusion
Frome
Frame
Prome
Preiheete Co
United Domain
Revision
Aroil
Atomic Donain
Aomic Domain
Donain
Donach
Learning
Paurbe
Promii
DROID Dataset
Salvability
Check
Feedbbrds
Closed-Loop Refinement
Domain
Meta
Meta Domain
Framer
Framle
P
Solution
Foodbaak
Verification
Domain
(d) Problem Generation
(e) Planning
(f) Execution
table orange bow,, drawer and put it back to the clseed chowerr
Meta Domain
Predicate
PDDL Solver
Planning
Grouping
O
Predicate & Operator Filtering
rontave, Ad (lo)
plat, Part rook (bow)
place on, Satale (boss)
Problem
place drawer (irwe))
whbe toole (lowed)
cisic. dhawer (crower)
Figure 2: Overview of UniDomain. detailed descriptions in Section
tied to the demonstrated task, and require task-specific priors (action, predicate sets or in-domain demonstrations) as input. UniDomain, however, learns unified, general- purpose PDDL domain from large-scale robotic manipulation datasets, to enable compositional generalization and support symbolic task planning in diverse manipulation tasks under complex constraints.
general-purpose
Method Overview
UniDomain (Figure 2 is a three-phase framework for sasiio--anguage-coontttooo task planning with symbolic structure. In the first phase, Domain Pretraining. atomic PDDL domains extracted from visual-language robot demonstrations using keyframe extraction (Figure 2 YLM-based domain construction, and LLM-based closed-loop refinement (FigureZp) These domains collectively form a unified domain capturing broad manipulation knowledge. the second phase, Domain Fusion (Figure 2+), baak--llass-eeevaane atomic domains retrieved and hierarchically merged into compact meta-domain by aligning functionalll-overlappe
predicates and operators. the final phase, Online Planning, a task instruction and scene image are used construct grounded PDDL problem (Figure[i), which is solved with classical planner (Figure using the fused meta-domain generate executable plans (Figure
4 Domain Pretraining
This section pre-trains general PDDL domain for robot manipulation tasks, which aggregates a large set of atomic domains learned from the DROID [18] dataset. Given each visual-language demonstration, UniDomain generates an atomic domain via two steps: unsupervised keyframe segmentation (Section 4.1b and LLM-guided domain construction with closed-loop verification (Section 4.2)
4.1 Khergy-Bassel Keyframe Extraction
Keyframe extraction is well-studied problem, though many robotics approaches rely on task-specific signals such as human-object contact or end-effector trajectories [39 40). Other methods [41] extract keyframes based on frame similarity using embeddings from pretrained vision-language models like CLIP 42 or SigL.P [3), but these approaches ineur high computational costs. We propose simple, domain-agnostic method that identifies semantic transitions in videos by detecting changes in

---

grayscale intensity (Figure 2a). Let I E Wx denote the grayscale version of the input frame t. We define the frame energy as
W H
E(I)
I(t,5)²
The energy sequence 11 is computed across all frames, and keyframes are selected by identifying local extrema using sliding window of size K: a frame is chosen keyframe if
E
max
ES or Es min Eg.
ie]t-K,4KK]
4.2 Closed-Loop Atomic Domain Generation
Given an ordered keyframe sequence {....... .,. &x) } and the associated task instruction T, we construct a symbolic PDDL domain through a multi-stage LLM-guided pipeline (Figure(2)) For each transition (ka, k441) vision-language model (VLM) infers the operator name, identifies preconditions and effects, and expands the predicate set if necessary, yielding an initial grounded domain Do- To improve consistency and generality, we pass Do and T through large language model (LLM) for holistic revision. The LLM enforces syntactic correctness, predicate reuse, and naming consistency, producing a revised domain D,. We then apply nested verification steps refine the domain:
Solvability Check. To assess domain correctness, we prompt the LLM with (Pr, I') to generate K test problems {P....., Pk) } of increasing diffficulyy. Only the predicate set P, is used for problem generation, preventing the LLM from compensating for incorrect operators D,. Each pair (Dr, Ps)
evaluated by PDDL planner. The solvability score is defined as
1 Σ K [PDDLSolver(D.; Pt) solves Ph..
S(D,)
If S(D,) 9 (default K 5, 0.6), the full, verbose feedback from the PDDL planner, including search process logs and any validation errors, is passed back to prompt the LLM for domain refinement, resulting in an updated atomic domain D,.
Solution Verification. We then verify the solution to the solvable and most challenging test problem, Ax PDDLSSolverD,,, Pk). using another LLM to check whether the plan satisfies physical constraints and commonsense expectations. The LLM reads the action sequence and identifies steps that violate physical operational commonsense. For example, it flags errors such as trying to pick an occluded object, stacking an object on itself, or applying an unsupported action to objects. If any violations are found, L.LM feedback is used prompt further domain refinement.
The two nested checks are repeated until both pass or a maximum of L = 5 iterations is reached. If convergence fails, the learned atomic domain is discarded. The entire closed-loop process can be restarted to regenerate the domain.
4.3 The Unified Domain
Using the domain learning pipeline described above, we process a total of 12,393 demonstrations from DROID, each yielding corresponding atomic domain. While each atomic domain Dr captures task-specific knowledge grounded in single demonstration, the complete set forms a comprehensive unified domain that spans the full task space present the dataset. Each atomic domain can be interpreted as a minimal symbolic knowledge graph D = (V, E), where the vertex set V = P U O O, and the edge set E Epre Eşa encodes precon- includes predicates p E P and operators
ditions (p pre o) and effects (o att, p). Aggregating all atomic domains (by taking the union of predicate and operator sets and merging large-scale symbolic knowledge graph for real-workh robotic task planning (Figure [I], comprising 3,137 operator nodes grouped into 170 semantic categories, 2,875 predicate nodes, and 16,481 causal edges. This unified representation connects otherwise isolated behaviors such as pick from_table, pour_ into___ooo b_wll,bowll____l stir__t.. bowl, remove lid, place_ in in_pot, and wipe_table, table, enabling the plan- ner to solve long-horizon tasks like "heat the milk using the pot and clean the table" through compositional generalization across atomic domains.
nodes), the unified domain forms a
5

---

Despite its broad coverage, the unified domain is not directly suitable for online planning. Its large scale poses challenges for LLMs and VLMs, making task grounding less reliable and increasing computational overhead. Second, semantic inconsistencies across atomic domains -uch varying predicate names for equivalent concepts- break symbolic continuity and reduce planning effec- tiveness. Therefore, to ensure the quality of domains and enable effective planning, for a specific task class, we first retrieve a relevant subset of atomic domains based on the relevance language instructions, and then fuse them into compact, high-quality meta-domain.
Domain Fusion
With retrieved set of task task-relevant atomic domains, {Di this section constructs meta- domain, D = U, D,, an integrated graph with improved symbolic and causal connectivity, so that better supports generalization across task variations. This achieved by merging functionally- overlapping nodes, via hierarchical fusion along a binary tree (Figure
5.1
Atomic Domain Retrieval
The set of task-relevant atomic domains can be retrieved either manually or automatically. For manual retrieval, atomic domains can be selected based on whether the associated language instructions show relevance to the target task class. For automatic retrieval, we prompt an LLM to infer the set of relevant actions based on the language description the target task class. We then use sentence embedding similarity to find the top-K matching operators in the unified domain. Atomic domains containing these relevant operators are thus retrieved for fusion.
5.2 Binary Tree Fusion
We construct the meta- meta-domain by recursively merging a set of atomic domains along binary tree. At each level I, a parent node Dj is formed by fusing its two ceitrarily-pairees child domains: Dj f(D'+ 2k- D ). where f(-,-) performs structured alignment of predicates and operators. This process proceeds bottom-up until a single, unified meta-domain resides at the root. Each node fusion is performed in two stages:
Let Pı and P2 be the predicate sets from two child domains. We compute predicate similarity using d(p;.P;) cos(E(p:), E(p;)), )). where E(-) denotes text embedding from a pretrained language model 44 Predicate pairs with Φ < Tp (with Tp = 0.3) are discarded. The remaining candidates are ranked by similarity and sequentially verified by an LLM for semantic cquivalence. Equivalent predicates are merged.
Predicate Merging.
Operator Merging, Following predicate aligmment, we update all operators referencing merged pred- icates using LLM assistance. Operator similarity computed via name embeddings: Onume (01, 02) cos(E(nnnm (o1 )), E(name(oz))
Pairs with @name < To (with maining operator pairs are ranked and passed the LLM along with their (name, pre, eff) tuples. Functionally equivalent ones are merged, inheriting the union of their preconditions and effects.
0.3) are filtered out. The re-
Task Planning with UniDomain
Given new task specified by language instruction T and scene image we apply the meta-domain D for symbolic planning. The pipeline consists of two stages: (1) constructing a task-specific PDDL problem P. and (2) solving the pair (D, P)) using classical planner such as Fast Downward [8] A central challenge is filtering task-relevant elements from 1) to reduce symbolic noise and improve solver efficiency.
Predicate Grouping. To help the LLM interpret the large predicate set P in D, we pre-organize predicates into four semantic groups: object category descriptors, state attribute indicators, spa- tial relations, and effordance-related predicates. This structured input improves the reliability of downstream problem construction.
Predicate and Operator Filtering. We first prompt a vision-language model with (D, T, T) to generate initial problem: Po LLM From Po. extract the predicate set Po used in

---

the initial and goal conditions, treating these as task-relevant predicates. Next, we extract from D the operators whose preconditions or effects involve
Po : pre (o O : Ap E Po: p
pre
0n fo 3p Po, 3 of
p),
(4)
and define the reduced operator set
O' O pre U
(5)
Using O', compact domain Dnew (Po: O') is constructed, and refined problem is generated: new ILLM (Deew T.
PDDL Planning. We then solve (D, P noww) using a symbolic planner, A = PDDDLSllver() D, Pnow). yielding an action sequence A that satisfies s9 under the symbolic constraints and minimizes cost. This filtering process improves both planning accuracy and computational efficiency by reducing irrelevant symbolic clutter.
7 Experiments
evaluate UniDomain on diverse real-world tasks. Results demonstrate that UniDomain achieves substantial improvements over the strongest baseline methods, obtaining up to 58% higher task success rate and 160% higher plan optimality. Specifically, UniDomain maintains consistently high success and optimality across diverse and previously unseen tasks, significantly reducing planning overhead compared to both LLM-only planners (c.g., Code-as Mideeas-Pors -eolciiis [21] and ReAct (22)) and hybrid LLM-PDDL methods (e.g.. ISR-LLM (23) and BoN-iVML [4]). Further ablation studies confirm that these performance gains result from learning a comprehensive domain via closed-loop verification and structured fusion and effectively filtering irrelevant predicates and operators during planning.
7.1 Experimental Setup
Tasks. The evaluation tasks span 4 unseen task domains: Block Desktop, Kitchen, and Combination. BlockWorld involves block sorting and stacking with ordering constraints; Desktop includes drawer use, wiping, folding, and document organization; Kitchen covers object transfers and food-tool manipulation; Combination mixes all domains to test cross-context generalization. There are 100 tasks in total, 40 atomic domains learned from DROID demonstrations are retrieved to construct meta-domain for all evaluation tasks, which includes 78 predicates and 61 operators. See details of the evaluation tasks and the meta-domain Appendix B and Appendix [ respectively.
Evaluation Metrics. We report success rate (SR), success-weighted relative path length (SPI.), and optimality rate (OR), defined as the fraction of plans whose cost e; falls within threshold K of the optimal oz Specifically,
N
SPL
(6)
N
sncc
Ci
1 N
OR(K)
>
110 c. Sc +K]
(7)
We additionally report the thinking time (LLM wall-clock runtime) and number of LLM calls per task to assess efficiency and overhcad.
Baselines. We compare UniDomain against two categories of methods. The first uses LLMs or VLMs as planners: Code-as-Police Policies (2 directly generates executable Python-style plans from language instructions; ReAct (22) improves robustness through closed-loop reasoning with feedback; VLM- CoT applies ch hhainoff-thought prompting [3I) in rero-shot language setting. The second category integrates LLMs with PDDL planning: ISR- LSR- 23 translates instructions into PDDL specifications for building LLM planning and iteratively refines plans with validator feedback: VLM-PDDL grounds scene and language into symbolic specifications and plans with classical solvers; BoN-iVML. [14] generates an initial PDDL domain Best-of-N sampling, refines it with verbalized feedback, and then constructs the problem file for planning.
7

---

UniDomain
ISR-LLM
BON-IVML VLM-COT
VUM-PDDL
UniDomain
ISR-LLM
React
For PDDL-integraedd planners, VLM-PDDL performs comparably to VLM-CoT, but is hindered by fragile domain and problem generation-nin minoo grounding or typing errors often lead to unsolvable plans. ISR-LLM achieves the highest task success rate among all baselines through iterative validator feedback, but its reliance on LLM-based planning results in sharp optimality drops (matching VLM- with verbalized refinement but still fails to Code as Policies
React
20
80
15
10
40
0.2
0.0
Success
OR(K=2)
OR(K-1)
DR/K-O)
(a)
(b)
(c)
Figure 3: Comparison results of UniDomain and state-of-the-art methods on unseen evaluation tasks: (a) success rates 1. success-wett weieghted relative path lengths 1, and optimality rates with thresholds (K = 2, 1, 0) 1: (b) Thinking time (s) of the top-peat ue-performiig methods; (c) number of LLM calls + of the top-performing methods. Average values are shown with standard errors.
an integrated robot system. To focus evaluation on the performance of high-level symbolic planning, we followed a standard practice in the task planning literature [45] [66 and assumed a perfect low-level control policy (human teleoperation in our experiments for both UniDomain and baselines), so that the measured performance does not get confounded with potential imperfections in the low-level controller. We used a semi-automatic evaluation approach, wherein an LLM reads the task and the plan to provide an initial assessment, followed by final verification by human experts. Example results in Appendix show that these judgments reflect well-defined objectives and commonsense constraints.
Evaluation Protocol. We evaluate UniDomain as high-level task planner,
Despite the stand-alone evaluation, UniDomain is ready for seamless integration into complete robotic system. Its high-level plan can be straightforwardly translated into natural language commands and input any low-level inguage-conditiones skill policy, like modern Vision-I Lanoon-Lngagg--cttiin (VLA) models 17). or modular approaches combining perception, motion planning, and affor- dance learning. See real-world demonstrations of such an integrated system built upon UniDomain on our project website: https::/roboticsjjee www.aug.t........... 1o/ /
7.2 Comparison Results
Figure 3/ reports performance across three metrics: Success Rate, SPL, and Optimality Rate increasing strictness levels (K 2K= 1, 0), transitioning focus from task feasibility to plan optimality. All methods were evaluated using GPT-4. via APT under a fixed temperature of 0.0.
Among LLM-only planners, Code-as-Policies achieves moderate success (51%) but degrades rapidly under stricter thresholds, highlighting its limited global reasoning capacity. VLM-CoT produces slightly more optimal plans due to stronger visual grounding, but struggles with task completion due to the absence of symbolic structure. ReAct yields the highest success among LLM-only methods by leveraging action feedback, yet its lack of explicit state tracking results in redundant or invalid steps-especially
long-horizer tasks.
CoT at K00). BoN-iVML improves over construct reliable high-quality domains the fly.
In contrast, UniDomain achieves strong and consistent performance across all metries, attaining 85% success rate and 83% optimality at K=0. It also incurs the lowest LLM thinking time and fewest LLM calls among top-performing methods (Figures and 3nd The meta-domain effectively supports compositional generalization. In 83% of tasks, our planner successfully produced not only feasible but also optimal plans through the composition of learned operators in unified domain. These results show that tre-training the unified domain, paired with post-training using domain fusion and
8

---

UniDomain (Ful)
W/O SC W/o SV
W/
DPSS
UniDomain
Structured
w/ R
OP-E
0.8
0.6
0.4-
0.2
0.0
Syntax Correct
Solvable
Solution Valid Task Performance
(
Figure 4: Results for ablation studies on domain generation: (a) ablation the atomic domain learning method; (b) ablation on the domain fusion method. All values are success rates 1 with standard errors.
test-time problem generation, enables symbolic planning to outperform both end-to-end LLM agents and the best existing hybrid planners in robustness, efficiency, and plan quality.
7.3 Ablation Studies
We conduct ablation studies to understand the contributions of core components in UniDomain. Results show that removing the closed-loop verification significantly reduces atomic domain quality, causing failures in solvability and task logic. Hierarchical fusion is critical, as a naive union of atomic domains or direct LLM-based merging yields unusable domains due semantic and structural inconsistencies. Additionally, predicate grouping and task-relevant filtering substantially boost planning performance, particularly in tasks requiring complex reasoning and compositional generalization.
Ablations on Domain Learning. In Figure 4r, we perform ablation studies atomic domain learning pipeline using 40 DROID demonstrations with paired instructions. A domain considered successful if it passes both the solvability check, ensuring that test problems PDDL planner, and the plan verification step, which checks whether the resulting plans conform to real-world physical constraints and commonsense logic. We report the success rate as the primary metric.
be solved by
To assess the contribution of each module, we compare against several ablated variants. Removing the LLLM-based revision step (w/o R) results in domains with more syntax or logical inconsistencies, increasing the average number of required refinement iterations (from 0.49 to 1.36). Disabling the solvability check (w/ SC) leads to domains that are syntactically valid but often fail due to disconnected operators and incomplete predicates. Removing the solution verification stage (w/o SV) produces domains that are solvable but fail to capture essential task logic, resulting in mis-aligned plans. Eliminating all feedback mechanisms (w/o CL) reduces the process single-pas mangl-pass LLM generation, which significantly degrades domain quality.
We also evaluate one-shot variants that use no refinement or validation, generating the domain from a single I.IM query. Using our energy-based keyframe extraction (OP-E) achieves 28% single- pass success, while the kmilarity--bsse approach [1 (OP-S) drops to 15%. In addition to higher accuracy, our energy-based method also substantially more efficient, reducing the processing time per demonstration from 47.8 seconds to just 0.6 seconds on average.
Ablations on Domain Fusion. In Figure 4p, we perform ablation studies on our hierarchical domain fusion method using test problems generated from evaluation tasks (sampled from those in Section 7.2). assessing whether the fused domain supports correct and generalizable planning. We report four metrics: syntax validity (as measured by PDDL syntax verifier), solvability (the rate of passing solvability tests), solution validity (the rate of passing plan verification), and task performance measured as the success rate of online planning.
Our full method, UniDomain, achieves perfect scores on syntax, solvability, and verification, and attains 85% task success. In contrast, using atomic domains without fusion (w/o Domain Fu- sion) where the planner selects the closest atomic domain for each task, akin to strieval-based
We performed similarity- based keyframe extraction using SigLIP-2 47 on an NVIDIA A800 GPU (80GB VRAM). running in parallel across batches. The energy-boo based method was executed in single threaded mode on an 17-14700HX CPU (32GB RAM).

---

UniDomain (Full)
w/o Filtering
w/o Grouping
1.0
0.6
0.2
0.0
BlockWorld
Desktop
Kitchen
Combination
Average
Figure 5: Results for ablation study of the UniDomain planner. Each bar shows average task success rates 1 with standard errors.
methods-yiee only 19% success, despite perfect syntax and solvability. This confirms the power of compositional generalization. Furthermore, replacing structured fusion with direct LLM-based merging strategy (w/ Structured Fusion) fails entirely: the merged domains contain structural errors, violate syntax rules, and unusable by downstream planners.
Ablations on the Planning Method. In Figure S we assess the impact of predicate organization and domain filtering on planning performance by measuring task success rates across all evaluation tasks. In w/o Grouping, we remove the structural organization predicates into semantic categories, providing the full flat list to the LLM during problem generation. In w/o Filtering, disable pruning based on task relevance and directly use the full meta-domain to perform a single-pass problem generation.
Removing predicate grouping degrades performance significantly, particularly in the Combination domain, where complex task composition requires the LLM interpret a rich and diverse predicate space. Without grouping, the flat structure overwhelms the L.LM's capacity to localize task-relevant semantics. Disabling predicate and operator filtering also leads to sharp performance drops, especially in the BlockWorld domain. These tasks rely on long-horizon action dependencies, and a compact, task- focused domain allows more coherent grounding and reasoning by reducing irrelevant information.
8 Conclusion and Limitations
We present UniDomain, a framework that addresses the challenge of task planning under complex, implicit constraints from language and vision. UniDomain learns reusable PDDL domain from large-scale visual demonstrations and applies it to zero-shot symbolic planning. By combining closed- loop domain learning, hierarchical fusion, and task-relevant filtering, UniDomain enables efficient and generalizable planning across diverse tasks. Experiments on 100 real-world task instances demonstrate that UniDomain substantially outperforms prior LLM-only and hybrid LLM-PDDL baselines, achieving higher success rates and plan optimality.
Despite its strong performance, UniDomain has a few limitations. First, automatically atomic domains can be redundant, thus construction of meta-domain can be time-consuming. Future work will focus improving the accuracy and efficiency of the domain retrieval and fusion methods. Second, UniDomain operates under the PDDL 1.0 formalism, which lacks support for temporal constraints, numeric fluents, and cost-sensitive planning. Extending the framework to richer representations such as PDDL 2.1 [88) is an important direction. Finally, our experiments assume full observability, ignoring real-world challenges like occlusion and perceptual noise. Incorporating probabilistic planning frameworks such as PPDDL or RDDL [50] a promising path toward handling uncertainty.
Acknowledgements
This work was supported in part by the National Key R&D Program of China (Grant No. 2024YFB4700600) and National Natural Science Foundation of China under grant No. 62303304.
We used generative to improve self-written texts to enhance readability. None of the presented methods and results (figures, equations, numbers, etc.) generated by Al.
10

---

References
[1] Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
[2] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint 13..:255...19233, 2025.
[3] Cassie Huang and Li Zhang, On the limit of language models as planning formalizers. arXiv preprint arXiv:2412.09875 029479999779 2024.
[4] Bo Liu, Yuqian Jiang. Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. Lim+ p: Empowering large language models with optimal planning proficiency, arXiv preprint arXiv:2304.11477 2023.
[5] Gautier Dagan, Frank Keller, and Alex Lascarides Keller. Dynamic planning with an Ilm. In Proceedings of the Language Gamification Workshop 2024 at NeurlPS, pages 1-14. Neural Information Processing Systems Foundation (NeurIPS), October 2024. Language Gamification Through 14-12-2024.
Workshop 2024 at NeurIPS: Conference date: 14-12-2024
[6] Li Zhang, Peter Jansen, Tianyi Zhang, Peter Clark, Chris Callison-Burch, and Niket Tandon. PDDLEGO: Iterative planning in textual environments. In Danushka Bollegala and Vered Shwartz, editors, Proceedings of the 13th Joint Conference on Lexical and Computational Semanties (*SEM 2024), pages 212-221, Mexico City, Mexico, June 2024. Association for Computational Linguistics.
[7] Constructions Acronautiques, Adele Howe, Craig Knoblock, ISI Drew McDermott, Ashwin Ram, Manuela Veloso, Daniel Weld, David Wilkins Sri, Anthony Barrett, Dave Christianson, al. PddIl the planning domain definition language. Technical Report, Tech. Rep., 1998.
[8] Malte Helmert. The fast downward planning system. Journal of Artificial Intelligence Research, 26:191-244 2006.
[9] Timo Birr, Christoph Pohl, Abdelrahman Younes, and Tamim Asfour. Autogpt+ p. Affordance- based task planning with large language models. arXiv preprint arXiv:2402.107778 2024.
[10] Keisuke Shirai, Cristian C. Beltran- Hertann-,rr,addzz, Masashi Hamaya, Atsushi Hashimoto, Shohei Tanaka, Kento Kawaharazuka, Kazutoshi Tanaka, Yoshitaka Ushiku, and Shinsuke Mori. Vision- language interpreter for robot task planning. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 2051-2058 2024.
[11] Lin Guan, Karthik Valmeekam, Sarath Sreedharan, and Subbarao Kambhampati, Leveraging pre-trained large language models construct and utilize world models for model-based task planning. In Thirty-seventth Conference on Neural Information Processing Systems, 2023.
[12] Muzhi Han, Yifeng Zhu, Song-Chun Zhu, Ying Nian Wu, and Yuke Zhu. Interpret: Interactive predicate learning from language feedback for generalizable task planning, arXiv preprint WrXiv:2405.19758 2024.
[13] Elliot Gestrin, Marco Kuhlmann, and Jendrik Seipp. Towards robust LLM-driven planning from minimal text descriptions. ICAPS 2024 Workshop on Human-Awar Explainable Planning. 2024.
[14] Zhouliang Yu, Yuhuan Yuan, Tim Xiao, Fuxiang Frank Xia, Jie Fu, Ge Zhang, Ge lin, and Weiyang Liu. Generating symbolic world models via test-time scaling of large language models. Transactions on Machine Learning Research, 2025.
[15] Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rb:- diffusion foundation model for bimanual manipulation, In The Thirteenth International Conference on Learning Representations, 2025.
11

---

[16] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Open- VLA: An open-souree vision-language-actinn model. In 8th Annual Conference on Robot Learning, 2024.
[17] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky, To: A vision-language-attinn flow model for general robot control, 2024.
[18] Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403. 12945, 2024.
[19] Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, et al. A comprehensive survey on pretrained foundation models: A history from hert to chatgpt. International Journal of Machine Learning and Cyhernetics, pages 1-65, 2024.
[20] Guiyao Tie, Zeli Zhao, Dingjie Song, Fuyang Wei, Rong Zhou, Yurou Dai, Wen Yin, Zhejian Yang, Jiangyue Yan, Yao Su, al. A survey on post-training of large language models. arXiv preprint arXiv:2503.06072 2025.
[21] Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 9493-9500, 2023.
[22] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
[23] Zhehua Zhou, Jiayang Song, Kunpeng Yao, Zhan Shu, and Lei Ma. Isr-lim: Iterative self-refined large language model for long-horizon sequential task planning. 2024 IEEE International Conference Robotics and Automation (ICRA), pages 2081-2088, 2024.
[24] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng. Chenyu Zhang, Chong Ruan, et al. Deepseck-v3 technical report. arXiv preprint 2MX1:22412.19437, 2024.
[25] Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. Progprompt: Generating situated robot task plans using large language models. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 1152-115330 2023.
[26] brian ichter, Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, Julian Ibarz, Alex Irpan, Eric Jang, Ryan Julian, Dmitry Kalashnikov, Sergey Levine, Yao Lu, Carolina Parada, Kanishka Rao, Pierre Sermanet, Alexander T Toshev, Vincent Vanhoucke, Fei Xia, Ted Xiao, Peng Xu, Mengyuan Yan, Noah Brown, Michael Ahn, Omar Cortes, Nicolas Sievers, Clayton Tan. Sichun Xu, Diego Reyes, Jarek Rettinghouse, Jornell Quiambao. Peter Pastor, Linda Luu, Kuang-Huei Lee, Yuheng Kuang, Sally Jesmonth, Kyle Jeffrey, Rosario Jauregui Ruano, Jasmine Hsu, Keerthana Gopalakrishnan, Byron David, Andy Zeng, and Chuyuan Kelly Fu. Do i can, not as say: Grounding language in robotic affordances. In 6th Annual Conference on Robot Learning, 2022.
[27] Rishi Hazra, Pedro Zuidberg Dos Martires, and Luc De Raedt. Saycanpay: Heuristic planning with large language models using learnable domain knowledge. In Proceedings of the AAA/ Conference on Artificial Intelligence, volume 38, pages 20123-20133 2024.
[28] Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan James Richard Tompson, Igor Mordatch, Yevgen Chebotar, Pierre Sermanet, Noah Brown, Tomas Jackson, Linda Luu, Sergey Levine, Karol Hausman, and Brian Andrew Ichter.
12

---

Innermonologue: Embodied reasoning through planning with language models. 2022. CoRL 2022 (to appear).
[29] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflex- ion: language agents with verbal reinforcement learning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances Neural Information Processing Systems, volume 36, pages 8634 -652. Curran Associates, Inc., 2023.
[30] Zirui Zhao, Wee Sun Lee, and David Hsu. Large language models as commonsense knowledge for large-scale task planning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 31967-31987. Curran Associates, Inc., 2023.
[31] Hanxu Hu, Hongyuan Lu, Huajian Zhang, Yun-Ze Song, Wai Lam, and Yue Zhang. Chain- of-symbol prompting for spatial reasoning in large language models. In First Conference on Language Modeling, 2024.
[32] Yingdong Hu, Fanqi Lin, Tong Zhang, Li Yi, and Yang Gao. Look before you leap: Unveiling the power of GPT-4v in robotic rision-langual planning. In First Workshop on Vision-Language Models for Navigation and Manipulation ICRA 2024, 2024.
[33] Lio Wong, Jiayuan Mao, Pratyusha Sharma, Zachary S. Siegel, Jiahai Feng, Noa Korneev, Joshua B. Tenenbaum, and Jacob Andreas. Learning adaptive planning representations with natural language guidance. In International Conference on Learning Representations (ICLR), 2024.
[34] Guanqi Chen, Lei Yang, Ruixing Jia, Zhe Hu,, Yizhou Chen, Wei Zhang, Wenping Wang, and Jia Pan. Language-augmented symbolic planner for open-world task planning. arXiv preprint arXiv:2407.09790 2024.
[35] Weiyu Liu, Neil Nie, Ruohan Zhang, Jiayuan Mao, and Jiajun Wu. Learning compositional behaviors from demonstration and language. In 8th Annual Conference Robot Learning, 2024.
[36] Ashay Athalye, Nishanth Kumar, Tom Silver, Yichao Liang. Tomás Lozano- Pérez, and Leslie Pack Kaelbling. Predicate invention from pixels via pretrained vision-language models. arXiv preprint htt::::5500000900 2024.
[37] Maximilian Diehl, Chris Paxton, and Karinne Ramirez-Amaro Automated generation of robotic planning domains from observations. In 2021 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6732-6738. IEEE, 2021.
[38] Jinbang Huang, Allen Tao, Rozilyn Marco, Miroslav Bogdanovic, Jonathan Kelly, and Florian Shkurti. Automated planning domain inference for task and motion planning. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 12534-1222 12540. IEEE, 2025.
[39] Zhenyu Jiang. Yuqi Xie, Kevin Lin, Zhenjia Xu, Weikang Wan, Ajay Mandlekar, Linxi Fan, and Yuke Zhu. Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning. In CoRL Workshop on Learning Robot Fine and Dexterous Manipulation: Perception and Control, 2024.
[40] Huayi Zhou, Ruixiang Wang, Yunxin Tai, Yueci Deng, Guiliang Liu, and Kui Jia. You only teach once; Learn one-shot bimanual robotic manipulation from video demonstrations, arXiv preprint arXiv:2501. 14208, 2025.
[41] Pingping Zhang, Jinlong Li, Kecheng Chen, Meng Wang, Long Xu, Haoliang Li, Nicu Sebe, Sam K Kwong, and Shiqi Wang. When video coding meets multimodal large language models: A unified paradigm for video coding. arXiv preprint vrXiv:2448.08093 2024.
[42] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages PmLR, 2021.
13

---

[43] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. Proceedings of the IEEE/CVF international conference on computer vision, pages 1975--19886 2023.
[44] Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie Liu. Mpnet: Masked and permuted pre-training for language understanding, Advances in neural information processing systems, 33:16857-16866 2020.
[45] Pierre Sermanet, Tianli Ding, Jeffrey Zhao, Fei Xia, Debidatta Dwibedi, Keerthana Gopalakrish- nan, Christine Chan, Gabriel Dulac-Arnold, Sharath Maddineni, Nikhil Joshi, et al. Robovqa: Multimodal long-horizon reasoning for robotics. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 645-652. IEEE, 2024.
[46] Wenqi Zhang, Mengna Wang, Gangao Liu, Xu Huixin, Yiwei Jiang, Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, et al. Embodied-reasee reasoner: Synergizing visual search, reasoning, and action for embodied interactive tasks. arXiv preprint arXiv:2503.2 2025.
[47] Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alab- dulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502.147588 2025.
[48] Maria Fox and Derek Long. Pdd12. 1: An extension to pddl for expressing temporal planning domains. Journal of artificial intelligence research, 20:61-124, 2003.
[49] Hákan LS Younes and Michael L Littman. PpddII. 0: An extension pddl for expressing planning domains with probabilistic effects. Techn. Rep. CMU-CS-04-162, 2:99, 2004.
[50] Scott Sanner et al. Relational dynamic influence diagram language (rddl): Language description. Unpublished ms. Australian National University, 32:27, 2010.
[51] Michel Plancherel and Mittag Leffler. Contribution é'étude de la représentation d' d'une fonction arbitraire par des intégrales définies. Rendiconti del Circolo Matematico di Palermo (1884- 1940), (0(1))289 -35, 1910.
[52] Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xuan Hu, Xu Huang, al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.0666 2025.
[53] Leslic Pack Kaelbling and Tomás Lozano-Pérez. Hierarchical task and motion planning in the now. In 201 IEEE international conference on robotics and automation, pages 1470-1477. IEEE, 2011.
[54] Zi Wang, Caelan Reed Garrett, Leslie Pack Kaelbling, and Tomás Lozano-Péres Learning compositional models of robot skills for task and motion planning. The International Journal of Robotics Research, 40(6-7):866-89) 2021.
[55] Meng Guo and Mathias Bürger. Geometric task networks: Learning efficient and explainable skill coordination for object manipulation. IEEE Transactions on Robotics, (8(3))1723-1733 2021.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction faithfully reflect the core contributions and scope of the paper. Specifically, the paper introduces UniDomain, novel framework that learns unified PDDL domain from large-scale real-world robotic manipulation demonstrations and applies it to enable zero-shot task planning. The three-stage pipeline- domain pretraining, hierarchical domain fusion, and online symbolic planning- is clearly outlined in both sections and substantiated by empirical results in later sections (e.g., Sections 4-7). The claims regarding improved task success, plan optimality, and generalization across unseen domains directly supported by the experiments distinct domains. Moreover, the abstract explicitly states the key innovations: learning from demonstrations, structured domain fusion, and zero-shot planning- all of which are realized and evaluated throughout the paper.
Justification:
100 real-world tasks from four
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper contains a dedicated discussion of limitations in Section 8 (Conclu- sion and Limitations). It explicitly identifies several practical and methodological constraints the current work. These include: (1) The reliance on PDDL 1.0, which limits expressive- ness in handling temporal constraints, numeric fluents, and cost-sensitive actions; (2) The assumption of full observability, which excludes challenges such as occlusion or sensory noise frequently encountered in real-world robotic planning.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
15

---

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
The paper does not present any formal theoretical results, theorems, or proofs. It is an empirical and methodological contribution focused on planning system design, evaluation, and application.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper provides comprehensive methodological and experimental details sufficient for reproducibility. clearly specifies the dataset used (DROID), the components of the UniDomain pipeline, domain fusion hyperparameters (Section 5.1), keyframe extrac- tion algorithm (Section 4),,, evaluation metrics and baselines (Section 7-1-2.2), and the LLM configuration (GPT-4.1 with temperature 0.0).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
16

---

While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c)
If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper plans to release core code upon acceptance and provides access test task data to support evaluation. Specifically, the learned unified PDDL domain mentioned in Section 4.3 will be released after acceptance. While full code and data are not yet released submission time, the authors commit to sharing them with appropriate documentation to support reproducibility. and the released task data can already assist in verifying the results. The code is available at: https://roboticst wtw.iiiitttuuuuutt.. iomubi.//nnn omain/
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nipe) cc/pu
blic/ htte//ggidees/ooodSSumissssssss
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https
sss.uoooiiiiiiooooociooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
17

---

The paper clearly describes the task domains and splits (Section 7.1), evaluation metrics (Section 7.1), ablation setups (Section 7.3), and fusion-related hyperparameters (Section 5.2). The planning setup, including use of GPT-4.1 and temperature setting, is also explicitly stated.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper reports standard errors for all major quantitative results, including main comparisons (Figure 3), ablations (Figures and 5), and task-level success rates (Figure the Appendix). These are described as averages with standard errors and visualize them in the figure as error bars error bands
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Answer: [Yes]
The paper reports relative execution time (e.g., planning time and LLM calls per task in Figure 3b, 3c) In the keyframe extraction experiment (Section 7.3), the computing devices (NVIDIA A800 with 80GB VRAM) and memory are described.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
18

---

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g,, preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect. with the NeurIPS Code of Ethics ltttp::/neurrpps wwwpuurippp.pppuliii....oop
Answer: [Yes]
The research does not involve human subjects, crowdsourcing, sensitive data. All datasets used are publicly available (e.g.. DROID), and the methods comply with standard academic practice. No unethical content, misuse risk, or unconsented data collection is present.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: The paper focuses on foundational research in robotics planning and does not explicitly discuss societal impacts. Potential negative applications risks (e.g., misuse, surveillance) are not addressed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
19

---

Justification: The work does not release high-risk models (e.g., LLMs or generative models). The planned release is limited to symbolic PDDL domain file derived from public robot demonstrations (Section 4.3), which poses minimal risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The paper explicitly cites the DROID dataset ([17)) and other models and techniques it builds upon, including GPT-4.1 and the Fast Downward planner. The referenced assets are standard research tools public datasets, and there is no indication that their terms use are violated.
Justification:
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of the package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The paper introduces unified PDDL domain constructed from over 12,000 robot demonstrations. This asset is described in detail in Section 4.3, including the number actions, predicates, causal edges, and its structure as a symbolic knowledge graph. The authors also state that the domain file will be released upon acceptance.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
20

---

The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing or human participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The research does not involve human subjects and thus does not require IRB approval or equivalent review.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, non-standard component of the core methods this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
The usage LLMs is central to the method and is thoroughly described throughout Sections 4-6. LLMs are used for domain generation, semantic fusion, task grounding, and planning. The specific model (GPT-4.1) is also mentioned explicitly in Section 7.2 along with its temperature setting.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
21

---

Please refer to our LLM policy http:s///oeeiss.crooootttrrrrsss for what should or should be described.
22

---

Appendix
A The Task Included in Overview
The task scene used in the overview is shown in Figure 6 corn from the pot into the orange bowl, wipe the table with the towel the drawer and put back the closed drawer." The meta-domain used is fused from 40 atomic domains learned from DROID demonstrations, and includes key operators for accomplishing such tasks, for example:
The language instruction is "Move the
(: acction pick_from_ran :parameters (?r ?o ?rk)
:precondition (and (on_rack ?o ?rk) thand_fred ?r) (rack ?rk))
:effect (and (not (on_rack ?o ?rk)) (not (reed_free?r) (holding ?r?o))
(iaction alace_on_tablee :parameters (?r ?o ?t)
iprecondition (and (holding ?r ?o) (table ?t))
:effect (and(on_tabl) ?o ?t) (hand_free ?r) (clear ?o) (not (holding ?r ?o))))
(:action pick_from_pot :parameters (?r ?o ?p)
iprecondition (and (in_pot ?o ?p) (is_open ?p) thand_free ?r))
reffect (and (holding ?r ?o) (not (in_pot ?o ?p)) (not (hand_free ?r))
(:action drawer :parametera (?r ?x)
:precondition (and (hand free ?r) (draver ?x) (not (is_open ?x))) :effect (and (is_open ?x))
The online planner generates the following PDDL. problem from the language and image input (showing only core predicates):
(iinit
(on_rack orange_bost bowl rack)
(in_pot corn pot)
cea__wipetablle towel)
(in_drawer towel yellov_drawer)
(hand_free robot)
(on lid pot)
(: goal
(and
(in_boul corn orange_boul)
(wiped table)
(in_draver towel aallowddrawer" (not (ie_open vellow_drawer).
The PDDL solver produces the following optimal plan:
(remove_lid robot lid pot)
(pick_from_rack robot orange_bow. bowl rack) (place_on_table robot wrange_boul table) (pick,from.pot robot corn pot)
(put_in_bovl robot corn orange_bovl table) (open_draver robot yellor_draver) (pick_fron_draver robot towel yellov_draver) (vipe_table robot towel table)
(place_in_draver robot tovel (illow_drawer) (close_draver robot yellov_draver)
23

---

Visual results of the plan execution are shown in Figure 7
Book of Bread
ATALIT IN FARIS
Figure 6: The initial scene of the task in the overview (Figure
Figure 7: Execution of the high-level plan generated by UniDomain.
24

---

B Evaluation Tasks
We provide representative examples of task images and corresponding language instructions for four real-world task domains -Bl--B Desktop, Kitchen, and Combination--each illustrated with two representative task instances used in our experimental evaluation.
B.1 Representative Tasks in Block World
Figure 8: Arrange all blocks into two separate stacks the table. The first stack should have blocks I, 3, 5, and in order from top to bottom. The second stack should have blocks 2, 4, 6, and 8 in order from top to bottom.
Figure 9: Arrange all blocks in a single stack (top bottom): purple, orange, blue, green, yellow, red.
B.2 Representative Tasks in Desktop
FarD
Book of Lam
Figure 10: There is a block in the green drawer. Please put it the table, push it and put the yellow drawer.
Figure 11: There is a tissue in the yellow drawer. Put it on the table and put the white book on the pink book.
25

---

B.3 Representative Tasks in Kitchen
Figure 12: Put the jujube in the green bowl. And put the white plate the rack.
Figure 13: Take the egg out of the pot and put it in the left bowl. Take the vegetable out of the pot and put it on the right bowl.
B.4 Representative Tasks in Combination
6
8
5
7
2
3
Figure 14: Put block 8 in the drawer, and arrange other blocks in a single stack the table (from top to bottom): 1, 2, 3, 4, 5, 6, 7.
Figure 15: There are spoon, a tissue, an orange block in the green drawer. Stir the bowl and put the spoon in the cup, put the orange block into the orange drawer, wipe the bowl and scrunch the tissue on the table.
26

---

C Verb Distribution in the Unified Domain and the Meta Domain
UniDomain processes 12,393 real-world demonstrations to construct unified domain consisting of 3,13 1,13 operators grouped into 170 verb types (Figure[T)) For planning evaluation, we retrieve and fuse 40 household-related atomic domains from this unified set form a compact meta-domain with 106 predicates (including negated predicates), 61 operators, and 332 causal edges (Figure) 17) This compositional meta domain integrates a diverse set of kitchen-related verbs (Figure 18).
399
300
250
290
190
100
(a) part
(b) part
(c) part 3
(d) part 4
Figure 16: Verb distribution of our pre-trained unified domain, divided into four segments (a)-d).
Figure 17: Visualization of meta domain used for planning across evaluation tasks, with 61 operator nodes (green) and 106 predicate nodes (purple).
27

---

12-
10
6-
4-
0
pick place
put pull
grasp
pour
press
wipe
open
fix drop release
fold scoop
stir remove
straighten
spread
push
slide
scrunch
unstack stack
bitt dose
Figure 18: Verb distribution of meta domain used for planning in evaluation tasks.
D Details and Justification on Energy-based Keyframe Extraction
Hyperparameter. To accommodate videos of varying lengths 1, we employ an adaptive window size
10, / < 100
20, 100 < < 150
K, specified as follows: K
30, 150 < & 200
40,200 200 < < 500
40 10 (L ([-501
+ 500
To explain how and why our seemingly-simple keyframe extraction works, we offer both theoretical and empirical justification below.
Theoretical Justification. Our method of summing squared grayscale intensities is equivalent to measuring the total energy of the image treated as a 2D signal. By Parseval's theorem [5T], this spatial-domain energy is proportional to the energy in the frequency domain. Semantic transitions video, such as an object being picked up or a drawer opening, cause significant changes in the image's 's structure and texture, which correspond to shifts in its frequency- space energy. Our method identifies keyframes by detecting the local extrema of this energy sequence, effectively capturing these of significant semantic change. Figure 19 shows an example set of keyframes extracted from DROID demonstration via our energy-based method. The key-frames successfully captured semantic phase changes (from approach, grasp, lift, to place), echoing our theoretical justification.
Empirical Justification. We use the Agibot World [52] dataset, which provides human- annotttd keyframes along with demonstrations, for empirical justification. We compare our automatically- extracted keyframes with annotate keyframes provided by Agibot World. Energy curves shown in Figure 20a and Figure 20b show that suman-annotatee keyframes consistently cluster around the local energy extrema we extracted, confirming the effectiveness of our method.
Connection to Primitive Actions. "Primitive actions", or "primitive skills". in a common terminology used in task planning (or task and motion planning) research (53 54 (55). A primitive skill is atomic, specialized skill for manipulating one object or one object-relation tuple, such as "pick an object" "open container". inducing atomic, local effects, and acting building block for longer tasks. Our energy-based keyframe extraction detects local extrema in frame-energy curves, either peak or trough, that consistently occurs at primitive skill boundaries (for example, when move ends and a pick begins). When spliting the demonstration at these extrema, each segment would span one primitive skill, resulting in the learned operator being atomic.
E Evaluation Results and Failure Examples Per Task Class
Figure 21 reports comparison results the four evaluation task classes separately, also evaluating three key metrics: Success Rate (SR), Success weighted by Path Length (SPL), and Optimality Rate
28

---

Figure 19: Energy-basee based keyframes on DROID. The automatically selected frames align with
intuitive manipulation boundaries
1.45
1.40
1.35
1.30
120
1.15
1200
1400
1500
(a)
1.65
1.50
1.55
1.45
1.40
1.30
200
a00
1200
(b)
Figure 20: (a)(b) are two example of the energy curve in Agibot World datasets. Red circles: keyframes extracted by our energy-based method: green crosses: manual annotations in Agibot World, The horizontal axis represents the frame index, and the vertical axis represents the energy value.
(OR) at increasing strictness levels (K=2,1,0). Below, we provide a qualitative comparison between the best-performing and the worst-performing methods in the four task classes.
BlockWorld. In Figure 8, the worst-performing method (React) attempted to perform an infeasible action, commanding to pick block 3 without first unstacking block 4 on top of it. The best-performing method (UniDomain) correctly recognized this physical constraint.
Task.
Arrange all blocks into two separate stacks on the table. The first stack should have blocks 1, 3, 5, and 7 in order from top to bottom. The second stack should have blocks 2, 4, 6, and 8 in order from top to bottom.
29

---

UniDomain
React
ISR-LLM
Code as Pelicies
BON-VML
VLM-COT
VLN-PODL
0.8
0.8
0.6
0.6
0.2
0.2
0.0
BlockWorld Desktop
Kitchen Combination
BlockWorld Desktop
Kitchen Combination
(a) SR
(b) SPI.
9.0
0.1
0.0
BloceWWorld
Desktug
Kitchen Combination
BlocsWorld Desktup
Kitchen Combination
BlockWorld Desktup
Kitchen Combination
(c) OR (K=2)
(d) OR (K=1)
(e) OR (K=0)
Figure 21: Separate comparison results of UniDomain and state-of- the-art baselines the four classes real-world tasks. Average values are shown with standard errors.
React plan (failed).
pick up block 1 from the top of the stack
put down block 1 on the table to start the first stack pick up block 5 from the top of the stack put down block 5 top of block
pick up block 7 from the top of the tall stack put down block 7 on top of block
pick block 3 from the bottom of the second stack pick up block 6 from the top of the second stack put doun block on the table temporarily
pick up block 2 from the top of the second stack put down block 2 on the table pick up block 6 from the table the table pick up block 2 from the table put down block 2 on top of block
put down block 6
pick block 6 from the table
pick up block 2 from the table
bottom pick up block 7 from the top of first stack
pick up block 3 from
the second stack
pick block 7 from the top of the first stack
pick up block 6 from the table
pick up block 6 from the table table
pick up block 6 from
UniDomain PDDL problem (showing only core predicates).
(iinit
(clear block1)
(on block1 block5)
(on block5 block7)
(on block7 block6)
(on block6 block2)
(on block2 block8)
(on block8 block4)
(on block4 block3)
(on_table block3 table)
(hand_free robot)
30

---

(igoal (and
(clear block1)
(on block1 block3)
(on block3 block5)
(on block5 block7)
(on_table block7 table)
(clear block2)
(on block2 block4)
(on block4 block6)
(on block6 block8)
ton_table block8 table)
UniDomain plan (succeeded).
(unstack robot block1 block5)
pplace_on_tatle robot block1 table)
(unstack robot block5
block7)
(place_on_ robot block5 table)
(unstack robot block7 block6)
(place_on_table robot block7 table)
fpick_from_ttble robot block5 table)
(stack robot block5 block7)
(unstack robot block6 block2)
(stack robot block6 block5) (unstack robot block2 block8)
[place_on_tati) robot block2 table)
(unstack robot block8 block4)
(place_on_table robot block8 table)
(unstack robot block6 block5)
(stack robot block6 block8) (unstack robot block4 block3)
(stack robot block4 block6)
(pick_from_tablll table robot block2 table) (stack robot block2 block4)
fpick_from_table robot block3 table) (stack robot block3 block5) fpick_from_table robot block1 table)
(stack robot block1 block3)
Desktop. In Figure 10, the worst-performing method (BoN-iVML) commanded to open the wrong drawer, distinct from the one instructed, and omitted critical "push" action. The pest-performing method (React) correctly understood these key task elements.
Task.
There is block in the green draver. Please put it on the table, push it and put it in the yellow draver.
BoN-IVML PDDL problem.
(iinit
block1)
(not (drawer_open green_drawer)) (not (drawer-open yellow_drawer))
(robot__andemppt))
(: (:goal
(block inneek_inyylllo_drreeee block1)
,
31

---

BoN-iVML plan (failed).
(open_draver yellov_draver)
(pick_up_bll) block. arro__grest g drawer block1 yellov_draver)
(put_down_ in yel.oellldddaeee block1 yellov_drawer
React plan (succeeded).
open the green drawer pick up the block from the green drawer put down the block on the table push the block on the table open the yellow draver pick up the block from the table put down the block in the yellow drawer
Kitchen. In Figure 12] the worst-performing method (Code as Policies) attempted to place an object into bowl that is still leaning vertical drying rack. The highest-performing method (UniDomain) correctly captured this operational commonsense.
Task.
Put the jujube the green bowl. And put the white plate on the rack.
Code as Policies plan (failed).
pick_up (jujube) put_down (green bowl) pick_up (white plate) put_down (rack)
UniDomain PDDL problem (showing only core predicates).
(iinit
(on_rack bowl_pink racki)
(on_rack bowl_greet rack1)
(on_table bowt_white table1)
(on_table plate_white table1)
(in_bovl jujubel bowl_vhite)
(hand_free robot1)
(:goal (and
(in_bowl jujubel bowl_green)
(on_rack White_ rack1)
UniDomain plan (succeeded).
(pick_from_rack robot1 bowl_green racki)
(place_on_ robot1 bowl-green table1)
[pick_from_bov robot1 jujubel bowl_white. (put_in_bowl robot1 jujubel bowl_green table1)
(pick_from_table roboti plate_whise white table1)
(place_on_rack robot1 plate_white rack1)
Combination. In Figure 15 the worst-performing method (VLM-PDDL)) suffered from syntax- errors and incomplete operators in its PDDL problems and domains. The best-penforminn method (UniDomain) ensured the quality of the PDDL domain and the syntax correctness of PDDL problems.
32

---

Task.
There are spoon, a tissue, an orange block the green drawer. Stir the boul and put the аpoon in the cup, put the orange block into the orange drawer, wipe the boul and scrunch the tissue on the table.
VLM-PDDL PDDL domain (part).
(:action open
:parameters (?dr draver)
:precondition
?dr)
reffect (and (drawer open ?dr) (not (draver - closed ?dr)))
)
(:action pick-up
:parameters (?obj object ?loc container) :precondition (and (at ?obj ?loc) (not hind Pany object)))
reffect (and (in-hand ?obj) (not (at ?obj ?loc)))
UniDomain PDDL domain (part).
(:action open drawer :parameters (?r ?x)
open_drawer
precondition (and thand_free ?r) (draver ?x) (not (is_open ?x)))
:effect (and (is_open ?x))
(:action fick_from_dran parameters (?r ?o ?d)
pprecondition (and (heed_free??r) (1n_drawer7)7d) (is_open?d) (drawer?d)
)
:effect (and (not (in (in_drawer?o7d) (not (hand_free ?r)) (holding ?r ?o))
UniDomain PDDL problem (showing only core predicates).
(iinit
(in_drawer orange_blockl draver_greem
(in_drawer spoon1 draver-green!
(in_drawer tissue1 drawer_green)
(on_table bowll table1)
(on_table cup1 table1)
(unfolded tissuel) (can_wipe_tae tissuel)
(can_wipe.bovl tissuel)
(can_stir_borf spoon1)
(hand_free robot1)
(: ggaal (and
(stirred bowl1)
(wiped boul1)
(scrunched tissuel)
(in_cup spoon1 cupl)
(in_draver orange_blockl drawer_orange)
(on_table tissue1 table1)
33

---

UniDomain plan (succeeded).
(open_draver roboti crawer-green
(pick_from_draat arroerdraver roboti spooni draver_green)
(stir_bowl robot1 spoon1 boull)
(place_in_cup robot1 spoon1 cup1)
(pick_from_draver roboti tissuel drawer-green) (vipe_bowl roboti tissuel bowl1)
(scrunch_on,
roboti tissuel tablel)
(open_drawer robotl drawer_orange (pick_from_dree drawer robot1 orangebblock draver.green) (place_ing in_ robot1 orange_block, draver_orange)
F UniDomain Main Failure Modes
In this section, we present the main failure modes of UniDomain.
Domain learning failures arise from: (1) Syntax errors, e.g., ill-formed parentheses, account for 48% of failures; (2) Missing operators-there are critical actions omitted in 39% of failed domains; (3) Logical conflicts contradictory preconditions and effects exist within operator, constituting 35% of failures; (4) Invalid tests the generated test problems contain invalid or unreachable goal states 26% of failures.
Crucially, our closed-loop refinement always successfully detects and corrects all the above issues; hence, the success rate of the closed-loop pipeline is 100%.
Online planning failures are mostly caused by perception and grounding issues: (1) Goal desintepreeation-incoreeee interpretation of task goals (e.g., reversed stacking order), occurred in 13.3% of failures; (2) Visual grounding criors-misoocalized or misclassified objects, leading to incorrect initial states, occurred in 53.4% of failures; (3) Insufficient domain coverage-the retrieved meta-domain lacks necessary operators or predicates to support the test task, which occurred 33.3% of failures.
34

---

G Prompt Design of the UniDomain Planner
To facilitate symbolic planning with the meta-domain, UniDomain employs prompt guide two crucial modules in the online planning: predicate/operator filtering and problem generation. This prompt used twice, first to identify task-relevant predicates and operators by analyzing the initial problem proposal, and second to refine the final problem file based on the filtered symbolic space.
Now you need to help robot to generate PDDL problem file based on the given PDDL domain.
The given image shows the initial scene.
The robot's 's hand is free initially (even though it shown in the image) and PDDL objects must include robot. The task is under table-top environment and PDDL objects must include at least a table.
Instructions: (instructions) given PDDL domain: (domain)
Your output should include four parts:
(1) reasoning: analyze the image and output reasons.
(2) objects: you need to locate objects related to the task from the iange.
(3) init: you need to describe PDDL init from the image based given
PDDL predicates.
(4) goal: you need to gnerate PDDL goal from human instructions.
When generating PDDL init, you should use type predicate, state predicates, spatial position relationship predicates and affordance predicates in order.
Your output should be in JSON format like below: {{
"reasoning:": "your analysis",
"objecta"; ["apple_1", "apple_2", "bowl"], "],
"init": ('(on_table boul)"], "goal": "(and (in apple_1, bovl) (in apple_2, bowl))
}}
35

---

H
other object, such lean Setting and Prompts for Baselines
H.1
Baseline Setting
In our evaluation, all LLM-based methods were adapted to use VLMs, to accept scene image as imput and support visual planning. For ReAct, we provide online visual feedback and action executability (success/failue only) to perform closed-loop planning. The maximum closed-loop action steps (including both execution and thinking steps) are constrained twice the optimal plan cost. For BoN-iVML, we make two modifications in our implementation: (1) employing multi-LLM voting for best-of-N selection, and (2) adding a PDDL problem file generation module, which uses VLM ensure visual grounding.
H.2 Prompt for VLM-COT.
Now you need to help single- robot to plan to finish the given task.
The given image shows the initial scene.
The available action APIs for the robot are:
(1) pick up: pick up some object.
(2) put down: put down the object in the robot's hand on/in some
object.
(3) open: open some object such as drawer, door etc.
(4) close: close some object such as draver, door, etc.
(5) fold: fold some object such as tovel, tissue, etc.
(6) wipe: wipe
object using the object in the robot 's hand. tissue,
(7) scrunch: scrunch some object such
(8) stir: stir some object using the object in the robot 's hand.
(9) push: push some object such as block, etc.
(10) slide: slide some object such as block, etc.
button etc. (12) turn on: turn on some object such as light, tap, etc.
(11) press: press
object such
(13) turn off: turn off some object such as light, tap, etc
(14) pull: pull some object such as rod, handle, etc.
(15) pour: pour some object using the object in the robot 's hand. object against board against a wall.
(16) lean: lean
When you make decisions, you should consider constraints based your common sense.
For example, the robot cannot pick up another object when it already holding one because it is single-armed. armed.
Instructions:
{instructions}
Your output should include two parts:
(1) reasoning: analyze the image and output the reasons.
(2) plan_sequence: output the sequence of actions in natural language complete the task.
Your output should be JSON format like below (Do NDT output comments):
{{
"reasoming:": "your amalysis", "plan_sequence": ["pick up the green block from table", "place the green block on the red block"]
}}
36

---

H.3
Prompt for Code as policies.
You are an expert at writing modular Python functions for single -armed robot to complete a task using a fixed set of atomic APIs and the image.
The given image shows the initial scene. Observe all objects and their spatial relationships, then write code based on this initial state to complete the task.
The available action APIs for the robot are: (1) pick_up(obii) pick up object.
(2) put_ (target))) put down the object in the robot's hand on/
in some object.
(3) (obj): open some object such as draver, door, etc.
object such drawer, door etc. (5) fold(: fold some object such as towel, tissue,
(4) cobse close
(6) vobe()b)): wipe some object using the object in the robot
hand.
(7) (erunch(ob)) scrunch some object such as tissue, etc
(8) stir( stir
object using the object in the robot 's
hand.
(9) push(obj). push some object such as block, etc.
(10) slide (obj): slide some object such as block, etc.
(11) press (obj); press
object such button, etc.
(12) turn on(obj): turn on some object such as light, tap, etc.
(13) turn (ob() turn off some object such as light, , tap, etc
(14) pull(obj): pull some object such as rod, handle, etc.
(15) pour()b)): pour some object using the object in the robot's
hand. (16) lobn( obj2): lean some object against some other object, such as lean a board against wall.
Instructions:
(instructions)
When you make decisions, you should consider constraints based your common sense. For example, the robot cannot pick up another object when it is already holding one because it is single-armed. Structure the code with reusable high-level functions that encapsulate meaningful sub-tasks using atomic action APIs This code will be directly executed, so must be syntactically correct Python. No markdown formatting like python or text outside the code.
Example: Open the draver, take out towel and tissue, fold both use the tissue wipe the table, and place the folded towel neatly on a shelf import numpy from robot_utils import pick_up, put_down, open, close, fold, sipe def retrieve_items.e draw (draver, towel, tissue):
np
open (draver)
backuup(tores
put_doen ("table")
pick_up put_doxn("table""
def prepare_items (tovel, tissue);
pick_up(tovel)
fold(tovel)
("t_alee("ttble")
pick_up fold(tisaue)
put_down ("taablen"
def sipe_table table):
pick.up(tissue) wipe (tablee put_dovn (tissue)
def store_tovel.onnshelll. shelf (tovel, shelf);
37

---

pick_up (towel)
pot_doon(shell:
Main execution
saarieve_ttemssfrmmdde drawer ("drawer", "towel", "tissue")
prepare_itemse ("ttove "tissue")
wipe_ tape_tabblettissee" "table") the ("tovel", "shelf")
38

---

H.4 Prompt for ReAct.
Now you need to help a single- armed robot to plan to finish the given
task. The given image shows the initial scene. The available action (4) close: close
APIs for the robot
(1) pick up: pick up some object.
(2) put down: put down the object in the robot's hand on/in some
object.
(3) open: open some object such draver, door, etc.
object such drawer, door, etc.
(5) fold: fold some object such as tovel, tissue, etc.
(6) wipe: sipe some object using the object in the robot hand.
(7) scrunch: scrunch some object such as tissue, etc.
object using the object in the robot's hand. (9) push: push some object such black,
(8) stir: stir
(10) slide: slide
object such as block, etc.
(11) press: press some object such as button, etc.
(12) turn on: turn some object such light, tap, etc.
object such light, tap, (14) pull: pull some object such as rod, handle, etc.
(13) turn off:
off
(15) pour: pour some object using the object in the robot 's hand.
(16) lean: lean some object against some other object, such as lean
board against wall.
When you make decisions, you should consider constraints based your common sense. For example, the robot cannot pick up another object when it is already holding one because it is single-armed.
You KUST use only the exact API formats listed above for your actions, and output the sequence of actions in natural language the tasks (e.g., pick up the green block from table). If you need to think through a problem first, start your response with 'think:' folloved by your reasoning, and you'll receive 'OK.' Then provide your action in the proper format.,
complete
react_example
Instruction: Put the blue cup into the open drawer Initial scene: [Image shoving table with blue cup, closed draver and other objects]
think: need open the drawer first, then pick up blue cup, and finally put it in the draver. The robot has only one arm, so I need to follow this sequence carefully. OK
open the drawer
Image: [Image showing the draver now open]
Feedback: Action successfully erecuted. pick the blue cup from the table.
Image: [Image showing the robot Feedback: Action successfully executed. put down the blue cup into the drawer Image: [Image showing the blue cup placed inside the open drawer] Action successfully executed.,
holding the blue cup]
Feedback:
react_ react_example__:
Instruction: Fold the towel and place it on the shelf. Initial scene: [Image shoving bathroom with an unfolded towel on rack and empty shelf]
pick up the towel from the rack Image: [Image showing the robot arm holding the unfolded towel]
Feedback: Action successfully executed. fold the tovel
Image: [Image showing the robot Feedback: Action successfully executed.
holding now folded towel]
> put down the towel on the shelf
Image: [Image showing the folded tovel placed on the shelf] Action successfully executed.
Feedback:
39

---

other object, such lean H.5 Prompt for VLM-PDDL.
Now a single-aran robot is going to finish the given task. The given image shows the initial scene.
The available action the robot are:
(1) pick up: pick up some object.
(2) put down: put down the object in the robot's hand on/in some
object.
(3) open: open
object such drawer, door
(4) close: close some object such as draver, door, etc.
(5) fold: fold some object such as tovel, tissue, etc.
(6) wipe: wipe some object using the object in the robot 's hand.
(7) scrunch: scrunch
object such
tissue, etc
object using the object in the robot hand. (9) push: push some object such as block, etc.
(8) stir:
(10) slide: slide some object such as block, etc.
(11) press: press some object such as button, etc.
light, tap, (13) turn off: turn off some object such as light, tap, etc
(12) turn
object such
(14) pull: pull some object such as rod, handle, etc.
(15) pour: pour some object using the object in the robot's hand.
object against board against wall.
(16) lean: lean
The robot must consider constraints based on common sense when making decisions.
For example, the robot cannot pick up another object when it already holding one because is
Now you need to help the robot to generate both PDDL domain file and PDDL problem file to finish the task, according to the given image scene and instructions.
Instructions: (instructions}
Your output should include three parts: (1) reasoning: analyze the image and output the reasons.
(2) domain: output the PDDL domain file.
(3) problem: output the PDDL problem file.
Your output should be in JSON format like below (Do NOT output comments):
{{
"reasoning:": "your analysis", "domain": "your PDDL domain file", "problen"; "your PDDL problem file"
}}
40

---

H.6 Prompts for ISR-LLM.
Translator.
You are a helpful, cnttern-folloving assistant that translates given task description into Planning Domain Definition Language (PDDL) domain and problem files.
armed Now you need to help single armed robot to plan finish the given task.
The given image will show the initial scene.
The available action APIs for the robot are:
(1) pick up: pick up
object.
(2) put down: put down the object
robot's hand on/in some
object.
(3) open: open some object such as draver, door, etc.
(4) close: close some object such as drawer, door, etc.
(5) fold: fold
object such towel, tisaue, etc.
(6) wipe: wipe some object using the object
the robot hand. (7) scrunch: scrunch some object such as tissue, etc
(8) stir: stir some object using the object in the robot 's hand.
(9) push: push some object such block, etc.
(10) slide: slide some object such as block, etc.
(11) press: press some object such as button, etc.
(12) turn on: turn on some object such as light, tap, etc.
(13) turn off: turn off some object such as light, tap, etc
object such rod, handle, etc. robot's hand. (16) lean: lean some object against some other object, such as lean
(14) pull: pull
(15) pour: pour some object using the object
board against a wall.
When you make decisions, you should consider constraints based your common sense.
For example, the robot cannot pick up another object when it is already holding one because it is dangle-armes Below are examples of PDDL files for the blocksworld problem. (examples)
Now instruction (instruction).
Planner.
You are a confident and pattern following assistant that determines action sequences complete given task, which is described by Planning Domain Definition Language (PDDL) domain and problem files.
Below are some examples: (examples)
Now the planning problem is (plamingg.robleas)
Validator.
You are a helpful, pattern- meneern-follee following assistant that examines the correctness of each action during task planning process. You work like state machine.
Below are some examples: (examples)
Question:
initial state: apddl_init_steee Goal state: (pddl_goal_statee Examined action sequence: (ection_description)
41

---

H.7 Prompts for BoN-iVML.
initialization.
Now single armed robot is going fimish the given task. The given image shows the initial scene.
Instructions: (instructions)
The available action for the robot are:
(1) pick up: pick up some object.
(2) put down: put down the object in the robot's hand on/in some
object.
(3) open: open
object such drawer, door,
(4) close: close some object such drawer, door, etc.
(5) fold: fold some object such as tovel, tissue, etc.
(6) wipe: wipe some object using the object in the robot 's hand.
(7) scrunch: scrunch some object such
tissue, etc
(8) stir: stir
object using the object in the robot hand. (9) push: push some object such as block, etc.
(10) slide: slide some object such as block, etc.
(11) press: press some object such as button, etc.
(12) turn
light, tap, (13) turn off: turn off some object such as light, tap, etc
object such
(14) pull: pull some object such as rod, handle, etc.
(15) pour: pour some object using the object in the robot's 's hand.
(16) lean: lean some object against some other object, such lean
board against wall.
The robot must consider constraints based on common sense when making decisions.
already holding one because is single armed angle-armed. For example, the robot cannot pick up another object when it is
Now are given the natural language instruction. Your task is to generate PDDL domain code. according to the given image scene and instructions.
This includes defining predicates and actions based on information provided.
Note that individual conditions in preconditions and effects should be listed separately. For example, "object1 is washed and heated" should be considered tuo separate conditions: "object1 washed" and object1 is heated".
Also, in PDDL, two predicates cannot have the same name even if they have different parameters. Each predicate must be explicitly defined in the predicate definition. It is recommended to define predicate names in an intuitive and readable way.
PDDL must have unique name, and its parameters
Remember: Ignore the information that you think is helpful for the planning task.
You are only responsible for domain generation. Before you generate the concrete domain code, you should first generate natural Language thought about the meaning step explanation of the domain code. Even if didn't provide the exact name of the predicates and actions, you should generate them based on the information provided in the natural language description.
each variable, and the step-by-
Note that you ONLY need to use PDDL 1.0!
Your output should be in JSON format like below:
{{
42

---

"reasoning"; "Analysis the image and output your reasoning, like: predicate_11 the name of predicate_l, explanation of predictate_1, predicate_n: the name predicate_n, explanation predictate_n, action_1: the name of action_1, explanation of action, action_n: the name of actio_n, explanation of
action_n.",
"domain": "The concrete pddl code for domain pddl, PDDL format
}}
Now 's your time to generate the solution, you have to follow the format provided above.
Best-of-N.
Now single armed robot is going fimish the given task.
The given image shows the initial scene.
Instructions: (instructions)
The available action the robot are:
(1) pick up: pick up some object.
(2) put down: put down the object in the robot's hand on/in some
object.
(3) open: open
object such draver, door, etc.
(4) close: close some object such as drawer, door, etc.
(5) fold: fold some object such as tovel, tissue, etc.
(6) wipe: wipe some object using the object in the robot 's hand.
(7) scrunch: scrunch some object such
tissue, etc
(8) stir:
object using the object in the robot hand. (9) push: push some object such block, etc.
(10) slide: slide some object such as block, etc.
(11) press: press some object such as button, etc.
(12) turn
turn some object such
light, , tap, etc.
(13) turn off:
off some object such as light, tap,
(14) pull: pull some object such as rod, handle, etc.
(15) pour: pour some object using the object in the robot 's hand.
(16) lean: lean some object against some other object, such lean
board against wall.
The robot must consider constraints based on common sense when making decisions.
For example, the robot cannot pick up another object when it is because is single-armo
already holding
Now a sequence of initial PDDL domains is provided, you must choose the best one that is the most relevant to the task according to the given image.
PDDL domains: {domains)
Your output should be in JSON format like below:
{{
"reasoning"; "Analysis the image and output your reasoning.", "domain_indexx;; "1"
}}
43

---

fopt (+)
Now single- robot is going to finish the given task. The given image shous the initial scene.
Instructions: (instructions)
The available action for the robot are:
(1) pick up: pick up some object.
(2) put down: put down the object in the robot's hand on/in
object.
(3) open: open some object such as draver, door etc.
(4) close: close some object such as drawer, door, etc.
(5) fold: fold some object such tovel, tissue, etc.
(6) wipe: wipe
object using the object in the robot's hand. tissue, etc
(7) scrunch: scrunch some object such
(8) stir: stir some object using the object in the robot's 's hand.
(9) push: push some object such as block, etc.
(10) slide: slide some object such as block, etc.
(11) press: press some object such button,
(12) turn on: turn on some object such as light, tap, etc.
(13) turn off: turn off some object such as light, tap, etc
(14) pull: pull some object such rod, handle, etc.
object using the object in the robot 's hand. (16) lean: lean some object against some other object, such lean
(15) pour: pour
board against a wall.
The robot must consider constraints based on common sense when making decisions.
For example, the robot cannot pick up another object when it already holding one because it is single armed.
Now PDDL domain about the task based the image is given with intermediate thoughts explaining each predicate and action Your task to generate critical feedback on the PDDL domain code based on the task and image scene.
You should evaluate the grammar and logic of the PDDL domain codes, and the logic
in the intermediate thoughts.
natural language chain of thoughts: (thought) Generated PDDL domain: (domain)
Note that you ONLY need
PDDL 1.0!
Your output should be in JSON format like belov: {{
"reasoning"; "Analysis the image and output your reasoning.", "feedback"; "Your final feedback.
}}
44

---

fupdate (+)
Now single- robot is going to finish the given task.
The given image shous the initial scene.
Instructions: (instructions)
The available action for the robot are:
(1) pick up: pick up some object.
(2) put down: put down the object in the robot's hand on/in
object.
(3) open: open some object such as draver, door etc.
(4) close: close some object such as drawer, door, etc.
(5) fold: fold some object such tovel, tissue, etc.
(6) wipe: wipe
object using the object in the robot's hand. tissue, etc
(7) scrunch: scrunch some object such
(8) stir: stir some object using the object in the robot's 's hand.
(9) push: push some object such as block, etc.
(10) slide: slide some object such as block, etc.
(11) press: press some object such button,
(12) turn on: turn on some object such as light, tap, etc.
(13) turn off: turn off some object such as light, tap, etc
(14) pull: pull some object such rod, handle, etc.
(15) pour: pour
object using the object in the robot 's hand. (16) lean: lean some object against some other object, such lean
board against a wall.
The robot must consider constraints based on common sense when making decisions.
For example, the robot cannot pick up another object when it already holding one because it is single armed.
You will be provided PDDL domain code about the task based the image and critical feedback the PDDL domain code based on the task and image.
Your task is to generate a new PDDL domain code that is more consistent with the task and update the chain of thoughts.
Natural language chain thoughts at the previous turn: {thought) Generated PDDL domain at the previous turn: (domain) The error of the PDDL domain (feedback)
Note that you ONLY need
PDDL 1.0!
Your output should be in JSON format like belov: {{
"reasoning"; "Analysis the image and output your reasoning.",
"thought"; "your updated thought", "domain": "your updated PDDL domain code in PDDL format"
}}
45

---

Problem Formulation.
Now single-armed armed robot is going to finish the given task.
The robot 's hand is free initially even though it is not shoun in the image.
The given image shows the initial
Now you need to help the robot to generate the PDDL problem file to finish the task, according to the given image scene and instructions. Instructions: (instructions) Given PDDL domain: (domain)
Note that you ONLY need to use PDDL 1.0!
Your output should be in JSON format like below (Do NOT output comments):
{{
"reasoning:": "analysis the image and output your analysia",
"problen": "your PDDL problem file in PDDL format"
}
46


Reviews:
[{"review_id": "IqPPRaSh5W", "reviewer": "Reviewer_z3bT", "summary": "This paper introduces UniDomain, a framework that extracts a unified PDDL domain from real-world robot manipulation demonstrations for generalizable task planning. This framework includes three parts: (1) Domain pretraining: extracts atomic domains from demonstration videos using keyframe segmentation and refines them via closed-loop verification with VLMs and LLMs; (2) Domain fusion: merges relevant predicate and operator; and (3) Online planning: constructs grounded PDDL problems for specific tasks and solves them using a symbolic planner. Experiments show it achieves significantly higher task success and plan optimality compared to LLM and LLM-PDDL baselines.", "questions": "See weaknesses.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper addresses a critical challenge in planning: the reliance on complete and manually constructed domain knowledge. This is a non-trivial bottleneck, as manually engineering domain models is both time-consuming and error-prone. The proposed approach of automatically constructing a unified domain from demonstrations presents a valuable direction for reducing manual effort and improving scalability.\n2. The proposed UniDomain framework demonstrates good performance across four task domains. The paper also includes comprehensive ablation studies, which effectively validate the contribution of individual components within the framework.\n\nWeaknesses:\n1. The unified domain is constructed from 40 manually selected demonstrations, raising concerns about the robustness and generalizability of the approach. It remains unclear whether alternative selections of demonstrations would produce similar results. Moreover, the potential impact of using the full DROID dataset—where a larger set of predicates and operators are present—has not been explored. Specifically, the paper does not discuss whether the increased domain complexity would affect the quality of the domain and problem files generated by the LLM during online planning, and whether this would result in a performance drop.\n2. The observation that the \"w/o grouping\" variant outperforms the full method in the Kitchen domain is counterintuitive and warrants further analysis. The lack of an explanation for this result makes it difficult to understand the trade-offs involved in the grouping component.\n3. The related work section overlooks prior efforts on domain generation from demonstrations. In particular, the following works are relevant and should be discussed:\n[1] Automated Generation of Robotic Planning Domains from Observations, IROS 2021.\n[2] Automated Planning Domain Inference for Task and Motion Planning, ICRA 2025.\n4. The paper does not clearly specify the control policy used for executing actions in the real-world experiments. Clarifying this aspect is important to assess the reproducibility and the completeness of the proposed system.", "was_revised": true, "final_justification": "Most of my concerns have been addressed. So I remain positive about this work.", "rebuttal": "We sincerely thank you for your detailed and insightful review. We are encouraged that you recognize the importance of the problem we are addressing—the bottleneck of manual PDDL domain engineering—and find our approach of automatically constructing a unified domain from demonstrations to be a \"valuable direction.\" We are also glad you found our experimental results and comprehensive ablation studies to be strong points of the paper.\n\nBelow is our response to your comments:\n\n---\n\n**Q: The unified domain is constructed from 40 manually selected demonstrations, raising concerns about the robustness and generalizability of the approach. It remains unclear whether alternative selections of demonstrations would produce similar results. Moreover, the potential impact of using the full DROID dataset has not been explored.**\n\n**A:** We would like to begin with a crucial clarification:\n\n* Our **Unified Domain** is pretrained on the full set of **12,393 demonstrations** from the DROID dataset, resulting in a large-scale symbolic knowledge graph with 3,137 operators and 2,875 predicates (as stated in Section 4.3).\n* The **40 demonstrations** you mentioned were used to construct a specific **meta-domain** for the evaluation tasks presented in the paper. In the original experiments, the atomic domains corresponding to these 40 demonstrations were manually retrieved based on their language instructions to demonstrate the effectiveness of our domain fusion and online planning stages.\n\nWe acknowledge your concern about the manual retrieval step. To address this and demonstrate the full potential and scalability of our framework, we have now implemented an **automated retrieval mechanism** that operates on the entire unified domain. For a given task class, an LLM infers relevant actions from the instruction. We then use sentence-embedding similarity to find the top-k best-matching operators in our unified domain. The atomic domains containing these top-matching operators are then selected and passed to our Domain Fusion pipeline to construct a meta-domain for online planning.\n\nWe present two new sets of results from this fully automated pipeline:\n\n1.  **Performance on Existing Tasks:** When applied to our original 100 evaluation tasks, this automated pipeline achieves a success rate of **83%** and an optimality rate (K=0)of **80%**, consitent with numbers reported in the paper (85% success and 83% optimality (K=0)).\n\n2.  **Generalization to New, Unseen Task Classes:** To further test the limits of our framework's generalization, we evaluated it on two new task classes significantly different from our original evaluation suite:\n\n    > **Household Object Rearrangement (sharing one meta-domain):**\n    >\n    > 1.  \"Put the tableware into the tray and the trash into the trash bin.\"\n    > 2.  \"Pull off the bedsheet, and put it into the laundry basket. Return the pillow and blanket from the table to the bed and fold them.\"\n    > 3.  \"Put all the remote controls on the living room coffee table into the storage box and wipe the coffee table.\"\n    > 4.  \"Put the hood on the table onto the person's head.\"\n    > 5.  \"Coil the cable on the table and place it into the storage box.\"\n    >\n    > **Room Cleaning (sharing one meta-domain):**\n    >\n    > 1.  \"Wipe the dust off the chair with a sponge.\"\n    > 2.  \"Erase all drawings on the whiteboard.\"\n    > 3.  \"Scrub the sink to remove stains.\"\n    > 4.  \"Pick up the cloth from the armrest of the chair and use it to wipe the chair and the dresser.\"\n    > 5.  \"Put the milk in the microwave and turn it on, then wipe the surface of the microwave with a towel.\"\n\n    **Results:** Using our new automated retrieval and fusion pipeline, UniDomain achieved a **99% success rate** and **99% optimality rate(K=0)** across these new tasks. In particular, these new tasks activated new operators, such as `coil_cable`, `put_in_basket`, `erase_whiteboard`, and `turn_on_microwave`, as well as predicates like `is_coiled`, `in_laundry_basket`, `on_whiteboard`, and `microwave_on`, which already existed in our unified domain.\n\n---\n\n**Q: The paper does not discuss whether the increased domain complexity would affect the quality of the domain and problem files generated by the LLM during online planning, and whether this would result in a performance drop.**\n\n**A:** You are absolutely correct that increased domain complexity can negatively impact LLM performance during online planning, by introducing noise and increasing the cognitive load for grounding. This is precisely the reason we designed the `Domain Fusion (Section 5)` and `Predicate & Operator Filtering (Section 6)` stages. These components are critical for managing complexity. Our ablation studies quantitatively demonstrate their importance:\n\n* Without **Domain Fusion**, planning success drops from **85% to 19%** (see Figure 4b), as it is highly challenging to ensure connectivity in the large-scale unified domain, whereas achieving the domain quality of a relevant subgraph (the meta-domain) is much easier.\n* Without **Predicate and Operator Filtering** at planning time, the success rate drops from **85% to 74%** (see Figure 5), as the LLM struggles to ground the task in a cluttered symbolic space.\n\n---\n\n**Q: The observation that the \"w/o grouping\" variant outperforms the full method in the Kitchen domain is counterintuitive and warrants further analysis.**\n\n**A:** This is indeed a counterintuitive result that we have investigated further. We believe this to be an experimental artifact specific to the \"Kitchen\" domain's setup rather than a systematic failure of the grouping strategy. On average, across all four domains, predicate grouping improves the success rate from 72% to 85% (see \"Average\" in Figure 5), demonstrating its overall effectiveness.\n\nOur empirical analysis suggests a potential cause for the anomaly in the Kitchen domain. We found that the failures in the \"with grouping\" case were often due to the VLM failing to correctly ground the initial state for the `(is_open pot)` and `(on_rack bowl)` predicates. The semantic grouping operation happened to shift the position of these two specific predicates significantly further down the list of 78 predicates provided in the prompt, compared to their original (random) order. We hypothesize that this reordering may have interacted with a known positional bias in LLMs, where attention can be weaker for tokens appearing later in a long context, leading to these specific grounding errors. While this is a hypothesis, it is supported by our failure analysis.\n\n---\n\n**Q: The following works are relevant and should be discussed: [1] Automated Generation of Robotic Planning Domains from Observations, IROS 2021. [2] Automated Planning Domain Inference for Task and Motion Planning, ICRA 2025.**\n\n**A:** Thank you for pointing us to this relevant work. UniDomain is distinguished from these two papers in its scale, generalization method, and verification process. Our framework is the first to **pre-train a unified PDDL domain** from over 12,000 real-world robot demonstrations, leveraging modern LLM/VLM priors for domain generation and grounding. In contrast, both [1] and [2] learn from single or few-shot demonstrations, often in simulated settings, and do not use these foundational models. Furthermore, UniDomain introduces a rigorous, multi-stage **closed-loop verification** process to ensure domain correctness and commonsense alignment, a more comprehensive approach than the limited or rule-based verification found in [1] and [2]. This large-scale, verified pre-training enables our framework's key contribution: **zero-shot compositional generalization** to solve complex, unseen, real-world tasks, whereas [1] and [2] require in-domain demonstrations to handle new domains. \n\n---\n\n**Q: The paper does not clearly specify the control policy used for executing actions in the real-world experiments.**\n\n**A:** To isolate the evaluation and focus purely on the performance of our high-level symbolic planner—which is the core contribution of this work—we followed a standard practice in the task planning literature and assumed a perfect low-level control policy (human teleoperation in our experiments for both UniDomain and baselines). This ensures that the measured task success and plan optimality are not confounded by potential imperfections in an autonomous low-level controller.\n\nHowever, UniDomain is designed for seamless integration into a complete robotic system. The high-level action sequence generated by our planner (e.g., `(pick_from_table bowl)` can serve as direct input to any language-conditioned low-level skill policy. It is particularly compatible with modern Vision-Language-Action (VLA) models, as our unified domain is learned directly from the DROID dataset, which is also used to train these VLAs.\n\nUniDomain enables a powerful hierarchical paradigm:\n\n1.  **High-Level Task Planning (UniDomain):** Interprets complex goals, reasons over long horizons and symbolic constraints, and generates an optimal sequence of primitive actions.\n2.  **Low-Level Motion Execution (VLA/etc.):** Translates each primitive action into a sequence of dexterous, adaptive robot motions.\n\nThis division of labor allows each component to address the challenges it is best suited for: symbolic reasoning for UniDomain, and sensorimotor control for the low-level policy.\n\n---\n\nWe hope these clarifications and new results have fully addressed your concerns. We thank you again for your constructive and valuable feedback."}, {"review_id": "Hc9eBf0MHB", "reviewer": "Reviewer_stD6", "summary": "The authors propose learning prior knowledge about PDDL domains required for solving robotics tasks from robot videos, and then utilizing that prior knowledge to perform better zero-shot planning in new environments. The authors show how the proposed method outperforms baselines in several real-world tasks, zero-shot solving planning problems that other methods struggle with.", "questions": "No additional questions.", "limitations": "The authors discuss the limitations of the approach. On the surface, this seems sufficient, but without more information to understand the internals of the approach better, it is difficult to say whether there are any additional ones.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "While the proposed approach might work well and would certainly be valuable if that is the case, there is almost no information in the paper to be able to clearly judge that.\n- It is unclear why a demonstration can be well segmented by simply looking at the total grayscale intensity in each frame. There is also no evaluation done on this in isolation to show that that is actually the case.\n- Not a single example of a domain for a task generated at test time is given, or an example of a generated plan. It is unclear what the approach qualitatively does based on its prior knowledge to outperform the baselines.\n- On a high level, it is unclear what the approach actually learns to enable it to perform better than the other approaches. Is it just a large set of preconditions for each action that are checked, where the VLM without prior knowledge might otherwise miss it?\n- While different domain variations and combinations get the total number of evaluation tasks to 100, they are based on only 3 relatively simple domains, covering a very small set of concepts. The majority of it seems to come down to: stacking blocks in order, putting things into bowls / onto plates / into drawers / onto dish racks, and performing table wiping.\n- Retrieval of relevant atomic domains being done manually seems like a significant downside. While it might be easy to overcome, it is not done so in the current version of the paper.", "was_revised": true, "final_justification": "As detailed in my response below, I think all my concerns about the paper largely remain. In addition, I only became aware that the low-level skills are realized using teleoperation with the reviewer cBvW noting it, as it was not stated in the paper. As detailed in my comment below, I do see this as a significant omission, but I will stay with my original rating for the paper.", "rebuttal": "We thank the reviewer for the critical and detailed feedback. The core concern raised is a lack of clarity regarding the internal workings and qualitative behavior of our method. We appreciate this feedback and will use this rebuttal to provide the necessary details, justifications, and concrete examples to address every point raised. Our goal is to offer a much clearer picture of what UniDomain learns, how it works, and why it is effective.\n\n---\n\n**Q: It is unclear what the approach qualitatively does based on its prior knowledge to outperform the baselines. On a high level, it is unclear what the approach actually learns to enable it to perform better than the other approaches.**\n\n**A:** Thanks for this crutial feedback. \nQualitatively, UniDomain's strength comes from a *principled division of labor*. It frees LLMs/VLMs from the burden of long-horizon reasoning under complex constraints—a known weakness of current models—and delegates it to a classical PDDL solver, which is explicitly designed for this.\n\n**What UniDomain Learns:**\nThe unified domain learns a large, structured, and grounded library of action models (operators with preconditions and effects) from real-world data. It's not just learning \"a large set of preconditions\"; it's learning a **compositional symbolic knowledge graph** where actions are connected through shared predicates, enabling complex plans.\n\n**How It Outperforms Baselines:**\n* **Grounding, Not Hallucinating:** LLM-only planners must reason about and generate action sequences from scratch, often hallucinating incorrect preconditions or missing crucial steps. UniDomain provides the VLM with a validated \"menu\" of predicates and operators gathered from **real-world atomic domains**. The VLM's role is simplified to **grounding** the current scene and instruction into a novel PDDL problem (extracting objects of interest and specifying the initial and goal conditions with predicates) using this menu, which is a task it excels at.\n* **Optimal, Constrained Reasoning:** Once the problem is formulated, the symbolic planner performs an efficient, optimal search that inherently respects all preconditions and dependencies. For example, in our BlockWorld tasks, the planner can handle the long chain of dependencies required for sorting, while LLM-based methods achieved success rates as low as 20% (see Figure 3a and Fig. 4a in the Appendix). UniDomain's pre-trained knowledge provides the essential structure that makes this division of labor possible.\n\n---\n\n**Q: Retrieval of relevant atomic domains being done manually seems like a significant downside. While it might be easy to overcome, it is not done so in the current version of the paper.**\n\n**A:** We are pleased to report that we have developed and evaluated a **fully automated retrieval pipeline**, removing the manual step entirely. For a given task class, an LLM infers relevant actions from the instruction. We then use sentence-embedding similarity to find the top-k best-matching operators in our unified domain. The atomic domains containing these top-matching operators are then selected and passed to our Domain Fusion pipeline to construct a meta-domain for online planning.\nWe have run two new experiments with this automated system:\n\n1.  **On Existing Tasks:** Applied to our original 100 evaluation tasks, the automated pipeline maintains the strong performance reported in the paper, achieving a consistent success rate of **83%** and an optimality rate (K=0) of **80%**. This confirms the viability of a fully automated approach.\n2.  **On New, Unseen Tasks:** We tested generalization on 10 new, diverse tasks from two new classes:\n    > **Household Object Rearrangement (sharing one meta-domain):**\n    >\n    > * \"Put the tableware into the tray and the trash into the trash bin.\"\n    > * \"Pull off the bedsheet, and put it into the laundry basket. Return the pillow and blanket from the table to the bed and fold them.\"\n    > * \"Put all the remote controls on the living room coffee table into the storage box and wipe the coffee table.\"\n    > * \"Put the hood on the table onto the person's head.\"\n    > * \"Coil the cable on the table and place it into the storage box.\"\n    >\n    > **Room Cleaning (sharing one meta-domain):**\n    >\n    > * \"Wipe the dust off the chair with a sponge.\"\n    > * \"Erase all drawings on the whiteboard.\"\n    > * \"Scrub the sink to remove stains.\"\n    > * \"Pick up the cloth from the armrest of the chair and use it to wipe the chair and the dresser.\"\n    > * \"Put the milk in the microwave and turn it on, then wipe the surface of the microwave with a towel.\"\n    >\n    \nOur automated pipeline achieved **99% success rate and 99% optimality rate(K=0)** on the new tasks, demonstrating robust zero-shot generalization to new concepts by composing operators like `put_hood_on_head`, `fold_sheet`, `scrub_sink`, `coil_cable`, and `wipe_chair`, etc..\n\n---\n\n**Q: While different domain variations and combinations get the total number of evaluation tasks to 100, they are based on only 3 relatively simple domains, covering a very small set of concepts. The majority of it seems to come down to: stacking blocks in order, putting things into bowls / onto plates / into drawers / onto dish racks, and performing table wiping.**\n\n**A:** We respectfully disagree with the characterization of the domains as simple and wish to clarify their complexity and richness.\n\n* **Conceptual Richness:** While the objects are common, their interactions activate a diverse set of concepts. These tasks required 25 distinct semantic operations, covering concepts such as `pick`, `place`, `pour`, `wipe`, `stir`, `open`, `close`, `stack`, `unstack`, `fold`, `remove lid`, `push`, `slide`, `press`, and `scrunch`, and many more.\n* **Deceptive Complexity:** The core challenge lies not in the individual actions but in their composition and the resulting long-horizon dependencies. As shown in Fig. 3a in the main text and Fig. 4a in the Appendix, strong baselines struggle significantly: ReAct achieves only 20% success in the \"simple\" BlockWorld domain, and Code-as-Policies scores only 25% in the Kitchen domain. This demonstrates that correctly identifying and satisfying the chained preconditions for these tasks is a non-trivial reasoning problem that LLM-only planners fail at.\n* **Scalability to Richer Domains:** As mentioned above, our new experiments on \"Household Object Rearrangement\" and \"Room Cleaning\" confirm that our framework readily scales to a wider variety of concepts, activating new operators to solve significantly different tasks.\n\n---\n\n**Q: It is unclear why a demonstration can be well segmented by simply looking at the total grayscale intensity in each frame. There is also no evaluation done on this in isolation to show that that is actually the case.**\n\n**A:** Thank you for this question. The justification for our energy-based keyframe extraction is both theoretical and empirical.\n\n* **Theoretical Justification:** Our method of summing squared grayscale intensities is equivalent to measuring the total energy of the image treated as a 2D signal. By Parseval's theorem [refer to paper \"Contribution à l'etude de la representation d'une fonction arbitraire par les integrales définies\"], this spatial-domain energy is proportional to the energy in the frequency domain. Semantic transitions in a video, such as an object being picked up or a drawer opening, cause significant changes in the image's structure and texture, which correspond to shifts in its frequency-space energy. Our method identifies keyframes by detecting the local extrema of this energy sequence, effectively capturing these points of significant semantic change.\n* **Empirical Justification:** We validated this method by comparing its output against human-annotated keyframes on a subset of the data. We observed that the human-annotated ground truth keyframes consistently cluster around the local energy extrema identified by our algorithm, demonstrating its effectiveness in practice. Furthermore, our ablation study (see Figure 4a) shows that this simple, efficient method (0.6 seconds/video) outperforms a more complex similarity-based approach (47.8 seconds/video) in generating successful atomic domains, with success rates of 28% vs. 15% in a single-pass setting.\n\n---\n\n**Q: Not a single example of a domain for a task generated at test time is given, or an example of a generated plan.**\n\n**A:** We apologize for this omission. To make our pipeline more concrete, here is a detailed example illustrating how the overview task (Figure 2) is solved.\n\n**Task:** \"Move the corn from the pot into the orange bowl, wipe the table with the towel in the drawer and put it back to the closed drawer.\"\n\nThe fused **meta-domain** for this task class includes key operators like `remove_lid`, `pick_from_rack`, `pick_from_pot`, `place_on_table`, `put_in_bowl`, and `wipe_table`.\n\nThe VLM grounds the scene and the instruction into this **PDDL problem**:\n```pddl\n(:init \n    (on_rack orange_bowl rack)\n    (in_pot corn pot)\n    (can_wipe_table towel)\n    (in_drawer towel yellow_drawer)\n    (hand_free robot)\n    (on lid pot)\n)\n(:goal \n    (and \n        (in_bowl corn orange_bowl) \n        (wiped table) \n        (in_drawer towel yellow_drawer) \n        (not (is_open yellow_drawer))\n    )\n)\n```\nThe PDDL solver produces the following optimal plan:\n\n```pddl\n(remove_lid robot lid pot)\n(pick_from_rack robot orange_bowl rack)\n(place_on_table robot orange_bowl table)\n(pick_from_pot robot corn pot)\n(put_in_bowl robot corn orange_bowl table)\n(open_drawer robot yellow_drawer)\n(pick_from_drawer robot towel yellow_drawer)\n(wipe_table robot towel table)\n(place_in_drawer robot towel yellow_drawer)\n(close_drawer robot yellow_drawer)\n```\n\nWe hope this example clarifies how UniDomain translates a real-world request into a structured problem and an executable plan."}, {"review_id": "L4fn1z9LZU", "reviewer": "Reviewer_n6yD", "summary": "This paper introduces a framework that constructs a unified PDDL domain from a large-scale dataset of robotic manipulation demonstrations. Instead of relying on manually designed symbolic domains, the proposed method builds a single domain containing over 3,000 operators and more than 2,000 predicates. The framework operates in three stages. First, in the domain pretraining stage, atomic PDDL domains are learned from individual demonstrations using energy-based keyframe extraction and iterative refinement with LLMs. Second, in the domain fusion stage, relevant atomic domains are hierarchically merged into a meta-domain based on text and semantic similarity. Third, during deployment, task-relevant operators and predicates are filtered from the meta-domain to generate executable plans using a symbolic planner. The framework is evaluated on 100 tasks across four domains, demonstrating improved performance compared to existing methods that use LLMs or VLMs either alone or in combination with PDDL.", "questions": "Besides the questions listed above, I have the following questions:\t\n1. What is the computational cost and inference time for constructing the unified domain?\n2. Can the authors provide the full list of generated operators?\n3. What are the 100 real-world task instances used in evaluation? What operators are required to complete each of them? Which operators are frequently reused, and which ones are rarely needed?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. While prior work has explored the integration of LLMs and VLMs with structured representations such as PDDL, this paper is unique in its attempt to learn a unified domain from a large collection of demonstration videos.\n2. The paper introduces several novel techniques to build and refine the unified domain, including the use of LLMs and PDDL planners in a verification loop, grouping of predicates and operators using text embeddings and LLMs, and pruning mechanisms to remove irrelevant elements from the domain for planning.\n\nWeaknesses:\n1. Although the paper claims that a unified domain with over 3,000 operators is constructed, the quality of this domain is not directly evaluated. Instead, the experiments use a smaller domain learned from only 40 videos.\n2. The paper is clearly written and well organized, but many implementation details are missing. These missing details, including in the supplementary material, limit the reproducibility of the method. For example:\n- In Line 151, what is the size of the sliding window used for keyframe detection?\n- In Line 161, how accurate is the LLM at generating test problems from a given predicate set and task instruction? What specific feedback is returned by the planner?\n- In Line 170, how does the LLM evaluate whether a plan satisfies physical constraints and commonsense expectations?\n- In Line 203, how are domain pairs selected for merging? Are the child domains chosen randomly?\n3. The paper provides very limited examples of generated operators and predicates. This makes it difficult to assess their quality.\n4. The paper misses an opportunity to analyze the quality of the generated domains in a systematic way. It would be valuable to examine common failure cases, such as preconditions or effects that violate physical constraints, incorrect groupings of predicates and operators, missing or redundant predicates in operators, and mismatches in abstraction levels (such as grounded versus lifted operators).", "was_revised": true, "final_justification": "The authors have provided many details in the rebuttal that address most of my concerns. My remaining concern is the lack of a systematic evaluation covering all 3,000 operators.", "rebuttal": "We thank the reviewer for recognizing the novelty of our work and for providing detailed, constructive feedback. We will use this response to provide the requested details, including a systematic failure analysis, concrete examples, new experimental results, and an analysis of operator usage.\n\n**Q: Although the paper claims that a unified domain with over 3,000 operators is constructed, the quality of this domain is not directly evaluated. Instead, the experiments use a smaller domain learned from only 40 videos.**  \n\n**A:** To demonstrate that the full unified domain (with 3,137 operators) is indeed valuable and can be used systematically, we have developed and evaluated a fully automated pipeline.  \n1. **Automated Pipeline**: For a given task class, an LLM infers relevant actions from the instruction. We then use sentence-embedding similarity to find the top-k best-matching operators in our unified domain. The atomic domains containing these matching operators are then selected and passed to our Domain Fusion pipeline to construct a meta-domain for online planning.\nThis pipeline removes any need for manual selection.  \n2. **New Experimental Results**:\n    * **On Existing Tasks:** Applied to our original 100 evaluation tasks, the automated pipeline, directly leveraging the full unified domain, maintains the strong performance reported in the paper, achieving a success rate of **83%** and an optimality rate of **80%**.\n    * **On New, Unseen Tasks:** We tested generalization on 10 new, diverse tasks from two new classes:\n    > **Household Object Rearrangement (sharing one meta-domain):**\n    >\n    > * \"Put the tableware into the tray and the trash into the trash bin.\"\n    > * \"Pull off the bedsheet, and put it into the laundry basket. Return the pillow and blanket from the table to the bed and fold them.\"\n    > * \"Put all the remote controls on the living room coffee table into the storage box and wipe the coffee table.\"\n    > * \"Put the hood on the table onto the person's head.\"\n    > * \"Coil the cable on the table and place it into the storage box.\"\n    >\n    > **Room Cleaning (sharing one meta-domain):**\n    >\n    > * \"Wipe the dust off the chair with a sponge.\"\n    > * \"Erase all drawings on the whiteboard.\"\n    > * \"Scrub the sink to remove stains.\"\n    > * \"Pick up the cloth from the armrest of the chair and use it to wipe the chair and the dresser.\"\n    > * \"Put the milk in the microwave and turn it on, then wipe the surface of the microwave with a towel.\"\n    >\n    > Our automated pipeline achieved **99% success rate and 99% optimality rate(K=0)** on the new tasks, demonstrating robust zero-shot generalization to new concepts by activating operators like `put_hood_on_head`, `fold_sheet`, `scrub_sink`, `coil_cable`, and `wipe_chair`, etc..\n\n---\n\n**Q: The paper is clearly written and well organized, but many implementation details are missing...**\n\n**A:** We apologize for the lack of these details in the manuscript and are happy to clarify them.  \n- **Sliding Window Size (Line 151)**: We use an adaptive sliding window size *K* based on the video's frame count *n* to handle varying demonstration lengths. The formula is:  \n\n  ```\n  K = 10                    if n ≤ 100\n      20                    if 100 < n ≤ 150\n      30                    if 150 < n ≤ 200\n      40                    if 200 < n ≤ 500\n      10 * (⌊(n - 501)/200⌋ + 1)  if n > 500\n  ```  \n\n- **Test Problem Generation & Planner Feedback (Line 161)**: We acknowledge that LLM‑generated test problems can sometimes be flawed. To account for this, our solvability check uses a soft threshold: a domain passes if it can solve at least 60% of the generated test problems, making the process robust to a few ill‑formed tests. For feedback, the full, verbose output from the Fast Downward planner, including search process logs and any validation errors, is passed back to the LLM for domain refinement.  \n\n- **Plan Commonsense Evaluation (Line 170)**: The LLM is prompted to act as a verifier, reading the action sequence and identifying steps that violate physical or commonsense logic. For example, it successfully flags errors like trying to pick an object that is still occluded, attempting to stack an object on itself `(stack b1 b1)`, or applying an action to an invalid object type `(open_drawer blue_plushie)`.  \n\n- **Domain Fusion Pairing (Line 203)**: In the current implementation, the child domains are paired randomly for merging along the binary tree. Optimizing the fusion order is a promising direction for future work.  \n\nTo further ensure reproducibility, we commit to **open‑source the UniDomain project**, including releasing the unified domain, the automatic retrieval, domain fusion, and online planning components, together with the final paper.\n\n---\n\n**Q: The paper provides very limited examples of generated operators and predicates. This makes it difficult to assess their quality. ... Can the authors provide the full list of generated operators?** \n\n**A:** Due to the 10k character limit, we can only present a minimal sample here. We will open‑source the entire unified domain file in the final paper.\n\n```pddl\n(:action fold_sheet\n    :parameters (?r ?s)\n    :precondition (and (holding ?r ?s) (sheet ?s) (not (folded ?s)))\n    :effect (and (folded ?s) (hand_free ?r) (not (holding ?r ?s)))\n)\n(:action cut_donut\n    :parameters (?r ?tool ?donut ?t)\n    :precondition (and (holding ?r ?tool) (on_table ?donut ?t) (table ?t) (can_cut_donut ?tool) (not (is_cut ?donut)))\n    :effect (and (is_cut ?donut))\n)\n(:action hang_cloth\n    :parameters (?r ?c ?obj)\n    :precondition (and (holding ?r ?c) (cloth ?c))\n    :effect (and (hung ?c ?obj) (hand_free ?r) (not (holding ?r ?c)))\n)\n(:action pair_shoes\n    :parameters (?r ?rs ?ls)\n    :precondition (and (holding ?r ?rs) (on_floor ?ls) (right_shoe ?rs) (left_shoe ?ls))\n    :effect (and (on_floor ?rs) (hand_free ?r) (not (holding ?r ?rs)) (paired ?ls ?rs))\n)\n```\n---\n\n**Q: The paper misses an opportunity to analyze the quality of the generated domains in a systematic way. It would be valuable to examine common failure cases...** \n\n**A:** This is an excellent suggestion. We have analyzed the failures and can categorize them as follows:\n\n- **Domain Learning Failure Modes**  \n  - *Syntax Errors*: generates a syntactically invalid PDDL file (e.g., mismatched brackets) that the planner cannot parse.  \n  - *Missing Operators*: fails to generate a critical operator needed to solve a test problem (e.g., generating `pick` but not `place`).  \n  - *Logical Violations*: contains logical errors, such as an effect that conflicts with a precondition in the same operator.  \n  - *Flawed Test Problems*: generates a test problem with an impossible goal state, such as `(and (is_open door) (not (is_open door)))`.  \n\n- **Online Planning Failure Modes**  \n  - *Goal Misinterpretation*: The VLM misunderstands the goal, for example, by reversing a required stacking order (top‑down vs. bottom‑up).  \n  - *Visual Grounding Errors*: The VLM misidentifies an object's state or location in the initial scene, leading to an incorrect `(:init)` block and planning failure.  \n  - *Domain Inherited Errors*: The planner generates a infeasible plan due to flaws in the learned domain itself (e.g., missing preconditions).\n\n---\n\n**Q: What is the computational cost and inference time for constructing the unified domain?** \n\n**A:** Generating a single atomic domain takes, on average, `5.27` calls to GPT‑4.1 (at `8.36 s/call`) and `4.47` calls to GPT‑o3-mini (at `74.51 s/call`). Note that, constructing the full unified domain from `12,393` demonstrations was a one‑time, offline cost. We will release the full unified domain in the final paper.\n\n---\n\n**Q: What are the 100 real-world task instances used in evaluation? ... Which operators are frequently reused, and which ones are rarely needed?** \n\n**A:** \nHere we include a representative set of 5 task instructions due to space constraints:  \n\n* \"Arrange all blocks into two separate stacks on the table. The first stack should have blocks green, red, blue, and yellow in order from top to bottom. The second stack should have blocks 1, 2, 3, and 4 in order from top to bottom.\" \n* \"There is a paper in the orange drawer. There is a towel in the yellow drawer. Please scrunch the paper on the table and wipe the table\" \n* \"There is a spoon in the cup. Please stir the two bowls and then put the spoon in the left cup\"  \n* \"There are a spoon, a tissue, an orange block in the green drawer. Stir the bowl and put the spoon in the cup, put the orange block into the orange drawer, wipe the bowl and scrunch the tissue on the table\"  \n* \"There is an unfolded towel in the yellow drawer. Wipe the table, wipe the bowl, and fold the towel on the table. Push the purple block, slide the blue block and press the white remote\"  \n\nBelow is a quantitative analysis of operator activations during online planning, across the four evaluation domains.\n\n| Domain      | # Operators | 3 Most-Frequent Operators (# Activations)                  | 3 Least-Frequent Operators (# Activations)             |\n| :---------- | :---------: | :--------------------------------------------------------- | :----------------------------------------------------- |\n| BlockWorld  |      4      | unstack (153), stack (143), put_on_table (84)              | pick_from_table (74)                                   |\n| Desktop     |     11      | open_drawer (30), pick_from_drawer (25), put_on_table (15) | fold_on_table (5), push_from_table (5), scrunch_on_table (5) |\n| Kitchen     |     12      | put_on_table (20), pick_from_rack (15), pick_from_bowl (15) | place_on_rack (5), remove_lid (5), fold_on_table (5)   |\n| Combination |     22      | pick_from_drawer (95), place_in_drawer (85), unstack (83)  | stir_bowl (5), slide_from_table (5), press_remote (5)  |\n|  |  |  |  |"}, {"review_id": "1X2QYjJUef", "reviewer": "Reviewer_cBvW", "summary": "This paper proposes a method for creating a large unified PDDL domain from real robot demonstrations. The domain generation process leverages VLMs and LLMs to ground predicates and to verify the syntax and solvability of the PDDL. Experiments in four domains compare the proposed approach to several baselines and ablations.", "questions": "1. Is there an easy way to resolve my concerns about non-executable task plans?  \n2. Can you provide more information about the procedure used by the human evaluators?  \n3. Can you clarify why predicate and operator filtering improves success rates?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "### Strengths\n\n**Novelty.** The paper does seem quite novel. I am not aware of other work that attempts to construct a large unified PDDL domain from real robot demonstrations and I really like that idea.\n\n**Baselines and ablations.** The baselines and ablations in experiments are fairly comprehensive (although I have concerns about the meaning of the metrics, see below).\n\n**Well-written papers.** I have a few concerns related to transparency (see below), but otherwise the paper is well-written and easy to read. I did not notice many sentence-level issues. I think the paper would be accessible to a broad audience.\n\n**Related work.** The related work section is thorough and the connections between UniDomain and previous work are clear.\n\n### Weaknesses\n\n**Non-executable task plans.** My biggest concern is that the plans produced by UniDomain are not executable, i.e., there is no way to directly map the task plans to actions. This seems like a fundamental limitation with several implications:\n\n* From a user’s perspective, UniDomain does not seem like it is ready for use. I don’t see a way that I could create a new task / domain and actually solve it with UniDomain, unless I am willing to manually map the outputs to executable robot actions, which may be very difficult or impossible.  \n* From an evaluation perspective, all of the reported metrics are more subjective than they initially appear. My understanding is that all of the metrics ultimately depend on a human judge. (This is buried in Appendix C: “We therefore reported results based on human evaluation to ensure their reliability.”) The optimality / cost metrics are particularly fraught: for example, what prevents every task from being solved with a single long-horizon operator? Or many small operators? There is arguably no ground-truth cost here, but the paper implies that there is.  \n* From a presentation perspective, the paper is not at all transparent about this limitation. For example, Figure 2(f) (“Execution”) and the robot videos heavily imply that UniDomain is solving the tasks shown (e.g., “UniDomain solves complex, unseen tasks in a zero-shot manner”), when that is not really the case.\n\n**Technical details.**\n\n* Not all robot plans can be easily expressed in PDDL. It is not clear what happens when UniDomain attempts to synthesize PDDL for a demonstration that really shouldn’t use it. From the paper’s description, it sounds like there is a potentially infinite loop that repeats until a domain is synthesized. It may be better to automatically determine if a domain is PDDL-able first.  \n* Fast Downward already performs “predicate and operator filtering” and does so in a way that maintains solvability. I’m confused about the predicate and operator filtering proposed here and how it apparently leads to improved task success rates (Figure 5). I would have thought that removing predicates and operators could only improve planning time, not success rate.  \n* The third claimed contribution is a “novel online task planner”, but I don’t think there is actually novelty in the planner.\n\n**Code promised but not delivered.** The paper indicates that code is available at [https://unidomain.github.io/](https://unidomain.github.io/) but it was not there during my review. I was also hoping that the final unified domain itself would be provided, but it was not.\n\n**Size of unified domain.** The paper emphasizes in several places that 12,393 demonstrations are used from DROID, leading to 3,137 operators, 170 semantic categories, 2,875 predicates, and 16,481 causal edges. These numbers are a bit misleading in my opinion because what ultimately matters is the size of the merged domain. Many of the operators and predicates in the original set are redundant and it would be easy to further inflate the reported numbers by simply copying theo original demonstrations. Furthermore, in experiments, it says: “40 DROID demonstrations are retrieved to construct a meta-domain for all 254 evaluation tasks, which includes 78 predicates and 61 operators.” It then refers to Appendix A.2, which says: “For planning evaluation, we retrieve and fuse 40 kitchen-related atomic domains from this unified set to form a compact meta-domain with 106 predicates, 61 operators, and 332 causal edges.” So there is some conflicting information about the number of predicates. Also, at the end of the day, only a subset of UniDomain has an empirical evaluation, so it’s not yet clear if there is value to the entire set.", "was_revised": true, "final_justification": "I am very borderline for the reasons mentioned in my review and follow up discussions, but I ultimately raised my score because the authors promise to clarify limitations in the writing.", "rebuttal": "We thank the reviewer for the thoughtful and detailed review. We appreciate the positive feedback on the novelty, clarity, and thoroughness of the paper. The reviewer raises several important concerns, primarily regarding plan executability, evaluation methodology, and implementation transparency. In this response, we will address each in detail to clarify our framework's intended use, justify our evaluation, and provide the missing technical details.\n\n---\n\n### **1. On Usability of the Approach and Objectiveness of Results**\n\n**Q: From a user’s perspective, UniDomain does not seem like it is ready for use... unless I am willing to manually map the outputs to executable robot actions.**  \n**A:** We would like to clarify UniDomain's role within a complete robotics system. UniDomain is designed as a **high-level task planner**, and its separation from low-level motor control is a deliberate feature that enables a **generalizable and scalable** robot system. This approach is not a limitation but a strength, creating a powerful division of labor.\n\nThe intended integration is automated, not manual:\n\n- UniDomain solves the long-horizon reasoning problem, producing a plan of symbolic operators like `(pick_from_table robot spoon table)`.\n- Each symbolic operator is straightforwardly translated into a natural-language command, e.g., \"Pick up the spoon from the table.\"\n- This command is then passed to any modern, language-conditioned Vision-Language-Action (VLA) model, which executes the physical motion.\n\nCrucially, UniDomain is designed for direct compatibility with this paradigm. Our symbolic operators are learned from the **same real-world demonstration data** used to train state-of-the-art VLAs, ensuring seamless integration. This allows the system to leverage the best of both worlds: UniDomain handles the **\"what to do\"** (strategic planning and reasoning), while VLAs handle the **\"how to do it\"** (dexterous motor execution).\n\n---\n\n**Q: From an evaluation perspective, all of the reported metrics are more subjective than they initially appear... ultimately depend on a human judge.**  \n**A:** We respectfully disagree with the characterization of our evaluation as subjective. To make our evaluation as objective and scalable as possible, we primarily used an **LLM‑as‑a‑judge**, with human verification to ensure reliability. The vast majority of failures are not subjective but are clear violations of physical or commonsense constraints that both an LLM and a human can easily identify. Common, objective failure cases include:\n\n- Attempting to place an object into a bowl that is still on a vertical drying rack.  \n- Attempting to pick an object from a stack without first removing the objects on top of it.  \n- Attempting to interact with a container (e.g., drawer, pot) without first opening it.  \n\nThe table below reveals high correlation between LLM-only judgement results and our reported numbers:\n\n| Method | LLM Score Mean ↑ | LLM Score Std | Human Score |\n| --- | --- | --- | --- |\n| VLM‑CoT | 38.11 | 2.20 | 40 |\n| Code‑as‑Policies | 51.33 | 2.24 | 51 |\n| ISR‑LLM | 52.89 | 2.32 | 52 |\n| **UniDomain** | **84.13** | **2.17** | **85** |\n\nFor complete transparency, **we will release all raw evaluation records**, including the generated plans and the LLM and human judgments, in the final version of the paper.\n\n---\n\n**Q: The optimality / cost metrics are particularly fraught... What prevents every task from being solved with a single long‑horizon operator? ... There is no ground-truth cost...**  \n**A:** \nWhile we use unit costs for simplicity, we argue that this cost design is valid and our framework readily supports more general cost functions.\n\n- **Validity of Unit Costs:** We ensure a consistent semantic granularity (one primitive skill per operator) through our domain‑learning process:\nEnergy‑based keyframe extraction identifies transition points between primitive skills, ensuring that long-horizon behaviors are chopped into primitive skills; \nIf a single skill is split across multiple keyframe pairs, our VLM‑based domain constructor (guided by prompt design) maps them to the **same operator**, preventing fragmentation.\n\n- **Supporting Better Costs:** Since we learn from real demonstrations, we can easily estimate costs like execution time (duration between keyframes), trajectory length, or motion energy from data. A PDDL solver can then directly use these non-unit costs to find a truly optimal plan.\n\n---\n\n**Q: Figure 2(f) (“Execution”) and the robot videos heavily imply that UniDomain is solving the tasks shown...**  \n**A:** Figure 2(f) and the videos show the physical execution of the **task plan generated by UniDomain**. It is a standard methodology in task planning research to isolate the evaluation to the high-level reasoning capability, without confounding results from an imperfect low-level controller. Following this standard, for our evaluation of both UniDomain and all baselines, the low-level execution was performed via human teleoperation, representing a perfect low-level policy. UniDomain does indeed solve the complex planning tasks in a zero-shot manner; the teleoperation only enacts the solution. We will clarify this in the final manuscript.\n\n---\n\n### **2. On Technical Details and Other Concerns**\n\n**Q: Not all robot plans can be easily expressed in PDDL... what happens when UniDomain attempts to synthesize PDDL for a demonstration that really shouldn’t use it?**  \n**A:** We agree that PDDL is best suited for problems with clear semantic structure, which is the focus of UniDomain (high-level task planning). For demonstrations that are not \"PDDL-able\" (e.g., continuous drawing, playing), we can detect this upfront by checking if the video's energy curve lacks distinct extrema or if the language instruction is vague (e.g., \"drawing stuff\", \"playing around\"). Furthermore, the closed-loop refinement is not an infinite loop; it is capped at L=5 iterations. If it fails to converge, which is rare (see Figure 4a), the demonstration can be discarded.\n\n---\n\n**Q: I’m confused about the predicate and operator filtering proposed here and how it apparently leads to improved task success rates...**  \n**A:** There is a potential misunderstanding. The filtering is **not for the PDDL solver**, but for the **LLM-based problem generator**. A large meta-domain can overwhelm an LLM, making it difficult to ground the specific task and generate a correct problem file. Our two-stage process uses a first pass to identify a small, task-relevant subset of predicates and operators. The second pass then uses this filtered, minimal domain to generate the final problem. This focused context significantly improves the reliability and accuracy of the LLM's problem generation, which in turn increases the final planning success rate by 11%, as shown in the ablation in Figure 5.\n\n---\n\n**Q: The third claimed contribution is a “novel online task planner”, but I don’t think there is actually novelty...**  \n**A:** The novelty lies specifically in the **two-stage problem generation process** described above. Using a first pass of problem generation as a filtering mechanism to create a minimal context for final problem generation is, to our knowledge, a new technique for improving the reliability of LLM/VLM-based PDDL planners. Figure 5 shows this is not a minor trick; removing it causes a significant drop, by 11%, in performance.\n\n\n---\n\n**Q: Size of unified domain… operators and predicates are redundant... conflicting predicate counts… only a subset evaluated.**  \n**A:**  \n\n- **Redundancy Control:** We took significant care to minimize redundancy in the unified domain. Our prompt designs enforce strict naming conventions for operators and predicates, causing functionally similar elements to merge into a single node in the knowledge graph. This process reduced an initial set of over 24,199 potential operators to the 3,137 unique ones reported.\n- **Predicate Count:** The \"`106` predicates\" in the appendix refers to the total nodes in the graph, including negated predicates (e.g., `(is_open door)` and `(not (is_open door))`). The \"`78` predicates\" in the main text refers to the number of unique predicate names. We will make this consistent.\n- **Value of the Full Set:** We have now conducted new experiments with a fully automated pipeline that retrieves relevant domains from the **entire** unified set. For a given task class, an LLM infers relevant actions from the instruction. We then use sentence-embedding similarity to find the top-k best-matching operators in our unified domain. The atomic domains containing these matching operators are then selected and passed to our Domain Fusion pipeline to construct a meta-domain for online planning. This fully-automated approach acheives **83%** success and **80%** optimality (K=0) on the existing evaluation set, **99%** success and **99%** optimality (K=0) on 10 new, diverse task classes, providing strong empirical evidence for the value of the entire knowledge graph.\n\n---\n\n**Q: Code promised but not delivered.**  \n**A:** We apologize for this oversight in the submitted manuscript. The statement and link regarding code availability were included in error during the paper's preparation. We are fully committed to reproducibility and will open-source the project, including the learned unified domain, the automated retrieval and domain fusion pipelines, and the online planner, with the final version of the paper.\n\n---\n\n*Thank you again for the constructive feedback; we believe these clarifications and new results strengthen the paper substantially.*"}]

Author rebuttal:

Meta-review: This submission proposes a method that automatically constructs a unified PDDL domain from large-scale robot manipulation demonstrations to enable generalizable task planning. The approach consists of three stages: domain pretraining using keyframe extraction and closed-loop verification with VLMs/LLMs to learn atomic domains from individual videos; domain fusion to merge these into a coherent meta-domain; and online planning, which grounds new tasks into PDDL problems for a symbolic planner. Evaluated across 100 tasks in four domains, the method demonstrates strong zero-shot performance, outperforming LLM-based and LLM-PDDL baselines in both task success rate and plan optimality. This submission received two borderline rejects, one weak accept and one accept. After rebuttal, one of the reviewers upgrades his / her score from borderline reject to borderline accept. The reviewers recognized the merits of this proposed method, However， some concerns are still remains. The authors are encouraged to address these concerns clearly in their final version.
Decision: Accept (poster)