Title: UniDomain: Pretraining a Unified PDDL Domain from Real-World Demonstrations for Generalizable Robot Task Planning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Robotics

Task Planning

Symbolic Planning

Vision Language Models

Robotic Knowledge Graph
Abstract: Robotic task planning in real-world environments requires reasoning over implicit constraints from language and vision. While LLMs and VLMs offer strong priors, they struggle with long-horizon structure and symbolic grounding. Existing meth-
ods that combine LLMs with symbolic planning often rely on handcrafted or narrow domains, limiting generalization. We propose UniDomain, a framework that pre-trains a PDDL domain from robot manipulation demonstrations and applies it for online robotic task planning. It extracts atomic domains from 12,393 manipulation videos to form a unified domain with 3137 operators, 2875 predicates, and 16481 causal edges. Given a target class of tasks, it retrieves relevant atomics from the unified domain and systematically fuses them into high-quality meta-domains for zero-shot planning. Experiments on diverse real-world tasks show that UniDomain solves complex, unseen tasks in a zero-shot manner, achieving up to 58% higher task success and 160% improvement in plan optimality over state-of-the-art LLM and LLM-PDDL baselines.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method for creating a large unified PDDL domain from real robot demonstrations. The domain generation process leverages VLMs and LLMs to ground predicates and to verify the syntax and solvability of the PDDL. Experiments in four domains compare the proposed approach to several baselines and ablations.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n**Novelty.** The paper does seem quite novel. I am not aware of other work that attempts to construct a large unified PDDL domain from real robot demonstrations and I really like that idea.\n\n**Baselines and ablations.** The baselines and ablations in experiments are fairly comprehensive (although I have concerns about the meaning of the metrics, see below).\n\n**Well-written papers.** I have a few concerns related to transparency (see below), but otherwise the paper is well-written and easy to read. I did not notice many sentence-level issues. I think the paper would be accessible to a broad audience.\n\n**Related work.** The related work section is thorough and the connections between UniDomain and previous work are clear.\n\n### Weaknesses\n\n**Non-executable task plans.** My biggest concern is that the plans produced by UniDomain are not executable, i.e., there is no way to directly map the task plans to actions. This seems like a fundamental limitation with several implications:\n\n* From a user’s perspective, UniDomain does not seem like it is ready for use. I don’t see a way that I could create a new task / domain and actually solve it with UniDomain, unless I am willing to manually map the outputs to executable robot actions, which may be very difficult or impossible.  \n* From an evaluation perspective, all of the reported metrics are more subjective than they initially appear. My understanding is that all of the metrics ultimately depend on a human judge. (This is buried in Appendix C: “We therefore reported results based on human evaluation to ensure their reliability.”) The optimality / cost metrics are particularly fraught: for example, what prevents every task from being solved with a single long-horizon operator? Or many small operators? There is arguably no ground-truth cost here, but the paper implies that there is.  \n* From a presentation perspective, the paper is not at all transparent about this limitation. For example, Figure 2(f) (“Execution”) and the robot videos heavily imply that UniDomain is solving the tasks shown (e.g., “UniDomain solves complex, unseen tasks in a zero-shot manner”), when that is not really the case.\n\n**Technical details.**\n\n* Not all robot plans can be easily expressed in PDDL. It is not clear what happens when UniDomain attempts to synthesize PDDL for a demonstration that really shouldn’t use it. From the paper’s description, it sounds like there is a potentially infinite loop that repeats until a domain is synthesized. It may be better to automatically determine if a domain is PDDL-able first.  \n* Fast Downward already performs “predicate and operator filtering” and does so in a way that maintains solvability. I’m confused about the predicate and operator filtering proposed here and how it apparently leads to improved task success rates (Figure 5). I would have thought that removing predicates and operators could only improve planning time, not success rate.  \n* The third claimed contribution is a “novel online task planner”, but I don’t think there is actually novelty in the planner.\n\n**Code promised but not delivered.** The paper indicates that code is available at [https://unidomain.github.io/](https://unidomain.github.io/) but it was not there during my review. I was also hoping that the final unified domain itself would be provided, but it was not.\n\n**Size of unified domain.** The paper emphasizes in several places that 12,393 demonstrations are used from DROID, leading to 3,137 operators, 170 semantic categories, 2,875 predicates, and 16,481 causal edges. These numbers are a bit misleading in my opinion because what ultimately matters is the size of the merged domain. Many of the operators and predicates in the original set are redundant and it would be easy to further inflate the reported numbers by simply copying theo original demonstrations. Furthermore, in experiments, it says: “40 DROID demonstrations are retrieved to construct a meta-domain for all 254 evaluation tasks, which includes 78 predicates and 61 operators.” It then refers to Appendix A.2, which says: “For planning evaluation, we retrieve and fuse 40 kitchen-related atomic domains from this unified set to form a compact meta-domain with 106 predicates, 61 operators, and 332 causal edges.” So there is some conflicting information about the number of predicates. Also, at the end of the day, only a subset of UniDomain has an empirical evaluation, so it’s not yet clear if there is value to the entire set.\n\nQUESTIONS:\n1. Is there an easy way to resolve my concerns about non-executable task plans?  \n2. Can you provide more information about the procedure used by the human evaluators?  \n3. Can you clarify why predicate and operator filtering improves success rates?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a framework that constructs a unified PDDL domain from a large-scale dataset of robotic manipulation demonstrations. Instead of relying on manually designed symbolic domains, the proposed method builds a single domain containing over 3,000 operators and more than 2,000 predicates. The framework operates in three stages. First, in the domain pretraining stage, atomic PDDL domains are learned from individual demonstrations using energy-based keyframe extraction and iterative refinement with LLMs. Second, in the domain fusion stage, relevant atomic domains are hierarchically merged into a meta-domain based on text and semantic similarity. Third, during deployment, task-relevant operators and predicates are filtered from the meta-domain to generate executable plans using a symbolic planner. The framework is evaluated on 100 tasks across four domains, demonstrating improved performance compared to existing methods that use LLMs or VLMs either alone or in combination with PDDL.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. While prior work has explored the integration of LLMs and VLMs with structured representations such as PDDL, this paper is unique in its attempt to learn a unified domain from a large collection of demonstration videos.\n2. The paper introduces several novel techniques to build and refine the unified domain, including the use of LLMs and PDDL planners in a verification loop, grouping of predicates and operators using text embeddings and LLMs, and pruning mechanisms to remove irrelevant elements from the domain for planning.\n\nWeaknesses:\n1. Although the paper claims that a unified domain with over 3,000 operators is constructed, the quality of this domain is not directly evaluated. Instead, the experiments use a smaller domain learned from only 40 videos.\n2. The paper is clearly written and well organized, but many implementation details are missing. These missing details, including in the supplementary material, limit the reproducibility of the method. For example:\n- In Line 151, what is the size of the sliding window used for keyframe detection?\n- In Line 161, how accurate is the LLM at generating test problems from a given predicate set and task instruction? What specific feedback is returned by the planner?\n- In Line 170, how does the LLM evaluate whether a plan satisfies physical constraints and commonsense expectations?\n- In Line 203, how are domain pairs selected for merging? Are the child domains chosen randomly?\n3. The paper provides very limited examples of generated operators and predicates. This makes it difficult to assess their quality.\n4. The paper misses an opportunity to analyze the quality of the generated domains in a systematic way. It would be valuable to examine common failure cases, such as preconditions or effects that violate physical constraints, incorrect groupings of predicates and operators, missing or redundant predicates in operators, and mismatches in abstraction levels (such as grounded versus lifted operators).\n\nQUESTIONS:\nBesides the questions listed above, I have the following questions:\t\n1. What is the computational cost and inference time for constructing the unified domain?\n2. Can the authors provide the full list of generated operators?\n3. What are the 100 real-world task instances used in evaluation? What operators are required to complete each of them? Which operators are frequently reused, and which ones are rarely needed?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose learning prior knowledge about PDDL domains required for solving robotics tasks from robot videos, and then utilizing that prior knowledge to perform better zero-shot planning in new environments. The authors show how the proposed method outperforms baselines in several real-world tasks, zero-shot solving planning problems that other methods struggle with.\n\nSTRENGTHS AND WEAKNESSES:\nWhile the proposed approach might work well and would certainly be valuable if that is the case, there is almost no information in the paper to be able to clearly judge that.\n- It is unclear why a demonstration can be well segmented by simply looking at the total grayscale intensity in each frame. There is also no evaluation done on this in isolation to show that that is actually the case.\n- Not a single example of a domain for a task generated at test time is given, or an example of a generated plan. It is unclear what the approach qualitatively does based on its prior knowledge to outperform the baselines.\n- On a high level, it is unclear what the approach actually learns to enable it to perform better than the other approaches. Is it just a large set of preconditions for each action that are checked, where the VLM without prior knowledge might otherwise miss it?\n- While different domain variations and combinations get the total number of evaluation tasks to 100, they are based on only 3 relatively simple domains, covering a very small set of concepts. The majority of it seems to come down to: stacking blocks in order, putting things into bowls / onto plates / into drawers / onto dish racks, and performing table wiping.\n- Retrieval of relevant atomic domains being done manually seems like a significant downside. While it might be easy to overcome, it is not done so in the current version of the paper.\n\nQUESTIONS:\nNo additional questions.\n\nLIMITATIONS:\nThe authors discuss the limitations of the approach. On the surface, this seems sufficient, but without more information to understand the internals of the approach better, it is difficult to say whether there are any additional ones.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces UniDomain, a framework that extracts a unified PDDL domain from real-world robot manipulation demonstrations for generalizable task planning. This framework includes three parts: (1) Domain pretraining: extracts atomic domains from demonstration videos using keyframe segmentation and refines them via closed-loop verification with VLMs and LLMs; (2) Domain fusion: merges relevant predicate and operator; and (3) Online planning: constructs grounded PDDL problems for specific tasks and solves them using a symbolic planner. Experiments show it achieves significantly higher task success and plan optimality compared to LLM and LLM-PDDL baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper addresses a critical challenge in planning: the reliance on complete and manually constructed domain knowledge. This is a non-trivial bottleneck, as manually engineering domain models is both time-consuming and error-prone. The proposed approach of automatically constructing a unified domain from demonstrations presents a valuable direction for reducing manual effort and improving scalability.\n2. The proposed UniDomain framework demonstrates good performance across four task domains. The paper also includes comprehensive ablation studies, which effectively validate the contribution of individual components within the framework.\n\nWeaknesses:\n1. The unified domain is constructed from 40 manually selected demonstrations, raising concerns about the robustness and generalizability of the approach. It remains unclear whether alternative selections of demonstrations would produce similar results. Moreover, the potential impact of using the full DROID dataset—where a larger set of predicates and operators are present—has not been explored. Specifically, the paper does not discuss whether the increased domain complexity would affect the quality of the domain and problem files generated by the LLM during online planning, and whether this would result in a performance drop.\n2. The observation that the \"w/o grouping\" variant outperforms the full method in the Kitchen domain is counterintuitive and warrants further analysis. The lack of an explanation for this result makes it difficult to understand the trade-offs involved in the grouping component.\n3. The related work section overlooks prior efforts on domain generation from demonstrations. In particular, the following works are relevant and should be discussed:\n[1] Automated Generation of Robotic Planning Domains from Observations, IROS 2021.\n[2] Automated Planning Domain Inference for Task and Motion Planning, ICRA 2025.\n4. The paper does not clearly specify the control policy used for executing actions in the real-world experiments. Clarifying this aspect is important to assess the reproducibility and the completeness of the proposed system.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}