Title: RDD: Retrieval-Based Demonstration Decomposer for Planner Alignment in Long-Horizon Tasks
Venue: NeurIPS 2025
Accepted: yes
Keywords: Sub-task Discovery

Robotic Learning
Abstract: To tackle long-horizon tasks, recent hierarchical vision-language-action (VLAs) frameworks employ vision-language model (VLM)-based planners to decompose complex manipulation tasks into simpler sub-tasks that low-level visuomotor policies can handle. Typically, the VLM planner needs finetuning to learn to decompose a new task, which requires target task demonstrations segmented into sub-tasks by either human annotation or heuristic rules. However, without prior knowledge, the heuristic sub-tasks can deviate significantly from the visuomotor policy's training data, thereby degrading task performance. To address these issues, we propose a Retrieval-based Demonstration Decomposer (RDD) that automatically decomposes video demonstrations into sub-tasks with prior by aligning the visual features of the decomposed sub-task intervals with those from the training data of the low-level visuomotor policies. RDD outperforms the state-of-the-art sub-task decomposer on both simulation and real-world tasks, demonstrating robustness across diverse settings. Code and more results are available at https://rdd-neurips.github.io


Peer reviews:
comments: SUMMARY:
This work introduced RDD, which can decompose demonstrations into sub-tasks without relying on human annotation or heuristics rules. RDD aligns the visual features of sub-task intervals with the training data of visuomotor policy. The author provides detailed demo video and code, which significantly enhances the reproducibility of the proposed method. RDD achieves good performance on RLBench and robustness.

STRENGTHS AND WEAKNESSES:
Strength: 
1. The author provides detailed demo video and code, which significantly enhances the reproducibility of the proposed method.
2. The author conduct ablations and present detailed analysis for the proposed method.
3. The paper is well-written.

Weaknesses:
1. How does the RDD perform compared to the decomposer based on code-generating LLM/VLMs, e.g., instruct2act [1], robotics programmer [2] and Code as policies [3].

[1] Huang, Siyuan, et al. "Instruct2act: Mapping multi-modality instructions to robotic actions with large language model." arXiv preprint arXiv:2305.11176 (2023).

[2] Xie, Senwei, et al. "Robotic Programmer: Video Instructed Policy Code Generation for Robotic Manipulation." arXiv preprint arXiv:2501.04268 (2025).

[3] Liang, Jacky, et al. "Code as policies: Language model programs for embodied control." ICRA, 2023.

QUESTIONS:
See Strengths And Weaknesses

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-06-23

comments: SUMMARY:
This paper proposes a method to decompose the demonstration into "sub-tasks" to improve the learning of VLA models. The authors formulate the decomposition as an optimal partition problem, where the partition objective relates to the learning data of low-level policies. The decomposition method is applied to a baseline hierarchical VLA and tested on 18 RLBench tasks for evaluation.

STRENGTHS AND WEAKNESSES:
[+] The overall idea is well-motivated and shares a lot of insights with recent hierarchical VLA models, where we can use long-horizon task demonstrations to facilitate the learning of plan and action, and also perhaps task augmentation.

[+] The overall description of the framework is clear and illustrative, with major points of the design easy and clear to follow.

[-] One major concern lies in the marginal improvement of this method to the tasks selected in the evaluation phase. The current result is a bit hard to validate the effectiveness of the current method, especially given several ties between the proposed and the Uniform or Heuristic methods.

[-] Another point is that I do feel the current framework can be extended to form many more interesting attempts and analyses, e.g., cross-domain or sim2real transfer (as the current decomposition should ideally support skill reuse). The current discussion on only RLBench is a bit narrow in my opinion, especially given the marginal improvement.

QUESTIONS:
See the strength and weakness section.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper identifies a critical alignment issue between high-level planners and low-level visuomotor policies in hierarchical vision-language-action (VLA) agents. To address this, the authors propose a Retrieval-based Demonstration Decomposer (RDD), a novel training-free method to automatically segment long-horizon demonstrations for planner fine-tuning. The core mechanism of RDD is to formulate the task as an optimal partitioning problem, which is solved by retrieving visually and temporally similar sub-tasks from the low-level policy's training data to guide the decomposition.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The paper tackles a highly relevant and significant problem in hierarchical robot learning. Ensuring that the high-level planner generates instructions that the low-level policy can reliably execute is crucial for the success of long-horizon tasks. The proposed direction of aligning the planner's fine-tuning data with the policy's existing capabilities is a valuable contribution to the field.\n2. The central concept of RDD is elegant and highly intuitive. Instead of relying on hand-engineered heuristics or expensive human annotations, RDD leverages the policy's own training data as a \"template\" for what constitutes a good, executable sub-task. \n3. The experimental evaluation is comprehensive, featuring strong baselines on a standard benchmark and extensive ablations that validate key design choices and demonstrate the method's effectiveness.\n\nWeakness\n1. The sub-task representation, using only start and end frames, is simplistic. It discards crucial intermediate dynamics and may fail to distinguish between sub-tasks with different motions but similar boundary states.\n2. The technical novelty is somewhat limited. The method is primarily a novel application and combination of existing techniques (dynamic programming, visual similarity search) rather than a fundamental algorithmic contribution.\n3. The retrieval-based nature of RDD raises questions about its robustness to novel, out-of-distribution sub-tasks. The paper lacks discussion and experiments for scenarios where a new demonstration contains actions not well-represented in the training database.\n\nQUESTIONS:\n1. Given RDD's reliance on the D_aug_train database, how does its performance depend on the quality and style of the upstream heuristic used to create that database? Is RDD robust to different or lower-quality source heuristics?\n2. What is the expected behavior of RDD when encountering a truly novel sub-task not present in its retrieval database? Have you considered fallback mechanisms for such low-confidence retrieval scenarios?\n3. To better isolate the contribution, have you considered a more direct evaluation of decomposition quality itself？\n\nLIMITATIONS:\nYes, the authors have acknowledged the existence of a limitations section in the appendix. However, given the significance of certain limitations (e.g., the reliance on the quality of the policy's training data), it would improve the paper to briefly discuss the most critical ones in the main text for better transparency.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces RDD (Retrieval-based Demonstration Decomposer), a training-free method for decomposing long-horizon robotic demonstrations into sub-tasks for finetuning hierarchical Vision-Language-Action (VLA) models. The key insight is that existing decomposition methods (human annotation or heuristics) often generate sub-tasks that deviate from the training data of low-level visuomotor policies, leading to suboptimal performance. RDD addresses this by formulating demonstration decomposition as an optimal partitioning problem that explicitly aligns decomposed sub-tasks with the visuomotor policy's training set through visual feature retrieval. The method uses dynamic programming to efficiently solve the optimization problem and demonstrates 6.7% improvement over state-of-the-art methods on 18 RLBench tasks.\n\nSTRENGTHS AND WEAKNESSES:\n=== Strengths ===\n\n1. The paper identifies an important gap in hierarchical VLAs - the misalignment between planner training data and visuomotor policy capabilities. This is a practically relevant insight that is overlooked in prior work.\n2. The formulation as an optimal partitioning problem with interval-wise additive scoring is mathematically sound. The dynamic programming solution with O(N²) complexity is efficient, and the theoretical analysis (including Corollary 3.1.1) is rigorous.\n3. Using only start/end frames for interval representation (Eq. 3.5) is clever, avoiding variable-length representations while maintaining efficiency for nearest neighbor search.\n\n=== Weaknesses ===\n\n1. The method is only evaluated on RLBench in simulation. Other more challenging manipulation benchmarks (e.g., LIBERO) and real-world robotic manipulation often involve more complex visual variations, occlusions, and dynamics that may challenge the visual similarity-based retrieval approach.\n2. RDD requires access to the full training dataset of the visuomotor policy to build the retrieval database. This may not always be feasible, especially for proprietary or large-scale policies.\n3. The experiments show that performance degrades when the retrieval database is down-sampled to 25% of its original size. This suggests that a large and comprehensive database is necessary for good performance, which could pose a scalability challenge as the number of tasks and skills grows.\n\nQUESTIONS:\n1. How does RDD handle demonstrations containing sub-tasks that are fundamentally novel and have no close visual or temporal analogues in the policy's training set? Could you discuss this potential failure case and how it might impact the fine-tuning of the planner and overall task success? A change in my score would depend on a convincing explanation of how the system avoids catastrophic failure in such out-of-distribution scenarios.\n2. Your method's success relies on the quality of the pre-existing visuomotor training set. How sensitive is RDD to the quality of the sub-task segmentation within this dataset? For example, if $\\mathcal{D}^{train}_{aug}$ was segmented using a poor or noisy heuristic, would RDD simply learn to replicate this flawed strategy, or does the optimization process offer some resilience?\n3. Can you provide examples where RDD fails to decompose demonstrations correctly? Specifically, are there cases where visually similar intervals have very different semantic meanings that confuse retrieval?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}