Title: Compositional Monte Carlo Tree Diffusion for Extendable Planning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Diffusion

MCTS

Long-term Planning

Offline RL

Goal-conditioned RL

Stitching sub-plans
Abstract: Monte Carlo Tree Diffusion (MCTD) integrates diffusion models with structured tree search to enable effective trajectory exploration through stepwise reasoning. However, MCTD remains fundamentally limited by training trajectory lengths. While periodic replanning allows plan concatenation for longer plan generation, the planning process remains locally confined, as MCTD searches within individual trajectories without access to global context. We propose Compositional Monte Carlo Tree Diffusion (C-MCTD), a framework that elevates planning from individual trajectory optimization to reasoning over complete plan compositions. C-MCTD introduces three complementary components: (1) Online Composer, which performs globally-aware planning by searching across entire plan compositions; (2) Distributed Composer, which reduces search complexity through parallel exploration from multiple starting points; and (3) Preplan Composer, which accelerates inference by leveraging cached plan graphs.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work tackles long-horizon planning tasks, applying methods trained on shorter planning datasets (e.g., upto 100 steps) to test environments that requires longer planning (e.g., 1000 steps). The authors proposed three \"stitching\" methods to extend planning ability of existing work (Monte Carlo Tree Diffusion). Specifically, the author introduces 1) Online-Composer which performs online MCTS over subplans to extend planning horizon; 2) Distributed Composer to speed up 1); and 3) Preplan Composer which utilizes offline plans to bootstrap online search performance. The authors evaluated their methods against other baselines in various environments in the OGBench, and showed improvements in long-horizon planning against other methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n- This paper proposed novel inference-time stitching methods to enables long-horizon planning beyond the length seen during \"training\".\n- The authors experimented with multiple tasks in the OGBench and showed their methods improves generalization in long horizon environments.\n\nWeaknesses:\n1. Planning runtime is only compared between the three methods proposed in this work, but not against other prior work. These should be reported in Table 1 along with other methods to provide a more transparent comparison.\n2. The best method (Preplan composer) takes significant runtime (mentioned in 228-229) as it involves an additional offline planning phase. This limits generalization ability as 1) the test environment may not be always accessible a priori; and 2) it is unclear how much offline planning is \"enough\".\n3. Preplan composer shows strong performance on giant mazes in Table 1 by utilizing plans composed offline in the same environment. Wouldn't this make comparisons against other planners unfair, since I assume all other planners only had access to small mazes (up to 100steps) during training? Without preplan composer, Table 1 shows that CompDiffuser (a prior work) achieved the best performance compared to the proposed methods.\n\nQUESTIONS:\nQuestion:\n- Why are baselines different in different experiments? Table 1 included baselines such as CompDiffuser, but they are not included in Table 3 and Table 4?\n\nTypos:\n- In L106 the subscripts 1:s and s+1:S are not formatted correctly\n- in L205-206 \"ParTS-MCTD\" is a new method name?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper try to address the limited planning horizon issue of MCTD framework by adding hierarchical planning strategy that stitch the plans in various different ways. It achieved substantially better performance or efficiency in out-of-domain long-horizon maze navigation tasks compared with Diffuser and MCTD.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n\n1. The overall performance is substantially strong in long-horizon maze navigation tasks compared with MCTD and Diffuser. \n2. The overall framework seems to be general. The paper applied the idea in both the maze navigation and robot arm cube manipulation. \n\n**Weaknesses**\n\n1. The algorithm could become quite complex due to the inherent computational cost of MCTS. In the worst case, UCT has an exponential complexity of the form $\\Omega(\\exp(\\exp(\\exp(...\\exp(1)))))$ (depth -1 times of $\\exp$) [1], and the actual worst-case performance could be even more prohibitive in practice when composing multiple UCT instances and diffusion process.\n2. If understand correctly, the algorithm seems to tackle the offline planning problems. However, MCTS is a online planning algorithm because of its anytime property and effectivenness in online MDP/POMDP (POMCP or DESPOT for POMDP). Given the essential slow runtime of diffusion, the proposed method could be very limited in real-time or low-latency planning scenarios.\n3. The paper might need a more comprehensive discussion of practical application scenarios where the proposed C-MCTD method offers a distinct advantage. For example, determinisitic pointmaze navigation could be tackled effectively by A* search. As for POMDP, there are also mature algorithms such as POMCP and DESPOT. It would strengthen the paper to include a discussion on when and why C-MCTD should be preferred over these traditional approaches, and under what conditions it significantly outperforms them.\n\n[1] Qoquelin & Munos, Bandit Algorithms for Tree Search, UAI 2007\n\nQUESTIONS:\n* One of the most important question is what is the formulation of the problem that C-MCTD works the best among the Diffuser, MCTD, traditional A* or MCTS algorithms such as UCT, PUCT, GumbelMuzero, etc. Where could we apply this algorithm in practice? What is the supporting evidence? \n* If applicable, I am curious about the performance of A* search with manhattan distance heuristics in pointmaze problem, including the runtime and successrate. \n* Looks like the MCTD and C-MCTD assume there is no need to use transition function to simulate the actual state changes for action execution? Are the environments in the experiment deterministic in transition? Will there be any impact if the environment has a highly random transition function? \n* In POMDP tasks, are you conducting the search offline as well and replan online? Is the low-performance is caused by no transition function's simulation, so that the gap between actual execution and plan is big?\n\nLIMITATIONS:\nI think in Discussion section, the author compare the limitations of different variants of C-MCTD, but did not actually discuss the limitation of C-MCTD compared with MCTD, Diffuser, and traditonal search algorithms. This should be included with supporting evidence to better define the scope of the application of C-MCTD.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work presents an extension of the Monte Carlo Tree Diffusion (MCTD) approach for long-horizon planning, termed C-MCTD. The proposed method enables the construction of long-horizon plans by effectively \"stitching\" together relatively short plans generated by MCTD. The authors introduce three strategies to achieve this—online, distributed, and preplan—and apply them to long-horizon maze navigation and multi-cube robot arm manipulation tasks.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- Divide-and-conquer approach for long-horizon tasks: This work builds upon the idea of MCTD by proposing a method that enables planning over longer horizons than the trajectory lengths supported by MCTD. By introducing and comparing three different composition strategies in C-MCTD, the authors allow for flexible selection of an appropriate strategy depending on the task requirements.\n- Clear explanation of some key algorithmic components: The differences among the proposed composition strategies are well explained, particularly in terms of the components that distinguish each approach. The inclusion of algorithmic details in the appendix and the release of open-source code contribute to the reproducibility of the proposed method.\n\n### Weaknesses\n- Shallow analysis of candidate tree origin position proposal : While the proposed method extends MCTD by composing multiple sub-trajectories via tree search—similar to hierarchical RL—it lacks in-depth discussion on the generation and maintenance of subgoals, which are crucial in hierarchical planning. The method treats tree origin positions as subgoals but does not provide substantial analysis on how these subgoals should be selected or maintained. In particular, for the distributed and preplan composers, subgoals are either randomly sampled or derived from previous trajectories, without further discussion on their semantic significance or strategic role.\n- Lack of clarity regarding the use and definition of the guidance set: Despite the frequent use of the term \"guidance\" throughout the manuscript and appendix, there is a lack of concrete mathematical formulation or detailed explanation regarding how the guidance set is actually utilized. A more explicit clarification—either in the main text or at least in the appendix—is necessary.\n- Insufficient discussion of the preplan composer: The authors claim that the preplan composer achieves 100% accuracy, which is not necessarily an incorrect observation. However, the emphasis on this result raises questions about the true contribution of the preplan composer. Without deeper analysis on what kinds of preplans are effective for long-horizon or multi-stage tasks—and how these preplans contribute to improved efficiency or accuracy—the method risks being perceived as a simple concatenation of ground-truth trajectory fragments.\n- Limited empirical validation of the proposed method: The model is only evaluated on two tasks (maze navigation and cube arrangement). While the results on these tasks demonstrate some level of robustness, the general applicability of the method remains uncertain. Additional experiments across diverse environments would be necessary to demonstrate that the proposed method is not overly specialized to these specific scenarios.\n- Lack of visualization for experimental results: The only visual comparison presented is the conceptual illustration in Figure 1. To better understand the behavior and differences between the proposed composition strategies, it would be helpful to include visualizations of the actual trajectories generated in the experiments.\n\nQUESTIONS:\n- How is the guidance set defined and implemented in the proposed method? Despite frequent references to \"guidance\" in the main text and appendix, its precise formulation and implementation remain unclear.\n- What is the size of the subgoal set, and how are subgoals sampled during planning? Is randomly scattering subgoals a valid strategy for complex long-horizon tasks, or could this limit planning efficiency and interpretability?\n- Could the authors provide further details on the hyperparameters used in the experiments? For instance, how was the epsilon threshold for stitching trajectories determined, and was the method robust to smaller epsilon values during execution?\n- (minor) spacing in Table 3 : Online Composer\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

comments: SUMMARY:
This paper proposes Compositional Monte Carlo Tree Diffusion (C-MCTD), a  framework that extend MCTD's planning capabilities beyond training trajectory lengths through three variants, online Composer, Distributed Composer, and Preplan Composer. 

The key idea is to stitch multiple short-horizon plans together using tree search at the plan level rather than the subplan level. Online Composer performs sequential plan stitching with hierarchical tree search, Distributed Composer uses parallel trees with strategic connections, and Preplan Composer leverages prebuilt plan graphs. The authors evaluate on maze navigation, robot manipulation, and visual planning tasks, showing improvements over replanning baselines.

STRENGTHS AND WEAKNESSES:
1. The paper addresses a real limitation of MCTD - its inability to generate plans longer than training trajectories. The hierarchical approach of searching over plans rather than subplans is reasonable.

2.The core contribution lacks novelty. Plan stitching is well-established, but,,,  the paper essentially applies existing tree search techniques at a higher level.
3.The experimental evaluation has several issues. The baselines are mostly weak replanning methods rather than SoTA long-horizon planners. The comparison lacks recent methods that directly address training length limitations. Results are inconsistent across environments - Distributed Composer sometimes outperforms Online Composer, sometimes doesn't, with no clear explanation.

QUESTIONS:
Why not compare against recent methods that specifically address training length limitations, such as hierarchical planning approaches or other compositional methods? The current baselines seem weak.

LIMITATIONS:
n/a
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03

comments: SUMMARY:
Building upon MCTD, the authors propose the C-MCTD framework, which incorporates three complementary strategies—Online, Distributed, and Preplan Composer—to perform compositional search over multiple short-horizon plans during the inference stage. This enables the generation of long-horizon trajectories exceeding 10× the training horizon. Experiments across OGBench’s Point-/Ant-Maze, vision-based mazes, and robotic block manipulation tasks demonstrate significant performance improvements over baselines such as MCTD-Replan and CompDiffuser.

STRENGTHS AND WEAKNESSES:
Strength:
The paper extends the MCTD to composable inference-time reasoning, allowing long-horizon plan generation without retraining. 

Weakness:
The paper lacks ablation studies on key design components of the Composers—for instance, the Fast Replanning and Plan-Cache modules in the Online Composer.

QUESTIONS:
- the paper only reports inference-time runtime (Table 2), how long does the offline graph construction take?
- the author states that the Online Composer relies on mechanisms such as guidance sets and fast replanning, but their specific contributions to performance are not demonstrated.

LIMITATIONS:
yes (section 5.4)
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-04