Compositional Visual Planning via Inference-Time Diffusion Scaling
Authors: Yixin Zhang
Yunhao Luo
Utkarsh Aashu Mishra
Woo Chul Shin
Yongxin Chen
Danfei Xu
Venue: ICLR 2026 Poster (2026)
Keywords: Planning
Compositionality
Diffusion Models
Robotics
Primary area: applications to robotics, autonomy, planning
TL;DR: We introduce an inference-time compositional sampling approach that scales to unseen and long-horizon tasks.

Abstract: Diffusion models excel at short-horizon robot planning, yet scaling them to long-horizon tasks remains challenging due to computational constraints and limited training data. 
Existing compositional approaches stitch together short segments by separately denoising each component and averaging overlapping regions. However, this suffers from instability as the factorization assumption breaks down in noisy data space, leading to inconsistent global plans. We propose that the key to stable compositional generation lies in enforcing boundary agreement on the estimated clean data (Tweedie estimates) rather than on noisy intermediate states. Our method formulates long-horizon planning as inference over a chain-structured factor graph of overlapping video chunks, where pretrained short-horizon video diffusion models provide local priors. At inference time, we enforce boundary agreement through a novel combination of synchronous and asynchronous message passing that operates on Tweedie estimates, producing globally consistent guidance without requiring additional training. Our training-free framework demonstrates significant improvements over existing baselines, effectively generalizing to unseen start-goal combinations that were not present in the original training data. Project website: https://comp-visual-planning.github.io/

Published as a conference paper at ICLR 2026
COMPOSITIONAL VISUAL PLANNING VIA INFERENCE- TIME DIFFUSION SCALING
Yixin Zhang'; Yunhao Luo 12 Utkarsh A. Mishra', Woo Chul Shin',
Yongxin Chen', Danfei Xu'
Geeorgia Institute of Technology University of Michigan
ABSTRACT
Diffusion models excel at short-horizon robot planning, yet scaling them to long- horizon tasks remains challenging due to computational constraints and limited training data. Existing compositional approaches stitch together short segments by separately denoising each component and averaging overlapping regions. However, this suffers from instability as the factorization assumption breaks down in noisy data space, leading inconsistent global plans. We propose that the key stable compositional generation lies in enforcing boundary agreement on the estimated clean data (Tweedie estimates) rather than on noisy intermediate states. Our method formulates Long-horizon planning as inference over chain-structured factor graph of overlapping video chunks, where pretrained short-horizon video diffusion models provide local priors. At inference time, we enforce boundary agreement through a novel combination of synchronous and asynchronous message passing that operates Tweedie estimates, producing globally consistent guidance without requiring additional training. Our training-free framework demonstrates significant improvements start-goal combinations that were not present in the original training data. Project website: https: comp- vi sual -pl ann ng g thub. /o/
existing baselines, effectively generalizing to unseen
INTRODUCTION
x
X
X
<<)))
x²
Training as a Unified Factor
Inference as
Factor Graph
Start Boundary Variable
Goal Boundary Variable <<< >>>>>>>>>>>>>>> Transition Boundary Variable
Figure 1: Compositional Visual Planning via Inference Time Diffuser Scaling. We train short-horizon visual diffusion model on clips treated as a single factor. At inference, we scale visual planning horizon without retraining by chaining overlapping factors into a linear factor graph: the start and goal boundary variables are anchored at the ends, while neighboring factors exchange information through shared transition boundary variables.
Generative diffusion models have shown strong capacity for modeling complex, high-dimensional distributions over images, videos, and robot plans. In planning, they offer a compelling alternative per-instance optimization: instead of solving can sample likely solutions from learned generator. However, extending video-based planning to long horizons remains challenging: most backbones trained short clips, compute and memory
search problem for every start-goal pair,
"Correspondence yzhang4 799gattcc..e.

---

Published as a conference paper at ICLR 2026
scale unfavorably with sequence length, and long-range constraints (contacts, object persistence, and start-goal satisfaction) must be maintained throughout the rollout.
Classical planning methods, such as Task and Motion Planning, decompose tasks into structured subproblems and enforce constraint satisfaction through symbolic operators, while hierarchical control methods first solve for a high-level task plan and then refine it into a low-level motion plan. Compositional diffusion planning follows these core principles but provides a data-driven, proba- bilistic alternative to hand-engineered classical and hierarchical planners. We adopt compositional generation perspective on long-horin morizoorzzonn planning. we compose plans from overlapping, short-horizon factors produced by pretrained diffusion model. The central challenge is the consistency of this composition: during forward diffusion, noisy variables become entangled across time, breaking factorization assumptions behind common compositional heuristics (e.g., score averaging) (Zhang et al., 2023; Mishra et al., 2023; 2024; Bar-Tal et al., 2023) and yielding brittle behavior when long-range constraints must propagate.
Our key insight is to compose where diffusion model estimationn are most reliable: on their Tweedie estimates, which provide stable domain in which strong and explicit compositional heuristics be applied. We formulate planning as inference in a chain-structured factor graph over overlapping video chunks. Local priors come from a short-horizon diffusion backbone; global coherence is enforced by boundary agreement on Tweedie predictions, not on noisy states. We instantiate this approach for compositional visual diffusion planning, representing plan as sequence of images. Crucially, the method operates purely at inference time: the short-horizon diffusion backbone is trained once on short clips and then frozen; at test time we compose long-horizon plans via message passing on Tweedie estimates, with no additional training, fine-tuning, or task-specific adapters.
In summary, the key contributions of this work are: (1) A diffusion planning framework that models long-horizon plans as chain-structured factor graphs over video segments and enforces boundary agreement on Tweedie estimates rather than on noisy diffusion states. (2) Joint synchronous and asynchronous message passing over denoised variables, coupled with a training- free sampler that guides DDIM steps with diffusion-sphere guidance derived from messaage-psssing residuals, preserving local sample quality, parallelism, and diversity while enforcing boundary agreement. (3) compositional planning benchmark and empirical study demonstrating significant improvements in temporal coherence, static quality, and task success held-out start-goal combinations compared with prior compositional baselines (Zhang et al., 2023) that operate on noisy diffusion states.
2 RELATED WORK
Diffusion Models For Planning. A flurry of work has leveraged diffusion models (Sohl-Dickstein et al., 2015; Ho et al., 2020) for planning (Janner et al., 2022; Ajay et al., 2022; Dong et al., 2024a; He et al., 2023a; Ubukata et al., 2024; Lu et al., 2025; Chen et al., 2024a; Liang et al., 2023; Dong et al., 2024b), with various applications such as path finding (Carvalho et al., 2025; Luo et al., 2024), roboties (Pearce et al., 2023; Fang al., 2024), and multi-agent (Zhu al., 2024; Shaoul al., 2024). Performance can be further improved by increasing test- fest-time compute such as tree search (Feng et al., 2024a; Yoon et al., 2025), hierarchical planning (Li et al., 2023: Chen et al., 2024b), and post-hoc refinement (Lee et al.. 2024; Wang et al., 2022). Despite strong results, most prior work studies diffusion planning in dow-dimesiional state spaces that yield 2D trajectories. While recent work (Xu et al., 2025; Xie et al., 2025; Huang et al., 2024a) begins to consider more complex state spaces, they typically target simple or task-specific scenarios. In this paper, we investigate visual diffussio plamning, where a plan is represented as a sequence of images, and we introduce training-fre free sampling method that scales to significantly longer horizons and to unseen start-goal combinations.
Compositional Diffusion Generation. Compositional diffusion models are now well studied (Du et al., 2020; Garipov et al., 2023; Du & Kaelbling, 2024; Mahajan et al., 2024; Okawa et al., 2024; Thornton et al., 2025). One thread develops samplers for logical conjunctions of conditions, combining multiple prompts or constraints into coherent generations (Liu et al., 2022; Bradley et al., 2025; Zhang et al., 2025; Yang et al., 2023). A complementary thread scales the number of inference- time tokens while reusing models trained on short horizons yielding wide-field panoramas (Zhang et al., 2023; Bar-Tal et al., 2023; Kim et al., 2024a; Lee et al., 2023), longer-duration videos (Wang et al., 2023; Kim et al., 2024b; 2025), and extended-horizon robotic plans (Zhang et al., 2023; Mishra et al., 2023; Luo et al., 2025). However, existing compositional methods suffer from instability
2

---

Published as a conference paper at ICLR 2026
when applied to noisy diffusion states, they typically rely on score averaging other heuristic combinations that assume factorization holds throughout the denoising process. In contrast, our approach operates clean Tweedie estimates rather than noisy intermediate states, formulates the problem as factor graph inference with explicit boundary constraints, and employs principled message passing to maintain global consistency- yieeldigg substantial improvements in both stability and plan quality over prior compositional planning methods.
Inference-Tinee Time Guidance for Diffussin Model. Inference-time guidance steers diffusion sampling without retraining, enabling adaptive, controllable behavior at test time. This flexibility has driven progress in image restoration (inpainting, deblurring) (Chung et al., 2024; Yang et al., 2024; Yu et al., 2023; Ye et al., 2024; Song et al., 2023a), style transfer (Bansal et al., 2023; He et al., 2023b), and robot motion/behavior generation (Liao al., 2025; Black et al., 2025; Du & Song, 2025; Song et al., 2023b; Feng et al., 2024b). However, most existing methods steers fixed-length output, we frame guidance as form of message passing between the sequence. This perspective lets us stitch together short behavioral fragments into long-range, temporally consistent visual plan.
information propagate across
3 PRELIMINARIES
3.1 FACTOR GRAPH FORMULATION FOR COMPOSITIONAL DISTRIBUTIONS
A factor graph
[u' u² u"] is bipartite graph connecting factor nodes {x²) yizı and variable nodes {u} where x C [u' u² ...., u²") An undirected edge between æ² and exists if and only if u² E x². Given a factor graph that represents the factorization of joint distribution, previous works DiffCollage upproximate with Bethe approximation (Zhang et al., 2023), and Generative Skill Chaining (GSC) extends the same formulation to robot task-and-motion planning as its follow-up work:
p(2)) = IT p(a))
(1)
IT p(u)) yd
where dy is the degree of each variable uj. Therefore, the estimated score is base Bethe approxi- mation is:
11
m -0-
V., logg(z)) = Σ at log
mn log(u)).
J=1 For example, consider a linear chain [ut u² uz u u²] with factors xı [ut u² w²)l and 22 = [u", u u²). We can represent the joint noisy distribution as:
1=1
p(z,) p(ut, ui: u²) puu). ut up) p(u))
Given the linear chain graph, the overlapping variable 2 (the one shared between neighboring factors Ij and x2)) has degree d = 2, while the non-overlapping ones have d = 1 (i.c., their distribution come from individual factors). However, Bethe approximation Eq. 1) holds in clean data (i.e., diffusion timestep f 0), but does not hold when > 0, we further prove its gap later (Theorem 1).
3.2 DIFFUSION MODEL AND TRAINING- FREE GUIDED DIFFUSION
Diffusion Models are a class of generative model that generates sample in the desired distribution from initial Gaussian distribution p(æT) by iteratively performing a denoising process. It has a pre-defined forward process q( (r)) = N(xx;; va,ao. (1 ()) where & is a scalar dependent on diffusion timestep t. In this work, we directly estimate Tweedie when training:
E. [ - no (2 t)] (20o predictor) and apply a DDIM step when sampling:
2 - = Vārzott + V1- ā. - of Tt - Var
+ +,Ct
(4)
VI az
where estimated Tweedie a the output of xg (2 t).
Classifier Guidance (Dhariwal & Nichol, 2021) proposes to train a time dependent classifier in conditional generative tasks. Specifically, the conditional distribution p(zxly) can be modeled by
3

---

Published as a conference paper at ICLR 2026
Vz, logg(xx/))) Vz. log)(2;) 22, logp(y mr). where y represents the condition or measurement. This paper focuses on conditional guidance in training-free ffree manner, all the guidance in this paper is in training-free manner. In training- free guidance setting, instead of explicitly training a classifier, the guidance term p(y x.) can be modeled potential function exp (-I(2))))) which simplifies to gradient-descent update during inference time:
Bayes Rules p(aa1y) (((x))p(ylz))
v. log m4) = Vz, log exp (o (-L(1))))) -
(5)
Z
where Z0 = x f) estimated by Tweedie's ' Predictor. Since exp (zo(())))) is point estimation of distribution of l (ro==p(zo(zz) (eapp///o///e/(((eeee_e (xxp(-L(ro)))) and the gap between them has upper bound (Chung al., 2024) and lower bound (Yang al., 2024), Diffusion sphere guidance (Yang et al., 2024) is proposed to eliminate this gap based by formulating constrained optimization problem over a
hypersphere with the mean to be v VāzToj: + vI - āg - 0² 21 Voy and radius (sar((s is the shape of v,), and derive a closed form solution for the update
vI-as
Vazzolt
V Zx (zoot)
I1-1
V1 at 0²
(6)
VI αi
V
(
4 METHOD
We formulate long-horizon planning as inference over chain-structured factor graph of overlap- ping video chunks, where pretrained short-horizon diffusion models provide local priors. Our key innovation is enforeing boundary agreement on estimated clean data (Tweedie estimates) rather than noisy intermediate states, addressing the core limitation that factorization assumptions break down during diffusion sampling. We achieve this through novel synchronous and asynchronous message passing that operates on Tweedie estimates, producing globally consistent guidance without additional training. The approach involves formulating the planning problem as a factor graph (Section 4.1), deriving its distribution (Section 4.2), and sampling via message passing scheme (Section 4.3).
4.1 PROBLEM FORMULATION
While a diffusion model can learn a prior over short, local behaviors, long- Long-horizon planning requires additional structure to ensure feasibility. Beyond satisfying the start and the goal, intermediate pieces must stitch with local consistency. We therefore train a short-horizon diffusion model Tg on local task segments; test time, given start image and goal image, we sample from Gaussian prior, partition the trajectory into overlapping chunks, generate each chunk with æg. and compose them into a coherent plan that finally mapped back to an action sequence through inverse dynamies model.
We represent the plan as a linear chain z = [u'....,,""" x² 2 1 2i
u=' and place n overlapping factors 22+1] i= I,..., n, each collecting three consecutive frames. The endpoints serve as the start and goal boundary variables. Let A; and B; denote linear
2 = and nm selectors that extract the first and last frames of factor z'. respectively. The feasibility of a plan is enforced by the following boundary agreements:
(Start/Goal Anchoring) Ais =8,
B 9:
(Transition Boundary) Bar A
This factorization reduces global planning to local generation with explicit boundary equalities and scales by reusing the same local model 2g across time while preserving consistency via start-goal anchoring and transition agreements. All factors/variable in latent space encoded by the Cosmos tokenizer (et. al., 2025) into a compact latent representation; we perform planning entirely this latent space rather than pixels, which reduces dimensionality to save compute.
4.2 DISTRIBUTION OF FACTOR GRAPH
Prior work (Zhang et al., 2023; Mishra et al., 2023; 2024) relies on the Bethe-style product of factors normalized by variables (Eq. 1), which is accurate on clean data. Forward diffusion, however, perturbs factorization assumption between factors and variables.
4

---

Published as a conference paper at ICLR 2026
Theorem 1 (Noisy-Bethe Gap Theorem). Consider linear chain [u', u² u²] with pairwise factors [u' [u',²²) and [uL, ,²), where u² is the transition boundary variable. As- sume the forward noising processes are plu,, un u' ),)). p(u, u. u² w²), and p(u u²). Let a(u²) = pp(u' u "p)p(,,, ut u², ) blu²) = p(uu u pp(pp uy u² due, c(u²) = p(u pplu))((ttt (u²), Z Z= ffd(uu du?, and q(u²) = c(u²)/ Denote by p(u), uf) the true noisy distribution and by plut, ui, ui) the estimator from Eq. Then the gap between true distribution and estimated distribution is:
(8) Interpretation. The proof is in appendix A. We can view a(u²) as the left-factor message into the boundary u²² b(u²) as the fight-factor message into the boundary, and c(u²) as the local boundary evidence. Intuitively, a(u²) and b(u²) quantify how the left and right pairwise factors, after passing through their respective nowward-noise channels, "vote" for different boundary values u². The term e(u²) provides the unary baseline that captures how plausible each boundary value on its own (and how it transmits noise). The noisy-Bethe gap A is exactly the covariance-unde the boundary weighting g g-between the two relative gains uncorrelated (or proportional) across u², the covariance vanishes and the Bethe approximation remains accurate. Forward diffusion typically introduces shared, heteroscedastic distortions u?, which make the two gains rise and fall together; this produces nonzero covariance and, consequently, a systematic gap.
A p(ul un. a.)) M(u) uZ.
Cov
[a
[
and b(u²)/ When these gains are
Figure 2 illustrates the core failure mode of DiffCollage which is de- ployed based noisy factorization assumption. A DiffCollage/GSC- style stitcher (middle) drifts and leaves boundary gaps, while our inference-time message passing (right) aligns shared boundaries and closes the loops. This mirrors the limitation of a Bethe-style product of factors (Eq. 1) under forward diffusion: noise corrupts factorization assumption. Motivated by the Noisy- Bethe gap, instead of enforcing dependencies directly among the noisy factors z!'", impose them on the concatenated Tweedie (denoised) estimates =by. factor-graph distribution is:
DiSCallaooOCC
Ouns
Motivating toy example. We train a short-horizon diffu-- sion model on circular are clips (left). At test time, three 120 are generators are composed to form a three-petal "flower".
Figure 2:
Accordingly, our approximation to the
p(=,) = p(a)) exp(-
where exp(-I(x)))) acts potential that penalizes inconsistencies- and thus enforces depen- dencies-amonp the estimated clean variables oofe
I=n x (x)")))
4.3 JOINTLY SYNCHRONOUS AND ASYNCHRONOUS MESSAGE PASSING
Message passing proceeds through boundary factors: when the transition boundaries together with the start and goal boundaries agree, the plan is feasible (see Eq. 7). We therefore optimize boundary agreement explicitly. Our synchronous scheme treats the chain as a Gaussian lincar system and via parallel updates, but can be numerically stiff. Our asynchronous scheme uses one-sided, stop-gradient targets propagate constraints forward and backward in a TD-style manner, yielding faster and more stable convergence at the cost of mild bias. Finally, diffusion-sphere guidance interpolates between unconditional sampling and loss-driven descent, balancing alignment and diversity.
drives single residual (E (E-">:lin
==b/ = 17)
4.3.1 SYNCHRONOUS MESSAGE PASSING
We encode the boundary condition as a Gaussian potential, 14-1, = exp(- B (12), where - denotes the variance.
5

---

Published as a conference paper at ICLR 2026
Proposition 1 (Synchronous Message Passing Constraint). Let x Waxtehon denote the concatenated intermediate factors in a shain-structured factor graph with transition boundaries V1-1,i- Given the start boundary Vs,1 and the goal boundary Vn.gr the joint constraints distribution over all intermediate factors is Gaussian:
Psyno
1:n
8, 9)
O exp( (1-2((x111
2 + 11 x 1 (
? --
(10)
A A1
B B1
B A2
A
00
B1
A Aa
B Ba
BI
6g
0
where E-1=
0
AG B2
AI As
B5 Ba
17
B
The detailed proof is in appendix B. We perform synchronous message passing the estimated
1:n by penalizing the deviation from the consistent linear system E Tweedie factors oft practice, we set c; = for all
-- DOt = 17. In
=0,- 0,..., n.:
L Layne
Σ
I:n off
(11)
Here, synchronous refers to lockstep update schedule, in which all updates computed from the same current iterate and applied simultancously. This scheme preserves parallelism, eliminates order-dependent effects, and guides the Tweedie estimates toward satisfying the chain constraints.
4.3.2 ASYNCHRONOUS MESSAGE PASSING
While the synchronous objective is conceptually clean, the resulting hard consistency constraint is difficult to optimize and often exhibits slow or unstable convergence (Ortiz et al., 2021). To improve stability and speed, we adopt an asynchronous scheme with bootstrapped targets and stop-gradient, akin to temporal-difference updates Hansen et al. (2024); Li et al. (2025). Concretely, we optimize:
n-1
Lasyne
+ Σ
sg (B(())))))) 1 ++1
forward passing
(12)
n-
Baabie - sg(AA4 2 (1t + Braeje - 9/
+
backward passing
.++ is target produced by diffusion model with latest where sg(-)is the stop-gradient operator, oof parameters, and s is produced by an EMA of the model parameters. The discount y down-weights messages as they move away from the start or the goal.
A A and BB==) of anchor the chain to the start and goal. The forward message loss penalizes mismatch between B, (outgoing) and A411 (incoming), with sg(.) enforcing one-way forward passing. Similarly, the backward message loss mirrors the same constraint in the reverse direction.
The boundary terms
4.3.3 DIFFUSION-SPHEES GUIDED MESSAGE PASSING
Having derived differentiable losses for synchronous and asynchronous message passing, we adopt the training-free guidance of DSG (Yang et al., 2024). As noted by DSG (Eq. 6), stronger guidance improves alignment but can reduce sample diversity. To balance alignment and exploration, we interpolate between the unconditional sampling direction and the normalized descent direction induced by our loss:
dm
dm = comple + (d' grample), x = Mr- +
(13)
|

---

Published as a conference paper at ICLR 2026
Algorithm 1 Compositional Generation
Require: Model EMA zg. Latest Model &
2: Hyperparameters: Diffusion Time Step T, number of factors chained n, guidance weight
3: Sample žr ~ I))
4: Split to overlapping chunks r//"
5: for = T do
6: "Ojt
forward model passing
7: G1mn
oft ig (at
VI āi 0 2 va
8: pi-1 = vātzojt
DDIM Step
vl
9: L Legne + Lasyne
Jointly syne and asyne message passing
V V.tn L
d= Voo.
Diffusion Sphere Guidance
11: dommpe Ofc
dm dsample + g(d deample)
TT-1 = - + +r |[dml]l
14: end for
merge chunks =/th get final plan zo
16: return 2g
Here deample = orE; is the unconditional annealing step, d' = - TV a
is the steepest
descent direction for our sync/async objective, and dm is subsequently normalized satisfy the apherical-Gausiaan constraint.
4.4 COMPOSITIONAL VIDEO PLANNING VIA INFERENCE- TIME DIFFUSION SCALING
The procedure for compositional generation is summarized in Algorithm 1. At high level. the goal is generate long-horizon trajectories by decomposing them into overlapping local factors and enforcing boundary agreement across those factors during the diffusion sampling process. At each timestep. DDIM provides the base update, while a joint synchronous aaachronous-ayynchooono message passing loss defines residual that Diffusion Sphere Guidance interpolates against, steering updates toward agreement without collapsing diversity. The resulting updates are local and parallel across overlapping factors yet collectively converge feasible, consistent plan. After all steps, merging the denoised chunks yields a smooth, temporally aligned trajectory 20-
For robot manipulation planning, we train a video diffusion model on randomly sampled short chunks from long-horizon demonstrations and an inverse dynamics model that predicts actions from consecutive frames. At test time, condition the diffuusin model on start and goal images and apply the compositional generation procedure to produce complete video plans. The resulting visual trajectory is converted into executable robot actions via the inverse dynamics model. The procedure training-fie free, plug-ands ang--ndd-pyy. and compatible with unconditional short-horizon diffusion backbones, enabling generalization unseen start-goal combinations without task-specific retraining.
5 EXPERIMENTS
We present experiment results on multiple robotic manipulation scenes spanning 100 tasks (18 in distribution, 82 out of distribution) of varying difficulties. Our objective is investigate (1) the visual fidelity of the generated video plans (Section 5. (2) how the proposed compositional visual planning can generalize
oong-horizon unseen tasks (Section 5.2) (3) how each proposed component affect the generation performance of our method (Section 5.3) (4) the effectiveness of our method in real robot manipulation tasks (Section 5.4).
We develop a simulation benchmark for compositional planning robotic manipulation based ManiSkill (Mu al., 2021), where each scene contains N start states and N goal states, resulting in N N tasks (i.e., different start/goal pairs) per scene, as shown in Appendix E.
Compositional Planning Benchmark.
Our training datasets only contains demonstrations for N start-goal pairs. At test time, we evaluate the planner on both the N seen start-goal pairs (in-distribution) and the remaining N N N
7

---

Published as a conference paper at ICLR 2026
unseen pairs dutt-o--distrbbtton)) since a capable planner should generalize new combinations the dataset covers all the required fragments. For example, in the Real World setting (Figure 3), demonstrations cover only the orange or blue regions; the planner must generalize across them to form cross-region plans unseen the dataset but composable from its fragments. We address this type of generation by learning from short demonstration chunks randomly taken from long-horizon tasks and compositionally generates multiple chunks at inference time to construct the final plan.
Figure 3: Real World Task Layout.Thi This task involves 2 start and 2 goal configurations. We also evaluate on challenging tasks. A complete list of task settings is provided in Appendix E.
Evaluation Setup. Given specified start and goal image as task context, our method first synthesize sequence of frames subgoals. We then use MLP-based inverse dynamics model to predict the pose of end-effector for the robot to execute, conditioned on adjacent images. The inverse dynamics models trained using the same demonstrations as the planner conditioned adjacent images. We report both the video quality metrics and the success rates the planners over the robotic manipulation tasks. An episode is counted as success if the target objects ends up the specified state within a small tolerance. For each environment, we report the success rate over all evaluation episodes. We evaluate all methods with 5 random seeds for each experiment and report the and standard deviation.
Baselines. For both bosecy and composition saaedd baseline, details are in Appendix D. For compositional baseline DiffCollage/GSC (where GSC is DiffCollage adapted to robotic planning), condition the start and goal images generate entire plan by noisy factorization Eq. 2, and then use an inverse dynamics model to execute that plan.
5.1 VIDEO GENERATION QUALITY STUDY
Beyond reporting task success rate, we also evaluate the visual fidelity of the synthesized video plans. Even when rollouts verify feasibility, perceptual quality still warrants careful analysis. Accordingly, we score generated videos with VBench++ (Huang et al., 2024b), focusing on robotics-centric metrics that matter for control: Dynamic Quality (inter-frame), which includes (i) motion smooth- ness capturinn temporal stability of robot/object motion-and (ii) background consistency- -esting whether the scene remains coherent over time; and Static Quality (frame-wise), which includes Aesthetic and Imaging Quality to ensure frames are clear and largely artifact-free. Our generation strategy substantially improves the time-dependent properties that matter for control: across all scenes and distributions, Motion Smoothness and Background Consistency far exceed DiffCollage (Zhang et al., 2023). This translates into dynamically executable trajectories and coherent spatiotemporal scenes. In terms of static quality, Aesthetic remains comparable, while Imaging shows a consistent and large advantage, directly reflecting fewer blurry frames and cleaner visuals.
8

---

Published as a conference paper at ICLR 2026
Dynamic Quality 1
Static Quality
Scene
Type
Motion Smoothness
Background Consistency
Aesthetic
Imaging
DiffCollage
Ours
DiffCollage
Ours
DiffCollage
Ours
DifiCollage
Ours
IND
0.88+0.04
0.94
0..04
0.90+6.04
0.400..0
0.50
0.60=o.06
0.7 0.70
Overall
OOD
0.87
0.97
0.80=a.07
0.90
0.4620.04
0.48
0.55=a.05
0.69 =0.00
Table Comparison across four scenes on Dynamic/Static Quality. Our results are averaged over 5 seeds and standard deviations are shown after the + sign.
5.2 COMPOSITIONAL PLANNING BENCHMARK
We present the robot manipulation success rates different escenes in Table 2. We separately report the success rates of IND and OOD tasks, where IND represents the N tasks seen in the training data while OOD represents the N N N unseen tasks. We observe that DiffCollage fails at almost all tasks. Qualitatively, we find that the synthesized images of DifiCollage tend be blurry or even unrealisite, perhaps due to its score averaging sampling scheme. Such suboptimal images will further confuses the inverse dynamic models, cause unstable behaviors and failures. In contrast, our method achieves significantly higher success rates, indicating that the generated visual plans are realistic and accurate for the inverse dynamic model follow. We also include several multiple representative policy learning baselines, such as strong policy learning baseline is able to perform well on IND tasks, their performance suffers from significant degradation on OOD tasks. In contrast, our method by the graphical chain formulation and message passing-min maintains stable performance regardless of the task distribution.
Diffusion Policy (GCDP). We notice that though
Scene
Type
LCBC
LCDP
GCBC
GCDP
DiffCollage
CompDiftuser
Ours
IND
8017
9512
851s
961a
1i2
60a
97..
Tool-Use
1513
371s
1316
4211a
0_e
5113
9612
IND
351s
541e
301s
50as
0.e
20a
53
Drawer
OOD
26.14
71s
18116
0.e
1841
52.
IND
28
58
26
60
0=
32#s
64-10
Cuhe
0OD
823
22412
5+5
24213
O=e
342
65
IND
2325
482s
1940
4723
1023
50_11
Puzzle
0OD
Ozo
1129
Ozo
12211
Ozo
923
50-13
IND
33418
57110
30
56216
O
172a
59
Overall
0OD
21
15112
15212
15113
1622
54_14
Table 2: Quantitative Results on Compositional Planning Bench. We benchmark our method on the 100 test-time tasks across scenes with 30 episodes per task. Our results are averaged over seeds and standard deviations are shown after the sign.
5.3 ABLATION STUDIES
Jointly Synchronous & Asynchronous Message Passing. We compare the success rates of three variants of our test-time compositional sampling scheme in Cube Scene: Only Synchronous Loss (Syne Only), Only Asynchronous Loss (Async Only), and Joint Synchronous and Asynchronous Loss (Sync & Async), as shown in Figure 4. Sync only suffers from overly tight constraints that are difficult to optimize, leading lower success rates. In contrast, the asynchronous variant performs better. Combining the two-Sync & soyyc--uuperformss either alone, likely due to its more effective balance of constraint enforcement and flexibility.
Scaling in Sampling Steps. We study how the number of diffusion sampling steps affects the planning performance Drawer Scene (Figure 5). Success rates improve the number of steps increases, demonstrating that our method scales effectively with additional test-time compute. We hypothesize that taking more steps enables deeper cross- cross-factor message passing through repeated

---

Published as a conference paper at ICLR 2026
denoising and guidance updates, which in turn reduces boundary inconsistencies and yields more accurate, temporally coherent plans.
Synchronous Asynchronous Message Passing,
Sculing Sumpling Steps.
Success Rate
Success Rate
Out-of-Distribetion
In-Dastribtiion
Out-of-Ditrrbution
Only
Asyne Only
100 200 Steps
300
Figure
Effect of synchronous and asyn- Figure 5: Efffet sampling steps on planning perfor- mance.
chronous message passing.
5.4 REAL ROBOT EXPERIMENT
For the real-world experiments, deploy method Franka Emika Panda robot, controlled 20Hz using joint impedance control. Visual image observations are captured using an Intel RealSense D435 depth camera. For data collection, the robot is teleoperated using Meta Quest 3 headset, with tracked Cartesian poses converted to joint configurations through inverse kinematics.
Figure 6: Hardware Setup.
We deploy our method on a Franka Emika Panda robot.
Real Scene
IND:Task1
IND:Task2
OOD:Task3
OOD:Task4
DiffCollage
1/10
1/10
0/10
0/10
Ours
9/10
7/10
10/10
8/10
Table 3: Real-robot success rates. Our method substantially outperforms DiffCollage across both in-distribution (IND) and out-of-distribution (OOD) tasks real hardware.
CONCLUSION
We introduced Compositional Visual Planning, an inference-time method that composes long-horizon plans by stitching overlapping video factors with message passing on Tweedie estimates. A chain- structured factor graph imposes global consistency, enforced via joint synchronous and asynchronous updates, while diffusion-spherr guidance balances alignment and diversity without retraining. Com- positional Visual Planning is plug-and-play with short-horizon diffusion video prediction model, scales with test-time compute, and generalizes to unseen start-goal combinations, Beyond roboties, the framework is applicable to broader domains, such panorama image generation and long-form text-to-vides synthesis, which we leave for future exploration.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We promise to provide all the source code to reproduce the results in this paper, including the proposed algorithm and the evaluation benchmark.
ACKNOWLEDGMENTS
This work was partially supported by the National Science Foundation under Awards NSF-2442393, NSF-2409016, and NSF-1142523, well by Samsung Research America. The views and conclusions expressed in this material are those of the authors and do not necessarily reflect the official policies or endorsements of the funding agencies. We would like to thank Shangzhe Li for helpful early discussion on trajectory stitching and feedback of the manuscript.
LLM USAGE DISCLOSURE
We confirm that no large language models (LLMs) were used to generate, edit, refine the scientific content, experimental design, results, or conclusions of this paper. All text, figures, algorithms, and analyses were produced entirely by the authors.
REFERENCES
Anurag Ajay. Yilun Du, Abhi Gupta, Joshua B Tenenbaum, Tommi S Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision making? The Eleventh International Conference on Learning Representations, 2022. 2
Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models, 2023. URL https:
//arxiv. orggabs// htt./bbs/22222 2302 200pb//11..........
07121.
Omer Bar,Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation, 2023. URL attps: // /arxiv org/ abss23022 08113.
Kevin Black, Manuel Y. Galliker, and Sergey Levine. Real-time execution of action chunking flow policies, 2025, URL https: org/ 225/ 07339.
Arwen Bradley, Preetum Nakkiran, David Berthelot, James Thornton, and Joshua M Susskind. Mechanisms of projective composition of diffusion models. arXiv preprint 0444445002.0444,, 2025. 2
Joao Carvalho, An T' Le, Piotr Kicki, Dorothea Koert, and Jan Peters. Motion planning diffusion: Learning and adapting robot motion planning with diffusion models. IEEE Transactions on Robotics, 2025. 2
Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. Shapenet: An information-rich 3d model repository, 2015. URL https: ptatpa://arxiv.... aaaii.iiiiv///ai///aaaaaaaaaavva org/ /bs/ 1512.00 03012.
Boyuan Chen, Diego Martí Monsó. Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems, 37:24081-24125 2024a.
Chang Chen, Fei Deng, Kenji Kawaguchi, Caglar Gulcehre, and Sungjin Ahn. Simple hierarchical planning with diffusion. arXiv preprint arXiv:2401.0266 2024b. 2
Hyungjin Chung, Jeongsol Kim, Michael T. Mccann, Mare I.. Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems, 2024. URL nttps://arxivv abs/2209. 14687. 3.4
Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis, 2021. URL https://arxiv wwwpabsssoo/////ooo/o/ooooooosss aat////s//1000 /210 05233.

---

Published as a conference paper at ICLR 2026
Zibin Dong, Jianye Hao, Yifu Yuan, Fei Ni, Yitian Wang, Pengyi Li, and Yan Zheng. Diffuserlite: Towards teml-time diffusion planning. Advances in Neural Information Processing Systems, 37: 122556-122583, 2024a. 2
Zibin Dong, Yifu Yuan, Jianye HAO, Fei Ni, Yao Mu, YAN ZHENG, Yujing Hu, Tangjie Lv, Changjie Fan. and Zhipeng Hu. Aligndiff: Aligning diverse human preferences via diffusion model. In The Twelfth International Conference on Learning Representations, 2024b.
Maximilian Du and Shuran Song. Dynaguide: Steering diffusion polices with active dynamic guidance, 2025. URL https::////... htttpp://arxxv.crrt http:///wrr/////////aaaaaaaaa///
abs 2506. 13922. 3
Yilun Du and Leslie Kaelbling. Compositional generative modeling: A single model is not all you
need. arXiv preprint arXiv:2402.01103, 2024. 2
Yilun Du, Shuang Li, and Igor Mordatch. Compositional visual generation with energy based models. Advances in Neural Information Processing Systems, 33:6637-6647, 2020. 2
NVIDIA et. al. Cosmos world foundation model platform for physical ai. arXiv preprint (rXiv:2501.03577 2025. 4
Xiaolin Fang, Caelan Reed Garrett, Clemens Eppner, Tomás Lozano-Pérez, Leslie Pack Kaelbling, and Dieter Fox. Dimsam: Diffusion models as samplers for task and motion planning under partial observability. In 2024 IEEF/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1412-1419. IEEE, 2024. 2
I Lang Feng, Pengjie Gu, Bo An, and Gang Pan. Resisting stochastic risks in diffusion planners with the trajectory aggregation tree. In Forty-first International Conference on Machine Learning.
2024a. 2
Zeyu Feng, Hao Luan, Pranav Goyal, and Harold Soh. Ltldog: Satisfying temporally-extendel symbolic constraints for safe diffusion-basee planning. IEEE Robotics and Automation Letters, doi: 10.11099ta.20244443350 URL http://dss doi. orgg 10. 10011119/1000 /R0. 2424.3443501.3
((10):8571-8778 October 2024b. ISSN 2377-3774.
Timur Garipov, Scbastiaan De Peuter, Ge Yang, Vikas Garg, Samuel Kaski, and Tommi Jaakkola. Compositional sculpting of iterative generative processes. arXiv preprint #xXiv23309.16133 2023.
Nicklas Hansen, Hao Su, and Xiaolong Wang. Td-mpc2: Scalable, robust world models for continuous control, 2024. URL http:://aaxvvviii.e httpr//aaab////31 16828.
Haoran He, Chenjia Bai, Kang Xu, Zhuoran Yang. Weinan Zhang, Dong Wang, Bin Zhao, and Xue- long Li. Diffusion model is an effective planner and data synthesizer for multi-task reinforcement learning. Advances in neural information processing systems, 36:64896 64917, 2023a. 2
Yutong He, Naoki Murata, Chich-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei- Hsiang Liao, Yuki Mitsufuji, J. Zico Kolter, Ruslan Salakhutdinov, and Stefano Ermon. Manifold preserving guided diffusion, 2023b. URL https: / /ttpiviivvvv////a///..////aaaav. org/ 2at1../21111.... 16424.
Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Shuaiyi Huang, Mara Levy, Zhenyu Jiang, Anima Anandkumar, Yuke Zhu, Linxi Fan, De-An Huang, and Abhinav Shrivastava. Ardup: Active region video diffusion for universal policies. In 2024 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pp. 8465-8472. IEEE, 2024a. 2
Ziqi Huang, Fan Zhang. Xiaojie Xu, Yinan He, Jiashuo Yu, Ziyue Dong, Qianli Ma, Nattapol Chanpaisit, Chenyang Si, Yuming Jiang, Yaohui Wang, Xinyuan Chen, Ying-Cong Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Vbench++: Comprehensive and versatile benchmark suite for video generative models, 2024b. URL https: /ttp///axvvvvvv wrg. abs/ /211l 11441.115555 13503.
12

---

Published as a conference paper at ICLR 2026
Michael Janner, Yilun Du, Joshua Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. In International Conference on Machine Learning, pp. 9902-9915. PMLR, 2022. 2
Jaihoon Kim, Juil Koo, Kyeongmin Yeo, and Minhyuk Sung. Synctweedies: A general generative framework based on synchronized diffusions, 2024a. URL 14370. 2
hwtp:/wwwwwwggrg//////aaaaaaaa// 2403.
Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. Advances in Neural Information Processing Systems, 37: 89834-89868, 2024b. 2
Subin Kim, Seoung Wug Oh, Jui-Hsien Wang. Joon- Joon-Young Lee, and Jinwoo Shin. Tuning-free multi- event long video generation via synchronized coupled sampling. arXiv preprint wcXiv:2503.0860)
2025. 2
Kyowoon Lee, Seongun Kim, and Jaesik Choi. Refining diffusion planner for reliable behavior synthesis by automatic detection of infeasible plans. Advances Neural Information Processing Systems, 36, 2024.
Yuseung Lee, Kunho Kim, Hyunjin Kim, and Minhyuk Sung. Syncdiffusion: Coherent montage via synchronized joint diffusions, 2023. URL https: //apiii.////xi....// atw.i.iiiiii///ee wot.vvorr////a. wttp:oorggabbbaooooooeeaeese 05178. 2
Shangzhe Li, Zhiao Huang, and Hao Su. Reward-free world models for online imitation learning, 2025. URL https: / arxivv. org/ abs /eee//21000 2224.1100000 6
Wenhao Li, Xiangfeng Wang, Bo Jin, and Hongyuan Zha. Hierarchical diffusiin for offline decision making. In International Conference on Machine Learning, pp. 20035-20064. PMLR, 2023. 2
Zhixuan Liang, Yao Mu, Mingyu Ding, Fei Ni, Masayoshi Tomizuka, and Ping Luo. Adaptdiffuser: Diffusion models as adaptive self-evolving planners. In International Conference on Machine PMLR, 2023.
Learning, pp.
Qiayuan Liao, Takara E. Truong, Xiaoyu Huang, Guy Tevet, Koushil Sreenath, and C. Karen Liu. Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion, 2025.
URL https: /awxiv.vviv//////i///a//////aavv org/ 22o.2 08241.
Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. In European Conference on Computer Vision, pp. 423-439. Springer, 2022. 2
Haofei Lu, Dongqi Han, Yifei Shen, and Dongsheng Li. What makes good diffusion planner for decision making? The Thirleenth International Conference on Learning Representations, 2025. 2
Yunhao Luo, Chen Sun, Joshua B Tenenbaum, and Yilun Du. Potential based diffusion motion planning. arXiv preprint arXiv:2407.06169 2024.
Yunhao Luo, Utkarsh Mishra, Yilun Du and Danfei Xu. Generative trajectory stitching through diffusion composition. arXiv preprint arXiv:2503.05/553 2025. 2
Divyat Mahajan, Mohammad Pezeshki, Ioannis Mitliagkas, Kartik Ahuja, and Pascal Vincent. Compositional risk minimization. arXiv preprint arXiv:2410.063003, 2024.
Utkarsh Aashu Mishra, Shangjie Xue, Yongxin Chen, and Danfei Xu. Generative skill chaining: Long- horizon skill planning with diffusion models. In Conference on Robot Learning, pp. 2905-2925. PMLR, 2023. 2, 4, 16
Utkarsh Aashu Mishra, Yongxin Chen, and Danfei Xu. Generative factor chaining: Coordinated manipulation with diffusion-based factor graph. In ICRA 2024 Workshop ('textemdash) Back to the Future: Robot Learning Going Probabilistic, 2024. 2, 4, 16
Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Yang, Xuanlin Li, Stone Tao, Zhiao Huang, Zhi- wei Jia, and Hao Su. Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations. arXiv preprint arXiv:2107. /4483, 2021. 7
13

---

Published as a conference paper at ICLR 2026
Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. In Robotics: Science and Systems (RSS), 2024. 19
Maya Okawa, Ekdeep S Lubana, Robert Dick, and Hidenori Tanaka. Compositional abilities emerge multiplicatively: Exploring diffusion models on synthetic task. Advances in Neural Information Processing Systems, 36, 2024.
Joseph Ortiz, Talfan Evans, and Andrew J. Davison. A visual introduction to gaussian belief propagation, 2021. URL / wttp:/wwwwwwgg//////////////////
6
Tim Pearce, Tabish Rashid, Anssi Kanervisto, Dave Bignell, Mingfei Sun, Raluca Georgescu, Sergio Valcarcel Macua, Shan Zheng Tan, Ida Momennejad, Katja Hofmann, et al. Imitating human behaviour with diffusion models. The Eleventh International Conference Learning Representations, 2023. 2
William Peebles and Saining Xie. Scalable diffusion models with transformers, 2023. URL https: //arxiv. otg. atc///bs////// 2212. 09748. 22
Yorai Shaoul, Itamar Mishani. Shivam Vats, Jiaoyang Li, and Maxim Likhachev. Multi-robot motion planning with diffusion models. arXiv preprint arXiv:2410.03072, 2024. 2
Jascha Sohl-Dickster Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pp. 2256-2265. PMLR, 2015. 2
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pscudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations, 2023a. URL /
Forum?id=$. Saa.iiddss_ssMMMRR.. 3
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-ple and-play controllable generation. In International Conference on Machine Learning (ICML). July 2023b.
James Thornton, Louis Bethune, Ruixiang Zhang, Arwen Bradley, Preetum Nakkiran, and Shuangfei Zhai. Composition and control with distilled energy diffusion models and sequential monte carlo. arXiv preprint arXiv:2502. 22255002.777666 /2786, 2025. 2
Toshihide Ubukata, Jialong Li, and Kenji Tei. Diffusion model for planning: systematic literature review. arXiv preprint arXiv:2408. 10266, 2024.
Yun Wang, Wenshuo Chen, Guanglu Song, Han-Jia Ye, Yu Liu, and Hongsheng Li. Gen-I-video: Multi-text to long video generation via temporal co-denoising. arXiv preprint 230.:22305.1864, 2023. 2
Zhendong Wang, Jonathan Hunt, and Mingyuan Zhou. Diffusion policies as expressive policy class for offline reinforcement learning. arXiv preprint arXiv:2208.061 06193, 2022.
Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, and Hao Su. SAPIEN: simulated bast- interactive environment. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2020. 19
Amber Xie, Oleh Rybkin, Dorsa Sadigh, and Chelsea Finn. Latent diffusion planning for imitation learning. arXiv preprint arXi::2504.16925 2025.
Zhengtong Xu, Qiang Qiu, and Yu She. Vilp: Imitation learning with latent video planning. IEEE Robotics and Automation Letters, 2025.
Lingxiao Yang, Shutong Ding, Yifan Cai, Jingyi Yu. Jingya Wang, and Ye Shi. Guidance with spherical gaussian constraint for conditional diffusion, 2024. URL htttps://axxiiiii //axxiv.ogg/ /202. 03201.350 3,
14

---

Published as a conference paper at ICLR 2026
Zhutian Yang, Jiayuan Mao, Yilun Du, Jiajun Wu, Joshua B Tenenbaum, Tomás Pezzno-Pérezz and Leslie Pack Kaelbling. Compositional diffusion- continuous constraint solvers. arXiv preprint arXiv:2339.0009000
2023. 2
Haotian Ye, Haowei Lin, Jiaqi Han, Minkai Xu, Sheng Liu, Yitao Liang, Jianzhu Ma, James Zou, and Stefano Ermon. Tfg: Unified training-free guidance for diffusion models, 2024. URL https:/ /atpiv//aaxvvvv//v org/ abs 2209240.... 15761. 3
Jaesik Yoon, Hyeonseo Cho, Doojin Baek, Yoshua Bengio, and Sungjin Ahn. Monte carlo tree diffusion for system 2 planning. arXiv preprint arXiv:2502.07202, 2025.
Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training-free ab303
energy-guided conditional diffusion model, 2023. URL 09833.
Jianrong Zhang, Hehe Fan, and Yi Yang. Energymogen: Compositional human motion generation with energy-based diffusion model in latent space. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 77592-17602 2025.
Qinsheng Zhang, Jiaming Song, Xun Huang, Yongxin Chen, and Ming- Liu, Diffcollage: Parallel generation of large content with diffusion models, 2023. URL 17076. 2, 3, 4, 8,
org/abs
Zhengbang Zhu, Minghuan Liu, Liyuan Mao, Bingyi Kang, Minkai Xu, Yong Yu, Stefano Ermon, and Weinan Zhang. Madiff: Offline multi-agent learning with diffusion models. Advances in Neural Information Processing Systems, 37:4177-4206, 2024. 2
15

---

Published as a conference paper at ICLR 2026
APPENDIX
A NOISY-BETHE GAP THEOREM
Theorem 1 (Noisy-Bethe Gap Theorem). Consider linear chain [u' u² u²] with pairwise factors [u', u²] and [u²,""), where u² is the transition boundary variable, As- sume the forward noising processes are pluj, uî uu. u²), p(uj, ud u² w²), and p(ui u²).. Let a(u²) = Ippl', u² pp((p, ut u', du', blu²) = ppuu,, u p((? us u², u² du', clu²) = p((? pplu))((u [²), Z = fffdu [(( du?, and q(u2) = c(u²)/ Denote by p(u), uf) the true noisy distribution and by p(u,, uy, uz) the estimator from Eq. Then the gap between true distribution and estimated distribution is:
A plu,, ur u.) plug uz) Cov x2 Cov
(8)
c
Proof. The true noisy distribution is:
/ plul uh, ur, ut u² )²) du' d
p(u|, u², u²²)
f plu' u² u² pluu u u u2 x2)) du du du
(14)
I plu', uz (p)p, uil w uz p( u² plu? oi u² u²) du du du
p(u?)p( (u² 2²)
{ I plu', u² p(ut ut [u,, u² ddu f p(u w² )p(pp uZ uz u² u² du² du²
p(u")p(??lu²))
The estimator used in (Zhang et al., 2023; Mishra et al., 2023; 2024) is:
p(ut. u², ut) f'(u', u²) p())p ut |ll, u²) du' du² ppuu, u² )((pp( uZ, uf u2, du?du? J p((2)) p(u? u²) du?
(15)
Define the left-factor message into u² as a(u²) = f p(u!, u² )p(pp us uu,, u²) du the right-factor message into 22 as b(u2)= I p(u? u²) u²)p((, (u, u, u² u²a du and the local boundary evidence
ab du² and p(u), u² uZ) [ dudu" f bu c(u²) = p(u²) )p(? u²). Then p(u!. uf uZ) c
f cdu
Introduce a change of measure by setting u(u²) with Z I [edvi duu. For the true distribution, we have:
plu,, ur, 33)
u² (p))p(pp uzl 1n2 u² ydu p(u?)p(u?(u?? Z du² / I pp(u), u2 )((( ui? ut, u² ud)du' p
p(u 2
p(u2) p(u))p(u?((r? Z
Z / Sppu'' u² (()pppu uf uu,, ud) J p(u 2 pu uf u² a dre p(u?)p((()))))))))))ppp)pppp)p lu²) du
(uu wz)
Z Eung(ra) b] For the estimator, observe that:
(16)
fat aal(?) aalu))duu du² = f plu', ur pp) ut w u²) du duz
=f
= f (f plu', u2 p( ufl un, u² du') ((u2)((ueluz) 7
p(us)p(urluz) z duz
(17)
2 f ( [pp(u', u² p() of un u²) du²)) (ua))((ullue dui 7
p(uz) (())(22)
Z [,
16

---

Published as a conference paper at ICLR 2026
By the same argument, f ((bb))(dt Z ((())) therefore the estimated distribution:
p(u u² ug) = 2 2 1 Bue
(18)
Finally, the difference between the true and estimated distributions is:
A = plu, 2 ui) - p(u;, ui ui)
= Z E Eungta - Z E, ( Bu
(19)
Z l m)
This shows that the estimator departs from the true distribution by a covariance term under the reweighted boundary measure q(u²), scaled by Z.
B SYNCHRONOUS MESSAGE PASSING
Proposition 1 (Synchronous Message Passing Constraint). Let x. E Tgxxtcho denote the concatenated intermediate factors in a chain-structured factor graph with transition boundaries -i Given the start boundary v..1 and the goal boundary Vnnor the joint constraints distribution over all intermediate factors is Gaussian:
Payne
1:n
8, 9)
O exp( exp(- (x 2' --1
+ 17 2.1))
(10)
A1
B
B A2
Go
A
B
A
BTB: B2
B
0
where Σ-1=
0
A Bg
A7 +
B Bg
17
C8
B 9
Proof. Payno(TIm) s, g) can be represented as a product of dependencies over all boundary variables:
p( s, g)syne (s,, x1) (2 x2) in- (1 In) in ( g).
(20)
The aim of this equation is to express the joint distribution over all intermediate states, given initial and final state:
p 1: 8, g) = vlo(s, 01)
1 (I1, x22) ... tn ( Un (In:g) g)
1
& exp
|s
A
2
B
A202
|Ba22
A
(
Ci
02
1
B
[I Inn-1 - Antn
2_
1
|B - -gl12
Cn-1
A
AT
= exp
8
+
T
+ 2
2
1 T
8
Co
BT B
BT
AT
B
A
A2
2 #1
81
+ 2 11
82
+ 2
F1
2
J22
32
C1
C1
C1
BI Ba
Bf Ag
A Ba
AT
Aa
22
02
+ 1 22
T
+ 1
32
I3
C/
C2
02
B B3
BT A4
A
B3
A A4
T
33
33
2 43
E4
+
1/
33
C3
C3
C3
BT B
BT
B
1
T
Fn
2 2
g +
1/
In
79
9
On
On
Cn
Cn
1
exp
---I
I.n
T
5
Fimm
+ n sim )
2
(21)
17

---

Published as a conference paper at ICLR 2026
A
B
A
0
A B1
A
B2
B Aa
where E-1
A
A AT Ag
B B3
B 0
C LIMITATIONS
Our method has several limitations. First, it relies on the accuracy of the estimated clean data (Tweedie estimates) during denoising, since guidance losses are computed on these estimates. This sensitivity could be mitigated by performing multi-step Tweedie estimation or by scaling up training data and model capacity. Second, as prior work, the number of test-time composed segments, n, must be specified manually. Developing procedures that automatically infer from task structure and uncertainty would an interesting future research direction. Lastly, our approach can more computationally demanding than direct averaging-basee sampling, because test-time guidance is implemented via gradient-basee based optimization (Talle 5). Exploring lighter optimization schedules could potentially reduce this overhead.
D BASELINE IMPLEMENTATIONS
To ensure clarity about how the prior baselines are instantiated in our setting, we briefly summarize the exact formulations of all baselines below. We'd like to clarify DiffCollage / GSC/ CompDiffuser are joint denoising over the full long-horizon trajectory, starting from single long-horizon Gaussian.
Language- Conditioond Behavioral Cloning (LCBC). The policy uses a T5 text encoder to embed natural-language instructions. We concatenate the text embedding with image features extracted by ResNet backbone, and feed the result into an MLP policy head. The model is trained in supervised manner predict a single action at each timestep, conditioned on both the language input and the current observation.
Irnguage-Conditioee Diffusion Policy (LCDPP). LCDP follows the text encoding pipeline as LCBC with T5 encoder, but replaces the MLP head with a Transformer- based policy head. The Transformer generates chunks of actions rather than single-step predictions, allowing multi-step reasoning conditioned on language and observations.
Goal-Conditioned Behavioral Cloning (GCBC), GCBC uses ResNet backbone encode both the current observation image and the goal image. The concatenated features are passed through MI.P policy head, which outputs single action. This provides goal-aware baseline without language conditioning.
Conl-Conditooned Diffusion Policy (GCDP). GCDP employs ResNet backbone with Transformer policy head, conditioned jointly on the current observation and the goal image. The model outputs action chunks, enabling multi-step planning toward the goal state.
DiffCollage / GSC. DiffCollage is compositional test time generation method, and GSC refers to its adaptation for robotic planning. Given start image and goal image, the model directly generates an entire visual plan (a sequence intermediate frames). composes a long-horizon score following Eq. 2 and iteratively denoise the entire long trajectory. We then employ separately trained inverse dynamics model to convert the visual plan into executable robot actions.
CompDiffuser is trained joint-denoising baseline: it trains the short-horizon model to condition on its preceding and following noisy chunks. At test time, it subdivides the long trajectory into overlapping chunks, and denoises them jointly while conditioning each chunk on its neighbors. We then employ a separately trained inverse dynamics model to convert the visual plan into executable robot actions. CompDiffuser is SOTA joini-den baseline OGBench 2D xy point maze navigation; however, its behavior on high-dimensional visual planning had not been evaluated prior our experiments.
CompDiffuser.
18

---

Published as a conference paper at ICLR 2026
E COMPOSITIONAL PLANNING TASKS
E. DATASET AND SIMULATION SETUP
Assets. Our asset library combines 3D models and textures from ShapeNet (Chang et al., 2015) and RoboCasa (Nasiriany et al., 2024). We additionally apply simple high-quality texture(e.g. wood, plastic, metal finishes) to increase visual fidelity. All simulations are conducted in the SAPIEN engine (Xiang et al., 2020), which provides high-fidelity physics and rendering for robotic manipulation.
State and Action Space. The observation space consists solely of RGB images with resolution 256 256 3, without access privileged information such as depth ground-truth states. The action space is parameterized by the end-effector position (3D Cartesian coordinates), orientation represented as a quaternion (4D), and binary scalar controlling the gripper open/close state. During initialization, all object poses are randomized within 0.2 m radius from their nominal positions, ensuring sufficient variability and out-of-distribution test cases.
Demonstrations. We provide 300 expert demonstrations for each of the N start-goal combinations across all scenes, resulting in thousands of trajectories spanning tool-use, drawer manipulation, cube rearrangement, and puzzle solving. Each demonstration is generated via scripted policies that guarantee feasibility and success. For the LCBC and LCDP baselines, we further annotate each demonstration with natural- natural-language descriptions of the task. Tese annotations also help evaluate how well language-condition
models can generalize across start-goal variations.
Success Condition. A rollout considered successful if all objects reach their target configurations
within a predefined spatial threshold.
E.2
TOOL
The Tool scene (Figure 7) requires the robot to manipulate a tool order to push the cube to targer location. Success is achieved when the cube reaches the designated target area within fixed distance threshold. Direct manipulation not possible, so the robot must use the provided tool to accomplish the task.
Figure 7: Visualization of Tool Scene.
19

---

Published as a conference paper at ICLR 2026
E.3 DRAWER
The Drawer scene (Figure 8) requires the robot to manipulate drawers into closed states and use the brush draw the canvas. Depending the start state, the brush may be located in different spaces, and the robot must go to the correct space and retrieve After grasping the brush, the drawer must often be closed again before drawing, to avoid collision between the and the drawer.
Figure 8: Visualization Drawer Scene.
E.4
CUBE
The Cube scene (Figure 9) requires the robot to manipulate multiple colored cubes, first arranging them into prescribed order and then placing each cube into its designated goal region. This task evaluates the planner's ability to identify and distinguish between object colors, maintain the correct ordering, and execute precise placement into multiple targets.
E.5 PUZZLE
The Puzzle scene (Figure 10) poses the most challenging test of compositional planning. The robot must first arrange multiple colored blocks into specific intermediate configuration, and then place them into distinct goal slots. This requires not only accurate object manipulation and ordering, but also the ability chain together multiple sub-tasks that were only observed in isolation during demonstrations.
F IMPLEMENTATION DETAILS
Software. All experiments conducted Ubuntu 20.04.6 with Python 3.10 and PyTorch 2.2.1.
20

---

Published as a conference paper at ICLR 2026
Figure Visualization of Cube Scene.
Figure 10: Visualization of Puzzle Scene.
21

---

Published as a conference paper at ICLR 2026
Training. Models are trained NVIDIA H200 GPUs.
Deployment Hardware. For deployment, we use a single NVIDIA L40S GPU.
Model Inputs and Outputs. All observations are first encoded into tokens using the Cosmos tokenizer. We adopt DiT' backbones for video generation, using DÝT-L. DiT-XL. (Peebles & Xie, 2023) depending on the scene. Video generation is performed entirely in the token space. For control, we employ a simple MLP-based inverse dynamics model, which predicts low-level actions conditioned on consecutive frames.
Hyperparameters. We report all hyperparameters used during both training and inference for full transparency. For fairness and reproducibility, we do not perform any hyperparameter search or use a learning scheduler; all experiments that performance gains arise from the method itself rather than extensive hyperparameter tuning. We also keep hyperparameters consistent across different tasks and scenes, unless otherwise specified, to highlight the robustness of our approach.
conducted with fixed values throughout. This ensures
Hyperparameter
Value
Diffusson Time Step
500
Batch Size
512
Optimizer
Adam
Learning Rate
10
Iterations
1M
Discount Factor
0.6
Sampling Time Steps
300
Guidance Weight gr
0.6
Table 4: Relevant hyperparameters used in our experiments.
G DEPLOYMENT TIME STUDY
G.1 TIME STUDY & NFES
All results in Table are reported under the same setting of 300 DDIM steps. Compared to DiffCol- lage, our method incurs higher wall-clock time because it requires test-time backpropagation through the diffusion model in order to enforce consistency via message passing. This extra computation accounts for the increase in sampling time, but is necessary to achieve the significant gains in success rates reported in the main results. Our implementation is fully batched, all factors require only a single forward pass at each diffusion steps. We follow the standard convention of counting only forward passes of the diffusion model as NFEs. At the same DDIM step count, method requires twice the NFEs because it performs two forward passes per step. The backward update does not increase NFEs, but does add a small amount of extra wall-clock time.
Scene
Models Composed
Sampling Time (s) J
DiffCollage
Ours
Tool-Use
3
7.1
17.2
Drawer
18.9
30.4
Cube
5
21.1
31.7
Puzzle
30.4
61.8
Table Sampling time during deployment. This is measured as the mean wall-clock time across all samples within single scene.
22

---

Published as a conference paper at ICLR 2026
H REAL WORLD QUALITATIVE RESULTS
H.1 IN DISTRIBUTION EVALUATION
Figure 11: Task1 Synthetic Plan.
Figure 12: Task2 Synthetic Plan.
23

---

Published as a conference paper at ICLR 2026
H.2 OUT OF DISTRIBUTION EVALUATION
Figure 13: Task3 Synthetic Plan.
Figure 14: Task4 Synthetic Plan.
24

---

Published as a conference paper at ICLR 2026
I FAILURE CASE OF SYNCHRONOUS MESSAGE PASSING
In our experiments, removing the discount factor does not cause plans to become globally incoherent, the generated videos remain smooth, and the intermediate states generally consistent. The main failure mode is more subtle: the generated start and goal frames remain semantically correct but exhibit small spatial misalignments relative to the precise test-time requirement. This misalignment leads to plans that are coherent in motion but anchored to an incorrect spatial configuration.
For example, in our failure cases and 2, the robot performs reasonable picking motion for the hook, yet the entire sequence centered around the wrong object location: the gray tool is consistently generated with small but noticeable orientation offset from its true pose required by start. We hypothesize that without the discount factor, the guidance signals from the start and goal conditions not sufficiently emphasized, allowing this spatial drift persist throughout the whole plan.
Sturt l'rame
Goal Frame
Synthetic Plan
Figure 15: Failure Mode of Synchronous Message Passing: Case 1.
Start l'rame
Goal
Synthetic Plam
Figure 16: Failure Mode Synchronous Message Passing: Case 2.
25


Reviews:
[{"review_id": "6KTdC2B3pD", "reviewer": "Reviewer_zF3D", "summary": "This paper builds on DiffCollage, but extends it from image composition to visual planning: instead of stitching small image patches into a larger picture, it stitches short video clips into coherent, long-horizon visual plans. The motivation is to overcome diffusion planners’ inherent limitation: while they excel at short-horizon trajectory generation, scaling them to long-horizon tasks leads to instability and broken temporal consistency when naively composing multiple short plans.\n\nProblem Setting\n- Long-horizon robot planning using diffusion models suffers from compounding noise and the breakdown of factorization assumptions when combining multiple short rollouts.\n- Prior compositional methods (e.g., DiffCollage, Generative Skill Chaining) operate in noisy latent space, leading to discontinuities at segment boundaries and failure to generalize to unseen start–goal pairs.\n\nMethod: Compositional Visual Planning (CVP in short)\n- CVP reframes long-horizon planning as inference over a chain-structured factor graph built from overlapping short-horizon video segments.\n- Key insight: enforce boundary consistency on Tweedie estimates (denoised predictions) rather than on noisy diffusion states.\n- Two complementary message-passing schemes ensure global coherence: (a) Synchronous message passing: treats all factors jointly as a Gaussian linear system for order-invariant, parallel consistency enforcement. (b) Asynchronous message passing: uses one-sided, stop-gradient updates (TD-style) for faster and more stable convergence.\n- The two are combined via Diffusion-Sphere Guidance (DSG) to interpolate between unconditional sampling and guided refinement — preserving both stability and diversity.\n- Importantly, the method is training-free: it reuses a pretrained short-horizon diffusion backbone without fine-tuning.\n\nThe authors evaluate the proposed approach on 100 robotic manipulation tasks across four scenes (Tool-Use, Drawer, Cube, Puzzle) built from ManiSkill.\n- In terms of visual quality, CVP achieves smoother motion, consistent backgrounds and better image quality quantified by VBench++ metrics.\n- When tested for task success rate (when combined with a inverse dynamics model), CVP achieves major improvements in out-of-distribution generalization, and stronger in-domain performance than existing diffusion policy baselines.", "questions": "- Language conditioning: Could this approach generalize to language-conditioned visual planning? In other words, can the boundary or goal condition g be replaced by a text or semantic embedding that specifies the desired outcome, rather than an explicit goal image?\n- Boundary artifacts in DiffCollage: Figure 2 shows DiffCollage leaving visible boundary gaps, but the original DiffCollage image-generation results generally do not show such artifacts. Could this discrepancy arise because the noisy factorization assumption breaks more severely in video or temporal domains than in static image composition? A short clarification would help reconcile this difference.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The paper presents a clear conceptual insight: enforcing boundary agreement on denoised Tweedie estimates rather than noisy diffusion states. This shift is both theoretically justified and empirically validated.\n- The toy example in Figure 2 effectively illustrates the key failure mode of noisy-state composition (boundary drift) and how the proposed approach closes those gaps. It’s an unusually clear and intuitive visualization of the underlying problem.\n- The proposed framework is training-free, operating entirely at inference time, which makes it easy to apply on top of existing short-horizon diffusion backbones.\n- The empirical results demonstrate strong improvements in temporal coherence and OOD generalization across a large set of manipulation tasks, validating both stability and scalability.\n- The method has solid theoretical grounding, bridging diffusion models with message-passing in factor graphs, which adds conceptual depth beyond heuristic composition.", "weaknesses": "- While OOD performance is impressive, the method still relies on the base video diffusion model having seen the intermediate motion fragments during training. The generalization is compositional rather than truly extrapolative, which limits deployment in fully novel environments.\n- The method requires a goal image as conditioning, restricting applicability to scenarios where both start and goal visual states are available. Many planning settings might not have explicit goal frames.\n- The approach introduces extra inference-time computation due to gradient-based guidance and message-passing updates. The paper lacks runtime comparisons with simpler compositional samplers.\n- The paper covers the preliminaries of factor-graph very quickly in section 3.1. Also. the way equation 1 puts the denominator as negative exponent makes it way less intuitive. This limits the accessibility for readers not already familiar with DiffCollage.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1SvL9HFafH", "reviewer": "Reviewer_PERF", "summary": "This paper proposes an inference-time compositional technique to extend existing short-horizon diffusion models to long-horizon robot planning without retraining. To solve the instability issue of prior work (e.g. DiffCollage/GSC) that composes segments in the noisy space ($x_t$)—a problem the paper theoretically identifies as the \"Noisy-Bethe Gap\" —this method enforces boundary consistency directly on the clean data estimates ($x_{0∣t}$). It achieves this by defining Synchronous ($L_{sync}$) and Asynchronous ($L_{async}$) message passing losses on $x_{0∣t}$ and guides the sampling process using Diffusion-Sphere Guidance (DSG).", "questions": "1. The justification for using Diffusion-Sphere Guidance (DSG) specifically is not entirely clear. Is there any reason that this paper uses the DSG instead of other guidance methods?\n\n2. I am curious about the qualitative adherence of the generated plans to the method's design. When the authors inspected the planning results, did they observe that the plans actually follow the underlying compositional structure (i.e., the chain of factors) described in the appendix? To put this another way: if a setting existed with a metric that specifically measures this 'compositionality,' would the authors hypothesize that proposed method would achieve the best score compared to the baselines?\n\n3. The proposed method introduces significant inference overhead by requiring back-propagation at every DDIM step. The paper lacks a comparison under a fixed same computational budget. For example, does the 300-step proposed method (with costly guidance) outperform a baseline like DiffCollage allowed to run for many more steps (e.g., 1000 or 3000 steps) such that the total wall-clock time/Number of Function Evalutations (NFEs) are equivalent?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. While using $x_0$ prediction (Tweedie estimates) for guidance is not new, its novel application to compositional generation is a impressive contribution, using it to enforce boundary consistency between factors.\n2. The paper provides a strong theoretical justification for why prior methods fail. By identifying and proving the \"Noisy-Bethe Gap\" (Theorem 1) , it formally explains that simple averaging in the noisy $x_t$​ space is fundamentally flawed.\n3. It addresses a critical and timely problem: how to achieve inference-time scaling by stitching pre-trained, short-horizon models to generate long-horizon content. This training-free, \"plug-and-play\" approach is highly practical and flexible, extending the capabilities of existing models without costly retraining.", "weaknesses": "1. The justification for using Diffusion-Sphere Guidance (DSG) specifically is not entirely clear. The paper would be significantly strengthened by an ablation study that compares DSG to standard, simpler guidance mechanisms (e.g., conventional gradient-based guidance or other proposed guidance methods). This would help demonstrate that the proposed method is general enough to improve performance even when paired with various other guidance techniques, not just DSG.\n2. The paper's claims regarding compositional performance would be significantly strengthened by a broader comparison against other diffusion-based stitching and planning methods. The current experiments are heavily focused on DiffCollage/GSC. Including comparisons to other relevant works, such as CompDiffuser [1], would provide a more comprehensive validation.\n\n\n[1] Luo, Y., Mishra, U. A., Du, Y., & Xu, D. (2025). Generative trajectory stitching through diffusion composition. arXiv preprint arXiv:2503.05153.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RLbkZpIDI9", "reviewer": "Reviewer_bi53", "summary": "This paper introduces a novel, training-free method for compositional planning leveraging diffusion models. The authors begin by theoretically interrogating the inaccuracies of prior compositional generation methods, such as DiffCollage, which rely on a Bethe approximation in the noised state. This inaccuracy is formalized in the paper as the \"Noisy-Bethe Gap Theorem\" (Theorem 1). To address this limitation, the paper proposes a novel inference-time guidance technique involving synchronous and asynchronous message passing. This mechanism is designed to enforce boundary agreement directly on the estimated Tweedie estimates (the predicted $x_0$​) rather than on the noisy intermediate states.\n\nThe proposed method is validated on a suite of robotic manipulation tasks. This is achieved by first generating long-horizon visual plans (scenes) using the diffusion model, and subsequently inferring executable actions via a separately trained inverse dynamics model. Furthermore, the authors conduct a comparative analysis against DiffCollage, evaluating not only task success but also the quality of the generated visual plans using metrics such as motion smoothness, background consistency, and aesthetic quality.\n\nThe results demonstrate that the proposed framework significantly outperforms the baseline (DiffCollage) across both task success rates and scene generation quality. Specifically, while DiffCollage fails on nearly all manipulation tasks, the proposed method achieves success rates ranging from 50-100%. Moreover, it demonstrates roughly a two-fold improvement in key video quality metrics like motion smoothness and background consistency.\n\nThe authors include ablation studies on the components of their novel composition rule (i.e., synchronous vs. asynchronous message passing) and the number of denoising steps. A practical comparison of sampling time against the baseline is provided in Appendix F, offering readers a concrete understanding of the computational trade-offs. The appendices also furnish the theoretical derivations for the inaccuracy of the noisy Bethe approximation (Theorem 1) and the formulation of synchronous message passing (Theorem 2). Finally, the authors provide comprehensive details on the experimental environments, tasks, and model hyperparameters used for validation.", "questions": "- **Rationale for Discount Factor (Eq. 11):** Regarding the asynchronous loss, why is a discount factor γ (set to 0.6) used to down-weight connections far from the start and goal? Intuitively, temporal smoothness and boundary agreement would seem equally important across the entire trajectory, not just near the endpoints. Could the authors elaborate on this design choice?\n    \n- **Performance Saturation (Fig. 5):** In Figure 5, the performance on the Drawer scene is still clearly improving at 300 sampling steps. Have the authors explored running this experiment with more steps until performance saturates? It would be valuable to know if it plateaus, or if performance eventually degrades.\n    \n- **Sensitivity to Loss Weighting (Fig. 4):** The ablation in Figure 4 compares \"Sync Only\", \"Async Only\", and their combination. This reviewer is curious about the sensitivity to the  weighting between these two losses. What would a sweep of $w \\cdot L_\\text{sync}​+(1−w) \\cdot L_\\text{async}$​ reveal? This would provide insight into the robustness of the method and the complementary nature of the two losses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- **Clear Articulation of Problem and Strong Theoretical Grounding:** The paper's primary strength lies in its clear theoretical articulation of a critical problem in compositional planning. The authors compellingly argue that prior methods (e.g., DiffCollage) are flawed due to their reliance on a Bethe approximation in the noisy state. This critique is not just qualitative; it is substantially supported by the \"Noisy-Bethe Gap Theorem\" (Theorem 1, detailed in Appendix A), which provides a strong formal foundation for the work.\n    \n- **Significant Empirical Gains and Thorough Analysis:** The proposed method demonstrates a striking and significant performance gap compared to the baseline. While DiffCollage fails on nearly all robotic manipulation tasks, the proposed method achieves high success rates (50-97%). This empirical success is well-supported by a thorough ablation study (Figures 4 and 5) that effectively isolates the contributions of the synchronous and asynchronous message passing components. The analysis also practically demonstrates how performance scales positively with increased denoising steps (up to 300).\n    \n- **Completeness and High Potential for Reproducibility:** The authors are commended for the paper's completeness. Beyond the main theorem, they provide a detailed theoretical derivation for the synchronous message passing mechanism (Appendix B) . This theoretical rigor, combined with comprehensive details on the simulation environments, tasks (Appendix D) , and hyperparameters (Appendix E) , ensures a high potential for reproducibility.", "weaknesses": "- **Insufficient Discussion on the Nature of the Contribution:** The paper attributes its success to composing in the Tweedie ($x_0$​ estimate) space. However, the powerful composing rules ($L_\\text{sync}$​ / $L_\\text{async}$​) are themselves very strong, explicit guidance heuristics. A key insight may be that the Tweedie estimation space is the first domain that enables such strong, explicit guidance to be applied stably (whereas it would fail in the noisy state). The authors are encouraged to reframe their contribution to reflect this: composing on clean estimates provides not just better Bethe approximation but also a crucial platform for powerful heuristics, and their proposed message passing is one such successful instance.\n    \n- **Potential for Generating Unrealistic Plans:** One of unaddressed limitations is the potential for the guidance to create unrealistic plans. The method guides sampling to enforce only boundary smoothness between short plans. It is plausible that this guidance could force the generation of a physically impossible trajectory within a short plan if the boundaries are in difficult-to-connect states (e.g., on opposite sides of a wall). The system might prioritize a smooth interpolation (e.g., passing through the obstacle) over maintaining physical plausibility.\n    \n- **Lack of Adaptivity in Plan Length:** As the authors acknowledge in their limitations, the method lacks adaptivity in plan length, requiring the number of factors ($n$) to be manually specified. This prevents the planner from dynamically adjusting to task complexity, potentially over-planning for simple tasks or under-planning for difficult ones.\n    \n- **(Minor) Presentation and Clarity Issues:**\n    \n    - **Equation 5:** the second term's numerator appears to have a typo. It is written as $x_t​−\\sqrt{\\bar{\\alpha}_t}x_0$, but based on the formulation in Equation 3, it should likely be ​$x_t​−\\sqrt{\\bar{\\alpha}_t}x_{0|t}$.\n\n    - **Figure 2 Caption:** The terms “DiffCollage” and \"GSC\" are used without any prior definition. Furthermore, \"Diffcollage\" should be capitalized consistently as \"DiffCollage\" .\n        \n    - **Qualitative Comparison:** The paper would greatly benefit from a direct qualitative comparison (e.g., side-by-side frames) showing the failure modes of DiffCollage (e.g., blurriness, incoherence) against the successful generations of the proposed method.\n        \n    - **Table 2:** The spacing between the caption and the table is too tight, harming readability.\n        \n- **Compliance with ICLR Policy:** The manuscript does not appear to include the mandatory statement on the use of Large Language Models (LLMs) in its preparation, as required by the ICLR 2026 Author Guide.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GBYigNV6B7", "reviewer": "Reviewer_dR6s", "summary": "The paper proposes a training-free, inference-time scheme to compose long-horizon visual plans by stitching overlapping short video segments from a pretrained diffusion model. The key idea is to enforce boundary agreement on Tweedie estimates (denoised (x_{0|t})) rather than on noisy states (x_t), implemented via (i) a synchronous Gaussian chain constraint and (ii) an asynchronous, stop-gradient message-passing loss. These constraints are injected into DDIM using diffusion-sphere guidance (DSG). Algorithm 1 describes the sampler. On a simulated benchmark of four manipulation scenes, the method reports higher success rates than a DiffCollage-style baseline and certain policy baselines.", "questions": "How does joint denoising baseline compare to this method and DiffCollage-style factorization? How much better is it as reference?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Clear modeling lens: Neat factor-graph formulation and explicit block-tridiagonal precision structure for boundary consistency; derivations are technically correct.\n\n- Practical sampler design: Integrates guidance into DDIM via DSG with synchronized/async constraints; the ablation indicates the async stabilization matters for performance.\n\n- Empirical gains on the paper’s benchmark: Improved temporal coherence metrics and task success relative to selected baselines.", "weaknesses": "- Enforcing guidance on (x_{0|t}) is a standard mechanism in training-free guidance; the synchronous GMRF derivation is textbook\n- The primary compositional baseline (DiffCollage/GSC) has near-zero success across tasks, making relative gains hard to interpret and raising concerns about implementation/tuning rather than superiority.\n- The Noisy-Bethe identity does not establish that Tweedie-space constraints are superior;", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Nzk6TKZnOC", "reviewer": "Reviewer_geNQ", "summary": "This paper tackles the challenge of generating long-horizon visual plans using diffusion models trained only on short video clips. The authors introduce CVP, a training-free inference-time framework that composes overlapping short-horizon segments through message passing on denoised (Tweedie) estimates rather than noisy diffusion states. This approach ensures global temporal consistency and smooth transitions across segments without retraining. Experiments on robotic manipulation and visual planning benchmarks show that CVP substantially improves video quality, coherence, and task success rates over prior compositional and policy-based baselines.", "questions": "- Since the current formulation assumes a linear chain structure, can the approach be extended to tree-structured or multi-goal planning problems? For example, tasks with subgoals or parallel branches.\n- To what extent are the observed gains attributable to the compositional inference versus the base diffusion model quality? Would weaker or stronger base models change the relative benefit?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- Composing diffusion-based video planners in a training-free manner by, shifting consistency enforcement to denoised Tweedie space is novel. \n- The proposed factor-graph formulation analysis provide a clear and rigorous justification for the method.\n- The approach achieves large performance gains in both visual quality and planning success.\n- The framework shows robust out-of-distribution generalization.", "weaknesses": "- The experiments, though extensive within robotic planning, are restricted to a single simulation domain (ManiSkill). Testing on real-world data or non-robotic visual domains would strengthen the claim of general applicability. Without at least some transfer or qualitative evidence on real-world data, it remains unclear how robust the approach is. \n- The paper could better position its contribution relative to classical planning and hierarchical control methods, making clearer how compositional diffusion planning aligns with them. \n- The paper doesn't analyze the computational efficiency or runtime implications of the added message-passing guidance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)