Foresight Diffusion: Improving Sampling Consistency in Predictive Diffusion Models
Authors: Yu Zhang
Xingzhuo Guo
Haoran Xu
Jialong Wu
Mingsheng Long
Venue: ICLR 2026 Poster (2026)
Keywords: diffusion models
flow-based models
predictive learning
generative models
Primary area: generative models
TL;DR: We propose Foresight Diffusion to improve the sampling consistency in predictive diffusion models through decoupling conditional understanding from target denoising.

Abstract: Diffusion and flow-based models have enabled significant progress in generation tasks across various modalities and have recently found applications in predictive learning. However, unlike typical generation tasks that encourage sample diversity, predictive learning entails different sources of stochasticity and requires sampling consistency aligned with the ground-truth trajectory, which is a limitation we empirically observe in diffusion models. We argue that a key bottleneck in learning sampling-consistent predictive diffusion models lies in suboptimal predictive ability, which we attribute to the entanglement of condition understanding and target denoising within shared architectures and co-training schemes. To address this, we propose **Foresight Diffusion (ForeDiff)**, a framework for predictive diffusion models that improves sampling consistency by decoupling condition understanding from target denoising. ForeDiff incorporates a separate deterministic predictive stream to process conditioning inputs independently of the denoising stream, and further leverages a pretrained predictor to extract informative representations that guide generation. Extensive experiments on robot video prediction and scientific spatiotemporal forecasting show that ForeDiff improves both predictive accuracy and sampling consistency over strong baselines, offering a promising direction for predictive diffusion models.

Published as a conference paper at ICLR 2026
FORESIGHT DIFFUSION: IMPROVING SAMPLING CONSISTENCY IN PREDICTIVE DIFFUSION MODELS
Yu Zhang, Xingzhuo Guo; Haoran Xu, Jialong Wu, Mingsheng Long"
School of Software, BNRist, Tsinghua University, Beijing 100084, China (zhangyu24, gxz23) gx2233
@ma i . tsinghua. edu en, mi mingsheng@tssi inghnn niguua edu. cn
ABSTRACT
Diffusion and flow-based models have enabled significant progress in generation tasks across various modalities and have recently found applications in predictive learning. However, unlike typical generation tasks that encourage sample diversity, predictive learning entails different sources of stochasticity and requires sampling consistency aligned with the ground-truth trajectory, which is a limitation we empirically observe in diffusion models. We argue that key bottleneck in learning sampling-consisten predictive diffusion models lies in suboptimal predictive ability, which we attribute the entanglement of condition understanding and target denoising within shared architectures and co-training schemes. To address this, we propose Foresight Diffusion (ForeDiff). framework for predictive diffusion models that improves sampling consistency by decoupling condition understanding from target denoising. ForeDiff incorporates separate deterministic predictive stream to process conditioning inputs independently of the denoising stream, and further leverages pretrained predictor extract informative representations that guide generation. Extensive experiments on robot video prediction and scientific spatiotemporal forecasting show that ForeDiff improves both predictive accuracy and sampling consistency over strong baselines, offering promising direction for predictive diffusion models.
INTRODUCTION
Diffusion models (Sol--Dickee et al., 2015: Song & Ermon 2019
Ho al. 2020; Song et al. 2023 Liu et al. 2023 Albergo & Vanden-Eijndenn
2020) and flow-based models (Lipman et al.
2023) class of generative models that has achieved ofate-of-the-art across wide range of tasks
and modalities, including image (Dhariwal & Nichol 2021), video (Ho et al. 2022), and cross-modal
generation (Saharia et al. 2022; Singer et al.) 2022 Rombach al., 2022). Owing to their ability
to model complex and multimode distributions, diffusion models have recently been adopted for
predictive learning (Voleti et al. [022] Chen et al., 22023, Gao et al.) (023) within the conditional
generation framework, where they serve as spatiotemporal predictors to learn real-world dynamics
and generate future trajectories conditioned on past observations.
Although both require high-fidelity stochastic outputs, pre- dictive learning fundamentally differs from generative tasks in the nature of stochasticity. In generative tasks (e.g., text- to-image synthesis), the target distribution corresponding to certain text prompt is inherently diverse, and mod- els are designed to pursue diversity, allowing imperfect or widely varying samples. In contrast, predictive learning (e.g., robot video prediction) aims to infer physically coher- ent futures from partial observations, where stochasticity stems mainly from incomplete or partial observational in- formation. Thus, predictive models, though require stochas- tic outcomes, prioritize per-sample accuracy and therefore should ensure sampling consistency to certain extent, "Equal contribution.
MiducPP
Vasilla
Forosigh Diflasion
Density
LPIPS
Figure 1: Kernel density estimation: LPIPS distributions of generated sam- ples. Shaded areas represent estimated probability densities; dashed lines in- dicate sample means. A lower LPIPS corresponds to better quality.

---

Published as a conference paper at ICLR 2026
Generative tasks
Predictive learning
Deterministic tasks
Diffusion models for image generation
Diffusion models for video prediction
Foresight Diffusion for Predictive learning
Class: <apple>
Past Frames:
ForeDiff
Text description: apple on a table
Actions:
Predictive Stream
Rotate 15 degrees, move down, and
Generative Stream
TexMo-image Diffusion
Vanilla Diffusion
Diversity desired
Diversity undesired
Sampling consistency
Figure 2: Aligning model stochasticity with task demands. (Left) Generative tasks usually favor diversity, making diffusion models ideal as they produce varied samples. (Middle) In contrast, predictive learning requires relatively precise predictions, and vanilla diffusion models demonstrate unsatisfactory sampling consistency. (Right) Foresight Diffusion strikes a balance between highly stochastic and fully deterministic models, making it well-suited for predictive learning.
which refers the ability produce concentrated, low-variance samples under identical conditions such that all the generated samples could closely align with ground-truth trajectories.
The specific demand of sampling consistency challenges the employment of diffusion models to predictive learning, which are prone to issues such as hallucinations, weak conditioning. and sample imperfection (Aithal al. 2024 Dhariwal & Nichol 202T). As illustrated in Figure although vanilla diffusion models outperform traditional auto-regressive models in terms of best-case as well as average performance, they exhibit higher sample variance and heavier worst-case tails that are undesirable for predictive learning. This highlights a fundamental mismatch between vanilla diffusion models and the requirements of predictive learning, and consequently there remains need for sampling-oe campling-consistent consistent diffusion models that effectively balance stochasticity and determinism.
To address these challenges, we propose Foresight Diffusion (ForeDiff). framework designed to enhance the sampling consistency of predictive diffusion models by decoupling condition under- standing from the denoising process. Diffusion models typically exhibit suboptimal predictive ability compared to deterministic approaches, condition understanding and target denoising entangled within shared architectures and co- to-irinning schemes, which impairs effective condition understanding and may finally lead diverse but unfaithful samples relative to the ground truth. Instead of directly applying conventional conditional diffusion model, Fife introduces a separate deterministic stream that processes condition inputs independently of the stochastic denoising stream. Further- more, leverages pretrained deterministic predictor to extract informative representations, thereby improving the model's predictive ability. Experiments across multiple modalities demonstrate that FofeDiff significantly improves both predictive performance and sampling consistency.
Our contributions are summarized follows:
We revisit diffusion models in the context of predictive learning, task intrinsically different from conventional content generation, and identify their sampling consistency issues.
We attribute the consistency issues of diffusion models to the entanglement of condition understanding and target denoising within shared architectures and co-training schemes.
We propose Foresight Diffusion, framework that improves sampling consistency by decoupling condition understanding and incorporating pretrained deterministic predictor.
Extensive experiments and analyses on video prediction and spatiotemporal forecasting demonstrate that ForeDiff achieves superior predictive accuracy and sampling consistency.
2

---

Published as a conference paper at ICLR 2026
2 PRELIMINARIES
Denoising-based generative models typically build on a forward process that progressively corrupts clean data samples with increasing amounts of noise. By default, we consider the simplest linear interpolation scheme (Lipman et al., 2023 Liu et al. (023) which has been widely used due to its analytical tractability and connections optimal transport. The forward process expressed as the linear interpolation between a clean sample Xy (x) and standard Gaussian noise e ~ (O,
X
tixo + te, t E [0, 1]
(1)
To recover the data distribution, we learn reverse this process by training a neural network vo vo(x,, to approximate the time- Lume-dependent velocity field (or its reparameterizationss) The training objective, known as conditional flow matching, is formulated as:
veclocit (0) Exn.e.t [Ivo(x,,))
(E-xx)ll xo)]²
At inference time, samples generated by integrating the learned velocity field backward in time:
- = X, V. (x, A)t,
starting from Gaussian noise sample x ~N(00, I) and progressing toward / = 0. This framework naturally extends to conditional generation by introducing conditioning variables into the model, resulting in conditional velocity field va (Xt, t, c). where с guides the generative dynamics.
METHOD
3.1 DIFFUSION MODELS FOR PREDICTIVE LEARNING
Predictive learning is fundamentally stochastic task. We formulate it conditional generation
problem, where s 00+1:: denotes a sequence of past visual observations and c represents a potential perturbation the environment (e.g., actions or goals) (Agarwal et al. 2025). The objective of predictive model is to approximate the conditional distribution of future Trames p(s1:9 000+1:0 c). which is inherently unknown and must be learned from data.
To reduce computational cost, we adopt the widely used latent diffusion paradigm (Rombach et al. 2022), which compresses the frames into a lower-dimensional latent space using a pretrained autoencoder composed of an encoder E and decoder D. The past and future frames are en- coded as -0+1- E(s ((s-00+:0) and II:S = E(sl:5), respectively. Denoting x z and y {z -0+1:0, c). the learning objective becomes modeling the conditional distribution p(xly). We focus on predictive diffusion models, which-as described Section - a conditional denoiser v y, that takes input both the condition y and noisy version of the target X
Desermnistic Prediction
iVideoGPT
Vanilla Diffission stop)
MiwkViT VIT
Vanilla Diffusion
128
FVD
EVD
LPIPS
63-
2.
ValleGPP FilVid
Vanilla Dilfusion
100
266
300
400
230
200
700
FVD
LPIPS LPIPS
LPIPS
Model Size (Million Parameters)
(Best) (Avemge)
Latent MSE
FVD
(a) Model efficiency: FVD vs. size.
(b) LPIPS for consistency analysis.
(c) Predictive ability.
Figure 3: Comparison between predictive diffusion models and existing baselines on RoboNet dataset. (a) Vanilla diffusion achieves competitive FVD with significantly fewer parameters, demonstrating high model efficiency.
(b) Vanilla diffusion performs well on best and average LPIPS, but suffers from higher worst-case error, highlighting poor sampling consistency. (c) Vanilla diffusion underperforms a deterministic predictor in absence noisy targets, revealing limited predictive ability.
3

---

Published as a conference paper at ICLR 2026
Predictive diffusion models are efficient, accurate, but not consistent. We evaluate the perfor- mance of predictive diffusion models in comparison to existing baselines, including auto-regressive and mask-bassed architectures (Wu et al., 2024] Gupta et al. 2023). We adopt adeo-adapted DiT, shown in Figure 4 a), as our vanilla diffusion model, and assess its performance using two standard metrics: Fréchet Video Distance (FVD) for distributional similarity, and LPIPS for perceptual sample- level quality (Unterthiner et al. 2018;; Zhang et al., 2018). Figure Ba presents a scatter plot comparing model size and FVD, while LPIPS scores under best-case, average, and worst-case conditions are reported in Figure 3b Our key findings are summarized below:
Despite having a smaller model size and no pretraining, the vanilla diffusion model outper- forms baseline models in FVD, best-case LPIPS, and average LPIPS, demonstrating strong efficiency and accuracy for predictive learning.
However, under worst-case LPIPS, the diffusion model underperforms iVideoGPT. Together with the results in Figure | this suggests a lack of sampling consistency, i... insufficient concentration of plausible outputs under the same condition.
These results reveal critical limitation of predictive diffusion models: although exhibiting strong best- case performance with compact architectures, they lack robust conditional control during generation, leading to significant variability across samples. As result, evaluating models only by best-case performance- as commonly done in prior works (Voleti et al.] 2022) Yu et al.) [0233] Wu et al.) 2024) can be misleading. Therefore, report average metries throughout the remainder of the paper unless otherwise specified.
3.2 OBSERVATIONS
To further explore the sampling inconsistency behavior in predictive diffusion models, we investigate their predictive ability, i.e., how well the model understands condition inputs (e.g., visual observations and actions) and predicts future trajectories based on the task's underlying dynamics. To assess how much the model relies solely y during denoising, (corresponding to = under notations in DDPM (Ho et al. 2020)), where X ~ NN (0,1) contains no signal and thus contributes useful information, which introduces the following lemme
consider the special
t 1
Lemma 3.1. For diffusion model as defined in Section 2 by reparameterizing the output as Xe (xt,t,y) = X₁ - t. (Xt, t, y) Karras et al. 2022), an ideal diffusion model at t = minimizes:
[| =) xol12]
(4)
where the noise is independent of both Xa and y. Furthermore, a diffusion model minimizing Eq. reduces to similar architectured deterministic predictor Ie minimizing the following objective:
(4)
Lacter (E) =Bxx,y [I1Se()) [1]f(())xxo)) xol12]
This lemma conveys two insights. First, evaluating performance at t 1 provides reasonable proxy for assessing the predictive ability of a diffusion model, as the model must rely entirely on y while discarding the irrelevant input Second, the predictive ability of diffusion model is inherently bounded and is possible to be equivalent to that of deterministic model. To empirically test whether the diffusion model can reach this upper bound, train ViT-based deterministic model fg. shown in Figure (b). with an architecture analogous to the diffusion model. We then compare its performance to the diffusion model evaluated with single-step inference at 1. The results, shown in Figure 3c, reveal that the diffusion model underperforms its deterministic counterpart in both latent and pixel space, verifying its limited predictive ability relative to its potential.
Discussion. Such observation indicates that the suboptimal predictive ability of diffusion models stems from the entanglement between condition understanding and target denoising due to the nature of diffusion models to train on various f. This entanglement constrains condition understanding by factors of both architecture and training. From the architecture perspective, network parameters must simultaneously learn both condition understanding of y and target denoising of x,, and this dual-role
Proof is referred Appendix
4

---

Published as a conference paper at ICLR 2026
Vanilla Diffusion
(b) Deterministic Prediction
Timestep
DIT Hook
DIT Block
DIT Mook
DIT Blok
Dilusion Generation
VIT Block
VIT Block
VIT Block
VIT Brock
Determinatic Predkction
Condrtion
Nuscal
Candition
(c) (ForeDiff-zero)
Foresight Diffusion backbone
(d) Foresight Difftsion (ForeDiff)
Predictive ability
Pretroining stoge only
Generative ability
Pred
Determinatic Prediction
Cundition
Dillusion
Mook
Natsed larget
Notsed target
Timodop
Timestep
Figure 4: Overview of Foresight Diffusion. (a) Vanilla diffusion jointly processes condition and noisy target, limiting its predictive ability. (b) A Deterministic model focuses solely condition understanding and achieves better predictive performance. (c) ForeDiff- zero introduces a separate predictive stream to isolate condition understanding from noise. (d) ForeDiff further adopts two- stage scheme: it pre-trains the predictive stream, then freezes representations to guide generation.
constraint can limit the model's ability to fully exploit the condition information. From the training perspective, the presence of x, informative input introduces shorteut, making it easier for the model to rely on generative priors from X rather than precise task-specific dynamics from
3.3 FORESIGHT DIFFUSION
Previous observations suggest that shared architectures and co-training schemes of condition y and noisy target limit the predictive ability of diffusion models, due to the need to simultaneously balance condition understanding and target denoising. To address these limitations, we introduce simple yet effective framework that enhances predictive ability through architectural decoupling and improved training scheme.
Architecture. Building architectures of vanilla diffusion models, propose architectural extension that integrates decoupled deterministic blocks for processing the condition y independently of xg. This design forms the Foresight Diffusion backbone (ForeDiff-zero)) illustrated in Figure/fc which separates the model into two distinct streams, the predictive stream and the generative stream, aiming focus on y and x, respectively. The predictive stream is composed of deterministic ViT blocks, while the generative stream follows the standard DiT-based denoising process. Formally, let M denote the number of ViT blocks and the number of DiT blocks. The process can then be formulated as:
go PatchEmbedly'), ho PatchEmbed(x.), h Fusion gM,
g: ViT,(g:1), i= M,
(6)
h (h,, i= 1..,,,, N,
OutHead(hyy-))
When M = 0, ForeDiff-zen reduces to vanilla conditional diffusion, where the fusion operates directly on the raw condition: h, = Fusion(ho, PatchEmbedl()).))
Unlike vanilla diffusion models, which ingest both and at the initial point of shared network, ForeDiff-zero processes the condition solely within its predictive stream and passes the resulting informative representation gM instead of to the generative stream. Since the predictive stream is entirely agnostic to x;, its parameters are fully dedicated to understanding y, thereby mitigating the architectural entanglement that limits the predictive ability.
Training scheme. To further ensure that the ViT blocks the predictive stream effectively acquire predictive ability instead of learning static representation, in addition the end-to-end training
5

---

Published as a conference paper at ICLR 2026
scheme of ForeDiff-zero, further enhance Foresight Diffusion (ForeDiff) by adopting two-stage training scheme demonstrated in Figure((d)
the first stage. the predictive stream is trained as standalone deterministic predictor. Inspired by the strong predictive abilitt of the deterministic model f& discussed in Section 3.2) ForeDiff trains the predictive stream by adding a prediction head Head form the architecture of Jg. which is defined by fc(y) PredHead (gadHeadggga)) (gas), and train it using the prediction loss in Eq. ( In the second stage, we freeze the pretrained predictive stream and remove the PredHead module. The resulting internal representation gmr, computed by the frozen ViT blocks, is then used as the conditioning input to train the generative stream.
We treat the predictive and generative streams as two independent models, denoted by Pe and Gg. respectively. Each stage is optimized with one of the following loss functions:
Laster = Exo.y [IP((y)
x
Leneisee =
Exp.v.e,t
(xx,
(6 (E-xx))
where P refers to the predictive stream excluding the PredHead module. This ensures that the information guiding generation are derived from the learned predictive representations, rather than the final outputs of the predictor. By combining architectural decoupling with dedicated predictive pretraining, Fofe leverages a deterministic predictor as a preparatory module for conditional generation. The design enables the model to "foresee" contextually rich representations, thereby enhancing predictive ability and improving both generation accuracy and sampling consistency.
EXPERIMENTS
We evaluate Foresight Diffusion in comparison to conventional conditional diffusion baselines across a range of tasks, covering both (action-conditioned) robot video prediction and scientific spatiotemporal forecasting. We adopt the model architecture across all tasks, with the only difference being the condition information provided to the model. The model components follow the standard configurations of VIT (Dosovitskiy et al.) 2020) and DiT (Peebles & Xie 2023) (or SiT (Ma iet al. ) blocks. Unless otherwise specified, we use 6 ViT blocks in the predictive stream and 12 DiT blocks in the generative stream. Additional implementation details provided in Appendix
4.1 ROBOT VIDEO PREDICTION
Setup. We begin by evaluating on RoboNet (Dasari et al. 2019) and RT-1 (Brohan et al. 2022)), two real-world video datasets widely used for assessing general video prediction performance. RoboNet contains 162k videos collected from 7 robots operating in diverse environments. Following previous works (Babaeizadeh et al., [021]] Wu al. 2024), the task is predict 10 future frames given past frames together with actions. RT-1 consists of 87k videos from 13 robots performing hundreds of real-world tasks. Here, the objective is to predict 14 future frames conditioned on past frames and the corresponding instructions. All video frames are resized to 64 64 pixels for both datasets. We evaluate model performance using widely adopted metrics including FVD (Unterthiner et al. 2018), PSNR (Huynh-The ThhyhhTThu & Ghanbari 2008). SSIM (Wang et al. 2004), and LPIPS (Zhang et al. 2018). In addition, we introduce STDpsNR. STDSSIM. and STDPPIPS defined as the standard deviation of metric values across multiple generated samples, numerically represent sampling consistency, where smaller values indicate more consistent predictions. See Appendix C for computation details.
Table Robot video prediction results on RoboNet and RT-1 datasets. SSIM and LPIPS scores are scaled by 100 for convenient display.
Dataset
Method
FVD 4
PSNR
SSIM f
LPIPS 4
STD +
STDssIM 4
STD 1
Vanilla Diffusion
53.8
27.1
88.2
5.65
0.66
1.33
0.65
RoboNet
ForeDiff-zoe
52.7
27.2
88.4
5.54
0.68
1.36
0.66
ForeDiff
51.5
27.4
88.8
5.25
0.37
0.70
0.35
Vanilla Diffusion
11.7
30.4
93.6
3.79
0.97
1.11
0.53
RT-1
ForeDiff-zero
11.1
30.7
93.9
3.60
0.95
1.03
0.50
ForeDiff
12.0
31.2
3.42
0.38
0.33
0.17

---

Published as a conference paper at ICLR 2026
RobuNet
5-6
5=8
5-0
5=2
S- 10
Ground
(Otservations)
Vimilla DifTusion Predictions
ForeDif Prodictions
Visualization of results RoboNet dataset (zoom in for details). In vanilla diffusion models, the pink shovel (left) appears distorted, while the toy object (right) collapses entirely. In contrast, FofeDiff produces more structurally plausible and visually coherent outputs.
Figure 5:
RF1
1-0 5-2
1-d
2
5
1-10
12
1-0
5-2
2-6
5
1-10
12
Ground
(Oharvaaima))
Vanila Didlasion Prodictions
FuezDifl Prafictions
Figure 6: Visualization of results RT-1 dataset (zoom in for details). Compared with vanilla diffusion, ForeDiff more accurately predicts the brightness of background and the position of robots.
Results. We present the experimental results in Table ForeDiff outperforms vanilla diffusion in each dataset, showing not only an improvement in accuracy (PSNR, LPIPS, etc.) but significant reduction in standard deviation (STD), which demonstrates its suitability for sampling-consister predictive learning. Moreover, there is notable difference between ForeDiff-Zero and vanilla diffusion in terms of STD, suggesting that the improved sampling consistency primarily stems from the deterministic pretraining procedure. The qualitative comparisons in Figures and further highlight the superior predictive ability of ForeDiff.
Since post-training strategies such as classifier-free guidance (CFG) (Ho & Salimans] (022) can also im- prove performance and consistency, we investigate their relationship with our approach. Specifically, we evalu- ate the effect of applying CFG during inference to both vanilla diffusion and ForeDiff, shown in 2 Our obser- vations are twofold: (1) the improvement brought by CFG on vanilla diffusion remains limited compared to using FofeDifff and (2) ForeDiff operates orthogonally CFG and be effectively combined with it.
Table 2: Robot video prediction results with CFG on RoboNet.
Model
Metric
w/o
CFG
Vanilla Diffusion
LPIPS
5.65
5.27
ForeDiff
LPIPS
5.25
5.05
Vanilla Diffusion ForeDiff
STDuriss
0.65
0.49
STD
0.35
0.24
Finally, we compare ForeDiff with prior methods using their evaluation settings, where metrics
reported the best of 100 samples (Top-1). As shown in Table ForeDiff achieves competi- tive performance even without accounting for its advantages in consistency, further confirming its effectiveness in predictive learning.
4.2 SCIENTIFIC SPATIOTEMPORAN FORECASTING
Setup. We then evaluate our method on HeterNS (Li et al. 2021 Zhou et al. 2025 which is generated from simulations of heterogencous 2D Navier-Stokes equations. Each sequence HeterNS contains 20 frames with a resolution of 64 x 64, where each pixel represents the vorticity of turbulence the corresponding space. The task predict the latter 10 frames based on the first 10 frames. L2 and relative L2 are reported according to previous works (Li et al. 2021
Zhou et al. 2025).
We present the experimental results in Table ForeDiff achieves a much lower relative L2 eтror compared to ForeDiff-Zers and both significantly outperform vanilla diffusion. These results suggest that incorporating deterministic blocks and pre-training them individually contribute to improved performance, demonstrating the applicability of ForeDiff for physical scenarios. Figure illustrates the qualitative advantage of Fife through visual comparisons.
Results.
7

---

Published as a conference paper at ICLR 2026
Table 3: Addition results compared with baselines on RoboNet dataset. Metrics are reported on the best of 100 samples. LPIPS and SSIM scores are scaled by 100 for convenient display.
Table 4: Scientific spatiotemporal forecast- ing results on HeterNS dataset. Metrics are scaled by 100 for convenient display.
Method
FVD 4
PSNR 1
SSIM t
LPIPS +
Method
1.2 +
Relative 1.2 4
Mask ViT 2023
133.5
23.2
80.5
4.2
SVG (2019)
123.2
23.9
87.8
6.0
Vanilla Diffusion
1.73
1.50
GHVÁE 2021
95.2
24.7
89.1
3.6
ForeDiff-zero
1.03
0.83
FitVid (02T)
62.5
28.2
89.3
2.4
ForeDiff
0.19
0.18
iVideoGPT (2024
63.2
27.8
90.6
4.9
ForeDiff
51.5
28.2
90.4
4.5
HeterNS
10
Truth
(Observations)
For Predictions
Residaal
Residual
Figure 7: Visualization of results on HeterNS dataset. As the simulation progresses, predictions from vanilla diffusion deviate increasingly, whereas ForeDiff maintains consistently accurate predictions.
4.3 ANALYSIS
For clarity, we report simplified results here, focusing on key trends. Complete numerical results across datasets and variants are provided in Appendix D supplements.
Effect of PredHead module. We investigate whether ForeDiff benefits more from the predictive ability of the ViT stream, represented by its learned internal features, or its explicit prediction outputs. To this end, conduct ablation where the DiT stream is conditioned on PeedHead outputs instead internal representations used in the standard ForeDiff. As shown in Figure Sa conditioning on PredHead outputs leads reduced model performance, which supports our hypothesis that the learned predictive representations, rather than the explicit prediction outputs, are more beneficial to the generative process.
Effect of ViT block number. To assess the extent of predictive ability required to benefit the generative process, we vary the number VIT blocks M in the predictive stream from vanilla diffusion) to M 12, while keeping the number of DiT blocks in the generative stream fixed. As shown in Figure 8b adding moderate number of ViT blocks noticeably improves performance, but further increasing M yields diminishing gains. This suggests that the predictive ability required to assist a fixed generative backbone can be achieved with minimal overhead, and that even lightweight deterministic auxiliary module can provide meaningful improvements.
(i.c.,
Effect of design beyond parameter scaling. We further validate that the performance gains of ForeDiff are attribuiable to its architectural design rather than the simple scaling of parameters. To isolate the effect of parameter scaling, we extend the vanilla diffusion model to 18 DiT blocks to match the total number of layers used in ForeDiff, and compare its performance with both ForeDiff-ZZ ForeDiff under identical VIT/DiT block configurations. We also evaluate the standalone deterministic predictive stream used in Fore As shown in Figure 8c ForeDiff outperforms both the extended vanilla diffusion model and the deterministic stream by substantial margin. These results indicate that the combination of deterministic prediction and conditional diffusion contributes synergistically, highlighting the effectiveness of the proposed hybrid architecture for predictive learning.
and
8

---

Published as a conference paper at ICLR 2026
15-
ForeDif (Read outpuis)
defamit
Diftiiio
HeteerS Itelutvve
Vanilla Diffusson (extended)
1P1PS
LPIPS
Dateeministi Prodiction
FoiDDir
112
RoboNet RT-1
HeterNS
VIT/DIT block numbers
RoboNet
RT-1
(a) Effect PredHead.
Effect of #ViT block.
(c) Effect beyond parameter scaling,
Figure 8: Ablation studies. (a) Conditioning PredHead outputs leads to degraded performance. (b) Increasing #VTT block improves accuracy, but further blocks yield diminishing gains. (c) ForeDiff outperforms both an extended vanilla diffusion and the standalone deterministic predictor.
RELATED WORK
Predictive Learning. Diffusion models have been widely adopted for predictive learning. Earlier approaches have explored architectural design for incorporating signals within the conditional
generation framework, including concatenation- and modulation- fusion (Voleti et al. 2022 Ho
al. 2022 Blattmann et al. 2023) with U-Net backbones, scalable multi-modal transtormer bbsoeder-bbaed
architectures (OpenAl 2024; Yang et al. 2024) and domain-specific architectures (Gao et al. 2023).
Beyond diffusion- approaches. various frameworks have been proposed for predictive tasks, including RNN-based (Shi et al. 2015; Villegas et al., [2099] Wang et al. (2022). auto-regressive (Yan) al. 2021; Wu et al. 2024), and mask-based (Gupta et al. [023] Yu al.) (2023) methods. As concluded in Section 3.1 diffusion models serve strong baselines but tend to exhibit greater sample
variability compared these non-diffusion counterparts. This observation motivates our focus on enhancing sampling consistency in predictive diffusion models.
Diffusion models components. While many approaches employ diffusion models in end-to- end manner, recent studies have integrated them as components within broader architectures. For example, Li al. (2024) and Liu et al. (2025) incorporate diffusion losses for image mask reconstruc- tion and auto-regressive time-series forecasting, respectively, thereby leveraging the generative ability in modular way. By comparison, Foresight Diffusion positions the diffusion model the central component, enhances its consistency within the generation framework while assigning deterministic modules an auxiliary role tailored for condition understanding prediction tasks.
Sampling methods. Beyond standard sampling strategies as Eq. 43, prior work has introduced post-training techniques to address imperfections in diffusion model learning, including classi- fier (Dhariwal & Nichol 2021) or classifier- free (Ho & Salimans, 2022) guidance, initial noise manipulation (Ahn et 2024, Zhou et al.] (024)) and inference-tine scaling (Ma et al., 2025). While effective for enhancing sample quality or conditioning, these methods serve as complementary post-hoc techniques alongside existing architectures, operating orthogonally ForeDiff.
CONCLUSION
We proposed Foresight Diffusion (ForeDiff), a framework that improves sampling consistency in predictive diffusion models by decoupling condition understanding from target denoising. Through a hybrid architecture that separates predictive and generative processes, and a two-stage training scheme that leverages pretrained deterministic predictors, ForeDiff overcomes key limitations of vanilla diffusion models particulaarlly their suboptimal predictive ability and high sample variance. Extensive experiments across real- gal-world robot video prediction and scientific forecasting demonstrate that ForeDiff achieves superior accuracy and significantly enhanced sampling consistency, marking a step toward reliable and controllable predictive diffusion models.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was supported by the National Natural Science Foundation of China (U2342217), the Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China (JY B2025XDXM803), the Beijing Scholar Program, and the National Engineering Research Center for Big Data Software.
We thank our colleagues for their support, especially Yuchen Zhang and Yuezhou Ma for valuable discussions, as well as Haoran Zhang and Huakun Luo for helpful suggestions on figure design.
REFERENCES
Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chat- topadhyay, Yongxin Chen, Yin Cui, Yifan Ding, al. Cosmos world foundation model platform for physical al. arXiv, 2025.
Donghoon Ahn, Jiwon Kang, Sanghyun Lee, Jaewon Min, Minjae Kim, Wooseok Jang, Hyoungwon Cho, Sayak Paul, SeonHwa Kim, Eunju Cha, et al. A noise is worth diffusion guidance. arXiv preprint arXiv:2412.03895, 2024.
Sumukh K Aithal, Pratyush Maini, Zachary Lipton, and J Zico Kolter. Understanding hallucinations in diffusion models through mode interpolation. In NeurIPS, 2024.
Michael S Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants. In ICLR, 2023.
Mohammad Babaeizadeh, Mohammad Taghi Saffar, Suraj Nair, Sergey Levine, Chelsea Finn, and Dumitru Erhan. Fitvid: Overfitting in pixel-level video prediction. arXiv, 2021.
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In CVPR, 2023.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Kuang-Huei Lee, Sergey Levine, Yao Lu, Utsav Malla, Deeksha Manjunath, Igor Mordatch, Ofir Nachum, Carolina Parada, Jodilyn Peralta, Emily Perez, Karl Pertsch, Jornell Quiambao, Kanishka Rao, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Kevin Sayed, Jaspiar Singh. Sumedh Sontakke, Austin Stone, Clayton Tan, Huong Tran, Vincent Van- houcke, Steve Vega, Quan Vuong, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. Rt-1: Robotics transformer for real-world control at scale. arXiv, 2022.
Xinyuan Chen, Yaohui Wang. Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang. Dahua Lin, Yu Qiao, and Ziwei Liu. Seine: Short-to-long video diffusion model for generative transition and prediction. ICLR, 2023.
Sudeep Dasari, Frederik Ebert, Stephen Tian, Suraj Nair, Bernadette Bucher, Karl Schmeckpeper, Siddharth Singh, Sergey Levine, and Chelsea Finn. Robonet: Large-scale multi-robot learning. In CoRL, 2019.
Prafulla Dhariwal and Alexander Nichol. Diffusson models beat gans on image synthesis. In NeurIPS, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2020.
Zhihan Gao, Xingjian Shi. Boran Han, Hao Wang, Xiaoyong Jin, Danielle Maddix. Yi Zhu, Mu Li, and Yuyang Bernie Wang. Prediff: Precipitation nowcasting with latent diffusion models. In NeurIPS, 2023.
10

---

Published as a conference paper at ICLR 2026
Agrim Gupta, Stephen Tian, Yunzhi Zhang, Jiajun Wu, Roberto Martin-Martin, and Li Fei-Fei. Maskvit: Masked visual pre-training for video prediction. In ICLR, 2023.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv, 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 2020.
Jonathan Ho. Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. In NeurlPS, 2022.
Quan Huynh- Thu and Mohammed Ghanbari. Scope of validity of psnr image/video quality assessment. In Electronics letters, 2008.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In NeurlPS, 2022.
Tianhong Li. Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. In NeurlPS, 2024.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In ICLR, 2021.
Yaron Lipman, Ricky TQ Chen, Heli Camuu Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In ICML, 2023.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2023.
Yong Liu, Guo Qin, Zhiyuan Shi, Zhi Chen, Caiyin Yang. Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Sundial: A family of highly capable time series foundation models. In ICLR, 2025.
Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric -ijnnen, and Saining Xie. Sit: Exploring flow and fiffusiin-based generative models with scalable interpolant transformers. In ECCV, 2024.
Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, al. Inference-time scaling for diffusion models beyond scaling denoising steps. arXiv preprint arXiv:2501.09) 09732, 2025.
OpenAl. Sora: Video generation models as world simulators. httpps://oppea.... com/ research/video generation models moneeliinnoddel----- as world simulators, 2024. Accessed: 2024-04-01.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang. Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. NewrIPS, 2022.
Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. Convolutional Istm network: A machine learning approach for precipitation nowcasting. In NeurlPS, 2015.
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-vided Text-to-video generation without text-video data. In ICLR, 2022.
Jascha Sohl- Dickssies,,,, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In 2015.
11

---

Published as a conference paper at ICLR 2026
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In NeurlPS, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2020.
Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv, 2018.
Ruben Villegas, Arkanath Pathak, Harini Kannan, Dumitru Erhan, Quoe V Le, and Honglak Lec. High fidelity video prediction with large stochastic recurrent neural networks. In NeurlPS, 2019.
Vikram Voleti, Alexia Jolicoeur-Martineau and Chris Pal. Mevd-masked conditional video diffusion for prediction, generation, and interpolation. In NeurIPS, 2022.
Yunbo Wang, Haixu Wu, Jianjin Zhang, Zhifeng Gao, Jianmin Wang, S Yu Philip,, and Mingsheng Long. Predrnn: A recurrent neural network for spatiotemporal predictive learning. In TPAMI, 2022.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. In TIP, 2004.
Bohan Wu, Suraj Nair, Roberto Martin-Martin Li Fei-Fei, and Chelsea Finn. Greedy hierarchical variational autoencoders for large-scale video prediction. In CVPR, 2021.
Jialong Wu, Shaofeng Yin, Ningya Feng, Xu He, Dong Li, Jianye Hao, and Mingsheng Long. ivideogpt: Interactive videogpts are scalable world models. In NeurlPS, 2024.
Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas. Videogpt: Video generation using vq-vae and transformers. In CVPR, 2021.
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv, 2024.
Lijun Yu, Yong Cheng, Kihyuk Sohn, José Lezama, Han Zhang, Huiwen Chang, Alexander G Hauptmann, Ming-Hsuan Yang, Yuan Hao, Irfan Essa, et al. Magvit: Masked generative video transformer. In CVPR, 2023.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as a perceptual metric. In CVPR, 2018
Hang Zhou, Yuezhou Ma, Haixu Wu, Haowen Wang, and Mingsheng Long. Unisolver: Pde- conditional transformers are universal pde solvers. In ICML, 2025.
Zikai Zhou, Shitong Shao, Lichen Bai, Shufei Zhang, Zhiqiang Xu, Bo Han, and Zeke Xie. Golden noise for diffusion models: A learning framework. arXiv preprint arXiv:2411.09500) 2024.
12

---

Published as a conference paper at ICLR 2026
A PROOF OF LEMMA
In this section we present the proof of Lemma 3.1
Proof. The proof structured in three parts: (1) & reparameterization, (2) e-agnostic property of
the optimal solution Kg, and (3) the reduction to deterministic prediction model fe in practice.
Starting from definition Xy (x(x,,t,> = X2 - vo(xe, t, y) with x; = (1 - t)xo + te, it holds that
Crrea (0) Exx.yy..t
|| (x (, ( c)
(x(x,,,,)) Xt Xo x,
(x (x)),,,,))))))) xoll
Exx.
and by letting 1 we arrive
(flt = 1) = Ex..y.e | 1,y) 1xo0))xoll))) = Ex..ve [ - x012
x0112
Since is independent of both network input y and target Xy, the involvement of e contributes no information in the case of t = 1, which matches the fact that e is sampled randomly.
Further, notice the varianar decomposition
Exp.y.e [|2oo(e, -x,1)
xo112] By E. (0 1,y)] Exo by [xo]
(e, D [x0]
By the convexity of the & loss, there exists eo(y) satisfying
(x)( Ex,ly [xol [e Exoly |xolv/xo]11
Therefore,
E. Be( 1, Exily (xo) || + Dxoly
60]1 By
2
2 Ey
>o((o(l); 1,y) Exxoyy [xo]
+ Dxx (Dxooylls [20]
1
Exo.y (xo(Eo(y))1,)) 1,y) xo
ie., the loss of diffusion models at is bounded by that of certain deterministic model f(y) = 8o(Eo(y),,, 1,y) which minimizes the following objective:
Laster = Exp.y [IIs(x) x
Finally, show that the inequalities can be turned into equalities in practical network architectures. which requires ( 1, = ( 1,y) for any 61 C2. Considering the first layer to process e as In this case, with b and absorbed into
We b, setting W 0 removes the dependence the network parameters, (o(e, 1, y, reduces to deterministic model (y). completing the proof.
13

---

Published as a conference paper at ICLR 2026
B IMPLEMENTATION DETAILS
We adopt the widely used latent diffusion paradigm, which operates in a compressed latent space and relies on pretrained autoencoder consisting of an encoder and a decoder. The autoencoder downsamples the input by factor of 4 x and maps it to latent representation with 3 channels. All prediction and denoising operations are performed this latent space. The autoencoder archi- tecture follows the standard design of Stable Diffusion (Rombach et al. 2022), and dataset-specific architectural and training details are summarized T
Table 5: Architecture and training details of the pretrained autoencoder for each dataset.
RoboNet
RT-1
HeterNS
Parameters
55M
Resolution
64 64
Input channels
3
3
1
Latent channels
Down blocks
3
Down layers per block
Down channels
[128. 256, 512] True
Mid block attention
Up blocks
Up layers per block
Up channels
[512, 256, 128] GroupNorm 32 SiLU
Normalization
Norm groups
Activation
Training steps
106
3 106
3 106
Discriminator start step
5 x 101
Batch size
Learning rate LR schedule Optimizer
4 x 10-5
Constant Adam
Our main model follows the DiT (Peebles & Xie 2023) and VIT (Dosovitskiy et al. 2020) architecture families, using unified design across all tasks. We adopt ViT-S and DIT-S configurations for the predictive and generative streams, respectively. Patch embeddings are extracted using minimal PatchEmbed module that partitions inputs into zon-overlapping 2 x patches, followed by linear projection. Sinusoidal positional encodings are used, with embedding dimensions partitioned in 3:3:2 ratio among the two spatial axes and the temporal axis. Detailed block settings are shown in Table All models are trained using the AdamW optimizer with constant learning rate of x 10 and dataset-specific training steps (see Appendix
Table 6: Architecture settings for ViT-S and DiT-S blocks used in ForeDiff.
ViT-S (predictive stream)
DiT-S (generative stream)
Number of blocks
6
12
Hidden size
384
384
MI.P ratio
4.0
4.0
Attention heads
6
Positional encoding
Sinusoidal
Sinusoidal
Normalization
LayerNorm
AdaLN
Activation
SiLU
SiLU
To construct the conditioning input, follow the masking- based strategy introduced in prior work (Blattmann et al. 2023). Specifically, temporal binary mask m E [0,1]0+5 defined over the full sequence s-0 - to indicate which frames observed (condition) and which are to be
14

---

Published as a conference paper at ICLR 2026
predicted (target), where O and S denote the numbers of observed and future frames, respectively After encoding the full sequence into the latent space via z E(s-00+1:5 R(O-S)xC*HXX we apply the mask m along the temporal axis by computing Zcomd m z, where the mask is broadcast across spatial and channel dimensions. To make the temporal structure explicit, we further concatenate the binary mask itself (channel-wisc) to the masked latent Zoond, and feed the result into the predictive stream.
When additional conditioning inputs such as actions or goals с are available (e.g., in RoboNet and RT-1), we directly concatenate them with the masked latent Zoodd along the channel axis. This enables the model to jointly reason over visual history and auxiliary task-specific signals.
Overall, this masking-basee based design supports unified conditioning mechanism for both video- only and video+action tasks. The masked latent input, along with any auxiliary information (e.g.. actions), is processed by the predictive stream to produce intermediate features gM, which serve as conditioning signals for the generative stream.
To inject this condition information into the generative stream, composed of an adaptive layer normalization (AdaIN) layer followed by two-layer MLP with GELU activation. The fusion operation is defined as:
lightweight Fusion module
h MLP(AdaLN( ((o: gM),
where ho denotes the noisy target, gMr is the output of the predictive stream (prior to the PredHead module), and is the diffusion timestep. We first concatenate ho and gM. apply AdaLN conditioned on t, and then pass the normalized features through the MLP to obtain the fused representation.
C EXPERIMENTAL SETUP
We evaluate ForeDiff on three benchmark datasets covering both real-world and simulated spa- tiotemporal dynamics: RoboNet (Dasari et al. 2019), RT-1 (Brohan et al. 2022), and HeterNS (Li
et al. 2021] Zhou et al.) 2025). This section provides additional details on dataset setups, task configurations, and evaluation metries. All experiments conducted in a single NVIDIA-A100 40G GPU.
RoboNet. RoboNet is large-scale real-world video dataset for vision-based robotic manipulation, consisting of approximately 162,000 trajectories collected across seven different robot platforms from four institutions. Each trajectory includes RGB video frames and associated action sequences, all represented in unified end-effector control space. The dataset captures wide range of variations in robot embodiment (e.g., Sawyer, Kuka, Franka), gripper design, camera viewpoints, surfaces, and lighting conditions.
Following prior works (Babaeizadeh et al. 202 Wu et al. 2024), we resize all frames to 64 x 64 and predict 10 future frames based on observed frames and corresponding actions. All models trained on RoboNet for x 10 steps with batch size of 16. We report PSNR, SSIM, LPIPS, and FVD as evaluation metrics. In addition, we compute standard deviation (STD) across multiple samples to quantify sampling consistency.
RT-1. RT-1 is a large-scale real-world robotics dataset comprising over 130,000 episodes collected from fleet of 13 mobile manipulators performing diverse manipulation tasks in office kitchen environments, Each episode includes RGB video sequence, a natural language instruction, and the executed robot actions. The dataset covers over 700 distinct tasks involving object interaction, long-horizon manipulation, and routine execution.
We formulate the prediction task by conditioning on 2 observed frames and task instruction, and predicting the next 14 frames. All frames are resized 64 x 64. Training on RT-1 is conducted for 5 x 10a steps with batch size of 16, and the evaluation protocol matches that of RoboNet, using PSNR. SSIM, LPIPS, FVD, and STD of these metrics across samples.
HeterNS. HeterNS is a synthetic dataset generated by numerically solving the two-dimensional incompressible Navier-Stokes equations in vortieity formulation periodic domain. The
15

---

Published as a conference paper at ICLR 2026
governing system is defined as:
0, 1) + u(x, Vw(22, 8) = v vAw(a, f(x), e (0, 1)3 te (0. T)
(9a)
V u(z,t) t) O, w(2r, 0) (x())
x e (0, 1)², te [0,T],
(9b)
x E (0, 0)²;
(9c)
where w is the vorticity field, u is the divergence- free velocity field recovered via the stream function & satisfying Aeb = -w, y/ is the viscosity coefficient, and f(x is a time-invariant external forcing term.
To construct a diverse collection of PDE instances, we vary two key physical parameters: the viscosity and the structure of the forcing term far). Specifically, the training set is constructed from Cartesian grid of parameter configurations:
E {1 10-5, 1 1x 10-*, x 10- f(x) (5 distinct variants),
yielding total of 3 = 15 unique PDE settings. Among the forcing terms, three are defined as:
f(x) = 0.1 (sin(w1 T + z2)) + cos(w27((1) + x2))))
with frequency pairs (w1, E ((2,)), (2,4), (4, 4)) The fourth variant defined as: f(x) = 0.1 (sin (21 x22) + cos(2 - x22))),
and the fifth as:
f(x) = 0.1 (sin(2m x cos(2 (z)
These five variants cover range of spatial patterns, including directional, cross-diagonal, and radially symmetric forcings.
For each of the 15 configurations, simulate 1000 trajectories, resulting in total of 15,000 samples. Each trajectory consists of 20 vorticity fields at a spatial resolution of 64 x 64. The prediction task involves forecasting the final 10 frames given the first 10 context, Each model is trained for x 10 steps using a batch size of 16.
Evaluation metries. We adopt a suite of evaluation metrics to assess model performance across multiple dimensions, including visual quality, perceptual fidelity, predictive accuracy, and sampling consistency. These include PSNR, SSIM, LPIPS, FVD, Relative L2, and STD, described as follows:
Peak Signal-to-Noise Ratio (PSNR) (Huynh- Thu & Ghanbari 2008) quantifies the ratio be- tween the maximum possible pixel intensity and the distortion introduced by reconstruction errors. It is defined in logarithmic scale (dB) and commonly used to evaluate frame-wiss reconstruction quality. A higher PSNR value indicates less pixel-level distortion and better fidelity to the original ground truth frame.
Structural Similarity Index Measure (SSIM) (Wang et al. 2004) measures perceptual similarity by comparing luminance, contrast, and structural information between images. Unlike PSNR, SSIM better correlates with human visual perception. The SSIM score ranges from 1 to 1, where 1 denotes perfect structural similarity. For better readability, we scale SSIM scores by a factor of 100 in all reported results.
Learned Perceptual Image Patch Similarity (LPPPSS) (Zhang et al.) 2018) evaluates perceptual similarity using deep features extracted from a pretrained neural network (e.g.. VGG or AlexNet). It computes the L2 distance between feature representations of two images, capturing differences beyond pixel values. Lower LPIPS values indicate better perceptual quality. Similar SSIM, LPIPS scores multiplied by 100 for display.
Fréchet Video Distance (FVD) (Unterthiner et al. 2018 extends the Fréchet Inception Distance (FID) to video generation by considering temporal dynamics. It compares the distributions of real and generated videos in feature space extracted from pretrained video recognition model. FVD is sensitive to both spatial quality and temporal consistency, making strong indicator of overall generative performance.
16

---

Published as a conference paper at ICLR 2026
Relative L2 Distance (Li et al. 2021) captures normalized regression emror in pixel or field-level prediction tasks. It is computed as:
Relative 1.2 | 2|12
|
where å and x denote the prediction and ground truth, respectively. A lower relative 1,2 value implies that the model produces more accurate outputs with respect both magnitude and structure, This metric is particularly suited for scientific forecasting tasks such as fluid dynamics.
Standard Deviation (STD) of metrics is used to evaluate sampling consistency across multiple generations from the same condition. For each condition input (e.g., past frames and action/insttruction) metric M (e.g., PSNR, SSIM, LPIPS) for each sample, and calculate the standard deviation of these scores. Let M M denote the metric values for the i-th condition, the per-condition standard deviation is:
generate N samples (typically N 100), compute chosen
N (M - MO) where M
N
STD
Σ
M
N
N
==1
The final STD score is then averaged over all C conditions:
STD Σ STD
Lower STD indicates that the model produces consistent outputs across stochastic samples, reflecting stronger reliability under the same input condition.
D MORE EXPERIMENTAL RESULTS
This section provides the full numerical results corresponding to the analyses discussed in Section 4.3 While the main paper focuses on reporting key trends and visualizations, the tables below include complete metric values across datasets and variants, serving as quantitative supplement.
In addition, we include extended experiments covering: (i) the necessity of architectural decoupling, (ii) the necessity the two-stage training scheme, (iii) the sensitivity of ForeDiff to predictor quality, and (iv) calibration-oriente evaluation (CRPS, NLL, and coverage curves). These results further support the conclusions drawn in the main text.
Effeet of PredHead module. Table 7 reports the full numerical results for the PredHead ablation across all datasets.
Table 7: Ablation results on the PredHead module. Across datasets, removing PredHcad consis- tently improves both perceptual and pixel-wise metrics. SSIM, LPIPS, L2, and Relative L2 are scaled by 100.
Dataset
Method
FVD 4
PSNR
SSIM + LPIPS
ForeDiff
88.8
5.25
RoboNet
ForeDiff (with PredHead)
53.7
27.3
88.7
5.35
ForeDiff
12.0
31.2
94.4
3.42
RT-1
ForeDiff (with PredHead)
12.4
31.0
94.1
3.60
Dataset
Method
L.2 4
Relative L22 4
ForeDiff
0.19
0.18
HeterNS
ForeDiff (with PredHead)
0.23
0.20
Effect of VIT block number. Table x presents detailed results varying the number of predictive (ViT) blocks, while keeping the number of generative (DiT) blocks fixed.
17

---

Published as a conference paper at ICLR 2026
Table 8: Effect of varying ViT block number (denoted by M) on performance, with DiT blocks fixed to 12. Adding moderate number of predictive blocks improves performance, while further increases yield diminishing returns. SSIM and LPIPS are scaled by 100.
Dataset
Method
FVD 4
PSNR 1
SSIM
LPIPS +
M=0 (vanilla)
53.8
27.1
88.2
5.65
M=3
53.3
27.1
88.3
5.51
RoboNet
(default)
51.5
27.4
5.25
M=9
50.8
27.5
89.0
5.17
M=12
52.1
27.5
89.1
5.14
M=0 (vanilla)
11.7
30.4
93.6
3.79
M=3
11.8
31.0
94.2
3.49
RT-1
M=6 (default)
12.0
31.2
3,42
M-9
12.4
31.3
94.4
3.41
M=12
12.4
31.3
94.4
3.41
Effect of design beyond parameter scaling. Table Paaleeprrovide.. the complete quantitative results for the ablation study on HeterNS dataset, used to isolate the effect of architectural design from mere parameter scaling.
Table 9: ForeDiff clearly outperforms both deterministic-one and extended vanilla diffusion models, confirming that its improvements stem from architectural design rather than model size alone. Metrics scaled by 100.
Method
L2 J
Relative L2 +
Deterministic Prediction
1.06
0.97
Vanilla Diffusion
1.73
1.50
Vanilla Diffusion (extended)
1.29
1.14
ForeDiff-Zero
1.03
0.83
ForeDiff
0.19
0.18
Necessity of architectural decoupling. We examine whether training-scheme decoupling alone, without architectural separation, is sufficient to disentangle condition understanding from target denoising. To this end, we first trained a diffusion model exclusively at timestep to mimic the behavior of deterministic predictor (see Lemma B.1.. and then fine-tuned it across all timesteps with uniform weighting. We refer this variant as training-only decoupling, since its architecture remains unchanged while the training procedure is modified through pretraining at
enabling
t 1 followed by fine-tuning. Table I0 reports the quantitative results on RoboNet. Such pretraining improves sampling consistency but still falls short of ForeDiff in both consistency and overall quality, most notably yielding worse FVD than vanilla diffusion. These findings align with our discussion in Sectior showing that parameter sharing between condition understanding and target denoising imposes an unavoidable dual-role constraint, thereby further validating the necessity of our joint decoupling design.
Table 10:
Necessity of architectural decoupling. Results comparing (1) Vanilla Diffusion, (2) Vanilla Diffusion with training-only decoupling (pretraining at / followed by fine-tuning across timesteps), and (3) ForeDiff with both training- and architectural decoupling.
Method
FVD J
PSNR t
SSIM
LPIPS 4
STDPsNR +
STDssns +
STD1rus
Vanilla Diffusion
53.8
27.1
88.2
5.65
0.66
1.33
0.65
Pretraining f =1
58.2
27.3
88.5
5.53
0.51
0.98
0.48
ForeDiff
51.5
27.4
88.8
5.25
0.37
0.70
0.35
Necessity of two-stage training. We suppose that jointly training the full architecture with composite loss but two-stage will reintroduces the entanglement problem that Fofe is designed
18

---

Published as a conference paper at ICLR 2026
to resolve. To validate this empirically, implemented the joint training variant using Gjiint Lonnose + deeeur and trained models on RoboNet with À = 0.1 and = 1. The results in Table [1 reveal clear pattern. With À 0.1. joint training brings only mild gains, but both the overall fidelity and the reduction of sample variance remain noticeably weaker than ForeDiff. Increasing the weight to À strengthens variance suppression but begins to harm generative quality, reflected by the increased -howing that the predictive and generative streams start to interfere with one another. In contrast, the full ForeDiff design consistently achieves the strongest performance across all metrics, indicating that only the two-stage training decoupling can stabilize predictive representations and support effective denoising.
Table 11: Necessity of two-stage training. While joint training can offer mild gains, ultimately converges to compromised middle ground. The two-stage decoupled design of ForeDiff remains the most effective and stable solution for leveraging deterministic foresight while preserving diffusion's ss generative strengths.
Method
FVD Y
PSNR
SSIM
LPIPS
STDPsNR + STDssIM +
STDurs 4
Vanilla Diffusion
53.8
27.1
88.2
5.65
0.66
1.33
0.65
ForeDiff-zero
52.7
27.2
88.4
5.54
0.68
1.36
0.66
joint training (A 0.1)
51.6
27.3
88.6
5,40
0.66
1.29
0.63
joint training (λ
52.7
27.3
88.6
5.36
0.48
0.92
0.44
ForeDiff
51.5
27.4
88.8
5.25
0.37
0.70
0.35
Sensitivity predictor quality. To assess how sensitive ForeDiff to the quality of the predictive stream, retrained the generative stream using predictive streams saved 0.5M and 0.8M iterations (instead the final 1.0M model). Table repports the quantitative results on RoboNet. Interestingly, even 50% of the predictor's training already provides clear improvements over vanilla diffuuson, and the 0.8M predictor performs even slightly better than our default 1.0M configuration. This suggests two things:
ForeDiff does not place strict requirements on predictor quality. Even a not-fully-converge predictor produces sufficiently structured intermediate representations to yield substantial gains.
The default 1.0M predictor may be mildly overfitted; more careful tuning could unlock additional gains. However, to minimize implementation complexity and ensure consistency across datasets, we deliberately adopt a single, unified configuration without fine-grained tuning.
Table 12: Sensitivity to predictor quality. ForeDiff's 's two-stage design yields consistent improvements and imposing critical requirements the predictive pretraining phase.
Method
FVD J. PSNR
SSIM
I.PIS
STDpsMR STDssIM 4 STDLrirs
Vanilla Diffusion
53.8
27.1
88.2
5.65
0.66
1.33
0.65
ForeDiff-zero
52.7
27.2
88.4
5.54
0.68
1.36
0.66
ForeDiff (predictive 0.5M)
51.5
27.2
88.6
5.35
0.48
0.91
0.45
ForeDiff (predictive stream at 0.8M)
50.9
27.3
88.8
5.26
0.39
0.74
0.37
ForeDiff (predictive stream at 1M)
51.5
27.4
88.8
5.25
0.37
0.70
0.35
Calibration-orientedd evaluation. To further verify ForeDiff's reduced variability corresponds to better probabilistic modeling, rather than just collapsing single mode, we computed calibration- oriented metrics including CRPS, NLL and coverage curves, as shown Table 13 and Figure 9 We observe that ForeDiff achieves lower CRPS and NLL, indicating that the samples become both more consistent and better calibrated. In other words, ForeDiff's ' reduced variance is not due to mode collapse but more accurate conditional alignment. For coverage, vanilla diffusion appears closer to the ideal y = x line, but this largely arises from inflated uncertainty: when the conditional mean is biased, larger variance (directly reflected by larger STD) artificially improves coverage. In contrast,
19

---

Published as a conference paper at ICLR 2026
ForeDiff yields narrower yet better-centered predictive distributions, leading to lower CRPS/NLL and more faithful calibration.
Table 13:
Calibration evaluation. ForeD- iff achieves improvement both consis- tency and sample/distribution similarity, confirming that ForeDiff enhances pre- dictive reliability without collapsing single deterministic mode.
Vanila Forehiiff Moxi
FoeeDeil Ideal
Eugiical comel
Impprcal
Dataset
Method
CRPS
NLL 4
422
RoboNet
Vanilla Diffusion ForeDiff
0.0175
0.0173
2.30
2.58
42
Nontial covergee
85
16
1/6 coverage
(d) RoboNct.
(c) RT-1.
Vanilla Diffusion
0.0157
-0.96
RT-1
ForeDiff
0.0128
-1.19
Figure 9: Coverage calibration curves.
E LIMITATIONS AND FUTURE WORK
While Foresight Diffusion demonstrates improvements in both predictive accuracy and sampling consistency, several avenues remain open for further exploration and could inspire future work:
Lack of large-scale validation. We did not perform large-scale experiments involving substantially larger models or training datasets. This choice primarily due to the substantial computational cost of scaling diffusion models, which often exceeds the capacity of academic research teams. Despite this, our setup follows established practices in prior work, and we believe the proposed method provides a fair and meaningful evaluation under moderate-scale settings. Importantly, our approach introduces algorithmic innovations that are orthogonal to scaling laws: ForeDiff does not alter the underlying generative paradigm, and thus its benefits should be complementary improvements from scaling. systematic study of scaling remains a valuable direction for future work.
Focus DiT-based architectures. Our experiments focus on DiT-based diffusion backbones, which currently represent the state of the art in video modeling and offer a natural foundation for validating new ideas. Nevertheless, ForeDiff is not tied to DiT-specific components; its de- sign-decoupling condition understanding from denoising- shouuld in principle extend to alternative backbones such as CNN- or hybrid-based diffusion models. Exploring these directions may further broaden the applicability of our approach.
Scope restricted to diffusion models. This study centers on predictive diffusion models, which currently achieve tfte-offthh-ar results standard benchmarks. While alternative model families (e.g., auto-regressive or energy-based based models) have demonstrated strengths in specific aspects, diffusion remains the most competitive framework in terms of overall predictive quality. Our work enhances diffusion further by improving both accuracy and consistency, ensuring it does not lag behind other approaches in robustness. We expect that the key insights of ForeDiff-particullrly the disentanglement of condition understanding and target denoising may generalize beyond diffusion, and extending similar architectural and training decoupling strategies other generative paradigms is an exciting avenue for future research.
Taken together, these limitations highlight promising directions for scaling, architectural diversity, and model family generalization, which we hope will guide future advances in predictive modeling.
20


Reviews:
[{"review_id": "vvl5p4yHGn", "reviewer": "Reviewer_bbPz", "summary": "This paper addresses a key discrepancy between the typical use of diffusion models for generative tasks and their application to predictive learning. While generative tasks often value sample diversity, predictive tasks (like video forecasting) require sampling consistency, where generated samples are tightly clustered around a single, physically plausible ground-truth future. The authors hypothesize that the suboptimal consistency of standard predictive diffusion models stems from the entanglement of condition understanding and target denoising within shared architectures and co-training schemes.\n\nTo solve this, the paper proposes Foresight Diffusion (ForeDiff), a framework that disentangles these two roles. ForeDiff consists of two main contributions:\n\n1. A Decoupled Architecture consists of a condition understanding branch and a generative branch.\n\n2. A Two-Stage Training Scheme that enables the predictive branch develops a strong foresight which can later provide a strong conditioning signal to the generative branch.\n\nExperiments on robotic video prediction and scientific spatiotemporal forecasting demonstrate that ForeDiff significantly improves both predictive accuracy and sampling consistency.", "questions": "1. Following on Weakness 1, how sensitive is the final ForeDiff model to the quality of the pretrained deterministic predictor from Stage 1? For instance, if the predictive branch is overfit, or trained for too few steps, does this significantly harm the generative stream's performance or consistency?\n\n2. Could the authors comment on the potential of the setting of jointly training the full architecture with a composite loss, i.e.,\n$\\\\mathcal{L}$ = $\\mathcal{L}\\_{denoise}$ + $\\lambda \\mathcal{L}\\_{deter}$? This ablation studies can further demonstrate the effectiveness of the two-stage training method.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This is a clear and well-motivated paper. It identifies an important issue with applying diffusion models to predictive tasks and proposes an effective solution. The experimental results on various tasks and multiple ablations studies provide evidence for the authors' claims. This work can potentially be applied to various fields such as forecasting, robotics, and scientific ML.", "weaknesses": "1. The proposed two-stage training scheme, while effective, introduces additional complexity to the training pipeline compared to a single-stage, end-to-end model. It requires a separate pretraining phase for the predictive branch, which may add to the overall engineering effort, training time, and need for hyperparameter tuning. A discussion of this trade-off (implementation complexity vs. consistency gain) would be beneficial.\n\n2. Using a frozen, deterministic predictor could also be a potential limitation. Predictive learning is not always purely deterministic; there can be legitimate stochasticity or multi-modality in the dataset. Forcing the model to foresee the future and training with a simple $L_2$ loss might suppress the model's ability to capture valid multi-modality, collapsing all potential futures into some form of averaged future prediction.\n\n3. The generative network is fundamentally conditioned on the output of the frozen predictive stream. If the predictive stream makes a significant error, the generative network has no mechanism to correct this. It is forced to denoise towards a faulty premise. Are there any ways to mitigate such scenarios?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "V00JasLrfK", "reviewer": "Reviewer_6hoQ", "summary": "This paper introduces Foresight Diffusion (ForeDiff), a framework designed to improve sampling consistency in predictive diffusion models. The authors identify that vanilla diffusion models, while effective for diverse generation tasks, suffer from high sample variance when applied to predictive learning where consistency is crucial. ForeDiff addresses this by architecturally decoupling condition understanding from target denoising through separate predictive and generative streams, and employing a two-stage training scheme with deterministic pretraining.", "questions": "- Do you anticipate ForeDiff will scale well to higher-resolution datasets and larger models (e.g., 256×256 or video generation)?\n- Have you attempted to adapt ForeDiff to non-DiT backbones, such as U-Net or latent diffusion architectures? If not, what are the main obstacles?\n- Since ForeDiff benefits from deterministic pretraining, could you provide a control where the baseline diffusion model is pretrained with a similar deterministic phase? \n- Could you clarify why the deterministic predictor must be trained separately rather than jointly fine-tuned with the diffusion model?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper presents a novel architectural approach by explicitly separating condition processing from denoising, which is a creative departure from standard conditional diffusion models.\n- The focus on sampling consistency as a distinct requirement for predictive tasks versus generative tasks is an important problem formulation.\n- Clear mathematical formulation and proof of the key lemma connecting diffusion and deterministic models\n- Comprehensive experimental evaluation across three diverse datasets (RoboNet, RT-1, HeterNS) covering both real-world robotics and scientific computing domains\n- Shows consistent (but modest) improvements across multiple metrics and datasets\n- Well-structured paper with clear motivation through visual examples (Figures 1 and 2) and illustration of architectural differences (Figure 4)", "weaknesses": "- As acknowledged by the authors, experiments are limited to moderate-scale settings (64×64 resolution, relatively small models). \n- Only DiT-based architectures are evaluated; generalization to U-Net or other diffusion backbones is assumed but not demonstrated\n- The connection between predictive ability and sampling consistency could be more rigorously established\n- The improvement margins, while consistent, are sometimes modest\n- The two-stage training could be seen as unfair comparison since vanilla diffusion does not benefit from deterministic pretraining", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0RnmCQVMOV", "reviewer": "Reviewer_SCh5", "summary": "The paper proposes Foresight Diffusion, which decouples a deterministic predictive stream (ViT blocks processing the condition $y$) from a generative/denoising stream (DiT blocks processing $x_t$), and then adopts a two-stage training scheme: first train the predictive stream as a standalone deterministic predictor with a prediction head, then freeze it and feed its internal representation $g_M$ to guide diffusion generation. The claim is that this design improves “predictive ability” and sampling consistency (lower across-seed variance of PSNR/SSIM/LPIPS) on datasets such as RoboNet, RT-1, and HeterNS.", "questions": "Can you report calibration metrics (e.g., NLL/CRPS/coverage) and analyze their relationship to the STD of PSNR/SSIM/LPIPS, on the same hardware and sampling budgets?\n\nPlease provide equal-budget comparisons (same steps, FLOPs, memory, sampling steps) between (i) end-to-end shared models and (ii) your decoupled two-stage pipeline, along with throughput/latency and memory use at training and inference.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The method is simple and relatively easy to reproduce: the architectural split and the two-stage schedule are clearly described, and the paper includes ablations on the prediction head and the number of ViT blocks in the predictive stream. Some datasets show moderate improvements in PSNR/LPIPS and reduced reported variability, and removing the prediction head appears beneficial across tasks per the appendix tables.", "weaknesses": "(1) Limited novelty and weak theory. The main idea—strengthening a condition encoder via standalone pretraining and then freezing its features to condition the denoiser—tracks common practice in conditional diffusion and teacher-feature guidance. Theoretical support is thin: the central formal argument reduces the $t=1$ case to a deterministic model by zeroing the first-layer weights, which does not yield general consistency or error bounds for multi-step diffusion.\n\n(2) Consistency is proxied narrowly and may conflate “agreement” with “accuracy.” Using the STD of PSNR/SSIM/LPIPS across repeated samples risks rewarding collapse or overly strong reliance on the condition signal. The paper lacks calibration-oriented metrics (e.g., NLL/CRPS/coverage) or uncertainty diagnostics to demonstrate that reduced variability corresponds to better probabilistic modeling, rather than just more concentrated—but possibly biased—outputs.\n\n(3) Distributional quality gains are not robust. On RT-1, the reported FVD degrades as predictive blocks increase (e.g., baseline 11.7 vs. 12.0 at the default $M=6$, undermining the claim that decoupling and two-stage training consistently improve generative quality. The paper should transparently analyze this trade-off.\n\n(4) Ablations do not fully isolate training-scheme and compute confounds. Although the appendix contrasts “training-only decoupling” (pretrain at $t=1$ then fine-tune) with the architectural variant, the study still does not establish equalized budgets (steps/FLOPs/memory/denoising steps) between end-to-end shared vs. decoupled + frozen designs. Nor are throughput/latency or memory overheads reported for the extra predictive stream and the freeze-and-feed pipeline.\n\n(5) Baselines and task breadth are insufficient. The paper mostly compares to its own variants and standard conditional diffusion. For scientific/physical forecasting, head-to-head evaluations against strong PDE/forecasting models (e.g., modern operator-learning and transformer baselines) are missing; for video prediction, broader masked/autoregressive competitors would better position the claimed advantages.\n\n(6) Method details and limits remain under-explained. The Fusion mechanism and temporal conditioning choices are specified but not systematically stress-tested for long-horizon prediction or out-of-distribution conditions; appendix tables list numbers (e.g., PredHead effects) without enough analysis about when and why the components help or hurt.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tlALDRdk1J", "reviewer": "Reviewer_Cbg4", "summary": "Diffusion models have achieved remarkable success in generative tasks, but they struggle with predictive learning, where sampling consistency with real trajectories is crucial. The authors identify this limitation as stemming from the entanglement of condition understanding and target denoising in shared architectures. To address this, they propose Foresight Diffusion (ForeDiff), which separates deterministic condition processing from stochastic denoising, leading to improved predictive accuracy and sampling consistency across robotic and spatiotemporal forecasting tasks.", "questions": "1. What is the difference between the conditioning injection mechanism in Stable Diffusion and that in ForeDiff-Zero? If they are similar, does Stable Diffusion already possess the capability to disentangle generation and conditional guidance?\n\n2. How does the proposed method compare to using a deterministic predictor directly for prediction tasks?\n\n3. Many text-to-diffusion methods also use textual information as conditional input. How would textual conditioning be incorporated into ForeDiff-Zero?\n\n4. The predictive task discussed here resembles the concept of a world model, which has recently gained significant attention. What is the conceptual and practical difference between your method and existing world models?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* This paper is overall well-written and easy to follow. The authors start from the discussion of the difference between standard generation tasks and predictive tasks via diffusion model. The predictive tasks require more accuracy on the dynamic, while diffusion model’s generation process has a large uncertainty. The investigated problem is interesting and practical.\n\n* The motivation of the proposed method, which is to detangle the generation and conditional guidance, is reasonable and clear.\n\n* The empirical results demonstrate the good results over vanilla diffusion model baseline on predictive tasks.", "weaknesses": "* While the motivation is clear and conceptually sound, the qualitative justification remains weak. For instance, in Figure 3(b), the authors use LPIPS to measure the similarity between generated and ground-truth samples, concluding that diffusion models exhibit greater uncertainty as they produce both the best and worst cases compared to iVideoGPT. However, it is unclear whether LPIPS is an appropriate metric for assessing dynamic prediction quality. LPIPS primarily measures perceptual similarity, which may not capture temporal or dynamic consistency crucial for predictive tasks.\n\n* The proposed methodological framework appears rather simple and incremental. The main difference lies in injecting the conditioning information into the first DiT block instead of concatenating it with the noised target input. Similar conditioning mechanisms have been used in many existing diffusion models (e.g., Stable Diffusion). If the primary novelty lies only in altering where the condition is injected, the technical contribution seems quite limited.\n\n* The experimental evaluation also lacks breadth. The authors only compare their method against a vanilla diffusion baseline. Without comparisons to other relevant or stronger baselines, it is difficult to convincingly demonstrate the effectiveness of the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)