Diffusion-based Decoupled Deterministic and Uncertain Framework for Probabilistic Multivariate Time Series Forecasting
Authors: Qi Li
Zhenyu Zhang
Lei Yao
Zhaoxia Li
Tianyi Zhong
Yong Zhang
Venue: ICLR 2025 Poster (2025)
Keywords: long-term time series forecasting
deep learning
diffusion model
Primary area: generative models

Abstract: Diffusion-based denoising models have demonstrated impressive performance in probabilistic forecasting for multivariate time series (MTS). Nonetheless, existing approaches often model the entire data distribution, neglecting the variability in uncertainty across different components of the time series. This paper introduces a Diffusion-based Decoupled Deterministic and Uncertain ($\mathrm{D^3U}$) framework for probabilistic MTS forecasting. The framework integrates non-probabilistic forecasting with conditional diffusion generation, enabling both accurate point predictions and probabilistic forecasting. $\mathrm{D^3U}$ utilizes a point forecasting model to non-probabilistically model high-certainty components in the time series, generating embedded representations that are conditionally injected into a diffusion model. To better model high-uncertainty components, a patch-based denoising network (PatchDN) is designed in the conditional diffusion model. Designed as a plug-and-play framework, $\mathrm{D^3U}$ can be seamlessly integrated into existing point forecasting models to provide probabilistic forecasting capabilities. It can also be applied to other conditional diffusion methods that incorporate point forecasting models. Experiments on six real-world datasets demonstrate that our method achieves over a 20\% improvement in both point and probabilistic forecasting performance in MTS long-term forecasting compared to state-of-the-art (SOTA) probabilistic forecasting methods. Additionally, extensive ablation studies further validate the effectiveness of the $\mathrm{D^3U}$ framework.

Published as a conference paper at ICLR 2025
DIFFUSION-BASED DECOUPLED DETERMINISTIC AND UNCERTAIN FRAMEWORK FOR PROBABILISTIC MUL- TIVARIATE TIME SERIES FORECASTING
Qi " Zhenyu Zhenyu Zhang ; Lei Yao', Zhaoxia Li,, Tianyi Zhong', Yong Zhang
'Beijing University of Posts and Telecommunications
BBijjin Key Laboratory of Work Safety Intelligent Monitoring
*China Unicom Cloud Data Co., Ltd., UUniversity of Melbourne
{li. q, zhangzhenyucad, Yao Lei,, y yongzhang) (bupt. eduu 1izx101 methooo@chhinnnnn com. cn, 2 70student. unime edu au
ABSTRACT
Diffusion-based denoising models have demonstrated impressive performance in probabilistic forecasting for multivariate time series (MTS). Nonetheless, existing approaches often model the entire data distribution, neglecting the variability in uncertainty across different components of the time series. This paper introduces Diffusion-based Decoupled Deterministic and Uncertain (D*U) framework for probabilistic MTS forecasting. The framework integrates non non-probabilistic fore- casting with conditional diffusion generation, enabling both accurate point predic- tions and probabilistic forecasting. D utilizes a point forecasting model to non- probabilistically model high-certainty components in the time series, generating embedded representations that are conditionally injected into diffusion model. To better model high-unnertaintty components, patch-based denoising network (PatchDN) is designed in the conditional diffusion model. Designed as plug-and- play framework, D can be seamlessly integrated into existing point forecasting models to provide probabilistic forecasting capabilities. It can also be applied to other conditional diffusion methods that incorporate point forecasting mod- els. Experiments six real-worke datasets demonstrate that method achieves over a 20% improvement in both point and probabilistic forecasting performance in MTS long-term forecasting compared to state-of-the-arr (SOTA) probabilistic forecasting methods. Additionally, extensive ablation studies further validate the effectiveness of the DO framework.
INTRODUCTION
Multivariate time series (MTS) are prevalent in real-world applications. Probabilistic MTS forecast- ing widely used as a decision support in domains such as finance (Wiese et al., 2020), healthcare (Teng et al. 2020) and power energy (Nowotarski & Weron 2018). Recently, autoregressive mod- els (Rasul et al. 2021; Li et al. 2022 Tashiro et al.] 2021 have achieved significant success in short-term forecasting tasks within the field of deep learning- -based probabilistic time series fore- casting. Nonetheless, the low generation efficiency of autoregressive models over long sequences limits their application long-term forecasting tasks. To address this challenge, non-autoregresssvve approaches have started to gain attention. (Shen & Kwok (2023) proposes a non-autoregressive dif- fusion model, introducing two novel conditioning mechanisms
achieve high-quality long-term time series forecasting. Li et al. (2024a) designs non non-autoregressive framework that integrates conditional diffusion process with Transformer model enable distributional forecasting for MTS data. (Shen al. (2024) decomposes time series into multiple scales, incorporating them into both the forward and reverse processes of the diffusion model, enabling probabilistic forecasting through non-autoregressive denoising.
Equual contribution. Code will be open-sourced at Correspondence Yong Zhang <yongzhhnge hupt.edu.cn.

---

Published as a conference paper at ICLR 2025
However, the aforementioned methods pay less attention to the differences in uncertainty across various components of time series data. We provide two case studies to analyze these differences:
(1) The first case study focuses two representative point forecasting models (Liu et al. 2022 Nie et al., 2023). The input series is first decomposed into trend, seasonal, and residual compo- nents, denoted as X,, Xs, and X,, respectively (see Appendix [B. for details the decomposition method). Each model is then trained using both the original series X and the series with the residual component removed, Xr + Xs. while keeping all other model parameters and settings unchanged. The model performances under both conditions are compared in Fig. la
(2) In the second case study, a patch-based vector quantization (VQ) model (Zhao al.; 2024) is employed to further analyze the uncertainty of the residual component. Specifically, the VQ model is first trained using XT + Xs. Subsequently, several patches are randomly selected from the training set and input into the trained VQ model in two forms: without decomposition (X) and with decomposition (Xr + X)), to observe their quantization distributions. The Kullback-Leible (KL) divergence between the quantization distribution and uniform distribution is used as measure of data uncertainty. larger KL divergence indicates that the input sequence is better quantized and more likely to exhibit lower uncertainty. Several examples are illustrated in Fig. 16
FFE
MrTni
BITTmi
T+5
All
0.-
NSfommn 5)
Oallbook Youne
PasdTSTTT (9)
KDL: 5.1492
KDL: 4.5810
KDL:
KDL:
4.9543
4.5430
0.4
case 020)
Pasodaally Janiy
bobiiiiy Insity
Paixdally lexiy
Traffid
Traffle
Trami
h:1122
Solentrrengy
0.0
Colliboni Youks
KDL: 4.1142
KDL: 2,8422
KDL.
KDL. 2,8123
0.2
0117 001
0.4
0.6
Prutabiity
Pistadity teuitt
(a) Case study
(b) Case study
Figure 1: Differences in modeling difficulty and uncertainty across different components of time series. (a) Both models show no significant changes in predictive performance after the removal of residual components. 'All' indicates training with the original series, while 'T+S' indicates training without the residual component. Results are averaged from all prediction lengths 166,3992336,7200 (b) Quantization distribution comparison of time series patches. KDL denotes the KI. divergence. 'lll' means containing residuals, 'T+S' means no residuals.
The results of the above case studies provide two key insights:
From the perspective of series decomposition, point forecasting models exhibit varying capabilities in modeling Xr, Xss and X,- When only the trend and seasonal components are retained input, the performance of these models does not significantly degrade and, in some cases, even improves. This suggests that the models are less effective at modeling the residual component compared the trend and seasonal components.
The difficulty in modeling the residual component appears to be related to the uncertainty it contains. Experiments using VQ-based point forecasting models show that the residual information increases the complexity of quantizing the input series. By randomly selecting five batches of data from the ETTml and Traffic datasets, the quantization distribution of all patches within each batch is analyzed. Results reveal that over 57% and 56% of the patches in the two datasets, respectively, exhibit worse quantization performance after the residual component is reintroduced. This suggests that the residual component of time series data tends to contain more uncertainty than the trend and seasonal components.
Based on insights from the aforementioned case studies, we propose a Diffusion-basse Decoupled Deterministic and Uncertain (D²U) framework for long-term MTS probabilistic forecasting. The
2

---

Published as a conference paper at ICLR 2025
Point forecasting workflow
Probabilittcc forccasting workflow
point forecasting ÿ R
Conditional DDPM
prediction emror
Forward Process
E BRUICA
Decoder
+x E R gaxesi ((.
(x)
Y BEOCKL
Forecner windoww
M
Condition Injection
Patch Embedding
E R(
Encoder fere
K steps
Reverse Process
back winddol yi ui ofll x PHCCK
Conditioning Network
Ly
Denoising Network go
Input
Encoder
probabilisne Eppstasting
Y Decoder
Output
BXXXXX
8g E 13
Gnall proggicntoiig
Sampling
An illustration of the proposed D framework. The framework consists of two main components: the conditioning network and the conditional Denoising Diffusion Probabilistic Model (DDPM). The conditioning network responsible for modeling the high-certainty components of the data, providing condition information for the reverse process of the conditional DDPM. The conditional DDPM focuses on modeling the distribution of the high- high-uncertainty components.
Figure 2:
D framework leverages pre-trained point forecasting models for non-probabilistic modeling of high-certainty components in the data, while employing a diffusion-based generative model to cap- ture the probabilistic distribution of high-uncertainty components. The D framework offers a key advantage: The pre-trained point forecasting model provides strong point forecasting capabiliitess for the overall framework and generates useful representations of high-certainty components, which serve as conditional information for the diffusion denoising model. This enhances the diffusion model's 's ability to capture the data distribution. Note that [i et al. (T024))(TMDM)) also combines a pre-trained point prediction model with a Transformer based diffusion model. The most significant difference between D and TMDM lies in their approach to modeling uncertainty in time series. TMDM directly models the overall uncertainty in the data, without focusing on the differences in uncertainty across different components. In contrast, D adopts a divide-and-come components with varying levels of uncertainty in the time series. In D*U, the diffusion model is only used to model the data distribution of high-uncertainty components, simplifying the difficulty of modeling the distribution and enables stable forecasting with fewer sampling steps. Additionally, the pre-trained model can be frozen during training, thereby reducing the overall computational cost.
approach
Our contributions are summarized follows:
1. Motivated by the phenomenon observed in case studies that point forecasting models hibit varying capabilities in modeling the deterministic and uncertain components time series, propose novel complementary modeling approach that combines point fore- casting models and probabilistic forecasting models from the perspective of decoupling the deterministic and uncertain components of time series data. Specifically, point forecast- ing models and probabilistic forecasting models are used to model the high-certainty and high-uncertainty components of time series data, respectively.
2. We propose D²U, long-term MTS probabilistic forecasting framework based on the com- plementary modeling. D²U leverages a pre-trained point forecasting model to learn the high-certainty components and injects valuable representations of these components as
3

---

Published as a conference paper at ICLR 2025
conditional information into the reverse process of conditional DDPM, aiding it in per- forming probabilistic forecasting on the high-uncertainty components. The final prediction combines the probabilistic forecast with the non- non-probabilistic forecast from the point fore- casting model, enabling DOU to achieve accurate data distribution modeling while main- taining strong point forecasting performance. D is plug-and-play framework that can be seamlessly integrated with existing point forecasting models and diffusion-basee based long- term forecasting models, improving both point and probabilistic forecasting capabilities.
3. Within the D framework, we design a patch-based denoising network, PatchDN, to en- hance the diffusion model's ability to represent the figh-uncertintty components in time series data. Our method demonstrates excellent probabilistic forecasting performance and competitive point forecasting capabilities across six real-world datasets.
2 BACKGROUND
2.1 UNCERTAINTY POINT FORECASTING MODELS
Point forecasting models are non-probabilistic models in nature. Given time series input X, these models directly learn a mapping from the input space to the output space, providing the conditional expectation of the prediction target, [() := F(X), where F-)) is parameterized function and Y is the model's output. We analyze the input series from the perspective of time series decompo- sition (Petropoulos et al. 2022), decomposing it as follows: X = X,,, + сx. where ex represents the inherent noise X, and Xn./ is the ideal noise-free time series. Further decomposition of X,, yields Xr + Xs. where Xy and Xs represent the trend and seasonal components, respectively. In an ideal scenario, the inherent noise 6x corresponds to the residual component, Xn- However, in practice, it is not possible to fully separate 6x from X, nor to obtain purely trend and seasonal com- ponents. Therefore, we decompose the residual follows: XR X2 + é'x, where X, represents the non-noise component of the series after removing the evident trend and seasonal information, and c'x the noise component of cx, excluding the noise that remains in Xy and Xs.
The prediction error of a point forecasting model, || - yl, includes both the error caused by insuf- ficient modeling of XT, Xs, and X,,, well as the error due to the inherent noise é'x. Therefore, |Y Y can be viewed as a combination of alcatoric uncertainty and epistemic uncertainty (Li et al. 2022), encompassing a majority of the uncertainty information in the sequence that is difficult for point forecasting models to capture.
2.2 PATCH IN TIME SERIES FORECASTING MODELS
recent years, advancements in natural language processing (NLP) (Radford et al. 2019) and com- puter vision (CV) (Dosovitskiy et al. 2020) have inspired the development of many basc time series forecasting models. Nie et al. (2023); Zhang & Yan (2023) segment input time series into subseries-level patches and encode them embedded representations for long-term forecast- ing tasks. /hhou et al. (003) further employs GPT-2 as a tokenizer to generate discrete embedded representations of patches. Jin al. (2023): Chang al. (2023) propose methods to align time series patches with pre-trained large language models (LLMs), improving the discrete representa- tions of time series. The use of patches for long-term forecasting has gained widespread consensus, as patches are more effective at extracting local semantic information. However, current diffusion- based probabilistic forecasting models for time series (Shen & Kwok 2023 Li al.) 2024a) pri- marily focus on pointwise modeling of time series data in the design of the denoising network.
2.3 DIFFUSION-BSEED PROBABILISTIC TIME SERIES FORECASTING
Diffusion- probabilistic models (Sohl-Dickstein et al. 2015) have recently gained prominence as a potent method in generative modeling. The DDPM (Ho et al. 2020). a widely known diffusion model, is extensively applied in probabilistic generation tasks. DDPM consists of two primary processes: the forward (diffusion) process and the reverse (denoising) process. Given an input vector yo. the forward process gradually adds zero-mean Gaussian noise into yo following Markov
4

---

Published as a conference paper at ICLR 2025
chain over K steps:
qly "K (o) = II (0 10o"ly"~1 N(y /1 Bx 1/1 .R.1),
k= 1........ "
K, (1)
where Be is a small positive constant denoting the variance the Gaussian noise added at step In practice, v² can be sampled directly from yo in a single step:
qly N(y": ãx you (1 ãg [)), y² = Vāa yo vI ag E,
where Cg. = I1 0; with a; == - 3t and is noise sampled from (0.
The reverse process aims to recover the original data yo from the noisy sample y through denois- ing procedure. This process is formulated as a Markov chain with learned transitions, where the noise is progressively removed at each step:
Po(y = p(y*) II po wn), Pol po(0"~110") NNy
Po ( k), Eole", .)) (3)
The variance (y k) is typically fixed of while the mean (y k) is parameterized by neural network with parameters ø. This network is generally used for noise estimation or data prediction. Once the model is trained, samples can be iteratively drawn from the reverse process Po(y k-1 1n*) to reconstruct yo-
PROPOSED METHOD
In this section, present D*U, novel diffusion-based probabilistic MTS forecasting framework that decouples the deterministic and uncertain components the MTS data. As illustrated in Fig. 2 D consists of two main parts: pre-trained point forecasting model (conditioning network) and patch-based conditional DDPM.
Given a history MTS data JI.H Cx and the corresponding prediction target Vı.E E RCL where H and L denote the lookback window size and forecast window size, respectively. C' denotes the number of variables in the MTS data. We use the conditioning network, denoted as fo. obtain the conditional expectation ú == E(g]æx; 0) E RCxL for the high-certainty components. where Th can either be the TT + æs or directly the original series T1:H- The specific choice depends on the modeling preferences and capabilities of the point forecasting model. We assume that the prediction error fe. Ily - contains most of the high-unncetainty components that are difficult to model using point forecasting model. The conditional DDPM is used model these components' distribution Po (y ÿ fenc (xn): Φ) where & denotes the parameters of the denoising network Ja- Here, fene denotes the encoder of and ( (Fn)) is the conditional probability density function of ÿ. fene injects information from 2G, as a condition into Jø- The final prediction is Ú, D. where Ú, sampled from pq-
3.1 CONDITIONING NETWORK
In the D framework, one of the primary roles of the conditioning network is to extract useful information from 24 effectively guide the denoising process in the conditional DDPM. Drawing inspiration from advancements in image generation (Rombach et al. 2022 Radford et al. 2021 the conditioning network implemented as a pre-trained, wetl-established point forecasting model, such as NSformer (Liu et al. (022) or PatchTST (Nie et al.] 2023). Moreover, existing MTS con- ditional DDPM models (Shen & Kwok 2023 Li et al. 2024a) typically utilize ş as the conditional input for the demoising network. However, this approach may overlook valuable intermediate fea- tures, particularly in long-term forecasting tasks. Experimental results demonstrate that leveraging (2x) the guiding signal provides more effective guidance, improving the denoising process and enhancing prediction accuracy.
In the following content, consistently Sparse (Zhao et al. 2024 the conditioning network. SparseVQ is a Transformer- based time series model that integrates VQ (Van Den Oord al. 2017) with patching techniques, demonstrating strong predictive performance MTS data.
5

---

Published as a conference paper at ICLR 2025
3.2 CONDITIONAL DDPM AND PATCH DENOISING NETWORK
3.2.1 1
CONDITIONAL DDPM FOR RESIDUAL DISTRIBUTION MODELING
In D² the conditional DDPM is employed to model the distribution of residual components in the prediction target. practice, the prediction error of the conditioning network is used the residual:
P... = y - # y (o(
Given the input residual components '1-2 E Cx and the condition c c= fene(Th), the conditional DDPM performs residual prediction by modeling the following distribution:
K
Ps (Ti:L c) - Po (Ti:L) II Ps (riiz rit
(5)
A=1
The reverse process at step is defined as:
po ( rri.,, c) = :
mo (c), o)
(6)
The mean us(ri k is parameterized as:
n 1o.1. k c) voue Ōk--1)
Vā=-1 ro (r (c),
I.L +
1 OEk
O.
where To is predicted by patch-based denoising network Ia that models the distribution of 0 The learnable parameters @ are optimized by minimizing the following loss function:
C, E e.kk [lrri.z rr [)]]]]]
Next, we present the design of an effective denoising network JG.
3.2.2 PATCH-BASED DENOISING NETWORK
To better represent MTS data, we design a patch-based denoising network (PatchDN). PatchDN adopts simple channel-independent (CI) setting (Nic et al. 2023] Woo et al. 2024 Goswami
ct al. 2024).
Patching: Specifically, the residual data ri,, is firstly divided into equilong patches. Let P repre- sent the patch length, and S' the stride, i.e., the non-overlapping portion between consecutive seg- ments. The generated sequence is denoted as r E RCx where C and N represent the number
Nie et al. 2023 Next, these patches will undergo the Patch Embedding: patches are projected into the latent space through linear laver W, E RPDD and fixed transformer's sinusoidal positional embeddings W Pos E <RINNxD (ILxXXD(VVaaann)) Vaswann 2017) are added to incorporate positional information. After Patch Embedding, the representation can be expressed as:
of variables and patches. respectively, N (H-P) S
= Patch Embedding
CxNxD
Time Embedding: As in Shen Kwok 2023 Li et al. 2024a),, the representation of the diffusion steps is obtained using the transtormer's ss sinusoidal position embedding (Vaswani 2017). followed by two fully connected layers project into the latent space.
Condition Injection: In the reverse process, effectively incorporating conditional information can guide the model to gradually denoise and generate time series samples that align with the given con- ditions. The patch-based denoising network adopts CI strategy, making the recovery of different scales across variables particularly important. We aim to leverage conditional information to apply fine-gratd fine-grained control over the scaling and shifting of different dimensions, thereby improving the recovery of noise-free time series samples.
In the field of CV, FiLM Freaturewwise Linear Modulation) (Perez et al.) 2018 is effective tech- nique that incorporates conditional information by dynamically adjusting the mean and variance in normalization layers, such as batch normalization and layer normalization. Similarly, the DiT model (Peebles & Xie 2023) has demonstrated the effectiveness of this approach in generating high-quality image samples. Inspired by this design, also introduce conditional information by controlling the

---

Published as a conference paper at ICLR 2025
scale and shift parameters of the adaptive layer normalization (AdaLN) layers in the Transformer encoder. Unlike the DiT model, which adjusts based on the batch dimension, the AdaLN layers in PatchDN dynamically adjust each variable dimension to handle the non-stationarity and heterogene- ity commonly found in time series data. The process can be described as follows:
h pd + Reshape(c) RXXIX Cscale, Cahift Linear(h) RCXIXD
(10)
AdaI.N( h) Cseate LayerNeLyy (rpe) + Cshift E E RXNxD
(11)
where Pd denotes the representation of diffusion step k.
Encoder and Decoder: Patches processed by series of Transformer encoders, Each encoder
layer consists of multi-head self-attention and feed-forward networks. Finally, flatten layer fol-
lowed by a linear layer serves as the decoder, outputting the reconstructed ri.z- Each encoder layer can be represented as follows:
TPE AdaLN + (ultti_tttn(rfe)) e RXXND,
(12)
PE AdaIN(rPE + Feed Forwardrrwrd((ye). RXNND
A
(13)
EXPERIMENTS
DATA AND EXPERIMENTAL SETTING 4.1
Six real-world MTS datasets are selected, each exhibiting distinct temporal dynamics, are selected: ETTml, ETTm2, Weather, Solar-Energy, Electricity, and Traffic. Further details can be found in Appendix D.I
Datasets:
The performance of probabilistic forecasting is evaluated using the Continuous Ranked Probability Score (CRPS) and CRPS for both the proposed model and baseline models. Ad- ditionally, point forecasting performance is assessed using Mean Squared Error (MSE) and Mean Absolute Error (MAE). Detailed descriptions of these metrics can be found in Appendix B
Metrics:
Baselines: Seven well-acknowledged long-term MTS forecasting models are carefully selected as baselines, including: (1) point forecasting methods: NSformer (Liu et al. (022), TimesNet (Wu
et al. 2023), DLinear (Zeng et al., 2023), PatchTST (Nie et al. (203), and VQ (Zhao et al.
2024
(2) probabilistic forecasting methods: TimeGrad (Rasul et al. 2021), CSDI (Tashiro et al.
2021 TimeDiff (Shen & Kwok 2023), and TMDM (Li et al. 2024a). Detailed descriptions can be
found in Appendix D.2]
Implementation details: In the experiments, the lookback window size H and prediction length L are set to 96 and 192, respectively. The diffusion process is configured with 100 steps, using a linear noise schedule where 31 10-4 and 3x 0.02. A total of 100 samples used approximate the estimated distribution. All experiments are implemented using Py PyTorch (Paszke et al. 2019) and executed on an NVIDIA RTX A6000 48GB GPU. Further implementation details are provided in Appendix D.3 Unless otherwise stated, the point forecasting model used in the DJ framework is VQ.
4.2
RESULTS
4.2.1
MAIN RESULT
In this section, our method integrates Sp ZZhao et al. 2024 and PatchDN within the D framework, where Sparse functions as the conditioning network and PatchDN serves as the de- noising network. Table[[]ighlight the outstanding performance of our method in point forecasting, achieving 28% improvement MSE and a 21% improvement in MAE compared to the current state-of-the-art (SOTA) probabilistic MTS long-term forecasting method, TMDM (Li et al. (024a). Additionally, our method's 's point forecasting capabilities are on par with the SOTA point forecasting models across four datasets. Compared to the VQ model, our method demonstrates either superior or comparable point forecasting performance across all datasets, with particularly notable improvements in the Solar and Traffic datasets.
7

---

Published as a conference paper at ICLR 2025
Table 1: Performance comparison on six real-world datasets based on MSE and MAE. The best/second results are highlighted in bold/underline, respectively. Lower MSE and MAE values indicate better performance. SparveVQ is used as the point forecasting model in the DdU (ours).
Model
Dataset
ETTml
ETTm2
Weather
Solar-Energy
Electricity
Traffic
Method
MSE
MAE
MSE
MAE
MSE
MAE
MSR
MAE
MSE
MAE
MSE
MAE
NSformer(20220)
0.440
0.430
0.277
0.343
0.226
0,270
0.266
0.270
0.391
0.295
0.653
0.360
TimesNNe(20023)
0.374
0.387
0.249
0.309
0.219
0.261
0,296
0,318
0.184
0.289
0.617
0.356
Point
forecasting
(202))
0.380
0.389
0.284
0.362
0.237
0.296
0.320
0.398
0.196
0.285
0.598
0.370
P22c))ITT(0233)
0.370
0.390
0.251
0.312
0.223
0.258
0,259
0.321
0.205
0.307
0.463
0.311
SpurseVQ(2024)
0.363
0.380
0.242
0.302
0.225
0.258
0.256
0.286
0.182
0.267
0.480
0.300
Trunsformer 20024)
0.377
0.391
0.250
0.309
0.221
0.254
0,233
0.261
0.164
0.255
0.418
0.284
TimeGrad(2021)
1,716
1.057
1,385
0.732
0.885
0.551
1,211
1.004
0.645
0.723
0.932
0.807
Probubilistic forecasting
(SD1(2021)
0.867
0.690
[.291
0.576
0.842
0.523
0.848
0.818
0.553
0.795
0.921
0.678
2022
0.796
0.577
0.284
0.342
0.277
0.331
1.169
0.936
0,730
0.690
1.465
0.851
TMDM/2024)
0.607
0.558
0.524
0.493
0.244
0.286
0.295
0.222
0.329
0.721
0.411
ours
0,363
0.386
0.241
0.302
0.222
0.264
0,237
0,270
0.179
0.267
0.468
0.299
Table 2 presents the superior probabilistic forecasting performance of our model, showing a 40% improvement in CRPS and 5% improvement in CRPS compared to TMDM. In the subsequent ablation studies, we analyze the contributions of the D framework and the design of PatchDN to the improvements in long-term forecasting performance.
Table 2: Performance comparisons on six real-world datasets regarding CRPS and CRPS The best/second results are highlighted in bold/underline. Lower CRPS and CRPS values indicate better performance. SparveVQ is used as the point forecasting model in the DOU (ours).
Model
Dataset
ETTml
ETTm2
Weather
Solar-Energy
Electricity
Traffic
Method
CRPS
CRPS....
CRPS
CRPS....
CRPS
CRPS....
CRPS
CRPS....
CRPS
CRPS....
CRPS
CRPS....
3RmoGraad((921)
06655
0.996
0.785
1.051
0.503
0.788
1.167
0.503
1.452
06657
1.683
Probabilistic
CSDI(2021)
0.773
0.852
0.782
0.465
0.649
0.465
0.823
0662
1,275
Forecasting
2022))
0.454
0.846
0.316
0.180
0.400
1.164
0.475
0.594
0.671
0.623
1MDM/22024
0.429
0.633
0.380
0.236
0.292
0.375
0.267
0.466
0.137
0.552
ours
0.285
0.574
0.343
0.141
0.307
0.283
0.186
0.266
0.282
0.160
0.232
0.186
4.2.2 ABLATION STUDY
To further demonstrate the advantages of the DOU framework, we modify the TMDM model to incorporate the DOU framework. Specifically, the NSformer (Liu et al. 2022) used in TMDM, which is originally co-trained with the denoising network, is replaced by pre-trained NSformer model with identical parameter settings. In the DdU framework, the pre- treirrained NSformer pre- diets the deterministic component and provides guiding information, while the diffusion model's denoising network, responsible for modeling the residual distribution, employs an MI. network with consistent parameter settings. The experimental setup ensures full consistency between the training and evaluation processes. Table 3 highlights the superior performance of the DOU frame- work, where both point and probabilistic forecasting capabilities are significantly improved across all datasets compared to the original TMDM. TMDM models the overall data distribution, while the DOU framework explicitly decouples the deterministic and uncertain components of the data, requiring the modeling of only the high-uncertainty component. This reduces the complexity of the distribution and allows the DDPM to achieve stable probabilistic forecasting with fewer sampling steps (Appendix A).
PatchDN design: In all ablation experiments examining the design of PatchDN, Sp serves as the conditioning network, providing guidance to the denoising network. In the comparative ex- periments, Table that replacing the PatchDN denoising network in the D framework with the MLP network from TMDM the UNet network from Tife results in decrease in both point and probabilistic forecasting performance. This demonstrates that PatchDN exhibits a stronger capability in modeling the high-uncertainty component.
8

---

Published as a conference paper at ICLR 2025
Table Performance promotion by applying DO to TMDM.
Mode
TMDM
TMDM (D*U)
Datasets
MSB
MAE
CRPS
CRPS.am
MSE
MAE
CRPS
CRPS.um
ETTml
0.607
0.558
0.429
0.633
0.441
0.432
0.324
0.616
FTTm2
0.524
0.493
0.380
0.226
0.317
0.302
Weather
0.244
0.286
0.226
0.292
0.215
0.267
0.196
0,273
Solar Energy
0.295
0.317
0.375
0.267
0.269
0.299
0.328
0.260
Electricity
0.222
0.329
0.446
0.137
0.216
0.328
0.381
0.157
Trallic
0.721
0.411
0.552
0.179
0.678
0.402
0.472
0,207
In PatchDN, two typical variants are designed for incorporating conditional information: the cross- attention method and the in-context method. Detailed descriptions of these two variants can be found in Appendix A Table shows that, compared to these two variants, the our method performs better in both point and probabilistic forecasting.
To further demonstrate the rationality behind the design of D'U, two variants of the framework are created. The first variant uses the ground truth as the starting point for the conditional DDPM, i.e., r = y. Experimental results show a significant decline in prediction performance. Compared to D²U, modeling the entire data distribution using the same diffusion steps and sample sizes proves to be more challenging, resulting in degraded probabilistic forecasting performance and inability to ensure accurate point forecasting. The second variant uses the final prediction output ý of the conditioning network as guidance for the conditional DDPM. Experimental results also reveal sig- nificant drop in prediction performance, suggesting that the encoder's 's output from a well-designed point forecasting model contains more beneficial information for guiding the denoising network.
Table 4: MSE, MAE and CRPS scores for different variants of the proposed method.
Ablation
Mode (/s + 9x)
ETTml
Solar-Energy
Traffic
Study
MSE
MAE
CRPS
MSE
MAE
CRPS
MSE
MAE
CRPS
Denoise
SVQ + MLP
0.372
0.294
0.330
0.313
0.242
0.525
0.331
0.297
Network
SVQ UNet"
0.385
0.410
0.301
0.267
0.266
0,219
0.469
0.301
0.289
Structure
SVQ 4 PaCttD
0,370
0.390
0.295
0.237
0.281
0.193
0.483
0.307
0.240
Design
SVQ (InC)
0.366
0,385
0.290
0.238
0.271
0,187
0.486
0.315
0.244
Framework
SVQ PatchDN(AAl]"
0.859
0.699
0.516
0.348
0.302
0.251
0.687
0.396
0.302
Design
SVQ + PatchDN(g)
0.408
0.421
0.312
0.259
0.301
0.241
0.479
0.308
0.251
0.385
0.284
0.236
0.270
0.186
0.468
0.299
0.232
SVQ is the abbreviation for S
layers; means the UNet. the denoissng network the TimeDill model, is buill using a comvouttional noural betwork-based UNet architecture.
means serves as demotsing network in TMDM model consists
marks PaichDN based on the cross attention method: d marks ParchDN based on the in contestst methed.
marks the framework variant that models the entire data distrbbtion; f marks the framework variant that employs guiddance.
Framework generality: The D framework is designed as a alug-and-plis solution that can be seamlessly integrated into existing point forecasting models. The conditioning network replaced with NSformer, PatchTST, and Sparse VQ, respectively. As shown in Table 5 applying the D framework improves each model's point forecasting capabilities, while also providing them with probabilistic forecasting abilities.
Table 5: Performance promotion by applying the proposed framework to point forecasting models.
Dataset
ETTml
Solar- Energy
Traffic
Method
MSF
MAR
CRPS
MSF:
MAF
CRPS
MSF
MAE
CRPS
NSformer
0.440
0.430
0.266
0.270
0.653
0.360
NStormer (D*U)
0.272
0.202
0.657
0.367
0.284
PatchTST
0.370
0.390
0.259
0.321
D.463
0.311
PatchTST (D*U)
0.387
0.405
0.299
0.233
0.281
0.221
0.452
0.297
0.234
VO
0.363
0,380
0.256
0.286
0.480
0.300
SparseQ (D*V)
0.361
0.385
0.284
0.237
0,270
0.185
0.475
0.309
0.232
means point forcasttngg models not have probubilistic forecasting abilities. degracks to the Normalized Mean Squire Eivor (NMAE), which is omniteed here.
CRPS value

---

Published as a conference paper at ICLR 2025
4.3 RETHINK OF UNCERTAINTY MODELING IN D
In the previous two sections, extensive experiments have validated the effectiveness of the D framework. However, some results in Table 2 and Table Suggest that D may face challenges in uncertainty modeling when applied to MTS data with extremely high variable dimensions (e.g.. Electricity, Traffic), particularly reflected in CRPSsum- This issue related the composition of uncertainty in the prediction eπrrr. As discussed in Section the prediction error of the point fore- casting model primarily consists of epistemic uncertainty and aleatoric uncertainty (Gaalikowski et al. 2023). In practice, a relatively simple yet effective approach is adopted by approximating the high-uunertainty components using the prediction error, y - ÿ. However, it is observed that higher epistemic uncertainty tends to result in wider probabilistic prediction intervals, which subsequently impacts the model's performance as measured by CRPS As shown in Fig. B when the point
Soond 1att
increditiinn
M
M
Ap
Figure 3: Cases of probabilistic prediction intervals with different uncertainty compositions.
forecasting model effectively models the deterministic components, the prediction eπror is mainly composed of aleatoric uncertainty, resulting in more accurate prediction intervals for probabilistic forecasting. Conversely, epistemic uncertainty becomes dominant when the point forecasting model struggles with the deterministic components, leading to wider prediction intervals. Therefore, em- ploying a more powerful point forecasting model can effectively mitigate this issue. As shown in 6 after replacing the conditioning network from Sparse with i'Transfommer, the D frame- work achieved significant improvement in the CRPS metric. This also suggests that potential direction for improving the D framework in future work would be to explore better methods for separating and addressing epistemic uncertainty and aleatoric uncertainty.
Table 6: Performence comparison of different point forecasting models employed by the D frame- work.
Electricity
Traffic
Mode
MSE
MAE
CRPS
CRPS
MSE
MAE
CRPS
CRPS
Sp PatchDN
0.179
0.267
0.202
0.160
0.469
0.299
0.232
0.186
iTranffommer + PatchDN
0.168
0.261
0.195
0.151
0.421
0.290
0.222
0.169
5 CONCLUSION
In this paper, we introduce the D framework for probabilistic long-term MTS forecasting. D decouples the deterministic and uncertain components of time series, leveraging pre-trained point forecasting models to model the high-certainty components, while employing a conditional DDPM to perform probabilistic forecasting on the high-uncertainty components, As plug-and-play frame- work, D can be seamlessly integrated into existing point forecasting models to enable proba- bilistic forecasting. To better capture the high-uncertainty components, PatchDN is designed within the conditional DDPM. Comprehensive experiments on six real-world datasets demonstrate the out- standing probabilistic forecasting performance and competitive point forecasting capability of D
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
We appreciate constructive feedback from anonymous reviewers and meta-reviewers. This work is supported by the National Natural Science Foundation of China under Grant No.61971057 and National Key R&D Program of China under Grant No. 2020YFC1522503.
REFERENCES
Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 22669-226337 2023.
Ching Chang, Wen-Chih Peng, and Tien-Fu Chen. Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms. arXiv preprint arXiv:2308.088.. 2023.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at
scale. CoRR, 0ts/2010.11929, 2020. UtLhttps: wtw.vviiivv..///vi///a.a//aaaava org/ abs 222/220100 110001000 11929
Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, et al. A survey of uncertainty in deep neural networks. Artificial Intelligence Review, 56(Suppl 1):1513-1589, 2023.
Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. Mo- ment: A family of open time-series foundation models. In International Conference on Machine Learning, 2024.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 2020.
Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin- Yu Chen, Yux- uan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-Ilm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.0172.
2023.
Yan Li, Xinjiang Lu, Yaqing Wang, and Dejing Dou. Generative time series forecasting with dif- fusion, denoise, and disentanglement. Advances in Neural Information Processing Systems, 35: 23009-23022, 2022.
Yuxin Li, Wenchao Chen, Xinyue Hu, Bo Chen, Mingyuan Zhou, et al. Transformer-modulaedd dif- fusion models for probabilistic multivariate time series forecasting. In The Twelfth International Conference on Learning Representations, 2024a.
Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, Dayou Chen, Jiajun He, Jiahao Li, Wenyue Li, Chen Zhang, Rongwei Quan, Jianxiang Lu, Jiabin Huang, Xiaoyan Yuan, Xiaoxiao Zheng, Yixuan Li, Jihong Zhang, Chao Zhang, Meng Chen, Jie Liu, Zheng Fang. Weiyan Wang, Jinbao Xue, Yangyu Tao, Jianchen Zhu, Kai Liu, Sihuan Lin, Yifu Sun, Yun Li, Dongdong Wang, Mingtao Chen, Zhichao Hu, Xiao Xiao, Yan Chen, Yuhong Liu, Wei Liu, Di Wang, Yong Yang, Jie Jiang, and Qinglin Lu. Hunyuan-dit: A powerful multi-resolution diffusion transformer with fine-grained chinese understanding, 2024b.
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long, Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 35:9881- 9893, 2022.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
James E Matheson and Robert L Winkler. Scoring rules for continuous probability distributions. Management science, 22(10):108--1096 1976.
11

---

Published as a conference paper at ICLR 2025
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations, 2023.
Jakub Nowotarski and Rafat Weron. Recent advances in electricity price forecasting: A review of probabilistic forecasting. Renewable and Sustainable Energy Reviews, 81:1548-1566 2018.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural information processing systems, 32, 2019.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4195-4205, 2023.
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Fotios Petropoulos, Daniele Apiletti, Vassilios Assimakopoulos, Mohamed Zied Babai, Devon K Barrow, Souhaib Ben Taieb, Christoph Bergmeir, Ricardo Bessa, Jakub Bijak, John Boylan, et al. Forecasting: theory and practice. International Journal of Forecasting. 38(3):705-871, 2022.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 9748-8766 PMLR, 2021.
Kashif Rasul, Calvin Seward, Ingmar Schuster, and Roland Vollgraf. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In International Conference on Machine Learning, pp. 8857-8868. PMLR, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Lifeng Shen and James Kwok. Non-autoregrressvv conditional diffusion models for time series prediction. In International Conference on Machine Learning, pp. 31016-31029. PMLR, 2023.
Lifeng Shen, Weiyu Chen, and James Kwok. Multi-resolution diffusion models for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024.
Jascha Sohl- Sohk-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International conference on machine learning, pp. 2256-2265. PMLR, 2015.
Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffu- sion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34:24804-244 14 2021.
Xian Teng, Sen Pei, and Yu-Ru Lin. Stocast: Stochastic discase forecasting with progression uncer- tainty. IEEE Journal of Biomedical and Health Informatics, 25(3):850-861) 2020.
Aaron Van Den Oord, Oriol Vinyals, al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Magnus Wiese, Robert Knobloch, Ralf Kom, and Peter Kretschmer. Quant gans: deep generation of financial time series. Quantitative Finance, 20(9): 1419-1444)) 2020.
12

---

Published as a conference paper at ICLR 2025
Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In International Conference on Machine Learning, 2024.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34:22419-224306 2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet:
Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations, 2023.
Xinyu Yuan and Yan Qiao. Diffusion- TS: Interpretable diffusion for general time series generation.
In The Twelfth International Conference Learning Representations, 2024. URL https:
/ //openreviev. net. www.///tt//omieeee fow.ffdddmmmmhapaaaaa j099
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 11121-11128, 2023.
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The eleventh international conference on learning representations, 2023.
Yanjun Zhao, Tian Zhou, Chao Chen, Liang Sun, Yi Qian, and Rong Jin. Sparse-vg transformer: An ffn-free framework with vector quantization for enhanced time series forecasting. ar preprint arXiv.2402.05830, 2024.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. International conference on machine learning, pp. 27268-27286 PMLR, 2022.
Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained Im. Advances neural information processing systems, 36:43322-43355 2023.
A APPENDIX: EXPERIMENTS
1. Cross- cooss-tttention method: conditional information serve keys and values in the attention mechanism, guiding the model to adaptively model the input patches (Li et al.) 2024b). 2. In-context method: the conditional information is treated as part of the input sequence the Transformer encoder, processed alongside other patch embeddings (Bao et al., 2023)..
Details of PatchDN Variants:
In Table 4 compare the performance of our method with the two structure variants. However, the second-best results in this experiment are also achieved by these two variants, prompting further analysis of the FLOPs and inference time for all three mechanisms. Table demonstrates that our method has lower computational complexity and inference time compared to the other two, which explains its superior overall performance.
Table 7: Comparison of FLOPs and Inference Time with the Two Variants (100 samples).
ETTml
Solar- Energ-
Traffic
Mode
#FLOPS
Inference time (min)
#FLOPS
Inference time
#FLOPS
Inference time
(G)
(G)
(min)
(G)
(min)
SVQ+ + Patum
4.933
0.094
20.69
0.446
130.0
0.452
SVQ Pinnh
4.632
20.94
131.7
0.336
Ours
3.569
0.050
15.74
0.238
99.03
0.299
SVQ is the abbreviation SpurseVQ..
marks PatchDN hased on the cross cross-attenting attention method: marks PaichDN hased on the in co-teneext method.
indicales oft-of- memory (OOM)
48GB GPU.
Diffusion Steps: Fig. Fig.4illuse Hillustrates the sensitivity of three time series probabilistic forecasting models to diffusion steps. Due to the decoupling advantage of the DOU framework, which reduces the
13

---

Published as a conference paper at ICLR 2025
complexity of distribution modeling, the figure shows that PatchDN achieves optimal performance at 70 steps on the ETTml dataset and 50 steps on the Weather dataset. Compared to other models, PatchDN exhibits lower sensitivity to diffusion steps.
0.56
TimeDiff
ET1ml
0.46
ET Tm
TMDM
0.43
0.52
DVCXSV PatcsVO--PachNN PatcDNN)
0.12
DJULSVQ-MLII)
041
0.50
MAP Value
0.48
0.46
0.36
0.75
Tim:Diff
0.44
0.36
TMDM
0.3
DSULSVQ PPccDNN
0.32
DSUASYO-MLP:
0.38
0:37
310
150
Diffission Steps K
Diffission Stope K
outher
0.34
DSUISVQ PoncED DEUISVOPPRRED DSUISVQ-MLPP
0.33
MAB Value
0.31
CRPS Value
0.78
.MDM
FimeDiiT
0.30
0.26
DSU/SVQ-
DAUISYG-MLPY
0.24
0.27
0.22
0.2
0.20
Diffision Stego
Diffisson Stege
Figure The impact of different number of diffusion steps K.
Prediction Strategy: Table 8 presents comparison of different prediction strategies. In low- dimensional datasets, the strategy of predicting 6s typically performs better, whereas in high- dimensional datasets, the strategy of predicting Te tends to be effective. This may be due to the fact that high-dimensional time series datasets contain more highly irregular noise components, making it more challenging to estimate the diffusion noise ed.
Table 8: Quantitative comparison of different prediction strategy.
Mode
Weather
Selar-Energy
Electricity
MSE
CRPS
19
0.251
03188
0.227
0.186
0.179
<o
0.363
0.382
0,286
0.241
0.302
0.236
0.223
0.264
0.203
0.254
0.276
0.186
0.184
0.269
0.201
0.481
0.298
0.225
B APPENDIX: SERIES DECOMPOSITION
B. SERIES DECOMPOSITION
In this paper, moving average is applied to the input sequence to obtain the component Xy. which contains trend information. The amount of trend information retained in X₂ is controlled by the moving average kernel k (Wu et al. 2021 Zeng et al. 2023):
Xς = AvgPool(X
(14)
For the seasonal component, Fourier layer based on Fourier bases is employed (Yuan & Qiao 2024). The number of retained frequency components is controlled by the Fourier factor f:
X5 Layer FFT(X-
Xx;
(15)
14

---

Published as a conference paper at ICLR 2025
The residual component is defined as:
XR := X XT Xs
(16)
B.2 SERIES DECOMPOSITION EXPERIMENTS
As mentioned in Section 3 the input of the conditional network, Th, can either use the original time series x or the 3r + g. Table 2 presents the impact of different kinds of Th on the forecasting performance of two point forecasting models (NSformer, PatchTST) and their DdU versions.
Table 9: The impact of different types of Ty on the point forecasting model and its D version's forecasting performance. 'All' means using the original series as input. 'K* F*1 means using the JT + Is as imput. For example, 'K3' denotes moving average kernel h = 3 in Eq. [4 and 'F2' denotes a Fourier Factor f 2.0 in Eq. [5] In this experiment, the lookback window size H is 96 and the prediction length L is 192.
Type of at.
All
K3
K7
K7
KIS
Datasel
Metrie
MSE
MAR
CRPS
MSF
MAE
CRPS
MSE
MAF
CRPS
MSF
MAF
CRPS
MSR
MAR
CRPS
NStormer
0.440
0,430
0,435
0.430
0.438
0.431
0.439
0.430
0.448
0.432
D"))
0,436
0,427
0.317
0,436
0,431
0.321
0.429
0,426
0.317
0,428
0,427
0.315
0.453
0.432
0.325
ET1ml
PoshEST
0.370
0,390
0.574
0.395
0.375
0.375
0.374
0.392
PUchTSTDD")
0.296
0.289
NStormer
0.226
0.218
10.255
0.268
0.227
0.270
0.232
0.272
0.233
0.281
0.209
0.226
0,275
0.207
0.237
0,283
0.212
0.245
0.282
0.213
0,240
0,284
0,212
Weather
PitcTTST
0.223
0.258
0.234
0.258
0.224
0.259
0.225
0.259
0.226
0.266
Uu)
0,236
0.293
0.218
0,224
0.267
0.205
0.236
0.288
0.215
0.224
0.267
0.206
0.225
0.264
0.206
C APPENDIX: TRAINING STRATEGY EXPERIMENTS
The D3U framework freezes the parameters of the pre-trained point forecasting model, which re- duces the number of trainable parameters and the overall training time. To demonstrate this advan- tage, we conducted experiments with three additional training configurations: (I)DaU with series decomposition (DCP), denoted as DUU-DCP: the inputs of the point prediction model and the dif- fusion model are FT + Vy and ZR := x - IT + Is. respectively. (2) D*U-FT: fine-tuning the pre-trained point forecasting model while training the diffusion model, and (3) D training the entire model from scratch without using the pre-trained point forecasting model. The details of the experimental results are shown in Table TO The results indicate that, most datasets, the fine-tuned and from-scratch training configurations do not show significant differences compared the original D* However, the training approach using the frozen pre-trained model results in the minimum number of training parameters and the shortest training time. In these three schemes, D*C-DCP performs the worst. We think the reason for this result is that DCP provides coarse approach to separating the high-certainty and high-uncertainty components of time series data. As shown in the experimental results of Table 9 the effectiveness of DCP is highly dependent on the choice of hyperparameters k and f. Therefore, we do not consider DCP a reliable approach for extracting high-certainty or high-uncertainty components in all cases. the setup we adopt in the paper, the undecomposed data is used as the input to the point forecasting model, and the predic- tion error y á from the point forecasting model is used as the input to the diffusion model. As analyzed in Section 2.1 the prediction error of the point forecasting model primarily consists of two types of uncertainty: aleatoric uncertainty and epistemic uncertainty. Thus, the prediction error is a reasonable approximation of the high-uncertainty component in the time series data that is difficult for the point forecasting model to capture. Details of the number of training parameters and the training time for the different training modes are presented in Table II
D APPENDIX: IMPLEMENTATION DETAILS
D.1 BENCNMARK DATASETS
For our experiments, we use ETTm1, ETTm2, Weather, Solar-Energy Electricity and Tra f fic open-source datasets, with their properties listed in Table 12 Following the methodolo-
15

---

Published as a conference paper at ICLR 2025
Table 10: MSE, MAE and CRPS scores for different training framework variants of the proposed method. D'U-DCP: The inputs to the point prediction model and the diffusion model are FT + Is and FR = 2r + zs, respectively. U-U- Fine-tuning the point forecasting model while training the diffusion model. DUU-FS: Training the entire model from scratch.
Training Modi
ET1ml
ETTm2
Weather
Solar Sobrr-Energy
Electricity
Truffic
MSR
MAR
CRPS
MAR
CRPS
MSE
MAR
CRPS
MSR
CRPS
MSE
CRPS
MSE
CRPS
DTU-DCP
0.395
0.405
0.330
0.251
0.312
0.265
0.228
0.368
0.240
0.204
0.324
0.272
0.275
0.377
0.780
0.790
0.510
DTC-PT
0.368
0.388
0.286
0.253
0.323
0.247
0.221
0.271
0.302
0.317
0.109
6.174
0.266
0.897
0.338
0.292
D7C-ES
0.364
0.366
0.284
0.248
0.317
0.243
0.200
0,210
0.220
0.262
0.172
0.172
0.266
0.199
0.16
0.304
0.233
DPU
0.36
0.366
0.283
0.241
0.302
0.243
0.223
0.264
0.202
0.237
0.270
0.186
0.177
0.672
0.202
0.468
0.299
0.232
Table 11: Trainable parameters and training time for different training framework variants of the proposed method. D*U-DCP: The inputs the point prediction model and the diffusion model JT + Es and n == x - 2r + Is, respectively. D-UFFT: Fine-tuning the point forecasting model while training the diffusion model. DOU-FS: Training the entire model from scratch.
Training Mode
ETTml
ETTm2
Weather
Ellar-Energy
Electricit
Traffic
Trainable Parametuns
Training
Trainable Parameters
Training
Trainable Parameters
Training time (hateh)
Trainuble Parameters
Training
Trainuble Paramoters (M)
Training time (whntch)
Trainable Parametery
Training
time (whatcc)
time (alatch)
time (whatch)
time (whotcc)
(M)
(M)
(XD)
(M)
(M)
D'C-DCP
1,899
4.912
1,699
4.922
3024
14,099
1.322
70.62
2,166
204
3,495
119.229
D'U-FT
7,580
7,471
7,350
7,403
4,855
18.106
3.504
105.5 155,555
10.249
386, 249
6.467
195
DTU-F3
7.300
7,552
7,360
7.451
4,85
18,050
3.504
105.159
10.249
316:112
6-467
340.347
D'
1,699
4.567
1,699
4.621
3024
13.712
1.322
76.627
2.166
198.87
3,495
316.330
gies Wu et al. (2021) and Zhou et al. (2022), the datasets are split chronologically into training, validation, and test sets. 6:2:2 ratio is used for ETTml and ETTm2, while 7:1:2 ratio is applied for Weather, Solar-Energy, Traffic, and Electricity. The dataset can be obtained through the links below.
() ETTm1, ETTm2:
(ii) Meather: http:///tt.as///w..oo./ooaaaaaar
Flletrioitt/ooaaalaaaaaa22222222
(iv)
Electricityloosbllerrs2201111111
(v) Traffic:
Table 12: Detailed information of the datasets used in our benchmark, including data frequency, number of time series (dimension), context length, and prediction length. Dataset size indicates the total number of time points the train, validation, and test splits, respectively.
Dataset
Dimension
Frequency
Dataset Size
Context length
Prediction length
ETTml ITTmm.ETTT22 .ETm2
7
15 Min
(34465, 11521, 11521)
96
192
Weather
21
10 Min
(36792, 5271, 10540)
96
192
Solar-Ense Energy
137
Min
(36601, 5161. 10417)
96
192
Electricity
321
1 Hour
(18317, 5261)
96
192
Traffic
862
1 Hour
(12185, 1757, 3509)
96
192
D.2 BASELINES IN MAIN EXPERIMENTS
Seven well- well-acknowledged long-term MTS forecasting models are carefully selected as baselines,
including: (1) point forecasting methods: NSformer (Liu et al. 2022). TimesNet (Wu al. 2023)
DLinear (Zeng et al. (023), PatchTST (Nie et al. 2023) and Sparse (Zhao et al.
2024): (2) prob-
abilistic forecasting methods: TimeGrad (Rasul et al. 2021). CSDI (Tashiro al.
2021 TimeDiff
(Shen & Kwok 2023), 3, and TMDM (Li et al. 2024a).
The code and descriptions of the baseline methods be obtained from the following sources.
16

---

Published as a conference paper at ICLR 2025
(i) NSformer: novel framework that enhances Transformer models for time series forecasting by integrating Series Stationarization to unify input statistics and De-stationary Attention recover intrinsic non-stationary information.
Code: https: / com/thumm //t////nnnnnnnniie ott.a//oooooo.....ooooooooooooo.
(ii) TimesNet: task-general backbone for time series analysis that transforms ID time series into 2D tensors to effectively model complex temporal variations through adaptive multi- periodicity discovery and a parameter- inception block.
Code:
wtt.ccco/////tttte. thum1 www.//iimeeeetrtttteeeeeeeeeeeee TimesNet.eeeeeeeeeeeeeiee
com/
(iii) DLinear: a simple one-layer linear model for long-term time series forecasting that out- performs sophisticated Transformer-based models by effectively preserving temporal rela- tions.
(iv) PatchTST: an efficient Transformer-basel model for multivariate time series forecasting that utilizes segmentation into subseries--leve and channel independence.
Code:
(v)
Sparse novel approach for time series analysis that utilizes sparse vector quantization and Reverse Instance Normalization to address distribution shifts and noise, replacing the
layer enhance computational efficiency and reduce overfitting.
Code: https: http///nnnnmmmmmmmmm 4wpp://muuuuuu....oo.oooom wwwence/eeepaaaeeeeeeeee
VQ-DCC2
(vi) TimeGrad: an autoregressive model for multivariate probabilistic time series forecasting which samples from the data distribution each time step by estimating its gradient.
Code: https: / thg.b. com/ min crosoft /trofff//rrr/
(vii) CSDI: novel time series imputation method that utilizes ecore-based diffusion models conditioned on observed data.
Code: https: thub com/ mmt//miiiee mircooooo/orrrrss
(viii) Tife a non-autoregresssvv diffusion model for time series prediction that leverages innovative conditioning rechanisms-funtt mixup and autoregressive initialization.
Code: There is no publicly available code; we obtained the code by emailing the author.
(ix) TMDM: combines conditional diffusion processes with transformers enable precise dis- tribution forecasting in multivariate time series while effectively incorporating uncertainty.
Code: thub. com Yuxin Yuxim321/mm13m
D.3 IMPLEMENTATION DETAILS
To accelerate the inference of the DDPM, DPM-Solver (Lu al. 2022) is employed, allowing the number of denoising steps to be empirically reduced to fewer than 20. The proposed model is trained using the Adam optimizer with learning rate of 10-4 Following the parameter settings outlined in Li al. 2024a), early stopping is applied after 15 epochs without improvement, with a maximum of 100 training epochs. Table presents the hyperparameters of the UparseVQ+Patchle both training and testing across each dataset. The conditioning network is the SparseVQ model, with the primary hyperparameters being encode poooddrrllyerss d_model, d_ff, nrm_codebook, and codebook_sizz ze. The hyperparameters for PatchDN are consistent across all datasets, consisting of single-layer Transformer encoder with 8 heads and latent space dimension of 128.
model for
Table 13: Hlyper-paramete values for the Sparse VQ+PatchDN Model.
Diffusion Train batch_size
Diffusion
Condition network encoder
Condition network d_model
Condition network d_ff
Condition network num_codebook
Condition network codebook_size
Datasct
batch_size
ETTmLETTT'
128
64
512
$12
256
Weather
128
64
256
512
256
Energy
512
1000
Electricity
64
512
512
256
Traffic
16
256
256
2
512
17

---

Published as a conference paper at ICLR 2025
E APPENDIX: METRICS
E. METRICS FOR POINT FORECASTING
Mean Squared Error (MSE). MSE is calculated the average of the squared differences between the predicted values and the actual values, defined mathematically as:
C
MSE CXL E0
(17)
CXL c=1
where C' represents the number of variates, denotes the length of the series, and rf and ff indicate the ground-truth value and the predicted value, respectively.
Mean Absolute Error (MAE). MAE measures the average magnitude of the errors in a set of
predictions, without considering their direction. It is calculated as:
C
MAE
rr - -1--
(18)
CxL
E.2 METRICS FOR PROBABILISTIC FORECASTING
Continuous Ranked Probability Score (CRPS). The CRPS (Matheson Winkler 1976) quanti- fies the difference between the cumulative distribution function (CDF) F of the predicted probabil- ities and the CDF of the observed outcomes r, represented as:
CRPS (1'()) [(((=)) -(r=))) dz,
(19)
where [(rs=) is the indicator function that equals one if < and zero otherwise. As proper scoring function, CRPS achieves its minimum when the predictive distribution I' matches the true data distribution. Using the empirical CDF F(2) 1 Σ I(R, S 2), CRPS can be computed from simulated samples of the conditional distribution po (r.
CRPS CRPS extends CRPS to multivariate time series. Specifically, CRPS is defindd as:
CRPS = E, [CRPS(F
Σ 1l
(20)
where F Fuoh is obtained by summing the samples across dimensions and then ordering them to derive the quantiles.
F APPENDIX: FORECAST SHOWCASES
F.1 CASE STUDY
To demonstrate the superiority of the proposed method, Fig. visualizes the ground truth and pre- dictions of time series across two dimensions in the ETTml and Weather datasets, along with the 50% and 90% prediction intervals. The results show that the proposed method achieves higher point forecasting accuracy (median prediction, represented by the dark green line) compared to the other two models, and demonstrates a stronger capability in estimating the distribution of the time series.
F.2 DENOISING PROCESS SHOWCASES
To clearly demonstrate the generation process of time series data, Fig. Fig. and Fig. illustrate the denoising process (aka reverse process) time series data across two dimensions in the ETTml, Electricity, and Traffic datasets, from diffusion step 100 (start point of denoising process) to step 0 (end point of denoising process). The results indicate that at diffusion step 100, point prediction models can provide prior knowledge to diffusion models, ensuring predictions oscillate near the true values rather than gradually denoising from pure Gaussian noise. This effectively reduces the difficulty of denoising. As the steps decrease, the width of the 50% and 90% prediction intervals gradually narrows, achieving accurate point prediction precision at step 0 (denoted by the median prediction in dark green lines) and effectively estimating the distribution of time series data.
18

---

Published as a conference paper at ICLR 2025
prodnie
TMON
25
25
-15
09
20
ETTml.
(b) Weather.
Figure 5: More comparison of prediction intervals for the ETTml and Weather Datasets. The black line representing the test set ground- truth.
F.3 PREDICTION RESULTS VISUALIZATION
To further illustrate the predictive performance the D3U framework, Fig. Fig. TO Fig.
Fig. 12 Fig. 13 Fig. 14 and Fig. [5] present the outputs of the point prediction model, the
probabilistic prediction model (provided by the mean of its samples), the overall prediction results of the framework, and the corresponding ground truth values.
These figures show that incorporating the results of probabilistic prediction enhances the overall accuracy of the model's predictions. This improvement stems from the D3U framework's diffu- sion model effectively capturing the distribution of components with high-uncertainty in the data. Additionally, mean of the diffusion model's s samples, denoted as Avg(ūg), exhibits noticeable periodicity. Visualizing these samples further reinforces this conclusion. If we treat the time series as a signal, the true distribution of the prediction сттог p(y - Ølt) inherently time-dependent random variable, i.c., stochastic process. Therefore, if the diffusion model successfully learns the distribution p(y y ğlt), it should also model an inherently time-dependent distribution. Fig. 9b) Fig. 10b Fig. 1Tb Fig. [2b] Fig. 13b Fig. 14b and Fig. 15b illustrate 100 samples generated from the distribution learned by the diffusion model. The range covered by these samples reflects the uncertainty of the data-the wider the range, the higher the uncertainty.
evident that the range of coverage exhibits clear periodicity over time and narrows significantly at time steps with lower prediction errors (lower uncertainty). This demonstrates that within the D3U framework, the diffu- sion model effectively captures the distribution of prediction errors rather than merely introducing -ndeieddeendent randomness into the point prediction results.
19

---

Published as a conference paper at ICLR 2025
60
356
120
100
220
(a) ETTml 0th dimension.
-01
Morovation
(b) ETTml 2t dimension.
Figure 6: Visualization of the Denoising Process of ETTml from Diffusion Step 100 Step 0.
20

---

Published as a conference paper at ICLR 2025
150
306
-50
100
200
(a) Electricity 138th dimension.
PRECOO
-21
Misovationn
(b) Electricity th dimension.
Figure 7: Visualization of the Denoising Process of Electricity from Diffusion Step 100 Step 0.
21

---

Published as a conference paper at ICLR 2025
1.5
24
100
300
-50
(a) Traffic th dimension.
NWW
75
Mosovationn
(b) Traffic th dimension.
Figure 8: Visualization of the Denoising Process of Traffic from Diffusion Step 100 to Step 0.
22

---

Published as a conference paper at ICLR 2025
Oevand Turts
65-
1.0
166
ise
200
250
-02
Probeb Estic Predictions CRPS-O.12
1.0
Overal Prodctions
Greand Fush
-05
300
Overall Prodictions 160 moc-0.00
3.0
Ground Tooh
Pratialora
Ovarall Pratitiona
-10
ste
156
282
Pobeblidk: Sarphs
0.-
-25-
-10-
-15
$
zio
Proteblliii Prediction Sample
Figure 9: Visualization of the Predictions of Eletricity (139th dimension). This case illustrates scenario with relatively low epistemic uncertainty. Probabilistic prediction: the mean the diffusion model's samples.
23

---

Published as a conference paper at ICLR 2025
20
Orvand Tirm
-10
-15
166
:s
262
250
64-
Probeb Estic Pradiction, CRPS 12.1
Greand Fush
Overal Prodicioon
15-
80
360
i5g Overall Prodictions, 180 moc-0.00 070
2.0
Groand Pooh
Prutialora
Ovenall Prstition
stc
155
202
Pobeblidk: Shactionarpp
0.-
-25-
-10-
-15
$5
zio
Protailliic Prediction Sample
Figure 10: Visualization of the Predictions of Eletricity (140th dimension). This case illustrates scenario with relatively low epistemic uncertainty. Probabilistic prediction: the mean the diffusion model's samples.
24

---

Published as a conference paper at ICLR 2025
Greand Trzm
0.5
366
Predictions, 3600 mac-0.300 301
15g
202
250
1.-
1.0
Pobabb Padcteee
-05
-10
Isti Predidions, CRPSu0.157
1.-
Orcend tuch Overgt Poeettors
1.
05
60
055
1.0
166
150
Prediction,, m0000.201, mae-o
Oveel Reeicions
100
154
200
250
(a)
anvictiinn Free
evbbaiil Proditionn Sample
Probabiistic Prodicion
Figure 11: Visualization of the Predictions of Solar th dimension). This illustrates
nario with relatively high epistemic uncertainty. Probabilistic prediction: the mean of the diffusion model's samples.
25

---

Published as a conference paper at ICLR 2025
2.0
Green Tun
166
Predictions, m3c-1.076
158
202
250
Pobabb Padctereeere
2.0
10-
Istic Predictons, CRPSH0,342
Orcend tunh
Overtl fooctoos
1.-
-05
150
Peedctiins, mox00,499, mae-o.me
Smane Timh
Ovell Arections
65
05
100
154
200
200
(a)
andicttinn Free
evbbaillio unte Proditionn Sample
Probabiistic
Figure 12: Visualization of the Predictions of Solar th dimension). This illustrates
nario with relatively high epistemic uncertainty. Probabilistic prediction: the mean of the diffusion model's samples.
26

---

Published as a conference paper at ICLR 2025
15-
Pradiction
10-
65-
60-
-65-
-10
115
Po nt Predictions, msc-0.112 mac-0.031
0.-
Prutction Probabi sichredkeerr
66
1.0
Protwb Estic Prediction, CAP 071
Greend Fush
166
150
760
290
15-
Redittee Preatt oi Predittee
Oseent
Dverall
(a)
Anadittion
Probebilncte Prediction Sumpll
Figure 13: Visualization of the Predictions of Traffic t20th dimension). Probabilistic prediction: the mean of the diffusion model's samples.
27

---

Published as a conference paper at ICLR 2025
25-
Pruditiooo
10-
-10
Po nt Predictions, msc-0.200 mac-0.151
1.0
Probubil sichhedkeere
Pruducton
Mond
0.
166
Proteb Estic Prediction, 132
Greend Tush
65
-10
166
15
200
290
Pretitd oF Peelitenn
Oseert
80
Dieeral
(a)
Anadittion
Probebilicte Prediction Samppl
Figure 14: Visualization of the Predictions of Traffic (77th dimension). Probabilistic prediction: the mean of the diffusion model's samples.
28

---

Published as a conference paper at ICLR 2025
Pradiction
60-
Po nt Predictions, msc-0.100 mac-0.02 0..2
Pruditton Probabi sichrrdkeeer
Ga maara
80-
006
166
Protwb Prediction, CAPS
1.-
Greend Push
166
150
200
290
3.0
Preatt oF Peeiriee
Oseent
65-
0.
Dverral
(a)
Anadittion
Probebilncie Prediction Gumpll
Figure 15: Visualization of the Predictions of Traffic hh dimension). Probabilistic prediction: the mean of the diffusion model's samples.
29


Reviews:
[{"review_id": "qXVRVPQHJS", "reviewer": "Reviewer_PGSo", "summary": "This work introduces a novel diffusion-based time series framework called D3U, designed to enhance probabilistic forecasting. D3U effectively decouples the deterministic and uncertain components of the forecasting process. The conditional network is focused on delivering precise point forecasts, while the conditional DDPM (PatchDN) handles high-uncertainty probabilistic forecasting. Additionally, D3U functions as a plug-and-play framework, allowing for easy integration into existing point forecasting models. Experimental results demonstrate substantial improvements over SOTA baselines.", "questions": "Given that other works also integrate diffusion models with deterministic models, what distinguishes your approach from theirs? How is your model superior in design? For example, how does it compare to models like TAMA, which you mentioned in your paper? And what about CARD[1] ?\n\n[1]Han, Xizewen, Huangjie Zheng, and Mingyuan Zhou. \"Card: Classification and regression diffusion models.\" Advances in Neural Information Processing Systems 35 (2022): 18100-18115.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The motivation behind this work is strong. The approach of decoupling point and probabilistic forecasting is well-founded.\n2. The design of the two components to address different forecasting targets is logical and effective.\n3. The experiments are comprehensive, and the improvements in performance are notable.\n4. The writing and presentation are clear and easy to follow.", "weaknesses": "1. A more detailed discussion of the differences between this work and related studies is essential to clarify its contributions. For example, TMDM also integrates deterministic transformer and diffusion models.\n2. Additionally, the selection of baselines could be more comprehensive. Models such as TimeGrad and CSDI, which are relevant probabilistic forecasting approaches, are not included. Currently, the focus is primarily on non-autoregressive models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "du82DaaSGQ", "reviewer": "Reviewer_jQCt", "summary": "The authors describe a novel framework (D3U) for probabilistic time series forecasting. It combines the point forecasting of seasonal and trend components with a generative sampling model for the residual component.\nThe point forecasting part consists of a well-known model (SparseVQ) and is used to produce a prediction $\\hat y$ of the input time series as well as a latent representation $f_{enc}(x)$ which serves as a condition for the generative part.\nThe generative part is a DDPM which shall produce samples of the residual $y-\\hat y$, the reverse process is hereby conditioned on the latent $f_enc(x)$ and the conditional expectation in the reverse process is modelled by a novel patch-based denoising network $g_\\phi$. \nThe network $g_\\phi$ is comprised of a sine-positional encoder for a strided windows representation of the residuals and transformer encoder blocks. It is conditioned on the latent $f_enc(x)$.\nIn experiments, the model shows competitive or better performance compared to SOTA models on different datasets. The authors conclude that the architecture can serve as framework to obtain a generative sampling model of improved performance for arbitrary point forecasting, conditioning models.", "questions": "p2: the residual component tends to contain more uncertainty than the trend and seasonal components: this is time-series specific. \n\np3: contribution 2: is this a contribution? It looks like a second description of D3U and hence an additional paragraph belonging to the first contribution.\n\np4: In statistics, $\\cal F(X)$ is the sigma-algebra generated by $X$. Better use another symbol.\n\np5: The symbols $x$, … are not introduced.\n\np5 l.243: $\\hat y_g+E(\\hat y|x_h)$ should be $\\hat y_g+\\hat y$\n\np6 l. 281: check the formula of $\\mu_\\phi$ (how is it derived? Shouldn't be $a$ instead of $\\bar a$ and the $\\beta$ outside the square root?).\n\np6 l. 297: check the formula $N=\\dots$.\n\np6 l. 322: $p^k$ is not introduced.\n\np7: check the formulas (12) and (13) for typos ($r^k_{Pembed}$ is used multiple times)\n\np10: section 4.3.: there were better results obtained when SparseVQ was replaced by iTransformer. It should be explained why SparseVQ is proposed in the rest of the paper.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The authors show a general way of extending classical point forecasting models to generative sampling models. The paper contains a comprehensive and well-founded survey of existing approaches and works. The results show improved performance compared to SOTA approaches.", "weaknesses": "The general idea to split the prediction of a time series in the prediction of trend/seasonal components and the residual is well-known, as is the usage of generative approaches to model the residual. However, the details of the architecture are new.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4TPSS8S4nb", "reviewer": "Reviewer_igxL", "summary": "The D3U framework is introduced for probabilistic multivariate time series (MTS) forecasting. D3U combines a point forecasting model for high-certainty components with a conditional diffusion model to capture high-uncertainty components. D3U achieves excellent performance across six real-world datasets and includes comprehensive ablation studies.", "questions": "The proposed method requires decomposing the time series data. As shown in Appendix B, decomposing the time series data requires some prior knowledge or specific hyperparameter settings. A natural question arises: when deploying the trained model in a new scenario, you often don’t know how the training data was decomposed. So, how would you decompose the data in the new scenario? Additionally, if the characteristics of the deterministic and uncertain components in the time series data differ between the new scenario and the training data, how can you ensure the model performs well?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Overall, the paper is well-written and easy to follow. The details in each section are well-explained, and the experiments are thorough.", "weaknesses": "The main concern is the originality of the paper. The concept is straightforward: decomposing multivariate time series data into deterministic and uncertain components, using a point forecasting model for the high-certainty component and a conditional diffusion model for the high-uncertainty component. Most of the diffusion design relies on previous work; for example, SparseVQ (Zhao et al., 2024) is used as the conditioning network, and the conditional DDPM is almost the same as the original. The only apparent innovation is the design of the Patch Denoising Network, which itself is not overly complex. Overall, this work feels more like a stitching together of existing methods applied to time series data, with insufficient original contributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "adVzEfcdD7", "reviewer": "Reviewer_pH5Z", "summary": "This paper introduces D3U for probabilistic multi-variable time series (MTS) forecasting, integrating non-probabilistic forecasting with conditional diffusion generation. This enables the framework to produce both accurate point predictions and probabilistic forecasts. Experiments conducted on six real-world datasets demonstrate the superior performance of D3U.", "questions": "1.What are the contribution proportions of the point forecasting model and the diffusion model to the final prediction? It is recommended that the authors provide a table of the respective proportions of point forecasting and probabilistic forecasting for different datasets and include visualizations of some examples. This would help readers understand the roles of the point forecasting model and the diffusion model within D3U and confirm the effectiveness of the diffusion model.\n\n2.The results presented in Figure 1 are interesting. Have the authors explored a similar approach to decompose the series and then used a point forecasting model to predict the deterministic part, while employing a diffusion model to predict the residual obtained from the decomposition?\n\n3.Are the weights of the pre-trained point forecasting model fixed in the D3U framework? Have the authors experimented with fine-tuning the point forecasting model simultaneously while training the diffusion model, or considered creating an end-to-end architecture for the entire model?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The D3U framework innovatively combines a point forecasting model to handle high-certainty components within the time series non-probabilistically, while employing a diffusion model to capture the probabilistic distribution of high-uncertainty components. This approach represents an advancement over previous diffusion models that typically model the entire series.\n\n2.D3U is designed as a plug-and-play framework that can be seamlessly integrated with existing point forecasting models. Additionally, the paper introduces a novel patch-based denoising network, PatchDN.\n\n3.The overall organization of the paper is clear, and the paper is understandable in most parts.", "weaknesses": "1.My primary concern is the effectiveness of the diffusion model within the framework. I have detailed my questions regarding the actual utility of the diffusion model in predictions in Question 1.\n\n2.The diffusion-based time series models included for comparison are somewhat limited, excluding several advanced diffusion-based models developed in the last two years.\n\n3.As indicated in Table 1, the proposed model performs worse than state-of-the-art point forecasting models in most settings.\n\n4.The model's complexity is notably higher compared to traditional point-to-point models, which might restrict its applicability in practical scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)