Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
Authors: J Rishi
GVS Mothish
Deepak Subramani
Venue: ICML 2025 poster (2025)
Keywords: diffusion model
time series forecasting
generative modeling
Primary area: deep_learning->generative_models_and_autoencoders
TL;DR: A generative framework that employs novel nonlinear transformations and learnable conditions in the forward process for time series forecasting

Abstract: Time-series forecasting finds application across domains such as finance, climate science, and energy systems. We introduce the Conditional Diffusion  with Nonlinear Data Transformation Model (CN-Diff), a generative framework that employs novel nonlinear transformations and learnable conditions in the forward process for time series forecasting. A new loss formulation for training is proposed, along with a detailed derivation of both forward and reverse process. The new additions improve the diffusion model's capacity to capture complex time series patterns, thus simplifying the reverse process. Our novel condition facilitates learning an efficient prior distribution. This also reduces the gap between the true negative log-likelihood and its variational approximation. CN-Diff is shown to perform better than other leading time series models on nine real-world datasets. Ablation studies are conducted to elucidate the role of each component of CN-Diff.

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
J Rishi" 1
GVS Mothish
Deepak Subramani
Abstract
networks (Hewamalage et al., 2021), convolutional neural networks (Yue et al., 2022), and transformers (Vaswani, 201 have been used for time-series modeling.
Time-series forecasting finds application across domains such as finance, climate science, and energy systems. We introduce the Conditional Diffusion with Nonlinear Data Transformation Model (CN-Diff), a generative framework that em- ploys novel nonlinear transformations and learn- able conditions in the forward process for time series forecasting. A new loss formulation for training is proposed, along with a detailed deriva- tion of both forward and reverse process. The new additions improve the diffusion model's ca- pacity to capture complex time series patterns, thus simplifying the reverse process. Our novel condition facilitates learning an efficient prior dis- tribution. This also reduces the gap between the true negative log- fog-likelihood and its variational ap- proximation. CN-Diff is shown to perform better than other leading time series models on nine real- world datasets. Ablation studies are conducted to elucidate the role of each component CN-Diff.
In addition to traditional deep learning architectures, diffu- sion models have recently gained prominence as effective tools for generative tasks. Their outstanding performance has been evidenced in various domains, exceeding the ca- pabilities of conventional generative approaches in image generation (Ho et al., 2020; Dhariwal & Nichol, 2021) 1), video synthesis (Harvey et al., 2022; Blattmann et al., 2023), robotics (Mothish et al., 2024) and cross- applications (Saharia et al., 2022). Given this impressive success, there is growing research interest leveraging the potential of diffusion models for time series forecasting, leading the development of several diffusion-based frameworks for time series modeling. Each of these frameworks is specifically designed to tackle the complexities and difficulties that arise in the context of temporal forecasting (Tashiro et al., 2021; Li et al., 2024b; Cao al., 2024; Shen & Kwok, 2023; Shen et al., 2024; Lopez Alcaraz & Strodthoff, 2023). These ap- proaches largely rely on conditional diffusion models. For instance, TimeDiff (Shen & Kwok, 2023) employs future mixup as condition-guidee diffusion model. TimeGrad (Rasul et al., 2021) merges standard diffusion model with the recurrent neural network's 's hidden states. CSDI (Tashiro et al., 2021) uses self-supervised masking to steer a non- autoregressive denoising process. While effective, these time-series diffusion models do not completely exploit dif- fusion model properties.
1. Introduction
Time series forecasting has become essential element in modern data-driven decision-making processes, spanning wide range of fields including financial pricing analy- sis (Kim, 2003), economics (Henrique et al., 2019), trans- portation planning (Huang et al., 2023), energy (Dumas et al., 2022) and various other fields (Li et al., 2024a; Rasul al., 2022). There has been substantial advancement in time series modeling, beginning with traditional statistical techniques such as ARIMA and state-space models, which initially dominated forecasting efforts. Later, various archi- tectures of deep neural networks, such as recurrent neural
Conventional training for conditional diffusion models in- volves gradually adding a fixed linear Gaussian noise at each forward stage. The reverse process is then optimized to closely replicate the forward process but with conditions only applied the reverse process. Firstly, this approach confines diffusion models in the non-learnable forward pro- cess. Secondly, simplifying into an isotropic Gaussian prior complicates the data generation process in the reverse pro- cess for time series applications.
'Equal contribution 'Department of Computational and Data Sciences, Indian Institute of Science, Bengaluru, India. Corre- spondence to: Rishi <rishij@iise ins>.ac.in>>. GVS Mothish <moth- ishg ishh@@ise... @isseiiiccccco. in. Deepak Subramani <deepakns @ in
This paper introduces the Conditional Diffusion with Nonlin- ear Data Transformation Model (CN-Diff) (Refer to Figure 1), a framework incorporating nonlinear time-dependent learnable data transformation into the forward process along-
Proceedings of the nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Conditional Diffusson Model with Nonlinear Data Transformation for Time Series Forecasting
side the learnable condition. This modification results in non-Markovian series of latent variables, each formed by transforming the data and subsequently adding noise, which is learned through the reverse process. Code is publicly available at
through Markov chain adopting the form
po (o p(a=)) II (otat1110))
with p(x²) = -oxx:00,) The integration of the forward process q with the reverse process po is akin (hierar- chical) variational autoencoder (Kingma, 2013a; Rezende et al., 2014). During training. the standard variational bound of the negative log-likelihood is minimized. In the case of DDPM (Ho et al., 2020), the loss to be minimized is
The main contributions of the present paper are as follows.
1. We introduce a nonlinear, time-dependent data trans- formation along with a learnable condition for data generation in the forward process for time series fore- casting.
C Byg
DKL (g(x (x:) (I((f))
log po po x:
2. We develop an objective function based on non- Markovian learnable reverse generative process to train CN-Diff.
Gyear
T
+ DKL (g(2" "1" (x,,2))) pol2" 1001)
3. Experimental results demonstrate that our model sur- passes or matches the performance of other leading time series forecasting models at the time of writing.
Can
Given the fixed nature of process y and distribution po(x") = p(x²), the prior term Lprior can be disre- garded as it does not depend on parameters 0. Since log po is often modeled by a Gaussian distribution with low variance, the reconstruction term Lree also re- mains unaffected by 0. Consequently, the diffusion term Lais is the only part that the model parameters 0 influ- ence. Laiss is the sum of Kullback-Leibler (KL) diver- gences between the posterior distribution in the forward process 0(x(-1/x" x) and the assumed normal distribu- tion p=(=-1|2"" = Na Ho t). (x", in the re- verse process. Here, the variance Dg (a", t) is set to I, while the mean No(a", t) is learned by a neural network with parameters 0. This process is typically viewed as noise estimation or data prediction problem (Benny & Wolf, 2022). To estimate noise, the network forecasts the noise in the diffused input x² and then computes po(x", t) using the formula ( (vor VI-a.ya, parameter 0 is learned by minimizing the loss function C. = Ez,.t ||ee | (x {=()") Alternatively, in the data prediction approach, a denoising network zg is employed to derive an estimate "a t) of the clean data x0 from x', and then set to
2. Preliminaries
2.1. Diffusion Model
Diffusion models are generative frameworks employing la- tent variables. For a data sample x ~ q(x), a forward noising process produces latent variables 2². xx! ,x² ..,"""""""
The reverse process aims reconstruct x by generating the same latent variables in reverse order.
In the standard diffusion model, the forward process fol- lows& a linear Gaussian Markov chain 2015; Ho et al., 2020). In denoising diffusion probabilistic models (DDPM), at step L, x is generated by modifying the previous state =1 (multiplied by var) with zero-mean Gaussian noise and variance (1 oz), that is, o(x*)=((1) = N (z'; Var="=1 (1 a)) We derive the marginal dis- tribution as (<>7(2z)) = NN(x;; /a,ar (1 (1-ōr) --- I), where o, is defined as I Qs.. Hence, x² = V aææ + /1 - âje, with e ~ (O, Leveraging these marginal distributions, the joint distribution for the latent variables z0, x² ,,......x" xT
et al.,
1-0. co('' +)) The
V Var ā(-1)
Von- 18g
Mo (o
x²
2o (t). ai
((2 [x)
g(x 11t1).
1 1- ae
Here, the parameter @ is learned by minimizing the loss
The forward process is typically fixed, lacking trainable parameters, and it is designed so that q(=='\ (x:) z 6(2 z) and g(xF)<)) (x) x NaTT" 0, I). If accessing ((x"-1[x+) were feasible, we could sample from x" NN(x";0,1 00 and reverse it to yield z0 ~ q(=²)= x q(x). The distribution g(at~l[=") depends implicitly on q(x), forming a complex relationship. Hence, we approximate the reverse process
L. = Ev., || xo(x", 1
For time series diffusion models, forecasting zg has been found to be more effective than predicting cg. as shown in (Feng et al., 2024; Shen & Kwok, 2023). Our work adopts this method.
2

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
2.2. Conditional Diffusion models for time series forecasting
In time series forecasting, the aim is to predict future values F1:H R based on past observations T-L+1:0 Rdx Here, L is the lookback window length, H is the forecast window length, and d is the count of variables, with super- scripts and subscripts indicating diffusion time steps and time series values, respectively. When using conditional diffusion models for time series prediction, the following distribution is considered.
(a) DDPM
pu(xe 1xx
a(x x
(b) Conditional DDPM
O:T po 11 (c) po(
II po
(af (i.H.)) c)-
Ta(
where 2HTH (0, 1) and represent the condition, which changes depending on the specific models used (Rasul et al., 2021; Shen & Kwok, 2023; Shen et al., 2024; Li et al., 2024b).
io xx,)
Tx((x",
rendam
scries
The procedure for denoising at step is
MnN
po p c) - ( (o(* Mc),
(c) CN-Diff
During the inference process, the generated sample linked to " is denoted as $'.H- We start by setting oT. as a noise vector sampled (0, I). By repeatedly executing the denoising step outlined in the above equation until reaching t I, we derive the final generated sample, $1.H.
Figure I. Directed graphical models of (a) DDPM (b) Conditional DDPM for time series and (c) CN-Diff
in the forward process can markedly decrease the dispar- ity between the log- log-likelihood and the variational bound, thereby enabling the model more effectively capture tem- poral dynamics and variable interrelationships. Furthermore, embedding conditional information in the formulation aids in synchronizing the prior distribution with the intrinsic temporal dynamics seen in real-world data.
3. Methodology
Consider time series data Jo.H = To. I1, 22, HH, where each z; represents a vector with one or several vari- ables with significant correlation between them at time (Liu al., 2023). Current conditional diffusion models for time series convert these correlated data distributions into an isotropic Gaussian prior by adjusting data points and progressively adding linear Gaussian noise to gener- ate latent variables. Conditional generation is performed only during the reverse process to help in learning these latent variables. This method presents two issues: first, it confines diffusion models during the forward process, ren- dering them fixed and untrainable. Second, converting simple isotropic Gaussian prior complicates data generation during the reverse process. To address these challenges, we propose CN-Diff, a nonlinear data transformation frame- work that learns time-dependent distributions of latent space and incorporates conditional distributions in the forward process.
3.1. CN-Diff formulation and objective
Let us define the nonlinear time dependent data transforma- tion of data for marginal distributions as
ge ( = (zi.n Va,To Va. -1
where (o(11.H,t) : R x [0, + d is a non- linear neural operator parameterized by á that applies a time-dependent transformation the data point 31.H. For of notation, replace Z1.H with in the following discussion.
We now introduce a learnable condition to the forward pro- cess, denoted e, inspired by a similar formulation used in image diffusion (Pandey et al., 2022). Consequently, the marginal distribution is (Figure 1):
In what follows, first, we describe the nonlinear, time- dependent data transformation devised for the forward pro- cess. By integrating conditional information into the for- ward formulation, we create diffusion model specifically designed for time series forecasting. Subsequently, we de- rive the variational bound loss pertinent to this model. We hypothesize that incorporating a more adaptable distribution
qs (x x, c) = Ax(x: V TT,TT
(I)
+ + (1- (1-v Vā)ce (1- a.)
Fort = T along with an appropriately regulated noise sched- ule 0t, or results in (x c) NNc, T). Simply
3

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
Algorithm 1 Training
Algorithm 2 Inference x" ~ (c, for t = T to do
Input: ((11:)).-444
repeat
10,, U[1,T]
= (c,))
x' (x)
x-1 = Cixt + <2C - To(K, () vár + To(,11
Compute the loss in Eq. (4)
() + (o) + o--16
Take numerical optimization step on: V Din- until until converged
end for 8 x~ple (x
put, the Gaussian (c, I) serves as our learnable prior distri- bution and inference requires executing our reverse process on it.
Note that forward process is parametrized by by the trans- formation (-)., making the objective different from (Ho et al., 2020). Consequently, the prior and reconstruction terms depend on Φ, and therefore must be included in the optimization process.
We get the following posterior distribution (x
11x x c)) that satisfies Eq.(1):
In the reverse process approximate posteriors as, (x e) as 1x(x-=1 So (),",)))c) c). Thus, Lasfy and Eprior are
x-1 = -ix' + Eee
xt-1
-x,(x, (xxxt)
(x, t- 1)(sivá + Co) aal
Lass DxL (2o(x' (x)xocc)lloox''111 (x²))
See Appendix A.1 for the detailed derivation.
Given that our forward process exhibits Markovvai characteristicx((qp..)) employing a Markovian reverse pro- cess is inadequate for accurately reconstructing the forward process. We therefore define framework that takes the reverse process non-Markovian, which is one of the many that could be taken. Our non- non-Markovian trainable reverse generative process pa px(x:2
(x, To (1,xxxx,),,0 ()),))=
20 201-111
(SIV + Co) (T (8o(x' 1x-xxxx),,1 -To(x (6)x((11))11--
Sprior
DKL (1o (x |x) xx)])((x?)
2 ar (x, -c12
cl2
0:2") is as follows:
Po (x
p(x)) II Pe (x' 11=0-111")) (xx'))(xx((x))
The complete derivation given in Appendix A.3. It is essential distinguish between the objectives of DDPM and CN-Diff. For DDPM, the objective is to accurately predict only the original data point x. On the other hand, CN-Diff is designed to predict the transformed data point To(2, t) along with the original data point z. Despite this modification, the optimization process for CN-Diff remains simulation-free. thereby facilitating its efficient training through KI. divergence computed samples at all time steps. The procedures for both training and inference are detailed in Algorithms and 2.
where p(x") Axxx :c, I).
For this choice, our Lemma A.I shows that, this approach results in a training objective analogous to that of DDPM even without non linear transform (Proof in Appendix A.
Consequently, the corresponding variational objective of CN-Diff is represented by following form (Appendix A.2).
LEN Dilr Egg
DKL (gs (x" xx, lpp (x* (c))
4. Experiments
Aptior
This section presents time series forecasting experiments using CN-Diff and compares its performance against re- cent deep neural models across nine popular real-world datasets. We begin by describing the datasets, architectural framework, and hyperparameters. We then present results benchmarked against existing forecasting outcomes. An ablation study follows to assess the unique components of our CN-Diff model.
log po (xix',
(4)
frce
DKL (gs (x"-) x, c) lipe (x'-1 (x"-1(x(,)) ((x(,c)
Caia

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
Conditionall network
2110
Bmbeed and Concat
Encoder
Decoder
Denoising neisork
Time Einded
= VorTo(x.1) vvojec vie
x = Gix' Eec TolRa Va
Tu(ks.t- 1KKiv o3
training only
inference ondy
fal/inal prr prediction)
Figure 2. An illustration of the CN-Diff architecture. During the training phase, historical data (Ø-L+1:0) is processed through a conditional network, concatenated with noised data, and subsequently passed through the denoising network. During the inference phase, the model initiates from (c,, f) and performs denoising to obtain (ān).
4.1. Experimental Setup
minutes (Zhou al., 2021). Additional details regarding the dataset's characteristics are available in Table 2.
1.1.1. DATASET
Following (Shen et al., 2024). we adopt a train.validation.ttes ratio of 7:1:2 for Exchange, Weather, Wind, Traffic and Electricity data sets, and 6:2:2 for ETTh1 and Ettml data sets. For Norpool, the training data consists of observations before April 2020; validation data spans from April 1 to October 1, 2020; and testing data is after October 1, 2020. For Caiso, the training set includes data before J January 1, 2020, the validation period extends from January to October 1, 2020, and the test set is post-October 1, 2020. Given that data sets exhibit different sampling intervals, we focus on prediction tasks with suitable prediction lengths based on the characteristics of the dataset (Shen & Kwok, 2023; Shen al., 2024).
We performed an experimental study on nine real-world time series datasets for daily, weekly, and monthly forecasts (Shen et al., 2024; Fan et al., 2022; Zhou et al., 2021; Wu et al., 2021). These datasets are (7) Wind wind power data for 2020-2021 every 15 minutes; (2) Caiso? hourly electricity loads over eight years from various California regions; (3) Traffic hourly road occupancy rates from San Francisco Bay area sensors; (4) Electricity hourly years; (5) Weather
electricity use of 321 clients 10-minute meteorological data for 2020-2021; (6) N eight years of hourly energy production data from various European countries; (7) Exchange" daily exchange rates for eight countries: Australia, United Kingdom, Canada, Switzerland, China, Japan, New Zealand and Singapore (Lai al., 2018); (8) ETThl and (9) ETTm1 are benchmarks for China's electricity transformer temperature data, over two years, ETThl reported hourly and ETTml every 15
4.1.2. IMPLEMENTATION DETAILS
Our CN-Diff model is trained using the Adam optimizer with a learning rate of 10 The training employs a batch size of 64 and incorporates early stopping (maximum of 100 epochs). We use T 100 diffusion steps with a quadratic variance schedule beginning at 91 10-4 and progress- ing 37 10-1 The length of the look-back window is selected from the set (96, 192, 336, 720, 1440), deter- mined by performance evaluations on the validation dataset averaged over ten runs. All experiments are executed on a single Nvidia RTX A6000 GPU with 48GB of vRAM. Figure 2 shows the architecture employed for the CN-Diff model. For condition c, a single dense layer is used. For Tép dense layers with tanh activation used, one layer handling the feature dimension and the other handling the
PicdLesptt.aa/reee///////apppppp datasets//KindPower
hhetgee//www.nnrrrrooLoeeeeococ "httpss:/peessddd...a...../
hitttasr/arcchiee.iss...dddddd
hittpss//www.bbg--jeaaanggg /eetter
hnttaar//aata.nocccoollgggapppep
"tttper//gitnabb.com///iioouo
mulivarratt--cmm--errlo----aaa
5

---

Conditional Diffusson Model with Nonlinear Data Transformation for Time Series Forecasting
Table 1. Multivariate prediction of MSEs on nine real-world time series datasetx(subserrpp) is the rank). CSDI runs out of memory on
Traffic and Electricity. Results of all baselines are from (Shen et al., 2024)
Method
NorPool
Caiso
Traffic
Electricity
Weather
Exchange
ETTh1
ETTml
Wind
Rank
Ours
(,5
0.094())
(,374
0.145(1)
(.296
0.016
0.405
0.988
1.7
mr-Diff
0,645ç3)
(4)
(.474())
(,156
(129
0.016(1)
(.411
(
(.881
3.0
TimeDiff
0.665
(.136(
0.564g)
0. (,193
(3311g3)
0.018m
0.407
0.336
0.896
4.6
TimeGrad
(21)
1.745/22) (225
0.76)/22)
0 (13)
(.079(21)
0. 0.993/23)
0,874(22)
(2209
21.0
CSDI
(_0
(19)
0.3
(2077
0.497
1.066 0.529
10)
15.0
SSSD
(387
(11
(.642
0.250
(.3
(.061
0.726
0.464414;
(8188/20)
14.2
DAVAE
p.7
(184
(.928
0.286
(2375
(2200
(1504
0.362
1.1 18
14.7
CPF
(26))(24)
(2383
1.625
0.793
1. 1,390(24)
0.016
0.482 (.730
1.140(18)
18.9
PSA-GAN
1.501
(.510
1.614g2))
0,535
1.220(22)
0.018
0.623
0.537
1.127
19.1
N-Hits
()
(.131(p)
(.386
0.152(g3
(33
0.017
0
(7
1.033
5.9
FiLM
(
(.628
0.210
0.327 (G)
0.016
0.426,b)
(.347
0.984(4)
6.7
Depts
0.062(4)
(3)
(009
(.319
0.761 (20)
(.0
()
0.350
(.082
12.6
NBeats
(183
(.14
(,373
0.269
1.344/20
0.016
0,586
0.391
(.069(11)
10.8
Scaleformer
0.983
0.207
(
0.195
(z4
0.036
(.48)
1.359
14.7
PatchTST
(18)
(11
(.831
(1225
0.782(21)
0.047 (15)
0.526
0.372
1.077(12)
13.3
FedFormer
(4873
(22
0.591
(22
(3342
0.133 (2)
0.541 (18)
0.426(13;
1.113(15)
13.1
Autoformer
(1940(1s)
0,226(16)
0.688(10)
0.201
(,360;11)
0.056
0.516
0,565
1,083
14.1
Pyraformer
1.008
(21)
(9659
0.273
(.394
(20
0.493 (.57
15.1
Informer
(.9
l (17)
(.664(14)
(.29
(1385
(1073
0.775
0.673
1.168 (196
17.7
Transformer
(18)
(120
0.671(u5)
0.328(19)
(1388(14)
(.062
0.759
0.992(23)
1.201(21)
18.0
SCINet
0.095
0,434(()
0.171(s)
(.329
0.036
0,465
0.350
1.055%))
6.4
NLinear
0. (,
0
(,430
0.147
0.313(4)
0.019(0)
0.410
0.349
0.989;s)
5.2
DLinear
(,
(2461
0,389
0.215
01488(18)
0.022(11)
0.415
0.345
0. 0.899;3)
9.3
LSTMa
1.481 (22)
(.217
0.966
0.414
(.662
0.403
1.14
1.030/24)
1.464(24)
21.2
for wind)) model suggests that wind patterns need to be analyzed at multiple time scales simultaneously. We be- lieve that incorporating CN-Diff at multiple resolutions to build multiresolution CN-Diff could help us to model such datasets better.
Table Dataset characteristics
datasct
dim
#obscrvations
freq.
II (steps)
NorPool
18
70,128
hour
month (720)
Caiso
10
74,472
hour
month (720)
Weather
21
52,696
10 mins
I week (672)
ETTml
7
69,680
15 mins
2 days (192)
Wind
7
48,673
15 mins
days (192)
Overall, the average ranking of the CN-Diff model is 1.7 showing its state-of-the performance. Section 5 has brief descriptions of the other leading models.
Traffic
862
17,544
hour
1 week (168)
Electricity
321
26,304
hour
I week (168)
ETThl
7
17,420
hour
1 week (168)
Exchange
8
7,588
1 day
2 weeks (14)
Table 3 displays the mean squared errors (MSEs) for multi- ple univariate time series datasets. The findings indicate that the CN-Diff excels in four of nine datasets. In the other five datasets, it secures second place in two instances. Overall, the average ranking of the CN-Diff surpasses all other lead- ing models. Additional results for the mean absolute error (MAEs) are provided in Appendix D. Appendix F provides qualitative comparison with various models and includes visualizations for different data sets. The visualizations show that CN-Diff is capable of capturing seasonal patterns without explicitly being trained to do so.
forecast window dimension. More details of the code and hyperparameters are in the Appendix B.
4.2. Main Results
Table I presents the mean squared errors (MSEs) in various multivariate time series datasets. The data reveal that the proposed CN-Diff method achieves superior performance in 6 out of 9 datasets. In particular, the enhancement is particularly pronounced in more complex datasets, namely Norpool and ETTh1. In the remaining three datasets, CN- Diff has Rank 2 in two cases, but only with a very narrow margin with respect to Rank 1 for those data sets. For Wind, CN-Diff achieves the fourth rank. The wind data set rep- resents the highly volatile wind power data. The superior performance of the multiresolution diffusion (mrDiff (rank
4.3. Qualitative analysis
Figure 3 illustrates the prediction results on ETTh1 gener- ated by CN-Diff alongside three competitive models: In- former, DLinear, and FiLM. It is evident that CN-Diff yields predictions of superior quality compared to the other models.
6

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
Table Univariate prediction of MSFs on nine real-world time series datasets (subscript is rank). Results of all other models are from
(Shen et al., 2024).
Method
NorPool
Caiso
Traffic
Electricity
Weather
Exchange
ETTh1
ETTml
Wind
Rank
Ours
2.606
(2
0.126
0.234(3)
0.
0.015
0.066
0.038
2.171(3)
2.0
mr- mr-Diff
(56
(.
0.119(1)
0.234(3)
0.002.))
0.016
0.066
(.039
2.182
2.8
TifeDiff
0,636gs)
(.
0.121
0.232(2)
0.002,1)
0.017(5)
0.066
g.040
2.407
4.2
TimeGrad
1.129
(.325
1.223(an)
0.920/24)
0.002
(2041
0.048 0.078
2.530
17.6
CSDI
0.967 (22)
(15)
0.393/21)
(.520
0.002
(.071(24)
0,083
0,050
2. 2.434(16)
16.7
SSSD
1. 1.145(24)
0.176(13)
0,151
0.370
0.004(12)
0.023
0.097
0.049
3.149(23)
16.1
VAE
(29
(24)
(,151
0,535
(1003
0.019
0.078
0.044
2.679(21)
15.4
CPF
(2260
(.279
0.609
(.002
0.016
0.080
0.04
(.43
13.1
PSA-GAN
0.658g4)
(.1
0.250
0.273
0.035
0.020
0.051 0.084
2.5
14.0
N-Hits
(21 (17)
0.147
0.346
0.
(.089
(.043
2. 2.406
9.4
FiLM
0. (,77)
(,185
0
(,260
(1007;15)
0.018
0.070
0.038 (1)
2. 2.143
8.2
Depts
(6668(6)
(1107(1)
0.
(,380
0. 0.024;20)
p.
0.070
0.046
3.457
11.6
NBeats
(22)
0.125
0.142
(.378
(333)
0.016
0.095
0.048
2.434(26)
12.4
Scaleformer
(17)
(17
(
0.361 (10)
0.035(20) (.009 (18)
0. (5
0.078/23;
2.646
18.2
PatchTST
(,595(1)
(11)
(.177
1.450(18)
0. (2026
(40
0. 0.106
0,052
2.698(22)
16.2
FedFormer
0.891
(1164
0.173
0.376
(.005(13)
(2050
0.076k))
0.065
2.351 (12)
14.4
Autoformer
00946
0.248(18)
0
0,650
(.003;10)
0.041
0,081
0.051,17)
2.349
17.4
Pyraformer
(99
(11
0.136(g)
(.389
(102
0.017
0.076
0.054(20)
2.279(7)
12.6
Informer
(48
(125
0.363
(10
(1023
0.076
0.049
2.297
13.6
Transformer
(192
0.250
(,238
(34)
(1007(15)
0.018(10)
0,092
0,058(21
2.306
16.0
SCINet
0.74 0174
0.1544
0,212
0.272(7)
0.002
0.018(10)
0.071
(3039
2.063(1)
7.1
NLincar
(,708(3)
(z)
0,124
(,231
0.002
0.017
0.070
0.039
2.193(6)
4.3
DLincar
0.67 (c))
0.118(2)
(.139(
0.244(6)
0.168(24)
0.017
0.078(11)
0.041(7)
2.171(3)
7.8
LSTMa
(283
(2253
1.032223)
0.596
0.005
0.031
0.167
0.091
2.299
18.8
For the ablation study on the time-dependent nonlinear data transformation Ta. we consider four variations (Table 5): (i) CN-Diff without introducing any nonlinear transformation; nonlinear transformations (ii) only along the feature dimen- sion, (iii) only along the look-back window dimension, and (iv) along both dimensions. From variation (i) to (ii), the MSE and MAE improved by 54% and 36% for ETTh1, 72% and 52% for Electricity, 57% and 35% for Caiso and 51 % and 30% for Wind. From (i) to (iii), the improvements are similar at 54% and 38% for ETTh1, 70% and 49% for Elec- tricity, 25% and 8% for Caiso and 48% and 27% for Wind. From (i) to (iv), an additional improvement is observed at 57% and 40% for ETTh1, 73% and 53% for Electricity, 61% and 38% for Caiso and 53% and 31% for Wind. Clearly, the incorporation of nonlinear data transformation both dimensions effectively captures the significant correlations present within both the sequence and feature dimensions of time series data.
4.4. Ablation Study
To evaluate the individual effects of each component of the CN-Diff model, we conducted various experiments. Initially, we illustrate the impact of the introduction of the condition (Eqn.1), 1), followed by an examination of the effects of the nonlinear transformation.
In the condition ablation study (Table 4), we present the forecast performance in terms of MSE and MAE when us- ing only the diffusion model (DDPM (Ho et al., 2020)) and when applying the condition only in the backward process, which represents the standard conditional diffusion setting for time series analysis). These numbers compared with the results with our CN-Diff conditional method, where the condition is used in both forward and backward processes. The MSE and MAE improve by 8% and 6% for ETTh] (fea- ture dimension of 7). 16% and 9% for electricity (feature dimension of 321) and 21% and 16% for traffic (feature dimension of 862). This clearly demonstrates that with an increase in the feature dimension, the effect of our condition- ing is enhanced while maintaining the same forecast window. We can infer that, as the feature dimension increases, leam- ing from an isotropic Gaussian prior becomes considerably more challenging and CN-Diff solves this challenge.
We present joint ablation study in Appendix 1, examin- ing the contributions due to variations of nonlinear transfor- mation and CN-Diff condition. Additional ablation studies for different diffusion parameters are found in Appendix E.3, and for various look-back windows in Appendix E.2. The results indicate notable enhancement with each com- ponent added to the diffusion model. Optimal performance with varying diffusion parameters is attained at diffusion
7

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
Tuth
Informer
(h) Dlincar
Ground Tuuth
Ground Trehh
Fradiction
FiLM
(d) CN-Diff
Figure Visualizations of ETTh1 dataset predictions by (a) Informer(Zhou al., 2021) (b) Dlinear(Zeng et al., 2023) (c) FiLM al., 2022a) and (d) CN-Diff
Table 4. Predicting MSEs and MAEs for DDPM, conditional DDPM and CN-Dift condition
Methods
Diff
Cond Diff
CN-Diff Cond
Datasets
MSE
MAE
MSE
MAE
MSE
MAE
ETThi
1.748
0.927
1.030
0.746
0.943
0.704
Electricity
1.784
1.018
0.637
0.564
0.536
0.512
Traffic
1.794
0.885
1.071
0.573
0.848
0.479
noise steps T = 100 and an upper variance schedule of 37 0.1.
strategy involving downsampling, convolution, and inter- action, to harness temporal dependencies in downsampled subsequences. NLinear (Zeng et al., 2023) implements basic approach by normalizing the time series before using linear layer for prediction. DLinear (Zeng et al., 2023) employs seasonal-trend decomposition akin to Autoformer (Wu et al., 2021).
5. Related work
Time series forecasting is utilized in numerous aspects of daily life, ranging from essential applications like traffic and electricity monitoring, and healthcare (Liu et al., 2022b), to more intricate uses such as predicting stock prices, weather, and wind patterns (Shen Kwok, 2023). Due its wide significance, research has advanced from traditional state space models contemporary deep learning methodologies.
Several models utilized the transformer (Vaswani, 2017b) and its variations for time series prediction. Informer (Zhou et al., 2021) used sparse attention to minimize computa- tional load and employed generative-style decoder for rapid long-sequence forecasts in one forward pass. Auto- former (Wu et al., 2021) substituted traditional self-attention with an autocorrelation layer, while Fedformer (Zhou et al., 2022b) incorporated frequency domain mapping through a Trequency-enhance module. Pyraformer (Liu et al., 2022b) introduced pyramidal attention for multi-resolution repre- sentation, and Scaleformer (Shabani al., 2023) adopted shared-weigh multilevel forecasting approach, from broad to fine scales. Inspired by vision transformers (Dosovitskiy et al., 2020), PatchTST (Nie et al., 2023) partitions time series data into subseries patches and uses self-supervised pre-training to extract local semantic features, thus improv- ing long-term prediction.
Basis expansion has been utilized by several models for time-series forecasting. FiLM (Zhou et al., 2022a) em- ployed Fourier analysis and low-rank matrix approximation to reduce noise, along with Legendre polynomial projection to maintain historical data representations. NBeats (Ore- shkin al., 2019) an interpretable architecture that com- bines polynomial trend modeling with Fourier techniques for detecting seasonality. Depts (Fan et al., 2022) builds NBeats by adding periodicity module for periodic series, while N-Hits (Challu et al., 2023) enhances the approach using multi-scale hierarchical interpolation.
Models such SCINet (Liu et al., 2022a) utilize recursive
8

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
Table Predicting MSFs and MAFs by applying Nonlinear Transformations across different configurations
Methods
Without To
Feature dim
Forecast window
Both
Datasets
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Electricity
0.536
0.512
0.150
0.246
0.160
0.260
0.145
0.243
ETTh1
0.943
0.704
0.429
0.447
0.430
0.439
0.405
0.421
Caiso
0.242
0.307
0.104
0.200
0.182
0.283
0.094
0.191
Wind
2.090
1.023
1.015
0.715
1.067
0.743
0.988
0.701
Recently, diffusion models have emerged as a promising method for time series forecasting. TimeGrad (Rasul al., 2021) pioneered the use of conditional diffusion models, leveraging autoregressive prediction informed by RNN hidden states. CSDI (Tashiro et al., 2021), applies non- autoregressive generation through self supervupe masking but relies on dual transformers, facing boundary inconsisten- cies and computational challenges with large datasets. SSSD (Lopez Alcaraz& & Strodthoff, 2023) aims reduce CSDI's computational load using structured state-space models, but it continues with masking- based conditioning, preserving boundary issues. TimeDiff (Shen & Kwok, 2023) intro- duced future mixup and autoregressive initialization within an encoder-decoder scheme for denoising. TMDM (Li et al., 2024b) integrates transformers with a diffusion process for probabilistic multivariate time series forecasting. TimeDiT (Cao et al., 2024) is foundational model for time series, which employed transformer type architecture to capture temporal dependencies and employs diffusion processes to generate samples. mr-Diff(Shen et al., 2024) is a recent study utilizing a multi-resolution strategy, incorporating fine-to-coarse patterns as latent variables aid in denois- ing.
(c,, 2 as we include learnable condition in the forward process. These fundamental differences set our work apart from the above.
6. Conclusion and Future work
This paper presents CN-Diff, a new framework for time series forecasting that combines non-linear data transforma- tion with conditional information in the forward process. We have derived the variational loss for the diffusion model from this framework. Our experiments indicate that CN- Diff matches or surpasses the performance of top time series models across nine real-world datasets. The avenues of fu- ture work include adding an explicit seasonal trend diffusion model, exploring different conditions using varied resolu- tions, and metadata- basee conditional generation. We also plan to extend our model for imputation, telemetry-based anomaly detection, and other engineering applications.
A Acknowledgment
This research partially supported by research grant (M0ES//6/001SSEXx//////000 from the Ministry
of Earth Sciences, Government of India. We are also grateful for the TCS PhD Fellowhip to J Rishi.
One step diffusion (Frans et al., 2024), family of generative models that use a single network and training phase to pro- duce high-quality samples in a single or multiple sampling steps in image diffusion. NDM(Bartosh et al., 2024) ap- plies a non-linear time-dependent transformation for image diffusion models with the Markovian reverse process. Poly- diffuse (Chen al., 2023) uses guided diffusion model for polygonal shape reconstruction, where the prior is learned via guidance networks (a,,, 1, i) and (zx f, i), which are in- dependent of the condition. Also they have not incorporated the condition in the forward process (ref. Fig. 3 [(Chen et al., 2023)]). In contrast, our approach incorporates con- dition in forward formulation results in condition-dependent prior. And our training procedure and loss function dif- fers significantly, as polydiffuse uses separate stages for prior learning and denoising. PriorGrad (Lee et al., 2021) is diffusion-based generative model for speech synthesis, which introduces a prior distribution based on data statis- ties rather than a learned prior. While their reverse process starts by sampling from (0, E), our method samples from
Impact Statement
This article presents new research exploring novel diffusion based framework for modeling time series data. The proposed model exhibits excellent performance when compared to existing diffusion- time series models for forecasting tasks. Our model shows great promise for improving forecast accuracy in diverse applications such as environmental, finance, climate, and industrial domains. Our research is centered on forecasting time series data, which on its own does not have any major ethical concerns. The downstream application developers must exercise cau- tion to ensure that ethical concerns arise when using model for societal applications.
9

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
References
Dumas, J., Wehenkel, A., Lanaspeze, D., Comélusse, B., and Sutera, A. A deep generative model for probabilistic energy forecasting in power systems: normalizing flows. Applied Energy, 305:1 2022.
Bahdanau, D., Cho, K. H., and Bengio, Y. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, 2015.
Esser, P., Kulal, S., Blattmann, A., Entezari, R.. Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F,, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024.
Bartosh, G., Vetrov, D., and Naesseth, C. A. Neural dif- fusion models. In Proceedings of the 41st International Conference on Machine Learning, pp. 3073-3095, 2024.
Benny, Y. and Wolf, L. Dynamic dual-output diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11482- 11491, 2022.
Fan, W., Zheng, S., Yi, X., Cao, W., Fu, Y., Bian, J., and Liu. T.-Y. Depts: Deep expansion learning for periodic time series forecasting. In International Conference on Learning Representations, 2022.
Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S. W., Fidler, S., and Kreis, K. Align your latents: High- resolution video synthesis with latent difftusion models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 22563-22575, 2023.
Feng, S., Miao, C., Zhang, Z., and Zhao, P. Latent diffusion transformer for probabilistic time series forecasting, In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pp. 11979-11987 2024.
Frans, K., Hafner, D., Levine, S., and Abbeel, P. One step diffusion via shorteut models. arXiv preprint 2024.
Cao, D., Ye, W., and Liu, Y. Timedit: General-purpon diffusion transformers for time series foundation model. In ICML 2024 Workshop on Foundation Models in the Wild, 2024.
Ge, S., Nah, S., Liu, G., Poon, T., Tao, A., Catanzaro, B., Ja- cobs, D.,, Huang, J.-B., Liu, M.-Y., and Balaji, Preserve your correlation: A noise prior for video diffusion models. In Proceedings of the IEEE/CVE International Conference on Computer Vision pp. 22930-22944 2023.
Challu, C., Olivares, K. G., Oreshkin, B. N., Ramirez, G., Canseco, M. M.. and Dubrawski. A. Nhits: Neural hi- erarchical interpolation for time series forecasting. In Proceedings of the AAAI conference artificial intelli- gence, volume 37, pp. 6989-6997, 2023.
Harvey, W., Naderiparizi, S., Masrani, V., Weilbach, C., and Wood, F. Flexible diffusion modeling of long videos. Advances in Neural Information Processing Systems, 35: 27953-27965. 2022.
Chen, J., Deng, R., and Furukawa, Y. Polydiffuse: Polygo- nal shape reconstruction via guided set diffusion models. Advances in Neural Information Processing Systems, 36: 1863-1888, 2023.
Henrique, M., Sobreiro, V. A., and Kimura, H. Literature review: Machine learning techniques applied financial market prediction. Expert Systems with Applications, 124: 226-251, 2019.
Das, A., Kong, W., Leach, A., Mathur, S., Sen, R., and Yu, R. Long-term forecasting with tide: Time-series dense encoder. arXiv preprint arXiv:2304.08422 2023a.
Hewamalage, H., Bergmeir, C., and Bandara, K. Recur- rent neural networks for time series forecasting: Current status and future directions. International Journal of Forecasting, 37(1):388-421 2021.
Das, A., Kong, W., Sen, R., and Zhou, Y. A decoder- only foundation model for time-series forecasting. arXiv preprint 200.:22800.088. 2023b.
Ho, J., Jain, A., and Abbeel, Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6851, 2020.
Dhariwal, P. and Nichol, A. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
Huang, X.,
Wu, D.. and Boulet, B. Metaprobformer for charging load probabilistic forecasting of electric vehi- cle charging stations. IEEE Transactions on Intelligent Transportation Systems, 24(10):10445-0045 2023.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2020.
Jeha, P.,
M., Mercado, P., Kapoor, S., Nir- wan, R. S., Flunkert, V., Gasthaus, J., and Januschowski, T. Psa-gan: Progressive self attention gans for synthetic
10

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
time series. In The Tenth International Conference on Learning Representations, 2022.
Liu, S., Yu, H., Liao, C., Li, Lin, W., Liu, A. X., and Dustdar, S. Pyraformer: Low-complexity pyramidal atten- tion for long-range time series modeling and forecasting. In # 2022b.
Jia, Y., Anaissi, A., and Suleiman, B. Resnls: An improved model for stock price forecasting. Computational Intelli- gence, 40(1):e 12608, 2024.
Liu, Y., Guo, H., Zhang, L.,, Liang, D., Zhu, Q., Liu, X., Lv, Z., Dou, X., and Gou, Y. Research on correlation analysis method of time series features based on dynamic time warping algorithm. IEEE Geoscience and Remote Sensing Letters, 20:1-5, 2023.
Kim, K.jj. Financial time series forecasting using sup- port vector machines. Neurocomputingg555( 55(1-2)3307-319, 2003.
Kingma, D. Auto-encoding variational bayes. arXiv 2013a.
preprint
Lopez Alcaraz, J. M. and Strodthoff, N. Diffusion based time series imputation and forecasting with structured atate apace models. Transactions on machine learning research, pp. 1-36, 2023.
Kingma, D. Auto-encoding variational bayes. arXiv preprint arXiv:1312.611 2013b.
Lai, G., Chang. W.-C., Yang, Y., and Liu, H. Modeling long-and short-term temporal patterns with deep neural networks. In The st international ACM SIGIR confer- on research & development in information retrieval, pp. 95-104, 2018.
Mothish, G., Tayal, M., and Kolathaya, S. Birodiff: Dif- fusion policies for bipedal robot locomotion on unseen terrains. In 2024 Tenth Indian Control Conference (ICC), pp. 385-390. IEEE, 2024.
Langley, P. Crafting papers machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICMI. 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations, 2023.
Lec, S.-g. Kim, H., Shin, C., Tan, X., Liu, C., Meng, Q. Qin, T., Chen, W. Yoon, S., and Liu, T.-Y. Prior- grad: Improving conditional denoising diffusion mod- els with data-dependent adaptive prior. arXiv preprint arXiv:2106.00 06406, 2021.
Oreshkin, B. N., Carpov, D., Chapados, N., and Bengio, Y. N-beats: Neural basis expansion analysis for interpretable time series forecasting. In International Conference on Learning Representations, 2019.
Pandey, K., Mukherjee, A., Rai, P., and Kumar, A. Diffuse- vae: Efficient, controllable and high-fidelity generation from low-dimensional latents. Transactions on Machine Learning Research, 2022.
Li, L.., Carver, R., Lopez-Gomez, I., Sha, F., and Anderson, J. Generative emulation of weather forecast ensembles with diffusion models. Science Advances, read4)9) 2024a.
Peebles, W. and Xie, S. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVE Interna- tional Conference on Computer Vision, pp. 4195-4205, 2023.
Li, Y., Lu, X., Wang, Y., and Dou, D. Generative time series forecasting with diffusion, denoise, and disentanglement. Advances in Neural Information Processing Systems, 35: 2022a.
Rangapuram, S. S., Kapoor, S., Nirwan, R. S., Mercado, P., Januschowski, T., Wang, Y., and Schhee-Schneider,, M. Coherent probabilistic forecasting of temporal hierarchies. In International Conference on Artificial Intelligence and Statistics, pp. 9362-9376. PMLR, 2023.
Li, Y., Lu, X., Wang, Y., and Dou, D. Generative time series forecasting with diffusion, denoise, and disentanglement. Advances in Neural Information Processing Systems, 35: 55009-23022 2022b.
Li, Y., Chen, W., Hu, X., Chen, B., Zhou, M., et al. Transformer-modulate diffusion models for probabilis- tic multivariate time series forecasting. In The Twelfth International Conference on Learning Representations, 2024b.
Rasul, K., Seward, C., Schuster, I., and Vollgraf, R. Au- toregressive denoising diffusion models for multivariate probabilistic time series forecasting. In International Con- ference on Machine Learning, pp. 8857-8868. PMI.R, 2021.
Liu, M.. Zeng, A., Chen, M., Xu, Z., Lai, Q. Ma, L., and Xu, Q. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 35:5816 118866588888 2022a.
Rasul, K., Park, Y.-J,, Ramström, M. N., and Kim, K.- M. Vq-ar: Vector quantized autoregressive probabilistic time series forecasting. arXiv preprint arXiv:205..5884 2022.

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
Rezende, D. J., Mohamed, S., and Wierstra, D. Stochastic hackpropagation and approximate inference in deep gen- erative models. In International conference machine learning, pp. 1278- 1286. PMLR, 2014.
time series. In Proceedings the AAAI Conference on Artificial Intelligence, volume 36, pp. 8980-8987, 2022.
Zeng, A., Chen, M. Zhang, L., and Xu, Q. Are transformers effective for time series forecasting? Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 11121-11122 11128, 2023.
Saharia, C., Chan, W. Saxena, S., Li, L.,, Whang, J., Denton, E. L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al. Photorealistic text-to-image dif- fusion models with deep language understanding. Ad- vances in neural information processing systems, 35; 36479-36494, 2022.
Zhang, Z., Meng. L., and Gu, Y. Sageformer. aware framework for long-term multivariate time-series fore- casting. IEEE Internet of Things Journal, 11(10))18435- 18448, 2024.
Shabani, M. A., Abdi, A. H., Meng, L., and Sylvain, T. Scaleformer: Iterative multi-scale refining transformers for time series forecasting. In The Eleventh International Conference on Learning Representations, 2023.
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11106-111 11115,111111111111111111111111 2021.
Shen, I.. and Kwok, Non-autoregressive conditional diffusion models for time series prediction. In Inter- national Conference on Machine Learning, pp. 31016- 31029, PMLR, 2023.
Zhou, T., Ma, Z., Wen, Q. Sun, L.,, Yao, T., Yin, W., Jin, R., et al. Film: Frequency improved legendre memory model for long-term time series forecasting. Advances in neural information processing systems, 35:12677-12690, 2022a.
Shen, L., Chen, W., and Kwok, J. Multi-resolution diffu- sion models for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024.
Zhou, T., Ma. Z., Wen, Q., Wang, X., Sun, L., and Jin, Fedformer: Frequency enhanced decomposed trans- former for long-term series forecasting. In Interna- tional conference machine learning pp. 20222-2222222 PMLR, 2022b.
Sohl-Dickstein, J., Weiss, E., Maheswaranathan. N., and Ganguli, S. Deep unsupervised learning using nonequi- librium thermodynamics. In International conference on machine learning, pp. 2256-2265, PMLR, 2015.
Song. J., Meng, C., and Ernon, S. Denoising diffusion implicit models. arXiv preprini arXiv:2010.02550 2020.
Taga, E. O., Ildiz, M. E., and Oymak, S. Timepfn: Effec- tive multivariate time series forecasting with synthetic data. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp.
2025.
Tashiro, Y., Song. J.. Song. Y., and Ermon, S. Csdi: Con- ditional score-based diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34:24804-24816, 2021.
Vaswani, Attention is all you need. Advances in Neural Information Processing Systems, 2017a.
Vaswani, A. Attention is all you need. Advances in Neurul Information Processing Systems, 2017b.
Wu, H., Xu, J., Wang, J., and Long, M. Autoformer: Decom- position transformers with auto-correlation for long-term series forecasting. Advances in neural information pro- cessing systems, 34:22419-22430, 2021.
Yue, Z., Wang, Y., Duan, J.. Yang, T., Huang, C., Tong, Y., and Xu, B. Ts2vec: Towards universal representation of
12

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
A. Formal Derivations
A.1. Forward Posterior
Given:
- (x(x'x,,)=
+ +11 -vajj, a))))
(5)
From (5), We can write,
x vārTo(x,1) + (1 Vājec vI dre
-V + Volec v1 0:04-116
-Vā,T((x(t) +(1 vāiee + VI - -1aa Vou a
(6)
= =VarTg(x,t)+ vāijc vI --c vaiv 01-16
(x, vā,ec v1 a ase var (x--1_ V
- (1 v van-iic)
= V 1) 11- va,jc +
vI- a,e
By introducing nonlinear, time-dependent transformation of data, along with condition, results in forward process that non-Markovian
q x. WN - Vār (Tx(x - To(x,t 1)) + (1 va,Jc, (1 az)
From (7 and 5), Posterior distribution can be derived as:
as (x- x) 2o(x--1 xxx x,
(x x,
x (x x .),c)
2 (x x, 90 (x .x,)
(8)
10 x 1-1
x, c) (x-11 (1,,,)))) ix,c)
1 (x' vaxx" (T (x, - Ts To (x, t- - 1)) - -(1 -{1-va))c
x exp
1 1- at
(x--1 Va.,
(x, - (1 - -00--va,__a vāy
+
-
at
1x"=1132
x
exp
1 1 -t-1
(2Vorx (x*
(T -To(x, -1))
1)) -
vaikc)
1-a1
-x
(o + (1 voz-)c)
Qt-1
By reformulating the above propotional form, akin to normal distribution, we can see that,
1 xx Nixx 11-110000 oi-1ee
(10)
Where,
Vaxxx= (a - Tx(xx 1)) varll (1 var)c
P4-1/t
1- Ot+
(11)
Vo 1) var-1le) of
1- --
13

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
o2-1/1 (1 0z)(( - a4-1)
(12)
04
Mean((:-1jp) can be simplified as,
(Vaux' Vāivoo -
(1 -a) - -ā
M4-1/
(1-4)
0t
Vo (1- Va 0z) [1 01-
(1-âr-1)
1 1 at
(13)
= -(Varx Vārvar(Ts(xx) - -Tb(x, 1)) varl varlc)
(1 O+-1)
1 ait
(1 a1)
+ 1o vā 1)c)
āq
Mt-1jt (1x + 62 -
Where,
1 0t
(1 Ō(-1)
(Var, - 1)( + vat-1)
6o
a1-1i
S1
Vass
6o =
1+
1 Ozt
1 at
A.2. Loss Formulation
Although our intention was not to explicitly make the forward process non-Markovian, our formulation has resulted in non-Markovian forward process Eq.7. So, we can make use of joint distribution from DDIM (Song et al., 2020) as:
((x0 (x)) = 4(a" (xs)
II
Next, we define Non-Markovian trainable generative process Po (Forr) as:
T po (a p(xx)) II (x t=1
(x' 1- 110 (e (20)z²)
(14)
where p(x") NN(xx (xxT;;
(15)
This formulation is similar to Equation (55) in DDIM (Song et al., 2020). However, the objective in the referenced work is minimize the number of sampling time steps; therefore, certain time steps are employed for image generation, and others included in the variational objective, ultimately demonstrating that the loss formulation aligns with DDPM.
Our definition of trainable reverse generative process (15) is result of introducing nonlinear transformation in the marginal distribution, which makes it challenging to learn from the reverse Markovian process. However, even without non-line non-linearity, the resulting loss will be similar to that of DDPM, (Refer Lemma A.1)
Now, by incorporating this into our variational loss term, the variational loss given as:
Egg
DKL (9s (x x, c) 1p (c))
log po (x/x*, c)
Eptior
Lrne
(16)
DKL. (gs (x x x. pl (x" (x c))
Caite
14

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
Lemma A.I. For pola" ( defined in Eq. 15, variational lower bound results in DDPM training objective. Proof. Based (Ho al., 2020), the variational objective (Fxa) can be expressed as follows:
Jz By [log ((x
mn) -logpo(x":))
T
Bg
log (x= 1x00)
log (0= ( 1xt 1,",,")
log po (x'-1(x²)
log po (2 (x') log po (x
Ey
DKL. (9(x² (x0) (1(x"110") (1x(x)))
DRa. (g(0"~1|a) x0)) 1o(x""=1<<")
Eng
Σ
log pa (z0)
(17)
Cyrior
Edifs
Crea
T Σ
T
priir
+
Eg
1| x0(x", ,t]1?
x,(xx,,)|1
+
Eg
1 - xo(x",
2
20
1=1
1=1
t)]l
CDMPM
Therefore, When predicting z", with trainable non markovian reverse process defined in Eq. 15 will result in DDPM training objective.
A.3. Variational Objective
To calculate the diffusion term Lasss of the objective (4), need to compute the KL divergence between the for- ward posterior distribution 4s (xxx=1[xx c) and the reverse distribution (x""="x",,) Since we use parameterization Po 0x c) = (x , & (x", t), c). both of these distributions are normal distributions with the same variance, so we can evaluate the KL. divergence between them analytically as follows:
DKL (xo(x' ||x" c)lpoo (x= 1x', (x(,))) ,))=
(T
(Sx (),1)) (SIV Co) (To(Ro(x') (- Tx(x, 1)))
1),1)
(18)
20 1-1
We compute the prior term follows:
DKL (x" x, xxcc)l((x(1c))
1
log
I
d T=(4--o00 (1--)))((06_0--0-)-----------
2
1
dog of d + || 1a 7) - T
2
(19)
1
(d (d log of 1) + âr [x](x <12)
2
2 ar (T) clf
The reconstruction term is considered for all latent variables similar to VAE(Kingma, 2013b)
B. Implementation Details
The MLP modules within the embeddings of the condition and noised input, well the decoder, from TiDE (Das et al., 2023a). This MLP framework is acknowledged as effective component for universal time series analysis models (Das et al., 2023b). The encoder employs the diffusion transformer block (DiT), with the diffusion time step adjusted by adaptive layer normalization (Peebles Xie, 2023; Esser et al., 2024). Hyperparameters are tuned specifically regarding embedding layers, encoder layers, and hidden dimensions while other hyperparameters remain constant as detailed in Section 4. 1.2.. Optimal hyperparameters are selected via validation and summarized for various datasets in Table B.
15

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
Table 6. Best hyperparameters for different datasets
datasets
hidden dimension
embedding layers
encoder layers
NorPool
256
Caiso
128
Weather
64
ETTml
512
Wind
512
1
Traffic
64
2
2
Electricity
256
2
ETThl
256
3
Exchange
64
C. Baselines
We have thoroughly benchmarked our CN-Diff model against 23 baseline models using variety of methodologies, from basis expansion techniques diffusion models. These include: (i) diffusion-based models (Shen al., 2024; Shen & Kwok, 2023; Rasul al., 2021; Tashiro et al., 2021; Lopez Alcaraz & Strodthoff, 2023); (ii) basis expansion methods such as those detailed in (Challu et al., 2023; Zhou et al., 2022a; Fan et al., 2022; Oreshkin et al., 2019); (iii) alternative generative techniques like GAN and VAE as noted in (Li et al., 2022a; Rangapuram et al., 2023; Jeha et al., 2022). We also evaluated CN-Diff against transformers (Shabani et al., 2023; Nie et al.. 2023; Zhou et al., 2022b; Wu et al., 2021; Liu et al., 2022b; Zhou et al., 2021; Vaswani, 2017b) and other deep learning approaches (Liu al., 2022a; Zeng et al., 2023; Bahdanau et al., 2015). Note that we don't implement these baseline models, Instead, we rely on the reported numbers literature.
D. Results of MAE
Tables 8 and present the mean absolute error (MAE) results for multivariate and univariate time series forecasting tasks. It is evident that CN-Diff continues to demonstrate a superior overall performance for MAE as the evaluation metric. The average rank difference for multivariate is significant, but for univariate, it is almost comparable to mr-Diff. Furthermore, acknowledging that the performance of deep models in time series forecasting can be affected by various random initializations, Table 7 illustrates the prediction results for univariate time series five distinct random runs.
Table Univariate prediction emror of CN-Diff obtained on five
Exchange
ETTh1
Weather MSE
Electricity
MSE
MAE
MSE
MAE
MAE
MSE
MAE
0
0.0152
0.0933
0.0660
0.1991
0.0025
0.0342
0.2370
0.3465
1
0.0154
0.0934
0.0649
0.1975
0.0021
0.0341
0.2268
0.3387
2
0.0153
0.0936
0.0656
0.1985
0.0025
0.0343
0.2289
0.3426
3
0.0161
0.0950
0.0670
0.2001
0.0027
0,0349
0,2393
0.3431
0.0157
0.0943
0.0644
0.1977
0.0024
0.0346
0.2392
0.3429
mean
0.0155
0.0939
0.0656
0.1986
0.0024
0.0344
0.2342
0.3427
std deviation
0.0004
0.0007
0.0010
0.0010
0.0002
0.0003
0.0059
0.0027
16

---

Conditional Diffusson Model with Nonlinear Data Transformation for Time Series Forecasting
Table Multivariate prediction of MAEs on nine real-world time series (ataascts(subserritss is the rank). CSDI runs out of memory on Traffic and Electricity. Results of all baselines are from (Shen ct al., 2024)
NorPool
Caiso
Traffic
Electricity
Weather
Exchange
ETTh1
ETTml
Wind
Rank
Ours
(3554(1)
0.191(1)
0.270
0.243/2)
(2324(g)
0.079
(.421
0.378
0.701
2.4
Diff
(.604/3)
(.219
(,3
0,252,1)
(.324(2)
(.
0.422
(,373
0.675
3.2
TimeDiff
0.611
(r234(r)
0,384(g)
0.305(r)
0.312
0.091 (8)
0.430
0.372(1)
0.687:3)
4.9
TimeGrad
0.821 (20)
(133
(284
0.630(22)
(.38
(2193
0.719
0.605
0.793
20.8
CSDI
0.777
(.345
0.374:13)
0.194(21)
0,438
0,442
(.74](11)
15.0
SSSD
(17)
(.2
0
0.363
(.350
0.127
0.561
0.406
0.778
13.6
D VAE
0.692(10) (0)
(133
(.483
(.372
(.380
0.301 (29)
(.502
0.391
0.779
15.7
CPF
(28)
(2424(22)
0.714
0.613
0.781 (24)
0,082(4)
(2597
0.472
0.757 (16)
19.0
PSA-GAN
(28)
(2477
(a6
0,533
0.578
0.087
0.546
0.488
0.756
18.7
N-Hits
(.643
(,221
0,268
0.245;3)
(3335
(.085
0.450
0.388
0.734:9)
6.0
FiLM
0.646/g)
(,278
(,398
()
(,336
0.079 (1)
(,436
(.374(3)
0.717
6.9
Depts
0.61
(.204(3) (9)
0.568(20)
(.40
(m394(mm)
0. (100
0.491 (13)
0.751 0.412
12.3
NBeats
0.832
(32)5
0.265(1)
0.370(14)
0. (1420
(_081(3)
0.52
0. (.409
0.741
11.6
Scaleformer
(131 (17
(.304())
0. (199
0. 0.138
(.579
0.475
0.864
15.4
PatchTST
(17
(,203
0,411am
0.348
0 0,555
0.147
0,489
0.392
0.720,p
13.0
FedFormer
0.744
(1417
(.385
0.341
0.347 (8)
(2233
0.484
0.413;14;
0.762
13.1
Autoformer
(175)
(13)
(33
0,313:5)
(.354
0.167
(.484
0.496;20)
0. 0.756(14)
13.3
Pyraformer
0.781
(217
(1390(11)
(.379
(1385
0.112
0.4933(14)
0,435
(,7
14.9
Informer
0.757
(.336
(139
0.383
(1364((1)
(1192(18)
0.605
0.542
0.772(18)
17.0
Transformer
(17)
(
0.410
0.405
0.370
0.178(18)
(.567
0.502
0.785
17.6
SCINet
0.001/2)
0.193
(7
0,280
(.344m
(.137
0.463
0.389
0.732
6.8
NLincar
0.636
(.223
(
0.239(1)
(.328
(.09
0,418
0.375(a)
0.706
4.4
DLinear
0,640(z)
(24
(,268
0,336
2.444
(1102
0.442m
0.378(g)
0.686
9.8
LSTMa
0.974(24)
(1305
0,510(19)
0.444(2)
0.501/20)
(2534
0.782(24)
0.699(24)
0.897 (24)
21.0
Table Univariate prediction of MAEs on nine real- world time series datasets(subseripts is the rank). Results of all baselines are from (Shen et al., 2024)
Method
NorPool
Caiso
Traffic
Electricity
Weather
Exchange
ETTH1
ETTml
Wind
Rank
Ours
(3
(2206(2)
0.206
0.343
0.034(4)
0.094,1)
0.199
0.14
L.18:
2.2
mr-Diff
(3609(3)
(.212(5)
0.197
0,332
0.032
0.094
0.196
0.149
L.168
1.9
TimeDiff
0.613(4)
(4)
(.207())
0,341(33
(3
0,102
0.202
0.154m
1.209
4.9
TimeGrad
(284
(,386
0.894(24)
(2898(24)
0.036mm)
0.155(22)
0.212
0,167
1.239(12)
17.6
CSDI
0. (27
(1282
(2468
0.540
0.037(8)
(.200
0.221
0.170
1.218:8)
16.1
SSSD
(2770
(,263/13)
(,226
0 0.403
0.041
0.118
0,250
0.169
1.356
15.3
DEVAE
0.774/23)
(2613
0.237
(.53
0.039 (10)
0.107
0.221 (18)
0. 0.160
1.321 (20)
15.9
CPF
(1710
0.338 (19)
0,385
0.502,22)
0. (,035
0.094(1)
0.221
0.153
1.256
12.8
PSA-GAN
(,623/6)
0.250
0.355(18)
0.373
0. (39
0. (15)
0.225
0.174(17)
1.287
14.3
N-Hits
(.646(g)
0.276/14)
(,2322())
0.419(10)
p,033
0.100
0.228
0.157
1.256(13)
9.8
FiLM
(.2 (3654
0.315
0.362
(1069(15)
0.104
0.210
0.149 (L)
1.189(4)
9.4
Depts
0.616
0.205 (1)
0.241
0.434 13)
0. 0.102(20)
0. 1.16
0.202
0. 0.165
1.472
11.2
NBeats
0.671
0. (,)
0,225kg)
0.430
0. 2.130(22)
0.096(a)
0.242
0. 0,165
1.236
11.4
Scaleformer
(.3 0.687
0.375
0.430
(.083
0. 0.148
0.302
0.210
1.348
18.4
PatchTST
0,590,1)
(,260(12)
0,269
0.478(18)
0. (.098(19)
0.111
0.260
0.174,17)
1.338(21)
15.3
FedFormer
(.725
0.254(10)
0,278(19)
(3453
(.057(24)
0,168
0.212
0.195
1.271
15.3
Autoformer
0,755
(.339
0.495(am)
0. 0.623
0. 0.132(gm) 0.040;11)
0.220
0.174(m7
1.319
18.3
Pyraformer
0.257 (.747
(11)
0.215
0.455
0. 0.107 (21)
(.104
0.211
0.179(20)
1.284
13.1
Informer
0.698 (14)
(2345
0.308
0.4
0. 0.118 (.069
0.212
0.172
1.236
14.1
Transformer
(.7)
(.345
0,336
0.469
0.071 (17)
0.10331a0
0.247
0,196
1.212
16.4
SCINet
(,653
0.244
0.322
0.377
0.037 (8)
0.101
0.205
0.150(5)
1.167
7.6
NLincar
(.
0,238
0.192
0.334
0.033
0.097
0.203
0.149
1.197
3.9
DLincar
0.671(11)
0,206
0,236
0.348
(3310
0.102
0.222
0.155
1.221
10.2
LSTMa
0.707
0.333 (18)
0.757
0.557
(.053
0. 0.136
0.332
0.239
1.298
19.3
17

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
E. Ablations
In this section, we present ablation studies for various hyperparameters, beginning with an examination of different components within CN-Diff, well as the look-back window, diffusion time steps, and Gaussian variance schedule.
E.1. Architecture components
Table E. presents an analysis of the various components integrated into CN-Diff. Beginning with DDPM (Ho et al., 2020), we provide results that illustrate the effects of incorporating only non non-linearity, the conditional aspect independently, and the combined integration of both into the DDPM, evaluated across multiple datasets.
Table 10. Prodicting MSE and MAE for different components of CN-Diff for Multivariate forecasting (improvement relative to Diffusion denoted within brackets.)
Methods
Diffusion
To
+ CN-Diff Cond
All (CN-Diff)
Datasets
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
1.748
0.927
1.320(25%)
0.730(21%)
0.943(46%)
0.704(24%)
0.405(77%)
0.421
Exchange
0.090
0.219
0.016(82%)
0.080(63%)
0.021(60%)
0.101(54%)
0.015(83%)
0.079664%)
Caiso
0.586
0.553
0.403(31%)
0.433(22%)
0.242(59%)
0.307(44%)
0.094(84%)
0.191(65%)
Norpool
1,819
1,047
1.41142%)
0.918(12%)
1.142(37%)
0.814(22%)
0.531(71%)
0.554(47%)
Electricity
1.824
1.030
1.597(12%)
0.951(8%)
0.536(71%)
0.512(50%)
0.145(92%)
0.243(76%)
Traffic
1.794
0.885
1.397(22%)
0.799(10%)
0.848(53%)
0.479(46%)
0.374(79%)
0.270(70%)
E.2. Look back Window
Table E.2 presents the prediction MSE and MAE for CN-Diff utilizing various lengths of the lookback window. The values considered are L= [96, 192, 336, 720, 1440). evident that on the datasets Electricity, ETTh1, and weather (corresponding to 168-step. 168-step, and 672-step-ahead predictions, respectively), satisfactory performance is achieved when L is set 336 higher.
Table II. Predicting MSE and MAE different Lookback windows (L) for Multivariate forecasting
Look back window
96
192
336
720
1440
Datasets
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
0.433
0.437
0.426
0.437
0.405
0.421
0.434
0.448
0.440
0.452
Weather
0.530
0.503
0.347
0.367
0.339
0.354
0.307
0.335
0.296
0.324
Electricity
0.167
0.262
0.152
0.248
0.149
0.246
0.148
0.246
0.145
0.243
E.3. Diffusion Parameters
Diffusion steps, denoted T, is critical parameter. Table E.3 presents the MSE and MAE of CN-Diff a function of T. It is evident that establishing T = 100 results in consistent performance in all three datasets examined. This observation is consistent with the findings in (Li et al., 2022b), which suggest that a minimal value of T may result in incomplete diffusion, while an excessively large K might lead to superfluous computational overhead. all datasets analyzed, the best length of the look-back window was considered.)
The variance of Gaussian noise Be, formulated through the implementation of a quadratic variance schedule, In present study, the variable 8r varied within the set (0.001, 0.01, 0.1. 0.9). while is maintained a constant value of 10-4, As evidenced in Table E.3, optimizing the parameter setting Br 0.1 consistently results in enhanced performance.
18

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
Table 12. Predicting MSE MAE for different diffusion noise steps (T) Multivariate forecasting
Noise Steps
50
100
200
500
Datasets
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1 0.415
0.431
0.405
0.421
0.409
0.426
0.422
0.437
Weather
0.299
0.330
0.296
0.324
0.325
0.360
0.362
0.378
Electricity
0.145
0.244
0.145
0.243
0.145
0.243
0.147
0.244
Table 13. Predicting MSE and MAE for different variance upper bound (fr) for Multivariate forecasting
Br 0.001
0.01
0.1
0.9
Datasets
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
0.807
0.656
0.462
0.463
0.405
0.421
0.425
0.438
Weather
0.312
0.347
0.340
0.375
0.296
0.324
0.357
0.373
Electricity
1.745
0.986
0.243
0.345
0.145
0.243
0.146
0.243
F. Qualitative analysis
Hittay
Groadd
Groand Trun
Weather
(b) Caiso
Mt. mie
(c) Electricity
(d) Norpool
Figure 4, Visualization CN-Diff forecasts across various datasets
The qualitative analysis presented in Figure demonstrates the forecasting capabilities of our CN-Diff model across four diverse datascts: Weather, Caiso, Electricity, and Norpool. These visualizations provide strong evidence of the model's effectiveness in capturing temporal patterns and generating accurate predictions across varying domains. Figure 4(a), the Weather dataset results show CN-Diff successfully tracking the overall trend and seasonal patterns, with predictions (green line) closely following the ground truth (blue line). For the Caiso dataset in Figure 4(b), the model effectively captures the high- figh-frequency oscillations that characterize this time series, with particularly strong alignment between the predicted and actual values in the latter portions of the sequence. The Electricity dataset results in Figure 4(c) further illustrate CN-Diff's ability to model periodic patterns with consistent amplitude and frequency matching the ground truth. Finally, Figure 4(d) showcases results on the Norpool dataset, where our model accurately predicts the irregular fluctuations and demonstrates robustness to the dataset's inherent volatility.
19

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
G. Computational analysis
We present the computational cost analysis to compare our model with other diffusion-based time series forecasting methods. Refer the tables for training (Table: 14) and inference (Table:15) comparisons (Results for other baselines are taken from (Shen et al., 2024)). Note that, SSSD utilizes structural state-space-based diffusion layers along with multiple dense layers; while TimeGrad leverages RNN hidden states; TimeDiff incorporates future mean and autoregressive initialization; and mr-Diff employs multiple diffusion models. In contrast, our innovation lies in the diffusion formulation rather than in the architectural modifications. Both key components we introduced, the nonlinear transform (-)( and the condition network, consist of simple linear layers with an activation function. This design choice allows us to achieve greater computational efficiency compared to existing diffusion models for time series forecasting, and this can be realized by the table below.
Table 14. Training time (ms) for different models and sequence lengths (If) for ETThl univariate.
H
96
H
168
H = 192
H
336
H
720
CN-Diff
0.21
0.26
0.27
0.31
0.39
mr-Diff
0.59
0.69
0.71
0.74
0.82
TimeDiff
0.71
0.75
0.77
0.82
0.85
TimeGrad
2.11
2.42
3.21
4.22
5.93
CSDI
5.72
7.09
7.59
10.59
17.21
SSSD
16.98
19.34
22.64
32.12
52.93
Table Inference (ms) different models sequence lengths (H) for ETThl univariate.
Model
Trainable Params
H
96
H
168
H
192
H
336
И 720
CN-Diff
1.1M
6.2
6.7
6.9
7.8
9.1
Mr-Diff
1.4M
12.5
14.3
14.9
16.8
27.5
TimeDiff
1.7M
16.2
17.3
17.6
26.5
34.6
TimeGrad
3.1M
870.2
1620.9
1854.5
3119.7
6724.1
CSDI
10M
90.4
128.3
142.8
398.9
513.1
SSSD
32M
418.6
590.2
645.4
1054.2
2516.9
H. Additional Results
H.1. Additional Datasets
In order further validate our approach, have carried out supplementary experiments employing multivariate synthetic data. This data utilizes Gaussian process kernels as described by (Taga al., 2025). Additionally, incorporated the CSI-300 dataset, which represents the wrightee average index of 300 prominent companies listed on the Shanghai and Shenzhen Stock Exchanges, cited in (Jia et al., 2024). The results of these experiments, detailed in Table 16 below, indicate that our model achieves higher level of performance when benchmarked against other existing models.
Table Performance comparison CN-Diff across models on Stock and Synthetic datasets using MSE and MAE metrics. Synthetic (MSE)
Model
Stock (MSE)
Stock (MAE)
Synthetic (MAE)
CN-Diff
0.020
0.109
0.722
0.308
Autoformer
0.058
0.190
0.767
0.339
PatchTST
0.028
0.129
0.771
0.345
FiLM
0.040
0.159
0.752
0.319
DLinear
0.044
0.170
0.747
0.311
CSDI
0.037
0.169
0.809
0.506
20

---

Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting
H.2. Additional Models
In our experiments, have compared our method with CPF(Rangapuram al., 2023), a model using Graph Neural Networks. To further strengthen analysis, we now incorporate comparison with SageFormerZZaang et al., 2024) well, which is series- graph-enhanced Transformer model for time series forecasting. The table below 17 demonstrates CN- Diff's superiority over graph-based based methods in time series forecasting.
Table 17. Comparison of Model Performance Metrics across different Datasets
Dataset
CN-Diff (MSE/MAE) Sage Former (MSE/MAE)
ETTml
0.340 0.378
0.370 0.388
Electricity
0.145 0.243
0.159 0.255
Stock
0.020 / 0.109
0.030 0.135
Exchange
0.016 / 0.079
0.016 0.081
ETTh1
0,405 / 0.421
0,421 0,419
Weather
0.296 0.324
0.324 0.344
H.3. Analysis of Non-linear Transformation(T..)
In Table and E.1, we have shown empirical evidence through the ablation results presented that CN-Diff performs better than the ablation without Tg. To understand what is learned by Té. take trained model and explore the correlation between the features in the learned latent representation space of T Ta (xr t) different diffusion model timesteps. We observed that there are increasing correlations between the features in learned latent space different timesteps as shown in tables below. Thus, we hypothesize that this increased correlation and time-dependent adaptability perhaps facilitate more effective diffusion process for time series forecasting. Similar observations of correlation in latent space that help diffusion models have been made in image and video diffusion works (Ge et al., 2023).
In the space of diffusion models for time series, paper (Tashiro al., 2021) has noted that learning the correlation between feature and temporal space is necessary for time series imputation tasks. We will add these the discussion in the ablation study section in the revision.
Table 18. Feature Correlation Matrix for Actual input (ETThl dataset)
Features
4
6
0 1.00
0.08
0.99
0.07
0.22
-0.20
-0.19
1
0.08
1.00
0.08
0.94
0.05
0.22
-0.06
2
0.99
0.08
1.00
0.09
0.12
-0.27
-0.19
3
0.07
0.94
0.09
1.00
-0.10
-0.06
-0.07
0.22
0.05
0.12
-0.10
1.00
0.56
-0.06
-0.20
0.22
-0.27
-0.06
0.56
1.00
0.11
-0.19
-0.06
-0.19
-0.07
-0.06
0.11
1.00
Table 19. Feature Correlation Matrix for Transformed Input T.(.) (ETThl dataset)
Features
1
4
6
0
1.00
-0.52
-0.60
0.99
0.93
-0.96
-0.56
-0.52
1.00
-0.31
-0.58
-0.68
0.60
0.63
-0.60
-0.31
1.00
-0.57
-0.33
0.55
-0.08
0.99
-0.58
-0.57
1.00
0.94
-0.97
-0.59
4
0.93
-0.68
-0.33
0.94
1.00
-0.86
-0.80
5
-0.96
0.60
0.55
-0.97
-0.86
1.00
0.43
-0.56
0.63
-0.08
-0.59
-0.80
0.43
1.00
21


Reviews:
[{"review_id": "RebBlCgPnH", "reviewer": "Reviewer_4fES", "summary": "The paper presents a conditional diffussion method for time series forecasting. The proposed method CN-Diff adds a non-linear  transformation in the forward process. This transformation is a learnable paramter and results in non-Markovian series of latent variables. These latent variables are learned in the reverse process. An objective function is developed for training.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **Questions For Authors:**\n**Q1: The proposed method is claimed to do a better job of learning the patterns in the time series.Will not the use of synthetic data will demonstrate better the effectiveness of the method?**\n\n**A1:** We agree with your reasoning that the structured nature of synthetic data will make the effectiveness of the model more apparent. To address this, we have conducted additional experiments using multivariate synthetic data generated with Gaussian process kernels following [1] . As shown in the table below, our model demonstrates superior performance compared to other models, even with synthetic data.\n\n**Table 1:** Performance comparison across different models on synthetic dataset.\n| Model       | **Synthetic (MSE)** | **Synthetic (MAE)** |\n|------------|--------------------|--------------------|\n| **CN-Diff**  | **0.722**          | **0.308**          |\n| Autoformer  | 0.767              | 0.339              |\n| PatchTST    | 0.771              | 0.345              |\n| FiLM        | 0.752              | 0.319              |\n| DLinear   | 0.747            | 0.311            |\n| CSDI        | 0.809              | 0.506              |\n\n- [1] TimePFN: Effective multivariate time series forecasting with synthetic data. arXiv preprint.\n\n**Q2: What are the specific reasons that in your experiments, for some datasets, other sota methods perform better than the proposed method? Can you please provide detailed reasoning that related to characteristics of these datasets?**\n\n**A2:** We reiterate that CN-Diff achieves the best average rank across multiple datasets. For Traffic and ETTm1 datasets, CN-Diff achieves the second rank. For Wind, CN-Diff achieves the fourth rank. The wind data set represents the highly volatile wind power data. The superior performance of the multiresolution diffusion (mrDiff (rank 1 for wind)) model suggests that wind patterns need to be analyzed at multiple time scales simultaneously. We believe that incorporating CN-Diff at multiple resolutions to build multiresolution CN-Diff could help us to model such datasets better.\n\n**Q3: How will the proposed approach perform on other domains e.g. Stock Market?**\n\n**A3:** We have added new experiments to further evaluate CN-Diff using stock market data. The table below highlights the effectiveness of CN-Diff, showing that it outperforms other models even in this setting. We have used CSI-300 (the price-weighted average index of 300 top-rated listed companies on the Shanghai and Shenzhen Stock Exchange) closing price as a stock market data following [2].\n\n**Table 2:** Performance Comparison Across Different Models on Stock Dataset\n\n| Model       | **Stock (MSE)** | **Stock (MAE)** |\n|------------|----------------|----------------|\n| **CN-Diff**  | **0.020**      | **0.109**      |\n| Autoformer  | 0.058          | 0.190          |\n| PatchTST  | 0.028       | 0.129   |\n| FiLM        | 0.040          | 0.159          |\n| DLinear     | 0.044          | 0.170          |\n| CSDI        | 0.037          | 0.169          |\n\n- [2] ResNLS: An improved model for stock price forecasting. Computational Intelligence.\n\n**Q4: How are the graph transformer methods, e.g., STGNN, compare with the diffusion methods?**\n\n**A4:** In our experiments, we have compared our method with CPF [3], a model using Graph Neural Networks. To further strengthen our analysis, we now incorporate a comparison with SageFormer [4] as well, which is a series-aware graph-enhanced Transformer model for time series forecasting. The table below demonstrates CN-Diff's superiority over graph-based methods in time series forecasting. \n\n**Table 3:** Comparison of CN-Diff and Sage Former across different datasets with MSE and MAE metrics\n\n| Model       | ETTm1 (MSE/MAE) | Electricity (MSE/MAE) | Stock (MSE/MAE) | Exchange (MSE/MAE) | ETTh1 (MSE/MAE)| Weather (MSE/MAE) |\n|------------|----------------|----------------|----------------|----------------|----------------|----------------|\n| **CN-Diff**    | **0.340** / **0.378**  | **0.145** / **0.243**  | **0.020** / **0.109**  | **0.016** / **0.079** | **0.405** / 0.421 | **0.296** / **0.324**  |\n| Sage Former | 0.370 / 0.388  | 0.159 / 0.255  | 0.030 / 0.135  | 0.016 / 0.081  | 0.421 / **0.419**  | 0.324 / 0.344  |\n\n- [3]  Coherent probabilistic forecasting of temporal hierarchies, PMLR.\n\n- [4] SageFormer: Series-aware framework for long-term multivariate time-series forecasting. IEEE IoT Journal. \n### **Other Comments Or Suggestions:**\n\nThanks for your valuable feedback. We will ensure that the figures and tables are closer to the corresponding text descriptions to improve readability. We will also include a reference to Table 1 in the dataset section.  The schematic Figure 2 explains the architecture and algorithms. We will slightly modify it to clearly call out algorithms 1 (training) and 2 (inference) in Figure 2. We appreciate your careful review and will make the necessary revisions accordingly. Thank you!", "claims_and_evidence": "1. The main claim is that introduction of non-linear transformation makes the reverse process non-Markovian is supported by mathematical proof in the main paper and the supplement.\n\n2. The effectiveness of developed objective function is supported by experiments and ablation study.\n\nThe overall effectiveness of the proposed approach needs more detailed analysis of the results, specifically, why other methods perform better in some of the cases. Furthermore, the use of synthetic data will greatly help in this regard.", "methods_and_evaluation_criteria": "The proposed method is evaluated across various datasets: wind power, Caiso (hourly load), electricity loads, Traffic, electricity and Weather. The comparisons are done with SOTA diffusion and other transformer methods.", "theoretical_claims": "I believe there no theoretical claims made in the paper, as the paper proposes a conditional diffussion method for time series. Although, I did not checked the correctness of the derivation of equations for the non-linear transformation and consequent non-Markovian process, objective function and proofs.", "experimental_designs_or_analyses": "The datasets used are usually the ones that are used in SOTA time series forecasting methods. The sota methods used for comparisons are the current diffusion and transformer-based methods.\nSome experiments should be done using synthetic data.", "relation_to_broader_scientific_literature": "The paper proposes a conditional diffusion approach for time series forecasting and I believe does not relate to broader scientific literature.", "essential_references_not_discussed": "The works referenced in the paper are adequate.\n\n Although I believe the Graph Deep learning methods (e.g. STGNN) should be discussed.", "other_strengths_and_weaknesses": "Strengths:\n1. non-linear transformation in the forward process resulting in non-Markovian reverse process.\n2. Derivation of the objective function\n3. Mathematical equations and proofs\n4. Ablation study\nWeaknesses\n1. Need to use Synthetic data to demonstrate the effectiveness of the approach\n2. Need to consider datasets from more domains e.g. stock market\n3. Need to discuss specifics in the comparison results as why for some datasets the proposed method is second in performance", "other_comments_or_suggestions": "1. Please add a figure that shows the network architecture of the proposed method and a figure that visualy discribes the proposed technique.\n2. Figures and tables should be closer to the text descriptions e.g. figure 1 is on a different page than the description, same is true of table 3.\n3. Table 1 is not described in text\n4. For algorithms, please provide detailed Figures rather than text descriptions.", "questions_for_authors": "1. The proposed method is claimed to do a better job of learning the patterns in the time series. Will not the use of synthetic data will demonstrate better the effectiveness of the method?\n2. What are the specific reasons that in your experiments, for some datasets, other sota methods perform better than the proposed method? Can you please provide detailed reasoning that related to characteristics of these datasets?\n3. How will the proposed approach perform on other domains e.g. Stock Market?\n4. How are the graph transformer methods, e.g. STGNN, compare with the diffusion methods?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "NA"}}, {"review_id": "UZHyjIoY58", "reviewer": "Reviewer_g2ux", "summary": "This paper introduces CN-Diff, a novel conditional diffusion model tailored for time-series forecasting. The core idea is to integrate a nonlinear data transformation and a learnable condition within the forward process of diffusion, as opposed to more conventional diffusion-based approaches that use only a fixed (typically Gaussian) forward process. The authors argue that a trainable forward process can better align the prior with the data manifold, thereby enhancing predictive performance. The paper includes a comprehensive derivation of the model’s variational training objective and provides empirical results on nine real-world datasets, demonstrating superior or on-par performance compared to state-of-the-art time-series forecasting models.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **Other Strengths And Weaknesses:**  \n\n**Q1: It would be better if the authors can also compared the computational and memory complexity of the CN-Diff with other methods.**  \n**A1:**  We now present the computational cost analysis to compare our model with other diffusion-based time series forecasting methods. **(Please refer to the response of Reviewer 9M88 Q1 for training and inference comparison tables)**. Note that, SSSD utilizes structural state-space-based diffusion layers along with multiple dense layers; while TimeGrad leverages RNN hidden states; TimeDiff incorporates future mixup and autoregressive initialization; and mr-Diff employs multiple diffusion models. In contrast, our innovation lies in the diffusion formulation rather than in the architectural modifications. Both key components we introduced, the non-linear transform ($T_\\phi(.)$) and the condition network, consist of simple linear layers with an activation function. This design choice allows us to achieve greater computational efficiency compared to existing diffusion models for time series forecasting.\n\n**Q2: The nonlinear transformation $T_\\phi(.)$ is a central innovation, yet the paper provides little insight into what it learns**\n\n**A2:** That's an interesting question. We offer empirical evidence through the ablation results presented in the article (Table 5 and 10) that CN-Diff performs better than the ablation run without $T_\\phi$. Furthermore, we also note that the ablation with nonlinear transformation only along the feature dimension leads to a slightly larger improvement over the baseline without $T_\\phi$ than the ablation with nonlinear transformation only along the forecast window (Table 5 in the paper).  \n\nTo understand what is learned by $T_\\phi$, we take a trained model and explore the correlation between the features in the learned latent representation space of $T_\\phi(\\textbf{x}, t)$ at different diffusion model timesteps. We observed that there are increasing correlations between the features in learned latent space at different timesteps as shown in tables below. Thus, we hypothesize that this increased correlation and time-dependent adaptability perhaps facilitate a more effective diffusion process for time series forecasting. Similar observations of correlation in latent space that help diffusion models have been made in image and video diffusion works [1].  \n\nIn the space of diffusion models for time series, paper [2] has noted that learning the correlation between feature and temporal space is necessary for time series imputation tasks. We will add these to the discussion in the ablation study section in the revision.  \n\n**Table 3:**  Feature Correlation Matrix for Actual input (ETTh1 dataset)  \n\n| Features | 0      | 1      | 2      | 3      | 4      | 5      | 6      |\n|----------|--------|--------|--------|--------|--------|--------|--------|\n| 0        | **1.00**  | 0.08   | **0.99**  | 0.07   | 0.22   | -0.20  | -0.19  |\n| 1        | 0.08   | **1.00**  | 0.08   | **0.94**  | 0.05   | 0.22   | -0.06  |\n| 2        | **0.99**  | 0.08   | **1.00**  | 0.09   | 0.12   | -0.27  | -0.19  |\n| 3        | 0.07   | **0.94**  | 0.09   | **1.00**  | -0.10  | 0.06   | -0.07  |\n| 4        | 0.22   | 0.05   | 0.12   | -0.10  | **1.00**  | **0.56**  | -0.06  |\n| 5        | -0.20  | 0.22   | -0.27  | 0.06   | **0.56**  | **1.00**  | 0.11   |\n| 6        | -0.19  | -0.06  | -0.19  | -0.07  | -0.06  | 0.11   | **1.00**  |\n\n\n**Table 4:** Feature Correlation Matrix for Transformed input $T_\\phi(.)$ (ETTh1 dataset)  \n\n| Features | 0      | 1      | 2      | 3      | 4      | 5      | 6      |\n|----------|--------|--------|--------|--------|--------|--------|--------|\n| 0        | **1.00**  | **-0.52** | **-0.60** | **0.99**  | **0.93**  | **-0.96** | **-0.56** |\n| 1        | **-0.52** | **1.00**  | -0.31  | **-0.58** | **-0.68** | **0.60**  | **0.63**  |\n| 2        | **-0.60** | -0.31  | **1.00**  | **-0.57** | -0.33  | **0.55**  | -0.08  |\n| 3        | **0.99**  | **-0.58** | **-0.57** | **1.00**  | **0.94**  | **-0.97** | **-0.59** |\n| 4        | **0.93**  | **-0.68** | -0.33  | **0.94**  | **1.00**  | **-0.86** | **-0.80** |\n| 5        | **-0.96** | **0.60**  | **0.55**  | **-0.97** | **-0.86** | **1.00**  | 0.43   |\n| 6        | **-0.56** | **0.63**  | -0.08  | **-0.59** | **-0.80** | 0.43   | **1.00**  |\n\n\n**References:**  \n\n- [1] Ge, S., Nah, S., Liu, G., Poon, T., Tao, A., Catanzaro, B., ... and Balaji, Y. (2023). Preserve your own correlation: A noise prior for video diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 22930-22941).\n- [2] Tashiro, Y., Song, J., Song, Y., and Ermon, S. (2021). Csdi: Conditional score-based diffusion models for probabilistic time series imputation. Advances in neural information processing systems, 34, 24804-24816.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Correct.", "experimental_designs_or_analyses": "Sound.", "relation_to_broader_scientific_literature": ".", "essential_references_not_discussed": ".", "other_strengths_and_weaknesses": "**Strength**:\n* The paper proposed a nonlinear transformation for the forward process of the diffusion model to enhance its representation power when learning the conditional time-series forecast. Based on this proposal, the paper derived rigorously a clear and easy training objective and inference procedures.\n* Experiments on nine diverse benchmark datasets (e.g., Electricity, Traffic, Wind, Caiso, NorPool, ETT, Exchange) cover multiple domains (energy systems, roads, weather). CN-Diff outperforms or is competitive with a variety of strong baselines, including both deep learning architectures (e.g., N-Linear, N-BEATS, various Transformers) and recent diffusion-based models (TimeDiff, mr-Diff).\n\n**Weakness**:\n* It would be better if the authors can also compared the computational and memory complexity of the CN-Diff with other methods.\n* The nonlinear transformation $ T_\\varphi $ is a central innovation, yet the paper provides little insight into what it learns.", "other_comments_or_suggestions": ".", "questions_for_authors": ".", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ndovw8QCwD", "reviewer": "Reviewer_4u7H", "summary": "This paper proposes an approach for time series forecasting using conditional diffusion models. The approach utilizes a learnable forward process, making the transition operator and ending points of the forward process learnable and controlled by the conditioning observations. It derives a non-Markovian reverse process based on the learnable forward process and a training objective following the frameworks of DDPM. Experiment results on standard time-series benchmark datasets show the superior performance of the model against the latest baseline methods.\n\n## update after rebuttal\nI'm satisfied with the response by the reviewer on making improvements on the presentations and terminologies. I appreciate the efforts of the reviewer by providing related works on learnable forward processes. I have no more serious issues with the work and would recommend the paper be accepted.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **Claims and Evidence:**\n**Q1: In the first Paragraph  $\\cdots$ . I would  $\\cdots$ straight reverses processes.**\n\n**A1:**\n\n**In the first $\\cdots$ untrainable.**\n\nOur claim follows the well-known observation that incorporating a flexible forward process can enhance performance and address the limitation of fixed and untrainable forward processes in existing diffusion models [e.g., 1,2,3]\n\n- [1] A flexible diffusion model. ICML, 2023.\n- [2] Soft diffusion: Score matching with general corruptions. TMLR, 2023.\n- [3] Maximum likelihood training of implicit nonlinear diffusion model. NeurIPS, 2022.\n\n**I would like $\\cdots$ processes.**\n\nTo address the above limitation, we propose a nonlinear, time-dependent data transformation combined with a learnable conditioning mechanism in the forward process for time series forecasting. This learnable condition that we have introduced in the forward process results in the final distribution converging to $N(c,I)$. In case of any misunderstanding due to the typo (see below), we apologize.\n\n**Q2: The author claims $\\cdots$ main paper.**\n\n**A2:** Thank you. We also considered this before submission. We agree with the suggestion of moving it to the main paper and shall do so.\n### **Theoretical Claims:**\n**Q3: The work requires $\\cdots$ of Equation 3**\n\n**A3:** Sorry for the typo and thank you for pointing it out. We sincerely appreciate your careful review.  Equation 3 should be correctly written as $p(\\textbf{x}^{T}) = \\mathcal{N}(\\textbf{x}^{T};c,I)$. The text in Algorithm 2 and line 132 (below Equation 1) are correct and consistent with our formulation. We carefully examined the manuscript and ensured that all remaining typos were corrected.\n\n### **Other Strengths and Weaknesses:**\n **Q4: The consistency $\\cdots$ time series data.**\n\n**A4:** Thank you for your comment. We will revise Equations 2 and 3 by replacing $x^{0:T}$ with $\\textbf{x}^{0:T}$  and $p_{\\theta}(x_{0:T})$ to $p_{\\theta}(\\textbf{x}^{0:T})$ for accurately reflect our notation. We believe that these changes will ensure clarity throughout the manuscript. \n\n**Q5: The presentation $\\cdots$ of Page 4.**\n\n**A5:** We will carefully address the presentation and layout improvements in the paper, including optimizing the whitespace on pages 4 and 6. \n\n**Q6: Some existing $\\cdots$ Not Discussed.**\n\n**A6:** Thanks for referring to these papers. We acknowledge the relevance of these works and clarify their distinctions from our approach. \n\n**Polydiffuse** uses a guided diffusion model for polygonal shape reconstruction, where the prior is learned via guidance networks $\\mu(x,t,i)$ and $\\sigma(x,t,i)$, which are independent of the condition. Also they have not incorporated the condition in the forward process (ref. Fig. 3 [Polydiffuse]). In contrast, our approach incorporates a condition in forward formulation results in condition-dependent prior. And our training procedure and loss function differs significantly, as polydiffuse uses separate stages for prior learning and denoising\n\n**PriorGrad** is diffusion-based generative model for speech synthesis, which introduces a prior distribution based on data statistics rather than a learned prior. While their reverse process starts by sampling from $\\mathcal{N}(0,\\Sigma)$, our method samples from $\\mathcal{N}(c,I)$ as we include a learnable condition in the forward process. \n\nThese fundamental differences set our work apart from the above. \n\n**Q7: Given the existing $\\cdots$ motivations.** \n\n**A7:** CN-Diff introduces a learnable condition in the forward process which depends on history data of the time series for forecasting. This leads to the convergence of the ending distribution $\\textbf{x}^T$ to $\\mathcal{N}(c,I)$ in the forward process. In the reverse process, this same distribution is used as prior to start sampling. Thus, CN-Diff completely differs from previous work including the provided references of polydiffuse and priorgrad.\n\nApart from this, we also introduced a novel nonlinear, time-dependent data transformation $ T_\\phi(\\textbf{x}, t)$ in the forward process for time series forecasting which reduces the gap between log-likehood and variational approximation.  \n\nIncorporating these concepts leads to a new training objective that we derived. The experimental results in Tables (4, 5, and 10) reveal notable improvements in time series forecasting due to our innovation.\n\nWe therefore humbly request the reviewer to reconsider their verdict. \n### **Questions For Authors:**\n**Q8: Can we use $\\cdots$ window lengths?**\n\n**A8:** We have trained the model for specific prediction lengths. However, an autoregressive approach during inference allows it to adapt to different prediction window lengths without any retraining. \n\n**Q9: Why is $T_\\phi(.) $ not conditioned on $c$?**\n\n**A9:** The proposal of further conditioning $T_\\phi(.)$ on $c$ is a worthwhile consideration that is already in our future work deliberations. We appreciate your insightful contribution.", "claims_and_evidence": "There are a few claims that requires further clarifications from the author:\n\n1. In the first Paragraph of Sec.3, the author claims that one issue of existing diffusion models is that their forward processes are fixed and untrainable. I would like to see the author providing further clarifications or evidence on why this could be a problem given that the ending distribution of the forward process is controlled by condition c and already learnable. This claim is also in contradiction with many modern diffusion models like rectified flow [1] or flow matching [2] which prefers simple and straight reverses processes.\n\n2. The author claims that the forward process defined by Eq.7 is non-Markov. This non-Markovian property of the forward process brings the framework of this work closer to DDIM[3] instead of DDPM. The author only mentioned the similarity between DDIM[3] and DDPM in Appendix A.2. I still think the author should state that in the main paper.\n\n[1] Liu, Xingchao, Chengyue Gong, and Qiang Liu. \"Flow straight and fast: Learning to generate and transfer data with rectified flow.\" arXiv preprint arXiv:2209.03003 (2022).\n\n[2] Lipman, Yaron, et al. \"Flow matching for generative modeling.\" arXiv preprint arXiv:2210.02747 (2022).\n\n[3] Song, Jiaming, Chenlin Meng, and Stefano Ermon. \"Denoising diffusion implicit models.\" arXiv preprint arXiv:2010.02502 (2020).", "methods_and_evaluation_criteria": "The works are evaluated using standard time-series benchmark datasets and settings. They are fit for the task of time-series forecasting, which the proposed CN-Diffuse method is trying to solve.", "theoretical_claims": "The work requires more clarification on the theoretical correctness of the proposed method. The forward process in the proposed work is conditioned on $c$, resulting in an ending distribution of $X^T$ parameterized by $c$. For the reverse process, Equation 3 states that $X^T$ follows an isotropic Gaussian distribution, but Algorithm 2 states that  $X^T$ follows a Gaussian distribution parameterized by $c$. This is an inconsistency. I'm not fully convinced that the proposed method is correct in the first case of Equation 3.", "experimental_designs_or_analyses": "The experiment design and analysis are thorough and valid. The experiments make comparisons against the latest baselines, and the ablation study results also thoroughly studied different ablated versions of the model. Experiment design and analysis are a strength of the work.", "relation_to_broader_scientific_literature": "The work is built on top of existing diffusion models like DDPM[1] and DDIM[2]. It is also closely related to existing conditional diffusion model works[3, 4] that parameterize the initial distribution of the reverse process using some conditions.\n\n[1] Ho, Jonathan, Ajay Jain, and Pieter Abbeel. \"Denoising diffusion probabilistic models.\" Advances in neural information processing systems 33 (2020): 6840-6851.\n\n[2] Song, Jiaming, Chenlin Meng, and Stefano Ermon. \"Denoising diffusion implicit models.\" arXiv preprint arXiv:2010.02502 (2020).\n\n[3] Lee, Sang-gil, et al. \"Priorgrad: Improving conditional denoising diffusion models with data-dependent adaptive prior.\" arXiv preprint arXiv:2106.06406 (2021).\n\n[4] Chen, Jiacheng, Ruizhi Deng, and Yasutaka Furukawa. \"Polydiffuse: Polygonal shape reconstruction via guided set diffusion models.\" Advances in Neural Information Processing Systems 36 (2023): 1863-1888.", "essential_references_not_discussed": "There are a few works on conditional diffusion model with a learnable starting distribution of the reverse process. They are closely related to the proposed approach in terms of methodology and should be cited:\n\n[1] Lee, Sang-gil, et al. \"Priorgrad: Improving conditional denoising diffusion models with data-dependent adaptive prior.\" arXiv preprint arXiv:2106.06406 (2021).\n\n[2] Chen, Jiacheng, Ruizhi Deng, and Yasutaka Furukawa. \"Polydiffuse: Polygonal shape reconstruction via guided set diffusion models.\" Advances in Neural Information Processing Systems 36 (2023): 1863-1888.", "other_strengths_and_weaknesses": "Other weaknesses:\n\n1. The consistency of notations needs significant improvements. For example, in $\\mathbf{x}^{0:T}$ and $x^{0:T}$ were used interchangeably in the Section 3. In the line above Equation 2, and $x_{0:T}$ is used in place of $x^{0:T}$ while $x_{0:H}$ uses subscript to denote time steps in time series data.\n\n2. The presentation and layout of the work could be further improved. For example there are plenty of white space in the left column of Page 4.\n\n3. Some existing but related works are missing in the citations. Please see **Essential References Not Discussed**.\n\n4. Given the existing works on conditional diffusion model and the first point of my concern in **Claims And Evidence**, I think the contributions of the work lack both originality and principled motivations.", "other_comments_or_suggestions": "Please see the section of **Other Strength and Weakness** and **Questions for Authors**.", "questions_for_authors": "1. Can we use the same model to make predictions for different prediction window lengths or we need to train different models for different prediction window lengths?\n\n2. Why is $T_{\\phi}(x, t)$ not conditioned on c?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HooQC8qwKT", "reviewer": "Reviewer_9M88", "summary": "The paper introduces a Conditional Diffusion Model (CDM) for generative modeling, leveraging denoising diffusion probabilistic models (DDPMs) to generate high-quality samples conditioned on specific inputs. The key contribution is a conditioning mechanism that guides the diffusion process, allowing for controlled generation tailored to input constraints. The proposed approach improves upon existing diffusion models by enhancing sample quality, diversity, and control over generated outputs. Empirical evaluations across multiple benchmarks demonstrate state-of-the-art performance, outperforming prior conditional generative models. Theoretical insights and ablation studies further validate the effectiveness of the conditioning mechanism in guiding diffusion-based generation.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **Methods And Evaluation Criteria:**\n\n**Q1: But the paper lacks efficiency comparisons to assess computational cost** \n\n**A1:** We now present the computational cost analysis to compare our model with other diffusion-based time series forecasting methods. Refer to the tables below for training and inference comparisons. Note that, SSSD utilizes structural state-space-based diffusion layers along with multiple dense layers; while TimeGrad leverages RNN hidden states; TimeDiff incorporates future mixup and autoregressive initialization; and mr-Diff employs multiple diffusion models. In contrast, our innovation lies in the diffusion formulation rather than in the architectural modifications. Both key components we introduced, the nonlinear transform ($T_\\phi(.)$) and the condition network, consist of simple linear layers with an activation function. This design choice allows us to achieve greater computational efficiency compared to existing diffusion models for time series forecasting, and this can be realized by the table below.\n\n**Table 1:** Training time (ms) for different models and sequence lengths ($H$) for ETTh1 univariate.\n\n| | H=96 | H=168 | H=192 | H=336 | H=720 |\n|------------|-------|--------|--------|--------|--------|\n| **CN-Diff** | **0.21**  | **0.26**   | **0.27**   | **0.31**   | **0.39**   |\n| mr-Diff    | 0.59  | 0.69   | 0.71   | 0.74   | 0.82   |\n| TimeDiff   | 0.71  | 0.75   | 0.77   | 0.82   | 0.85   |\n| TimeGrad   | 2.11  | 2.42   | 3.21   | 4.22   | 5.93   |\n| CSDI       | 5.72  | 7.09   | 7.59   | 10.59  | 17.21  |\n| SSSD       | 16.98 | 19.34  | 22.64  | 32.12  | 52.93  |\n\n\n**Table 2**: Inference time (ms) for different models and sequence lengths ($H$) for ETTh1 univariate.\n\n| Model     | Trainable Params | H=96  | H=168  | H=192  | H=336  | H=720  |\n|-----------|----------------|-------|--------|--------|--------|--------|\n| **CN-diff** | **1.1M**  |**6.2**   |**6.7**   | **6.9**   |**7.8**   |**9.1**   |\n| mr-Diff | 1.4M           | 12.5  | 14.3   | 14.9   | 16.8   | 27.5   |\n| TimeDiff | 1.7M          | 16.2  | 17.3   | 17.6   | 26.5   | 34.6   |\n| TimeGrad | 3.1M          | 870.2 | 1620.9 | 1854.5 | 3119.7 | 6724.1 |\n| CSDI    | 10M           | 90.4  | 128.3  | 142.8  | 398.9  | 513.1  |\n| SSSD    | 32M           | 418.6 | 590.2  | 645.4  | 1054.2 | 2516.9 |\n\n\n**Q2: Additionally, while state-of-the-art baselines are included, a broader range of conditional generative models (e.g., GANs, VAEs) could provide a more comprehensive evaluation**\n\n**A2:** We have already benchmarked our results with variants of GANs **(PSA-GAN)** and VAEs **(D³ VAE)** as shown in the tables in the Experimental section.  Additionally, we are also benchmarking our CN-Diff against Graph transformer methods. Please refer to the response to reviewer 1 for Graph transformer results. \n\n### **Essential References:**\n**Q3: A recent ICLR paper can be discussed**\n\n**A3:** This work focuses on introducing a novel formulation for time series forecasting rather than optimizing computational time. The papers suggested focus on reducing sampling time in image diffusion models. From recent work in time series diffusion [1, 2, 3], timesteps for the diffusion is around 100 which is very less compared to images. But even this might cause a longer sampling time in long sequences. So, we acknowledge its importance and will consider it for future work, and will include the suggested references in our paper. \n\n**References:**\n\n- [1] Shen, L. and Kwok, J. Non-autoregressive conditional diffusion models for time series prediction. In International Conference on Machine Learning, pp. 31016– 31029. PMLR, 2023.  \n\n- [2] Tashiro, Y., Song, J., Song, Y., and Ermon, S. Csdi: Conditional score-based diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34:24804–24816, 2021. \n\n- [3] Rasul, K., Seward, C., Schuster, I., and Vollgraf, R. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In International Conference on Machine Learning, pp. 8857–8868. PMLR, 2021.", "claims_and_evidence": "The paper’s claims are generally well-supported by empirical results and theoretical insights. The authors claim that their Conditional Diffusion Model (CDM) improves sample quality, diversity, and controllability, which is backed by quantitative evaluations on benchmark datasets. The inclusion of state-of-the-art comparisons strengthens the claim that CDM outperforms prior methods. However, the paper does not clearly discuss the computational cost of CDM compared to existing models, which is crucial since diffusion models are computationally intensive.", "methods_and_evaluation_criteria": "The proposed Conditional Diffusion Model (CDM) is well-aligned with the controlled generative modeling task, and the evaluation on standard benchmark datasets is appropriate. The use of sample quality metrics (e.g., FID, Inception Score) makes sense, but the paper lacks efficiency comparisons to assess computational cost. Additionally, while state-of-the-art baselines are included, a broader range of conditional generative models (e.g., GANs, VAEs) could provide a more comprehensive evaluation. Including robustness tests for different conditioning strategies would further strengthen the experimental design.", "theoretical_claims": "This is not a theoretical-oriented paper. No comments here.", "experimental_designs_or_analyses": "The experimental design is solid, with comparisons against state-of-the-art methods, quantitative evaluations using standard metrics (FID, IS), and ablation studies. However, the paper lacks efficiency analysis, making it unclear how computational cost compares to existing diffusion models.", "relation_to_broader_scientific_literature": "The contribution of this paper strengthens the application of diffusion models in time-series forecasting. It is highly recommended if the training and generating complexity can be well handled.", "essential_references_not_discussed": "A recent ICLR paper can be discussed https://openreview.net/forum?id=OlzB6LnXcS", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a new diffusion-model based approach for time-series forecasting. The idea is to allow a learnable forward diffusion process, where the mean of the Gaussian forward distribution is a time-dependent learnable function determined by the clean data and the conditioning variable. This learnable forward drift is designed in a way such that in the backward pass the diffusion model can be trained by minimising the L2 regression error towards a target with analytic form (dependent on the forward drift). Extensive experiments show the proposed approach in most datasets perform the best or second-best when compared with a lot of time-series forecasting baselines.

Reviewers found the method to be sound and convincing, although they asked a few questions regarding various aspects of the experiments as well as related work regarding learnable forward process. Overall the author rebuttal has addressed most of the reviewers' concerns.

I briefly read the paper and I found the proposed design of forward diffusion process very interesting in that it still admits an analytic regression target for the denoising network. Given that in the broader literature of diffusion models learnable forward processes have also been tried out, it would be more convincing if further justification can be provided regarding this specific forward diffusion process design as compared with free-form learnable forward drift.
Decision: Accept (poster)