Multi-Resolution Diffusion Models for Time Series Forecasting
Authors: Lifeng Shen
Weiyu Chen
James Kwok
Venue: ICLR 2024 poster (2024)
Keywords: diffusion model
time series
multiscale
Primary area: generative models

Abstract: The diffusion model has been successfully used in many computer vision applications, such as text-guided image generation and image-to-image translation. Recently, there have been attempts on extending the diffusion model for time series data. However, these extensions are fairly straightforward and do not utilize the unique properties of time series data. As different patterns are usually exhibited at multiple scales of a time series, we in this paper leverage this multi-resolution temporal structure and propose the multi-resolution diffusion model (mr-Diff). By using the seasonal-trend decomposition, we sequentially extract fine-to-coarse trends from the time series for forward diffusion. The denoising process then proceeds in an easy-to-hard non-autoregressive manner. The coarsest trend is generated first. Finer details are progressively added, using the predicted coarser trends as condition variables. Experimental results on nine real-world time series datasets demonstrate that mr-Diff outperforms state-of-the-art time series diffusion models. It is also better than or comparable across a wide variety of advanced time series prediction models.

Published as a conference paper at ICLR 2024
MULTI-RESOLUTION DIFFUSION MODELS FOR TIME SERIES FORECASTING
Lifeng Shen' Weiyu James T. Kwok
1 Division of Emerging Interdisciplinary Areas, Hong Kong University of Science and Technology
2 Department of Computer Science and Engineering, Hong Kong University of Science and Technology jameskecse. ust.hk
1 Ashenaecconnect..... ust.hk, ochenbx@cse.ust.
ABSTRACT
The diffusion model has been successfully used in many computer vision appli- cations, such as text-guided image generation and image-to-image translation. Recently, there have been attempts on extending the diffusion model for time series data. However, these extensions are fairly straightforward and do not utilize the unique properties of time series data. As different patterns are usually exhibited at multiple scales of a time series, we in this paper leverage this multi-resolution temporal structure and propose the multi-resolution diffusion model (mr-Diff), By using the seasonal- -trend decomposition. we sequentially extract fine-to-coare trends from the time series for forward diffusion. The denoising process then proceeds in an easy-to-hard non-autoregressiive manner. The coarsest trend is generated first. Finer details are progressively added, using the predicted coarser trends as condition variables. Experimental results nine real-world time series datasets demonstrate that mr-Di ff outperforms state-of tart time series diffu- sion models. It is also better than comparable across wide variety of advanced time series prediction models.
INTRODUCTION
Time series data are prevalent in many real-world applications. particular, time series forecasting facilitates users to identify patterns and make predictions based on historical data. Examples include stock price prediction in finance, patient health monitoring healthcare, machine monitoring man- ufacturing, and traffic flow optimization in transportation. Over the years, significant advancements in time series analysis have been made through the development of various deep neural networks, including recurrent neural networks (Hewamalage et al., 2021), convolutional neural networks (Yue et al., 2022), and transformers (Vaswani et al., 2017).
Besides these prominent deep neural networks, the diffusion model has recently emerged as strong generative modeling tool. It has outperformed many other generative models in areas such as image synthesis (Ho et al., 2020; Dhariwal & Nichol, 2021), video generation (Harvey al., 2022; Blattmann et al., 2023), and multi-modal applications (Rombach et al., 2022; Saharia et al., 2022). Very recently, researchers have sought leverage its strong generative capacity in the time-series domain. A number of time-series diffuusin models have been developed (Rasul et al., 2021; Tashiro al., 2021; Alcaraz & Strodthoff, 2022: Shen & Kwok, 2023). For example, TimeGrad (Rasul et al., 2021) integrates the standard diffusion model with recurrent neural network's 's hidden states. CSDI (Tashiro al., 2021) uses self ceff-supervised masking to guide an con-autoregressvve denoising process. While these time series diffusion models have demonstrated their efficacy, they do not fully utilize the unique structural properties in time series data and are still constrained to generate the time series directly from random vectors (Figure 1(a)). This pose significant challenge when working with real-world time series that are non-stationary and noisy.
As time series usually exhibit complex patterns over multiple scales, using the underlying multi- resolution temporal structure has been a cornerstone traditional time series analysis. In particular, the seasonal-tren decomposition (Robert et al., 1990) can extract the seasonal and trend components, and the coarser temporal patterns can be used to help modeling the finer patterns. Recently, some deep time series prediction models (Oreshkin et al., 2019; Wu et al., 2021: Zeng al., 2023) have also

---

Published as a conference paper at ICLR 2024
incorporated multi-resolution analysis techniques. For example, NBeats (Oreshkin et al., 2019) uses the Fourier and polynomial basis to approximate the seasonal and trend components in multiple layers, respectively. Autoformer (Wu et al., 2021) uses average pooling extract seasonal components in various transformer layers. DLinear (Zeng et al., 2023) introduces an MIP with seasonal-trent branches. N-Hits (Challu al., 2023) uses hierarchical interpolation better leverage the multiscale temporal patterns. Fedformer (Zhou et al.. 2022b) improves Autoformer by using mixture-of-expee decomposition in the frequency domain. While these recent transformer and MLP models demonstrate the effectiveness of resoutonn analysis in deep time series modeling, the use of analysis in time series diffusion models has yet to be explored.
In this paper, we bridge this gap by propos- ing the multi-resolution diffusion (mr-Di ff) model for time series forecasting. Unlike exist- ing time series diffusion models that directly denoises from random vectors (Figure I(a)), mr mr--ii ff decomposes the denoising objective into several sub-objectives (Figure 1(b)), each of which corresponds to a trend extracted from sequence of fine-to-coarse seasonal-trent de- compositions. This encourages the denoising process proceed in an easy-to-hard manner. The coarser trends are generated first and the finer details are then progressively added. By better exploiting the seasonal-trent structure and different temporal resolutions, this leads more accurate generation of the time series.
M.NNWe MMl
random noises
time series
(a) Standard diffission with direct denoising.
(b) Proposed diffusion model with multi-resolution.
Figure 1: Direct denoising versus proposed multi- resolution denoising.
The contributions of this paper can be sum- marized as follows: (i) We propose the multi- resolution diffusion (mr-Diff) model, which is the first to integrate the seasonal-trend decomposition- based multi-resolution analysis into time series diffusion models. (ii) We perform progressive denoising in an easy-to-hard manner, generating coarser signals first and then finer details. This allows more accurate prediction of time series. (iii) Extensive experiments show that mr-Diff ff outperforms state-of-the-ar time series diffusion models. It is also better than or comparable across wide variety of advanced time series prediction models.
RELATED WORKS: DEEP TIME SERIES MODELS
Recently, a number of deep time series models have been proposed that use the transformer (Vaswani et al., 2017) to capture temporal dependencies. Informer (Zhou et al., 2021) improves the vanilla transformer by avoiding its quadratic time complexity with sparse attention, and improves the infer- ence speed by decoding in non-autoregressive manner. Autoformer (Wu et al., 2021) replaces the transformer's self-attention block with an auto-correlation layer. Fedformer (Zhou et al., 2022b) uses frequency-enhancee module to capture important temporal structures by frequency domain mapping. Pyraformer (Liu et al., 2021) uses pyramidal attention module for multi-resolution representation of the time series. Scaleformer (Shabani al., 2023) generates the forecast progressively, starting from the coarser level and then progressing to the finer levels. PatchTST (Nie et al., 2022) is similar to the vision transformer (ViT) (Dosovitskiy et al., 2020), and performs time series prediction by patching the time series and self-supervised pre-training to extract local semantic information. It also replaces the transformer's 's decoder with a linear mapping and uses indeeededdee strategy to achieve good performance in multivariate time series prediction.
Besides the transformer-based models, some recent models leverage basis expansion to decompose the time series. FiLM (Zhou et al., 2022a) uses Legendre Polynomials projections to approximate historical information and Fourier projections to remove noise. NBeats (Oreshkin et al., 2019) represents the trends in the time series by polynomial coefficients and the seasonal patterns by Fourier coefficients. Depts (Fan et al., 2022) improves NBeats by using a periodicity module to model periodic time series. N-Hits (Challu al., 2023) uses multi-scale hierarchical interpolations to further
2

---

Published as a conference paper at ICLR 2024
improve NBeats. In general, these models are easier to train than transformer- maasddrreer-- models, though their performance can vary depending on the choice of the basis.
Besides these two types of deep learning models, other recent models are also very competitive. SCINet (Liu et al., 2022) uses a recursive downsample-convolv--ine temporal features from the downsampled sub-sequences or features. NLinear (Zeng et al., 2023) normalizes the time series and uses a linear layer for prediction. DLinear (Zeng et al., 2023) follows the Autoformer and uses seasonal-trend decomposition.
architecture to extract
Very recently, diffusion models have also been developed for time series data. TimeGrad (Rasul et al., 2021) is conditional diffusion model which predicts in an autoregressive manner, with the denoising process guided by the hidden state of a recurrent neural network. However, it suffers from slow inference on long time series because of the use of autoregressive decoding. To alleviate this problem, CSDI (Tashiro et al., 2021) uses von-autoregressive generation, and uses self-supervised masking to guide the denoising process. However, it needs two transformers to capture dependencies in the channel and time dimensions. Moreover, its complexity is quadratic in the number of variables and length of time series as in other transformer models. Besides, masking-basee conditioning is similar to the task of image inpainting, and can cause disharmony at the boundaries between the masked and observed regions (Lugmayr et al., 2022; Shen & Kwok, 2023). SSSD (Alcaraz & Strodthoff, 2022) reduces the computational complexity of CSDI by replacing the transformers with structured state space model. However, it uses the same masking- baaseng--baedd conditioning as in CSDI, and thus still suffers from the problem of boundary disharmony. To alleviate this problem, the von-autoreeressive diffusion model TimeDiff (Shen & Kwok, 2023) uses future mixup and autoregressive initialization for conditioning. However, all these time series diffusion models do not leverage the multi-resolution temporal structures and denoise directly from random vectors as in standard diffusion models.
In this paper, we propose decompose the time series into multiple resolutions using seasonal- trend decomposition, and use the fine-to-coarse trends as intermediate latent variables to guide the denoising process. Recently, other multiresolution analysis techniques besides using seasonal- trend decomposition have also been used for time series modeling. For example, Yu et al. (2021) propose a U-Net (Ronneberger al., 2015) for graph-structured time series, and leverage temporal information from different resolutions by pooling and unpooling. Mu2ReST (Niu et al., 2022) works on spatio-temporal data and recursively outputs predictions from coarser to finer resolutions. Yformer (Madhusudhanan et al., 2021) captures temporal dependencies by combining downscaling/uussampiig with sparse attention. PSA-GAN (Jeha et al., 2022) trains a growing U-Net, and captures multi- resolution patterns by progressively adding trainable modules at different levels. However, all these methods need to design very specific U-Net structures.
3 BACKGROUND
3.1 DENOISING DIFFUSION PROBABILISTIC MODELS
A well-known diffusion model is the denoising diffusion probabilistic model (DDPM) (Ho et al., 2020). is a latent variable model with forward diffusion and backward denoising processes. During forward diffusion, an input x" is gradually corrupted to Gaussian noise vector. Specifically, at the kth step, x is generated by corrupting the previous iterate x (scaled by VI BB) with zero-mean Gaussian noise (with variance Be E [0, I)):
qx**|x* k --) Axxx; v1 3,x² ,3,I), k k= L..,,,, K.
It can be shown that this can also be rewritten as g(x* (x²) Nx(x;; / vāxx", rI) where og = 1Tk 0,, and og = Bk- Thus, x can be simply obtained as
Voxxa v1 age,
where is a noise from (O. This equation also allows x" to be easily recovered from *
In DDPM, backward denoising is defined as Markovian process. Specifically, at the kth denoising step, x k-1 is generated from x by sampling from the following normal distribution:
po x (x*))= -N(x* po(x ((xxt,)))))))),))
k)), Ea kk))
3

---

Published as a conference paper at ICLR 2024
Here, the variance Ex(x", k) is usually fixed as ofI. while the mean po(x*, k) is defined by neural network (parameterized by 8). This is usually formulated as a noise estimation or data prediction problem (Benny& & Wolf, 2022). For noise estimation, network eo predicts the noise of the diffused input x*, and then obtains no(x*, k) as Vak x
1-ac VT-orya <o(x", k). Parameter 0 is learned by
minimizing the loss L. = xx,x" [||e - <x(x*, ; k) [?]] Alternatively, the data prediction strategy uses a denoising network X9 to obtain an estimate X xx(x", ,)) of the clean data x" given x", and then set
,ø(x*, vax āk 1) āz Parameter 0 is learned by minimizing the loss
x vāk- 18
x6 (xx,
1 1 ai
Lx = ,x,,,,x/1<" xo (xx "ox,,,K ,:)11?
3.2 CONDITIONAL DIFFUSION MODELS FOR TIME SERIES PREDICTION
In time series forecasting, aims predict the future values x R given the past observations x" L++1:0 Rdx,. of the time series. Here, d is the number of variables. H is the length of the forecast window, and L is the length of the lookback window. When using conditional diffusion models for time series prediction, the following distribution is considered (Rasul al., 2021; Tashiro et al., 2021; ; Shen & Kwok, 2023)
K (x c) (x II po | xi.H, c), c F(x=L410)
(5)
where x observations x" 1+100 as input. Correspondingly, the denoising process at step & is given by
(N, I), the condition, and is a conditioning network that takes the past
po p I c) (x
: po(xi. of)) A K, 1,k.-,...3 1,...,l..
(6)
During inference, denote the generated sample corresponding to x by x We first initialize X11 as noise vector from (0, I). By repeatedly running the denoising step in (6) till k 1. the final generated sample is x x
MR-DIFF: RELIIIRESSOLTION DIFFUSION MODEL
As discussed in Section 1, recent transformer and MLP models demonstrate the effectiveness of seasonal-trend decomposition- multi-resolution analysis in deep time series modeling. However, the use of multi-resolution temporal patterns in the diffusion model has yet to be explored. In this paper, address this gap by proposing the multi-resolution diffusion (mr-Diff) model. An overview the proposed model is shown in Figure
The proposed mr mr-Diff can be viewed cascaded diffusion model (Ho al., 2022), and proceeds in S stages, with the resolution getting coarser as the stage proceeds (Section 4.1). This allows capturing the temporal dynamics multiple temporal resolutions. In each stage. the diffusion process interleaved with seasonal-tre -rrenn decomposition. For simplicity of notations, we use = X 1:00 and Y X1:H1 for the time series segments in the lookback and forecast windows, respectively. Let the trend component of the lookback (resp. forecast) segment at stage 8 + be X, (resp. Y.). The trend gets coarser as s increases, and with X,- X and Yo Y. In each stage s + 1, conditional diffusion model is learned to reconstruct the trend component Y. extracted from the forecast window (Section 4.2). The reconstruction at stage then corresponds to the target time series forecast.
While the forward diffusion process in this diffusion model is straightforward and similar to existing diffusion models, design of the denoising process, particularly on the denoising conditions and denoising network, are less trivial. During training, guide the reconstruction of Y., the proposed model takes the lookback segment X, (which has the same resolution as Y.) and the coarser trend Yo41 (which provides an overall picture of the finer Y.) as denoising condition. On inference, the ground-truth Y.-1 is not available, and is replaced by its estimate *+1 produced by the denoising process at stage s + 1. By combining the diffusion model and seasonal-trews decomposition a multi-resolution manner, the proposed model encourages better modeling of real-world time series.
4

---

Published as a conference paper at ICLR 2024
%
Training
lock window
Tiiadd
Trund
Trand
inference
Yi
Y
window
M
Extraction
Treed
Hol
Extraction
Freid
Fandin oring
consitioning
Centi Tooning
Final prediction
natword
network
retwerk
Yi
affisee
diftuiion
difici
Stage
Stege 2
Stage:
Figure 2: The proposed multi-resolltion diffusion model mr -irffff. For simplicity illustration, we use S = 3 stages. At stage + 1, Y, denotes the corresponding trend component extracted from the segment in the forecast window, Y is the diffusion sample at diffusion step k, and Y! is the denoised output.
4.1 EXTRACTING FINE-TO-COARS: TRENDS
For the given time series segment Xg in the lookback window, its trend components successively extracted by the TrendExt wacttxttraatiii ion module as:
X, Avgpool(Pacding((X-
L
where AvgPool is the average pooling operation (Wu et al., 2021), Padding keeps the lengths of X.-1 and X, the same, and Tg is the smoothing kernel size which increases with s so as to generate fine-to-coarse trends. Processing for the segment Yo in the forecast window is analogous, and its trend components [Y+)= are extracted.
Note that while the seasonal-trend decomposition obtains both the seasonal and trend components, the focus here is on the trend. On the other hand, models such as the Autoformer (Wu et al., 2021) and Fedformer (Zhou al., 2022b) focus progressively decomposing the seasonal component. As we use the diffusion model for time series reconstruction at various stages/resolutions (Section 4.2), intuitively, easier to predict finer trend from coarser trend. On the other hand, reconstruction of finer seasonal component from a coarser seasonal component may be difficult, especially as the seasonal component may not present clear patterns.
4.2 TEMPORAL MULTI-RESOLLUTION RECONSTRUCTION
In each stage + 1, we use conditional diffusion model to reconstruct the future trend Y. extracted in Section 4.1. As in the standard diffusion model, it consists of forward diffusion process and backward denoising process. Forward diffusion does not involve learnable parameters, while the training procedure of the backward denoising process needs optimization shown in Algorithm 1.
An d' dimeesiional embedding P of the diffusion step & is used in both the forward and backward denoising processes. As in (Rasul al., 2021; Tashiro al., 2021; Kong al., 2020), this is obtained by first taking the sinusoidal position embedding (Vaswani et al., 2017): Kanbadding = d'
0x4 (10 --1 i).... sin (10 w- I), cos 10 -000-1/)
wxa
6x4
cos(10==1)) ()] ()...)..s....... 2x4
where w z and then passing
through two cally-comnected (FC) layers to obtain
p SSIUUPCCSSTAUPPCC/aaadddda
(7)
where SiLU is the weighted linear unit (Elfwing et al., 2017). By default, d' is set to 128.
5

---

Published as a conference paper at ICLR 2024
X,
Canditiaing
Trolnin inference
lookbeck window
Prudfilyy
Thaur meup
MY
Heapy
randon n016
foreccat window
futrre whup
coerner future trand
stage
Sing networks output
Network
Traiiing ebjective
Docore
L Yh - No Rl
Denosinn
V Vo-iAA
Y,
Forward van
Figure 3: The conditioning network and denoising network.
4.2.1
FORWARD DIFFUSION
Forward diffusion is straightforward. Analogous to (I), with Y Y.. we obtain at step k; az Y + v1 - cgge k = 1,... K,
(8)
where the noise matrix is sampled from (O. I) with the same size as Y.
4.2.2 BACKWARD DENOISING
Standard diffusion models perform one-stage denoising directly from random vectors. this section, we decompose the denoising objective of mr Difff into S sub-objectives, one for each stage. As will be seen, this encourages the denoising process to proceed in an easy-to-hard manner, such that the coarser trends are generated first and the finer details are then progressively added.
Conditioning network. The conditioning network (Figure 3, top) constructs condition to guide the denoising network (to be discussed). Existing time series diffusion models (Rasul al., 2021; Tashiro et al., 2021) simply use the original time series' lookback segment X, condition c in (5). With the proposed multi-resolution seasonal-trend decomposition, we instead use the lookback segment X, at the same decomposition stage This allows better and easier reconstruction, as X, has the same resolution as Y, to be reconstructed. On the other hand, when X is used as in existing time series diffusion models, the denoising network may overfit temporal details at the finer level.
linear mapping is applied on X, to produce a tensor Zhistory dd For better denoising performance, during training combines Eaistory and the (ground-truth) future observation Y with mixup (Zhang et al., 2018):
future-mixup (Shen & Kwok, 2023) to enhance Zhistory-
Zaix m Zaistory + (1 m) Y
where denotes the Hadamard product, and element is randomly sampled from the uniform distribution on [0, 1). Future-mixup is similar to teacher forcing (Williams & Zipser, 1989), which mixes the ground truth with previous prediction output at each decoding step of autoregressive decoding.
[0, 1) is mixing matrix in which each
Besides using X, on training, the coarser trend Y (= Y can provide an overall picture of the finer trend Y, and is thus also useful for conditioning. Hence, Zuiz in (9) is concatenated with Y along the channel dimension to produce the condition c, (a 2d H tensor). For s (the last stage), there is no coarser trend and c, is simply equal to
On inference, the ground-truth Y is no longer available. Hence, future-mixup in (9) is not used, and we simply set Zaix ziitorry. Moreover, the coarser trend Y is also not available, and we concatenate Zaix with the estimate Lr generated from stage + 2 instead.

---

Published as a conference paper at ICLR 2024
Denoising network.
Analogous to (6), the denoising process step is of stage + 1 is given by (10)
Y c.) = =N Hu. (Y k/c,, o))) k ,,.,,,, 1, pe, -
where 02 includes all parameters in the conditional network and denoising network at stage + 1, and
Mo, (Y 8 k/c., of /as (1 (1-061) Y + vāa- Yo, (Y= klc.).
(11)
-r
As in (3), Yo, (Y,, klc,) is an estimate of Y
The denoising network (also shown in Figure 3) outputs Yo, (Y, k/c,) with guidance from the condition c, (output by the conditioning network). Specifically, it first maps Y to the embedding 2º E Rd' xH by an input projection block consisting of several convolutional layers. This z² together with the diffusion-step k's embedding p* E Ied in (7), is fed to an encoder (which is convolution network) to obtain the representation z" E Rd xH Next, we concatenate z and c, along the variable dimension form tensor of size (2d + d") x H. This then fed to a decoder, which is also a convolution network, and outputs Yo, (Y& klc.). Finally, analogous to (4), 0, is obtained by minimizing the following denoising objective
min L. (02) min Ey! (Y Y Ye, (Y Ye,(Y, k(c, 0,
(12)
0,
On inference, for each 8 = S..... : 1, we start from YK NO I) (step Algorithm 2). Based on the data prediction strategy in (11), l), each denoising step from Ý (an estimate of Y. to Ý is:
Y Var ax-1) Yx
13 Yo, ( klc,) + oRC,
(13)
1 1- ā
1-āx
where e 10, when 1, and e otherwise.
The pseudocode for the training and inference procedures of the backward denoising process can be found in Appendix A.
The proposed mr-Di ff is related to the Scaleformer (Shabani et al., 2023), which also generates the forecast progressively from the coarser level to the finer ones. However, besides the obvious difference that Scaleformer is based on the transformer while mr - is based on the diffusion model, Scaleformer uses one single forecasting module transformer) all resolutions. However, as the time series patterns at different resolutions may be different, in mr mr-Diff, each resolution has its own denoising network and thus more flexible. Moreover, moving from a lower resolution to a higher resolution during both training and inference, Scaleformer needs linear interpolation and an additional cross-scale normalization mechanism. On the other hand, in mr-Diff. the lower-resolution prediction is fed into the conditioning network as condition. The conditioning network can learn a more flexible nonlinear mapping to fuse the cross-scale trends. Coherent probabilistic forecasting (Rangapuram et al., 2023) also uses temporal hierarchy to produce forecasts at multiple resolutions. However, it is more stringent than mr-Diff E and requires the model to produce consistent forecasts at all resolutions. As will be shown in Section 5, this enables mr-Di -iiff ff to have better empirical performance.
Discussion.
EXPERIMENTS
In this section, we conduct time series prediction experiments by comparing 22 recent strong pre- diction models on 9 popular real-worid time series datasets. Due to space constraints, detailed introductions about the datasets and selected baselines are in Appendix B and Appendix C, respec- tively. For performance evaluation, we use the mean absolute error (MAE) and mean squared error (MSE) averaged over ten randomly sampled trajectories. Because of the lack of space, results on MSE are in Appendix D. Furthermore, ablation studies are provided in Appendix E.
Implementation Details. We train the proposed model using Adam with a learning rate of 10 10-3, The batch size is 64, and training with early stopping for maximum of 100 epochs. K = 100 diffusion steps are used, with a linear variance schedule (Rasul et al., 2021) starting from 81 = 10 to 8x = 10 S = 5 stages are used. The history length (in (96. 192, 336, 720, 1440)) is selected by using the validation set. All experiments are run on an Nvidia RTX A6000 GPU with 48GB memory. More details can be found in Appendix F.
7

---

Published as a conference paper at ICLR 2024
Table I: Univariate prediction MAEs the real-world time series datasets (subscript the rank).
Results of all baselines (except NLinear, SCINet, and N-Hits) are from (Shen & Kwok, 2023).
NorPool
Caiso
Trulic
Electricity
Weather
Exchonge
ET7M
6T7ml
Wind
agg
mr-Dift
(.609
(212(a)
0.197
0,33
(,032
0.09
0.196
0.14%()
1168/2)
1.7
TimelDiff
0.613
(.209
0.207
0.34)
(,035(4)
0.102
2220
0.154
(.209
4.0
TimeGrad
(2841/2))
0.386/29)
0.84:22;
0.888
0,036
0.155
(2)2(
(.167
(.239
1666
CSDI
(2763
(4282
0.468
0.037
(.200
(222
0.170
1.218
15.1
SSSD
0.770/223
0.263
0.226ge)
(84
(134
0.118
(220
(.169
1,356
DAVAE
(a77
0.237
0.539 (1))
0.03%
(.107
(22)
(.
(19
14.9
CPF
(17 (.338(18)
(1385
0.502
0.035
0.054
(122
0.153(a)
(,256
11.9
PSA-GAN
(.623(5)
(.250
0.355
(63
(333
(.109
(.225(16))
(.17
(.267
13.3
N-Hits
0.646
(,232
0.419
0.033
0. (3
0.157 (2228
(.256
8.9
FILM
0.654m)
(1290
0.315
0.362
(.069
(10)
(.210
()
()
8.6
Depts
(.)6(a)
(12
(.24
0,434
(1)
(12
(2202
(.165
(.472
NBeuts
(167
0.228 (s)
0.225
0.43% (18)
0.130
0.000
(1242
0.165
(,236
10.4
Scaleformer
0.687
(63
0.375
0.430
(m (17)
e.302
02200
(23
PaccTST
(1
(12)
0.26%/13)
0.478
0.098 0.098 18: (.11
0.260
0.174
1.338
14.4
FedFormer
(17
(.25%
0.278
0.453
0.057
0.
0.16% (22)
(22
1,271 (.1
(L4)
14.3
Autoformer
(.75
(13
0.495:21;
0.623 (22)
0.040
(2152
(1220 0.174(16)
(1319(1))
17.3
Pyraformer
(.74
(127 0.215())
0,455
0.107 (0) (.101 (.21l(7)
(91
(:281(:5)
Informer
0.698
0.308 (3345(20)
(133
(.069
0.118(16)
0.212())
0.172
1,236
13.2
Transfommer
(.725(16)
0.345/20)
0.336q16;
(146
(16)
0. (.103(g)
(124
0.196(11)
(.212(0)
15.4
SCINet
(.653
(.24
0.3
0.377
(.037
(.101
(g205
4.150
(.167
NLincar
0.637
0.238
0.192:13
0.334
0.033
(.097
(.203
0.149
1.197
3.3
DLinear
(667
0.206
0,236:a)
0,348 (1)
(33
(.102
(122
0.155
1.221
93
LSTMa
0.707
(3333
0.757
(20)
(105
(183
(.332
(2))
(22
(17
5.1
MAIN RESULTS
Table 1 shows the MAES on the univariate time series. As can be seen, the proposed mr-Diff ff is the best of the 9 datasets. The improvement particularly significant on the more complicated datasets, such as Exchange and ETThI. On the remaining 4 datasets, mr mr-Dife Difff ranks second in 3 of them. Overall, its average ranking is better than all other baselines (which include the most recent diffusion models). Note that there is no improvement on datasets such as Caiso. This is because there is no complex trend information on this dataset that can be leveraged (as can be seen in Figure 5(b) Appendix B).
Figure 4 shows example prediction results on ETTh! by mr- and three competitive models: SCINet, NLinear and the recent diffusion model TimeDiff. As can be seen, mr-Diff produces higher-qualite predictions than the others.
(a) SCINet.
(b) NLinear.
(c) TimeDiff.
(d) mr-Diff.
Figure 4: Visualizations on E7Th! by (a) SCINet (Liu et al., 2022), (b) NLinear (Zeng et al., 2023).
(c) TimeDiff (Shen & Kwok, 2023) and (d) the proposed mr -ffffff.
Table 2 shows the MAES on the multivariate time series. Baselines N-Hits, FiLM and SCINet have strong performance because they also leverage multiresolution information in prediction. However, the proposed mr-Diff ff is still very competitive compared to these strong baselines. It ranks among the 2 methods on of the 9 datasets, and is the best overall. This is then followed by the recent iiffussionbbased model Dife
5.2 INFERENCE EFFICIENCY
In this experiment, compare the inference efficiency of mr-Diff (with 1-4 stages) with the other time series diffusion model baselines (TifeDiff, TimeGrad, CSDI, SSSD) on the univariate ETTh/ dataset. Due space constraints, compare the training efficiency in Appendix G. Besides using
8

---

Published as a conference paper at ICLR 2024
Table 2: Multivariate prediction MAËs on the real-world time series datasets (subscript is the rank). CSDI runs out of memory on Traffic and Electricity. Results of all baselines (except for NLinear, SCINet, and N-Hits) are from (Shen & Kwok, 2023).
NorPool
Caiso
Traffic
Electriitty
Weazher
Exchange
ETThl
ETTm!
Wond
avg rank
(0609
0211%
0.320ka)
(,252
0,324(2)
(082233
0/4222g2)
0,373(2)
(.675
TimeDitt
0.234::
(.384(a)
(3305
0,312(1)
(00
0,430
(,372
(.687
4.2
TimeGrd
(.821
1.3
(220
(.381
(3195
(.219
(2605
(2193
19.8
(,777
0.345.19
0.374
(90)
(.438
(144((15)
(174
SSSD
0.753
0.295
0,398
p23
0.350
p.
0.561
0.406
(.778
12.6
D VAE
0.092
(.3
(43
(.380
(230
0.502 (4)
(,391
0.799
14.7
CPF
(31
0.424
0.714
(224
0,781
(.082
(.597
(,472(10)
0.257
18.0
PSA-GAN
(,890
0.477
0.697
(a533
0,578
(.087
0,516(16)
0.488(18)
0.756
17.7
N-His
(613
0.221
(.268
0.245
0.335
(,08
0/180
0.388
0.734p8)
5.1
FiLM
0.4661))
(
(33
(3300
0.336
0.07%
0.436
0.574
0.717(5)
6.1
Depes
0411
0:204:3:
1.568
(340
(.394
p0100
(.49
(442
(.25
NBeaats
(.832
0.235
0.365())
(.370
1.420
(0
0.52
(.40
(374
Scaleformer
(.769
0,3
0,579
(3304:3)
(.458
(4138
0.57%
(,475 0.864 (296
14,4
PatcTTST
(0710
0.293
0.411a
(3348(11)
0.555ra1)
(.547
(1489
(.392
0.720
FedFormmrr
0.744
0.317
0.385(a)
0.34
0.347(7)
(2233223)
0.181(9)
(.313
0.762
12.1
Autoformer
(37
(.321
0.392(12)
0,513
(.354(a)
(.167
0/481(9)
0.496(19)
0.756
12.3
Paraformer
(378) (23
220)
(.390
(.379/15)
(.385(1))
0.112(11)
(.4
0.735 (.45(114)
13.9
Informer
(.757
0.336:17)
(.39);11)
(3383
0.364(10)
(.192
0.605/21)
(2542(20)
(.772
Transformer
(:765
0.321
0.410
(.405
0.370
0.178
0.567
03592(21)
0.785 (20)
SCINet
0,193
0.335
0,250
0,344,0)
(3)37
(.463
0.389
(.732
5.8
NLinear
0.636
0.223
0.293
(.23
0.328
0.091
0.418
(.375(4)
0,706
DLinear
0610
2.1
(.268
0
(.44
0.102
0:412
0,378 ())
0.686 ()
9.1
LSTMa
0,974
0,305
0,510
(9444
0,501
0.534:20) 0,782
0.699
0.897
20.3
prediction horizon of H = 168 as in the previous experiment, we also use H = -96, 336 as in (Wu et al., 2021; Zhou al., 2022b).
336 and 720
Table 3 shows the number of trainable parameters and inference time. As can be seen, mr mr-Diff has smaller number of trainable parameters. Moreover, even with S or 5 stages, its inference is more efficient than SSSD, CSDI, and TimeGrad. When S = 3 (the setting used in the previous experiments), mr Dar-ifff f even faster than TimeDiff. The inference efficiency of mr-Dif ff over existing diffusion models is due to: (i) It is non- autoregressive, which avoids autoregressive decoding in each time step as TimeGrad. (ii) It uses convolution layers. This avoids the scaling problem in CSDI, whose complexity is quadratic with the number of variables and time series length. (iii) Use of the acceleration technique DPM-Solver (Lu et al., 2022), which further reduces the number of denoising steps in each stage. Moreover, it is also faster than Tife when S = 2 or 3, thanks to its use of a linear mapping for future mixup instead of employing additional deep layers for this purpose.
Table 3: Number of trainable parameters and inference time (in ms) of various time series diffusion models with different prediction horizons (H) on the univariate ETTh!.
Infermnce time (mm) H
of tminnslle
F7 96 f7
(08
152
H=39:
336
FT =20720
mz-uit (8+2)
0.9M
9.5
9.8
ai- (8+3)
1.4
14.3
14.9
(8-4)
1.M
19.1
(5-5)
TimeDitf
17.3
1620.9
1854.5
6724.1
CSDI
128.3
142.8
S3SD
418.6
3902
645.4
1054.2
2516.9
CONCLUSION
In this paper, we propose the mutli-resolution diffusion model mr-Diff. Different from the existing time series diffusion model, mr-Diff ff incorporates seasonal-trend decomposition and uses multiple temporal resolutions in both the diffusion and denoising processes. By progressively denoising the time series in a coarse-to-fie fine manner, mr Di-iiff is able to produce more reliable predictions. Experiments on a number of real-world univariate and multivariate time series datasets show that mr-Diff ff outperforms the state-of- ofe---f-thehaee time series diffusion models. It also achieves very competitive performance even when compared to various families of non non- diffusfioinn--o tiffusion-based time series prediction models.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This research was supported in part by the Research Grants Council of the Hong Kong Special Administrative Region (Grant 16200021), and the Science and Technology Planning Project of Guangdong Province (Grant 2023,050505000)..
REFERENCES
Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecast- ing with structured state space models. Technical report, arXiv, 2022.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations, 2015.
Yaniv Benny and Lior Wolf. Dynamic -ual-output diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
Cristian Challu, Kin G Olivares, Boris N Oreshkin, Federico Garza, Max Mergenthaler-Canseeo. and Artur Dubrawski. N-HITS: Neural hierarchical interpolation for time series forecasting. In AAAI Conference on Artificial Intelligence, 2023.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In Advances
in Neural Information Processing Systems, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsbym. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2020.
Stefan Elfwing, Eiji Uchibe, and Kenji Doya. function approximation in reinforcement learning. Technical report, arXiv, 2017.
linear units for neural network
Wei Fan, Shun Zheng, Xiaohan Yi, Wei Cao, Yanjie Fu, Jiang Bian, and Tie- Liu. DEPTS: Deep expansion learning for periodic time series forecasting. In International Conference on Learning Representations, 2022.
William Harvey, Saeid Naderiparizi, Vaden Masrani, Christian Weilbach, and Frank Wood. Flexible diffusion modeling of long videos. Advances in Neural Information Processing Systems, 2022.
Hansika Hewamalage, Christoph Bergmeir, and Kasun Bandara. Recurrent neural networks for time series forecasting: Current status and future directions. International Journal of Forecasting, 37 (1):388-422) 427, 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models, In Advances in Neural Information Processing Systems, 2020.
Jonathan Ho, Chitwan Saharia, William Chan, Da Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research, 23(4)):1-33, 2022.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):
1735-1780, 1997.
Paul Jeha, Michael Sohhke-Schheider, Pedro Mercado, Shubham Kapoor, Rajbir Singh Nirwan, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. PSA-GAN: Progressive self attention GANs for synthetic time series. In International Conference on Learning Representations, 2022.
Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Hon Choi, and Jaegul Choo. Re- versible instance normalization for accurate time-series forecasting against distribution shift. In International Conference Learning Representations, 2021.
10

---

Published as a conference paper at ICLR 2024
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations. 2020.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-an and short-term temporal patterns with deep neural networks. In International ACM SIGIR Conference on Research & Development in Information Retrieval, 2018.
Yan Li, Xinjiang Lu, Yaqing Wang, and Dejing Dou. Generative time series forecasting with diffusion, denoise, and disentanglement. In Advances in Neural Information Processing Systems, 2022.
Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet:
Time series modeling and forecasting with sample convolution and interaction. In Advances in Neural Information Processing Systems, 2022.
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X. Liu, and Schahram Dust- dar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International Conference Learning Representations. 2021.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In Advances in Neural Information Processing Systems, 2022.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In IEEE/CVF Conference on Computer Vision and Patlern Recognition, 2022.
Kiran Madhusudhanan, Johannes Burchert, Nghia Duong-Trung, Stefan Born, and Lars Schmidt- Thieme. Yformer: U-net inspired transformer architecture for far horizon time series forecasting. Technical report, arXiv, 2021.
Yuqi Nie, Nam Nguyen, Phanwadee Sinthong. and Jayant Kalagnanam. time series is worth 64 words: Long-term forecasting with transformers. Technical report, arXiv, 2022.
Hao Niu, Chuizheng Meng, Defu Cao, Guillaume Habault, Roberto I Legaspi, Shinya Wada, Chihiro Ono, and Yan Liu. Mu2ReST: Multi-resolution recursive spatio-temporal transformer for long-term prediction. In Advances in Knowledge Discovery and Data Mining, 2022.
Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. In International Conference on Learning Representations, 2019.
Syama Sundar Rangapuram, Shubham Kapoor, Rajbir Singh Nirwan, Pedro Mercado, Tim Januschowski, Yuyang Wang, and Michael Bohike-Schneider. Coherent probabilistic forecasting of temporal hierarchies. In International Conference on Artificial Intelligence and Statistics, 2023.
Kashif Rasul, Calvin Seward, Ingmar Schuster, and Roland Vollgraf. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In International Conference on Machine Learning. 2021.
Cleveland Robert, C William, and Terpenning Irma. STL: seasonal-trend decomposition procedure based on loess. Journal of Official Statistics, 1990.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention, 2015.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to poxt-to-ime image diffusion models with deep language understanding. In Advances in Neural Information Processing Systems, 2022.

---

Published as a conference paper at ICLR 2024
Mohammad Amin Shabani, Amir H. Abdi, Lili Meng, and Tristan Sylvain. Scaleformer: Iterative multi-scale refining transformers for time series forecasting. In International Conference on Learning Representations, 2023.
Lifeng Shen and James Kwok. Non-autoreeressive conditional diffusion models for time series prediction. In International Conference on Machine Learning, 2023.
Yusuke Tashiro Jiaming Song, Yang Song, and Stefano Ermon. CSDI: Conditional score-basee cosre-based
diffusion models for probabilistic time series imputation. In Advances Neural Information Processing Systems, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Ronald Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1(2):270-280, 1989.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transform- ers with auto-correlation for long-term series forecasting. In Advances Neural Information Processing Systems, 2021.
Bing Yu, Haoteng Yin, and Zhanxing Zhu. St-unet: A spatio- temtoral U-network for strupurtt time series modeling. Technical report, arXiv, 2021.
Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, and Bixiong Xu. TS2Vec: Towards universal representation of time series. In AAAI Conference on Artificial Intelligence, 2022.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In AAAI Conference on Artificial Intelligence, 2023.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference Learning Representations, 2018.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang, Informer: Beyond efficient transformer for long sequence time-series forecasting, In AAAI Conference on Artificial Intelligence, 2021.
Tian Zhou, Ziqing MA, Xue Wang, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, and Rong Jin. FiLM: Frequency improved legendre memory model for long-term time series forecasting. In Advances in Neural Information Processing Systems, 2022a.
Tian Zhou, Ziqing Ma. Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning. 2022b.
12

---

Published as a conference paper at ICLR 2024
A
PSEUDOCODE
The training and inference procedures are shown in Algorithms and 2, respectively.
Algorithm 1 Training procedure of the backward denoising process.
1:
repeat
(Xu: Yu) ~ q(X, Y)
{X S- = TrendExtractien Xo)
4:
TrendExtracticn Yo)
5:
for --1511.... -
6:
&~ Und.orrm(11.......
7:
I)
8:
Generate diffused sample Y by =
Y. +v
arc
9:
Obtain diffusion step k's embedding p using
10:
Randomly generate matrix in (9)
11:
Obtain Zaistory using linear mapping X,
12:
Obtain Zeix using (9)
13:
s < S-1 1 then
14:
Obtain condition
concatenating 7aix and Y ++1
15:
else
16:
Obtain condition c, Znix
17:
end if
18:
Calculate the loss C (0.) in (12)
Take a gradient descent step based on Vo, (0s)
20:
end for
21:
until converged
Algorithm 2 Inference procedure.
1:
[X.]==1....S-1 = rrendextraction//
2:
for 80 = ..... do
Obtain Zaiszory using linear mapping on X,
then
Obtain condition e, by concatenating Enistory x
else
7:
Obtain condition Bristory
8:
end if
Initialization: Y.K I)
10:
for k..., do
e~~( 1). if clsc e=
12:
Obtain diffusion step k's embedding pd using (7)
13:
Obtain the denoised output k by (13)
14:
end for
15:
end for
16:
return Y
B DATASETS
Experiments are performed on nine commonly-used real-world time series datasets (Zhou al., 2021; Wu et al., 2021; Fan et al., 2022): (i) NorPool which includes eight years of hourly energy production volume series in multiple European countries; (ii) Caiso 2, which contains eight years of hourly actual electricity load series in different zones of California; (ii) Traffic 3, which records the hourly road occupancy rates generated by sensors in the San Francisco Bay area freeways; (iv) Electricity" 4, which includes the hourly electricity consumption of 321 clients over two years; (v) Weather which records 21 meteorological indicators at 10-minute intervals from 2020 to 2021; (vi)
Phttp://wwe http://wwwweeeeeyylnne......oo
'http://pems.dos cav.vv
'https://archiveeiiss uui.ii.......................... edu/mi
120044
htttps://uw.bbc--eeett wwt.:jeaaaaaa.....mmmmmmmm de/vetter/
13

---

Published as a conference paper at ICLR 2024
Exchange (Lai et al., 2018), which describes the daily exchange rates of eight countries (Australia, British, Canada, Switzerland, China, Japan. New Zealand, and Singapore); (vii)-(viii) ETTh/ and ET7ml,', which contain two years of electricity transformer temperature data (Zhou et al., 2021) collected in China, at 1-hour and 15 15-minute intervals, respectively; (ix) Wind & which contains wind power records from 2020-2021 information. As different datasets have different sampling interval lengths, we follow (Shen & Kwok, 2023) and consider prediction tasks with more reasonable prediction lengths.
15-minute intervals (Li et al., 2022). Table 4 shows the dataset
Table 4: Summary dataset statistics, including the dimension, total number of observations, sampling frequency, and prediction length H used in the experiments.
dimension
#observations
frequency
steps (HI)
NorPool
18
70,128
1 hour
720 (1 month)
Caiso
10
74,472
I hour
720 (1 month)
Traffic
862
17,544
I hour I hour 10 mins
168 (1 week)
Electricity
321
26,304
168 week)
Weather
21
$2,696
672 week)
Exchange
7,588
day
weeks)
ETTh]
17,420
I hour
168 (1 week)
ETTml
7
69.680
15 mins
192 (2 days)
Wind
48,673
15 mins
192 (2 days)
Besides running directly on these multivariate datasets, we also convert them to univariate time series for performance comparison. For NorPool and Caiso, the univariate time series are extracted from all variables in (Fan et al., 2022). For the other datasets, we follow (Wu et al., 2021; Zhou et al., 2021) and extract the univariate time series by using the last variable only.
Figure 5 shows examples of the time series data used in the experiments. Since all of them multivariate, we only show the last variate. As can be seen, these datasets are representative in exhibiting various temporal characteristics (different stationary states, periodicities, and sampling rates). For example, Electricity contains obvious periodic patterns, while E7Th/ and ETTm/ exhibit non-stationary trends.
Real-world time series often have irregular dynamics over time or across variables. As in many deep time series models (Kim et al., 2021; Zhou al., 2022a; Liu et al., 2022: Zeng et al., 2023), it is often useful to normalize the scales of the time series in each window. In the proposed method, we use instance normalization, which has also been used in RevIN (Kim et al., 2021) (which is referred to as reversible instance normalization) and FiLM (Zhou et al., 2022a). Specifically, we subtract the time series value in each window by its lookback window mean, and then divide by the lookback window's ' standard deviation. During inference, the mean and standard deviation are added back to the prediction.
C BASELINES IN MAIN EXPERIMENTS
We compare with several types of baselines, including: (i) recent time series diffusion models: non-autoregressive diffusion model TimeDiff (Shen & Kwok, 2023), TimeGrad (Rasul et al., 2021), conditional score-based diffusion model for imputation (CSDI) (Tashiro al., 2021), structured state space model-based diffusion (SSSD) (Alcaraz & Strodthoff, 2022); (ii) recent generative models for time series prediction: variational autoencoder with diffusion, denoise and disentanglement (D&VAE) (Li et al., 2022), coherent probabilistic forecasting (CPF) (Rangapuram et al., 2023), and PSA-GAN (Jeha et al., 2022); (ii)) recent prediction models based on basis expansion: N- Hits (Challu et al., 2023), frequency improved Legendre memory model (FiLM) (Zhou et al., 2022a), Depts (Fan et al., 2022) and NBeats (Oreshkin al., 2019); (iv) time series transformers: Scaleformer (Shabani et al., 2023), PatchTST (Nie et al., 2022), Fedformer (Zhou et al., 2022b), Autoformer (Wu al., 2021), Pyraformer (Liu al., 2021), Informer (Zhou al., 2021) and the
hhttps://githb. http://thhuu.coo///oooaeaaeeeeee http://..wwooooooaaaoy///aaaaaaa
latasets/windde http::/gg...hhccoooo///aeeeeaaaa http:///ea..aapaaaaaaaeeeeeeaaaa
14

---

Published as a conference paper at ICLR 2024
w/h
(a) NorPool.
(b) Caiso.
M MoMtoaW Mooi Min Ma Matherlll
(c) Traffic.
(d) Electricity.
(c) Weather.
(f) Exchange.
CANNN www. Mamn
g ETThI.
(h) ETTml.
(i) Wind.
Figure Visualization of the time series datasets.
standard Transformer (Vaswani al., 2017); and (v) other competitive baselines: SCINet (Liu et al., 2022) that introduces sample convolution and interaction for time series prediction. NLinear (Zeng al., 2023), DLinear (Zeng al., 2023) LSTMa (Bahdanau al., 2015), and an attention-based LSTM (Hochreiter & Schmidhuber, 1997). We do not compare with (Yu et al., 2021; Niu et al., 2022; Madhusudhanan et al., 2021) because their codes are not publicly available.
D RESULTS ON MSE
Tables 5 and 6 show the MSE results the univariate and multivariate time series, respectively, for the experiment Section 5.1. As can be seen, the proposed mr-Dif iff ff still achieves the best overall MSE performance in terms of average rank. Moreover, since deep models for time series forecasting may be influenced by different random initializations, Table 7 shows the prediction results on the univariate time series over 5 random runs. As can be seen, the largest standard deviation is 0.0042, indicating that the proposed model is robust towards different initializations.
E ABLATION STUDIES
In this section, we perform a number of ablation experiments on mz-Diff using the Electricity, ETTh] and ETTm/ datasets. These datasets representative in exhibiting various temporal char- acteristics (different stationary states, periodicities, and sampling rates). For example, Electricity contains obvious periodic patterns, while E7Th/ and ETTm/ exhibit non-stationary trends.
15

---

Published as a conference paper at ICLR 2024
Table 5: Univariate prediction MSEs on the real-world time series datasets (subscript is the rank). Results of all baselines (except NLinear, SCINet, and N-Hits) are from (Shen & Kwok, 2023).
Method
NorPool
Caiso
Truffic
Electricity
Weather
Exchange
ETThl
ETTml
Wind
Rank
mr-Diff'
(4)
(3)
(1)1
(,234(a;
(10
0.016.1)
1.066
(.039(2)
(41
2.2
TimeDiff
(,636
(22 (212 (0
(1
0.017a)
()
0.040;s)
(.407
3,7
TimeGrad
(22)
(.325(22)
(23))
(23)
(1
(.041
0.078(10)
0.048(11)
2.530
16.7
CSDI
(29)
(1
(200
(15)
(1
0.071
(18)
(.050
2,434(15)
15.8
SSSD
(214
(.17
(.)
(137
(100
(.023
(209
(.049(18)
(.149(22)
15.1
VAR
0.521 (20)
(25)
(,15
(193
(.
(.019/12)
(107
(.044(9)
(.679/20)
14.4
CPP
(135
(2)
(187
(20)
0.002
0.016
(3080
0.041
2,430
12.2
PSA GAN
(,
mm
(,250
(,273
0.035
0.020
(.084
0.05
2.510
13.0
N-Hits
0.73%())
(11
(.147
(.346
(10
0.012:4)
(17
(.043;))
2.406(12)
8.6
Fil
(m
(33)
(13)
0.007
0.018
(.070
0.03(()
2.143(2)
7.4
Depts
(5)
0.107()
(,)
(4380
(1024(19)
0.020/133
0.070
0.046
(,457
10.7
NBeats
(,7
(.)25())
(,142())
(.378
(2137
0.016:13
(109
0,048(11)
(,434(16)
11.4
Scaleformer
0,778
(12
(.286
(,361
(.)
(.03
(.150(22)
(.078
(.646
17.2
PatchTST
0.595
(18)
(127
(745
(.026
(30
0,106(21)
0.052(18)
(.698(21)
21
FodFormer
(189
0,164(0)
0.173(m))
0,376(12)
0.005
0.050/22)
0.076(7)
0.065/21)
2,351
13.4
Autoformer
0,946(19)
(124 (,47
(2659
0.003
02041:20)
(40
(.051
2.349(10)
16.4
Pyraformer
(183
(10)
0.136(1)
(1389
(.02
0.017:7)
0.076(1)
0.054(19)
2.279(6)
11.2
Informer
0,804
(185
(22
(136
(140)
(.023;16)
0.076(7)
0.049
(.297
12.6
Transforner
(,928
(1250
(,258
0.430(14)
(.00
0.018;0)
(1092
(205
2.306(D)
15.2
SCINet
0.746 (.154())
0,212
0,002(1)
0.018:9)
(.071
0.039:2)
(.063
6.3
NLincar
0,708gs)
(,
0.124g))
(1231
0.002
0.017:4)
0.070
0.039
2.193(5)
3.7
DLincar
0,671,)) 0.118(2)
0.13%(z)
0.24(()
0.168
0.012(4)
0,078(16)
0.041
2.17
L.TMa
0.836/14) (22)) (,032
0.596(20)
0.005
(.03
0.167(28)
0.091(23)
2.299(8)
17.8
Table 6: Multivariate prediction MSEs on the real-world time series datasets (subscript is the rank). CSDI runs out of memory on Traffic and Electricity. Results of all baselines (except for NLinear, SCINet, and N-Hits) are from (Shen & Kwok, 2023).
Method
Norfool
Caiso
Traffie
Electricity
Weather
Exchange
ETThl
ET7ml
Wind
Rank
mr-Diff
(,45(1))
0.127(a)
0.474(6:
0.155(a)
0.296(1)
0.016(1)
0.411(a)
(.34D
0.881(1)
2.4
TimeDiff
(4665
0.136r6)
0,564(m:
0,193(s)
0
0.018;c)
0,407:1)
0,336
0.896/2)
3.8
TimeGrad
1.152/20)
(.258(19)
(,745
0.736(21)
0.392
0.079
0.993/22)
0.874(21)
(220
20.0
CSDI
(01l
(1253
0.356
0.077
0.497
0.52
(,060())
14.0
SSSD
(,872(12)
0.195(10)
0.642
(.255
0.349(8)
0.061
0.726
0.464(13)
(118
13.2
D
(,745
0.241
(,928(17)
(,28
(1375
0,200
0,504:g)
0,362(e)
(118
13.7
CPF
1,63)pa)
0,383
(.625
0. (279)
(330
0.016
0,730
(,482
I. 1.140
18.0
PSA-GAN
1,501
0.510
1,614/20)
0.535
(,220
0.018
0.623
0.537
1.127
18.1
N-Hits
0,716
0.131;4)
0
0.152(2)
0.323
0,017,0)
0.498
0,353
())
4.9
FiLM
(,723
0.17%(s)
(.628(10)
0.210(a)
0,327
0.016.1)
0.426:3)
0,347
0.984(4)
5.9
Depts
0,662
0. 106
1019
(.319
0,761
0.020
0,579(15)
0,380
I. 1.082
11.6
NBeats
(1832
0.141(7)
(13
(126
(.344(22)
0.016(1)
0.586
0,391
(.06
10.0
Scaleformer
(,983
0,618 (.2
(.1
(146
0,036
0.481 0.613/16)
(,35
13.7
PatchTST
(155
0.193
0.831 (10)
0.225
0.782
0.047
0.526
0,372
1070
11)
12.3
FedFormer
(387 0,205(11)
0,591
0,238(11)
0.342(m)
0,135
0,541
0.426
L.13
12.1
Autoformer
0,940 0.226/16)
(.688
0,201(7)
0.360
0.056(15)
0.516;10)
0.565
1.08
13.1
Pyralormer
(100
0.273
(165
0.273
(.394(16)
0.032(11)
0.579
0.493
L.06l
14.1
Informer
0.985
0,231(a))
0.664(1))
0.29
0.38
0.073
0,775
0.673
(18)
16.7
Transfomerr
(.00
0,206(12)
0.671
(.328
(138
0,062(17)
0.759
0,992
1.201
17.0
SCINet
0.61
0.095(1)
0.434(6;
0.171
0.329
0.036
0.465
0.359
1.055(7)
5.4
NLinear
(6707
0.135(s)
(4:
0.147
0.313
0.019(s)
0.410:2)
0.349
0.989(s)
DLincar
0.670(5)
0.461(25)
0.389
0.215
(1488
0.022
0.415(4)
0,345
(.89)
8.4
LSTMa
(2481
0.217/14)
0,966(18)
0.414(19)
0.662
0.403
1.149
1.030
1.464(23)
20.2
Length of lookback window L. Table 8 shows the prediction MAE of mr-Diff with different
lengths (L) of the lookback window. We consider L = (96, 336, 720, 1440) as used in (Shen & Kwok, 2023; Zeng al., 2023; Wu et al., 2021). As be seen, on Electricity, ETTh/ and
16

---

Published as a conference paper at ICLR 2024
Table 7: Univariate prediction errors of mr-Di obtained on five
NorPbol
Traffic
Electricity
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSB
0 0.5096
0.6668
0.2129
0.1225
0.1973
0.1192
0.3322
0.2331
0.6092
0,6664
0.1226
0.2293
0.6087
0.1220
3
0.6099
0.6672
0.2129
0.1225
0.1976
0.1192
0.3382
0.2347
4
0.6091
0.6665
0,2120
0.1221
0.1973
0.1195
0.3321
0,2339
mean
0.6093
0.6666
0.2120
0.1223
0.1973
0.0005
0.0017
0.0003
Wearher
Exclange
ETThi
E77m/
Wind
MAE
MSE
MAE
MSE
MAE
MSB
MAE
MSE
MAE
MSE
0.U322
0.0019
0.0936
0.0155
0.1964
0.0663
0.1486
U.0389
1.1704
2.1862
1.1682
0.0323
0.0018
0.0947
0.0158
0.1956
0.0660
0.1483
0.0384
1.1690
2.1822
0.0322
0.0019
0.0954
0.0160
0.1959
0.0661
0,1486
0.0387
1.1676
2.1807
0.0322
0.0018
0.0946
0.0159
0.1965
0.0671
0.1489
0.0389
1.1670
2.1804
0.0664
1.1684
2.1823
std deviation
0.0001
0.0001
0,0008
0,0002
0.0005
0.0004
0.0003
0.0002
0.0013
0.0023
ETTml (corresponding to 168/168/192-step-ahaaee prediction, respectively), good performance can be obtained when L is 336 or above.
X
training only
lookbock window
ferencee only
N2
rondom noise
forecast
netwerkk
Yg
findl prediction
network
forward diffusion
Figure 6:
The baseline without scasonal-tres trend decomposition blocks.
Table 8: Prediction MAE versus lookback window length L..
Electricity
ETThl
ETTm]
96
0.449
0,210
0.165
192
0.376
0.202
0.163
336
0.362
0.196
0.157
720
0,332
0.196
0.152
1,440
0.346
0.199
0.149
Number of stages In this experiment, we vary the number of stages S. The kernel size тg is generate fine-to-coarse trends. We also compare
set to increase with the stage number with a variant that does not perform seasonal-trend decomposition (illustrated in Figure 6). This variant directly generates the forecast window from random noise vector without using the coarser
17

---

Published as a conference paper at ICLR 2024
trend s+1- Table shows the prediction errors. As can be seen, using multiple stages improves performance, and not using the ccasonal-trens decomposition leads to the worst performance.
Table Prediction MAE versus number stages S. set to the optimal setting Table (i.e.,
Electricity: 720, ETThl: 336, ETTml: 1440.
Electricity
ETTml ETThl
0.403
0.208
0.1573
2
0.389
0.200
0.1529
0.363
0.196
0.1525
0.346
0.197
0.1509
5
0.332
0.197
0.1496
Number of difflusion steps K. Table 10 shows the prediction error of mr mr-Diff with K, the number of diffusion steps. As can be seen, setting K 100 leads to stable performance across all four datasets. This also agrees with (Li al., 2022) in that a small K may lead incomplete diffusion, while K too large may involve unncessary computations.
Table 10: Prediction MAE versus number of diffusion steps K.
K
ETThl Electricity
ETTml
50
0.348
0.199
0.151
100
0.332
0.196
0.149
500
0.335
0.198
0.151
1,000
0.337
0.199
0.151
Gaussian noise variance 3g- Recall from Section 5 that 3g is generated by a linear variance schedule (Rasul et al., 2021). In this experiment, vary 8x in (0.001,0.01.0.10000)) with 81 fixed to 10-1, As can seen from Table 11, setting 3x = 0.1 always has the best performance. Li et al. (2022) has similar conclusion that a 38x small can lead unsatisfactory diffusion, while 8x too large can make the diffusion out of control.
Table 11: Prediction MAE versus Gaussian noise variance upper bound 34-
3x
Electricity
ETThl
ETTml
0.001
0.403
0.198
0.151
0.01
0.365
0.198
0.152
0.1
0.332
0.196
0.149
0.9
0.372
0.201
0.154
Future mixup. Recall that each element of the future mixup matrix m in (9) randomly sampled from the uniform distribution on [0; 1). In this experiment, study the effect of m by sampling each of its elements from the Beta distribution Beta(y, where y E {0.1, 1, 2}. Note that Beta (1, 1) reduces to the uniform distribution. Also, include mr mr-Diff variant that does not future mixup (by replacing (9) with Zιix = Znistory). Table 12 shows the prediction results. As can be seen, the uniform distribution (with 7 1) used in mr-Di ff has the best performance.
F MORE IMPLEMENTATION DETAILS
In the proposed mr -ir ff, the conditioning network and the denoising network's 's encoder/decoder built by stacking a number of convolutional blocks. The default configuration of each convolutional block is shown in Table 13.
The number selected from 12,3,4.5) (the number of stages S is greater than the number of
decompositions by 1). When S 2, the kernel size is selected from (5, 25); When S
18

---

Published as a conference paper at ICLR 2024
Table 12: Prediction MAE with future mixup matrix m from different Beta distributions Beta(r, 7)-
Electricity
ETThl
ET7ml
0.1
0.348
0.202
0.154
0.332
0.196
0.149
2
0.776
0.222
0.200
variant without future mixup
0.349
0.198
0.151
Table 13: Configuration the convolutional block.
layer
operator
default parameters
1
Convid ld
in channel-256, out channel=256, kernel size=3, stride=1, padding=1 number fcaturex=256
2
BatchNormid
3
LeakyReLU
negative slope=0.1 dropout rate=0.1
4
Dropout
the kernel size (2) is selected from (5, 25), (25, 51), (51, 201)); When S = 4, the kernel size (TI, T2, т3) is selected from {(5. 25, 51), (25, 51, 201)). When 5, the kernel size (7 T2, T3, T4) is selected from [(5, 25,51, 51, 201)]. In practice, we run grid search over S for each dataset. This is not expensive as the number of choices is small (as can be seen from above). Usually, combinations (5,25) and (25,51) achieve promising performance.
and the
G TRAINING EFFICIENCY
In Section 5.2, discussed inference efficiency. In this section, further compare training efficiency mr-Dii ff with other time series diffusion model baselines (TimeDiff, TimeGrad, CSDI, SSSD) the univariate ETThl dataset. As can be seen from Table 14, the proposed model requires less training time compared to TimeGrad, CSDI, and SSSD. This suggests that the proposed model is more efficient in terms of training. This is because, during training, mr-Dife ff uses convolution layers in its denoising networks, which eliminates the need for large modules like residual blocks (in TimeGrad and SSSD) and self-attention layers (in CSDI). Moreover, it is also faster than TifeDiff, thanks to its use of linear mapping for future mixup instead of employing additional deep layers for this purpose.
Table 14: Training time (ms) of various time series diffusion models with different prediction horizons (H) on the univariate ETTh/.
training time (ms)
Fl
Ff 168
Hf 192
Hf
336
Ff 720
mr-Dife (S=2)
D1 (S=3)
0.59
0.69
0.71
0.74
0.82
mi-Difd (S=4)
0.78
0.96
0.99
1.07
(S=5)
1.19
1,26
1.27
1.31
1.72
TimeDill
TimeGrad
2.11
2.42
3.21
4.22
5.93
CSDI
5.72
7.09
7.59
10.59
17.21
SSSD
16.98
19.34
22.64
32.12
52.93
19


Reviews:
[{"review_id": "xWP5EtuiRA", "reviewer": "Reviewer_aTA7", "summary": "This paper combines a multi-resolution structure and a diffusion model to forecast a time series. Similar to TimeDiff, it has a forward diffusion process that sequentially extracts fine-to-coarse trends from the time series and a backward denoising process in an easy-to-hard non-autoregressive manner.", "questions": "See above", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The main contribution of this work compared to the TimeDiff is that the authors added a multi-resolution structure. The authors have conducted extensive experiments to demonstrate the superiority of the proposed method compared with other existing models.", "weaknesses": "The authors mentioned that there have been recent works using U-Net to capture multi-resolution patterns. Considering that the main contribution of this work compared with the TimeDiff is incorporating a multi-resolution structure in a diffusion model, it is essential to compare the proposed seasonal-trend decomposition and the U-Net style decomposition.\n\nIn terms of computation complexity, the authors should also compare the model training time, especially with TimeDiff. For inference, the authors stated that \"mr-Diff is even faster than TimeDiff\" which is also suspicious. All factors presented by the authors apply to TimeDiff as well. Specifically, TimeDiff can be regarded as mr-Diff with S = 1.\n\nThere are some claims not well justified.\n1. Why “while the seasonal-trend decomposition obtains both the seasonal and trend components, the focus here is on the trend.”?\n2. Why “when X0 is used as in existing time series diffusion models, the denoising network may overfit temporal details at the finer level”?\n\nMinor:\nunlike existing --> Unlike existing\nthat directly denoise --> that directly denoises", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WDsHNlAiGG", "reviewer": "Reviewer_V8ZH", "summary": "This paper proposes and studies a novel diffusion model for forecasting time series. The model utilizes the seasonal-trend decomposition to sequentially extract fine-to-coarse trends from the time series for forward diffusion, and the denoising process proceeds by generating the coarsest trend first and the finer details are progressively added (using the predicted coarser trends as condition variables).", "questions": "- MAE and MSE are used as metrics to evaluate the forecasting performance. How about using MAPE? \n- How effective is the model for long-term forecasting (i.e. when H is large)? Can it reconstruct the long term statistics of the underlying dynamical system accurately? \n- What are the limitations of the proposed model? How will the proposed model perform on all the tasks of the Monash benchmark?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Overall the paper is well written \n- A solid contribution to extend recent diffusion models for time series forecasting by leveraging a seasonal-trend decomposition", "weaknesses": "- The proposed model is restricted to/evaluated on only forecasting tasks but not on generation tasks and downstream tasks like classification (c.f. https://arxiv.org/abs/2303.09489) \n- The proposed model is somewhat complicated, consisting of many components, and it is not clear how the model performance would be affected if one of the components is not used, as well as how this depends on the multiscale nature of the time series. A systematic approach is to carefully evaluating the model on synthetic time series data generated by multiscale dynamical systems; e.g., multiscale Lorenz-96 systems\n- Missing discussions on training efficiency and number of trainable parameters, and how do they compare to other models \n- Missing evaluations on other forecasting tasks in the Monash benchmark: \nhttps://forecastingdata.org/\n- Error bars evaluation is not completed (provided for only Electricity, ETTh1, ETTm1 but not the others)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "08wOuOJRr3", "reviewer": "Reviewer_xEhe", "summary": "In this paper, the authors proposed a diffusion model for the time series forecasting problem. Specifically, they introduced a solution for applying multi-resolution analysis in a time series diffusion model.", "questions": "The authors proposed a neat solution to make use of the multi-resolution analysis in a diffusion model for the time series forecasting. I have following questions regarding the experiment results:\n1. The experiment results seem suggest that the performance does not benefit a lot from the multi-resolution analysis. The performance gain is not significant after comparing it with the best models in diffusion models, basis expansion-based models, and the last category of models. The authors claimed that the lack of complex trend information is responsible for the absence of improvement. Could the authors provide further elaboration on this point? What could be considered as complex trends that the proposed model can take advantage of?\n2. What is the inference time of the proposed model compared to NLinear and DLinear?\n3. The performance gain in multivariate prediction is even less significant when comparing the data in Table 1 and 2, as well as in Table 13 and 14. Can the authors elaborate on this observation?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The problem studied in this paper is interesting and important\n2. The proposed solution enables the diffusion model to handle and utilize multi-resolution time series.", "weaknesses": "1. More analysis is needed on the experiment results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "goT1uDgDl6", "reviewer": "Reviewer_WNFV", "summary": "In this paper, the authors proposed a new diffusion-based time-series forecasting model that involves a coarse-to-fine trend generation process.  The empirical study shows that the proposed model outperforms the state-of-the-art diffusion models proposed in this area and has the comparable performance in comparison to the best existing time-series forecasting models.", "questions": "Please check the Weaknesses section. \n\nAdditionally, the paper claims that the proposed model enjoys a lower computational complexity by dropping the attention layer in their architecture; however, this may result that when the generated time series is sufficiently long, the beginning and the end of the series might be independent of each other and cause some inconsistency problem. I wonder if the authors could give some comments on this issue.\n\n\n\n---- post rebuttal ----\n\nI thank the authors for the clarifications and the additional results. All my concerns have been addressed, and I have adjusted the score correspondingly. I hope the authors can include the results in the final version if the paper gets accepted.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed model achieves the state-of-the-art among the diffusion-based forecasting models in terms of the inference speed and the output quality. Additionally, according to Table 1, it also has the best performance on average among the listed forecasting models.", "weaknesses": "1. It appears that the networks at different stages are independent, which suggests that the total number of parameters in the proposed model is significantly higher than the ones in other models. If so, the performance of mr-diff would be less impressive than what appears in the paper. I think the authors should discuss this issue in the paper to avoid potential misinterpretation.\n2. More details should be given regarding the seasonal-trend decomposition and the generation of the coarse trends.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)