ANT: Adaptive Noise Schedule for Time Series Diffusion Models
Authors: Seunghan Lee
Kibok Lee
Taeyoung Park
Venue: NeurIPS 2024 poster (2024)
Keywords: Diffusion Model
Noise Schedule
Time Series
Primary area: diffusion_based_models
TL;DR: We propose ANT,  Adaptive Noise schedule for TS diffusion models,  to select an adaptive schedule based on the non-stationarity statistics of time series.

Abstract: Advances in diffusion models for generative artificial intelligence have recently propagated to the time series (TS) domain, demonstrating state-of-the-art performance on various tasks. However, prior works on TS diffusion models often borrow the framework of existing works proposed in other domains without considering the characteristics of TS data, leading to suboptimal performance. In this work, we
propose Adaptive Noise schedule for Time series diffusion models (ANT), which automatically predetermines proper noise schedules for given TS datasets based on their statistics representing non-stationarity. Our intuition is that an optimal noise schedule should satisfy the following desiderata: 1) It linearly reduces the non-stationarity of TS data so that all diffusion steps are equally meaningful, 2) the data is corrupted to the random noise at the final step, and 3) the number of steps is sufficiently large. The proposed method is practical for use in that it eliminates the necessity of finding the optimal noise schedule with a small additional cost to compute the statistics for given datasets, which can be done offline before training. We validate the effectiveness of our method across various tasks, including TS forecasting, refinement, and generation, on datasets from diverse domains. Code is available at this repository: https://github.com/seunghan96/ANT.

ANT: Adaptive Noise Schedule for Time Series Diffusion Models
Seunghan Lee, Kibok Lee; Taeyoung Park
Department of Statistics and Data Science, Yonsei University {seunghan9613, kibok, teark)peoonsseese ac
Abstract
Advances in diffusion models for generative artificial intelligence have recently propagated to the time series (TS) domain, demonstrating state-of-the-a perfor- mance various tasks. However, prior works on TS diffusion models often borrow the framework of existing works proposed in other domains without considering the characteristics of 'T data, leading suboptimal performance. In this work, we propose Adaptive Noise schedule for Time series diffusion models (ANT), which automatically predetermines proper noise schedules for given TS datasets based on their statistics representing non-stationarity. Our intuition is that an optimal noise schedule should satisfy the following desiderata: 1) It linearly reduces the non-stationarity of TS data so that all diffusion steps are equally meaningful, 2) the data is corrupted the random noise at the final step, and 3) the number of steps is sufficiently large. The proposed method is practical for use in that it eliminates the necessity of finding the optimal noise schedule with small additional cost compute the statistics for given datasets, which can be done offline before training. We validate the effectiveness of our method across various tasks, including TS forecasting, refinement, and generation, on datasets from diverse domains. Code is available this repository: https: //github. com/ wemm//uuunann6/ANT
Introduction
Diffusion models have demonstrated outstanding performance in generative tasks across diverse domains, and various methods have been proposed in the time series (TS) domain to address a range
of tasks, including forecasting, imputation, and generation 26 3d 38 works primarily focus on determining which architecture to use for a diffusion model, overlooking the useful information from the domain knowledge for other components, e.g., noise schedule.
18 30 I.. However, recent
Noise schedules control the noise added to the data across the diffusion process, with the choice of schedule being crucial for performance [4]. Several works have explored the design of noise schedules 124211 however, they do not consider the
0.041 0.040 0.040
0.039
Transformer
0.036
CRPS
+ANT (Ours)
characteristics of data, resulting in suboptimal performance, especially in the TS domain. Figure [ shows the forecasting performance of various models including our method applied TSDiff [15] on the M4 dataset (23]. Among the various schedules, our method chooses cosine schedule for the M4
DeepState
TSDiff
CSDI
0.026
dataset, yielding 27.5 27.8% gain compared to a linear schedule of TSDiff, which is common schedule across TS datasets. This highlights the importance of selecting an appropriate schedule for each dataset.
Figure/
Performance gain by ANT.
In this work, we propose Adaptive Noise schedule for Time series diffusion models (ANT), method for choosing an adaptive noise schedule based on the statistics representing non-stationarity of the TS dataset. These statistics measure the patterns appeared in the TS, with TS from the real world exhibiting high non- stationarttt while white noise TS exhibiting low statttaiinn.rity. We argue that a desirable schedule should gradually transform con-stationary TS into stationary ones, in line with
"qqual advising.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

prior research suggesting that the noise level at each step should remain consistent (24] Furthermore, we opt for schedule that corrupts the TS into random noise the final step to align the training and inference stages [1]] and a schedule that has a sufficient number of diffusion steps to enhance sample quality [II]. Specifically, we discover that a schedule which decreases the non-stationarity of TS on linear scale corrupts the TS into random noise gradually. making TS across steps more distinguishable, i.c., making each step equally meaningful. Figure 2 shows the forward process using two schedules: the base schedule of TSDiff' (wlo ANT) and the schedule proposed by our method (w/ ANT) which aims to decrease the non-stationarity linearly.
Additionally, we investigate the usefulness of diffusion step embedding (DE) in TS diffusion models and argue that it not necessary when using a linear schedule, as the step information is inherent in the data. We also discover that a non-linear schedule is more robust to total diffusion steps (T) compared to a linear schedule, yielding consistent performance across various resource constraints on T. The main contributions of this paper are summarized as follows:
We introduce ANT. an algorithm designed to select an appropriate noise schedule, which is I) adaptive in that the schedule is selected based on the statisties of datasets, 2) flexible in that any noise schedule can be a candidate, 3) model-agnoste in that it only depends on the statistics of datasets, and 4) efficient in that no training is required, as the statistics can be precomputed offline.
We study the components of diffusion models regarding noise schedules, arguing that diffusion step embedding is unnecessary when using a linear schedule. Additionally, we find that a non-linear schedule exhibits greater robustness to the number of diffusion steps compared to linear schedule.
We provide extensive experimental results across various datasets, demonstrating that our method outperforms the baseline in a range of tasks, including TS forecasting, refinement, and generation.
2 Background and Related Works
Denoising diffusion probabilistic model (DDPM). DDPM 12 is a well-known diffusion model where input x" is corrupted to Gaussian noise during the forward process and x" is denoised from xJ during the backward process with total T diffusion steps. For the forward process, x' corrupted
from x 1-1 iteratively with Gaussian noise of variance 3, E [0, 1]:
(x")
x x)= - N (x': - ...).
Bix-=1
1=
Using property of Gaussian transition kernel, the forward process of multiple steps can be written as (x') x²) = N (x'; vārx", (1 - āτ) I). where āg =I a, and a = 1 B. For the backward process, x is denoised from x² by sampling from the following distribution:
Po (x'-1 x²) = N (x"-1: (x',t) Eg (x',t))
where 1áo (x', ,xt,) 1) is defined by a neural network and Eg (x', 1) is usually fixed as IfI. DDPM formulates this task as noise estimation problem, where predicts the noise added to x'. With the predicted noise co (xe, .t), no (x', .) can be obtained by
1 Vou
1 ar VI- at Eo (xt (x²:1))
ps (x²,)=
x"
and Eo is optimized using C. = = [Ile-ea Eo (x', [)]] as a training objective.
Unconditional TS diffusion models. Unlike most TS diffusion models which are conditional models [26] 38
1 30 [6 29 40 19), unconditional TS diffusion models do not use conditions (i.e. information of the observed values x(h.) as explicit inputs during the training stage. Instead, they utilize them as guidance during the inference stage through seff-guidance mechanism. The backward process of unconditional models with the self- self-guidance term can be expressed as
Po (x'-1 x², xXa..) = N' px (x', t) + + x! log Po (x4 |x'),,,)) ,T,
where s is the scale parameter controlling the self- guidance term. As the self- guidance mechanism helps avoid the need for architectural changes depending on the condition or task, we apply our method to TSDiff [5), which is the SOTA unconditional TS diffusion model. However, our method not limited to the unconditional model; application to a conditional model, such as CSDI 34 is
also discussed in Appendix M

---

w/o ANT (a)
Stop 0%
Step 2015
Stop 40%
Non-etadoonnrigg
Linear (Ideal)
(c)
Performance (CRPS) (%)
d.
0:
+ 25.1%
? 4.1%
2010
2000
Stop 60%
Stop 90%
Stop 100%s
1.n
Lineer (Ideal)
Datanet
20.8
a.
%f Step
Non-stationarit
ca
txxchang
Hecuiitty
1500 TSD/f--0
w/ ANT
Step US
Step 20%
A1FS
Non Non-atattoonarit
Stap 60%
2000
20
Step 80%
Step 100%
2000
0.2
2.
20
60
100
Step
Step
Figure 2: Overall framework of ANT. (a) shows that base schedule (w/o ANT) abruptly corrupts TS at the earlier diffusion step. while a schedule proposed by ANT gradually corrupts it until the final step. (b) visualizes the son-stationarity curves of both schedules and their discrepancy from a linear line, with the schedule that gradually decreases the non-stationarity (w/ ANT) showing lower discrepancy. (c) shows that better performance is achieved as the curves get closer to a linear line.
Noise schedules for diffusion models. The choice of the noise schedule is crucial for the performance of diffusion models [4], and various methods have been proposed in the computer vision domain [12] including cosine schedule [24] to maintain the noise level at each step consistent, and rescaling method 21 to achieve zero signal-to-noise ratio (SNR) at the terminal step to address the misalignment between the training and inference stages. However, these methods are handcrafted for each dataset, without considering the characteristics of TS datasets. Moreover, recent TS diffusion models overlook the importance of noise schedules, treating them merely hyperparameters, which motivates us to develop a noise schedule specifically tailored for TS datasets using their characteristics.
32 13
Non-stationarity of TS. The non-stationarity of TS indicates how different it is from white noise, and various statisties using autocorrelation (AC) exist to assess this. Integrated autocorrelation time (IAT) [22. defined 7IA lags, Additionally, lag-one autocorrelation (LagIAC) quantifies AC between adjacent observations, and variance of autocorrelation (VarAC) measures the variance of AC over different lags (37). These statistics, which capture the noisiness of TS, inform the speed at which the TS collapses into noise during the diffusion process. In this paper, we propose new statistic based on IAT that captures the non-stationarity of TS, accounting for both positive and negative AC.
2 1x=1 Pi: where Pk is AC lag k. quantifies AC over different
Methodology
In this section, we introduce our main contribution, ANT, an adaptive noise schedule for TS diffusion models. ANT proposes a noise schedule resembling an ideal one that gradually diminishes the non-stationarity of TS as the diffusion step progresses. Subsequently, we investigate the properties of commonly used noise schedules and argue that: 1) The diffusion step embedding (DE) is unnecessary for TS diffusion models when a linear schedule is employed, and 2) non-linear schedules are more robust to the change of the number of diffusion steps than linear schedules in terms of the performance and the scale parameter of the self- suff-guidance mechanism of unconditional diffusion models.
3.1 ANT: Adaptive Noise Schedule for TS Diffusion Models
The overall framework of ANT is illustrated in Figure where we aim to find noise schedule that decreases the non-stationarity of TS on a linear scale. Figure 2 shows how the non-stationarity of a TS gradually decreases with ANT, while it decreases abruptly without ANT. This difference can be explained by computing the con-stationarity curve of a TS for a given noise schedule as shown in Figure ANT proposes noise schedule that minimizes the discrepancy between the ideal linear line and the non-stationarity curve of the schedule, whose r-axis and y-axis represent the progress of the step (%) and the (normalized) statistics of non-stationarity, respectively. As illustrated in Figure 2 reduction in discrepancy with ANT yields better performance compared to without ANT. Statisties of non-stationarity. While various statisties be used to quantify the non-stationarity of TS, we propose the integrated absolute autocorrelation time (IAAT). IAAT is a variation of LAT [22) that takes the absolute value of the autocorrelation to account for positive and negative correlations without canceling them out: TIAXT = 1 + 2 Σ |ps]- Although IAAT shows the best performance, note that ANT is robust across different statistics as shown in Table g
Adaptive schedule. Figure 3 illustrates the non-stationarity curves of various schedules for two datasets [17]. with line width indicating total diffusion steps. The figure reveals that different
3

---

Solar
Exchange
Alinear Linear reduction of non-stationarity
1.
1.0
Unear
MM
M
stationaart
Ideal
Non statioraaatyyy
Signaad (mm0.5)
Cos (1m0.a)
Anoiso Noise collapse
+
2.6
64
04
Ideal
022
Cosine (T+6.5)
Liter
Adlep Sufficient number of steps
gnol [t=0 51
00
94
06
1o
00
5.6
0.1
10
Figure 3: Non-stationarity curves of various schedules.
Figure 4: Desiderata of noise schedules.
Linear schedule
Non-linear schedule
Channel
Channel #2
Channel
Channel
130
P1
Step ratio (UT) 100%
80%
60% 40%
0%
Bietngusstatle differnce
Indeetogulesstii differnnee
(Noise)
(Original)
(a) Confusion matrix of proxy task.
(b) I-SNE visualizations of CNN features.
Figure 5: Proxy task classification & t-SNE visualization.
schedules yield different shapes of curves, and that the same schedule may yield different curve shapes per dataset, highlighting the importance of selecting an adaptive schedule for each dataset.
ANT score. To select an adaptive noise schedule for each dataset, ANT takes a dataset D and a schedule s as inputs and returns the ANT score that measures how well s works on D, with lower score indicating a better schedule. The key component of the score the discrepancy between a linear line 1 and the non non-stationarity curve l,, which is normalized to the range [0, 1] to account for potential differences scales across statistics and datasets. On top of that, as shown in Figure 4 ANT considers two other desiderata of noise schedules: 1) Noise collapse: A noise schedule should corrupt data to random noise at the final step of the forward process to align the training and inference stages 21. 2) Sufficient # of steps: A noise schedule should have sufficiently large T,, as increasing generally improves sample quality [T] To meet the desiderata, ANT opts for schedule with relatively low non-stationarity at the final step ((T) than the first step ( and large T. Then, given D and s, the ANT score is defined as:
ANT(D, 8) Alinear Anoise
Astep
where Aiinaar = d(I", (i(I*,[x) [4), Amoise =1 1 /1 and Xtep = 11 1/T are the terms considering linear reduction of non-stationarity, noise collapse, and sufficient steps, respectively, and is discrepancy metric between the two lines. Among various metrics, we use the difference in the area under the curve (AUC) using the trapezoidal rule, as empirically performs the best in our experiments. However, we note that ANT is robust across different metrics d, as shown in Table The pseudocode for calculating the ANT score is described in Appendix
3.2 Diffusion Step Embedding for TS Diffusion Models
Diffusion models take the t-th step data (x') and the step number t as inputs, where is typically encoded in a DE. However, we argue that DE is not necessary for TS diffusion models employing linear schedule, because information about the step is inherent in the data. To validate our claim, we conduct two experiments: 1) A proxy classification task to predict t with x", and 2) visualization of the embeddings of x" with various
Proxy classification. We design a proxy classification task that identifies the step number in the forward diffusion pass given a noisy TS. For the task, we sample multiple subseries from M4 (2) each with varying steps of up to 100 steps. We build a ID convolutional neural network (CNN) classifier with the architecture of [ConviD
ReLU Flatten Linear], where Conv1D has the kernel size of 3 and 4 output channels. Figure 5a shows the results in confusion matrices, indicating that TS corrupted using a linear schedule retains step information (high accuracy), whereas TS corrupted using a non-linear schedule does not (low accuracy), making DE redundant for a model employing a linear schedule. We argue that this due to the variance of noise for each step (Ba), as non-linear schedule adds most of the noise at the end of the steps, making TS less distinguishable for most of the steps. In contrast, a linear schedule that gradually increases 13, makes TS more distinguishable across steps, allowing to eliminate the DE in the model.

---

Coefficient of varlation (o/ju)
T= 10
T= 50
100
I'E 100)
Linear
1EEE SORRER
0.3
Exchange
Cosine
0.00
Linear
Signnid
Cosine
(a) Robust CRPS to I.
(b) Robust -tattiiionartty curves T'
Figure 6: Robustness of performance to T. (a) shows that CRPS of non-line con-linear schedules are robust to T, supported by (b) which shows the robustness of non non-stationarity curves to T.
Optimal
Linear
Sigmoid
Cosine
Sigmeid
Exchwnng
6.2
2.
UberTTC
08:10
01:100
2.5
(3:3:44
12:3.8
(2:3.4
2.0
075.8.33
(7.2.4)
(220 11.81
0200 3.00
033.
1.0
0.5
of xlape (T)
of stage (T)
cfl steps (T)
(Region) R: Relative ratio of posterior variances
of steps
(b) E of of various schedules. Figure 7: Robustness of optimal T. (a) shows that optimal of non-linear schedules are robust to T, supported by (b) which shows the robustness of sum of the posterior variances to T.
(a) Robust optimal to
t-SNE visualization. (igure55bjeppcee the t-SNE visualizations CNN features extracted from the proxy classification model, representing noisy TS at various steps corrupted by a linear schedule. The results show directional point movement across output channels as the step progresses, implying that diffusion step information is inherently present in the TS. As discussed in Appendix PP t-SNE visualizations employing a non-lincat schedule also show directional point movement but with common pattern across all output channels, indicating limited step information compared to linear schedule. Based on these observations, we remove the DE from the diffusion models when employing linear schedule. In experiments, observe that DE is useful for models with non-linear schedules, but not for models with linear schedules, as shown in Table 6
3.3 Robustness of Non-linear Schedules
In diffusion models, the number of steps T determines the efficiency of overall process and sample quality. We argue that non-linear schedules are more robust to ' than linear schedules in two aspects: 1) performance and the optimal scale parameter s controlling the sefdd-uidnnce in Eq. (4).
Robustness of performance to T. Figure 6a depicts the coefficient of variation of continuous ranked probability score (CRPS) [ of forecasting task with various T, illustrating that the performance is robust to T when employing a non-linear schedule. This is supported by Figure 6b) which shows the non-stationarity curves of various schedules, with each color representing different dataset. The figure indicates that the shape of the curve, which determines the ANT score, remains consistent regardless of T' when employing a non-linear schedule. It is important note that the ANT score is highly correlated with the performance, as discussed in Section 4.4
Robustness of optimal to T. Figure 7a shows that the optimal is sensitive to T when employing linear schedules, whereas robust when using non-linear schedules. We discover that this robustness stems from the posterior variance (a?) of schedule, which affects the self-guidance term in Eq. (4). Figure 7 illustrates the sum of the posterior variances ( of) of various schedules, indicating that this sum is more robust to the choice of T for non-linear schedules compared to linear schedules.
4 Experiments
Experimental setup. We demonstrate the effectiveness of our proposed method three tasks: TS forecasting, refinement, and generation. For evaluation, we adopt ANT to TSDiff (15), where we use the official code replicate the results. All experimental protocols adhere to that of TSDiff, where
utilize the CRPS 99] to assess the quality of probabilistic forecasts. We conduct experiments on eight TS datasets from different domains Solar [17), Electricity (3], Traffic [3), Exchange [17,, M4 (23).
UberTLC (7]. KDDCup 10. and Wikipedia [ We present the mean and standard deviations calculated from three independent trials.
5

---

Mathod
Solar
Electricity
Traffic
Exchange
M4
UherTLC
KDDCup
Wikipedia
DeepAR
40.00
0.054-0000
4.000
0.0
0.161
0.4142002
4..31
DeepState
0.379 00
0.07520004
0.146g9.00
2.011-0m
0.041
g.288
0.318 010
Transfomerr
0.419g..000 oea
0.76
0.1022..00m
0.000-0000
0.04022.044
0.1922
0.411.00m
0.214 g9 20
TFT
0.417
0.086....cs
1.134
0.007
0.039
a.
0.581
0.22%
CSDI
0. 0.352ga 1os
0.005
0.150go oon
0.033
2.0
0.206ga.00
0.318
0.289 g0.00
TSDift
0.399 g0.00
0.049
0.105
2.012gmmm
0.336
(0.221
TSDIFI
0.326 10.000
0.047.00.00 10.000
0.101 ooe
0:009 20.000
0.026 L oon
0.1633
0.325 on0
0.0
Gaia (%)
22.4%
44.1%
33.9%
26.15
27 8%
5.2%
13.0%
6.8%
(a) TS forccasting task: Standard horizon H.
TSDiE
Solar
Truffic
Electricity
Exchango
M1
UberTLC
KDDCup
Wikipodia
1/4
000
10000
0.033_..000
0.158,n.om oon
0.275_...00
0.18210.000
w/o ANT
1/2
2
0.184 000
2.359
0.097
ao.co
0.047 10000
0.062
1.012
0.01
0.058
0.01....c0
0.157 10.002
2.196
0,452 10:003
0.5022n.004
0.241 20.000
0.189 10.000
4
0.38310000
0.1 0.0
0.05610000
0.015_0000
0.046.0.800
0.264_0.00
0.406200.0
0. 0.277.0.00)
1/4
0.041
0.006_.
0.025
0.162_a.00
0.164 a.
w/ ANT
1/2
0.356 ono
0000
0.098 0.000
0. sa.cco
0. 1..01:0.000
0.0443.0.000
0.009 00.000
0011::0.00
0.040:...00 0.000
0.028_.o.0m
0.184:2.002 000
0.162_...0m
0.497
0.247 10.000
n.235.0.000
0.000
0,376
0.103
0. 0.053:0.000
0,0251umac
0.043:0.001
0 0.25922.008
0.45332.00z
0.205
(b) TS forecasting task: Various horizons H.
Table 2: Results of TS forecasting.
TSDif'
Refinement
Electricity
Traffic
Exchange
M4
UberTLC
KDDCup
LMC-MS
0.49420.015
0.050=0004
0. 0.113.0.c01
0.013
0.040
0. 0.187
0.458
LMC-Q
0.516.0.000
0.11940.00 0.050
0.009
0.03422000:
0:22840.010
0.34620010
w/o ANT
ML-MS
0.503 10.80.00
0.063 Loocs
0.117 10.002
0.015 1
0.045 Loor
1.208
0.472 in0ss
ML-Q
0.5233.0.0
0.856ga0cc
0.121
0.010g0.01
0.03220.0m
0.240g0.000
0.350g0011
Avg.
0,509
0.058
0.118
0.012
0.038
0.215
LMC-MS
0.447=0.012
0.055
0.1 0.118
0.009.20.00
0 035
0.1800.0.01
014
LMC-Q
0.495 nes
0.054
0.115
0.009
0 033 ooo
0,186 ..0m
0.364
w/ ANT
ML-MS
0.44920.000 oos
0.058 o0o
2.11520.00
0.010=0.000 000
037 oce
0.1822.0.01
0.419
ML-Q
0.49920.000
0.056i
0.115:0.0oz
0.010 20.000
0 08-120 oce
O. 0.187
0 0.367
Avg.
0.472
0.056
0.115
0.009
0.035
0.184
0.389
Gain (9)
+7.3%
+3.5%%
+2.0%
25.0%
+7.9%
+14.4%
+4.4%
Table 3: Results of TS refinement.
Noise schedules. ANT finds the best schedule for a given dataset from candidate schedules based on the ANT score. Table candidate schedules that we use for our experiments based on the three components: noise function (f), temperature (7), and the number
1)
2) T
3)
Lincar (Lin)
110,20,50,55.100
Cosine (Cos)
[0.5.1.02.00] [0.3,0.5.100
[10.20,50,75 21.20,50,75,100 (10,20,50.75.100)
Sigmoid (Sig)
Table 1: Candidate schedules.
of diffusion steps (T)). We denote f(T") for a linear schedule and f(T, r) for a non-linear schedule. Note that TSDiff 15 employs Lin(100) as a common
schedule for all datasets, following TimeGrad (26).
4.1 Time Series Forecasting
For baseline methods in forecasting tasks, we compare our method with various approaches, includ- ing DeepAR AR (28), MQ-CNN [36], DeepState (25). Transformer (35) TFT (20). CSDI [4. and TSDiff 15). We note that we do not aim to outperform all SOTA methods, but rather to demonstrate the efficacy of our proposed adaptive noise schedules when applied within the framework of TS diffusion models. Nonetheless, Table indincaes that TSDiff employing the noise schedule proposed by our method achieves the best performance on all datasets.
To show the effectivvness of our method across various prediction horizons, we conduct an experiment with different horizons using the same input window. Specifically, we multiply a by the standard horizon H used Table 2a where the range of a depends on the sequence length of the dataset. Table 2b shows that ANT enhances TSDiff with variable prediction lengths.
4.2 Time Series Refinement
We conduct an experiment refining the forecasting results of other base forecasters to evaluate the quality of the implicit probability density of TSDiff applied with ANT. For the experiment, we employ Gaussian (MS) and asymmetric Laplace negative log-likelshoods (Q) as regularizers for both energy (LMC) and maximum likelihood (ML)eb refinement, following TSDiff. Table 3 displays the results using a linear model as the base forecaster, showing that TSDiff, when applied with our method, yields performance gains across all datasets and refinement methods.

---

Generator
Solar
Traffic
Exchange
M4
UberTLC
KDDCup
Wikipedia
VAE
19.33
0.024g0.004
0.074go.00s
0.354 one
1.00 179
0.643
TimeGAN
1.140
0.308
0.011 000
0.140g0.053
0.665 104
0.713 oo9
4.421
Lanear
TSDift
0.597
00.181 g00000
0.012
g.045go0007
0.291
0.504
0.392
TSDiIF+NTT
0.533 on0
0.177
0.177:0.000
20.00000
0.012
0.040;0.000
0.232 4.
0. 0.426 one
0.350..0.00
Gain (%)
10.7%%
+..2%
+0.0%
+11.1%
+ 20.3%
+ 15. 5%
+10.7%
VAm
0. 0.493 10.022
0.155g0.006
0.009
0.039
0.278.10.00
0.62110.000 000
g.000
DeepAR
TimeGAN
g.
2.4
0.008g0.00
0.121 go
0.594 195
0.690 con
go.00s
TSDiff
0.478 o0
0. 00.40go.ou
0.02910.017
g.002g0002
0,268 00.007
=0.416
0.225
ISDiff+ANT
0.42400.006
20.00.
0.01300.000
0..3000.000
0. 0.185
0 0.343 012
0. 0.240
Gain (5)
-1.3%
+0.7%
+55.2%
+21.4%
+300 1%
+ 17.5%
6.7%
VAE
0 2.0000
0.16320.a1
0.011
0.035g0.011
0,291 20.008
0.717
0.451
Transfommed
TimeGAN
0.972
0.41320
0.114go.052
0.685 eos
0.314
TSDiff'
0.437 20.000
+0 0.153
0.03110.000
0.069 co.00s
0.272:0.018
1.003
ISDifI+ANT
0.444 002
0.148
0.01 400.00
0.039
0. 0.225
0.366 ooe
0.234_..0m
Gain (%)
+2.8%
3.3%
+ +54.8%
+43.5%
+17.3%
-8.0%
- -2.1%
Table 4: Results of TS generation.
Solar
Electricity
Traffic
Exchange
M4
UberTLC
KDDCup
Wikipedia
Arg.
Schedulle
w/o ANT
Lin(100)
ISDiff
ANT
Lin(50)
Lin(100)
Cos(75,2.0)
Cos(50,0.5)
Cos(100,1.0)
Lin220
Lin(50)
Cos(75,2.0)
Orncle
(os(75,2.0)
wdo ANT
0.399
0.049
0.105
0.012
0.036
0.172
0.335
0:221
0.166
CRPS
TSDiff
w/ ANT
0.101
0.150
0.326
0.009
0.163
0.325
Ornele
0.099
0.150
Table 5: Schedules proposed by ANT and forecasting results with them.
4.3
Time Series Generation
To assess the generation quality of TS diffusion models with ANT, we evaluate the forecasting performance of downstream forecasters trained on the synthetic samples generated by various models. For the baseline methods, we employ TimeGAN (39], Time [3], and TSDiff [1] For the downstream models, we employ a simple linear (ridge) regression model, DeepAR 22 and Transformer [35]. The results are shown in Table 4 indicating that ANT improves the performance
of TSDiff, leading to SOTA performance in most cases. Further details regarding the generation task
with Electricity are discussed in Section O
4.4 Analysis
Proposed noise schedule. J the schedules proposed by ANT and the schedules that yield the best performance (oracle) among all candidate schedules, along with the forecasting performance obtained when using these schedules. The result indicates that the schedules selected by ANT are mostly consistent with the oracle, and even for an exceptional case that ANT fails to select the best schedule, e.g., on Traffic, the performance gap from the oracle is not significant.
Necessity of DE. Table ( displays the forecasting results of TSDiff applied with ANT. both with and without the use of DE across eight datasets. The result indicates that for all datasets except for Traffic, models trained with a linear schedule perform better when trained without DE, while models trained with non-linear schedule perform better when trained with DE. Based on these findings, we do not use DE when employing a linear schedule.
TND
Dataset
TSDiff
DE
UberTLC KDDCup Traflic
20
0.163
0.169
0.172
Linear
0.335
50
0.101
0.095
0.105
Solar
100
0.336
0.399
0.399
Exchange Electricity Wikipedia
Cosipe
2.0
75
0.050
0.010
0.047
0.049
2.0
75
0.206
0.206
0.221
M4
1.0
100
0.052
0.026
0.036
Ablation study. To show the effectiveness of each com- ponent in ANT, we conduct an ablation study in Table The result demonstrates that enforcing the schedule to gradually drop the non -taatioooonartt is the most important component (Aumear), and ensuring noise collapse (Aaoise) and sufficient steps (Aatep also help ANT achieve oracle for three datasets 23 the correlation between the ANT score and CRPS on forcasting tasks via a scatter plot of candidate schedules and a bar plot of schedules selected in Table implying that there a strong correlation between the ANT score and CRPS, and the components ANT strengthen the correlation further.
Table 6: TS forecasting w/ & w/o DE.
when combined together. Figure g shows
7

---

ANT companents
Solar
Electricity
Traffic
Exchange
M4
UberTLC
KDDCup
Wikipedia
Aisert
Araess
Aarp
Lin(75)
Cost 10.10)
Lin(20)
Cos(50,0.5)
Cos(10.0.5)
Lin(20)
Lin(50)
Cos(75,2.0)
Lin(75)
Cos( 10,
Lin(20)
Cos(50,0.5)
Cos(10.0.5)
Lin(20)
Lin(50)
Cos(75,2.0)
Lim(75)
10000)
Lin(20)
Co.(50,0.5)
Cost 10,0.5)
Lin(20)
Lin(50)
Cos(75,2.0)
Lin(100)
Cos(75,2.0)
Lin(50)
Cos(50,0.5)
Cost (0s(10011.0)
Lin(20)
Lim(5))
Cos(75,2.0)
Oracle
Lin(100)
Cos(75.2.0)
Cos(75,2.0)
C0s(50,0.5)
Cos(100.1.0)
Lin(20)
Lin(50)
Cos(75,2.0)
Table Ablation study. The table shows the schedules proposed by ANT when using some or all components of the ANT score.
KDDCup
0.38
M4
Solar
0.25
UberTLC
KDDCup
Paresse
0.25
0.45
0.50
6.65
0:36
0.20
0.40
5.43
0.25
0.15
0.25
2.53
0.30
0.25
0.2
0.3
Aliocar
Astep
Anoise
+>step
Anoine
AMT score
(a) ANT score CRPS.
(h) Correlation between ANT score and CRPS.
Figure 8: Ablation study. (a) depicts the scatter plot of ANT score and forecasting result of various schedules, where a schedule with a lower score yields better performance in general. (b) shows that using all three components of ANT score results in the highest correlation.
Avg.
Generation
Non-stationarity
Avg.
w/o ANT
0,166
TSDI
w/o ANT
0.166
MSE
0.152
VarAC
0.158
MAE
0.151
w/ ANT
LaglAC IAAT
0.151
w/ ANT
Cor:.
0.151
AMT 15900
0.150
R 0.152
AUC
0.150
Oracle
0.150
Oracle
0.150
Table 8: Robustness to statistics.
Figure 9: Generation process.
Table 9: Robustness to d.
Robustness statisties. To see ANT is sensitive to the choice of statistics of non-stationarity. compare IAAT with other statistics including LaglAC and VarAC. Table 8 shows the average CRPS across eight datasets in forecasting tasks, demonstrating that ANT with different statistics still outperforms the model without ANT, while IAAT performs best. We conjecture that this is because IAAT considers AC at all time lags, whereas LaglAC only considers a single lag and VarAC focuses on the variance of AC rather than their exact values.
Robustness to d. Among the various metrics that can be employed for d to measure the discrepancy between the two lines, compare AUC with the mean-squared error (MSE), meca-abbolute error (MAE), Pearson correlation (Corr.), and R-squared (R²). Table presents the average CRPS of eight datasets on the forecasting task, demonstrating the robustness of our method the choice of metric.
Generation process. Figure 9 illustrates the generation process of a TSDiff model trained on M4 with and without ANT, where the color changes from black to red as the step progresses. The figure on the left shows the generated TS for each step, while the one on the right shows the non-stationarity of each step in terms of IAAT. As shown in Figure 9 ANT indeed makes the generation process linearly increase the non-stationarity.
Effect of I on performance. While it was believed that the larger the T, the better the performance diffusion models (31. we argue that this not always true in the TS domain. Table 10 shows the oracle performance for each T among candidate schedules on forecasting tasks, indicating that the best performance is not always achieved with the largest e-g.. the best performance is achieved with T - 20 on UberTLC (7]. Furthermore, as shown in Figure TO, ANT often outperforms the baseline method TSDiff with smaller T, achieving the oracle performance in most cases.
Efficiency of ANT. ANT offers efficiency in two aspects: 1) Statistics of non-stationarity can be precomputed offline with small additional cost, and 2) the proposed schedule with 7' smaller than that of the base schedule results in faster inference. First, Table diaislas the time spent calculating IAAT using the entire train dataset of Traffic with schedules of five different values for T. The results indicate that the computational cost is negligible compared to the training time shown in Table [Tb making ANT practical for use. Second, Table shows the training and inference time on Traffic, where we report the training time with 1000 epochs and the inference time for a TS data averaged on the test dataset. The results demonstrate that we can save up to 50% of the inference time and reduce the training time by climinating the DE.

---

ANT Oracle (per vs. Base
Solar
Electriary
Truthe
Exchange
UberTLC
KDDDClap
Wikipedia
TSDiff
0.399
0.019
0.105
0,012
0,036
0.172
0.535
0.221
TSDIE++ANT
0.326
0.047
0
0.009
0.026
0,163
0.325
0.206
10
0.321
0:062
0.127
0.011
0.990
0.190
Onalie
20
0.382
0,050
0,103
0,010
0.031
0.163
0.306
0.210
50
0.330
0.042
0.100
0.009
0.027
0.165
0.325
0.207
75
0.351
0.047
0.899
0.010
0.028
0.189
0.314
0.396
100
0.326
0.048
0.101
0.010
0.026
0.189
0.368
0.208
0.5
Table 10: Oracle (lowest CRPS) under fixed T.
Reletive ratio (flestoo, Worst
Figure 10: ANT vs. Oracle.
Dataset: Traffic
Daraset: Traffic
Avg.
Train (mina
Inference (5oc)
w/o ANT
100
0.166
Time
T
11.1
10
22.6
20
60.2
50
93.2
75
120.0
100
w/o ANT
100.5
92.0
WAT
weo
1.28
w/ ANT
0.64
ANT
w/O con.
w/ con.
2 100
50
0.157
0.150
(a) IAAT calculation time (sec).
(b) Train & inference time.
(c) With constraint on T.
Table 11: Efficiency of ANT.
UberTLC
Traffic
w/o Cos"
w/ Cos"
w/o Cos"
w/ Cos"
A:
Schedule
Lin(20)
Cos" (20,0.2)
Lin(50)
Cos' (20,2.0)
ANT score
0.054
0.053
0,088
0,050
CRPS
0.163
0.161
0.101
0.099
Table 12: Flexibility of ANT.
Figure 11: 3e of Cos*.
cost trade-off in that it can be optimized with a constraint on T by selecting a schedule among candidates with T no larger than T (T I T*). Table shows the forecasting results with a constraint of T = 50, achieving better performance with no more than half of the steps of the base schedule. Further analyses regarding computational time and forecasting results under constraints on T are discussed in Appendix [ respectively.
Furthermore, ANT offers a performance-computane
Flexibility of ANT. As ANT does not require direct access to the noise schedule but the TS data corrupted by the schedule, any noise schedule can be candidate. To confirm if ANT can judge the usefulness of non-trivial noise schedules other than the functions described in Table [ we experiment an ensemble of cosine functions (Cos"), illustrated in Figure IT] Table 12 shows the results on two datasets 7 3 that achieve lower ANT scores with Cos", indicating the potential for a better schedule to be selected by relaxing structural constraints, or even without any functional form S.
Comparison with other schedules. To demonstrate the effectiveness of ANT compared to noise schedules from other domains, we compare ANT with a cosine schedule (Cos) (24] and schedule that enforces zero terminal SNR (Zero) [1]] applied to TSDiff. Table [3] shows the average CRPS across eight datasets on forecasting tasks, where ANT outperforms the others. The results highlight the importance of schedule being adaptive to the given TS dataset and considering the corruption speed.
Schedule
Avg.
Linear
0.166
Cos 24
0.160
Zero (2
0.185
ANT
0.150
Table 13:
Comparison
with other schedules.
5 Conclusion
In this work, introduce ANT, a method designed select adaptive noise schedule for TS diffusion models using the statistics of non-stationarity. ANT is practical in that it predetermines a noise schedule before training, as the statistics can be precomputed offline with minimal additional cost. The proposed ANT is a simple yet effective method for TS diffusion models across various tasks, e.g., ANT improves the SOTA method TSDiff by 9.5% on average across eight forecasting tasks, highlighting the importance of adaptive schedules tailored to the characteristics of TS datasets. Limitations and future work. We note that our contribution is not in finding the optimal schedule but in proposing a criterion for efficiently selecting a better noise schedule from a set of candidates based on the dataset characteristics. While our experiments search for appropriate schedule from 35 candidates derived from linear, cosine, and sigmoid functions, any schedule can be considered a candidate for ANT, and incorporating additional schedules could lead further performance gains at the expense of longer search times. We leave developing algorithms to determine the optimal schedule parameters based on our proposed criterion for future work. We hope that our research motivates further research across various domains to incorporate domain knowledge in the design of machine learning models, extending beyond noise schedules for diffusion models.

---

Acknowledgements
This work was supported by the National Research Foundation Korea (NRF) grant funded by the Korea government (MSIT) (2020R 1 A2C1 A01005949, 2022R1 A4A 10333443 35-2023-002177003 RS-2024-00341 749). the Miniryin of Science and ICT), Korea, under the ICAN(ICT Chal- lenge and Advanced Network of HRD) support program RRS-2023-00259933 UTP(Instiiute for Information & Communications Technology Planning & Evaluation), the Yonsei University Research Fund (2024-22-0148), and the Son Jiho Research Grant of Yonsei University (2023-22-0006).
supervised by the
References
[1] Juan Miguel Lopez. Alcaraz and Nils Strodthoff. Diffusion- time series imputation and forecasting with structured state space models. TMLR, 2023.
[2] Alexander Alexandrov, Konstantinos Benidis, Michael Bohlke-Schneider, Valentin Flunkert, Jan Gasthaus, Tim Januschowski, Danielle Maddix, Syama Rangapuram, David Salinas, Jasper Schulz, al. Gluonts: Probabilistic and neural time series modeling in python. Journal of Machine Learning Research, 21( 16): 2020.
[3] Arthur Asuncion and David Newman. Uci machine learning repository. 2007.
[4] Ting Chen. On the importance of noise scheduling for diffusion models. arXiv preprint a7X1v:2301.10972, 2023.
[5] Abhyuday Desai, Cynthia Freeman, Zuhui Wang, and Ian Beaver. Timevae: A variational auto-encoder for multivariate time series generation. arXiv preprins 1..0092/11.08095, 2021.
[6] Xinyao Fan, Yueying Wu, Chang Xu, Yuhao Huang, Weiqing Liu, and Jiang Bian. Mg-tsd: Multi- Muatilgramularrtty time series diffusion models with guided learning process. In ICLR, 2024.
[7] FiveThirtyEight. Uber tle foil response. hutt:s///gituubb.coom///evttttty fooo-ttc--oil-respooe 2016.
[8] Jan Gasthaus, Konstantinos Benidis, Yuyang Wang, Syama Sundar Rangapuram, David Salinas, Valentin Flunkert, and Tim Januschowski. Probabilistic forecasting with spline quantile function mns. In AISTATS, 2019.
[9] Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477):359-3777 2007.
[10] Rakshitha Godahewa, Christoph Bergmeir, Geoffrey Webb, Rob J Hyndman, and Pablo Montero-Manso, Monash time series forecasting archive. arXiv preprint arXi::2105.06643; 2021.
[11] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(1 1):139 144, 2020.
[12] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurlPS, 2020.
[13] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In NeurlPS, 2022.
[14] Diederik P Kingma and Max Welling, Auto-encoding variational bayes. arXiv preprint 13X1v:1312.6114. 2013.
[15] Marcel Kollovieh, Abdul Fatir Ansari, Michael Bohlke-Schmeider, Jasper Zschiegner, Hao Wang, and Yuyang Bernie Wang. Predict, refine, synthesize: Self-guiding diffusion models for probabilistic time series forecasting, NewrIPS, 2023.
[16] Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: versatile diffusion model for audio synthesis. In ICLR, 2021.
10

---

[17] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The slst international ACM SIGIR conference on research & development in information retrieval, pages 95-104, 2018.
[18] Yan Li, Xinjiang Lu. Yaqing Wang, and Dejing Dou. Generative time series forecasting with diffusion, denoise, and disentanglement. In Neur!PS, 2022.
[19] Yuxin Li, Wenchao Chen, Xinyue Hu, Bo Chen, Mingyuan Zhou, et al. Transformer-modulate diffusion models for probabilistic multivariate time series forecasting. In ICLR, 2024.
[20] Bryan Lim, Sercan Ank, Nicolas Loeff, and Tomas Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37(4):1748-177 1764, 2021.
[21] Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang. Common diffusion noise schedules and sample steps are flawed. In WACV, pages 5404 5411, 2024.
[22] Neal Madras and Alan D Sokal. The pivot algorithm: A highly efficient monte carlo method for the self-avoiding walk. Journal of Statistical Physics, 50:109-186 1988.
[23] Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36(1))54- 74, 2020.
[24] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, 2021.
[25] Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang. and Tim Januschowski. Deep state space models for time series forecasting, In NeurlPS, 2018.
[26] Kashif Rasul, Calvin Seward, Ingmar Schuster, and Roland Vollgraf. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In ICML, 2021.
[27] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In ICLR, 2022.
[28] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181-1191 2020.
[29] Lifeng Shen, Weiyu Chen, and James Kwok. Multi-resolution diffusion models for time series forecasting, In ICLR, 2024.
[30] Lifeng Shen and James Kwok. -on-autoreeresssvve conditional diffusion models for time series prediction. In ICML, 2023.
[31] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML, 2023.
[32] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021.
[33] Gabriel Stoltz, Mathias Rousset, et al. Free energy computations: mathematical perspective. World Scientific, 2010.
[34] Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffusion models for probabilistic time series imputation. In NeurIPS, 2021.
[35] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS. 2017.
[36] Ruofeng Wen, kari Torkkola, Balakrishnan Narayanaswamy, and Dhruy Madeka. multi- horizon quantile recurrent forecaster. In NewrIPS, 2017.

---

[37] Annette Witt, Jürgen Kurths, and A Pikovsky. Testing stationarity in time series. physical Review E, 58(2):1800, 1998.
[38] Tijin Yan, Hongwei Zhang, Tong Zhou, Yufeng Zhan, and Yuanqing Xia. Scoregrad: Multivari- ate probabilistic time series forecasting with continuous energy-based generative models. arXiv preprint N.Ni::2006.112...0 2021.
[39] Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. Time-series generative adversarial networks. In NeurIPS, 2019.
[40] Xinyu Yuan and Yan Qiao. Diffusion-ts: Interpretable diffusion for general time series genera- tion. In ICLR, 2024.
12

---

A Dataset Description
In our experiments, we employ eight widely-used univariate datasets from various fields which can be found in Gluon'TS [2] in their preprocessed form, with training and test splits provided. Following TSDiff [15], the validation set is created by splitting a portion of the training dataset, with the split ratio determined by the sizes of the training and test datasets. Table Shlles..Ishhoss the statistics of the dataset, annotated with their corresponding frequencies (daily or hourly) and lengths of predictions. For the daily datasets, the sequence lengths are hourly datasets, we utilize sequence lengths of 360, corresponding to 15 days. These sequences are created by randomly slicing the original time series various time steps. Below, we provide a brief description of each dataset.
390, roughly equivalent 13 months. For the
This dataset contains data on solar power production in 2006 from 137 solar power plants located in Alabama.
Solar 17
Electricity 17 3
This dataset includes electricity consumption data from 370 customers.
Traffic 17 This dataset features hourly occupancy rates on the freeways in the San Francisco Bay area from 2015 2016.
Exchange [17] This data is comprised of daily exchange rates from eight countries-Australl the UK, Canada, Switzerland, China, Japan, New Zealand, and Singapore- from 1990 to 2016.
M4 23
This is hourly data subset derived from the M4 forecasting competition,
UberTLC 77 This dataset includes data Uber pickups in NYC from January to June 2015, collected by the NYC Taxi & Limousine Commission.
KDDCup [00: This dataset consists air quality indices (AQIs) for Beijing and London, used in the KDD Cup 2018.
Wikipedia g This dataset contains daily number of hits on 2000 Wikipedia pages.
Dataset
Narain
Niest
Domain
Freq.
Median Length
L
H
Solar
137
959
R
H
7009
336
24
Electricity
370
2590
R
H
5833
336
24
Traffic
963
6741
(0, 1)
H
4001
336
24
Exchange
8
40
R
D
6071
360
30
M4
414
414
N
H
960
312
48
UberTLC
262
262
H
4320
336
24
KDDCup
270
270
N
H
10850
312
48
Wikipedia
2000
10000
D
792
360
30
Table A.1: Statistics of datasets.
Soolar; http://www...aanoooo/o//////olor
*Traffic: http:///ss..aaoooooo//////2/2331
'Exchange: http:////hhhcccoooooooaaaaaaaaaa MM4 ha4k:p///.hhcccckkaaaataa///aaar
'UherTLC: http:h//hhhh..tt.o..////aaaaaaaa $ *KDDCup: http://///./aooo/////////////aa.
WWikipedia:
13

---

B Experimental Settings
Metries, Following TSDiff, we employ the Continuous Ranked Probability Score (CRPS) as our evaluation metric. It is defined by the integral of the pinball loss over the interval from 0 to
CRPS(F = /. 2A, (F y)
(B.1)
where (g, v) (x - (y denotes the pinball loss for a given quantile level K, F-1 denotes the predicted inverse cumulative distribution function, and denotes the observed value. Since the quantile function is not analytically accessible practice, it is approximated using discrete quantile levels derived from sample forecasts. For the experiment, we utilize the CRPS implementa- tion in GSuonTS (2). which uses 100 sample forecasts to calculate nine predefined quantile levels {0.1, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9).
Hyperparameters. Table |.I presents the hyperparameters of the backbone model utilized in our experiment, which are aligned with those used in TSDiff. Note that the diffusion step embeddings 35 are only applied to the model employing non-linear schedule. Moreover, we employ skip connec- tions for certain datasets, following the TSDiff approach, which results in improvements in validation performance. For the normalization method, we employ the mean scaler from GluonTS [2]. which normalizes each TS by dividing by the mean of the absolute values of its observations. The scale parameter which controls the self-guidance term is searched over the form 2 where a ranges through the set [-1, 1, 0, 1, 2, 5, 7|.
Hyperparameter
Value
Learning rate Optimizer
le-3
Adam
Batch size Epochs
64
1000
Grad. clipping threshold
0.5
Residual layers/channels
3/64
Dim of DE
128
Normalization
Mean-scaling
Self-guidance
Quantile
Solar
4
Electricity
8
Traffic
4
Exchange
32
scale
M4
16
UberTLC
16
KDDCup
Wikipedia
8
Table B.1: Hyperparameters.
14

---

C Baseline Methods
For the baseline methods TS forecasting and generation tasks, we adhere to the methods outlined in TSDiff
15
C.1
Methods for TS Forecasting
DeepAR 28
probabilistic forecasting method that outputs the parameters for distribu- tions, optimized with the (conditional) log-likelihood. The model employs RNN-based autoregressive framework that leverages past lags and external features to predict future values.
DeepState (25 integrates RNNs with linear dynamical systems (LDS), where the LDS parame- ters, such as transition and emission matrices, are predefined to reflect trend and seasonality of TS. The parameters of LDS and RNN are simultaneously optimized using maximum likelihood estimation.
Transformer utilizes a self attfntttention mechanism [35 for sequence- -o-nnccee----uueneee forecasting, employing TS lags and additional covariates to predict future distribution parameters.
TFT 220 is a Transformer-based forecasting method that incorporates that employs variable selection mechanism to filter out irrelevant features while training on quantile loss.
CSDI 34 is conditional diffusion model that adapts the DiffWave framework [66 to multi- Variate TS forecasting and imputation, incorporating transformer layers into its architecture.
TSDiff [5 is unconditional diffusion model that utilizes self self-guidance mechanism, that
leverages conditioning information during the inference stage with an unconditional network, without requiring additional architectural changes.
C.2 Methods for TS Generation
Tim utilizes variational autoencoders handle trends and seasonality in TS.
14
with its architecture tailored for TS analysis to
TimeGAN utilizes an autoencoder framework to train a generative adversarial network
(GAN) [ in the latent space, which is optimized with composite loss function that integrates supervised, unsupervised, and discriminative losses.
D TS Diffusion Models
The goal of TS prediction tasks is to predict x E Rdx given x E RR where d, H. L are the number of variables, the length of the target window, and the length of the input window, respectively,
Most of TS diffusion models for prediction tasks are conditional models [26 38 I8
30 I 6 29
40 19], employing conditioning network F construct a condition using x which contains
the information of the observed values used to predict the unobserved values. The distribution for conditional diffusion models is formulated as
Po
miss
0:T' Po
c)
miss
Po
1-1 (xima
X c).
(D.1)
where F (x((m)) and XT (O, The corresponding backward process can be expressed as (x x x c) = = (x no (x t c),
t= T, T.....l I.
(D.2)
15

---

E Pseudocode for ANT Score
Algorithm shows the pseudocode for calculating the ANT score, which evaluates the suitability of
schedules for given dataset.
Algorithm 1 Calculation of ANT score
Input: Dataset D = |= Schedule (schedule function: f. temperature: т, number of steps: T)
for to N in parallel do
2:
x : X;
3:
for f to do
4:
Add noise: x² ~ q (x} x based
5: Calculate non-stationarity: g = Gx{),, where G is non-stationarity metric
6: end for
7: end for
8: Define non-stationarity curve: 4g = ( ( where 18th - g
9: Normalize non-stationarity curve: MinMaxScalo((s))
10: Calculate ANT score: (D, s) = Ainear Anaise Astep
where Aiinar d(l", [q), Ancisc T /16") Astep
1 1/T, LLinspaceeL,,. 0, T)
Output: ANT score: AN)(D,s)
F TS Refinement Methods
TSDiff offers strategy to improve predictions from base forecasters, adaptable any type of base forecaster as it only requires their initial outputs. These initial forecasts are iteratively enhanced through an implicit density learned by the diffusion model, which acts prior. The refinement process can be conducted using one of two methods: (a) sampling from an energy function (LMC), or (b) maximizing the likelihood to determine the most probable sequence (ML). Furthermore, the approach includes an option between two types of regularizers: (a) Gaussian (MS) and (b) Laplace negative log- likelihoodds (Q).
Energy based sampling (LMC). LMC improves the initial forecast of a given base forecaster g. by framing the refinement task of sampling from regularized energy- -waeed model (EBM). This can be expressed as
y) log (y (y= (y)
(F.I)
where y represents the time series formed by combining yobs with g(yoss), R denotes regularizer, and À a Lagrange multiplier controlling the strength regularization.
To sample from the specified EBM, overdamped Langevin Monte Carlo (LMC) B3 is used, with (()) initialized y. The iterative refinement process outlined as follows:
((+1) (()) V ycy Fg ()():)) + 22 where & ~(0,).
(F.2)
where 1) represents the step size, and is the noise scale.
Maximizing the likelihood (ML). The refinement procedure can alternatively be viewed as regularized optimization problem aimed at identifying the most probable TS while adhering to specific constraints on the observed time steps. Formally, this expressed as
arg min |- (o) XR(y,))
(F.3)
which can be optimized using gradient descent.
16

---

G Statistics of Non-stationarity
Integrated autocorrelation time (IAT) [22]. Integrated autocorrelation time (IAT) quantifies the efficiency of an MCMC sampler by estimating the effective number of independent samples produced by the sampler through calculating the autocorrelation (AC) within a chain. A sampler with lower IAT is considered more efficient. Additionally. IAT can be used diagnose the level of noise in TS, as a TS with higher noise levels will exhibit smaller AC in their absolute values. The LAT and its absolute version (IAAT) are defined as
TAX = -1
Pk: TIAAT 1 +
Σ |pal
(G.1)
A=
1=
respectively, where is an AC at lag k. Note that this computation needs to be truncated at practical lag where the ACF values are essentially zero.
Lag-one autocorrelation (LagAC) [37]. LagAC (p1) is a measure of AC that quantifies the correla- tion between adjacent observations TS, which is defined
1=1 (It ()(++11
(G.2)
Σ1=1
Z)²
where 2y and 2++1 are consecutive observations in the TS, 7 is the mean of the TS observations, and is the total number of observations in the TS.
Variance of autocorrelation (VarAC) [37]. VarAC (o)) measures the variance of the AC over different lags, which is defined
Pk - p)
(G.3)
m k=1
where ý is the mean of the AC over m lags, and m is the total number lags considered.
H Robustness to Statistics of Non-stationarity
Table H.I presents the CRPS on forecasting tasks with eight datasets using various statistics repre- senting non-stationariiy. The result demonstrates our method's robustness across various statistics.
Solar
Electricity
Traffic
Exchange
M4
UberTLC
KDDCup
Wikipedia
Avg.
TSDiff
0.399
0.105
0.012
0.036
0.172
0.335
0.221
0.166
VarAC
0.369
0.050
0.102
0,010
0.029
0.169
0.325
0.209
0.158
TSDIFFANT
LayAC
0.326
0.053
0.101
0.009
0.026
0.163
0.325
0.210
0.151
IAAT
0.326
0.047
0.101
0.009
0.026
0.163
0.325
0.206
0.150
Oracle
0.326
0.047
0.099
0.009
0.026
0.163
0.325
0.206
0.150
Table H.I: Robustness statistics of statinttinn.
17

---

I Robustness to Discrepancy Metric
Tables [.1 and present the schedules proposed by ANT and the CRPS on forecasting tasks with eight datasets using various discrepancy metrics between the non-stationarity curve and linear line, respectively. For the metrics, we employ mean-squared error (MSE), mean-absolute error (MAE), Pearson correlation (Corr.), and R-squared (R²), along with the area under the curve (AUC). The result demonstrates our method's 's robustmess across various metrics for d.
Solar
Electricity
Traftic
Exchange
M4
UberTLC
KDDCup
Wikipodia
MSE
Lin(100)
Cos(75.2.0)
Lin(20)
Cos(50,0.5)
Sig(100,1.0)
Lin(20)
Lin(50)
Cos(75,2.0)
MAE
(0n))
Cos(75.2.0)
Lin(20)
Cos(50,0.5)
Cos(100,1.0)
Lin(20)
Lin(50)
Cos(75,2.0)
Corr.
Lin(100) Sig(100.0.3)
Lin(20)
Cos(50,0.5)
Cos(100,0.5)
Lin(20)
Lin(S0)
Cos(100.2.00
Lin(100)
1os(50.1.0)
Lin(20)
Cos(50,0.5)
Sig(100,1.0)
Lin(20)
Lin(50)
Cos(75,220)
AUC
Lin(100)
Cos(75,2.0)
Lin(50)
Cos(50,0.5)
Cos(100,1.0)
Lin(20)
Lin(50)
Cos(75,2.0)
Oracle
Lin(100)
Cos(75,2.0)
Cos(75,2.0)
Cos(50,0.5)
Cos(100,110)
Lim
Lin(50)
Cos(7552.0)
Table I.1: Robustness of schedule to discrepancy metric.
Solar
Electricity
Traffic
Exchange
M4
UberTLC
KDDCup
Wikipedia
Avg.
TSDiff
0.399
0.049
0.105
0.012
0.036
0.172
0.335
0.221
0.166
MSE
0.326
0.047
0.108
0.009
0.028
0.163
0.206
0.152
MAF
0.326
0.047
0.108
0.009
0.026
0.163
0.325
0.206
0.151
TSDif++ANT
Cor..
0.326
0.047
0.108
0.009
0.026
0.163
0.325
0.208
0.151
0.326
0.049
0.108
0,028
0.163
0,206
0,152
AUC
0.326
0,047
0.101
0.009
0.026
0.163
0.325
0.206
0.150
Oracle
0.326
0.049
0.099
0.009
0.026
0.163
0.325
0.206
0.150
Table 1.2: Robustness of CRPS to discrepancy metric.
18

---

J ANT under Constraint on T
J.1 Schedule with Constraint on T
ANT can propose a schedule under constraint on T. which offers a flexible trade-off between performance and computational cost. As shown in Table D.1 the proposed noise schedules may vary based on the constraint, highlighting the importance of using an appropriate noise schedule under different resource constraints.
T
Solar
Electricity
Traffic
Exchunge
M4
UberTLC
KDDCup
Wikipedia
10
Linear(
Cos(10,1.0)
Linear(
Cos(10,0.5)
Cos(10.0.5)
Linear(
Linear( 10)
Cos(10,1.0)
20
Linear(20)
Cos(20,1.0)
Lincar(20 (20)
Cos(20,0.5)
Sig(20,1.0)
Linear(20) 20)
Lincar(20)
Cos(20,2.0)
Linear(50)
Cos(50.1.0)
Lincar(50)
Cos(50.0.5)
Cos(50.1.0)
Linear(50)
Linear(S0)
Cos(5002.0)
75
Linear(75) Cos(75,2.0)
Li5car(75)
Cos(75.0.5)
Cos(75.1.0)
Linear755
Lincar(100)
Cos(75.2.0)
100
Lincar(100)
Cos(100,2.0)
Lineur(100)
Cos( (00(000,0.5)
Cos(100,1.0)
Lineur(100)
Lineur( (100)
Cos(1000,.0)
Table J.1: Schedule with constraint on T. The schedule proposed by ANT varies based the constraint on T, with the schedules proposed without the constraint in red.
J.2 TS Forecasting with Constraint on T
Table J.2 shows the CRPS on forecasting tasks with a constraint of T 50 where ANT selects schedule among candidates with 7' no larger than 7 (T' < T). The figure indicates that ANT achieves better performance across all datasets even with Tf 100.
than half of the steps of the base schedule
TSDIf
T
Solar
Electricity
'Traflie
Exchange
M4
UberTLC
KDDCUpp
Wikipedia
Avg.
100
0.399
0.049
0.105
0.012
0.036
0.172
0.335
0.221
0.166
0.372
f.499
0.101
0.009
0.029
0.165
0.325
0,210
0,157
+ANT
w/o con.
100
0.326
0.047
0.101
0.009
0.026
0.163
0.325
0.206
0.150
Table J.2: Forecasting result with constraint T. TSDiff applied with ANT under constraint on
diffusion steps (T' < 50) still yields better performance despite the reduced number of steps.
K Comparison with Other Noise Schedules
Table K. ssows the comparison of our method with a cosine schedule 24 and a recently proposed method [21] in the computer vision domain that utilizes schedule enforcing
signal-to- to-i---t--ieee ratio (SNR) at the terminal step and employs u-prediction [27). Note that we tune the hyperparameters for the above methods using the same range
method. The results show that method outperforms the two other methods across all eight datasets terms of CRPS on forecasting tasks, emphasizing the importance of using adaptive schedule that considers the rate of data corruption during the forward process.
Schedule
Solar
Electricity
Traftic
Exchange
M4
UberTLC
KDDCup
Wikipedia
Avg.
Linear
0.399
0.049
0.105
0.012
0.036
0.172
0.335
0.221
0.166
Cosine 24
0.346
0.048
0.102
0.010
0.026
0.174
0,368
0.209
0.160
Zero
21
0.441
0.055
0.123
0.010
0.046
0.184
0.387
0.240
0.185
ANT
0.326
0.047
0.101
0.009
0.026
0.163
0.325
0.206
0.150
Table K.1: Comparison with other schedules.
19

---

L IAAT for Multivariate TS
IAAT for multivariate TS can be applied in two different ways: (1) by calculating IAAT for each variable individually, and (2) by calculating single IAAT that considers all variables together, which we denote as mIAAT (multivariate IAAT).
Calculation of mLAAT. First, the autocorrelation function is defined for each pair of variables at different lags. For multivariate TS X = (X1t, ..,...., Xat) with d variables of length L, the at lag k for variable pairs (i,j) is computed
ΣL (xi x;) (xit+ xj)
(3.5) Pk
(L..1)
VI (x. x2) El= (x]1 x,)
where x; and X, are the means of the TS for variables and j. respectively.
Then, for each variable ((d), IAAT is calculated by computing the of autocorrelations across all lags and variables:
20
IAAT = 1 +
(L.2)
To aggregate the IA across multiple variables, a weighted average of the IAAT values for in- dividual variables can be employed. In this method, the weights are determined by the variances of the individual variables, thereby assigning greater importance to variables with higher variance. Consequently, the IAAT for the multivariate TS (mIAAT) can be calculated follows:
Ed of (8)
TIIAAT
IAAI
(L.3)
where of is the variance of the i-th variable.
20

---

M Application to CSDI and Multivariate TS
To demonstrate the effectiveness of our method with other model architectures and multivariate TS, apply method to CSDI [34], which is a score-based conditional diffusion model that uses transformer layers for multivariate TS fore- casting and imputation. For the experiment, we conduct fore- casting task using Solar [77. M4
Dataset
D
L
11
Solar
137
168
24
Electricity
370
168
24
M4
414
96
48
2X and Electricity [773
employing the CRPS an evaluation metric and Lin(100) base schedule. Table M. MM..ddisplan displays the statistics the datasets, where D, L, and represent the number of variables, the length of the input window, and the length of the target window, respectively. Following the original paper, we normalize each feature to have zero mean and unit variance, and set 31 = 0.0001 and 8T = 0.5 for the schedule. Note that as CSDI adds noise only to certain parts of the TS which are randomly masked with ratio Uniform(O,1) I), calculate the IAAT with partially noised TS by randomly masking certain series with ratio r and only performing the forward process on the masked parts, excluding the unmasked parts.
Table M.1: Statistics of datasets.
As we use a multivariate TS dataset, we apply our method two ways: (1) using a common schedule across all variables (with mIAAT) where we compute the ANT score based on Equation L..3] and (2) using a variable-specific schedule for all variables (with IAAT). For the variable-specific schedule, choose a schedule with common 7 for all variables, and Figure M.1 shows the ratio of schedules chosen as adaptive schedules for each variable.
Solar
M4
Electricity variables 370
variables 137
variables 414
47.0%
43.1%
2.2%
3.3%
linear cosine(1.0)
cosine(2.0)
sigmoid(0.5)
sigmoid(0.3)
Figure M.I: Ratio of adaptive schedule for each variable.
Table M.2
shows the results of TS forecasting, indicating that our method improves performance when using a common schedule for all variables, but hampers performance when using an adaptive schedule for each variable, underscoring the importance of employing common schedule for all variables using mIAAT. Furthermore, Figure [h2]shows the non-stationarity curves for all variables of Solar, with the 5th and 95th percentiles shaded, using both the base schedule and the schedule proposed by ANT. It is evident that ANT resembles the ideal linear line (black) more closely.
CSDI+ANT
CSDI
Solar
M4
Electricity
(Scaled) (ed) IAAT
CSDI
0.402
0.109
0.044
0.6
w/ mIAAT
0.351
0.035
0.041
CSDI+ANT
w/ IAAT
0.706
0.063
0.050
Oracle
0.349
0.034
0.041
Table M.2: Multivariate TS forecasting task.
of Step
Figure M.2: Non-stationarity curves.
21

---

Efficiency of ANT
N.1 Inference Time by T
Table N.1 displays the inference time TSDiff with schedules of different T'.. Specifically, we report inference time for TS data averaged on the test dataset of Wikipedia [8] The results highlight efficiency of using a smaller T, achieving the best forecasting performance (low CRPS) with schedule of T 75, proposed by ANT. Notably, the base schedule performs worse despite using larger T of = 100.
Inference
Oracle
w/ ANT
w/o ANT
T
10
20
50
75
100
75
100
Time (sec)
0.15
0.29
0.71
1.06
1.40
1.06
1.40
CRPS
0.225
0.210
0.207
0.206
0.208
0.206
0.221
Table N.1: Inference time (sec) by T.
N.2
Comparison of Training & Inference Time
Table N.2
compares the training and inference times of TSDiff with and without the application of ANT across eight datasets, where we train for 1000 epochs and perform inference with the entire test dataset. The results demonstrate that training time reduced for datasets where linear schedules are selected from ANT, as diffusion step embedding is eliminated. In terms of inference time, efficiency gained through the reduced T. For instance, on the UberTLC dataset, efficiency improves by 78. %% with T reduced from 100 to 20.
Train (min)
Inference (sec)
w/o ANT
w/ANT
Gain(%)
w/o ANT
w/NNT
+ Gain %)
Solar
93.7
85.2
+9.1% %
1.28
1.24
+3.4%
Electricity
95.4
+0.0%
1.28
0.96
+24.1 %
Traffic
100.5
92.0
+8.5%
1.28
0.64
+50.0%
Exchange
92.0
+0.0%
1.48
0.68
+46.2%
M4
85.2
+0.0%
1.20
+0.0%
UberTLC
94.5
86.0
+9.0%
1.28
0.27
+78.1 %
KDDCup
106.5
99.6
+6.5%
1.24
0.62
+49.4%
Wikipedia
98.9
+0.0%
1.40
1.06
+24.0%
Table N.2: Train & Inference time w/ and w/o ANT,
22

---

TS Generation with Electricity
Histogram of IAAT
KDE piot of IAAT
Fitting IAAT with GMM
700
Real
Real
IT the-NNT
TSDIY
600
0.023
Count
500
12 0.020
400
0.015
100
0.035
200
(a) Histogram.
(b) Kernel density estimation. Figure O.1: IAAT plots of real and generated TS.
(c) Gaussian mixture model.
Data
Modality
Parameters
Ratio
Data
CRPS
JSD
(103.8 28.
45.6%
Linear
DeepAR
Truns.
Real
Multi
2)
(185.3, 13.1")
54.4%
Real
0.088
0.054
0.076
w/o ANT
(109.00 39.5*)
100%
w/n ANT
0.065
0.058
0.056
0.343
TSDiff
Uni
TSDiff
w/ ANT
(1113.8, 31.9")
100%
w/ ANT
0.067
0.058
0.061
0.369
Table O.1: Result of Gaussian fitting for IAATs.
Table O.2: CRPS & JSD with Real.
Figure O. and 0.11 disppyyyyyy the IAAT distributions of real TS and TS samples generated from TSDiff using Electric- ity [7, with and without ANT, using histogram and kernel density estimation (KDE) plots, respectively. The figures indicate that while the distribution of real TS exhibits mul- timodal characteristics, TSDiff without ANT captures only one modality, whereas TSDiff with ANT captures the other modality.
KDE plot of IAAT
0.014
0.010
Density
0.008
0.006
0.004
a visualization of real TS by the IAAT, exhibiting distinct patterns among different TS. TS (A), characterized by the lowest IA exhibits no discernible pat- terns, whereas TS (E), with the highest IAAT, demonstrates a pronounced seasonality.
0.002
A
E
0.000
100
150
200
250
IAAT
To assess the similarity between the real TS and the generated TS, we employ Gaussian mixture model (GMM) fitting on the IAAT distributions, with the results presented in Figure O.Ic Table provides the mean and variance of each modality. revealing that the modality captured by TSDiff without ANT accounts for 45.6% of the data, while the modality captured by TSDiff with ANT constitutes 54. 44.4% of the dataset.
To analyze the the performance of TS generation task in terms of the IAAT distribution, we calculate the Jensen-Shannon divergence (JSD) between the IAAT distributions of the real TS and the generated TS. Taking into account the ratio of data in each modality, we compute the weighted JSD with weights derived from Table The results Table (.2) indicate that
E VM
TS generated from TSDiff without ANT exhibits lower JSD compared to those generated with ANT, resulting slightly lower average CRPS across three different base forecasters (Linear, DeepAR, and Transformer).
Figure O.2: Visualization by IAAT.
It's noteworthy that the performance is the worst with the real samples, suggesting the inability of the forecasters to predict TS with different modalities using a single model. This also suggests that the quality of the synthetic dataset cannot be fully captured by the TS generation task of (TSTR), indicating its insufficiency.
23

---

t-SNE Visualizations
depicts the t-SNE visualizations of embeddings CNN features extracted from a ID-CNN encoder of the proxy classification model, with both linear and non-linear schedules. The figure shows that both visualizations of the linear and non-linear schedules yield directional point movement across all output channels as the step progresses. However, with the linear schedule, the directions appear more diverse, indicating that step information is captured by both schedules, but more limited with non-linear schedule.
Figure P.I
Channel
Channel #
Channel #
Channel # 4
Linear
Channel # 1
Channel #
Channel #
Channel # 4
Non-linear
-
Step ratio (t/T)
100% (Noise)
80%
60%
40%
20%
0% (Original)
Figure P.I:
t-SNE visualizations with linear and non- non-linear schedules.
24

---

Visualization of Non-stationarity Curves
Figure Q.1 presents the visualization of the non-stationarity curves of eight datasets when employing and adaptive schedules proposed by ANT. The result indicates
a base schedule of TSDIff that, in general, the larger the reduction in discrepancy between the linear line and the non-stationarity curve by the proposed schedule, the greater the performance gain achieved. Note that for the Solar [17] dataset, the schedule proposed by ANT is identical to the base schedule, and any performance gain is solely attributed to the elimination of the diffusion step embedding.
M4 (CRPS gain: +27.8%
UberTLC CCPSS gain: +5.2%)
the Peae
% Step
Exchange (CRPS gain: +25.1%)
Traffic CRPS gain: +3.9%) )
500p
Wikipedia ( (CRPS gain: +6.8%
KDDCup (CRSS gain: +3.0%)
Electricity (CRPS gain: .4.1%) )
Solar (CRPS gain: +22.4%
TEDM
Figure Q.I: Visualization of non-stationarity curves. The figure shows the non-stationarity curves of eight datasets, when employing a base schedule and and an adaptive schedule proposed by ANT.
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's s contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The paper discuss the limitations of the work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution low images taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
26

---

Justification:
The paper provides the full set of assumptions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper fully disclose all the information needed reproduce the main experimental results.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case of large language model), releasing of model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reprodueibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g.. large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., registered users), but should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
27

---

Answer: [Yes]
Justification: The paper provide open access to the data and code, with sufficient instructions to reproduce the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.ce ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this central the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/otddeSSmmmsssssiooooooo.ooo ssi ssionPolicy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper specify all the training and test details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper reports standard deviations of the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" the results are accompanied by error bars, confidence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
28

---

It should be clear whether the enror bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides sufficient information the computer resources, including computational time.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The research conform with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure preserve anonymity (e.g., if there is a special consider- ation due laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The proposed method for selecting an adaptive noise schedule for time series diffusion models has the potential to significantly impact the field of diffusion models across various domains, including time series. The ability to utilize the characteristics of datasets for noise schedules is promising, as different statistics can be employed depending on the domain. To ensure responsible use of the algorithm, we make the datasets and code publicly available htttt:://// h/tt:::///tthbb com/ seunghan96/ ANT Public availability ensures transparency and reproducibility, enabling other researchers to evaluate and address potential biases and misuse.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The research does not contain any data or models that have a high risk for
misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The creators or original owners of assets (e.g., code, data, models) used in the paper are properly credited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
30

---

For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperssiitheeee thcode com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer:[Yen
Justification:
The new assets introduced in the paper are well documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: There are no crowdsourcing experiments or research with human subjects in this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not contain any potential risks incurred by study participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
31

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
32


Reviews:
[{"review_id": "Cops0cMzrf", "reviewer": "Reviewer_YUJG", "summary": "This paper addresses the non-stationarity in time series data by proposing an ANT score to enable an adaptive noise schedule for diffusion models. It provides extensive experimental results on time series forecasting and generation tasks.", "questions": "1. I wonder whether the authors can provide more rigorous theoretical analysis on ANT score.\n2. It would be nice to see whether the ANT can be adapted to other diffusion models.", "limitations": "The paper limits the noise functions to linear, cosine, and sigmoid, and it may be more imperative to be extended to other cases.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The idea of adaptively select noise schedule in diffusion models is original and interesting, which can effectively addresses the number of steps issue.\n2. The paper introduces an ANT score to enable an adaptive noise schedule, which measures the discrepancy between a linear line and the non-stationary curve.\n3. Experiments have been conducted on several different time series tasks including time series forecasting, refinement, and generation, which is generally solid.", "weaknesses": "1. Though the ANT score seems effective in empirical evaluations, the paper does not provide theoretical analysis on how the score plays a role in addressing the proposed limitations of traditional diffusion models.\n2. The proposed ANT noise schedule can be applied to different types of diffusion models, however in the experiments the ANT is only adapted to TSDiff, which lacks enough evidence to demonstrate its generalizability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **W1, Q1. Theoretical analysis on the ANT score**\n\nPrevious works on noise schedules have demonstrated their effectiveness primarily through **empirical justification** rather than theoretical analysis [4,21,24]. In line with this approach, we have conducted extensive experiments to support our proposal, as highlighted by reviewers xQJK and eJYD.\n\nFurthermore, following the previous works regarding the schedules, we focus on the intuitive aspects that schedules should meet, where the proposed ANT score aligns with the three desiderata of noise schedules: \n- **1) $\\lambda_{\\text{linear}}$: Reducing non-stationarity on a linear scale**\n\nPrevious work of DDPM [24] proposed a cosine schedule (instead of linear) and argued that maintaining a consistent noise level at each step leads to better quality, as mentioned in **L38--39**.\n\n- **2) $\\lambda_{\\text{noise}}$: Corruption to random noise**\n\nPrevious work [21] suggested that the schedule must be capable of corrupting the TS into random noise at the final step to ensure the generation of high-quality samples, as the reverse process (sampling) of the diffusion model begins with random noise, as mentioned in **L40--41** and **L133--135**.\n\n- **3) $\\lambda_{\\text{step}}$: Sufficient steps**\n\nPrevious work of consistency models [31] emphasized the need for a sufficient number of steps to generate high-quality samples, as mentioned in **L41--42** and **L135--136**.\n\nWhile the advances in noise scheduling for diffusion models has mostly been made empirically, we hope our work motivates future works on the theoretical understanding of a relationship between optimal noise scheduling and the characteristics of the datasets.\n\n&nbsp;\n\n### **W2, Q2. Application to other diffusion models.**\n\nWe appreciate the reviewer YUJG's concern about the **limited range of models** used to demonstrate the effectiveness of ANT. We completely agree and, as discussed in **Appendix M**, we have *already* applied our method to another well-known TS diffusion model, CSDI [34]. We believe this further validates the applicability and effectiveness of our method across different TS diffusion models (in both univariate and multivariate settings).\n\n&nbsp;\n\n### **L1. Functions used for candidate schedules.** \nWe acknowledge the reviewer YUJG's point on expanding the range of noise functions. However, our focus is **not on identifying the optimal schedule across various noise functions**, but rather on proposing the criterion of choosing effective noise schedules tailored to the dataset's characteristics.\n\nHowever, we emphasize that the linear, cosine, and sigmoid functions are well-established and widely used in literature [4], and recent TS diffusion models [1, 26, 29, 30] primarily employ linear or cosine schedules. Although developing the best noise function is not our primary goal, we have explored extending our approach with a more flexible noise (and non-trivial) function, such as an ensemble of cosine functions, as demonstrated in **Figure 11** and **Table 12**, to illustrate the potential applicability of ANT for future advancements in noise functions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mMyccU8354", "reviewer": "Reviewer_eJYD", "summary": "This paper proposes ANT, which adaptively selects the optimal noise schedule for time series diffusion models. The ANT score is computed for each schedule offline based on the datasets' statistics, which is the basis for selection. Extensive experiments demonstrate the method's superior performance in multiple time series tasks.", "questions": "- What is the basis for the design of your candidate schedule? What if they do not include the optimal schedule? Will it have a big impact on the results of the experiment?\n- Although a lower score indicates a better schedule, are there many cases where schedules with higher scores have better results?\n- Are the optimal schedules for the same data set significantly different between different diffusion methods? Can the proposed method consider the characteristics of specific methods when selecting a schedule?", "limitations": "Yes", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- This method is innovative. Different from previous work that directly applied the existing general framework, it takes into account the characteristics of time series data, that is, non-stationarity, to select the scheduler.\n- The experiments cover a variety of tasks and data sets, and the experimental results are convincing.", "weaknesses": "- These candidate schedules are manually set in advance and are not guaranteed to include the optimal schedules.\n- The ANT score is a simple multiplication of three factors, which is relatively intuitive. Their high or low values ​​cannot guarantee the quality of the schedules.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **W1, Q1. Candidate schedules do not guarantee to include the optimal schedule.** \nAs the reviewer eJYD mentioned, the candidate schedule may not include the optimal schedule. However, we note that we do **not aim to find the optimal schedule**; our proposed ANT is a **criterion for choosing a better noise schedule from candidates based on the characteristics of the dataset**.\n\nFor experiments, we selected three widely used schedules (linear, cosine, sigmoid) [4] as our candidates, incorporating five different diffusion steps and three different temperatures for non-linear schedules. This resulted in a total of **35 schedules**, which we believe are sufficient, as they encompass most of the schedules used in previous TS diffusion studies [1,26,29,30].\n\nAs shown in **Figure 11** and **Table 12**, ANT is applicable to non-trivial schedules, i.e., it gives a higher score to sensibly designed noise schedules if they perform better than trivial ones. However, as mentioned in the conclusion, finding the optimal schedule for a given dataset using our ANT score (via optimization) remains a future work.\n\n&nbsp;\n\n### **W2. Relationship between ANT score and the quality of schedule**\nThe reviewer raised concerns about the relationship between the ANT score and the quality of the schedule. However, the proposed ANT score aligns with the three desiderata of noise schedules, which can be considered indicators of schedule quality. These desiderata have been discussed in prior works and our paper:\n\n- **1) $\\lambda_{\\text{linear}}$: Reducing non-stationarity on a linear scale**\n\nPrevious work of DDPM [24] proposed a cosine schedule (instead of linear) and argued that maintaining a consistent noise level at each step leads to better quality, as mentioned in **L38--39**.\n\n- **2) $\\lambda_{\\text{noise}}$: Corruption to random noise**\n\nPrevious work [21] suggested that the schedule must be capable of corrupting the TS into random noise at the final step to ensure the generation of high-quality samples, as the reverse process (sampling) of the diffusion model begins with random noise, as mentioned in **L40--41** and **L133--135**.\n\n- **3) $\\lambda_{\\text{step}}$: Sufficient steps**\n\nPrevious work of consistency models [31] emphasized the need for a sufficient number of steps to generate high-quality samples, as mentioned in **L41--42** and **L135--136**.\nFurthermore, as shown in **Table 7**, using all the three components of ANT mostly results in the oracle, backing up our argument. For example, with the M4 dataset [23], using all three components results in a value of 0.026 with a Cosine(100,1.0) schedule, whereas the ANT score without considering $\\lambda_\\text{noise}$ yields a CRPS of 0.094 with a Cosine(10,0.5) schedule. Notably, the base schedule of TSDiff (Linear(100)) yields a CRPS of 0.036.\n\n&nbsp;\n\n### **Q2. Case for higher ANT score but with better result.** \nA lower ANT score generally indicates a better schedule, as shown in **Figure 8(a)**, and this tendency becomes stronger when using all three components of the ANT score. However, as the reviewer has noted, there can be exceptions where better performance is achieved with a higher ANT score, maybe because of the stochasticity of the forward diffusion process when computing ANT. Nonetheless, the schedule with the lowest ANT score should return a reasonably good performance.\nIn our extensive analysis, we found only one such case with the Traffic dataset [3], as shown in **Table 5**. Among the 35 candidate schedules, Linear(50) had the lowest ANT score, yet the best performance was obtained with Cosine(75, 2.0), with a marginal difference in CRPS(0.101 vs. 0.099). It is worth noting that the base schedule Linear(100) yielded a CRPS of 0.105, which is noticeably higher than both Linear(50) and Cosine(75, 2.0).\n\n&nbsp;\n\n### **Q3. Optimal noise schedule depending on diffusion models** \nNote that the ANT score is computed **without access to the diffusion model**, such that the schedule is selected based on **the characteristics of the dataset**, rather than the **specific design choice on the model architecture** being used.\n\nNonetheless, as the reviewer eJYD concerned, some specific design choices for diffusion models could affect the optimality of the schedule. As we had a similar concern, in **Appendix M**, we have applied our method to another well-known TS diffusion model, CSDI [34], where we found that ANT is effective for this model as well. We believe this further validates the applicability and effectiveness of our method across different TS diffusion models.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xWWabL3sre", "reviewer": "Reviewer_xQJK", "summary": "Paper introduces the new method, adaptive noise schedule  for time series diffusion model(ANT) . This is a new methodology as per my knowledge and hence extends the boundaries of the field. Diffusion models are highly effective in generating data but their inclusion in Time Series tasks overlooks the significance of properties of time series and don't focus on dedicated noise scheduling. \n\nThe paper introduces an algorithm to choose an adaptive noise scheduling which has not been done in the past for time series. Authors clearly define the statistics used to quantify the non-stationarity and ANT score. The proposed schedule noise aims to corrupt the TS data gradually instead of abruptly as done with Linear scheduling. It also demonstrates the need for different scheduling needed for different datasets. One size fit all solution does not serve the purpose. \n\nAnother important aspect of the paper discusses and proves why diffusion embedding is not needed for linear schedule. In addition, there was several analysis which shows the importance and stability of ANT.", "questions": "1. There are several places in the text where language can be improved to avoid ambiguity and enhance the clarity of paper.  E.g. L27 text: \"Several works have been proposed to design appropriate schedules [24, 21], but they are not tailored to the TS domain, considering the characteristics of TS data.\". L31: it should be M4 dataset and a reference added to it. \n\n2. Conclusion/Summary does not discuss/report the main results quantitatively.", "limitations": "Authors does mention about the limitation at the end of the draft that ANT could take long time if to find the noise schedule if there are too many candidate schedules and this motivated future work in this direction.   It is also noted that the comparison for forecasting task is done using very limited models to show the effectiveness of the ANT but more advanced models (DLinear/FEDFormer etc) could be used in comparison for completeness and also give a hint how big is the difference in metric.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Originality: The paper introduce new method ANT for adaptive noise scheduling for time series. This is an original idea in TS domain. In addition the a new metric is introduced (IIAT) to quantify the importance of ANT. \n\nQuality: Overall the quality of the paper is good, all the statements/claims are backed with experimental results. Authors conducted extensive experiments and analysis across several datasets and tasks which However, the details about the experiments/analyses and evaluation is limited and can be explained further in details. \n\nClarity: The paper is well structured, clearly written making is accessible for reading to different level of background knowledge with diffusion model for TS. The new algorithm, metric and concepts are introduced clearly in the draft under review. The terms are defined and experimented are backed with tables and figures which make it clear to understand the new algorithm better. \n\nSignificance: ANT significantly improve the accuracy of the TS forecasting models by used adaptive noise scheduling and have wide applications. The paper describes how non-stationarity can be used to enhance the performane for generative tasks.", "weaknesses": "Although paper is well structures and provide tables and figured for all the experiments conducted, details on the evaluation of experiments would help the reader further. E.g. How is metric calculated for the time series forecasting task; what fraction of data is used for inference and which evaluation method is used.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **W1. Details on the evaluation of experiments** \nWe agree that some details on the evaluation of experiments were omitted due to space limitations. Here, we provide the additional details and will incorporate them into Appendix.\n\n\n&nbsp; \n\n\n> 1. **Metric for TS forecasting task**\n\n\n\n\nFollowing prior works on TS diffusion models [6,18,19,26,34,38] including TSDiff [15], we utilize the **Continuous Ranked Probability Score (CRPS)** [9] to assess the quality of *probabilistic* forecasts, where we describe the details in **Appendix B**. Note that TS diffusion models predict the *probabilistic* distribution rather than the *deterministic* value, so CRPS accounts for the uncertainty of the predicted results, unlike MSE or MAE.\n\n\n&nbsp; \n\n\n> 2. **Fraction of (test) data used for inference** \n\n\n\n\nAll datasets used in our experiments are sourced from GluonTS [2], where training and test splits are provided. We follow the setting of TSDiff [15], where the validation set is created by splitting a portion of the training dataset, with the split ratio determined by the size of the train and test datasets.\nFurther details about the training and test sets are provided in **Table A.1**.\n\n\n&nbsp; \n\n\n> 3. **Evaluation method** \n\n\n\n\nFor evaluation, we calculate the CRPS using only the **test dataset**. Specifically, we follow the approach used in TSDiff [15] by utilizing the `gluonts.transform.sampler.TestSplitSampler` from GluonTS [2], which leverages the final time steps of the test TS instances to predict future values.\n\n\n&nbsp; \n\n\n### **Q1. Clarifying sentences and adding references**\nThank you for addressing this. We are planning to improve overall writing in the revision for better clarity including your suggestions. For sure, what we meant in **L27** is that noise schedules used in [24, 21] are not tailored to the TS domain (but images). Regarding M4 in **L31**, we have already mentioned **the M4 dataset** with a reference in the previous sentence (**L28**), but we agree that the term **M4** might be confusing when used standalone.\n\n\n&nbsp; \n\n\n### **Q2. Main quantitative results missing in the conclusion.** \nThank you for pointing this out. As we have emphasized our results quantitatively in the introduction, we will also summarize and reiterate the main findings quantitatively in the conclusion section.\n\n\n&nbsp; \n\n\n### **L1-2. Comparison with more advanced models (DLinear/FEDFormer etc).**\n\n\n> **1) Application to CSDI**\n\n\n\n\nWe appreciate the reviewer xQJK's concern about the **limited range of models** used to demonstrate the effectiveness of ANT. We completely agree and, as discussed in **Appendix M**, we have *already* applied our method to another well-known TS diffusion model, CSDI [34]. We believe this further validates the applicability and effectiveness of our method across different TS diffusion models (in both univariate and multivariate settings).\n\n\n&nbsp; \n\n\n> **2) Comparing with other TS forecasting models [A,B]**\n\nIn short, DLinear [A] and Fedformer [B] are not compared because they are for *deterministic* forecasting, while our task is *probabilistic* forecasting; TS forecasting models for these two different tasks have been developed independently.\nWe provide more detailed reasons as below:\n- **Model**) TS diffusion models, including TSDiff, are probabilistic forecasting models, while models like DLinear and Fedformer are deterministic forecasting models, which have developed independently in prior works.\n- **Metric**) For the above reason, previous works on TS diffusions [6,18,19,26,34,38] do not use them as baselines and instead employ a different metric, CRPS, to account for the uncertainty of the predicted results, rather than the MSE/MAE metrics commonly used in deterministic models.\n- **Dataset**) TS diffusion models are mostly designed for short-term TS forecasting (STSF) [C], whereas models like DLinear and Fedformer are intended for long-term TS forecasting (LTSF). In LTSF tasks, datasets generally consist of long TS with fewer instances, and training and testing splits are usually based on different time periods of the same instances. In contrast, STSF tasks involve shorter time series with more instances, and the split is based on different instances of the TS.\n\n\n\n\nNonetheless, to address the reviewer xQJK's concern, we conducted an additional experiment with DLinear [A] using the Solar dataset [17]. Since DLinear is a deterministic model and cannot be directly compared with our task which uses CRPS as a metric, we employ a common technique for probabilistic forecasting [28] by predicting the mean and standard deviation of a normal distribution rather than exact values. The results show that DLinear yields a CRPS of **0.660**, while TSDiff and {TSDiff with ANT} achieve CRPS scores of **0.399** and **0.326**, respectively.\n\n&nbsp; \n\n[A] Zeng, Ailing, et al. \"Are transformers effective for time series forecasting?\" AAAI 2023\n\n\n[B] Zhou, Tian, et al. \"Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting.\" ICML 2022\n\n\n[C] Meijer, Caspar, and Lydia Y. Chen. \"The Rise of Diffusion Models in Time-Series Forecasting.\" arXiv 2024", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# General Comments
First of all, we deeply appreciate your time and effort in reviewing our paper.
Our work introduces **ANT** (**A**daptive **N**oise Schedule for **T**ime Series), a method for automatically predetermining proper noise schedules based on the characteristics of the dataset.


As agreed by all reviewers, our proposed ANT is original/novel and its effectiveness is supported by extensive experiments. In our responses, we addressed the concerns raised by the reviewers and supplemented our claims with additional analyses. Here are some key highlights to assist with your post-rebuttal discussion:


&nbsp; 


### **1. Application to other TS diffusion models (All)**

We found that reviewers are interested in whether ANT is applicable to other TS diffusion models, and we have *already* applied our method to another well-known TS diffusion model, CSDI [34], as detailed in **Appendix M** and guided in **L83--84**.


&nbsp; 



### **2. Comparison with other baselines (Reviewer xQJK)**

Reviewer xQJK asked us to compare other baseline methods like DLinear and FEDformer with ours. However, they are proposed for *deterministic* forecasting, while our task with diffusion models is *probabilistic* forecasting, such that they have been developed independently and are not directly comparable. Nonetheless, to address reviewer xQJK's concern, we conducted an additional experiment with DLinear on the Solar dataset and confirmed the superiority of ours (with TS diffusion models) over DLinear.


&nbsp; 


### **3. Finding the optimal schedule from finite number of candidates (Reviewer eJYD, Reviewer YUJG)**

We emphasize that our contribution is *not on finding the optimal schedule*, but on proposing a ***criterion for efficiently selecting a better noise schedule from candidates based on the dataset's characteristics***, where we used candidate functions that are well-established and widely used in literature [4]. We also confirmed that our ANT score is able to assign a high score to non-trivial schedules if they result in good performance in **Figure 11** and **Table 12**. We leave the way to find the optimal schedule for a given dataset using our ANT score (via optimization) as a future work.

&nbsp; 


### **4. Theoretical analysis on the ANT score (Reviewer YUJG)**

We note that previous works on noise schedules primarily relied on empirical justification rather than theoretical analysis [4, 21, 24], and we have also supported our proposal through extensive experiments following these works. While the advances in noise scheduling for diffusion models has mostly been made empirically, we hope our work inspires future research into the theoretical understanding of the relationship between optimal noise schedule and dataset characteristics.

&nbsp; 


We sincerely appreciate the reviewers' valuable feedback and insights. We believe these will significantly enhance the contribution of our paper, and we will integrate them into the final version. Should there be any additional points we may have overlooked or if you have further questions or suggestions, please let us know; we are eager to address them and refine our work.

&nbsp; 

Thank you very much.

Authors.

Meta-review: This paper introduces a novel methodology to build time series diffusion models. All the reviewers are positive about the contribution. The authors should incorporate the reviewers' feedback in the final version of their paper.
Decision: Accept (poster)