Retrieval-Augmented Diffusion Models for Time Series Forecasting
Authors: Jingwei Liu
Ling Yang
Hongyan Li
Shenda Hong
Venue: NeurIPS 2024 poster (2024)
Keywords: Diffusion Models
Times Series Forecasting. Retrieval-Augmented Mechanism
Primary area: generative_models
TL;DR: We propose a Retrieval-Augmented Time series Diffusion framework for time series forecasting for the first time.

Abstract: While time series diffusion models have received considerable focus from many recent works, the performance of existing models remains highly unstable. Factors limiting time series diffusion models include insufficient time series datasets and the absence of guidance. To address these limitations, we propose a Retrieval-Augmented Time series Diffusion model (RATD). The framework of RATD consists of two parts: an embedding-based retrieval process and a reference-guided diffusion model. In the first part, RATD retrieves the time series that are most relevant to historical time series from the database as references. The references are utilized to guide the denoising process in the second part. Our approach allows leveraging meaningful samples within the database to aid in sampling, thus maximizing the utilization of datasets. Meanwhile, this reference-guided mechanism also compensates for the deficiencies of existing time series diffusion models in terms of guidance. Experiments and visualizations on multiple datasets demonstrate the effectiveness of our approach, particularly in complicated prediction tasks. Our code is available at https://github.com/stanliu96/RATD

Retrieval-Augmented Diffusion Models for Time Series Forecasting
Jingwei Liu 1.2 Ling Yang" Hongyan Li Shenda Hong t
School of Intelligence Science and Technology, Peking University
2 National Key Laboratory of General Artificial Intelligence, Peking University Institute of Medical Technology, Peking University Health Science Center
National Institute of Health Data Science, Peking University
5 Institute for Artificial Intelligence, Peking University nngve11lu1996016.. com, yangling0818016: com (leehy, hongshenda> edu.
Abstract
While time series diffusion models have received considerable focus from many recent works, the performance of existing models remains highly unstable. Factors limiting time series diffusion models include insufficient time series datasets and the absence of guidance. To address these limitations, we propose a Retrieval- Augmented Time series Diffusion model (RATD). The framework of RATD con- sists of two parts: an embedding- diffusion model. In the first part. RATD retrieves the time series that most relevant to historical time series from the database as references. The references are utilized to guide the denoising process in the second part. Our approach allows leveraging meaningful samples within the database to aid in sampling, thus maxi- mizing the utilization of datasets. Meanwhile, this reference guided mechanism also compensates for the deficiencies of existing time series diffusion models terms of guidance. Experiments and visualizations multiple datasets demon- strate the effectiveness of our approach, particularly in complicated prediction tasks. Our code is available at httpss//gttubb.ommmstmnn091111
retrieval process and a reference-guidee
serrencceguidedd
Introduction
Time series forecasting plays a critical role in variety of applications including weather forecasting [1 11,, finance forecasting 57, earthquake prediction [19] and energy planning 66 One way to approach time series forecasting tasks is to view them conditional generation tasks 32 42 where conditional generative models are used to learn the conditional distribution P(x P Hf of predicting the target time series x given the observed historical sequence " As the current state-of-the- art conditional generative model, diffusion models [12] have been utilized in many works for time series
forecasting tasks 28 36 30
Although the performance of the existing time series diffusion models is reasonably well on some time series forecasting tasks, remains unstable in certain scenarios (an example is provided in [c). The factors limiting the performance particularly evident. First, most time series lack direct semantic label correspondences, which often results in time series diffusion models lacking meaningful guidance during the generation
time series diffusion models are complex, two of them are
"Contact: Jingwei Liu, jingweiliu 1996@ 163.com
'Contributed equally.
*Corresponding Authors: Hongyan Li, Shenda Hong
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

process(such as text guidance or label guidance in image diffusion models). This also limits the potential of time series diffusion models.
Groand Fiush
Model
Time Sence
Tram Soam
Dooks
CSDI
Pedicctee
(Our)
Proditte
Suries
(4)
(b)
(0)
Figure I: (a) The figure shows the differences in forecasting results between the CSDI 336 (left) and
RATD (right). Due the very small proportion of such cases in the training set, CSDI struggles to make accurate predictions, often predicting more common results. Our method, by retrieving meaningful references as guidance, makes much more accurate predictions. (b) A comparison between our method's (bamtwork(botttom) and the conventional time series diffusion model framework((opp) (c) We randomly selected 25 forecasting tasks from the electricity dataset. Compared our method, CSDI and MG-TSD [9] exhibited significantly higher instability. This indicates that the RATD is better handling complex tasks that are challenging for the other two methods.
The second limiting factor arises from two shortcomings the time series datasets: size insufficient and imbalanced. Compared to image datasets, time series datasets typically have smaller scale. Popular image datasets (such as LAION series datasets usually only contain tens of thousands of data points. Training a diffusion model to learn the precise distribution of datasets with insufficient size challenging. Additionally, real- world time series datasets exhibit significant imbalance. For example, in the existing electrocardiogram dataset MIMIC-IV, records related to diagnosed pre-excitation syndrome (PS) account for less than 0.025% of the total records. This imbalance phenomenon may cause models to overlook some extremely rare complex samples, leading to a tendency to generate more common predictions during training. thus making it difficult handle complex prediction tasks, as illustrated in Figure !
contain 400 million sample pairs, while most time
To address these limitations, we propose the Retrieval-Augmentee Augmented Time series Diffusion Model (RATD) for complex time series forecasting tasks. Our approach consists of two parts: the embedding- based retrieval and the reference-guee -ureeed diffusion model. After obtaining historical time series, input into the embedding- based retrieval process to retrieve the k nearest samples as references. The references are utilized as guidance the denoising process. RATD focuses on making maximum utilization of existing time series datasets by finding the most relevant references in the dataset to historical time series, thereby providing meaningful guidance for the denoising process. RATD focuses on maximizing the utilization of insufficient time series data and to some extent mitigates the issues caused by data imbalance. Mcanwhile, this reference guidedd mechanism also compensates for the deficiencies of guidance in existing time series diffusion models. Our approach demonstrates strong performance across multiple datasets, particularly on more complex tasks.
To summarize, our main contributions are summarized as follows:
To handle complex time series forecasting, we for the first time introduce Retrieval- Augmented Time series Diffusion (RATD), allowing for greater utilization of the dataset and providing meaningful guidance in the denoising process.
Extra Reference Modulated Attention (RMA) module is designed to provide reasonable guidance from the reference during the denoising process. RMA effectively simply integrates information without introducing excessive additional computational costs.
We conducted experiments on five real-world datasets and provided comprehensive pre- sentation and analysis of the results using multiple metrics. The experimental results demonstrate that our approach achieves comparable better results compared to baselines.

---

2 Related Work
2.1 Diffusion Models for Time Series Forecasting
Recent advancements have been made in the utilization of diffusion models for time series forecasting. In l'meCGrad (28), the conditional diffusion model was first employed
autoregressive approach for prediction, with the denoising process guided by the hidden state. CSDI [36] adopted a non-
autoregressive generation strategy to achieve faster predictions. SSSD [ replaced the noise-matching network with structured state space model for prediction. TimeDiff (30] incorporated future mix-up and autoregressive initialization into a non-autoregressiive framework for forecasting. MG-TSD [ utilized multi-scale generation strategy to sequentially predict the main components and details of the time series. Meanwhile, mr-diff [31] utilized diffusion models to separately predict the trend and seasonal components of time series. These methods have shown promising results in some prediction tasks, but they often perform poorly in challenging prediction tasks. We propose a serrieaal-augmenn framework to address this issue.
2.2 Retrival-Augmented Generation
The vetrieval-augmente mechanism is one of the classic mechanisms for generative models. Numer- ous works have demonstrated the benefits of incorporating explicit retrieval steps into neural networks. Classic works in the field of natural language processing leverage retrieval augmentation mechanisms to enhance the quality of language generation [T6] TO 4]. In the domain of image generation, some strieval-augmennee models focus on utilizing samples from the database to generate more realistic images (2) [4]. Similarly, (3] employed memorized similarity information from training data for retrieval during inference enhance results. MQ-ReTCNN [40] is specifically designed for complex time series forecasting tasks involving multiple entities and variables. Re'Time [3] creates a relation graph based on the temporal closeness between sequences and employs relational retrieval instead of mentent-based retrieval. Although the aforementioned three methods successfully utilize retrieval mechanisms to enhance time series forecasting results, our approach still holds significant advan- tages. This advantage stems from the iterative structure of the diffusion model, where references can repeatedly influence the generation process, allowing references to exert a stronger influence on the entire conditional generation process.
3 Preliminary
The forecasting task and the background knowledge about the conditional time series diffusion model will be discussed in this section. To avoid conflicts, we use the symbol "s" to represent the time series, and the "t" denotes the 1-th step in the diffusion process.
Generative Time Series Forecasting, Suppose we have an observed historical time series x {s1, $2, St 8 Rd), where / is the historical time length, observation and 8; is the observation at time step i. The x" is the corresponding prediction target
the number of features per
{5141, 8142, ... ,14h 9144 E Rdd } (d' < d) where h is the prediction horizon. The task of generative time series forecasting is to learn density pa(x">x" that best approximates p(æ" (x "), which can be written as:
min D (po D6
|| (l"))ppp=P11 (())))))))))) =²)).
(1)
where 0 denotes parameters and D is some appropriate measure of distance between distributions.
Given observation æ the target time series can be obtained directly by sampling from po (xx ((
Therefore, we obtain the time series {51, {2,,82, s2. ,8,44] [a"" .x"] ".
Conditional Time Series Diffusion Models. With observed time series "" the diffusion model progressively destructs target time series x (equals to the x mentioned in the previous context) by injecting noise, then learns to reverse this process starting from of for sample generation. For the convenience of expression, in this paper, we use D, to refer to the t-th time series in the diffusion process, with the letter "P" omitted. The forward process can be formulated as a Gaussian process with Markovian structure:
q (x,=1) := := VI B,o,- H ,B,I),
(2)
(0x))x0) = N(xx vorzo, "1 (1 I))
3

---

Porttaleed Ensoder
Historical
1 pola,, x1, .,
Time Serice
Nodes
X
X
xt-1
Prediction
X
Vector Strilbrity
Ristoiical
Time Seriee
V
Guide
tediees
Retrieval
x
Database
Forward
Detabere
References
D
Retrieval
Mechanism
Referenccos
Main Framework
Figure Overview of the proposed RATD. The historical time series x " is inputted into the retrieval module to for the corresponding references æR, After that, z is concatenated with the noise as the main input for the model μıg. xt will be utilized as the guidance for the denoising process.
where . Br denotes fixed variance schedule with a == Br and ag II. as. This forward process progressively injects noise into data until all structures are lost, which is well- approximated by (V, I). The reverse diffusion process lears model pe po(x;-11 [ x that approximates the true posterior:
po(::-1|<1; (0) := (i De (zt), (x)),,)))))))) ))
where μg and Mg are often computed by the Transformer. Ho et al. 12 improve the diffusion training process and optimize following objective:
((vo)=
=
[1 1 1/xl) - ja(x, mo "=o"
Am ( [x)
where A(æ, vg (t is the mean of the posterior ( (zo, 2i) which is a closed from Gaussian, and pa t|x is the predicted mean of (0 2e (x(x") computed by neural network.
4 Method
We first describe the overall architecture of the proposed method in 4.1 Then we will introduce the strategy of building datasets in Section 4.2] The embedding- retrieval mechanisms and reference-guide time series diffusion model are introduced in Section 4.3
4.1 Framework Overview
Figure shows the overall architecture of RATD. We built the entire process based DiffWave [17], which combines the traditional diffusion model framework and 2D transformer structure. In the forecasting task, RATD first retrieves motion sequences from the database base D based on the input sequence historical events. These retrieved samples are then fed into the Reference- Modulated Attention (RMA) references. In the RMA layer, we integrate the features of the input [x x at time step with side information I, and the references R. Through this integration, the references guide the generation process. We will introduce these processes in the following subsections.
4.2 Constructing Retrieval Database for Time Series
Before retrieval, it is necessary to construct proper database. We propose strategy for constructing databases from time series datasets with different characteristics. Some time series datasets are size- insufficient and difficult to annotate with single category label (e.g., electricity time series), while some datasets contain complete category labels but exhibit a significant degree class imbalance (e.g., medical time series). We use two different definitions of databases for these two different types of datasets. For the first definition, the entire training set is directly defined as the database DR:
R (æ. Var;
Dirain)
where m. = [8i, +++ the time series with length 1 + h, and Dirain is the training set. the second way, the subset containing samples from all categories in the dataset is defined as the database

---

La
[2r41
xx1
Qlaxd)
2/ax d)
Tanporal Fnanoore Layer
X
Kijn x)
A[d=d]
(
Relorrnce Facturs
Vimax d)
Output
Uooo Projectias
()) Model Architecture
(b) Reference Modulated Attention
Figure 3: The structure of μag (a) The main architecture of μ4 is the time series transformer structure that proved effective. (b) The structure of the proposed RMA. We integrate three different features through matrix multiplication. Di::
DR
{ .. oeiVe E CJ
(6)
where zf is the i-th sample in the k-th class of the training set, with length of (+h. h. C is the category set of the original dataset. For brevity, we represent both databases as DR.
4.3 Retrieval-Augmented Time Series Diffusion
Gimeeddin-Bassee Retrieval Mechanism For time forecasting tasks, the ideal references (s,,... S would be samples where preceding n points {5:-m>'.. s is relevant to the historical time series {sj. ... Sjtn in the DR. In our approach, the overall similarity be- tween time series of greater concern. We quantify the reference between time series using the distance between their embeddings. To ensure that embeddings can effectively represent the entire time series, pre-trained encoders Eg are utilized. Eg is trained on representation learning tasks, and the parameter set ø is frozen in our retrieval mechanism. For time series (with length + h) in DR their first n points are encoded, thus the R can be represented as DR
(7) where (p g] refers to the subsequence formed by the p-th point to the g-th point in the time series.
D {(i, (a(o))))) (rrem))))///ana) (
D
The embedding corresponding to the historical time series can be represented as " = Eg (x
We calculate the distances between w and all embeddings in DR and retrieve the references corresponding to the k smallest distances. This process be expressed as:
index(v arg min || E2fo011))
s #'EDR
(8)
x" {0 pann MVj index(""))
where index(-) represents retrieved index given Thus, we obtain subset R of DR based on query # i.e. " DD R where a: k.
Reference-Guided Time Series Diffusion Model guided time series diffusion model. In the diffusion process, the forward process is identical the traditional diffusion process, as shown in Equation (2). Following [34] 12] (35 the objective of the reverse process is to infer the posterior distribution p( 2ar (2*) through the subsequent expression:
In this section, we will introduce our reference-
T
p(a
II
(ac, |x,, at " DDEIT
where p(xr|=") xe (arll=" I). pa p0 1 " x²) is the reverse transition kernel from æ; to
Φ;-1 with a learnable parameter 0. Following most of the literature in the diffusion model, we adopt the assumption:
p0 æ) -N(x "
". ", ())), Co(ws, x .)))
(10)
where uap is a deep neural network with parameter 0. After similar computations as those in 12] Eg (mn, .H R 1)) in the backward process is approximated as fixed. In other words, we can achieve reference-guiin guided denoising by designing rational and robust μgg
5

---

Denoising Network Architecture Similar to Wave [77] and CSDI [3], our pipeline is con- structed on the foundation of transformer layers, as shown in Figure However, the existing framework cannot effectively utilize the reference as guidance. Considering attention modules to integrate the R and 24 as a reasonable intuition, we propose a novel module called Reference Mod- ulated Attention (RMA). Unlike normal attention modules, we realize the fusion of three features in RMA: the current time series feature, the side feature, and the reference feature. To be specific, RMA was set at the beginning of each residual module Figure 3 We use ID-CNN to extract features from the input Øt references ". and side information. Notably, we concatenate all references together for feature extraction. Side information consists of two parts, representing the correlation between variables and time steps in the current time series dataset Appendix B We adjust the dimensions of these three features with linear layers and fuse them through matrix dot products. Similar to text-image diffusion models [29]. RMA can effectively utilize reference information to guide the denoising process, while appropriate parameter settings prevent the results from overly depending the reference.
Training Procedure To train RATD (i.e., optimize the evidence lower bound induced by RATD), we use the same objective function previous work. The loss time step 1 are defined as follows respectively:
1
4-1 =
|1 20) (()x1;0001'
2,3
(11)
Yts 20g Boll
&. 18?
where do are predicted from æt and 7/2 2/5(11 (z))) are hyperparameters in diffusion process. We summarize the training procedure of RATD in Algorithm I and highlight the differences from the conventional models, in cyan. The process of sampling is shown in Appendix
Algorithm 1 Training Procedure of RATD
Require: Time series dataset Dimin neural network pe, diffusion step T, external database D pre-trained encoder Eg, number of references
1: Retrieve references with top-k high similarity from D using E to obtain x as described in Section
2: while de not converge do
3: Sample diffusion time f E U(.....,
4: Compute the side feature I.
5: Perturb æn obtain æt Predict do from ær. I, and x Compute loss L with 2p and 2o Update 0 by minimizing L end while
6:
(Equation 10 (Equation
7:
8:
9:
Experiments
5.1 Experimental Setup
Datasets Following previous work [45] 38 30), experiments are performed on four popular real-world time series datasets: (1) Electricity" which includes the hourly electricity consumption data from 321 clients over two years.; (2) Wind [20], which contains wind power records from
2020-2021. (3) Exchange [18), which describes the daily exchange rates of eight countries (Australia, British, Canada, Switzerland, China, Japan, New Zealand, and Singapore); (4) Weathe
F which
documents 21 meteorological indicators at 10-minute intervals spanning from 2020 2021.; Besides, we also applied our method to a large ECG time series dataset: MIMIC-IV-ECO [4]. The MIMIC- IV-ECG dataset contains clinical electrocardiogram data from over 190,000 patients and 450,000 hospitalizations at Beth Israel Deaconess Medical Center (BIDMC).
12014

---

Table 1: Performance comparisons on four
datasets in terms of MSE, MAE, and CRPS.
The best is in bold, while the second best is underlined.
Dataset
Exchange
Wind
Flectricity MAE
Weather
Metric
MSE
MAE
CRPS
MSE
MAE
CRPS
MSE
CRPS
MSE
MAE
CRPS
RATD (ours)
0.013
0.073
0.339
0.784
0.579
0.673
0.151
0.246
0.373
0.281
0.293
0.301
TifeDiff
0.018
0.091
0.589
0.896
0.687
0.917
0.193
0.305
0.490
0.327
0.312
0.410
CSDI
0.077
0.194
0.397
1.066
0.741
0.941
0.379
0.579
0.480
0.356
0.374
0.354
mr-Diff
0.016
0.082
0.397
0.881
0.675
0.881
0.173
0.258
0.429
0.296
0.324
0.347
D& VAE
0.200
0.301
0.401
1.118
0.779
0.979
0.286
0.372
0.389
0.315
0.380
0.381
Fedformer
0.133
0.233
0.631
1.113
0.762
1.235
0.238
0.341
0.561
0.342
0.347
0.319
FreTS
0.039
0.140
0.440
1.004
0.703
0.943
0.269
0.371
0.634
0.351
0.354
0.391
FiLM
0.016
0.079
0.349
0.984
0.717
0.798
0,210
0.320
0.671
0.327
0.336
0,556
Traansformer
0.016
0,074
0.343
0.932
0.676
0.811
0.192
0.262
0.402
0.358
0,401
0.318
Autoformer
0.056
0.167
0.769
1.083
0.756
1.201
1.026
0.313
0.602
0,360
0.354
0.754
Pyraformer
0.032
0.112
0.532
1.061
0.735
0.994
0.273
0.379
0.732
0.394
0.385
0.485
Informer
0.073
0.192
0.631
1.168
0.772
1.065
0.292
0.383
0.749
0.385
0.364
0.821
PutchTST
0.047
0.153
0.629
1.001
0.672
1.026
0.225
0.394
0.801
0.782
0.670
0.370
SCINet
0.038
0.137
0.624
1,055
0.732
0,997
0.171
0.280
0.499
0.329
0.344
0.814
DLincar
0,022
0.102
0,538
0.899
0.686
0.957
0.215
0.336
0.527
0.488
0.444
0.791
NLincar
0.019
0.091
0.481
0.989
0.706
0,974
0.147
0.239
0.419
0.369
0.328
0.738
TimesNet
0.023
0.120
0.520
0.982
0.771
1.001
0.141
0.361
0.403
0.313
0.364
0.491
NBeats
0.016
0.081
0.399
1.069
0.741
0.981
0.269
0.697
0.744
0.420
0.871
Baseline Methods To comprehensively demonstrate the effectiveness of our method, we compare RATD with four kinds of time series forecasting methods. Our baselines include (1) Time series dif- fusion models, including CSDI (36), mr-Diff [31], V [20]. Tiff [30]: (2) Recent time series
forecasting methods with frequency information, including FiLM [46, Fedformer [47] and FreTT
41
(3) Time series transformers, including PStchTST 25 Autoformer (38] Pyraformer [22], In-
former 45 and i'Transformer 23 (4) Other popular methods, including TimesNet (39]. SciNet 21
Nlinear 43 DLinear
43 and NBeats (26).
Evaluation Metric To comprehensively assess our proposed methodology, our experiment employs three metries: (1) Probabilistic forecasting metrics: Continuous Ranked Probability Score (CRPS) on each time series dimension
[24]. (2) Distance metries: Mean Squared Error (MSE), and Mean Average Error(MAEE) are employed to measure the distance between predictions and ground truths.
Implementation Details The length of the historical time series was 168, and the prediction lengths Nvidia RTX
were (96, 192, 336), with results averaged. All experiments were conducted A6000 GPU with 40GB memory. During the experiments, the second strategy of conducting D was employed for the MIMIC dataset, while the first strategy was utilized for the other four datasets. To reduce the training cost, we preprocessed the retrieval process by storing the reference indices of each sample in the training in a dictionary. During the training on the diffusion model, accessed this dictionary directly to avoid redundant retrieval processes. More details are shown in Appendix B
5.2 Main Results
T the primary results of our experiments on four daily datasets. Our approach surpasses existing time series diffusion models. Compared to other time series forecasting methods, our approach exhibits superior performance on three out of four datasets, with competitive performance on the remaining dataset. Notably, we achieve outstanding results on the wind dataset. Due to the lack of clear short-term periodicity (daily or hourly), some prediction tasks in this dataset exceedingly challenging for other models. netrieval-augmenen mechanisms can effectively assist in addressing these challenging prediction tasks.
Figure presents a case study randomly selected from our experiments on the wind dataset. We compare our prediction with iTransformer and two popular open-source time series diffusion models, CSDI and D, VAE. Although CSDI and VAE provide accurate predictions in the initial short-term period, their long-term predictions deviate significantly from the ground truth due to the lack of guidance. I'Transformer captures rough trends and periodic patterns, yet our method offers higher- quality predictions than the others. Furthermore, through the comparison between the predicted
7

---

groundd prediction
ground prediction
grounst mruth prediction
130
139
140
2
254
CSDI
D VAE
frrnnformer
hieoryy ground_ 1T007
prediciion
RATD
Reference
Figure 4: Visualizations on wind by CSDI, D3 VAE, iTrrasformmr and the proposed RATD (with reference).
results and references in the figure, although references provide strong guidance, they do not explicitly substitute for the entire generated results. This further validates the rationality of our approach.
Table presents the testing results our method on the MIMIC-IV-ECG dataset. We selected some powerful open-source methods as baselines for comparison. Our experiments are divided into two parts: in the first part, we evaluate the entire test set, while in the second part, we select rare cases (those accounting for less than 2% of total cases) from the test set subset for evaluation. Prediction tasks in the second part are more challenging for deep models. In the first experiment, our method achieved results close to iTransformer, while the second task, model significantly outperformed other methods, demonstrating the effectiveness of our approach in addressing challenging tasks.
5.3 Model Analysis
Influence of Retrieval Mechanism To investigate the impact of the retrieval augmentation mecha- nism on the generation process, we conducted an ablation study and presented the results Table 3 The study addresses two questions: whether the retrieval augmentation mechanism is effective and which retrieval method is most effective. Firstly, we removed our retrieval augmentation mechanism from the RATD baseline. Besides, the model with random time series guidance another baseline. The references retrieved by other methods have all positively impacted the prediction results. This suggests that reasonable references highly effective in guiding the generation process.
We also compared two different retrieval mechanisms: correlation- based retrieval. The first method directly retrieves the reference in the time domain (e.g.. using Dynamic Time Warping (DTW) or Pearson correlation coefficient). Our approach adopts the second mechanism: retrieving references through the embedding of time series. From the results, the purclation-basee methods are significantly inferior to the embedding methods fail to capture the key features of the time series, making difficult to retrieve the best references for forecasting. We also evaluate the embedding- based methods with various encoders for comparison. The comprehensive results show that methods with different encoders do not significantly differ. This indicates that different methods can all extract meaningful references. thereby producing similar improvements in results. TCN was utilized in experiment because TCN strikes the best balance between computational cost and performance.
retrieval and embedding-
methods. The former
Effect of Retrieval Database We conducted an ablation study on two variables, n and k. to investigate the influence of retrieval database D RATD, where n represents the number of samples in each category of the database, and k represents the number of reference exemplars. The results in Figure g can benefit the model in terms of prediction accuracy because larger DR brings higher diversity, thereby providing more details beneficial for prediction and enhancing the generation

---

Table 2: Performance comparisons on MIMIC datasets with popular time series forecasting meth- ods. Here, "MIMIC-IV (All)" refers to the model's s testing results on the complete test set, while "MIMIC(Rare)" indicates the model's testing results on rare disease subset.
Method
Transfomer
PTtchTST
TimesNer
CSDI
RATD
Metric
MSE
CRPS
MSH
CRPS
MSR
MAE
CRPS
MAR
CRPS
MSE
MAE
CRPS
MIMIC-IV (AII)
0.174
0.263
0.299
0.219
0.301
0.307
0.193
0.311
0.310
0.268
0.331
0.369
0.172
0,270
0.293
MIMIC-IV (Rac)
0.423
0.315
0.379
0.183
0.379
0.407
0.627
0.359
0.161
0.499
0.359
0.371
0.206
0,299
0.301
Table 3: Ablation study on different retrieval mechanisms. "_" means no references was utilized and "Random" means references selected randomly. Others refer what model use for retrieval references.
Dataset
Exchange
Wind
Electricity
Weather
Metric
MSF
MAR
CRPS
MSF
MAF
CRPS
MSR
MAE
CRPS
MSF
MAR
CRPS
0.077
0.194
0.397
1.066
0,741
0.941
0.379
0,579
0.480
0,356
0.374
0.354
Random
0.153
0.203
0.599
1.593
0.903
0.996
0.471
0.639
0.701
0.431
0.473
0.461
DTW
0,075
0.195
0.403
1.073
0.791
0.942
0.357
0.564
0.449
0.361
0.375
0.356
Pearson
0.091
0.207
0.411
1.099
0.831
0.953
0.361
0,571
0.483
0.370
0.364
0.391
DI Linear
0.022
0.081
0.361
0.941
0.735
0.895
0.159
0.250
0.390
0.297
0.304
0.332
Informer
0,019
0.078
0.371
0.841
0.645
0.861
0.170
0.263
0.411
0.291
0.305
0.330
TimesNet
0,013
0.074
0.341
0.781
0,572
0.669
0.167
0.263
0.397
0,286
0.295
0.311
TCN
0.013
0.073
0.339
0.784
0.579
0.673
0.161
0.256
0.391
0.281
0.293
0.313
process. Simply increasing k does not show significant improvement, utilizing more references may introduce more noise into the denoising process. our experiment, the settings n and & are 256 and 3, respectively.
Inference Efficiency In this experiment, we evaluate the inference efficiency of the proposed RATD in comparison to other baseline time series diffusion models (TimeGrad, MG-TSD, SSSD). T the inference time on the multivariate weather dataset with varying values of the prediction horizon (h). While method introduces additional retrieval module, the sampling efficiency of the RATD is not low due to the non-autore oon-autoregressve transformer framework. even slightly outperforms other baselines across all h values. Notably, TimeGrad is observed to be the slowest, attributed to its utilization of auto-regressive decoding.
MG130
10000
F000
0315
0000
0110
2000
150
202
400
500
3.2
2000
320
-200
5.0
702
Precicton on 2on
Figure 5: The effect of hyper-parameter n and k. Figure 6: Inference time (ms) the Electricity with different prediction horizon
Effectiveness of Reference Modulated Attention To validate the effectiveness of the proposed RMA, we designed additional ablation experiments. In these experiments, architecture as the baseline method and added extra fusion modules to compare the performance of these modules (linear layer, cross-attention layer, and RMA). The results are shown in the Table 4
used the CSDI
Through our experiments, we found that compared the basic cooss-attention-te based approach, RMA can integrate an edge information matrix (representing correlations between time and feature dimensions) more effectively. The extra fusion is highly beneficial experiments, guiding the model to capture relationships between different variables. In contrast, linear-based methods concatenate inputs and references initially, which prevents the direct extraction of meaningful information from references, resulting in comparatively modest performance.

---

Table Performance comparison between CSDI-based methods, CSDI represents the basic network framework, LSDi+L denotes the approach where inputs and references are concatenated via a linear layer and fed into the network together, CSDI+CroosAttention signifies the use of cross attention to fuse features from inputs and references, and finally, CSDI+RMA, which incorporates an additional RMA.
Datasct
Exchange
Electricity
Wind
Weather
Solar
MIMIC-IV
CSDI
0.077
0.379
1.066
0.356
0.381
0.268
CSDi+Linear
0,075
0,316
0,932
0.349
0.369
0.265
Cros+Cross Attention
0.028
0.173
0.829
0.291
0.340
0. 0.183
CSDI+RMA
0.013
0.151
0.784
0.281
0.327
0.172
Predicting 2p vs Predicting Following the formulation in Section 4.3 our network is designed
forecast the latent variable 20. Since some existing models (28 36 have been trained by predicting
an additional noise term we conducted comparative experiment to determine which approach is more suitable for our framework. Specifically, we maintained the network structure unchanged, only modifying the prediction target to be e. The results are presented in Table Predicting zo proves to be more effective. This may because the relationship between the reference and v, is more direct, making the denoising task relatively easier.
Table MSEs of denoising strategies: Predicting vo vs predicting e.
denoising strategy
Wind
Weather
Exchange
270
0.784
0.281
0.013
0.841
0.331
0.018
RMA position We investigate the best position of RMA the model. Front, middle, and back means we set the RMA in the front of, in the middle of, and the back of two transformer layers, respectively. We found that placing RMA before the bidirectional transformer resulted in the most significant improvement in model performance. This also aligns with the intuition of network design: cross-attention modules placed at the front the model tend to have a greater impact.
Table 6: Ablation study on different RMA positions. The best in bold.
Dataset
Exchange
Wind
Electricity
Weather
Metric
MSE
MAE
CRPS
MSE
MAE
CRPS
MSE
MAE
CRPS
MSE
MAE
CRPS
0.077
0.194
0.397
1.066
0.741
0.941
0.379
0.579
0.480
0.356
0.374
0.354
Back
0.031
0.105
0.373
0.673
0.611
0.842
0.267
0.434
0.426
0.301
0.321
0.322
Middle
0.057
0.141
0.381
0.799
0.631
0.833
0.291
0.481
0.451
0.333
0.331
0.336
Front
0.013
0.063
0.331
0.784
0.579
0.673
0.161
0.256
0.391
0.281
0.293
0.313
Discussion
Limitation and Future Work As transformer-basee diffusion model structure, our approach still faces some challenges brought by the transformer framework. Our model consumes a significant amount of computational resources dealing with time series consisting of too many variables. Addi- tionally, our approach requires additional preprocessing (retrieval process) during training, which incurs additional costs on training time (around ten hours).
Conclusion In this paper, we propose new framework for time series diffusion modeling to address the forecasting performance limitations of existing diffusion models. RATD retrieves samples most relevant to the historical time series from the constructed database and utilize them as references to guide the denoising process of the diffusion model, thereby obtaining more accurate predictions. RATD is highly effective in solving challenging time series prediction tasks, evaluated by experiments on five real-world datasets.
Acknowledgements
This work is supported by the National Natural Science Foundation of China (No.6217201 No.62 No.62102008) and Wuhan East Lake High-Tech Development Zone National Comprehensive Experi- mental Base for Governance of Intelligent Society.
10

---

References
[1] Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion- based time series imputation and forecasting with structured state space models. arXiv preprint arXiv:2208.09:99 2022.
[2] Andreas Blattmann, Robin Rombach, Kaan Oktay, Jonas Müller, and Bjöm Ommer. Retrieval- augmented diffusion models. Advances in Neural Information Processing Systems, 35:15309- 15324, 2022.
[3] Giovanni Bonetta, Rossella Cancelliere, Ding Liu, and Paul Vozila. Retrieval-augmentel transformer-xl for close-domain dialog generation. arXiv preprint arXiv:2 2021.
[4] Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Sean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pp. 2206-2240. PMLR, 2022.
[5] Jian Cao, Zhi Li, and Jian Li. Financial time series forecasting model based on ceemdan and Istm. Physica A: Statistical mechanics and its applications, 519:127-139, 2019.
[6] Jui-Sheng Chou and Duc-Son Tran. Forecasting energy consumption time series using machine learning techniques based on usage patterns of residential householders. Energy, 165:709-726, 2018.
[7] Alexiei Dingli and Karl Sant Fournier. Financial time series forecasting-a deep learning approach. International Journal of Machine Learning and Computing, ((5):188 122, 2017.
[8] Wei Fan, Shun Zheng, Xiaohan Yi, Wei Cao, Yanjie Fu, Jiang Bian, and Tic-Yan Yan-Yan Liu. Depts: deep expansion learning for periodic time series forecasting, arXiv preprint arXiv:2203. 07681,, 2022.
[9] Xinyao Fan, Yueying Wu, Chang Xu, Yuhao Huang, Weiqing Liu, and Jiang Bian. Mg-tsd: Multi-granularin time series diffusion models with guided learning process. arXiv preprint arXiv:2403.05751. 2024.
[10] Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pre-training. In International conference machine learning, pp. 1929-39338 PMLR, 2020.
[11] Pradeep Hewage Ardhendu Behera, Marcello Trovati, Ella Pereira, Morteza Ghahremani, Francesco Palmieri, and Yonghuai Liu. Temporal convolutional neural (ten) network for an effective weather forecasting using time-series data from the local weather station. Soft Computing, 24: 24:16453-16482, 2020.
[12] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-68511 2020.
[13] Baoyu Jing, Si Zhang, Yada Zhu, Bin Peng, Kaiyu Guan, Andrew Margenot, and Hanghang Tong. Retrieval based time series forecasting. arXiv preprint arXiv:2209. /3525, 2022.
[14] Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, al. Mimic-iv, freely accessible electronic health record dataset. Scientific data, 10(1):1, 2023.
[15] Zahra Karevan and Johan AK Suykens. Transductive 1stm for time-series prediction: An application to weather forecasting. Neural Networks, 125:1-9, 2020.
[16] Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Gen- cralization through memorization: Nearest neighbor language models. arXiv preprint arXiv:1911.00177 2019.
[17] Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2020.

---

[18] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterss with deep neural networks. In The st international ACM SIGIR conference on research & development in information retrieval, pp. 93-104, 2018.
[19] S Sri Lakshmi and RK Tiwari. Model dissection from earthquake time series: A compara- tive analysis using modern non-linear forecasting and artificial neural network approaches. Computers & Geosciences, 35(2):191-200 2009.
[20] Yan Li, Xinjiang Lu, Yaqing Wang, and Dejing Dou. Generative time series forecasting with diffusion, denoise, and disentanglement. Advances in Neural Information Processing Systems, 35:23009-23022, 2022.
[21] Minhao Liu, Ailing Zeng. Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 35:5816-58285 2022.
[22] Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International conference on learning representations, 2021.
[23] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long, itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625, 2023.
[24] James E. Matheson and Robert I. Winkler. Scoring rules for continuous probability distributions. Management science, 1087-1169, 1976.
[25] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-tern term-termmne forecasting with transformers. arXiv preprint arX1v:2211.147.00 2022.
[26] Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv: 1045,77777 2019.
[27] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[28] Kashif Rasul, Calvin Seward, Ingmar Schuster, and Roland Vollgraf. Autoregressive denois- ing diffusion models for multivariate probabilistic time series forecasting. In International Conference on Machine Learning, pp. 8857-8868. PMIR, 2021.
[29] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
[30] Lifeng Shen and James Kwok. Non-autoreeressive conditional diffusion models for time series prediction. arXiv preprint arXiv:2306.00 05043, 2023.
[31] Lifeng Shen, Weiyu Chen, and James Kwok. Multi-resolution diffusion models for time series forecasting, In The Twelfth International Conference on Learning Representations, 2023.
[32] Zhipeng Shen, Yuanming Zhang, Jiawei Lu, Jun Xu, and Gang Xiao. Seriesnet: generative time series forecasting model. In 2018 international joint conference on neural networks (IJCNN), pp. 1-8. IEEE, 2018
[33] Zhuoran Shen, Mingyuan Zhang, Haiyu Zhao, Shuai Yi, and Hongsheng Li. Efficient attention: Attention with linear complexities. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp.
2021.
[34] Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International conference on machine learning, pp. 2256-2263 PMLR, 2015.
12

---

[35] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprini arXiv:2011. 13456, 2020.
[36] Yusuke Tashrro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems,
Conditional score-based
2021.
[37] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[38] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long, Autoformer: Decomposition trans- formers with core-correlation for long-term series forecasting. Advances in Neural Information Processing Systems, 34:22419 222300, 2021.
[39] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The eleventh international conference on learning representations, 2022.
[40] Sitan Yang, Carson Eisenach, and Dhruv Madeka. Mqretun: Multi-horizon time series forecast- ing with retrieval augmentation. arXiv preprint arXiv:2207. 2022.
[41] Kun Yi. Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domain mips are more effective learners in time series forecasting. In Seirth-seventh Conference on Neural Information Processing Systems, 2023.
[42] Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. Time-series generative adversarial networks. Advances neural information processing systems, 32, 2019.
[43] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 11121-11128, 2023.
[44] Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, and Ziwei Liu. Remodiffuse: detrieval-augmenntr motion diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 364-373, 2023.
[45] Haoyi Zhou, Shanghang Zhang. Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting, In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11115, 2021.
[46] Tian Zhou, Ziqing Ma, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, Rong Jin, al. Film: Frequency improved legendre memory model for long-term time series forecasting. Advances in Neural Information Processing Systems, 35:12677-12690, 2022.
[47] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning, pp. 27268-27286. PMLR, 2022.
13

---

A Sampling Procedure
Like Algorithm J
we summarize the sampling procedure RATD in Algorithm and highlight the differences from conventional diffusion models in
Algorithm 2 Sampling Procedure of RATD
Require: The historical time series " the learned model po, external database D pre-trained Eg. the number of references k
Ensure: Prediction x corresponding the history x
1: Sample initial target time series æT
2: Embed x into v
3: Retrieval the reference a: with of
4: Compute the side feature I,
5: for in T - 1,..., 1 do
6: Predict to from 1r, I. and at
(Equation 10
7: Sample -t-1 from the posterior g(0r)(0))
(Equation 2
8: end for
B Impletion Details
B.1
Training Details
Our dataset split in the proportion of 7:1:2 (Train: Validation: Test), utilizing random splitting strategy to ensure diversity in the training set. We sample the ECG signals 125Hz for the MIMIC-IV dataset and extract fixed-length windows as samples. For training, we utilized the Adam optimizer with initial learning of 10 -3, betas (0.95, 0.999). During the training process of shifted diffusion, the batch size was set to 64, and early stopping was applied for a maximum of 200 epochs. The diffusion steps T were set 100.
B.2 Side Information
We combine temporal embedding and feature embedding as side information v.. We use 128- dimensions temporal embedding following previous studies 337:
Sembedding (5c)
o (12)
where r = 10000. Following [3), st represents the timestamp corresponding to the 1-th point in the time series. This setup is designed to capture the irregular sampling in the dataset and convey it to the model. Additionally, we utilize learnable embedding to handle feature dimensions. Specifically, feature embedding is represented
16-dimensional learnable vectors that capture relationships between dimensions. According to [17], we combine time embedding and feature embedding, collectively referred to side information I.,
The shape I, not fixed and varies with datasets. Taking the Exchange dataset as an example, the shape of forecasting target R is [Batchsize (64), 7(numeer of variables). 168 (time-dimension), 12 (tim-ddimension) and the corresponding shape of I, is [Batchsize (64), total channel (144) time: + feature:16)), 320
latent channel), 12 (time-dimension).
B.3 Transformers Details
Our approach employs the Transformer architecture from CSDI, with the distinction of expanding the channel dimension to 128. The network comprises temporal and feature layers, ensuring the comprehensiveness of the model handling the time-frequency domain latent while maintaining relatively simple structure. Regarding the transformer layer, we utilized a 1-layer Transformer encoder implemented PyTorch (27), comprising multi- head attention layers, fully connected layers, and layer normalization. We adopted the "linear attention transformer" package to enhance
http::///ghthaooooc//cccccaaaaat transffrrmmt
14

---

computational efficiency. The inclusion of numerous features and long sequences prompted this decision. The package implements an efficient attention mechanism [33 and we exclusively utilized the global attention feature within the package.
B.4 Metrics
We will introduce the metrics in experiments. We summarize them below:
CRPS. CRPS (2 is univariate strictly proper scoring rule which measures the compatibility of cumulative distribution function F with an observation
CRPS(F, J. (F(v) ((rs))
(13)
where (esy)) is the indicator function, which is if x S y and O otherwise. The CRPS attains the minimum value when the predictive distribution same as the data distribution.
MAE and MSE. MAE and MSE are calculated in the formula below, x² represents the predicted time series, and x represents the ground truth time series. MAE calculates the average absolute difference between predictions and true values, while MSE calculates the average squared difference between predictions and true values. A smaller MAE or MSE implies better predictions.
MAE = mean (1x xx I)
(14)
MSE Vmean((ap 1!)
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction include the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of our approach at Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
16

---

Justification:
The paper does not include theoretical results
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide all the information needed in the Experiment part.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer:
[No]
Justification:
The full code will be uploaded in the future.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
ooooooooocccy lc for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ctpppp..cppulli//guuii wwwiiicgguie///eeeeeeeeic CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide all the details about the dataset in appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
We do not report the error bars in our experiemnt.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
18

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provide sufficient information on the computer resources in para- graph implement details.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: pesttp://neuripsttt
Answer: [Yes]
Justification:
The research conducted in the paper conform with the NeurIPS Code of Ethics
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
There is no obvious societal impact of the work performed
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All the original work are sited properly.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
21


Reviews:
[{"review_id": "RLILdjXQqB", "reviewer": "Reviewer_HjSX", "summary": "This paper proposes a retrieval-augmented time series diffusion model that uses an embedding-based retrieval process and a reference-guided diffusion model. The proposed RATD retrieves relevant time series from an external database as references, which is later utilized to guide the denoising process of the diffusion model for future forecasting. Extensive experiments are conducted on five real-world datasets that demonstrate the effectiveness of the proposed approach.", "questions": "- How is the time series encoder $E_\\phi$ pretrained with representation learning tasks (Line148)? Since the embedding quality highly affects the retrieval precision and quality, some analysis and evaluations should be conducted on the time series embedding quality.\n- What is the architecture of the pre-trained encoder $E_\\phi$? How do you encode the multivariate time series into a single embedding? Do you use any pooling operation? More analysis and explanations on the encoder are beneficial for paper clarity.", "limitations": "Yes. The authors discussed the limitations of the proposed approach in the paper.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper is clearly written and easy to understand. The authors provide clear and detailed explanations and illustrations for the proposed architecture.\n- The idea of applying retrieval augmentation on the diffusion model for time series forecasting is novel and insightful.", "weaknesses": "- The proposed architecture requires retrieval database construction and pretraining the time series encoder $E_\\phi$ before training the forecasting model. The additional complexity and computation cost may limit the efficiency of the proposed method.\n- Experimental results do not contain standard deviation, which potentially limits the confidence and significance of the performance superiority.\n- Some baselines are missing. For instance, aurhors mentioned MQ-ReTCNN and ReTime as retrieval-augemented time series models. However, they are not invovled as baselines in experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*We thank Reviewer HjSX for the thorough and valuable feedback. We are glad that the reviewer found that the proposed idea is novel. The reviewer's main concern is the use of pre-trained models. Please see below for our responses to your concerns.* \n\n**Q1**: The additional complexity and computation cost may limit the efficiency of the proposed method.\n\n**A1**: The demand for pre-trained models does indeed incur additional computational costs. However, the increased training costs do not impose severe constraints on the practical application of the model. The additional training costs stem from two components: pretraining the encoder and the retrieval process. Specifically, pretraining the encoder is straightforward, as we employ the structurally simple yet effective TCN. Typically, TCN can be trained on a dataset within a few hours on a standard single GPU (e.g., Nvidia RTX 3090), thereby avoiding excessively high time costs. Similarly, the entire retrieval process also incurs a few extra hours of computational time. Compared to the time costs required for training diffusion models (ranging from a day to several days), the extra cost is not a serious issue. Furthermore, it is worth noting that our model does not incur excessively high sampling costs (as depicted in Figure 6 of the original text), which is crucial because sampling costs of diffusion models are a more pertinent concern compared to training costs.\n\n**Q2**: Experimental results do not contain standard deviation, which potentially limits the confidence and significance of the performance superiority.\n\n**A2**: Thank you very much for your suggestion! Calculating the standard deviation will indeed further assess the stability of the model's generated results. Following your advice, we conducted additional experiments including some popular baselines, and the MSE results are shown in the table below (with the same experimental settings as in this paper, we repeat the test 15 times to calculate the standard deviation ). \n\n| |Exchange| Wind | Traffic | Weather|\n| :-----| :---- | :---- |:---- |:---- |\n|RATD（Ours）  | 0.013(0.001)  | 0.784(0.005)  | 0.151(0.002) |0.281(0.002) |\n|iTransformer| 0.016(0.001) | 0.932(0.007) |0.192(0.003) |0.358(0.003) |\n|PatchTST| 0.047(0.009) | 1.001(0.009) |0.225(0.003)|0.782(0.008)|\n|CSDI| 0.077(0.003) | 1.066(0.008) |0.379(0.003) |0.356(0.002)|\n\nOur method also demonstrates superior certainty in the results based on additional experiments.\n\n**Q3**: Some baselines are missing. For instance, the authors mentioned MQ-ReTCNN and ReTime as retrieval-augmented time series models.\n\n**A3**: We did not compare our method with the previous RAG-based time series methods in the paper because these papers did not provide comprehensive evaluations on commonly used time series datasets or publicly available codes. The implementation details in these papers were also very limited, making replication of their methods challenging. Nevertheless, as you mentioned, for the sake of comprehensive experimental comparison, we present here the MSE results of our replicated method under our experimental settings, demonstrating that our approach shows noticeable advantages.\n| |Exchange| Wind | Traffic | Weather|\n| :-----| :---- | :---- |:---- |:---- |\n|RATD（Ours）  | 0.013(0.001)  | 0.784(0.005)  | 0.151(0.002) |0.281(0.002) |\n|MQRetNN| 0.063(0.004) | 1.116(0.008) |0.346(0.003) |0.668(0.004) |\n|ReTime| 0.059(0.003) | 1.043(0.007) |0.330(0.005)|0.489(0.005)|\n\nRATD exhibits significant performance advantages, indicating that our method can better utilize references for prediction.\n\n**Q4**: How is the time series encoder Eϕ pre-trained with representation learning tasks (Line148)? Since the embedding quality highly affects the retrieval precision and quality, some analysis and evaluations should be conducted on the time series embedding quality.\n\n**A4**: We trained TCN for time series prediction tasks for pre-training. In Section 5.3（Influence of Retrieval Mechanism， line 240）of the paper, we extensively discussed the question of which encoder structure to adopt. I believe this discussion holds the same significance as the \"discussion on embedding quality\" you mentioned because the quality of embeddings is difficult to assess directly and can only be judged by comparing their contributions to experiment improvements. It is worth noting that through comparative experiments, we found that the differences in results brought by different encoders are not significant. This also demonstrates that our approach has strong robustness.\n\n**Q5**: What is the architecture of the pre-trained encoder Eϕ? How do you encode the multivariate time series into a single embedding? Do you use any pooling operation? More analysis and explanations on the encoder are beneficial for paper clarity.\n\n**A5**: Your concerns may be solved after reading Section 5.3（Influence of Retrieval Mechanism， line 240）. As we addressed above: we conducted a comprehensive discussion on the structure of the encoder. The TCN architecture supports the encoding of multivariate time series (resulting embeddings are not one-dimensional) without the need for additional pooling layers.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XvtqmQNnJH", "reviewer": "Reviewer_AGBG", "summary": "Existing time series diffusion models are unstable due to insufficient datasets and lack of guidance. The RATD model combines an embedding-based retrieval process with a reference-guided diffusion model to improve stability and accuracy. RATD retrieves relevant time series from a database to guide the denoising process, maximizing dataset utilization and compensating for guidance deficiencies. Experiments on multiple datasets demonstrate RATD's effectiveness, particularly in complex prediction tasks.", "questions": "- What is the main difference between RAG and proposed work?\n- How to define stability in time-series data? Someone may not agree with stability in Figure 1-(c). Some may disagree that the red plots are unstable.\n- Why necessarily the proposed RATD is needed in time-series forecasting? In other words, how to link the relationship between RATD and TSAD?\n- There is a typo in the last sentence of the second paragraph from Section 5.2. “.t”\n- Why Table 2 is separated from Table 1 independently?\n- What can you define as the best retrieval for forecasting?", "limitations": "They described their limitations themselves.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper introduces a creative combination of embedding-based retrieval and reference-guided diffusion. This approach is innovative in addressing the limitations of existing time series diffusion models.\n- The authors conducted extensive experiments on multiple datasets, demonstrating the effectiveness of RATD in complex prediction tasks. The results show that RATD outperforms existing methods in terms of stability and accuracy.\nThe paper provides a thorough explanation of the model architecture, retrieval mechanism, and training procedure, ensuring reproducibility.\n- The paper is well-organized, with clear sections on introduction, related work, methodology, experiments, and conclusions. The use of figures and tables enhances understanding.\n- The authors explain complex concepts in a concise manner, making the paper accessible to readers with varying levels of expertise.", "weaknesses": "- The proposed RATD lacks justification for architecture.\n- Please refer to the below questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*We thank Reviewer AGBG for the thorough and valuable feedback. We are glad that the reviewer found that the proposed model is effective and our paper is easy to read. The main concern of the reviewer is that the architecture of RATD lacks justification. Please see below for our responses to your concerns.* \n\n**Q1**: The proposed RATD lacks justification for architecture.\n\n**A1**: We here justify the effectivness of our architecture from following aspects:\n\n* We **for the first time** introduce the concept of RAG to time series diffusion models. In other words, and **introduces a new guiding mechanism** for time series diffusion models.  Describing time series data directly is challenging, and currently, there is no universally recognized effective guidance mechanism that guarantees the effectiveness of diffusion models in time series conditional generation tasks.\n* The reference-augmented mechanism we propose is highly effective.  Our model framework is based on CSDI (mentioned in Line 167), and building upon CSDI, our approach achieves superior performance. We have listed the performance improvement ratios (MSE) in the table below. \n\n| | Exchange |  Electricity | Wind | Weather | \n| :-----| :---- | :---- |:---- | :---- |\n| Performance improvement (%) | 79.22 |  60.18 | 26.45 | 21.06 | \n\n* The **RMA (Reference Modulated Attention, L167-180)** we propose is also robust and effective. In terms of robustness, the performance does not rely excessively on pre-trained encoders（from line 250 on paper). Additionally, we designed extra ablation experiments to demonstrate the effectiveness of RMA (Metric: MSE).\n|Dataset | Exchange|  Electricity | Wind | \n| :-----| :---- | :---- |:---- | \n| CSDI | 0.077  |  0.379 | 1.066 | \n| CSDI+Linear| 0.075 |  0.316  | 0.932  | \n| CSDI+Cross Attention| 0.028  |  0.173 |0.829 |\n| CSDI+RMA|0.013 |  0.151  | 0.784 | \n \nWhere CSDI represents the most basic network framework, CSDI+Linear denotes the approach where inputs and references are concatenated via a linear layer and fed into the network together, CSDI+CrossAttention signifies the use of cross attention to fuse features from inputs and references, and finally, CSDI+RMA, which incorporates an additional RMA. Clearly, RMA is highly effective in leveraging references for references.\n\n**Q2**: What is the main difference between RAG and the proposed work?\n\n**A2**: In our related work (section 2.2, line 76), we mentioned RAG and positioned our proposed RATD as a new application of RAG in the study of time series diffusion models. Compared to previous time series RAG methods, our approach still holds significant advantages. This advantage stems from the iterative structure of the diffusion model and our proposed Reference Modulated Attention, where references can repeatedly influence the generation process, allowing references to provide a informative guidance for the conditional generation process.\n\n**Q3**: How to define stability in time-series data? Someone may not agree with stability in Figure 1-(c)...\n\n**A3**: To validate our conclusions about 'stability,' we designed additional experiments. In these experiments, we calculated the variance of 15 repeated prediction results, which are presented in the table below. By evaluating the variance, we found that our method demonstrates a clear advantage in stability through the use of additional references for guidance.\n\n| |Exchange| Wind | Traffic | Weather|\n| :-----| :---- | :---- |:---- |:---- |\n|RATD（Ours）  | 0.013(0.001)  | 0.784(0.005)  | 0.151(0.002) |0.281(0.002) |\n|iTransformer| 0.016(0.001) | 0.932(0.007) |0.192(0.003) |0.358(0.003) |\n|PatchTST| 0.047(0.009) | 1.001(0.009) |0.225(0.003)|0.782(0.008)|\n|CSDI| 0.077(0.003) | 1.066(0.008) |0.379(0.003) |0.356(0.002)|\n\n**Q4**: Why necessarily the proposed RATD is needed in time-series forecasting? In other words, how to link the relationship between RATD and TSAD?\n\n**A4** : As we mentioned in **A1**, our method provides a new guiding mechanism for time series diffusion models, and it proves to be highly effective in experiments. Furthermore, our method is plug-in, allowing its application to other transformer-based time series diffusion models. The design principles and motivations behind RATD may also offer new insights for future work, particularly in designing novel guiding mechanisms. All of these aspects underscore the ‘necessity’ of our approach.\nIf TSAD refers to Time Series Anomaly Detection, directly applying the retrieval process may not yield optimal results, as the anchors used for retrieval could be anomalous. However, with appropriate design, it might be possible to cleverly mitigate this issue, thereby making retrieval-based methods potentially viable for TSAD.\n\n**Q5**: There is a typo in the last sentence of the second paragraph from Section 5.2. “.t”\n\n**A5**: I apologize for the typo. I will correct the paper again. Thank you for bringing it to my attention.\n\n**Q6**: Why Table 2 is separated from Table 1 independently?\n\n**A6**: The main reason we separated Table 1 and Table 2 is due to our use of different strategies for constructing the retrieval databases (as mentioned in line 211). Additionally, most of the popular baselines have not yet been evaluated on the MIMIC-IV dataset, given its later release. \n\n**Q7**: What can you define as the best retrieval for forecasting?\n\n**A7**: Generally, a good pre-trained time series encoder can be a good retriever because it can effectively embed the trend of time series. In this paper, we leverage state-of-the-art pre-trained encoder models, and use them for retrieving the most similar anchor time series as reference. Such reference can naturally be the best retrieval for the conditional forecasting, which have been further proved by our empirical results in paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tFgJi6iwpY", "reviewer": "Reviewer_uLZU", "summary": "This article proposes a retrieval-augmented diffusion model for time series prediction, featuring a simple and straightforward approach. It aims to address two key issues: \n\n1. The lack of semantics and labels in time series data, leading to insufficient guidance during the diffusion model's generation process. \n\n2. The problem of size insufficiency and imbalance in time series data. \n\nSpecifically, the method involves embedding-based retrieval to gather similar time series data from a database to serve as references during the denoising process. Their proposed RMA module performs feature fusion between the retrieved time series and the current diffusion step's results, thereby enhancing the denoising and generation process of the diffusion model.  \n\nIn the experimental section, they explore the impact of different retrieval methods on the performance of the diffusion model. They also investigate the effects of varying the size of the retrieval database and the number of retrieved sequences (k) on the prediction performance.", "questions": "1. What would be the result if the retrieved sequences were directly used for a simple weighted average for the final generation output, or if the performance of selecting the sequence closest to the ground truth (gt) from the retrieved ones was evaluated?\n2. I am uncertain whether the non-diffusion time-series prediction methods used in the author's comparative experiments include retrieval-enhanced modules. If not, I recommend adding experiments with this component to demonstrate the efficacy of the proposed method.\n3. In Formula 1, the symbol $\\mathcal{D}$ is used ambiguously, representing both distance and feature dimensions.", "limitations": "The approach may heavily rely on the capabilities of the embedding model and the construction of the database.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "An additional database was utilized to enhance the diffusion generation process. This method is straightforward and easy to follow. The newly proposed RMA module aids in integrating features from different datasets. The paper is clearly written and achieves state-of-the-art (SOTA) results on the majority of the datasets they selected.", "weaknesses": "The paper lacks innovation as the retrieval-enhanced diffusion method has already been proposed in the field of image generation, and similar retrieval-enhanced approaches exist in the time-series prediction domain, although their backbone is not a diffusion model. Additionally, the performance improvement largely stems from the representational capabilities of the pre-trained embedding model used.\n\nTypo in title: \"Augment\" -> \"augument\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*Thank you very much to reviewer uLZU for acknowledging some advantages of our proposed method. We understand you may have concerns regarding the overall novelty of the entire paper. We will address this point.*\n\n**Q1**: The paper lacks innovation as the retrieval-enhanced diffusion method has already been proposed in the field of image generation...\n\n**A1**： The innovation of this method is reflected in the following aspects：\n* We **for the first time utilize RAG on time series diffusion models**. In text2image diffusion models, cross-modal guidance mechanisms are well-established, and retrieval-enhanced methods emphasize using semi-parametric approaches to enhance generation efficiency. However, time series diffusion models lack a universally recognized effective guidance mechanism.  **We are the first to provide a promising RAG-based guidance mechanism for the diffusion generation of time series, which represents a pioneering advancement in time series diffusion model research**. Additionally, while existing work utilizes retrieval mechanisms for time series prediction, previous efforts have not adequately analyzed the use of references nor extensively evaluated and compared model performance in experiments. Our novel framework addresses these gaps in previous research and significantly advances the study of retrieval-enhanced time series prediction methods.\n* Our method **introduces a novel attentional mechanism for time series diffusion, called RMA (Reference Modulated Attention, L167-180)**, designed to enhance the guidance provided by references in the generation process. While the overall structure is not complex, our proposed RMA is effective and similarly groundbreaking. Additionally, we designed extra ablation experiments to demonstrate the effectiveness of RMA.\n |Dataset | Exchange|  Electricity | Wind | \n| :-----| :---- | :---- |:---- | \n| CSDI | 0.077  |  0.379 | 1.066 | \n| CSDI+Linear| 0.075 |  0.316  | 0.932  | \n| CSDI+Cross Attention| 0.028  |  0.173 |0.829 |\n| CSDI+RMA|0.013 |  0.151  | 0.784 | \n \nWhere CSDI represents the most basic network framework, CSDI+Linear denotes the approach where inputs and references are concatenated via a linear layer and fed into the network together, CSDI+CrossAttention signifies the use of cross attention to fuse features from inputs and references, and finally, CSDI+RMA, which incorporates an additional  RMA. Clearly, RMA is highly effective in leveraging references for guidance.\n\nRegarding the issue of dependency on pre-trained encoders, we discussed this in Section 5.3 (line 240). In the embedding-based retrieval mechanism, different encoders (feature extractors) do indeed introduce performance differences. However, these differences are not decisive. This variability may stem from the effectiveness of the methods we selected and the quality of the retrieved references. This also indicates that our method exhibits high robustness and does not rely excessively on the choice of pre-trained encoders.\n\n**Q2**: There is a typo in the last sentence of the second paragraph from Section 5.2. “.t”\n\n**A2**: I apologize for the typo. It was an oversight on our part, and I will review the paper again. Thank you for bringing it to my attention.\n\n**Q3**: What would be the result if the retrieved sequences were directly used for a simple weighted average for the final generation output...\n\n**A3**: Your suggestion is very interesting! Indeed, we paid close attention to this issue during our experimental phase. If the references are too similar to the ground truth (GT), the model's predictions can overly rely on the references, thereby failing to learn the true conditional distribution. We assessed this visually during experiments, such as in our analysis in Figure 4 from the paper. Following your advice, we conducted experiments on two datasets. \n\n|Dataset |Exchange| Wind | \n| :---- | :---- | :---- |\n|RATD（Ours）  | 0.013 | 0.784  | \n|Reference (Closest)| 0.153 | 2.487|\n|Reference (Average)| 0.197| 2.597 |\n\nSpecifically, \"Reference (Closest)\" refers to the closest reference. The results demonstrate that on datasets with strong periodic patterns, the references and predicted results are indeed very similar, whereas, on more complex datasets(wind), the references cannot replace the prediction at all.\n\n**Q4**: I am uncertain whether the non-diffusion time-series prediction methods used in the author's comparative experiments include retrieval-enhanced modules...\n\n**A4**: We did not compare our method with the previous RAG-based time series methods in the paper because these papers did not provide comprehensive evaluations on commonly used time series datasets or publicly available code. The implementation details in these papers were also very limited, making replication of their methods challenging. Displaying replicated results directly in the main text could potentially lead to unnecessary misinterpretation. Nevertheless, as you mentioned, for the sake of comprehensive experimental comparison, we present here the MSE results of our replicated method under our experimental settings, demonstrating that our approach shows noticeable advantages.\n\n| |Exchange| Wind | Traffic | Weather|\n| :-----| :---- | :---- |:---- |:---- |\n|RATD（Ours）  | 0.013(0.001)  | 0.784(0.005)  | 0.151(0.002) |0.281(0.002) |\n|MQRetNN| 0.063(0.004) | 1.116(0.008) |0.346(0.003) |0.668(0.004) |\n|ReTime| 0.059(0.003) | 1.043(0.007) |0.330(0.005)|0.489(0.005)|\n\nClearly, RATD exhibits significant performance advantages, indicating that our method can better utilize references for forecasting.\n\n**Q5**: In Formula 1, the symbol D is used ambiguously, representing both distance and feature dimensions.\n\n**A5**: I apologize for the mistake in using symbols. I will correct the paper and remove any potentially misleading parts.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GmHpYQcQRY", "reviewer": "Reviewer_FgUz", "summary": "This paper introduces the Retrieval-Augmented Time series Diffusion model (RATD) to address the limitations of existing time series diffusion models, such as insufficient datasets and lack of guidance. RATD consists of two components: an embedding-based retrieval process and a reference-guided diffusion model. The model retrieves the most relevant time series from a database to guide the denoising process, thereby enhancing the utilization of the dataset and providing meaningful guidance. Experiments on multiple datasets demonstrate the effectiveness of RATD, particularly in complex prediction tasks.", "questions": "- More comprehensive and detailed comparisons with existing studies\n  - Covering necessary datasets and both short and long horizons\n- Systematic ablation tests to illustrate your specific architecture designs\n  - Especially for Figure 3, more explanations and ablation tests are indispensable for readers to catch up", "limitations": "The idea of retrieval-augmented diffusion time-series model is interesting and reasonable.\nMy major concerns are about the incomplete experiments and the reproducibility.\n\nI may consider raising the score if my major concerns are properly addressed.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Novel Approach**: The idea of leveraging retrieved similar historical time series to guide forecasting for current timesteps is novel. References can potentially accelerate the convergence of the diffusion process and improve prediction accuracy.\n- **Guidance Mechanism**: The reference-guided mechanism compensates for the lack of guidance in existing time series diffusion models, which is a significant contribution to the field.", "weaknesses": "- **Incomplete Experimental Comparisons**: The experimental setup and comparisons are insufficient and not aligned with existing studies, making it difficult to verify the performance and correct implementation of this work. For example, the datasets and forecasting horizons used differ from those in classical studies like CSDI and TimeGrad. This discrepancy makes it challenging to ensure fair comparisons and reproducibility.\n  - In CSDI and TimeGrad, solar, electricity, traffic, taxi, wiki datasets and forecasting horizon=24 are used for comparisons. While this work adopts datasets of exchange, wind, electricity, and weather and only report the average results of horizons of 96, 192, and 336. \n  - Given that the code is also not provided at the current stage, though I like the idea, I am not sure about the reproducibility. Hence I suggest the author to align with existing classical studies and provide a more comprehensive experimental results.\n- **Insufficient Ablation Tests**: While Table 3 indicates the importance of choosing good retriever embeddings, there are no systematic ablation tests of the architecture developed (Figure 3). Detailed explanations of model architectures and extensive ablation tests are crucial to understanding the success of conditional diffusion and ensuring computational efficiency. The current paper lacks this information. It is easy for me to understand the conditional diffusion process. but it is extremely challenging for me to understand the insights and tricks in the architectures shown in Figure 3. I believe these are crucial to ensure the success of conditional diffusion and provide efficient computation. I cannot find any contents in the main paper or appendix informing me about these.\n  - Detailed explanation of model architectures and extensive ablation tests are important for me to effectively distinguish it from CSDI. For example, you can still follow the architecture of CSDI while only introducing the depdence on retrived references x^R.\n- **Unknown GPU Memory Usage**: There is no information on GPU memory usage and computation efficiency. Understanding how the introduction of reference series impacts computation and how these challenges are addressed by specific architectural designs is important for evaluating the practicality of the model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*Thank you very much to reviewer FgUZ for recognizing our proposed method. Following your suggestion, we have conducted additional experiments and provided further explanations of the model's structure. We hope to receive your approval.*\n\n**Q1**: The experimental setup and comparisons are insufficient and not aligned with existing studies, making it difficult to verify the performance and correct implementation of this work.\n\n**A1**: We extended the horizon length in our experiments, which is currently a reasonable practice as many popular forecasting methods [1][2] evaluate results by predicting horizon lengths of 96 and above. Of course, we agree with your point, so we supplemented the experiments according to the settings of CSDI, and the results are as follows.\n\n|Dataset | Traffic |  Electricity | Wind | \n| :-----| :---- | :---- |:---- | \n|Metric | MSE CRPS-sum |  MSE CRPS-sum | MSE CRPS-sum | \n| CSDI [3]| 0.027 0.020 |  0.112 0.017 | 0.462 0.042| \n| TimeGrad [4]| 0.041 0.044 |  0.121 0.021 |0.491 0.047 | \n| RATD(Ours)|0.023 0.018 |  0.096 0.014 | 0.311 0.031 | \n \nWe compared CSDI, TimeGrad, and RATD under the experimental setting of CSDI (with a prediction window length of 24). It is noteworthy that in this experimental setup, the differences in results among different methods were narrowed. This may be due to the reduced difficulty of the task with a smaller prediction window. From the extra experiment, our method still exhibits obvious advantages.\n\n **W2**: Insufficient Ablation Tests: While Table 3 indicates the importance of choosing good retriever embeddings, there are no systematic ablation tests of the architecture developed (Figure 3). \n\n **A2**: Thank you very much for your suggestions! I truly appreciate that you took the time to read our paper thoroughly and provided very constructive feedback! To validate the ideas you mentioned, we conducted additional ablation experiments and performed preliminary tests on three datasets. The results are shown in the table below:\n \n|Dataset | Exchange|  Electricity | Wind | \n| :-----| :---- | :---- |:---- |\n| CSDI [3]| 0.077  |  0.379 | 1.066 | \n| CSDI+Linear| 0.075 |  0.316  | 0.932  | \n| CSDI+Cross Attention| 0.028  |  0.173 |0.829 |\n| CSDI+RMA|0.013 |  0.151  | 0.784 | \n \nWhere CSDI represents the most basic network framework, CSDI+Linear denotes the approach where inputs and references are concatenated via a linear layer and fed into the network together (following [6]), CSDI+CrossAttention signifies the use of cross attention to fuse features from inputs and references (following [5]), and finally, CSDI+RMA, which incorporates an additional Reference Modulated Attention (RMA).\n\nThrough our experiments, we found that compared to the basic Cross-attention-based method, RMA can integrate an edge information matrix (representing correlations between time and feature dimensions) more effectively. The extra fusion is highly beneficial in experiments, guiding the model to capture relationships between different variables. In contrast, linear-based methods concatenate inputs and references initially, which prevents the direct extraction of meaningful information from references, resulting in comparatively modest performance.\n \n **W3**: There is no information on GPU memory usage and computation efficiency...\n\n **A3**: As mentioned in our conclusion and limitation section, the retrieval process incurs additional computational costs. To minimize this cost, we pre-retrieve references for all samples during training and record the corresponding indices (as noted in line 215 of the paper). This means that during actual training and inference, we only need to consider the additional cost introduced by RMA. Since you are concerned about this aspect, we provide a rough overview of GPU memory usage and computational efficiency (all the experimental settings are the same as the paper, we complete the comparison on the wind dataset).\n|| GPU memory usage (training)|  Time Cost (for test batch)| \n| :-----| :---- | :---- |\n| CSDI | ~11.1 (GB) |  ~240s |\n| RATD (ours)| ~12.3 (GB) | ~270s  | \n \nIn general, compared to CSDI, our method has a slight disadvantage in terms of memory consumption and computational efficiency. However,  this disadvantage is not significant and does not pose serious challenges to applications.\n\n **Q4**: More comprehensive and detailed comparisons with existing studies...\n\n **A4**: We have provided supplementary experimental results above in hopes of addressing your concerns. Thank you.\n\n **Q5**: Systematic ablation tests to illustrate your specific architecture designs...\n\n **A5**: Similarly, we hope our ablation experiments have addressed your concerns. If you have any further questions, please feel free to reply. Thank you.\n\n \n[1]Liu Y, Hu T, Zhang H, et al. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting[C]//The Twelfth International Conference on Learning Representations.\n[2]Nie Y, Nguyen N H, Sinthong P, et al. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers[C]//The Eleventh International Conference on Learning Representations.\n[3]Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34:24804–24816, 2021. \n[4]Rasul K, Seward C, Schuster I, et al. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting[C]//International Conference on Machine Learning. PMLR, 2021: 8857-8868.\n[5]Yang S, Eisenach C, Madeka D. MQ-ReTCNN: Multi-Horizon Time Series Forecasting with Retrieval-Augmentation[J]. 2022.\n[6]Jing B, Zhang S, Zhu Y, et al. Retrieval based time series forecasting[J]. arXiv preprint arXiv:2209.13525, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8wGyyvVUNr", "reviewer": "Reviewer_zh2V", "summary": "This paper proposed Retrieval-Augmented Time series Diffusion (RATD) model for complex time series forecasting tasks, and designed an Extra Reference Modulated Attention (RMA) module to enable the guidance from the retrieved reference time series during the denoising process of the diffusion models. On five real-world datasets, this paper demonstrated that RATD performed on-par or better on multiple metrics compared to various time series forecasting baselines including diffusion models, transformers, and other methods.", "questions": "1. In section 4.2, how is $D^{R}$ different from $D^{R'}$ if $\\mathcal{C}$ represents all categories? In 5.3, it seems that when it's $D^{R'}$, only $n$ samples from each category are used to construct the database. Is $n$ a hyperparameter for $D^{R'}$ that also needs to be introduced in section 4.2?\n2. In section 4.3, under $\\textbf{Denoising Network Architecture}$, it is mentioned that all references are concatenated together for feature extraction. I am wondering how this scales with the length of time series, i.e. would there be memory concerns if you are working with long-horizon time series forecasting / long context windows?", "limitations": "The authors discussed the limitations of this work with regards to computational costs incurred by the transformer based diffusion model and the retrieval process, which is also my concern.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper is well-written and easy to follow.\n2. The methodology design is clear and reasonable to me.\n3. The experiments are comprehensive, and demonstrated the effectiveness of RATD across various retrieval mechanisms on multiple time series datasets using multiple metrics.\n4. In the experiment results, RATD is effective with k = 3 retrieved reference time series, and the inference time is also on the lower side compared to other baselines, which showed that RATD should be relatively computationally scalable and efficient.", "weaknesses": "1. Based on the related work section, there have been several works which investigated  time series forecasting using diffusion models and achieved promising results. RAG is also a well-studied topic in text generation. Given this, I think the technical novelty of this work is moderate, though it's a nice direction to combine RAG with time series forecasting and the experiment results demonstrated its effectiveness. I am wondering whether there is any previous work on time series RAG?\n2. Probably out of the scope of this paper, it would be great if the authors can provide some discussion or intuitive insights on the advantage of using diffusion models for time series forecasting compared to other methods, e.g. is it more robust or generalizable, and also the advantage of diffusion models when it comes to retrieval augmented forecasting. As in table 1, several transformer based methods also performed well on some datasets, thus I am wondering whether there is any justification in choosing diffusion models for time series forecasting given its potential extra computational cost.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*Thank you very much to reviewer zh2V for the careful reading and consideration, as well as for acknowledging the innovative aspects of our approach* .\n\n**Q1**: I am wondering whether there is any previous work on time series RAG?\n\n**A1**: In the related work section, I mentioned that previous studies have combined time series prediction with RAG[1][2]. Our proposed method differs from these prior works in several key aspects:\n\n**Different Utilization of References**: Previous methods either directly fused features using existing cross-attention modules or concatenated them for fusion. In contrast, we introduce a novel RMA(Reference Modulated Attention) module for feature fusion.\n\n**Distinct Research Motivation**: We believe that time series diffusion models may offer advantages over transformer networks in certain prediction tasks. However, these advantages are not always pronounced (as mentioned in your subsequent question). We identify the lack of guidance mechanisms as a major barrier to achieving greater performance with diffusion models. Due to this absence, diffusion models must directly learn the mapping from Gaussian distribution to data distribution, which can be challenging. We propose an enhanced retrieval-guided mechanism and aim to provide new insights for future research on time series diffusion models, particularly in the study of guidance mechanisms.\n\n**Differences in Experimental Completeness**: Previous studies did not provide comprehensive evaluations of public datasets, or experiment details, making it difficult to assess model performance. for the sake of comprehensive experimental comparison, we present here the MSE results of our replicated method under our experimental settings, demonstrating that our approach shows noticeable advantages.\n|Dataset |Exchange| Wind | Traffic | Weather|\n| :-----| :---- | :---- |:---- |:---- |\n|RATD（Ours）  | 0.013(0.001)  | 0.784(0.005)  | 0.151(0.002) |0.281(0.002) |\n|MQRetNN[1]| 0.063(0.004) | 1.116(0.008) |0.346(0.003) |0.668(0.004) |\n|ReTime[2]| 0.059(0.003) | 1.043(0.007) |0.330(0.005)|0.489(0.005)|\n\nClearly, RATD exhibits significant performance advantages, indicating that our method can better utilize references for prediction.\n\nIn summary, while there are similarities between prior works and our method to some extent, there are significant differences as well. We believe our approach advances the integration of RAG with time series prediction tasks.\n\n**Q2**:  I am wondering whether there is any justification in choosing diffusion models for time series forecasting given its potential extra computational cost.\n\n**A2**: The diffusion model is a generative framework and the transformer is a specific model structure. Two methods follow different technical paths, making it challenging to assess their relative performance. To be specific, many existing efforts focus on enhancing performance by updating the architecture of transformers [3], while related works on diffusion models concentrate on updating the framework as a whole [4].  However, I believe time series diffusion models hold greater research potential.\nFirstly, time series diffusion models are currently constrained by inference costs, preventing the direct adoption of powerful time series transformer models as their denoising network. In the future, if transformer models with significantly lower inference costs and high performance emerge, the performance of diffusion models could have substantial improvements.\nSecondly, current time series diffusion models lack clear and effective guidance mechanisms. This work is among the few that address this issue, and I believe that appropriate guidance mechanisms could greatly enhance the performance of diffusion models (as demonstrated in the experimental section of this paper). Further research into guidance mechanisms may activate the potential of time series diffusion models in the future.\n\n**Q3**: In section 4.2, how is DR different from DR′ if C represents all categories? Is n a hyperparameter for DR′ that also needs to be introduced in section 4.2?\n\n**A3**: In fact, we employed two different methods for constructing retrieval datasets, primarily to correspond to two types of time series datasets: one smaller and lacking clear category labels, such as wind, electric...; and one with complete category annotations, often larger, such as MIMIC-IV. For the former, we could only use the first method to construct the retrieval dataset, while for the latter, both methods were applicable. If all category data from the latter case were applied, as you mentioned, then there would be no difference between the two construction methods. \n\nAdditionally, you are correct in noting that the sample counts for each category should also be included in the formula. Thank you for your correction, and I will review it again.\n\n**Q4**: ...would there be memory concerns if you are working with long-horizon time series forecasting / long context windows?\n\n**A4**: Your concerns are reasonable, and we have considered them. Therefore, we used a linear layer (shown on the right side of Figure 3) to reduce the length of the concatenated sequence, which facilitates subsequent computations. We will emphasize this point in the final version of the paper.\n\n[1]Yang S, Eisenach C, Madeka D. MQ-ReTCNN: Multi-Horizon Time Series Forecasting with Retrieval-Augmentation[J]. 2022.\n[2]Jing B, Zhang S, Zhu Y, et al. Retrieval-based time series forecasting[J]. arXiv preprint arXiv:2209.13525, 2022.\n[3] Zhang Y, Yan J. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting[C]//The eleventh international conference on learning representations. 2022.\n[4] Rasul K, Seward C, Schuster I, et al. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting[C]//International Conference on Machine Learning. PMLR, 2021: 8857-8868.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all the reviewers for the thorough reviews and valuable feedback. We are glad to hear that the idea is novel or innovative (Reviewer HJsX, AGBG, and FgUz ), this paper is well-written and easy to follow (Reviewer HJsX, AGBG, uLZU and hz2v), and provide clear explanations for the proposed architecture (Reviewer HJsX, AGBG and hz2v).

Here, we want to highlight the novelty and contributions of our proposed method as follows:

* **First retrieval-augmented time series diffusion model**: We for the first time introduce a retrieval-augmented framework for time series diffusion model. The framework is concise, effective, and robust. RATD will inspire future research in time series diffusion models.
* **Specialized attention mechanism for reference-guided time series diffusion**: We introduce a new attentional mechanism (Reference Modulated Attention, RMA) to effectively utilize references for guiding the generation process. Compared to the cross-attention module, RMA more effectively integrates multiple features, making it a better for diffusion-based time series forecasting.
* **Comprehensive evaluation**: We evaluate our approach on five real-world datasets, consistently achieving state-of-the-art performance.

We summarized our responses to reviewers as follows:

* We analyzed the differences between our method and the existing time-series RAG method, and quantified these differences through experiments. (Reviewer HJsX, uLZU and hz2v)

* We supplemented the paper with ablation experiments for the newly proposed module (Reference Modulated Attention, RMA) to validate its effectiveness. (Reviewer AGBG and Fguz)

* By forecasting repeatedly, we obtained variance in the prediction processes for evaluation, providing the basis for model stability analysis. (Reviewer HJsX and uLZU)

* We provided more explanations for the innovation of the proposed method and the issue of the performance being overly reliant on pre-trained models. (Reviewer AGBG, uLZU and hz2v)

We reply to each reviewer's questions in detail below their reviews. Please kindly check out them. Thank you and please feel free to ask any further questions.

Meta-review: This well written paper has been assessed by five knowledgeable reviewers. Four of them leaned towards accepting it (one full accept, one weak accept and two borderline accept ratings), while one leaned the other way (borderline rejection). The reviewers predominantly highlighted novelly of the presented approach and clarity of its presentation.  One of the reviewers expressed a dissenting opinion on the novelty of the proposed approach that is a clever combination of otherwise well established techniques. The authors provided a comprehensive rebuttal in response to the initial critique and engaged the reviewers in discussions, which all helped resolve most of the concerns. Given the prevailing sentiment, with which I agree, I conclude that this work satisfies the requirements for being presentable at NeurIPS.
Decision: Accept (poster)