Retrieval Augmented Time Series Forecasting
Authors: Sungwon Han
Seungeon Lee
Meeyoung Cha
Sercan O Arik
Jinsung Yoon
Venue: ICML 2025 poster (2025)
Keywords: Time series forecasting
Retrieval augmented model
Deep learning
Primary area: deep_learning->sequential_models_time_series

Abstract: Time series forecasting uses historical data to predict future trends, leveraging the relationships between past observations and available features. In this paper, we propose RAFT, a retrieval-augmented time series forecasting method to provide sufficient inductive biases and complement the model's learning capacity. When forecasting the subsequent time frames, we directly retrieve historical data candidates from the training dataset with patterns most similar to the input, and utilize the future values of these candidates alongside the inputs to obtain predictions. This simple approach augments the model's capacity by externally providing information about past patterns via retrieval modules. Our empirical evaluations on ten benchmark datasets show that RAFT consistently outperforms contemporary baselines with an average win ratio of 86%.

Retrieval Augmented Time Series Forecasting
Sungwon Han 12 Seungeon Lee *2 Meeyoung Cha
Sercan Ö. Arik
Jinsung Yoon"
Abstract
Use retrieved results alongside the input
Time series forecasting uses historical data to predict future trends, leveraging the relation- ships between past observations and available features. In this paper, propose RAFT, retrieval-augmented time series forecasting method to provide sufficient inductive biases and complement the model's learning capacity. When forecasting the subsequent time frames, we di- rectly retrieve historical data candidates from the training dataset with patterns most similar the input, and utilize the future values of these can- didates alongside the inputs to obtain predictions. This simple approach augments the model's 's ca- pacity by externally providing information about past patterns via retrieval modules. Our empirical evaluations on ten benchmark datasets show that RAFT consistently outperforms contemporary baselines with average win ratio of 86%.
MMa, Wem
W
Forecast
Retrieve relevant historical patters
Figure 1. Illustration of motivating example of retrieval in time- series forecasting.
effectivvness in capturing patterns of change in historical observations, leading to the development of various deep learning models tailored for time series forecasting, Espe- cially, the advent of attention-basse transformers (Vaswani et al., 2017) has made a significant impact on the time series domain. The architecture has shown be effective in modeling dependencies between inputs, resulting in variants like Informer (Zhou et al., 2021), AutoFormer (Wu et al., 2021), and FedFormer (Zhou et al., 2022). Additionally, recent methods utilize time series decomposition (Wang et al., 2023), which isolates trends or seasonal patterns, and multi multi-ppriodicity analysis which involves downsam- pling/upsampling of the series at various periods (Lin et al., 2024; Wang al., 2024). Furthermore, lightweight models like multi-layer perceptrons (MLP) have demonstrated strong performance along with these decomposition techniques and multi-periodiciyy analysis (Chen et al., 2023; Zeng et al., 2023; Zhang et al., 2022).
1. Introduction
Accurately predicting future trends is crucial for making informed decisions in various fields. Time series forecast- ing, which analyzes past data to anticipate future outcomes, plays vital role in diverse areas like climate model- ing (Zhu & Shasha, 2002), energy (Martín al., 2010), economics (Granger& & Newbold, 2014), traffic flow (Chen et al., 2001), and user behavior (Benevenuto et al., 2009). By providing reliable predictions, it empowers us to develop effective strategies and policies across these domains.
However, real-world time series exhibit complex, non- stationary patterns with varying periods and shapes. These patterns may lack inherent temporal correlation and arise from non-deterministic processes, resulting in infrequent repetitions and diverse distributions (Kim et al., 2021). This raises concerns about the effectiveness of models in extrap- olating from such infrequent patterns. Moreover, the advan- tages indiseriminately memorizing all patterns, including noisy and uncorrelated ones, are questionable terms of both generalizability and efficiency (Weigend al., 1995).
Over the past decade, deep learning models such as CNNs (Bai et al., 2018; Borovykh et al., 2017) and RNNs (Hewamalage et al., 2021) have proven their
"Equal contribution 'Department of AI Convergence, GIST, Gwangju, South Korea. This work is done while the author was in KAIST. "School of Computing, KAIST. Daejeon, South Ko- rea 'Daaa Science for Humanity Group, Max Planck Institute for Security and Privacy, Bochum, Germany "Google Cloud AI, Sun- nyvale, United States. Correspondence to: Jinsung Yoon <jin- sungyoon@ googlc.coo........
We show advancement in time-series forecasting mod- els by expanding the models' capacity (implicitly via the trained weights) to learn patterns. We directly provide infor- mation about historical patterns that are complex to learn, as a way of bringing relevant information via the input to reduce the burden on the forecasting model. Inspired
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Retricval Augmented Time Series Forecasting
by the retrieval-augmenned generation (RAG) approaches used in large language models (Lewis et al., 2020), our method retrieves similar historical patterns from the training dataset based on given inputs and utilizes them along with the model's learned knowledge to forecast the next time frame (see Figure 1).
categorized based on the employed architecture. Prior to the advent of transformers (Vaswani et al., 2017), time series analysis often relied on CNNs to capture local temporal pat- terns (Bai et al., 2018; Borovykh et al.. 2017) or RNNs to model sequential dependencies (Hewamalage et al., 2021). Following the advent of transformers, several approaches emerged to better tailor the transformer architecture for time-series forecasting. For example, LogTrans (Li et al., 2019) used convolutional self-attention layer, while In- former (Zhou et al., 2021) employed ProbSparse attention module along with distilling technique to efficiently reduce network size. Both Autoformer (Wu et al., 2021) and FedFormer (Zhou et al., 2022) decomposed time series into components like trend and seasonal patterns for prediction.
Our new approach, Netrieval-Augmeene Forecasting of Time-series (RAFT), offers two key advantages. First, by directly utilizing retrieved information, the useful patterns from the past become explicitly available at inference time, rather than utilizing them via the learned information in model weights. Learning hence covers patterns that lack temporal correlation or do not share common characteristics with other patterns, thereby reducing the learning burden and enhancing generalizability. Second, even if a pattern rarely appears in historical data and is difficult for the model to memorize, the retrieval module allows the model to easily leverage historical patterns when they reappear (Miller et al., 2024; Laptev et al., 2017).
Despite advancements in transformer-based models, (Zeng et al., 2023) reported that even simple linear model can achieve strong forecasting performance. sequently, lightweight MLP-based time-series models such as TiDE (?), TSMixer (Chen et al., 2023), and TimeMixer (Wang et al., 2024) were introduced with the ad- vantages in both forecasting latency and training efficiency. These models utilize various approaches such as series de- composition similar to transformer-bass et al., 2023) or introduced multi- qulti-periodicity analysis by downsampling or upsampling the series at various period intervals (Lin et al., 2024), to accurately extract the rele- vant information from time-scries for MLPs to effectively fit on them. Recently, several studies have constructed a large time-series databases to build large foundation models, achieving strong zero-shot and few-shot performance (Das et al., 2024; Woo et al., 2024).
Sub-
We demonstrate that the proposed judiciously-desiinedd inductive bias, implemented through a simple retrieval module, enables an MLP architecture to achieve strong forecasting performance. Inspired by existing literature that downsamples series at various period intervals (Lin et al., 2024; Wang et al., 2024), RAFT also generates multiple series by downsampling the given series at different periods and attaches a retrieval module to each series. This allows for effectively capturing both short-term and long-tern pat- terns for more accurate forecasting. As demonstrated on ten time-series benchmark datasets, RAFT outperforms other contemporary baselines with average win ratio of 86%. Overall, our contributions can be summarized as follows:'
studies (Zeng
Our proposed RAFT is based on shallow MLP architecture, following simplicity and efficiency motivations. Through the retrieval module, the model retrieves subsequent pat- terns that follow the patterns most similar to the current input from the single time series, allowing it to reference past patterns for future predictions without the burden of memorizing all temporal patterns during training. Our re- trieval differs from transformer variants that typically learn relationships only within a fixed lookback window. RAFT goes beyond the lookback window by retrieving relevant data points from the entire time series and incorporating them into the input.
We propose a retrieval-augmeeted time series forecast- ing method, RAFT, which retrieves observations with similar temporal patterns from the training dataset and effectively leverage retrieved patterns for future predictions.
Our empirical studies on ten different benchmark datasets show that RAFT outperforms other contem- porary baselines with an average win ratio of 86%.
We further explore the scenarios where retrieval mod- ules can be beneficial for forecasting by conducting analyses using synthetic and real-world datasets.
2.2. Retrieval augmented models
Retrieval-augmenned models typically work by first retriev- ing relevant instances from a dataset based on given input. Then, they combine the input with these retrieved instances to generate prediction. Retrieval-augmenter generation (RAG) in natural language domain is an active research area that utilizes this scheme. (Lewis et al., 2020; Guu et al., 2020). RAG retrieves document chunks from external
2. Related Work
2.1. Deep learning for time-series forecasting
large body of research employs deep learning for time-series forecasting. Existing methods can be broadly
Code in https://gttthhiieee thub. 101.oonn19//10 59//RFT
2

---

Retricval Augmented Time Series Forecasting
corpora that are relevant the input task, helping large lan- guage models (LLMs) generate responses related the task without hallucination (Shuster et al., 2021; Borgeaud et al., 2022). This not only supplements the LLM's limited prior knowledge but also enables the L.MM to handle complex, knowledge-intensive tasks more effectively by providing additional information from the retrieved documents (Gao et al., 2023).
weighted sum (Sec. 3.2). The main difference of our model from attention- forecasting models, such transform- ers, lies in its ability to retrieve relevant data from the entire time series rather than relying on a fixed lookback window. Since the time series shows distinct characteristics across periods, we utilize the retrieval modules into multiple peri- ods. RAFT generates multiple time series by downsampling the time series S with different periods and applies the re- trieval module to each time series. The retrieval results from multiple series are processed through linear projection and aggregated by summation. Finally, the input and the aggre- gated retrieval result are concatenated and passed through a linear model to produce the final prediction (Sec. 3.3). Details of each component are described below.
Beyond natural language processing, retrieval-augmented models have also been used to solve structured data prob- lems. A simple illustrative example is the K-nearest neigh- bor model (Zhang. 2016). Other approaches have intro- duced kernel-based neighbor methods (Nader et al., 2022), prototype-band approaches (Arik& Pfister, 2020), or con- sidered all training samples as retrieved instances (Kossen et al., 2021). More recently, models leveraging attention- like mechanisms have incorporated the similarity between retrieved instances and the input into the prediction, achiev- ing superior performance compared to traditional deep tab- ular models (Gorishniy et al., 2024). There also exists a method that has explored the potential of retrieving similar entities in time-series forecasting, involving multiple time series entities (Iwata& & Kumagai, 2020; Yang al., 2022). Assuming the training set contains various types of time series entities, they aggregate the information needed for each entity's s prediction based on the similarities across all time series entities.
3.2. Retrieval module architecture
We transform the time series S to be appropriate for re- trieval. First, we find all key patches within S that are to be compared with given x E RCxL Using the sliding window method of stride 12, we extract patches of window size L and define this collection as K. {kı,... ..., kT- (1-((++))1) } where i indicates the starting time step of the patch k, E RCxL, Note that any patch that overlaps with the given x must be excluded from K. during the training phase. Then, we find all value patches that sequentially follow each key patch k; E K. in the time series. We define the collection of value patches as V {Vi,..., VT-(L+F)+1)) J. where each V, RCCF sequentially follows after k, in the time series.
In this paper, we aim to demonstrate that retrieval can be ef- fective, even when applied to the single time-series. Similar to how RAG supplements LLMs with additional informa- tion for knowledge-intensive tasks, our approach seeks to reduce the learning complexity in time-series forecasting. Instead of forcing the model to learn every possible com- plex pattern, the retrieval module provides information that simplifies the learning process.
After preparing the key patch set KC and value patch set V for retrieval, we use the input x as a query to retrieve similar key patches along with their corresponding value patches with following steps. We first account for the distributional deviation between the query, key, and value patches used in the retrieval process. Let define x = (x')1e (e where x' e RC denotes the values of C variates at t-th time
step within the input x (i.e., xt = {zi, ..., =x&)). }). Inspired by existing literature (Zeng et al., 2023), we treat the final time step value in each patch as an offset and subtract this value from the patch as a form of preprocessing to make the patterns more meaningful to compare:
3. Method
3.1. Overview
Problem formulation. Given single time series S E RCT of length T with C' observed variates (i.e., channels), RAFT utilizes historical observation x E CxL to predict future values y E CxF that is close to the actual future values yo E C L denotes look-back window size and F denotes forecasting window size.
X
[x'
x- hE(()....))
where & represent the input queries with the offset sub- tracted. Similarly, we subtract the offset from all key patches k; E IC and v, E V. denoting them as k, E IC and v, E D. respectively. Then, we calculate the similarity P; between given and all key patches in IC using similarity function s:
Given input x, RAFT utilizes retrieval module to find the most relevant patch from S. Then, the subsequent patches of the relevant patch are retrieved as additional information for forecasting. The retrieval process follows an attention-like structure, where the importance weights are calculated based on the similarity between the input and the patches, and the retrieved patches are aggregated through
p: = x(x k, E K.
Here, we use Pearson's 's correlation as a similarity function s to exclude the effects of scale variations and value offsets
The stride can be adjusted according the demand of compu- tational efficiency.
3

---

Retrieval Augmented Time Series Forecasting
Time Series (S)
{
Values
Attention weight
Sliding window
Viw myy
AA
Retrieved Results
{
Keys
Top-m selection
{
Corn W
Key Value
SoftMax
Input
query
Candidate Set
Retrieval module
Figure 2. Illustration of retrieval module architecture. First, we consider consecutive time frames from the entire time series S as key-value pairs and construct a candidate set using a sliding window approach. Given an input time series as the query, the retrieval module computes the similarity between the query and the keys in the candidate set that do not overlap temporally. Based the similarity, the top-m candidates are selected, and attention weights are calculated via SoftMax. The final result obtained through a weighted sum of the corresponding values.
in the time series, focusing on capturing the increasing and decreasing tendencies We then retrieve the patches with top-m correlation values:
where linear projections f maps I to IRF, 9 maps R to RF, h maps 2RF to RF, and represents concatenation operation.
Multiple periods. Time series at different periods display unique characteristics patterns in a small time window typically reveal local patterns, while patterns a large time window might correspond to global trends. We extend the retrieval process consider periods P. For each p P, downsample the query all key patches in KC, and all value patches in V of period by average pooling with period p. This results in x(p) E RC K(P),, and V(P) as the respective query, key patch set, and value patch set for period p. where a key patch k.") E RCx151 and a value patch v(P) C f1, Then, we conduct the retrieval process described in Sec. 3.2 using x(P), K(P), and v(o), and obtain the retrieval result (() E c for each p. Each (P) is processed through linear layer glp) to project all retrieval results in the same embedding space, mapping I to RFF,, respectively. Finally, we concatenate & with sum of linear projections and process through lincar predictor h, which replaces Eq. 6 to following equation:
J = arg top-m (fpi S (CI)),
(3)
where T denotes the indices of top-m patches. Given temperature calculate the weight of value patches with following equation:
exp (p: E exp (p5/7)>
if J otherwise
W/
(4)
0.
Note that this is equivalent to conduct SoftMax only with top-m correlation values. Finally, we obtain the final retrieval result V E RCXF as the weighted sum of value patches:
V
Σ
w; V..
(5)
RE(I....IVD
Figure illustrates the architecture of our retrieval module.
ý = h([(())) e Σ (((())
3.3. Forecast with retrieval module
PEP
Single period. Consider the given input x E RCxL and the retrieved patch & e CCF Similar to the retrieval module, subtract the offset from and define & the input with the offset removed. Next, concatenate f(x) with g(v), and process concatenated result through h to obtain y:
Denoting ÿ as the value at the t-th time step within ý, we restore the original offset by adding x to ÿ, resulting in the final forecast y:
ÿ fxf()) @ g(V)),
(6)
We train the model by minimizing the following MSE loss:
'See Appendix C.I for comparison results with different simi- larity metrics.
L MSE(y, Yo)

---

Retrieval Augmented Time Series Forecasting
: Key
Retrieval
Linear Projection
Value
Module
Input
Retrieval Module
Linear Projection
Linear Predictor
Concat
Prediction
Retrieval
Linear
111 Mi
w
Module
Projection
Linear
Projection
Figure 3. Illustration of the proposed architecture, RAFT. The input time series and the entire past observed time series S are first downsampled to generate multiple series with different periods. Then, a retrieval module is applied to each series to retrieve information relevant to the current input. The retrieved results projected the dimension via linear layer, and the results from different periods are summed to aggregate the information. Finally, the input time series is concatenated with the aggregated retrieved results, and a linear layer is applied produce the final prediction.
Figure 3 illustrates our model's forecasting process with multiple periods of retrieval. Hyperparameters such as m chosen based the performance in the validation set.
consists 21 weather-related indicators in Germany over one year". Data summary provided in the Appendix A.
Baselines. We compare against contemporary time-series forecasting baselines, including: (1) Autoformer (Wu et al., 2021), (2) Informer (Zhou et al., 2021), (3) Sta- tionary (Liu et al., 2022b), (4) Fedformer (Zhou et al., 2022), and (5) PatchTST (Nie et al., 2023), all of which use Transformer-baset architectures; (6) DLinear (Zeng et al., 2023), which are lightweight models with simple linear architectures; (7) MICN (Wang al., 2023), which leverages both local features and global correlations through convolutional structure; (8) TimesNet (Wu et al., 2023), which utilizes Fourier Transformation to decompose time-series data within a modular architecture; and (9) TimeMixer (Wang al., 2024), which utilizes decomposition and multi-periodicity for forecasting'.
4. Experiments
We evaluate RAFT across multiple time series forecasting benchmark datasets. We analyze how our proposed retrieval module contributes to performance improvement time- series forecasting, and in which scenarios retrieval partic- ularly beneficial. The full results, visualizations, and addi- tional analyses of our model are provided in the Appendix.
4.1. Experimental settings
Datasets. We consider ten different benchmark datasets, each with diverse range of variates, dataset lengths, and frequencies: (1-4) The ETT dataset contains 2 years of electricity transformer temperature data, divided into four subsets ETTh1, ETTh2, ETTm1, and ETTm2 (Zhou et al., 2021); (5) The Electricity dataset records household electric power consumption over approximately years (Trindade, 2015); (6) The Exchange dataset includes the daily exchange rates of eight countries over 27 years (1990-2016 (Lai et al., 2018); (7) The Illness dataset includes the weekly ratio of pa- tients with influenza-like illness over 20 years (2002-2021)4- (8) The Solar dataset contains 10-minute solar power fore- casts collected from power plants in 2006 (Liu et al., 2022a);
Implementation details. RAFT employs the retrieval mod- ule with following detailed settings. The periods are set to {1, 2, 4) (n = 3), following existing literature (Wang et al., 2024), and the temperature 7 is set to 0.1. Batch size is set to 32. The initial learning rate, the number of patches used in the retrieval (m), and the size of the look-back window (T.) are determined via grid search based on performance on the validation set, following the prior work (Wang et al., 2024). For fair comparison, pprempprreette tuning was performed for both our model and all baselines using the validation set. The learning rate is chosen from le-5 to 0.05, look back
(9) The Traffic dataset contains hourly road occupancy rates on freeways over 48 months²: and (10) The Weather dataset
'httpa: www.. bgw.bbg---e jena.,, .pg-- de/ /etttettr///// "We compare our model with general time-series forecasting models. Other retrieval-based time-series models mentioned in the related work assume the presence of multiple time-series instances, which are outside the scope of study.
hhttps://gist edc. hpvp/grasp////oo http/grrap//fluieee// fluview/ auuportaldashbaard.oo html 'https://pems.d dot ca.gov/
5

---

Retrieval Augmented Time Series Forecasting
Table Comparison RAFT and baseline methods across 10 datasets using MSE. For all datasets except Illness, results are averaged over forecasting horizons of 96, 192, 336, and 720. For the Illness dataset, forecasting horizons of 24, 36, 48, and 60 are used. Best performances are bolded, and our framework's performances, when second-hest, are underlined.
Methods
RAFT
TimeMixer
PatchTST
TimesNet
MICN
DLinear
FEDformer
Stationary
Autoformer
Informer
ETTh1
0.420
0,447
0,516
0.495
0.475
0,461
0.498
0.570
0.496
1.040
ETTh2
0.359
0.364
0.391
0.414
0.574
0.563
0.437
0.526
0.450
4,431
ETTml
0.348
0,381
0.406
0.400
0,423
0,404
0.448
0,481
0.588
0.961
ETTm2
0.254
0.275
0.290
0.291
0.353
0.354
0.305
0.306
0.327
1,410
Electricity
0.160
0,182
0.216
0.193
0.196
0,225
0.214
0.193
0.227
0.311
Exchange
0.441
0.386
0.564
0,416
0.315
0.643
1.195
0.461
1.447
2.478
Illness
2.097
2.024
1.480
2.139
2.664
2.169
2.847
2.077
3.006
5.137
Solar
0.231
0,216
0.287
0,403
0.283
0.330
0.328
0.350
0.586
0.331
Traffic 0.434
0.484
0.529
0.620
0.593
0.625
0.610
0.624
0.628
0.764
Weather 0.241
0,240
0.265
0,251
0.268
0.265
0.309
0.288
0.338
0.634
window size from (96, 192, 336, 720}, and the number of
using both benchmark time series datasets and synthetic time series datasets.
patches used in retrieval m from {1, 5, 10, 203. The cho-
sen values of each setting are presented in the Appendix B. referred the publicly available
For implementation, time-series repository (TSLib)². For all experiments, the average results from three runs are reported, with each exper- iment conducted on a single NVIDIA 1100 40GB GPU. For more details about the computational complexity analysis of RAFT, see Appendix D.
5.1. Better retrieval results lead to better performance.
Two criteria are important for our retrieval method to enhance the forecasting performance. First, the value patches V iddetified through the similarity between the input query x and key patches K. should closely match the actual future value yo which sequentially follows the input query. Second, the model should efficiently leverage the informa- tion in the value patches for forecasting. From these, we can draw the insight that higher similarity between input query and key patches (i.e., key similarity) will lead to the higher similarity between the actual value and value patches (i.e.. value similarity), eventually resulting in better performance.
Evaluation. We consider two metrics for evaluation: MSE and MAE. We varied the forecasting horizon length to mea- sure performance (i.e., F = 96, 192, 336, 720), and each ex- periment setting was run with three different random seeds to compute the average results. For the Illness dataset, fore- casting horizons of 24, 36, 48, and 60 are used, following the prior work (Nie al., 2023; Wang et al., 2024). The evaluation was conducted in multivariate settings, where both the input and forecasting target have multiple channels.
Figure 4 presents the correlation analysis conducted on the ETTh I dataset. Figure 4a shows that retrieving key patches with higher similarity leads to value patches that are more closely aligned with the actual future value. Figure lustrates that the value patches with greater similarity to the actual future values tend improve RAFT's perfor- mance more significantly. This trend is also consistent across datasets; datasets with higher key similarity show higher value similarity, resulting in larger performance gains. Spearman's 's correlation coefficient validate this trend, show- ing a correlation of 0.60 between key similarity and value similarity, and correlation of -0.54 between value simi- larity and performance gain across datasets. The negative correlation with performance is due to the of MSE as the metric (lower the better). These results demonstrate that better retrieval results from the retrieval module lead to improved performance of RAFT.
4.2. Experimental results on forecasting benchmarks
Table presents comparisons between the performance of time series forecasting methods and RAFT. The results represent the average MSE performance evaluated across different forecasting horizon lengths. We observe that our model consistently outperforms other contemporary basclines on average, supporting the effectiveness of retrieval in time series forecasting. Full results and comparisons using different evaluation metric (i.e., MAE) provided in Appendix G.
5. Discussions
In this section, we explore scenarios where retrieval shows substantial advantage by empirically analyzing its effect,
5.2. Retrieval is helpful when rare patterns repeat.
'ttpps: / http///ttttt../t////////t/ wwt.ttt.......ttmmeeeete com/thumm/
RAFT can complement scenarios where particular pattern does not frequently appear in the training dataset, making it
Time- Series- Libbarybbryy
6

---

Retrieval Augmented Time Series Forecasting
1.0
50
Value Similarity
MSE Change (%)
0.5
0.0
-0.51
-50
0.5
1.0
-0.5
0.0
0.5
1.0
Key Similarity
Value Similarity
(a) Scatter plot of key and value similarity
(h) Scatter plot value similarity and MSE change (%)
Figure 4. Analysis of the correlation between (a) the key similarity and value similarity, and (b) the value similarity and model performance changes measured hy MSE (%). Each dot represents each input patch from ETTh test dataset. Key similarity refers to the average similarity between input query (x) and all retrieved key patches (CC). Value similarity refers to the average similarity between actual future value (yb) and all retrieved value patches (V).
difficult for the model to memorize. By utilizing retrieved information, the model can overcome this challenge. To an- alyze this effect, we conducted experiments using synthetic time series datasets.
Table 2. Analysis between forecasting accuracy and the rarity of the pattern over the synthetic time series with an autoregressive model. Forecasting accuracy was evaluated using MSE, averaged across 120 different time series short-term patterns. The num- bers in the last row indicate the ratio by which the MSE decreases when retrieval is appended.
Synthetic data generation with autoregressive model. The synthetic time series was constructed by combining three components: trend, seasonality, and event-based short- term patterns. Trend and seasonality were generated using sinusoidal functions with varying periods, amplitudes, and offsets, representing long-term consistent patterns. Short- term patterns, modeled as event- went-based dynamics, were cre- ated using autoregressive model:
Pattern occurrences
2
4
TimeMixer
0.2360
0.2166
0.2276
TimesNet
0,2282
0.1970
0.1925
MICN
0.2285
0,2331
0.2033
DLinear
0.2640
0.2552
0.2502
RAFT without Retrieval
0.2590
0.2310
0.2344
RAFT with Retrieval
0.2209
0.2064
0.2128
20
Dt - + Et.
(10)
MSE decrease ratio
-14.7%
-10.7%
-9.2%
where yoi autoregressive parameters, and E, is noise pled from a uniform distribution. The short-term pattern length was fixed at 200. To test retrieval effectiveness for rare patterns, we generated three distinct short-term patterns and varied their frequency in the training dataset. Forecast- ing accuracy (MSE) was evaluated when each short-term pattern appeared in the test set, with input and forecasting horizon lengths fixed at 96. Additional dataset details and figures are available in Figure 5a and Appendix E.
reduction in MSE was more significant. Similar to RAFT without retrieval, the baseline models exhibited decrease in performance the pattern occurrences decreased. When we also visualize the predictions of models with and without retrieval modules over the rare pattern (see Figure 5b), the model utilizing retrieval aligns well with the pattern's 'ss periodicity and offset during forecasting, while the model relying solely on learning fails to capture these aspects. This suggests that the model struggles to learn rare patterns, and the retrieval module effectively complements this deficiency.
Results. Table presents the number of occurrences of the short-term patterns and the corresponding performance of RAFT with and without retrieval, as well as baseline did not consider
5.3. Retrieval is helpful when patterns are temporally less correlated.
models. Note that, this experiment, multiple periods in order to isolate the effect of retrieval, so RAFT without retrieval has identical structure to the NLinear (Zeng et al., 2023). The results show that our model, utilizing retrieval, consistently outperformed the model without retrieval on the synthetic dataset; 9.27~1.77% increase in performance depending on the pattern occur- rences. Notably, as the pattern occurrences decreased, the
If short-term patterns very similar across time, there's ' less unique information for the model to learn, making it easier to achieve accurate predictions. On the other hand, if the short-term patterns time series data are similar to a random walk without any specific temporal correlation, the model would need to memorize all changes within

---

Retrieval Augmented Time Series Forecasting
Ground Truth
w/o Retrieval
with Retrieval
100
-0.2
0.4
-100
-066
-.8
-200
110
0
500
1000
1500
20
40
60
80
(a) Example plot of the synthetic time series
(b) Predictions with and without retrieval module
Figure 5. Visualization of a synthetic time series with short-term patterns and the corresponding predictions over the rare short-term pattern from models with and without the retrieval module. MSE of predictions in this example without retrieval is 0.087, while with retrieval, it improves to 0.035.
short-term pattern for accurate forecasting. Based on this hypothesis, we expect the retrieval module to be especially helpful when patterns are temporally less correlated, as retrieval can easily detect similarities between patterns that temporal correlation alone cannot capture. We again use the synthetic dataset for validation.
series data. Again, the retrieval module improves perfor- mance across all cases, particularly for rare patterns. Fur- thermore, the performance improvement is more signifi- cant for temporally less correlated patterns (16.0~31.5%% decrease of MSE depending on pattern occurrences), com- pared to temporally more correlated ones shown in Table (9.2~144 14.7%). The baseline models exhibited a similar trend to that observed in Table 2, while the performance gap compared to RAFT with retrieval has become more signif- icant. This confirms that the proposed retrieval module is more beneficial when dealing with temporally less corre- lated or near-random patterns that are more challenging for the model to learn.
Synthetic data generation with random walk model. Instead of generating short-term patterns using the autore- gressive model as before, we utilize random walk-based change patterns, following the equation:
Et = It-1 + Et.
(11)
The step size for the walk ε, was sampled from a uniform distribution within the range of [-20, 20]. The generated short-term patterns were then inserted into the training data, as in the previous synthetic time-series approach.
5.4. Retrieval is also helpful for Transformer-variantt
We investigate the effectiveness of the retrieval module cunsformer-variints. using AutoFormer. Instead of modi- fying the internal Transformer architecture to integrate retrieval module, we directly added retrieval results to Aut- oFormer's s predictions at the final stage. Table 4 demon- strates that our retrieval module successfully enhances the forecasting performance of the Transformer- based model, highlighting its broader applicability to other architectures.
Table Forecasting accuracy the rarity of the pattern. Syn- thetic time series with random walk based patterns (temporally less correlated) used. Forecasting accuracy was evaluated using MSE, averaged across 120 different time series and short-term patterns. The numbers the last row indicate the ratio by which the MSE decreases when retrieval is appended.
Pattern occurrences
2
TimeMixer
0.2863
0.2305
0.2249
TimeNet
0.2448
0.1877
0.1938
Table 4, Performance comparison between AutoFormer and Aut- oFormer with our proposed retrieval module. The average MSE across different forecasting horizon lengths is reported.
MICN
0.2536
0.2445
0.2450
DLinear
0.3175
0.2059
0.2798
RAFT without retrieval
0.2694
0.2649
0.1894
ETTh1
ETTh2
ETTml
ETTm2
RAFT with retrieval
0.1845
0.1818
0.1592
Autoformer
0.496
0.450
0.588
0.327
MSE decrease ratio
-3155%
-31.4%
-16.0%
Retrieval
0.471
0.444
0.454
0.326
Results. Table 3 shows the results of applying the same experiment as in Table 2, but with different synthetic time-
8

---

Retrieval Augmented Time Series Forecasting
6. Conclusion
In Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement, pp. 49 49-62, 2009.
In this paper, we introduce RAFT, a time-series forecasting method that leverages retrieval from training data aug- ment the input. Our retrieval module lessens the model to absorb all unique patterns in its weights, particularly those that lack temporal correlation or do not share common char- acteristics with other patterns. This overall is demonstrated as an effective inductive bias for deep learning architectures for time-series. Our extensive evaluations on numerous real- world and synthetic datasets confirm that RAFT achieves performance improvements over contemporary baselines. As various retrieval-based models are being proposed, there remains room for improvement in retrieval techniques specifically tailored for semess data (beyond the simple approaches used), including determining when, where, and how to apply retrieval based on dataset characteristics and capture more complex similarity measures that depend on nonlinear and nonstationary characteristics. Our work is expected open new avenues in the time-series forecasting field through the use of eetrivvalaunmmenee approaches.
Borgeaud, S., Mensch, A., Hoffmann, J.. Cai, T., Rutherford, E., Millican, K., Van Den Driessche, G. B., Lespiau, J.-B., Damoc, B., Clark, A., et al. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pp. 2206-2240. PMLR, 2022.
Borovykh, A., Bohte, S., and Oosterlee, C. W. Condi- tional time series forecasting with convolutional neural networks. stat, 1050:16, 2017.
Chen, C., Petty, K. Skabardonis, A., Varaiya, P., and Jia, Z. Freeway performance measurement system: mining loop detector data. Transportation research record, 1748(1): 96-102, 2001.
Chen, S.-A., Li, C.-L.,, Arik, S. O., Yoder, N. C.,, and Pfister, T. Tsmixer: An all-mlp architecture for time series fore- casting. Trunsactions on Machine Learning Research, 2023.
Acknowledgement
Das, A., Kong, W., Sen, R., and Zhou, Y. A decoder- only foundation model for time-series forecasting. In Forty-first International Conference on Machine Learn- ing. 2024.
We would like to thank Tomas Pfister and Dhruv Madeka for their valuable feedback during the review of our paper. Han, Lee, and Cha were partially supported by the National Research Foundation of Korea grant RAS-2022-00165333 Han was partially supported by GIST (Al-based Research Scientist Project).
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., and Wang, H. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312. 1...1....999... 2023.
Impact Statement
Gorishniy, Y., Rubachev, I., Kartashev, N., Shlenskii, D., Kotelnikov, A., and Babenko, A. Tabr: Tabular deep learning meets nearest neighbors. In The Twelfth Interna- tional Conference on Learning Representations, 2024.
This paper advances time series forecasting by enhancing deep learning models with the retrieval of information from historical training data. By reducing the learning burden and mitigating memorization, this approach improves the forecasting performance in various real-world applications, such as weather prediction and financial analysis. We be- lieve this work offers a new perspective on integrating the concept of retrieval into the time-series domain.
Granger, C. W. J. and Newbold, P. Forecasting economic time series. Academic press, 2014.
Guu, K., Lee, K., Tung, Z., Pasupat. P., and Chang, M. Retrieval augmented language model pre-training. In International conference on machine learning, pp. 3929- 3938. PMLR, 2020.
References
Hewamalage, H., Bergmeir, C., and Bandara, K. Recur- rent neural networks for time series forecasting: Current status and future directions. International Journal of Forecasting, 37(1):388-42) 2021.
Arik, S. O. and Pfister, T. Protoattend: Attention-basee prototypical learning. Journal of Machine Learning Re- search, 21(210):1-35, 2020.
Bai, S., Kolter, J. Z., and Koltun, V. An empirical evalua- tion of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271, 2018.
Iwata, T. and Kumagai, A. Few-shot learning for time-series forecasting. arXiv preprint arXiv:2009. 14379, 2020.
Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.-H., and Choo, J. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2021.
Benevenuto, F., Rodrigues, T., Cha, M., and Almeida, V. Characterizing user behavior in online social networks.

---

Retricval Augmented Time Series Forecasting
Kossen, J., Band, N., Lyle, C., Gomez, A. N., Rainforth, T., and Gal, Y. Self-attention between datapoints: Going beyond individual input-output pairs in deep learning. Advances in Neural Information Processing Systems, 34: 28742-28756, 2021.
Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations, 2023.
Shuster, K., Poff, S., Chen, M., Kiela, D., and Weston, J. Retrieval augmentation reduces hallucination in conver- sation. Findings of the Association for Computational Linguisties: EMNLP, 2021.
Lai, G., Chang, W.-C.,, Yang, Y., and Liu, H. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR confer- ence on research & development in information retrieval, pp. 95-104, 2018.
Trindade, A. UCI Machine Learning Repository, 2015. DOI: https///bi.org/11.2444325288
LoecticiyyLodDDiggaam20011 12014.
Laptev, N., Yosinski, J., Li, L. E., and Smyl, S. Time- series extreme event forecasting with neural networks at uber. In International conference on machine learning, volume 34. pp. 1-5. sn, 2017.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L.., Gomez, A. N., Kaiser, t.., and Polosukhin, Atten- tion is all you need. In Advances in Neural Information Processing Systems, pp. 5998-6008, 2017.
Lewis, P., Perez, E., Piktus, A., Petroni, F,, Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., et al. eetrieval-augmentee generation for knowledge- intensive nlp tasks. Advances in Neural Information Pro- cessing Systems, 33:9459-9474 2020.
Wang. H., Peng, J.. Huang, F., Wang. J., Chen, J., and Xiao, Y. Mien: Multi-scale local and global context modeling for long-term series forecasting. In The eleventh interna- tional conference on learning representations, 2023.
Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang, Y.- X., and Yan, X. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecast- ing. Advances in neural information processing systems, 32, 2019.
Wang, S., Wu, H., Shi, X., Hu, T., Luo, H., Ma, L.., Zhang, J. Y., and ZHOU, J. Timemixer: Decomposable multi- scale mixing for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024.
Lin, S., Lin, W., Wu, W., Chen, H., and Yang, Sparsetsf: Modeling long-tem term time series forecasting with 1k* parameters. In Forty-first International Conference Machine Learning, 2024.
Weigend, A. S., Mangeas, M., and Srivastava, A. N. Non- linear gated experts for time series: Discovering regimes and avoiding overfitting. International journal of neural systems, 6(04):373-39) 1995.
Liu, M., Zeng, A., Chen, M., Xu, Z., Lai, Q., Ma, L., and Xu, Q. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 55:5816-5828 2022a.
Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., and Sahoo, D. Unified training of universal time series fore- casting transformers. In Forty-first International Confer- ence Machine Learning, 2024.
Liu, Y., Wu, H., Wang, J., and Long, M. Non-stationary transformers: Exploring the stationarity in time series forecasting, Advances in Neural Information Processing Systems, 35:9881 1 19993, 2022b.
Wu, H., Xu, J., Wang. J., and Long, M. Autoformer: Decom- position transformers with core-correlttion for long-term series forecasting. Advances in neural information pro- cessing systems, 222240,,,,,, 2021.
Martín, L., Zarzalejo, L. F. Polo, J., Navarro, A., Marchante, R., and Cony, M. Prediction of global solar irradiance based on time series analysis: Application to solar ther- mal power plants energy production planning. Solar Energy, 84(10): 2010.
Wu, H., Hu, T., Liu, Y.. Zhou, H., Wang. J., and Long, M. Timesnet: Temporal 2d variation modeling for gen- eral time series analysis. In The Eleventh International Conference on Learning Representations, 2023.
Miller, J. A., Aldosari, M., Saeed, F., Barna, N. H., Rana, S., Arpinar, I. B., and Liu, N. A survey of deep learning and foundation models for time series forecasting, arXiv preprint ar XrXiv:2401.11 13912, 2024.
Yang, S., Eisenach, C., and Madeka, D. Mq-retenn: Multi-horizon time series forecasting with retrieval- augmentation. In 8th SIGKDD International Workshop on Mining and Learning fromm Time Series Deep Fore- casting: Models, Interpretability, and Applications, 2022.
Nader, Y., Sixt, L., and Landgraf, T. Dnnr: Differential nearest neighbors regression. In International Conference on Machine Learning, pp. 16296-16317. PMIR, 2022.
Zeng, A., Chen, M., Zhang, L., and Xu, Q. Are transformers effective for time series forecasting? In Proceedings of
10

---

Retrieval Augmented Time Series Forecasting
the AAAI conference on artificial intelligence, volume 37, pp. 11121-11128, 2023.
Zhang, T., Zhang, Y., Cao, W., Bian, J., Yi, X., Zheng, S., and Li, Less is more: Fast multivariate time series forecasting with light sampling-oriented mip structures. arXiv preprini 2rXiv:2207.01 186, 2022.
Zhang, Z. Introduction to machine learning: k-nearest neighbors. Annals of translational medicine, 4(11), 2016.
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11106-11115, 2021.
Zhou, T., Ma, Z., Wen, Q. Wang. X., Sun, L., and Jin, R. Fedformer: Frequency enhanced decomposed trans- former for long-term series forecasting. In Interna- tional conference on machine learning. pp. 27268-27286. PMLR, 2022.
Zhu, Y. and Shasha, D. Statstream: Statistical monitoring of thousands of data streams in real time. VLDB 02: Proceedings of the 28th International Conference on Very Large Databases, pp. 358-369, Elsevier, 2002.

---

Retrieval Augmented Time Series Forecasting
A. Dataset Details
this work, we use widely-used time series datasets. The detailed information each dataset are shown Table 5. The dataset size is presented in (Train, Validation, Test). The targets used the univariate setting are as follows: oil temperature for the ETTh1, ETTh2, ETTml, ETTm2 datasets; the consumption of client for the Electricity dataset; the exchange rate of Singapore for the Exchange Rate dataset; the weekly ratio of patients for Illness dataset; 10-minute solar power forecasts collected from power plants for the Solar dataset; the road occupancy rates measured by sensor for the Traffic dataset; and CO2 (ppm) for the Weather dataset.
Table 5. Basic information of datasets used for evaluation.
Dataset
# of variates
Dataset Size
Frequency
ETTh1
7
(8449, 2785, 2785)
Hourly
ETTh2
7
(8449, 2785, 2785)
Hourly
ETTml
7
(34369, 11425, 11425)
15 min
ETTm2
7
(34369, 11425, 11425)
15 min
Electricity
321
(18221, 2537, 5165)
Hourly
Exchange Rate
8
(5120, 665, 1422)
Daily
Illness
7
(485, 2, 98) (36601, 5161, 10417)
Weekly
Solar
137
10 min
Traffic
862
(12089, 1661, 3413)
Hourly
Weather
21
(36696, 5175, 10444)
10min
12

---

Retrieval Augmented Time Series Forecasting
B. Implementation Details
RAFT employs retrieval module with the following detailed settings. The periods are set to 1, 4 (n = 3), following existing literature (Wang et al., 2024). The temperature is set to 0.1. The remaining settings, including the look back window size the learning rate, and the number of patches used in retrieval m are determined through grid search based on validation set performance, consistent with prior work (Wang al., 2024). The effect of haper-parametert (L, m, 7) on the performance are analyzed in the Section C.3-C.4.
Table provides the parameter settings of our model for each dataset. We observed that some parameters vary across different datasets.
Table 6. The chosen parameter values of cach setting via grid search over the validation set.
Forecasting horizon size
Look back window size
Learning rate
Number of retrievals
ETThl
96
720
1.00E-03
20
192
720
1.00E-02
20
336
720
1.00E-02
20
720
720
1.00E-04
20
ETTh2
96
720
1.00E-02
10
192
720
1. 1.00E-03
10
336
720
1. 1.00E-03
20
720
720
1.00-004
20
ETTml
96
720
1.00E-02
1
192
720
1. 1.00E-03
20
336
720
1.00E-03
20
720
720
1. L00E--2
20
ETTm2
96
720
1. .00E-03
5
192
720
1 1.00E-03
20
336
720
10000004
20
720
720
1. .00F-04
20
Electricity
96
720
L00E-02
192
720
.00E-03
336
720
1. 1000-03
1
720
720
1000-03333
Exchange
720
.00E-04
192
720
1. .00E-03
336
720
1 10000003
10
720
720
1000000
20
Mlnness
96
96
.00F-02
192
96
1 (00F-02
336
96
1000-002
20
720
96
1000-02
20
Solar
96
720
1. .00E-03
192
720
.00E-02
336
720
1.00E-03
1
720
720
1. 0000-03
1
Traffic
96
720
1. 1000-022
192
720
1000-003
336
720
1. 1.00E-03
720
720
1. 1.00E-03
Weather
96
720
1.00E-02
192
720
1.00E-03
336
720
1.00E-03
720
720
1.00E-03
13

---

Retrieval Augmented Time Series Forecasting
C. Component Analysis
In this section, we analyze the impact of each component of RAFT on performance.
C.1. Different Similarity Metrics for Retrieval
We compared RAFT using various similarity metrics, including Pearson's correlation, cosine similarity, cosine similarity with projection, and negative L2 distance. Cosine similarity with projection employs a trainable linear projection head for the input query and key vectors, respectively, and measures cosine similarity between the embeddings after projection rather than between the raw query and key. Table presents the comparison results across datasets, where Pearson's 's correlation shows the best performance among the various similarity metrics. We also observe that the linear projection provide comparable performance measuring similarity with the raw query and key.
Beyond performance, this projection-basee based approach offers computational advantages. By mapping sparse, high- -igh-dimensional inputs into dense, lower-dimensional embeddings, it reduces the time complexity of retrieval especially when integrated with preprocessing, parallelization, and efficient vector search techniques such as approximate nearest neighbor (ANN) methods. Additionally, the projection mechanism enables incorporation of external features or correlated time series. In such scenarios, separate encoders can project external segments and input features into shared embedding space. Once aligned, cosine similarity can be used retrieve relevant external segments for each input query. These retrieved segments can then be leveraged alongside the original input to enhance predictive performance, with both encoders optimized jointly during training.
Table 7. Comparison of various similarity metrics with RAFT the univariate setting.
Pearson's Correlation
Cosine Similarity Cosine Sim with Projection
Negative Distance
ETThl
0.0559
0.0561
0.0562
0.0562
ETTh2
0.1231
0.1235
0.1298
0.1271
ETTml
0,0299
0.0298
0.0294
0.0296
ETTm2
0.0647
0.0649
0.0699
0.0666
Electricity
0.3307
0.3343
0.3981
0.3388
Exchange Rate
0.0915
0.0917
0.0933
0.0922
Traffic
0.2737
0.2773
0.2943
0.2925
Weather
0.0118
0.0129
0.0026
0.0278
C.2. Ablation Study on Retrieval Module
To thoroughly analyze the impact of the proposed retrieval design on performance, we conducted an ablation study on the retrieval module. The ablations were as follows: (1) Random Retrieval Key patches retrieved randomly, without considering similarity the query; (2) Without Attention When aggregating value patches, use equal weights instead of similarity-basee weights (Eq. 5); (3) With One Period - Only period is used for retrieval (P {1)); (4) Without Retrieval Retrieval is entirely removed, leaving only the linear predictor. The experiments were conducted under identical hyper-paramett and learning settings and evaluated on multivariate forecasting tasks. Table 8 presents the MSE results for each dataset across the ablations. As shown the results, our model with all components included consistently achieved the best performance compared to the baselines across all datasets. Notably, we observed that when retrieval was conducted randomly, without attention with one period, performance was sometimes even worse than without retrieval, which demonstrates that retrieving relevant data is crucial for achieving high performance.
14

---

Retricval Augmented Time Series Forecasting
Table
Ahlation study retrieval module in the multivariate setting.
ETTh1
ETTh2
ETTml
ETTm2
Electricity
Exchange Rate
Traffic
Weather
RAFT
0.367
0.276
0.302
0.164
0.133
0.091
0.378
0.165
Random Retrieval
0.382
0.282
0.305
0.171
0.150
0.092
0.413
0.188
Without Attention
0.379
0.281
0.300
0.165
0.148
0.090
0.409
0.172
With One Period
0.369
0.276
0.303
0.164
0.133
0.088
0.379
0.168
Without Retrieval
0.379
0.282
0.306
0.167
0.143
0.089
0.410
0.182
C.3. Effect of Look Back Window Size (L)
We analyze the effect of look back window size (I.) on forecasting performance. Keeping all other experimental settings fixed, we varied the look back window size between 96, 192, 336, and 720 to observe performance changes. The experiments were conducted in a multivariate setting across four datasets, with the prediction length set 96. Table 9 compares the MSE results for different look back window sizes. Consistent with prior works (Wang al., 2024; Zeng al., 2023), observed that RAFT, based on a linear model, also achieves better forecasting performance the look back window size increases.
Table 9. Comparison results over different look back window size.
Look back window size (T.)
96
192
336
720
ETTh1
0.387
0.390
0.386
0.367
ETTh2
0.296
0.292
0.281
0.276
ETTml
0.348
0.310
0.306
0.302
ETTm2
0.179
0.171
0.166
0.164
C.4. Hyper-Parameter Analysis
RAFT has two key internal model parameters. The first is the number of patches retrieved by the retrieval module, and the second is the temperature used in the softmax function to calculate weights. Each hyper-parameter is optimally tuned for each dataset based on the validation set. Table 10-11 below illustrates examples of performance variations (MSE) across four datasets with different hyper-parameter values. As shown, the optimal values of the prpeemparaeetes vary depending on the dataset.
Table 10. Effect of the number of retrievals (m) on performance.
The number retrievals (m)
10
20
ETTh1
0.370
0.368
0.367
0.367
BTTh2
0.280
0.278
0.276
0.275
ETTml
0.302
0.300
0.298
0.297
ETTm2
0.164
0.164
0.164
0.164
Table 11. Effect the temperature (r) on performance.
Temperature (r)
0.01
0.1
10
ETTh1 0.383
0.367
0.378
0.381
ETTh2
0.285
0.276
0.280
0.281
ETTml
0.303
0.302
0.300
0.304
ETTm2
0.165
0.164
0.165
0.167
15

---

Retrieval Augmented Time Series Forecasting
C.5. Effect of Training Time Series Length
When the available time series is short, the number of historical patches for retrieval naturally decreases, potentially limiting performance gains. To investigate this phenomenon, without retrieval, varying the length of the training data from our benchmark datasets and evaluated performance test set while adjusting the training data proportion to 20%, 60%, and 100%. The results presented in Table 12 indicate that the impact of the training dataset's length on performance gains varies significantly across different datasets. However, our approach consistently outperformed the baseline model, even with limited historical data.
conducted additional experiments comparing model with and
fixed
Table 12. Effect training time series length on RAFT performance.
Dataset
Training data proportion
20%
60%
100%
ETThl
RAFT without retrieval
0.590
0.444
0.379
RAFT with retrieval
0.562
0.428
0.367
ETTh2
RAFT without retrieval
0.251
0.255
0.282
RAFT with retrieval
0.260
0.255
0.276
ETTml
RAFT without retrieval
1.492
0.692
0.306
RAFT with retrieval
0.975
0.684
0.302
ETTm2
RAFT without retrieval
1.364
0.608
0.167
RAFT with retrieval
1.312
0.603
0.164
16

---

Retrieval Augmented Time Series Forecasting
D. Computational Complexity for Retrieval
Our model incorporates retrieval process to find similar patches the given data. For efficient training, the retrieval process is pre-computed for the training and validation data, requiring computation only once during training. We analyzed the wall time (in seconds) for retrieval pre-computation, training, and inference on the ETTml dataset (see Table 13). The lookback window size set 720, and the forecasting horizon length was set to 96.
Table 13. Wall time for each process of RAFT over ETTml.
Pre-computation
Training time per epoch Total Inference time
Wall time (sec)
42.2
7.3
1.9
The pre-computation speed for retrieval of our model O(N3), where N denotes the size of the time-series in the training data. To reduce this time, approach is increase the stride of the sliding window beyond speeding up the search process. Table 14 records the changes in wall time as the stride of the sliding window increases. As the stride increases, the time required for the search process decreases significantly.
Table 14. Wall time across different number of strides over ETTml.
Stride
1
2
4
8
Wall time for pre-computation (sec)
42.2
19.8
9.3
4.7
Lastly, we examined the impact of increasing the stride on forecasting performance. Table 15 presents the changes in MSE across four datasets (ETTh1, ETTh2, ETTm1, ETTm2) the stride increases. While increasing the stride introduced performance trade-off, we observed that the decrease in performance not significant.
Table 15. MSE changes of RAFT over four datasets across the different number of strides.
Stride
2
4
8
ETThl
0.367
0.379
0.381
0.383
BTTh2
0.276
0.279
0.279
0.280
ETTm1
0.302
0.298
0.299
0.300
6TTm2
0.164
0.164
0.165
0.165
17

---

Retrieval Augmented Time Series Forecasting
E. Synthetic Dataset Generation Details
The synthetic time series was created by combining three different components. Two of these components represent trend and seasonality, which exhibit long-term consistent patterss throughout the entire time series. The third component represents event-based short-term patterns. The generation details for each component are follows:
Trend and seasonality components. To generate the trend and seasonality components, we synthesized sinusoidal functions with varying periods, amplitudes, and offsets. The total length of the time series was set to 18,000. The period of the sinusoidal function for the trend was sampled from a uniform distribution between [1000, 4000), while the period for seasonality was shorter, sampled from [500, 1000]. The amplitude of each component was randomly chosen from the ranges [200, 300] for the trend and [100, 200] for the seasonality. Offsets were sampled from the range [100, 200].
Short-ferm patterns from the autoregressive model. The length of each short-term pattern was 200. In the case of the autoregressive model, the value of the next time step was determined by the previous 20 time steps, following the equation below:
21 P + Et.
(12)
where yi represents the parameters in the autoregressive model, and 6, is the noise. The parameters were sampled from uniform distribution within [-5, 5], and the noise was sampled from a uniform distribution within [-10, 10]. The length of the short-term pattern was set to 200. To prevent the short-term patterns from producing extreme values compared to the trend and seasonal components, we clamped the values within the range [-100, 100].
Short-term patterns from the random-wald model. In the case of the random-walk model, the length of the short-term pattern was also fixed 200. Unlike the autoregressive model, in the random-wald walk model, the value of the next time step depends only on the previous time step, described by the equation:
-t-1 Eg.:
(13)
where the step size for the walk sampled from uniform distribution within the range of [0, 20). Again, to prevent the short-term patterns from producing extreme values compared to the trend and seasonal components, clamped the values within the range [-100, 100).
Finally, the trend, seasonality, and short-term patterns were combined to create the synthetic time series. Example visualizations of the autoregressive short-term pattern, the fandom-walle pattern, and the resulting synthetic time series can be seen in Figure
18

---

Retricval Augmented Time Series Forecasting
100
100
50
0
0
AM
-100
-50
-20
-100
0
50
100
150
200
0
500
1000
1500
(a) Autoregressive short-term pattern
(b) Synthetic data with autoregressive patterns
50
25
100
wm
0
M
-25
-100
-50
-20
50
100
150
200
0
500
1000
1500
(c) handom-walk short-term pattern
(d) Synthetic data with random- walk patterns
Figure 6. Visualization of an example synthetic time series with short-term patterns.
19

---

Retrieval Augmented Time Series Forecasting
F. Qualitative Analysis on Retrieval
this section, we provide examples of our retrieval results. Figure 7-9 illustrate a comparison between the input query and the retrieved key patch, as well as comparison between the ground truth and the retrieved value patch, with retrievals by and 4 periods. Note that we retrieve the key patch with the top-1 similarity and its following value patch. The results demonstrate that retrieval module effectively delivers useful information for forecasting future predictions.
Query
Retrieved Key
Ground Truth
Retrieved Value
1.0
1.0
0.5
0.5
0.0
0.0
-0.5
-0.5
0
20
40
60
80
0
20
40
60
80
(a) Input query and retrieved key patch (period
(b) Ground and retrieved value patch (period 1)
1.0
1.0
0.5
0.5
0.0
0.0
-0.5
-0.5
0
20
40
60
80
0
20
40
60
80
(c) Input query and retrieved key patch (period
(d) Ground truth and retrieved value patch (period 2)
1.0
1.0
0.5
0.5
0.0
0.0
-0.5
0.5
0
20
40
60
80
0
20
40
60
80
(c) Input query and retrieved key patch (period
(f) Ground truth and retrieved value patch (period 4)
Figure 7. The example of retrieval results ETThl dataset. The key patches retrieved by period 1, and 4 compared with input query in (a), (c), and (e), respectively. The value patches retrieved by period 1, 2, and 4 are compared with ground truth in (b), (d), and (f), respectively. Note that the figures in the right side sequentially follows after the figures in the left side within the time series.
20

---

Retrieval Augmented Time Series Forecasting
Query
Retrieved Key
Ground Truth
Retrieved Value
1.5
1.5
1.0
1.0
0.5
0.5
0.0
0.0
-0.5
-0.5
20
40
60
80
0
20
40
60
80
(a) Input query and retrieved key patch (period 1)
(b) Ground truth and retrieved value patch (periad 1)
1.5
1.5
1.0
1.0
0.5
0.5
0.0
0.0
-0.5
005
20
40
60
80
0
20
40
60
80
(c) Input query and retrieved key patch (period 2)
(d) Ground truth and retrieved value patch (period 2)
1.5
1.5
1.0
1.0
0.5
0.5
0.0
0.0
-0.5
-05
20
40
60
80
0
20
40
60
80
(c) Input query and retrieved key patch (period 4)
(f) Ground truth and retrieved value patch (periad 4)
Figure The example our retrieval results on Exchange Rate dataset. The key patches retrieved by period with input query in (a), (c), and (c), respectively. The value patches retrieved period 1, 2, and 4 are compared with ground truth in (b), (d), and (f), respectively. Note that figures the right side sequentially follows after figures the left side within the time series.
are compared
21

---

Retrieval Augmented Time Series Forecasting
Query
Retrieved Key
Ground Truth
Retrieved Value
4
4
2
0
0
-2
-2
0
20
40
60
80
0
20
40
60
80
(a) Input query and retrieved key patch (period
(b) Ground truth and retrieved value patch (period 1)
4
2
0
0
-2
-2
0
20
40
60
80
0
20
40
60
80
(c) Input query and retrieved key patch (period 2)
(d) Ground truth and retrieved value patch (period 2)
4
2
2
0
-2
-2
20
40
60
80
0
20
40
60
80
(e) Input query and retrieved key patch (period
(f) Ground truth and retrieved value patch (period 4)
Figure 9. The example of our retrieval results on Traffic datasct. The key patches retrieved by period 1, 2, and 4 are compared with input query in (a), (c), and (e), respectively. value patches retrieved by period 4 are compared with ground truth in (b), (d), (f), respectively. Note that the figures in the right side sequentially follows after the figures in the left side within the time series.
22

---

Retrieval Augmented Time Series Forecasting
G. Full Results
G.I. Evaluation Results with MSE
Table 16, Full evaluation results with MSE are provided, with some baseline results excerpted from prior works (Wang et al., 2024; Nie
al., 2023).
Methods
TimeMixer
PatchTST
TimesNet
MICN DLineur
FEDfomer Stationary
Autoformer
Informer
ETTh1
96
0.367
0.375
0.460
0.384
0.426
0.397
0.395
0.513
0.449
0.865
192
0.411
0.429
0,512
0.436
0.454
0.446
0.4699
0.534
0,500
1.008
336
0.436
0.484
0,546
0.638
0.493
0.489
0.530
0.588
0.521
1,107
720
0.467
0.498
0,544
0,521
0.526
0,513
0.598
0.643
0.514
1,181
Avg 0.420
0.447
0.516
0.495
0.475
0.461
0.498
0.570
0.496
1.040
ETTh2
0..766
0.289
0.308
0.340
0.372
0.340
0.358
0.346
0.347
0.372
0,393
0.492
0.482
0,429
0.456
5.602
336 0.376
0.386
0.427
0.452
0.607
0.591
0.496
0.552
0.482
4.721
720 0.436
0.412
0.436
0.462
0.824
0.839
0.463
0.562
0,515
3.647
Avg 0.359
0.365
0.391
0.414
0.574
0.563
0.437
0.526
0.450
4,431
ETTml
96 0..02
0.320
0.352
0.338
0.365
0.346
0.379
0.386
0.505
0.672
192 0.329
0.361
0.390
0.374
0.403
0.382
0.426
0.459
0.553
0.795
336 0.355
0.390
0.421
0.410
0.436
0.415
0.445
0.495
0.621
1,212
720 0.406
0.454
0.462
0.478
0.489
0.473
0.543
0.585
0.671
1.166
Avg 0.348
0.381
0.406
0.423
0.404
0.448
0.481
0,588
ETTm2
96 0.166
0.175
0.183
0.187
0.197
0.193
0,203
0.192
0.255
0,365
192 0.219
0.237
0.255
0.249
0.284
0.284
0.269
0.280
0.281
0,533
336
0.275
0.298
0.309
0.321
0,381
0.382
0.325
0.334
0.339
1.363
720
0.359
0.391
0.412
0.408
0.549
0.558
0.421
0.417
0.433
3.379
Avg 0..254
0.275
0.290
0.291
0.353
0.354
0.305
0.306
0.327
1.410
Electricity
0.133
0.153
0.190
0.168
0.180
0,210
0.193
0,201
0.274
192 0.149
0.166
0.199
0.184
0.189
0,210
0.201
0,222
0.296
336 0.161
0.185
0.217
0.198
0.198
0.223
0.214
0.200
0,231
0,300
720 0.197
0.225
0.258
0.220
0.217
0,258
0.246
0.222
0,254
0.373
Avg 0.
0.182
0.216
0.193
0.196
0.225
0.214
0.193
0.227
0.311
Exchange
96 00.011
0.095
0.084
0.107
0.102
0.081
0.148
0.111
0.197
0.847
192 0.205
0.201
0.180
0.226
0.172
0.157
0.271
0.219
0.300
1,204
336 0.353
0.350
0.510
0.367
0.272
0.305
0.460
0.421
0.509
1.672
720 1.115
0.898
1,480
0.964
0.714
0.643
1.195
1.092
1.447
2.478
Avg
0.441
0.386
0.564
0.416
0.315
0,297
0.519
0.461
0.613
1.550
Illness
24
2.076
1.896
1,319
2.317
2.684
2.215
3.228
2.294
3.483
5,764
36
2.183
1.928
1.579
1.972
2.667
1,963
2.679
1.825
3.103
4,755
48
2.073
2.132
1,553
2.238
2.558
2.130
2.622
2.010
2.669
4,763
60
2.058
2.141
1,470
2.027
2.747
2,368
2,857
2.178
2,770
5,264
Avg 2.097
2.024
1,480
2.139
2.664
2.169
2.847
2.077
3.006
5.137
Solar
0.192
0.189
0.265
0.373
0.257
0.290
0.286
0.321
0.456
0.287
192 0.247
0.222
0.288
0.397
0.278
0.320
0.291
0.346
0,588
0.297
336 0.240
0.231
0.301
0.420
0.298
0.353
0.354
0.357
0,595
0.367
720 0.246
0.223
0.295
0.420
0.299
0.357
0.380
0.375
0.733
0.374
Avg (0.231
0.216
0.287
0.403
0.283
0.330
0.328
0.350
0.593
0.331
Traffic
96 00.378
0.462
0.526
0.593
0.577
0.650
0.587
0.612
0.613
0.719
192 0.391
0.473
0.522
0.617
0.589
0.598
0.604
0.613
0.616
0.696
336 0.402
0.498
0,517
0.629
0.594
0.605
0,621
0.618
0.622
0.777
720 0.434
0.506
0.552
0.640
0.613
0.645
0.626
0.653
0.660
0.864
Avg 0.402
0.485
0.529
0.620
0.593
0.625
0.610
0.624
0.628
0.764
Weather
96 00.655
0.163
0.186
0.172
0.198
0.195
0.217
0.173
0.266
0,300
192 00.211
0.208
0.234
0.219
0.239
0.237
0.276
0.245
0.307
0.598
336
00.660
0.251
0.284
0.246
0.285
0.282
0.339
0.321
0.359
0,578
720
0..277
0.339
0.356
0.365
0.351
0.345
0.403
0.414
0.419
1.059
Avg 0.241
0.240
0.265
0.251
0.268
0.265
0.309
0.288
0.338
0.634
23

---

Retrieval Augmented Time Series Forecasting
G.2. Evaluation Results with MAE
Table 17. Full evaluation results with MAE are provided, with some baseline results excerpted from prior works (Wang et al., 2024; Nie
al., 2023).
Methods
Ours
TimeMixer
PatchTST
TimesNet MICN DLineur FEDformer Stationary
Autoformer
Informer
ETTh!
96
0.397
0,400
0.447
0.402
0.446
0.412
0.424
0.491
0.459
0.713
192 0.427
0.421
0.477
0.429
0.464
0.441
0.470
0.504
0.482
0.792
336 0.442
0.458
0.496
0.469
0.487
0.467
0.499
0.535
0.496
0.809
720 0.478
0.482
0.517
0.500
0.526
0,510
0.544
0.616
0.512
0.865
Avg 0.436
0.440
0.484
0.450
0.481
0.458
0.484
0.537
0.487
0.795
ETTh2
96 0.344
0.341
0.355
0.374
0.424
0.394
0.397
0.458
0.388
1.525
192
0.393
0.392
0.405
0.414
0.492
0.479
0.439
0,493
0.452
1,931
336
0.425
0.414
0.436
0.452
0.555
0.541
0.487
0.551
0.486
L835
720 0.473
0.434
0.450
0.468
0.655
0.661
0.474
0.560
0.511
1.625
Avg 0.409
0.395
0.412
0.427
0.532
0,519
0.449
0.516
0.459
1.729
ETTml
96 00.349
0.357
0.374
0.375
0.387
0.374
0.419
0.398
0.475
0,571
192 0.367
0.381
0.393
0,387
0.408
0.391
0.441
0.444
0,496
0.669
336
0.383
0.404
0.414
0,411
0.431
0,415
0.459
0.464
0,537
0.871
720 0.413
0.441
0.449
0.450
0.462
0.451
0.490
0,516
0,561
0.823
Avg 0.378
0.396
0.408
0.406
0,422
0,408
0.452
0.456
0.517
0.734
ETTm2
0..266
0.258
0,270
0.267
0.296
0.293
0.287
0.274
0.339
0.453
192 0.296
0.299
0.314
0.309
0,361
0,361
0.328
0.339
0,340
0.563
336 0.336
0.340
0,347
0.351
0.429
0.429
0.366
0.361
0.372
0.887
720 0.392
0.396
0.404
0.403
0,522
0.525
0.415
0.413
0,432
1.338
Avg 00.300
0.323
0.334
0.333
0,402
0.402
0.349
0.347
0.371
0.810
Electricity
96 0.222
0.247
0.296
0.272
0.293
0.302
0.308
0.273
0.317
0.368
192 0.247
0.256
0.304
0.322
0.302
0.305
0.315
0.286
0.334
0.386
336 0.259
0.277
0.319
0.300
0.312
0.319
0.329
0.304
0.443
0.394
720 0.297
0.310
0.352
0.320
0.330
0.350
0.355
0.321
0.361
0.439
Avg 0.259
0.273
0.318
0.304
0,309
0.319
0.327
0.296
0,364
0.397
Exchange
96 00.209
0.214
0.203
0.234
0.235
0.203
0.278
0.237
0.323
0.752
192 0.324
0.320
0.302
0.344
0.316
0.293
0.380
0.335
0.369
0.895
336 0..31
0.427
0.531
0.448
0.407
0.414
0,500
0.476
0.524
1.036
720
0.801
0.702
0.959
0.746
0.658
0.601
0.841
0.769
0,941
1,310
Avg 0.441
0.416
0.499
0.443
0.404
0.378
0.500
0.454
0.539
0.998
Illness
0.956
0.860
0.754
0.934
1.112
1,081
1.260
0.945
1.287
1.677
1.008
0.910
0.870
0.920
1.068
0.963
1.080
0.848
1.148
1.467
48
0.972
0.956
0.815
0.940
1.052
1.024
1.078
0.900
1,085
1.469
60
0..944
0.956
0.788
0.928
1.110
1.096
1.157
0.963
1.125
1,564
Avg (0.977
0.920
0.807
0.931
1. 1,066
1.041
1.144
0.914
1.161
1.544
Solar
96
00.251
0.259
0.323
0.358
0.325
0.378
0.341
0.380
0.446
0.323
192
0.323
0.283
0.332
0.376
0.354
0.398
0.337
0.369
0.561
0,341
336
0.300
0.292
0.339
0.380
0.375
0.415
0.416
0.387
0.588
0.429
720
0.311
0.285
0.336
0.381
0.379
0.413
0.437
0.424
0.633
0.431
Avg 0.296
0.280
0.333
0.374
0.358
0.401
0.383
0.390
0.557
0.381
Traffic
96 (0.273
0.285
0.347
0.321
0.350
0.396
0.366
0.338
0.388
0.391
192 00.277
0.296
0.332
0.336
0.356
0.370
0.373
0.340
0,382
0.379
336 0.282
0.296
0.334
0.336
0.358
0.373
0,383
0.328
0.337
0.420
720 0.297
0.313
0.352
0.350
0.361
0.394
0,382
0.355
0,408
0.472
Avg 0.282
0.298
0.341
0.336
0.356
0.383
0.376
0.340
0.379
0.416
Weather
00.222
0.209
0.227
0.220
0.261
0.252
0.296
0.336
0.384
192 0.264
0.250
0.265
0.261
0.299
0.295
0.336
0.285
0.367
0.544
336
0.302
0.287
0.301
0.337
0.336
0.331
0.380
0.338
0.395
0.523
720
0.355
0.341
0.349
0.359
0.388
0.382
0..288
0.410
0.428
0.741
Avg 00.266
0.272
0.286
0.294
0.321
0.315
0,360
0.314
0,382
0,548
24


Reviews:
[{"review_id": "W2j924hLkA", "reviewer": "Reviewer_AD5i", "summary": "This paper presents RAFT (Retrieval-Augmented Forecasting of Time Series), a method for enhancing time series forecasting models by retrieving relevant “patches” from the training dataset that match the current input pattern. These retrieved patches—subsequent future values corresponding to historically similar inputs—are then treated as an additional signal alongside the traditional model input. The authors demonstrate that RAFT boosts performance over numerous strong baselines (including Transformer-based approaches and simple linear or MLP models) across ten benchmark datasets. They also perform extensive analyses on synthetic data to highlight in which scenarios retrieval offers clear benefits (e.g., when patterns are rare or have low temporal correlation).\n\nMain contributions\n\n1. Proposing a retrieval-augmented architecture, specifically designed for single time-series forecasting, leveraging historical key-value “patches” in a memory-like module.\n2. Showing consistent performance gains on real-world benchmarks (ETT, Electricity, Exchange, etc.) and describing scenarios where retrieval is particularly beneficial (e.g., repeated rare patterns).", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful comments. Please see our point-by-point response. \n\n> W1, Q1. Scalability to extremely long series.\n\nThank you for highlighting this important point. We fully agree that searching patches in extremely long time-series can be computationally intensive. In light of this, we had introduced a stride into the sliding window approach used during retrieval to reduce the number of searched segments in the training data (see Appendix D). Our experiments demonstrated that increasing the stride substantially decreases computational costs with minimal impact on performance. For instance, using a stride of 8 reduces inference time by a factor of 8 (which could be even parallelized) with less than 5% degradation in performance.\n\nAnother strategy could be to train a simple encoder to project the time segments of training data into low-dimensional dense embeddings and perform the search using cosine similarity (refer to Appendix C.1 - Cosine Similarity with Projection). According to our results presented in the Appendix C.1, this method maintains comparable performance with Pearson’s Correlation (default RAFT setting). This approach can significantly reduce inference-time complexity through precomputing dense embeddings of the training data segments. The entire process can be even parallelized, enabling it to handle datasets of a larger-scale such as billions of data points at inference. We can further accelerate the search efficiency via vector search solution through the approximated nearest neighbor (ANN) technique. We will incorporate this discussion into the revised manuscript.\n\n> W2, Q3. Extension to handling confounding variables (e.g., external or exogenous features)\n\nThank you for this comment. Our model can be extended to incorporate external features or correlated time series by training encoders to project external information into low-dimensional dense embeddings within the same embedding space as the input features. One natural extension is to consider additional external time series. In this scenario, one encoder can project external segments into dense embeddings and another encoder can project input features into the same embedding space. When embeddings are aligned within a shared space, retrieval can use cosine similarity to identify relevant external segments for each input query patch. The retrieved external segments can then be used alongside the original input for enhanced prediction and encoders would be optimized jointly during training. This idea is consistent with our ablation study described in Appendix C.1 (\"Cosine Similarity with Projection\"). We will include this extension and its potential benefits in the revised manuscript.\n\n> Q2. Does RAFT degrade gracefully when the time series is short (and thus historical patches are limited)? Are there diminishing returns?\n\nIt is correct that when the available time series is short, the number of historical patches for retrieval naturally decreases, potentially limiting performance gains. To investigate this phenomenon, we conducted additional experiments comparing our model with and without retrieval, varying the length of the training data from our benchmark datasets and evaluated performance on a fixed test set while adjusting the training data proportion to 20%, 60%, and 100%. Our results indicate that the impact of the training dataset's length on performance gains varies significantly across different datasets. However, our approach consistently outperformed the baseline model, even with limited historical data. We will include these findings and further discussion in our revised manuscript. We will include these findings and further discussion in our revised manuscript.\n\n| Dataset | Training data proportion | 20% | 60% | 100% |\n|:-:|-|:-:|:-:|:-:|\n| ETTh1 | RAFT without retrieval | 0.590 | 0.444 | 0.379 |\n|  | RAFT with retrieval | 0.562 | 0.428 | 0.367 |\n| ETTh2 | RAFT without retrieval | 0.251 | 0.255 | 0.282 |\n|  | RAFT with retrieval | 0.260 | 0.255 | 0.276 |\n| ETTm1 | RAFT without retrieval | 1.492 | 0.692 | 0.306 |\n|  | RAFT with retrieval | 0.975 | 0.684 | 0.302 |\n| ETTm2 | RAFT without retrieval | 1.364 | 0.608 | 0.167 |\n|  | RAFT with retrieval | 1.312 | 0.603 | 0.164 |\n\n\n> Suggestion. It would be interesting to see more real-world interpretability examples instead of the standard datasets.\n\nThank you for the suggestion to showcase practical use cases with real-world data, which will enhance the interpretability of our model. We will include new interpretability examples using real-world scenarios, such as disease spread data from the COVID-19 pandemic (e.g., https://data.who.int/dashboards/covid19/) in the revised manuscript.", "claims_and_evidence": "* The authors claim that retrieval can help a model avoid “memorizing” rarely repeating patterns in its weights, instead leveraging a near-external memory approach to handle such patterns at inference time. Experimental results on synthetic data indeed show RAFT obtains substantial gains—particularly when short-term patterns reappear rarely and have random, less-correlated structures.\n* They claim RAFT outperforms state-of-the-art baselines like Autoformer, FedFormer, PatchTST, etc. The tables (Table 1, for example) show RAFT surpassing these baselines on average MSE by a clear margin.", "methods_and_evaluation_criteria": "Method Overview: RAFT uses a sliding window approach to generate “key-value” patches from the historical data, where keys are windows of size L and values are the subsequent F steps. Given a query window, RAFT calculates similarity (via Pearson correlation by default) to find top-m “key” matches, then aggregates the corresponding “value” patches. This aggregated retrieval is appended to the original input for final forecasting via a linear projection.\n\nEvaluation: They compare MSE and MAE across multiple standard forecasting horizons (F = 96, 192, 336, 720) on widely used popular datasets. They provide ablations, sensitivity analyses, and comparisons to existing SOTA.", "theoretical_claims": "This paper is applied.", "experimental_designs_or_analyses": "1. The authors systematically test RAFT on 10 benchmarks commonly used in time-series forecasting. They vary the forecasting horizon and measure MSE/MAE. They also provide comparative results for 9+ baselines.\n2. They do thorough ablations: e.g., removing the retrieval module, random retrieval, retrieval but no attention weighting, etc., to demonstrate each component’s impact.\n3. Hyperparameter search is described (e.g., number of retrieved patches m, temperature for weighting, stride for scanning the dataset). \n4. The authors present separate analyses on synthetic data to illustrate “rare patterns” and “less correlated random-walk patterns.” This is a strong demonstration of the retrieval approach’s value in difficult scenarios.", "relation_to_broader_scientific_literature": "1. Retrieval-based frameworks are used extensively in large language models to incorporate external knowledge. This work adapts that principle to time-series forecasting.\n2. It aligns with the wave of “memory” or “nonparametric” approaches to handle patterns not frequently observed, reminiscent of k-NN or kernel-based time-series methods, but integrated into a deep learning pipeline.", "essential_references_not_discussed": "Overall, the references are comprehensive enough.", "other_strengths_and_weaknesses": "Strengths:\n\n1. Strong empirical performance: RAFT  outperforms multiple strong baselines across standard real-world datasets.\n2. Solid ablation and sensitivity studies: They carefully show how retrieval stride, temperature, offset normalization, etc. affect results, enhancing transparency.\n\nWeakness:\n\n1. Scalability: For extremely long series (millions of points), storing and searching patches might be expensive. More discussion on large-scale feasibility would be beneficial.\n2. Confounding variables: Potential external or exogenous features (holidays, discrete events, etc.) might also be important to consider or retrieve.", "other_comments_or_suggestions": "It would be interesting to see more real-world interpretability examples instead of the standard datasets.", "questions_for_authors": "Q: As T grows large (say, 10 million+ data points), any efficient way of retrieval ?\n\nQ: Does RAFT degrade gracefully when the time series is short (and thus historical patches are limited)? Are there diminishing returns?\n\nQ: Could RAFT be extended to incorporate side-information retrieval from multiple correlated time series or external knowledge for each query patch?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CVl2KCjmBb", "reviewer": "Reviewer_kZHz", "summary": "This paper proposes the RAFT framework, which leverages retrieval-augmented generation (RAG) to retrieve similar time series patterns and integrate them to enhance future predictions. The effectiveness of the RAFT framework has been evaluated on well-adopted time series benchmarks, with comparisons against state-of-the-art models.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for acknowledging the novelty and wide applicability of our proposed retrieval method. We plan to further revise our manuscript by including additional experiments and clarifications in the revised manuscript.", "claims_and_evidence": "Yes, the claims are well supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "Yes, the evaluation criteria make sense for the problem.", "theoretical_claims": "There is no theoretical claim in the paper.", "experimental_designs_or_analyses": "The design and validity of the experiment are well-founded.", "relation_to_broader_scientific_literature": "This paper will have a broad impact on real-life applications of time series prediction models. It paves the way for applying retrieval-augmented generation (RAG) in the time series domain.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "The paper is well-written and clear, with a concise results comparison and convincing visualization of similar patterns. Well done!", "other_comments_or_suggestions": "No.", "questions_for_authors": "No.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "M7o6al2Tv8", "reviewer": "Reviewer_43iV", "summary": "The author introduces retrieval-augmented methods into the time series forecasting problem, proposing Retrieval-Augmented Forecasting of Time-series (RAFT). During the forecasting process, the method retrieves the most similar historical windows from the training set to predict future data. The RAFT method has achieved favorable results on multiple datasets.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful comments. Please see our point-by-point response.\n\n> Claims and Evidence & W1. Rationale and ablation study of multiple periods in retrieval\n\nThank you for raising this important point. First, we would like to clarify that our model performs retrieval across multiple periods (scales) and does not retrieve segments from identical positions across different periods. Instead, it retrieves segments at varying positions for each period based on their similarity to the input segment. Perhaps this misunderstanding arose because Figure 3 depicted keys and values from identical positions across multiple periods. In the revised manuscript, we will update Figure 3 to show different positions of retrieval in each period and revise the corresponding explanation on lines 193–197 of the right column.\n\nIn addition to our response above, we conducted new ablation tests to empirically validate the effectiveness of multi-period retrieval by comparing performance with and without multiple periods. This new result is shown in the table below, which shows multi-period retrieval shows positive impacts for most datasets. We hope this answers your question and we will be happy to add these findings in the  revised manuscript in Appendix C.2.\n\n||ETTh1|ETTh2|ETTm1|ETTm2|Electricity|Exchange Rate|Traffic|Weather|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|RAFT|0.367|0.276|0.302|0.164|0.133|0.091|0.378|0.165|\n|Without Retrieval|0.379|0.282|0.306|0.167|0.143|0.089|0.41|0.182|\n|Retrieval With 1 Period|0.369|0.276|0.303|0.164|0.133|0.088|0.379|0.168|\n\n\n> W2, W3. Baseline comparison in Section 5.2 and 5.3\n\nThank you for the suggestion. As recommended, we have conducted additional experiments by comparing RAFT against strong baselines (represented in Table 1) on carefully synthesized datasets, extending our analyses presented in Sections 5.2 and 5.3. The comparison with the four strongest baselines selected based on winning ratio is summarized in the table below. Our results demonstrate that, in the experiments from Section 5.2, the proposed method showed comparable overall performance to the baselines but outperformed them in capturing rare patterns (e.g., occurrences = 1). In the Section 5.3 experiments, our method demonstrated superior performance in modeling less correlated patterns compared to all other baselines. We will include these new experimental results in the revised manuscript.\n\n|Pattern Occurrences (Section 5.2)|1|2|4|\n|--|:-:|:-:|:-:|\n|TimeMixer|0.236|0.217|0.228|\n|TimesNet|0.228|0.197|0.192|\n|MICN|0.228|0.233|0.203|\n|DLinear|0.264|0.255|0.250|\n|RAFT without Retrieval|0.259|0.231|0.234|\n|RAFT with Retrieval|0.221|0.206|0.213|\n\n| Pattern Occurrences (Section 5.3)|1|2|4|\n|-|:-:|:-:|:-:|\n|TimeMixer|0.286|0.230|0.225|\n|TimeNet|0.245|0.188|0.194|\n|MICN|0.254|0.245|0.245|\n|DLinear|0.318|0.206|0.280|\n|RAFT without Retrieval|0.269|0.265|0.189|\n|RAFT with Retrieval|0.185|0.182|0.159|\n\n\n> W4. Details of implementing RAFT in AutoFormer\n\nRather than modifying the internal Transformer architecture to integrate our retrieval module, we directly appended retrieval results to AutoFormer’s predictions at the final stage. Here are some details: AutoFormer divides the input patches into trend and seasonal components, processes them separately, and combines them by addition at the final stage. Our retrieval results, $\\sum_{p \\in \\mathcal{P}}{g^{(p)}(\\tilde{\\mathbf{v}}^{(p)})}$, are added at the final stage of AutoFormer. We will clarify these details in our revised manuscript.\n\n> Suggestion 1. Impact of stride on computational complexity and performance\n\nWe agree that searching patches in extremely long time-series can be computationally intensive. To address this, we had introduced increasing a stride into the sliding window approach used during retrieval in Appendix D. Our experiments demonstrated that increasing the stride substantially decreases computational costs with minimal impact on performance. Results show that until stride 8, the performance loss is less than 5% while we could decrease the computational cost by 89%. As mentioned in our response to Reviewer 1 (W1), our method can be easily scalable to extremely long time series.\n\n> Suggestion 2. During training, if the input sequence x is located in the middle of the training dataset S, does the retrieval module search for sequences both before and after x?\n\nTo prevent information leakage during training, any patches overlapping with the input sequence x were excluded from the retrieval candidates. After removing these overlapping patches, the retrieval module utilized patches from both before and after the sequence x. At inference time, we ensured a chronological split between the training and test sets, guaranteeing that the input sequence at test time always occurs after the entire training dataset. Therefore, there is no overlap, and the retrieval module leverages the full training set.", "claims_and_evidence": "The article suggests that retrieval windows of different scales provide information at varying levels, such as local or global information. However, as mentioned in the article, when performing retrieval at different scales, RAFT scales the retrieval dataset, k, V, and input x, resulting in the sequences involved in the retrieval process corresponding to the same positions in the original sequence. It is unclear how this processing captures information at different scales. Furthermore, the paper does not include an ablation study to validate this claim, which may lead to confusion. I suggest that the authors conduct corresponding experiments to provide empirical evidence.", "methods_and_evaluation_criteria": "The proposed method and evaluation criteria are appropriate for the time series forecasting problem.", "theoretical_claims": "The theoretical claims are basically correct.", "experimental_designs_or_analyses": "1. The multiple periods used in RAFT, as mentioned in the \"Claims and Evidence\" section, lack experimental validation.\n2. In Section 5.2,  the authors compare RAFT with and without the retrieval module to demonstrate the benefit of retrieval for rare events. Since retrieval augmentation inherently aims to improve forecasting accuracy, it is expected to enhance results. To better highlight the retrieval module’s ability to capture rare patterns, I suggest the authors conduct experiments on carefully synthesized datasets to compare RAFT against strong baselines.\n3. Similarly, in Section 5.3, I recommend evaluating different baselines on the same dataset to substantiate the claim. Otherwise, comparing RAFT with and without retrieval alone may not be meaningful.\n4. Section 5.4 states that retrieval is also effective for Transformer-based models and evaluates AutoFormer. Could the authors provide more details on how the retrieval module is incorporated into AutoFormer?", "relation_to_broader_scientific_literature": "There is a strong connection to retrieval augmentation in NLP.", "essential_references_not_discussed": "Not at all.", "other_strengths_and_weaknesses": "Weakness see \"Experimental Designs\"", "other_comments_or_suggestions": "1. Since setting the stride to 1 when obtaining K and V may lead to high computational complexity, could increasing the stride be beneficial? How would it impact the results?\n2. During training, if the input sequence x is located in the middle of the training dataset S, does the retrieval module search for sequences both before and after x?", "questions_for_authors": "See above comments.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes RAFT, a retrieval-augmented method for time series forecasting, which enhances predictions by retrieving historical patterns similar to the current input and utilizing their subsequent values. The method demonstrates strong and consistent performance improvements over various state-of-the-art baselines across multiple benchmark datasets. Reviewers acknowledged the approach's novelty in applying retrieval to time series, its effectiveness, and the thoroughness of the empirical validation, including ablation and sensitivity studies. Initial concerns regarding scalability, the handling of multiple periods/scales, and specific experimental comparisons were effectively addressed by the authors during the rebuttal phase with additional experiments and clarifications. Therefore, I recommend acceptance of this paper.
Decision: Accept (poster)