Rethinking the Power of Timestamps for Robust Time Series Forecasting: A Global-Local Fusion Perspective
Authors: Chengsen Wang
Qi Qi
Jingyu Wang
Haifeng Sun
Zirui Zhuang
Jinming Wu
Jianxin Liao
Venue: NeurIPS 2024 poster (2024)
Keywords: Time Series Forecasting
Primary area: machine_learning_for_other_sciences_and_fields
TL;DR: We propose a plugin to enhance the robust prediction capability of time series forecasting backbones in the real world.

Abstract: Time series forecasting has played a pivotal role across various industries, including finance, transportation, energy, healthcare, and climate. Due to the abundant seasonal information they contain, timestamps possess the potential to offer robust global guidance for forecasting techniques. However, existing works primarily focus on local observations, with timestamps being treated merely as an optional supplement that remains underutilized. When data gathered from the real world is polluted, the absence of global information will damage the robust prediction capability of these algorithms. To address these problems, we propose a novel framework named GLAFF. Within this framework, the timestamps are modeled individually to capture the global dependencies. Working as a plugin, GLAFF adaptively adjusts the combined weights for global and local information, enabling seamless collaboration with any time series forecasting backbone. Extensive experiments conducted on nine real-world datasets demonstrate that GLAFF significantly enhances the average performance of widely used mainstream forecasting models by 12.5\%, surpassing the previous state-of-the-art method by 5.5\%.

Rethinking the Power of Timestamps for Robust Time Series Forecasting: A Global-Local Fusion Perspective
Chengsen Wang!* Qi Qi Jingyu Wang 12f
Haifeng Sun'
Zirui Zhuang' Jinming Wu! Jianxin Liao
Beijing University of Posts and Telccommunication, Beijing, China ?Pengcheng Laboratory. Shenzhen, China {eswang, qiq18266, {hfsun, zhuangzirui, wjm_18, liaojx//bupt edu.
edu cn
Abstract
Time series forecasting has played a pivotal role across various industries, includ- ing finance, transportation, energy. healthcare, and climate. Due to the abundant seasonal information they contain, timestamps possess the potential to offer robust global guidance for forecasting techniques. However, existing works primarily focus local observations, with timestamps being treated merely as optional supplement that remains underutilized. When data gathered from the real world is polluted, the absence of global information will damage the robust prediction capability of these algorithms. To address these problems, we propose a novel framework named GLAFF. Within this framework, the timestamps are modeled individually to capture the global dependencies. Working as plugin, GLAFF adaptively adjusts the combined weights for global and local information, enabling seamless collaboration with any time series forecasting backbone. Extensive experi- ments conducted on nine real-world datasets demonstrate that GLAFF significantly enhances the average performance of widely used mainstream forecasting mod- els by 12.5%, surpassing the previous state-of-the-an method by 5.5%. Code is available at httpp///ggtbbbboom///sssssig/gi
1 Introduction
Time series forecasting holds significant importance across various industries, including finance [2], transportation 4 [1, energy [28] 31), healthcare[16] 29), and climate g With the development of deep learning techniques, neural network-based methods [15]
18
40
47 have
notably propelled advancements owing to their strong capability in capturing dependencies within time series. The relevant models have evolved from statistical models to RNNs, CNNs, Transformers, and LLMs. However, existing research primarily concentrates on local observations within history sliding windows, overlooking the significance of timestamps.
Due to the abundant seasonal information they contain, timestamps possess the potential to offer robust global guidance for forecasting techniques. For instance, traffic volumes on weekdays typically exhibit high peaks. Regrettably, existing works primarily focus on local observations, with timestamps being treated merely an optional supplement that remains underutilized. DLinear [1] and FPT (7 completely overlook timestamps. Informer (45) and TimesNet (38) incorporate timestamps by summing their embeddings with position embeddings and data embeddings. These intertwined
Equual contribution. Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(a) Prediction Error
(b) Prediction Showcase
0.48
10.0
7.3
0.34
4.7
0.27
2.0
Informet
Timeseel
iTrnnsdommer
48
96
144
192.
240
288
336
384
412
480
S38
576
624
672.
720
768
816
Uirand Maretone
0.M
7.3
7.3
0.29
4.7
4.7
2.0
0.20
-0.6
-0.5
Informer
TimesNal
iTressGrmer
192
672
Figure I: The experimental results on Traffic dataset. (a) illustrates the outcomes of the ablation study on mainstream forecasting models and their variants. (b) depicts the visualization of traffic volume (upper), successful prediction case (lower right), and failed prediction case (lower left), respectively.
patterns encourage networks to extract information from more intuitive observations. iTransformer 24) embeds timestamp features separately into tokens employed by the attention mechanism. This embedding method across time points damaged the physical significance of timestamps. To validate this proposition, conduct ablation study the aforementioned models using the Traffic dataset. The results depicted in Figure [a) indicate that the performance of the models exhibits no significant decline after removing timestamps. Meanwhile, proposed GLAFF demonstrates notable enhancement in mainstream forecasting models.
Moreover, Time series collected from the real world often be polluted [5] For example, spike in electricity consumption coupled with short circuits can induce point anomalies, while reduction in traffic volume coinciding with holidays can evoke contextual anomalies. When local information gathered from the real world contains anomalies, the absence of global information will damage the robust prediction capability of most forecasting techniques [7] 43 46
We illustrate traffic
volume for San Francisco Bay area freeways at an hourly granularity in Figure [b). Typically, this sequence exhibits clear periodic pattern, alternating with five high peaks (weekdays) and two low peaks (weekends). However, due to a holiday, the week from 24 to 192 shows a deviation, resulting in three high peaks and four low peaks. As evident from the illustration in the lower right corner of Figure [b). the mainstream forecasting models [24 38 41 usually demonstrate reliable prediction capability. Nonetheless, when the observations within the history window include anomalies, as illustrated in the lower left corner of Figure [b), these models are significantly affected and yield notably underestimated predictions. Therefore, it is necessary reasonably incorporate more robust global information into the existing forecasting technique.
To address the aforementioned problems, propose generalized framework named GLAFF (short for Global-Local Adaptive Fusion Framework), aimed at enhancing the robust prediction capability of time series forecasting models in the real world leveraging global information. Specifically, GLAFF initially employs the Attention-basel Mapper to individually model the timestamps containing global standard distribution. Subsequently,
information and maps them to observations conforming to handle scenarios where anomalies are present within the observations of the sliding window, we utilize the Robust Denormalizer to inverse normalize the initial mappings, thereby mitigating the impact of data drift [17]. Finally, the Adaptive Combiner dynamically adjusts the combined weights for global mapping and local prediction within the prediction window, yielding the final prediction outcome. By fusing the robustness of global information with the flexibility of local information, GLAFF demonstrates substantial enhancement in the robust prediction capability of mainstream forecasting models. Additionally, GLAFF serves as a model-agnostic and and-play framework that can seamlessly collaborate with any time series forecasting backbone.
In general, the contributions of our paper are summarised as follows:
We propose GLAFF that leverages global information, represented by timestamps, to improve the robust prediction capability of time series forecasting models. GLAFF is a plug- and-play module that seamlessly collaborates with any time series forecasting backbone.

---

We design Robust Denormalization module to facilitate the adaptation of GLAFF for data drift, even when the observations encompass anomalies, alongside an Adaptive Combiner module for dynamically fusing global and local information.
We conduct comprehensive experiments on nine real-world benchmark datasets across five domains. The result demonstrates that GLAFF significantly improves the robust prediction capability of mainstream forecasting models.
2
Related Work
As significant real-world challenge, time series forecasting has garnered considerable attention. Initially, ARIMA (2 establishes an autoregressive model and performs forecasts in moving av- erage manner. However, the inherent complexity of the real world often renders such statistical methodologies 2
challenging to adapt. With the development of deep learning techniques, neural network- methods have become increasingly important. Recurrent neural networks 30 dynamically capture temporal dependencies by modeling semantic information within a sequential structure. Unfortunately, this architecture suffers from gradient vanishing/exploding and information forgetting when dealing with long sequences. To further improve prediction performance, self-attention mechanisms [19 [2 45 and convolutional networks 21
14
36 38 have been introduced to capture long-range dependencies. Additionally, prior research [41] has demonstrated that a simple linear network augmented by decomposition can also achieve competitive performance. Nowadays, with fast growth and remarkable performances of large language models, there is a growing interest 3 32 47 in utilizing L.M to analyze time series data. Recently, the i'Transfrrmer (24) has emerged as the state-of state-of-theearr method for time series forecasting tasks by embedding series from different channels into the variate tokens utilized by the attention mechanism.
Most time series forecasting techniques focus on local observations, with timestamps being treated merely as optional supplement that remains underutilized. DLinear 41). FPT 47 and other models (26 40, 433 completely overlook timestamps. When data gathered from the real world is polluted, the absence of global information will damage the robust prediction capability of these algorithms. Informer [45), TimesNet [38], and other models [23] 37 46] incorporate timestamps by summing their embeddings with position embeddings and data embeddings. These intertwined patterns encourage networks to extract information from more intuitive observations. i'Transfrmme 24 embeds timestamp features separately into tokens employed by the attention mechanism. This embedding method across time points damaged the physical significance of timestamps.
The processing of timestamps by the previous baselines and our proposed GLAFF can be abstracted as early fusion (feature-level fusion) and late fusion (decision-level fusion). Early fusion integrates modalities into single representation at the input level and processes the fused representation through the model. Late fusion allows each modality to run independently through its own model and fuses the outputs of each modality. Compared to early fusion, late fusion maximizes the processing effectiveness each modality and less susceptible to the noise single modality, resulting in greater robustness and reliability. This has been validated by extensive previous work 20 35
3 Methodology
We propose a model-agnostic and plug-and-play framework, GLAFF, which utilizes global informa- tion, represented by timestamps, to enhance the robust prediction capability of mainstream time series forecasting models in real-world scenarios. In multivariate time series forecasting, given the history observations of e channels within h time steps X = {xi..... X } Rgxx,, we aim to forecast the subsequent p time steps Y = (Xh-1,.... x E _Rxxe In addition to observations, we incorporate timestamps to provide global information. For each timestamp, we extract month, day, weekday, hour, minute, and second as timestamp features, respectively. For instance, for the timestamp 2018- 06-02 12:00:00 at moment f, its feature representation s; = [06, 02, 05, 12, 00, 00) I The holiday markers may also be included if accessible. Unlike observations, the timestamp features within the history window tion window T = (S.++,.....S)) 3ktr) RRxN are known. In this section, we describe the detailed workflow of the entire GLAFF framework and explain how it fuses local information X and global information S, T to predict Y.
(S),-. .Si....,, Sn. Sp.).................
E R and the timestamp features within the predic-
3

---

201 8-06-06 12:00:00
Eimbed
Alten
Project
2018-06-06 13:00:00
2018-06-10 11:00:00
Mapper
Denormalizer
Combiner
2018-06-02 12:00:00
Embed
Prnject
Denarm
201 8-06-0 13:00:00
Weight
Stats
201 2018-06-06 11:00:00
Mapper
Denormalizer
Statistics-bused
RNN-based
CNN-based
Transforme-based
Lincar-based
LLM-based
Figure 2: The overall architecture of GLAFF mainly consists of three primary components: Attention- based Mapper, Robust Denormalizer, and Adaptive Combiner.
3.1 Overview
GLAFF a plug-and-play framework that seamlessly collaborates with any time series forecasting backbone. The overall architecture of the plugin is depicted in Figure comprising three primary
components: Attention-bass Mapper, Robust Denormalizer, and Adaptive Combiner. Following local prediction Y E R2xe provided by the backbone network based on history observations X (maybe including underutilized history timestamps and future timestamps T), GLAFF leverages global information to revise it. Initially, the Attention-based Mapper captures dependencies between timestamps through an attention mechanism, mapping timestamp features and into an initial history mapping X E RRxx and an initial future mapping Y RPxe conforming to standard distribution. Subsequently, the Robust Denormalizer inverse normalizes the initial mappings X and Y to X gaxxe and Ý IRPxc based on quantile deviation between the initial mapping X and the actual observations X within the history window, mitigating the impact of data drift. Lastly, the Adaptive Combiner dynamically adjusts the combined weights of the global mapping Y and the local prediction Y within the prediction window according to the disparity between the final mapping X and the actual observations X within the history window, yielding the final prediction outcome Y. By fusing the robustness of global information and the flexibility of local information, GLAFF significantly enhances the robust prediction capability of mainstream forecasting models.
3.2 Attention-based Mapper
As depicted the green segment of Figure 2 our proposed Attention- Mapper employs a sim- plified encoder-only architecture within the Transformer (34] framework, comprising embedding layer, attention blocks, and a projection layer. Analogous to typical Transformer-based encoders 37 45), each timestamp feature is initially tokenized by embedding layer describe its properties, applied by self-attention for mutual interactions, and individually processed by feed-forward networks for series representations. Subsequently, projection layer is utilized acquire the initial mappings. Leveraging the capability of the attention mechanism for capturing long-range dependencies and parallel computation, the Attention-based Mapper can sufficiently model the global information embodied by timestamps.
Specifically, in Attention-based Mapper, the procedure for obtaining its corresponding initial mapping X, which conforms to the standard distribution, based on the history timestamps S, is succinctly delineated as follows:
T Embedding (S)
H Attention (H') i = -0...., ,
(1)
X Projection (H')

---

where H' E Rxxd denotes the intermediate feature variable output from the i-th attention block, and represents the dimension of the intermediate feature variable. The attention blocks are stacked with 2 layers to capture the high-level semantic information hidden within the timestamps. To maintain simplicity in implementation, both the embedding and projection layers are comprised of a single linear layer. Following conventional protocol, the primary computation steps for the i-th attention block are outlined as:
H' = LayerNorm (H' + MSA (H', H' HH))
H ++ = LayerNorm (H' + FeedForward (H')
where LayerNorm (.) represents the commonly adopted layer normalization and FeedForward (.) denotes the multilayer feedforward network. The MSA (Q, K, V) indicates the Multihead Self- Attention mechanism [3],, where Q. K. V serve as the query, key, and value respectively. Addi- tionally, dropout mechanism is incorporated to alleviate overfitting and enhance the generalization of the network. The process of obtaining the corresponding initial mapping Y based the future timestamps T, conforming to the standard distribution, mirrors the aforementioned procedure, simply substituting S and X in Equation I with T and Y respectively.
3.3 Robust Denormalizer
Due to the inherent variability of the real world, time series observations typically undergo rapid evolution over time, a phenomenon commonly referred to data drift [17]. This phenomenon can result in discrepancies across different time spans and hinder the generalization ability of deep learning models. Recognizing the presence of data drift, GLAFF employs two-phase untangling modeling strategy to address the global information represented by timestamps. In the first phase, the network in Attention-bast based Mapper produces initial mappings, denoted as X and Y, which are assumed to satisfy a standard distribution for reducing the difficulty of modeling the dependencies between timestamps and observations. Subsequently, in the second phase, leveraging the distribution deviations between the initial mapping X and the actual observations X within the history window, the Robust Denormalizer separately inverse normalizes the initial mappings X and Y to produce the final mappings X and Y, mitigating the impact of data drift.
To alleviate the impact of data drift, a feasible solution p dynamic factors from the original data through a normalization procedure before feeding them into the deep learning model, and subsequently reintroducing these dynamic factors via inverse normalization procedure after output from the deep learning model. The conventional inverse normalization procedure typically considers distribution deviations in mean and standard deviation. Nonetheless, this approach is susceptible to extreme values and lacks robustness when the observations contain anomalies. Instead of relying on mean and standard deviation, we employ median and quantile ranges 6). respectively, to enhance the robustness of the Robust Denormalizer against anomalies. As depicted in the yellow segment of Figure [ the procedure for Robust Denormalizer to inverse normalize initial mappings X and Y into final mappings X and Y can be succinetly expressed as:
23 25
has been proposed: removing
X-
X
x
where j E gixe and μ E 1xc represent the median of the initial mapping X and the actual observation X for each channel, respectively. Similarly, 5 E gx and E gixe denote the quantile range (the distance between the q quantile and the 1 q quantile) of the initial mapping X and the actual observation X for each channel. Specifically, when q = 0.75, and a correspond to the inter-quartile range (IQRE) for each channel of the initial mapping X and the actual observation X.
3.4 Adaptive Combiner
Owing to the intricacies of real-world scenarios, data preferences for model bias will continuously change with online concept drifts [42). Therefore, need data-dependent strategy to change
'IQR is defined as the difference between the 1st and 3rd quartiles of distribution or set of values, and is a robust measure of the distribution spread.
5

---

the model selection policy continuously. In other words, the combined weights of global and local information necessitate adaptive and dynamic updates. When the time series pattern exhibits clarity and stability, greater emphasis should be placed on robust global information. Conversely, increased attention should be directed towards flexible local information when the time series pattern appears ambiguous and variable. Within the framework of GLAFF, we employ an Adaptive Combiner to realize the adaptive adjustment combined weights.
As illustrated in the red segment of Figure 2 the Adaptive Combiner initially dynamically adjusts the combined weights of the global mapping Y and the local prediction Y within the prediction window, based on the deviation between the final mapping X and the actual observation X within the history window. Subsequently, we aggregate the dual-source information based on the combined weights to yield the final prediction Y. Specifically, the primary computation procedure of the Adaptive Combiner is represented as:
W MLP X (--X) x)
Y= Ew
W (YY
where W E 1xxex2 signifies the dynamically generated combined weight by the network based on the deviation between the final mapping X and the actual observation X within the history window. The @ denotes the concatenation operation based on the additional last dimension, and Σ denotes the summation operation performed across the last dimension. For simplicity, the weight generation network consists solely of a Multilayer Perceptron (MIP) containing a hidden layer and a layer of Softmax for weight normalization.
Through adaptive adjustment of combined weights, our method can effectively fuses the robustness of global information and the flexibility of local information, thereby enhancing its suitability for intricate and fluctuating real-world scenarios.
4 Experiment
4.1 Experimental Setup
Dataset We conduct extensive experiments on nine datasets across five domains, including Electric- ity, Exchange, Traffic, Weather, and ILI, along with four ETT datasets. Detailed dataset information provided in Appendix A.I We follow the standard segmentation protocol 24 37 45).
strictly di-
viding each dataset into training, validation, and testing sets chronologically to ensure no information leakage issues. The segmentation ratio for each dataset set to 6:2:2. Regarding prediction settings, we also adhere to established mainstream protocols [26] 38, 41) Specifically, we set the length of the history window to 96 for the Electricity, Exchange, Traffic, Weather, and four ETT datasets, while the prediction length varies within (96, 192, 336. 720). For the ILI, which has fewer time points, the length of the history window fixed at 36, and the prediction length varies within (24, 36, 48, 60).
Backbone To demonstrate the effectiveness of the framework, we select several mainstream forecasting models based on different architectures, including the Transformer- Informer (2021) [45] and iTransformer (2024) [24]. the Linear-basee DLinear (2023) 41, and the Convolution-based
TimesNet (2023) 338 Notably, iTransformer represents the previous state-of-the-att method in time series forecasting tasks. Further details regarding the backbone models are provided in Appendix
A.2 As described in Section [] these backbones encompass three different treatments for timestamps employed in prior forecasting techniques, namely summation (Informer, TimesNet), concatenation (iTransformer), and omission (DLinear).
The details of experimental setup can be found in Appendix A.3 All experiments are based on our runs, utilizing the same hardware configurations, and repeated 3 times with different random seeds.
4.2 Main Result
Table[[ the prediction outcomes for mainstream baselines and GLAFF We present a detailed version of this table in Appendix B.1 The results indicate that GLAFF significantly surpasses all four widely used mainstream baselines across all nine real-world benchmark datasets. In particular, GLAFF enhances the respective backbones by average of 12.5%.

---

Table 1: The forecasting errors for multivariate time series among GLAFF and mainstream baselines.
lower outcome indicates a better prediction. The best results are highlighted in bold.
Method
Informer MSE
MAE
MSB
Ours
MAE
D2Lineu MAE
MSE
Ours MAE
MSB
TimesNet
MAE
MSE
Ours
MAE
irranffommer MSE
MAE
MSE
Ours
MAE
Impr
11.1 Excbunge ET1m22 RTTOI F:712 ETThi Fleatritt
96
0.333
0.414
0.217
0.323
0.196
0.283
0.147
0.23%
0.175
0.288
0.154
0.248
0.153
0.246
0.120
0.19%
336
192
0,352
0.362
0.434
0.444
0.230
0.220
0.337
0.339
0.208
0.196
0.301
0.286
0.197
0.172
0.274
0.253
0.191
0.211
0.293
0.310
0.169
0.185
0.276
0.261
0.183
0.167
0.276
0.259
0.1143
0.168
0.240
11.216
15.7%
720
0.364
0.443
0.247
0.351
0.239
0.331
0.239
0.308
0.235
0.325
0.226
0.303
0.220
0.310
0.217
0.279
96
0.926
0.736
0.609
0.569
0.409
0.440
0.391
0.418
0.453
0.481
0.435
0.464
0.420
0.454
0,411
0.441
192
1,235
0.841
0.831
0.680
0.457
0.475
0.446
0.457
0.533
0.53]
0.530
0.517
0.191
0.502
0.474
0.482
8.8%
336
1.354
0.875
0.882
0.698
0.500
0.506
0.492
0.488
0.621
0.580
0.5%
0.560
0.538
0.528
0.534
0.519
720
1.764
0,857
0,937
0.730
0.576
0.609
0.556
0.844
0.697
0.773
0.661
0.716
0.679
0.704
0.615
96
0.708
0.549
0.423
0.443
0.159
0.278
0.128
0.205
0.183
0.298
0.174
0.276
0.177
0.287
0.172
01.271
3.36
192
0.997
1.133
0.667
0.688
0.747
0.860
0.570
0.599
0.207
0.187
0.330
0.309
0.206
0.165
0.238
0.265
0.218
(12240
0.529
0.345
0.219
0.294
0.319
0.306
0.220
0.199
0.329
0.311
0.196
012213
01330
0.295
12.
720
1.607
0.815
1.255
0.730
0.262
0.378
0.214
0.288
0.281
0.376
0.278
0.363
0.271
0.366
0.270
0.356
0.593
0.548
0.503
0.486
0.339
0.388
0.309
0.353
0.449
0.448
0.381
0.396
0.383
0.415
0.349
0.386
3356
192
0,588
0.611
0.726
0.576
0.534
0.707
0.707-000
0.628
0.325
0:450
0.391
0.450-0000
0.418
0.451
0.442
0.362
0.432
0.386
0,668
0.550
0.461
0.501
0.494
0.440
0.432
0.462
0.429
0.185
0.179
0.445
0.468
0.403
0.460
0.420
8.1%
720
1.037
0.786
0.925
0.719
0.508
1.5080000
0,493
0.493
0.467
0.619
0.559
0.563
0.507
0.566
0.532
0.564
0,519
0.186
0,311
0.147
0.266
0.115
0.252
0,080
0.165
0.121
0.234
0.110
0.212
0.120
0.235
0.221
192
0.242
0.348
0:230
(9381
01143
0:261
0.109
0.193
0.155
0.267
0.136
0.239
0.149
0.266
0.144
0.252
12.1%
336
0.454
0.466
0.308
0.380
0.176
0.294
0.148
0.229
0.190
0.295
0.175
0.269
0.185
0.293
0.182
0.283
720
0,861
0.616
0.719
0.561
0.225
0.340
0.221
0.274
0.242
0.334
0.225
0.309
0.233
0.333
0.232
0,327
96
0,735
0.728
0.223
0.391
0.051
0.164
0,046
0.155
0,076
0.198
0.066
0.177
0.058
0.172
0.051
0.158
192
1.016
0.861
0.421
0.547
0.099
0.238
0.093
0.225
0.135
0.272
0.115
0.242
0.113
0.245
0.108
0.231
16.4%
336
1.331
0.971
0.691
0.694
0.174
0.317
0.161
0.299
0.237
0.363
0:219
0.336
02100
0.339
0.196
0.314
720
2,054
1,263
1.152
0.922
0.314
0.446
0.308
0.439
0.636
0.618
0.395
0,582
0.517
0.551
0,510
0,529
24
3.374
1,356
2.487
1.106
2,087
1,131
1,875
0.963
1,478
0.713
1.333
0.684
1,188
0.659
1.129
0.658
3,094
1,299
2.617
1.157
1.107
1.756
0.957
1.294
0.748
1.204
0.695
1.061
0.695
1,039
0.6N2
9.9%
3,383
1.370
2.879
1.220
2,059
1.639
0.912
1.280
0.736
1.278
0.721
1.309
0.735
1.164
1.715
60
3.610
1.415
3,086
1.274
2.186
1.097
1.644
0.889
1.291
0.775
1.191
0.713
1.222
0.758
1.196
0.731
96
0.467
0.375
0.352
0.297
0.482
0.378
0_301
0.260
01360
0,314
0.322
0.278
0.308
0.272
0.283
11.249
Weather Traftic
192
0.455
0.462
0.371
0.378
0,343
0.335
0.281
0.288
0.453
0.449
0.358
0.356
0.302
0.306
0.263
0.261
0.373
0,364
0.320
0.314
0.331
0.325
0.282
0.277
0.338
0.327
0.285
0.279
0.291
0.301
0.253
0.259
19.5%
720
0.195
0.100
0.340
0.287
0:475
0.374
0.327
0.278
0
0.337
0.339
0,289
0.357
0,302
0.320
0.273
96
1.422
0.867
0.642
0.554
0.198
0.258
0.176
0.244
0.188
0.238
0.171
0.226
0.178
0,223
0.159
0,220
192
1.429
0,880
0.877
0.664
01851
0.237
0.282
0.296
0.219
0.265
0.280
0.312
0.234
0.293
0.317
0.278
0.233
0.288
0.277
0.314
0.231
0.310
0.268
0,214
0.273
0.265
0.300
9.6%
720
1.542
0.946
1.427
0.853
0.343
0.379
0.330
0.360
0.368
0.365
0.364
0.362
0.370
0.363
0.352
0L355
Impr
23.8%
13.1%
7.5%
5.5%
12.3%
By fusing the robustness of global information with the flexibility of local information, GLAFF can significantly improve the robust prediction capability of mainstream forecasting models. Specifically, in the case of DLinear, a Linear-based model that entirely disregards timestamps, GLAFF enhances its prediction accuracy by 13.1% 1%. For Transformer based Informer and Convolution- based TimesNet utilizing simple timestamp summation, GLAFF yields performance improvements of 23. 23.8% and 7.5%, respectively. In the case of Transformer-based i'Transformer employing direct timestamp concatenation, GLAAF still produces a 5.5% improvement in accuracy. Additionally, we note a diminishing boosting effect of GLAFF as the modeling prowess of the backbone increases, indicative of the complementary nature of global and local information. Nonetheless, for the state-of- tfrhh-r iTransformer, GLAFF continues to offer substantial benefits.
It evident that the enhancement of GLAFF varies across datasets with different characteristics. For datasets such as Traffic and Electricity, characterized by a significant number of channels and clear periodic patterns, GLAFF demonstrates superior capability in capturing the dependencies between timestamps and observations, resulting in performance enhancements of 19.5% and 15.7%, respectively. For the non-stationary datasets (25,, such as ETTh2, ETTm2, and Exchange, the Robust Denormalizer effectively alleviates the impact of data drift, thereby augmenting prediction accuracy by 12.1%, 12.1%, and 16.4%, respectively. Regarding common datasets like ETThI, ETTml, Weather, and ILL, although the performance of GLAFF may not be remarkable, it still yields improvements of 8.8%, 8.1, 9.6%, and 9.6%, respectively.
Given the burgeoning interest in leveraging LLMs for time series, we assess the performance augmentation of GLÄFF when applied to widely recognized FPT (2023) [7] as our baseline. FPT completely disregards timestamps similar to DLinear. We deploy two structures, GPT2(3) and GPT2(6), of FPT as outlined their paper. The ILI dataset has a history window length of 36 and a prediction window length of 48, while other datasets have a history window length of 96 and a prediction window length of 192. As delineated in the findings presented in Table 2 GLAFF also offer significant benefits to the LLM-based basclines.
backbones. Specifically, we employ the
7

---

Table 2: The forecasting errors for multivariate time series among GLAFF and LLM-based baselines. lower outcome indicates better prediction. The best results are highlighted in bold.
Method
GPT2(3)
+Ours
GPT2(6)
+Ours
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Electricity
0.19410.000
0.278.10.002
0.168 0.005
0.258.10.00
0.194110.001
0.279_0.002
0.171
0.258. 0.208.10.001
ETTh1
0.46620.001
0,483=00001
0.454=00.003
0,462::0.003
0.468:20.002
0.483=0.002
0.445:0.002
0.001=0.002
ETTh2
0.190=0.000
0.304+0.000
0.178=0.017
0.284=0.010
0.190=0.002
0.303=0.002
0.177=00.005
0.286==0000
ETTml
1400005
0.431+0.0033
0.388-0.000
0.409:00.005
0.412+00002
0.431=0.002
0.390=-0.012
0.413-00007
ETTm2
0.142=0.001
0.257=00.01
0.120=0.001
0.235=00.002
0.142=00002
0.256=200002
0.119::0.002
0.233==00002
Exchange
0. 10+00.000
0.238++0.002
0.090=0.0022
0.219:00.002
0,106++0.001
0.234=0.001
0.088
0.216=0.000
1.58510.041
0.909
0.782
0.002
0.85410.011
1.396 0.026
0.018
Traffic
0.370==0.022
0.309+0.002
1.2%%=0.002
0.256-00.001
0.301
0.312=0.001
0.301
0.262=0.000
Weather
100000
0.206
0.004
0.000
0.278:10000
0.002
0.002
Informer
DLineur
TimesNet et
iTransfomme
10.0
10.0
10.0
10:2
Traffic
7.3
////
-0.5
01
46
06
144
193
240
7801
.4
W5
144
140
240
790
48
66
144
193
340
7000
48
164
192
240
240
10.7
10.7
Eleetricity
4.-
-1.4
1.8
-18
-14
48
95
144
192
240
183
43
96
144
132
240
288
48
56
144
192
340
208
240 258
Ground Truch
Backhone
Backhone
Figure 3: The illustration of prediction showcases among GLAFF and mainstream baselines.
Additionally. validate the practical applicability of GLAFF, we assess the computation costs in Appendix B.4, The results indicate that GLAFF has little effect on model training and deployment across most scenarios, particularly when considering its significant accuracy enhancement.
4.3 Prediction Showcase
In addition to evaluation metrics, forecasting quality is crucial. To further compare GLAFF and the four mainstream forecasting models, we illustrate prediction showcases for two representative datasets in Figure 3 We provide the full prediction showcases for the nine datasets in Appendix B.5 It is evident that GL.AFF can yield more realistically robust predictions. the same time, the respective backbones susceptible to abnormal local information.
The Traffic dataset records traffic volume in hourly granularity. Typically, this sequence exhibits clear periodic pattern, alternating with five high peaks (weekdays) and two low peaks (weekends). However, owing holiday, the initial two days within the example history window do not exhibit the high peaks as usual. Due to the limited local information containing such contextual anomalies, Informer, DLinear, and iTransformer all think the prediction window should also consist of only low peaks. Although TimesNet generates predictions with high peaks, it displays an incorrect alternation between five high peaks and one low peak. By introducing sufficiently modeled global information, GLAFF has enabled the four mainstream forecasting backbones to recognize the existence of high peaks and the correct periodic patterns, thus yielding more accurate forecasts.
The Electricity dataset records electricity consumption in hourly granularity. Typically, this sequence exhibits a clear periodic pattern, alternating with five peaks (weekdays) and two flat segments (weekends). However, owing to a short circuit, the middle two days within the example history window show a spike in electricity consumption. Due the limited local information containing such point anomalies, DLinear, TimesNet, and iTransformer all think the flat segments in the prediction window should also contain a spike. Although Informer generates predictions without spikes, it completely ignores the presence of flat segments. By introducing sufficiently modeled global information, GLAFF has enabled the four mainstream forecasting backbones to recognize the contingency of spikes and the correct periodic patterns, thus yielding more robust predictions.
8

---

Table 3: The forecasting eттоrs for multivariate time series of ablation study among GLAFF and variants. lower outcome indicates a better prediction. The best results are highlighted in bold.
Methed
iTransforner
+
Ours
w/o Backhone MSE
w/o Attention MSB
w/o Quantile
w/o Adaptive
MSB
MAE
MSE
MAE
MAE
MAE
MSE
MAE
MSE
MAE
Traffic Electricity
96
0.1525
0.2460
0.1197
0.1979
0.2058
0.2663
0.1518
0.2450
0.1467
0.2465
0.1574
0.2502
192
0.1674
0.2593
0.1434
0.2157
0.2097
0.2793
0.1684
0.2610
0.1677
0.2662
0.1740
0.2662
336
0,1830
0.2762
0.1683
0,2395
0,2454
0.3014
0.1832
0.2775
0.1993
0.2954
0.1953
0.2877
720
0.2199
0.3097
0.2169
0.2786
0.2984
0.3386
0.2182
0.3092
0.2593
0.3403
0.2330
0.3171
96
0,3084
0.2717
0.2828
0.2485
0.3348
0.2723
0.3172
0.2806
0.2909
0,2684
0.2930
0.2612
192
0.3267
0.2794
0.2909
0.2528
0.3387
0.2736
0.3357
0,2884
0.2948
0.2737
0.2970
0.2610
336
0,3381
0.2850
0.3005
0.2594
0,3466
0.2794
0.3482
0.2958
0.3023
0,2804
0.3082
0.2706
720
0,3574
0.3015
0.3201
0.2730
0,3558
0,2906
0,3684
0.3113
0.3249
0.2984
0.3212
0.2819
Weather
96
0.1784
0.2229
0.1587
0.2199
0,2382
0.2695
0.1780
0.2214
0.1811
0,2270
0.1914
0.2379
192
0.2308
0.2675
0,2138
0.2654
0,2882
0.3105
0.2383
0.2733
0.2364
0.2768
0.2489
0.2832
336
0.2892
0.3099
0.2733
0.3058
0.3381
0,3414
0.2932
0,3146
0.2905
0.3134
0.3070
0.3251
720
0.3701
0,3634
0.3520
0.3547
0.4011
0,3813
0.3752
0.3664
0.3727
0.3649
0.3829
0.3722
Avg.
0.2602
0,2827
0.2367
0,2593
0.3000
0.3004
0,2646
0.2870
0.2555
0,2876
0.2591
0.2845
4.4 Ablation Study
We provide comprchensive ablation study to validate the necessity of the GLAFF components. We implement our approach and its four variants on the iTransformer backbone. The results of our experiments on three representative benchmark datasets are presented in Table 3 Detailed results for the nine real-world benchmark datasets are available in Appendix B..2
In w/o Backbone, we completely remove the backbone network within the GLAFF and map the future using only timestamps. Surprisingly, GLAFF still demonstrates favorable prediction performance without any observations. The average prediction accuracy of GLAFF even outperforms Informer and DI .iner,, and is also competitive with TimesNet and iTTransformer. Global information proves adequate in scenarios featuring clear periodic and stable distributions.
In w/o Attention, we substitute the stacked attention blocks with MLP networks having the equivalent size. Following the replacement of attention blocks, GLAFF fails to capture the dependencies among timestamps adequately. It proves challenging to map out precise observations solely from a single timestamp. Particularly notable is the most marked decline in performance on the Traffic dataset, which has the largest number of channels, indicating the greatest modeling challenge for GLAFF
In w/o Quantile, we replace the Robust Denormalizer with conventional inverse normalization. The experimental results illustrate that our design yields enhancements across all three datasets, particularly in the Electricity dataset. When the history window encompasses anomalies, conventional inverse normalization yields inaccurate estimates for distribution. Leveraging more robust quantiles, our Robust Denormalizer demonstrates enhanced robustness in mitigating the impacts of data drift.
In w/o Adaptive, we substitute the Adaptive Combiner with straightforward averaging for global mapping and local prediction. We distinctly find that dynamically adjusting the combined weights can prove more efficacious in accommodating fluctuating real-world scenarios, particularly evident in the non-stationary Weather dataset. By fusing global and local information adaptively, GLAFF can seamlessly collaborate with any time series forecasting backbone.
5 Conclusion
In this work, our focus lies in leveraging global information, as denoted by timestamps, to enhance the robust prediction capability of time series forecasting models in the real world. We introduce a new approach named GLAFF, serving as model-agnosti and plug-and-play framework. Within this framework, the timestamps are modeled individually to capture the global dependencies. Through adaptive adjustment of combined weights for global and local information, GLAFF facilitates seamless collaboration with any time series forecasting backbone. To substantiate the superiority of our approach, we have conducted comprehensive experiments on widely used benchmark datasets, demonstrating the substantial enhancement GLAFF provides to mainstream forecasting models. We hope that GLAFF can be used as foundational component for time series forecasting and call on the community to give more attention to global information represented by timestamps.

---

Acknowledgments and Disclosure of Funding
This work was supported by the National Natural Science Foundation of China under Grants (U23B2001, 62171057, 62101064, 62201072, 62001054, 62071067), the Ministry of Education and China Mobile Joint Fund (MCM20200202, MCM201 80101), Beijing University of Posts and Tellcommunication-CCiin Mobile Research Institute Joint Innovation Center, China Postdoctoral Science Foundation (2023TQ0039. National Postdoctoral Program for Innovative Talents under Grant BX20230052, and the BUPT Excellent Ph.D. Students Foundation (CX20241016).
References
[1] Adebiyi Ariyo Ariyo, Aderemi Oluyinka Adewumi, and Charles K. Ayo. Stock price prediction using the ARIMA model. In International Conference on Computer Modelling and Simulation, pages 106-112, 2014.
[2] G. E. P. Box and G. M. Jenkins. Some recent advances in forecasting and control. Journal of the Royal Statistical Society, 17:91-109, 1968.
[3] Ching Chang, Wen-Chih Peng, and Tien-Fu Chen. LLM4TS: two-stage fine-tuning for time- series forecasting with pre-trained 1lms. arXiv, 2308.08469, 2023.
[4] Yuzhou Chen, Ignacio Segovia-Domingee Dominguez, Baris Coskunuzer, and Yulia R. Gel. Tamp- s2genets: Coupling time-aware multipersistence knowledge representation with spatio-supra graph convolutional networks for time-series forecasting. In International Conference on Learning Representations, 2022.
[5] Hao Cheng, Qingsong Wen, Yang Liu, and Liang Sun. Robusttsf: Towards theory and design of robust time series forecasting with anomalies. In International Conference Learning Representations, 2024.
[6] Ailin Deng and Bryan Hooi. Graph neural network-based anomaly detection multivariate time series. In AAAI Conference Artificial Intelligence, pages 4027-4035, 2021.
[7] Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. Simmtm: simple pre-training framework for masked time-series modeling. In Annual Conference on Neural Information Processing Systems, 2023.
[8] Shengdong Du, Tianrui Li, Yan Yang, and Shi-Jinn Horng. Deep air quality forecasting using hybrid deep learning framework. IEEE Transactions on Knowledge and Data Engineering, 33: 2412-2424, 2021.
[9] Wei Fan, Pengyang Wang, Dongkun Wang, Dongjie Wang, Yuanchun Zhou, and Yanjie Fu. Dish-ts: A general paradigm for alleviating distribution shift in time series forecasting. In AAAI Conference on Artificial Intelligence, pages 7522-7529, 2023.
[10] Valentin Flunkert, David Salinas, and Jan Gasthaus. Deepar: Probabilistic forecasting with autoregressive recurrent networks. arXiv, 2201.00332 2017.
[11) Hui He, Qi Zhang, Simeng Bai, Kun Yi, and Zhendong Niu. CATN: cross attentive tree-aware network for multivariate time series forecasting. In AAAI Conference on Artificial Intelligence, pages 4030-4038, 2022.
[12] Qi-Qiao He, Shirley Weng In Siu, and Yain-Whar Si. Instance-based deep transfer learning with attention for stock movement prediction. Applied Intelligence, 53:6887-6908, 2023.
[13] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9: 1735-1780, 1997.
[14] Rob J Hyndman and George Athanasopoulos. Forecasting: Principles and Practice. OTexts, 2018.
[15] Yuxin Jia, Youfang Lin, Xinyan Hao, Yan Lin, Shengnan Guo, and Huaiyu Wan. WITRAN: water-wave information transmission and recurrent acceleration network for long-range time series forecasting. In Annual Conference Neural Information Processing Systems, 2023.
10

---

[16] Shruti Kaushik, Abhinav Choudhury, Pankaj Kumar Sheron, Nataraj Dasgupta, Sayee Natarajan, Larry A. Pickett, and Varun Dutt. AI in healthcare: Time-serie forecasting using statistical, neural, and ensemble architectures. Frontiers Big Data, 3:4, 2020.
[17] Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2022.
[18] Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In International Conference on Learning Representations, 2020.
[19] Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In Annual Conference on Neural Information Processing Systems, pages 5244-5254, 2019.
[20] Paul Liang, Amir Zadeh, and Louis-Philippe Morency. Foundations & trends in multimodal machine learning: Principles, challenges, and open questions. ACM Compuling Surveys, 56: 264, 2024.
[21] Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. In Annual Conference on Neural Information Processing Systems, 2022.
[22] Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X. Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International Conference on Learning Representations, 2022.
[23] Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Ex- ploring the stationarity in time series forecasting. In Annual Conference on Neural Information Processing Systems, 2022.
[24] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In International Conference Learning Representations, 2024.
[25] Zhiding Liu, Mingyue Cheng. Zhi Li, Zhenya Huang, Qi Liu, Yanhu Xie, and Enhong Chen. Adaptive normalization for non-stationary time series forecasting: A temporal slice perspective. In Annual Conference on Neural Information Processing Systems, 2023.
[26] Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations, 2023.
[27] Luis Manuel Pereira, Addisson Salazar, and Luis Vergara. A comparative analysis of early and late fusion for the multimodal two-class problem. IEEE Access, 11:84283 84300, 2023.
[28] Tiago Pinto, Isabel Praça, Zita A. Vale, and José Silva. Ensemble learning for electricity consumption forecasting in office buildings. Neurocomputing, 423:747-755, 2021.
[29] Chetanya Puri, Gerben Kooijman, Bart Vanrumste, and Stijn Luca. Forecasting time series in healthcare with gaussian processes and dynamic time warping based subset selection. IEEE Journal of Biomedical and Health Informatics, 26:6126-6137 2022.
[30] Syama Sundar Rangapuram, Matthias W. Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting. In Annual Conference on Neural Information Processing Systems, pages 7796-7805, 2018.
[31] Tárik S. Salem, Karan Kathuria, Heri Ramampiaro, and Helge Langseth. Forecasting intra-hour imbalances in electric power systems. In AAAI Conference on Artificial Intelligence, pages 2019.
[32] Chenxi Sun, Yaliang Li, Hongyan Li, and Shenda Hong. TEST: text prototype aligned em- bedding to activate Imm' ability for time series. In International Conference on Learning Representations, 2024.

---

[33] Sean J. Taylor and Benjamin Letham. Forecasting at scale. PeerJ PrePrints, 5:e3190, 2017.
[34] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Annual Conference on Neural Information Processing Systems, pages 5998-6008, 2017.
[35] Chengsen Wang. Zirui Zhuang, Qi Qi, Jingyu Wang, Xingyu Wang, Haifeng Sun, and Jianxin Liao. Drift doesn't 't matter: Dynamic decomposition with diffusion reconstruction for unstable multivariate time series anomaly detection. In Annual Conference Neural Information Processing Systems, 2023.
[36] Huiqiang Wang. Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. MICN: multi-scale local and global context modeling for long-term series forecasting, International Conference on Learning Representations, 2023.
[37] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Annual Conference on Neural Information Processing Systems, pages 22419-22453 2021.
[38] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations, 2023.
[39] Kun Yi, Qi Zhang, Wei Fan, Hui He, Liang Hu, Pengyang Wang, Ning An, Longbing Cao, and Zhendong Niu. Fouriergnn: Rethinking multivariate time series forecasting from a pure graph perspective. In Annual Conference on Neural Information Processing Systems, 2023.
[40] Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang. Hui He. Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequeney-domain mips are more effective learners in time series forecasting. In Annual Conference on Neural Information Processing Systems, 2023.
[41] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In AAAI Conference on Arrificial Intelligence, pages 1121-11128, 2023.
[42] Yifan Zhang, Qingsong Wen, Xue Wang, Weiqi Chen, Liang Sun, Zhang Zhang, Liang Wang, Rong Jin, and Tieniu Tan. Onenet: Enhancing time series forecasting models under concept drift by online ensembling. In Annual Conference on Neural Information Processing Systems, 2023.
[43] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In International Conference on Learning Representa- tions, 2023.
[44] Yu Zheng, Xiuwen Yi, Ming Li, Ruiyuan Li, Zhangqing Shan, Eric Chang, and Tianrui Li. Forecasting fine-grained air quality based on big data. In ACM SIGKDD International Conference Knowledge Discovery and Data Mining, pages 2267-2276, 2015.
[45] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang. Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI Conference on Artificial Intelligence, pages 11 1110-11115. 2021.
[46] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning, pages 27268-27286, 2022.
[47] Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin. One fits all: Power general time series analysis by pretrained LM. In Annual Conference on Neural Information Processing Systems, 2023.
12

---

A Detailed Experimental Setup
A.1 Dataset
We conduct extensive experiments on nine real-world datasets acorss five domains, including Elec- tricity. Exchange, Traffic, Weather, and ILI, along with four ETT datasets. Table4summarize the statisties of these datasets. These datasets have been widely utilized for benchmarking purposes and are publicly available. (1) Electricity] Ellectricit"fcommpiiss
comprises hourly electricity consumption for 321 customers
from 2012 to 2014. (2) Exchange encompassss panel data on daily exchange rates for 8 countries aggregates Traafii@aggreeatess hourly road occupancy rates measured by 862 sensors on San Francisco Bay Area freeways from 2015 to 2016. (4) Weather captures 21 weather parameters monitored every 10 minutes from Germany in 2020. (5) ILL records the percentage of patients with influenza- like illness and the total number of such patients collected weekly by the United States Centers for Disease Control and Prevention from 2002 2020. (6) ETT records the oil temperature and load characteristics of two power transformers from 2016 to 2018, each at 2 different resolutions (15 minutes and hour), resulting in a total of four datasets: ETTml, ETTm2, ETTh1, and ETTh2.
from 1990 to 2019. (3) Traffic
Table 4: The statistics of each dataset. Channel represents the variate number of each dataset. Length indicates the total number of time points. Frequency denotes the sampling interval of time points.
Dataset
Channel
Length
Frequency
Information
Electricity Exchange Traffic Weather
321
26304
I Hour
Energy
7588
Day
Finance
862
17544
1 Hour
Transportation
21
52696
10 Minutes
Climate
ILI
966
Week
Healthcare
ETTh1 & ETTh2
7
17420
I Hour 15 Minutes
Energy
ETTml & ETTm2
69680
Energy
A.2 Backbone
To demonstrate the effectiveness of our framework, we select several mainstream forecasting models based on different architectures, including the Transformer-based Informer (2021) and iTrrasformer (2024), the Linear-based DLinear (2023), and the Convolution-bass mentioned models are non-autoregressive forecasting models. (1) Informen 110 utilizes the ProbSparse attention and distillation mechanism to manage exceedingly long input sequences efficiently and incorporates a generative decoder to mitigate the еттоr accumulation inherent in autoregressive fore- commposition-enhaneee to attain competitive forecasting performance. (3) TemesNe 1
TimesNet (2023). All afore-
simple linear networks accurately models two-dimensional dependencies by transforming the mi-demnnsional time series into a collection of two-dimensional tensors, leveraging multiple periods to embed intra-periodic and inter-periode variations along the columns and rows the tensor, respectively. (4) i'Transformer embeds individual channel into token employed by the attention mechanism, facilitating the capture of inter-channel multivariate correlations while applying a feed-forward network to each token acquire nonlinear representations.
! casting methodologies. (2) DLinear employs
All backbones are based on our runs, using the same hardware. We utilize official or open-source implementations and follow the hyperparameter configurations recommended their papers.
bttps://githb..co http://hhww...oo//////u//u////// dimesseeeetttmm-meeeee-d-ddda
https:/ppmmsdott. dot.ca.gov
https://www www.wwwwwwwwwww............... www.wwp.en.aa....... mpg. de/wetter
edc,
https://github.co
wtwphhhwww..oooooohohhai//l///te Dataset
http::/ggihhb..cmm///ooaa///////
13

---

Algorithm GLAFF
import torch
from torch inport
class (LAFF
__init__ (salf, hist_len, channel, dim=512, dff=2048, dropouts0.1, head_nun=8, jayar_nun=2).
:paran hist_len: the length of the history window :param channel: the number the dataset channel
the MultiHeadhttettion iparen dff: the dimension of the feedforvard network
sparem dim: the dinension
pparam dropout: the dropeut proportion of the Multiläadkttentiion
pparam head_ nun: the number the attention head in Mult:Headtttention
paren layer_num: the nunber of the attention block in the Attention-based Mapper
super (GLAFF, self)....nit.
Mapper Sequential( En. (insar((, dim), Bn. TransformerBncoder
Traneforrarracoodeiaaye d_model=din, shead-head_ num, fii_ffffdroorrrdff. dropout-dropout, activati on gelu", batch,firattt first-True,
num, layers-lyyeeer nun, noruran. (ayerNorm(dam))
Dn. Lineer (din, channel)
self.Combiner
self Combiner un Sequentiii Linear (hist_len, dff), GELU(),
pn. Ln.iinear (dtr, 2), So.Softmax (din--11)
def forvard(eelf, hist.gt, hist_ts, pred_pt, pred_te, 9-0.75):
sparem hiet_gt: the true value the history vindow paran hist_ts: to: the timestamps in the history vindor paran pred_pr:
prediction paren pred_ te: the tinestamps the prediction vindov
the backbone prediction
sparam the quantile of the Robust Denormalizer one
map
hist_map self Nappar (hint_ta) # mapping value mapper in history window pred_nap aelf aelI.Napper (pred_ta) # the mapping value the napper in the prediction window
inverse normalize
means_gt gt
gt, True)
means means_nap torch. nadian (hist_nap, 1, True) [0]
etdey_g torch
stdey_nap torch.quantilee quantile (bist._gt 9. True) torch werehhguuniileebhetttteo 9, 1, True)
quantile map,, q, 1. True) torch.quantile (hiet__ppp q,
hist hist_map map (hist_nap _ap means_nan nap) stdav_nag atdev_gt gt neams_gt
(hist
pred_nap (pred_nap neans_nap) stdev_nax stdev_gt neana_gt
combine
arror bist_gt hist_nap
veight Combine permurrreee.eemmeee....... 2, unsqueeze (1)
pred corchhstacki [pred_napp pred_pr), dim--1) corch.sun(pre) x weight, dim=-1)
return pred
A.3 Implementation
We employ the Adam optimizer and L2 loss for model optimization, initializing the learning rate 10-4. The batch size uniformly
32, and the number of training epochs is fixed 10. Optimal hyperparameters for GLAFF are determined through grid search, employing common setup shared across all datasets and backbones. The number of the attention blocks Mapper is designated as 2 (selected from (1,2,3,4.5)), and while proportion p of dropout is set to 0.1 (selected from (0.000.1,,02,,,3,,0.., to 0.75, selected from (0.55, 0.65, 0.75, 0.85, 0.95). We provide details of each hyperparameter in
Attention-based
The quantile g the Robust Denormalizer configured
14

---

Appendix B.3
[.. All experiments are conducted using Python 3.10.13 and PyTorch 2.1.2, executed on an Ubuntu server equipped with an AMD Ryzen 9 7950X 16-Core processor and a single NVIDIA GeForce RTX 4090 graphies card, with each experiment repeated three times using different random seeds. If not explicitly stated, we report the Mean Square Error (MSE) and Mean Absolute Error (MAR) as evaluation metries, with lower values indicating superior performance.
The detailed implementation of GLAFF delineated in Algorithm I For simplification. the Adaptive Combiner is implemented through a straightforward two-layer linear network. We employ the nn. nn. TransformerEncoderr within PyTorch to realize the Attention- based Mapper. The source code and checkpoints have been made openly accessible to facilitate future research.
B Full Experimental Result
B.1 Robustness Analysis
We report in Table the means and standard deviations of the evaluation metrics for the baselines and GLAFF under three runs using different random seeds, facilitating the assessment of their robustness in long-range and ultra-long-range time series forecasting tasks. Evident from the mean of the experimental outcomes, GLAFF consistently demonstrates marked superiority over all four mainstream forecasting models across all nine real-world benchmark datasets. The standard deviation indicates the consistent stability and robustness of our proposed framework.
B.2 Ablation Study
We provide comprehensive ablation study to validate the necessity of the GLAFF components. We implement our approach and its four variants on the iTransformer backbone. Specifically, in w/o Backbone, completely remove the backbone network within the GLAFF and map the future using only timestamps. In w/o Attention, we substitute the stacked attention blocks with MLP networks having the equivalent size. In w/o Quantile, we replace the Robust Denormalizer with conventional inverse normalization. In w/o Adaptive, we substitute the Adaptive Combiner with a straightforward averaging mechanism for global mapping and local prediction. Due to space limitations, Section 4.4 presents experimental results for only three representative datasets, so we provide the complete results for nine real-world datasets in also Table 6
When excluding any observations within the history window, GLAFF demonstrates an average prediction accuracy superior to Informer across nine real-world datasets and remains competitive with DLinear. However, contrast to the three representative datasets outlined in Section 4.4 the average prediction accuracy of GLAFF across the nine datasets still lags behind TimesNet and iTransformer. We find that this discrepancy primarily stems from the highly non-stationary IIII dataset. In highly non-stationary scenarios, relying solely global information fails to adapt to the intricate and fluctuating nature of real-world conditions. Our experimental outcomes validate the indispensable nature of both the robustness of global information and the flexibility of local information for accurate time series prediction.
The experimental results show that the adaptive adjustment of combined weights within the Adaptive Combiner is the most important among various designs. When the time series pattern exhibits clarity and stability, greater emphasis should be placed on robust global information. Conversely, increased attention should directed towards flexible local information when the time series pattern appears ambiguous and variable. By fusing global and local information adaptively, GLAFF can seamlessly collaborate with any time series forecasting backbone to adapt to intricate and fluctuating real world scenarios. Furthermore, the stacked attention blocks within the Attention-basee Mapper and the robust inverse normalization within the Robust Denormalizer also yield notable contributions to enhancing the forecasting performance of GLAFF through efficient modeling of timestamps and robust mitigation of data drift.
It is noteworthy that the lack of either component design will corrupt the outcome of the GLAFF, resulting a prediction accuracy lower than the standard iTransformer baseline. This aligns with both common knowledge and theoretical expectations. In terms of global information, the introduction of insufficient (w/o Attention), inaccurate (w/o Quantile), or crude (w/o Adaptive) will all hurt GLAFFR This again validates that each component design in GLAFF is reasonable and necessary.
15

---

Table 5: The complete forecasting errors for multivariate time series among GLAFF and mainstream baselines. A lower outcome indicates a better prediction. The best results are highlighted in bold.
Method
MSF
Informer
MAB
MSE
+ Ours
MAF
MSE
DLinear
MAE
MSE
-Ours
MAE
I1.1 Eachunge ET R37ml RT E77Tl Fllemntty
0.3528=000005
041138000000
0.2173:0.0007
0.3232: 0001077
0.1962-+0.0001
0 0.2830++00000
0.1470=0.0008
0.2077
0.362310.0224
0.404200044
0.220110.0027
0.3292-0.0034
0.196210.0001
0.286110.0000
0.1715-.0.0039
0.2530-0.0000
336
0.3519
0.3322000004
0.2302.10.0015
0.33744.00000 00.0044
0.2080LL.. 0.0000
0,5008.10 10000000.0003
0.1965
0.2736 00.0050
720
0.3640:0.0119
0.447:00.0147
0.2472:00.011
0,3507
00.000
0.2592::0.0002
0.3305-10.000
0.2390:0.0007
U.3002:11MHH
96
0.9255::0.0694
0.7361:000367
0.0155
M.5688:10.000
0.4085
0,4399:10.000 00000
0.3909:0000099
0,4181 10.0097
192
1.2554-00.0679
08437-000146
0.8312+0.0976
0.6797-110444
0.4572+0.0006
0.4752-00.000
0.4462-0.0106
0145565-
HJ, ILI00K3
1.354111,00277
0.87511000107
0.881610.0249
0.69781000003
0.5012.10.0005
0.506110..001
0.4922-0.00040
0,4679
720
1.2641::0.0222
0:8572:0000001
0.936610.0036
0..009220.0199
0.609910.0012
0.5757.10.000)
0.6088-00.0025
0.5560
10.0011
96
0.7063.10.0800
0.54911000330
0.4427 10.02710 0.4019.0.0339
10.0000
0.2776.10.0013
0.1282. 0.0051
0.20-49 1 10,055
192
1.1529::0.1889
06889=000499 0493
0.06990000622
O.
11869=00.000
O 1.501=000000 00000
0.1651=0.0138
0.2382
+ 10,0068
336
0.9972.0.1608
0.6666000000
0.747110.1531
0.5702210.0558
0.2066.10.002
0.33011000002
0.206410.0224
0.2650
0.0077
1.60661003017
0.815210.0768
0.2550.10.04889
0.7203.00.020
0.2621 10.0006
0.3783.10.002)
0.2135 0.0077
0.2875
96
0.5932.10.0320
0.5475
1.503110.0000
0,4862 0026
L.3385L0.000
O 0 0006
0.3085 0.0230
0.3529
1000330
192
0.6111:0.00117
0.5763-000008
0.53991000111
0.5248.00000
U. 0.3936
0.4179-00.0007
0.0619
0.3857
3356
-8881--0.0366
0.7257-0.0206
0.7066-0.0932
1.6278- 11.137
0. 0.4502-0.0009
0.514400.0004
0.4416 0.0208
0,4320
10,0990
720
1.0574.0,00180
0.7859L0.0104
0.925410.00230
0..185
0.5075
0,1932.000010
0.4930L00.006
0,4669L10,0O
0.186010,0089
0.31091000003
0.1473L00.0003
0.2656-10.0279
0.11481000000
0.200810.0012
0.000300.0000
0.1649 10.0000
192
0.242110.00266
0,3481 100022
0.2300100.0657
0.3405 10. 03.0551
0.1425 0002
12.0510.00005
A.1085L.0.0024
0.1932.10.000
336
0.554200.0614 14
O 04664-00.022
0,30N2-000007
0L379K-00JNTT
0.1760-00.011
0 2039=00.0019
0.1477:000027
0.224Z 1-202--1-8836
720
0.8609 1..000002000
0.6156..0.0065
0.7192.0.0489
0.5611
0.2 0.2252.10.0010 10
0.339610000011
0.2211
0.2737 10,000
96
0.754910.0501
0.7282-0.0226
0.3905.0.0042 0.222610.0059
0.0505.100000
O. 0.161210.0013
0.006210.0004
0.1552.10.007
192
1.015810.0258
0.0001000016
0.221410.0237
0.547110.0147
1.098810.0000
0.00510000033
0.0929-0.0015
0.2244
10,0022
336
1.3807:00.0715
0.6912:0.0292
0.6939 20.0000011100 50
0.1739:20.0003 003K
0.3100000000
0.1612:0.0028
0.2943 08003R
720
2.0556-0.0588
1.231-001132
1.1516+0.071X
0.9222-0.1225
0.5137+0.0001
D 04400-0000044
0.3077-0.0019
0.4386
-0.0039
24
3.3735:10.1341
0.56L=000083
2.4865
L1055
2.0071
J. 1.331420.003
08745-000551
1.0099
36
1.927_00.0099
2.17330.1699
1..00
2,0651
L.007
L.7557..0.08
0,9571
10,0201
3.382810.006 14147=00033
000000
2.792200.099
10.0415
2.056610.0138
0.59910.0000
1.2033.0.0430
0.91151000166
3.6102+0.16338
0.1401
0.274
2.1861 00.0222
1099=00.0030
1.6441-0.033
0.8894 10.0
Traftic
192
96
0.4868=0000265
0.054910.0044
0.3748-20.0150
0.0012
0.3521-0.0125
0.343010.0094
0 0.2882-00.006
1.290910.0007
000000
0.44671100000
0.37821000001
0.3561
0.0007
0.3019
0.2011
0.2609
10,001
336
0.6622.10.0088
0.3778.0.0054
0.3352.00.000
0.2812.10.0023
0,4526.10.00
0.057710.00002
0.3062..0.001
0,2631
10,0002
0.4945:000116
0.3097:000073
0..403:000166
0.2070
0,4749
10.0001
0.3735:0000002
0.3267:0.0022
0.2776.10.001
Weather
192
14216::0,2777
1.4292:0.3063
0.6770000077
0.860000.0916
0.87671000006
0.642010.0339
0.663610.0004
06
0.237) 1000005
0.1975:10.001 0013
0.2963100.0010
0
0.2194100.0041
0.1758:0.0022
0.2795
L.2435510.003
336
10.6671
10076.10.1661
1.506310.1420
0.851110.0366
0.2819
0.003110.0009
0.052_10.0088
0.3121 10.003
720
1.542410.0315
0.945700.0200
1.2771000059
0.833
0.5427
0.00410000002
0,3896 0.3297.00.010
Method
MSE
TimesNet
MAB
MSE
Ours
MAE
MSE
iTransfommnrr
MAE
MSE
+Ours
MAE
Exchange E71m2 ET1bll FT122 FT/h1 Fllctricity
96
0.1753=0000110
0.2090000000
0.1541=0.0016
0.2475-000019
0.1525:10.000
0.2400+0.0004
0.1197=000005
0.19791000000
192
0.190820000433
0.203310.0018
0.169410.0017
00.008
0.1674-0.0003
0.259210.0007
0.1434_0.0009
0.2157 1.0011
336
0,2112110.004
0.310110.0075
0.185010.0041
0.2762.10.0039
0.1830.10.0016
0:2762.10:0016
0.1683..0.0023
0,2395L10.011
0.2553:00.003
0.3257:000008
0.226110.0112
0.3027
0.2199:0.0007
0.3007110.0010
0.2169:0.00.99
0.2776
10.066
0.483:=0.0095
0.8111=00004
0.4349=0.0055
0.4643=000007
0.4200+0.0039
0.536=000005
0.4112:00.017
0.4007=0.0014
192
0.5331:10.0019
0.5308-000064
0.5195
0.5170 0066
0.0011110.0111
0.5020110.0059
0.4739:00.0039
0.4816:1.0021
336
0.620910.0027
0.5796.000003
0.596110.0014
0.559810.0014
0.5382L10.0035
0.5276.10.0029
0.5336_00.0033
0.5188 100031
720
0.814410.0717
0.0003
0.773010.0163
1..005
0.715910.0117
06293110.006
0,7040-0.0034
0.6153
0,1831=00.0028
02290=000012
0.1740=0.0064
0.2764=0.0035
0.1769+0.0039
0 00007
0.1720-0.00-9
0.2708 0.0013
192
0.2177=0.0020
0:3294=000000
0.2050=0000026
0.3060=0.0017
0.1992:00.019
0.310900.0021
0.0057
0.2054
336
0.23961.0.0026
0.3457.000000
0.218510000033
03319410.00333
0.219710.0000
0.329010.0003
0.2127-000008
0.3061 10,0023
720
0.2805110.0103
0.576110.000
0.2779.0.0137
0.363410.0117
0.2708.10.0140 40
0.3661 110.010
0.0032
0.3562 00.094
96
0.649310.0025
0.1177000017
0.3805
0.3983
0.3832.10.007 0071
0.0045
0,3489 0.0065
0.0065
0.3856110000
192
0.4478=000007
0.408=0000085
0.1400=000000
0.4315=0.0120
0.4285=000009
0.455
0,4834:
0.0080
0.4201 = 000055
0.5502=0.0629
0.0011200113
0.904410.0046
04616110.0226
0.485
0,479420.0020
0.4680=000064
0.0003
720
0.618510000341
0.5588.000167
0.563010.0006
0.5072.10.0199
0.566010.0004
0 0.5023100.004
0.0056
0.519010.0041
96
0.1212.0.0022
0.23361000033
0.110310.0004
02120100.0335
0.119910.0030
022471000066
0.0012-0.0027
0.2205.0.0002
192
0.1550==0.0076
0 033665000065
0.1358=0000111
0.2385=000015
0.1491=0.0031
0 0.2656=0000088
0.1498
0.2517=0.0000
336
0.190==00.0070
0:299900.008
0.1750=20.0041
0.268620.0016
0.184520.0026
02222:00:0025
0.1816:0.0017
0.2829-00.009
720
0.241710.0047
0.303510.001
0.225410.0020
0.309310.0004
0.333110.0051
0333442000322
0.2319-0.0084
0.3267 10.006
96
0.0757:00.0000
01000100000
0.065610.0001
0.176910.0044
0.058010.0025
0.171910.0037
0.001410.0022
0.0000
192
0.1347.10.0074
0.271510.0075
0.1145
0.241510.004
0.1126.10.012
0.241710.0025
0.1078_0.0062
0.2306
336
0.2367=:0.0078
03331=00000)
0.2187=00.0046
0.3364-0.0022
0.2096=00.0053
0.3091:0.0041
0.1963:0.0028
0.3144
720
0.635720.0185
0.617900.0094
0.594510.0266
0.5822-0.0168
0.517210.0134
0.500520.0076
0.0102
0.529110.00990
1.4781=0.2225
0.9128:000491
0.330020.0889
0.0944:0.0119
1.17772000227
0.6991-000222
L.289000.0025
0.66761000067
11.1
36
1.295110,0892
0.74761000257
1.203910.0124
1.698810.000 0098
1.0608100000 0.0062
0.005110.0155
L.031
0.6822 0.0174
1.2800=00.088
0.7363-0000 0064
1.277-000339
0.7217
1.2094+0000277
0.7051
11637=000268
0.71448-0.0094
1.290900006888
0.7730=0.0261
1.190620.0354
0.7126=0.0093
1.2223=0.10022
0.7575=0.0409
0.016
0.0017
96
0.3001=0.0085
0::41:000055
0.2215:0.000
0.2784=0.0056
0.3084=0.0014
0.777220.0018
0.2827
0.0004
Tralfic
192
0.363910.0026
031132200003
0.305110.0060
0.27681000006
0.32671000020
0.2794 0023
0.000410.0144
0.0076
336
0.373410.0044
0.300000.001
1.3308L0000000
0.2824.0.0000 0067
1.33811100000 0..000
0.285010.0007
0.3005L000000
L.259440000043
720
0.3958=000005
0.3368-000008
0.0049
0.2055=00.0000
0.3574=0.0010
0.3015:0.0012
0.1201
0.2730
Waater
96
0.1880=0.0170
0:330=000016
0.1706=00.0034
0.2257
0.1784=0.0005
0.2229=0.0029
0.1587=0.0061
1.2119:00.0067
192
0.23442:0.0122
0.277=00.0070
0.2331:0.0062
0.2772-00.0039
0.200810.0011
0.275
0.2138 0.008
0.2684
0.292810.00-8
0.111110.0006
0.287510.0062
0.2892110.0019 0.3144L0.0053
0.309910.0030
0.2733
0.3058
720
0.3684100.002
0.365410.0016
0.364010.0022
0.3622.10.0019
0.5701 10.0015
0 0.363410.0011
0.352010.0044.
0.3547110.0026
16

---

Table 6: The complete forecasting errors for multivariate time series of ablation study among GLAFF and variants. A lower outcome indicates a better prediction. The best results are highlighted in bold.
Method
iTrrasformer MSB
+
Ours
w/o Backhone MSE
w/o Attention MSB
w/o Quantile MSE
w/o Adaptive
MAE
MSE
MAE
MAE
MAE
MAE
MSE
MAE
ETTh2 ETThl Electricity
96
0.1525
0.2460
0.1197
0.1979
0.2058
0.2663
0.1518
0.2450
0.1467
0.2465
0.1574
0.2502
192
0.1674
0.2593
0.1434
0.2157
0.2097
0.2793
0.1684
0.2610
0.1677
0.2662
0.1740
0.2662
336
0,1830
0.2762
0.1683
0,2395
0,2454
0.3014
0.1832
0.2775
0.1993
0.2954
0.1953
0.2877
720
0.2199
0.3097
0.2169
0.2786
0.2984
0.3386
0.2182
0.3092
0.2593
0.3403
0.2330
0.3171
96
0,4200
0.4536
0.4112
0.4407
0.5741
0,5150
0.4282
0.4560
0,4230
0,4598
0.4590
0.4754
192
0,4944
0.5020
0.4739
0,4816
0.6422
0.5409
0.5166
0.5112
0,4861
0.5019
0.5427
0.5258
333
0,5382
0,5276
0,5336
0,5188
0.7020
0,5833
0.5797
0.5513
0,5496
0,5386
0.5836
0.5469
720
0.7159
0,6293
0.7040
0.6153
0.8481
0.6541
0.7564
0.6442
0.7065
0.6294
0.7429
0.6429
96
0.1769
0.2871
0.1720
0,2708
0,2095
0.2786
0.1941
0.3270
0,1969
0,3308
0.2028
0.3337
192
0.1992
0.3109
0.1957
0.2954
0.2847
0.3274
0.2325
0.3618
0.2223
0.3562
0.2335
0.3631
336
0.2197
0,3290
0.2127
0,3061
0.2937
0,3384
0.2432
0,3722
0,26339
0,3896
0.2611
0,3848
720
0.2708
0,3661
0.2698
0,3562
0.3082
0.3567
0.2909
0.4051
0,2786
0.4036
0.3041
0.4197
ETTm2 ElTml
96
0,3832
0.4145
0.3489
0,3856
0,4161
0.4100
0.4106
0.4233
0.3595
0.4021
0.3841
0.4180
192
0.4285
0,4451
0.4034
0.4201
0.4932
0.4587
0.4547
0.4522
0,4138
0.4385
0.4397
0.4511
336
0.4851
0,4794
0.4680
0,4604
0.5467
0,4883
0,5278
0.4990
0.4797
0.4806
0.4984
0,4881
720
0,5660
0,5323
0.5636
0,5190
0.6210
0,5316
0,5942
0,5441
0,5814
0,5383
0.5883
0,5400
96
0,1199
0.2347
0,1112
0.2205
0.1334
0.2340
0.1197
0.2337
0,1129
0,2293
0.1
0.2296
192
0.1491
0,2656
0.1438
0.2517
0,1612
0,2624
0.1554
0.2677
0,1459
0.2627
0.1470
0,2614
336
0.1845
0.2933
0.1816
0,2829
0.19979
0.2901
0.1884
0.2944
0.1935
0,3006
0.1837
0.2924
720
0.2331
0,3334
0,2319
0.3267
0,2464
0,3272
0,2332
0.3297
0.2791
0,3627
0.2324
0,3306
ILI Exchange
96
0,0580
0.1719
0.0514
0.1580
0.0972
0,2212
0.0595
0.1748
0.0613
0.1771
0.0669
0.1870
192
0.1126
0.2447
0.1078
0,2306
0,1448
0,2779
0.1
0.2475
0.1154
0,2470
0.1203
0.2539
336
0.2095
0.3391
0.1963
0.3144
0,2614
0.3772
0,2079
0.3371
0.2081
0.3359
0.2129
0.3404
720
0.5172
0,5505
0,5102
0.5291
0.6089
0.5952
0,5308
0,5582
0,5359
0.5605
0.5461
0.5668
24
1.1477
0,6593
1.1289
0.6576
2.3361
1,0211
1.1321
0.6759
1.2641
0.6907
1.4264
0,7593
36
1.06088
0.6951
1.0391
0.6822
1,9812
0.9408
1.1258
0.7193
1.0839
0.7035
1.2890
0.7669
48
1.2094
0.7351
1.1637
0.7148
1.8142
0.9166
1.2250
0.7509
1.2472
0.7483
1.3530
0.7789
60
1.2223
0.7575
1.1956
0,7308
1.5257
0.8484
1,2615
0.7734
1,2239
0.7551
1,2319
0.7593
96
0.3084
0.2717
0,2828
0,2485
0.3348
0,2723
0.3172
0.2806
0,2909
0,2684
0.2930
0.2612
Weather Traffic
192
0,3267
0.2794
0.2909
0,2528
0,3387
0.2736
0.3357
0.2884
0.2948
0,2737
0,2970
0,2610
336
0,3381
0,2850
0.3005
0,2594
0,3460
0.2794
0,3482
0.2958
0.3023
0,2804
0,3082
0.2706
720
0.3574
0.3015
0.3201
0,2730
0.3558
0,2906
0,3684
0.3113
0.3249
0,2984
0.3212
0,2819
96
0.1784
0.2229
0.1587
0.2199
0.2382
0.2695
0.1780
0.2214
0,1811
0.2270
0.1914
0,2379
192
0.2308
0.2675
0.2138
0.2654
0,2882
0.3105
0.2383
0.2733
0.2364
0,2768
0.2489
0.2832
336
0,2892
0.3099
0.2733
0,3058
0,3381
0.3414
0.2932
0.3146
0,2905
0,3134
0.3070
0.3251
720
0.3701
0.3634
0.3520
0.3547
0.4011
0.3813
0.3752
0.3664
0.3727
0.3649
0.3829
0.3722
Avg.
0.3957
0.3875
0.3794
0.3689
0.5291
0.4278
0.4100
0.3987
0.4027
0.3971
0.4243
0.4036
B.3
Hyperparameter Analysis
To assess the sensitivity of GLAFF for various hyperparameter configurations, we conduct a com- prehensive hyperparameter analysis. It is worth noting that, in theory, GLAFF introduces only one additional core hyperparameter, the quantile g in the Robust Denormalizer. Therefore, conducting hyperparameter selection for GLAFF requires only very little work. However, to comprehensively compare the effects for different parameter configurations, we also explore the number of attention blocks and the proportion p of dropout in the Attention-based Mapper. We implement our method on the iTransformer backbone. The length of the history window is fixed at 96, while the length of the prediction window is set to 192. The experimental results for the eight data-rich datasets are depicted in Figure
The quantile in the Robust Denormalizer impacts the ability of GLAFF to adapt to data drift, particularly when anomalies are present within the sliding window. Higher quantiles can render GLAFF overly sensitive to magnitude, diminishing its robustness against outlier data. Conversely, smaller quantiles may hinder the ability of GLAFF to detect distribution changes, consequently impairing its capacity to adapt to data drift. For datasets ETTh1, ETTml, and Traffic, higher quantiles correlate with increased prediction accuracy, while for datasets ETTh2 and Weather, lower quantiles yield higher prediction accuracy. Moreover, datasets ETTm2, Electricity, and Exchange demonstrate relative robustness to quantile variations. To equalize prediction accuracy, we adopt quantile g of 0.75 for all datasets and backbones.
17

---

64940
64921
0,4088
64011
0,3976
0,3472
MSE
MSE
MSE
0.3822
0..5
0.65
6.75
0.85
0.95
00
0.
02
63
Electicity
Exchange
Figure 4: The forecasting emrors for multivariate time series of hyperparameter analysis among different configurations for GLAFF. A lower outcome indicates a better prediction.
The number of attention blocks in the Attention-base Mapper influences the modeling process for global information representation. greater depth attention blocks enables GLAFF model times- lamp dependencies more sufficiently and uncover latent high-level semantic features. Nevertheless, augmenting the depth the network also escalates computation costs and exacerbates convergence challenges. According to experimental findings, all benchmark datasets exhibit robustness to varia- tions in the number of attention blocks, thus facilitating the deployment of GLAFF. Considering training cost and model performance, choose the number / of attention blocks for all datasets and backbones.
The proportion of dropout in the Attention-based Mapper impacts the generalization performance of GLAFF. When the dropout proportion is too small, indicating the retention of too many neurons, the model tends to overfit, thereby diminishing its generalization capacity. Conversely, if the dropout proportion is excessively large, meaning the exclusion of too many neurons, the model may struggle to effectively capture the underlying features, decreasing prediction accuracy. Moreover, high dropout proportion can also impede the convergence of model training due to the varying network structures observed in each training sample. From the experimental results, except for the ETTml and Traffic datasets, most datasets exhibit insensitivity to the choice of dropout proportion p. To simplify the parameter selection challenge, we adopt dropout proportion p of 0.1 for all datasets and backbones.
B.4 Efficiency Evaluation
To assess the practical applicability of GLAFF in real-world environments. we comprehensively compare training time and memory usage between baseline models and GLAFF across nine datasets. The experimental results are summarized in Table 7 Specifically, incorporating GLAFF results in an average 23.4382s increase in the training time and 25. 06.8MB increase in memory usage. Presently, with hardware resources evolving rapidly, this computation costs may not affect the training and deployment GLAFF in most scenarios, particularly when considering the significant accuracy enhancement it offers. Moreover, it is common practice to trade off computation costs for enhanced prediction accuracy. For instance, the TimesNet baseline exhibits an average training time increase of 355s and an average memory cost increase of 145MB compared to the DLinear baseline, yielding mere 8.4% increase in average prediction accuracy. In contrast, our GLAFF incurs a training time increase of 14s and a memory usage increase of 25MB while achieving a 13.1% 1 increase in prediction accuracy compared to the DLinear backbone. In scenarios demanding high prediction accuracy, this computation costs are acceptable.
Notably, owing to the unique forecasting mechanism (mapping), the memory usage incurred by GLAFF remains insensitive to both the dataset and the prediction length, consistently hovering at approximately 25MB. Additionally, we observe that the introduction of GLAFF not only results in substantial enhancements in prediction accuracy in all scenarios, but also significantly reduces the training time in some scenarios (such as the TimesNet backbone on the ETTh2 dataset, the Informer backbone on the Traffic dataset, the DLinear backbone, and the TimesNet backbone). We postulate
18

---

Table 7: The training time and memory usage for multivariate time series forecasting among GLAFF and mainstream baselines. lower outcome indicates a better efficiency.
Informer
+
DLinear
Ours
TimesNet
Ours
iramsformer
Ourx
Methol
Time
Size
Time
Size
Time
Size
Time
Size
Time
Size
TTme
Size
Time
Size
Time
Size
(6)
(MB)
(8)
(MB)
(MB)
(MB)
(MB)
(8)
(MB)
(6)
(MB)
(6)
(MB)
ETTal ETTal ETTh2 ETThl Elletiiity
96
15.92
67.05
17.63
92.52
5,701
0.071
8.226
86.18
146.2
100.5
171.6
27.71
48.48
49
73.95
192
20.84
67.05
22.50
92.52
7,663
0.142
11.74
25.61
161.9
146.2
162.1
171.7
29.37
48.67
34.86
74.14
336
29.05
67.05
31.40
92.52
10.58
0.249
17.49
25.72
396.2
146.3
279.5
171.7
31.82
48,95
40.43
74.42
720
50.30
67.05
60.02
92.52
18.88
0.533
35.79
26.00
489.2
146.4
433.4
171.9
38.24
49.70
58.18
75.17
96
5,400
62.76
8.828
67.61
0.394
0.071
3.708
24.93
82.26
145.5
89.64
170.4
2,057
48.48
5.486
73.34
192
6.195
62.76
11.55
87.61
5.528
$5.00
145.6
169.0
170.4
2.107
7.191
73.53
336
8.104
62.76
16.49
87.61
0.543
0.249
8.687
25. 10
274.1
145.6
312.7
170.5
2,095
9:
10:25
73,81
720
12.78
62.76
30.18
87.61
0.727
0.535
18.44
25.39
336.6
145.8
314.0
170.6
2.269
49.70
19.81
74.56
96
5.605
62.76
8,833
87.61
0.405
0.071
3,740
24.93
98.52
145.5
105.6
170.4
1.945
48.48
5.476
73.34
192
6.155
62.76
1 11.60
87.61
0.459
0.142
5.517
25.00
179.3
145.6
171.1
170.4
2,027
48.67
7.214
73.53
336
8.125
62.76
16.51
87.61
0.535
0.249
8.702
25.
375-4
145.6
221.7
170.5
2,084
10.30
73.81
720
12:73
62.76
30.10
87.61
0.710
0.533
18:45
25.39
372.7
145.8
299.7
170.6
2.072
49.70
19.81
74.56
96
21.43
62.76
35.98
87.61
1.550
0.071
15.114
24.93
293.0
145.5
286.9
170.4
7.929
48.48
21.96
73.34
192
25.07
62.76
47.45
87
1,833
0.142
22.47
25.00 15.00
382.0
145.6
556.5
170.4
8.757
67
73.53
336
33.71
62.76
68,
87.61
2,096
0.249
35.95
25.
1017
145.6
1296
170.5
8.003
48.95
-12.53
73.81
720
54.73
62.76
129..8
67.61
3.099
0.533
79.09
25.39
1530
145,8
1405
170.6
8.719
49.70
84.83
74.56
96
21.83
62.76
35.44
H7.61
1,553
0.071
15.11
24.93
306.7
145.5
308.4
170.4
8.144
48.48
22. 09
73.34
192
15.12
62.76
47.47
87.0
1,801
0.142
22.58
25.00
393.7
145.6
786.6
170.4
8.590
48.67
29. 54
73.53
336
33.41
62.76
87.61
2.126
0.249
35.93
2631
170.5
8.318
73.81
720
55.16
62.76
129.6
87.61
3.177
0.533
79.13
25. 39
1499
145.8
1470
170.6
9.696
49.70
84.85
74.56
ILI Exchange
96
2.242
62.77
3.823
87.63
0.173
0.071
1,600
24.93
32.51
145.5
37.20
170.4
0,865
48,
2.336
73.34
192
2,580
62.77
4,929
87.65 67.63
0,188
0.142
2.301
25. 00
66.52
145.6
66,49
170.4
0.928
48. 67
3.031
73.55
336
3,340
62.77
6.752
N7.63
0.218
0.249
3.543
25.11
159.0
145.6
209.2
170.5
0.859
48.95
4,209
73.81
720
4.852
62.77
11.46
87.63
0,280
0.533
7.005
25. 25.39
144,4
145.8
138.3
170.6
0.797
49.70
7.563
74.56
24
0.245
62.76
0,343
87.14
0.019
0.007
0.128
24.39
1.608
145.5
1.650
169.9
0.162
48.22
0.253
72.61
36
0.244
62,76
0.326
87.14
0.019
0,010
0.129
24.40
1.664
145.5
1.717
169.9
0,146
48.25
0.252
72.63
0.246
62.76
0.336
87.14
0.018
0.014
0.132
1.688
145.5
169.9
0.146
0.249
72.66
60
0.248
62,76
0.354
87.14
0.019
0.017
0,143
24.40
1,769
145.5
1,701
169.9
0,141
48,30
0,270
72.68
96
18.17
74.45
14.9/7
0
8.166
0.071
8.067
26.60 60
75.01
147.2
74.67
173.7
60.
48,48
62.07
75.01
Weather Traffic
192
25.29
74.45
19.77
101.0
12.17
0.142
11.49
26.67
131.2
147.3
102.1
173.8
63.71
48.67
64.85
75.20
336
37.10
74.45
27.89
101.0
17.77
0.249
16.94
26.78
189.1
147.3
177.3
173.8
67.71
48.95
69.42
75.48
720
65.19
74.45
49.28
101.0
31.00
0.533
266.4
147.5
286.0
174.0
77.75
82.50
76.23
96
16.69
62.95
27.68
87.83
1,645
0.071
11.71
24.95
206 8
145.6
274.0
170.5
6.877
48.48
17.06
73.37
192
19.88
62,95
35.94
87.83
1.988
0,142
17.23
25.02
363.6
145.6
500.2
170.5
7,318
48.67
23:05
73.55
336
26
62.95
51.92
87
2.472
0.249
27,46
653.6
145.7
743.1
170.5
7.614
48.95
32.99
73,84
720
43.84
62.95
98.33
87.83
3.918
0.533
59.82
25.42
785.8
145.8
786.7
170.7
8.277
49.70
64.74
74.59
that this phenomenon can attributed to the incorporation of global information, which accelerates the convergence of the network.
B.5 Prediction Showcase
In addition to evaluation metrics, forecasting quality is crucial. To comprehensively compare GLAFF and the four mainstream forecasting models, we present the complete prediction showcases for the nine real-world datasets in F For the ILI dataset, the length of the history window is set to 36, while the length of the prediction window is set to 48. Except for the ILI dataset, the length of the history window is set to 96, and the length of the prediction window is set to 192. We can observe that by fusing the robustness of the global information and the flexibility of the local information, GLAFF demonstrates superior suitability for intricate and fluctuating real-world scenarios, thereby enhancing the ability of the backbone model to generate predictions that closely correspond to the ground truth.
C Broader Impact
The GLAFF proposed in our paper focuses on leveraging global information, as denoted by times- tamps, to enhance the robust prediction capability of time series forecasting models in the real world. As a model-agnostic and plug-and-play module, GLAFF significantly improves mainstream forecasting models, positively influencing domains such as finance, transportation, energy, healthcare, climate, etc. Furthermore, GLAFF may inspire the community to give more attention to the utilization of global information and catalyze further development of time series forecasting techniques. The source code and checkpoints have been made publicly available to support future research. This paper only focuses on the algorithm design. Using all the codes and datasets strictly follows the
19

---

Informer
DLincar
TimesNet
iTransformer
10.7
10.7
16.7
10.7
Electricity
7.6
4.4
-1.8
-1.8
0.3
-22
-22
-.2
96
146
192
240
45
96
144
190
240
200
144
192
2.00
208
168
192
240
248
ETTh2
1.7
1.7
1.7
08
96
144
193
240
2801
48
96
1:4
190
240
205
48
56
164
192
260
2001
192
240
245
1.4
1.4
ET1ml
-1.2
-37
-3.7
3.7
146
144
192
240
240
2000
0.3
0.3
0.3
ET1m2
0.7
4.7
-03
-1.6
-1.6
-1.6
-15
Exchange
1/
1.4
0.8
0.8
96
144
192
240
76R
340
4.0
4.38
4.8
4.1
1.5
1.3
1.3
13
12
34
16
40
14
12
30
36
48
40
14
12
34
50
73
16
12
24
48
80
72
100
Truffic
/V
.06
.05
144
96
144
192
340
268
240
218
24k
1.7
1.7
1.7
1.7
Weather
0.5
0.5
0.5
0.5
.07
4.7
0.7
Ground Truth
Backbone
Backbone Ours
Figure The complete illustration prediction showcases among GLAFF and mainstream baselines.
20

---

corresponding licenses (Appendix A.1 and Appendix A.2 negative social impact.
There is no potential ethical risk or
D Limitation
While GLAFF exhibits encouraging performance benchmark datasets, it is subject to certain limi- tations. As model-agnostic and plug-and-plas framework, GLAFF incurs considerable computation costs attributed to the utilization of stacked attention blocks in Attention- Mapper (Appendix B.4). Presently, with hardware resources evolving rapidly, this computation costs may not affect the training and deployment of GLAFF in most scenarios. However, GLAFF will likely encounter operational challenges resource-constrainee edge devices, thus restricting its applicability. In future work, plan to explore lighter weight and more efficient architectures, such as dilated convolutional or graph neural networks, to replace the conventional attention mechanism.
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please refer to Abstract and Section Introduction
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please refer to Appendix D Limitation
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
22

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibiliyy
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Please refer to Section Experimental Experimental Sctup
Setup and Appendix A Detailed
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
23

---

Answer: [Yes]
Justification: The data is publicly available as said in Appendix [A.] [..] Dataset Please refer Appendix A.3 Implementation for the core code. We submit the full code as part of the supplemental material. After the review period, we will open-source the full code and checkpoints, but not before in order to not break anonymity of the submission.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co co/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurIPS code and data submission guidelines (https "pipps.c/puullc//guu /
/CodeSubmeei ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer to Section 4.1 Experimental Setup Appendix A Detailed Experi mental Setup and Appendix B.3 Hyperparameter Analysis
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification; Please refer Section 4.2 Main Resuliland Appendix B. Guidelines:
Robustness Analysis
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer Appendix B.4iixx.B444iee Efffciency Evaluation
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: This research conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer to Appendix C Broader Impact
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
25

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Please refer to Reference, Appendix A.] Dataset and Appendix A.2 Backbone
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
26

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Please refer Appendix A.33 Implementation for the code. We submit the full code as part of the supplemental material. After the review period, we will open- source the full code and checkpoints, but not before in order not break anonymity of the submission.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
27


Reviews:
[{"review_id": "8CMSpcdv6L", "reviewer": "Reviewer_NzBw", "summary": "The authors proposed a plugin to utilize global information from timestamps in time series forecasting tasks. The proposed plugin consists of three main components: attention-based timestamp mapper, robust denormalizer and adaptive combiner. The authors found that using the proposed plugin in combination with various backbone models helped achieve an average of 12.5% increase in forecasting prediction performance measured as mean square error and mean absolute error on 9 real-world datasets.", "questions": "•\tPlease refer to the weaknesses section.\n\n•\tDo the authors see a way to use the proposed method in datasets where timestamps are not available?\n\n•\tDo the authors plan to publish the exact same datasets used in the experiments?\n\n•\tIt may be beneficial to see sample outputs from historical and future mappers.", "limitations": "As the authors mentioned, adding the plugin to backbone network cause additional computational time and memory usage.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "•\tThe paper is well structured and easy to follow.\n\n•\tThe authors compared the proposed method with different timestamps treatments like summation, concatenation and omission.\n\n•\tWell-illustrated prediction showcases provide examples of the usefulness of the proposed plugin.\n\n•\tThe experimental results are comprehensive and impressive. Extensive results on 9 real-world datasets in five domains confirm the superiority of the plugin models used. The authors presented the average percentage improvement of individual models and datasets, which is useful for reviewing the results.\n\n•\tThe results with the proposed plugin are best for all datasets and backbone models used.\n\n•\tThe prosed method requires only one core hyperparameter, the quantile q in the robust denormalizer, which is 0.75 by default.\n\n•\tThe proposed method is flexible and can be used with any backbone model.\n\n•\tThe authors conducted an ablation study, proving that all components are important. The authors explained how core components can be helpful data drift and concept drift mitigation.\n\n•\tThe authors conducted computation time and memory usage study.", "weaknesses": "•\tThe overhead for lightweight models like DLinear is significant.\n\n•\tThe authors did not provide average percentage increases in computation time and memory consumption for individual backbone models and datasets, which would be useful for quickly reviewing results.\n\n•\tFigure 2 suggests that statistics are calculated separately for historical and future mappings denormalization which is not true.\n\n•\tLack of theoretical proof for the proposed plugin.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your positive comments and insightful suggestions. Please find our response below.\n\n**Q1: Explanation of computation time and memory consumption.**\n\nAs stated in Appendix D, higher prediction accuracy is typically accompanied by a higher computational cost. Balancing these needs requires consideration within the context of actual production demands. This study primarily focuses on enhancing prediction accuracy. In future work, we will consider model reduction to broaden the applicability of the proposed method. Additionally, we have not provided the percentage increase in computation time and memory consumption due to insufficient statistical significance. In the case of memory consumption, our proposed method increases consumption by 25MB for both DLinear (from 0.1MB) and TimesNet (from 150MB), representing percentage increases of 25000% and 17%, respectively. This statistical method may cause confusion and misunderstanding for readers.\n\n**Q2: Explanation of the misunderstanding caused by Figure 2.**\n\nWe apologize for the confusion. As you noted, the statistics used in the inverse normalization for the historical mapping $\\tilde{\\mathbf{X}}$ and future mapping $\\tilde{\\mathbf{Y}}$ are indeed identical, as indicated in the parentheses in Equation 3. The two \"Stats\" are plotted in Figure 2 primarily for a more organized framework diagram.\n\n**Q3: Theoretical proof of the proposed method.**\n\niTransformer does not utilize the timestamp information of the prediction window, which may contribute to its underutilization of timestamps. Therefore, we focus solely on the comparison between our proposed method and the summation scheme represented by Informer. The method proposed by Informer and our approach can be abstracted as early fusion (feature-level fusion) and late fusion (decision-level fusion). Early fusion (Informer) integrates modalities into a single representation at the input level and processes the fused representation through the model. Late fusion (GLAFF) allows each modality to run independently through its own model and fuses the outputs of each modality. Compared to early fusion, late fusion maximizes the processing effectiveness of each modality and is less susceptible to the noise of a single modality, resulting in greater robustness and reliability. This has been validated by extensive previous work [1, 2]. To mitigate the effect of noise and fully exploit the robustness of global information represented by timestamps, our proposed method adopts late fusion.\n\n**Q4: Applicability of the proposed method in the absence of timestamps.**\n\nThanks to the timestep-level embedding and attention mechanism, our proposed approach can tolerate missing timestamps. Specifically, it only requires adding the corresponding mask matrix for the Attention-based Mapper to exclude the missing timestamps, allowing the entire framework to function seamlessly. We present the MSE results of the iTransformer backbone on two representative datasets. The history window is set to 96, and the prediction window to 192. The results in the table below clearly demonstrate the robustness of our proposed method to varying percentages of missing timestamps.\n\n|   Dataset   | iTransformer | + Ours 0% | + Ours 10% | + Ours 20% | + Ours 30% |\n| :---------: | :----------: | :-------: | :--------: | :--------: | :--------: |\n|   Traffic   |    0.3267    |  0.2909   |   0.3067   |   0.3225   |   0.3429   |\n| Electricity |    0.1674    |  0.1434   |   0.1490   |   0.1554   |   0.1691   |\n\n**Q5: Availability of the datasets.**\n\nThe timestamp information we integrate has been extensively utilized, though ineffectively, by various existing methods. We introduce no additional information or assumptions. As detailed in Appendix A.1, all datasets employed (including timestamp information) are publicly accessible. We will also release the code and datasets upon completion of the paper review.\n\n**Q6: Visualization of the output from the mapper.**\n\nWe visualize the Mapper and Denormalizer outputs for the two prediction cases in Section 4.3 of the paper **in Figure 1 of the Global Response attachment PDF**. From the experimental results, the Attention-based Mapper captures the majority of the shape information, while the Robust Denormalizer aligns the distribution of mapping values. Together, they provide comprehensive and robust assistance for accurate prediction of the backbone model.\n\n**References**\n\n[1] 2024, Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions\n\n[2] 2023, A Comparative Analysis of Early and Late Fusion for the Multimodal Two-Class Problem", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n0xm1QIiiA", "reviewer": "Reviewer_wbSq", "summary": "This paper proposes GLAFF which encodes the time stamps of time series and performs self attention across the encodings of the time dimensions, combining it with the output of a global time series forecaster via a learned weighting scheme. As GLAFF is a set of feature constructions, it’s generally additive in performance against any backbone architecture. The main requirement is that GLAFF seems to require knowledge of when the prediction should be produced for.\n\nMore specifically, the time stamps themselves are used to generate de-medianed and de-quantized predictions. This is extra side information, and so it should generally tend to be helpful – some of the tested datasets have clear time information. For example, traffic tends to spike in the morning and afternoon hours, while electricity also has daily and monthly peaks and valleys. This is a natural encoding of seasonality and other similar regular events.", "questions": "-\tIn Eq 3, where do the median and quantiles of Y come from? Do they come from the global set of sequences? This is I think more for clarity at this point but want to confirm.\n\n-\tIt seems that most of the improvement in exchange and weather, which are not periodic, is driven by improvement in the worst model (informer). In general, should we expect that there is less improvement on aperiodic data because the time embeddings are less helpful?\n\n-\tOften in time series prediction tasks, there is other side information, is this naturally best situated to being encoded solely inside the backbone architecture or should it be modeled alongside the time stamps?\n\n-\tTables 1 and 2: I’d suggest using bar charts to make the presentation more engaging, and moving the tables to the appendix.\n\n-\tI find it fairly surprising that informer, timesNet, and iTransformer tend to have minimal dropoff when removing the timestamp pieces. Is this lack of dropoff consistent across datasets?\n\n-\tCould the authors include an average improvement across methods for a fixed horizon? It would be interesting to see if DLinear improves the most as a result of adding in the time step information.", "limitations": "n/a", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality:\n\n-\tMost other papers in this literature tend to focus primarily on architecture, and the ones that do take into account some sort of feature information seem to not incorporate it that well.\n\n-\tOverall, I like the idea of separating the time encoding into what’s essentially its own network.\n\nQuality:\n-\tI appreciate the knockout studies of the varying parts, which is good experimental design.\n\n-\tI also appreciate the knockout study of iTransformer and would like to see more of these types of results.\n\nClarity: \n\n-\tOverall, the paper is pretty well written. It’s pretty much clear what’s going on.\n\nSignificance: \n\n-\tImproving time series forecasting is obviously a highly important problem, and improving the base model is a general purpose technique that should generally be quite useful.", "weaknesses": "Quality:\n\n-\tMy biggest concern in terms of usefulness (and unfortunately, this is somewhat a critique of the entire vein of literature here) is that the time stamps of the prediction window are trained on. This can be, in some sense, a pretty strong lookahead bias. In most deployment settings, we cannot pre-train on the time stamps that we’re going to use to generate the sequence because we do not know the absolute value for the time stamp during train time. \n\n       o\tThus, the method is limited to explicitly semi-regular time series where the forecast clock is known ahead of time. By non-regular forecasting, we can think of wanting to predict the inventory levels of a product after the next sixty sales, or to predict the price of a foreign currency after some amount of trading volume in it. Instead, in regular forecasting, at the start of the day, we may wish to predict the next day’s electricity demands (one of the benchmark datasets).\n\nClarity:\n\n-\tI personally find $\\tilde X$, $\\hat X$, $\\hat Y$ to be quite confusing in terms of notation. Perhaps $\\hat T$, $\\tilde S$, $\\hat S$ could be used instead throughout to denote the fact that these values come from the encoded time stamps. \n\n       o\tIn line 197, I would encourage against setting the final prediction as $Y$, but rather as $\\hat Y$.\n\n-\tI also find the methods section somewhat unclear, as it’s a bit tricky to parse out that the median / quantile stats are dependent on the input time series, X. Perhaps it would be better to not separate X and Y entirely, but rather point out that the time series is really cat(X, Y). \n\n-\tFrom the intro and contributions, it could probably be made clear that other papers _do_ consider time stamp information; however, it doesn’t seem to be helpful in knockout studies. This is more of a writing note than something major.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your positive comments and insightful suggestions. Please find our response below.\n\n**Q1: Usefulness of the proposed method in non-regular forecasting.**\n\nThe initial point to state is that the timestamp information we integrate has been extensively utilized, though ineffectively, by various existing methods. We introduce no additional information or assumptions. Regarding the non-regular forecasting scenario you describe, we consider it a novel situation distinct from the current task. It appears to utilize transaction counts instead of time intervals to denote timing changes. Due to the differing domains, our proposed approach cannot natively support this scenario. As mainstream methods like iTransformer cannot address time series forecasting with missing values, we cannot expect a framework to solve all forecasting challenges. To my knowledge, neither the datasets nor the baselines examined in our paper are relevant to the new scenario you mentioned. Nevertheless, the idea of information fusion that we have proposed may be beneficial. Furthermore, the non-regular forecasting appears intriguing, and we look forward to further discussion after the review period concludes.\n\n**Q2: Confusion caused by the notations.**\n\nWe use $\\mathbf{S}$ and $\\mathbf{T}$ for timestamps, where the last dimension is 6, and $\\mathbf{X}$ and $\\mathbf{Y}$ for observations, where the last dimension represents the number of channels in the multivariate time series. We apologize for the confusion and will incorporate your suggestion to enhance the readability of our article. Regarding the output in row 197, a marker on $\\mathbf{Y}$ is indeed necessary to differentiate it from the actual labels, and we will address this in the final version.\n\n**Q3: Explanation of parsing statistics and Equation 3.**\n\nThe median and quantile in Equation 3 are derived from the historical actual observation $\\mathbf{X}$ and the historical initial mapping $\\tilde{\\mathbf{X}}$ within a local sliding window to accommodate the dynamics of the time series. We apply the same statistics, as indicated in the parentheses in Equation 3, to both the historical mapping $\\tilde{\\mathbf{X}}$ and future mapping $\\tilde{\\mathbf{Y}}$. We apologize for any confusion, but we do not advise concatenating $\\tilde{\\mathbf{X}}$ and $\\tilde{\\mathbf{Y}}$, as the following Adaptive Combiner will require separate historical mapping $\\hat{\\mathbf{X}}$ and future mapping $\\hat{\\mathbf{Y}}$ .\n\n**Q4: Performance of the proposed method on aperiodic data.**\n\nRobust global information represented by timestamps indeed offers less assistance for predicting non-periodic data than periodic domains such as transportation and electricity. In fact, this paper primarily focuses on effectively integrating available assistive information. Accurate prediction of dynamic and non-periodic time series is more challenging and requires the inclusion of more assistive information. The information fusion approach we have proposed is also valuable for integrating other supplementary information.\n\n**Q5: Modeling approaches for other side information.**\n\nWe should design the modeling method for each type of assistive information based on its characteristics and practical requirements. According to the findings of our paper, simple fusion techniques such as summation or concatenation often fail to fully utilize the assistive information. However, more nuanced modeling generally entails greater resource consumption. In practical applications, it is essential to carefully balance prediction accuracy with computational cost.\n\n**Q6: Enhancing the engagement level of presentations.**\n\nIt is true that bar charts provide a more visually engaging presentation of backbone performance improvements. However, arranging various combinations of backbones, metrics, datasets, and prediction lengths within a limited space remains a considerable challenge. We will persist in seeking more effective presentation methods.\n\n**Q7: Impact of timestamps on baseline methods.**\n\nWe supplement the ablation results about timestamp across the nine datasets. The history window is set to 96 and the prediction window to 192 for all datasets except ILI, where the history and prediction windows are both set to 36. The MSE results **in Table 1 of the Global Response attachment PDF** clearly indicate that simple fusion methods, such as summation or concatenation, are ineffective with timestamps.\n\n**Q8: Results across varying lookback window lengths.**\n\nWe supplement the MSE results on two representative datasets with fixed prediction windows by varying the history windows. The prediction window is fixed at 192. From the experimental results **in Table 2 of the Global Response attachment PDF**, iTransformer and DLinear, which derive their final predictions from a linear layer, demonstrate a general trend of enhanced prediction accuracy with longer history windows. Furthermore, our proposed methods consistently enhance prediction performance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KYYIUqBA3x", "reviewer": "Reviewer_XBcx", "summary": "The paper introduces GLAFF, a novel framework that enhances time series forecasting by modeling timestamps to capture global dependencies and adaptively balancing global and local information, resulting in a significant improvement of 12.5% over existing methods in experiments across nine real-world datasets.", "questions": "Please check concerns", "limitations": "The paper does not discuss limitations. Please elaborate more on when the method may perform poorly.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The writing is clear and easy to understand.\n2. Code is provided.\n3. Comprehensive experiments consistently enhance performance.", "weaknesses": "My concerns are as follows:\n1. This paper focuses on utilizing timestamps but only discusses and compares some general time-series forecasting methods. Please discuss the differences and performance comparison with existing methods that focus on better utilizing timestamps, such as methods across temporal scales [1][2] and representation for each timestamp [3].\n   - [1] AutoCTS: Automated correlated time series forecasting\n   - [2] METRO: a generic graph neural network framework for multivariate time series forecasting\n   - [3] TS2Vec: Towards Universal Representation of Time Series\n\n2. The method appears simple and lacks technical contributions, resembling a straightforward dual-pathway combination; it lacks theoretical backing, particularly in theoretical analysis of existing backbones' capabilities with timestamps, and only provides a case study for discussion.\n\n3. I noticed that the adaptive weight has a minimal effect, and I am curious about how the weights in the Combiner are initialized, how they are tuned, and whether they are sensitive.\n\n4. The usage has limitations, requiring high-precision timestamps. Please analyze the impact of timestamp granularity and noise on the results.\n\n5. By simply increasing the lookback window, existing methods usually have better performance (better leverage periodicity). Please analyze the performance improvements under different lookback window lengths, especially longer lengths, to validate more realistic effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments. We will answer the questions one by one.\n\n**Q1: Comparison with additional baselines.**\n\nThank you very much for your supplement. All three articles have significant contributions to our field. However, there seems to be some misunderstanding regarding our work. The timestamps mentioned in our paper denote external assistive information, such as \"2024-08-01 12:00:00,\" along with their embeddings. Upon reviewing their papers and codes, it is evident that none of these three works incorporate this type of information, thereby classifying them in the same category as the baseline DLinear. The term \"timestamp\" used in their papers is more accurately replaced with \"timestep\", which indicates a concept of temporal position. Generally, these three works do not utilize assistive information represented by timestamps and do not belong to \"existing methods that focus on better utilizing timestamps\". Furthermore, the superiority of the baselines adopted in our paper (updated to 2024) compared to these three articles (published in 2021) has been extensively validated by previous work[1, 2]. Due to spatial limitations in the response, we will refrain from repeating it here.\n\n**Q2: Theoretical analysis of the proposed method.**\n\niTransformer does not utilize the timestamp information of the prediction window, which may contribute to its underutilization of timestamps. Therefore, we focus solely on the comparison between the summation scheme represented by Informer and our proposed method, which can be abstracted as early fusion (feature-level fusion) and late fusion (decision-level fusion). Early fusion (Informer) integrates modalities into a single representation at the input level and processes the fused representation through the model. Late fusion (GLAFF) allows each modality to run independently through its own model and fuses the outputs of each modality. Compared to early fusion, late fusion maximizes the processing effectiveness of each modality and is less susceptible to the noise of a single modality, resulting in greater robustness and reliability. This has been validated by extensive previous work [3, 4]. To mitigate the effect of noise and fully exploit the robustness of global information represented by timestamps, our proposed method adopts late fusion. Furthermore, we supplement the ablation results about timestamp across the nine datasets. The MSE results **in Table 1 of the Global Response attachment PDF** clearly indicate that simple fusion methods, such as summation or concatenation, are ineffective with timestamps.\n\n**Q3: Explanation of the Adaptive Combiner.**\n\nFirstly, according to the ablation results on the nine datasets in Appendix B.2, the effect of the Adaptive Combiner is not \"minimal\", second only to the complete removal of the prediction backbone. Secondly, the combined weight $\\mathbf{W}$ of the global mapping $\\hat{\\mathbf{Y}}$ and local prediction $\\bar{\\mathbf{Y}}$ in the Adaptive Combiner is derived from the MLP weight generation network. We do not specifically set the initial model weights of the MLP layer. They are entirely adapted autonomously based on the MSE loss using gradient descent from the randomly initialized values. It is important to note that the combination weights $\\mathbf{W}$, which cannot be directly initialized, are not the model weights of the MLP.\n\n**Q4: Effects of timestamp granularity and noise.**\n\nFirstly, it must be noted that the timestamp information we integrate has been extensively utilized but ineffectively by various baselines. We do not introduce any supplementary information or assumptions. In other words, our approach does not impose more stringent requirements on timestamps (e.g., higher precision sampling granularity) compared to the Informer (2021) backbone. Secondly, maintaining the stability of prediction models in noisy environments is another extensive area where numerous dedicated works, such as RobustTSF [5], have provided excellent solutions. Nonetheless, to assess the robustness of our proposed method, we present the MSE results of the iTransformer backbone on two representative datasets. The results in the table below clearly demonstrate that our proposed method is robust against varying proportions of Gaussian noise, owing to components such as Robust Denormalizer.\n\n|   Dataset   | iTransformer | + Ours 0% | + Ours 10% | + Ours 20% | + Ours 30% |\n| :---------: | :----------: | :-------: | :--------: | :--------: | :--------: |\n|   Traffic   |    0.3267    |  0.2909   |   0.2932   |   0.2943   |   0.2957   |\n| Electricity |    0.1674    |  0.1434   |   0.1548   |   0.1551   |   0.1555   |\n\n**Q5: Results across varying lookback window lengths.**\n\nWe supplement the MSE results on two representative datasets with fixed prediction windows by varying the history windows. The prediction window is fixed at 192. From the experimental results **in Table 2 of the Global Response attachment PDF**, iTransformer and DLinear, which derive their final predictions from a linear layer, demonstrate a general trend of enhanced prediction accuracy with longer history windows. Furthermore, our proposed methods consistently enhance prediction performance.\n\n**Q6: Limitations of the proposed method.**\n\nWe have discussed the limitations of our study in Appendix D. The primary emphasis is on the computational cost imposed by the attention mechanism, and a potential solution has been proposed.\n\n**References**\n\n[1] 2024, Self-Supervised Contrastive Learning for Long-Term Forecasting\n\n[2] 2024, Multi-Patch Prediction: Adapting Language Models for Time Series Representation Learning\n\n[3] 2024, Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions\n\n[4] 2023, A Comparative Analysis of Early and Late Fusion for the Multimodal Two-Class Problem\n\n[5] 2024, RobustTSF: Towards Theory and Design of Ro-Bust Time Series Forecasting With Anomalies", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XnZS8LCnsX", "reviewer": "Reviewer_Y7JY", "summary": "This paper introduces the GLAFF framework where time series models are adapted to also capture \"global\" information by using information content in the datetime parsed in components of habitual meaning to complement baseline models (\"backbones\").  The global information is represented through a Mapper, quantile-based Denormalizer, and Combiner, and is combined in an MLP head with the backbone output to form the prediction.  While extracting the time units from datetimes is used in other NLP literatures, it has not been applied in time series forecasting models as far as I am aware.", "questions": "See first two limitations above.The", "limitations": "The limitation section is in the appendix and describe computational cost.  So the limitations section should be expanded upon, e.g. with respect to moving away from small benchmark datasets, irregular sampling, data drift settings, etc.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "The work appears to be original and augments leading time series models, leading to performance improvements on a suite of time series prediction benchmarks as compared to the respective baseline models without GLAFF.  The baseline models held previous SOTA performances in the last 5 years or so.  Relevant alternatives (GPT2, Table 2) and relevant ablations are done for the best performing baseline iTransformer (Table 3).  The results are presented clearly, with full tables of results in the appendix.", "weaknesses": "The intuition paragraphs describing the motivation for the components could be made clearer, in particular with respect to \"mitigating data drift\"? What assumptions are made explicitly, and what is the precise formulation?\n\nThe GLAFF framework could have be applied to a larger set of time series models, or at least applied to the best performing one for each dataset, since they are readily accessible and compared against: (e.g. https://github.com/thuml/Time-Series-Library).\n\nHyperparameter searches were done on axes rather than a grid.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your positive comments and insightful suggestions. Please find our response below.\n\n**Q1: Explanation of the motivation for the component.**\n\nWe apologize for the confusion. We will further explain the role of the denormalizer in mitigating data drift. The statistical characteristics (e.g., mean or variance) of time series data collected from the real world typically change over time as the system evolves dynamically. For instance, an increase in the popularity of a service can cause customer metrics (e.g., request counts) to rise over time. Previous studies [1, 2] have shown that models trained on the training set may underperform on the test set when the distributional difference between the training and test sets is significant. We assume that the difference between the distributions of the history and prediction windows is negligible within a small sliding window. As illustrated in Equation 3, the inverse denormalizer mitigates data drift by aligning the output of the mapper to the distribution of the local history window, thereby smoothing the difference between the training and test sets.\n\n**Q2: Additional results for more backbones.**\n\nThe principle for selecting the backbone in our study is to balance model architecture, prediction accuracy, and timestamp treatment within a constrained space. Based on the library you provided, the current top three rankings for long-term forecasting are iTransformer, TimeMixer, and TimesNet. iTransformer and TimesNet have already been discussed in our paper, and we now supplement experimental results for TimeMixer. The history window is set to 96 and the prediction window to 192 for all datasets except ILI, where the history and prediction windows are both set to 36. The MSE are presented in the table below. The results demonstrate that our proposed method enhances performance across various mainstream backbones.\n\n|   Dataset   | iTransformer | + Ours | TimeMixer | + Ours | TimesNet | + Ours |\n| :---------: | :----------: | :----: | :-------: | :----: | :------: | :----: |\n| Electricity |    0.1674    | 0.1434 |  0.1890   | 0.1576 |  0.1908  | 0.1694 |\n|    ETTh1    |    0.4944    | 0.4739 |  0.4978   | 0.4897 |  0.5331  | 0.5195 |\n|    ETTh2    |    0.1992    | 0.1957 |  0.2041   | 0.2028 |  0.2177  | 0.2040 |\n|    ETTm1    |    0.4285    | 0.4034 |  0.4429   | 0.4282 |  0.4478  | 0.4400 |\n|    ETTm2    |    0.1491    | 0.1438 |  0.1443   | 0.1393 |  0.1550  | 0.1358 |\n|  Exchange   |    0.1126    | 0.1078 |  0.1207   | 0.1174 |  0.1347  | 0.1145 |\n|     ILI     |    1.0608    | 1.0391 |  1.2741   | 1.2252 |  1.2944  | 1.2039 |\n|   Traffic   |    0.3267    | 0.2909 |  0.3405   | 0.3009 |  0.3639  | 0.3251 |\n|   Weather   |    0.2308    | 0.2138 |  0.2337   | 0.2246 |  0.2344  | 0.2331 |\n|    Avg.     |    0.3522    | 0.3346 |  0.3830   | 0.3651 |  0.3969  | 0.3717 |\n\n**References**\n\n[1] 2022, Reversible Instance Normalization for Accurate Time-Series Forecasting Against Distribution Shift\n\n[2] 2023, Adaptive Normalization for Non-stationary Time Series Forecasting: a Temporal Slice Perspective", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OoyfhUpAE3", "reviewer": "Reviewer_N7JP", "summary": "This article focuses on enhancing time series forecasting capabilities using timestamps and introduces a plug-and-play module called GLAFF. Overall, GLAFF is designed to be simple and lightweight, significantly improving the predictive performance of existing time series forecasting algorithms such as ITransformer and DLinear.", "questions": "Please refer to the weaknesses section.", "limitations": "The authors have discussed the limitations of the article in the appendix.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The article is well-structured, and the writing is clear and appropriate.\n2. The motivation is explicit, and the method is straightforward and efficient. The core code is provided in the appendix for readers' convenience.\n3. The experiments are relatively comprehensive, involving both large language models and task-specific general models.", "weaknesses": "1. The article contains some typos, such as \"damaged\" in line 32 and \"aimed\" in line 52.\n2. Formula 3 uses quantile for denormalization, but the reason for this choice is not explained. Why is quantile better than std for this purpose?\n3. Observing Figure 3, I notice that while GLAFF improves the forecasting performance, it still fails to fully capture the traffic spikes. Can you explain why this is the case and suggest any other potential solutions to address this issue?\n4. For the ablation experiment w/o Quantile, I would like to see the results of completely removing robust denormalization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your positive comments and insightful suggestions. Please find our response below.\n\n**Q1: Some typos in the paper.**\n\nWe apologize for the oversight that led to grammatical errors in the paper, causing issues with your reading. These typos will be corrected in the final version.\n\n**Q2: Reasons for using quantile.**\n\nAs demonstrated in the Electricity dataset in Figure 3, history windows may contain anomalous noise, such as spikes, due to various complex factors in the real world. The mean and variance are highly sensitive to such spiky noise and tend to misnormalize the output of the mapper to an incorrect distribution. In contrast, quantiles are more robust to such noise and align better with our initial objective of providing robust global information. This is also confirmed by the ablation experiments in Section 4.4.\n\n**Q3: Challenges in capturing traffic spikes.**\n\nWe visualize the full traffic flow **in Figure 2 of the Global Response Attachment PDF**. The test set has demonstrated drift, with peaks significantly higher in the latter half compared to the training set. Unless the history window incorporates relevant information (e.g., sharper peaks) or the model undergoes additional adaptive training, resolving this issue is challenging.\n\n**Q4: Additional results for ablation experiments.**\n\nWe supplement the ablation experiments based on the iTransformer backbone across two representative datasets. The history window is set to 96, and the prediction window to 192. The results about MSE are presented in the table below. The results underscore the necessity of denormalization and the superiority of employing quartiles for robust denormalization.\n\n|   Dataset   | iTransformer | + Ours | w/o Quantile | w/o Denormalization |\n| :---------: | :----------: | :----: | :----------: | :-----------------: |\n|   Traffic   |    0.3267    | 0.2909 |    0.2948    |       0.3233        |\n| Electricity |    0.1674    | 0.1434 |    0.1677    |       0.1742        |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We provide some images and tables **in the PDF attachment of the global response**, accompanied by detailed descriptions in the individual responses for each reviewer.

Meta-review: This well-written paper has been assessed  by five knowledgeable reviewers who predominantly voted to have it accepted for NeurIPS (two full accept ratings, two weak acceptances, and one borderline rejection). The authors provided a comprehensive rebuttal to the initial criticism, and engaged in conversations with most of the reviewers. Those reviewers who communicated with the authors have their main concerns predominantly addressed. In summary, this work is viable for presentation at NeurIPS in spite of a more critical view of it by one of the five reviewers.
Decision: Accept (poster)