CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns
Authors: Shengsheng Lin
Weiwei Lin
Xinyi HU
Wentai Wu
Ruichao Mo
Haocheng Zhong
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Time Series Forecasting
Time Series Analysis
Machine Learning
Primary area: machine_learning_for_other_sciences_and_fields
TL;DR: This paper pioneers the exploration of explicitly modeling periodic patterns in time-series data to enhance the accuracy of long-term time series forecasting (LTSF) tasks.

Abstract: The stable periodic patterns present in time series data serve as the foundation for conducting long-horizon forecasts. In this paper, we pioneer the exploration of explicitly modeling this periodicity to enhance the performance of models in long-term time series forecasting (LTSF) tasks. Specifically, we introduce the Residual Cycle Forecasting (RCF) technique, which utilizes learnable recurrent cycles to model the inherent periodic patterns within sequences, and then performs predictions on the residual components of the modeled cycles. Combining RCF with a Linear layer or a shallow MLP forms the simple yet powerful method proposed in this paper, called CycleNet. CycleNet achieves state-of-the-art prediction accuracy in multiple domains including electricity, weather, and energy, while offering significant efficiency advantages by reducing over 90% of the required parameter quantity. Furthermore, as a novel plug-and-play technique, the RCF can also significantly improve the prediction accuracy of existing models, including PatchTST and iTransformer. The source code is available at: https://github.com/ACAT-SCUT/CycleNet.

CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns
Shengsheng Lin', Weiwei Lin! Xinyi Hu², Wentai Wu', Ruichao Mo', Haocheng Zhong' Schooo of Computer Science and Engineering, South China University of Technology, China "Pengcheng Laboratory, China
Department of Computer Science and Engineering, The Chinese University of Hong Kong *College of Information Science and Technology. Jinan University, China scut. edu. cn, linsw@scut. edc.. cn, xyhu@cse cuhk. edu. hk, ventaiwuQjnu. edu. cn, {es_moruichao, [shezhong]@maie scut, edu. edu.
csl inshengshenn@mmall scutt.
Abstract
The stable periodic patterns present in time series data serve as the foundation for conducting long-horizon forecasts. In this paper, pioneer the exploration explicitly modeling this periodicity to enhance the performance of models in long-temn time series forecasting (LTSF) tasks. Specifically, the Residual Cycle Forecasting (RCF) technique, which utilizes learnable recur- rent cycles to model the inherent periodic patterns within sequences, and then performs predictions on the residual components of the modeled cycles. Com- bining RCF with Linear layer shallow MLP forms the simple yet powerful method proposed in this paper, called CycleNet. CycleNet achieves state-of-the- art prediction accuracy in multiple domains including electricity, weather, and energy, while offering significant efficiency advantages by reducing over 90% the required parameter quantity. Furthermore, as novel plug-and-play tech- nique, the RCF can also significantly improve the prediction accuracy of existing models, including PatchTST and iTTrasformer. The source code is available http:///thhlbbccommAACACCCCCCCC
introduce
1
Introduction
Time series forecasting (TSF) plays a crucial role in various domains such as weather forecasting, transportation, and energy management, providing insights for early warnings and facilitating proac- tive planning. Particularly, accurate predictions over long horizons (e.g.. spanning several days months) offer increased convenience, referred to Long-term Time Series Forecasting (LTSF) However, the principle enabling long-horizon prediction lies understanding the inherent periodicity within the data [32]. Unlike short-term forecasting, long-term predictions cannot rely solely on recent temporal information (including means, trends, etc.). For instance, a user's electricity consumption thirty days ahead not only correlates with their consumption patterns
59
(66]
17
42 6
in the past few days.
In such cases, long-term dependencies, or in other words, underlying stable periodicity within the data, serve as the practical foundation for conducting long-term predictions [32] This why existing models emphasize their capability to extract features with long-term dependencies. Models like Informer [59], Autoformer [51] and PatchTST (40] utilize the Transformer's ability for long-distance modeling to address LTSF tasks. ModernTCN 38 employs large convolutional kemelss to enhance
TCNs' ability to capture long-range dependencies, and SegRNN [3] uses segment-wise iterations to
"Corresponiing author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

improve RNN methods' handling of long sequences. If a model can accurately capture long-range dependencies. it can precisely extract periodic patterns from historical long sequences, enabling more accurate long-horizon predictions.
However, if the purpose of constructing deep and complex models is solely hetter extract periodic features from long-range dependencies, why not directly model the patterns? As illustrated in Figure [ electricity data exhibits clear daily periodic patterns (in addition to possible weekly patterns). We can use a globally shared daily segment to represent the periodic pattern in electricity consumption, By repeating this daily segment N times, we can continuously represent the cyclic components of N days' electricity consumption sequences.
Based on the above motivation, pioneer plicit modeling of periodic patterns in the data to enhance the model's performance LTSF tasks in this paper. Specifically, we propose the Residual Cycle Forecasting (RCF) technique. It involves using learnable recurrent cycles to explicitly model the inherent periodic patterns within time series data, followed by predicting the residual components of the modeled cycles. Combining the RCF technique with either single-layer Linear or a dual-layer MLP results in CycleNet, simple yet powerful method. Cy- cleNet achieves consistent mate-offthe-aa per- formance across multiple domains and offers significant efficiency advantages.
Shared Periodic Pattern
time
Figure 1: Shared daily periodic patterns present in the Electricity dataset.
In summary, this paper contributes:
We identify the presence of shared periodic patterns in Long-horizon forecasting domains and propose explicit modeling of these patterns to enhance the model's 's performance on LTSF tasks.
Technically, we introduce the RCF technique, which utilizes learnable recurrent cycles to explicitly model the inherent periodic patterns within time series data, followed by predicting the residual components the modeled cycles. The RCF technique significantly enhances the performance of basic (or existing) models.
Applying RCF with Linear layer or shallow MIP forms the proposed simple yet powerful method, called CycleNet. CycleNet achieves consistent state-of-the-a performance across multiple domains and offers significant efficiency advantages.
2
Related work
In fact, utilizing periodic information to enhance model prediction accuracy is not novel concept. Numerous studies, in particular, have introduced series of Geasonal-Trente Decomposition (STD) techniques that allow models to better leverage periodic information. Popular models such as Autoformer [5T]. FEDformer [60), and DLinear [56] utilize the classical STD approach to decompose the original time series into two equally sized subsequences: seasonal and trend components, which are then modeled independently. These classical STD methods typically use a basic moving average (MOV) kernel to perform sliding aggregation to obtain the trend component. Recently, Leddam (55) proposed replacing the traditional MOV kernel in STD with a Learnable Decomposition (LD) kernel, leading to improved performance. Additionally, DEPTS ( treats the periodicity of sequences parameterized function with respect to time, and learns periodic and residual components layer-wise through its periodic and local blocks. SparseTSF [32] another recent work, utilizes cross-periee pprsio sparse forecasting technique to decouple cycles and trends, achieving impressive performance at extremely low cost.
The RCF technique proposed in this paper can essentially be considered a type of STD method. The key difference from existing techniques lies in its explicit modeling of global periodic patterns within independent sequences using learnable recurrent cycles. The proposed RCF technique conceptually simple, computationally efficient, and yields significant improvements in prediction accuracy. The

---

further proposed CycleNet, which combines the RCF technique with simple backbone, is a Linear- or MLP-based model that simple, efficient, and powerful for time series forecasting. To correctly position CycleNet, we have provided detailed review of the development of different categories of time series forecasting methods (including Transformer-basee RNN-based, etc.) Appendix A
3 CycleNet
Given a time series X with D variables or channels, the objective of time series forecasting is to predict future horizons H steps ahead based on past observations, mathematically represented as II-L+1: R #2+1++ IR #x In fact, the inherent periodicity within time series fundamental for accurate prediction, particularly when forecasting over large horizons, such as 96-720 steps (corresponding to several days months). To enhance the model's performance long-term prediction tasks, we propose the Residual Cycle Forecasting (RCF) technique. It combines Linear layer shallow MLP to form simple yet powerful method CycleNet, illustrated in Figure 2 with detailed pseudocode provided in Appendix B.1
Leamable Recurrent Cycles Bxw
Align and Repeat
M
Align Repeat
Remove Cycle
Restore Cycle
mUI
MAU
MMMM
Linear or MLP
MUM
WW
M
WW
M
Ne-141o
ROKL
X E-tase E RRNL
Reas BRX
Figure 2: CycleNet architecture. CycleNet/Linear and CycleNet/MLP represent using single-layer Linear model and dual-layer MLP model, respectively, as the backbone of CycleNet. Here, = 3.
3.1 Residual cycle forecasting
The RCF technique comprises two steps: the first step involves modeling the periodic patterns of sequences through learnable recurrent cycles within independent channels, and the second step entails predicting the residual components of the modeled cycles.
Periodic patterns modeling Given D channels with priori cycle length W, we first generate learnable recurrent cycles Q E RRW xD initialized to zeros. These recurrent cycles are globally shared within channels, meaning that by performing cyclic replications, we can obtain cyclic com- ponents C' of the sequence X of the same length. These recurrent cycles Q of length W undergo gradient backpropagation training along with the backbone module for prediction, yielding learned representations (distinct from the originally initialized zeros) that unveil the internal cyclic patterns within the sequence.
Here, the cycle length W depends on the priori characteristics of the dataset and should be set the maximum stable cycle within the dataset. Considering that scenes requiring long-term predictions usually exhibit prominent, explicit eycles (e.g., electrical consumption and traffic data exhibit clear daily and weekly cycles), determining the specific cycle length is available and straightforward. Ad- ditionally, the dataset's cycles can be further examined through autocorrelation functions (ACF) (39] revealed in Appendix B.2

---

Residual forecasting Predictions made on the residual components of the modeled cycles, termed residual forecasting, are follows:
1. Remove the cyclic components Ct-++1: from the original input 1+=1+ to obtain residual components 14-1+1:--
2. Pass f1-1+1: through the backhone to obtain predictions for the residual components, F441441-
3. Add the predicted residual components o #1114 to the cyclic components C++1++4 to obtain 8--1+HH-
It is important to note that, since the cyclic components C' are virtual sequences derived from the cyclic replications of Q. we cannot directly obtain the aforementioned sub-sequences ct I:t and C++1-++H- Therefore, as illustrated in Figure appropriate alignments and repetitions of the recurrent cycles Q are needed to obtain equivalent sub-sequences: (i) Left-shift Q by mod W positions to obtain Q(+,, Here, mod W be viewed as the relative positional index of the current sequence sample within Q. (ii) Repeat Q(+) [L/W] times and concatenate Q0:L mod W Mathematically, these two equivalent subsequences be represented as:
G4112411 B
Align
gl (holl
6 E B
Q Boww
@ ( T
and
(1 13 moll
|/W1 x W
H//W
mod
Figure 3: Alignments and repetitions of the recurrent cycles Q. Here, D = 1.
Ct-++1: 'Q(0 (Q."",..,Q""""" ,((() Q
mod wl:
(I)
[L/W]
C411+H = [Q(++)) (L+L) (++L)
mod wl-
(2)
[H/W]
Backbone The original prediction task is transformed into cyclic residual component modeling, which serve normal sequence modeling. Therefore, any existing time series forecast model can be employed as backbone. In this paper. our aim is to propose and examine a method for enhancing time series prediction by explicitly modeling cycles (i.c., RCF). Thus, we opt for the most basic backbone, namely a single-layer Linear and dual-layer MLP, forming our simple yet powerful methods, CycleNet//Line and CycleNet/MLPP Herein, each channel utilizes the same backbone with parameter sharing for modeling, which is also referred to as the Channel Independent strategy [13]
3.2 Instance normalization
The statistical properties of time series data, such the mean, often vary over time, which referred to as distributional shifts. This can lead to poor performance of models trained on historical training sets when applied future data. To address this issue, recent research has introduced Instance Normalization strategies like RevIN [5] 22 26]. Mainstream approaches such as iTrrasformer [37], PatchTST 40 and Sparse (32 have widely adopted similar techniques to enhance performance. To improve the robustness of CycleNet, we also incorporate similar optional strategy (see the full ablation study in Appendix C.4). Specifically, we remove the varying statistical properties from the model's 's internal representations outside of CycleNet's input and output steps:

---

21-++1 P
Lt-1+14
(3)
Va+c
T+114H = F++1++H X Vo+
where i and a represent the mean and standard deviation of the imput window, respectively, and is small constant for numerical stability. This method aligns with the RevIN version that excludes learnable affine parameters 22
3.3
Loss function
To remain consistent with current mainstream methods, CycleNet defaults to using Mean Squared Error (MSE) as the loss function to ensure fair comparison with other methods, formulated as:
Loss = |04+144 12+1.
(5)
4 Experiments
4.1 Setup
We utilized widely adopted benchmark datasets including the ETT series [59]. Weather, Traffic, Electricity, and Solar-Energy (24). Preprocessing operations on the datasets, such as dataset splitting and normalization methods, remained consistent with prior works (e.g., Autoformer [5]], iTransformer (37], etc.).
Datasets
Note that these datasets all exhibit stable cyclic patterns, such as daily and weekly, which form the realistic basis for performing long-horizon forecasting. Combined with the sampling frequency of the datasets, we can infer the maximum cycle length of the datasets, such as 24 for ETTh and 168 for Electricity. These manually inferred cycle lengths can be further confirmed through the ACF analysis, details of which provided in Appendix B.2. The hyperparameter W of CycleNet is set by default to match the cycle length in Table 1
The information of the datasets is shown in Table [
Table Dataset Information.
Dataset
ETTh ETTh2
ETTml & ETTm2
Electricity
Solar-Energy
Traftic
Weather
Timesteps
17,420
69.680
26,304
$2,560
17,544
$2,696
Channels
321
137
862
21
Proquency
hour
hour
mins
Cyelic Patterss
Daily
Daily
Daily & Woekly
Daily
Daily & Weekly
Daily
Cycle Length
Baselines We compared CycleNet against state-of-the-an models in recent years, including iTrans- former (37]. PStchTST (40), Crossformer [58). TiDE (5). TimesNet (52). DI DLinnarr [56). SCINet (34), FEDformer [60), Autoformer (51). To comprehensively evaluate CycleNet's performance, the Mean Squared Error (MSE) and Mean Absolute Error (MAE) metrics were employed.
Environments the Adam
All experiments in this paper were implemented using Py PoTch 41]. trained using optimizer, and executed on single NVIDIA GeForce RT' 4090 GPU with 24 GB
2
memory.
4.2 Main results
shows the comparison results of CycleNet with other models on multivariate LTSF tasks. OveralT, CycleNet achieves state-of-the-art performance (except for the Traffic dataset), with Cy- cleNet/MLP ranking first overall, and CycleNet/Linaar ranking second overall. Due to the nonlinear
Table
2

---

mapping capability of MLP compared to Linear, CycleNet/MLPP performs better on high-dimensional datasets such as Electricity and Solar-Enery (i.e., datasets with more than 100 channels). In summary. with the support of the RCF technique, even very simple and basic model (i.e., Linear and MIP) can achieve the current best performance, surpassing other deep models. This fully demonstrates the advantages of the RCF technique.
Table 2: Multivariate long-term time series forecasting results. The look-back length L is fixed
as 96 and the results are averaged from all prediction horizons of H E {96, 192, 336, 720}. Full results and more comparison results on longer look-back lengths are available in Appendix C.2 The results of other models are sourced from iTTranffrmmer (37 and TimeMixer 48 highlighted in bold and the second best are underlined.
The best results
Datsset
ETThi
ETTh2
ETTml
ETTm2
Fleemiiity
Solar-Faergy
Traftic
Weather
Merric
MSF
MAF
MSF
MAF
MSE
MAF
MSF
MAF
MSR
MAF
MSF
MAF
MSF
MAR
MSF:
MAE
Autoformer
2021
0.496
0.487
01450)
01459
01.58
0.517
00.327
0.371
0.227
0.338
0.885
0.711
0.678
0.379
0.338
0.382
PEDfarmer
0.440
0.460
0.437
0.449
0.448
0.452
0.305
0.349
0.214
0.327
0.291
0.381
0.610
0.376
0.309
0.360
SCINet
0.571
0.537
0.363
DLincar
2024
0.456
0.452
0.559
0.515
0.403
0.407
0.350
0.401
0.212
0.300
0.330
0.401
0.625
0.383
0.265
0.317
TemesNer
A024
0.458
0.450
0.414
0.427
0.400
0.406
0.291
0.333
0.192
0.295
0.301
0.319
0.620
0.336
0.259
0.287
TIDE
0.541
0.507
0.611
0.550
0.419
0.358
0.404
0.251
0.344
0.347
0.417
0.760
0.473
0.271
0.320
Cressforntrr
21/33
0.304
0.259
0.315
PacchTT
20x3
0.669
0.154
0.387
0.107
0.387
0.100
0.281
0.326
0.205
0.290
0.270
0.307
0.481
0.301
0.259
0.281
TimeMisur
21/4
0.447
0.421
0.364
0.395
0.388
0.395
0.275
0.323
0.182
0.272
0.216
0.250
0.484
0.297
0.240
0.271
(Trnsformer
0.454
0.447
01383
0.407
0.407
0.410
0.288
0.332
0.178
0.270
0.235
0.762
0.428
0.282
0.258
0.278
CycleNest
0.485
0.313
0.254
0.279
CycleNerMM.P
0.457
0.266
0.168
0.210
0.261
0.472
0.301
0.243
0.271
Furthermore, we can observe that CycleNet's performance on the Traffic dataset is inferior to iTTrans- former, which models multivariate relationships in time series data using an inverted Transformer. This is because the Traffic dataset cxhibits spatiotemporal characteristics and temporal lag char- acteristics, where the traffic flow at certain detection point significantly affects the future values of neighboring detection points. In such cases, modeling sufficient inter-channel relationships is necessary, and iTTranffommer accomplishes this. In contrast, CycleNet independently models the temporal dependencies of each channel, hence it suffers a disadvantage in this scenario. However, CycleNet still significantly outperforms other baselines on the Traffic dataset, demonstrating the competitiveness of CycleNet. Additionally, we have included more analysis of CycleNet in traffic scenarios in Appendix including a full comparison of results on the PEMS datasets.
4.3 Efficiency analysis
The proposed RCF technique, as a plug-and- play module, requires minimal overhead, need- ing only additional W x D learnable param- eters and no additional Multiply-Acumuultt Operations (MACs). The backbones of Cy- cleNet, namely single-layer Linear and dual- layer MLP, are also significantly lightweight compared to other multi-layer stacked models. the efficiency comparison between CycleNet and other mainstream models, where CycleNet shows significant advantages. Particularly, compared to fTransformer, which also possesses strong capabilities modeling long-term dependencies and nonlinear learning, CycleNe/MMPP has over ten times fewer param- eters and MACs. As for CycleNet/Linene which shares the same single-layer linear backbone as DLinear, it also has fewer parameters and MACs. However, in terms of training speed, DLinear is still faster than CycleNet/Liness This is because the RCF technique requires aligning the recurrent cycles with each data sample, which incurs additional CPU time. Overall, considering the significant improvement in prediction accuracy brought by the RCF technique, CycleNet achieves the best balance between performance and efficiency.
Table 3: Efficiency comparison between CycleNet and other models on the Electricity dataset with look-back length
96 and forecast horizon H = 720. Training Time denotes the average time required per epoch for the model.
Model
Parameters
MACS
Training Time(s)
Table J
Informer 200
12.53M
3.97G
70.1
Autoformer
2021
12.22M
4.41G
107.7
FEDformer
2022
17.98M
4.41G
238.7
DLineur
2753
139.6K
44.9MM
18.1
PatchTST
2015
10.74M
25.87G
129.5
Transformer
2024
5.15M
1.65G
35.1
CycleNet/MLP
472.9K
134.84M
30.8
CycleNet/LLinuur
123.7K
22.42M
29.6
RCF part
53.9K
0
12.8

---

4.4 Ablation study and analysis
Effectiveness of RCF To investigate the effectiveness of RCF, we conducted comprehensive ablation experiments on two datasets with significant periodicity: Electricity and Traffic. The results are shown Table 4
Table 4: Ablation study of RCF technique. The Linear and MLP backbones apply the same instance normalization strategy as CycleNet by default to fully demonstrate the effect of RCF technique.
Donasct
Eleatioay
Traffic
Horizon
96
192
96
336
Metric
MSB
MAB
MSE
MAB
MSE
MAB
MSE
MAB
MSE
MAB
MSE
MAB
MSE
MAB
MSE
MAB
Lincer
0.197
0.274
0.197
0.277
0.212
0.292
11.533
0.524
0.645
01333
0.598
0.361
0.605
0.362
0.643
0.381
RCF
0.141
0:234
0.155
0.347
0.172
0.364
0.210
0.206
0.430
0.314
0.482
0.313
0.476
0. 0.503
0.503
0.320
Improse
28.6%
14.6%
21.4%
10.8%
18.8%
9,5%
17.1%
8.7%
25.6%
18.0%
19.5%
13.2%
21.3%
16.2%
21.8%
16.1%
MLP
0.175
0.259
0.181
0.265
0.197
0.282
0.240
0.517
0,500
0.325
0
0.321
0.509
0.325
0.542
0.342
RCF
0.136
0:229
0.152
0.344
0.370
0.264
10.112
0.209
11458
1.259
0.457
1.254
0.470
0.299
01502
013114
Improve
22.2%
11.6%
15.9%
8.0%
13.6%
6.3%
11.6%
5.7%
8.5%
8.9%
7.9%
8.3%
7.7%
8.0%
7,3%
8.1%
DLinear
0.195
0.278
0.194
0.281
0.207
0.297
0.243
0.331
0.549
0.398
0.599
0.372
0.806
0.375
0.516
0.396
RCY
0.143
01240
0.356
0.253
0.337
00,899
0.317
0.325
0.343
Improse
13.3%
PascITST
0.260
0.266
0.285
0.310
RCF
0.136
0.153
0.246
0.370
0.264
0.211
0.299
0,438
0.264
0.457
0.270
0,469
0.275
0.509
0.292
Improve
19.0%
11.0%
13.0%
7.6%
11.7%
6.6%
9,4%
5.7%%
-0.5%
6.1%
-1.85
5.5%
6.3%
-2.0%
6.1%
Transfomncr
0.148
0.240
0.162
0.253
0.178
0.369
0.317
0.395
0.266
0.417
0.276
0.433
0.283
0.467
0.302
RCT
0.136
0.231
0.153
0.247
0.168
0.263
0.194
0.287
0.415
0,263
0.540
0.271
0.456
0.278
0.491
0.294
Impcove
8.1%
3.7%
5.6%
2.4%
5,8%
2.2%
13.8%
9.5%
-5.156
1.9%
-5.55
1.8%
-5.3%8
1.8%
-5.358
2.6%
Firstly, when combining the basic Linear and MLP backbones (both utilizing instance normalization by default) with the RCF technique, a significant improvement in prediction accuracy (approximately 10% 20%) is observed. This demonstrates that the success of CycleNet is largely attributed to the RCF technique rather than the backbones themselves or the instance normalization strategy. Overall, the performance of MLP is stronger than that of Linear, regardless of whether the RCF technique is applied. This indicates that non-linear mapping capability is necessary when modeling high-dimensional datasets with the Channel-independerr strategy (sharing parameters across each channel), aligning with previous research findings (26).
Secondly, we further verified whether RCF can enhance the prediction accuracy of existing models, as RCF is essentially a plug-and-plas fiexible technique. It is observed that incorporating RCF still improves the performance of existing complex designed, deep stacked models (approximately 5% to 10%), such as PStchTST (40) and i'Trasformee (37]. Even for DLinear, which already employs the classical MOV-based STD technique, RCF was able to provide improvement of approximately 20%. This further indicates the effectiveness and portability of RCF
However, an interesting phenomenon was observed: although the MAE decreases when PatchTST and iTransformer are combined with RCF, the MSE increases. The most important reason behind this is that there are extreme points in the Traffic dataset that could affect the effectiveness of RCF, which fundamentally relies on learning the historical average cycles in the dataset. We further analyze this phenomenon in detail in Appendix C.5 and suggest potential directions for improving the RCF technique.
Comparison of different STD techniques The proposed RCF technique is essentially a more powerful STD approach. Unlike existing methods that decompose the periodic (seasonal) component from a limited look-back window, RCF learns the global periodic component from the training set. Here, we compare the effectiveness of RCF with existing STD techniques, using pure Linear model as the backbone (without applying any instance normalization strategies). The comparison includes LD from Leddam (55), MOV from DLincar [56), and Sparse technique from Sparse [32). As shown in Table 5 RCF significantly outperforms other STD methods, particularly on datasets with strong periodicity, such as Electricity and Solar-Energy. In contrast, the other STD methods did not show significiniiii significant advantages over the pure Linear model.
There are several reasons for this. First, MOV and LD-based STD methods achieve trend estimation by sliding aggregation within the look-back window, which suffers from inherent issues 27 26 (i The sliding window of the moving average needs to be larger than the maximum period of the seasonal component; otherwise, the decomposition may be incomplete (especially when the period length exceeds the look-back sequence length, making decomposition potentially impossible).
7

---

(ii) Lero-padding is required at the edges of the sequence samples to obtain equally sized moving average sequences, leading to distortion of the sequence edges, As for the Sparse technique, being a lightweight decomposition method, it relys more on longer look-back windows and instance normalization strategies to ensure adequate performance.
Additionally, these meth- ods that decouple trend and seasonality within the look-back window are es- sentially equivalent to un- constrained or weakly con- strained linear regression 44 which means that after full training convergence, linear-based models com- bined with these methods are theoretically equivalent to pure linear models. In contrast, the periodic com- ponents obtained by the RCF technique are globally estimated from the training set, allowing it to surpass the limitations of finite-
Table 5: Comparison of different STD techniques. To directly compare the effects of STD, the configuration used here is consistent with that of DLinear [56]. with sufficient look-back window length of 336 and no additional instance normalization strategies. Thus, CLinear here refers to CycleNet/Lingg Lener without RevIN. The reported results are averaged across all prediction horizons of H E {96, 192, 336, 720), with full results available in Appendix C.3
CI Linear
L.D.inear (LD+Linear)
DL innar -Liner)
SI. SLinear (SpureeLiinaar
Setup
(RCF+Lineer)
Lincar
Metric
MSE
MAE
MSE
MAE
MSC
MAE
MSE
MAE
MSE
MAE
ETTal
0.418
0.434
0.427
0.439
0.425
0.437
0.424
0.436
0.427
0.439
ETTb2
0.451
0.456
0.455
0.457
0.471
0.467
0.460
0.460
0.460
0.462
ETTml
0.349
0.382
0.365
0.387
0.367
0.390
0.362
0.383
0.362
0.384
ETTm2
0.266
0,330
0.273
0.336
0.280
0.341
0.290
0.352
0.269
0.331
Electricity
0.157
0.255
0.264
0.264
0.268
0.167
0.265
Solar- Eieer-Eneegyy
0.220
0.259
0.253
0.316
0.254
0.318
0.255
0.315
0.253
0,318
Traflic
0.423
0.289
0.434
0.296
0.434
0.296
0.435
0.292
0.434
0.296
0.300
0.244
0.297
0.244
0.246
0.298
0.297
length look-back window, and thus, its capabilities extend beyond standard linear regression.
Impact of hyperparame- ter W The hyperparame- ter W determines the length of the learnable recurrent cycles Q in the RCF tech- nique. In principle, must match the maximum pri-
Table 6: Performance of the CycleNet/Linear model with varied W. The forecast horizon is set as 96.
Sctup
RCF/W=168
RCE/W=144
RCF/W=96
RCF/W=24
W/o, RCF
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Electricity
0.142
0.234
0.196
0.275
0.196
0.274
0.195
0.274
0.197
0.274
mary cycle length in the data to correctly model the periodic patterns of the se- quence. We investigate the performance of the CycleNet/L T When correctly setting the hyperparameter W to the max cycle length of the dataset (i.e., the cycle length pre-inferred in Table []] RCF can play a significant role, yielding a large performance gap compared to the cases when it is not correctly set. This indicates the necessity of inferring and setting the correct W for RCF to function properly. Furthermore, when W is incorrectly set, the model's performance is almost the same as when RCF is not used at all. This suggests that even in the worst-case scenario, RCF does not bring significant negative effects.
Trafic
0.480
0.314
0.617
0.386
0.617
0.385
0.618
0.385
0.645
0.383
Solar Energy
0.289
0.376
0.208
0.256
0.276
0.365
0.287
0.375
0.286
0.375
ETTml
0.350
0.369
0.340
0.366
0.325
0,363
0,348
0.367
0.351
0.372
ETThl 0.395
0.402
0.384
0.395
0.383
0.393
0.377
0.391
0.384
0.392
model under different settings of W for different datasets in
Visualization of the learned periodic patterns The purpose of the RCF technique to utilize the learnable recurrent cycles Q (initialized to zero) to model the periodic patterns in time series data. After co-training with the backbone, the recurrent cycles can represent the inherent periodic patterns of the sequence. Figure 4 illustrates the different periodic patterns learned from different datasets and channels. For example, Fi shows the daily operating pattern of solar photovoltaic generation, while Figure 4 d) displays the weekly operating pattern of traffic flow, featuring peak traffic in the mornings on weekdays. These periodic patterns learned from the global sequence provide important supplementary information to the prediction model, especially when the length of the look-back window is limited and may not provide sufficient cyclic information when the cycle length is long.
Furthermore, although the cycle length is the same for different channels within the same dataset, the specific periodic patterns differ, as shown in Feghr Particularly, Figure demonstrates the intermittent periodicity of household electricity consumption on weekdays, while others exhibit relatively uniform weekday patterns in their respective channels. This highlights the necessity of separately modeling the periodic patterns for each channel.
8

---

ET 7th
(b) Weather, 7th
(c) Solar- Solar-Energy, 137th
(d) Traffic, 607th
M
(e) Electricity, 311st
(1) Electricity, 318th
(g) Electricity, 320th
(h) Electricity. 321st
Figure 4: Visualization the periodic patterns learned by CycleNet/Linear, Panels (a-d) display different periodic patterns learned from different datasets, and panels (e-h) show different periodic patterns learned from different channels within the same dataset. The th indicates the index of the channel within the dataset.
In conclusion, these findings demonstrate that the RCF technique effectively learn the inherent periodic patterns in time series data, serving as a crucial explanatory factor contributing to the state- of-the-art performance of CycleNet. Additionally, we have included further analysis in Appendix C.T showcasing the learned periodic patterns of RCF under different configurations to better illustrate how RCF operates.
0.26
DLinear
0.9
DLinear
PatchTST
0.8
PatchTST
Transformer
0.22
LycleNettLinear
0.7
CycleNet/Linear
CycleNet/MLP
CycleNet/MLP
0.20
0.18
0.16
0.5
0.14
0.4
0.12
48
96
192
336
528
720
48 96
192
336
528
720
Lookback Length
Lookback Length
(a) Electricity
(b) Traffic
Figure 5: Performance of CycleNet and comparative models with different look-back lengths. The forecast horizon is set as 96.
Performance with varied look-back length The look-back length determines the richness of historical information that can be utilized. Theoretically, the larger it is, the better the model performance should be, especially for models capable of capturing long-term dependencies. Figure shows the performance of different models under different look-back lengths. It can be observed that CycleNet, as well as representatives of current state-of-the-art models such as iTransformer (37], PStchTST 400 and DLinear [56), all achieve better performance with longer look-back lengths. This indicates that these models all possess strong capabilities in modeling long-term dependencies.
It is worth highlighting that (i) on the Electricity dataset, CycleNet outperforms current statc-of-the- art models any prediction length; (ii) on the Traffic dataset, CycleNet still falls short compared to powerful existing multivariate forecasting models, such as iTransformer. This indicates that in scenarios with strong periodicity without additional spatiotemporal relationships, fully leveraging the periodic components is sufficient to achieve high-accuracy predictions. However, in more complex scenarios that require thorough modeling of relationships between variables, a simple channel-independen strategy combined with basic backbone, like CycleNet, still struggles to fully meet the demands. Therefore, Appendix C.5 we further analyze the current limitations of the current RCF technique in spatiotemporal scenarios (such the traffic domain) and and point out

---

potential directions for future improvements. Finally, we also provide a comparison of CycleNet with existing models on full datasets using longer look-back windows in Appendix C.2
5 Discussion
CycleNet demonstrates its efficacy in LTSF scenarios characterized by prominent and explicit periodic patterns. However, there several potential limitations of CycleNet that warrant discussion here:
Potential limitations
Unstable cycle length: CycleNet may not be suitable for datasets where the cycle length (or frequeney) varies over time, such as electrocardiogram (ECG) data, because CycleNet can only learn a fixed-length cycle.
Varying cycle lengths across chamnels: When different channels within a dataset exhibit cycles of varying lengths, CycleNet may encounter challenges because it defaults to mod- eling all channels with the same cycle length W. Given CycleNet's 's fraanee-independeee modeling strategy, one potential solution is to pre-process the dataset by splitting it based on cycle lengths or to independently model each channel as a separate dataset.
Impact of outliers: If the dataset contains significant outliers, CycleNet's performance may be affected. This is because the fundamental working principle of RCF is to learn the historical average cycles in the dataset. When significant outliers exist, the mean of certain point in the cycle learned by RCF can be exaggerated, leading to inaccurate estimation of both the periodic and residual components, which subsequently impacts the prediction process.
Long-range cycle modeling: The RCF technique is effective for modeling miidrrange stable cycles (e.g., daily or weekly). However, considering longer dependencies (such as yearly cycles) presents more challenging task for the RCF technique. Although, in theory, CycleNet's W can be set to yearly cycle length to model annual cycles, the biggest difficulty lies in collecting sufficiently long historical data to train a complete yearly cycle, which might require decades of data. In this case, future research needs to develop more advanced techniques to specifically address long-range cycle modeling.
Future work: further modeling inter-channel relationships The RCF technique enhances the model's 's ability to model the periodicity of time series data but does not explicitly consider the relationships between multiple variables. In some spatio-temporal scenarios where spatial and temporal dependencies between variables exist, these relationships are crucial. For example, recent studies such as iTransformer (37 and SOFTS 112 indicate that appropriately modeling inter-channel relationships can improve performance
traffic scenarios. However, directly applying the RCF technique to iTransformer does not lead to significant improvement (at least for the MSE metric), as demonstrated Table We believe that devising a more reasonable multivariate modeling approach that combines CycleNet could be promising and valuable, and we leave it for future exploration.
6 Conclusion
This paper reveals the presence of inherent periodic patterns in time series data and pioneers the exploration of explicitly modeling this periodicity to enhance the performance of time series fore- casting models. Technically, we propose the Residual Cycle Forecasting (RCF) technique, which models the shared periodic patterns in sequences through recurrent cycles and predicts the residual cyclic components via a backbone. Furthermore, we introduce the simple yet powerful LTSF meth- ods CycleNet/Linnea and CycleNet/MLP. which combine single-layer Linear and dual-layer MLP respectively with the RCF technique. Extensive experiments demonstrate the effectiveness of the RCF technique, and CycleNet as a novel and simple method achieves sfate-of-the- art results with significant efficiency advantages. The findings this paper underscore the importance of periodicity as a key characteristic for accurate time series prediction, which should be given greater emphasis in the modeling process. Finally, integrating CycleNet with effective inter-channel relationship modeling methods serves promising and valuable future research direction.
10

---

Acknowledgments
This work is supported by Guangdong Major Project of Basic and Applied Basic Research (20198033300002) National Natural Science Foundation of China (62072187), Guangzhou Devel- opment Zone Science and Technology Project (2023GH02) and the Major Key Project PCL, China under Grant PCL2023A09.
References
[1] Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprini arXiv: 1803.0127,, 2018.
[2] Shane Bergsma, Tim Zeyl, and Lei Guo. Sutranets: Sub-series autoregressive networks for long-sequence, probabilistic forecasting, Advances in Neural Information Processing Systems, 36:30518-30533 2023.
[3] Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. Tempo: Prompt-based generative pre-trained transformer for time series forecasting. arXiv preprint arXiv:2310.04944 2023.
[4] Cristian Challu, Kin G Olivares, Boris N Oreshkin, Federico Garza Ramirez, Max Mergenthaler Canseco, and Artur Dubrawski. Nhits: Neural hierarchical interpolation for time series fore- casting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 1999-6997, 2023.
[5] Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan Mathur, Rajat Sen, and Rose Yu. Long- term forecasting with tide: Time-series dense encoder. arXiv preprint arXiv:2304.0842. 2023.
[6] Jinliang Deng. Feiyang Ye, Du Yin, Xuan Song, Ivor Wai-Hung Tsang, and Hui Xiong. Parsi- mony or capability? decomposition delivers both in long-term time series forecasting. 2024.
http::ttp:ssatt wewanticschollat........ org/ CorpusID 267068391
[7] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprins arXiv:2010.119299 2020.
[8] Wei Fan, Shun Zheng, Xiaohan Yi, Wei Cao, Yanjie Fu, Jiang Bian, and Tie- Yan Liu. Depts: Deep expansion learning for periodic time series forecasting. arXiv preprint arXiv:2203.00 076611, 2022.
[9] Jean-Yve Franceschi, Aymeric Dieuleveut, and Martin Jaggi. Unsupervised scalable repre- sentation learning for multivariate time series. In Wallach, Larochelle, A. Beygelzimer, F. d' d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Sys- tems, volume 32, Curran Associates, Inc., 2019. URL http:ttss//prccceiigggeeeee neuripps...e........ cc/ pdf
paper_ http://pppppppooetooo/oorreaaaaa
[10] Zeying Gong, Yujin Tang, and Junwei Liang. Patchmixer: patch-mixing architecture for long-term time series forecasting. arXiv preprint arXiv:2310.00655. 2023.
[11] Nate Gruver, Mare Finzi, Shikai Qiu, and Andrew Wilson. Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems, 36, 2024.
[12] Lu Han, Xu-Yang Chen, Han-Jia Ye, and De-Chuan Zhan. Softs: Efficient multivariate time series forecasting with series-core fusion. arXiv preprint arXiv:2404. /4197, 2024.
[13] Lu Han, Han-Jia Ye, and De-Chuan Zhan. The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting. IEEE Transactions on Knowledge and Data Engineering, 2024.

---

[14] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 16000-16009, 2022.
[15] Haowen Hou and F Richard Yu. Rwkv-ts: Beyond traditional recurrent neural network for time series tasks, arXiv preprint arXiv:2401 0.9099, 2024.
[16] Qihe Huang, Lei Shen, Ruixin Zhang, Jiahuan Cheng, Shouhong Ding, Zhengyang Zhou, and Yang Wang. Hdmixer: Hierarchical dependency with extendable patch for multivariate time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 12608 12616, 2024.
[17] Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. Crossgnn: Confronting noisy multivariate time series via cross interaction refine- ment. Advances in Neural Information Processing Systems, 36, 2024.
[18] Qihe Huang, Zhengyang Zhou, Kuo Yang, Gengyu Lin, Zhongchao Yi, and Yang Wang. Leret: Language-empowered retentive network for time series forecasting. In Proceedings of the Thirty-Thite International Joint Conference on Artificial Intelligence, IJCAI-24, 2024.
[19] Yuxin Jia, Youfang Lin, Xinyan Hao, Yan Lin, Shengnan Guo, and Huaiyu Wan. Witran: Water-wave information transmission and recurrent acceleration network for long-range time series forecasting. Advances in Neural Information Processing Systems, 36, 2024.
[20] Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin- Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728, 2023.
[21] Ming Jin, Yifan Zhang, Wei Chen, Kexin Zhang, Yuxuan Liang, Bin Yang, Jindong Wang. Shirui Pan, and Qingsong Wen. Position: What large language models tell us about time series analysis. In Forty-first International Conference on Machine Learning. 2024.
[22] Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2021.
[23] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.690 12.69880 2014.
[24] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The s/ international ACM SIGIR conference on research development in information retrieval, pages 95-104, 2018
[25] Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting, Advances in neural information processing systems, 32, 2019.
[26] Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint 205. 2023.
[27] Zhe Li, Zhongwen Rao, Lujia Pan, Pengyun Wang, and Zenglin Xu. Ti-mae: Self-supervise masked time series autoencoders. arXiv preprint arXiv:2301.08877, 2023.
[28] Zhe Li, Zhongwen Rao, Lujia Pan, and Zenglin Xu. Mts-mixers: Multivariate time series forecasting via factorized temporal and channel mixing. arXiv preprint arXiv:2302.044 04501, 2023.
[29] Bryan Lim, Sercan Ank, Nicolas Loeff, and Tomas Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37() 1748-1764, 2021.
[30] Shengsheng Lin, Weiwei Lin, Wentai Wu, Songbo Wang, and Yongxiang Wang. Petformer: Long-term time series forecasting via placeholder-enhaneed transformer. arXiv preprint 2xX1v:2308.04791, 2023.
12

---

[31] Shengsheng Lin, Weiwei Lin, Wentai Wu, Feiyu Zhao, Ruichao Mo, and Haotong Zhang Segrnn: Segment recurrent neural network for long-term time series forecasting. arXiv preprint arXiv:2308. 1200, 2023.
[32] Shengsheng Lin, Weiwei Lin, Wentai Wu, Haojun Chen, and Junjie Yang. Sparsetsf: Modeling long-term time series forecasting with IK parameters. In Forty-first International Conference on Machine Learning, 2024.
[33] Haoxin Liu, Zhiyuan Zhao, Jindong Wang, Harshavardhan Kamarthi, and B Aditya Prakash. Lstprompt: Large language models as zero-shot time series forecasters by long-short-term prompting. arXiv preprint arXiv:2402. /6/32, 2024.
[34] Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 35:5816-5825 2022.
[35] Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International conference on learning representations, 2021.
[36] Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 35:9881 -9893, 2022.
[37] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang. Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024.
[38] Donghao Luo and Xue Wang. Modernten: A modern pure convolution structure for general time series analysis. In The Twelfth International Conference on Learning Representations, 2024.
[39] Henrik Madsen. Time series analysis. CRC Press, 2007.
[40] Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations, 2023.
[41] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[42] Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. Tib: Towards comprehensive and fair benchmarking of time series forecasting methods. Proc. VLDB Endow., 17(9):2363- 2377, 2024.
[43] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International journal of forecasting, 36(3): 1181-1191, 2020.
[44] William Toner and Luke Nicholas Darlow. An analysis of linear time series forecasting models.
In Forty-first International Conference on Machine Learning. 2024.
[45] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing 2016
ingredient for fast stylization. arXiv preprini
[46] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit. Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
[47] Huiqiang Wang, Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. Micn: Mclti-scale local and global context modeling for long-term series forecasting. The Eleventh International Conference on Learning Representations, 2022.
13

---

[48] Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and JUN ZHOU Timemixer: Decomposable multiscale mixing for time series forecasting, In The Twelfth International Conference on Learning Representations, 2024.
[49] Qingsong Wen, Tian Zhou, Chaoli Zhang. Weiqi Chen, Ziqing Ma. Junchi Yan, and Liang Sun. Transformers in time series: A survey. arXiv preprint arXiv:2202.07125, 2022.
[50] Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Etsformer: Expo- nential smoothing transformers for time-series forecasting. arXiv preprint arXiv:2202.01381, 2022.
[51] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long, Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34:22419-224333 2021.
[52] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. International Conference on Learning Representations, 2023,
[53] Zhijian Xu, Ailing Zeng, and Qiang Xu. Fits: Modeling time series with 10k parameters. In The Twelfth International Conference on Learning Representations, 2024.
[54] Hao Xue and Flora D Salim. Promptcast: A new prompt-based learning paradigm for time series forecasting. IEEE Transactions Knowledge and Data Engineering, 2023.
[55] Guoqi Yu, Jing Zou, Xiaowei Hu, Angelica Aviles-Rivers Jing Qin, and Shujun Wang. Revitalizing multivariate time series forecasting: Learnable decomposition with inter-series dependencies and intra-series variations modeling. Forty-first International Conference on Machine Learning, 2024.
[56] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121-11128, 2023.
[57] Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less more: Fast multivariate time series forecasting with light sampling-orien oriented mlp structures. arXiv preprint arXiv:2207.011888 2022.
[58] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In International Conference on Learning Representa- tions, 2023.
[59] Haoyi Zhou, Shanghang Zhang. Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106-11115, 2021.
[60] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning, pages 27268 27286. PMLR, 2022.
[61] Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained Im. Advances in neural information processing systems, 36, 2024.
14

---

A Development of time series forecasting
In recent years, the time series analysis community has shifted its focus from short-term forecasting to tasks with longer prediction horizons, also known as LTSF tasks This shift offers greater convenience but also poses increased challenges. Mainstream approaches be roughly classified into the following five distinct classes:
Transformer-basee Models It is widely recognized that Transformers possess impressive capabil- ities for long-distance modeling. and thus researchers have high expectations for their adaptation to long time series tasks 46 49]. Early research works, such as Trans [25], TFT (29], Informer [59]
Autoformer [51], Pyraformer [35]. FEDformer [60]. ETSformer [50], and NSTransformer [36],
focused on optimizing the original Transformer architecture for time series analysis tasks. However, more recent research has found that satisfactory performance can be achieved by simply partitioning patches, drawing inspiration from patch techniques used in the computer vision community 7 14)
Approaches like PatchTST [40]. PETformer (30, and Crossformer (58) have demonstrated promising results by adopting this patch-based approach.
Linear- and MLP-based Models Linear- and MLP-based methods are often fighter-weigh,, especially compared to Transformer methods that require stacking multiple blocks [57 4 A particularly notable breakthrough the observation made by DLinear [56], which demonstrates that a single-layer linear approach could outperform many complex Transformer designs. This observation leads to a sequence of works, including TiDE [5), MTS-Mixers (28). TSMixer (28). TimeMixer [48], HDMixer [16], SOFTS [2) FITS [53] TSF B2). and SSCNN [6. The proposed CycleNet in this paper is also a Linear- or MLPP-baed model that is simple, efficient, and powerful for time series forecasting.
RNN-based Models Conceptually, Recurrent Neural Networks (RNN) are considered to be the most suitable models for modeling time series data [24] 43. However, due to difficulties in parallelization and modeling long sequences, RNNs are not the most popular choice in works on LTSF tasks. Recent works aim to revitalize RNN models in long sequence modeling tasks, such as SegRNN 31), WITRAN [9), SutraNets (2]. and RWKV-TS [15].
TCN-based Models Because of the parallclizability of convolution operations and their ability to capture features different time scales, Temporal Convolutional Networks (TCN) methods are considered strong competitors for addressing time series tasks 91.. Recent works that apply TCN
methods to LTSF tasks include SCINet (34, MICN 47,, TimesNet [52] PatchMixer [10, and
ModernTCN [8].
LLM-based Models The remarkable capabilities demonstrated by large language models (LLM) have sparked interest among researchers from various fields, including those working on time series forecasting tasks [21 18] Some works consider fine-tuning pre-trained LLMs perform time
series analysis tasks, including OFA [61), Time-LLM (20). and TEMPO 3. Other works aim to achieve zero-shot inference using large pre-trained LLMs through prompt engineering, including LLMTime [1]] PrompiCast [54]. and LSTPrompt [3].
B
More details of CycleNet
B.1 Overall pseudocode
Algorithm| demonstrates the implementation of modeling periodic patterns through recurrent cycles. Specifically, the first line defines the learnable parameter queue Q and initializes it to zero. Lines 2-11 define the getCycle function, which will be called by CycleNet to obtain the corresponding truncated equivalent cyclic subsequences. This function takes two parameters, and i, where i represents the relative positional index for Q, and represents the length of the required subsequence. Q lcarns internal periodic patterns within the sequence through co-training with the backbone.
Furthermore, Algorithm 2 iillustrates the workflow of CycleNet. The first step is to normalize the samples based on their mean and standard deviation, then call the getCycle function to remove the cyclic components of the input data. Subsequently, predict the residual components through the backbone. Finally, add back the cyclic components of the output data, and perform instance denormalization to obtain the final prediction result. Here, the cycle index corresponds to mod W, as described in Section 3.1
15

---

Algorithm Modeling periodic patterns through recurrent cycles
Require: Number of channels and cycle length W
Ensure: Learned periodic patterns Q E RR W W
1: Initialize learnable parameters Q <- 0
> Q E R
2: function GETCYCLE(i.)
Define function Roll the queue to the appropriate index b Retrieve the required part directly from Q'
3: Q' - Roll( Q, shifts = -ē, dim 0)
4: if W then
5: return Vaa
6: else
Repeat Q' match the required length
7:
- L1/W d=/ mod W
8:
9: return Concat([Q)) x n, (Qo.))
Concatenate replicated Q' and the remaining part
end
11: end function
Algorithm 2 Workflow of CycleNet
Require: Look-back length L, forecast horizon H, cycle index i, and input 24-141: RLXD
Ensure: Forecast output 11414 E RD
1: if RevIN is applied then
2: 14,0 + Mean(fr-r+1:d) STD(
D. Compute mean and standard deviation
3:
Tt-1+1:1 f- -
b Remove instance-specific statistics
4: end if
5: 14-1+11 + -+-++ getCycle(i, L)
Remove the cycle component D Forecast using backbone model Restore the cycle component
6: F4114H
7: E4+1:+ E 14414 getCycle(i + H)
8: RevIN is applied then
9: I++1++ FesitH x (c+
Restore instance-specific statistics
10: end if
B.2 Utilizing ACF analysis to determine cycle length
The RCF technique utilizes recurrent cycles Q E WXD to model the internal periodic patterns of sequences. Here, the hyperparameter W determines the length of the recurrent cycles, which should precisely match the length of the periodic patterns within the data. As shown the results of Table 6 when W' is not accurately set, the RCF technique fails to fulfill its intended purpose. Although. in practice, we can infer the maximum cycle length of the dataset by considering the data's 's sampling frequency and the potential existing periodic patterns (as shown in Table T). this manual inference method may introduce errors. Therefore, we may need more scientific and precise approach to find the hyperparameter W.
In such cases, the autocorrelation function (ACF) [39 serves as a powerful mathematical tool to help us determine the periodicity within the data. The autocorrelation function measures the correlation between time series and its lagged values, indicating the presence autocorrelation within the data. Mathematically, this can be expressed as:
N 11=1 (x, (I+ - ( Z)
ACF
(6)
ΣN (2, 8)
where N represents the total number of observations, I; denotes the value of the time series at time t, is the lag time, and 5 is the mean of the time series values.
Here, when the lag time I aligns with the data's 's cycle, the ACF value exhibits a significant peak. Specifically, the largest peak corresponds to the lag that aligns with the length of the maximum cycle present the dataset. Conversely, the data lacks periodicity, no significant peaks or troughs will be observed.
present the ACF results for each dataset in Figure 6 can be observed that these datasets all display evident periodicity, indicated by prominent peaks and troughs in the plots. More importantly,
16

---

1.0
0.0
0.0
0.0
Autocomdtation
0.6
2.6
0.2
0.2
0.0
0.6
0.0
0.7
100
(a) ETThl, W 24
(b) ETTh2, W 24
(c) ETTml, W 96
(d) ETTm2, W 96
1.0
3.4
75
Auboorr
0.2
0.4
0.0
0.00
0.29
0.00
-.3
-02
-023
50
100 Logs
1.5
200
100
150
200
320
150
0090
50
100
202
Lags
Lags
Lags
(c) Electricity, W = 168 (f) Solar-Energy. W = 144
(g) Traffic, W = 168
(h) Weather, W = 144
Figure 6: Visualization of ACF results on the training set of different datasets. The hyperparameter
W should be set to the lag corresponding to the observed maximum peak.
the maximum cycles shown in the plots align with the pre-inferred cycle lengths from Table 1 This indicates the correctness of the pre-inferred lengths, and W should be strictly these values.
B.3 Experimental details
We utilized widely used benchmark datasets for LTSF tasks, including the ETT series, Electricity, Solar-Energy, Traffic, and Weather. Following prior works such Autoformer [51] and iTTrnss- former [37]. we split the ETTs dataset into training, validation, and test sets with a ratio of 6:2:2,
while the other datasets were split in a ratio of 7:1:2.
We implemented CycleNet using PyTorch [41] and conducted experiments on single NVIDIA RTX 4090 GPU with 24GB of memory. CycleNet patience of 5 on the validation set. The batch size was set uniformly to 256 for ETTss and the Weather dataset, and 64 for the remaining datasets. This adjustment a larger number of channels, requiring a relatively smaller batch size to avoid out-of-memory issues. The learning rate selected from the range (0.002, 0.005, 0.01) J based the performance on the validation set. The hyperparameter W was set consistently to the pre-inferred cycle length as shown in Table [ Additionally, the hidden layer size of CycleNet/MLP
trained for 30 epochs with early stopping based
made because the latter datasets have
uniformly set to 512.
By default, CycleNet uses RevIN without learnable affine parameters [22]. However, we found that on the Solar dataset, using RevIN leads to significant performance drop, as shown in Appendix C.4 The primary reason for this may be that photovoltaic power generation data contains continuous segments of zero values (no power generation at night). When the look-back windows integer multiple of a day, the calculation of means in RevIN can be significantly affected, leading to decreased performance. Therefore, for this dataset, did not apply the RevIN strategy.
C More experimental results
C.1 Periodic patterns learned under different configurations
The proposed RCF technique can effectively learn the inherent periodic patterns within time series data. This capability is a significant advantage, revealing the potential value of RCF or its underlying cyclic modeling approach as superior method to assist data engineers in analyzing patterns time series data. To further elucidate the working principle behind RCF, we delve into the periodic patterns learned by the RCF technique under different configurations, as illustrated in Figure 7
Forecast horizon H: The learned patterns remain almost unchanged as the horizon length varies. This indicates that the horizon length does not affect the learned pattern results.
17

---

MMMM
MM MMMMMM
MMMM
MMMM
(a) Horizon-99
(b) Horizon- 192
(c) Horizon-336
(d) Horizon-720
MMmM M
(e) Lookback-96
(f) Lookbackk199 192
(g) Lookback-336
(h) Lookback-7200
MMmm MM
M MMMMmm
(i) Backbone- -inebane-Linear
(j) Backbone-DLinnar
(k) Backbone- PatchTTTT
(l) Backbone-
Backbone-TTransfrmmee Trannsformer
MMmm MM
M MMM
63
(m) W-168
nn) W-96
(0) W-24
(p) W-23
Figure 7: Periodic patterns of the 321st st channel in the Electricity dataset, learned under different configurations. The basic configuration includes both look Linear model as the backbone, and the correct cycle length W set to 168.
look-back back and horizon length of 96, simple
Look-back length L: The overall pattern remains unchanged the look-back window changes. However, with closer observation, it is noticeable that the learned pattern becomes smoother with increased look-back. This is because a longer look-back provides the backbone with richer periodic information, thereby reducing the reliance on the learned pattern component.
The patterns vary somewhat with different backbones. When DLinear is used as the backbone, the learned patterns are smoother, as DLinear's 's decomposition techmique itself extracts certain periodic features. When iTransformer is the backbone, the learned patterns differ more, as additionally models multichannel relationships, so the learned periodic patterns may consider multichannel feature interactions. PatchTST's performance is more similar to that of Linear, as it is also a regular single-channel modeling method, though with stronger nonlinear learning capabilities compared to the Linear model.
Backbone:
Cycle length W: When W is set to 168 (the weekly cycle length for the Electricity dataset), the recurrent cycle Q learns the complete periodic pattern, including both weekly and daily cycles. When W is set to 24 (the daily cycle length), the recurrent cycle Q only learns the daily cycle pattern. When W is set 96 (four times the daily cycle length), the recurrent cycle Q learns four repeated daily cycle models. However, when W is set to 23 (without matching any semantic meaning), the recurrent cycle Q fails to learn any meaningful pattern, resulting in straight line.
C.2 Full results with different look-back lengths
Table 2 presents the comparison results of CycleNet with other models on the mean performance at look-back length L = 96 for various forecast horizons H {96, 192, 336, 720]. Here, we further
18

---

Table 7: Full results of different models with the look-back length L. = 96. The reported results with standard deviation of CycleNet are averaged from 5 runs (with different random seeds of {2024, 2025, 2026, 2027, 2028)). The results of other models are sourced from iTransformer [37 The best results are highlighted in bold and the second best are underlined.
Model
FEDfomer
TimesNet
iTransformer
CycleNet/LLinear
CycleNet/MLP
Metric
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MAE
96
0.376
0,419
0.384
0.402
0.386
0.405
0.378=0.001
0.391=000001
0.375=00001
0.395=0.001
ETThl
192
0.420
0.448
0.436
0.429
0.441
0.436
0.426-0.001
0.419 0.001
0.436 0.0022
0.028=00002
336
0.459
0,465
0.491
0.469
0.487
0.458
0,464-0.001
0.009
0.496:10.001
0.455.000003
720
0.506
0.507
0.521
0.500
0.503
0.491
0.461+0.001
0.000
0.520+0.021
0.484
Avg
0.440
0.460
0.458
0.450
0.454
0.448
0.402:10.001
0.407
0.457:10.006
0.441 0.0004
96
0,358
0.397
0.340
0.374
0.297
0.349
0.285=0.001
0.305=100001
0.298
0.344=0.001
ETTh2
192
0.429
0,439
0,402
0.414
0,380
0,400
0.373-00001
0.301
0.372:00002
0. 0.002
336
0.496
0,487
0.452
0.452
0,428
0,432
0.401
0.431 0.403
0,439=10000
720
0.463
0.474
0.462
0.468
0.427
0.445
0.453-0.0033
0.458
0.450=0.010
0.058
Avg
0..477
0,449
0.414
0.427
0.383
0.407
0.383-10.001
0.404
0.388=0.005
0. 0.409
96
0.379
0.419
0.338
0.375
0.334
0.368
0.325=00001
0.363=0.001
0.319=0.001
0.360=0.001
ETTml
192
0.426
0.441
0.374
0.387
0.377
0.391
0.366=0.001
0.382-00001
0.360=0.002
0.001=0.001
336
0,445
0,459
0,410
0.411
0,426
0.420
0.096=00001
0.401=0.001
0.08910:001
0.403=0.001
720
0,543
0,490
0.478
0.450
0.491
0.459
0.457=0.001
0.433=0.001
0.447=0.001
0.441=0.001
Avg
0.448
0.452
0.400
0.406
0.407
0.410
0.386+0.001
0.305
0.379+0.001
0.396
96
0.203
0.287
0.187
0.267
0,180
0.264
0.166=0.001
0.248=0.001
0.163=0.001
0.246=0.001
ETTm2
192
0,269
0.328
0.249
0.309
0.250
0.309
0.233-0.001
0.291+0.001
0.290+0.00
336
0,325
0.366
0.321
0.351
0.311
0.348
0.293
0.330
0.284 0.000
0.327 0.007
720
0.421
0.415
0.408
0.403
0.412
0.407
0.395=0.001
0.389=0.001
0.389=0.003
0.391=0.002
Avg
0.305
0.349
0.291
0.333
0.288
0.332
0.272:0.001
0.315=00001
0.266 0.001
0.314 0.000
Electricity
96
0,193
0.308
0.168
0.272
0.148
0.240
0.001
0.234110000
0.136=:0000
0.229-00.00
192
0.201
0.315
0.184
0.289
0.162
0.253
0..001000011
0.247+0.00
0.152#0.001
0.244 + 0.001
336
0.214
0.329
0.198
0.300
0.178
0.269
0.102
0.004
0.170
0.264 001
720
0,246
0.355
0.220
0,320
0.225
0.317
0.210=0.001
0.2961100001
0.212=0.001
0.299=0.000
Avg
0,214
0,327
0.193
0:295
0,178
0.270
0.000
0.200
0.168 0.000
0.259
Solar- Energy
96
0,242
0.342
0.250
0,292
0,203
0.237
0.209-0.001
0.260
0.190
0.247-0.003
192
0,285
0,380
0.296
0,318
0.235
0.261
0.231=0.002
0.269=00002
0.210=0.004
0.266=0.008
336
0,282
0.376
0.319
0.330
0.248
0.273
0.246-00002
0.005
0.217
0.266
720
0.357
0,427
0.338
0.337
0.249
0.275
0.255=0.001
0.274=00003
0.223
0.266=0.003
Avg
0.292
0.381
0.301
0.319
0.233
0.262
0.235=0.001
0.270=00002
0.210=0.005
0.261=0.005
96
0.587
0.366
0.593
0.321
0.395
0.268
0.480=0.001
0.458=0.001 0.314=0.001
0.296=0.001
Traffic
192
0.604
0.373
0.617
0,336
0.417
0.276
0.482=0.001
0.313=00001
0.457=0.001
0.294=0.001
336
0.621
0.383
0.629
0.336
0.433
0.283
0.476+0.001
0.303++0001
0.470+0.001
0.299+0.000
720
0.626
0,382
0.640
0.350
0.467
0.302
0.503=0.001
0.320=0.001
0.502=0.001
0.314=0.001
Avg
0.610
0.376
0.620
0.336
0.428
0.282
0.485+0.001
0.313+0.001
0
0.301 0.000
0,217
0.296
0.172
0.220
0.174
0.214
0.000
0.216
0.158
0.203
Weather
192
0.276
0.336
0.219
0.261
0.221
0.254
0.222+0.001
0.259
0.207+0.001
0.247 0.000
336
0.339
0.380
0.280
0.306
0.278
0.296
0.005
0.296
0.262+0.001
0.289 0.001
720
0.403
0.428
0.365
0.359
0.358
0.349
0.349-0.001
0.345
0.344
0.34410.001
Avg
0.309
0.360
0.259
0.287
0.258
0.278
0.254
0.279
0.243 0.001
0,271 +00001
showcase the complete comparison results for different forecast horizons in Table 7 It can be observed that in most settings, CycleNet achieves state-of-the-ar results, consistent with the findings in Table 2 Additionally, the standard deviation of CycleNet's results is mostly below 0.001. This strongly indicates the robustness of CycleNet.
Additionally, the look-back length is crucial hyperparameter that significantly impacts the perfor- mance of time series forecasting models, as it determines the richness of information the model can leverage. Initially, the community focused primarily on exploring the application of Transformers in time series forecasting tasks. Due to the inherent complexity of Transformers, using excessively long look-back windows resulted in a significant increase in runtime. As result, many popular models at the time, such as Informer (59], Autoformer [5], and FEDformer (60) employed shorter look-
windows, typically with L = 96.
19

---

Table 8: Full results of different models with longer look-back lengths L E 720). The reported results of CycleNet are averaged from 5 runs (with different random seeds of {2024, 2025, 2026, 2027, 2026,2027,220 2028)). The results of other models are reproduced after fixing long- standing bug (discarding the last batch of data during the test phase). The best results are highlighted in bold and the second best are underlined.
I Lookhack
336
720
DL Dlinear
PatcTTST 2023
CycleNer /Lincar
CycleNet /MLP
Se5RNN
SparseTTF
CycleNet
CycleNet
Model
2023
2025
2024
/Lincar
/MLP
Metric
MSB
MAB
MSB
MAB
MSE
MAE
MSE
MAE
|| MSB
MAB
MSB
MAE
MSE
MAB
MSE
MAB
96
0.374
0.398
0.385
0.405
0.374
0.396
0.382
0.403
0.351
0.392
0.362
0.388
0.379
0.403
0.385
0.412
BTTb
192
0.430
0.440
0.414
0.421
0.406
0.415
0.421
0.426
0.390
0.418
0.405
0.411
0.416
0.425
0.424
0.438
336
0.442
0.445
0.440
0.440
0.431
0.430
0.449
0.444
0.449
0.452
0.434
0.428
0.447
0.445
0.460
0.463
730
0.497
0.507
0.456
0.470
0.450
0.464
0.497
0.485
0.492
0.494
0.426
0.447
0.477
0.483
0.486
0.487
Avg
0.436
0.448
0.424
0.415
0,426
0.437
0.440
0.421
0.439
0.406
0.419
0.430
0.439
0.439
0.450
96
0.281
0.347
0.275
0.337
0.279
0.341
0.300
0.355
0.275
0. 0.338
0.294
0. 1.346
0.271
0.337
0.293
0.352
ET1h2
192
0.367
0.404
0.338
0.379
0.342
0.385
0.573
0.403
0.338
0. 0.380
0.339
0.377
0.332
0.380
0.359
0.395
336
0.438
0.454
0.365
0,298
0.971
0:413
0,384
0.419
0.419
0.445
0.359
0.297
0.362
0.408
0.392
0.423
720
0.598
0.549
0.391
0,429
0.426
0.451
0,428
0.450
0.431
0.464
0,383
0.424
0.415
0.449
0.425
0.451
Aug
0.421
0.439
0.342
0.386
0 0.355
0.398
0.371
0.407
0.366
0.407
0.344
0.386
0.345
0.394
0.367
0.405
96
0.307
0.350
0.291
0.343
0.299
0.348
0.297
0.351
0.295
0.356
0.312
0.354
0.307
0.353
0.301
0.357
BTTaa
192
0,340
0.373
0.334
0.370
0.334
0.367
0,338
0.377
0.334
0. 0.382
0.347
0.376
0.337
0.371
0.341
0.377
336
0.377
0.397
0.367
0.392
0,368
0.386
0.374
0,400
0.359
0.401
0.367
0.386
0.361
0.387
0,376
0.396
720
0.433
0.433
0.422
0.426
0.417
0.414
0.436
0.431
0.415
0.435
0.419
0.413
0.410
0.411
0.431
0.425
Avg
0,364
0.388
0.354
0.383
0.355
0.379
0,361
0,390
0,351
0.394
0.361
0.382
0.355
0,381
0.362
0.389
96
0.165
02257
0.164
0.254
0.159
0.247
0.178
0,262
0.165
0:251
0.163
0,252
0.159
0.249
0.176
0.365
ET1o2
192
0.227
03307
0.221
0.203
0.214
0,286
0.238
0.303
0.226
0,300
0.217
0.2 0.20
0.214
0,289
0.231
0.305
336
0,504
0.362
0.276
0:328
0.269
0.322
0,292
0.339
0.282
0.341
0.270
0.327
0.268
0.326
0.282
0.338
720
0.431
0.441
0.306
0,383
0.363
0.382
0.374
0.391
0.361
0.392
0.352
0.379
0.353
0.384
0.361
0.388
Avg
0.282
0.342
0.257
0.315
0.251
0.309
0.271
0.324
0,259
0.321
0.251
0.312
0.249
0.312
0.263
0.324
Elestricity
0.140
0,237
0.131
0.225
0.128
0.223
0.126
0.221
0.130
0.228
0.138
0.128
0.223
0.127
0.223
192
0.153
0.250
0.148
0.240
0.144
0.237
0.144
0.237
0.152
0.251
0.151
0.344
0.143
0.237
0.144
0.239
336
0,169
0.267
0.165
0.259
0.160
0,254
0,160
0,255
0.170
0.272
0,166
0. 16260
0.159
0.254
0.159
0.255
720
0,203
0,202
0.291
0.198
0,287
0.199
0.203
0.304
0.205
0,293
0.197
0.287
0.290
Avg
0.166
0.263
0.162
0.254
0.158
0.250
0.157
0.251
0.164
0,264
0.165
0,258
0.157
0.250
0.157
0,252
Emurgy
96
0.222
0.292
0.190
0.278
0,200
0.250
0.182
0.245
0.175
0.236
0.195
0.243
0.194
0.255
0.174
0.232
192
0.249
0,313
0.206
0.252
0.221
0.261
0,191
0.254
0,193
0.268
0.215
0.254
0.205
0.251
0.187
0.246
336
0.268
0.527
0.217
0.254
0.236
0,272
0.197
0.257
0,209
0,263
0.232
0.262
0,218
0.257
0.194
0,252
720
0.271
0.326
0.219
0.255
0,245
0.277
0.207
0.264
0.205
0.264
0.237
0. 0.263
0.239
0.278
0.201
0.259
Avg
0.253
0.315
0 0.208
0.260
0,226
0.265
0.194
0.255
0.196
0.258
0,220
0. 25556
0,214
0.260
0.189
0,247
96
0.410
0.282
0.373
0.254
0.397
0.271
0.386
0, 0.268
0,356
0.255
0,389
0 162
0.381
0.266
0.374
0.368
Traliic
0.423
0.288
0.391
0.262
0,411
0.283
0,404
0.276
0.268
0.398
0.270
0.394
0.273
0.275
336
0.436
0.296
0.404
0.269
0.424
0.289
0.416
0.281
0.393
0.273
0.411
0.275
0.406
0.279
0.405
0,282
720
0.466
0.315
0.436
0.287
0.450
0.305
0.445
0.300
0.434
0.294
0.448
0. 0.29
0.441
0.300
0.441
0.302
Avg.
0.434
0:295
0.401
0.268
0.421
0.289
0.413
0.281
0.389
0.273
0.412
0.278
0.406
0.280
0.403
0.282
96
0.174
0.235
0.155
0.204
0.167
0.221
0.148
0.200
0.141
0. 0.205
0.169
0.223
0,164
0.220
0.149
0.203
Weather
0.219
0.195
0.242
0,212
0.258
0.240
0.250
0.214
0.262
0.209
0.258
0,244
336
0.264
0.317
0.249
0.283
0,260
0.293
0.243
0,283
0.241
0.297
0.257
0.293
0.255
0.292
0.242
0,283
730
0.324
0.363
0.321
0.334
0.728
0.339
0.322
0.359
0.318
0.382
0.321
0.340
0.320
0.358
0.312
0.333
Avg
0.245
0.230
0.266
0.242
0.278
0.226
0.266
0.221
0.276
0.240
0.237
0.277
0.224
0,266
With the recent development of model lightweighting techniques, particularly the adoption of channel- independent strategies (first applied in DLinear (56) and PatchTST [40), more models have started to experiment with longer back-back windows in pursuit of higher predictive accuracy. For instance, DLinear and PatchTST default to using look-back windows of L = 336, while SegRNN [31 and Sparse 32 default to using L = 720. To explore CycleNet's performance with longer look-back windows, we compared CycleNet with these advanced models using their respective default, longer look-back windows in Table g
It is important to note that we re-ran the official open-source code of these basclines to obtain the corresponding results, using the same MSE as the loss function (as SegRNN originally used MAE as its loss). Additionally, there was a long-standing bug in their original repositories, where the data from the last batch was discarded during testing [42] [53] This issue could have affected the model's s performance, so we fixed this problem before re-running the experiments.
It can be observed that even with longer look-back length, CycleNet generally maintains a significant advantage, achieving otate-of-the-arr performance in most scenarios. This demonstrates CycleNet's excellent performance across different look-back lengths. It is worth noting that both PStchTST and
20

---

SegRNN outperform CycleNet on the Traffic dataset, even though they are also channel- indeeedddnde models. This is partly because the Traffic dataset contains more outliers (see more discussion in Appendix C.5 which may impact the performance of RCF; additionally, PatchTST and SegRNN are more complex deep models with stronger nonlinear capabilities, enabling them to fit various patterns across numerous channels (the Traffic dataset has up to 862 channels).
C.3 Full results with different STD techniques
Table 9: Full results of comparison of different STD techniques. The configuration used here is consistent with that of DLincar [56], where a pure Linear model serves as the backbone, look-back length of 336 is employed, and no additional instance normalization strategies are applied. Thus, CLinear here refers CycleNet/Linnaar without RevIN. The best results are highlighted in bold and the second best are underlined.
CLincar (RCF+Linear)
LDLinear (ID++Linear)
DLincar (MOV+I +I Linear)
SLincar Sgaaree+Linaar
Model
Linear
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.370
0.395
0.372
0.394
0.372
0.394
0.366
0.388
0.374
0.395
192
0.404
0.417
0,410
0.420
0.408
0,417
0.406
0.414
0,409
0.418
336
0.434
0.440
0.449
0.452
0.441
0.442
0.440
0.442
0.442
0.444
720
0.465
0.486
0.476
0.492
0.480
0.494
0.483
0.501
0.484
0.498
Avg
0.418
0.434
0.427
0.439
0.425
0.437
0.424
0.436
0.427
0.439
96
0.308
0.369
0.292
0.357
0.297
0.362
0,340
0.389
0.305
0.368
ETTh2
192
0,382
0.416
0.372
0.409
0.398
0,426
0.379
0.413
0,385
0.419
336
0.454
0.465
0,479
0,480
0.496
0,489
0.404
0.437
0,458
0.470
720
0
0.575
0.675
0,582
0.694
0,592
0.720
0.600
0,691
0.592
Avg
0.451
0.456
0.455
0.457
0.471
0.467
0.460
0.460
0.460
0.462
96
0.298
0.350
0.305
0,350
0.309
0.356
0,306
0.349
0.305
0.349
ETTml
192
0.330
0.370
0.335
0.366
0.346
0.380
0.339
0.370
0.338
0.369
336
0.359
0.388
0.372
0.390
0.373
0.391
0.372
0.389
0.371
0.389
720
0.410
0.421
0.445
0.443
0.439
0.435
0.430
0.426
0,433
0.428
Avg
0,349
0,382
0.365
0,387
0.367
0.390
0.362
0.383
0,362
0,384
96
0.164
0.260
0.165
0.257
0,165
0.257
0.177
0.272
0.166
0.259
ElTm2
192
0.225
0,304
0,240
0.318
0.232
0.310
0,246
0.325
0.228
0.305
336
0.271
0.332
0,290
0.349
0.295
0,356
0,309
0.370
0.275
0.334
720
0.406
0.423
0.396
0.419
0.427
0.442
0.427
0.440
0.407
0.425
Avg
0.266
0.330
0.273
0.336
0.280
0.341
0.290
0.352
0.269
0.331
Eleetricity
96
0.131
0.228
0.140
0.237
0.140
0.237
0.148
0.243
0.140
0.238
192
0.145
0.242
0.154
0.250
0.154
0.250
0.159
0.254
0.154
0.251
336
0.160
0.260
0.170
0,268
0.169
0.268
0.173
0.271
0.170
0,269
720
0.193
0,292
0.204
0,300
0.204
0.301
0.207
0,303
0.204
0.301
Avg
0.157
0.255
0.167
0.264
0.167
0.264
0.172
0.268
0.167
0.265
Solar- Energy
96
0.192
0.251
0.222
0.294
0.222
0.298
0.226
0.296
0.224
0.302
192
0.218
0.258
0.249
0.315
0.250
0.312
0.252
0.312
0.250
0.310
336
0.231
0.262
0.268
0.326
0.270
0.335
0.270
0.326
0.269
0.325
720
0.239
0.265
0.271
0.327
0.272
0.327
0.271
0.327
0.270
0.333
Avg
0.220
0.259
0.253
0.316
0.254
0,318
0.255
0.315
0,253
0.318
96
0.397
0.275
0.411
0.285
0.411
0.284
0.414
0.281
0.411
0.283
Traffic
192
0.412
0.282
0.423
0.288
0.423
0,289
0.425
0.285
0.423
0.289
336
0.426
0.290
0.436
0.296
0.436
0.296
0.436
0.293
0.437
0.297
720
0.456
0.308
0.466
0.315
0.466
0.316
0.464
0.310
0.466
0.316
Avg
0.423
0.289
0.434
0.296
0.434
0.296
0.435
0.292
0.434
0.296
96
0.174
0.240
0.174
0.235
0.175
0.237
0.176
0.235
0.175
0.235
Weather
192
0.218
0.279
0.215
0,271
0.215
0.273
0.218
0.277
0.218
0,276
336
0.262
0.314
0.263
0.315
0.261
0.311
0.265
0.316
0.262
0.312
720
0.328
0.367
0.325
0.365
0.324
0.363
0.325
0.363
0.327
0.366
Avg
0.245
0.300
0.244
0.297
0.244
0.296
0.246
0.298
0.245
0.297
The proposed RCF technique is essentially a type of Seasonal-Treen Decomposition (STD) method. To directly compare RCF with existing related STD techniques, we adopted a strategy consistent with DLinear, using pure Linear model as the backbone and not applying any instance normalization
21

---

techniques. We previously reported the mean performance of these techniques across different horizons H E [96, 192, 336, 720) in Table Here, we further present the complete comparative results for all horizons in Table 2
The results show that the RCF technique consistently outperforms other techniques. A notable exception is the relatively noisy weather dataset, where RCF does not show a significant advantage. However, in this case, the performance of several STD techniques similar to that of the pure Linear model. Overall, these findings strongly support RCF as new STD method that enhances model performance in scenarios with strong periodicity.
C.4
Ablation study of RevIN
Table 10: Ablatioin results of RevIN.
CycleNet/l. w. RevIN
CycleNet/I. w/o. RevIN
RLinear 2023
CycleNet/M w. ReviN
CycleNet/M w/o, RevIN
RMIP
Model
[2023
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.377
0.391
0.379
0.399
0.385
0.393
0.378
0.397
0.383
0.401
0.383
0.401
1:1112 ETThl
192
0,426
0.419
0.423
0.428
0,439
0.424
0.440
0.431
0.431
0.436
0.437
0.432
336
0,464
0.439
0.460
0.452
0,483
0.448
0.495
0.453
0.486
0.467
0.494
0.461
720
0.462
0.460
0.484
0.494
0.481
0.470
0.502
0.473
0.547
0.516
0.540
0.499
96
0,286
0.336
0.328
0.381
0.291
0.339
0.298
0.344
0.326
0.377
0.299
0.345
192
0.372
0.391
0.467
0,464
0.375
0.389
0.374
0.400
0.421
0.435
0.371
0.394
336
0,422
0,433
0.570
0.523
0,414
0,425
0.425
0.435
0.522
0.490
0.420
0.429
720
0.457
0,460
0.773
0.630
0.420
0,440
0.442
0.454
0.876
0.647
0.438
0.450
96
0,325
0.363
0.327
0.371
0.351
0.372
0.320
0.361
0.338
0.383
0.327
0.366
HTTm2 HTTml
192
0.366
0.382
0.359
0.388
0.390
0.390
0.361
0.382
0.367
0.393
0.370
0.386
336
0.396
0.402
0,391
0.414
0.423
0.414
0.392
0.404
0.396
0.419
0,404
0.410
720
0.457
0.434
0,434
0.442
0.486
0.448
0.448
0.441
0.447
0.448
0.462
0.445
96
0,168
0,249
0.176
0.272
0.184
0.266
0.164
0.246
0.174
0.266
0.178
0.259
192
0.232
0.290
0.249
0.324
0.248
0.305
0.232
0.291
0.248
0.318
0.242
0.302
336
0,293
0.330
0.325
0.378
0.307
0.342
0.283
0.328
0.304
0.361
0.299
0.340
720
0.394
0.389
0.526
0.495
0.408
0.397
0.385
0.389
0.512
0..788
0.400
0.398
Electricity
96
0.142
0.234
0.142
0.239
0.198
0.275
0.136
0.230
0.138
0.235
0.182
0.265
192
0.156
0.247
0,155
0.252
0.198
0.277
0.153
0.245
0.154
0.250
0.187
0.270
336
0.173
0.265
0,170
0.269
0,212
0.293
0.170
0.264
0.171
0.269
0.203
0.287
720
0.211
0.297
0.199
0,298
0.254
0.325
0.212
0.300
0,206
0.302
0.244
0.319
96
0.250
0.277
0.208
0.256
0.308
0.332
0.195
0.252
0.187
0.245
0.236
0.270
Solar
192
0.289
0.299
0.231
0.269
0.345
0.349
0.225
0.272
0.215
0.275
0.270
0.290
336
0.338
0.323
0.247
0.272
0.387
0.364
0.248
0.289
0.212
0.257
0.296
0.305
720
0.351
0.326
0.258
0.275
0.390
0.358
0.253
0.286
0.228
0.269
0.296
0.303
96
0.480
0.314
0.475
0.302
0.647
0.386
0.459
0.297
0.469
0.298
0.510
0.331
Truffic
192
0.482
0.313
0.475
0.305
0.600
0.362
0.457
0.295
0.477
0.304
0.505
0.327
336
0.476
0.303
0.489
0.313
0.607
0.365
0.470
0.300
0.487
0.302
0,518
0.332
720
0.505
0.321
0,518
0.327
0.644
0,383
0.502
0.314
0,522
0.315
0,553
0.350
Weather
96
0.170
0.216
0.209
0.284
0.197
0.236
0.158
0.203
0.179
0.247
0.181
0.219
192
0.222
0.260
0.265
0.334
0.239
0.270
0.207
0.248
0.220
0.284
0.228
0.259
336
0.276
0.296
0.314
0.368
0.292
0.307
0.263
0.290
0.273
0.325
0.282
0.299
720
0.350
0.345
0.378
0.410
0.365
0.353
0.344
0.345
0.345
0.377
0.357
0.347
Instance normalization strategies constitute essential factors for the success of current models, such as PatchTST [40] TiDE [5], iTransformer [37]. Sparse [32], etc. By default, CycleNet also
adopts this strategy, namely the version of RevIN without learnable affine parameters 22). Here, we meticulously investigate the impact of RevIN on the performance of CycleNet, and the results are shown in Table T0 On the ETTh2 and Weather datasets, RevIN significantly enhances the performance of CycleNet, possibly due to more severe distribution drift issues in these datasets. However, on the Solar dataset, RevIN leads to poorer performance, likely because the photovoltaic power generation data contains continuous segments of zero values (no power generation at night), which significantly affects the calculation of means in RevIN.
22

---

Overall, in most cases, RevIN leads to better performance. We acknowledge that RevIN is an indispensable cornerstone of CycleNet's success, but it is not the key factor that sets CycleNet apart from other models in terms of performance. As shown in the comparison results in Table TO CycleNet exhibits a significant advantage over RLinear and RMLP, which can be viewed as CycleNet without RCF technique. This clearly demonstrates that the RCF technique is the key factor that significantly enhances the model's prediction accuracy, constituting the core contribution of this paper.
C.5 Further Analysis in Traffic Scenarios
Table 11: Comparison results on the PEMS datasets. The look-back length L is fixed at 96, and the forecast horizons are set to H E {12, 24, 48, 96). The results of other models are sourced from iTransformer [37.. The best results are highlighted in bold, and the second-bes are underlined.
Model
CycleNet /MLP
CycleNet /Lincar
RLinear 2023
iTrunsformer [2024]
PutchTST
Crossformer 2023
DLinear 2023
SCINet 2022
2023
Metric
MSE
MAE
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
PEMS00
12
0
0.172
0.080
0.192
0.126
0.236
0.071
0.174
0.099
0.216
0.090
0.203
0.122
0.243
0.066
0.172
24
0.089
0.201
0.120
0.237
0.246
0.334
0.093
0.201
0.142
0.259
0.121
0.240
0.201
0.317
0.085
0.19%
48
0.136
0.247
0.156
0.551
0.236
0.211
0.319
0.317
0.333
0.425
0.127
0.238
96
0.182
0.282
0.199
0.292
1.057
0.787
0.164
0.275
0.269
0.370
0.262
0.367
0.457
0.515
0.178
0.287
PEMS00
12
0.078
0.069
0.201
0.138
0.252
0.078
0.183
0.105
0.224
0.098
0.218
0.148
0.272
0.073
0.177
24
0.099
0.212
0.127
0.245
0.258
0.348
0.095
0.205
0.153
0.275
0.131
0.256
0.224
0,340
0.0144
0.193
48
0.133
0.248
0.169
0,286
0.572
0.544
0.120
0,233
0.229
0.339
0.205
0.326
0.355
0,437
0.099
0.211
0.167
0.189
0.293
1.137
0.820
0.150
0.262
0.291
0.389
0.457
0.504
0.114
0.227
PEMSON PEMS07
12
0
0.162
0.075
0.183
0.118
0.235
0.067
0, 0,165
0.095
0.207
0.094
0.200
0.115
0.242
0.068
0,171
0,086
0.192
0.113
0.088
0.262
0.247
0.329
0.225
48
0.128
0.234
0.157
0.254
0,562
0.541
0.110
0.215
0.253
0,340
0.311
0.369
0.398
0.458
0.149
0.237
96
0.176
0,268
0.207
0,291
1.096
0.795
0,139
0.245
0.3-16
0,404
0.396
0.442
0.594
0.553
0,141
0.234
12
0.082
0.185
0.091
0
0.133
0.247
0.079
0.182
0.168
0.232
0.165
0.214
0.154
0,276
0.087
0.184
24
0.117
0.226
0.140
0,251
0.249
0.343
0.115
0.219
0.224
0.281
0.215
0.260
0.248
0.353
0,122
0.221
0.169
0.268
0,200
0.291
0.569
0.544
0.186
0.235
0.321
0.354
0.315
0.355
0.440
0.470
0.189
0.270
96
0.233
0.306
0.272
0.328
1.166
0.814
0.221
0,267
0.408
0.417
0.377
0.397
0.674
0.565
0.236
0,300
Avg.
0.125
0.229
0.149
0.252
0.514
0.482
0.119
0.218
0.217
0.306
0.220
0.304
0.320
0.394
0.121
0.222
CycleNet, formed by combining the RCF technique with simple backbone, achieved state-of-the-art performance across multiple domains but fell short in the traffic domain. To further investigate the reasons behind this, we supplemented the complete performance of CycleNet on the PEMS dataset (the same four public subsets adopted in SCINet 34 in Table TT The results show that: (i) CycleNet still achieved top-tier prediction accuracy, and (ii) although CycleNet underperformed compared to iTransformer in this scenario, the gap in MSE on the Traffic dataset was reduced from approximately 10% to about 5%.
Regarding the first point, it is important highlight the effectiveness of RCF CycleNet's backbone is merely a single-layer Linear or a two-layer MLP, without any additional design or deep stacking, yet it still delivers excellent results. Specifically, when comparing CycleNet/I Linear with Rlinear and Diinear, it becomes evident that RCF is the major contributor to narrowing the gap between the simple Linear model and those state-of-the-an models.
Table 12: Statistical characteristies of datasets, including average number of extreme points per channel (Z-Score 6), average maximum extreme value per channel, and cosine similarity between channels.
Traffic
Electricity
Solar-Energy
ETThl
PEMS03
PEMS04
PEMS07
PEMS08
Avg. Extreme Points
23.8
1.4
0
0.9
0.1
3.5
4.8
Avg. Max Extreme
9.27
4.14
2.92
4,08
2,87
2.66
2.61
2.77
Cosine Similarity
0.56
0.46
0.92
0.21
0.84
0.77
0.80
0.78
For the second point, we further analyzed the statistical characteristics of the datasets to explore the underlying reasons in Table 12 Specifically, we examined the presence of extreme values in the channels and the cosine similanty between channels. It was found that the Traffic dataset contains very significant outliers, both in terms of quantity and magnitude. The presence of these outliers:
(i) May affect the effectiveness of RCF The fundamental working principle of RCF is to learn the historical average cycles in the dataset. In such cases, the average cycles learned in RCF can be skewed by these significant outliers, such as the mean of certain point in the cycle being exaggerated.
23

---

Consequently, during each prediction process, the original sequence subtracts a locally exaggerated average cycle, resulting an inaccurate residual component and affecting the local point predictions within each cycle. The more inaccurate these local point predictions are, the larger the discrepancy between MSE and MAE, as MSE significantly amplifies the impact of a few large errors. This combining ITransformer with RCF decreases MAE but increases MSE, indicating overall prediction accuracy improvement but anomalies local point predictions.
explains why in Table 4
(ii) Highlight the necessity of stronger spatiotemporal relationship modeling. Models like iTransformer and GNN, which accurately model inter-channel relationships, are more suitable for scenarios with extreme points and temporal lag characteristics. For example, when sudden traffic surge occurs at certain junction, these models, having correctly modeled the spatiotemporal relationships. can accurately predict possible traffic surges at other junctions. In contrast, the current CycleNet only considers single-channel relationship modeling, making it somewhat limited this scenario.
These underlying reasons explain why CycleNet did not achieve the best performance on the Traffic dataset and showed a relative large performance gap. On the PEMS dataset, although it is also a traffic dataset, the presence of extreme points is significantly less severe compared the Traffic dataset. Therefore, CycleNet's performance on the PEMS dataset improved compared to the Traffic dataset (the gap MSE compared to the state-of the-art reduced from approximately 10% to about 5%). This further validates the effectiveness of RCF but also indicates that in more complex traffic scenarios, reasonable spatiotemporal relationship modeling (or multivariate relationship modeling) is essential.
Additionally, while intuitively the solar scenarios might also involve significant spatiotemporal relationships, in practice, these relationships are much weaker compared to the traffic scenarios. Firstly, the weather conditions in the same region often similar, leading similar power generation curves. For instance, the Solar-Energy dataset's channels have a cosine similarity as high as 0.92 (shown in Table 12). which indirectly indicates weaker spatial characteristics. Secondly, extreme points are rare in the solar scenarios because photovoltaic systems have maximum power threshold. Fewer extreme points mean that the impact of temporal lag characteristics is smaller. This explains why, compared to the Traffic dataset, the gains from the RCF technique are much more significant on the Solar- VolaryEnergy dataset.
In summary, when dealing with traffic scenarios that may involve significant outliers and emphasize spatiotemporal relationship modeling. the current version of CycleNet may not be fully adequate. There are two direct and meaningful directions for improvement that could address this issue: (I) Enhaneing the current RCF technique to be more robust to the presence of outliers; (2) Exploring a more reasonable multi-channel modeling technique within the RCF framework. We leave these challenges for future work and encourage the community to further research more robust and powerful periodic modeling techniques.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims in the abstract accurately reflect our contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of this work Section 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
25

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide complete experimental details in Appendix B.3
Additionally, we have shared the full reproducible code in an anonymous repository (link provided in the abstract).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
26

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide anonymous link to the code and describe how to reproduce the experimental results in the README file the code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cc.
cublic//uuides/Coeeetubii. watpssiinooollyyy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips. wttps/cccppplccggua.eeaeaeccuuep Sttelee//ooSSSSeissSioiaa www.ssssiicggcooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We describe the complete experimental details and hyperparameter choices in Appendix B.3
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the standard deviations of the results for our proposed method under different settings in Table
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
27

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We report the computational resource requirements of proposed method in Table
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httths/s/heerripa..cppbbbb///// iett
Answer: [Yes]
Justification:
Our research aligns with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
The paper focuses advancing the field of machine learning. While work may have various societal implications, we believe none are significant enough to warrant specific mention here.
Guidelines:
28

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
The code and datasets used in the paper are publicly available and properly credited.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
29

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We will make the code publicly available upon acceptance of the paper and provide detailed documentation.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
30

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "3akiOZRbHj", "reviewer": "Reviewer_SdEc", "summary": "This paper proposes a learnable Seasonal-Trend Decomposition method (CycleNet) to improve the prediction performance of current long-term multivariate time series forecasting models. Specifically, it firstly model the periodic patterns of sequences through globally shared recurrent cycles and then predicts the residual components of the modeled cycles. Extensive experiments are conducted to evaluate the proposed method.", "questions": "Please refer to the weaknesses.", "limitations": "Yes", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The proposed method is a model-agnostic solution applicable to different kinds of models.\n2. Although it is simple, it is able to achieve good performance improvement in many cases.\n3. Extensive experiments are conduct to evaluate the proposal.", "weaknesses": "1. It seems that the proposal (CycleNet) does not work well for complex datasets, e.g., Traffic. It is better to show more results on the same kind of datasets like the PEMS datasets used in the iTransformer paper.\n2. The Section 3 is not well written and lacks a lot of details, especially about the Learnable Recurrent Cycles. The authors may consider to reorganize Section 3 and Appendix A.1 to make Section 3 more clear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you for your valuable comment!**\n\n> **W1:** It seems that the proposal (CycleNet) does not work well for complex datasets, e.g., Traffic. It is better to show more results on the same kind of datasets like the PEMS datasets used in the iTransformer paper.\n\nThe current CycleNet did not achieve SOTA on the Traffic dataset because **it is a simple single-channel modeling method that uses the proposed RCF technique to achieve a balance of performance and efficiency.** Its backbone for prediction is only a single Linear layer or a dual-layer MLP. For scenarios like Traffic, which exhibit spatiotemporal characteristics and temporal lag characteristics, more complex multi-channel relationship modeling is indeed required.\n\n*As we know, traffic scenarios may experience sudden surges in flow (i.e., extreme points), and the surge at one intersection can affect the flow at other intersections over time.* The table below shows the average number of extreme points per channel, counted with Z-Score > 6:\n\n| Dataset    | Traffic | Electricity | Solar | ETTh1 |\n| ---------- | ------- | ----------- | ----- | ----- |\n| Avg. Count | 63.9    | 0.5         | 0     | 0     |\n\n*We can see that the Traffic dataset has significantly more extreme points than other datasets.* In this case, models that fully model inter-channel relationships, like iTransformer, can accurately predict the flow at other intersections after an extreme point appears. In contrast, univariate models or over-parameterized multivariate models are less capable in such situations. This is why in Table 2, iTransformer significantly outperforms other models as it can accurately predict the flow at other intersections after an extreme event.\n\n**However, it is also notable that, aside from iTransformer, CycleNet still significantly outperforms other models**. Especially, this result is achieved with CycleNet's backbone being simple linear and MLP layers. Therefore, returning to CycleNet's core contribution, which is exploring a simpler yet effective use of periodicity and establishing a method that balances performance and efficiency, it is undoubtedly successful.\n\nAdditionally, based on your suggestion, **we have added results of CycleNet on the PEMS datasets.** Here, it is essential to clarify that although PEMS and Traffic are both transportation datasets, PEMS has significantly fewer extreme points than Traffic:\n\n| Dataset    | Traffic | PEMS03 | PEMS04 | PEMS07 | PEMS08 |\n| ---------- | ------- | ------ | ------ | ------ | ------ |\n| Avg. Count | 63.9    | 0.9    | 0.1    | 3.5    | 4.8    |\n\n\nThus, in this case, CycleNet is expected to perform better on PEMS than on Traffic. Below are the MSE comparison results of CycleNet and other models in the scenario where the lookback length is 96 and the prediction horizon is 12:\n\n| Dataset | CycleNet/MLP | CycleNet/Linear | iTransformer | PatchTST | Crossformer | TimesNet | DLinear | RLinear |\n| ------- | ------------ | --------------- | ------------ | -------- | ----------- | -------- | ------- | ------- |\n| PEMS03  | **0.066**    | 0.080           | 0.071        | 0.099    | 0.090       | 0.085    | 0.122   | 0.126   |\n| PEMS04  | **0.078**    | 0.089           | **0.078**    | 0.105    | 0.098       | 0.087    | 0.148   | 0.138   |\n| PEMS07  | **0.062**    | 0.075           | 0.067        | 0.095    | 0.094       | 0.082    | 0.115   | 0.118   |\n| PEMS08  | 0.082        | 0.091           | **0.079**    | 0.168    | 0.165       | 0.112    | 0.154   | 0.133   |\n\n**In this case, CycleNet/MLP performs on par with iTransformer, which models multi-channel relationships. Even the CycleNet/Linear, with a single linear layer as the backbone, outperforms other deep nonlinear models.** Therefore, these results further validate the effectiveness of the proposed RCF technique and demonstrate that the CycleNet model achieves a balance of performance and efficiency. *We will include these analyses and full experimental results in the revised paper.*\n\n> **W2:** The Section 3 is not well written and lacks a lot of details, especially about the Learnable Recurrent Cycles. The authors may consider reorganizing Section 3 and Appendix A.1 to make Section 3 clearer.\n\nThank you for pointing out this issue. We will carefully optimize Section 3 in the revised paper to make it clearer for readers. Additionally, **we have supplemented it with a schematic figure** in *Figure 1 of the attached pdf*, which more intuitively describes the workflow of the recurrent cycle $Q$. *We will add this schematic figure to Section 3 of the revised paper.*\n\n**Thank you again for your valuable review, and we hope our response can address your concerns.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FR2G9XP2t0", "reviewer": "Reviewer_24Zo", "summary": "This paper presents a novel technique for improving the accuracy of multivariate long-term time series forecasting. The technique, called Residual Cycle Forecasting (RCF), involves learning the cyclical patterns of time series through recurrent cycles, which can be used as a pre-processing step for any forecasting model. The authors also propose CycleNet, a linear-based model that uses RCF to enhance its predictions. CycleNet first uses RevIn to account for distribution shift, then subtracts the learned RCF from the input data. The backbone of the model predicts the future residual, adds the learned RCF, and reverses RevIN from the outputs to obtain the final prediction. \n\nThe proposed method is evaluated on eight multivariate time series datasets and compared against several baselines. The results show competitive performance and resource consumption.", "questions": "## Baselines\n\nThe paper fails to properly compare the proposed CycleNet model with appropriate baselines.\n\nSince CycleNet is based on RevIN, it would be more complete to include other RevIN-based baselines such as RLinear [1] for linear-based models and for instance RevInformer (from the RevIN paper) for transformer-based solutions. Although PatchTST uses RevIN, including these additional baselines would avoid any doubt and provide a more comprehensive comparison.\n\nAdditionally, the paper misses important baselines such as LD [2], which also uses RevIn and also argues for using learnable decomposition rather than moving average. According to the published results, CycleNet appears to be behind LD. But it would be interesting to compare LD against backbone + RCF.\nPotential references to [3] is also missing.\n\n[1] https://arxiv.org/pdf/2305.10721\n\n[2] https://arxiv.org/abs/2402.12694\n\n[3] https://arxiv.org/pdf/2308.13386\n\nFinally, despite being describes as main baselines, why SparseTSF was not included in this paper? Especially, as SparseTSF seems to produce better results than proposal especially for ETT and not for Electricity and Traffic. What would be the reason for such differences?\n\nIncluding such comparisons is required to correctly position CycleNet with all the other baselines and further discuss why CycleNet offers better performance with some datasets and not others. Such discussion is important to further investigate the benefit of time series decomposition for forecasting tasks.\n\n## Visualization\nThe visualizations of the learned cycles are informative, but it is unclear whether these cycles change depending on the prediction horizon or lookback. If they do, it would be interesting to see the differences and whether they can explain the variation in performance for different prediction horizons.\n\nSince the cycles are learned along with the backbone model, there could be differences in the learned cycles for different experiments. It would be helpful to plot the cycles for different prediction lengths, such as 96 and 720, and for different lookback lengths, such as 48, 96, and 336. \n\nIn addition, the cycle may change depending on the backbone. It would be interesting to show the difference if any when RCF is used with iTransformer, DLinear, etc. This would allow readers to better understand whether there are any differences in the learned cycles and whether they are due to the parameters, backbone model or other factors.\n\nFigure 3(f) is surprising, and it is unclear whether it represents a household or an industrial building with specific working hours. In my opinion, it looks like there are only 3 main weekly patterns (the large pattern on the left seems to have ~50 time steps = 2 days) and weekends (one pattern seems to have ~24 time steps = 1 day). Therefore, it could be a factory or industrial building with specific working hours (6 days a week).\n\nIt would be interesting to discuss how RCF impacts previous backbones, such as DLinear. Specifically, it would be helpful to know how much DLinear's performance changes with the addition of RCF and plot it similarly to Figure 4.\n\nFinally, it is important to provide Figure 4 and Table 4 for other datasets to give readers a global view of the impact of RCF/CycleNet depending on the dataset and its nature. This would help avoid generalizations that could be incorrect based only on results depicted in the current version, especially for datasets such as Traffic and Solar Energy where the impact might be drastically different.\n\n## Discussion\n“Traffic dataset exhibits spatiotemporal characteristics and temporal lag characteristics, where the traffic flow at a certain detection point significantly affects the future values of neighboring detection points.” However, the authors do not explain why this is not an issue for the Solar-Energy dataset, where weather conditions at one location may slowly impact neighboring locations in the future. Can authors comment on that?\n\nRegarding the electricity consumption dataset, the authors state that \"a user’s electricity consumption thirty days ahead does not directly correlate with their consumption patterns in the past few days.\" However, it is possible that long-term habits may still influence consumption patterns, for instance with a monthly routine.\n\nThe authors suggest that \"the cycle length W depends on the a priori characteristics of the dataset and should be set to the maximum stable cycle within the dataset.\" However, they do not address how to account for yearly cycles, such as those found in weather datasets that repeat each year with the seasons or datasets influenced by human behavior, such as electricity consumption, which may increase in winter and summer due to heating and cooling needs. It would be helpful to clarify whether RCF is only suitable for mid-range stable cycles or if there is a way to account for longer cycles. This might need to be included in the limitations section.\n\n## Additional points\nThe paper lacks a computation cost study to determine the overhead imposed by RCF on the existing backbone in terms of memory and consumption. It is important to evaluate whether the gain in accuracy justifies the increase in complexity and associated cost.\n\n## Proof-read\n * “a single-layer Linear or a dual-layer MLP” maybe find another way to express it to avoid repeating too much (cf., abstract, introduction, etc.)\n * “surpassing complexly designed deep models”->  redundant, ”complex design” or “deep models”\n * “leveraging this discovery to enhance” should change this as the fact that time series have cycle is not new leveraging this knowledge is the innovation", "limitations": "Authors have discussed some limitations of their proposal and especially when each channel have different period cycle.", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Clear and concise writing style\n * Novel approach to time series decomposition\n * Comparison of performance and resource consumption with baseline methods\n * Ablation study and parameter impact analysis (e.g., $W$)\n * Easy-to-understand model design and components\n * Informative figures to illustrate model and results\n * Thorough discussion of results, including strengths and limitations\n * Code and data provided for reproducibility and transparency.", "weaknesses": "* Lack of clarity to specify which results are from the authors (reproduced or produced) and which ones are collected from previous papers (if so, which ones)\n * Incomplete comparison with existing time series decomposition baselines, such as LD, TDFNet, or SparseTSF (even though this one was in the related works)\n * Model was not compared to RLinear (also relying on RevIN)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you for your detailed and thoughtful review!**\n\n>  **W1:** Lack of clarity of the source of results.\n\nWe will clarify this in the main text of the revised paper. Previously, we clarified this in Appendix A.4.\n\n> **W2 & W3 & Q1: Baseline:** Add more appropriate baselines to correctly position the RCF technique.\n\nThank you for your reminder.\n\n(i) **Leddam (LD) and SparseTSF** are both accepted papers at ICML 2024. We timely noted the latter because it emphasizes the importance of periodicity, similar to our work. LD upgrades the Moving Average kernel (MOV) technique in STD to a weights-learnable module, while SparseTSF uses sparse techniques to decompose sequences. We will compare these two techniques with our proposed RCF technique.\n\n(ii) **TFDNet** extracts features in the Time-Frequency domain after using STD techniques to decompose sequences. Since it fundamentally uses regular MOV-based STD techniques, it may not be necessary to compare it directly here.\n\n(iii) **RLinear**: We will add a comprehensive comparison with it in the main text. In fact, the Linear and MLP in Table 4 are combined with RevIN, so they represent RLinear and RMLP, and CycleNet significantly outperforms them. Additionally, we thoroughly evaluated the impact of RevIN on CycleNet in Appendix Table 8.\n\n**In summary, we compared CycleNet/Linear (RCF+Linear), LDLinear (LD+Linear), DLinear (MOV+Linear), SparseTSF (Sparse+Linear), and Linear** in *Table 1 of the attached pdf*. To ensure fairness, we did not use RevIN since DLinear originally did not use it.\n\nIt can be observed that:\n\n(i) **CycleNet significantly outperforms other methods**, proving the superiority of RCF as a new STD technique.\n\n(ii) As a sparse prediction method, SparseTSF may rely more on longer lookback length and RevIN, thus performing poorly here.\n\n(iii) Most surprisingly, **the performance of LDLinear and DLinear is almost identical** in our setup. This result differs from the results reported in the LD paper (Table 4). Other researchers have also noted this phenomenon on the LD project's GitHub (Issue #1), but have not yet received a response from the authors. *To some extent, LD is essentially a weight-trainable MOV, so it is hard to achieve high performance gains compared to the original MOV.* We are not sure if there is any mistake here, so we will further confirm with the authors after the double-blind review phase.\n\n> **Q2: Visualization:** Add more plots under different configurations.\n\n**We have added these visualizations** in *Figure 2 of the attached pdf*. The basic configuration is Electricity-Lookback=96-Horizon=96-Backbone=Linear-W=168. It can be observed that:\n\n**(i) Horizon:** *The learned patterns remain almost unchanged as the horizon changes.* This indicates that the horizon length does not affect the learned pattern results.\n\n**(ii) Lookback:** The overall pattern remains unchanged as lookback changes. However, upon closer observation, *it can be seen that the learned pattern becomes smoother with increased lookback.* This is because a longer lookback provides the backbone with richer periodic information, thereby reducing the importance of the learned pattern component.\n\n**(iii) Backbone:** *The patterns change somewhat with different backbones.* When DLinear is the backbone, the learned patterns are smoother, as DLinear's decomposition technique itself extracts certain periodic features. When iTransformer is the backbone, the learned patterns differ more, as it additionally models multichannel relationships, so the learned periodic patterns may consider multichannel feature interactions. PatchTST's performance is more similar to Linear, as it is also a regular single-channel modeling method, only with stronger nonlinear learning capabilities compared to the Linear model.\n\n**(iv) Cycle length $W$:** When $W$ is 144 (the weekly cycle length for the Electricity dataset), the recurrent cycle $Q$ learns the complete periodic pattern, including weekly and daily cycles. When $W$ is set to 24 (the daily cycle length), the recurrent cycle $Q$ only learns the daily cycle pattern. When $W$ is set to 96 (four times the daily cycle length), the recurrent cycle $Q$ learns four repeated daily cycle models. However, when $W$ is set to 23 (without matching semantic meaning), the recurrent cycle $Q$ learns nothing, i.e., a straight line.\n\n> **Q3: Discussion and other points.**\n\n*Very sorry that the response here cannot cover every point you have mentioned **due to the text length limitation***. We will carefully incorporate your suggestions into the revision (e.g., providing more results of other datasets on Table 4 and Figure 4), and **we can further discuss any uncovered points during the discussion phase.**\n\n> **Q4: Additional points:** The computation cost of RCF on other Backbones.\n\n**In fact, the cost of RCF is fixed and independent of the Backbone.** The primary additional fixed overhead is the parameter quantity of $D×W$, and the CPU time required for alignments and repetitions of the recurrent cycles (~10 seconds per epoch in our experimental environment). **Overall, this additional overhead is very small compared to the other deep backbones.** We will supplement this quantitative analysis in Table 3 (i.e., the additional overhead introduced by RCF). By combining these overheads and the ablation results of other backbones in Table 4, it can be inferred whether the gain in accuracy justifies the increase in complexity and associated cost.\n\n> **Q5: Proof-read.**\n\nThank you, and we will revise these points in the revised paper.\n\n**Again, thank you for your thorough review, and we hope our responses address your concerns.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6zeDmNHfYg", "reviewer": "Reviewer_zzRb", "summary": "The paper introduces CycleNet, a novel time series forecasting method that enhances long-term prediction accuracy by explicitly modeling the inherent periodic patterns present in time series data. The core contribution of paper is introducing the Residual Cycle Forecasting (RCF) technique, which leverages learnable recurrent cycles to represent these periodic patterns and predicts the residuals, significantly improving upon the performance of existing models with reduced computational complexity. CycleNet demonstrates state-of-the-art results across various domains, such as electricity, weather, and energy forecasting, while offering over 90% reduction in parameter quantity, highlighting its efficiency and effectiveness in capturing long-term dependencies for accurate forecasting.", "questions": "Please refer to the weakness", "limitations": "Please refer to the weakness", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The author tries to model the period information explicitly in the time series prediction task, and the motivation is intuitive and reasonable.\n\n2. The method designed by the author is reasonable and closely related to motivation. Combined with the experimental results, the author gives a simple but effective method.\n\n3. The author exposes the code and gives a detailed description, which increases the reproducibility of the model.\n\n4. The limitations of this method are clearly discussed and the possible problems are pointed out.", "weaknesses": "1. In the introduction, the author's statement establishes a close relationship between long-term prediction and periodic information. In the absence of some experimental support, this is not rigorous. Periodic information may be useful for long-term forecasting in certain situations, but it is not appropriate for all tasks, nor is it the only important information that these tasks require. The author slightly obfuscates these to highlight the motivation of this article.\n\n2. The authors select data sets with different periodicity to illustrate the validity of the model. However, the authors only demonstrate the validity, and experiments can be added to show how the proposed method performs differently on periodically different data sets and discuss the underlying rules. At the same time, it is also worth showing how CycleNet performs on data sets where there is no obvious periodicity.\n\n3. Combined with the results in Table 2 and Table 4, the results predicted with Linear alone are even better than some well-designed methods, whether this is due to the particularity of the data set or some other reason.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you for your kind and careful review!**\n\n> **W1:** In the introduction, the author's statement establishes a close relationship between long-term prediction and periodic information. Etc.\n\n**In fact, periodic information is indeed one of the most important factors for achieving long-term forecasting.** Recent works have demonstrated that by effectively utilizing periodic information, it is possible to achieve near state-of-the-art prediction accuracy with fewer than 1,000 parameters [1]. This strongly underscores the importance of periodic information in long-term forecasting tasks. Additionally, the popular work DLinear has shown that a single-layer linear model can outperform many well-designed models in long-term forecasting tasks [2]. This is because a simple linear layer can robustly extract periodic information from the historical data of a single channel (evident from the clear periodic patterns in the weight distribution of these linear models) [3]. \n\n*Without periodicity, long-term forecasting becomes very challenging.* For instance, DLinear shows that on financial datasets, even the most advanced deep learning models cannot outperform simply copying the most recent data point [2]. \n\nThus, existing evidence highlights that the presence of periodicity in data is crucial for accurate long-term predictions. **Our paper builds on this premise, exploring a simple yet effective method of leveraging periodicity through RCF.** We acknowledge that periodic information is not the sole factor for accurate time series predictions. Other factors like short-term trends, multivariate dependencies, and seasonality are also critical, especially for short-term forecasting tasks. However, the core of this paper focuses on improving long-term forecasting performance by better utilizing periodicity in data. Therefore, this motivation and contribution here may be not in conflict with scenarios where other factors are more influential.\n\n> **W2:** The authors select data sets with different periodicity to illustrate the validity of the model. However, the authors only demonstrate the validity, and experiments can be added to show how the proposed method performs differently on periodically different data sets and discuss the underlying rules. At the same time, it is also worth showing how CycleNet performs on data sets where there is no obvious periodicity.\n\n*Sorry if we misunderstood your first point here.* We have already shown the performance of our model on datasets with different cycle lengths in Table 5 and Figure 3 of our submission, **indicating that when the model's hyperparameter $W$ is set correctly, the RCF technique significantly improves prediction accuracy, and the trainable recurrent cycle $Q$ can accurately learn the corresponding cycle patterns.**\n\nWe assume that your question is about how the proposed method performs differently with different cycle lengths and we have supplemented *Figure 2 (m-p) in the attached PDF* to visualize the learned recurrent cycle $Q$ when setting different cycle lengths $W$. We found that when $W$ is correctly set to 144 (the weekly cycle length of the Electricity dataset), $Q$ learns the complete cycle pattern, including weekly and daily cycles. When $W$ is set to 24 (the daily cycle length), $Q$ captures only the daily cycle. With $W$ set to 96 (four times the daily cycle), $Q$ learns four repeated daily cycles. However, with $W$ set to 23 (with no matching semantic cycle length), $Q$ learns nothing, resulting in a flat line. For datasets without obvious periodicity (e.g., Exchange-Rate dataset), the behavior of $Q$ is similar, learning a flat line. *We will further include these results in the revised paper.*\n\n> **W3:** Combined with the results in Table 2 and Table 4, the results predicted with Linear alone are even better than some well-designed methods, whether this is due to the particularity of the data set or some other reason.\n\nThe results in Table 2 and Table 4 cannot be directly compared because Table 2 reports results averaged across all prediction horizons $H ∈ {96, 192, 336, 720}$, whereas Table 4 presents results for specific horizons individually.\n\nHowever, if we average the results from Table 4, *the Linear model does indeed outperform some well-designed models*, such as Autoformer. This phenomenon can be traced back to the DLinear paper [2], **which demonstrated that a purely linear layer can outperform many well-designed Transformer models at the time**. DLinear's success is attributed to its channel-independent approach for multivariate forecasting, where each channel is modeled with a shared linear layer. This approach allows the model to focus on extracting historical information from individual channels, leading to more robust periodic information for long-term forecasting (as evidenced by the striped patterns in the learned weights of its linear layers). Previous models like Autoformer mixed information across multiple channels, making it difficult to extract robust periodic information. Since DLinear, many models have adopted the channel-independent approach for long-term forecasting, including PatchTST, FITS, SparseTSF, and our CycleNet.\n\n**Therefore, it is reasonable that a simple linear model can outperform some well-designed models in this context.**\n\n[1] Lin, Shengsheng, et al. \"SparseTSF: Modeling Long-term Time Series Forecasting with *1k* Parameters.\" In International Conference on Machine Learning, 2024.\n\n[2] Zeng, Ailing, et al. \"Are transformers effective for time series forecasting?.\" Proceedings of the AAAI conference on artificial intelligence. Vol. 37. No. 9. 2023.\n\n[3] Toner, William, and Luke Darlow. \"An Analysis of Linear Time Series Forecasting Models.\" In International Conference on Machine Learning, 2024.\n\n**Thank you again for your kind review, and we hope our response can address your concerns.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HlxCIfpP1G", "reviewer": "Reviewer_EqMa", "summary": "This paper proposes a novel and effective technique to enhance long-term time series forecasting. The proposed technique, Residual Cycle Forecasting (RCF), directly models periodic cycles with learnable parameters, decomposing the learning of time series into periodic cycles and residual components. The residual can be learned with a simple architecture like Linear or MLP, and this technique can be integrated into existing forecasting models. Results show significant improvement achieved by using this technique.", "questions": "1. Since Figure 2 depicts the entire CycleNet architecture, instance normalization should also be included.\n\n2. The description of alignment and repetition of $Q$ (Lines 152-157) can be further improved. The link to Algorithm 1 should be added, and a schematic figure (even in Appendix) would make it clearer to understand.\n\n3. How extreme points affect RCF in the Traffic dataset should be further explained (Line 255). Extreme points can also affect other methods. It is unclear why this would specifically deteriorate RCF.", "limitations": "See Weaknesses and Questions", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "* The proposed method is simple and effective. It not only improves accuracy but is also more parameter efficient and can be easily integrated into different backbones.\n* The paper is well-written with good clarity. Both the problem and analysis are presented clearly.\n* The experiments are conducted with high quality, providing extensive and comprehensive analysis of the proposed technique. The results are consistent with prior related work. Limitation of the proposed method is also clearly stated.", "weaknesses": "The notations are not strictly consistent. The notations for instance normalization (in Section 3.2 and Algorithm 2) are independent from the whole framework and not consistent with the previous problem definition.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you for your valuable comment!**\n\n> **W1:** The notations are not strictly consistent. The notations for instance normalization (in Section 3.2 and Algorithm 2) are independent from the whole framework and not consistent with the previous problem definition.\n\nThanks for pointing this out. **We will correct it to make the notations in the paper more consistent.** In the original version of our paper, we used simpler notations (e.g., $x_{in}$, $y_{out}$) in Section 3.2 and Algorithm 2 to help readers better understand the model's input-output workflow. However, this indeed led to inconsistency in the notations. *Therefore, we will properly revise these in the revised paper.*\n\n> **Q1:** Since Figure 2 depicts the entire CycleNet architecture, instance normalization should also be included.\n\nThanks, and **we will include instance normalization in Figure 2** to present the complete workflow of CycleNet.\n\n> **Q2:** The description of alignment and repetition of Q (Lines 152-157) can be further improved. The link to Algorithm 1 should be added, and a schematic figure (even in Appendix) would make it clearer to understand.\n\nThanks for your suggestion. In our submission, due to space limitations in the main text, we adopted a concise writing style. **We will further elaborate on the alignment and repetition of $Q$ in the revised paper**, including more textual explanations in the main text, adding the link to Algorithm 1, and most importantly, supplementing with an intuitive schematic figure to help readers better understand this process (as shown in ***Figure 1 of the attached pdf***).\n\n> **Q3:** How extreme points affect RCF in the Traffic dataset should be further explained (Line 255). Extreme points can also affect other methods. It is unclear why this would specifically deteriorate RCF.\n\nThe reason behind this is indeed complex. Firstly, the basic fact is that **there are indeed some outliers with extremely high values in the Traffic dataset.** For instance, the 12630 value of the 857th channel in the Traffic dataset, after global standard normalization, still exceeds 25, while the normal range of values for surrounding points is [-2, 2]. Secondly, **the fundamental working principle of RCF is to learn the historical average cycles in the dataset.** \n\nIn such cases, the average cycles learned in RCF can be affected by these significant outliers, *such as the mean of a certain point in the cycle being exaggerated.* Consequently, in each prediction process, the original sequence subtracts a locally exaggerated average cycle, resulting in an inaccurate residual component, thereby affecting the local point predictions within each cycle. The more inaccurate these local point predictions are, the larger the discrepancy between MSE and MAE, as MSE significantly amplifies the impact of a few large errors. **This explains why in Table 4, combining iTransformer with RCF decreases MAE but increases MSE, indicating overall prediction accuracy improvement but anomalies in local point predictions.**\n\nTherefore, models like iTransformer and GNN, which accurately model inter-channel relationships, are more suitable for scenarios with extreme points and temporal lag characteristics. For example, when a sudden traffic surge occurs at a certain junction, these models, having correctly modeled the spatiotemporal relationships, can accurately predict possible traffic surges at other junctions. In contrast, the current CycleNet only considers single-channel relationship modeling, thus being somewhat limited in this scenario.\n\n*However, **the core of CycleNet still lies in exploring a more effective periodic modeling approach and proposing a model that balances performance and efficiency.*** We believe that further addressing the issue of RCF being affected by extreme points and investigating how to incorporate channel relationship modeling within CycleNet in future work would be highly promising. *We will include these discussions in the revised paper.*\n\n **Thank you again for your careful review, and we hope our responses address your concerns.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**Dear AC and Reviewers,**

**Thank you very much for your time and effort in reviewing our submission.** The valuable comments provided are highly beneficial for improving the quality of our paper. 

In this paper, **we propose the RCF technique**, which *utilizes learnable recurrent cycles to explicitly model the inherent periodic patterns within time series data*, followed by predicting the residual components of the modeled cycles. The RCF technique significantly enhances the performance of basic (or existing) models. **CycleNet** (which *combines RCF with basic Linear or MLP models*) **achieves consistent state-of-the-art performance** across multiple domains and offers **significant efficiency advantages**.

Overall, the four reviewers highly recognize the contributions of our submission, including comments such as "**Novel approach"**, "**Simple and effective**", "**Thorough discussion of results**", and "**Well-written with good clarity**". At the same time, the four reviewers have provided specific suggestions to improve the quality of our paper:

- **Reviewer SdEc** suggested supplementing experimental results on the PEMS dataset, and **we have supplemented these results** and demonstrated that the proposed method **remains very effective in this scenario**. In addition, Reviewer SdEc also suggested enhancing the description of the core technique proposed, and we will meticulously improve this part of the description in the final paper, and supplement an intuitive illustration to further explain the proposed technique (as shown in *Figure 1 of the attached pdf*).
- **Reviewer 24Zo** suggested adding comparisons with recent related work, and we have included this comparison in *Table 1 of the attached pdf*, **demonstrating the superiority of our methods**. Reviewer 24Zo also suggested showing more visualization results under different configurations, and **we have supplemented these results** in *Figure 2 of the attached pdf*. Additionally, Reviewer 24Zo raised more specific discussions and many detailed suggestions, including more discussion and some proof-reading. We will carefully revise the paper according to these suggestions.
- **Reviewer zzRb** suggested supplementing how the proposed method performs differently with different cycle lengths. **We have included this part of the experiment and discussion**. Additionally, Reviewer zzRb raised some issues about the introduction and results, and we have thoroughly analyzed these in the response.
- **Reviewer EqMa** provided suggestions for optimizing the writing, including notation consistency, figure completeness, and supplementary schematic figures. We will revise the paper and **have included the required schematic figure** in *Figure 1 of the attached pdf*. Additionally, Reviewer EqMa raised the issue of how extreme points affect the RCF technique, and we have **conducted an in-depth analysis**.

**Finally, thank you again for your valuable review.** We hope our response can further address your concerns, and we will carefully revise the paper according to the review.

Sincerely,

The Authors of Submission 9084

Meta-review: This paper has been assessed by four knowledgeable reviewers who agreed on accepting it. Two of the reviewers gave it strong accept ratings, one marked it as straight accept and one as weak accept. The paper presents a novel yet simple approach to time series  decomposition that can be used for long term forecasting of time series. It is well written and well structured and avoids most of the common pitfalls in how the ideas are presented and supported with empirical evidence. The authors provided a thorough rebuttal and engaged the reviewers in focused discussions.
Decision: Accept (spotlight)