Title: CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns
Venue: NeurIPS 2024
Accepted: yes
Keywords: Time Series Forecasting

Time Series Analysis

Machine Learning
Abstract: The stable periodic patterns present in time series data serve as the foundation for conducting long-horizon forecasts. In this paper, we pioneer the exploration of explicitly modeling this periodicity to enhance the performance of models in long-term time series forecasting (LTSF) tasks. Specifically, we introduce the Residual Cycle Forecasting (RCF) technique, which utilizes learnable recurrent cycles to model the inherent periodic patterns within sequences, and then performs predictions on the residual components of the modeled cycles. Combining RCF with a Linear layer or a shallow MLP forms the simple yet powerful method proposed in this paper, called CycleNet. CycleNet achieves state-of-the-art prediction accuracy in multiple domains including electricity, weather, and energy, while offering significant efficiency advantages by reducing over 90% of the required parameter quantity. Furthermore, as a novel plug-and-play technique, the RCF can also significantly improve the prediction accuracy of existing models, including PatchTST and iTransformer. The source code is available at: https://github.com/ACAT-SCUT/CycleNet.

CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns

Abstract

The stable periodic patterns present in time series data serve as the foundation for conducting long-horizon forecasts. In this paper, we pioneer the exploration of explicitly modeling this periodicity to enhance the performance of models in long-term time series forecasting (LTSF) tasks. Specifically, we introduce the Residual Cycle Forecasting (RCF) technique, which utilizes learnable recurrent cycles to model the inherent periodic patterns within sequences, and then performs predictions on the residual components of the modeled cycles. Combining RCF with a Linear layer or a shallow MLP forms the simple yet powerful method proposed in this paper, called CycleNet. CycleNet achieves state-of-theart prediction accuracy in multiple domains including electricity, weather, and energy, while offering significant efficiency advantages by reducing over 90% of the required parameter quantity. Furthermore, as a novel plug-and-play technique, the RCF can also significantly improve the prediction accuracy of existing models, including PatchTST and iTransformer. The source code is available at: https://github.com/ACAT-SCUT/CycleNet.

1 Introduction

Time series forecasting (TSF) plays a crucial role in various domains such as weather forecasting, transportation, and energy management, providing insights for early warnings and facilitating proactive planning. Particularly, accurate predictions over long horizons (e.g., spanning several days or months) offer increased convenience, referred to as Long-term Time Series Forecasting (LTSF) [59, 56, 17, 42, 6]. However, the principle enabling long-horizon prediction lies in understanding the inherent periodicity within the data [32]. Unlike short-term forecasting, long-term predictions cannot rely solely on recent temporal information (including means, trends, etc.). For instance, a user's electricity consumption thirty days ahead not only correlates with their consumption patterns in the past few days.

In such cases, long-term dependencies, or in other words, underlying stable periodicity within the data, serve as the practical foundation for conducting long-term predictions [32]. This is why existing models emphasize their capability to extract features with long-term dependencies. Models like Informer [59], Autoformer [51], and PatchTST [40] utilize the Transformer's ability for long-distance modeling to address LTSF tasks. ModernTCN [38] employs large convolutional kernels to enhance TCNs' ability to capture long-range dependencies, and SegRNN [31] uses segment-wise iterations to

improve RNN methods' handling of long sequences. If a model can accurately capture long-range dependencies, it can precisely extract periodic patterns from historical long sequences, enabling more accurate long-horizon predictions.

However, if the purpose of constructing deep and complex models is solely to better extract periodic features from long-range dependencies, why not directly model the patterns? As illustrated in Figure 1, electricity data exhibits clear daily periodic patterns (in addition to possible weekly patterns). We can use a globally shared daily segment to represent the periodic pattern in electricity consumption. By repeating this daily segment N times, we can continuously represent the cyclic components of N days' electricity consumption sequences.

Based on the above motivation, we pioneer explicit modeling of periodic patterns in the data to enhance the model's performance on LTSF tasks in this paper. Specifically, we propose the Residual Cycle Forecasting (RCF) technique. It involves using learnable recurrent cycles to explicitly model the inherent periodic patterns within time series data, followed by predicting the residual components of the modeled cycles. Combining the RCF technique with either a single-layer Linear or a dual-layer MLP results in CycleNet , a simple yet powerful method. CycleNet achieves consistent state-of-the-art performance across multiple domains and offers significant efficiency advantages.

In summary, this paper contributes:

We identify the presence of shared periodic patterns in long-horizon forecasting domains and propose explicit modeling of these patterns to enhance the model's performance on LTSF tasks.

Technically, we introduce the RCF technique, which utilizes learnable recurrent cycles to explicitly model the inherent periodic patterns within time series data, followed by predicting the residual components of the modeled cycles. The RCF technique significantly enhances the performance of basic (or existing) models.

Applying RCF with a Linear layer or a shallow MLP forms the proposed simple yet powerful method, called CycleNet. CycleNet achieves consistent state-of-the-art performance across multiple domains and offers significant efficiency advantages.

2 Related work

In fact, utilizing periodic information to enhance model prediction accuracy is not a novel concept. Numerous studies, in particular, have introduced a series of Seasonal-Trend Decomposition (STD) techniques that allow models to better leverage periodic information. Popular models such as Autoformer [51], FEDformer [60], and DLinear [56] utilize the classical STD approach to decompose the original time series into two equally sized subsequences: seasonal and trend components, which are then modeled independently. These classical STD methods typically use a basic moving average (MOV) kernel to perform a sliding aggregation to obtain the trend component. Recently, Leddam [55] proposed replacing the traditional MOV kernel in STD with a Learnable Decomposition (LD) kernel, leading to improved performance. Additionally, DEPTS [8] treats the periodicity of sequences as a parameterized function with respect to time, and learns periodic and residual components layer-wise through its periodic and local blocks. SparseTSF [32], another recent work, utilizes cross-period sparse forecasting technique to decouple cycles and trends, achieving impressive performance at extremely low cost.

The RCF technique proposed in this paper can essentially be considered a type of STD method. The key difference from existing techniques lies in its explicit modeling of global periodic patterns within independent sequences using learnable recurrent cycles. The proposed RCF technique is conceptually simple , computationally efficient , and yields significant improvements in prediction accuracy. The

time

further proposed CycleNet, which combines the RCF technique with a simple backbone, is a Linearor MLP-based model that is simple, efficient, and powerful for time series forecasting. To correctly position CycleNet, we have provided a detailed review of the development of different categories of time series forecasting methods (including Transformer-based, RNN-based, etc.) in Appendix A.

3 CycleNet

Given a time series X with D variables or channels, the objective of time series forecasting is to predict future horizons H steps ahead based on past L observations, mathematically represented as f : x t -L +1: t ∈ R L × D → ¯ x t +1: t + H ∈ R H × D . In fact, the inherent periodicity within time series is fundamental for accurate prediction, particularly when forecasting over large horizons, such as 96-720 steps (corresponding to several days or months). To enhance the model's performance on long-term prediction tasks, we propose the Residual Cycle Forecasting (RCF) technique. It combines a Linear layer or a shallow MLP to form a simple yet powerful method CycleNet, as illustrated in Figure 2, with detailed pseudocode provided in Appendix B.1.

ҧ

Linear

or

MLP

Restore Cycle

Learnable Recurrent Cycles

𝑄 ∈ ℝ 𝐷×𝑊

Align and Repeat

Align and Repeat

𝑥𝑡-𝐿+1:𝑡

𝑐𝑡-𝐿+1:𝑡

𝑥′

𝑡-𝐿+1:𝑡

𝑥′ 𝑡+1:𝑡+𝐻

∈ ℝ 𝐷×𝐻

𝑐𝑡+1:𝑡+𝐻

∈ ℝ 𝐷×𝐻

𝑥𝑡+1:𝑡+𝐻

∈ ℝ 𝐷×𝐻

/

1

√

M

ҧ

3.1 Residual cycle forecasting

The RCF technique comprises two steps: the first step involves modeling the periodic patterns of sequences through learnable recurrent cycles within independent channels, and the second step entails predicting the residual components of the modeled cycles.

Periodic patterns modeling Given D channels with a priori cycle length W , we first generate learnable recurrent cycles Q ∈ R W × D , all initialized to zeros. These recurrent cycles are globally shared within channels, meaning that by performing cyclic replications, we can obtain cyclic components C of the sequence X of the same length. These recurrent cycles Q of length W undergo gradient backpropagation training along with the backbone module for prediction, yielding learned representations (distinct from the originally initialized zeros) that unveil the internal cyclic patterns within the sequence.

Here, the cycle length W depends on the a priori characteristics of the dataset and should be set to the maximum stable cycle within the dataset. Considering that scenes requiring long-term predictions usually exhibit prominent, explicit cycles (e.g., electrical consumption and traffic data exhibit clear daily and weekly cycles), determining the specific cycle length is available and straightforward. Additionally, the dataset's cycles can be further examined through autocorrelation functions (ACF) [39], as revealed in Appendix B.2.

Residual forecasting Predictions made on the residual components of the modeled cycles, termed residual forecasting, are as follows:

Remove the cyclic components c t -L +1: t from the original input x t -L +1: t to obtain residual components x ′ t -L +1: t .

Pass x ′ t -L +1: t through the backbone to obtain predictions for the residual components, ¯ x ′ t +1: t + H .

Add the predicted residual components ¯ x ′ t +1: t + H to the cyclic components c t +1: t + H to obtain ¯ x t +1: t + H .

It is important to note that, since the cyclic components C are virtual sequences derived from the cyclic replications of Q , we cannot directly obtain the aforementioned sub-sequences c t -L +1: t and c t +1: t + H . Therefore, as illustrated in Figure 3, appropriate alignments and repetitions of the recurrent cycles Q are needed to obtain equivalent sub-sequences: (i) Left-shift Q by t mod W positions to obtain Q ( t ) . Here, t mod W can be viewed as the relative positional index of the current sequence sample within Q . (ii) Repeat Q ( t ) ⌊ L/W ⌋ times and concatenate Q ( t ) 0: L mod W . Mathematically, these two equivalent subsequences can be represented as:

𝑄 ∈ ℝ 𝐷×𝑊

𝑐𝑡-𝐿+1:𝑡

𝑄 (𝑡)

∈ ℝ 𝐷×𝑊

𝑄 (𝑡+𝐿)

∈ ℝ 𝐷×𝑊

𝐿/𝑊 ×

W

𝐿 mod 𝑊

𝑡 mod 𝑊

(𝑡 + 𝐿) mod 𝑊

𝐻/𝑊 ×

W

𝐻 mod 𝑊

2. Repeat and Concat

2. Repeat and Concat

𝑐𝑡+1:𝑡+𝐻

∈ ℝ 𝐷×𝐻

1. Align (Roll)

1. Align (Roll)

Backbone The original prediction task is transformed into cyclic residual component modeling, which can serve as normal sequence modeling. Therefore, any existing time series forecast model can be employed as a backbone. In this paper, our aim is to propose and examine a method for enhancing time series prediction by explicitly modeling cycles (i.e., RCF). Thus, we opt for the most basic backbone, namely a single-layer Linear and a dual-layer MLP, forming our simple yet powerful methods, CycleNet/Linear and CycleNet/MLP. Herein, each channel utilizes the same backbone with parameter sharing for modeling, which is also referred to as the Channel Independent strategy [13].

3.2 Instance normalization

The statistical properties of time series data, such as the mean, often vary over time, which is referred to as distributional shifts. This can lead to poor performance of models trained on historical training sets when applied to future data. To address this issue, recent research has introduced Instance Normalization strategies like RevIN [45, 22, 26]. Mainstream approaches such as iTransformer [37], PatchTST [40], and SparseTSF [32] have widely adopted similar techniques to enhance performance. To improve the robustness of CycleNet, we also incorporate a similar optional strategy (see the full ablation study in Appendix C.4). Specifically, we remove the varying statistical properties from the model's internal representations outside of CycleNet's input and output steps:

4 Experiments

4.1 Setup

Datasets We utilized widely adopted benchmark datasets including the ETT series [59], Weather, Traffic, Electricity, and Solar-Energy [24]. Preprocessing operations on the datasets, such as dataset splitting and normalization methods, remained consistent with prior works (e.g., Autoformer [51], iTransformer [37], etc.).

The information of the datasets is shown in Table 1. Note that these datasets all exhibit stable cyclic patterns, such as daily and weekly, which form the realistic basis for performing long-horizon forecasting. Combined with the sampling frequency of the datasets, we can infer the maximum cycle length of the datasets, such as 24 for ETTh1 and 168 for Electricity. These manually inferred cycle lengths can be further confirmed through the ACF analysis, details of which are provided in Appendix B.2. The hyperparameter W of CycleNet is set by default to match the cycle length in Table 1.

Baselines We compared CycleNet against state-of-the-art models in recent years, including iTransformer [37], PatchTST [40], Crossformer [58], TiDE [5], TimesNet [52], DLinear [56], SCINet [34], FEDformer [60], Autoformer [51]. To comprehensively evaluate CycleNet's performance, the Mean Squared Error (MSE) and Mean Absolute Error (MAE) metrics were employed.

Environments All experiments in this paper were implemented using PyTorch [41], trained using the Adam [23] optimizer, and executed on a single NVIDIA GeForce RTX 4090 GPU with 24 GB memory.

4.2 Main results

Table 2 shows the comparison results of CycleNet with other models on multivariate LTSF tasks. Overall, CycleNet achieves state-of-the-art performance (except for the Traffic dataset), with CycleNet/MLP ranking first overall, and CycleNet/Linear ranking second overall. Due to the nonlinear

where µ and σ represent the mean and standard deviation of the input window, respectively, and ϵ is a small constant for numerical stability. This method aligns with the RevIN version that excludes learnable affine parameters [22].

3.3 Loss function

To remain consistent with current mainstream methods, CycleNet defaults to using Mean Squared Error (MSE) as the loss function to ensure fair comparison with other methods, formulated as:

mapping capability of MLP compared to Linear, CycleNet/MLP performs better on high-dimensional datasets such as Electricity and Solar-Energy (i.e., datasets with more than 100 channels). In summary, with the support of the RCF technique, even a very simple and basic model (i.e., Linear and MLP) can achieve the current best performance, surpassing other deep models. This fully demonstrates the advantages of the RCF technique.

Furthermore, we can observe that CycleNet's performance on the Traffic dataset is inferior to iTransformer, which models multivariate relationships in time series data using an inverted Transformer. This is because the Traffic dataset exhibits spatiotemporal characteristics and temporal lag characteristics, where the traffic flow at a certain detection point significantly affects the future values of neighboring detection points. In such cases, modeling sufficient inter-channel relationships is necessary, and iTransformer accomplishes this. In contrast, CycleNet independently models the temporal dependencies of each channel, hence it suffers a disadvantage in this scenario. However, CycleNet still significantly outperforms other baselines on the Traffic dataset, demonstrating the competitiveness of CycleNet. Additionally, we have included more analysis of CycleNet in traffic scenarios in Appendix C.5, including a full comparison of results on the PEMS datasets.

4.3 Efficiency analysis

The proposed RCF technique, as a plug-andplay module, requires minimal overhead, needing only additional W × D learnable parameters and no additional Multiply-Accumulate Operations (MACs). The backbones of CycleNet, namely single-layer Linear and duallayer MLP, are also significantly lightweight compared to other multi-layer stacked models. Table 3 demonstrates the efficiency comparison between CycleNet and other mainstream models, where CycleNet shows significant advantages. Particularly, compared to iTransformer, which also possesses strong capabilities in modeling long-term dependencies and nonlinear learning, CycleNet/MLP has over ten times fewer parameters and MACs. As for CycleNet/Linear, which shares the same single-layer linear backbone as

DLinear, it also has fewer parameters and MACs. However, in terms of training speed, DLinear is still faster than CycleNet/Linear. This is because the RCF technique requires aligning the recurrent cycles with each data sample, which incurs additional CPU time. Overall, considering the significant improvement in prediction accuracy brought by the RCF technique, CycleNet achieves the best balance between performance and efficiency.

4.4 Ablation study and analysis

Effectiveness of RCF To investigate the effectiveness of RCF, we conducted comprehensive ablation experiments on two datasets with significant periodicity: Electricity and Traffic. The results are shown in Table 4.

Firstly, when combining the basic Linear and MLP backbones (both utilizing instance normalization by default) with the RCF technique, a significant improvement in prediction accuracy (approximately 10% to 20%) is observed. This demonstrates that the success of CycleNet is largely attributed to the RCF technique rather than the backbones themselves or the instance normalization strategy. Overall, the performance of MLP is stronger than that of Linear, regardless of whether the RCF technique is applied. This indicates that non-linear mapping capability is necessary when modeling high-dimensional datasets with the channel-independent strategy (sharing parameters across each channel), aligning with previous research findings [26].

Secondly, we further verified whether RCF can enhance the prediction accuracy of existing models, as RCF is essentially a plug-and-play flexible technique. It is observed that incorporating RCF still improves the performance of existing complex designed, deep stacked models (approximately 5% to 10%), such as PatchTST [40] and iTransformer [37]. Even for DLinear, which already employs the classical MOV-based STD technique, RCF was able to provide an improvement of approximately 20%. This further indicates the effectiveness and portability of RCF.

However, an interesting phenomenon was observed: although the MAE decreases when PatchTST and iTransformer are combined with RCF, the MSE increases. The most important reason behind this is that there are extreme points in the Traffic dataset that could affect the effectiveness of RCF, which fundamentally relies on learning the historical average cycles in the dataset. We further analyze this phenomenon in detail in Appendix C.5 and suggest potential directions for improving the RCF technique.

Comparison of different STD techniques The proposed RCF technique is essentially a more powerful STD approach. Unlike existing methods that decompose the periodic (seasonal) component from a limited look-back window, RCF learns the global periodic component from the training set. Here, we compare the effectiveness of RCF with existing STD techniques, using a pure Linear model as the backbone (without applying any instance normalization strategies). The comparison includes LD from Leddam [55], MOV from DLinear [56], and Sparse technique from SparseTSF [32]. As shown in Table 5, RCF significantly outperforms other STD methods, particularly on datasets with strong periodicity, such as Electricity and Solar-Energy. In contrast, the other STD methods did not show significant advantages over the pure Linear model.

There are several reasons for this. First, MOV and LD-based STD methods achieve trend estimation by sliding aggregation within the look-back window, which suffers from inherent issues [27, 26]: (i) The sliding window of the moving average needs to be larger than the maximum period of the seasonal component; otherwise, the decomposition may be incomplete (especially when the period length exceeds the look-back sequence length, making decomposition potentially impossible).

(ii) Zero-padding is required at the edges of the sequence samples to obtain equally sized moving average sequences, leading to distortion of the sequence edges. As for the Sparse technique, being a lightweight decomposition method, it relys more on longer look-back windows and instance normalization strategies to ensure adequate performance.

Additionally, these methods that decouple trend and seasonality within the look-back window are essentially equivalent to unconstrained or weakly constrained linear regression [44], which means that after full training convergence, linear-based models combined with these methods are theoretically equivalent to pure linear models. In contrast, the periodic components obtained by the RCF technique are globally estimated from the training set, allowing it to surpass the limitations of a finite-

Table 5: Comparison of different STD techniques. To directly compare the effects of STD, the configuration used here is consistent with that of DLinear [56], with a sufficient look-back window length of 336 and no additional instance normalization strategies. Thus, CLinear here refers to CycleNet/Linear without RevIN. The reported results are averaged across all prediction horizons of H ∈ { 96 , 192 , 336 , 720 } , with full results available in Appendix C.3.

length look-back window, and thus, its capabilities extend beyond standard linear regression.

Impact of hyperparame-

ter W The hyperparameter W determines the length of the learnable recurrent cycles Q in the RCF technique. In principle, it must match the maximum primary cycle length in the data to correctly model the periodic patterns of the sequence. We investigate the

Table 6: Performance of the CycleNet/Linear model with varied W . The forecast horizon is set as 96.

performance of the CycleNet/Linear model under different settings of W for different datasets in Table 6. When correctly setting the hyperparameter W to the max cycle length of the dataset (i.e., the cycle length pre-inferred in Table 1), RCF can play a significant role, yielding a large performance gap compared to the cases when it is not correctly set. This indicates the necessity of inferring and setting the correct W for RCF to function properly. Furthermore, when W is incorrectly set, the model's performance is almost the same as when RCF is not used at all. This suggests that even in the worst-case scenario, RCF does not bring significant negative effects.

Visualization of the learned periodic patterns The purpose of the RCF technique is to utilize the learnable recurrent cycles Q (initialized to zero) to model the periodic patterns in time series data. After co-training with the backbone, the recurrent cycles can represent the inherent periodic patterns of the sequence. Figure 4 illustrates the different periodic patterns learned from different datasets and channels. For example, Figure 4(c) shows the daily operating pattern of solar photovoltaic generation, while Figure 4(d) displays the weekly operating pattern of traffic flow, featuring peak traffic in the mornings on weekdays. These periodic patterns learned from the global sequence provide important supplementary information to the prediction model, especially when the length of the look-back window is limited and may not provide sufficient cyclic information when the cycle length is long.

Furthermore, although the cycle length is the same for different channels within the same dataset, the specific periodic patterns differ, as shown in Figure 4(e-h). Particularly, Figure 4(f) demonstrates the intermittent periodicity of household electricity consumption on weekdays, while others exhibit relatively uniform weekday patterns in their respective channels. This highlights the necessity of separately modeling the periodic patterns for each channel.

0

20

40

60

80

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

(a) ETTm1, 7th

0

20

40

60

80

100

120

140

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

(b) Weather, 7th

0

20

40

60

80

100

120

140

-0.5

0.0

0.5

1.0

1.5

(c) Solar-Energy, 137th

0

25

50

75

100

125

150

175

-1.0

-0.5

0.0

0.5

1.0

1.5

(d) Traffic, 607th

0

25

50

75

100

125

150

175

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

(e) Electricity, 311st

0

25

50

75

100

125

150

175

-1.0

-0.5

0.0

0.5

1.0

1.5

(f) Electricity, 318th

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

(g) Electricity, 320th

0

25

50

75

100

125

150

175

-1.0

-0.5

0.0

0.5

1.0

(h) Electricity, 321st

In conclusion, these findings demonstrate that the RCF technique can effectively learn the inherent periodic patterns in time series data, serving as a crucial explanatory factor contributing to the stateof-the-art performance of CycleNet. Additionally, we have included further analysis in Appendix C.1, showcasing the learned periodic patterns of RCF under different configurations to better illustrate how RCF operates.

48

96

192

336

528

720

Lookback Length

0.12

0.14

0.16

0.18

0.20

0.22

0.24

0.26

MSE

DLinear

PatchTST

iT ransformer

CycleNet/Linear

CycleNet/MLP

(a) Electricity

48

96

192

336

528

720

Lookback Length

0.4

0.5

0.6

0.7

0.8

0.9

MSE

DLinear

PatchTST

iT ransformer

CycleNet/Linear

CycleNet/MLP

(b) Traffic

Performance with varied look-back length The look-back length determines the richness of historical information that can be utilized. Theoretically, the larger it is, the better the model performance should be, especially for models capable of capturing long-term dependencies. Figure 5 shows the performance of different models under different look-back lengths. It can be observed that CycleNet, as well as representatives of current state-of-the-art models such as iTransformer [37], PatchTST [40], and DLinear [56], all achieve better performance with longer look-back lengths. This indicates that these models all possess strong capabilities in modeling long-term dependencies.

It is worth highlighting that (i) on the Electricity dataset, CycleNet outperforms current state-of-theart models at any prediction length; (ii) on the Traffic dataset, CycleNet still falls short compared to powerful existing multivariate forecasting models, such as iTransformer. This indicates that in scenarios with strong periodicity but without additional spatiotemporal relationships, fully leveraging the periodic components is sufficient to achieve high-accuracy predictions. However, in more complex scenarios that require thorough modeling of relationships between variables, a simple channel-independent strategy combined with a basic backbone, like CycleNet, still struggles to fully meet the demands. Therefore, in Appendix C.5, we further analyze the current limitations of the current RCF technique in spatiotemporal scenarios (such as the traffic domain) and and point out

potential directions for future improvements. Finally, we also provide a comparison of CycleNet with existing models on full datasets using longer look-back windows in Appendix C.2.

5 Discussion

Potential limitations CycleNet demonstrates its efficacy in LTSF scenarios characterized by prominent and explicit periodic patterns. However, there are several potential limitations of CycleNet that warrant discussion here:

Unstable cycle length: CycleNet may not be suitable for datasets where the cycle length (or frequency) varies over time, such as electrocardiogram (ECG) data, because CycleNet can only learn a fixed-length cycle.

Varying cycle lengths across channels: When different channels within a dataset exhibit cycles of varying lengths, CycleNet may encounter challenges because it defaults to modeling all channels with the same cycle length W . Given CycleNet's channel-independent modeling strategy, one potential solution is to pre-process the dataset by splitting it based on cycle lengths or to independently model each channel as a separate dataset.

Impact of outliers: If the dataset contains significant outliers, CycleNet's performance may be affected. This is because the fundamental working principle of RCF is to learn the historical average cycles in the dataset. When significant outliers exist, the mean of a certain point in the cycle learned by RCF can be exaggerated, leading to inaccurate estimation of both the periodic and residual components, which subsequently impacts the prediction process.

Long-range cycle modeling: The RCF technique is effective for modeling mid-range stable cycles (e.g., daily or weekly). However, considering longer dependencies (such as yearly cycles) presents a more challenging task for the RCF technique. Although, in theory, CycleNet's W can be set to a yearly cycle length to model annual cycles, the biggest difficulty lies in collecting sufficiently long historical data to train a complete yearly cycle, which might require decades of data. In this case, future research needs to develop more advanced techniques to specifically address long-range cycle modeling.

Future work: further modeling inter-channel relationships The RCF technique enhances the model's ability to model the periodicity of time series data but does not explicitly consider the relationships between multiple variables. In some spatio-temporal scenarios where spatial and temporal dependencies between variables exist, these relationships are crucial. For example, recent studies such as iTransformer [37] and SOFTS [12] indicate that appropriately modeling inter-channel relationships can improve performance in traffic scenarios. However, directly applying the RCF technique to iTransformer does not lead to significant improvement (at least for the MSE metric), as demonstrated in Table 4. We believe that devising a more reasonable multivariate modeling approach that combines CycleNet could be promising and valuable, and we leave it for future exploration.

6 Conclusion

This paper reveals the presence of inherent periodic patterns in time series data and pioneers the exploration of explicitly modeling this periodicity to enhance the performance of time series forecasting models. Technically, we propose the Residual Cycle Forecasting (RCF) technique, which models the shared periodic patterns in sequences through recurrent cycles and predicts the residual cyclic components via a backbone. Furthermore, we introduce the simple yet powerful LTSF methods CycleNet/Linear and CycleNet/MLP, which combine single-layer Linear and dual-layer MLP respectively with the RCF technique. Extensive experiments demonstrate the effectiveness of the RCF technique, and CycleNet as a novel and simple method achieves state-of-the-art results with significant efficiency advantages. The findings in this paper underscore the importance of periodicity as a key characteristic for accurate time series prediction, which should be given greater emphasis in the modeling process. Finally, integrating CycleNet with effective inter-channel relationship modeling methods serves as a promising and valuable future research direction.

Acknowledgments

This work is supported by Guangdong Major Project of Basic and Applied Basic Research (2019B030302002), National Natural Science Foundation of China (62072187), Guangzhou Development Zone Science and Technology Project (2023GH02) and the Major Key Project of PCL, China under Grant PCL2023A09.

References

Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 , 2018.

Shane Bergsma, Tim Zeyl, and Lei Guo. Sutranets: Sub-series autoregressive networks for long-sequence, probabilistic forecasting. Advances in Neural Information Processing Systems , 36:30518-30533, 2023.

Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. Tempo: Prompt-based generative pre-trained transformer for time series forecasting. arXiv preprint arXiv:2310.04948 , 2023.

Cristian Challu, Kin G Olivares, Boris N Oreshkin, Federico Garza Ramirez, Max Mergenthaler Canseco, and Artur Dubrawski. Nhits: Neural hierarchical interpolation for time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 6989-6997, 2023.

Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan Mathur, Rajat Sen, and Rose Yu. Longterm forecasting with tide: Time-series dense encoder. arXiv preprint arXiv:2304.08424 , 2023.

Jinliang Deng, Feiyang Ye, Du Yin, Xuan Song, Ivor Wai-Hung Tsang, and Hui Xiong. Parsimony or capability? decomposition delivers both in long-term time series forecasting. 2024. URL https://api.semanticscholar.org/CorpusID:267068391 .

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

Wei Fan, Shun Zheng, Xiaohan Yi, Wei Cao, Yanjie Fu, Jiang Bian, and Tie-Yan Liu. Depts: Deep expansion learning for periodic time series forecasting. arXiv preprint arXiv:2203.07681 , 2022.

Jean-Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. Unsupervised scalable representation learning for multivariate time series. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/ paper_files/paper/2019/file/53c6de78244e9f528eb3e1cda69699bb-Paper.pdf .

Zeying Gong, Yujin Tang, and Junwei Liang. Patchmixer: A patch-mixing architecture for long-term time series forecasting. arXiv preprint arXiv:2310.00655 , 2023.

Nate Gruver, Marc Finzi, Shikai Qiu, and Andrew G Wilson. Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems , 36, 2024.

Lu Han, Xu-Yang Chen, Han-Jia Ye, and De-Chuan Zhan. Softs: Efficient multivariate time series forecasting with series-core fusion. arXiv preprint arXiv:2404.14197 , 2024.

Lu Han, Han-Jia Ye, and De-Chuan Zhan. The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting. IEEE Transactions on Knowledge and Data Engineering , 2024.

Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16000-16009, 2022.

Haowen Hou and F Richard Yu. Rwkv-ts: Beyond traditional recurrent neural network for time series tasks. arXiv preprint arXiv:2401.09093 , 2024.

Qihe Huang, Lei Shen, Ruixin Zhang, Jiahuan Cheng, Shouhong Ding, Zhengyang Zhou, and Yang Wang. Hdmixer: Hierarchical dependency with extendable patch for multivariate time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 12608-12616, 2024.

Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. Crossgnn: Confronting noisy multivariate time series via cross interaction refinement. Advances in Neural Information Processing Systems , 36, 2024.

Qihe Huang, Zhengyang Zhou, Kuo Yang, Gengyu Lin, Zhongchao Yi, and Yang Wang. Leret: Language-empowered retentive network for time series forecasting. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24 , 2024.

Yuxin Jia, Youfang Lin, Xinyan Hao, Yan Lin, Shengnan Guo, and Huaiyu Wan. Witran: Water-wave information transmission and recurrent acceleration network for long-range time series forecasting. Advances in Neural Information Processing Systems , 36, 2024.

Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728 , 2023.

Ming Jin, Yifan Zhang, Wei Chen, Kexin Zhang, Yuxuan Liang, Bin Yang, Jindong Wang, Shirui Pan, and Qingsong Wen. Position: What can large language models tell us about time series analysis. In Forty-first International Conference on Machine Learning , 2024.

Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations , 2021.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval , pages 95-104, 2018.

Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in neural information processing systems , 32, 2019.

Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305.10721 , 2023.

Zhe Li, Zhongwen Rao, Lujia Pan, Pengyun Wang, and Zenglin Xu. Ti-mae: Self-supervised masked time series autoencoders. arXiv preprint arXiv:2301.08871 , 2023.

Zhe Li, Zhongwen Rao, Lujia Pan, and Zenglin Xu. Mts-mixers: Multivariate time series forecasting via factorized temporal and channel mixing. arXiv preprint arXiv:2302.04501 , 2023.

Bryan Lim, Sercan Ö Arık, Nicolas Loeff, and Tomas Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting , 37(4): 1748-1764, 2021.

Shengsheng Lin, Weiwei Lin, Wentai Wu, Songbo Wang, and Yongxiang Wang. Petformer: Long-term time series forecasting via placeholder-enhanced transformer. arXiv preprint arXiv:2308.04791 , 2023.

Shengsheng Lin, Weiwei Lin, Wentai Wu, Feiyu Zhao, Ruichao Mo, and Haotong Zhang. Segrnn: Segment recurrent neural network for long-term time series forecasting. arXiv preprint arXiv:2308.11200 , 2023.

Shengsheng Lin, Weiwei Lin, Wentai Wu, Haojun Chen, and Junjie Yang. Sparsetsf: Modeling long-term time series forecasting with 1k parameters. In Forty-first International Conference on Machine Learning , 2024.

Haoxin Liu, Zhiyuan Zhao, Jindong Wang, Harshavardhan Kamarthi, and B Aditya Prakash. Lstprompt: Large language models as zero-shot time series forecasters by long-short-term prompting. arXiv preprint arXiv:2402.16132 , 2024.

Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems , 35:5816-5828, 2022.

Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International conference on learning representations , 2021.

Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems , 35:9881-9893, 2022.

Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations , 2024.

Donghao Luo and Xue Wang. Moderntcn: A modern pure convolution structure for general time series analysis. In The Twelfth International Conference on Learning Representations , 2024.

Henrik Madsen. Time series analysis . CRC Press, 2007.

Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations , 2023.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems , 32, 2019.

Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. Tfb: Towards comprehensive and fair benchmarking of time series forecasting methods. Proc. VLDB Endow. , 17(9):23632377, 2024.

David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International journal of forecasting , 36(3): 1181-1191, 2020.

William Toner and Luke Nicholas Darlow. An analysis of linear time series forecasting models. In Forty-first International Conference on Machine Learning , 2024.

Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 , 2016.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

Huiqiang Wang, Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. Micn: Multi-scale local and global context modeling for long-term series forecasting. In The Eleventh International Conference on Learning Representations , 2022.

Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and JUN ZHOU. Timemixer: Decomposable multiscale mixing for time series forecasting. In The Twelfth International Conference on Learning Representations , 2024.

Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transformers in time series: A survey. arXiv preprint arXiv:2202.07125 , 2022.

Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Etsformer: Exponential smoothing transformers for time-series forecasting. arXiv preprint arXiv:2202.01381 , 2022.

Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural information processing systems , 34:22419-22430, 2021.

Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations , 2023.

Zhijian Xu, Ailing Zeng, and Qiang Xu. Fits: Modeling time series with 10 k parameters. In The Twelfth International Conference on Learning Representations , 2024.

Hao Xue and Flora D Salim. Promptcast: A new prompt-based learning paradigm for time series forecasting. IEEE Transactions on Knowledge and Data Engineering , 2023.

Guoqi Yu, Jing Zou, Xiaowei Hu, Angelica I Aviles-Rivero, Jing Qin, and Shujun Wang. Revitalizing multivariate time series forecasting: Learnable decomposition with inter-series dependencies and intra-series variations modeling. In Forty-first International Conference on Machine Learning , 2024.

Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence , volume 37, pages 11121-11128, 2023.

Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less is more: Fast multivariate time series forecasting with light sampling-oriented mlp structures. arXiv preprint arXiv:2207.01186 , 2022.

Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In International Conference on Learning Representations , 2023.

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence , volume 35, pages 11106-11115, 2021.

Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning , pages 27268-27286. PMLR, 2022.

Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained lm. Advances in neural information processing systems , 36, 2024.

A Development of time series forecasting

In recent years, the time series analysis community has shifted its focus from short-term forecasting to tasks with longer prediction horizons, also known as LTSF tasks . This shift offers greater convenience but also poses increased challenges. Mainstream approaches can be roughly classified into the following five distinct classes:

Transformer-based Models It is widely recognized that Transformers possess impressive capabilities for long-distance modeling, and thus researchers have high expectations for their adaptation to long time series tasks [46, 49]. Early research works, such as LogTrans [25], TFT [29], Informer [59], Autoformer [51], Pyraformer [35], FEDformer [60], ETSformer [50], and NSTransformer [36], focused on optimizing the original Transformer architecture for time series analysis tasks. However, more recent research has found that satisfactory performance can be achieved by simply partitioning patches, drawing inspiration from patch techniques used in the computer vision community [7, 14]). Approaches like PatchTST [40], PETformer [30], and Crossformer [58] have demonstrated promising results by adopting this patch-based approach.

Linear- and MLP-based Models Linear- and MLP-based methods are often lighter-weight, especially compared to Transformer methods that require stacking multiple blocks [57, 4]. A particularly notable breakthrough is the observation made by DLinear [56], which demonstrates that a single-layer linear approach could outperform many complex Transformer designs. This observation leads to a sequence of works, including TiDE [5], MTS-Mixers [28], TSMixer [28], TimeMixer [48], HDMixer [16], SOFTS [12], FITS [53], SparseTSF [32], and SSCNN [6]. The proposed CycleNet in this paper is also a Linear- or MLP-based model that is simple, efficient, and powerful for time series forecasting.

RNN-based Models Conceptually, Recurrent Neural Networks (RNN) are considered to be the most suitable models for modeling time series data [24, 43]. However, due to difficulties in parallelization and modeling long sequences, RNNs are not the most popular choice in works on LTSF tasks. Recent works aim to revitalize RNN models in long sequence modeling tasks, such as SegRNN [31], WITRAN [19], SutraNets [2], and RWKV-TS [15].

TCN-based Models Because of the parallelizability of convolution operations and their ability to capture features at different time scales, Temporal Convolutional Networks (TCN) methods are considered strong competitors for addressing time series tasks [1, 9]. Recent works that apply TCN methods to LTSF tasks include SCINet [34], MICN [47], TimesNet [52], PatchMixer [10], and ModernTCN [38].

LLM-based Models The remarkable capabilities demonstrated by large language models (LLM) have sparked interest among researchers from various fields, including those working on time series forecasting tasks [21, 18]. Some works consider fine-tuning pre-trained LLMs to perform time series analysis tasks, including OFA [61], Time-LLM [20], and TEMPO [3]. Other works aim to achieve zero-shot inference using large pre-trained LLMs through prompt engineering, including LLMTime [11], PromptCast [54], and LSTPrompt [33].

B More details of CycleNet

B.1 Overall pseudocode

Algorithm 1 demonstrates the implementation of modeling periodic patterns through recurrent cycles. Specifically, the first line defines the learnable parameter queue Q and initializes it to zero. Lines 2-11 define the getCycle function, which will be called by CycleNet to obtain the corresponding truncated equivalent cyclic subsequences. This function takes two parameters, i and l , where i represents the relative positional index for Q , and l represents the length of the required subsequence. Q learns internal periodic patterns within the sequence through co-training with the backbone.

Furthermore, Algorithm 2 illustrates the workflow of CycleNet. The first step is to normalize the samples based on their mean and standard deviation, then call the getCycle function to remove the cyclic components of the input data. Subsequently, predict the residual components through the backbone. Finally, add back the cyclic components of the output data, and perform instance denormalization to obtain the final prediction result. Here, the cycle index i corresponds to t mod W , as described in Section 3.1.

Algorithm 1 Modeling periodic patterns through recurrent cycles

Algorithm 2 Workflow of CycleNet

B.2 Utilizing ACF analysis to determine cycle length

The RCF technique utilizes recurrent cycles Q ∈ R W × D to model the internal periodic patterns of sequences. Here, the hyperparameter W determines the length of the recurrent cycles, which should precisely match the length of the periodic patterns within the data. As shown in the results of Table 6, when W is not accurately set, the RCF technique fails to fulfill its intended purpose. Although, in practice, we can infer the maximum cycle length of the dataset by considering the data's sampling frequency and the potential existing periodic patterns (as shown in Table 1), this manual inference method may introduce errors. Therefore, we may need a more scientific and precise approach to find the hyperparameter W .

In such cases, the autocorrelation function (ACF) [39] serves as a powerful mathematical tool to help us determine the periodicity within the data. The autocorrelation function measures the correlation between a time series and its lagged values, indicating the presence of autocorrelation within the data. Mathematically, this can be expressed as:

where N represents the total number of observations, x t denotes the value of the time series at time t , k is the lag time, and ¯ x is the mean of the time series values.

Here, when the lag time k aligns with the data's cycle, the ACF value exhibits a significant peak. Specifically, the largest peak corresponds to the lag that aligns with the length of the maximum cycle present in the dataset. Conversely, if the data lacks periodicity, no significant peaks or troughs will be observed.

We present the ACF results for each dataset in Figure 6. It can be observed that these datasets all display evident periodicity, indicated by prominent peaks and troughs in the plots. More importantly,

0

50

100

150

200

0.0

0.2

0.4

0.6

0.8

1.0

Autocorrelation

(a) ETTh1,

W

= 24

0

50

100

150

200

0.6

0.7

0.8

0.9

1.0

Autocorrelation

(b) ETTh2,

W

= 24

0

50

100

150

200

0.0

0.2

0.4

0.6

0.8

1.0

Autocorrelation

(c) ETTm1,

W

= 96

0

50

100

150

200

0.7

0.8

0.9

1.0

Autocorrelation

(d) ETTm2,

W

= 96

0

50

100

150

200

0.0

0.2

0.4

0.6

0.8

1.0

Autocorrelation

0

50

100

150

200

-0.5

0.0

0.5

1.0

Autocorrelation

0

50

100

150

200

-0.25

0.00

0.25

0.50

0.75

1.00

Autocorrelation

0

50

100

150

200

-0.50

-0.25

0.00

0.25

0.50

0.75

1.00

Autocorrelation

(e) Electricity,

W

= 168

(f) Solar-Energy,

W

= 144

(g) Traffic,

W

= 168

(h) Weather,

W

= 144

the maximum cycles shown in the plots align with the pre-inferred cycle lengths from Table 1. This indicates the correctness of the pre-inferred lengths, and W should be strictly set to these values.

B.3 Experimental details

We utilized widely used benchmark datasets for LTSF tasks, including the ETT series, Electricity, Solar-Energy, Traffic, and Weather. Following prior works such as Autoformer [51] and iTransformer [37], we split the ETTs dataset into training, validation, and test sets with a ratio of 6:2:2, while the other datasets were split in a ratio of 7:1:2.

We implemented CycleNet using PyTorch [41] and conducted experiments on a single NVIDIA RTX 4090 GPU with 24GB of memory. CycleNet was trained for 30 epochs with early stopping based on a patience of 5 on the validation set. The batch size was set uniformly to 256 for ETTs and the Weather dataset, and 64 for the remaining datasets. This adjustment was made because the latter datasets have a larger number of channels, requiring a relatively smaller batch size to avoid out-of-memory issues. The learning rate was selected from the range {0.002, 0.005, 0.01} based on the performance on the validation set. The hyperparameter W was set consistently to the pre-inferred cycle length as shown in Table 1. Additionally, the hidden layer size of CycleNet/MLP was uniformly set to 512.

By default, CycleNet uses RevIN without learnable affine parameters [22]. However, we found that on the Solar dataset, using RevIN leads to a significant performance drop, as shown in Appendix C.4. The primary reason for this may be that photovoltaic power generation data contains continuous segments of zero values (no power generation at night). When the look-back windows are not an integer multiple of a day, the calculation of means in RevIN can be significantly affected, leading to decreased performance. Therefore, for this dataset, we did not apply the RevIN strategy.

C More experimental results

C.1 Periodic patterns learned under different configurations

The proposed RCF technique can effectively learn the inherent periodic patterns within time series data. This capability is a significant advantage, revealing the potential value of RCF or its underlying cyclic modeling approach as a superior method to assist data engineers in analyzing patterns in time series data. To further elucidate the working principle behind RCF, we delve into the periodic patterns learned by the RCF technique under different configurations, as illustrated in Figure 7:

Forecast horizon H : The learned patterns remain almost unchanged as the horizon length varies. This indicates that the horizon length does not affect the learned pattern results.

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(a) Horizon-96

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(b) Horizon-192

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(c) Horizon-336

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(d) Horizon-720

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(e) Lookback-96

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(f) Lookback-192

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(g) Lookback-336

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(h) Lookback-720

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(i) Backbone-Linear

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(j) Backbone-DLinear

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(k) Backbone-PatchTST

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(l) Backbone-iTransformer

0

25

50

75

100

125

150

175

-1.5

-1.0

-0.5

0.0

0.5

1.0

(m) W-168

0

20

40

60

80

-1.5

-1.0

-0.5

0.0

0.5

1.0

(n) W-96

0

5

10

15

20

-1.5

-1.0

-0.5

0.0

0.5

1.0

(o) W-24

0

5

10

15

20

-1.5

-1.0

-0.5

0.0

0.5

1.0

(p) W-23

Look-back length L : The overall pattern remains unchanged as the look-back window changes. However, with closer observation, it is noticeable that the learned pattern becomes smoother with an increased look-back. This is because a longer look-back provides the backbone with richer periodic information, thereby reducing the reliance on the learned pattern component.

Backbone : The patterns vary somewhat with different backbones. When DLinear is used as the backbone, the learned patterns are smoother, as DLinear's decomposition technique itself extracts certain periodic features. When iTransformer is the backbone, the learned patterns differ more, as it additionally models multichannel relationships, so the learned periodic patterns may consider multichannel feature interactions. PatchTST's performance is more similar to that of Linear, as it is also a regular single-channel modeling method, though with stronger nonlinear learning capabilities compared to the Linear model.

Cycle length W : When W is set to 168 (the weekly cycle length for the Electricity dataset), the recurrent cycle Q learns the complete periodic pattern, including both weekly and daily cycles. When W is set to 24 (the daily cycle length), the recurrent cycle Q only learns the daily cycle pattern. When W is set to 96 (four times the daily cycle length), the recurrent cycle Q learns four repeated daily cycle models. However, when W is set to 23 (without matching any semantic meaning), the recurrent cycle Q fails to learn any meaningful pattern, resulting in a straight line.

C.2 Full results with different look-back lengths

Table 2 presents the comparison results of CycleNet with other models on the mean performance at look-back length L = 96 for various forecast horizons H ∈ { 96 , 192 , 336 , 720 } . Here, we further

showcase the complete comparison results for different forecast horizons in Table 7. It can be observed that in most settings, CycleNet achieves state-of-the-art results, consistent with the findings in Table 2. Additionally, the standard deviation of CycleNet's results is mostly below 0.001. This strongly indicates the robustness of CycleNet.

Additionally, the look-back length is a crucial hyperparameter that significantly impacts the performance of time series forecasting models, as it determines the richness of information the model can leverage. Initially, the community focused primarily on exploring the application of Transformers in time series forecasting tasks. Due to the inherent complexity of Transformers, using excessively long look-back windows resulted in a significant increase in runtime. As a result, many popular models at the time, such as Informer [59], Autoformer [51], and FEDformer [60], employed shorter look-back windows, typically with L = 96 .

With the recent development of model lightweighting techniques, particularly the adoption of channelindependent strategies (first applied in DLinear [56] and PatchTST [40]), more models have started to experiment with longer look-back windows in pursuit of higher predictive accuracy. For instance, DLinear and PatchTST default to using look-back windows of L = 336 , while SegRNN [31] and SparseTSF [32] default to using L = 720 . To explore CycleNet's performance with longer look-back windows, we compared CycleNet with these advanced models using their respective default, longer look-back windows in Table 8.

It is important to note that we re-ran the official open-source code of these baselines to obtain the corresponding results, using the same MSE as the loss function (as SegRNN originally used MAE as its loss). Additionally, there was a long-standing bug in their original repositories, where the data from the last batch was discarded during testing [42, 53]. This issue could have affected the model's performance, so we fixed this problem before re-running the experiments.

It can be observed that even with a longer look-back length, CycleNet generally maintains a significant advantage, achieving state-of-the-art performance in most scenarios. This demonstrates CycleNet's excellent performance across different look-back lengths. It is worth noting that both PatchTST and

SegRNN outperform CycleNet on the Traffic dataset, even though they are also channel-independent models. This is partly because the Traffic dataset contains more outliers (see more discussion in Appendix C.5), which may impact the performance of RCF; additionally, PatchTST and SegRNN are more complex deep models with stronger nonlinear capabilities, enabling them to fit various patterns across numerous channels (the Traffic dataset has up to 862 channels).

C.3 Full results with different STD techniques

The proposed RCF technique is essentially a type of Seasonal-Trend Decomposition (STD) method. To directly compare RCF with existing related STD techniques, we adopted a strategy consistent with DLinear, using a pure Linear model as the backbone and not applying any instance normalization

techniques. We previously reported the mean performance of these techniques across different horizons H ∈ { 96 , 192 , 336 , 720 } in Table 5. Here, we further present the complete comparative results for all horizons in Table 9.

The results show that the RCF technique consistently outperforms other techniques. A notable exception is the relatively noisy weather dataset, where RCF does not show a significant advantage. However, in this case, the performance of several STD techniques is similar to that of the pure Linear model. Overall, these findings strongly support RCF as a new STD method that enhances model performance in scenarios with strong periodicity.

C.4 Ablation study of RevIN

Instance normalization strategies constitute essential factors for the success of current models, such as PatchTST [40], TiDE [5], iTransformer [37], SparseTSF [32], etc. By default, CycleNet also adopts this strategy, namely the version of RevIN without learnable affine parameters [22]. Here, we meticulously investigate the impact of RevIN on the performance of CycleNet, and the results are shown in Table 10. On the ETTh2 and Weather datasets, RevIN significantly enhances the performance of CycleNet, possibly due to more severe distribution drift issues in these datasets. However, on the Solar dataset, RevIN leads to poorer performance, likely because the photovoltaic power generation data contains continuous segments of zero values (no power generation at night), which significantly affects the calculation of means in RevIN.

Overall, in most cases, RevIN leads to better performance. We acknowledge that RevIN is an indispensable cornerstone of CycleNet's success, but it is not the key factor that sets CycleNet apart from other models in terms of performance. As shown in the comparison results in Table 10, CycleNet exhibits a significant advantage over RLinear and RMLP, which can be viewed as CycleNet without RCF technique. This clearly demonstrates that the RCF technique is the key factor that significantly enhances the model's prediction accuracy, constituting the core contribution of this paper.

C.5 Further Analysis in Traffic Scenarios

CycleNet, formed by combining the RCF technique with a simple backbone, achieved state-of-the-art performance across multiple domains but fell short in the traffic domain. To further investigate the reasons behind this, we supplemented the complete performance of CycleNet on the PEMS dataset (the same four public subsets adopted in SCINet [34]) in Table 11. The results show that: (i) CycleNet still achieved top-tier prediction accuracy, and (ii) although CycleNet underperformed compared to iTransformer in this scenario, the gap in MSE on the Traffic dataset was reduced from approximately 10% to about 5%.

Regarding the first point, it is important to highlight the effectiveness of RCF. CycleNet's backbone is merely a single-layer Linear or a two-layer MLP, without any additional design or deep stacking, yet it still delivers excellent results. Specifically, when comparing CycleNet/Linear with RLinear and DLinear, it becomes evident that RCF is the major contributor to narrowing the gap between the simple Linear model and those state-of-the-art models.

For the second point, we further analyzed the statistical characteristics of the datasets to explore the underlying reasons in Table 12. Specifically, we examined the presence of extreme values in the channels and the cosine similarity between channels. It was found that the Traffic dataset contains very significant outliers, both in terms of quantity and magnitude. The presence of these outliers:

(i) May affect the effectiveness of RCF. The fundamental working principle of RCF is to learn the historical average cycles in the dataset. In such cases, the average cycles learned in RCF can be skewed by these significant outliers, such as the mean of a certain point in the cycle being exaggerated.

Consequently, during each prediction process, the original sequence subtracts a locally exaggerated average cycle, resulting in an inaccurate residual component and affecting the local point predictions within each cycle. The more inaccurate these local point predictions are, the larger the discrepancy between MSE and MAE, as MSE significantly amplifies the impact of a few large errors. This explains why in Table 4, combining iTransformer with RCF decreases MAE but increases MSE, indicating overall prediction accuracy improvement but anomalies in local point predictions.

(ii) Highlight the necessity of stronger spatiotemporal relationship modeling. Models like iTransformer and GNN, which accurately model inter-channel relationships, are more suitable for scenarios with extreme points and temporal lag characteristics. For example, when a sudden traffic surge occurs at a certain junction, these models, having correctly modeled the spatiotemporal relationships, can accurately predict possible traffic surges at other junctions. In contrast, the current CycleNet only considers single-channel relationship modeling, making it somewhat limited in this scenario.

These underlying reasons explain why CycleNet did not achieve the best performance on the Traffic dataset and showed a relative large performance gap. On the PEMS dataset, although it is also a traffic dataset, the presence of extreme points is significantly less severe compared to the Traffic dataset. Therefore, CycleNet's performance on the PEMS dataset improved compared to the Traffic dataset (the gap in MSE compared to the state-of-the-art reduced from approximately 10% to about 5%). This further validates the effectiveness of RCF but also indicates that in more complex traffic scenarios, reasonable spatiotemporal relationship modeling (or multivariate relationship modeling) is essential.

Additionally, while intuitively the solar scenarios might also involve significant spatiotemporal relationships, in practice, these relationships are much weaker compared to the traffic scenarios. Firstly, the weather conditions in the same region are often similar, leading to similar power generation curves. For instance, the Solar-Energy dataset's channels have a cosine similarity as high as 0.92 (shown in Table 12), which indirectly indicates weaker spatial characteristics. Secondly, extreme points are rare in the solar scenarios because photovoltaic systems have a maximum power threshold. Fewer extreme points mean that the impact of temporal lag characteristics is smaller. This explains why, compared to the Traffic dataset, the gains from the RCF technique are much more significant on the Solar-Energy dataset.

In summary, when dealing with traffic scenarios that may involve significant outliers and emphasize spatiotemporal relationship modeling, the current version of CycleNet may not be fully adequate. There are two direct and meaningful directions for improvement that could address this issue: (1) Enhancing the current RCF technique to be more robust to the presence of outliers; (2) Exploring a more reasonable multi-channel modeling technique within the RCF framework. We leave these challenges for future work and encourage the community to further research more robust and powerful periodic modeling techniques.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main claims in the abstract accurately reflect our contributions.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations of this work in Section 5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: This paper does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide complete experimental details in Appendix B.3. Additionally, we have shared the full reproducible code in an anonymous repository (link provided in the abstract).

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide an anonymous link to the code and describe how to reproduce the experimental results in the README file of the code.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We describe the complete experimental details and hyperparameter choices in Appendix B.3.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We report the standard deviations of the results for our proposed method under different settings in Table 7.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We report the computational resource requirements of our proposed method in Table 3.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our research aligns with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: The paper focuses on advancing the field of machine learning. While our work may have various societal implications, we believe none are significant enough to warrant specific mention here.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The code and datasets used in the paper are publicly available and properly credited.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We will make the code publicly available upon acceptance of the paper and provide detailed documentation.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This work does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This work does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel and effective technique to enhance long-term time series forecasting. The proposed technique, Residual Cycle Forecasting (RCF), directly models periodic cycles with learnable parameters, decomposing the learning of time series into periodic cycles and residual components. The residual can be learned with a simple architecture like Linear or MLP, and this technique can be integrated into existing forecasting models. Results show significant improvement achieved by using this technique.\n\nSTRENGTHS:\n* The proposed method is simple and effective. It not only improves accuracy but is also more parameter efficient and can be easily integrated into different backbones.\n* The paper is well-written with good clarity. Both the problem and analysis are presented clearly.\n* The experiments are conducted with high quality, providing extensive and comprehensive analysis of the proposed technique. The results are consistent with prior related work. Limitation of the proposed method is also clearly stated.\n\nWEAKNESSES:\nThe notations are not strictly consistent. The notations for instance normalization (in Section 3.2 and Algorithm 2) are independent from the whole framework and not consistent with the previous problem definition.\n\nQUESTIONS:\n1. Since Figure 2 depicts the entire CycleNet architecture, instance normalization should also be included.\n\n2. The description of alignment and repetition of $Q$ (Lines 152-157) can be further improved. The link to Algorithm 1 should be added, and a schematic figure (even in Appendix) would make it clearer to understand.\n\n3. How extreme points affect RCF in the Traffic dataset should be further explained (Line 255). Extreme points can also affect other methods. It is unclear why this would specifically deteriorate RCF.\n\nLIMITATIONS:\nSee Weaknesses and Questions", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces CycleNet, a novel time series forecasting method that enhances long-term prediction accuracy by explicitly modeling the inherent periodic patterns present in time series data. The core contribution of paper is introducing the Residual Cycle Forecasting (RCF) technique, which leverages learnable recurrent cycles to represent these periodic patterns and predicts the residuals, significantly improving upon the performance of existing models with reduced computational complexity. CycleNet demonstrates state-of-the-art results across various domains, such as electricity, weather, and energy forecasting, while offering over 90% reduction in parameter quantity, highlighting its efficiency and effectiveness in capturing long-term dependencies for accurate forecasting.\n\nSTRENGTHS:\n1. The author tries to model the period information explicitly in the time series prediction task, and the motivation is intuitive and reasonable.\n\n2. The method designed by the author is reasonable and closely related to motivation. Combined with the experimental results, the author gives a simple but effective method.\n\n3. The author exposes the code and gives a detailed description, which increases the reproducibility of the model.\n\n4. The limitations of this method are clearly discussed and the possible problems are pointed out.\n\nWEAKNESSES:\n1. In the introduction, the author's statement establishes a close relationship between long-term prediction and periodic information. In the absence of some experimental support, this is not rigorous. Periodic information may be useful for long-term forecasting in certain situations, but it is not appropriate for all tasks, nor is it the only important information that these tasks require. The author slightly obfuscates these to highlight the motivation of this article.\n\n2. The authors select data sets with different periodicity to illustrate the validity of the model. However, the authors only demonstrate the validity, and experiments can be added to show how the proposed method performs differently on periodically different data sets and discuss the underlying rules. At the same time, it is also worth showing how CycleNet performs on data sets where there is no obvious periodicity.\n\n3. Combined with the results in Table 2 and Table 4, the results predicted with Linear alone are even better than some well-designed methods, whether this is due to the particularity of the data set or some other reason.\n\nQUESTIONS:\nPlease refer to the weakness\n\nLIMITATIONS:\nPlease refer to the weakness", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel technique for improving the accuracy of multivariate long-term time series forecasting. The technique, called Residual Cycle Forecasting (RCF), involves learning the cyclical patterns of time series through recurrent cycles, which can be used as a pre-processing step for any forecasting model. The authors also propose CycleNet, a linear-based model that uses RCF to enhance its predictions. CycleNet first uses RevIn to account for distribution shift, then subtracts the learned RCF from the input data. The backbone of the model predicts the future residual, adds the learned RCF, and reverses RevIN from the outputs to obtain the final prediction. \n\nThe proposed method is evaluated on eight multivariate time series datasets and compared against several baselines. The results show competitive performance and resource consumption.\n\nSTRENGTHS:\n* Clear and concise writing style\n * Novel approach to time series decomposition\n * Comparison of performance and resource consumption with baseline methods\n * Ablation study and parameter impact analysis (e.g., $W$)\n * Easy-to-understand model design and components\n * Informative figures to illustrate model and results\n * Thorough discussion of results, including strengths and limitations\n * Code and data provided for reproducibility and transparency.\n\nWEAKNESSES:\n* Lack of clarity to specify which results are from the authors (reproduced or produced) and which ones are collected from previous papers (if so, which ones)\n * Incomplete comparison with existing time series decomposition baselines, such as LD, TDFNet, or SparseTSF (even though this one was in the related works)\n * Model was not compared to RLinear (also relying on RevIN)\n\nQUESTIONS:\n## Baselines\n\nThe paper fails to properly compare the proposed CycleNet model with appropriate baselines.\n\nSince CycleNet is based on RevIN, it would be more complete to include other RevIN-based baselines such as RLinear [1] for linear-based models and for instance RevInformer (from the RevIN paper) for transformer-based solutions. Although PatchTST uses RevIN, including these additional baselines would avoid any doubt and provide a more comprehensive comparison.\n\nAdditionally, the paper misses important baselines such as LD [2], which also uses RevIn and also argues for using learnable decomposition rather than moving average. According to the published results, CycleNet appears to be behind LD. But it would be interesting to compare LD against backbone + RCF.\nPotential references to [3] is also missing.\n\n[1] https://arxiv.org/pdf/2305.10721\n\n[2] https://arxiv.org/abs/2402.12694\n\n[3] https://arxiv.org/pdf/2308.13386\n\nFinally, despite being describes as main baselines, why SparseTSF was not included in this paper? Especially, as SparseTSF seems to produce better results than proposal especially for ETT and not for Electricity and Traffic. What would be the reason for such differences?\n\nIncluding such comparisons is required to correctly position CycleNet with all the other baselines and further discuss why CycleNet offers better performance with some datasets and not others. Such discussion is important to further investigate the benefit of time series decomposition for forecasting tasks.\n\n## Visualization\nThe visualizations of the learned cycles are informative, but it is unclear whether these cycles change depending on the prediction horizon or lookback. If they do, it would be interesting to see the differences and whether they can explain the variation in performance for different prediction horizons.\n\nSince the cycles are learned along with the backbone model, there could be differences in the learned cycles for different experiments. It would be helpful to plot the cycles for different prediction lengths, such as 96 and 720, and for different lookback lengths, such as 48, 96, and 336. \n\nIn addition, the cycle may change depending on the backbone. It would be interesting to show the difference if any when RCF is used with iTransformer, DLinear, etc. This would allow readers to better understand whether there are any differences in the learned cycles and whether they are due to the parameters, backbone model or other factors.\n\nFigure 3(f) is surprising, and it is unclear whether it represents a household or an industrial building with specific working hours. In my opinion, it looks like there are only 3 main weekly patterns (the large pattern on the left seems to have ~50 time steps = 2 days) and weekends (one pattern seems to have ~24 time steps = 1 day). Therefore, it could be a factory or industrial building with specific working hours (6 days a week).\n\nIt would be interesting to discuss how RCF impacts previous backbones, such as DLinear. Specifically, it would be helpful to know how much DLinear's performance changes with the addition of RCF and plot it similarly to Figure 4.\n\nFinally, it is important to provide Figure 4 and Table 4 for other datasets to give readers a global view of the impact of RCF/CycleNet depending on the dataset and its nature. This would help avoid generalizations that could be incorrect based only on results depicted in the current version, especially for datasets such as Traffic and Solar Energy where the impact might be drastically different.\n\n## Discussion\n“Traffic dataset exhibits spatiotemporal characteristics and temporal lag characteristics, where the traffic flow at a certain detection point significantly affects the future values of neighboring detection points.” However, the authors do not explain why this is not an issue for the Solar-Energy dataset, where weather conditions at one location may slowly impact neighboring locations in the future. Can authors comment on that?\n\nRegarding the electricity consumption dataset, the authors state that \"a user’s electricity consumption thirty days ahead does not directly correlate with their consumption patterns in the past few days.\" However, it is possible that long-term habits may still influence consumption patterns, for instance with a monthly routine.\n\nThe authors suggest that \"the cycle length W depends on the a priori characteristics of the dataset and should be set to the maximum stable cycle within the dataset.\" However, they do not address how to account for yearly cycles, such as those found in weather datasets that repeat each year with the seasons or datasets influenced by human behavior, such as electricity consumption, which may increase in winter and summer due to heating and cooling needs. It would be helpful to clarify whether RCF is only suitable for mid-range stable cycles or if there is a way to account for longer cycles. This might need to be included in the limitations section.\n\n## Additional points\nThe paper lacks a computation cost study to determine the overhead imposed by RCF on the existing backbone in terms of memory and consumption. It is important to evaluate whether the gain in accuracy justifies the increase in complexity and associated cost.\n\n## Proof-read\n * “a single-layer Linear or a dual-layer MLP” maybe find another way to express it to avoid repeating too much (cf., abstract, introduction, etc.)\n * “surpassing complexly designed deep models”->  redundant, ”complex design” or “deep models”\n * “leveraging this discovery to enhance” should change this as the fact that time series have cycle is not new leveraging this knowledge is the innovation\n\nLIMITATIONS:\nAuthors have discussed some limitations of their proposal and especially when each channel have different period cycle.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

comments: SUMMARY:
This paper proposes a learnable Seasonal-Trend Decomposition method (CycleNet) to improve the prediction performance of current long-term multivariate time series forecasting models. Specifically, it firstly model the periodic patterns of sequences through globally shared recurrent cycles and then predicts the residual components of the modeled cycles. Extensive experiments are conducted to evaluate the proposed method.

STRENGTHS:
1. The proposed method is a model-agnostic solution applicable to different kinds of models.
2. Although it is simple, it is able to achieve good performance improvement in many cases.
3. Extensive experiments are conduct to evaluate the proposal.

WEAKNESSES:
1. It seems that the proposal (CycleNet) does not work well for complex datasets, e.g., Traffic. It is better to show more results on the same kind of datasets like the PEMS datasets used in the iTransformer paper.
2. The Section 3 is not well written and lacks a lot of details, especially about the Learnable Recurrent Cycles. The authors may consider to reorganize Section 3 and Appendix A.1 to make Section 3 more clear.

QUESTIONS:
Please refer to the weaknesses.

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-13