CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns

Authors: shengsheng  lin
weiwei  lin
xinyi  hu
wentai  wu
ruichao  mo
haocheng  zhong
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 10:12:57

Abstract: The stable periodic patterns present in time series data serve as the foundation for conducting long-horizon forecasts. In this paper, we pioneer the exploration of explicitly modeling this periodicity to enhance the performance of models in long-term time series forecasting (LTSF) tasks. Specifically, we introduce the Residual Cycle Forecasting (RCF) technique, which utilizes learnable recurrent cycles to model the inherent periodic patterns within sequences, and then performs predictions on the residual components of the modeled cycles. Combining RCF with a Linear layer or a shallow MLP forms the simple yet powerful method proposed in this paper, called CycleNet. CycleNet achieves state-of-the-art prediction accuracy in multiple domains including electricity, weather, and energy, while offering significant efficiency advantages by reducing over 90% of the required parameter quantity. Furthermore, as a novel plug-and-play technique, the RCF can also significantly improve the prediction accuracy of existing models, including PatchTST and iTransformer. The source code is available at: https://github.com/ACAT-SCUT/CycleNet.

Main paper: 
 introduction: 
 Time series forecasting (TSF) plays a crucial role in various domains such as weather forecasting, transportation, and energy management, providing insights for early warnings and facilitating proactive planning. Particularly, accurate predictions over long horizons (e.g., spanning several days or months) offer increased convenience, referred to as Long-term Time Series Forecasting (LTSF) [59, 56, 17, 42, 6] . However, the principle enabling long-horizon prediction lies in understanding the inherent periodicity within the data [32] . Unlike short-term forecasting, long-term predictions cannot rely solely on recent temporal information (including means, trends, etc.). For instance, a user's electricity consumption thirty days ahead not only correlates with their consumption patterns in the past few days.In such cases, long-term dependencies, or in other words, underlying stable periodicity within the data, serve as the practical foundation for conducting long-term predictions [32] . This is why existing models emphasize their capability to extract features with long-term dependencies. Models like Informer [59] , Autoformer [51] , and PatchTST [40] utilize the Transformer's ability for long-distance modeling to address LTSF tasks. ModernTCN [38] employs large convolutional kernels to enhance TCNs' ability to capture long-range dependencies, and SegRNN [31] uses segment-wise iterations to improve RNN methods' handling of long sequences. If a model can accurately capture long-range dependencies, it can precisely extract periodic patterns from historical long sequences, enabling more accurate long-horizon predictions.However, if the purpose of constructing deep and complex models is solely to better extract periodic features from long-range dependencies, why not directly model the patterns? As illustrated in Figure 1 , electricity data exhibits clear daily periodic patterns (in addition to possible weekly patterns). We can use a globally shared daily segment to represent the periodic pattern in electricity consumption. By repeating this daily segment N times, we can continuously represent the cyclic components of N days' electricity consumption sequences. Based on the above motivation, we pioneer explicit modeling of periodic patterns in the data to enhance the model's performance on LTSF tasks in this paper. Specifically, we propose the Residual Cycle Forecasting (RCF) technique. It involves using learnable recurrent cycles to explicitly model the inherent periodic patterns within time series data, followed by predicting the residual components of the modeled cycles. Combining the RCF technique with either a single-layer Linear or a dual-layer MLP results in CycleNet, a simple yet powerful method. Cy-cleNet achieves consistent state-of-the-art performance across multiple domains and offers significant efficiency advantages.In summary, this paper contributes:• We identify the presence of shared periodic patterns in long-horizon forecasting domains and propose explicit modeling of these patterns to enhance the model's performance on LTSF tasks.• Technically, we introduce the RCF technique, which utilizes learnable recurrent cycles to explicitly model the inherent periodic patterns within time series data, followed by predicting the residual components of the modeled cycles. The RCF technique significantly enhances the performance of basic (or existing) models.• Applying RCF with a Linear layer or a shallow MLP forms the proposed simple yet powerful method, called CycleNet. CycleNet achieves consistent state-of-the-art performance across multiple domains and offers significant efficiency advantages.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes]Justification: The main claims in the abstract accurately reflect our contributions.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
related work: 
 In fact, utilizing periodic information to enhance model prediction accuracy is not a novel concept. Numerous studies, in particular, have introduced a series of Seasonal-Trend Decomposition (STD) techniques that allow models to better leverage periodic information. Popular models such as Autoformer [51] , FEDformer [60] , and DLinear [56] utilize the classical STD approach to decompose the original time series into two equally sized subsequences: seasonal and trend components, which are then modeled independently. These classical STD methods typically use a basic moving average (MOV) kernel to perform a sliding aggregation to obtain the trend component. Recently, Leddam [55] proposed replacing the traditional MOV kernel in STD with a Learnable Decomposition (LD) kernel, leading to improved performance. Additionally, DEPTS [8] treats the periodicity of sequences as a parameterized function with respect to time, and learns periodic and residual components layer-wise through its periodic and local blocks. SparseTSF [32] , another recent work, utilizes cross-period sparse forecasting technique to decouple cycles and trends, achieving impressive performance at extremely low cost.The RCF technique proposed in this paper can essentially be considered a type of STD method. The key difference from existing techniques lies in its explicit modeling of global periodic patterns within independent sequences using learnable recurrent cycles. The proposed RCF technique is conceptually simple, computationally efficient, and yields significant improvements in prediction accuracy. The further proposed CycleNet, which combines the RCF technique with a simple backbone, is a Linearor MLP-based model that is simple, efficient, and powerful for time series forecasting. To correctly position CycleNet, we have provided a detailed review of the development of different categories of time series forecasting methods (including Transformer-based, RNN-based, etc.) in Appendix A.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We discuss the limitations of this work in Section 5.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
cyclenet: 
 Given a time series X with D variables or channels, the objective of time series forecasting is to predict future horizons H steps ahead based on past L observations, mathematically represented as f : x t-L+1:t ∈ R L×D → xt+1:t+H ∈ R H×D . In fact, the inherent periodicity within time series is fundamental for accurate prediction, particularly when forecasting over large horizons, such as 96-720 steps (corresponding to several days or months). To enhance the model's performance on long-term prediction tasks, we propose the Residual Cycle Forecasting (RCF) technique. It combines a Linear layer or a shallow MLP to form a simple yet powerful method CycleNet, as illustrated in Figure 2 , with detailed pseudocode provided in Appendix B.1. RemoveThe RCF technique comprises two steps: the first step involves modeling the periodic patterns of sequences through learnable recurrent cycles within independent channels, and the second step entails predicting the residual components of the modeled cycles.Periodic patterns modeling Given D channels with a priori cycle length W , we first generate learnable recurrent cycles Q ∈ R W ×D , all initialized to zeros. These recurrent cycles are globally shared within channels, meaning that by performing cyclic replications, we can obtain cyclic components C of the sequence X of the same length. These recurrent cycles Q of length W undergo gradient backpropagation training along with the backbone module for prediction, yielding learned representations (distinct from the originally initialized zeros) that unveil the internal cyclic patterns within the sequence.Here, the cycle length W depends on the a priori characteristics of the dataset and should be set to the maximum stable cycle within the dataset. Considering that scenes requiring long-term predictions usually exhibit prominent, explicit cycles (e.g., electrical consumption and traffic data exhibit clear daily and weekly cycles), determining the specific cycle length is available and straightforward. Additionally, the dataset's cycles can be further examined through autocorrelation functions (ACF) [39] , as revealed in Appendix B.2.Residual forecasting Predictions made on the residual components of the modeled cycles, termed residual forecasting, are as follows:1. Remove the cyclic components c t-L+1:t from the original input x t-L+1:t to obtain residual components x ′ t-L+1:t .2. Pass x ′ t-L+1:t through the backbone to obtain predictions for the residual components, x′ t+1:t+H .3. Add the predicted residual components x′ t+1:t+H to the cyclic components c t+1:t+H to obtain xt+1:t+H .It is important to note that, since the cyclic components C are virtual sequences derived from the cyclic replications of Q, we cannot directly obtain the aforementioned sub-sequences c t-L+1:t and c t+1:t+H . Therefore, as illustrated in Figure 3 , appropriate alignments and repetitions of the recurrent cycles Q are needed to obtain equivalent sub-sequences: (i) Left-shift Q by t mod W positions to obtain Q (t) . Here, t mod W can be viewed as the relative positional index of the current sequence sample within Q. (ii) Repeat Q (t) ⌊L/W ⌋ times and concatenate Q (t) 0:L mod W . Mathematically, these two equivalent subsequences can be represented as: 𝑄 ∈ ℝ 𝐷×𝑊 𝑐 𝑡-𝐿+1:𝑡 ∈ ℝ 𝐷×𝐿 𝑄 (𝑡) ∈ ℝ 𝐷×𝑊 𝑄 (𝑡+𝐿) ∈ ℝ 𝐷×𝑊 𝐿/𝑊 × W 𝐿 mod 𝑊 𝑡 mod 𝑊 (𝑡 + 𝐿) mod 𝑊 𝐻/𝑊 × W 𝐻 mod 𝑊EQUATIONEQUATIONBackbone The original prediction task is transformed into cyclic residual component modeling, which can serve as normal sequence modeling. Therefore, any existing time series forecast model can be employed as a backbone. In this paper, our aim is to propose and examine a method for enhancing time series prediction by explicitly modeling cycles (i.e., RCF). Thus, we opt for the most basic backbone, namely a single-layer Linear and a dual-layer MLP, forming our simple yet powerful methods, CycleNet/Linear and CycleNet/MLP. Herein, each channel utilizes the same backbone with parameter sharing for modeling, which is also referred to as the Channel Independent strategy [13] .The statistical properties of time series data, such as the mean, often vary over time, which is referred to as distributional shifts. This can lead to poor performance of models trained on historical training sets when applied to future data. To address this issue, recent research has introduced Instance Normalization strategies like RevIN [45, 22, 26] . Mainstream approaches such as iTransformer [37] , PatchTST [40] , and SparseTSF [32] have widely adopted similar techniques to enhance performance.To improve the robustness of CycleNet, we also incorporate a similar optional strategy (see the full ablation study in Appendix C.4). Specifically, we remove the varying statistical properties from the model's internal representations outside of CycleNet's input and output steps:EQUATIONEQUATIONwhere µ and σ represent the mean and standard deviation of the input window, respectively, and ϵ is a small constant for numerical stability. This method aligns with the RevIN version that excludes learnable affine parameters [22] .To remain consistent with current mainstream methods, CycleNet defaults to using Mean Squared Error (MSE) as the loss function to ensure fair comparison with other methods, formulated as:EQUATION4 ExperimentsQuestion: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [NA]• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
setup: 
 Datasets We utilized widely adopted benchmark datasets including the ETT series [59] , Weather, Traffic, Electricity, and Solar-Energy [24] . Preprocessing operations on the datasets, such as dataset splitting and normalization methods, remained consistent with prior works (e.g., Autoformer [51] , iTransformer [37] , etc.).The information of the datasets is shown in Table 1 . Note that these datasets all exhibit stable cyclic patterns, such as daily and weekly, which form the realistic basis for performing long-horizon forecasting. Combined with the sampling frequency of the datasets, we can infer the maximum cycle length of the datasets, such as 24 for ETTh1 and 168 for Electricity. These manually inferred cycle lengths can be further confirmed through the ACF analysis, details of which are provided in Appendix B.2. The hyperparameter W of CycleNet is set by default to match the cycle length in Table 1 . Baselines We compared CycleNet against state-of-the-art models in recent years, including iTransformer [37] , PatchTST [40] , Crossformer [58] , TiDE [5] , TimesNet [52] , DLinear [56] , SCINet [34] , FEDformer [60] , Autoformer [51] . To comprehensively evaluate CycleNet's performance, the Mean Squared Error (MSE) and Mean Absolute Error (MAE) metrics were employed.Environments All experiments in this paper were implemented using PyTorch [41] , trained using the Adam [23] optimizer, and executed on a single NVIDIA GeForce RTX 4090 GPU with 24 GB memory. Firstly, when combining the basic Linear and MLP backbones (both utilizing instance normalization by default) with the RCF technique, a significant improvement in prediction accuracy (approximately 10% to 20%) is observed. This demonstrates that the success of CycleNet is largely attributed to the RCF technique rather than the backbones themselves or the instance normalization strategy.Overall, the performance of MLP is stronger than that of Linear, regardless of whether the RCF technique is applied. This indicates that non-linear mapping capability is necessary when modeling high-dimensional datasets with the channel-independent strategy (sharing parameters across each channel), aligning with previous research findings [26] .Secondly, we further verified whether RCF can enhance the prediction accuracy of existing models, as RCF is essentially a plug-and-play flexible technique. It is observed that incorporating RCF still improves the performance of existing complex designed, deep stacked models (approximately 5% to 10%), such as PatchTST [40] and iTransformer [37] . Even for DLinear, which already employs the classical MOV-based STD technique, RCF was able to provide an improvement of approximately 20%. This further indicates the effectiveness and portability of RCF.However, an interesting phenomenon was observed: although the MAE decreases when PatchTST and iTransformer are combined with RCF, the MSE increases. The most important reason behind this is that there are extreme points in the Traffic dataset that could affect the effectiveness of RCF, which fundamentally relies on learning the historical average cycles in the dataset. We further analyze this phenomenon in detail in Appendix C.5 and suggest potential directions for improving the RCF technique.The proposed RCF technique is essentially a more powerful STD approach. Unlike existing methods that decompose the periodic (seasonal) component from a limited look-back window, RCF learns the global periodic component from the training set.Here, we compare the effectiveness of RCF with existing STD techniques, using a pure Linear model as the backbone (without applying any instance normalization strategies). The comparison includes LD from Leddam [55] , MOV from DLinear [56] , and Sparse technique from SparseTSF [32] . As shown in Table 5 , RCF significantly outperforms other STD methods, particularly on datasets with strong periodicity, such as Electricity and Solar-Energy. In contrast, the other STD methods did not show significant advantages over the pure Linear model.There are several reasons for this. First, MOV and LD-based STD methods achieve trend estimation by sliding aggregation within the look-back window, which suffers from inherent issues [27, 26] : (i) The sliding window of the moving average needs to be larger than the maximum period of the seasonal component; otherwise, the decomposition may be incomplete (especially when the period length exceeds the look-back sequence length, making decomposition potentially impossible).(ii) Zero-padding is required at the edges of the sequence samples to obtain equally sized moving average sequences, leading to distortion of the sequence edges. As for the Sparse technique, being a lightweight decomposition method, it relys more on longer look-back windows and instance normalization strategies to ensure adequate performance. 6 . When correctly setting the hyperparameter W to the max cycle length of the dataset (i.e., the cycle length pre-inferred in Table 1 ), RCF can play a significant role, yielding a large performance gap compared to the cases when it is not correctly set. This indicates the necessity of inferring and setting the correct W for RCF to function properly. Furthermore, when W is incorrectly set, the model's performance is almost the same as when RCF is not used at all. This suggests that even in the worst-case scenario, RCF does not bring significant negative effects.Visualization of the learned periodic patterns The purpose of the RCF technique is to utilize the learnable recurrent cycles Q (initialized to zero) to model the periodic patterns in time series data.After co-training with the backbone, the recurrent cycles can represent the inherent periodic patterns of the sequence. Figure 4 illustrates the different periodic patterns learned from different datasets and channels. For example, Figure 4 (c) shows the daily operating pattern of solar photovoltaic generation, while Figure 4 (d) displays the weekly operating pattern of traffic flow, featuring peak traffic in the mornings on weekdays. These periodic patterns learned from the global sequence provide important supplementary information to the prediction model, especially when the length of the look-back window is limited and may not provide sufficient cyclic information when the cycle length is long.Furthermore, although the cycle length is the same for different channels within the same dataset, the specific periodic patterns differ, as shown in Figure 4 (e-h). Particularly, Figure 4 (f) demonstrates the intermittent periodicity of household electricity consumption on weekdays, while others exhibit relatively uniform weekday patterns in their respective channels. This highlights the necessity of separately modeling the periodic patterns for each channel. In Performance with varied look-back length The look-back length determines the richness of historical information that can be utilized. Theoretically, the larger it is, the better the model performance should be, especially for models capable of capturing long-term dependencies. Figure 5 shows the performance of different models under different look-back lengths. It can be observed that CycleNet, as well as representatives of current state-of-the-art models such as iTransformer [37] , PatchTST [40] , and DLinear [56] , all achieve better performance with longer look-back lengths. This indicates that these models all possess strong capabilities in modeling long-term dependencies.It is worth highlighting that (i) on the Electricity dataset, CycleNet outperforms current state-of-theart models at any prediction length; (ii) on the Traffic dataset, CycleNet still falls short compared to powerful existing multivariate forecasting models, such as iTransformer. This indicates that in scenarios with strong periodicity but without additional spatiotemporal relationships, fully leveraging the periodic components is sufficient to achieve high-accuracy predictions. However, in more complex scenarios that require thorough modeling of relationships between variables, a simple channel-independent strategy combined with a basic backbone, like CycleNet, still struggles to fully meet the demands. Therefore, in Appendix C.5, we further analyze the current limitations of the current RCF technique in spatiotemporal scenarios (such as the traffic domain) and and point out potential directions for future improvements. Finally, we also provide a comparison of CycleNet with existing models on full datasets using longer look-back windows in Appendix C.2. 
discussion: 
 Potential limitations CycleNet demonstrates its efficacy in LTSF scenarios characterized by prominent and explicit periodic patterns. However, there are several potential limitations of CycleNet that warrant discussion here:• Unstable cycle length: CycleNet may not be suitable for datasets where the cycle length (or frequency) varies over time, such as electrocardiogram (ECG) data, because CycleNet can only learn a fixed-length cycle.• Varying cycle lengths across channels: When different channels within a dataset exhibit cycles of varying lengths, CycleNet may encounter challenges because it defaults to modeling all channels with the same cycle length W . Given CycleNet's channel-independent modeling strategy, one potential solution is to pre-process the dataset by splitting it based on cycle lengths or to independently model each channel as a separate dataset.• Impact of outliers: If the dataset contains significant outliers, CycleNet's performance may be affected. This is because the fundamental working principle of RCF is to learn the historical average cycles in the dataset. When significant outliers exist, the mean of a certain point in the cycle learned by RCF can be exaggerated, leading to inaccurate estimation of both the periodic and residual components, which subsequently impacts the prediction process.• Long-range cycle modeling: The RCF technique is effective for modeling mid-range stable cycles (e.g., daily or weekly). However, considering longer dependencies (such as yearly cycles) presents a more challenging task for the RCF technique. Although, in theory, CycleNet's W can be set to a yearly cycle length to model annual cycles, the biggest difficulty lies in collecting sufficiently long historical data to train a complete yearly cycle, which might require decades of data. In this case, future research needs to develop more advanced techniques to specifically address long-range cycle modeling.Future work: further modeling inter-channel relationships The RCF technique enhances the model's ability to model the periodicity of time series data but does not explicitly consider the relationships between multiple variables. In some spatio-temporal scenarios where spatial and temporal dependencies between variables exist, these relationships are crucial. For example, recent studies such as iTransformer [37] and SOFTS [12] indicate that appropriately modeling inter-channel relationships can improve performance in traffic scenarios. However, directly applying the RCF technique to iTransformer does not lead to significant improvement (at least for the MSE metric), as demonstrated in Table 4 . We believe that devising a more reasonable multivariate modeling approach that combines CycleNet could be promising and valuable, and we leave it for future exploration. 
conclusion: 
 This paper reveals the presence of inherent periodic patterns in time series data and pioneers the exploration of explicitly modeling this periodicity to enhance the performance of time series forecasting models. Technically, we propose the Residual Cycle Forecasting (RCF) technique, which models the shared periodic patterns in sequences through recurrent cycles and predicts the residual cyclic components via a backbone. Furthermore, we introduce the simple yet powerful LTSF methods CycleNet/Linear and CycleNet/MLP, which combine single-layer Linear and dual-layer MLP respectively with the RCF technique. Extensive experiments demonstrate the effectiveness of the RCF technique, and CycleNet as a novel and simple method achieves state-of-the-art results with significant efficiency advantages. The findings in this paper underscore the importance of periodicity as a key characteristic for accurate time series prediction, which should be given greater emphasis in the modeling process. Finally, integrating CycleNet with effective inter-channel relationship modeling methods serves as a promising and valuable future research direction. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: We report the computational resource requirements of our proposed method in Table 3 .Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: The paper focuses on advancing the field of machine learning. While our work may have various societal implications, we believe none are significant enough to warrant specific mention here.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The code and datasets used in the paper are publicly available and properly credited.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: In recent years, the time series analysis community has shifted its focus from short-term forecasting to tasks with longer prediction horizons, also known as LTSF tasks . This shift offers greater convenience but also poses increased challenges. Mainstream approaches can be roughly classified into the following five distinct classes:Transformer-based Models It is widely recognized that Transformers possess impressive capabilities for long-distance modeling, and thus researchers have high expectations for their adaptation to long time series tasks [46, 49] . Early research works, such as LogTrans [25] , TFT [29] , Informer [59] , Autoformer [51] , Pyraformer [35] , FEDformer [60] , ETSformer [50] , and NSTransformer [36] , focused on optimizing the original Transformer architecture for time series analysis tasks. However, more recent research has found that satisfactory performance can be achieved by simply partitioning patches, drawing inspiration from patch techniques used in the computer vision community [7, 14] ). Approaches like PatchTST [40] , PETformer [30] , and Crossformer [58] have demonstrated promising results by adopting this patch-based approach.Linear-and MLP-based Models Linear-and MLP-based methods are often lighter-weight, especially compared to Transformer methods that require stacking multiple blocks [57, 4] . A particularly notable breakthrough is the observation made by DLinear [56] , which demonstrates that a single-layer linear approach could outperform many complex Transformer designs. This observation leads to a sequence of works, including TiDE [5] , MTS-Mixers [28] , TSMixer [28] , TimeMixer [48] , HDMixer [16] , SOFTS [12] , FITS [53] , SparseTSF [32] , and SSCNN [6] . The proposed CycleNet in this paper is also a Linear-or MLP-based model that is simple, efficient, and powerful for time series forecasting.RNN-based Models Conceptually, Recurrent Neural Networks (RNN) are considered to be the most suitable models for modeling time series data [24, 43] . However, due to difficulties in parallelization and modeling long sequences, RNNs are not the most popular choice in works on LTSF tasks. Recent works aim to revitalize RNN models in long sequence modeling tasks, such as SegRNN [31] , WITRAN [19] , SutraNets [2] , and RWKV-TS [15] .Because of the parallelizability of convolution operations and their ability to capture features at different time scales, Temporal Convolutional Networks (TCN) methods are considered strong competitors for addressing time series tasks [1, 9] . Recent works that apply TCN methods to LTSF tasks include SCINet [34] , MICN [47] , TimesNet [52] , PatchMixer [10] , and ModernTCN [38] .The remarkable capabilities demonstrated by large language models (LLM) have sparked interest among researchers from various fields, including those working on time series forecasting tasks [21, 18] . Some works consider fine-tuning pre-trained LLMs to perform time series analysis tasks, including OFA [61] , Time-LLM [20] , and TEMPO [3] . Other works aim to achieve zero-shot inference using large pre-trained LLMs through prompt engineering, including LLMTime [11] , PromptCast [54] , and LSTPrompt [33] .Algorithm 1 demonstrates the implementation of modeling periodic patterns through recurrent cycles. Specifically, the first line defines the learnable parameter queue Q and initializes it to zero. Lines 2-11 define the getCycle function, which will be called by CycleNet to obtain the corresponding truncated equivalent cyclic subsequences. This function takes two parameters, i and l, where i represents the relative positional index for Q, and l represents the length of the required subsequence. Q learns internal periodic patterns within the sequence through co-training with the backbone.Furthermore, Algorithm 2 illustrates the workflow of CycleNet. The first step is to normalize the samples based on their mean and standard deviation, then call the getCycle function to remove the cyclic components of the input data. Subsequently, predict the residual components through the backbone. Finally, add back the cyclic components of the output data, and perform instance denormalization to obtain the final prediction result. Here, the cycle index i corresponds to t mod W , as described in Section 3.1.Require: Number of channels D and cycle length W Ensure:Learned periodic patterns Q ∈ R W ×D 1: Initialize learnable parameters Q ← 0 ▷ Q ∈ R W ×D 2: function GETCYCLE(i, l) ▷ Define function 3: Q ′ ← Roll(Q, shifts = -i, dim = 0)▷ Roll the queue to the appropriate index 4:if l < W then ▷ Retrieve the required part directly from Q ′ 5:return Q ′ 0:l 6:else ▷ Repeat Q ′ to match the required length 7:n ← ⌊l/W ⌋ 8: d ← l mod W 9: return Concat([Q ′ ] × n, [Q ′ 0:d ]) ▷ Concatenate replicated Q ′ andThe RCF technique utilizes recurrent cycles Q ∈ R W ×D to model the internal periodic patterns of sequences. Here, the hyperparameter W determines the length of the recurrent cycles, which should precisely match the length of the periodic patterns within the data. As shown in the results of Table 6 , when W is not accurately set, the RCF technique fails to fulfill its intended purpose. Although, in practice, we can infer the maximum cycle length of the dataset by considering the data's sampling frequency and the potential existing periodic patterns (as shown in Table 1 ), this manual inference method may introduce errors. Therefore, we may need a more scientific and precise approach to find the hyperparameter W .In such cases, the autocorrelation function (ACF) [39] serves as a powerful mathematical tool to help us determine the periodicity within the data. The autocorrelation function measures the correlation between a time series and its lagged values, indicating the presence of autocorrelation within the data. Mathematically, this can be expressed as:EQUATION)where N represents the total number of observations, x t denotes the value of the time series at time t, k is the lag time, and x is the mean of the time series values.Here, when the lag time k aligns with the data's cycle, the ACF value exhibits a significant peak. Specifically, the largest peak corresponds to the lag that aligns with the length of the maximum cycle present in the dataset. Conversely, if the data lacks periodicity, no significant peaks or troughs will be observed.We present the ACF results for each dataset in Figure 6 . It can be observed that these datasets all display evident periodicity, indicated by prominent peaks and troughs in the plots. More importantly, the maximum cycles shown in the plots align with the pre-inferred cycle lengths from Table 1 . This indicates the correctness of the pre-inferred lengths, and W should be strictly set to these values.We utilized widely used benchmark datasets for LTSF tasks, including the ETT series, Electricity, Solar-Energy, Traffic, and Weather. Following prior works such as Autoformer [51] and iTransformer [37] , we split the ETTs dataset into training, validation, and test sets with a ratio of 6:2:2, while the other datasets were split in a ratio of 7:1:2.We implemented CycleNet using PyTorch [41] and conducted experiments on a single NVIDIA RTX 4090 GPU with 24GB of memory. CycleNet was trained for 30 epochs with early stopping based on a patience of 5 on the validation set. The batch size was set uniformly to 256 for ETTs and the Weather dataset, and 64 for the remaining datasets. This adjustment was made because the latter datasets have a larger number of channels, requiring a relatively smaller batch size to avoid out-of-memory issues. The learning rate was selected from the range {0.002, 0.005, 0.01} based on the performance on the validation set. The hyperparameter W was set consistently to the pre-inferred cycle length as shown in Table 1 . Additionally, the hidden layer size of CycleNet/MLP was uniformly set to 512. By default, CycleNet uses RevIN without learnable affine parameters [22] . However, we found that on the Solar dataset, using RevIN leads to a significant performance drop, as shown in Appendix C.4. The primary reason for this may be that photovoltaic power generation data contains continuous segments of zero values (no power generation at night). When the look-back windows are not an integer multiple of a day, the calculation of means in RevIN can be significantly affected, leading to decreased performance. Therefore, for this dataset, we did not apply the RevIN strategy.The proposed RCF technique can effectively learn the inherent periodic patterns within time series data. This capability is a significant advantage, revealing the potential value of RCF or its underlying cyclic modeling approach as a superior method to assist data engineers in analyzing patterns in time series data. To further elucidate the working principle behind RCF, we delve into the periodic patterns learned by the RCF technique under different configurations, as illustrated in Figure 7: • Forecast horizon H: The learned patterns remain almost unchanged as the horizon length varies. This indicates that the horizon length does not affect the learned pattern results. • Look-back length L: The overall pattern remains unchanged as the look-back window changes. However, with closer observation, it is noticeable that the learned pattern becomes smoother with an increased look-back. This is because a longer look-back provides the backbone with richer periodic information, thereby reducing the reliance on the learned pattern component. • Backbone: The patterns vary somewhat with different backbones. When DLinear is used as the backbone, the learned patterns are smoother, as DLinear's decomposition technique itself extracts certain periodic features. When iTransformer is the backbone, the learned patterns differ more, as it additionally models multichannel relationships, so the learned periodic patterns may consider multichannel feature interactions. PatchTST's performance is more similar to that of Linear, as it is also a regular single-channel modeling method, though with stronger nonlinear learning capabilities compared to the Linear model. • Cycle length W : When W is set to 168 (the weekly cycle length for the Electricity dataset), the recurrent cycle Q learns the complete periodic pattern, including both weekly and daily cycles. When W is set to 24 (the daily cycle length), the recurrent cycle Q only learns the daily cycle pattern. When W is set to 96 (four times the daily cycle length), the recurrent cycle Q learns four repeated daily cycle models. However, when W is set to 23 (without matching any semantic meaning), the recurrent cycle Q fails to learn any meaningful pattern, resulting in a straight line. ). The results of other models are sourced from iTransformer [37] .The best results are highlighted in bold and the second best are underlined.showcase the complete comparison results for different forecast horizons in Table 7 . It can be observed that in most settings, CycleNet achieves state-of-the-art results, consistent with the findings in Table 2 . Additionally, the standard deviation of CycleNet's results is mostly below 0.001. This strongly indicates the robustness of CycleNet.Additionally, the look-back length is a crucial hyperparameter that significantly impacts the performance of time series forecasting models, as it determines the richness of information the model can leverage. Initially, the community focused primarily on exploring the application of Transformers in time series forecasting tasks. Due to the inherent complexity of Transformers, using excessively long look-back windows resulted in a significant increase in runtime. As a result, many popular models at the time, such as Informer [59] , Autoformer [51] , and FEDformer [60] , employed shorter look-back windows, typically with L = 96. With the recent development of model lightweighting techniques, particularly the adoption of channelindependent strategies (first applied in DLinear [56] and PatchTST [40] ), more models have started to experiment with longer look-back windows in pursuit of higher predictive accuracy. For instance, DLinear and PatchTST default to using look-back windows of L = 336, while SegRNN [31] and SparseTSF [32] default to using L = 720. To explore CycleNet's performance with longer look-back windows, we compared CycleNet with these advanced models using their respective default, longer look-back windows in Table 8 .It is important to note that we re-ran the official open-source code of these baselines to obtain the corresponding results, using the same MSE as the loss function (as SegRNN originally used MAE as its loss). Additionally, there was a long-standing bug in their original repositories, where the data from the last batch was discarded during testing [42, 53] . This issue could have affected the model's performance, so we fixed this problem before re-running the experiments.It can be observed that even with a longer look-back length, CycleNet generally maintains a significant advantage, achieving state-of-the-art performance in most scenarios. This demonstrates CycleNet's excellent performance across different look-back lengths. It is worth noting that both PatchTST and techniques. We previously reported the mean performance of these techniques across different horizons H ∈ {96, 192, 336, 720} in Table 5 . Here, we further present the complete comparative results for all horizons in Table 9 .The results show that the RCF technique consistently outperforms other techniques. A notable exception is the relatively noisy weather dataset, where RCF does not show a significant advantage. However, in this case, the performance of several STD techniques is similar to that of the pure Linear model. Overall, these findings strongly support RCF as a new STD method that enhances model performance in scenarios with strong periodicity. Instance normalization strategies constitute essential factors for the success of current models, such as PatchTST [40] , TiDE [5] , iTransformer [37] , SparseTSF [32] , etc. By default, CycleNet also adopts this strategy, namely the version of RevIN without learnable affine parameters [22] . Here, we meticulously investigate the impact of RevIN on the performance of CycleNet, and the results are shown in Table 10 . On the ETTh2 and Weather datasets, RevIN significantly enhances the performance of CycleNet, possibly due to more severe distribution drift issues in these datasets. However, on the Solar dataset, RevIN leads to poorer performance, likely because the photovoltaic power generation data contains continuous segments of zero values (no power generation at night), which significantly affects the calculation of means in RevIN.Overall, in most cases, RevIN leads to better performance. We acknowledge that RevIN is an indispensable cornerstone of CycleNet's success, but it is not the key factor that sets CycleNet apart from other models in terms of performance. As shown in the comparison results in Table 10 , CycleNet exhibits a significant advantage over RLinear and RMLP, which can be viewed as CycleNet without RCF technique. This clearly demonstrates that the RCF technique is the key factor that significantly enhances the model's prediction accuracy, constituting the core contribution of this paper. CycleNet, formed by combining the RCF technique with a simple backbone, achieved state-of-the-art performance across multiple domains but fell short in the traffic domain. To further investigate the reasons behind this, we supplemented the complete performance of CycleNet on the PEMS dataset (the same four public subsets adopted in SCINet [34] ) in Table 11 . The results show that: (i) CycleNet still achieved top-tier prediction accuracy, and (ii) although CycleNet underperformed compared to iTransformer in this scenario, the gap in MSE on the Traffic dataset was reduced from approximately 10% to about 5%.Regarding the first point, it is important to highlight the effectiveness of RCF. CycleNet's backbone is merely a single-layer Linear or a two-layer MLP, without any additional design or deep stacking, yet it still delivers excellent results. Specifically, when comparing CycleNet/Linear with RLinear and DLinear, it becomes evident that RCF is the major contributor to narrowing the gap between the simple Linear model and those state-of-the-art models. For the second point, we further analyzed the statistical characteristics of the datasets to explore the underlying reasons in Table 12 . Specifically, we examined the presence of extreme values in the channels and the cosine similarity between channels. It was found that the Traffic dataset contains very significant outliers, both in terms of quantity and magnitude. The presence of these outliers:(i) May affect the effectiveness of RCF. The fundamental working principle of RCF is to learn the historical average cycles in the dataset. In such cases, the average cycles learned in RCF can be skewed by these significant outliers, such as the mean of a certain point in the cycle being exaggerated.Consequently, during each prediction process, the original sequence subtracts a locally exaggerated average cycle, resulting in an inaccurate residual component and affecting the local point predictions within each cycle. The more inaccurate these local point predictions are, the larger the discrepancy between MSE and MAE, as MSE significantly amplifies the impact of a few large errors. This explains why in Table 4 , combining iTransformer with RCF decreases MAE but increases MSE, indicating overall prediction accuracy improvement but anomalies in local point predictions.(ii) Highlight the necessity of stronger spatiotemporal relationship modeling. Models like iTransformer and GNN, which accurately model inter-channel relationships, are more suitable for scenarios with extreme points and temporal lag characteristics. For example, when a sudden traffic surge occurs at a certain junction, these models, having correctly modeled the spatiotemporal relationships, can accurately predict possible traffic surges at other junctions. In contrast, the current CycleNet only considers single-channel relationship modeling, making it somewhat limited in this scenario.These underlying reasons explain why CycleNet did not achieve the best performance on the Traffic dataset and showed a relative large performance gap. On the PEMS dataset, although it is also a traffic dataset, the presence of extreme points is significantly less severe compared to the Traffic dataset. Therefore, CycleNet's performance on the PEMS dataset improved compared to the Traffic dataset (the gap in MSE compared to the state-of-the-art reduced from approximately 10% to about 5%). This further validates the effectiveness of RCF but also indicates that in more complex traffic scenarios, reasonable spatiotemporal relationship modeling (or multivariate relationship modeling) is essential.Additionally, while intuitively the solar scenarios might also involve significant spatiotemporal relationships, in practice, these relationships are much weaker compared to the traffic scenarios. Firstly, the weather conditions in the same region are often similar, leading to similar power generation curves. For instance, the Solar-Energy dataset's channels have a cosine similarity as high as 0.92 (shown in Table 12 ), which indirectly indicates weaker spatial characteristics. Secondly, extreme points are rare in the solar scenarios because photovoltaic systems have a maximum power threshold. Fewer extreme points mean that the impact of temporal lag characteristics is smaller. This explains why, compared to the Traffic dataset, the gains from the RCF technique are much more significant on the Solar-Energy dataset.In summary, when dealing with traffic scenarios that may involve significant outliers and emphasize spatiotemporal relationship modeling, the current version of CycleNet may not be fully adequate.There are two direct and meaningful directions for improvement that could address this issue: (1) Enhancing the current RCF technique to be more robust to the presence of outliers; (2) Exploring a more reasonable multi-channel modeling technique within the RCF framework. We leave these challenges for future work and encourage the community to further research more robust and powerful periodic modeling techniques.NeurIPS Paper ChecklistJustification: Our research aligns with the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 


Decision: This paper has been assessed by four knowledgeable reviewers who agreed on accepting it. Two of the reviewers gave it strong accept ratings, one marked it as straight accept and one as weak accept. The paper presents a novel yet simple approach to time series  decomposition that can be used for long term forecasting of time series. It is well written and well structured and avoids most of the common pitfalls in how the ideas are presented and supported with empirical evidence. The authors provided a thorough rebuttal and engaged the reviewers in focused discussions.

Reviews: review: paper_summary: paper_summary: This paper proposes a learnable Seasonal-Trend Decomposition method (CycleNet) to improve the prediction performance of current long-term multivariate time series forecasting models. Specifically, it firstly model the periodic patterns of sequences through globally shared recurrent cycles and then predicts the residual components of the modeled cycles. Extensive experiments are conducted to evaluate the proposed method.
strength_weakness: strength_weakness: 1. It seems that the proposal (CycleNet) does not work well for complex datasets, e.g., Traffic. It is better to show more results on the same kind of datasets like the PEMS datasets used in the iTransformer paper.
2. The Section 3 is not well written and lacks a lot of details, especially about the Learnable Recurrent Cycles. The authors may consider to reorganize Section 3 and Appendix A.1 to make Section 3 more clear.
questions: questions: Please refer to the weaknesses.
limitations: limitations: Yes
paper_summary: paper_summary: This paper presents a novel technique for improving the accuracy of multivariate long-term time series forecasting. The technique, called Residual Cycle Forecasting (RCF), involves learning the cyclical patterns of time series through recurrent cycles, which can be used as a pre-processing step for any forecasting model. The authors also propose CycleNet, a linear-based model that uses RCF to enhance its predictions. CycleNet first uses RevIn to account for distribution shift, then subtracts the learned RCF from the input data. The backbone of the model predicts the future residual, adds the learned RCF, and reverses RevIN from the outputs to obtain the final prediction. 

The proposed method is evaluated on eight multivariate time series datasets and compared against several baselines. The results show competitive performance and resource consumption.
strength_weakness: strength_weakness: * Lack of clarity to specify which results are from the authors (reproduced or produced) and which ones are collected from previous papers (if so, which ones)
 * Incomplete comparison with existing time series decomposition baselines, such as LD, TDFNet, or SparseTSF (even though this one was in the related works)
 * Model was not compared to RLinear (also relying on RevIN)
questions: questions: ## Baselines

The paper fails to properly compare the proposed CycleNet model with appropriate baselines.

Since CycleNet is based on RevIN, it would be more complete to include other RevIN-based baselines such as RLinear [1] for linear-based models and for instance RevInformer (from the RevIN paper) for transformer-based solutions. Although PatchTST uses RevIN, including these additional baselines would avoid any doubt and provide a more comprehensive comparison.

Additionally, the paper misses important baselines such as LD [2], which also uses RevIn and also argues for using learnable decomposition rather than moving average. According to the published results, CycleNet appears to be behind LD. But it would be interesting to compare LD against backbone + RCF.
Potential references to [3] is also missing.

[1] https://arxiv.org/pdf/2305.10721

[2] https://arxiv.org/abs/2402.12694

[3] https://arxiv.org/pdf/2308.13386

Finally, despite being describes as main baselines, why SparseTSF was not included in this paper? Especially, as SparseTSF seems to produce better results than proposal especially for ETT and not for Electricity and Traffic. What would be the reason for such differences?

Including such comparisons is required to correctly position CycleNet with all the other baselines and further discuss why CycleNet offers better performance with some datasets and not others. Such discussion is important to further investigate the benefit of time series decomposition for forecasting tasks.

## Visualization
The visualizations of the learned cycles are informative, but it is unclear whether these cycles change depending on the prediction horizon or lookback. If they do, it would be interesting to see the differences and whether they can explain the variation in performance for different prediction horizons.

Since the cycles are learned along with the backbone model, there could be differences in the learned cycles for different experiments. It would be helpful to plot the cycles for different prediction lengths, such as 96 and 720, and for different lookback lengths, such as 48, 96, and 336. 

In addition, the cycle may change depending on the backbone. It would be interesting to show the difference if any when RCF is used with iTransformer, DLinear, etc. This would allow readers to better understand whether there are any differences in the learned cycles and whether they are due to the parameters, backbone model or other factors.

Figure 3(f) is surprising, and it is unclear whether it represents a household or an industrial building with specific working hours. In my opinion, it looks like there are only 3 main weekly patterns (the large pattern on the left seems to have ~50 time steps = 2 days) and weekends (one pattern seems to have ~24 time steps = 1 day). Therefore, it could be a factory or industrial building with specific working hours (6 days a week).

It would be interesting to discuss how RCF impacts previous backbones, such as DLinear. Specifically, it would be helpful to know how much DLinear's performance changes with the addition of RCF and plot it similarly to Figure 4.

Finally, it is important to provide Figure 4 and Table 4 for other datasets to give readers a global view of the impact of RCF/CycleNet depending on the dataset and its nature. This would help avoid generalizations that could be incorrect based only on results depicted in the current version, especially for datasets such as Traffic and Solar Energy where the impact might be drastically different.

## Discussion
“Traffic dataset exhibits spatiotemporal characteristics and temporal lag characteristics, where the traffic flow at a certain detection point significantly affects the future values of neighboring detection points.” However, the authors do not explain why this is not an issue for the Solar-Energy dataset, where weather conditions at one location may slowly impact neighboring locations in the future. Can authors comment on that?

Regarding the electricity consumption dataset, the authors state that "a user’s electricity consumption thirty days ahead does not directly correlate with their consumption patterns in the past few days." However, it is possible that long-term habits may still influence consumption patterns, for instance with a monthly routine.

The authors suggest that "the cycle length W depends on the a priori characteristics of the dataset and should be set to the maximum stable cycle within the dataset." However, they do not address how to account for yearly cycles, such as those found in weather datasets that repeat each year with the seasons or datasets influenced by human behavior, such as electricity consumption, which may increase in winter and summer due to heating and cooling needs. It would be helpful to clarify whether RCF is only suitable for mid-range stable cycles or if there is a way to account for longer cycles. This might need to be included in the limitations section.

## Additional points
The paper lacks a computation cost study to determine the overhead imposed by RCF on the existing backbone in terms of memory and consumption. It is important to evaluate whether the gain in accuracy justifies the increase in complexity and associated cost.

## Proof-read
 * “a single-layer Linear or a dual-layer MLP” maybe find another way to express it to avoid repeating too much (cf., abstract, introduction, etc.)
 * “surpassing complexly designed deep models”->  redundant, ”complex design” or “deep models”
 * “leveraging this discovery to enhance” should change this as the fact that time series have cycle is not new leveraging this knowledge is the innovation
limitations: limitations: Authors have discussed some limitations of their proposal and especially when each channel have different period cycle.
paper_summary: paper_summary: The paper introduces CycleNet, a novel time series forecasting method that enhances long-term prediction accuracy by explicitly modeling the inherent periodic patterns present in time series data. The core contribution of paper is introducing the Residual Cycle Forecasting (RCF) technique, which leverages learnable recurrent cycles to represent these periodic patterns and predicts the residuals, significantly improving upon the performance of existing models with reduced computational complexity. CycleNet demonstrates state-of-the-art results across various domains, such as electricity, weather, and energy forecasting, while offering over 90% reduction in parameter quantity, highlighting its efficiency and effectiveness in capturing long-term dependencies for accurate forecasting.
strength_weakness: strength_weakness: 1. In the introduction, the author's statement establishes a close relationship between long-term prediction and periodic information. In the absence of some experimental support, this is not rigorous. Periodic information may be useful for long-term forecasting in certain situations, but it is not appropriate for all tasks, nor is it the only important information that these tasks require. The author slightly obfuscates these to highlight the motivation of this article.

2. The authors select data sets with different periodicity to illustrate the validity of the model. However, the authors only demonstrate the validity, and experiments can be added to show how the proposed method performs differently on periodically different data sets and discuss the underlying rules. At the same time, it is also worth showing how CycleNet performs on data sets where there is no obvious periodicity.

3. Combined with the results in Table 2 and Table 4, the results predicted with Linear alone are even better than some well-designed methods, whether this is due to the particularity of the data set or some other reason.
questions: questions: Please refer to the weakness
limitations: limitations: Please refer to the weakness
paper_summary: paper_summary: This paper proposes a novel and effective technique to enhance long-term time series forecasting. The proposed technique, Residual Cycle Forecasting (RCF), directly models periodic cycles with learnable parameters, decomposing the learning of time series into periodic cycles and residual components. The residual can be learned with a simple architecture like Linear or MLP, and this technique can be integrated into existing forecasting models. Results show significant improvement achieved by using this technique.
strength_weakness: strength_weakness: The notations are not strictly consistent. The notations for instance normalization (in Section 3.2 and Algorithm 2) are independent from the whole framework and not consistent with the previous problem definition.
questions: questions: 1. Since Figure 2 depicts the entire CycleNet architecture, instance normalization should also be included.

2. The description of alignment and repetition of $Q$ (Lines 152-157) can be further improved. The link to Algorithm 1 should be added, and a schematic figure (even in Appendix) would make it clearer to understand.

3. How extreme points affect RCF in the Traffic dataset should be further explained (Line 255). Extreme points can also affect other methods. It is unclear why this would specifically deteriorate RCF.
limitations: limitations: See Weaknesses and Questions

Comments: title: Additional Rebuttal (Optional Reading) # Part 2
Additional Rebuttal (Optional Reading) # Part 1
comment: Dear reviewer, thank you very much! We will do our best to incorporate the mentioned new results and analyses into the main text or the appendix. Once again, we are very grateful for your detailed and constructive review!
> Added visualizations

Thank you, these are very interesting results. Again, make sure to include them as it might be very useful for other researchers tackling cyclical patterns or decomposition of time series data. 

> We will include full experiments on more datasets in the revised paper.

Much appreciated.

> It can be seen that the power generation curves of different channels in the Solar-Energy dataset are very similar, which indirectly indicates weaker spatial characteristics.

> It can be seen that the number of extreme points in the Traffic dataset is significantly higher than in other datasets.

These are very compelling explanations. Thank you for computing them and enlightening me as well as future readers.
The discussion regarding the limitations of existing solutions, as well as your proposal, is very interesting. Please ensure that you include them in your final revision.

> We will include this in the limitations section.

Thank you!
> Other researchers have also noted this phenomenon on the LD project's GitHub (Issue #1), but have not yet received a response from the authors. 

I was not aware of the existing issue with LD. It is good to include these results in your final revision and mention the existing reproducibility issue with LD. Such information will be helpful for future readers.
Thank you for your extensive effort during this rebuttal phase and for providing such detailed responses to my comments and those of other reviewers.
Overall, the authors have addressed all my points, and I still do not see any issues preventing the acceptance of this paper.
Even though it is a difficult task, please ensure that all the outputs provided during the rebuttal and discussion phase are added to the main paper (or at least a quick summary of them, with a reference to the appendix). These details are crucial for future readers to fully understand your work, its advantages compared to existing solutions, its limitations, and areas that require further investigation.

In light of this, I have increased my score from 7 to 8.
Dear reviewer, thank you for your timely response. Once again, we sincerely thank you for taking the time to review our paper!
Thanks for the response. I will keep my original score.
**Dear Reviewer zzRb,**

Sorry to bother you. **We are eager to know whether our response has addressed your concerns as the discussion phase is nearing its end.** *If not, or if you have any additional questions, we would be more than happy to further address them.* Thank you for your time.
**Dear Reviewer 24Zo,**

Sorry to bother you. **We are eager to know whether our response has addressed your concerns as the discussion phase is nearing its end.** *By the way, we would like to further include the remaining results of other datasets for Table 1 of the attached PDF,* which were not included initially due to space constraints. Thank you for your time.

|             |      | RCF+Linear |           | LD+Linear |           | MOV+Linear |           | Sparse+Linear |           |  Linear   |           |
| :---------: | :--: | :--------: | :-------: | :-------: | :-------: | :--------: | :-------: | :-----------: | :-------: | :-------: | :-------: |
|             |      |    MSE     |    MAE    |    MSE    |    MAE    |    MSE     |    MAE    |      MSE      |    MAE    |    MSE    |    MAE    |
|    ETTh2    |  96  |   0.308    |   0.369   | **0.292** | **0.357** |   0.297    |   0.362   |     0.340     |   0.389   |   0.305   |   0.368   |
|             | 192  |   0.382    |   0.416   | **0.372** | **0.409** |   0.398    |   0.426   |     0.379     |   0.413   |   0.385   |   0.419   |
|             | 336  |   0.454    |   0.465   |   0.479   |   0.480   |   0.496    |   0.489   |   **0.404**   | **0.437** |   0.458   |   0.470   |
|             | 720  | **0.661**  | **0.575** |   0.675   |   0.582   |   0.694    |   0.592   |     0.720     |   0.600   |   0.691   |   0.592   |
|    ETTm1    |  96  | **0.298**  |   0.350   |   0.305   |   0.350   |   0.309    |   0.356   |     0.306     | **0.349** |   0.305   |   0.349   |
|             | 192  | **0.330**  |   0.370   |   0.335   | **0.366** |   0.346    |   0.380   |     0.339     |   0.370   |   0.338   |   0.369   |
|             | 336  | **0.359**  | **0.388** |   0.372   |   0.390   |   0.373    |   0.391   |     0.372     |   0.389   |   0.371   |   0.389   |
|             | 720  | **0.410**  | **0.421** |   0.445   |   0.443   |   0.439    |   0.435   |     0.430     |   0.426   |   0.433   |   0.428   |
|    ETTm2    |  96  | **0.164**  |   0.260   |   0.165   | **0.257** |   0.165    |   0.257   |     0.177     |   0.272   |   0.166   |   0.259   |
|             | 192  | **0.225**  | **0.304** |   0.240   |   0.318   |   0.232    |   0.310   |     0.246     |   0.325   |   0.228   |   0.305   |
|             | 336  | **0.271**  | **0.332** |   0.290   |   0.349   |   0.295    |   0.356   |     0.309     |   0.370   |   0.275   |   0.334   |
|             | 720  |   0.406    |   0.423   | **0.396** | **0.419** |   0.427    |   0.442   |     0.427     |   0.440   |   0.407   |   0.425   |
| Electricity |  96  | **0.131**  | **0.228** |   0.140   |   0.237   |   0.140    |   0.237   |     0.148     |   0.243   |   0.140   |   0.238   |
|             | 192  | **0.145**  | **0.242** |   0.154   |   0.250   |   0.154    |   0.250   |     0.159     |   0.254   |   0.154   |   0.251   |
|             | 336  | **0.160**  | **0.260** |   0.170   |   0.268   |   0.169    |   0.268   |     0.173     |   0.271   |   0.170   |   0.269   |
|             | 720  | **0.193**  | **0.292** |   0.204   |   0.300   |   0.204    |   0.301   |     0.207     |   0.303   |   0.204   |   0.301   |
|    Solar    |  96  | **0.192**  | **0.251** |   0.222   |   0.294   |   0.222    |   0.298   |     0.226     |   0.296   |   0.224   |   0.302   |
|             | 192  | **0.218**  | **0.258** |   0.249   |   0.315   |   0.250    |   0.312   |     0.252     |   0.312   |   0.250   |   0.310   |
|             | 336  | **0.231**  | **0.262** |   0.268   |   0.326   |   0.270    |   0.335   |     0.270     |   0.326   |   0.269   |   0.325   |
|             | 720  | **0.239**  | **0.265** |   0.271   |   0.327   |   0.272    |   0.327   |     0.271     |   0.327   |   0.270   |   0.333   |
|   Traffic   |  96  | **0.397**  | **0.275** |   0.411   |   0.285   |   0.411    |   0.284   |     0.414     |   0.281   |   0.411   |   0.283   |
|             | 192  | **0.412**  | **0.282** |   0.423   |   0.288   |   0.423    |   0.289   |     0.425     |   0.285   |   0.423   |   0.289   |
|             | 336  | **0.426**  | **0.290** |   0.436   |   0.296   |   0.436    |   0.296   |     0.436     |   0.293   |   0.437   |   0.297   |
|             | 720  | **0.456**  | **0.308** |   0.466   |   0.315   |   0.466    |   0.316   |     0.464     |   0.310   |   0.466   |   0.316   |
|   Weather   |  96  | **0.174**  |   0.240   |   0.174   | **0.235** |   0.177    |   0.241   |     0.177     |   0.238   |   0.176   |   0.238   |
|             | 192  |   0.218    |   0.279   |   0.227   |   0.292   |   0.222    |   0.283   |     0.224     |   0.287   | **0.217** | **0.276** |
|             | 336  |   0.262    |   0.314   |   0.270   |   0.326   | **0.261**  | **0.312** |     0.264     |   0.316   |   0.271   |   0.326   |
|             | 720  |   0.328    |   0.367   |   0.341   |   0.386   |   0.330    |   0.370   |   **0.326**   |   0.366   |   0.333   |   0.373   |
Thank you very much! We will include the complete results in the final paper. Once again, thank you for taking the time to review our paper and for improving our rating!
Thank your for the results. I have updated my rating. Please include the full results for readers to have a more complete understanding of the proposal.
Due to length constraints, the results are attached here:



|        |      | CycleNet/MLP  |           | iTransformer |           |  SCINet   |           | CycleNet W/o. RCF |       | iTransformer W/. RCF |           |
| :----: | :--: | :-------: | :-------: | :----------: | :-------: | :-------: | :-------: | :---------------: | :---: | :------------------: | :-------: |
|        |      |    MSE    |    MAE    |     MSE      |    MAE    |    MSE    |    MAE    |        MSE        |  MAE  |         MSE          |    MAE    |
| PEMS03 |  12  |  *0.066*  |   0.172   |    0.071     |   0.174   |   0.066   |  *0.172*  |       0.077       | 0.186 |      **0.064**       | **0.170** |
|        |  24  |   0.089   |   0.201   |    0.093     |   0.201   |  *0.085*  |  *0.198*  |       0.116       | 0.228 |      **0.084**       | **0.194** |
|        |  48  |   0.136   |   0.247   |   *0.125*    |  *0.236*  |   0.127   |   0.238   |       0.181       | 0.289 |      **0.116**       | **0.228** |
|        |  96  |   0.182   |   0.282   |   *0.164*    |  *0.275*  |   0.178   |   0.287   |       0.234       | 0.336 |      **0.163**       | **0.268** |
| PEMS04 |  12  |   0.078   |   0.186   |    0.078     |   0.183   | **0.073** | **0.177** |       0.092       | 0.201 |       *0.075*        |  *0.182*  |
|        |  24  |   0.099   |   0.212   |    0.095     |   0.205   | **0.084** | **0.193** |       0.133       | 0.248 |       *0.089*        |  *0.201*  |
|        |  48  |   0.133   |   0.248   |    0.120     |   0.233   | **0.099** | **0.211** |       0.203       | 0.314 |       *0.110*        |  *0.225*  |
|        |  96  |   0.167   |   0.281   |    0.150     |   0.262   | **0.114** | **0.227** |       0.257       | 0.357 |       *0.142*        |  *0.256*  |
| PEMS07 |  12  | **0.062** | **0.162** |    0.067     |   0.165   |   0.068   |   0.171   |       0.073       | 0.177 |       *0.063*        | **0.162** |
|        |  24  |  *0.086*  |   0.192   |    0.088     |  *0.190*  |   0.119   |   0.225   |       0.116       | 0.226 |      **0.078**       | **0.181** |
|        |  48  |   0.128   |   0.234   |   *0.110*    |  *0.215*  |   0.149   |   0.237   |       0.201       | 0.301 |      **0.100**       | **0.202** |
|        |  96  |   0.176   |   0.268   |   *0.139*    |   0.245   |   0.141   |  *0.234*  |       0.287       | 0.364 |      **0.126**       | **0.230** |
| PEMS08 |  12  |   0.082   |   0.185   |   *0.079*    |  *0.182*  |   0.087   |   0.184   |       0.094       | 0.199 |      **0.076**       | **0.179** |
|        |  24  |   0.117   |   0.226   |   *0.115*    |  *0.219*  |   0.122   |   0.221   |       0.151       | 0.255 |      **0.108**       | **0.213** |
|        |  48  | **0.169** |   0.268   |   *0.186*    | **0.235** |   0.189   |   0.270   |       0.231       | 0.312 |        0.188         |  *0.238*  |
|        |  96  |   0.233   |   0.306   |  **0.221**   |  *0.267*  |   0.236   |   0.300   |       0.332       | 0.380 |       *0.226*        | **0.265** |
**Dear Reviewer SdEc,**

Thank you for your questions.

We have provided the full results based on your requested settings. I apologize for the time it took to conduct these experiments. We will include the full results, including comparisons with other models, in the revised paper.

As mentioned earlier, CycleNet is a simple, single-channel modeling method that uses only a Linear layer or a shallow MLP as the backbone. **Its purpose is to validate the proposed RCF technique, demonstrating that even with a simple backbone, combining RCF can achieve state-of-the-art prediction performance** (*except in the traffic scenario*) with very low computational overhead, balancing performance and efficiency.

In scenarios like traffic, where spatiotemporal relationships need to be considered, independent channel modeling methods (including PatchTST, etc.) may struggle to fully capture the dynamics, necessitating additional multichannel relationship modeling techniques, such as those employed by iTransformer. Therefore, it is reasonable that the simple CycleNet has certain limitations in these spatiotemporal scenarios. *We had pointed out these limitations when analyzing the experimental results in the original submission and further elaborated on them in the limitations and future work sections.*

In fact, iTransformer and SCINet are powerful models that achieve the best performance on the PEMS dataset. **The fact that CycleNet can nearly match their performance with just a simple backbone and independent channel modeling is noteworthy.** CycleNet’s backbone is merely a two-layer MLP, without any further design or deep stacking. (Due to length constraints, the results are attached in the next comments.)

Moreover, *when the RCF technique is removed from CycleNet, its performance drops significantly*, demonstrating that RCF is a major contributor to narrowing the gap between the shallow MLP and these state-of-the-art models. *When the proposed RCF technique from CycleNet is integrated into iTransformer*, which already achieves state-of-the-art performance, *iTransformer’s predictive accuracy is further enhanced*.

**Overall, for a basic two-layer MLP backbone, RCF brings about a 28% improvement in MSE and a 16% improvement in MAE.** **For iTransformer, which currently leads in the field, RCF brings an additional 4.9% improvement in MSE and 2.7% in MAE.** *This further validates the effectiveness of our RCF technique, which is our core contribution: a simple and novel method for better extracting periodicity in time series data.* In  addition to improving predictive accuracy, it can also serve as a novel decomposition method for helping us to further analyze the patterns present in time series data (as shown in Figure 3 in the paper).

**I hope this addresses your concerns. Thank you again for your time and for reviewing our paper.**
Thank you for the responses.  Could you give the full results on the PEMS dataset where the lookback length is 96 and the prediction horizon is {12, 24, 48, 96} (only the results of CycleNet, iTransformer and SCINet are OK).

*Sorry, the prediction horizon should be  {12, 24, 48, 96} instead of {12, 24, 36, 48} according to Table 9 in iTransformer's paper.
Thank you very much! We will incorporate the revisions mentioned in the rebuttal into the final paper. Once again, thank you for your careful review and for increasing our score!
Thank you for your detailed response. I have carefully reviewed your rebuttal and the attached PDF. All my questions have been clearly addressed. My rating has been updated.
**First, the spatial characteristics of the solar scenario are weaker than those of the traffic scenario.** In the traffic scenario, traffic flow dynamically changes at each location, with complex internal spatial dependencies. In the solar power generation scenario, as in the Solar-Energy dataset, which records the solar power production of 137 PV plants in Alabama State, the spatial dependencies are much weaker because the weather conditions within a region are usually similar. Even if there are spatial relationships, such as differences in power generation due to longitude differences, they are minor and easily learned. The following table shows the average cosine similarity between different channels in the training set of each dataset (closer to 1 indicates more similar channels):

| Dataset               | Traffic | Electricity | Solar-Energy | ETTh1 |
| --------------------- | ------- | ----------- | ------------ | ----- |
| **Cosine Similarity** | 0.578   | 0.471       | **0.913**    | 0.261 |

*It can be seen that the power generation curves of different channels in the Solar-Energy dataset are very similar, which indirectly indicates weaker spatial characteristics.*

**Second, the solar scenario has fewer extreme points compared to the traffic scenario, so the impact of temporal lag characteristics is smaller.** In the traffic scenario, unexpected situations may cause a sudden increase in flow (i.e., extreme points), and a traffic surge at one intersection can affect the flow at other intersections over time. In this case, adequately modeling inter-channel relationships (i.e., temporal lag) can accurately predict traffic flows at other intersections after an extreme point occurs. In contrast, extreme points are rare in the solar scenario because the power generation of photovoltaic systems has a maximum power threshold. The following table shows the average number of extreme points per channel using Z-Score > 6:

| Dataset            | Traffic | Electricity | Solar-Energy | ETTh1 |
| ------------------ | ------- | ----------- | ------------ | ----- |
| **Extreme Points** | 63.9    | 0.5         | 0            | 0     |

*It can be seen that the number of extreme points in the Traffic dataset is significantly higher than in other datasets.*

> Regarding the electricity consumption dataset, the authors state that "a user’s electricity consumption thirty days ahead does not directly correlate with their consumption patterns in the past few days." However, it is possible that long-term habits may still influence consumption patterns, for instance with a monthly routine.

Thank you for pointing this out. What we intended to convey is more along the lines of "a user’s electricity consumption thirty days ahead ***not only*** correlates with their consumption patterns in the past few days." That is, to achieve accurate long-term predictions, the long-term patterns play a crucial role, not just the short-term fluctuations. **We will revise the original statement.**

> The authors suggest that "the cycle length W depends on the a priori characteristics of the dataset and should be set to the maximum stable cycle within the dataset." However, they do not address how to account for yearly cycles, such as those found in weather datasets that repeat each year with the seasons or datasets influenced by human behavior, such as electricity consumption, which may increase in winter and summer due to heating and cooling needs. It would be helpful to clarify whether RCF is only suitable for mid-range stable cycles or if there is a way to account for longer cycles. This might need to be included in the limitations section.

Thank you for your suggestion. **We will include this in the limitations section.** Considering longer dependencies (such as yearly cycles) is indeed a more challenging task. Although theoretically, CycleNet’s $W$ can be set to a yearly cycle length to model annual cycles, the biggest difficulty lies in collecting sufficiently long historical data to train a complete yearly cycle (possibly requiring decades of data). For example, the Electricity training set covers only 2 years, and the Weather dataset includes less than a year. In this case, other existing models might also struggle to effectively model such long cycles. Therefore, CycleNet is more suitable for mid-range stable cycle modeling, and we believe future research needs to develop more advanced techniques to address this issue specifically.

**Thank you again for your nice review.**
**Dear Reviewer 24Zo,**

Thanks again for you careful review. **It’s the discussion phase now, so we would like to continue to address the uncovered points from the rebuttal phase** (*due to the text length limits in the rebuttal*). *If you find these additional comments any inappropriate (e.g., out of rebuttal limits), please ignore these comments.*

> Finally, despite being describes as main baselines, why SparseTSF was not included in this paper? Especially, as SparseTSF seems to produce better results than proposal especially for ETT and not for Electricity and Traffic. What would be the reason for such differences?

As mentioned in the rebuttal, we noticed SparseTSF, which was recently accepted at ICML 2024, because it emphasizes the importance of periodicity in long-term forecasting tasks, similar to our work. Thus, we included it in the related work but did not perform a complete comparison due to the submission deadline constraints. We will supplement a complete comparison in the revised paper.

Additionally, SparseTSF performs better on the ETT dataset rather than the Electricity and Traffic datasets for two reasons. First, **the ETT dataset is smaller and noisier**. In such cases, the sparse techniques proposed in SparseTSF help the model focus more directly on corresponding elements in historical cycles, reducing interference from less relevant elements and improving overall performance. Second, the ETT dataset's dimensionality (number of channels) is significantly lower than that of the Electricity and Traffic datasets, with the former having only 7 channels compared to the latter's 321 and 862 channels. Note that SparseTSF fundamentally employs a channel-independent modeling approach (parameter-shared) and linear-based methods, which limit its performance on high-dimensional datasets. This is because such scenarios require non-linear capabilities to remember different patterns of multiple channels, or a separate linear layer for each channel (non-parameter-shared) to model each channel's patterns separately [1]. *In contrast, CycleNet’s RCF technique is a fully channel-independent modeling scheme (non-parameter-shared)*, meaning it models each channel's periodic pattern separately. In this case, even with Linear as the backbone, it performs better on high-dimensional datasets because it enhances the model's ability to capture different patterns of multiple channels. Therefore, as shown in our comparison results in *Table 1 of the attached PDF*, **our RCF technique overall still outperforms the Sparse technique** (even on the ETT dataset). *We will include full experiments on more datasets in the revised paper.*

[1] Li, Zhe, et al. "Revisiting long-term time series forecasting: An investigation on linear mapping." *arXiv preprint arXiv:2305.10721* (2023).

> Figure 3(f) is surprising, and it is unclear whether it represents a household or an industrial building with specific working hours. In my opinion, it looks like there are only 3 main weekly patterns (the large pattern on the left seems to have ~50 time steps = 2 days) and weekends (one pattern seems to have ~24 time steps = 1 day). Therefore, it could be a factory or industrial building with specific working hours (6 days a week).

Yes, you are correct. This might represent the typical working hours of a user, for example, working on Monday, Wednesday, and Friday, with the other days off. Indeed, this interesting phenomenon further reveals *the potential value of our proposed RCF technique*, as it may be **a superior way to help data engineers analyze patterns in time series data**.

> “Traffic dataset exhibits spatiotemporal characteristics and temporal lag characteristics, where the traffic flow at a certain detection point significantly affects the future values of neighboring detection points.” However, the authors do not explain why this is not an issue for the Solar-Energy dataset, where weather conditions at one location may slowly impact neighboring locations in the future. Can authors comment on that?

The Traffic and Solar-Energy datasets are indeed quite different.