PGN: The RNN's New Successor is Effective for Long-Range Time Series Forecasting
Authors: Yuxin Jia
Youfang Lin
Jing Yu
Shuo Wang
Tianhao Liu
Huaiyu Wan
Venue: NeurIPS 2024 poster (2024)
Keywords: information propagation paths
the RNN's new successor
long-range time series forecasting
comprehensive semantic information
Primary area: deep_learning_architectures
TL;DR: We propose a novel paradigm, PGN, as the new successor to RNN, and propose a novel temporal modeling framework called TPGN, based on PGN.

Abstract: Due to the recurrent structure of RNN, the long information propagation path poses limitations in capturing long-term dependencies, gradient explosion/vanishing issues, and inefficient sequential execution. Based on this, we propose a novel paradigm called Parallel Gated Network (PGN) as the new successor to RNN. PGN directly captures information from previous time steps through the designed Historical Information Extraction (HIE) layer and leverages gated mechanisms to select and fuse it with the current time step information. This reduces the information propagation path to $\mathcal{O}(1)$, effectively addressing the limitations of RNN. To enhance PGN's performance in long-range time series forecasting tasks, we propose a novel temporal modeling framework called Temporal PGN (TPGN). TPGN incorporates two branches to comprehensively capture the semantic information of time series. One branch utilizes PGN to capture long-term periodic patterns while preserving their local characteristics. The other branch employs patches to capture short-term information and aggregate the global representation of the series. TPGN achieves a theoretical complexity of $\mathcal{O}(\sqrt{L})$, ensuring efficiency in its operations. Experimental results on five benchmark datasets demonstrate the state-of-the-art (SOTA) performance and high efficiency of TPGN, further confirming the effectiveness of PGN as the new successor to RNN in long-range time series forecasting. The code is available in this repository: https://github.com/Water2sea/TPGN.

PGN: The RNN'S New Successor is Effective for Long-Range Time Series Forecasting
Yuxin Jia Youfang Lin Jing Yu!.2 Shuo Wang Tianhao Liu Huaiyu Wan 22,
School of Computer Science & Technology, Beijing Jiaotong University, China 2 Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing, China School of Cyberspace Science and Technology, Beijing Jiaotong University, China fyuxinjia, yflin, jingyul, shuo. wang, leolth, hywan) edu. cn
Abstract
Due to the recurrent structure of RNN, the long information propagation path poses limitations capturing long-term dependencies, gradient explosion/van- ishing issues, and inefficient sequential execution. Based this, propose novel paradigm called Parallel Gated Network (PGN) as the new successor to RNN. PGN directly captures information from previous time steps through the designed Historical Information Extraction (HIE) layer and leverages gated mech- anisms to select and fuse with the current time step information. This reduces the information propagation path to (1), effectively addressing the limitations of RNN. To enhance PGN's performance in long-range time series forecasting tasks, we propose a novel temporal modeling framework called Temporal PGN (TPGN). TPGN incorporates two branches comprehensively capture the mantic information of time series. One branch utilizes PGN to capture long-term periodic patterns while preserving their local characteristics. The other branch employs patches to capture short-term information and aggregate the global rep- resentation of the series. TPGN achieves theoretical complexity of O(VL), ensuring efficiency in its operations. Experimental results on five benchmark datasets demonstrate the state-of-the-arr (SOTA) performance and high efficiency TPGN, further confirming the effectiveness of PGN as the new successor RNN in lang-range time series forecasting. The code is available in this repository: https://gittaie www.//aaatrsseeeeeeeeeee
Introduction
Under the premise of accurate time series forecasting, long-range forecasting tasks offer advantage over short-range forecasting tasks as they provide more comprehensive information for individuals and organizations to thoroughly assess future changes and make well-informed decisions. Due its practical applicability across various fields (i.e., energy [Zhou et al. [02T], climate [Angryk et al. 2020), traffic (Yin and Shang 2016), etc), long-range forecasting has attracted significant attention from researchers in recent years.
Long-range time series forecasting tasks can be broadly classified into two categories. One task is to
utilize abundant inputs to forecast future outputs [Liu et al. 2022a Jia et al. 2023). while another
task is predict longer-range futures with fewer historical inputs /hou al. [022]] 2022a Wang
et al, 2023). Although existing studies have shown that ample historical inputs can introduce more
information improve prediction performance (Jia et al. (0033 Liu al. 2024 considering factors
such as the load capacity of training devices and data collection, the utilization of limited historical
CCorresponiing author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

inputs to predict longer-range futures remains an important research topic. Therefore, this paper sets the task goal as predicting longer outputs with fewer inputs.
In recent years, dlep-learning-basee methods have achieved remarkable success in time series forecasting (for further diseussions, please refer to Section and Appendix [B). These methods can be
2021 Wu et al. 2021 Liu et al. 2022a Zhou et al. 2022a Nie et al. 2023 Ni al. [023 Liu et al. 2024 Dai et 2024
roughly categorized into four based paradigms: Transformers [Zhou ct al.
CNNs [u et al. 2023 Wang et al. [0033 Luo and Wang 2024], MLPs and Linears (Zeng et al.
2023
Xu et al. 2024 Wang et al. [024], , and RNNs Pia et al. [023]. It is worth noting that RNNs
have received relatively less attention over an extended period of time. This discrepancy is primarily
attributed to the limitation of RNNs' recurrent structure, which leads the persistence of
excessive long pathways for information propagation.
In fact, shorter information propagation paths lead less information loss (Tishby and Zaslavsky 2013). better captured dependencies [Liu et al. 2022a], and lower training difficulty [Wang et al. 2023). However, RNNs heavily rely on sequential recurrent structure to transmit information, making it challenging for them to capture long-term dependencies and suffer from the issue of gradient vanishing/exploding [Pascanu et al. 2013]. Meanwhile, due to its sequential computation, even though RNNs have a theoretical complexity that is linear with respect to sequence length L, their actual running speed can be even slower than the O(L2) complexity of the Vanilla- Traislla-rn Vaswani et al. 2017]. Some RNN-based models [Hochreiter and Schmidhuber [997/ [Cung et al. [0144] have tried to enhance performance by incorporating specialized gated mechanisms. However. compared to the inherent limitations of the RNN structure, these improvements in information selection and fusion are merely drop in the bucket.
Based on this motivation, this paper proposes a novel general paradigm called Parallel Gated Network (PGN) as the new successor to RNN. PGN introduces Historical Information Extraction (HIE) layer to replace the recurrent structure of RNN, and then further selects and fuses information through gated mechanisms, effectively reducing the information propagation paths O(1), as shown in [ This enables PGN to better capture long-ten long-term dependencies in input signals. Additionally, since computations for each time step can be parallelized, PGN achieves significantly faster execution speed while maintaining the same theoretical complexity of OLL) as RNN.
eicw
Figure The information propagation illustration of different models.
Despite the advantages of PGN in terms of efficiency and capturing long-term information, it cannot be directly applied to time series forecasting tasks for optimal performance. This is because, based
on ID modeling, PGN struggles to capture periodic semantic information (Jia al., [023] effectively,
Fortunately, the idea of transforming data from 1D to 2D and modeling it [Wu et al. 2023 Jia
al., [023] Dai et al.] [2024), proves effective in addressing above limitation. When employing 2D modeling for time series, information captured along rows reflects short-term changes, while
information along columns represents long-term periodic patterns. Due to the distinct characteristics of these two types of information, it is reasonable to model them separately. Furthermore, considering that periodicity is present throughout the entire time series, both in the past and in the future, is important to prioritize this consideration when modeling.

---

Based on these motivations, we propose novel PGN-based temporal modeling framework called Temporal Parallel Gated Network (TPGN). TPGN establishes two distinct branches to capture long-term and short-term information in the 2D input series. To focus on modeling long-term information, we utilize PGN to model each column of the 2D inputs, preserving their respective local periodic characteristics. Simultaneously. leveraging the advantages of patch (Nie et al. 2023) in capturing short-term changes, TPGN initially aggregates the short-term information into patches and subsequently merges them to obtain global information.
By integrating the information from both branches, TPGN achieves comprehensive semantic infor- mation capture for accurate predictions. It also should be noted that other methods can substitute PGN and be used in the long-term information extraction branch of TPGN, undoubtedly enabling TPGN to be general framework to model temporal dependencies. Furthermore, TPGN maintains a efficient computational complexity of ((VL). To better illustrate the advantages of TPGN, inspired by (Tia et al. 2023], we have provided a information propagation comparative diagram in Figure] ] and an analysis table in Table 3
The main contributions of this paper can be summarized as follows:
We propose a novel general paradigm called PGN as the new successor to RNN, as shown in Figure (l). (l). It reduces the information propagation path to O1)), enabling better capture of long-term dependencies
input signals and addressing the limitations of RNNs.
We propose TPGN, a novel temporal modeling framework based on PGN, which compre- hensively captures semantic information through two branches, as shown in Figure [ (m). One branch utilizes PGN capture long-term periodic patterns and preserve their local characteristics, while the other branch employs patches to capture short-term information and aggregates them to obtain global representation of the series. Notably, TPGN can also accommodate other models, making it be a general temporal modeling framework.
terms of efficiency, PGN maintains the same complexity of OL)) as RNN. However, due to its parallelizable calculations, PGN achieves higher actual efficiency. On the other hand, TPGN, serving as a general temporal modeling framework, exhibits a favorable complexity of O(VL). For a more detailed comparison of complexities, please refer to Table 3
We conducted experiments five benchmark datasets, and the results indicated that TPGN achieved an average MSE improvement of 12.35% in various long-range time series fore- casting tasks compared to the previous best-performing models. Furthermore, in comparison to the average performance of specific models across all tasks, TPGN achieved an average MSE improvement ranging from 14.08% to 39.65%. Additionally, experimental evaluations on computational complexity confirmed the efficiency of TPGN.
2 Related Works
2.1 Modeling Interaction Cross Temporal Dimension
The methods that focus on temporal modeling can be broadly categorized into four paradigms:
RNN- CNN-, MLP- (Linear-), and Transformer-basel The limitations of RNNs (Hochreiter and
Schmidhuber 1997| Chung et al.] 2014] (aalnna et al.) 2020 have been discussed in Section [
Despite some methods [Chang et al. [207] 7, Yu and Liu 2018 Jia al. 2023) trying to alleviate
these limitations, the recurrent structure still hinders their further development. CNNs [Franceschi
al. 2019 Sen al. 2019] offer advantages in efficiency and shorter information propagation paths, but primarily constrained by limited receptive fields (Wu et al. [023). resulting in an increase in the information propagation path as the length of the processed signal increases. Although some methods [Wang et al. 2023] Luo and Wang] 2024 have increased the receptive field to address these issues, the ID modeling approach makes challenging for them to directly capture periodicity. The advantages of Linear [Zeng et al. 2023) lie in its simple structure and high operational efficiency. Some advanced models have further enhanced the performance of MLP or Linear by applying them in the frequency domain [Xu et al.] 2024] or introducing multi scales [Wang et al. 2024].
which could lead to higher execution overhead. Classic Transformer based methods either struggle to capture semantic information (wu et al., 2023 Nie et al.) 2023) due to point-wise attention mechanisms /Vaswani al.] 2017 Zhou et al. 2021 [022a] or have high complexity [Wu al. 2021
Liu et al. 2022a limiting their ability. Subsequently, this problem was effectively addressed by
3

---

utilizing patches [Nie et al. 2023). However, they still suffer from the ID modeling issue mentioned earlier or the problem of limited receptive fields. More detailed discussion and analysis can be found in Appendix B
2.2 Modeling Interaction Cross Variable Dimension
For handling variable dimensions, there are generally four categories: variable fusion processing, variable independent processing, modeling based on Transformers, and modeling based on Graph Neural Networks (GNNs). Traditional fusion processing methods, due to the heterogeneity of multiple variables [Zhou et al.) 2021], introduce excessive noise, resulting in worse performance compared to independent processing of variables (Nie et al.! 2023). However, by applying attention mechanisms and Graph Neural Networks (GNN) on the variable dimension to replace independent modeling of variables, it possible to successfully capture the correlations and differences between variables, thereby significantly improving the performance of multivariate modeling, Representative methods for modeling variable relationships based on Transformers include Crossformer (Zhang and Yan 2022] and iTransformer [Liu et al., (2224), while GNN-based representative methods include CrossGNN (Huang et al., [023] and FourierGNN (Yi et al. 2023). They provide excellent inspiration for analyzing and modeling multivariate time series.
3 Methodology
In this section, we first introduce our proposed novel paradigm called Parallel Gated Network (PGN), and explain how it reduces the information propagation paths, overcomes the limitations of RNNs, and emerges the new successor to RNNs. Next, we present our newly designed temporal modeling framework called Temporal PGN (TPGN), which incorporates two separate branches to comprehensively capture semantic information. Finally, we provide comprehensive complexity analysis to evaluate the computational efficiency of our methods.
3.1 Parallel Gated Network (PGN)
Building upon the previous analysis, the limitation of RNNs lies in the excessively long information propagation paths of its recurrent structure, which directly leads to series of issues, such as difficulty in capturing long-term dependencies (performance). low efficiency (efficiency), and gradient exploding/vanishinn specialized gated mechanisms, such as LSTM [Hochreiter and Schmidhuber [997] and GRU [Chung iet al. 2014]. which do have advantages in information selection and fusion. However, when faced with the disastrous limitation of RNNs, their advantages become insignificant.
sequential computations (training difficulty). Indeed, some RNNs leverage
Based on this, we propose a novel general paradigm called PGN as the new successor to RNNs. PGN draws the advantages of RNNs while reducing information propagation paths (1), thereby addressing the limitation of RNNs. The information propagation illustration and structure of PGN shown in Figure (1) and Figure 2 (a). respectively. On one hand, to enable PGN capture information from all preceding time steps within short information propagation paths, we introduce a linear Historical Information Extraction (HIE) layer to aggregate information from the entire history at each time step. Importantly, at this stage, the computation of each time step of the signal is independent of others, allowing for effective parallel processing. On the other hand, PGN leverages gated mechanisms to inherit the advantages of information selection and fusion. It is important to emphasize that in PGN, we utilize only a single gate to simultaneously control the information selection and fusion in parallel manner across all time steps of the sequence, resulting in reduced computational overhead. When given an input signal X E R. of length L, the computation process of PGN can be formalized as follows:
H HIE X)),
G a ( [X, H] + bg),
(I)
A tanh (anh(W,(X H + bt).
Out = G & + (1 G) o f where Padding(-) represents the operation of filling the front of the processed signal along the length dimension with zero-filled vector of size (L-1), HIE(-) is a linear layer with weight matrices

---

W1 E (4.x(1-11) and bias vectors b₁ R It aggregates all relevant historical information for each time step in parallel by sliding along the sequence length dimension, and H R R.xdm represents the output of this operation. The weight matrices Wg, g, W E (damx(d++1) and bias vectors bg. by E d are utilized in the computations. G and are the intermediate variables involved the gated mechanism. The symbol @ represents the element-wiss product, while o(-) and tanh(-) denote the sigmoid and tanh activation functions. Out E I 1xdddm dm represents the output of PGN.
Time Stepe
Eepnorettess Zzu Pukkin Pandkl Aumepiion
TPUN Famework
Forecasting
Mudule
IHE Layer
Good Madharien
Medale
Information Extractoo itrnch
Cenast
Ouet
PGN
LinetTane
Nern
100000
back
Rollups
Sheet boin Infocmotior Extraction Beanch
Linrar
Grait
Libear
Reshage
Xmn
()) The etructtten of PCN
(b) Ovenall eracture TINN
Figure 2: The structures of PGN and TPGN.
3.2 Temporal Parallel Gated Network (TPGN)
The specific objective time series forecasting task is to predict the future series of length Lf given a historical sequence of length 4n As stated in Section | PGN may not be effective in directly extracting periodic semantic information, which limits its application in time series forecasting tasks. Inspired by [Wu et al. 2023 Tia et al., 2023 Dai et al.) 2024]. we transform the input series from 1D 2D for modeling. To fully capture the short-term changes and long-term periodic patterns with different characteristics in the rows and columns of the 2D input, we introduce two branches to model them separately. The information propagation diagram and overall structure of TPGN are shown in Figure I (m) and Figure (b).
Input Preparation Module enable TPGN to directly capture periodic semantic information, inspired by previous works [Wu et al. 2023 Jia et al. 2023 Dai et al., 2024]. we reshape the series from ID to 2D. Notably, we do not need to introduce multiple scales of periods like in TimesNet [Wu] et al. 2023 and PDF (Dai et al. 2024), as it would result in increased computational overhead. Instead, we draw inspiration from WITRAN [Jia et al. 2023] and solely reset the sequence based on the natural scale of the time series. In addition, to minimize the negative impact of data fluctuations on model training, inspired by (Liu et al.) 2022b 2024], we have introduced a normalization layer along the temporal dimension. When given input sequence XID {x1, 22, Ø2,.... ..... LL.. } IR gg/4 X
and temporal external feature Fene E IR 4g Came (C and Ctime represent the number of variables and temporal external features), this module can be mathematically expressed as:
Lb
Ln
px =
Ci,
2 =
Σ i=1
(x; xx)²
Lu
Lu
i=1
0 Reshape([XXI0 TFene]).
[XID.
(2)
ID
((II - -xx)/ox norm 1
XạD
Here, X E R4 represent the normalized series, [.] represents the concat operation, and the hyperparameters norm should be determined based the characteristics of different datasets. To combine each variable of the input series with the temporal feature, we need to expand Norm by adding an extra dimension to match the shape of CCx1 Additionally, TFeece needs to be expanded by adding dimension and repeated C' times to match the shape of x XxxxxxxCr XCimeee
Afterwards, we concatenate the results and reshape them according to the natural period P of series represents the output this module. R and P represent the number of rows and columns in the 2D input, respectively.
through Reshape(-). :), and X2D E fxx (xC
TPGN As TPGN focuses on modeling the temporal dimension, which crucial for any variable in the time series. In the following discussions, will focus example variable m provide
5

---

detailed explanation, where X E RR represents the input. To better capture long- and short- term information while preserving their respective characteristics, we have designed two branches as illustrated in Figure (m) and Figure (b).
In the long-term information extraction branch, directly capture the information using PGN. On one hand, it effectively captures the long-term repetitive historical information for each time step. On the other hand, through the gated mechanism, selects and fuses the current and historical information at each time step, thereby preserving the long-term periodic characteristies to the maximum extent. Specifically, this branch can be formulated as follows:
Long PGN(X))), Hong Lineariong loug):
where (. represents the input being passed through the PGN paradigm. It is important to note that PGN operates along the f dimension. The advantage of this approach is it preserves the individual characteristics of each column, better serving forecasting. The output denoted as X E R Rxx Pxxxx To facilitate the utilization of long-term information for prediction purposes, we aggregate the information from all rows in each column using linear layer Linearing (-). The output of this branch is denoted as H E R Pxdm.
In the short-term information extraction branch, considering the advantage of patch in aggregation short-term information, we first utilize a linear layer to aggregate the short-term information into patches. Then, another linear layer is used to further fuse the patches into the global information of the series. The specific process can be formulated as follows:
short Linear short (X2D). global Chort (H short):
sbort (-) operates along the P dimension, and H E RRxd is its output. Subse- quently, L (-) further aggregates the patches Hghott and obtains the global representation Hm
where Linear row
global E TR. xxdm of the sequence. Finally, to facilitate subsequent predictions, we repeat Hm global along the first dimension P times to obtain new representation with the same dimension RP Pxda as the output of the long-term information extraction branch.
Forecasting Module In this module, begin by concatenating the information representations derived from the two branches of TPGN. The concatenated information encompasses the local long-term periodic characteristics observed across various columns in the 2D input series, along with the globally aggregated short-term information. Subsequently, we take the previously aggregated representation with comprehensive semantic information and pass it through linear layer to predict future values at different positions within the period. The formulation of this module is as follows:
Out" Desshape(Linear(FFFF)
Hglobal)),
where |-] represents the concat operation. The output dimension after Linear(-) is RPx where Ry multiplied by P equal forecasting series length L,. Finally, the above output will be permuted and reshaped to ID dimension by Reshape(-) operation, the result Out E R-4,
3.3
Complexity Analysis
PGN While PGN processes signals in the time dimension in parallel, each step still involves processing all its historical information. Therefore, the theoretical complexity of this part is still linear with respect to the length L. of the signal being processed. The complexity of the gated mechanism is independent of the signal length, so the complexity of PGN can be expressed O(L). Noted that PGN has the same theoretical complexity as RNN, but due to the parallelized ability of PGN computations, it has much higher efficiency in practice compared to RNN.
TPGN Since TPGN has two separate branches, it is necessary to analyze them separately.
For the long-term information extraction branch, TPGN applies the PGN paradigm along the number of R, the complexity this step indeed linear with respect to R. denoted as (R). The aggregation of all rows of information is accomplished through linear layer, which still has a complexity proportional to (R). Therefore, the complexity of the long-term information extraction branch can be expressed as (R).

---

For the short-term information extraction branch, TPGN applies two linear layers. The first linear layer compresses the time dimension from P 1, while the second linear layer compresses the other time dimension R to 1, therefore, their complexities are respectively (P)) and (R)).
Since R multiplied by P equals the input sequence length Ta (T.), the complexities ( and (PP) are both equal to O(VL). For the two branches of TPGN, the complexities are both O (V))).
Therefore, the complexity of TPGN is also (VT.).
Experiments
Datasets
To validate the performance of TPGN, we followed WITRAN (Jia al., 2023] and conducted experiments on five real-world benchmark datasets that span across energy, traffic, and weather domains. More details about the datasets can be found in Appendix C
Baselines We conducted a comprehensive comparison of thirteen methods in our study. These
methods include two RNN-based methods: WITRAN [Jia et al. 2023). SegRNN [Lin et al.
2023
three CNN-based methods: Modern'T TCN (Luo and Wang [024]] TimesNet [Wu et al.
2023
MICN [Wang et al.] 2023), three MLPPbaaedd methods: FITS [xu et al. 2024). TimeMixer Wang
ct al.
2024].
DLinear [Zng et al.) 2023). four Transformer-based methods: iTransformer [Liu
ct al.
2024 PDF [Dai et al., 2024 Basisformer [Ni et al. 2023). PatchTST [Nie et al.) 2023],
and FiLM Zhou et al. 20226]. It should be noted that certain earlier methods such as Vanilla-
Transformer Vaswani et al. 2017]. Informer [Zhou et al. [02T], Autoformer [Wu et al. [021]
Pyraformer [Liu et al.] 2022a). and FEDformer [Zhou et al. [022a] have been extensively surpassed by the methods we selected. Hence, we did not include these earlier methods as baselines in our
comparison. For further discussion on these methods and details of the experimental setup, please
refer to Appendix
B and Appendix D
4.1 Experimental Results
It is important to emphasize that while there have been numerous works focusing on modeling the relationships among multiple variables in time series, they still need to effectively capture information along the temporal dimension to better accommodate multivariate time series. In contrast, our method primarily concentrates on modeling the temporal dimension. To mitigate any potential negative impact caused by the heterogeneity of multivariate data, we followed the experimental setup of WITRAN [Jia et al. [2023), conducted experiments using a single variable. To ensure fairness, we conducted parameter search for each baseline model, enabling them to achieve their respective optimal performance across different tasks. For further experiment details, please refer to Appendix D
Table I: Long-range forecasting results. A lower MSE or MAE indicates a better prediction. The best results are highlighted in bold and the second best results are underlined.
HAVE 1100 CLONT
SCE
HALT
2177
1300 01177
41.0% $2..4
$.10
120
01.21
8.114
9.005
76/1 11001
0.775
9.00 67755
07100 87275
196.23 114
01354
11861
1.224
0123 1:201
51121 52022
11570
1220
11122 52100
1113
01211 52141
1100 1V7NA
144 116
93774
07411 1271
1107 15:0
ET1 V1R07
130
9,244
81100
640% 0OM6
02040
12000 8305
0028
0179
8760
14th
6138 0181
870 8810
9014
E.09
(80)
E1100 m400
Belkgg Aigrosse #TRO3
30.0
2137
3017
R31 3MR
2..0 1430
3.17 1107
3-30: 2300
23R 10.01
11.31
31174
30101
SEA1- 17.67
77.F 2310
We conducted four tasks on each dataset for long-range forecast- ing, and the results are shown in Table For instance, considering the task setting 168- 1440 on the left side of Table [] it signifies that the input length is 168, and the forecasting length is 1440. It is worth noting that our proposed TPGN achieves state-of-the-art (SOTA) performance across all tasks, with an average improvement of MSE by 12.35% and MAE by 7.25% compared to the previous best methods. In particular, TPGN demonstrates an average reduction MSE of 17.31% for the ECL. datasct, 9.38% for the Traffic dataset, 3.79% for the ETTh; dataset, 12.26% for the ETTh; dataset,
Long-range Forecasting Results
7

---

and 19.09% for the Weather dataset. Furthermore, we calculated the average improvement values of TPGN compared to each method across all tasks and displayed them in the last row of Table Based on the aforementioned results, it can be concluded that TPGN is capable of effectively handling long-range forecasting tasks in various domains. For further experimental results and showcases, please refer to Appendix and Appendix
Performance of Variations with Different Forecasting Lengths It is important to emphasize that through [ we observed that as the forecasting task length increased, all models generally experienced varying degrees of performance decline. However, TPGN appeared to exhibit slower decline trend. To further validate the performance of TPGN, we expanded our experimental settings by selecting representative methods from different paradigms in Table I WITRAN TimesNet (CNN-Bar BANN-Based), TimeMixer compared them with TPGN. The experimental results on the ECL dataset are depicted in Figure 3 and more experimental results can be found in Appendix [ It can be observed that the forecasting task length gradually increases, TPGN exhibits a stable decline in performance and consistently outperforms the other comparative methods. This strongly indicates that TPGN effectively captures the comprehensive information contained in fewer inputs and applies it well to the forecasting tasks.
-Basedd), and iTransformer TTranfformer-Base/e
and
MSE over Dilicce Poccentine Taii
1.5
MAE over Deficen Percensing lonks
WilKAS
0.
Treski
350
1014
1350
1800
352
623
921
1000
Procusting Lengs
Figure 3: Experimental results with different forecasting lengths on the ECL dataset.
For other aspects experiments and analysis of our methods can be found in Appendix G
4.2 Ablation Study
To validate the roles of the two information extraction branches in TPGN, we conducted tests on the performance of the model when using only one branch. Additionally, to verify the effectiveness of PGN, we performed ablation experiments by replacing PGN with GRU and MLP, respectively. "TPGN-long" represents only using the long-term information extraction branch, while "TPGN- respectively represent
short" represents only using the short-term. "TPGN- GRUUALSTM/MMILI replacing PGN in the long-term information extraction branch with GRU, LSTM, MLP and self- attention. The results of these experiments are presented in Table 2
Table 2: Results of the ablation study on long-range forecasting tasks, A lower MSE or MAE indicates better prediction. The best results are highlighted in bold.
Mabees Mare
TPG.
MUNLEE
DINNGA HSS MAR
X54
3045
1450
M4A
345E
MAA
3544
MAS
345E
MAE
BJ284
32223
0.39%
E332
142353
0,222
B3344
E3317 13425
100 712
1/h11 8 2114
11864
81213
5.157
1899
1967
0.702
1014 133/74
146 1440
0.1300
8.2144
01361
6.314
1550
0L140)
02164
01435
83155
01004 03507
BLEAE 0,553
01113
6.366
1JC08 0,2441
0.395
01113
01103
9.344
01133
01111
1L1306 8.2965
0.311
9.396
(1300)
01:92
$1100 MNIES
9.30%
750-104
0,274
83623
(A531
1L572
0,509
012577 03313
03472 13300
03300
166-1491
11211 8,773
0.505
046
1.312 64841
12811
03365
8.423
8-300
141%
0A336 033147
1.305
E3117
83333
63384
0L103
3.381
0.2322
03190 8.475
0376
786-22
83225
03300
1.78 BARTS
0.387
137e 11:044
166-1441
01883
83218
HA972
IL199 0,300
01895
8.384
84008
60411 (1467))
Through the ablation experiments, we can draw the following conclusions: (1) The two branches designed in TPGN are reasonable as they capture long-term and short-term information respectively. while preserving their respective characteristics. most cases, using only one branch leads to subpar results due to incomplete capture of essential features. (2) The branch capturing long-term information
8

---

in TPGN is more significant. This can be observed by comparing the performance degradation when using only one branch versus using both branches together. Especially for strongly periodic data like traffic, in some tasks, using only the long-term information capture branch can achieve good results. This also aligns with our earlier mention Section Iaroonnaaal Tabout the significance of prioritizing the modeling of periodicity. (3) Compared to GRU and LSTM, which have more gates, PGN introduces only one gate but still achieves better performance. This strongly demonstrates the ability of PGN to serve as the new successor to RNN. (4) The comparison between
and the baseline results demonstrates the strong generality and performance the TPGN framework. Despite their inferior performance compared to TPGN, in some tasks, they even surpass the previous SOTA time series forecasting methods.
4.3 Efficiency of Execution
Although this paper primarily focuses predicting longer-range future outputs using short-range historical inputs, we conducted two sets of comparative experiments to comprchensively evaluate the efficiency of our proposed method. In the first set of experiments, we kept the input length fixed at 168 and varied the output length to 168/336/720/ 1440 to study the impact of forecasting length on the actual runtime efficiency of the model. In the second set of experiments, we fixed the output length at 1440 while varying the input length to 02//336//20/444// to investigate the influence of historical input series length the actual runtime of the model. The efficiency analysis considered both time and memory aspects. We selected representative methods from each paradigm based on the experimental results in Table a the comparative methods. We fixed the batch size 32, the model dimension size at 128, and conducted the tests using a single-layer model. The results are shown in Figure Due the much higher time and memory overhead of TimesNet compared to the other comparative models, we have omitted it from Figure 4 provide a clearer illustration of the overhead details of the other models. Similarly, FiLM not included in the time comparison chart,
Dechectionaa tie www.eetttttttaaeeoo
10 Time crcilaal lnghii Bool 150)
Manary encllcal Dipee kigfh Fasl 108)
(6 Time onathed Inapa ligh la Fandal 1448)
M0 snstadd toupu baga Bhades 1443)
Figure 4: Time and memory overhead of different models.
From Figure 4 it can be observed that although TPGN does not have the lowest time and memory overhead, it achieves decent level of efficiency in both time and space aspects. It is important to note that TPGN is a model with only one layer, while most of other models require the introduction of deeper layers, which inevitably leads higher overhead. This undoubtedly further demonstrates that our method not only achieves SOTA performance but also delivers satisfactory efficiency.
5 Conclusions
In this paper, we propose a novel general paradigm called Parallel Gated Network (PGN). With its (1) information propagation paths and parallel computing capability, PGN achieves faster runtime speed while maintaining the same theoretical complexity as RNN (O()). To enhance the application of PGN in long-range time series forecasting tasks, we introduce a novel temporal modeling framework called Temporal PGN (TPGN) with an excellent complexity of O(VL). By employing two branches to separate the modeling of long-term and short-term information, TPGN effectively capture periodicity and local-global semantic information while preserving their respective characteristics. The experimental results on five benchmark datasets demonstrate that our PGN-based framework, TPGN, achieves SOTA performance and high efficiency. These findings further confirm the effectiveness of PGN as the new successor to RNN in long-range time series forecasting tasks.

---

Acknowledgments
This work was supported by the National Natural Science Foundation of China (No. 62372031
References
Haoyi Zhou, Shanghang Zhang, Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence semies forecasting. In Proceedings the AAAI conference on artificial intelligence, volume 35, pages 11106-11115, 2021.
Rafal Angryk, Petrus C Martens, Berkay Aydin, Dustin Kempton, Sushant S Mahajan, Sunitha Basodi, Azim Ahmadzadeh, Xumin Cai, Soukaina Filali Boubrahimi, Shah Muhammad Hamdi, et al. Multivariate time series dataset for space weather data analytics. Scientific data, 7(1):227, 2020.
Yi Yin and Pengjian Shang. Forecasting traffic time series with multivariate predicting method. Applied Mathematics and Computation, 191:666-278, 2016.
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting, In International conference learning representations, 2022a.
Yuxin Jia, Youfang Lin, Xinyan Hao, Yan Lin, Shengnan Guo, and Huaiyu Wan. Witran: Water-wave information transmission and recurrent acceleration network for long-range time series forecasting. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequeney enhanced decomposed transformer for long-term series forecasting, In International Conference on Machine Learning, pages 27268-226677 27286. PMLR, 2022a.
Huiqiang Wang, Jian Peng, Feihu Huang. Jince Wang, Junhui Chen, and Yifei Xiao. Mien: Multi-scale local and global context modeling long-term series forecasting. The Eleventh International Conference on Learning Representations, 2023.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024.
Haixu Wu, Jichui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34: 22419-22430, 2021.
Yuqi Nie. Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long- term forecasting with transformers, In The Eleventh International Conference on Learning Representations, 2023.
Zelin Ni, Hang Yu, Shizhan Liu, Jianguo Li, Weiyao Lin. Basisformer: Attention-based time series forecasting with learnable and interpretable basis. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Tao Dai, Beiliang Wu, Peiyuan Liu, Naiqi Li, Jigang Bao, Yong Jiang, and Shu-Tao Xia. Periodicity decou- pling framework for long-term series forccasting. In The Twelfth International Conference on Learning Representations, 2024.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The eleventh international conference on learning representations, 2023.
Donghao Luo and Xue Wang. Modernten: A modem pure convolution structure for general time series analysis. In The Twelfth International Conference on Learning Representations, 2024.
Ailing Zeng. Muxi Chen, Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121-11122 2023.
Zhijian Xu, Ailing Zeng, and Qiang Xu. Fits: Modeling time series with 10k parameters. In The Tweifih International Conference on Learning Representations, 2024.
10

---

Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo. Lintao Ma. James Y Zhang, and JUN ZHOU. Timemixer: Decomposable multiscale mixing for time series forecasting. In The Twelfth International Conference on Learning Representations. 2024.
Naftali Tishby and Noga 7aslavsky. Deep learning and the information bottleneck principle. In 2015 ieee information theory workshop (inw), pages 1-5. IEEF, 2015.
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks, In International conference on machine learning, pages 1310-1318. Pmlr, 2013.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Sepp Hochreiter and Jürgen Schmidhuher. Long short-term memory. Neural computation, 9()::1335-178) 1997.
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:141233555 2014.
David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski, Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3): 181-1191, 2020.
Shiyu Chang, Yang Zhang. Wei Han, Mo Yu, Xiaoxiao Guo, Wei Tan, Xiaodong Cui, Michael Witbrock, Mark Hasegawa-Johnson, and Thomas S Huang. Dilated recurrent neural networks. Advances in neural information processing systems, 30, 2017.
Zeping Yu and Gongshen Liu. Sliced recurrent neural networks. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2953-2964, 2018.
Jean-Y Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. Unsupervised scalable representation learning for multivariate time series. Advances in neural information processing systems, 32, 2019.
Rajat Sen, Hsiang-Fu Yu, and Inderjit S Dhillon. Think globally, locally: deep neural network approach high-dimensional time series forecasting. Advances neural information processing systems, 32, 2019.
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. The eleventh international conference on learning representations, 2022.
Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. Crossgnn: Confronting noisy multivariate time series via cross interaction refinement. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Kun Yi, Qi Zhang, Wei Fan, Hui Hc, Liang Hu, Pengyang Wang, Ning An, Longbing Cao, and Zhendong Niu. Fouriergnn: Rethinking multivariate time series forecasting from pure graph perspective. In Thirty-seventh Conference on Neural Information Processing Syxtems, 2023.
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity time series forecasting. Advances Neural Information Processing Systems, 35:9881 19993, 2022b.
Shengsheng Lin, Weiwei Lin, Wentai Wu, Feiyu Zhao, Ruichao Mo, and Haotong Zhang. Segrnn: Segment recurrent neural network for long-term time series forecasting. arXiv preprint arXiv:2308. 1200, 2023.
Tian Zhou, Ziqing Ma, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, Rong Jin, et al. Film: Frequency improved legendre memory model for long-term time series forecasting. Advances Neural Information Processing Systems, 15:77777-1,690, 2022b.
George EP Box and Gwilym M Jenkins. Some recent advances in forecasting and control. Journal of the Royal Statistical Society. Series C (Applied Statistics), 17(2):91-109, 1968.
Sean Taylor and Benjamin Letham. Forecasting at scale. The American Statistician. 72(1):37-44 2018.
G Athanasopoulos and RJ Hyndman. Forccasting: principle and practice: O texts; 2018, 2020.
Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting. Advances in neural information processing systems, 31, 2018.
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint S.3v:1803.01277 2018
11

---

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 141.6004,,.6980. 2014.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high sigh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
12

---

A Limitation and Future Outlook
It is important to acknowledge that our focus in this work was primarily on temporal modeling, without specifically addressing the modeling of relationships between variables. Nevertheless, we can draw inspiration from other methods specialized in variable modeling. Incorporating an additional component to model variable relationships and integrating it into the TPGN framework is a promising direction for better adaptation to multivariate prediction tasks, which we plan explore in future research. Additionally, we will continue to investigate the broader application of the PGN paradigm as a replacement for RNN in various time series analysis tasks and other research areas.
B More Detailed Discussions of Related Works
Traditional methods such as ARIMA [Box and Jenkins 1968 Prophet [Talorr and Letham 2018), and Holt-Winters [Athanasopoulos and Hyndman] 2020 are often constrained by theoretical assumptions, which limits their applicability in time series forecasting tasks with dynamic data changes. In recent years, deep neural networks (DNNs) have made significant advancements in the field of time series analysis. DNNS can be categorized into four main paradigms: RNN-based, CNN-based, MLP-based, and Transformer-basee methods.
RNN-based methods [Hochreiter and Schmidhuber 1997 Chung et al. 2014 Rangapuram et al. 2018 Salinas al. 2020 rely on recurrent structures to capture sequential information, which leads to long information propagation paths and brings about various limitations, as discussed in Section In terms of performance, RNN-based methods struggle to capture long-term dependencies effectively. Moreover, their theoretical complexity scales linearly with the sequence length L, but their practical efficiency is often low due to sequential computation. Additionally, during training, RNNs are prone to the issues of gradient exploding/vanishinn [Pascanu et al. 2013).
To alleviate these issues, some methods have modified the conventional information propagation approach. DilatedRNN [Chang et al., 2017] introduces a multi-scale dilated mechanism, which aggregates information at each time step. Although it can shorten the information propagation path by selecting the branch with the maximum skipping step, the path remains linearly related to the sequence length L, which is still relatively long. SlicedRNN (Yu and Liu| 2018 addresses the efficiency problem of RNNs by dividing the sequence into multiple slices for parallel computation. However, the length of the information propagation path remains the same as the traditional RNNs. WITRAN [Jia et al. 2023 as an emerging time series forecasting method, reshapes the sequence into 2D dimension and performs simultaneous information propagation in both directions. This approach improves computational efficiency and reduces the information propagation path to O(VL) However, it is still relatively long for effective information extraction.
Overall, the limitations imposed by the recurrent structures of RNNs have hindered their further development.
CNN-based methods Bai et al.) 2018 Franceschi et al., 2019 Sen et al. 2019 have a theoretical complexity of (L). due to their parallel ability, they often exhibit higher practical efficiency compared to RNNs. However, CNNs are typically constrained by limited receptive fields, requiring the stacking of multiple module layers to capture global information from inputs. The number of modular layers grows superlinearly with the sequence length, leading to an information propagation path of (G) in CNN-based methods. Here, G is superlinearly related to the sequence length L. In the case of the 2D modeling method TimesNet [Wu et al.) 2003).. where the input lengths in both directions are (VTT),, the information propagation path would be O(VG). MICN [Wang et al. 2023) and ModernTNN [Luo and Wang 2024] effectively reduce the information propagation path by enlarging the receptive field of the convolutional kernel. However, due to their ID modeling approach, they may not perform as well as TimesNet [Wu et al. 2023 in capturing periodic characteristics.
MLP-based methods are highly favored due to their simple structure, resulting in lower computational complexity and information propagation path. This simplicity makes MLP models easy to implement and train, contributing to their popularity. Dlinear and NLinear [Zeng et al.) 2023 optimize the original Linear model through the methods of sequence decomposition and re-normalization methods, enabling direct future prediction based on historical inputs. However, due to their limited ability to extract deep semantic information, they may not achieve excellent forecasting performance.
TimeMixer [Wang et al. 2024 employs two specialized modules to analyze and predict time series
13

---

data from multiple scales. While this approach can effectively capture periodicity, incorporating multiple scales in computations inevitably leads to increased computational costs and training difficulties. FITS [Xu et al. 2024] treats time series prediction as interpolation and transforms the time series into the frequency domain. It operates on the frequency domain using a specially designed block LPF (Low-Pass Filter) and a complex-valued linear layer for final forecasting. However, FITS may still overlook explicit local variations present in the sequence.
Transformer-bant based methods still dominate the majority of the field. The advantage of methods based on point-wise attention mechanism, such as Taaill--Tannformer (Vaswani al. 2017]. Informer [Zhou et al. 2,21], and FEDformer [Zhou et al. 2022a], lies in their ((1) information propagation path. However, previous studies [Wu al., |023] Jia et al. 2023] have clearly pointed out their limitation in capturing semantic information from time steps. On the other hand, other methods that utilize non-point-wise attention mechanisms still have other limitations. Although Autoformer [Wu et al. 2021] can capture the periodicity of time series to some extent through sequence decomposition, it is far less direct compared to methods like TimesNet (Wu et al. 2023].
Additionally, its complexity remains high at OLL log L). Pyraformer (Liu et al. 2022a through the special design of its pyramidal structure, also able to effectively capture the periodic characteristics of sequences. However, it is still constrained by the limitations of the convolution kernel when initializing the node of pyramidal structure. Additionally, Pyraformer still maintains a high complexity of OLL). PStchTST [Nie et al. 2023] captures local semantic information through patches, where represents the stride length, and further reduces the complexity to ((L//S²², However, it still cannot directly capture the periodic characteristics of series. iTransformer [Liu et al. 2024] primarily focuses on modeling the relationships between variables, including the relationships between time series variables and external time features. For the temporal dimension, iTTransfrmmer adopts direct patch-based approach, which makes it challenging to effectively extract periodic patterns and other local characteristics. PDF [Dai et al. 2024) also follows the approach of transposing the original ID sequence into 2D representation for modeling. Specifically, it utilizes CNNs to process short-term information, which is undoubtedly constrained by the limitations of convolution itself. When it comes to handling long-term periodic information, PDF also adopts a patch-based approach, which may not fully capture all the periodic characteristics present in the sequence.
To highlight the advantages of our proposed PGN paradigm and TPGN framework compared to previous methods, have organized an information propagation diagram shown in Figure 1 Based on the above analysis, we further compiled the various strengths, information propagation paths, and theoretical complexities of different models, which are presented in Table
Table 3:
Comparison of strengths, complexities and the maximum information propagation paths of different models. is superlinearly related to the sequence length and S represents the stride.
Capturing non- point- wise semantic information
Directly capturing periodic semantic information
Maximum temporal
Complexity of encoder per
Parallel Computing
Methods
information propagation path
Capability in the in Termporal Dimension
model layer
RNN
O(L)
O(L)
x
WITRAN
v ((2D)
O(VL)
O(vL)
CNN
O(G)
O(L)
MICN
V
A (Decompusition)
0(1)
O(L)
TimesNet
/
/(DD)
((vG)
(UL)
ModemTCN
O(G)
O(L/8)
Transformer
X
0(1)
O(12)
X
0(1)
O(1log L))
Autoformer
(Decemposition)
0(1)
O(L L)
Pyraformrr
/ (Pyramidal Streture)
((1)
O(L)
PatchTST
X
0(1)
O((L/S)))
Transformer
0(1)
((L)
PDF
/ (2D)
((vō)
O(L/8)
PGN (ours)
x
O(1)
O(L)
TPGN (ours)
/ (aa))
((1)
O(VD)
14

---

C More Detailed Description of the Datasets
In this section, we will provide comprehensive overview of the datasets utilized in this paper. (1) Electricing (ECL) contains the hourly electricity consumption of 321 customers from 2012 to 2014.
(2) Traffic contains the hourly data the road occupancy rates measured by different sensors on San Francisco Bay area freeways, collected from California Department of Transportation, (3) ETT contains the load and oil temperature data recorded every 15 minutes from electricity transformers in two different areas, spans from July 2016 to July 2018. (4) Weather[contaii contains 21 meteorological demperree
indicators (such as air temperature, humidity, etc.) and was recorded every 10 minutes for 2020 whole year.
Due to the varying granularity of data acquisition for each dataset, in order ensure that they contain
the same semantic information for the same task, we followed (Jia et al. [0033 to aggregate them at an hourly level for experimentation. The target value for ECL is 'MT 3000, for Traffic is 'Node 862', for ETT is "oil temperature (OT)', and for Weather is 'wet_bulb" They are all split into the training set, validation set and test set by the ratio of 6:2:2 during modeling.
Table 4: MSE and MAE with error bars are measured for all methods in long-range forecasting tasks. Each experiment is repeated 5 times.
Widder
Mark
MAA
SI5E
SEAL
SASE
MAE
MIL
SIAE
MAA
9964
01815-0100%
68776
30316
0100544
3250--0000
20144
66377
158 164
0
03661-00300 3156
B 0LI05-010640
00008800000 DOSTS
6515
0100-000013
148 731
6:336-10 39766
B 0156-61601
0.03846 6613
774-0199155
146-444
015541801125
155-166
032122002011 2012
ModeaTTNN
155-123
3404-0001 185
04476-307711
04012-000035
TraodNe:
01-55
63671-0 S0OVE
0230640 50588
305-003054
0338448 1045
12011-42 8096
02531-48 8:410
5115-000100
MICN
0423222983
01200
200R
1007740 14480
LL20220 BEC0)
638-5293309
EL 114400015
22802001444
BA20610 8250
047%:2001221
022152200077
07022011011 (QU
LUT0722 R011
01500-00R227
0.399
RESHCLDOROL
621540 30540
0221510
86350
03
0158 10000
6230-0
02716443007
16216
B277542
31500-351000
3056
02300-1339
3500000155
01570220111
RC05441100 1010)
DE Sanur
6339953 2015
0241292215 2012
01500120110
61041-1 2004
04410-10100
C
L201151100 101001
2013
P0C
03902-250111
04011201307
0.35
EL220 11200
EL00415 ELS27
1076
0336512 B3527
0-4773000015
0440000114
00000 00
623560 20134
64772033642
11251
0773448 6036
0
198
01935-35365
01501-000077
B 015576610M5
01703140 104%
2251-008671
0290-35863
6385255 18
1496-00860
01977-19141
135-355
84094110343
C 62918-02663
241142001177
55500
63815.0 DMAC
64561 2011291202
21130000111
6171011 D4307
055360001111
02287-006022
03382225315
0130
010011000077
022661301441
22122000111
0242515
02022120014
0221112
155-33
8238603 2962
12381309903
04757200R612
148:440
0.11442000127
02585.30000
419011000057
026:5002014
029471910110
37500000103
01T0511 6112
0344320000134
D Experimental Setup Details
To ensure a fair comparison of the performance of each model, we set the same search space for the common parameters included in each model. Specifically, (1) The model dimension size dm is set to: 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024. (2) The number of layers for the model's 's encoder and decoder is set to: 1, 2, 3. (3) The number of heads for the attention mechanism is set to: 1, 2, 4. 8. In addition, for the individual hyperparameters specific to each model, we also referred to their
www.::hhwwiiiiiuuu......... uci wtt.///wwwhh//teeeeccccccccaaaaa
http:/ http://pems.dot.. dot. ga.
bttpp::/www.bgc-.. jec-jjna..apppee..ooooo..a. dwt.tt///..eerr//////////
15

---

respective original papers and conducted parameter search accordingly. This ensured that we took into account the optimal configurations for each model in our experiments. For models that have variants, such as MICN-regre and MICN-mean, we treat the variants as separate hyperparameters and include them in the search process. The aforementioned procedures ensure that the reported results represent the optimal performance of each model under the same comparison conditions.
TPGN and all baselines were trained using L2 loss and the ADAM optimizer [Kingma and Ba 2014 with an initial learning rate of 10-3 -3_ All of them are implemented using Py PyTorch Paszke et al. 2019] and conducted on NVIDIA RTX A4000 16GB GPUs. The batch maximum training epochs was set to 25. If there was no degradation in loss on the validation set after epochs, the training process would be early stopped. We saved the model with the lowest loss on the validation set for final testing. The mean square error (MSE) and mean absolute error (MAE) are used as metrics. We followed [Jia al. [2033 and set the seed to 2023 ensure the reproducibility of the results. All experiments are repeated 5 times and we set the mean of the metrics as the final
was set 32 and the
results, as shown in Table 1
E Error Bars
During the model training process, we conducted tests using the parameters that achieved the lowest loss on the validation set. This process was repeated five times, and the error bars were calculated and presented in Table 4 The results in Table cableAclenn cleearly demonstrate the stability of our proposed method, TPGN, further confirming its superior overall performance compared to other baseline models, being SOTA approach.
F Comprehensive Experiment for Performance Variations with Different Forecasting Lengths
We conducted a comprehensive experiment to analyze the performance variation of the models across different forecasting lengths, mentioned in Subsection 4.1 According the experimental results in Table we selected representative methods from different paradigms: WITRAN -AN--Based), TimesNet BANN-B TimeMixer (MLP-Based) and iTransformer (Transformer- BBased, and compared them with TPGN. The results of dataset ECL have already been presented in Figure 3 Therefore, in this section, we present the other dataset results, as shown Figure 5 Figure and further analysis them.
MSE evez Difiree Focrenting Tialo
MAB uvez Diffcern Fenccniig Tacks
0.2
024
0.1
5 022
3022
021
JPG4
021
TFCV
TeaeWber
023
200
-00
600
100
350
1200
200
400
480
100
1200
1400
Figure 5: Experimental results with different forecasting lengths the Traffic dataset.
0.66
TP12V
0.22
Teatiterr
0.44
0.23
037
mandeesc
WITMAN
Popuudingggg Lengs
Focsuding Lengl
Figure 6: Experimental results with different forecasting lengths on the ETTh; dataset.
16

---

Diffact Roceaing
TPCN
0.5
TPWN
0322
142
0222
036
Figure 7: Experimental results with different forecasting lengths on the ETTh, dataset.
035
0.34
TPEV
022
034
030
MITRAN
WIFRAN
032
Figure 8: Experimental results with different forecasting lengths on the Weather dataset.
Through the results in Figure B Figure Figure Figure 7 and Figure X can clearly observe that as the sequence length increases, all models show varying degrees of performance degradation across different datasets. However, TPGN not only consistently maintains SOTA performance across all tasks but also exhibits slower performance decay with increasing forecasting lengths in most cases. This demonstrates the significant advantage of TPGN in predicting longer-range tasks, further confirming its ability to effectively extract information from limited inputs and apply them well to prediction tasks.
G Robustness Analysis
To assess the robustness of TPGN, we conducted experiments following the settings of MICN [Wang et al. 2023) and WITRAN (Jia et al. 2023), and introduced simple white noise injection. Specif- ically, a random proportion & data was selected from the original input sequence, and random perturbations within the range [-2X,, 2X, were applied to the selected data, where X; represents the original data. Subsequently, the injected noisy data was used for training, and we recorded the MSE and MAE metrics in Table
can be observed that as the perturbation ratio increases, there a slight increase in the MSE and MAE metrics in terms of forecasting. This indicates that TPGN exhibits good robustness in handling data with low noise levels (up to 10%) and has a significant advantage in effectively handling various abnormal data fluctuations.
H Parameter Sensitivity
In our model, we have only two hyperparameters. One is the number of hidden units, denoted as dm. which is a common hyperparameter in many models. Its value is determined through parameter search and further validation the validation set. During our parameter search process, have observed that different models exhibit significant variations in the optimal dm values for the same task. Similarly, even for the same model, this variability persists when facing different tasks. Therefore, in this paper, we focus our analysis solely on the parameter norm, as it showcases the notable differences and its impact on the model's performance.
In the Subsection 3.2 we mentioned that the parameter norm should be determined based on dataset, which has been previously acknowledged in related works [Jia et al. 2023). However, to validate the appropriateness our norm selection in the preparatory work, we conducted a statistical analysis on the variations across different task datasets, shown in Table
17

---

Table 5: Robustness experiments of TPGN's forecasting results. Different e indicates different proportions of noise injection.
Tasks
168-168
168-336
168-720
168-1440
Metric
MSB
MAE
MSE
MAB
MSE
MAE
MSE
MAE
0%
0.2107
0.3264
0.2276
0.3446
0,2303
0.3550
0.2484
0.3775
1%
0,2116 0,3248
0.2277
0.3441
0.2311
0.3557
0.2484
0.3776
5%
0,2117
0.3270
0.2295
0.3464
0.2314
0.3553
0.2495
0.3783
10%
0.2142
0.3288
0.2303
0.3466
0.2332
0.3561
0.2497
0.3784
Traficc
0%
1%
0.1201
0.1196
0.1861
0.1857
0.1154
0.1156
0. 181868
0.1869
0,1295
0.1293
0,2059
0.2057
0.1395
0.1390
0.2120
0.2114
5%
0.1899
0.1176
0.1913
0,1296
0,2064
0.2200
10%
0.1261
0.1964
0.1205
0.1960
0.1324
0.21 19
0,14
0.2241
0%
0.1061
0.2533
0.1110
0,2625
0.1346
0.2908
0.1343
0.2941
1%
0.1061
0.2534
0.1110
0.26228
0.1348
0,2908
0.1350
0.2944
5%
0,1067
0,2546
0.1111
0.2631
0.2916 0.1355
0.1407
0.2990
10%
0.1113
0,2630
Weather ETThg
0%
0.2174
0,3623
0.2237
0.3769
0.2356
0.3898
0.2514
0.4070
1%
0,21778
0.3629
0.2237
0.3770
0,2358
0..900
0.2521
0.4086
5%
0,2180
0.2240
0.3772
0,2358
0.3901
0.4090
10%
0,2187
O. 163699
0.2242
0,3775
0,2360
0,3904
0.2525
0.4097
0%
0.1877
0.3166
0,1978
0,3278
0.1925
0.3255
0.1786
0.3184
1%
0.1884
0,3184
0.1980
0.3282
0.1944
0.3273
0.1788
0.3186
5%
0,1886
0.3184
0.1987
0.3295
0.1950
0.3278
0.1788 0.3188
10%
0.1889 0.3185
1993
0.3320
0.1952 0.3281
0.1797
0.3197
Through statistical analysis of the dataset, we have found that there are some differences between the training set and the validation set. These differences are due to the inherent characteristics of time series data and are considered normal fluctuations. If the variances of the training set and validation set are roughly in the same range, it can be assumed that their fluctuations are roughly consistent. In this case, there are no significant differences in data distribution between the training set and validation set, so there is no need for normalization. The hyperparameter norm should be set O.
However, when there is relatively large difference in variance between the training set and the validation set (approximately twice half), it indicates a significant difference in data distribution between the two sets, In such cases, the hyperparameter norm should be set to I to facilitate better training of the model.
Additionally, weather datasets are a special case because they contain negative values, which leads to their mean being close to zero and significant difference between the variance and mean. This is reasonable for weather data. On the other hand, traffic datasets do not have negative values. Therefore, even if the variances of the training set and validation set are similar, their differences still be observed when analyzed in conjunction with the mean.
Table The distribution (Mean and STD) of dataset in the training and validation sets.
Disasta Talas
Trils
mairag mn
132711 63-3
A1ASTAL (4)
027141
LSW9- 24015
I Case Study
To highlight the advantages of TPGN in long-range time series forecasting tasks, we compared TPGN's showcase with the showcases of the second-best and third-hest models, selected based on MSE as the evaluation metric, for each dataset. The forecasting results showcases are shown in Figure to Figure 23
The comparison of the aforementioned cases clearly demonstrates the superiority of our method in terms of forecasting results. This unequivocally reaffirms the SOTA performance of TPGN across various domains. Moreover, it provides further evidence that PGN, as the novel successor to RNNs, can be effectively applied to long-range time series forecasting tasks.
18

---

TRGX
Hirar
TPON 168728
TION 100-340
MM Mww Mad
Mad
Figure 9: Forecasting cases of TPGN for all tasks in dataset ECL.
WITRAN 166 998
WITRAN 168 936
Unauad
M MM (M M000 Model MMl MW
200
Figure 10: Forecasting cases of WITRAN for all tasks in dataset ECL.
Hicar
Goocad Stach
Sidtom Phodeehee
MMdd
W00 Moo Mad / MM 1 M NW
Thensey
Figure 11: Forecasting cases of FiLM for all tasks in dataset ECL.
19

---

TRGX
TPON
TPON 168728
TPON 168-3440
Figure 12: Forecasting cases of TPGN for all tasks in dataset Traffic.
165-385
TinsMiser
Chua
260
200
80
300
1000
110
200
180
100
Figure 13: Forecasting cases of TimeMixer for all tasks dataset Traffic.
Sinciry
Hitra Stach
Sidto Phodleke
1993
Figure 14: Forecasting cases of iTransformer for all tasks in dataset Traffic.
20

---

198119
lizan Focse
Poote
164
TPON tec 1480
Prodecter
Figure 15: Forecasting cases of TPGN for all tasks in dataset ETTh1-
Tina ser 188 164
Trndhtnc 1el:25
Unaved
14
-15
20
Xakiy
Sodece
Time Mine 158 1648
Pradiciio
200
180
1000
Figure 16: Forecasting cases of TimeMixer for tasks dataset ETTh,
Sibit
Hilas Gooad
FIS 160-33)
Zedicke
10
1
Orand
Theusey
Figure 17: Forecasting cases of FITS for all tasks in dataset ETTh
21

---

Hiran
IPEN
Figure 18: Forecasting cases of TPGN for all tasks in dataset ETTh2-
WITRAN 166 99
WITRAN 168 996
200
Figure 19: Forecasting cases of WITRAN for all tasks dataset ETTh2-
Trandhone 198-10
Pormer 28-333
Hiteyy
Toansfemee 198-200
Thenway
Figure 20: Forecasting cases of iTransformer for all tasks in dataset ETTh
22

---

Canad Fosidd
IPEN lec
Figure 21: Forecasting cases of TPGN for all tasks in dataset Weather.
WITRAN 166 99R
WITRAN 168 995
Phrcle dey
WITRAN
200
180
1000
Figure 22: Forecasting cases of WITRAN for all tasks in dataset Weather.
Sabip
Dasaaberrc 108 18-720
Thensey
Figure 23: Forecasting cases of Basisformer for all tasks in dataset Weather.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: This paper has include the main claims in the abstract and introduction, which reflect the contributions and scope. For more details, please refer to Section T Section and Subsection
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We extensively discuss the limitation and future outlook of this paper in
Appendix A
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
24

---

Answer: [Yes]
Justification: Detailed derivations for the theoretical complexity analysis can be found in Subsection 3.3 of this paper.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversell, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have included the code repository address in the abstract. Additionally, detailed descriptions of the datasets we used can be found in Appendix while Appendix provides a comprehensive overview of all the experimental details. And the experimental results can be found in Section of this paper.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
25

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We utilized publicly available real-world benchmark datasets for research, and we provide detailed descriptions of these datasets in Appendix C Furthermore, at the end of the abstract, have included an anonymous link to access code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https
/
CuomeSumbeessc Sonte///ggideeeeeeeuuuuuiiiii ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if' applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Appendix C we provide an introduction to the data splits. Additionally, in Appendix D we offer detailed explanations of all experimental details, including the search space for hyperparameters, the selection of loss functions and optimizers, as well as process of selecting optimal hyperparameters based on performance the validation set. These sections are designed to enhance understanding.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We provide a detailed report on error bars in Appendix and a comprehensive analysis robustness experiments in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
26

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eπror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide detailed information about our experimental setup and computa- tional devices in Appendix D
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: After carefully reviewing the information on the url, we have confirmed that all aspects of our study conform with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
27

---

Justification: In Section 1
we introduce the background of our work and highlight its positive societal impacts. To the best of our knowledge, there are currently no apparent negative impacts associated with our work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out, For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper utilizes benchmark datasets, and poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cite the original paper that produced the code package or dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
28

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperssi thoode com/ com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the datasct/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
29

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "jpzTU4OIxe", "reviewer": "Reviewer_K1qz", "summary": "This paper proposes a new network called PGN to capture the long-term dependencies of time series. Based on PGN, this paper further design TPGN for long-range time series forecasting. TPGN consists of two branches to respectively capture the long-term periodic patterns and short-term information of time series. Extensive experiments are conducted to show the effectiveness and efficiency of the TPGN.", "questions": "In table of experiment comparison, could you explain why TPGN-long outperform TPGN in some cases.", "limitations": "Yes, the authors have fully discussed the limitations of their work.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. This paper is easy to follow. The motivations are clearly described by figures. The authors thoroughly analyze the information propagation modes of different time series forecasting models and explore new information propagation path to improve TS forecasting effectiveness.\n\nS2. The design of the PGN is novel, which is a completely new network architecture and can effectively solve the inherent problems of classic RNN models. Both experimental results and theoretical analysis show the effectiveness and efficiency of PGN.\n\nS3. This paper proposes TPGN upon PGN, which capture both the long-term and short-term characteristics of the time series with low computational complexity.\n\nS4. Experiments are sufficient. Five benchmark datasets are evaluated and the most representative models proposed recently are included in the experiments.", "weaknesses": "W1. The computational complexity of TPGN is not well discussed in this paper, and it would be better if the inference efficiency was adequately discussed as the time series size increases.\n\nW2. Some presentation needs to be improved. For example, it is difficult for readers to quickly get important conclusions on Table 1 and Table 4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate Reviewer K1qz for offering valuable insights and recognizing of our work.\n\n> Q1: The computational complexity of TPGN is not well discussed in this paper, and it would be better if the inference efficiency was adequately discussed as the time series size increases.\n\nThe TPGN architecture is designed to capture information more effectively from historical sequences. The complexity of processing input time series can be referenced in Sec. 3.3. For the forecasting module, our approach involves concatenating the local long-term and globally\naggregated short-term information obtained by TPGN to predict the future at different time points within a period. As a result, the dimensionality of the forecasting module changes from $P \\times (2 \\times dm)$ to $P \\times R_\\mathrm{f}$, with a complexity of $O(R_\\mathrm{f})$. As $R_\\mathrm{f} \\times P = L_f (L)$, the complexity of the forecasting module is $O(\\sqrt{L})$. Indeed, it is also important to note that since both the input and output serires must be fully stored in memory, the total memory complexity should be $O(L)$. However, this does not conflict with the $O(\\sqrt{L})$ complexity of TPGN. As the input seriers grows linearly, the complexity of TPGN also increases in a square root manner, with the actual time and memory overhead reflected in Figures 4(c) and (d). When the forecasting series grows linearly, although theoretically the complexity of TPGN should also increase in a square root manner, when $d_\\mathrm{m}$ is relatively large, the overhead of the forecasting module is much smaller compared to the encoder part. Therefore, as shown in Figures 4(a) and (b), the actual cost variation is not significant as the forecasting sequence grows.\n\n> Q2: Some presentation needs to be improved. For example, it is difficult for readers to quickly get important conclusions on Table 1 and Table 4.\n\nThank you for the reminder. In the revised version, we will carefully check the entire paper and enhance its presentation by adjusting colors, bolding text, particularly to emphasize the key conclusions in Table 1 and Table 4.\n\n> Q3: In table of experiment comparison, could you explain why TPGN-long outperform TPGN in some cases.\n\nFrom Table 2, we observe that in most datasets across various forecasting tasks, TPGN outperforms TPGN-long, with the exception of a few instances where TPGN-long performs better than TPGN in the Traffic dataset. Observing Figure 12, the Traffic dataset primarily exhibits strong periodicity along with short-term variations in different patterns. In this context, the long-term module in TPGN is already adept at capturing the predominant strong periodicity in sequences. However, in certain cases where the short-term variation patterns exhibit strong randomness, it may introduce disturbances to the short-term module of TPGN. Hence, there are scenarios in specific forecasting tasks where TPGN-long outperforms TPGN in terms of the MAE metric. Nevertheless, we also observe that TPGN remains best performance in terms of the MSE metric in these forecasting tasks.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iOm5tJy5Iz", "reviewer": "Reviewer_fqFX", "summary": "The paper introduces a new model paradigm which aims to solve the traditional bottlenecks of RNN models, such as non-parallel computation, gradient explosion/vanishing issues, etc.", "questions": "The model proposed in this paper is pretty neat and easy to follow. My questions mainly focus on the time complexity and experiments:\n1. I think the total amount of computation done in the PGN should be O(L^2) because it is 1 + 2 + … + L, which is O(L^2). Although the PGN removes half of the computations of the self-attention, the self-attention and PGN still share the same asymptotic complexity. Thus, technically, replacing the PGN with self-attention won’t change the time complexity asymptotically. But I agree that it should be faster than RNN since it enables parallel computation as self-attention does. Also, this is the reason why I think the overall contribution is less exciting than the paper title claims. Can the authors kindly address this issue?\n2. The above discussion can also be tested in the Efficiency of Execution experiment.\n3. Regarding the experiments, only an input length of 168 is tested. Why did the authors choose to fixate on this input length instead of testing some other options?\n4. In the ablation test, it seems that on some datasets (e.g., ETTh1), TPGN’s improvement is very slight compared to its LSTM/GRU/MLP ablations. Can the authors provide some analysis on these cases?\n5. I am also interested to see some analysis on what would happen if the PGN is replaced by self-attention.", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "1. An important problem is studied in this paper.\n2. The overall representation is clear and easy to follow.\n3. A comprehensive summary of the related work is provided.", "weaknesses": "1. The overall contribution is not very significant.\n2. Some questions regarding the time complexity and experiments need to be clarified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We express our sincere gratitude to Reviewer fqFX for providing comprehensive review, insightful perspectives, and thought-provoking questions.\n\n> Q1: I think the total amount of computation done in the PGN should be O(L^2) ... Can the authors kindly address this issue?\n\nSincerest gratitude for your detailed comments. Here we would like to provide more explanations about our model:\n\nFirstly, regarding the complexity analysis:\n\n(1) The computational complexity related to sequences in PGN mainly focuses on the HIE layer. In our specific implementation, we were inspired by the implementation details of methods like Transformer/iTransformer for the FFN part and used conv1D to replace Linear. Indeed, our actual convolution kernel (lookback window) size is set to $L-1$. However, typically in CNN computations, the convolution kernel size is considered a hyperparameter. For example, the convolution kernel in MICN[1] is actually linearly related to $L$, but the theoretical complexity of MICN remains $O(L)$. Therefore, when calculating the theoretical complexity in our case, we followed a similar approach.\n\n(2) The HIE layer can still be implemented using Linear. In this scenario, due to parameter sharing, the way each time step processes information from its previous $L-1$ time steps is exactly the same. Hence, we can follow the approach of WITRAN[2] in handling slices by expanding the dimensions that need to be processed in parallel and merging them into the sample dimension. In this case, Linear reduces $L-1$ dimensions to 1, so its theoretical complexity remains $O(L)$.\n\n(3) In our time series forecasting task, the lookback window size of PGN is set to $L-1$ to fully capture long-term information from the input signal. However, since PGN is a general paradigm, when applied to other tasks, the lookback window can be fixed to a certain value or adjusted. In such cases, regardless of whether implementation (1) or (2) is used, the theoretical complexity remains $O(L)$ or lower.\n\n(4) For self-attention, its computation between sequences is entirely different from PGN because it requires matrix operations to obtain information between every two time steps. Therefore, the necessary $O(L^2)$ theoretical complexity cannot be reduced using the aforementioned ways. From this, we can see that the essence of PGN is fundamentally different from self-attention.\n\nSecondly, In terms of innovation, we have focused on proposing PGN as the successor to RNN to address the issues stemming from RNN's excessive reliance on recurrent information propagation and overly long information pathways, including: (a) difficulty in capturing long-term dependencies in signals; (b) gradient vanishing/exploding; (c) low efficiency in serial computation.\n\n(1) Analysis of information propagation paths: PGN, through the HIE layer, can capture historical information at each time step with an $O(1)$ information propagation path, followed by information selection and fusion through Gated Mechanism, thereby altering RNN's $O(L)$ information propagation path. A shorter information propagation path lead to less information loss, better captured dependencies, and lower training difficulty. Consequently, PGN can more effectively capture long-term dependencies of inputs signals. And, as the information path shortens, it naturally resolves the issue of gradient vanishing/exploding.\n\n(2) Regarding efficiency, the thoughtful design of PGN allows for shared parameters in HIE, enabling information from subsequent time steps to be computed in parallel without waiting for the completion of computations from preceding time steps. This approach achieves good efficiency and resolves the inefficiency in serial computation of RNN.\n\nIn conclusion, through its innovative approach, PGN addresses all the limitations of RNN. Therefore, we claimed that PGN can be considered as the new successor to RNN.\n\n**Refs [1][2] details are in Sec. G of Global Rebuttal.**\n\n> Q2: The above discussion can also be tested in the Efficiency of Execution experiment.\n\nOkay, thank you for your suggestion. We replaced PGN with self-attention and conducted efficiency experiments. **Due to space constraints, we have placed the results in newly submitted PDF file. More details please refer to Sec. C of the Global Rebuttal**.\n\n> Q3: Why did the authors choose to fixate on input length of 168 instead of testing some other options?\n\nThis is because, on the one hand, considering factors such as the load capacity of training devices and data collection, forecasting longer futures with shorter sequence lengths is an important research problem, which has been the focus of much previous works, such as Informer, MICN, etc. On the other hand, in order for the model to have sufficient historical information for prediction from a limited sequence length, we aim for sequence length could be longer. Therefore, we chose 168 historical time steps (7 days) to predict the future (7 days, 14 days, 30 days, 60 days) as the forecasting tasks. This setting satisfies both considerations and serves as the basis for evaluating the performance of all models. Additionally, we also conducted experiments to demonstrate the reasonableness of our second consideration. **Due to space constraints, more details can be found in Sec. D in the Global Rebuttal**.\n\n> Q4: In the ablation test, some datasets ... Can the authors provide some analysis on these cases?\n\nThank you again for your detailed question. We have conducted relevant statistics and analysis, but **due to space constraints, we regret to inform you that we present all the details in Sec. E of the Global Rebuttal**.\n\n> Q5: I am also interested to ... what would happen if the PGN is replaced by self-attention.\n\nWithin the framework of TPGN, we replaced PGN with self-attention and conducted experiments on five datasets. **Due to space limitations, for experimental results and more detailed analysis, please refer to Sec. F in the Global Rebuttal**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FVXeiAOY8u", "reviewer": "Reviewer_t9pC", "summary": "This paper focuses on long-range time series forecasting problems. To address the limitations of RNNs, a novel paradigm called PGN is introduced as an alternative, providing shorter information propagation paths. Building upon PGN, the paper further presents a generic temporal modeling framework named as TPGN, which effectively captures both long-term and short-term periodic patterns, as well as local and global information, through a dual-branch design. The experimental results in this paper demonstrate that TPGN exhibits excellent performance in time series forecasting.", "questions": "Q1. Why was the Gated Mechanism designed in PGN this way in Figure 2 (a)? Can this part be replaced with GRU or other RNN variants?\n\nQ2. Is it necessary to have two Linear layers in the design of the short-term branch in TPGN?", "limitations": "yes", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1: This paper proposed a novel paradigm called PGN, which effectively tackles the inherent issues of RNNs through a simple yet powerful design. PGN exhibits a high level of innovation and holds the potential to replace traditional RNNs.\n\nS2: TPGN primarily focuses on modeling the temporal dimension. Its dual-branch design makes sense as it captures both long-term and short-term periodicity, as well as the local and global characteristics of time series. Additionally, it is reasonable to set up different univariate forecasting tasks to evaluate TPGN's performance.\n\nS3: This paper is well-written, and the presentation of the figures and tables is clear, making it easy to understand and follow. The experimental comparisons are comprehensive, including numerous advanced baseline models such as iTransformer, ModernTCN, FITS, TimeMixer, PDF, WITRAN, and Basisformer.", "weaknesses": "W1. For tables with a large amount of content, such as Table 1, it may be beneficial to consider using different colors for highlighting, as it could enhance clarity. Additionally, another option to consider is moving some of the experimental results to an appendix.\n\nW2. While TPGN exhibits some advantages in terms of efficiency, I have noticed that it still appears to be challenging to reach the optimal level. Specifically, I have noticed that as the input sequence size increases, the efficiency of TPGN may gradually become inferior to that of iTransformer.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We extend our sincere appreciation to Reviewer t9pC for providing valuable feedback and acknowledgment of our research.\n\n> Q1: For tables with a large amount of content, such as Table 1, it may be beneficial to consider using different colors for highlighting, as it could enhance clarity. Additionally, another option to consider is moving some of the experimental results to an appendix.\n\nThank you for your suggestion. In the revised version, we will enhance the table presentation by utilizing color changes, bolding, and other methods. Additionally, we will thoroughly check and optimize the entire paper.\n\n> Q2:  While TPGN exhibits some advantages in terms of efficiency, I have noticed that it still appears to be challenging to reach the optimal level. Specifically, I have noticed that as the input sequence size increases, the efficiency of TPGN may gradually become inferior to that of iTransformer.\n\nWe have noticed that in terms of efficiency, linear-based methods such as FITS do indeed perform better. And for iTransformer, since it maps the entire sequence into a patch for processing the time dimension, while TPGN requires long-term and short-term information extraction branches in the time dimension, the efficiency of TPGN may slightly lag behind iTransformer when the input sequence length varies. However, TPGN still holds significant advantages as it can maintain good efficiency while ensuring optimal performance.\n\n> Q3: Why was the Gated Mechanism designed in PGN this way in Figure 2 (a)? Can this part be replaced with GRU or other RNN variants?\n\nIn our design, only one gate is needed for information selection and fusion, which incurs smaller overhead compared to GRU's two gates and LSTM's three gates. While PGN, as a general architecture, can have its gated mechanism replaced with GRU and other RNN variants, this substitution would introduce higher costs. In terms of performance, we have observed that despite having only one gate, PGN still outperforms GRU or LSTM, more details of the experiments can be found in Table 2.\n\n> Q4: Is it necessary to have two Linear layers in the design of the short-term branch in TPGN?\n\nYes, because in the short-term branch, the semantic information extracted by the two linear layers carries different meanings. The first layer primarily aggregates short-term information, extracting localized information. The second layer further aggregates the representation from the first layer at a higher level, extracting global information. While it is possible to extract global semantic information using a single linear layer, it would overlook the extraction of information at different granularity levels. Additionally, in terms of efficiency, the theoretical complexity of using only a single linear layer is $O(L)$, which is larger than the current $O(\\sqrt{L})$.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QzOGXcqoUk", "reviewer": "Reviewer_Bhsw", "summary": "This paper proposed a Parallel Gated Network (PGN) as a successor to RNN, featuring a Historical Information Extraction (HIE) layer to directly capture information from previous time steps. Additionally, it introduces a Temporal PGN (TPGN) framework with two branches to capture both long-term periodic and short-term semantic patterns, demonstrating state-of-the-art performance in long-range time series forecasting.", "questions": "See Weaknesses.", "limitations": "The authors have already described the limitation of this work in the paper, namely the lack of modeling multivariate relationships. To some extent, this is not a significant issue because many current cutting-edge studies have demonstrated that focusing solely on univariate temporal relationships can also be effective in multivariate tasks.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. This paper compares a variety of cutting-edge methods.\n2. The experiments are generally thorough.", "weaknesses": "1. The major issue with this paper is the lack of analysis and comparison with significant literature. The entire paper's premise is the traditional RNN's failure in long-term sequence problems due to the long information propagation paths of its recurrent structure. However, as far as I know, SegRNN[1] has already addressed these shortcomings of traditional RNN in long-term forecasting through segmented iteration and parallel prediction. Yet, there is no discussion on this in the paper. Please compare your method with it and clarify your differences and advantages.\n\n2. In Section 2, you should distinguish between Linear-based and MLP-based methods. The former has only single-layer parameter connections, while the latter has multiple layers and can learn non-linear features due to the presence of activation functions. Methods like DLinear and FITS should be classified as Linear-based methods.\n\n3. The description of HIE is unclear: (i) The process shown in Figure 2(a) suggests first performing linear mapping and then zero-padding, which conflicts with Equation 1 in the paper, where H = HIE(Padding(X)), and the actual code. It is recommended to modify Figure 2 to make this clearer. (ii) Line 169 describes that “HIE(·) is a linear layer,” but in practice, the behavior of HIE is more like a sliding aggregation operation of CNN (or TCN) rather than a sorely linear mapping. Given **(ii)**, calling the proposed method RNN-based is debatable since it is more likely TCN-based. \n\n4. You should include an ablation analysis of the normalization layer, explaining its impact on TPGN achieving state-of-the-art results.\n\n5. Although the authors provide source code, it does not include the hyperparameter settings required to reproduce the key results in the paper, meaning there is no directly runnable script. Are the hyperparameters in the main results all defaults? For instance, is TPGN_period=24? If not, providing a complete script file that can be run directly is necessary.\n\n\n[1] Lin, S., Lin, W., Wu, W., Zhao, F., Mo, R., & Zhang, H. (2023). Segrnn: Segment recurrent neural network for long-term time series forecasting. arXiv preprint arXiv:2308.11200", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the comprehensive review, detailed feedback, and valuable suggestions from the reviewer Bhsw.\n\n> Q1: The major issue with this paper is the lack of analysis and comparison with significant literature. Please compare your method with SegRNN and clarify your differences and advantages.\n\n| **Advantages** | Information propagation paths | Whether a decoder is unnecessary | Directly capturing periodic semantic information |\n|:--------------:|:-----------------------------:|:--------------------------------:|:------------------------------------------------:|\n| **TPGN (PGN)** |              $O(1)$             |                 ✓                |                         ✓                        |\n|   **SegRNN**   |             $O(L/W)$            |                 ✗                |                         ✗                        |\n\n(1) Regarding information propagation paths: Shorter information propagation paths lead to less information loss, better captured dependencies, and lower training difficulty. In this regard, PGN can achieve an information propagation path of $O(1)$, while SegRNN, although capable of segment-wise iteration, still maintains an information propagation path of $O(L/W)$, where $W$ represents the number of segments. This is where PGN holds a greater advantage.\n\n(2) In terms of time series forecasting design: Although SegRNN adopts parallel prediction in the decoder stage, successfully addressing the error accumulation issue of iterative RNN predictions, our approach, TPGN, draws inspiration from Informer's design, omitting the need for a decoder module. This not only resolves the error accumulation problem but also results in a more concise model design.\n\n(3) Direct extraction of periodic semantic information: TPGN can directly extract semantic information related to the periodicity of time series data, while SegRNN segments the time series first and then feeds the representation of each segment into GRU units, which hinders direct capture of the periodicity within the time series.\n\nThe segmented iteration and parallel prediction of SegRNN are indeed thought-provoking. We have conducted preliminary experiments and observed that TPGN (PGN) has shown some improvement compared to SegRNN. In the revised version, we will incorporate the above analysis.\n\n> Q2: In Section 2, you should distinguish between Linear-based and MLP-based methods. Methods like DLinear and FITS should be classified as Linear-based methods.\n\nThanks again for your valuable suggestion. We will conduct a full check and modification in the revised version.\n\n> Q3: The description of HIE is unclear: (i) Figure 2(a) conflicts with Equation 1 in the paper, where H = HIE(Padding(X)), and the actual code. It is recommended to modify Figure 2 to make this clearer. (ii) Line 169 describes that “HIE(·) is a linear layer,” but in practice, the behavior of HIE is more like CNN (or TCN) rather than a sorely linear mapping. Given (ii), calling the proposed method RNN-based is debatable since it is more likely TCN-based.\n\n(i) Thank you for pointing out the issue. **We have made modifications to Figure 2 (a), and the new figure is shown as Figure A in the PDF file (newly submitted)**. \n\n(ii) Thanks again for your detailed review. In fact, HIE is indeed a linear layer. However, during our implementation, we were inspired by the design of methods like Transformer and iTransformer for the FFN part, and we also used conv1D to replace Linear. \n\n(iii) The function of the HIE layer is to replace the recurrent structure of RNN. In terms of specific code implementation, whether using conv1D or Linear, it does not affect its function of linear mapping. Additionally, the linear operation of HIE is just a part of the structure in PGN. Following HIE, there is the Gated Mechanism, and together they form the entirety of PGN. Therefore, PGN can be considered as the new successor to RNN-based methods.\n\n> Q4: You should include an ablation analysis of the normalization layer, explaining its impact on TPGN achieving state-of-the-art results.\n\nOkay, since the normalization layer was not used in the ECL, ETTh2, and WTH datasets, we focused on conducting ablation experiments on the normalization layer for the Traffic and ETTh1 datasets. **Due to space limitations, the experimental results have been placed in Table A of the PDF file. For details, please refer to Sec. A of the Global Rebuttal.**\n\nWe observed a noticeable decrease in performance when the normalization layer was removed from the two datasets. This drop can be attributed to the inherent characteristics of the datasets. In Table 6, we calculated the distribution (Mean and STD) of the training and validation set for each dataset. The results indicate significant differences in the distributions of the Traffic dataset and the ETTh1 dataset. These differences can introduce disturbances to the model, hence the $norm$ should be set to 1 in this case. More details can be found in Appendix H.\n\n> Q5: It does not include the hyperparameter settings required to reproduce the key results in the paper. Are the hyperparameters in the main results all defaults? For instance, is TPGN_period=24? If not, providing a complete script file that can be run directly is necessary.\n\nFor the 'TPGN_period', we were inspired by WITRAN and aimed to partition time series based on natural period. Therefore, for each task in the five datasets, 'TPGN_period' was set to 24. The selection of $norm$ varied depending on the dataset, as detailed in Table 6. The value of $dm$ for each forecasting task was determined through the validation set, with specific details available in Appendix D. **Due to space limitations, we have included the hyperparameters of our model for each task in Table B of the PDF file for result reproducibility, please refer to Sec. B of the Global Rebuttal for more details**. Furthermore, we will also enhance the files in the source code moving forward.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank reviewers for their thorough review and valuable suggestions.

# A  Ablation analysis of the normalization layer on the Traffic and ETTh1 datasets

**The experimental results can be found in Table A in the PDF file (newly submitted).** We observed a noticeable decrease in performance when the normalization layer was removed from the two datasets. This drop can be attributed to the inherent characteristics of the datasets. In Table 6, we calculated the distribution (Mean and STD) of the training and validation set for each dataset. The results indicate significant differences in the distributions of the Traffic dataset and the ETTh1 dataset. These differences can introduce disturbances to the model, hence the $norm$ should be set to 1 in this case. More details can be found in Appendix H.

# B Specific hyperparameters for reproducing our model

**We have presented the specific hyperparameters of our model for different tasks across all datasets in Table B of the PDF file (newly submitted)**, facilitating the direct reproducibility of our experimental results.

# C Efficiency of Execution experiment of replacing PGN with self-attention in TPGN

Within the TPGN framework, we replaced PGN with self-attention (referred to as TPGN-Attn). The results indicate that the overall practical overhead of TPGN-Attn is inferior to TPGN, **as shown in Figure B of the newly submitted PDF file.**

Specifically, following the setup outlined in Sec. 4.3, we conducted two sets of efficiency experiments on TPGN-Attn. In one set of experiments, we kept the input length fixed at 168 and varied the output length to 168/336/720/1440, while in the other set, we fixed the output length at 1440 and varied the input length to 168/336/720/1440. We integrated the results of TPGN-Attn with Figure 4, resulting in a new Figure B (For specific details, please refer to the newly submitted PDF file).

# D The experimental results for forecasting 1440 steps using 96 steps as input

We compared the performance of forecasting 1440 steps using 96 steps input to using 168 steps input. The results show that, in the 96-1440 scenario compared to the 168-1440 scenario, all models exhibited varying degrees of performance decline, with TPGN still demonstrating its advantage. This suggests that increasing the input length appropriately when predicting longer futures with shorter inputs provides the model with more information for improved performance. **The detailed experimental results can be found in Table C in PDF file (newly submitted).**

Specifically, we selected the three models ranked second in Table 1, WITRAN, iTransformer, and TimeMixer, and compared their performance in experiments predicting 1440 steps using 96 steps. 


# E An analysis of the experimental results regarding TPGN and its LSTM/GRU/MLP ablations

We calculated the average improvements of TPGN compared to TPGN-LSTM/TPGN-GRU/TPGN-MLP across various tasks on each dataset. We found that the improvements were not as pronounced on the ETTh1 and Traffic datasets compared to the other datasets. Additionally, we also calculated the improvements of TPGN over the second-best model performance in Table 1 in terms of MSE across various tasks. We have summarized the results as follows. The table presents the average improvement of TPGN over the compared models across various tasks.

| Dataset | The second-best model | TPGN-LSTM | TPGN-GRU | TPGN-MLP |
|:-------:|:---------------------:|:---------:|:--------:|:--------:|
| Traffic |         9.38%         |   3.54%   |   3.33%  |  19.54%  |
|  ETTh1  |         3.79%         |   5.14%   |   5.52%  |   4.02%  |

(1) On the ETTh1 dataset, although the numerical improvement of TPGN over its LSTM/GRU/MLP ablations is not substantial, comparing TPGN with the second-best model reveals a significant degree of improvement. This discrepancy can be attributed to the characteristics of the ETTh1 dataset, making the improvement appear less pronounced compared to other datasets.

(2) On the Traffic dataset, TPGN shows less improvement over its LSTM/GRU ablations. This is due to the strong periodicity of the Traffic dataset, as observed in Figure 12. In this scenario, the TPGN framework plays a crucial role in capturing this strong periodicity effectively. Despite PGN in TPGN using only one gate, its performance surpasses that of GRU with two gates and LSTM with three gates, demonstrating that PGN can serve as the new successor to RNN.

(3) Furthermore, the results in the above Table reveal that the performance of the ablated variants of LSTM/GRU/MLP ablations across different datasets, indicating their limited adaptability to diverse domains. In contrast, TPGN demonstrates a greater advantage in this regard.

# F Forecasting performance of replacing PGN with self-attention in TPGN

Within the framework of TPGN, we replaced PGN with self-attention (called TPGN-Attn). The results show that TPGN exhibits a significant advantage over TPGN-Attn. **The specific experimental results are presented in Table D in the PDF file (newly submitted).**

Based on the results in the Table D, we calculated that TPGN outperforms TPGN-Attn in terms of MSE, with an average improvement of 26.97%. In particular, TPGN demonstrates an average reduction in MSE of 27.18% for the ECL dataset, 24.87% for the Traffic dataset, 6.99% for the ETTh1 dataset, 29.62% for the ETTh2 dataset, and 46.19% for the Weather dataset. The analysis above demonstrates the advantages of TPGN.

# G References

MICN[1]: Wang, H., et al. (2023). Micn: Multi-scale local and global context modeling for long-term series forecasting. In The eleventh international conference on learning representations.

WITRAN[2]: Jia, Y., et al. (2023). WITRAN: water-wave information transmission and recurrent acceleration network for long-range time series forecasting. In Proceedings of the 37th International Conference on Neural Information Processing Systems.

Meta-review: The submission addresses the problem of lengthy information paths in RNNs by proposing a fully parallel gated model with O(1) path length. The approach is novel and well supported by discussions and empirical results. The rebuttal discussion led to a convergence in the reviewers' opinion, which broadly support acceptance of the work.
Decision: Accept (poster)