Parsimony or Capability? Decomposition Delivers Both in Long-term Time Series Forecasting
Authors: Jinliang Deng
Feiyang Ye
Du Yin
Xuan Song
Ivor Tsang
Hui Xiong
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Time Series Forecasting
Spatial-temporal Prediction
Decomposition
Primary area: other

Abstract: Long-term time series forecasting (LTSF) represents a critical frontier in time series analysis, characterized by extensive input sequences, as opposed to the shorter spans typical of traditional approaches. While longer sequences inherently offer richer information for enhanced predictive precision, prevailing studies often respond by escalating model complexity. These intricate models can inflate into millions of parameters, resulting in prohibitive parameter scales. Our study demonstrates, through both theoretical and empirical evidence, that decomposition is key to containing excessive model inflation while achieving uniformly superior and robust results across various datasets. Remarkably, by tailoring decomposition to the intrinsic dynamics of time series data, our proposed model outperforms existing benchmarks, using over 99\% fewer parameters than the majority of competing methods. Through this work, we aim to unleash the power of a restricted set of parameters by capitalizing on domain characteristics—a timely reminder that in the realm of LTSF, bigger is not invariably better. The code is available at \url{https://anonymous.4open.science/r/SSCNN-321D/}.

Parsimony or Capability? Decomposition Delivers Both in Long-term Time Series Forecasting
Jinliang Deng 1.2
Du Yin" Xuan Song Ivor Tsang" Hong Kong Generative Al Research and Development Center 2 Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology (SUSTech),
Feiyang Ye
Hui Xiong 7
3 University of Technology Sydney, University of New South Wales,
CFAR and IHPC, Agency for Science, Technology and Research, Singapore, School of Artificial Intelligence, Jilin University,
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou).
8 CSE, The Hong Kong University Science and Technology
{dengjinliang, xionghui>eust .hk, feiyang. ye uts@gmail........ com
du. wut.yinnunss.c edu. songxuan@jlu edu. cn, Ivor "or_TTann@cfan ar a-star. www...rddd..........eee
Abstract
Long-term time series forecasting (LTSF) represents critical frontier in time series analysis, characterized by extensive input sequences, as opposed to the shorter spans typical of traditional approaches. While longer sequences inherently offer richer information for enhanced predictive precision, prevailing studies often respond by escalating model complexity. These intricate models can inflate into millions parameters, resulting in prohibitive parameter scales. Our study demonstrates, through both analytical and empirical evidence, that decomposition is key containing excessive model inflation while achieving uniformly superior and robust results across various datasets. Remarkably, by tailoring decomposition the intrinsic dynamics of time series data, our proposed model outperforms existing benchmarks, using over 99% fewer parameters than the majority of competing methods. Through this work, we aim to unleash the power of a restricted set parameters by capitalizing on domain characteristics- timely reminder that in the realm of LTSF, bigger is not invariably better. The code is available at https::/ h/gi::://gttaan thg.. com/JLDeng/ /SSCN
Introduction
Time series forecasting is cornerstone in the fields of data mining, machine learning, and statistics, with wide-ranging applications in finance, meteorology, city management, telecommunications, and beyond (Jiang et al. 2021 Han et al. 2023a Zhang et al. [2020] Wu al. 2020 [209 Cui et al. 2023; Zhang et al. 2017]] Liang et al. 2018 Zhu et al. 2024 Fan et al.] 2022, 2023). Traditional univariate time series models, such as Auto- Regressive Integrated Moving Average (ARIMA) and Exponential Smoothing, fail to capture the intricate complexities present in open, dynamic systems. Fortunately, the advent of deep learning has marked significant shift in this domain. Recently, the utilization the Transformer /Vaswani et al.] (017)) model has revolutionized time series forecasting, setting new benchmarks in the accuracy of forecasting models due to its capability of depicting intricate pairwise dependencies and extracting multi-level representations from sequences.
Inspired by the extraordinary power exhibited by large language models (LLMs),, expanding model scale has increasingly become the dominant direction in the pursuit of improvement for time series
"Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

forecasting. Currently, the majority of advanced models require millions of parameters Nie et al. 2023] Zhang & Yan 2023 Wu et al. 2023). With the recent introduction of pre-trained large language models, such as I. Ma and GPT-2, the parameter scale has inflated to billions (Jin et al. 2024 Cao
jtt al. 2024; Jia et al. 2024a Zhou et al., 2023 Gruver et al. (024)) Despite the significant increase in the number of parameters to levels comparable to foundational models for language and image, the efficacy of these models has seen only marginal improvements. In particular, these large models have shown up to only 30% improvement in MSE and MAE, yet at the cost of 100 to 1000 times more
parameters compared to a simple linear model across representative tasks (Nie et al. 2023] Cao et al.
2024, Jim et al. 2024). Additionally, we have observed a convergence in the abilities demonstrated by the models since the advent of PatchTST (Nie et al. 2023). with recent advancements achieving only incremental improvements.
These evidences indicate that large model may not necessarily be a prerequisite for the future of time series forecasting, motivating us to explore the opposite trend-minimizing the number of parameters. Before introducing our design, we reflect on why existing methods struggle to maintain their optimal effectiveness with a reduced number of parameters. Existing methods predominantly adopt data patching over either the temporal or spatial dimensions (Zhou et al. 2021 Wu et al. 2021 Zhou et al. 2022 Nie et al., 2023] Liu et al. 2024a; Zhang & Yan| (023), which, in conjunction with the attention mechanism, allows them to capture complex temporal and spatial dependencies. However, significant drawback of data patching is the elimination of temporal (or spatial) identities along with the destruction of temporal (or spatial) correlations, resulting in the potential loss of complex temporal (or spatial) information. To counteract this undesirable information loss, these methods establish a high-dimensional latent space to accommodate the encodings of the temporal and spatial identities in addition to the embeddings of the real-time observations. As result, the dimensionality of the latent space typically scales with the number of identities to be encoded, inevitably leading to the exponential inflation of parameter scale. Moreover, the expansion of model size makes the models prone to overfitting, a common challenge in time series tasks where data is often limited.
To achieve capable yet parsimonious model, it is fundamentally paramount to reinvent the paradigm to maintain and harness the spatial and temporal regularities, eliminating unnecessary and redundant parameters for encoding them into the latent space. Recent studies (Deng et al.) [021]] [202 Wang ct al. 2024) showcase the potential of feature decomposition in attaining improved efficacy with limited parameters. While remarkable progress has been made, these methods struggle with long-term forecasting, especially for datasets exhibiting intricate temporal and spatial correlations, such as the Traffic and Electricity datasets (Deng et al.) (024). Moreover, the analytical aspect of decomposition along with its relation to patching is under-explored, hindering further advancements in this line of research.
In response to these limitations, we propose a Selective Structured Components- based Neural Network (SSCNN). For the first time, we address the amalytical gap in feature decomposition, providing insights into its rationale for capability and parsimony compared to patching. In addition, SSCNN enhances plain feature decomposition with a sclection mechanism, enabling the model to distinguish fine- grained dependencies across individual time steps, which is crucial for improving the accuracy of the decomposed structured components and, ultimately, the overall prediction accuracy. SSCNN has been benchmarked against state-of-the--e (SOTA) methods, demonstrating consistent improvements ranging from 2% to 10% in efficacy while using 99% fewer parameters than SOTA LTSF methods, including PatchTST (Nie et al. 2023) and iTransformer (Liu et al.) 2024a). Remarkably, it uses 87% fewer parameters than DLinear (Zeng et al., 2023) when tasked with extensive long-term forecasting.
Our contributions can be summarized as follows:
1. We introduce SSCNN, decomposition based model innovatively enhanced with selection mechanism. This model is specifically designed to adeptly capture complex regularities in data while maintaining a minimal parameter scale.
2. We conduct in-depth comparison between decomposition and patching, examining both capability and parsimony.
3. We carry out comprehensive experiments to demonstrate SSCNN's 's superior performance across various dimensions. These extensive evaluations not only prove its effectiveness but also highlight its versatility in handling diverse time series forecasting scenarios.

---

2
Related Work
The field of time series forecasting spatial-temporal prediction has traditionally leveraged multi- layer perceptrons (MLPs) (Zhng et al. (017). recurrent neural networks (RNNs) (BAT et al. 2020
Zhao et al.
2019 Jiang et 2023; Jia al. 2024b), graph convolution networks (GCNs) Yu et al.
2018), and temporal convolution networks (TCNs) (Bai et al., 2018). The recent development of
ST-Norm (Deng et al. 2.2T), STID (Shao al. 2022a) and STAEformer (Liu et al. 2023) shows
promise in enhancing model capabilities to distinguish spatial and temporal features more effectively. Motivated by the success of self-supervised learning and pre-training in natural language processing (NLP) and computer vision (CV), these two techniques are also gaining attention and application in
this field (Guo et al. 2021 Shao et al. 2022b)
Over the last few years, the focus has shifted to long-term sequence forecasting (LTSF). The majority of studies have concentrated on adapting the Transformer (Vaswani et al. 2017). successful in NLP (Devlin et al. (0188) and CV (Khan et al. 2022). for LTSF tasks. Pioneering works like LogTrans (Li 2019) addressed the computational challenges of long sequences through sparse attention mechanisms. Subsequent developments, such as Informer (Zhou et al. 2021). Autoformer (Wu ct al. 202I), and Fedformer (Zhou et al. 2022). introduced innovative approaches to improve predictive accuracy with temporal feature characterization, autocorrelation-bee based series similarities, and frequency domain conversions, respectively. Other notable contributions include the Non- stationary Transformer (Liu et al. 2022) and Triformer (Cirstea et al. 2022). Furthermore, diverse normalization techniques have been developed to mitigate the distribution shift present in time series data (Liu et al.) 2024b) Kim et al. 2021). Probabilistic forecasting (Kollovieh et al., 2024) and irregular time series forecasting (Chen et al. 2024 Ansari et al. 2023) are two growing subfields receiving increasing attention.
significant shift in LTSF research occurred with DLinear (Zeng al. 2025), an embarrassingly simple linear model. DLinear highlighted the limitation of Transformers in capturing the unique ordering information of time series data (Zeng al. (023). To overcome this limitation, recent methods like PatchTST (Nic et al.) 2023). TimesNet (Wu et al. (023). Crossformer (Zhang & Yan 2023). and iTrransfommer (Liu et al. 2024a) blend the global dependency capabilities of Transformers with the local order modeling strengths of MLPs. However, the success of these methods is achieved the cost of an enormous number of parameters.
A handful of emerging studies have sought to reduce parameter usage in pursuit of a parsimonious
model (Lin al., 2024, Xu et al. 2024; Wang al., 2024 Deng al., 2024). The techniques
they employed to remove redundancies fall into three categones: downsampling (Lin et al. 2024). decomposition (Deng al.) [024 Wang et al., 2024). and Fourier transform (Xu et al. 20244] Yi et al.,
2024). Despite efficient parameter usage, these methods often compromise accuracy for specific datasets, especially those presenting complex yet predictable patterns, such as Traffic (Deng et 2024 Xu et al., (004)). Our study, as far as we know, is the first to realize a parsimonious model without any sacrifice of capability.
3 Selective Structured Components- Neural Network
In multivariate time series forecasting, given historical observations X = fxi,""", ,NN E NTT. with N variates and Tin time steps, we predict the future Tour time steps X E R-NxTm. The input data goes through the processing, visualized in Fig. [ for predicting the unknown, future data. Over the course of prediction, sequence of intermediate representations structured into two distinct branches: the top branch illustrates the inference process used to derive the components accompanied by the residuals, while the bottom branch depicts the extrapolation process, forecasting the potential evolution of these components. The components and residuals obtained are combined into wide vector, which is then input into a polynomial regression layer to capture their complex interrelations. In the following sections, we detail the inference and extrapolation processes for these components, respectively.
yielded. Essentially, SSCNN is
3.1 Temporal Component
We invent temporal attention-based normalization (T-AttnNorm) to decompose the temporal compo- nents, consisting of the long-term component, the seasonal component, and the short-term component,
3

---

Input
Embedding
T-AttnNorm
T-AttnNorm
T-AttnNorm
S-AttnNorm
Polynomial Regression
H
F
H
R
MatMul
MatMul
MatMul
R 12
Regression
Polynomial
Projection
Output
Conv
Conv
Conv Conv
Figure 1: An overview of the SSCNN. The grids are used exemplify the selection maps I' and E as defined the main text, with Ti. Tou and instantiated as 4, 4 and 3, respectively.
in sequential manner. The inference for each of these three components is applied to the repre- sentation of each series individually along the temporal dimension, with selection/attention map characterizing the dynamics of the component of interest. Upon inference, our model disentangles the derived structured component from the data, resulting in residual term summarizing other remaining components. The initial representation of the time series, the derived structured component, and the residual term are denoted as H H",,"" R E N xxxxxxx, xd, respectively. The selection map is denoted as I' E R xT, T-AttnNorm is formulated as follows:
Mr R T-AAttNorml :H; (H;
(1)
where n4 I -T²=-'H;² - 142 c, R - To ensure the unbiasedness of nu and oj the sum of each row of I" is constrained 1. The distinction between the three components is the realization of I" The residual term resulting from the current block is taken as the input for the subsequent block, e.g.. _ R
Then, the component series and the residual series are respectively extrapolated to forward horizons by mapping parameterized with E" X
p. R Extrapolate R(;""),
where i; En - with Ar R xd leading R R² E N.Touxx Similar to I", E" is also defined as matrix with the sum of each row equal to 1. Next, we introduce how to realize I" and * to extract and simulate the dynamics of the considered components, respectively.
Long-Tere Term Component The long-term component aims to characterize the trend patterns time series data. To acquire an estimation of the long-term component with less bias, we aggregate the samples collected across multiple seasons, eliminating the seasonal and short-term impacts that impose only local effects. The realizations of the inference and extrapolation matrices for the

---

long-term component are given by:
(i,,)) T m
Ti
The attention mechanism is excluded from the long-term component it does not improve forecasting accuracy in our datasets. The attention mechanism beneficial when a component's distribution changes significantly over time, helping reduce estimation bias. However, for our long-term component, the distribution remains stable throughout the input period. In such cases, the attention mechanism adds value, making redundant and unnecessary.
Seasonal Component The seasonal component created to model the seasonal fluctuation. In- ference of the seasonal component operates under the assumption of consistent eycle duration to streamline the detection of seasonal trends. We introduce to indicate the length of cycle, Tin to signify the maximal count of cycles encompassed by the input sequence, i.e., Tin Tin and Tout to denote the minimal number of cycles covering the output sequence, i.e., Tour 2 Tour To simplify the notation, we assume that Ti multiple of c, i.c., Ti Tin C.
To acquire unbiased and precise seasonal components, we introduce a parameter matrix We RTx allocating distinet weights to each pair of periods to represent the inter-cycle correlations. The matrix undergoes row-wise normalization via a softmax operation to satisfy the constraint on the sum of 1. The selection map for inferring the seasonal component is defined as:
W
== 1 121, (mod c)
(T,
Σ A exp(W
(4)
Otherwise
where [.] denotes the floor function.
When dealing with extrapolation, we define Wae Th as the parameter matrix capturing the correlations between each pair of cycles encompassed by the input and output sequences, respectively. The selection map for extrapolating the seasonal component is written as follows:
exp( Wie W.e)
[. LeJ, 0 (mod e).
80h (1,))=
Σ
exp(War Nao
(5)
Otherwise
Short-Term Component The short-term component discerns irregularities and ephemeral phe- nomena unaccounted for by the seasonal and long-term components. contrast the long-term component, it necessitates only limited window size, 6, encapsulating recent observations with immediate relevance. Moreover, these observations exhibit varying degrees correlation depending on the associated lag. Therefore, the inference of the short-term component involves parameter vector w i and is expressed mathematically as:
explw!) M exp(w))
(i ()
(1,1))=
(6)
Otherwise
The extrapolation the short-term component bifurcates based on the targeted horizon. Immediate horizons retain correlations with preceding estimations of the short-term component, prompting regression-bel based forecasting with a parameter vector w E 8 Conversely, as the horizon extends ahead, it accumulates compounded uncertainties, decreasing the predictability. Herein, opt for zero-padee paro-padding to climinate unnecessary parameters. The entire extrapolation formalized as follows:
exp(we
(<< Tm --
8dd (i.
M exp(w),)
Otherwise
3.2 Spatial Component
The spatial component refers to the component that is temporally irregular, i.e., cannot be captured by the aforementioned three temporal components, but spatially regular, i.e., showing consistent behavior across group of residual series. The inference of the spatial component, referred to
5

---

as spatial attention based normalization (S (S-AttnNorm), shares similar formula representation as the temporal component, except that it is applied each frame independently along the spatial dimension:
R = S-AttnNormHH... (
(8)
where p., I',,,, = = m.t cc, R Hi We acquire the similarities die
among the series in terms of the spatial component by applying correlation to the residuals post controlling the long-term, seasonal, and short-term components. Given that each series is represented by a Tin x d matrix in the hidden layer. we vectorize them before performing the measurement. This results in similarity matrix 2sd TRNN with each entry I (i,j) representing the conditional correlation between the ith series and jth series:
(1,)) exp(vec(H,) vec (oo(H)))
k= exp(vec(H,)) vec vec (H))
Considering the erratic and unpredictable nature of the spatial component along time, we simply realize component extrapolation with zero aero-padding: P = 0 and R 0.
3.3
Component Fusion
The Polynomial Regression layer is inspired by the work of Deng et al. (2024), where we extend the original module include both additive and multiplicative relations. This extension allows us to model more complex interactions between the decomposed components.
H,- Conv (Conv& (S:) & Conva + Conv&
(10)
where
S. [ p4 R 14 P,, RT The resulting H, is fed to the next layer as H.! for further manipulation.
Comparison Between Decomposition and Patching
Capability Analysis We first demonstrate that the representation space accommodating only plain single-step observation is ill-structured due to the entanglement of diverse component signals. Then, elucidate how patching and decomposition adjust the structure of the representation space, facilitating the capture of faithful and reliable relations among the samples. The detailed derivations for this section provided in Appendix A
For simplicity, we assume that time series & is driven by two distinct components a and b, but the analysis can be trivially extended to cases with multiple components. Consider a triplet (It, Tty Itg) collected at three time steps f1, 12, and f3, respectively, where Itz = at + b, for = 1, 2, 3, subject to az, at,, bt, big. and a,, # aty. We expect that 21, should be closer 2t, than Fty, given llar, - atall < |a:, at, along with |lbr, - b = bb, bt, ll. However, this relationship may not necessarily hold with hepe observations, i.e., there exist specified triplets (fr, &rg Lig) such that || - $ > || - St,ll. |l. due to the entanglement of a and
Patching can rectify sample relations by creating orthogonality between distinct components, resulting in a well-structured representation space. By incrementally augmenting the time series represen- tation with preceding observations, i.c., [zt-p. 21.. the a-component ([at-p... a.) and the b-component (ib-ps> ... be]) become increasingly orthogonal, reducing negative interference caused by their interactions. It is worth noting that while patching is widely acknowledged for enriching the semantic information of step-wise data (Nie et al. 2023), no formal explanation from the perspective of disentanglement has been offered yet.
Distinct from patching, which implicitly attenuates the interactions among distinct components, decomposition can explicitly avert any interactions. This is done by first recovering and subsequently distributing different components into disjoint sets of dimensions, completely isolating them in independent subspaces. Specifically, this involves finding decomposition mapping D at, + by.

---

(ae, b such that - D -D(zt)))((z,,))) The effectiveness of decomposition depends on how accurately the model can recover and b. Thus, it paramount to ensure that no irrelevant information is included in estimating the component of interest, suggesting the necessity of using a selection mechanism. The above analysis is based on deterministic data, but similar results can be obtained if the data possesses stochasticity, which is the case for real-world applications. See detailed discussion in Appendix
Parsimony Analysis To justify that SSCNN is fundamentally more parameter-efficient than patching-based methods, we analyze how the number of required parameters scales with Tim Tour- and dpashh//soop. where dpatch and daee represent the dimensionalities of the representation spaces for patching- based and decomposition-basee methods, respectively.
Patching- Ratching-based methods establish intense connections from backward variables to forward variables via multiple layers of patch-wist wise fully-comnected networks, typically necessitating (dgacch Tin Tou + de (anch) parameters (Nie et al. 2023). Empirically, the optimal performance of these models achieved when dpuch is adjusted within the range of 64 and 256, resulting in an explosion in the total number of parameters, especially for large Tin and Tou--
In the case of SSCNN, the portion scaling with Tin. Tour, and dsee contains O + Ti
I + don parameters. These come from three sources: the convolution operators in the polynomial regression module, the attention map responsible for characterizing seasonal patterns, and the predictor. In contrast to dpuchh daep can be assigned a small number, e.g., 8, which sufficient to prompt the model to exhibit its full potential, making daap orders of magnitude smaller than Lunch Additionally, SSCNN significantly reduces the number of connections scaling with Tin and Tour from duuch in T to 1 T + zz T + dstep Tout, thanks to the capability of decomposition in eliminating redundant correlations across variables, as further analyzed Appendix [ The presence of the scaling factor 1 makes SSCNN even more parameter-efficient than DLinear (Zeng et al. 2023) for large Tin and Tout. when the sum of terms related to Tin and Tout dominates O(diep). The actual
results are presented in Sec. 5.2
5
Evaluations
In this section, we conduct comprehensive experiments across standard datasets to substantiate from various perspectives that SSCNN achieves SOTA performance with minimal parameter count.
5.1
Experiment Setting
Datasets We evaluate the performance of our proposed SSCNN on seven popular datasets with diverse regularities, including Weather, Traffic, Electricity, and four ETT datasets (ETTh1, ETTh2, ETTm1, ETTm2). Among these seven datasets, Traffic and Electricity consistently show more regular patterns over time, while the rest contain more volatile data. Detailed dataset descriptions and data processing are provided in Appendix B
Evaluation Metries In line with established practices LTSF (Nie et al. 2023 Wu et al. 2021
evaluate the model performance using mean squared error (MSH) and mean absolute (MAE).
Baseline Models We compare SSCNN with the state-of-the-an thr- models, including (1) Transformer-
based methods: Autoformer (Wu et al. 202T), Crossformer (Zhang Yan 2023), TST (Nie)
al. 2023) and i'Transforee| Liu et al. 2024a), (2) Linear-based method: DLinear Zeng et al. (2023);
(3) TCN- Cassedd method: TimesNet Wu et a.. (2023); (4) Decomposition- -aaed method: Mixer
(Wang et al.) 2024), SCNN (Deng et al. (2024). For implementing state-of-tee of-the-arrr- models (SOTAs),
we adhere to the default settings as provided in the Samn--Seres-Libuuu
Network Setting All the experiments NVIDIA 2080тi IIGB GPU. For ECL and Traffic, SSCNN is configured with 4 layers, and the number of hidden channels d is to 8. We set to a value from {2, 8, 16). The kernel size for the
implemented in PyTorch and conducted on single
hptps://gttubb.co///uu//// Series Serrrs-
7

---

Table 1: Multivariate forecasting results with prediction lengths S [3, 24, 96, 192, 336). Results are averaged from all prediction lengths. Full results are listed in Appendix C
The best result is
highlighted in bold, and the second best is highlighted with underline.
SSCNN
SCNN
Traansormer
TaneMiser
PaccTTT
TimesNet
Crossformer
DLinear
Autolormer
Models
(Ours)
(2024)
(2024)
(2024)
(2023)
(2023)
(2023)
(2023)
(2021)
Metric
MSE MAE
MSE
MAE
MSE
MAE
MSE MAE
MSB MAE
MSE
MAE
MSE
MAF
MSE MAP
MSE MAE
ECL
0.118
0.212
0.128
0.222
0.121
0.216
0.123
0.215
0.125
0.219
0.163
0.267
0.124
0.221
0.141
0.236
0.191
0.308
Traffic
0.338 0.236
0.360
0.254
0.343
0.246
0.387
0.271
0.349
0.241
0.579
0.314
0.375
0.254
0.426 0.290
0.583 0.420
ETThl
0.330
0.363
0.339
0.368
0.359
0.387
0.348
0.375
0.335
0.372
0.399
0.418
0.349
0.378
0.368
0.393
0.452 0.466
ETTh2
0.255
0.257
0.274
0.331
0.264
0.264
0.356
0.289
0.338
0.282
0.365
ETTml
0.242
0.300
0.244
0.302
0.261
0.315
0.258
0.316
0:248
0,305
0.271
0.318
0.272
0.318
0:259
0.312
0,464
0.445
ETTm2
0.158
0.236
0.158
0.235
0.175
0.251
0.164
0.241
0.167
0.240
0.180
0.258
0.168
0.244
0.167
0.251
0.225 0.306
Weather
0.139
0.175
0.140
0.178
0.158
0.190
0.143
0.182
0.153
0.184
0.165
0.206
0.150
0.194
0.161
0.209
0.185 0.231
Log10(e Params)
log30re FLORS)
SOWS
250N
VICTS Trandferee
Timellaar
Thentheerr
Trandel
DUne
Cuner
192 236
330
Forward window
Forward window size
Backward window 52c
Backward window ozc
(a)
(b)
(c)
(d)
Figure 2: Examination of parameter scale and computation scale against the forward window size and the backward window size on the ECL dataset.
convolution used in polynomial regression is chosen as 2. The total number of parameters resulting from this configuration is around 25,000. For the other five datasets showing unstable and volatile patterns, the spatial component is found to be useless due to the absence of spatial correlations, thus this component is disabled. Additionally, the number of layers is configured as 2, resulting in around 5,000 parameters. During the training phase, we employ L2 loss for model optimization. In addition, the batch size is set 8, and Adam optimizer is used with learning rate of 0.0005.
5.2 Comparative Analysis with Baselines
Forecasting Accuracy As reported in Table I for Electricity and Traffic, SSCNN showcased remarkable proficiency by achieving the lowest MSE and MAE. Specifically, on the Electricity dataset, SSCNN outperformed iTransformer and PatchTST, two representative baselines, by 2% and 4%, respectively. On the Traffic dataset, SSCNN again surpassed them by 2% and 3%, respectively. By comparing SSCNN to SCNN, the benefit of the selection mechanism is evident, leading to approximate 8% improvement, highlighting the necessity of modeling fine-graine aru-ggriied correlations for complex yet regular patterns. On the other five datasets, we find that the attention mechanism benefits little, resulting in comparable performance between SSCNN and SCNN. When it comes to the comparative analysis with three leading Transformer- based models, including iTransformer, PatchTST, and Crossformer, SSCNN exhibits notable improvements by 8.8%, 4.4%, and 8.3%, respectively, on average. This suggests the competitiveness of SSCNN in handling irregular dynamics.
Parameter Scale Parameter scale is represented by the number of parameters. This metric varies with the lookback window size and the forward window size for all models, as illustrated in Fig. 22 and Fig. 2c| respectively. Apparently, SSCNN and DLinear emerge the top parameter-efficien models, requiring 99% % fewer parameters than other models. Remarkably, SSCNN proves to be even more parsimonious than DLincar when tasked with extensively long-term forecasting, showing considerable reduction of 87% in parameter scale at the forward window size of 336.
Computation Scale Computation scale measured using the number of floating point operations
(FLOPs), as illustrated in Fig. 2b and Fig. 2d respectively. SSCNN falls short in computation scale compared to iTransformer. This is because, instead of performing compression using fully connected layers, it manages and processes the entire sequence each layer. As a result, the computational
8

---

0.20
0.33
Trarsforma
5SChN
0.18
0.19
SSCKN
Transoommer
0.52
SSCNN
ITrasslomma
0.50
Irrenfoommd
862
0.12
OLincor
PaschtST
Crasstmmer
0.26
0.15
Cressformer
0.31
FatchTST
0.43
PatchTss
near
0.10
0.34
0.3
0.29
05
330
185 Backward s widdow size
100 Barkward window size
336
335
Bactward window size
Baccward window size
(a) ECL. Tou 24
(h) ECI. Tow: 96
(c) ETTh2 Tour 96
(d) ETTh2 Tou: 336
Figure 3: Impacts of backward window size.
0.26
ECL
EGLL Trafflc
ECL Traffic
ECL
Traffic
Traffic
0.25
ECL Traffic
0.24
0.23
0.22
2
3
5
2
12
2
5
12
24
42
Layers
Hidden
Kemel Size
Cycle Length
Short-tem Span
Figure 4: Sensitivity analysis of haperepssameters on the ECL and Traffic datasets.
cost scales with the sum of the backward window and forward window sizes, raising issue to be addressed in future work. Despite the relatively high computational cost, SSCNN has the potential to be accelerated through parallel computing.
Sensitivity to Lookback Window Size As shown in Fig. for ECL data, although SSCNN initially lags behind other state-of-ther models at window size of 96, progressively outperforms them as more historical data is included. In contrast, iTransformer and Crossformer achieve ad advanta- geous performance with shorter input ranges but exhibit diminished gains from extended historical data. Furthermore, when handling the volatile fluctuations of ETTh2 data, SSCNN demonstrates indisputably enhanced capability with respect the lookback window size compared to competing methods, some of which even show degraded performance due to the overfitting problem.
5.3 Comparative Analysis of Model Configurations
Analysis of Syper-parameters As shown in Fig. 4 the efficacy of SSCNN is impacted by adjustments the five considered hyperparameters, showing up to 10% difference between the best and the worst records, Focusing on the performance change against cycle length, we find that the misalignment between the configured value and the authentic value of results significant drop in effectiveness, verifying our presumption. Additionally, the short-term span also has noglneglig influence on the outcome.
Ablation Study We create seven variants: The first six variants are used assess the individual contributions of the four components as well as the two introduced attention maps, respec- tively. The last variant, represented as w FCN', is constructed by inserting an additional fully-connected network (FCN) between backward variables and forward variables to verify the re- dundaney of the FCN, which is preva- lently adopted by previous works (Liu et al. 2024a Nie et al. (023) to cap- ture temporal correlations, under the framework of SSCNN.
MSE
0.15
MSE
0.41
0.14
0.39
0.3
0.7
812
035
SSCNIN w/o It
w/o sta
w/o st
w/o sea se
FCN
SSCNNN w/o It w/o sta
w/o st
w/o sea
w/b se
w FCN
w/l
w/o
w
win
(a) ECL
(b) Traffic
Figure 5: Performance comparison with various component in ECL and Traffic dataset.
evident
from Fig.
5 that all these components

---

are vital, with the seasonal component being the most prominently important. Interestingly, our approach shows that basing dependencies on the evaluation inter-channel conditional correlations actually enhances outcomes, contradicting the evidence delivered by Han et al. (2023b); Nie et al. (2023), where channel-dependent strategy was seen as detrimental to model performance. Fur- thermore, we note that 'w FCN' does not bring any benefit, proving the sufficiency of the proposed inference and extrapolation operations for characterizing useful dynamics.
Conclusions, Limitations and Broader Impacts
In this research, developed SSCNN, a structured component-based neural network augmented with a selection mechanism, adept at capturing complex data regularities with a reduced parameter scale. Through extensive experimentation, SSCNN has demonstrated exceptional performance and versatility, effectively handling variety of time series forecasting scenarios and proving its efficacy.
While the proposed SSCNN model achieves satisfactory results in reducing parameter usage, it falls short terms of computational complexity, as shown in Fig. [b and Fig. 2d This drawback, however, has the potential to be mitigated in future work by identifying and eliminating redundant computations with down-sampling techniques. Moreover, our future research will explore the potential for automating the process of identifying the optimal neural architecture, using these fundamental modules and operations as building blocks. This approach promises to alleviate the laborious task of manually testing various combinations in search of the optimal architecture for each new dataset encountered.
Time series forecasting can significantly benefit various sectors, including meteorology, economics, traffic management, and healthcare. However, also poses potential negative impacts. Inaccurate forecasts can lead to significant economic operational disruptions. Additionally, the large amounts of data required for accurate forecasting may include personal or sensitive information, raising concerns about data privacy and potential misuse.
7
Acknowledgment
The research was supported by theme-based Research Scheme (T45-205/21-N) from Hong Kong RGC, Generative Al Research and Development Centre from InnoHK, and the Open Project Pro- gram of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University (No. VRLAB2024A **). This work was also partially supported by the grants of National Key Research and Development Project (2021 171114444 114400) of China, Jilin Provincial International Cooperation Key Laboratory for Super Smart City and Zhujiang Project (2019QN01S744) This work was supported in part by the National Key R&D Program of China (Grant No. No.2023YFF0725001, part by the National Natural Science Foundation of China (Grant No. 02372244,, in part by the guangdong Basic and Applied Basic Research Foundation (Grant No.2023B1511 15SBBB555122000 51552007),, and in part by Boangzhou-HUUST(GGG Joint Funding Program (Grant No. 20.20023A00 A03J0000,, Education Bureau of Guangzhou Municipality.
10

---

References
Ansari, A. F., Heng, A., Lim, A., and Soh, H. Neural continuous-diseets state space models for irreullrly-samplle time series. In International Conference Machine Learning, pp. 926-951. PMLR, 2023.
BAI, L., Yao, L., Li, C., Wang, X., and Wang, C. Adaptive graph convolutional recurrent network for traffic forecasting. Advances in Neural Information Processing Systems, 33, 2020.
Bai, S., Kolter, J. Z., and Koltun, V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv: 100. 01271, 2018.
Cao, D., Jia, F., Arik, S. O., Pfister, T., Zheng, Y.. Ye, W., and Liu, Y. TEMPO: Prompt-based genera- tive pre-trained transformer for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024. URL htLhhtts::// https:s:: /ttptr:e///pppeevieeeee net
Chen, Y., Ren, K., Wang, Y., Fang. Y., Sun, W., and Li, D. Contiformer: Continuous-time transformer for irregular time series modeling. Advances in Neural Information Processing Systems, 36, 2024.
Cirstea, R.-G., Guo, C., Yang, B.. Kieu, T., Dong, X., and Pan, S. Triformer: Triangular, variable- specific attentions for long sequence multivariate time series forecasting- version. arXiv preprint (ttt::22041376677 2022.
Cui, Y., Li, S., Deng, W., Zhang, 7., Zhao, J., Zheng. K., and Zhou, X. Roi-demand traffic prediction: A pre-train, query and fine-tune framework. In 2023 IEEE 39th International Conference Data Engineering (ICDE). pp. 1340-1352. IEEE, 2023.
Deng, J., Chen, X., Jiang, R., Song, X., and Tsang, I. W. St-norm: Spatial and temporal normalization for vulti-variate time series forecasting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 269-278, 2021.
Deng, J., Chen, X., Jiang, R., Yin, D., Yang. Y.. Song, X., and Tsang. I. W. Disentangling struc- tured components: Towards adaptive, interpretable and scalable time series forecasting. IEEE Transactions on Knowledge and Data Engineering. 2024.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: 04.000 2018
Fan. W., Zheng. S., Yi, X., Cao, W., Fu, Y., Bian, J., and Liu, T.-Y. Depts: Deep expansion learning for periodic time series forecasting. arXiv preprint Xii::2203.07681, 2022.
Fan, W., Wang, P., Wang, D., Wang, D., Zhou, Y., and Fu, Dish-ts: general paradigm for alleviating distribution shift in time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 7522-7529, 2023.
Gruver, N., Finzi, M., Qiu, S., and Wilson, A. G. Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems, 36, 2024.
Guo, K., Hu, Y., Sun, Y., Qian, S., Gao, J., and Yin, B. Hierarchical graph convolution networks for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 151-159, 2021.
Han, J., Liu, H., Zhu, H., and Xiong, H. Kill two birds with one stone: A multi-view multi-adversarial learning approach for joint air quality and weather prediction. IEEE Transactions on Knowledge and Data Engineering, 2023a.
Han, L., Ye, H.-J., and Zhan, D.C. The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting. arXiv preprint 2304:2330 2023b.
Jia, F., Wang, K., Zheng, Y., Cao, D., and Liu, Y. Gpt4mts: Prompt-based large language model for multimodal time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 23343-23355 2024a.

---

Jia, Y., Lin, Y.. Hao, X., Lin, Y., Guo, S., and Wan, H. Witran: Water-wave information transmission and recurrent acceleration network for long-range time series forecasting. Advances in Neural Information Processing Systems, 36, 2024b.
Jiang, R., Yin, D., Wang, Z., Wang, Y., Deng, J., Liu, H., Cai, Z., Deng. J., Song, X., and Shibasaki, R. DI-traff: Survey and benchmark of deep learning models for urban traffic prediction. In Proceedings the 30th ACM international conference on information & knowledge management, 2021.
pp.
Jiang, R., Wang, Z., Yong, J., Jeph, P., Chen, Q. Kobayashi, Y., Song, X., Fukushima, S., and Suzumura, T. Spatio-temporal meta-graph learning for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 8078-80888 8086, 2023.
Jin, M., Wang, S., Ma, L.,, Chu, Z., Zhang, Y., Shi, X., Chen, P.-Y.. Liang, Y., Li, Y.-F,, Pan, S., and Wen, Q. Time-LLM: Time series forecasting by reprogramming large language models. In The Twelfth International Conference on Learning Representations, 2024. URL https:// openreview
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., and Shah, M. Transformers in vision: A survey. ACM computing surveys (CSUR), 54(10s): 41,, 2022.
Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.-H., and Choo, J. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2021.
Kollovieh, M.,
Ansari, A. F., Bohlke-Schneider, M., Zschiegner, J., Wang, H., and Wang, Y. B. Predict, refine, synthesize: Self-guiding diffusion models for probabilistic time series forecasting. Advances in Neural Information Processing Systems, 36, 2024.
Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang, Y-X., and Yan, X. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting, Advances in Neural Information Processing Systems, 32:5243-522 -2533, 2019.
Liang, Y., Ke, S., Zhang, J., Yi, X., and Zheng, Y. Geoman: Multi-level attention networks for geo-sensory time series prediction. In ISCAI, volume 2018, pp. 3428-3434, 2018.
Lin. S., Lin, W., Wu, W., Chen, H., and Yang. J. Sparsetsf: Modeling long-term time series forecasting with Ik parameters. arXiv preprint arXiv: 009444 2024.
Liu, H., Dong, Z., Jiang, R., Deng. J., Deng, J.. Chen, Q. and Song, X. Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting. In Proceedings of the 32nd ACM international conference on information and knowledge management, pp. 4125-4129, 2023.
Liu. Y., Liu, Q. Zhang, J..W., Feng, H., Wang, Z., Zhou, Z., and Chen, W. Multivariate time- series forecasting with temporal polynomial graph neural networks. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openrreiiet
12EJh
Liu, Y., Hu, T. Zhang, H., Wu, H., Wang, S., Ma, L.,, and Long, M. itransformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024a. htttpt:t:///:ttttreet
new. http///ooouumdddaaaAAaa.a.a.
Liu, Z.,, Cheng, M., Li, Z., Huang, 7.,, Liu, Q., Xie, Y., and Chen, Adaptive normalization for non-stationary time series forecasting: A temporal slice perspective. Advances in Neural Information Processing Systems, 36, 2024b.
Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long- term forecasting with transformers. In The Eleventh International Conference Learning Representations, 2023.
Shao, 7., Zhang, Z., Wang. F., Wei, W., and Xu, Y. Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting. Proceedings of the 31st ACM International Conference Information & Knowledge Management, pp. 4454-4458, 2022a.
12

---

Shao, Z., Zhang, Z., Wang, F., and Xu, Y. Pre-training enhanced spatial-tempones graph neural network for multivariate time series forecasting. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1567-1577, 2022b.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L.,, Gomez, A. N., Kaiser, E., and Polosukhin, I. Attention is all you need, Advances in neural information processing systems, 30, 2017.
Wang, S., Wu, H., Shi, X., Hu, T., Luo, H., Ma, L., Zhang, J. Y., and Z.OU, Timemixer: Decom- posable multiscale mixing for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024. URL https:
Wu, H., Xu, J., Wang. J., and Long, M. Autoformer: Decomposition transformers with auto- correlation for long-term series forecasting. In Thirty-Fifth Conference on Neural Information Processing Systems, 2021.
Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., and Long, M. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations, 2023.
Wu, Z., Pan, S., Long, G., Jiang. J., and Zhang, C. Graph wavenet for deep spatial-temporal graph modeling. In International Joint Conference on Artificial Intelligence 2019, pp. 1907-1913. Association for the Advancement of Artificial Intelligence (AAAI), 2019.
Wu, Z., Pan, S., Long, G., Jiang, J., Chang, X., and Zhang, C. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 753-763, 2020.
Xu, Z., Zeng, A., and Xu, Q. FITS: Modeling time series with $10k$ parameters. In The Twelfth International Conference on Learning Representations, 2024. URL http:::/ net/ forun?id=bWcav23qk
Yi, K., Zhang, Q., Fan, W., Wang, S., Wang. P., He, H., An, N., Lian, D., Cao, L.,, and Niu, Z. Frequency-domain mips are more effective learners in time series forecasting. Advances in Neural Information Processing Systems, 36, 2024.
Yu, B., Yin, H., and Zhu, Z. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp. 3634-3333 26400, 2018.
Zeng, A., Chen, M., Zhang, L.,, and Xu, Q. Are transformers effective for time series forecasting? 2023.
Zhang, J., Zheng, Y., and Qi, D. Deep spatio-temporal residual networks for citywide crowd flows prediction. In Thirty-first AAAI conference on artificial intelligence, 2017.
Zhang, W., Liu, H., Liu, Y., Zhou, J., Xu, T., and Xiong. H. Semi-supervised city-wide parking availability prediction via hierarchical recurrent graph neural network. IEEE Transactions on Knowledge and Data Engineering, 34(8):3984-38) 3996, 2020.
Zhang, Y. and Yan, J. Crossformer: Transformer utilizing cross- doss-dimension dependency for multivari- ate time series forecasting. In The Eleventh International Conference on Learning Representations, 2023.
Zhao, L., Song, Y., Zhang, C., Liu, Y., Wang, P., Lin, T., Deng, M., and Li, H. T-gen: A temporal graph convolutional network for traffic prediction. IEEE Transactions on Intelligent Transportation Systems, 21()):3848-3885 3858, 2019.
Zhou, H., Zhang, S., Peng. J., Zhang, S., Li, J., Xiong, H., and Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of AAAI, 2021.
Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., and Jin, R. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning, pp. 27268-77286. PMLR, 2022.
13

---

Zhou, T., Niu, P., Sun, L., Jin, R., et al. One fits all: Power general time series analysis by pretrained Im. Advances in neural information processing systems, 36:43322--4333 2023.
Zhu, Y., Ye, Y., Zhang, S., Zhao, X., and Yu, J. Difftrajj: Generating gps trajectory with diffusion probabiliistcc model. Advances in Neural Information Processing Systems, 36, 2024.
14

---

A Additional Materials for Section
Suppose the time series x is driven by two distinct vector components a and b. Considering a triplet
(ata -u (ty) sampled from three time steps 81, 22 and i3. respectively, where 2t, = Ct + be, for
i = 1,,,3, subjecting to ats = atg. Dig = btg and at, # atg. To examine the relationship between
||2,, Ftall and || 2tg ll. we analyze the difference between the squares of them:
| - Ftall 112 - 1 - -rta 11 = ||ar, + bb - 012 - b12 112 - -Iaa + bbg - 0es - ltal?
llbr, bta 112 - -Ilot, + bb: att beal?
=-llac, atsll 112 - 2(ar. ats bs. biz)-
(II)
Therefore, the sign of the difference is determined by the relation between dt, asg and br, bra If (r1- at and bt, big are orthogonal, implying that their inner product is zero, then Eq. can be further reduced to - -laa, ats 11 resulting in a rigorously negative number, which means that 2i2 is constantly closer to It, than zty- Whereas, if they are not perpendicular to each other, which is the case for the step-wise representations of time series data, the sign can be arbitrary. To make it more accessible, we express b,, btg as the addition of two components, i..., U1 + U2 where in = Maa, - At2),, representing the projection of by bt, on at, 0tg. and it2 = by htg un, explaining the residual part. Substituting this expression of bi, btg into Eq. 11). we obtain -(1 + 2 - atall² At this point, it is easy to capture that the relationship between [x,, Dt, I and || Tta |l depends on the value taken by X. Therefore, 2t₁ is likely to be farther Ft₂ than Ttg. even Ie, and It, share one same component.
(I)
The patching operation, i.c., P x4 - (Zi-p--.. 22) restructures the representation space by adjusting the distances among the samples. in particular, the difference between ||,,, Fit ll and ||0t stall |l is transformed into the following form:
[P()))) - P(x)))l) 111 (I - P(
[ - Vig p: ... ft, - F12 11/2 111 [ - -ps-pp" Dt, F'g 1|
= [[001-p- a12-p:' :.' ,1, - -a+,)) ar,)11 - [(b,,pp- - bp ..... bb, )12
+ 2(((t1-p (11-- 0t, at2), (b) b12-p 112-p" , br2)) )}
[1at)) Otg 02, a:. || [1(br, bts br btg 11
2((at1-p Ots-p,
at at,), (be, bbg- p bb, bea))-
(12)
Given that and display independent dynamics, the correlation between (at-p:" aa) and (be-p.... be) is increasingly approaching zero as p grows. Therefore, any terms involving the inner product between and b can be ignored. However, another critical prerequisite needs to be additionally satisfied for the success of rectification:
[0t,-p- ... .a,, should be closer to [Atz-p: [ats-ps '.. atg] provided that an = at, and at, # Ozg. This also applies to the b-component. For components showing stable and consistent behaviors, such as long-term and seasonal components, this prerequisite readily holds for arbitrarily large patch sizes. In contrast, for volatile components. such as the short-term component, the patch is likely to contain increasingly irrelevant information as p grows, resulting in an adversely increased disparity between 21 and Z2. Therefore, patching- based methods fall short in capturing the short-term component.
(t2, than
By using the decomposition mapping D : ax + bt. + (ar, b, we have [|D D(x..)) (K,, by - by I|| < (1(r, Atg. br, b,,))l = ()x,),,) D(xt,))l. Therefore, we obtain || D(=+,)) < (
[
D(rtg) Il. We now extend our analysis to the case where the time series is sampled from different data distributions, e.g.. Gaussian mixture models. Specifically, we consider a triplet (zt, Ity Fty)). where Tt, is sampled from the distribution AIN oal) + V2,.( o,.)). Here, the Gaussian distributions A N(Ha.s, oal) and A ((H.., o,.) represent two different components for zt,, and X; represents the weights for the Gaussian mixture model. The goal of the decomposition approach is to decouple each component. Therefore, the decomposition projection will take the form of D: : \v + A2N, + (A,Na, A2N.)T This decomposition projection can adjust the structure of the representation space and better separate data with different distributions. To show this, we suppose that |a,1 = Ma,2 holds. Then for any I3 sampled from Nu I..) /) + X2 (Hb,3, a,() where Hb,3 = n4, holds, we expect that the distance between I and 2a is more likely to be closer than the distance between 21 and v3, since Ti and I2 have the same distribution component. However, the original representation cannot guarantee that the inequality E|[a1 x2 1 < E| 21112
15

---

holds. By using the decomposition mapping, 24, is mapped to the vector (Ai )2b1,) ) where at and bt, follow the distributions (H T,I) and (H TTJ), respectively. Then we have
E ( -D( )l2 > at, at, 2| + Aل b,, b Since Pa.1 Na.2. we have E dt, 0tg 1 a,, at, It follows that E| (z,,, D(x,,)]1 I a2 at, l ( Therefore, the decomposition mapping ensures that the inequality [E] ((x)) ((22) (()))) (2aa) holds. This helps rectify the incorrect
b - =
relations between data.
B
Data Details
B.1 Dataset Description
We evaluate the performance our proposed SSCNN on popular datasets with diverse regularities, including Weather, Traffic, Electricity and 4 ETT datasets (ETTh1, ETTTh,,ETTh,, ETTm1, ETTm2). Among these 7 datasets, Traffic and Electricity consistently show more regular patterns over time, while the rest datasets contain more volatile data. Weather dataset collects 21 meteorological indicators in Germany, such as humidity and air temperature. Traffic dataset records the road occupancy rates from different sensors on San Francisco freeways. Electricity is dataset that describes 321 customers' hourly electricity consumption. ETT (Electricity Transformer Temperature) datasets are collected from two different electric transformers labeled with I and 2, and each of them contains 2 different resolutions (15 minutes and hour) denoted with m and h. Thus, in total we have 4 ETT datasets: ETTm1, ETTm2, ETTh1, and ETTh2. These datasets have been extensively utilized for benchmarking and publicly available on Time-SSries-LLine Library
B.2
Data Preprocessing
For the datasets with 1 1-hour sampling rate Sate-EElcttiiiny Traffic, ETTh1, and ETTh2-thh the input length Ti uniformly
168 for all models. Conversely, for datasets with 10-minute or 15- minute sampling rate, Tin is adjusted to 432 or 384, respectively. to ensure coverage of multiple periods. The output length varies among [3, 24, 96, 192, 336), accommodating both short-term and long-term forecasts. To ensure fairness in the comparison, all competing methods adopt this window size configuration. Additionally, data are pre-processed by normalization and post- prot-processed by de- de-normalization across all baselines.
C Full Results
The full multivariate forecasting results provided in the following section due to the space limita- tion of the main text. We extensively evaluate competitive counterparts on challenging forecasting tasks. Table contains the detailed results of all prediction lengths of the seven well-acknowledgee forecasting benchmarks. The proposed model achieves comprehensive state-of-the-art in real-word world forecasting applications. We also report the standard deviation of SSCNN performance under five runs with different random seeds in Table which exhibits that the performance of SSCNN is stable.
D Implications of Decomposition on Spatial-Tempore Correlations
We showcase that decomposition facilitates to wipe out redundant correlations, leading to sparse temporal and spatial correlation maps. We first define two concepts, namely the conditional correlation and conditional auto-correlation. Then, we utilize these two concepts to analyze the temporal correlations and spatial correlations, respectively.
D.1 Conditional Correlation and Conditional Auto-correlation
Conditional correlation, or also known as partial correlation, refers to the degree of association between two series, with the effect of set of controlling series removed, while conditional auto- correlation refers to the correlation of a series with delayed copy of itself, given a specific set of controlling series.
Phaps//gittubb.co//uuu//iiiii Series--eeeeeeee Series-Librrry Library
16

---

Table 2: Long-tern term forecasting results on 7 real-world datasets in MSE and MAE. The best result is
highlighted in bold, and the second best is highlighted with underline.
Models
SSCNN
SCNN
FTransfomner
Tixer
PaichTST
TumexNel
Crosslormer
DLineur
Autofommer
(Ours)
(2024)
(2024)
(2024)
(2023)
(2023)
(2023)
(2023)
(2021)
Metric
MSE
MAE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE MAE
MSE
MAE
MSE
MSE MAE
3
0.057
0.149
0.059
0.152
0.059
0.152
0.060
0.153
0.063
0.160
0.119
0.252
0.058
0,151
0.077
0.175
0.147 0.273
24
0.093
0.187
0.096
0.192
0.094
0.189
0.097
0.191
0.100
0.197
0.135 0.245
0.098
0.195
0.122
0.221
0. 0.168 0.286
96
0.128
0.221
0.148
0.241
0.133
0.229
0.134
0.226
0.136
0.230
0.169
0.272
0.136
0.238
0.154
0.248
0.186
0.301
192
0.151
0.243
0.160
0,252
0.154
0.247
0.152
0,244
0.152
0.244
0.191
0.288
0.158
0.255
0.168
0.260
0.218 0.328
336
0.165
0.261
0.181
0.274
0.169
0.263
0.171
0.262
0.172
0.264
0.203
0.300
0.173
0,268
0.186
0.277
0.238 0.351
3
0.241
0.189
0.246
0.194
0.250
0.197
0.266
0.198
0.252
0.195
0.511
0.283
0.289
0.210
0.331
0.255
0.524
0.344
Tmfffi
24
0.310
0.223
0.316
0. 0.234
0.316
0.234
0.343
0, 0.246
0.323
0.229
0.531
0.293
0.335
0.231 0,402
0.281
0.548
0.335
96
0.358
0.244
0.386
0.271
0.258
0.407
0.278
0.602 0.319
0.392
0.272 0.452
0.302
0.603
0.368
192
0.384
0.258
0.416
0,280
0.389
0.270
0.452
0,315
0.394
0.266l
0.615
0.321
0.423
0.269
0.465
0.304
0.628
0.385
336
0.398
0.264
0.437
0.293
0.398
0.274
0.472
0.321
0.409
0.275
0.640
0.355
0.440
0,299
0,480
0.311
0.616
0.371
3
0.144
0.239
0.146
0.242
0.165
0.262
0.145
0.245
0,148
0.248
0.272
0.357
0.142
0.241
0.170
0.260
0.299
0.382
24
0.297
0.344
0.304
0.353
0.367 0.320
0.299
0.350
0.299
0.355
0.352
0.393
0.318
0.366
0.319
0.362
0.442 0.466
96
0.363
0.386
0.379
0.398
0.400
0,362
0.391
0.402
0
0,383
0.456
0.469
192
0.404
0.413
0.427
0.423
0.444 0.446
0.443
0.437
0.413
0.425
0.464
0.459
0.433
0.431
0.429
0.428
0.505 0.491
336
0.438
0.428
0.424
0.457 0.476
0.476
0.444
0.44
0.506
0.482
0.471
0,451
0.464
0.444
0.560
0.522
3
0.079
0.177
0.079
0.177
0
0.193
0.082
0.179
0.081
0.178
0.119
0,252
0.079
0.176 0.083
081
0.203
0.310
ETTh2
24
0.165
0.258
0.163
0,253
0.187
0.278
0.174
0.266
0.176
0. 0.264
0.210
0.301
0.180
0.271
0.167
0.261
0.318 0.393
96
0,285
0.339
0.289
0.340
0,304
0.355
0.302
0.355
0,300
0.348
0.340
0.379
0.328
0.376
0.290
0.349
0.378
0,417
192
0.356
0
0.356
0. 0.388
0.378
0.401
0.361
0.392
0.359
0.390
0.402
0.417
0.796
0.416
0.389
0.416
0.437 0.452
336
0.392
0419
0.395
0.417
0.417
0.428
0.405
0.429
0,404 0.434
0.451
0.454
0.438
0,430
0,484
0.482
0.493
0.486
3
0.057
0.150
0:058
0.151
0.062
0.161
0.069
0.175
0.056
0.149
0.067
0.168
0.057
0.151
0.062
0.156
0.227 0.315
E11m2 ET7ml
24
0.185
0.265
0.193
0.270
0.211
0.290
0.204
0. 0.284
0.196
0.277
0.201
0.282
0.209
0.282
0.214
0.288
0.466 0.446
96
0.290
0.339
0.287
0.339
0,305
0.354
0.314
0.355
0,293
0.342
0.324
0.370
0.319
0.355
0.308
0.358
0.471
0.445
192
0.325
0.362
0.327
0,366
0.377 0,346
0.331
0,373
0.335
0.369
0.393
0.413
0.374
0,390
0,341 0.373
0.566
0.498
336
0.357
0.384
0.385
0,381
0.397
0.373
0.396
0,364
0.389
0.371
0.399
0.405
0.414
0.372
0.388
0.594 0.523
3
0.041
0.117
0.042
0.119
0.044
0.127
0.043
0.122
0.042
0.120
0.051
0.143
0.042
0,120 0,044
0.125
0.120 0.234
24
0.094
0.190
0.095
0.192
0.102
0.200
0.097
0.195
0.093
0. 191
0.808
0.2.10
0.098
0.197 0.098
0. 198
0. 151 0.262
96
0.165
0.254
0.163
0.250
0.274
0.170
0.258
0.255
0.1
0.278
0.177
0.264 0.177
0.231 0.317
192
0.221
0.293
0.221
0,292
0,244
0,312
0.229
0,300
0,233
0.296
0.241
0.315
0.231
0.303
0,228
0.306
0.288 0.343
336
0.270
0.326
0:271
0.326
0,299
0.345
0.282
0.333
0,296
0.339
0.309
0.345
0.292
0,339
0.291
0.353
0.339 0.377
3
0,044
0.060
0.046
0.066
0.046
0.062
0.045
0,062
0.045
0.064
0.055
0.091
0.045
0.064
0.048
0.074
0.054
0.087
Wember
24
0.087
0.116
0.089
020
0.097
0.1
0.092
0.133
0.093
0.121
0.100
0.142
0.093
0.134
0.102
0.147
0.119
0.167
0.142
0.196
0.192
0.216
0.198
0.207
0.173
0.155
0.171
0.201
0.242
192
0.186
0.231
0.186
0.234
0,213
0.252
0.191
0,240
0,203
0.243
0.216
0.266
0.194
0.247
0,219
0.282
0.253
0.311
336
0.238
0.274
0.276
0.269
0.294
0.243
0,280
0,262
0.287
0.285
0.314
0,264
0.317
0.266
0.321
0.302
0.350
Table 3: Robustness of SSCNN performance. The results are obtained from five random seeds.
Dataset
ECL
Traffic
ETThl
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
0.057100000
0.149=0.000 0.241
0.189=00001
0.14410.004
0.239-0.003
24
0.093=0.001 0.187=0.002
0.310=0.004
0.223=0.003
0.297
0.344=0.003
96
0.128:10.002 0.22111.003
0.00810.002
0.244-0.002
0.363:10.002
0.386=0.004
192
0.00000.000 0.243=00001
0.384400.004
0.258=0.002
0.404
0.413=0.005
336
0.165=+0.003
0.261=0.003
0.39811.002
0.264=0.002
0.435:100003
0.428=0.004
Datasets
ETTh2
ETTml
ETTm2
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
0.079=10.000
0.177=0.001
0.057:10001
0.150=0.002
0.041
0.117=0.002
24
0.165110.001
0.25810.002
0.185:10.002
0.205-00.03
0.09411.001
0.100
0.285:10.001 0.339:1.001
0.29011.000
0.339=0.001
0.165:1.0001
0.254=0.002
192
0.356:10.004 0.385
0.323++0.003
0.362=0.004
0.201
0.293=0.002
336
0.392=#0.044 0.419=0.002
0.357=00003
0.384=0.001
0.270
0.326=0.003
Definition 1 (Conditional Correlation). Let X and be two real-valued, one-dimensional series, and Z re-dimensional control series. Denote X,, Y,. and Zi the 2th observations of each series, respectively. The conditional correlation is defined based on the residuals in X and Y that are unexplained by Z. The residuals are calculated as follows:
Rx, = X; - Wx Zi:
Ry. = Y,- WY Zi:
17

---

where wx and wy are obtained by minimizing the respective squared differences:
N
Wx = arg min w
(X; W. Z
N (Y, W ZZ,²
wy = arg min
The conditional correlation PXYIz z is then computed using these residuals:
Em Ry.
PXYIZ
Σ R E
Definition 2 (Conditional Auto-correlation) Extending the concept of conditional correlation to auto-correlation, we analyze a series' correlation with its own past values in the context of other variables. For time series Y and control variables Z. the residual is computed as:
R = Y w Zi,
where determined by minimizing the squared differences:
arg min
Σ (Y: - z.Z)²
The conditional auto-correlation is then defined as:
Σn 147 R₁ R4
DYIZ(T)
VEN R2 V Ei R2
Here, J represents the time lag. The measure (yy) quantifies the correlation between the time series and its r-lagged series, conditional on Z.
Both conditional correlation and conditional auto-correlation offer flexible and diverse instantiation possibilitics, relying on the definition of the conditional set. When this set is null, these measures simplify their standard forms of correlation and auto-correlation, respectively. This adaptability allows for a broad range analytical applications and interpretations. However, when utilizing conditional auto-correlation for time series data analysis, encounter significant challenge: the limited information of the controlling variable Z. In practice, identifying and evaluating the factors influencing our time series often requires external data sources, which are not always available. Instead, we use the method proposed by SSCNN to approximate the components that can be explained by factors with specific regularities. Figure 6 showcases the estimated components alongside the residuals obtained by incrementally controlling the long-term (c.g., resulting from resident population), seasonal (e.g., resulting from time of day). and short-term (e.g., resulting from occurrence of an event) components.
D.2 Temporal Regularity
Temporal regularity can be captured by the evolution of conditional auto-correlation, as depicted in Fig. (d))) indicating how the structured components impact the temporal regularity time series. We can observe that the original time series degenerates into sequence of nearly uncorrelated residuals upon progressively controlling for long-term, seasonal, and short-term components. This observation implies that these diverse components collectively elucidate the dynamics of time series, rendering any remaining correlations superfluous. Therefore, isolating these components is imperative to pinpoint and eliminate potential redundancy. Moreover, employing operator with an extensive receptive field, such as the MLP used Nie et al.] (2023). becomes unnecessary for processing decoupled data due to the minimal relevance of current observations distant lags.
In addition, conditional auto-correlations display variations with increasing lag intervals, likely attributed to cunulative noise. This fluctuation highlights the necessity for the selection mechanism, to discern subtle differences, moving beyond basic approaches like moving averages that uniformly treat all lags.
18

---

Residual
Mean
15
Standard Deviation
Long-term Component
Seasonal Component
Short-term Component
Figure 6: Disentangling structured components from the original time series data.
(a) (T)
(b) py (TT) Pre (T)
(c) py Ph (T) PR
(d) py pyl (T) Pra
22532
(c) PXY
(f) px Priy nq
PxY PAy RV
(h) pxY Prg. my
Figure 7: (a)))) The evolution conditional correlation between each pair of series when progressively controlling for three distinct types of factors. (h): evolution of conditional auto-correlation when progressively controlling for three distinct types factors.
D.3 Spatial Regularity
Spatial regularity is recognized as another vital information source for enhancing prediction accuracy and reliability in many datasets. However, an long-standing question in spatial regularity research persists: Why the channel-mixing strategy struggles to capture beneficial spatial regularities Han et al. (2023b); Nie et al. (202))
Our principal discovery is that spatial and temporal correlations frequently intersect, leading to redundant information. This overlap necessitates the identification of distinct spatial correlations that augment temporal correlations. We concentrate on conditional correlations, acquired by methodically controlling for long-term, seasonal, and short-term components. As Fig. (f)) illustrates, spatial correlations gradually wane as more temporal elements are regulated. Eventually, when all three factors are considered, the majority of series pairs exhibit significant relation, underscoring the criticality of discerning genuinely impactful correlations.
This phenomenon is intuitively understood. Temporal influences, such as seasonal trends, commonly affect multiple time series concurrently, fostering parallel evolutions and visible spatial correlations. Once these influences are adjusted for, the resulting spatial correlations wane, uncovering less correlated series. the correlations among these residual series that genuinely refine forecasting by supplying unique spatial insights. Thus, the key insight for spatial modeling is that conditional correlation instrumental in highlighting spatial variables that offer information complementary to temporal data.
19

---

260
15c
126
250
250
(a) SSCNN
(b) iTrrnnffrmeer
(c) PatchTST
(d) TimeMixer
(e) TimesNet
(1) Crossformer
Figure 8: Visualization of input-168-precee 16--prediet-96 results on the Traffic dataset.
0.59
Ora offruh
GroudShur
0.00
0.0-
0.7-
0.77
0.00
0.3
0.6
0.34
0.06
4M
(a) SSCNN
(b) iTransformer
(c) PatchTST
0.36-
0:00
3.22
8.55
0.06
0.00
(d) TimeMixer
(c) TimesNet
(f) Crossformer
Figure 9: Visualization of input-432-predict-199 192 results on the Weather dataset.
E Visualization of Prediction Results
To provide clear comparison among different models, we list supplementary prediction showcases of two representative datasets Fig. and Fig. 9 respectively, which are given by the following
models: SSCNN, iTTrassforrer (?), PStchTST (Nie et al. 2023). Crossformer (Zhang & Yan 2023
TimesNet (Wu et al. 2023), TimeMixer (Wang et al. (024). Among the various models, SSCNN predicts the most precise future series variations and exhibits superior performance
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The empirical results and analysis support the claims.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We analyze the limitations of the proposed method in a separate section, specifically pointing out its disadvantageous computational efficiency compared against baseline models.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
21

---

Answer: [NA]
Justification: No new theoretical results are necessary for this work, but we do include analysis in the supplemental material. The analysis primarily aims for casting light on the rationale of the proposed methodooogy, which may found to be somewhat unlike rigorous theoretical proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We detail the configurations of the model and the settings of the experimental environments with the code being made publicly available.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
22

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: Please find code in Sttp: The
datasets can be downloaded from https: https: /tbpss//drive... htoodvi..oooolllee.... com/ sarive/folders
which is maintained by existing study.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental details mentioned in the paper, and also found in the released code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We repeated the experiments 5 times with different settings of the random seed for each dataset, and reported the I-sigma errors.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
23

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We train our model with a single NVIDIA 2080Ti 11GB GPU,
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: This work conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We create an individual section to discuss the social impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The study poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
This paper properly cites the related datasets and codes used in our work.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
25

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "nFHJz7uCKV", "reviewer": "Reviewer_a5mv", "summary": "The paper approaches the problem of long term time series forecasting (LTSF) using a compositional technique to reduce the model size without compromising the quality of solution. The proposed technique is a transformer based architecture with a lower number of parameters, and delivers similar performance as state of the art models for LTSF.\n\nThe limitation of existing approaches, such as data patching, is that they fail to take into account the spatio-temporal dependencies, and end up with a blow up in the number of latent variables. This results only in a very small improvement even if the model size is increased substantially. The proposed technique in the paper is based on a inference step and an extrapolation step without any information loss.\n\nThe paper evaluates the proposed approach, called SSCNN, with seven datasets, which has a combination of regular and volatile patterns. The baseline and state of the art approaches compared against include iTransformer, TimeMixer, and PatchTST. SSCNN consistently achieves the best scores, with respect to MSE and MAE. The paper also conducts ablation studies to show that each new component in the architecture is vital to the performance.", "questions": "1. Are there any non neural network based time series forecasting models which make use of compositionality?\n2. Do any of the introduced layers (temporal, seasonal, short-term, etc) have similarities with any existing literature? What I mean is that,  is the novelty in getting the layers to work together, or, also in defining the individual layers?\n3. In order to better study the computational cost, can you share the total/average running time for each method for each dataset?\n4. Why is SSCNN missing Figure 3(a)?\n5. What is the value of T_{out} in Figure 2?\n6. What is the forward window size in Figure 3?", "limitations": "Yes, the paper has outlined the limitation of computational efficiency and provided some insight into how it can be improved in future work.", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "The work studies an important and hard problem in time series forecasting which is the problem of efficient and accurate long term forecasting. Compositional techniques have been successful in other areas of AI including reinforcement learning, planning, and finite state controller synthesis. So, it makes sense to apply similar ideas in the space of long term time series forecasting.", "weaknesses": "While the high level message is presented well, I found the details of the proposed method and experiments are hard to follow. A running example with the explanation of the new layers will be useful.\n\nThe main contribution with respect to results is somewhat hard to grasp and align with the theoretical claims of the paper. Overall, I think there is room for improvement in the presentation of experimental results. I found some missing details in the experimental section that include:\n\n1. Why is SSCNN missing Figure 3(a)?\n2. What is the value of T_{out} in Figure 2?\n3. What is the forward window size in Figure 3?\n\nIn figure 2, it would be useful to move some of the methods to the appendix, and keep only the critical ones  in the main body of the paper. Same is true for Figure 3. It is hard to go back and forth between figures 1, and figures2&3.\n\nMinor:\n\n1. I would suggest providing some more details about the experimental results in point 3 of the contributions (lines 80-82)\n2. Figure 3 is hard to read in print.\n3. Having only one legend for all the subplots (Figure 2(a)-(d) and Figure 3(a)-(d)) will better than repeating the legends in all subplots.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and constructive comments.\n\n---\n**W1:** We apologize if the current symbolic formulas present challenges to comprehension. To enhance understanding, we have tried to make the selection mechanisms formulated by Equations (3)-(7) more accessible with the example attention maps provided in Figure 1. To further illustrate the computational flow, we present a toy example focusing on the inference of long-term, seasonal, and short-term components, with the scaling operation and the selection mechanism being disabled for simplicity:\n\nGiven a historical sequence of observations: 1.1, 3.1, 1.0, 3.0, 0.9, 2.9:\n\nLong-term Component: Derived as 2, 2, 2, 2, 2, 2, resulting in the sequence of residuals: -0.9, 1.1, -1, 1, -1.1, 0.9. \n\nSeasonal Component: Computed as -1, 1, -1, 1, -1, 1, leading to the residual series: 0.1, 0.1, 0, 0, -0.1, -0.1. \n\nShort-term Component: Derived as 0, 0.1, 0.05, 0, -0.05, -0.1 with a window size of 2, obtaining the residual series: 0.1, 0, -0.05, 0, -0.05, 0. \n\n---\n**W2:** We apologize for any confusion caused by the perceived disconnect between the experimental results and the theoretical claims of the paper. The experimental results presented in Table 1 demonstrate SSCNN's improved efficacy compared to state-of-the-art methods, including the patching-based methods PatchTST and iTransformer. These results align with the theoretical claims made in Section 4.1, suggesting that SSCNN's decomposition-based approach either matches or exceeds the capabilities of patching-based methods. Additionally, Figure 2 illustrates the optimal parsimony of SSCNN, supporting the analysis in Section 4.2. Furthermore, Figure 3 highlights SSCNN's superior scalability with an increasing lookback window size. Unlike other baselines, SSCNN consistently benefits from more historical data points, regardless of the dataset's dynamic nature. This scalability is crucial for time series forecasting, as more data generally leads to more accurate predictions following our expectation.\n\n ---\n**W3:**  (1). Thank you for catching this error. The label \"HDformer\" in Figure 3(a) is indeed a typo and should be replaced with \"SSCNN.\" We will correct this in the revised version. \n\n(2). In Figure 2, for subfigures (a) and (b), $T_{in}$  is set to 96, while $T_{out}$  is represented by the x-axis. For subfigures (c) and (d), $T_{out}$  is set to 96, while $T_{in}$  is represented by the x-axis. We will clarify this information in the revised figure caption to improve understanding. \n\n(3). The forward window size in Figure 3 is indicated by $T_{out}$, which is noted in each subfigure. We will make sure this information is more clearly presented in the revised version for better clarity. \n\n---\n**W4:** We will revise the third contribution as follows in the final version: \nWe conducted comprehensive experiments to evaluate SSCNN across various dimensions. SSCNN demonstrates improved effectiveness, with increases ranging from 4% to 20%, and a reduction in model size ranging from 70% to 99% on average across all datasets, compared to state-of-the-art baselines.\n\n---\n**W5:** We will also address the other concerns you raise about the layout and formating in the final version. \n\n---\n**Q1:** The concept of compositionality in time series forecasting has roots in classical statistical methods, most notably STL [1]. Our method builds on this foundational idea by integrating it with deep learning techniques, enabling nonlinear decomposition and re-composition of the time series data. Additionally, we also extend STL to consider the short-term and spatial components. In this way, our method achieves a more nuanced and effective modeling of complex time series patterns. \n\n[1] Cleveland, Robert B., et al. \"STL: A seasonal-trend decomposition.\" J. off. Stat 6.1 (1990): 3-73.\n\n---\n**Q2:** The novelty of our approach lies in the reinvention of these layers that are inspired from the existing literature. As mentioned in the introduction and related work sections, some existing works have also explored the idea of modeling different heterogeneous components separately. These works typically employed techniques such as plain moving averages or MLPs. However, they often struggled to outperform advanced baselines like PatchTST across various benchmark datasets, as shown in Tables 1 and 2. Moreover, these approaches lacked scalability, requiring exponentially more parameters than SSCNN for large forward and backward window sizes, as demonstrated in Figures 2(a) and 2(c). Our method significantly advances these prior approaches by integrating customized selection mechanisms for the components of different behaviors within both backward and forward estimation processes. This extension enhances our model's capability and parsimony, resulting in superior performance.  \n\n---\n**Q3:** We have preliminarily reported the measurements of training and inference times for SSCNN compared to two representative baselines, PatchTST and iTransformer, on the Electricity and ETTh1 datasets. We will improve this evaluation study in the revision. Our preliminary findings indicate that iTransformer is the most efficient model among the three, requiring the least time for both training and inference. The relatively higher running time of SSCNN can be in part attributed to its sequential handling of the considered components. In contrast, iTransformer handles these components in parallel using a unified MLP, contributing to its efficiency advantage. We have mentioned this limitation of SSCNN in the section of \"Conclusions, Limitations and Broader Impacts\".\n\n| Models| ECL |  ETTh1 |\n|-|-|-|\n|iTransformer | 11min (training), 0.05s (inference) |  2min (training), 0.01s (inference) |\nPatchTST|95 min (training), 0.3s (inference)|10min (training), 0.09s (inference) \n|SSCNN|72 min (training), 0.2s(inference)|8min (training), 0.07s (inference) |\n\n---\n**Q4, Q5, Q6:** Please check the above response to W3.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RA1LHzsE2L", "reviewer": "Reviewer_UAfk", "summary": "Title: Parsimony or Capability? Decomposition delivers both in long term time series forecasting.\n\nLong term time series forecasting has been an important research problem which applies to different problem domains. This paper proposes a decomposition method which shows significant performance on the benchmarks with less parameters. This method been evaluated extensively on the various datasets and been competitive to existing models. With such approach models can be enhanced to adapt domain characteristics more effectively in various time series applications.", "questions": "1. Please rewrite the contributions to make them clearer (2nd and 3rd).\n2. Add descriptions in figure 2 and 3\n3. In section D) Implications of Decomposition on Spatial-Temporal Correlations, please correct the captions of figures in temporal and spatial recovery.", "limitations": "Authors have adequately addressed the limitations related to computational efficiency and capability of model.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. SSCNN reduces the parameter count substantially compared to traditional models, holding onto less than 1% of the parameters while still performing well across different datasets.\n2. The model captures complex data patterns effectively using fewer parameters, utilizing a structured component-based approach with a selection mechanism to improve prediction accuracy.\n3. SSCNN excels in time series forecasting, managing diverse data types and confirming its effectiveness through thorough experimentation.\n4. SSCNN improves plain feature decomposition by incorporating a selection mechanism. This allows the model to identify fine-grained dependencies at each time step, which is essential for enhancing the accuracy of the decomposed structured components and, consequently, the overall prediction accuracy.\n5. Extensive analysis has been performed to validate the method on existing benchmarks and compared with state-of-the-art methods.\n6. Supplementary materials are satisfactory and provide explanation about the dataset and the implementation.", "weaknesses": "1. Figures lack captions.\n2. Include some limitations of the model as well.\n3. second contribution and third one looks quite similar.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and constructive comments.\n\n---\n**W1:** We apologize for the lack of captions for Figure 2 and Figure 3. We also include the caption for Figure 1, as requested by Reviewer aS9R. The captions for these figures are shown below\n\nFigure 2: Examination of parameter scale and computation scale against the forward window size and the backward window size on the ECL dataset. (a) (c) For short forward window size or backward window size, SSCNN consumes more parameters than DLinear only. As the window size grows, the SSCNN scales up at a much slower inflation rate than DLinear. (b) (d) The consumption of FLOPS by SSCNN ranks in the middle place among SOTAs, less efficient than iTransformer and Autoformer yet more efficient than PatchTST and Crossformer. \n\nFigure 3: Impacts of backward window size. While the performance of other SOTA forecasters does not necessarily benefit from the increased lookback length, SSCNN improved performance on the enlarged lookback window. \n\nFigure1: (a) Embedding: Raw data points are independently mapped to a high-dimensional representation space using a shared 1x1 convolution operator across all variables and horizons. (b) In-sample Inference: The historical time series representations are processed by three temporal attention-based normalization (T-AttnNorm) layers with different instantiations of the attention map ($\\mathcal{I}$), followed by a spatial attention-based normalization (S-AttnNorm) layer. This process yields estimations of four structured components ($\\mu$) along with the residual ($R$). (c) Out-of-sample extrapolation: The decomposed components corresponding to each variable are individually extrapolated to future horizons using the matrix multiplication (MatMul) operator, with distinct instantiations of the attention map ($\\mathcal{E}$). (d) Component fusion: Finally, all four components, combined with the residuals, are input into a polynomial regression layer. \n\n ---\n**W2:** We apologize for not highlighting the limitations of the model prominently enough. Due to space constraints, we discussed the limitations in the last section together with broader impacts and conclusions. We will discuss the limitations in detail and put them into a new separate section in the revision\n\nBriefly, the identified limitations include: (a) While our approach effectively reduces model size, its computational complexity remains comparable to Transformer-based models, which is more demanding than linear models. This suggests potential for further optimization in terms of computational efficiency. (b) Currently, the combination of the proposed decomposition modules is determined with statistical insight into the data, which may not be optimal. Future work could explore methods for efficiently searching for the optimal combination of these modules. These points outline practical areas for improvement and future exploration. We will ensure these limitations are clearly presented in the final version of the paper. \n\n---\n**W3:** We apologize for the confusion between the second and third contributions. The second contribution relates to Section 4, where we theoretically explore the equivalence in capability between the patching operator and the decomposition operator. We also highlight the advantage of decomposition in terms of parameter efficiency, suggesting that decomposition can serve as a parameter-efficient alternative to patching, which is often considered essential in many state-of-the-art (SOTA) models. The third contribution pertains to the empirical evaluation of SSCNN against SOTA baselines, detailed in Section 5. \n\nTo clarify, we will revise the second contribution as follows: \n\n\"We conduct an in-depth theoretical comparison between decomposition and patching, analyzing both capability and parsimony. This analysis challenges the necessity of patching as an essential component in modern time series forecasting models, proposing decomposition as a more parameter-efficient alternative.\"\n\n---\n**Q1:** Please check the response to W3.\n\n---\n**Q2:** Please check the response to W1. \n\n---\n**Q3:** We will rectify this issue in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OCBTbuYeuc", "reviewer": "Reviewer_oTnm", "summary": "This study unveils a groundbreaking approach to time series forecasting, notable for its minimal parameter count. It stands as the first model to consistently outperform state-of-the-art (SOTA) techniques while remaining compact. Unlike prevalent methods such as PatchTST and iTransformer, which are powerful but cumbersome, and emerging methods like TimeMixer and SCNN, which are lightweight yet inadequate for complex tasks, this model achieves superior performance without the associated heft.", "questions": "If the authors can address the weak points, I would reconsider the score.", "limitations": "The authors have raised the limitation of the model concerning computational efficiency, along with potential solutions.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The model consistently delivers superior accuracy compared to state-of-the-art (SOTA) methods while maintaining a minimal model size. This accomplishment distinguishes it from other methods.\n\n2. The framework unifies the ability to capture various patterns in time series data, offering a streamlined and enhanced alternative to existing models built with MLPs or Transformers.\n\n3. The authors conduct extensive experiments, showcasing the model's strong performance compared to selected SOTA models, which are sufficiently representative of the latest advancements in the field.", "weaknesses": "1. There is a gap between the introduction and Section 3 regarding the decomposition of the time series into four components. The authors do not explain why these four components are sufficient. For longer sequences, is there a need for more components? Are there references that support this approach? This discussion should be included at the beginning of Section 3.\n\n2. Manually disabling the spatial component for certain datasets appears suboptimal. It would be more effective if the algorithm could automatically determine whether including the spatial component is beneficial for each dataset.\n\n3. The paper's formatting needs improvements. It seems the authors may have additional content to include. Although the figures in the methodology section are clear and informative, resizing and rearranging them could provide more space for adding valuable content to the main text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and constructive comments.\n\n---\n**W1:** Thank you for highlighting this important point. We apologize for not providing sufficient context on the four components used in our model. Disentangling these components has been shown to be effective for time series forecasting in numerous studies [1] [2] [3] [4]. Our approach builds on these established methods with a novel selection mechanism, which provably reduces estimation bias for the considered components. Our method advances the performance of decomposition-based approaches to the SOTA level, which can be achieved by only large-scale models before our work. The fact that our model achieves this with only four components suggests that, regardless of model complexity, the representation power of these large-scale models does not exceed the scope covered by these four components. As you pointed out, this raises a critical question for future research: are there additional components that could be modeled to further improve forecasting accuracy, especially for longer sequences? Currently, this question remains open and unanswered. We will discuss this issue and provide relevant references at the beginning of Section 3 in the revised version, ensuring a more comprehensive and informative explanation. \n\n[1] Cleveland, Robert B., et al. \"STL: A seasonal-trend decomposition.\" J. off. Stat 6.1 (1990): 3-73. \n\n[2] Wen, Qingsong, et al. \"RobustSTL: A robust seasonal-trend decomposition algorithm for long time series.\" Proceedings of the AAAI conference on artificial intelligence. Vol. 33. No. 01. 2019. \n\n[3] Deng, Jinliang, et al. \"Disentangling Structured Components: Towards Adaptive, Interpretable and Scalable Time Series Forecasting.\" IEEE Transactions on Knowledge and Data Engineering (2024). \n\n[4] Wang, Shiyu, et al. \"TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting.\" The Twelfth International Conference on Learning Representations (2024). \n\n---\n**W2:**  Thank you for your insightful suggestion. Our primary aim is to demonstrate the pioneering predictive capability of a purely decomposition-based model with an exceptionally low parameter count—less than 1% of what large-scale models require. The current architecture leverages the statistical characteristics of the dataset of interest, ensuring that the design is not arbitrary. In particular, our analysis of correlation, autocorrelation, conditional correlation, and conditional autocorrelation—visualized in Figure 7—guided our decisions regarding the architecture's configuration and critical hyper-parameters, such as cycle length and the connection between the variables. While these decisions are based on our judgment, they can be automated through a Python script, a straightforward implementation. Despite its simplicity, this method has already resulted in a sensible architecture that outperforms baselines, thereby validating our contributions. We acknowledge that the current method may not achieve the absolute upper limit of performance possible with all potential decomposition-based model configurations. This limitation is discussed in the \"Conclusions, Limitations, and Social Impacts\" section. We agree that learning-based methods, such as neural architecture search (NAS), offer a promising avenue for further improvement and optimization. This remains an exciting area for future work. \n\n---\n**W3:**  Thank you for your valuable suggestion. We will enhance the layout and formatting in the final version of the paper. This will involve resizing and rearranging the figures in the methodology section to ensure they are clear and informative while providing additional space for valuable content in the main text. Our goal is to make the main body of the paper more self-contained and comprehensive.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QTBEilvyCO", "reviewer": "Reviewer_aFLy", "summary": "This paper addresses long-term time series forecasting and critiques the reliance on complex models with extensive parameters. It proposes a decomposition method specifically designed for time series dynamics, achieving better forecasting performance across various datasets. Remarkably, the new model uses over 99% fewer parameters than other methods, highlighting the efficiency of domain-specific approaches. This research calls for a move away from complexity in LTSF, showcasing the effectiveness of focused decomposition techniques rather than relying on large-scale models.", "questions": "1.\tThe methodology would benefit from additional explanation regarding the structure and rationale of the Polynomial Regression layer depicted in Figure 1. If this layer represents a standard approach, including references would enhance clarity.\n\n2.\tClarifying the decision to exclude an attention mechanism from the long-term component, despite its presence in other components like seasonal, short-term, and spatial, would strengthen the methodological coherence and aid reader comprehension.\n\n3.\tFigure 1 requires clarification on several elements. Specifically, the purpose of the 4x4 blocks and addressing inconsistent text formatting (e.g., $\\mathcal{E}$) compared to the main text would improve comprehensibility.", "limitations": "Whether this model can be applied to other types of time series data, e.g. trajectory.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe paper is praiseworthy for its intuitive approach. It tackles a significant problem by proposing a method that matches or surpasses current state-of-the-art models like PatchTST while using a smaller model footprint. The experimental results strongly validate this approach.\n\n2.\tThe model consistently performs well under various experimental conditions, including different input window sizes and hyperparameter settings. Statistical tests demonstrate the reliability of the results across multiple initializations, strengthening the study's credibility.\n\n3.\tThe authors provide a thorough comparison between decomposition and patching in terms of effectiveness and simplicity, demonstrating the superior benefits of decomposition over patching.", "weaknesses": "1.\tThe clarity of the methodology could be improved with further elaboration.\n2.  The evaluation could be strengthened by including comparisons with LLM-based models, such as:\n\n [1] Jin, Ming, et al. \"Time-LLM: Time Series Forecasting by Reprogramming Large Language Models.\" The Twelfth International Conference on Learning Representations.\n\n[2] Bian, Yuxuan, et al. \"Multi-patch prediction: Adapting llms for time series representation learning.\" arXiv preprint arXiv:2402.04852 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and constructive comments.\n\n---\n**W1:** We have addressed the concerns raised in your questions. Please kindly refer to the response to Q1 and Q2 below. \n\n---\n**W2:**  We have conducted a comparative analysis between SSCNN and MTSF models based on LLM [1] [2]. Due to the time constraint, we directly copy the results obtained by baselines from an ICML 2024 publication [1]. The results, which are reported at the end of the reply box, demonstrate SSCNN's advantages in both performance and efficiency. This analysis strengthens our claim that SSCNN is a highly competitive model in the field of time series forecasting. We will include this comparative analysis in the final version of the paper to provide a more comprehensive evaluation of SSCNN's capabilities. \n\n[1] Bian, Yuxuan, et al. \"Multi-patch prediction: Adapting llms for time series representation learning.\" arXiv preprint arXiv:2402.04852 (2024). \n\n[2] Jin, Ming, et al. \"Time-LLM: Time Series Forecasting by Reprogramming Large Language Models.\" The Twelfth International Conference on Learning Representations. \n\n| Electricity | 96 | 192 | 336 | 720 |\n|-----------|-----------|-----------|-------------|-------------|\n| PatchTST (Transformer-based ICLR 2023) | 0.129 | 0.147 | 0.167 | 0.202 |\n| ALLM4TS (LLM-based, ICML 2024) | 0.127 | 0.145 | 0.163 | 0.206 \n|Time-LLM (LLM-based, ICLR 2024) | 0.140 | 0.151 | 0.171 | 0.210 |\n|SSCNN (Ours) |   **0.126** | **0.145** | **0.161** | **0.191** | \n\n\n| Traffic | 96 | 192 | 336 | 720 |\n|-----------|-----------|-----------|-------------|-------------|\n| PatchTST (Transformer-based ICLR 2023) | 0.367 | 0.385 | 0.399 | 0.434 \n| ALLM4TS (LLM-based, ICML 2024) | 0.372 | 0.383 | 0.396 | 0.433 \n|Time-LLM (LLM-based, ICLR 2024) | 0.383 | 0.398 | 0.407 | 0.434 |\n|SSCNN (Ours) |   **0.352** | **0.380** | **0.390** | **0.423** |\n\n| ETTh1 | 96 | 192 | 336 | 720 |\n|-----------|-----------|-----------|-------------|-------------|\n| PatchTST (Transformer-based ICLR 2023) | 0.379 | 0.413 | 0.422 | 0.447 |\n| ALLM4TS (LLM-based, ICML 2024) | 0.380 | **0.396** | 0.413 | 0.461 |\n|Time-LLM (LLM-based, ICLR 2024) | 0.399 | 0.433 | 0.469 | 0.473 |\n|SSCNN (Ours) |   **0.361** | 0.401 | **0.410** | **0.424** |\n \n---\n**Q1:** The Polynomial Regression layer depicted in Figure 1 is inspired by the work of [3], where we extend the original module to include both additive and multiplicative relations. This extension allows us to model more complex interactions between the decomposed components. We will provide a more detailed explanation of the Polynomial Regression layer in the methodology section, including its structure and rationale. Additionally, we will cite relevant references to enhance the clarity and contextual understanding of this approach.  \n\n[3] Deng, Jinliang, et al. \"Disentangling Structured Components: Towards Adaptive, Interpretable and Scalable Time Series Forecasting.\" IEEE Transactions on Knowledge and Data Engineering (2024). \n\n---\n**Q2:** Thank you for highlighting this point. The decision to exclude an attention mechanism from the long-term component was made after careful consideration. Although it can be incorporated, as we have done with the seasonal, short-term, and spatial components, we found that it brought little to no gain in forecasting accuracy across the datasets we analyzed. To clarify, the attention mechanism is beneficial when a component exhibits a significantly evolving distribution over time. It reduces estimation bias by assigning higher weights to data points with higher correlation. However, for the long-term component in our case, the distribution tends to be stable throughout the input period. In such scenarios, the attention mechanism does not contribute additional value, rendering it redundant and unnecessary. We will include this explanation in the main text of the final version to enhance the coherence of our methodology and improve reader comprehension. \n\n---\n**Q3:** The 4x4 blocks in Figure 1 are used to exemplify the selection maps $\\mathcal{I}^*$ and $\\mathcal{E}^*$ as defined in the main text, with $T_{in}$  and $T_{out}$  both instantiated as 4. To enhance clarity, we will explicitly label these blocks in the figure to indicate that they are examples of the associated selection maps. Additionally, we will address the inconsistent text formatting, such as the use of E, to ensure consistency between the figure and the main text.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "igVDERDp1U", "reviewer": "Reviewer_aS9R", "summary": "This paper identifies data decomposition as a core bottleneck in time series forecasting and proposes a novel model named SSCNN, a decomposition-based model innovatively enhanced with a selection mechanism. SSCNN is specifically designed to adeptly capture complex regularities in data while maintaining a minimal parameter scale. This paper also provides an in-depth comparison between decomposition and patching, examining both capability and parsimony.  Comprehensive experiments show the superior performance of SSCNN.", "questions": "See Weakness.", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Strong Points:\n\n1. The insight of this paper is attractive and compelling. One of the most crucial characteristics of time series is that they can be viewed as composed of components with different natures, e.g., season, trend, and residual. However, this characteristic has been rarely utilized in related works, or it has been implemented in trivial ways. This paper identifies data decomposition as a core bottleneck in time series forecasting and proves its effectiveness. By decomposing complex data into more learnable components, SSCNN achieves state-of-the-art performance with a minimal number of parameters.\n\n2. The writing of this paper is very clear. I can easily follow the author's logic and understand their points.\n\n3. The experimental results are extensive, including overall performance results, ablation studies of each component, hyperparameter experiments, etc., which validate the effectiveness of SSCNN.\n\n4. The code is reproducible and well documented. I have successfully replicated the authors' results.\n\n5. The authors also provide an in-depth comparative analysis and experimental results between patching and decomposition, which help readers understand the advantages of SSCNN’s insights.\n\nThis paper emphasizes the importance of decomposition in long-term time series forecasting, addressing the analytical gap in feature decomposition for the first time and providing insights into its rationale for capability and parsimony compared to patching.", "weaknesses": "I have some minor questions and suggestions. If the author addresses the following points, I will increase my score.\n\nWeak Points:\n\nExperimental Setting: Most works using the Time-Series-Library repository predict up to 720 steps, yet your results do not include this prediction horizon. It would be beneficial to explain why 720-step predictions were not included.\n\nFigures: I suggest the authors add more explanatory information to Figure 1 to help readers grasp the main architecture of SSCNN from the figure and its caption alone. Moreover, some font styles (italic) in Figure 1 seem different from the character styles in the main text. I recommend unifying the styles.\n\nMinor Issues: The operator $\\lfloor \\cdot \\rfloor$ is used in the paper but not explained. In Figure 3(a), if I understand correctly, “HDformer” should be replaced by “SSCNN.”\n\nFigures: The text size of the legends in the figures is too small, making them difficult to read. Adjusting the text size to be consistent with the main text would enhance the readability of the figures and improve the overall presentation quality of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and constructive comments.\n\n---\n**W1:** We acknowledge the importance of including 720-step predictions. In our experiments, we observed that the ranking of models for 720 output steps differed only slightly from the ranking for 336 output steps. This suggests that the models maintain consistent performance over long-term horizons. To support this observation, we conducted additional experiments with SSCNN and selected baselines on the ECL, Traffic, and ETTh1 datasets for output steps within 96, 192, 336 and 720. \n\n| Electricity | 96 | 192 | 336 | 720 |\n|-----------|-----------|-----------|-------------|-------------|\n| PatchTST (Transformer-based ICLR 2023) | 0.129 | 0.147 | 0.167 | 0.202 |\n| FITS (lightweight, ICLR 2024)  |       0.138| 0.152 | 0.166 | 0.205 |\n| SparseTSF (lightweight, ICML 2024)| 0.138 | 0.146 | 0.164 | 0.203 |\n| ALLM4TS (LLM-based, ICML 2024) | 0.127 | 0.145 | 0.163 | 0.206 |\n|Time-LLM (LLM-based, ICLR 2024) | 0.140 | 0.151 | 0.171 | 0.210 |\n|SSCNN (Ours) |   **0.126** | **0.145** | **0.161** | **0.191** | \n\n\n| Traffic | 96 | 192 | 336 | 720 |\n|-----------|-----------|-----------|-------------|-------------|\n| PatchTST (Transformer-based ICLR 2023) | 0.367 | 0.385 | 0.399 | 0.434 \n| ALLM4TS (LLM-based, ICML 2024) | 0.372 | 0.383 | 0.396 | 0.433 \n|Time-LLM (LLM-based, ICLR 2024) | 0.383 | 0.398 | 0.407 | 0.434 |\n|FITS (lightweight, ICLR 2024)    |    0.401| 0.407 | 0.420 | 0.456 |\n|SparseTSF (lightweight, ICML 2024) | 0.382 | 0.388 | 0.402 | 0.445 |\n|SSCNN (Ours) |   **0.352** | **0.380** | **0.390** | **0.423** |\n\n| ETTh1 | 96 | 192 | 336 | 720 |\n|-----------|-----------|-----------|-------------|-------------|\n| PatchTST (Transformer-based ICLR 2023) | 0.379 | 0.413 | 0.422 | 0.447 |\n| ALLM4TS (LLM-based, ICML 2024) | 0.380 | **0.396** | 0.413 | 0.461 |\n|Time-LLM (LLM-based, ICLR 2024) | 0.399 | 0.433 | 0.469 | 0.473 |\nFITS (lightweight, ICLR 2024)    |     0.375| 0.408 | 0.429 | 0.427 |\n|SparseTSF (lightweight, ICML 2024) | **0.359** | 0.397 | **0.404** | **0.417** |\n|SSCNN (Ours) |   **0.361** | **0.401** | **0.410** | **0.424** |\n\n---\n**W2:**  To provide a clearer understanding of the SSCNN architecture, we will enhance Figure 1 with more explanatory information. Here's an overview: (a) Embedding: Raw data points are independently mapped to a high-dimensional representation space using a shared 1x1 convolution operator across all variables and horizons. (b) In-sample Inference: The historical time series representations are processed by three temporal attention-based normalization (T-AttnNorm) layers with different instantiations of the attention map ($\\mathcal{I}$), followed by a spatial attention-based normalization (S-AttnNorm) layer. This process yields estimations of four structured components ($\\mu$) along with the residual ($R$). (c) Out-of-sample extrapolation: The decomposed components corresponding to each variable are individually extrapolated to future horizons using the matrix multiplication (MatMul) operator, with distinct instantiations of the attention map ($\\mathcal{E}$). (d) Component fusion: Finally, all four components, combined with the residuals, are input into a polynomial regression layer to capture their complex interrelations. \n\nAdditionally, we will ensure that the font styles in Figure 1 are consistent with the main text in the final version.  \n\n---\n**W3:** We appreciate your attention to these issues. To address them: \n\n(a) We will include a clear definition and explanation of the operator $\\lfloor . \\rfloor$, which denotes the floor function, in the paper. \n\n(b) Regarding Figure 3(a), we will correct the labeling error by replacing “HDformer” with “SSCNN”. \n\n(c) We acknowledge the issue with the text size of the legends in the figures. We will adjust the text size to ensure consistency with the main text, thereby enhancing the readability and overall presentation quality of the paper. \n\nThank you for your helpful feedback.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZST3gA2RiF", "reviewer": "Reviewer_p9WT", "summary": "This paper proposes  a Selective Structured Components-based Neural Network for Long-term Time Series Forecasting", "questions": "See weakness", "limitations": "No problem", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper demonstrates originality by addressing a crucial limitation in existing SOTA methods, maintains high quality through thorough experimentation and clear presentation, offers significant advancements to the field of time series forecasting, and ensures clarity that aids in understanding and reproduction of the work.\n\n2. The motivation of this paper is intuitive and compelling. Given the large model sizes of current SOTA methods like PatchTST, the idea of using a smaller model to achieve comparable or better performance is highly attractive. \n\n3. The experiments are thorough, and the proposed method achieves state-of-the-art performance. The effectiveness of each sub-module is demonstrated through detailed ablation studies. \n\n4. The code is open source and reproducible, with a straightforward and clear usage process.", "weaknesses": "1. In the experiment section, it is noted that most papers use the ETT dataset for ablation studies, likely due to its smaller size, which allows for quicker results. However, you chose the ECL and Traffic datasets instead of ETT, which is a more comprehensive and reliable approach. While this choice is commendable, there is no explanation provided for not using the ETT dataset. \n\n2.It would be more informative to report the model size directly in Table 1. Including the model size would provide a clearer comparison with other SOTA methods and highlight the efficiency of your proposed model. \n\n3.Baselines: Some MTSF models based on LLM have been widely applied [1]. If the authors can demonstrate that SSCNN has advantages in both performance and efficiency, this paper will be more convincing.\n\n4.Some extremely lightweight models have also been proven to have satisfactory performance [2] . Compared to these methods, what are the main advantages of SSCNN? \n\n[1]One Fits All:Power General Time Series Analysis by Pretrained LM\n\n[2]FITS: Modeling Time Series with 10k Parameters", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and constructive comments.\n\n---\n**W1:**  We chose the ECL and Traffic datasets for our experiments primarily because they present more challenging tasks due to their larger data scale. Specifically, these datasets include a mass of variables and cover an extensive data collection period. This increased data scale results in a less biased estimate of the impact of each module, providing a more comprehensive evaluation of our model's performance. To address the concern regarding the exclusion of the ETT dataset, we have conducted an additional ablation study using the ETTh1 dataset. The results from this study are provided below for your reference. This inclusion further validates the robustness and generalizability of our approach across different dataset scales and complexities. We will improve this study in the revision.\n\n\n|SSCNN|w/o lta |  w/o lt | w/o sta | w/o st | w/o sea | w/o se | w si |\n|---|----|---|---|---|---|---|---|\n|0.363| 0.364 | 0.379 |  0.364 |  0.368 |  0.364 |  0.415|  0.378 |\n\n\n---\n**W2:** We will include the model size in Table 1 in the final version of the paper.  \n\n---\n**W3:** We have conducted a comparative analysis between SSCNN and MTSF models based on LLM. Due to the time constraint, we directly copy the results obtained by baselines from an ICML 2024 publication [1]. The results, which are reported at the end of the reply box, demonstrate SSCNN's advantages in both performance and efficiency. This analysis strengthens our claim that SSCNN is a highly competitive model in the field of time series forecasting. We will include this comparative analysis in the final version of the paper to provide a more comprehensive evaluation of SSCNN's capabilities. \n\n---\n**W4:** We appreciate your concern, which prompts us to emphasize the key contributions of SSCNN that might otherwise be underestimated. As discussed in the last paragraph of Section 2, many lightweight models achieve a reduction in parameters, however, at the cost of sacrificing prediction accuracy. In contrast, SSCNN manages to maintain strong competitive performance with comparably reduced parameters. To substantiate this claim, we offer an additional comparison between SSCNN and some representative lightweight models, including FITS, highlighting the balance SSCNN strikes between model complexity and prediction accuracy. Due to the time constraint, we directly borrow the results obtained by baselines from an ICML 2024 publication [2]. This comparison will be included in the supplementary materials of the final version, showcasing SSCNN's advantages in terms of both efficiency and effectiveness in time series forecasting. \n\n[1] Bian, Yuxuan, et al. \"Multi-patch prediction: Adapting llms for time series representation learning.\" arXiv preprint arXiv:2402.04852 (2024). \n\n[2] Lin, Shengsheng, et al. \"SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters.\" arXiv preprint arXiv:2405.00946 (2024). \n\n\n| Electricity | 96 | 192 | 336 | 720 |\n|-----------|-----------|-----------|-------------|-------------|\n| PatchTST (Transformer-based ICLR 2023) | 0.129 | 0.147 | 0.167 | 0.202 |\n| FITS (lightweight, ICLR 2024)  |       0.138| 0.152 | 0.166 | 0.205 |\n| SparseTSF (lightweight, ICML 2024)| 0.138 | 0.146 | 0.164 | 0.203 |\n| ALLM4TS (LLM-based, ICML 2024) | 0.127 | 0.145 | 0.163 | 0.206 |\n|Time-LLM (LLM-based, ICLR 2024) | 0.140 | 0.151 | 0.171 | 0.210 |\n|SSCNN (Ours) |   **0.126** | **0.145** | **0.161** | **0.191** | \n\n\n| Traffic | 96 | 192 | 336 | 720 |\n|-----------|-----------|-----------|-------------|-------------|\n| PatchTST (Transformer-based ICLR 2023) | 0.367 | 0.385 | 0.399 | 0.434 \n| ALLM4TS (LLM-based, ICML 2024) | 0.372 | 0.383 | 0.396 | 0.433 \n|Time-LLM (LLM-based, ICLR 2024) | 0.383 | 0.398 | 0.407 | 0.434 |\n|FITS (lightweight, ICLR 2024)    |    0.401| 0.407 | 0.420 | 0.456 |\n|SparseTSF (lightweight, ICML 2024) | 0.382 | 0.388 | 0.402 | 0.445 |\n|SSCNN (Ours) |   **0.352** | **0.380** | **0.390** | **0.423** |\n\n| ETTh1 | 96 | 192 | 336 | 720 |\n|-----------|-----------|-----------|-------------|-------------|\n| PatchTST (Transformer-based ICLR 2023) | 0.379 | 0.413 | 0.422 | 0.447 |\n| ALLM4TS (LLM-based, ICML 2024) | 0.380 | **0.396** | 0.413 | 0.461 |\n|Time-LLM (LLM-based, ICLR 2024) | 0.399 | 0.433 | 0.469 | 0.473 |\nFITS (lightweight, ICLR 2024)    |     0.375| 0.408 | 0.429 | 0.427 |\n|SparseTSF (lightweight, ICML 2024) | **0.359** | 0.397 | **0.404** | **0.417** |\n|SSCNN (Ours) |   **0.361** | **0.401** | **0.410** | **0.424** |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We are grateful for the detailed and constructive feedback provided by the reviewers. The positive reception of our work, particularly the recognition of its innovative approach and significant contributions to the field of time series forecasting, is highly encouraging. Below, we summarize the key strengths of our paper as highlighted by the reviewers:

1. **Importance and Relevance** (**Reviewer a5mv**): Acknowledged the significance of tackling efficient and accurate long-term time series forecasting.

2. **Innovative Approach and Original Contributions** (**Reviewers a5mv, UAfk, p9WT, oTnm, aS9R**): Highlighted the innovative use of decomposition with a selection mechanism. The reviewers appreciated the originality of the approach in feature decomposition, effectively capturing complex data patterns with a minimal model size. The work addresses a critical gap in SOTA methods, demonstrating the potential for smaller models to match or surpass existing methods.

3. **Lightweight and Strong Performance** (**Reviewers UAfk, oTnm, aFLy, p9WT**): Noted the substantial reduction in parameter count while maintaining or surpassing the performance of SOTA models. The reviewers appreciated the model's capability to handle diverse datasets and achieve state-of-the-art accuracy.

4. **Comprehensive Evaluation** (**Reviewers UAfk, aS9R, aFLy, p9WT**): Praised the extensive and thorough experiments, including overall performance results, ablation studies, and comparisons with other SOTA methods. The reviewers also noted the consistent performance across different experimental conditions and the clear documentation of results.

5. **Clarity and Quality** (**Reviewers aS9R, UAfk, p9WT**): Commended the clarity of the writing, logical flow, and the comprehensibility of the paper. Reviewers appreciated the supplementary materials and clear usage instructions for the open-source code, which facilitated reproducibility.

We have carefully addressed each of the reviewers' concerns and have made corresponding revisions and clarifications throughout the manuscript. We believe these changes enhance the clarity, coherence, and overall quality of the paper. We invite the reviewers to refer to the specific reply box associated with their review for detailed responses and updates. Thank you for your thoughtful consideration, and we look forward to your feedback on the responses.

Meta-review: The paper proposes a novel decomposition-based approach for long-term time-series forecasting, which significantly reduces model complexity while maintaining or surpassing the performance of state-of-the-art methods. The model is particularly effective in capturing complex data patterns with minimal parameters, offering a robust and scalable solution for diverse datasets.

**Strengths**
- Parameter Efficiency: The model achieves superior performance using over 99% fewer parameters than competing methods, addressing a critical challenge in long-term time-series forecasting problems that generally require more parameters to maintain history and to avoid catastrophic forgetting.
- Enhanced Interpretability: The decomposition approach not only improves accuracy but also enhances model interpretability, making the results more actionable in practical applications, supporting confidence in its value in real-world scenarios (Reviewers UAfk and aFLy).
- Comprehensive Empirical Validation: The paper presents extensive empirical evaluations across multiple datasets, showcasing the generalizability and effectiveness of the proposed method. Reviewer UAfk specifically praised the thoroughness of the experiments and the clear demonstration of the model’s superiority over existing SotA methods.
- Clarity and Reproducibility: The writing is generally clear, and the code is well-documented and reproducible, facilitating future research and application.

**Areas Needing (1st, 2nd) or Recommended for (3rd) Improvement (Weaknesses)**
- Methodological Clarity: Aspects of the methodological decisions, particularly the choice of decomposition techniques, lack clarity (Reviewer aFLy).  
--Providing more detailed justifications and discussions on these choices would improve the paper’s overall robustness in the claims of their algorithm's capabilities and results.
- Presentation and Formatting Issues: (Reviewer aS9R) highlighted minor presentation issues that detract from the paper’s readability. While these do not undermine the scientific contribution, they should be addressed to ensure the paper’s professionalism and clarity.
- Limited Comparison with LLM-based Models: The paper would benefit from a more detailed comparison with large language model (LLM)-based approaches, which are increasingly relevant in the forecasting domain. Addressing this would strengthen the paper’s position within the broader research landscape. (Reviewer p9WT)

The paper offers a significant contribution to long-term time-series forecasting through its innovative approach, sound experimental design, and comprehensive empirical validation. 

The identified issues 1 and 2 are important and easily addressable. *If* this paper is accepted, these issues must be corrected for the camera-ready version. Improve the methodological clarity and aim to improve presentation and formatting issues. Additionally, comparisons with LLM-based models are suggested for future work but are not required nor expected for the current work.

---

**[Update: 9-20-24]** After manually reviewing the paper in entirety as well as all reviews, rebuttals, and discussions a week ago, I believe that this work shows progress in multiple key areas time-series forecasting, which is a large, growing domain that will continue to grow over time. Namely this progress is demonstrated in (1) better forecasting performance over multiple time horizons on (2) several, diverse problem settings against (3) state-of-the-art and/or reasonably comparable baselines, with (4) order of magnitudes fewer parameters than comparable models due to a focus on decomposition.

The Authors proactively list limitations in the time taken for training and inference versus certain alternative approaches and provide sound logical reasoning as to why this is the case. This along with the inclusion of other limitations helps the research community in identifying where they should or should not focus their efforts.

As a result of the strong generalizable results in a large problem domain due to an innovative approach that deviates from the norm in a refreshing way, I recommend this paper for acceptance and spotlight.
Decision: Accept (spotlight)