Ada-MSHyper: Adaptive Multi-Scale Hypergraph Transformer for Time Series Forecasting
Authors: Zongjiang Shang
Ling Chen
Binqing Wu
Dongliang Cui
Venue: NeurIPS 2024 poster (2024)
Keywords: Time series forecasting
transformer
multi-scale modeling
hypergraph neural network
hypergraph learning
Primary area: deep_learning_architectures
TL;DR: We propose an adaptive multi-scale hypergraph  transformer for time series forecasting.

Abstract: Although transformer-based methods have achieved great success in multi-scale temporal pattern interaction modeling, two key challenges limit their further development: (1) Individual time points contain less semantic information, and leveraging attention to model pair-wise interactions may cause the information utilization bottleneck. (2) Multiple inherent temporal variations (e.g., rising, falling, and fluctuating) entangled in temporal patterns. To this end, we propose Adaptive Multi-Scale Hypergraph Transformer (Ada-MSHyper) for time series forecasting. Specifically, an adaptive hypergraph learning module is designed to provide foundations for modeling group-wise interactions, then a multi-scale interaction module is introduced to promote more comprehensive pattern interactions at different scales. In addition, a node and hyperedge constraint mechanism is introduced to cluster nodes with similar semantic information and differentiate the temporal variations within each scales. Extensive experiments on 11 real-world datasets demonstrate that Ada-MSHyper achieves state-of-the-art performance, reducing prediction errors by an average of 4.56%, 10.38%, and 4.97% in MSE for long-range, short-range, and ultra-long-range time series forecasting, respectively.  Code is available at https://github.com/shangzongjiang/Ada-MSHyper.

Ada-MSHyper: Adaptive Multi-Scale Hypergraph Transformer for Time Series Forecasting
Zongjiang Shang, Ling Chen," Binqing Wu, Dongliang Cui
State Key Laboratory of Blockchain and Data Security College of Computer Science and Technology Zhejiang University
{zongjiange Langgjang,,,n,, lingchen, bingingau, runnercdl)@cs. zju edu..du.cco
Abstract
Although transformer- based methods have achieved great success in multi-scale temporal pattern interaction modeling, two key challenges limit their further de- velopment: (1) Individual time points contain less semantic information, and leveraging attention model pair-wise interactions may cause the information utilization bottleneck. (2) Multiple inherent temporal variations (e.g., rising, falling, and fluctuating) entangled in temporal patterns. To this end. propose Adaptive Multi-Scale Hypergraph Transformer (Ada-MSHyper) for time series forecast- ing. Specifically, an adaptive hypergraph learning module is designed to provide foundations for modeling group-wise interactions, then a multi-scale interaction module introduced to promote more comprehensive pattern interactions differ- ent scales. In addition, node and hyperedge constraint mechanism is introduced cluster nodes with similar semantic information and differentiate the temporal variations within each scales. Extensive experiments on 1 real-world datasets demonstrate that Ada-MSHyper achieves state-of-the-an performance, reducing prediction errors by average of 4.56%, 10.38%, and 4.97% in MSE for long- range, short-range, and ttra--long-rane time series forecasting, respectively. Code is available at httpss//githhbb http::///ghbbbb.omm//aggggggggg/ cowm//hanggoogiingg/ddddSSiii
Introduction
Time series forecasting has demonstrated its wide applications across many fields [30 37
c.g.,
energy consumption planning, traffic and economics prediction, and disease propagation forecasting. In these real-world applications, the observed time series often demonstrate complex and diverse temporal patterns at different scales[6] 28] For example, due to periodic human activities, traffic occupation and electricity consumption show clear daily patterns (e.g., afternoon or evening), weekly patterns (e.g., weekday or weekend). and even monthly patterns (e.g., summer or winter).
Recently, deep models have achieved great success in time series forecasting. To tackle intricate temporal patterns and their interactions at different scales, numerous foundational backbones have emerged, including recurrent neural networks (RNNs) [7] 9 10] convolutional neural networks (CNNs) [1 graph neural networks (GNNs) 4 g and transformers 200 39). Particularly, due the capabilities of depicting pair-wise interactions and extracting multi-scale representations in sequences, transformers are widely used in time series forecasting. However, some recent studies show that simple multi-scale MLP |T
35 or naïve series decomposition methods [3] 15] can
outperform monsformer-bbsed methods on various benchmarks. We argue the challenges that limit the effectiveness of transformers time series forecasting are as follows.
"Corresponding author: Ling Chen.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

The first one is semantic information sparsity Different from natural language processing (NLP) and computer vision (CV), individual time point in time series contains less semantic information S 29 Compared to pair-wise interactions, group- wise interactions among time points with similar semantic information (e-g., neighboring time points or distant but strongly correlated time points) are more emphasized in time series forecasting, To address the problem of semantic information sparsity, some recent works employ patch-based approaches 23) and hypergraph structures [26] to enhance locality and capture group-wise interactions. However, simple partitioning of patches and predefined hypergraph structures may introduce a large amount of noise and be hard to discover implicit interactions.
The second one is temporal variations entanglement. Due to the complexity and non-stationary of real-world time series, the temporal patterns of observed time series often contain large number of inherent variations (e.g,, rising, falling, and fluctuating), which may mix and overlap with each other. Especially when there are distinct temporal patterns at different scales, multiple temporal variations are deeply entangled, bringing extreme challenges for time series forecasting. To tackle the problem of temporal variations entanglement, recent studies employ series decomposition [30] 39] and multi- periodicity analysis [2 29] to differentiate temporal variations at different scales. However, existing methods lack the ability to differentiate temporal variations within each scale, making temporal variations within each scale overlap and become entangled with redundant information.
Motivated by the above. we propose Ada-MSHyper, an Adaptive Multi-Scale Hypergraph Trans- former for time series forecasting. Specifically, Ada-MSHyper map the input sequence into multi- scale feature representations, then by treating the multi-scale feature representations as nodes, an adaptive multi-scale hypergraph structure introduced to discover the abundant and implicit group- wise node interactions at different scales. To the best of our knowledge, Ada-Mr is the first work that incorporates adaptive hypergraph modeling into time series forecasting. The main contributions are summarized as follows:
We design adaptive hypergraph learning (AHL) module model the abundant and implicit group-wise node interactions at different scales and a multi-scale interaction module to perform hypergraph convolution attention, which empower transformers with the ability to model group-wise pattern interactions at different scales.
We introduce a node and hyperedge constraint (NHC) mechanism during hypergraph learning phase, which utilizes semantic similarity to cluster nodes with similar semantic information and leverages distance similarity to differentiate the temporal variations within each scales.
We conduct extensive experiments on 11 real-world datasets. The experimental results demonstrate that Ada-MSHyper achieves state-of-the-att (SOTA) performance, reducing error by an average of 4.56%, 10.38% and 4.97% in MSE for long-range, short-range, and ultra-long-range time series forecasting, respectively, compared to the best baseline.
2
Related Work
Deep Models for Time Series Forecasting. Deep models have shown promising results in time series forecasting. To model temporal patterns at different scales and their interactions, large number of specially designed backbones have emerged. TAMS-RNNY [00 captures periodic temporal dependencies through multi-scale recurrent structures with different update frequencies. TimesNet [29 extends the ID time series into the 2D space, and models multi-scale temporal pattern interactions through 2D convolution inception blocks. Benefiting from the attention mechanism, transformers have gone beyond contemporancous RNN- and CNN-based methods and achieved promising results in time series forecasting. FEDformer (39 combines mixture of expert and frequency attention to capture multi-scale temporal dependencies. Pyraformer 20 extends the input sequence into multi-scale representations and models the interactions between nodes at different scales through pyramidal attention. Nevertheless, with the rapid emergence of linear forecasters 22 35 the effectiveness of transformers in this direction is being questioned.
Recently, some methods have attempted to fully utilize transformers and paid attention to the inherent properties of time series. Some of these methods are dedicated to addressing the problem of semantic information sparsity time series forecasting. PatchTST [3] segments the input sequence into subseries-level patches to enhance locality and capture group-wise interactions. MSHyper (26)
2

---

models group-wise interactions through multi-scale hypergraph structures and introduces k-hop connections to aggregate information from different range of neighbors. However, constrained by the fixed windows and predefined rules, these methods cannot discover implicit interactions. Others emphasize on addressing the problem of temporal variations entanglement time series forecasting. FilM (38) differentiates temporal variations at different scales by decomposing the input series into different period lengths. iTransformer [21] combines inverted structures with transformer to learn entangled global temporal variations. However, these methods cannot differentiate temporal variations within each scale, making temporal variations within each scale overlap and become entangled with redundant information.
Hypergraph Neural Networks. As generalized form of GNNs, hypergraph neural networks (HGNNs) have been applied in different fields, e.g., video object segmentation [17]. stock selection [2]. multi-agent trajectory prediction [31, and time series forecasting [26]. HyperGCC [32
the first work that incorporates convolution operation into hypergraphs, which demonstrates the
superiority of HGNNs over ordinary GNNs in capture group-wise interactions. Recent studies 2
show that HGNNs are promising to model group-wise pattern interactions. LBSN2Vec++ 34
uses hypergraphs for location-based social networks, which leverages heterogeneous hypergraph embeddings to capture mobility and social relationship pattern interactions. GroupNet [3] utilizes multi-scale hypergraph for trajectory prediction, which combines relational reasoning with hypergraph structures to capture group wise pattern interactions among multiple agents.
Considering the capability of HGNNs in modeling group-wise interactions, in this work, an adaptive multi-scale hypergraph transformer framework is proposed to model the group-wise pattern interac- tions at different scales. Specifically, an AHL model designed to model the abundant and implicit group-wise node interactions. In addition, NHC mechanism is introduced to cluster nodes with similar semantic information and differentiate temporal variations within each scale, respectively.
Preliminaries
hypergraph defined as G E), where & {El..... c...... eM} } is the hyperedge set and V {VI,.... Was.... UN } is the node set. Each hyperedge represents group-wise interactions by connecting a set of nodes {V1, U2 ..... vn) S V. The topology of hypergraph can be represented as an incidence matrix H E R N M. with entries Ham defined as follows:
Hypergraph.
{ 1,
On E cm $ em
Homm
(I)
0,
The degree of the with node is defined as d(v,)) -
Ham and the degree of the mtth hyperedge is defined as ddvm) = >n Ham Further, the node degrees and hyperedge degrees are sorted in diagonal matirces D, E N and D. E MMM respectively.
M
Problem Formulation. Given the input sequence X { az, x; ER., [1, [1, where æa represents the values at time step t, T is the input length, and D is the feature dimension. The task of time series forecasting is to predict the future H steps, which can be formulated as follows:
O TIITH = F (Xi 0) HxD
where X + denotes the forecasting results, F denotes the mapping functioon and denotes the learnable parameters of F. The description of the key notations are given in Appendix A
4 Ada-MSHyper
As previously mentioned, the core of Ada Ada-MSHyper is to promote more comprehensive pattern interactions at different scales. To accomplish this goal, we first map the input sequence into sub- sequences at different scales through the multi-scale feature extraction (MFE) module. Then, by treating multi-scale feature representations as nodes, the AHL module is introduced to model the abundant and implicit group-wise node interactions at different scales. Finally, the multi-scale interaction module is introduced to model group-wise pattern interactions at different scales. Notably, during the hypergraph learning phase, NHC mechanism is introduced to cluster nodes with similar
3

---

tat Multóalic Festers Extradion (MPE) Hodus
Fetra Interaation Modiale Seake
Mommaalle Furker Nodak
Sead
bade kapeoapp
lopacdgoend:
mak: Searc
topeeda: (h) Mbglie Ryperaaphh Leurning (AHIL) Mudak:
Predictios
updaed
cedo-oppndde topevdaelopeudge topecdar fanar
Vetakaed 11'
nemn spanity
Figure 1: The framework of Ada Adyperr
semantic information and differentiate temporal variations within each scale. The overall framework of Ada-MSHyper is shown in Figure
4.1 Multi-Scale Feature Extraction (MFE) Module
The MFE module is designed to get the feature representations at different scales. As shown in Figure a), suppose X {a: o E RD, , E [1, })} denotes the sub-sequence at scale s, where
.... denotes the scale index and the total number of scales. N [ is the number
of nodes at scale s and 1a-1 denotes size of the aggregation window scale 1. X 1i the raw input sequence and the aggregation process can be formulated as follows:
Agg X 11000)00 E N , 2 2,
(3)
where Agg is the aggregation function, e.g., ID convolution average pooling, and 6-11 denotes the learnable parameters of the aggregation function at scale
4.2 Adaptive Hypergraph Learning (AHL) Module
The AHL module automatically generates incidence matrices to model implicit group-wise node interactions different scales. As shown in Figure we first initialize two kinds of parameters, i.., node embeddings Eiode V xD and hyperedge embeddings Enyper E RR' at scale s, where M is hyperparameters, representing the number of hyperedges at scale 8. Then, we can obtain the scale-sper spelifi incidence matrix H by similarity calculation, which can be formulated follows:
H SSftMMaat ReLU (Eaodo (Eiyper)"))
where the ReLU activation function is used eliminate weak connections and the SoftMat function is applied to normalize the value of H In order to reduce subsequent computational costs and noise interference, the following strategy designed to sparsify the incidence matrix:
H O,
H
H Fmm E TopK(H; & TopK (H;,.))
H 2m
where 7j is the threshold of TopK function and denotes the max number of neighboring hyperedges connected to node. The final values of H f can be obtained as follows:
H 1. H H
H H <B
where 3 denotes the threshold, and the final scale-specific incidence matrices can be represented as H Compared to previous methods, our adaptive hypergraph learning is novel
[H' ........ H from two aspects. Firstly, our methods can capture group-wise interactions at different scales, while
most previous methods [5] 21 can only model pair-wise interactions at single scale. Secondly, our
methods can model abundant and implicit interactions, while many previous methods 23 26 depend
on fixed windows and predefined rules.

---

Figure 2: The node and hyperedge constraint mechanism.
4.3 Node and Hyperedge Constraint (NHC) Mechanism
Although the AHL module can help discover implicit group-wise node interactions at different scales, we argue that the pure data-driven approach faces two limitations, i.e., unable to efficiently cluster nodes with similar semantic information and differentiate temporal variations within each scale. To tackle the above dilemmas, we introduce the NHC mechanism during hypergraph learning phase.
Given the multi-scale feature representations {X) .'. ule, and the scale-specific incidence matrices {H' ... H ........ H generated from the AHL module, first get the initialized node feature representations V f(X) scale 8, where f can be implemented by the multi-layer perceptron (MLP). As shown Figure the initialized hyper- edge feature representations can be obtained by the aggregation operation based on H Specifically, for the ith hyperedge ef at scale 8, its feature representations ef can be formulated as follows:
X generated from the MFE mod-
= augl Σ (EN()))) Dj)E RD, RR,
where aug is the average operation, .Ne,)) represents the neighboring nodes connected by ej at scale s. and v3 V is the jth node feature representations scale s. The initialized hyperedge feature representations at different scales can be represented as {E ...."..
E Then, based on semantic similarity and distance similarity, we introduce node constraint to cluster nodes with similar semantic information and leverage hyperedge constraint to differentiate the temporal variations of temporal patterns.
Node Constraint. In the data-driven hypergraph, we observe that some nodes connected by the same hyperedge contain distinct semantic information. To cluster nodes with similar semantic information and reduce irrelevant noise interference, we introduce node constraint based on the semantic similarity between nodes and their corresponding hyperedges. As shown in Figure bb), for the jth node at scale 8, we first obtain semantic similarity difference 03 with its corresponding hyperedges:
= {abs(v - - (N()),
(8)
where abs refers to the operation of calculating the absolute value. The node loss Lnode scale based on node constraint can be formulated as follows:
N
I L'node
N
where N the number of nodes at scale 8. Empowered by the node constraint, our method can enjoy more advantageous group-wise semantic information than pure data-driven hypergraph. Further experimental results and visualization analysis in Section 5.3 and Appendix H demonstrate the effectiveness of the node constraint in clustering nodes with similar semantic information.
Hyperedge Constraint. Since time series is collection of data points arranged in chronological order, some recent works 24 2d show that connecting multiple nodes sequentially through patches or hyperedges can represent specific temporal variations. Therefore, to deal with the problem of temporal variations entanglement, we introduce hyperedge constraint based on distance similarity. As shown in Figure Cc), we first compute the cosine similarity to reflect the correlation of any two hyperedge representations at scale which can be formulated as follows:
e (e))T
(10)
|e:1l2
eel112
5

---

where az, represents the correlation weight. e and ej are the ith and jth hyperedge representation at scale s, respectively. Then, we use Euclidean distance Di., to measure the differentiation magnitude between any two hyperedge representations, which can be formulated as follows:
Dij = e - e e V - ((e:)d (eg)))²,
(11)
The hyperedge loss hypeer at scale based the correlation weight and Euclidean distance can be formulated as follows:
(May² E M E M j=1 Dis + (1 04. (4)))/xx( Da.. 0)),
Lyper
(12)
where y > O denotes the threshold. Notably, when oij = 1, indicating that ef and ej are deemed similar, the hyperedge loss turns to Lhyper (M*)² 2i=1 2j=1 Ci, Di.j. where the loss will increase if Di,j becomes large. Conversely, when Cijj = 0, meaning e; and are regarded as dissimilar, the hyperedge loss turns to Thyper = (M*)² Di=
M'
D 0), where the loss will increase if Di.j falls below the threshold and turns smaller. Other cases lie between the above cireumstances. We further provide the visualization results in Section [5.3 and appendix H to verify that our constraint loss can differentiate temporary variations of temporary patterns within each scale and promote forecasting performance. The final constraint loss Leanst based on node constraint and hyperedge constraint can be formulated as follows:
M - ( 03,;)
Leonst
Liode + (1 (1- (1-A) Σ
s=1 Lipper
(13)
wherer À denotes the hyperparameter controlling the balance between node loss and hyperedge loss.
4.4 Multi-Scale Interaction Module
To promote more comprehensive pattern interactions at different scales, direct way is to mix multi-scale node feature representations at different scales. However, we argue that intra-scale interactions and inter-scale interactions reflect different aspects of pattern interactions, where intra- scale interactions mainly depict detailed interactions between nodes with similar semantic information and inter-scale interactions highlight macroscopic variations interactions 27). Therefore, instead of directly mixing multi-scale pattern information as a whole, we introduce the multi-scale interaction module perform inter-scale interactions and intra-scale interactions.
Intra-Scale Interaction Module. Due to the semantic information sparsity of time series, traditional pair-wise attention may may cause the information utilization bottleneck (5). In contrast, some recent studies 2x 26
show that group-wise interactions can provide more informative insights in time series forecasting. To capture group-wiss wise interactions among nodes with similar semantic information within each scale, we introduce hypergraph convolution attention within the intra-scale interaction module. Specifically, given H we first use attention mechanism to capture the interaction strength of each node of E V and its related hyperedges at scale s, which can be formulated as follows:
expla(filu,,ee) eiD)
Hig E EegEN(r)) cxplo of
(14)
where [..-] denotes the concatenation operation of the ith node and its related hyperedges. is trainable MLP, and N (uf) is the neighboring hyperedges connected to vf, which can be accessed using H Then, considering the symmetric normalized hypergraph Laplacian convolution A
D 1 'HD 1/2 used in HGNN [33,, the multi-head hypergraph convolution attention can be formulated follows:
" E NDD
(15)
where 13th is the updated node feature representations at scale 8, + the aggregation function used for combing the outputs of multi-head, e.g., concatenation average pooling, is the activation

---

function, e.g., LeakyReLU and ELU. H and P; are the enriched incidence matrix and the learnable weight matrix of the jth head at scale s, respectively. is the number of heads.
Inter-Scale Interaction Module. The inter-scale interaction module is introduced to capture pattern interactions at different scales. To achieve this goal, direct way is to model group-wise node interac- tions across all scales. However, detailed group-wise node interactions across all scales can introduce redundant information and increase computation complexity. Therefore, we adopt a hyperedge attention within the inter-scale interaction module to capture macroscopic variations interactions at , E.5 we
different scales. Technically, based on the hyperedge representations & = {() E. ,.... first adopt linear projections to get queries, keys, and values Q, K, V Rx Then the hyperedge attention can be formulated as follows:
QKT
sof softmme tmaar
VV,
(16)
vDx
where V is the updated hyperedge feature representations.
4.5 Prediction Module Loss Function
After obtaining the updated node and hyperedge feature representations, we concatenate them and feed them into a linear layer for prediction. We choose Mean Squared Error (MSE) as our forecasting loss, which can be formulated as follows:
1
Lysse
H T+IT+H
X 1
(17)
where + and XO AT+1:++ are ground truth and forecasting results, respectively. Notably, during training phase, Limse is used to regulate the overall learning process, while Leonst only used to constrain hypergraph learning process.
4.6 Complexity Analysis
For the MFE module, the time complexity is O(N)), where N is the number of nodes the finest scale and N is equal to the input length T. is the aggregation window size at the finest scale. For the AHL module, the time complexity is ((M + M²), where M is the number hypergraphs at the finest scale. For the intra-scale interaction module, since D and D are diagonal matrices, the time complexity ((M N). For the inter-scale interaction module, the time complexity is M In practical operation, M and 1 is the hyperparameter and is much smaller than N. As a result, the total time complexity of Ada-MSHyper is bounded by (NN).
Experiment
5.1 Experimental Setup
Datasets. For long-range time series fore- casting, we conduct experiments on 7 com- monly used benchmarks, including ETT (ETThl, ETTh2, ETTml, and ETTm2), Traffic, Electricity, and Weather datasets fol- lowing (30 21 26]. For short-range time
Table 1: Dataset statistics.
Dassor
Verlowe
Proferie
Pispuecy
Ixtermation
UTT edurs)
(M). 192, 336,700
(15 min. Hooey)
18.360
Timperatns
Weater
336.70)
325
Woyter
(Wh. 191, 116.700
Honryy Hoady
0077
series forecasting, we adopt benchmarks from PEMS (PEMS03, PEMS04, PEMS07, For ultra-
Tnlls
(0N, 192, 336.72)
366
Sranpeetaiinn
FEAN (bwen) (1788557)
(12,24.44)
1843-0.000
Toite nursert
and PEMS08) following 21 27
long-range time series forecasting, we adopt ETT datasets following [8]. statistics. In addition, the forecastability is derived from one minus the entropy of the Fourier
Table gives the dataset
decomposition of time series 27 14]. Higher values mean greater forecastability.
Baselines.
We compare Ada-MSHyper with 15 competitive baselines, i.e., iTransformer 211, MSHy-
per (26). PStchTST 23 TimeMixer (27] MSGNet 4. CrossGNN [16), TimesNet (29, WITRAN
18. SCINet [19,, Crossformer 36
FiLM
38 DI .nnear 35 FRDformer [39], Pyraformer 20
and Autoformer [30].
7

---

Experimental Settings. Ada- Ada-MSHyper is trained/tested on a single NVIDIA Geforce RTX 3090 GPU, MSE and MAE are used as evaluation metrics and lower values mean better performance. Adam is set as the optimizer with the initial learning rate of 10 It notable that the above mentioned baseline results cannot be used directly due to different input and output lengths. For fair comparison, we set the commonly used input length T = 96 and output lengths H E {96. 192, 336. 720) for long-ram oong-range forecasting. H E {12, 24, 48) for short-range forecasting, and H E {1080, 1440, 1800, 2 for ultra-long-range forecasting. More descriptions about datasets, baselines, and experimental settings are given in Appendix
B
and D respectively.
5.2 Main Results
Long-Range Forecasting. Table 2 shows the results of long-range time series forecasting under multivariate settings. We can observe that: (1) Ada-MSHyper achieves the SOTA results in all datasets, with an average error reduction of 4.56% and 3.47% compared to the best baseline in MSE and MAE, respectively. (2) FEDformer and Autoformer exhibit relatively poor predictive performance. This may be that vanilla attention and simplistic decomposition techniques are insufficient in capturing multi- (3) By considering multi-scale pattern interactions, TimeMixer achieves competitive results. However, its performance deteriorates on the datasets with low forecastability (e.g., ETThl and ETTh2 datasets). In contrast, Ada-MSHyper still maintains superiority on low forecastability datasets by modeling group-wise pattern interactions. Notably, despite modeling group-wise pattern interactions, the performance of MSHyper and PatchTST still lags behind that of Ada-MSHyper, indicating that predefined rules may overlook implicit interactions and introduce noise interference for forecasting. Moreover, for long-range time series forecasting under univariate settings, Ada-M Ada-Mr gives an average error reduction of 7.57% and 4.65% compared to the best baseline in MSE and MAE, respectively. The univariate results are given in Appendix
scale pattern interactions.
Table 2: Results of long-range time series forecasting under multivariate settings. The best results are bolded and the second best results are underlined. Results are averaged from all prediction lengths. Full results are listed in Appendix E
Maeds
Ada aiitgg
MSHlge
Conftttatt
Timaske
FEM
Auadama:
Merk
MAE.AA
MAME
509 MAR
ADR MAR
MAR
5090 MAR
MBe MAR
MAEMAR
MBe MAR
565 MAB
MAE
MSEMAE
Wester
02318344
170009
03590
RTTM
83000455
03090655
04860c99
64560852
05160/53
04900053
0371 83M
9303 8401
1315 D40
8:30 6412
03060407
8300640
633F
09433604
64140427
05990315
64220423
B437044
0300 8.M
8410
5412040
1260441
03670400
4417148
04800446
0411040
04400491
0:0000000
01850
01000 911
Tulk
0333
639
86410330
05586033
0.56333
66200336
66250030
06370354
Short-Range Forecasting. Table 3 summarizes the results of short-range time series forecasting under multivariate settings. It is notable that the PEMS datasets record multiple time series of citywide traffic networks and show complex -patial-temporal correlations among multiple variates. We adopt the same settings as iTransformer (26) and TimeMixer [20]. Ada-MSHyper still achieves the best performance in PEMS datasets, verifying its effectiveness in handling complex multivariate short-range time series forecasting. Specifically, Ada-MSHyper gives an average error reduction of 10.38% and 3.82% compared to the best baseline in terms of MSE and MAE, respectively.
Table 3: Results of short-range time series forecasting under multivariate settings. Results are averaged from all prediction lengths. Full results are listed in Appendix B
Modeb
Adt Mihhyar
(Dun))
monedommer (3124)
MSilsper
Timchturr (200))
PWEAINT (2023)
Timeeka
(3025)
Cooafmmr
(2023)
SCTNa (2822)
ITDXmnee (2803)
Assofomee (301))
Maunie
MISE MAR.
MSR MARI
MSI MAT:
MST MAE:
MSE MAR
MSR MAR:
MAT
MISE MAR:
MSR MAT:
MSN MAT:
MSE MAR
PEM500
6455 0.193
0.006 0.204
0.123 0226
0336
0.151 0266
0.19 0225
0219-0338
0.338 0.388-0213 0253
0,009 0203
0.67
0.46 0.53
PMM50M
0.388 8.199
0.0800000 0:07
01830.901 963
370
100 8:700
D.150.000 590
038500007
01065
308
051000557
01330:351
0700
038100777
01120:011
0.000
PLM1500
0.138 0.10
0.12/0212
0220
013903374
0:239
11399
0.100
012610.366
0222002066
0.138
0,244
0323
Ultra-Long-Raan Forecasting, Table 4 summarizes the results of ultra-long-range time series
forecasting under multivariate settings. We can see that: (1) Ada-MSHyper achieves SOTA results in
8

---

almost all benchmarks, with an average error reduction of 4.97% and 2.21% compared to the best baseline in MSE and MAE, respectively. (2) Compared with other baselines, PatchTST and MSHyper achieve competitive results. The reason may he that group-wise interactions can help mitigate the issue of semantic information sparsity. (3) Compared to PatchTST and MSHyper, Ada-MSHyper achieves superior performance, the reason may be that the inter-scale interaction module can help capture macroscopic variations interactions, especially for the ultra-long-rang time series.
Table 4: Results of ultra-long-range time series forecasting under multivariate settings, Results are averaged from all prediction lengths. Full results are listed in Appendix E
Medels
Ads-Miillppr
(Oum)
T Tirnnformee
(2024)
MSHtyper
(2004)
TimeMliser
(2074)
WITKAN
(2023)
Paad
(2025)
D mnrer
(1023)
Crossforncr (2023)
FEEtioreer
(2022)
Pynifoenee (1022)
Auteloener
(2021)
Meuri
MSE MAE
MSE MAE
MSE
MAE
MSE MAE
MSE MAE
MSE MAE
MSE MAE
MSE MAE
MSE MAE
MSE MAE
MSE MAE
ETTh
IU65 0.567
0.66606611
1.745
0.610
0.840
0.734 0.33
0.588
0.606 0.624
0.921
1.091
0.763 0.637
0.03
0.808 0.675
ETThZ
0,490 0.450
0.541 0.513
10512
0.495
0.563
0.447 0.337
0.505 0.488
0.77
2.530
0.574
3,263 1,509
0,658 0.648
(1883
494
0.544
0:540
3,555
1.0%00.511
0.25 (4M
0.465
(1449
0.464
0.447
0,465
0.466 6,434
0,462
0,468
0,595
3,55
1,885
0.46)
0,875
4,566 1,745
0.16 0,891
5.3 Ablation Studies
AHL Module. To investigate the effectiveness of the AHL model, we conduct ablation studies by designing the following three variations: (1) Replacing the AHI. module with adaptive graph learning module (-AGL). (2) Replacing the AHI model with one incidence matrix to capture group-wise node interactions at different scales (-one). (3) Replacing the AHL module with predefined multi-scale hypergraphs (-PH), i.e., each hyperedge connected fixed number of nodes (4 in our experiment) in chronological order. The experimental results on ETThl dataset are shown in Table [ We can observe that -AGL gets the worst forecasting results, indicating the importance of modeling group-wise interactions. In addition, -PH and -ne perform worse than Ada-MSHyper, showing the effectiveness of adaptive hypergraph and multi-scale hypergraph, respectively.
Table 5: Results of different adaptive hypergraph learning methods and constraint mechanisms.
Variation
AGL
P11
who NC
wioo II
wfo NICC
Adx Add-MSilyper
Metric
MSE
MAR
MSF
MAF
MSF
MAF
MAF
MSE
MSE
MAF
0,542
0.423
0.386
0.390
0.403
0.416
0.903
0.372
0,303
0.559
0.502
0.448
0.423
0.437
0.435
0.425
0.441
0.422
0.433
720
0.563
0.617
0.456
0.458
0.448
0.457
0.451
0.460
0.449
0.466
0.445
0,459
NHC Mechanism. To investigate the effectiveness of the NHC mechanism, we conduct ablation studies by designing the following three variations: (1) Removing the node constraint (-w/o NC). (2) Removing the hyperedge constraint (-w/o HC). (3) Removing the NHC mechanism (-w/o NHC). The experimental results on ETTh1 dataset are shown Table [ We can observe that Ada-MSHyper performs better than -w/o NC and -w/o HC, showing the effectiveness of node constraint and hyperedge constraint, respectively. In addition, -w/o NHC gets the worst forecasting results, which demonstrates the superiority of the NHC mechanism in adaptive hypergraph learning. More results about ablation studies are shown in Appendix F
To further demonstrate the effectiveness of the node constraint in clustering nodes with similar semantic information, we present case visualization with -w/o NHC and -w/o HC on Electricity dataset. We randomly select one sample and plot the node values at the finest scale. We categorize the nodes into four groups based on the node values. Nodes with the same color indicate that they may have similar semantic information. As shown in Figure 3a for the target node, nodes of other colors may be considered as noise. We drew the nodes related to the target node in black color based on incidence matrix H'. As shown in Figure 3b) due to the lack of node constraint, -w/o NHC can only capture the interactions among the target node and neighboring nodes and cannot distinguish nuanced noise information. In Figure 3c with the node constraint, -w/o HC can cluster neighboring and distant but still strongly correlated nodes. In Figure with the NHC mechanism, Ada-MSHyper cannot only cluster nodes with similar semantic information but can differentiate temporal variations. The full visualization results are shown in Appendix HH
5.4 Parameter Studies
9

---

400
24
300
300-
2257
350
200
200
200
200
100
300
100
100
20
60
100
60
80
100
20
40
80
100
0
20
40
60
80
100
(a) Input sequence
(b) (b)-w/o NHC
(c) -w/o HC
(d) Ada- Ada-MSHyper
Figure 3: Visualization the node constraint effect on Electricity dataset.
We perform parameter studies to measure the impact of the number of scales (#scaale) and
0.55
0.55
0-46
0.50
0.50
the max number of hyperedges connected node (#hyperedges). The experimental results on ETTh dataset are shown Figure 4 we can see that: (1) the best performance can be obtained when #scales is The reason that smaller #scales cannot provide sufficient pattern information and larger #scales may introduce excessive parameters and result in overfitting problems. (2) The optimal #hyperedges is The reason is that smaller #hyperedges cannot capture group-wiss wise interactions sufficiently and larger #hyperedges may introduce noise. More results about parameter studies are shown in Appendix
Ow720
0.45
C-368
Cw220
C-320
0.35
0.35
Number Seales
Number Hyperedges
Figure 4: The impact of hyperparameters.
5.5 Computational Cost
We compare Ada-MSHyper with the two latest transformer-based methods, i.c., iTransformer and PatchTST, traffic datasets with the output length of 96. The experimental results shown in Table Although we have a larger number of parameters, we achieve lower training time and lower GPU occupation due the matrix sparsity operation in the model and the optimization of hypergraph computation provided by torch seecchhgemttr___ Considering the forecasting performance and the computation cost, Ada- Ada-MSSyppr demonstrates its superiority existing methods.
Table 6: Computation cost.
Methods
Training Time
Parameters
GPU Occopation
MSE results
Adh-MSHyper
6.4%%
8,965,30
6,542MB
0,384
Traantommer
7.863s
6,731.984
6,738MB
0.395
PatcTTT
17.603c
548,704
9,78MB
0.526
Conclusions and Future Work
In this paper, we propose Ada-MSHyper with adaptive multi-scale hypergraph for time series forecasting. Empowered by the AHL module and multi-scale interaction module, Ada-MSHyper can promote more comprehensive multi-scale group-wise pattern interactions, addressing the problem of semantic information sparsity. Experimentally, Ada-MSHyper achieves the SOTA performance, reducing prediction errors by an average of 4.56%. 10.38%, and 4.97% in MSE for long-range, short-range, and ultra-long-rang range time series forecasting, respectively. In addition, the visualization analysis and the ablation studies demonstrate the effectiveness of NHC mechanism in clustering nodes with similar semantic information and addressing the issue of temporal variations entanglement.
In the future, this work can be extended in the following two aspects. First, since 2D spectrogram data may offer a better representation for time series forecasting, we will adapt our framework to the 2D spectrogram data in time-freene fime-freyuency domain. Second, since the features extracted by the MFE module may contain redundant information, we will design a disentangled multi-scale feature extraction module to extract more independent and representative features.
10

---

7 Acknowledgement
This work was supported by the Science Foundation of Donghai Laboratory (Grant No. DH- 2022ZY0013).
References
[1] Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:/803.011 1803.01271, 2018.
[2] Song Bai, Feihu Zhang, and Philip HS Torr. Hypergraph convolution and hypergraph attention. Pattern Recognition, 110: 100677 2021.
[3] George EP Box and Gwilym M Jenkins. Some recent advances in forecasting and control. Journal of the Royal Statistical Society. Series C (Applied Statistics), 17(2):11109, 1968.
[4] Wanlin Cai, Yuxuan Liang, Xianggen Liu, Jianshuai Feng, and Yuankai Wu. MSGNet: Learning multi-scale inter-series correlations for multivariate time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 11141-11149 2024.
[5] Haizhou Cao. Zhenhao Huang, Tiechui Yao, Jue Wang, Hui He, and Yangang Wang. In- Parformer: Evolutionary decomposition transformers with interactive parallel attention for long-term time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 37, pages 6906 6915, 2023.
[6] Donghui Chen, Ling Chen, Zongjiang Shang, Youdong Zhang, Bo Wen, and Chenghu Yang. Scale-aware neural architecture search for multivariate time series forecasting. ACM Transac- tions on Knowledge Discovery from Data, 2024.
[7] Donghui Chen, Ling Chen, Youdong Zhang, Bo Wen, and Chenghu Yang. A multiscale interactive recurrent network for time-series forecasting. IEEE Transactions on Cyberneties, 52(9):8793-888) -8099, 2021.
[8] Ling Chen, Donghui Chen, Zongjiang Shang, Binqing Wu, Cen Zheng, Bo Wen, and Wei Zhang. Multi-scale adaptive graph neural network for multivariate time series forecasting, IEEE Transactions on Knowledge and Data Engineering, pages 10748-10761, 2023.
[9] Ling Chen and Jiahua Cui. TPRNN: A top-down pyramidal recurrent neural network for time series forecasting. arXiv preprint arXiv:2312.06322 2023.
[10] Zipeng Chen, Qianli Ma, and Zhenxi Lin. Time-aware multi-scale RNNs for time series modeling. In IJCAI, pages 2285-2291, 2021.
[11] Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan Mathur, Rajat Sen, and Rose Yu. Long- term forecasting with TiDE: Time-series dense encoder. Transactions Machine Learning Research, 2023.
[12] Vijay Ekambaram, Arindam Jati, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. Tsmixer: Lightweight mip-mixer model for multivariate time series forecasting. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 459 469, 2023.
[13] Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, pages 3558-3565. 2019.
[14] Georg Goerg. Forecastable component analysis. In International conference machine learning, pages 64-22. PMLR, 2013.
[15] Hansika Hewamalage, Klaus Ackermann, and Christoph Bergmeir. Forecast evaluation for data scientists: common pitfalls and best practices. Data Mining and Knowledge Discovery, 37(2):788-833 832, 2023.

---

[16] Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. CrossGNN: Confronting noisy multivariate time series via cross interaction refinement. Advances in Neural Information Processing Systems, 36:46885-46902, 20233
[17] Yuchi Huang, Qingshan Liu, and Dimitris Metaxas. I video object segmentation by hypergraph cut. In 2009 IEEE conference on computer vision and pattern recognition, pages 1738-1745. IEEE, 2009.
[18] Yuxin Jia, Youfang Lin, Xinyan Hao, Yan Lin, Shengnan Guo, and Huaiyu Wan. WITRAN: Water-wave information transmission and recurrent acceleration network for long-range time series forecasting. Advances in Neural Information Processing Systems, 36, 2024.
[19] Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. SCINet: Time series modeling and forecasting with sample convolution and interaction. Ad- vances in Neural Information Processing Systems, 35:5816-555 1628, 2022.
[20] Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In Proceedings of the International Conference Learning Representations, 2021.
[21] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang. Lintao Ma, and Mingsheng Long. iTrrasformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations, 2023.
[22] Yong Liu, Chenyu Li, Jianmin Wang, and Mingsheng Long, Koopa: Learning non-stationary time series dynamies with koopman predictors. In A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, vol- ume 36. pages 1227 12290.. Curran Associates, Inc., 2023.
[23] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. The Eleventh International Conference on Learning Representations, 2022.
[24] Ramit Sawhney, Shivam Agarwal, Arna Wadhwa, Tyler Derr, and Rajiv Ratn Shah. Stock selection via spatiotemporal hypergraph attention network: learning rank approach. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 497-504, 2021.
[25] Rajat Sen, Hsiang-Fu Fu Yu, and Inderjit S Dhillon. Think globally, act locally: A deep neural network approach to -igh-dimensional time series forecasting. Advances in neural information processing systems, 32, 2019.
[26] Zongjiang Shang and Ling Chen. MSHyper: Multi-scale hypergraph transformer for long-range time series forecasting. arXiv preprint arXiv:2401.09200 2024.
[27] Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and JUN ZHOU. TimeMixer: Decomposable multiscale mixing for time series forecasting. In The Twelfth International Conference on Learning Representations, 2023.
[28] Binqing Wu, Weiqi Chen, Wengwei Wang. Bingqing Peng. Liang Sun, and Ling Chen. Weath- erGNN: Exploiting meteo- and spatial- depeeeeieess for local numerical weather prediction birs-corr, In Proceedings of the International Joint Conference on Artificial Intelligence, pages 2433-2441, 2024.
[29] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long, TimesNet: Temporal 2d-variation modeling for general time series analysis. In The eleventh international conference on learning representations, 2022.
[30] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems, pages 22419-224300 2021.
[31] Chenxin Xu, Maosen Li, Zhenyang Ni, Ya Zhang, and Siheng Chen. GroupNet: Multiscale hypergraph neural networks for trajectory prediction with relational reasoning. In Proceedings of the IEFF/CVE Conference on Computer Vision and Pattern Recognition, pages 6498-6507, 2022.
12

---

[32] Naganand Yadati, Madhav Nimishakavi, Prateck Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. HyperGCN: new method for training graph convolutional networks on hypergraphs. Advances in Neural Information Processing Systems, 32, 2019.
[33] Yichao Yan, Jie Qin, Jiaxin Chen, Li Liu, Fan Zhu, Ying Tai, and Ling Shao. Learning multi- granular hypergraphs for rideo-based person re-identification. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 2899-2999999 1999-2908, 2020.
[34] Dingqi Yang. Bingqing Qu, Jie Yang, and Philippe Cudré-Maurows LBSN2Vec++: Hetero- geneous hypergraph embedding for location-based social networks. IEEE Transactions on Knowledge and Data Engineering, (114)):183--855) 2020.
[35] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? arXiv preprint arXiv:2205.13504 13504, 2022.
[36] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. Proceedings of the International Conference on Learning Representations, 2023.
[37] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 11 11106-11115, 2021.
[38] Tian Zhou, Ziqing Ma. Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, Rong Jin, al. FiLM: Frequency improved legendre memory model for long-term time series forecasting. Advances in Neural Information Processing Systems, 15:12677- 2022.
[39] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Proceedings of the International Conference on Machine Learning, pages 27268-27286 2022.
13

---

A Descriptions of Notations
To help understand the symbols used throughout the paper, we provide a detailed list of the key notations in Table 7
Table 7: Description of the key notations.
Notation
Descripions
Hlyeergaph
Hyyeredge set
Node set
N
Number of nodes
M
Number of hypenddges
Trgut longth
11
Outpat length
Feature dimension
Total namber of temponal scales
Scale index
Historical input sequence
X"
Sub sequence at scale a
Values time slep
*9+17++ E D
Forecasting results
Enoda E xDN
Node embeeklings at scale
Ehyper R ARA*** x
Flypenegge emboddiings scale
<7
hyperedge scale
node at scale
<9
hyperaige feature representation scale
17
node feature represcutation scale
Threshold of TopK function
Threshold of the scalo-pociific incidence matrices
Threshold of the Enclidean distance
Halancing hyperparameter between node loss and hyperodge loss
H1
Incidence masrix scale
15-1
Size of the agzregation window scile
V"
Initislized node festure reprosentations scall
Inniiliizdd hyperadge feature representations at scale
Updated node fouture representations scale
V
Upáated hypeeedge feature representations
Number heads
76
Enriched incidence matris of yth head scale
A((e()
Neighhoring hyperedges connected to of
.((ef)
Neighboring nodes connected by cf
Q.K. E MXD
Queries, keys, and values
Conrelation weight between sih and jih hypersdge repersenaations
De.
Eaclddean distance between sth and jthh bypenedge representations
Node constraint loss al scale
Mageer
Hypensdges constraint at scale
Leanet
Final constraiar loss
Lmnn
MSE loss
14

---

B Descriptions of Datasets
Datasets. For long-range time series forecasting, we conduct experiments on commonly used benchmarks, including Electricity Transformers Temperature (ETT), Traffic Electricity and Weathe datasets following 30 21 26 ETT datasets include data from two counties in the same Chinese province, each data point comprising seven variables: the target variable "oil temperature" and six power load features. The datasets vary in granularity, with "h" indicating hourly data and "m" indicating 15-minute intervals. Weather dataset contains 21 weather indicators collected every 10 minutes from a weather station in Germany. Electricity dataset records hourly electricity consumption of 321 clients. Traffic dataset provides hourly road occupancy rates from 821 freeway sensors. For short-range time series forecasting, we use four benchmarks from PEMS (PEMS03, PEMS04, PEMS07, and PEMS08), referenced in [27 19]. These datasets capture 5-minute traffic flow data from freeway sensors. For loira-long-raggee time series forecasting, we adopt ETTh1, ETTh2, ETTmI, and ETTm2 following [18]. Table 8 gives the detailed dataset statistics. In addition, the forecastability derived from one minus the entropy of the Fourier decomposition a time series 27 14]. Higher
values mean greater forecastability.
Table 8: Detailed dataset statistics.
Dataset
a Variates
Prodiction Length
Frequency
Rorocstaabllit
Information
ErThL ErTh2 7
(96, 236,720)
Hourly
Temperature
Long-ter
FTT=l, FTT=2
7
336,720)
Temperature
Weather
(96, 192, 336,720)
mins
0.75
Weather
Electricity
321
(96, 192, 336,720)
Hourly
0.77
Electricity
Trafic
862
(96, 192, 236,720)
Houtly
0.68
Transportation
PEMS03
358
12
Smin
0.65
Transporiaionn
Short-temm
PEMS04
307
12
Smins
0.45
Transpottatinn
PFM507
883
12
Smins 0.58
Trmngrctation
PEMSO8
170
12
5mins
0.52
Tranppcttttonn
We adopt the same data processing and train-validation-tess split protocol in existing works [26 21 23]. We split each dataset into training, validation, and test sets based on chronological order. For PEMS (PEMS03, PEMS04, PEMS07, and PEMS08) dataset and ETT (ETThl, ETTh2, ETTml, and ETTm2) dataset, the train-validation-tes) split ratio is 6:2:2. For Weather, Traffic, and Electricity dataset, the main-validation-tee test split ratio is 7:2:1.
Metric details. Following existing methods (26 21), we employ Mean Squared Error (MSE) and Mean Absolute Error (MAE) as our evaluation metrics, which can be formulated follows:
Linse
Xo
T
X X
(18)
1
Lmee
T
X
(19)
H
where T and H are the input and output lengths. X0 and ground truth.
7+1T+ and X
x the predicted results
C Descriptions of Baselines
We compare Ada- Ada-MSHyper with 15 competitive baselines. Below are brief descriptions of the baselines: (1) iTransformer [21]: Applies the attention and feed-forwee forward network on the inverted dimensions, i.c.. the time points of individual series are embedded into variate tokens, and the feed- forward network is applied for each variate token to learn nonlinear representations. (2) MSHyper [26]: Utilizes rule-hased multi-scale hypergraphs to model high-order pattern interactions in univariate time series. (3) PatchTST 23 : Uses channel-independen techniques and treats subseries-level patches as
dot gov
httpa://archive. wtw.:i.w......eee//eeeeeeeee.ee.
https::/www. bec-bebc-ajnn.e. meg.mppg... de/wetter/
15

---

input tokens to Transformer, facilitating semantic extraction of multiple time steps in time series. (4) TimesMixer [27]: Employs fully MLP-based architecture with past-decomposable-me mixing and futurr-multipprdiicoo--iiii
blocks to leverage disentangled multiscale series. (5) MSGNet Leverages frequency domain analysis to extract periodie patterns and combines an attention mechanism with adaptive graph convolution to capture multi-scale pattern interactions. (6) CrossGNN
[6]: Uses an adaptive multi-scale identifier to construct multi-scale representations and utilizes a cross-scale GNN to capture multi-scale pattern interactions.
(7) TimesNet [29]: Conducts multi- periodicity analysis by extending ID time series into set of 2D tensors, modeling complex temporal variations from 2D perspective. (8) WITRAN [8] Proposes an RNN-bbaser architecture that handles univariate input sequences from a 2D space perspective, maintaining a fixed scale throughout the processing. (9) SCINet [9]]: Uses recursive downsample-coe wwwplll--ooooooeieneeeeeeeeeeeee
architecture to extract temporal features from downsampled sub-sequences or features. (10) Crossformer [36]: Adopts cross-dimension attention capture inter-series dependencies for multivariate time series forecasting. (11) FiLM 3 Applies Legendre polynomial projections to approximate historical information, uses Fourier projections remove noise, and adds low-rank approximation to speed up computation. (12) DLinear [35]: Decomposes time series into two different components and uses single linear layer for each component to model temporal dependencies. (13) FEDformer [39]: Utilizes a seasonal-trend decomposition method to capture the global profile of time series and a frequency-enhai pyramidal attention module to extract inter-scale features at different resolutions and intra-scale features at different ranges with linear complexity. (15) Autoformer [30]: Uses an auto-correlation mechanism based on series periodicity to capture features at the sub-series level.
Transformer to capture more detailed structures. (14) Pyraformer [20]: Utilizes
D Experimental Settings
We repeat all experiments 3 times and use the mean of the metrics as the final results. The training process is early stopped when there no improvement within 5 epochs. Following existing works 21 27 351, we use instance normalization to normalize all datasets. The max number of scale S set to We use ID convolution as our aggregation function. For other hyperparameters, we use Neural Network Intelligence (NNI toolkit to automatically search the best hyperparameters, which can greatly reduce computation cost compared to the grid search approach. The detailed search space of hyperparameters is given in Table 9 The source code of Ada-MSHyper is released on GitHub 6
tNolktoolkii
Table The search space of hyperparameters.
Parameters
Choise
Batch size
[8, 16, 32, 64, 128)
Number of hyperedges scale
(10, 20, 30, 50J
Number of byperedges scale 2
[5, 10, 15)
Number of byperedges scale 3 [1,2, 8, 12)
Aggrogation window at scale
(2,4,8) 8]
Aggregation window at scale 2
[2,4)
[1,3,5, 10, 15,
8
(.3.0.)))
(..200.3.04...5)
E
Full Results
We compare Ada-MSHyper with 13 baselines across four tasks: long-range forecasting for multi- variate time series, long-range forecasting for univariate time series, ultra-long-range forecasting for multivariate time series, and short-range forecasting for multivariate time series. For fair compari- son, we evaluate Ada-MSHyper and baselines under unified experimental settings of cach task. The average results from all prediction lengths are presented in tables, where the best results are bolded
http:p////ni..eeccheeeeee.///ee Ada ASatan//Aaa-MMMyerrr
16

---

and the second best results are underlined. * indicates that some baselines do not meet our settings, thus we rerun these baselines using their official code and fine-tune their key hyperparameters.
Long- Range Time Series Forecasting Under Multivariate Settings. Table 10 summarizes the results of long-range time series forecasting under multivariate settings, where the results of baselines without are cited from iTrrnsformer [21]. We can see from Table that Ada-MSHyper achieves the SOTA results on all datasets. Specifically, Ada-MSHyper gives an average error reduction of 4.56% and 3,47% compared to the best baseline in MSE and MAE, respectively.
Table 10: Full results of long- range time series forecasting under multivariate settings.
-2004
22.24
(2324)
(2104)
(2103)
(2423)
(2/23)
(2/02)
1466
MAR
1466
MAA.
MAS
136
0.20
00296
19202
6229
1257
11225
0.300
1220
1.341
0.335
8:338
0.28
8.3
0.26
5.30
0.350
8.74
1121
0229
0314
0322
0.36
0333
0:36
0.24
8:30
625
1.31
0.22
8.54
130
0.200
8.50
1.255
00.39
0225
02211
0032
0231
0335
0246
0036
0280
0.50
04200
0.30
0.40
0.300
0.25
0.30
0.46
1.30
0.54
$.30
19.03
19.04
RAM
039
015
0.06
0.6
1116
0.88
ETMh:
11283
HANI
0.401
0.61
8.8m
1.50
0311
8.11
6:31
0.7
E.41
0.6%
1.60
1.16
1.454
1.11
RTT
11345
0335
0220
0331
042
033
134
E332
0353
6310
037%
2415
0353
5415
0.07
8.46
0.590
0.01
0.6
040
0151
0462
0666
6586
64%
6400
6424
0461
6551
1.00
6471
8%
133
1:30
0330
DTTW2
336
0385
8.38
03511
0346
0335
0641
0305
0302
636
1052
6346
0385
654
5351
036
842
138
0.30
0333
1.34
0356
8.11
11:81
61%
13%%
(100)
636
61%
(10
0911
0316
8.361
0.43
9.30
0382
0431
0237
0022
0289
0.23
R240
0.30
6.3
0.91
0226
1.30
5950
0.48
Long-Range Time Series Forecasting Under Univariate Settings. Table and summarize the average results and full results of long-range time series forecasting under univariate settings, where the results of baselines without * are cited from DLinear [35). Following existing works ETT only predicting target variate "oil temperature" given inputs from all variables. We can see from Table [I] that Ada-MSHyper achieves the SOTA results all datasets. Specifically, Ada-MSHyper gives average error reduction of 7.57% and 4.65% compared to the best baseline in terms of MSE and MAE, respectively.
33 26 23 30) we set the univariate forecasting
Table 11: Long- range time series forecasting results under univariate settings.
Midells
Adi-Miitpper
(Dus)
(Trarsfomer*
(2024)
MSllyper*
(2024)
TimeMixee*
(2024)
PackIST
(2023)
Dliter
(2023)
Creseformer
(2023)
Pyrafoence (2022)
Pettfmee
(2022)
Autofoener
(2021)
Infoemee
(2021)
Merr
MSEMAE
MSU MAB
MSB MALI
MSE MAE
MSU MAB
MSE MAB
MSE MAB
MSI MAE
MSE MAI
MSU MAB
MSU MAE
ETTbi
0,303
0213
0.0800.318
0.044
0,310
0.020.221
0.24
00999
0.170
0.111 0,257
0252
0,199 0.377
0.329
0.198.946
| 9500
0.201
021550973
0.206
0:363:04000
ET1ml
0,07
0,159
0.14
0.050..122
0.053
0.122
0004 (U1S8
0.086
0.099 0.255
0061 (1221
0,281 0.441
ETTu2
0L163 0.163
0.128 0.68
0.120.0.258
0.121 0:258
0.121
0.258
| 0.112 0:288
0.119.0.56
0.133
0.1 0,262
0.130:0271
| 0.175 0.320
Table 12: Full results of loog-range time series forecasting under univariate settings.
Medii
Als MSRipe
Thanchoee 20030)
MSSBpee (2004)
(7074)
(3003)
DC Diiver (2633)
Consoome (2073)
Pyrforrer (3077)
FERSS ner
(3002)
Astrrmee
(2071)
Irformer
(2007)
Merio
MSE MAT
MSE MAE
MSE MAS
MSE MAE
MSE MAB
MSE MAR
MSE MAS
MSR MAG
MSE MAR
MSS MAB
MSR MAK
06
8J1T 173
0259 0185
0505T 191
0.8601 180
0,266 0.266
0.299 0.277
010000 215
0,71 0.206
0.195-03 373
0.055
356
8.006
01223
0.0001236
257
05540332
0.44
98 0.500
01190 270
0.258
720
0160-02 225
10099 EL256
14544445
1L883 11.277
13 3.30 14350
0.13943355
41:25 01367
11304 11344
0.162.0.00 2009
01:36 01365
0.135 13333-2203 385
46
8.116-8.36
11300
01 033041376
0.3330375
43:25 01775
0:12602211
0.115-331 313
ETM2
195
8.168-11
345
1383 16331
0.17643335
11:87 01334
12:02 11753
0.30002300
01.344 01351
0
720
0.22
11555
11250 01404
12224 11435
0.2360 420
01/00)
0.201 436
45
8307 0.118
01244 EL122
000
11.00
11035 61.15
11:27 (1:61
0405.0 140
11256 ELUST
101090122 211
37Tml
356
190
8185
8108-1345
1005 EL162
0229 ELESA
D40 152
11:4511330 550
1L003
11005 (L18)
11302 11557
15305 11312
0156.0 116
016110311
11270 1L214
06011 0.316
01337
01151 310
3TTM2
556
192
8.80
0.314-1540
11008 0.304
0141 02200
1235 02214
0 13339224
83531337
0 (2200
11:55 1L252
03.43 02255
211
[1:14 0125)
IL 0L903
0.112.022 263
0.1500 315
720
0.1560.310
0.990 0.33
0.175 0.334
0.183
0.399
0.39 0.355
0.15503355
0393
0.30010435
Short-Range Time Series Forecasting Under Multivariate Settings. Table K summarizes the results of short-range time series forecasting under multivariate settings, where the results of baselines
17

---

without are cited from ITTransfrmeer (2I]. We can see from Table L3 that Ada-MSHyper achieves the SOTA results on all datasets. Specifically, ada-Me gives an average error reduction of 10.38% and 3.82% compared to the best baseline in terms of MSE and MAE, respectively.
Table 13: Full results of short-range time series forecasting under multivariate settings.
Models
(Oum))
Trassforaer pettyper (2004)
pS (2004)
TimeMliver (3004)
PachTST
(2025)
TimesNet (2003)
Dliteer (2021)
Crosforne (2003)
SCINet
(2022)
PED
(2002)
Autodome: (2021)
Metric
MISE MAR
MISE MAR
MSE MAR
MSR MAN
MNF MAB
MSR MAH
MSSI MAR
MSR STAR
MSE MAR
MSB MAR
MSR MAR:
0L060 IL165
0.073 0.174
0.106.0:00 0:30
161
0.0990.116
0.55
0.22 0.243
0.000
0.172
125 0.251
0.372.0.33
PUMSOS
0.129-0:117
0.118
0.317 0.33
U111
0.120 0.230
0.236
0.365
0.211
0.3
0.255
0.345
1,022 0.792
12
0J068 EL173
0.078 0.183
0.103.0.190
166 0.34
0.234
0.057 0.105
0.145 0.272
0.098 0.218
0 0,172
0. 0.138 0.262
0.424
PEMS04
24
0J80 1.189
0 0.305
0.148 0.245
183 0.363
0.153:0:275
11008 0.215
0.224 0.540
11131 0.256
0:064 1.195
177 0.243
0.5590.510
48
0.093 0.204
0.233
0.191 0.308
0. 199
0.385
0.229 0.359
0.136 250
0.555 0.437
0.205 0.326
0.099
0:211
0.270
0.368
0.646...100
0.055 0.154
0.165
0.137 0.256
0.153
0.115 0.342
0
0.068 61171
0.309
0,199
PEM907
0,214
0011
196 0.354
0.340
0,258 0.458
0 10330
0.257
0.165 0.255
0,470
12
0.063 0J165
0,009 DL1R2
0.313 0.209
0. 1626200 0.337
0.168 0.252
0.112 0.212
0.154 0.276
0.165 0.214
0.067 0.154
0.2730.273
0.436
PEMS08
24
0L109 0.239
OCTES R.21
0.259:03448
181 B
0:224 0:361
[1:41 0.226
01348 0.553
0.215.0.5 0.360
0.132-.221
02100 50
0-467 8.50
0.159 01235
OLBE 0.235
0.317
0.224 0:422
0.321 0.354
0.195 0.253
0.440
0.315
0,189
0.399
0,566.0.73
Sarra-Long-Range Time Series Forecasting Under Multivariate Settings. We conduct ultra-long- range time series forecasting by taking fixed input length (T 96) to predict ultra-long lengths (H = {1080, 1440, 1800, 21 2160)). We run all results by ourselves. Table [ summarries the results of ultra-long-range time series forecasting under multivariate settings, where indicates that the method fails to produce any results on that prediction length due to the -ut-of-memory problems. We can see from Table 14 that Ada-MSHyper achieves SOTA results almost all datasets. Specifically, Ada-MSHyper gives an average error reduction of 4.97% and 2.21% 11 compared to the best baseline in terms of MSE and MAE, respectively.
Table 14: Full results of ultra-long-range time series forecasting results under multivariate settings.
por
WITRANY
Pacils*
Dixea
Createran
TEDteraa
Pytaterac
Austmmee
M45
MAG
MSE
M3E
MBE
MAG
MB5
MAB
MBE
1882
1,59
0557
06600555 554
0.551
04772 324
2415
1 0.766
3:684 6626
ETTM
1440
1882
02500365
BU2H0 555
0258
0607 0557
03770843
0275
677
RASSS
666 B
0,55
0,501 159
03651 176
05511 5.67
2049
1
1.075 3333
0552
3676 BJG6
1960
0:986
116
0:850.10 180
0:550 134
0.953
1.514 0/0
5.551 618
3,546 547
ETT12
1882
1440
8.65
(L66-1333
05130 527
03222
D5532.00 506
02000344
05570555
03557 0.00
(1540
3.009 485
1991 011
0.545 2.04
0757 2-16
3.96 1.15
2.775
1060
0.53.0443
05535 155
155
154
336
05115
3:55 01551
2TTml
1802
1642
0371192
455563455
115220.417
LU225-048 485
11563-1402 407
0.500 EA67
8-434
0.534 01491
0L507
23366 340
4.115 822
01314 2.40
01511 2.04
1 2.12
11036 2.34
2:44
3,002 ELSAD
0.969
5.556
1060
0.3500444
042901407 402
0.41594434
0.422
0,519
3.901 236
01501 0.466
3:523 0L000
KTTWW
1802
1442
8,05 11452
0 682
0.68/1453
1U475-2445 449
444
0u86-0 402
[u47
Dult2 13442
01456
01475 0.43
0/221 6682
0.660 (L9K3
23446 344
4.1031.85 822
01:77 3474
01405 2480
-.4293.700 7.708
3:33 EL-00
0.00
F Ablation Studies
To investigate the performance of Ada-MSHyper on longer prediction lengths, we compare the forecasting results of Ada-MSHyper with those of six variations (i.c., AGL, one, PH, -w/o NC. -wo HC, and -w/o NHC) on ETThl datasct. The experimental results are shown in Table [5] We can observe that for longer prediction lengths, -wo NC has smaller performance degradation than other variations. The reason may be that when the prediction length increases, the model tends to focus more on macroscopic variation interactions and diminishes its emphasis fine- ane-grained node constraint. In addition, Ada Ada-MSHyppr performs better than other six variations even with longer prediction length, showing the effectiveness of our AHL module and NHC mechanism.
Table 15: Results of different adaptive hypergraph learning methods and constraint mechanisms.
Variation
AGL
ose
PH
.wlo NC
-wl HC
wloo NHC.
Ado MSHiyper
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MSE
MAE
MSE
MAE
0,685
0.679
0.640
0.591
0.539
0.515
0.574
0.525
0.534
0.509
0.855
0.857
0.673
0.503
0.679
0.734
0.498
To investigate the impact of node and hypergraph constraints mechanism on the adaptive hypergraph learning (AHL) module, we design two variants: (1) Removing the NHC mechanism (-w/o NHC).
18

---

(2) Only optimizing the hypergraph learning module (-OH). We illustrate these two variants for better
understanding Figure
Hud
Halp
Feature Extr.
Feature Extr.
Constrain Loss
MSE Loss
Loss Backward
Loss Backward
Lass Bockward
-iio-NHC
-OHI
Ada-MSHyyper
Figure 5: Different optimization strategies for AHL.
To investigate the impact of the multi-scale feature extraction (MFE) module, we design two variants: (1) Replacing the aggregation function in the MFE module with average pooling (-avg). (2) Replacing the aggregation function in the MFE module with max pooling (-max).
The results for the four variants are shown in Table [6] We can observe that: (1) -w/o NHC gets the worst results. This may be because lacking the constraints makes the model fail to capture implicit semantic features of the clustered nodes and learned hyperedges. (2) Compared to -OH, Ada-MSHyper yields slightly better results. The reason may be that the NHC mechanism facilitates the MFE model to more effectively aggregate similar features during multi-scale feature extraction and reduce the interference of noise. (3) -avg and -max get relatively worse performance. This may be because the lack of parameters leads to the reduction of the representative ability of Ada-MSHyper.
Table 16: Results of different AHL, MFE and multi-scale interaction methods.
Methods
-wla NIIC
-011
-2rg
stalt
Ada-MSillper
Matric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.393
0.422
0.379
0.397
0.372
0.413
0.382
0.408
0.418
0.419
0,372
0.393
0.459
0.423
0.439
0.429
0.440
0.426
0.483
0.454
0.422
0.433
0.487
0.457
0.507
To investigate the effectiveness of hypergraph convolution attention, we design one variant: Replacing the hypergraph convolution attention with the attention mechanism used in the inter-scale interaction module to update node features (-r/att). The experimental results on ETTh1 dataset are shown in Table [6 We can observe that Ada-MSHyper performs better than -r/ att, which demonstrates the effectiveness of the hyperedge convolution attention used in the intra-scale interaction module.
G Parameter Studies
We perform parameter studies to measure the impact of the threshold 77, which influences the effectiveness of the sparsity strategy. The experimental results ETTh1 dataset are shown in Table I7 We can see that the best performance can obtained when 77 is 3. The reason is that a small may filter out useful information and large y would introduce noise interference.
Table 17: Results of Ada- MdaaMer with different 17.
Uyparrmettrr
gel
n-2
1-3
7-4
0-5
Meuic
MSE
MAE
MSC
MAB
MSE
MAG
MSE
MAD
MSE
MAB
96
0.407
0.415
0.390
0.397
0.372
0.393
0.387
0.396
0.419
0.418
336
0.547
0.500
0.476
0.43
0.422
0.433
0.438
0.435
0.560
0.510
720
0440
0.463
0.476
0.465
0.445
0.459
0.460
0.459
0.473
0.474
H
Visualization
Visualization of Node Constraint. As shown in Figure 6a each time step denoted node of the hypergraph at the finest scale. We categorize the nodes into four groups based the node values of
19

---

500
500
Large
400
400
400
300
nolse
300
300
200
200
100
20
40
60
80
100
20
40
60
80
100
Node Indes
Node Index
Nade Index
(a) Input sequence
(h) Ada-MSHyper-w/6 NHC
(c) -dy-MSSyper-w/o NC.
500
400-
Valber
200
200
100
100
100
100
Node
(d)
Ada-MSHyper-wio HC
(c) Ada-MSHyper
Figure 6: Visualization the node constrain effect Electricity dataset.
original inputs, and draw them using different colors. For target node, nodes of the same color may be regarded as those sharing similar semantic information with the target node, while nodes of other colors may be regarded noise. Then, we draw the nodes related to the target node based on the incidence matrix of the learned hypergraph in the black color.
We random select samples at the same time step from three variants, i.e., without node constraint (-w/o NC), without hyperedge constraint (-w/o HC), and without node and hyperedges constraints (-w/o NHC), and plot these three samples with samples from original inputs and Ada-MHyper.
We observe that: (1) In Figure 6b and Figure the related nodes of the target node are almost neighboring nodes. However, some noise, plotted as orange, is included as well. The reason may be that -w/o NHC and -w/o NC cannot distinguish noise information without node constraint, i.c., cannot consider the semantic similarity to cluster nodes. (2) In Figure (6d and Figure [6c| since -w/o HC and the proposed Ada- MSHyper have node constraint, both of them can cluster neighboring and distant but strongly correlated nodes, and they can also mitigate the interference of noise, indicating the effectiveness of node constraint.
Visualization of Hyperedge Constraint. We use the samples which are used in the visualization of node constraint. We visualize the sequentially connecting nodes that belong the same hyperedges whose indices are {4, 8, 12}. Figure 7 shows three types of temporal variations learned by -wfo NC, -w/o HC, -w/o NHC, and Ada-MSHyper, respectively. We can observe that: (1) -w/o HC and -w/o NHC exhibit irregular temporal variations in comparison to -w/o NC and Ada-MSHyper. The reason may be that without the hyperedge constraint, these methods are unable adequately differentiate temporal variations entangled in temporal patterns. (2) Compared to wwo NC, Ada- Ada-MSHyper exhibits relatively simple temporal variations. The reason may be that influenced by node constraint, the temporal variations extracted by Ada-MSHyper contain less noise.
Adb-Miilyperrwll NIIL
Ad-MSllypennis
MoMMMiityer-sio NC
Ads-MtSllyper
Figure 7: Different temporal variations learned by different methods.
20

---

We also matched the temporal variations extracted by Ada AdH-Mrryper to the sample sequences. As shown Figure 8b] we can observe that these variations can represent inherent changes. We speculate that by introducing hyperedge constraint, the model will treat temporal variations with different shapes as distinct positive and negative examples. addition, the differentiated temporal variations are like kind of Shapelet, akin to those used in NLP and CV, enabling better representation of temporal patterns within time series.
006-
130
190
100
Nade Indks
Nede
(a) Input sequence
(b) Ada-MSHypor
Figure 8: Visualization the hyperedge constraint effect on Electricity dataset.
I Limitations and Future Works
the future, we will extend our work in the following directions. Firstly, due to our NHC mechanism can cluster nodes with similar semantic information and differentiate temporal variations within each scales, It is interesting to correlate the inherent temporal variations with corpora used in natural language processing, and leverage large language models to investigate deeper correlations between corpora and TS data. Secondly, compared to natural language processing and computer vision, time series analysis has access to fewer datasets, which may limit the expressive power of the models. Therefore, in the future, we plan to compile larger datasets to validate the generalization capabilities of our models more extensive data.
J Broader Impacts
In this paper, we propose Ada-MSHyper for time series forecasting, Extensive experimental results demonstrate the effectiveness of Ada-MSHyper. Our paper mainly focuses on scientific research and has no obvious negative social impact.
21

---

1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer:
[Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the contributions and scope of the paper (see Abstract and Introduction)
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of our work in Appendix[]
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairess..
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers penalize honesty concerning limitations.
will be specifically instructed
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
22

---

The paper provides corresponding experimental validation in Section5.2 and AppendixE] provides ablation studies in Section [..3 and Appendix F and provides visualization analysis in Appendix H to support the claimed capabilities of the model.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have provided the details regarding computational platforms, dataset descriptions, network architectures, hyper-parameter settings, and the training process of our method in Section B in the main paper and Appendix [ and D In addition, we provide source codes on anonymous Github as stated in the abstract.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
23

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: Our codes released at anonymous Github stated in the abstract. The down- load links of the public datasets are provided the project homepage and pre-processing functions included in the codes. The
settings given in Appendix D
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
htap://eguuides//deeeuubmii. wtt.:/uwmmsssiooooooooooooo. for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
ctipss.cc/pullic/uuiii gwwiis/gguee//eeeeeoeooi.ocsssee CodeeSummiee onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have provided the details regarding computational platforms, dataset descriptions, network architectures, syper-parameter settings, and the training process of our method in Section in the main paper and Appendix and D
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We repeat all experiments 3 times and use the mean of the metrics as the final results as illustrated in Appedix D
Guidelines:
The answer NA means that the paper does not include experiments.

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errоrs is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have stated the experimental platforms (i.c.. GPUs and CPUs) and software (the version of Pytorch) used in our paper in Section [5.]] and Appendix D and have analyzed the computational efficiency in Appendix ??.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported didn't it make it into the paper).
paper (e.g., preliminary or failed experiments that NeurlPS Code of Ethics https: httpr//s....cbb.accacccccceeeaea
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the
Answer: [Yes]
Justification: anonymity.
We have reviewed the NeurIPS Code of Ethics and we have ensured to preserve
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
25

---

Justification: We discuss both potential positive societal impacts and negative societal impacts of our work in Appendix
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses such risks, as we focus on theoretical analysis and general research areas, and conduct our experiments on public datasets.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We make sure to cite the original papers (or URLs) of the code packages or datasets that are used in our paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
26

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperssi thoode com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New Assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification: Our codes are provided on the project homepage at anonymous Github.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the datasct/cod//model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
No crowdsourcing nor research with human subjects is involved.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No crowdsourcing research with human subjects is involved.
Guidelines:
27

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "7eWwmM8Diy", "reviewer": "Reviewer_Y3pA", "summary": "This paper introduces a hypergraph-based multi-scale time series forecasting model. By treating multi-scale feature representations as nodes, the proposed AHL module automatically generates incidence matrices to model implicit group-wise node interactions at different scales. Node constraints and hyperedge constraints are introduced to effectively aggregate similar semantic nodes and distinguish temporal variations at different scales. The experimental results confirm the predictive ability of the model, and it can still maintain low MSE and MAE on the ultra-long range series.", "questions": "1. The paper claims that most of the experimental results for long-range prediction come from the DLinear model. However, there are no experimental results of the Crossformer model in the DLinear paper. Moreover, the results presented in this paper differ significantly from those in the ITransformer paper for certain datasets. It would be helpful to have an explanation for these differences.\n\n2. In Section 4.1, the paper introduces multi-scale sequence construction, while Section 4.2 discusses the AHL module. One aspect that raises curiosity is the process of mapping from the sequence to the hypergraph. Specifically, is each time point with multiple features in the time series mapped to a node in the hypergraph?\n\n3. Regarding model efficiency, the paper provides a comparison of model parameters and training time on a specific dataset. However, a single experiment result may not be sufficient to establish convincing evidence. It would be beneficial to explain the model's efficiency from the perspective of theoretical complexity or provide information on model parameters and training time for longer input sequences and on additional datasets.\n\n4. In the ablation experiment section, it would be beneficial to include experiments with longer prediction lengths to further evaluate the model's performance.", "limitations": "The authors adequately addressed the limitations", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The authors propose the AHL module, which is applied to a hypergraph constructed from time series, enabling the discovery of pattern interactions at various scales.\n\n2. The experimental results demonstrate that the model has achieved state-of-the-art (SOTA) performance in short, long, and ultra-long sequence prediction.\n\n3. The paper exhibits a clear structure and substantial content. It provides a detailed introduction to the structure and function of each module in the model.", "weaknesses": "1. In the ablation experiment, only ETTh1 dataset was utilized to conduct experiments at three prediction lengths: {96, 326, 720}. This setup lacks support for evaluating ultra-long-range performance. For instance, in the case of w/o NC (without Noise Conditioning), the performance difference between the proposed model and its counterpart becomes only marginally noticeable as the sequence length increases. Since there is a lack of additional datasets and ultra-long-range experiments, it is reasonable to interpret this result as a normal experimental error rather than a significant performance difference.\n\n2. The experiments conducted to assess model efficiency are too simplistic to effectively validate the author's viewpoint.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Comment:\n\nMany thanks to Reviewer Y3pA for providing the insightful reviews and comments.\n\n**Q1**: About some different long-range prediction results from those in the DLinear and iTransformer paper.\n\nThanks for your careful check and scientific rigor. As for the long-range time series forecasting, we have two kinds of settings, i.e., **multivariate settings** and **univariate settings**. Due to some methods (e.g., iTransformer and Crossformer) do not have results under univariate settings, we use their official codes and fine-tune their key hyperparameters. We have added an explanation about the Crossformer results in the $\\underline{\\text{revised paper}}$. As for the multivariate settings, some newly added baselines are rerun by us and other results are from iTransformer. We have carefully rechecked the experimental results and addressed the aforementioned issues in the $\\underline{\\text{revised paper}}$. See **Table 1** in $\\underline{\\text{global response}}$.\n\n**Q2**: Is each time point with multiple features in the time series mapped to a node in the hypergraph?\n\nYes, to be precise, Ada-MSHyper maps the input sequence into subsequences at different scales. The features of these subsequences are then treated as nodes in the hypergraph.\n\n**Q3**: It would be beneficial to explain the model's efficiency from theoretical complexity or provide results on longer input sequence and additional datasets.\n\nTo provide a more comprehensive evaluation of the model's efficiency, we have included the **theoretical complexity analysis** and added additional **computation cost results** for longer input sequence and additional datasets. The results are shown as follows, which has also been included in the $\\underline{\\text{revised paper}}$.\n\n**Theoretical complexity analysis**: For the MFE module, the time complexity is $\\mathcal{O}(Nl)$, where $N$ is the number of nodes at the finest scale and $N$ is equal to the input length $T$. $l$ is the aggregation window size at the finest scale. For the AHL module, the time complexity is $\\mathcal{O}(MN+M^2)$, where $M$ is the number of hypergraphs at the finest scale. For the intra-scale interaction module, since $\\mathbf{D}_v$ and $\\mathbf{D}_e$ are diagonal matrices, the time complexity is $\\mathcal{O}(MN)$. For the inter-scale interaction module, the time complexity is $M^2$. In practical operation, $M$ and $l$ is the hyperparameter and is much smaller than $N$. As a result, the total time complexity is bounded by $\\mathcal{O}(N)$.\n\n**Computation cost results**: We have newly added computation cost on Traffic and Weather datasets with the 720-96 input-output length. The experimental results are shown in Table 1 and Table 2, from which we can observe that:\n\n* Although Ada-MSHyper has a large number of parameters, it achieves **lower training time and lower GPU occupation** due to the matrix sparsity strategy in the model and the optimization of hypergraph computation provided by torch\\_geometry in PyTorch.\n  \n* **Ada-MSHyper can maintain better performance even with longer input length**. Considering the forecasting performance and computation cost, Ada-MSHyper demonstrates its superiority over existing methods.\n  \n\nTable 1.\n\n| Methods | Input-Output | Dataset | Training Time/epochs | Parameters | GPU Occupation | MSE results |\n| --- | --- | --- | --- | --- | --- | --- |\n| Ada-MSHyper | 720-96 | Weather | **2.273s** | 5,684,229 | **1,249MB** | **0.149** |\n| iTransformer | 720-96 | Weather | 2.482s | 5,153,376 | 1,538MB | 0.180 |\n| PatchTST | 720-96 | Weather | 11.546s | **1,517,152** | 14,100MB | 0.152 |\n\nTable 2.\n\n| Methods | Input-Output | Dataset | Training Time/epochs | Parameters | GPU Occupation | MSE results |\n| --- | --- | --- | --- | --- | --- | --- |\n| Ada-MSHyper | 720-96 | Traffic | **10.093s** | 19,575,100 | **6,154MB** | **0.342** |\n| iTransformer | 720-96 | Traffic | 12.352s | **18,490,786** | 7,113MB | 0.348 |\n| PatchTST | 720-96 | Traffic | —   | —   | —   | —   |\n\n**Q4**: In the ablation experiment section, Include longer prediction lengths to further evaluate the model's performance.\n\nWe have added additional ablation studies on ETTh1 dataset to verify the performance of Ada-MSHyper on longer prediction lengths. The results are shown in Table 3, which has also been included in the $\\underline{\\text{revised paper}}$. From Table 3 we have the additional observation:\n\n* For longer prediction length, **-w/o NC has smaller performance degradation than other variations.** The reason may be that when the prediction length increases, the model tends to focus more on macroscopic variation interactions and diminishes its emphasis on fine-grained node constraint.\n* **Ada-MSHyper performs better than -w/o NC and -w/o HC even with longer prediction length**, showing the effectiveness of node constraint and hyperedge constraint, respectively.\n\nTable 3.\n\n| Variation | AGL | one | PH  | -w/o NC | -w/o HC | -w/o NHC | Ada-MSHyper |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Metric | MSE MAE | MSE MAE | MSE MAE | MSE MAE | MSE MAE | MSE MAE | MSE MAE |\n| 1080 | --  -- | 0.685 0.679 | 0.640 0.591 | 0.539 0.515 | 0.574 0.516 | 0.597 0.525 | **0.534 0.509** |\n| 1440 | --  -- | 0.855 0.857 | 0.783 0.673 | 0.621 0.503 | 0.679 0.568 | 0.734 0.585 | **0.616 0.498** |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "04P6IITRRX", "reviewer": "Reviewer_HQ9E", "summary": "This paper presents a Time Series Forecasting method Ada-MSHyper that uses a hyper graph to capture the group-wise interactions at different time scales rather than Point-Wise interaction. Experiments are performed on 8 data sets and the proposed method is compared with SOTA methods", "questions": "1. Why SOTA methods do not include Graph-Transformer methods e.g. STGNN?\n2. Will this process of \"reduce subsequent computational costs and noise interference\", introduce any loss of usefull information?\n3. Have you studied and quantified the computational cost reduction by the above approach?\n4. Is the linear layer for forecasting a linear regression?\n5. Will the proposed method work on Stock Market Data?\n6. Have you analyzed the data sets to check and quantified the presence of \"multi-scale pattern interactions\"?", "limitations": "No Applicable", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The use of Hyper Graph\n2. Differentiation of variation at Each Time Scale\n3. Comparisons with SOTA Methods\n4. Ablation Study\n5. Comparisons of Computational Cost with 3 SOTA", "weaknesses": "1. Related works does not include Graph-Transformer methods e.g. STGNN\n2. Graph-Transformer methods e.g. STGNN are not used as SOTA for comparisons.\n3. Data Sets should include financial data e.g. Stock Market\n4. In results of \"Ultra-Long-Range Forecasting\", the reason for comparebale accuracies of \"WITRAN\" on ETTm2\n5. The ablation study should include the effect of \"η is the threshold of T op K function\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Comment:\n\nMany thanks to Reviewer HQ9E for providing the insightful reviews and comments.\n\n**Q1**: Graph-Transformer methods should be included in Related Works and used for comparisons.\n\nThanks for your valuable suggestions and scientific rigor. We have added two latest Graph-Transformer methods, i.e., MSGNet (AAAI 2024) and CrossGNN (NeurIPS 2023) for comparison. The descriptions of these methods and their long-range time series forecasting results are shown as follows, which have also been included in the $\\underline{\\text{revised paper}}$.\n\n* **MSGNet**: MSGNet leverages frequency domain analysis to extract periodic patterns and combines an attention mechanism with adaptive graph convolution to capture multi-scale pattern interactions.\n* **CrossGNN**: CrossGNN uses an adaptive multi-scale identifier to construct multi-scale representations and utilize a cross-scale GNN to capture multi-scale pattern interactions.\n\nThe long-range time series forecasting results under multivariate settings are shown in Table 1 of the $\\underline{\\text{global response}}$ and the following tendencies can be discerned:\n\n* MSGNet and CorssGNN are the state-of-the-art graph learning methods that use graph learning modules to capture multi-scale pattern interactions. However, **they can only capture pair-wise interactions instead of group-wise interactions and get worse performance than Ada-MSHyper in most cases.**\n\n**Q2**: Will the sparsity strategy introduce any loss of useful information?\n\nThe sparsity strategy is employed to reduce subsequent computation costs and noise interference. However, the effectiveness of the sparsity strategy is influenced by the hyperparameter $\\eta$. **When $\\eta$ is set to a smaller value, some useful information may be filtered out.** We have performed parameter studies to measure the impact of $\\eta$. The results are shown in Table 2, which has also been included in the $\\underline{\\text{revised paper}}$. From Table 2 we have the additional observation:\n\n* The best performance can be obtained when $\\eta$=3. The reason may be that **a small $\\eta$ may filter out useful information and a large $\\eta$ would introduce noise interference.**\n\nTable 2.\n\n| Hyparameter | $\\eta$=1 | $\\eta$=2 | $\\eta$=3 | $\\eta$=4 | $\\eta$=5 |\n| --- | --- | --- | --- | --- | --- |\n| Metric | MSE MAE | MSE MAE | MSE MAE | MSE MAE | MSE MAE |\n| 96  | 0.407 0.415 | 0.390 0.397 | **0.372 0.393** | 0.387 0.396 | 0.419 0.418 |\n| 336 | 0.547 0.500 | 0.476 0.443 | **0.422 0.433** | 0.438 0.435 | 0.560 0.510 |\n| 720 | 0.450 0.463 | 0.476 0.465 | **0.445 0.459** | 0.460 0.459 | 0.473 0.474 |\n\n**Q3**: Study and quantify the computational cost reduction by the above approach.\n\nTo investigate the effectiveness of the sparsity strategy, we conduct ablation studies by carefully designing the following variant:\n\n* -w/o SO: It removes the sparsity strategy in the AHL module.\n\nWe compare the computation cost on Electricity dataset under 96-96 input-output settings, the experimental results are shown in Table 3, which has also been included in the $\\underline{\\text{revised paper.}}$\n\nTable 3.\n\n| Methods | Training Time/epochs | Parameters | GPU Occupation | MSE results |\n| --- | --- | --- | --- | --- |\n| -w/o SO | 9.525s | 14,519,292 | 8,454MB | 0.392 |\n| Ada-MSHyper | **6.499s** | **8,965,392** | **6,542MB** | **0.384** |\n\nFrom Table 3 we can observe that Ada-MSHyper can achieve better performance with faster speed and lower GPU occupation compared to -w/o SO, which demonstrates the effectiveness of the sparsity strategy.\n\n**Q4**: Is the linear layer for forecasting a linear regression?\n\nYes, the linear layer used for forecasting can be considered as a form of linear regression as it maps the updated multi-scale features to the final predictions through a linear relationship.\n\n**Q5**: Datasets should include financial data, e.g., Stock Market.\n\nWe have added Nasdaq 100 Stock dataset for comparison. The detailed descriptions of the public dataset are shown as follows:\n\n* Nasdaq: This dataset includes the stock prices of 82 major corporations, which are sampled 390 times every day from July 2016 to December 2016.\n\nThe full long-range time series forecasting results on Nasdaq dataset will be included in the $\\underline{\\text{revised paper}}$. Due to time and space limitations, we list the comparison results between Ada-MSHyper and three latest baselines. The experimental results are shown as follows:\n\nTable 4.\n\n| Methods | Ada-MSHyper | iTransformer | MSHyper | TimeMixer |\n| --- | --- | --- | --- | --- |\n| Metric | MSE MAE | MSE MAE | MSE MAE | MSE MAE |\n| 96  | **0.027 0.090** | 0.057 0.141 | 0.034 0.102 | **0.027** 0.094 |\n| 192 | **0.054 0.131** | 0.095 0.183 | 0.076 0.153 | 0.059 0.137 |\n| 336 | **0.091 0.179** | 0.153 0.237 | 0.111 0.199 | 0.092 0.182 |\n| 720 | **0.182 0.268** | 0.315 0.349 | 0.257 0.310 | 0.184 0.270 |\n\nFrom Table 4 we can observe that **Ada-MSHyper achieves the best performance in almost all cases.** The experimental results demonstrate the effectiveness of Ada-MSHyper on stock market dataset.\n\n**Q6**: Analyze the datasets to check and quantify the presence of \"multi-scale pattern interactions\".\n\nWe have added weight visualization results, see **Figure 2** in $\\underline{\\text{global response}}$, the detailed analysis has been included in the $\\underline{\\text{revised paper}}$.\n\n**Q7**: The reason for comparable accuracies of WITRAN on ETTm2 dataset.\n\n**WITRAN employs a Recurrent Acceleration Network within its framework.** For ETTm2 dataset (high forecastability), it can make effective predictions with fewer parameters and a shallower network structure. However, recurrent structure may face underfitting risks when handling more challenging datasets (low forecastability), e.g., ETTh1 and ETTh2. This is why WITRAN achieves comparable accuracies on ETTm2 but performs worse on other datasets (e.g., ETTh1 and ETTh2).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kXEZQYxj5n", "reviewer": "Reviewer_HJMp", "summary": "(1) Design an AHL module to model the abundant and implicit group-wise node interactions and a multi-scale interaction module to model group-wise pattern interactions at different scales.\n(2)Introduce a NHC mechanism to cluster nodes with similar semantic information and differentiate the temporal variations within each scales.", "questions": "(1) In your Hyperedge Constraint, you use both cosine similarity α and Euclidean distance D, are your Euclidean distance normalized? It seems that cosine similarity and normalized Euclidean distance are similar, so will it provide too much redundant information to use αD as your hyperedge loss? Can you do some ablation studies?\n(2) I am kind of confused about your Multi-Scale Interaction Module, why your Intra-Scale Interaction part use HGNN for message passing, and your Inter-Scale Interaction Module part use transformer for updating features, is there any reason for this? Cause both interaction part are using attention, can they just use the same method for updating features?\n(3) Note that you use lots of attention mechanism to model group-wise pattern interactions, you use the cosine similarity and the Euclidean distance, both are known to us well. I wonder whether there are some other distance for modeling the pattern interactions we can try? Like Wasserstein distance.", "limitations": "Authors addressed the limitations in appendix I that their datasets are not really large, so the generalization capabilities of their models may not be really good. The paper mainly focuses on scientific research and has no obvious negative social impact.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The first work that incorporates adaptive hypergraph modeling into time series forecasting;\n(2) Design AHL module to solve semantic information sparsity, and NHC mechanism to solve temporal variations entanglement, it is interesting;\n(3) Achieve state-of-the-art (SOTA) performance.\n(4) Lots of experiments and ablation studies, and visualizations to demonstrate the effectiveness of the proposed methodology.", "weaknesses": "(1) The pipeline is kind of complicated, and the color of it can be improved;\n(2) Some expression in the paper can be more formal, like “entries hnm” can be replace by “entries Hnm” for more formal.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Comment:\n\nMany thanks to Reviewer HJMp for providing the insightful reviews and comments.\n\n**Q1**: The pipeline can be improved and some expressions in the paper can be more formal.\n\nThanks for your valuable suggestions and scientific rigor. We have improved the color of the pipeline, see **Figure 1** in $\\underline{\\text{global response}}$. In addition, we have performed thorough proofreading and used more formal expressions.\n\n**Q2**: Are your Euclidean distance normalized? Do some ablation studies to verify whether using $\\alpha D$ as hyperedge loss will introduce redundant information.\n\n**We do not normalize the Euclidean distance $D$.** The cosine similarity $\\alpha$ and normalized Euclidean distance are indeed similar as both of them compare **the direction** of two vectors in feature space, reducing the impact of feature scale differences. However, relying solely on cosine similarity **neglects differences in hyperedge representations regarding relative scale and distance** (i.e., **the magnitude** of two vectors). For instance, one hyperedge connects group-wise nodes with larger values indicating a \"peak variation\", while another hyperedge connects group-wise nodes with smaller values indicating a \"trough variation\". After normalization, the differences between hyperedge representations would become less noticeable. To address this, we add Euclidean distance and use $\\alpha D$ as hyperedge loss.\n\nTo investigate the effectiveness of the hyperedge loss, we conduct ablation studies by carefully designing the following three variants:\n\n* -w/o $\\alpha$: It removes cosine similarity and uses Euclidean distance as the hyperedge loss.\n  \n* -w/o $D$: It removes Euclidean distance and uses cosine similarity as the hyperedge loss.\n  \n* -Wass: It uses Wasserstein distance as the hyperedge loss.\n  \n\nThe experimental results on ETTh1 dataset are shown in Table 1, which has also been included in the $\\underline{\\text{revised paper}}$. From Table 1 we can observe that: (1) -w/o $\\alpha$ and -w/o $D$ perform worse than Ada-MShyper, showing **the effectiveness of Euclidean distance and cosine similarity in hyperedge loss, respectively**. (2) Ada-MSHyper performs better than -Wass, which demonstrates **the effectiveness of the hyperedge loss in the adaptive hypergraph learning module.**\n\nTable 1.\n\n| Variation | -w/o $ \\alpha $ | -w/o $D$ | -Wass | Ada-MSHyper |\n| --- | --- | --- | --- | --- |\n| Metric | MSE MAE | MSE MAE | MSE MAE | MSE MAE |\n| 96  | 0.400 0.415 | 0.406 0.414 | 0.405 0.421 | **0.372 0.393** |\n| 336 | 0.494 0.460 | 0.457 0.440 | 0.482 0.447 | **0.422 0.433** |\n| 720 | 0.525 0.495 | 0.536 0.502 | 0.492 0.479 | **0.445 0.459** |\n\n**Q3**: Explain the reasons for the different designs of the intra-scale and inter-scale modules. Can they just use the same method for updating features?\n\nThe intra-scale interaction module is used to capture detailed group-wise interactions between nodes with similar semantic information, while the inter-scale interaction module focuses on capturing the interactions of macroscopic variations through hyperedges. If both modules use hypergraph convolution attention for updating features, there are two limitations for the inter-scale interaction modules: (1) **There is no hypergraph structure regarding the relationships between hyperedges.** (2) Hyperedges already represent group-wise interactions by connecting multiple nodes. Modeling group-wise hyperedges interactions through the hypergraph structure may **introduce redundant information and cause the overfitting problem.**\n\nIf we want to use the same method for updating features, one direct way is to use attention for both modules. However, this would cause the intra-scale interaction module **lack the ability to capture group-wise interaction between nodes with similar semantic information** and pair-wise attention would **result in $\\mathcal{O}(N^2)$ computation cost.**\n\nTo investigate the feasibility of using the same method for updating features, we conduct ablation studies by carefully designing the following variant:\n\n* -r/ att: It replaces the hypergraph convolution attention with the attention mechanism used in the inter-scale interaction module to update node features.\n\nThe experimental results on ETTh1 dataset are shown in Table 2, which have also been included in the $\\underline{\\text{revised paper}}$. From Table 2 we can observe that: Ada-MSHyper performs better than -r/ att, which demonstrates **the effectiveness of the hyperedge convolution attention used in the intra-scale interaction module.**\n\nTable 2.\n\n| Variation | -r/ att | Ada-MSHyper |\n| --- | --- | --- |\n| Metric | MSE MAE | MSE MAE |\n| 96  | 0.418 0.419 | **0.372 0.393** |\n| 336 | 0.483 0.454 | **0.422 0.433** |\n| 720 | 0.514 0.507 | **0.445 0.459** |\n\n**Q4**: Whether there are some other distances for modeling the pattern interactions we can try? Like Wasserstein distance.\n\nOther distances can also be introduced as constraint for modeling pattern interactions. As per your suggestion, we normalize the hyperedge features and use Wasserstein distance as the hyperedge loss. See response to **Q2** for detailed analysis.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5Xs3i62MSS", "reviewer": "Reviewer_VVbq", "summary": "This paper presents an adaptive hypergraph learning module for modeling group-wise multi-scale interactions to improve transformer-based model for time series data. Given a time series data, Multi-Scale Feature Extraction (MFE) Module first converts it to a hypergraph. Then, intra-scale and inter-scale learning modules are performed. Comprehensive experiments demonstrate the effectiveness of the proposed method.", "questions": "1. Considering that one of the challenges addressed by this paper is \"temporal variations entanglement,\" how do the authors compare their method with frequency domain analysis techniques? For instance, have they considered converting the time-series data into a spectrogram using Short-Time Fourier Transform? Can Ada-MSHyper easily adapt to the 2D spectrogram data in time-frequency domain?", "limitations": "More insightful analysis could be added to the experiments. See weaknesses and questions.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The motivation behind Ada-MSHyper is well-founded and insightful.\n2. The design of the proposed method effectively addresses the target challenge.\n3. Extensive experiments were carried out, demonstrating the clear superiority of Ada-MSHyper.\n4. The writing of the paper is clear and easy to follow.\n\nOverall, this is a solid paper with excellent presentation.", "weaknesses": "This paper is generally well-written. My only suggestion is to provide more insights into the performance differences of Ada-MSHyper across different datasets. Consider analyzing the reasons behind its performance on long-range, short-range, and ultra-long-range datasets. Does Ada-MSHyper perform better on one type compared to the others? What might be the reasons for this? For example, the node constraints on different dataset could be quite different. Node clustering might behave poorly on datasets with weaker temporal patterns. Answering these questions can help understand the effectiveness and limitations of Ada-MSHyper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Comment:\n\nMany thanks to Reviewer VVbq for providing the insightful reviews and comments.\n\n**Q1**: In long-range, short-range, and ultra-long-range time series forecasting, does Ada-MSHyper perform better on one type compared to the others? What might be the reasons for this?\n\nThanks for your valuable feedback and scientific rigor. Compared to long-range and ultra-long-range time series forecasting, **short-range time series forecasting has better performance** (reducing prediction errors by an average of **10.38%**). The reason may be that we use PEMS datasets in short-range time series forecasting, which are influenced by human activities and have obvious multi-scale pattern information. This also explains why Ada-MSHyper also performs better on Electricity and Traffic datasets for long-range time series forecasting (achieving the best performance on **all forecasting horizons**).\n\n**As observed by the reviewer, the cause of the above phenomenon is the design of the node constraint,** which is used to cluster nodes with similar semantic information. To investigate the impact of node constraint, we have newly added ablation studies on Electricity dataset by carefully designing the following variant:\n\n* -w/o NC: It removes the node constraint.\n\nThe experimental results are shown in Table 1, which has also been included in the $\\underline{\\text{revised paper}}$.\n\nTable 1.\n\n| Variation | -w/o NC | Ada-MSHyper |\n| --- | --- | --- |\n| Metric | MSE MAE | MSE MAE |\n| 96  | 0.169 0.245 | **0.135 0.238** |\n| 336 | 0.184 0.275 | **0.168 0.266** |\n| 720 | 0.237 0.403 | **0.212 0.293** |\n\nFrom Table 1 and the experimental results of -w/o NC in $\\underline{\\text{Section 5.3 of the original paper}}$, we have the following observation: (1) **The performance drop of -w/o NC is smaller compared to other variants (e.g., -w/o NHC) on ETTh1 dataset**, possibly because there are weaker temporal patterns on ETTh1 dataset, and on that dataset Ada-MSHyper may focus more on macroscopic variation interactions rather than detailed group-wise interactions between nodes with similar semantic information. (2) Compared to its performance on ETTh1 dataset, the performance of **-w/o NC shows a significant drop on Electricity dataset**. The reason may be that Electricity dataset has obvious multi-scale pattern information, making the NC mechanism used to cluster nodes with similar semantic information appear to be more important. (3) Ada-MSHyper still performs better than -w/o NC in almost all cases, showing the effectiveness of node constraint.\n\n**Q2**: How do the authors compare their method with frequency domain analysis techniques and can Ada-MSHyper easily adapt to the 2D spectrogram data in time-frequency domain?\n\nTo solve the problem of **temporal variations entanglement**, some frequency domain analysis methods (e.g., TimesNet [1], FiLM [2], FEDformer [3], and Autoformer [4]) adopt simplistic series decomposition with frequency domain analysis techniques to differentiate temporal variations at different scales. We have compared Ada-MSHyper with these methods and added analysis in $\\underline{\\text{Section 5.2 of the paper}}$.\n\nIn addition, Ada-MSHyper may not be directly applicable to 2D data. To investigate the performance of Ada-MSHyper in the frequency domain, **we flatten the 2D spectrogram data into 1D features** and conduct ablation studies by carefully designing the following variant:\n\n* w/ STFT: It converts the multi-scale subsequences into 2D spectrogram data using Short-Time Fourier Transform (STFT) and then flattens it into 1D feature representations before send to the AHL model.\n\nThe experimental results on ETTh1 dataset are shown in Table 2, which has also been included in the $\\underline{\\text{revised paper}}$. From Table 2 we can observe that:\n\n-w/ STFT performs worse than Ada-MSHyper. The reason may be that flattening 2D spectrogram data into 1D features will mix frequency domain features with time domain features, thereby impacting the forecasting performance of the model. However, the valuable feedback from the reviewer inspired us to consider that time series features **need no to be limited to the form of \"scalars\" or \"vectors\". The \"tensors\" (e.g., 2D spectrogram data) may offer a better representation for time series forecasting.** Modifying the model to support 2D spectrogram data instead of flattening it into 1D features may have better performance. Considering the scope of our paper, we would like to leave the exploration in the future work.\n\nTable 2.\n\n| Variation | -w/ STFT | Ada-MSHyper |\n| --- | --- | --- |\n| Metric | MSE MAE | MSE MAE |\n| 96  | 0.390 0.399 | **0.372 0.393** |\n| 336 | 0.478 0.443 | **0.422 0.433** |\n| 720 | 0.479 0.466 | **0.445 0.459** |\n\n[1] Wu H, Hu T, Liu Y, et al. TimesNet: Temporal 2d-variation modeling for general time series analysis. ICLR, 2023.\n\n[2] Zhou T, Ma Z, Wen Q, et al. FiLM: Frequency improved legendre memory model for long-term time series forecasting. NeurIPS, 2022.\n\n[3] Zhou T, Ma Z, Wen Q, et al. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. ICLR, 2022.\n\n[4] Wu H, Xu J, Wang J, et al. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. NeurIPS, 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all the reviewers for their insightful reviews and valuable comments, which are instructive for us to improve our paper further.

The reviewers generally hold positive opinions of our paper, in that they perceive our approach as **interesting**, **detailed**, and **clear**. The reviewers also acknowledge our research topic **is the first work that incorporates adaptive hypergraph modeling into time series forecasting** and the motivation **is well-founded and insightful**. In addition, the reviewers deem our paper **exhibits a clear structure and substantial content** and the experiments are **extensive**, **solid**, and **effective**.

The reviewers also raise insightful and constructive concerns. We made every effort to address all the concerns by providing sufficient evidence and requested results. Here is the summary of the major revisions:

**Add more ablation studies and model analysis (Reviewer Y3pA, HQ9E, HJMp, and VVbq):** Following the suggestions of the reviewers, we have added more than 10 ablation studies to investigate the effectiveness of the node constraint, the sparsity strategy, the cosine similarity and Euclidean distance in hyperedge loss, and the hypergraph convolution attention. We have added parameter studies to measure the impact of $\eta$. In addition, we have added more computation cost analysis for longer input sequence and additional datasets.

**Add additional baselines and datasets (Reviewer HQ9E)**: Following the suggestions of the reviewer, We have added stock market data for comparison and included two graph-transformer methods as baselines. The experimental results demonstrate the effectiveness of Ada-MSHyper over existing methods.

**Polish the writings (Reviewer Y3pA and HJMp):** We have performed thorough proofreading and revisions with helpful suggestions from the reviewers. We have improved the color of the pipeline, used more formal symbols, and added additional explanations about the experimental results.

**Provide frequency domain analysis to results (Reviewer VVbq):** Following the suggestions of the reviewer, we have added Short-Time Fourier Transform (STFT) in our model for comparison.

**After 7 full days of experiments (with 4 RTX 3090 GPUs), we have added more than 150 new experimental results to address the mentioned issues. All the revisions have been included in the revised paper.**

The valuable suggestions from reviewers are very helpful for us to revise the paper to a better shape. We would be very happy to answer any further questions.

Looking forward to the feedback of the reviewers.

Meta-review: This paper introduces a novel time series forecasting method, Ada-MSHyper, which leverages a hypergraph-based approach to model group-wise multi-scale interactions rather than traditional point-wise interactions. The proposed adaptive hypergraph learning (AHL) module, along with multi-scale interaction and node clustering mechanisms, effectively captures implicit group-wise patterns and distinguishes temporal variations at different scales. Extensive experiments on multiple datasets demonstrate the model's superior predictive performance, maintaining low MSE and MAE even on ultra-long-range time series.

Reviewers have expressed concerns regarding the presentation of the paper and the experimental setup. Despite these issues, the significance of the contributions justifies recommending acceptance. It is advised that the authors incorporate the reviewers' feedback when preparing the final version of the paper.
Decision: Accept (poster)