TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
Authors: Xiaowen Ma
Zhen-Liang Ni
Shuai Xiao
Xinghao Chen
Venue: ICML 2025 poster (2025)
Keywords: multivariate time series forecasting
hyperstate
efficiency
Primary area: applications->time_series
TL;DR: We present a powerful and efficient multivariate time series forecasting model that introduces variable and time-aware hyper-states based on Mamba.

Abstract: In long-term time series forecasting, different variables often influence the target variable over distinct time intervals, a challenge known as the multi-delay issue. Traditional models typically process all variables or time points uniformly, which limits their ability to capture complex variable relationships and obtain non-trivial time representations. To address this issue, we propose TimePro, an innovative Mamba-based model that constructs variate- and time-aware hyper-states. Unlike conventional approaches that merely transfer plain states across variable or time dimensions, TimePro preserves the fine-grained temporal features of each variate token and adaptively selects the focused time points to tune the plain state. The reconstructed hyper-state can perceive both variable relationships and salient temporal information, which helps the model make accurate forecasting. In experiments, TimePro performs competitively on eight real-world long-term forecasting benchmarks with satisfactory linear complexity. Code is available at https://github.com/xwmaxwma/TimePro.

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
Xiaowen Ma '1 Zhenliang Ni
Shuai Xiao Xinghao Chen'
https//gthhb..ooo/wwwwwwm//////
Abstract
in multivariate modeling (Liang et al., 2024; Ahamed & Cheng, 2024; Ma et al., 2024).
In long-term time series forecasting, different variables often influence the target variable over distinct time intervals, challenge known as the multi-delay issue. Traditional models typically process all variables or time points uniformly, which limits their ability to capture complex vari- able relationships and obtain non-trivial time rep- resentations. To address this issue, we propose TimePro, an innovative Mamba-based model that constructs variate- and time-aware hyper-states. Unlike conventional approaches that merely trans- fer plain states across variable time dimensions, TimePro preserves the fine-grat mne-grained temporal fea- tures of each variate token and adaptively selects the focused time points to tune the plain state. The reconstructed hyper-state can perceive both variable relationships and salient temporal infor- mation, which helps the model make accurate forecasting. In experiments, TimePro performs competitively eight real-world long-term fore- casting benchmarks with satisfactory linear com- plexity.
Bi-Mamba+ (Liang et al., 2024) introduces bidirectional structure and splits time series into smaller segments to more comprehensively model the data, incorporating for- getting gate selectively integrate new features with his- torical ones, thereby retaining historical information over longer ranges. S-Mamba (Wang et al., 2025) marks the time points of each variable through the autonomous lin- ear layer, and extracts the correlation between variables through the bidirectional Mamba layer. It also incorporates feedforward neural network (FNN) learn time depen- dence. TimeMachine (Ahamed & Cheng, 2024) combines four Mamba modules to capture both channel-mixed and channel-indeppnndent contexts through multi-scale contex- tual cues. TSMamba (Ma et al., 2024) further optimizes time series forecasting performance through channel-cone comperssed attention modules and a two-stage training strategy, with experiments showing that its innovative components en- hance forecasting accuracy while reducing computational overhead. The mentioned methods often employ different ways to scan features from various directions. However, these methods overlook the fact that different variables have different impact durations on the target variable, i.e., the multi- multi-delay issue, which limits their performance.
1. Introduction
The multi-delay issue in multivariate time series forecasting (Xu et al., 2016; Chandereng & Gitter, 2020) is defined as the temporal discrepancy in the propagation of influence from different predictor variables to the target variable, char- acterized by the presence of distinct and non-uniform time lags between the changes in predictor variables and their corresponding effects on the target variable. However, exist- ing mamba models process all variables or time points in a uniform manner, making it difficult capture critical time points and obtain non-trivial time representations. Moreover, this problem also occurs in transformer-basee models. For example, PatchTST (Nie et al., 2023) captures global tem- poral dependencies in a channel-independen manner, while treating them uniformly for all variables. i'Transformer (Liu et al., 2024b) focuses modeling variable relationships, yet uniformly performs a coarse linear projection for different
Mamba (Gu & Dao, 2024) has shown significant advan- tages in time series forecasting (Ahamed & Cheng, 2024), characterized by its linear computational complexity, and efficient long-term dependency capture capability. Unlike Transformers, which have quadratic complexity concern- ing sequence length, Mamba achieves linear complexity, making it highly scalable for long sequences. Mamba can also capture long-term dependencies, which is particularly beneficial in time series forecasting. Furthermore, can not only handle long series of data but also perform well
Equal contributions. 'Huawei Noah's Ark Lab. Correspon- dence to: Xinghao Chen <xinghao.cheu @ hum>
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time--ware Hyper-state
BCT.
ETIM
Exchange
larter
040
FTTm2
Weather
Mianha
SIFTS
PHcbIST
TimeNel
Figure I: Forecasting performance comparison of Time- Pro with other state-of the-thhettt methods. Average results (MSE) are reported following iTransformer (Liu et al., 2024b). Visualization results show that TimePro out- performs previous methods on the popular multivariate long-term forecasting benchmarks.
Figure 2: Efficiency comparison of TimePro with other state-of-the- art methods. We set the lookback window 96,6, forecast horizon H = 720, and batch size to 16 in the Electricity dataset. The train and inference times are measured on the Nvidia V100 GPU. Compared to other methods, TimePro achieves satisfactory performance with minimal parameters, FLOPs, memory consumption and competitive training and inference speeds.
time points, making it difficult to capture complex internal temporal variations. To address this limitation, propose novel Mamba-based model called TimePro, which incorpo- rates variable and time-aware hyper-state construction. This innovation allows the model to dynamically adapt to the distinet temporal characteristics of each variable, thereby enhancing its ability capture the complex dynamics of multivariate time series data and effectively deal with the multi-delay issue.
ing tasks, particularly those involving long sequences high-dimensional data. Our main contributions are as fol- lows:
We devise novel time-tune strategy that adjusts the variable states by adaptively selecting important time points and uses reconstructed hyper-states to obtain the output. The ability of the hyper-state to perceive complex variable relationships and intra-variable time changes facilitates accurate prediction.
Unlike traditional approaches that merely transfer plain states across variables, TimePro preserves the fine-grained temporal features of each variate token and adaptively se- lects the focused time points to tune the plain state. Specifi- cally, we first scan the variable dimension to obtain the hid- den state, which contains the correlation between variables. This hidden state serves foundation for subsequent tem- poral refinement. Following this, specialized network is employed to learn the offsets of critical time points. By adaptively selecting these key time points, TimePro dynam- ically updates the hidden states to reflect the most salient temporal information. This adaptive mechanism enables the reconstructed hyper-state integrate both variable-specific information and subtle temporal changes, empowering the model to produce highly accurate forecasts.
By combining the hyper-state reconstruction and hardware-aware implementation, we propose ef- ficient multivariate long-term time series prediction model TimePro.
TimePro achieves competitive performance on eight real-world datasets, significantly surpassing existing mamba and transformer-based methods.
2. Related Work
2.1. Time Series Forecasting
In recent years, various methods have emerged in the field of time series prediction (Nie et al., 2024a;b), including those based on RNNs, MLPs, Transformers, and Mamba. RNN-based methods (Hewamalage et al., 2021; Rangapu- ram et al., 2018; Hou et al., 2025), once dominant, have gradually been overshadowed by newer approaches due to their limited capacity to model long-range dependencies and their high computational complexity, which often leads
As shown in Fig. and Fig. 2, TimePro has consistently demonstrated competitive performance across eight real- world forecasting benchmarks. Notably, achieves these results while maintaining linear complexity, ensuring that computational efficiency is not compromised. This balance between accuracy and efficiency makes TimePro powerful tool for tackling complex multivariate time series forecast-
2

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
to inefficient training and inference. The MLP-based mod- els include DLinear (Zeng et al., 2023), LightTS (Zhang et al., 2022) and TimeMixer (Liu et al., 2024a). The MLP model has small number of parameters and low compu- tational complexity, but the ability to model long sequence feature relationships is weak. The Transform-basee method has strong global information modeling ability and is help- ful to capture long-term dependencies in time series (Hou et al., 2024). For example, i'Transformer (Liu al., 2024b) and PStchTST (Nie et al., 2023). However, the transformer method has high computational complexity and is prone to overfitting in time series forecasting tasks. Mamba-base based methods can model long time series relationships and have linear complexity, so this kind of method has great advan- tages in time series prediction tasks (Wang et al., 2025; Ahamed & Cheng, 2024; Liang et al., 2024).
could be denoted as follows:
1 (t) = Ah(1) + Bx((t,
(1)
y(() = Ch((),
where A E RRN. is the evolution parameter, B, C' E R.N.DD, D. denote the learnable projection parameters, and N₂ is the state size.
Discretization. The above continuous- SSMs are not well compatible with deep learning algorithms. Therefore, discretization is needed to align the model with the sampling frequency of the input signal to improve computational ef- ficiency (Gu et al., 2021). Following the previous work (Gupta et al., 2022), given the sampling time scale parame- ter A, the above continuous SSMs are discretized through zero-order hold rule, thus converting the continuous tittimmuuu-tmeee parameters (A, B) to their corresponding discrete counter- parts (A, B):
2.2. Mamba
A = e AA
In recent years, a series of mamba-based methods have been proposed for time series forecasting. Bi-Mamba+ (Liang et al., 2024) introduces bidirectional structure and splits time series into smaller segments to more comprehensively model the data, incorporating forgetting gate to selec- tively integrate new features with historical ones, thereby retaining historical information over longer ranges. S- Mamba (Wang et al., 2025) marks the time points of each variable through the autonomous linear layer, and extracts the correlation between variables through the bidirectional Mamba layer. In addition, S-Mamba also incorporates feedforward neural network (FNN) to learn time depen- dence. Machine (Ahamed & Cheng, 2024) combines four Mamba modules to capture both channel-mixed and channel-independent contexts through multi-scale contex- tual cues. MSMamba (Ma et al., 2024) further optimizes time series forecasting performance through chamnel-compressed attention modules and two-stage training strategy, with ex- periments showing that its innovative components enhance forecasting accuracy while reducing computational over- head. SST (Xu et al., 2024) deals with long-range global patterns and short-range local changes simultaneously by Mamba and LWT, which solves the shortage of traditional models in modeling long-term and short-term features (Liuu et al., 2023). However, the above methods ignore the dif- ferent time lags between multivariate and prediction results, which limits their performance.
(2)
B = (A 1). AB.
Then, The discretized formulation of Eq. 1 is formulated as:
h =Ah,11+ + Bz,,
Y/ = Chh,
where A E N.N.N. B R.N.D. In addition, the iter- ative process described in Eq. 3 can be performed by the parallel computing mode of global convolution (Gu & Dao, 2024) to improve the computational efficiency:
y == K
(4)
with K= (CB.CAB. ".BB....AAAAAA B),
where @ denotes the convolution operation, and K e R serves as the kernel of the SSMs.
Selective State Space Models (S6). Conventional SSMs (i.c., S4) have been implemented to capture sequence con- text under linear time complexity, despite the fact that they are constrained by static parameterization and cannot per- form content-based reasoning. To address this problem, the selective state space model (i.e., Mamba (Gu & Dao, 2024)) has been proposed, which allows the model to selectively propagate or forget information based on the latitude of the current token along the length of the sequence by simply setting the parameters the SSM as a function of the inputs. In S6, the parameters B, C. and A are computed directly from the input sequence (t)), thus enabling sequence-awan parameterization.
3. Preliminaries
State Space Models (S4). State space models (SSMs) proposed in deep learning as common sequence mod- els (Gu et al., 2022), which transform an input sequence ar()) E RL,xD, into an output sequence y(t) E g4,X1 xD, by utilizing a learnable hidden state h(t) E N.N, The process
4. Method
Multivariate Time Series Forecasting (MTSF) deals with time series data containing multiple variables or channels at
3

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
Projection
Hyper.Mamba
value
ProBlock
Add & Norm
Scam
TimeFFN
time
Inpul
State
Hyper State
Patching
0
Hyper-Scan
Bi-direction
Add & Norm
Linear
Linear
HyperMamba
varfate
Embedding
Embedding
Figure 3: Overview of our TimePro method. The multivariate time series is first embedded along the temporal dimension with the patching operation to get the series representation for each variable. Then the variable correlation and time representation of variables are captured by multiple layers of ProBlock modules. The core component of Problok is HyperMamba, which adaptively selects important time points to regulate the plain state of the variable dimension. The reconstructed time- and variable-aware hyper-states are then applied to obtain the output.
each time step. Given historical value X N where L denotes the length of the lookback window and N denotes the number of variables, the goal of MTSF is to predict future value Y E Nx where H > 0 is the forecast horizon. In this paper, we focus on long-term forecasting N >= 96, which is more challenging.
time series x,: RR into overlapping patches and preserve the dimensions of the variables,
En = EmmbedddiggX") (X),
(5)
where embedding & NxPxxD and D denote the number and feature dimensions of patches, respectively.
4.1. Overview
We apply multiple layers of ProBlock to opti- mize the embedding, which enables information interactions across time and variables,
ProBlock
As shown in Fig. 3, TimePro adopts a transformer-like pure encoder architecture (Vaswani et al., 2023), which consists of the following components:
E = ProBlook(EE)), i 0, 1..
(6)
Reversible instance normalization Training and test data often suffer from distributional shifts, and directly inputting raw sequences into the model is not conducive to stable predictions. Following previous work (Liu et al., 2024b; Han et al., 2024), we use reversible instance normalization (RevIN) (Kim et al., 2022) to concentrate the sequence to zero mean and scale unit variance before inputting the original sequence X into the model. Then, the predicted sequence is reverse normalized to obtain the output Y.
where y the number of layers. In addition, as shown in Fig. 3, ProBlock composed of HyperMamba and Tim- eFFN, which complete the information interaction between variables and the capture of complex changes within vari- ables respectively. We'll describe the HyperMamba module in the next subsection, which is the focus of this paper.
Linear projection After y layers of ProBlock, we first flatten the embedding of each variable, and then apply simple linear projection obtain the forecast results,
Time and variable preserved embedding Previous works have either used sequential embedding (Liu et al., 2024b; Wang et al., 2025: Han et al., 2024) or channel- independent patch embedding (Nie et al., 2023). However, this style of uniform processing of all variables time points limits the ability of models to capture complex vari- able relationships and obtain non-trivial time representa- tions. Therefore, we use time and variable preserved em- beddings, which facilitate the construction of subsequent htper-states. Specifically, we divide each input univariate
Y Projection(Er.).
4.2. HyperMamma Module
We propose HyperMamba to optimize the modeling of vari- able dependencies. Existing popular methods (Liu et al., 2024b) mainly use attention to capture this relationship. However, they suffer from quadratic complexity with the number of variables, which severely limits the deployment

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
Project
A,
C
Ah
Bx
Time tune
GPL
GPU HBM
Input (N, PxD)
Scan along variable
State PsD)
Hyper-State (N PsD)
Output (N, PxD)
GPU memory hierarchy
Reshape
Reshape
Sample
State: h (N, P. D)
Conv
Offsets: 8n
Sampled State: h (N, P.
Conv
HyperState: ho (N,
Time tune
Figure 4: Implementation details of hardware-aware hyper-scan. We effectively apply the GPU memory hierarchy, i.e., perform plain state acquisition GPU SRAM (implemented in the grey box above), and other operations on GPU HBM. Specifically, we follow the original Mamba implementation by first scanning the embedding along the variables and acquiring the plain state. Then, we perform reshape on the plain state to recover the fine-grained time dimension of the embedding, Next, we adaptively select important time points for each variable to adjust the plain state to obtain time- and variable-aware hyper-states. Finally, the reconstructed hyper-states are applied to obtain the augmented embeddings through gating mechanism.
of models in real-world scenarios. Although some methods introduce Mamba (Wang et al.. 2025) or MLP (Han et al., 2024) to extract variable dependencies, they use sequence embedding to process each variable sequence, which is not enough capture complex internal time changes within variables. particular, in the real world, the relationship between various physical variables is not static and tends to fluctuate with changes in local time (Zhou et al., 2021). Therefore, we propose a novel time tune strategy to con- struct the variable- and time-aware hyper-state, so as to model the information interaction between variables better.
HyperMamba Design The structure diagram of the Hy- perMamba module can be seen in Fig. 3, which is slightly modified from the original Mamba (Gu & Dao, 2024). First, we replace the selective sean with a hardware-aware Hyper Scan that underpins the construction of hyperstates. Com- pared to the original selective scan module, the hyper scan does not reduce the efficiency and achieves better perfor- mance. In addition, we remove the depthwise convolution before the scan and the linear projection after the scan, which are verified as unnecessary in Sec. 5.3. In addition, we add the scanning inception design, i.e., we split two parts along the channel and scan in opposite variable di- rections, thereby enhancing the model's ability capture global variable dependencies.
Specifically, for input & NxPxXD) we first get & E (XPxD) and & (xxx(P through two linear projec- tions. Then, split the & into two parts along the channel and input the Hyper-scan module along the opposite variable direction to capture the global variable dependencies,
The Vanilla Mamba transfers the state inde- pendently along each dimension of the embedding and gets the output through the parameter C. Therefore, when we apply the original selective scanning module to scan along the variable dimension, the state contains only variable in- formation. is not conducive capturing accurate variable dependencies, which fluctuate with time changes within the variable. Therefore, we introduce time tune strategy to adjust the initial state adaptively according to the time points intra-variable.
Hyper-scan
PD 1),
εn (xpe
PD E,2 (&/: PD)),
(8)
& Con,
Finally, we apply the SiLU activation function (Ramachan- dran et al., 2017) to E, and dot-multiply it with the aug-
mented & get the output E
The process Hyper-scan is shown Fig. 4. For the input &, E (x(xD) is first scanned along the variable
E E. SITUUEE)..
5

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
dimension and the initial state h E R-x((PD) is obtained. This step is implemented in SRAM reduce the repeated read and write of HBM memory. We then reshape the dimensions of state h and obtain an initial shift 8y, through a convolution,
5. Experiments
5.1. Datasets and Implementation Details
In order to comprchensively evaluate the performance of the proposed TimePro, we conduct extensive experiments on five widely used real datasets, including ETT (4 subsets), Exchange, Electricity, Weather (Zhou et al., 2021; Wu et al., 2021), and Solar-Energy (Lai et al., 2018). All experiments are implemented on four Tesla V100 GPUs and follow the common training settings in (Liu et al., 2024b). A detailed description can be found in Appendix A.
5n = Chnv(h).
(10)
Next, we add the reference point to the learnable offset to get the sample point, which serves as the final coordinate for extracting the important time point from the state. In prac- tice, we follow (Xiong et al., 2024) by linear interpolation & to make the sampling process differentiable,
5.2. Main Results
haamp liref + Ōh.
(11)
As shown in Table 2. TimePro achieved state-of-the-ard per- formance on multiple datasets, with the first place marked in red and the second place marked in blue. TimePro scored 12 firsts and seconds out of 16 metrics. With the exception of the Solar-Energy dataset, our approach outperforms the state-of-ther the-trrttttrtttr methods SOFTS and iTransformer. On the weather dataset, the MSE of our method is 2.0% lower than that of SOFTS and 3% lower than that of iTransformer. On the weather dataset, the MSE of our method is reduced by 2% compared with that of SOFTS and 3.1% 11% compared with that of iTransformer. Compared with SOTFS and iTrans- former the ETTml dataset, TimePro reduces mae by 1.2% and 2.9%, respectively. On most data sets, Time- Pro outperforms past advanced methods such as TimesNet, PStchTST and Dlinear. This is because these methods deal with the time dimension in a unified manner, and there is no way to capture complex time relationships.
h- href + (h; haamp);
(12)
where h E XXNXPXXX and M is the number of sampling time points. Then, linear mapping is used to fuse the sam- pled time points to obtain the hyperstate ho E RNXxxD
ho = Lh)
(13)
Finally, the reconstructed htper-state is multiplied with the parameter matrix C in Mamba to get the output.
4.3. Complexity Analysis
We analyze the complexity of the core component Hyper- Mamba from patch number P. feature dimension D, vari- able number N, and sampling point M. First, the input passes through two linear projections with a complexity of D(NP2D") Then, we get the parameter matrix A, B, C' by linear mapping and structured operation respectively, which has a complexity of OD) since we set the num- ber of states in each dimension 1. During the scanning process, the initial state is PD) complex because only needs to be scanned independently along the variable di- mension. The time tune process involves the generation of offsets (i.e., convolution) and linear projection, requires a complexity of PMM.. Given that D is a relatively small constant and M also constant (i.c., 9), these two terms can be ignored. In addition, P is affected by the length of the series L, so the complexity of the entire hy- perMamba is T.)..). As shown in Table 1, two classical comparison models, such as iTransformer and PatchTST, require significantly more complexity than TimePro.
Fig. 2 also provides a detailed comparison of efficiency met- rics including parameters, FLOPs, training time and infer- ence time. It can be observed that TimePro possesses better computational efficiency than previous methods. Specifi- cally, TimePro has training time and inference time similar to S-Mamba and significantly outperforms other convolu- tional or transformer- based methods. For example, TimePro obtains lower prediction errors with 2.7 and 14.4 times the inference speed of PatchTST and TimesNet, respec- tively. Moreover, TimePro has the minimal parameters, FLOPs, and memory consumption. For example, Time- Pro requires only 67% parameters and 78% GFLOPs of S-Mamba. In addition to satisfactory efficiency, TimePro also outperforms recent advanced models including iTrans- former and S-Mamba in forecasting performance. These results demonstrate TimePro's 's lightweight and suitability for deployment in a variety of real-world scenarios where resources are constrained.
Table 1:
Complexity comparison between popular time series forecasters concerning series length L, number of variables N. TimePro achieves only linear complexity.
To further validate the robustness of our approach, we con- duct a comparative analysis of various models across dif- ferent lookback window lengths. As depicted in Figure 7, TimePro consistently achieves the lowest MAE across all
TimePro (ours)
iTansformer
PachTST
Transformer
Complexity
O(NL)
ON NLJ
ONNI")
O
6

---

TimePro:
Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
Table 2: Multivariate long-term forecasting results with horizon H E (96, 192, 330, 720} and fixed lookback window length
L = 96. Results are averaged from all prediction horizons.
TimePro
S-Mamba
SOFTS
Frransformer
PatchTST
Crossformer
TDDE
TimesNet
DLincur
SCINet
Stationary
Models
ours
(2015
(2024)
(2024))
(2023)
(2013)
(2023)
(2023)
(2023)
(2022)
(2002)
Metric
MSF.
MAF
MSF
MAR
MSF
MAF
MSE
MAF
MSF
MAF
MSE
MAR
MSE
MAF
MSF
MAP
MSR
MAR
MSF
MAF
MSE.
MAA
ECL
0.169
0.262
0.170
0.265
0.174
0.264
0.178
0.270
0.189
0.276
0.244
0,334
0.251
0.344
0.192
0.295
0.212 0,300
0,268 0.365
0.193
0.296
Exchange
0.352 0.399
0.367
0.408
0.361
0.402
0.360
0.403
0.367 0.404
0.940
0.707
0.370
0.413
0.416
0.443
0.354 0.414
0.750 0.626
0.461
0.454
Weather
0.251
0.276
0.251
0.276
0.255
0.278
0.258
0.278
0,256
0.279
0.259
0.315
0.271
0,320
0.259
0.287
0.265
0.317
0,292
0.363
0,288
0.314
Solar-Fnergy
0.232
0.266
0.240
0.273
00.256
0.233
0.262
0.236
0.266
0.641
0.639
0.347
0.417
0.301
0.319
0.330
0.401
0.282
0.375
0.261
0.381
FTTml
0.391
0..400
0.798
0.405
0..993
0.403
0.407
0.410
0.396
0.406
0.513
0.496
0,419
0.419
0.400
0.406
0.403
0.407
0.485
0.481
0.481
0.456
ETTm2
0.281
0.326
0.288
0.332
0.287
0.330
0.288
0.332
0.287
0,330
0.757
0.610
0.358
0,404
0.291
0.333
0.350
0.401
0.571
0.537
0.306
0.347
ETThi
0.438
0.438
0.455
0,450
0.449
0:442
0.454
0.447
0.453
0.446
0.529
0.522
0.541
0,507
0.458
0,450
0.456
0.452
0.747
0.647
0,570
0.537
ETTh2
0.377
0.403
0.388
0.405
0.373
0.400
0.383
0.407
0.385 0.410
0.942
0.684
0.611
0.550
0.414
0,427
0.559
0.515
0.954
0.723
0.526 0,516
tested lookback windows. Specifically, when the lookback window is set to 48, our method demonstrates signifi- cantly lower MAE compared to SOFTS. This is attributed to TimePro's 's capability to effectively capture crucial tempo- ral information, enabling it to maintain strong performance even with limited data. Notably, on the ETTm2 and Ex- change datasets, we observe substantial reduction in MAE when the lookback window is extended to 336. While it is generally expected that a larger lookback window would provide more information and thus lead to more consistent model performance, our model's ability to excel under such conditions is noteworthy. This is largely due to TimePro's s dynamic feature selection mechanism, which allows it to identify and focus on key temporal features, thereby pre- venting the relevant information from being overwhelmed by the sheer volume of time series data.
suggests that TimePro effectively mitigates the detrimental effects of the multi-delay issue on accurate variable relation- ship modeling.
Ablation of the feature dimensions We first explore the effect of feature dimensions on forecasting performance, as shown in Fig. 6. For most datasets, once the feature dimen- sion increases to a certain point (such as 48 or 64), further increasing the feature dimension may not significantly im- prove model performance, and can sometimes even lead to a decrease in performance. For example, the impact of feature dimensions on the Solar dataset is complex, with initial increase in MAE (from 32 to 48), but subsequent decrease in MAE when the dimensions are further increased (from 48 to 96). Therefore, selecting a moderate feature dimen- sion is a better strategy. Moreover, TimePro achieves the best performance on most of the datasets when the feature dimension is 48. The small feature dimension ensures that TimePro is efficient.
5.3. Ablation study
We conduct series of ablation experiments to investigate the effect of different hyperparameters and the effectiveness of the model structure design.
Ablation of the number of encoder layers Regarding the number of encoder layers in Fig. 6, for the ECL dataset, increasing the number of encoder layers significantly im- proves performance, with the MAE gradually decreasing as the number of layers increases. For the Exchange dataset, increasing the number of layers sigmificantly improves per- formance, but the improvement becomes smaller when the number of layers exceeds 2, eventually stabilizing at 3 and 4 layers. For the Weather dataset, increasing the number of layers has little impact on MAE, indicating that this dataset has a lower demand for the number of layers. For the Solar dataset, increasing the number of layers significantly im- proves MAE, especially when increasing from to 3 layers, with a noticeable performance boost, but a slight increase at 4 layers. In summary, increasing the number of encoder layers generally improves model performance, but this im- provement tends to level off or slow down after a certain point. For most datasets, an encoder layer count between 2 to 4 is likely to be a good choice, as it balances model complexity and performance.
Effect on multi-delay issue We first add a visualization experiment to further validate the validity and interpretabil- ity of TimePro for the multi-delay issue, as shown in Fig. 5. We choose test sequences from the ETTml and E datasets as examples. Specially, we first calculate the cor- relation of label sequences (i.e.. groundtruth) by Pearson correlation coefficient,
DL. (x. - 3) (21- - 17)
Tzy =
(14)
1 (z; - -3)². Ete (y:- - -2)
where Fz; V/ E RR run through all time points of the paired variates to be correlated. We then visualize the correlation between the variable features before and after HyperMamba. Fig. 5 shows that TimePro selects important time points through the time-tune strategy, which drives the learned multivariate correlations closer to the label sequences. This
7

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
GT Correlations
Before HyperMamba
After HyperMamba
GT Correlations
Before HyperMamba After HyperMamba
Figure 5: Visualization for multivariate correlation analysis ETTm] (upper) and ETTh I (bottom) dataset. The visualization implemented based the Pearson Correlation Coefficient. The GT Correlations denote the correlation between the variables of the forecast sequence (groundtruth). The two columns the right denote the correlation between the variables before and after the HyperMamba module, respectively. It shows that TimePro drives the learned multivariate correlations closer to the forecast sequence through the HyperMamba module.
Ablation of the patch length Fig. shows that the im- pact of patch length model performance also varies from one dataset another. For some datasets, increasing the patch length can significantly improve performance, while for others, the effect may be less pronounced. For the ECL dataset, increasing the patch length (from 8 to 32) reduces MAE, with significant performance improvement due to patch length. For the Exchange dataset, the impact of patch length MAE is relatively small, with only minor performance changes, indicating that this dataset has lower demand for patch length. For the Weather dataset, the im- pact of patch length is more complex, with an initial increase in MAE (from to 16), but a subsequent decrease in MAE when the patch length is further increased (from 16 to 32). For the Solar dataset, increasing the patch length from 8 to 32 significantly reduces the MAE, with clear performance improvement due patch length. Overall, moderate patch length (such 16 to 32) may be a good starting point, as it can capture sufficient temporal sequence information without excessively increasing model complexity.
Table 3: Ablation experiment of the time tune design on the Exchange and ETTh1 dataset. Results are averaged with horizon H {96, 192, 336, 720].
Exchange MSE
ETThl
Variant
MAE
MSE
MAE
Non-Adaptive
0,360 0.402
0.451
0.44T
Adaptive
0.352 0.399
0.438
0.438
in Table 3. An optional strategy is to integrate the variables states along the time dimension using a linear projection, i.c. non-adaptive. The results show that the non-adaptive variant obtains MSEs of 0.360 and 0.351 on Exchange and ETTh1, respectively. TimePro achieves smaller MSEs and MAEs on Exchange and ETThl compared the non-adaptive variant. These experimental results validate that the introduction of an adaptive strategy can better tune the state of the variables and improve the prediction performance.
Ablation of the HyperMamba structure In addition, we also carry out an ablation analysis of HyperMamba' 's struc- ture. Compared to the original Mamba, we remove two components, the depth-wist wise convolution before scanning
Ablation of the time tune design We next perform a further ablation analysis of the time tune strategy, as shown

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
BCL
Exchange
Weather -+ Solar
ECL
Exchange
Weather -+- SSollr
ECL
Exchange
Weather
Solar
0.43
0.43
0.43
0.41
0.41
0.41
0.39
0.39
0.39
0.37
0.37
0.37
0.33
0.31
0.31
0.31
0.29
0.29
0.29
0.27
0.27
0.27
0.25
0.25
0.25
32
48
64
80
96
16
24
32
Dimemsoo features
Layer Encoder
Length Patches
Figure 6: Influence of the hidden dimension of features D, Layer of encoder L and length of patches. We select ECL, Exchange, Weather and Solar-energy for visualization.
ETTm2
ECL
Exchanng
0.4
0.33
0.43
PasiiST SOFTS
PAwATST
SOFTS
3OFTS TinAFraUurr)
0.38
TireFrutlerc
0.31
ThacPratttnn
MAE
MAE
MAE
0.36
0.29
0.41
0.34
0.4
0.32
0.25
0.39
336
96
192
336
48
96
192
336
Lookhack Window Length
Lookbork Window Lougth
Lookback Window Langth
Figure 7: Influence of lookback window length L E 48, 96, 192, 336 on ETTm2, ECL and Exchange dataset. TimePro performs almost consistently better than other models under different lookback window lengths.
Table 4: Ablation experiment of the HyperMamba structure on the Exchange and ETTh1 dataset. Results are averaged with horizon H E {96, 192, 336, 720).
6. Conclusion
In this paper, we begin by examining previous work on time series forecasting. These works tend to treat all variables or time points uniformly, which limits their ability simulta- neously capture complex variable relationships and obtain nontrivial temporal representations. Inspired by the recent popularity of state-space models (i.e., Mamba), we propose an efficient multivariate long-term time series forecasting model TimePro. contrast to state transfer only in the vari- able dimension, TimePro adjusts the variable states through time-tune strategy and uses the reconstructed hyper-states to obtain the output. The prediction performance is im- proved due to the ability of the hyperstates to perceive both complex variable relationships and intra-variable time vari- ations. More remarkably, TimePro also absorbs the advan- tages of hardware awareness and linear complexity of the original mamba, thus ensuring high efficiency. Comprehen- sive experiments verify the effectiveness of TimePro. In future work, plan to build a large time series prediction foundation model based on TimePro, thus providing the community with more directions to explore.
Exchange MSE MAE
ETTh1
Variant
MSE
MAE
Mamba + Hyper-Scan
0.358 0.403
0.449
0.439
+DWConv
0.358 0.402
0.447
0.440
-Linear
0.356 0.401
0.447
0.441
HyperMamba
0.352 0.399
0.438
0.438
and the linear projection after scanning. As shown in Table 4, we replace the selective scanning module of the original Mamba with the Hyper-Scan and use it as the baseline, i.e., Mamba+Hlyper-Scan It can be observed that the forecast- ing performance is slightly improved after the removal of both components. We believe that the possible reason for this is that there is no localization between the variables and therefore depth-wise convolution is not necessary. In addition, the output linear within the original Mamba mod- ule is also redundant due to our introduction of TiMeFFN. Therefore, we remove these two components, which could also improve the efficiency of TimePro.
9

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
Impact Statement
status and future directions. International Journal of Forecasting, 37(1):388 427, 2021.
This paper presents work whose goal to advance the field of Machine Learning. There are many potential societal consequences of our work, none which feel must be specifically highlighted here.
Hou, J., Cao. Z., Zheng, N., Li, X., Chen, X., Liu, X., Cong, X., Hong, D., and Zhou, M. Linearly-evolved transformer for pan-sharpening. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 1486-1494, 2024.
References
Ahamed, M. A. and Cheng, Q. Timemachine: A time series is worth 4 mambas for long-term forecasting. arXiv preprint arXiv:2403.09896 2024.
Hou, J., Liu, X., Wu, C., Cong, X., Huang, C., Deng, L.-J,, and You, J. W. Bidomain uncertainty gated recursive network for pan-sharpening. Information Fusion, 1 118 102938, 2025.
Chandereng, T. and Gitter, A. Lag penalized weighted cor- relation for time series clustering. BMC bioinformatics, 21:1-15, 2020.
Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.-H., and Choo, Reversible instance normalization for accu- rate time-series forecasting against distribution shift. In International Conference Learning Representations, 2022. URL https: sogenreview.netee net//orumttt id-cGDAKOelCCp
Das, A., Kong, W., Leach, A., Mathur, S., Sen, R., and Yu, R. Long-term forecasting with tide: Time-series dense encoder, 2023. URL https: //tp:///a/vv////a//aa org/ abs/
2304. 08424.
Gu, A. and Dao, T. Mamba: Linear-time sequence mod- eling with selective state spaces, 2024. URL https: / ///axxiv.oo orgg/ abs/ /2122 00752.
Kingma, D. P. and Ba, Adam: A method for stochastic op- timization, 2017. URL https: http://axxvvv//o aabs/ 6980.
Gu, A., Johnson, I., Goel, K., Saab, K., Dao, T. Rudra, A., and Ré, C. Combining recurrent, con- volutional, and continuous-time models with linear state space layers. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 572-585. Curran Associates, Inc., 2021. URL ca/ sc/paper_see feaper__flees/pperr//2///// 0554 (8446b0e3888 60t.0e/3aab9155c500eeeeeeeeeeeee pdf.
Lai, G., Chang, W.-C., Yang, Y., and Liu, H. Model- ing long- and short-term temporal patterns with deep neural networks. SIGIR '18, pp. 95-104, New York, NY. USA, 2018. Association for Computing Machin- ery. ISBN 97814 9781450356572. doi: 10.1145/3209978. http:://ooo..ggggg/ 3210006. URL http:://cco..orr/11.1155 0109978.321066
Liang, A., Jiang, X., Sun, Y., and Lu, C. Bi-mambat: Bidirectional mamba for time series forecasting, arXiv preprint arXiv:2404.15772 2024.
Paper.
Gu, A., Goel, K., and Ré, C. Efficiently modeling long sequences with structured state spaces, 2022. URL https:///arse org/ abs/2bll11 00396.
Liu, H., Zhang, J.. Li, Y., Zhou, H., Zhang, S., and Xie. X. Timemixer: Decomposable multiscale mixing for time series forecasting. In International Conference on Learning Representations (ICLR), 2024a. URL https: //arxiiv.orgg..................a http://dd//2400./ 14616.
Gupta, A., Gu, A., and Berant, J. Diagonal state spaces are as effective structured state spaces. Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Pro- cessing Systems, volume 35, pp. 22982-22994. Curran Associates, Inc., 2022.
LIU, M., Zeng, A., Chen, M., Xu, Z.,, LAI, Q.. Ma, I.., and Xu, Q. Scinet: Time series modeling and forecasting with sample convolution and interaction. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 5816-5828, Curran Associates, Inc., 2022.
Han, L., Chen, X.-Y,, Ye, H.J.,, and Zhan, D.-C. SOFTS: Efficient multivariate time series forecasting with series-core fusion. In The eiirty- Annual Conference on Neural Information Processing Systems, 2024. URL hictps://operrviiii net forum? id-89AUi5IDD 511uA.
Liu, X., Hou, J., Cong, X., Shen, H., Lou, Z., Deng, L.-J,, and You, J. W. Rethinking pan-sharpening via spectral- band modulation. IEEE Transactions on Geoscience and Remote Sensing, 62:1-16, 2023.
Hewamalage, H., Bergmeir, C., and Bandara, K. Recur- rent neural networks for time series forecasting: Current
10

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
Liu, Y., Wu, H., Wang, J., and Long, M. Non-stationary transformers: Exploring the stationarity in time series forecasting, In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D.. Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 9881-9893. Curran Associates, Inc., 2022.
Wu, H., Xu, J., Wang, J., and Long, M. Autoformer: De- composition transformers with auto-correlation for long- term series forecasting. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, vol- ume 34, pp. 22419-22436 Curran Associates, Inc., 2021.
Wu, H., Hu, T., Liu, Y., Zhou, H., Wang. J., and Long, M. Timesnet: Temporal 2d- 2d-variation modeling for general time series analysis. In The Eleventh International Confer- ence Learning Representations, 2023. URL https: / //openeeieww net forum?id forumiid-juuuqw ju__qw384uupp
Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M. itransformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024b. URL https:::: pttttp:://ppeee http//pprrvvveeeeeeee net/ forum? Aa=JePfAIBfal
Xiong. Y., Li, Z.. Chen, Y., Wang, F,, Zhu, X., Luo, J., Wang, W., Lu, Li, H., Qiao, Y., Lu, L., Zhou, J., and Dai. J. Efficient deformable convnets: Rethinking dynamic and sparse operator for vision applications. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5652-5661, June 2024.
Ma, H., Chen, Y., Zhao, W., Yang, J., Ji, Y., Xu, X., Liu, X., Jing, H., Liu, S., and Yang, G. A mamba founda- tion model for time series forecasting. arXiv preprint arXiv:2411.022 0294411 2024.
Nie, T. Mei, Y., Qin, G., Sun, J., and Ma, W. Channel- aware low-rank adaptation in time series forecasting. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 3959- 3963, 2024a.
Xu, X., Chen, C., Liang, Y., Huang, B., Bai, G., Zhao, L., and Shu, K. Sst: Multi-scale hybrid namba-transformer experts for long-short range time series forecasting, 2024, URL https: / arxiv. org/ haap/ab///// abs/ 14757.
Xu, Y., Liu, Y., and Zhu, Q. Multivariate time de- lay analysis based local kpca fault prognosis approach for nonlinear processes. Chinese Journal of Chem- ical Engineering, 24(10):1413-1422 2016. ISSN
Nie, T., Qin, G., Sun, L.,, Ma, W., Mei, Y., and Sun, J. Con- textualizing mip-mixers spatiotemporally for urban traffic data forecast at scale. IEEE Transactions on Intelligent Transportation Systems, 2024b.
1004-9541. doi: http:///ooioogg/lloo0////ccceooe 011. URL UtL.tttpps//ww..seeeedddeeeeee peckenee/artiaeeepp/////osssssss
Nie, Y. H. Nguyen, N., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long- Nong-termmfe forecasting with transformers. In International Conference on Learning Representations, 2023.
Zeng, A., Chen, M., Zhang, L., and Xu, Q. Are trans- formers effective for time series forecasting? Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, (1(9):1121-1112:8 Jun. 2023. doi: 10.1609/ Stl.v3799.26313. URL, hwwp::tttssste htt.htttss:/oos..aaa orr/ index phpp AAAI /tt////accceeeee///1111111
Ramachandran, P., Zoph, B., and Le, Q. V. Searching for activation functions, 2017. URL htt.t:://// //attps://arivv.. org/abs/171i 05941.
Zhang, T., Zhang, Y., Cao, W., Bian, J., Yi, X., Zheng, S., and Li, J. Less more: Fast multivariate time series fore- casting with light sampling-oriented mlp structures, 2022. URL https: / arxiv. .ww.iiiorrr///o.... org www//rgg/////////////////eaeee/e aaaaaaas/222// 01186.
Rangapuram, S. S., Seeger, M. W., Gasthaus, J., Stella, L., Wang, Y., and Januschowski, T. Deep state space models for time series forecasting. In Bengio, S., Wal- N., lach, H., Larochelle, H., Grauman, K., Information and Garnett, R. (eds.), Advances in Neural Processing Systems, volume 31. Curran Associates, Inc., 2018.
Zhang, Y. and Yan, J. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The Eleventh International Conference on Learning Representations, 2023. URL https:// openreview net / forum i id=
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need, 2023. URL https: //apsiv/////vv////a org/ abs /176 03762.
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. Informer: Beyond efficient trans- former for long sequence time-series forecasting. Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, 35(12): (122):11100-01) 106-11115, May 2021. doi: 10.1609/ vaai.v35i12.17325. URL
Wang, Z., Kong, F., Feng, S., Wang, M., Yang, X., Zhao, H., Wang. D., and Zhang, Y. Is mamba effective for time series forecasting? Neurocomputing 619:1 129178, 2025.
index, AAA./AAAAA/AAAAAAA httppp//w...aartttcee//eeeee///a

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
A. Dataset
We conduct experiments on eight real-world datasets. (1) ETT datasets: ETT (Zhou et al., 2021) (Electricity Transformer Temperature) comprises data on load and oil temperature, collected from electricity transformers over the period from July 2016 July 2018. It contains four subsets, ETTmI, ETTm2, ETThl and ETTh2. ETThl and ETTh2 are recorded every hour, and ETTml and ETTm2 are recorded every 15 minutes. In addition, ETT datasets have few varieties and weak regularity. (2) Exchange (Wu et al., 2021) collects the panel data of daily exchange rates from 8 countries from 1990 to 2016. (3) Weather includes 21 indicators of weather, such as air temperature, and humidity. Its data is recorded every 10 min for 2020 in Germany. (4) Solar-Energy (Lai et al., 2018) records the solar power production of 137 PV plants in 2006, which is sampled every 10 minutes. (5) ECL (Wu et al., 2021) records the hourly electricity consumption data of 321 clients. The details of datasets are also provided in Table 5.
Table 5: Detailed dataset descriptions. Chamnels denotes the number of channels in each dataset. Dataset Split denotes the total number of time points in (Train, Validation, Test) split respectively. Prediction Length denotes the future time points to be predicted and four prediction settings are included each dataset. Granularity denotes the sampling interval of time points.
Dataset
Channels
Prediction Length
Dataset Split
Granularity
Domain
ETTh2
ETTh2
7
{96, 192, 336, 720}
(8545, 2881, 2881)
Hourly
Electricity
ETTml, ETTm2
7
(96, 192, 336, 720)
(34465, 11521, 11521)
15min
Electricity
Weather
21
{96, 192, 336, 720}
(36792. 10540)
10min
Weather
ECL
321
(96, 192, 336, 720}
(18317, 2633, 5261)
Hourly
Electricity
Exchange
8
{96, 192, 336, 720}
(5120, 665, 1422)
Daily
Economy
Solar-Energy
137
(96, 192, 336, 720}
(36601, 5161, 10417)
10min
Energy
B. Implementation Details
B.1. Details of time and variable preserved embedding
Following PStchTST (Nie et al., 2023), we divide each input univariate time series X E IR into patches that can overlap. Let the patch length be Pt and the stride between two consecutive patches be Sj. The patching process generates a sequence of patchesX?;, X." 1.: E Px where P is the number of patches and P = شيك + 2. Here, we the last value of X,, E IR_ with the number of S repetitions to the end of the original sequence and then patch it. By using patching, the number of input tokens can be reduced from I. to approximately L. This means that TimePro's 's memory usage and computational complexity reduced by factor of St.
B.2. Experiment Details
All experiments are performed on 4 NVIDIA Tesla V100 GPUs with 32G VRAM. The model is optimized using the Mean Squared Error (MSE) loss function. The performance of the different methods is compared based on two main evaluation metrics: the Mean Squared Error (MSE) and the Mean Absolute Error (MAE). We use the ADAM optimizer (Kingma & Ba, 2017) with an initial learning rate of 5 x 10-4. This rate is modulated by cosine learning rate scheduler. We study the layer of Problocks within the set (1,2,3.4) and the dimension of the embedding / within (32,48,64,96)
C. Full Results
Table 6 shows the full results of the prediction benchmarks. We conduct experiments using eight widely used real- world datasets and compare our method to ten previous state-of-the-art models. The comparison models use variety of architectures including convolution, transformer, mlp, and mamba. In these tests, our TimePro exhibits strong performance. Specifically, TimePro achieves the best performance on most of the datasets. Both at different prediction lengths and on average, TimePro possesses smaller MSE and MAE.
In addition, we implement visualizations on the ETTh2 and ECL. dataset to further validate the predictive effect of TimePro.
12

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
As shown in Fig. 8 and 9, compared to iTTransfommer and S-Mamba, TimePro's prediction curves are closer to the groundtruth, and the curve distances are smaller at the inflection point locations. The visualization analysis further validates the effectiveness of TimePro.
Table 6: Multivariate long-term forecasting results with prediction lengths H E (96, 192, 336, 720) and fixed lookback window length L = 96. The results are are taken from iTransformer (Liu et al., 2024b) and S- S-Mamba (Wang et al., 2025).
Medel::
TinsPo
Morntha
0115:
SOFTS
2226
ITonsfommer
(2004)
PORITSST
2022
Choiffener
2422)
TinE
2023)
TiredNei
(2423)
(2223)
SCINe1
Stalocary
(2322)
Mirric
N15P
M/411
MSS
MAP
MCSP
MA.TT
KE5P
MAP
ME5T
MAATT
MAAF
MSSF
1A.E
5051
MA.TT
50511
MAI
0.350
3.344
0.335
0.395
0,335
0.361
0,508
0.325
0.365
0,404
0.425
0.364
0.567
0.375
0.345
0.372
I1415
0.458
0.356
0.396
192
08.677
11371
0.391
11335
0.299
113377
02.91
0.3300
0.984
11450
02.451
1139%
0.4404
11.344
02.987
14380
0.300
14439
03.58)
14429
0.464
336
0.400
0.391
0.406
0.113
0.405
0.412
0.325
0.400
0.400
2.410
0.533
0.515
0.428
0.425
0.310
0.411
0.413
0.413
0.650
0.485
0.855
0.464
1231
18.40)
0.446
11475
0.445
0.466
0.487
11.91
0.49
01475
0.435
11.86
0,500
04487
0.461
14.77
0.450
01.44
0.435
14595
0.550
0.395
0.316
0.398
0,317
0.260
01
0.263
0,380
0.261
01.80
0.264
0.184
0.364
01287
0.500
0.200
0.345
0.357
0.367
0.398
0.212
11256
0,577
0.392
0.214
195
0.342
0:311
(1.50)
1.9
0.208
3.385
0.250
0.50
0.315
0.305
11:14
0.007
07700
11.00
87000
0.764
0 0.667
11309
0.605
0.790
0.50
3.6
0.303
0.342
00312
0,343
0.35
0.332
01311
0.548
0.206
0,535
10.97
0,542
0.377
0.422
01321
0.351
0.350
0.427
0U637
0.591
0.334
0.361
1251
0..00)
8.799
11:11
0.196
0.405
0.411
01412
0.407
0.409
0.417
1730
1947
16.58
0.504
11:08
034013
0.554
11.00
0.417
0.413
0.281
07.58
0.282
2.300
0.756
0.550
0.782
0.390
0.757
0.356
0.004
0.291
0.553
0.350
0.001
0,571
0.306
0.567
55
1..75
0.705
11:36
0:315
11381
0:34
11336
0.44
0.796
0.986
11423
02.02
14479
0.064
11.88
0,4112
0.796
0.000
11856
(3:00
0.513
0.0M1
192
0.437
0.429
0.665
0.437
0.55
0.31
0.441
0.36
0.640
0.45
0.671
0.474
0.555
0.000
0.436
0.574
0.437
0.452
0.719
0.651
0.534
0.504
33h
0.472
6-221
1169
0-485
11403
0.42
0
B.491
0/422
11.70
0.500
16280
0.313
11491
03489
16461
6.09
1071
03809
0.318
0.55
0.676
0,50
0.491
0.485
0.653
0.594
0.531
0.519
0.76
0.543
0.616
0.438
0.234
0.455
0.43
0.440
0.442
0.454
0.447
0445
0:446
0.529
0,522
0.541
0.307
0.458
0.430
0.456
0.452
0.747
0.667
0,570
0.537
0.265
2.355
0220
0.348
0.297
0.547
0.397
0.59
0.388
6.381
0.155
0.581
0.00
0.600
0.300
0.354
0.33
0.
0.307
0.621
0.416
0.058
192
0.207
0.20
0.5%6
0,333
0.34
0.00
0.376
0.95
0.577
0.58
0.500
04422
0.414
0.477
0.476
0.850
0.512
0.110
0.420
0.45
1.304
0.544
1,349
01302
5-411
18391
0,915
0.575
0.331
11335
0/477
0.700
0410
11942
0.684
04511
0.50
16416
0.427
0.359
0.35
16986
0.723
0.336
0.316
0:300
0.100
0.143
0.23
0.290
0.164
0251
02319
0.37
0.20
0.272
0.197
0.347
0.360
0.273
0177
0.599
0.39
0.300
0.259
0.366
0.30
0.544
0.209
0.301
0.259
0.200
5299
11304
0.241
0.218
0.315
0317
0.330
0315
10330
0,363
11284
0.575
112200
0.300
01245
0.335
14299
0.398
01222
0.321
Avg
0.300
0.302
1.170
0.265
0.174
2.204
0.178
0.20
0.150
0.276
01244
0.334
0.251
0.344
0.392
0.205
0.212
0.500
0.258
0.365
0.355
0.2%
Eeclange
0.55
8.591
0118
0.911
0.176
0.911
0.299
0.376
0.550
01544
0.356
0.504
0.376
0.315
0.511
0.319
0.535
336
0.355
5414
0.352
0.488
0.329
3.46
0.331
0417
0,301
0.307
1258
0,583
0.340
0.431
0.367
0.448
0.313
0.427
1.324
0.553
0421
0.476
130
03017
11667
0.701
086
0.991
0167
0.911
0.001
0.714
1767
1800
01553
0.008
11964
0.796
01110
0.001
1.055
0.797
1.000
0.799
0.352
8,799
0.367
0.435
0.261
0/412
01250
0,413
0.761
0.444
11.40
0.007
0.310
0.413
11416
0.443
04354
0.414
14220
0.600
04451
0.454
0.106
8.307
0.385
0.210
0.106
0.315
0.374
02214
0.776
0.217
16358
0.203
11300
0.261
11:12
02:200
01396
0.205
11231
03:00
0.315
0.221
192
0.216
0.254
0.314
0.252
0.317
3.53
0.331
0.254
0.221
0.256
0.306
0.277
0.42
0.298
0.219
0.361
0.237
0.246
0.251
0.340
0.345
0.285
376
8.2%
11210
0.221
0.382
0.311
11.77
0.2%
11315
8.2%6
11312
01.00
18:37
0.304
11.300
0,330
11.353
02300
14300
0.3775
16323
0.333
720
01551
0.365
0.350
0.345
0.356
0.351
0.358
0.347
0.553
0.565
0.356
0.418
0.351
0.586
0.365
0.359
0.346
0.381
0.337
0.427
0.414
0.430
11302
Eeegg
0.19%
0.237
0.216
0.244
0.210
0.230
0.210
0.297
0.215
0.286
11310
0.351
11312
0.39
0.250
0.292
0.200
0.376
11227
0.344
0.215
0.299
195
0.231
0.567
0.37
0.770
0.329
8:251
03335
2.30
0337
0.367
01734
0.775
033
0.416
01306
0.318
03300
0 1.00
0.780
0. 6.300
0.751
0.777
236
0.250
0.291
0.158
0.285
0.243
8.309
0.248
0.75
0.250
0.276
10350
0.735
0.368
0.200
1.310
0.550
0.333
0.415
0L304
0.399
0.250
0.2%
170
0.285
0.785
01200
0.781
0.345
0.272
0.309
0.275
0.555
0.225
11789
0.770
0.425
0.378
0.717
0.776
0.415
01308
0.888
0.765
0.700
0.550
0.566
0.340
0.273
0.339
0.256
0.30
0.256
0.366
0.641
0.34
0.417
0.319
0.330
0.282
0.261
0.581
D. More ablation results
Ablation of the Hyper-Sean We explore the effectiveness of hyper-scan as shown in Table 7. Specifically, we remove the HyperMamba module as the baseline and set up several variants, including scanning in the variable dimension only (Var only), scanning in the time dimension only (Time only), and scanning in the variable dimension first and then in the time dimension (Time& & Var). The results show that both time only and var only improve the prediction performance of the baseline, which is also attributed to the powerful global dependency capture of the original Mamba. In addition, the combination of the two further reduces the MSE and MAE of the prediction. However, this structure does not model the relationship between time and variables in a single module, and thus performance is limited. In addition, it can lead to large computational costs that impair the efficiency of the model. In contrast, TimePro models more accurate variable dependencies through an efficient time tune strategy, requires only linear computational complexity, and achieves the best prediction performance.
We also provide some visualization results to validate the effectiveness of Hyper-Scan, as shown in Fig. 10. It can be observed that suboptimal results are obtained for the prediction curves of models that scan only in the time dimension or the variable dimension. Specifically, when scanning only in the variable dimension (Fig. a), the model's prediction curves are smoother, with a poorer fit the extremes. We attribute this to the model's lack of ability to capture the details of local changes within variables. When scanning in the time dimension only (Fig. b), the model's prediction ability for the extremes improves, but the average accuracy remains poor. This is due to the lack of capturing complex variable relationships. And when we use non-adaptive hyper-scanning (Fig. c), the model's average accuracy and predictive ability for extreme values are improved, but the performance is still unsatisfactory. And when we apply the adaptive hyper-scan, i.e., TimePro (Fig. d), the model can perceive both variable relationships and significant temporal information, resulting in a more accurate prediction performance.
13

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
Table 7: Ablation experiment of the Hyper-Scan on the Exchange and ETThl dataset. Results are averaged with horizon He {96, 192, 336, 720).
Exchange
ETTh1
Variant
MSE
MAE
MSE
MAE
Baseline
0.374
0,418
0,467
0,452
Time only
0.367
0.414
0.459
0.447
Var only
0.367
0.410
0.454
0.443
Time& & Var
0.360
0.404
0.450
0.439
TimePro
0.352
0.399
0.438
0.438
030
330
0.50
-0.50
-350
-5.0
CrevneTuAl
OroAneT A0
Prediction
Aradittion
1.0
130
(a) iTransformer
(b) S-Mamba
(c) TimePro
Figure 8: Comparison of forecasts between TimePro, S-Mamba and iTransformer on ETTh2 dataset when the input length 96 and the forecast length 96. The blue line represents the ground truth and the orange line represents the forecast.
Efficiency effect of variable channels We further explore the efficiency of TimePro with different variable channels in Fig. 11, which can be seen as complement to Table 1. It can be observed that as the number of variable channels increases, iTransformer shows quadratic increase in both the memory and inference time, which severely compromises the efficiency of the model and limits practical applications. In addition, PatchTST has unsatisfactory efficiency under all variable channels. contrast, the linear scaling ability compared to variable channels, small memory consumption and high inference speed validate TimePro's 's efficiency.
14

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
Groundhrmm
Grundhrur
Granndhuth
15.
15
13-
05.
3.-
0.-
2.-
50-
10-
10c
14-
25
50
100
225
150
175
200
23
50
130
125
150
175
200
25
52
100
325
150
375
208
(a) iTTansformer
(b) S-Mamba
(c) TimePro
Figure 9: Comparison of forecasts between TimePro, S-Mamba and iTransformer on ECL dataset when the input length is 96 and the forecast length 96. The blue line represents the ground truth and the orange line represents the forecast.
Prediccion
Prediction
0.25
-0.25
-0.25
-0.50
-0.50
0.75
0.75
1.00
1.00
25
50
100
125
150
175
200
50
75
100
125
150
175
200
(a) Var only
Time only
6.75
GroundTruth
0.75
Ground truth
Prediction
Prediction
0.50
0.50
6.25
175
125
(c) Non-Adaptive Hyper-Scan
(d) TimePro
Figure 10: Visual comparison of different Hyper-scan designs on ETTh2 dataset when the input length is 96 and the forecast length 96. Var only and Time only denote we only apply the scanning the variable and time dimension, respectively. Non-Adaptive Hyper-Scan denote we integrate the variables states along the time dimension using linear projection, i.c.. non-adaptive. The better prediction curves of TimePro prove the validity of Hyper-Scan 's structure.
15

---

TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
3000
TimePro(ours) Irannfomer PatchTST
TimeProlouns) Irassfomer Patch1ST
350
2003
Inferace Time (mxileed)
Memory (MB8)
15000
200
10000
1000
2000
3000
1000
1500
2000
2500
3000
3500
4000
4500
5000
Channels
of Channels
Figure 11: Memory and inference time of different methods. We set the lookback window L 96, forecast horizon 720 in synthetic dataset conduct. We set the batch size to 4 due the limited GPU memory. The inference times measured Nvidia V100 GPU. It reveals that TimePro scales to large number of channels more effectively than Transformer-basee models (i.e., iTransformer and PatchTST).
16


Reviews:
[{"review_id": "uncluY1tqx", "reviewer": "Reviewer_gQxg", "summary": "This paper proposes TimePro, a Mamba-based model for multivariate long-term time series forecasting. TimePro adaptively selects critical time points to refine variable states, preserving temporal granularity and capturing dynamic variable relationships. Experiments on various benchmarks show competitive performance with existing Mamba- and Transformer-based methods.\n\n\n## =================update after rebuttal==============\n\nThanks for the authors' responses. My concerns have been well addressed.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and efforts in reviewing our work and providing valuable feedback that can further strengthen our manuscript. We have added figures with more experiments following your suggestions and they are available at  https://anonymous.4open.science/r/Anonymous_figure-2319/figure_response.pdf. Please see our detailed responses to your comments below.\n\n### 1 computational overhead discussion\nWith reference to your comments, we have added two experiments, including Figures R1 and R2 in the figure response.\nIn Figure R1, we provide a detailed comparison of efficiency metrics including parameters, FLOPs, training time and inference time. It can be observed that TimePro possesses better computational efficiency than previous methods. Specifically, TimePro has a training time and inference time similar to S-Mamba and significantly outperforms other convolutional or transformer-based methods. For example, TimePro obtains lower prediction errors with 2.7 and 14.4 times the inference speed of PatchTST and TimesNet, respectively. Moreover, TimePro has the minimal parameters, FLOPs, and memory consumption. For example, TimePro requires only 67% parameters and 78% GFLOPs of S-Mamba. In addition to satisfactory efficiency, TimePro also outperforms recent advanced models including iTransformer and S-Mamba in forecasting performance. These results demonstrate TimePro's lightweight and suitability for deployment in a variety of real-world scenarios where resources are constrained.\n\nIn Figure R2, we further explore the efficiency of TimePro with different variable channels, which can be seen as a complement to Table 1 in our manuscript. It can be observed that as the number of variable channels increases, iTransformer shows a quadratic increase in both the memory and inference time, which severely compromises the efficiency of the model and limits practical applications. In addition, PatchTST has unsatisfactory efficiency under all variable channels. In contrast, the linear scaling ability compared to variable channels, small memory consumption and high inference speed validate TimePro's efficiency.\n\n### 2 visualization effect of Hyper-Scan\nWe provide some visualization results to validate the effectiveness of Hyper-Scan, as shown in Figure R6 in the figure response. It can be observed that suboptimal results are obtained for the prediction curves of models that scan only in the time dimension or the variable dimension. Specifically, when scanning only in the variable dimension (Fig. a), the model's prediction curves are smoother, with a poorer fit at the extremes. We attribute this to the model's lack of ability to capture the details of local changes within variables. When scanning in the time dimension only (Fig. b), the model's prediction ability for the extremes improves, but the average accuracy remains poor. This is due to the lack of capturing complex variable relationships. And when we use non-adaptive hyper-scanning (Fig. c), the model's average accuracy and predictive ability for extreme values are improved, but the performance is still unsatisfactory. And when we apply the adaptive hyper-scan, i.e., TimePro (Fig. d), the model can perceive both variable relationships and significant temporal information, resulting in a more accurate prediction performance.\n\nWe will add these experiments (i.e., Figures R1, R2, R6) and the corresponding analyses to the final version. Thanks again for your valuable comments!", "claims_and_evidence": "This paper claims TimePro achieves state-of-the-art results with linear complexity. Table 2 reports superior MSE/MAE across datasets. Complexity analysis (Table 1) confirms linear scaling with sequence length.", "methods_and_evaluation_criteria": "This paper follows standard evaluation criteria with prior methods.", "theoretical_claims": "NA.", "experimental_designs_or_analyses": "The experimental designs and analyses are technically sound.", "relation_to_broader_scientific_literature": "The work builds effectively on Mamba-based models for multivariate long-term time series forecasting.", "essential_references_not_discussed": "NA.", "other_strengths_and_weaknesses": "**Strengths**\n\n1.\tThis paper is well organized and well-written.\n\n2.\tThe idea of incorporating variable and time-aware superstate construction is reasonable and interesting.\n\n3.\tComprehensive ablation studies (e.g., feature dimensions, encoder layers) provide insights into design choices.\n\n**Weaknesses**\n\n1.\tLimited discussion on computational overhead (e.g., training time, inference time) compared to baselines.\n\n2.\tIs it possible to provide some visualization results for better understand the proposed modules?", "other_comments_or_suggestions": "See weakness.", "questions_for_authors": "NA.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XSUWAYClgF", "reviewer": "Reviewer_FUHr", "summary": "This paper proposes TimePro, a Mamba-based model for multivariate long-term time series forecasting. By introducing a hyper-state mechanism that adaptively selects critical temporal intervals, TimePro aims to address the multi-delay problem, where variables influence targets over heterogeneous time spans. Empirical results across various benchmarks demonstrate competitive performance compared to Mamba- and Transformer-based baselines, with claims of linear computational complexity.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and efforts in reviewing our work and providing thoughtful feedback that can further strengthen our manuscript. We have added some experiments following your suggestions and they are available at  https://anonymous.4open.science/r/Anonymous_figure-2319/figure_response.pdf. Please see our detailed responses to your comments below.\n\n### 1 real-world metrics are absent\nConcerning your comments, we have added two experiments, including Figures R1 and R2 in the figure response.\nIn Figure R1, we provide a detailed comparison of efficiency metrics including parameters, FLOPs, training time and inference time. It can be observed that TimePro possesses better computational efficiency than previous methods. Specifically, TimePro has a training time and inference time similar to S-Mamba and significantly outperforms other convolutional or transformer-based methods. For example, TimePro obtains lower prediction errors with 2.7 and 14.4 times the inference speed of PatchTST and TimesNet, respectively. Moreover, TimePro has the minimal parameters, FLOPs, and memory consumption. For example, TimePro requires only 67% parameters and 78% GFLOPs of S-Mamba. In addition to satisfactory efficiency, TimePro also outperforms recent advanced models including iTransformer and S-Mamba in forecasting performance. These results demonstrate TimePro's lightweight and suitability for deployment in a variety of real-world scenarios where resources are constrained.\n\nIn Figure R2, we further explore the efficiency of TimePro with different variable channels, which can be seen as a complement to Table 1 in our manuscript. It can be observed that as the number of variable channels increases, iTransformer shows a quadratic increase in both the memory and inference time, which severely compromises the efficiency of the model and limits practical applications. In addition, PatchTST has unsatisfactory efficiency under all variable channels. In contrast, the linear scaling ability compared to variable channels, small memory consumption and high inference speed validate TimePro's efficiency.\n\n### 2 lacks details in Fig. 3\nThanks to your comments, we have added some details in Figure R3 in the figure response. These details include: 1) textual explanations of GPU SRAM and HBM; 2) additions to the formulas in the grey boxes so that the reader can understand the initial state generation process; 3) some mathematical symbol additions such as the state h and hyperstate ho, which correspond to Eq. 10-13 in the manuscript; 4) optimizations of some process components to enhance aesthetics and clarity; 5) textual additions to the captions, which further enhances the reader's understanding of hyper-scan.\n\n### 3 More discussions of Fig. 4\nIn Figure 4, as the number of layers increases from 1 to 4, the prediction error of the model first decreases and then gradually increases. This is due to the fact that when the number of layers is 1, the model is shallow and can not capture complex time-varying and variable relationships. When the number of layers is large, for example, 3 or 4, the model suffers from overfitting, which impairs the model's generalizability to the test set, and therefore a slight increase in prediction error occurs.\n\nWe will add the above experiments and the corresponding analyses to the final version. Thanks again for your constructive suggestions!", "claims_and_evidence": "1. TimePro achieves state-of-the-art performance in multivariate long-term forecasting by adaptively modeling variable-specific temporal dependencies, which is justified by Table 2 and Fig. 1, where TimePro obtains superior MSE/MAE on various datasets.\n2. This paper claims that the proposed hyper-state mechanism effectively captures both variable interactions and intra-variable temporal dynamics. Ablation studies (Tables 3–5) demonstrate that combining bidirectional variable scanning and time-aware offset learning reduces MSE and improves the performance.", "methods_and_evaluation_criteria": "This paper aligns with standard protocols (MSE/MAE metrics) across various benchmarks (e.g., ETT, Weather).", "theoretical_claims": "NA", "experimental_designs_or_analyses": "The experimental framework is rigorous, incorporating comprehensive ablation studies (e.g., feature dimensions, encoder depth) and complexity analyses to isolate the contributions of key components.", "relation_to_broader_scientific_literature": "TimePro advances Mamba-based time series forecasting by addressing the multi-delay problem through dynamic time-aware hyper-states.", "essential_references_not_discussed": "NA.", "other_strengths_and_weaknesses": "**Strengths**\n\n1.\tThis paper is well written and the proposed method is technically sound.\n2.\tThe integration of bidirectional variable scanning with time-aware offset learning effectively captures dynamic variable-temporal interactions, addressing a clear limitation in existing Mamba variants.\n3.\tExtensive experiments on diverse datasets (e.g., ETT, Weather) demonstrate the effectiveness of the proposed method. The design choices are well justified by ablation studies.\n\n**Weaknesses**\n\n1.\tWhile theoretical complexity is analyzed (Table 1), real-world metrics (e.g., inference speed) are absent, which are important for justifying the advantages of the proposed method.\n2.\tThe hardware-aware implementation of hyper-scan (Fig. 3) lacks details and poses challenges for understanding this part.\n3.\tAs shown in Fig. 4, For the Exchange dataset, increasing the number of layers from 1 to 2 obtains slightly poorer performance. More discussions and insights are encouraged.", "other_comments_or_suggestions": "NA", "questions_for_authors": "NA", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9Mf5eLBgj4", "reviewer": "Reviewer_Kihx", "summary": "This paper introduces TimePro, a novel Mamba-based framework designed for multivariate long-term time series forecasting. The core contribution lies in its variable- and time-aware hyper-state mechanism, which dynamically refines hidden states by adaptively selecting critical temporal intervals to address the multi-delay problem. Extensive experiments across popular benchmarks demonstrate superior performance over existing methods.\n\n## update after rebuttal\n\nI will keep my score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and efforts in reviewing our work and providing thoughtful feedback that can further strengthen our manuscript. We have added some experiments following your suggestions and they are available at  https://anonymous.4open.science/r/Anonymous_figure-2319/figure_response.pdf. Please see our detailed responses to your comments below.\n\n### 1 Analysis of the improvement\nFirst, TimePro outperforms S-Mamba on seven datasets, and the improvement is significant on some datasets. For example, TimePro has a 0.009 and 0.015 improvement in MSE and MAE for the ETTh1 dataset, respectively. In addition, TimePro has a 0.011 and 0.008 improvement in MSE and MAE, for the Exchange dataset, respectively. These improvements fully validate the effectiveness of TimePro. \n\nSecond, as mentioned in Fig. 5 of the manuscript, TimePro can significantly benefit from longer lookback window lengths because it preserves more local details in the time dimension. So, when the lookback window length is 96, TimePro's improvement on the Weather and ETTm1 datasets is 0.001 and 0.006, respectively. However, when the lookback window length is 336, the improvement is 0.008 and 0.01, respectively. This improvement is significant and further validates TimePro's potential. Finally, we provide a detailed efficiency analysis in Figure R1 in the figure response. It shows that TimePro has a better efficiency performance compared to S-Mamba. In summary, TimePro strikes a better balance between efficiency and performance than S-Mamba.\n\n### 2 details for this statement\nAs shown in Figure R3 of the figure response, we add some details in Fig. R3 and modify the corresponding caption. These details include: 1) textual explanations of GPU SRAM and HBM; 2) additions to the formulas in the grey boxes so that the reader can understand the initial state generation process; 3) some symbolic additions such as the state $h$ and hyperstate $h_o$, which correspond to Eq. 10-13 in the manuscript; 4) optimizations of some process components to enhance aesthetics and clarity; 5) textual additions to the captions, which further enhances the reader's understanding of hyper-scan. In addition, given that the final version has an extra page, we will add a more detailed description in Sec. 4.2. I hope these modifications can solve your confusion.\n\n### 3 Fig. 6 Blurring\nI'm sorry that Figure 6 isn't clear enough. We have uploaded the image in PDF format as shown in Figure R4 of the figure response. In addition, we additionally upload the visualization comparison experiment on the ECL dataset in PDF format as shown in Figure R5. The clarity of these images is guaranteed. It can be observed that the prediction curve of TimePro is much closer to the groundtruth. This further validates the effectiveness of TimePro.\n\nWe will add the above modifications and the corresponding analyses to the final version. If the rebuttal helps address your concerns, we kindly ask that you increase your score to give TimePro a fighting chance!", "claims_and_evidence": "The claims are supported by evidence.", "methods_and_evaluation_criteria": "The proposed method is evaluated on five widely used real datasets, including ETT (4 subsets), Exchange, Electricity, Weather, and Solar-Energy.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Reasonable experimental designs and analyses.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "**Strengths**\n\n1.\tThe proposed method is technically sound.\n\n2.\tExtensive experiments across popular benchmarks demonstrate superior performance over existing methods.\n\n3.\tDetailed ablation studies justify the design choices. \n\n**Weaknesses**\n\n1.\tI am not sure that the improvement over S-Mamba is significant enough, e.g., 0.251 vs. 0.250 for Weather and 0.398 vs. 0.392 for ETTm1.\n\n2.\tIn Fig. 3“It consists of two parts, including plain state acquisition in the GPU SRAM and time tuning in the GPU HBM.” However, there are no details for this statement and it is not easy to understand.", "other_comments_or_suggestions": "Figure 6 is insufficiently clear, making it difficult to discern critical details.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v3AyfshBWs", "reviewer": "Reviewer_1d3n", "summary": "This paper introduces TimePro, a model designed for multivariate long-term time series forecasting, but it is marred by significant writing issues. The main problem lies in the overly complex and unclear explanations. The terminology used, such as \"variable- and time-aware hyper-states,\" is vague and confusing, making the methodology difficult to follow, especially for readers not already familiar with the specific framework. The paper overcomplicates simple ideas with jargon that could have been explained more clearly, detracting from its overall accessibility and readability. Additionally, there is a lack of coherence between sections, with abrupt shifts in focus that make the paper feel disjointed. The presentation of the model and its components lacks clarity, and the descriptions of the model's workings are not intuitive, making it harder for readers to understand the core innovations. The overall structure feels cumbersome, and a more straightforward and concise approach would significantly improve the readability and impact of the paper.\n\n## Update after rebuttal\nI checked the rebuttal, and most of my concerns are addressed. Therefore, I raise my score from 2 to 3.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and efforts in reviewing our work and providing thoughtful feedback that can further strengthen our manuscript. We have added some experiments following your suggestions and they are available at  https://anonymous.4open.science/r/Anonymous_figure-2319/figure_response.pdf. Please see our detailed responses to your comments below.\n### 1 Multi-delay issue definition\nWe have defined the multi-delay issue in both the abstract and paragraph 3 of the introduction. To make this definition strict, we modify the definition as follows.\nThe multi-delay issue in multivariate time series forecasting is defined as the temporal discrepancy in the propagation of influence from different predictor variables to the target variable, characterized by the presence of distinct and non-uniform time lags between the changes in predictor variables and their corresponding effects on the target variable.\n### 2 Logical connection\nTo effectively address the multi-delay issue, it is necessary for the model to identify and capture the critical temporal points of each variable within the input time series data. However, traditional models uniformly process different time points of the same variable, thereby causing the key temporal information to be obscured by a vast number of irrelevant time series features. Therefore, the traditional model can not solve the multi-delay issue. We will revise the above content into the paper.\n### 3 Addressing multi-delay issue\nWe have detailed how TimePro solves the multi-delay problem in paragraph 4 of introduction. During the scan of mamba, a specialized network is employed to learn the offsets of critical time points. By adaptively selecting these key time points, TimePro dynamically updates the hidden states to reflect the most salient temporal information. Through adaptive selection of key temporal features, the model focuses more on the delay times of each variable, thus solving the multi-delay problem.\n### 4 Excessive Mamba detail\nI'm sorry for your concerns. Taking your suggestion into consideration, we will revise the first section to mainly focus on the advantages of Mamba in time-series forecasting, thereby better aligning with the theme of time-series forecasting. In the second paragraph we introduce some Mamba-based works that is relevant to our approach, which can facilitate the reader to understand our contribution and differences from previous work. We will simplify the descriptions of these methods and enhance the summarization of the methods in relation to the theme. We will revise the last sentence of the second paragraph as follows: The mentioned related works often employ different ways to scan features from various directions. However, these methods overlook the fact that different variables have different impact durations on the target variable, which limits their performance.\n### 5 Reason of Transformer-based methods introduction\nAlthough TimePro is based on Mamba, transformer-based models dominate the time series community. Therefore, we need to describe these models in the introduction and motivate the reader to understand the shortcomings of these models. As we mention in lines 59-66 of our paper, transformer-based models also can’t capture critical time points. We believe that using some space to describe this work is necessary to help make our motivation clearer.\n### 6 Clarify inputs and outputs\nThe inputs and outputs of the model refer to arbitrary sequences, as shown in lines 145-146 of our paper. Considering the complex mechanism of Mamba, we need to depict the working mechanism of Mamba before introducing our TimePro (i.e., Preliminaries), which facilitates the understanding of our approach. This writing structure is also adopted by other Mamba-based models, including VMamba (NeurIPS2024), ViM (ICML2024) and S-Mamba.\n### 7 Lacking explanations of the motivations\nWe present our motivation before introducing our core modules (HyperMamba and Hyper-Scan), as shown in lines 227-241 and lines 220-229. In addition, we also introduce the motivation for Time and variable preserved embedding, as shown in lines 191-197 of our paper. For other parts such as ProBlock and Linear projection, we do not present the motivation behind them as they do not involve architectural innovations. We will be more specific in describing the motivation before each module description in the final version.\n### 8 Supplementary experiments\nWe have provided some other experimental results in the figure response, including R1: Efficiency comparison, R2: Memory and inference time with different channels, etc. In addition, we will put Fig. R1 and R2 into the final version to increase the space of the comparison experiment. Other experiments will be placed in the appendix.\n\nWe will follow your suggestions for the writing to reorganize our final version. If the rebuttal helps address your concerns, we kindly ask that you increase your score to give TimePro a fighting chance!", "claims_and_evidence": "The claim of this paper are not supported by clear evidence. See \"Other Strengths and Weaknesses\" for details.", "methods_and_evaluation_criteria": "The proposed method simply applies Mamba to time series forecasting, and the improvements in various modules lack motivation.", "theoretical_claims": "The paper does not contain any theoretical claims.", "experimental_designs_or_analyses": "The paper includes some ablation experiments; however, due to the lack of motivation behind the module design, these ablation experiments appear meaningless.", "relation_to_broader_scientific_literature": "This paper merely applies the Mamba tool to time series forecasting, making the paper seem incremental.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "## Strengths：\n\nThe paper presents extensive experiments on several real-world datasets, validating the model’s robustness across different domains and prediction horizons.\n\n## Weakness:\n\n1. The \"multi-delay issue\" should be given a strict definition rather than a simple description, as it is the core problem addressed by the paper.\n\n2. The authors claim in the abstract, \"Traditional models typically process all variables or time points uniformly, which limits their ability to capture complex variable relationships and obtain non-trivial time representations.\" How does this statement logically connect to the previously mentioned \"multi-delay issue\"? It feels like there is a lack of logical linkage between these two sentences, making the paragraph appear confusing.\n\n3. The description of the method in the introduction and abstract does not seem to address the \"multi-delay issue\" proposed by the authors, which makes the proposed method seem meaningless.\n\n4. In the first and second paragraphs of the introduction, a large portion of the content focuses on the Mamba model, which is not directly related to the research problem (time series forecasting). This deviates from the main topic and wastes space that should focus on the challenges of time series forecasting and the contributions of this research. While Mamba provides relevant background value, the excessive detail introduced does not directly relate to the core issue of this paper and may confuse readers, affecting the overall coherence and academic rigor. The authors should consider streamlining this content and focusing more on the research background that is directly related to the paper's goals, thus better highlighting its innovation and practical significance.\n\n5. Why introduce Transformer-based time series forecasting methods in the third paragraph of the introduction when this paper is based on a Mamba-based time series forecasting method?\n\n6 . When introducing \"Preliminaries\" in Section 3, please first clarify what the inputs and outputs are, as this will prevent confusion for readers who are not familiar with Mamba and time series forecasting.\n\n7. The method section of the paper reads like code documentation, lacking explanations of the motivations behind the module design.\n\n8 . The experimental section spends nearly 80% of its space on ablation experiments, leaving very little room for other important information. This makes the experimental section seem sparse.", "other_comments_or_suggestions": "See \"Other Strengths and Weaknesses\".", "questions_for_authors": "See \"Other Strengths and Weaknesses\".", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces TimePro, a Mamba-based model designed for forecasting multivariate long-term time series. It uses a hyper-state mechanism to adaptively choose important temporal intervals, targeting the multi-delay issue where variables affect targets across different time spans. Experiments on multiple benchmarks show that TimePro performs comparably to Mamba- and Transformer-based baseline models, and it is claimed to have linear computational complexity.

All reviewers agree to accept this work. One reviewer has raised his score (from weak reject to weak accept) and all problems have been solved.

Thus, the AC recommend‌s acceptance of this work.
Decision: Accept (poster)