Temporal Query Network for Efficient Multivariate Time Series Forecasting
Authors: Shengsheng Lin
Haojun Chen
Haijie Wu
Chunyun Qiu
Weiwei Lin
Venue: ICML 2025 poster (2025)
Keywords: Multivariate time series forecasting
correlation modeling
time series analysis
machine learning
Primary area: applications->time_series
TL;DR: This paper introduces Temporal Query Network (TQNet), a lightweight model with a novel Temporal Query technique that efficiently captures multivariate correlations.

Abstract: Sufficiently modeling the correlations among variables (aka channels) is crucial for achieving accurate multivariate time series forecasting (MTSF). In this paper, we propose a novel technique called Temporal Query (TQ) to more effectively capture multivariate correlations, thereby improving model performance in MTSF tasks. Technically, the TQ technique employs periodically shifted learnable vectors as queries in the attention mechanism to capture global inter-variable patterns, while the keys and values are derived from the raw input data to encode local, sample-level correlations. Building upon the TQ technique, we develop a simple yet efficient model named Temporal Query Network (TQNet), which employs only a single-layer attention mechanism and a lightweight multi-layer perceptron (MLP). Extensive experiments demonstrate that TQNet learns more robust multivariate correlations, achieving state-of-the-art forecasting accuracy across 12 challenging real-world datasets. Furthermore, TQNet achieves high efficiency comparable to linear-based methods even on high-dimensional datasets, balancing performance and computational cost. The code is available at: https://github.com/ACAT-SCUT/TQNet.

Temporal Query Network for Efficient Multivariate Time Series Forecasting
Shengsheng Lin Haojun Chen
Haijie Wu
Chunyun Qiu
Weiwei Lin 12
Abstract
(a) Global Correlation
[b] Learned Correlation of TQ
Sufficiently modeling the correlations among vari- ables (aka channels) is crucial for achieving accu- rate multivariate time series forecasting (MTSF). In this paper, we propose a novel technique called Temporal Query (TQ) to more effectively cap- ture multivariate correlations, thereby improv- ing model performance in MTSF tasks. Tech- nically, the TQ technique employs periodically shifted learnable vectors queries in the atten- tion mechanism to capture global inter-variable patterns, while the keys and values are derived from the raw input data to encode local, sample- level correlations. Building upon the TQ tech- nique, we develop a simple yet efficient model named Temporal Query Network (TQNet), which employs only a single-layer attention mecha- nism and lightweight multi-layer perceptron (MLP). Extensive experiments demonstrate that TQNet learns more robust multivariate correla- tions, achieving state-of-the-art forecasting accu- racy across 12 challenging real-world datasets. Furthermore, TQNet achieves high efficiency comparable baser- methods even on high- dimensional datasets, balancing performance and computational cost. The code is available at: https///iitbb.com//AXXtSSCUTTTpp
88
14
14.
04
04
02
00
-1
30
30-
Sample Correlation
Extreme Point
1.0
088
15
Missing Value
MM
16
-0
10
-07
Noise Disturbances
-00
Figure I: Comparison of inter-variable correlation patterns on the Traffic dataset. (a) Global correlations computed across the entire training set; (b) Correlations learned by the proposed TQ technique; (c) Correlations from individual samples. which may be distorted by non-stationary distur- bances such as extreme values, missing data, and noise. The comparison between panels (a) and (b) shows that TQ effec- tively captures global correlation structures, while panel (c) illustrates the instability of sample-level correlations.
1. Introduction
inter-variable correlations, as these correlations significantly influence the quality of predictions (Wang et al., 2024c).
Effectively modeling correlations among variables (aka channels) is crucial for multivariate time series forecasting (MTSF), a task that plays critical role in various real-world applications, including energy planning, traffic monitoring, medical forecasting, and climate modeling (Qiu et al., 2024; Wang et al., 2024b; Wen et al., 2023; Luo et al., 2024). The accuracy of MTSF relies heavily on capturing precise
However, accurately identifying the correlation patterns among multivariate input sequences is inherently challeng- ing, particularly when the sequences are affected by non- stationary disturbances, such as extreme values, missing data, and noisy perturbations (Huang et al., 2023). As illus- trated in Figure (a) and (c), these disturbances can create significant discrepancies between the variall corre- lations observed in individual samples and the global cor- relations derived from the entire training set. Bridging the gap between local (sample-level) and global (dataset-level) correlation patterns is essential for improving predictive performance (Cai et al., 2024).
'School of Computer Science and Engineering, South China University of Technology. Guangzhou 510006, China PPnggheng Laboratory, Shenzhen 518066, China. Correspondence to: Weiwei Lin <linww @ scut.edu.cn2..
Proceedings of the 4dnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
To address this issue, we propose the Temporal Query (TQ)

---

TQNet for Efficient Multivariate Time Series Forecasting
technique, which integrates global and local correlations within the attention mechanism. It uses periodically shifted learnable vectors as queries to emphasize stable, global de- pendencies across variables, while the raw imput sequences serve as keys and values to capture instance-specific (i.e., local) relationships. The TQ technique offers two key ad- vantages: its learnable nature enables adaptive modeling to capture optimal representations of variaba relation- ships, while its periodic time-shifting mechanism facilitates parameter reuse, resulting in more robust and stable cor- relation learning. As shown in Figure 1 (a) and (b), the correlations of queries captured by the learned TQ align more closely with the global correlations of the training set, enabling the model to establish stable dependencies during prediction and achieve more accurate forecasts.
advances reflect a transformation in the strategies for mod- eling multivariate dependencies, which can be categorized as follows:
Channel Mixing (CM): CM-based methods treat each time step as an input unit, meaning that multiple observed val- ues (i.e., variables) at a given time step are mixed and em- bedded together. This approach is adopted by many early Transformer-based models (Vaswani et al., 2017), such as LogTrans (Li et al., 2019), Informer (Zhou et al., 2021), Aut- oformer (Wu et al., 2021), FEDformer (Zhou et al., 2022), ETSformer (Woo et al., 2022), and Pyraformer (Liu et al., 2021). However, such methods often make variable relation- ships indistinguishable within the model (Liu et al., 2024c). As a result, even with their large parameter sizes, these models sometimes fail to outperform simple linear-based baselines (Zeng et al., 2023; Lin et al., 2024a; Toner & Darlow, 2024).
Building upon the strong capability of the TQ technique in modeling inter-variable correlations, we propose sim- ple yet effective multivariate forecaster, termed Tempo- ral Query Network (TQNet). TQNet consists of only a single-layer multi-head attention (MHA) mechanism and shallow multi-layer perceptron (MLP). Extensive exper- iments demonstrate that TQNet achieves state-of-the-art performance on 12 real-world multivariate datasets by effec- tively capturing robust variable dependencies. More- over, its lightweight architecture enables TQNet to maintain high computational efficiency comparable to lnnedrbaaeed methods (c.g., DLinear (Zeng et al., 2023)), even on high- dimensional datasets, while delivering superior forecasting accuracy.
Channel Independence (CI): To address the limitations of CM-based methods, PatchTST (Nie et al., 2023) introduced the concept of channel independence (CI), This strategy sep- arates the modeling of each variable and uses a parameter- sharing unified model make independent forecasts for each channel. Although these methods reduce the capac- ity for modeling vater-variable dependencies, they enhance model robustness, which is often more critical for time se- ries forecasting tasks (Han et al., 2024b). Consequently, CI-based methods, including SegRNN (Lin et al., 2023), TiDE (Das al., 2023), N-HiTS (Challu et al., 2023), FITS (Xu et al., 2024), STID (Shao et al., 2022), TimeMixer (Wang al., 2024a), PDF (Dai et al., 2024), CATS (Kim et al., 2024), Sparse (Lin et al., 2024a), and CycleNet (Lin et al., 2024b), have seen widespread adoption in recent years. Nonetheless, their lack of explicit inter-variable rela- tionship modeling has become a bottleneck, limiting their performance in multivariate scenarios.
In summary, the contributions of this paper are as follows:
We propose novel Temporal Query (TQ) technique, which models global inter-variable correlations within the attention mechanism using periodically shifted learnable vectors as queries.
Based the TQ technique, we introduce TQNet, simple yet powerful model that effectively captures multivariate and temporal dependencies using only a single-layer multi-head attention mechanism and a shallow MLP. This lightweight architecture ensures TQNet achieves efficiency comparable to linear-based models across datasets of varying dimensions.
To overcome the limitations of CI-based methods, researchers have developed advanced techniques to model inter-variable dependencies. These methods often treat segments of time series or entire chan- nels as input units, or employ advanced mechanisms to bet- ter understand multivariate relationships. Such approaches include various attention- based methods, such as TimeXer (Wang et al., 2024c), iTransformer (Liu et al., 2024c), Cross- former (Zhang & Yan, 2023). CARD (Xue et al., 2024), SAMformer (Ilbert et al., 2024), Leddam (Yu et al., 2024) UniTST (Liu et al., 2024a), SSCNN (Deng et al., 2024), TimeBridge (Liu et al., 2024b), and DUET (Qiu et al., 2025).
Channel Dependence (CD):
Extensive experiments on 12 challenging real-world multivariate datasets demonstrate that TQNet achieves overall state-of- sf-te-of-the-rr performance by extracting ro- bust inter-variable correlations.
Additionally, other paradigms have also been proposed to tackle this challenge. These include graph neural networks (e.g., CrossGNN (Huang et al., 2023), FourierGNN (Yi et al., 2023)), convolutional neural networks (e.g.. MICN (Wang et al., 2023), TimesNet (Wu et al., 2023), Mod-
2. Related Work
In recent years, deep learning- based approaches have made significant progress in MTSF tasks (Wang et al., 2024b; Qiu et al., 2024; Shao et al., 2024; Lin et al., 2025). These
2

---

TQNet for Efficient Multivariate Time Series Forecasting
emTCN (Luo & Wang. 2024)), and MLP-based solutions (e.g., LighTTS (Zhang et al., 2022), MTSSMixers (Li al.,
parameters are initialized to zeros and dynamically updated during training to adaptively capture the underlying correla- tions among variables.
2023b). TSMixer (Chen et al., 2023; Ekambaram et al., 2023), HDMixer (Huang et al., 2024), SOFTS (Han et al., 2024a)). These methodologies demonstrate diverse and ef- fective ways to capture and model the intricate dependencies among multivariate time series.
Herein, the hyperparameter W determines the length of the learnable vectors and the interval for periodic shifts. Its value should align with the stable periodicity of the dataset, which can be identified either through prior knowl- edge about the data (Lin et al., 2024b) or via computational methods, such the autocorrelation function (ACF) (Mad- sen, 2007). In Section 4.3, we will provide a detailed dis- cussion on the criteria for selecting W and its influence on the performance of the TQ technique.
In this paper, we focus on attention- based approaches within the scope of CD-based methods and aim to design min- imalist yet efficient network for MTSF. Inspired by the periodicity modeling in CycleNet (Lin et al., 2024b), which employs learnable parameters to capture period patterns, we propose the Temporal Query (TQ) technique. The TQ technique introduces periodically shifted learnable vectors as query inputs to the attention mechanism, enabling the model to incorporate inherent periodic structures in the data while more effectively capturing robust inter-variable rela- tionships.
T'Q-Enhanced Multi-Head Attention (TQ-MHA) To effectively capture multivariate dependencies, TQNet em- ploys a single-layer multi Malti-head attention (MHA) mechanism enhanced by the proposed TQ technique. Conventional self- attention mechanisms, such as those in i'Transfommer (Liu et al., 2024c), derive the queries, keys, and values (Q, K, V) directly from the input sample sequences. However, as discussed earlier, non-stationary disturbances in real-world data, such as noise or extreme values, can impede the ac- curate modeling of correlations within individual samples. To address this, we propose TQ-enhanced MHA mecha- nism, where the queries are generated from the TQ vectors to capture global correlations:
3. Methodology
The goal of multivariate time series forecasting (MTSF) is to predict future sequences Y, E Cx based on observed historical sequences Xx E RCLL at time step f, where C represents the number of variables (or channels), H is the forecasting horizon, and L is the length of the look- back window. To address the MTSF task, the workflow of the proposed Temporal Query Network (TQNet) is illustrated in Figure 2, with the corresponding pseudocode provided in Appendix A.
Q = Q= E RXL
Here, 6% is a segment of the learnable parameters 0ro RCxW, extracted periodically for each input sample. Specif- ically, for given time step f, the starting index is computed as mod W, and segment of length L is cyclically se- lected from Örq (as shown in Figure 3). This implies that for samples spaced W time steps apart, the extracted TQ vectors will remain identical, which can be expressed as:
3.1. Overview of TQNet
TQNet consists of a series of simple yet essential compo- nents. Specifically, given an input sequence X, E RCXL the temporal query- query-enhanced multi-head attention (TQ- MHA) layer is first applied to capture multivariate correla- tions. This is followed by a shallow multi-layer perceptron (MLP), which models temporal dependencies. Both the TQ- MHA and MLP are enhanced with residual connections (He et al., 2016) to improve learning stability. Finally, a linear layer with Dropout (Srivastava et al., 2014) is employed to project the leamed hidden representations onto the target outputs Y, E RC*H Below, we detail the key components of TQNet.
0%
(l(+ TO
i.N.
(2)
The periodic shifting mechanism ensures that 026 cycles periodically over time, aligning with the prior knowledge of periodic variations in real-world data (Lin al., 2024b). This mechanism also facilitates efficient parameter reuse, enabling the modeling of more robust sequence correlations by mitigating (i.e., averaging) the impact of localized noisy perturbations. By integrating TQ into the MHA mechanism, the attention process for each head h becomes:
3.2. Components of TQNet
Temporal Query Technique The Temporal Query (TQ) technique uses periodically shifted learnable vectors as queries in the attention mechanism to capture global inter- variable correlations (see Figure 3). Mathematically, given the periodic length W of the dataset, we initialize OTQ E RCXW which serves as a set of learnable parameters. These
Kn VI
Head Softmax
Vh.
(3)
where Qn = 0% w K₁, = X,W and V/2 = =X,WY. h
After computing the attention outputs for all heads, the
3

---

TQNet for Efficient Multivariate Time Series Forecasting
X IR
9, E R
Mulli-Hadd Helli-HHdd Altention (TQ-MHAA)
Instance Normalization (IN))
Temporal Query-E E nhhanced
Multi- Layer Perceptron (MLL P)
Look Length
www M
Outpult rojection
Instance De-
Mmmm
ForecastHorizon H
C Channele
Figure 2: The architecture of the proposed TQNet model. It comprises the lightweight and efficient TQ-MHA and MLP modules, with an optional module to mitigate distributional drift.
in TQNet consists of two fully connected layers interspersed with GeLU activations (Hendrycks & Gimpel, 2016), de- fined as:
Learnale Temporal Query
W
mod
hmp Linear (innarr(ee.U(LLneea((aaa))))
(5)
Multi-Head Attention
K
where hatm represents the output of the TQ-MHA module.
Extract
The output of the MLP is fed into a linear layer, optionally with Dropout (Srivastava et al., 2014), to project the learned representations onto the target forecasting horizon:
Output Projection
X2 CXL
CxL
rTQ E C
Figure 3: Temporal Query-enhanced Multi-Head Attention (TQ-MHA). The proposed TQ technique employs periodi- cally shifted learnable parameters as queries to adaptively model inter- variable correlations within individual samples.
Y, Linear (hooaa(Dropou((hmn)))
(6)
Instance Normalization (IN) Distributional shifts are common in real-world data and significantly impact the generalization performance of models. To address this challenge, recent research has explored various approaches, such as RevIN (Kim et al., 2021) and FAN (Ye et al., 2024). In this paper, we adopt a simple yet effective IN method that used in i'Tranffrmmer (Liu et al., 2024c) and CycleNet (Lin et al., 2024b), which involves removing the mean and vari- ance of the input data before and after the model's operation:
MHA mechanism concatenates the outputs:
MHA(Q, V) Concat(head head W (4)
where WO is the output projection matrix of MHA module. By incorporating 0'0 as query vectors, the TQ-enhanced MHA mechanism fuses global priors with local input fea- tures, enabling more robust modeling of vatealla cor- relations. Specifically, the periodic design of 0f captures stable global patterns (as shown in Figure ((b),, while keys and values from X, retain sample-specinf information (as shown in Figure (c)).. This design improves resilience to noise and missing data by balancing global consistency and local adaptability.
X, = X, - P
V +E
Y, = Y, x Voz+etN
where μ and o² denote the mean and variance of the input data, and is a small constant added for numerical stability.
4. Experiments
Multi-Layer Perceptron (MLP) Following the TQ- enhanced attention mechanism, lightweight MLP is em- ployed to model temporal dependencies within the sequence. Previous studies have demonstrated the robustness of MLPs in extracting temporal features (Li al., 2023a). The ML.P
4.1. Setup
The experiments this section are implemented using Py- Torch (Paszke et al., 2019), with the models optimized using 1.2 loss and evaluated based on both Mean Squared Error

---

TQNet for Efficient Multivariate Time Series Forecasting
(MSE) and Mean Absolute Error (MAE). Further details about the experimental setup can be found in Appendix A.2. The datasets and baseline models used for evaluation are outlined as follows.
earlier, these methods rely entirely on self-attention to ex- tract dependencies, making them more vulnerable noisy disturbances in input samples. This susceptibility limits their potential for further improvements in predictive perfor- mance. Conversely, methods like PatchT'TT and DLinear, which are CI-based approaches lacking explicit mechanisms for modeling inter-variable dependencies, exhibit subopti- mal performance on nigh-dimensional multivariate datasets.
Table 1: Detailed information about the datasets. The hyper- parameter W of TQ technique configured to align with the stable cycle length of each dataset, following the guide- lines in CycleNet (Lin et al., 2024b).
Overall, despite the simplicity of TQNet's architecture, it achieves superior performance. This significant improve- ment is primarily attributed to the TQ technique, which effectively enhances the model's ability to capture com- plex multivariate correlations. In the following sections, we delve deeper into the contributions of the TQ technique in modeling multivariate dependencies.
Dataset
Channels
Timesteps
Interval
HParam.
W
Domain
ETTh1
14,400
hour
24
Electricity
ETTh2
14,400
hour
24
Electricity
ETTml
$7,600
mins
Electricity
ETTm2
57,600
mins
Electricity
Electricity
321
26,304
hour
168
Electricity
Solar
137
52,560
10 mins
144
Energy
Traffic
862
17,544
hour
168
Transportation
Weather
52,696
10 mins
144
Weather
4.3. Ablation Studies and Analysis
PEMS03
358
26,208
mins
288
Transportation Trunsportation Transpotation Transportation
PEMS04
307
16,992
mins
288
A Ablation Study on the Design of TQ-MHA The goal of the proposed TQ technique to better integrate both global and local correlations within the attention mechanism. Specifically, the queries Q are generated from periodically shifted learnable vectors θrQ. while the keys K are derived from the raw input sequences. To examine the effects of global versus local correlations, we compare the following three settings:
PEMS07
883
28,224
mins
288
PEMS08
170
17,856
mins
288
Datasets We evaluate the proposed method on 12 widely used real-world datasets, including the ETT series (Zhou et al., 2021), PEMS series (Liu et al., 2022a), Electricity, Solar, Traffic, and Weather datasets (Wu et al., 2021). These datasets vary in scale, dimensionality (i.e., the number of variables), frequency, and domain. Detailed information about the datasets is provided Table 1.
1. Both Q and K are generated from raw data, capturing only per-sample correlations. This is the conventional approach used in models such as iTransformer and TimeXer.
Baselines To evaluate the performance of the proposed TQNet, we compare it against several representative mod- els from recent years, including TimeXer (Wang et al., 2024c), CycleNet (Lin et al., 2024b), iTransformer (Liu et al.. 2024c), MSGNet (Cai et al., 2024), TimesNet (Wu et al., 2023), PatchTST (Nie et al., 2023), Crossformer (Zhang & Yan, 2023), DLinear (Zeng et al., 2023), SCINet (Liu et al., 2022a). Following the setting of i'Transformer, TQNet defaults to a look-back length of 96.
2. Q is generated from the learnable TQ vectors, while K is derived from the raw data. This enables the at- tention score computation (i.e., QK vL to incorporate both global and samplc-specific correlations. This cor- responds to the design used in our TQNet.
3. Both Q and K are generated from the learnable TQ vec- tors, thereby focusing the attention mechanism solely on global correlations, ignoring local variations.
4.2. Main Results
Table 3 presents the average performance across four fore- casting horizons on large-scale multivariate datasets (those with more than 100 variables). As shown, the TQNet strat- egy (i.e., both global and per-sample correlations are con- sidered) yields the best performance. This is followed by the setting that considers only per-sample correlations, and then the purely global setting. These results validate that proper integration of both global and local correlations is indeed beneficial for multivariate forecasting.
Table 2 compares the forecasting errors of TQNet with base- line models across 12 real-world datasets. Lower MSE and MAE values indicate higher forecasting accuracy. TQNet consistently achieves Top 2 performance in 22 out of 24 fore- casting error metrics, demonstrating overall state-of-the-art accuracy, Notably, TQNet excels in high-dimensiona mul- tivariate datasets, such as Electricity and PEMS (over 100 variables). This highlights TQNet's advantages in handling complex multivariate data.
In addition, iTransformer and TimeXer, which are also ad- vanced attention- based methods, exhibit strong performance in complex multivariate scenarios. However, as mentioned
Ablation Study on TQNet Components To further in- vestigate the individual contributions of TQNet's compo- nents, we conduct ablation experiments on high-dimensional
5

---

TQNet for Efficient Multivariate Time Series Forecasting
Table 2: Comparison of multivariate time series forecasting results across 12 real-world datasets. The reported results are averaged over all prediction horizons H E [96, 192, 336, 720), with detailed results available in Table 5. The look-back length I uniformly fixed at 96. The best results are highlighted in bold, the second best are underlined, and the Count row counts the number of times each model ranks in the top
TQNet
TimeXer (2024c)
CycleNet (2024b)
iTransoommrr (2024c)
MSGNet (2024)
TimesNet (2023)
PatchTST (2023)
Crossformer (2023)
DLinear (2023)
SCINet (2022a)
Model
(Ours)
Metric
MSE
MAE
MSE
MAE
MSB
MAE
MSE
MAE
MSE
MAB
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
ETTh
0.441
0.434
0.437
0.437
0.457
0.441
0.454
0.448
0.453
0.458
0.450
0.469
0.455
0.529
0.522
0.456
0.452
0.747
0.647
ETTh2
0.378
0.402
0,368
0.396
0.388
0,409
0.383
0.407
0.413
0:427
0,414
0.427
0.387
0,407
0.942
0.684
0.559
0.515
0,954
0.723
ETTml
0.377
0.393
0.382
0.397
0.379
0.396
0.407
0.410
0.400
0.412
0.400
0.406
0.387
0.400
0.513
0.495
0.403
0.407
0.486
0.481
FTTm2
0.277
0.323
0,274
0.322
0.266
0.314
0.288
0.332
0.289
0.330
0.291
0.333
0.281
0.326
0.757
0.611
0.350
0.401
0.571
0.537
Electricity
0.164
0.259
0.171
0.270
0.168
0.259
0.178
0.270
0.194
0.301
0.193
0.295
0.205
0.290
0.244
0.334
0.212
0.300
0.571
0.537
Solar
0.256
0.237
0.210
0.261
0.233
0.262
0.292
0.319
0.270
0.307
0.641
0.330
0.401
0.282
0.375
Traffic
0.445
0.2776
01.66
0.287
0.472
0.301
0.428
0,282
0.660
0.382
0.620
0.336
0.481
0,300
0.550
0.304
0.625
0.383
0.804
0.509
Weather
0.242
0.269
0.241
0.243
0.271
0.258
0.278
0.249
0.278
0.259
0.287
0.259
0.273
0.259
0.315
0.265
0.317
0.292
0.363
PEMS03
0.197
0.200
0.112
0.214
0.118
0.236
0.113
0.222
0.150
0.251
0,147
0.248
0.180
0.291
0.169
0.282
0.278
0.375
0.114
0.224
PEMS04
0
0.197
0,105
0.209
0.119
0.232
0.111
0.221
0.122
0.239
0.129
0.241
0.195
0.307
0.209
0.314
0.295
0.388
0.093
0.202
PEMS07
0.075
0.171
0,085
08182
0.113
0.214
0.101
0.204
0.122
0.227
0,125
0.226
0.211
0.303
0.235
0.315
0.329
0.396
0.119
0.217
PEMS08
0.142
0.229
0.175
0.250
0.1
0.246
0.150
0.226
0.205
0.285
0,193
0.271
0.280
0.321
0.268
0.307
0.379
0.416
0.159
0.244
Count
22
11
Table 3: Performance comparison of different Query-Key configurations in TQ-MHA The setting (Q = TQ, K = Raw), where Q is generated from global learnable TQ vec- tors and K is derived from local raw data, corresponds to the default configuration in TQNet.
enables effective channel-wise information interaction. Fi- nally, the pure MLP model performs the worst due to its lack of any mechanism for channel correlations modeling. As a result, these findings underscore the effectiveness of the TQ technique and demonstrate that its integration with the at- tention mechanism significantly enhances the performance of multivariate forecasting tasks.
Setup
(Q=Raw. )Raw)
(Q=TQ.K=Raw)
(Q=TQ.K=TQQ)
Metric
MSE
MAE
MSE
MAE
MSE
MAE
Electricity
0.175
0.267
0.164
0.259
0.179
0.269
Beyond validating the critical role of the TQ technique within TQNet, we further explore its porta- bility and effectiveness in improving the multivariate fore- casting capabilities of existing models. Specifically, we integrate the TQ technique into several mainstream time series forecasting models to examine whether it enhances their predictive performance.
Integration Study
Solar
0,208
0.257
0.198
0.256
0.213
0.268
Traftic
0.426
0.279
0.445
0.276
0.429
0.281
PEMS03
0.114
0.222
0.097
0.203
0.111
0.221
PEMS04
0.112
0.222
0.091
0.197
0.113
0.222
PEMS07
0.094
0.195
0.075
0.171
0.092
0.195
PEMS08
0.170
0.252
0.142
0.229
0.174
0.257
Notably, i) For iTransformer, a Transformer- method that uses self-attention to model multivariate correlations, we directly replace the original query inputs with the pro- posed TQ vectors. ii) For PatchTST and DLinear, which are Cl-based methods without dedicated attention mechanisms for modeling channel correlations, we embed a complete attention mechanism (i.e., MHA) incorporating the TQ tech- nique to jointly model channel dependencies.
datasets. Specifically, the core of TQNet consists of an atten- tion mechanism (i.e., MHA) enhanced by the TQ technique for modeling multivariate correlations and a MLP module for capturing temporal dependencies.
Herein, when the TQ technique is removed, TQNet reduces to standard self-attention module combined with an MLP module, where channel correlations are more susceptible to noisy disturbances. When the attention mechanism is removed, TQNet becomes a variant of an MLP augmented with channel identifiers (Shao et al., 2022), which explicitly distinguish different channels but lack the ability to directly model inter-channel correlations. When both the TQ module and the attention mechanism are removed, TQNet further simplifies into pure MLP model.
The results on the right side of Table 4 demonstrate that in- corporating the TQ technique (either by enhancing existing attention mechanisms or by embedding a complete attention module with TQ) consistently improves the original models' performance. This indicates the versatility and effectiveness of the TQ technique in elevating the predictive capabilities of diverse forecasting methods, showcasing its portability across different architectures.
As shown in the results on the left side of Table 4, the TQ technique plays pivotal role in TQNet's performance, as removing the TQ module leads to the most significant perfor- mance degradation. Additionally, the attention mechanism also critical for modeling inter-variable correlations, as it
Representation Learning of the TQ Technique To fur- ther explore the underlying factors contributing to the suc- cess of TQNet, we examine the intrinsic representations
6

---

TQNet for Efficient Multivariate Time Series Forecasting
Table 4: Ablation and integration studies of the TQ technique. Left: Ablation study revealing the effectiveness of the TQ technique within TQNet. Right: Integration study exploring the portability and adaptability of the TQ technique.
Model
TQNet (MLP+ TQ & MHA)
irranfoormrr (2024c)
PanchTST (2023)
DLincar (2023)
Sctup
Original
MHHA
None
TQ + None
Pure MLP
I|
Original
+TQ
Original
+TQ
Original
+TQ & MHA
Merric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
||
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAE
96
0.134
0.229
0.138
0.233
0.149
0.241
0.165
0.252
0.150
0.241
0.133
0.230
0.164
0.254
0.136
0.236
0.195
0.277
0.154
0.252
Eleetioity
192
0.154
0.247
0.155
0.248
0.162
0.254
0,173
0.260
0.164
0.255
0.152
0.247
0.174
0.265
0.157
0.254
0.194
0.279
0.168
0.264
336
0.169
0.264
0.172
0.267
0.178
0.271
0.190
0.277
0.178
0.272
0.166
0.264
0.193
0.285
0.177
0.277
0.207
0.295
0.181
0.279
720
0.201
0.294
0.210
0.300
0.210
0.300
0,250
0.312
0.210
0.300
0.202
0.298
0.232
0.320
0.215
0.313
0.244
0.330
0.224
0.319
Avg
0.164
0.259
0.169
0.262
0.175
0.267
0.190
0.276
||
0.175
0.267
0.163
0.260
0.191
0.281
0.171
0.270
0.210
0.295
0.182
0.279
12
0.161
0.061
0.161
0.065
0.177
0.219
0.067
0.179
0.068
0.105
0.221
PEMS08
24
0.077
0.182
0.078
0.186
0.086
0.195
0.105
0.312
0.050
0.199
0.085
0.185
0.102
0.213
0.084
0.191
0.185
0.299
0.172
0.227
0.215
0.110
0.218
0.126
0.158
0.265
0.301
0.155
0.263
0.315
0.407
96
0.148
0.253
0.157
0.255
0.174
0.284
0,208
0.309
0.242
0.348
0,156
0.248
0.204
0.305
0.155
0.263
0.455
0.508
0,532
0.382
Avr
0.097
0.203
0.101
0.206
0.113
0.221
0.135
0.241
0.159
0.268
0.104
0.133
0,240
0.106
0.211
0.265
0.359
0.282
12
10067
0.166
0.072
0.176
0.076
0.180
0.086
0.193
0.081
0.189
0.967
0.167
0.087
0.195
0.069
0.172
0.114
0.228
0.074
0.177
PEMS04
24
0.077
0.181
0.086
0.194
0.094
0.204
0.119
0.229
0.097
0.207
0.079
0.182
0.119
0.231
0.082
0.189
0.187
0.298
0.093
0.202
48
1.097
0.206
0.111
D.223
0.124
0.238
0.174
0.283
0.128
0.241
0.092
0.199
0.172
0.279
0.101
0.212
0.319
0.417
0.138
0.248
96
0.123
0.233
0.1
0.246
0.156
0.268
0.224
0.328
0.176
0.284
0.128
0.233
0.221
0.323
0.131
0.247
0:424
0.481
0.211
0.313
Avg
0.091
0.197
0.102
0.210
0.113
0.223
0.151
0.258
0.120
0.230
0.091
0.195
0.150
0.257
0.096
0.205
0.261
0.352
0.129
0.235
30
0.5
Channel 69
0.0
20
-0.5
50
100
150
200
250
300
350
10-
2
Channnll 460
-10
0-
100
150
200
250
350
Channel 69
Channel 578
Value
Channel 460
-30
Channel
-40
-20
20
40
50
100
150
200
250
300
350
Dimension
Time Steps
Figure 4: Visualization of the learned TQ representations using t-SNE (Van der Maaten & Hinton, 2008) and their corresponding raw channel sequences. The data is sourced from the Traffic dataset.
learned by the TQ technique. Specifically, Figure 4 presents a visualization of the t-SNE (Van der Maaten & Hinton, 2008) compressed representations of the TQ vectors after TQNet has been trained to convergence.
forecasting accuracy and learning meaningful semantic rep- resentations. In this section, we further investigate whether TQ can capture more robust multivariate dependencies that contribute to better predictive performance.
Remarkably, channels that are closer in the t-SNE represen- tation tend to exhibit similar sequence patterns in their raw time series, whereas channels that are farther apart demon- strate distinct sequence patterns. This suggests that the TQ technique effectively captures intrinsic correlations among different channels. This ability allows the model to leverage information from correlated channels to stabilize predictions for given channel, particularly in the presence of noisy dis- turbances. As result, this characteristic not only improves forecasting accuracy but also enhances the interpretability of the model.
To this end, conduct the multivariate- -niviaareatte fore- casting task, where the objective is to predict single target channel using a varying number of input variables, rang- ing from no covariate information to utilizing all available variables. As shown in Figure 5, incorporating a moderate amount of covariate information notably enhances the fore- casting accuracy of TQNet on the target channel. Overall, using more covariates generally leads better performance. These results provide direct evidence that TQNet effectively leverages multivariate dependencies to improve prediction quality.
Dependency Study In previous experiments, we demon- strated the effectiveness of the TQ technique improving
Impact of the Hyperparameter W In the TQ technique, the length of the learnable TQ vectors, denoted as the hyper-
7

---

TQNet for Efficient Multivariate Time Series Forecasting
0.35
None
20
100
Full
0.30
0.25
MSE
0.20
0.15
0.10
0.05
0.00
Electricity
Traffic
Solar
PEMS03
PEMS04
PEMS07
PEMS08
Figure 5: Performance of TQNet with varying amounts of covariate information. The forecasting target is the last channel of the dataset, and the results are averaged across forecasting horizons H E (96, 192, 336, 720). "None" denotes that no covariates are used, "Full" indicates the use of all available covariates, and the numbers represent the use of a partial subset of covariates.
0.185
0.280 mance as the model captures only partial periodic features. In addition, when W is to the length of two weeks (W = 168 336). TQNet still achieves competitive performance. This is primarily because setting W to an in- teger multiple of the true period merely reduces the number of training samples allocated to each TQ parameter propor- tionally, without significantly impairing the effectiveness of the TQ mechanism itself.
MSE
MAE
0.180
0.180
0.275
0.175
0.175
0.267
0.172
D.265
0.265
0.262
0.167
6.167
0.165
0:1M
0.260
In contrast, when W does align with any true periodic length (e.g.. W 23 W = 167), the TQ technique introduces semantic inconsistencies, which lead to perfor- mance degradation. Additionally, special case arises when W = 1. Although it does not correspond to any periodic cycle, it functions as a weakened form of channel identifier, allowing the model to distinguish between different chan- nels (Shao et al., 2022). This enables TQNet still achieve performance improvements over the absence of the TQ tech- nique, highlighting its robustness and inherent ability to enhance forecasting accuracy. Finally, we further provide more detailed discussion in Appendix A.3 on how to deter- mine a proper value of the hyperparameter W and analyze the challenges posed by multi-periodic patterns.
0.160
0.255
TQ
Hyperparam. TQ
Figure 6: Performance of TQNet on the Electricity dataset with varying hyperparameter W. Results are averaged across forecasting horizons H (96, 192, 196, 720).
parameter W, is a critical factor as it determines the periodic shifting interval of the TQ vectors. In principle, this length W should align with the maximum periodic length of the dataset (Lin et al., 2024b), such as the length of a day for daily datasets the length of week for weekly datasets, to maximize its effectiveness. To investigate the impact of the hyperparameter W, we conduct experiments on the Electricity dataset, which exhibits both daily and weekly periodicity.
Efficiency Analysis Benefiting from the powerful TQ technique and the lightweight architectural design of TQNet, the model achieves exceptional multivariate forecasting per- formance while maintaining high computational efficiency, As shown Figure Left, TQNet delivers state-of-the-ae forecasting accuracy with significantly smaller parameter sizes and shorter training times compared to other mod- els. This highlights TQNet's ability to balance forecasting performance with computational cost effectively.
As shown Figure 6, TQNet achieves its best perfor- mance when W matches the weekly periodicity of the dataset (W 1681). When W matches only the daily periodicity (W 24), TQNet delivers suboptimal perfor-
'Tee Electricity dataset sampled hourly, the weekly length corresponds to x 168
Moreover, while TQNet demonstrates impressive capabili- ties in modeling multivariate correlations, its attention mech-
8

---

TQNet for Efficient Multivariate Time Series Forecasting
0.25
DLinear
PatchTST
50
TONet (ours)
Traiming Time diccadssepochhg
TimeXer
0.24
ITransformer
(Transformer
40
PatchTST
0.23
DLinear
TimeXer
MSE
0.22
CycleNet
Parameters
0.21
TQNet (ours)
0.20
3N
10H
0.19
40
80
120
400
800 862
Training Time (seconds/epochs)
Number of Variables
Figure 7: Computational efficiency of TQNet. Left: Comparison of prediction accuracy, parameter size, and training time on the Electricity dataset (forecast horizon H 720). Right: Comparison of training time with varying channel numbers the Traffic dataset, which includes up to 862 channels in the complete setting.
anism has a quadratic complexity, which may raise concerns for certain real-world applications. However, due to the overall lightweight design of TQNet and advancements in modern GPU parallelization technologies, the model can operate with near-linear computational overhead across vary- ing numbers of channels. As illustrated in Figure 7 Right, TQNet maintains minimal computational overhead even with different channel counts, closely approaching the ef- ficiency of linear-based methods (i.e., DLinear). This indi- cates that in most practical scenarios (e.g., when the number of channels C 1000), TQNet can operate with remark- ably low computational costs, making it highly suitable for real-world applications.
which can partially substitute for multivariate cues. On the other hand, longer input windows introduce more noise, increasing the risk of overfitting in correlation modeling.
We believe that addressing these challenges will further enhance the robustness and generalizability of TQNet, and leave this for future exploration.
6. Conclusion
This paper presented TQNet, a simple yet efficient model for multivariate time series forecasting. At the core of TQNet lies the Temporal Query (TQ) technique, an inno- vative method that employs periodically shifted learnable parameters queries in the attention mechanism model global inter-variable correlations. The TQ technique en- hances the model's ability to capture complex multivariate dependencies, offering improved robustness, interpretabil- ity, and adaptability. Extensive experiments on 12 chal- lenging real-world datasets demonstrated the effectiveness of TQNet, achieving state-of-the the-ar performance across diverse scenarios. Remarkably, TQNet maintains high putational efficiency, comparable to linear-based methods, even when applied to high- nigh-dimensional datasets with nearly 1,000 variables. This balance between accuracy and effi- ciency underscores TQNet's potential as a practical and robust solution for real-world forecasting challenges.
5. Limitations
This paper presents an efficient and robust approach for modeling multivariate correlations time series forecasting. However, TQNet still has several practical limitations worth noting:
Similar to CycleNet (Lin et al., 2024b), TQNet heavily relies on the inherent periodicity of the data to deter- mine the hyperparameter W. This dependency may limit its generalization datasets without clear peri- odic patterns.
When the underlying multivariate correlations in the data are weak or insignificant, enforcing strong multi- variate modeling may introduce unnecessary complex- ity and even negatively impact performance.
A Acknowledgements
This work is supported by Guangdong Major Project of Basic and Applied Basic Research (20198030302002, Guangdong Provincial Natural Science Foundation Project (2025A1515000113 and the Major Key Project of PCL, China under Grant PCL.202A00.
As shown in Figure in Appendix B, the benefits of multivariate modeling diminish as the bock-bcck win-
dow becomes sufficiently long. On the one hand, a longer look-back provides richer temporal information,

---

TQNet for Efficient Multivariate Time Series Forecasting
Impact Statement
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEFE conference computer vision and pattern recognition, pp. 770-778, 2016.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Hendrycks, D. and Gimpel, K. Gaussian error linear units (gelus). arXiv preprini arXiv: (666084155 2016.
References
Huang, Q., Shen, L.,, Zhang, R., Ding, S., Wang. B., Zhou, Z., and Wang. Y. Crossgnn: Confronting noisy multivari- ate time series via cross interaction refinement. Advances in Neural Information P Processing Systems, 36, 2023.
Cai, W., Liang, Y., Liu, X., Feng, J., and Wu, Y. Msgnet: Learning multi-scale inter-series correlations for multi- variate time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 11141-11149, 149, 2024.
Huang, Q., Shen, L.,, Zhang, R., Cheng, J.,, Ding, S., Zhou, Z., and Wang. Y. Hdmixer: Hierarchical dependency with extendable patch for multivariate time series forecasting. In Proceedings of the AAAI Conference on Artificial In- telligence, volume 38, pp. 12208-126111 2024.
Challu, C., Olivares, K. G., Oreshkin, B. N., Ramirez, F. G., Canseco, M. M., and Dubrawski, A. Nhits: Neural hi- erarchical interpolation for time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelli 2023.
Hyndman, R. J. and Athanasopoulos, G. Forecasting: prin- ciples and practice. OTexts, 2018
gence, volume 37, pp.
Chen, S.-A., Li, C.-L., Yoder, N., Arik, S. O., and Pfis- ter, T. Tsmixer: An all-mlp architecture for time series forecasting. arXiv preprint 2rXiv:2303.0005. 2023.
Ilbert, R., Odonnat, A., Feofanov, V., Virmaux, A., Paolo, G., Palpanas, T., and Redko, Samformer: Unlock- ing the potential of transformers in time series fore- casting with sharpness-awars minimization and channel- wise attention. In Proceedings of the 41st International Conference Machine Learning, volume 235. PMLR, 2024. URL https: wttphhtpps//pppeeeeeeeeeeee /ttp//ppeoeee...e///e//e/ mlr. ppress/ v235 / 11brtt24a html.
Dai, T., Wu, B., Liu, P., Li, N., Bao, J., Jiang, Y., and Xia, S.- T. Periodicity decoupling framework for long-ter -erm-termmme series forecasting. In The Twelfth International Conference Learning Representations, 2024.
Das, A., Kong, W., Leach, A., Mathur, S. K., Sen, R., and Yu, R. Long-term forecasting with tide: Time-series dense encoder. Transactions on Machine Learning Research, 2023.
Kim, D., Park, J., Lee, J., and Kim, H. Are self atffaattentions effective for time series forecasting? In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024.
Deng. J., Ye, F., Yin, D.. Song, X., Tsang, I., and Xiong, H. Parsimony or capability? decomposition delivers both in long-term time series forecasting. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.-H., and Choo, J. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2021.
Kingma, D. and Ba, Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.66 2014.
Ekambaram, V., Jati, A., Nguyen, N., Sinthong, P., and Kalagnanam, Tsmixer: Lightweight mll-mixer model for multivariate time series forecasting. In Proceedings of the 29th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 469, 2023.
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICMI. 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Han, L., Chen, X.-Y,, Yc, H.-J.. and Zhan, D.-C. Softs: Efficient multivariate time series forecasting with series- core fusion. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024a.
Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang, Y.- X., and Yan, X. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecast- ing. Advances neural information processing systems, 32, 2019.
Han, L., Ye, H.-J., and Zhan, D.-. The capacity and robust- ness trade-off off: Revisiting the channel independent strat- egy for multivariate time series forecasting. IEEE Trans- actions on Knowledge and Data Engineering, 2024b.
Li, Li, Z. QI, S., Y. and Xu, Z. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305. 10721, 2023a.
10

---

TQNet for Efficient Multivariate Time Series Forecasting
Li, Z.,, Rao, Z., Pan, L.., and Xu, Z. Mis-mixers: Multi- variate time series forecasting via factorized temporal and channel mixing, arXiv preprint arXiv:2302.04501, 2023b.
Luo, D. and Wang, X. Modernten: A modern pure convo- lution structure for general time series analysis. In The Twelfth International Conference on Learning Represen- tations, 2024.
Lin, S., Lin, W. Wu, W., Zhao, F., Mo, R., and Zhang, H. Segrnn: Segment recurrent neural network for long-term time series forecasting, arXiv preprint arXiv:2308. 11200, 2023.
Luo, Y., Liu, Z., Wang, L., Wu, B., Zheng, J., and Ma, Q. nowledge-empowerr dynamic graph network for irreg- ularly sampled medical time series. Advances in Neural Information Processing Systems, 37:67172-67199 2024.
Lin, S., Lin, W., Wu, W., Chen, H., and Yang, J. Sparsetsf: Modeling long-term time series forecasting with 1k pa- rameters, In Forty-first International Conference on Ma- chine Learning, 2024a.
Madsen, H. Time series analysis. CRC Press, 2007.
Nie, Y., H. Nguyen, N., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations, 2023.
Lin, S., Lin, W., Xinyi. H., Wu, W., Mo, R., and Zhong, Enhancing time series forecasting through modeling periodic patterns. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024b.
H. Cyclenet:
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Lin, S., Lin, W., Wu, W., Wang. S., and Wang, Y. Petformer: Long-term time series forecasting via placeholder- enhanced transformer. IEEE Transactions on Emerging Topics in Computational Intelligence, 18)):119--201, 2025. doi: 2..100//ETCC12204.555222222
Qiu, X., Hu, Zhou, L.,, Wu, X., Du, J., Zhang, B., Guo, C., Zhou, A., Jensen, C. S., Sheng. Z., and Yang, B. Tfb: Towards comprehensive and fair benchmarking of time series forecasting methods. Proc. VLDB Endow., 17(9): 2363-2377, 2024.
Liu, J., Liu, C., Woo, G., Wang, Y., Hooi, B., Xiong, C., and Sahoo, Unitst: Effectively modeling inter-series and intra-series dependencies for multivariate time series forecasting, arXiv preprint arXiv:2406.04990 2024a.
Qiu, X., Wu, X., Lin, Y., Guo, C., Hu, J., and Yang, B. Duet: Dual clustering enhanced multivariate time series forecasting. In SIGKDD, 2025.
Liu, M.. Zeng, A., Chen, M., Xu, Z., Lai, Q.. Ma, L., and Xu, Q. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neurul Information Processing Systems, 35:5816-588 2022a.
Shao, Z., Zhang, Z., Wang, F,, Wei, W., and Xu, Y. Spatial- temporal identity: A simple yet effective baseline for multivariate time series forecasting. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp. 4454-4458, 2022.
Liu, P.. Wu, B., Hu, Y., Li. N., Dai, T., Bao, J., and Xia, S.-t. Timebridge: Non-stationarity matters for long-term time series forecasting, arXiv preprint vrXiv:2410.04443) 2024b.
Shao, Z., Wang, Xu, Y., Wei, W., Yu, C., Zhang, Z., Yao, D., Sun, T., Jin, G., Cao, X., al. Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transac- tions on Knowledge and Data Engineering, 2024.
Liu, S., Yu, H., Liao, C.. J., Lin, W. Liu, A. X., and Dustdar, S. Pyraformer: Low-complexity pyramidal atten- tion for long-range time series modeling and forecasting. In International conference learning representations, 2021.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov. R. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-1988, 2014.
Liu, Y., Wu, H., Wang, J., and Long, M. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 35:9881 -99933, 2022b.
Toner, W. and Darlow, L. N. An analysis of linear time series forecasting models. In Forty-first International Conference on Machine Learning, 2024.
Liu, Y., Hu, T., Zhang. H., Wu, H., Wang, S., Ma, L., and Long, M. itransformer: Inverted transformers are effec- tive for time series forecasting. In The Twelfth Interna- tional Conference on Learning Representations, 2024c.
Van der Maaten, L. and Hinton, G. Visualizing data using
t-sne. Journal of machine learning research, 9(11), 2008

---

TQNet for Efficient Multivariate Time Series Forecasting
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L.,, Gomez, A. N., Kaiser, t.,, and Polosukhin, I. At- tention is all you need. Advances in neural information processing systems, 30, 2017.
Yi, K., Zhang, Q., Fan, W., He, H., Hu, L., Wang, P., An, N., Cao, L.,, and Niu, Z. Fouriergnn: Rethinking multivariate time series forecasting from pure graph perspective. Advances in Neural Information Processing Systems, 36, 2023.
Wang, H., Peng. J., Huang, F., Wang, J., Chen, J., and Xiao, Y. Mien: Multi-scal local and global context modeling for long-term series forecasting. In International Confer- ence on Learning Representations, 2023.
Yu, G., Zou, J., Hu, X., Aviles-Rivero, A. I., Qin, J., and Wang. S. Revitalizing multivariate time series forecast- ing: Learnable decomposition with inter-series depen- dencies and intra-series variations modeling. In Forty- first International Conference on Machine Learning, 2024. URL https:/ wpwpp/eeeeeeeeeeeeeepeep net/ forum? id=8 CCYNyGGO...
Wang, S., Wu, H., Shi, X., Hu, T., Luo, H., Ma, I.., Zhang, Y., and ZHOU, J. Timemixer: Decomposable multi- scale mixing for time series forecasting. The Twelfth International Conference on Learning Representations, 2024a.
Zeng, A., Chen, M.,, Zhang, L., and Xu, Q. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 11121-11121 1188, 2023.
Wang, Y., Wu, H., Dong, J., Liu, Y., Long, M., and Wang, J. Deep time series models: A comprchensive survey and benchmark. arXiv preprint 1rXi: 2024b.
Wang, Y.. Wu, H., Dong, J., Qin, G., Zhang, H., Liu, Y., Qiu, Y., Wang, J., and Long, M. Timexer: Empowering transformers for time series forecasting with exogenous variables. In The eighty- Annual Conference on Neural Information Processing Systems, 2024c.
Zhang, T., Zhang, Y., Cao, W., Bian, J., Yi, X., Zheng, S., and Li, J. Less is more: Fast multivariate time series forecasting with light sampling-orient mlp structures. arXiv preprint arXiv:2207.01 186, 2022.
Zhang, Y. and Yan, J. Crossformer:
Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In International Conference on Learning Representations, 2023.
Wen, Q.. Zhou, T., Zhang. C., Chen, W., Ma, Z., Yan, J., and Sun, L. Transformers time series: survey. In Proceed- ings of the Thirty-Second International Joint Conference on Artificial Intelligence, pp. 6778-6786, 2023.
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11106-11115, 2021.
Woo, G., Liu, C., Sahoo, D., Kumar, A., and Hoi, S. Ets- former: Exponential smoothing transformers for time- series forecasting. arXiv preprint arXiv:2202.01381, 2022.
Wu, H., Xu, J., Wang. J., and Long, M. Autoformer: Decom- position transformers with auto-correlation for long-term series forecasting. Advances in neural information pro- cessing systems, 14222419-22430 2021.
Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, I.., and Jin, R. Fedformer: Frequency enhanced decomposed trans- former for long-term series forecasting. In Interna- tional conference on machine learning, pp. 77268-272866. PMLR, 2022.
Wu, H., Hu, T,, Liu, Y., Zhou, H., Wang, J., and Long, M. Timesnet: Temporal 2d- variation modeling for gen- eral time series analysis. International Conference on Learning Representations, 2023.
Xu, Z., Zeng, A., and Xu, Q Fits: Modeling time series with 10k parameters. In The Twelfth International Conference on Learning Representations, 2024.
Xue, W., Zhou, T., Wen, Q., Gao, J., Ding, B., and Jin, R. Card: Channel aligned robust blend transformer for time series forecasting. In International Conference on Learning Representations (ICLR), 2024.
Ye, W., Deng, S., Zou, Q., and Gui, N. Frequency adaptive normalization for non-stationary time series forecasting. In The Thirty-eighn Annual Conference on Neural Infor- mation Processing Systems, 2024.
12

---

TQNet for Efficient Multivariate Time Series Forecasting
A. More Details of TQNet
mechanism. Specifically, O is used as the query (Q), while the input sequence X, serves as the key (K) and value (V). To stabilize training and enhance robustness, a residual connection is applied by adding X₃ to the output of the MHA, yielding Matn E RCxL
Algorithm Overall Pseudocode of TONet
Require: Historical look-back input X, E RCXL, cycle index mod W E No
Ensure: Forecasting output Y, E ROXH
In the temporal dependency modeling stage, the dimension- ality of ham is first transformed from RC RCxd via a linear layer, resulting in K Then, a two-layer multi-layer perceptron (MLP) with GeLU activation is used extract non-linear temporal dependencies. Similar to the attention mechanism, residual connection is added to stabilize train- ing. This process outputs mmp E Cxdd
1: Initialize learnable parameters Pro RCXw t
2: if Instance Normalization is applied then
3: 11.0 < Mean(X,), (TD(X))
4: Xt XI-
Votte
5: end if
6: indexro E No. + (t mod W Range(L)) mod W
7: 0i0 E EROXL t- Orol: indexro]
In the output stage, a linear layer with Dropout maps the learned hidden representations hmpp to the desired forecast- ing horizon H1, resulting in the final output Y, E RC*H If instance normalization was applied during the input stage, ee-normalization is performed here to restore the predictions to their original scale.
8: Mann E ROXL t Multi-Head Attention (Q =
0i lp Xc, V x,)+
9: Kaan RRxd Linear(h (hmm)
kmpp &RCxd= Multi- Layer Perceptron(H h
11: Y, E RCCH +- Linear (Dropout(hhh mipp)
12: if Instance Normalization is applied then
13: Ye + Ye vo
A.2. Experimental Details
14: end if
The complete experimental details can be found in our open- source code repository?. Specifically, the experiments in this paper are implemented using Py PyTorch (Paszke et al., 2019) and executed on single NVIDIA GeForce RTX 4090 GPU with 24 GB memory. The models are trained using the Adam optimizer (Kingma & Ba, 2014) and optimized with the L2 loss function. The training-vali validation-tess splits are consistent with prior works, such as iTTransformer (Liu et al., 2024c) and TimesNet (Wu et al., 2023). Specifically, the data splits follow a 6:2:2 ratio for the ETT and PEMS series datasets and a 7:1:2 ratio for the remaining datasets.
A.1. Method Details
The pseudocode of TQNet is presented Algorithm takes as input the historical look-back sequence X, E RCXL and the cycle index t mod W E No, and outputs the predicted sequence Y, RCHH
The algorithm begins by initializing the learnable param- eters Oro E CCC to zeros. Here, W is a hyperparameter that represents the prior cycle length of the dataset. During training, Oro is optimized to capture the underlying inter- variable correlations adaptively.
TQNet is trained for 30 epochs with early stopping based patience of on the validation The learning rate is set to 3 x 10 for most datasets, except for smaller datasets (i.e., the ETT series), where it is reduced to 1 x 10-3 The batch size varies based on the dataset's 's scale to maximize GPU utilization while avoiding out-of-memory errors. For instance, a batch size of is used for the Traffic dataset, while 64 is used for the Weather dataset. In TONet, the number of attention heads in the multi-head attention mechanism is fixed 4, and the dropout rate is set to 0.5 by default. Additionally, the dropout rate in the output layer is set to 0.5 for smaller datasets (c.g., the ETT series and Weather) and 0 for larger datasets. To ensure reproducibility. all experiments are conducted with a fixed random seed of 2024.
In the input processing stage, an optional instance normal- ization layer can be applied to X,- This step removes the sample's 's mean and variance to mitigate potential issues arising from distributional shifts and ensure that the input sequence is stationary. instance normalization is applied, the corresponding de-normalization is performed on the model output recover the original scale.
Next, the indexro E N/ is computed, representing the in- dices in fro to extract the segment t1o for the current sam- ple. This is efficiently calculated as:
indexro (t mod W Range(T.)) mod W.
L-1]. where Range(L) L)) generates the array [0, 1,..., 1]. Using these indices, f i. indexro] is extracted obtain 0%00 e RCx, which serves as the TQ vectors for the sample.
In the multivariate dependency modeling stage, the TQ vec- tors are incorporated into the multi-head attention (MHA)
13

---

TQNet for Efficient Multivariate Time Series Forecasting
A.3. Discussion about Hyperparameter W
In TQ-MHA, the queries Q are generated from globally shared, periodically shifted learnable vectors, while the keys K and values V are directly derived from the raw input sequence X₁. thereby encoding more localized, sample- specific patterns. For simplicity, we assume only a single attention head in MHA. During training, the model is opti- mized to maximize the attention output:
The hyperparameter W, which represents the prior cy- cle length of the dataset, is determined based on domain- specific knowledge of periodic patterns and sampling in- tervals, suggested by CycleNet (Lin et al., 2024b), or alternatively through computational techniques such as the autocorrelation function (ACF) (Madsen, 2007). The se- lected W values for TQNet are aligned with the cycle length settings used in CycleNet, as summarized in Table
Q'KtT VL
For detailed explanation of ACF and its use in identifying periodicities in time series data, we strongly refer readers to Appendix B.2 of the CycleNet paper (Lin et al., 2024b). We have provided the corresponding runnable code snippets in our open-source repository", and omit further details here for brevity.
0 = Softmax
V
(10)
which implies that, for each input sample i, the learnable queries Q' is encouraged to align with the corresponding key K This alignment allows the queries to capture more relevant inter-variable information, which is erucial for ac- curate forecasting. This alignment can be interpreted as maximizing their directional similarity:
Moreover, since TQNet considers only single periodic length W per dataset, it may encounter challenges when dealing with multi-period scenarios (e.g.. hourly, daily, weekly, or monthly patterns). This issue can be discussed in two representative cases:
Corr(Q') ~ Corr(K)),
(11)
1. Overlapping periodicities. This scenario is gener- ally manageable. For example, the Traffic dataset ex- hibits both daily (24-hour) and weekly (7x24-hour) patterns. In such cases, selecting the longest period- icity (weekly) is typically sufficient, as implicitly captures the shorter cycle. In fact, most real-world datasets fall into this category, meaning that TQNet remains effective in practical applications.
where Corr()) (-,,,10) A AT, diag( (A) = 1}
denotes a normalized correlation measure between the vectors.
Furthermore, since the query Q' is extracted periodically from the shared learnable parameter θTg, multiple samples spaced W time steps apart will share the same queries. As result, the effective correlation learned by Q' remains consistent across periods and is effectively an average over multiple local samples:
2. Irregularly interwoven periodicities. In contrast, datasets with irregularly overlapping periodicities, such as weekly (7x24-hour) and monthly (00x24-hour] pat- terns, pose a greater challenge. Simply choosing the longest periodicity may fail to capture the nuances of the overlapping weekly cycle. A practical compromise in such cases is focus solely on the daily periodicity (24-hour). Alternatively, one could ensemble multi- ple TQNet models, each configured with a different W, to improve adaptability to complex periodic struc- tures. Overall, effectively modeling multi-periodicity remains an open research question, even for existing methods.
Corr(Q') = Con((uitul), n= 0. N - 1, (12)
and
N-1 Cor(K k
(13)
m=0
where W the periodic length, N is the number of sampled periods, and Ki+nW denotes the keys derived from the raw inputs at those respective time steps.
A.4. Effectiveness analysis
The TQ technique lies at the core of TQNet, aiming to enhance the robustness of global inter-variable correlation modeling within the attention mechanism. In this subsection, provide theoretical insights into why the TQ technique facilitates more stable and expressive representations of variable correlations.
Therefore, in practice, the TQ-based queries Q' serves as averaged representation of correlations across multiple periodic samples in the dataset. This averaging effect miti- gates the influence of local non-stationary noise or outliers, leading to more stable and robust inter-vrriblee correlation modeling.
acc ipyt. in http:///ghhhh.ccom/AAAA.SCU///
14

---

TQNet for Efficient Multivariate Time Series Forecasting
Table 5: Full multivariate time series forecasting results of Table 2 for all prediction horizons H E {96, 192, 336, 720). The look- length L is fixed at 96, and the reproduced baseline results are sourced from TimeXer (Wang et al., 2024c), i'Transfomme (Liu et al., 2024c), and CycleNet (Lin et al., 2024b). The best results are highlighted in bold, while the second- best results are underlined.
TQNet (Ours)
TimeXer (2024c)
CycleNet (2024b)
iTransformer (2024ec
MSGNNe (2024)
TimesNet (2023)
PatchTST (2023)
Crossformer (2023)
DLinca (2023)
SCINer (2022a)
Model
Metric
MSE
MAE|
| MSE
MAE|
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
|SE
MAE
MSE
MAE
96
0.371
0.393
0,382
0.403
0.375
0.395
0.386
0.405
0.390
0.411
0.384
0.402
0.414
0.419
0.423
0.448
0.386
0.400
0.654
0.599
192
0.428
0.426
0.429
0.435
0.436
0.428
0.441
0,436
0.443
0.442
0.436
0.429
0.460
0.445
0.471
0.474
0.437
0.432
0.719
0.631
336
0.476
0.446
0.468
0.448
0.496
0.455
0.487
0.458
0.482
0.469
0.491
0.469
0.501
0.466
0.570
0.546
0.481
0.459
0.778
0.659
720
0:487
0.470
0.169
0.461
0.520
0.484
0.503
0.491
0.496
0.488
0.521
0.500
0.500
0.488
0.653
0.621
0.519
0.516
0.836
0.699
Avg
0.441
0,434
0.437
0.437
0.457
0.441
0.454
0.448
0.453
0.453
0.458
0.450
0.469
0.455
0.529
0.522
0.456
0.452
0.747
0.647
96
0.295
0.343
0.286
0.338
0.298
0.344
0.297
0.349
0.329
0.371
0.340
0.374
0.302
0.348
0.745
0.584
0.333
0.387
0.707
0.621
ETT1
192
0:367
0.393
0.36
0.
0.372
0.396
0.380
0,400
0.402
0.414
0.402
0.414
0.388
0.400
0.877
0.656
0.477
0.476
0.860
0.689
336
0.417
0.427
0.414
0.423
0.431
0.439
0.428
0.432
0,440
0.445
0.452
0.452
0.426
0.433
1.043
0.731
0,594
0.541
1,000
0.744
720
0.433
0,446
0.408
0.432
0.450
0,458
0.427
0.445
0.480
0.477
0,462
0.468
0.431
0.446
1.104
0.763
0.831
0.657
1.249
0.838
Avg
0.3778
0.402
0.368
0.396
0.388
0.409
0.383
0.407
0.413
0.427
0.414
0.427
0.387
0.407
0.942
0.684
0.559
0.515
0.954
0.723
96
0.311
0.353
0.318
0.356
0.319
0.360
0.334
0.368
0.319
0.366
0.338
0.375
0.329
0.367
0.404
0,426
0.345
0.372
0.418
0.438
ETTal
192
0.356
0.378
0.362
0,385
0.360
0
0.377
0,391
0.377
0.397
0.374
0.387
0.367
0.385
0.450
0.451
0,380
0.389
0.439
0.450
336
0.390
0.401
0.395
0.407
0.389
0.403
0.426
0,420
0.417
0.422
0.410
0.411
0,399
0.410
0.532
0.515
0.413
0.413
0.490
0.485
720
0:452
0.440
0.452
0.441
0.447
0.441
0.491
0.459
0.487
0.463
0.478
0,450
0.484
0.439
0.666
0.589
0,,474
0.483
0,595
0.550
Avg
0.377
0.393
0.382
0.397
0.379
0.396
0:407
0.410
0,400
0.412
0,400
0.406
0.387
0.400
0.513
0.495
0:103
0.407
0.486
0.481
96
0.173
0,256
0.171
0.256
0.163
0.246
0.180
0,264
0.182
0.266
0.187
0.267
0.175
0,259
0.287
0,366
0.193
0.292
0.286
0.377
ETTo2
192
0.238
0.298
0.237
0.299
0.229
0.290
0.250
0,309
0.248
0.306
0.249
0.309
0.241
0.302
0.414
0.492
0,284
0.362
0.399
0.445
336
0.301
0.340
0.296
0.338
0.284
0.327
0.311
0.348
0.312
0.346
0.521
0.351
0.305
0.343
0.597
0.542
0.369
0.427
0.637
0.591
720
0.397
0.396
0.392
0. 1.494
0.389
0.391
0.412
0.407
0.414
0.404
0.408
0.403
0,402
0,400
1,730
1.042
0.554
0.522
0,960
0.735
Avg
0.277
0.323
0.274
0.322
0.266
0.314
0.288
0.332
0.289
0.330
0.291
0.333
0.281
0.326
0.757
0.611
0.350
0.401
0.571
0.537
Electricity
96
0.134
0.229
0.140
0.242
0.136
0.229
0.148
0.240
0.165
0.274
0.168
0.272
0.181
0.270
0.219
0.314
0.197
0.282
0,286
0.377
192
0.154
0.247
0.157
0,256
0.152
0.244
0.162
0,253
0.185
0.292
0.184
0,289
0.188
0,274
0.231
0.322
0.196
0.285
0.399
0.445
336
0.169
0.264
0.176
0.275
0.170
0.264
0.178
0.269
0.197
0.304
0.198
0.300
0.204
0.293
0.246
0.337
0.209
0.301
0.637
0.591
720
0.201
0.294
0.211
0.306
0.212
0,299
0.225
0,317
0.231
0.332
0.220
0.320
0.246
0.324
0,280
0.363
0,244
0.333
0,960
0.735
Avg
0.164
0.259|
|
0.171
0,270
0.168
0.259
0.178
0.270
| 0.194
0.301
0.193
0.295
0,205
0.290
0.244
0.334
0,212
0,300
0.571
0.537
Solar Energy
96
0.173
0.233
0.215
0.295
0.190
0.247
0.203
0,237
0.210
0.246
0.250
0.292
0.234
0,286
0,310
0.331
0.290
0.378
0.237
0.344
192
0.199
0.257
0.236
0,301
0.210
0,266
0.233
0
0.265
0,290
0.296
0,318
0.267
0,310
0.734
0.725
0.320
0.398
0,280
0.380
336
0.211
0.263
0.252
0.307
0.217
0,266
0.248
0.273
0.294
0.318
0.319
0.330
0.290
0.315
0.750
0.735
0.353
0.415
0.304
0.389
720
0.209
0.270
0,244
0,305
0.223
0.266
0.249
0.275
0.285
0.315
0.338
0.337
0.289
0.317
0.769
0.765
0.356
0.413
0.308
0.388
Avg.
0.198
0.256
0.237
0.302
0.210
0.261
0:233
0,262
0.263
0.292
0.301
0.319
0.230
0.307
0.641
0.639
0.330
0.401
0,282
0.375
96
0.413
0.261
0:428
0.271
0.458
0.296
0.395
0,268
0.608
0.349
0.593
0.321
0,462
0.290
0.522
0.290
0.650
0.396
0.788
0.499
Traflie
192
0.432
0.271
0.448
0,282
0.457
0.294
0.417
0.236
0.634
0,371
0.617
0,336
0.466
0,290
0.530
0.293
0.598
0.370
0.789
0.505
336
0.450
0.277
0,473
0.289
0.470
0.299
0.433
0,283
0.669
0.388
0.629
0.336
0,482
0.300
0.558
0,305
0.605
0.373
0.797
0.508
720
0.486
0.295
0,516
0,307
0.502
0.314
0.467
0.302
0.729
0.420
0.640
0.350
0,514
0.320
0.589
0,328
0.645
0.394
0.841
0.523
Avg
0.445
0.276
|
0.466
0,287
0.472
0.301
0.428
0,282
0.660
0.382
0.620
0.336
0.481
0.300
0.550
0.304
0.625
0.383
0.804
0.509
96
0.157
0.200
0.157
0.205
0.158
0.203
0.174
0,214
0.163
0.212
0.172
0.220
0,177
0,210
0.158
0.230
0.196
0.255
0,221
0.306
Wealter
192
0.206
0.245
0.204
0.247
0.207
0.247
0.221
0.254
0.211
0.254
0.219
0,261
0.225
0.250
0.206
0.277
0,237
0.296
0.261
0.340
336
0.262
0.287
0,261
0,290
0.262
0,289
0.278
0,296
0.273
0.299
0.280
0.306
0.278
0.290
0,272
0.335
0,283
0:335
0.309
0.378
720
0,344
0,342
0.340
0.341
0.344
0.344
0.358
0.349
0.351
0.348
0,365
0.359
0.354
0,340
0.398
0.418
0.345
0.381
0,377
0.427
Avg
0.242
0.269
|
0.241
0.271
0.243
0.271
0.258
0.278
0.249
0.278
0.259
0.287
0.259
0.273
0.259
0.315
0.265
0.317
0.292
0.363
96
0,060
0.161
0.070
0.173
0.066
0.172
0.071
0.174
0.078
0.187
0,085
0.192
0,099
0,216
0.090
0.20
0.203
0.122
0.243
0.066
0.172
PEMS00
192
0.077
0.182
0.092
0.194
0.089
0.201
0.093
0.201
0.108
0.218
0.118
0.223
0.142
0.259
0.121
0.240
0,201
0.317
0.085
0.198
336
0.104
0.215
0.129
0,229
0.136
0.247
0.125
0,236
0.178
0.272
0.155
0.260
0.211
0.319
0,202
0.317
0.333
0.425
0.127
0.238
720
0.148
0.253
0,157
0.261
0.182
0,282
0.164
0,275
0.238
0.328
0.228
0,317
0.269
0,370
0.262
0,367
0.457
0.515
0.178
0.287
Avg
0.097
0.203
0.112
0.214
0.118
0.226
0.113
0.222
0.150
0.251
0.147
0.248
0.180
0.291
0.169
0.282
0.278
0.375
0.114
0.224
96
0.067
0.166
0.074
0.178
0.078
0.186
0.078
0,183
0.086
0.199
0.087
0.195
0,105
0.224
0.098
0.218
0.148
0.272
0.073
0.177
PEMSO
192
0.077
0.181
0.087
0.195
0.099
0,212
0.095
0.205
0.101
0.218
0.103
0.215
0.153
0.275
0.131
0.256
0,224
0.340
0.084
0. 0.193
336
0.097
0.206
0.110
0.214
0.133
0,248
0.120
0,233
0.127
0.247
0.136
0.250
0,229
0,339
0.205
0.326
0.355
0.437
0.099
0.211
720
0.123
0.233
0.148
0,251
0.167
0.281
0.150
0,262
0.174
0.292
0.190
0.303
0.291
0.389
0.402
0.457
0.452
0.504
0.114
0.227
Avg.
0.091
0.197
0.105
0,209
0.119
0.232
0.111
0,221
0.122
0.239
0.129
0.241
0.195
0.307
0.209
0.314
0.295
0.388
0.093
0.202
96
0.051
0.143
0.057
0,152
0.062
0.162
0.067
0,165
0.079
0.182
0.082
0.181
0.095
0.207
0.094
0.200
0.115
0.242
0.068
0.171
PEMS07
192
0.063
0.159
0.079
0.179
0.086
0.192
0.088
0.190
0.099
0.206
0.101
0,204
0.150
0.262
0.139
0.247
0,210
0.329
0.119
0.225
336
0,081
0.179
0.099
0.191
0.128
0,234
0.110
0,215
0.133
0.239
0.134
0.238
0,253
0,340
0.311
0.369
0.398
0.458
0.149
0.237
720
0.103
0.203
0.107
0 0.205
0.176
0.268
0.139
0.245
0.179
0.279
0.181
0.279
0.346
0.404
0.396
0.442
0.594
0.553
0.141
0.234
Avg.
0.075
0.171
0.085
0.182
0.113
0.214
0.101
0.204
0.122
0.227
0.125
0.226
0.211
0.303
0.235
0.315
0.329
0.396
0.119
0:217
96
0.071
0.170
0.075
0.176
0.082
0.185
0.079
0.182
0.105
0.211
0.112
0.212
0.168
0.232
0.165
0.214
0.154
0.276
0.087
0.184
PEMSO8
192
0.096
0.196
0.102
0
0.117
0.226
0.115
0.219
0.141
0.243
0.141
0.238
0.224
0.281
0.215
0.260
0.248
0.353
0.122
0.221
336
0.149
0.244
0.158
0,248
0.169
0,268
0.186
0.235
0.211
0.300
0.198
0.283
0,321
0.354
0,315
0.355
0.440
0.470
0.189
0.270
720
0.253
0.309
0.366
0.377
0.233
0.306
0.221
0.267
0.364
0.387
0.320
0.351
0,408
0.417
0.377
0,397
0.674
0.565
0.236
0.300
Avg
0.142
0.229
0.175
0.250
0.150
0.246
0.150
0.326
0.205
0.285
0.193
0.271
0,280
0.321
0.268
0.307
0.379
0.416
0.159
0.244
15

---

TQNet for Efficient Multivariate Time Series Forecasting
B. More Results of TQNet
of TQNet. The following points are worth noting when interpreting the results:
B.1. Full Comparison Results
Table 5 presents the full comparison results of TQNet against several baseline methods across 12 real-world multi- variate datasets. The results demonstrate that TQNet con- sistently achieves state-of-the-art forecasting performance under most experimental settings, underscoring the effec- tiveness of the proposed approach.
ETS is fundamentally a univariate forecasting method and cannot leverage multivariate information, whereas this is a key capability emphasized by TQNet. This limitation puts ETS disadvantage in case where inter-variable dependencies play an important role.
2.
ETS requires the look-back window L satisfy L Z 2W, where W denotes the period length. Accordingly, we set L 336 and L 720 for ETS to meet this constraint, instead of using the default I. = 96 adopted in the TQNet experiments.
B.2. Univariate Forecasting Results
Previously, we primarily demonstrated the performance of TQNet in multivariate forecasting scenarios. How- ever, in real-world applications, a more common setting is cnntrarraateto-uivvaiiii variables are utilized to predict a single target variable. Therefore, we further present comparison of forecasting results under this setting (see Table 6). As shown, TQNet exhibits strong competitiveness compared to state-of-the-art models specifically designed for multivariate- tosttattte-t--iiiieet to-uiiiiaiitttiiiiiiitiee univariate tasks, such as TimeXer.
3. ETS is primarily designed for short-term forecasting based on statistical extrapolation. When applied to long-horizon forecasting, its predictions often diverge significantly from the ground truth, with errors com- pounding over time. This explains the substantial in- crease MSE observed for ETS under large forecast- ing horizons (c.g. H =720).
forecasting, where exogenous
B.3. Impact of Module Stacking
B.5. Impact of Look-back Length
Table 8 illustrates the impact of varying look-back lengths on the performance of TQNet and other models. The results reveal that under shorter input lengths, CD-based methods (e.g.. iTransformer) exhibit a significant advantage over CI- based methods (e.g., PStchTST). In these scenarios, TQNet outperforms other CD-based methods, demonstrating the effectiveness of the proposed TQ technique in capturing robust multivariate dependencies.
TQNet achieves sfate-of-the-ar performance using only the most essential components: a single-layer attention mech- anism and a single MIP. This design achieves an effective balance between forecasting accuracy and computational efficiency. To investigate whether increasing model capacity leads to further improvements, we experiment with stacking three layers of TQ-MHA and MLP modules within TQNet.
The results in Figure 7 show that adding more stacked mod- ules does not yield significant performance gains. In most datasets, performance slightly declines, although modest improvements are observed on the PEMS datasets. This outcome highlights the robustness and soundness of the original TQNet design, which already delivers near-optimal performance with minimal architectural complexity.
When the input length increases, models can leverage more temporal dependencies to compensate for the limitations in modeling multivariate relationships. Consequently, the performance gap between CD-based and Cl-based methods diminishes. However, overall, CD-based methods maintain distinct advantage and are more suitable for practical de- ployment. For instance, in real-world scenarios where it may be challenging to obtain sufficiently long historical data, CD-based methods can effectively enhance predictive per- formance by leveraging superior multivariate dependency modeling.
These findings not only validate the effectiveness of our approach but also reinforce our central claim: TQNet strikes an ideal trade-off between forecasting accuracy and com- putational cost. Finally, we also advocate for lightweight model designs in time series forecasting, as they enhance in- terpretability, facilitate practical deployment, and maintain strong predictive performance.
B.6. Robustness of TQNet
To evaluate the robustness of TQNet, we conducted multiple runs of the model under different random seeds and learning rates. The results indicate that TQNet consistently maintains low standard deviations across all settings, demonstrating its stability and reliability. This robustness suggests that TONet is resilient to variations in initialization and hyperpa- rameter configurations, making it a dependable choice for multivariate time series forecasting tasks.
B.4. Comparison with ETS
ETS (Exponential Smoothing (Hyndman & Athanasopou- los, 2018)) is a classical forecasting method that leverages periodic patterns in time series data. In Table 8, we com- pare the performance of TQNet and ETS across multiple datasets, demonstrating the superior forecasting capabilities
16

---

TQNet for Efficient Multivariate Time Series Forecasting
Table 6: Multivariate- aitiivariate-t-univvaiiia to-univariate forecasting time series forecasting results. The look-back length L is fixed at 96, and the reproduced baseline results are sourced from TimeXer (Wang et al., 2024c). The best results are highlighted in bold, while the second-best results are underlined.
TQNet (Ours)
TimeXer (2024c)
iTransfommer (2024c)
TimesNet (2023)
PatchTST (2023)
Cressformer (2023)
DLinear (2023)
SCINet
NSformer (20226)
Autefommer (2021)
Model
(2022a)
Metric
MSR
MAF]
MSR
MAF
MSR
MAF
MSF
MAF
MSF
MAF
MSE
MAP
MSE
MAF
MSE
MAR
MSF
MAR
MSR
MAR
96
0.064
0.181
0.067
0.188
0.071
0.194
0.073
0.200
0,068
0.188
0.149
0.309
0.072
0.195
0.253
0.427
0.098
0.229
0.135
0.282
ETTa2
192
0.097
0.231
0.101
0.236
0.108
0.247
0.106
0.247
0.100
0.236
0.686
0.740
0.105
0.240
0.592
0.677
0.161
0.302
0.143
0.294
336
0.128
0.272
0.130
0.275
0.140
0.288
0.150
0.296
0.128
0.271
0.546
0.602
0.136
0.280
0.777
0.790
0.243
0.362
0.156
0.308
720
0.181
0.330
0.182
0.332
0.188
0.340
0.186
0.338
0.185
0.335
2.524
1.424
0.191
0.335
1.117
0.960
0.326
0.441
0.184
0.333
Avg
0,118
0.254
0.120
0.258
0.127
0.267
0.129
0.270
0.120
0.258
0.976
0.769
0.126
0.263
0.685
0,714
0,207
0.334
0.154
0.304
96
0.239
0.348
0.261
0.366
0.299
0.342
0.339
0.412
0.364
0.387
0.451
0.390
0.462
0.298
0.407
0.502
Electricity
192
0.283
0.375
0.316
0.397
0.321
0.413
0.384
0.461
0.361
0.425
0.313
0.390
0.365
0.436
0.375
0.456
0.340
0.433
0.492
0.492
336
0.342
0.415
0.367
0.429
0.379
0.446
0.439
0.493
0.393
0.440
0.380
0.431
0.391
0.453
0.468
0.519
0.405
0.471
0.508
0.548
720
0.427
0.477
0.365
0.439
0.461
0.504
0.473
0.482
0.507
0.418
0.463
0.428
0.487
0.477
0.524
0,444
0.489
0.547
0.569
Avg
0..23
0.404
0.327
0.408
0.365
0.442
0.410
0.476
0.394
0.446
0.344
0.412
0.393
0.457
0.428
0.490
0.372
0.450
0.495
0.528
96
0.129
0.201
0.151
0.224
0.156
0.236
0.154
0.349
0.176
0.253
0.154
0.230
0.268
0.351
0.371
0.448
0.214
0.323
0.290
0.290
Trafiic
192
0.131
0.204
0.152
0.229
0.156
0.237
0.164
0,255
0.162
0.243
0.180
0.256
0,302
0,387
0,450
0,503
0.195
0.307
0.291
0.291
336
0.131
0.208
0.150
0.232
0.154
0.243
0.167
0.259
0.164
0.248
0.164
0.241
0.298
0.384
0.447
0.501
0.198
0.309
0.322
0.416
720
0.148
0.228
0.172
0.253
0.177
0,268
0.197
0.292
0.189
0.267
0,203
0.277
0.340
0.416
0.521
0.548
0.835
0.507
0.307
0.414
Avg.
0.135
0,210
0.156
0.235
0.161
0.246
0.171
0.264
0.173
0.253
0.175
0.251
0,302
0.385
0..47
0.500
0.361
0.362
0,303
0.353
0.26
0.34
TONet(ours)
TimeXer
0.24
0.32
iTransformer
0.22
0.30
PatchTST
MAE
0.20
DLinear
0.28
0.18
0.26
0.16
0.24
0.14
0.22
100
200
300
400
500
600
700
100
200
300
400
500
600
700
Look-back Length
Look-back Length
Figure 8: Performance of TQNet and comparative models on the Electricity dataset with different look-back lengths. The forecast horizon is set as 96.
Table 7: Comparison of the performance between the origi- nal TQNet model and the stacked version (three layers) on multiple datasets. Results are averaged across forecasting horizons H {96, 192, 336, 720).
Table 8: Comparison of TQNet and ETS for multivariate- to-univariate forecasting across multiple datasets. Re- sults are averaged across forecasting horizons H (96, 192, 336, 72).
Model
Original
Stacking
Model
TQNet
ETS
Metric
MSE
MAE
MSE
MAE
Metric
MSE
MAE
MSE
MAE
ETTh1
0.441
0.434
0.443
0.447
ETTh1
0.074
0.213
0.206
0.337
ETTh2
0.378
0.402
0.382
0.405
ETTh2
0.178
0.337
231.3
8.006
ETTml
0.377
0.393
0.391
0.404
E
0.048
0.164
0.169
0.268
ETTm2
0.277
0.323
0.280
0.324
ETTm2
0.121
0.263
12.19
1.788
Electricity
0.164
0.259
0.167
0.262
Electricity
0.299
0.388
0.778
0.599
Solar
0.198
0.256
0.203
0.265
Solar
0.260
0.299
18,06
1.160
traffic
0.445
0.276
0.451
0.285
Traffic
0.122
0.201
92.69
0.490
weather
0.242
0.269
0.242
0,271
Weather
0.002
0.029
0.007
0.044
PEMS03
0.097
0.203
0.095
0.197
PEMS03
0.151
0.245
0.618
0.357
PEMS04
0.091
0.197
0.084
0.189
PEMS04
0.063
0.186
0.345
0.352
PEMS07
0.075
0.171
0.072
0.167
PFMS07
0.094
0.233
0.253
0.359
PEMSO8
0.142
0.229
0.147
0.225
PEMS08
0.178
0.309
0.426
0.458
17

---

TQNet for Efficient Multivariate Time Series Forecasting
Table 9: Performance of TQNet under different random seeds and learning rates. Mean represents the average value, and Std denotes the standard deviation.
Random Seed
Learning Rate
Sctup
2024
2025
2026
Mean
Std
0.001
0,002
0,003
Mosn
Std
Metric
MSE
MAE
MSE
MAE|
MSE
MAE
MSE
MAE
|
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.371
0.393
0.371
0.394
0.372
0.393
0.371
0.393
0.001
0,000
0.371
0.393
0.376
0.398
0.378
0.396
0.375
0.395
0.004
0.002
192
0.428
0.426
0.430
0.424
0.430
0.423
0.429
0.424
0.001
0.002
0.428
0.426
0.435
0.429
0.435
0.432
0.433
0.429
0.004
0.003
3366
0.476
0.446
0.481
0.453
0.476
0.446
0.478
0.448
0.003
0.004
0,476
0.446
0,480
0.448
0.494
0.458
0.483
0.451
0.010
0.007
720
0.487
0.470
0,510
0.487
0.491
0.472
0.496
0.476
0.012
0.009
0,487
0.470
0,510
0.486
0.521
0.493
0.506
0.483
0.017
0.012
96
0.295
0.343
0.295
0.346
0.295
0.343
0.295
0.344
0.000
0.002
0.295
0.343
0.302
0.346
0.301
0.348
0.299
0.346
0.004
0.002
ETTh2
192
0.367
0.393
0.370
0.394
0.366
0.391
0.368
0.392
0.002
0.002
0.367
0.393
0.374
0.393
0.376
0.396
0.372
0.394
0.005
0.002
3366
0.417
0.427
0.418
0.427
0.415
0.428
0.417
0.428
0.001
0.001
0.417
0.427
0,418
0.433
0.424
0.434
0.419
0.431
0.004
0.004
720
0.433
0.446
0.441
0.441
0.451
0.438
0.449
0.004
0.003
0.433
0:446
0.437
0.441
0.437
0.4-19
0.436
0.448
0.002
0.002
96
0
0.353
0,316
0.355
0.310
0.352
0.312
0.353
0.003
0.001
0.311
0.353
0,315
0.356
0.321
0.359
0.316
0.356
0.005
0.003
ETTml
192
0.356
0.378
0.358
0.380
0,362
0.582
0.359
0.380
0.003
0,002
0.356
0.378
0.365
0.384
0.361
0.381
0.361
0.381
0.004
0.003
336
0.390
0.401
0.389
0.402
0.392
0.402
0.390
0.402
0.001
0.001
0.390
0.401
0.398
0,407
0.393
0.405
0.393
0.404
0.004
0.003
720
0.452
0. 0.440
0.451
0.441
0.452
0.441
0.452
0.441
0.001
0.001
0,452
0.440
0.460
0.447
0.455
0.443
0.456
0.443
0.004
0.003
96
0.173
0.256
0.172
0.255
0.174
0.255
0.173
0.255
0.001
0.001
0.256
0.174
0,256
0.175
0,258
0.174
0.256
0.001
0.001
Electricity ETTu2
192
0,238
0.298
0,239
0.299
0,241
0.300
0,239
0.299
0.002
0.001
0,238
0.298
0,240
0.299
0.244
0.301
0.241
0.299
0.003
0.002
336
0
0.340
0,305
0.342
0.298
0.338
0.301
0.340
0.003
0.002
0.301
0.340
0,306
0.342
0.302
0.340
0.303
0.341
0.003
0.001
720
0,397
0.396
0.397
0.396
0.398
0.396
0.397
0.396
0.000
0.000
0.397
0.396
0,396
0.395
0,396
0.395
0.396
0.395
0.001
0.001
96
0.134
0.229
0.135
0.230
0.135
0.229
0.134
0.230
0.000
0.000
0.136
0.231
0.135
0:230
0.135
0.230
0.135
0.230
0.001
0.000
192
0,154
0.247
0,153
0,246
0.155
0.246
0.153
0.246
0,000
0.000
0.154
0.247
0,152
0.246
0.152
0,246
0.153
0.246
0.001
336
0,169
0.26-1
0.168
0.264
0.169
0.264
0.169
0.264
0.000
0.000
0.169
0.263
0,169
0.264
0.168
0,263
0.169
0.263
0.001
0,000
720
0,201
0.294
0.204
0.296
0,210
0.302
0.205
0.298
0.005
0.004
0,202
0.292
0.201
0.292
0.201
0.294
0,201
0.293
0.001
0.001
12
0.060
161
0.061
0.161
0.060
0.161
0.060
0.161
0.000
0.000
0.061
0.163
0,061
0,162
0,061
0.161
0.061
0.162
0,000
0.001
PEMSU0
24
0.077
0.182
0.076
0.181
0.077
0.182
0.076
0.182
0.001
0,001
0.077
0.184
0.076
0.182
0.077
0.183
0.077
0.183
0.000
0.001
48
0,104
0.215
0,107
0.215
0.110
0.216
0.107
0.215
0.003
0.001
0,110
0.218
0,107
0.215
0,109
0.215
0.109
0.216
0.001
0.002
96
0.148
0.253
0,150
0.255
0.144
0.251
0.148
0.253
0.003
0.002
0.146
0.256
0,152
0.256
0,152
0.256
0.150
0.256
0.003
0.000
12
0.067
0.166
0.067
0.167
0.066
0.166
0.067
0.167
0.000
0.000
0.069
0.170
0.067
0.167
0.067
0.166
0.068
0.168
0.001
0.002
PEMSI
24
0.077
181
0.077
0.182
0,077
0.181
0.077
0.181
0.000
0.000
0.082
0.187
0.078
0.183
0.078
0.182
0.079
0.184
0.002
0.003
48
0.007
0.206
0.095
0.205
0.096
0.205
0.096
0.206
0.001
0.001
0.103
0.213
0.097
0,207
0.096
0.206
0.099
0.209
0.005
0.004
96
0,123
0,233
0.122
0,233
0.123
0.232
0.123
0.233
0.000
0.000
0.129
0.242
0,132
0,233
0.126
0.235
0.125
0.236
0.004
0.005
12
0.051
0.143
0.082
0.143
0.052
0.144
0.052
0.144
0.000
0.000
0.053
0.147
0.052
0.144
0.051
0.143
0.052
0.145
0.001
0.002
PEMSUT
0.063
0.159
0.063
0.159
0.063
0.159
0.063
0.159
0.000
0.000
0.065
0.164
0.063
0.160
0.063
0.160
0.064
0.161
0.001
0.002
48
0.081
0.179
0.080
0.180
0.081
0.180
0.080
0.180
0.000
0.000
0.084
0.188
0.081
0.181
0.080
0.180
0.082
0.183
0.002
0.004
0.103
0.203
0.103
0.203
0.108
0.207
0.105
0.204
0.003
0.002
0.113
0.217
0.106
0.207
0.102
0.202
0.107
0.209
0.005
0.008
12
0.071
0.170
0.072
0.170
0.070
0,170
0.071
0.170
0.001
0.000
0.073
0.174
0,071
0.170
0.070
0.169
0.071
0,171
0.002
0.003
PEMSSS
24
0.096
0.196
0.097
0.196
0.094
0.195
0.096
0.196
0.001
0,001
0.102
0.204
0.097
0.198
0.095
0.194
0.098
0.199
0.003
0.005
0.149
0.244
0.152
0.247
0.150
0.245
0.150
0.245
0.001
0.002
0,166
0.259
0,155
0.249
0.155
0.249
0.158
0.252
0.006
0.006
96
0,253
0. 0,309
0.265
0.310
0.256
0.306
0,258
0.308
0.006
0.002
0.297
0.331
0,266
0,310
0.259
0. 0.306
0.274
0.316
0.020
0.013
96
0.173
0.233
0.191
0.254
0.181
0.245
0.182
0.264
0.010
0.192
0.256
0.184
0.236
0.184
0.235
0.187
0.243
0.005
0.012
Solat
192
0.199
0.257
0.191
0.250
0.191
0.254
0.194
0.254
0.004
0.003
0.196
0.248
0.191
0.252
0.193
0.255
0.193
0.252
0.002
0.003
3366
0.211
0.263
0.202
0.263
0,213
0.272
0.209
0.266
0.006
0.005
0.203
0:256
0.206
0.259
0.205
0. 0.264
0,205
0.260
0.002
0.004
720
0,209
0,270
0.217
0.270
0.220
0.269
0,216
0.270
0.006
0.001
0,208
0.260
0,207
0,265
0.214
0.271
0.210
0.265
0.004
0.005
96
0.413
0.261
0.412
0.261
0.410
0.260
0.412
0.260
0.001
0.000
0.417
0.266
0.412
0:262
0.414
0.261
0.414
0.263
0.005
0.003
Traffic
192
0.432
0.271
0.435
0.431
0.270
0.432
0.271
0.001
0.434
0.276
0.432
0.272
0.431
0.270
0.432
0.273
0.002
0.003
336
0.450
0.277
0.451
0.277
0.450
0.277
0.450
0.277
0.001
0.000
0.448
0.283
0.443
0.279
0.448
0.277
0.446
0.280
0.003
0.003
720
0.486
0.295
0.493
0.297
0.485
0.296
0.488
0.296
0.004
0.001
0.491
0.301
0.488
0.296
0.477
0.296
0.483
0.298
0.007
0.003
96
0,157
0.200
0.158
0.201
0.158
0.201
0.158
0.201
0.001
0.001
0.157
0.200
0,160
0.204
0.158
0.201
0.158
0.202
0.002
0.002
Weather
192
0.206
0.245
0.206
0.245
0,206
0.246
0.206
0.245
0.000
0,000
0.206
0.245
0.205
0.244
0.206
0.246
0,206
0.245
0.001
0.001
336
0,262
0.287
0,262
0.287
0.264
0.288
0.263
0.288
0.001
0.001
0,262
0.287
0,264
0.288
0,264
0.289
0.263
0.288
0.001
0.001
720
0.344
0.342
0.343
0.342
0.343
0.342
0.343
0.342
0.001
0.000
0. 0.344
0.342
0.344
0.342
0.344
0. 344
0.344
0.342
0.000
0.000
18


Reviews:
[{"review_id": "oaq40DMCDU", "reviewer": "Reviewer_kXGG", "summary": "This paper proposes Temporal Query Network (TQNet), a new approach for multivariate time series forecasting (MTSF). The key idea is the Temporal Query (TQ) technique, where periodically shifted learnable vectors serve as the query in a single-layer multi-head attention (MHA) module. TQ provides one vector per channel and shift them by a chosen cycle length. This mechanism aims to capture global inter-variable correlations more robustly than conventional self-attention, which often derives queries/keys/values solely from the raw data and is prone to get influenced by noise or missing values. The authors demonstrate state-of-the-art performance on 12 real-world benchmarking datasets of 3 different domains, showing that TQNet is both accurate and efficient, even with a large number of channels (e.g., nearly 1,000).", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you for your valuable comments!** \n\n> C1: TQ is fixed across different samples, whereas per-example correlations differ from global correlations.\n\nThis concern is valid in general, but it is effectively addressed within the TQ-MHA mechanism. In TQ-MHA, **the learnable shifted TQ serves *only* as the $Q$**, while the raw series acts as the $K$ and $V$. This design means that the correlations among **$Q$  emphasize more stable global dependencies**, while those among **$K$ and $V$ focus on per-example correlations**. Through the attention mechanism, the model integrates both global and local dependencies, rather than relying solely on per-example correlations. *We will properly revise the original claim to clarify this point.*\n\n> C2: How does TQ capture inter-variable correlations if there are no explicit constraints?\n\nIn fact, TQ introduces **two implicit biases** that facilitate the learning of inter-variable correlations:\n\n1. **Adaptive learning through backpropagation** – TQ is inherently designed to learn correlations via backpropagation optimization. As shown in Figure 4 of the paper, TQ effectively models the relative relationships among variables.\n2. **Periodic shifting mechanism** – This ensures that TQ-based learned representations are averaged per period, mitigating the effects of random noise perturbations. Figure 1 demonstrates that the correlations derived from TQ-generated $Q$ are more stable and globally consistent.\n\n> C3: TQ seems to violate time-invariance because it assigns fixed ( C × L ) vectors to each timestamp.\n\nIn fact, **only sample on timestamps $t$ and $t+W$ share the same $Q$**, since $Q$ ($C×L$) is extracted from $\\theta_{TQ}$ ($C×W$) via a periodic shifting mechanism (with a period of $W$ timesteps). However, **the remaining samples within the interval $[t, t+W-1]$, the $Q$ are different** across timestamps (manifesting as a shift along the time axis). Therefore, this setting does not violate time-invariance. On the contrary, by explicitly considering periodic fluctuations, it better aligns with the inherent biases of real-world time series.\n\n> W1:  There is no clearly way to easily and automatically determine determine W given a specific dataset..\n\nThere are many simple methods to clearly determine the hyperparameter $W$. On one hand, real-world time series data are often influenced by **social or natural factors**, meaning they typically exhibit **at least a daily or weekly periodic fluctuation**. Therefore, considering the sampling interval of the data (e.g., 15 min), it is easy to infer the potential periodic length. On the other hand, the **autocorrelation function (ACF) serves as an effective mathematical tool to verify the periodic length**, where peaks in the ACF correspond to potential periodic lengths. Therefore, adjusting and selecting $W$ is even easier than tuning the learning rate in the practical training scenarios. *We will provide a more detailed explanation of how to determine $W$ in the revised paper, along with a script tool (utilizing ACF) in the open-source code to facilitate direct usage by users.*\n\n> W2: It might overlook the cases where there are multiple periodic lengths.\n\nThanks for pointing this out. This issue can be discussed in two cases:\n\n1. **When multiple periodicities overlap, it can be easily handled**. For example, the Traffic dataset exhibits both daily (24-hour) and weekly (7×24-hour) periodic patterns. In this case, considering only the longest periodicity (weekly) is sufficient, as it inherently encompasses the shorter daily cycle. **In fact, most real-world scenarios fall into this category, so this does not pose a significant challenge for the practical application of TQNet**.\n2. **When multiple periodicities are irregularly interwoven**, such as weekly (7×24-hour) and monthly (30×24-hour) cycles, it introduces some challenges for TQNet. Simply considering the longest periodicity cannot precisely capture the overlapping weekly patterns. In such cases, a practical compromise is to focus only on the daily periodicity (24-hour). Additionally, an alternative approach is to integrate multiple TQNet models, each configured with different $W$ values, to enhance adaptability in such scenarios. **Overall, fully addressing this issue remains an open research question, even for existing models.** *We will include a more detailed discussion in the revised paper to provide readers with a clearer understanding of TQNet and suggest directions for further investigation.*\n\n> Suggestion to clearly put the message in the Figure 1 caption.\n\nThanks for your suggestion! *We will add a clearer message to the caption of Figure 1.*\n\nFinally, sorry about the concise response due to length limits. For any unclear points, we are happy to provide further clarification in the next stage. **Thanks again!**", "claims_and_evidence": "The main claims of this paper are (1). TQ can better capture inter-variable correlations, (2). TQ is more robust to noise and missing values, and (3). TQ is more efficient. While claim (2) and (3) seem to be intuitively reasonable, claim (1). is non-trivial and requires more convincing intuition and direct evidence. \n\nThe problems to be clarified include but not limited to:\n\nC1. TQ is fixed across different samples. However, as shown in figure 1(a) and 1(c), the global correlations and the per-example correlations can be different, which can reveal some properties of each example. It is not clear how much this difference will play a role in MTSF, and how well TQ can tackle this problem\n\nC2. As TQ generating vectors from CxW learnable space, it is not clear how the correlations among different variables can be learned - there seem to be no explicit constrains on the inter-variable correlations\n\nC3. TQ seem to violate the invariance of the patterns in the time dimension, i.e., patten alpha in channel 1 and pattern beta in channel 2 might happen in different timestamps. However, in TQ, as each different timestamp will take a fixed CxL vector, patterns seem to be fixed in the time domain.", "methods_and_evaluation_criteria": "Though the main claims of the method need to be further polished and supported, the methods follows the main claims well.\n\nThe datasets, the experimental settings (lock-back length and forecasting length) and the main evaluation metrics (MSE and MAE, training time, etc.) follow the common practice in MTSF.", "theoretical_claims": "To my understanding, there are no theoretical claims in this paper.", "experimental_designs_or_analyses": "The experiment designs, i.e., main results, ablation studies, MSE vs. Training efficient, sensitivity of W, against number of variables, seem reasonably complete and can empirically back up the advantage of the proposed method.", "relation_to_broader_scientific_literature": "To my understanding, there are no clear relations to a potential broader scientific literature, besides MTSF.", "essential_references_not_discussed": "To my knowledge, this paper refers to a reasonably good number of references.", "other_strengths_and_weaknesses": "S1. The paper is largely well-written and easy to follow.\n\nBesides the claim questions C1, C2 and C3 to be clarified, some other weaknesses include:\n\nW1. The periodic length W is dataset-dependent, and there is no clearly way to easily and automatically determine W given a specific dataset.\n\nW2. The fact that there is only one periodic length W for each dataset, might overlook the cases where there are multiple periodic lengths, e.g., hourly, daily, weekly and monthly, etc.", "other_comments_or_suggestions": "C1. The message of Figure 1 is not clear. I suggest the authors to clearly put the message in the figure caption. For example, the message by comparing Figure 1(a) and 1(b) is that TQ can replicate well the global correlation.", "questions_for_authors": "Please refer to C1, C2, C3 and W1, W2.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "R6DBGhDbBq", "reviewer": "Reviewer_Tew6", "summary": "This paper proposed Temporal Query technique for multivariate time series forecasting framework, which is aiming at capturing optimal representations of inter-variable relationships. The lightweight improvement show advanced performance on real-world datasets and can be integrated easily to existing models.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you for your insightful review!**\n\n> Claims And Evidence and Essential References Not Discussed. \n\nThanks for pointing this out. Indeed, changes in time scales can cause variations in inter-variable correlations, and in fact, Figure 1 of TQNet also demonstrates this. The key differences are:\n\n1. MSGNet's multi-scale approach focuses on **different time scales within the look-back window** (e.g., variations within 24 and 96-time steps in a window of length 96).\n2. TQNet's multi-scale nature manifests in two ways: **the local sample look-back window** represents short-term scales, while **the global sequence (the entire training set)** represents long-term scales.\n\n**Thus, MSGNet’s findings actually support our claim that considering correlations at different scales is necessary.** *We will supplement the revised paper with discussions and comparisons with MSGNet.* The complete comparison results can be found at this link: https://anonymous.4open.science/r/TQNet-MSGNet-7EBE, which demonstrates the significant advantage of TQNet.\n\n> Questions For Authors: How does temporal query technique benefit representation learning of inter-variable relationships? Why can TQ-enhanced MHA mechanism learns a globally consistent and adaptive representation of inter-variable correlations within individual samples? How does it deal with non-stationary disturbances in real-world data? Maybe some theoretical explanation or case discussion will help.\n\nThe benefits of the Temporal Query (TQ) technique arise from **its learnable nature combined with a periodic shifting mechanism**. In the TQ-enhanced MHA, the query $Q$ is generated from globally shared, periodically shifted learnable vectors, while the keys $K$ and values $V$ are directly derived from the raw time series and thus capture more localized correlations. During training, the model is encouraged to maximize the attention output:  \n\n$$\nO = \\operatorname{Softmax}\\left( \\frac{Q_i K_i^\\top}{\\sqrt{L}} \\right) V_i,\n$$\n\nwhich implies that for each sample $i$, *the learned query $Q_i$ tends to align with the key $K_i$, meaning that it actively incorporates more relevant inter-variable information for accurate forecasting*. This alignment can be formulated as:\n\n$$\n\\operatorname{Corr}(Q_i) \\approx \\operatorname{Corr}(K_i) = \\frac{Q_i K_i^\\top}{\\|Q_i\\|\\,\\|K_i\\|},\n$$\n\nwhere $\\operatorname{Corr}(\\cdot)$ represents a normalized measure of correlation between the vectors.  \n\nMoreover, since the query $Q_i$ is periodically extracted from the shared learnable parameter $\\theta_{\\text{TQ}}$ (i.e., multiple samples share the same $Q$ due to the periodic shift), its effective correlation remains consistent across periods and is averaged over multiple local samples.\n\n$$\n\\operatorname{Corr}(Q_i) = \\operatorname{Corr}(Q_{i+nW}), \\quad n=0,1,\\dots,N-1,\n$$\n\nand  \n\n$$\n\\operatorname{Corr}(Q_i) \\approx \\frac{1}{N} \\sum_{n=0}^{N-1} \\operatorname{Corr}(K_{i+nW}),\n$$\n\nwhere $W$ is the periodic length, $N$ is the number of sampled periods, and $K_{i+nW}$ represents the keys obtained from the raw time series, capturing localized correlations.  \n\n**Thus, in practice, $Q_i$ serves as an averaged representation of correlations across all samples within the dataset over multiple periods**, *mitigating the impact of non-stationary disturbances in local samples.)*\n\n*This also explains why the TQ technique enables each sample's $Q$​ to learn a globally consistent and adaptive representation of inter-variable correlations, addressing the limitation of conventional attention mechanisms, which can only capture localized sample correlations. Since the learned correlations incorporate information from all periodic samples in the training set, they approximate the overall dataset correlation, thereby enhancing the representation learning of inter-variable relationships.*\n\n> Theoretical Claims.\n\nWe hope the above theoretical analysis addresses your concerns. In summary, the TQ technique, **through its periodic cyclic sharing mechanism, effectively neutralizes non-stationary disturbances across multiple periods**, thereby improving the robustness of the correlation learned in the attention mechanism.\n\n**Additionally, Figure 1 serves as a case study demonstrating the effectiveness of the TQ technique.** The data is collected from the first sample of the real-world Traffic dataset (where Figure 1d highlights several channels with strong disturbances). It can be observed that even in the presence of noisy disturbances, the correlation of $Q$ generated by the TQ technique is more stable and aligns more closely with the dataset's global correlation. This characteristic enables TQ-MHA to comprehensively consider correlations at different time scales—where $Q$ models the global correlation via cross-period contributions, while $K$ and $V$ model localized correlation with noisy perturbations.\n\n**Finally, thank you again for your informative review. We hope our response addresses your concerns.**", "claims_and_evidence": "Evidence insufficient. In Figure 1, the author's viewpoint is that the inter-variable correlations observed in individual samples is significantly differ from global correlations because of non-stationary disturbances, such as extreme values, missing data, and noise. However, more intuitive reason may be that the correlation varies across different time scales[1] even if there are no extreme values, missing data, and noise.\n\n[1] MSGNet: Learning Multi-Scale Inter-Series Correlations for Multivariate Time Series Forecasting.", "methods_and_evaluation_criteria": "1. Lightweight and exquisite improvement. The TQ technique consumes smaller parameter sizes and shorter training times. \n2. Good portability. The TQ technique can be integrated into several time series forecasting models easily.\n3. Sufficient evaluations.\n(1) State-of-the-art performance for long-term forecasting. The model achieves state-of-the-art performance on some real-world multivariate datasets.\n(2) Experiments on representation learning suggests that the TQ technique is useful for captureing intrinsic correlations among different channels.\n(3) The dependency study is interesting, which evaluate whether the TQ technique captures more robust multivariate dependencies.", "theoretical_claims": "Lack or insufficient theoretical analysis. The article lacks theoretical discussion on how can TQ address the non-stationary disturbances in real-world data and how it finally enhances the robustness of the learned correlations. This doesn't sound intuitive, and there’s no theoretical explanation or case discussion.", "experimental_designs_or_analyses": "The experiments is conducted with commonly used settings and full results of repeated experiments are shown in appendix.", "relation_to_broader_scientific_literature": "This work contributes to improving the predictive performance of time series models and will be highly effective in low-cost industrial scenarios.", "essential_references_not_discussed": "The following references may help for the similar motivation, which is to discuss the variate correlations:\n[1] MSGNet: Learning Multi-Scale Inter-Series Correlations for Multivariate Time Series Forecasting.", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1.How does temporal query technique benefit representation learning of inter-variable relationships? Why can TQ-enhanced MHA mechanism learns a globally consistent and adaptive representation of inter-variable correlations within individual samples? How does it deal with non-stationary disturbances in real-world data? Maybe some theoretical explanation or case discussion will help.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0SHnFro7Ag", "reviewer": "Reviewer_thzz", "summary": "This paper introduces the Temporal Query Network (TQNet) to address multivariate time series forecasting (MTSF) tasks. At its core, it employs periodically shifted learnable parameters to model more stable inter-variable correlations adaptively. Extensive experiments are conducted to demonstrate the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you very much for your detailed review!**\n\n> W1: The paper only provides results for multivariate-to-multivariate forecasting.\n\nWe have further supplemented the comparison between TQNet and baseline models in the multivariate-to-univariate forecasting scenario. **The results show that TQNet still exhibits a significant advantage in this setting**, demonstrating its superior capability in multivariate modeling.\n\n|| TQNet|| TimeXer || iTrans || TimesNet || PatchTST ||\n| ----------- | --------- | --------- | ------- | ----- | ------ | ----- | -------- | ----- | -------- | ----- |\n|| MSE| MAE| MSE| MAE| MSE| MAE| MSE| MAE| MSE| MAE|\n| ETTm2| **0.118** | **0.254** | 0.120   | 0.258 | 0.127  | 0.267 | 0.129    | 0.270 | 0.120    | 0.258 |\n| Electricity | **0.323** | **0.404** | 0.327   | 0.408 | 0.365  | 0.442 | 0.410    | 0.476 | 0.394    | 0.446 |\n| Traffic| **0.135** | **0.210** | 0.156   | 0.235 | 0.161  | 0.246 | 0.171    | 0.264 | 0.173    | 0.253 |\n\n> W2 (also mentioned in Suggestions): There are some inconsistencies between Figure 2, Algorithm 1, and their descriptions in the main text.\n\nWe appreciate your careful examination and suggestions! We will correct these inconsistencies in the final version.\n\n> W3: The paper does not discuss potential limitations, such as cases where there is no periodicity or no significant multivariate dependencies.\n\nThank you for highlighting this. **We will add a discussion of this potential limitations** of TQNet in the revised version to provide readers with a clearer understanding of TQNet and identify areas that require further investigation. Specifically, in scenarios where there is no significant periodicity, TQNet may not perform optimally, as its TQ mechanism relies on periodic shift operations. However, long-horizon forecasting in non-periodic settings is inherently challenging. In such cases, incorporating additional prior features may help compensate for the lack of periodic information. We will include these clarifications in the revised paper.  \n\n> Q1: What happens when a suitable $W$ cannot be found?\n\nMany recent studies have demonstrated that **periodicity is a crucial factor for achieving long-horizon forecasting** [1]. Moreover, real-world time series data are often influenced by social or natural factors, meaning they typically exhibit at least a daily periodic fluctuation. Therefore, identifying a suitable $W$ is feasible and straightforward. Additionally, the results in Figure 6 indicate that even without leveraging the dataset's periodicity (e.g., setting $W$ to 1), **the TQ mechanism can still improve forecasting accuracy**. This is because, in such cases, TQ can act as an implicit channel identifier [2], enhancing the model's ability to distinguish between multivariate channels.\n\n[1] Lin S, Lin W, Hu X, et al. Cyclenet: Enhancing time series forecasting through modeling periodic patterns. NeurIPS, 2024.\n[2] Shao Z, Zhang Z, Wang F, et al. Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting. CIKM, 2022.\n\n> Q2: Why are the learnable parameters in TQ initialized to zero? \n\nTQ can adaptively learn optimal representations of inter-variable relationships through backpropagation. Therefore, **its initialization does not significantly impact the final learned representations**. Specifically, we verified this by experimenting with different initialization strategies on the Electricity dataset and found that the performance remained consistent across different initialization methods. \n\n|      | Zero  | Uniform | Normal | Xavier | Kaiming |\n| ---- | ----- | ------- | ------ | ------ | ------- |\n| MSE  | 0.164 | 0.166   | 0.166  | 0.166  | 0.166   |\n| MAE  | 0.259 | 0.260   | 0.260  | 0.260  | 0.260   |\n\n\n> Q3: Why can TQNet achieve nearly the same computational efficiency as DLinear (as shown in Figure 7)? \n\nThis is primarily due to two factors: (i) **The lightweight design of TQNe**t, which includes only a single attention layer and a two-layer MLP, maximizing efficiency. (ii) **The powerful parallel computing capabilities of modern GPUs**. Specifically, the attention computations and MLP structures in TQNet can be highly parallelized, allowing it to achieve computational efficiency comparable to linear models.\n\n> Q4: Are the results reported in Table 2 averaged over multiple runs?\n\n**Table 5 presents results from multiple runs of TQNet with different random seeds and learning rates, demonstrating its robustness**. The results in Table 2 for TQNet are from a single run with a random seed of 2024, following the experimental setup described in Appendix A.2. Notably, these results are consistent with those in Table 5. Furthermore, the baseline results in Table 2 are sourced from their official reports (or reproduced from the iTransformer paper) to ensure reliability. We have clarified this in the caption of Table 4, where the full results are provided.\n\n**Thanks again!**", "claims_and_evidence": "Yes. The authors design lots of experimental evidence to support their claims.", "methods_and_evaluation_criteria": "Yes. The proposed method is aligned to advance the field of MTSF.", "theoretical_claims": "No theoretical claims are made in the paper.", "experimental_designs_or_analyses": "Yes. I have examined the experimental design.", "relation_to_broader_scientific_literature": "This paper proposes a multivariate time series forecasting model, which is particularly relevant to real-world applications such as traffic prediction, power demand forecasting, and weather forecasting.", "essential_references_not_discussed": "Not at all.", "other_strengths_and_weaknesses": "**Strengths:**\n1. The paper is well-organized and easy to follow.\n2. The proposed method is logically sound and computationally efficient.\n3. The experimental design is well-structured to demonstrate the effectiveness of the method, including ablation studies, exploratory experiments, and efficiency analysis.\n4. The figures and tables are clear and visually appealing.\n\n\n**Weaknesses:**\n1. The paper only provides results for multivariate-to-multivariate forecasting. However, in real-world applications, a more common scenario is multivariate-to-univariate forecasting, where exogenous variables are used to predict a single target variable.\n2. There are some inconsistencies between Figure 2, Algorithm 1, and their descriptions in the main text.\n3. The paper does not discuss potential limitations, such as cases where there is no periodicity or no significant multivariate dependencies.", "other_comments_or_suggestions": "1. There is an inconsistency in Algorithm 1 regarding the RevIN formula, which does not match Equation 8.\n2. Figure 2 is missing the linear transformation mentioned in line 9 of Algorithm 1.", "questions_for_authors": "1. The proposed technique heavily depends on the hyperparameter W. What happens when a suitable W cannot be found (i.e., when the dataset lacks clear periodicity)? \n2. Why are the learnable parameters in TQ initialized to zero? How does this differ from random initialization?\n3. Why can TQNet achieve nearly the same computational efficiency as DLinear (as shown in Figure 7)? This seems difficult to achieve, as the additional attention mechanism and deep network should require a significant amount of computation.\n4. Are the results reported in Table 2 averaged over multiple runs? Were the baseline results reproduced, or were they sourced from existing work?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ToDvxwiAnJ", "reviewer": "Reviewer_yR9p", "summary": "The paper presents a new model to do point-forecasting of multivariate time series: TQNet.\n\nTQNet combines a single attention layer to handle multi-time-steps interactions and an MLP to handle multi-channel interactions. The main novel contribution of TQNet is that the attention layer doesn't use the input to create its query vectors, but instead use trained parameters based on the absolute time position modulo the data periodicity. Another particularity is that all channels in a single timestep is taken as a single \"token\", instead of having each timestep/channel pair identifying its own token.\n\nThe model is benchmarked against multiple recently published models on multiple multivariate datasets, and comes ahead.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you for your detailed and thoughtful review! We must apologize for the concise response earlier due to length limits.**\n\n> Summary: TQNet combines a single attention layer to handle multi-time-steps interactions and an MLP to handle multi-channel interactions. \n\nSorry for any unclear points in our previous version. In fact, it is the **TQ-enhanced attention layer that handles multi-channel interactions**, followed by an MLP that handles multi-time-step interactions. We will improve the clarity of this statement in the revised paper.\n\n> Claims: The main evidence which is lacking is how good the model is at using multivariate information.\n\nThank you for pointing this out. Figure 5 only indirectly demonstrates TQNet's ability to utilize multivariate information but does not directly show how effective it is in leveraging such information. To address this, we have added experiments on several large-scale multivariate datasets. In these experiments, the goal is to predict the target channel using multiple variables, ranging from not utilizing any additional variables to fully leveraging all available variables. **The results show that incorporating covariate information significantly improves the prediction accuracy of TQNet for the target channel.(even for Solar dataset)** This provides direct evidence that TQNet can effectively utilize multivariate information.\n\n| # covariant | Electricity | Traffic | Solar | PEMS03 | PEMS04 | PEMS07 |\n| - | - | - | - | - | - | - |\n| 0| 0.355| 0.153| 0.290|0.070|0.060| 0.125|\n| 5| 0.325| 0.150| 0.264|0.061|0.054| 0.088|\n| 20| 0.328| 0.149| 0.259|0.064|0.048| 0.081|\n| 100| 0.330| 0.138| 0.263|0.057|0.049| 0.075|\n| Full| 0.323| 0.137| 0.260|0.062|0.048| 0.079|\n\n> M1. This makes it plausible that the results' quality is solely due to the improved time-wise interactions.\n\nOur goal is to explore a more elegant mechanism for modeling multivariate correlations, for which we designed TQ-MHA. Time-wise interactions are not the primary focus of our study, so we adopted a simple two-layer MLP for temporal dependency modeling, which has been shown in recent studies to be sufficiently capable of capturing temporal dependencies. **As shown in Figure 2, TQNet's major improvements are particularly evident in large-scale multivariate datasets.**\n\n> M2. It is not mentioned whether any of the baselines have also been given this information.\n\n**Many baselines have indeed utilized temporal (time) information**. For example, CycleNet explicitly models periodic sequences, while iTransformer and TimeXer use timestamp sequences as tokens. *In fact, how to leverage this information to build better forecasting models remains an open research question, and TQNet provides an elegant solution in this regard.*\n\n> M3 (also mentioned in Essential References Not Discussed). Classical and older models are absent from the baselines, such as ETS.\n\nWe have added comparison experiments with ETS, which clearly demonstrate the superiority of TQNet. Please refer to this link for the results: https://anonymous.4open.science/r/TQNet-ETS-CD4F.\n\n> Other Comments: Please add the hyperparameters used for TQNet.\n\nYes, we have detailed the hyperparameters of TQNet in Appendix A.2, where they are set relatively consistently accross datasets. The results for other baselines are sourced from their official results or the iTransformer paper to ensure reliability.\n\n> Q1 (also mentioned in Supplementary Material):  The provided link does not work.\n\nWe sincerely apologize for this issue. The anonymous website encountered a bug displaying \"The requested file is not found\" when trying to browse the code online. To resolve this, **you may directly download the source code from the anonymous website or access it through the Supplementary Material on OpenReview.** We hope this works for you.\n\n> Q2: What is the impact of giving an incorrect W to the model?\n\nIn Figure 6, we have systematically explored the impact of incorrect W settings, except for the specific case you mentioned where W is set to a multiple of the full periodicity. **Our additional experiments show that when W = 2 × 168, TQNet's performance declines but remains close to that when W = 168.** This is primarily because setting W to an integer multiple only reduces the number of training samples allocated to TQ parameters proportionally, without significantly affecting the effectiveness of the TQ mechanism itself. We will add this result to Figure 6.\n\n|| 168| 2*168 |\n| -| - | - |\n|MSE|0.164 | 0.167 |\n|MAE| 0.259 | 0.261 |\n\n> Q3: Whether interactions between time steps t and t' and those between t and t' + W are identical.\n\nThey are not identical. **This is because the TQ mechanism only affects the correlation of Q in the attention mechanism, while K and V remain dependent on local samples.** Therefore, the fundamental purpose of the TQ mechanism is to provide a globally stable correlation supplement.\n\n**Apologies again for the concise text, and thank you again!**", "claims_and_evidence": "The major portion of the evidence is done through the benchmark and the ablation experiments. The main evidence which is lacking is how good the model is at using multivariate information. While Figure 5 shows that TQNet is still better than baselines when the data is disrupted, this doesn't give much evidence that TQNet actively use multivariate information a lot. The results could just as well be explained by the baselines mostly having learned univariate information. Furthermore, this could also be explained by TQNet being relatively weak at using intra-channel information, and compensating this using inter-channel information.", "methods_and_evaluation_criteria": "The benchmark contains many baselines and datasets, which gives credence to the point that TQNet is a useful model in practice.\n\nHowever, there are three concerns about these results:\n1. Many of the datasets used are known to be easily forecasted using univariate forecasting (such as the ETTh and Solar datasets). This makes it plausible that the results quality is solely due to the improved time-wise interactions.\n2. Giving the model the periodicity of the dataset can be a very useful information which the model doesn't have to handle itself. It is not mentioned whether any of the baselines have also been given this information.\n3. Classical and older models are absent from the baselines, with the oldest baseline being from 2022. At the very least, methods which strongly leverage the periodicity of the data should be included, such as ETS.", "theoretical_claims": "There are no theoretical claim in the paper.", "experimental_designs_or_analyses": "Nothing to mention besides what is already in \"Methods And Evaluation Criteria\".", "relation_to_broader_scientific_literature": "The ablation where the paper shows the TQ attention can be applied to other architecture is a good sign that the ideas presented in the paper may be used to build new models. In particular, while I have doubts about it due to the lack of discrimination between interactions between time steps dt or dt+W time steps apart, the TQ attention could be mixed with other ways to encode temporal distance in a time series model.", "essential_references_not_discussed": "The related work section only considers very recent models. It is therefore lacking in reference to classical statistical techniques, some of which having features that would deserve to be compared with TQNet. The main one would be the ETS technique, which also takes a strong advantage of being given the periodicity of the signal to improve a forecast.", "other_strengths_and_weaknesses": "Beside what is already mentioned in other sections, my main concern is that the quality of the results are solely due to TQNet strongly imposing a specific period to the forecast. To truly determine whether or not this is the case, further experiments should be added. I would personally suggest adding experiments with synthetic data, since they can be tailored to test specific property of the model.", "other_comments_or_suggestions": "Please add the hyper parameters used for both TQNet and the baselines in the appendix. Additionally, if any hyper parameter search was done, it should be detailed in said appendix.", "questions_for_authors": "1. Is there an updated link to the anonymous code repository?\n2. What is the impact of giving an incorrect W to the model? In particular, how does the model react when W is a multiple or a divisor of the full periodicity? An example would be to use 1 day instead of 1 week for the traffic dataset, or using 2 days instead of 1 day for the solar dataset.\n3. If the query vectors are computed only through the time steps up to the period, does this mean that the interactions between time steps t and t' are identical to the interactions between time steps t and t'+W? If this is indeed the case, did you test how TQNet fare on datasets with a strong short-term causal interaction between time steps? One simple example could be a sine wave summed with a random walk.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes the Temporal Query Network (TQNet) for multivariate time series forecasting. Its core innovation is the Temporal Query (TQ) mechanism, which uses periodically shifted learnable vectors as the query (Q) in a single-layer attention module, while the keys (K) and values (V) are derived from the input data. This design aims to capture stable, global inter-variable correlations more robustly and efficiently than traditional self-attention, especially in the presence of noise or non-stationarity. Reviewers acknowledged the model's strong performance on benchmarks, its efficiency, and the novelty of the TQ idea. However, several concerns were raised, particularly regarding the fundamental justification and intuition behind TQ: how fixed (though periodically shifted) queries capture dynamic per-sample correlations (R-kXGG), how inter-variable correlations are learned without explicit constraints (R-kXGG, R-Tew6), potential time-invariance issues (R-kXGG), and the lack of theoretical grounding for robustness claims (R-Tew6). Other key issues included the reliance on a manually set period (W), handling multiple periodicities (R-kXGG), insufficient direct evidence of effective multivariate information use (R-yR9p), the need for comparisons with classical methods like ETS (R-yR9p), and minor clarity issues in figures and algorithms (R-thzz, R-kXGG).

The authors provided extensive rebuttals, including theoretical arguments, clarifications, and significant new experimental results. They explained that TQ captures global patterns while K/V capture local ones, with the attention mechanism integrating both. They argued TQ learns correlations implicitly via optimization and averaging effects from periodic shifting, mitigating noise. They provided new experiments comparing different Q/K origins, showing TQ(Q)/Raw(K) was optimal, and experiments demonstrating how TQ learns global correlations through Q/K interaction. They addressed periodicity concerns by suggesting methods (ACF) to determine W, discussing multi-periodicity handling, and showing robustness to incorrect W values. New experiments demonstrated improved performance with more covariates, multivariate-to-univariate results, and comparisons against ETS and MSGNet. Reviewer responses were mixed: R-thzz was fully convinced, raising their score to Accept after follow-up questions on the model's lightweight design were answered. R-kXGG remained unconvinced by the core TQ explanations and maintained Weak Reject. R-Tew6 acknowledged the rebuttal but did not change their Weak Reject score or provide further feedback. R-yR9p acknowledged the improvements but felt they weren't substantial enough to raise their Weak Accept score.
Decision: Accept (poster)