OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain
Authors: Wenzhen Yue
Yong Liu
Hao Wang
Haoxuan Li
Xianghua Ying
Ruohao Guo
Bowei Xing
Ji Shi
Venue: NeurIPS 2025 poster ()
Keywords: Time series forecasting; Linear-based model; Transformer
Primary area: deep_learning
TL;DR: OLinear is a linear-based time series forecasting model that outperforms state-of-the-art Transformer-based methods with significantly higher efficiency.

Abstract: This paper presents $\mathbf{OLinear}$, a $\mathbf{linear}$-based multivariate time series forecasting model that operates in an $\mathbf{o}$rthogonally transformed domain. Recent forecasting models typically adopt the temporal forecast (TF) paradigm, which directly encode and decode time series in the time domain. However, the entangled step-wise dependencies in series data can hinder the performance of TF. To address this, some forecasters conduct encoding and decoding in the transformed domain using fixed, dataset-independent bases (e.g., sine and cosine signals in the Fourier transform). In contrast, we propose $\mathbf{OrthoTrans}$, a data-adaptive transformation based on an orthogonal matrix that diagonalizes the series' temporal Pearson correlation matrix. This approach enables more effective encoding and decoding in the decorrelated feature domain and can serve as a plug-in module to enhance existing forecasters. To enhance the representation learning for multivariate time series, we introduce a customized linear layer, $\mathbf{NormLin}$, which employs a normalized weight matrix to capture multivariate dependencies. Empirically, the NormLin module shows a surprising performance advantage over multi-head self-attention, while requiring nearly half the FLOPs. Extensive experiments on 24 benchmarks and 140 forecasting tasks demonstrate that OLinear consistently achieves state-of-the-art performance with high efficiency. Notably, as a plug-in replacement for self-attention, the NormLin module consistently enhances Transformer-based forecasters. The code and datasets are available at https://github.com/jackyue1994/OLinear.

OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain
Wenzhen Yue', Yong Liu?, Hao Wang", Haoxuan Li Xianghua Ying"," Ruohao Guo', Bowei Xing', Ji Shi!
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science
and Technology, Peking University, "School of Software, BNRist, Tsinghua University, CCollege of Control Science and Engineering. Zhejiang University, CCenter for Data Science, Peking University (yuewenzhen, hxli 8stu. pku. edu. en, xhying@pku.edu...
Abstract
This paper presents OLinear, a linear-basee multivariate time series forecasting model that operates in orthogonally transformed domain. Recent forecasting models typically adopt the temporal forecast (TF) paradigm, which directly encode and decode time series in the time domain. However, the entangled step-wise de- pendencies in series data can hinder the performance of TF. To address this, some forecasters conduct encoding and decoding in the transformed domain using fixed, dataset-independent bases (e.g., sine and cosine signals in the Fourier transform). In contrast, we utilize OrthoTrans, a data-adaptive transformation based on an orthogonal matrix that diagonalizes the series' temporal Pearson correlation matrix. This approach enables more effective encoding and decoding in the decorrelated feature domain and can serve as a plug-in module to enhance existing forecasters. To enhance the representation learning for multivariate time series, we introduce a customized linear layer, NormLin, which employs a normalized weight matrix capture multivariate dependencies. Empirically, the NormLin module shows a surprising performance advantage over multi-head self-attention, while requiring nearly half the FL.OPs. Extensive experiments on 24 benchmarks and 140 forecast- ing tasks demonstrate that OL consistently achieves trate-oftthe-ee perfor- mance with high efficiency, Notably, as a plug-in replacement for self-attention, the NormLin module consistently enhances Transformer forecasters. The code and datasets are available at wtthtts: :/ggttub/bttt//// wtwm:/acccyue199400aa.....
1 Introduction
Multivariate time series forecasting is critical in fields such weather [wu al. 2023a], trans- portation [Ma et al.) 2021], energy [Zhou et al. 202T]. and finance [Chen et al. 2023 Time series forecasters typically adopt the temporal forecast (TF) paradigm (Wang al. 2025a Liu et al. 2024a
Nie et al. 2023 Wang et al. 2024a], which encodes time series into latent representations and decodes them back, all within the time domain. However, this paradigm struggles to fully exploit the forecasting potential in the presence of entangled intra-series dependencies Yi et al. 2023 Yue
al. 2025). To mitigate this issue, recent studies apply Fourier (Yi et al.) 2024a] 2023 Yue al.
2025 or wavelet [Masserano et al.] 2024] transforms to obtain the decorrelated feature sequence and
perform encoding and decoding in the transformed domain. Nevertheless, these methods rely on teteeet-indeeendeett bases, which fail to exploit the dataset-specific temporal correlation information.
Haoxuan Li and Xianghua Ying are the corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In this paper, we introduce Ortho'Tras a dataset- adaptive transformation scheme that constructs an orthogonal basis via eigenvalue decomposition of the temporal Pearson correlation matrix [Gray and Davisson 2004). Projecting the series onto this basis obtains the decorrelated feature domain, providing a disentangled input for linear encoding and empirically improving forecasting performance. Notably, Ortho'Tran is modular and can be integrated into existing forecasters to enhance their performance. In-depth ablation studies reveal that Ortho promotes representation diversity and increases the rank of attention matrices in Transformer-basee models.
As TrahoTrans transforms complex temporal variations into decorrelated features, the representation learning process can be effectively handled by linear layers (Yi et al., 2023]. Specifically, we employ linear-basee Cross-Series Learner (CSL) and Intra-Series Learner (ISL) to model multivariate correlations and sequential dynamics, respectively. To motivate our design, we note that in the classic self-attention mechanism, the attention entries are all positive with the row-wise I.1 norm fixed as 1. Inspired by this, we design a new linear layer in CSL, called NormLin, where the weight matrix entries are made positive via the Softplus function and then row-wise normalized. Surprisingly, the NormLin module consistently outperforms the self-attention mechanism while improving computational efficiency in the field of time series forecasting. The overall model is referred to as OLinear, and our contributions can be summarized as follows:
contrast to the commonly used TF paradigm, we introduce OrthoTrans, a dataset- adaptive transformation scheme which leverages the orthogonal matrix derived from the eigenvalue decomposition of the temporal Pearson correlation matrix. As a plug-in, consistently improves the performance of existing forecasters.
For better representation learning, we present the NormLin layer, which employs a row- normalized weight matrix to capture multivariate correlations. Notably, plug-in, the NormLin module improves both the accuracy and efficiency of Transformer-base forecast- ers. It also adapts well to decoder architectures and large-scale time series models.
Extensive experiments on 24 benchmarks and 140 forecasting tasks (covering various datasets and prediction settings) demonstrate that OLinear consistently achieves state-of- the-art performance with competitive computational efficiency.
2 Related work
2.1 Transformed domain in time series forecasting
Deep learning based forecasters typically adopt the TF paradigm Wang et al. 2025a 2024a Yu
ct al. 2024 Wang et al. 2024b] Liu et al. 2024a Nie al. 2023 Wu et al. [003b Zeng et al., 2023] with the entire process performed in the time domain. However, this paradigm may underperform
in the presence of strong intra-series correlations (Yi et al.. 2023 Wang et al. 20256). To mitigate
this, recent models propose to forecast in the frequency domain Yi et al. 2024a| Xu et al. 2024
Yi al. 2023) or the wavelet domain [Masserano et al. [024]] where step-wise correlations are reduced and performance improvements are observed. FreDF [Wang et al. 2025b) incorporates frequency-domain regularization term into the loss function to address the issue of biased forecast. However, classical Fourier and wavelet transforms employ fixed, dataset-agnostic bases that do not explicitly use the specific statistical characteristics of the dataset. In this work, we leverage orthogonal matrices derived from the temporal Pearson correlation matrix decorrelate the series data. This approach provides a more suitable input for linear encoding and improves forecasting performance.
2.2 Time series forecasters
Deep learning based time series forecasters can generally be categorized into Transformer-basee
(Wang et al. 2025a Nie al. [203 Liu et 2024a Wang et al. 2024b Yu et al. 2024 linear-
based [ang et al. 2024a Zeng et al. 2023 Yi et al. 2023 Xu et al. 2024, Yi et al.) 2024a].
TCN-based Wu et al. [202b Luo and Wang 2024], RNN based Lai et al. 2018 Rangapuram et al..
2018], and GNN-based [Huang et al. 2023a Yi et al. 2024b) models. Recently, research interests increasingly focus on Transformer-based and linear-based methods, each offering distinct advantages. Transformer-based forecasters typically exhibit strong expressiveness, whereas linear- based models offer better computational efficiency. In this paper, we aim to achieve state-of-the-art performance using efficient based model. Specifically, we employ the linear- based NormLin module to
2

---

rans
TxT
Orthogonally Transfomed Domain
QN
Qo
Representation Leaming
Pattten Linon
Linear Frassding
Dappocsr
Instra Nom
Mat Mul
Cross- Series Learner (CSL.
Setra-Series
Mat Mul
Nor
Leamer (ISL.
NxdxT
Nxdxr
N<T
NxdxT
Nkr
Cross- Series Linear Learner
Intra- Series Linear Learner
Mat. Nx N
Softplus
L1 Norm wice
NormLin
Module
Addd LaperNoom
Add & Layer
Lines
NormLin
Layer
Mat
N.d=
N=dx
Figure 1: Overall structure of OLinear. The orthogonal matrix derived from the series' temporal Pearson correlation matrix transforms the series into the feature domain with removed sequential correlations. The cross-series learners and intra-series learners are then employed for robust represen- tation learning. Specifically, introduce the NormLin layer, whose weight matrix is processed by the Softplus function and row-wise LI normalization, to capture multivariate correlations.
robustly model inter-variate correlations. Remarkably, the NormLin module consistently outperforms the self-attention mechanism Transformer- forecasters, highlighting the strong capability of linear layers for time series forecasting.
3 Preliminaries
Real-world time series often exhibit temporal dependencies, where each time step is influenced by its predecessors. In this section, we derive the expected value of future step given its past, assuming multivariate Gaussian distribution. Without loss of generality, we consider a series x E RR, and the following time step y E R. We now present the following theorem.
Theorem (Expected value of y). We assume that NN (Hxx Ex), N (Mys oz), and the joint Here, N ((,)) denotes Gaussian distribution oy
distribution (x, N (pxx
Σx ET
Exy
with the specified mean vector and covariance matrix. Ex Btxt is symmetric and positive definite. Exy E Rix! denotes the covariance between x and y. Then, the expected value of given x is
5- (x px) xy
Myyx = My
The proof of Theorem is provided in Appendix A Equation [ shows that for temporally correlated series [x, y/. the expected value of y depends not only on its own mean By. but also on its past observations
The second term in Equation [ introduces additional difficulty to forecasting under the TF paradigm. In contrast, decorrelating the series simplifies the forecasting task. this work, introduce Ortho'Trans to transform the original series into decorrelated transformed domain, converting the temporal forecasting problem into the feature prediction task. Experiments in Section 5.2 validate the effectiveness and generality of this transformation strategy.
4 Method
Problem formulation For multivariate time series forecasting, given a historical sequence X = {xI,-- xx) E Nxx with I' time steps and N variates, the task to predict the future time steps Y = (xr+1, (x Nxx Our goal is to approximate the ground truth Y as closely
possible with predictions Y.

---

4.1 Overall architecture
As shown in Figure T OLinear adopts a simple yet effective architecture. Given the input series X, RevIN [Kim et al. 2021 layer first performs instance normalization to mitigate non-stationarity. A dimension extension module then enhances expressiveness Yi et al. [023] by computing the outer product with learnable vector &a RR where is the embedding size. Next. the time domain is decorrelated by multiplying with a transposed orthogonal matrix Q! E TR T'xT which will be detailed Section 4.2 This process can be formulated as Z = (RevIN Norm where & denotes the outer product.
(X) 8 ②a) Q [RNxdx"'
We then perform encoding and forecasting in the transformed domain. Specifically, a linear layer first encodes the decorrelated features Z to the model dimension D. The cross cross-scries series learner (CSL) and intra-series learner (ISI.) subsequently capture multivariate correlations and model intra-series dynamics, respectively. After passing through I. stacked blocks, the representation is decoded to the desired prediction length r, and then mapped back to the time domain via multiplication with the orthogonal matrix Qo- The overall process is summarized as:
H LinearEncode (Z) NxdxD
H ISL (CSL
[
1 0, L-
Y LincarDecode
H²)) Qo E Rxxxxr
Finally, the flattened output is mapped to shape N x T via a linear layer, then de-normalized to yield the final prediction: Y RevvIDeNNrm FlattenLinear
Y
4.2 Orthogonal transformation (OrthoTrans,
One effective approach for decorrelating the series is based the Pearson relation matrix. Let trrain E IRN N
Time Domain
Trunsformed Domain
CorrMaae
First Halt
1.0
BCL
denote the training set, where M is the length of the training series. For each variate j. we generate T lagged series with temporal offsets from O to 1: s = X i: M- + i - T where si denotes the i-th tem- porally lagged series of variate j. We then compute the Pearson correlation matrix CorrMat of {s'),, whose (p. g) entry is Cov(ep.8,) (Varee))-Var(e.)) P. E (0.....
100
Ch
0.5
PEMSOS
Half Second Hall"
0.0
Figure 2: Comparison between the time and trans- formed domains. (1) The series exhibits strong con- sistency in the transformed domain, which is desirable for forecasting. (2) Temporal correlations are effectively reduced the transformed domain. (3) Periodicity is clearly reflected in the correlation matrix, which facili- tates modeling. (4) The new domain typically exhibits energy compaction, with only few dominant compo-
1}.
Here, Cov(-) denotes the covariance of two series, and Var(.) denotes the vari- ance. The final Pearson correlation matrix then obtained by averaging over all vari-
nents. More cases are shown in Figure 7
ates: CorrMat, = Σ 0 CorrMat/-
Following the above procedure, the resulting CorrMat, symmetric with all diagonal entries equal to Based on the properties of symmetric matrices, we perform eigenvalue decomposition as CorrMat, = A.Q.. where Q; an orthogonal matrix (Horn and Johnson 2012) whose columns are the eigenvectors of CorrMat, [Bishop and Nasrabadi 2006). and A is diagonal matrix containing the corresponding eigenvalues.
For the input series x with temporal correlation matrix CorrMat, and unit variance, the covariance ma- trix of the transformed vector Q, is CovMat (Q₂ x) Q CorrMat, Q; = Q Q.Q, Q; A. which is diagonal matrix. Therefore, the entries of Q x are linearly uncorrelated, removing sequen- tial dependencies. Similarly, we can compute Q E I and recover the temporal correlations

---

Incorporating the other components in by multiplying with Qo. Note that Q and Q are pre-computed and used throughout training and inference. Ablation studies on these two matrices are discussed in Appendix [.3] Moreover, as shown in Appendix [.T0] OLinear remains robust even when the orthogonal matrices Q; and Qo are computed with limited training data.
Discussion (1) Mathematically, Q. x projects onto the eigenvectors of the temporal correlation matrix CorrMat, The transformed domain exhibits energy compaction, and noise is suppressed in the primary components (Bishop and Nasrabadi| 2006). (2) Compared DFT and wavelet transforms, or to the TF paradigm without any transformation, Ortho'Tran produces higher-rank attention matrices for Transformer- based forecasters (see Figures [0 [4], indicating greater representation diversity and enhanced model expressiveness. This partly explains why OrthoTrans enhances other forecasters when used as a plug-in module.
4.3 Representation learning
CSL and NormLin Inspired by the properties of attention matrices in self-attention Vaswani et al. 2017]. we impose two constraints on the weight matrix of the linear layer for robust multivariate representation learning: (I) all entries must be positive, and (2) each row must sum to 1. To enforce these constraints, we apply the Softplus function followed by row-wise LI normalization to the learnable weight matrix W. The resulting layer is referred to as NormLin, and is defined as follows:
NormLin (x) RowNormLI (Softplus (W)) x, W N E
The variants of NormLin are discussed in Appendix J.1 Figure (b). the CSL process formulated as:
CSL [ LayerNorm [ Linear (NormLin (Linear
H
where Linear(-) and NormLin(-) operate on the sequence and variate dimensions, respectively. We define the NormLin module as NormLinModule() (-) & Linear (NormLin (Lincar ()). Ablation studies on the two linear layers in NornLinModule (-) are presented in Appendix J.2
ISL We adopt two linear layers separated by the GELU activation function as a powerful predictive representation learner [Li et al. 2023). It has been well established that MLPs are highly effective for encoding sequential dynamics and decoding future series (Zeng et al. 2023 Similar to CSL, residual connections and LayerNorm are also applied. The complete ISL process defined as:
ISL H = LayerNorm H + Linear
(ELU
Linear
H
(5)
Discussion Compared to the self-attention mechanism, the NormLin module offers the following advantages: (1) As shown in Table [ it reduces computational com- plexity by half and decreases memory footprint by factor of 1 /hh, where h is the number of attention heads; (2) The learned weight matrix in NormLin Lin naturally ex- hibits high rank, in contrast to the low-rank nature of self-attention (see Appendix EE.2. The low-rank issue in self-attention could arise from the sharp value concen- tration of the Softmax function. A higher-rank weight (or attention) matrix often better preserves the rank of the representation space and thus improves the model's expressiveness (Han et al. 2023). (3) From the perspective of gradient flow, the NormLin layer provides a more direct backpropagation path for optimizing weight entries. Appendix B shows self-attention and Normr,
Table I: Comparison of FLOPs and mem- ory usage between the NormLin mod- ule and multi-head self-attention (MHSA). Here, h denotes the number of attention heads, typically set to 8.
Module
Norml Module
MHSA
FLOPPs O (N D 2NDP) o (2N2 AND?)
Memory
O (N* + ND) O (hN2 - ND)
(Softplus(-)) (Soll, in the NormLin layer share a similar structure, but the latter offers greater flexibility. (4) The query-key mechanism of self-attention induces correlations in attention matrix entries, restricting the optimization space. In contrast, NormLin decouples these dependencies, allowing for more flexible parameter updates.
that the Jacobian matrices Soft
5

---

Table 2: Long-term forecasting results with prediction lengths {12, 24, 48, 96) for PEMS and T E (96, 192, 336, 720) for others. Lookback horizon T = 96. Results are averaged over four
prediction lengths. Aug denotes further averaging subsets. Full results are shown in Table I7
OLinear
TimeMix.
Fitterett
DLineur
TimeMix.++
Loddum
CARD
Frediormer
iThns..
PatchTST
TimesNet
Model
(Ours)
2024a
202
2023
2025a
2024
2024bb
2024
2024a
2023
20236
Metric
MAE
MSE
MAE
MAE
MSE
MAE
(TT((vg)
0.359 0.376 0.367 0.388 0.375 0.394 0.442 0.444 0.349
0.577 0.367 0.387 0,366 0,380 0.366 0.385 0.383 0.399 0,380 0.396 0.391 0.404
ECL
0.159 0.248 0,182 0.273 0.173 0.268 0.212 0,300 0,165 0.253 0.169 0.263 0,168 0.258 0.1 0.269 0.178 0.270 0,208 0,295 0.192 0.295
Exchange
0.355 0.399 0,387 0.416 0,388 0.419 0.354 0.414 0.357
0.409 D. 8.354 0,402 O. 0.362 0,402 0,333 0.391 0,360 0:403 0.367 0:404 0.416 0.443
Traffic
0.451 0.247 0.485 0. 298 0,463 0.310 0.625 0.383 0.416
0.264 0.467 0.294 0.453 0,282 0.433 0.291 0,428 0.282 0.531 0.343 0.620 0.336
Weather
0.237 0.260 0.240 0 0.272 0.245 0.272 0.265 0.317 0.226 0.262 0.242 0.272 0.239 0.265 0,246 22 0,258 0.279 0,259 0.281 0.259 0.287
Solar 0.215 0.217 0.216 0.280 0.235
0:401
0.258
0.230 0.264
0.237 0.226
0,233
0:307 0,301 0.319
PEMS03
0.199 0.167
0.145 0.251 0.375
0.263
0.107 0.210 O. 0.275 0.135 0.113 0.221 0.291
PEMS04
0.091 0.190 0,185 2828 16 0.258 0,388
0.251 0.103 0.210 206 0.299 162 261 0,111 0.221 0.195 0:307 0.241
PEMS07
0.077 0.164 0.181 0.271 0.123 0.229 0.329 0.395
0.152
0.258
0.084 180 O. 0.247 0.121 0.222 0.101 0.204 0.211 0.303
0.124 0.225
PEMS08 0.113 0.194 0.226 0.299
0.172 0.260 0.379
0.416
0.200
0.279
0.122 0.211 0.201 0.280 0.161 0.250 0.150 0.226
0,280
0.321
0.193 0.271
1 Count
0
0
0
0
0
0
0
0
0
Table 3: Short-term forecasting results. Two settings are applied: S1 (Input-12 12, Predict- (3, 6, 9, 12)) and S2 (Input-36, Predict- (24, 36, 48, 60)). Average results across eight prediction lengths are reported. Full results are provided in Tables and 19
OLinear
TimeMix.
FilterNet
DLinear
+it
Leddam
CARD
Fredformer
iTrans.
PatchTST
TimesNet
Model
(Ours)
2024a
2024g
2023
2025a
2024
2024b
2024
202-4a
2023
20236
Metric
MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE
MSE
MAB
IIJI
1,429 0.690 1,864 0.806 1,793 0.791 2 2.722 1,126 1,805 0.793 1.725 0.777 1.959 0.822 1,732 0.797 1715 0.773 1,905 0.804 1,809 0.807
COVID-19 5.187 1,211 5,919 5,607 1,322 8.279 1,601 5,974 1,369 5.251 1,285 5,536 1,314 5,279 1,287 5 5.301 1,293 5,836 1.362 6,106 1.369
METR-LA
0,587 0.311 0.608 0.372 0,603 0,366 O. 58580 0,422 0,567 0,363 O 0.603 0,367 0.639 O 0.3 0.617 617700 0.369 O 6.627 0,373 0.614 0,372 0.617 0,370
NASDAQ
0.121 0.201 0,120 0.204 0.127 0.231 I 1,150 0.251 0,125 0,210 0.128 0.21 0.125 0,207 0.127 0,210 0.133 0.217 0.128 0.209 0.161 0.247
Wiki
6.395 0.415 6.443 0:439 6.457 0.439 6,420 0,510 6.430 0.443 6.417 0.435 6.419 0.427 6,318 0.429 6:422 0.432 6,368 0.424 7.633 0,572
SP500
0.146 0.250 O. O 262255 O. 164 0.279 D. 178 298 0.157 0,270 0.163 0,282 D. 142 0.252 167 0.286 O. 0.161 0.279 0.1.9 0.277 0.150 0.262
Dowlones
7.686 0.619 8.499 0 0.633 8.283 0.633 7.893 0.626 8.895 0.643 8.257 0.633 7. 7.699 0.619 8.041 O. 0.625 8.177 0.630 7.991 0.626 10.960 0.737
CarSales
0,330 0.305 0,322 328
387000000 0.3
0.321 0.335 O. 3.47 0,324 O.3
0.322 0.318 0.334
1.248 0.835 0,840 1,309 1.278 0.870
0.841
0.868
0.847
1.324 0.874
Website
0.225 0,311 0,279 D. 0.297
0,302
0.389
0,344
0,264
0,348
0.303 O. 0,266 0.351 0.179 0.297 0,362
Unemp
0.729 0.461 1,581 0.708 1,286 0.627 0.565 0.509
1.506
0.678
1.502 0.689 1.163 0.596 2.048 0.789 1,088 0.666
1.237
0.624
2.328 0.852
1s Count
4
10
0
0
1
0
0
0
0
2
0
0
5 Experiments
Datasets and implementation details We extensively evaluate OLinear using 24 diverse real- world datasets: ETT (four subsets). Weather, ECL, Traffic, Exchange, Solar-Energy. PEMS (four subsets), ILI, COVID-19, METR-LA, NASDAQ. Wiki, SP500. DowJones, CarSales. Power, Website, Unemp. The weighted LI loss function from CARD [Wang et al. 2024b] adopted. The embedding size is set as 16. Dataset description and more implementation details are presented in Appendices and D respectively.
5.1 Forecasting performance
Baselines We carefully choose 11 well-acknowledged -tate-of-the-art forecasting models as our
baselines. including (1) Linear-based models: TimeMixer [Wang et al.) 2024a]. FilterNet [Y)
ct al., 2024a]. FITS [Xu et al., 2024]. DLinear [Zeng et al. 2023 (2) Transformer-bet based mod-
els: TimeMixer++ [Wang et al. 2025a]. Leddam (Yu et al. 2024], CARD [Wang al.) 2024b
Fredformer [Piao et al. [024]. rannffrrer [Liu et al. 2024a]. PatchTST [Nie et al. 2023
TCN-based model: TimesNet [Wu 2023b
Main results Comprehensive long-term and short-term forecasting results are presented in Tables 2 and 3 respectively, with the best results highlighted in bold and the second-best underlined. Lower MSE values indicate more accurate predictions. Across wide range of benchmarks, OLinear

---

Weather
ECL
Traffic
Solar-Energy
PEMS03
CarSales
II.1
0.15
Ground Truch
Prediction
Figure 4: Visualization of the forecasting results of OLinear, demonstrating great accuracy.
Table 4: Ablation study transformation bases. S/ and S2 represent 'Input-12, Predict- 6, 12)" and IIpuut36,, Predict-(24, (24, 36, 48, 60)", respectively. Waveletl and Waveler2 use the Haar and discrete Meyer wavelets. Identity denotes transformation. Full results are in Table 20
Dutaset
ECL
Solar-Energy
PEMSU3
Power (S2)
ILI (S1)
COVID (S2)
METR-LA (S2)
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Ours
0.159
0.248
0.215
0.217
0.095
0.199
1.487
0.922
1.094
0.578
8.467
1.754
0.838
0.402
Fourier
0.161
0.250
0,219
0.219
0.101
0.204
1.614
0.967
1.268
0.584
9.165
1.839
0.843
0.403
Waveletl
0.160
0.249
0.221
0.221
0.107
0,210
1.663
0.987
0.116
0.580
8.666
1.799
0.840
0.404
Wavelet2
0.162
0,251
0.226
0.224
0.108
0.210
1.664
0.987
1.177
0.594
8.949
1,840
0.843
0,406
Chebyshev
0,218
0,295
0,226
0.226
0.105
0.207
1.570
0.965
1.217
0.597
9.330
1.875
0,407
Laguerre
0.167
0.255
0.233
0.230
0.111
0.214
1.659
1.353
0.651
9.302
1.890
0.868
0.420
Legendre
0.161
0.250
0,243
0.235
0.109
0.213
1.685
0.995
1.177
0.603
8.550
1.798
0,841
0.404
Identity
0.163 0.252
0.227
0.225
0.106
0.209
1.542
0.945
1.153
0.587
8.856
1,819
0.848
0.408
consistently outperforms state-of of-the-rt Transformer-bass and based forecasters. Notably, these gains achieved with high computational efficiency. (Figure 3 and Table 4T)
We attribute this superior performance to the adopted Orahs and NormLin modules. The effectiveness of the simpler NormLin module challenges the necessity of the widely adopted multi-head self-attention mechanism, which has been dominant design in prior works. To further validate robustness, we evaluate OLin- under varying lookback lengths (Table 29). where it consistently outperforms existing state- of-the-art methods.
Solar Energy
Traflee
PRESTITT
ITreAs
Freftin
Prodfie
Levontlan)
CART
Poach1 NT
CARD
Tmitiing Sime Staiiiey
Trainnn timittery
Figure 3: Model efficiency comparison. Bubble areas represent GPU memory usage during train- ing, scaled independently in the two subfigures. Resource footprint data are from Table 41
Figure shows the prediction visualizations of OLincar. Moreover, Table 14 demonstrates that OLinear exhibits greater robustness to random seeds compared to state-of-the-ar Transformer-based forecasters such as TimeMixer++ and iTransformer.
5.2 Model analysis
Transformation bases We replace Ortho- Trans with several commonly used bases, includ- ing the Fourier basis, two wavelet bases (Haar and discrete Meyer wavelets), and three polyno- mial bases. As shown in Table 4 our method consistently outperforms all alternatives. Specif- ically, it achieves 5.5% reduction in MSE over the Fourier basis, and a 10.0% improvement the no-transformation baseline (i.e., the TF paradigm) on PEMS03.
Table 5: Applying OrthoTrans (O.Trans) to iTrans- former, PStchTST and RLinear. Average MSEs are reported. Full results are in Table 21
TTrans. 2024g
PatchTST 2023
RLingar
Model
2023
Van.
+O.Trans
Van.
+O.Trans
Van.
+O.Trans
ETTml BCL
0.407
0.404
0.385
0.384
0.414
0.408
0.171
0.208
0.214
PEM503
0.113
0.103
0.180
0.163
0.495
0.477
PEMS07
0.101
0.085
0.211
0.147
0.504
0.485
Solar
0.233
0.228
0.270
0.239
0.369
0.354
OrthoTrans as plug-in We further integrate TrahoTrans into three classic forecasters: iTrans-
Weather
0.258
0.252
0.259
0.246
0.272
0.269
METR-LA

---

former, PatchTST, and RLinear [i ct al. 2023]. As shown in Table OrthoTrans yields average 1 iTransformer and PatchTST, respectively, highlighting
MSE improvements of 5.1% and 10.1% for the benefit of incorporating dataset-specific statistical information into the model design. This im- provement can be attributed to the increased attention matrix rank introduced by OrthoTrans (see which indicates enlarged representation space and enhanced model capacity.
Appendix E.3
Representation learning validate the rationality our CSL and ISL designs, we conduct ablation studies by replacing removing their core components -i and (standard) linear layers. As shown in Table []] our design- design-applying NormLin along the variate dimension and standard linear layers along the temporal Notably, applying NormLin along the variate dimension consistently outperforms its self-attention counterpart (last row), with reduced computational cost. Furthermore, removing the ISL module (third-last row) results in a 6.2% performance decline, highlighting the importance temporal representations. Interestingly, on small-scale datasets with fewer variates (e.g., NASDAQ and ILD, the model with only temporal linear layers (third row) exhibits competitive performance, implying that NormLin is more beneficial when handling larger number of variates.
coneension-cossiteeene achieves the best performance.
updating
Table 6: Ablations on the CSL and ISL design. 'NormLin (Temporal)' replaces the linear layer in Eq. 5 with the NormLin layer, and 'Linear (Variate)' replaces the NormLin layer in Eq. with standard linear layer. 'Attn.' denotes the self-attention mechanism. Results are averaged over four prediction lengths. Full results are shown in Table 22
Traflic
PEMS03
Weather
ETTml
NASDAQ (SI)
(S2)
Variate
Temp.
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
NormLin
Linear
0.159
0.248
0.451
0.247
0.215
0.217
0.095
0.199
0.237
0.260
0.374
0.377
0.055
0.125
1.764
0.802
Linear
Linear
0.178
0.320
0.121
0.261
0.377
0.837
w/o
Lincar
0.178
0.259
0,482
0.257
0.241
0.232
0.147
0.234
0.247
0,266
0.378
0.379
0.054
0.124
1.864
0.823
NormLin NormLin 0.169 0.257
0.275
0.214
0.261
0.055
0.126
0.836
Linear
NormLin
0.183
0.276 0,578
0.339
0.262
0.254
0.143
0.246
0.240
0.261
0.383
0.384
0.057
0.133
2.037
0.867
w/o
in
0.185
0.290
0.182
0.269
0.265
0.384
0.874
NormLin
w/o
0. 0.169
0.257
0.460
0.275
0.253
0.241
0.114
0.215
0.239
0,261
0.380
0.382
0.055
0.126
1.940
0.837
Lincar
0.183 0.276
0.341
0.262
0.254
0.142
0.246
0.262
0.384
0.384
0.057
0.132
2.073
0.874
Atin.
Linear
0.166 0.255 0.457
0.251
0.220 0.221
0.097 0.202
0.244
0.265
0.391
0.389
0.056
0.126
2,022
0.847
OLinear-C As shown in Figure 5 the learned weight matrix resemble Softmax (CorrMat,)) where CorrMat, is the Pearson correlation matrix across variates. Mo-
ECL
PEMS06
Weather
CorMont Nir)) Nommin in Weight
0.35
tivated by this, we replace the Icarnable weights in
NormLin with the pre-computed Softmax (CorrMat,)), resulting in a simplified variant: NormLin, (x) Softmax (CorrMat,)) matrix in NirmLin is optimized to align with the cor- relation matrix among channels CorrMat,, We refer the model with this NormLin Li variant as OLinear-C. Ta- ble shows that OLinear-C performs comparably to OLin- ear with reduced learnable parameters. Comprehensive evaluation of OLinear-C and the variants of NormLin, are discussed in Appendix H
0.10
This indicates that the weight
Figure The learned NormLin weights exhibit similarity to the multivariate cor- relation matrix (after Sof tmax).
5.3 Generality and scalability of the NormLin module
In our CSL module, we employ the NormLin conduue-comprisinng the NormLin layer and its associated pre- and post-lincar layers (Equation (- capture multivariate correlations. Despite its simple architecture, the NormLin module demonstrates strong capability, generality, and scalability, Specifically, it consistently outperforms the classic multi-head self-attention mechanism and its variants, offering a compelling alternative for token dependency modeling in time series forecasting.
Comparison with attention mechanisms To assess its effectiveness, we compare Normlin Ln with classic attention variants, such as Reformer (Kitaev al. 2020), Flowformer [Wu et al.) [022].
FLatten [Dao et al. 2022). and Mamba [Gu and Dao 2023 As shown in Table 8 NormLin
8

---

Table 7: Performance comparison of OLinear and OLinear-C. Full results are shown in Table 25
Dataset
ECL
TTraflic
ETT
Solar
PEMS
CarSales
ILI
COVID-19
Unemp
Metric
MSE
MAE
MSE MAE
MSE
MAE
MSE
MAB
MSE
MAE MSE MAE
MSE
MAE
MSE
MAE
MSE
MAE
OLincar
0.159 0.248
0,451
0.247
0.359
0.376
0.215
0.217 0.094 0.187 0.330
0.305
1.429
0.690
5.187
1,111
0.461 0.766 0.474
0.729
OLinear C. 0.161 0.249 0.451
0.247 0.359 0.376 0,215 0.217 0.094 0.187 0.330 0.305
1.463
0.698
5.346
1.247
Table 8: Comparison of the NormLin module with state-of-the the-art attention variants. Average MSEs are reported. Full results are in Table 23
Table 9: Applying NormLin (N.Lin) to state-of- the-art Transformer-bat forecasters. Average MSEs are reported. Full results are in Table 24
NomaLin Trans.
Refm. Flowfm.
Flatten
Mamba E.Atin.
iTmns
PatchTST
Leddam
Fredformer
Dataset
Dataset
(Ours)
2017
2020
2022
2023
2023
2025
Van.
N.Lin
Van.
NiLin Van.
N.Lin
Van.
N.Lio
ECL
0.159
0,166
0.167
0.165
0.164
0.176
0.159
ETTml
0.407
0.388 0.387
0.379 0.386 0.381 0.384 0.381
Traffic
0.457
0.459
0.460
0.464
0.439
ECL
0.169 0.165
0.169
PEMS03
0.097
0.101
0.097
PEMS03
0,113
0.102 0.180 0.146 0.107 0.103 0,134 0.108
Weather
0.237
0,244
0.241
0.242
0.246
0.242
0.241
PEMS07
0.101
0.086 0.211 0.168 0.084 0.082 0.121 0.096
Solar
0.215
0.223
0.216
0,222
0.231
0,228
0.217
Solar
0,233 0.226 0.270 0.237 0.230 0.222 0.226 0.226
ILI
1.764
2.022
1.821
1,881
2.134
1,950
1.878
Weather
0.258
0.245 0.259 0.245
0.242
0.242
0.246 0.240
NASDAQ
0.055
0.056
0.055
0.055
0.057
0.055
0.055
METR-LA
0.338
0.328
0.335
0.333 0.327
0.320
0,336 0.329
consistently outperforms these methods, indicating that simply normalizing the weight matrices can be more effective than self-attention for time series forecasting (see Paragraph 4.3).
NormLin as a plug-in To demonstrate the generality of the NormLin module, we replace self- attention with NormLin in transformer-basee forecasters. As shown in Table this substitution leads to notable MSE improvements 6.7% for iTransformer and 10.3% for aog-and-play effectiveness. These results also validate NormLin's in's capability to model dependencies across multiple token types (e.g., variate, temporal patch, and frequency-domai patch), highlighting its potential as a universal token dependency learner for time series forecasting. Furthermore, the Norml .in module consistently improves both training and inference efficiency across these models (see Table 42): for example, it boosts iTransformer's inference efficiency by average of 53%.
its
Scalability of the NormLin module Decoder-only Transformers are currently the de facto architecture of large time series models [Liu et al. 2024b Ansari let al. 2024 Das et al. 2024). We take Timer [Liu et al. 2024b as a representative model and replace its self-attention with our NormLin module. To align with Timer's decoder-only structure, we apply a causal mask by zeroing out the upper triangular part of NormLin's 's weight matrix. The modified model is pre- pre-trained on the UTSD dataset (Liu et al. 2024b]. which spans seven domains and contains up to 1 bil- lion time points. As shown Table TO No improves performance in both zero-shot and fine- tuning scenarios. For example, the zero-shot MSE on ETTh2 and ETTml reduced by 12.7% and 8.4%, respectively. These results demonstrate that the NormL.in module adapts well to decoder-only architectures and scales effectively to large-scale pre-training scenarios.
Table 10: Applying NormLin to the large time series model Timer. For fine-tuning, 5% sam- ples of trainset are used. MSEs are reported.
Fine-tuning
Zero-shot
Dataset
Van.
NormLin
Van.
NormL.n
ETTh1
0.362
0.360
0.438
0.404
ETTh2
0,280
0.269
0.314
0.274
ETTml
0.321
0.309
0.690
0.632
ETTm2
0.176
0.172
0.213
0.212
ECL
0.132
0.130
0.192
0.183
Traffic
0.361
0.353
0.458
0.462
Weather
0.151
0.149
0.181
0.174
Conclusion
In this work, we present OLinear, simple yet effective linear-based forecaster that achieves state-of- the-art performance, built on two core components: (1) OrthoTrans an orthogonal transformation that decorrelates temporal dependencies to facilitate better encoding and forecasting, and (2) the Normlin module, a powerful and general-purpose token dependency learner. Notably, both modules consistently improve existing forecasters when used as plug-ins. We hope Ol .neer will serve as strong baseline and inspire future studies in time series forecasting.

---

Acknowledgments
This work was supported by Beijing Natural Science Foundation under Grant No. L227029, and the National Natural Science Foundation of China (NSFC) under Grant No. 6237 and Grant No. 623B2002.
References
Haixu Wu, Hang Zhou, Mingsheng Long, and Jianmin Wang. Interpretable weather forecasting for worldwide stations with unified deep model. Nature Machine Intelligence, 5(6):602-611, 2023a.
Changxi Ma, Guowen Dai, and Jibiao Zhou. Short-term traffic flow prediction for urban road sections based on time series analysis and lstm_ bilstm method. IEEE Transactions on Intelligent Transportation Systems, 23(6):5615-566 5624, 2021.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, volume 35, pages 1 11106-111 111 15, 2021.
Zonglei Chen, Minbo Ma. Tianrui Li, Hongjun Wang, and Chongshou Li. Long sequence time-series forecasting with deep learning: A survey. Information Fusion, 97:101819, 2023.
Shiyu Wang. Jiawei Li, Xiaoming Shi, Zhou Ye, Baichuan Mo, Wenze Lin, Shengtong Ju, Zhixuan Chu, and Ming Jin. Timemixer++: general time series pattern machine for universal predictive analysis. In ICLR, 2025a
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma. and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In ICLR, 2024a.
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. ICLR, 2023.
Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun I Luo, Lintao Ma, James Y. Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting. ICLR, 2024a.
Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao. and Zhendong Niu. Frequency-domain mips are more effective learners in time series forecasting. In NeurlPS, 2023.
Wenzhen Yue, Yong Liu, Xianghua Ying, Bowei Xing, Ruohao Guo, and Ji Shi. Freeformer: Frequency enhanced transformer for multivariate time series forecasting. arXiv preprini arXiv:2501. 139999 2025.
Kun Yi, Jingru Fei, Qi Zhang, Hui Hc, Shufeng Hao, Defu Lian, and Wei Fan. Filternet: Harnessing frequency filters for time series forecasting. Advances in Neural Information Processing Systems, 17:55115-55140. 2024a.
Luca Masserano, Abdul Fatir Ansari, Boran Han, Xiyuan Zhang, Christos Faloutsos, Michael W Mahoney, Andrew Gordon Wilson, Youngsuk Park, Syama Rangapuram, Danielle C Maddix, et al. Enhancing foundation models for time series forecasting via wavelet-based tokenization. arXiv preprint arXiv:24/2.05244. 2024.
Robert M Gray and Lee D Davisson. An introduction to statistical signal processing. Cambridge University Press, 2004.
Guoqi Yu, Jing Zou, Xiaowei Hu, Angelica I Aviles-Rivent Jing Qin, and Shujun Wang. Revitalizing multivariate time series forecasting: Learnable decomposition with inter-series dependencies and intra-series variations modeling. In JCML. 2024.
Xue Wang, Tian Zhou, Qingsong Wen, Jinyang Gao, Bolin Ding, and Rong Jin. Card: Channel aligned robust blend transformer for time series forecasting. In ICLR, 2024b.
10

---

Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet:
Temporal 2d variation modeling for general time series analysis. In ICLR, 2023b.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In AAAI, volume 37, pages 11121-11128, 2023.
Hao Wang, Lichen Pan, Yuan Shen, Zhichao Chen, Degui Yang, Yifei Yang, Sen Zhang, Xinggao Liu, Haoxuan Li, and Dacheng Tao. Fredf: Learning to forecast in the frequency domain. In ICLR, 2025b.
Zhijian Xu, Ailing Zeng, and Qiang Xu. Fits: Modeling time series with 10k parameters. In ICLR, 2024.
Donghao Luo and Xue Wang. Modernten: A modern pure convolution structure for general time series analysis. ICLR, 2024.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and and-aad short-term temporal patterns with deep neural networks. In SIGIR, pages 95-104, 2018.
Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting, NewrIPS, 31, 2018.
Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. Crossgnn: Confronting noisy multivariate time series via interaction refinement. NeurlPS, 36:46885-46902, 2023a.
Kun Yi, Qi Zhang, Wei Fan, Hui He, Liang Hu, Pengyang Wang, Ning An, Longbing Cao, and Zhendong Niu. Fouriergnn: Rethinking multivariate time series forecasting from pure graph perspective. Advances in Neural Information Processing Systems, 36, 2024b.
Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. ICLR. 2021.
Roger A Horn and Charles R Johnson. Matrix analysis. Cambridge university press, 2012.
Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. NIPS, 30, 2017.
Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint 17211 2023.
Dongehen Han, Xuran Pan, Yizeng Han, Shiji Song, and Gao Huang. Flatten transformer: Vision transformer using focused linear attention. In ICCV, pages 5961-5971, 2023.
Xihao Piao, Zheng Chen, Taichi Murayama, Yasuko Matsubara, and Yasushi Sakurai. Fredformer: Frequency debiased transformer for time series forecasting. In SIGKDD, 2024.
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. ICLR, 2020.
Haixu Wu, Jialong Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Flowformer: Linearizing transformers with conservation flows. In ICML, 2022.
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752 2023.
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. NeurIPS, 35:16344-16359, 2022.
Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer: Transformers for time series analysis at scale. In ICML, 2024b.
11

---

Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, et al. Chronos: Learning the language of time series. arXiv preprint arXiv:2403.07788 2024.
Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. decoder-only foundation model for time-series forecasting. In Forty-first International Conference on Machine Learning, 2024.
Hao Wang, Zhiyu Wang, Yunlong Niu, Zhaoran Liu, Haozhe Li, Yilin Liao, Yuxin Huang, and Xinggao Liu. An accurate and interpretable framework for trustworthy process monitoring. IEEE Transactions on Artificial Intelligence, 5(5):2241-2255 2023.
Haotian Wang, Haoxuan Li, Hao Zou, Haoang Chi, Long Lan, Wanrong Huang, and Wenjing Yang. Effective and efficient time-varying counterfactual prediction with sate--pace models. In The Thirteenth International Conference Learning Representations, 2025c.
Lei Wang. Shanshan Huang, Chunyuan Zheng, Jun Liao, Xiaofei Zhu, Haoxuan Li, and Li Liu. Mitigating data imbalance in time series classification based on counterfactual minority samples augmentation. In Proceedings of the 31tt ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 2973, 2025d.
Hao Wang, Haoxuan Li, Xu Chen, Mingming Gong. Zhichao Chen, et al. Optimal transport for time series imputation. In The Thirteenth International Conference on Learning Representations, 2025e.
Hao Wang, Licheng Pan, Zhichao Chen, Xu Chen, Qingyang Dai, Lei Wang, Haoxuan Li, and Zhouchen Lin. Time-ol: Time-series forecasting needs transformed label alignment. Advances in Neural Information Processing Systems, 2025f.
Xinyu Li, Yuchen Luo, Hao Wang, Haoxuan Li, Liuhua Peng. Feng Liu, Yandong Guo, Kun Zhang, and Mingming Gong. Towards accurate time series forecasting via implicit decoding. Advances in Neural Information Processing Systems, 2025.
Xingjian Wu, Xiangfei Qiu, Zhengyu Li, Yihang Wang, Jilin Hu, Chenjuan Guo, Hui Xiong, and Bin Yang. CATCH: Channel-aware multivariate time series anomaly detection via frequency patching. In ICLR, 2025a.
Xingjian Wu, Xiangfei Qiu, Hongfan Gao, Jilin Hu, Bin Yang. and Chenjuan Guo. VAV:E: koopman-kalman enhanced variational autoencoder for probabilistic time series forecasting. In ICML, 2025b.
Xingjian Wu, Xiangfei Qiu, Hanyin Cheng, Zhengyu Li, Jilin Hu, Chenjuan Guo, and Bin Yang. Enhancing time series forecasting through selective representation spaces: A patch perspective. In NeurlPS, 2025c.
Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. TFB: Towards comprehensive and fair benchmarking of time series forecasting methods. In Proc. VLDB Endow, pages 2363-2377, 2024.
Xiangfei Qiu, Xingjian Wu, Hanyin Cheng, Xvyuan Liu, Chenjuan Guo, Jilin Hu, and Bin Yang. DBLoss: Decomposition- loss function for time series forecasting. In NeurIPS, 2025a.
Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo, Jilin Hu, and Bin Yang. DUET: Dual clustering enhanced multivariate time series forecasting. In SIGKDD, pages 1185-1196 2025b.
Xiangfei Qiu, Zhe Li, Wanghui Qiu, Shiyan Hu, Lekui Zhou, Xingjian Wu, Zhengyu Li, Chenjuan Guo, Aoying Zhou, Zhenli Sheng, Jilin Hu, Christian S. Jensen, and Bin Yang. Tab: Unified benchmarking of time series anomaly detection methods. In Proc. VLDB Endow. pages 2775-2789, 2025c.
Jiaming Ma, Binwu Wang, Pengkun Wang. Zhengyang Zhou, Xu Wang, and Yang Wang. Bist: A lightweight and efficient bi-directional model for spatiotemporal prediction. Proceedings of the VLDB Endowment, 1866):1663-1666 2025a.
12

---

Jiaming Ma, Zhiqing Cui, Binwu Wang, Pengkun Wang. Zhengyang Zhou, Zhe Zhao, and Yang Wang. Causal learning meet covariates: Empowering lightweight and effective nationwide air quality forecasting. In International Joint Conference on Artificial Intelligence, 2025b.
Jiaming Ma, Binwu Wang, Guanjun Wang, Kuo Yang. Zhengyang Zhou, Pengkun Wang, Xu Wang, and Yang Wang. Less but more: Linear adaptive graph learning empowering spatiotemporal forecasting. In Advances in Neural Information Processing Systems, 2025c.
Jiaming Ma, Binwu Wang, Pengkun Wang, Zhengyang Zhou, Yudong Zhang, Xu Wang. and Yang Wang. Mobimixer: A multi-scale spatiotemporal mixing model for mobile traffic prediction. IEEE Transactions on Mobile Computing, 2025d.
Jiaming Ma. Binwu Wang. Qihe Huang, Guanjun Wang. Pengkun Wang, Zhengyang Zhou, and Yang Wang. Mofo: Empowering long-ter term time series forecasting with periodic pattern modeling. In Advances in Neural Information Processing Systems, 2025e.
Jiaming Ma, Binwu Wang, Pengkun Wang. Zhengyang Zhou, Xu Wang, and Yang Wang. Robust spatio-temporal centralized interaction for ood learning. In Forty-second International Conference on Machine Learning, 2025f.
Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. Crossgnn: Confronting noisy multivariate time series via cross interaction refinement. Advances in Neural Information Processing Systems, 36:46885-46902 2023b.
Qihe Huang, Lei Shen, Ruixin Zhang, Jiahuan Cheng, Shouhong Ding, Zhengyang Zhou, and Yang Wang. Hdmixer: Hierarchical dependency with extendable patch for multivariate time series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 12608-12616, 2024.
Qihe Huang, Zhengyang Zhou, Yangze Li, Kuo Yang, Binwu Wang, and Yang Wang. Many minds, one goal: Time series forecasting via sub-task specialization and inter-agent cooperation. In Advances in Neural Information Processing Systems, 2025a.
Qihe Huang, Zhengyang Zhou, Kuo Yang, Zhongchao Yi, Xu Wang, and Yang Wang. Timebase: The power of minimalism in efficient long-term time series forecasting. In Forty-second International Conference Machine Learning, 2025b.
Wenxuan Wang, Kai Wu, Yujian Betterest Li, Dan Wang, and Xiaoyu Zhang. Synthetic series-symbol data generation for time series foundation models. Advances in Neural Information Processing Systems, 2025g. URL /ttttps://aaxx..orrtte org/ 2510. 08445
Sai Surya Duvvuri and Inderjit S Dhillon. Laser: Attention with exponential transformation. arXiv e-prints, pages arXiv 2411, 2024.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, NeurIPS, 0-22411-224400 2021.
Minhao Liu, Ailing Zeng. Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. NeurIPS, 35: 5816-5828, 2022.
Yuzhou Chen, Ignacio
Baris Coskunuzer, and Yulia Gel. Tamp-s2genets: cou- pling time-aware multipersistence knowledge representation with spatio-supra graph convolutional networks for Line-seriss forecasting. In ICLR, 2022.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, hegh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Ian T Jolliffe. Principal component analysis for special types of data. Springer, 2002.
13

---

Qihe Huang, Zhengyang Zhou, Kuo Yang, Zhongchao Yi, Xu Wang, and Yang Wang. Timebase: The power minimalism in efficient long-term time series forecasting. In ICML, 2025c.
Wenzhen Yue, Xianghua Ying, Ruohao Guo, DongDong Chen, Ji Shi, Bowei Xing, Yuqing Zhu, and Taiyan Chen. Sub-adjacent transformer: Improving time series anomaly detection with reconstruction error from sub-adjacent neighborhoods. arXiv preprint arXiv:2404.1894 2024.
Hui Chen, Viet Luong, Lopamudra Mukherjee, and Vikas Singh. SimpleTM: A simple baseline for multivariate time series forecasting. In ICLR, 2025.
Shengsheng Lin, Haojun Chen, Haijie Wu, Chunyun Qiu, and Weiwei Lin. Temporal query network for efficient multivariate time series forecasting, In JCML, 2025.
Xiaowen Ma, Zhenliang Ni, Shuai Xiao, and Xinghao Chen. Timepro: Efficient multivariate long- term time series forecasting with variable- and time-aware hyper-state. In ICMI., 2025g.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. NIPS, 13:1 2020.
Abhimanyu Das, Weihao Kong, Andrew Leach, Rajat Sen, and Rose Yu. Long-term forecasting with tide: Time-series dense encoder. arXiv preprini arXiv:2304.0844 2044004444 2023.
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In ICLR, 2023.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. ICMI., pages 27268-27286 PMLR, 2022.
Nate Gruver, Mare Finzi, Shikai Qiu, and Andrew G Wilson. Large language models zero-shot time series forecasters. Advances in Neural Information Processing Systems, 36:19622-19635 2023.
Xiaoming Shi, Shiyu Wang, Yuqi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. Time-moe: Billion-scale time series foundation models with mixture of experts. In ICLR, 2025.
Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In ICML, 2024.
Yong Liu, Guo Qin, Zhiyuan Shi, Zhi Chen, Caiyin Yang, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Sundial: A family of highly capable time series foundation models. arXiv preprint arXiv:2502.008/1 2025.
Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. Moment: A family of open time-series foundation models. In ICML, 2024.
Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, and Vikas Singh. Nyströnformer: nyström-based algorithm for approximating self-attention. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 14138-14148. 2021.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our claims in the abstract and introduction clearly state contributions and accurately match the theoretical and experimental results in Sections 3 and 5 respectively.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are discussed in Appendix L
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
Theorem iis formally stated in Section 3 with its complete proof provided in Appendix [A] Additionally, the derivations of the Jacobian matrices of the NormLin layer are detailed in Appendix B
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The implementation details are presented in Appendix D The source code and datasets are available at https: com http://hwb.cco//jaaaa//e/19////0 190te///1/9900ii00rrr
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: Data and code are available at this anonymous repository: https://githu: com/ jackyue www.yyeee94/0.immm 1994/0Linear
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurIPS code and data submission guidelines (https
//ipsscccppbllt... wwtppbblcggguiiieecceeeeeeooeeet http:/uidess/eee/e https//CoSSubbiiisce seiooniiisiooPoooiyy.. for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Experimental settings and implementation details are provided in Appendices and D The source code is publicly available.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Error bars are reported in Appendices F and
H.2 Specifically, Table 13
presents standard deviations over seven random seeds, and Table 14 shows 99% confidence intervals of averaged performance across four prediction lengths.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The compute resources are presented in Appendix
K
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: pecttps:/neurriie hww./puur..../. cccpuullc//thiccccuieelllll
Answer: [Yes]
Justification:
This work strictly conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This is a technical work on time series forecasting without direct societal impact, hence we select [NA]. Broader technical implications are discussed in Appendix
Guidelines:
The answer NA means that there is no societal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The datasets and models used in this work are standard in time series forecast- ing and pose no known safety or misuse risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We use publicly available datasets and open-source code, all of which are properly cited in the main text. All terms of use have been respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
19

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswipphaeeoooccaaappp thcode. com/ com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The released code repository includes all necessary source code to reproduce the experimental results and well documented with usage instructions. All assets are anonymized for review.
Justification:
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details of the dataset/codeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs are not used as an important, original, or non-standard component of methods.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurd) auttep://eearrpp..ccc//oofffeeer //tp://///oooeeeeee/////////// for what should or should not be described.
21

---

A Proof of Theorem 1
Ux My
Vix
Dxxy
Proof. For clarity, we denote
R Mr -
RR' and Σ
Σxg
According to the definition, the probability density function of i.e, the joint density of x and y, 18
p(z) p(x. y) (2))(411///111///22 exp (-2 :a) H4)
(6)
where |ع] denotes the determinant of Σ. the following, we ignore the constant coefficient and focus on the exponential term.
Using the standard block matrix inverse formula [Horn and Johnson
2012),
have
E- E + d Exy FT T
xy xy -
xx Ex
-d
xx
A
-b
d
d
b
d-1
where the scalar oy 2 >x R(xl denote d 1x Exy. the matrix becomes Σx + d bbb..
the Schur complement (Horn and Johnson 2012). Letting
For notational simplicity, we define a ف x Px E [ and B a y - My R. Then, the exponent in Equation 6 (ignoring the constant factor 1/2) becomes
R-1 x
+ d. bbT
-b
(z uz) TI uz) = JaT
B]
bb-
d-1
3
a TIxx a d. a =b 28a T + +
(8)
a Tx atdd (3² 2dd-2.100 a B a
a =a'xaaad (3 da b))
Since p(x) x exp Txx a). it follows that
p x) p(x, y)
p(x)
T - 2x
d (8 Tb)²
-a xx - x)=aaa) a))
exp
+
2
(3 da
dabb) b)²
exp
2d
(y By
da TTTb"
exp
2d
Therefore, the conditional density of y give x is also Gaussian distribution, whose mean is
By + da Tb = My + db
= My + d dt T xx (x pz) xy
(10)
My IT D (x [xx)
and the variance is dl = 03
TT 5-
This completes the proof.
xx -x Exy
22

---

B Jacobian matrix comparison of self-attention and NormLin
In this section, analyze the gradients of the non-linear transformations in the self-attention mechanism and the NormLin module, focusing on the attention/weight rows, denoted as E IR
B.1 Jacobian matrix of Softmax self-attention
Ler A Softmax(a) RR We first compute the partial derivatives element-wise, and then rewrite the results in matrix form. Based on the definition of Softmax, the i-th element of с is The partial derivative of c; with respect to a, is:
ΣN
dc;
aa Σ
ca e.
= C2 - c.
(11)
da,
Ek=1
Eat
(Ei= Eau
For j # i, the partial derivative of c; with respect to is:
CjCj
(12)
0aj (Ek=1 eak
Combining Equations and 12 we obtain:
dc: ci(61s. 82...... dNN] C.c.
(13)
da
where 8j. = if = i, otherwise 0. Therefore, the Jacobian matrix Ba
can be written as
Oc Diag(c) cc Òa
(14)
where Diag(c) the diagonal matrix with с its diagonal. We observe that the Jacobian matrix a function of c. Since the Softmax function could cause sharp value concentration, the Jacobian matrix may exhibit sparsity, with most entries being close zero [Surya Duvvuri and Dhillon 2024
B.2 Jacobian matrix of the NormLin layer
Let c NormL1 (Softplus (a)) RVV, where NormLi(-) denotes the LI normalization. For de Since all entries of b are positive, notational simplicity, let b Softplus(a). We first analyze ab we have:
b
C;
(15)
b
Therefore, follows that
Oc;
b;
(|bbl,
b.).
(16)
bb,
Σx-1 by
N
2m b
|bb
des is: For o i, the partial derivative
ōb,
Oc;
bi
1
b
(17)
Ckml
bz)²
|
1
Combining Equations
16 and
17 we obtain the Jacobian matrix:
dc
([bb) I I- b1
(18)
Øb
|b]]
23

---

Weather
Solar-Energy
ECL
PEMS03
Self Self-Attention
0.03
0.02
NormLin
0.01
Figure 6: Comparison of Jacobian matrices of Softmax (+) in self-attention and NormL1 (Softplus() (-)) the NormLin layer. The absolute values the matrix entries are visualized. NormLin and self-attention exhibit similar patterns; however, NormLin generally yields diagonal entries of higher magnitude, suggesting more effective gradient propagation. The same input vector is used for both ensure fair comparison.
where 1 E R the all-ones vector.
For the derivative of b, with respect to aj with 1 S i,j N, we have
bb;
0
dij
dij
In eah)
oaj
0a,
0a,
(19)
dij 1 ca.
dij Sigmoid(a.)
Therefore, 8a
0b can be written as
db da
Diag (Sigmoid(a)) Diag (6)
(20)
where Sigmoid(-) operates element-wise and b - Sigmoid(a) is defined for clarity. Using the chain rule, the Jacobian matrix with respect can be derived as follows:
Oc de
1
(|bbl1 I --bi- Diag 6b)
Oa
Ub
da
|b|l
1
[|b]], Diag 6)
-bbb]
(21)
1
||b]] (Diag 66) bb
where we use the fact that DDag (6) GT. and |[ is the normalized b.
Equations 21 and (4) share similar structures, with the former introducing an additional learnable scaling factor [b]] which offers greater flexibility. The entries of b Sigmoid(a) in Equation (2T)
are generally larger than those
Soft max(() in Equation [4]. particularly when a contains
small values near zero. This property may contribute to stronger gradients in NormLin. Figure 6
illustrates the Jacobian matrices the self-attention mechanism and the Normlin layer under same input highlighting that NormLin tends to produce stronger gradient values.

---

C Dataset description
In this work, the following real-world datasets are used for performance evaluation. The dataset details are presented in Table
ETT datasets Zhou et al. 2021 record seven channels related to electricity transformers from July 2016 July 2018. It contains four datasets: ETTh1 and ETTh2, with hourly recordings, and ETTml and ETTm2, with 15-minute recordings.
Weather (Wu et al., 2021] contains 21 meteorological variables (e.g., air temperature, humidity) recorded every 10 minutes in 2020.
ECL [Wu et al. 2021] records hourly electricity consumption of 321 consumers from July
2016 July 2019.
Traffic [Wu ct al.
202 includes hourly road occupancy rates from 862 sensors in the Bay Area from January 2015 to December 2016.
Exchange (Wu et al.
2021 collects daily exchange rates for eight countries from January
1990 to October 2010
Solar-Energy Lai et al.
2018 records the solar power output every 10 minutes from 137
photovoltaic plants in 2006.
PEMS [Luu et al. 2022 provides public traffic sensor data from California, collected every 5 minutes. We use its four subsets (PEMS03, PEMS04, PEMS07, PEMS08) in this study.
ILII contains weekly records of influenza-like illness patient counts provided by the U.S.
CDC from 2002 to 2021.
COVID-19 [Chen al. 2022 includes daily records of COVID-19 hospitalizations in California in 2020, provided by Johns Hopkins University.
METR-LA 3 collects traffic network data in Los Angeles every 5 minutes from March to June 2012. A total of 207 channels are included.
NASDA includes daily NASDAQ index and key economic indicators (e.g., interest rate and gold price) from 2010 to 2024.
daily page view counts for Wikipedia articles over two years (2018-2019). The first 99 channels are used in this study.
SP500 records daily SP500 index data (c.g., opening price, closing price, and trading volume) from January 1993 to February 2025.
Dow collects daily stock prices of 27 Dow Jones Industrial Average (DJIA) component companies from January 1999 to March 2025.
CarSales collects daily sales of 10 vehicle brands (e.g., Toyota, Honda) in the U.S. from January 2005 to June 2023. The data are compiled from the Vehicles Sales dataset on Kaggle.
Power contains daily wind and solar energy production (in MW) records for the French grid from April 2020 to June 2023, The data are compiled from the Wind Solar Daily Power Production dataset On Kaggle.
Website PoosstteeContinn contains six years of daily visit data (c.g., first-time and returning visits) to an academic website, spanning from September 2014 to August 2020.
Unemp contains monthly unemployment figures for 50 U.S. states and three other territories from January 1976 to March 2020, sourced from the official website of the U.S. Bureau of Labor Statistic
https: http://wwww..a...acaacc//aaaaaaa
https://www.. bls http://wwwwbb..ggg//e//////uuupp
25

---

Table 11: Detailed dataset descriptions and statistics. Dim denotes the number of variates for each dataset. Frequency refers the time interval between consecutive steps. Split indicates the (Train, Validation, Test) ratio. Prediction len. represents the prediction lengths. For long-term forecasting, the input length fixed 96. For short-term forecasting, we adopt two settings: S1 (Input-12. Predict-(3, 6, 9, 12}) and S2 (Input-36, Predict-(244 36, 48, 60)). In total, 140 prediction tasks across various datasets and prediction length settings are evaluated in this work.
Dataset
Dim
Frequency
Total len.
Split
Prediction len.
Information
ETThI, ETTh2
7
Hourly
17420
6:2:2
(96,192,336,720]
Electricity
ETTml, ETTm2
7
15 mins
69680
6:2:2
(96,192,336,720]
Electricity
Weather
21
10 mins
52696
7:1:2
19,,1923336,720)
Weather
ECL
321
Hourly
26304
7:1:2
[96.192,336.720]
Electricity
Traffic
862
Hourly
17544
7:1:2
[96,192,336,720]
Transportation
Exchange
Daily
7588
7:1:2
(96,192,336.720|
Economy
Solar-Energy
137
10 mins
52560
7:1:2
[96,192,336,720)
Energy
PEMS03
358
5 mins
26209
6:2:2
(12,24,48,96)
Transportation
PEMS04
307
5 mins
16992
6:2:2
(12,24,48,96)
Transportation
PEMS07
883
mins
28224
6:2:2
(12.24,48,96)
Transportation
PEMS08
170
5 mins
17856
6:2:2
(12,24,48,96)
Transportation
(3,6,9,12)
ILI
Weekly
966
7:1:2
Health
(24.36,48.60)
7:1:2
(3,6,9122
COVID-19
55
Daily
335
Health
6:2:2
(24,36,48,60)
(3.6,9.12)
METR-LA
207
5 mins
34272
7:1:2
Transportation
(24,36,48,60)
[3,6,9,12)
NASDAQ
Daily
3914
7:1:2
Finance
(24,36,48,60)
(3,6,9,12)
Wiki
99
Daily
730
7:1:2
Web
(24,36,48,60)
(3,6.9.12)
SP500
5
Daily
8077
7:1:2
Finance
(24,36,48,60)
(3,6,912)
DowJones
27
Daily
6577
7:1:2
Finance
(24.36,48,60)
(3,6,9,12)
CarSales
10
Daily
6728
7:1:2
Market
(24.36,48,60)
13,6.9,12)
Power
2
Daily
1186
7:1:2
Energy
(24,36,48,60)
(3.6.9,12)
Website
4
Daily
2167
7:1:2
Web
(24,36,48,60)
(3,6,911)
Unemp
53
Monthly
531
6:2:2
Society
(24,36,48,60)
26

---

D Implementation details
OLinear is optimized using the ADAM optimizer [Kingma and Ba 2015), with the initial learning rate selected from {10-,,2 x 10-4,5 x 10 10-4) The model dimension is chosen from {128, 256, 512]. while the embedding size dl is set to 16. The batch size is selected from {4, 8, 16, 16,32) 32) depending on the dataset scale. The block number is chosen from (1,2,3) Training is performed for up to 50 epochs with early stopping, which halts training if the validation performance does not improve for 10 consecutive epochs. We adopt the weighted loss function following CARD [Wang et al. 2024b]. The experiments are implemented in PyTorch (Paszke al. [019] and conducted NVIDIA GPU with 24 GB of memory. Hyperparameter sensitivity is discussed in Appendix [8] For baseline models, we use the reported values from the original papers when available; otherwise, we produce the results using the official code. For the model FilterNet (Yi et al. 2024a], TexFilter is adopted in this work. The code and datasets are available at the following anonymous repository: https://gi /htpsbu thub com//aactyuu1994//0heea
E
Showcases
E.1
Time domain and transformed domain
Figure iillsstatts the temporal domain and its corresponding transformed domain. the transformed domain, the correlations along the sequence are effectively suppressed. Adjacent time series exhibit strong consistency in this new domain, which desirable for forecasting tasks. The orthogonal transformation corresponds to projecting the time series onto the eigenvectors of the temporal Pearson correlation matrix CorrMat,. The eigenvalues
this symmetric positive semi-definite matrix CorrMat, typically decay rapidly [Jolliffe] 2002), with only few being dominant. Consequently, the transformed series exhibits sparsity, with most of the energy concentrated in just a few dimensions. Moreover, since noise tends to be evenly distributed across the transformed dimensions, the signal-to- noise ratio (SNR) in the leading components is improved [Bishop and Nasrabadi 2006].
E.2 Low-rank attention matrix and high-rank NormLin weight matrix
We replace the NormLin module in OLinear with standard self-attention mechanism and observe the typical low-rank property in the resulting attention matrices, as shown in Figure 9 This phenomenon be attributed the sparsity induced by the transformed domain and the sharp focus introduced by the Softmax operation. The low-rank attention matrix could limit the expressive capacity of the model. In contrast, the weight matrices in NormLin exhibit higher rank, better preserving representation diversity.
Furthermore, the learned NormLin weights closely resemble the across- variss-variate Pearson correlation matrix CorrMat,, suggesting that the NormLin layer effectively captures multivariate correlations. By directly replacing the learnable weight matrix with Softmax( Softmax(CorrMatt)) we obtain OLinear-C, which also achieves competitive performance, as demonstrated in Appendix H
Reason analysis A potential reason behind this phenomenon is that Ortho'Trans can mitigate the intrinsic low-rank property of time series data. The empirical results are presented below. Following TimeBase (Huang et al., 2025cl, we compute the median singular values of the correlation matrix among non-overlapping time series patches (patch length = 24, number of patches 30). As shown in Table 12, Ortho yields larger singular values than other transformation bases (with "Identity" denoting no transformation). In other words, OrthoTrans alleviates low-rank tendencies of time series data and enhances data diversity.
E.3 OrthoTrans enhances attention matrix rank
Figures TO 12 visualize the attention matrices of OLinear (with the NormLin module replaced by self-attention) under different transformation bases. Similar results for ITransformer and PatchTST are presented Figures [3] [4] respectively. As shown, OrthoTrans typically yields higher-rank attention matrices compared to DFT, wavelet transforms, transformation. As stated earlier,
27

---

Time Domaii
Transformned Domatn
Cor:Mate
Q Mal
Sevnettel
150
23
0.6
120
Weathe
0.4
Secodiaar
156
200
20
Solar
-02
Cariiles
Per
200
Soundila
Figure 7: Visualization of temporal series, transformed series, the temporal Pearson correlation matrix, and its corresponding orthogonal matrix. The series are normalized prior to transformation, and the absolute values in the transformed domain are shown for clarity. The X-axis in the first and second columns represents the temporal and feature dimensions, respectively. Adjacent series exhibit consistency in the transformed domain, with sparsity clearly observable. Temporal Pearson correlation matrices on more datasets are presented in Figure g
higher-rank attention matrices better preserve the representation space and can potentially enhance the model's expressive capacity (Han al.) 2023). This may explain why integrating OrthoTrans as a plug-in consistently improves the performance of Traasforrer-based forecasters (as shown in 55).
ECL
Taffic
Weather
1.0
0.8
Solan-Energy
PEM503
ETT+1
0.6
0.4
0.2
ILI
Power
Website
0.0
-0.2
Figure 8: Temporal Pearson correlation matrices various datasets. Periodic patterns are clearly reflected in the correlation matrices. We incorporate such a priori statistical information into the model design, leading to improved forecasting performance.
28

---

Table 12: TranoTrans can effectively mitigate the low-rank characteristics of time series data. The median singular values of the correlation matrices among temporal patches are reported. The orthogonally transformed domain exhibits better data diversity.
Dataset
Ortho Trans
DFT
Wavelet
Identity
ECL
0.446
0.332
0.381
0.162
Traffic
0.568
0.177
0.049
0.002
Weather
0.572
0.352
0.292
0.012
Solar
0.481
0.032
0.023
0.007
PEMS03
0.450
0.356
0.395
0.241
Table 13: Robustness of OLinear performance. Standard deviations calculated over random seeds. SI and S2 correspond to Input 12, Predict {3, 6, 9, 12] and Input 36, Predict {24, 36, 48, 60), respectively.
Dataset
ECL
Traftic
ETTml
Solar- Solar-Energy
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.131#4e-4
0.221#4e-4 0.39843e-3
0.226*2e-4
0.30247c-4
0.33443e-4
0.179#6c-4
0.19145e-4
Horizon
192
0.150#1e-3
0.238#1e-3
0.439#3e-3
0.24114e-4
0.357=9e-4
0.363+5e-4
0.209+8e-4
0.213+2e-4
336
0.165#1e-3
0.254=1e-3
0.464+4e-3
0.250#3e-4
0.387+2e-3
0.38545e-4
0.231+8e-4
0.229*7e-5
720
0.191#2e-3
0.279=2e-3
0.502#4e-3
0.270=4e-4
0.452#1e-3
0.426#6e-4
0,241#1e-3
0.23614e-4
Dataset
Weather
PEMS03
NASDAQ (S2)
Wiki (SI)
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
H1
0.153#1e-3
0.190=le-3
0.060#3e-4
0.159=3e-4
0.121#1e-3
0.21629c-4
6.161#1e-2
0.368+7e-4
Horizon
0,20042e-3
0.235a2e-3
0.078#6e-4
0.179a5e-4
0.163#7e-4
0.26119e-4
6.453s9e-3
0.3
H3
0.25843e-3
0.280#2e-3
0.10446e-4
0.21016e-4
0.20542e-3
0.296#2e-3
6.66626e-3
0.398+le-3
H4
0.33744e-3
0.333#2e-3
0. 0.140:2e-3
0.247#le-3
0.25942e-3
0.336*2e-3
6.83424e-3
0,40644e-4
Dataset
Dowlones (S2)
SP500 (S2)
CarSales (S1)
Power (S1)
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
H1
7.43243e-2 0.664#9c-4
0.155+2e-3
0.271#2e-3
0.303+2e-3
0.277+le-3
0.86447e-3
0.688t4e-3
Horizon
H2
10.848=7e-2
0.20942e-3 0.799=1e-3
0.317#26-3
0,31542e-3
0.28542e-3
0.99147c-3
0.74243e-3
H3
14.045=1e-1
0,914=1e:3
0,258+2e-3
0,358#le-3
0.32748e-4
0.293*8e-4
1.062+1e-2
0.770#4e-3
H4
16.95918e-2
1.017=3e-3
0.305#3e-3
0.38742e-3
0.336=5e-4
0.30114e-4
1.11942e-2
0.78916e-3
Robustness under various random seeds
Table 13
presents the standard deviations for different datasets and prediction lengths using seven random seeds. OLinear demonstrates strong robustness across independent runs. Furthermore, as shown in Table [4] our model exhibits better robustness than state-of-the-ar Transformer-basee models, TimeMixer++ and iTTransfommer, measured by the 99% confidence intervals.
Note that averaging over M prediction lengths reduces the standard deviation by factor of VM making the results more robust the choice random seeds. Therefore, we prefer report the average results in this work to mitigate the influence of randomness.
We perform significance with 7 random seeds using Student's iTransformer across multiple datasets and prediction horizons. As shown in Table [5] OLinear consistently outperforms iTransformer, with the improvements being statistically sigmificant the 0.05 level (p < 0.05).
compare OLinear and
Full results G
Table 16 lists the simplified tables from the main text and their full versions in the appendix.
29

---

Attn. Mat.
NormLin Weight
Softnax(CorrMatt)
CorMaay
Rank:
Rank:
Rank: 15
Weather
1.0
Rank:
Rank: 131
Rank: 137
Rank: 137
0.8
Solar- Energy
0.6
Rank: 196
Rank: 310
Rank: 321
Ronk: 321
0.4
ECL
0.2
Rank: 224
Rank: 344
Rank:
Rank: 342
PEMS03
Figure Illustration of attention/weight matrices from self-attention, NormLin, and the across- variate Pearson correlation matrices (with and without Softmax). When applied in OLinear, the self-attention mechanism results in low-rank attention matrices, whereas NormLin produces high- rank weight matrices that better preserve representation diversity. Moreover, the learned NormLin weights resemble the across-variate correlation matrices, suggesting effective modeling multivariate relationships.
H
OLinear-C
H.1
Forecasting performance
T5ble255compares compares OL and OLinear-C on both short- and long-term forecasting tasks. OLinear-C achieves performance comparable to OLinear while using fewer learnable parameters and offering improved efficiency (see Table However, due to its fixed weight matrix, NormLin, lacks the flexibility required serve plug-and-play module for other forecasters. Therefore, we focus on OLinear in this work.
H.2 Robustness
Table 26 reports the standard deviation of OLinear-C across seven random seeds, demonstrating its robustness independent runs. For a broader robustness comparison, Table presents 99% confidence intervals for OLinear, OLinear-C, TimeMixer++, and iTranfformer.
30

---

Identity
Ortho
Fourier Basis
Wavelet Basis
Rank: 14
Runk: 16
Rank: 1
Rank: 2
Weather
Rank: 23
Rank: 43
Rank: 44
Rank: 18
Solar Enneggy
Rank: 190
Rank: 196
Rank: 217
Rank: 178
ECL
Rank:
Rank: 238
Rank: 201
Rank: 121
PEMS03
Figure 10: Comparison of attention matrices under various transformation bases for OLinear with the NormLin module replaced by self-attention (Example 1). For a fair comparison, the same input series is used for different bases on each dataset. 'Identity' denotes no transformation, and the Haar wavelet is used the wavelet basis. In general, TrthoTrans yields higher-rank attention matrices and thus better preserves the representation space.
H.3
Ablation studies
We conduct ablation studies different transformations of the Pearson correlation matrix CorrMat.. including Softplus, Sigmoid, ReLU, and the identity function (i.c., no transformation). All variants followed by row-wise L1 normalization. As shown in Table 27 Softmax Softmax(CorrMat,) consistently outperforms the other options and is adopted in the main experiments.
I More experiments
I.1 Quantitation of temporal decorrelation of OrthoTrans
We evaluate the decorrelation effectiveness of OrthoTrans, DFT, and Haar wavelet transforms using window size 96. The metric is the off-diagonal Frobenius norm temporal correlation matrix, normalized by the number of off-diagonal elements. As shown in Table 28 Ortho'Trans reduces the correlation in the original series by 96%, and achieves relative improvements of 84% and 93% over DFT and Haar wavelet transforms, respectively.
31

---

Identity
Ortho
Fourier Basis
Wavelet Basis
Rank: 13
Runk: 14
Rank: 2
Rank: 1
Weather
Rank: 24
Rank: 74
Rank: 49
Rank: 35
Solar Eneegyy
Rank: 175
Rank: 276
Rank: 193
Rank: 200
ECL
Rank: 183
Rank: 244
Rank: 180
Rank: 225
PEMS03
Figure 11: Comparison of attention matrices under various transformation bases for OLinear with the NormLin module replaced by self-attention (Example 2). For a fair comparison, the same input series is used across bases on each dataset. 'Identity' denotes no transformation, and the Haar wavelet used the wavelet basis. In general, OrthoTrans yields higher-rank attention matrices and thus better preserves the representation space.
I.2 Performance under varying lookback horizons
We evaluate the forecasting performance of OLinear and its variant, OLinear-C, under varying lookback horizons. Instead of using a fixed lookback length, search for the optimal horizon within the range of 96 to 720. As shown in Table 29] both models consistently achieve wate-of-the--a art results across different forecasting lengths and datasets, demonstrating their robustness and strong performance.
I.3
More baselines
We further compare OLinear with several lightweight baselines, including SimpleTM [Chen et
2025]. TQNet [Lin et al.) 2025), TimePro [Ma et al., 2025g), and TimeBase [Huang al.) [025c]] The lookback length uniformly set 96 for all methods. shown Table 30 OLinear achieves the lowest MAE across all datasets, outperforming SimpleTM, TQNet, TimePro, and TimeBase by
5%, 5%, 7%, and 16% on average, respectively. These results demonstrate that OL serves as strong and efficient new bascline for lightweight time series forecasting.
32

---

Identity
Ortho
Fourier Basis
Wavelet Basis
Rank:
Runk: 17
Rank: 3
Rank: 2
Weather
Rank: 20
Rank: 21
Rank: 25
Rank: 48
Solar Eneeggy
Rank: 202
Rank: 264
Rank: 226
Rank: 219
ECL
Rank: 201
Rank: 224
Rank: 84
Rank: 178
PEMS03
Figure 12: Comparison of attention matrices under various transformation bases for OLinear with the NormLin module replaced by self-attention (Example 3). For a fair comparison, the same input series is used across bases on each dataset. 'Identity' denotes no transformation, and the Haar wavelet used the wavelet basis. In general, OrthoTrans yields higher-rank attention matrices and thus better preserves the representation space.
L.4 Few-shot and zero-shot forecasting
To evaluate OLinear's adaptability to sparse data and transferability to unseen datasets, we conduct few-shot and zero-shot learning experiments, with the results shown in Tables [3] and 32 respectively. As a Linear-based model, OLinear exhibits surprisingly strong generalization capabilities, characteristic more commonly associated with Transformer based models [Brown al. 2020). Notably, OLincar achieves this with high computational efficiency (see Table 41). Furthermore, OLinear delivers performance par with superior to TixeMixer+ [Wang al.) 2025a], despite the latter's s architectural complexity, which includes multi-scale design, series imaging, and dual-axis attention mechanisms.
This performance advantage may be attributed to the high-rank property of the NormLin weight matrix (see Appendix E.2) and its enhanced gradient flow (see Appendix BB. In contrast, attention matrices derived from the classic query-key mechanism typically exhibit Tow-rank characteristics (see Figures 10 [4), lcading to compressed representation space (Han et al. 2023
33

---

Identity Rank: 9
Trans
Fourier Basis
Wavelet Basis
Rank: 12
Rank: 12
Rank: 9
Weather
RRnk: 18
Rank: 2.3
Rank: 9
Rank: 11
Solar Eneegyy
Rank: 183
Rank: 203
Rank: 175
Rank: 186
ECL
Rank: 97
Rank: 148
Rank: 111
Rank: 148
PEMS03
Figure 13: Comparison of attention matrices under various transformation bases for iTransformer. For fair comparison, the same input series is used across bases on each dataset. 'Identity' denotes no transformation, and the Haar wavelet is used as the wavelet basis. In general, OrthoTrans yields higher-rank attention matrices and thus better preserves the representation space.
1.5 Increasing lookback lengths
Performance under increasing lookback lengths reflects model's ability effectively utilize histori- cal information [Liu et al.] 2024a). Linear- based forecasters have strong potential to benefit from longer lookback windows [Zng et 2023). As shown Figure [5] OLinear demonstrates consis- tent improvements as the lookback horizon increases from 48 to 720, and consistently outperforms state-of-the-arr forecasters.
I.6 Training with less data
Training with less data reflects model's adaptability and learning efficacy. As shown in Figure I6 OLinear adapts well to decreasing training set sizes, from 100% down to 5%. Our model consistently outperforms others across all training ratios, highlighting its robustness to data-sparse conditions.
L.7 OLinear versus large time series models
We further compare OLinear with the large pre-trained time series models the full-shot, few- shot and zero-shot scenarios. As shown in Table 33 OLinear surprisingly achieves comparable
34

---

Identity Rank: 5
Ortho Trans
Fourier Basis
Wavelet Basis
Rank: 6
Rank: 5
Rank: 2
Weather
Rank: 5
Rank: 9
Rank: 9
Rank: 9
Solar Eneegyyeeyyy
Rank: 10
Rank: 9
Rank: 9
Rank: 10
ECL
Rank: 3
Rank: 11
Rank: 9
Rank: 10
PEMS03
Figure 14: Comparison of attention matrices under various transformation bases for PatchTST. For fair comparison, the same input series used across all bases for each dataset. 'Identity' denotes no transformation, and the Haar wavelet is used as the wavelet basis. In general, OrthoTrans yields higher-rank attention matrices and thus better preserves the representation space.
performance to Timer [iu et al. 2024b), despite not being pre-trained on large-scale datasets. When the full training set used, OLinear outperforms the fine-tuned Timer on the ECL, Traffic, Weather, and PEMS datasets. Even when only 5% of training data are available, OLinear still performs better on the ETTml and ETTm2 datasets. In zero-shot forecasting, our model trained on ETTh1 state-of-thed -fttte-rrttttt large time series models when evaluated on
and ETTml respecively--outperfomm ETTh2 and ETTm2. These results not only demonstrate OLinear's strong generalization capability, but also imply that large time series models are still in the early stages of development.
I.8 Hyperparameter sensitivity
We evaluate the hyperparameter sensitivity of OLinear with respect to four key factors: the learning rate (Ir), the number of blocks (L), the model dimension (D), and the embedding size (d). As shown in Figure [7] OLinear generally maintains stable performance across different choices of Ir, L, and D. although each dataset exhibits its own preferences. Regarding the embedding size d, Table 34 indicates that 16 represents a sweet spot, which we adopt as the default setting in our experiments.
35

---

Table 14: 99% confidence intervals of the average performance across four prediction lengths, computed using three times the standard deviation from seven random seeds. The smallest standard deviations are highlighted in bold. OLinear- C denotes the variant of OLinear where the weight matrix NormLin is set Softmax (CorrMat,). with CorrMat, being the multivariate correlation matrix.
OLincar
OLinear-C
TimeMixer++
iTransfommer
Model
(Ours)
(Ours)
2025a
2024a
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Weather
0.237+0.006
0.260:0.003
0.238+0.005
0.259#0.004 0.226=0.008
0.262:0.007
0.258=0.009 0.278=0.006
0.215+0.001
0.217=0.001
0.215+0.001
0.21743e-4
0.203#0.001
0.238+0.010
0.233=0.009
0.262+0.007
ECL
0.159#0.001 0.248:0.002
0.161#0.006
0.249#0.005
0.165=0.011
0.253:0.001
0.178:0.002
0.270=0.005
Traffic
0.451#0.003
0.247:0.001
0.451#0.006
0.247=0.001
0.416#0.015
0.264*0.013
0.428=0.008
0.282+0.002
ETThl
0.424=0.003
0,424=0.002
0.424+0.002
0.424=0.001
0.419=0.011
0.432+0.015
0.454=0.004
0,447=0.007
ETTh2
0.367+0.002
0,388+0.002
0.368+0.002
0.389=0.001
0.339#0.009
0.380+0.002
0,383+0.004
0.407=0.007
ETTml
0.37440.001
0.377=0.001
0.375#0.001
0.378=0.001
0.369=0.005
0.378+0.007
0.407=0.004 0.410=0.009
ETTm2
0.270=3e-4
0.313+2e-4
0.270=1c-4
0.313+2e-4
0.269=0.002
0.320#0.012
0.288=0.010
0.332+0.003
Table 15: Significance test with Student's t-test and performance comparison between Ol inner and iTransformer across various datasets and prediction horizons.
Dataset & Horizon
OLinear (MAE)
(MAE))
p-value between OLinear and i'Trans.
ECL: 96
0.2
0.240#4e-4
5.12E-11
ECL: 192
0.238+1e-3
0.253+2e-3
3.06E-06
ECL: 336
0.254#1e-3
0.269#1e-3
9.96E-07
ECL: 720
0.27942e-3
0.317=7e-3
5,59B-05
Traffic: 96
0.226+2e-4
0.268#
1.61B-12
Traffic: 192
0.241+4e-4
0.276*
9.48E-10
Traffic: 336
0.25013e-4
0.283#4c-4
1.87E-13
Traffic: 720
0.270+4e-4
0,302+4e-4
6.15E-13
Weather: 96
0.190+1e-3
0.214#3e-4
1.29E-08
Weather: 192
0.235+2e-3
0.254+le-3
1.04E-06
Weather: 336
0.280+2e-3
0.296*1e-3
1.93E-06
Weather: 720
0.333+2e-3
0.349:4e
5.68E-06
I.9 More metrics
We further conduct a comprehensive evaluation of forecasting performance using three scale-free metrics: the Coefficient of Determination (R²), the Pearson Correlation Coefficient (r", and the Mean Absolute Scaled Error (MASE), which are defined as follows:
Y
Y
R²
1
Y
11:
N
Σn
(
-
(Y n:
r
(22)
N
V EE=
(Ynt-Vi
2
VE (Y
N
Yn: - -Y:
MASE
--- Ynne Y
36

---

Table 16: Overview of tables in the main paper and their full versions in the appendix.
Tables in paper
Tables in Appendix
Content
Table
Table
Long-term forecasting
Table
Tables
and
Short-term forecasting
Table
Table
Ablation studies on various bases
Table
Table
OrthoTrans plug-in
Table
Table 25
Performance of OI Linear-C.
Table 6
Table 22
Ablation studies of OLinear
Table
Table 23
NormLin vs self-attention and its variants NormLin as plug-in
Table
Table 24
ECL
Traffic
0.30
0.35
0.30
MAB
0.26
MAB
0.24
0.25
0.22
48
96
192
336
720
48
96
192
336
720
Lookback length
Lookback length
Ours
CARD
iTrans.
TimeMixer
TimesNet
Leddam
Fredformer
PatchTST
DLinear
FreTS
Figure 15: Performance change with increasing lookback lengths. The prediction length is 96.
Here Yn: and Y, n: denote the n-th variate of the ground truth Y and the prediction Y, respectively.
Yn,t refers to the f-th timestep of the n-th variate, and (.) represents the mean.
As shown in Table 35 OLinear consistently achieves state-of-the-art performance these metrics. For example, an average Pearson correlation coefficient of r = 0.903 on the ECL dataset, indicating strong alignment between predictions and ground truth. On the Traffic dataset, OLinear achieves MASE of 0.764, outperforming the second-best model, CARD [Wang et al. 2024b), by a notable margin of 1..4%.
I.10 Less training data to compute Q matrices
To evaluate the amount of training data required for computing the orthogonal transformation matrices Q; and Qo. conduct experiments using only partial data of the training set. As shown in Table 36 OLinear exhibits strong robustness, with minimal performance variation across different data proportions. This property not only enables more efficient preprocessing before training, but also ensures robust performance in few-shot learning scenarios (see Appendix I.4).
More ablation studies
J.1 Ablations NormLin design
In the NormLin layer, the Softplus function followed by L1 row-wise normalization is applied to the learnable weight matrix W to produce attention-like matrices, i.e., RowNNrrm, (Softplus( (W). To assess the effects of different transformations and normalization strategies, we consider transformation
37

---

Table 17: Full results for the long-term forecasting task. The lookback length is set to 96 for all baselines. Avg means the average results across four prediction lengths. This table presents the full version of Table 2
Liscu Based
Turafoomer Buss
TCN Bosed
Model
OLinen
(0am)
PEL J0GA
DLinev 302
TangMisgr 2025
Lobiars IIZA
CARD AUDRA
Filffcues 2034
Tiass 2024
PINLTST
TaresNet 2223h
Mezic
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MEAE
XSE
MAE
MSE
MAE
MSE
MAE
2:310
11
0,445
0.300
0.399
0.348
0.363
0,383
0.363
0.570
0.363
0.500
0.300
0.531
0413
0,413
0.376
0.393
DANC
0.392
0.300
0.395
0.403
0/400
0411
0.426
0.454
(U477
0.448
0.000
0.513
0.414
0.533
0.400
0,423
0.460
D.422
0.459
04453
0.438
0.491
0,499
0.439
0.478
0.450
0.574
0.381
01.455
0,390
0.801
0.450
0/403
0.407
B.30
0.37%
0.997
0.381
01764
0.784
0,395
0/407
0,410
01:307
0,000
0.109
(17:1M
0.175
01704
1.175
0.716
6 187
0.766
0.191
11:07
2L1TII
0.245
8. 1.776
0.757
0.169
11:00
0U1TT
0.204
1.190
0.764
0.175
(.759)
6.
0.767
192
0.223
(1.200
0.317
(1:290
(1.34]
0:901
0.251
0.112
0.764
11:60
1.319
0.241
0.705
0.234
113077
0,745
01301
(1.54)
0.339
01:441
01300
0.349
336
0.291
0.331
0.258
0.340
0,311
0.347
0,313
0.349
0,369
0.427
0.305
0.343
0.303
0.341
0.294
1.550
0.392
0,540
0.311
0,315
0.305
0.345
0.321
0.351
730
0.389
0.387
0,351
0.395
0.414
0,825
0.416
0.806
0.554
0,522
0,373
0.399
0.403
0.758
0,390
11.80
0.397
0.396
0.412
0.407
0.402
0.400
0,878
0.403
Ang
0.710
0.313
0.715
0.323
0.785
0.378
0.291
0.315
3.350
0.401
8.369
0.370
0.281
0.375
0.273
0.312
0.779
0.504
01788
0.313
002881
0.326
0.333
0.375
0.519
0.516
D/667
0:451
0.471
0.461
0.467
0.503
0.424
0.434
0.447
0.443
0.441
0,439
0.442
0.411
0.455
0.452
0.419
0.422
0.431
0429
0:442
11429
0.435
0.425
0.454
0.447
0.40)
0.454
0.458
0.450
96
0.254
0.320
0.259
0.34]
0,290
0.343
0.292
0.340
0.333
0,387
0.276
0.325
0.292
0.345
0.291
0.350
0.295
0.342
0,207
0.349
0.292
0.342
0.340
0.574
192
0.560
I.379
0.312
0.392
[L334
0:395
0.577
0.191
0.477
11476
1.342
0.379
01:67
0.789
01:561
11.81
01371
0.389
(1790)
0:400
01:587
(U40))
0.802
0.414
399
0,409
0.415
0.386
0.414
I1417
0.410
0.416
0/45
0.394
11.41
1.346
0.315
11412
D424
0.411
T1415
0.18:
0.405
I1425
0.412
01425
0L455
0.422
720
0.415
0.431
0/412
0.434
(1440
0.450
0.415
0.437
0.531
01697
0.392
0.415
01419
0448
0,415
11451
0.415
0.434
0.427
0:445
0.431
0.446
0.462
0.468
Jux
0.367
0.388
0.365
0.395
0.363
0.407
0.376
0,358
0.359
0.515
0.339
0.300
0.373
0,350
0.365
0.300
0.365
0.303
0.353
0.437
0.415
0.414
0.427
0.131
0.221
0.153
0.274
0.197
0,282
5:135
0.222
0.235
0.141
0.253
0.147
0.341
0.148
0.340
0.150
0.166
0.255
0.196
0.285
1.147
0.235
0.160
0.250
0.165
0.258
0.162
0.165
0.55
0.185
0,490
0.209
0.301
6.164
0.345
0.173
0.363
0.117
0.17%
0.259
0.192
0.215
0.304
Ang
0.159
0.348
0.187
03
0.773
0.173
0.76
0 133
0.454
0.713
007403
0.753
6 116
0.763
0.168
0.176
0.709
11778
0.770
01708
0.795
0.157
0.795
0J002
0.210)
0.086
0.205
0.991
0:711
0,877
0.306
0.084
0.218
DISSS
0:214
01095
02007
01084
0.210
0.201
01006
0:206
01083
0.305
0.94
Enshonge
192
0.171
0.293
0.193
0.312
0.186
0.305
0.306
0.176
0.315
5.175
0,313
0.175
0
0.77
11.09
0.503
0.177
0.259
0.176
0.259
0.236
0.34
356
0.331
0.414
0.316
0.433
0.350)
0.440
0.342
0.455
0.313
01427
0.316
0.420
0325
0.15
0.333
11.18
0:339
0.008
0.351
0.417
0.301
0.397
0.367
0.4
730
0.837
0.088
0/913
0.712
01596
0.713
0
0.594
03819
015055
0.851
0.689
0831
0/186,
0.851
01001
0:749
0.847
0.691
01901
0.714
0.964
0776
Aug
0.414
3:357
0.909
0.41P
0.908
0.126
0.463
01:335
(.15)
0:264
0.001
0.161
0.500
0.:96
E.392
0.251
0.426
0.378
0.415
0.209
0.426
0.277
1.195
0.258
01.455
0.365
0.903
0.521
192
0.439
0.241
0.415
0.295
0.452
0:301
0.365
0.598
0.500
8.462
0.25%
0.658
0:289
0:441
11.06
0.426
0.200
D4417
0.215
01.400
0.354
0/67
0.396
0.454
0.290
0.456
0.245
8:428
0.363
01.55
0.297
01283
0.45T
0.293
0:43
0.356
0.907
0.270
0.506
0.615
0.394
0.441
0.742
11:00
0.452
0.305
0.467
0,300
0.640
0.39)
Aua
0,451
0.347
0.485
0.295
0.463
0.310
0.615
0,370
0,625
0,383
0.416
0.264
0.467
0.294
0.453
0.282
0.433
0.201
0.425
0.292
0.531
0.343
0.620
0.336
0.153
0.100
0.863
0.209
0.196
0.255
D1555
0.205
0.156
0,302
0.158
0.188
0.163
0.207
0.174
0.214
0.218
0.172
0.20
192
0.206
0.250
(1110)
0:250
0.271
0:231
01296
01201
0.245
0,207
0:250
0.202
01238
0:211
0.251
0:221
0.254
0.359
0.219
0.261
396
0.25
0.290
0.251
0.257
0.265
0.290
0.292
0.307
0:293
0,385
0:237
0.265
0.262
0.291
0.260
[1182
0.257
0.201
0.228
0.296
0.228
0.297
0.280
0.30
770
0.337
0.333
0.319
0.541
01347
6.340
0
0.354
0:315
01381
0.112
0.314
0.541
0.343
0.311
0.945
0.541
0.340
0.384
0.348
0.99
0.237
0.200
0.340
0.292
0.255
0,317
0.226
0.262
0.239
0,365
0.246
0.272
0.258
0.287
96
0.179
0.191
0.189
0.259
0.315
0.242
0.319
0.253
0.290
0.378
6.171
0.231
0.197
0.241
0.197
0.111
0.155
0.233
0.216
0.237
0.234
0.256
0.250
0.09
192
0.309
0.313
0.332
0.381
0.253
0.265
0.367
0.387
0.330
01398
0.318
0.263
0251
0.364
0.234
11554
0.217
0.253
16333
0.261
0.367
0.310
0.256
0318
Solan
720
396
0.211
0.241
0.336
0.129
0.211
0.233
0.291
0.265
0.349
0.153
0.211
0.218
0.411
0.408
0.403
0.395
0.355
0.353
0.415
0.413
01212
6.212
0.269
0.200
0.241
0.250
0281
0268
0.256
0.260
10000
01254
0.266
0.247
0.276
0.264
0.249
0.248
0.213
0.215
0.299
0.290
0.317
0.315
0.319
0.338
0.300
0.397
Ang
0.315
0.317
0.116
0.
0.780
0.315
0:365
0 176
0,764
0.710
0.401
8.383
0.21
0.790
0.764
0.213
0.232
0.116
0.367
0.753
0.763
00200
0.310
6.
0.301
0.19
0.343
0.204
0,073
1.177
0.086
0.179
0.113
0.225
0.201
0.317
01100
0230
01155
0107
0.177
0,091
0:201
0.118
0.511
0.475
D11TH
0.73
0.76
0194
0.30
0.146
11125
0.140
0.347
0.258
0,363
0,344
0.340
0,750
0.457
0.515
0.214
0.342
1110
0.323
0.402
0.225
0.330
0.164
0.275
0.30
0.370
0.228
0,317
0,195
0.199
0.167
0.267
0.365
0.251
0 01
0/465
0.27%
0.325
0.261
0 11107
0:310
0.174
0.275
0.135
0.241
0.115
0.331
0.180
0.391
0.147
0.338
12
0J065
IL163
0,002
0.204
(L082
0.190
0.239
0.148
01272
D009
0214
0.172
0.089
11194
0.199
0.163
0.105
(.324
0.887
8 1999
24
0.079
IL176
0.128
0.443
0.110
0:224
0.205
0.357
0:234
11.40
0211
0.125
0.254
0L117
0.224
1L005
0:206
01153
(L275
0.103
0.215
0.197
0.313
0.315
0.376
0
0,57M
0.355
0.637
01164
0.261
1113
0.35
0.234
11331
0.174
0.376
121
0.333
01.22
0.359
6.136
0.122
0.126
0.307
01 1844
(1334
0.343
1 181
0.64%
0.453
0.504
0.747
1.111
0.383
D1665
0.735
0.354
1.150
0.763
0.180
0.150
Aux
0,091
0.190
0.185
0.267
0.346
0.255
0.531
0,459
0.295
0.388
0.136
0.251
0,103
0.210
0.206
0.299
0.162
0.261
0.111
0.221
0.307
0.129
0.41
0.052
0.138
0.013
0.184
0.064
0.163
0.222
0.115
0.242
5,090
0.197
0.055
0.055
0.166
0,063
0.158
0.067
0.165
0.085
0.18
POMERT
0.054
0.395
0.165
0.288
0.136
0.75
Avg
0.077
0.164
0.181
0.271
0.123
0.239
0.500
0.412
0.329
0,395
01152
0.255
0.164
0.150
0.149
0.347
0.121
0.222
0.101
0.204
0.211
0.308
0.124
0.25
13
0,068
0.189
0.091
0.201
0.060
0.182
0.122
0.335
0.154
0.276
5.119
0.223
0,071
0.171
0.080
0.181
0/051
0.185
0.079
0.183
0.168
0.232
0.112
0212
8.189
0.178
0.137
0.345
0.114
0.219
0.336
0.330
0.245
03353
0.249
0.114
(1220
0.112
0.214
0.115
0.219
0.224
0.281
0.141
0
0.123
0.204
0.265
0,343
0.164
0.264
0.562
0.540
0,440
1L470
5.206
0.292
0.199
(1,00
0.174
0.263
0L186
0.235
0.321
0.354
0.198
0.283
96
0.173
0.236
0/410
0.407
0.300
0.356
1,216
0.546
0,674
0.565
0.329
0.355
0.266
0,405
TUA51
0,277
0.385
0.221
0.257
0.405
0.417
0.320
0.51
0.113
0.194
0.226
0.299
0.172
0.260
0.534
0.457
0.379
0.416
0.200
0.279
0.122
0211
0.201
0.280
0.161
0.250
0.150
0,226
0.321
0.21
2
2
38

---

Table 18: Full results for the short-term forecasting task (Part 1). For prediction lengths {3. 6,9. 9. 12), the lookback horizon is 7 = 12; for T E {24, 36, 48, 60). it is T = 36. The best results highlighted in bold, and the second-best results are underlined. This table presents the detailed
version of 3
Model
Orthelicaar
Timchlic
Fiterles
Dlaae
Tinchlic
Lodcare
CARD
Fradloneer
PaddiEST 2002
Time Not
Metric
MSE
MAB
MSE
MAE
MSE
MAB
MSE
MAE
MSB
MAB
MSE
MAE
MSB
MAB
MAE
MSE
MAE
MAE
USE
MAE
0.465
0.349
0.659
1.455
0.000
0,433
1.611
D0945
1,30
0,547
0.55
0.400
0.551
0.585
0.500
0.992
0.528
0.555
0.395
0.46
0.477
0.527
0.620
3:337
1.773
1.249
0/655
2
0,542
1555
0.798
3.397
1.197
2.771
L138
2.099
0,840
1.551
0.795
2042
0:829
1.804
0.756
1.744
0.772
2,021
0,530
1.796
0.785
2.792
1,018
2.435
0.953
4.244
1.361
3.407
1.244
2,633
1,004
2,421
0.915
2.785
1,016
2340
0.920
Avg
1.094
0.575
1.707
0.754
1.512
0.608
2.860
1.009
2.400
1.054
1,056
0.730
1.468
0.670
1.659
0.707
1.518
0.696
1.452
0.652
1.681
0.723
1.480
0.684
744
1.737
0.000
3,110
D.579
2:190
4.765
1.335
1,555
1.241
1.877
0.876
7,085
01881
7.
0.470
7.096
0.804
2.04
01800
7,045
0.649
2,317
0.94
36
1.214
1,064
0.500
1,002
0.962
3,216
1.365
3,000
1.200
2.216
0912
2,012
0.892
2.524
0.948
0.867
1.910
0.880
2.344
0.912
1,912
0.900
-49
1.821
0.884
1.561
0.966
2.051
1.994
1.422
2.994
1994
1.921
0,910
1.500
2.157
0.911
3,056
0.952
2,096
0.891
7.133
0.865
2.238
0.90
60
1.745
0.810
1.526
0.578
2.151
0.955
4.543
1.554
3,172
1.232
1.745
0.818
1.907
0.575
2.177
0.921
1.925
0.913
2 0.02
0.919
2.001
0.595
2.007
0.925
Ang
1.764
0.882
3,000
2,071
0.885
4.130
1.465
5.001
1.217
1.915
0.852
1.982
0.876
2 2.200
0.916
1.947
0.809
1.91
01881
3.178
0.635
2,114
0.9
1.108
0.487
1.239
0.547
1.195
0,555
2,039
0.P90
2,586
0.909
1.295
0.584
1.216
0,500
1.103
0.521
1.885
0.548
1,193
0.561
1.220
0.513
2,021
0.794
L.750
0.619
2,006
0.759
1.539
0.713
2.633
D:919
3.220
1.033
1.833
0,682
1,782
0.689
1.919
0.735
1.465
0.685
1.933
0.755
1.982
0,762
2,405
0.005
0.560
1,050
3,803
2,472
2.88
0.99
COVID.9
12
2:538
0.831
3.105
0.961
2.782
0.965
1.630
1.356
4,534
1.285
3:273
1.284
2,551
0.991
2,899
0.911
2.435
0.964
2819
0.964
3,050
1,000
2,90
0.91
Arx
1,003
0.688
2.234
0.782
2.06
0.790
2,875
0:979
3,483
1.02
2.219
0.795
2,064
0.779
2.059
0.767
1.962
ILTES
2,096
0.795
2.221
0.830
2,500
0.861
24
4.474
1.150
6.335
1.554
5.906
1.517
5.246
1,829
0,700
1851
6,539
1,615
4.500
1.342
51133
4.759
5.525
1,450
5:634
1.422
36
7.341
1,670
8.222
1.587
7,996
1.733
10.545
2,000
12:804
2,003
7,085
1,710
7.37%
1.708
7.503
1.725
7.536
1,727
7.29
1698
8.351
1.530
9.114
1884
2003
60
17:079
7.187
17.15
184
2.175
11.311
2/866
15
155
3.752
15/472
2.275
12
T54
3.755
11:45
1.1
17.558
2.935
12
455
7:309
11.871
2.155
17.66
066
7,325
17.848
Ang
3.463
1.754
9.604
1.918
9.126
19964
10.944
2.09
130755
2.099
9.228
1995
5.499
1,702
9.13
1.362
8.655
1.66
8.505
1.291
9.451
1,306
9.844
1.877
0.314
0,501
0.207
0.297
0.230
0.2%%
0.226
0.306
D.25T
0,307
0.278
0.295
0,230
0.293
0.277
0.513
0.219
0.256
0.227
0.300
0.229
0.255
0.227
0.308
0.234
0.382
0.236
0.361
0.364
0.577
0.263
0.392
DU3T1
0.396
0,316
0.384
0.266
0,369
0.264
0.403
0.253
0.365
0.363
0,342
0,263
0.357
0.23
METR-LA
0.482
0.263
0.455
0.205
0.449
0.200
0.465
0.300
11452
0.553
0.455
0.294
0.442
0.202
0.434
0.281
0.457
0.292
0.460
0.295
0.455
0.292
0.482
0.294
Ark
0,335
0.230
0.334
0.245
0.331
0.242
0.344
D1251
0,341
0.254
0,335
0.245
0.327
0.243
0.349
0.326
0.244
0.315
0.243
0.344
0253
24
0,600
0.337
0.671
0411
0.600
0.50
0/116
0L5AS
0.459
0.617
0.394
0.500
0.405
0.700
0.278
0.56
0.406
0.700
0.413
0,679
0440
0.688
0.415
0,890
0.188
0.941
0-480
085
0:514
2.400
0.531
0.781
0.457
0.611
0.471
03514
0.118
0.112
0.80
0.880
0.345
0.464
0.866
0.475
019015
0.437
0.564
0.551
0.95
0.531
11011
0.546
0.585
0.812
0.510
0.965
0.504
1007
0.058
0.982
0.506
1.017
0.539
0.973
0.516
0.972
0.514
1.000
0.563
1.122
0.559
0.623
0.555
0.551
0.43
Arg
049T
3,510
D1STM
0.500
0.003
0.193
0.635
0.093
0.008
0.100
D1042
0:111
0L064
0.123
0.096
0,040
0,103
0307
0.096
0.039
0.112
0.000
0.105
0.035
0.059
0.040
0.123
0.155
0.42
0J062
0.137
6.862
1.1
0.065
0.145
0.59
0.154
01022
0.189
0.533
0.144
0:066
12147
0.141
0:067
0.147
0.065
0.150
0.065
0.145
0.003
0.161
0:15
0.07
NASDAU
Ang
0.125
6,855
0.126
0.008
0.131
0051
0.142
0.002
0.170
0.55
0.131
0,069
01185
0.07
0.130
0:099
0.155
0.060
0.137
0.58
0.112
0.088
0.151
24
0.121
0.216
0.122
0.221
0.130
0.230
0.140
5.344
0L155
0.224
0.132
0.233
0.125
0.222
0.124
0.229
0.128
0.226
0.130
0.237
0.127
0.224
0.106
0.29
0.175
0.271
0.201
0.246
0.200
1.700
0.534
0.514
0.754
7.776
0.564
0.715
0.311
0,32
0.11
0.21
0
307
0.718
0.306
01200
0.514
0.715
0.316
0.367
0.55
0.315
0.338
0.338
0.299
0.540
0.281
0.357
0.518
0.40L
0.242
0.337
0.264
0.341
0.264
0.341
0.262
0.339
0.279
0.362
0.265
0.319
0.327
0.394
AuR
0.187
0.277
0.186
0.381
0.197
0.299
0,210
0.30C
0.225
0,33L
0.193
0.250
0.196
0.286
0.193
0.284
0.194
0.265
0.200
0.207
0.195
0.266
0.255
0.443
6.161
0.368
6.709
0.392
6:334
0.402
7.470
D.4R6
6:254
01438
6.149
0.389
6.148
0.383
6.133
0.375
6,190
0.367
6,577
0.393
6.112
0.780
7.507
0.510
5,475
0,402
6.400
0.401
5.325
0.544
6.5T9
0.467
5.436
0.401
6.455
0,397
6.465
0.393
6:696
0.403
6.415
0.395
7.002
0.315
0.696
0.395
5/702
0418
6,007
0.155
$.359
0.554
6.776
0.505
6.744
0440
6.577
0.412
6.714
0-415
6.788
0.411
0,000
0.411
6:343
0.436
6.150
0.524
6,334
0.486
5:900
0.426
0.599
0.42
9.394
0.608
6.007
0,513
6.852
0.421
6.590
0.404
0332
0.415
7.368
0.424
6.66
0.419
6.814
0.414
K.117
0.533
WELLS
Arg
6,578
0.389
6.570
6.572
0:41
5.515
0.555
6.634
6.535
0.406
6,547
0. 6.004
6,553
0-000
6.705
D.
0.006
6.508
0.405
6.533
0.401
7.9%
0.500
24
6.894
0.423
5,00
0.416
6,907
0.441
7.381
DA1S
0.500
6.902
0-460
5.19
0. 0.450
0.445
6.551
0.452
889
0.431
5.318
0:410
8:023
0.612
36
0.439
5,500
0.667
6.514
0.007
3.774
3.060
0.5
6.519
0.473
6.456
0.457
6
0.451
5.035
0.453
0.853
6.400
0:415
7.229
0.591
48
9.446
6.106
0.484
6.135
0.478
7.757
0,506
5.00
0.547
6.115
0.457
6:01
0.468
6003
5:559
0.410
7.154
0.041
60
5.205
0.454
5.33
0.476
5.11
0.482
7.855
0.346
5.505
0.582
5.736
0.497
5.500
0.478
5.723
0.463
5.359
01463
5.681
0.462
5,033
0.462
6.816
0.645
Arg
6.302
0.410
6.315
0.668
6334
0.467
7,413
0:681
0,205
0,538
6,333
0,479
6.266
0.463
6
0:453
5:931
0.453
6,275
0.458
6,212
0.414
7,310
0.023
1
39

---

Table 19: Full results for the short-term forecasting task (Part 2). For prediction lengths 7 E {3, 6, 9, 12), the lookback horizon is I = 12; for E {24, 36, 48, 60), it is T = 36. The best and second-best results are highlighted in bold, and underlined, respectively. This table presents the detailed version of Table g
Model
OrthoLizaar
Timehtic
Fiterlee 20044
Dilan 2071
Timchlic .+ 3027
Ledcare 20M
CARLI 1UP-4
Peatlonner 2004
PaadiET (001)
Time Not
Metric
MSE
MAD
MSB
MAE:
MSE.
MAB
MSB
MAE
MSE
MAB
MSI
MAE
MSB
MAB
MSE
MAE
MSC
MAII
MIE
MAL
MSB
MAE
MSE
MAB
0.035
0.136
0.638
0.137
0.000
0.151
0/066
0:557
0.047
0.552
0140
0.141
D:4T
1.155
0104
0.130
0.045
0.153
0.046
0.155
0.042
0.146
0.087
0.133
0067
0.193
0.57
0.065
0.192
0.000
0.203
DOST
D:315
0.091
0.220
0076
0.166
0:81
9:099
0.203
0.307
0:187
0.005
0.264
0/092
0.213
0.002
0.224
0.106
0.342
0.113
0.25
0.93
0.213
0.104
0.256
01082
0.206
0.008
0.234
0.099
0.228
0.101
0.238
0.000
0.200
50500
Apg
(1.661
0.187
0.085
1.177
0.073
0.191
01076
0/700
0.000
0.205
0.065
0.179
0
0.001
0.178
0:015
0.195
0.073
0.191
0.013
0.162
0.063
24
0.159
0.358
0.181
0.317
0.192
0.334
0.189
0,530
0.172
0.305
0.175
0,300
0.155
0.276
0.181
0.315
0.180
0.309
0.164
0.358
0.102
36
0.209
0.317
0.216
0.345
0.224
0.541
0.259
0.359
1U250
0.363
0.227
0.344
0.232
0.358
0.296
0.319
0.239
0.365
0.225
0.345
0:221
0.341
0.219
0.34
48
0.255
9.358
0.264
0.367
0.290
0.364
0.324
9.439
0.291
0.598
0.272
0.363
0.2
0.588
0.258
0.354
0.265
0.394
0.275
0,383
0.275
0.397
0.262
0.37L
0.416
0.416
0.391
0.456
0.475
0.413
01.231
0.313
0.241
0.755
0.251
0.565
0.791
0/412
0,377
00300
0.247
0.361
0,252
0,304
0.231
0.313
0.761
0.37%
012200
0.364
0.346
0.761
0.237
0.251
3
1.580
0.276
1.568
0.376
1.597
0.276
1,11
0.283
1.574
0.290
1.555
0.275
1.538
0.272
1.503
0.276
1.565
0.217
1.551
0.276
1.556
0.276
2.290
0.325
2651
0:310
2,815
0.364
0.162
5.964
0:436
3.364
D.431
3.651
0:344
3.204
D.443
3,621
0.436
3,572
0.435
3,544
0.440
3.509
0.430
3.575
0.432
3.599
0.433
3.585
0/422
4.715
0.492
Davloaas
4.517
0.490
4.412
0.400
4.634
0.43
4.656
0.505
4.506
01407
4.630
0.496
4.522
0.471
4,525
0.490
4.605
0.493
4.579
0.403
4,533
0.412
5,999
0.56
Avg
3.000
0.389
3,849
0.390
3.125
0.393
3.160
5.400
3,009
0,305
31087
0.395
3,086
0,
3 3100
0.389
3,090
0.391
3.073
0.391
3,065
0.390
4,229
0.402
24
2.432
0.664
5.327
0.683
8,000
0.683
3.934
9,500
7,500
0.670
9.293
0.699
8,009
0.879
7.416
0.655
0.572
7.925
0.673
7.641
0.500
11.535
0834
36
10.548
0.799
11.192
0.513
12.011
0.823
11.907
0.537
10:056
0,503
14.754
0.856
11.962
0.524
10.790
0.795
11 11:455
0.508
12/057
0.821
11.210
0.507
16.922
0.92
14:045
0.914
15:27
0.965
14.31
0.913
18821
0.999
14.157
0.922
16,933
0.910
15:255
0.992
13,881
0.912
14,005
0.921
14267
0.935
669
0.915
19.501
1.00
12,121
0.545
13.91
13.419
0.571
15.255
3.90%
17A48
0,857
14. T1T
0.995
13.416
0.77
12.31%
0.549
12
19.9
0.858
11.314
0.560
12.916
0.852
17.600
10001
01.003
0.271
0.301
0.206
0.304
0.291
0.411
11:96
0.583
0.309
0.269
0.316
0.500
0.518
0.295
0.111
0.300
0.281
0.276
0.380
0,250
0.315
0.303
0.307
0.11
0.500
0:401
0.367
0.700
0.376
0.319
0,923
0.281
0.116
0.115
0.504
0.318
0-11
5.180
0.406
0.500
0:311
0.708
0,352
0.511
0.319
0.119
0.318
0L305
0.291
0.114
0.306
0.388
0351
CusSacs
0.336
0.381
0.345
0.319
0.555
0.512
0.434
0.785
0.410
01.000
0.543
0.314
0.356
0.513
0.389
0.524
0.348
0L324
0.314
0.295
0.311
0.310
0.358
0.500
Avg
0.320
8.289
0.325
0.300
0.501
0.303
0.413
0.377
0.400
0,385
0.325
0.302
0.326
0.506
0
0.311
0.331
0.313
0.295
0.285
0.315
0.300
0.327
0.312
24
0,500
0.383
0.330
0.118
0.518
0.519
0.319
D:347
0.354
0.390
0,225
0.310
0.125
0.322
0.317
0.311
0.119
0.176
0.312
0.319
0:119
0.116
0.524
36
0.334
0.315
0.312
0.351
0.551
0.530
0L313
1L:W5
01365
0.351
0.34K
0.357
0.553
0.548
0.315
0.315
1.355
0.315
0.523
0.312
0.330
0.356
0.543
0.327
0.365
0.165
0.541
0.385
0,567
0.375
0.351
0.351
0.366
0.349
0.344
0.313
0:347
0.344
0.34
0.351
3.785
Arg
0,340
0.330
0.315
0.356
0.350
0.335
0.379
3.365
0.373
0.364
0,347
0.340
0.345
0.354
0,355
0.336
0.340
0.338
0.304
0.327
0.335
0.335
0.340
0345
11.944
0.688
0.550
0:681
0.890
2.712
0.576
0.708
0.559
0.066
018955
0.691
0.965
0.567
11861
0.699
0,510
0.667
0.909
0.116
0.743
0.765
0.008
0.747
1.079
0.785
0.591
0.761
0.997
0.745
1,023
0,761
1.013
0.750
0.994
0.345
6:741
1,000
0.733
1.063
0.770
1.834
0.663
1.003
0.774
1.150
0.804
1.001
0.790
1.050
0.767
1,071
0.783
1885
0.781
1.073
0.784
1.005
0.780
1.027
0.773
1.104
0.785
17
1.109
0.769
1.887
1.718
1125
0.307
1.766
0.551
1.100
01514
1107
0.745
1.170
033
1.1
0.793
1:135
0.807
1.107
0.201
1.883
0.797
1.317
0.075
Aag
1.009
0.741
0:963
1.764
1.041
0.753
1.006
0.788
1.007
0.765
0.900
0.748
1.001
0.761
11023
0.755
1,017
0. 0.718)
1.015
0.744
0.972
0.749
1.009
0.775
24
1.343
0.870
1.341
0.581
1.410
0.916
1.491
3.544
1.300
0,916
1.340
0,817
1,257
0,509
1 1.405
0.886
1,410
0.913
1.462
0.924
1,465
0.935
1.694
0.924
36
1.445
0.983
1.430
19911
1.500
03965
1471
1/594
1.518
11957
1.446
0.5T6
1.500
01951
1.95
DNT1
1.518
0.955
1.507
0.964
1.593
0.917
1.556
0999
1.505
1.567
1,009
1.775
1,052
1,610
0,995
1.467
0:933
005T
1,020
0.981
1.776
1,053
1.956
1.122
1.67M
1.000
1,066
1.006
1,737
1.750
1.064
1.625
1.00
1.484
0.587
1.514
0.985
1,211
1,008
1,549
0.972
1.470
0,975
1.543
1.650
1.556
0.966
0.199
0/066
0.215
0.084
0.213
0.191
0.320
(.159
0.294
0.033
0,206
0.999
0.195
0.059
0.217
0.075
0.500
0.103
0.124
0.344
0.116
0.342
0.356
0.182
03002
0.11
0.33
0:11-
0239
0135
0.255
0:11
0.341
0.098
0.227
6:131
0346
0.005
0232
0.135
0.251
0.159
0.375
0.151
0.269
0.276
0.372
1.220
0,390
0.156
0.273
0.142
0.266
0.171
0.242
0.150
0,268
0.124
0.253
0.157
0:213
0.158
0.202
0.176
0.241
0.204
0.306
0.194
0.237
0.409
0.404
0.255
0.335
0.193
0.296
0.196
0.294
0.213
0.307
0.156
0.301
0.144
0.369
0.200
0.30
0.100
0.285
Weleie
Aag
0.123
0.240
0.145
0.161
0.136
0.255
0.216
0.165
11.044
0,319
0.133
0.255
0.135
0.253
0.154
0.267
0.135
0.154
0.109
0.237
0.141
0.259
11122
0.444
24
0.186
0.306
0:229
0.335
0.273
0.351
0:431
1.469
1L315
0,303
0231
0.349
0.240
0.545
0.525
0.370
6.216
0.335
08181
0.315
0.245
0:310
0.226
0.354
36
0.222
0.356
0.361
0.420
0.401
0:441
0.554
0.552
11385
0.447
0.325
0,397
0.317
0.405
0:425
0.442
0,331
0.411
0.226
0.342
0.319
0.43
0:37
0.436
0.501
0.530
0.532
06644
0/647
014%
0.496
0.450
0.413
0.475
0.451
0.370
0.504
0.115
0.36
0.411
0.571
0.502
0.630
0.991
0.736
0.673
0.464
0.510
0.525
0.517
0.561
0,540
0.595
0.565
0.556
0.547
0.410
0.547
0.366
0.478
0.307
Ang
0.074
0.644
0.555
0:354
0,015
0.004
0.012
0.062
0.151
5.359
0.072
0.200
0.014
0.012
0.062
0.104
0.057
151
0.130
0.710
3.365
1:115
0.255
0.77
0.194
0.500
01077
0.135
0.041
0.137
0.111
0:277
0.100
0313
0.107
0.266
0.369
D.445
0.191
0,525
0.101
0.200
0.119
0:095
0.150
0.176
0.21
Useeep
(1.33
0.230
0.195
0.595
0.155
0.255
0.475
0,500
(1240)
0.386
0.196
0.291
0.141
0
0.155
0.256
0.148
0.250
155
0.256
0.164
0.261
0.340
0.525
0.136
0.094
0.153
0.079
0.165
0.306
0.354
0.154
0.292
0.095
0.165
0.078
0L162
0.075
BUOTS
0.163
0,075
0.160
0.147
0.51
24
0.447
0.666
0.549
0.605
0.545
2.520
1,355
0.551
0.645
0.779
0.575
0.561
0.645
0.523
0.545
0.719
0.515
0.620
0.536
1.798
0.96
36
11.70
0.619
1,500
0.566
1.347
0.769
4.47%
1.705
11506
1.717
1.316
0.717
1.741
0.994
1.205
0.763
1.107
TL725
2,212
1.103
2.245
1.120
2.530
1.171
48
1.611
01400
1.700
1.46%
2.700
1.000
3,052
1L974
117477
7.977
1.71%
3.576
1.477
3:777
1.718
6.100
3,005
2
2,577
111
7.641
1.191
5.900
1.000
1.182
6.320
2,000
5.370
1,890
5.460
2.392
1.071
0,798
6.582
2,060
5,574
4.475
1.799
7.511
2.033
Are
1,901
0,281
3,088
1.132
2.403
LO09
5.451
1,550
(L975
0,727
2.914
1.168
1/925
1,212
2.250
L031
4.020
1,415
2.700
1.169
2.315
1,069
4,500
1.482
1
Coasti
18
35
5
6
0
0
0
3
3
3
2
0
21
14
40

---

Table 20: Comparison of different transformation bases. For SI and S2, the prediction lengths are
7 E [3, 6, 9, ]2 and r E {24,36, 48, 60), respectively. Wavelet and Wavele12 denote the classic Haar and discrete Meyer wavelets, respectively. Identity means that no transformation is applied.
This is complete version of Table +
Base
ECL
Solar Energy
PEMS03
Power (S2)
ILI (SI)
COVID- 19 (S2)
METR-LA (S2)
Metric
MSE
MAE
MSR
MAE
MSR
MAE
MSR
MAE
MSE
MAE
MSE
MAE
MSE
MAR
H1
0.131
0.221
0.179
0.191
0.060
0.159
1.343
0.870
0.468
0.349
4.474
1.180
0.650
0.337
H2
0.150
0.238
0.209
0.213
0.078
0.179
1,445
0.903
0.923
0.516
7.241
1,670
0.800
0.388
H3
0.165
0.254
0.231
0.229
0.104
0.210
1.559
0.946
1,289
0.655
10.076
1.985
0.905
0.427
Ours
H4
0.191
0.279
0.241
0.236
0.140
0.247
1.602
0.971
1.698
0.791
12.079
2,182
0.999
0.457
Avg
0.159
0.248
0.215
0.217
0.095
0.199
1.487
0.922
1.094
0.578
8.467
1.754
0.838
0.402
H1
0.132
0.222
0.181
0.193
0.061
0.161
1,469
0.917
0.486
0.350
5.192
1.378
0.654
0.337
H2
0.153
0.241
0.214
0.215
0.078
0.182
1.584
0.962
0,885
0.500
7.553
1.699
0.799
0,390
HB
0.165
0.253
0.236
0.232
0.110
0.216
1.625
0.970
1.438
0.664
11.096
2.070
0.910
0.427
Fourier
H4
0.195
0.283
0.247
0.238
0.155
0.258
1.778
1.020
2.264
0.822
12.819
2.209
1.007
0,459
Avg
0.161
0.250
0.219
0.219
0.101
0.204
1.614
0.967
1.268
0.584
9.165
1.839
0.843
0.403
H1
0.134
0.223
0.182
0.194
0.061
0,161
1.476
0.917
0.483
0.351
4.826
1.317
0.647
0.336
H2
0.153
0.241
0.217
0.216
0.079
0.183
1,586
0.959
0.943
0.520
7.117
1.652
0.801
0,390
H3
0.166
0.255
0.238
0.234
0.114
0.219
1.711
1.011
1.251
0.653
10.133
1.995
0.903
0.428
Wavelet
H4
0.186
0.276
0.250
0.240
0.176
0.278
1.880
1.060
1.786
0.795
12.588
2.233
1.011
0.464
Avg
0.160
0.249
0.221
0.221
0.107
0.210
1.663
0.987
1.116
0.580
8.666
1.799
0.840
0.404
HI
0.133
0.222
0.186
0.198
0.062
0.162
1.511
0.929
0.461
0.349
5.398
1.413
0.653
0.340
H2
0.154
0,243
0.222
0.220
0.080
0.184
1,570
0.953
0.935
0.526
7.199
1.663
0.803
0,392
0.165
0.255
0.242
0.236
0.115
0.221
1,701
1.005
1.393
0.683
10.616
2.051
0.909
0.432
Waveler2
H4
0.196
0,283
0.254
0.243
0.175
0.272
1.876
1.060
1.911
0.818
12.583
2.232
1.007
0.462
Avg.
0.162
0.251
0.226
0.224
0.108
0.210
1.664
0.987
1.177
0.594
8.949
1.840
0.843
0.406
H1 0.134
0.223
0.184
0.198
0.061
0.160
1.338
0.864
0.456
0.341
5.381
1.391
0.661
0.339
H2
0.156
0.244
0.220
0.222
0.079
0.182
1.564
0.969
0.945
0.524
8.001
1.767
0.803
0.391
H3
0.182
0,270
0.244
0.240
0.118
0.220
1.604
0.950
1.527
0.700
11.244
2.108
0.923
0,433
Chebyshev
H4
0.399
0.444
0.256
0.246
0.163
0.266
1.774
1.076
1.940
0.824
12.695
2,233
1.028
0.466
Avg. 0.218
0.295
0.226
0.226
0.105
0.207
1.570
0.965
1.217
0.597
9.330
1.875
0.854
0.407
H1 0.139
0.227
0.192
0.204
0.065
0.167
1.425
0.899
0.595
0.395
5.632
1.456
0.659
0.346
H2
0.156
0.243
0.227
0.227
0.085
0.191
1.658
0.985
1.083
0.577
8.729
1.859
0.829
0.407
H3
0.173
0.262
0.251
0.244
0.122
0.226
1.774
1.013
1.581
0.737
10.381
2.024
0.933
0,444
Laguerre
H4
0.199
0.286
0.261
0.248
0.171
0.271
1,780
1.039
2.155
0.896
12.467
2.221
1.053
0.484
Avg
0.167
0.255
0.233
0.111
0.214
1.659
0.984
1.353
0.651
9.302
1.890
0.868
0.420
H1 0.134
0.224
0.200
0.208
0.074
0,181
1,502
0.926
0.501
0.366
4.965
1,350
0.652
0.337
H2
0.152
0.239
0.238
0.231
0.080
0.183
1.603
0.962
0.964
0.529
7.396
1.703
0.799
0.389
HB
0.165
0,256
0.262
0.247
0.113
0.219
1.790
1.031
1.446
0.695
9.481
1.935
0.900
0.427
Legendre
H4
0.193
0.282
0.274
0.253
0.168
0.269
1.846
1.060
1.799
0.822
12.358
2.203
1.015
0.463
Avg
0.161
0.250
0.243
0.235
0.109
0.213
1.685
0.995
1.177
0.603
8.550
1,798
0.841
0.404
HI
0.134
0,224
0.186
0.199
0.063
0.163
1.419
0.900
0.477
0.352
4.841
1,284
0.659
0,342
H2
0.156
0.245
0.221
0.220
0.080
0.184
1.453
0.903
0.956
0.523
7.828
1.757
0.807
0.395
H3
0.167
0.257
0.244
0.237
0.117
0.222
L582
0.956
1.307
0.659
10.153
2.003
0.921
0.433
Identity
H4
0.194
0.282
0.256
0.244
0.165
0.266
1.715
1.023
1.874
0.812
12.603
2.232
1.006
0.461
Avg
0.163
0.252
0.227
0.225
0.106
0.209
1.542
0.945
1.153
0.587
8.856
1.819
0.848
0.408
functions from the set (Softplus, Identity, Softmax, Sigmoid, ReLU) and normalization methods from {L1, L2} (all applied row-wise). Here, Identity denotes no transformation.
As shown in Table 37 the combination 'Softplus L1' yields the best performance among the 10 candidates. On average, 1.1 normalization leads to 10% performance improvement over 1.2, while Softplus slightly outperforms Softmax and other transformation functions.
41

---

ECL
Traffic
0.2
0.25
0.3
MAE
MAE
0.30
0.4
0.35
0.5
100%
80%
60%
40%
20%
10%
5%
100%
80%
60%
40%
20%
10%
5%
Training ratio
Training ratio
Ours
CARD
iTrans.
TimeMixer
TimesNet FreTS
Leddam
Fredformer
PatchTST
DLinear
Figure 16: Performance change with less training data. The lookback and prediction lengths are set
to 96. The Y-axis inverted for clarity.
Learning Rate
Block Number
Model Dimension
0.5
0.5
0.5
0.4
0.4
0.4
MSE
MSE
MSE
0.3
0.3
0.3
0.2
0.2
0.2
0.1
0.1
1e.4
2e4
Se-4
le-3
4
128
256
512
1024
ETT
ECL
Traffic
Weather
Hyperparameter sensitivity with respect to the learning rate lr, the number of blocks L, and the model dimension D. Average MSEs are reported across four prediction lengths {96, 192, 336, 720). ETT denotes the average performance over four subsets: ETT (h1, h2, m1, m2). The lookback length T is uniformly set as 96.
Figure 17:
J.2 Pre- and post-linear layers in NormLin module
In CSL, the pre- and post-linear layers, named based their positions in the execution sequence, are applied around the NormLin layer (see Equation 4 Post_ Lin (NormLin (Pre_Lin(-)))
We conduct ablation studies to evaluate the contributions of these two layers. As shown in Table 38 incorporating both pre- and post-linear layers yields average performance gain of 6% compared to the variant without them, highlighting their effectiveness in refining inputs for multivariate correlation modeling and downstream series representation learning.
J.3 Ablations on Q matrices
The effectiveness of Trans has been demonstrated in Tables 4 and To further analyze the individual contributions of Q; and Qo. we conduct ablation studies on these transformation matrices.
As shown in Table 39 using both Q; and Qu achieves the best performance reeeeeineednggggeeeeee the average MSE by 5.3% compared to the baseline without transformations, by 4.4% compared to using Q. only, and by 0.8% compared to using Q; only. Notably, the variant using only Q outperforms the one using only Q.. highlighting the critical role of temporal decorrelation effective encoding and representation learning.
42

---

J.4 Swapping Q matrices
Table 40 reports the forecasting performance when both training and testing are conducted using Q matrices (Q; and Q₂) derived from different datasets. As shown, using the Q matrices from the same dataset yields the best performance, highlighting the importance of incorporating -ataset-specific temporal correlation information in time series forecasting.
K
Model efficiency
As linear model, OLinear achieves remarkable training and inference efficiency while delivering state-of-the oof-the-art forecasting performance. Figure (/ouws that OLinear achieves better efficiency than Transformer- baasee forecasters, benefiting from its relatively simple architecture. Detailed resource consumption for both training and inference is summarized in Table [4] Moreover, Table 42 presents the efficiency differences before and after applying the NormLin module to Transformer-base forecasters.
L Limitations
While this work includes extensive experiments, it primarily focuses on time series forecasting, and our claims are confined this setting. Future work includes larger-scale pre-training and wider range of time series analysis tasks (e.g.. imputation and anomaly detection), in order to further validate the robustness of OLinear and its core components, Ortho'rrans and NormLin. In addition, although NormLin is linear-based it still incurs (N?) computational and memory complexity with respect the number of variates N. Developing simpler yet effective alternatives for modeling multivariate correlations and more generally, for learning general-purposs. token dependencies- remimiis promising direction.
M Broader impacts
Multivariate time series forecasting is fundamental field with wide range of real-world applica- tions across domains such as energy, finance, healthcare, and transportation. While this work does not target specific application domain, OLinear could have general real-world applicability (e.g., weather forecasting and traffic planning). We contribute two key insights: (1) the Ortho'Tram scheme, which reformulates temporally correlated forecasting inter-independent feature prediction; and (2) the linear-based NormLin module, which effectively models multivariate correlations, enhances representation learning among temporal and frequency tokens, and remains compatible with decoder architectures and large-scale pre-trate pre-training. As step toward simpler and more efficient token depen- dency learners, the NormLin module shows potential for broader use deep learning and may inspire future research in this direction.
43

---

Table 21: Full results of applying OrthoTrans to iTransformer, PatchTST and RLinear. 'Imp.' denotes the improvement over the corresponding vanilla model. Note that RLinear consists of only a single linear layer that projects the lookback length T to the prediction horizon Nevertheless, OrthoTTrans consistently improves RLinear's performance, demonstrating its ability to enhance model capacity. This table is the complete version of Table
iTransformer
PStchTST
Rline
2024a
2023
2023
Model
Vanilla
+OrthoTrans
Vanilla
TranhoTrans
Vanilla
TrathoTrans
Metric
MSF:
MAR
MSR
MAF
MSF
MAE
MSR
MAR
MSF
MAF
MSR
MAF
96
0.334
0.368
0.334
0.367
0.329
0.367
0.322
0.359
0.355
0.376
0.348
0.366
192
0.377
0.391
0.377
0.389
0.367
0.385
0.363
0.383
0.391
0.392
0.386
0.385
336
0.426
0.420
0.414
0.413
0,399
0,410
0,394
0.405
0.424
0.415
0,419
0,406
FTTml
720
0.491
0.459
0.490
0,452
0,454
0.439
0.457
0.445
0.487
0.450
0.478
0.439
Avg
0.407
0.410
0.404
0,405
0.387
0.400
0.384
0.398
0.414
0.407
0.408
0.399
Imp.
0.80%
1.16%
0.82%
0.53%
1.51%
1.97%
96
0.148
0.240
0.136
0.233
0.161
0.250
0.153
0.243
0.201
0.281
0.197
0.273
192
0.162
0.253
0.155
0.250
0,199
0.289
0.165
0.254
0.201
0.283
0.196
0.276
336
0.178
0.269
0.169
0.265
0.215
0.305
0.182
0.272
0.215
0.298
0.211
0.291
ECL
720
0.225
0.317
0.225
0.311
0.256
0.337
0.222
0.306
0.257
0.331
0.253
0.324
Avg
0.178
0.270
0.171
0.265
0.208
0.295
0.181
0.269
0.219
0.298
0,214
0.291
Imp.
3.97%
1.87%
13.12%
8.98%
2.17%
2.35%
0.071
0.174
0.062
0.164
0.099
0.216
0.073
0.176
0.126
0.236
0.117
0.226
24
0.093
0.201
0.080
0.185
0.142
0.259
0.104
0.210
0.246
0.334
0.233
0.322
48
0.125
0.236
0.112
0.222
0.211
0.319
0.181
0.268
0.551
0.529
0.532
0.514
PEMS03
96
0.164
0.275
0.160
0.269
0,269
0.370
0.293
0.338
1.057
0.787
1.024
0.760
Avg
0.113
0.221
0.103
0.210
0,180
0.291
0.163
0.248
0.495
0.472
0.477
0.456
Imp.
8.45%
5.02%
9.69%
14.81%
3.74%
3.50%
0.067
0.165
0.055
0.147
0.095
0.207
0.065
0.160
0.118
0.235
0.109
0.221
24
0.088
0.190
0.071
0.165
0.150
0.262
0.097
0.192
0.242
0.341
0.228
0.324
48
0.110
0.215
0.093
0.188
0.253
0.340
0.164
0.247
0.562
0.541
0.539
0.522
PEMS07
96
0.139
0.245
0.122
0.218
0.346
0.404
0.264
0.311
1.096
0.795
1.065
0.773
Avg 0.101
0.204
0.085
0.179
0,211
0.303
0.147
0.227
0.504
0.478
0.485
0.460
Imp.
15.59%
11.94%
30.17%
24.94%%
3.76%
3.77%
96
0,203
0.237
0.194
0.237
0,234
0.286
0.208
0.248
0,322
0.339
0.302
0.327
192
0.233
0.261
0.231
0.265
0.267
0.310
0.237
0.268
0.359
0.356
0.342
0.346
336
0.248
0.273
0.242
0.272
0.290
0.315
0.255
0.281
0.397
0.369
0.384
0.362
Solar-Energy
720
0.249
0.275
0.245
0.275
0.289
0.317
0.255
0.281
0.397
0.356
0.387
0.356
Avg
0.233
0.262
0.228
0.262
0.270
0.307
0.239
0.269
0.369
0.356
0.354
0.348
Imp.
2.16%
0.00%
11.57%
12.22%
4.14%
2.37%
96
0.174
0.214
0.164
0.208
0.177
0.218
0.164
0.209
0.192
0.232
0.192
0.232
192
0.221
0.254
0.221
0.259
0.225
0.259
0.211
0.251
0.240
0.271
0.236
0.268
336
0.278
0.296
0.271
0.295
0.278
0.297
0.266
0.292
0.292
0.307
0.288
0.304
Weather
720
0.358
0.349
0.351
0.347
0.354
0.348
0.345
0.343
0.364
0.353
0.359
0.350
Avg
0.258
0.279
0.252
0.277
0.259
0.281
0.246
0.274
0.272
0.291
0,269
0.289
Imp.
2.42%
0.63%
4.86%
2.59%
1.22%
0.86%
0.205
0.188
0.205
0.189
0.204
0.190
0.203
0.187
0.210
0.195
0.208
0.194
0.300
0.229
0.292
0.227
0.298
0.227
0.296
0.228
0.304
0.234
0.303
0.235
0.386
0.265
0.375
0.260
0.382
0.263
0,380
0.263
0.389
0.268
0.387
0.266
METR-LA
12
0.460
0.295
0.444
0.291
0.456
0.292
0.452
0.294
0.465
0.298
0.464
0.297
Avg
0.338
0.244
0.329
0.242
0.335
0.243
0.333
0.243
0.342
0.249
0.341
0.248
Imp.
2.55%
1.02%
0.67%
0.00%
0.44%
0.28%
44

---

Table 22: Ablation results for replacing or removing components along the variate and temporal dimensions. 'Hor.' denotes the prediction horizon. The simplified results are presented in Table 6
ECL
Traffic
Solar
PEMS03
Weather
ETTml
NASDAQ (S1)
I1.I (S2)
Var.
Temp.
Hor.
MSE
MAE
MSE
MAE
MSB
MAB
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
111
0.131
0.221
0.398
0.226
0.179
0.191
0.060
0.159
0.153
0.190
0.302
0.334
0.036
0.092
1.737
0.800
H2
0.150
0.238
0.439
0.241
0.209
0.213
0.078
0.179
0.200
0.235
0.357
0,363
0.049
0.117
1.714
0.795
Normlin in
Lioear
H3
0.165
0.254
0.464
0.250
0.231
0.229
0.104
0.210
0.258
0.280
0.387
0.385
0.062
0.137
1.821
0.804
H4
0.191
0.279
0.502
0.270
0.241
0.236
0.140
0.247
0.337
0.333
0.452
0.426
0.073
0.154
1.785
0.810
Avg.
0.159
0.248
0.451
0.247
0.215
0.217
0.095
0.199
0.237
0.260
0.374
0.377
0.055
0.125
1.764
0.802
HI
0.147
0.244
0.564
0.308
0.207
0,213
0.069
0.173
0.149
0.187
0,307
0.339
0.038
0.101
2.015
0.828
H2
0.168
0,262
0.585
0.311
0,246
0,238
0,090
0.197
0.200
0,236
0,358
0,365
0.051
0.123
1.938
0.865
Lineur
Linear
H3
0.184
0.279
0.607
0.319
0.265
0.251
0.135
0.241
0.264
0,286
0,394
0,390
0.064
0.143
1.856
0.810
0.304
0.668
0.342
0.268
0,250
0.189
0.292
0.338
0.334
0.451
0.425
0.075
0.160
1.942
0.847
Aug
0.178
0.272
0.606
0.320
0,246
0.238
0.121
0.226
0.238
0.261
0.377
0.380
0.057
0.132
1.938
0.837
H1
0,147
0,230
0.441
0.241
0.201
0,208
0.069
0.170
0.164
0.199
0.304
0.336
0.035
0.091
1.767
0.805
0.165
0.246
0.471
0.251
0.236
0.229
0.099
0.200
0.210
0.243
0,362
0.366
0.049
0.116
1.905
0.826
Linear
H3
0.181
0.263
0.486
0.259
0.260
0.244
0.168
0.254
0.264
0.283
0.392
0.388
0.061
0.136
1.909
0.826
H4
0.219
0.297
0.529
0.278
0.267
0,248
0,254
0.314
0.349
0.339
0.456
0.428
0.073
0.153
1.876
0.834
Avg.
0.178
0.259
0.482
0.257
0.241
0.232
0.147
0.234
0.247
0.266
0.378
0.379
0.054
0.124
1.864
0.823
0.138
0.229
0.427
0.260
0.205
0,213
0.060
0.160
0.152
0.190
0,312
0.342
0.035
0.093
2,093
0.850
H2
0.155
0.243
0.447
0.269
0.245
0,235
0.079
0.184
0.202
0.238
0,361
0.368
0.049
0.118
1.818
0.822
NorLLnn
NomLin
113
0.173
0.261
0.466
0.276
0.277
0,255
0,120
0.227
0.259
0.280
0.393
0.388
0.062
0.138
1.974
0.842
114
0,212
0.294
0.500
0.294
0.283
0,258
0.189
0.286
0.342
0.335
0.457
0.430
0.073
0.154
1.905
0.832
Avg.
0.169
0.257
0,460
0.275
0,252
0,240
0.112
0.214
0.239
0.261
0.381
0.382
0.055
0.126
1.947
0.836
H1
0.156
0.252
0.557
0.329
0,214
0.226
0.072
0.178
0.155
0.191
0,313
0.345
0.038
0.101
2.135
0.885
H2
0.171
0.264
0.555
0.333
0.255
0.250
0.100
0.209
0.202
0.238
0.364
0.369
0.051
0.122
1.986
0.865
Lineir
NormLin
H3
0.187
0,282
0.567
0.336
0.287
0,270
0.154
0.264
0.263
0.283
0.396
0.392
0.064
0.144
1.968
0.848
H4
0.220
0.306
0.632
0.359
0,294
0,272
0.245
0.334
0.340
0.334
0.461
0.431
0.077
0.164
2.059
0.870
Avg
0,183
0.276
0.578
0.339
0.262
0,254
0.143
0.246
0,240
0.261
0,383
0.384
0.057
0.133
2,037
0.867
HI
0.158
0.242
0.474
0,282
0.237
0.237
0.073
0.178
0.163
0.199
0,314
0,346
0.035
0.092
2.154
0.886
H2
0.168
0,251
0.479
0.284
0,273
0,257
0,111
0.218
0.210
0.242
0,365
0.370
0.049
0.117
2.074
0.870
NormaLnn
113
0.185
0,268
0.493
0.289
0,310
0,277
0,202
0.293
0.264
0,282
0.397
0.392
0.062
0,138
2.105
0.874
0.304
0.526
0.307
0.313
0.344
0.387
0.346
0.336
0.462
0.433
0.073
0.154
2.039
0.866
Aug
0.185
0.266
0.493
0.290
0.283
0,262
0.182
0.269
0.246
0,265
0.384
0.385
0.055
0.125
2.093
0.874
H1
0.138
0.228
0.427
0.260
0.207
0,214
0.060
0.160
0.151
0.189
0,311
0.342
0.036
0.093
2.117
0.852
H2
0.155
0.243
0.446
0.269
0,246
0,236
0.079
0.184
0.202
0.238
0,360
0.367
0.049
0.118
1.767
0.812
w/o
0.171
0.260
0.467
0.277
0.280
0.257
0.121
0.227
0.260
0.282
0,392
0.388
0.063
0.139
1.960
0.846
H4
0.214
0.296
0.500
0.295
0,281
0,258
0.195
0.291
0.343
0.336
0.459
0.431
0.073
0.155
1.916
0.837
Avg
0.257
0.460
0.275
0.253
0,241
0.114
0.215
0.239
0.261
0,380
0.382
0.055
0.126
0.837
HI
0.155
0.252
0.575
0.331
0.216
0.228
0.073
0.179
0.152
0.190
0,315
0.346
0.037
0.099
2.190
0.891
0,266
0.575
0.333
0.254
0,249
0.100
0.210
0.204
0.238
0,362
0.369
0.052
0.123
1.978
0.861
w/
113
0.189
0.282
0.572
0.334
0.286
0.269
0.154
0.264
0.262
0,283
0,396
0.392
0.064
0.145
2.074
0.874
H4
0.217
0.306
0.642
0,368
0.292
0,270
0.242
0.332
0.342
0.337
0.462
0.431
0.076
0.162
2.051
0.872
Avg
0.183
0.276
0.591
0.341
0.262
0.254
0.142
0.246
0.240
0.262
0.384
0.384
0.057
0.132
2.073
0.874
H1
0.226
0.407
0.228
0.185
0.194
0.060
0.160
0.155
0.192
0.317
0.346
0.036
0.094
2.404
0.888
H12
0.157
0.245
0.445.00
0.243
0.218
0,218
0.075
0.179
0.206
0.240
0.369
0.375
0.050
0.118
1.979
0.857
Lineer
H3
0,180
0.267
0.461
0.252
0.235
0.234
0.105
0.212
0.267
0.288
0.416
0.403
0.063
0.138
1.895
0.827
H4
0.190
0.281
0.516
0.282
0.240
0.237
0.149
0.256
0.350
0,339
0.465
0.433
0.074
0.155
1.811
0.818
Avg
0.166
0.255
0.457
0.251
0.220
0.221
0.097
0.202
0.244
0.265
0,391
0.389
0.056
0.126
2.022
0.847
45

---

Table 23: Comparison of NormLin with the attention variants using the OLinear architecture. This is a complete version of Table 8
OLinear
Trans 2017
Reformer 2020
Flowformer
Flashformer
FLailen 2023
Mamba 2023
Lin.Altn. 2024
Enh.Atin.
Model
(Ours)
2022
2022
2025
Metric
MSE
MAE
MSE
MAB
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.131
0.221
0.138
0.226
0.134
0.223
0.135
0.225
0.138
0.227
0.135
0.225
0.142
0.231
0.134
0.222
0.131
0.221
192
0.150
0.238
0,157
0.245
0.153
0.240
0.157
0.245
0.155
0.243
0.156
0.245
0.164
0.250
0.155
0.243
0.154
0.242
BCL
336
0.165
0.254
0,180
0.267
0.169
0.257
0.171
0.261
0.177
0.266
0,172
0.262
0.180
0.267
0.174
0.261
0.166
0.255
0.191
0.279
0.190
0.281
0.211
0.293
0.196
0.283
0.197
0.285
0.194
0.283
0.219
0.302
0.190
0.279
0.187
Avg
0.159
0.248
0.166
0.255
0.167
0.253
0.165
0.253
0.167
0.255
0.164
0.254
0.176
0.262
0.163
0.251
0.159
0.249
0.398
0.226
0.407
0.228
0.413
0.226
0.411
0.229
0.407
0,229
0.417
0.229
0.420
0.250
0.409
0.228
0.403
0.229
Traffie
0.439
0.241
0.445
0.243
0.440
0.241
0.446
0.246
0.441
0.243
0.441
0.242
0.445
0.263
0.441
0.243
0.424
0.243
336
0.464
0.250
0.461
0.252
0,470
0.250
0.468
0.464
0.474
0.252
0.463
0.469
0.252
0.447
0.252
720
0.502
0,270
0.516
0,282
0.514
0.270
0.516
0.275
0.528
0.283
0.526
0.273
0.496
0.290
0.513
0,274
0.482
0.271
Avg
0.451
0.247
0.457
0.251
0.459
0.247
0.460
0.251
0.460
0.251
0.464
0.249
0.456
0.269
0.458
0.249
0.439
0.249
0.060
0.159
0.060
0.160
0.060
0.159
0.060
0.160
0.061
0.160
0.060
0.160
0.062
0.062
0.162
0.061
0.160
PEMS03
0.078
0.179
0.075
0.179
0.077
0.178
0.076
0.179
0.075
0.178
0.077
0.181
0.078
0.181
0.078
0.182 0.078
0,181
48
0.104
0.210
0.105
0,212
0.103
0.208
0.109
0,214
0.104
0.210
0.112
0.216
0.115
0.230
0.111
0.215
0.108
0,213
96
0.140
0.247
0.149
0,256
0,144
0.248
0.149
0.255
0.147
0.255
0.155
0.259
0.163
0.266
0. 156
0,261
0.142
0.251
Avg
0.095
0.199
0.097
0.202
0.096
0.198
0.099
0.202
0.097
0.201
10101
0.204
104
0.207
102
0.205
0.097
0.201
96
0.153
0.190
0,155
0,192
0.153
0.190
0.156
0.191
0.157
0.194
0.156
0.193
0.155
0.193
0.156
0.194
0.155
0.191
Weather
192
0.200
0.235
0,206
0,240
0.204
0.237
0.209
0.242
0.205
0.240
0.211
0.245
0.205
0.241
0.211
0,245
0.202
0,239
336
0.258
0.280
0.267
0,288
0.267
0.285
0.264
0.284
0.264
0.285
0.268
0.288
0.263
0.285
0,266
0,285
0.267
0,2W2
720
0.337
0.333
0,350
0.339
0.340
0.332
0.341
0.336
0.341
0.337
0.349
0.340
0.345
0.339
0.361
0.347
0.341
0,336
Avg
0.237
0.260
0.244
0.265
0.241
0.261
0.242
0.263
0.242
0.264
0.246
0.266
0.242
0.264
0.248
0.267
0.241
0.262
12
0.068
0.159
0.070
0.164
0.069
0.157
0.068
0.161
0.070
0.164
0.071
0.164
0.072
0.167
0.070
0,164
0.068
0.159
PEMSO8
24
0.089
0.178
0.095
0.187
0.087
0.175
0.090
0.181
0.096
0.188
0.094
0.585
0. 0.095
0. 1.00
0.094
0.184
0.084
0.178
48
0.123
0.204
0.130
0,215
0.118
0.201
0.130
0.212
0.128
0.216
0.141
0.220
0.139
0.228
0.133
0.217
0.117
0.204
0.173
0,236
0,207
0,265
0.174
0.234
0.195
0.252
0.206
0.263
0.213
0.260
0.226
0.276
0. 0.192 0.255
0.183
0,240
Avg
0.113
0.194
0.125
0.208
0.112
0.192
0.121
0.201
0.125
0.208
0.130
0.207
0.133
0.215
0.122 0.205
0.113
0.195
96
0.179
0.191
0.192
0.198
0.179
0.192
0.180
0.190
0.193
0.198
0.200
0.198
0.188
0.202
0.193
0.195
0.180
0.192
Solar-Einerg
192
0.209
0.213
0,220
0,219
0.211
0,214
0.218
0.220
0.215
0,215
0.230
0.222
0.221
0.224
0.230
0.223
0.211
0.213
0.231
0.229
0.237
0.236
0.232
0,230
0.237
0.234
0.240
0.246
0.247
0.244
0.234
0,230
0.241
0.236
0.243
0.240
0.243
0.236
0.252
0.242
0.246
0.251
0.242
0.257
0.245
0.249
0,242
0.244
0.236
Avg
0.215
0.217
0.223
0,223
0.216
0.218
0.222
0.221
0.223
0.223
0.231
0.225
0.228
0.228
0.229
0.224
0.217
0.218
1.737
0.800
2.404
0.888
1.776
0.805
2.015
0.842
2.565
0.895
2.723
0.909
1.867
0.795
2.580
1.846
1.714
0.795
1.979
0.857
1.769
0.806
1.820
0.832
1.973
0.856
1.944
0.864
2.002
0.846
1.832
0.835
1.794
0.829
1.821
0.804
1.895
0.827
1.911
0.820
1.867
0.824
1.891
0.831
1.884
0.839
1.996
0.837
1.854
0.829
1.890
0.824
1.785
0.810
1.811
0.818
1.827
0.814
1.823
0.815
1.837
0.816
1.987
0.872
1.935
0.835
1.972
0.868
1.982
0.868
Avg
1.764
0.802
2.022
0.847
1.821
0.811
1.881
0.828
2.066
0.849
2.134
0.871
1.950
0.828
2.059
0.859
1.878
0.836
0.036
0.092
0,036
0.094
0.035
0.092
0.036
0.093
0.036
0.094
0.037
0.096
0.036
0.093
0.036
0.094
0.036
0.093
NASDAQ
0.049
0.117
0.050
0.118
0.049
0.118
0.049
0.117
0.050
0.119
0.052
0.122
0.050
0.118
0.051
0.119
0.049
0,118
0.062
0.137
0.063
0.138
0.062
0.137
0.062
0.137
0.063
0.138
0.064
0.140
0.062
0.138
0.062
0.139
0.062
0.138
12
0.073
0.154
0.074
0.155
0.074
0.155
0.073
0.154
0.074
0.155
0.075
0.157
0.074
0.155
0.074
0.156
0.074
0.155
Avg
0.055
0.125
0.056
0.126
0.055
0.125
0.055
0.125
0.056
0.126
0.057
0.055
0.056
0.127
0.055
0.126
1d Count
27
26
3
0
10
211
6
5
2
0
2
0
15
46

---

Table 24: Applying the NormLin module to Transformer-based forecasters: iTransformer, PSt Leddam, and Fredformer. For a fair comparison, model hyperparameters, loss functions, and training strategies are kept unchanged. In the case of Leddam, only the cross-channel attention module is updated. This table presents the complete results corresponding to Table 9 in the main text.
iTransformer 2024a
PatchTST 2023
Leddam
Fredformer
2024
2024
Model
Vanilla
NormLin
Vanilla
NormLin
Vanilla
NornLin
Vanilla
NormLin
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
96 0.334 0.368 0.320 0.359 0.329 0.367 0.323 0.361 0.319
0.359
0.319
0.357
0.326
0.361 0.318
0.356
ETTml
192 0.377 0.391 0.365
0.382 0.367 0.385 0.357 0.384 0.369
0.383
0.360
0,380
0.363 0.380
0,362 0,383
336
0.426
0.420
0.400
0.405 0.399 0.410 0.390
0.408
0.394
0.402
0.391
0.401
0.395
0.403
0.391
0,404
720
0.491 0.459
0.467
0.443 0,454 0.439 0.448
0.443
0.460
0.442
0.455
0.439
0.453
0.438
0.454
0,440
Avg 0.407
0.410
0.388
0.397
0.387 0,400
0.379
0.399
0.386
0.397
0,381
0.394
0.384
0.396
0.381
0.396
96 0.148 0.240
0.138
0.232
0,161 0.250 0.155 0.246
0.141
0.235
0.139
0.234
0.147 0.241
0.138
0.235
192
0.162 0.253
0.154
0.248
0.199 0.289 0.167 0,256 0.159
0.252
0.155
0.248
0.165 0.258
0.155
0.251
336
0.178
0.269
0.171
0.265
0.215 0.305
0.183
0.273
0.173
0.268
0.171
0.265
0.177 0.273
0.172
0.271
720
0.225 0.317
0.201
0.297
0.256
0.337
0.221 0.307
0.201
0.295
0.195
0.293
0.213
0.304
0.210
0.304
Avg 0.178 0.270
0.166
0,260
0.208
0.295
0.181
0,270
0.169
0.263
0.165
0.260
0.176
0.269
0.169
0.265
12
0.071
0.174
0.062
0.164
0.099
0,216 0.071
0.176
0.063
0.164
0.063
0.164
0.068
0.174
0.064
0.168
PEMS03
24
0.093
0.201
0.078
0.184
0.142 0,259 0.103
0.211
0,080
0,185
0.079
0.182 0.093 0.202
0.081
0.188
48
0.125
0.236
0.110
0.219
0.211 0,319
0.160
0.248
0.124
0.226
0.110
0.217
0.146
0.258
0.119
0.228
96
0.164
0.275
0.159
0.269
0.269
0.370
0.251
0.331
0.160
0.266
0.159
0.266
0.228 0,330
0.170
0.284
Avg 0.113
0.221
0.102
0.209
0.180
0.291
0.146
0.241
0.107
0.210
0.103
0.207
0.134
0.241
0.108
0.217
12 0.067
0.5
0.055
0.145
0.095 0,207 0.084
0.199
0.055
0.145
0.054
0.145
0.063
0.158
0.057
0.150
PEMS07
24
0.088
190 0.070
0.162 0.150 0,262 0.098
0.198 0.070
0.164 0.067
0.160 0.089 0.190 0.073
0.168
48
0.110
0.215
0.091
0.187
0.253 0.340
0.243
0,341 0.094
0.192
0.087
0.184
0.135
0.239 0.099
0,201
96
0.139
0.245
0.128 0.220 0.346 0.404
0,249
0.310
0.117
0.217
0.119
0.222
0.196
0.294
0.157 0.267
Avg 0.101
0.204
0.086
0.178
0.211 0.303
0.168
0.262
0.084
0.180
0.082
0.178
0.121
0.220
0.096 0.196
Solar-Energy
96 0.203 0.237
0.194
0.233
0.234
0.286
0,208
0,249
0.197
0.241
0.186
0.227
0.185
0.233
0.190
0.231
192 0.233
0.261 0.225
0.258
0.267 0.310 0.234
0,266
0.231
0.264
0.221
0,258
0.227
0.253
0.222 0.254
336 0,248
0.273
0.241
0.273
0,290
0.315 0.252
0.278
0.241
0.268
0.242
0,270
0.246
0,284
0.245 0.275
720
0.249 0.275
0.244
0.277
0.289
0.317 0.253
0.277
0.250
0.281
0.239
0.271
0.247
0.276
0.249
0.281
Avg 0.233
0.262
0.226
0.260
0.270
0.307
0.237
0.267
0.230
0.264
0.222
0.257
0.226
0.262
0.226 0.260
96 0.174 0.214
0.160
0.205
0.177
0.218
0.161
0.206
0.156
0.202
0.156
0.201
0.163
0.207
0.156
0.202
Weather
192
0.221
0.254
0.209
0.250
0.225 0.259
0.207
0.249
0.207
0.250
0.206
0.249
0.211
0.251
0.205
0.248
336
0.278
0.296
0.268
0.294
0.278
0.297
0.266
0,291
0.262
0.291
0.262
0.291
0.267
0.292
0.261
0.289
720
0.358 0.349
0.344
0.344
0.354
0.348
0.345
0.344
0.343
0,343
0.345
0.344
0.343
0.341
0.340
0.341
Avg 0.258
0.279
0.245
0.273
0.259
0.281
0.245
0.272
0.242
0.272
0.242
0.271
0.246
0.273
0.240
0.270
METR-LA
0.205
0.188
0.202
0.189
0.204
0.190
0.205
0,188
0.204
0.191
0.202
0.189
0.205
0.188
0.200
0.186
0.300
0.229
0.293
0.226
0.298
0.227
0.297
0.229 0.293
0.227
0.289
0.226
0.298
0.227
0.296
0.227
0.386
0.265
0.375
0.259
0.382
0.263
0.379
0.262
0.369
0.264
0.366
0.259
0.385
0.263
0.369
0.263
12 0.460
0.295
0.442
0.289
0.456
0.292
0,453
0.293
0.442
0.292
0.423
0.287
0.457
0.292
0.451
0.291
Avg 0.338
0.244
0.328
0.241
0.335
0.243
0.333
0.243
0.327
0.243
0.320
0.240
0.336
0.242
0.329
0.242
47

---

Table 25: Full evaluation of OLinear-C. 51 and S2 correspond to 'Input-12. 12, Predict- 6, 9, 12 and 'Input-36, Predict-(24, 48, 60)", respectively. This table presents the full results of Table 7
Dataset
ETTml
ETTm2
ETThl
FTTh2
ECL
Exchange
Traffic
Metric
MSB
MAE
MSE
MAB
MSB
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.334
0.249
0.382
0.284
0.329
0.131
0.082
0.200
0.398
0.226
OLinea-C Olinear
192
0.357
0.363
0.232
0.290
0.416
0.414
0.360
0.379
0.150
0.238
0.171
0.293
0.439
0.241
336
0.387
0.385
0.291
0.328
0.457
0.438
0.409
0.415
0.165
0.254
0.331
0.414
0.464
0.250
720
0.426
0.389
0.387
0.462
0.415
0.431
0.191
0.279
0.837
0.688
0.502
0.270
Avg 0.374
0.377
0.270
0.313
0.424
0.424
0.367
0.388
0.159
0.248
0.355
0.399
0.451
0.247
96
0.335
0.169
0.249
0.383
0,285
0.329
0.220
0.198
0.404
0.227
192
0.357
0.364
0.232
0.290
0.415
0.414
0.358
0.377
0.156
0.243
0.171
0.293
0.435
0.241
336
0.389
0,386
0.291
0.328
0.460
0,440
0.407
0.415
0.167
0,256
0.325
0.410
0.465
0.250
720
0.426
0.389
0.387
0.460
0,423
0.436
0.192
0.279
0.845
0.692
0.502
Avg
0.375
0.378
0.270
0.313
0.424
0.424
0.368
0.389
0.161
0.249
0.355
0.398
0.451
0.247
Dataset
Weather
Solar-Fnergg
PEMS03
PEMS04
PFM500
PEMS08
(SII
Metric
MSB
MAB
MSE
MAB
MSB
MAB
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
H1
0.153
0.190
0.179
0.191
0.060
0.159
0.068
0.163
0.052
0.138
0.068
0.159
0.468
0.349
OLincaar
H2
0.200
0.235
0.209
0.213
0.078
0.179
0.079
0.176
0.065
0.151
0.089
0.178
0.923
0.516
113
0.258
0,280
0.231
0.229
0.104
0,210
0.095
0.197
0.084
0.171
0.123
0.204
1.289
0.655
H4
0.337
0.333
0.241
0.236
0.140
0.247
0.122
0.226
0.108
0.196
0.173
0,236
1.698
0.791
Avg
0,237
0.260
0.215
0.217
0.095
0.199
0.091
0.190
0.077
0.164
0.113
0.194
1.094
0.578
OLinnar--C
111
0.152
0.189
0.178
0.191
0.061
0.160
0.068
0.163
0.052
0.138
0.068
0.159
0.485
0.352
0.202
0.237
0.209
0.213
0.076
0.178
0.078
0.176
0.064
0.151
0.089
0.177
0.951
0.523
H3
0.258
0,279
0.231
0.229
0.105
0,211
0.097
0. 0.197
0.083
0.171
0.120
0.203
1.336
0.667
H4
0.340
0.333
0.241
0,236
0.142
0.248
0.121
0.227
0.108
0.195
0.177
0,237
1.693
0.790
Avg
0.238
0.259
0.215
0.217
0.096
0.199
0.091
0.191
0.077
0.164
0.113
0.194
1.116
0.583
Dataset
ILT (S2)
COVID-19 (SI)
COVID-19 (S2)
METR-LA (SI)
METR-LA (S2)
NASDAQ (S1)
NASDAQ (S2)
Metrie
MSB
MAE
MSF
MAF
MSB
MAE
MSF
MAF
MSE
MAE
MSF
MAE
MSF
MAE
H1
1.737
0,800
1.100
0.487
4.474
1.180
0.207
0.171
0.650
0.337
0.036
0.092
0.121
0.216
OLinear
0.795
0.619
7.241
1.670
0.301
0.207
0.388
0.049
0.117
0.163
H3
1.821
0.804
2.239
0.734
10.076
1.985
0.382
0.238
0.905
0.427
0.062
0.137
0.205
0.296
114
1.785
0,810
2.538
0.831
12.079
2.182
0.452
0.263
0.999
0.457
0.073
0.154
0.259
0.336
Avg
1.764
0.802
1.907
0.668
8.467
1.754
0.335
0.220
0.838
0.402
0.055
0.125
0.187
0.277
OLinear-C
111
18850
0,835
1.107
0.493
4.980
1.336
0,207
0.172
0.654
0.337
0.035
0.092
0.119
0.214
H2
1.749
0,801
1.693
0.630
7.605
1,719
0.301
0.207
0.804
0.388
0.049
0.117
0.162
0.259
H3
1.847
0,808
2.251
0.740
10.336
2.015
0.382
0.238
0.916
0.428
0.061
0.137
0.207
0.298
H4
1.792
0,808
2,545
0.843
12.251
2,204
0.450
0.263
1.016
0.462
0.073
0.153
0.255
0.334
Avg
1,809
0.813
1,899
0.676
8.793
1.819
0.335
0.220
0.847
0.404
0.054
0.125
0.186
0.276
Dataset
Wiki(S))
Wiki (S2)
SP500
(S1)
SP500
(S2)
Dowlones (S1)
Dowdenes (S2)
CarSales (S1)
Metric
MSB
MAE
MSE
MAE
MSB
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
H1
6.161
0.368
6.894
0.423
0.035
0.126
0.155
0.271
1.550
0.276
7.432
0.664
0.303
0.277
OLincar
H2
6.453
0.385
6.446
0.439
0.053
0.158
0.209
0.317
2.569
0.361
10.848
0.799
0.315
0.285
0.398
6.004
0.446
0.070
0.181
0.258
0.358
3.565
0.430
14.045
0.914
0.327
0.293
H4
6.834
0.406
5.705
0.454
0.088
0.204
0.305
0.387
4.517
0.490
16.959
1.017
0.356
0.301
Avg
6.528
0.389
6.262
0,440
0.061
0.167
0.231
0.333
3,050
0.389
12.321
0.848
0.320
0.289
OLinear-C
H1
6.187
0,368
6.888
0,422
0.035
0.126
0.156
0.271
1.553
0.276
7.496
0.665
0.303
0.276
H12
6.457
0.384
6.437
0.438
0.054
0.159
0,210
0.319
2.570
0.361
10.965
0.802
0.317
0.287
H3
6.652
0.396
6.003
0.446
0.071
0.182
0.262
0.358
3.573
0.431
14,161
0.915
0.328
0.294
H4
6.835
0.406
5.717
0,456
0.088
0.205
0,296
0.378
4.538
0.490
17.084
1.018
0.337
0.300
Avg
6.533
0.388
6.261
0.440
0.062
0.168
0.231
0.331
3,058
0.389
12.427
0.850
0.321
0.289
Dataset
CarSales (S2)
Power (S1)
Power (S2)
Website (SI)
Website (S2)
Unemp (S1)
Unemp (S2)
Metric
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
H1
0.320
0.302
0.864
0.688
1.343
0.870
0.077
0.199
0.186
0.306
0.012
0.047
0.458
0.447
OLinear-C QLinear
H2
0.334
0,315
0.991
0.742
1.445
0.903
0.103
0.227
0.272
0.356
0.041
0.108
0.870
0.619
113
0.347
0.327
1.062
0.770
1.559
0.946
0,135
0.251
0.365
0.391
0.084
0.170
1.643
0.900
0.358
0.337
1.119
0.789
1.602
0.971
0.176
0.281
0.486
0.481
0.131
0.220
2.593
1.182
Avg
0.320
1.009
1.487
0.922
0.123
0.240
0.383
0.067
0.136
1.391
0.787
0.320
0.301
0.873
0.696
1.460
0.922
0.222
0.176
0.304
0.012
0.047
0.437
0.434
H2
0.334
0.315
0.997
0.744
1.527
0.939
0.110
0.232
0.283
0. 0.364
0.039
0.105
0.899
0.633
H3
0.347
0.327
1.060
0.768
1.721
1.016
0.144
0.257
0.402
0.421
0.079
0.163
1.963
1.026
0.358
0.337
1.128
0.796
1,869
1.077
0.188
0.286
0.517
0.490
0.132
0.222
2.571
1.162
Avg
0.340
0.320
1.014
0.751
1.644
0.988
0.133
0.249
0.344
0.395
0.065
0.134
1.467
48

---

Table 26: Robustness of OLinear-C performance. Standard deviations are calculated over seven random seeds.
Dataset
ECL
Traffic
ETTml
Bolar-Energy
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.130=3e-4
0.22014e-4 0,40424e-3
0,227+7e-4
0.303=7e-4
0.335+7e-4
0.178+1e-3
0.19147e-4
Horizon
192
0.156=1e-3
0.24349e-4
0.435+2e-3
0.35742e-4 0.241#4e-4
0.364#2e-4
0.20948e-4
0,21343e-4
336
0.167=1e-3
0.25618e-4
0.465#5e-3
0.250+2e-4
0.389
0.386:5e-4
0.231+5e-4
0.22914e-5
720
0.192#9e-3
0.279*6e-3 0.502+5e-3
0.271#6e-4
0.452#6e-4
0.426+44e-4
0.241+4e-4
0.236t-4e-4
Dataset
Weather
PEMS03
NASDAQ (SI)
Wiki (S1)
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
H1
0.154=le:3
0.191
0.061
0.160:4e-4
0.035#2e-4
0.09245e-4
6.187+9e-3
0.
Honizon
H2
0,20443-3
0.238+2e-3
0.076*8e-4
0,178+8c-4
0.049=8c-5
0.117:4c-4
6.457+7e-3
0.384
H3
0.259+4e-3
0,280#4e-3
0.105+9e-4
0,211=6e-4
0.061#0.000
0.137#3e-4
6.652+4e-3
0.396#66-4
H4
0.343=6e-3
0.334+3e-3
0.142+2c-3
0.248+
0.073=5e-5
0.153+2c-4
6.83543e-3
0.406+2c-4
Dataset
DowJones (S2)
SP500 (S2)
CarSales (S1)
Power (S2)
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
H1
7.49624e-2 0.66549e-4 0.156+2e-3
0.271+2e-3
0.303#1e-3
0.276*1e-3
1.46014e-2
0.922+1e-2
Horizon
H2
10.96547e-2
0.802+1e:3 0.210#le-3 0.319#1e:3
0.31747e-4
0.287
1.52743e-2
0,93942e-2
H3
14.16145e-2
0.91547e.4 0.26242e-3 0.358+2e-3
0.328=7e-4
1.721=7e-2 0.294
1.016#3e-2
H4
17.084#7e-2
1.018+le33
0.296*3e-3
0.378+3e-3
0.337+4c-4
0.300+44-4
1.869#1e-2
1.077+5e-3
Table 27: Ablation study on different transformations of CorrMat,
Dataset
ECL
Trallic
Solar
PEMS03
Weather
NASDAQ
ILI
Web_S1
Car_S2
Metric
MSE
MSE
MAE
MAE
MSE
MSE
MSE
MAE
MSE
MAE
MAE
H1
0.130
0.220 0.404
0.227
0.178
0.191 0.061 0.160
0,152
0.189
0.035
0.092
1.850
0.835
0.078
0,200
0.320
0. 0.302
Sofbmax
0.156
0,243
0.435
0.241 0.209
0.213
0.076
0.178
0.202
0.237
0.049
0.117
1.749
0.801
0.108
0.230
0.334
0.315
H3
0.167
0,256
0.465
0.250
0.231
0.229
0.105
0.211
0,258
0.279
0.061
0.137
1.847
0.808
0.143
0.256
0.347
0.327
0.192
0,279
0.502
0.271
0.236
0.142
0.340
0.333
0.073
0.153
1.792
0.185
0.285
0.337
Avg
0.161
0.249
0.451
0.247
0.215
0.217
0.096
0.199
0.238
0.259
0.054
0.125
1.809
0.813
0.128
0.243
0.340
0.320
111 0.131
0,221
0.401
0.227
0.178
0.191
0.061
0.160
0.154
0.191
0.035
0.092
1.854
0,838
0.078
0.200
0.320
0.302
Softplus
0.153
0,241
0.436
0.241
0.213
0.078
0.204
0.240
0.049
0.117
1,741
0.801
0.108
0.230
0.315
H3
0.166
0,255
0.461
0.249
0.231
0.229
0,106
0.211
0.262
0.281
0.061
0.737
1.843
0.807
0,143
0.256
0.347
0.327
H4
0.191
0,280
0,511
0.271
0.242
0,236
0.140
0.247
0,342
0.336
0.073
0.154
1.793
0.808
0.185
0.285
0.358
0.337
Avg 0.160
0.249
0.452
0.247
0.215
0.217
0.096
0.199
0.240
0.262
0.054
0.125
1.808
0.813
0.128
0.243
0.340
0.320
H1 0.131
0,221
0.398
0.227
0.178
0.192
0,060
0.159
0.152
0.189
0.036
0.093
1.727
0.796
0.078
0.200
0.320
0.302
Signoid
H2
0.152
0.240
0.441
0.241
0.213
0,214
0.077
0.179
0,206
0.241
0.049
0.117
1.988
0.832
0.108
0.230
0.334
0.315
11
0.166
0,256
0.460
0,250
0,230
0.229
0.104
0.210
0,266
0.285
0.061
0.137
1.845
0.807
0,143
0.256
0.347
0.527
H4
0.191
0.280
0.509
0.270
0.241
0.236
0.140
0.247
0.345
0.337
0.073
0.14
1.795
0.809
0.185
0.285
0.358
0.337
Avg
0.160
0.249
0.452
0.247
0.216
0,218
0.095
0.199
0.242
0.263
0.055
0.125
1,819
0.811
0.128
0.243
0.340
0.320
H1
0.132
0.222
0.403
0.227
179
0.192
0.060
0.159
0.154
0.190
0.035
0.092
1.844
0.832
0.078
0.201
0.320
0.302
R+LU
H2
0.153
0,241
0.446
0.241
0.213
0,214
0.076
0.178
0.205
0.241
0.049
0.116
1.757
0.802
0.108
0.229
0.334
0.315
0.171
0.250
0.229
0.258
0.136
1.841
0.256
0.327
114
0.193
0.282
0.519
0.271
0.242
0.236
0.141
0.248
0,343
0.335
0.073
0.153
1.790
0.808
0.185
0.285
0.358
0.337
Avg
0.162
0,251
0.458
0.247
0,216
0.218
0.096
0.199
0.240
0.261
0.054
0.124
1.808
0.812
0.128
0.243
0,340
0.320
111
0.132
0.222
0.402
0.226
0.179
0.192
0.060
0.150
1.843
0.201
0.301
Idenuty
H2
0.152
0,240
0.445
0.241
0,213
0,214
0.076
0.178
0,204
0.240
0.049
0.116
1.757
0.802
0,108
0.229
0.334
0.315
HG
0.170
0.259
0.464
0.250
0.230
0.229
0.106
0.211
0.262
0.282
0.061
0.1
1.842
0.807
0.144
0.257
0.347
0.327
H4
0.194
0,284
0.520
0.271
0.242
0.236
0.141
0.248
0.341
0.334
0.073
0.153
1.791
0.808
0.188
0.285
0.358
0.337
Avg.
0.162
0.251 0.458
0.247
0,216
0.218
0.096
0.199
0.239
0.261
0.054
0.124
1.808
0.812
0.129
0.243
0.340 0.320
49

---

Temporal decorrelation comparison of OrthoTrans, DFT and Wavelet transforms. We report the off- off-diagonal Frobenius norm of the temporal correlation matrix, divided by the number of off-diagonal elements.
Table 28:
Training Set
Test Set
Dataset
Original
OrthoTrans
DFT
Wavelet
Original
OrthoTraas
DFT
Wavelet
ETTh1
6.7E-03
1.6E-04
5,8E-04
3.7E-03 5.2E-03
2.1E-04
6.0E-04
3.1E-03
ETTm2
9.1E-03
1.4E-04
1.1E-03
4.6E-03 7.2E-03
2.2E-04
1.3E-03
3.7E-03
ECL
5,0E-03
1.9E-04
4.7E-04
3.6E-03 5.0E-03
2.2E-04
4.9E-04
3.6E-03
Traffic
3.8E-03
1.9E-04
5,5E-04
2.6E-03
3.8E-03
2.3E-04
5.7E-04
2.6E-03
Weather
6.7E-03
3.1E-04
4.1E-03
3.6E-03
7.2E-03
4.0E-04
4.6E-03
3.7E-03
Avg
6.3E-03
2.0E-04
1.4F-03
3.6F-03 5.7E-03
2.6E-04
1.5F-03
3.3E-03
Table 29: Performance with varying lookback horizons. The best and second-best results are highlighted in bold and underlined, respectively, OLinear-C refers to the OLinear variant where the weight matrix in NormLin is replaced by Softmax (CorrMat,), with CorrMat, being the cross-variate correlation matrix.
OLinear
OLinear-C
Leddam
CARD
Frefformer
irans.
TimeMixer
PatchTST
TemesNet
DLinear
Model
(Ours)
(Ours)
2024
2024b
2024
2024a
2024a
2023
2023b
2023
Metric
MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE
96
0.123 0.212 0.123 0.212 134 0.227 0.129 0.223 0,129 0,226 0.132 0.227 0.129 0.224 0.129 0.222 0.168 0,272 0.140 0.237
192
0.143 0,232 0.144 0.233 0.156 0.248 0,154 0,245 0,148 0,244 0,154 0,251 0,140 0.220 0.147 0.240 0,184 0.322 0.153 0,249
ECL
336
0,157
0.247 0,156 0.247 166 0.264 0.161 0.257 0,165 0,262 0, 0,268 061 0.255 0,166 0,259 0.198 0,300 0, 0,267
720
0,181
0,270 0.180 0.270 0.195 0.291 0.185 0.278 0.193 0,286 0,193 0,288 0.194 0.287 0.197 0.290 0,220 0.320 0,203 0,301
Avg
0.151
0.240 0.240 06163 0.257 0.157 0.251 0.159 0.254 O.
0.1 0.247 0.1 0.253 0.193 0.304 0.166 0.264
96
0.338
0.221 0,340 0.221 0.366 0,260 0.341 0.229 0.358 0.257 0.359 0.262 0,360 0.249 0.360 0.249 0.593 0.321 0,410 0,282
Tralic
192
0,361 0.233 0,369 0.235 0.394 0,270 0.367 0.243 0.381 0,272 0,376 0,270 0.375 0.250 0.379 0.256 0.617 0.336 0.423 0,287
336
0,386
0.241 0,390 0.241 0.400 0.283 0,388 0.254 0.396 0,277 0,393 0,279 0,385 0.270 0.392 0.264 0.629 0.336 0.436 0,296
720
0.444
0.264 0,451 0.266 0.442 0.297 0.427 0.276 0.424 0.296 0,434 0.293 0.430 0.281 0.432 0.286 0.640 0.350 O 0.466 0,315
Avg.
0.382
0.240 0.387 0.241 O. 0.400 0.278 0,381 0.251 0.390 0.275 0,390 0,276 0,388 0.263 0.391 0.264 0.620 0.336 0.434 0.295
96
0.144
0.144 0.144 0.183 0. 149 0.199 0.545 0.186 150 0.203 0,165 0,214 O. 147 0. 197 0.149 0.198 0.172 0.220 0.176 0,237
Weather
192
0.190
0.230 1.9 0.228 O. 196 0.243 18187 0.227 0.194 0,246 0,208 0.253 189 0.239 0.194 0.241 0.219 0.261 0.220 0,282
336
0.235
0.268
0,236 0.268 0.243 0.280 0,238 0.258 0.243 0,284 0,257 0.292 0.241 0.280 0.306 0,282 0.246 0.357 0,265 0,319
720
0.316
0.323 0.316
0.323 0.321 0.334 0.308 0.321 0.308 0.333 0.331 0.343 0.310 0.330 0.314 0.334 0.365 0.359 0.323 0.362
Avg
0.221
0,251 0.221 0.250 0.227 0.264 0.220 0.248 0.224 0,266 0,240 0.275 0.222 0.262 0.241 0.264 0.251 0.294 0,246 0,300
Solur Energy
0.169
0.194 0.195 0.207 0.186 0.242 0.170 0.207 0.187 0,236 0.190 0.241 0.167 0.220 0.224 0.278 0.219 0.314 0,289 0,377
192
0.190
0.212 0. 0.212 0.208 0.262 D. 0.219 0.251 0.233 0.261 0.187 0.249 0.253 0.298 0.231 0.322 0,319 0.397
336
0.192
0.215 0.194 0.219 0.218 0.265 0.226 0.233 0,208 0.265 0.226
0.200 0.258 0.273
0.337 0.352 0.415
720
0.204
0.225 0,203 0.226 0.208 0.273 0.217 0.243 0.209 0.272 0.220 0.282 0.215 0.250 0.272 0.308 0.280 0.363 0.356 0.412
Avg.
0.189
0.211 195 0.216 0.205 0.261 0.201 0.225 0,200 0.256 0.217 0.265 192 0.244 0.256 0.298 0.244 0.334 0.329 0,400
0.275
0.326 0.275 0.326 0.294 0.347
0,284 0,338
0.291 0.340 0.293
0.375 0,299 0.343
ET1m]
192
0.318
0,352 0.317 0.351 0.336 0.369 0.332 0.357 0.323 0,364 0,346 0,383 0.327 0.365 0.333 0.370 0.374 0,387 0,335 0.365
336
0.352
0.373 0.352 0.373 0.364 0.389 0.364 0,376
0,358 0,387 0,385 0.410
0.360 0.381 0.369 0.392 0.410 0.411 0,369 0,386
720
0.406
0.408 0.406 0.408 0.421 0.419 0.414 0.407 0.420 0.417 0,440 0.442 0.415 0.417 0.416 0.420 0.478 0.450 01.425 0.421
Avg
0.338
0.365 0.338
0.364 0.354 0.381 0.350 0.368 0.346 0,376 0,370 0.398 0.348 0.376 0.353 0.382 0.400 0.406 0.357 0.379
1d Count
12
16
0
0
4
2
0
0
0
0
50

---

Table 30: Comparison OLinear with more baselines. MAEs are reported.
OLinear (Ours)
SimpleTM
TQNet 2025
TimePro
TimeBase
Model
2025
2025g
2025c
96
0.334
0.361
0.353
0.364
0.388
192
0.363
0.380
0.378
0.383
0.409
336
0.385
0.404
0,401
0,409
0.421
ETTml
720
0.426
0.438
0.440
0.446
0.461
Avg
0.377
0.396
0.393
0.400
0.420
96
0.249
0.257
0.256
0.260
0.271
192
0.290
0.299
0.298
0.303
0.309
336
0.328
0.338
0.340
0.342
0.346
ETTm2
720
0.387
0.395
0.396
0.399
0,401
Avg
0.313
0.322
0.323
0.326
0.332
96
0.382
0.392
0.393
0.398
0.392
192
0.414
0.421
0,426
0.429
0.423
336
0.438
0.438
0,446
0,450
0.443
ETTh1
720
0,462
0.462
0.470
0.474
0.458
Avg
0.424
0.428
0.434
0.438
0.429
96
0.329
0.338
0.343
0.345
0.376
192
0.379
0.387
0.393
0,394
0,405
336
0,415
0.401
0,427
0.431
0.440
ETTh2
720
0.431
0.436
0.446
0.445
0.477
Avg
0.388
0.391
0.402
0.403
0.424
96
0.221
0.235
0.229
0.234
0.279
0.238
0.247
0.247
0.249
0.281
336
0.254
0.267
0.264
0.267
0.295
ECL
720
0.279
0.293
0.294
0.299
0.327
Avg
0.248
0.260
0.259
0.262
0.295
96
0.226
0.274
0.261
0.269
0.384
192
0.241
0.280
0.271
0.276
0.362
336
0.250
0.290
0.277
0.287
0.365
Traffic
720
0.270
0.309
0.295
0.312
0.386
Avg
0.247
0.289
0.276
0.286
0.374
96
0.190
0.207
0.200
0.207
0.215
192
0.235
0.248
0.245
0,254
0,256
336
0.280
0,290
0,287
0.296
0.297
Weather
720
0.333
0.341
0.342
0.346
0.348
Avg
0.260
0.271
0.269
0.276
0.279
96
0.191
0,232
0,233
0.237
0.363
192
0.213
0.247
0.257
0.263
0.404
336
0.229
0.257
0,263
0.281
0.398
Solar-Energy
720
0.236
0.252
0.270
0.285
0.388
Avg
0.217
0.247
0.256
0.266
0.388
51

---

Table 31: Few-shot forecasting performance with 10% training data. Results are averaged over four prediction lengths (96, 192, 336, 720)
OLincar
TimeMixcr+
TimeMixer
ITans
PatchISTT
TimesNet
DLincar 2023
TIDE 2023
Crossfm. 2023
FEDformer
Madel
(Ours)
2025a
2024a
2024a
2023
2023h
2022
Metric
MSE MAE
MSE
MAE
MSE MAE
MSE MAE MSE
MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE
ETTh1 0.478 0.456 0.517
0.512
0.613 0,520 0,510 0.597 0.633 0,542 0.869 0.628 0.691 0.600 0,589 0,535 0.645 0,558 0.639 0,561
ETTh2
0.390 0.407 0.379
0.391
0.402 0.433 0.455 0.461 0.415 0.431 0.479 0.465 0.605 0.538 0.395 0.412 0.428 0.447 0.466 0.475
ETTml
0.405 0,400 0.398
0.431
0.487
0.461 0.491 0.516 0.501 0.466 0.677 0,537 0.411 0.429 0.425 0.458 0.462 0.489 0.722 0.605
ETTm2
0.287 0.330 0.291
0.351
0.311
0.367 0.375 0.412 0.296 0.343 0.320 0.353 0.316 0.368 0.317 0.371 0.343 0.389 0.463 0.488
Weather
0.245 0.265 0.241
0.271
0.242
0.331 0.242 0.279 0.279 0,301 0.241 0.283 0.249
0.267
0.284 0.324
Table 32: Zero-shot forecasting performance. Results are averaged over four prediction lengths {96, 192, 336, 7203. A - B denotes that the model is trained on dataset A but evaluated on B.
Model
OLineur
TimeMixer++
TimeMixer
LLMTime
DLinear
PuccTTS
TimesNet
iTrans.
Crossfm.
FEDformer
(Ours)
2025c
2024g
2023
2023
2023
2023b
2024a
2023
2022
Metric
MSE MAE MSE
MAE
MSE
MAE MSE MAE
MSE
MAE
MSE
MAE
MSE MAE MSE
MAE
MSE
MAE
MSE MAE
ETTbl ET
0.367
0.391 0.367
0.391
0.427 0.424 0.992 0,708 0.493 0.488 0.380 0.405 0.421 0.431 0.481 0.474 0,555 0.574 0.712 0.693
ETTh-2
0.511
0.352 0.301
0,357
0,561
0.397 1 1666 0.869 0,415 0,452 0.314 0.360 0.327 0,361 0,311 0,361 0.613 0.629 0.681 0.588
ETTh2-ETTh
0.507
0.479 0.511
0.498
0.679
0.577 1,961 0.981 0.703 0,574 0.565 0.513 0.865 0.621 0.552 0,511 0.587 0,518 0.612 0.624
ETTal-ETTh2
0.413
0.425 0.417
0.422
0.452
0.441 0.992 0,708 0.464 0.475 0.439 0.438 0.457 0.454 0.434 0.438 0.621 0,541 0,533 0.591
ETim)-EETin
0.291 0.327 0.291
0.351
0.529
0.357 1 866 0.869 0,355 0,389 0.296 0.334 0,322 0,354 0,324 0,8511 0.595 0,572 0.612 0.611
ETTm2-ETTml
0.480
0.445 0.427
0.448
0.554
0.478 1.933 0.984 0.649 0.537 0.568 0,492 0.769 0.567 0.559 0.491 0.611 0.593 0.577 0.601
Table 33: Performance comparison of OLinear and large time series models on fine-tuning and zero-shot scenarios. MSEs are reported, with prediction length of 96. Better results are highlighted in bold. For OLinar's zero-shot learning, we employ a cross-dataset strategy: training on ETThl and evaluating on ETTh2 (and vice versa), and similarly for ETTml and ETTm2. For Time-MoE (Shi
al. 2025], J. Moirai (Woo et al. 2024). and Chronos [Ansari et al., 2024] we use the largest model
variants from their respective families. Baseline results are sourced from Liu et al., 2024b and Liu
ct al. 2025
Scenaria
100% Target
20% Target
5% Target
0% Target (hot)
Model
OLinear (Ours)
Timer
OLinear (Ours)
Timer 2024b
OLinear (Ours)
Timer 2024b
OLinear (Ours)
Timer-28B 2024b
Time--Mli 2025
Moirai 2024
Moment
TimestM 2024
Chronos
2024b
2024
2024
UTSD+
O(100))
13 Public
Pre trained
UTSD
UTSD
Time:
LOTSA
Pile
LOTSA
Timepoints
Datasets
0.363
0.358
0.396
0.359
0.413
0.362
0.416
0.393
0.349
0.381
0.674
0.414
0.441
ETTh2
0,276
0.289
0.303
0.287
0.308
0.292
0.296
0.330
0.320
ETTml
0,275
0.300
0.321
0.321
0.321
0.355
0.420
0.281
0.380
0.670
0.361
0.457
ETTm2
0.161
0.167
0.187
0.170
0.176
0.176
0.247
0.198
0.211
0.257
0.202
0.197
BCL
0.123
0.136
0.131
0.134
0.153
0.132
0.147
0.153
0.744
Traffic
0.338
0.351
0.371
0.352
0.419
0.361
0.414
0.399
1.293
Weather
0.144
0.154
0.156
0.151
0.168
0.151
0.243
0.157
0.199
0.255
PEMS03
0.103
0.118
0.116
0.116
0.226
0.125
PEM504
0.086
0,107
0.195
0.120
0.542
0.135
52

---

Table 34: Performance with various embedding size settings. In main experiments, we d = 16 without specific statements.
Emb. size
8
16
64
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.133
0.223
0.132
0.221
0.132
0.221
0.131
0.221
0.131
0.221
0.131
0.221
192
0.152
0.240
0.153
0.241
0.152
0.240
0.150
0.238
0.153
0.241
0.152
0.240
ECL
336
0.166
0.256
0,169
0.257
0.165
0.255
0.165
0.254
0.165
0.255
0.168
0.258
720
0,188
0.277
0.198
0.282
0.207
0.290
0.191
0.279
0.193
0.280
0,200
0.286
Avg
0.159
0.249
0.163
0.250
0.164
0.252
0.159
0,248
0.161
0.249
0.163
0.251
96
0.398
0.230
0.411
0.227
0.403
0.226
0.398
0.226
0.402
0.226
0.404
0.226
Traftic
192
0.425
0.243
0.433
0.241
0.443
0.241
0.439
0.241
0.436
0.241
0,435
0.242
336
0,450
0.251
0,462
0.250
0.465
0.249
0.464
0.250
0.460
0.251
0.459
0.254
720
0.496
0.269
0.518
0.269
0,520
0.270
0.502
0.270
0.507
0.274
0,515
0.275
Avg
0.442
0.248
0.456
0.247
0.458
0.246
0.451
0.247
0.451
0.248
0.453
0.249
Solar- Energy
96
0.180
0.194
0.179
0.193
0.180
0.191
0.179
0.191
0.177
0.190
0.182
0.192
192
0.212
0.216
0.211
0.214
0.209
0.213
0.209
0.213
0.212
0.213
0.211
0.214
336
0.234
0.233
0.232
0.230
0.230
0.230
0.231
0,229
0.231
0.230
0,233
0.231
720
0.244
0.238
0.241
0.236
0.241
0.235
0.241
0.236
0.242
0.236
0.243
0.238
Avg
0.217
0.220
0.216
0.218
0.215
0.217
0.215
0.217
0.215
0.217
0,217
0.219
0.152
0.189
0.158
0.192
0.150
0.187
0.152
0.189
0.152
0.190
0.157
0.194
Weather
192
0.203
0.238
0.200
0.234
0.207
0.242
0.207
0.243
0.202
0.238
0.205
0.239
336
0.261
0.282
0.259
0,280
0.259
0.280
0.261
0.282
0.263
0.282
0.269
0.287
720
0.342
0.337
0.343
0.338
0.343
0.337
0.344
0.337
0.350
0.339
0.344
0.336
Avg
0,240
0.261
0.240
0.261
0.240
0.261
0.241
0.262
0.242
0.262
0.244
0.264
0.487
0.355
0.545
0,380
0,500
0.353
0.468
0.349
0.515
0.362
0,486
0.352
0.915
0.515
1,021
0.533
0.890
0.506
0.923
0.516
0.914
0.515
1.059
0.558
1,301
0.662
1.363
0.666
1.376
0.667
1.289
0.655
1.265
0.644
1.264
0.645
12
1.638
0.783
1.693
0.788
1,810
0.799
1.698
0.791
1.619
0.769
1.747
0.773
Avg
1.085
0.579
1.155
0.592
1.144
0.581
1.094
0.578
1.078
0.572
1.139
0.582
0.060
0.159
0.060
0.159
0.060
0.159
0.060
0.1.5
0.061
0.160
0.059
0.158
PEMS03
0.077
0.179
0.078
0.179
0.078
0.180
0.078
0.179
0.077
0.179
0.077
0.179
48
0.107
0,213
0.106
0.211
0.106
0.211
0.104
0.210
0.106
0.211
0.105
0.210
96
0.146
0.253
0.143
0.248
0.141
0.248
0.140
0.247
0.140
0.247
0.143
0.248
Avg
0.097
0
0.097
0.199
0.096
0.200
0.095
0.199
0.096
0.199
0.096
0.199
1s Count
9
4
8
9
12
12
14
12
4
53

---

Table 35: Forecasting performance on the scale-free metrics: Coefficient of Determination (R²2), Pearson Correlation Coefficient (r), and MASE. The symbols & and indicate that lower and higher values are better, respectively. The best and second-best best results are highlighted in bold and underlined, respectively. The lookback length T is uniformly set as 96.
OLinear (Ours)
Leddam
CARD
Trans.
TimeMixer
Model
2024
2024b
2024a
2024a
Metric
R² (t) r (f) MASE (1) R² (?) (t) MASE (4) R² (t)
(7)
MASE (4) R² (f) (f) MASE (U)
(t)
(t) MASE (U)
96
0.640
0.916
0.902
0.527
0.911
0.964
0.533
0.905
0.975
0.550
0.907
0.990
0.456
0.897
1.022
192
0.638
0.908
0,983
0.593
0.902
1,041
0.547
0.899
1.028
0.590
0,900
1.057
0.575
0.893
1.076
336
0.710
0.901
1.061
0.670
0.893
1,134
0.678
0.891
1.113
0.678
0.893
1.133
0.640
0,886
1.155
720
0.701
0.889
1.174
0.661
0.882
1,251
0.653
0.877
1,273
0.638
0.872
1.315
0.634
0,871
1.314
Avg
0.903
1.098
1,997
0.614
1.124
0.887
96
0.737
0.899
0.729
0.681
0.885
0.909
0.690
0.882
0.855
0.690
0.889
0.860
0.640
0,888
0.979
192
0,719
0.884
0,756
0.666
0.865
0.949
0.683
0.869
0.856
0.689
0.877
0.879
0.660
0.862
0.959
336
0.730
0.875
0.758
0.707
0.867
0.897
0.702
0.862
0.843
0.713
0.867
0.883
0.685
0.852
0.960
720
0.703
0.860
0.815
0.687
0.853
0.947
0.679
0.848
0.894
0.693
0.857
0.913
0.672
0.842
0.986
Aug
0.722
0.879
0.764
0.685
0.868
0.926
0.689
0.865
0.862
0.696
0.873
0.884
0.664
0.861
0.971
FilterNet
FITS
DJ Minear
PatchTST
TimesNet
Model
2024a
2024
2023
2023
20236
Metric
It" (1) (t) MASE (L) R²3
(t)
MASE (4) nt3 (t)
MASE (4) St (t) (t) MASE Re (1)
(t) MASE (L)
96
0,537
0.905
0.996
0.352
0.883
1.144
0.120
0.870
1.300
0.426
0.890
1.119
0.405
0.892
1.118
192
0.576
0.898
1.052
0.446
0.880
1,153
0.215
0.867
1.312
0.509
0.887
1.140
0.503
0.885
1.195
336
0.670
0.891
1.139
0.588
0.873
1,229
0.247
0.858
1,387
0.604
0.880
1.216
0.592
0.875
1.262
1.376
1,536
1.371
0.864
Avg
0.603
0.892
1.129
0.495
0.873
1,225
0.182
0.860
1,384
0.536
0.880
1.211
0.526
0,879
1.241
96
0.637
0.876
0.987
0.458
0.811
1,284
0.441
0.811
1.346
0.668
0.880
0.903
0.625
0.877
1.023
192
0.665
0.867
0,950
0.542
0.814
1,151
0.551
0.815
1.189
0.677
0.871
0.887
0.635
0.867
1.009
336
0.696
0.861
0.582
1.119
0.582
0.810
1.167
0.700
0.863
0.874
0.654
0,852
1.079
720
0.678
0.847
0,982
0.568
0.794
1.169
0.564
0.794
1.224
0
0.850
0.925
0.661
0,850
1.047
Avg.
0.669
0.862
0.962
0.537
0.807
1.181
0.535
0.808
1.232
0.682
0.866
0.897
0.644
0.862
1.040
54

---

Table 36: Forecasting performance when orthogonal matrices are computed from a subset of the training data "Q_ratio') ) while model training still uses the full trainset.
ratio
100%
80%
60%
40%
20%
10%
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.131
0.221
0.131
0.221
0.131
0.221
0.131
0.221
0.130
0.221
0.131
0.221
192
0.150
0.238
0.155
0.243
0.152
0.239
0.153
0.240
0.151
0.240
0.152
0.240
336
0.165
0.254
0.169
0.257
0.164
0.253
0.166
0.255
0.165
0.254
0.166
0.255
ECL
720
0.191
0.279
0.193
0.284
0.188
0.277
0.193
0.284
0.193
0.280
0.189
0.278
Avg
0.159
0.248
0.162
0.251
0.159
0.247
0.161
0,250
0.160
0.248
0.159
0.248
96
0.398
0.226
0.403
0.226
0.403
0.227
0.401
0.226
0.404
0.226
0.402
0.226
192
0.439
0.241
0.436
0.240
0.434
0.240
0.432
0.240
0.434
0.240
0.431
0.240
336
0.464
0.250
0.461
0.250
0,462
0.250
0.463
0.250
0.459
0.250
0.465
0.250
Traffic
720
0.502
0.270
0.499
0.271
0.499
0.271
0,501
0,271
0,502
0,271
0,512
0,271
Avg
0.451
0.247
0.450
0.247
0.449
0.247
0.449
0.247
0.450
0.247
0.453
0.247
96
0.153
0.190
0.150
0.188
0.153
0.191
0.153
0.190
0.154
0.191
0.149
0.187
192
0.200
0.235
0.202
0.237
0.204
0.239
0.203
0,238
0,202
0.237
0.203
0,239
336
0.258
0.280
0,260
0.280
0,260
0.281
0.264
0.284
0,260
0.281
0.259
0,281
Weather
720
0.337
0.333
0.339
0.333
0.338
0.332
0.337
0.332
0.335
0.330
0.339
0.332
Avg
0.237
0.260
0.238
0.259
0.239
0.261
0.239
0.261
0.238
0.260
0.237
0.260
12
0.060
0.159
0.060
0.159
0.060
0.159
0.060
0.159
0.061
0.160
0.060
0.159
24
0.078
0.179
0.077
0.179
0.077
0.180
0.077
0.179
0.077
0.180
0.077
0.180
0.104
0.210
0.105
0.210
0.104
0.210
0.104
0.210
0.104
0.210
0.105
0,210
PEMSU3
96
0.140
0.247
0.141
0.248
0.142
0.248
0.141
0.248
0.143
0.249
0.140
0.249
Avg
0.095
0.199
0.096
0.199
0.096
0.199
0.095
0.199
0.096
0.199
0.095
0.199
0.036
0.092
0.036
0.092
0.036
0.092
0.036
0.093
0.036
0.093
0.036
0.093
0.049
0.117
0.049
0.117
0.049
0.117
0.049
0.117
0.049
0.117
0.049
0.118
9
0.062
0.137
0.062
0.137
0.062
0.137
0.062
0.1
0.062
0.137
0.062
0.137
NASDAQ
12
0.073
0.154
0.073
0.154
0.073
0.154
0.073
0.154
0.073
0.154
0.073
0.154
Avg
0.055
0.125
0.055
0.125
0.055
0.125
0.055
0.125
0.055
0.125
0.055
0.125
24
0.320
0.302
0.319
0.302
0.319
0.301
0.319
0.302
0.320
0.302
0.320
0.302
36
0.334
0.315
0.334
0.315
0.334
0.315
0.334
0.315
0.334
0.315
0.334
0.316
48
0.347
0.327
0.347
0.327
0.347
0.327
0.347
0.327
0.347
0.327
0.347
0.328
CarSales
60
0.358
0.337
0.358
0.337
0.358
0.337
0.358
0.337
0.358
0.337
0.358
0.337
Avg
0.340
0.320
0.339
0.320
0.339
0.320
0.340
0.320
0.340
0.320
0.340
0.320
55

---

Table 37: Ablation study on transformation functions and normalization methods in NormLin. For S/ and S2, the prediction lengths are {3,6, 9, 12) and {24, 36, 48, 60), respectively.
Fun.
Norm
Hoor.
ECL
Traflic
Solar
PEMS03
Weather
NASDAQ (SI)
ILI (S2)
111
0.131
0.221
0.398
0.226
0.179
0.191
0.060
0.159
0.153
0.190
0,036
0.092
1.737
0.800
H2
0.150
0.238
0,439
0.241
0.209
0.213
0,078
0.179
0.200
0.235
0.049
0.117
1.714
0.795
H3
0.165
0.254
0.464
0.250
0.231
0.229
0.104
0.210
0.258
0.280
0.062
0.137
1.821
0.804
Softplu
1.1
H4
0.191
0.279
0.502
0.270
0.241
0.236
0.140
0.247
0.337
0.333
0.073
0.154
1,785
0.810
Avg
0.159
0.248
0.451
0.247
0.215
0.217
0.095
0.199
0.237
0.260
0.055
0.125
1,764
0.802
HI
0.140
0.229
0.455
0,233
0.182
0.193
0.062
0.161
0.154
0.191
0.036
0.093
1,783
0.807
H2
0.157
0.245
0.486
0,246
0.216
0.215
0.078
0.179
0.203
0.240
0.049
0.117
1.844
0.810
113
0.170
0.260
0.506
0.255
0.239
0.233
0.103
0.208
0.267
0.285
0.061
0.137
1,909
0.826
Identity
1.1
114
0,200
0.287
0.551
0,276
0.506
0.255
0.143
0,250
0.350
0.339
0.073
0.154
1.841
0.830
Avg
0.167
0.255
0.499
0,252
0.285
0.224
0.096
0.199
0.243
0.264
0.055
0.125
1.844
0.818
HI
0.131
0.221
0.398
0.226
0.179
0.192
0.061
0.160
0.150
0.187
0.036
0.092
1.707
0.800
H2
0.155
0.242
0.441
0.241
0.209
0.213
0.077
0.179
0.202
0.237
0.049
0.117
1.886
0.828
113
0.167
0.255
0.467
0.250
0.232
0.229
0.107
0.211
0.259
0.280
0.062
0.138
1,820
0.803
Softmax
L1
H4
0.199
0,286
0.504
0.270
0.241
0.236
0.139
0.247
0.335
0.332
0.073
0.154
1,795
0.814
Avg
0.163
0.251
0.452
0.247
0.215
0.217
0.096
0.199
0.236
0.259
0.055
0.125
1.802
0.811
H1
0.131
0.221
0.398
0.226
0.178
0.191
0.061
0.159
0.152
0.189
0.035
0.093
1,742
0.799
H2
0.150
0.238
0.241
0.209
0.213
0.077
0.179
0.203
0.238
0.049
0.117
1.719
0.797
Sigmoid
L1
H3
0.167
0.256
0.250
0.231
0.229
0.106
0.211
0.262
0.062
0.137
1,826
0.804
H4
0.199
0.285
0.500
0.270
0.241
0.236
0.139
0.248
0.338
0.332
0.073
0.154
1,780
0.808
Avg
0.161
0.250
0.450
0.247
0.215
0.217
0.096
0.199
0.239
0.260
0.055
0.125
1,767
0.802
111
0.132
0.222
0.398
0.226
0.179
0.191
0.061
0.160
0.153
0.189
0
0.092
1,711
0.799
H2
0.153
0.241
0.443
0.241
0.210
0.213
0,078
0.179
0.202
0,238
0.049
0.117
1,721
0.795
H3
0,170
0.259
0.467
0,250
0.231
0.229
0,106
0.211
0.264
0,285
0.062
0.138
1.816
0.804
ReLU
L1
H4
0.197
0.285
0.507
0.271
0.241
0.236
0.143
0.248
0.338
0.335
0.073
0.154
1,790
0.812
Avg
0.163
0.251
0.454
0.247
0.215
0.217
0.097
0.199
0.239
0.262
0.055
0.125
1.759
0.802
111
0.192
0.285
0.613
0.313
0.234
0.244
0.095
0.199
0.151
0.188
0.036
0.093
1,928
0.852
H2
0.194
0,288
0.640
0,325
0,280
0.267
0.121
0.221
0.203
0.238
0.050
0.119
1,929
0.837
113
0,202
0.297
0.653
0.329
0.323
0.288
0.174
0.265
0.259
0.281
0.063
0.139
1.845
0.806
Sofplus
1.2
H4
0,236
0.322
0.683
0.344
0.337
0.292
0,264
0.331
0.339
0.335
0.073
0.154
1,794
0.813
Avg 0.206
0.298
0.647
0.328
0.293
0.273
0.163
0.254
0.238
0.266
0.055
0.126
1,874
0.827
HI 0.139
0.230
0.456
0.238
0.187
0.198
0.063
0.163
0.151
0.189
0.036
0.092
1,718
0.795
H2
0.159
0.249
0.516
0.251
0.221
0.219
0.080
0.184
0.201
0,238
0.050
0.118
1.855
0.812
113
0.171
0.261
0.529
0.261
0.239
0.234
0.112
0.217
0.266
0.284
0.062
0.137
1,947
0.831
Identity
1.2
114
0.215
0.298
0.573
0.288
0.248
0.239
0.153
0.258
0.347
0.339
0.073
0.154
1,832
0.829
Avg
0.259
0.518
0.259
0.224
0.222
0.102
0.205
0.241
0.262
0.055
0.125
1.838
0.817
HI
0.183
0.278
0.612
0,312
0.233
0.242
0.096
0.199
0.153
0.190
0.036
0.093
1,876
0.837
H2
0.192
0.286
0.641
0.325
0,266
0.256
0.118
0.219
0.199
0.235
0.050
0.119
1.875
0.825
113
0.202
0.297
0.653
0.330
0.313
0.280
0.174
0.265
0.259
0.282
0.063
0.139
1.823
0.803
Softmax
L.2
H4
0.226
0.316
0.686
0.346
0.336
0.292
0.268
0.334
0.337
0.333
0.073
0.154
1.798
0.814
Avg
0.201
0.294
0.648
0.328
0.287
0.267
0.164
0.254
0.237
0.260
0.055
0.126
1.843
0.820
111
0.193
0.285
0.613
0,313
0.234
0.244
0.096
0.200
0.149
0.187
0.036
0.094
1.993
0.861
H2
0,206
0.296
0.640
0,324
0.279
0.267
0.119
0.220
0.204
0,239
0.050
0.119
1.762
0.807
Sigmoid
L2
H3
0,201
0.296
0.653
0.329
0.324
0.288
0,175
0.267
0.262
0,284
0.063
0.139
1.866
0.811
H4
0.256
0.334
0.684
0.345
0.336
0.291
0.265
0.332
0.337
0.333
0.073
0.155
1.797
0.813
Avg
0,214
0.302
0.647
0.328
0.293
0.272
0.164
0.255
0.238
0.261
0.056
0.127
1.854
0.823
111
0.181
0.277
0.612
0.312
0.231
0.241
0.095
0.197
0.153
0.190
0.036
0.093
1,860
0.838
H2
0.190
0.284
0.640
0.325
0.265
0.255
0.118
0.219
0.201
0.237
0.050
0.119
1.900
0.833
H3
0,200
0.295
0.653
0,329
0,309
0.278
0.176
0.268
0.260
0.282
0.063
0.139
1.803
0.800
ReLU
L2
H4
0.232
0.320
0.686
0.346
0.335
0.291
0.263
0.330
0.336
0.332
0.073
0.154
1,798
0.813
Avg
0.201
0.294
0.648
0.328
0.285
0.266
0.163
0.254
0.237
0.260
0.055
0.126
1,841
0.821
56

---

Table 38: Ablation study on pre- and post-linear layers in the NormLin module. 'Hor.' denotes prediction horizons. For SI and S2, the prediction horizons are {3, 6, 9, 12] and {24, 36, 48, 60). respectively.
ECL
Trallic
Solar
PEMS03
Weather NASDAQ (S1)
ILI
Wiki (SI)
ETTml
PreLin PostLin Hor.
MSE MAE MSE MAE MSE MAE MSE MAE MSE
MSE
MSE
MSE MAE MSE MAE
III 0.135 0,229 0.474 0.253 0,192 0.206 0.065 0.167 0.159 0.195 0.040
0.104
1.744 0.861 6,184 0.372 0.320 0.352
H2 0.153 0,246 0.492 0.264 0,225 0.228 0.085 0,191 0.207 0.241 0.053
0.126
1,970 0.891 6.471 0.387 0.373 0.384
0.168 0.264 0.273 0.247 0.244 0.125 0.233 0.285 0,065
2.153
0.892 0,403 0.401 0.400
X X
0.204 0.297 0.294 0.255 0.247 0.178 0.281
0.077
6.846 0,410 0.461 0.435
Avg 0.165 0,259 0.508 0.271 0.230 0.231 0.113 0,218 0.243
0.264
0.058
0.134
1,989
0.883
6.543 0.393 0.389 0.393
III 0.131 0,222 0.403 0.228 0.179 0.192 0.061 0.161 0,155 0.192 0.036
0.094
1,926
0.858 6.176 0.368 0,303 0,334
H2 0.151 0,240 0.434 0.242 0.210 0.213 0.077 0.180 0.205 0.240 0.049
0.118
1.796
0.827 6,447 0.384 0.356 0.364
H3 0.166 0,256 0.457 0.251 0.231 0.230 0.107 0.212 0.262 0,282 0.062
0.137
1.960
0,833 6.650 0.399 0.389 0.387
X
H4 0.211 0.298 0,500 0.272 0,243 0.236 0.146 0,253 0.352 0.340 0.073
0.155
1.799
0.811 6.839 0.406 0.452 0.426
Avg 0.165 0.254 0.448 0.248 U.216 0,218 0.098 0.201 0.243 0.263 0.055
0.126
1.870
0.832 6,528 0,389 0,375 0.378
HI 0.131 0,222 0.404 0.228 0.178 0.192 0,061 0.160 0.154 0.192 0.036
0.994
1.924
0.858 6.175 0.368 0,303 0.334
H2 0,155 0.243 0.434 0.242 0.211 0.214 0.077 0.180 0.204 0.239 0.049
0.118
1,796
0.828 6.448 0.385 0.356 0,364
H3 0.165 0,255 0.457 0.251 0.231 0.230 0.107 0.212 0.263 0,282 0.062
0.137
1,959
0,833 6,649 0.399 0.389 0,387
114 0.209 0.293 0.499 0.272 0,242 0.236 0.144 0,252 0.344 0,339 0.073
0.155
1.799
0.811 6,839 0.406 0.452 0.426
Avg 0.165 0.253 0.448 0.248 0.215 0.218 0.097 0.201 0.241 0.263 0.055
0.126
18869
0.832 6.528 0.389 0,375 0.378
0.131 0.221 0.226 U.179 0.191 0.060 0.159
0.036
0.092
1.737
6.161 0.368 0,302 0.334
H2 0.150 0.238 0.439 0.241 0.209 0.213 0.078 0.179 0.207
0.243 0.049
0.117
1,714
0.795 6.453 0.385 0.357 0,364
0.165 0.254 0.464 0.250 0.231 0.229 0.104 0,210 0.261 0,282
0.062
0.137
1.821
0.804 6.666 0.398 0,387 0.385
6 V
114 0.191 0,279 0.502 0.270 0.241 0.236 0.140 0,247 0.344 0.337 0.073
0.154
1.785
0.810 6.834 0,406 0,452 0.426
Avg 0.159 0.248
0.215 0.217 0.095 0.199 0.263
0.055
0.125
1.764
0.802
6.528 0.389 0.374 0.377
Table 39: Ablation study on Q and Qo 'Hor.' denotes prediction horizons. For SI and S2, the prediction lengths are (3.6, 9, 12} and (24, 36, 48, 60), respectively.
FCT.
Solar: Fnergy
PFMS03
PFMS08
NASDAQ (51)
(S2
COVID- 19 (S2)
Exchange
Q
+
Hor:
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAB
0.134
0.186
0.199
0.063
0.163
0.162
0.036
0.093
1.891
0.847
4.841
1.284
0.203
H2
0.156
0.245
0.221
0.220
0.080
0.184
0.091
0.183
0.050
0.119
1.889
0.844
7,828
1.757
0.173
0.295
H3
0.167
0.257
0.244
0.237
0.117
0.222
0.128
0.216
0.063
0.139
1.854
0.811
10.153
2,003
0.335
0.416
X
X
H14
0.194
0.282
0.256
0.244
0.165
0.266
0.231
0.262
0.074
0.156
1.802
0.816
12.603
2.232
0.846
0.692
Avg
0.163
0.252
0.227
0.225
0.106
0.209
0.130
0.206
0.056
0.127
1.859
0.829
8.856
1.819
0.401
HI
0.132
0.223
0,186
0.198
0.062
0.162
0,069
0,162
0.036
0.093
1.844
0.830
4,645
1,260
0,083
0,203
0.153
0.243
0.219
0.219
0.079
0.183
0.092
0.183
0.050
0.119
1.779
0.811
8.1
1.798
0.173
0.295
0.167
0.258
0.246
0.238
0.115
0.221
0.132
0.217
0.063
0.139
1.854
0.809
10.054
1.990
0.416
X
0.194
0.285
0.244
0.164
0.267
0.216
0.261
0.074
0.156
1.787
12,000
2.189
0.693
Avg
0.161
0.252
0.226
0.235
0.105
0.208
0.127
0,206
0.056
0.127
1.816
0.816
8.709
1.809
0.359
0.401
HI
0.131
0.221
0.179
0.191
0.061
0.159
0.068
0.159
0.036
0.092
1.897
0.847
4.957
1,330
0
0.200
H2
0.154
0.241
0.210
0.212
0.076
0.178
0.087
0.177
0.049
0.117
1.886
0.836
7.100
1.654
0.172
0.293
H3
0.165
0.254
0.230
0.229
0.105
0.211
0.121
0.204
0.062
0.137
1.809
0.802
10.224
2.009
0.352
0.414
x
114
0.188
0.279
0.241
0.235
0.142
0.249
0.182
0,238
0.073
0.154
1.784
0.811
11.779
2.160
0.837
0.688
Avg
0.159
0.244
0.215
0.217
0.096
0.199
0.114
0.194
0.055
0.125
1.844
0.824
8.515
1.788
0.355
0.399
111
0.131
0.221
0.179
0.191
0.060
0.159
0.068
0.159
0.036
0.092
1.737
0.800
4.474
1.180
0.082
0.200
H2
0.150
0.238
0.209
0.213
0.078
0.179
0.089
0.178
0.049
0.117
1.714
0.795
7.241
1.670
0.171
0.293
H3
0.165
0.254
0.231
0.229
0.104
0.210
0.123
0.204
0.062
0.137
1.821
0.804
10.076
1.985
0.331
0.414
H4
0.191
0.279
0.241
0.236
0.140
0.247
0.173
0.236
0.073
0.154
1.785
0.810
12.079
2. 2.182
0.837
0.688
Avg
0.159
0.248
0.215
0,217
0.095
0.199
0.113
0.194
0.055
0.125
1.764
0.802
8.467
1.754
0.355
0.399
57

---

Table 40: Forecasting performance when both training and testing use Q matrices Q and Q from different datasets. A B indicates that the Q matrices computed from dataset A are applied in experiments on dataset B.
Setting
ILI
Power-Il.
Wcb-sIII
Metric
MSE
MAE
MSE
MAE
MSE
MAE
24
1.737
0,800
1.846
0.830
1.887
0.842
36
1.714
0.795
1.721
0.801
1.743
0.807
48
1.821
0.804
1.849
0.810
1.875
0.817
60
1.785
0.810
1.824
0.814
1.815
0.813
Avg
1.764
0.802
1.810
1.830
0.820
Setting
ILw-:Power
Power Power
New-PPower
Metric
MSE
MAR
MSF
MAF
MSF
MAE
24
1.391
0.895
1.343
0.870
1.393
0.889
36
1.457
0.902
1.445
1.493
0.915
48
1.612
0.972
1.559
0.946
1.680
0.997
60
1.035
1.602
1.773
Avg
1.548
0.951
1.487
0.922
1.585
0.962
Setting
Web
Power Web
Web
Metric
MSE
MAB
MSE
МАE
MSE
MAE
24
0.208
0.319
0.185
0.306
0.186
0.306
36
0.285
0.369
0.289
0.364
0.272
0.356
48
0.377
0.403
0,400
0.427
0.365
0.391
60
0.480
0.476
0.467
0.466
0.486
0.481
Avg.
0.337
0.391
0.335
0.391
0.327
0.383
58

---

Table 41: Training and inference resource footprint for OLinear and baseline forecasters. 'T.T." 'T.M.', 'I.T.' and 'IM.. denote training time, training GPU memory usage, inference time, and inference GPU memory usage, respectively. All experiments use a lookback/prediction of 96 and a batch size of 16. To ensure a fair comparison, the number of layers (or blocks) is fixed at 2, except for DLinear, which employs only linear layer. 'OL' refers to OLinear. 'OOM' indicates that the experiment out of memory
24 GB GPU.
OL
OL-C
Leddam
CARD
Fredformer
Trans. 2024g
TimeMix 2025a
TimeMix. 2024a
Dlinear 2023
TimesNet 2023b
PatchTST
FS 2023
Model
(Ours)
2024
2024b
2024
2023
Params/(M)
4.52
4.52
8.55
0.03
8.59
4.83
1.19
0.08
0.02
299.94
3.76
0.42
FLOPs(M)
33.74
33.74
111.54
135.01
33.99
1396.90
0.13
289708
272.20
2.94
T.T (mns/tee)
8.09
7,26
25.02
29.10
26.48
10.11
74.74
27.99
1.42
501.98
8,37
3.63
T.M.GB)
L.T.(ns/tter
1.19
1.09
2.62
4.05
3.73
1.72
28.15
3.86
0.21
155.97
1.38
0.49
1.M. (MB)
156.41
156.41
51.04
11.02
78.04
156.00
149.58
18.06
8,48
1396.24
51.83
14.47
Params(M)
4.79
4.58
8.56
1.39
12.12
4.83
1.19
0.12
0.02
300.58
3.76
0.42
FLOPs(G)
5.32
5.55
1.87
293.98
12.48
T.T. (msfiter)
7.75
6.94
65.19
78.25
26.59
10.66
106.06
57.09
1.37
509.17
65.83
3.78
(.M(GB)
0.45
0.42
1.44
5.08
1.24
0.70
1.02
4.24
0.04
5.82
3.02
0,30
L.T.msfiter)
1.M. (GB)
0.17
0.17
0.49
0.63
0.26
0.23
0.36
0.82
0.02
1.41
0.93
0.23
Params(M)
1.74
1.74
8.56
1.39
8.59
4.83
1.19
0.08
0.02
299 94
3.76
0.42
Exchange
FLOPs(M)
16.27
16.27
127.49
114.83
135.01
38.88
1595.72
12.25
0.15
287909
311.08
3.36
TT. (ms/iter)
7,38
24.81
35.57
28.56
10.14
27,46
1.47
505.39
9.75
T.M.BB
0.17
0.17
0,18
0.17
0.24
0.21
0.28
0.06
0,02
5.80
0.15
0.03
(.T.(msfitrr
1.18
1.89
2.62
6.02
6,86
1,70
28.46
3.81
0.21
156.33
2.47
0.48
LM. (GB)
0.05
0.08
0.16
0.05
Params/MM
6.17
4.69
8.56
0.98
11.09
4,83
4.73
0.12
0.02
301 69
3.76
0.42
FLOPs(G)
4.91
4.91
15.24
10.47
14.78
6.45
588.47
4,66
0.02
288.72
33.52
0.36
T.T. (sfiter)
0.02
0.02
0.19
0.18
0.10
0.04
OOM
0,17
0.001
0.50
0.17
0.01
TM.GBB
3.74
2.95
I.T.(ns/iter)
5.71
5.67
45.68
61.21
31.60
13:45
1610.66
67.21
0,20
15.64
55.10
3.48
L.M. (GB)
0.43
0.43
1.25
1.69
1.84
1.27
9.35
2.20
0.04
1.42
2.44
0.59
Params(M)
4.52
4,52
8.56
0.03
0.50
4.83
2.37
0.10
0.02
299.97
3.76
0.42
FLOPs(G)
0.004
0.01
0.000
291.21
Weather
T.T. (msitte))
7.47
6.90
27.55
28.69
37.85
9.45
102.04
33.30
1.38
505.88
9.89
3.53
(MM)
0.21
0.21
0.21
0.05
0.05
0.22
0.86
0.16
0.02
5.80
0.27
0.04
L.(ensfiter
3.15
L.M. (MB)
39.26
39.26
70.06
14.45
18.28
34.88
316.49
36,03
9.06
1397.18
90.78
24.37
Params(M)
4.58
4.54
8.56
1.39
4.61
4.83
2.37
0.12
0.02
300.21
3.76
0.42
FLOPs(G)
0.68
0.68
2.22
2.05
1.20
0.72
46.81
0.74
0.003
290.64
5.33
0.06
T.T. (ms/iter)
31.77
9.97
35.90
1.44
506.58
Solar
T.M.GB)
0.30
0.30
0.68
2.22
0.61
0.37
5.11
1.83
0.03
5.81
1.34
0.15
(.T((ffiter)
1.20
1.08
7.65
10.44
5.16
1.96
189.20
8.54
0.20
156.66
8.49
0.48
LMM (GB)
0.21
0.23
0.28
0.16
0.18
0.01
0.42
Params/MM
4.75
3.19
8.56
1.39
13.44
4.83
2.38
0.12
0.02
301.73
3.76
0.42
FLOPs(G)
3.67
3.67
15.65
16.10
20.53
6.66
301.92
4.77
0.02
290.97
34.34
0.37
PEMSOT
T.T. (sfiter)
0.01
0.01
0.19
0.28
0.11
0.04
OOM
0.17
0.002
0.51
0.19
0.01
TMMIBB
LT((msfiter)
1.21
1.10
47.06
94.01
36,54
13.92
1370.91
68.96
0.20
156.78
56.49
0.50
LM. (GB)
0.43
0.43
1.28
1.74
2.03
1.32
12.37
2.25
0.04
1.43
2.50
0.61
59

---

Table 42: Efficiency comparison of Leddam, iTrransfommer, PatchTST, and Fredformer before and after replacing multi-head self-attention with the in module. In Leddam, only the cross-channel attention module is updated. Metrics include number of learnable parameters (Params), FLOPs, training time (T.T.), training memory (T.M.), inference time (I.T.), and inference memory (I.M.). All experiments use a llokbackkprediccteiec
length of 96 and a batch size of 16. Notably, in its official implementation, Fredformer employs Nystromformer [iiong et al. 202T an approximate yet more efficient self-attend attentonn mechanism for the ETTml, ECL., and Weather datasets.
Leddam
iTrans. 2024a
PatchTST
Fredformer
Model
2024
2023
2024
Attn.
NormLin
Attn.
NomnLin
Attn.
NormLin
Attn.
NornLin
Params(M)
8.55
7.51
4.83
2.21
3.76
2.71
8.59
8.39
FLOPs(M)
111.54
104.15
33.99
15.51
272.20
182.20
135.01
79.10
T.T. (msfiter)
25.02
23.67
10.11
7.10
8.37
6.82
26.48
10.06
ETTml
(MB)
0.18
0.16
0.21
0.17
0.14
0.12
0.24
0.18
2.62
2.50
1.72
1.25
1.38
1.14
3.73
1.36
L.M. (MB)
51.04
47.03
156.00
146.20
51.83
50.45
78.04
52.44
Params(M)
8.56
7.71
4.83
2.41
3.76
2.71
12.12
12.92
FLOPs(G)
5.32
4.88
1.87
12.48
8.35
5.55
6.04
T.T. (ms/iter)
65.19
64,93
10.66
7.39
65.83
50.82
26.59
21.03
ECL
(GB)
1.44
1.41
0.70
0.41
3.02
2.78
1.24
1.27
(.T.ms/iter)
14.78
14.52
3.46
1.01
20.52
14.08
7.08
6.71
I.M. (GB)
0.49
0.48
0.23
0.10
0.93
0.99
0.26
0.26
Params(M)
8.56
7.52
4.83
2.21
3.76
2.71
0.50
0.48
FLOPs(()
0.34
0.31
0.10
0.05
0.82
0.55
0.01
0.003
(msfiter)
27.55
27.21
9.45
7.49
7.47
37.85
28.19
Weather
(.M.
0.21
0.19
0.22
0.05
0.03
(ms/iter)
3.15
3.08
1.66
0.99
2.96
1.12
7.19
3.55
L.M. (MB)
70.06
66.05
34,88
25.51
90.78
92.69
18.3
13.77
Params(M)
8.56
7.54
4.83
2.24
3.76
2.71
4.61
4.32
FLOPs(G)
2.22
2.06
0.72
0.32
5.33
3.56
1.20
0.91
T.T. (ms/iter)
48.64
48.12
9.97
7.40
28.89
22.31
31.77
12.32
Solar-Energy
(.M.GG))
0.68
0.67
0.37
0.26
0.61
0.46
(ms/ite))
7.65
7.59
1.96
5.16
2.89
LM. (GB)
0.23
0.22
0.18
0.18
0.42
0.44
0.16
0.09
Params(M)
8.56
9.07
4.83
3.77
2.71
13.44
15.29
FLOPs(G)
15.65
13.93
6.66
2.75
34.34
22.96
20.53
17.57
T.T. (sfiter)
0.19
0.18
0.04
0.01
0.19
0.14
0.11
0.06
PEMS07
(GM.
3.85
3.74
3.07
0.82
9.18
7.57
6.42
3.16
(Ts(ftteer)
47.06
45.16
13.92
3.02
56.49
38.63
36.54
19.16
L.M. (GB)
1.28
1.28
1.32
2.70
2.03
60


Reviews:
[{"review_id": "6Vo2LFE9Bz", "reviewer": "Reviewer_rQjb", "summary": "This paper tackles entangled temporal dependencies in multivariate time series forecasting through two main contributions: a data-adaptive orthogonal transformation based on eigenvalue decomposition of Pearson correlation matrices, and a computationally efficient normalized linear layer (NormLin) that serves as an alternative to self-attention mechanisms. The proposed method leverages dataset-specific statistical properties to decorrelate temporal features before applying linear forecasting models. The comprehensive evaluation across multiple datasets and and tasks in this paper demonstrates consistent performance gains over strong baselines, with both components showing modularity as plug-in enhancements for existing architectures.", "questions": "1. How does the method perform on highly non-stationary time series where the correlation structure changes significantly over time? Have you considered adaptive or online updates to the orthogonal matrices? \n\n2. You mention that OrthoTrans improves the rank of attention matrices in Transformer models. Can you provide more detailed analysis or proof of this phenomenon? What is the relationship between decorrelation and representation diversity?\n\n3. Have you compared against other adaptive bases beyond the eigenvalue decomposition, such as those learned through neural networks or other unsupervised methods?\n\n4. The orthogonal matrices Qi and Qo are computed from the training data. The paper only briefly mentions robustness to limited training data in the appendix but doesn't thoroughly investigate how performance degrades with smaller training sets or distribution shifts.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\nThis paper presents a well-executed combination of theoretical innovation and empirical rigor. The core contribution using eigenvalue decomposition of temporal Pearson correlation matrices for orthogonal transformation, provides an elegant theoretical foundation with clear justification. The work demonstrates state-of-the-art performance across multiple datasets and tasks, with both components serving as effective plug-in modules that consistently improve various baseline models. I am particularly impressed by the comprehensive experimental evaluation, which includes extensive ablation studies, and evaluation across carious tasks with different prediction horizons.\n\n\n**Weaknesses:**\n\nSee questions below", "was_revised": true, "final_justification": "The authors have provided comprehensive response to my questions. I choose to maintain my current score.", "rebuttal": "Thank you for your constructive response. We sincerely appreciate the reviewer’s recognition of OLinear as \"**a well-executed combination of theoretical innovation and empirical rigor**\". Below are our responses to the concerns:\n\n> Q1: On non-stationary time series\n- Financial data (e.g., stock prices and exchange rates) are widely recognized as non-stationary time series. As shown in Tables 2 and 3 of the paper, OLinear consistently achieves state-of-the-art performance on the real-world non-stationary time series datasets: Exchange, NASDAQ, SP500, and DowJones.\n- Regarding online updates of the Q matrices, we further conduct experiments where Q matrices are replaced by those computed on the validation set. After retraining with the updated Q matrices (using the same trainset), the model exhibits similar forecasting performance (as shown below). This phenomenon implies that the Q matrices obtained by OrthoTrans with the trainset can **generalize well to the full dataset**. Further discussion on online learning could be a **future direction** of this work.\n\n| **Dataset** | **Hor.** | **Data in paper** |   | **Val_Q and Retrain** |    |\n|-------------|----------|--------------------------------|--------|-----------------------|--------|\n|             |          | MSE                            | MAE    | MSE                   | MAE    |\n| Exchange    | 96       | 0.082                          | 0.200  | 0.082                 | 0.200  |\n|             | 192      | 0.171                          | 0.293  | 0.171                 | 0.293  |\n|             | 336      | 0.331                          | 0.414  | 0.331                 | 0.414  |\n|             | 720      | 0.837                          | 0.688  | 0.836                 | 0.688  |\n|             | Avg      | 0.355                          | 0.399  | 0.355                 | 0.399  |\n| NASDAQ      | 3        | 0.036                          | 0.092  | 0.036                 | 0.092  |\n|             | 6        | 0.049                          | 0.117  | 0.049                 | 0.117  |\n|             | 9        | 0.062                          | 0.137  | 0.062                 | 0.137  |\n|             | 12       | 0.073                          | 0.154  | 0.073                 | 0.154  |\n|             | Avg      | 0.055                          | 0.125  | 0.055                 | 0.125  |\n| SP500       | 3        | 0.035                          | 0.126  | 0.035                 | 0.127  |\n|             | 6        | 0.053                          | 0.158  | 0.054                 | 0.158  |\n|             | 9        | 0.070                          | 0.181  | 0.070                 | 0.181  |\n|             | 12       | 0.088                          | 0.204  | 0.088                 | 0.204  |\n|             | Avg      | 0.061                          | 0.167  | 0.061                 | 0.167  |\n\n\n\n> Q2: On the rank improvement of attention matrices \n- The observation that OrthoTrans improves the rank of attention matrices is interesting and partly explains why OrthoTrans, as a plug-in, consistently enhances Transformer-based forecasters (see Table 5 of the paper). A potential reason behind this phenomenon is that **OrthoTrans can mitigate the intrinsic low-rank property** of time series data. The empirical results are presented below.\n\n-  | **Dataset** | **Identity** | **OrthoTrans** | **DFT** | **Wavelet** |\n|-------------|--------------|----------------|---------|-------------|\n| ECL         | 0.162        | **0.446**          | 0.332   | 0.381       |\n| Traffic     | 0.002        | **0.568**          | 0.177   | 0.049       |\n| Weather     | 0.012        | **0.572**          | 0.352   | 0.292       |\n| Solar       | 0.007        | **0.481**          | 0.032   | 0.023       |\n| PEMS03      | 0.241        | **0.450**          | 0.356   | 0.395       |\n\n\n- Following TimeBase (ICML 2025), we compute the **median singular values** of the correlation matrix among non-overlapping time series patches (patch length = 24, number of patches = 30); see Figure 1 in TimeBase for more details. The results are presented above. As shown, OrthoTrans yields **larger singular values** than other transformation bases (with “Identity” denoting no transformation). In other words, OrthoTrans alleviates low-rank tendencies [1,2] and **enhances data diversity**.\n\n\n> Q3: On adaptive bases \n\n- We have considered learnable Q matrices for potential performance improvements. The most straightforward approach is adding a learnable matrix to Q. More advanced alternatives include using a learnable vector $\\mathbf{v}$ with a Householder transformation to ensure orthogonality: \n$$\\mathbf{Q}'=\\mathbf{Q} \\left ( \\mathbf{I} -2\\frac{\\mathbf{v}\\mathbf{v}^T}{\\left \\| \\mathbf{v}  \\right \\|^2_2 }  \\right ), $$\nor applying the Cayley transform:\n$$\\mathbf{Q}'= \\left ( \\mathbf{I}+\\mathbf{A} +\\mathbf{V}   \\right )^{-1} \\left ( \\mathbf{I}-\\mathbf{A} -\\mathbf{V}   \\right ),$$\nwhere $\\mathbf{V}$ is the learnable skew-symmetric matrix and $\\mathbf{A}=\\left ( \\mathbf{I} -\\mathbf{Q}  \\right ) \\left ( \\mathbf{I} + \\mathbf{Q}  \\right )^{-1}$. \nWe also explored an input-adaptive version: $$\\mathbf{Q}'=\\mathbf{Q}+ \\alpha \\cdot \\mathrm{Linear_1}(\\mathbf{x}) \\mathrm{Linear_2}(\\mathbf{x})^\\mathrm{T} , $$\nwhere $\\mathbf{x}$ is the input series and $\\alpha$ is the learnable parameter. The results are reported below:\n\n| **Dataset** | **Horizon** | **OLinear** |   | **Plus Delta** |    | **Householder** |    | **Cayley** |    | **Adapt_Q** |   |\n|-------------|-------------|-------------|--------|----------------|--------|-----------------|--------|------------|--------|-------------|--------|\n|             |             | MSE         | MAE    | MSE            | MAE    | MSE             | MAE    | MSE        | MAE    | MSE         | MAE    |\n| ECL         | 96          | 0.131       | 0.221  | 0.131          | 0.221  | 0.131           | 0.221  | 0.130      | 0.221  | 0.131       | 0.221  |\n|             | 192         | 0.150       | 0.238  | 0.152          | 0.240  | 0.153           | 0.241  | 0.153      | 0.241  | 0.153       | 0.241  |\n|             | 336         | 0.165       | 0.254  | 0.166          | 0.256  | 0.166           | 0.255  | 0.165      | 0.255  | 0.164       | 0.254  |\n|             | 720         | 0.191       | 0.279  | 0.188          | 0.277  | 0.193           | 0.282  | 0.188      | 0.278  | 0.192       | 0.281  |\n|             | Avg         | 0.159       | 0.248  | 0.159          | 0.248  | 0.161           | 0.250  | 0.159      | 0.249  | 0.160       | 0.249  |\n| Traffic     | 96          | 0.398       | 0.226  | 0.397          | 0.227  | 0.400           | 0.226  | 0.401      | 0.226  | 0.399       | 0.226  |\n|             | 192         | 0.439       | 0.241  | 0.436          | 0.241  | 0.435           | 0.240  | 0.437      | 0.241  | 0.438       | 0.240  |\n|             | 336         | 0.464       | 0.250  | 0.460          | 0.250  | 0.461           | 0.250  | 0.466      | 0.250  | 0.460       | 0.250  |\n|             | 720         | 0.502       | 0.270  | 0.510          | 0.271  | 0.500           | 0.271  | 0.505      | 0.271  | 0.507       | 0.271  |\n|             | Avg         | 0.451       | 0.247  | 0.451          | 0.247  | 0.449           | 0.247  | 0.452      | 0.247  | 0.451       | 0.247  |\n| Weather     | 96          | 0.153       | 0.190  | 0.149          | 0.187  | 0.152           | 0.189  | 0.151      | 0.189  | 0.151       | 0.189  |\n|             | 192         | 0.200       | 0.235  | 0.200          | 0.236  | 0.204           | 0.239  | 0.205      | 0.240  | 0.203       | 0.238  |\n|             | 336         | 0.258       | 0.280  | 0.266          | 0.285  | 0.263           | 0.284  | 0.263      | 0.284  | 0.260       | 0.282  |\n|             | 720         | 0.337       | 0.333  | 0.334          | 0.331  | 0.339           | 0.334  | 0.343      | 0.337  | 0.337       | 0.332  |\n|             | Avg         | 0.237       | 0.260  | 0.237          | 0.260  | 0.239           | 0.261  | 0.240      | 0.262  | 0.238       | 0.260  |\n\n- However, these attempts do not yield performance gains. These results highlight the importance of temporal decorrelation: modifications to the Q matrices **reintroduce correlations** into the output series (since Q is theoretically the optimal orthogonal matrix for temporal decorrelation). This further demonstrates the superiority of the proposed OrthoTrans scheme.\n\n> Q4: On smaller training sets or distribution shifts\n\n- We report performance with smaller training sets in **Appendix I.4**. As a linear-based forecaster, OLinear does not heavily rely on large training data and adapts well when the training size is reduced from 100% to 5% (see Figure 16 on Page 32 of the paper).\n\n- Distribution shifts are challenging for any deep forecasting model. Tables 2 and 3 in the paper show that OLinear performs competitively on real-world non-stationary datasets. In Table 28, OLinear also demonstrates strong **zero-shot** and generalization capabilities.\n\n\n**We hope these responses can fully address your concerns. Thank you again for your detailed feedback!**\n\n\n[1] TimeBase: The Power of Minimalism in Efficient Long-term Time Series Forecasting. ICML 2025.\n\n[2] Robust recovery of subspace structures by low-rank representation. TPAMI 2012."}, {"review_id": "14nXd7CnPX", "reviewer": "Reviewer_CEMa", "summary": "The paper introduces OLinear, a computationally efficient alternative to transformer-based models for multivariate time series forecasting. The architecture combines two key components that can also be used as plugins: OrthoTrans, which applies orthogonal transformations to decorrelate input features, and NormLin, a linear layer that aims to reproduce channel-wise attention. OLinear consists of a Cross-Series Learner (CSL) using NormLin layers for multivariate correlations and an Intra-Series Learner (ISL) with MLPs for individual series processing. Results show OLinear achieves competitive or superior performance while reducing computations.", "questions": "- On the asymmetric normalization: The cov_matrix / diag_vec normalization produces an asymmetric matrix that isn't a true correlation matrix. Is this an implementation error or deliberate choice? If intentional, what's the theoretical justification? Why is this implementation in a notebook? Maybe I didn't find the right file that saves the matrices.\n\n- On the decorrelation architecture: Have you explored architectures where each variable has its own $Q^{(j)}$ matrix to preserve variable-specific temporal structures? What information is lost by averaging correlations across all variables? Did you try the ablation studies with PCA on $XX^\\top$? What is the intuition? Did you try an approach with learnable $Q^{(j)}$? \n\n- What is the intuition behind the dimension extension? To what extent does it differ from the linear encoder? What does it bring? Why decorrelating after the dimension extension?\n\n- In the CSL: What is the purpose of the first linear layer, what is the intuition behind it? From my understanding, you want to reproduce the architecture of self-attention blocks, but then I don't see why we need the first layer. CSL:\n$$\\tilde{H} = \\text{LayerNorm}\\left(\\tilde{H} + \\text{Linear}\\left(\\text{NormLin}\\left(\\text{Linear}\\left(\\tilde{H}\\right)\\right)\\right)\\right)$$\n\n- In Table 6, could you elaborate on what \"Temporal NormLin\" baseline exactly is?  Do you keep the intra-series learner? Could you be more explicit about this architecture?\n\n- Could you also have a comparison with other lightweight multivariate methods such as SAMFormer [1] and TSMixer [2]?\n\n\n*References:*\n\n - [1] (Ilbert et al., 2024) SAMFormer: Unlocking the potential of transformers in time series forecasting with sharpness-aware minimization and channel-wise attention.\n - [2] (Chen et al., 2023) TSMixer: An all-mlp architecture for time series forecasting.", "limitations": "- As said earlier, the memory complexity remains quadratic with respect to D.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "*Strengths:*\n- The two-component design (OrthoTrans, NormLin for cross-variable interactions) creates a clean separation of concerns where temporal decorrelation is handled by preprocessing and multivariate correlations by learned linear layers.\n- These two modules can work as plugins for existing transformer architectures, with validated improvements on iTransformer (+5.1%) and PatchTST (+10.1%), which is really interesting \n- The empirical benchmark is exhaustive, covering 24 datasets and 140 configurations, including zero-shot and few-shot scenarios and seems to show strong results. The ablation study is extensive.\n- Rank analysis demonstrates good expressiveness of the model.\n\n*Weaknesses:*\n- The proposed way to evaluate the correlation matrix is confusing and lacks proper formalization. I guess you want to estimate the autocorrelation of a block of T timesteps (context length) but what does this mean formally? The mathematical framework should be more rigorously defined as the concept is non-trivial.\n- In addition, when reading the description of the approach, two intuitive baselines came to my mind: (a) instead of eigenvector decomposition of the correlation matrix, we can directly perform data rotation by PCA (i.e., working with $XX^T$), (b) the correlation matrix can alternatively be estimated through a sliding window approach by computing cross-correlation between the timestamps across different windows. It will be valuable to perform additional experiments to compare with these approaches in order to have more insights on the chosen methodology.\n- Averaging correlations across N variables dilutes variable-specific temporal structures without theoretical justification. Why did not you apply decorrelations for each channel separately?\n- Decorrelation effectiveness is not shown, no metrics show Q^T CorrMat Q ≈ Λ or residual correlation analysis on a validation set, for example.\n- When looking at the source code, it appears that there are some implementation errors. The authors consider the Pearson correlation, but in `dataset/Generate_corrmat.ipynb`, we have `cov_matrix = cov_matrix / diag_vec` which seems to divide only by σ_i, not σ_i × σ_j, producing an invalid asymmetric correlation matrix.\n- Some other architecture choices are not well-explained nor well-motivated (e.g., the CSL block, the dimension extension). Please see my questions below.\n- Reducing the number of FLOPs is beneficial, but the complexity (memory for example) remains quadratic with respect to D, which can still result in substantial computational overhead for high-dimensional multivariate series.", "was_revised": true, "final_justification": "I am raising my score as the authors have clarified most of my concerns including:\n* Clarification on the definition of the temporal correlation matrix and the way it is calculated,\n* Empirical justification that the approach effectively decorrelates the channels,\n* Extension of the experiments by incorporating more baselines,\n* Other concerns related to ablation study.", "rebuttal": "Thank you for your invaluable review. We sincerely appreciate the reviewer’s recognition that our plug-and-play OrthoTrans and NormLin modules are \"**really interesting**\" and the experiments are \"**exhaustive**\" and \"**extensive**\". Below are our responses to the concerns:\n\n> Q1: On the computation of $\\mathrm{CorrMat}\\_{t}$ and 'asymmetric' normalization\n\n- We first clarify our process of computing the temporal correlation matrix $\\mathrm{CorrMat}\\_{t}$. For clarity, we use the univariate series. Let  $\\mathbf{x}^{\\text{train}} \\in \\mathbb{R}^{M}$  denote the training series. We then generate $T$ lagged sub-series (whose length is $M-T+1$): $$\n\\mathbf{s}\\_{i} = \\mathbf{x}[i : M - T + i],\\quad i = 0, 1, \\dots, T-1.\n$$  The correlation matrix among these sub-series is then computed. The entire process is elaborated in Section 4.2 of the paper.\n\n- The reason for using the cov_matrix / diag_vec normalization in our code is that the sub-series $  \\mathbf{s}\\_{i}, i=0,1,\\cdots,T-1  $ share almost identical standard deviations because they consist of the same $M-T+1$ data points. Since $M\\gg T$, this overlapping part constitutes the majority of these sub-series. We further report the metric: $\\frac{\\sigma_{max}}{\\sigma_{min}}-1 $, where $\\sigma_{max}$ and $\\sigma_{min}$ are the maximum and minimum standard deviations of the sub-series, respectively.\n\n| **Dataset** | **std_max/std_min-1** |\n|-------------|---------------|\n| ECL | 2E-03|\n| Traffic| 3E-03|\n| Weather| 1E-03|\n| Solar  | 2E-04|\n| ETTm1  | 6E-04|\n\n- Because the standard deviations of these sub-series are (almost) the same, i.e., $\\sigma_{i} \\approx \\sigma_{j}$ for $0 \\leq i,j \\leq T-1$, and diag_vec[i] = $\\sigma_{i}^2$, we can simply use cov_matrix / diag_vec to obtain the Pearson correlation matrix. Another advantage of this approach is that the resulting $\\mathrm{CorrMat}_{t}$ always has diagonal elements equal to 1, as required for correlation matrices.\n\n- We compare forecasting performance using Q matrices computed from cov_matrix / diag_vec (**Mode 1**) and cov_matrix / $\\sigma \\sigma^T$ (**Mode 2**), and find that the results are identical:\n\n| **Dataset**   | **Mode 1** |   | **Mode 2** |    |\n|---------------|------------|--------|------------|--------|\n|               | MSE        | MAE    | MSE        | MAE    |\n| ECL (Avg)     | 0.159      | 0.248  | 0.159      | 0.248  |\n| Traffic (Avg) | 0.451      | 0.247  | 0.451      | 0.247  |\n| Weather (Avg) | 0.237      | 0.260  | 0.237      | 0.260  |\n\n- We further verify the orthogonality of Q matrices computed from Mode 1, using the metric $\\|| \\mathbf{Q}\\mathbf{Q}^T - \\mathbf{I} \\||_F$ (where $\\|| \\cdot \\||_F$ denotes the Frobenius norm):\n\n| **Dataset** | **96**    | **192**   | **336**   | **720**   |\n|-------------|-----------|-----------|-----------|-----------|\n| ECL         | 1.580E-06 | 3.349E-06 | 4.237E-06 | 7.706E-06 |\n| Traffic     | 1.697E-06 | 3.283E-06 | 4.171E-06 | 7.747E-06 |\n| Weather     | 1.777E-06 | 3.311E-06 | 4.178E-06 | 7.704E-06 |\n| Solar       | 1.743E-06 | 3.405E-06 | 4.371E-06 | 7.876E-06 |\n| ETTm1       | 1.755E-06 | 3.377E-06 | 4.213E-06 | 7.606E-06 |\n\n- Furthermore, as shown in Appendix I.8, OLinear is relatively robust even when Q matrices are computed with limited training data. \n\n- Since the Q matrices are **pre‑computed** and used throughout training and inference, we generate them once in a Jupyter Notebook, save them as ```.npy``` files, and load them during model initialization.\n\n> Q2: On the decorrelation architecture\n\n- Our choice of channel averaging (CA) over channel individual (CI) for computing the temporal correlation matrix is **based on the empirical results**. As shown below, CA consistently outperforms CI in forecasting performance. A possible reason is that temporal correlations for individual channels may be sensitive and time-varying, and CA can mitigate this effect. Furthermore, CI Q matrices are much larger than their CA counterparts. For example, for the Traffic dataset, the CI Q matrix (862 × 720 × 720) is 862 times larger than the CA Q matrix (720 × 720). In addition, the classic work TimesNet also adopts CA to determine the top‑k frequencies.\n\n| **Dataset** | **Horizon** | **CA** || **CI** ||\n|-------------|-------------|--------|--------|--------|--------|\n| | | MSE| MAE| MSE| MAE|\n| ECL | 96| 0.131| 0.221| 0.141| 0.232|\n| | 192| 0.150| 0.238| 0.164| 0.255|\n| | 336| 0.165| 0.254| 0.176| 0.268|\n| | 720| 0.191| 0.279| 0.204| 0.292|\n| | Avg| **0.159**| **0.248**| 0.171| 0.262|\n| Solar| 96| 0.179| 0.191| 0.182| 0.197|\n| | 192| 0.209| 0.213| 0.214| 0.219|\n|| 336| 0.231| 0.229| 0.237| 0.237|\n| | 720| 0.241| 0.236| 0.252| 0.244|\n| | Avg| **0.215**| **0.217**| 0.221| 0.224|\n| Weather| 96| 0.153| 0.190| 0.153| 0.191|\n| | 192| 0.200| 0.235| 0.205| 0.241|\n| | 336| 0.258| 0.280| 0.263| 0.285|\n| | 720| 0.337| 0.333| 0.350| 0.342|\n| | Avg| **0.237**| **0.260**| 0.243| 0.264|\n\n\n- OrthoTrans is technically equivalent to PCA without dimensionality reduction. Since orthogonal transformation is also a conformal transformation, OrthoTrans **rotates** the data into a new feature space where correlations are removed. Moreover, according to PCA theory, noise is suppressed in the primary components.\n\n- Regarding learnable Q matrices, we kindly refer to our response to Reviewer rQjb’s Q3. In short, we tried several updating schemes but observed no performance gain. Theoretically, Q is already the optimal orthogonal matrix for temporal decorrelation; altering it risks reintroducing correlations. These experimental results further confirm the effectiveness of OrthoTrans.\n\n> Q3: On the dimension extension\n\n- Unlike the linear encoder ($\\mathbf{W} \\mathbf{x} +\\mathbf{b}$), the dimension extension module is an **outer product** of $\\mathbf{x}$ with a learnable vector $\\phi \\_d \\in \\mathbb{R}^d$, which can be easily implemented using the $*$ operator in PyTorch. Further implementation details can be found in Lines 84–91 of ```model/OLinear.py``` in our code repository.\n\n- We report performance with various embedding sizes $d$ in Table 30 on Page 42. The results show that this module ($d = 16$) improves forecasting performance compared with its removal ($d = 1$). (In our implementation the module is skipped when $d = 1$.) Because OrthoTrans is a linear operation (matrix multiplication), its **relative order with the dimension-extension module has no impact on the final result**.\n\n> Q4: On the linear layers in the CSL\n\n- The two layers in the CSL, referred to as the pre‑ and post‑linear layers according to their execution sequence, are inspired by the classic self‑attention mechanism. As shown in **Appendix J.2**, incorporating both pre‑ and post‑linear layers yields an average performance improvement of 6% (see **Table 34**, Page 46) compared to the variant without them, highlighting their effectiveness in refining inputs for multivariate correlation modeling (pre‑linear) and downstream series representation learning (post-linear).\n\n> Q5: On the temporal NormLin\n\n- In **Table 6**, we present ablation studies in which NormLin and standard linear layers are either replaced or removed. The *temporal NormLin* baseline indicates that the ISL is **retained** while its two linear layers (**Eq. 5** in the paper) are **replaced with two NormLin layers**, which operate on the temporal dimension. Now the weight matrices in the two NormLin layers are $\\mathbf{W}\\_1  \\in \\mathbb{R}^{d_{model}\\times d_{ff}}$ and $\\mathbf{W}\\_2  \\in \\mathbb{R}^{d_{ff}\\times d_{model}}$, rather than the weight matrix $\\mathbf{W}  \\in \\mathbb{R}^{N \\times N}$ in Eqs. 3 and 4. Please refer to line 299 in ```layers/Transformer_EncDec.py``` of our code repository for details.\n\n- Table 6 shows that our design—applying NormLin along the variate dimension and standard linear layers along the temporal dimension—consistently achieves the best performance.\n\n> Q6: Comparison with SAMformer and TSMixer\n\n- We provide MSEs and standard deviations to compare OLinear with SAMformer and TSMixer. Results of SAMformer and TSMixer are from the SAMformer paper. Note that SAMformer and TSMixer use a lookback window of 512 and thus enjoy extra performance gains. Despite this, OLinear achieves lower MSE and smaller standard deviations in most cases.  \n\n- More results are presented in our response to Reviewer jUXb's Q2. When the lookback length is uniformly set as 96, OLinear outperforms SAMformer **by a substantial margin of 12.1%**.\n\n- We will incorporate a discussion of these two classic forecasters in the final draft.\n\n| **Dataset** | **Hor.** | **OLinear (Lookback: 96)** | **SAMformer (Lookback: 512)** | **TSMixer (Lookback: 512)** |\n|-------------|------|-------------|---------------|-------------|\n| ETTm2       | 96   | **0.169±1e-4**  | 0.181±0.005   | 0.211±0.014 |\n|             | 192  | **0.232±1e-4**  | 0.233±0.002   | 0.252±0.005 |\n|             | 336  | 0.291±2e-4  | **0.285±0.001**   | 0.303±0.004 |\n|             | 720  | 0.389±4e-4  | **0.375±0.001**   | 0.390±0.003 |\n| ECL         | 96   | **0.131±4e-4**  | 0.155±0.002   | 0.173±0.004 |\n|             | 192  | **0.150±1e-3**  | 0.168±0.001   | 0.204±0.027 |\n|             | 336  | **0.165±1e-3**  | 0.183±0.000   | 0.217±0.018 |\n|             | 720  | **0.191±2e-3**  | 0.219±0.000   | 0.242±0.015 |\n| Exchange    | 96   | **0.082±3e-4**  | 0.161±0.007   | 0.343±0.082 |\n|             | 192  | **0.171±2e-3**  | 0.246±0.009   | 0.342±0.031 |\n|             | 336  | **0.331±3e-3**  | 0.368±0.006   | 0.484±0.062 |\n|             | 720  | **0.837±0.018** | 1.003±0.018   | 1.204±0.028 |\n| Weather     | 96   | **0.153±1e-3**  | 0.197±0.001   | 0.214±0.004 |\n|             | 192  | **0.200±2e-3**  | 0.235±0.000   | 0.231±0.003 |\n|             | 336  | **0.258±3e-3**  | 0.276±0.001   | 0.279±0.007 |\n|             | 720  | 0.337±4e-3  | **0.334±0.000**   | 0.343±0.024 |\n\n**We hope these responses can fully address your concerns. Thank you again for your detailed feedback!**"}, {"review_id": "S1OxUde4lD", "reviewer": "Reviewer_5fFn", "summary": "The paper introduces OLinear, a linear-based model for multivariate time series forecasting with two key components: \n- OrthoTrans: A data-adaptive transformation that diagonalizes the temporal Pearson correlation matrix, decorrelating features and simplifying forecasting.\n\n- NormLin: A lightweight linear layer with row-wise normalization that outperforms self-attention in accuracy and efficiency.\n\nExperiments on 24 datasets demonstrate state-of-the-art performance, particularly for long-term forecasting, while maintaining low computational overhead.", "questions": "- Why does NormLin show better gradient dynamics than attention? Are there any theoretic supports.\n- What is the relationship OrthoTrans with PCA?\n- Theorem 1 assumes Gaussian. How sensitive is OLinear to deviations from this assumption in real-world data?\n- How does performance degrade with noisy/missing data or non-stationary correlations? \n- What is the inference latency for ultra-long sequence (1M)? And some related references for long-term forecasting such as Pyraformer (ICLR 2022) should be discussed.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n+ Theorem 1 formally connects the temporal-correlation structure of the data with the orthogonal transformation used by OrthoTrans, giving the model a clear theoretical underpinning and enhancing interpretability.\n+ NormLin replaces multi-head self-attention with a row-normalized linear layer, cutting both FLOPs and memory footprint by roughly 50 % while preserving—or improving—forecasting accuracy.\n+ OLinear achieves state-of-the-art results on 140 forecasting tasks drawn from 24 public datasets, demonstrating robustness across domains and long-horizon scenarios.\n+ Both OrthoTrans and NormLin are modular, training-free components that can be plugged into existing forecasters to boost performance with minimal engineering effort.\n+ The experimental section is notably comprehensive, featuring extensive ablations, scalability tests, and comparisons against a wide range of baselines.\n\nWeaknesses\n- While training-time speed and memory are reported, the paper does not quantify inference-time latency or serving cost—metrics that are crucial in streaming or high-frequency settings; deployment-cost analysis is therefore incomplete.\n- Important modern linear or correlation-based baselines (e.g., RLinear, TSMixer, StemGNN, DEPTS) are omitted, leaving open whether OLinear’s gains hold against the strongest current alternatives.\n- The claim that NormLin is a “universal token dependency learner” appears overstated: its superiority is demonstrated only on time-series data, with no evidence provided for other modalities or tasks.", "was_revised": true, "final_justification": "Thanks the authors for providing detailed response especially those extra data support.\nI keep my original rating, and think this paper is acceptable.", "rebuttal": "Thank you for your positive comments on our work. We sincerely appreciate the reviewer’s recognition of our work as **theoretically well-founded**, experimentally **comprehensive**, and achieving **state-of-the-art** performance. Here are our responses to your concerns:\n\n> W1: On the deployment cost\n- We report the training and inference resource footprint of OLinear and baseline forecasters in **Table 37 (Page 48)**, including **inference time and GPU memory** consumption. OLinear achieves higher inference efficiency than state-of-the-art Transformer-based forecasters (e.g., TimeMixer++, iTransformer, and CARD), benefiting from the simplified NormLin module. Moreover, **Table 38** shows the efficiency differences before and after applying the NormLin module to Transformer-based forecasters.\n\n> W2: On more baselines\n\n- We compare OLinear with RLinear and TSMixer using the MSE metric (see below). The results of RLinear and TSMixer are taken from iTransformer and SAMformer, respectively. On average, OLinear outperforms these two forecasters by 17.3% and 25.6%, respectively.\n\n| **Dataset** | **Hor.** | **OLinear** | **RLinear** | **TSMixer** |\n|-------------|------|-------------|-------------|-------------|\n| ETTm2       | 96   | **0.169**       | 0.182       | 0.211       |\n|             | 192  | **0.232**       | 0.246       | 0.252       |\n|             | 336  | **0.291**       | 0.307       | 0.303       |\n|             | 720  | **0.389**       | 0.407       | 0.390       |\n| ECL         | 96   | **0.131**       | 0.201       | 0.173       |\n|             | 192  | **0.150**       | 0.201       | 0.204       |\n|             | 336  | **0.165**       | 0.215       | 0.217       |\n|             | 720  | **0.191**       | 0.257       | 0.242       |\n| Exchange    | 96   | **0.082**       | 0.093       | 0.343       |\n|             | 192  | **0.171**       | 0.184       | 0.342       |\n|             | 336  | **0.331**       | 0.351       | 0.484       |\n|             | 720  | **0.837**       | 0.886       | 1.204       |\n| Traffic     | 96   | **0.398**       | 0.649       | 0.409       |\n|             | 192  | **0.439**       | 0.601       | 0.637       |\n|             | 336  | **0.464**       | 0.609       | 0.747       |\n|             | 720  | **0.502**       | 0.647       | 0.688       |\n| Weather     | 96   | **0.153**       | 0.192       | 0.214       |\n|             | 192  | **0.200**       | 0.240        | 0.231       |\n|             | 336  | **0.258**       | 0.292       | 0.279       |\n|             | 720  | **0.337**       | 0.364       | 0.343       |\n\n- In Section 5.1, OLinear is compared with 11 carefully selected and widely acknowledged state-of-the-art forecasting models. We believe these results demonstrate the performance superiority of OLinear. For completeness, we will also include discussions on RLinear, TSMixer, StemGNN, and DEPTS in the final draft.\n\n> W3: On the claim of “universal token dependency learner” \n\n- On Line 289, we explicitly restrict the claim “universal token dependency learner” to the field of “time series forecasting.” However, on Lines 278 and 312, this field restriction was unintentionally omitted. We will revise these two sentences in the final draft to ensure accuracy. Thank you for pointing this out!\n\n> Q1: On the gradient dynamics\n\n- In **Appendix B**, we analyze the Jacobian matrix of the non-linear transformations in the self-attention mechanism and the NormLin layer. For self-attention, we have \n$$ \\frac{\\partial \\mathbf{c} }{\\partial \\mathbf{a} }  = \\mathrm{Diag} (\\mathbf{c}) - \\mathbf{c}\\mathbf{c}^{\\mathsf{T}},$$ where $\\mathbf{c} \\triangleq \\mathrm{Softmax}(\\mathbf{a}) \\in \\mathbb{R}^N$, and $\\mathrm{Diag} (\\mathbf{c})$ is the diagonal matrix with $\\mathbf{c}$ as its diagonal. \n\n- For the NormLin layer, it holds that \n$$\\frac{\\partial \\mathbf{c}}{\\partial \\mathbf{a}}=\\frac{1}{\\left \\|| \\mathbf{b} \\right \\||_1 } \n\\left (  \\mathrm{Diag}  \\left ( \\tilde{\\mathbf{b}} \\right ) - \\bar{\\mathbf{b}} \\tilde{\\mathbf{b}}^{\\mathsf{T}} \\right ),$$\nwhere $\\mathbf{c} \\triangleq \\mathrm{Norm}\\_{\\mathrm{L1} }\\left ( \\mathrm{Softplus} \\left ( \\mathbf{a}  \\right )  \\right )  \\in \\mathbb{R}^N$, $\\mathbf{b} \\triangleq \\mathrm{Softplus}(\\mathbf{a})$, $\\bar{\\mathbf{b}} = \\frac{\\mathbf{b}}{\\left \\|| \\mathbf{b} \\right \\||_1 }$ is the normalized $\\mathbf{b}$, $\\tilde{\\mathbf{b}} \\triangleq \\mathrm{Sigmoid} (\\mathbf{a})$. The detailed derivation process is presented in Appendix B. \nThe two Jacobian matrices share similar structures, while the Jacobian matrix of NormLin has an additional learnable scaling factor $\\frac{1}{\\left \\|| \\mathbf{b} \\right \\||_1}$, providing greater flexibility.\n\n- **Figure 6** (Page 15) illustrates the Jacobian matrices of the self-attention mechanism and the NormLin layer under the same input $\\mathbf{a}$, highlighting that NormLin tends to produce stronger gradient values.\n\n> Q2: On the relationship of OrthoTrans and PCA\n\n- Essentially, OrthoTrans is PCA with all features retained. It is adapted to the time series modality through its computation of $\\mathrm{CorrMat}_{t}$. One contribution of our work is showing that decorrelating along the time dimension boosts forecasting performance. As shown in **Table 5**, OrthoTrans, as a plug-in, improves the performance of iTransformer, PatchTST, and RLinear.\n\n- Further analysis shows that OrthoTrans can **mitigate the intrinsic low-rank property of time series data**. In our reply to Q2 of Reviewer rQjb, we compared the median singular values of correlation matrices of time patches with and without OrthoTrans. The results show that OrthoTrans produces larger singular values, thereby increasing data diversity. This partly explains the performance gains brought by OrthoTrans.\n\n> Q3: On Theorem 1 \n\n- Theorem 1 shows that temporally decorrelated inputs facilitate forecasting, providing OLinear with a clear theoretical underpinning. The widely used Gaussian assumption is applied in Theorem 1. We acknowledge that real-world datasets can be more complex and difficult to model mathematically. Despite this, OLinear performs robustly across real-world datasets. We conduct extensive experiments, covering 24 datasets and 140 configurations, and OLinear consistently exhibits state-of-the-art performance.\n\n> Q4: On non-stationary and noisy dataset\n\n- Financial data are commonly regarded as non-stationary. We report average MAEs on Exchange, NASDAQ, SP500, and DowJones datasets. The results below show that OLinear remains robust on these datasets.\n\n| **Dataset**   | **OLinear** | **DLinear** | **iTransformer** | **PatchTST** |\n|---------------|-------------|-------------|------------------|--------------|\n| Exchange      | **0.399**       | 0.414       | 0.403            | 0.404        |\n| NASDAQ (S1)   | **0.125**       | 0.170       | 0.137            | 0.132        |\n| SP500 (S1)    | **0.167**       | 0.205       | 0.193            | 0.192        |\n| DowJones (S2) | **0.848**       | 0.857       | 0.869            | 0.862        |\n\n- Regarding noisy/missing data, we evaluate OLinear on traffic and weather datasets. (The widely used Weather dataset contains missing data, which are filled with -9999.) OLinear demonstrates clear performance advantages under these challenging conditions.\n\n| **Dataset** | **OLinear** | **DLinear** | **iTransformer** | **PatchTST** |\n|-------------|-------------|-------------|------------------|--------------|\n| Traffic     | **0.247**       | 0.383       | 0.282            | 0.343        |\n| PEMS03      | **0.199**       | 0.375       | 0.221            | 0.291        |\n| PEMS07      | **0.164**       | 0.395       | 0.204            | 0.303        |\n| Weather     | **0.260**       | 0.317       | 0.279            | 0.281        |\n\n> Q5: On inference latency and related work\n\n- For ultra-long sequences (1M), the inference latency varies with the number of channels. Using the data in **Table 37** (Page 48), we estimate that for a 1M-long series with 21 channels, the inference latency is about 0.77 seconds, which is suitable for streaming scenarios.\n\n| **# Channels** | **Infer. Time for 1M-long Series (seconds)** |\n|----------------|----------------------------------------|\n| 21             | 0.77                                   |\n| 137            | 0.78                                   |\n| 321            | 1.37                                   |\n| 862            | 3.72                                   |\n\n\n- We further compare OLinear with Pyraformer and report the average results across four prediction lengths. In the final draft, we will include discussions of more baselines including TSMixer, StemGNN, DEPTS, and Pyraformer.\n\n| **Dataset** | **OLinear** |    | **Pyraformer** |   |\n|-------------|-------------|--------|----------------|-------|\n|             | MSE         | MAE    | MSE            | MAE   |\n| ETTm1       | **0.374**       | **0.377**  | 0.691          | 0.607 |\n| ETTm2       | **0.270**       | **0.313**  | 1.498          | 0.869 |\n| ETTh1       | **0.424**       | **0.424**  | 0.827          | 0.703 |\n| ETTh2       | **0.367**       | **0.388**  | 0.826          | 0.703 |\n| ECL         | **0.159**       | **0.248**  | 0.379          | 0.445 |\n| Traffic     | **0.451**       | **0.247**  | 0.878          | 0.469 |\n| Weather     | **0.237**       | **0.260**  | 0.946          | 0.717 |\n| Exchange    | **0.355**       | **0.399**  | 1.913          | 1.159 |\n\n\n**We hope these responses can fully address your concerns. Thank you again for your detailed feedback!**"}, {"review_id": "uAP7B2kztT", "reviewer": "Reviewer_jUXb", "summary": "The paper introduces *OLinear*, a multivariate time-series forecaster that operates in an *orthogonally transformed* domain rather than the raw time domain. For each input window, the authors compute the temporal Pearson-correlation matrix, perform an eigen-decomposition, and project the signals onto the resulting orthogonal basis. They then replace self-attention with *NormLin*. Extensive experiments indicate improvements over strong baselines, and the accompanying code is available.", "questions": "1. *Could you run a statistical significance test across ≥ 5 random seeds for every dataset–horizon pair and report the resulting p-values?*\n\n2. The existing work already demonstrates that a specific optimiser combined with a single channel-wise attention head (complexity $O(N^2)$) raises attention-matrix rank and boosts accuracy while using a channel attention mechanism to capture multivariate correlations. *Please clarify the novelty and compare the baseline with this existing work. Especially, how the model compares to a shallow transformer trained with a strong regularization like SAM and a channel-wise attention mechanism ?*  \n\n3. The claim that OrthoTrans removes step-wise correlations needs quantitative confirmation in both domains.  *Could you provide a metric that shows (i) a drop in the off-diagonal magnitude of the time-domain covariance matrix and (ii) that no new dependencies appear in the frequency domain—for instance, by comparing the Frobenius norm of the off-diagonal entries in the covariance matrix and in the cross-spectral density before and after applying OrthoTrans?*\n\n4. NormLin removes non-linearity yet keeps quadratic cost.  *Is there a way to reduce this cost?*\n\n5. In the experiments, you use different learning rate, model dimension, embedding size and number of blocks for each dataset. *Could you please freeze one single architecture, run it on all the datasets and compare this performance with the best architecture for each dataset?*", "limitations": "While the authors include a brief limitations paragraph in Appendix, a fuller discussion into the main paper, covering details like compute cost and dataset-specific hyper-parameters, is needed.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n1. The authors evaluate OLinear on 24 publicly available datasets spanning electricity, traffic, weather and financial domains, and consider multiple forecast horizons for each dataset. Reporting 99 % confidence intervals indicates an effort to convey result variability rather than single-run numbers.\n\n2. The paper is well structured: problem statement, method, ablations and main tables flow logically. Figures are labelled and referenced properly, and notation is introduced before use, which makes the technical content easy to follow.\n\n3. Code and data-processing scripts are released, allowing reviewers and future researchers to reproduce the results or to integrate the proposed method into their own pipelines with minimal effort.\n\n4. The plug-and-play lowers the entry barrier for practitioners who already have Transformer-based models and wish to test the proposed ideas without redesigning their entire architecture.\n\n5. While Fourier and wavelet bases dominate current transform-domain forecasters, constructing an orthogonal basis from the data’s own temporal correlation matrix has been less explored. This adaptive approach could inspire further work. \n\n## Weaknesses\n\n1. Results seem to degrade when the look-back window exceeds 96, casting doubt on long-horizon robustness. For example, with 15-minute data, 96 steps cover only 24 h, yet the model may need to predict up to 720 steps (≈ 1 week) ahead in long-term forecasting, raising concerns about interpretability.  \n\n2. Many of the hyper-parameters, such as model dimension, embedding size, number of blocks and learning rate, are dataset-dependent.\n\n3. The assumption of decorrelation after OrthoTrans is not demonstrated. Two series can be decorrelated in time yet correlated in frequency; an orthogonal transform does not necessarily remove such cross-frequency dependencies. Moreover, decorrelating in the time domain removes second-order dependencies but says nothing about higher-order or non-linear dependencies.\n\n4. *The novelty remains unclear.* The manuscript does not cite the previous existing work [1]. That work already establishes that forecasting Transformers suffer from excessive sharpness, and that this can be mitigated by (i) using a different optimiser and (ii) reducing the architecture to a *single* channel-wise attention head—whose complexity is exactly $O(N^2)$, the same as NormLin with $h=1$ (see the Table 1). These two changes were shown to raise the attention-matrix rank and therefore boost the model’s expressivity. The study re-uses those insights in a purely linear setting, which by construction loses the non-linear capacity of a Transformer, yet none of this prior art is acknowledged. Consequently, the claims regarding higher rank, multivariate correlations, and expressivity are not novel in the context of multivariate time-series forecasting.  \n\n5. NormLin trades away non-linearity while retaining the same $O(N^2)$ cost.   \n\n**References**\n\n[1] SAMformer : Unlocking the potential of Transformers in Time Series Forecasting", "was_revised": true, "final_justification": "The authors have addressed my initial concerns. They have also committed to revise the narrative on weight-matrix rank in the camera-ready.", "rebuttal": "Thank you for your comments. We sincerely appreciate the reviewer’s recognition of our \"**well-structured**\" presentation, the plug-and-play modules that \"**lower the entry barrier**\", and an adaptive design that could \"**inspire further work**\". Below are our responses:\n\n> Q1: On the p-value\n\n- We conduct a significance test (using 7 random seeds) using Student’s t-test between OLinear and iTransformer across various datasets and prediction horizons. As shown below, the better model is in bold when the improvement is statistically significant at the level 0.05 (p-value < 0.05). In other words, OLinear exhibits a clear performance advantage over iTransformer.\n\n| **Setting**  | **OLinear (MAE)** | **iTrans.(MAE)** | **p-value between OLinear and iTrans.** |\n|-|-|-|-|\n| ECL: 96 | **0.221±4e-4**| 0.240±4e-4 | 5.12E-11|\n| ECL: 192 | **0.238±1e-3**| 0.253±2e-3 | 3.06E-06|\n| ECL: 336 | **0.254±1e-3**| 0.269±1e-3| 9.96E-07|\n| ECL: 720 | **0.279±2e-3**| 0.317±7e-3 | 5.59E-05|\n| Traffic: 96  | **0.226±2e-4**| 0.268±1e-3 | 1.61E-12|\n| Traffic: 192 | **0.241±4e-4**| 0.276±1e-3 | 9.48E-10|\n| Traffic: 336 | **0.250±3e-4**| 0.283±4e-4| 1.87E-13|\n| Traffic: 720 | **0.270±4e-4**| 0.302±4e-4| 6.15E-13|\n| Weather: 96  | **0.190±1e-3**| 0.214±3e-4| 1.29E-08|\n| Weather: 192 | **0.235±2e-3**| 0.254±1e-3| 1.04E-06|\n| Weather: 336 | **0.280±2e-3**| 0.296±1e-3| 1.93E-06|\n| Weather: 720 | **0.333±2e-3**| 0.349±4e-4| 5.68E-06|\n\n\n- We report the performance with increasing lookback lengths in **Appendix I.3**. OLinear demonstrates consistent improvements as the lookback horizon increases from 48 to 720, and consistently outperforms state-of-the-art forecasters.\n\n- Regarding robustness across random seeds, please refer to **Appendix F**, where OLinear exhibits smaller standard deviations and narrower 99% confidence intervals compared to TimeMixer++ and iTransformer. \n\n- We would like to emphasize that MSE and MAE metrics typically degrade in long-term forecasting as the prediction horizon increases. As shown in **Table 15** (Page 28), this phenomenon is observed in state-of-the-art time series forecasters, such as TimeMixer++ (ICLR 2025), iTransformer (ICLR 2024), FilterNet (NeurIPS 2024). This phenomenon is reasonable because future series inherently contain greater uncertainty as the horizon extends, much like how predicting the temperature two weeks ahead is less accurate than predicting tomorrow’s temperature.\n\n> Q2: On novelty and comparison with SAMformer\n\n- SAMformer (ICML 2024) introduces the Sharpness-Aware Minimization (SAM) mechanism—originally proposed in [1] (ICLR 2021)—into time series forecasting. While channel-wise attention was first introduced by iTransformer, SAMformer adopts the same practice. Its key contribution is a new **loss function** that smooths the loss landscape of Transformers: $$\\mathcal{L}^{SAM}\\_{train}(\\omega ) = \\underset{\\left \\| \\epsilon  \\right \\| <\\rho  }{max}\\mathcal{L}\\_{train}(\\omega +\\rho), $$ where $\\rho >0$ is a hyper-parameter. This can be implemented by the SAM optimizer [1]. \n\n- Instead of modifying the loss function, we focus on the **architecture design** and propose NormLin. Compared with vanilla self-attention, NormLin offers four advantages: lower computational cost, higher-rank attention matrices, smoother gradient flow, and an enlarged optimization space (see Lines 182-198 in the paper). NormLin \"can be plugged into existing forecasters to boost performance **with minimal engineering effort\" (by Reviewer 5fFn)**. Moreover, our **linear-based** model, OLinear, outperforms state-of-the-art Transformer-based forecasters—including SAMformer. We will **cite and discuss SAMformer in our final draft**.\n\n- We compare MSE and standard deviation between OLinear and SAMformer. The SAMformer numbers are taken from its original paper, which uses a look-back window of 512 and thus enjoys extra performance gains. Even under this handicap, OLinear achieves **lower MSE** and **smaller standard deviations** in most cases.\n\n| **Dataset** | **Hor.** | **OLinear (Look-back: 96)** | **SAMformer (Look-back: 512)** |\n|-|-|-|-|\n| ETTm2| 96   | **0.169±1e-4**  | 0.181±0.005 |\n| | 192  | **0.232±1e-4**  | 0.233±0.002 |\n| | 336  | 0.291±2e-4| **0.285±0.001** |\n| | 720  | 0.389±4e-4| **0.375±0.001** |\n| ECL  | 96   | **0.131±4e-4**| 0.155±0.002 |\n| | 192  | **0.150±1e-3**| 0.168±0.001 |\n| | 336  | **0.165±1e-3**| 0.183±0.000 |\n| | 720  | **0.191±2e-3**| 0.219±0.000 |\n| Exchange    | 96   | **0.082±3e-4**| 0.161±0.007 |\n| | 192  | **0.171±2e-3**| 0.246±0.009 |\n| | 336  | **0.331±3e-3**| 0.368±0.006 |\n| | 720  | **0.837±0.018**     | 1.003±0.018 |\n| Weather     | 96   | **0.153±1e-3**| 0.197±0.001 |\n| | 192  | **0.200±2e-3**| 0.235±0.000 |\n| | 336  | **0.258±3e-3**| 0.276±0.001 |\n| | 720  | 0.337±4e-3| **0.334±0.000** |\n\n- To ensure a fair comparison, we uniformly set the lookback length to 96 and report the following MAEs. The average results are reported across 3 random seeds for SAMformer, using the recommended learning rates and $\\rho$ values.\n  - On average, OLinear outperforms SAMformer by a **substantial margin of 12.1%**.\n  - Notably, the proposed **NormLin layer improves the performance of SAMformer** by 10.6% and 5.8% on the Traffic and ECL datasets, respectively.\n\n| **Dataset** | **Hor.** | **OLinear** | **SAMformer** | **SAMformer+NormLin** |\n|-|-|-|-|--|\n| ETTh1 | 96 | 0.382 | 0.396 | 0.392 |\n| | 192| 0.414 | 0.425 | 0.419 |\n| | 336| 0.438 | 0.444 | 0.439 |\n| | 720| 0.462 | 0.469 | 0.459 |\n| ETTm2 | 96 | 0.249 | 0.268 | 0.262 |\n| | 192| 0.290 | 0.305 | 0.303 |\n| | 336| 0.328 | 0.343 | 0.340 |\n| | 720| 0.387 | 0.399 | 0.397 |\n| ECL | 96 | 0.221 | 0.285 | 0.267 |\n| | 192| 0.238 | 0.288 | 0.269 |\n| | 336| 0.254 | 0.301 | 0.284 |\n| | 720| 0.279 | 0.335 | 0.318 |\n| Traffic | 96 | 0.226 | 0.404 | 0.355 |\n| | 192| 0.241 | 0.367 | 0.335 |\n| | 336| 0.250 | 0.379 | 0.339 |\n| | 720| 0.270 | 0.401 | 0.358 |\n| Weather | 96 | 0.190 | 0.231 | 0.208 |\n| | 192| 0.235 | 0.269 | 0.251 |\n| | 336| 0.280 | 0.304 | 0.290 |\n| | 720| 0.333 | 0.350 | 0.339 |\n| Echange | 96 | 0.200 | 0.203 | 0.210 |\n| | 192| 0.293 | 0.303 | 0.302 |\n| | 336| 0.414 | 0.426 | 0.416 |\n| | 720| 0.688 | 0.712 | 0.689 |\n\n\n\n> Q3: On the quantitation of temporally decorrelation\n\n- We report the **off-diagonal Frobenius norm** of the correlation matrices (size: $96\\times 96$) for the original series, and for the series after OrthoTrans or FFT, using a window size of 96. The results below show that OrthoTrans effectively removes temporal correlations and performs better than the FFT operation in decorrelation.\n\n| **Dataset** | **Original** | **OrthoTrans** | **FFT** |\n|-|--|--|-|\n| ETTh1 | 61.36| 1.46 | 2.58|\n| ETTm2 | 83.04| 1.26 | 5.22|\n| ECL | 45.95| 1.78 | 2.26|\n| Traffic | 34.80| 1.79 | 3.29|\n| Weather | 61.70| 2.86 | 19.61 |\n\n- We also argue that examining correlations in the frequency domain of the feature series after OrthoTrans is of limited relevance: 1) OLinear does not involve any FFT operations. 2) Whether there exists an orthogonal transformation that simultaneously mitigates both temporal- and frequency-domain correlations is a nontrivial question. Further exploration could be a future direction of this work. We will emphasize that the decorrelation is performed **temporally** in the final draft.\n\n> Q4: On quadratic cost of NormLin\n\n- In our view, $\\mathcal{O}(N^2)$ is the minimal cost required to model $N$-channel correlations, as even a single linear layer incurs this complexity.\n\n- A straightforward way to reduce this cost is to introduce a **bottleneck** block: the channels are first projected down to a **fixed width** (e.g., 64), the NormLin module is applied in the fixed-dimensional space, and then the representation is projected back to $N$ channels. This reduces the computational complexity to $\\mathcal{O}(N)$. As shown below, this variant tends to degrade performance, particularly on datasets with a large number of channels, such as ECL and Traffic.\n\n| **Dataset** | **Hor.** | **Original (MAE)** | **Bottleneck (MAE, fixed width=64)** |\n|-|-|-|-|\n| ETTh1 | 96 | 0.382| 0.382|\n| | 192| 0.414| 0.415|\n| | 336| 0.438| 0.443|\n| | 720| 0.462| 0.468|\n| ECL | 96 | 0.221| 0.241|\n| | 192| 0.238| 0.256|\n| | 336| 0.254| 0.270|\n| | 720| 0.279| 0.291|\n| Traffic | 96 | 0.226| 0.263|\n| | 192| 0.241| 0.278|\n| | 336| 0.250| 0.283|\n| | 720| 0.270| 0.298|\n| Weather | 96 | 0.190| 0.190|\n| | 192| 0.235| 0.239|\n| | 336| 0.280| 0.281|\n| | 720| 0.333| 0.337|\n| PEMS-03 | 12 | 0.159| 0.160|\n| | 24 | 0.179| 0.182|\n| | 48 | 0.210| 0.224|\n| | 96 | 0.247| 0.264|\n\n> Q5: On hyperparameters\n\n- Because there are substantial differences among datasets, it is common practice (e.g., iTransformer and SAMformer) to fine-tune certain hyperparameters to maximize model performance.\n\n- In the paper, the hyperparameter value ranges are provided in Appendix D. Note that the embedding size $d$ is fixed at 16 for all experiments. We also evaluate the **hyperparameter sensitivity** of OLinear in **Appendix I.6**. As shown in **Figure 17** and **Table 30**, OLinear exhibits robust performance across these hyperparameters.\n\n- Furthermore, we freeze a single architecture (learning rate $lr = 5\\times10^{-4}$, model dimension $D = 512$, number of blocks $L = 3$, embedding size $d = 16$) and report the average results across four prediction lengths. Compared to the results reported in the paper, the performance degradation is within 1%. \n\n| **Dataset** | **In paper** |  | **Fixed Para.** |  |\n|-|-|-|-|-|\n| | MSE| MAE| MSE | MAE|\n| ETTh1 |0.424| 0.424| 0.426 | 0.425|\n| ETTh2 |0.367| 0.388| 0.369 | 0.390|\n| ETTm1 |0.374| 0.377| 0.377 | 0.381|\n| ETTm2 |0.270| 0.313| 0.271 | 0.313|\n| ECL |0.159| 0.248| 0.159 | 0.248|\n| Traffic |0.451| 0.247| 0.451 | 0.247|\n| Weather | 0.237| 0.260| 0.240 | 0.262|\n\n**We hope these responses can fully address your concerns. Thank you again for your comments!**\n\n*Reference*\n\n[1] Sharpness-aware minimization for efficiently improving generalization. ICLR 2021."}]

Author rebuttal:

Meta-review: This paper introduces OLinear, a new linear multivariate time series forecasting model, where the O indicates that the model operates in an orthogonally transformed domain. It includes two key components: OrthoTrans,  a data-adaptive orthogonal transformation that uses temporal Pearson correlation matrices; NormLin: a lightweight, normalized linear layer that effectively replaces more complex self-attention mechanisms, leading to gains in both accuracy and efficiency. The empirical study shows good performance. 

All reviewers gave positive ratings, and indicated that the rebuttal comments are useful to clarify unclear things in the original submission. The authors promised to include the rebuttall comments into the final version to improve the paper. Overall, the paper presents an interesting linear model for time series forecasting.
Decision: Accept (poster)