Characteristic Root Analysis and Regularization for Linear Time Series Forecasting
Authors: Zheng Wang
Kaixuan Zhang
Wanfang Chen
Xiaonan Lu
Longyuan Li
Tobias Schlagenhauf
Venue: ICLR 2026 Poster (2026)
Keywords: long term time series forecasting
linear model
characteristic roots
modes
noise robustness
rank reduction
root purge
Primary area: learning on time series and dynamical systems

Abstract: Time series forecasting remains a critical challenge across numerous domains, yet the effectiveness of complex models often varies unpredictably across datasets. Recent studies highlight the surprising competitiveness of simple linear models,  suggesting that their robustness and interpretability warrant deeper theoretical investigation. This paper presents a systematic study of linear models for time series forecasting, with a focus on the role of characteristic roots in temporal dynamics. We begin by analyzing the noise-free setting, where we show that characteristic roots govern long-term behavior and explain how design choices such as instance normalization and channel independence affect model capabilities. We then extend our analysis to the noisy regime, revealing that models tend to produce spurious roots. This leads to the identification of a key data-scaling property: mitigating the influence of noise requires disproportionately large training data, highlighting the need for structural regularization. To address these challenges, we propose two complementary strategies for robust root restructuring. The first uses rank reduction techniques, including Reduced-Rank Regression (RRR) and Direct Weight Rank Reduction (DWRR), to recover the low-dimensional latent dynamics. The second, a novel adaptive method called Root Purge, encourages the model to learn a noise-suppressing null space during training. Extensive experiments on standard benchmarks demonstrate the effectiveness of both approaches, validating our theoretical insights and achieving state-of-the-art results in several settings. Our findings underscore the potential of integrating classical theories for linear systems with modern learning techniques to build robust, interpretable, and data-efficient forecasting models. The code is publicly available at: https://github.com/Wangzzzzzzzz/RootPurge.

Published as a conference paper at ICLR 2026
CHARACTERISTIC ROOT ANALYSIS AND REGULARIZA- TION FOR LINEAR TIME SERIES FORECASTING
Zheng Wang .1, Kaixuan Zhang Wanfang Chen Xiaonan Lu
Longyuan Li Tobias Schlagenhauf
Bosch (China) Investment Co., Ltd., "Robert Bosch GmbH f [david. wanq3, kaixuan. zhang, wanfang. chen, xiaonan, lu longyuan. 1i] @enn www.bos.......... com
ABSTRACT
Time series forecasting remains critical challenge across numerous domains, yet the effectiveness of complex models often varies unpredictably across datasets. Recent studies highlight the surprising competitiveness of simple linear models, suggesting that their robustness and interpretability warrant deeper theoretical investigation. This paper presents systematic study of linear models for time series forecasting, with a focus on the role of characteristic roots in temporal dynamics. We begin by analyzing the noise-free setting, where we show that characteristic roots govern long-term behavior and explain how design choices such as instance normalization and channel independence affect model capabilities. We then extend our analysis to the noisy regime, revealing that models tend to pro- duce spurious roots. This leads to the identification mitigating the influence of noise requires disproportionately large training data, highlighting the need for structural regularization. To address these challenges, we propose two complementary strategies for robust root restructuring. The first uses rank reduction techniques, including Reduced-Ran -aak Regression (RRR) and Direct Weight Rank Reduction (DWRR), recover the low-dimensional latent dynamics. The second, a novel adaptive method called Root Purge, encour- ages the model learn noise-suppressing null space during training. Extensive experiments on standard benchmarks demonstrate the effectiveness of both ap- proaches, validating our theoretical insights and achieving state-of-the-a of-the-art results in several settings. Our findings underscore the potential of integrating classical theories for linear systems with modern learning techniques to build robust, inter- pretable, and data-efficient forecasting models. The code is publicly available at: https://github..co wwngaaauuzzzuuooouuuuuuu
key data-scaling property:
INTRODUCTION
Time series forecasting is foundational task in wide range of critical applications, including finance, weather prediction, traffic modeling, and energy systems (Hamilton 2020). Despite its importance, long-term forecasting remains particularly challenging problem due to inherent uncer- tainty, noise, and complex temporal dependencies real-world data (Kong et al. 2025). In response, the research community has devoted substantial efforts to developing increasingly sophisticated model architectures (Woo et al., 20033] Xu et al. 2023 Zhou et al. 2022; Nie et al. 2022), from deep recurrent networks to attention- based transformers, in attempt capture long-range dependencies and improve accuracy.
However, emphasized in the position paper (Brigato al. 2025). ), single model consistently outperforms others across all datasets and forecasting horizons. This observation echoes the "No Free Lunch" theorem (Adam al.] 2019); in the absence of strong assumptions domain-specific priors, it is impossible to design a universally superior forecasting model. This limitation prompts rethinking of current approaches and highlights the need for more fundamental, theory-driven insights into what makes a time series model effective.
Corresponding davi wang3lcn www.seen.............mmm. com

---

Published as a conference paper at ICLR 2026
Linear Models on Time Series Analysis: Y XW
Generalization foundations guide analysiss of modaling principles
Algorithm connects back to characteristic roots: Prop.
Generalization Foundations: Characteristic Roots
Y; + 0: + + 4p3t-n = 0 e> (7,
Fact 1 + Prop.
General Modeling Principles
Algorithm Design
Noise-fres freee-freeeee Case:
Noisy Case:
Rank Reduction Prop, 2,7 V
Essence:
Roat Purge
Lookback Window
Scaling Channel Independence Prop. 1, 6,6
Analysis mothvates algorithm design
Rank Pank-Nulityy Thm.
|Y XW]= I[Y xW
Forerasting Horizon
Instance Normalization Fact 2 + Prop.5
Error Bound: Prop. &
Relationship
V Error Bound: Prop 12
Robustness: Prop.
Convergence: Prop. 15
Figure 1: Structure of the paper and its main contributions.
Recent studies show that linear systems, despite their simplicity, often match or exceed the perfor- mance of complex nonlinear models, particularly in long-term forecasting (Zeng et al.) [0233 Toner and Darlow 2024 Li et al. 2023). Their robustness, interpretability, and analytical clarity make them strong foundation for forecasting. Motivated by this, we develop a theoretical framework to
study the core properties of linear systems, focusing on the role of dominant characteristic roots that encode the essential structure of the data. At the same time, real-world time series are rarely clean; they are typically contaminated with noise that obscures the underlying structure (Lim and Zohren 2021). Complex deep learning models especially prone to overfitting such noise, often resulting in poor generalization and unreliable forecasts. To address this, we draw inspiration from recent findings (Shi et al. 2024) showing that only a small subset of data components those capturing the essential structure meaningfully impact the prediction accuracy. In linear dynamical systems, these correspond to dominant characteristic roots that govern the system behavior. Our approach focuses on identifying these core roots while systematically suppressing noise and spurious dynamics.
In this work, we present systematic analysis of linear models for time series forecasting, emphasizing the role of characteristic roots in determining model expressivity and dynamic behavior (refer to [ a road map of this paper). We begin by examining how forecasting horizon and lookback window interact with characteristic roots in noise-free settings, showing that common practices such modeling naturally arise from this framework.
as instance normalization and Srannel-independeen We then extend our analysis to noisy settings, where models tend to learn spurious roots aatifacts of noise that distort prediction and obscure true dynamics. This reveals a key data-scaling property: mitigating the impact of noise demands significantly more training data. This reduces data efficiency and underscores the need for structural regularization. Motivated by these insights, we propose two complementary strategies for robust root identification. The first leverages rank reduction, including Reduced- -ank Regression (Izenman 1975) and Direct Weight Rank Reduction, to enforce a low-dimensional structure aligned with the latent dynamics. The second introduces Root Purge, a novel adaptive training method that promotes the learning of an appropriate null space, actively suppressing noise while preserving informative signal components. We evaluate both approaches on range of standard time series forecasting benchmarks and demonstrate that they consistently enhance model robustness and accuracy, often achieving state-of-the-arr performance. Side experiments and controlled toy examples further validate our theoretical insights. Overall, our findings emphasize the value of integrating classical linear system theory with modern optimization techniques, and pave the way for future work on robust, interpretable, and scalable forecasting models. The contributions in this paper can be summarized as follows:
Theoretical Analysis: We provide a systematic study of linear models for time series forecasting, analyzing the role of characteristic roots in both noise- free and noisy settings, and uncovering a key data-scaling property that motivates structural regularization.
Proposed Methods:
We introduce two strategies for robust root identification: rank reduction techniques enforce low-dimensional structure, and Root Purge, novel adaptive training method that suppresses noise via null-space learning.
Empirical Validation: We verify the effectiveness of our approaches through strong performance standard benchmarks. We also validate theory with additional controlled experiments.
2

---

Published as a conference paper at ICLR 2026
2 BACKGROUND
Related Work. Time series forecasting aims to predict future values by learning patterns like trends and seasonality from historical data. Various modeling paradigms have emerged to tackle long-term
forecasting challenges (Kong et al. 2025). Transformer-based models have been prominent: NNe et al. (2022) segments series into patch-lever tokens with channel-wise modeling, while Zhou al. (2022)
combines aysonall-rend decomposition with transformers, Woo et al. (2023) proposes time- index models that condition on future timestamps to support both interpolation and extrapolation. Beyond time-domain methods, frequency-based approaches like Xu et al. (2023) apply low-pass filtering and complex linear layers to enhance long-term modeling. In parallel, minimalist models have gained traction: Lin et al. (2024) achieves strong results with just 1,000 parameters via structured sparsity, and Zeng et al. (2023) shows that simple linear models can outperform transformers by separately modeling trend and seasonality. Recent evaluations (Brigato et al. 2025) show that no single method dominates across all settings, emphasizing the need for diverse benchmarks and context-aware model selection. Motivated by this, we pursue a theoretical perspective on time series forecasting, aiming for models that balance simplicity, interpretability. and robustness across diverse temporal dynamics. To detailed related work and further background, please refer Appendix B.
Problem Formulation. Let {yr)em be a multi-channel time series of length T, where each observation yt T represents an m-dimensional vector time t. The goal of multi-horizon forecasting is to learn mapping f that predicts future values
Y YOLHHH yc VI-L+1),
where Y (++++ 18411 Yt+Hl is an m H matrix of forecasts. Here, H is the length of forecasting horizon, I. is the length of lookback window, and f is the forecast model, which is required capture temporal dependencies, account for noise and non-stationarity, and generalize historical patterns unseen future data.
Preliminary. We consider the class of single-channel (i.c., m = 1) homogeneous linear difference equations due their analytical tractability in modeling temporal dependencies, taking the form:
yt + 019t-1 + ..... Op3/tp O,
where p is the order of the difference equation and {ar) are constant coefficients. The general solution at time step a linear combination of characteristic roots raised to power
y1 Ciri + C2rá + .... Cpr
where {C+)/- are constants determined by initial conditions, and {r:)p=1 are obtained by solving the characteristic equation: rP + a-rd-1 +... .... + ap = O.
3 THEORETICAL ANALYSIS
Linear dynamical models strike a valuable balance between analytical simplicity and expressive power, making them effective capturing temporal patterns such as trends, oscillations, and exponential behaviors. In this section, we explore the foundations of linear modeling in time series forecasting. Our goal is to understand the principles governing linear dynamics through the lens of characteristic roots, enabling us to interpret, justify, and generalize widely used modeling choices.
We begin by formalizing the core problem following standard conventions. Given normalized single-channel time series, we define linear forecasting as the solution to the least-squares objective?
min Y Y W W
where Yis. E R NXL and Yim E NR represent the collection of N history and future segments from the same normalized sequence, W R represents the coefficient matrix. Each row in Y his
'erre we consider the simplest case where all characteristic roots are distinct.
"The omission of hias term is justified by the fact that linear difference equations with constant biases be algebraically transformed into equivalent homogeneous systems; Appendix H.. and
and C. T for justifications.
3

---

Published as a conference paper at ICLR 2026
and Yun. denoted as Vhis E RR Ytu RR, corresponds to observed history and future segment, respectively. Solving Equation (1) is equivalent to independently estimating the j-th column Y fut. corresponding to the j-th forecasting horizon (j steps ahead to the future), by solving:
min W
Σ
yt+s
Σ Wky 1155yy1k41 v1-k41)
j j= H.
This formulation reflects a regression-bes based interpretation of temporal forecasting and serves as a direct proxy for learning the underlying recurrence relations.
3.1 ROLE CHARACTERISTIC ROOTS AND NOISE- LINEAR MODELING PRINCIPLES
Recall from Section 2 that for linear models governed by difference equations, solutions are de- termined by their characteristic roots. Therefore, we can arrive at Fact as follows, derived from the properties of general solutions. This highlights a key generalization property of linear models: the ability to cover wide range of temporal behaviors (details Appendix D.1] Corollary [ and Corollary 2 through appropriate choice of roots, rather than through complex parameterization.
Fact 1. A linear model can represent any time series whose characteristic roots are subset of its
For better illustration of this fact, we provide toy example in Appendix C.3 Based Fact [ in the noise-free setting, the optimization objective defined in Equation ①1 can often achieve zero given appropriate model capacity. This allows us convert linear time series model into difference equation and study the impact of key design choices of forecasting horizon and lookback window.
It is common practice (Xu al. 2023 Lin al. 2024 to model each forecast step independently as in Equation (2). resulting in // separate regression problems, each corresponding to specific forecasting horizon. Further, is common that we use long lookback window length L, whereas the underlying process follows a simpler minimal recurrence relation of order K, with K < L. Both of these choices introduce redundancy not detrimental to learning and can, in fact, offer flexibility in parameterization. We summarize our key insight as follows, with more details in Appendix C.2 and D.2
input representation. Nonetheless, this redundancy
Fact 2. The characteristic root set of a linear higher-horizon model, or one with an extended lookback window, always preserves, as subset, the roots that govern the fundamental system dynamics.
This observation offers theoretical support for independently modeling each horizon, as higher- horizon models remain consistent with the system's 's true dynamics. Moreover, increasing the lookback window does not alter the set of roots but rather introduces multiple equivalent representations.
We highlight two common time series modeling techniques that align with our framework above, with more details in Appendix C.4 A further empirical study of and an alternative technique called INC is provided Table 3 Section 5.4
(CI) modeling
Remark 1. Instance normalization introduces unit root, allowing the model to generalize across sequences with arbitrary mean shifts. inddpee-independent modeling remains effective when the model has sufficient degrees of freedom capture the union of characteristic roots across all channels.
3.2 DATA SCALING PROPERTY UNDER NOISY OBSERVATIONS
In practical applications, time series data are often contaminated with stochastic noise, which presents fundamental challenge for model estimation and generalization. In the presence of noise, the least-squares loss for a linear forecasting model takes the following form for single segment:
E |I( W yir.) (E W [m))
where Yiur yi, are noise-free free history and future segments, respectively. and Emis- tu denote additive noise these segments. This loss naturally decomposes into two components: signal fitting error and noise-induced error. In the over- iee-ideel case where the learned weight matrix W perfectly recovers the signal dynamics, only the noise term E [|| T emis remains.
4

---

Published as a conference paper at ICLR 2026
To better understand the noise sensitivity of mean squared error (MSE)-based linear models, analyze a simplified setting in which both inputs and outputs consist solely of Gaussian noise. The following proposition summarizes such asymptotic behavior (see details in Appendix C.5 and D33
For linear model forecasting a zero-mean noise with finite second moments, the learned weights converge at rate proportional to OL1/VT), where T is the length of the observed time series.
Proposition
This result illustrates an important limitation of classical least squares training: despite its consisteney and unbiasedness, convergence is slow under noise. Even with perfect signal recovery, noise effects decay only a sublinear rate. This scaling behavior highlights that MSE-based training is datf-inefficient in high-noise regimes, requiring a substantial number of observations to achieve low-variance estimates of the underlying dynamics.
4 NOISE-AWARE LINEAR FORECASTING VIA ROOT RESTRUCTURING
In noisy environments, the data-scaling property highlights key limitation of linear forecasting models: robustness to noise requires a substantial amount of data. This challenge is compounded when the model must also accurately recover the underlying signal dynamics, misidentifying them leads to fitting errors. From the perspective of characteristic roots, reliable forecasting hinges on correctly identifying and preserving the roots that capture the signal's temporal structure. However, noise can obscure these roots. To address this, we next present two complementary approaches designed to improve root restructuring and enhance robustness in practical settings.
4.1 RANK REDUCTION METHODS
In time series forecasting, the lookback window length L and the structure of the underlying dynamics jointly determine the model's expressive capacity. In the absence of noise, the matrix Y formed by slicing deterministic time series has rank min(L. (L, K), where K denotes the number of characteristic roots. However, once noise introduced, this low-rank structure becomes obscured the observed data matrix typically becomes full rank with high probability, even when the underlying signal remains confined lower-dimensional subspace.
This observation motivates the use of rank reduction a denoising strategy. A straightforward, though suboptimal, approach involves applying truncated singular value decomposition (SVD) directly to the history matrix Yii and future matrix Y By projecting these matrices onto lower-dimensional subspaces, one can suppress the variance introduced by noise and partially recover the latent signal structure. However, direct manipulation of Y his and Yıu is neither necessary shown that imposing a low-rank constraint on the model's weight matrix W achieves an equivalent effect (see Proposition 2 below). This constraint implicitly projects the input and output data learned low-dimensional subspaces during training, without the need to alter the raw sequences themselves. low-rank W functions as a bottleneck, aligning Yiis and fur along directions of maximal shared variation and filtering out noise-dominated components.
ideal. It can be
Proposition 2. Constraining W to be low-rank implicitly projects Yos and Y onto low-dimensions subspaces.
See Appendix D.4 for a detailed version and its proof. See also Appendix D.6 for how this improves noise robustness. To operationalize this idea, we propose two practical strategies for incorporating rank constraints into linear models, with detailed algorithms provided in Algorithms [ndd
Reduced- Rank Regression (RRR). This approach (Izenman 1975) explicitly enforces low-rank constraint in the optimization target. It begins by computing the ordinary least squares (OLS) solution, then projects the forecast outputs onto lower-dimensional subspace using truncated SVD. The weight matrix is re-estimated to best match this projected output. By restricting the model to operate within smaller set of latent directions, RRR compresses the mapping from inputs to outputs and aligns both with a shared low-dimensional representation. This joint dimensionality reduction often improves generalization, particularly when the underlying data-generating process is itself low-rank.
5

---

Published as a conference paper at ICLR 2026
Algorithm 1 Reduced-Raale Regression
Algorithm 2 Direct Weight Rank Reduction
Input:
Yiis. Yiu P;
Input: Y Yiis. Yur. P;
Output: W:
Output: W;
1: compute the OLS solution:
WOLS
I: compute the OLS solution:
WOLS
(Y Y his Y
his fm:
(Y his Y his) -
his Yimi
2: calculate estimation: Yiu- = Yis Wors
2: perform SVD on WOLS: WOLS UEV
3: perform SVD on Yuu: Yíu UEVI:
4: truncate to rank pt: W RRRR WoLS V
3: truncate to rank pt WDWRR = U,
5: return WRRR;
4: return WDWRR:
Direct Weight Rank Reduction (DWRR). In contrast to RRR, DWRR applies rank reduction directly on the weight. The model first trained without any constraints. Truncated SVD is then applied post-processing step to produce low-rank approximation by discarding the smaller singular values.
See Appendix D.5 for additional theoretical analysis of Bank-Reduction methods.
4.2 ROOT PURGE FOR DYNAMIC RANK ADJUSTMENT
Previously, we described how post-training SVD-based root restructuring offers simple way to enforce low-rank structure. However, this approach is rule-based and depends on fixed assumptions, such accurate rank estimates, that may not hold in practice. To overcome these limitations, propose a more flexible alternative: an adaptive, training-integrated method for root restructuring
We introduce modified training loss designed enable online root restructuring by encouraging the model to learn a dynamically adjusted null space. Specifically, we define the following loss function:
min Yiu Sw (iis) +A 19 P (Y fm Gw mn))))l?
roct-soeking
root-purging
R NxH denotes linear transformation governed by the learned weight matrix NxH + R xxI. ensures dimensional consistency by either cropping or zero- padding the outpu The loss consists of two complementary components: (1) The root-secking term reflects the standard prediction loss commonly used in time series forecasting: (2) The root- porringgg term serves regularizer, encouraging the model learn null space that suppresses noise.
Here, Gw: RRXL
W. The operator
Intuition Behind Root Purge. The root-purging term is grounded in straightforward yet powerful principle: when a model encounters pure noise which is assumed to have zero mean, its output should ideally be zero. This term operates in stages. First, it estimates the noise by computing the residual between the model's prediction and the ground truth. Second, it re-applies the model to this residual assess whether lies in the null space of the transformation. If the residual indeed represents noise and the model has correctly captured its null space, the resulting output should be close zero. Although the purging term is unlikely to vanish completely since noise typically spans a full-rank space its minimization encourages the model to distinguish signal from noise. In doing so, it promotes the discovery of a low-rank structure that aligns with the underlying temporal dynamics, enhancing robustness to stochastic perturbations.
Optimization in Time vs. Frequency Domains. The proposed loss formulation is agnostic to the domain in which the optimization is performed, as long as the transformation applied by the model remains linear. Specifically, the function Gw used in both the root-secking and root-purging terms can be defined differently depending on the chosen representation:
IIn practice: if H < we zero-pad the output length T. and scale X by T./ L/H;if L,, crop to the first columns and keep À unchanged.
*For more memory-efficient training, apply stop-gradient operation to P(-). Empirically, this has minimal impact on the performance of Root Purge. See Appendix F.. for further details.

---

Published as a conference paper at ICLR 2026
Time domain: Gw is the linear transformation defined by the weight matrix Wr-
Frequency domain: Gw defined as F We F, where FF and FF1 denote the forward and inverse discrete Fourier transforms, respectively.
The second formulation effectively applies linear filter the frequency domain, making especially well-suited for signals with pronounced periodic or oscillatory patterns. By enabling optimization in either the time or the frequency domain, our framework offers practitioners the flexibility align model design with the inherent structure of their data, enhancing both computational efficiency and interpretability, without compromising the theoretical guarantees of our approach.
Relationship between Root Purge and Rank Reduction. The root-purging mechanism funda- mentally connected to rank reduction through the rank-nullity theorem (Axler (015), which states that increasing the dimension of a linear transformation's null space necessarily reduces its rank. This relationship allows the model dynamically adjust the rank of its transformation matrix during training. The optimization balances two opposing forces: if the rank is too low to capture the underlying dynamics, the prediction (root-seeking) loss increases, encouraging the model to raise its rank; if the rank is high enough to fit the signal, the root-purging term becomes more influential, driving the model expand its null space-thus reducing rank and suppressing noise. Through this adaptive process, the model self-regulates its capacity, learning to extract the underlying temporal structure while remaining robust stochastic perturbations.
To see additional theoretical analysis of Root Purge, please refer to Appendix D.7
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Datasets. We evaluate our methods on several widely used real-world datasets for long-term time series forecasting, including Traffic, Electricity, Weather, Exchange, and ETT (Zhou et al. (021). Due to our focus on data scaling properties, we present results from smaller-scale datasets like ETT, Exchange, and Weather in the main text, with full set of results provided in the Appendix E
Baselines. Despite our thcoretical analysis being primarily grounded in linear models, include baselines from a broad range of state-of-the-art time series forecasting methods. Specifically, we evaluate against three major categories: (1) transformer-basee models: FEDformer (Zhou al.)
2022) and PatchTST (Nie et al. (2022); ;); (2) convolution- based models: TimesNet (Wu et al. (022)..
TSLANet (Eldele et al. 20224). and Plain FilterNet (Yi al. [2024); ; and (3) linear models: FITS (Xu
et al.
2023) SparseTTF (Lin et al. 2024 and DLinear (Zeng et al.
2023
5.2
MAIN RESULTS
We evaluate our structural regularization approaches for linear models. Table [ shows the results with rank reduction using RRR and Root Purge the frequency domain, and results on time domain models are given in Appendix E.. and E.8 Experiments are repeated five times, and we report the average performance. The confidence interval for Root Purge onitted, as run-to-run differences are negligible except for the Exchange dataset (within #0.002). Both RRR and Root Purge consistently outperform other baselines. RRR surpasses methods that require extensive hyperparameter tuning, providing simpler rank adjustment without retraining. Meanwhile, Root Purge pushes the performance limits of linear time series forecasting across multiple benchmarks while maintaining simplicity and computational efficiency. These methods are especially effective smaller datasets, where models relying solely on data scaling tend to underperform, in line with our theoretical analysis.
5.3 THEORETICAL PROPERTIES VERIFICATION
In this section, validate the theoretical properties of our proposed methods in three stages: from singular value shrinkage (the mathematical mechanism) to improved data scaling (the key practical challenge) and finally back characteristic root restructuring (the foundational motivation). We
7

---

Published as a conference paper at ICLR 2026
Table 1: Forecasting result for horizon H {96, 192, 336, 720} with lookback window of length L 720. For RRR, we tune the rank on the validation set, select the top three with lowest validation MSE, and report the best test result. For Root Purge, we select the best test MSE over A E [0.125, 0.25, 0.5]. The best results are highlighted in red, and second-best in blue.
Dataset
H
FEDiormer
FillerNet
TSLANet
TimesNet
PStchTST
DLinear
SpareTTF
HITS
RRR
Root Purge
0.375
0.386
0.387
0.384
0.385
0.384
0.379
0.367
0.359
Exelange Weateer B7722 ETTml ET EPhh
192
0.427
0.420
0.421
0.436
0.413
0,443
0.4013
0.414
0.401
0,394
336
0.459
0.449
0.468
0.491
0.440
0.446
0.434
0.435
0.430
0.423
720
0.484
0.500
0.529
0.521
0.456
0.504
0.426
0.431
0.425
0.421
0.340
0.309
0.299
0.340
0.274
0.282
0.294
0.272
0.268
0.268
0.433
0.376
0.369
0.402
0.338
0,350
0.331
0.329
0.328
336
0.508
0.418
0.390
0.452
0.367
0.414
0.359
0.354
0.352
0.355
0.484
0.462
0.588
0.376
0.377
96
0.362
0.315
0.307
0.338
0.292
0,101
0.314
0.310
0.305
0.305
0.374
336
0.442
0.391
0.384
0,410
0.365
0.371
0.369
0.366
0.363
0,360
720
0,483
0.457
0.471
0.478
0.419
0.426
0.418
0.415
0.414
0.412
96
0.189
0.100
0.197
0.187
0.163
0.171
0.165
0.163
0.161
0.161
192
0,256
0.236
0.251
0.249
0.219
0.237
0.218
0.217
0.216
0.216
336
0.326
0,292
0.303
0.321
0.276
0.294
0.272
0.269
0.268
0.269
720
0,437
0.366
0,378
0.408
0.368
0,426
0,350
0.350
0.348
0.350
0.149
0.171
0.172
0.151
0.174
0.140
0.142
192
0,292
0.194
0.219
0.219
0.195
0,217
0.215
0.188
0.182
0.186
720
0.447
0.316
0.332
0.365
0.321
0.352
0,318
0.309
0,304
0,310
0.088
0.084
192
0,271
0.230
0.243
0.226
0.284
0.182
0.182
0.177
0.174
0.172
336
0,460
0.384
0.484
0.367
0.448
0.330
0.330
0.331
0.324
0.324
720
1,195
1.062
1.079
0.964
1.092
1,060
1.051
0.936
0.915
0.941
Number of 1 Places
0
0
0
0
2
0
0
9
13
then further assess their sensitivity hyperparameters and the possibility of alternative architectural choices in the next section.
Effect on Singular Value Spectrum. While RRR performs explicit rank reduction on the weight matrix, the Root Purge performs rank reduction implicitly via the rank-nullity tradcoff. In this section, we perform a case study on ETTh1 I and ETTml to show how the singular values of W change after training via Root Purge. Since our model is trained in the frequency domain, we revert the transformed model back to the temporal domain for singular value decomposition (SVD). By leveraging the properties of linear operators, we input the identity matrix to obtain the corresponding weight matrix in the time domain. As shown in Figure 2] the degree of Root Purge does not affect the overall trend of the singular value distribution across both datasets. Increasing the regularization coefficient X leads to a more pronounced shrinkage of singular values. Notably, significant singular values remain unchanged even with large regularization coefficient. This underscores the role of the root-seeking loss, which prevents the noise-free signal subspaces to be suppressed.
Singular Spectrum ETT-1
Singular Value
Nogntudd stgas)
102
10-
26
100
350
200
250
53
230
15c
250
303
Sigular/onte
Siguuarr Value
Figure 2: First 336 singular value magnitudes ETThi and ETTml under different values of (log scale). As \ increases, Root Purge pushes the weight matrix W to have more smaller singular values, while the significant singular values remain largely unaffected.
Data Scaling& & Noise Robustness Property. We further use a toy example to illustrate the scaling behavior of different methods under noisy conditions. Specifically, we examine the effects of two key factors: training data volume and noise magnitude. We construct synthetic signal {ye )ee with both trend and periodic components: y/ = (2t) cos(5t) + 0.5l Et, where ~ (0 1)
8

---

Published as a conference paper at ICLR 2026
and observations are taken at increments of 0.01 in t. To examine data scaling behavior, fix the noise level at o = 0.5 and vary the size of the training dataset. As shown in (leff,,)(eeft), baseline models struggle under limited data and require substantially more samples to mitigate the impact of noise, revealing a clear data scaling inefficiency. In contrast, our proposed methods, RRR and Root Purge, maintain consistently strong performance across all dataset sizes, with minimal performance drop as data size decreases. To assess noise robustness, we fix the training interval to & E [00] and vary the noise level o. Results in Figure (right) show that while baseline models degrade more significantly with increasing noise, RRR and Root Purge remain stable and robust throughout.
Scaling Property Linear
Nose Robustness SOTA Linear Models
Mean 2x0
from GT TOL
0000-
Ground
Lheer
Linear
Sparattr Rext Paran
6.06
Rest Parge
0.04
0.29
3.2
324
100
200
400
00.03
3.50
Notsc
Figure 3: Data scaling and noise robustness of state-of-the-ar linear time-series models. (left) RRR and Root Purge exhibit near-constant performance in data-scaling benchmarks. (right) Both methods exhibit robust performance under increasing noise levels, outperforming bascline models.
Table 2: Comparison of Root Distance to Ground Truth across models (root pairings determined by Hungarian algorithm (Kuhn 1955)). The results demonstrate the effectiveness of Rank Reduction and Root Purge in recovering roots closer to the true values compared standard linear models.
Model
Root Distance to Ground Truth (mean + std)
RRR
0.036 1 0.014
Root Purge
0.045 + 0.009
Standard Linear Model (OLS)
0.064 + 0.025
Analysis of Characteristic Roots. Finally, to close the loop of on our framework, we hereby directly evaluate how Rank Reduction Regression (RRR) and Root Purge affect the learned root distribution in a controlled synthetic setting. Such a direct comparison is usually infeasible on real-world data because the underlying noise-free dynamics and their true characteristic roots unknown. Therefore, we conduct experiments using the aforementioned synthetic dynamics: = sin(2t) + cos( + 0.5l
St- For each method, we compute the mean and standard deviation of the distances between all learned roots and the ground truth roots, with root pairs matched using the Hungarian algorithm (Kuhn) 1955). As shown in Table 2 the roots estimated by OLS tend to drift farther from the ground-truth characteristic roots, whereas those estimated by Root Purge and RRR remain closer. This observation confirms that our proposed techniques indeed help prevent the learning of spurious roots, thereby leading to the performance improvements demonstrated earlier. Additional experimental details and qualitative evaluations are also provided in Appendix E.13
5.4 PRACTICAL ROBUSTNESS ANALYSIS
Hyperparameter Sensitivity. In our Root Purge method, as in Equation (3) the only tunable hyperparameter X, which controls the balance between prediction error and regularizer for noise suppression. This section investigates the model's sensitivity to this hyperparameter on the E and ETTml datasets. We train a linear forecasting model with Root Purge using: E [1/22, 1/16, 1/8, 1/2, 2), and report the average performance across all forecasting hori- zons. Figure shoows that the performance improves initially with increasing X, reaches an optimum, and then deteriorates when À becomes too large. Notably, even very small values of À enhance model performance. This positive effect remains consistent across a broad range of À values, highlighting the robustness of our method.

---

Published as a conference paper at ICLR 2026
Root
ETTh1
Sendiivity ETTm1
Sesciter Lamt)
1.622
3.435
Avcrigg
2.40
0.90
612
20.6
214
1/4
1022
244
62
Regubatzation Confrideet
Regulviiee Confficcan (41
Figure Average forecasting MSE on ETThl and ETTml across horizons = (96, 192, 336, 720] for different values of A. Results indicate that wide range of À improves predictions, whereas larger values may cause over-regularization. A break-down table for each horizon is in Appendix E..
Channel Independent vs. Individual Channel Modeling. The results presented so far are mainly based the channel independent (CI) strategy, which models all channels using shared weight matrix. A key advantage of CI is its ability to aggregate noise from all channels. This serves as implicit data augmentation and, by Proposition [1] enhances model robustness. However, the expressivity of CI is inherently limited by its shared parameterization, creating a tradeoff between robustness and modeling capacity. Here, we consider an alternative design: individual channel (INC) modeling, where each channel is assigned its own linear model (see Table for more details of CI and INC). This increases model capacity and allows the system better adapt to channel -speciffc dynamics. However, without the benefit of cross-channe noise sharing, INC models are more sensitive to noise and often underperform in raw form, as shown in Table 3 Notably, when combined with our proposed Root Purge strategy, the INC formulation becomes significantly more robust. Root Purge enables each individual model to effectively suppress channel-specific noise and isolate meaningful dynamics. In several cases, this leads to substantial performance gains over CI models. For example, on the ETTml dataset, the INC+Root Purge model even outperforms the previous non-linear state-of-the-art PatchTST (Nie et al. 2022) on the 96- and 192 step forecasting tasks. These results highlight how our method unlocks alternative, more expressive architectures that would otherwise be infeasible.
Table 3: Performance of INC linear models with Root Purge. While standalone INC models typically underperform due to limited data, applying Root Purge significantly mitigates this issue.
Dalaeei
ETThi
11
E17ml
ErTin
Honizn
192
720
192
330
729
95
336
720
0L375
11411
0331
0.353
0.144
0.152
11217
0.260
0.350
Ponge
0.182
0.188
0.239
INC Parge
0.418
0.161
CONCLUSIONS
This paper presents a comprehensive study of linear models for time series forecasting, focusing on the role of characteristic roots in shaping model expressivity. We propose two complementary methods, Rank Reduction and Root Purge, for improved forecasting through root identification. Experiments validate our theoretical claims and demonstrate the effectiveness of both methods across a range of forecasting tasks, More detailed discussions and limitations are provided in Appendix [F]
REPRODUCIBILITY STATEMENT
Detailed descriptions of the metric and experimental setup, dataset we used, and baseline information
are provided in Section 5 Appendix F.I F.2 and E.3 Detailed descriptions, assumptions, and proofs
of theories (Facts, Propositions, Remarks, Claims, and Corollaries) can be found Appendix D Source code is available in the supplementary material as a zip file. You can also find our source code at htttts://github.bc http::/gggthuucooo//a Wangzzzzzzz/RooPPrrgg
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Emmanuel Abbe. Community detection and stochastic block models: recent developments. Journal of Machine Learning Research, 18(177): 2018.
Stavros P Adam, Aaimatios-Agglloo N Alexandropoulos, Panos M Pardalos, and Michael N Vrahatis. No free lunch theorem: A review. Approximation and optimization: Algorithms, complexity and applications, pages 57-82, 2019.
Anish Agarwal, Muhammad Jehangir Amjad, Devavrat Shah, and Dennis Shen. Model agnostic time series analysis via matrix estimation. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 2(3):1-39, 2018.
Sheldon Axler. Linear algebra done right. Springer, 2015.
Lorenzo Brigato, Rafael Morand, Knut Strømmen, Maria Panagiotou, Markus Schmidt, and Stavroula Mougiakakou. Position: There are no champions in long-term time series forecasting. arXiv preprint arXiv:2502.14044 2025.
Steven L Brunton, Marko Budišić, Eurika Kaiser, and J Nathan Kutz. Modern koopman theory for dynamical systems. arXiv preprint arXiv:2102. 12086, 2021.
Marko Budišić, Ryan Mohr, and Igor Mezić. Applied koopmanism. Chaos: An Interdisciplinary Journal of Nonlinear Science, 22),), 2012.
John Parker Burg. A new analysis technique for time series data. Paper presented at NATO Advanced Study Institute on Signal Processing, Enschede, Netherlands, 1968, 1968.
Yuxin Chen, Jianqing Fan, Cong Ma, and Kaizheng Wang. Spectral method and regularized mle both optimal for top-k ranking. Annals of statistics, 47(4)22204, 2019.
Yuxin Chen, Yuejie Chi, Jianqing Fan, Cong Ma, et al. Spectral methods for data science: statistical perspective. Foundations and Trends@ in Machine Learning, 14(5):566-800) 2021.
Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan Mathur, Rajat Sen, and Rose Yu. Long-tem abs/
forecasting with tide: Time-series dense encoder, 2024. URL
2304. 08424
Chandler Davis and William Morton Kahan. The rotation of eigenvectors by perturbation. iii. SIAM Journal on Numerical Analysis, 7(1):1-46, 1970.
James Durbin. The fitting of time-series models. Revue de l'Institut International de Statistique, pages 233-244, 1960.
Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, and Xiaoli Li. Tslanet: Rethinking transformers for time series representation learning, 2024.
http://:///xxw...///// /
2404.088 08472
Walter Gautschi. On inverses of vandermonde and confluent vandermonde matrices. Numer Math.,
4(1):117-123, December 1962. ISSN 0029-599X. doi: 10.10077BF00338300. URL https
7/doi http:/100.0007/560 0000000FFF0 386302
Matan Gavish and David L. Donoho. Optimal shrinkage of singular values. IEEE Transactions on Information Theory, 63(4):2337-2122 2017.
James D Hamilton. Time series analysis. Princeton university press, 2020.
Hossein Hassani. Singular spectrum analysis: methodology and comparison. 2007.
Zhanxuan Hu, Feiping Nie, Rong Wang, and Xuelong Li. Low rank regularization: A review. Neural Networks, 136:218-233 2021.
Qihe Huang, Zhengyang Zhou, Kuo Yang, Zhongchao Yi, Xu Wang. and Yang Wang. Timebase: The power of minimalism in efficient long-term time series forecasting. In Forry-second International Conference on Machine Learning, 2025. URL https:://///c. httpp:://pperreviww net wtt.::/.eweeeeeeeeeeeemmm forum?id=
GhTONOMEOD
11

---

Published as a conference paper at ICLR 2026
Alan Julian Izenman. Reduced-ran regression for the multivariate linear model. Journal
of Multivariate Analysis. 5(2):248-200) 1975. ISSN 0047-259X. doi: https://oi.org//
1016/00441-559X7759900222 URL
att.iclle/511/00477557777
Ravindran Kannan, Santosh Vempala, et al. Spectral algorithms. Foundations and Trends@ in Theoretical Computer Science, 4(3-4))157-288, 2009.
Raghunandan Keshavan, Andrea Montanari, and Sewoong Oh. Matrix completion from a few entries. IEEE transactions on information theory, 56(6):2980-23) 2998, 2010.
N Kishore Kumar and Jan Schneider. Literature survey low rank approximation of matrices. Linear and Multilinear Algebra, 65(11):2212-2244, 2017.
Andrei Kolmogoroff. Über die analytischen methoden in der walnscheinlichkeirrccchnngg Mathe- matische Annalen, 104:415-458, 1931.
Xiangjie Kong, Zhenghao Chen, Weiyao Liu, Kaili Ning, Lechao Zhang, Syauqie Muhammad Marier, Yichen Liu, Yuhao Chen, and Feng Xia. Deep learning for time series forecasting: a survey. International Journal of Machine Learning and Cybernetics, pages 1-34, 2025.
Sebastian Krämer. Affine rank minimization via asymptotic log-det iteratively reweighted least squares. Journal of Machine Leurning Research, 26(92):1 2025.
H. W. Kuhn. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2):83 97, 1955. doi: comm/dol/aban /wwo/ob//////oo wttpa///...........oo..o.
URL https
Lonelinell www.ienelebbrrrr wi Ley. cwm
Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305.107211 2023.
Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: survey, Philosophical Transactions of the Royal Society A, 37922194))222200009 2021.
Shengsheng Lin, Weiwei Lin, Wentai Wu, Haojun Chen, and Junjie Yang. Sparsetsf: Modeling long-term time series forecasting with Ik parameters, 2024. URL
2405. 00946
Guangcan Liu. Time series forecasting via learning convolutionally low-rank models. IEEE Transac- fions on Information Theory, 68(5):3362-3380, 2022.
Yong Liu, Chenyu Li, Jianmin Wang, and Mingsheng Long. Koopa: Learning non-stationary time series dynamics with koopman predictors. Advances in neural information processing systems, 36: 12271-12290, 2023.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting, 2024. URL URLhttps
//axxiv.org/abs/22333 06625
Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecosting, 36(1):54-74 74, 2020. ISSN 0169-2070. doi: htt.a//cc..................aaaaa
URL https
/ttww.sciiitt wtw.ci/ene.reee..coooooooooooooe htwp://cceeeeeeeeecccccccc/ereee Competition.
M4
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730, 2022.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Mohsen Pourahmadi. Foundations of time series analysis and prediction theory. John Wiley & Sons, 2001.
12

---

Published as a conference paper at ICLR 2026
Adityanarayaman Radhakrishnan, Mikhail Belkin, and Dmitriy Drusvyatskiy. Linear recursive feature machines provably recover low-rank matrices. Proceedings of the National Academy of Sciences, (2((3))x2241122522, 2025.
Peter Schmid. Dynamic mode decomposition of numerical and experimental data, Journal fluid mechanics, 656:5-28, 2010.
Jingzhe Shi, Qinwei Ma, Huan Ma, and Lei Li. Scaling law for time series forecasting. arXiv preprint 1rXiv:2405.15124, 2024.
Robert H Shumway and David Stoffer. Time series analysis and its applications: with examples. Springer, 2006.
Michael Singer. Introduction the galois theory of linear differential equations. Algebraic theory of differential equations, 357:1-82, 2009.
Terence Tao. Topics in random matrix theory, volume 132. American Mathematical Soc., 2012.
Vincent Thibeault, Antoine Allard, and Patrick Desrosiers. The low-rank hypothesis of complex systems. Nature Physics, 20(2):294-300 2024.
William Toner and Luke Darlow. An analysis of linear time series forecasting models. arXiv preprint 1:Xi::2403314587 2024.
Joel Tropp et al. An introduction to matrix concentration inequalities. Foundations and Trends in Machine Learning, 8(1-2):1 2015.
Palghat P Vaidyanathan. The theory of linear prediction. Morgan Claypool Publishers, 2007.
Marius Van der Put and Michael F Singer. Galois theory of linear disferential equations, volume 328. Springer Science & Business Media, 2012.
Martin J Wainwright. High- digeesiimnsi statistics: non-asymptotic viewpoint, volume 48. Cam- bridge university press, 2019.
Hermann Weyl. Das asymptotische verteilungsgesetz der eigenwerte linearer partieller differentialgle- ichungen (mit einer anwendung auf die theorie der bohlraumstrahlung). Mathematische Annalen, 71(4):441-477 1912.
N Wiener. The interpolation, extrapolation and smoothing of stationary time series. ndre report, 1942.
Matthew O Williams, Joannis G Kevrekidis, and Clarence W Rowley. A driven approximation of the koopman operator: Extending dynamic mode decomposition. Journal of Nonlinear Science, 25(6):1307-13 2015.
Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Learning deep inde- models for time series forecasting. In International Conference on Machine Learning. pages 37217-37237. PMLR, 2023.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-temn term series forecasting, 2022.
2106. 13008
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet:
Temporal 2d-variation modeling for general time series analysis, 2023. URL httthhhtp:://aett
org abs abs 2210. 02186
bogg/bbs/0
Zhijian Xu, Ailing Zeng, and Qiang Xu. Fits: Modeling time series with 10k parameters. arXiv preprint arXiv:2307.03376. 2023.
Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Defu Lian, Ning An, Longbing Cao. and Zhendong Niu. Frequency-domee mlps are more effective learners time series forecasting, 2023. ItR.hhttss://arxttt org/ abs /211 06184]
13

---

Published as a conference paper at ICLR 2026
Kun Yi, Jingru Fei, Qi Zhang, Hui He, Shufeng Hao, Defu Lian, and Wei Fan. Filternet Harnessing frequency filters for time series forecasting, 2024. URL
2411
01623
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121-11128, 2023.
Yitian Zhang, Liheng Ma, Antonios Valkanas, Boris N Oreshkin, and Mark Coates. Skolr: Structured koopman operator linear rnn for time-series forecasting. arXiv preprint arXiv:2506.14111 2025,
Anatoly Alexandrovich Zhigljavsky. Singular spectrum analysis for time series: Introduction to this special issue. Statistics and its Interface, 3(3):255 258, 2010.
Haoyi Zhou, Shanghang Zhang. Jiegi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang Beyond efficient transformer for long sequence time-series forecasting, In The Thirty- Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Conference, volume 35, pages
Informer:
11 11115. AAAI Press, 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on PMLR, 2022.
machine learning, pages
14

---

Published as a conference paper at ICLR 2026
CONTENTS
Introduction
2
Background
3
Theoretical Analysis
3.1
Role of Characteristic Roots and Noise-Free Linear Modcling Principles
3.2
Data Scaling Property Under Noisy Observations
4
Noise-Aware Linear Forecasting via Root Restructuring
Rank Reduction Methods
4.2
Root Purge for Dynamic Rank Adjustment
6
5
Experiments
5.1
Experimental Setup
Main Results
5.3
Theoretical Properties Verification
5.4
Practical Robustness Analysis
6
Conclusions
10
Mind Map, Notations, and Summary of Theorems
17
A.I
Mind Map
17
Notation
18
A.3
Summary of Theorems
19
B Background
21
Related Works
B.2
Terminology
23
Preliminary
23
Reinterpreting Design Choices in Linear Time Series Models
26
C.I
Bras-Free Linear Models: Justification and Implications
26
Higher-Horizon Forecasting and Extended Lookback Window
................................
26
Toy Example
27
Instance Normalization and Channel Independent Modeling
................................
27
C.5
Scaling Pure Noise Series
31
D
Formal Theories and Proofs
33
Characteristic Roots Generalization
33
D.2
Linear Modeling Property under Noise-free Setting
35
15

---

Published as a conference paper at ICLR 2026
D.3
Scaling Property White Noise Series
36
Effect of Model Parameter Rank Reduction
37
D.5
Analysis of Rank Reduction
38
D.6
Rank reduction and noise robustness
40
D.7
Analysis of Root Purge
43
Analysis of Characteristic Roots
48
Experiments
49
Dataset
49
Implementation Details
50
Details of Baseline Results and Comparing with Additional Baselines
E,4
Comparing Traffic and Electricity Datasel
52
Computational Cost Complexity Analysis
52
E.6
Linear Forecaster with Bias and without Bias
53
Additional Empirical Results for Root Purge
54
E.8
Additional Empirical Results for Rank Reduction Methods
57
E.9
RRR and Root Purge on Different Lookback Window/Forecasting Horizon
58
E.
10
Short-term Forecasting An Extreme Case
59
Higher-Order Root Purge
59
E.
12
Additional Results on Data Scaling & Noise Robustness on Simulation Data
61
13 Details of Root Analysis Simulation Data and Quantitative Evaluation
61
14 Stop-Gradient and Full- Graddent Root Purge
62
Discussion
63
Limitation and Future Work
63
2 Broader Impact
63
Declaration
64
16

---

Published as a conference paper at ICLR 2026
A
MIND MAP, NOTATIONS, AND SUMMARY OF THEOREMS
A.1 MIND MAP
Thee series brrcidiig con modila tove wntersabee rnburter
surpiiing our coppriiionss antforttoon
Backroondd& &
Conduct of Shanctessiio nott Eweay
Core Objective
gtrshggs to addrus naire-
sibuad anrryeetaal data
Propoditon
Foot
Constary 2-4y dy-daanii this acirring selve.
roic Gece
Notse-Tre Dayire
rocti of rightr-
lodkitee modks
Propesitin minairiss
cyster reart
Punderentiss
Chenssweees roos mocsl
seiight for whita converte si
Proposttion Greinfrros Props
Deeortical Analydi
Asayyi lrac di aroervelty
Kairy
Ohearaation intokkom 1o01s ratkgrti
dilk
Peplication vatance rooma
Realis Meddeling Principtex
aval fir
Prepocation &-
Chaanee Chaivellllllanee www.pandeeedeeccc you rect unto
periprectvv
Chanacterttt
Pripostióe mat farlurcs Pop tans deduct suporasse
Reputatcation Linea Forecasting
Theseeicl
ree reakiity Nutur regherto
and
Mpprito ton delgg
aubepecce.
Algaritana
Hork Reduattion
DIAR
Preperitio newe nuthi-cs
Algorther
(500)
Propecation
Frodannu
Propesition impact
Treeectioal Anebyahi
Theneefood Book
Prepositon 14:
Praudtion you famertor
Roat Purge (Aclgtivv
Propecation hurga ecciniard Cus
Prepositon 18 enor to the OLD
Prepositon 15
REBU Nare sulperfom
palas than
deteetts Nodudame
sodee batee mebicd
(kppaddi Learned
buth
Experimental Valicatioo
of book Purge
Pare
Rart Purge
Sugglamertary. Anagas
Robcsirees #500boot Parge
Chassa Hoddtigg
OR ver laers
Figure 5: A mind map of the paper describing its logic flow and how organize this paper.
17

---

Published as a conference paper at ICLR 2026
This mind map summarizes the logical flow of the study, showing how the paper moves from theory to modeling principles and then to algorithm design. It begins with generalization foundations derived from the characteristic roots of linear recurrence relations, which guide the analysis of modeling principles (including lookback window, prediction horizon, instance normalization, channel indepen- dence, and data scaling laws) under both noise-free and noisy conditions. These theoretical insights motivate the development of two complementary algorithmic strategies-rank reduction (using re- duced rank regression direct weight rank reduction) and the proposed root purge method-which in turn connect back to the characteristic roots, forming closed loop between theoretical analysis and practical algorithm design. Figure 5 thus provides roadmap for understanding how the paper in- tegrates classical linear systems theory with modern learning techniques to build robust, interpretable, and data-efficient time series forecasting models.
A.2 NOTATION
Mutt-Chhmnnl Serins Nolee
No Clenonatoo
Channel 2ith Knioe
Charnel 2ith Noioe
Channd With
Langen limoaness
Niciea rree Single-Chunne Time Semser (s N0
Single Channel Thme Sorito with Notse: Dele
Sepvent
Collecting Beegmeent
Obsemvttiu Netter Ie
Collecting All Segiuutta
Sugment I
Loodback WWindian
Frowsaer
Looktsack Windows..
Freecast Uavinon:
Axingle angment: Ya. and Nii
nolio free segment:
Yig and
Hanry Matrie Ya.
Notes Tnea: Hatni: Via g.l4l
Future Marria: Yez
Notse Tres Puture Matris: Yia 1g
Figure 6: Mind map for common notations used in paper for time series dataset. A more detailed tabular description can be found in Table
Remark 2 (On the Construction of his and ria)- In our framework, Yis and Y jus are constructed such that each row corresponds to a univariate segment drawn from a shared underlying dynamical system. Specifically,
V1,1
V1,2
VI.L
91L+1
V1,
U1
92.1
V2.2
32.L
42,L+1
V2,
22
Y his
Y
LUN.1
UN.2
UN.L.
LUNLL1
UN,L+2
JN.L+
In this setting, the future segment in each row of Y immediately follows the corresponding history segment in his along the time axis. This construction especially suitable for caltti- time series where independent samples (e.g. across entities sensors) share the same temporal dynamics.
18

---

Published as a conference paper at ICLR 2026
Table 4: Summary of Notation
Symbol
Deseription
m
Number channels (multivariate time series dimension)
T
Total length of time series observations
K
Order of the underlying recurrence/dynamic system
L
Lookback window length (history segment length)
H
Forecasting horizon (future segment length)
N
Number of segments the dataset Undetermined coefficients in the general solution Characteristic roots of the recurrence relation Coefficients of the recurrence relation Regularization coefficient for root purging
(C
(ri)
(a:)
Noise level (standard deviation) in observations Selected rank for low-rank approximation Dimension alignment operator
P
Gw
Linear operator parameterized by weight matrix W Fourier transform and its inverse
F.F-1
Yos
Collection of noise-free history segments
Y
Collection of noise-free future segments
Y his
Collection of noisy history segments
fut
Collection of noisy future segments Single noise-free history and future segment
Biis Yiu
Yus: Via
Single noisy history and future segment
Ye, He
Time series observation at step (multi- and single-channel)
Y fut
Collection of all predicted future segments
W
Weight matrix of linear model; Wjs. denotes element at (j, OLS solution for weight matrix
W OLs
W RRR
Reduced- -ank Regression (RRR) solution
W DWRR
Direct Weight Rank Reduction (DWRR) solution
WT
Weight matrix in the time domain for root purging
W F
Weight matrix in the frequency domain for root purging SVD of matrix
U, U,E, V
Up. Ep, V
Rank-p truncated SVD components
Ehis, Sfut
Noise on history and future segments
Alternatively, in traditional time series analysis, common approach is Hankelization. Given sufficiently long univariate time series {yr)- the data is organized by applying a sliding window the temporal axis extract overlapping segments. In this case, N T -H L and
91
1/2
3L
UL+1
9L+2
UL+H UL+H+1
V2
V3
UL+1
VL+2
VL+3
Y
Y fur
LYN
9N+1
1N+L-1
WLL+
VL+N+1
VL+N+H-11
This formulation is widely adopted in signal processing and control theory. Under this view, each row represents segment from single long sequence, with consecutive rows overlapping temporally. Both formulations compatible with our proposed framework, and the choice between them be made depending on the application scenario and data availability.
A.3 SUMMARY OF THEOREMS
In our paper, we had proposed multiple theorems, facts, propositions, and corollaries about linear forecasting models and proposed methods. To provide comprehensive idea of how each of them contributes to our analysis and what they mean in an intuitive sense, we summarize them in Table shown below. For the detailed assumption and proofs, please refer Appendix D
19

---

Proposition 9 Published as a conference paper at ICLR 2026
Table 5: Summary of Key Theorems, Facts, Propositions, and Corollaries
Theorem
Plain-Language Explanation
Role and Significance in the Paper
Fact
A linear model can accurately represent time series if the time series' charac- teristic roots subset of the model's characteristic roots.
1. Defines core generalization logic: adapta- tion depends root subset relationships. 2. Provides theoretical basis for fitting diverse time series.
Fact
Justifies independent modeling per fore- For linear models with longer forecast- ing horizons or extended lookback win- dows, their set of characteristic roots always includes the key roots of the sys-
cast horizon. 2. Shows parameter redun- dancy enhances flexibility without harming core dynamics.
Proposition 1 (Informal version Prop. 6)
For eero-mean noise with finite ond moments, convergence rate OLS weights is O(1/vT).
Reveals limitations of least-squares in noisy data. Motivates structural regulariza- tion (RRR, Root Purge).
Proposition 2 (Informal version of Prop. 7)
Constraining weight matrix W to low- rank is equivalent projecting in- put and output into shared low- dimensional subspace.
Basis for RRR/DWRR methods. 2. Ex- plains denoising via low-rank constraints.
Proposition
Multi-channel time series with diago- nal recurrence per channel can be trans- formed to a shared-paramettr recur- rence.
Proves effectiveness of channel- independent modeling. 2. Shared weights improve robustness and reduce parameters.
Proposition 4
Two exponential combinations with dif- ferent characteristic roots that match points must match everywhere.
1. Strengthens Fact 1: root subset relation determines expressivity. 2. Provides general- ization guarantee.
Proposition
Characteristic polynomial of minimum- order recurrence divides higher-order recurrences.
Deepens Fact 2 from polynomial theory. 2. High-order models only add redundant roots.
Proposition 6 (Formal version of Prop. 1)
For a zero-mean noise with finite sec- ond moments. convergence rate of OLS weights is O(1/VT).
Reveals limitations of least-squares in noisy data. 2. Motivates structural regulariza- tion (RRR, Root Purge).
Proposition (Formal version of Prop. 2)
Constraining weight matrix to low- rank is equivalent to projecting in- put and output into shared low- dimensional subspace.
Basis for RRR/DWRR methods. 2. Ex- plains denoising via low-rank constraints.
Proposition 8
DWRR's low-rank approximation of OLS is optimal (Frobenius norm). ror sum of discarded singular values squared.
Proves DWRR optimality. 2. Guides engi- neering implementation.
RRR error is bounded by minimal sin-
Quantifics RRR approximation accuracy.
gular values of input and output data.
2. Guides rank selection.
Proposition 10
If true matrix is low-rank, truncated SVD error after noise is controlled by noise spectral norm.
Provides matrix denoising interpretation.
2. Low-rank constraints robustly isolate noise.
Proposition 11
Erorr of estimating true principal sub- space decreases with sample size.
1. Ensures statistical reliability of rank reduc- tion. 2. Supports tuning rank p with valida- tion data.
Proposition 12
OLS a critical point of Root Purge ob- jective residual and weights satisfy null-space condition.
Shows relationship between Root Purge and OLS. 2. Guides regularization for noise suppression.
Proposition 13
Deviation between Root Purge and OLS weights is bounded by minimal singular values and regularization X.
1. Quantifies correction magnitude. 2. En- sures stability of Root Purge.
Continued on next page
20

---

Published as a conference paper at ICLR 2026
Table 5 Continued from previous page
Theorem
Plain-Languaae Explanation
Role and Significance in the Paper
Proposition 14
Each singular value Root Purge weights I corresponding OLS weight singular value.
Shows Root Purge implicitly enforces low- rankness. 2. Connects Root Purge to rank reduction.
Proposition 15
Gradient descent with proper step size converges to stationary point of Root Purge objective.
1. Ensures feasihility of Root Purge optimiza- tion. 2. Justifies using OLS warm starts.
Proposition 16
If weight matrix approximation error E, root perturbation O(e).
1. Connects weight approximation to root stability, Ensures regularized model pre- serves core dynamics.
Corollary 1
Sinusoids, polynomial, exponential, and combinations can be represented by linear difference equations.
Shows linear models fit common temporal patterns. 2. Supports long-term forecasting.
Corollary 2
A linear model represents a time series iff the time series' PV extension is subextension of the model's s PV exten- sion.
1. Uses differential Galois theory to charac- terize model expressivity. 2. Shows linear models' generality for complex dynamics.
B BACKGROUND
B.1 RELATED WORKS
Time Series Forecasting, Time series forecasting (TSF) involves predicting future values by learn- ing patterns such as trends and seasonality from historical data. Over time, it has given rise to variety of modeling paradigms, each addressing the challenges of long-term forecasting from different per- spectives (Kong et al. 2025). Transformer-bart
architectures have been a dominant line of research. For example, Nie al. (022)) proposes segmenting time series into subseries-leved patches, treating each patch as an input token while modeling channels independently. Similarly, [Zoou et al. (2022) combines transformer networks with seasonal-trend decomposition, capturing broad temporal profiles via decomposition while leveraging transformers for detailed pattern learning. Meanwhile, Woo et al.) (2023) introduces deep time-index models, explicitly conditioning predictions future time indexes to better handle temporal variations and support both interpolation and extrapolation tasks. Beyond time-domain approaches, alternative methods explore Trequency-domann representations, Xu et al.) (2023) applies a low-pass filter followed by complex-valued linear transformations to strengthen long- term dependencies and smooth high-frequency noise. parallel, model efficiency and minimalism have gained attention. [Ln et al. (2024) demonstrates that strong long-term forecasting performance can be achieved with as few as 1,000 parameters through structured sparsity. Zeng et al. (2023) further questions the necessity of complex architectures, showing that simple linear models by separately modeling trend and seasonal components-can outperform transformer based models on long-term forecasting benchmarks. Finally, comprehensive evaluations (Brigato al. 2025) suggest that no single model consistently outperforms others across all scenarios. This highlights the importance of diverse evaluation settings and context-aware model selection, given the heterogeneous nature of real- world time series data. Motivated by these developments, we aim to approach long-term time series forecasting from theoretical perspective, seeking models that balance complexity, interpretability, and robustness across diverse temporal structures.
Similarly[ZZhou
Linear Prediction Theory. Linear prediction theory. rooted in the early 20th century, emerged from the study of time series analysis and signal processing, with foundational contributions from mathe- maticians like Norbert Wiener and Andrey Kolmogorov in the 1940s (Vaidyanathan (007). Wiener's work on optimal filtering (Wiener 1942) and Kolmogorov's theory on extrapolation (Kolmogoroff 1931) laid the theoretical groundwork, formalizing the idea that future values of discrete-time signal can be estimated as a linear combination of past observations. The theory gained prominence in the 1960s and 1970s with the development of efficient algorithms, such as the Levinson cevinson-Durbin recur- sion (Durbin 1960). which solved the Yule- Walker equations for autoregressive (AR) modeling (Burg) 1968). The connection between time series forecasting and linear prediction is fundamental, both
21

---

Published as a conference paper at ICLR 2026
disciplines aim to capture and exploit temporal dependencies in sequential data (Pourahmadi) 2001)) Linear prediction provides a mathematically tractable framework for time series analysis by express- ing future values as weighted linear combinations of past observations an approach particularly well-suited for stationary processes (Hamilton 2020). Starting with DLinear (Zeng et al.] 2023), which extends the linear framework by separately y modeling trend and seasonal components, recent studies (Li et al., 2023: Toner and Darlow] (024) have further explored the properties of linear models, providing practical insights for improving forecasting accuracy. Additionally, the concept of scaling laws, as explored in Shi et al. (2024), demonstrates how performance improves with larger data sizes and model capacities, highlighting both the potential and limitations of scaling in linear forecasting models. Building on these developments, we aim to enhance linear prediction models to better handle the complexities of real-world time series data.
Spectral Methods, Low Rank Structure and Koopman Theory. Spectral methods, which are algorithms built on eigenvalues and eigenvectors or singular values and singular vectors of data dependent matrices, have become widely used toolkit for extracting structure from large, noisy, and incomplete data (Kannan al.] 2009; Chen et al. 202T). They have been successfully applied in machine learning, signal processing, imaging science, econometrics and many other areas, both as stand alone estimators and as initializations for more sophisticated procedures (Abbe) 2018 Keshavan al.] 2010|/Chen Chen et al.] 2019). In the time series domain, particularly important instance of spectral methodology is singular spectrum analysis (SSA) (Hassani 2007 Zhigljavsky 2010). SSA forms a Hankel trajectory matrix from lagged copies of the series, performs an SVD based decomposition. and reconstructs interpretable components such as trend, periodicities and noise for denoising, gap filling and forecasting.
Low-rank structure is a pervasive modeling assumption for high-dimensional dynamical systems Agarwal et al. (2018) Liu (022). By positing an effective low-rank interaction matrix, one can capture large-scale behavior in a tractable form. Recent theoretical and empirical studies (Thibeault al. 2024) probe this hypothesis by analyzing singular value decay in random graphs and real networks, quantifying effective ranks, and linking rapid singular value decay to accurate dimension reductions of nonlinear dynamics. These findings show that low-rank representations enable faith- ful reduced-order models of recurrent neural networks and other complex systems. Furthermore, rank reduction and low-rank recovery play a central role across applications (Kishore Kumar and Schneider, 2017: Hu et al., (021). Beyond classical truncated SVD (hard thresholding), rich set of algorithms- convex relaxations with the nuclear norm, nonconvex optimization, iteratively reweighted schemes, and modern SVD-free iterative methods-has emerged (Chen et al., 2021 l Radhakrishnan et al. 2025). Recent advances (Krämer 2025) revisit affine rank minimization and show that asymptotic log- degeeremrnnn reweighted least squares recover low-rank solutions under broad conditions, clarifying both computational and statistical trade-offs of low-rank modeling at scale. In the time-series domain, low-rank structure also provides a powerful inductive bias. For example, Agarwal et al. (2018) reformulates univariate time series into structured low-rank Page matrices so that denoising, imputation, and forecasting reduce to classical matrix estimation problems. More recently, Huang et al. (2025) shows that even ultra-lightweight networks with fewer than 0.4k parameters can achieve state-of-the-arr long-term forecasting performance by exploiting low-rank periodic structures and www.ggonllltyooosrrriinn representations, drastically reducing model size and computation while maintaining accuracy.
An Sperator-theorticc perspective. in which nonlinear dynamics are represented by the infinite- dimensional linear Koopman operator acting on observables, has emerged as a powerful data- driven framework for the prediction, estimation, and control of nonlinear systems (Brunton et al.) 2021 Budišić et al., 2012). In practice, this viewpoint is implemented through finiteddimensional approximations of the Koopman operator, most notably Dynamic Mode Decomposition (DMD) and its many extensions, including Extended DMD and kernel-based DMD (Schmid) 2010 Williams et al. 2015). An expanding body of research (Zhang et al.) 2025 Liu et al. 2023) now leverages these Koopman- inspired methods to analyze and forecast time series, demonstrating their versatility in extracting low-dimensional linear representations from complex, high-dimensional dynamical data.
22

---

Published as a conference paper at ICLR 2026
B.2 TERMINOLOGY
Instance Normalization Given a time series historical segment, Yhis E IR4 Phis and This the mean and standard deviation of Yhis respectively, and 0 small constant for numerical stability, Instance normalization (IN) performs the following operations for forecasting:
Yms (Yuis Mais) / Ims + 6)
Fz1 = (Y
Yiu = Yiu (o + 6) + Phis
is also shown in Zeng al. (2023) and Lin et (2024) that simpler operations, such as directly
fixing This + 6 = 1, also works.
All of the operations mentioned above have the same effect on W for a single-channel linear time series forecasting model, which forces the column sum of W to become Toner and Darlow 2024).
Channel Independent, Channel Dependent, and Individual Channel Modeling Channel Inde- pendent (CI), Channel Dependent (CD) (Nie et al. 2022), and Individual Channel Modeling (INC)
are three means of handling multi-channel time series.
Consider a multi-channel time series {y+) each of channel to m in this time series can be (m)
considered as single-channel time series denoted as {vr ) " {vt (x( Ji=l...., {3/2
Jt=1 The
T
corresponding collection of history and future segments are denoted as Y Y-2) Y (mm) N and (1) y fur Y E IR NxH respectivvlyy
his
his
füt
Firstly, we introduce CD, which has general modeling target follows:
min
stack
(Y: fut fut
(2(2)
y-(m) fut
- fo
( his
(
his
his
CD is the most general target in time series. Potentially, CD allows errors that arise from any channel to impact predictions of all channels. INC can be viewed as constrained version of CD. For INC, errors that arise from the i-th channel only impact predictions the i-th channel, and all interactions between channels are dropped. The INC target is as follows, where (()) are individual models for each the i-th channel:
S6 (X
min
fut
Finally, we give the CI target, which can be viewed posing additional constraints on INC. Unlike INC, a shared model fø is used to minimize the following target:
( his I|:
min
fo
fut
Particularly, for linear time series models, the CD paradigm can be considered fitting matrix of shape L. x m, consisting m x m blocks of Wayig E LR (iz and in are two separate indexer over the channels). In this case, the linear CD target is as follows:
172
min
Y fut
Σ
Y his
1==1
11=1
Following previous logic, in linear time series models, CD pose no constraint on all W iiig INC forced W Wi = for all in # i2, and further forces W = W2 22 relationships visualized in Figure 7
W Wmm- Their
B.3 PRELIMINARY
Time series Models and Difference Equations. Following the common practice of using CL. where the model operates single-channel time series (Nie al. 2022 Zeng al. 2023 Das et al.) 2024), we consider single-channel noise-free discrete time series { Jt= There are two main approaches to modeling such time series:
17
23

---

Published as a conference paper at ICLR 2026
v(3)
y(3)
V(m)
[3)
y
y(S)
v(m]
[3) v(2) y/30
hut
his
Mii
(80
his
his
Y
(a) Linear CD
(b) Linear INC
(c) Linear
Figure 7: Visualization of CD. INC, and CI for linear time series models. We color 0 matrix blocks with light grey. Matrix blocks that are forced to be equal are colored the same.
Time-based models: These models directly map each time step to a predicted value using function (t), such that
y. fo(t..
While conceptually straightforward. this method can be difficult to scale or generalize, especially for long complex sequences, because it treats time as the primary input without explicitly modeling the dynamics between observations.
Structural models (difference equations): Instead of predicting each value independently. this approach models dependencies between time steps using function fa(+). It imposes constraint across consecutive values in the time series:
fo Vi
27 97 1. Vi 1+
historical segment
This form, known as difference equation, the discrete analogue of differential equations used in continuous systems.
Modern time series forecasting methods, particularly those based on deep learning, typically focus learning such structural dependencies. While these models often achieve high predictive performance by implicitly capturing difference equations, their internal mechanisms usually not interpretable.
Homogeneous Linear Difference Equations. We consider the class of homogeneous linear dif- ference equations due to their analytical tractability and structural clarity in modeling temporal dependencies. These equations take the form
vi 0197 +... apPi-p = O,
where is the order of the difference equation and aI,...,, Op E C.
More general linear difference equations may include external forcing constant terms, known as nonhomogeneous difference equations. Nonhomogeneous difference equations can often be transformed into homogeneous equations. For instance, it is common obtain a nonhomogencous difference equation as follows:
V + +a
anDi-p
where some constant bias. In this example with constant bias, conversion to the homogencous form is done simply by taking algebraic reduction on the following:
y1 + aivi-1 + + unDiip = c v + aivi +... OpHi-p-1-
This justifies our focus on the homogeneous case. The solution to the homogencous linear difference equation obtained via the characteristic equation
p a1 P-1
Up 0,
24

---

Published as a conference paper at ICLR 2026
whose roots, called characteristie roots {r.)i=1 dictate the structure of the general solution. When the roots are distinct, the solution is expressed as
Vi = Ciri + +.......... CLTY
where the constants C₁'s are determined by initial conditions. Repeated roots introduce polynomial factors of the form (Ag + - + 11.-1th-1)__ where is the multiplicity root r. This method provides closed-form description of the system's evolution, revealing direct and interpretable link between the underlying structural formulation and explicit predictions across time steps. Such models form foundational baseline for analyzing and approximating more intricate non-linear learned temporal systems.
Another way of obtaining the linear difference equation comes from the linear recurrence relation with the general expression given below:
ap-1
ap
V
E
1/+
3
0
+
(4)
U1
11
Ut
step segmeett
bias
F the companion matris
2-1 step sogment
Under the noise-free condition (E, 0 for t), the homogeneous form (c 0) of Equation equation[4] can be further simplified. In this case, the higher-step future segment is directly obtained through the matrix power of the companion matrix F. This observation naturally leads us consider the eigenvalue decomposition of F. Notably, the characteristic polynomial of F coincides with that of the underlying linear difference equation, which was given earlier this section.
The Multi- Channel Case. Our discussion of homogeneous linear difference equations focuses mainly on Longle-channe time series. While common practice to model multi-channel time series by first converting them into multiple single-channel time series in recent works (Zeng et al.) 2023 Nie al., 2022] Eldele al., 2024), we also cover Vector Autoregression (VAR), which fundamental multi multi-channcl time series model that characterizes the joint dynamics of multiple variables. VAR without bias for an m-dimensional,, noise-free time series {y? ! is multivariate generalization of the homogeneous linear difference equation mentioned above:
yi
A,yi-1
where (A,) E mm are coefficient matrices. A more general form of a VAR(p) model for an (2)
m-dimensional (not necessarily noise free) time series yt (vt2) y1 ...,yt (")) ) is given by:
yr - Σ A t
(5)
1=1
where e is an x vector of intercept terms, and e, R.m is white noise process with TE(e,] 0 and Cov(E;) Σ.
For homogeneous system (c 0) without noise (e, 0 for all described in Equation (5) the underlying system's dynamics determined by the characteristic polynomial as follows:
det
I
A
= 0.
(6)
The companion form representation for VAR is given as follows, which is also a multidimennional generalization of the companion matrix for the univariate case in Equation
Yt
A
A2
Ap-1
A
Y(-1"
E
y -
I
0
0
0
Yt-2
0
Lt-p+1
I
0
Ly
step segment
bias
F the companion matrix
f-1 step segment
25

---

Published as a conference paper at ICLR 2026
where F is the mp x mp companion matrix. Notice solving for the characteristic roots by Equation O is equivalent to solving for the eigenvalues of the companion matrix in Equation (7). VAR models are widely employed in forecasting and structural analysis, with tools such as impulse response functions and variance decompositions providing insights into dynamic interactions. Extensions like structural VAR (SVAR) incorporate identifying restrictions for causal inference, while Bayesian VAR (BVAR) introduces shrinkage priors to improve estimation in high-dimension settings.
Stability of AR/VAR Models. In classical time series analysis, the stability of AR and VAR models is usually an important consideration. Stability requires that the characteristic roots lie outside the unit eircle, ensuring that the process is stationary and that forecasts do not diverge over time (Shumway and Stoffer 2006). This condition matters when the model is used to describe the underlying data- generating process or when iterative forecasts are produced by repeatedly feeding predictions back into the model. However, in the context of direct multi-horizon forecasting, stability is not strict requirement. Each forecast horizon is modeled and estimated independently, and the prediction for a given horizon is produced only once, without recursive dependence earlier predictions (as shown in Equation (2.. As a result, even if the estimated AR/VAR coefficients correspond to an unstable system, this does not affect the validity of the forecasts at the specified horizons. The forecasts remain well-defined because they are not generated through repeated iteration of the unstable dynamics.
C REINTERPRETING DESIGN CHOICES IN LINEAR TIME SERIES MODELS
C.I BIAS-FREE LINEAR MODELS: JUSTIFICATION AND IMPLICATIONS
A common yet sometimes unexamined modeling decision is the omission of bias term. We offer three principled reasons for this choice. First, from a theoretical standpoint, linear difference equations that include constant bias term can be algebraically transformed into equivalent homogeneous forms; Second, standard normalization, especially instance normalization, centers input and target sequences, making bias terms negligible for sufficiently long sequences; Third, empirical results (Appendix [E66 show bias-free models match or outperform biased alternatives. This simplification preserves expressive power while streamlining the architecture.
C.2 HIGHER-HORIZON FORECASTING AND EXTENDED LOOKBACK WINDOW
Recent time series models tend to directly give predictions for the entire forecasting horizon without recursive calls (Zhou et al.. 202T Zeng et al.) 2023). This is because such practice may help alleviate cumulative errors in recursive predictions and improve performance. We thus turn our focus to the higher-horizon forecasting setting. Rather than predicting the next time step and applying iterative rollout, we model each forecast horizon independently. This results in H distinct regressions, one for each future time step. Specifically, the predictive equation for the j-th horizon takes the following form when the model achieves zero training loss in ideal noise-free situation:
Yo+s
Wks Vt-x+1 = O.
(8)
This formulation can be interpreted as a linear difference equation. Each such equation implicitly defines set of characteristic roots that govern the dynamics at the corresponding forecast horizon. Notably, models for higher horizons tend to incorporate more temporal information. This leads to the following structural observation:
Claim (Part of Fact 2). The characteristic root set of higher-horizon linear model contains, subset, the roots from basic dynamics.
This claim suggests a hierarchical organization in temporal dynamics: models for longer-term forecasts preserve the essential roots needed for short-term accuracy while allowing more expressive representations. This supports the common practice of treating each horizon independently and further implies that higher-horizon models do not contradict, but rather generalize, the underlying system dynamics.
Another fundamental modeling choice is the length of the input (lookback) window L. When the underlying dynamics admit minimal recurrence relation of order K, choosing I, K introduces
26

---

Published as a conference paper at ICLR 2026
redundancy. In such cases, the regression admits multiple parameterizaaiion that yield identical predictive behavior:
Claim 2 (Part II of Fact the minimal recurrence order is K, then any lookback window of length -> yields non-unique representations that all preserve the characteristic roots of the underlying dynamics.
While this non-uniqueness does not harm predictive performance, underscores the flexibility of the model encode equivalent dynamics in multiple ways.
The formal version and proof Fact 2 are provided in Section D.2
C.3 TOY EXAMPLE
To illustrate how characteristic roots govern the forecasting behavior of linear time series models, consider a simple noise-free example:
yt) = 0.01f + sint.
This system admits five characteristic roots, {e', -e', 1,,, corresponding to the following general solution with coefficients A, B, D, E:
Vgen (t) (At² + Bt C) 1' D(cos t isn + E(cos t sin
This general solution can be exactly represented by linear forecasting model with lookback window of length and a forecasting horizon of length 1:
Yt + d1 1 + 45 5
Following standard practice, we form the history and future matrices, Yis E RN5 and Y fur E RNX. and compute
his Y fut-
his
Empirically, we find that the characteristic polynomial of W R yields the roots fe', e 1, 1,1,1} 1}. Since the roots {e", e=t, 1, 1] are preserved, the model recovers the dynamics exactly, consistent with our observation of zero training and test errors.
Based on Fact 1 the role of roots also explains the model's generalization ability in the following two cases:
1. W can also perfectly forecast st))= = 4+ cos f, which has characteristic roots {et, -ef 1,
a subset of (et, e 1, 1, 1): see Figure 8 Left;
2. W cannot forecast z(t) cos(1.1t). which has characteristic roots (el.1i,e-1.1y 1...., not subset of { 1, 1, 1 see Figure Right.
More generally, for any lookback window length > 5 and arbitrary forecast horizon, the resulting matrix W general introduces additional roots. However, these roots play no role in the forecasts, provided that {er, -e¹, 1, 1, 1} remain among the characteristic roots. this case, perfect forecasting always guaranteed.
C.4 INSTANCE NORMALIZATION AND CHANNEL INDEPENDENT MODELING
In this section, we discuss detail how Instance Normalization (IN), Channel Independent (CI) Modeling, and other channel modeling methods (CD, INC) fits in framework. You can refer to Appendix B.2 for more explanation of these terms.
Instance Normalization (IN) As an application of characteristie-noot- tion 3 instance normalization introduces a structural inductive bias aligned with this framework. By centering each input sequence, normalization implicitly enforces unit root at r 1, corresponding to level invariance in the dynamics. To see how we end up with this special root, we follow the proofs in Toner and Darlow (2024). For linear model, the effect of IN on W e T Lx is that the column sum of W is 1, as follows:
analysis in Sec-
Σ W 1, 1,2 H.
27

---

Published as a conference paper at ICLR 2026
Mode Genirra ltation Test on.s. F+
Gencralizal on coe(.1))
-1
-20 -30
-350
32
Line
Figure 8: Generalization test of W forecasting the time series v() 0.01 + sin f. Left: success of generalization to xrt) t + cos t. Right: failure of generalization to <t) = cos(1.
We then follow Equation S to establish the difference equation. It important to notice that yt++j has a natural leading coefficient of 1, as a result, the leading coefficients of the characteristic polynomial sum to 0, suggesting the existence of characteristic root of 1.
This special root enables the model to maintain consistency across sequences with varying mean shifts. This should be intuitive to see, the general solution with root being is now the following:
y/ Carl + Car2
C,1' Cirf C2r2 + Cp-
The undetermined Cp in the general solution allows any mean shifts to become automatically generalizable.
Channel Independent (CT) Modeling Our analysis Section 3 also helps to explain the empirical success of channel-independen modeling in multi-channel time series. Although individual channels may exhibit distinct dynamics, modeling them in a shared fashion can still be effective. The key insight is that the overparameterizattoo introduced by a sufficiently large L provides enough representational capacity to accommodate the union of characteristic roots across channels, even without explicit cross-channel interactions. Without loss of generality, the first channel has linear dynamics with roots Ti,1, 11, K, and the second channel has linear dynamics with roots 12 1, r2,2, ...y K2 By Fact long as the model's characteristic root set R contains the following set as subset, the model can generalize to both the first and the second channel:
assume that we have two channels, where
Ru P12 } (2,,,,
For instance, if the model is used forecast the first channel now, it can be done by simply setting the O's for roots in R FT,,,. Ti,K as O. Similar things can also be done for the second channel to achieve generalization. When the roots are highly shared between an efficient strategy as the cardinality of Ru is much smaller than the capacity required to learn (r1,1, fi, K, } and {r2,1. 112.1.22. F2,2 .... K2,Ka separately.
channels, CI is
The root sharing mechanism explains why CI models can use a shared weight W to model many channels, allowing better parameter efficiency. The performance improvements, on the other hand, explained with the data scaling property as described in Proposition ] as training the shared weight with more data improves its robustness.
VAR Perspective on CI Another valuable perspective on channel- indeeenden modeling is through vectorized autoregressive (VAR) modeling, where the system's 's behavior can also be captured by the characteristic roots obtained from solving the characteristic equation (with the multivariate generalization as in Equation (6)). This approach is particularly interesting because the number of characteristic roots directly related to both the order of the process, p. and the number of channels, m. Specifically, the total number of roots equals p X m, reflecting the fact that the system evolves with given order over multiple independent channels. In this framework, channel independence can be seen degenerate associated with the variables as diagonal matrices, with each diagonal element identical. More formally, this means that the coefficient matrices, denoted A;, are of the form A, k, x I, where
where the autoregressive process is simplified by treating all matrices
28

---

Published as a conference paper at ICLR 2026
k; is scalar constant, and is the identity matrix. This simplification implies that each channel behaves independently. with its evolution governed by the same recurrence parameter k; and no interactions between different channels occur. Furthermore, we have the following results:
} be vector Proposition 3 (Equivalent Representation of Diagonal Recurrences). Let {y+ [R'" time series following a diagonal matrix recurrence of order p.
Y1 D.y.ii:
1=1
where D; = diag(d) () ...., ) are diagonal matrices. Then, there exists an equivalent order-L. recurrence (where L S mp) that can be written
D
Y1
where each D Ii scaled identity matrix, with E R and Im being the identity matrix of size m.
Proof. Let Yr = (y) ..... D1 recurrences:
(m))
The given recurrence decomposes into m independent scalar
( (1) Dt-
for j = =..;..... 1..... m.
Each scalar recurrence has an associated characteristic polynomial:
Pş(r) = r
d p j j=1...,,m
Let Q2 be the minimal polynomial of (vi () which divides (r)... The Least Common Multiple (LCM) of all (r), denoted as Q(r), is monic polynomial of degree L mp that annihilates every {ul ()))) }. Thus, all components satisfy:
I. Σ
Kyr for j 1...., m,
where (hilt are the coefficients of Q()). In vector form, this becomes:
y: Σ k-y,,
D'yt-i
where D = k, This proves the equivalence.
The equivalence between the original diagonal matrix recurrence and the transformed recurrence indicates that while the original system uses different diagonal matrices for each lag, the equivalent system employs uniform scalar scaling (represented by kęIm) across all channels at each time step. This simplification removes the structure of individual diagonal elements in favor of single scaling factor for each lag, which effectively reduces the complexity of the model. The key takeaway that both formulations describe the same system, but the latter offers more compact and simplified representation. This result implies that a system with independent channels-each governed by diagonal matrix with potentially different coefficients-ee where the same scalar k; governs all channels at each time step. This reduction in complexity is particularly useful for simplifying the analysis of digh-dimensional systems, it shows how the system dynamics can be captured using fewer parameters, namely the scalars k; rather than a full set of diagonal matrices for each lag.
be equivalently represented by system
29

---

Published as a conference paper at ICLR 2026
Remark 3 (Technical Condition for General Matrices). Considering a more general case of matrix recurrence of the form
y= Σ A.y.-
where A; E IRmxx are arbilrary matrices (nol necessarily diagonal), the components of the sequence Y: R interact at each time step due their dependence on multiple previous states Yt-i- Unlike simpler cases, where recurrences can sometimes be expressed using scaled identity matrices, this general form does not have a similar proposition or a straightforward structure. The matrices A; rypically cannot be reduced to a form involving only scaled identity matrices unless they satisfy the restrictive condition of being simulannousty diagonalizable. In cases where the matrices are not simultaneously diagonalizable, the recurrence introduces coupled dynamics between the channels of yt, making the analysis significantly more complex. These coupled interactions prevent an easy separation simplification of the system, which means that more advanced techniques required to analyze the interdependencies and predict the behavior of the system over time.
Connections of VAR and Linear Channel Dependent Modeling Notice that the perspective of restricting VAR to model channel independence is very similar to the process in Figure 7 Indeed, VAR and linear CD models are deeply connected. Precisely, linear CD model (with no bias) with a lookback window of length I. and a forecasting horizon H is equivalent to a collection of H VAR(I) models as follows:
Y1+1 = - -1-1 y + E111
Y1+2 -i=
1-1
A
(
y E142
(9)
YI+H = 2i=0 A, y E1+H
E
The parameter collection described in Equation (9) aligns with those described in Appendix B.2 via
basic tensor operations- we can stack all (") appeared to make a 4-D tensor of shape L x m x m. By permuting the axes orders and flattening it to make matrix of shape L x H m, we can arrive at the CD learnable parameter set discussed before.
Following the convention in VAR, here utilize the form of the parameter set described in Equa- tion (5). However, it is important to see that it is closely connected to channel-dependen modeling used in the field of machine learning.
CI, INC, and CD in Linear Time Series Models Building on our previous discussion of CI from both top-down and bottom-up views, we summarize our theoretical and empirical insights on using CI modeling, even in when the channels of multivariate time series interact in complex ways. Furthermore, we go beyond from CI and discuss the practical potential of additional key paradigms- Individual Channel (INC), and Channel-Dependent (CD) as defined in Appendix B.2
Broad Use of CI is a Consequence: Specifically, if each channel can be individually represented by a linear model (a much weaker and realistic assumption by Corollary aad Corollary 2 then the multivariate time series as whole can be effectively modeled by applying the same linear operator across channels. We emphasize that no assumption on channel relationships is needed. This significantly broadens the applicability of CI modeling to real-world datasets, where channels may be correlated or interact in complex ways.
Benefit of CI in Practice: of CI modeling. In particular, CI methods aggregate noise across channels, which effectively acts implicit data augmentation. This makes CI models more robust under finite-sample conditions, especially when training data is noisy or limited.
Beyond expressiveness, Proposition[exxpeiee @xxlains the robustness benefits
Empirical Justification: To address the challenge of limited data, we propose two inductive bias strategies - rank reduction and root purge which are designed to improve model robustness by introducing structural regularization. The effectiveness of these strategies is reflected in our main experimental results (Table IT] In addition, conducted controlled experiments comparing channel-ine hauneel-ndeeendem (CI) and channel-individual (INC) modeling (see Table These experiments reveal several key tradeoffs:
30

---

Published as a conference paper at ICLR 2026
CI models already benefit from shared information across channels, which reduces the marginal benefit from root purge.
IICC models, in contrast, have higher capacity and flexibility by design, which allows them to benefit more from root purge, especially under large-scale datasets such as ETTm and ETTh, where data sparsity is less of a concern.
Table 6: Comparison of common channel modeling approaches (CI, INC, and CD) in linear time series forecasting models.
Modeling Choices
Attributes
Description
channels share the same linear representation
Capacity
Low (I. x H)
(Chamel--ndeeendent)))
Data Scaling
Works well
Benefit from Our Methods
Moderate
Practicality
More practical
Description
Fach channel uses separate linear model High (C' H L)
Capacity
INC (Individual Channel)
Data Scaling
Suffers
Benefit from Our Methods
Good
Practicality
Feasihle when the numher of channels is small
Description
channels are jointly modeled using linear model High (C' II x L)
Capacity
CD (Channel-Dpenndnt)
Data Scaling
Suffers
Benefit from Our Methods
Moderate (difficult to optimize at scale) Hard to implement due to memory overhead
Practicality
To finally clarify the distinction of CI, INC, and CD, now compare these three primary channel modeling paradigms and summarize their key similarities and differences across several critical aspects in Table
C.5 SCALING IN PURE NOISE SERIES
Pure Notse
Lincar Nobs 17--000)
Oyoneroonce
Oytinal Hivces
Timts Strip
Time Stop
(a) OLS on small scale of pure noise series
(b) OLS on large scale of pure noise series
Figure 9: Qualitative visualization of an OLS model forecasting pure noise. Since noise is inherently unpredictable, the optimal forecast is horizontal line at zero. (a) With limited data, the OLS fit shows significant deviation from this optimum. (b) As the dataset size increases, the model's 's forecast converges to the optimal solution, consistent with the theoretical scaling law in Proposition [
In practice, time series models will almost never encounter the noise-free case. Thus, typical time series can viewed as a superposition of two components: the true underlying signal and additive noise. Given the linearity of the forecasting model, the learned weight matrix W naturally acts both components simultaneously, as follows:
[ W y + (Etu
W Em. )
In the idcalized analysis presented in Section 3.2 we separate the desired properties an "ideal" linear model W into parts:
31

---

Published as a conference paper at ICLR 2026
Purge
Opriaal
ACOA Prossaa
Tima
(a) Root Purge small scale of pure noise series
(b) RRR small scale of pure noise series
Figure 10: Qualitative demonstration of forecasting performance on pure noise. Models fitted with proposed methods (RRR and Root Purge) generate predictions that converge significantly toward the optimal forecast (zero). Notably, their performance is comparable to an Ordinary Least Squares (OLS) model trained dataset 16 times larger.
1. It should perfectly recover and predict the signal dynamics (not including the noise term), i.c., map the historical signal part of the time series to the corresponding future signal values;
2. It should completely ignore the historical noise component, i.c., map it to zero, since noise is unpredictable by nature.
Data Scaling Property Pure Noise Series
0.32
Linear model
RRR
Root Purge
0.30
Test MSE
0.28
0.26
50
70
100
200
400
600
Training Set Size (T)
Figure 11: Mean Squared Error on pure noise series of varying sizes. Consistent with Proposition [ OLS-optimized linear models require substantial data to converge to the optimal forecast, despite the fitting target being pure noise and the optimal prediction being trivially zero. Conversely, both RRR and Root Purge demonstrate robust performance, showing good performance even with limited data.
Both properties are essential for the forecasting performance: even if W can perfectly capture the signal dynamics, it also maps noise history to a biased quantity in the forecast, the prediction quality will suffer.
This effect can be better demonstrated with Figure a Figure TO We can imagine that even if linear time series forecasting model can perfectly predict the noise- free dynamics, the output will still be corrupted by the term W I Shis- W Ehis is a large noise as shown Figure 9a this corruption will be strong, making the forecast suboptimal. On the contrary, if W Ehis produce output as shown in Figure Pa (by using lots of data), and Figures 10a and TOb (by using our methods), the forecasting performance can be significantly improved.
Notice that the previous analysis naturally establishes a lower bound for how well linear time series forecasting model can perform by assuming that model fits the noise-free dynamics perfectly. This
32

---

Published as a conference paper at ICLR 2026
similar our analysis in Section 3.2 where we stated "In the idealized case where W perfectly recovers the signal dynamics, only the noise term remains." Studying this lower bound allows us to clearly see the fundamental limitation of learning linear time series forecasting model with solely the MSE loss, as the lower bound of test MSE remains high, especially with limited training cases. This result is promised theoretically by Proposition aas well empirically by Figure 1T
On the other hand, we demonstrate the superior scaling behavior of RRR and Root Purge in Figure 11 they actively suppress noises in their optimization processes. Therefore, RRR and Root Purge unlock the potential of the linear time series forecasting model to perform better than those trained with solely MSE losses, and their empirical improvements are evidently shown in Table !
D FORMAL THEORIES AND PROOFS
D.1 CHARACTERISTIC ROOTS GENERALIZATION
We give detailed and formal version of Fact I and its proof as follows:
Proposition (Characteristic Roots Generalize up to Sequence Initial Conditions). Let ri,..., rr E C be distinct complex numbers, and let (ri....,Tp) C {ri,...,TI.) for some p < L. Suppose C1 C1...-, Cp E C and b, by....,by by E C are such that
Eer Ebrl, for I,...,,,
where tt E Z are distinct integers. Then, for all E Z, holds that
Σ
CST
byrg.
1=1
Proof. Define the function
E Σ j=1
h(()
= Σ
cjr'
bgrl
dgr's.
j=1
j=1
where
Cj - bj
for j 1,... ,
dj
bj for j = p L.
Then h(t,)) = 0 for i = 1,..., L by assumption. Let V E Clx be the generalized Vandermonde matrix with entries Vi.j = r Since the ry are distinct and the & are distinct, V is invertible. Hence, the equation
vd=0 0
implies = O, and thus all d, = 0. Therefore,
Σ J=1
C
byrl
for all / E Z.
f=1
The proposition demonstrates a powerful uniqueness result: if a linear combination of exponential functions (with distinct complex bases r;) agrees with another (potentially overcomplete) combination at L distinct time points, then both expressions must be identical at all time points. This result can be extended to situations where roots are repeated. In such cases, the basis functions generalize to include terms of the form (Ao + A,t of the root r. These generalized exponentials still form a linearly independent set under suitable conditions, and similar linear algebraic arguments (c.g., involving confluent Vandermonde matrices or the Wronskian) show that equality on a sufficient number of time points again guarantees equality everywhere.
+ A 1 =Irr,, where k accounts for the multiplicity
33

---

Published as a conference paper at ICLR 2026
This result directly implies that linear model can represent any time series whose characteristic roots form a subset of the model's '' own roots. In other words, a time series is generated by a set of exponentials associated with certain roots, then any linear model whose characteristic polynomial includes those roots, possibly along with additional ones, will still be able to reproduce the series exactly, provided the coefficients associated with the redundant roots are set to zero
Based on the earlier proposition, we can formulate a corollary that highlights the expressivity of linear models:
Corollary 1 (Expressivity of Linear Recurrence Models), Let a time-dependent signal y(t) be composed of functions of the following
}
u(t)
II MEN
Σ (a. cos(un,,t) + +b sin(wn, LiEt.
+
Σ PEP
apth
+
Σ Ien
apert
where In, Pn, Rn are finite index sets for each n E N. and all coefficients ag b,, ap. ar E C, frequencies Wn,i E R. and exponents E C. Then any such function y(t) can be perfectly represented as a solution to a linear difference equation with constant coefficients.
This corollary highlights the expressive power of linear models with constant coefficients. Such models can perfectly represent signals composed of sinusoids, polynomials, exponentials, and their combinations - forms commonly found in real-world time series. Moreover, the exponential and sinusoidal functions form a complete basis in many function spaces. This means that even when exact representation is impossible, linear models can approximate complex signals with small error, making them highly effective for both modeling and prediction.
The expressivity of linear recurrence models can be understood more formally through the lens of differential Galois theory (Singer) 2009; Van der Put and Singer] 2012). In this framework. consider a time-dependent signal y(f) that satisfies a linear differential (or difference) equation with constant coefficients Let K denote the base field (e.g., rational functions in t), and let L be the Picard-Vessior (PV) extension generated by (t) over K. Then the set of all solutions to the associated linear system forms L, which is minimal differential field extension closed under the operations required to solve the equation.
In this language, the linear recurrence model is expressive for given signal v(() if and only if the Picard-Vessiot extension generated by y(t) is contained in the extension generated by the model's characteristic roots. In other words, linear model can exactly represent y(t) precisely when the solution field L of (t) is a subextension of the model's PV extension. This perspective allows a clean generalization: for any family of signals whose PV extensions are nested subextensions of a model's extension, a single linear recurrence system is guaranteed to represent all signals in the family. Conversely, if the PV extension a target signal lies outside the model's 's extension, no linear model with the given characteristic roots can represent it exactly. Formally, we can restate Corollary as follows:
Corollary (Differential Galois formulation). Let y(1) be time-dependent signal and K the base differential field. Let Ly/ K denote the Picard -icard-Vessior extension generated by y(t), and let LM / K denote the PV extension corresponding to a linear recurrence model M with characteristic roots (x.). Then y(t can be represented by M if and only if
Ly S LMM
Equivalently, in terms of differential Galois groups,
DGal(T. K) - DGal(T.g/K).
i.e. the differential Galois group of Ly is quotient of the differential Galois group of LM-
The differential Galois perspective can be further extended to the case where the coefficients of the linear recurrence or differential equation not constant, but are functions of time other variables. In this setting, the classical notion of characteristic roots does not directly apply. Nevertheless, one
Unlike in differential Galois theory, the Picard-Vessiot extension in difference Galois theory is often only a ring (not field) and one usually needs localize it to handle zero divisors. In the sequel, we focus the differential equation case for ease of exposition.
34

---

Published as a conference paper at ICLR 2026
still describe the expressive generalization capacity of model in terms of Picard-Vessiot extensions. Let the linear system now be
y (t) + an-1(t)y - +... (u (t) = =,
base differential field K. Denote by Ly/ the Picard- Vessiot extension generated by target signal y(1), and by LM/ K the PV extension generated by the linear model .M with functional coefficients. Then, as the constant coefficient case, the model M exactly represent y(t) if and only if Ly S LM- This formulation shows that the PV extension framework naturally generalizes beyond constant coefficient systems. In particular, it provides a unified algebraic criterion for model expressivity and generalization: linear model with functional coefficients can capture any signal whose solution field is contained within its PV extension, even when classical characteristic roots do not exist. Consequently, the concept of subextension remains the fundamental criterion for representability and generalization in the broader class of time- -imendepnndent linear systems.
where the coefficients a, (t) are functions
D.2 LINEAR MODELING PROPERTY UNDER NOISE-FREE SETTING
We give detailed and formal version of Fact 2 and its proof as follows:
Proposition (Characteristic Polynomial Divisibility under Recurrence Extension). Let univariate linear dynamical system be governed by minimal-order linear recurrence relation
M++1 019t + 022ht-1 + ...... 0p ptl:
with characteristic polynomial
Ag. Suppose the same dynamics can also be expressed via a higher-onder recurrence
P(r) = p8 aird-1
Vt'+1 = biyr b2 + byHl' q 2 P.
with characteristic polynomial
by. Then P(r) divides Q(r); that there exists a polynomial S(r) of degree q - p such that
Q(r) = r4 b
Q(r) = P(r)S(r).
Proof. Let P()) R[rr be the minimal annihilating polynomial for the sequence {ur). By definition, P(r) is the monic polynomial of least degree such that the associated linear difference operator annihilates the sequence:
P(r) {vr 0.
Now, suppose Qrr) R[r] is another characteristic polynomial corresponding to recurrence relation that also annihilates [ych.c.. i.c.
Q(r) {v 0.
We claim that P(r) Q(r). Assume for contradiction that P(r) Q(r). Then write: @(r) = (()) R(r),
where R(r) * is the non-zero remainder polynomial with deg(R) (R) < dog(P) = Applying both sides to the sequence { obtain:
Q(r) {v+} = [P(r)S(r) + R(r)] {w:} = P(r) {vr} + R(r) {v.).
The first term vanishes by assumption, since P(r): {vc} Thus:
Q(r)- (v.) R(r) {ur } 0.
This implies that R(r) also annihilates {yc) contradicting the minimality of P(r), deg( < deg(P). Therefore, no such non-zero R(r) can exist.
We conclude that there exists S(r) E [K[r] such that
Q(r) = P(r)s(r),
and hence P(r) Q()).
35

---

Published as a conference paper at ICLR 2026
The result implies that any higher-order recurrence relation that annihilates given sequence must preserve the characteristic roots of the minimal recurrence, since its characteristic polynomial contains the minimal one factor. Two notable special cases of such higher-order recurrences are:
the j-step ahead regressor, where yt+1 = a19t + ...... + apDttp++1 and
(ii) extended lookback models of the form yt+1 a1ys + L+1 with L > p.
Both formulations represent higher-order linear relations that are consistent with the original dynamics, and therefore preserve the same characteristic roots. Importantly, increasing the order of the recurrence (i.e., enlarging the polynomial degree) introduces additional degrees of freedom. This can enable the model to generalize better, especially noisy or non-stationary settings by capturing richer set of dependencies while still being anchored to the original root structure.
Remark 4. In the noise-free setting, the closed-fom solution to the optimization problem of Equa- tion ① can always achieve zero error, confirming that a higher-order recurrence relation holds. However, if one uses the Sbore-Penrroee pseudoinverse to express this solution, the result will be the solution with minimal f2 norm. In general, this solution does not correspond the minimal-order recurrence and may include unnecessary higher-order terms that preserve the roots but do not reflect the most compact representation of the underlying dynamics.
D.3 SCALING PROPERTY ON WHITE NOISE SERIES
We give formal version of Proposition 1 and its proof as follows:
Proposition 6 (Asymptotic Distribution of vec (W) under Cross Cooss-ovvriiacce. Let X RNx
and
Y E N be random matrices with the following moment conditions:
E ( [((
(ve(()) (X) (Y)((Y))
(ee(X)) vec rYc(Y))
Σxy) Σ XY Σy
Ex
O, E
where Ex E RNINIL Ey E RNNHXNH and Exy E RNLXNH To fit the linear model Y = XW + E, where W ERAX is the weight matrix, and RN is independent noise, then, for large N, the least-squares estimator W= - X) Y satisfies
VN (vec(W) ( vec(w.)) 4 (0,0),
where:
W. E-1 Exx Ex y is the population coefficient matrix with Ex E X] and EXY E
r DIO x with Kronecker product, and I denotes the identity matrix of size N.
Exx Ex Exy & I @ Iv. 8 denotes the
Proof. First, the population coefficient matrix W, minimizes the expected squared emoor:
W,- arg min [||Y - XWI = '''Êxv E
W
Substituting Y = XW. + E, the least- estimator can be written as:
X
[
x
x (XW. + E.) = W. + (x TT..
Thus we have
W W. w (XTx) '
Vectorizing both sides and using the identity vec( (ABC) = (CT & A) vec(B) gives:
vec W W,) (I, & (X -)- vee( X E.).
To derive the limiting distribution, study the asymptotic properties of X X and (x E.). By the law of large numbers, we have:
X X 1 Ex 7 (x
~
Σx
N
36

---

Published as a conference paper at ICLR 2026
Multiplying this asymptotic approximation gives
VNvee W.) = IH Ex'1 (Jr vec(X (TEE)
Next, we apply the central limit theorem to the matrix product E., which gives:
vec E.) 4 No E Σx).
Now applying Slutsky's theorem gives:
N - vec(W.)) 4 N,,,))
where
r= (IN & Σx
& EI) 8 ΣX
[ tes
Σー1
The informal claim of Proposition I Ithat the weights learned by a linear model forecasting zero- mean noise with finite second moments converge at a rate proportional to O(1/VT). where the length of the observed time series, can be formally justified by the asymptotic normality theorem for least-squares estimators. In this context, each observation time can be treated as an independent sample, allowing the time horizon T play the role of the sample size N in the multivariate analysis. The theorem establishes that, under suitable regularity conditions, the least-squares estimator W converges in distribution to a normal random variable centered at the true coefficient matrix W.. with convergence rate (I/vVN).
When the time series data consist of zero-mean noise with finite second moments, the assumptions of the theorem are particularly well-suited. The proposition provides a rigorous statistical foundation for the empirical observation that forecasting models trained on longer time series tend yield more stable and accurate parameter estimates. However, the convergence rate of the estimator is on the order of O(1 which implies that disproportionately large amount of data required to significantly reduce the estimation variance. This highlights an inherent inefficiency in learning from high-variance signals like white noise: although increasing the sample size improves estimation accuracy, the marginal gains diminish. Therefore, to overcome the adverse effects introduced by noise, substantially large dataset is often necessary.
peplinear-ssecifiinn
D.4 EFFECT OF MODEL PARAMETER RANK REDUCTION
We give formal version of Proposition 2 and its proof as follows:
Proposition' 7 (Model Parameter Rank Reduction). Let W E [RL be a matrix with rank W) S p S min(L, H). Then there exist matrices U, E LL Σ, Rexp, and V, E Hxx such that
W = U
Consequently, for any x E R. and y E H the model y WT x can be equivalently written as
V.y = - or y V U.E.U.
Proof. Since rank(W) < < I min(L, H), the singular value decomposition (SVD) yields orthogo- nal matrices U E RLXL E Ri*H' and diagonal matrix Σ LxH with at most p non-zero singular values, such that
W UEVT
Let U, E RL and V, Hxp consist of the first p columns of U and V, respectively, and let Σ, E RRxP contain the top p singular values. Then,
W V.
37

---

Published as a conference paper at ICLR 2026
For any x E RR', the model W becomes
y - V x.
Left-multiplying by V and using the orthogonality property V V = =Ip gives the equivalent form
completing the proof.
From the Proposition we see that fitting a rank-p linear map is equivalent to:
projecting x into an p-dimensional latent space via U..
projecting y into the same latent space via V and
learning a linear map Ep between them.
This expresses the model as:
(projected output) = linear map o (projected input).
So. constraining the rank of W is equivalent to simultaneously reducing the dimensionality of both inputs and outputs, and learning relationship that shared latent space.
Proposition shows a constructive duality between two operations in time series forecasting:
imposing rank-p constraint on the model weight matrix W, and
explicitly projecting both the input and output data into an p-dimensional latent space, and then learning full-rank linear map between them.
In this work, we prefer the model-basee rank constraint than data projection for the following reasons:
Directly projecting data breaks the sequential manifold structure of the time series, poten- tially harming downstream performance;
Model-based regularization enables tross-valdation to efficiently tune the rank hyperparam- eter, while data pre-procee pee-poceessing fixes it priori;
We empirically tested both strategies: constraining the model rank consistently outperformed pre-projected data. Other related approaches (e.g., |Hanng et al. (2025) also use low-rank projections of data but similarly underperform our method.
D.5 ANALYSIS OF RANK REDUCTION
We analyze rank reduction in the linear mapping Y The focus is on how Reduced-Raee Rank Regression (RRR) and Direct Weight RRR (DWRR) alter the solution in terms of the Frobenius norm.
XW, where E RxL and Y NxH
Proposition (Frobenius Norm Bound for DWRR Approximation). The Frobenius norm of the approximation error satisfies:
min(L.
WDwRr Woisl
Σ i=p+1
o..
where 02 2 .... min((,, 2 0 are the singular values of W Wots-
Moreover, this is the best possible rank-p approximation error the Frobenius norm, meaning that for any rank-p matrix B.
W DWRR Wots|| < B Worsllr.
38

---

Published as a conference paper at ICLR 2026
Proof. Let the full SVD of WOLS E R be
min(L.
WOLS
Σ o,
where 01 2 02 2 ... mi) 2 0 are the singular values, and u, V, are the corresponding
singular vectors. The rank-p approximation W WDWRR retains the first terms:
W DWRR-
Σ V.
The approximation error is then:
min(L.
W WDWRR WOLS
Σ
o.u.V
The Frobenius norm the square root of the sum of the squared singular values. Hence,
min(L,H)
DwRr WoLs|lr= =
Σ
of
i
By the Bicart-Young-Minn theorem, the truncated SVD provides the best rank-p approximation in Frobenius norm. That is, for any rank-p matrix B,
WDWR Wors| S [B WoLs|IF.
Therefore, W WDWR is the optimal rank-p approximation of WOLS in the Frobenius norm.
Similarly, derive analogous results for Reduced-Rank Regression (RRR).
Proposition 9 (Frobenius Norm Bound for RRR OLS). Assume X Nx has full column rank (i.e.. rank(X) L). Then, the Frobenius norm of the difference between W RRR and WOLS is bounded by:
min(N.II)
VL
WRRR Wots|FF
Σ
oi
Omin (X)
i=p+1
where Tmin (X) is the smallest singular value of X, and (\) are the singular values of Ý X ols E IR XH
Proof. The OLS solution is
WOLS (X x) XWOLS (X xx (X x) ''
where -= UEVT the SVD of Ý. RRR applies a rank constraint by projecting the solution onto
the top p right singular vectors of Y (i.c. top singular vectors in V), resulting in
WRRR WoLs V,V V = (X (x ' UEV
Σ 0
x (U, U_ = =(x
V.V
Σ 0
where U, E Nxp, V, xx are the first columns and V, respectively, U-p Nx(L-P) V-p E (HH- are the remaining columns, Σ, is the truncated diagonal matrix with the top p singular values, and Σ E-p is the diagonal matrix with the remaining singular values.
39

---

Published as a conference paper at ICLR 2026
The difference between the estimators is
WRRR Wols = (X X) x
(U,D, -UEV))
Σp V
(v..
(x' X
U.E.V -U, U-p)
0 E-p (VT)
-- X U D V
where _mp retains only the lower min(N, (N, H) singular values.
Using the submaltiplicative property of the Frobenius norm and the orthogonality of U and V, we have:
WRRR
WoLs/le & IIx x)- ||
1 -ell---
Since
min(N.H)
|> 1>=p|1==
Σ oi(Y),
i=p+1
and
||(x xx-xxxxX
1x 11: = -trace X(X X) x)= 2 T <
o²
min (X)
we conclude that
VL
mint H.H)
W RRR Wotslle
Σ p+p1
(Y
Mmin (X)
The Frobenius norm bound shows that when the discarded singular values are small, the RRR solution remains close to the OLS estimator. Thus, RRR can effectively suppress noise while preserving most of the signal, especially when the true underlying dynamics is of low rank.
D.6 RANK REDUCTION AND NOISE ROBUSTNESS
Rank reduction (via truncated singular value decomposition or spectral projection) is widely used and effective heuristic for denoising high- nigh-dimensional data. Intuitively, when the signal lives in low-dimensional subspace and the noise (approximately) isotropic or spread across many directions, projecting the noisy data onto the dominant rank-p subspace "concentrates" signal energy while discarding much of the high-dimensional noise.
Below we formalize two complementary theoretical explanations for why rank reduction reduces noise:
Result I (Matrix Denoising). Treat the (symmetric) data weight matrix itself as the object denoise: if M is low-rank and observe M M E, then the rank-p spectral truncation of M approximates M' with error controlled by the noise spectral norm [E]|
Result II (Linear Factor Model). When each observation arises from a linear low-rank model y; L'x + (latent factors x; and idiosyncratic noise < the sample covariance's s top-p eigenspace estimates the population principal subspace with an error that decreases with sample size n: projecting observations onto this estimated low-rank subspace reduces variance of the fitted signals and yields improved reconstruction.
The statements below make these ideas precise and give high-probability error bounds.
'Note that || [|2 with matrix inside is known the spectral norm of matrix. Specifically, for some matrix M. |M ||| Tmax (M), where Tmax (M) is the maximum singular value of M.
TThi classical linear factor model should not be confused with our linear forecast model; it aims at recovering
a low-rank factor structure from independent samples, whereas ours focuses on forecasting future values from past observations.
40

---

Published as a conference paper at ICLR 2026
Proposition 10 (Low-rank Matrix Estimation Error). Let M Ruxn be rank-p symmettic matrix with eigen- degeoppccoooooiioonn
M UUAUU+
where U TR= has orthonormal columns, and A diagg (A, (5) with M1l
0. Suppose we observe
M M E,
where E is symmetric noise matrix with entries {Eiptiss independently distributed as Eig lid (0, )²). Let U [u...... Up] and A diag(A1,-. Ap) be the top-p eigenvectors and eigenval- ues of M (in descending order of absolute value), and define
M = UAUT.
the noise level satisfies
1 v2 1xple
ovn S
then with probability least 1 O(n we have:
M - M2 S 10
- - S 10 2pm.
Sketch of the proof. We outline the main steps of the proof.
Step 1: Truncating the noise matrix. Since the Gaussian entries of the noise matrix E unbounded, we introduce a truncated version
Eg = Eig E1 s 50 /logn) 1 15455 By Gaussian tail bounds and the union bound.
E - 21
Hence, with overwhelming probability, we may replace E by E.
Step 2: Bounding the spectral norm of the noise. All entries of E satisfy Egj < B = 50 /lg
and E F S o². A matrix Bernstein/Winner-ypee inequality (Wainwright
2019) shows that
E|l2 <
40 C₁ log
with high probability. For sufficiently large n, the second term is negligible, leading to
E
50
with high probability.
Step 3: Controlling the subspace estimation error. Since the target matrix M has rank p with eigenvalue gap [*)], then under the signal-to-noise condition El|2 I (1 1/v Aal. the theorem (Davis and Kahan 1970) gives
2 E| A>
dist (U.((UUU)) <
Step 4: Bounds on eigenvalues and low-rank reconstruction. Weyl's inequality (Weyl 1912) implies that all spurious eigenvalues beyond rank are bounded by [E]]e. For the low-rank recon- struction M UAUT, the triangle inequality yields
IM M 2/ s E|
Since the difference has rank most 2p. we further obtain
M M I 2//2p |E||.
41

---

Published as a conference paper at ICLR 2026
The above result also extends to rectangular (asymmetric) matrices with independent entries via the standard symmetric dilation trick symmetric dilation
(Tropp et al. 2015). Specifically, for X E I x define its
S(X)
X
R (
X
[S(X) 2. Applying the theorem to S(X) yields the same type of bounds for X itself. For conciseness, may sometimes apply the theorem directly to asymmetric matrices without explicitly invoking the dilation.
which satisfies ||
Proposition 11 (Estimation Error of the Principal Subspace). Suppose we observe independent samples (yr)== C Rº obeying
yi= L'xi + Ei:
1sisn,
where itd Ip) are latent factors, Lid. O0 are idiosyncratic noises, and L (A Here U Rexe has orthonormal columns and A diag(Xi,-..g (2)) satisfies A 2 ... X O. Define the condition number X/X
Let the sample covariance matrix be
M
y
and denote by X 2 ... Xo its top-p eigenvalues with associated eigenvectors u Up
Let
U [uI..... Up] E Roxp be the spectral estimator of the principal subspace U*.
If the sample size satisfies
2 c 1²pp plog" (n p) +
RO²p 0Ap X
log log² +n+pp)
(xp)²
for some sufficiently large constant > 0, then with probability least 1 OOn p)00), one has
(
dist dist(U,UU". 2
kp
p
P +
log
1/2 (n +p),
n
X
n
where
tist((U, (U, U+) = min REO,xp
-R-
Please refer Chen al, (2021) for the proof of the Proposition
Note that rank reduction approach, truncated singular value decomposition (TSVD) corresponds to a Shard-thresholding operation on the singular values: retaining the top p singular values and setting the rest to zero. A more desirable approach is to perform singular value shrinkage, that is, to apply nonlinear function ()()) to each singular value o,, yielding the estimator
X,
Σ n(o:)u,v;
This compensates for the "inflation" effect of noise on the singular values and thus improves estimation accuracy. Within an asymptotic framework provided by random matrix theory (Tao) 2012) (as m, - with aspect ratio 3 fixed), the authors (Gavish and Donoho 2017) show that for given loss function (such as the Frobenius norm MSE), there exists a unique asymptotically optimal singular value shrinkage function whose asymptotic risk is no worse than that of any other shrinkage rule across all low-rank models. This implies that in this framework, once the loss function specified, the optimal form of singular value shrinkage is uniquely and rationally determined.
42

---

Published as a conference paper at ICLR 2026
D.7 ANALYSIS OF RooT PURGE
When X and Y share the same dimensions (i.e., X, Y E RNx,), the modified objective function of Root Purge:
J(W) XXIIXXXX--IIIXXWVWE-E Y A |
-
+
admits special structure. Here, W E RLX2 is now square matrix. In practice, the optimization of J(W can also be carried out using a stop-gradient strategy, where the residual in the purge term R = Y - X is treated as constant when updating W. Since the stop gradient case tends to be easier, we mainly analyze the full full-gradient version of root purge and give conclusions of stop-gradient version in the remarks.
Proposition 12 (Equivalence Condition). The ordinary least squares solution Wo (X - is a critical point of the modified objective J(W) if and only if the residuals E Y XWWoss satisfy:
EW OLS
O.
Proof. The gradient of J(W) with respect W given by:
((W = 2x R + 2X(R RW - -x RWW
where R = Y XW denotes the residue. Consider the least squares solution WOLS (X X and define the corresponding residual as E = Y XWoLs. Since Wols mini- mizes the squared loss YY XWII. the residual is orthogonal the column space of X, which implies:
X E 0.
Substituting W WOLS into the gradient expression yields:
JWoL(W = -2 E 2XE EWOLS 2AX EWOLS W
OLS 2XE EWoLs.
Therefore, Vwos(WoLs) if and only if
E E = 0.
This proposition highlights a key geometric insight about when the ordinary least squares (OLS) solution remains optimal under a modified loss that penalizes residuals the direction the solution itself. Specifically, the condition E EW OLS 0 implies that WOLS must lie in the null space of the residual matrix E. In other words, the prediction directions encoded WOLS must be orthogonal to the residual errors. This condition ensures that the additional penalty term, which measures how much the residuals are aligned with the predictions, has no effect at W WOLs- Geometrically, means that the model's learned directions do not amplify interact with the error left over after fitting, preserving OLS as a critical point of the new objective. This equivalence gives a deeper understanding of how structured residuals model alignment can impact regularized learning. Moreover, we have the following result:
Proposition 13 (Bounded Deviation of Root Purge). For the minimizer W of J(W its deviation from W Ols is bounded by:
E @EWosslir
IW" Worslle Omin (XT (XXX)(1/X + o oiin (Wols)) + Jmia E)
where o auin (-) denotes the smallest singular value of matrix.
*Our analysis focuses on the local behavior around Wols. as preserves the most informative dynamics of the system. This restricts W to neighborhood of Wols. allowing us to neglect higher-orde order terms in A if deviates too far from this neighhorhood, the root-seeking term may incur large loss, potentially compromising the system's integrity.
43

---

Published as a conference paper at ICLR 2026
Proof. The gradient of J(W with respect W is given by:
J(W) - R + 2A(R RW X RWW)).
where Y denotes the residue. Setting V JwJ(WW) = 0 gives:
X X(R R X R W W = 0.
Then from E = Y Y-XWoxWo OLS. we have R Y -x- = E+ XWOLS - XW -E- E
WOLs). Substituting this into the above equation gives:
-x EE X(W" Wors))
+ E X(W WoLS)) LE-X(W" Wors)) W
x 20-XWW- WoLs) W W = 0,
Applying X E = 0 and dropping the higher-order terms gives:
X XA + AE EWS + AE EA + XX XA WOLS W = 0,
where W Wors. Vectorize this equation using vee(.) and the Kronecker product @:
[I+ XWLLS O @ X X + AI
vec(A) - EWors).
Let A- [I + XWLLS W OLS & X AI 2 E E. Then we have:
| | |/vec( ||) < A 'Ilellvec(AE) EWois) A||A-Ila EWolsle (10)
The singular values satisfy:
Omin (A) 2 mmin X X J (I XWo oLs WOs)) + Aa (E E)
min X X)(1 + A (ors)) + Aomin E E)
where we used muin (I AWOLS W OLS) 1 2 (WOLs). Thus have:
|| 1 S miin
X (1 + oo min (WoLs)) Am E E)
Substituting this into Equation (T0) yields the final bound:
E EWoLslIF
|W WolsllF
T m
X (T)(11/A + o (Wo.s) + J (E E)
This proposition provides meaningful upper bound on the deviation between the minimizer W of the modified objective J(W) and the ordinary least squares (OLS) solution W Wors- Several important insights emerge from this result:
1. Bounded Deviation:
This bound quantifies how much W noise and regularization. A smaller X, larger singular values (i.e., hetter-condiiioned matrices), or well-structured WoLs all help tighten the bound.
away from WOLS due to the influence
2. Bias vs. Robustness Trade-off:
Although W may not be an unbiased estimator-aa as deviates from the OLS solution the bound shows that this deviation is explicitly controlled by the regularization parameter X, the norm of Wols. and the spectral properties of X X and E. This introduces robustness: rather than fitting noise in the data (as OLS can), the solution remains stable under perturbations, especially when the noise matrix E is significant.
3. Optimization Warm Start:
Since the bound suggests that W lies close to WOLS. practical implication is initialize iterative optimization algorithms at W Wols- This warm start can lead to faster convergence by starting closer to the final solution.
44

---

Published as a conference paper at ICLR 2026
In summary, while the estimator W is biased relative to WOLs, the trade-off results in enhanced robustness to noise, controlled deviation, and practical optimization benefits, making this formulation useful in settings where noise and overfitting concerns. Combining Proposition 8 Proposition 9 and Proposition [3 with the triangle inequality allows us to bound the Frobenius norm errors | WRRR |r and | WDWRRr [[,, suggesting that Root Purge and rank reduction techniques can have similar effects on controlling model complexity and noise sensitivity. Therefore, these methods provide alternative yet related strategies for regularization in multi-channel forecasting problems.
Remark (Equivalence Condition of Stop-Gradient Root Purge). For the poopggrddien objective
J(w) = I - XWI + X R|
where the residual matrix R treated as constant during optimization, the same conclusion from Proposition 12 holds.
Remark (Bounded Deviation of Stop-Gradient Root Purge). For the stop- graddird objective
J(w) = - + RW
where the residual matrix R treated constant during optimization, the gradient term become T R -2X
(((W)
2XR RW, and the bound as stated in Proposition 13 becomes
E EWoLss
I Worsll S
1/A Tmin
XXX
Omin
E)
To further illustrate the relationship between Root Purge and Rank Reduction, we have the following results:
Proposition 14 (Anisotropic Shrinkage and Conditional Singular Value Shrinkage at the Stationary
Point), Assume that > 0, and W* satisfies the orthogonality condition
X XW'
Then there exists symmetric positive semidefinite matrix S = such that
W S WOLs,
where
S = (1+ R*) R')
we further assume that X X and R. R.R.R. simultaneously diagonalizable. Then the unique minimizer W satisfies
az (,W))) (Woss), Vi.
Proof. Ret XW. Using standard matrix calculus, the gradient of is given by VJ(W) = - R + 2X (R RW - RRWWT).
At stationary point W VJ(W*) 0, which yields
X R' AX R
AR- R'W'
Under the assumed orthogonality condition X R W* = 0, the second term on the left-hand side vanishes, and the above equation reduces
[ + AR
R R* X
= =
R 2 0, and = X TY Then
W - (A+B)- -C,
WOLS A-
Using the identity
( + -I+ A
45

---

Published as a conference paper at ICLR 2026
obtain
W = [I+A- B)
WOLS.
5
Since A- Z O, follows that 0 < S 3 I, completing the proof of our first conclusion. Addi- tionally, by further assumptions, A and B are simultancously diagonalizable, Hence there exists an orthogonal matrix U such that
A diiag(a)
B B=U
U diag(b;)
b, 2 0.
Therefore,
( + A)+B) U- Udiag
UT. U
UT. A = Udiag
+ b,
Let Č = UTC. Then
W
U diag
C
WoLs
U diag
1 Č
a; + by
Since a, + 6 2 a; for all i, it follows that
Vi.
ag di
Consequently, each singular value of W is bounded above by the corresponding singular value of WOLS, which completes the proof.
The first part of Proposition 14 establishes that. under a mild orthogonality condition, the stationary solution W can be expressed as linear transformation of the OLS estimator via a contraction operator S z I. This result reveals anisotropic shrinkage mechanism, where different directions in parameter space are suppressed to different degrees depending on the residual covariance. With further assumption in the second part, we can additionally perform component wise comparison of singular values between W' and Won VO.s.
The simultaneous diagonalizability of X and R R. should be interpreted as analytical choice of coordinate system rather than a restrictive modeling assumption. Indeed, the result characterizes shrinkage behavior in the common eigenbasis of the data covariance and the residual covariance. In particular, directions associated with large residual energy experience stronger suppression, while directions well explained by the data are preserved. As the regularization parameter À increases, this selective attenuation progressively eliminates low-energy singular modes, effectively reducing the rank of the solution. This theoretical behavior is clearly reflected in our empirical results, as shown in Figure and Figure [4] where increasing À progressively attenuates smaller singular values. This supports the interpretation that Root Purge acts similarly to rank reduction techniques by selectively suppressing spurious or noisy components (roots) in the solution. Hence, Root Purge can be seen as principled mechanism for regularizing overparameterized models by promoting low-rank structure.
Remark (Anisotropic vs. isotropic shrinkage). Unlike ridge regression, which applies uniform (isotropic) shrinkage across all directions, the shrinkage operator S depends on the residual covari- ance R R and therefore induces dependen,,,,nt, directionally selective contraction of the OLS estimator.
Remark 8 (Norm contraction). As a direct consequence of 0 < S - I, we have
| < | ooss 12,
|| || < Wots||F
Remark (Shrinkage of Stop-Gradient Root Purge). Consider the stop- nop-gradient objective
J(W) Y XW
R|
where the residual matrix R is treated constant during optimization. We can drop the orthogonality condition that X XT XW W 0. We still arrive at the conclusion of Proposition 14 given the rest of assumptions are satisfied.
46

---

Published as a conference paper at ICLR 2026
Finally, to support the implementation and theoretical analysis of our algorithm, present classical convergence result for gradient descent in the non-convex setting. Although our objective function is non-convex, the following proposition guarantees that gradient descent with properly chosen step size converges to stationary point. This forms the basis for understanding the optimization behavior in our setting.
Proposition 15 (Convergence of Gradient Descent Algorithm), Let ((W be a differentiable with L-Lisschitz continuous gradient:
[VJ((1)) - JJ(Wa)llr & LI - W2lle, W1,, W2.
Assume J(W is bounded below by J Consider the gradient descent update:
W W W with 1
L
Then, after K iterations, the minimum gradient norm satisfies:
2L(J(Wo) JJ)
min
[VJ(W)) &
OSKcK
K
Proof. By Lipschitz continuity of VJ(W). for any W and W W -VW)):
J(W") s J(W) + (VJ(W), W W) + L
W WIE.
Substituting W - = ---nVJ(W) gives:
J(W' S ((W m/W + Li²
2 [J]()
With L. we have:
J(W 1
S J(W.) 2T. [VJ(W,)
Summing over k = 0
K - gives:
K- Σ
K-1
VJ(W S Σ
- J(W.) J(WK)
2L
k=0
k=
Since J(W K) Jr, get:
VJ(W S 22(((WWo) JT).
Taking the average gives:
K
2L()(Wo) JJ)
VJ(W 1)
K
K
k=0
implying:
2L(J(Wo) J
min oskck
[[J(W))
K
As a consequence of this proposition, we see that gradient descent reaches an e-stationary point ([) S c) O(1cc) iterations. While recursively solving least square may provides faster convergence through its second-order approximation, it requires solving a linear system with an additional computational cost of O(L3) per iteration due to the matrix inversion of X T X AR, R2 (computational costs in matrix multiplications are not counted as it is also required for other algorithms). In contrast, gradient descent avoids this cubic scaling, making it preferable for extended lookback window problems where L > 1. Note that this convergence result applies to any objective function satisfying the Lipschitz gradient condition. In particular, applies to our specific optimization problem:
47

---

Published as a conference paper at ICLR 2026
Remark 10. The gradient (WW)) is Lipschitz continuous with constant L satrisfyin
L S 2 X + 2A(6M | 2 2 + 2
(1I)
where:
M maxwew
W for W - J(Wo)):
|X||2 and || spectral norms.
Remark 11. the stopgradieett objective
J(W)- || XWII X|RWII
where the residual matrix R treated as constant during optimization, the gradient VJ( is Lipschitz continuous with constant I, satisfying:
5 2 2M(3M 1X 4M| Y +
(12)
D.8 ANALYSIS OF CHARACTERISTIC ROOTS
To this point, have explicitly formalized the connection between rank reduction and the resulting restructuring of polynomial roots. In the following proposition, we establish this relationship rigorously. highlighting how the structure of the underlying system governs the interplay between low-rank approximations and root perturbations an idea central to the focus of this paper.
Proposition 16 (Root Perturbation under Low-Rank Approximation). Let W RL be coeffi- cient matrix where each column w, defines a monic polynomial:
p = 1
W Wigrh-i
with distinct roots (r()) ...... ,...., "....,,LLt--- 11 - each -,...,, H. Let W be approximated coefficient
(1)
matrix with error bound ||W - W | S. Then for each column polynomial Pr(r) of W with roots (r(5) ...., ,...,, the following root perturbation bound holds:
max ISiSL+j- 1 1gi 1 1
min
where Kj is the condition number of the root-finding problem for Pr(r), dependent on the minimal separation between roots.
the perturbation in each column satisfies: [|w, will2 S For each polynomial (r),, consider its companion matrix:
Proof. Given | W
C₁ = [S w/l D(L+S--((Lt)-)
where:
01-1] w
w/
(zero-padded coefficient vector);
S is the shift matrix:
0
E [[(++j-)x((+t)->
=
0
with ones on the first subdiagonal and zeros elsewhere.
9The Lipschitz constant L for (J(W) can be estimated as two parts: root-seeking term Y XW] contributes to 2||X| I1 and root root-purging term |I XW)WII has two gradient components: 2 RW and 2AX RWW! The bound of L can be found looking each component separately. The first component give us the bound 2A [||]||| Y||a ||||22M f (|Y))> l| + ||X|/MM and the later give us the bound of 2A [|| Y| + |X M) + | M? via triangle inequalities and submultinlicatiity.
48

---

Published as a conference paper at ICLR 2026
Note that the eigenvalues of C are exactly {rt (1 L+j-1
( Under perturbation A; C2 Cj. where
C is the companion matrix for (r)), we have:
A = Ilw w S
Ji=1 By assumption {r( ( (+)-++111 from C are all distinct, applying Bauer-Fike theorem to a eigenvalues
()) from Cj, we have the following:
min 1 100 ,(VV))
where V is the Vandermonde matrix that diagonalizes C, as <(V; x(V V,ll2llV,'I1 2|2 the condition number of the eigenvector matrix for C,, For distinct roots, the following form controls the upper bound of the conditioning (Gautschi) 1962):
A, V,, and <(V,) =
L+j-1 Σ VTX
1+ |rm =: Kj
<(V,) S [-+j-10 1)
max
P
max
m=
Pm
directly evalunting
\ ol
Gautschi's bound of 1 11
The leading term of the above equation is result of the inequality between || and ... 1ox controlled solely by matrix shapes. In this case we have ||V S VL+j- 1| flo as V E (((+)-I)x(L+--I), Combining these results yields:
max min (R!0 - 51 S kg 1
From this proposition, we see that the bound on root perturbation is directly controlled by the Frobenius norm of the approximation error || W. Since we have explicitly derived Frobenius norm bounds for each method, including Root Purge, RRR. and DWRR, the proposition ensures that the resulting perturbations the root structure are similarly bounded. Consequently, the leading roots are preserved, while less significant roots may shift or collapse. This behavior reflecs the structural influence imposed by the rank constraint and reinforces the central theme of the paper: that rank reduction reshapes the root landscape in principled and quantifiable manner.
E EXPERIMENTS
E. DATASET
We provide summary of the dataset we used in Table For ETT (Zhou al. (02T) data, we follow previous works (Lin et al. 2024 Xu et al. 2023 Wu et al., 2022 and divide the dataset into train, val, and test sets with a 2 2 ratio. For the rest of the datasets, they are divided in to train, validation, and test sets with a 7 1 ratio. The characteristics of these datasets are further illustrated by representative samples Figure I2
For the synthetic dataset we used in Section 5.3 to study data scaling property, noise robustness, and root distribution, the train, validation, and test sets are split with a 5: 2.5 2.5 ratio.
Table Detailed information of the datasets we used in this paper.
Datasets
ETTh1
ETT12
ETTml
ETTm2
Exchange
Weather
Traffic
Electricity
No. of Channels
7
7
862
321
Timesteps
17420
17420
69680
69680
7588 Daily Economy
$2696
17544
26304
Frequency
Hourly
Hondy
15min
15min
10min
Hourly Traffic
Houryy
Application Domain
Electricity
Electricity
Electricity
Electricity
Weather
Electricity
Additionally, we also use M4 (Makridakis et al. 2020) dataset to test the robustness of our methods under short lookback window well their performances in short-term forecasting tasks. The M4
49

---

Published as a conference paper at ICLR 2026
Visuaization
Visudieatiion Weathhr
Taining catenn
Trining VONASAaOO
2008
Lobes
Steps
Time
(a) ETThl
(b) Weather
Visualization Exchange Rate
Visualization of Traffic
Tuining
Vell oaten
0.22
16-
0.05
6/08
Skops
Stops
(c) Exchange Rate
(d) Traffic
Figure 12: Representative samples from the ETTh1, weather, exchange rate, and Traffic datasets,
illustrating the complex, non-stationary trends characteristic of real-world time series data.
dataset consists 100,000 real-world time series from diverse domains such as finance, economics, demographics, and industry. Each subset of M4 is collected in yearly, quarterly, monthly, weekly, daily, and hourly frequencies, with a mixture of data from the domains mentioned before. For the M4 dataset, the lookback window L is set to H, where H is the forecasting horizon, which differs from long-term forecasting datasets. The details of M4 dataset are listed in Tabel g
Table 8: Detailed information of the M4 dataset.
Datasets
Yearly
Quarterly
Monthly
Weekly
Daily
Hourly
No. of Channels
1
Forecasting Horizon (H)
13
48
Training Set Size
23000
24000
48000
359
4227
414
E.2 IMPLEMENTATION DETAILS
Main Experiments. In experiments for both RRR and Root Purge, the length of lookback window is chosen to be 720. The major reason is that the lookback window determines the number of roots linear model can capture, which is the key to the generalization behavior of linear time series models. This setting is also consistent with prior works in linear time series models (Xu et al. 2023 Lin et al. 2024). For experiments with RRR, we use singular value decomposition (SVD) to directly solve linear regression for the least squares (OLS) in the time domain on the CPU. Each experiment for RRR is only done once, as the results are produced in close form and therefore static.
We adopt frequency-dome domain linear layer for experiments with Root Purge for its better training stability (Xu et al., 2023 Yi et al. 2023). Due to the linearity of the Fourier Transform and the inverse Fourier Transform, learning complex-valued linear layer in the frequency domain still results in an overall linear mapping, on which the Root Purge algorithm has theoretical guarantees for the effectiveness of the rank-nullity trade-off. We provide comparison of linear models in the time domain and the frequency domain in Table 12 Further, we provide the results performing Root Purge the time domain in Table 13
50

---

Published as a conference paper at ICLR 2026
Transforming Frequency-Doman Weights to the Time Domain. We explain how to transform learned weight matrix in the frequency domain, denoted WF, F, into its corresponding iimeid representation Wr. Consider general setting where the model defines a linear operator Gw (-) (which may involve a composition matrix W equivalent to G()), L.e. Gw (X) = XW for any input X. A convenient way to obtain W by evaluating Gw on the identity matrix I:
multiple linear transforms). Our goal is to recover a single
W (I
Since Gw represents the trained model, we can compute W directly by feeding the identity matrix into the model. This method is general and applies to any linear operator (so that we must have W Wr). whether defined in the time or frequency domain. Specifically, when the model is implemented as frequency -ommmnn transform,
Gw = WF
which yields the full time-domain weight matrix WT associated with the learned operator.
A similar approach can be used to approximate nonlinear operator G with linear one. Evaluating G on the identity matrix yields: W ~ GTT), which defines a linear surrogate W such that: g(X) x WX. This is particularly useful when G is locally linear, offering tractable way to analyze or approximate nonlinear models, including neural networks or nonlinear filters.
Computing Resource. All experiments of Root Purge are implemented using Pytorch (Paszke ct al. 2017) and conducted on a single NVIDIA V100 32GB GPU. We employ MSE (Mean Squared Error) as both our loss function and the evaluation metrics, following prior works in linear time series forecasting and our theoretical analysis. Each of the experiments for Root Purge is repeated 5 times remove run-to-run differences.
For more details on the running settings of Root Purge, please refer to our code and running scripts in the supplementary materia 10
E.3 DETAILS OF BASELINE RESULTS AND COMPARING WITH ADDITIONAL BASELINES
We provide details for obtaining baseline results. For many previous works, the results listed impacted by long-standing bug" mistakenly setting drop asst=ruue for the test and validation dataset. For this reason, either cite results with this bug fixed, reproduce the results with the official open source code with this bug fixed. The details of our baseline results are as follows:
FEDformer (Zhou al. 2022) Transformer-based model with sparse attention. We cite the results with this bug fixed from Xu et al. (2023) and reproduce missing results with the open-source implementation https:://///..
PatchTST (Nie et al. 2022) Transformer-based model considering a short time se- single token. We cite the results with this bug fixed from Xu et al. (202)) and reproduce missing results with the open-source implementation at https: : //tp///ggtt/./// wwwp///tmmmmiieeeeeeeeeeeeee
ries patch
TimesNet (Wu et al. 2023) Convolution-oe based model applied on time series that is converted to 2D. We cite the results with this bug fixed from Xu et al. (2023) and reproduce missing results with the open-source implementation at httpp::/ tttpbb..ttt...og.. www/to/mmuute.ecce.. /T me- es- bra
TSLANet (Eldele al. 2024) Convolution-basee based model. The official implementation affected by the bug mentioned before. Thus, we reproduce the result using the open-source implementation at ht https: / / /github. con/ emadel deen: 2: / Ttc//SLAA TSLANet/. / For fair comparison with our results, we use fixed lookback window of 720. The rest of the setup follows the script provided in the repository.
Plain FilterNet (Yi al. 2024 Convolution based model with 2-layer MLP head. The official implementation is affected by the bug mentioned before. Thus, we reproduce the result using the
OOur code is available at http:///ihhh.commaaaazzzr//tttor
See details in htt.://nnnt////gggt.ooooooooonon
51

---

Published as a conference paper at ICLR 2026
open-source implementation at bttps://qithub /qtthtbtttuh/gggg/eee//.e....... wwtlli..oaaaaiiaaaeeiaee.eee.ece comm/akkunii/PPitrrree...
For fair comparison with our results, we use a fixed lookback window of 720. The rest of the setup follows the script provided in the repository. When no official shell script is provided, use the default
settings in the code.
FITS (Xu et al. 2023) Linear model the frequency domain. FITS takes lookback win- dow as a hyperparameter. Thus, for a fair comparison with our results, we use a fixed lookback window of 720 and reproduce the result using the open-source implementation https: / /httbbb....uu COom/VEWXIC//// /FITSI/ The rest of the hyperparameters follow the offi- cial implementation that yields the best results.
SparseTSF (Lin et al.
2024 Linear model with convolution backbones. We cite the result from the
original paper, as already contains the result after fixing the bug.
DLinear (Liu et al.
Linear model with moving average. We cite the result with the bug 2023) and reproduce missing results with the open-source implementation at
2024)
fixed from Xu et al.
https: / htt//githuu huu......... wtt.e////////// won.vvvwa//Lineaa.
E.4
COMPARING TRAFFIC AND ELECTRICITY DATASET
Due to data scaling property in Proposition 1 we expect RRR and Root Purge to show more significant improvements on small-scale datasets. Under CI settings, large datasets such as Electricity and Traffic constitute huge amount of data for training. It is clear in Table Table@9tha that complex non-linear models achieve better results than linear models, taking advantage of larger model capacity.
For linear models, these large-scale datasets allow any linear model to approach the optimum via only the root-secking loss. In Appendix [E.8. we reaffirm that although the numerical analysis from RRR and DWRR still shows that the optimal rank of W is not full, the performance of full-rank models can come close to that of their rank- trknccted counterparts.
As shown in Table 9 the performance of all linear models is very close these two datasets. RRR and Root Purge either match or outperform other linear models (i.c., DLinear, FITS, and SparseTSF). Notably, for Root Purge, while data scaling has already pushed the model close its optimality, performance remains robust with a relatively large regularization coefficient (A = 0.125). This reaffirms the robustness of our methods.
Table 9: Forecasting result for ECL and Traffic with horizon H E {96, 192, 336, 720} with lookback
window of length L 720. Following the main experiment settings, for RRR, we tune the rank on the validation set, select the top three with lowest validation MSE, and report the best test result. For Root Purge, we select the best test MSE from a hyperparameter search over [0.125, 0.25, 0.5). The best results are highlighted in red, and the second-best in blue. Methods we proposed in this paper are bolded this table.
Dataset H
PEDformer
FilterNet
ANet
TimesNet
PutchTST
DI.inear
SparseTS
FITS
RRR
Root Purge
96
0.188
0.136
0.129
0.168
0.129
0.140
0.138
0.135
0.133
0.134
BCL
192
0.197
0.147
0.184
0.153
0.151
0.148
0.148
336
0.212
0.168
0.162
0.198
0.166
0.169
0.166
0.165
0.164
0.165
720
0.244
0.207
0.196
0.220
0.210
0.204
0.205
0.205
0.203
0.205
96
0.573
0.393
0.362
0.593
0.366
0.413
0.389
0.386
0.385
0.386
Trallic
192
0.611
0.408
0.377
0.617
0.388
0.423
0.398
0.397
0.396
0.397
336
0.621
0.424
0.391
0.629
0.398
0.437
0.411
0.411
0.410
0.411
720
0.630
0.462
0.430
0.640
0.457
0.466
0.448
0.449
0.448
0.450
E.5 COMPUTATIONAL COST & COMPLEXITY ANALYSIS
While the focuses of RRR, DWRR, and Root Purge are not maximizing computational costs, provide the following analysis of the computational complexity.
RRR and DWRR For inference, under the most general setting and within a channel-independent framework, if consider dataset with channels, the computational complexity for all linear models is Oc L- H), where L is the input sequence length and H is the output sequence length. However, due the rank limitation we impose the intermediate weight matrix, the complexity
52

---

Published as a conference paper at ICLR 2026
in methods (RRR and DWRR) be significantly reduced to Olc p (H L)), where < min(H, L) denotes the rank the matrix.
For training, the computational complexity of the RRR algorithm can be broken down into four steps:
Estimating W Woss: O(NL2 NLH L3 L2H)
Calculating the estimation: O(N
Performing SVD: (minnNNH² ,N²
Rank truncation: OHHp²² p)
Overall, given that the sample size is generally much larger than both the imput and output lengths, and due to the low-rank property of the weight matrix, the total computational complexity can be approximated as O + LH).
For the DWRR algorithm, the training complexity involves three steps:
Estimating WOls: O(NI? NLH 1 + I.²H)
Performing SVD: (min(LHH² I2H))
Rank truncation: p(LHp)
Similarly, under general conditions, the total computational complexity is approximately O = NLH).
Root Purge For the Root Purge algorithm, since its training process relies on gradient-based optimization, we measure the training memory consumption and iteration speed on an NVIDIA V100 GPU. In addition, we also measure the inference time and parameter size. We summarize the results on the ETTh2 and ECL datasets Table In]and and Table respectively.
Table 10: Computational measurements based ETTh2 dataset with batch 8. Lookback window length is set to L 720 and forecasting horizon is set to = 192.
Model
Train GPU Memory (MB
Training Time (ms/iter
Parameter Counts (J)
Inference Time
SparseTSF
8.6K
0.28
DLinear
410
1.5
415.3K
0.30
FITS
414
1.9
48.9K
0.53
FilterNet
434
4.0
235.3K
0.55
TSLANet
604
3.7
605 4K
1.25
Koopa
25.1
252.8K
2.79
PatchTST
1028
13.9
2,611.84
3.03
Root Purge
432
1.7
165.0K
0.29
Table 11: Computational measurements results based on ECL dataset with batch 8. Lookback window length set to L 720 and forecasting horizon set to 192.
Model
Train GPU Memory (MB ())
Training Time (ms/iter
Parameter Counts (L)
Inference Time (ms J)
SparscTSF
488
8.3
8.6K
0.42
DLincar
472
5.2
413.3
0.49
130.0K
FilteNett
550
6.2
235.3K
0.81
TSLANee
4382
91.5
1,581.1K
2.57
Koopa
1920
56.9
63,777.2K
50.89
PatchTST
15266
264.0
2,61 1.8K
3.73
Root Purge
534
6.1
165 OK
0.47
As you can see, Root Purge is able to achieve SOTA performances while using training/inference resources.
E.6 LINEAR FORECASTER WITH BIAS AND WITHOUT BIAS
We list our experimental results comparing the setting of with/without bias in Table 12 The experiments are done for both the time-domain linear block and the frequency domain linear block.
53

---

Published as a conference paper at ICLR 2026
We repeat our experiments 5 times and include the error bars in the table, and the experimental settings in this section follow our main experiments.
Although the gap tends to be larger when fitting time domain model, for both frequency and time domain models, the one without bias is performing better than/equally well as the model with bias.
The result from this section can also serve non-regularized baseline for Root Purge, where use = O.
Table 12: Comparison of the time/frequency--onne -omain linear model with bias and without bias. The model without bias tends to perform better/equally well (better results in red) compared to the model with bias in both time and frequency domains. Overall, the time domain model without bias performs similarly to the frequency domain model without bias, with only minor degradations.
Time Domain
Frequency Domain
Dataset Horizon
With
Without
With Bias
Without Bias
96
0.379 = 0.001
0.378 1 0.001
0.375 + 0.001
0.374 + 0.001
192
0.415 = 0.000
0.415 + 0.001
0.410 + 0.000
0.410 + 0.000
336
0.448 0.002
0.443 + 0.001
0.432 + 0.001
0.431 +: 0.001
720
0.459 1 0.001
0.435 1. 0.002
0.428 1 0.000
0.427 1. 0.000
0.275 0.001
0.276 0.001
0.272 0.000
0.273 0,000
192
0.334 = 0.000
0.334 1. 0.001
0.332 1: 6,000
0.332 1: 0.000
0.361 0.002
0.360 + 0.002
0.355 +: 0.001
0.354 + 0.001
720
0.396
0.001
0.384 1 0.003
0.379 +: 0,000
0.378 +: 0.000
ECL Exchanng Weathe ET1m2 k77ml
96
0.311 = 002
0.310 + 0.001
0.310 + 0.001
0.308 = 0.002
0.343 0.003
+ 0.340
0.337
336
0.369
0.001
0.369 1 0.001
0.367 1 0.000
0.366 1. 0.000
720
0.418 0.002
0.417 + 0.001
0.416 + 0.000
0.415 + 0.000
0.163
0.001
0.163 1 0.001
0.164 1 0.000
0.164 1 0.000
192
0.217 = 0.001
0.217 + 0.000
0.218 = 0.000
0.219 = 0.000
0.270
D.001
0.270 1
0.269
0.269 1
720
0.354
0.001
0.353 + 0.001
0.350 + 0.000
0.350 + 0.000
96
0.143
D.000
0.143 1 0.0D1
0.145 1 0.000
0.145 E 0.000
192
0.186
0.000
0.186 1 0.001
0.188 1: 0.001
0.187 1 0.000
336
0.237 = 0.001
0.237 + 0.001
0.238 + 0.000
0.238 + 0.000
720
0,310
0.000
0.308 1 0.001
0.309 0.000
0.309 1. 0.000
0,091 = 0.001
0.087 + 0.000
0,088 - 0.001
0.084 Σ 0.000
192
0.192 0.001
0.178 1 0.001
0.186 1 0.000
0.176 1 0.000
336
0.370 0.001
0.328 1 0.003
0.346 + 0.001
0.329 + 0.001
720
1,084 = O. 0.022
0.925 + 0.011
0.961 - 0.003
0.985 + 0.002
0.134 = 0.000
0.134 1 0.000
0.135 0.000
0.135 1 0.001
192
0.149 0.000
0.149 + 0.000
0,149 + 0.000
0.149 + 0.000
336
0.165 = 0.000
0.165 1 0.000
0.165 1 0.000
0.165 1 0.000
720
0.205 0.000
0.204 + 0.000
0.206 0.000
0.205 + 0.000
96
0.386 0.001
0.386 1 0.000
0.385 1 0.000
0.385 1 0.000
Traffic
0.397 0.000
0.397 + 0.000
0.397 + 0.000
0.397 + 0.000
336
0.411
0.001
0.411 + 0.001
0.411 - 0.001
0.410 + 0.000
720
0.449 0.000
0.449 1 0.000
0.449 + 0.000
0.449 1 0.000
E.7 ADDITIONAL EMPIRICAL RESULTS FOR ROOT PURGE
Root Purge on Time Domain In our main experiment (Section 5). we use a frequency-domain linear block following Yi et al. (2023) and Xu
Xu et al. (2023). In this section, we show that Root Purge also works well on the time-domain linear models. Following the experimental setting in the main experiment, we perform hyperparameter selection with E [0.125, 0.25, 0.5] to obtain our results. The full experimental results for each choice of hyperparameter can be found in Table [5] and we give a summary table for the result with the best hyperparameter in Table [3]
Hyperparameter sensitivity and singular values are also studied for time-domain linear models with Root Purge. We provide hyperparameter sensitivity in Figure [13 and singular value spectrum in Figure 14 Overall, the hyperparameter sensitivities and singular value trend for Root Purge in the time domain are similar to those in the frequency domain.
Full Experimental Results of the Impact of Root Purge Hyperparameter We conduct a broader hyperparameter search with x E [1/32, 1/16. 1/8, 1/4, 1/2. 1, 2] for ETThl and ETTm1; we show a complete result for each of H = {96, 192, 336, 720} in Table 14, Overall, a large range of will yield improved results. Still, there are some differences in the hyperparameter sensitivity across
54

---

Published as a conference paper at ICLR 2026
Purge hyperparameter Sensitivity
Purge Hyparpacameter SensRivity
1.415
1.455
3.444
0.90
0.00
12.00 9mh 18-1)
0.054
1.2
2/3
1/4
10
121
314 12
Figure 13: (Time Domain) Average forecasting MSE on ETThl and ETTml across horizons H (96, 192, 336, 720} for different values of X. The hyperparameter sensitivity of the time-domain linear model is similar that of the frequency-doman linear model, a break-down table for each horizon for the time-domain linear model is in Table [4]
linear model. As for the frequency-domat
Singular Value Spectrum WonE
Singular Value Spectrum of W on ETTm]
Magmtoas lop 30300
10
Siipuaor
Figure (Time Domain) First 336 singular value magnitudes on ETThl and ETTml under different values of À (log scale) with W. As À increases, Root Purge pushes the weight matrix W to have more smaller singular values, while the significant singular values remain largely unaffected. The overall effect of singular values is consistent with what found in the main text, where W is learned in the frequency domain.
Table 13: Root Purge result on time-domain linear model. This shows that Root Purge is also effective in time-domain linear models. The error bars omitted for simplicity. A full set of results is also shown in Table with error bars.
Datasel
FTTm2
Weather
Exchange
Horizon
96
0.362
0,271
0.305
0.161
0.142
0.085
0.133
0.385
192
0.397
0.330
0.335
0.216
0.185
0.175
0.148
0.397
336
0.432
0.359
0.361
0.268
0.235
0.324
0.165
0.411
720
0.423
0.381
0.415
0.352
0.308
0.932
0.205
0.450
55

---

Published as a conference paper at ICLR 2026
forecasting horizons. For instance, H = 720 is relatively more sensitive to large regularization, and for smaller H. performance tends to be more robust.
Furthermore, since the Root Purge method involves some hyperparameter tuning, we also provide empirical results with À E [0.125, 0.25, 0.5] across all datasets and aforementioned forecasting horizons. The results are shown in [S]
Table 14: Break-down results for Root Purge hyperparameter sensitivity testing on ETThl and ETTmI. Overall, the trend for the frequency domain model and the time domain model is similar. While a large range of X will yield improved results, across different forecasting horizons, H = 720 is relatively more sensitive to large regularization. For smaller H, performance tends to be robust even when a large À is used.
Datasel H
1/sa
1/16
1/s
1/4
1/2
1
2
Frequency Domuin
96
0,374
0,373
0.371
0.368
0.364
0.359
0.354
0.351
192
0.410
0.408
0.406
0.403
0,399
0.394
0.389
0.388
336
0.431
0.431
0.430
0.427
0.425
0.423
0.423
0.424
720
0.427
0.425
0.424
0.423
0.421
0.423
0.434
0.485
96
0,308
0.307
0.307
0.306
0.306
0.305
0.305
0.310
k ETThl FlTml
192
0.337
0.336
0.336
0.335
0.334
0.333
0.333
0.318
336
0,366
0,365
0.364
0.363
0.361
0.360
0.361
0.368
720
0.415
0.414
0.414
0.413
0.412
0.413
0.420
0.456
96
0.378
0.378
0.376
0.373
0,368
0,362
0.356
0.354
Time Domain
0.415
0,413
0
0.408
0.403
0.398
0.392
0.390
336
0,443
0,442
0.440
0.438
0.435
0,432
0,430
0,430
720
0.435
0.434
0.432
0.429
0.426
0,423
0,432
0.484
96
0,310
0.309
0.310
0.309
0,308
0,306
0.306
0.311
0.340
0.340
0.339
0.337
0.337
0.335
0.335
0.339
336
0,369
0,368
0.368
0.366
0.364
0.361
0.362
0.368
720
0.417
0.416
0.415
0.415
0.415
0,415
0.421
0.456
Table 15: Full experimental results with Root Purge. The hyperparameter sets, following the main experiment, are À E 10.125, 0.25, 0.5].
Frequency Domain Linear
Time Domain Linear
Dataset
Horizon
À = 0.125
A= 0.25
A = 0.5
A 0.125
A= 0.25
A 0.5
96
0.368 + 0.000
0.364 :: 0.000
0.359 :: 0.000
0.373 + 0.002
0.367 + 0.002
0.362 : 0.000
BCL Exchange Weather ET1m2 811ml E1Th2 ET
192
0,103 0.000
0.399 + 0.000
0.391 + 0.000
0.408 + 0.001
0.403 + 0.001
0.397 = 0.001
336
0.427
+ 0.000
0.425 + 0.000
0.423 + 0.000
0.438
+ 0.001
0.435 + 0.001
0.432 - 0.001
720
0.423
0.421 + 0.000
0.423
0.000
0.129
+ 0.001
0.423 0.000
96
0,270 1: 0.000
0.269 1 0.000
0.268 1: 0.000
0.274
1: 0.001
0.273 1773:1000 0.001
0.271 :: 0.001
0,330
+ 0.000
0.000
0.332
0.001
336
0.354 + 0.000
0.356 + 0.000
0.366 + 0.000
0.360
+ 0.002
0.359 + 0.001
0.360 - 0.001
720
0.377 0.000
0.377 + 0.000
0.379
0.000
0.383 0.003
0.382 0.002
0.381 0.000
96
0,306 0.000
0.306 1 0.000
0.305 1 0.000
0.309 1 0.001
0.308 1 0.001
0.305 3: 0.001
192
0.335 0.000
0.331 + 0.000
0.333 + 0.000
0.337 + 0.001
0.337 + 0.001
0.335 0.002
336
0.363 0. 000
0.361 + 0.000
0.360 + 0.000
0.366 0.001
0.364 0.001
0.361 = 0.001
730
0.413 + 0.000
0.412 + 0.000
0.413 + 0.000
0.415 + 0.000
0.415 + 0.001
0.416 - 0.001
96
0. 162 D. 000
0.161 : 0.000
0.161 - 0.000
0.161 + 0.000
0.161 + 0.001
0.161 = 0.001
192
0.217
+ 0.000
0.217 + 0.000
0.216 + 0.000
0.217 + 0.001
0.216 + 0.001
0.216 - 0.001
336
0,269
0.269 +
0.000
0,268 0.001
0.268 0.001
720
0.350 + 0.000
0.350 + 0.000
0.365 + 0.000
0.352 + 0.000
0.352 + 0.001
0.363 - 0.001
0,1
0.142 0.000
0.000
0.142 0.000
192
1.187 + 0.000
0.186 + 0.000
0.186 + 0.000
0.186
+ 0.000
0. 186 + 0.000
0.186 = 0.001
336
0.238 0.000
0.238 1 0.000
0.238 4 0.000
0.236
0.000
0.236 0.001
0.235 0.001
720
0.310 U. 000
0.310 : 0.000
0.315 + 0.000
0.308 + 0.001
0.310 + 0.000
0.313 = 0.000
96
0,083 + 0.000
0,083 + 0.000
0.082 = 0.000
0.086 + 0.000
0.085 + 0.001
0.085 = 0.000
192
O. 0.172 0.001
0.172 + 0.000
0.172 + 0.000
0.176 + 0.001
0.176 + 0.000
0.176 - 0.000
336
0.324
+ 0.001
0.324 1 0.001
0.324 + 0.001
0.325 + 0.001
0.324 + 0.001
0.324 - 0.002
720
0.941 0.002
0.962 + 0.002
1.010 +. 0.006
0.932
0.009
0.954 0.008
1.002 + 0.009
96
0.134 + 0.000
0.133 = 0.600
0.134 + 0.000
0.133 1 11,000
0.134 4 0.000
0.134 = 0.001
0.1418
0.148 + 0.000
0.000
0.000
336
16.655222 + 0.000
0.166 + 0.000
0.167 + 0.000
0.165
+: 01.000
166 + 0.000
0.167 - 0.000
720
0.205
0.000
0.207 + 0.000
0.210 4 0.000
0.205 0.000
0.207
0.000
0,211 0.000
96
0.386
1 0.000
0.386 1 0.000
0.387 1 0.000
0.385 1 01.00
0.386 1 0.000
0.386 = 0:000
Trillie
192
0.397 0.000
0.397 1 0.000
0.399 1 0.000
0.397 0.000
0.397 1 0.000
0.398 0.000
336
0.411
0.000
0.412 : 0.000
0.416 + 0.000
0.411 : 0.000
0.413 4 0.000
0.416 - 0.001
730
0.450
1: 0.000
0.452 4: 0.000
0.457 1: 0.000
0.450 1 0.000
0.452 1: 0.000
0.457 :: 0.000
56

---

Published as a conference paper at ICLR 2026
E.8 ADDITIONAL EMPIRICAL RESULTS FOR RANK REDUCTION METHODS
Performing SVD Directly on W (DWRR) As mentioned in Section 4 it is also straightforward to directly perform rank reduction on the W matrix. In this section, we summarize the results of DWRR in Table The experimental setting of DWRR is exactly the same as RRR. We tune the rank on the validation set, select the top three ranks with the lowest validation MSE, and report the best test MSE achieved using these ranks. Despite the DWRR's ' strong empirical performance, we argue against its robustness and recommend readers with RRR. An example is shown Figure T6] with full set of results at the end of the Appendix. It's obvious that RRR's 's validation and test curve show much higher consistency.
Table 16: DWRR performance. In comparison to RRR shown in the main text, DWRR can frequently perform better, except for ETTh2, where DWRR shows degradation.
Datasel
ETTbl
ETTh2
ETTml
ETTm2
Weather
Exchange
ECL
Traflic
Horizon
96
0,365
0,270
0.306
0.162
0,140
0.084
0.133
0.385
192
0,399
0.331
0.332
0.215
0.182
0.173
0.148
0.396
336
0.426
0,355
0.365
0.268
0.232
0.323
0.164
0.410
720
0.427
0.384
0.414
0.349
0.304
0.911
0.203
0.448
RRR ETTN1_4720_0 Trade off
ARR ETTh2_A720_ 2 Rank Trade-oft
-0.77
2.00
+0.713
MODE
3.44
3.05
3.000
0,200
3.852
3.42
2,00
0.220
50
106
325
324
375
200
2.5
20
75
330
120
150
175
206
Bank
Bank of
(a) Rank-MSE Tradc-off on ETTh1
(b) Rank-MSE Trade-off on ETTh2
RRR BC1_1720_0111100 F192 RanieMSE Trade-off
RRR Traffic_ F192 Rank-MSE Trade-off
0.22
1e4 952
0.02
022
Vel 128
022
6050
3.2
333
0.2
+140
332
Ranc
(c) Rank-MSE Tra on ECL.
(d) Rank-MSF: Trade-off on Traffic
Figure 15: RRR Rank-MSE Trade-off 4 example datasets. (a) & (b): On smaller datasets, there are clear benefits in truncating the rank of W. (c) & (d): On larger datasets (Electricity, Traffic), the performance curve is robust the rank of W, and full-rank matrix does not cause significant performance degradation-a result of data scaling, where sufficient data helps suppress spurious roots. However, although the curves for these larger datasets appear flat at this scale, numerical analysis reveals that the optimal model is not achieved at full rank, indicating that slight rank truncation remains beneficial despite data scaling.
RRR & DWRR Rank-MSE Trade-off Curves While no retraining is needed for RRR and DWRR, both methods incur repetitive evaluation of the model performance the validation data in order to determine the optimal rank to use for W. For a better reading experience, we showcase some representative examples.
In Figure [5] it can be seen that for smaller datasets such as ETThl and ETTh2, it is usually beneficial to truncate the rank, suggesting the existence of spurious roots described in our theoretical analysis.
57

---

Published as a conference paper at ICLR 2026
RA ETTh2 L720 Rank NSE Trade
DWRR ETTh2 L720 F192 RankMSE Trado
0.200
33315
1228
0.005
6.3
Val MOE
422
0.02
028
01.00
25
220
2.
Rark ot
Rank of
(a) Rank MSE Trf of RRR
(b) Kank-MSE Trade-off of DWRR
Figure 16: RRR Rank-MSE Trade-off compared to DWRR on ETTh2. The validation trade-off curve of RRR highly matches the test trade-off curve. For DWRR, we found that there can be significant gaps for some datasets, and an example (ETTh2) is shown here. Overall, RRR tends to generate more consistent trade-off curves, suggesting better robustness than DWRR.
Conversely, for the larger and more complex Electricity and Traffic datasets, a full-rank W does not significantly degrade performance, which aligns with the data scaling effect in Proposition [ sufficient data suppresses spurious roots. That said, our experiments with RRR and DWRR reveal that even for the Electricity and Traffic datasets, the optimal model is not full rank; slight truncation remains beneficial despite data scaling. This indicates that a low-rank structure is relatively common in time series forecasting. Consequently, applying regularization by default can be generally beneficial practice for time series forecasting tasks.
In Figure T6 we observe that DWRR has less consistent rank-MSE trade-off curves for validation and test datasets. This might suggest DWRE aathhuhth showing better performance in many datasets and is inherently computationally cheaper- can be less robust than RRR.
We leave the full plot with all datasets and all forecasting horizons H = (96, 192, 336, 720} at the end of the Appendix, in Figures 19 20
22
23
24
25
and 26
E.9 RRR AND ROOT PURGE ON DIFFERENT LOOKBACK WINDOW/FORECCASTING HORIZON
The commonly used lookback windows of previous time series forecasting models of lengths 96, 192, 336, and 720. We perform a thorough study on the performance of RRR and Root Purge on all of these lookback windows and summarize the results in Table 18| Since we use Root Purge a frequency-doman section. Each experiment for Root Purge, following the setting of the main experiment, is repeated times. Overall, there is a clear trend that with a longer lookback window, the performance of both RRR and Root Purge improves. Still, the performance of RRR and Root Purge remains competitive even when the lookback window length is set to shorter values.
linear model in the main experiment, we continue to use these settings this
is important to notice that reducing the lookback window is a much more inefficient method in characteristic root selection than RRR and Root Purge. For instance, for ETThl, as shown in Appendix E.8 the optimal rank obtained by RRR is much less than 96, and decreasing the rank will result in improvements. However, if we use a lookback window of 96 directly to keep our root set small, the result is much worse.
In addition, we notice that previous works (Liu et al. 2023 Zhang et al., 2025 has also been using the forecasting horizon of 48 and 144. Thus, we summary RRR and Root Purge performances for these forecasting tasks in Table 17 for better comparing with methods using these settings.
Table 17: Performance of Root Purge and RRR models at forecasting horizon of 48 and 144, The lookback window is still set to 720, aligning with the setting in main experiments.
Dutsset
ETThl
ETT22
ETTm]
ET7m2
Weather
Eachangs
Traffic
ECL
Hunzon
46
144
48
144
46
144
48
144
46
144
48
144
48
144
4k
144
0332
0.379
0333
0.325
1.134
0.042
B1544
0.118
0.143
0.500
0.326
0.128
0,162
0.044
0.127
0.117
0.142
58

---

Published as a conference paper at ICLR 2026
Table 18:
Impact of lookback window length L on the performance of RRR and Root Purge. Given that the lookback window length directly impacts the number of characteristic roots of the corresponding linear difference equation, the lookback window alone can serve as strong regularization. Thus, we expect RRR and Root Purge to perform better with higher lookback windows, The error bars are omitted for RRR as it produces static results.
Dotnset
Horizon
Root Purge (Frequency Docain)
96
192
L= 336
[ 720
L. 96
L= 192
I 336
L. = 720
96
0,388
0.370
0.309
0.367
0.383
0.000
0.373 0.000
0.364 0.000
0.355 # 0,000
192
0.001
+: 0.000
0.394
336
0.474
0.448
0.424
0.430
0.473
0.000
0.446
0.000
0.421 0.000
0.423 + 0.000
720
0.461
0.433
0.418
0.425
0.461
0.003
0.433 + 0.000
0.421 #: 0.001
0.421 :: 0.000
96
0.280
0.282
0.272
0.268
0.293
0.001
0.288
0.000
0.277 0.000
0.268 :: 0.000
192
0.364
0.000
0.000
0.328
336
0.414
0.308
0.302
0.000 0.416
0,386
0.000
0.353 0.000
0.355 + 0.000
720
0.416
0.399
0.2844
0.376
0.001 0.419
0.403 + 0.000
0.387 +: 0.000
0,377 4: 0,000
96
0.351
0.308
0.301
0.306
0.359
0.000
0.311 0.000
0.300 + 0.000
0.306 + 0.000
192
0.000
0.346
0.336
0.333 0.000
236
0.422
0,380
0.370
0,385
0.000 0.428
0,381
0.370 0.000
0.380 0.000
720
0,483
0.439
0.426
0.414
0.487
0.000
0. 439
000
0.425 0.000
0.412 1. 0, 0,000
Exchange Weaher ETTm2
96
0.182
0.172
0.165
0.161
0.184
0.000
0.174
0.000
0.185 + 0.000
0.361 + 0,000
192
0.246
0,231
0.000
0.233 0.000
0.221 0.000
0.216 1: 0,000
0.307
0.000
0,289
720
0.407
0.382
0.367
0.348
0.408
0.000
0.
0000
0.368
0.000
0.350 +: 0.0
96
0.184
0.150
0.143
0.140
0.168
0.000
0.155 0.000
0.136 1: 0.000
0.142 1: 0,000
192
0.211
0,194
0.186
0.162
0.217 0.000
0. 0.000
0.191 1: 0.000
0.186 1: 0,000
336
0.287
0,246 0.000
0.236
720
0.348
0.328
0.316
0.304
0.353
0.000
0.334 0.000
0.321 +: 0.000
0.310 +
96
0.082
0.082
0.083
0.084
0.082 1 0.000
0.063 1 0.000
0.085 1: 0.000
0.082 1 0.000
192
0.169
0.170
0.171
0.174
0.170 0.000
0.173 0. 000
0.171 2: 0.000
0.1 4: 0.001
336
0.324
720
0.813
0.816
0.543
0.915
0.821
0.000
0.829
0.000
0.892 0.004
0.041 + 0.002
Average
0.350
0.330
0.321
0.321
0.353
0.334
0.325
0.322
E. SHORT-TERM FORECASTING-AN EXTREME CASE
The performance of linear models the M4 dataset is often limited (Xu et al. 2023 Liu et al. 2023 Wu et al., 2023). Our theoretical analysis in Section 3 and Appendix attributes this primarily to Two factors:
1. The restricted lookback window sizes, which range from only 12 to 96.
2. The requirement for single model to fit large number of time series from diverse domains, each with potentially different dynamics.
For example. fitting linear model to the M4 yearly data is equivalent to capturing all dynamics across 23,000 different series using only 12 characteristic roots. The severe constraint on the number of roots, combined with the diversity of dynamics to be captured, implies that linear model will significantly underfit the M4 dataset, explaining its subpar performance.
Root Purge and RRR leverage rank and nullity manipulation to eliminate spurious roots. However, when a linear model is already significantly underfitting, no spurious roots are present. Consequently, we expect little to no performance gain from applying Root Purge or RRR to linear models on M4. On the other hand, our approaches especially Root Purge do not over-regularize the model even in these extreme cases (as shown in Table 19). suggesting their potential as default regularizer for linear time series forecasting models.
E.11 HIGHER-ORDER RooT PURGE
The root purge we considered in the main text has the following target, where Gw(-) defines a linear transformation:
min W
Yıu - Sw Gw (I +A 1lGw P (Y - Gw Y )ns)))??
most-seeking
mot mat-purging
We now define the order of root purge as 1, forcing the noise to be contained the null-space of G.(.). To extend from order-1 Root Purge, it seems also reasonable to perform Root Purge on
59

---

Published as a conference paper at ICLR 2026
Table 19:
Forecasting performance on the M4 benchmark for linear models with and without Root Purge. The suboptimal performance of the baseline linear model is consistent with our theoretical observations Section 3 a consequence of limited capacity. Crucially, these results demonstrate the robustness of Root Purge. Even in these extreme cases of underfitting, the regularizer maintains stability and introduces negligible negative effects, affrrming its utility as a default regularization.
Frequency Domain Linear Model
Time Domain Lincar Model
Metric M4 Subset
Baseline
Root Pure
Baseline
Root Purge
Yearly
14.914
14.911
14.871
14,880
sMAPE
Quarterly
10.977
10.977
10,860
10,860
Monthly
13.402
13.404
13.354
13.351
Others
4.861
4,856
4,913
4.911
Average
12.742
12.741
12.682
12.683
Yearly
3.170
3.171
3.044
3.044
MASE
Quarterly
1.349
1.347
1.322
1.323
Monthly
1.022
1.022
1.010
1,010
Others
3.313
3.313
3.343
3.344
Average
1.709
1.709
1.670
1,670
Yearly
0.855
0.855
0.838
0.838
OWA
Quarterly
0.990
0.990
0.975
0.975
Monthly
0.945
0.945
0.938
0.938
Others
1.034
1.033
1.044
1.044
Average
0.916
0.916
0.904
0.904
higher-order composition Gw o P(-). J, as follows:
(Gw o P)* (+) = 9w o P(Gw o P(.... o P( P(-)))
I such composition
In this section, we investigate this general form of Root Purge and conclude that Root Purge of order I works best, As summarized in Table 20 higher-order Root Purge has attenuated effects on regularization, and the performance tends degrade as order increases. The order-1 Root Purge is able to attain the best result in almost all the benchmarks. To rule out run-to-run variance, we repeat each experiment 5 times and report the mean with standard deviations in our table.
Table 20: Root Purge results across different orders. The best results are highlighted in red.
Dataset
H
Order-I Ront Purge (Original)
Order-2 Ront Purge
Order:3 Root Purge
Order-4 Root Purge
+:
0.373
ETTbl
192
0.394 + 0.000
0.401 = 0.000
0.406 + 0.000
0.408 + 0.000
0.423 = 0.000
0.131 0.000
0.432 + 0.000
0.432 0.000
720
0.421 + 0.000
0.427 0.000
0.428 + 0.000
0.428
0.000
ETTh2
96
0.268 1: 0.000
0.270 1 0.000
0.271 1. 0.000
0.271 0.000
0.330 0.000
+
0.331 0.000
336
0.355 + 0.000
0.354 0.000
0.351 = 0.000
0.354 0.000
720
0.377 + 0.000
0.377 = 0.000
0.377 + 0.000
0.377 0.000
BTTm2 ETT
0.305 = 0.000
0.306 0.000
0.307 = 0.000
0.308 0.000
192
0.333 - 0.000
0.335 = 0.000
0.336 + 0.000
0.337 +: 0.000
720
0,412 + 0.000
0.414 000
0.415 + 0.000
0.415 0.000
96
0.161 + 0.000
0.161 = 0.000
0.3 + 0.000
0.163 1 0.000
0.216
0.000
0.217 0.000
336
0 0.269 + 0.000
0.269 = U.000
0.269 + 0.000
0.269 1: 0.000
0.350 0.000
0.350
Exchanga Weather
0.142 1: 0.000
0.143 0.000
0.143 1 0.000
0.144 1 0.000
192
0.186 - 0.000
0.187 = 0.000
088 + 0. 0.000
0.888 + 0.000
0,238
720
0.310 1 0.000
0.309 1: 0.000
0.309 1 0.000
0.309 1: 0.000
0.082 = 0.000
0.083 0.000
0.081 - 0.000
0.087 0.002
192
0.172 + 0.001
0.173 0.000
0.174 + 0.000
0.174 0.000
0.324 1 0.001
0.324 0.001
0.325 1 0.001
0.326 0.001
60

---

Published as a conference paper at ICLR 2026
E.12 ADDITIONAL RESULTS ON DATA SCALING & NOISE ROBUSTNESS ON SIMULATION DATA
We provide additional visualization for the experiment in Section 5.3 Data Scaling & Noise Robustness, where we examine the data scaling property and noise robustness of our methods on a toy example. In Figure 17a we show a qualitative result. When there is a scarce amount of data, the root-seeking target drives the linear model to overfit and mimic patterns of noise, which are, in fact, unpredictable. In Figure 17b, we show the noise robustness curve based on the raw MSE we obtained. It is still visible that both RRR and Root Purge closest to the ground truth. A clearer relative degradation plot in noise scaling is provided in Section 5
Qualitative Visualization Notse/Data
Noise Robustress SOTA Linea Modiel
10
Grann Tuch
Suruma
3.
30
Grand Uieer Rest Purge
Trdd thte Start Time Stepe
Oe
3.0
0.25
8:58
975 3.50
(a) Qualitative Visualization
(b) Noise Robustness (On MSE)
Figure 17: Additional visualization of the data scaling & noise robustness on simulation data. (a) A qualitative visualization of what went wrong for pure root seeking when data is scarce. (b) Test MSE against noise level. It is still visible that RRR and Root Purge show better noise robustness, but clearer degradation plot in noise scaling is provided in Figure
E.13 DETAILS OF ROOT ANALYSIS ON SIMULATION DATA AND QUANTITATIVE EVALUATION
As mentioned in the main text, a direct root comparison is infeasible with real-world data due to the lack of access to the underlying foeee-free dynamics and their true characteristic roots. We employ the same synthetic dynamics used previously to illustrate the data scaling and noise robustness properties of our methods.
(t) sin(2t) + cos 0.5t + -
The additive Gaussian noise term is given by a εt. In the noise-free case (a = 0), the signal can be represented exactly using six characteristic roots. We fit linear model with a lookback window and forecasting horizon each of length 25st limits the total number of learned roots and offers sufficient capacity capture the noise-free dynamics exactly. We then compare the characteristic roots estimated under four conditions against the ground truth roots.
Ground Truth (noise-free): A linear model is fitted to the clean signal via SVD, and its characteristic roots serve as the reference.
Standard Linear Model Noise:
A standard linear model fitted using Ordinary Least Squares (OLS) the noisy signal, and the root distribution from the first and the twelfth forecasting head studied.
2. Root Purge Noise:
Our Root Purge method is applied to the same noisy data, and the learned roots are examined.
3. Rank Reduction + Noise:
Similarly, our Rank Reduction Regression (RRR) approach is applied. The learned roots examined.
4. Standard Linear Model Gradient Method:
To control for the optimizer, we include a baseline where a standard linear model is optimized using the gradient method (matching the optimizer used in Root Purge).
61

---

Published as a conference paper at ICLR 2026
Complex Tor
AAA
622
420
Imaghary
Smrggh
(a) Root Distribution on the Ist forecasting horizon
(b) Root Distribution on 12th forecasting horizon
Figure 18: Root distribution on the (a) 1st and (b) 12th linear time series forecasting horizons, as defined by Section 3 Equation ( Both RRR and Root Purge yield root distributions visibly closer to the noise-free ground truth than the standard linear model. The arrows denote the ground truth roots toward which the respective methods should converge.
In section 5.3
we see that roots obtained via RRR and Root Purge are closer to the ground truth than those from the standard model. Here, qualitatively show the root recovery performance in Figure by visualizing the characteristic roots from these models in the two-dimensional complex plane. The roots obtained via RRR and Root Purge visibly closer the ground truth than those from the standard model.
Additionally, since Root Purge is optimized with gradient method, also introduced "Standard Linear Model (gradient method)" as a further baseline in Table to investigate the impact of using gradient based-optimizations.
Table 21:
Comparison of Root Distance to Ground Truth across models. The results demonstrate the effectiveness of Rank Reduction and Root Purge in recovering roots closer to the true values compared to standard linear models.
Model
Root Distance to Ground Truth (mean 1 std)
Rank Reduction
0.036 0.014
Root Purge
0.045 0.009
Standard Linear Model (OLS)
0.064 + 0.025
Standard Linear Model (gradient method)
0.151 + 0.015
Our analysis yields the following key observations:
Both RRR and Root Purge yield root distributions much closer to the noise-free ground truth than the standard linear model.
The root distributions from the standard model exhibit high variance across runs, whereas our proposed methods produce more stable and concentrated distributions.
Although the standard model optimized with the gradient method performs worse than its OLS counterpart, Root Purge successfully overcomes this optimizer disadvantage to achieve smaller root distance.
These results align with the results in Section 5.3 and demonstrate that our proposed techniques enhance the robustness of root learning in the presence of noise.
E. STOP-GRADIENT AND FULL-GRADIENT ROOT PURGE
In Appendix D.7 we analyzed the properties of the two variants of Root Purge from a theoretical perspective. Our results indicate that their theoretical behavior should be highly similar. In this
62

---

Published as a conference paper at ICLR 2026
section, we empirically compare the full- -ill-gradientt version with the stop-gra gradient version. The results, summarized in [able[22] for the ETT datasets, show that both variants yield very similar performance. Therefore, we primarily use the stop-gradien version in our experiments, it greatly simplifies gradient computation and is significantly less memory-intensive than the full-gradient version.
Table 22:
Experimental results comparing the stop-gradin noo-ggradeent and bull-gradient versions of Root Purge on ETT datasets with different À values. The stop-gratde graddeet variant produces nearly identical performance while being more efficient.
Stop Gradient Version
Full Gradient Version
Datssct
Horizon
A 0.125
A 0.25
A = 0.5
== 0.125
A 0.25
À 0.5
96
0.368
0.000
0.364 1: 0.000
0.300 +: 0.000
0.368
01.000
0.364 +: 0.000
0.364 :: 0.000
ET1m2 ET1ml BT112 B7Tb]
192
0.403 1: 0.000
0.399 1: 0.000
0.394 1: 0.000
0.403 1: 0.000
0.390 1: 0.000
0.394
1: 0.000
336
0.427 0.000
0.425 4: 0.000
0.423 4: 0.000
0.427 +: 0.000
0.425 +: 0.001
0.423 :: 0.000
720
0,423 0.000
0.421 + 0.000
0.423 + 0.000
0.423 +: 0.000
0.421 + 0.000
0.421 = 0.001
96
0.270 +: 0.000
0.269 4: 0,000
0.268 :: 0.000
0.270 +: 0.000
0.269 4: 0.000
0.266 :: 0.000
192
0.330 + 0.000
0.329 + 0.000
0.328 + 0.000
0.330 + 0.000
0.329 + 0.000
0.328 = 0.000
336
0,354 4: 0.000
0.355 4: 0.000
0.356 4: 0.000
0.354 1: 0.000
0.355 + 0.000
0.356 #: 0.001
0.377 0.000
0.377 + 0.000
0.000
0.000
0.377 0.000
0.000
96
0,306 1 0.000
0,306 1 0.000
0.305 1 0.000
0.306 1: 0.001
0.305 1 0.001
0.305 E 0.001
0,335
0.331 0.000
0.000
0.331
0.000
336
0. 363 + 0.000
0.361 + 0.000
0.360 + 0.000
0.362 + 0.001
0.361 + 0.000
0.360 = 0.000
720
0.413 0.000
0.412 + 0.000
0.413 + 0.000
0.413 0.000
0.412 + 0.000
0.412 0.000
96
0.1 1 0.000
0.161 1. 0.000
0.161 1. 0.000
0.161 1 0.000
0.161 1 0.000
0.161 = 0.001
192
0.217 0.000
0,217 + 0.000
0.216 + 0.000
0.216 0.000
0.216 + 0.000
0.216 + 0.000
336
0,269
0.000
0.269 + 0.000
0.269 + 0.000
0.269 + 0 0.000
0.269 + 0.001
0.269 = 0.000
730
0.350
+ 0.000
0.350 + 0.000
0.355 + 0.000
0.350 + 0.000
0.351 + 0.000
0.354 = 0.000
F DISCUSSION
F. LIMITATION AND FUTURE WORK
While our analysis provides valuable insights into linear models for time series forecasting, a key limitation is their inability to fully capture nonlinear dynamics present in many real-world signals. Although linear systems offer strong interpretability and robustness, their expressiveness inherently constrained, especially when the lookback window is limited to a small number (e.g. in short-term forecasting tasks such as M4; see Appendix I0 for details). To allow a linear model to capture complex dynamics, the lookback window must be sufficiently long. A promising direction for future work is to extend our framework toward nonlinear dynamics, investigating whether similar core principles-suce as dominant modes or root structures-can be identified and generalized. This could inspire the design of more interpretable and theoretically grounded nonlinear models. Additionally, our focus has been on forecasting tasks; it would be interesting to explore the implications of characteristic root structures in classification settings. In particular, understanding how random projections may act as a form of null space learning- both geometrically as classifiers provide new perspective on the design and
and algebraically as approximations of regularization of time-series classifiers.
F.2 BROADER IMPACT
The broader impact of this work lies in its potential to reshape how time series forecasting is approached in both research and applied settings. By grounding model design in classical linear systems Specry--pecffiaallly. through the lens of characteristic roots-this study offers a rigorous foundation for building more interpretable and robust forecasting models. The insights into noise sensitivity and data- sata-scaling inefficiency can inform the development of lightweight, resource- alternatives to overparameterized deep models, making high-quality forecasting more accessible in low-data or low-compute environments. Moreover, the proposed root-centric regularization techniques may be extended to other domains involving dynamical systems, such as control, finance, and climate modeling. Ultimately, this work invites the time series and broader AI communities to rethink the foundational principles guiding model design. Rather than relying on increasingly intricate architectures large-scale pretraining, we advocate for theory-driver approaches that focus on capturing the essential dynamics of time series data. Our study demonstrates that a deeper understanding of fundamental structures-such as characteristic roots-can lead to models that
63

---

Published as a conference paper at ICLR 2026
not only more interpretable and robust, but also more efficient and generalizable. By highlighting the power of simple, principled modeling grounded in classical system theory. we hope to shift the focus toward uncovering the core mechanisms that govern temporal behavior. This perspective encourages the community to pursue models that align with the inherent structure of time series, fostering progress through insight rather than complexity.
G DECLARATION
We used large language models (LLMs) to polish writing.
Full set of rank-MSE trade-off curves for RRR and DWRR in the following pages.
64

---

Published as a conference paper at ICLR 2026
www.
(a) RRR Rank-MSE Trade-off Curves
(b) DWRR Bank-MEE Trade-off Curves
Figure 19: Rank-MSE Trade-off Curves on ETTh1
(a) RRR Rank- MSE Tra Curves
(b) DWRR Rank-MSE Trade-off Curves
Figure 20: Rank-MSE Trade-off Curves on ETTh2
(a) RRR Rank-MSE Trade-off Curves
No9e mTT v1.77 LUTE
DWNS 1170
FTF 1775
(b) DWRR Rank-MSE Trade-off Curves
Figure 21: Rank-MSE Trade-off Curves on ETTml
65

---

Published as a conference paper at ICLR 2026
1PW FTT ST0 (30
1206
(a) RRR Rank-MSE Trade-off Curves
140
(b) DWRR Lank-MSE Trade-off Curves
Figure 22: Rank-MSE Trade-off Curves on ETTm2
0108
01004
20 15
(a) RRR CAn Tra Curves
LiVit
122
(b) DWRR Rank-MSE Trade-off Curves
Figure 23: Rank-MSE Trade-off Curves on Weather
44240
(a) RRR Rank-MSE Trade-off Curves
1509 fatuerr /7Th Turr-al
D1 100// fum Tude
mope Eushunge /20 yor Tute-
07077
(b) DWRR Rank-MSE Trade-off Curves
Figure 24: Rank-MSE Trade-off Curves on Exchange
66

---

Published as a conference paper at ICLR 2026
BOL,77L Trate
PATIOL 2002 PM2
(a) RRR Cank-MSE Trade-off Curves
(b) DWRR Rank-MSE MSE Trade-off Curves
Figure 25: Rank-MSE Trade-off Curves on Electricity. Although the curves for these larger datasets appear flat at this scale, numerical analysis reveals that the optimal model is not achieved at full rank, indicating that slight rank truncation remains beneficial despite data scaling.
Eam Tratre LTD P5S Triicett
Thrtt (739 Runk-ND To
AVA Porte UTE 7314 Kun A/Z Tramo
N. Trafti LTD8 Tedkoet
(a) RRR Rank-MSE Trade-off Curves
(b) DWRR Rank-MSE Trade-off Curves
Figure 26: Rank-MSE Trade-off Curves on Traffic. Although the curves for these larger datasets appear flat at this scale, numerical analysis reveals that the optimal model is not achieved at full rank, indicating that slight rank truncation remains beneficial despite data scaling.
67


Reviews:
[{"review_id": "nvnue0O5xW", "reviewer": "Reviewer_xcu7", "summary": "This paper analyzes linear time series forecasting via characteristic roots , identifying that noise creates \"spurious roots\" and a \"data scaling law\" that demands excessive data to mitigate this noise. To improve robustness and data efficiency, it proposes two structural regularization strategies: (1) post-hoc Rank Reduction (RRR) and (2) a novel \"Root Purge\" method. Root Purge is an adaptive training regularizer that encourages the model to learn a null space for the estimated noise (residuals).", "questions": "1. Does the Low-Rank Assumption limit the generality of method, making it effective only for a specific subset of (simpler) time series datasets? \n\n2.If we encounter a complex trend in the future that Instance Normalization can't flatten, does your Root Purge method just fail? How can you convince us that it's still robust in that scenario?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1.Originality and Clarity: The \"Root Purge\" loss function is a novel, elegant, and intuitive regularization technique.\n\n2.Quality and Significance: The paper provides a valuable insight into why simple linear models fail in noisy, low-data regimes.", "weaknesses": "1.Hyperparameter Sensitivity: The methods' performance relies heavily on tuning $\\rho$ or $\\lambda$. This data-specific tuning is difficult to estimate a priori and undermines the claim of robustness.\n\n2.Contingent on Low-Rank Assumption: The methods are motivated by an assumption that the true signal is low-rank. However, on large, complex datasets, the paper's results show the optimal rank is often full , and the proposed methods offer minimal benefit . This limits the methods' generality.\n\n3.Theory-Practice Gap (Trends): The theoretical analysis assumes homogeneous equations (constant/zero bias) . This is a poor fit for real-world data with non-stationary, dynamic trends.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9xePgM3DO9", "reviewer": "Reviewer_G42f", "summary": "This work presents a comprehensive study of linear models for time series forecasting, focusing on the role of characteristic roots in shaping model expressivity. Experiments validate our theoretical claims and demonstrate the effectiveness of both methods across a range of forecasting tasks.", "questions": "1. What is the transferability of the method?Can linear models be used in other areas than time series learning?\n\n2. Why is the frequency-domain linear layer necessary for Root-Purge instead of a mere implementation detail?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The research motivation of this manuscript is clear and its content is very rich。\n\n2. This manuscript is theoretical.\n\n3. Sufficient experiments have verified the effectiveness of the proposed method.", "weaknesses": "1. Theoretical analysis is an advantage of this work, but it can easily make it difficult for readers to understand. It is necessary to provide intuitive explanations in the key derivation process\n\n2. The manuscript lacks a comprehensive discussion of related work, which hampers readers, particularly those less familiar with the domain. Thus, it remains unclear how the proposed approach advances the state of the art or distinguishes itself from existing methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "luT2Azb5Mv", "reviewer": "Reviewer_zgN9", "summary": "This paper advances a linear time-series forecasting framework centered on characteristic roots: in the noise-free regime, it shows that characteristic roots govern long-term dynamics; in noisy settings, it exposes the MSE training data-scaling law and the risk of spurious roots, indicating that simply enlarging the sample size is insufficient for robust generalization. Accordingly, the authors introduce two complementary structural regularization strategies: (i) low-rank constraints on the parameter matrix via Reduced-Rank Regression (RRR) and Direct Weight Rank Reduction (DWRR) to recover low-dimensional latent dynamics; and (ii) Root Purge, an adaptive loss that learns a noise null space and performs online root restructuring, with training applicable in either the time or frequency domain. Evaluated on benchmarks including Traffic, Electricity, Weather, Exchange, and ETT, the methods deliver strong results.", "questions": "1. Under complex, non-stationary noise typical of real-world series, do the proposed methods retain theoretical advantages?  \n\n2. Can the approach be extended to more complex deep learning architectures?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. According to the reported results, the proposed methods achieve strong empirical performance across multiple datasets, which is particularly notable given the simplicity of the designs.  \n\n2. The paper provides substantial theoretical analysis with clear exposition to clarify the core concepts.  \n\n3. A clear roadmap is presented, and overall readability is high.", "weaknesses": "1. The core insight “preserving dominant characteristic roots to suppress noise” is not novel and aligns with fundamental conclusions from classical Singular Spectrum Analysis (SSA).  \n\n2. The technical contributions of DWRR and Root Purge appear simple.  \n\n3. The Root Purge objective may induce model degeneration: the model could prefer mapping any input to zero, and the paper offers no theoretical guarantees to prevent or mitigate this failure mode.  \n\n4. The theoretical analysis depends on i.i.d. Gaussian noise and stationarity assumptions to justify data efficiency and the regularizer; these assumptions often do not hold in practice, and no generalization beyond them is provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)