MixLinear: Extreme Low Resource Multivariate Time Series Forecasting with $0.1K$ Parameters
Authors: Aitian Ma
Dongsheng Luo
Mo Sha
Venue: ICLR 2026 Poster (2026)
Keywords: Long-term Time Series Forecasting
Segmentation
Adaptive Low-Rank Spectral Filtering
Primary area: learning on time series and dynamical systems

Abstract: Recently, there has been a growing interest in Long-term Time Series Forecasting (LTSF), which involves predicting long-term future values by analyzing a large amount of historical time-series data to identify patterns and trends. Significant challenges exist in LTSF due to its complex temporal dependencies and high computational demands. Although Transformer-based models offer high forecasting accuracy, they are often too compute-intensive to be deployed on devices with hardware constraints. 
In this paper, we propose MixLinear, which synergistically combines orthogonal segment-based trend extraction in the time domain with adaptive low-rank spectral filtering in the frequency domain. Our approach exploits the complementary structural sparsity of time series: local temporal patterns are efficiently captured through mathematically linear transformations that separate intra-segment and inter-segment correlations, while global trends are compressed into an ultra-low-dimensional frequency latent space through learnable rank-constrained filters. By reducing the parameter scale of a downsampled $n$-length input/output one-layer linear model from $O(n^2)$ to $O(n)$, MixLinear achieves efficient computation without sacrificing accuracy.
Extensive evaluations show that MixLinear achieves forecasting performance comparable to, or surpasses, state-of-the-art models with significantly fewer parameters ($0.1K$), which makes it well suited for deployment on devices with limited computational capacity.

Published as a conference paper at ICLR 2026
MIXLINEAR: EXTREME Low RESOURCE MULTIVARI- ATE TIME SERIES FORECASTING WITH 0.1K PARAME-
TERS
Aitian Ma, Dongsheng Luo, Mo Sha" Knight Foundation School of Computing and Information Sciences Florida International University {ama003, dluo, msha) ) @fiu.edu
ABSTRACT
Recently, there has been growing interest in Long-term Time Series Forecasting (LTSF), which involves predicting long-term future values by analyzing large amount of historical time-series data to identify patterns and trends. Significant challenges exist LTSF due to its complex temporal dependencies and high com- putational demands. Although Transformer baasoo models offer high forecasting accuracy, they are often too compute-intensive to be deployed on devices with hardware constraints. In this paper, propose MixL cally combines segment-based trend extraction in the time domain with adaptive low-rank spectral filtering the frequency domain. Our approach exploits the com- plementary structural sparsity of time series; local temporal patterns are efficiently captured through mathematically linear transformations that separate intra-segment and inter-segmen correlations, while global trends are compressed into an ultra- low-dimensional frequency latent space through learnable constronttrieedd filters. By reducing the parameter scale of a downsampled n-length input/output one- layer linear model from (n²) to ((n), Mixl_inear achieves efficient computation without sacrificing accuracy. Extensive evaluations show that MixLinear achieves forecasting performance comparable to existing models with significantly fewer parameters (O.1K), which makes it well-suited for deployment devices with limited computational capacity.
which synergisti-
INTRODUCTION
Deep learning models have recently achieved state-of-the-an accuracy on time series forecasting in various applications (Moon & Wettlaufer, 2017 Nunnari & Nunnari, [017 Sezer et al.) 2020
Ma et al.) 2025c). However, this success is marked by a trend of escalating computational cost and parameter counts, mirroring the trajectory of models in NLP and vision (Brown et al. 2020
Kaplan et al. 2020: Dosovitskiy et al. 2022]] Radford et al. 2021]. With leading models comprising millions parameters, their deployment on embedded devices, edge sensors, and other resource- constrained systems is often infeasible. We argue this parameter explosion is not an inevitable price for performance but symptom of a structural inefficiency in how current models represent time
series patterns (Zhou et al.) 2022b Wu et al. 2021 2023 Xu et al. 2024).
The core of this inefficiency lies monolithic representational strategy. Prevailing architectures attempt to capture both high-frequency local variations and low-frequency global patterns using the same set of mechanisms, despite their inherently different statistical properties. Local features (e.g.. short-term fluctuations) are best characterized by their temporal locality, while global structures (c.g.. long-term trends and scasonalities) are well-known to be sparse in the frequency domain (Zhou et al., 2022b Yi al. 2023). Forcing single, uniform architecture model these disparate characteristies is a mismatch that leads directly to parameter redundancy and computational waste.
Recent research has started to process local and global components differently. One approach, found in models like DeepGate (Park et al. 2022) decomposes the time series first. However, such method
Corresponding author.

---

Published as a conference paper at ICLR 2026
often still relies on complex, parameter- heavy modules for both components, failing to achieve the desired level efficiency. Another approach uses the frequency domain. Models like FITS (Xu et al. 2024) are highly efficient for global patterns. However, using global frequency components to model localized, transient variations inherently inefficient; it requires a disproportionate number of coefficients to capture sharp features that would be simple represent in the time domain. This diminishes the efficiency gains from spectral compression. Therefore, the central challenge is to design framework that can effectively model both global and local patterns while remaining computationally efficient.
Our work is motivated by a core insight about time series patterns: local trends can be efficiently captured through segment-based extraction in the time domain, while global trends exhibit sparsity in the frequency domain and benefit from adaptive low-rank filtering. This insight suggests that optimal efficiency requires not just separating patterns, but processing each in its most natural domain time for local trends and frequency for global trends. To realize this vision, we introduce MixLinear, a dual- pathway framework that systematically processes local trends through factorized linear decomposition in the time domain and global trends through adaptive low-rank spectral filtering in the frequency domain. MixLinear addresses three fundamental challenges: (1) developing parameter- efficceent segment-basee based extraction that reduces complexity from (n²) to O(n) (for an original series length L and downsampling factor л, = L//T) while preserving hierarchical temporal structures, (2) achieving extreme compression through rank-constrained spectral filtering that focuses on dominant frequency modes, and (3) integrating dual pathways through learnable upsampling without parameter explosion.
Our contributions are as follows:
We introduce an efficient approach for local trend processing that reduces complexity from O(n²) O(n) through strategic segmentation and dual linear transformations for segmeen and inter-segment dependencies.
We propose a complex-vail valued transformation approach with adaptive low-rank filtering that com- presses global trends into a minimal latent space, achieving unprecedented parameter efficiency while preserving essential spectral information.
We create novel synthesis that seamlessly combines time-domain and frequency-domain process- ing to achieve an unprecedented operating point on the efficiency-accuray curve.
Extensive experiments on benchmark datasets demonstrate that MixLinear achieves competitive forecasting performance with remarkable efficiency - orders of magnitude fewer parameters than existing lightweight models. Our work opens new possibilities for forecasting resource- constrained devices and provides a practical framework for efficient time series representation learning.
2 MIXLINEAR DESIGN
2.1 PRELIMINARY
LTSF involves predicting future values over an extended horizon using previously observed multi- variate time series data. It is formalized as f2+1:4H =f ( where Iz-L+1:; E R and f IR #CC In this formulation, denotes the length of the historical observation window, C represents the number of distinct features channels, and H denotes the length of the forecast hori- zon. Extending the forecast horizon H is crucial for LTSF allows more comprehensive long-term planning and decision making in practical applications. However, increasing the forecast horizon H usually requires more parameters and significantly increases the computational complexity of the forecasting model. Efficiently balancing extended forecast horizons with model complexity remains a key challenge in LTSF research, necessitating novel architectural designs and parameter-efficiee strategies maintain both accuracy and scalability.
2.2 FRAMEWORK OVERVIEW
We propose MixLinear, a dual-domain architecture that fundamentally reconceptualizes temporal pattern modeling through factorized representational learning. Our theoretical contribution rests the ppectral-Tempore Decomposition Principle: real-world time series exhibit intrinsic structural duality local dynamics manifest segment-wing correlations amenable factorized linear
2

---

Published as a conference paper at ICLR 2026
segment-based Trend Extraction
Xrrend E RR
Kseg ERV
X Intra Sog E RVM Xinter_Seg RR
Xyp Rym
EyER"
Aggregation & Downsampling
Linear Layer
Linean Layer
Upsampling
M
Intra-segment Correlation
Inter-segment Correlation
IFFT
XER Rd
X
75EC*
Xsp E CT
X Rm
XM ER YE R"
Adaptive Low Rank Filter
Adaptive Low Rank Filter-based Frequency Domain Trend Extraction
Figure 1: MixLinear Architecture Overview. Our dual-pathway framework efficiently processes time series data. The Segment-based pathway (top) downsamples input X E IR into segments Xeeg E R1/"" applies linear transformations for intra-segment (blue) and inter-segment (orange) correlations, then upsamples to Xr E R" The Frequency- domain pathway (bottom) transforms segments via FFT (Xs E C4/"), compresses trends through adaptive low-rank filtering to latent space Zs C"= reconstructs via iFFT, and outputs XF R Final predictions R combine both outputs, achieving competitive forecasting with only O.1K parameters.
decomposition, while global patterns demonstrate spectral concentration enabling extreme low-rank compression in the frequency domain (Zhou al. 2022b)
MixLinear operates through a sophisticated dual-pathway architecture that synergistically processes complementary temporal manifolds. Given input X R LLC. framework produces predictions through the learned compositional mapping:
Y Segment X: O,) Fineuencc (X; O)),
where Fsegment performs segment-based trend extraction with parameters O., and F Frequency performs adaptive low-rank spectral filtering with parameters θ, This additive composition preserves the independence of domain-spesn sspeifc representations while enabling joint optimization through backpropa- gation, contrasting with multiplicative fusion approaches that suffer from gradient instability (Vaswani al. 2017).
The architecture achieves parameter efficiency (0.1K parameters) through principled exploitation of inherent structural sparsity: factorized linear projections capture segment-wist correlations (Ki taev et al., 2020), while rank-constrained spectral filtering enables extreme compression of global modes (Tay et al. 2020). This decomposition fundamentally avoids the parameter explosion endemic to hybrid time-frequency models (Zhou et al. 2021 Liu et al. 2021)
2.3 SEGMENT- BASED TREND EXTRACTION
The segment-based pathway introduces Factorized Linear Decompoxition framework that disen- tangles intra-segment and inter-segment correlations through specialized linear projections. This approach transcends traditional segmentation methods (Zeng et al. (2023) by introducing transforma- tions that separate local shape modeling from global trend aggregation.
implement a principled temporal decomposition strategy through adaptive downsampling and non-overlapping segmentation. Given input X RxC we first apply downsampling by factor π to obtain X (L4/x)xC implementing implicit low-pass filtering that attenuates high-frequency noise while preserving trend information (Wu et al. 2021 Cleveland al.) 1990). The downsampled
sequence undergoes structured partitioning into M segments of length L/(x M):
X
(x(e)
As

---

Published as a conference paper at ICLR 2026
This decomposition enables parallel processing of local patterns while maintaining computational tractability, analogous to efficient attention mechanisms (Wang et al. 2020 Choromanski et al. 2021).
Within each segment x(*) E R we apply complementary linear transformations that disentangle
distinct correlation structures. For intra-segment modeling, we compute h h = Lin (x( E RdxC which compresses r temporal samples into a d-dimensional summary that captures short- range waveform information including local slopes, short periodicity, and morphological features through parameter-efficien linear mapping. For ieter-segment dependencies, we stack intra-segment
embeddings to form m iotra:.......................... h [M)) E RMxxxCC and apply a second transformation Himer Linearmmer (ma) R XxxX that models dependencies across segments, capturing slow drift, segment-level periodicity, and cross segment correlations. The separation between intra and inter projections ensures parameter efficiency and stable training (Tolstikhin al. 202I).
Following the factorized linear transformations, we implement reconstruction with upsampling: XT = Upsample(Reshapel m; H) E Hx The segment- based pathway requires dr + dMM d + M parameters, achieving linear complexity O(n) (n = L/x²) while preserving hierarchical temporal structures through factorized linear decomposition. This enables our model to achieve complexity reduction from On²²) to O(n) while maintaining expressiveness via two-layer linear networks over segmented inputs (Hornik et al. 1989).
2.4 ADAPTIVE LOW-RANK SPECTRAL FILTERING
The frequency domain pathway introduces an Adaptive Low-Rank Spectral Filtering framework that explicitly targets persistent spectral components through cunk-consrriined matrix factorization. This approach advances beyond conventional frequency domain methods (Oppenheim Schafer 1999) by learning adaptive spectral bases with extreme compression.
We first use FFT on the data to see spectral makeup (how much of each frequency contains). More specifically, we apply FFT to the down-sampled series to form the spectral tensor:
F FFT (Xdow)) E C(L/a)xC)
This approach enables parallel spectral processing while preserving temporal locality.
Traditional filtering can be complex and prone to overfitting (getting too specific to the training data and failing data). Instead of using enormous, full-size filter, low-rank factorization. Instead of learning computationally expensive full (L/T) x (L/T) complex filters prone to overfitting, we parameterize the frequency transform as rank-n, operator:
technique called
+ Φ = U(VF) ((L/T)xC
where U E ((L/a)xn, and V E CL,x(L/x) with n, < (L/x). This factorization projects each segment spectrum into shared low-dimensional latent space, then reconstructs filtered spectra through adaptive basis U. The rank constraint π1 2 enforces extreme compression that focuses representational capacity on dominant spectral modes, leveraging the low-rank structure of natural signals in the frequency domain (Donoho 2006 Halko et al. 2011.
From the rank-constrained spectral representation, we reconstruct temporal signals via:
X F = (Psamp
"xx
The frequency pathway requires only 4rn, real parameters, achieving extreme compression while preserving essential global spectral information through adaptive filtering (Howard et al. 2017 Sandler al. 2018
2.5
COMPLEXITY ANALYSIS
MixLinear introduces dual-domain decomposition that models data through the exploitation of complementary structural sparsity in time series data. Our comprehensive analysis demonstrates substantial improvements over existing approaches across multiple dimensions.
Time Complexity. Given an effective processing length n = L/x after downsampling. the segment- based pathway operates with O(n) complexity for orthogonal linear transformations. This represents
4

---

Published as a conference paper at ICLR 2026
exponential improvement over the O(L²) complexity of self-attention mechanisms Vaswani et al. 2017). The spectral pathway requires (n log n) operations for FFT transformations (Cooley & Tukey 1965) and (rn.) for constrandd filtering, which is significant reduction from the quadratic (r²²) complexity of full filtering approaches. The overall time complexity simplifies to O log n), dominated by the FFT operations in the spectral pathway.
Space Complexity, Memory requirements scale linearly as ((n), incorporating storage for downsam- pling operations, dual-pathway processing, and consttontt filtering components. This shows an important improvement over attention-based models that require (L)2) memory allocation (Tay et al. 2022]. enabling deployment on resource-constraine devices while processing longer sequences.
The linear scaling properties enable MixLinear to process sequences orders of magnitude longer than existing methods without proportional increases in computational overhead. The orthogonal temporal factorization and rank rank-constrained spectral filtering create an unprecedented efficiency efficiency-expressiii expressiveness trade-off that maintains state-of -otee-o---hhh-rr--e predictive capabilities even resource-constrainge environ-
ments (Chen & Ran 2019 Lim et al. 2021)
EXPERIMENT
In this section, we first outline our experimental setup. We then compare MixLinear with baseline models to evaluate its performance across eight LTSF benchmarks. Next, we assess the effectiveness of time domain segmentation and frequency domain filtering, Prediction visualizations (Appendix (D are provided in the Appendix.
3.1 EXPERIMENT SETUP
Datasets. We conduct experiments using eight benchmark LTSF datasets: ETTh1, ETTh2, ETTml, ETTm2, Exchange, Solar, Electricity, and Traffic. More details on this are provided in Appendix B.3
Baselines. We conduct comparative analysis of MixLincar against transformer based state-of-the the art baselines in the field TimesNet (Wu et al. (2022) and PatchTST (Nie et al. (203)). addition, we compare MixLinear against three linear based state-of-the- -hre--rrtt baselines models: DLinear (Zeng et al.
2023), FITS (Xu et al. 2024), and SparselTFF (Lin et al. 2024). More details on these baselines can
be found in Appendix B.2
Environment. MixLinear and our baselines are implemented using PyTorch (Paszke et al. 2019). All experiments are performed single NVIDIA A GPU with 80GB of memory. More details on our experimental setup are presented in Appendix B.
3.2 MAIN RESULTS
We evaluate the performance of MixLinear with eight benchmark LTSF datasets. We consider
look-back window of 720 and four forecast horizons: 96, 192, 336, and 720. Table provides a detailed comparison of MixLinear against five competitive baseline models, including SparseTSF FITS, DLinear, PStchTST, and TimesNet. The evaluation considers two key metrics: Multiply- Accumulate Operations (MACs) and Mean Square Error (MSE). MACs quantify the computational cost per prediction, which serves as an important measure on model efficiency, while MSE is metric quantify prediction accuracy. In addition, the Relative Percentage Difference (RPD) in MSE relative to Sparse'TSF offers insights into the comparative performance of the models. A positive RPD indicates superior performance over Sparse whereas a negative RPD implies reduced accuracy.
Small Parameter Size (81% % Parameter Reduction). As Table lists, MixLinear provides competi- tive forecasting accuracy while using significantly fewer parameters than our baselines. It contains only O.IK subsuttttiyssss smaller than SSF (1K) and FITS (10K). Figure 2 further highlights its parameter efficiency across different look- back windows on the Electricity dataset: MixLinear maintains near-linear growth in parameter count the forecast horizon increases, in con- trast to the much steeper scaling of SparseTSFF and FITS. Consistent with our O(n) complexoyyyy analysis, MixL iinear requires only 45-176 parameters across all configurations 1 11-81% % reduction
5

---

Published as a conference paper at ICLR 2026
Table Comparisons on MACs and MSE among various LTSF models when being applied in eight data sets. The best MACs are highlighted with bold fonts. RPD denotes the relative percentage difference in MSE compared to SparseTSF's MSE. A higher RPD indicates a larger improvement. RPD values are marked with: *** for RPD greater than 10%, ** for RPD between 3% and 10%," * for RPD between 0% and 3%, and no marking for negative RPD (%)
Madele
Spura77FF
Ancai
DLiserr
PuchTST
TimesNet
(2024)
(ours)
(3024)
(2025)
(2023)
(2025)
Horizon
MACx
1
MACx
RPD(R)
MACs 1 RPD(9))
M/CCs
RPD(%)
MACs
RFDXS)
MACs..
RFD(S)
96
0.362
3
-5.5%
0.98M
-11.0%
4.13G
4901,12G
192
166.32K
0.403
174 DSK
184.73K
-3.5%
195M
-7.0%
4.100
-10.0%
5481.17G
-2.5%
336
196.56K
0434
181,0K
5.39
214.16X
-0.5%
3.4M
-2.7%
4.21G
-1.4%
6357.34G
-13.2%
720
277:20K
0.426
196.56K
0.718
292.32K
-1.6%
7.28M
-18.3%
4,33G
-7.0%
S681114G
-22.5%
Traffic Fleenieity ETT12 E77ml ET7h2
192
96
146.16K
166.12K
0.294
0.339
167 86K
154
3,7%8
0.9%
165.44K
184.75K
7.5%
1.8%*
0.9MM
1.55
4.158"
-0.9%
4.13G
4,160
6.898
0.5%
42G
5481.17G
-15.6%
-18.6%
336
195.568
0.359
181,0K
2.3%
214.16K
2.3%6
3.4M
-15.3%
4.21G
2.2%
6337.24G
25. 35.9%
720
277 20K
0.383
196.56K
0.88
292,52K
-1.3%
7.28M
-50.3%
4,33G
-2.1%
8640.14
-20.69
96
146.16K
0.314
167 66K
1.9%
165.44K
2.95
0.98M
4,8%
4,13G
6.7%
4901.33G
-7.6%
192
166.32K
0.34
174 O5K
1.7%
184.73K
1.28
1.95
2.3%
4.16G
2.9%
5481.17G
-9.0%
336
196-56K
0.369
214.16K
0.515
3.4M
0.0%
4,21G
0.9%
6357.346
-11.1%
730
0.418
202.33K
0.0%
7.28M
-1.7%
4,330
8640/40
-14.4%
96
146,10K
66K
0.0%%
490
-13.3%
192
166.32
0218
-0.5%
184.73K
1.5M
-144%
4.16G
5881.17G
-14.2%
336
19636K
0:222
18L-10K
1.1%
164
1,1%
3.4M
20.7%
4.31G
-0.7%
6337.34G
-18.0%
720
277
0.350
196.6K
-0.3%
292.32K
0.9%
7.28M
-5.1%
4.33G
-3.4%
8640.14
16.6%%
96
6.7M
0.138
7.69M
0.0%%
7.4MM
5.1%
44.91M
-1.4%
189.39G
6.5%
4905.29G
21.7%
192
76621
01151
-2.0%5
8.4M
-5.3%
89.42M
-1.3%
190.77G
1,385
5485.72G
-21.9% 33..9%
336
901M
0.8
8.30M
-2.4%
9.2M
-5.4%
156 09M
-1.8%
193.0G
0.0%%
6342/60G
-19.3%
720
12.71M
0.205
9
9,01
2.0%
13.40M
-3.4%
333.75M
0.5%
198.68G
2.44
8647.11G
-7.3%
96
18,001
0.389
2005M
0.0%
20.37M
-2.3%
120.61M
-6.2%
508.50G
6.018
4902.216
-52.4%
192
20.4XM
0.388
-1.396
22 74M
-2.8%
230.04M
-6.3%
512:27G
2,5%
5482.17G
-55.0%
336
24,30M
0,11
2234
11.2%
26.37M
2.4%
418.95M
-6.3%
518.3G
3.2%
6338.19G
53.0%
34.14M
0.44W
24.38M
40.9%5
3600
-2.0%
89625M
533.45G
-42.9%
Solar Exchunge
96
16704K
189.06K
1.12M
17.15
17
1905.19G
190,8K
0.106
10.79
211.12K
-28.
6.6%
$485.72G
-15.3%
336
224.64K
0.358
206-98K
11.25
244.76X
7.0%
389M
-1.7%
4.51G
2..5%
730
31680K
0.084
234.64K
334.08K
1,4%
8.37M
.3.00
4.95C
-8.8%
8647.31G
-1.0%
96
2,86M
0:111
2.26M
0.0%
3:24M
3,7%
19.17M
46.3%
80.84G
24.7%
4906.51G
76.3%
192
3.26MI
0.225
2-1M
-0.9%
3.62M
4.0%
35.13M
-2.2%
81 355
-25.0%
5486.976
-76.4%
336
3.85
0.241
3.54M
0.45*
4.19M
3.7%
66.58M
-6.5%
82: 38G
-24 24.9%
6345.200
-74.3%
730
3.43M
0.241
3.55
0.4%
5.72M
-0,4%
142.44M
-8. 155
84.78G
-22.4%
8639. 88
-74.3%
compared to SPa and 94 98% reduction compared to FITS. At the longest setting (720
look-back/horizon), MixLinear only uses 176 parameters, whiles FITS uses 10,512 parameters.
10
10
10
10+
MixLinear TSF
MixLinear SparseTSF
MixLinear SparseT
MixLinear
SparseTSF
Parameters
10
FITS
Parameters
10a
FITS
Parameters
10a
FITS
Parameters
102
FITS
10a
10a
10a
102
10 96
192
336 Horizon
720
10a
96
192
336
720
10a
96
192
336 Horizon
720
10
96
192
336
720
Horizon
Horizon
(a) Look-back- = 96
(b) Look-back 192
(c) Look-back 336
(d) Look-back = 720
Figure 2: Parameter count comparisons across different look- back windows on Electricity dataset. MixLinear demonstrates consistently better parameter efficiency compared to Sparse'TSFF and FITS across all configurations.
High Efficiency. As Table I shows, MixLinear exhibits a significantly slower increase in MACs compared to FITS and Sparse as the forecast horizon expands. This characteristic highlights Mix- Linear's scalability advantages, making it particularly well-suited for more difficult and challenging long-horizon forecasting applications. In datasets with fewer channels flw-dimensional datasets), such as ETThl with a channel size 7, MixLinear achieves the smallest MAC at 196.56K under the forecast horizon 720, outperforming Spa at 277.20K (a 41.32% increase) and FITS at 292.32K (a 48.98% increase). For datasets with a large number of channels (high- thigh-dimensional datasets), such as Traffic with channel size of 862, MixLinear maintains its efficiency. For example, under the forecast horizon 720, it achieves the lowest MAC at 24.2M, compared to Sparse at 34. 14M (a 41.67% increase) and FITS at 36.00M (a 48.76% increase).

---

Published as a conference paper at ICLR 2026
0.4
0.4
0.2-
0.50
0.25
0.0
0.0
0.00
0.0
ETTh1
ETTh2
Exchange
Solar
0.4
0.2
0.6
0.3
0.4
0.2
0.2
0.1
0.1-
0.0
0.0
0.0
0.0
ETTm1
ETTm2
Electricity
Traffic
MixLinear
Z
SparseTSF
FITS
DLinear
PatchTST
TimesNet
Figure 3: Comparisons on MSE among various LTSF models forecast horizon 720.
State-of-the-art Forecasting Performance. Table demonstrates that MixLinear outperforms TSF achieving to a 16.2% improvement on the Exchange dataset, 5.3% improvement on ETThI, and a 3.7% improvement on ETTh2. As Figure shows. MixLinear consistently deliv- competitive, and in some instances superior, predictive performance across all eight datasets, demonstrating its effectiveness in capturing long-term dependencies. More experimental results with additional baselines provided in Appendix C.I
3.3 RUNTIME EFFICIENCY
To examine the runtime efficiency of MixLinear, measure the inference time (Tin).
Speedup in Low- Dimensional Scenarios (Up to 3.2x). Figure shows, MixLinear consistently outperforms baseline models in inference time. For example, MixLinear achieves inference time of 0.25ms on the Exchange dataset, significantly outperforming TSF (0.80ms) and FITS (0.43ms). These results represent inference speedups of 3.2x compared to SparseTSF and 1.72x compared to FITS, establishing Mixar as the most computationally efficient model in the comparison.
FITS
10.0
FITS
Time (ms)
SparseTSF
Time (ms)
SparseTSF
0.6
MixLinear
7.5
MixLinear
0.4
5.0
0.2
2.5
0.0
ETTh1
ETTh2
Exchange
0.0
Solar
Electricity
Traffic
Now-Dimensional scenarios
High- Righh-Dimensionl scenarios
Figure 4: Inference time among efficient LTSF models in low- and High-Dimensional scenarios.
Speedup in High--imensional Scenarios (Up to 2.58x). The efficiency advantage becomes even more pronounced in High-Dimensional scenarios. MixLinear achieves inference time of 2.05ms on the Electricity dataset, demonstrating notable improvements over Sparse (4.20ms) and FITS (4.77ms). These results represent inference speedups of 2.12x compared SparseTTSF and 2.58 compared to FITS.
3.4 ABLATION STUDY
To evaluate the contribution each pathway our dual-domain design, we conduct an ablation study by comparing MixLinear against two single-pathway variants: w/o Segment (removing the segment- based dual linear transformations) and w/o Filtering (removing the adaptive low-rank spectral filter). The w/o Segment variant captures only global spectral dynamics, while the who Filtering variant relies
7

---

Published as a conference paper at ICLR 2026
solely local temporal processing through segmentation. This setup highlights the complementary
roles of local and global trend extraction in achieving high forecasting accuracy.
Table 2: Performance for MixLinear and its
variants forecast horizon 720.
Model
ETThl
ETTh2
Exchunge
Solar
Electricity
Trallic
MACs
MSE
MACs
MSR
MACs
MSE
MACs
MSE
MACs
MSE
MACs
MSE
w/o Filtering
44K
0.425
14L
0.389
207.36K
0.954
3.55M
0,262
8.32M
0.245
22.34M
0.528
w/o Segment
141.2KK
0.474
141.12K
0.411
161.28K
0.949
2.76M
0.267
6.47M
0.245
17.38M
0.478
MixLincar
196,56K
0.423
196.5KK
0,380
224.64K
0.923
3.85M
0.240
9.01M
0.209
24.20M
0,452
As shown in Table 2. MixLinear consistently achieves lower MSE than both single-pathway variants while maintaining O (n log n) complexity. In low-dimessional datasets (ETThI/ETTh2) w/o Filtering outperforms w/o Segment, confirming that local dual linear transformations are more effective when inter-channel correlations are limited (e.g.. 0.425 vs. 0.474 MSE on ETTh1). MixLinear further improves performance (0.423 on ETThI, 0.380 on ETTh2) by integrating spectral modeling. In high- dimensional datasets (Electricity/Trrffii)) w/o Segment achieves lower error than w/o Filtering (e.g.. 0.478 vs. 0.528 MSE on Traffic), showing the benefit of low-rank spectral filtering in compressing global dynamics. Again, MixLinear delivers the best overall results (0.209 Electricity, 0.452 on Traffic) by unifying both local and global pathways. From an efficiency standpoint, MixLinear adds only marginal cost (224.64K vs. 207.36K MACs on Exchange), mainly from FFT/FFFT operations, yet still achieves superior accuracy-efficiency trade-offss with only ~0.1 parameters.
3.5
HYPERPARAMETER STUDY
230
0.45
0.45
MACs (K)
210
220
MACS
MSE
0.40
MSE
MACs (K)
210
220
MACS
MSE
0.40
MSE
0.35
200
200
190
0.30
0.30
6 Segment Length
10
10
Segment Length
(a) ETThl
(b) ETTh2
Figure 5: Impact of segment length on MACs and MSE of MixLinear at forecast horizon 720.
Impact of Segment Length. The sensitivity analysis of segment length demonstrates the robustness of our orthogonal linear projections across varying temporal granularities while maintaining compu- tational efficiency. As shown in Figure[5] both ETTh1 and ETTh2 exhibit stable MSE performance across segment lengths from 2 to 16, with ETTh1 maintaining MSE values in the 0.42-0.43 range and minimal variation. Computational cost decreases substantially as segment length increases, with ETThl showing a reduction from 290K to 220K MACs due to reduced inter-segment projection complexity. The computational cost reduction reflects the inverse relationship between segment count M and individual segment length r in our O ((r + +dM) parameter complexity, where longer segments require fewer inter-segment operations while maintaining sufficient intra-segment modeling capacity. This robustness validates our orthogonal factorization design, demonstrating that the separation of local shape modeling from global trend aggregation remains effective across different temporal scales without requiring careful hyperparameter tuning. More discussions are in Appendix
0.6
0.6
350
350
MACs (K)
MACS
MSE
-0.5
MACs (K)
MACS
MSE
0.5
300-
300
MSE
250
-0.4
0.4
200
200
16
24
0.3
8
24
0.3
Rank
Rank
(a) E dataset at forecast horizon 720
(b) ETTh2 dataset forecast horizon 720
Figure 6: Impact of spectral rank MACs and MSE of MixLinear at forecast horizon 720.
Impact of Spectral Rank. The spectral rank sensitivity analysis of our adaptive low-rank spectral filtering reveals that extremely low-rank approximations achieve near-optimal performance with
8

---

Published as a conference paper at ICLR 2026
minimal computational overhead. As demonstrated in Figure increasing n; from 2 to 24 on ETThl yields only marginal MSE improvement (approximately 0.005) while computational cost increases substantially from 275K to 350K MACs. ETTh2 exhibits similar performance saturation beyond nz = 4, providing empirical evidence for the spectral sparsity assumption underlying our frequency domain pathway. This confirms that global temporal patterns concentrate in low- dimensional spectral subspace corresponding to dominant periodicities. The rank-2 approximation achieves 6x parameter reduction compared n2 16 while maintaining comparable accuracy, enabling deployment in ssource-constrainel environments without sacrificing predictive capability, The gradual increase computational cost with rank reflects the linear scaling of our low-rank formulation (O(rn))) compared to quadratic complexity (O(r²) of full spectral filtering, validating the effectiveness of our rank-constrained parameterization for scalable spectral processing. More discussions are provided in Appendix
4 RELATED WORK
Long-term Time Series Forecasting. LTSF is challenging due to the complexity and high dimension-
ality of temporal data (Zheng et al. 2024 2023
Ma et al. [024] 2025b a Ma & Sha 2025 Ma et al.)
2025d). Traditional methods, such ARIMA Contreras et al. 2003) and Holt-Winters (Chatfield)
& Yar 1988), perform well in short-term settings but often fail at long horizons. Machine learn- ing models, including SVM (Wang & Hu 2003). Random Forests (Breiman 200T), and Gradient Boosting (Natekin& & Knoll 2013). improve forecasting accuracy via non-linear modeling but rely on manual feature engineering. Deep learning models, such as RNNs, LSTMs, GRUs, and Transformers, such as Informer and Autoformer, are good at capturing long-range dependencies. Hybrid approaches that combine statistical and neural models further improve forecast performance. Recently developed models, such FEDformer (Zhou et al., 2022b), FiLM (Zhou et al. 2022a), PatchTST (Nie et al. 2023), and Sparse (Lin et al. (002)) employ frequency-domain processing and efficient attention mechanisms to improve scalability and accuracy. Despite these advances, existing models largely overlook how to effectively and efficiently integrate time and frequency domain features- an area with significant potential to further advance the state of LTSF.
Time Series Decomposition. Time series often comprise trend, seasonal, and residual components. Classical decomposition techniques, such as STL (Cleveland et al., 1990), TBATS (De Livera jet al.] 2011). and trend filtering (Moghtaderi et al., 2011 Tibshirani (014)) are effective but limited by seasonal shifts, long periods, and sensitivity to noise (Gao et 2020). Alternatively, frequency-domain decomposition provides compact, expressive representations (Xu et al. 2020).
FEDformer (Zhou et al. 20226) and TimesNet (Wu al., 2022) extract frequency modes to capture periodic structure, while FITS (Xu et al. (2024) uses sinusoidal decomposition to preserve signal fidelity. However, extracting robust temporal features in the frequency domain remains challenging and requires task-specific neural designs. Moreover, little attention has been given to compressing features the frequency domain to enhance model efficiency, leaving a gap in the design of scalable and lightweight forecasting models in the frequency domain.
CONCLUSION
In this paper, we present MixLinear, dual-domain framework that achieves competitive long-term time series forecasting performance with only O.1K parameters. By processing local trends through segment-based extraction in the time domain and global trends through adaptive low-rank spectral filtering in the frequency domain, MixLinear exploits the complementary structural sparsity inherent in time series data, reducing complexity from O(n2) ((n) while maintaining comparable accuracy. Extensive experiments across eight benchmark datasets demonstrate to 16.2% improvement in forecasting accuracy and 3.2x speedup in inference time, with robust performance across both low-dimensional and high-dimensional scenarios. The significant parameter reduction enables new deployments of LTSF models on eesource-constran real-time forecasting in edge computing scenarios for many applications, such as flooding detection, environmental health monitoring, and traffic control, where traditional deep learning models are computationally prohibitive. Besides, the underlying idea is also applicable to the development of more efficient Large Language Models and foundation models. Our implementation of MixLinear can be found at Ma et al.]
devices and offers new opportunities for

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENT
This work was supported in part by the National Science Foundation under grants CNS-2150010, ECCS-2242700, and IIS-2529283.
REFERENCES
Leo Breiman. Random Forests. Machine Learning, 45:5-32, 2001.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems (NeurIPS), 2020.
Chris Chatfield and Mohammad Yar. Holt-Winters Forecasting: Some Practical Issues. Journal of the Royal Statistical Society Series D: The Statistician, 37(2):129-140, 1988.
Jiasi Chen and Xukan Ran. Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing. Proceedings of the IEEE, 107(8):1738-1777 1762, 2019.
Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, David Belanger, Lucy Colwell, and Adrian Weller. Rethinking Attention with Performers. In International Conference on Learning Representations (ICLR), 2021.
Robert B Cleveland, William S Cleveland, Jean E McRae, and Irma Terpenning. STL: A Seasonal- Trend Decomposition Procedure Based Loess. Journal of Official Statistics, 6(1 6(1):3-73, 1990.
Javier Contreras, Rosario Espinola, Francisco Nogales, and Antonio J Conejo. ARIMA Models to Predict Next-day Electricity Prices. IEEE Transactions on Power Systems, 1014):011 2003.
James Cooley and John W Tukey. An Algorithm for the Machine Calculation of Complex Fourier Series. Mathematics of Computation, 19900)2277331, 1965.
Alysha M De Livera, Rob J Hyndman, and Ralph D Snyder. Forecasting Time Series with Complex Seasonal Patterns Using Exponential Smoothing. Journal of the American Statistical Association, 1011456) 2011.
P Kingma Diederik. Adam: A Method for Stochastic Optimization. International Conference on Learning Representations (ICLR), 2015.
David L Donoho. Compressed Sensing. IEEE Transactions on Information Theory, 1289-163 2006.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations (ICIR), 2021.
Jingkun Gao, Xiaomin Song, Qingsong Wen, Pichao Wang, Liang Sun, and Huan Xu. Robustrad: Robust Time Series Anomaly Detection via Decomposition and Convolutional Neural Networks. In ACM SIGKDD Workshop on Mining and Learning from Time Series
2020.
Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions. SIAM Review, 53 (2):217-288, 2011.
Kurt Hornik, Maxwell Stincheombe, and Halbert White. Multilayer Feedforward Networks Universal Approximators. Neural Networks, 255):359-366, 1989.
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. In arXiv preprint arXiv: /00. 04861, 2017.
10

---

Published as a conference paper at ICLR 2026
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling Laws for Neural Language Models. arXiv preprint 20X1v:2001.08361, 2020.
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The Efficient Transformer. In International Conference Learning Representations (ICLR), 2020.
Bryan Lim, Sercan Ô Ank, Nicolas Loeff, and Tomas Pfister. Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting, International Journal of Forecasting, 37(4): 1748-1764, 2021.
Shengsheng Lin, Weiwei Lin, Wentai Wu, Haojun Chen, and Junjie Yang. SparseTSF: Modeling Long-term Time Series Forecasting with Ik Parameters. In International Conference on Machine Learning (ICML), 2024.
Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time Series Modeling and Forecasting with Sample Convolution and Interaction. Advances in Neural Information Processing Systems (NeurIPS), 2022.
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-Complexity Pyramidal Attention for Long-Range Time Series Modeling and Forecasting. In International Conference on Learning Representations (ICLR), 2021.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. In International Conference on Learning Representations (ICLR), 2023.
Aitian Ma and Mo Sha. WMN-CDA; Contrastive Domain Adaptation for Wireless Mesh Network Configuration. In ACM/SIGAPP Symposium on Applied Computing, 2025.
Aitian Ma, Dongsheng Luo, and Mo Sha. MixLinear Implementation. URL https::/githhn:
Aitian Ma, Jean C Tonday Rodriguez, and Mo Sha. Enabling Reliable Environmental Sensing with LoRa, Energy Harvesting, and Domain Adaptation. In IEEE International Conference on Computer Communications and Networks (ICCCN), 2024.
Aitian Ma, Jean Tonday Rodriguez, and Mo Sha. A Basa- Energy- Haregy-H Sensing System for Living Environment. In IEEE International Workshop Metrology for Living Environment (MetroLivEnv), 2025a.
Aitian Ma. Jean Tonday Rodriguez, Mo Sha, and Dongsheng Luo. A Feasibility Study on Using LoRa Link Characteristics to Predict Ambient Air Temperature. In IEEE International Conference Computer Communications (INFOCOM), 2025b.
Aitian Ma, Jean Tonday Rodriguez, Mo Sha, and Dongsheng Luo. Sensorless Air Temperature Sensing Using LoRa Link Characteristics. In IEEF International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-LOT)) 2025c.
Dongyi Ma, Aitian Ma, Sirui Luo, Martin de Jode, Andrew Hudson-Smith, and Mo Sha. LORADOC- TOR: LLM-Driven Diagnosis and Adaptive Policy Optimization for Reducing Packet Error Rate in LoRaWAN Networks. In IEEE International Conference on Machine Learning and Applications (ICMLA), 2025d.
Azadeh Moghtaderi, Pierre Borgnat, and Patrick Flandrin. Trend Filtering: Empirical Mode Decom- positions Versus 11 and Hodrick-Prescott. Advances in Adaptive Data Analysis, 301161 2011.
Woosok Moon and John S Wettlaufer. A Unified Nonlinear Stochastic Time Series Analysis for Climate Science. Scientific Reports, 711):44228, 2017.
Alexey Natekin and Alois Knoll. Gradient Boosting Machines, a Tutorial, Frontiers in Neurorobotics, 7:21, 2013.
11

---

Published as a conference paper at ICLR 2026
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A Time Series is Worth 64 Words: Long-Tene Term Forecasting with Transformers. In International Conference on Learning Representations (ICLR), 2023.
Giuseppe Nunnari and Valeria Nunnari. Forecasting Monthly Sales Retail Time Series: A Case Study. In IEEE Conference on Business Informatics (CBI), 2017.
Alan V Oppenheim and Ronald W Schafer. Discrete-Time Signal Processing. Prentice Hall, 1999.
Jinuk Park, Chanhee Park, Jonghwan Choi, and Sanghyun Park. DeepGate: Global-local Decomposi- tion for Multivariate Time Series Modeling. Information sciences, 590:158-178, 2022.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems (NeurIPS), 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision. International Confer- ence on Machine Learning (ICML), 2021.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mo- bileNetV2: Inverted Residuals and Linear Bottlenecks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Omer Berat Sezer, Mehmet Ugur Gudelek, and Ahmet Murat Ozbayoglu. Financial Time Series Forecasting with Deep Learning: A Systematic Literature Review: 2005-2019. Applied Soft Computing, 90:106181, 2020.
Yi Tay, Dara Bahri, Donald Metzler, Da-Cheng Juan, Zhe Zhao, and Che Zheng. Synthesizer: Rethinking Self-Attention for Transformer Models. In International Conference Machine Learning (ICML), 2020.
Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, and Donald Metzler. Efficient Transformers: Survey. ACM Computing Surveys, 55(6):1-28, 2022.
Ryan J Tibshirani. Adaptive Piecewise Polynomial Estimation via Trend Filtering. The Annals of Statistics, 42(1))285-3, 2014.
Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Un- terthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. MLP-Mixer: An AlI-MLP Architecture for Vision. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is All You Need. In Advances Neural Information Processing Systems (NeurIPS), 2017.
Haifeng Wang and Dejin Hu. Comparison of sym and Is-svm for regression. In International Conference on Neural Networks and Brain (ICNNB), 2005.
Sinong Wang, Belinda 7. Li, Madian Khabsa, Han Fang, and Hao Ma. Linformer: Self-Attention with Linear Complexity. In arXiv preprint arXiv:2006. 2020.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long, Autoformer: Decomposition Transform- ers with Auto-Correlation for Long-Tere Term Series Forecasting. Advances Neural Information Processing Systems (NeurIPS), 2021.
Haixu WU, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d Variation Modeling for General Time Series Analysis. International Conference on Learning Representations (ICLR), 2022.
12

---

Published as a conference paper at ICLR 2026
Haixu Wu, Jun Huang, Jichui Xu, Jianmin Wang. and Mingsheng Long. TimesNet: Temporal 2D- Variation Modeling for General Time Series Analysis. International Conference on Learning Representations (ICLR), 2023.
Kai Xu, Minghai Qin, Fei Sun, Yuhao Wang, Yen-Kuang Chen, and Fengbo Ren. Learning in the Frequency Domain. In JEEE/CVF Computer Vision and Puttern Recognition Conference (CVPR), 2020.
Zhijian Xu, Ailing Zeng, and Qiang Xu. FITS: Modeling Time Series with 10K Parameters. In International Conference Learning Representations (ICLR), 2024.
Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang. Pengyang Wang. Hui He, Defu Lian, Ning An, Longbing Cao, and Zhendong Niu. Frequency-Domain MLPs are More Effective Learners in Time Series Forecasting. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are Transformers Effective for Time Series Forecasting? In AAAI conference on artificial intelligence (AAAT), 2023.
Xu Zheng. Tianchun Wang, Wei Cheng, Aitian Ma, Haifeng Chen, Mo Sha, and Dongsheng Luo. Auto TCL. Automated Time Series Contrastive Learning with Adaptive Augmentations. In International Joint Conference Artificial Intelligence (IJCAI), 2023.
Xu Zheng. Tianchun Wang, Wei Cheng, Aitian Ma, Haifeng Chen, Mo Sha, and Dongsheng Luo. Parametric Augmentation for Time Series Contrastive Learning. In International Conference on Learning Representations (ICLR), 2024.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. In AAAI Conference on Artificial Intelligence (AAAI), 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, Rong Jin, et al. Film: Frequency improved legendre memory model for long-term time series forecasting. In Advances in Neural Information Processing Systems (NeurIPS), 2022a.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Fre- quency Enhanced Decomposed Transformer for Long-Term Series Forecasting. In International Conference on Machine Learning (ICML), 2022b.
13

---

Published as a conference paper at ICLR 2026
A IMPLEMENTATION DETAILS
A.1 TRAINING THE DUAL-DOMAIN TREND EXTRACTOR
MixLinear employs dual-domain architecture that processes temporal patterns through both segment- based and eeqquency-baseer pathways. We formulate the forecasting function fa) R + R where represents the learnable parameters, L is the lookback window length, and is the forecast horizon. For multivariate time series forecasting, MixLinear adopts Obannel-Indepenee strategy where multiple channels are modeled using shared parameters to enhance generalization and reduce computational overhead.
The regment-based pathway captures local temporal dependencies through orthogonal linear transfor- mations applied to reshaped trend sequences. Given downsampled trend sequence Zrrend E 12", reshape it into a square matrix Xseg E IR VVviv where á = [v 2 The dual linear transformation is formulated as:
X
W (W1 (Wex
where Wi, W22 Rvax Iearnable projection matrices that capture intra-segment and inter- segment correlations respectively.
The frequency-basee based pathway leverages spectral sparsity through adaptive low-rank filtering in the frequency domain. After applying FFT to the trend sequence, we compress the spectral representation using a low-rank approximation:
Zs Wen: (PF(FFT(FInea))))
Xire iFFee Zs):
(8)
where Wene E R xr and Weee Rrxa, are encoding and decoding matrices, n, is the latent dimension, and LPF denotes low-pass filtering.
A.2 TRAINING ALGORITHM
The MixLinear training process integrates dual-domain processing with efficient downsampling and upsampling operations. Algorithm describes the complete workflow for single-step inference.
Algorithm 1: MixLinear Training Algorithm
Input:
Historical window 2t-L+1a E R_, downsampling period w, forecast horizon H
Output: Forecasted output 1441:4 R
Preprocessing and Downsampling:
Tmean E Z4
// Compute temporal mean
Tnorm -- Tt- L+1.t - Tmean
// Gero-mean mean normalization // Temporal aggregation
Tagg ConCColldda Cdonvld(nnomm) w) + Tnorm
+ [L/w]. â
[/n
// Compute dimensions // Extract trend sequence
Turend & Reshape( Tagg w));
Segment-based Pathway:
X < Reshape(Fuunds (Vn, n)):
// Square matrix formation // Intra-segment transformation
X Kemp + W1 Wxxse
Xom WT Temp
XT
// Inter-segment transformation // Temporal output
Iy Reshape(X,, Xonts m) where HH/w];
Frequency-baser Pathway:
.ss FFF(I
// Frequency domain transformation
J.S LPFF Neutor))
// Low-pass filtering
29 Wem
// Latent space encoding
Trezon <- Wdecis;
// Spectral reconstzuction // Time domain conversion
IF +- iFFT (2recon)
Combination and Output:
TM ØT + IF + Zmean ;
// Dual-domal fusion
ht 1--++# + Upsample(zM. H):
// Final forecast
14

---

Published as a conference paper at ICLR 2026
The training objective minimizes the Mean Squared Error (MSE) between predicted and actual values:
L =
aiti - i(ti)² H
During training. we optimize all parameters = ( 22. Wene Wdec) simultaneously using Adam optimizer with learning rate 0.02. The model converges within 30 epochs due the limited parameter space and efficient gradient flow through linear transformations.
B EXPERIMENTAL SETTINGS
B.1 HYPERPARAMETERS
We implement MixLinear with PyTorch (Paszke et al. 2019) and optimize hyperparameters to ensure fair comparison with baseline methods. Due MixLinear's minimal design complexity, hyperparam- eter tuning is straightforward and requires limited search space. We provide all hyperparameters and their configurations as follows:
Optimizer: Adam optimizer (Diederik 2015) with learning rate set to 0.02 for all datasets.
The relatively large learning rate accelerates training convergence given the small parameter count in MixLinear. Default decay rates are set to (0.9, 0.999).
Training configuration: Training conducted for 30 epochs with early stopping based on validation loss with patience of 10 epochs. This prevents overfitting while ensuring sufficient training iterations.
Batch size: Batch sizes determined by dataset channel dimensions to maximize GPU parallelism while preventing out-of-memory issues. Specifically, batch size is set to 256 for datasets with fewer than 100 channels (e.g., ETThI, ETTh2, Exchange) and 128 for datasets with fewer than 300 channels (e.g., Electricity, Traffic).
Segment-based pathway hyperparameters: Segment length Lseg is tuned in {2, 4, 6, 10, 12, 16) with optimal values typically ranging from 4 to across datasets. The dual linear transformation dimensions and M are set to and respectively based on sensitivity analysis.
Frequency- prruency-based based pathway hyperparameters: Frequency latent dimension n, searched in (2, 4, 6, 8, 12, 16, 24) with n. 2 achieving optimal accuracy-efficiene trade-offs across most datasets. Rank parameter r is set to 8 for all experiments.
Input configuration: Following FITS baseline setup, input length is uniformly set 720 for all models to ensure fair comparison. Forecast horizons are set to (96, 192, 336, 720) as standard evaluation metrics.
Dataset splitting and normalization: We follow the procedures outlined in FITS and Autoformer (Wu et al.) 202I) for dataset splitting. ETT datasets are divided into training, validation, and test sets with 6:2:2 ratio, while other datasets use a 7:1:2 ratio. Both our model and baselines use StandardScaler normalization to ensure consistent preprocessing. The baseline results reported come from the first version of the FITS paper for direct comparison.
B.2 BASELINE SETTINGS
As shown in Table 3 we summarize the baseline models used in this study, covering a diverse set of architectures, including Transformer-bases CNN-based, MLP-based, and trequency-domalll models. Each of these models has been proposed to address different challenges in long-term time series forecasting (LTSF) by leveraging unique architectural designs, such as dccomposition techniques, convolutional feature extraction, frequency-domain transformations, and lightweight linear modeling.
FEDformer. FEDformer (Zhou et al. 2022b) is a Transformer-batt based model that introduces a seasonal-trend decomposition mechanism and exploits the sparsity of time series in the frequency do- main. By leveraging Fourier transform properties, it selectively learns relevant frequency components, improving efficiency in LTSF tasks.
15

---

Published as a conference paper at ICLR 2026
Table 3: Summary of baseline models used in our experiments.
Model
Year
Description
Code
FEDlormer
2022
Translormer- model that incorporales seasonal -rrendd decom-
https: /
position and exploits the sparsity time series in the frequency domain.
com/
TCM 2M22 20022220ooo
TimesNet
2023
bused model with Block lask- menk-gerrral backbone. It transforms ID time series into 2D lensors to capture intraperiod and interperiod variations.
https: https://1tthbb.oo
tuuml/ /iimi//iimeetttttt
SCINet
2022
Recursive downsample-on convolve--nneeeeeeeoeeee architecture that applics multiple convolutional filters to extract distinct yet valuable tem- pural features from downsampled sub-sequences.
cuze cure-lab/5C1m /0..ab/
Transformer
2022
Transformer-hawe architecture that applies attention and feed-
https://github
forward networks inverted dimensions.
Tuml//rransforme
PatchTST
2022
Transfommer-bsee model uttliiigg patching and channel-
independent (CI) techniques. It also enables effective pre-truining and transfer learning across datasets.
(aginie3//PatchTS)
DLincar
2023
MLP-bbaed model with a single lincar layer that outperforms
kt hctps://githa
Transformer- models in long-term time series forccasting
com/ wwre-uueeeee lab/
(LTSF) tasks.
LTSF -Lnear
FITS
2023
Linear model processes series through interpolation
https://q1this
the complex frequency domain.
VEMOXIC/2I13
TSI
2024
Extremely lighteiigtt model designed for LTSF, addressing the challenges of modeling complex temporal dependencies over extended horizons with minimal computational resources.
httpa://9ithab
1ss 1138/Sparse1SF
TimesNet (Liu et al. 2023) is a CNN-based model that introduces the TimesBlock, a task-general backbone designed to process temporal patterns effectively. It converts ID time series into 2D tensors, allowing for better feature extraction both intraperiod and interperiod variations through convolutional layers.
TimesNet.
SCINet. SCINet (Liu et al. 2022 is recursive downsample-convole--nneee architecture de- signed to capture multi-scale temporal dependencies. It applies multiple convolutional filters to downsampled uub-sequences, enhancing feature extraction across different resolutions while main- taining computational efficiency.
iTranssfrmer. iTransformer (Wu et al., 2022) a Transformer-based model that applies self- attention mechanisms and feed-forward networks inverted dimensions. By reordering dimensions before applying traditional Transformer operations, iTransformer enhances dependency modeling across time series data.
PatchTST. PatchTST (Ne et al. 2023) is Transformer-based model that utilizes patching tech- niques and a channel-independen (CI) approach. This model segments time series into patches, allowing the self-attenti mechanism to focus localized patterns, leading improved learning efficiency. Additionally, it supports pre-training and transfer learning across different datasets.
DLinear. DLinear (Zeng et al. 2023) is an MLP-based model with a single linear layer, demon- strating that Transformer-based models are not always necessary for LTSF tasks. By reducing architectural complexity while maintaining strong predictive performance, DLinear outperforms many Transformer-based approaches in efficiency and generalization.
FITS. FITS (Xu et al. 2024) is frequency-domain model that manipulates time series data through interpolation in the complex frequency domain. By transforming signals into frequency space, FITS enables smooth interpolation and efficient learning, making it well-suited for capturing periodic and long-range dependencies.
Sparse SpF (Lin et al. 2024) is a novel, extremely lightweight model designed specifi- cally for LTSF tasks. It addresses the challenge of modeling long-term dependencies while main- taining minimal computational cost. SparseTSF achieves this by strategically reducing redundant computations and focusing essential temporal patterns.
16

---

Published as a conference paper at ICLR 2026
B.3 DATASET DESCRIPTION
This section provides a detailed description of the datasets used in our experiments. Table4 marizes the eight benchmark datasets, including the number of channels, sampling rate, and total timesteps available for each dataset.
Table Benchmark dataset description.
Dataset
Traffic
Electricity
Solar
Exchange
ETThl
ETTh2
ETTml
ETTm2
Channels
862
321
137
8
7
7
7
Sampling Rate
hour
hour
10 min
day
hour
hour
15 min
15 min
Total Timesteps
17,544
26,304
52,560
7,588
17,420
17,420
69,680
69,680
Traffic Datase
The Traffic dataset contains hourly road occupancy rates recorded by 862 sensors on major freeways in the San Francisco Bay Area. Provided by the California Department of Transportation, it consists of 17,544 total timesteps and captures key traffic patterns such as rush-hour congestion, seasonal variations, and long-term trends in freeway usage.
Electricity Datase The Electricity dataset comprises hourly electricity consumption measurements from 321 residential and commercial customers in Portugal, covering the period from 2012 to 2014. It consists of 26,304 total timesteps and is useful for analyzing demand patterns, seasonal variations, and peak load forecasting.
Solar Datase The Solar-Energy dataset records solar power generation from 137 photovoltaic (PV) power plants located in Alabama. This dataset contains 52,560 total timesteps sampled at 10-minute resolution throughout 2016, offering detailed insights into solar energy production dynamics, including seasonal trends, weather influences, and variations in power output.
Exchange Datase The Exchange-Rate dataset records the daily exchange rates of eight major foreign currencies, including those of Australia, the United Kingdom, Canada, Switzerland, China, Japan, New Zealand, and Singapore, spanning from 1990 2016. With total of 7,588 timesteps, this dataset captures global economic trends, market fluctuations, and currency volatility, making it valuable for financial time series forecasting.
ETT Datase The ETTh1, ETTh2, ETTml, and ETTm2 datasets, originally introduced in In- former (Zhou et al. (02T), contain industrial load and oil temperature readings collected from an electrical transformer station. These datasets span from July 2016 July 2018 and differ in their temporal resolutions. ETThl and ETTh2 are sampled at 1-hour intervals, each containing 17,420 total timesteps across 7 channels. In contrast, ETTml I and ETTm2 significantly increasing the total number of timesteps to 69,680 while maintaining the same number of channels. These datasets widely used for evaluating long- tengttemm forecasting performance, capturing both short-term fluctuations and long-term trends in electricity consumption and environmental conditions.
sampled at 15-minute intervals,
C FULL EXPERIMENT RESULTS
In this section, we compare Mixl inear with more baseline models, examine the effectiveness of time domain segmentation, and frequency domain filtering.
C.1 MAIN RESULTS
We evaluate the performance of MixLinear across eight benchmark LTSF datasets, including ETTh1, ETTh2, Exchange, Solar, ETTml, ETTm2, Electricity, and Traffic. Our experiments use a look-back window of 720 and four forecast horizons: 96, 192, 336, and 720. Table provides comprehensive
http: / o
17

---

Published as a conference paper at ICLR 2026
Table MSE results of multivariate long-term time series forecasting comparing MixLinear against baselines.
Models
MixLinear
SSarseTSF
FITS
DLinear
PatcTTST
iTransformer
SCINet
TimesNet
FEDromer
Data
Horizon
(ours)
(2024)
(2024)
(2023)
(2023)
(2023)
(2022)
(2022)
(2022)
P2rameter(720)
0.176K
0.925K
41.76K
485.3K
6.31M
7.04M
23.62M
301.7M
17.98M
0.351
0.362
0.384
0.385
0.386
192
0.395
0.403
0.417
0.443
0.413
0.441
0.429
0.436
0.427
336
0.411
0.434
0.436
0.446
0.440
0.487
0.504
0.491
0.459
720
0.423
0.426
0.433
0.504
0.456
0.503
0.544
0.521
0.484
96
0.283
0.294
0.272
0.282
0.274
0.297
0.289
0.340
0,340
Exchange E1
192
0.336
0.339
0.333
0.340
0.338
0,380
0.372
0.402
0.433
336
0.355
0.359
0.355
0.414
0,367
0.428
0.365
0.452
0.508
720
0,380
0.383
0.378
0.588
0.391
0.427
0.475
0.462
0,480
0,088
0,105
0.086
0.087
0.087
0.086
0.267
0.107
0,148
192
0.175
0.196
0,180
0.251
0.183
0.177
0,351
0.226
0,271
336
0,318
0.358
0.333
0.403
0.390
0.331
0.424
0.367
0,460
720
0.923
0.954
1.364
1.038
0.970
1.058
0.964
1.195
0.211
0.211
0.195
0.290
0,266
0.203
0.237
0.373
0.286
Weather BTTm2 ETTml Solar
192
0.227
0.225
0.216
0.320
0.288
0,233
0.280
0.397
0.291
336
0.240
0.241
0.232
0.353
0.301
0.248
0.304
0.420
0,354
720
0.240
0.241
0.242
0.357
0.295
0,249
0.308
0.420
0,380
96
0,308
0.314
0.305
0.299
0.293
0.334
0.418
0.338
0.379
192
0.337
0,343
0.339
0.335
0.333
0.377
0,439
0.374
0.426
336
0.365
0.369
0.367
0.369
0,369
0.429
0.490
0.410
0.445
0.415
0,418
0,418
0.425
0.416
0.491
0.595
0,478
0.165
0.165
0.164
0.167
166
0.180
0,286
0.187
0.203
192
0.219
0.218
0.217
0.221
0,223
0,250
0.399
0.249
0,269
0.269
0.274
0.311
720
0.351
0.350
0.347
0.368
0,362
0.412
0.960
0,408
0.421
96
0.170
0.172
0.145
0.176
0,149
0.174
0.221
0.172
0,217
0.212
0.215
0.218
0.194
0.221
336
0.257
0.260
0.236
0,262
0.245
0.278
0.309
0.280
0.339
720
0.321
0.318
0.308
0.323
0.314
0.358
0.377
0.365
0,403
Electricity
0.138
0.138
0.145
0.140
0.129
0.148
0.168
192
0.154
0.151
0.159
0.153
0,149
0.162
0.175
0,184
0,197
336
0.170
0.166
0.175
0.169
0.166
0.178
0.189
0.198
0.212
720
0,209
0.205
0,212
0.204
0,210
0.225
0,231
0.220
0.244
96
0.389
0.389
0.398
0.413
0,366
0.395
0.613
0.593
0,573
Traffic
0,403
0.398
0.409
0.423
0.388
0.417
0.535
0.617
0.611
336
0.416
0.411
0.421
0.437
0.398
0.433
0.540
0.629
0.621
720
0.452
0.448
0.457
0.466
0.457
0.467
0.620
0.640
0.630
comparison between MixLinear and leading baseline models, such as SparseTSF, FITS, DLinear, PatchTST, iTransformer, SCINet, TimesNet, and FEDformer.
Compact Model with 81 % Parameter Reduction. As shown in Tad Table MixLinear achieves exceptional parameter efficiency, requiring only 0.1K parameters, significantly fewer than SparseTSF (1K) and FITS (10K). This results in an 81 % reduction in parameter size at a forecast horizon of 720 compared to SparseTTSF, the second-best linear-based LTSF model. Figure 7 further illustrates the stark contrast in parameter sizes among models, where TimesNet, PStchTST, DLinear, and FITS exhibit considerably larger parameter footprints, while Sparse represents moderate reduction. By achieving the smallest parameter size among all compared models, MixLinear drastically minimizes computational overhead and accelerates inference speed, making it highly suitable for resource-coneen conotrceconntrrineed environments.
State-of-the-Arr Forecasting Performance. Table highhlghts MixLinear's superior forecasting capabilities across diverse datasets and forecasting horizons. In www-Dimensional scenarios, such as ETTh1 and ETTh2 (each with 7 channels), MixLinear consistently outperforms baseline models, achieving the lowest MSE across multiple horizons. Notably, on ETTh1 at horizon 336, Mixennnear reduces MSE by 5.3% (+0.023) compared to the best baseline, demonstrating its effectiveness in capturing essential time-series patterns in data with limited variates. In High- Hiihh-Dimnnsional scenarios, such as Electricity (321 channels) and Traffic (862 channels), MixLinear maintains strong perfor- Even when compared to lightweight
mance, ranking among the top two models across most models like DLinear, FITS, and SparseTSFF MixLinear remains highly competitive. Moreover, despite its significantly smaller parameter size (0. 1K), MixLinear performs on par with parameter-heavy
18

---

Published as a conference paper at ICLR 2026
10f
FEDfarmer
10t
TimesNet
Number Parameters
104
SCINet
ITanstomme
Patch
DLinear
10
FITS
10
SporseTSF
MixLinear
10
Figure 7: Comparisons on number of parameters among various LTSF models at forecast horizon 720. The Y-axis uses logarithmic scale.
architectures, such PatchTST (6M parameters), highlighting its efficiency in learning complex temporal dependencies without the need for excessive computational resources. At an extended forecast horizon of 720, MixLinear further demonstrates its robustness in long-term forecasting, ranking within the top two models across all datasets except Electricity. On ETTh1, MixLinear reduces MSE by 0.003 at this horizon, while maintaining MSE increase less than 0.005 on other datasets. These results underscore MixLinear's 's ability to effectively model long-range dependencies while maintaining exceptional efficiency both computation and parameter utilization.
C.2 IMPACT OF SEGMENT LENGTH
The sensitivity analysis of segment length demonstrates the robustness of our segment segment-based trend extraction across varying temporal granularities while maintaining computational efficiency. As shown in Figures 8 and the experimental results reveal distinct behavioral patterns between Low- Dimensional and High- Dimennsoonii datasets, providing insights into the fundamental relationship between temporal decomposition and forecasting performance.
For Low-Dimensional datasets (Figure 8 ETThl and ETTh2 exhibit optimal performance with moderate segment lengths (4-8 time points), maintaining MSE valuesaround 0.38-0.42 with controlled variation across different segmentation strategies. The ETTh1 dataset demonstrates remarkable stability, with MSE fluctuating minimally between 1.38-0.40 across both forecast horizons, while computational cost (MACs) decreases substantially from approximately 300K to 180K segment length increases from 2 to 16. This inverse relationship reflects the reduced inter-segment projection complexity inherent in our temporal decomposition framework. Conversely, the Exchange dataset exhibits different sensitivity patterns, showing optimal performance at longer segment lengths (8-16) 16), particularly evident in the 720-horizon forecasting where MSE stabilizes around 0.9-
Now-Dimensional Dataset Analysis:
High--imensionnl Dataset Robustness: High- High--Dimessionl datasets (Figure 9 demonstrate superior stability across varying segment lengths, validating the hypothesis that cross-channel information compensates for suboptimal temporal segmentation choices. The Electricity dataset maintains consistent MSE performance across all tested segment configurations while achieving significant computational savings, with MACs reducing from 28K to 8K. Most remarkably, the Traffic dataset exhibits exceptional robustness, showing virtually flat MSE curves across all segment lengths for both forecast horizons, with computational costs decreasing linearly from 45K to 15K MACs. The Solar dataset presents an intermediate behavior, with slight performance variations but overall stable trends, particularly at longer forecast horizons.
The computational cost analysis reveals a consistent inverse relationship between segment length and MACs across all datasets. This efficiency gain is particularly pronounced in High-Dimensional datasets, where the Traffic dataset achieves 3x computational reduction while maintaining performance stability. The computational savings validate our design philosophy of separating local temporal pattern extraction from global trend aggregation.
Computational Efficiency Patterns:
Forecast Horizon Sensitivity: Comparing short-term (336) and long-term (720) forecasting reveals interesting temporal dynamics.
datasets show increased sensitivity to segment length at longer horizons, suggesting that extended predictions require more careful temporal decom- position strategies. High- Dimensional datasets maintain their robustness across forecast horizons, with Traffic and Solar showing minimal performance degradation 720-step predictions.
19

---

Published as a conference paper at ICLR 2026
0.6
195-
MACs
MSE
MACs (K
0.5
220
190
MSE
0.44
MACs
210
185-
0.35
0.3
200
180
2
10
0.2
190
10
0.30
Segment Length
Segment Length
(a) Dataset ETThl the forecast horizon 336
(b) Dataset FTTh1 the forecast horizon 720
0.6
230
0.45
MACs (K)
195
MACS
MSE
0.5
220
MACs
MSE
0.40
190-
MACs
210
185
0.35
180
2
10
0.2
190
8
10
0.30
Segment Length
Segment Length
(c) Dataset ETTh2 at the forecast horizon 336
(d) Dataset ETTh2 at the forecast horizon 720
0.6
1.2
240
MACs
MSE
260
MACS
MSE
MACs (K)
0.5
MACs 8
1.1
250
0.4
M5
1.00/
240
210
0.3
230
0.9
220
10
10
Segment Length
Segment Length
(e) Dataset Exchange at the forecast horizon 336 (f) Dataset Exchange at the forecast horizon 720
Figure 8: Performance of MixLinear with different segment lengths on low-dimensional datasets.
Universal Performance Improvements: Across all tested configurations, the segmented approach consistently outperforms baseline methods, with improvement margins varying systematically by dataset characteristics. The convergence patterns observed indicate natural temporal scales that, once captured, provide stable forecasting benefits regardless of further segmentation refinement. This validates our segmentation- based design, demonstrating that the orthogonal separation of temporal components remains effective across diverse datasets and forecast horizons without requiring extensive hyperparameter tuning.
C.3 IMPACT OF SPECTRAL RANK
The spectral rank sensitivity analysis provides compelling evidence for the effectiveness of our adaptive low-rank spectral filtering, demonstrating that extremely low-rank approximations achieve near-optimal performance with minimal computational overhead. As illustrated in Figures 10 and [T] the relationship between spectral rank and forecasting performance reveals distinct patterns across dataset categories, validating our theoretical premise that global temporal patterns concentrate within low-dimensional spectral subspaces.
Low-Dimensional Dataset Analysis: For Now-Dimensional datasets (Figure 10p, both ETTh and ETTh2 exhibit remarkable performance stability across varying spectral ranks, with MSE remaining virtually constant around 0.38-0.42 despite rank variations from 2 to 24. This stability demonstrates that dominant periodicities in these datasets can be effectively captured with minimal spectral dimensions. Computational complexity increases linearly from approximately 200K to 350K MACs rank increases, reflecting low-rank formulation's O(n,z) scaling, Notably, the Exchange dataset shows slightly different behavior with modest MSE fluctuations, but maintains overall stability, suggesting that currency exchange patterns also concentrate in low-dimensional frequency subspaces.
20

---

Published as a conference paper at ICLR 2026
9.2
MACs
MSE
10.5
MACs
MSE
MACs (K)
9.0-
MACs (K)
MSC
10.0
MSE
0.22
8.6
9.5
8.4
10
0.1
2
10
0.1
Segment Length
Segment Length
(a) Dataset Electricity the forecast horizon 336 (b) Dataset Electricity the forecast horizon 720
0.6
28-
0.6
MACS
MSE
MACS
MSE
0.55
0.5
23.5
MACs
MACs
0.4
23.0
22.5
0.2
0.2
2
6
10
2
10
Segment Length
Segment Length
(c) Dataset Traffic at the forecast horizon 336
(d) Dataset Traffic at the forecast horizon 720
0.4
0.4
MACs
MSE
4.4
MACs
MSE
MACs (K)
MACs (K)
0.3
0.3
3.7
MSE
4.2
MSE
3.6
4.0
10
10
Segment Length
Segment Length
(e) Dataset Solar the forecast horizon 336
(f) Dataset Solar the forecast horizon 720
Figure 9: Performance of MixLinear with different segment lengths on high-dimensional datasets.
High-Dimensional Dataset Robustness: High- Dimensional datasets (Figure IT] demonstrate excep- tional robustness to rank variations, with Traffic and Solar datasets maintaining virtually flat MSE curves across tested ranks for both forecast horizons. The Electricity dataset exhibits minimal performance variations (MSE x 0.2) while computational costs scale linearly from 8K to 17K MACs. This robustness suggests that the cross-channel information in multivariate time series provides natu- ral regularization against rank selection, enabling stable performance even with extremely low-rank approximations.
Computational Efficiency Validation: The consistent linear scaling of MACs across all datasets confirms the effectiveness of our rank-constrained parameterization compared to quadratic (1²2) complexity of full spectral filtering methods. Remarkably, rank-2 approximations achieve 6 parameter reduction compared to rank-24 while maintaining comparable accuracy across all tested configurations. This efficiency gain is particularly pronounced in High- Sige--iinnsional datasets where the Traffic dataset maintains optimal performance with minimal computational overhead.
Forecast Horizon Consistency: Comparing performance across different forecast horizons (336 vs. 720) reveals that spectral rank sensitivity remains consistent regardless of prediction length. This temporal invariance suggests that the oow-dimesiional spectral structure captures fundamental periodicities that persist across different forecasting contexts, validating our frequency domain processing approach for both short-term and long-term predictions.
Spectral Sparsity Confirmation: The universal performance saturation observed beyond rank-4 to rank-8 across all datasets provides strong empirical validation for the spectral sparsity hypothesis underlying our frequency domain pathway. This confirms that time series data naturally exhibit low-rank spectral structure, where a small number of dominant frequency components capture the majority of temporal dynamics. The minimal performance degradation at extremely low ranks (rank-2) demonstrates the effectiveness of our adaptive spectral filtering in identifying and preserving the most informative frequency components while discarding redundant spectral information.
21

---

Published as a conference paper at ICLR 2026
300
0.6
0.6
350-
MACs (M)
0.5
MACs
MSE
MACs
MSE
250
0.44
MACs
300
MSE
250
0..4
0.3
200
200
8 Rank
16
24
0.2
8 Rank
16
24
0.3
(a) Dataset ETThl the forecast horizon 336
(b) Dataset ETThi the forecast horizon 720
300
0.6
0.6
MACs (M)
0.5
350
MACS
MSE
MACs
-00-
MSE
0.5
250-
MACs
300
250
0.4
200
200
8
16
24
0.2
8 Rank
16
24
0.3
Rank
(c) Dataset ETTh2 at the forecast horizon 336
(d) Dataset ETTh2 at the forecast horizon 720
400
0.4
1.2
MACs (M)
350
MACs
MSE
0.3
MACs (M)
400
MACs
MSE
1.1
300
0.2
MSE
1.00/
250
0.1
300
0.9
8 Rank
24
24
Rank
(e) Dataset Exchange at the forecast horizon 336 (f) Dataset Exchange at the forecast horizon 720
Figure 10: Performance of MixLinear with different spectral ranks low-dimensional datasets.
C.4 EXPERIMENTAL RESULTS EFFICIENCY
We evaluate the computational efficiency of the models by measuring MACs under different horizons. As Figure shows, MixLinear consistently demonstrates the lowest computational overhead and the most stable growth rate under all tested scenarios. This best scalability is particularly evident in high- dimensional, long-hone monghhorioon tasks (H = 336 and H = 720). In these regimes, MixLinear achieves up to 40% reduction in MACs compared to SparseTSF. The more flat curve confirms MixLinear's time complexity of (n fn log n.). This contrasts sharply with the significantly steeper growth rates observed for Sparse FSF and FITS. In low-dimensional datasets with few channels (e.g., ETTh1 with 7 variables), MixLinear attains the lowest MAC cost-196. 1986-696.56K at horizon out-ourpeooonning TSF (277.20K. +41.32%) and FITS (292.32K, +48.98%) This efficiency persists in high- dimensional settings: on the Traffic dataset with 862 channels, MixLinear again achieves the smallest MAC 24.2M for horizon 720, compared 34.14M for Sparse (+41.67%% and 36.00M for FITS (+48.7698) Overall, the experimental results unequivocally confirm that MixLinear substantially reduces computational overhead across various datasets and forecasting horizons while maintaining competitive predictive performance.
C.5 EFFECT OF DOWNSAMPLING FACTOR ON MIXLINEAR PERFORMANCE
To evaluate the effect of the downsampling factor π model performance, we perform ablation studies on both low-dimensional (ETTh2) and high-dimensional (Solar) datasets. Table X demon- strates that MixLinearr achieves optimal performance at æ 24 across most configurations, while maintaining robust forecasting accuracy across a wide range of downsampling factors. The results reveal several key insights: First, minimal downsampling (ı 2) does not yield the best performance, suggesting that moderate temporal aggregation helps the model capture more robust patterns by reducing noise and focusing on essential trends. Second, the optimal value 24 strikes effective balance between computational efficiency and information preservation, achieving the lowest MSE values of 0.282, 0.336, 0.356, and 0.380 for ETTh2 across horizons 96, 192, 336, and 720 respectively.
22

---

Published as a conference paper at ICLR 2026
0.4
MACs (M)
MACs
MSE
0.3
17.5
MACs
MSE
0.3
0.2
MACs
0.2
12.5
0.1
0.1
10.0
24
0.0
16
24
0.0
Rank
Rank
(a) Dataset Electricity the forecast horizon 336 Dataset Electricity the forecast horizon 720
0.6
0.6
45-
MACs
MSE
00.5
MACs (M)
MACS
MSE
0.55
MACs
30-
30
0.2
0.2
16
24
16
24
Rank
Rank
(c) Dataset Traffic at the forecast horizon 336
(d) Dataset Traffic at the forecast horizon 720
0.4
0.4
MACs (M)
MACs
MSE
0.3
MACs (M)
MACs
0.3
0.2
M5
0.2
9.
0.1
0.1
16
16
24
Rank
Rank
(e) Dataset Solar the forecast horizon 336
(f) Dataset Solar the forecast horizon 720
Figure I: Performance of MixLinear with different spectral ranks on high-dimensional datasets.
280
MixLinear
325
5.5
MixLinear
35.0
260
300
32.5
240
SparseTSF
2 275
SparseTSF
Sparse SSF
30.0
Sparse ISF
220
250
MACs
4.5
27.5
200
4.0
180
200
22.5
160
20.0
175
3.0
140
336
192
720
17.5
336
720
Horizon
Horizon
Horizon
Horizon
(a) ETT (Low Channel)
(b) Excahnge (Low Chan- (c) Solar (High Channel) (d) Traffic (High Chan- nel)
nel)
Figure 12: Comparisons on MACs across different forecast horizons.
Third, excessive downsampling (т 36) leads to performance degradation, indicating information loss when compression becomes too aggressive. Notably, the performance variations across different и values remain relatively small (typically within 2-3% MSE difference), demonstrating MixLinear's inherent robustness the choice of downsampling factor. This stability across different downsam- pling rates makes Mixlar adaptable to various computational budgets and deployment scenarios without requiring extensive hyperparameter tuning.
23

---

Published as a conference paper at ICLR 2026
Table 6: MSEs MixLinear with different downsampling factor и ETTh2 and Solar datasets.
Bold values indicate best performance for each horizon.
Dataset
Horizon
16
24
96
0.287
0.286
0.284
0.289
0.282
0.305
ET
192
0.351
0.358
0.341
0.360
0.336
0.348
336
0.366
0.364
0.365
0.364
0.356
0.370
720
0.390
0.383
0.383
0.389
0.380
0.391
96
0.207
0.205
0.215
0.209
0.211
0.218
Solar
192
0.231
0.233
0.238
0.238
0.227
0.230
336
0.250
0.254
0.258
0.251
0.240
0.242
720
0.252
0.250
0.259
0.252
0.240
0.243
D VISUALIZATION
To showcase the prediction performance of MixLinear and compare with other models, we present visualizations of their prediction results. Figures 13 and Figures 14 display the prediction results the Exchange dataset for different models under two settings: 1mput-72-ppreicee 1etttt-2---redit--11 192 (Figure T4). In these figures, the blue lines represent the ground truth values, while the orange lines denote the model predictions.
(Figure [3) and
3.5
3.5
3.0
3.0
2.5
2.5
Ground Truth Prediction
Ground Truth Prediction
2.0
Prediction Start
Prediction Start
200
400
600
800
200
400
600
800
(a) MixLinear
(b) SSa
3.0
3.0
2.5
2.5
Ground Truth Prediction
Ground Truth
Prediction
2.0
2.0
Prediction Start
Prediction Start
200
400
600
800
0
200
400
600
800
(c) FITS
(d) DLinear
3.5
3.5
3.0
3.0
2.5
2.5
Ground Truth Prediction
Ground Truth
Prediction Prediction Start
2.0
2.0
Prediction Start
0
200
400
600
800
200
400
600
800
(e) PStchTST
(f) iTransformerr
Figure 13: Prediction cases from Exchange by different models under the input- 720- 9720-predict-96 settings. Blue lines the ground truths, and orange lines are the model predictions.
24

---

Published as a conference paper at ICLR 2026
3.5
2.5
2.5
Ground Truth Prediction
Ground Truth Prediction
2.0
2.0
Prediction Start
Prediction Start
200
400
600
800
200
400
600
800
(a) Mixl.inear
(h) Sparse
3.5
3.5
3.0
3.0
2.5
2.5
Ground Truth Prediction
Ground Truth
Prediction
2.0
2.0
Prediction Start
Prediction
200
400
600
800
0
200
400
600
800
(c) FITS
(d) DLinear
3.5
3.5
3.0
3.0
2.5
Ground Truth
Ground Truth Prediction Prediction Start
Prediction
Prediction Start
200
400
600
800
200
400
600
800
(e) PatchTST
(I) iTransformer
Figure 14: Prediction cases from Exchange by different models under the moput-720-predict- settings. Blue lines are the ground truths and orange lines are the model predictions.
In Figure 13
the models predict 96 future time steps based on 720 past time steps. MixLinear shows strong alignment with the ground truth, capturing short-term patterns effectively. Sparse performs reasonably well, though slight deviations are observed. FITS and PatchTST closely follow the ground truth, demonstrating robust short-term forecasting capabilities. DLinear and ITransformer, however, exhibit larger deviations, indicating less accuracy for short-term predictions.
In Figure 14] the models are tasked with predicting 192 future time steps using 720 past time steps. Mnxar continues to perform accurately with minimal deviations, proving its effectiveness for longer prediction horizons. SparseTSFF and FITS display moderate accuracy but show occasional mismatches in trends. PatchTST maintains strong performance, similar to the 96-step setting, while DLinear and iTransformer show greater discrepancies and instability, struggling with the extended horizon.
Overall, these figures highlight the strengths and weaknesses of the models. MixLinear and PatchTST consistently deliver accurate predictions across both settings, whereas DLinear and iTTrnsformee face challenges in capturing longer-term temporal patterns. This comparison underscores the importance of robust model design for both short-term and long-term forecasting.
E
USE OF LARGE LANGUAGE MODELS
ChatGPT was used as a general-purpose purpose writing assistance tool to improve the grammar and clarity of writing during the preparation of this paper. L.LSs did not contribute the formulation of the
25

---

Published as a conference paper at ICLR 2026
research idea, the experimental design, the data analysis, or the formulation of scientific conclusions.
The authors assume full responsibility for all contents presented in this paper.
26


Reviews:
[{"review_id": "lFFHSVc4qr", "reviewer": "Reviewer_v3WY", "summary": "MixLinear proposes an extreme low-parameter (0.1K) dual-domain framework for multivariate time series forecasting. It processes local trends via segment-based linear decomposition in the time domain (O(n) complexity) and global patterns via adaptive low-rank spectral filtering in the frequency domain. Experiments show competitive accuracy with 3.2× inference speedup and 16.2% error reduction versus baselines while enabling deployment on resource-constrained devices.", "questions": "- Why does the frequency pathway use per-segment FFT instead of global FFT given the focus on global trends?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Achieves SOTA-comparable results with only 0.1K parameters.\n- Linear memory complexity (O(n)) enables edge/IoT deployment.", "weaknesses": "- Only MACs are compared, authors should provide the comparison of number of parameters.\n- While authors claim that the number of parameter is reduced down to $0.1k$, the number of MACs of the proposed MixLinear is with similar scale compared to SparseTSF and FITS.\n- The core idea of this paper is inherited from FITS. Good but not that impressive as FITS and SparseTSF.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2Ly07AplLB", "reviewer": "Reviewer_Saee", "summary": "To bridge the gap in the design of scalable and lightweight forecasting models in the frequency domain, this work present MixLinear, a dual-domain framework that achieves competitive long-term time series forecasting performance with only 0.1K parameters. The extreme parameter reduction enables deployment on resource-constrained devices, opening new possibilities for real-time forecasting applications in IoT environments and edge computing.", "questions": "See Weaknesses", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The research direction of this work is very interesting and has practical significance.\n\n2. This work is a highly innovative work. A large amount of evidence shows that it is different from the existing works.\n\n3. The experiments in this work are thorough.", "weaknesses": "The research presented in this manuscript is highly intriguing, and I have personally found it to be quite rewarding. However, as I am not an expert in this particular domain, I would like to raise two minor questions:\n\n1. While I appreciate the overall design of this work, the main text reads more like a technical report than a research paper, it lacks analysis or critical discussion to guide the reader toward a deeper understanding.\n\n2. Can more interpretable evidence be derived from lightweight research to bolster its applicability across diverse domains?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nkwU5MggvO", "reviewer": "Reviewer_GAKK", "summary": "This paper introduces MixLinear, a novel and extremely lightweight model for long-term time series forecasting (LTSF). The core idea is to tackle the \"parameter explosion\" problem in current deep learning models by adopting a dual-domain architecture that processes time series patterns in their most natural domains. Specifically, MixLinear uses a segment-based pathway with factorized linear transformations in the time domain to capture local temporal patterns, and a frequency domain pathway that uses an adaptive low-rank spectral filter to compress and model global trends. The authors claim this hybrid approach reduces the parameter scale of a linear model from O(n^2) to O(n). The main contribution is a model that achieves forecasting performance competitive with or even surpassing some state-of-the-art methods, while using only ~0.1K parameters, making it exceptionally well-suited for resource-constrained environments.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) The most prominent strength is the model's minuscule size (~0.1K parameters) and low computational cost (MACs). An 81% parameter reduction compared to the next-lightest model (SparseTSF) is a remarkable achievement. This has profound practical significance for deployment on resource-constrained hardware, which is a major bottleneck for many modern deep learning models.\n\n2) The dual-domain design is well-motivated by the \"Spectral-Temporal Decomposition Principle.\" The idea of processing local patterns in the time domain and global patterns in the frequency domain is intuitive and elegant. The ablation study provides strong empirical evidence that this separation of concerns is effective and that the two pathways are complementary.\n\n3) The paper is backed by a thorough experimental evaluation on eight benchmark datasets. The authors compare against a strong and diverse set of baselines. The inclusion of detailed ablation and hyperparameter sensitivity studies adds significant credibility to the design choices and validates the core hypotheses, especially the finding that a spectral rank as low as 2 is often sufficient.", "weaknesses": "1) The paper's primary weakness is that the individual components of the architecture are not conceptually new. Time series segmentation (PatchTST), linear forecasting models (DLinear), and frequency-domain analysis (FEDformer, FITS) are all established techniques. The novelty lies in the specific integration and extreme simplification of these ideas. While the final result is impressive, the contribution is more of an engineering and simplification achievement than a fundamental theoretical breakthrough. To strengthen the paper, the authors could be more explicit about this, framing the contribution as a novel synthesis that achieves an unprecedented operating point on the efficiency-accuracy curve.\n\n2) The abstract and main results sometimes use strong phrasing like \"surpasses, state-of-the-art models.\" While MixLinear does outperform SOTA on some dataset/horizon combinations (e.g., Exchange), Table 1 and Table 5 show that it is more accurately described as being highly \"competitive\" or \"comparable.\" For instance, on several datasets, FITS or even DLinear achieve slightly better or similar MSE. A more nuanced description of the performance would strengthen the paper's credibility, for example by emphasizing that it achieves this competitiveness with orders of magnitude fewer parameters.\n\n3) The explanation of how the factorized linear projections in Section 2.3 disentangle \"intra-segment\" and \"inter-segment\" correlations is slightly imprecise. The mathematical formulation in the appendix ($W^T_2$ ($W_1 X_seg)^T$) suggests a token-mixing and channel-mixing mechanism on a reshaped tensor, akin to MLP-Mixer, rather than a hierarchical processing of segment embeddings. It's unclear how one linear layer acts on \"intra-segment\" features and the other on \"inter-segment\" features when they are applied in this manner. Clarifying this mechanism and its connection to the intuition would improve the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GFBxnYacjL", "reviewer": "Reviewer_rVBw", "summary": "This paper introduces MixLinear, a novel model for time series forecasting. The central contribution is a dual-domain architecture that processes time series in parallel: (1) a segment-based, factorized linear pathway in the time domain to capture local patterns, and (2) an adaptive low-rank spectral filtering pathway in the frequency domain to model global trends. The authors claim that this approach achieves SOTA-comparable forecasting accuracy while using an extremely small parameter budget of only 0.1K. The model is reported to be significantly more efficient in terms of parameters and inference speed compared to baselines.", "questions": "1.Downsampling Factor: Could the authors provide a sensitivity analysis for the downsampling factor \n? How does the model's performance change with different values of \n (e.g., 2, 4, 8, 16)?\n\n2.The ablation study suggests the time-domain path is better for low-dim data and the freq-domain path is better for high-dim data . Does this not undermine the universality of the core assumption? How does the model perform on data that is known to be highly non-linear or non-stationary, which might not fit either the factorized-linear or low-rank-spectral assumptions?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.Efficiency: the ultra-lightweight (0.1K params) model that matches SOTA performance and the reported gains in inference speed are impressive.\n\n2.Design: The core design principle—exploiting complementary structural sparsity by separating local time-domain patterns (via factorized linear ops) from global frequency-domain patterns (via low-rank filtering)—is intuitive and elegant.", "weaknesses": "1.Marginal Accuracy Gain: The paper's contribution to forecasting accuracy is marginal, it places the entire burden of the paper's contribution on the efficiency claim.\n\n2.Ambiguous Core Mechanisms: The paper lack of important details. The \"learnable upsampling\" needs to be explained.How is it implemented?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)