LiNo: Advancing Recursive Residual Decomposition of Linear and Nonlinear Patterns for Robust Time Series Forecasting

## Abstract

Abstract Forecasting models are pivotal in a data-driven world with vast volumes of time series data that appear as a compound of vast Li near and No nlinear patterns.
Recent deep time series forecasting models struggle to utilize seasonal and trend decomposition to separate the entangled components. Such a strategy only explicitly extracts simple linear patterns like trends, leaving the other linear modes and vast unexplored nonlinear patterns to the residual. Their flawed linear and nonlinear feature extraction models and shallow-level decomposition limit their adaptation to the diverse patterns present in real-world scenarios.
Given this, we innovate Recursive Residual Decomposition by introducing explicit extraction of both linear and nonlinear patterns. This deeper-level decomposition framework, which is named LiNo , captures linear patterns using a Li block which can be a moving average kernel, and models nonlinear patterns using a No block which can be a Transformer encoder. The extraction of these two patterns is performed alternatively and recursively. To achieve the full potential of LiNo, we develop the current simple linear pattern extractor to a general learnable autoregressive model, and design a novel No block that can handle all essential nonlinear patterns.
Remarkably, the proposed LiNo achieves state-of-the-art on thirteen real-world benchmarks under univariate and multivariate forecasting scenarios. Experiments show that current forecasting models can deliver more robust and precise results through this advanced Recursive Residual Decomposition. We hope this work could offer insight into designing more effective forecasting models. Code is available at this Repository : https://github.com/Levi-Ackman/LiNo .

## 1 Introduction

Time series forecasting (TSF) is a long-established task (Lim & Zohren, ; Wang et al., ), with a wide range of applications (Zhou et al., ; LIU et al., ; Piao et al., ).
Notably, numerous deep learning methods have been employed to address the TSF problem, utilizing architectures such as Recurrent Neural Networks (RNNs) (Elman, ; Lin et al., ), Temporal Convolutional Networks (TCNs) (donghao & wang xue, ; Wu et al., ), Multilayer Perceptron (MLP) (Liu et al., ; Challu et al., ) and Transformers (Liu et al., ; Kitaev et al., ).
Recent advancements in the time series forecasting community have suggested that seasonal (nonlinear) and trend (linear) decomposition can enhance forecasting performance (Zhang et al., ; Wu et al., ). This is typically performed once, with the trend component being extracted using methods such as the simple moving average kernel (MOV) (Zhou et al., ; Wang et al., ), the exponential smoothing function (ESF) (Woo et al., ), or a learnable 1D convolution kernel (LD) (Yu et al., ). The seasonal component is then obtained by subtracting the trend part from the original time series.

Figure: Figure 1: Example of the multi-level linear and nonlinear patterns in real-world time series. We take the ETTh2 dataset as an example and decompose a raw time series (Raw) into four signals through linear and nonlinear patterns decomposition. Linear 1&2 is the obtained linear patterns using the proposed Li block, and Nonlinear 1&2 is the obtained nonlinear patterns using No block.
Refer to caption: x1.png

Real-world time series data often exhibit more complex structures, combining multiple levels of linear and nonlinear characteristics (Stock & Watson, ; Dama & Sinoquet, ; Agrawal & Adhikari, ), as illustrated in Figure 1.
When deployed in real-world time series data, simple seasonal and trend decomposition faces three main challenges.
Firstly, these methods rely on simple techniques (MOV, LD, ESF). MOV and LD employed a fixed window size, which cannot fully extract all linear patterns, while the non-learnability of ESF limits its performance. They can only extract simple linear features such as trends, while other linear features like cyclic patterns and autoregression (Chatfield & Xing, ) remain underutilized.
Secondly, we argue the obtained seasonal part is actually a residual consisting of all the unextracted linear and nonlinear information. Without further separating the nonlinear part from the residual severely hinders its extraction. Since it’s extremely challenging for deep models to extract useful information from such mixtures (Bengio et al., ; Tishby & Zaslavsky, ). Another problem is the design of current nonlinear models, which mainly focus on one or two types of nonlinear patterns. Such designs cannot satisfy our requirements in the real world.
Thirdly, their shallow-level decomposition is incompatible with the multi-level characteristics of real-world time series. In contrast, numerous studies have shown that deeper-level decomposition can lead to better time series analysis (Huang et al., ; Rilling et al., ).
Therefore, it is crucial to develop advanced decomposition methods capable of multi-granularity separation of various modes, and better linear and nonlinear pattern extractors to provide a more nuanced understanding of the signal’s structure and potentially improve forecasting performance.

To address the challenges above, we propose adopting Recursive Residual Decomposition (RRD), a method used in Empirical Mode Decomposition (EMD) (Huang et al., ; Rilling et al., ), to decompose a time series into multiple patterns. This process is performed recursively. Each pattern is extracted based on the residual obtained by subtracting previously extracted patterns from the original signal, utilizing Intrinsic Mode Functions (IMF) to identify similar characteristics. Instead of traditional RRD, we explicitly and alternately extract both linear and nonlinear patterns in this paper.
Specifically, We can employ a Li block (MOV, LD, ESF, etc.) as the IMF for extracting linear patterns and a No block (Transformer, RNN, etc.) as the IMF for extracting nonlinear patterns. Then, the RRD is performed on a deeper level.
We denote the proposed overall framework as LiNo. To fully realize RRD’s potential, we propose an advanced LiNo, Specifically, we enhance the Li block to a general learnable autoregressive model with a full receptive field and propose a novel No block capable of modeling essential nonlinear features, such as temporal variations, period information, and inter-series dependencies in multivariate forecasting.

In summary, our contributions can be delineated as follows.

- •
We advance current shallow linear and nonlinear decomposition by innovating Recursive Residual Decomposition.
- •
Proposed No block demonstrates better nonlinear pattern extraction ability than current SOTA nonlinear models, such as iTransformer and TSMxier.
- •
LiNo consistently delivers top-tier performance in both multivariate and univariate forecasting scenarios, demonstrating robust resilience against various noise disturbances.
- •
The significant improvement by more nuanced and deeper linear and nonlinear decomposition provides insight for designing more effective and robust forecasting models.

## 2 Related Work

#### Advancement in recent time series forecasting.

Time series forecasting is a critical area of research that finds applications in both industry and academia. With the powerful representation capability of neural networks, deep forecasting models have undergone a rapid development (Wang et al., ; Lim & Zohren, ; Torres et al., ).
Recent research endeavors have focused on segmenting the sequence into a series of patches (Nie et al., ; Zhang & Yan, ), better modeling the relationships between variables (Ng et al., ; Chen et al., ), the dynamic changes within a sequence (Wu et al., ; Du et al., ), or both (Yu et al., ; Liu et al., ).
Some works strive for more efficient forecasting solutions (Lin et al., ; Xu et al., ). Other efforts aim to revitalize existing architectures, such as RNN (Lin et al., ), Transformer (Liu et al., ), TCN (donghao & wang xue, ), with new ideas, or to explore the potential of outstanding architectures from other domains, such as MLP-Mixer (Chen et al., ; Wang et al., ), Mamba (Ahamed & Cheng, ; Wang et al., ), Graph Neural Network (Yi et al., ; Shao et al., ), even Large Language Models (Jin et al., ; Liu et al., ; Pan et al., ; Bian et al., ; Gruver et al., ), for application in time series forecasting. Notably, some efforts also begin to ponder the role of self-attention in time series forecasting (Ilbert et al., ).

#### Importance of linear and nonlinear patterns.

Deep time series models that are dedicated to model nonlinear patterns such as non-stationarity (Liu et al., ), time-variant and time-invariant features (Liu et al., ), and frequency bias (Piao et al., ) have delivered outstanding performance in various domains. In contrast, recent advances have proved the importance of attention to linear patterns in time series (Toner & Darlow, ). For instance, DLinear (Zeng et al., ) and RLinear (Li et al., ) achieve results comparable to, or even surpassing, many intricately designed nonlinear models in certain scenarios, using only simple linear layers. FITS (Xu et al., ) even achieves SOTA performance with merely $10k$ parameters. This suggests that balanced consideration of both linear and nonlinear patterns can be crucial for enhancing predictive performance in time series forecasting, which is unexplored in the current simple seasonal and trend decomposition.

## 3 Methodology

### 3.1 Preliminary

Given a multivariate time series $X\in\mathbb{R}^{B\times C\times T}$ with a length of $T$ time steps, time series forecasting tasks are designed to predict its future $F$ time steps $\hat{Y}\in\mathbb{R}^{B\times C\times F}$, where $B$ is $batch\ size$, $C$ is the number of variate or channel ($C=1$ in univariate case), and $T$ represents the look-back window. We aim to make $\hat{Y}$ closely close to the ground truth $Y\in\mathbb{R}^{B\times C\times F}$.

Without loss of generality and to simplify the analysis, we assume a real-world univariate time series $X$ consisting of $S$ linear patterns, $S$ nonlinear patterns, and a white noise $\varepsilon$.
We denote $X=L_{1}+N_{1}+\cdots+L_{S}+N_{S}+\varepsilon$,
where $L_{i}$ is a Linear signal, $N_{i}$ is a Nonlinear signal.
During the ‘Recursive Residual Decomposition’ (RRD) process of LiNo, if we denote the extracted linear and nonlinear patterns using different IMFs (such as MOV, LD, ESF for linear, Transformer, MLP-Mixer for nonlinear) at each time as $\hat{L}_{i}$ and $\hat{N}_{i}$, we have

$$ $\displaystyle R^{L}_{1}$ $\displaystyle=X-\hat{L}_{1},\ \ R^{N}_{1}=R^{L}_{1}-\hat{N}_{1},$ (1) $\displaystyle\quad\quad\quad\quad\quad\dots$ $\displaystyle R^{L}_{S}$ $\displaystyle=R^{N}_{S-1}-\hat{L}_{S},\ \ R^{N}_{S}=R^{L}_{S}-\hat{N}_{S}.$ $$

where $\lim\limits_{i\to\infty}R^{N}_{i}=0$. The modeling error will be $\delta=X-(\hat{L}_{1}+\hat{N}_{1}+\cdots+\hat{L}_{n}+\hat{N}_{n})=(L_{1}-\hat{
L}_{1})+(N_{1}-\hat{N}_{1})+(L_{2}-\hat{L}_{2})+(N_{2}-\hat{N}_{2})+\cdots+(L_
{S}-\hat{L}_{S})+(N_{S}-\hat{N}_{S})+\varepsilon$.
By alternating the extraction of linear and nonlinear features with subtracting the extracted features from the original input representation, we ensure previously extracted features do not affect the extraction of subsequent features, guaranteeing the independence of the resulting linear and nonlinear patterns.
Moreover, $\lim\limits_{i\to\infty}R^{N}_{i}=0$ ensures that all valuable information in the sequence is fully extracted if the decomposition level is deep enough, preventing any loss of information.
This design takes the existing shallow RRD to a deeper level, introducing the extraction of nonlinear patterns. Such a refined design ensures that the model can achieve more robust forecasting results.
Notably, if set the RRD level to $1$, then we get $Trend=IMF(X)=\hat{L}_{1}$, and $Seasonal=X-Trend=R^{L}_{1}$, where $IMF$ can be MOV, LD, or ESF, which is equivalent to the former seasonal and trend decomposition.

### 3.2 LiNo Pipeline

Figure: Figure 2: Framework of LiNo. Li block and No block extract patterns from the embedded input alternatively, in an RRD manner. The final prediction is aggregated from all blocks.
Refer to caption: x2.png

The structure of the LiNo framework is illustrated in Figure 2. Initially, we extract the whole series embedding, which is then processed through $N$ LiNo blocks to forecast future values of the time series. In this subsection, we will provide a detailed explanation of the whole series embedding, Li block, and No block, step by step.

#### Whole series embedding.

Following iTransformer (Liu et al., ), we first map time series data $X\in\mathbb{R}^{B\times C\times T}$ from the original space to a new space to get $X_{embed}\in\mathbb{R}^{B\times C\times D}$ using a simple linear projection, where $D$ denotes the dimension of the layer. Such a design can better preserve the unique patterns of each variate.

#### Li block.

The fixed window size of the MOV and learnable 1D convolution kernel (LD), and the non-learnability of the exponential smoothing function (ESF), prevent them from fully extracting all linear patterns. So we introduce a learnable autoregressive model with a full receptive field to replace them, where MOV, LD, and ESF are its subset.

Given its input feature $H_{i}\in\mathbb{R}^{B\times C\times D}$ where $H_{0}=X_{embed}$, we get $i$-th linear pattern $L_{i}\in\mathbb{R}^{B\times C\times D}$ by:

$$ $\begin{array}[]{c}\hat{L}_{i}[:,c,d]=\phi_{i}[c,1]*H_{i}[:,c,1]+\phi_{i}[c,2]* H_{i}[:,c,2]\\ +\cdots+\phi_{i}[c,d]*H_{i}[:,c,d]+\beta_{i}[c],\\ L_{i}=\textbf{Dropout}(\hat{L}_{i}).\end{array}$ (2) $$

Here, $\phi_{i}\in\mathbb{R}^{C\times D}$ represents the autoregressive coefficients, $\beta_{i}\in\mathbb{R}^{C}$ denotes the white noise error term. We assume the noise error term of each channel across all time stamps to be constant. So the whole process of extracting the linear part of the input feature can be easily deployed in a convolution fashion by setting the weight of the convolution kernel to $\phi_{i}$, and the weight of bias to $\beta_{i}$.
Before applying the convolution, we pad the input feature $\mathbf{H}_{i}$. The padded input feature $\mathbf{H}_{i}^{\prime}$ becomes

$$ $\displaystyle\mathbf{H}_{i}^{\prime}[:,:,t]=\begin{cases}\mathbf{H}_{i}[:,:,t- D],&\text{for }t\geq D\\ 0,&\text{for }t<0\end{cases}.$ (3) $$

We perform convolutions on each channel independently across the last dimension of $\mathbf{H}_{i}^{\prime}$. Following this, we apply dropout for generalization. The linear prediction $P^{li}_{i}\in\mathbb{R}^{B\times C\times F}$ for this level is obtained by mapping the extracted linear component $L_{i}$. Subsequently, we pass $R^{L}_{i}=H_{i}-L_{i}$ to the next stage for nonlinear pattern modeling.

#### No block.

Typical nonlinear time series characteristics include temporal variations, frequency information, inter-series dependencies, etc. While all of these factors are crucial for precise forecasting, current nonlinear models can only address one (Wu et al., ; Nie et al., ; Zhang et al., ; Piao et al., ) or two (Liu et al., ; Li et al., ; Chen et al., ) of these aspects. Therefore, we design a No block that can handle all these characteristics simultaneously.

Given a feature $R^{L}_{i}\in\mathbb{R}^{B\times C\times D}$, both temporal variation patterns and frequency information are accessible through linear projection. Hence, we start by applying a linear projection in the time domain to obtain temporal variation patterns $N^{T}_{i}\in\mathbb{R}^{B\times C\times D}$ in a classical manner. Correspondingly, we apply a linear projection in the frequency domain, which is transformed from $R^{L}_{i}$ using the Fast Fourier Transform (FFT), following FITS (Xu et al., ). Thereafter, the frequency domain representation is converted back using the Inverse Fast Fourier Transform (IFFT) to obtain frequency information patterns $N^{F}_{i}\in\mathbb{R}^{B\times C\times D}$. To leverage the complementary strengths of both domains, features extracted from both the time and frequency domains are fused and activated by: $N^{TF}_{i}=\textbf{Tanh}(N^{T}_{i}+N^{F}_{i})$.

To model inter-series dependencies, we first normalize $N^{TF}_{i}$ across the channel dimension using a softmax function. We then compute a weighted mean by multiplying the softmax weights with $N^{TF}_{i}$ and summing over the channel dimension. This weighted mean is repeated to match and concatenate with $R^{L}_{i}$. The concatenated result is passed through a Feedforward Network to obtain inter-series dependencies information $N^{C}_{i}$.

To integrate temporal variations, frequency information, and inter-series dependencies, we first apply Layer Normalization to the sum of $N^{TF}_{i}$ and $N^{C}_{i}$, resulting in $N^{TFC}_{i}$. A subsequent Feedforward Network is applied, and the result is added back to $N^{TFC}_{i}$, followed by a final Layer Normalization to produce the overall nonlinear pattern $N_{i}$. The nonlinear prediction $P^{no}_{i}\in\mathbb{R}^{B\times C\times F}$ for this level is then obtained by mapping the extracted nonlinear part $N_{i}$. Finally, $R^{N}_{i}=R^{L}_{i}-N_{i}$ is passed to the next LiNo block, where $H_{i+1}=R^{N}_{i}$.

#### ReVIN and forecasting results.

We used ReVIN (Kim et al., ) to counter the distribution problem following iTransformerLiu et al. (). The input first performs an Instance Normalization (Ulyanov, ) before being embedded. The final output is reversed using the Mean and Standard Deviation of Instance Normalization. The final prediction result is aggregated from multi-level by:

$$ $\displaystyle\hat{Y}=$ $\displaystyle\sum^{N}_{i=1}(P^{li}_{i})+\sum^{N}_{i=1}(P^{no}_{i}).$ (4) $$

## 4 Experiments

#### Datasets.

To thoroughly evaluate the performance of our proposed LiNo, we conduct extensive experiments on 13 widely used, real-world datasets including ETT (4 subsets) (Zhou et al., ), Traffic, Exchange, Electricity(ECL), Weather (Wu et al., ), Solar-Energy(Solar) (Lai et al., ) and PEMS (4 subsets) (LIU et al., ). Detailed descriptions of the datasets can be found in Appendix A. We select both univariate and multivariate time series forecasting tasks, ensuring a comprehensive assessment.

#### Experimental setting.

All the experiments are conducted on a single NVIDIA GeForce RTX 4090 with 24G VRAM. The mean squared error (MSE) loss function is utilized for model optimization. We use the ADAM optimizer with an early stop parameter $patience=6$. To foster reproducibility, we make our code, training scripts, and some visualization examples available in this GitHub Repository(^1^11[https://github.com/Levi-Ackman/LiNo](https://github.com/Levi-Ackman/LiNo)). Full implementation details and other information can be found in Appendix B.1.

### 4.1 Multivariate Time Series Forecasting Results

#### Compared methods and benchmarks.

We extensively compare the recent Linear-based or MLP-based methods, including DLinear (Zeng et al., ), TSMixer (Chen et al., ), TiDE (Das et al., ), RLinear (Li et al., ). We also consider Transformer-based methods including FEDformer (Zhou et al., ), Autoformer (Wu et al., ), PatchTST (Nie et al., ), Crossformer (Zhang & Yan, ), iTransformer (Liu et al., ) and a CNN-based method TimesNet (Wu et al., ). These models represent the latest advancements in multivariate time series forecasting and encompass all mainstream prediction model types.
The multivariate time series forecasting benchmarks follow the setting in iTransformer (Liu et al., ). The lookback window is set to $T=96$ for all datasets. We set the prediction horizon to $F\in\{12,24,48,96\}$ for PEMS dataset and $F\in\{96,192,336,720\}$ for others. Performance comparison among different methods is conducted based on two primary evaluation metrics: Mean Squared Error (MSE) and Mean Absolute Error (MAE). The results of TSMixer are reproduced following Time Series Library (Wang et al., ) and other results are taken from iTransformer (Liu et al., ).

#### Result analysis.

**Table 1: Multivariate forecasting results with prediction lengths $F\in\{12,24,36,48\}$ for PEMS dataset while $F\in\{96,192,336,720\}$ for others with fixed lookback window $T=96$. Results are averaged from all prediction lengths. *Avg* means further averaged by subsets. Full result is left in Appendix E.1 due to space limit.**
| Models | LiNo | iTransformer | RLinear | PatchTST | TSMixer | Crossformer | TiDE | TimesNet | DLinear | FEDformer | Autoformer |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| (Ours) | () | () | () | () | () | () | () | () | () | () |  |  |  |  |  |  |  |  |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |
| ETT (Avg) | 0.368 | 0.387 | 0.383 | 0.399 | 0.380 | 0.392 | 0.381 | 0.397 | 0.388 | 0.402 | 0.685 | 0.578 | 0.482 | 0.470 | 0.391 | 0.404 | 0.442 | 0.444 | 0.408 | 0.428 | 0.465 | 0.459 |
| ECL | 0.164 | 0.260 | 0.178 | 0.270 | 0.219 | 0.298 | 0.205 | 0.290 | 0.186 | 0.287 | 0.244 | 0.334 | 0.251 | 0.344 | 0.192 | 0.295 | 0.212 | 0.300 | 0.214 | 0.327 | 0.227 | 0.338 |
| Exchange | 0.350 | 0.398 | 0.360 | 0.403 | 0.378 | 0.417 | 0.367 | 0.404 | 0.376 | 0.414 | 0.940 | 0.707 | 0.370 | 0.413 | 0.416 | 0.443 | 0.354 | 0.414 | 0.519 | 0.429 | 0.613 | 0.539 |
| Traffic | 0.465 | 0.296 | 0.428 | 0.282 | 0.626 | 0.378 | 0.481 | 0.304 | 0.522 | 0.357 | 0.550 | 0.304 | 0.760 | 0.473 | 0.620 | 0.336 | 0.625 | 0.383 | 0.610 | 0.376 | 0.628 | 0.379 |
| Weather | 0.241 | 0.270 | 0.258 | 0.279 | 0.272 | 0.291 | 0.259 | 0.281 | 0.256 | 0.279 | 0.259 | 0.315 | 0.271 | 0.320 | 0.259 | 0.287 | 0.265 | 0.317 | 0.309 | 0.360 | 0.338 | 0.382 |
| Solar-Energy | 0.230 | 0.270 | 0.233 | 0.262 | 0.369 | 0.356 | 0.270 | 0.307 | 0.260 | 0.297 | 0.641 | 0.639 | 0.347 | 0.417 | 0.301 | 0.319 | 0.330 | 0.401 | 0.291 | 0.381 | 0.885 | 0.711 |
| PEMS03 | 0.096 | 0.197 | 0.113 | 0.221 | 0.495 | 0.472 | 0.119 | 0.233 | 0.180 | 0.291 | 0.169 | 0.281 | 0.326 | 0.419 | 0.147 | 0.248 | 0.278 | 0.375 | 0.213 | 0.327 | 0.667 | 0.601 |
| PEMS04 | 0.098 | 0.203 | 0.111 | 0.221 | 0.526 | 0.491 | 0.103 | 0.215 | 0.195 | 0.307 | 0.209 | 0.314 | 0.353 | 0.437 | 0.129 | 0.241 | 0.295 | 0.388 | 0.231 | 0.337 | 0.610 | 0.590 |
| PEMS07 | 0.088 | 0.181 | 0.101 | 0.204 | 0.504 | 0.478 | 0.112 | 0.217 | 0.211 | 0.303 | 0.235 | 0.315 | 0.380 | 0.440 | 0.124 | 0.225 | 0.329 | 0.395 | 0.165 | 0.283 | 0.367 | 0.451 |
| PEMS08 | 0.138 | 0.217 | 0.150 | 0.226 | 0.529 | 0.487 | 0.165 | 0.261 | 0.280 | 0.321 | 0.268 | 0.307 | 0.441 | 0.464 | 0.193 | 0.271 | 0.379 | 0.416 | 0.286 | 0.358 | 0.814 | 0.659 |
| $1^{\text{st}}$ Count | 9 | 8 | 1 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

As shown in Table 1, LiNo performed remarkably across 10 benchmark datasets. It achieved first place in 9 out of 10 datasets in MSE and 8 datasets in MAE, underscoring its leading position in multivariate time series forecasting tasks. LiNo successfully reduced the MSE metric by 3.41% compared to the previous state-of-the-art method, iTransformer, across all 10 datasets.
Notably, the PEMS datasets (PEMS03: $358$ variates, PEMS04: $307$ variates, PEMS07: $883$ variates, PEMS08: $170$ variates) and the ECL dataset ($321$ variates) present notorious challenges to multivariate time series forecasting models due to their high dimensionality and complex nonlinearity. LiNo demonstrated its superiority in nonlinear pattern extraction by achieving a substantial relative decrease of 11.89% in average MSE on the four PEMS-relevant benchmarks. On the ECL dataset, LiNo decreased the average MSE from 0.178 to 0.164, representing a significant reduction of about 7.87%.

Previous research suggests that simple linear models can outperform complex deep neural networks in certain scenarios (Zeng et al., ; Li et al., ). For instance, in scenarios where the dataset displays clear nonlinear patterns, nonlinear models like iTransformer excel. However, on the ETT datasets (four subsets), RLinear, which consists of a linear layer combined with ReVIN (Kim et al., ), easily surpassed all previous sophisticated deep models.
We argue that this is because most of these models focus solely on either linear or nonlinear patterns, neglecting the other, leading to inconsistent performance across different scenarios. In contrast, LiNo performs outstandingly across various scenarios, demonstrating the importance of a balanced approach to handling both linear and nonlinear patterns.

### 4.2 Univariate Time Series Forecasting Results

#### Compared methods and benchmarks.

The models and results used for comparing univariate time series forecasting performance were collected from MICN (Wang et al., ), including MICN (Wang et al., ), FEDformer (Zhou et al., ), Autoformer (Wu et al., ), Informer (Zhou et al., ), LogTrans (Li et al., ).
We follow the setting in MICN (Liu et al., ) where the lookback window length is set to $T=96$ and the prediction horizon to $F\in\{96,192,336,720\}$ for all datasets.

#### Result analysis.

**Table 2: Univariate forecasting results with prediction lengths $F\in\{96,192,336,720\}$ and fixed lookback length $T=96$ for all datasets. Results are averaged from all prediction lengths. Full result is left in Appendix E.2 due to space limit.**
| Models | LiNo | MICN | FEDformer | Autoformer | Informer | LogTrans |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| (Ours) | () | () | () | () | () |  |  |  |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |
| ETTm1 | 0.053 | 0.172 | 0.064 | 0.185 | 0.069 | 0.202 | 0.081 | 0.221 | 0.281 | 0.441 | 0.231 | 0.382 |
| ETTm2 | 0.118 | 0.255 | 0.131 | 0.266 | 0.119 | 0.262 | 0.130 | 0.271 | 0.175 | 0.320 | 0.130 | 0.277 |
| ETTh1 | 0.074 | 0.209 | 0.092 | 0.233 | 0.111 | 0.257 | 0.105 | 0.252 | 0.199 | 0.377 | 0.345 | 0.513 |
| ETTh2 | 0.180 | 0.332 | 0.252 | 0.390 | 0.206 | 0.350 | 0.218 | 0.364 | 0.243 | 0.400 | 0.252 | 0.408 |
| Traffic | 0.143 | 0.222 | 0.165 | 0.246 | 0.219 | 0.323 | 0.261 | 0.365 | 0.309 | 0.388 | 0.355 | 0.404 |
| Weather | 0.0016 | 0.030 | 0.0030 | 0.040 | 0.0055 | 0.058 | 0.0083 | 0.070 | 0.0033 | 0.044 | 0.0058 | 0.057 |
| $1^{\text{st}}$ Count | 6 | 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

Table 2 demonstrates the top-notch performance of LiNo in univariate time series forecasting tasks, achieving the best predictive results across all 6 datasets. On six datasets, LiNo reduced the MSE metric by 19.37% and the MAE by 10.28% compared to the previous SOTA method, MICN. Notably, on the Weather, ETTh2, and Traffic datasets, the MSE decreased by 47.11%, 28.64%, and 12.97%, respectively, marking a significant improvement. The consistent superior advancement in both univariate and multivariate time series forecasting demonstrates the wide applicability of LiNo across various scenarios.

### 4.3 LiNo Analysis

#### Ablation study on LiNo components.

**Table 3: Ablations on LiNo. ‘w/o’ means remove this design. ‘TE’ stands for temporal variations extraction, ‘FE’ strands for frequency information extraction, and ‘CD’ means the channel mixing step for inter-series dependencies modeling.**
| Dataset | LiNo | w/o Li Block | w/o No Block | w/o TE | w/o FE | w/o CD |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |  |
| ECL | 96 | 0.138 | 0.233 | 0.149 | 0.242 | 0.186 | 0.267 | 0.143 | 0.241 | 0.140 | 0.237 | 0.154 | 0.248 |
| 720 | 0.191 | 0.290 | 0.210 | 0.300 | 0.245 | 0.320 | 0.197 | 0.293 | 0.196 | 0.294 | 0.223 | 0.311 |  |
| Weather | 96 | 0.154 | 0.199 | 0.211 | 0.300 | 0.162 | 0.206 | 0.158 | 0.204 | 0.159 | 0.204 | 0.164 | 0.209 |
| 720 | 0.343 | 0.342 | 0.355 | 0.348 | 0.342 | 0.338 | 0.347 | 0.344 | 0.345 | 0.343 | 0.347 | 0.345 |  |
| ETTm2 | 96 | 0.171 | 0.254 | 0.177 | 0.260 | 0.180 | 0.261 | 0.173 | 0.256 | 0.173 | 0.255 | 0.175 | 0.257 |
| 720 | 0.395 | 0.393 | 0.399 | 0.396 | 0.409 | 0.400 | 0.399 | 0.396 | 0.397 | 0.395 | 0.391 | 0.392 |  |
| ETTh2 | 96 | 0.292 | 0.340 | 0.295 | 0.344 | 0.301 | 0.348 | 0.294 | 0.342 | 0.294 | 0.341 | 0.291 | 0.340 |
| 720 | 0.422 | 0.441 | 0.416 | 0.436 | 0.429 | 0.446 | 0.424 | 0.442 | 0.426 | 0.443 | 0.424 | 0.442 |  |
| PEMS04 | 12 | 0.069 | 0.169 | 0.083 | 0.186 | 0.121 | 0.232 | 0.070 | 0.173 | 0.070 | 0.171 | 0.083 | 0.186 |
| 96 | 0.137 | 0.247 | 0.278 | 0.359 | 1.016 | 0.762 | 0.149 | 0.261 | 0.146 | 0.259 | 0.296 | 0.349 |  |
| PEMS08 | 12 | 0.070 | 0.166 | 0.076 | 0.176 | 0.119 | 0.230 | 0.075 | 0.180 | 0.074 | 0.176 | 0.085 | 0.189 |
| 96 | 0.247 | 0.283 | 0.359 | 0.359 | 1.075 | 0.771 | 0.258 | 0.299 | 0.253 | 0.292 | 0.431 | 0.384 |  |
| Solar-Energy | 96 | 0.200 | 0.250 | 0.238 | 0.310 | 0.338 | 0.258 | 0.203 | 0.257 | 0.204 | 0.258 | 0.226 | 0.271 |
| 720 | 0.250 | 0.283 | 0.251 | 0.395 | 0.369 | 0.292 | 0.260 | 0.296 | 0.267 | 0.297 | 0.277 | 0.297 |  |
| avg-promote | 0 | 0 | -10.00% | -6.48% | -71.82% | -35.97% | -2.27% | -2.52% | -2.27% | -1.80% | -15.91% | -8.27% |  |

**Table 4: Impact of the number of LiNo blocks (layers) $N$ on the model’s performance. The task is input-96-predict-96 for PEMS04&08, and input-96-predict-720 for others. We set $N\in\{1,2,3,4\}$. The best results are bold in red.**
| Number of LiNo blocks | ETTh1 | ETTh2 | ECL | Weather | Solar | PEMS04 | PEMS08 |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Value \Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |  |
| N | 1 | 0.472 | 0.466 | 0.427 | 0.446 | 0.197 | 0.295 | 0.346 | 0.344 | 0.268 | 0.299 | 0.152 | 0.262 | 0.267 | 0.308 |
|  | 2 | 0.471 | 0.466 | 0.421 | 0.440 | 0.191 | 0.290 | 0.343 | 0.342 | 0.250 | 0.283 | 0.145 | 0.256 | 0.247 | 0.283 |
|  | 3 | 0.459 | 0.456 | 0.423 | 0.442 | 0.192 | 0.292 | 0.346 | 0.344 | 0.255 | 0.285 | 0.143 | 0.250 | 0.258 | 0.296 |
|  | 4 | 0.468 | 0.463 | 0.424 | 0.442 | 0.194 | 0.293 | 0.347 | 0.345 | 0.257 | 0.286 | 0.137 | 0.247 | 0.258 | 0.296 |

To verify the effectiveness of LiNo components, we conducted ablation studies by removing components (w/o) on 7 multivariate time series forecasting benchmarks with a lookback window of $T=96$ and prediction lengths $F\in\{96,720\}$. The results are presented in Table 3.
Every design in LiNo is crucial. Removing the No block results in significant performance degradation, with an MSE increase of up to 71.82%. Similarly, the absence of the Li block leads to a 10.00% rise in MSE, underscoring the importance of modeling both linear and nonlinear patterns. Further ablation of the No block reveals that temporal variation and frequency information extraction are essential. The absence of inter-series dependencies modeling results in a 15.91% increase in MSE, highlighting its critical importance.

#### Sensitivity to the number of LiNo blocks.

We investigate the impact of the number of LiNo blocks (layers) $N$ on the model’s performance, as shown in Table 4. The best forecasting results for each dataset are generally achieved when $N>1$, indicating the necessity of deeper RRD. The variation in optimal $N$ across datasets suggests that LiNo can flexibly adapt to different RRD requirements.

#### Superiority of the proposed No block.

**Table 5: Ablation study of different No block choice and different model design.**
| Models | Ori LiNo | $\rightarrow$ iTransformer | $\rightarrow$ TSMixer |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE |  |
| ETTm2 | 96 | 0.171 | 0.254 | 0.174 | 0.259 | 0.173 | 0.256 |
| 192 | 0.237 | 0.298 | 0.238 | 0.299 | 0.239 | 0.301 |  |
| 336 | 0.296 | 0.336 | 0.302 | 0.340 | 0.304 | 0.343 |  |
| 720 | 0.395 | 0.393 | 0.402 | 0.398 | 0.406 | 0.402 |  |
| Avg | 0.275 | 0.320 | 0.279 | 0.324 | 0.281 | 0.326 |  |
| ECL | 96 | 0.138 | 0.233 | 0.141 | 0.239 | 0.145 | 0.249 |
| 192 | 0.155 | 0.250 | 0.163 | 0.254 | 0.164 | 0.263 |  |
| 336 | 0.171 | 0.267 | 0.175 | 0.271 | 0.187 | 0.285 |  |
| 720 | 0.191 | 0.290 | 0.201 | 0.295 | 0.228 | 0.320 |  |
| Avg | 0.164 | 0.260 | 0.170 | 0.265 | 0.181 | 0.279 |  |
| Weather | 96 | 0.154 | 0.199 | 0.156 | 0.201 | 0.157 | 0.203 |
| 192 | 0.205 | 0.248 | 0.206 | 0.248 | 0.206 | 0.250 |  |
| 336 | 0.262 | 0.290 | 0.264 | 0.291 | 0.267 | 0.295 |  |
| 720 | 0.343 | 0.342 | 0.345 | 0.343 | 0.349 | 0.347 |  |
| Avg | 0.241 | 0.270 | 0.243 | 0.271 | 0.245 | 0.274 |  |

Extracting nonlinear patterns, such as inter-series dependencies, temporal variations, and frequency information, is crucial for accurate predictions. To demonstrate the competence of the proposed No block, we sequentially replaced it with two renowned nonlinear time series forecasting models: iTransformer and TSMixer.
As shown in Table 5 (a), our proposed No block consistently outperforms other nonlinear pattern extractors. It delivers superior performance across ETTm2 (7 variates), Weather (21 variates), and ECL (321 variates), showcasing its remarkable ability to extract nonlinear patterns.

#### Increasing lookback length.

Figure: Figure 3: Multivariate forecasting performance improves with the increase of lookback window $T\in\{48,96,192,336,720\}$ and a fixed prediction length $F=720$. Notably, LiNo consistently and stably enhances its forecasting performance as the lookback window size increases.
Refer to caption: x3.png

It is generally expected that increasing the input length will enhance forecasting performance by incorporating more information (Zeng et al., ). This improvement is typically observed in linear forecasts, supported by statistical methods (Liu et al., ) that utilize extended historical data.
Figure 3 evaluates the performance of LiNo and other prestigious baselines. LiNo effectively leverages longer lookback windows, showing a positive correlation between predictive performance and input length. It significantly outperforms other baselines on the Weather and ECL datasets and achieves comparable results on the ETTm2 dataset. Further analysis of LiNo is provided in Appendix C.

### 4.4 Analysis of Different Forecasting Model Designs

#### Forecasting performance comparison.

We use iTransformer, a leading transformer-based time series forecasting model, as the backbone. We evaluate three model designs: ‘Raw’ (classical design), ‘Mu’ (recursive splitting of representations for prediction), and ‘LiNo’ (our proposed framework with further recursive splitting into linear and nonlinear patterns).
The forecasting performance in Table 5 (b) demonstrates the effectiveness of the LiNo framework. Compared to ‘Raw’, ‘Mu’ reduces the MSE on ETTm2, ECL, and Weather by 1.3%, 0.28%, and 1.06%, respectively. LiNo further reduces the MSE by 2.96%, 6.34%, and 6.72%. These results indicate that while ‘Mu’ improves forecasting performance, it remains suboptimal due to the entanglement of linear and nonlinear predictions. LiNo effectively separates these patterns, achieving more accurate results.

#### Noise robustness.

Figure: Figure 4: Multivariate forecasting performance of three different model designs using iTransformer as backbone under different noise levels across datasets of ECL, ETTm2, and Weather.
Refer to caption: x4.png

To investigate the robustness of different forecasting model designs to noise, we conducted experiments using iTransformer as the backbone. Given an input multivariate time series signal $X\in\mathbb{R}^{B\times T\times N}$, we added Gaussian noise to obtain:
$\hat{X}=X+\alpha\cdot\text{noise},$
where $\alpha\in\{0\%,25\%,50\%,75\%,100\%\}$ is the noise intensity coefficient, and $\text{noise}\in\mathbb{R}^{B\times T\times N}$ is Gaussian noise with mean 0 0 and standard deviation $1$. The noisy input $\hat{X}$ was used during training. A more robust forecasting model will be less affected by this noise.

Our LiNo design consistently outperforms the ‘Mu’ and ‘Raw’ models across various noise levels, demonstrating superior robustness and reliability in forecasting, as shown in Figure 4. This result supports our hypothesis that separating linear and nonlinear patterns enhances model robustness.

#### Visualization of linear and nonlinear predictions.

Figure: Figure 5: Visualization of multivariate forecasting result (last channel/variate) of the proposed LiNo on ETTh1 and ECL datasets. We set the number of LiNo blocks (layers) to $N=3$. The task is multivariate time series forecasting with input $T=96$ and target $F=96$. ‘LP’ denotes Linear prediction, and ‘NP’ stands for Nonlinear prediction. LP $i$ or NP $i$ ($i\in\{1,2,3\}$) is the linear or nonlinear prediction of $i$-th layer (level).
Refer to caption: x5.png

We visualize LiNo’s forecasting results on the ETTh1 and ECL datasets in Figure 5. The Li block primarily captures linear patterns such as long-term trends, while the No block effectively captures nonlinear signals like fluctuations and seasonality. The forecasting results for ECL and ETTh1 reveal three distinct linear modes and three different nonlinear patterns, enhancing the interpretability of time series forecasting and aiding in understanding the underlying data dynamics.
Additional visualization results of LiNo can be found in Appendix D.

## 5 Conclusion

The commonly used seasonal and trend decomposition (STD) in previous methods still rely on flawed linear pattern extractors. Their lack of separating the nonlinear component from residual and shallow-level decomposition severely hinders its modeling capability. This work advances them by incorporating a more general linear extraction model and introducing a novel and powerful nonlinear extraction model into RRD. By performing RRD at a deeper and more nuanced level, we achieve a more refined decomposition, leading to more accurate and robust forecasting results.
The proposed No block excels in capturing nonlinear features. Experiments across multiple benchmarks demonstrated LiNo’s superior performance in both univariate and multivariate forecasting tasks, offering improved accuracy and stability. These findings could offer opportunities to design more robust and precise forecasting models.

## 6 Ethics Statement

Our work only focuses on the scientific problem, all datasets are publicly available, so there is no potential ethical risk.

## 7 Reproducibility Statement

We involve the implementation details in Appendix B. The source code is accessible in GitHub ([https://github.com/Levi-Ackman/LiNo](https://github.com/Levi-Ackman/LiNo)) for reproducibility.

## Appendix A Datasets Description

**Table 6: Detailed dataset descriptions. *Channels* denotes the number of channels in each dataset. *Dataset Split* denotes the total number of time points in (Train, Validation, Test) split respectively. *Prediction Length* denotes the future time points to be predicted, and four prediction settings are included in each dataset. *Granularity* denotes the sampling interval of time points.**
| Dataset | Channels | Prediction Length | Dataset Split | Granularity | Domain |
| --- | --- | --- | --- | --- | --- |
| ETTh1, ETTh2 | 7 | {96, 192, 336, 720} | (8545, 2881, 2881) | Hourly | Electricity |
| ETTm1, ETTm2 | 7 | {96, 192, 336, 720} | (34465, 11521, 11521) | 15min | Electricity |
| Exchange | 8 | {96, 192, 336, 720} | (5120, 665, 1422) | Daily | Economy |
| Weather | 21 | {96, 192, 336, 720} | (36792, 5271, 10540) | 10min | Weather |
| ECL | 321 | {96, 192, 336, 720} | (18317, 2633, 5261) | Hourly | Electricity |
| Traffic | 862 | {96, 192, 336, 720} | (12185, 1757, 3509) | Hourly | Transportation |
| Solar-Energy | 137 | {96, 192, 336, 720} | (36601, 5161, 10417) | 10min | Energy |
| PEMS03 | 358 | {12, 24, 48, 96} | (15617,5135,5135) | 5min | Transportation |
| PEMS04 | 307 | {12, 24, 48, 96} | (10172,3375,281) | 5min | Transportation |
| PEMS07 | 883 | {12, 24, 48, 96} | (16911,5622,468) | 5min | Transportation |
| PEMS08 | 170 | {12, 24, 48, 96} | (10690,3548,265) | 5min | Transportation |

We elaborate on the datasets employed in this study with the following details.

- 1.
ETT (Electricity Transformer Temperature) Zhou et al. () (^2^22[https://github.com/zhouhaoyi/ETDataset](https://github.com/zhouhaoyi/ETDataset))
comprises two hourly-level datasets (ETTh) and two 15-minute-level datasets (ETTm). Each dataset contains seven oil and load features of electricity transformers from July $2016$ to July $2018$.
- 2.
Exchange (Wu et al., ) (^3^33[https://github.com/thuml/iTransformer](https://github.com/thuml/iTransformer)) collects the panel data of daily exchange rates from 8 countries from 1990 to 2016.
- 3.
Traffic (Wu et al., ) (^4^44[http://pems.dot.ca.gov](http://pems.dot.ca.gov))
describes the road occupancy rates. It contains the hourly data recorded by the sensors of San Francisco freeways from $2015$ to $2016$.
- 4.
Electricity (Wu et al., ) (^5^55[https://archive.ics.uci.edu/ml/datasets/ElectricityLoadDiagrams20112014](https://archive.ics.uci.edu/ml/datasets/ElectricityLoadDiagrams20112014))
collects the hourly electricity consumption of $321$ clients from $2012$ to $2014$.
- 5.
Weather (Wu et al., ) (^6^66[https://www.bgc-jena.mpg.de/wetter/](https://www.bgc-jena.mpg.de/wetter/))
includes $21$ indicators of weather, such as air temperature, and humidity. Its data is recorded every 10 min for $2020$ in Germany.
- 6.
Solar-Energy Lai et al. () (^7^77[https://github.com/thuml/iTransformer](https://github.com/thuml/iTransformer)) records the solar power production of 137 PV plants in 2006, which is sampled every 10 minutes.
- 7.
PEMS (LIU et al., ) (^8^88[https://pems.dot.ca.gov/](https://pems.dot.ca.gov/)) contains public traffic network data in California collected by 5-minute windows.

We follow the same data processing and train-validation-test set split protocol used in iTransformer (Liu et al., ), where the train, validation, and test datasets are strictly divided according to chronological order to make sure there are no data leakage issues. We fix the length of the lookback series as $T=96$ for all datasets, and the prediction length $F\in\{12,24,48,96\}$ for PEMS datasets, and $F\in\{96,192,336,720\}$ for others. Other details of these datasets is concluded in Table 6.

## Appendix B Implement Details

### B.1 Experiment Details

To foster reproducibility, we make our code, training scripts, and some visualization examples available in this GitHub Repository(^9^99[https://github.com/Levi-Ackman/LiNo](https://github.com/Levi-Ackman/LiNo)). All the experiments are conducted on a single NVIDIA GeForce RTX 4090 with 24G VRAM. The mean squared error (MSE) loss function is utilized for model optimization. We use the ADAM optimizer with an early stop parameter $patience=6$. We explore the number of LiNo blocks $N\in\{1,2,3,4\}$, dropout ratio $dp\in\{0.0,0.2,0.5\}$, and the dimension of layers $dim\in\{256,512\}$. The $learning\ rate\in\{1e-3,1e-4,1e-5\}$ and $batch\ size\in\{32,64,128,256\}$ are adjusted based on the size and dimensionality of the dataset, as well as the specific conditions of our experimental setup. All the compared multivariate forecasting baseline models that we reproduced are implemented based on the benchmark of Time series Lab (Wang et al., ) Repository, which is fairly built on the configurations provided by each model’s original paper or official code.

Performance comparison among different methods is conducted based on two primary evaluation metrics: Mean Squared Error (MSE) and Mean Absolute Error (MAE). The formula is below:
Mean Squared Error (MSE):

$$ $\text{MSE}=\frac{1}{F}\sum_{i=1}^{F}(\mathbf{Y}_{i}-\hat{\mathbf{Y}}_{i})^{2}.$ (5) $$

Mean Absolute Error (MAE):

$$ $\text{MAE}=\frac{1}{F}\sum_{i=1}^{F}|\mathbf{Y}_{i}-\hat{\mathbf{Y}}_{i}|.$ (6) $$

where $\mathbf{Y},\hat{\mathbf{Y}}\in\mathbb{R}^{F\times C}$ are the ground truth and prediction results of the future with $F$ time points and $C$ channels. $\mathbf{Y}_{i}$ denotes the $i$-th future time point.

## Appendix C Further Model Analysis

### C.1 Model Robustness

**Table 7: Error Bar ($Mean\pm Std$) of LiNo’s multivariate forecasting result on ETTh2, ETTm2, Weather, PEMS04, PEMS08. We set lookback window $T=96$, and prediction length $F\in\{12,24,48,96\}$ for PEMS04 and PEMS08, and $F\in\{96,192,336,720\}$ for others. Mean and standard deviation were obtained on 5 runs with different random seeds.**
| Models | ETTh2 | ETTm2 | Weather | PEMS04 | PEMS08 |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |
| 96/12 | $0.293\pm 0.0017$ | $0.341\pm 0.0012$ | $0.172\pm 0.0004$ | $0.254\pm 0.0005$ | $0.156\pm 0.0013$ | $0.201\pm 0.0013$ | $0.069\pm 0.0002$ | $0.168\pm 0.0009$ | $0.071\pm 0.0004$ | $0.169\pm 0.0026$ |
| 192/24 | $0.377\pm 0.0015$ | $0.392\pm 0.0008$ | $0.238\pm 0.0006$ | $0.298\pm 0.0005$ | $0.206\pm 0.0021$ | $0.248\pm 0.0019$ | $0.081\pm 0.0015$ | $0.186\pm 0.0031$ | $0.094\pm 0.0011$ | $0.191\pm 0.0027$ |
| 336/48 | $0.417\pm 0.0009$ | $0.426\pm 0.0005$ | $0.297\pm 0.0005$ | $0.336\pm 0.0004$ | $0.265\pm 0.0020$ | $0.291\pm 0.0017$ | $0.104\pm 0.0027$ | $0.214\pm 0.0036$ | $0.139\pm 0.0036$ | $0.227\pm 0.0064$ |
| 720/96 | $0.422\pm 0.0007$ | $0.440\pm 0.0007$ | $0.395\pm 0.0011$ | $0.393\pm 0.0009$ | $0.343\pm 0.0016$ | $0.342\pm 0.0012$ | $0.139\pm 0.0021$ | $0.247\pm 0.0031$ | $0.254\pm 0.0079$ | $0.296\pm 0.0091$ |

We provide LiNo’s Error Bar ($Mean\pm Std$) on several representative datasets in Table 7. LiNo demonstrated a relatively lower Error Bar across results with different random seeds, indicating consistent performance, high stability, and considerable generalization ability.

### C.2 Model Efficiency

**Table 8: Model efficiency analysis. We evaluated the parameter count, and the inference time (average of 5 runs on a single NVIDIA 4090 24GB GPU) with $batch_size=1$ on ETTh1 and ECL dataset. We set the dimension of layer $dim\in\{256,512\}$, and the number of network layers $N=2$. The task is input-96-forecast-720. * means ‘former.’ Para means ‘Parameter count(M).’ Time means ‘inference time(ms).’**
| Datasets/Models | dim | LiNo | PatchTST | Cross* | iTrans* | FED* |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Param | Time | Para | Time | Para | Time | Para | Time | Para | Time |  |  |
| ETTh1 | 256 | 1.59 | 143.81 | 3.27 | 251.00 | 8.19 | 399.00 | 1.27 | 177.67 | 3.43 | 303.556 |
| 512 | 4.82 | 145.68 | 8.64 | 266.66 | 32.11 | 445.74 | 4.63 | 190.92 | 13.68 | 345.736 |  |
| Electricity | 256 | 1.75 | 151.33 | 3.27 | 322.53 | 13.66 | 432.40 | 1.27 | 192.12 | 4.24 | 347.634 |
| 512 | 5.14 | 152.24 | 8.64 | 411.96 | 43.04 | 507.54 | 4.63 | 249.60 | 15.29 | 398.599 |  |

We evaluated the parameter count, and the inference time of four cutting-edge transformer-based multivariate time series forecasting models: iTransformer, Crossformer, PatchTST, and FEDformer. Results can be found in Table 8. Although LiNo introduces slightly more trainable parameters compared to iTransformer well-known for its simple and efficient design, its inference speed and prediction performance are significantly superior to iTransformer.

## Appendix D Visualization

### D.1 Visualization of LiNo’s Forecasting Results

To provide insights that help readers better understand the working mechanism of LiNo and to intuitively grasp the effects of advanced ‘Recursive Residual Decomposition’ (RRD), we present visualizations of LiNo’s forecasting results (Last channel/variate) across 13 multivariate forecasting benchmarks. We set the input length $T=96$, prediction length $F=96$, and number of LiNo blocks (layers) $N=3$. ‘LP’ denotes Linear prediction, and ‘NP’ stands for Nonlinear prediction. LP $i$ or NP $i$ ($i\in\{1,2,3\}$) is the linear or nonlinear prediction of $i$-th layer (level). Results are in Figure 6– 18.

### D.2 Visualization of Weight of Li blocks and No blocks

We present visualizations of the weight of Li blocks and No blocks obtained across 13 multivariate forecasting benchmarks to help better understand the proposed LiNo. We set the input length $T=96$, prediction length $F=96$, and number of LiNo blocks (layers) $N=3$. Results are in Figure 19– 31.

The method used for getting the weight follows the approach outlined in Analysis of linear model (Toner & Darlow, ). Plotting the learned matrices as in Figure 19 requires us to first convert each trained model into the form $f(\vec{x})=A\vec{x}+\vec{b}$. To do this we note that $f(\vec{0})=A\vec{0}+\vec{b}=\vec{b}$. Thus, the bias can be found by passing the zero vector into the trained model. We can determine $A$ in a similar manner. Let $\vec{e_{i}}$ denote the $i$-th coordinate vector, that is $\vec{e_{i}}$ is the vector which is 1 at position $i$ and zero elsewhere. Then $f(\vec{e_{i}})=A\vec{e_{i}}+\vec{b}=A_{\cdot,i}+\vec{b}$ where $A_{\cdot,i}$ is the $i$-th column of $A$. Hence, given that we have already computed the bias term, we may derive $A$ simply by passing through each coordinate vector $\vec{e_{i}}$ and subtracting $\vec{b}$. Then, we repeat this process separately to each Li block and No block to get their weight, since they each output a forecasting result.

Take the ETTh1 dataset as an example, we observe in Figure 7 that each block (Li or No block) produces significantly different weights. This indicates that each block focuses on different patterns. The three No blocks all generate weights with noticeable periodicity, while the three Li blocks’ weights are more concentrated on the most recent points in the input series. These interesting findings help us better understand how neural networks behave when extracting features from time series.

## Appendix E Full Results

### E.1 Full Results of Multivariate Forecasting Benchmark

The full multivariate forecasting results are provided in Table 9 and Table 10 due to the space limitation of the main text. The proposed model achieves comprehensive state-of-the-art in real-world multivariate time series forecasting applications.

### E.2 Full Results of Univariate Forecasting Benchmark

Table 11 provides the full univariate forecasting results to save space in the main text. LiNo surpasses previous state-of-the-art MICN (Wang et al., ) by a large, earning its prominent place in univariate time series forecasting tasks.

Figure: Figure 6: Visualization of LiNo’s multivariate forecasting result on ECL dataset. ‘LP’ denotes Linear prediction, and ‘NP’ stands for Nonlinear prediction. LP $i$ or NP $i$ ($i\in\{1,2,3\}$) is the linear or nonlinear prediction of $i$-th layer (level). Same to followed figures.
Refer to caption: x6.png

Figure: Figure 7: Visualization of LiNo’s multivariate forecasting result on ETTh1 dataset.
Refer to caption: x7.png

Figure: Figure 8: Visualization of LiNo’s multivariate forecasting result on ETTh2 dataset.
Refer to caption: x8.png

Figure: Figure 9: Visualization of LiNo’s multivariate forecasting result on ETTm1 dataset.
Refer to caption: x9.png

Figure: Figure 10: Visualization of LiNo’s multivariate forecasting result on ETTm2 dataset.
Refer to caption: x10.png

Figure: Figure 11: Visualization of LiNo’s multivariate forecasting result on Exchange dataset.
Refer to caption: x11.png

Figure: Figure 12: Visualization of LiNo’s multivariate forecasting result on PEMS03 dataset.
Refer to caption: x12.png

Figure: Figure 13: Visualization of LiNo’s multivariate forecasting result on PEMS04 dataset.
Refer to caption: x13.png

Figure: Figure 14: Visualization of LiNo’s multivariate forecasting result on PEMS07 dataset.
Refer to caption: x14.png

Figure: Figure 15: Visualization of LiNo’s multivariate forecasting result on PEMS08 dataset.
Refer to caption: x15.png

Figure: Figure 16: Visualization of LiNo’s multivariate forecasting result on Solar dataset.
Refer to caption: x16.png

Figure: Figure 17: Visualization of LiNo’s multivariate forecasting result on Traffic dataset.
Refer to caption: x17.png

Figure: Figure 18: Visualization of LiNo’s multivariate forecasting result on Weather dataset.
Refer to caption: x18.png

Figure: Figure 19: Visualization of LiNo’s weight on ECL dataset.
Refer to caption: x19.png

Figure: Figure 20: Visualization of LiNo’s weight on ETTh1 dataset.
Refer to caption: x20.png

Figure: Figure 21: Visualization of LiNo’s weight on ETTh2 dataset.
Refer to caption: x21.png

Figure: Figure 22: Visualization of LiNo’s weight on ETTm1 dataset.
Refer to caption: x22.png

Figure: Figure 23: Visualization of LiNo’s weight on ETTm2 dataset.
Refer to caption: x23.png

Figure: Figure 24: Visualization of LiNo’s weight on Exchange dataset.
Refer to caption: x24.png

Figure: Figure 25: Visualization of LiNo’s weight on PEMS03 dataset.
Refer to caption: x25.png

Figure: Figure 26: Visualization of LiNo’s weight on PEMS04 dataset.
Refer to caption: x26.png

Figure: Figure 27: Visualization of LiNo’s weight on PEMS07 dataset.
Refer to caption: x27.png

Figure: Figure 28: Visualization of LiNo’s weight on PEMS08 dataset.
Refer to caption: x28.png

Figure: Figure 29: Visualization of LiNo’s weight on Solar dataset.
Refer to caption: x29.png

Figure: Figure 30: Visualization of LiNo’s weight on Traffic dataset.
Refer to caption: x30.png

Figure: Figure 31: Visualization of LiNo’s weight on Weather dataset.
Refer to caption: x31.png

**Table 9: Full results of the long-term forecasting task. The input sequence length is set to $T=96$ for all baselines. *Avg* means the average results from all four prediction lengths.**
| Models | LiNo | iTransformer | Rlinear | TSMixer | PatchTST | Crossformer | TiDE | TimesNet | DLinear | FEDformer | Autoformer |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| (Ours) | () | () | () | () | () | () | () | () | () | () |  |  |  |  |  |  |  |  |  |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |  |
| ETTm1 | 96 | 0.322 | 0.361 | 0.334 | 0.368 | 0.355 | 0.376 | 0.323 | 0.363 | 0.329 | 0.367 | 0.404 | 0.426 | 0.364 | 0.387 | 0.338 | 0.375 | 0.345 | 0.372 | 0.379 | 0.419 | 0.505 | 0.475 |
| 192 | 0.365 | 0.383 | 0.377 | 0.391 | 0.391 | 0.392 | 0.376 | 0.392 | 0.367 | 0.385 | 0.450 | 0.451 | 0.398 | 0.404 | 0.374 | 0.387 | 0.380 | 0.389 | 0.426 | 0.441 | 0.553 | 0.496 |  |
| 336 | 0.401 | 0.408 | 0.426 | 0.420 | 0.424 | 0.415 | 0.407 | 0.413 | 0.399 | 0.410 | 0.532 | 0.515 | 0.428 | 0.425 | 0.410 | 0.411 | 0.413 | 0.413 | 0.445 | 0.459 | 0.621 | 0.537 |  |
| 720 | 0.469 | 0.447 | 0.491 | 0.459 | 0.487 | 0.450 | 0.485 | 0.459 | 0.454 | 0.439 | 0.666 | 0.589 | 0.487 | 0.461 | 0.478 | 0.450 | 0.474 | 0.453 | 0.543 | 0.490 | 0.671 | 0.561 |  |
| Avg | 0.389 | 0.400 | 0.407 | 0.410 | 0.414 | 0.407 | 0.398 | 0.407 | 0.387 | 0.400 | 0.513 | 0.496 | 0.419 | 0.419 | 0.400 | 0.406 | 0.403 | 0.407 | 0.448 | 0.452 | 0.588 | 0.517 |  |
| ETTm2 | 96 | 0.171 | 0.254 | 0.180 | 0.264 | 0.182 | 0.265 | 0.182 | 0.266 | 0.175 | 0.259 | 0.287 | 0.366 | 0.207 | 0.305 | 0.187 | 0.267 | 0.193 | 0.292 | 0.203 | 0.287 | 0.255 | 0.339 |
| 192 | 0.237 | 0.298 | 0.250 | 0.309 | 0.246 | 0.304 | 0.249 | 0.309 | 0.241 | 0.302 | 0.414 | 0.492 | 0.290 | 0.364 | 0.249 | 0.309 | 0.284 | 0.362 | 0.269 | 0.328 | 0.281 | 0.340 |  |
| 336 | 0.296 | 0.336 | 0.311 | 0.348 | 0.307 | 0.342 | 0.309 | 0.347 | 0.305 | 0.343 | 0.597 | 0.542 | 0.377 | 0.422 | 0.321 | 0.351 | 0.369 | 0.427 | 0.325 | 0.366 | 0.339 | 0.372 |  |
| 720 | 0.395 | 0.393 | 0.412 | 0.407 | 0.407 | 0.398 | 0.416 | 0.408 | 0.402 | 0.400 | 1.730 | 1.042 | 0.588 | 0.524 | 0.408 | 0.403 | 0.554 | 0.522 | 0.421 | 0.415 | 0.433 | 0.432 |  |
| Avg | 0.275 | 0.320 | 0.288 | 0.332 | 0.286 | 0.327 | 0.289 | 0.333 | 0.281 | 0.326 | 0.757 | 0.610 | 0.358 | 0.404 | 0.291 | 0.333 | 0.350 | 0.401 | 0.305 | 0.349 | 0.327 | 0.371 |  |
| ETTh1 | 96 | 0.378 | 0.395 | 0.386 | 0.405 | 0.386 | 0.395 | 0.401 | 0.412 | 0.414 | 0.419 | 0.423 | 0.448 | 0.479 | 0.464 | 0.384 | 0.402 | 0.386 | 0.400 | 0.376 | 0.419 | 0.449 | 0.459 |
| 192 | 0.423 | 0.423 | 0.441 | 0.436 | 0.437 | 0.424 | 0.452 | 0.442 | 0.460 | 0.445 | 0.471 | 0.474 | 0.525 | 0.492 | 0.436 | 0.429 | 0.437 | 0.432 | 0.420 | 0.448 | 0.500 | 0.482 |  |
| 336 | 0.455 | 0.438 | 0.487 | 0.458 | 0.479 | 0.446 | 0.492 | 0.463 | 0.501 | 0.466 | 0.570 | 0.546 | 0.565 | 0.515 | 0.491 | 0.469 | 0.481 | 0.459 | 0.459 | 0.465 | 0.521 | 0.496 |  |
| 720 | 0.459 | 0.456 | 0.503 | 0.491 | 0.481 | 0.470 | 0.507 | 0.490 | 0.500 | 0.488 | 0.653 | 0.621 | 0.594 | 0.558 | 0.521 | 0.500 | 0.519 | 0.516 | 0.506 | 0.507 | 0.514 | 0.512 |  |
| Avg | 0.429 | 0.428 | 0.454 | 0.447 | 0.446 | 0.434 | 0.463 | 0.452 | 0.469 | 0.454 | 0.529 | 0.522 | 0.541 | 0.507 | 0.458 | 0.450 | 0.456 | 0.452 | 0.440 | 0.460 | 0.496 | 0.487 |  |
| ETTh2 | 96 | 0.292 | 0.340 | 0.297 | 0.349 | 0.288 | 0.338 | 0.319 | 0.361 | 0.302 | 0.348 | 0.745 | 0.584 | 0.400 | 0.440 | 0.340 | 0.374 | 0.333 | 0.387 | 0.358 | 0.397 | 0.346 | 0.388 |
| 192 | 0.375 | 0.391 | 0.380 | 0.400 | 0.374 | 0.390 | 0.402 | 0.410 | 0.388 | 0.400 | 0.877 | 0.656 | 0.528 | 0.509 | 0.402 | 0.414 | 0.477 | 0.476 | 0.429 | 0.439 | 0.456 | 0.452 |  |
| 336 | 0.418 | 0.426 | 0.428 | 0.432 | 0.415 | 0.426 | 0.444 | 0.446 | 0.426 | 0.433 | 1.043 | 0.731 | 0.643 | 0.571 | 0.452 | 0.452 | 0.594 | 0.541 | 0.496 | 0.487 | 0.482 | 0.486 |  |
| 720 | 0.422 | 0.441 | 0.427 | 0.445 | 0.420 | 0.440 | 0.441 | 0.450 | 0.431 | 0.446 | 1.104 | 0.763 | 0.874 | 0.679 | 0.462 | 0.468 | 0.831 | 0.657 | 0.463 | 0.474 | 0.515 | 0.511 |  |
| Avg | 0.377 | 0.400 | 0.383 | 0.407 | 0.374 | 0.398 | 0.401 | 0.417 | 0.387 | 0.407 | 0.942 | 0.684 | 0.611 | 0.550 | 0.414 | 0.427 | 0.559 | 0.515 | 0.437 | 0.449 | 0.450 | 0.459 |  |
| ECL | 96 | 0.138 | 0.233 | 0.148 | 0.240 | 0.201 | 0.281 | 0.157 | 0.260 | 0.181 | 0.270 | 0.219 | 0.314 | 0.237 | 0.329 | 0.168 | 0.272 | 0.197 | 0.282 | 0.193 | 0.308 | 0.201 | 0.317 |
| 192 | 0.155 | 0.250 | 0.162 | 0.253 | 0.201 | 0.283 | 0.173 | 0.274 | 0.188 | 0.274 | 0.231 | 0.322 | 0.236 | 0.330 | 0.184 | 0.289 | 0.196 | 0.285 | 0.201 | 0.315 | 0.222 | 0.334 |  |
| 336 | 0.171 | 0.267 | 0.178 | 0.269 | 0.215 | 0.298 | 0.192 | 0.295 | 0.204 | 0.293 | 0.246 | 0.337 | 0.249 | 0.344 | 0.198 | 0.300 | 0.209 | 0.301 | 0.214 | 0.329 | 0.231 | 0.338 |  |
| 720 | 0.191 | 0.290 | 0.225 | 0.317 | 0.257 | 0.331 | 0.223 | 0.318 | 0.246 | 0.324 | 0.280 | 0.363 | 0.284 | 0.373 | 0.220 | 0.320 | 0.245 | 0.333 | 0.246 | 0.355 | 0.254 | 0.361 |  |
| Avg | 0.164 | 0.260 | 0.178 | 0.270 | 0.219 | 0.298 | 0.186 | 0.287 | 0.205 | 0.290 | 0.244 | 0.334 | 0.251 | 0.344 | 0.192 | 0.295 | 0.212 | 0.300 | 0.214 | 0.327 | 0.227 | 0.338 |  |
| Exchange | 96 | 0.084 | 0.203 | 0.086 | 0.206 | 0.093 | 0.217 | 0.089 | 0.211 | 0.088 | 0.205 | 0.256 | 0.367 | 0.094 | 0.218 | 0.107 | 0.234 | 0.088 | 0.218 | 0.148 | 0.278 | 0.197 | 0.323 |
| 192 | 0.176 | 0.298 | 0.177 | 0.299 | 0.184 | 0.307 | 0.177 | 0.302 | 0.176 | 0.299 | 0.470 | 0.509 | 0.184 | 0.307 | 0.226 | 0.344 | 0.176 | 0.315 | 0.271 | 0.315 | 0.300 | 0.369 |  |
| 336 | 0.316 | 0.409 | 0.331 | 0.417 | 0.351 | 0.432 | 0.327 | 0.415 | 0.301 | 0.397 | 1.268 | 0.883 | 0.349 | 0.431 | 0.367 | 0.448 | 0.313 | 0.427 | 0.460 | 0.427 | 0.509 | 0.524 |  |
| 720 | 0.823 | 0.682 | 0.847 | 0.691 | 0.886 | 0.714 | 0.912 | 0.727 | 0.901 | 0.714 | 1.767 | 1.068 | 0.852 | 0.698 | 0.964 | 0.746 | 0.839 | 0.695 | 1.195 | 0.695 | 1.447 | 0.941 |  |
| Avg | 0.350 | 0.398 | 0.360 | 0.403 | 0.378 | 0.417 | 0.376 | 0.414 | 0.367 | 0.404 | 0.940 | 0.707 | 0.370 | 0.413 | 0.416 | 0.443 | 0.354 | 0.414 | 0.519 | 0.429 | 0.613 | 0.539 |  |
| Traffic | 96 | 0.429 | 0.276 | 0.395 | 0.268 | 0.649 | 0.389 | 0.493 | 0.336 | 0.462 | 0.295 | 0.522 | 0.290 | 0.805 | 0.493 | 0.593 | 0.321 | 0.650 | 0.396 | 0.587 | 0.366 | 0.613 | 0.388 |
| 192 | 0.450 | 0.289 | 0.417 | 0.276 | 0.601 | 0.366 | 0.497 | 0.351 | 0.466 | 0.296 | 0.530 | 0.293 | 0.756 | 0.474 | 0.617 | 0.336 | 0.598 | 0.370 | 0.604 | 0.373 | 0.616 | 0.382 |  |
| 336 | 0.468 | 0.297 | 0.433 | 0.283 | 0.609 | 0.369 | 0.528 | 0.361 | 0.482 | 0.304 | 0.558 | 0.305 | 0.762 | 0.477 | 0.629 | 0.336 | 0.605 | 0.373 | 0.621 | 0.383 | 0.622 | 0.337 |  |
| 720 | 0.514 | 0.320 | 0.467 | 0.302 | 0.647 | 0.387 | 0.569 | 0.380 | 0.514 | 0.322 | 0.589 | 0.328 | 0.719 | 0.449 | 0.640 | 0.350 | 0.645 | 0.394 | 0.626 | 0.382 | 0.660 | 0.408 |  |
| Avg | 0.465 | 0.296 | 0.428 | 0.282 | 0.626 | 0.378 | 0.522 | 0.357 | 0.481 | 0.304 | 0.550 | 0.304 | 0.760 | 0.473 | 0.620 | 0.336 | 0.625 | 0.383 | 0.610 | 0.376 | 0.628 | 0.379 |  |
| Weather | 96 | 0.154 | 0.199 | 0.174 | 0.214 | 0.192 | 0.232 | 0.166 | 0.210 | 0.177 | 0.218 | 0.158 | 0.230 | 0.202 | 0.261 | 0.172 | 0.220 | 0.196 | 0.255 | 0.217 | 0.296 | 0.266 | 0.336 |
| 192 | 0.205 | 0.248 | 0.221 | 0.254 | 0.240 | 0.271 | 0.215 | 0.256 | 0.225 | 0.259 | 0.206 | 0.277 | 0.242 | 0.298 | 0.219 | 0.261 | 0.237 | 0.296 | 0.276 | 0.336 | 0.307 | 0.367 |  |
| 336 | 0.262 | 0.290 | 0.278 | 0.296 | 0.292 | 0.307 | 0.287 | 0.300 | 0.278 | 0.297 | 0.272 | 0.335 | 0.287 | 0.335 | 0.280 | 0.306 | 0.283 | 0.335 | 0.339 | 0.380 | 0.359 | 0.395 |  |
| 720 | 0.343 | 0.342 | 0.358 | 0.347 | 0.364 | 0.353 | 0.355 | 0.348 | 0.354 | 0.348 | 0.398 | 0.418 | 0.351 | 0.386 | 0.365 | 0.359 | 0.345 | 0.381 | 0.403 | 0.428 | 0.419 | 0.428 |  |
| Avg | 0.241 | 0.270 | 0.258 | 0.278 | 0.272 | 0.291 | 0.256 | 0.279 | 0.259 | 0.281 | 0.259 | 0.315 | 0.271 | 0.320 | 0.259 | 0.287 | 0.265 | 0.317 | 0.309 | 0.360 | 0.338 | 0.382 |  |
| Solar-Energy | 96 | 0.200 | 0.250 | 0.203 | 0.237 | 0.322 | 0.339 | 0.221 | 0.275 | 0.234 | 0.286 | 0.310 | 0.331 | 0.312 | 0.399 | 0.250 | 0.292 | 0.290 | 0.378 | 0.242 | 0.342 | 0.884 | 0.711 |
| 192 | 0.225 | 0.265 | 0.233 | 0.261 | 0.359 | 0.356 | 0.268 | 0.306 | 0.267 | 0.310 | 0.734 | 0.725 | 0.339 | 0.416 | 0.296 | 0.318 | 0.320 | 0.398 | 0.285 | 0.380 | 0.834 | 0.692 |  |
| 336 | 0.243 | 0.283 | 0.248 | 0.273 | 0.397 | 0.369 | 0.272 | 0.294 | 0.290 | 0.315 | 0.750 | 0.735 | 0.368 | 0.430 | 0.319 | 0.330 | 0.353 | 0.415 | 0.282 | 0.376 | 0.941 | 0.723 |  |
| 720 | 0.250 | 0.283 | 0.249 | 0.275 | 0.397 | 0.356 | 0.281 | 0.313 | 0.289 | 0.317 | 0.769 | 0.765 | 0.370 | 0.425 | 0.338 | 0.337 | 0.356 | 0.413 | 0.357 | 0.427 | 0.882 | 0.717 |  |
| Avg | 0.230 | 0.270 | 0.233 | 0.262 | 0.369 | 0.356 | 0.260 | 0.297 | 0.270 | 0.307 | 0.641 | 0.639 | 0.347 | 0.417 | 0.301 | 0.319 | 0.330 | 0.401 | 0.291 | 0.381 | 0.885 | 0.711 |  |
| $1^{\text{st}}$ Count | 28 | 29 | 6 | 10 | 5 | 4 | 0 | 0 | 4 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 | 0 |  |

**Table 10: Full results of the PEMS forecasting task. The input length is set to $T=96$ for all baselines. *Avg* means the average results from all four prediction lengths.**
| Models | LiNo | iTransformer | Rlinear | TSMixer | PatchTST | Crossformer | TiDE | TimesNet | DLinear | FEDformer | Autoformer |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| (Ours) | () | () | () | () | () | () | () | () | () | () |  |  |  |  |  |  |  |  |  |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |  |
| PEMS03 | 12 | 0.061 | 0.163 | 0.071 | 0.174 | 0.126 | 0.236 | 0.075 | 0.186 | 0.099 | 0.216 | 0.090 | 0.203 | 0.178 | 0.305 | 0.085 | 0.192 | 0.122 | 0.243 | 0.126 | 0.251 | 0.272 | 0.385 |
| 24 | 0.077 | 0.181 | 0.093 | 0.201 | 0.246 | 0.334 | 0.095 | 0.210 | 0.142 | 0.259 | 0.121 | 0.240 | 0.257 | 0.371 | 0.118 | 0.223 | 0.201 | 0.317 | 0.149 | 0.275 | 0.334 | 0.440 |  |
| 48 | 0.113 | 0.217 | 0.125 | 0.236 | 0.551 | 0.529 | 0.121 | 0.240 | 0.211 | 0.319 | 0.202 | 0.317 | 0.379 | 0.463 | 0.155 | 0.260 | 0.333 | 0.425 | 0.227 | 0.348 | 1.032 | 0.782 |  |
| 96 | 0.132 | 0.225 | 0.164 | 0.275 | 1.057 | 0.787 | 0.184 | 0.295 | 0.269 | 0.370 | 0.262 | 0.367 | 0.490 | 0.539 | 0.228 | 0.317 | 0.457 | 0.515 | 0.348 | 0.434 | 1.031 | 0.796 |  |
| Avg | 0.096 | 0.197 | 0.113 | 0.221 | 0.495 | 0.472 | 0.119 | 0.233 | 0.180 | 0.291 | 0.169 | 0.281 | 0.326 | 0.419 | 0.147 | 0.248 | 0.278 | 0.375 | 0.213 | 0.327 | 0.667 | 0.601 |  |
| PEMS04 | 12 | 0.069 | 0.169 | 0.078 | 0.183 | 0.138 | 0.252 | 0.079 | 0.188 | 0.105 | 0.224 | 0.098 | 0.218 | 0.219 | 0.340 | 0.087 | 0.195 | 0.148 | 0.272 | 0.138 | 0.262 | 0.424 | 0.491 |
| 24 | 0.081 | 0.184 | 0.095 | 0.205 | 0.258 | 0.348 | 0.089 | 0.201 | 0.153 | 0.275 | 0.131 | 0.256 | 0.292 | 0.398 | 0.103 | 0.215 | 0.224 | 0.340 | 0.177 | 0.293 | 0.459 | 0.509 |  |
| 48 | 0.103 | 0.212 | 0.120 | 0.233 | 0.572 | 0.544 | 0.111 | 0.222 | 0.229 | 0.339 | 0.205 | 0.326 | 0.409 | 0.478 | 0.136 | 0.250 | 0.355 | 0.437 | 0.270 | 0.368 | 0.646 | 0.610 |  |
| 96 | 0.137 | 0.247 | 0.150 | 0.262 | 1.137 | 0.820 | 0.133 | 0.247 | 0.291 | 0.389 | 0.402 | 0.457 | 0.492 | 0.532 | 0.190 | 0.303 | 0.452 | 0.504 | 0.341 | 0.427 | 0.912 | 0.748 |  |
| Avg | 0.098 | 0.203 | 0.111 | 0.221 | 0.526 | 0.491 | 0.103 | 0.215 | 0.195 | 0.307 | 0.209 | 0.314 | 0.353 | 0.437 | 0.129 | 0.241 | 0.295 | 0.388 | 0.231 | 0.337 | 0.610 | 0.590 |  |
| PEMS07 | 12 | 0.055 | 0.146 | 0.067 | 0.165 | 0.118 | 0.235 | 0.073 | 0.181 | 0.095 | 0.207 | 0.094 | 0.200 | 0.173 | 0.304 | 0.082 | 0.181 | 0.115 | 0.242 | 0.109 | 0.225 | 0.199 | 0.336 |
| 24 | 0.070 | 0.162 | 0.088 | 0.190 | 0.242 | 0.341 | 0.090 | 0.199 | 0.150 | 0.262 | 0.139 | 0.247 | 0.271 | 0.383 | 0.101 | 0.204 | 0.210 | 0.329 | 0.125 | 0.244 | 0.323 | 0.420 |  |
| 48 | 0.095 | 0.189 | 0.110 | 0.215 | 0.562 | 0.541 | 0.124 | 0.231 | 0.253 | 0.340 | 0.311 | 0.369 | 0.446 | 0.495 | 0.134 | 0.238 | 0.398 | 0.458 | 0.165 | 0.288 | 0.390 | 0.470 |  |
| 96 | 0.132 | 0.225 | 0.139 | 0.245 | 1.096 | 0.795 | 0.163 | 0.255 | 0.346 | 0.404 | 0.396 | 0.442 | 0.628 | 0.577 | 0.181 | 0.279 | 0.594 | 0.533 | 0.262 | 0.376 | 0.554 | 0.578 |  |
| Avg | 0.088 | 0.181 | 0.101 | 0.204 | 0.504 | 0.478 | 0.112 | 0.217 | 0.211 | 0.303 | 0.235 | 0.315 | 0.380 | 0.440 | 0.124 | 0.225 | 0.329 | 0.395 | 0.165 | 0.283 | 0.367 | 0.451 |  |
| PEMS08 | 12 | 0.070 | 0.166 | 0.079 | 0.182 | 0.133 | 0.247 | 0.083 | 0.189 | 0.168 | 0.232 | 0.165 | 0.214 | 0.227 | 0.343 | 0.112 | 0.212 | 0.154 | 0.276 | 0.173 | 0.273 | 0.436 | 0.485 |
| 24 | 0.093 | 0.190 | 0.115 | 0.219 | 0.249 | 0.343 | 0.117 | 0.226 | 0.224 | 0.281 | 0.215 | 0.260 | 0.318 | 0.409 | 0.141 | 0.238 | 0.248 | 0.353 | 0.210 | 0.301 | 0.467 | 0.502 |  |
| 48 | 0.140 | 0.227 | 0.186 | 0.235 | 0.596 | 0.544 | 0.196 | 0.299 | 0.321 | 0.354 | 0.315 | 0.355 | 0.497 | 0.510 | 0.198 | 0.283 | 0.440 | 0.470 | 0.320 | 0.394 | 0.966 | 0.733 |  |
| 96 | 0.247 | 0.283 | 0.221 | 0.267 | 1.166 | 0.814 | 0.266 | 0.331 | 0.408 | 0.417 | 0.377 | 0.397 | 0.721 | 0.592 | 0.320 | 0.351 | 0.674 | 0.565 | 0.442 | 0.465 | 1.385 | 0.915 |  |
| Avg | 0.138 | 0.217 | 0.150 | 0.226 | 0.529 | 0.487 | 0.165 | 0.261 | 0.280 | 0.321 | 0.268 | 0.307 | 0.441 | 0.464 | 0.193 | 0.271 | 0.379 | 0.416 | 0.286 | 0.358 | 0.814 | 0.659 |  |
| $1^{\text{st}}$ Count | 18 | 19 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |  |

**Table 11: Full univariate forecasting results with prediction lengths $F\in\{96,192,336,720\}$ and fixed lookback winodw $T=96$ for all datasets.**
| Models | LiNo | MICN | FEDformer | Autoformer | Informer | LogTrans |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | (Ours) | () | () | () | () | () |  |  |  |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |  |
| ETTm1 | 96 | 0.029 | 0.126 | 0.033 | 0.134 | 0.033 | 0.140 | 0.056 | 0.183 | 0.109 | 0.277 | 0.049 | 0.171 |
| 192 | 0.044 | 0.160 | 0.048 | 0.164 | 0.058 | 0.186 | 0.081 | 0.216 | 0.151 | 0.310 | 0.157 | 0.317 |  |
| 336 | 0.058 | 0.185 | 0.079 | 0.210 | 0.084 | 0.231 | 0.076 | 0.218 | 0.427 | 0.591 | 0.289 | 0.459 |  |
| 720 | 0.081 | 0.217 | 0.096 | 0.233 | 0.102 | 0.250 | 0.110 | 0.267 | 0.438 | 0.586 | 0.430 | 0.579 |  |
| Avg | 0.053 | 0.172 | 0.064 | 0.185 | 0.069 | 0.202 | 0.081 | 0.221 | 0.281 | 0.441 | 0.231 | 0.382 |  |
| ETTm2 | 96 | 0.066 | 0.185 | 0.059 | 0.176 | 0.067 | 0.198 | 0.065 | 0.189 | 0.088 | 0.225 | 0.075 | 0.208 |
| 192 | 0.100 | 0.235 | 0.100 | 0.234 | 0.102 | 0.245 | 0.118 | 0.256 | 0.132 | 0.283 | 0.129 | 0.275 |  |
| 336 | 0.130 | 0.273 | 0.153 | 0.301 | 0.130 | 0.279 | 0.154 | 0.305 | 0.180 | 0.336 | 0.154 | 0.302 |  |
| 720 | 0.176 | 0.328 | 0.210 | 0.354 | 0.178 | 0.325 | 0.182 | 0.335 | 0.300 | 0.435 | 0.160 | 0.321 |  |
| Avg | 0.118 | 0.255 | 0.131 | 0.266 | 0.119 | 0.262 | 0.130 | 0.271 | 0.175 | 0.320 | 0.130 | 0.277 |  |
| ETTh1 | 96 | 0.056 | 0.180 | 0.058 | 0.186 | 0.079 | 0.215 | 0.071 | 0.206 | 0.193 | 0.377 | 0.283 | 0.468 |
| 192 | 0.071 | 0.203 | 0.079 | 0.210 | 0.104 | 0.245 | 0.114 | 0.262 | 0.217 | 0.395 | 0.234 | 0.409 |  |
| 336 | 0.085 | 0.228 | 0.092 | 0.237 | 0.119 | 0.270 | 0.107 | 0.258 | 0.202 | 0.381 | 0.386 | 0.546 |  |
| 720 | 0.082 | 0.226 | 0.138 | 0.298 | 0.142 | 0.299 | 0.126 | 0.283 | 0.183 | 0.355 | 0.475 | 0.628 |  |
| Avg | 0.074 | 0.209 | 0.092 | 0.233 | 0.111 | 0.257 | 0.105 | 0.252 | 0.199 | 0.377 | 0.345 | 0.513 |  |
| ETTh2 | 96 | 0.127 | 0.273 | 0.155 | 0.300 | 0.128 | 0.271 | 0.153 | 0.306 | 0.213 | 0.373 | 0.217 | 0.379 |
| 192 | 0.176 | 0.326 | 0.169 | 0.316 | 0.185 | 0.330 | 0.204 | 0.351 | 0.227 | 0.387 | 0.281 | 0.429 |  |
| 336 | 0.203 | 0.359 | 0.238 | 0.384 | 0.231 | 0.378 | 0.246 | 0.389 | 0.242 | 0.401 | 0.293 | 0.437 |  |
| 720 | 0.214 | 0.371 | 0.447 | 0.561 | 0.278 | 0.420 | 0.268 | 0.409 | 0.291 | 0.439 | 0.218 | 0.387 |  |
| Avg | 0.180 | 0.332 | 0.252 | 0.390 | 0.206 | 0.350 | 0.218 | 0.364 | 0.243 | 0.400 | 0.252 | 0.408 |  |
| Traffic | 96 | 0.138 | 0.214 | 0.158 | 0.241 | 0.207 | 0.312 | 0.246 | 0.346 | 0.257 | 0.353 | 0.226 | 0.317 |
| 192 | 0.134 | 0.214 | 0.154 | 0.236 | 0.205 | 0.312 | 0.266 | 0.370 | 0.299 | 0.376 | 0.314 | 0.408 |  |
| 336 | 0.142 | 0.223 | 0.165 | 0.243 | 0.219 | 0.323 | 0.263 | 0.371 | 0.312 | 0.387 | 0.387 | 0.453 |  |
| 720 | 0.160 | 0.238 | 0.182 | 0.264 | 0.244 | 0.344 | 0.269 | 0.372 | 0.366 | 0.436 | 0.491 | 0.437 |  |
| Avg | 0.143 | 0.222 | 0.165 | 0.246 | 0.219 | 0.323 | 0.261 | 0.365 | 0.309 | 0.388 | 0.355 | 0.404 |  |
| Weather | 96 | 0.0012 | 0.026 | 0.0029 | 0.039 | 0.0062 | 0.062 | 0.0110 | 0.081 | 0.0038 | 0.044 | 0.0046 | 0.052 |
| 192 | 0.0015 | 0.029 | 0.0021 | 0.034 | 0.0060 | 0.062 | 0.0075 | 0.067 | 0.0023 | 0.040 | 0.0056 | 0.060 |  |
| 336 | 0.0016 | 0.029 | 0.0023 | 0.034 | 0.0041 | 0.050 | 0.0063 | 0.062 | 0.0041 | 0.049 | 0.0060 | 0.054 |  |
| 720 | 0.0021 | 0.035 | 0.0048 | 0.054 | 0.0055 | 0.059 | 0.0085 | 0.070 | 0.0031 | 0.042 | 0.0071 | 0.063 |  |
| Avg | 0.0016 | 0.030 | 0.0030 | 0.040 | 0.0055 | 0.058 | 0.0083 | 0.070 | 0.0033 | 0.044 | 0.0058 | 0.057 |  |
| $1^{\text{st}}$ Count | 28 | 25 | 2 | 3 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 |  |

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: - What is the advantage of using frequency domain operations? How do they benefit the predictions? Is it only useful if there are cyclical seasonal patterns? Will it be useful in cases where there is no seasonality or the frequency of the seasonal pattern can shift?
- Line 239 to 243: Can you elaborate on how the softmax is applied and the weighted mean is computed? It would be helpful to include a figure showing this operation in detail.
- How is the FFT and IFFT calculated? In particular, is this operation differentiable is is back-propagated through?
- Where does H_i sit in figure 2?
- Figure 2 No Block: Why is the MLP applied to the final prediction (right before adding the prediction from the Li Block) but not when computing the residual for the next block?
- Figure 2: What does the backbone consist of?
- What is the purpose of dropout in Equation 2? Does it help with spreading out the coefficients across the whole input window rather than have it concentrated on a few time steps?

Some suggested clarifications in the text:
- Line 078: "Another problem is the design of current nonlinear models, which mainly focus on
one or two types of nonlinear patterns."
Which types of non-linear patterns are being talked about here.
- Line 090: What is "traditional RRD"?
- Line 143: Typo "closely close"
- Eq 3: did you mean T<D?
soundness: 3
strengths: Strengths:
- The author presented a novel time series forecasting method based on the idea of iteratively extracting linear and non-linear features from the residuals. While the idea has been introduced earlier in the literature (RRD), this paper extends the idea using neural networks.
- The experiments performed in this paper are extensive and the baselines used are state of the art. Comparison with several state of the art baselines on benchmark performance shows improved prediction accuracy.
- Additionally, the analysis presented in the paper including the ablation studies are extensive furthering strengthening the proposed method.
summary: This paper presents a new neural network architecture for multi-variate time series forecasting. The main idea is to iteratively extract linear and non-linear patterns from the time series which are easier to learn and forecast. The linear patterns are extracted using 1D convolutions (part of the Li Block) where as the non linear features (part of the No Block) are extracted using linear projections on the frequency (FFT) and time domain (temporal variation features). The inter time-series dependencies are modeled using a weighted mean (akin to a pooling function) where the weights are computed using a softmax function.

The LiNo blocks are iteratively applied to the residuals to build predictors for each residual. All the predictors are finally added to produce the final prediction. The authors have presented extensive experimentation showing the effectiveness of the approach yielding state of the art results on several benchmark datasets.
weaknesses: Weaknesses
- Several parts of the paper are unclear. In particular the design of the model (see questions below)
- The paper builds on the ideas of residual modeling and non-linear feature extraction using FFT, both which have been explored earlier in the literature [1, 2, 3] ([1] has not been cited), weakening the novelty of the approach slightly.

[1] Oreshkin, Boris N., et al. "N-BEATS: Neural basis expansion analysis for interpretable time series forecasting." arXiv preprint arXiv:1905.10437 (2019).

[2] Challu, Cristian, et al. "Nhits: Neural hierarchical interpolation for time series forecasting." Proceedings of the AAAI conference on artificial intelligence. Vol. 37. No. 6. 2023.

[3] Xu, Zhijian, Ailing Zeng, and Qiang Xu. "FITS: Modeling time series with $10 k $ parameters." arXiv preprint arXiv:2307.03756 (2023).

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 2
questions: Note: The most important questions are listed first.

1. What is meant by "multi-level characteristics of real-world time series" (Sec. 1)? How do the time series in Fig. 1 relate to each other? How are they combined (additively?)?
2. L. 299f states, "These models represent the latest advancements in multivariate time series forecasting and encompass all mainstream prediction model types". However, recurrent models such as SegRNN (Lin et al. 2023) and the empirically strong TimeMixer model (Wang et al. 2024) are missing. Was this a deliberate choice?
3. Regarding the Li Block (Sec. 3.2): Why is padding performed? Is the linear prediction in l. 221 the same as the (not very linear) MLP in Fig. 2? Which of the model components are learned via gradient descent?
4. Regarding Fig. 2: What is the "backbone"?
5. Regarding the No Block (Sec. 3.2): Again, MLP blocks are suddenly linear projections (matrix multiplications). If the addition in l. 238 is element-wise, isn't the overall $N_i^{TF}$  entirely linear up to the point of activation. How can it be beneficial over a simple linear layer? Why is Tanh chosen, given that ReLU is a more common choice today? I do not understand the motivation behind the construction in l. 239-250.
6. The iterative pattern extraction reminds me of N-BEATS (Oreshkin et al., 2019). Are they that closely related? If so, this might be relevant for Sec. 2.
7. What are the diagonal stripes in, for instance, Fig. 19 (bottom, left)?
soundness: 3
strengths: - I am not aware of these ideas being proposed before. However, I am not sufficiently familiar with the topic to rule it out definitively.
- The quality of the presentation ranges from poor (method section) to good (experiments).
- Time series forecasting is relevant in many domains and is far from being solved. The proposed method investigates a method that is interesting due to its interpretability and fair performance.
- The experiments go beyond mere performance metrics, performing an ablation study, other variations, depth sensitivity analysis, and lookback length investigations.
summary: - The paper identifies explicitly and iteratively extracting linear and nonlinear patterns from time series as an opportunity to learn highly effective forecasting models.
- The extracted patterns are combined via simple linear layers to form the forecast.
- The method is then evaluated in a variety of experiments.

I did not check the entire appendix.
weaknesses: 1. The method presentation in (mainly) Sec. 3.2 is poor. The notation is more complicated than necessary (e.g., it includes a batch dimension) while not being sufficiently precise to fully specify the computation (e.g., in l. 239 to 243). The accompanying Figure 2 further complicates things by its inconsistent colors, the red cutout not showing what it points at (on the left, the projections were independent), the MLPs only being linear layers, the "Cat" being a Hadamard product, arrows leading into the nothingness, and inconsistent formatting. See also the Questions section.
2. It is unclear to me why the Li block is called Linear. Going by Fig. 2 or Fig. 5, neither the pattern nor the prediction is linear in the time index, and the learned function is neither (e.g., there is a Dropout layer).
3. The learned representations are somewhat redundant. See, e.g., LP2 and LP3 in Fig. 5 (ETTh1, center) or NP1 and NP2 in Fig. 5 (ETTh1, right). These representations are not used for model interpretability.

#### Minor Comments
- The text size for the figure captions was very likely reduced from what is specified in the template.
- The reference's formatting is very sloppy (missing venues, capitalization of titles and names, broken math notation, and inconsistency).
- Section 3 could be simplified by omitting the batch dimension, as every operation only operates on a single time series. Given the deep learning context, a batch dimension can be conceptually added simply in one sentence without complicating notation.
- Language: "TSMxier" typo in l. 103. Reference in l. 118. Used citet instead of citep in l. 252. ...
- The convention of typesetting vectors and matrices in bold might ease readability (starting in Sec. 3.1).
- The images are not included as vector graphics, limiting accessibility, the ability to zoom in (cf. Fig. 3), and increasing file size.
#### References
- Oreshkin, Boris N., Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. “N-BEATS: Neural Basis Expansion Analysis for Interpretable Time Series Forecasting.” In International Conference on Learning Representations, 2019.
- Lin, Shengsheng, Weiwei Lin, Wentai Wu, Feiyu Zhao, Ruichao Mo, and Haotong Zhang. “SegRNN: Segment Recurrent Neural Network for Long-Term Time Series Forecasting.” arXiv, August 22, 2023. https://doi.org/10.48550/arXiv.2308.11200.
- Wang, Shiyu, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y. Zhang, and Jun Zhou. “TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting.” In The Twelfth International Conference on Learning Representations, 2024.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: Please see the weaknesses above.
soundness: 2
strengths: 1. In experimental evaluations, the proposed method surpasses state-of-the-art models in both univariate and multivariate forecasting scenarios.
2. The method is easy to understand.
3. This paper proposes an efficient method to capture the linear and nonlinear patterns for time series forecasting.
summary: This paper introduces LiNo, a Recursive Residual Decomposition framework for time series forecasting. LiNo addresses the limitations of existing deep learning models by extracting both linear and nonlinear patterns. The framework employs a Li block to capture linear patterns and a No block to model nonlinear patterns. This recursive and deeper-level decomposition enhances the extraction of complex, multi-granular patterns in real-world time series data. Experimental results on thirteen real-world benchmarks show that LiNo outperforms state-of-the-art models in both univariate and multivariate forecasting scenarios, providing robust and precise predictions.
weaknesses: 1. The components of the method, while effective, are not entirely novel. Recursive residual decomposition has been demonstrated to be effective in N-BEATS [1], and the utility of frequency-domain MLPs is highlighted in FreMLP [2]. Additionally, the channel mixing technique used in LiNo is similar to that used in SOFTS [3], and it would be beneficial to include a more detailed comparison and discussion of these similarities.
2. The authors argue that the primary distribution involves linear and nonlinear decomposition, with ablation studies primarily focused on multivariate time series forecasting. In line 428, they state, "Extracting nonlinear patterns, such as inter-series dependencies, temporal variations, and frequency information, is crucial for accurate predictions." The classification of inter-series dependencies as nonlinear patterns may be debatable, as methods like the Multivariate Autoregressive Model, which are inherently linear, can also capture such dependencies. It would be valuable to separately study the effects of the channel mixing method and the linear-nonlinear decomposition. The claimed superiority of the No block ablation, using iTransformer and TSMixer, which mainly attributes its performance to the channel mixing part, could benefit from further evidence (Table 5, Figure 4).
3. The paper lacks an error bar analysis. To demonstrate the robustness of the proposed method, the experiments should be conducted multiple times using different random seeds.
4. The paper contains a few minor typographical errors. For instance, on line 103, "TSMxier" should be corrected to "TSMixer," and on line 252, "ReVIN" should be changed to "RevIN."

[1] Oreshkin B N, Carpov D, Chapados N, et al. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting[J]. ICLR, 2020.

[2] Yi K, Zhang Q, Fan W, et al. Frequency-domain MLPs are more effective learners in time series forecasting[J]. NeurIPS, 2023.

[3] Lu H, XY Chen, et al. SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion. ArXiv, 2024.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: Could you elaborate on the design choices for the complex components? Beyond being a complicated module featuring recursive, residual, frequency, and linear elements, what insights can we gain from your design?
soundness: 3
strengths: The numerical performance is relative good.
summary: It's a complex time series forecasting model incorporating recursive residual, frequency, and linear/non-linear designs
weaknesses: I'm concerned that the work resembles a LEGO model: it combines various components into a high-level narrative without detailed mapping. The motivation and underlying problem are weak. The ablation should focus on the concept of Recursive Residual Decomposition instead of individual components. Removing entire modules that result in extremely poor outcomes doesn't provide meaningful insights. It would be more effective to compare it with similar residual-based designs or variations to evaluate the overarching idea.