xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories

## Abstract

Abstract Time series data is prevalent across numerous fields, necessitating the development of robust and accurate forecasting models.
Capturing patterns both within and between temporal and multivariate components is crucial for reliable predictions.
We introduce xLSTM-Mixer, a model designed to effectively integrate temporal sequences, joint time-variate information, and multiple perspectives for robust forecasting.
Our approach begins with a linear forecast shared across variates, which is then refined by xLSTM blocks.
They serve as key elements for modeling the complex dynamics of challenging time series data.
xLSTM-Mixer ultimately reconciles two distinct views to produce the final forecast.
Our extensive evaluations demonstrate its superior long-term forecasting performance compared to recent state-of-the-art methods.
A thorough model analysis provides further insights into its key components and confirms its robustness and effectiveness.
This work contributes to the resurgence of recurrent models in time series forecasting.

## 1 Introduction

Time series are an essential data modality ubiquitous in many critical fields of application, such as medicine (Hosseini et al., ), manufacturing (Essien & Giannetti, ), logistics (Seyedan & Mafakheri, ), traffic management (Lippi et al., ), finance (Lin et al., ), audio processing (Latif et al., ), and weather modeling (Lam et al., ).
While significant progress in time series forecasting has been made over the decades, the field is still far from being solved.
The regular appearance of yet better models and improved combinations of existing approaches exemplifies this.
Further increasing the forecast quality obtained from machine learning models promises a manifold of improvements, such as higher efficiency in manufacturing and transportation as well as more accurate medical treatments.

Historically, recurrent neural networks (RNNs) and their powerful successors were natural choices for deep learning-based time series forecasting (Hochreiter & Schmidhuber, ; Cho et al., ).
Today, large Transformers (Vaswani et al., ) are applied extensively to time series tasks, including forecasting.
Many improvements to the vanilla architecture have since been proposed, including patching (Nie et al., ), decompositions (Zeng et al., ), and tokenization inversions (Liu et al., ).
However, some of their limitations are yet to be lifted.
For instance, they typically require large datasets to train successfully, restricting their use to only a subset of conceivable applications.
Furthermore, they are inefficient when applied to long sequences due to the cost of the attention mechanism being quadratic in the number of variates and time steps, depending on the specific choice of tokenization.
Therefore, recurrent and state space models (SSMs) (Patro & Agneeswaran, ) are experiencing a resurgence of interest in overcoming such limitations. Specifically, Beck et al. () revisited recurrent models by borrowing insights gained from Transformers applied to many domains, specifically to natural language processing. They propose Extended Long Short-Term Memory (xLSTM) models as a viable alternative to current sequence models.

We propose xLSTM-Mixer(^1^11[https://github.com/mauricekraus/xLSTM-Mixer](https://github.com/mauricekraus/xLSTM-Mixer)), a new state-of-the-art method for time series forecasting using recurrent deep learning methods.
Specifically, we augment the highly expressive xLSTM architecture with carefully crafted time, variate, and multi-view mixing.
These operations regularize the training and limit the model parameters by weight-sharing, effectively improving the learning of features necessary for accurate forecasting.
xLSTM-Mixer initially computes a channel-independent linear forecast shared over the variates.
It is then up-projected to a higher hidden dimension and subsequently refined by an xLSTM stack.
It performs multi-view forecasting by producing a forecast from the original and reversed up-projected embedding.
The powerful xLSTM cells thereby jointly mix time and variate information to capture complex patterns from the data.
Both forecasts are eventually reconciled by a learned linear projection into the final prediction, again by mixing time.
An overview of our method is shown in Figure 1.

Figure: Figure 1: The xLSTM-Mixer architecture consists of three stages: (1) An initial NLinear forecast assuming channel independence and performing *time mixing*; (2) subsequent joint mixing, which mixes variate and time information through crucial applications of sLSTM blocks in two views; and (3) a final view mixing, where the two latent forecast views are reconciled into a coherent final forecast.
Refer to caption: x1.png

Overall, we make the following contributions:

- (i)
We investigate time and variate mixing in the context of recurrent models and propose a joint multistage approach that is highly effective for multivariate time series forecasting. We argue that marching over the variates instead of the temporal axis yields better results if suitably combined with temporal mixing.
- (ii)
We propose xLSTM-Mixer, a state-of-the-art method for time series forecasting using recurrent deep learning methods.
- (iii)
We extensively compare xLSTM-Mixer with existing methods for multivariate long-term time series forecasting and perform in-depth model analyses. The experiments demonstrate that xLSTM-Mixer consistently achieves state-of-the-art performance in a wide range of benchmarks.

The following work is structured as follows:
In the upcoming Sec. 2, we introduce preliminaries to then motivate and explain xLSTM-Mixer in Sec. 3.
We then present comprehensive experiments on its effectiveness and inner workings in Sec. 4.
We finally contextualize the findings within the related work in Sec. 5 and close with a conclusion and outlook in Sec. 6.

## 2 Background

After introducing the notation used throughout this work, we review xLSTM blocks and discuss leveraging channel mixing or their independence in time series models.

### 2.1 Notation

In multivariate time series forecasting, the model is presented with a time series $\bm{X}=\left(\bm{x}_{1},\dots,\bm{x}_{T}\right)\in\mathbb{R}^{V\times T}$ consisting of $T$ time steps with $V$ variates each.
Given this context, the forecaster shall predict the future values $\bm{Y}=\left(\bm{x}_{T+1},\dots,\bm{x}_{T+H}\right)\in\mathbb{R}^{V\times H}$ up to a horizon $H$.
A variate (also called a channel) can be any scalar measurement, such as the occupancy of a road or the oil temperature in a power plant.
The measurements are assumed to be carried out jointly, such that the $T+H$ time steps reflect a regularly sampled multivariate signal.
A time series dataset consists of $N$ such pairs $\left\{\left(\bm{X}^{(i)},\bm{Y}^{(i)}\right)\right\}_{i\in\{1,\dots,N\}}$ divided into train, validation, and test portions.

### 2.2 Extended Long Short-Term Memory (xLSTM)

Beck et al. () propose xLSTM architectures consisting of two building blocks, namely the sLSTM and mLSTM modules.
To harness the full expressivity of xLSTMs within each step and across the computation sequence, we employ a stack of sLSTM blocks without any mLSTM blocks.
The latter are less suited for joint mixing due to their independent treatment of the sequence elements, making it impossible to learn any relationships between them directly.
We will continue by recalling the construction of sLSTM cells.

The standard LSTM architecture of Hochreiter & Schmidhuber () involves updating the cell state $\mathbf{c}_{t}$ through a combination of input, forget, and output gates, which regulate the flow of information across tokens.
sLSTM blocks enhance this by incorporating exponential gating and memory mixing (Greff et al., ) to handle complex temporal and cross-variate dependencies more effectively.
The sLSTM updates the cell $\bm{c}_{t}$ and hidden state $\bm{h}_{t}$ using three gates as follows:

$$ $\displaystyle\bm{c}_{t}$ $\displaystyle=\bm{f}_{t}\odot c_{t-1}+\bm{i}_{t}\odot\bm{z}_{t}$ cell state (1) $\displaystyle\bm{n}_{t}$ $\displaystyle=\bm{f}_{t}\cdot\bm{n}_{t-1}+\bm{i}_{t}$ normalizer state (2) $\displaystyle\bm{h}_{t}$ $\displaystyle=\bm{o}_{t}\odot\bm{c}_{t}\odot\bm{n}^{-1}_{t}$ hidden state (3) $\displaystyle\bm{z}_{t}$ $\displaystyle=\tanh\bigl{(}\bm{W}_{z}\bm{x}_{t}+\bm{R}_{z}h_{t-1}+\bm{b}_{z} \bigr{)}$ cell input (4) $\displaystyle\bm{i}_{t}$ $\displaystyle=\exp\bigl{(}\bm{\tilde{i}}_{t}-\bm{m}_{t}\bigr{)}$ $\displaystyle\bm{\tilde{i}}_{t}$ $\displaystyle=\bm{W}_{i}\bm{x}_{t}+\bm{R}_{i}\bm{h}_{t-1}+\bm{b}_{i}$ input gate (5) $\displaystyle\bm{f}_{t}$ $\displaystyle=\exp\bigl{(}\bm{\tilde{f}}_{t}+\bm{m}_{t-1}-\bm{m}_{t}\bigr{)}$ $\displaystyle\bm{\tilde{f}}_{t}$ $\displaystyle=\bm{W}_{f}\bm{x}_{t}+\bm{R}_{f}\bm{h}_{t-1}+\bm{b}_{f}$ forget gate (6) $\displaystyle\bm{o}_{t}$ $\displaystyle=\sigma\bigl{(}\bm{W}_{o}\bm{x}_{t}+\bm{R}_{o}\bm{h}_{t-1}+\bm{b} _{o}\bigr{)}$ output gate (7) $\displaystyle\bm{m}_{t}$ $\displaystyle=\max\bigl{(}\bm{\tilde{f}}_{t}+\bm{m}_{t-1},\bm{\tilde{i}}_{t} \bigr{)}$ stabilizer state (8) $$

In this setup, the matrices $\bm{W}_{z},\bm{W}_{i},\bm{W}_{f},$ and $\bm{W}_{o}$ are input weights mapping the input token $\bm{x}_{t}$ to the cell input $\bm{z}_{t}$, input gate, forget gate, and output gate, respectively.
The states $\bm{n}_{t}$ and $\bm{m}_{t}$ serve as necessary normalization and training stabilization, respectively.

As have shown, it is beneficial to restrict the memory mixing performed by the recurrent weight matrices $\bm{R}_{z},\bm{R}_{i},\bm{R}_{f},$ and $\bm{R}_{o}$ to individual *heads*, inspired by the multi-head setup of Transformers (Zeng et al., ), yet more restricted and therefore more efficient to compute.
In particular, each token gets broken up into separate pieces, where the input weights $\bm{W}_{z,i,f,o}$ act across all of them, but the recurrence matrices $\bm{R}_{z,i,f,o}$ are implemented as block-diagonals and therefore only act within each piece.
This permits specialization of the individual heads to patterns specific to the respective section of the tokens and empirically does not sacrifice expressivity.

### 2.3 Channel Independence and Mixing in Time Series Models

Multiple works have investigated whether it is beneficial to learn representations of the time and variate dimensions jointly or separately.
Intuitively, because joint mixing is strictly more expressive, one might think it should always be preferred.
It is indeed used by many methods such as Temporal Convolutional Networks (TCN) (Lea et al., ), N-BEATS (Oreshkin et al., ), N-HiTS (Challu et al., ), and many Transformers (Vaswani et al., ), including Temporal Fusion Transformer (TFT) (Lim et al., ), Autoformer (Wu et al., ), and FEDFormer (Zhou et al., ).
However, treating slices of the input data independently assumes an invariance to temporal or variate positions and serves as a strong regularization against overfitting, reminiscent of kernels in CNNs.
Prominent models implementing some aspects of channel independence in multivariate time series forecasting are PatchTST (Nie et al., ) and iTransformer (Liu et al., ).
TiDE (Das et al., ), on the other hand, contains a time-step shared feature projection and temporal decoder but treats variates jointly.
As Tolstikhin et al. () have shown with MLP-Mixer, interleaving mixing of all channels in each token and all tokens per channel does not empirically sacrifice any expressivity and instead improves performance.
This idea has since been applied to time series, too, namely in architectures such as TimeMixer (Chen et al., ) and TSMixer (Chen et al., ), and is one of the key components of our method xLSTM-Mixer.

## 3 xLSTM-Mixer

Now we have everything at hand to introduce xLSTM-Mixer as depicted in Fig. 1. It carefully integrates three key components: (1) an initial linear forecast with time mixing, (2) joint mixing using powerful sLSTM modules, and (3) an eventual combination of two views by a final fully connected layer.
The transposing steps between the key components enable capturing complex temporal and intra-variate patterns while facilitating easy trainability and limiting parameter counts.
The sLSTM block, in particular, can learn intricate non-linear relationships hidden within the data along both the time and variate dimensions.
The xLSTM-Mixer architecture is furthermore equipped with normalization layers and skip connections to improve training stability and overall effectiveness.

### 3.1 Key Component 1: Normalization and Initial Linear Forecast

Normalization has become an essential ingredient of modern deep learning architectures (Huang et al., ).
For time series in particular, reversible instance norm (RevIN) (Kim et al., ) is a general recipe for improving forecasting performance, where each time series instance is normalized by its mean and variance and furthermore scaled and offset by learnable scalars $\gamma$ and $\beta$:

$$ $\bm{x}_{t}^{\text{norm}}=\operatorname{RevIN}(\bm{x}_{t})=\gamma\left(\frac{ \bm{x}_{t}-\mathbb{E}\left[\bm{x}\right]}{\sqrt{\mathrm{Var}\left[\bm{x}\right ]}+\epsilon}\right)+\beta.$ $$

We apply it as part of xLSTM-Mixer, and at the end of the entire pipeline, we invert the RevIN operation to obtain the final prediction.
In the case of xLSTM-Mixer, the typical skip connections found in mixer acrchitectures (Tolstikhin et al., ; Chen et al., ) are taken up by RevIN, the normalization in the NLinear forecast explained shortly, and the integral skip connections within each sLSTM block.

It has been shown previously that simple linear models equipped with appropriate normalization schemes are, already by themselves, decent long-term forecasters (Zeng et al., ; Li et al., ).
Our observations confirm this finding.
Therefore, we first process each variate separately by an NLinear model by computing:

$$ $\bm{x}^{\text{initial}}=\operatorname{NLinear}(\bm{x}^{\text{norm}})= \operatorname{FC}\left(\bm{x}_{1:T}^{\text{norm}}-x_{T}^{\text{norm}}\right)+x _{T}^{\text{norm}}\,,$ $$

where $\operatorname{FC}(\cdot)$ denotes a fully-connected linear layer with bias term.
Sharing this model across variates limits parameter counts, and the weight-tying serves as a useful regularization.
The quality of this initial forecast will be investigated in Sec. 4.1 and 4.2.

### 3.2 Key Component 2: sLSTM Refinement

While the NLinear forecast $\bm{x}^{\text{initial}}$ captures the basic patterns between the historic and future time steps, its quality alone is insufficient for today’s challenging time series datasets.
We, therefore, refine it using powerful sLSTM blocks.
As a first step, it is crucial to increase the embedding dimension of the data to provide enough latent dimensions $D$ for the sLSTM cells: $\bm{x}^{\text{up}}=\operatorname{FC}^{\text{up}}\left(\bm{x}^{\text{initial}}\right)$.
This pre-up-projection is similar to what is commonly performed in SSMs (Beck et al., ).
We weight-share $\operatorname{FC}^{\text{up}}$ across variates to perform time-mixing similar to the initial forecast.
Note that this step does not maintain the temporal ordering within the embedding token dimensions, as was the case up until this step, and instead embeds it into a higher latent dimension.

The stack of $M$ sLSTM blocks $\mathcal{S}(\cdot)$ transforms $\bm{x}^{\text{up}}$ as defined in Eq. 1 to 8.
The recurrent model strides over the data in variate order, i.e., where each token represents all time steps from a single variate as in the work of Liu et al. ().
The sLSTM blocks learn intricate non-linear relationships hidden within the data along both the time and variate dimensions.
The mixing of the hidden state is still limited to blocks of consecutive dimensions, aiding efficient learning and inference while allowing for effective cross-variate interaction during the recurrent processing.
Striding over variates has the benefit of linear runtime scaling in the number of variates at a constant number of parameters.
It, however, comes at the cost of possibly fixing a suboptimal order of variates.
While this is empirically not a significant limitation, we leave investigations into how to find a suitable ordering for future work.
In addition to a large embedding dim, we observed a high number of heads being crucial for effective forecasting.

The sLSTM cells’ first hidden state $\bm{h}_{t-1}$ must be initialized before each sequence of tokens can be processed.
Extending the initial description of these blocks, we propose learning a single initial embedding token $\bm{\eta}\in\mathbb{R}^{D}$ that gets prepended to each encoded time series $\bm{x}^{\text{up}}$.
These initial embeddings draw from recent advances in Large Language Models, where learnable "soft prompt" tokens are used to condition models and improve their ability to generate coherent outputs (Lester et al., ; Li & Liang, ; Chen et al., ; ).
Recent research has extended the application of soft prompts to LLM-based time series forecasting (Cao et al., ; Sun et al., ), emphasizing their adaptability and effectiveness in improving model performance across modalities.
These tokens enable greater flexibility and conditioning, allowing the model to adapt its initial memory representation to specific dataset characteristics and to dynamically interact with the time and variate data.
Soft prompts can be readily optimized through back-propagation with very little overhead.

### 3.3 Key Component 3: Multi-View Mixing

To further regularize the training of the sLSTM as with the linear projections, we compute forecasts from the original embedding $\bm{x}^{\text{up}}$ as well as the reversed embedding $\bm{\widehat{x}}^{\text{up}}$, where the order of the latent dimensions including the representation of $\bm{\eta}$ is inverted.
Learning forecasts $\bm{y}^{\prime}$ and $\bm{y}^{\prime\prime}$ for both views while sharing weights helps learn better representations.
Such multi-task learning settings are known to benefit training (Zhang & Yang, ).
The final forecast is obtained by a linear projection $\operatorname{FC}^{\text{view}}$ of the two concatenated forecasts, again per-variate.
Specifically, we compute:

$$ $\bm{y}^{\text{norm}}=\operatorname{FC}^{\text{view}}\left(\bm{y}^{\prime},\bm{ y}^{\prime\prime}\right),\hskip 4.30554pt\text{where}\hskip 4.30554pt\bm{y}^{ \prime}=\mathcal{S}(\bm{x}^{\text{up}})\hskip 4.30554pt\text{and}\hskip 4.3055 4pt\bm{y}^{\prime\prime}=\mathcal{S}(\bm{\widehat{x}}^{\text{up}}).$ $$

The final forecast is obtained after de-normalizing the reconciled forecasts as $\bm{y}=\operatorname{RevIN}^{-1}(\bm{y}^{\text{norm}})$.

## 4 Experimental Evaluation

Our intention here is to evaluate the forecasting capabilities of xLSTM-Mixer, aiming to provide comprehensive insights into its performance. To this end, we conducted a series of experiments with the primary focus on long-term forecasting, following the work of Das et al. () and Chen et al. ().
An evaluation of xLSTM-Mixer’s competitiveness in short-term forecasting on the PEMS dataset is provided in Sec. A.2.
Additionally, we perform an extensive model analysis consisting of an ablation study to identify the contributions of individual components of xLSTM-Mixer, followed by an inspection of the initial embedding tokens, a hyperparameter sensitivity analysis, and an investigation into its robustness.

**Table 1: The long-term forecasting benchmark datasets and their key properties.**
| Dataset | Source | Domain | Horizons | Sampling | #Variates |
| --- | --- | --- | --- | --- | --- |
| Weather | Zhou et al. () | Weather | 96–720 | 10 min | 21 |
| Electricity | Zhou et al. () | Power Usage | 96–720 | 1 hour | 321 |
| Traffic | Wu et al. () | Traffic Load | 96–720 | 1 hour | 862 |
| ETT | Zhou et al. () | Power Production | 96–720 | 15&60 min | 7 |

Datasets.
We generally follow the established benchmark procedure of Wu et al. () and Zhou et al. () for best backward and future comparability. The datasets we thus used are summarized in Tab. 1.

Training.
We follow standard practice in the forecasting literature by evaluating long-term forecasts using the mean squared error (MSE) and the mean absolute error (MAE). Based on our experiments, we used the MAE as the training loss function since it yielded the best results. The datasets were standardized for consistency across features. Further details on hyperparameter selection, metrics, and the implementation can be found in Sec. A.1.

Baseline Models.
We compare xLSTM-Mixer to the recurrent models xLSTMTime (Alharthi & Mahmood, ) and LSTM (Hochreiter & Schmidhuber, ); multi-perceptron (MLP) based models TimeMixer (Wang et al., ), TSMixer (Chen et al., ), DLinear (Zeng et al., ), and TiDE (Das et al., ); the Transformers PatchTST (Nie et al., ), iTransformer (Liu et al., ), FEDFormer (Zhou et al., ), and Autoformer (Wu et al., ); and the convolutional architectures MICN (Wang et al., ) and TimesNet (Wu et al., ).

### 4.1 Long-Term Time Series Forecasting

**Table 2: xLSTM-Mixer is effective in long-term forecasting. The results are averaged from 4 different prediction lengths {96, 192, 336, 720}. A lower MSE or MAE indicates a better prediction. The best result for each dataset is highlighted in bold red, while the second-best result is blue and underlined. Wins for each model out of all 28 settings are shown at the bottom.**
| Models | Recurrent | MLP | Transformer | Convolutional |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| xLSTM-Mixer | xLSTMTime | LSTM | TimeMixer | TSMixer | DLinear | TiDE | PatchTST | iTransformer | FEDFormer | Autoformer | MICN | TimesNet |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| (Ours) |  | ^a |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| Dataset | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |
| Weather | 0.219 | 0.250 | 0.222 | 0.255 | 0.444 | 0.454 | 0.222 | 0.262 | 0.225 | 0.264 | 0.246 | 0.300 | 0.236 | 0.282 | 0.241 | 0.264 | 0.258 | 0.278 | 0.309 | 0.360 | 0.338 | 0.382 | 0.242 | 0.299 | 0.259 | 0.287 |
| Electricity | 0.153 | 0.245 | 0.157 | 0.250 | 0.559 | 0.549 | 0.156 | 0.246 | 0.160 | 0.256 | 0.166 | 0.264 | 0.159 | 0.257 | 0.159 | 0.253 | 0.178 | 0.270 | 0.214 | 0.321 | 0.227 | 0.338 | 0.186 | 0.295 | 0.192 | 0.295 |
| Traffic | 0.392 | 0.253 | 0.391 | 0.261 | 1.011 | 0.541 | 0.387 | 0.262 | 0.408 | 0.284 | 0.434 | 0.295 | 0.356 | 0.261 | 0.391 | 0.264 | 0.428 | 0.282 | 0.609 | 0.376 | 0.628 | 0.379 | 0.541 | 0.315 | 0.620 | 0.336 |
| ETTh1 | 0.397 | 0.420 | 0.408 | 0.428 | 1.198 | 0.821 | 0.411 | 0.423 | 0.412 | 0.428 | 0.423 | 0.437 | 0.419 | 0.430 | 0.413 | 0.434 | 0.454 | 0.448 | 0.440 | 0.460 | 0.496 | 0.487 | 0.558 | 0.535 | 0.458 | 0.450 |
| ETTh2 | 0.340 | 0.382 | 0.346 | 0.386 | 3.095 | 1.352 | 0.316 | 0.384 | 0.355 | 0.401 | 0.431 | 0.447 | 0.345 | 0.394 | 0.324 | 0.381 | 0.383 | 0.407 | 0.433 | 0.447 | 0.453 | 0.462 | 0.588 | 0.525 | 0.414 | 0.427 |
| ETTm1 | 0.339 | 0.366 | 0.347 | 0.372 | 1.142 | 0.782 | 0.348 | 0.375 | 0.347 | 0.375 | 0.357 | 0.379 | 0.355 | 0.378 | 0.353 | 0.382 | 0.407 | 0.410 | 0.448 | 0.452 | 0.588 | 0.517 | 0.392 | 0.413 | 0.400 | 0.406 |
| ETTm2 | 0.248 | 0.307 | 0.254 | 0.310 | 2.395 | 1.177 | 0.256 | 0.315 | 0.267 | 0.322 | 0.267 | 0.332 | 0.249 | 0.312 | 0.256 | 0.317 | 0.288 | 0.332 | 0.304 | 0.349 | 0.324 | 0.368 | 0.328 | 0.382 | 0.291 | 0.333 |
| Wins | 18 | 23 | 1 | 2 | 0 | 0 | 3 | 5 | 0 | 0 | 0 | 0 | 5 | 1 | 2 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

Figure: Figure 2: Example Forecasts Across Models and Datasets. This figure shows example forecasts on the Weather and ETTm1 datasets for multiple models. The blue lines represent the ground truth target, and the orange lines show the predictions. The first panel illustrates the forecast from xLSTM-Mixer, while the second panel shows the forecast extracted before the up-projection step, highlighting the effectiveness of our added components. Comparisons with xLSTMTime, TimeMixer, and PatchTST confirm the favorable performance of xLSTM-Mixer relative to these baseline models.
Refer to caption: x2.png

We present the performance of xLSTM-Mixer compared to prior models in Tab. 2. As shown, xLSTM-Mixer consistently delivers highly accurate forecasts across a wide range of datasets. It achieves the best results in 18 out of 28 cases for MSE and 22 out of 28 cases for MAE, demonstrating its superior performance in long-term forecasting.
In particular, xLSTM-Mixer exhibits exceptional forecasting accuracy, as evidenced particularly by its strong MAE performance across all datasets. Notably, on Weather, xLSTM-Mixer reduces the MAE by 2% compared to xLSTMTime and 4.6% compared to TimeMixer. Similarly, for ETTm1, xLSTM-Mixer outperforms TimeMixer by 2.4% in MAE and shows a strong competitive edge over xLSTMTime.
Although xLSTM-Mixer performs slightly less well on the Traffic and ETTh2 datasets, where it encounters challenges with handling outliers, it remains highly competitive and outperforms the majority of baseline models. This suggests that despite these few cases, xLSTM-Mixer can consistently deliver state-of-the-art performance in long-term forecasting. A qualitative inspection of several baseline models, including the initial forecast extracted before the sLSTM refinement, is shown in Fig. 2.
In this comparison, the lookback window and forecasting horizon are both fixed at 96.

### 4.2 Model Analysis

##### Ablation Study.

To assess the contributions of each component in xLSTM-Mixer to its strong overall forecast performance, we conducted an extensive ablation study with the results listed in Tab. 3. Each configuration represents a different combination of the four key components: mixing time with NLinear, using sLSTM blocks, learning an initial embedding token, and multi-view mixing. We evaluated the performance using the MSE and MAE across the prediction lengths $\{96,192,336,720\}$.

**Table 3: Ablation Study on the Weather and ETTm1 Datasets. The MSE (↓) and MAE (↓) are reported for each numbered configuration across all four forecast horizons. The best results are highlighted in bold red, while the second-best results are blue and underlined.**
|  | #1 (full) | #2 | #3 | #4 | #5 | #6 | #7 | #8 | #9 | #10 |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Mix Time | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ |  |  |  |  |  |  |  |  |  |  |  |
| sLSTM | Variates | Time | Variates | Variates | Variates | None | Variates | Variates | Variates | Variates |  |  |  |  |  |  |  |  |  |  |  |
| Init. Token | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ |  |  |  |  |  |  |  |  |  |  |  |
| Mix View | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ |  |  |  |  |  |  |  |  |  |  |  |
|  | Horizon | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |
| Weather | 96 | 0.143 | 0.184 | 0.148 | 0.194 | 0.145 | 0.186 | 0.144 | 0.185 | 0.144 | 0.186 | 0.173 | 0.223 | 0.149 | 0.193 | 0.151 | 0.195 | 0.149 | 0.192 | 0.152 | 0.195 |
| 192 | 0.186 | 0.226 | 0.196 | 0.239 | 0.188 | 0.228 | 0.186 | 0.226 | 0.188 | 0.228 | 0.219 | 0.257 | 0.192 | 0.233 | 0.192 | 0.234 | 0.191 | 0.234 | 0.193 | 0.236 |  |
| 336 | 0.237 | 0.266 | 0.252 | 0.281 | 0.239 | 0.267 | 0.241 | 0.270 | 0.242 | 0.270 | 0.261 | 0.288 | 0.240 | 0.271 | 0.242 | 0.273 | 0.242 | 0.273 | 0.244 | 0.274 |  |
| 720 | 0.310 | 0.324 | 0.315 | 0.328 | 0.310 | 0.324 | 0.309 | 0.323 | 0.309 | 0.323 | 0.320 | 0.334 | 0.320 | 0.329 | 0.319 | 0.329 | 0.322 | 0.330 | 0.319 | 0.328 |  |
| ETTm1 | 96 | 0.275 | 0.328 | 0.298 | 0.348 | 0.277 | 0.329 | 0.278 | 0.331 | 0.279 | 0.333 | 0.295 | 0.338 | 0.282 | 0.339 | 0.285 | 0.341 | 0.281 | 0.337 | 0.284 | 0.339 |
| 192 | 0.319 | 0.354 | 0.337 | 0.369 | 0.321 | 0.354 | 0.321 | 0.356 | 0.322 | 0.358 | 0.329 | 0.357 | 0.329 | 0.364 | 0.330 | 0.365 | 0.337 | 0.367 | 0.335 | 0.366 |  |
| 336 | 0.353 | 0.374 | 0.368 | 0.388 | 0.354 | 0.375 | 0.355 | 0.377 | 0.357 | 0.379 | 0.359 | 0.376 | 0.367 | 0.385 | 0.367 | 0.385 | 0.366 | 0.384 | 0.366 | 0.385 |  |
| 720 | 0.409 | 0.407 | 0.420 | 0.416 | 0.411 | 0.408 | 0.413 | 0.411 | 0.414 | 0.411 | 0.412 | 0.407 | 0.422 | 0.412 | 0.422 | 0.413 | 0.417 | 0.410 | 0.418 | 0.411 |  |

The full version of xLSTM-Mixer (#1), which integrates all components, achieves the best performance overall. However, we also observe that some configurations of xLSTM-Mixer, which exclude specific components, remain competitive.
For instance, #3, which excludes the initial embedding token, still performs reasonably well.
This suggests that while it contributes positively to the overall performance, the model can sometimes still achieve competitive results without it.
In general, removing any specific component leads to a performance drop. For example, removing the time mixing (#7) increases the MAE by 3.4% on ETTm1 at length 96 or 2.8% at length 192, highlighting its critical role in capturing intratemporal dependencies.
When we now omit everything except for time mixing on Weather at 192, we suffer a 13.7% performance decrease.
In summary, the ablation study confirms that all components of xLSTM-Mixer contribute to its effectiveness, with the full configuration yielding the best results. Furthermore, we identified the sLSTM blocks and time-mixing as critical components for ensuring high accuracy across datasets and prediction lengths.

Figure: Figure 3: Initial tokens capture dataset characteristics. The plot illustrates the learned tokens across multiple datasets and prediction lengths. The lookback length is set to 96 for all evaluations. For clarity and the high noise levels in ETTm1 and ETTh2, we only show three seeds for Weather.
Refer to caption: x3.png

##### Initial Token Embedding.

We qualitatively inspect decodings of the initial embedding tokens $\bm{\eta}$ on multiple datasets to further understand and interpret the initializations learned by xLSTM-Mixer.
$\bm{\eta}$ are decoded to a forecast $\bm{y}$ by transforming them through the sLSTM stack $\mathcal{S}$ and applying multi-view mixing.
The resulting output of $\operatorname{FC}^{\text{view}}$ can then be interpreted as the conditioning forecast used to initialize the sLSTM blocks.
Fig. 3 shows the dataset-specific patterns the initial embedding tokens have learned on Weather, ETTm1, and ETTh2 for various prediction horizons.
With increasing prediction horizons, we observe longer spans of time, eventually revealing underlying seasonal patterns and respective dataset dynamics.

Figure: Figure 4: Sensitivity analysis of the xLSTM hidden dimension $D$. The shaded area indicates one standard deviation from the mean MSE (↓). Increasing the up-projection dimension becomes beneficial with increasing prediction length.
Refer to caption: x4.png

##### Sensitivity to xLSTM Hidden Dimension.

In Fig. 4, we visualize the performance of xLSTM-Mixer on the Electricity dataset with increasing sLSTM embedding (hidden) dimension realized by $\operatorname{FC}^{\text{up}}$. The results indicate that larger hidden dimensions consistently enhance the model’s performance, particularly for longer forecast horizons.
This suggests that a larger embedding dimension enables xLSTM-Mixer to capture better the higher complexity of the time series data over extended horizons, leading to improved forecasting accuracy.

##### Robustness to Lookback Length.

Fig. 5 illustrates the performance of xLSTM-Mixer across varying lookback lengths and prediction horizons. We observe that xLSTM-Mixer can effectively utilize longer lookback windows than the baselines, especially when compared to transformer-based models.
This advantage stems from xLSTM-Mixer’s avoidance of self-attention, allowing it to handle extended lookback lengths efficiently.
Additionally, xLSTM-Mixer demonstrates stable and consistent performance with low variance.
These results confirm that increasing the lookback length improves forecasting accuracy and enhances robustness, particularly for longer prediction horizons.

Figure: Figure 5: Increasing the lookback window increases forecasting performance, with xLSTM-Mixer virtually always providing the best results. Shows the mean MSE (↓) and std. dev. on ETTm1.
Refer to caption: x5.png

## 5 Related Work

##### Time Series Forecasting.

A long line of machine learning research led from early statistical methods like ARIMA (Box & Jenkins, ) to contemporary models based on deep learning, where four architectural families take center stage: The ones based on recurrence, convolutions, Multilayer Perceptrons (MLPs), and Transformers.
While all of them are used by practitioners today, the research focus is gradually shifting over time.
Initially, the naturally sequential recurrent models such as Long Short-Term Memory (LSTM) (Hochreiter & Schmidhuber, ) and Gated Recurrent Units (GRUs) (Cho et al., ) were used for time series analysis.
Their main benefits are the high inference efficiency and arbitrary input and output lengths due to their autoregressive nature.
While their effectiveness has historically been constrained by a limited ability to capture long-range dependencies, active research continues to alleviate these limitations (Salinas et al., ), including the xLSTM architecture presented in Sec. 2 (Beck et al., ; Alharthi & Mahmood, ).
Similarly efficient as RNNs, yet more restricted in their output length, are the location-invariant CNNs (Li et al., ; Lara-Benítez et al., ), such as TCN (Lea et al., ), TimesNet (Wu et al., ), and MICN (Wang et al., ).
Recently, some MLP-based architectures have also shown good success, including the simplistic DLinear and NLinear models (Zeng et al., ), the encoder-decoder architecture of TiDE (Das et al., ), the mixing architectures TimeMixer (Wang et al., ) and TSMixer (Chen et al., ), as well as the hierarchical N-BEATS Oreshkin et al. () and N-HiTS (Challu et al., ) models.
Finally, a lot of models have been proposed based on Transformers (Vaswani et al., ), such as Autoformer (Wu et al., ), TFT (Lim et al., ), FEDFormer (Zhou et al., ), PatchTST (Nie et al., ), and iTransformer (Liu et al., ).

##### xLSTM Models for Time Series.

Some initial experiments of applying xLSTMs (Beck et al., ) to time series were already performed by Alharthi & Mahmood () with their proposed xLSTMTime model.
While it showed promising forecasting performance, these initial soundings did not surpass stronger recent models such as TimeMixer (Wang et al., ) on multivariate benchmarks, and the reported performance is challenging to reproduce.
We ensure that our method xLSTM-Mixer is well suited as a foundation for further research by providing extensive model analysis, including an ablation study with ten variants, and ensuring that results are readily reproducible.
Our methodology draws from xLSTMTime yet improves on it by several key components.
Most importantly, our novel multi-view mixing consistently enhances forecasting performance.
Furthermore, we find the trend-seasonality decomposition to be redundant and a simple NLinear normalization scheme (Zeng et al., ) to suffice.

## 6 Conclusion

In this work, we introduced xLSTM-Mixer, a method that combines a linear forecast with further refinement using xLSTM blocks. Our architecture effectively integrates time, joint, and view mixing to capture complex dependencies. In long-term forecasting, xLSTM-Mixer consistently achieved state-of-the-art performance, outperforming previous methods in 41 out of 56 cases. Furthermore, our detailed model analysis provided valuable insights into the contribution of each component and demonstrated its robustness to varying hyperparameter settings.

While xLSTM-Mixer has shown extraordinary performance in long-term forecasting, it should be noted that due to the transpose of the input, i.e., processing the variates as sequence elements, the number of variates may limit the overall performance.
To overcome this, we plan to explore how different variate orderings influence performance and whether incorporating more than two views could lead to further improvements.
This study focused on long-term forecasting, yet extending xLSTM-Mixer to tasks such as short-term forecasting, time series classification, or imputation offers promising directions for future research.

### Ethics Statement

Our research advances machine learning by enhancing the capabilities of long-term forecasting in time series models, significantly improving both accuracy and efficiency. By developing xLSTM-Mixer, we introduce a robust framework that can be applied across various industries, including finance, healthcare, energy, and logistics. The improved forecasting accuracy enables better decision-making in critical areas, such as optimizing resource allocation, predicting market trends, and managing risk.

However, we also recognize the potential risks associated with the misuse of these advanced models. Time series forecasting models could be leveraged for malicious purposes, especially when applied at scale. For example, in the financial sector, adversarial agents might manipulate forecasts to create market instability. In political or social contexts, these models could be exploited to predict and influence public opinion or destabilize economies. Additionally, the application of these models in sensitive domains like healthcare and security may lead to unintended consequences if not carefully regulated and ethically deployed.

Therefore, it is essential that the use of xLSTM-Mixer, like all machine learning technologies, is guided by responsible practices and ethical considerations. We encourage stakeholders to adopt rigorous evaluation processes to ensure fairness, transparency, and accountability in its deployment, and to remain vigilant to the broader societal implications of time series forecasting technologies.

### Reproducibility Statement

All implementation details, including dataset descriptions, metric calculations, and experiment configurations, are provided in Sec. 4 and Sec. A.1.
We make sure to exclusively use openly available software and datasets and provide the source code for full reproducibility at
[https://github.com/mauricekraus/xLSTM-Mixer](https://github.com/mauricekraus/xLSTM-Mixer).

#### Acknowledgments

This work received funding from the EU project EXPLAIN, under the Federal Ministry of Education and Research (BMBF) (grant 01—S22030D). Furthermore, it was funded by the ACATIS Investment KVG mbH project “Temporal Machine Learning for Long-Term Value Investing” and the BMBF KompAKI project within the “The Future of Value Creation – Research on Production, Services and Work” program (funding number 02L19C150), managed by the Project Management Agency Karlsruhe (PTKA). The author of Eindhoven University of Technology received support from their Department of Mathematics and Computer Science and the Eindhoven Artificial Intelligence Systems Institute. Furthermore, this work benefited from the HMWK project “The Third Wave of Artificial Intelligence - 3AI”.

## Appendix A Appendix

### A.1 Implementation Details

##### Experimental Details

Our codebase is implemented in Python 3.11, leveraging PyTorch 2.4
(Paszke et al., ) in combination with Lightning 2.4(^2^22[https://lightning.ai/pytorch-lightning](https://lightning.ai/pytorch-lightning)) for model training and optimization.
We used the custom CUDA implementation of Beck et al. () for sLSTM(^3^33[https://github.com/NX-AI/xlstm](https://github.com/NX-AI/xlstm)) which relies on the NVIDIA Compute Capability 8.0 or higher. Thus, our experiments were conducted on a single NVIDIA A100 80GB GPU.
The majority of our baseline implementations, along with data loading and preprocessing steps, are adapted from the Time-Series-Library(^4^44[https://github.com/thuml/Time-Series-Library](https://github.com/thuml/Time-Series-Library)) of Wang et al. ().
Additionally, for xLSTMTime we used code based on the official repository(^5^55[https://github.com/muslehal/xLSTMTime](https://github.com/muslehal/xLSTMTime)) of Alharthi & Mahmood ().

##### Training and Hyperparameters

We optimized xLSTM-Mixer for up to 60 epochs with a cosine-annealing scheduler with the Adam optimizer (Kingma & Ba, ), using $\beta_{1}=0.9$ and $\beta_{2}=0.999$ and no weight decay. Hyperparameter tuning was conducted using Optuna (Akiba et al., ) with the choices provided in Tab. 4.
We optimized for the L1 forecast error, also known as the Mean Absolute Error (MAE). To further stabilize the training process, gradient clipping with a maximum norm of $1.0$ was applied. All experiments were run with three different random seeds {2021, 2022, 2023}.

**Table 4: Hyperparameters and their choices.**
| Hyperparameter | Choices |
| --- | --- |
| Batch size | {16, 32, 64, 128, 256, 512} |
| Initial learning rate | {$1\cdot 10^{-2}$, $3\cdot 10^{-3}$, $1\cdot 10^{-3}$, $5\cdot 10^{-4}$, $2\cdot 10^{-4}$, $1\cdot 10^{-4}$} |
| Scheduler warmup steps | {5, 10, 15} |
| Lookback length $T$ | {96, 256, 512, 768, 1024, 2048} |
| Embedding dimension $D$ | {32, 64, 128, 256, 512, 768, 1024} |
| sLSTM conv. kernel width | {disabled, 2, 4} |
| sLSTM dropout rate | {0.1, 0.25} |
| # sLSTM blocks in $\mathcal{S}$ | {1, 2, 3, 4} |
| # sLSTM heads | {4, 8, 16, 32} |

##### Metrics

We follow common practice in the literature (Wu et al., ; Wang et al., ) for maximum comparability and, therefore, evaluate long-term forecasting of all models on the mean absolute error (MAE), mean squared error (MSE), and for short-term forecasting, using the MAE, root mean squared error (RMSE), and mean absolute percentage error (MAPE).
The metrics are averaged over all variates and computed as:

$$ $\displaystyle\operatorname{MAE}(\bm{y},\bm{\hat{y}})$ $\displaystyle=\sum_{i=1}^{H}\left|y_{i}-\hat{y_{i}}\right|$ $\displaystyle\operatorname{MSE}(\bm{y},\bm{\hat{y}})$ $\displaystyle=\sum_{i=1}^{H}(y_{i}-\hat{y_{i}})^{2}$ $\displaystyle\operatorname{RMSE}(\bm{y},\bm{\hat{y}})$ $\displaystyle=\sqrt{\operatorname{MSE}(\bm{y},\bm{\hat{y}})}$ $\displaystyle\operatorname{MAPE}(\bm{y},\bm{\hat{y}})$ $\displaystyle=\frac{100}{H}\sum_{i=1}^{H}\frac{\left|y_{i}-\hat{y}_{i}\right|} {\left|y_{i}\right|+\epsilon},$ $$

where $\bm{y}$ are the targets, $\bm{\hat{y}}$ the predictions, and $\epsilon$ a small constant added for numerical stability.

### A.2 Outlook: Short-Term Time Series Forecasting

Having shown superior long-term forecasting accuracies in Sec. 4.1, we also provide an initial exploration of the effectiveness of xLSTM-Mixer to short-term forecasts.
To this end, we compare it to applicable baselines on PEMS datasets with input lengths uniformly set to 96 and prediction lengths to 12.
The results in Tab. 5 show that the performance of xLSTM-Mixer is competitive with existing methods.
We provide the MAE, MAPE, and RMSE as is common practice.

**Table 5: Short-term forecasting evaluation of xLSTM-Mixer and baselines on the multivariate PEMS datasets. A lower MAE, MAPE, or RMSE indicates a better prediction. The best result for each dataset is highlighted in bold red, while the second-best result is blue and underlined.**
| Models | Recurrent | MLP | Transformer | Convolutional |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| xLSTM-Mixer | xLSTMTime | LSTM | TimeMixer | DLinear | PatchTST | FEDFormer | Autoformer | MICN | TimesNet |  |  |
| (Ours) |  | ^a |  |  |  |  |  |  |  |  |  |
| PEMS03 | MAE | 15.71 | 16.59 | 18.65 | 14.63 | 19.70 | 18.95 | 19.00 | 18.08 | 15.71 | 16.41 |
| MAPE | 14.92 | 15.31 | 17.39 | 14.54 | 18.35 | 17.29 | 18.57 | 18.75 | 15.67 | 15.17 |  |
| RMSE | 24.82 | 26.47 | 31.73 | 23.28 | 32.35 | 30.15 | 30.05 | 27.82 | 24.55 | 26.72 |  |
| PEMS08 | MAE | 16.56 | 17.44 | 20.34 | 15.22 | 20.26 | 20.35 | 20.56 | 20.47 | 17.76 | 19.01 |
| MAPE | 10.24 | 10.58 | 13.05 | 9.67 | 12.09 | 13.15 | 12.41 | 12.27 | 10.76 | 11.83 |  |
| RMSE | 26.65 | 28.13 | 31.90 | 24.26 | 32.38 | 31.04 | 32.97 | 31.52 | 27.26 | 30.65 |  |

### A.3 Full Results for Long-Term Forecasting

Tab. 6 shows the full results for long-term forecasting on all four separate forecast horizons.

**Table 6: Full long-term forecasting results for Tab. 2. *Avg* is averaged from all four prediction lengths {96, 192, 336, 720}. A lower MSE or MAE indicates a better prediction. The best result for each dataset is highlighted in bold red, while the second-best result is blue and underlined. Wins for each model are shown at the bottom.**
| Models | Recurrent | MLP | Transformer | Convolutional |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| xLSTM-Mixer | xLSTMTime | LSTM | TimeMixer | TSMixer | DLinear | TiDE | PatchTST | iTransformer | FEDFormer | Autoformer | MICN | TimesNet |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| (Ours) |  | ^a |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |  |
| Weather | 96 | 0.143 | 0.184 | 0.144 | 0.187 | 0.369 | 0.406 | 0.147 | 0.197 | 0.145 | 0.198 | 0.176 | 0.237 | 0.166 | 0.222 | 0.149 | 0.198 | 0.174 | 0.214 | 0.217 | 0.296 | 0.266 | 0.336 | 0.161 | 0.229 | 0.172 | 0.220 |
| 192 | 0.186 | 0.226 | 0.192 | 0.236 | 0.416 | 0.435 | 0.189 | 0.239 | 0.191 | 0.242 | 0.220 | 0.282 | 0.209 | 0.263 | 0.194 | 0.241 | 0.221 | 0.254 | 0.276 | 0.336 | 0.307 | 0.367 | 0.220 | 0.281 | 0.219 | 0.261 |  |
| 336 | 0.236 | 0.266 | 0.237 | 0.272 | 0.455 | 0.454 | 0.241 | 0.280 | 0.242 | 0.280 | 0.265 | 0.319 | 0.254 | 0.301 | 0.306 | 0.282 | 0.278 | 0.296 | 0.339 | 0.380 | 0.359 | 0.395 | 0.278 | 0.331 | 0.280 | 0.306 |  |
| 720 | 0.310 | 0.323 | 0.313 | 0.326 | 0.535 | 0.520 | 0.310 | 0.330 | 0.320 | 0.336 | 0.323 | 0.362 | 0.313 | 0.340 | 0.314 | 0.334 | 0.358 | 0.347 | 0.403 | 0.428 | 0.419 | 0.428 | 0.311 | 0.356 | 0.365 | 0.359 |  |
| Avg | 0.219 | 0.250 | 0.222 | 0.255 | 0.444 | 0.454 | 0.222 | 0.262 | 0.225 | 0.264 | 0.246 | 0.300 | 0.236 | 0.282 | 0.241 | 0.264 | 0.258 | 0.278 | 0.309 | 0.360 | 0.338 | 0.382 | 0.242 | 0.299 | 0.259 | 0.287 |  |
| Electricity | 96 | 0.126 | 0.218 | 0.128 | 0.221 | 0.375 | 0.437 | 0.129 | 0.224 | 0.131 | 0.229 | 0.140 | 0.237 | 0.132 | 0.229 | 0.129 | 0.222 | 0.148 | 0.240 | 0.193 | 0.308 | 0.201 | 0.317 | 0.164 | 0.269 | 0.168 | 0.272 |
| 192 | 0.144 | 0.235 | 0.150 | 0.243 | 0.442 | 0.473 | 0.140 | 0.220 | 0.151 | 0.246 | 0.153 | 0.249 | 0.147 | 0.243 | 0.147 | 0.240 | 0.162 | 0.253 | 0.201 | 0.315 | 0.222 | 0.334 | 0.177 | 0.285 | 0.184 | 0.289 |  |
| 336 | 0.157 | 0.250 | 0.166 | 0.259 | 0.439 | 0.473 | 0.161 | 0.255 | 0.161 | 0.261 | 0.169 | 0.267 | 0.161 | 0.261 | 0.163 | 0.259 | 0.178 | 0.269 | 0.214 | 0.329 | 0.231 | 0.338 | 0.193 | 0.304 | 0.198 | 0.300 |  |
| 720 | 0.183 | 0.276 | 0.185 | 0.276 | 0.980 | 0.814 | 0.194 | 0.287 | 0.197 | 0.293 | 0.203 | 0.301 | 0.196 | 0.294 | 0.197 | 0.290 | 0.225 | 0.317 | 0.246 | 0.355 | 0.254 | 0.361 | 0.212 | 0.321 | 0.220 | 0.320 |  |
| Avg | 0.153 | 0.245 | 0.157 | 0.250 | 0.559 | 0.549 | 0.156 | 0.246 | 0.160 | 0.256 | 0.166 | 0.264 | 0.159 | 0.257 | 0.159 | 0.253 | 0.178 | 0.270 | 0.214 | 0.321 | 0.227 | 0.338 | 0.186 | 0.295 | 0.192 | 0.295 |  |
| Traffic | 96 | 0.357 | 0.236 | 0.358 | 0.242 | 0.843 | 0.453 | 0.360 | 0.249 | 0.376 | 0.264 | 0.410 | 0.282 | 0.336 | 0.253 | 0.360 | 0.249 | 0.395 | 0.268 | 0.587 | 0.366 | 0.613 | 0.388 | 0.519 | 0.309 | 0.593 | 0.321 |
| 192 | 0.377 | 0.241 | 0.378 | 0.253 | 0.847 | 0.453 | 0.375 | 0.250 | 0.397 | 0.277 | 0.423 | 0.287 | 0.346 | 0.257 | 0.379 | 0.256 | 0.417 | 0.276 | 0.604 | 0.373 | 0.616 | 0.382 | 0.537 | 0.315 | 0.617 | 0.336 |  |
| 336 | 0.394 | 0.250 | 0.392 | 0.261 | 0.853 | 0.455 | 0.385 | 0.270 | 0.413 | 0.290 | 0.436 | 0.296 | 0.355 | 0.260 | 0.392 | 0.264 | 0.433 | 0.283 | 0.621 | 0.383 | 0.622 | 0.337 | 0.534 | 0.313 | 0.629 | 0.336 |  |
| 720 | 0.439 | 0.283 | 0.434 | 0.287 | 1.500 | 0.805 | 0.430 | 0.281 | 0.444 | 0.306 | 0.466 | 0.315 | 0.386 | 0.273 | 0.432 | 0.286 | 0.467 | 0.302 | 0.626 | 0.382 | 0.660 | 0.408 | 0.577 | 0.325 | 0.640 | 0.350 |  |
| Avg | 0.392 | 0.253 | 0.391 | 0.261 | 1.011 | 0.541 | 0.387 | 0.262 | 0.408 | 0.284 | 0.434 | 0.295 | 0.356 | 0.261 | 0.391 | 0.264 | 0.428 | 0.282 | 0.609 | 0.376 | 0.628 | 0.379 | 0.541 | 0.315 | 0.620 | 0.336 |  |
| ETTh1 | 96 | 0.359 | 0.386 | 0.368 | 0.395 | 1.044 | 0.773 | 0.361 | 0.390 | 0.361 | 0.392 | 0.375 | 0.399 | 0.375 | 0.398 | 0.370 | 0.400 | 0.386 | 0.405 | 0.376 | 0.419 | 0.449 | 0.459 | 0.421 | 0.431 | 0.384 | 0.402 |
| 192 | 0.402 | 0.417 | 0.401 | 0.416 | 1.217 | 0.832 | 0.409 | 0.414 | 0.404 | 0.418 | 0.405 | 0.416 | 0.412 | 0.422 | 0.413 | 0.429 | 0.441 | 0.436 | 0.420 | 0.448 | 0.500 | 0.482 | 0.474 | 0.487 | 0.436 | 0.429 |  |
| 336 | 0.408 | 0.429 | 0.422 | 0.437 | 1.259 | 0.841 | 0.430 | 0.429 | 0.420 | 0.431 | 0.439 | 0.443 | 0.435 | 0.433 | 0.422 | 0.440 | 0.487 | 0.458 | 0.459 | 0.465 | 0.521 | 0.496 | 0.569 | 0.551 | 0.491 | 0.469 |  |
| 720 | 0.419 | 0.448 | 0.441 | 0.465 | 1.271 | 0.838 | 0.445 | 0.460 | 0.463 | 0.472 | 0.472 | 0.490 | 0.454 | 0.465 | 0.447 | 0.468 | 0.503 | 0.491 | 0.506 | 0.507 | 0.514 | 0.512 | 0.770 | 0.672 | 0.521 | 0.500 |  |
| Avg | 0.397 | 0.420 | 0.408 | 0.428 | 1.198 | 0.821 | 0.411 | 0.423 | 0.412 | 0.428 | 0.423 | 0.437 | 0.419 | 0.430 | 0.413 | 0.434 | 0.454 | 0.448 | 0.440 | 0.460 | 0.496 | 0.487 | 0.558 | 0.535 | 0.458 | 0.450 |  |
| ETTh2 | 96 | 0.267 | 0.329 | 0.273 | 0.333 | 2.522 | 1.278 | 0.271 | 0.330 | 0.274 | 0.341 | 0.289 | 0.353 | 0.270 | 0.336 | 0.274 | 0.337 | 0.297 | 0.349 | 0.346 | 0.388 | 0.358 | 0.397 | 0.299 | 0.364 | 0.340 | 0.374 |
| 192 | 0.338 | 0.375 | 0.340 | 0.378 | 3.312 | 1.384 | 0.317 | 0.402 | 0.339 | 0.385 | 0.383 | 0.418 | 0.332 | 0.380 | 0.314 | 0.382 | 0.380 | 0.400 | 0.429 | 0.439 | 0.456 | 0.452 | 0.441 | 0.454 | 0.402 | 0.414 |  |
| 336 | 0.367 | 0.401 | 0.373 | 0.403 | 3.291 | 1.388 | 0.332 | 0.396 | 0.361 | 0.406 | 0.448 | 0.465 | 0.360 | 0.407 | 0.329 | 0.384 | 0.428 | 0.432 | 0.496 | 0.487 | 0.482 | 0.486 | 0.654 | 0.567 | 0.452 | 0.452 |  |
| 720 | 0.388 | 0.424 | 0.398 | 0.430 | 3.257 | 1.357 | 0.342 | 0.408 | 0.445 | 0.470 | 0.605 | 0.551 | 0.419 | 0.451 | 0.379 | 0.422 | 0.427 | 0.445 | 0.463 | 0.474 | 0.515 | 0.511 | 0.956 | 0.716 | 0.462 | 0.468 |  |
| Avg | 0.340 | 0.382 | 0.346 | 0.386 | 3.095 | 1.352 | 0.316 | 0.384 | 0.355 | 0.401 | 0.431 | 0.447 | 0.345 | 0.394 | 0.324 | 0.381 | 0.383 | 0.407 | 0.433 | 0.447 | 0.453 | 0.462 | 0.588 | 0.525 | 0.414 | 0.427 |  |
| ETTm1 | 96 | 0.275 | 0.328 | 0.286 | 0.335 | 0.863 | 0.664 | 0.291 | 0.340 | 0.285 | 0.339 | 0.299 | 0.343 | 0.306 | 0.349 | 0.293 | 0.346 | 0.334 | 0.368 | 0.379 | 0.419 | 0.505 | 0.475 | 0.316 | 0.362 | 0.338 | 0.375 |
| 192 | 0.319 | 0.354 | 0.329 | 0.361 | 1.113 | 0.776 | 0.327 | 0.365 | 0.327 | 0.365 | 0.335 | 0.365 | 0.335 | 0.366 | 0.333 | 0.370 | 0.377 | 0.391 | 0.426 | 0.441 | 0.553 | 0.496 | 0.363 | 0.390 | 0.374 | 0.387 |  |
| 336 | 0.353 | 0.374 | 0.358 | 0.379 | 1.267 | 0.832 | 0.360 | 0.381 | 0.356 | 0.382 | 0.369 | 0.386 | 0.364 | 0.384 | 0.369 | 0.392 | 0.426 | 0.420 | 0.445 | 0.459 | 0.621 | 0.537 | 0.408 | 0.426 | 0.410 | 0.411 |  |
| 720 | 0.409 | 0.407 | 0.416 | 0.411 | 1.324 | 0.858 | 0.415 | 0.417 | 0.419 | 0.414 | 0.425 | 0.421 | 0.413 | 0.413 | 0.416 | 0.420 | 0.491 | 0.459 | 0.543 | 0.490 | 0.671 | 0.561 | 0.481 | 0.476 | 0.478 | 0.450 |  |
| Avg | 0.339 | 0.366 | 0.347 | 0.372 | 1.142 | 0.782 | 0.348 | 0.375 | 0.347 | 0.375 | 0.357 | 0.379 | 0.355 | 0.378 | 0.353 | 0.382 | 0.407 | 0.410 | 0.448 | 0.452 | 0.588 | 0.517 | 0.392 | 0.413 | 0.400 | 0.406 |  |
| ETTm2 | 96 | 0.157 | 0.244 | 0.164 | 0.250 | 2.041 | 1.073 | 0.164 | 0.254 | 0.163 | 0.252 | 0.167 | 0.260 | 0.161 | 0.251 | 0.166 | 0.256 | 0.180 | 0.264 | 0.203 | 0.287 | 0.255 | 0.339 | 0.179 | 0.275 | 0.187 | 0.267 |
| 192 | 0.213 | 0.285 | 0.218 | 0.288 | 2.249 | 1.112 | 0.223 | 0.295 | 0.216 | 0.290 | 0.224 | 0.303 | 0.215 | 0.289 | 0.223 | 0.296 | 0.250 | 0.309 | 0.269 | 0.328 | 0.281 | 0.340 | 0.307 | 0.376 | 0.249 | 0.309 |  |
| 336 | 0.269 | 0.322 | 0.271 | 0.322 | 2.568 | 1.238 | 0.279 | 0.330 | 0.268 | 0.324 | 0.281 | 0.342 | 0.267 | 0.326 | 0.274 | 0.329 | 0.311 | 0.348 | 0.325 | 0.366 | 0.339 | 0.372 | 0.325 | 0.388 | 0.321 | 0.351 |  |
| 720 | 0.351 | 0.377 | 0.361 | 0.380 | 2.720 | 1.287 | 0.359 | 0.383 | 0.420 | 0.422 | 0.397 | 0.421 | 0.352 | 0.383 | 0.362 | 0.385 | 0.412 | 0.407 | 0.421 | 0.415 | 0.422 | 0.419 | 0.502 | 0.490 | 0.408 | 0.403 |  |
| Avg | 0.248 | 0.307 | 0.254 | 0.310 | 2.395 | 1.177 | 0.256 | 0.315 | 0.267 | 0.322 | 0.267 | 0.332 | 0.249 | 0.312 | 0.256 | 0.317 | 0.288 | 0.332 | 0.304 | 0.349 | 0.324 | 0.368 | 0.328 | 0.382 | 0.291 | 0.333 |  |
|  | Wins | 18 | 23 | 1 | 2 | 0 | 0 | 3 | 5 | 0 | 0 | 0 | 0 | 5 | 1 | 2 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 6
presentation: 2
questions: 1. The term "Mix View" in Table 3, line 332, is unclear and challenging to interpret. Could you clarify if this refers to the performance of channel mixing?

2. The dataset selection could be expanded to include the Exchange from the  (Informer long-horizon forecasting datasets) datasets and more extensive ablation studies across additional datasets to provide a thorough evaluation of the xLSTM-Mixer’s performance, including an exploration of cross-correlation between variables and model effectiveness on different datasets or synthetic ones.

3. Consider incorporating synthetic data to facilitate verification of learnable concepts and addressing the model’s sensitivity to the order of variates, as it lacks permutation invariance.

4. I would highly encourage you to back the claims of challenges in the reproducibility of prior works and why trend-seasonality decomposition is redundant.
soundness: 2
strengths: **Originality**
xLSTM-Mixer combines temporal sequences, joint time-variate information, and multi-view perspectives to handle multivariate time series complexities, capturing temporal and cross-variate patterns for accurate predictions, it builds on prior work, but combines it in an intricate way. The work enhances the xLSTM architecture, however limiting itself to the sLSTM.

**Quality**
The work is written with enough clarity, related work is addressed and the paper is fairly easy to understand. The graphical presentation of the model allows for easy following.

**Significance**
Authors do not claim state-of-the-art performance, however, they do beat the current baselines to which they compare themselves. It is the second work relating to the use of xLSTM to time series, and authors raise concerns about the first one, those concerns must be backed in the future.
summary: The paper introduces xLSTM-Mixer, a new model for forecasting complex time series data with multiple variables. It combines the xLSTM architecture with multi-view processing and variate mixing. The model works in three stages: first, it creates an initial forecast, then it uses xLSTM blocks to combine time and variable patterns, and finally, it merges two different forecast views for results in most cases beating baseline models. xLSTM-Mixer addresses challenges associated with Transformer models, such as high computational costs and data requirements for long sequences.
weaknesses: **Weaknesses**

1. The authors conduct experiments on only a subset of datasets, specifically focusing on a subset of the Informer long-horizon forecasting datasets and omitting the Exchange dataset, which would add considerable value if included. The reviewers would appreciate a deeper analysis of why the XLSTM-Mixer performs better on some datasets and worse on others, particularly through an exploration of cross-correlation between variables and model performance.

2. The ablation study, conducted on only two datasets, lacks breadth. Expanding this experiment across additional datasets would significantly strengthen the findings.

3. The experiments do not address a critical aspect: the model's sensitivity to the order of variates, as it is not permutation invariant in this regard.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: - The manuscript fails to convincingly argue the substantial novelty of xLSTM-Mixer compared to prior xLSTM models. Authors should clarify that xLSTM-Mixer is not merely a combination of xLSTM and Multi-View Mixing, as the current description lacks the depth to establish its significance.

- While the manuscript acknowledges xLSTM-Mixer's performance with varying lookback windows (Figure 5), it lacks a thorough experimental setup to explore different input/output ratios. A comprehensive analysis should include varied input lengths relative to output horizons to assess model robustness. The absence of this setup limits understanding of performance in diverse forecasting conditions. Additional experiments with varying input/output ratios, as suggested by Table 6, are recommended.

- The paper does not explain the increase in MSE at a lookback length of 1024, leaving readers unclear about the model's behavior. To strengthen the manuscript, the authors should provide a plausible explanation or conduct additional experiments to verify performance consistency across different lookback lengths.

- A significant issue is the claim that a simple NLinear normalization scheme is sufficient without rigorous testing or theoretical backing. The decision to forgo trend-seasonality decomposition and rely solely on NLinear normalization lacks the necessary validation, raising questions about the model's robustness. More experimental corroboration against established methods is needed.
soundness: 3
strengths: - Superior long-term forecasting performance: xLSTM-Mixer outperforms recent state-of-the-art methods in terms of predictive accuracy on various benchmark datasets.

- Robustness and effectiveness: Detailed model analysis, including A/B testing, sensitivity analysis of hyperparameters, and research on initial embedding vectors, further validates the effectiveness and robustness of the model.

- Contribution to the resurgence of recurrent models in time series forecasting: By combining linear prediction with xLSTM blocks and integrating time, variate, and multi-view mixing, xLSTM-Mixer enhances the learning of necessary features and offers a viable alternative to current sequence models, contributing to the resurgence of recurrent models in time series forecasting.
summary: This paper introduces a new time series prediction model called xLSTM-Mixer, which aims to effectively capture and integrate complex patterns in time series data by combining linear prediction with xLSTM blocks. The model first uses a shared linear model to make initial predictions for each variable, then fine-tunes and mixes these predictions through a series of sLSTM modules, and finally generates the final prediction result by integrating two different perspectives' predictions. Experimental results show that xLSTM-Mixer has significant advantages in long-term time series prediction tasks, achieving better predictive accuracy than current state-of-the-art methods on multiple benchmark datasets. In addition, the article conducts detailed analysis of the model, including A/B testing, sensitivity analysis of hyperparameters, and research on initial embedding vectors, further validating the effectiveness and robustness of the model. This work not only improves the level of deep learning-based time series prediction technology but also provides new directions for future related research.
weaknesses: - The contribution regarding advancement of  xLSTM-Mixer over the base xLSTM model is not clearly defined. The novelty and significant contributions need further clarification to distinguish xLSTM-Mixer as more than a trivial modification.

- The paper asserts that longer lookback lengths improve forecasting but Figure 5 shows increased MSE at a lookback length of 1024 compared to 768. This requires further exploration as it conflicts with the claims.

- The manuscript dismisses trend-seasonality decomposition without empirical support, which is a significant weakness given the technique's established value in forecasting.

- The paper omits a comparison of the xLSTM-Mixer's computational complexity and cost against other methods, a critical evaluation factor for practical model deployment.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: **More discussion on the use of Multivariate information**:
- Firstly, the primary innovation claimed by the authors is the use of multivariate time series forecasting. However, simply explaining how the new method works and benchmarking with other multivariate models is insufficient. Since the impact of using multivariate information varies across different datasets and variables, and given that multivariate mixing is the core innovation, additional analysis on this aspect should be provided.
- Secondly, whats are the authors' opinion on the heterogeneity of variates, and how important is this compare to the temporal dependency in terms of forecasting? 
- Thirdly, the authors mention that variate ordering may impact performance but leave this for future work. Given the importance of variate dependencies in time series, a preliminary exploration of this factor could provide additional insights into potential model improvements.
soundness: 2
strengths: **Innovation in Model Architecture**: The integration of time and variate mixing with xLSTM blocks is an innovative approach. This design tackles some limitations of conventional recurrent and transformer-based models in capturing complex temporal dependencies, which is especially challenging in multivariate time series data. 

**Empirical Performance**: The experiments show that xLSTM-Mixer outperforms a variety of established methods (e.g., Time-Mixer, iTransformer, MICN) in terms of mean squared error (MSE) and mean absolute error (MAE) across multiple datasets, together with comprehensive ablation studies on initializations, size of hidden dimension and length of lookback window shows the model’s robustness and generalizability.
summary: The paper presents a novel design - xLSTM-Mixer which is based on LSTM and a set of mixing machanism for multivariate time series forecasting. The model builds on the idea of recurrent structures, integrating three mixing strategy: an initial linear forecast with time mixing, joint mixing with sLSTM blocks, and a final view mixing stage. Evaluations show that xLSTM-Mixer achieves state-of-the-art forecasting performance across multiple datasets which is a key contribution to the ongoing development of recurrent models.
weaknesses: **Model structure**: 
LSTM-Mixer appears to be an incremental improvement over xLSTM, and the mixing machisim does not seems to be very siginificant regards to the Ablation Study on Table 3. An experiment with alternative recurrent structures may help here. Instead of relying solely on the sLSTM block with the joint mixing machinism, it would be usefule to assess the performance of other recurrent models, such as GRU and Mamba.

**Computational Efficiency**: 
The complexity of xLSTM-Mixer’s mixing machanism and recurrent structures would be computationally intensive. Further discussion on the computational costs relative to attentions and SSMs would be beneficial. 

**Evaluation on Short-Term Forecasting**:
While the model demonstrates strong performance in long-term forecasting, the short-term forecasting evaluation is inadequate. Adding a more robust short-term evaluation could highlight the model’s flexibility across different forecasting horizons.

**Evaluation on Long-Term Forecasting**:
Although most studies follow the 96, 192, 336, 720 prediction length on benchmarks, regards to this work highlighted long-term forecasting as a distinctive feature, a prediction length of 720 is insufficient.

---

confidence: 4
contribution: 4
initial_rating: 8
presentation: 3
questions: Suggestions:
- Adding dimensions to Fig. 1 would help clarify input shapes after higher-dimensional projections and transpositions.
- Lines 41-42: Regarding Transformer architectures, the authors state, “For instance, they typically require large datasets to train successfully, restricting their use to only a subset of applications.” However, I do not believe this claim is fully supported by the literature. For example, common benchmarks for long-horizon forecasting include datasets like ILI, Traffic, and Weather. The ILI dataset, with only 966 timestamps, could be considered relatively small, yet PatchTST outperforms other Transformer and linear models on this dataset [1]. Similarly, for ILI, Autoformer outperforms other models, including recurrent architectures like LSTMs [2].
- It would help motivate the proposed approach to illustrate the trade-offs in computational complexity between the proposed model, Transformer-based models, and MLP-based models for both training and inference.
- While model hyperparameters for the proposed model are included in the appendix, it would be helpful to provide hyperparameters and the number of training epochs for the baseline models as well.
- Additional benchmark datasets could include ILI and Exchange datasets which are used in prior work [2].

References:
1. “A Time Series is Worth 64 Words: Long-term Forecasting with Transformers”
2. “Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting”
soundness: 4
strengths: - Figure 1 provides a clear overview of the method.
- Several commonly used benchmark datasets are included.
- The proposed xLSTM-Mixer model consistently achieves state-of-the-art performance across a broad range of benchmarks.
- Informative forecast plots are included.
- A comprehensive set of baseline models from different architecture categories including, Transformer-based, CNN-based, MLP-based, and Recurrent architectures.
- The ablation study on architecture choices is highly insightful and supports the combined aspects of the proposed model.
- Open-source code is available with reproducible experiments
summary: The paper introduce  a novel time series forecasting model with time and variate mixing in the context of recurrent models.
weaknesses: 1. Random seeds were included as a hyperparameter, but it would be beneficial to train the model over several trials to compute the average and standard deviation of the results. This approach would provide a more robust measure of performance consistency across runs.
2. The approach does not reference similar architectures, such as iTransformer, which utilizes the attention and feed-forward network on inverted dimensions. In iTransformer, individual time points are embedded as variate tokens, leveraging the attention mechanism to capture multivariate correlations. This bears a resemblance to the proposed method, which jointly combines time and variate information to capture complex patterns in the data by employing transposed input dimensions to invert the series. Mentioning this may help emphasize the translatable similarities across architecture backbones.
3. The motivation for the reverse-up projected embedding is not explicitly explained. Including an ablation study of the model with and without this embedding would be useful to help justify this architectural choice.
4. Some models cited in prior work are missing as baselines, including TFT, N-HiTS, and N-BEATS.

---

confidence: 3
contribution: 1
initial_rating: 3
presentation: 2
questions: 1. Has the author tried to train on time series with randomly shuffled temporal orders, and what will happen to the results?
2. About the experiments of enlarging lookback (Figure 3): In the DLinear paper, the performance can also be improved by increasing the lookback using a single linear layer. It is suggested that the author introduce such linear models into the baseline. Otherwise, the advantage can only be due to NLinear. It is also encouraged to provide the model performance with a longer lookback length to verify whether the proposed can continuously improve the performance (or keep it stable in a very long lookback length).
soundness: 2
strengths: 1. The motivations are well-summarized and the paper's writing is easy to follow.
2. The author provides the code to ensure the reproducibility of the paper.
summary: The work proposed xLSTM-Mixer for multivariate time series forecasting that combines linear forecasters (NLinear) with efficient recurrent xLSTM blocks. By adapting xLSTM modules for the temporal and variate dimensions, the model refined the predictions of NLinear with the utilization of time-variate information. The model also employs techniques like reversible instance normalization and soft prompt tokens to further improve the performance. Empirical experiments demonstrate that xLSTM-Mixer can outperform several baseline methods in most cases in long-term forecasting.
weaknesses: 1. The main contribution of this paper is to refine the predictions obtained from NLinear by introducing xLSTM modules, which may lack novelty and motivation in technology. On the one hand, please clarify more about the advantages of xLSTM modules that have been refined over (linear forecaster + renorm). For example, to support the mentioned advantages of modeling variate relationships, you can provide a comparative analysis of xLSTM against MLP mixers and efficient attention modules in the Mixer architecture. Also, extending ablation studies (Table 3) to more datasets and different variate-mixing modules would provide a more comprehensive evaluation of the model's effectiveness.
2. It is uncertain whether the authors use the unidirectional xLSTM, which is generally adopted for sequential modeling, to model the correlations of variates. If it is unidirectional, please explain how it addresses the non-sequential nature of the variate correlation, and discuss the potential benefits or drawbacks of using a bidirectional approach in this context.
3. This paper mainly provides the results of long-term forecasting. The lookback seems to be fixed at 96. Considering the extensive overfitting of current deep models on these datasets, it is recommended that the author provide more comparisons with varying lookback lengths and on additional datasets that are less prone to overfitting. This would provide a more robust evaluation of the model's performance.
4. Compared with related recurrent models (such as xLSTMTime), the performance improvement of the proposed model is not significant shown in Table 2. The author should add more recent recurrent models (e.g., SutraNets and P-sLSTM) and more discussions with related work.
5. The showcases in Figure 2 do not reflect the advantages of xLSTM-Mixer in multivariate modeling. To make it more specific, please include visualizations that explicitly show how xLSTM-Mixer captures relationships between multiple variates in its predictions.
6. The paper concludes by highlighting the potential of xLSTM-Mixer for future research in areas such as short-term forecasting, time series classification, and imputation. I wonder if there are some results on these analysis tasks using xLSTMs.

Meta-review: This paper has been evaluated by 5 knowledgeable reviewers. Their opinions varied: 1 straight acceptance, 1 marginal acceptance, 1 marginal rejection and 2 straight rejections. The authors provided extensive rebuttals which addressed some of the reviewers’ complaints. The paper considers long-term time series forecasting using xLTM as a mixer adapted to capture both the temporal and cross-channel information. It shows good empirical performance but leaves some issues unaddressed. Reviewers brought up the lack of analysis of the impact of relative lengths of the observed input time series and the forecast horizon, they also wished for more comparisons against more data and a broader range of relevant algorithmic baselines, to comprehensively assess the relative utility of the proposed method. Reviewers also pointed out incremental novelty of the presented methods. This promising work will need a few substantial upgrades to be accepted for publication at ICLR.

Decision: Reject