Title: TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: xLSTM

Time Series Forecasting

Zero-Shot Forecasting

Pre-Trained Models
Abstract: In-context learning, the ability of large language models to perform tasks using only examples provided in the prompt, has recently been adapted for time series forecasting. 
This paradigm enables zero-shot prediction, where past values serve as context for forecasting future values, 
making powerful forecasting tools accessible to non-experts and increasing the performance when training data are scarce.
Most existing zero-shot forecasting approaches rely on transformer architectures, which, despite their success in language, often fall short of expectations in time series forecasting, where recurrent models like LSTMs frequently have the edge. Conversely, while LSTMs are well-suited for time series modeling due to their state-tracking capabilities, they lack strong in-context learning abilities.
We introduce *TiRex* that closes this gap by leveraging xLSTM, an enhanced LSTM with competitive in-context learning skills. Unlike transformers, state-space models, or parallelizable RNNs such as RWKV, TiRex retains state tracking, a critical property for long-horizon forecasting. To further facilitate its state-tracking ability, we propose a training-time masking strategy called CPM.
TiRex sets a new state of the art in zero-shot time series forecasting on the Hugging Face benchmarks *GiftEval* and *Chronos-ZS*, outperforming significantly larger models including *TabPFN-TS* (Prior Labs), *Chronos Bolt* (Amazon), *TimesFM* (Google), and *Moirai* (Salesforce) across both short- and long-term forecasts.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces TiRex, a pre-trained time series forecasting model leveraging xLSTM, a modern LSTM variant, to address the limitations of existing zero-shot forecasting approaches. TiRex integrates Contiguous Patch Masking (CPM), a novel training-time masking strategy, and three data augmentation techniques to enhance state-tracking and generalization. Experiments on HuggingFace benchmarks GiftEval and Chronos-ZS demonstrate that TiRex outperforms significantly larger models (e.g., TabPFN-TS, Chronos Bolt) in both short- and long-term forecasting, with fewer parameters (35M) and faster inference speed. The model’s success stems from combining xLSTM’s in-context learning with CPM to mitigate autoregressive error accumulation and augmentations to handle diverse time series patterns.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths** \n\n1. By integrating xLSTM, TiRex bridges the gap between LSTM's state-tracking and transformer's in-context learning, enabling robust long-horizon forecasting.\n\n2. CPM enhances multi-patch prediction stability, while augmentations (amplitude modulation, censor augmentation, spike injection) improve generalization to rare patterns.\n\n3. Sets new SOTA on zero-shot benchmarks, outperforming larger models with 11× faster inference than TimesFM-2.0.\n\n**Weaknesses**\n\n1. Currently focuses on univariate time series analysis, with multivariate modeling explicitly noted as a direction for future research.\n\n2. Omits evaluation on widely adopted benchmark datasets (e.g., ETTH1, ETTH2, ETTM1, ETTM2, Weather, Electricity) that are routinely used in state-of-the-art (SOTA) studies [1]. This gap limits direct comparability with prior works and may obscure the model's generalizability across diverse time series modalities.\n\n3. Lacks publicly available codebase for independent verification, which hinders reproducibility and community validation of the reported findings.\n\n[1] Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., & Sahoo, D. (2024). Unified Training of Universal Time Series Forecasting Transformers. In Proceedings of the Forty-First International Conference on Machine Learning.\n\nQUESTIONS:\nPlease make responses to these weaknesses.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-18"}

comments: SUMMARY:
This paper leverages the recently introduced xLSTM architecture to build a time-series foundation model and shows that it outperforms competing models in terms of both efficiency and forecast accuracy.

STRENGTHS AND WEAKNESSES:
**Strengths**:
1. The experiments are well designed, well thought out, and extensive, which lends credibility to the results.
2. The improvements over existing zero-shot models (Chronos, TimesFM, etc.) are quite substantial, and the underlying architecture of TiRex is quite different from transformer-based models, which will help move the field forward.
3. The paper is well written.

**Weaknesses**:
1. There is very little theoretical analysis, it is not clear on a fundamental level why TiRex does better than other models, and what its potential failure modes are.

QUESTIONS:
1. The authors showed that TiRex can predict periodic spikes well, which is a common failure mode for other time-series foundation models. This is not surprising given the specific data augmentation techniques (e.g., spike injection) employed to train TiRex. But how well can TiRex generalize to situations not well covered in the training data. For example, what will TiRex predict when faced with aperiodic spike trains (e.g., from neuroscience)? It would be interesting to test TiRex on synthetic time series where different aspects of the signal can be controlled.
2. What are some failure modes of TiRex that the authors have observed? Understanding these failure modes will help further improve the model.
3. Did the authors find any interpretable strategies TiRex is using to make zero-shot forecasts?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-30

comments: SUMMARY:
There has been lots of recent work on zero-shot time series forecasting with pre-trained time series foundation models. Most of these models are based on transformers, which often struggle with time-series data. On the other hand LSTM are excellent at state-tracking, making them a classic choice for time-series, but they traditionally lack the strong in-context learning abilities of Transformers. This paper aims to bridge this gap. The authors introduce TiRex, a new zero-shot forecasting model that has the state-tracking strengths of LSTMs and the powerful in-context learning ability. The model is built on xLSTM, an enhanced LSTM with strong in-context learning ability. To help address the quantile collapsing problem, the model used contiguous patch masking (CPM) to random mask consecutive patches during training. Multiple training data augmentation methods are proposed to improve the model’s prediction ability. The model achieves state-of-the-art results, outperforms other foundation models such as TimsFM, Chronos, Moirai.

STRENGTHS AND WEAKNESSES:
The paper leveraged new model structure (xLSTM), new training method (CPM) and better data augmentation specifically censor augmentation, amplitude modulation and spike injection to improve state-of-the-art results by a significant amount (Figure 4). Among the three points, I believe 1, 2 are new and the specific data augmentation method is also new although I could be missing something here. In terms of the impact, I think CPM is the most impactful for two reasons 1) addressing collapsing quantiles problems in other models such as Chronos Bolt and Times FM. 2) looking at Table 1, not using CPM seems to lead to the most performance loss.

QUESTIONS:
No

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-06-30

comments: SUMMARY:
The paper proposes a novel LSTM based architecture for pre-trained time-series models that leveraged the xLSTM architecture. The model shows that it can perform well on zero-shot and few-shot forecasting tasks with much lower size, and efficient inference compared to other founadational time-series models

STRENGTHS AND WEAKNESSES:
## Strengths
1. The improvement in performance given the efficiency of the model is very significant.
2. The architecture is quite novel for time-series models. 
3. The evaluation is well done with comparison with state-of-art foundational models and benchmark suites
4. The performance improvement is very significant shocasing that innovations in model backbone is important for founfational time-series models.
5. Ablations on data augmentation, patching are very useful for practioners

## Weaknesses
1. It is a improvement over the xLSTM architecture with additional novelty only in input patching and simple data augmentation

QUESTIONS:
1. Can the model be used for other tasks like imputation, anomaly detection and classification?
2. How does the performance vary with horizon length and frequency of time-series?
3. Can the hidden states be interpreted to model different dependencies and explanations similar to attention modules?

LIMITATIONS:
1. See Weaknesses
2. The model can only do forecasting and not other time-series analysis tasks like imputation, anomaly detection and classification
3. It is not as interpretable as some simpler statistical models
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 2
DATE: 2025-07-03