Title: Investigating Hallucinations of Time Series Foundation Models through Signal Subspace Analysis
Venue: NeurIPS 2025
Accepted: yes
Keywords: foundation models

hallucinations

time series forecasting
Abstract: Times series foundation models (TSFMs) have emerged as a promising paradigm for time series analyses and forecasting, showing remarkable generalization performance across different domains. Despite the efforts made on hallucinations of foundation models, hallucinations of TSFMs have been underexplored in existing literature. In this paper, we formally define TSFM hallucinations in the zero-shot forecasting setting by examining whether a generated forecast exhibits different dynamics from those of the context. Our study reveals that TSFM hallucinations are associated with the loss of context information in hidden states during forward propagation. As such, we propose a methodology to identify signal subspaces of TSFMs and magnify the information through intervention. Experiments demonstrate that our proposed intervention approach effectively mitigates hallucinations and improves forecasting performance. The signal strength measure computed from signal subspaces shows strong predictive power of hallucinations and forecasting performance of the model. Our work contributes to deeper understanding of TSFM trustworthiness that could foster future research in this direction.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses hallucinations in Time Series Foundation Models (TSFMs), particularly in zero-shot forecasting settings, by defining what constitutes a hallucination based on violations of time-series dynamics inferred from the context. It proposes a novel intervention approach (SSIM) that operates on signal subspaces of hidden representations to mitigate hallucinations, and introduces a signal strength score (SSAS) to detect them. Experiments are conducted across multiple TSFMs and datasets.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- Problem Definition: The paper makes an effort to formalize the notion of hallucination in TSFMs, filling a relatively unexplored gap in the literature.\n\n- Methodological Clarity: The intervention mechanism (CPS and SSIM) is well-described, mathematically clean, and efficient to compute at inference time.\n\n- Empirical Evaluation: Experiments are conducted on multiple models and both synthetic and real-world datasets, showing moderate improvements over baselines in both hallucination reduction and predictive performance.\n\n- Potential Impact: If substantiated, the idea that hidden-state subspace structure directly correlates with hallucination likelihood could provide useful insights into TSFM interpretability.\n\n**Weakness**\n\n- The hallucination definition relies on handcrafted rules (trend, frequency, ARMA, etc.) that are domain- and scale-sensitive, and often ill-defined or unstable in noisy, non-stationary time series.\n\n- It is unclear whether failure on these rules always corresponds to a hallucination, or if these mismatches are natural consequences of generalization to difficult signals.\n\n- While the CPS operation is clean, the intervention idea (projecting onto a signal subspace and magnifying) is conceptually similar to many LLM steering methods (e.g., activation steering, neuron editing).\n\n- The main novelty seems incremental in the context of adapting these ideas to TSFMs, and prior work (e.g., [38]) has already discussed intervention in TSFM hidden spaces.\n\n- The baseline comparisons (e.g., denoising, perturbation + averaging) are relatively weak. Stronger baselines, such as diffusion-based or fine-tuned TSFMs with dynamic attention masking, are missing.\n\n- The correlation between SSAS and hallucination is heavily empirical; no theoretical insights are provided about why neuron variance should reliably reflect semantic grounding.\n\n- Many insights are drawn from synthetic datasets (e.g., sinusoidal signals with additive noise), which may not reflect the complex patterns of real-world time series.\n\n- The lack of domain-specific case studies reduces confidence in the generalizability of the findings.\n\nQUESTIONS:\n- How sensitive are the hallucination rules (especially ARMA/pattern-based) to scale normalization and window size?\n- Have you compared SSIM against any stronger TSFM-specific debiasing or attention control methods?\n- Can the proposed signal subspace activity score (SSAS) generalize to classification or anomaly detection tasks?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper defines the hallucination of time series foundation models.  The underlying mechanisms are investigated and the signal strength of hidden states are used for checking of hallucination.  Activity score is proposed to  rank the signal neurons. Then the center-project-scale (CPS) intervention approach is proposed for mitigation of the hallucination is proposed.   It magnifies the signal information in hidden states.  Experimental results have shown the effects of the hallucination detection and mitigation.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is clear written and well organized.  The definition of hallucination in time series based on the knowledge set is reasonable.   The paper presents the observed the patterns of the hidden states and hence form the central idea of this paper.   The only question is whether this signal and noise space of the hidden states can be generalized to all kinds of hallucination and TSFMs.  The proposed hallucination detection and hallucination mitigation methods are clearly presented.  The works are original and it may have substantial impact to future research work in this area.   The use of the knowledge set is important in this work.  Currently, trend, frequency, pattern and ARMA are considered.  It is not clear why pattern and ARMA rules are combined in this study.   Also, is it possible to include other domain knowledge in the knowledge set?\n\nQUESTIONS:\nSection 4.1 describes a hallucination example and illustrates that the hidden states gives us clues about the presence of hallucinated forecasts.  It is not clear if this is just an example or if this can be generalized to most cases.   Also, are all TSFMs showing similar phenomena?  The authors should elaborate this point in order to make the proposed work convincing.\n\nIt is said that type 3 hallucinations is the most common one which include violation of the pattern and ARMA rules. It is also claimed that SSIM has the greatest impact on Type 3 hallucinations.  It is puzzling that the authors did not separate these two types of rules and analyze them individually.  As in terms of prediction, any small adjustments of ARMA coefficients may affect the prediction results.  The adjustment could be caused by the training data in the learning process.    Thus it may be fruitful to study them separately.\n\nThe legend of figure 1 may have the sub-figures (b) and (c) mixed up.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}

comments: SUMMARY:
Disclaimer: While I have experience in time series forecasting and deep learning, I do not have sufficient expertise in the specific techniques or theoretical frameworks employed in this paper (e.g., signal subspace analysis, internal representation intervention). My review may not fully reflect the technical depth of this work, and I encourage the AC to consider assigning an additional reviewer with relevant background.

Recommendation to AC: Given the specific technical depth of this work, particularly in representation manipulation and signal processing, I believe this paper would benefit from an additional review by someone with a strong background in neural representation interpretability or signal-space analysis in deep networks.

This paper investigates hallucination phenomena in Time Series Foundation Models (TSFMs). It proposes a formal definition of hallucination, introduces a method for identifying signal subspaces, and applies test-time interventions to reduce hallucination. The methods are validated on both synthetic and real-world datasets.

STRENGTHS AND WEAKNESSES:
While the motivation is clear and the empirical results appear thorough, I find it difficult to evaluate the novelty and technical correctness of the proposed intervention method and subspace decomposition approach. These appear to rely on internal mechanisms of TSFMs and assumptions that I am not qualified to fully assess.

QUESTIONS:
While the motivation is clear and the empirical results appear thorough, I find it difficult to evaluate the novelty and technical correctness of the proposed intervention method and subspace decomposition approach. These appear to rely on internal mechanisms of TSFMs and assumptions that I am not qualified to fully assess.

LIMITATIONS:
N/A
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores the phenomenon of hallucinations in Time Series Foundation Models (TSFMs), introduces a formal definition in the zero-shot forecasting setting, and proposes a test-time intervention method based on identifying and amplifying signal subspaces in model hidden states.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- Timely Topic: Hallucination analysis in TSFMs is underexplored. Drawing parallels with hallucinations in NLP and vision models is meaningful and relevant.\n- Insightful Hypothesis: The idea that hallucinations stem from diminished signal representation in hidden states is plausible and interesting.\n- Signal Subspace Analysis and Intervention Strategy:  Proposing a Signal Subspace Activity Score (SSAS) and identifying \"signal neurons\" using contrastive analysis adds a fresh angle to understanding hidden states in TSFMs. The proposed test-time intervention (via signal subspace magnification) is simple and avoids retraining, which is attractive for practical deployments.\n\n**Weaknesses**\n\n1. Literature Positioning & Missing Related Work\n - There's insufficient discussion on similar latent space analyses in time series models or comparable intervention techniques in other domains like disentanglement for time series\n\n2. Framing and Definitions\n - The framing of hallucination is imprecise and overly broad in places. For example, defining hallucinations strictly based on deviations from handcrafted rules (trend, frequency, ARMA) is fragile and may not generalize across domains. \n- Definitions (esp. Definition 2 and 3) are complicated and sometimes tautological,  hallucinations are defined via violations of rules, but those rules are subjective and can depend on thresholds chosen arbitrarily.\n\n3. Unclear Methodology\n - The contrastive neuron activity score is underdeveloped theoretically. Why this specific measure? Why should variance across TS steps signify \"signal\" rather than something else (e.g., noise sensitivity or instability)?\n - The intervention mechanism remains vague: What exact operation is applied to hidden states during test-time intervention?\n\n4. Experimental Validation\n - a) The experiments lack rigorous baselines. There's no comparison to other hallucination mitigation strategies or any ablation on how each rule contributes to hallucination detection.\n - b) The evaluation of the proposed SSAS metric as a predictor for hallucinations and forecast quality is unconvincing. Could the authors clarify under which conditions SSIM might fail or be less effective, especially given the varying results of baseline methods? Additionally, how does SSIM’s performance scale with model complexity and computational overhead compared to perturbation averaging? \n - c) There is a lack of variation analysis across different seeds for each method, such as Denoising and Perturbation+Averaging. These methods may be sensitive to hyperparameter settings.\n\n5. Clarity & Presentation\n - a) Clarity and Organization: The paper is cluttered and difficult to follow in several sections. The overall presentation would benefit from clearer structure and more focused explanations.\n\n - b) Figures Not Well Integrated: Many figures, such as Figure 1, are not clearly explained or referenced in the main text. For example, it’s unclear what exactly is being visualized on the right-hand side of Figure 1, and how it supports the authors' claims.\n\n - c) Cosine Similarity Ambiguity: It is not clear what the cosine similarity measures in this context. Is it computed between embeddings, hidden states, or the projection layer weights? This distinction is important for interpreting the results.\n\n - d) Gap in Activation Statistics: The difference in standard deviation between hallucinated and non-hallucinated forecasts appears small. Why is such a narrow statistical gap sufficient to explain the shift from hallucination to accurate forecasting?\n\n - e) Interpretation of Representation Alignment: The authors seem to suggest that when representations are aligned in a certain direction, hallucinations are less likely. If this is the case, it could be an interesting and valuable insight for the community. However, this idea is currently underexplained and would benefit from a more rigorous and intuitive justification.\n\n - f) UMAP Projection: The direction in the UMAP projection (Figure 1, right) corresponding to models that do not hallucinate is not clearly defined. Without a clear definition, it is difficult to assess the significance of the observed alignment and determine whether it meaningfully correlates with hallucination behavior.\n\nQUESTIONS:\nKindly see the questions outlined in the weaknesses section.\n\nLIMITATIONS:\nThis work has some limitations. It focuses on zero-shot forecasting, leaving other probing and finetuning unexplored. Identifying signal subspaces may not generalize to all TSFMs, and the signal strength measure needs further interpretation. The intervention adds test-time overhead, and the evaluation criteria might not cover all types of forecast errors in practice.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}