Title: Glocal Information Bottleneck for Time Series Imputation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Time Series Imputation; Time Series Analysis; Information Bottleneck
Abstract: Time Series Imputation (TSI), which aims to recover missing values in temporal data, remains a fundamental challenge due to the complex and often high-rate missingness in real-world scenarios. Existing models typically optimize the point-wise reconstruction loss, focusing on recovering numerical values (local information). However, we observe that under high missing rates, these models still perform well in the training phase yet produce poor imputations and distorted latent representation distributions (global information) in the inference phase. This reveals a critical optimization dilemma: current objectives lack global guidance, leading models to overfit local noise and fail to capture global information of the data. To address this issue, we propose a new training paradigm, **Glocal** **I**nformation **B**ottleneck (**Glocal-IB**). Glocal-IB is model-agnostic and extends the standard IB framework by introducing a Global Alignment loss, derived from a tractable mutual information approximation. This loss aligns the latent representations of masked inputs with those of their originally observed counterparts. It helps the model retain global structure and local details while suppressing noise caused by missing values, giving rise to better generalization under high missingness. Extensive experiments on nine datasets confirm that Glocal-IB leads to consistently improved performance and aligned latent representations under missingness. Our code implementation is available in [https://github.com/Muyiiiii/NeurIPS-25-Glocal-IB](https://github.com/Muyiiiii/NeurIPS-25-Glocal-IB).


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nSummary: The authors propose a model-agnostic deep learning approach for time series imputation based on the Information Bottleneck (IB) theory. \n\nStrengths: 1. The proposed method is model-agnostic, enabling compatibility with various architectures. 2. It demonstrates superior imputation accuracy compared to common time series models. 3. The writing is clear and the narrative flow is accessible to readers. \n\nWeaknesses: 1. Limited innovation: Global Mutual Information Maximization and related solutions build on prior work without substantial novel contributions. 2. Motivational concerns: Does time series imputation inherently require temporal semantic meaning? The paper fails to quantify how such semantic information benefits real-world applications, lacking concrete justifications. 3. Inadequate comparative experiments: The study lacks sufficient baselines to rigorously validate the method’s effectiveness.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: 1. The proposed method is model-agnostic, enabling compatibility with various architectures. 2. It demonstrates superior imputation accuracy compared to common time series models. 3. The writing is clear and the narrative flow is accessible to readers. \n\nWeaknesses: 1. Limited innovation: Global Mutual Information Maximization and related solutions build on prior work without substantial novel contributions. 2. Motivational concerns: Does time series imputation inherently require temporal semantic meaning? The paper fails to quantify how such semantic information benefits real-world applications, lacking concrete justifications. 3. Inadequate comparative experiments: The study lacks sufficient baselines to rigorously validate the method’s effectiveness.\n\nQUESTIONS:\nQuestions: 1. Appendix Equation 16 contains a mispositioned multiplication sign in the second line, potentially causing interpretational errors. 2. Reference formatting is inconsistent; underlines in citations should be removed. 3. Missing visualizations: The main text lacks graphical representations of Glocal-IB's performance dynamics. 4. Unfair baselines: The comparative experiments rely on prediction-focused models rather than imputation-specific architectures. At least 4–6 imputation-oriented models  should be added to ensure valid comparisons. 5. Ambiguities in Table 1: Define \"IMP\" and clarify the methodology for generating missing data patterns. 6. Flawed conclusion in Observation 5: TimesNet and SAITS already exhibit strong performance without Glocal-IB (e.g., SAITS maintains similarity to original data at 50% missing rate). Glocal-IB introduces minimal improvements, undermining the claim of significant enhancement. \n7. Unexplained mechanism in Observation 7: Why does a regularization weight of 0.01 degrade model performance? The authors must provide theoretical analysis of this phenomenon.\n8. Over-reliance on complex models: The study focuses on TimesNet, a sophisticated architecture. Adding analyses of simpler models (DLinear, MLP, TCN, Transformer) would mitigate model-specific biases.\n9. The publicly available code by the authors is incomplete.\n\nLIMITATIONS:\nNo", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses time series imputation (TSI) under high missingness, observing that many models achieve low training loss (focusing on pointwise reconstruction) but perform poorly at inference. The authors identify an “optimization dilemma”: under severe missingness, models overfit local noise and fail to capture the global structure of the data. To remedy this, they propose the Glocal Information Bottleneck (Glocal-IB) training paradigm, which extends the information bottleneck framework by adding a Global Alignment objective. Concretely, Glocal-IB applies the model’s encoder to both the original (fully observed) sequence and the masked sequence. A contrastive/InfoNCE-style loss (via a one-layer MLP projector) aligns the latent of the masked input with that of its unmasked counterpart, and a KL-divergence term regularizes the latent (suppressing noise). In effect, the model is trained to retain global, semantic-level information (from the full data) even when much of the input is missing.\n\nExperiments on real-world datasets show that Glocal-IB consistently reduces imputation error compared to strong baselines. For example, on the ETT benchmarks, Glocal-IB achieves the lowest MAE and MSE in all cases, with up to 40% reduction in MSE over prior methods. The authors also visualize latent representations: without Glocal-IB, the latent distribution collapses under high missingness, whereas with Glocal-IB, the latent remains well-structured up to 90% missing. In summary, the key contribution is a model-agnostic training strategy that enforces alignment between masked and original latents, which empirically yields more robust, generalizable imputation under extreme missing data.\n\nSTRENGTHS AND WEAKNESSES:\n- Quality\n  - Strengths\n    - The paper does an excellent job of identifying and illustrating the \"optimization dilemma\" with clear empirical evidence (Figure 1). This convincingly frames the core problem that existing methods, which focus solely on local reconstruction, fail to learn generalizable representations when data is sparse. This insight itself is a valuable contribution.\n    - The authors clearly identify a practical failure mode of existing TSI methods (good training fit but poor test imputation) and back it with diagnostic plots. The Glocal-IB method is well-specified: adding a contrastive alignment loss and KL regularizer to the standard encoder–decoder framework. Experiments are extensive (nine datasets, five missing rates, five random seeds), and both quantitative results (MAE/MSE tables) and qualitative latent plots are provided. Results indicate strong improvements across diverse datasets, suggesting robustness. Code is made available, aiding reproducibility.\n\n  - Weaknesses:\n    - The choice of baselines is incomplete: notable recent methods for generative imputation are missing. Comparing to Conditional Information Bottleneck (CIB) or other IB-based imputation would strengthen claims of superiority.\n- Clarity\n  - Strengths:\n    - Overall, the writing is clear and systematic. Key ideas (local vs. global information) are introduced in plain language, and Figure 1 nicely illustrates the problem. Figure 2 then contrasts the proposed paradigm with baselines. The Appendix is comprehensive (including a Societal Impact Statement and detailed limitations, per NeurIPS requirements).\n  - Weaknesses:\n    - The information-theoretic notation (Section 3.1) may be heavy for readers unfamiliar with IB, and the derivation of the InfoNCE alignment loss (Eq. 12–13) could use more intuition.\n    - The technical exposition is dense (heavy on IB theory and mutual information bounds); while Figure 2 helps, some readers may find the derivations hard to parse. A clearer summary of the final loss terms and training procedure could improve readability.\n\n- Significance\n  - Strengths:\n    - This work addresses a highly significant and practical problem in time series analysis. Imputation under high missingness is a common challenge, and the proposed model-agnostic training paradigm offers a robust solution. The insights into the failure modes of current TSI methods are also an important contribution to the field. The demonstrated improvements could have a considerable impact on applications in healthcare, finance, and energy systems.\n  - Weaknesses:\n    - The ablation study (Figure 5) shows that the model's performance is quite sensitive to the weight of the regularization loss (LRegθ​). While the authors acknowledge this, the paper would benefit from a more detailed discussion on the practical aspects of tuning the loss weights (α,β1​,β2​). Without a clear strategy, practitioners might find it challenging to apply the method effectively to new datasets or architectures, which could be a minor barrier to adoption.\n- Originality\n  - Strengths:\n    - The core idea — combining an information bottleneck with a global latent alignment — appears novel in the TSI literature. Prior work on IB for TSI (e.g. Choi & Lee’s CIB) still relied on local reconstruction losses, whereas here a separate global mutual-information objective is added. The paper’s formulation (alignment via InfoNCE between full vs. masked embeddings) seems new.\n\nQUESTIONS:\n1. **Computational Overhead:** You describe Glocal-IB as a \"lightweight\" paradigm. Could you please quantify the computational overhead? Specifically, what is the increase in training time and memory usage when applying Glocal-IB to a backbone like the Transformer, compared to its standard training?\n2. **Guidance on Hyperparameter Tuning:** Your sensitivity analysis shows that performance can depend significantly on the choice of loss weights, especially the regularization term. Could you provide some practical guidance or a heuristic for setting the hyperparameters α,β1​, and β2​ when applying Glocal-IB to a new dataset or model architecture? A more robust tuning strategy would significantly enhance the method's practical utility.\n3. **Characterizing \"Global Information\":** The Global Alignment loss is motivated by preserving \"global semantic features.\" Beyond the excellent t-SNE visualizations, can you provide more insight into what specific properties this alignment captures? For example, does it preferentially preserve long-term trends, periodicity, or specific types of inter-channel correlations?\n4. **Comparisons to other imputation methods:** The paper omits an empirical comparison with some recent imputation frameworks. How does Glocal-IB perform relative to diffusion-based methods (e.g., CSDI) or GAN-based imputation (e.g., ImputeGAN)? Including one of these baselines could strengthen the experimental claims. Similarly, how does Glocal-IB differ from the Conditional IB approach of Choi & Lee? A brief discussion or additional results would be helpful.\n5. **Training details – teacher/stop-gradient:** In Figure 2(c), the encoder processes the original data to produce a “target” latent Z′. Is this encoder a frozen copy or a separate network? In other words, during training, is the latent from the original sequence allowed to update the encoder weights? (The stop-gradient symbol suggests no.) Clarifying the implementation of the “alignment” (e.g., is the original-branch encoder a fixed copy?) would help. Also, is there a risk of “leaking” information from the original into the masked branch that could lead to overfitting?\n6. **Generalization to non-random missingness:** The experiments use random masking patterns. How would Glocal-IB handle more structured or non-random missing patterns (e.g., entire variables missing for long periods)? Since the global alignment uses the same sequence’s full version as the positive, if missingness occurs in long gaps, the alignment signal might be weak. Can the authors comment on this, or test a few non-Random scenarios?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents Glocal-IB, a novel training framework for Time Series Imputation (TSI) that tackles a core optimization issue: under high missing rates, existing models often minimize training loss but fail to generalize, leading to distorted latent embeddings and inaccurate imputations. Glocal-IB extends the Information Bottleneck (IB) principle by incorporating a Global Alignment (GA) loss, which enforces consistency between latent representations of masked and fully observed inputs. This guides the model to preserve both global temporal structure and local signal fidelity, while mitigating noise. The approach is model-agnostic, lightweight (adding only a single MLP), and demonstrates superior imputation accuracy and latent space stability across nine benchmark datasets, consistently outperforming state-of-the-art methods.\nHowever, overall, this paper’s concept is rather vague and not well motivated. It may have limited novelty for NeruIPS.\n\nSTRENGTHS AND WEAKNESSES:\n•\tStrengths:\nStrong theoretical foundation with clear derivations and mutual information approximations.\nComprehensive empirical validation across 9 diverse datasets and missingness levels.\nAblation and sensitivity analyses support the importance of each component.\nDemonstrates robustness and generalizability across multiple model backbones.\nThe combination of local and global mutual information objectives is innovative.\n•\tWeaknesses:\nPerformance gains under extreme missingness (e.g., 90%) are less pronounced.\nEvaluation is limited to a few model architectures due to computational constraints.\nSome mathematical derivations are dense and could benefit from more intuitive explanations or diagrams.\nBuilds on existing IB and contrastive learning principles, though the integration is novel.\n\nQUESTIONS:\n1.\tThe paper notes that Glocal-IB is less effective at 90% missingness. Could the authors explore or suggest mechanisms (e.g., stronger priors or pretrained encoders) to improve performance in such cases? How reliable are studies exploring missing at such high level?\n2.\tHow does Glocal-IB scale with longer sequences or higher-dimensional time series? Any insights on computational overhead?\n3.\tThe paper compares Glocal-IB with foundation model alignment. Could the authors elaborate on why foundation models underperform and whether combining both approaches might help?\n4.\tCan the authors comment on the interpretability of the learned latent space or the imputed values, especially in high-stakes domains like healthcare?\n5.\tCould Glocal-IB be extended to other structured data imputation tasks (e.g., spatiotemporal graphs or tabular data)?\n6.\tCombine Glocal-IB’s global-local mutual information loss with a flexible’s per-variable model selection to improve robustness under high missingness.\n7.\tAdjust the KL divergence or alignment loss weights based on variable skewness to avoid over-penalizing informative but skewed features.\n8.\tExtend Glocal-IB to support multiple imputations and ensemble predictions to reflect uncertainty.\n\nLIMITATIONS:\nThe authors discuss limitations related to extreme missingness and model coverage, and suggest future directions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles a critical optimization dilemma in time series imputation where models achieve low training loss under high missing rates but suffer substantial degradation in imputation quality and distorted latent representations during inference. The authors propose Glocal Information Bottleneck (Glocal-IB) as a model-agnostic solution that extends the Information Bottleneck framework. Validated across diverse datasets, Glocal-IB demonstrates good performance.\n\nSTRENGTHS AND WEAKNESSES:\nS1. The paper compellingly diagnoses a critical limitation in time series imputation methods where models minimize training loss under high missing rates yet produce distorted latent representations and poor inference performance.\nS2. Glocal-IB extends the Information Bottleneck framework with a novel Global Alignment loss derived from tractable mutual information approximation. This innovation aligns latent representations of masked inputs with original observations while maintaining local reconstruction capabilities.\nS3. The paper presents a thorough evaluation across nine datasets, demonstrating consistent improvements over baselines, particularly under high missing rates. Latent space visualizations provide empirical support for the claim that the proposed method better preserves distributional structure compared to existing approaches.\n\nW1. While the paper is motivated by an insightful observation on memorization under high missing rates, the choice of the Information Bottleneck (IB) as the solution is not sufficiently justified. Table 1 shows that IB-based methods like GPVAE do not consistently outperform non-IB approaches. It remains unclear why IB is preferred over alternative strategies that promote global structure (e.g., multi-directional self-attention [1]).\nW2. The paper does not clearly report the specific values of \\beta_1 and \\beta_2 used in the experiments, nor does it provide practical guidance for selecting them. Given that the sensitivity analysis indicates the method’s performance is strongly affected by these parameters, it would be helpful to include tuning heuristics or recommendations to support practical adoption.\nW3. The paper claims that high missing rates lead to memorization, where models achieve low training loss but fail to generalize due to poor semantic representations. However, Figure 1 does not clearly demonstrate increased overfitting under high missingness; in fact, the training dynamics appear similar across missingness levels. This raises concerns about the strength of the empirical evidence supporting the central hypothesis.\n\n[1] Suo, Q., Zhong, W., Xun, G., Sun, J., Chen, C., & Zhang, A. (2020, December). GLIMA: Global and local time series imputation with multi-directional attention learning. In 2020 IEEE International Conference on Big Data (Big Data) (pp. 798-807). IEEE.\n\nQUESTIONS:\nQ1. Could the authors further elaborate on why IB is preferred over other strategies that encourage global structure in representations?\nQ2. How should users determine an appropriate value of \\beta_1 and \\beta_2 when applying this method to real-world datasets, especially those with unknown or varying missingness characteristics? Could the authors suggest a principled method, heuristic, or adaptive tuning strategy for selecting \\beta_1 and \\beta_2 in practice?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-05"}