Glocal Information Bottleneck for Time Series Imputation
Authors: Jie Yang
Kexin Zhang
Guibin Zhang
Philip S. Yu
Kaize Ding
Venue: NeurIPS 2025 poster ()
Keywords: Time Series Imputation; Time Series Analysis; Information Bottleneck
Primary area: deep_learning

Abstract: Time Series Imputation (TSI), which aims to recover missing values in temporal data, remains a fundamental challenge due to the complex and often high-rate missingness in real-world scenarios. Existing models typically optimize the point-wise reconstruction loss, focusing on recovering numerical values (local information). However, we observe that under high missing rates, these models still perform well in the training phase yet produce poor imputations and distorted latent representation distributions (global information) in the inference phase. This reveals a critical optimization dilemma: current objectives lack global guidance, leading models to overfit local noise and fail to capture global information of the data. To address this issue, we propose a new training paradigm, **Glocal** **I**nformation **B**ottleneck (**Glocal-IB**). Glocal-IB is model-agnostic and extends the standard IB framework by introducing a Global Alignment loss, derived from a tractable mutual information approximation. This loss aligns the latent representations of masked inputs with those of their originally observed counterparts. It helps the model retain global structure and local details while suppressing noise caused by missing values, giving rise to better generalization under high missingness. Extensive experiments on nine datasets confirm that Glocal-IB leads to consistently improved performance and aligned latent representations under missingness. Our code implementation is available in [https://github.com/Muyiiiii/NeurIPS-25-Glocal-IB](https://github.com/Muyiiiii/NeurIPS-25-Glocal-IB).

Glocal Information Bottleneck for Time Series Imputation
Jie Yang! Kexin Zhang Guibin Zhang', Philip S. Yu', Kaize Ding"
University of Illinois Chicago
Werthwestern University
National University of Singapore
Primary contact:
jyang265uuic.edu
Abstract
Time Series Imputation (TSI), which aims to recover missing values in temporal data, remains fundamental challenge due to the complex and often high-rate miss- ingness in real-world scenarios. Existing models typically optimize the point- wise reconstruction loss, focusing recovering numerical values (local information). However, we observe that under high missing rates, these models still perform well in the training phase yet produce poor imputations and distorted latent rep- resentation distributions (global information) in the inference phase. This reveals critical optimization dilemma: current objectives lack global guidance, leading models to overfit local noise and fail to capture global information of the data. To address this issue, we propose new training paradigm, Glocal Information Bottleneck (Glocal-IB). Glocal-IB is model-agnostic and extends the standard IB framework by introducing Global Alignment loss, derived from a tractable mutual information approximation. This loss aligns the latent representations of masked inputs with those of their originally observed counterparts. helps the model retain global structure and local details while suppressing noise caused by missing values, giving rise better generalization under high missingness. Extensive experiments nine datasets confirm that Glocal-IB leads to consistently improved performance and aligned latent representations under missingness. Our code implementation is available in https: /ttps://github com/ wwt///yyy111i//eerrrrPee. 25- 155--1- IB
Introduction
Missing values are pervasive in real-world time series due to device malfunctions, transmission failures, and manual collection errors [52 72 These missing values occur with varying rates and
patterns across domains such as healthcare 69 41 43., transportation [25), and energy systems [18
20]. thus substantially impairing the integrity of time series data and the performance of downstream tasks [1]. Consequently, Time Series Imputation (TSI), which aims to reconstruct missing values from partially observed data, has emerged critical problem with broad practical significance (51).
Missing values disrupt the original structure time series data, acting as structured noise that corrupts temporal dependencies and statistical patterns [27 34] To address this, existing TSI methods typically adopt dncoder-decoder architectures [30 () trained by randomly masking observed values to simulate missingness [58] 13 42). The goal is to learn the global data distribution from corrupted observations, enabling the model to reconstruct masked values during training and serve as a conditional generative model for imputation inference time [56] 53). However, critical optimization dilemma has emerged in this paradigm: Under high missing rates, models achieve training losses comparable
'Work done during internship Northwestern University. CCoresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Original
Miss 10%
Miss 30%
Miss 50%
Miss 70%
Miss 90%
BAT 936
B.
(a) TimesNet
a5
Bus
8:71
30 epeclss
8.04
RAY
0.87
(b) TimesNet
100 epochs
8.70
846
1.0
(c) GIVAA
30 epochs
Traia MAE Text MAR
Reconstruction
stable reconstruction
Imputation
worse imputation & more distortion
Figure 1: Illustration of optimization dilemma in TSI. We visualize the latent space of two representative models-Timesshe (a-b) and GPVAE (c) (rrined under different missing rates and training epochs. Training and test losses are shown in green and orange boxes, respectively,
-iwmmissinnness scenarios. suggesting successful convergence, yet suffer drastic performance degradation in imputation quality. To investigate this discrepancy, we conduct an empirical analysis of representative TSI models under varying missing rates. Our results highlight gap between latent representations learned during training and their utility for accurate imputation. Fig. T (a-b) illustrates this phenomenon using TimesNet [60] (a-b) and GPVAE [44 (c) on the ETTh1 dataset 75]; additional results are provided in the Appendix D.I Our findings highlight two key phenomena:
Low training loss does not necessarily imply good imputation. As the missing rate increases, the model still performs well in reconstructing the training data, but their inference-time imputation quality drops substantially. Even more surprisingly, reducing the number of training epochs (re- sulting in slightly higher training loss) achieves better imputation results during inference. This suggests that the training objective under high missingness fails to guide the model toward gen- eralizable representations, encouraging memorization of local observations rather than learning meaningful global structures and information.
Well-aligned representations are strongly related to good imputation. We further visualize the latent space distributions of the models and observe that better imputation performance corre- sponds to representations that remain well-aligned with those derived from fully observed data, However, as missingness increases, the distributions become increasingly distorted, despite low training losses. This distortion correlates with poor imputation, suggesting that reconstruction losses (e.g.. MAE/MSE) fail to preserve globally coherent structure under severe missingness.
These observations suggest that a fundamental limitation of current TSI methods lies in their training objectives. The focus local numerical accuracy each timestamp makes these models sensitive to temporal noise and redundant patterns (53] 7 744, hindering their ability to capture the underlying global distribution. To address this issue, researchers have explored the Information Bottleneck (IB) principle [19] [ 47]. which encourages representations that discard irrelevant noise while preserving task-relevant information. However, most IB-based TSI methods [4 7 still rely on local recon- struction loss to increase tesk-relevant mutual information, which is inadequate for capturing global structure. As result, these models remain vulnerable the same optimization dilemma. For exam- ple, GPVAE [14] as shown in Fig. (c), suffers from severe latent space distortion and performance degradation as the missing rate increases. Its MAE degrades to 0.8, similar to the non-IB -o--edd TimesNet 660 under the same conditions. Therefore, key research question is raised: Can we design a training paradigm that encourages TSI models to capture both global and local information from incomplete data, without overfitting to noise?

---

Our approach. To answer this question, we propose new training paradigm, Glocal Information Bottleneck (Glocal-IB), which is based on the trade-off between compactness (suppressing noise) and informativeness (preserving both global and local information). Unlike previous IB-based methods [35 21 32] that rely solely on reconstruction losses to increase the mutual information between latent representations and imputation targets, Glocal-IB goes one step further. Specifically, it extends the standard IB framework by introducing a Global Alignment loss, derived from a tractable mutual information approximation. This loss aligns the latent representations of masked inputs and their corresponding original inputs. Remarkably, Glocal-IB requires only a single Multilayer Perceptron (MLP) to implement the alignment loss, making it model-agnostic and easily integrable into existing encoder-decoder frameworks.
The main contributions of this paper are as follows:
We identify a critical optimization dilemma in existing TSI methods: under high missing rates, models achieve low training loss but fail to learn globally semantic latent representations, leading to substantial degradation in imputation quality and severe latent space distortion.
We propose a novel IB-based training paradigm, Glocal-IB, which explicitly enforces latent space consistency via lightweight global alignment loss, alongside local reconstruction, thereby improving both global and local feature learning while removing irrelevant noise.
Our empirical results validate the effectiveness of Glocal-IB. On nine benchmark datasets, consistently achieves top imputation performance and helps form smooth, structured latent space when applied to a vanilla Transformer. Similar improvements are observed across other backbones, showing strong generalization and better robustness under high missing rates.
2 Related Work
Time Series Imputation TSI has received increasing attention due to its critical impact in real-
world applications [44 76 61. Most recent methods adopt an encoder-decoder architecture (53) differing mainly in how they capture temporal dependencies. RNN-based models like GRU-D [ and
BRITS (3] handle temporal decay and bidirectional inference, while transformer-based models such as ImputeFormer (36) use attention mechanisms for long-range temporal modeling. Spatial correlations across variables are modeled by methods like GRIN (8 and SPIN [31), which integrate Graph Neural Networks (GNNs) [2 49). Moreover, CSDI 46) GPVAE [4), CIB [7], and USGAN [33 are
proposed to learn probability distributions from the observed data. Despite architectural progress, most models remain sensitive to noise and temporal redundancy in the observed values [42 [7].
due to the point-wise reconstruction loss. To address this, we introduce a new training paradigm, Glocal-IB, with dual emphasis on global structure and local detail, thereby encouraging semantically stable representations and improving imputation under severe missingness.
Optimization Dilemma Recent studies have observed a mismatch between low training loss and poor test-time performance. For instance, in latent diffusion models (LDMs) for computer vision [12] 67 (0), high-capacity models produce over-concentrated latent spaces that capture low-level details at the cost of semantic coherence. Solutions to this in computer vision, such as VA-VAE [67 and REPA 70,, leverage vision foundation models [39 15
16 to align the latent space, promoting richer semantics. However, time-series foundation models 45
29 54 are primarily trained with predictive reconstruction losses and lack sufficient semantic information needed to mitigate this issue. To address this, we introduce a Global Alignment objective that encourages the latent representations of masked observed sequences to remain close to those of their original observed counterparts. In addition, compared to solutions that rely on foundation models, our approach is lightweight and efficient, requiring only one extra MLP.
3 Methodology
Problem Definition. Given an original multivariate time series X = { 1 - N) E N where N is the number of variables and T is the sequence length. To simulate missingness, binary mask M E (0, 1) NxT is applied, where M = 1 indicates that 2 is observed and M indicates it is missing. The masked input is then defined as X = X @ M. TSI models outputs the imputed result X NxT based on the X aiming at estimating the missing values in X
3

---

Recomtrucion Lous
Reconsruction
Reconstruction Lows
gradiast
gradiost
gradient
Decoder du
Decoder
Decoder
stop- grad
stop-grad
Z
Z
Proj
Similarity
Z'
Proj
Allgument
Z'
Loss
Encoder
Enceder
Foundation
Model
KL Loss
Encoder
Encoder
Only Trate
Only Train
Masked Dota: X"
Manual Mask
Original Datn: X
Masked Doca: X"
Manual Mask
Original Datn: X
Masked
Manual
Original
Doto:
Mask
Doca:
(a) Current TSI Training
(b) Foundation Model Alignment
(c) Our proposed Glocal-IB
Figure 2: Framework comparison of three TSI training paradigms. Three paradigms differ in how to deal with the latent representations and how the key encoder and decoder are updated. (a): The encoder and decoder are updated end-to-end by back-propagation of reconstruction loss. (b): The latent representations are aligned with a frozen time series foundation model with original data. (c): Glocal-IB utilizes encoder itself and KL divergence to regularize the latent representations.
Throughout this paper, we refer to X as the imputation target, X as the masked input, and X as the model's s imputation result.
Information Bottleneck Theory for Time Series Imputation IB principle provides a theoretical framework for identifying informative parts of the input by balancing two competing objectives: compactness (regularization) and informativeness (task performance). This trade-off shapes the latent representations to retain only the essential structure for solving a specific task.
Let X and y denote the original input data and the targets of a specific task, respectively. To get the balance between regularizing input data and maintaining good performance, there is well-designed formula about XBB ylB and the bottleneck variable 218 as follows:
min [T(Z (B 3 I( 21
(1)
where I(2 x1B and T( IBB) represent the mutual information of (Z x and (Y-18 ZTB), and E R is Lagrange multiplier that balance the two mutual information. This offers good understanding of what contributes most to the task from an Furthermore, according to the previous IB literature [I] [4], we assume factorization of the joint distribution as follows:
perspective.
p(x1B 1B /BB PB
X y-18 )p (V 1 )p p(7 XB )p (Y | )p( X (2)
namely, there is a Markov chain y1B + X ++ IB indicating that the latent representations ZBB not directly depend the targets y1B, Then, following Eq. (I). can define the TSI supervised task as follows:
min[lo(Z:X') - IxX; 0.
where 3 E I and Z E R.N xd denote preset hyperparameter and the latent representations, respectively. and denote the learnable parameters of the encoder (+(-) and decoder g. of our method Glocal-IB. Therefore, we can accomplish the TSI tasks by modeling crucial information from the partially observed data while filtering out redundant noise.
3.1 Overview
In this section, we introduce our proposed training paradigm Glocal-IB, which is grounded in the IB principle, and present the derivation of two components in Eq As shown in Fig (c), Glocal-IB is simple to use, adds only one MLP projector for alignment. and can be applied to a wide range of existing methods. Glocal-IB aims to balance two goals in the latent space: reducing noise and retaining both global and local information. To achieve this, it minimizes the mutual information between the masked input X and the latent representations Z, which helps remove noise introduced by incomplete data. Mcanwhile, it maximizes the mutual information between Z and the imputation

---

target X, to capture both fine- aiteggriieed local details and global semantic features. This combination encourages the model learn well-aligned representation of the original data distribution, thereby addressing the aforementioned optimization dilemma and achieving accurate imputation.
3.2 Regularizing Partially Observed Input: min (2;
Based on variational inference [50], we derive an upper bound for the regularization term in Eq. The full derivation is shown in Appendix A.T
T(z; J pla"). (x..p(2(<"))l(())) dx -s p(>))2) DKL. Pxa.pp(=)ll(() de",
(4)
- J p(x)). DxLLp(t/> de = pppcr DKL (
where the inequality follows from the non-negativity of KL divergence. Due the difficulty in posterior calculation, we use our encoder po(z æ²) to approximate the true posterior distribution p(z 2²), so that the Regularization loss is defined follows:
T(Z; X²) Bap(ar) (=)(p
LReE
(5)
Meanwhile, we set an isotropic Gaussian as the prior distribution of the latent representations Z, i.., p p(Z) = (0, I). Therefore, the encoder is defined model partially observed time series data
through a multivariate Gaussian distribution shown below:
po(2 = (Ha((X) (xa)(
(6)
where (+).) and (-(-) are designed neural networks with parameter 9. During inference,
the latent variable as Z = μl (X"), and sample from the approximate posterior ~ po (Z Xº)
using the reparameterization trick:
Z =e -x("))
where E (N, 1) and denote element-wiss multiplication. Under this formulation, the Regular-
ization loss in Eq. can be computed and differentiated analytically as follows, without the need for stochastic estimation (23]:
1 danne DKL.
Σ j=1
1+ 1+ log
(8)
2
Here, dmedall denotes the dimensionality of the latent representations, and (1) (Xx) and o ( (x9) represent the j-th elements of the mean and standard deviation vectors, respectively.
3.3 Maximizing Global and Local Inforamtion: max (X; Z)
Local Mutual Information Maximization Following the derivations introduced in previous IB- relevant literature [7] [ we can obtain a lower bound for the informative term, which aims to maximize the mutual information between the latent representations Z and the original data (full
derivation is illustrated in Appendix A.2.
log [0(x|=)]
I(X; 2) = Ep(z,))
p(r) + / (x.Ip(al)))loo dz, 2 Ep(,,s) [log qu (x|=)] Epiz,)) [logp(x)],
(9)
2 Ep(,,)) [log qo (x|=)]
def -C C
where the inequality holds due the non-negativity of KL divergence and entropy. As we assume that time series data follow a Gaussian distribution with fixed variance 23,, i.e, qs (z//=) = (Nt the derived Local loss can be further reduced the form of MSE loss follows:
T
L -p((x,)) [log qo (x|22) Ep(x,))
|
112
2 log(2mo²)
20
(10)
x Ep(z.)) [||e - 21
5

---

where å denotes the imputation results generated by the model, and T is the length of the time series. However, although this MSE-based Local loss provides valid way maximize [[x; Z). inherently emphasizes accurate reconstruction of local numerical values. These values often contain noise introduced by data collection errors and provide little guidance at the global level. Consequently, under high missing rates, the model tends to memorize these noisy details rather than learn the true data distribution, leading to poor generalization, degraded imputation quality, and severe distortion in the latent space. We identify this noise memorization the key reason why both non-IB and IB-based TSI methods fail such settings.
Global Mutual Information Maximization To overcome the limitations of point-wise reconstruction losses, we introduce a complementary formulation that explicitly targets the global (semantic-level) mutual information between the latent representations Z and the original data X. Inspired by the InfoNCE objective from contrastive learning [38] we derive an alternative lower bound of
T(X; Z) (full derivation is illustrated in Appendix A.2.2
p(ar)
(p(x|z) NN) ]
p(x)
[(x; 2) = -1E [p(,,)))
log
NN)
log N
2 E pla,s)
log
p(<|z))
p(a|z) p(z)
(11)
2 ()t)))
log
p(z|=]
piz, p(R.)
Instead of reconstructing directly with a decoder 4s (x12), we model a density ratio f(x, 2)
exp(proj(=)T -po (a:) that preserves mutual information T(X; 2), as it is proportional to p(a[z) And p(z)
we denote Z' po()). This yields the following Global Alignment loss:
f(z, 2) f(z, 2) + Σ f(zz))) 8,EXa
I(X; 2 Ep(a.s)
log
def
Gio
(12)
where proj(-) and (o(-) are simple one-layer MLP and the model's encoder, respectively. Since our goal is to maximize global semantic-level mutual information, we treat the embedding of the original data at the same timestamp the positive sample for the partially observed input. For negatives, use embeddings from other timestamps of the original data. This setup pushes the model to align partially observed inputs with their original counterparts, encouraging it to capture semantic-level features such as temporal dynamics and global data distribution.
Moreover, considering the evolution of the training paradigm of the contrastive learning 15 can further simplify the Global Alignment loss L to simple alignment loss as follows:
5
we
G Ep, [f 2)] Ep(,,)
[exp (proj(z) enc(x))] def
(13)
3.4 Overall Training Objective
We now present the overall training objective of our proposed training paradigm Glocal-IB. This framework is simple apply any encoder-decoder architecture and requires only one additional
MLP. By combining all components, including Regularization loss Cleys Local loss L and Global Alignment loss C we optimize the time series imputation objective defined in Eq.
min a C 31 LL.c + 82 f
(14)
0.6
where a, 31, and B2 are hyper-parameters that balance the mutual information. Global Alignment loss Lg can be implemented by L or C%
Experiments
4.1 Experimental Settings
Datasets: Comprehensive experiments are conducted on nine public time-series datasets (59 75 26 73,, including ETTh1, ETTh2, ETTml, ETTm2, Beijing Air, PEMS-Traffic, Electricity, Weather, and

---

Table Imputation performance on 9 datasets (average MAE and MSE across 10% to 90% missing rates). Best is bold and second-best is underlined. We use OOM to denote out of memory.
Models
IMP
Ours
SAITS
Transformer
DLinear
TimesNet
FreTS
PatchTST
ITuunffrmer
GPVAE
TimeMixer
Metric
MSE
MAE MSB
MAE MSE
MMAE MSE
MAE MSE
MAE MSE MAE MSE
MAE MSE
MAE MSE
MMA MSE
MAE MSB
ETTM
38%
[0.283 0.197 (1.402 0.376 0.399 0.373 390 0.316 0.602 o. (02/ 0. 0.394/00224 0.780/0.441 0.406
(.7731 0.928|0.678 0.88
ETTa2
40%
(0.249 0.132
0,340 0.256 00307
0.218
0.352
0.243 0.800 1.140|0.434 0.370|0.525
0.575 0.321
[06686
0.769l0.529 0.535
ETTml
28%
[0.157
0.069
0.206
0.099
(0.202
0.096
(1:284
0.172
0,789 1.008
0.234
0:208
(0.588
0.627/0
0.242
ETTm2
25%
(0.157
0.069
0.206
0.099
(.2002
0.096
0.284
0.172
0.789
0..100
0.195
0.194
0.18
0.315
0.208
(05888
0.627 0.359 0.242
Beijing Air
7%
|0.223 0.320
0.256
0.353
0.2668
0.375
(0.279
0.338 0.264 o. 0.370/0.289
0.349
0.300
0.472
0.366
0.473
0.330
0.448 0.628
PEMS: Traffic
6%
|0.318 0.630)
0.336
0.674
(0.355
0.695
|0.401
0.696 0.336 0.683 (0.441
0.745
0.870
OOM
[0.383
0.650|0.528
Electricity
8%
|0.372 0.296 0.397 0.343
(0.410
0.358
0.443 0.433 0,390 p 0.322 (0.544
0.534
0.783
0.363
(0.443
0.394|0.648
0.724
Weather
34%
|0.0%6
0.056 0.056
0.093
139
0.091
(0.161 0.089 0.262 0.211 10.167 0.08
188
0.11/
0.102
J0.278 0.195/0.239 18180
Men-LA
17% (0.267 0.293 0.301 0.392 (0.306 0.387 0.387 0.412 0.289 o 0.354/0.414 0.453/0.423 0.544/0.427 0.477 |0.420 0.465/0.607 1.000
During the experiments, we follow the point-wise missing patterns to randomly mask the time series [I]]. We follow the standard train/validationntest splits provided by PyPOTS| 9. More
LA.
details are shown in the Appendix [
Baselines: We select nine representative time series methods as our baselines, including: (1) Transformer-basee methods: SAITS [10], Transformer [48), PatchTST (37), iTrrnsformer [28];
(2) Lasear-based methods: DLinear 771. FreTS [68] TimeMixer [55]: (3) Generative-based method:
GPVAE 14] and (4) CNN- method: TimesNet [60].
Evaluation Metrics: Following previous studies 62 63 utilize MAE and MSE to evaluate the
imputation performance by measuring feature-wise imputation quality. Lower values indicate better.
To demonstrate the effectiveness of Glocal-IB, we apply it to vanilla 2-layer Transformer. This simple backbone, equipped with our training strategy, serves as our demonstration model and is compared against all baselines. More information is in Appendix
Implementation Details:
4.2 Overall Comparison
We comprehensively compare the imputation performance of different methods over 9 datasets with various missing rates and visualize the latent representation distributions of SAITS, TimesNet, and our proposed method, which are the best three TSI methods. Due to space limits, we report the average imputation results over five missing rates (0.1, 0.3, 0.5, 0.7, and 0.9) in Table Full results are provided in Appendix D.I. Based on the comparison results, we summarize our observations (Obs.):
Obs. O: Glocal-IB demonstrates superior performance improvement in TSI tasks. As shown in Table | [nd Glocal-IB achieves the lowest MAE and MSE across all 9 datasets, with several cases showing a substantial margin. Notably, ETTh ETTh2, ETTml, and ETTm2, Glocal-IB shows substantial reductions in MSE (up to 40%) compared to all baselines. Even on more challenging real-world datasets like Beijing Air, PEMS-Trafite Electricity, and Metr-LA, which contain complex temporal patterns and noise that the vanilla Transformer is not good at processing, Glocal- helps the Transformer to surpass SAITS and TimesNet by non-trivial margins in both MAE and MSE. Moreover, on the Weather dataset, Glocal-IB outperforms the second-bes, method by large margin, reflecting strong robustness to seasonal patterns.
Obs. 0: The distortions of the latent representation distribution can be well solved by Glocal-IB. As shown in the Fig. (8] existing representative TSI methods produce increasingly distorted latent distributions as the missing rate increases. These distortions suggest that the models fail to preserve the underlying temporal or structural properties of the original data under high missingness. In contrast, our proposed Glocal-IB maintains a stable and coherent latent structure from 10% to 70% missing rates. Even at 90% missingness, Glocal-IB still enables the model capture the global
7

---

0.9
0.8
0.8
0.7
Gln_1
0.7
Glo
0.6
Glo
MAE
Glo
Glo_1
0.5
Glo.2
01.
FM
0.5
0.4
FMgnlion
0.3
0.4
0.3
0.2
0.3
0.2
0.1
0.2
0.1
0.3
0.5
0.9
0.1
0.5
0.7
0.9
0.1
0.3
0.5
0.7
0.9
0.9
1.3
1.0
0.8
Glo.1
Ori
1.1
Glo
0.8
Glo.
Orl
MSE
Glo
0.9
Glo
Glo
0.5
0..4
FM. Alige
FM Align
0.4
FM Align
0.3
0.2
0.1
0.0
0.3
0.5
0.1
0.5
0.7
0.9
0.1
0.3
0.7
0.9
ETTh1 Missing (Transformer)
ETThl Missing (TimesNet)
ETTh1 Missing (SAITS)
Figure 3: Imputation performance the ETThl dataset of Transformer, TimesNet, and SAITS with four different training methods.
TimesNet Glocal-IB SAITS Glocal-1B
Fully Observed
Miss 10%
Míss 30%
Miss 50%
Miss 70%
Míss 90%
Figure 4: Latent space of SAITS and TimesNet with Glocal-IB the ETTh dataset. Comparison with original models is in the Appendix D.I
shape of the original distribution, while other methods show significant collapse or fragmentation in the latent space. This observation supports that Glocal-IB preserves informative global-local dependencies under extreme data degradation.
4.3 Generality Analysis
We conduct series of studies to investigate how different training paradigms affect the performance of TSI models. Specifically, we select two the most effective TSI methods-TTimeset and SAITS-and evaluate them under four training paradigms: (1) Ori: Standard reconstruction-ban based training without any external alignment. (2) FM align: Representation alignment with a time series foundation model, specifically using the latest Time-MoE [5] (3) Glo 1: Employ Eq. 12 as the
usage of Global Alignment loss G And (4) Glo 2: Employ Eq. 13 as the usage of Global
Alignment loss Ce. to compare with Glo_ From Fig. and we observe that:
Obs. @: Glocal-IB improves the learning capability of existing imputation models. From 10% 70% missing, models trained with Glo Glo consistently outperform the original version of baselines (Ori) and foundation model-aligned (FM (FM_align) counterparts. Even under an extreme missing rate of 90%, where global information little only 10% observed data, Glocal-IB continues to enhance performance for both Transformer and SAITS, highlighting its effectiveness to boost TSI methods. Importantly, this improvement achieved with minimal architecturll modifications-nnle lightweight MLP is introduced.

---

(a) ETTh2 50% Point Missing
(c) Memory Footprint
(d) Execution Time
TCN
SAITS
TCN
saITs
MAE
0.2
0.1
0.0
(b) ET 50% Black Missing
Usage (MB)
(+)
0.8
0.4
DL DLincar
Tenes Net
DL Linear
Times Ne
0.2
0.0
ESGAN
DI.ineur
TCN_GIB
Ori
wi Glocal-18
Foundation Align (Time-M63)
Dlinear
TCN
Ours
Figure 5: Different Missing Pattern Imputation and Efficiency results. (a, b): Comparison of imputation performance on the ETTh2 dataset with 50% Point and Block missing rates. Additional results for various missing rates are presented in Appendix D.3 (c, d): Efficiency comparison of four representative models on the ETThl dataset, evaluating the original models against their variants with Glocal-IB and foundation model alignment (Time-MoB). The radial axes are on logarithmic scale.
The Time series foundation model provides limited benefit. We observe that Time-Mote based alignment yields only marginal improvements for TSI tasks. This discrepancy is likely due to the nature of the pretraining objectives used in current time series foundation models 45 29 (54,, which are predominantly forecasting tasks. Such tasks may not impose sufficient semantic constraints on the learned representations, thereby limiting the benefit of alignment when transferred imputation.
Obs. O:
Obs. Glocal-IB mitigates latent representation distortion. Figure and illustrate the impact of Glocal-IB on latent representations. In the original SAITS and TimesNet model, the upper portion of the latent space becomes increasingly distorted as the missing rate grows from 10% to 30%. From 50% to 90% missing, the latent distribution collapses, indicating that the model fails to capture meaningful structure. contrast, when they are trained with Glocal-IB, the latent distributions remain well-structuree up to 90% missing rate. This indicates that Glocal-IB introduces strong global regularization, enabling the model to preserve semantic coherence even under severe missingness.
4.4 Missing Pattern and Efficiency Analysis
We conduct experiments to analyze the effectiveness of Glocal-IB under various missing patterns and its efficiency. Our evaluation includes suite of representative baselines: USGAN (33. DLinear (7]), TCN 22. SAITS [10], and TimesNet [60]. Based on the results, we have the following observations:
Obs. @: Our proposed method remains highly effective for challenging block-wise missing patterns. Figure 5 (b) shows that when contiguous blocks data are missing- a scenario that disrupts local temporal dependencies margin. This demonstrates its robustness and superior capability in reconstructing structured data loss compared to other baselines.
method still achieves the lowest MAE by significant
Obs. Glocal-IB enhances the capability existing models across different missing patterns. As shown in Figures [S] (a,b), applying Glocal-IB on current methods (e.g.. DLinear_CC GIB and TCN GIB) leads to improved imputation accuracy over the base models. This enhancement is particularly significant in the more challenging Block Missing scenario, where both DLincar GIB GIB and TCN CCB achieve a lower MAE. This demonstrates Glocal-IB's broad utility in strengthening existing imputation methods.
Obs. O: Glocal-IB is computationally efficient module. Figures 5 (c.d) reveal that augmenting existing models with our proposed Glocal-IB (w/ Glocal-IB) results in only marginal increase memory footprint and execution time compared to the original (Ori) versions. This efficiency stands in stark contrast to the Foundation Align based on the Time-MOE (45) method, which incurs substantial computational overhead. This highlights Glocal-IB as practical, lightweight solution for enhancing model performance.

---

0,8
0.7
144
1.6
0.7
0.6
13
0.5
0.8
04
8.6
0.4
0.2
0.2
0.1
0.
L.1 0 ph
0.1
Loc Weight
Reg Weight (log)
-00
Glo Weight
2
Missing
Figure 6: Hyperparameter sensitivity experiment results, More results are in Appendix D.4
0.7
MAE
0.85
MSR
0.5
MAE
0.5
MSE
0.65
0.3
0.3
0.15
50%
700%
911%
70%
90%
30%
%/4
70%
90%
300%
50%
70%%
90%
Ablation study results ETThl dataset
(b) Ablation study results ETTh2 dataset
Figure 7: Visualization of ablation study results on ETTh1 dataset. More results are in Appendix D.4
4.5
Ablation Study and Sensitivity Analysis
We conduct an ablation study and a parameter sensitivity analysis to examine the contribution and robustness of each component Glocal-IB. The experiments are performed on four datasets: ETTh1, ETTh2, ETTml, and ETTm2. In the Ablation Study (Fig. (7 and
we compare the following configurations: (1) Entire: the full Glocal-IB method. (2) w/o Reg: Glocal-IB without the mutual information minimization term, Regularization loss. (3) w/o Glo: Glocal-IB without the global
mutual information maximization, Global Alignment loss. (4) only Loc: only the reconstruction objective, i.e., Local loss, is used, corresponding to local mutual information maximization. In the Sensitivity Analysis (Fig, 6). we vary the weights assigned the Local loss Co Regularization loss Reg. and Global Alignment loss 4te to study how each impacts model performance.
Obs. @: Both the Reg and Ce. are critical for improving imputation quality. As demonstrated in Fig. when either the Regularization loss Cley or the Global Alignment loss C is removed, the
model performance deteriorates more significantly as the missing rate increases. This indicates that Keg is effective at suppressing irrelevant variations in the latent space, while the L helps the model maintain global semantic information of the data.
Obs. @: Imputation quality is sensitive to the weight of Ckagr As shown in Fig. increasing
the weight of the Global Alignment loss Loo or Local loss L. leads to stable performance trends. However, the imputation quality drops sharply when the weight of Regularization loss L exceeds 0.01. This suggests that a small amount of KL regularization beneficial for filtering noise, but excessive regularization would suppress useful latent information excessively, resulting in significantly degraded imputation performance.
Conclusion
This paper studies the optimization dilemma in current TSI methods. To address this issue, we introduce a novel training paradigm, Glocal-IB. It extends standard IB based objectives by adding Global Alignment loss based on tractable mutual information approximation. This loss encourages the latent representations of masked inputs to match those of their fully observed counterparts, helping the model retain global structure and local detail while reducing the impact of noise. Extensive experiments on nine datasets show that Glocal-IB consistently improves imputation accuracy and leads to more stable latent representation distributions under varying missing rates.
10

---

References
[1] Alexander A Alemi, lan Fischer, Joshua V Dillon, and Kevin Murphy. Deep variational information bottleneck. arXiv preprint urXiv:1612.00410, 2016.
[2] Shaojie Bai, Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling, arXiv preprint arXiv:1803.01277 2018.
[3] Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. Brits: Bidirectional recurrent imputation for time series. Advances in neural information processing systems, 31, 2018.
[4] Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1 8(1):6085, 2018.
[5] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15750-15758, 2021.
[6] Zhichao Chen, Haoxuan Li, Fangyikang Wang. Odin Zhang, Hu Xu, Xiaoyu Jiang. Zhihuan Song, and Hao Wang. Rethinking the diffusion models missing data imputation: gradient flow perspective. Advances in Neural Information Processing Systems, 37: 12050- 12050-112000 112103, 2024.
[7] MinGyu Choi and Changhee Lec. Conditional information bottleneck approach for time series imputation. In The Twelfth International Conference on Learning Representarions, 2023.
[8] Andrea Cini, Ivan Marisca, and Cesare Alippi. Filling the g_ap_s: Multivariate time series imputation by graph neural networks, arXiv preprint
2021.
[9] Wenjie Du Pypots: python toolbox for data mining partially-oberred time series. arXiv preprint arXiv:2305. 18 2023.
[10] Wenjie Du, David Côté. and Yan Liu. Saits: Self-attention- imputation for time series. Expert Systems with Applications. 219:119619, 2023.
[11] Wenjie Du, Jun Wang, Linglong Qian, Yiyuan Yang, Zina Ibrahim, Fanxing Liu, Zepu Wang, Haoxin Liu, Zhiyuan Zhao, Yingjie Zhou, et al. Tsi-bench: Benchmarking time series imputation. arXiv preprint arXiv:2406. 12747, 2024.
[12] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
[13] Yangxin Fan, Xuanji Yu, Raymond Wieser, David Meakin, Avishai Shaton, Jean-Nicolas Jaubert. Robert Flottemesch, Michael Howell, Jennifer Braid, Laura Bruckman, et al. Spatio-temporal denoising graph autoencoders with data augmentation for photovoltaic data imputation. Proceedings of the ACM on Management of Data, 1(1):1 19, 2023.
[14] Vincent Fortuin, Dmitry Baranchuk, Gunnar Rätsch, and Stephan Mandt, Gp-vae: Deep probabilistic time series imputation. In International conference on artificial intelligence and statistics, pages 1651 1651-1661. PMI.R, 2020.
[15] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729 9738, 2020.
[16] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li. Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 16000-16009, 2022.
[17] Yifan Hu, Peiyuan Liu, Yuante Li, Dawei Cheng, Naiqi Li, Tao Dai, Jigang Bao, and Shu-Tao Xia. Finmamha: Market-aware graph enhanced multi-level mamba for stock movement prediction. arXiv preprint Xrv:225022.66077 2025.
[18] Yifan Hu, Peiyuan Liu, Peng Zhu, Dawei Cheng, and Tao Dai. Adaptive multi-scale decomposition framework for time series forecasting, In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 17359 17367, 2025.
[19] Yifan Hu, Jie Yang. Tian Zhou, Peiyuan Liu, Yujin Tang, Rong Jin, and Liang Sun. Bridging past and future: Distribution-awe aware alignment for time series forecasting. arXiv preprint arXiv:2509. 14181, 2025.
11

---

[20] Yifan Hu, Guibin Zhang, Peiyuan Liu, Disen Lan, Naiqi Li. Dawei Cheng. Tao Dai. Shu-Tao Xia, and Shirui Pan. Timefilter: Patch-specific spatial-temporal graph filtration for time series forecasting. arXiv preprint wxXi:22501.30441 2025.
[21] SeungHyun Kim, Hyunsu Kim, Eunggu Yun. Hwangrae Lee, Jachun Lee, and Juho Lee. Probabilistic imputation for time-series classification with missing data. In International Conference on Machine Learning, pages 16654-16667. PMLR, 2023.
[22] Diederik Kingma. Adam: A method for stochastic optimization. arXiv preprint arw.iv:412..9900 2014.
[23] Diederik Kingma, Max Welling, al. Auto-encoding variational bayes.
[24] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv: 1e09.02007 2016.
[25] Huiping Li, Meng Li, Xi Lin, Fang He, and Yinhai Wang- A spatiotemporal approach for traffic data imputation with complicated missing patterns. Transportation research part C: emerging technologies, 119:102730, 2020.
[26] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint 17XXvv::07.0099900 2017.
[27] Qi Liu and Wanjing Ma. Navigating data corruption in machine learning: Balancing quality, quantity, and imputation strategies. arXiv preprint arXiv:2412. 18296, 2024.
[28] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itrans- former: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625, 2023.
[29] Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer: Transformers for time series analysis at scale. arXiv e-prints, pages arXiv-2402, 2024.
[30] Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, et al. Multivariate time series imputation with generative adversarial networks. Advances in neural information processing systems, 31, 2018.
[31] Ivan Marisca, Andrea Cini, and Cesare Alippi. Learning to reconstruct missing data from spatiotemporal graphs with sparse observations. Advances in neural information processing systems, 35:32069-32082, 2022.
[32] Pierre-Alexanee Alexandre Mattei and Jes Frellsen. Miwae: Deep generative modelling and imputation of incomplete data sets. In International conference machine learning, pages 4413-4423, PMLR, 2019.
[33] Xiaoye Miao, Yangyang Wu, Jun Wang, Yunjun Gao, Xudong Mao, and Jianwei Yin. Generative semi- supervised learning for multivariate time series imputation. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 8983-8991 8993,899,,222 2021.
[34] Robin Mitra, Sarah F McGough, Tapabrata Chakraborti, Chris Holmes, Ryan Copping. Niels Hagenbuch, Stefanie Biedermann, Jack Noonan, Bricue Lchmann, Aditi Shenvi, al. Learning from data with structured missingness. Nature Machine Intelligence, 5(1):13-23, 2023.
[35] Ahmad Wisnu Mulyadi, Eunji Jun, and Heung-Il Il Suk. Uncertainty- aware iniiational-recuren imputation network for clinical time series. IEEE Transactions on Cybernetics, 52(9):684-9999, 2021.
[36] Tong Nie, Guoyang Qin, Wei Ma, Yuewen Mei, and Jian Sun. Imputeformer: Low rankness-induced transformers for generalizable spatiotemporal imputation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2260-2271, 2024. 2024.
[37] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint 22.i::222/.113300 2022.
[38] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint arXiv:1807.0331 037043788 2018
[39] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaacldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXin:2304.07199, 2023.
[40] A Paszke. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint 102iv.1912.01703.200 2019.
12

---

[41] Mattia Prosperi, Yi Guo, Matt Sperrin, James S Koopman, Jae S Min, Xing He. Shannan Rich, Mo Wang. lain E Buchan, and Jiang Bian. Causal inference and counterfactual prediction in machine learning for actionable healthcare. Nature Machine Intelligence, 2(7):369-375, 2020.
[42] Rui Qin and Yong Wang. Imputegan: Generative adversarial network for multivariate time series imputation. Entropy, 2571) 2023.
[43] Xiangfci Qiu, Jilin Hu, Lckui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S Jensen, Zhenli Sheng, et al. Tlb: Towards comprehensive and fair benchmarking of time series forecasting methods. arXiv preprint arXiv:2403.201 2024.
[44] Xiaobin Ren, Kaiqi Zhao, Patricia Riddle, Katerina Taskova, Qingyi Pan, and Lianyan Li. Damr: dynamic adjacency matrix representation learning for multivariate time series imputation. Proceedings of the ACM on Management of Data, 1(2):1255, 2023.
[45] Xiaoming Shi, Shiyu Wang, Yuqi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. Time-moe: Billion-scale time series foundation models with mixture of experts. arXiv preprint arXiv:2409.16040
2024.
[46] Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffusion models for probabilistic time series imputation. Advances neural information processing systems, 34: #4804-24866, 2021.
[47] Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 ieee information theory workshop (itw), pages 1-5, IEEE, 2015.
[48] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances neural information processing systems, 30, 2017.
[49] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pictro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv: 10903, 2017.
[50] Slava Voloshynovskiy, Mouad Kondah, Shideh Rezaeifar, Olga Taran, Taras Holotyak, and Danilo Jimenez Rezende. Information bottleneck through variational glasses. arXiv preprint arXiv: 19/2.0080, 2019.
[51] Hao Wang, Haoxuan Li. Xu Chen, Mingming Gong, Zhichao Chen, et al. Optimal transport for time series imputation. In The Thirteenth International Conference on Learning Representations.
[52] Hao Wang, Zhichao Chen, Zhaoran Liu, Licheng Pan, Hu Xu, Yilin Liao, Haozhe Li, and Xinggao Liu. Spot-i: Similarity preserved optimal transport for industrial iot data imputation. IEEE Transactions on Industrial Informatics, 2024.
[53] Jun Wang. Wenjie Du, Yiyuan Yang, Linglong Qian, Wei Cao, Keli Zhang, Wenjia Wang, Yuxuan Liang, and Qingsong Wen. Deep learning for multivariate time series imputation: A survey. arXiv preprint arXiv:2402.04m: 2024.
[54] Shiyu Wang, Yinbo Sun, Xiaoming Shi, Shiyi Zhu, Lin- Lin-Tao Ma, James Zhang, Yifei Zheng, and Jian Liu. Full scaling automation for sustainable development of green data centers. arXiv preprint arXiv:2305.00770 000066 2023.
[55] Shiyu Wang, Haixu Wu, Xiaoming Shi. Tengge Hu, Huakun Luo, Lintao Ma. James Y Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting. arXiv preprint 24Xiv:2405.14611 2024.
[56] Xu Wang, Hongbo Zhang, Pengkun Wang. Yudong Zhang. Binwu Wang. Zhengyang Zhou, and Yang Wang. An observed value consistent diffusion model for imputing missing values in multivariate time series. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2419-2418, 2023.
[57] Zhixian Wang. Linxiao Yang, Liang Sun, Qingsong Wen, and Yi Wang. Task-oriented time series imputation evaluation via generalized representers. Advances in Neural Information Processing Systems, 37:137403-13748 2024.
[58] Hyowon Wi. Yehjin Shin, and Noseong Park. Continuous -itetooou---eeeeeee autoencoders for regular and irregular time series imputation. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 2024.
13

---

[59] Haixu Wu, Jiehui Xu, Jianmin Wang. and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34:22419-22430,
2021.
[60] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang. and Mingsheng Long. Timesnet: Temporal 2d- variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186, 2022.
[61] Qianxiong Xu, Sijie Ruan, Cheng Long, Liang Yu, and Chen Zhang. Traffic speed imputation with seatio-temporal attentions and perceeperceptual training. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 2280-2288 2022.
[62] Hanchen Yang. Jiannong Cao, Wengen Li, Yu Yang. Xiaoyi Li, Lingbai Kong, Yichao Zhang, Jihong Guan, and Shuigeng Zhou. Towards robust and interpretable temporal graph modeling for traffic prediction. ACM Transactions on Knowledge Discovery from Dato, 2025.
[63] Hanchen Yang. Jiaqi Wang, Jiannong Cao, Wengen Li, Jialun Zheng, Yangning Li, Chunyu Miao, Jihong Guan, Shuigeng Zhou, and Philip S Yu. Okg-Ilm: Aligning ocean know inowledge graph with observation data via Ilss for global sea surface temperature prediction. arXiv preprint arXiv:2508.00050
2025.
[64] Yang. Yifan Hu, Kexin Zhang, Luyang Niu, Yushun Dong. Philip Yu, and Kaize Ding. Revisiting multivariate time series forecasting with missing values. arXiv preprint arXiv:2509.23494, 2025.
[65] Jie Yang, Rui Zhang. Ziyang Cheng, Dawei Cheng, Guang Yang. and Bo Wang. Grad: Guided relation diff'usion generation for graph augmentation graph fraud detection. In Proceedings of the ACM on Web Conference 2025, pages 5308-5319, 2025.
[66] Xinyu Yang. Yu Sun, Xinyang Chen, al. Frequency generative models for multivariate time series imputation. Advances in Neural Information Processing Systems, 37:52595-5222 2024.
[67] Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstruction vs. generation: Taming optimization dilemma in latent diffusion models. arXiv preprint arXiv:2501 SE0I::2550.01223,, 2025.
[68] Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang. Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency- domain mlps are more effective learners in time series forecasting. Advances in Neural Information Processing Systems, 36:76656 76679, 2023.
[69] Ruoxi Yu, Yali Zheng, Ruikai Zhang, Yuqi Jiang, and Carmen CY Poon. Using multi-task recurrent neural network with attention mechanisms to predict hospital mortality of patients. IEEE journal of biomedical and health informatics, 24(2):486 492, 2019.
[70] Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is casier than you think. arXiv preprint arXiv:2410.0694 2024.
[71] Ailing Zeng, Muxi Chen, Lci Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121-11128, 2023.
[72] Kexin Zhang, Baoyu Jing, K Selçuk Candan, Dawei Zhou, Qingsong Wen, Han Liu, and Kaize Ding. Cross-domain conditional diffusion models for time series imputation. arXiv preprint arXiv:2506. 12412, 2025.
[73] Shuyi Zhang, Bin Guo, Anlan Dong, Jing He, Ziping Xu, and Song Xi Chen. Cautionary tales on air-quality improvement beijing. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473(2205):201704577 2017.
[74] Jialun Zheng, Jie Liu, Jiannong Cao, Xiao Wang. Hanchen Yang. Yankai Chen, and Philip S Yu. Dp-dga:: generalist dynamic graph anomaly detector with dynamic prototypes. arXiv preprint arXiv:2508. 00664, 2025.
[75] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence serieserieseee forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106-11118 11115, 2021.
[76] Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, al. One fits all: Power general time series analysis by pretrained Im. Advances in neural information processing systems, 36:43322 -43355, 2023.
[77] Lvxing Zhu, Kexin Zhang, Hao Chen, Chao Wei, Weiru Zhang, Haihong Tang, and Xiu Li. Hel4ge: Incorporating hierarchical category structures into contrastive leamning for e-commerce query classification. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. pages 3647-3656, 2023.
14

---

A Theoretical Analysis
A.1 Variational Approximation of Mutual Information Minimization for Lee
Following prior work [ 7
approximate r(Z; X*) using variational upper bound. We begin by rewriting the mutual information definition into an equivalent KL form:
I X") ((),2r)
log p22) p(x))]
log ((= p(x=))
Ep(x,2*)
(15)
((2) p(x0)
log p(2
p(2)
To make the equation tractable, we follow the variational inference by introducing a variational marginal (==). We convert TZZ; X*) as follows:
log p==|20)
T(Z; X²) pp(c,2))
p(2)
log [22(2=)
((2)
p(2)
((2)
log p(=(20)
log p(z)
Bp(c,2))
(16)
g(2)
((2)
[. / p(2,x") log p==|=0)
-(22x=") log p(=)] ((2)
g(2) de dr.
[ / plx)) =(=|==) log
p(z p(0012) p(z) log p(z)
dz dx
((2)
((=)
where the first and second terms are both calculations of KI. divergence, so we can get as follows:
1(2;X)) [ pl=") p(z|=") log p(z|2")
((2) p(z1)z) p(2) log p(2)
dz do,,
(())
((0"). Dri (==la")))) dx p((012) DKi (xn.p(())ll(s do,
(17)
/ p(=") DRL (xup(()(>))ll(())) dz",
Ep(ay) DRI. [pl[p((=/// x [l{(=)).
where the last inequality follows from the non-negativity of KL divergence.
A.2
Approximation of Mutual Information Maximization L and Le.
A.2.1
Derivation of C.oc
Here, we illustrate the entire derivation of the mutual information I(X; Z) in Eq.9. Similar the calculation Eq. 15 by utilizing variational inference, we can get lower bound of [(X; Z):
log 2xx,2)
I(X; Z) E, E,(,,,)
p(z) p(2)
log p(x|=) p(2)
Ep(r,))
p(x) p(z)
log p(x[2)]
Ep(a.a)
(18)
p(x)
log p(x|=) go (x(2))
Ep(x,a)
p(z) go (z|2))
log [6 22]2)
log p( 2))
Ep(x,.)
+ Fp(z,))
p(x)
(x)
15

---

Note that the second term can be calculated as KL divergence, so we calculate as follows:
log (x(2)2
/ p(a, log p(a|2) dæ dz,
T(X;2) = E p(xx,)
p(a)
(x)22)
log (x(222)
+ p(x|2)) -p(2) log p(z|z) dx dz, q (
1 p)),,=)
(19)
p(a)
log (x(22
pp(x..)
p(z) DKI Dw..p(o/a)laa dz.
p(z)
Finally, because of the non-negativiyy of KL divergence:
log (0(2))
+ / p(()) (wlp(aaee))lll dz.
I(X; 2) = Ep(2,2)
p(x)
log g0 (x(2))
Ep(x,))
(20)
p(r)
[lg go (x|2)]
pp(,,) [logp(x)),
E pp(x,))
[log qo (x|22]
A.2.2 Derivation of C
To provide the model with global-level guidance, we approximate the ((x; Z) into contrastive form, which is similar to the CPC
(8).
2(x,2)
]
2( Ep(a,:)
log
p(x) p(z)
p(alz) -p(2)
E Bp(x,=)
log
p(x:) p(2))
p(xl2)
pp(x.))
log
p(x)
p(xr
- Ep(u,z)
log
p(=]=))
p(r)
=-Ep Ep(a,=)
log
-
log N
p(x|z)
p(r)
- Ep(a,z)
log
N
(21)
1+ p(x)
p(r[2) (N (N-1) 1)- .1) ]
E p((,,))
log
1 p(x)
p(ayy (z) p(x,,)
- Ep(x,=)
log
(N - 1) Ep(as)
p(x|2)
p(aaz)
Ep(a,))
log
p(æ)
p(als) p(z)
play(=) plaz)
+
ExEX Tog
f(x,2)
= =p((r.)
log
[(x, 2) + E Majj 2)
xyEXas
Here, we use mini-batch approach [7] that n chosen from other timestamps' data in the same
mini-batch. And fxx 2) is density ratio that is proportional to p(z)
p(z[z)
Moreover, inspired by the evolution of the contrastive learning Eq. 21 as shown below:
15
5 77
we further simplify the
T(X: Z) x Ep(x,2) [f
(22)
Therefore, get simpler alignment loss in Eq. 13
16

---

B Datasets
We conduct experiments 9 real-world datasets evaluate the imputation performance. Now describe the detailed information of these datasets as follows:
ETT [5 records 7 power-related factors from electricity transformers between 2016/07 and 2018/07. includes four subsets: ETTh1 and ETTh2 are sampled hourly, while ETTml and ETTm2 are sampled every 15 minutes.
Beijing Air 733 provides hourly air quality data from 12 monitoring stations in Beijing, collected from 2013/03/01 to 2017/02/08. Each station measures 11 variables, resulting in 132 combined features.
PEMS-Traffic 59 contains hourly road occupancy rates from 862 sensors San Francisco
Bay area highways, spanning 2015/01 to 2016/02.
Electricity 59 records hourly electricity usage of 321 clients from 2012 2014.
Weather 59 includes 21 meteorological variables collected every 10 minutes at the Max
Planck Biogeochemistry Institute throughout 2021.
Metr-LA (26] captures traffic speeds every 5 minutes from 207 road sensors across Los
Angeles County, covering the period from 2012/03 to 2012/06.
C Implementation Details
We follow the data processing and split protocol from PyPOTS (.. The training, validation, and test sets are divided (60%, 20%, and 20%) chronological order to avoid data leakage. For all datasets, the input sequence length is set to 96.
All experiments implemented in PyTorch 40 2.6.0 and run single NVIDIA 4090 GPU with
24GB memory. We use the Adam optimizer [22] with learning rate of 0.001. The batch size is 64, and the number of training epochs is fixed to 30. The hidden dimension set to 256.
All baseline models are built upon the PyPOTS 9 benchmark, where each model follows the settings from its original paper and official implementation. We report the average results over different random seeds in this paper.
D
Full Experiments
D.1 Full Comparison Results
Table provides comprehensive comparison Globcal-IB with a vanilla Transformer against baseline methods, with results of missing rate of 10%, 30%, 50%, 70%, and 90% listed separately. Additionally, we visualize the latent space of ours and three representative TSI methods in Fig.
D.2 Generality Analysis
Fig. 9 provides the latent space TimesNet, SAITS, and their Glocal-IB counterparts, across missing rate 10%. 30%, 50%, 70%, and 90%. Glocal-IB remarkably improves the alignment of the latent space while the missing rate increases.
D.3 Missing Pattern Analysis
Fig. 10
12 and 13 provide the entire performance comparison results on ETTh1, ETTh2,
ETTml. and ETTm2, across missing rate 10%. 30%, 50%, 70%, and 90%. These indicate that our proposed method achieves the best imputation performance while remarkably improving the imputation performance of current methods.
D.4
Ablation Study and Sensitivity Analysis
Fig. 15
illustrates the ablation studies on datasets, including ETTh1, ETTh2, ETTm], and E
17

---

Fig. 14 demonstrates all the parameter sensitivity analyses on 4 datasets, including ETTh1, ETTh2, ET1ml. and ETTm2.
E
Societal Impact Statement
Similar to previous TSI works [66 31. the development of Glocal-IB has the potential to benefit wide range of real-world applications. In healthcare, for example, improved imputation models can enhance the reliability of patient monitoring systems by recovering missing clinical measurements. This may support earlier diagnosis, enable timely interventions, and help reduce overall medical costs by facilitating more informed decision-maker making.
However, the deployment of advanced imputation techniques also introduces several risks. sensitive domains such surveillance, these models may reconstruct incomplete data in ways that raise privacy concerns, particularly if used infer personal information without consent. Moreover, over-reliance on automated imputation may lead to overlooked errors, potentially resulting in biased unreliable decisions downstream tasks.
To mitigate these risks, it is important to establish clear guidelines for the ethical use of imputation models. This includes enforcing data protection regulations, ensuring transparency model behavior, and incorporating fairness-awand validation protocols. Broadening access to such technologies and conducting regular audits can further promote responsible deployment and prevent unintended harm.
Limitation and Discussion
Due to computational constraints, only apply Glocal-IB three representative back- Jones-Timesie SAITS, and Transformer-on four datasets from the ETT benchmark: ETThl, ETTh2, ETTm1, and ETTm2. While these results sufficient demonstrate the effectiveness of our approach, future work can explore broader model families and larger-scale datasets to further validate generalization.
We also observe that the performance gain under extreme missingness (e.g., 90%) is less pronounced than at moderate levels (10%-70%). A possible reason is that in the inference phase when only small portion of the input is available (e.g., 10%), the preserved global structure too weak to offer meaningful alignment signals. In such cases, the model has limited capacity to distinguish signal from noise, resulting in less reliable imputation.
This limitation highlights an important future direction: how to enhance global guidance under limited observations. One possible solution is incorporate stronger structural priors or pretrained knowledge to better inform the latent space.
18

---

SAFES
Thmesset
GPVAE
Ours
Fully
Figure 8: Latent space comparison of Glocal-IB on Transformer and three representative TSI methods, including SAITS, TimesNet, and GPVAE.
SAITS Ori
SAITS Glocal-TB
Timesett Ori
TimesNer Glocal-IB
Fully Observed
30%
Miss 501%
Miss 70%
Miss
Figure 9: Latent space comparison of TimesNet and SAITS with Glocal-IB against their original implementations.
19

---

Table 2: Imputation performance on 9 datasets (average MAE and MSE across 10% to 90% missing rates). Best is bold and second-best is underlined. We use OOM to denote out of memory.
Models IMP
Ours
SAITS
Transformer
DLinear
TimesNet
FreTS
PatcTTST
Trranformer
GPVAE
TineMixer
Metric
MSE
MAE MSE
MAE MSE
MAE MSE
MAE MSE MAE MSE MAE
MAE MSE
MAB
MSE
MAE MSE MAE MSE
0.1
50%
0.165
0.060
0.245 0.
lo 251 0.138 132 0,281 0.158 0.339 0.209 0.324 0.191 0,449 0,480
0,350 0.246
0.664 0.736
0.736/0.626 0.626 0.730
0.3
49%
0.214
0.096 0.306
0.306
0.350
0.194 0.316 0.193 00.506 0.464 0.348 0.221 0.583 0,743 0.368
0.267
0.690 0.827 0.655 0.824
0.5
47%
00.239
0.128 0.257
0.360
0.267 0.356 0,244 (0.67 0,706 01,344 0.566
0,546
0.614
0:401
0.314
0.735
0.901 0.683 0.925
0.7
36% %
(0.303
0.202 0,443 0.422 0.443 0.455 0.410
0.315 00.718 0,917 0.382 0.700 0,927
00:453
0.401 0.760
1.009 0.703 0.968
26% 96
(0.506
0.497 0.497 (0665 0 0.893 0.814
0.585 0.670 1.216
0. 0.796
1.134
0.633
0.799
0.808
1.166 0.721 0.983
0.1
500%
(0.180
0.06310.265 0. 0.142|0.235
0.126 0.289 0.1557 (0.423 0.342 0.315 0.189
0.312
0.318
0.522
0.458 0.463 0.390
ETTh2
0.3
48%
00.205
0.080 0.276 0. 0.167 0.260
0.155 0.317 0.192 0.756 0.955 0,383 0.271 0.423
0.347
0.387
0.282
0.545
0.496 0.465 0,383
0.5
37%
(0.243
0.115 0.209 0.286
183 0.340 0.223 (0.862 1.222 0..001 0.309 0.327 0.416
0.321
0.640
0.649 0..495 0.449
0.7
44%
00.254
0.132 0.350 0.132
0.324
0.235 0,363 0.262 0..977 1.511 00,456 0.381 0.525 0.381
0.520
0.381
0.267
0.764
0.889 0.531 0.549
0.9
30.
(0.361
0.270 0.498
0. 508 0.442 0,390 0.451 0.384 1 1.002 1.672 (..613 0. 0.702 1,368
0,460
0.415
0,958
1.351 (0,692 0. 0.903
0.1
45%
(0.097
0.024/0.138
0.042l0. 145 0.047 0.215 0.095 007 0.876 0.245 0.119 0.094
0.094|0.243
0.243
0.123
0.469
0.377 0..318 0.187
ETTm2 ET7ml
0.3
35%
(0.117 0.036 158 0.055
162 0.059 0.230 0.1 0. 1.012
0.258 0.131 0.237
0.362
0.137
0.520
0.321 0.195
0.5
33%
00.135
0.049 p.075
0.15
185 0.075 0.251 0.129 00.799 1.112 0.279 0.151 00.248
0.122
01.294
16188
0.546
0.5 0,33440.335 0,208
0.7
30%
00.165
0.076
0.107
0
0,215
0.101 0.292 0.169 0.832 1.206 0.3118 0.193
0.294 0,165 0.324
0.201
0.627
0.680 0,360 0.236
0.9
15%
(0.271
0.167
(0:324
0.217
304
0.197 0.430 0.358 00.844 1,231 00.452 0.381
0.447
0.445)0.450
0.411
0.777
1.102 0.463 0.381
0.1
18%
(0.113
0.032/0.151
0.043|0
13.9 139 0.039 0.229
0.101 (0.751 0.974 (0.267 0,141 (0.24) 0.109
00.316
0.206
0.4666
0.38| / (0,337 0.209
0.3
25%
00.126
0.039 0.160 0.052 0. 169 0.056 0.253 0.129 0.866 1,263 0.287 0,155
0.305 0.177 0..17
0:207
0.479
0.407 0.319 0.193
37%
00.138
0.042 0.183 0.065 183 0.065 0.290 0.169 00.33 1.450 0.291 0.165 0.162
0..34
0.230
0.482
0.404)
0.199
0.7
18%
(0.169
0.06310:231
0. 0.102 199 0.078 0.305
0.188 (0,978 1,588 0.337
0.222 0.162
0.330
0.220
0.498
0.439 0,366 0.266
0.9
23%
(0.277
0.109 0.299 0.169
252 0.142 0.392 0.313 1.011 1.694 (0.412 0.326 0.289
01.775
0.273
0.746
0.872 0.452 0.405
Beijing Air
0.1
10.192
0.289/0.216
0.216
0.291 10.231
0.344
0.253
0.294
(0.231 0.359
0.272 0.316
0.382
(0.298
0.362
0..39
0.420004040000 0.606
0.3
7%
0.200
0.299 0.231
0.325
0.325
0.247
0,352 0,263 0.312 00.44 0.342 0.271 0.319 0.389
0.327
0.418
0.350
0:436
0:436 0.445 0.627
0.5
6%
(0.215
0.312 0.250
0,363 0.269 0.334
0.244 0,337 0.276 0.334
0,407
0,347
0.445
0.371
0.471 0.445 0.602
0.7
5%
0.234
0.227
0.327 0.266
0.277 0,380 0.287
0.341 0,2555 0,353 0..296 0.359
0.497
0.373
0.474
0.388
0.490 0.448 0,628
0.9
5%
00.272
0.374 0.431
0.436
0.322 0.406 00.448 0.462 0.333 0.417
0.686
0.480
0.666
0.454
0.597 0.466 0.674
PEMS -
0.1
9%
|0.303
0.602|0.330
0.664
0,685
0.394 0.668 (0.333 0.681 0.455
0.770
0.7M
OOM
0.388 0.680) 0.680/o.522
0.522 0,984
0.3
7%
0.,08
0.334
0.670
0.346
0.091 0.396 o. 0.680
0.333 0.333 0.677 0..39 0,442
0,844 00M
00M
0.383
0,525 1.014
0.5
6%
0.318
0.334
0.349
0.691 0.393 0.678 0.333 0.688 0.439 0.740 0.855
OOM 0.374
0.671 0.526 0.998
0.7
0.324
0.332
p.673
0.335 0.699 0.403
00.336
0.448
0.744
0.905
ODM
0.674 0.528 1.028
0.9
5%
(0.339
0.657
0.335
0.691
0.375
0.710
0.417 0.757 00.45 0 0.693
0.425 0. 0.500 0.962
OOM
0.394
0.536 1.067
0.1
5%
[0.340
0.252
0 0.278/0.360
0,286
0.459 0.390 0.297 0.498 0.456
0.770
0.266
0.440
0.373 0.651 0.727
Electricity
0.3
00.349
0.335
0.282 0.366 0.294 0.465 0.399 00,773 0.300 0.571 0.682
0.811
0.811
0.393
0.289
0.430
0.373 0.655 0.739
0.5
5%
00.354
0.275
0.275
0.360
0.397
0.290
0.374 0,303
0.471 0.415 (0.376 0.304 0.524 0.492 0.718 0:419
0.324
0.436
0.384 0.652 0.731
0.7
4%
(0.379
0.310 0.435
0.310
0.393 0.455
0.422 0.483 0.434
0.434 00.392 0.324 0.579 0.597 0.094
0.812 0.456
0.378
0.447
0.401 0.643 0.712
0.9
1%
(0.437 0.380
0.486
0.493
0.485 0.538 0,526 0.440 0. 0,384 (1.548 0.539 0.805
0.560
0.559
0.474
0.44 00.639 0.710
0.1
38%
/0.070
0.034/0
103
0.072lo.
111 0.069
0.150
0.068 15 0.053 0.16 0.065
0.084
0,144
0.080
0.239
0.151 0..224 |0.224 0.167
Weather
0.3
41%
0..17
0.039
0.030 0.110
0.075
0.076
0.150 0.073 13. 0.075 0.149 0.066
0.157
0.087
0..158
0.083
0.252
0.161 0.235 0.170
0.5
42%
00.084
0.047...12
0.080
136 0.085 0.154 0.080
0.080 0.15 0, 120 0.165 0.081 0.178
0,100
0.172
0.089
0.273
0.179 0,229 0,174
0.7
32%
0.103
0.059
0,143
0.091
0 142 0.093 0.159 0.093 00.366 0,269 06165 0.087
0.204
0,118
0.199
0.110
0.265
0.180 0.235 0.175
0.9
19%
00.150
0.101
0.200
(.2202 0. 0.14
188 031 0.1 0.129 00.447 0.537 0.203 0.125
0.164
(0.238
0.149
0.366
0.302 0.270 0.202
0.211
0.1
23%
(0.247
0.250/0.28,
0.350/0.290
0.353 0.381 0.383
(0.267
0.323 0.409
0.423/0.388
0.459
00.383
0.381
(0.409
0.436 0.645 1.045
Metr-LA
21.
0.251
0.262 0.283 0. 0.292 0.292 0.363 0.391
10.271 0.331 0.411
0.432 0.392
0.468
0.397
0.410
0.383
0.414 0.608 0.966
0.5
20%
0.2259
0.277 01222 0.375 0.372 0.380 0.397
00.779 0.346 0.439 0,509 0.420
0,509
0.420
0.464
0.380
0.411 0.605 1.003
0.7
20%
0.267
0.294
0.391 0.311 0,393 0,379 0.411
10.290
0.366 0.401 0.43200.106 0.534
00.39
0.508
0.436
0.487
0.580 0.963
0.9
5%
(0.308
0.345
0.342 0.456 0.406 0.468 00.336 0 404
0.442 0.472 0,750
0.
0.498
0.624
0.499
0.567
(0,597 1.025
20

---

a ETTh1- 10% Missing Rate
0.7
0.6
0.5
MAE
0.4
0.3
0.2
0.1
0.0
(b) ETTh1 -30% Missing Rate
0.7
0.6
0.5
MAE
0.4
0.3
0.2
0.1
0.0
(c) ETTh1-50% Missing Rate
0.8
0.7
0.6
MAE
0.5
0.4
0.3
0.2
0.1
0.0
(d) ETTh1-7000 -0% Missing Rate
0.8
0.7
0.6
MAE
0.5
0.4
0.3
0.2
0.1
0.0
(c) ETTh1-90% Missing Rate
0.9
0.8
0.7
MAE
0.6
0.5
0.4
0.3
0.2
0.1
0.0
Point
Block
USGAN
DLinear GIB
TCN GIB
DLinear
TCN
Ours
Figure 10: Full comparison results on ETTh1 with Point and Block missing patterns.
21

---

a ETT 10% Missing Rate
0.8
0.7
0.6
MAE
0.5
0.4
0.3
0.2
0.1
0.0
(b) ETTh2-30% Missing Rate
0.9
0.8
0.7
MAE
0.6
0.5
0.4
0.3
0.2
0.1
0.0
(c) ETTh2-50% Missing Rate
1.0
0.9
0.8
0.7
MAE
0.6
0.5
0.4
0.3
0.2
0.1
0.0
(d) ETTh2-70% Missing Rate
1.0
0.9
0.8
0.7
MAE
0.6
0.5
0.4
0.3
0.2
0.1
0.0
(c) ETTh2-90% Missing Rate
1.2
1.0
MAE
0.8
0.6
0.4
0.2
0.0
Point
Block
USGAN
DLinear GIB
TCN GIB
DLinear
TCN
Ours
Figure 11: Full comparison results on ETTh2 with Point and Block missing patterns.
22

---

(a) ETTml 1- 10% Missing Rate
0.5
0.4
MAE
0.3
0.2
0.1
0.0
(b) ETTm 1-3%% Missing Rate
0.6
0.5
MAE
0.4
0.3
0.2
0.1
0.0
(c) CTTm1-50% Missing Rate
1.0
0.8
MAE
0.6
0.4
0.2
0.0
(d) STTm1-70%) Missing Rate
1.0
0.8
MAE
0.6
0.4
0.2
0.0
(e) 1TTm1-900) Missing Rate
1.2
1.0
MAE
0.8
0.6
0.4
0.2
0.0
Point
Block
USGAN
DLinear GIB
TCN GIB
DLinear
TCN
Ours
Figure 12: Full comparison results on ETTml with Point and Block missing patterns.
23

---

aa) ETTm2- 10% Missing Rate
0.6
0.5
MAE
0.4
0.3
0.2
0.1
0.0
(b) ETTm2-30%, Missing Rate
0.7
0.6
0.5
MAE
0.4
0.3
0.2
0.1
0.0
(c) ETTm2-50% Missing Rate
0.8
0.6
MAE
0.4
0.2
0.0
(d) ETTm2-70% Missing Rate
0.9
0.8
MAE
0.6
0.4
0.2
0.0
(e) ETTm2-90% Missing Rate
0.9
0.8
MAE
0.6
0.4
0.2
0.0
Point
Block
USGAN
DLinear GIB
TCN GIB
DLinear
TCN
Ours
Figure 13: Full comparison results on ETTm2 with Point and Block missing patterns.
24

---

0.8
16
0.7
0,6
0.5
0.4
06
04
0.2
0.2
1. Loe Weight
Missing
.00 Reg Weight (log)
a 0 10
Missing
Glo Weight
0.1 0.5
0 0.3 10
Missing
ETTh2
0.6
0.6
05
0.6
0.4
0.2
0.0
0. 1 Loc Weight
050 ,0,9 Missing
Reg Weight (log)
2 -10
oo 2 20 Missing
0.5
Weight
10 20 Missing
ETTml
1.2
0.4
0.0
Loe Weight
1g00 Mi Missing
-10 Reg Weight (log)
a. 0. oo 2ll Mig
Missing
Glo Weight
0u
0 90, Missing
ETTm2
0.4
0.4
1.4
113
0.3
0.4
02
0.2
0.0
Loe Weight
0.1
9
Missing
-10 Reg Weight (log)
0
0.
Missing
Glo Weight
0.1 n5
0 0.3 20 ygg g9
Missing
Figure 14: Full sensitivity results on ETTh1, ETTh2, ETTml, and ETTm2.
25

---

ETTh1-MAE
ETThI-MSE
0.7
0.85
0.6
0.65
0.5
0.45
0.4
0.3
0.25
0.2
0.05
10%
30%
50%
70%
90%
10%
30%
50%
70%
90%
ETTh2-MAE
ETTh2-MSE
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.1
0.2
0.0
10%
30%
50%
70%
90%
10%
30%
50%
70%
90%
ETTmI-MAE
ETTm1-MSE
0.4
0.25
0.20
0.3
0.15
0.10
0.2
0.05
0.1
0.00
10%
30%
50%
70%
90%
10%
30%
50%
70%
90%
ETTm2-MAE
ETTm2-MSE
0.35
0.20
0.30
0.15
0.25
0.10
0.20
0.05
0.15
0.10
10%
30%
50%
70%
90%
10%
30%
50%
70%
90%
Entire
w/o Reg
w/o Glo
only Loc
Figure 15: Full ablation study results ETTh1, ETTh2, ETTml, and ETTm2.
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of this work in Appendix F
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
27

---

Justification:
We include the theoretical proof in Appendix A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We include the detailed experimental settings in Section 4.1 and Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
28

---

Answer: [Yes]
Justification: The anonymous source code and datasets are available on https:// anonymous 4open
2localll-lll IB--FFO//
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https
ssiicyolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We include the detailed experimental settings in Section 4.1 and Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: seeds.
We report the results averaged from five experiments with different random
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
29

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report the necessary computer resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
In every respect in the paper, we follow the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We include the impact statement in Appendix E
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
30

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All data, models, and code in the paper respect the license.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
31

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
32

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
33


Reviews:
[{"review_id": "BfuYMJzdcj", "reviewer": "Reviewer_VQkt", "summary": "This paper tackles a critical optimization dilemma in time series imputation where models achieve low training loss under high missing rates but suffer substantial degradation in imputation quality and distorted latent representations during inference. The authors propose Glocal Information Bottleneck (Glocal-IB) as a model-agnostic solution that extends the Information Bottleneck framework. Validated across diverse datasets, Glocal-IB demonstrates good performance.", "questions": "Q1. Could the authors further elaborate on why IB is preferred over other strategies that encourage global structure in representations?\nQ2. How should users determine an appropriate value of \\beta_1 and \\beta_2 when applying this method to real-world datasets, especially those with unknown or varying missingness characteristics? Could the authors suggest a principled method, heuristic, or adaptive tuning strategy for selecting \\beta_1 and \\beta_2 in practice?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "S1. The paper compellingly diagnoses a critical limitation in time series imputation methods where models minimize training loss under high missing rates yet produce distorted latent representations and poor inference performance.\nS2. Glocal-IB extends the Information Bottleneck framework with a novel Global Alignment loss derived from tractable mutual information approximation. This innovation aligns latent representations of masked inputs with original observations while maintaining local reconstruction capabilities.\nS3. The paper presents a thorough evaluation across nine datasets, demonstrating consistent improvements over baselines, particularly under high missing rates. Latent space visualizations provide empirical support for the claim that the proposed method better preserves distributional structure compared to existing approaches.\n\nW1. While the paper is motivated by an insightful observation on memorization under high missing rates, the choice of the Information Bottleneck (IB) as the solution is not sufficiently justified. Table 1 shows that IB-based methods like GPVAE do not consistently outperform non-IB approaches. It remains unclear why IB is preferred over alternative strategies that promote global structure (e.g., multi-directional self-attention [1]).\nW2. The paper does not clearly report the specific values of \\beta_1 and \\beta_2 used in the experiments, nor does it provide practical guidance for selecting them. Given that the sensitivity analysis indicates the method’s performance is strongly affected by these parameters, it would be helpful to include tuning heuristics or recommendations to support practical adoption.\nW3. The paper claims that high missing rates lead to memorization, where models achieve low training loss but fail to generalize due to poor semantic representations. However, Figure 1 does not clearly demonstrate increased overfitting under high missingness; in fact, the training dynamics appear similar across missingness levels. This raises concerns about the strength of the empirical evidence supporting the central hypothesis.\n\n[1] Suo, Q., Zhong, W., Xun, G., Sun, J., Chen, C., & Zhang, A. (2020, December). GLIMA: Global and local time series imputation with multi-directional attention learning. In 2020 IEEE International Conference on Big Data (Big Data) (pp. 798-807). IEEE.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your valuable comments and detailed suggestions. Below is a report about how we address your comments.\n\n> **W1**: While the paper is motivated by an insightful observation on memorization under high missing rates, the choice of the Information Bottleneck (IB) as the solution is not sufficiently justified. Table 1 shows that IB-based methods like GPVAE do not consistently outperform non-IB approaches. It remains unclear why IB is preferred over alternative strategies that promote global structure (e.g., multi-directional self-attention [1]).\n>\n> **Q1**: Could the authors further elaborate on why IB is preferred over other strategies that encourage global structure in representations? \n>\n> [1] Suo, Q., Zhong, W., Xun, G., Sun, J., Chen, C., & Zhang, A. (2020, December). GLIMA: Global and local time series imputation with multi-directional attention learning. In 2020 IEEE International Conference on Big Data (Big Data) (pp. 798-807). IEEE.\n\n**A**: Existing Time Series Imputation (TSI) methods, guided by training objectives such as MAE or MSE, exhibit a tendency to over-focus on numerical details. This makes them susceptible to noise and redundant signals within the data, leading them to neglect crucial information about the overall distribution. Consequently, these methods encounter a significant optimization dilemma under high missing rates: a lower training loss paradoxically results in a more distorted embedding distribution and, ultimately, inferior test performance. Therefore, **how to encourage TSI models to *capture both global and local information* from incomplete data *without overfitting to noise*** emerges as a critical research problem.\n\n**The Information Bottleneck (IB) principle offers a powerful theoretical framework for this challenge, as it seeks to compress data to its minimal sufficient statistics (capture both global and local information), thereby filtering noise (avoid overfitting) while preserving essential information.** However, current IB-based methods like GPVAE and TimeCIB fall short. They primarily introduce regularization terms to filter out irrelevant signals, but their training objective remains identical to non-IB methods, relying on MAE/MSE for reconstruction. This fails to provide the model with additional, more informative signals. Moreover, the stringent regularization, which often forces the latent distribution towards a standard normal, $\\mathcal{N}(0,I)$, can unnaturally constrain the model's capacity. This results in a simplistic spherical embedding distribution as shown in **Figure 1's Line 3**, which has a detrimental effect on performance.\n\nWhile other approaches, such as multi-faceted learning strategies, have been suggested, they are often tailored to specific model architectures and can be complex to implement and train. **We contend that the true potential of the IB principle lies not in creating a single, specialized model, but in establishing a new, more effective training paradigm**.\n\nIn this paper, we depart from the design of a specific architecture and instead propose a general training paradigm based on the IB theory. **Our approach enhances the global mutual information often ignored by previous models**, enabling them to capture more comprehensive data signals for superior imputation. A key advantage of our paradigm is its flexibility; it is designed as a \"plug-in\" that can be applied to a wide range of existing TSI models to boost their performance. We have validated this through extensive experiments, demonstrating consistent improvements across various benchmarks.\n\n\n\n\n\n> **W2**: The paper does not clearly report the specific values of $\\beta_1$ and $\\beta_2$ used in the experiments, nor does it provide practical guidance for selecting them. Given that the sensitivity analysis indicates the method’s performance is strongly affected by these parameters, it would be helpful to include tuning heuristics or recommendations to support practical adoption.\n>\n> **Q2**: How should users determine an appropriate value of $\\beta_1$ and $\\beta_2$ when applying this method to real-world datasets, especially those with unknown or varying missingness characteristics? Could the authors suggest a principled method, heuristic, or adaptive tuning strategy for selecting $\\beta_1$ and $\\beta_2$ in practice?\n\n**A**: As illustrated in **Figure 8** of the appendix, our model exhibits consistent sensitivity to hyperparameters across four datasets (ETTh1, ETTh2, ETTm1, ETTm2). The performance trends are remarkably similar, uniformly achieving optimal results when the hyperparameters are set to **0.5, 1e-6, and 0.5**, respectively. In accordance with this finding, we adopted this single, robust set of hyperparameters for all experiments conducted on the nine datasets presented in the main paper. We appreciate the constructive feedback and will incorporate these revisions into the final version of our manuscript.\n\n\n\n> **W3**: The paper claims that high missing rates lead to memorization, where models achieve low training loss but fail to generalize due to poor semantic representations. However, Figure 1 does not clearly demonstrate increased overfitting under high missingness; in fact, the training dynamics appear similar across missingness levels. This raises concerns about the strength of the empirical evidence supporting the central hypothesis.\n\n**A**: **Figure 1** presents the training and testing MAE for all distributions. Taking TimesNet trained for 30 epochs as a representative example (**Line 1**), we observe a clear trend. As the missing rate increases, the training MAE rises slowly from 0.07 to 0.12—an increase of only 0.05. In stark contrast, the test MAE escalates dramatically from 0.36 to 0.81, a jump of 0.45. This growing disparity between training and test performance becomes increasingly pronounced.\n\nConcurrently, we observe that the learned embedding distribution progressively distorts from the original data distribution. The degree of this distortion is exacerbated as the missing rate climbs. \n\nThese two phenomena—**the widening performance gap** and **the increasing distributional dissimilarity**—provide strong evidence of a deepening overfitting to noise and a failure to capture high-level semantic information. This empirical result substantiates our core hypothesis."}, {"review_id": "Akz3BhacTn", "reviewer": "Reviewer_cTKT", "summary": "This paper presents Glocal-IB, a novel training framework for Time Series Imputation (TSI) that tackles a core optimization issue: under high missing rates, existing models often minimize training loss but fail to generalize, leading to distorted latent embeddings and inaccurate imputations. Glocal-IB extends the Information Bottleneck (IB) principle by incorporating a Global Alignment (GA) loss, which enforces consistency between latent representations of masked and fully observed inputs. This guides the model to preserve both global temporal structure and local signal fidelity, while mitigating noise. The approach is model-agnostic, lightweight (adding only a single MLP), and demonstrates superior imputation accuracy and latent space stability across nine benchmark datasets, consistently outperforming state-of-the-art methods.\nHowever, overall, this paper’s concept is rather vague and not well motivated. It may have limited novelty for NeruIPS.", "questions": "1.\tThe paper notes that Glocal-IB is less effective at 90% missingness. Could the authors explore or suggest mechanisms (e.g., stronger priors or pretrained encoders) to improve performance in such cases? How reliable are studies exploring missing at such high level?\n2.\tHow does Glocal-IB scale with longer sequences or higher-dimensional time series? Any insights on computational overhead?\n3.\tThe paper compares Glocal-IB with foundation model alignment. Could the authors elaborate on why foundation models underperform and whether combining both approaches might help?\n4.\tCan the authors comment on the interpretability of the learned latent space or the imputed values, especially in high-stakes domains like healthcare?\n5.\tCould Glocal-IB be extended to other structured data imputation tasks (e.g., spatiotemporal graphs or tabular data)?\n6.\tCombine Glocal-IB’s global-local mutual information loss with a flexible’s per-variable model selection to improve robustness under high missingness.\n7.\tAdjust the KL divergence or alignment loss weights based on variable skewness to avoid over-penalizing informative but skewed features.\n8.\tExtend Glocal-IB to support multiple imputations and ensemble predictions to reflect uncertainty.", "limitations": "The authors discuss limitations related to extreme missingness and model coverage, and suggest future directions.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "•\tStrengths:\nStrong theoretical foundation with clear derivations and mutual information approximations.\nComprehensive empirical validation across 9 diverse datasets and missingness levels.\nAblation and sensitivity analyses support the importance of each component.\nDemonstrates robustness and generalizability across multiple model backbones.\nThe combination of local and global mutual information objectives is innovative.\n•\tWeaknesses:\nPerformance gains under extreme missingness (e.g., 90%) are less pronounced.\nEvaluation is limited to a few model architectures due to computational constraints.\nSome mathematical derivations are dense and could benefit from more intuitive explanations or diagrams.\nBuilds on existing IB and contrastive learning principles, though the integration is novel.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your valuable comments and detailed suggestions. Below is our answer.\n\n> **W1**: Performance gains under extreme missingness (e.g., 90%) are less pronounced. \n\n**A**: While the performance gain of our method is less pronounced under extreme missingness (e.g., 90%), this is likely because the highly sparse input during inference provides an insufficient signal for stable imputation. With only 10% of the data visible—a sample whose own distribution may have shifted—the input provides an insufficient signal for the model to generate a stable and accurate imputation.\n\nNevertheless, our approach remains highly effective across diverse architectures. As demonstrated in **Figure 3** and confirmed by extension experiments on DLinear and TCN, our method consistently delivers substantial performance gains even in these demanding conditions. **1st**, `2nd`\n\n|PointMissing-MAE|Ours|CSDI|TimeCIB|SSSD|USGAN|DLinear|DLinear_MY|TCN|TCN_MY|\n|:-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|ETTh1-point-0.1|**0.165**|0.2615|0.7019|0.5147|0.2604|0.2740|`0.2582`|0.3346|0.3027|\n|ETTh1-point-0.3|**0.204**|0.3435|0.7025|0.5577|`0.3016`|0.3159|0.3091|0.3946|0.3397|\n|ETTh1-point-0.5|**0.239**|0.4516|0.7414|0.6421|0.3751|0.3524|`0.3485`|0.4552|0.4020|\n|ETTh1-point-0.7|**0.303**|0.6095|0.8550|0.7683|0.5264|0.4238|`0.4053`|0.5722|0.5224|\n|ETTh1-point-0.9|**0.506**|1.1069|1.0767|1.3492|0.7970|0.6565|`0.5977`|0.7938|0.7661|\n|ETTh2-point-0.1|**0.180**|0.3247|0.5904|0.4820|0.2643|0.2926|`0.2535`|0.3152|0.2667|\n|ETTh2-point-0.3|**0.205**|0.4363|0.5934|0.5969|0.3218|0.3138|`0.2769`|0.3509|0.3203|\n|ETTh2-point-0.5|**0.243**|0.5668|0.6108|0.7140|0.3823|0.3487|`0.3169`|0.4429|0.3773|\n|ETTh2-point-0.7|**0.254**|0.7855|0.6311|0.9069|0.4680|0.3708|`0.3665`|0.5239|0.5152|\n|ETTh2-point-0.9|**0.361**|1.4136|0.6705|1.6021|0.8069|0.4740|`0.4695`|0.8837|0.8425|\n|ETTm1-point-0.1|**0.097**|0.1746|0.2352|0.2310|`0.1443`|0.2142|0.2116|0.2052|0.1891|\n|ETTm1-point-0.3|**0.117**|0.2122|0.2573|0.2603|`0.1602`|0.2332|0.2331|0.2602|0.2364|\n|ETTm1-point-0.5|**0.135**|0.2611|0.2889|0.3102|`0.1807`|0.2530|0.2520|0.3147|0.2773|\n|ETTm1-point-0.7|**0.165**|0.3483|0.3161|0.3928|`0.2322`|0.2924|0.2902|0.3814|0.3714|\n|ETTm1-point-0.9|**0.271**|0.5765|0.5143|0.6466|0.5037|0.4367|`0.4152`|0.5806|0.5687|\n|ETTm2-point-0.1|**0.113**|0.2150|0.4438|0.2485|`0.1415`|0.2411|0.2401|0.2411|0.2310|\n|ETTm2-point-0.3|**0.126**|0.2752|0.4555|0.2994|`0.1600`|0.2626|0.2543|0.2471|0.2408|\n|ETTm2-point-0.5|**0.138**|0.3391|0.4622|0.3754|`0.2212`|0.2938|0.2725|0.3025|0.2948|\n|ETTm2-point-0.7|**0.169**|0.4282|0.4660|0.5036|0.3592|0.2992|`0.2904`|0.3897|0.3706|\n|ETTm2-point-0.9|**0.227**|0.6734|0.4876|0.8811|0.5969|0.4426|`0.4155`|0.6640|0.6433|\n\n\n\n> **W2**: Evaluation is limited to a few model architectures due to computational constraints. \n\n**A**: Due to computational constraints, our primary experiments apply Glocal-IB to three representative backbones—TimesNet, SAITS, and Transformer—on the four ETT benchmark datasets: ETTh1, ETTh2, ETTm1, and ETTm2. As evidenced by the significant improvements on models like SAITS and Transformer, these results are sufficient to demonstrate the effectiveness of our approach.\n\nFurthermore, as shown in the table in **W1**, to validate the general applicability of our method, we conducted additional experiments on Dlinear and TCN. These results confirm that our paradigm consistently enhances performance across a broader range of models.\n\n\n\n> **Q1**: The paper notes that Glocal-IB is less effective at 90% missingness. Could the authors explore or suggest mechanisms (e.g., stronger priors or pretrained encoders) to improve performance in such cases? How reliable are studies exploring missing at such high level?\n\n**A**: As demonstrated in **Figure 3** and our **extension experiments on DLinear and TCN**, our Glocal-IB model is effective in most scenarios. We agree that under conditions of extreme sparsity, such as a 90% missing rate, models require stronger external knowledge. Two promising directions are:\n\n- **Stronger Priors**: Introducing priors, such as a Fourier prior for periodic data or a Gaussian Process prior for smooth data. These provide a reliable \"structural backbone\" for the model, preventing unrealistic imputations when the data signal is weak.\n- **Pretrained Encoders**: Pretraining a powerful encoder on a large, complete dataset using self-supervised methods. This encoder can learn general, robust feature representations that are then fine-tuned for the sparse imputation task.\n\nWhile research at a 90% missing rate is challenging, we believe such an investigation has irreplaceable value for the following reasons:\n\n1. **A Stress Test for Models**: A model that can recover global structure from only 10% of the original data is fundamentally more robust. This helps us design models that are more reliable under all conditions.\n2. **Relevance to Critical Applications**: Scenarios with extreme data sparsity are prevalent in many critical domains. For example:\n   - **Healthcare**: Modeling a patient's year-long health trajectory in longitudinal studies where exams occur only once or twice a year.\n\n\n\n> **Q2**: How does Glocal-IB scale with longer sequences or higher-dimensional time series? Any insights on computational overhead?\n\n**A**: The computational demands related to sequence length and data dimensionality are dictated solely by the chosen backbone model. Glocal-IB introduces negligible computational overhead. This efficiency is by design: we only use the encoder for a single forward pass to get an embedding without gradient backpropagation. \n\nTo validate this, we measured Glocal-IB's GPU memory and training time on datasets with sequence lengths (L) and feature dimensions (D) configured to `L=96, D=256` and `L=192, D=512`. Across all tests, the total memory and time consumption remained **nearly identical** to training the backbone model alone.\n\n|Model GPU Usage (MB)|Ori|w/ Glocal-IB|Foundation Align (TimeMOE)|\n|-|:-:|:-:|:-:|\n|SAITS`L=96,D=256`|759|905|7553|\n|SAITS`L=192,D=512`|989|1151|13385|\n|TimesNet`L=96,D=256`|869|909|7503|\n|TimesNet`L=192,D=512`|905|965|13429|\n|DLinear`L=96,D=256`|591|619|7433|\n|DLinear`L=192,D=512`|655|765|13251|\n|TCN`L=96,D=256`|611|715|7525|\n|TCN`L=192,D=512`|723|1029|13883|\n\n|Model Training Time (s)|Ori|w/ Glocal-IB|Foundation Align (TimeMOE)|\n|-|:-:|:-:|:-:|\n|SAITS`L=96,D=256`|8.44|9.33|78.56|\n|SAITS`L=192,D=512`|5.80|7.56|49.14|\n|TimesNet`L=96,D=256`|5.84|7.89|77.86|\n|TimesNet`L=192,D=512`|4.77|7.32|49.14|\n|DLinear`L=96,D=256`|4.53|5.52|73.25|\n|DLinear`L=192,D=512`|2.86|5.16|46.59|\n|TCN`L=96,D=256`|20.36|25.76|97.90|\n|TCN`L=192,D=512`|14.48|22.25|66.02|\n\n\n\n> **Q3**: The paper compares Glocal-IB with foundation model alignment. Could the authors elaborate on why foundation models underperform and whether combining both approaches might help?\n\n**A**: This is because recent foundation models are typically trained on forecasting tasks with MAE/MSE. Consequently, the embeddings they produce are heavily biased towards signals relevant for prediction and often lack the rich semantic information necessary for imputation.\n\nThis limitation is evident in our experiments. For instance, when we used **TimeMoE**, a SOTA model from ICLR 2025, to align embeddings, the resulting performance gain was marginal, as shown in **Figure 3**. \n\nFurthermore, deploying such large foundation models incurs a substantial GPU memory cost and training time, as shown in the **table in Q2**. \n\nGiven these limitations, we chose to utilize our lightweight Glocal-IB paradigm to assist the model's training.\n\n\n\n> **Q4**: Can the authors comment on the interpretability of the learned latent space or the imputed values, especially in high-stakes domains like healthcare?\n\n**A**: In high-stakes domains like healthcare, clinicians often need to model a patient's year-long health trajectory from sparse data, such as only one or two annual check-ups. \n\nFrom an **imputed value perspective**, more accurate data gives clinicians a clearer view of a patient's health trends, enabling better interventions. \n\nFrom a **latent space perspective**, an accurate representation offers significant clinical information. When visualized with techniques like t-SNE, the latent space becomes an intuitive tool for patient assessment. Embeddings similar to healthy profiles suggest positive outcomes, while divergent ones can indicate potential disease risks. \n\nThe **Information Bottleneck (IB) principle** is key to this benefit. IB aims to learn a \"minimal yet sufficient\" representation. As a result, the learned latent space is more compact and more likely to correlate with key, interpretable clinical factors compared to standard models.\n\n\n\n> **Q5**: Could Glocal-IB be extended to other structured data imputation tasks (e.g., spatiotemporal graphs or tabular data)?\n\n**A**: A key advantage of our method is its generality. It is designed as a plug-in paradigm, making it agnostic to the specific backbone architecture or downstream task. We contend that the principle of aligning embedding spaces provides a fundamental benefit, enabling our method to enhance performance across a diverse range of models and tasks.\n\n\n\n> **W3**: Some mathematical derivations are dense and could benefit from more intuitive explanations or diagrams.\n>\n> **W4**: Builds on existing IB and contrastive learning principles, though the integration is novel.\n>\n> **Q6**: Combine Glocal-IB’s global-local mutual information loss with a flexible’s per-variable model selection to improve robustness under high missingness.\n>\n> **Q7**: Adjust the KL divergence or alignment loss weights based on variable skewness to avoid over-penalizing informative but skewed features.\n>\n> **Q8**: Extend Glocal-IB to support multiple imputations and ensemble predictions to reflect uncertainty.\n\n**A**: We appreciate the constructive feedback and will incorporate these revisions into the final version of our manuscript."}, {"review_id": "XhX6W24vDC", "reviewer": "Reviewer_3qMw", "summary": "The paper addresses time series imputation (TSI) under high missingness, observing that many models achieve low training loss (focusing on pointwise reconstruction) but perform poorly at inference. The authors identify an “optimization dilemma”: under severe missingness, models overfit local noise and fail to capture the global structure of the data. To remedy this, they propose the Glocal Information Bottleneck (Glocal-IB) training paradigm, which extends the information bottleneck framework by adding a Global Alignment objective. Concretely, Glocal-IB applies the model’s encoder to both the original (fully observed) sequence and the masked sequence. A contrastive/InfoNCE-style loss (via a one-layer MLP projector) aligns the latent of the masked input with that of its unmasked counterpart, and a KL-divergence term regularizes the latent (suppressing noise). In effect, the model is trained to retain global, semantic-level information (from the full data) even when much of the input is missing.\n\nExperiments on real-world datasets show that Glocal-IB consistently reduces imputation error compared to strong baselines. For example, on the ETT benchmarks, Glocal-IB achieves the lowest MAE and MSE in all cases, with up to 40% reduction in MSE over prior methods. The authors also visualize latent representations: without Glocal-IB, the latent distribution collapses under high missingness, whereas with Glocal-IB, the latent remains well-structured up to 90% missing. In summary, the key contribution is a model-agnostic training strategy that enforces alignment between masked and original latents, which empirically yields more robust, generalizable imputation under extreme missing data.", "questions": "1. **Computational Overhead:** You describe Glocal-IB as a \"lightweight\" paradigm. Could you please quantify the computational overhead? Specifically, what is the increase in training time and memory usage when applying Glocal-IB to a backbone like the Transformer, compared to its standard training?\n2. **Guidance on Hyperparameter Tuning:** Your sensitivity analysis shows that performance can depend significantly on the choice of loss weights, especially the regularization term. Could you provide some practical guidance or a heuristic for setting the hyperparameters α,β1​, and β2​ when applying Glocal-IB to a new dataset or model architecture? A more robust tuning strategy would significantly enhance the method's practical utility.\n3. **Characterizing \"Global Information\":** The Global Alignment loss is motivated by preserving \"global semantic features.\" Beyond the excellent t-SNE visualizations, can you provide more insight into what specific properties this alignment captures? For example, does it preferentially preserve long-term trends, periodicity, or specific types of inter-channel correlations?\n4. **Comparisons to other imputation methods:** The paper omits an empirical comparison with some recent imputation frameworks. How does Glocal-IB perform relative to diffusion-based methods (e.g., CSDI) or GAN-based imputation (e.g., ImputeGAN)? Including one of these baselines could strengthen the experimental claims. Similarly, how does Glocal-IB differ from the Conditional IB approach of Choi & Lee? A brief discussion or additional results would be helpful.\n5. **Training details – teacher/stop-gradient:** In Figure 2(c), the encoder processes the original data to produce a “target” latent Z′. Is this encoder a frozen copy or a separate network? In other words, during training, is the latent from the original sequence allowed to update the encoder weights? (The stop-gradient symbol suggests no.) Clarifying the implementation of the “alignment” (e.g., is the original-branch encoder a fixed copy?) would help. Also, is there a risk of “leaking” information from the original into the masked branch that could lead to overfitting?\n6. **Generalization to non-random missingness:** The experiments use random masking patterns. How would Glocal-IB handle more structured or non-random missing patterns (e.g., entire variables missing for long periods)? Since the global alignment uses the same sequence’s full version as the positive, if missingness occurs in long gaps, the alignment signal might be weak. Can the authors comment on this, or test a few non-Random scenarios?", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "- Quality\n  - Strengths\n    - The paper does an excellent job of identifying and illustrating the \"optimization dilemma\" with clear empirical evidence (Figure 1). This convincingly frames the core problem that existing methods, which focus solely on local reconstruction, fail to learn generalizable representations when data is sparse. This insight itself is a valuable contribution.\n    - The authors clearly identify a practical failure mode of existing TSI methods (good training fit but poor test imputation) and back it with diagnostic plots. The Glocal-IB method is well-specified: adding a contrastive alignment loss and KL regularizer to the standard encoder–decoder framework. Experiments are extensive (nine datasets, five missing rates, five random seeds), and both quantitative results (MAE/MSE tables) and qualitative latent plots are provided. Results indicate strong improvements across diverse datasets, suggesting robustness. Code is made available, aiding reproducibility.\n\n  - Weaknesses:\n    - The choice of baselines is incomplete: notable recent methods for generative imputation are missing. Comparing to Conditional Information Bottleneck (CIB) or other IB-based imputation would strengthen claims of superiority.\n- Clarity\n  - Strengths:\n    - Overall, the writing is clear and systematic. Key ideas (local vs. global information) are introduced in plain language, and Figure 1 nicely illustrates the problem. Figure 2 then contrasts the proposed paradigm with baselines. The Appendix is comprehensive (including a Societal Impact Statement and detailed limitations, per NeurIPS requirements).\n  - Weaknesses:\n    - The information-theoretic notation (Section 3.1) may be heavy for readers unfamiliar with IB, and the derivation of the InfoNCE alignment loss (Eq. 12–13) could use more intuition.\n    - The technical exposition is dense (heavy on IB theory and mutual information bounds); while Figure 2 helps, some readers may find the derivations hard to parse. A clearer summary of the final loss terms and training procedure could improve readability.\n\n- Significance\n  - Strengths:\n    - This work addresses a highly significant and practical problem in time series analysis. Imputation under high missingness is a common challenge, and the proposed model-agnostic training paradigm offers a robust solution. The insights into the failure modes of current TSI methods are also an important contribution to the field. The demonstrated improvements could have a considerable impact on applications in healthcare, finance, and energy systems.\n  - Weaknesses:\n    - The ablation study (Figure 5) shows that the model's performance is quite sensitive to the weight of the regularization loss (LRegθ​). While the authors acknowledge this, the paper would benefit from a more detailed discussion on the practical aspects of tuning the loss weights (α,β1​,β2​). Without a clear strategy, practitioners might find it challenging to apply the method effectively to new datasets or architectures, which could be a minor barrier to adoption.\n- Originality\n  - Strengths:\n    - The core idea — combining an information bottleneck with a global latent alignment — appears novel in the TSI literature. Prior work on IB for TSI (e.g. Choi & Lee’s CIB) still relied on local reconstruction losses, whereas here a separate global mutual-information objective is added. The paper’s formulation (alignment via InfoNCE between full vs. masked embeddings) seems new.", "was_revised": true, "final_justification": "Authors properly answered my questions and addressed my concerns and I believe this is a good paper and my evaluation would remain same.", "rebuttal": "Thank you very much for your valuable comments and detailed suggestions. Below is a report about how we address your comments.\n\n> **W1 and Q4**: Comparisons to other imputation methods: The paper omits an empirical comparison with some recent imputation frameworks. How does Glocal-IB perform relative to diffusion-based methods (e.g., CSDI) or GAN-based imputation (e.g., ImputeGAN)? Including one of these baselines could strengthen the experimental claims. Similarly, how does Glocal-IB differ from the Conditional IB approach of Choi & Lee? A brief discussion or additional results would be helpful.\n\n**A**: As shown in the **Table in reviewer cTKT's W1**, to validate its broad applicability, we conducted extensive experiments. We benchmarked against strong methods, including TimeCIB, CSDI, and US-GAN, and further demonstrated the efficacy of our approach by integrating it with diverse models such as DLinear and TCN. The results confirm that Glocal-IB consistently enhances imputation performance across this wide range of architectures.\n\n**Different from TimeCIB:** We contend the Information Bottleneck (IB) principle's true potential lies in creating a new training paradigm, not a single specialized model. Therefore, instead of designing a specific architecture, we propose a general training paradigm based on IB theory. Our approach enhances the global mutual information often ignored by previous models, helping them capture more comprehensive signals for superior imputation. A key advantage is its flexibility: it is a \"plug-in\" that can boost the performance of a wide range of existing Time-Series Imputation (TSI) models.\n\n\n\n> **W2**: The information-theoretic notation (Section 3.1) may be heavy for readers unfamiliar with IB, and the derivation of the InfoNCE alignment loss (Eq. 12–13) could use more intuition.\n>\n> **W3**: The technical exposition is dense (heavy on IB theory and mutual information bounds); while Figure 2 helps, some readers may find the derivations hard to parse. A clearer summary of the final loss terms and training procedure could improve readability.\n\n**A**: We appreciate the constructive feedback and will incorporate these revisions into the final version of our manuscript.\n\n\n\n> **W4 and Q2**: Guidance on Hyperparameter Tuning: Your sensitivity analysis shows that performance can depend significantly on the choice of loss weights, especially the regularization term. Could you provide some practical guidance or a heuristic for setting the hyperparameters α,β1, and β2 when applying Glocal-IB to a new dataset or model architecture? A more robust tuning strategy would significantly enhance the method's practical utility.\n\n**A**: As illustrated in **Figure 8** of the appendix, our model exhibits consistent sensitivity to hyperparameters across four datasets (ETTh1, ETTh2, ETTm1, ETTm2). The performance trends are remarkably similar, uniformly achieving optimal results when the hyperparameters are set to **0.5, 1e-6, and 0.5**, respectively. In accordance with this finding, we adopted this single, robust set of hyperparameters for all experiments conducted on the nine datasets presented in the main paper. We appreciate the constructive feedback and will incorporate these revisions into the final version of our manuscript.\n\n\n\n> **Q1**: Computational Overhead: You describe Glocal-IB as a \"lightweight\" paradigm. Could you please quantify the computational overhead? Specifically, what is the increase in training time and memory usage when applying Glocal-IB to a backbone like the Transformer, compared to its standard training?\n\n**A**: The computational demands are dictated solely by the chosen backbone model. Glocal-IB introduces negligible computational overhead. This efficiency is by design: we only use the encoder for a single forward pass to get an embedding without gradient backpropagation. \n\nTo validate this, we measured Glocal-IB's GPU memory and training time on datasets with sequence lengths (L) and feature dimensions (D) configured to `L=96, D=256` and `L=192, D=512`. Across all tests, the total memory and time consumption remained **nearly identical** to training the backbone model alone.\n\n|Model GPU Usage (MB)|Ori|w/ Glocal-IB|Foundation Align|\n|-|:-:|:-:|:-:|\n|SAITS`L=96,D=256`|759|905|7553|\n|SAITS`L=192,D=512`|989|1151|13385|\n|TimesNet`L=96,D=256`|869|909|7503|\n|TimesNet`L=192,D=512`|905|965|13429|\n|DLinear`L=96,D=256`|591|619|7433|\n|DLinear`L=192,D=512`|655|765|13251|\n|TCN`L=96,D=256`|611|715|7525|\n|TCN`L=192,D=512`|723|1029|13883|\n\n|Model Training Time (s)|Ori|w/ Glocal-IB|Foundation Align|\n|-|:-:|:-:|:-:|\n|SAITS`L=96,D=256`|8.44|9.33|78.56|\n|SAITS`L=192,D=512`|5.80|7.56|49.14|\n|TimesNet`L=96,D=256`|5.84|7.89|77.86|\n|TimesNet`L=192,D=512`|4.77|7.32|49.14|\n|DLinear`L=96,D=256`|4.53|5.52|73.25|\n|DLinear`L=192,D=512`|2.86|5.16|46.59|\n|TCN`L=96,D=256`|20.36|25.76|97.90|\n|TCN`L=192,D=512`|14.48|22.25|66.02|\n\n\n\n> **Q3**: Characterizing \"Global Information\": The Global Alignment loss is motivated by preserving \"global semantic features.\" Beyond the excellent t-SNE visualizations, can you provide more insight into what specific properties this alignment captures? For example, does it preferentially preserve long-term trends, periodicity, or specific types of inter-channel correlations?\n\n**A**: The t-SNE visualizations in **Figures 1, 4, 6, and 7** reveal a consistently well-aligned latent space for partially observed data. This provides strong evidence that global information—including long-term dependencies, periodicity, and even inter-channel correlations—is effectively captured and understood by models equipped with our Glocal-IB paradigm. \n\nTo further substantiate this claim, we will include visualizations of attention maps from the Transformer-based models in the final version of this paper.\n\n\n\n> **Q5**: Training details – teacher/stop-gradient: In Figure 2(c), the encoder processes the original data to produce a “target” latent Z′. Is this encoder a frozen copy or a separate network? In other words, during training, is the latent from the original sequence allowed to update the encoder weights? (The stop-gradient symbol suggests no.) Clarifying the implementation of the “alignment” (e.g., is the original-branch encoder a fixed copy?) would help. Also, is there a risk of “leaking” information from the original into the masked branch that could lead to overfitting?\n\n**A**: **On the Encoder Implementation**: The original and masked data are processed by the same encoder. However, they are handled differently with respect to gradient flow. Crucially, the encoder path for the original data functions is a frozen copy of the encoder and purely for inference; its gradients are detached and are not used to update the encoder's weights.\n\n**On Data Leakage**: The concern of data leakage is unfounded. Our method exclusively encodes the ground-truth values from the masked portions of the sequence. This is the very same information that a standard imputation model would use as its reconstruction target or label. Therefore, our paradigm introduces no additional information into the training process that would constitute a data leak.\n\n**On Overfitting**: Regarding overfitting, the high dimensionality of our embedding space (256-D) substantially increases the difficulty of the learning task. Consequently, it is highly unlikely for the model to overfit, particularly within the brief training period of 30 epochs used in our experiments.\n\n\n\n> **Q6**: **Generalization to non-random missingness:** The experiments use random masking patterns. How would Glocal-IB handle more structured or non-random missing patterns (e.g., entire variables missing for long periods)? Since the global alignment uses the same sequence’s full version as the positive, if missingness occurs in long gaps, the alignment signal might be weak. Can the authors comment on this, or test a few non-Random scenarios?\n\n**A**: To further evaluate the robustness of our method, we conducted additional experiments using **a block-wise missing pattern where contiguous 5x5 blocks** of data were masked. The results demonstrate that our approach still provides significant benefits.\n\nThe case of missingness over long, contiguous time periods can be considered analogous to the extreme 90% random missing rate setting. As our previous results show in the **Table in reviewer cTKT's W1** and **Table below**, even under these highly demanding conditions, our method consistently provides performance gains for models such as SAITS, and TCN.\n\n|BlockMissing-MAE|Ours|USGAN|DLinear|DLinear_MY|TCN|TCN_MY|\n|:-|:-:|:-:|:-:|:-:|:-:|:-:|\n|ETTh1-block-0.1|**0.3608**|0.4507|0.4435|`0.3923`|0.6242|0.6200|\n|ETTh1-block-0.3|**0.4226**|0.5113|0.5211|`0.4650`|0.6495|0.6366|\n|ETTh1-block-0.5|**0.4699**|0.5964|0.5778|`0.5346`|0.7365|0.7185|\n|ETTh1-block-0.7|**0.5109**|0.6923|0.7081|`0.6080`|0.7699|0.7521|\n|ETTh1-block-0.9|**0.5812**|0.7539|0.8970|`0.6758`|0.8341|0.8169|\n|ETTh2-block-0.1|**0.2732**|0.4135|0.4520|`0.3148`|0.7023|0.7002|\n|ETTh2-block-0.3|**0.3373**|0.5985|0.6287|`0.4434`|0.8057|0.7889|\n|ETTh2-block-0.5|**0.4280**|0.6922|0.7580|`0.5107`|0.8967|0.8597|\n|ETTh2-block-0.7|**0.4420**|0.7412|0.8511|`0.5669`|0.9500|0.9353|\n|ETTh2-block-0.9|**0.5285**|0.7466|1.1131|`0.6096`|0.9417|0.9268|\n|ETTm1-block-0.1|**0.2001**|`0.2516`|0.3887|0.3562|0.4923|0.4709|\n|ETTm1-block-0.3|**0.2208**|`0.3211`|0.4614|0.4097|0.5356|0.5276|\n|ETTm1-block-0.5|**0.2830**|`0.3882`|0.5266|0.5011|0.6130|0.5969|\n|ETTm1-block-0.7|**0.2916**|`0.4539`|0.5594|0.5141|0.6821|0.6491|\n|ETTm1-block-0.9|**0.3654**|`0.5604`|0.6385|0.5699|0.7346|0.7097|\n|ETTm2-block-0.1|**0.1899**|`0.2923`|0.3943|0.3605|0.5467|0.5097|\n|ETTm2-block-0.3|**0.2142**|`0.3803`|0.5453|0.4494|0.6886|0.6314|\n|ETTm2-block-0.5|**0.2807**|`0.4703`|0.6364|0.5227|0.7667|0.7381|\n|ETTm2-block-0.7|**0.3289**|0.5836|0.7091|`0.5661`|0.8409|0.8218|\n|ETTm2-block-0.9|**0.3581**|0.6193|0.7412|`0.5757`|0.8792|0.8529|"}, {"review_id": "CdPQQs0Ehx", "reviewer": "Reviewer_9tLB", "summary": "Summary: The authors propose a model-agnostic deep learning approach for time series imputation based on the Information Bottleneck (IB) theory. \n\nStrengths: 1. The proposed method is model-agnostic, enabling compatibility with various architectures. 2. It demonstrates superior imputation accuracy compared to common time series models. 3. The writing is clear and the narrative flow is accessible to readers. \n\nWeaknesses: 1. Limited innovation: Global Mutual Information Maximization and related solutions build on prior work without substantial novel contributions. 2. Motivational concerns: Does time series imputation inherently require temporal semantic meaning? The paper fails to quantify how such semantic information benefits real-world applications, lacking concrete justifications. 3. Inadequate comparative experiments: The study lacks sufficient baselines to rigorously validate the method’s effectiveness.", "questions": "Questions: 1. Appendix Equation 16 contains a mispositioned multiplication sign in the second line, potentially causing interpretational errors. 2. Reference formatting is inconsistent; underlines in citations should be removed. 3. Missing visualizations: The main text lacks graphical representations of Glocal-IB's performance dynamics. 4. Unfair baselines: The comparative experiments rely on prediction-focused models rather than imputation-specific architectures. At least 4–6 imputation-oriented models  should be added to ensure valid comparisons. 5. Ambiguities in Table 1: Define \"IMP\" and clarify the methodology for generating missing data patterns. 6. Flawed conclusion in Observation 5: TimesNet and SAITS already exhibit strong performance without Glocal-IB (e.g., SAITS maintains similarity to original data at 50% missing rate). Glocal-IB introduces minimal improvements, undermining the claim of significant enhancement. \n7. Unexplained mechanism in Observation 7: Why does a regularization weight of 0.01 degrade model performance? The authors must provide theoretical analysis of this phenomenon.\n8. Over-reliance on complex models: The study focuses on TimesNet, a sophisticated architecture. Adding analyses of simpler models (DLinear, MLP, TCN, Transformer) would mitigate model-specific biases.\n9. The publicly available code by the authors is incomplete.", "limitations": "No", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths: 1. The proposed method is model-agnostic, enabling compatibility with various architectures. 2. It demonstrates superior imputation accuracy compared to common time series models. 3. The writing is clear and the narrative flow is accessible to readers. \n\nWeaknesses: 1. Limited innovation: Global Mutual Information Maximization and related solutions build on prior work without substantial novel contributions. 2. Motivational concerns: Does time series imputation inherently require temporal semantic meaning? The paper fails to quantify how such semantic information benefits real-world applications, lacking concrete justifications. 3. Inadequate comparative experiments: The study lacks sufficient baselines to rigorously validate the method’s effectiveness.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your valuable comments and detailed suggestions. Below is a report about how we address your comments.\n\n> **W1**: Limited innovation: Global Mutual Information Maximization and related solutions build on prior work without substantial novel contributions.\n\n**A**: Existing Time Series Imputation (TSI) methods, guided by training objectives such as MAE or MSE, exhibit a tendency to over-focus on numerical details. This makes them susceptible to noise and redundant signals within the data, leading them to neglect crucial information about the overall distribution. Consequently, these methods encounter a significant optimization dilemma under high missing rates: a lower training loss paradoxically results in a more distorted embedding distribution and, ultimately, inferior test performance. Therefore, **how to encourage TSI models to *capture both global and local information* from incomplete data *without overfitting to noise*** emerges as a critical research problem.\n\n**The Information Bottleneck (IB) principle offers a powerful theoretical framework for this challenge, as it seeks to compress data to its minimal sufficient statistics (capture both global and local information), thereby filtering noise (avoid overfitting) while preserving essential information.** However, current IB-based methods like GPVAE and TimeCIB fall short. They primarily introduce regularization terms to filter out irrelevant signals, but their training objective remains identical to non-IB methods, relying on MAE/MSE for reconstruction. This fails to provide the model with additional, more informative signals. Moreover, the stringent regularization, which often forces the latent distribution towards a standard normal, $\\mathcal{N}(0,I)$, can unnaturally constrain the model's capacity. This results in a simplistic spherical embedding distribution as shown in **Figure 1's Line 3**, which has a detrimental effect on performance.\n\n**We contend that the true potential of the IB principle lies not in creating a single, specialized model, but in establishing a new, more effective training paradigm**. In this paper, we depart from the design of a specific architecture and instead propose a general training paradigm based on the IB theory. **Our approach enhances the global mutual information often ignored by previous models**, enabling them to capture more comprehensive data signals for superior imputation. A key advantage of our paradigm is its flexibility; it is designed as a \"plug-in\" that can be applied to a wide range of existing TSI models to boost their performance. We have validated this through extensive experiments, demonstrating consistent improvements across various benchmarks.\n\n\n\n> **W2**: Motivational concerns: Does time series imputation inherently require temporal semantic meaning? The paper fails to quantify how such semantic information benefits real-world applications, lacking concrete justifications.\n\n**A**: To isolate the contribution of the semantic-level guidance provided by our method, we first conducted a detailed ablation study. As shown in **Figures 5(a) and 9**, we compared model configurations trained **with** our global loss term, $\\mathcal{L}_{Glo}$ (labeled as \"Entire\" and \"w/o Reg\"), against those trained **without** it (\"w/o Glo\" and \"only Loc\"). The results consistently demonstrate that models incorporating the semantic-level guidance from LGlo achieve superior performance, which validates the direct benefit of this component.\n\n\n\n> **Q8**: Over-reliance on complex models: The study focuses on TimesNet, a sophisticated architecture. Adding analyses of simpler models (DLinear, MLP, TCN, Transformer) would mitigate model-specific biases. \n\nTo demonstrate its broad applicability, as shown in **Figure 3**, the **Table in reviewer cTKT's W1** and the **Table in reviewer 3qMw's Q6**, we integrated the full Glocal-IB paradigm with several TSI methods, including **SAITS, TimesNet (in the paper), DLinear, and TCN (new)**. These experiments confirm that applying our training paradigm leads to significant performance gains over standard training with only an MAE/MSE loss. \n\n\n\n\n> **W3**: Inadequate comparative experiments: The study lacks sufficient baselines to rigorously validate the method’s effectiveness.\n>\n> **Q4**: Unfair baselines: The comparative experiments rely on prediction-focused models rather than imputation-specific architectures. At least 4–6 imputation-oriented models should be added to ensure valid comparisons.\n\n**A**: As shown in the **tables in W2**, to ensure a more comprehensive evaluation, we have expanded our experiments to include additional strong baselines: TimeCIB, CSDI, US-GAN, and TCN. Furthermore, to demonstrate the broad applicability of our proposed method, we have integrated it with DLinear and TCN. The results confirm that our approach yields significant performance improvements for both of these models.\n\n\n\n> **Q1**: Appendix Equation 16 contains a mispositioned multiplication sign in the second line, potentially causing interpretational errors. \n>\n> **Q2**: Reference formatting is inconsistent; underlines in citations should be removed.\n\n**A**: We appreciate the constructive feedback and will incorporate these revisions into the final version of our manuscript.\n\n\n\n> **Q3**: Missing visualizations: The main text lacks graphical representations of Glocal-IB's performance dynamics.\n\n**A**: Thank you for the constructive feedback. To ensure we address your comment effectively, we would appreciate it if you could clarify what you mean by 'performance dynamics'.\n\n\n\n> **Q5**: Ambiguities in Table 1: Define \"IMP\" and clarify the methodology for generating missing data patterns.\n\n**A**: In our evaluation, we measure the performance gain as \"Improvement (IMP)\". The missing data for these experiments were generated following the standard random masking protocol from the **`PYPOTS`** benchmark. Specifically, for each time-series sample, a binary mask of identical dimensions is created, where the proportion of masked entries (represented as 0s) corresponds directly to the specified missing rate. We will revise this in the final version.\n\n\n\n> **Q6**: Flawed conclusion in Observation 5: TimesNet and SAITS already exhibit strong performance without Glocal-IB (e.g., SAITS maintains similarity to original data at 50% missing rate). Glocal-IB introduces minimal improvements, undermining the claim of significant enhancement.\n\n**A**: The effectiveness of our approach is demonstrated through multiple visual and quantitative results. In **Figure 4 and 7**, we visualize the embedding distributions for SAITS and TimesNet after applying Glocal-IB. It is evident that for missing rates between 0% and 70%, the learned distributions maintain a high degree of similarity to the original data distribution, indicating a successful preservation of global structure.\n\nThe magnitude of this enhancement is further highlighted in **Figure 3**, where our method boosts the models' performance to a level equivalent to what they would achieve with a 20% lower missing rate—a substantial and significant improvement.\n\n\n\n> **Q7**: Unexplained mechanism in Observation 7: Why does a regularization weight of 0.01 degrade model performance? The authors must provide theoretical analysis of this phenomenon.\n\n**A**: This phenomenon is attributed to a significant disparity in the orders of magnitude among the different loss components. The KL divergence loss, for instance, often reaches a scale of approximately $10^{5}$. In contrast, the other two loss terms (global and local) are substantially smaller, on the orders of $10^{1}$ and $10^{−1}$, respectively.\n\nConsequently, a small scaling factor, such as 0.01, is insufficient to bring the gradient from the KL loss into a range comparable to that of the other terms. This large imbalance effectively marginalizes the guidance from the global and local losses, severely impairing their influence on the model during training and ultimately compromising the final performance.\n\n\n\n> **Q9**: The publicly available code by the authors is incomplete.\n\n**A**: To foster reproducibility and encourage future research, we will release the complete open-source code in an upcoming version. The implementation of Glocal-IB will be refactored into a modular, plug-and-play format to facilitate its seamless integration with arbitrary models."}]

Author rebuttal:

Meta-review: This paper addresses a key challenge in time series imputation: models often attain low training loss under high missing rates but experience significant drops in imputation quality and distorted latent representations at inference. To mitigate this issue, the authors propose Glocal Information Bottleneck (Glocal-IB), a model-agnostic extension of the Information Bottleneck framework. Experiments across diverse datasets show that Glocal-IB consistently improves performance.

In the rebuttal, the authors have carefully addressed most of the concerns raised by the reviewers. Three reviewers gave positive scores on the paper, and the only negative reviewer did not participate in the author-reviewer discussion phase.

I have read all the comments and responses from the reviewers and authors, and believe the strength of the paper is sufficient for acceptance.
Decision: Accept (poster)