Long Short-Term Imputer: Handling Consecutive Missing Values in Time Series
Authors: Jiacheng You
Xinyang Chen
Yu Sun
Weili Guan
Liqiang Nie
Venue: Accepted by TMLR ()

Abstract: Encountered frequently in time series data, missing values can significantly impede time-series analysis. With the progression of deep learning, advanced imputation models delve into the temporal dependencies inherent in time series data, showcasing remarkable performance. This positions them as intuitive selections for time series imputation tasks which assume ``Miss Completely at Random''. Nonetheless, long-interval consecutive missing values may obstruct the model's ability to grasp long-term temporal dependencies, consequently hampering the efficacy of imputation performance. To tackle this challenge, we propose Long Short-term Imputer (LSTI) to impute consecutive missing values with different length of intervals. Long-term Imputer is designed using the idea of bi-directional autoregression. A forward prediction model and a backward prediction model are trained with a consistency regularization, which is designed to capture long-time dependency and can adapt to long-interval consecutive missing values. Short-term Imputer is designed to capture short-time dependency and can impute the short-interval consecutive missing values effectively. A meta-weighting network is then proposed to take advantage of the strengths of two imputers. As a result, LSTI can impute consecutive missing values with different intervals effectively. Experiments demonstrate that our approach, on average, reduces the error by 57.4% compared to state-of-the-art deep models across five datasets.

Published in Transactions on Machine Learning Research (02/2025)
Long Short- Short-Term Imputer:
Handling Consecutive Missing Values in Time Series
Jiacheng You
jiachengyon2hit@@maal.cc
School of Computer Science and Technology, Harbin Institute of Technology (Shenahen)
Xinyang Chen
chencinnynng55@@maall.cm
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)
Yu Sun*
sungu@nonkai.edui edu.ccm
College of Computer Science, DISSec, Nankai University
Weili Guan
guamwelti@hit.edue edu.cn
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)
Liqiang Nie
nieliqiang@gmail. com
School of Computer Science and Technology, Harbin Institute of Technology (Shenahen)
Reviewed on OpenReview: https: // openreview. net/ forum? id= 9NVJ02gEfT
Abstract
Encountered frequently time series data, missing values can significantly impede time-series analysis. With the progression of deep learning, advanced imputation models delve into the temporal dependencies inherent in time scries data, showcasing remarkable performance. This positions them as intuitive selections for time series impulation lasks which assume "Miss Completely at Random". Nonetheless, long-interval consecutive missing values may obstruct the model's 's ability to grasp long-term temporal dependencies, consequently hampering the efficacy of imputation performance. To tackle this challenge, we propose Long Short-term Imputer (T.TTT) impute consecutive missing values with different. length of intervals. Tong- term Imputer is designed using the idea of bi-directional autoregression. A forward prediction model and backward prediction model trained with a consistency regularization, which is designed capture long-tin tong-time dependency and adapt to internnerval consecutive missing values. Short-term Imputer is designed to capture short-time dependency and can impute the short-interval consecutive missing values effectively. A meta-weighting network is then proposed to take advantage of the strengths two imputers. As a result, LSTI can impute consecutive missing values with different intervals effectively. Experiments demonstrate that our approach, on average, reduces the error by 57.4% compared state-of-tho-art deep models across five datasets.
1 Introduction
Multivariate time series are abundant in real-world applications, including meteorology, finance, engineering,
science, and healthcare (Hajihashemi & Popescu 2015 Prakosa et al. 2012 Gupta et al. 2020 Tascikaraoglu
et al. 2016 Zheng al. 2015). These time series data often contain missing values due to reasons such
scnsor failures and communication losses (Silva et al. 2012 Yi et al. 2016 These missing values impair the
integrity and interpretability of the data, posing significant challenges to time series analysis (Hasan et. al.
2021 Therefore, effectively imputing missing data before analysis essential.
* *Corresponding authors.

---

Published in Transactions on Machine Learning Research (02/2025)
Timeline
Variate
Missing Completely at Random
Consecutive Missing Interval
Figure 1: The difference between MCAR and intginntrvval consecutive missing values.
Many studies have addressed the task of imputing missing values using deep learning techniques (Wu et al.
2022: Du et al. 2023 Tashiro al. 2021), and most of them are designed for scenarios involving Missing Completely at Random (MCAR). However, in the real world, missing values often occur consecutively and in clusters due to factors such as signal loss, environmental interference, and equipment failure. For example, when travels through tunnel, all satellite signals related to the car will be blocked by the tunnel, resulting in long interval of consecutive missing values in the time series data.
Following the scheme in Khayati et al. (2020), we categorize the conseculive lime series missing patterns inlo four distinct types: 'Disjoint', 'Overlap" 'MCAR_D B', and 'Blackout', Each pattern represents a specific form of long-interval consocutive missing data. This paper concentrates on the 'Blackout' pattern, considered the most challenging, where missing values across all channels are aligned in the same positions. This synchronized absence of data introduces significant difficulties for standard imputation methods, it eliminates the possibility of leveraging information from other channels time points to reconstruct the missing values. illustrates the differences between MCAR and Blackout missing patterns.
Figure |
Traditional imputation methods perform poorly on this type of data due to their inadequate long-term modeling capabilities. Some methods attempt to address this problem in specific application domains (Ma) ct al. 2020 Chen et al., 2021 Guo al. (021), and other methods utilize mathematical and statistical techniques to tackle this issue [Wongoutong] 2020). However, these methods require domain knowledge and complex processing workflows, and they are not end-to-end deep learning solutions, thus limiting the range of problems they can effectively solve.
To address the problem of imputing Blackout data, we propose LSTI, Long Short-Term Imputer specifically designed for this purpose. The model consists of three main components: first, a Long-Term Imputer, which is composed of a forward prediction network and a backward prediction network. During training, an idnalional consistency loss introduced to minimize the discrepancy between the prediction of two networks. During imputation, the two prediction networks autoregressively impute the entire sequence in forward and backward directions, respectively, with the final imputation result obtained a weighted sum of the two which follows the idea that the closer the values in the prediction window are to the lookback window, the smaller cumulative errors are. This structure effectively captures long-term dependencies in time series data and is suitable for imputing long-interval consecutive missing values.
The second component the Short-Tern Imputer. Missing data real-world often a mixture of various missing patterns, including MCAR and long-interval consocutive missing values. This requires a model that can adaptively impute different types of missing data. The Short-Term Imputer consists of self-mapping network, trained with randomly generated consecutive missing mask to capture short-term dependencies in the time series data. Subsoquently, LSTI employs a Meta-weighting module to learn the importance of long-term and short-term dependencies in the current data, and uses this to weight the outputs of the long-term and short-term imputers to obtain the final imputation result. This structure can simultaneously learn both long-term and short-term dependencies in time series data, enabling adaptive and effective imputation across various missing data patterns.
Our main contributions summarized as follows:
2

---

Published in Transactions on Machine Learning Research (02/2025)
We propose the Long-Term Imputer to capture long-term dependencies in time series data. It uses forward and backward prediction networks to antoregressively impute missing values from both directions. Additionally, a consistency loss is introduced during training to minimize the discrepancy between the two predictions.
We propose the Short-Term Imputer to caplure short-term dependencies in time series data. A Meta- wettawweghttig module is used learn the proportion of long-term and short-term dependencies in specific data, adaptively balancing the weights of long-term and short-term dependencies. This allows LSTI to effectively impute data under various missing patterns.
We conduct extensive experiments on five real-world datasets, and our method outperformed the current state-of-the-art imputation methods on all datasets.
2 Related Work
2.1
Time Series Imputation
Deep learning models are inherently good at capluring complex and non-linear relationships in data. RNN- based models are more commonly used in earlier works. GRU-D (Che et al. 2018), variant of the Gated Recurrent Unit (GRU), addresses missing data in time series classification tasks. Soon after, BRITS (Cao et al. 2018 imputes missing values using a bidirectional recurrent dynamical system, without specific assumptions. Tn the case of M-RNN (Yoon et al. 2018). it imputes missing values based on hidden states derived from bidirectional RNNs. Since the generation ability naturally suited to the imputation task, generative models are then widely used in filling missing values. E2GAN (Luo et al., 2019) incorporates generator based on GRUT within aulo-encoder framework. For spatiotemporal sequence imputation, NAOMI (Liu et al.) 2019) introduces non-autoregressive model to comprise a bidirectional encoder and a multiresolution decoder. With diffusion models gaining popularity, CSDI (Tashiro et al. 2021 emerges, which is a conditional score-based diffusion model designed for time-series imputation. Advancing further, TimesNet (Wu et al., 2022) expands the examination of temporal variations into the two-dimensional space, while considering the presence of multiple periodicity in time series data. NRTSI (Shan et al., 2023 is an approach for time-series imputation that treats time series as collection of (time, data) pairs. SAITS (Dul et al. 2023 conducts simultaneous reconstruction and imputation by employing a weighted combination of two dingonally-masked self-attention blocks.
Each method discussed above possesses distinct advantages, but they are all designed for scenarios involving Missing Completely at Random (MCAR). When handling long-interval consecutive missing data, these methods may produce suboptimal results. Some mothods attempt to address the imputation of consecutive
missing data using specific domain knowledge in specific applications, such as air pollution (Ma et al., 2020 water quality prediction (Chen et al.] (002). and electrical engineering (Guo al. 2021]. Another method attempts tackle this problem using statistical approach (Wongoutong) 2020). However, these approaches require complex processing workflows and are not end-to-end methods, leading to limited applicability.
2.2 Time Series Forecasting
Time series forecasting methods potentially can be used to predict long-interval consecutive missing data. Specifically, the values prior to the missing interval are used to predict the entire missing segment. Some RNN-based models demonstrate robust forecasting performance in earlier studies, such as LSTNet (Lai et al.,
2018), which combines networks of DNNs, RNNs, and Skip-RNNs. DeepAR (Salinas et al. 2020 utilizes autoregressive recurrent network architecture to output the probabilities of prediction points. In more recent rescarch, Transformer-band based models gain significant prominence. Early on, Informer (Zhou et al. 2021 utilizes self-atlention distilling to significantly accelerate the training elficiency of Transformers; following this, Autoformer (Wu et al. 2021) embeds time series decomposition into the model, enhancing its capability discover temporal dependencies; Non-stationary Transformers (Liu et al. 2022) takes a novel approach by incorporating non-stationarity factors into the attention mechanism to prevent the over-stabiliation of
time series data; PatchTST (Nie et al. 2022) achieves remarkable results through the use of patches and
3

---

Published in Transactions on Machine Learning Research (02/2025)
SMNet
Short-Term
Imputer
Timeline
Mask
Gate
1
Long-Ten -erm Imputer
Lb
f(x)
BPNet
Lc
11
FPNet
The overall training process of LSTI. LSTI consists of three main parts: (1) the Long-Term Term Imputer, which is composed of two prediction networks operating in different directions; (2) the Short-Term Imputer, which consists of a malping module; (3) the weightiigg module, which learns to weigh the outputs of the Imputers and can adapt specific input data.
Figure 2:
channel independence; most recently, iTransformer (Liu et al. 2023) transposes the input matrix and reverses the roles of the attention mechanism and feed-forward network, better capturing the correlations among multivariate time serics.
Although forecasting methods appear promising for addressing long-interval consecutive missing data, various issues arise in practice. First, a series requiring imputation may have multiple missing points, and there is no guarantee that each missing segment precoded by a complete, available segment for forecasting. Second, the endpoints of the predicted sequence cannot be effectively aligned with the observed values following the missing segment, leading deviations later forecasted values from actual values. Therefore, forecasting methods cannot be directly applied to impute long-interval consecutive missing data.
3 Long Shrrt-Term Imputer
Problem Formulation We consider a collection multivariate time series X = fFITTI RTXC with T timestamps and C channels (attributes). The imputation task to impute the missing values in X. Formally, observation mask is defined M (m1T11c RT'CC where my,c 0 if diec missing, and my if Ft.c is observed.
In practical scenarios, consccutive missing values can impact the matrix M. The initial length of consecutive missing values is denoted as Lm- There are random submatrices Xagar 1.1:0 C. X where values are missing. Specifically, the observation mask can then be expressed as:
K
0. if Ft. missing,
my. II (1 1aax St< as + Lm)), Mi.c
(1)
1. if Tte is observed.
4

---

Published in Transactions on Machine Learning Research (02/2025)
where ag represents the randomly selected starting position of the missing interval, and K denotes the total number of missing intervals that appeared in the original data.
Motivation Current imputation methods generally assume that test data is missing completely at random (MCAR). However, in the real world, missing values often occur consecutively and in clusters due to factors such as signal loss, environmental interference, and equipment failure. Mainstream imputation methods may not perform well when dealing with long-interval consecutive missing data because they are limited at learning long-term dependencies. Based on this practical concern, we design the Long Short-Term Imputer (LSTI), which can effectively impute consecutively missing data by learning both long-term dependencies and short-term dependencies with specially designed expert models.
3.1 Overall structure
The LSTI model consists of the following modules: (1) Long-term Imputer: This module captures long-term dependencies in time series data. It includes forward and backward prediction networks, which use an additional consistency loss during training to minimize the discrepancy between them. During imputation, the forward and backward networks autoregressively impute the sequence in both directions, followed by a weighted ensemble for each missing interval. (2) Short-term Imputer: This module captures short-term dependencies in time series data. It includes a self-mapping imputation network, trained with a randomly generated consecutive missing mask. During imputation, it uses a sliding window approach for local imputation of the sequence. (3) Mcta-weighting modulc: This module learns to weigh the outputs of the long- fingttemm and short-term Imputer considering the properties of specific input data. The final imputation result is obtained by ensembling the outputs of two imputers using the Meta-weighting approach. The overall framework of LSTI is shown in Figure It is worth mentioning that the backbone of Long-term Imputer and Short-term Imputer can be any advanced backbone designed for time series, e.g., transformer- baase model, RNN-basod model or TCN-based model.
3.2 Long-term Imputer
The long-interval consecutive missing values may hinder the model from learning long dependencies. To alleviate the problem, two insights are employed to model long-term dependencies. First, the long-term dependencies can be learned in two directions. Thus Long-term Imputer includes a forward prediction network and backward prediction network, which aim to learn long-term dependencies in cach direction and impute missing data in an autoregressive manner. Second, the prediction results using the aforementioned two kinds of long-term dependencies should be consistent. As a result, during training, we incorporate a consistency loss to reduce the output discrepancy between the two networks to further enhance the effectiveness of learning long-term dependencies.
3.2.1 Training process
Bidirectional Prediction. Denote the input length of the prediction network S and the prediction length as L, for training dataset X,, R(S+ composed of input for forward prediction Xy RSC the ground-truth values to be predicted X, E RLXC, input for backward prediction X5 E RSCC we reverse time order of X to obtain X Specifically, we have:
X or
X1), (xf)ss (xb)s-
(2)
This indicates that X is a submatrix consisting of the first S rows of Xtr, while X is a submatrix formed by reversing the last S rows of X,r- Xy and X are fed into the forward prediction network (FPNet) and the backward prediction network (BPNet), respectively, to obtain the network outputs Y,, Y E ((S+L)xC Y, consists of prodicted lookback window Y E RSXC and prediction window fp E RLxC and similarly for Y₁-- Specifically, we have:
Y, (P),
Y/= = (PNNe((())
(3)
Y, = YYl.
Y = [X Y/J,
5

---

Published in Transactions on Machine Learning Research (02/2025)
where y: Yo Yop is the reversal of Yo Yos. bp. The network outputs both the lookback window and the prediction window values to ensure that the model's output matches its input, resulting in comprehensive training process and a more stable autoregressive process. Therefore, the forward prediction loss Lj and backward prediction loss Ls need to match both the lookback window and the prediction window values:
= MSE(( X,J, y),, La = MMMM( Yn).
(4)
Consistency Loss. The prediction windows output fFp Yip by the two networks corresponding to the same segment, X,, If the long-term dependencies can be captured by two networks, their predicted values for the same segment should be as close possible. As result, we propose consistency loss to further reduce potential error accumulation in the autoregressive process:
Le = MSE(Y /p, Ytp).
55)
Emsembling. Given prodictions from both directions, we can ensemble them as the output of Long-term Imputer. An intuitive idea is that the closer the values in the prediction window are to the lookback window, the smaller cumulative errors are. Therefore, values at the beginning of the interval should be more aligned with the forward network's output, while values at the end of the interval should be more aligned with the backward network's output. We use linear function f(i) to weigh and ensemble the two prediction outputs, and compute the loss between the final sequence and the ground truth:
i-1 --0-1-1 L-1'
f(1)
Y; (1- f) Yip + f Yop. C, MSE(X, Y.).
(6)
The overall training objective of Long-tern term Imputer C the sum of the aforementioned losses:
Li Ls Lo Le byp
(7)
3.2.2 Imputation process
In real-world datasets, there may be multiple consecutive missing segments, which can result in missing values in the network's input. To address this issue, the Long-term Imputer uses a bidiroctional autoregressive approach impute the entire missing data once during the imputation process. This ensures that the data input to the network is either true value a previously imputed value, thereby avoiding the issue of missing data in the input.
Bidirectional Autoregression. Consider the case of the forward prediction network. We scan the entire dataset from front to back along the time dimension. If missing values are found in the data segment D& E R4XC and the corresponding missing mask M2, we select the preceding data segment D E RRSx from D3 and input it into the forward prediction network:
Eat, Esp] (Du), D M D3 + (1-M+) Exp-
(8)
where Dj is the imputed D2 sequence. We use D2 to replace D ensuring that this sequence no longer contains any missing values. Then, we continue scanning the entire dataset and process the next segment. D
The backward prediction network performs the same autoregressive imputation for the entire missing data from the opposite direction. Subsequently, we rescan the entire original missing data, and for each consecutive missing interval H2 RLxxC we emsemble the forward and backward imputation results using the method in Equation 6 to obtain the final imputed sequence H.
3.3 Short-term Imputer
Missing data in the real world is typically represented as a combination of single-point missing values and long consecutive missing intervals. To accommodate various missing patterns, follow the idea of vanilla

---

Published in Transactions on Machine Learning Research (02/2025)
imputation models and introduce Short-term Imputer to capture short-term dependencies in time series data. This module consists of single self-mapping network (SMNet). For the training data in Short-term Imputer, X E RLxC which is the middle segments of training data in Long-term Imputer (which is also illustrated in Figure 2 is used as input, and we randomly gencrate a consecutive missing mask M, and input them into the self-mapping nerwork:
Z, SMNet(X,, M.), L. = MSE((1 M.) X,, (1-M)) Z.).
During imputation, partition the entire dataset using sliding window of length L and input the segments into the Short-term Imputer for impulation. Finally, the Short-term Impuler outpuls the imputed data. I.
3.4 Meta-weighting module
To adaptively impute data with mixture of long- terin and short-term missing values, we designed the Meta-weighting module. This module contains standalone MLP that learns the characteristics of the missing data to adaptively allocate weights to the long-term and short-term imputer outputs. During training, after obtaining the output Y, from the Long-tere term Imputer and Z, from the Short-term Imputer, we input the data X, and the missing mask M, into the Meta- weightteigging module:
X = MLP(X, M.), =1 (1-11-)) Y, 7 Z..
(10)
The training loss for Meta- weightin output is:
L. MSE((1 M,) X (1- M,) &.).
(11)
The overall training objective of LSTT is the sum of the losses from all modules:
L = Li + L. + Lt.
(12)
During imputation, after obtaining the output H from the Long-term Imputer and i from the Short-term Imputer, we use the same sliding window as the Short-term Imputer to get D2 and its missing mask M, from the original data. These partitions are then input into the Meta-weighting module to obtain Ag- For each window, we use the method in Equation 10 to weighttig Hg and fg in proportion to get the final imputed data.
4 Experiments
In this section, we conduct experiments five real-world datasets and compare them with the current. mainstream imputation and forecasting methods. The experimental results indicate that our approach significantly outperforms existing methods in cases of long-interval consecutive missing data.
4.1 Experimental Setup
Datasets We use five real-world datasets in experiments. (1) Electricity
(UCI collects hourly electricity
consumption data of 321 customers from 2012 to 2014. (2) Traffic (PeMS) contains hourly road occupancy rates measured by 862 sensors on San Francisco Bay area frecways from Jamuary 2015 to December 2016.
(3) METR-LA (Metro) records four months of statistics on traffic speed on 207 sensors on the highways of Los Angeles County. (4) Guangzhou (Chen et al. 2018) records traffic speeds per ten minutes on 214 anonymous roads in Guangzhou from August 1, 2016 to September 30, 2016. (5) PEMS04 (Chen al., 2001 is subset of PEMS, collected by 307 detectors over continuous period of 59 days, starting from January 1, 2018.
Baselines We compare our method with several current mainstreeam imputation approaches, including Transformer |Vaswani et al. 2017), SAITS (Du et al.) 2023), TimesNet (Wu et al. 2022). CSDI Tashiro et al. 2021), and ImputeFormer (Nic al. (024).. Additionally, we also conduct comparisons
with long-term forecasting methods using TimesNet (Wu 2022 and iTransformer (Liu al. 2023
to validate the effectiveness of our approach.

---

Published in Transactions on Machine Learning Research (02/2025)
Blackout Window
Blackout Global
0
-2
-2
window_ 1
window_2 2
window_3
window 1
window_2 2
window
(a) Each window contains gap of uniform length.
(b) The gaps can vary across windows.
Figure 3: Blackout missing patterns under Lm 50 and R.. 50%. On the left, cach window has fixed missing segment of length 50. On the right, the missing segments are independent of the windows, potentially occurring at any position. The missing length and ratio may vary across windows.
Generating missing data The Blackout missing pattern has been explored in previous studies (Khayati et al. 2020 Alcaraz & Strodthoff 2022). but it often constructed with missing data strictly aligned to imputation windows, scenario that is uncommon in real-world datasets. In contrast, we propose a more realistic approach, where Blackout segments are randomly scattered across the entire dataset, independent of any specific imputation windows. As shown in Figure 3 generate missing datasets using the following method: For a given missing length Lm and a missing rate Rm: we randomly seleet intervals of length I, in the data and mark them as missing until the overall missing rate Rm is reached. In our experiments, we conduct controlled variable comparisons by keeping either Lm or Rm constant while varying the other.
Implementation details Our LSTI is general training framework. Hence the backbone of FPNet, BPNet and SMNet can be any advanced deep network specially designed for time series. In our experiments, we use TemesNet as the backbone in LSTI, employing the same hyperparameter settings as the original paper, namely S is 96 and 96. In other baseline models, hyperparameters are according to the configurations specified in their original papers. We use MAE and MSE as the metrics for our experiments.
4.2 Main Results
To investigate the imputation capabilities of LSTI in various scenarios and its performance under different missing lengths Im and missing rates Rm, conduct controlled variable comparisons by keeping cither Im or Rm constant while varying the other. Experimental results show that our method outperforms the current. state-of-the-art models in all cases.
4.2.1 Varying missing length
We investigate the imputation performance of different methods real datasets with varying missing lengths, and the corresponding results are presented in Table 1 From the table, it can be observed that our method outperforms traditional imputation and forecasting methods across all datasets. For instance, regarding the MSE metric, our method leads to an average improvement of 72.32%, 55.43%, 40.27%, 54.54%, 68.29% compare to other methods on Flectricity, Traffie, METR-LA, Guangzhou, and PEMS04 datasets, respectively. Additionally, we observe that as the length of consccutive missing data incrcases, the performance traditional imputation and forecasting methods declines significantly, whereas our method exhibits a smaller decline across most datasets. This indicates that the length of consecutive missing data has substantial impact traditional imputation methods, whercas our method effectively imputes consocutive missing data and demonstrates better adaptability to datasets with consecutive missing values.
We conduct experiments on the Electricity dataset with varying lengths of consecutive missing data, and the results are shown in Figure 4al As the length of consecutive missing data increases, the performance of all imputation methods declines varying degrees. Models that perform reasonably well under low levels of consecutive missing data exhibit poor performance at higher levels. In contrast, method is less impacted
8

---

Published in Transactions on Machine Learning Research (02/2025)
Table 1: Imputation results on real datasets with different missing lengths and a fixed 30% missing rate. The
label F indicates models using forecasting methods.
LSTI
Transformer
CSDI
SAITS
TimesNet
Imputeformer
TimosNet
F
[Transoormer
Lm
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE]
MSE
Electricity
0.249
0.138
0.739
0.842
0.441
0.424
0.785
0.936
0.494
0.474
0.759
0.868
0.348
0.245
0.382
0.297
30
0.242
0.129
0.782
0.923
0.546
0.616
0.798
0.965
0.571
0.618
0.788
0.926
0.330
0.222
0.372
0.290
50
0.255
0.144
0.806
0.961
0.658
0.765
0.809
0.982
0.686
0.844
0.793
0.920
0.375
0.296
0.395
0.339
100
0.278
0.171
0.830
1.029
0.774
1.005
0.828
1.041
0.784
1.008
0.814
0.976
0.433
0.407
0.434
0.414
300
0.370
0.251
0.858
1.088
0.837
1.107
0.835
1.063
0.820
1.079
0.815
0.976
0.699
0.805
0.695
0.802
10
0.269
0.296
0.743
1.129
0.590
0.816
0.762
1.158
0.603
0.830
0.739
1.143
0.374
0.456
0.401
0.497
Traffic
30
0.281
0.316
0.750
1.134
0.599
0.863
0.762
1.150
0.654
0.951
0.742
1.132
0,402
0.511
0.408
0.504
50
0.301
0.341
0.762
1.164
0.555
0.871
0.769
1.166
0.697
1.056
0.750
1.124
0.389
0.478
0.373
0.454
100
0.334
0.381
0.773
1.194
0.546
0.808
0.777
1.195
0.790
1,298
0.771
1.193
0,470
0,614
0.476
0,630
300
0.450
0.547
0.782
1.213
0.635
0.972
0.782
1.210
0.785
1.240
0.769
1.207
0.663
0.986
0.670
1.002
METR-LA
0.342
0.353
0.674
1.119
0.411
0.826
0.580
0.881
0.460
0.636
0.677
1.006
0.565
0.848
0.644
1.074
30
0.434
0.568
0.756
1.203
0.543
1.129
0.730
1.340
0.559
0.862
0.804
1.368
0.586
0.933
0.625
1.060
50
0.508
0.747
0.795
1.272
0.586
1.345
0.742
1.381
0.559
0.926
0.773
1.216
0.592
0.951
0.612
0.987
100
0.478
0.714
0.652
0.831
0.625
1.527
0.710
1.363
0.617
1.083
0.720
1.223
0.611
1.008
0.610
0.966
300
0.712
1.225
0.842
1.567
0.768
1.729
0.811
1.658
0.812
1.476
0.773
1.365
0.817
1.519
0.828
1.527
Guangzhou
10
0.367
0.293
0.689
0.823
0.487
0.469
0.758
1.001
0.659
0.773
0.709
0.832
0.447
0.396
0.508
0.513
30
0.387
0.323
0.698
0.845
0.606
0.678
0.747
0.975
0.797
1.080
0.714
0.871
0.449
0.411
0.512
0.528
0.395
0.321
0.750
0.920
0.581
0.664
0.752
0.971
0.893
1.293
0.787
0.930
0.527
0.533
0.588
0.633
100
0.420
0.350
0.748
0.909
0.588
0.672
0.752
0.938
0.828
1.126
0.762
0.915
0.540
0.545
0.559
0.584
300
0.495
0.466
0.758
0.968
0.635
0.749
0.794
1.084
0.943
1.411
0.758
0.986
0.670
0.790
0.669
0. 0.794
10
0.213
0.099
0.662
0.622
0.294
0.160
0.714
0.741
0.461
0.389
0.820
0.908
0.485
0.495
0.649
0.808
PEMS04
30
0.263
0.132
0.779
0.833
0.457
0.356
0.760
0.820
0.685
0.828
0.813
0.908
0.523
0.668
0.648
0.901
50
0.267
0.142
0.827
0.897
0.553
0.505
0.762
0.813
0.711
0.865
0.854
0.965
0,531
0,611
0.531
0.532
0.425
0.328
0.888
1.037
0.706
0.767
0.867
1.020
0.872
1.126
0.876
1.015
0.612
0.680
0.627
0.658
300
0.569
0.513
0.883
1.025
0.731
0.785
0.899
1.097
0.891
1.167
0.923
1.096
0.776
0.876
0.800
0.906
by the length of consecutive missing data, exhibiting more stable performance across different lengths. It performs well under both low and high levels of consecutive missing data.
Furthermore, we observe that some models reach their worst performance even with very short lengths of consecutive missing data, whereas they perform much better under MCAR. This indicates that different models have varying levels of robustness to consecutive missing data. Some models experience a significant performance drop with only small amount of consecutive missing data, which poses substantial risk in real-world scenarios. However, our hod demonstrates excellent robustness to consecutive missing data lengths and can effectively handle large-scale missing data problems in real-world applications.
4.2.2 Varying missing rate
In addition to handling data with different lengths of consecutive missing values, the imputation model should also adapt to data with varying missing rates. Table P presents the experimental results under different. missing rates with a fixed missing length of 50. As shown, our method outperforms all basclines in every case. For instance, regarding the MSE metric, our method leads to an average improvement of 74.62%, 57.81%, 28.19%, 52.38%, 70.15% compare to other methods on Electricity, Traffic, METR-LA, Guangzhou, and PEMS04 datasets, respectively. This demonstrates that our model has strong robustness and can adapt to various types of missing data.
Additionally, we observe an anomalous phenomenon in some models where increasing the missing rate on certain datascts actually improves the results. This occurs because, at lower missing rates, the randomly consecutive missing values may be unevenly distributed, causing instability in the imputation models. Different random missing data can produce significantly varying results. If the missing intervals happen to concentrate
9

---

Published in Transactions on Machine Learning Research (02/2025)
Table 2: Imputation results on real datasets with different missing rates and a fixed 50 missing length. The
label F indicates imputation using forecasting methods.
LSTI
Transformer
CSDI
SAITS
TimesNet
Imputeformer
TimesNet
F
iTransformer
Rm
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
Electricity
10%
0.224
0.108
0.775
0.896
0.653
0.765
0.769
0.907
0.607
0.687
0.752
0.860
0.290
0.181
0.286
0.192
30%
0.255
0.144
0.806
0.961
0.658
0.766
0.809
0.982
0.686
0.844
0.793
0.920
0.375
0.296
0.395
0.339
50%
0.269
0.157
0.802
0.971
0.770
0.980
0.809
0.987
0.698
0.876
0.795
0.932
0.478
0.431
0.524
0.504
0.321 70%
0.209
0.863
1.121
0.766
0.979
0.821
1.019
0.733
0.945
0.803
0.940
0.578
0.572
0.637
0.668
10%
0.265
0.308
0.781
1.246
0.601
0.896
0.784
1.238
0.707
1.078
0.755
1.160
0.323
0.392
0.288
0.343
METR-LA Traffic
30%
0.301
0.341
0.762
1.164
0.555
0.871
0.769
1.166
0.697
1.056
0.756
1.170
0.389
0,478
0.373
0.454
50%
0.292
0.322
0.761
1.163
0.610
0.913
0.773
1.181
0.782
1.255
0.753
1.119
0.514
0.689
0.541
0.745
70%
0.366
0.428
0.766
1.166
0.619
0.961
0.767
1.156
0.826
1.397
0.773
1.244
0.604
0.833
0.629
0.877
10%
0.478
0.710
0.770
1.099
0.509
0.938
0.704
1.266
0.594
0.988
0.690
1.003
0.638
1.249
0.577
0.928
30%
0.508
0.747
0.795
1.272
0.606
1.345
0.742
1.381
0.559
0.926
0.804
1.382
0.592
0.951
0.612
0.987
50%
0.568
0.867
0.744
1.210
0.626
1.375
0.727
1.351
0.589
1.048
0.788
1.344
0.674
1.150
0.672
1.093
70%
0.621
1.025
0.764
1.301
0.653
1.396
0.754
1.450
0.629
1.156
0.799
1.386
0.716
1.219
0.725
1.188
Guangzhou
10%
0.410
0.355
0.766
1.024
0.557
0.607
0.800
1.107
0.856
1.246
0.712
0.867
0.443
0.399
0.481
0.491
30%
0.395
0.321
0.750
0.920
0.581
0.664
0.752
0.971
0.893
1.293
0.787
0.971
0.527
0.533
0.588
0.633
50%
0.428
0.368
0.733
0.904
0.563
0.631
0.766
1.007
0.888
1.306
0.757
0.950
0.595
0.639
0.637
0.728
70%
0.491
0.457
0.747
0.928
0.631
0.736
0.775
0.990
0.860
1.238
0.779
0.995
0.692
0.813
0.713
0.847
PEMS04
10%
0.232
0.109
0.819
0.915
0.447
0.319
0.780
0.834
0.728
0.861
0.854
0.965
0.281
0.160
0.366
0.279
30%
0.267
0.142
0.827
0.897
0.553
0.505
0.762
0.813
0.711
0.865
0.845
0.969
0.531
0.611
0.531
0.532
50%
0.349
0.235
0.812
0.901
0.622
0.628
0.799
0.895
0.719
0.921
0.863
0.994
0.743
1.040
0.737
0.940
70%
0.478
0.388
0.821
0.926
0.639
0.630
0.870
1.045
0.776
0.998
0.956
1.133
0.809
1.056
0.893
1.264
100
825
META
PEM504
0.0
Transformer SAITS TimesNot CSDI
Average (%)
71.5
Average (%)
63.5
43.2
0.6
60
MAE
40
Ours
$1.5
0.4
22:3
0.3
100
200
80
100 200
300
50
80
100
200
300
Lg
Lm
Lm
(a) MAE on Electricity.
(b) À on META LA and PEMS04.
Figure& 4: Results of MAE and average À with varying lengths of consecutive missing data.
within the model's imputation window, the model has to impute at higher missing rate within that interval, leading to unexpectedly poor results.
4.3 Analysis X
We Lest the output À of the Mela-weighting module on two datasets and record the average values of all X during testing. The relevant results are shown in Figure 4b As can be seen, À generally decreases as Lm increases, indicating that the Short-Term Imputer more dominant datascts with shorter Im, whercas the Long-Term Imputer is more dominant in datasets with longer L,. This is because shorter consecutive missing data can more easily capture information near the missing values, making short-term dependencies
10

---

Published in Transactions on Machine Learning Research (02/2025)
more effective for imputation. Conversely, longer consecutive missing data can only obtain useful information from distant points, necessitating the use of long-term dependencies for better imputation. This demonstrates that our Meta-weighting module effectively learns the importance of long-term and short-term dependencies in different missing datasets and can adaptively adjust the weights based on the characteristics of the data.
4.4 Analysis data missing type
In real-world scenarios, various missing data patterns can occur. The straightforward Blackout missing pattern fails to fully capture real-world scenarios. In this section, we explore and conduct experiments under various missing patterns to assess the adaptability and effectiveness of LSTI across different settings.
4.4.1 Analysis MCAR
Missing Completely At Random (MCAR) is widely adopted missing data pattern in time-series imputation. It describes a scenario where the occurrence of missing data is independent of the data itself or any other variables. Under the MCAR pattern, missing data points are typically distributed uniformly across the dataset. Figure [ illustrates the experimental results of various methods on the PEMS04 dataset under the MCAR missing pattern, with all models trained using the same MCAR configuration. As shown, despite not being specifically designed for MCAR, LSTI consistently achieves superior performance across different. missing rales. This highlights the robustness and versatility of LSTI.
4.4.2 Analysis other consecutive missing types
To evaluate the performance of LSTI under different consecutive missing patterns, we follow the scheme in Khayati et al. (2020), which categorizes these patterns into four types: 'Disjoint,' 'Overlap,' 'MCAR_BB B,' and 'Blakkou."" Specilically, the 'Disjoint' pattern requires the missing blocks to have litule overlap as possible, while the 'Overlap' pattern demands the missing blocks to have as much overlap as possible. The 'MCAR B' pattern involves randomly selecting missing blocks for each channel independently, whereas the 'Blackout' pattern randomly selects identical missing blocks across all channels. All experiments were conducted with mm= and =,.=30% settings.
The experimental results are shown in Figure 6 It can be seen that our LSTI outperforms TimesNet across all datasets and missing patterns, demonstrating that our method has exceptional adaptability to various continuous missing patterns and exhibits high robustness, showing minimal sensitivity to the specific missing pattern. The figure also reveals that while TimesNet's performance remains consistent across the Disjoint, Overlap, and MCAR B missing patterns, it significantly deteriorates under the Blackout missing pattern. This indicates that traditional imputation methods are less adaptable to the Blackout missing pattern, where missing values span all channels, making inference from other channels infeasible. In contrast, our LSTI
10% missing rate
missing rate
missing rate
SAITS
Imputeoormw TimesNet
SAITS
Impuleforme Timeshet
SATS Transforar CSDI
Ingudtformee
0.10
Transformer
0.60
Transfommrr
0,10
Timestet
CSDI
LSM
CSDI
LSTI
LSTI
0.35
0.305
0.30
0.38
0.30
0.5
0.235
0.23
0.25
M00
0.175
0.179
0.156
0.153 0,152
0.180
0.101
0.159
0.157
0.168 0.166
0.15
0:35
0.15
0.1
0.10
0.6
0.00
Mothod
2.00
Method
0.00
Method
(a) 10% missing rate
(b) 30% missing rate
(c) 50% missing rate
Figure 5: Experimental results of various methods the PEMS04 dataset under the MCAR missing type.
11

---

Published in Transactions on Machine Learning Research (02/2025)
TimesNet
TimesNet
0.8
LSTI
0.77
0.8
LSTI
0.79
0.67
0.69
0.6
0.58
0.6
0.53
MSE
0.44
MSE
0.30
0.4
0.33
0.35
0.25
0.26
0.2
0.17
0.2
0.16
0.18
0.0
0.0
Electricity
Traffic
METR-LA Guangzhou
PEMS04
Electricity
Traffic
METR-LA Guangzhou
PEMS04
(a) Disjoint
(b) Overlap
TimesNet
1.4
TimesNet
0.78
1.29
LSTI
LSTI
0.71
1.2
0.65
1.06
0.6
1.0
0.93
0.86
MSE
MSE
0.75
0.4
0.2
0.34
0.32
0.2
0.0
0.0
Electricity
METR-LA Guangzhou PEMS04
Electricity
Traffic
Guangzhou
(c) MCAR B
(d) Blackout
Figure 6: Comparison of MSE between TimesNet and LSTI across different datasets and missing patterns. All experiments were conducted with -5 and RS, =30% settings.
Table 3: Ablation study results of different LSTI modules, with Lm values of [10, 30, 50, 100, 300), and all results averaged these five different Im values.
META-LA
PEMS04
IT.
bl.
T
S
L.S
MAE
MSE
MAE
MSE
0.794
1.423
0.437
0.376
0.692
1.362
0.413
0.367
0.611
0.973
0.380
0.324
0.558
0.963
0.476
0.435
/
0.495
0.721
0.347
0.243
model effectively imputes data even under the challenging Blackout pattern, addressing a key limitation of traditional imputation approaches.
4.5 Ablation study
To investigate the impact of different LSTI modules on imputation performance, we conduct an ablation study on LSTI with the experimental results presented in Table 3 The abbreviations fL, bL, L, S, and LS represent the following configurations: only the forward prediction network, only the backward prediction network, only the Long- Term Imputer, only the Short-Term Imputer, and the complete LSTI model, respectively. We find that in all cases, the Long-Tern Term Imputer outperformed both the forward and backward prediction networks
12

---

Published in Transactions on Machine Learning Research (02/2025)
when used individually. This suggests that the strategy of weighted fusion of predictions made before and after the missing interval is both practical and highly effective.
Furthermore, while the Short-Term Imputer generally performed worse than the Long-Term Imputer on most datasets, their combined results achieved optimal performance. This is attributed to the fact that the Short-Term Imputer performs better when I.m small, but performance significantly declines as Lm increases. However, due to the presence of the weighteigg module, the negative impact of the Short-Term Imputer is minimized, resulting in the final averaged outcome being optimal. This demonstrates the importance of integrating both short-term and long-tern term components in the LSTI model for achieving robust imputation outcomes.
5 Conclusion
Long-interval consecutive missing data is a common issue in real-world time series datasets, which traditional deep learning imputation methods struggle to address effectively. To tackle this, we propose the Long Short- Term Imputer (LSTI), consisting of a Long-Term Imputer, Short-Term Imputer, and a Meta-weightiig module. This model can capture both long-term and short-term dependencies in time series and performs weighted merging based data characteristics. Experiments on five real-world datasets demonstrate that our method can effectively impute long-intenn interiatrval consecutive missing data.
Acknowledgements
This work was supported by the Shenzhen Science and Technology Program (20S15202302209120000)) National Natural Science Foundation of China (62306085, 62302241, 62476071, 62236003), Shenzhen College Stability Support Plan (GXWD2023113015122900 GXWDD020081114428005)
References
Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models. arxiv preprint or arXiv:2208.0040 09399, 2022.
Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. Brits: Bidirectional recurrent imputation for time series. Advances in neural information processing systems, 31, 2018.
Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1):6085, 2018.
Chao Chen, Karl Petty, Alexander Skabardonis, Pravin Varaiya, and Zhanfeng Jia. Freeway performance measurement system: mining loop detector data. Transportation research record, 1748(1):96 102, 2001.
Xinyu Chen, Yixian Chen, and Zhaocheng He. Urban traflic speed dataset of guangzhou, china. March 22, 2018. Distributed by Zenodo., 2018
Zeng Chen, Huan Xu, Peng Jiang, Shanen Yu, Guang Lin, Igor Bychkov, Alexey Hmelnov, Gennady Ruzhnikov, Ning Zhu, and Zhen Liu. A transfer barning-based lstm strategy for imputing large-scale consecutive missing data and its application in water quality prediction system. Journal of Hydrology, 602:126573, 2021.
Wenjie Du, David Côté, and Yan Lin. Saits: Selfattention-baset imputation for time series. Expert Systems with Applications, 219:119619, 2023.
Xiaolong Guo, Shijia Zhu, Zhiwei Yang, Hao Liu, and Tianshu Bi. Consecutive missing data recovery method based on long-short term momory network. In 2021 3rd Asia Energy and Electrical Engineering Symposium (AEEES), pp. 988 992. IEEE, 2021.
Ashish Gupta, Hari Prabhat Gupta, Bhaskar Biswas, and Tanima Dutta. An carly classification approach for multivariate time series of on-vehicle sensors in transportation. IEEE Transactions on Intelligent Transportation Systems, 21(1)):5316 5322557777,, 2020.
13

---

Published in Transactions on Machine Learning Research (02/2025)
Zahra Hajihashemi and Mihail Popeseu. A multidimensionaal time-series similarity measure with applications to eldercare monitoring. IEEE Journal of biomedical and health informalics, 20(3):953 0020399622220 2015.
Md Kamrul Hasan, Md Ashraful Alam, Shidhartho Roy, Aishwariya Dutta, Md Tasnim Jawad, and Sunanda Das. Missing value imputation affects the performance machine learning: A review and analysis of the literature (2010 2021). Informaties in Mediccine Unlocked, 27:100799, 2021.
Mourad Khayati, Alberto Lerner, Zakhar Tymchenko, and Philippe MaurruM Mind the gap. Proceedings of the VLDB Endowment, 13:768-788 2020.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 4 1st international ACM SIGIR conference on research & development in information relrieval, pp. 95 104, 2018.
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting, Advances in Neural Information Processing Systems, 35:9881 1-998, 2022.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint W:Xi:2310006625, 2023.
Yukai Liu, Rose Yu,, Stephan Zheng, Fric Zhan, and Yisong Yue. Naomi: Non-autorrgressive multiresolution sequence imputation. Advances in neural information processing systems, 32, 2019.
Yonghong Luo, Ying Zhang, Xiangrui Cai, and Xiaojic Yuan. E2gan: End-to-end generative adversarial network for multivariate time series imputation. ion. Proceedings of the 28th international joinl conference on artificial intelligence, pp. 3094 0094-3100, AAAI Press Palo Alto, CA, USA, 2019.
Jun Ma, Jack CP Cheng, Yuexiong Ding, Changqing Lin, Feifeng Jiang, Mingzhu Wang, and Chong Zhai. Transfer learning for long-interval consecutive missing values imputation without external features in air pollution time series. Advanced Engineering Informatics, 44:101092, 2020.
Metro.
Metr-la dataset.
Etienne Le Naour, Louis Serrano, Léon Migus, Yuan Yin, Ghislain Agoua, Nicolas Baskiotis, Patrick Gallinari, and Vincent Guigue. Time series continuous modeling for imputation and forecasting with implicit neural representations. arXio preprinl Tfi::2306008888 2023.
Tong Nie, Guoyang Qin, Wei Ma, Yuewen Mei, and Jian Sun. Imputeformer: Low rankness-induced transformers for generalizable spatiotemporal imputation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2260 2271, 2024.
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations, 2022.
PeMS. Traffic dataset. http://pems... wwtppesss.ott...m cot.ma.ooocca.ooo/ go.
Adityo Prakosa, Maxime Sermesant, Hervé Delingette, Stéphanic Marchesscau, Eric Saloux, Pascal Allain, Nicolas Villain, and Nicholas Ayache. Generation of synthetic but visually realistic time series of cardiac images combining a biophysical model and clinical images. IEEE transactions on medical imaging, 32(1): 99 10-109,44 2012.
David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Jamuschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International journal of forecasting, 36(3):1181 1191, 2020.
Siyuan Shan, Yang Li, and Junier B Oliva. Nrtsi: Non-recureent time series imputation. In ICASSP 2023-2023 IEFE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEFE, 2023.
14

---

Published in Transactions on Machine Learning Research (02/2025)
Ikaro Silva, George Moody, Daniel J Scott, Leo A Celi, and Roger G Mark. Predicting in-hospital mortality in cardiology challenge 2012. In 2012 Compuling in Cardiology-
of icu patients: The pp. 245 248. IEEE, 2012.
Akin Tascikaraoglı, Borhan M Sanandaji, Gianfranco Chicco, Valeria Cocina, Filippo Spertino, Ozan Erdine, Nikolaos G Paterakis, and João PS Catalão. Compressive spatio-temporal forecasting of metcorological quantities and photovoltaic power. IEEE Transaclions on Suslainable Eneryy, ((3):1295-1305) 2016.
Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-basod diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34: 24804-24816, 2021.
UCI
Electricity dataset.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Wetterstation. Weather dataset. https: htttps:/www.bgee.. map.ea..ppp........m www.dde/eeett..
Chantha Wongoutong. Imputation for consecutive missing values non-stationary time series data. Advances and Applications in Statistics, 64(1):87-102,, 2020.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34: 22419 22430, 2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv preprint orXiv:2210.02186, 2022.
Xiuwen Yi, Yu Zheng, Junbo Zhang, and Tianrui Li. St-mvl: filling missing values in geo-sensory time series data. In Proceedings of the 25th International Joint Conference Artificial Intelligence, 2016.
Jinsung Yoon, William R Zame, and Mihaela van der Schaar. Estimating missing data in temporal data streams using mirtt-direetional recurrent neural networks. IEEE Tronsactions on Biomedical Engineering, 6675):1477,,490, 2018.
Yu Zheng, Xiuwen Yi, Ming Li, Ruiyuan Li, Zhangqing Shan, Eric Chang, and Tianrui Li. Forecasting fine-grained air quality based on big data. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 2267-2276, 2015.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11106-11115, 2021.
15

---

Published in Transactions on Machine Learning Research (02/2025)
A Variability and Confidence Estimation
A.1 Standard deviation
The standard deviation (8) is a key statistical metric used to quantify the dispersion or spread of a data set. It measures the extent to which individual data points deviate from the mean, offering valuable insight into the variability or consistency of the data. A larger standard deviation indicates greater variability, with data points more widely dispersed from the mean, whereas a smaller standard deviation signifies less variability, suggesting that the data points are more tightly clustered around the mean.
1 11
I
Tis
(2) x)²
(13)
1=1
In our experiment, each result is the average of three repeated trials, i.e., n = 3. Table standard deviations for all experimental results. It can be observed that, in the majority of experimental settings, LSTI exhibits a relatively small standard deviation, which further validates the consistency and stability of our findings.
4 and 5 present the
Table 4: Standard deviation on real datasets with different missing lengths and a fixed 30% missing rate.
The label " F" indicates models using forecasting methods.
LSTTI
CSDI
TemesNet
SAITS
Transformer
NetesNNt
F
|Transformer F
Lm
MAE|
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
Electricity
10
0.008
0.011
0.004
0.145
0.031
0.059
0,005
0,007
0.018
0.038
0.017
0.032
0.007
0.018
30
0.006
0.010
0.021
0.028
0.038
0.072
0.009
0.025
0.023
0.057
0.001
0.002
0.011
0.009
50
0.003
0.007
0.023
0.033
0.032
0.082
0.011
0.025
0.014
0.035
0.002
0.010
0.023
0.026
100
0.022
0.028
0.027
0.047
0.025
0.037
0.014
0.016
0.013
0.036
0.026
0.029
0.029
0.021
300
0.025
0.025
0.014
0.045
0.027
0.137
0.002
0.019
0.012
0.032
0.033
0.056
0.033
0.056
10
0.005
0.016
0.003
0.011
0.015
0.018
0,012
0.040
0.014
0.041
0.021
0.060
0.017
0.044
Traffic
30
0.008
0.019
0.025
0.052
0.030
0.059
0.014
0.036
0.013
0.043
0.020
0.071
0.018
0.046
50
0.028
0.038
0.025
0.050
0.037
0.099
0.015
0.064
0.014
0.067
0.042
0.068
0.018
0.026
100
0.023
0.031
0.008
0.046
0.015
0.073
0.011
0.051
0.012
0.047
0.048
0.081
0.049
0.086
300
0.031
0.073
0.042
0.108
0.013
0.054
0.010
0.042
0.007
0.037
0.039
0.075
0.041
0.083
METR-LA
10
0.024
0.058
0.070
0.288
0.034
0.078
0.116
0.397
0.100
0.254
0.042
0.155
0.051
0.093
30
0.018
0.061
0.076
0.274
0.026
0.100
0.046
0.179
0.044
0.158
0.057
0.231
0.032
0.043
50
0.004
0.027
0.053
0.215
0.055
0.207
0.037
0.115
0.013
0.075
0.023
0.115
0.044
0.181
100
0.1
0.352
0.122
0.450
0.050
0.222
0.092
0.261
0.087
0.297
0.027
0.100
0.046
0.217
300
0.131
0.280
0.188
0.790
0.102
0.370
0.193
0.657
0.179
0.642
0.182
0.602
0.171
0.591
Guangzhou
0.006
0.011
0.065
0.078
0.030
0.062
0.018
0.052
0.034
0.061
0.023
0.032
0.014
0.017
30
0.050
0.082
0,086
0.121
0.019
0.075
0.021
0.092
0.021
0.064
0.026
0.049
0.007
0.026
50
0.007
0.028
0.031
0.074
0.082
0.168
0.025
0.073
0.021
0.041
0.023
0.038
0.037
0.029
100
0.053
0.055
0.032
0.063
0.008
0.014
0.013
0.032
0.010
0.040
0.077
0.121
0.069
0.110
300
0.009
0.023
0.036
0.026
0.106
0.294
0.033
0.084
0.019
0.047
0.014
0.012
0.016
0.009
PEMS04
0.006
0.011
0.024
0.025
0.005
0.006
0.016
0.027
0.008
0.012
0.118
0.238
0.028
0.108
30
0.023
0.015
0.052
0.082
0.029
0.053
0.051
0.085
0.073
0.117
0.056
0.223
0.032
0.205
50
0.027
0.037
0.037
0.076
0.024
0.072
0.027
0.050
0.026
0.044
0.029
0.094
0.028
0.035
100
0.061
0.098
0.049
0.108
0.035
0.054
0.047
0.084
0.040
0.070
0.124
0.236
0.118
0.181
300
0.080
0.8
0.042
0.1
0.027
0.1
0.018
0.065
0.011
0.018
0.039
0.059
0.026
0.037
16

---

Published in Transactions on Machine Learning Research (02/2025)
Table 5: Standard deviation on real datasets with different missing rates and a fixed 50 missing length. The
label "_ indicates imputation using forecasting methods.
LSTI
CSDI
TimesNet
SAITS
Transformer
TimcsNet
F
iTrranformmrr F
Rm
MAE
MSE
MAE
MSE
MMAE
MSE
MAE
MSE
MAE
MSE
MAE
MSL
MAE
MSE
Electricity
10%
0.003
0.003
0.092
0.158
0.030
0.055
0.023
0.052
0.034
0.079
0.014
0.017
0.012
0.015
30%
0.003
0.007
0.038
0.095
0.032
0.082
0.011
0.025
0.014
0.035
0.002
0.010
0.023
0.026
50%
0.007
0.011
0.044
0.101
0.016
0.049
0.008
0.012
0.031
0.053
0.021
0.036
0.006
0.015
70%
0.018
0.023
0.035
0.070
0.018
0.033
0.007
0.016
0.023
0.046
0.020
0.038
0.015
0.027
10%
0.014
0.009
0.084
0.179
0.049
0.115
0.021
0.049
0.020
0.061
0.033
0.051
0.052
0.074
Guangerhou METR-LA Traffic
30%
0.028
0.038
0.025
0,050
0.037
0.099
0.015
0.064
0.014
0.067
0.042
0.068
0.018
0.026
50%
0.012
0.018
0.075
0.105
0.026
0.088
0.011
0.051
0.016
0.055
0.070
0.162
0.025
0.060
70%
0.029
0.046
0.065
0.109
0.018
0.029
0.007
0.014
0.014
0.021
0.023
0.046
0.027
0.050
10%
0.164
0.393
0.071
0.246
0.171
0.534
0.191
0.653
0.171
0.525
0.238
0.767
0.144
0.469
30%
0.004
0.027
0.053
0.215
0.055
0.207
0.037
0.115
0.013
0.075
0.023
0.115
0.044
0.181
50%
0.050
0.164
0.046
0.181
0.034
0.105
0.037
0.174
0.014
0.101
0.029
0.070
0.048
0.168
70%
0.034
0.134
0.0-1
0.178
0.041
0.096
0.003
0.020
0.016
0.024
0.046
0.145
0.027
0.039
10%
0.020
0.048
0.090
0.148
0.039
0.089
0.048
0.133
0.048
0.110
0.013
0.041
0.054
0.134
30%
0.007
0.028
0.031
0.074
0.082
0.168
0.025
0.073
0.021
0.041
0.023
0.038
0.037
0.029
50%
0.019
0.026
0.040
0.075
0.024
0.043
0.022
0,053
0.007
0.024
0.029
0.048
0.015
0.041
70%
0.008
0.002
0.016
0.053
0.036
0.114
0.006
0.005
0.011
0.017
0.039
0.091
0.022
0.038
PEMS04
10%
0.056
0.042
0.046
0.053
0.043
0,094
0.032
0,049
0.046
0,093
0.037
0.024
0.027
0,040
30%
0.027
0.037
0.037
0.076
0.024
0.072
0.027
0,050
0.026
0.044
0.029
0.094
0.028
0.035
50%
0.046
0.072
0.065
0.103
0.047
0.136
0.032
0,056
0.033
0.049
0.055
0.280
0.036
0.121
70%
0.008
0.013
0.067
0.102
0.035
0.055
0.049
0.135
0.007
0.008
0.072
0.227
0.039
0.138
Table 6: The 95% confidence intervals of LSTT under a fixed 30% missing rate across all datasets. Here, [x,y] indicates the confidence interval, where x is the lower bound and y is the upper bound.
Electricity
Traffic
Metr-la
GuangZhou
PEMS04
Lm
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
10
10:24,
0.25]
00113, 0.14]
[0:29, 0.35|
[0.30, 0.43)
[0.40, 0.47]
[0.46, 0.65]
[0:39, 0.42)
[0.32, 0.37]
[0:23, 0.26]
00.12, 0.15]
30
J0:24,
0.24|
[0.12, 0.13]
|0.28, 0.47]
[0.32, 0.58)
[0.45, 0.54]
[0.57, 0.77]
[0.35, 0.43]
[0.27, 0.38]
[0.23, 0.27]
[0.10, 0.19]
50|
0.26] |0.25,
[0.14, 0.16|
[0.30,
0.36]
[0.32, 0.44
[0.47, 0.59
[0.66, 0.93]
[0.38, 0.43]
[0.29, 0.39]
[0.23, 0.31]
00.10, 0.22
100
0.25,
0.30]
00.14, 0.20
00.34, 0.40]
[0.38, 0.47
[0.40, 0.67
[0338, 1.27
[0.39, 0.49]
[0.33, 0.43]
[0.28, 0.46]
00.15 0.39
300
0.377,
0.43
|0.25, 0.32|
|0.45, 0.56]
[0.52, 0.77]
[0.63, 0.89]
[1002, 1.53]
[0.50, 0.52]
[0.47, 0.49]
[0.47, 0.56]
[0.34, 0.50]
A.2 Confidence interval
A confidence interval (CT) a statistical tool used to estimate the range within which a true population parameter is likely to lie, based on sample data. It quantifies the uncertainty of the estimate, providing both an estimate of the population mean and an indication of the precision of that estimate. A 95% confidence interval, for example, means that if the same experiment were repeated multiple times, approximately 95% of the calculated confidence intervals would contain the true population mean, offering a mcasure of the reliability of the estimete. The key benefits of calculating a CI include quantifying uncertainly, enabling more informed decision Docision-making, and assessing the statistical significance of experimental results. The confidence interval for the sample mean is calculated using the formula:
C1 =
x
2-
(14)
17

---

Published in Transactions on Machine Learning Research (02/2025)
Table The 95% confidence intervals of LSTI under a fixed 50 missing length across all datasets. Here, [x,y] indicates the confidence interval, where x is the lower bound and y is the upper bound.
Electricity
Traffic
Mctr-la
GuangZhou
PEMS04
Rm
MAB
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MAE
MSE
0.1
[0.22,
0.23]
[0.10, 0.11]
[0.25, 0.28]
[0.30, 0.32
[0.29, 0.66]
[0.27, 1.16
[0.39, 0.43]
[0.30, 0.41]
[0.17. 0.30]
[0.06, 0.16)
0.3
[0.25,
0.26
(0.14, 0.15
[0.27, 0.33
[0.30, 0.38
[0.50, 0.51
(0.72, 0.78
[0.39, 0.40]
[0.29, 0.35
[0.24, 0.30
[0.10, 0.18)
0.5
[0.26,
0.28
00.14. 0.17]
[0.28. 0.31]
[0.30, 0.34|
[0.51, 0.63
[0.68, 1.05]
[0.41, 0.45)
[0.34, 0.40)
[0.30, 0.40)
00155, 0.32
0.7
[0.30, 0.34]
[0.18, 0.24]
[0.33, 0.40]
[0.38, 0.48)
[0.58, 0.66]
[0.87, 1.18]
[0.48, 0.50]
[0.45, 0.46]
[0.47, 0.49)
[0.37, 0.40)
where æ represents the sample mean, s is the sample standard deviation, n denotes the sample size (n in our experiment), and is the critical value corresponding the desired confidence level (1.96 for n = 3 and 95% CT). Table and illustrate the 95% confidence intervals for LSTI across all datasets, providing comprehensive view of its performance variability.
A.3 Cumulative Average
To examine how many repetitions are needed for the average result converge, we conducted 30 repeated experiments on the GuangZhou dataset with Lm 50 and R,, 30%. The results shown in Figure 7 where the x-axis represents the number of repeated experiments and the y-axis represents the average MAE of the first x experiments. The average MAE across all 30 experiments is shown as a dashed line. As observed, the average value converges to the overall mean after approximately the 5th experiment, with subsequent fluctuations remaining small around the mean. This demonstrates that our method exhibits strong stability.
Cumulative Average
0.390
Cumulative Average
Overall Average
0.385
Mean MAE
0.380
0.375
0.370
0
5
10
15
20
25
30
Number of Experiments
Figure 7: Repeated experiments conducted on the GuangZhou dataset with I. 50 and R,, = 30%, where the missing data mask identical for each experiment.
18

---

Published in Transactions on Machine Learning Research (02/2025)
B Ablation Study on Loss Function
In the Long-term Imputer, we utilize four distinct loss functions: forward prediction loss Ly; backward prediction loss Lo. consistency loss Le, and combined result loss Lp. They play an indispensable role in the training process. In the following, we provide detailed description of each loss function, along with the corresponding ablation study results.
B.1 Detailed explanation
The forward prediction loss C, and backward prediction loss Ls serve as the direct loss functions for FPNet and BPNet, respectively. Their primary distinction lies in the prediction direction of the networks. For instance, Ly calculates the loss for the prediction window generated by FPNet while simultaneouse compuling the reconstruction loss for the look-back window produced by FPNet. This design improves the stability of the Long-term Imputer during autoregressive processes and reduces the risk of significant error accumulation over successive iterations. These two loss functions directly govern the training of FPNet and BPNet, aiming optimize the values generated by both networks.
The consistency loss Le applied to both FPNet and BPNet. As shown in Figure 2 during training, the prediction windows generated by FPNet and BPNet correspond to the same segment of the time series. Ideally, their predictions should be identical. The consistency loss f. measures the difference between the prediction windows produced by FPNet and BPNet, with the goal of aligning their prediction spaces and ensuring consistent oulputs. This design allows the Long-term Imputer to produce stable results when imputing the same target segment, whether in the forward or backward direction, thereby mitigating the instability often associated with antoregressive processes.
The loss Lp is associated with the final combined result produced by the Long-term Imputer. This is the most crucial loss, it directly reflects the imputation capability of the Long-term Imputer. The individual performance of FPNet and BPNet does not necessarily translate to optimal imputation performance, as the prediction windows they generate must be combined using scalar weighting mechanism to produce the final imputed result. The role of Lp is to integrate this mechanism into the network, aiming to optimize the combined outputs of FPNet and BPNet and, in turn, directly enhance the imputation capability of the Long-term Imputer.
Table 8: Ablation study on different losses with fixed 50 missing length and 30% missing rate.
GuangZhou
Electricity
L
Lb
Le
Cp
MAE
MSE
MAE
MSE
/
/
/
/
0.395
0.323
0.272
0.154
/
/
0.426
0.357
0.340
0.278
/
/
/
0.427
0.373
0.347
0.234
/
/
/
0.418
0.355
0.292
0.181
/
0.423
0.375
0.293
0.190
/
0.440
0.382
0.296
0.184
B.2 Ablation study
We conducte ablation experiments different losses across various datasets. In these experiments, modify the training loss Li of the Long-term Imputer by combining different losses to investigate their individual contributions. The training processes of other modules, such as the Short-term Imputer and the meta-learning weighting module, remain unchanged.
The results of the experiments are shown in Table As observed, the combination of all four losses in Lr consistently produced the best performance across all datasets. This demonstrates that each of the four losses plays crucial and complementary role in the training process.
19

---

Published in Transactions on Machine Learning Research (02/2025)
C Details of the Short-term Imputer
The Short-term Imputer is a crucial component of LSTI, designed to capture short-term dependencies in time series data. It is particularly effective in datasets with densely distributed observation points. While the main text describes the Short-term Imputer as a self-mapping network (SMNet); this section offers a more detailed explanation, covering both its training and inference processes.
C.1 The training process of Short-term Imputer
At cach training step, a data segment X E R(S-L+5)xC is obtained and divided into three parts: X,r X,, N.]. The Short-term Imputer trained exchusively on the middle segment, X₃ E RTxC_ the same time, we obtain the corresponding mask matrix M, for X,, We first normalize the valid values each channel of X,, and set all invalid values to O. The processing formula is as follows:
Me El= (mi,e [t.e)
o? E's (21,c - Mc)² miee
Tt.
(Ii,e po) m.,
(15)
El=1 Mi,e
El=1 m4,c
de
The normalization method described above excludes zero-filled missing values, using only valid observations to compute the mean and standard deviation. This reduces the negative impact of zero values, cnsuring that the computed isaiis are more representative of the ground truth distribution and improving the accuracy and reliability of the imputation process.
The normalized matrix X₁ transformed into feature tokens via a linear fully connected layer and then summed with fully trainable positional encoding matrix to preserve the temporal order of the time series. The detailed data encoding process is described follows:
Xenc = X, W enc + Bene + Epos.
(16)
where W encs Boney and Epss are learnable parameters. The projection sequence Xene constitutes the input sequence of the SMNet. We then feed Xenc into the SMNet to obtain the imputed result Z, produced by the network. Subsequently, Ze is de-normalized using the previously obtained mean and standard deviation o to obtain the final result. The Short-term Imputer loss L, is then computed based on this final output.
2, = SMene, Z: = 11=Z.-o0 - -. MSE((1 - M,) X, (1.1MM)) Z.).
(17)
The SMNet can be replaced with any imputation network. In our experiments, we use TimesNet as the SMNet.
C.2
The inference process of Short-term Imputer
During the inference process, we partition the entire dataset using a sliding window of length L and a sliding step of L. Each window is input as X, into the Short-term Imputer, which outputs the imputed result Z, for the current. window. After imputing all the windows, obtain the final imputed result î from the Short-term Imputer.
20

---

Published in Transactions on Machine Learning Research (02/2025)
D Details of the Meta-Learning Module
The Mcta-Learning Module (referred to as the Gate) is pivotal component of our system, tasked with learning the characteristics of the missing data to adaptively allocate weights to the Long-term Imputer and Short-term Imputer outputs. This section offers comprehensive explanation of the module's architecture, highlighting the design rationale alongside its training and inference processes.
D.1 The training process
At each training step, a data segment X R(S-L+8)xC is obtained and divided into three parts: X Xe, Xz]. Similar to the Short-term Imputer, the Gate's training process also utilizes only the middle segment, X, E RRLXC_ Upon obtaining X, and M,, these matrices are concatenated to form composite input matrix, Xinp E R2LXC This input is then passed through linear fully connected layer, augmented by the addition of fully trainable positional encoding matrix. Formally, this process is expressed as:
Xip [X,, Mr],
Xene Xinp W enc + B + E E'pas
(18)
Following this, the encoded data Xeno is processed through one-dimensional convolutional layer, which effectively extracts local temporal patterns while addressing boundary effects to ensure robust feature representation across the sequence. The convolution operalion can be formally expressed as:
X Xcone = Conv1D(X (no)
(19)
The processed data Xoon is then passed through a series of hidden linear layers, enabling the model to learn complex relationships and interactions within the input features. Finally, a Softmax function applied map the embeddings to values between 0 and representing the adaptive mixing ratio. The overall operation can be expressed as:
À = Softmax (MMLP(Xcoov)))
(20)
where À represents the mixing ratio, it is used to combine the outputs of the Long-term Imputer, Y,, and the Short-term Imputer, Ze. The meta-learning module is subsequently updated through backpropagation using the loss function L..
D.2 The inference process
During inference, the Short-term Imputer partitions the data into windows. The corresponding data and missing mask for each window are then provided as input to the Gate, which computes the mixing ratio for that window. Once the inferences from both the Long-term and Short-term Imputers are complete, the mixing ratio is applied to combine their results, yielding the final imputed output.
E
Experimental Setup and Analysis
E.1 Dataset descriptions
Table summaarizes the details of all datasets used in our experiments. To ensure consistency and comparability, we adopted the data processing methodology from TimesNet. Each dataset was partitioned into training, validation, and test sets in a 7:1:2 ratio. This division allows for robust model evaluation and minimizes potential biases. The final performance metrics were computed on the test set, which serves as an unseen dataset for assessing the generalization capabilities of the model.
21

---

Published in Transactions on Machine Learning Research (02/2025)
Table Details the Datasets.
dataset
Electricity
Traffic
Metr-la
GuangZhou
PEMS04
channel 321
862
207
214
307
time step
26304
17544
34272
8784
16992
E.2
Showcase results
Figure iilustrates the imputation performance of our LSTI, compared with TimesNet and Transformer on the Electricity dataset under the Blackout setting with Im 50 and R. 30%. In this case, the data is missing from the 26th point to the end of the window. Notably, under our global Blackout missing setting, result in overlapping intervals, leading to extended missing
the random selection of missing segments segments. This overlap explains why the missing segment in this case exceeds the predefined length of 50.
As illustrated in the figure, the Transformer model performs poorly when dealing with continuous missing data, with imputation results approximating simple mean imputation. TimesNet demonstrates better performance, with some reconstructed values aligning closely with the ground truth. However, it struggles capture sharp transitions and maintain consistency in highly dynamic regions. In contrast, LSTI addresses these challenges effectively, reconstructing missing segments with high accuracy while preserving short-term variability and long-term temporal dependencies. This leads to imputation results that closely align with the ground truth, underscoring the robustness and adaptability of LSTI, positioning it as a highly effective solution for time series data imputation, particularly scenarios involving continuons data loss.
LSTI
TimesNet
Transformer
Imputation GroundTruth
Imputation
Imputation
Ground/Truth
GroundTrruth
-1
-2
60
(a) LSTI
(b) TimesNet
(c) Transformer
Figure 8: Visualization of imputation results on the Electricity dataset under the Blackout setting with Lm = 50 and Rm = 30%. The black line represents the ground truth, while the orange line indicates the imputed values. The missing segment begins at the 26th point the sequence.
E.3 Analysis of computational resource
The experiments in this study were conducted on high-performance computing system with the following specifications: eight NVIDIA GeForce RTX 4090 GPUs, each equipped with 24GB of VRAM, enabling efficient execution of deep learning and high-performance computational tasks. The system is powered by a 128-core AMD EPYC 7513 processor, which provides substantial parallel processing power. Additionally, the server is configured with 503GB of RAM, facilitating the handling of large datasets and memory-intensive operations.
We comprehensively evaluate the model efficiency across various datasets, assessing metrics such as memory consumption, overall runtime, and imputation performance. Figure g presents these results, where the x-axis
22

---

Published in Transactions on Machine Learning Research (02/2025)
Guangzhou
Electricity
1.0
Models
LSTI:
Transformer: 0.9G
Timeshet: 2.1G
SAITS: 2.76
Memory Footprint
0.8
ImputeFormer: 11.76
0.7
3G
6G
0.7
0.6
MSE
Memory Footprint
MSE
0.6
0.5
3G
0.4
0.3
LSTI: 5.3G
Transformer: 0.9G
0.3
CSDI: 18.8G
Imputeli mee:: 17.1G
0.2
10a
10
10
0.1 1z
10a
10"
10
Overall Time Cost (seconds)
Overall Time Cost (seconds)
PEMS04
METR-LA
Models
LSTI: 5:2G
Transtorme: 0.9G
TimesNet: 2.1G
SAITS: 3.6G
Imputeformer: 11.3G
1.0
0.6
Models
Transforme: 0.8G
TimesNet: 2.1G
CSDI:
0.4
SAITS: 3.4G
16.3G
0.6
Memory Footprint
3G
6G
0.0 10-
10a
10
10a
10a
10
Overall Time Cost (seconds)
Overall Time Cost (seconds)
Figure 9: A comprehensive comparison of model efficiency on the Guangzhou, Electricity, PEMS04, and METR-LA datascts. All results are obtainod under the scttings of Lm = 10 and Rm = 30%.
represents the overall runtime of each model, the y-axis indicates the imputation MSE, and the bubble size reflects the memory usage of each model. All experiments are conducted with TimesNet as the backbone, using settings of L,, 10 and R, 30%.
The results demonstrate that, although LSTT incurs higher computational overhead compared to some lightweight models, the significant improvement in accuracy more than offsets this cost. Additionally, for resourve-intensive models like CSDI, LSTI not only reduces computational demands but also delivers superior performance. By striking a balance between computational efficiency and tatte-of-theearr accuracy, LSTI establishes itsolf as robust and reliable solution for time series data imputation, particularly in addressing datasets with long consecutive missing segments.
23

---

Published in Transactions on Machine Learning Research (02/2025)
F
Additional experiments on more datasets.
To evaluate LSTI's performance across diverse domains and datasets with varying distributions, conducted additional experiments on two widely recognized benchmarks: Weather (Wetterstation) and ETTml (Zhou et al. 2021). The Weather dataset consists of 10-minute interval data from 2020, encompassing 21 meteorological indicators such as air temperature and humidity. The ETTml datasct provides 15-minute interval data, capturing load characteristics of seven oil and power transformers over a two-year period from July 2016 to July 2018. The results of these experiments are summarized in Tables [1 and 12 To ensure a fair and consistent comparison, all experimental settings align with those outlined in the main text.
Table 10: Details of the Dalasets.
dataset
Weather
ETTml
channel
21
7
time step
52696
69680
Table 11: Imputation results on real datasets with different missing lengths and fixed 30% missing rate. The label " F" indicates models using forecasting methods.
LSTI
Transformer
CSDI
SAITS
TimesNet
Imputeformer
TimesNet
Transformer
Tm
MAE
MSE
MAF
MSF
MAF
MSE
MAE
MSE
MAE
MSE
MAF
MSE
MAE
MSE
MAE
MSE
10
0.120
0.071
0.388
0.298
0.104
0.143
0.411
0.342
0.154
0.098
0.269
0.194
0.298
0.264
0.283
0.215
Weather
30
0.153
0.108
0.486
0.431
0.167
0.574
0.500
0.468
0.205
0.164
0.468
0.430
0.336
0.306
0.297
0.256
0.197
0.144
0.503
0.450
0.239
0.657
0.548
0.537
0.240
0.192
0.508
0.500
0.345
0.343
0.286
0.244
100
0.295
0.239
0.546
0.505
0.367
0.759
0.641
0.708
0.321
0.278
0.564
0.562
0.386
0.360
0.365
0.332
300
0.413
0.358
0.546
0.512
0.512
1.030
0.675
0.757
0.449
0.409
0.556
0.538
0.486
0.464
0.486
0.466
10
0.263
0.158
0.559
0.628
0.287
0.206
0.624
0.848
0.536
0.593
0.435
0.378
0.460
0.453
0.429
0.386
ETTml
30
0.362
0.293
0.718
0.889
0.346
0,673
0.704
1.000
0.721
1.025
0.646
0.726
0.492
0.518
0.464
0.472
0.430
0.418
0.716
0.900
0.637
0,832
0.716
1.023
0.718
1.051
0.721
0.898
0.495
0.534
0.471
0.483
100
0.509
0.547
0.694
0.855
0.747
1.073
0.695
0.930
0.774
1.114
0.699
0.864
0.546
0.623
0.513
0.551
300
0.578
0.632
0.769
1.023
0.976
1.609
0.777
1.123
0.733
1.124
0.778
1.045
0.705
0.917
0.687
0.880
Table 12: Imputation results on real datasets with different missing rates and fixed 50 missing length. The label F" indicates imputation using forecasting methods.
LSTI
Transformer
CSDI
SAITS
TimesNet
[Imputeformer
TimesNet F
iTransformer
Rm..
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
10%
0.183
0.134
0.512
0.492
0.243
0.274
0.443
0.402
0.220
0.175
0.483
0.465
0.256
0.217
0.235
0.177
Weather
30%
0.196
0.143
0.503
0.450
0.206
0.375
0.548
0.537
0.240
0.192
0.508
0.500
0.304
0.269
0.286
0.244
50%
0.213
0.154
0.509
0.460
0.288
0.715
0.600
0.622
0.259
0.224
0.540
0.536
0.430
0.441
0.395
0.358
70%
0.256
0.192
0.425
0.372
0.333
1.221
0.600
0.652
0.301
0.273
0.538
0.538
0.490
0.502
0.479
0.453
10%
0.428
0.409
0.677
0.822
0.550
0.668
0.677
0.931
0.718
1.051
0.640
0.749
0.434
0.424
0.395
0.363
ET'ml
30%
0.439
0.412
0.716
0.900
0.677
0.928
0.716
1.023
0.722
1.099
0.721
0.898
0.499
0.539
0.471
0.483
50%
0.454
0.424
0.687
0.899
0.693
0.985
0.737
1.095
0.761
1.129
0.733
0.954
0.556
0.638
0.541
0.606
70%
0.469
0.471
0.640
0.797
0.741
1.061
0.743
1.085
0.739
1.128
0.731
0.946
0.623
0.744
0.607
0.711
As illustrated in the tables, LSTI consistently outperforms bascline methods across the majority of experimental settings, with particularly notable gains under conditions of high missing rates and long consocutive missing segments. Specifically, LSTT achieves an average improvement of 49.25% on the Weather dataset. and 52.67%
24

---

Published in Transactions on Machine Learning Research (02/2025)
the ETTml dataset. These substantial enhancements underscore LSTI's strong imputation capabilities in scenarios characterized by challenging data gaps. Furthermore, the results highlight LSTT's remarkable generalizability across diverse datasets with varying characteristics, solidifying its position as a robust and effective solution for addressing data imputation challenges complex real-world applications.
G Baseline Comparison and Analysis: SSSD
In this section, analyze the SSSD method (Alcaraz & Strodthoff 2022). which has been reported to outperform baselines such CSDI and SAITS. Additionally, SSSD reported evaluation results under the Blackout missing pattern in its original paper. To further investigate its performance, we conducted additional experiments comparing LSTI with SSSD and examined why SSSD struggles to effectively address the global Blackout missing pattern configured in our experiments.
G.1 Comparison between LSTI and SSSD
To further evaluate the SSSD method, we conducted additional experiments following the experimental settings outlined in the main text. All hyperparameters for SSSD were set to their default values as specified in its original implementation. To ensure a fair and consistent comparison, we applied the standardized data preprocessing method used in this study to all datasets. It is worth noting that SSSD employs dataset- specific preprocessing strategies in its original experiments, which could potentially influence its performance outcomes.
Across all datasets and experimental configurations, SSSD consistently underperformed when compared to proposed LSTT method. This performance gap was particularly pronounced datasets such Traffic and Electricity, where SSSD exhibited significantly suboptimal results. These findings not only underscore the robustness and adaptability of LSTI but also highlight the challenges faced by SSSD in effectively handling the global Blackout missing pattern adopted in our experiments.
The experimental results are summarized in Tables and 14
Table 13: Comparison of imputation results between LSTI and SSSD across different datasets under fixed 30% missing rate.
Electricity
Traffic
Metr-la
GuangZhou
PEMS04
Lm
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
10
0.249
0.138
0.269
0.296
0.342
0.353
0.367
0.293
0.213
0.099
30
0.242
0.129
0.281
0.316
0.434
0.568
0.387
0.323
0.263
0.132
LSTI
50
0.255
0.144
0.301
0.341
0.508
0.747
0.395
0.321
0.267
0.142
100
0.278
0.171
0.334
0.381
0.478
0.714
0.420
0.350
0.425
0.328
300
0.370
0.251
0.450
0.547
0.712
1.225
0.495
0.466
0.569
0.513
10
1.308
3.667
1.450
4.435
1.046
2.635
0.739
1.290
1.132
2.734
30
1.309
3.672
1.455
4.465
1.057
2.664
0.757
1.345
1.245
3.425
SSSD
50
1.308
3.666
1.454
4.457
1.157
3.028
0.993
2.252
1.250
3.441
100
1.310
3.676
1.458
4.487
1.233
3.283
1.149
2.923
1.248
3.444
300
1.307
3.663
1.462
4.507
1.239
3.316
1.147
2.913
1.255
3.467
G.2
Analysis of SSSD
In the following, analyze the factors leading to SSSD's suboptimal performance from several perspectives.
25

---

Published in Transactions on Machine Learning Research (02/2025)
Table 14:
Comparison of imputation results between LSTI and SSSD across different datasets under a fixed
50 missing length.
Flectricity
Traffic
Metr-la
GuangZhou
PEMS04
Rm
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
10%
0.224
0.108
0.265
0.308
0.478
0.710
0.410
0.355
0.232
0.109
30%
0.255
0.144
0.301
0.341
0.508
0.747
0.395
0.321
0.267
0.142
LSTI
50%
0.269
0.157
0.292
0.322
0.568
0.867
0.428
0.368
0.349
0.235
70%
0.321
0.209
0.366
0.428
0.621
1.025
0.491
0.457
0.478
0.388
10%
1.186
3.055
1.432
4.335
0.980
2.357
0.531
0.768
1.032
2.601
30%
1.255
3.446
1.454
4.457
1.057
2.664
0.757
1.345
1.182
3.135
SSSD
50%
1.256
3.449
1.451
4.432
1.134
2.879
1.044
2.450
1.192
3.176
70%
1.309
3.672
1.455
4.460
1.225
3.247
1.150
2.927
1.250
3.447
First, while SSSD reports evaluation results under the Blackout missing pattern in its original paper, the implementation in its publicly available code adopts simplified and less realistic approach. Specifically, SSSD randomly masks fixed-length segment within each imputation window, ensuring uniform missing ratios and segment, lengths across all windows. This significantly reduces the complexity of the missing data patterns, making the model's training and inference processes easier. contrast, the global Blackout missing pattern addressed in this study better reflects real-world scenarios, where missing segments are randomly distributed across the entire dataset rather than confined to specific imputation windows. This introduces variability in missing ratios and continuous missing segment lengths across different windows, substantially increasing the difficulty of the imputation tasks.
Second, the original SSSD paper reports experimental results under the Blackout missing pattern only for the PTB-XL datasct, while results for other datascts, such as MuJoCo and Electricity, are limited to the raudom missing pattern. This omission raises concerns about the generalizability of the SSSD model. As evidenced by the results in Tables 13 and [4] as well as additional experiments, SSSD demonstrates limited effectiveness larger datasets, particularly under the more complex global Blackout missing pattern. These findings underscore its challenges in addressing more demanding data imputation scenarios.
Third, SSSD employs identical missing pattern during both training and testing. In other words, during training, the model is provided with prior knowledge of the fixed missing ratio and continuous missing length for each training window. However, such prior knowledge is often impractical to obtain real-world applications. While this approach allows SSSD) to perform exceptionally well under a predefined fixed missing pattern, its effectiveness diminishes when faced with more complex missing scenarios, such as varying missing ratios and continuous missing lengths across different windows.
In summary, our proposed method, L.ST,, demonstrates significant ad advantage over SSSD, particularly when evaluated under the global Blackout missing pattern. This highlights its robustness and adaptability to complex, real-world conditions.
26

---

Published in Transactions on Machine Learning Research (02/2025)
H Discussion on Implicit Neural Representations (INR)
Implicit Neural Representations (INRs) constitute a class of neural networks designed to parameterize contimuous functions by mapping imput coordinates, such as temporal indices, to corresponding output values. As an emerging area time series analysis, INRs represent data as continuous functions rather than discrete samples, facilitating the modeling of temporal dynamics in coordinate-based manner. This framework is particularly effective for applications involving irregularly sampled data, interpolation, or the reconstruction of missing values, making it well-suited for addressing the global Blackout missing pattern discussed in this paper.
H.1 Comparison between LSTI and TimeFlow
In this section, present TimeFlow (Naour et al. 2023 additional baseline for comparison with LSTI and provide detailed discussion of the strengths and limitations of INR-based methods in addressing the challenges posed by the global Blackout missing data pattern.
Table 15: Comparison of imputation results between LSTI and TimeFlow across different datasets under a fixed 30% missing rate
Electricity
Traffic
Metr-la
GuangZhou
PEMS04
Lm
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
10
0.249
0.138
0.269
0.296
0.342
0.353
0.367
0.293
0.213
0.099
30
0.242
0.129
0.281
0.316
0.434
0.568
0.387
0.323
0.263
0.132
LSTI
50
0.255
0.144
0.301
0.341
0.508
0.747
0.395
0.321
0.267
0.142
100
0.278
0.171
0.334
0.381
0.478
0.714
0.420
0.350
0.425
0.328
300
0.370
0.251
0.450
0.547
0.712
1.225
0.495
0.466
0.569
0.513
10
0.811
0.984
0.744
1.106
0.261
0.277
0.441
0.378
0.226
0.106
30
0.884
1.134
0.941
1.491
0.383
0.417
0.529
0.528
0.238
0.122
TimeFlow
50
0.934
1.397
1.228
2.353
0.699
1.202
0.709
0.933
0.309
0.255
100
0.972
1.514
1.477
3.389
0.892
1.909
1.081
2.097
0.580
0.630
300
1.484
4.467
1.787
5.476
1.142
3.387
1.114
2.371
1.292
2.730
Tables and 16 summarize the performance of TimeFlow on our datasets under the experimental settings. The results indicate that LSTI consistently outperforms TimeFlow across most scenarios, reaffirming LSTI the state-of-the-art solution for addressing the challenges posed by the Blackout missing pattern. Specifically, LSTT demonstrates superior robustness and adaptability, effectively handling variations in missing rates and continuous missing lengths.
It important to note that TimeFlow is not suitable for imputing large-scale datasets. To ensure a fair comparison, we adopted the same training and testing split as L.STI and used the full set of observed data points for training. Under this sctup, TimeFlow requires significant computational resources, taking considerable time and memory complete single training run. This is corroborated by the original TimeFlow paper, where the datasets used were sub-samples of the original data, each containing fewer than 2000 data points. When we trained TineFlow using sub-sample of similar size under the Blackout missing pattern, the results were highly unsatisfactory, with some cases yielding an MSE greater than 1000. Therefore, TimeFlow has notable limitations when applied to large-scale datasets.
It can be observed that TimeFlow exhibits commendable performance on specific datascts when the missing rates are lower, and the lengths of continuous missing segments are relatively short. However, on other datasets, TimeFlow struggles significantly, showing subpar results even at the low-missing-rate settings.
27

---

Published in Transactions on Machine Learning Research (02/2025)
Table 16: Comparison imputation results between LSTI and Timel across different datasets under a
fixed 50 missing length.
Flectricity
Traffic
Metr-la
GuangZhou
PEMS04
Rm
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
10%
0.224
0.108
0.265
0.308
0.478
0.710
0.410
0.355
0.232
0.109
30%
0.255
0.144
0.301
0.341
0.508
0.747
0.395
0.321
0.267
0.142
LSTI
50%
0.269
0.157
0.292
0.322
0.568
0.867
0.428
0.368
0.349
0.235
70%
0.321
0.209
0.366
0.428
0.621
1.025
0.491
0.457
0.478
0.388
10%
0.834
1.020
0.918
1.591
0.469
0.538
0.622
0.678
0.307
0.167
30%
0.934
1.397
1.228
2.353
0.624
1.131
0.709
0.933
0.309
0.255
TimeFlow
50%
1.133
2.114
1.487
3.614
0.699
1.202
0.728
0.943
0.389
0.299
70%
1.356
3.159
1.635
4.460
1.031
2.486
1.452
4.475
0.847
1.639
As the missing rate and continuous missing length increase, TimeFlow's performance deteriorates sharply, sometimes leading to severe deviations that render the imputation results unreliable.
These observations underscore the strengths and limitations of INR-based methods like TimeFlow. While they offer distinct advantages over traditional imputation techniques, especially for structured missing data, their effectiveness not universal. The results suggest that INR-based methods require careful calibration and may not al always generalize well to more complex diverse missing data scenarios. Consequently, practitioners should excrcise caution when adopting such methods.
H.2 Analysis of INR-based methods
The primary distinction between INR-based methods and traditional deep learning imputation techniques is how they conceptualize time series data. Traditional methods treat time series as discrete data points, while INR-based approaches model lime series as continuous functions, with each data point associated with a specific time coordinate. In this section, we examine the factors contributing to the strong performance of INR-based methods, excmplified by TimeFlow, on specific datascts and missing rates, as well as the reasons for their poor performance under other conditions.
First, traditional deep learning methods often impute missing data by replacing missing values with zeros in imputation windows, relying the network to infer missing information. However, this approach struggles with real-world scenarios where varying missing rates and patterns across windows introduce inconsistencies, degrading performance. INR, in contrast, employs coordinate-basel representation, mapping data to temporal coordinates. This eliminates imputation windows and avoids zero placeholders, enabling more effective handling missing data. Similarly, LSTI's bidirectional autoregressive imputation strategy removes missing values during regression, ensuring the model receives complete data and mitigating issues caused by varying missing rates, thereby cnhancing robustness and generalization.
Secondly, the ability model contimuous functions provides INR-based methods with advantage in handling long segments of continuous missing data. However, this approach also has limitations, particularly in terms of generalization across diverse datascts. Since INRs rely on deep learning networks to model implicit function fitting, the resulting function is strongly influenced by the network architecture. In other words, a specific INR structure may excel in fitting certain data distributions, but may struggle significantly with others. For instance, as shown in Tables 15) and 16. TimcFlow performs well on datasets like PEMS04 and Metr-la, but yields poor results on datasets such as Electricity and Traffic. This underscores the selective nature of INR methods and their limited generalization ability across varying datasets.
28

---

Published in Transactions on Machine Learning Research (02/2025)
Third, INR methods perform well in imputing MCAR (Missing Completely at Random) data even under high missing rates but struggle with Blackout missing data under similar conditions. Table 17 shows the results of TimeFlow on the Guangzhou dataset with an MCAR missing pattern, demonstrating notable performance improvement without a sharp decline as the missing rate increases. This underscores the strengths of INR-based methods in handling high-missing-raan MCAR patterns. However, their performance under the Blackout missing pattern remains suboptimal, with a substantial gap compared to LSTI.
Table 17: Imputation results of TlmeFlow on the GuangZhou Dataset under different missing patterns
Rm
10%
30%
50%
70%
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MCAR
0.364
0.325
0.336
0.258
0.322
0.260
0.359
0.293
Blackout
0.622
0.678
0.709
0.933
0.728
0.943
1.452
4.475
In summary, while INR-based methods exhibit strengths in certain scenarios, LSTI clearly outperforms them when dealing with large-scale Blackout missing patterns, making it the more effective choice in such cases.
Broader Impact
The LSTI presents significant potential in healthcare, finance, and environmental science by addressing missing data in time series. In healthcare, it can enhance patient monitoring and personalized care by accurately imputing gaps in vital signs data. However, there are ethical concerns regarding bias, particularly in vulnerable populations, that may affect clinical outcomes. In finance, LSTI could improve predictive models for market behavior, yet imputed data must be rigorously validated to prevent financial instability and market manipulation. In environmental science, the method can support more accurate climate models, but improper imputation may lead to flawed policy decisions. In summary, across all domains, transparency, model validation, and ethical oversight critical to ensuring fairness and mitigating potential risks.
29


Reviews:
[{"review_id": "sE4TDZuAbN", "reviewer": "Reviewer_bUFZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The work presents a well-thought-out approach to tackling the difficult problem of imputing long stretches of consecutive missing values in time series data. This is a common challenge in real-world applications where many existing methods fall short. The proposed Long Short-Term Imputer (LSTI) stands out by combining both long-term and short-term imputation strategies. What I find particularly noteworthy is how the model adaptively balances the contributions of these two components, allowing it to handle datasets with different patterns of missing data effectively. This demonstrates a solid understanding of the complexities involved in time series analysis. By capturing diverse temporal dependencies, LSTI not only improves imputation quality but also offers a practical solution for practitioners working with real-world datasets. Overall, this is a valuable contribution to the field of time series imputation, with the potential for significant real-world impact. \n\nI found one of the standout strengths of this work to be its thoughtful and comprehensive approach to addressing both long-term and short-term dependencies in time series data. The use of the Long-Term Imputer, which incorporates forward and backward prediction networks along with a consistency loss, is particularly effective in capturing long-range temporal relationships. This bidirectional strategy, combined with an autoregressive imputation method, demonstrates a robust ability to handle extended missing intervals, setting it apart from many existing approaches in the field. The addition of a Short-Term Imputer is a thoughtful and practical choice, as it allows the model to effectively capture local temporal patterns. This flexibility enhances the model's adaptability, making it well-suited for handling a range of missing data scenarios, from long consecutive gaps to more sporadic omissions. Such a dual-focused strategy reflects a deep and nuanced understanding of the real-world challenges associated with time series data.\n\nThe Meta-weighting module is an impressive and well-thought-out addition to the LSTI architecture. It dynamically adjusts the importance of long-term and short-term dependencies based on the characteristics of the input data, which is a critical capability for handling diverse missing data patterns. This flexibility makes the module particularly effective in challenging scenarios, such as those involving long stretches of consecutive missing values—situations where traditional methods often fall short. The approach demonstrates a thoughtful and forward-looking perspective, highlighting the authors' recognition of the importance of adaptive methodologies in addressing real-world data challenges. This is a meaningful step forward compared to static imputation strategies. \n\nOne of the key strengths of this submission is the thorough experimental evaluation. The authors present comprehensive comparisons across five real-world datasets, clearly demonstrating the superior performance of LSTI compared to state-of-the-art imputation and forecasting methods. The substantial reduction in error rates highlights its strong potential for real-world applications. Moreover, the inclusion of ablation studies offers valuable insights into the roles of individual components, effectively justifying the design choices made in the LSTI architecture. This careful and rigorous validation process adds significant weight to the theoretical claims and bolsters the reliability of the proposed approach. The authors' thorough exploration of various missing data patterns, such as MCAR and different types of consecutive missing patterns, demonstrates the robustness and adaptability of their model. By addressing these scenarios, the evaluation effectively highlights the method's potential for practical use in a wide range of real-world applications.\n\nThe end-to-end learning framework of LSTI is another notable strength, as it simplifies the imputation process and makes it more accessible for practical use. This is especially important because many practitioners may not have in-depth expertise in deep learning or time series analysis. By offering a more straightforward approach, the authors effectively enhance the usability of their model, which is crucial for promoting its adoption across different fields and ensuring it can be implemented with ease in real-world applications.\n\nThe theoretical foundation of the approach is well-established, particularly in how the consistency loss and weighted ensemble strategy for the Long-Term Imputer are formulated. This demonstrates a clear and deep understanding of the core challenges involved in time series imputation.\n\nWhile this work has several notable strengths, there are a few areas that warrant further consideration. For instance, the computational complexity and training time of the LSTI model, especially in comparison to existing methods, are not explicitly discussed. These factors could be important when evaluating the model's suitability for practical deployment. Additionally, while the model performs well across various datasets, a deeper analysis of its behavior on datasets with differing characteristics—such as varying levels of seasonality or trend—could provide more insight into its generalizability and robustness in diverse real-world scenarios.\n\nOne area that could benefit from further development is the explanation of the training process and the specific roles of the different components within the LSTI framework. Although the overall structure is presented, a more in-depth description of how the Long-term Imputer, Short-term Imputer, and Meta-weighting module interact would help clarify the model’s functionality. It would be helpful to provide more detail on the implementation of consistency regularization and its impact on the training dynamics of both the forward and backward prediction networks. This additional information would improve the reader's comprehension of the model's inner workings.\n\nAdditionally, the paper would benefit from a more detailed discussion on the limitations of the proposed method, especially in situations where the missing data characteristics differ significantly from those in the training datasets. For example, it would be insightful to examine the model's performance under conditions of non-random missingness or when the temporal patterns in the data are irregular. Highlighting these aspects would offer a more balanced perspective on the model's strengths and potential challenges, which is crucial for shaping future research directions and its real-world applications.\n\nAdditionally, the interpretability of the model’s decisions, especially in relation to the Meta-weighting module, could benefit from further elaboration. Offering more detailed explanations of how the model manages the balance between long-term and short-term dependencies in various scenarios would provide deeper insight into its decision-making process. Lastly, while the authors compare their method to several baselines, including recent state-of-the-art models, incorporating comparisons with more specialized techniques tailored for handling long-interval missing data in specific domains would add further value to the paper’s contribution.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The submission introduces valuable contributions to time series imputation, particularly focusing on the challenge of consecutive missing values. The proposed Long Short-term Imputer (LSTI) framework is a notable highlight, combining long-term and short-term imputation strategies to address varying patterns of missing data. By adaptively weighting the outputs of both imputers, LSTI effectively improves the quality of imputation. A key strength of this framework is the Meta-weighting module, which dynamically adjusts the imputation strategy based on input data characteristics. This innovation proves especially beneficial for handling long-interval consecutive missing data. The authors validate their approach through experiments on five real-world datasets, showing that LSTI consistently outperforms existing imputation and forecasting methods, making a strong case for its practical utility. Additionally, the framework’s end-to-end design streamlines the imputation process, reducing complexity compared to traditional methods. This makes the approach more accessible and efficient for real-world applications. Overall, the submission offers a robust and adaptable solution for time series imputation, filling a critical gap in current methodologies while demonstrating its effectiveness through thorough experimentation!", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Based on my review, I would propose several adjustments that would strengthen the work and potentially secure a recommendation for acceptance at TMLR. These adjustments range from critical improvements to suggestions that would enhance the overall quality and impact of the research. A critical adjustment that I believe is a more comprehensive analysis of the computational complexity and training time of the LSTI model compared to existing methods. While the work demonstrates impressive performance improvements, it lacks a detailed discussion of the computational resources required to achieve these results. This information is crucial for assessing the practical applicability of the proposed method in real-world scenarios. The authors should provide benchmarks comparing the training and inference times of LSTI with baseline models across different datasets and missing data patterns.  \n\nA key area for improvement would be to broaden the scope of the experimental evaluation by including a more diverse set of datasets, especially from domains that are not currently represented in the study. While the choice of datasets used is valid, incorporating data from sectors like healthcare, finance, or environmental monitoring could significantly enhance the paper’s claims about the model's generalizability and its broader applicability. Additionally, providing a deeper analysis of how the model performs on datasets with different characteristics—such as varying seasonality or trend—would further strengthen the work. This would offer valuable insights into the model's flexibility and help pinpoint any potential weaknesses or areas that could be refined in future research. \n\nA valuable addition to the paper would be a more in-depth discussion on the interpretability of the model's decisions, especially in relation to the Meta-weighting module. Visualizations or case studies showcasing how the model balances long-term and short-term dependencies in various scenarios could significantly enhance the reader's understanding of the decision-making process and increase its practical applicability. Although not essential for acceptance, a comparison with more specialized methods tailored to handling long-interval missing data in specific domains could further solidify the paper's contribution. This would position LSTI within the wider field of imputation techniques and highlight its unique strengths over domain-specific approaches.\n\nA useful addition to the work would be a more detailed exploration of how the model performs under various hyperparameter settings. This analysis would help assess the robustness of the method and provide practical guidance for users looking to implement LSTI in different contexts. Additionally, to improve the paper’s impact and support reproducibility, I strongly recommend that the authors release their code and pre-trained models, accompanied by clear documentation on how to use and extend the LSTI framework. This would make it easier for other researchers to build upon the work and promote its application in the wider scientific community.\n\nI have one final comment regarding the ablation study. While the current analysis is certainly informative, I believe it could be further strengthened by incorporating a more detailed breakdown of the contributions of each component. In particular, examining the roles of the consistency loss and the Meta-weighting module in greater depth would offer a clearer understanding of how these elements influence the model’s overall performance. This level of detail would provide valuable insights into the design choices and their impact.", "broader_impact_concerns": "In my view, the authors could strengthen their Broader Impact Statement by delving deeper into the ethical considerations associated with their work. The proposed Long Short-Term Imputer (LSTI) demonstrates great potential for addressing consecutive missing values in time series data, with applications spanning critical areas like healthcare, finance, and environmental monitoring. While the current statement mentions the absence of obvious negative social impacts, this assessment might oversimplify the broader ethical implications tied to advancements in time series imputation. A more nuanced discussion could enhance the paper’s perspective on its societal and ethical dimensions. \n\nA critical point for further exploration is the potential application of this technology in sensitive areas where data privacy and accuracy are crucial. For example, in healthcare, more precise imputation of missing patient data could significantly enhance diagnostic accuracy and treatment outcomes. However, this also highlights concerns about the reliability of imputed data and the risks of making critical medical decisions based on reconstructed information. It would be valuable for the authors to discuss how their method can be validated and constrained to prioritize patient safety and uphold ethical standards in such high-stakes environments. Similarly, the financial sector could see notable benefits, such as improved market predictions and risk management. Yet, this raises important issues, including the potential for market manipulation and unfair competitive advantages. The authors are encouraged to address safeguards or regulatory frameworks that could mitigate these risks, ensuring that the method does not unintentionally deepen existing inequalities or create new ethical dilemmas in financial markets.\n\nAn important consideration that seems to be missing from the discussion is the environmental impact of the LSTI model. While the paper does an excellent job of highlighting the model's performance, it overlooks the computational resources required for training and deploying the model at scale. With growing concerns about the carbon footprint associated with AI technologies, it would be valuable for the authors to address the energy efficiency of their approach in comparison to existing methods. Additionally, exploring strategies to reduce the model's environmental impact would strengthen the overall contribution and align it with the broader goals of sustainable AI.\n\nThe authors should also consider discussing the potential risks associated with their method being misused in scenarios like data manipulation or falsification. While the primary goal is to accurately impute missing data, there’s an inherent risk that such techniques could be misapplied to create misleading or fraudulent datasets. Including a thoughtful discussion on how the approach could be misused, along with strategies to mitigate these risks, would add an important layer of ethical consideration to the paper. \n\nIn closing, I’d like to highlight the importance of considering the broader societal implications of improved time series imputation. For example, it would be valuable to explore how this technology could influence decision-making in areas like urban planning, climate change modeling, or economic forecasting. Additionally, reflecting on the potential societal benefits and risks could provide a more well-rounded perspective on its broader impact.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "BTxZib2Y6p", "reviewer": "Reviewer_WCjL", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nOverall, the method handle the imputation of not just short-interval but also long-interval missing values with what it seems (good) reasonable performance.\n\n\nWeaknesses:\n\n1) lack of comparison with state-of-the-art models for long term time series imputations, e.g. SSSD https://openreview.net/pdf?id=hHiIbk7ApW which certainly overcome CSDI, SAITS and other baselines. I would suggest at least a proper citation and justification on why not included in the analysis. \n\n2) the definition of missingness scenarios is certainly clear however not consistent with previous works, e.g. SSSD uses completely at random (Random missing), and consecutive missing interval (blackout missing), but the mentioned baseline also investigate random block missing. \nI could suggest the authors mention/justify from which literature is that they bring the missigness scenarios definition. \n\n3) lack of computational complexity results. I suggest the authors provide a short section on the number of model parameters, size, and the actual training and generation times for a different datasets of varying lengths and number of channels. \n\n4) lack of confidence intervals. I suggest the authors also provide if not for all experiments for a subset of them confidence intervals e.g. 95% CI. across 100 or so imputations. \n\n5) an extension on the previous point, it would be really interesting to know at how many imputations the variance of the error converges, e.g. for a specific length and number of channels, how many imputations would be ideal to make to get a converged mean.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "title: Long Short-Term Imputer: Handling Consecutive Missing Values in Time Series\n\nsummary: this paper addresses the challenge of imputing missing values in time series data, especially focusing on long-interval consecutive missing values. The authors introduce the Long Short-Term Imputer (LSTI), framework that combines two specialized models: a Long-term Imputer and a Short-term Imputer, along with a Meta-weighting module for adaptive integration. The Long-term Imputer employs bidirectional autoregressive prediction with a consistency regularization loss, designed to capture long-term temporal dependencies. Meanwhile, the Short-term Imputer uses a self-mapping network to handle short-term dependencies effectively. The Meta-weighting module adaptively blends the strengths of both models based on the nature of missing data.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weaknesses.", "broader_impact_concerns": "The statement could improve clarity on why the method is not directly applicable to real-world tasks and how this limitation impacts its utility.\n\nThe authors fail to explore potential risks, such as biases in imputation or over-reliance on imputed data. See lack of confidence intervals in review.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "enegjzi4Tq", "reviewer": "Reviewer_5gmT", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- **S1.** The blackout missing values setting is particularly relevant in practice and challenging to address, making the chosen problem both important and interesting.\n- **S2.** The Long Short-Term Imputer model is well-explained and thoughtfully designed. I found the use of the scalar weighting mechanism, which adjusts the emphasis on FPNet or BPNet based on the location of the missing values, to be intuitive and elegant. It was surprising that this part of the model (the long term imputer) does not rely solely on the $\\mathcal{L}_p$ term.\n- **S3.** The experimental results are impressive, with significant improvements over the considered baselines.\n\n\n**Weaknesses**\n\n-  **W1.**  The paper is hard to follow, particularly in Section 3 (Method) and Section 4 (Experiments). Specific examples include:\n    - Certain aspects of the problem formulation, such as the introduction of the Q matrices, feel unnecessary and overcomplicate the explanations without contributing much to understanding.\n    - In the Long-Term Imputer, four terms are used in the loss function, yet no ablation study is provided. As mentioned earlier, it is surprising that the network is not trained solely using the $\\mathcal{L}_p$ term. This complexity in the loss terms might hinder reader comprehension.\n    - The Short-Term Imputer (SMNet) is insufficiently discussed. Beyond being described as a single self-mapping network, there are no details provided. An explanation or a figure (even in the appendix) would improve clarity.\n    - The experimental setup is unclear. For instance, two hyperparameters are mentioned for generating missing data: the overall missing rate ($\\mathcal{R}_m$) and the missing length ($\\mathcal{L}_m$) of the window. Does this mean that, for example, in Table 1, first row (electricity dataset), you used windows of length 10 with blocks of 30% missing values (i.e., three consecutive values)? Additionally, there seems to be a typo in Table 1, where $\\mathcal{R}_m$ is indicated, but $\\mathcal{L}_m$ is likely intended.\n    - Including figures to clarify the experimental setup would greatly enhance understanding.\n-  **W2.** While the results are promising, there are no visualizations to evaluate the quality of imputations compared to the baselines. Adding a few imputation plots would significantly strengthen the paper.\n-  **W3.** The paper does not discuss Implicit Neural Representations for time series imputation [1, 2, 3], which is an emerging area in the literature. These models show strong performance and appear particularly suited for blackout scenarios. Including a discussion (and some experiments, if the authors have time) on this topic would add valuable context.\n\n\n[1] Time Series Continuous Modeling for Imputation and Forecasting with Implicit Neural Representations, Le Naour et al., TMLR 2024\n\n[2] ImputeINR: Enhancing Time Series Imputation with Adaptive Group-based Implicit Neural Representations, Li et al. (https://openreview.net/forum?id=xcPN6Or88c)\n\n[3] MADS: Modulated Auto-Decoding SIREN for time series imputation, Bamford et al, (https://arxiv.org/abs/2307.00868)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces the Long Short-Term Imputer (LTSI) model for time series imputation. The LTSI model is specifically designed to effectively handle missing values with a ‘blackout’ pattern, where all channels are missing simultaneously for consecutive timestamps. The model relies on three key components:\n\n- **Long-Term Imputer**: Leverages and combines forward and backward neural predictors.\n- **Short-Term Imputer**: Focuses on local missing values.\n- **Meta-Weighing Module**: Takes the data and missing mask as input and outputs a scalar weight to combine the long- and short-term imputers.\n\nExperiments on five real-world datasets demonstrate that the LTSI model achieves strong performance compared to the considered baselines.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see weaknesses section.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers are leaning towards accepting the paper. The authors fully respond to the majority of the reviewers' remarks, the paper improved after the revision, especially it is much clearer to read and the experimental results are good. 

Therefore I accept (with minor revision) the paper for publication in the TMLR.

However, the following concerns listed by Reviewer 5gmT remain, and I encourage the authors to incorporate appropriate changes in the final revision of their paper:
"The improvements compared to all other considered baselines are impressive but sometimes questionable. All the considered datasets exhibit clear patterns and frequencies, and I do not understand why some results for strong baselines are this poor. There may be aspects of the training procedure that I do not fully understand. The paper still lacks qualitative comparisons, such as more plots and a specific focus on a dataset in the appendix."
Decision: Accept (poster)