Investigating a Model-Agnostic and Imputation-Free Approach for Irregularly-Sampled Multivariate Time-Series Modeling
Authors: Abhilash Neog
Arka Daw
Sepideh Fatemi
Medha Sawhney
Aanish Pradhan
Mary E. Lofton
Bennett J. McAfee
Adrienne Breef-Pilz
Heather L. Wander
Dexter W Howard
Cayelan C. Carey
Paul Hanson
Anuj Karpatne
Venue: Accepted by TMLR ()

Abstract: Modeling Irregularly-sampled and Multivariate Time Series (IMTS) is crucial across a variety of applications where different sets of variates may be missing at different time-steps due to sensor malfunctions or high data acquisition costs. Existing approaches for IMTS either
consider a two-stage impute-then-model framework or involve specialized architectures specific to a particular model and task. We perform a series of experiments to derive insights about the performance of IMTS methods on a variety of semi-synthetic and real-world datasets for both classification and forecasting. We also introduce Missing Feature-aware Time Series Modeling (MissTSM) or MissTSM, a simple model-agnostic and imputation-free approach for IMTS modeling. We show that MissTSM shows competitive performance compared to other IMTS approaches, especially when the amount of missing values is large and the data lacks simplistic periodic structures - conditions common to real-world IMTS applications.

Published in Transactions on Machine Learning Research (01/2026)
Investigating a Model-Agnostic and Imputation-Free Ap- proach for Irregulary-Samppled Multivariate Time-Series Mod- eling
Abhilash Neog"", Arka Daw?, Sepideh Fatemi", Medha Sawhney!, Aanish Pradhan', Mary E. Lofton', Bennett J. McAfee Adricnne Breef-Pilz', Heather L. Wander', Dexter W Howard', Cayelan C. Carey', Paul Hanson Anuj Karpatne Virginia Tech, *Oak Ridge National Lab, $ Universiiy of Wisconsin-MMddion
Reviewed on OpenReview: https: // openreview. net/ forum? id= JODDDVAAA3
Abstract
Modeling Irregularly-sampled and Multivariate Time Series (IMTS) is crucial across a variety of applications where different sets of variates may be missing at different time-steps due to sensor malfunctions or high data acquisition costs. Existing approaches for LMTS either consider two-stage impute-then-mode framework or involve specialized architectures specific lo a particular model and task. We perform series of experiments to derive insights about the performance of IMTS methods on a variety of semi-synthetic and real-world datasets for both classification and forecasting. We also introduce Missing Featuro-aware Time Scries Modeling (MissTSM) or MissTSM, a simple model-agnostic and imputation-fres approach for IMTS modeling. We show that MissTSM shows competitive performance compared to other IMTS approaches, especially when the amount of missing values large and the data lacks simplistic periodic structures-conditions common to roal-world IMTS applications.
1 Introduction
Deep Learning for modeling multivariate Time-Series (MTS) is a rapidly growing field, with two major downstream tasks: forecasting and classífication. Research (Dong et al. 2024 Nie al., 2022a, Liu al., 2023) in this field has been fueled by the availability of benchmark MTS datasets spanning diverse applications such electric load forecasting and health monitoring containing fixed sets of variates regularly sampled over time. However, real-world MTS applications are plagued by missing values occurring over arbitrary sets of variates at every time-step (e.g. due to sensor malfunctions), resulting in Irregularly-sampled MTS (IMTS) datasets. IMTS modeling is particularly challenging because the misalignment of variates across time impairs transformer models that assume a fixed set of variates to be observed at every time-step
A common approach for IMTS modeling is to use a two-step framework where we first use imputation methods (Ahn et al. 2022: Batista et. al., 2002) to fill in missing values based observed data, followed by feeding the imputed sime-series to an MTS model (see Figure D). Note that the choice of imputation method is agnostic to the MTS model, making it "model-agnostic" However, the effectiveness of this framework relies on the quality of performed imputation, which can degrade if the time-series lacks periodic ure or if the imputation method is overly simplistic. Imputation can also introduce artificial patterns or artifacts into the data, which MTS models may interpret as genuine trends or observations. Moreover, deep learning-based imputation methods require training, which adds the overall computational cost of IMTS modeling.
Imputation-free approaches for IMTS have also been developed in recent literature (Che et al. 2018 Rubanoval et al. 2019). that involve specialized architectures to handle missing values in time-series for specific
" *Corresponding author: abbilashh22vtt.ddh karpatne@vvt.ddu Code available https://githin htw.:///www.......aaeee/oo/aaaaa

---

Published in Transactions on Machine Learning Research (01/2026)
downstream tasks such classification (see Figure However, these approaches have been empirically shown to struggle with capturing long-term temporal dependencies that are central to the problem of forecasting, are difficult to parallelize, and incur high computational costs. Furthermore, existing imputation-fiet IMTS approaches are not model-agnostic, i.e., they have been developed as specialized model architectures that cannot be used as a generic wrapper with latest advances in MTS models, restricting their adaptability and performance.
Given the complementary strengths and weaknesses of existing approaches in IMTS modeling, we ask the following question can we develop model-agnostic and impulation-free approach for IMTS modeling that can be used in a variety of dounstream tasks (e.g., fore- casting and classification)? To address this question, we analyze the prevailing strategies for converting time- series into Lokens in existing transformer-based MTS models. There are two primary ways for converting time scries into tokens: (a) treating cach time step (all variates) as a token, or (b) considering each variate (all time steps) as a token (Liu et al. 2023 While these strategies work well for regular MTS data, they are not suited to handle IMTS data because we may be missing some variates for variate, making the computation of tokens infea- sible. In contrast, we explore a different perspective for creating time-series tokens: independently embed- ding each combination of time-step and variate as a token. Time-variate combinations with missing values can then be handled using masked cross-atlention without performing any explicit, imputation. Building on this intuition, we introduce MissTTM,, a simple model-agnostic and imputation-fies approach for IMTS modeling, designed as "plug-and-play" layer that can be integrated into any backbone MTS model to handle IMTS data. The advantage of such an approach is that it (a) does not introduce any impulation artifacls, and (b) can act as a wrapper around any MTS model.
Mixsnn
(?) imputatlon-bssdd Approaches
Impuped Values
Imputation Model
Medil-Agreako
(2) Imputation-free & Modal-Agnostie
Time stapue
MaxTTMM Layer (Wrthh/peerstt
Any MTS Model
(3) Imputaaiion-ree & Specialized Models
Downsorem Worccasting, dlassfication, ...]
Figure 1: We investigate the relative importance of three categories of approaches for modeling irregular and multivariate time-series: (1) imputation-base ap- proaches, (2) model-agnostic and imputation-fre ap- proaches (proposed MissTSM layer), and (3) imputation- free approaches involving specialized architectures.
time step or some time-steps
In this work, we make the following contributions: (1) We introduce MissTSM, a model-agnostic and imputation-free approach; (2) We conduct a comprehensive experimental study on a variety of datasets for both classification and forecasting tasks, using synthetic masking techniques as well as real-world occurrence (a) sensitivily of impulation-
frumeworks on the choice of imputation technique and the nature of missing values, and (b) the performance of IMTS approaches the fraction of missing values varies; (3) We demonstrate that Miss achieves competitive performance compared to other IMTS approaches, especially when the amount of missing values is large and the data lacks simplistic periodic structures conditions common to real-world IMTS applications.
of missing values. This study investigates
2 Related Works
Time-series Forecasting. With the introduction of attention mechanisms via transformer models [Vaswani et al. 2017), number of transformer-based time-series models have been developed in the last few years (Wu et al. 2021 Nie et al., 2022b Dong et al.
Liu et al.) 2023). While Transformer-bast based models have shown great promise, recently there has been strong interest in exploring the use of simple linear models for time-scries forecasting as well (Zeng et al.
2024
Ekambaram et al., 2023). In addition, with the risc of self-supervised learning-based models such as masked auto-encoders (MAEs) (He et al. 2022), a new calegory of MAE-style time-series models have emerged (Dong et al., 2024] that have received a lot of recent interest owing to their ability of learning both low-level and high-level representations for varied downstream tasks such as forecasting and classification. However, while these methods can deal with missing values in the temporal domain, they are unable to handle missing values across both variates and time.
2023
2

---

Published in Transactions on Machine Learning Research (01/2026)
Imputation Methods. Traditionally, most imputation techniques for handling missing values in time-series
have been based on statistical approaches (Fung) 2006 Batista et al.
2002 Dempster et al. 1977 Mnih &
Salakhutdinov 2007]. In recent years, there is a growing trend to use deep learning methods for time-series
imputation, such as SAITS (Du et al. 2023 CSDI
Tashiro et al.
2021a). GAIN (Yoon et al.] 2018a),
and BRITS (Cao et al. leverage cross-channel correlations (Batista et al., 2002: Acuna & Rodriguez 2004) and those that exploit temporal dynamics (Box et al. 2015). Recently, deep learning-
2018 Imputation Lechniques can be broadly classilied into two classes: those that,
approaches for imputation have been developed (Tashiro et al. 2021b Cini et al. 2021; Liu et al.) 2019 Cao et al. 2018 Du et al.) 2023), which can jointly learn the temporal dynamics with cross-channel correlations. These methods, however, rely on a single entangled representation (or hidden state) to model nonlinear dynamics (Woo et al., 2022) which can be a limitation in capluring the multifaceted nature of time-series. Matrix factorization based techniques Liu et al., 2022) have also been proposed that offer disentangled temporal representations, enhancing the ability to differentiate and model distinct temporal features. While these deep lcarning- caaning-bassdd models are highly efficient during inference, they require additional training time, which add to the already large time complexity of MTS models.
Imputation-free IMTS Models. In the last decade, there has been significant growth of models and architectures for learning from IMTS data. Some of the simpler approaches to deal with IMTS data involve working with fixed temporal discretization (Marlin et al. 2012 Lipton et al., 2016). The primary drawback with these approaches is that they make ad-hoc choices in terms of discretization window width and aggregation functions within the windows (Shukla & Marlin 2020). A popular set of approaches for handling IMTS data are neurrence-based approaches, which includes RNN-based methods such as GRU-D (Che et al., 2018). However, GRU-D has limited scalability to long sequences. Other recurrence-based approaches based on Ordinary Differential Equations (ODE) (Chen et al. 2018 Rubanova et al., 2019) provide an effective solution in modeling the continuous time semantics. These methods are however significantly slow and memory-intensive, as they constantly need to apply the ODE solver and solving ODEs require numerical integration, thus making it impractical for long-term forecasting and large datasets.
Transformer- based methods such as ContiFormer (Chen et al.) 2023) and mTAN (Shukla & Marlin 2021 addressos these limitations by explicitly integrating the modeling abilities of Neural ODEs into the attention mechanism and introducing continuous time attention mechanism that learns time embeddings dynamically, respectively. However, despite ContiFormer being principled and effective approach, solving an ODE for each key and value incurs a high computational cost. Also, while the runtime speed of mTAN is relatively faster, it is however, inherently optimized toward interpolating missing values by learning representations at fixed set of reference points, thus limiting it's extrapolation or forecasting ability.
This is another limitation of IMTS approaches their evaluation is mostly limited to a single task, most often to time series classification, thus limiting their applicability. ContiFormer performs evaluation forecasting tasks, however, they consider regular and clean benchmark time-series datasets in their evaluation. Another limitation in terms of evaluation is that the prior works primarily focus on other IMTS models for comparison, completely ignoring the two-stage imputation approach, which is a more common and practical way of dealing with tissing-value data. Our work aims to solve these issues by providing a comprehensive comparison against both existing imputation- free and two-stage imputation- based approaches, and proposing a model-agnostic transformation-allovig any task-specific SOTA model to be applied on any irregularly sampled time-series data with minimal data transformations.
3 Proposed Missing Feature Time-Series Modeling (MissTSM) Framework
3.1 Notations and Problem Formulations
Let us represent a multivariate time-series as X E RTXN, where T is the number of time-steps, and N is the dimensionality (mumber of variates) of the time-series. We assume a subset of variates (or features) to be missing at some time-steps of X, represented in the form of a missing-value mask M E [0, 1|TxN, where M(ra) represents the value of the mask at t-th time-step and d-th dimension. M(.a) = 1 denotes that the corresponding value in (() is missing, while M(r.a) O denotes that (t.d) is observed. Furthermore, let
3

---

Published in Transactions on Machine Learning Research (01/2026)
denote (t.)) RR the multiple variates of the time-series at a particular time-step and ((.d) R the uni-variate time-series for the variate d. In this paper, we consider two downstream tasks for time-series modeling; forecasting and classification. For forecasting, the goal is to predict the future S time-steps of X represented as Y E RSXN, Alternatively, for time-serics classification, the goal is to prodict output labels Y (1, 21,,,..,, C} given X, where C the number of classes.
3.2 Learning Embeddings for Time-Series with Missing Features
Limitations of Existing Transformer Methods: The first step in time-series modeling using transformer-basee architectures is to learn embedding of the time-series X that can be sent to the transformer encoder. Traditionally, this is done using an Embedding layer (typically implemented us- ing a multi-layered perceptron) as Embedding IN + RRD that maps X E RTxN to the embedding HE E RTXD, where D is the embedding dimension. The Embedding layer operates on every time-step independently such that the set of variates observed at time-step t, ((.))) considered as single token and mapped to the embedding vector he R as h, Embedding( (see Figure 2())). An alternate embedding scheme was recently in- troduced in the framework of inverted Trans- former ((Transformer) (Liu et al. 2023)), where the
Tiken
Tikes
uni-variate time-series for the d-th variate, (i.d)) is considered as a single token and mapped to the embedding vector: ha = Embedding (x.))) (see Figure 2 b)).
Thas depe
Time saps
Tie
Enderdding
Endonttios
While both these embedding schemes have their unique advantages, they are unfit to handle time- series with arbitrary sets of missing values at every time-step. In particular, the imput tokens to the Embedding layer of Transformer or iTransformer requires all components of (t.)) or ((.d) to be observed, respectively. If any of the components in these tokens missing, we will not be able to compute their embeddings and thus will have to discard either the time-step or the variate, leading to loss of information.
Tiee dips
Time
Tranddaarrarr
Figure 2: Schematic of the Time Feature Independent (TFI) Embedding of MissTSM that learns different embedding for every combination of time-step and variate, contrast the time-only embeddings of Transformer (Vaswani et al. 2017 and the variate-only embeddings of iTransformers (Liu et al. 2023
Time-Feature Independent (TFI) Embed-
ding: To address this challenge as well as to utilize inter-variate interactions similar to Wei et al. (2023), consider Time-Feature Independent (TFI) Embedding scheme for time-series with missing features, where the value at each combination of time-step t and variate d is considered a single token
X (t,d)) and is independently mapped to an embodding using TFIEmbedding IR + R as follows:
((t.d) = TF (TEEmbeeding(X(t..
(1)
In other words, the TFIEmbedding Layer (which is simple MLP layer) maps X E TxNN into the TFI embedding HIFI E R (see Figure 2 c)). The TFIEmbedding is applied only on tokens X(t.4) that are observed (for missing tokens, i.e., M( 1, we generate a dummy embedding that gets masked out the MFAA layer). The advantage of such an approach that even if particular value in the time-series is missing, other obscrved values in the time-scries can be embodded "independently" without being affected by the missing values. Moreover, it allows the MPAA layer to leverage the high-dimensional embeddings to store richer representations bringing in the context of time and variate by computing masked cross-attention among the observed features at a time-step to account for the missing features.
2D Positional Encodings: We add Positional Encoding vectors PE to the TFI embedding HIF obtain positionally-encodded embeddings, Z = PE + HTFI Since TFI embeddings treat every time-feature
4

---

Published in Transactions on Machine Learning Research (01/2026)
Missing Feature-Aware Attention (MFAA)
Encoder
Decador
Erooodet Em becoings
Oriberding
Lear Leareable
Mask
Latent Reepesentation (et time
Trensformrr
Decoder Embeddiing
(time only)
Time stops
Matrix Mubplicatioo
Mazsing
lampant
D-Positionel Encoding
MFA Atlention Score
Unbea
Transformer
Layerfiomm
Decader Blocks
Softmax
Missing Feature Awcrre Aftention (MIAA)
Skip COO
Mask
Po Pofiioall
Prediction Head
(at time
Sca Product
Enpoding
Timeel nature indepencent Entedding (TFI)
Query
Key
Value
Reconatructed Time-seriss
MissTaM Layer
Tim-seres
Varisies
Varatre
Leernable
Quary Vector
Variates
steps
Timm stops
Figure 3: Overview of the MissTSM layer integrated within the Masked 2023 A zoomed-in view of the MFAA is shown on the left.
framework (Li et al.
combination a token, we use 2D-positional encoding scheme defined as follows:
PE(t, d, 21) = sin
(00DD(4:/D)
(2)
PE(L, d, 26 + 1) cos
10000(4:/D)
d
PB(I, 2j D/2) = sin
10000(477D)
PE(1, d, 2j 13+++++ + D/2) = cos
10000(4;/D)
where is the time-step, d is the feature, and i,j E [0, D/4) are integers.
3.3 Missing Feature- Aware Attention (MFAA)
The MFAA Layer illustrated in Figure leverage the power of "masked-attention" for learning latent representations at every time-step using partially observed features. MFAA works by computing attention scores based on the partially observed features at lime-step I, which are then used to perform weighted sum of observed features to obtain the latent representation L, As shown in Figure 3 these latent representations are projected back using a linear layer, to the original input shape before being fed into the downstream model (here, the encoder-decoder based self-supervised learning framework). Note that MEAA is not designed for long-term temporal modeling. Its primary role is to handle missing values a time-step by using the observed variates at the same time-step as context. It relies on the subsequent backbone to model long-range temporal dynamics MFAA performs masked cross-attention using learnable query vector and observed data keys and values. This separation of roles is inspired by similar architectures in multi-modal grounding, for example, in Carion et al. (2020), where learnable object queries serve as abstract object representations to focus on distinct objects in an image without requiring predefined region proposals, enabling set-based prediction. Similarly, in our setting, the learnable queries capture the interactions among variates independent of time, cnabling the model to attend to the most informative aspocts of observed variates any time-step fod through keys and values. This intuition aligns with the query- mechanism in mTAN (Shukla & Marlin 2021), which introduces structured way to aggregate information observed time-series data.
5

---

Published in Transactions on Machine Learning Research (01/2026)
However, there's is key difference in the nature of the query- while mTAN uses discrete reference points a fixed temporal grid to achieve this, our single learnable query generalizes across variates al every time step, allowing for a more flexible representation of feature interactions. Another key difference is in the architecture flow mAAN first performs a "temporal-first variate-independent interpolation' to perform temporal attention for each feature to create a regular sequence, then performs a linear combination to generate single per timestep representation. In contrast, MissTSM 'variate-first', with the MFAA layer performing cross-variate attention to create a representation L at each timestep. Then, the backbone model learns the temporal relationships from this sequence of summaries. Also note that goal is not to propose new attention mechanism, rather, our novelty lies in the novel adaptation of masked cross-attention for effectively modeling IMTS data. To the best of our knowledge, this novel adaptation of masked cross-attention for IMTS modeling has not been tried before in any previous works.
Mathematical Formulation: To obtain attention scores from partially observed features at time-step, we apply a masked scaled-dot product operation followed by softmax operation described as follows. We first define a learnable query veclor Q E IRIID which is independent of the variates and time-steps. The positionally-enctn encodeed embeddings at time-step t, Z(::)) are used as key and value inputs in the MFAA Layer. Specifically, The query, key, and value vectors are defined using linear projections as follows: Q QwQ K, ((..)) WK v WV Here, Q E 1xxd and K,, V. E RNNxd, where dy is the dimension of the vectors after linear projection. The linear projection matrices for the query, key, and values are defined as: Wº, W WV E gDxd respectively. Note that the key K, and value V. vectors depend on the time-step t, while the query vector doesn't 't change with time. We then define the Missing Prature-Aaad Attention Score at given time-step & as masked scalar dot-product of the query and key vector followed by normalization of the scores using Softmax operation, formally defined follows:
A MFAAScore Q K. M(aa)
QKJ
Sof
+ +nMaa
(4)
ddl
where A& R' is the MFAA Score vector of size N corresponding to the N variates, and 17 oo is large negative bias. The negative bias term y) forces the masked-elements that correspond to the missing variates in the time-scrics to have an attention score of zero. Thus, by dcfinition, the i-th clement of the MFAA Score ((.)) #0 - Ma.. = O. We compute the latent representation L, as a weighted sum of the MFAA score A, and the Value vector V, follows:
L MFAA(A, V.) = E
(5)
Similar to multi-head attention used in traditional transformers, we extend MFAA to multiple heads as follows:
MultiHeadMFAN ((.:).M..))) ConcattLi .!...., L 1--1). wo
(6)
where h is the number of heads, W E Rddde D. L; the latent representation obtained from the i-th MHAA Layer, and D, is the output-dimension of the MultiHeadMFAA Layer
3.4 Putting Everything Together: Plugging MSs with any MTS Model
Figure 3 shows the overall framework of a Masked Auto-Encoder (MAE) (He et al., 2022 based time-series model integrated with MissTSM. For input time-series X, we apply the TFI embedding layer followed by the MFAA layer to learn latent representation for every time-step. The latent representations are then projected back to the original input shape to be fed into the downstream model. In this work, we opted for a MAE-based time-series model as the default downstream or base model, primarily due to its recent success in time-series modeling and its ability to perform both time-series forecasting and classification tasks, Furthermore, out of the several stato-of-the-art masked time-sorics modeling techniques, we intentionally chose the simplest variation of MAE, namely Ti-MAE (Li al. 2023), to highlight the effectiveness of TFI and MFAA layers in handling missing values.

---

Published in Transactions on Machine Learning Research (01/2026)
4
Experimental Setup
Baselines:
We benchmark against two categories of models. For MTS, consider SimMTM
Dong et al.
2024), PatchTST (Nee et al. 2022b), AutoFormer (Wu el al., 2021 DLinear (eng el. al.
2023), and
iTransformrr (Liu et al. 2023) Imputation strategies used are, 2nd -order spline interpolation MMcKinley &
Levine
1998). k-Nearest Neighbor (Tan et al.) 2019), and SAITS (Du et al. 2023) and BRITS
(Cao et al.,
2018).
For IMTS, we evaluate GRU-D (Che et al., 2018), Latent ODE (Rubanova et al., 2019). SeFT (Horn
et al.
2020), mTANND (Shukla & Marlin 2021). p. Raindrop (Zhang et al. 2021), and MTGNN (Wu al.
2020
Baseline choice is aligned with the task each model was originally designed for.
We considered three popular time-series forecasting datasets: ETTh2, ETTm2 (Zhou et al., 2021) and Weather (Weather) 202T). For classification, we considered three real-world datasets, namely, Epilepsy (Andrzejak ct al., 2001), EMG (Goldberger et al. 2000a), and Gesture (Liu et al. 2009). We follow the same evaluation setups as proposed in TF-C (Zhang et al., 2022). To simulate varying scenarios of missing values appearing real-world time-series datasets, we adopt two synthetic masking schemes that we apply on these benchmark datasots, namely missing completely at random (MCAR) masking and periodic masking. Furthermore, we compared our performance on five real-world datasets: PhysioNet-2012 (Silva) et al. 2012). P12 [Goldberger et al. 2000b) and P19 (Reyna al. 2020 for health monitoring; Falling Creek Reservoir (FCR) dataset for modeling lake water quality, and Lake Mendota from the North Temperate Lakes Long-Tern Term Ecological Research program (NTL-LTER; Magmuson et al., 2024 also for modeling lakes. See Appendix for more details.
Datasets:
5 Results and Discussions
Here, we discuss our findings with respect to imputation-basee vs. imputation free methods, and model- agnostic vs. specialized methods across a variely of datasets, lasks, and missing value settings.
5.1
Imputation-based vs. Imputation-free
5.1.1
Impact of Missing Data Fractions.
ETTh2 Io 720, MCAR
ETTmZ, 7=020, MEAR
BTTNZ 72720, Periodi
ETTmZ T 720, Periodic
Mean Squered Eror (MSB)
1.44
1.4
Mol Seuard Broy 18
1.50
marolama
-150
Magn Squared Erarr (MSE)
2.0
Manarre
Auteforme:
2.0
Squared Brerr (MSE)
1.50
5MANTM
SENNN
NexTSN
1.0
1.0
1.25
1.5
2.25
1.25
1.00
1.00
1.0
1.0
0.4
0.4
0.50
0.3
0.5
0.50
0.25
0.25
0.25
0.25
Fractions
Fractions
Practions
Practions
(a) ETTh2, MCAR
(b) ETTm2, MCAR
(c) ETTh2, Periodic
(d) ETTm2, Periodic
Figure 4: Performance comparison against different TS Baselines imputed with SAITS, across different missing data fractions.
To understand the effect of varying masking fractions on the forecasting performance, we consider five forecasting models trained on SAITS-imputed data the set of imputation-based baselines. We compare their results with MissTSM integrated within the MAE framework an imputation-fres approach. Figure shows variations in the Mean Squared Error (MSE) as increase the missing value fraction in MCAR and periodic masking scheme from 0.6 to 0.9 for forecasting horizon T = 720 on two ETT datascts. We can baselines
see that, on average, as we increase the amount of missing values in the data, imputation and MissTSM show an increasing trend in MSE. This is expected as larger missing value fractions starve IMTS models with greater amount of information degrading their performance. However, the risc in MSE of MissTSM with missing value fractions is much less pronounced than imputation based baselines consistently across the two datasets and synthetic masking schemes (MCAR and Periodic Masking). Further, note that.
7

---

Published in Transactions on Machine Learning Research (01/2026)
MissTSM shows smaller standard deviations compared to the large and varying standard deviations of the imputation-based approaches (w.r.t the increasing missingness). These results suggest that imputation-based frameworks struggle when the amount of missing values is high, possibly due to the poor performance of imputation methods when the number observations is small.
75
FMG
100
Fpiiepsy
Gesture
Miclie
70
Camimy 2ot
Seart the
Stemm Spll
65
60
Score
F1 $i5ccre
55
150
50
45
80
35
0.2
0.4
0.6
0.8
50-
0.2
0.6
0.5
0.8
0.2
0.4
0.6
0.8
Fractions
Figure 5: Classification F1 scores on three datasets, EMG, Epilepsy, and Gesture. Masking fractions
considered: 0.2, 0.4, 0.6, 0.8.
We conduct a similar study to understand the impact missing data fractions on classification tasks with MCAR masking scheme (see Figure 5). Similar forecasting, we see, on average, a gradual decrease the F1 scores with increasing missing fractions of the imputation-based approaches. We also observe a high range of variability in the Spline-imputed baselines, which suggests that the polynomial order of spline imputation can be further fine-tuned specific to the data. On the other hand, MissTSM shows consistently strong performance across all the three datasets.
5.1.2 Impact of the Nature of Missing Values.
To understand the performance of imputation-based and imputation-free approaches under varying conditions of missing data, we compared their results across the two synthetic masking patterns: MCAR and Periodic Masking. From Fig. G we can observe that for the ETTTh dataset, models perform consistently better under random masking compared to periodic. We can also see that the performance difference between MCAR and Periodic masking is, on an average, higher for SATTS-imputed models compared to Spline. This suggests that the typer-parameters of SAITS can be further fine-tuned on the Periodic dataset, which is relatively casier for Spline to model. Additionally, the performance under MCAR and Periodic missingness Weather dataset is comparatively similar, which hint towards high seasonality within the weather dataset, thus helping the imputation-based baselines on this dataset.
5.1.3 Impact of Imputation Methods.
The choice of imputation method dictates the overall performance of imputation-based frame- works. In Figure 7a we compare four impu- lation techniques: Spline, kNN, BRITS, and SAITS, paired with two MTS models (iTrans- former and Patch at a forecasting horizon of T 720 and 70% missing data fraction. Model performance
Ana yoing Impact of Offerent iosutation Models
Clscsiication FI F1-scores
Mastso PRECTTTT mundorme
Inpuistinn om
SAIS
Spline
Uiit
DRITS
1AITS
MANSTON
ERITS-imputed data is relatively poor, whereas models trained on SAITS-imputed data performs relatively good. This difference in performance indicates the impact of imputation models on downstream tasks within imputation-based frameworks. No- tably, MissTSM- based imputation-Fi model achieves relatively low MSE scores compared to most imputation-based frameworks.
0.5
M5E
F2 sccoore
(a) Varying imputation models. (b) Classification results of im- Performance on ETTh2.
putation models on PhysioNet.
Figure 7: Comparison MSE (lower is better) and F1-score acToss imputation methods.

---

Published in Transactions on Machine Learning Research (01/2020)
Neeoods
Mea1SN
Swaryy
SuMTR 5P
Asshama 50
SanMIM
Asshama
RECNTTT
Tandomerc
Tandsmme
SAchISI_SA
Paterommr: 3P
AursFormrr 54
50
Duicar SA
mansforerr
Masking MDA
0.25
0.50
0.75
MIEE
VISE
MSE
Figure 6: Comparison of different masking methods (70% missing fraction): MCAR vs. Periodic Masking for ETTh2, ETTm2, and Weather datasets. SA stands for SAITS and SP stands for Spline. Lower MSE better
Figure 7b we compare MissTTSM with six imputation baselines M-RNN Yoon et al. 2018b), GP-VAE Fortuin 2020), BRITS (Cao al. 2018). Transformer (Vaswani al. 2017 and SAITS Du al., 2023) a popular real-world classification dataset, PhysioNet. (Silva et al. 2012) following the same evaluation sctup proposed in (Du et al. 2023). MissTSM achieves impressive Fi-score of 57.84%, representing approxinately 15% improvement over the best-performing model (trained SAITS imputed data). This substantial performance gain on a real-world dataset with missing values highlights the potential of imputation-free or single-stage approaches compared to imputation-based approaches.
5.2
Comparing Agnss-Ag vs. Specialized Models
5.2.1
Analyzing MissTSM on IMTS Classification and Forecasting
We evaluate MissTSM on both classification and forecasting tasks for irregular multivariate time series. To illustrate the generality of our approach, we study two case models: (i) GRU-D, a specialized classifier for irregularly sampled data, and (ii) Latent ODE, a continuous-time generative model not originally designed for forocasting but adapted here to a long-term ter-term prediction setting. These case studics emphasize how specializod methods struggle when moved beyond their intended use, underscoring the value of modellaggnstic approaches.
IMTS Classification. We conduct experiments on the IMTS classification task using the P12 GGoldberger et al. 2000b) and P19 (Reyna et al. 2020) datasets, following the same evaluation protocol as Luo et al.
(2025] We report the baseline results for the considered models directly from Luo et. al. 2025 Table
highlights the strong potential of model-agnostic approaches; integrating the MissTSM layer, can achieve
performance on par with or exceeding that of several well-known IMTS models.
Comparing MisSTSM with GRU-D on Classification. To analyze the potential of model-agnostic approaches we apply the same MSM-intggrrte MAE model on synthetically masked (80%) classification datasets and compare against GRU-D. From Table 2 observe that while GRU-D specialized model for IMTS data, the proposed model-agnostic still outperforms it significantly. Please refer to the Appendix for more implementation details.
Comparing MissTSM with Latent ODE on ETTh2. As discussed above, specialized IMTS models cannot be easily adapted to a different task. To analyze this further, we adapt the Latent ODE model (with ODE-RNN encoder) for a long-term forecasting problem and compare it against our model-agnostic approach. We consider a simple setup with 336 context length and 96 prediction length under MCAR masking with varying fractions. From Table 3 we that Latent ODE struggles to perform long-sequence modeling, with

---

Published in Transactions on Machine Learning Research (01/2026)
Table Performance comparison on P19 and P12. Best bold, second- best underlined.
P19
P12
Methods
AUROC
AUPRC
AUROC
AUPRC
GRU-D
88.7 1.2
56.2 2.3
79.6 0.6
41.7 1.8
ODE-RNN
87.1
62.6 8.2
78.8
37.4 2.6
SeFT
84.0
49.3 0.5
78.1
35.9
m mAAND
82.9
32.2 1.5
85.3 0.3
49.3
Raindrop
87.6 2.7
61.1 1.4
82.0 0.6
42.7 1.7
MTGNN
88.5 1.0
55.8 1.5
82.1 1.5
41.8 2.1
MissTSM
88.8 1.3
56.5 1.2
82.2 0.5
43.8 1.1
Table Comparing (F1 scores) MissTSM approach against GRU-D for classification datasets.
Table 3: Comparing (MSE values) MissTSM with Latent ODE adapted for forecasting
Dataset GRU-D
MissTSM
Fraction
Latent ODE
MissTSM
Epilepsy
6.52%
64.9%
60%
4.25
0.243
Gesture
3.16%
55.70%
70%
3.181
0.250
EMG
2.78%
59.45%
80%
2.543
0.264
90%
2.624
0.316
significantly high MSE values. Moreover, ODE-based methods incur considerable computational costs, which grow even more pronounced for long-term modcling.
5.2.2 Analyzing Agnel-Agnostic Nature of MissTSM.
To further analyze model-agnostic capability of the proposed approach we integrate MisSTSM with other MTS models like PatchTST and i'Tranffrmer. Tables and 5 show competitive performance of Miss integrated with PatchTST, revealing potential for plugging MissTSM with advanced MTS models for improvod performance on downstream tasks even in the presence of missing values with minimal change to the MTS model architecture. Please refer to appendix for additional results.
Table 4: MSE (meanara) for PatchTST with Mis- sTSM under 60% masking.
Table 5: MSE (meanata) for PatchTST with Mis- sTSM under 70% masking.
Horizon PatchTST PatchTST PatchTST Window
Horizon PatchTST PatchTST PatchTST Window
Dataset
Dataset
MissTSM
SAITS
Spline
MissTSM
SAITS
+ Spline
96
0.3174.004
0.503p.013
0.324..013
96
0.322
0.548
0.317
192
0.377
0.512p.0m
0.399p.017
192
0.382
0.561
0.380
ETTh2
ETTh2
336
0.380p.am
0.410p 012
0.4311.005
336
0.384..006
0.468n.058
0.372
720
0.514p.0s
0.411g.002
0.436p.017
720
0.621g.025
0.497n.0ss
0.4199.m1
96
0.202o.uus
0.322p.045
0.169
96
0.2133.006
0.405g.079
0.177a.0m9
192
0.261
0.359
0.227
192
0.266
0.447a.0s8
0.2366.009
ETTm2
ETTm2
336
0.313p 001
0.408p 043
0.285p.001
336
0.315 13115..00 004
0.494g.008
0.2933.om
720
0.420p 027
0.4599.023 033
0.3766.001
720
0.432 025
0.529b.002
0.3866.009
96
0.206p
0.169
0.270b.110
96
0.2044.014
0.1660.013
0.262.. 122
192
0.276p.u27
0.212
0.287o.0sm
0.249h.030
0.2077.008
0.279q.0w2
Weather
West her
336
0.309 024
0.263
0.3255.065
336
0.304..08 .206
0.257..007
0.317a.075
720
0.340p 003
0.333
0.3911.060
720
0.358
0.329
0.383p.071
5.2.3
Impact on Real-world Datasets
We observed that existing benchmark datasets used for forecasting represent certain level of seasonality which makes it easier for imputation-based models to show adequate performance. However, in many
10

---

Published in Transactions on Machine Learning Research (01/2026)
Lake Mendota
Lake FCR
0.8
Masked MSE
0.4
0.0
28
35
42
28
35
42
Prediction Horizon
Prediction Horizon
PatchTST MissTSM
PatchTST + Spline
KIZ
PatchTST + SAITS
iTransformer + MissTSM
iTransformer Spline
iTransformer SAITS
Figure 8: Forecasting performance comparison on Lake datasets across different prediction horizon windows. Lower MSE (y-axis) is better
real-world datasets such as those encountered in ecology, there are complex forms of lemporal structure in the data beyond simple seasonality. We compare the performance of MissTSM integrated with two MTS models, Trransformer and PatchTST, on two Lake Datasets: Falling Creeks Reservoir and Mendota. Figure masked MSE MSE computed only on observed points comparing MissTSM against imputation-based baselines. Competitive performance shown by MissTSM both the real-world missing datasets further motivates the idea of imputation-free and model-agnostic approaches for IMTS modeling.
S reports
5.3
Ablations
ETTh2, T=720, Periodic
ETTh2, T=720, MCAR
0.50
0.45
MSE
0.40
0.35
0.6
0.7
0.8
0.9
0.6
0.7
0.8
0.9
Missing Value Probability
Missing Value Probability
MAE Splinc
-As
MAE SAITS
MissTSM
Figure 9: Ablations of MissTSM with and without TFA+ layer on Forecasting datasets.
EMG
100
Epllepsy
70
Gesture
No0 AITS
NAC Sol
95
65
PAC
60
60
F1 spore
Score
50
75
40
70
30
0.2
0.4
0.6
0.5
55
0.2
0.4
0.5
0.8
35
0.2
0.4
0.6
0.8
Fractions
Fractions
Fractions
Figure 10: Ablations of Mis:TSM with and without the TF1+MFAA layer the classification tasks.
11

---

Published in Transactions on Machine Learning Research (01/2026)
In the ablation experiments, we evaluate the impact of integrating the MissTSM layer. We compare MAE with MissTSM against standard MAE (without MissTSM), using spline and SATTS imputation additional baselines. The goal here is to understand the additional value of adding the MissTSM layer instead of
modcling on imputed data. For forecasting (Fig. 9 and classification (Fig. [0), MissTSM consistently improves performance. In forecasting, MissTSMM-AF outperforms MAE variants, while classification, is consistently comparable or superior across all three datasets.
6 Conclusion
We investigate the performance existing IMTS models as well as our proposed MissTSM framework on a variety of datasets and tasks with varying conditions of missing values. We show that imputation-basee frameworks built on simple imputations perform well when the amount of missingness is small or there is periodic structure in the data (c.g., in Weather data) that is casy to approximate. However, impatation-based approaches show poor performance at larger missing value fractions and when missing values have limited periodic patterns (e.g., on the lake datasets). We also show that MissTSM, which is an imputation-fre and model-agnostic framework, demonstrates competitive performance across most datasets, tasks, and settings compared to imputation- based and existing imputation-fre specialized models. We hope our findings could inspire further research into developing flexible, model-agnostic adapters for handling the challenges irregularly-sampled time-series data.
Limitations and Future Directions. (1) A limitation of the MFAA layer is that it doesn't learn the non-linear temporal dynamics and relies on the subsequent transformer encoder blocks to learn the dynamics. Future work can explore modifications of the MFAA layer such that it can jointly learn the cross-channel correlations with the non-linear temporal dynamics. (2) Independent embedding of each lime-feature token can become computationally expensive in high-dimensional multivariate systems.
Broader Impact Statement
Validation requirements and safety considerations when used in healthcare sectors MissTSM is a decision-suppon tool and would require rigorous clinical validation to ensure reliable predictions. We would recommend that the model must be used in an "expert-in-the-loop" system
Guidance on when the method is reliable vs when it may fail The method reliable in challenging scenarios with low periodicity, high missingness and high number of variates. On low variate, highly periodic data, a simpler approach like Spline + Backbone can be more effective
The potential issue of over-trust in imputation-free outputs We agree that this is critical concern. Our recommendation would be to pair MissTSM's predictions with a reliable uncertainty quantification metric. This metric should reflect the sparsity (or quality) of the input data, warning the user when the predictions are of low confidence.
Acknowledgments
This work supported in part by NSF awards IIS-2239328 and DEB-2213550. This manuscript has been authored by UT-Battelle, LLC, under contract DBE-AC05-000R2777 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public wwwprr/wwgoooooouuuubueeeee
Access Plan (https:/
References
Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. Practical approach to asynchronous multivariate time series anomaly detection and localization. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & dala mining, pp. 2485-2484 2021.
12

---

Published in Transactions on Machine Learning Research (01/2026)
Edgar Acuna and Caroline Rodriguez. The treatment of missing values and its effect on classifier accuracy. In Classification, Clustering, and Data Mining Applications: Proceedings of the Meeting of the International Federation of Classification Societies (IFCS), Illinois Institute of Technology, Chicago, 15 18 July 2004, pp. Springer, 2004.
Hyun Ahn, Kyunghee Sun, and Kwanghoon Pio Kim. Comparison of missing data imputation methods in time series forecasting. Computers, Materials & Continua, 70(1):767-77) 2022.
Ralph G Andrzejak, Klaus Lehnertz, Florian Mormann, Christoph Rieke, Peter David, and Christian Elger.
Indications of nonlinear deterministic and fiitee-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state. Physical Review E, 64(())061107, 2001.
Gustavo EAPA Batista, Maria Carolina Monard, et al. A study of k-nearest neighbour as an imputation method. His, (7(251-260):48, 2002.
George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.
Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. Brits: Bidirectional recurrent imputation for time series. Adoances in neural information processing systems, 31, 2018.
Cayelan C Carey and Adrienne Breef-Pilz. Discharge time series for the primary inflow tributary entering falling creck reservoir, vinton, virginia, usa 2013-2022. 2023a.
Cayelan C Carey and Adrienne Breef-Pilz. Time series of high-frequency meteorological data at falling creek reservoir, virginia, usa 2015-2022. 2023b.
Cayelan C Carey, Adrienne Breef-Pilz, Heather L Wander, Beckett Geisler, and George Haynie. Secchi depth data and discrete depth profiles of water temperature, dissolved oxygen, conductivity, specific conductance, photosyuthetic active radiation, redox potential, and ph for beaverdam reservoir, carvins cove reservoir, falling creek reservoir, gatewood reservoir, and spring hollow reservoir in southwestern virginia, usa 2013-2022. 2023a.
Cayelan C Carey, Adrienne Breef-Pilz, and Whitney M Woelmer. Time series of high-frequency sensor data measuring water temperature, dissolved oxygen, pressure, conductivity, specific conductance, total dissolved solids, chlorophyll a, phycocyanin, fluorescent dissolved organic matter, and turbidity at discrete depths in falling creck reservoir, virginia, usa in 2018-2022. 2023b.
Cayclan C Carey, Dexter W Howard, Kathryn K Hoffman, Heather L Wander, Adricence Breef-Pilz, BR Nieder- lehner, George Haynie, Ryan Keverline, Michael Kricheldorf, and Evelyn Tipper. Water chemistry time series for beaverdam reservoir, carvins cove reservoir, falling creek reservoir, gatewood reservoir, and spring hollow reservoir in southwestern virginia, usa 2013-2023. 2024.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pp. 213-229. Springer, 2020.
Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1):6085, 2018.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018
Yuqi Chen, Kan Ren, Yansen Wang, Yuchen Fang, Weiwei Sun, and Dongsheng Li. Contiformer: Contimuous- time transformer for irregular time scries modcling. Advances in Neural Information Processing Systems, 36:47143-4711 2023.
Hao Cheng, Qingsong Wen, Yang Liu, and Liang Sun. Robusttsf: Towards theory and design of robust time series forecasting with anomalies. Xrxio preprint or Xrxi::2402 020002, 2024.
13

---

Published in Transactions on Machine Learning Research (01/2026)
Andrea Cini, Ivan Marisca, and Cesare Alippi. Multivariate time series imputation by graph neural networks. corr abs/2108.00298 (2021). arXie preprint arXiv2108.000290 2021.
Arthur Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series B (methodological), 39(1):1-22, 1977.
Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. Simmtm: A simple pre-training framework for masked time-scries modeling. Advances in Neural Information Processing Systems, 36, 2024.
Wenjie Du, David Côté, and Yan Liu. Saits: Self-attention-ben based imputation for time series. Expert Systems with Applications, 219:119619, 2023.
Vijay Ekambaram, Arindam Jati, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. Tsmixer: Lightweight mlp-mixer model for multivariate time series forecasting. In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, pp. 459 469, 2023.
Vincent Fortuin, Dmitry Baranchuk, Gunnar Rätsch, and Stephan Mandt. Gp-vac: Deep probabilistic time series imputation. In International conference artificial intelligence and stalistics, pp. 1651-1661. PMLR, 2020.
David S Fung. Methods for the estimation of missing values in time series. 2006.
Ary L Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph E Mictus, George B Moody, Chung-Kang Peng, and H Eugene Stanley. Physiobank, physiotoolkit, and physionet: components of new research resource for complex physiologic signals. circulation, 101(23): e215-e220, 2000a.
Ary Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph E Mietus, George B Moody, Chung-Kang Peng, and H Eugene Stanley. Physiobank, physiotoolkit, and physionet: components of new research resource for complex physiologic signals. circulation, 121()
e215-e220, 2000b.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pottern recognition, pp. 16000-16009, 2022.
Max Horn, Michael Moor, Christian Bock, Bastian Rieck, and Karsten Borgwardt. Set functions for time series. In International Conference on Machine Learning, pp. 1353-4363. PMLR, 2020.
Diederik P Kingma.. Adam: A method for stochastic optimization. ar preprint arXiv:1412.6980 2014.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. Tn The 41st international ACM SIGIR conference on research & development in information retrieval, pp. 95 104, 2018
Zhe Li, Zhongwen Rao, Lujia Pan, Pengyun Wang, and Zenglin Xu. Ti-mae: Self-supervised masked time series autoencoders. Xr preprint or 2rXiv:2301.08871, 2023.
Zachary C. Lipton, David Kale, and Randall Wetzel. Directly modeling missing data sequences with rnns: Improved classification of clinical time series. Machine learning for healthcare conference, pp. 253 270, 2016.
Jiayang Liu, Lin Zhong, Jehan Wickramasuriya, and Venu Vasudevan. uwave: Accelerometer-bassel personal- ized gesture recognition and its applications. Pervasive and Mobile Computing, 5(6):657-675, 2009.
Shuai Liu, Xiucheng Li, Gao Cong, Yile Chen, and Yue Jiang. Multivariate time-serics imputation with disen- tangled temporal representations. In The Eleventh International Conference on Learning Representations, 2022.
14

---

Published in Transactions on Machine Learning Research (01/2026)
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. or arXiv preprint vrXiu23300066225 2023.
Yukai Liu, Rose Yu, Stephan Zheng, Eric Zhan, and Yisong Yue. Naomi: Non-autoregressive multiresolution sequence imputation. Advances in neural information processing systems, 32, 2019.
Yicheng Luo, Bowen Zhang, Zhen Liu, and Qianli Ma. Hi-patch: Hierarchical patch gun for irregular multivariate time series. In Porty-second International Conference on Machine Learning, 2025.
John J Magnuson, Stephen R Carpenter, and Emily H Stanley. North Temperate Lakes LTER: High Frequency Data: Meteorological, Dissolved Oxygen, Chlorophyll, Phycocyanin Lake Mendota Buoy 2006 current ver 39. Environmental Data Initiative, 2024. URL
Bonkkageid=kmb--ter-ee Stter-ntl.10 ne9.-.nt112.9.99 129.39
Benjamin M Marlin, David Kale, Robinder G Khemani, and Randall Wetzel. Unsupervised pattern discovery in electronic health care data using probabilistic clustering models. In Proceedings of the 2nd ACM SIGHIT international health informatics symposium, pp. 389-398, 2012.
Sky McKinley and Megan Levine. Cubic spline interpolation. College of the Redwoods, 45(1):1049 1060, 1998.
Andriy Mnih and Russ R Salakhutdinov. Probabilistic matrix factorization. Advances in neural information processing systems, 20, 2007.
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint 17X.422211.14790, 2022a.
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. time series worth 64 words: Long-tery term forecasting with transformers. In The Eleventh International Conference on Learning Representations, 2022b.
Matthew A Reyna, Christopher S Josef, Russell Jeter, Supreeth P Shashikumar, M Brandon Westover, Shamim Nemati, Gari D Clifford, and Ashish Sharma. Early prediction of sepsis from clinical data: the blyssionet/oomputing in cardiology challenge 2019. Crilical care medicine, 48(2):210 217, 2020.
Yulia Rubanova, Ricky TQ Chen, and David K Duvenaud. Latent ordinary differential equations for irregularly-samplee time series. Advances in neural information processing systems, 32, 2019.
Satya Narayan Shukla and Benjamin M. Marlin. A survey on principles, models and methods for learning from irregularly sampled time series. arXiv preprinl arXic:2012.00 00168 (2020, 2020.
Salya Narayan Shukla and Benjamin M Marlin. Multi-time attention networks for irregularly sampled time
series. arXie preprint or 2ri::22001.1033 2021.
Ikaro Silva, George Moody, Daniel J Scott, Leo A Celi, and Roger G Mark. Predicting in-hospital mortality of icu patients: The cerssinct/computtin in cardiology challenge 2012. In 2012 computing in cardiology, pp. 245 248. IEEE, 2012.
P.N. Tan, M. Steinbach, A. Karpatne, and V. Kumar. Introduction to Data Mining. What's New in Computer Science Series. Pearson, 2019. ISBN 9780133128901. URL https:ttboossss.ttttto google www/ com/books? id= [G=_204MCAAAAJJ
Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional seore-based diffusion models for probabilistic time series imputation. Advances in neural informalion processing systems, 34: 24804-24816, 2021a.
Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional basod diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34: 24816, 2021b.
15

---

Published in Transactions on Machine Learning Research (01/2026)
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Weather. https://wwwlagg--om..mgg..g wetterr//eeeee/e.eeeee 2021.
Yuxi Wei, Juntong Peng, Tong He, Chenxin Xu, Jian Zhang, Shirui Pan, and Siheng Chen. Compatible transformer for irregularly sampled multivariate time series. 2023 IEEE International Conference on Data Mining (ICDM), pp. 1409-1414. IEEE, 2023.
Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Cost: Contrastive learning of disentangled seasonal-trend representations for time series forecasting, or Xiv preprint urXav:2202.01572 2022.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, Advances in neural information processing systems, 34, 2021.
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chongqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 753-763, 2020.
Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer: Time series anomaly detection with association discrepancy. arXiv preprint wxXiv:2100002444 2021.
Jinsung Yoon, James Jordon, and Mihaela Schaar. Gain: Missing data imputation using generative adversarial nets. International conference on machine learning, pp. 5689 5698. PMLR, 2018a.
Jinsung Yoon, William R Zame, and Mihaela van der Schaar. Estimating missing data in temporal data streams using multi-directional recurrent neural networks. IEEE Trunsactions on Biomedical Engineering, 66(5):1477-1440, 2018b.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAT conference artificial intelligence, volume 37, pp. 11121-11128, 2023.
Xiang Zhang, Marko Zeman, Theodoros Tsiligkaridis, and Marinka Zitnik. Graph-guided network for irregularly sampled multivariate time series. arXiv preprint arXio:2110.05333 2021.
Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, and Marinka Zitnik. Self-supervised contrastive pre- training for time series via time- frequeryy consistency. Advances neural information processing systems, 35, 2022.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting, In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11106 11106-11115, 2021.
16

---

Published in Transactions on Machine Learning Research (01/2026)
A Dataset Details
A.1 Forecasting
ETT. The ETT (Zhou et al. 2021) dataset captures load and oil temperature data from electricity transformers.
ETTh2 includes 17,420 hourly observations, while ETTm2 comprises 69,680 15-minute observations. Both datasets span two years and contain variales each.
Weather. Weather (Weather 2021 is a 10-minute frequency time-series dataset recorded throughout the year 2020 and consists of 21 metcorological indicators, like humidity, air temperature, etc.
Solar-Energy. Solar-energy (Lai et al. 2018) dataset captures solar power generation data from 137 photovoltaic (PV) plants in the year 2006, with data sampled at resolution of every 10 min, providing rich MTS data. We report results on Solar-Energy dataset in the appendix only, as supplementary forecasting evaluation.
Following Wu et al. (2021) and Lai et al. (2018), we use a train-valldatiion-tes split of 6:2:2 for the ETT and Solar-Energy datasets and 7:1:2 for the Weather datasel.
Ecology Dataset Falling Creek Reservoir (FCR). The FCR dataset is generated by combining data from the following data packages, Carcy ct al. (2023b), published in the Environmental Data Initialive repository. The dataset contains daily median meteorological and water quality observations from FCR (Virginia, USA), collected over a period of 3 years spanning from 2018-08-01 to 2021-12-22. As part of the pre-processing, we removed variables with more than 90% missing values. Specifically, two columns were excluded with the remaining dataset consisting of 1240 time points with an overall missing rate of 53.12% 12%.
(202a 2024 ) Carcy & Piref-PPizz (2023b a); Carcy et al.
Ecology Dataset Mendota. The Mendota dataset contains daily averaged meteorological and water quality observations from Lake Mendota (Dane County, WI, USA). Data were collected using an instrumented buoy at the surface of the water during the ice-free season from July 2006 through November 2023 (Magmuson et al. 2024). The data were cleaned by filtering out flagged values before use. We applied preprocessing and removed variables with more than 90% missing data. This resulted in dropping three columns with 99% missing values, and one column with 98.1% missing values. The resulting dataset contains 6321 time points, with overall missing rate of 50.07%.
For both datasets, used split of 70% for training, 10% for validation, and 20% for lesting.
A.2 Classification
Epilepsy (Andrzejak et al. 2001) contains univariate brainwaves (single-channel EEG) sampled from 500 subjects (with 11,500 samples in total), with each sample classified as having epilepsy or not (binary classification).
Epilepsy.
Gesture. Gesture Liu et al. 2009) dataset consists of 560 samples, each having 3 variates (corresponding to the accelerometer data) and each sample corresponding to one of the 8 hand gestures (or classes)
EMG. EMG /Goldberger et al. 2000at dataset contains 163 EMG (Electromyograppy)) samples corresponding to 3-classes of muscular diseases.
We make use of the following readily available data splits (train, validation, test) for each of the datascts: Epilepsy 60 (30 samples per each class)/20 (10 samples per each class)/11420 (Train/Val Val/Testt) Gesture 320/20/120 (Train/ (Train/Val/Ter Vea/TTett EMG 122 41/41 Airain/Val/TTesti
Physio-Net Dataset: 2012 Mortality Prediction Challenge Silva et al. 2012) contains 12k multivariate clinical time-series samples that were collected from patients in ICU. The time-series contains 37 variables, such as temperature, heart rate, blood pressure, etc. that can vary depending on the type of patient. Each of the samples are recorded during the first 48 hours of admission in ICU. PhysioNet has a high degree of 80% missing values. We follow the experimental setup in Du et al. (2023), and split the dataset into 80%, 10% and 10% train/val/uen ttest split.
17

---

Published in Transactions on Machine Learning Research (01/2026)
P12 Dataset Derived from the PhysioNet 2012 Challenge, the P12 GGoldbrgerr et al. 2000b) dataset includes multivariate TCU time-series data for 11 18898 patients after filtering out 12 invalid entries. Each patient record contains up to 48 hours of sensor data across 36 physiological variables (excluding weight), plus a 9-dimensional static vector with demographic attributes such as age and gender. The binary label indicates ICU length of stay: 53 days (negative) vs. 23 days (positive).
P19 Dataset (Reyna et al. 2020) This dataset is part of the PhysioNet 2019 Sepsis Early Prediction Challenge and contains longitudinal EHR. data from 38,803 ICU patients. Each patient record comprises multivariate time-series data with 34 clinical variables, sampled at irregular intervals, alongside static vector containing demographic and admission information (e.g., age, gender, ICU type, hospital-to-ICU delay, ICU length of stay). A binary label indicates whether the patient will develop sepsis within the next 6 hours.
A.3 Synthetic Masked Data Generation
We generated masks by randomly selecting data points across all variates and time-steps, assigning them as missing with a likelihood determined by p (masking fraction). The selected data points were then removed, effectively simulating missing values at random. For multiple runs, we created multiple such versions of the synthetic datasets and compared all baseline methods and MissTSM the same datasets.
Random Masking:
Periodic Masking: We use sine curve to generate the masking periodicity with given phase and frequency values for different features. Specifically, the time-dependent periodic probability of seeing missing values is defined as p(t)= = + a11 - p2 + 6), where, 6 and v are randomly chosen across the feature space, a is a scale factor, and p is an offset term. We vary from low to high values to get different fractions of periodic missing values in the data. To implement this masking stralegy, each feature in the dataset was assigned a unique frequency, randomly selected from the range [0.2, 0.8). This was done reduce bias and increase randomness in periodicity across the feature space. Additionally, the phase shift was chosen randomly from the range [0, 2r.. This was applied to each feature to offseet the sinusoidal function over time. Like frequency, the phase value was different. for different features. This generated a periodic pattern for the likelihood of missing data.
B Baselines
Regular MTS Methods. iTrransormer LLu et al. 2023 TST Nie et al. 2022b). DLinear ZZng
et al. 2023), Autoformer (Wu et al., 2021), SimMTM (Dong et al. 2024). MAE (He et al, 2022) (adapted
for Time series similar to Ti-MAE (Li et al. 2023) ). We use the default parameters reported for each models in their official code for the respective datasets. We use the parameters reported in Table D and Table for
the MAE implementation, code for Ti-MAE (Li et al. 2023) is not publicly available.
Imputation Methods oorder spline imputation (McKinley & Levine 1998 N-Nearest Neighbor (kNN)
(Tan et al.) 2019) (with neighbors=10), SAITS (Du et al. 2023). and BRITS (Cao et al. 2018). For SAITS, we use the following setting in our experiments n 1_ embedding. saatldiinndi=2256. num heads=4. For BRITS, we use hidden hiidee_ddieenneeee dimension=64. We follow the official code for implementation of these models.
IMTS Methods GRU-D {Che al. 2018), Latent ODE (Rubanova et al. 2019). We used the availa ble GRU-D re-implementation as the official code was not available. For Latent ODE we used their official code.
The other baselines used are the following, SeFT (Horn el al. 2020), mnTAN Shukla. & Marlin 2021
Raindrop (Zhang et al. 2021). MTGNN Wu et al. 2020)
C Implementation Details
The experiments have been implemented in PyTorch using NVIDIA TITAN 24 GB GPU. The baselines have been implemented following their official code and configurations. In the default implementation code of MissTSM with MAE, we integrate the MissTSM layer directly within the backbone without the lincar projection to the original input shape. We consider Mean Squared Error (MSE) as the metric for time-series forecasting and F1-score, AUROC, AUPRC for the classification tasks. We generate five different versions of
18

---

Published in Transactions on Machine Learning Research (01/2026)
synthetic data and report the average metric from these 5 runs along with their standard deviation, wherever applicable.
C.1 hyper-parameter Details
The hyperparameters for MissTSM selected after integration with the base model, with embedding dimensions searched in the range [4, 128] and the number of attention heads in the range [1, 8). For the base models, use the default configurations for the iTransformer (Liu et al. 2023) and PatchTST (Nie et al.) 2022b) models. For the MAE base model, we start with the same set of hyper-parameterd SimMTM paper (scc Table 6 as initialization, and then search for the best learning rate in factors of 10, encoder/decoder layers in the range [2, 4], number heads in the range [2,16] and embedding dimensions the range [4, 128]. Note that we only perform and use the same hyper-parameters for all experiments involving the dataset, such as different missing value probabilities. Table 7 captures the MissTSM parameters across datasets. Note that, for classification datasets (EMG, Epilepsy, Gesture), we use the same set of parameters with qeim Table presents the sensitivity analysis of the MissTSMMMMA model with respect to key hyperparameters (mcasured using MSE).
reported in the
tuning on 100% (or fully observed) data,
= kdin = 32 and Whead 16.
Table 6: Hyperparameters for Forecasting and Classification Tasks used for base MAE model
Hyperparameter ETTh2 ETTm2
Weather Classification
Encoder Layers
#
Decoder Layers
2
2
2
2
# Encoder Heads
16
# Decoder Heads
1
1
1
16
Encoder Embed Dim
Decoder Embed Dim
32
32
32
32
Table 7: Hyperparameters of the MissTSM layer
Base Model
Qdim
kdim
Udim
Thhaads
Dataset
PatchTST
128
iTrrassormer
128
ETTH2 & ETTm2
MAE
8
8
8
8
PatchTST
128
8
8
1
iTransformer
128
Weather
MAE
64
32
32
8
MAE
32
32
32
16
EMG, Epilepsy, Gesture
MAE
64
PhysioNet, P12, P19
PatchTST
i'Transformer
128
FCR & Mendota
C.2 Forecasting experiments
The models were trained with the MSE loss, using the Adam (Kingma 2014] optimizer with learning rate of 1c-3 during pre-training for 50 epochs and a learning ate of 1c-4 during fine-tuning with an carly stopping counter of 3 epochs. Batch size was set to 16. All the reported missing data experiment results are obtained over trials (5 different. masked versions). During fine-tuning for different Prediction lengths (96, 192, 336, 720), we used the same pre-trained encoder and added linear layer at the top of the encoder.
For our experiments, we use the Latent ODE model with the following configuration. The number of training iterations is set to 50, using dataset size of 10,000 samples and batch size of 32. The latent state dimension
19

---

Published in Transactions on Machine Learning Research (01/2026)
Table 8: Hyper-panee flyperparaameter sensitivity of MAE model integrated with MissTSM on ETTh2 with 70% Masking Fraction, MCAR. Best results shown in bold, second best underlined. Hyper- parameer settings used in the paper are italicized.
Enc.
Heads
Enc. Layers
Enc.
Embed
Dim
4
8
2
3
8
16
32
96
0.246
0.245
0.246
0.249
0.243
0.244
0.243
0.248
0.285
192
0.261
0.273
0.266
0.287
0.267
0.271
0.267
0.266
0.340
336
0.312
0.279
0.310
0.294
0.392
0.307
0.392
0.316
0.369
720 0.326 0.346
0.333
0.351
0.323
0.355
0.323
0.338 0.446
Dec. Heads
Dec.
Layers
Dec.
Embed Dim
8
3
32
96 0.261 0.248
0.252
0.276 0.242
0.248
0,250
0.259
0.243
192
0.276 0.267
0.272
0.266
0.268
0.268
0.257
0.272
0.267
336
0.319
0.392
0.301
0.262
0.352
0.271
0.289
0.266
0.392
720
0.324 0.323
0.330
0.323
0.364
0.341
0.353
0.384
0.323
is 16, and the dataset used is ETTh2. We operate in forecasting mode, using a sequence length of 336 and a prediction horizon of 96. The recognition model (ODE/RNN) has 30 dimensions and 3 layers, while the generative ODE function has layers. Each ODE function layer has 300 units, and the GRU update networks have 100 units.
For the lake dataset, both the i'ransformer and PatchTST models were trained using a sequence length of 21 and prediction lengths of 28, 35, and 42. The illranffrmeer model was trained with a batch size of 16 and a learning rate of 0.0001. The model uses 2 encoder layers, 8 attention heads, and a model dimension of 128. The PStchTST model was trained with the same batch size and learning rate. Its encoder consisted of 3 layers, attention heads, and a model dimension of 16, with a dropout rate of 0.3. The patch length was sot. to 16, with a stride of 8.
C.3 Classification experiments
The models were trained using the Adam (Kingma pre-training and Cross-Entropp loss during fine-tuning. During fine-tuning, we plugged a 64-D linear layer at the top of the pre-trained encoder. We pre-trained and fine-tuned for 100 epochs. For the GRU-D experiments, we use the available re- eeimplemennation of the code, with hidden size as 16. For the PhysioNet experiments, we follow the same evaluation sctup as proposed in Du et al. (2023). The basclines imputation models use a simple RNN model as classification model on top of the imputed data.
2014) optimizer, with MSE as the loss function during
C.4 Discussion on Design Choices
We discuss some of the design choices behind the MissTSM layer.
Why cross-attention instead of a self-attention at each timestep? In our current formulation, we perform a masked cross-attention at every timestep using a single learnable query. While we agree that the conventional approach of applying self-attention can capture better cross-variate interactions, we specifically docided to Iearn a single query voctor to roduce the computational cost of lcarning attention weights (we only need N x 1 attention weights, instead of a full N N matrix). While we empirically show that our simple attention architecture still captures valuable information across variates, work can be easily extended in future works to model all variate-pairs interactions, e.g., by introducing hierarchical or grouped aggregation mechanism that first models variate dependencies for single time-step and then performs a global pooling across time-steps.
Why a single time-independent query? The goal of the MFAA layer is to learn a feature representation at each time-step using the observed variates while ignoring the missing variates. It does so by using a learnable query. The goal of learning this query vector be equated to the task of extracting the best.
20

---

Published in Transactions on Machine Learning Research (01/2026)
representation from the observed features every timestep. Since this task can be assumed to be invariant of time (I.e., the same feature extractor can be used to learn representations of observed variates at any time-step), we consider learning a single time-independent query. Alternatively, if we used time-dependent queries, it would require learning T separate query vectors, which would increase the number of learnable parameters. Moreover, by learning a different query for every lime-step, the model would be forced to learn T different rules, potentially making it susceptible to overfitting artifacts at specific time indices.
D Complexity and Scalability Analysis
D.1 Theoretical Analysis of Computational Costs
We analyze the computational and memory complexity of the proposed MissTSM module. Let L represent the context. length, N the number of variates (features), d, the embedding dimension, h, the number of heads, and Q, K, V are the Query, Key and Value matrices
Time Complexity The initial feature embedding (i.e. the Time-feature independent embed- ding TFT) processes each of the N variates at each of the L time steps, mapping them to d-dimensional space. This operation has a time complexity of (I.Nd).. The MFAA layer performs cross-attention across variates at each time-step, where single query attends to N embedded variates. The complexity is domi- nated by the linear projections to Q, and or V, ((d)) and ((Nd)) re- spectively. It results in a complexity of O(LNd). The final projection layer maps the d-dimensional out- put to the variate embedding space, adding (LdNN to the time com-
Table 9: Time, memory, and parameter complexity of MissTSM. Here, L is the context length, N the number of variates, h, number of heads and d the embedding dimension. dn and di are hidden and latent dimensions of mTAND, respectively. G and 4innr corresponds to the number groups within the DMSA blocks and number of layers within each block in SAITS
Model
Time
Memory
Params
MissTSM
O(LNd)
O(LNd)
Old + Nd)
O(LA(d + H(N + d..)) O(L(N + d, + + LLdd.)
Old + hNd,
ITTAND
+hL4)
hdl d,d))
SimMTM
O(NTAE(T+ d))
O + hT?)
O + I2'd))
O[L(Nd+N' L) +Glmeer L'd)
O(L(N + + +Glnmer (Ld (Ld+ hLL))
Oliiner df N L))
SAITS
plexity, overall resulting in a total time complexity of MisSTSM as OLLAND + dN)), = O(LNE)
Parameter and Memory Complexity The main learnable parameters lie in the aross-attention layer (approximately 4d parameters from the Q, K, V and the output projection layers), the TFI embedding (2Nd parameters from the weight and bias matrices), and the final linear layer (dN parameters). Thus, the total parameter count is + 4d + 2Nd + 2d, yielding an overall paramcter complexity of O(d Nd) quadratic in d and linear in N
The largest intermediate tensors during the forward pass are the embedded features (O(LNd) and the Q,K.V tensors (O(LNd)), and the attention scores (O(LN)). Hence, the peak activation memory scales as O(Ldd), which is linear in N.
Table summarizes the time, memory and parameter complexity of MissTSM, compared against three basclines- mTAND (IMTS method), SimMTM (MTS method) and SAITS (imputation method). The scalability of MisSTSM is primarily governed by three factors embedding dimension, number of variates and sequence length. In contrast, the baselines exhibit complexity that depends multiple interacting components within each dimension (time, space, parameters), making their scalability less straightforward.
D.2 Empirical Analysis of Computational Costs
To evaluate the efficiency of different IMTS pipelines, we benchmark their training runtime, inference throughput, and peak GPU memory usage. Table 10 presents these metrics across models.
21

---

Published in Transactions on Machine Learning Research (01/2026)
Table 10:
Computational cost comparison between different pipelines and ours for IMTS modeling. We consider a classification task on the PhysioNet P19 dataset
Training Time (avg. epoch, sec) (samples/sec)
Inference Throughput
Peak Active Tensor
Learnable
Pipeline
AUROC
Memory (train, GB)
Params
13.2770.77 0.279b.13 13.3560.cm
1,373, 576
236
SAITS
438 438.13547.783
0.74
77.40
1,136,812
mTAND
1.95
9593.36121.44
1.6
142.370
82.9
Raindrop
17.65lo.554
964.75es.a1
1.4
1,947,804
87.6
MissTSM IAA
0.322a.077
19371.58:s15.0z
1.866
365,506
88.8
The training time reported as the average per epoch over five trials. Inference throughput, is computed as the total number test samples divided by the total inference time. For two-stage IMTS pipeline, the total inference time is the sum across both the stages. We report the peak GPU memory allocated using PyTorch. For two-stage pipeline, we report the maximum GPU memory usage across both stages. All the experiments were conducted Ubuntu 18.04, Python 3.9, PyTorch 1.12 (CUDA 12.8) and single NVIDIA TITAN 24GB GPU, using& constant batch size of 64 for all models and with mixed precision disabled
The empirical results demonstrate that the MissTSM-MAN framework is approximately 42x faster per training epoch and achieves over 44x higher inference throughput than the two-stage SAITS+MAN pipeline. However, the peak memory of the MSA (1.866 GB) slightly higher than the SAITS+MÄE pipeline (0.74 GB), which can be further improved using more efficient code optimization in future works.
D.3 MSs Scalability
We conducted an empirical study of runtime and memory usage N (number of variates) increases, analyze the scalability of our T N lokenization in the TFI layer. We generated multiple synthetic datasets of approx- imately 56k observations with 70% MCAR missing fraction, varying N in 10, 20, 50, 100, 200, 500, 800, 1000. For this analysis, we considered MissTSM integrated with an iTrrasformer backbone and compare its perfor- mance with a baseline approach of using SAITS imputation with iTransformer. We report the peak GPU memory and average time per epoch (Figure 12 as well as per forward pass (Figure 11.. All the results are summarized in Table [1 Figure 12 and iTransformer model, and Figurc 11 we compare the metrics per forward pass between iTransformer and the MissTSM layer. The sequence length T and the embedding dimension d were held constant to 336 and 16 respectively, to isolate the effect of N. Note that the reported baseline results reflect only the runtime and memory requirements of the iTransformer model itself, excluding the additional computational cost. associated with the SAITS imputation process, for case of analysis.
we compare the per epoch metrics between Miss MiisTSSM-TTaaassooeeee
Table 11: Scalability analysis with respect to N.
Forward Pass (per epoch)
Forward + Backward Pass (per epoch)
Time (s)
Peak GPU (GB)
Time (s)
Peak GPU (GB)
Miss
iTrans
MissTSM
iTrans
Tras
iTrans
-irans
iTTrns
2.095
1.784
0.080
0.064
22.416
13.495
0.271
0.177
20
2.954
1.911
0.149
0.114
27.312
16.275
0.410
0.236
50
6.004
3.163
0.359
0.275
49.903
18.514
0.847
0.427
100
10.227
12.487
0.711
0.576
96.618
48.387
2.764
1.925
200
20.140
24.188
1.410
1,309
174.575
69.040
3.313
1.636
52.917
62.014
3.515
4.536
456.296
185.875
9.182
5.034
800
83.468
113.758
5.613
9.309
1204.278
335.082
17.028
10.403
From Figure 11
we observe that during the forward pass, the MissTSM layer scales almost linearly with N, consistent with the theoretical complexity in Table However, Figure T2 shows that the runtime and
22

---

Published in Transactions on Machine Learning Research (01/2026)
(a) Forward rd pass Time vs Number Variates
(b) GPU Memory (per forwar rd pass) vs Number of Variates
Forward
MissTSM Forward Time
MissTSM Forward Peak GPU Mem.
Paak GPU Memory (GB)
(seconds)
400
100
700
Number of Variates (N)
Number of Variates (N)
Figure 11: Time and peak GPU memory scaling w.r.t. N, observed per forward pass
(a) Forward Backward (per Epoch) time vs Number of Variates
(b) GPU Memory vs Number of Variates
1200
ITranssMis MissTSM Epoch Time
ITrans-MisTSSI GPU Memory
ITrans Epoch Time
iTtans Peak GPU Memory
1000
Peak GPU Memory (GB)
12.5
(seconds)
800
10.0
7.5
5.0
200
255
1.00
200
300
600
500
000
700
800
100
200
4.00
500
700
800
Number of Variates (N)
Number of Variates (N)
Figure 12: Time and peak GPU memory scaling w.r.t. N, observed per epoch
memory for the combined forward and backward pass scale linearly only up to certain point. Till N = 100, TTransformer and MussTT---TTanffoo the scaling curves in Figurcs I and [2] and the forward versus we find that the backward pass dominates both runtime and peak GPU memory, leading to a sharp increase runtime after N 100. On single 24 GB NVIDIA RTX, training MissTSM becomes limiting N 1000. However, note that N > 100 represents an extreme setup involving very large number of variates, while for common practical ranges of N (e.g., N 50), the computational overhead of backpropagation with MissTSM remains minimal.
models scale almost in parallel, but diverges after that. Comparing
backward metrics for iTransformer,
D.4 Imputation Time and Error Propagation Case Study
We conduct case study examine the tradc-offs between models for IMTS modeling in terms of com- putational complexity and efficiency, and to investigate potential correlations between imputation errors and downstream performance. We consider classification task the Epilepsy dataset that is 80% masked under MCAR. Spline and SAITS are the imputation techniques and SimMTM is the MTS model used. We report the total modeling time as the sum of imputation time and the time-series model training time. The experiments are conducted on NVIDIA TITAN 24 GB GPUs.
In Table 12 we observe that, while SImMTM integrated with SAITS achioves a higher F1 score than Spline, but the total imputation time for SAITS is significantly higher than that Spline. This additional computational overhead substantially increases the overall modeling time. Moreover, SATTS has approximately 1.3 million
23

---

Published in Transactions on Machine Learning Research (01/2026)
Table 12: Total computational cost comparison between MisTTSM-MAE (model-agnostic and imputation and SimMTM (MTS model) for an IMTS classification task
Model
Imp. Model Imp. Time (sec) MTS Model Train Time (sec) Total Time (sec)
F1 Score
SimMTM
SAITS
949 1 42.9
397.59 1 2.64
13416.99 1 45.54
61.0 1 9.20
Spline
8.74 1 0.38
397.59 1 2.64 346.8 7.32
406.33 1 3.02
59.16 1 3.67
MissTTM
N/A
N/A
346.8 + 7.32
64.93 + 4.57
Weather Error Plot
2.0
Imputation Error
1.5
1.0
MissTSM
SAITS
0.5
Spline
BRITS
kNN
0.0
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Forecasting Error for Horizon 720
Figure 13: Imputation error vs Forecasting error across trials for missing fractions, 0.6, 0.7, 0.8, 0.9
trainable parameters, further increasing the overall model complexity of the time-series modeling task. This highlights a potential trade-off between imputation efficiency and model complexity. With our proposed model-agnostic approach, we do not have the extra overhead of imputation complexity. Interestingly, the MissTSMMinnlegratee model leads to superior performance.
Figure 13 captures the propagation of imputation errors and forecasting errors for Weather dataset 720 forecast horizon. It indicates that there is an overall positive correlation between the imputation error and forecasting errors, thereby demonstrating propagation of the impulation errors into the downstream time-series models.
E Additional Results
E.1
Embedding of 1D data and the Effect of Varying Embedding Sizes
To understand the usefulness of mapping 1D data to multi-dimennionall data in TFI embedding, present (in Table 13) an ablation comparing performances on ETTh2 with and without using high-dimensional projections in TFI Embedding under the no missing value scenario. Projecting 1D scalars independently to higher-dimensional vectors may look wasteful at the time of initialization of TFI Embedding, when the context of time and variates are not incorporated. However, it during the cross-attention stage (using MFAA layer or later using the Transformer encoder block) that we can leverage the high-dimensional embeddings to store richer representations bringing in the context of time and variate in which every data point resides.
24

---

Published in Transactions on Machine Learning Research (01/2026)
that TFI embedding with 8-dimensional vectors consistently outperform the ablation with 1D representations, empirically demonstrating the importance of high-dimensional projections in our proposed framework.
From Table 13
Table 13: Effect of TFI Embedding with embedding size=1 and embedding size=8 under masking scenario. Dalaset=ETThh
Horizon
TFI Embedding with embedding size 1
TFI Embedding with embedding
size = 8
96
0.283 0.048
0.245 0.011
192
0.285 0.078
0.260 0.023
336
0.319 0.023
0.300 0.016
720
0.378 0.022
0.334 0.032
E.2 Effectiveness of TFI Embedding
To understand the standalone performance of the MFAA layer, we conducted a simple ablation study where we fixed the weights of the TFI embedding layers to a constant value of with a bias of 0, to simulate a scenario of no TFI embedding. This corresponds using every time-feature combination as a scalar value rather than a multidimensionall embedding. For this study, we considered the ETTh2 dataset across 60-90% MCAR missing fractions. The results summarized in Table 14 We observe that MissTSM suffers
Table 14: Performance comparison with and without TFI embedding across fractions and horizons
Mask Ratio
Horizon
W/o TFI
With TFI
96
0.53
0.24
192
0.47
0.26
60%
336
0.49
0.28
720
0.83
0.33
96
0.42
0.25
192
0.46
0.27
70%
336
0.58
0.30
720
1.73
0.35
96
0.47
0.26
192
0.68
0.28
80%
336
1.02
0.29
720
0.69
0.37
96
0.50
0.32
192
0.68
0.35
90%
336
1.05
0.32
720
0.66
0.35
significantly on removing the TFI embeddings, suggesting the importance of using learnable multidimensional embeddings for every dime-featree combination rather than scalar values.
E.3 Additional Forecasting Dataset
To further strengthen our understanding of the behaviors of regular IMTS models under the two-step pipeline, and to explore the potential of model-agnostic imputation-Fi approaches, we conduct forecasting experiments another dataset, Solar-Fnergy (Tai et al. 2018), using the MCAR masking scheme two missing data
25

---

Published in Transactions on Machine Learning Research (01/2026)
14 summarizes the results. As expected, the performance of traditional MTS methods degrades with increasing missingness, reflecting their reliance on separate imputation strategies. While IMTS methods provide a practical solution for partially observed time series, they tend to underperform compared to regular MTS models in long-range forocasting scenarios. This is where we believe the gap could potentially be addressed by imputation-fress model-agnostic approaches such as MisTTSM.
fractions 60 and 70%. Figure
Masking Fraction 60%
Masking Fraction 70%
0.25
0.20
MSE
0.15
0.10
0.05
0.00
96
192
336
720
96
192
336
720
Horizon
Horizon
iTransformer + SAITS
MissTSM-MAE
Figure 14: Results the Solar-Energy dataset with MCAR masking across four horizons at 60% and 70% masking fractions. Lower MSE is better
E.4
Analyzing the Impact of Forecast Horizon
ETTm2 60% MCAR
0.9
Diterdonmerr
TRansformar
0.55
0.7
0.8
DLinear
SimMTM
MisiTEM
Autiloemer
0.8
NisATSM
Dlinear
SimMITM
Autatormrr
0.50
0.45
0.40
DLinear
SinMITR
MIsTSM
Autaformr
0.35
0.5
0.6
0.4
0.25
0.3
0.20
720
0.15
Forecast Herizons
(a) ETTH2 60% MCAR
(b) ETTm2 60% MCAR
(c) Weather 60% MCAR
Figure 15: Forecasting performance with horizon length T (96, 192,336 720} and fixed lookback length S 336. Baseline models are imputed with SAITS.
To analyze the performance of regular MTS models under IMTS setting with increasing forecasting horizon T = {96, 192, 336, 720), we consider 60% MCAR masking scheme with five regular MTS models and Miss inL,,rated-MAEE as shown in Figure 15 As expected, a common trend across all three datasets is that the forecasting MSE increases with the forecasting horizon for all methods. While this pattern generally holds for most MTS models, few exhibit irregular behavior, with noticcably higher variability their predictions. This could suggest sensitivity to the quality of the underlying imputation model need for greater robustness in certain MTS approaches.
26

---

Published in Transactions on Machine Learning Research (01/2026)
Table 15: Performance (MAE) of iTransformer with different imputation methods (LOCF, Spline, SAITS)
compared to MissTSM on the ETTm2 dataset across varying missing fractions and forecasting horizons.
Mask Ratio
iT
iT
IT
MisSTSM
+
LOCF
+ Spline
+ SAITS
60%
0.18 0.002
0.18 0.002
0.37 0.08
0.22 0.005
70%
0.18 0.002
0.18 0.006
0.46 0.11
0.23 0.006
96
80%
0.19
0.002
0.18 0.008
0.64 0.14
0.23 0.012
90%
0.21
0.002
0.20
0.009
0.86 0.22
0.24 0.034
60%
0.25 0.003
0.24
0.005
0.43 b.08
0.25 0.009
192
70%
0.26
0.002
0.25
0.007
0.54 0.14
0.26 0.018
80%
0.25
0.002
0.25
0.005
0.69 0.13
0.27 0.013
90%
0.27
0,003
0.27
0.02
0.93 0.19
0.27 0.010
60%
0.30
0.006
0.29
0.002
0.48 0.08
0.29 0.019
336
70%
0.30
0.002
0.30
0.009
0.61 0.17
0.30 0.012
80%
0.31
0.004
0.29
0.006
0.73 0.13
0.28 0.010
90%
0.32
0.005
0.32
0.008
1.04 0.18
0.32 0.043
60%
0.38
0.002
0.38
0.008
0.54 0.07
0.31
0.014
720
70%
0.38
0.002
0.38
0.008
0.64 0.14
0.31
0.013
80%
0.38
0.003
0.39
0.009
0.80 D.1G
0.32
0.020
90%
0.39
0.006
0.40 0.01
1.00 0.21
0.34
0.024
E.5 Simpler Imputation Methods
We investigate the role of simple imputation strategics, such as Last Observation Carried Forward (LOCF), within the IMTS modeling pipeline for regular MTS methods. As shown in Table 15 on seasonal datasets such as ETTm2, such straightforward methods at times outperform more complex alternatives.
While this might raise questions about the relative benefits of complex imputation methods such as SAITS, more important consideration is whether trends observed on simpler datasets reliably translate to real-world scenarios.
Table 16: Performance on the PhysioNet dataset (higher is better).
The SAITS paper (Du et al., 2023) addresses this by comparing SAITS and LOCF using a standard baseline (RNN classifier) on the PhysioNet dataset, challenging real-world benchmark with over 80% missing values, and demonstrates the clear advantage of SAITS (see Table 16
Method
LOCF
SAITS
MissTSM
Accuracy (%)
39.5
42.7
57.84
Notably, our proposed model-agnostic approach exhibits highly competitive performance at shorter horizons and shows even greater promise as the forecasting horizon increases.
E.6 Hyper-paramette Tuning of Baseline Imputation Models
To strengthen the the claim on performance gain of MissTSM, we carry out rigorous hyper-parameter tuming of the SAITS imputation model on the ETTh2 dataset under both 70% periodic and 70% MCAR missing scenarios. The hyperparameters and their ranges considered were n lllaers: [1,2,3), a d_ffn = (64, 128, 256] and (d_ m_modell, n heads) = (64, 1), (128, 2), (256, 4) resulting in 27 total combinations. We report the results of iTransformer and PatchTST in the table below. We first trained SAITS on the 27 combinations. Then we trained the downstream model (iTransformer and PatchTST) on all the imputed files to obtain 27 different error metrics. The best configuration, based on test set results, is selected and reported in the following table. The results of the tuning experiment are presented in Table
The best Transformer results are obtained with the following SAITS configurations
MCAR: n_ m_layers=1, a_ d_model=64, a_ ffn=1118 n heads=1
Periodic: a_layers=1, d_ 1_model=128, a 64, n, heads=2
27

---

Published in Transactions on Machine Learning Research (01/2026)
Table 17: Comparison MissTTM with i'Trassfrmee, and PStchTST trained on HP Tuned SAITS model.
Under MCAR and Periodic settings.
i'Transformer (SAITS)
iTransformer (HP tuned SAITS)
PatchTST
PStchTST
Horizon
MissTSM
(SAITS)
(HP tuned SAITS)
MCAR
96
0.243
0.492
0.465
0.440
0.389
192
0.259
0.545
0.513
0.484
0.434
336
0.283
0.579
0.546
0.434
0.387
720
0.329
0.540
0.494
0.436
0.378
Periodic
96
0.246
0.691
0.456
0.581
0.389
192
0.263
0.715
0.511
0.620
0.427
336
0.301
0.763
0.552
0.592
0.394
720
0.353
0.773
0.484
0.644
0.408
The best PatchTST results are obtained with the following SAITS configurations
MCAR: n_ a_layers=3, d_ m_model=256, d_ d_ffn=128, n n_heads=4
Periodic: n_layers-j) d_model-2500 d_ffn-64, n_heads-4
Tuning the SATTS model significantly improved the downstream performance (e.g., MSE dropping from 0.773 to 0.484 for T-720 under periodic masking) for both MCAR and periodic masking strategies. This comparison demonstrates that. However, while the imputation-based baselines do improve with specific tuning, the MissTSM framework still consistently demonstrate competitive performance compared to the newly-optimized basclines across all horizons and on both masking patterns.
ETTh2
ETTm2
Weather
Methods
MissTSM
SImNTM_S SSP
SimMTM_SS
SimMIM SA
PetchTST_5P
SimMTM
PatchTST SA
Autoformer 50
PatchTST SP
AutoFormer
DLinear: SP
PatchTST
DLinear SA
Iiransformer:
AutoFormer SP
iaanstorme SA
AutoFormer SA
DLinear
DLinear SA
iTransformer SP
Masking
Transformer SA
Perfodic
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.2 MSE
MSE
MSE
Figure 16: Comparison of different masking methods: MCAR Periodic Masking for ETTh2, ETTm2 and Weather datasets. SA stands for SAITS and SP stands for Spline. Horizon Window: 96. Lower MSE is better
E.7 Additional Results Masking strategies
Figures 16 and 18 show additional results on comparison between random and periodic masking strategics for different horizon lengths 96, 192, 336. Please refer to the main paper for horizon length 720.
28

---

Published in Transactions on Machine Learning Research (01/2026)
ETTh2
ETTm2
Weather
Methods
MissTSM
MisTTMM
SiMMIM SP
SimMTM SP
SimMTM, SA
PeachTST_S1
SimMTM_SA SA
PetchTST_ SA
AutoFormer SP
PatchTST_Sr SP
AutoFormer SA
PatchTST_SA SA
DLineer SA
mansformer
AutoFormer_st
Itranffomme: 5A
AutoFormer_si SA
DLinear_St SP
DLinear_SM
iTranformer_st _P
Masking
Transformer_so SA
MCAR
Perfodic
0.0
0.5
0.0
0.5 MSE
0.0
0.2
0.4
MSE
MSE
Figure 17: Comparison of different masking methods: MCAR Periodic Masking for ETTh2, ETTm2 and Weather datasets. SA stands for SAITS and SP stands for Spline. Horizon Window: 192. Lower MSE is better
ETTh2
Weather
MissTSM
MisSTM
SimMTM_S SP
SimMTM_S
PetchTST_ _
SimMTM_SA
PetchTST_ SA
AutoFormer_
PatchTST_SP
Autoformer SA
DLinear SP
PatchTST_SA
DLinear
Transformer:
AutoFormer SP
Transforme: SA
AutoFormer_5t
DLinear_S5 SP
DLinear_SA
ITranstormer SP
Masking
ilransfomer SA
MCAR
Periodic
0.0
0.5
1.0
0.0
0.2
0.4
MSE
MSE
MSE
Figure 18: Comparison of different masking methods: MCAR vs. Periodic Masking for ETTh2, ETTm2 and Weather datasets. SA stands for SAITS and SP stands for Spline. Horizon Window: 336. Lower MSE is better
E.8
Model Agnostic Experiments
E.8.1
Transformer based backbones
Tables 18
presents performance of MissTSM integrated with iTransformer, under 60% and 70% masking
fractions.
29

---

Published in Transactions on Machine Learning Research (01/2026)
Table 18: MSE (meansta) for iTTranffommer integrated with MissTSM vs with different imputation methods: SAITS and Spline for 60% masking.
Table 19: MSE (mean)st)) for iTTransformer integrated with MissTSM against different imputation methods: SAITS and Spline under 70% masking.
Horizon Trransfommrr Trannformmrr iTransformer Spline
Horizon
iTransformee iTransformer SAITS
ITransformer + Spline
Dataset Window
Dataset
MisTTMM
+ SATTS
Window
+ MisTTSM
96
0.350k.com
0.6455o.09
0.325p.015
96
0.3866.020
0. 0.2002.054
0.3466.015
192
0.482o.c27
0.612
0.340
192
0.530 ugs
0.680
0.430p.011
ETT12
ETTh2
336
0.4044.000
0.526
0.431a.005
336
0.400
0 0.573p.061
0. 0.4663.026
720
0.490o.o12
0.505p.0o6
0.4366.017
720
0.5
0.575p
0.4755.020
0.20% cos
0.361o.0e8
0.176
0 0.226
0. 0.18
0.1833.007
ETTm2
192
0.26%o.oue
0.4100.064
0.240p.om
ETT m2
192
0.3000
0.555
0.2444.008 0os
336
0.325c.ous
0.458o.00s
0.290o.on2
336
0. n.498
0. 0.5922.167
0.2922.007
720
1,0800 188
0.521 OGG
0.3855.007
720
0. 9.731
0.650
0.390 007
0. 19.193..02
0.1944.004
0.2022.00L
96
0.1 0.1888.012
0.177
0.3655.266
192
0.240 ood
0.230
0.240
192
0 0,251.mm
0.218g
0.314p.189
Weatthr
Weather
336
0. 0.29%o.co4
0.2800po01
0.2910.001
336
0.300p.o02
0.273a.02
0.3166..00
720
0.36-4o.oo
0.348
0.3577.00L
720
0.3668..00
0.3440
0.421o.112
E.8.2 Non-transffrrmer-asss backbones
Tables 20 and 21 presents performance of MissTSM integrated with non-transformer backbone LSTM, under 60% and 70% MCAR and Periodic masking fractions.
Table 20: MSE (mean)ad) for LSTM integrated with MissTSM vs with different imputation methods: SAITS and Spline for 60% masking.
Table 21: MSE (mean for LSTM integrated with MissTSM vs with different imputation methods: SAITS and Spline for 70% masking,
Dataset
Window
Horizon
MisSTSM
LSTM
LSTM SAITS
LSTM Spline
Dataset
Horizon Window
MisSTSM
LSTM
+ SAITS
LSTM
+ Spline
LSTM
0.43.04
0.53
0.39p
96
0.46
0.51o.02
0.42,.02
192
0.50g.02
0.590.03
0.44p.004
0.52o.m1
0.60o.u6
0.45
ETTH2
ETTh2
336
0.40p.01
0.51o.09
0 0.42p.02
0.40g.03
0.460.02
0.42o.0 01
720
0.44p.0
0.56p..1
0.50g.os
720
0.46g.01
0.48o
0.56g.00
96
0.33
0.41g.00
0.220.01
96
0.44o.07
0.47o.m
0.22p.01
192
0.46g.02
0.42o.07
0.270.01
192
0.51..02
0.540.04
0.30
ETTm2
ETTm2
336
0.46n.01
0.490.07
0.33p.002
0.60o.a2
0.53o.n4
0.36
720
0.49g.06
0.47o 02
0.440.00
1.63..77
0.630.15
0.45%..1
0.34g.005
0.27
0.35g.02
96
0.38p
0.26
0.28p
Weather
336
192
0.38p.015
0.32p.03
0.34g.02
0.42 0.420.01
0.37
0.38
Weather
336
192
0.330
0.40o.02
0.32p.04
0.38g.00
0.346
0.44o.
720
0.39
0.49, os
0.45p.00
0.470.01
0.48.
0.49p
Similar transformer-based backbones, MissTSM, with a non-transformer backbone, consistently achieves competitive performance across datasets and forecasting horizons. We see that Spline imputed model generally performs well on the ETT datasets with fewer variates (N = 7) but is not effective on the Weather dataset, which has 3x many variates (N 21)
F Analysis of impact of frequency and phase parameters
In this section, we provide additional details regarding an ablation we conducted to understand the impact of frequency and phase parameters under periodic masking. Given the varying frequency and phase for each feature, modify the intervals of both to assess their impact the results. Dataset=FTThh, Fraction=90%
30

---

Published in Transactions on Machine Learning Research (01/2026)
Table 22: Effect (MSE) of sampling from different frequency intervals. Original refers to original periodic masking MSE. The best results are in bold and second- best are italicized
Horizon
Original
High Frequency
Low
Frequency
96
0.268 0.015
0.281 0.028
0.285 0.023
192
0.295 0.029
0.301 0.037
0.316 0.049
336
0.319 0.018
0.308 0.014
0.307 0.011
720
0.356 0.031
0.339 0.043
0.351 0.058
Table 23: Effect (MSE) of sampling from different phase intervals. Original refers to the original periodic masking MSE. The best results are in bold and second-best are italicized
Horizon
Original
(+) Half Cycle
(-) Half Cycle
96
0.268 0.015
0.287 0.037
0.293 0.040
192
0.295 0.029
0.309 0.050
0.313 0.057
336
0.319 0.018
0.316
0.022
0.311 0.013
720
0.356 0.031
0.343
0.035
0.340 0.040
Case 1. With the phase interval held constant, we lower the frequency range and examine two intervals: one in the high frequency region ([0.6, 0.9]) and one the low frequency region ([0.1, 0.3]). The performance comparison between these new strategies and the original configuration shown in Table 22
We observe that with reduced frequency range, for both high and low frequency intervals, the performance improves as the prediction window increases.
Case 2. Following similar approach as Case 1, we keep the frequency interval constant and lower the range of phase values. We examine the following intervals: the positive half-cycle [0, x and the negative half-cycle (r, 2x]. Table 23 presents the results of this ablation
We observe a similar pattern here well, with the performance improving as the prediction window increases when we sample from either the positive negative cycle.
As shown in the tables above, frequency and phase values clearly impact model performance. The new strategies reduce frequency or phase-related randomness among the variates of the dataset, resulting in more consistent values. This appears to enhance the model's ability in long-tow term forecasting.
G
Case Studies
G.1
Comparison against noise-agnostic methods
In this section, we compare the proposed MissTSM approach against robust methods like RobustTSF CCheng et al. 2024) which performs time-series modeling with noisy data, where missing values and anomalics are considered a type of noise. While Cheng al. (2024) is an impactful work, there are some key differences compared to MissTSM, such as, (a) RobustTSF treats missing anomalous values anomalies and filters them out during training, while MissTTMM aim to learn latent representations from all observed time-feature pairs without dropping any timc-points, fully leveraging available data; (b) While Robust relies on assumptions like smooth trend filtering and Dirac-weighted anomaly scores with user-sel thresholds, which may not hold in practice, MissTSM avoids such assumptions, depending purely on supervision from observed data; (c) RobustTSF assumes no missing values during testing and only imputes zeros if missing. MissTSM robustly handles missing values both during training and testing: (d) While RobustTSF is mainly univariate, MSs is explicitly designed for multivariate time-series, capturing dependencies across variates.
31

---

Published in Transactions on Machine Learning Research (01/2026)
Table 24 compares the forecasting performance between MissTSM with MAE RobustTSF applied to MTS model, Autoformer, on the ETThh2 dataset. across 60% to 90% MCAR masking fractions.
Table 24: Performance comparison between RobustTSSF + Autoformer and MissTSM across fractions and horizons
RobustTSF Autoformer
Fraction Horizon
MissTSM
96
1.5 0,054
0.24
1.45 0.065
0.26 0.002
60%
336
1.34 0.082
0.28 0.009
720
1.33 0.024
0.33 0.012
96
1.81 0.117
0.25 0.009
192
1.76 0.066
0.27 0.013
70%
336
1.76 0.049
0.30 0.008
720
1.69 0.032
0.35 0.010
0.084
0.26 0.025
192
2.23 0.091
0.28 0.021
80%
336
2.2 0,085
0.29 0,029
720
2.11 0.041
0.37 0.056
96
2.72 0.077
0.32 0.029
192
2.74 0.038
0.35 0.072
90%
336
2.77 0.024
0.32 0.014
720
2.67
0.35
G.2 Adapting MissTSM to Anomaly Detection
Table 25:
Comparison between RobustTSF Autoformer and Anomaly Transformer + MissTSM across
forecasting horizons, on the PSM dataset
RobustTSF + Autoformer (MSE)
Anomaly Transformer +
Horizon
MissTSM (MSE)
96
0.349
0.360
192
0.524
0.465
336
0.731
0.603
720
0.958
0.815
While handling anomalies is not the primary focus of our work, this case study was carried out explore the possibility of our framework to be adapted to handle other data quality issues such anomalies. The experiment carried out the Pooled Server Metrics (PSM) (Abdulaal et al. 2021) dataset using the following sctup- we first use an off-thc-shcl anomaly detection method, Anomaly Transformer (Xu et al., 2021), to identify anomalies in the input datasel, which are then treated as missing values in the training of MissTSM. We call this combined framework: Anomaly Transformer + MissTSM. We use 134K samples for training the Anomaly Transformer and use the remaining 84K samples for training and evaluating two forecasting models, namely, Anomaly Transformer + MissTSM, and RobustTSF (with AutoFormer backbone). We specifically split the 84K samples into train, test, and validation sets using 7:2:1 ratio. Once trained, evaluate (see Table 25) both MissTSM and RobustTSF on the test set ignoring anomalous points in the forecasting window (using ground truth labels of anomalics).
H Visualization
Visualizations on Lake (see Figure 19 and benchmark datasets (see Figures
21
22
23
24
32

---

Published in Transactions on Machine Learning Research (01/2026)
MissISM
SAITS
Spline
[0.1 BAB
Fredktion Groand Trush
Ground Trath
Crouse T2ua
Time
Time
Time
MissTSM
SAITS
Spline
Tredcias
avg phyco
-1
Tredutas
Time
MissTSM
SAITS
Spline
avg par below
Predekee Tha
Prodktio Oreand Truts
Predecss Ceoaue Touls
Time
Time
Time
MissTSM
SAITS
Spline
Gra
avg. do raw
6000 800
1200
200
600
1000 12000
2000
600
1000 1200
Figure 19: Comparison of T+1 step-ahead predictions Lake Mendota dataset using three methods for handling missing data: MissTSM, SAITS, and Spline (shown across columns). Blue lines represent model predictions, while red stars indicate the ground truth values. This comparison is based on forecasts generated by the iTransformer model, configured with a prediction length of 28 and context length of 21. Each row corresponds to a different feature. The selected features include avg_rel. rel _um,, representing average relative humidity with a total 49.5% missing values; avg wrrg_pyyc_____ __u,, which captures average phycocyanin fluorescence and has 48.2% missingness; avg_par _s__pa__below, measuring photosynthetically active radiation below the water surface with 72.8% of its values missing; and avg do raw, representing average dissolved oxygen with 48.4 48 missing data.
33

---

Published in Transactions on Machine Learning Research (01/2026)
(a) PStchTST MisSTSM
(b) PatchTST + SAITS
(c)
PStchTST Spline
Figure 20: Qualitative comparison ETTm2 datasct at 70% missing fraction and horizon window 336.
(a) PatchTST integrated with Mis-
(b) PatchTST imputed with SATTS
(c) PatchTST imputed with Spline
sTSM
Figure 21: Qualitative comparison ETTm2 dataset at 70% missing fraction and horizon window-96
(a) PatchTST + MissTSM
(b) PatchTST + SAITS
(c) PatchTST + Spline
Figure 22: Qualitative comparison on Weather dataset at 60% missing fraction and horizon window = 336.
smay Malern
(a) iTransformer integrated with Mis- (b) iTransformer imputed with SAITS sTSM
(c) iTransformer imputed with Spline
Figure 23: Qualitative comparison Weather datasct 60% missing fraction and horizon window=96.
(a) iTransformer integrated with Mis- (b) iTransformer imputed with SAITS
(c) iTransformer imputed with Spline
3TSM
Figure 24: Qualitative comparison on Weather dataset at 70% missing fraction and horizon window=96.
34

---

Published in Transactions on Machine Learning Research (01/2026)
Summary of Results
Table 26 strategies
presents the full set of results of all models over different masking fractions, horizons, and masking
Table 26: Comparing forecasting performance of baseline methods using mean squared error (MSE) as the evaluation metric under no masking, MCAR masking, and periodic masking. For every dataset, we consider multiple forecasting horizons, T (96, 192, 336, 720}. Results are color-coded as Best , Second best. We report the mean and standard deviations (in brackets) across random sampling of the masking schemes (fraction 70%). Subscript SP refer to Spline and SA refer to SAITS
ETThh
ETTm2
Weather
Avg
96
192
336
720
96
192
336
720
96
192
336
720
Rank
MisETSM
0.255
0,234
0.316
0.305
0.183
0.209
0.261
0.311
0.164
0.210
0.254
0.324
1.9
No Masking
SimMTM
0.295
0.356
0.375
0.404
0.172
0.223
0.282
0.374
0.163
0. 0.203
0.255
0.326
2.9
PatchTST
0.274
0.338
0.350
0.378
0.164
0.220
0.367
0.196
0.249
0.319
1.7
AutoFormer
0.501
0.516
0.565
0,462
0.352
0.337
0.474
0.306
0.431
0.437
0.414
5.9
DLinear
0,288
0.383
0.605
0.168
0.224
0.414
0.175
0.219
0.265
0.323
4.1
iTransformer
0.304
0.392
0.425
0.415
0.176
0.246
0.289
0.379
0.163
0.203
0,256
0.326
4.5
MisXTSM
0.243
0.250
0.2833
b.329
0:224
0.253
0.293amm
0.316
0. 0.234
0,281
0.322
2.7
SinMTMar
0.309
0.372px00
0.
0.118
0.185
0:243pm03
0,298
0.3
0.203
0.212pmm
0,2844mm
0.386
5.0
MCAR Masking
MiAMTMaA
0.457am
0.510gas
0.
0.503ums
0.472
0.287
0 0.3200mss
0.3422mm
0.413
0.187a,002
0.240puaa
0.2800
0.386
6.2
PatchTSTsp sT
0.336
0.345
0.300
0.1699
0 0.228
0.288 own
0.378
0.1833.000
0. 0.2266.000
0.277oom
0.3399
2.1
PatchTSTTAA
0.100mme 0.484a
0.1334mm
0.32 0.136
0.362
0.410mom
0. 0.162.oor
0.175x.00e
0.211aoco
0,2642mm
0.335
1.6
AutaFormersp
0.550km
0.628
0.525
0.550g.43
0.2800g.0m
0
0.390
0.3600ums
0 0.4775o.ca
0.321
0.413ums
0.508
0.467
8.9
AutoFormersA
0.767
0.526
0.550ums
0.449
0.6100..12
0. 0.850
Guast
1. L.045g.00
0.353
0.4130us
0.4744ure
0.504q.040
10.2
DLineurgp
on
0.400la,o1s
0.445pmm
a.607
0.458c.10
0.228u00
0.3002
0.53Ilo.14
0.205g
0.241oo07
0,2822
0.3733
6.5
DLinearsA
0.154amma 0.51
0.512ams
0.
0.050s.ea
0.330k
0.305
0.427ams
0.538
0.190k.o0
0. 0,2333um
0,276
0.333
6.8
Transforrmegg
0.313mmm
0.3944,014
0.436um:
0.4295.00
0.1
0.243
0.253
0.3844.co
0.197a.006
0. 0.2600me77
0.315
0.340
4.9
Traasforruera
0.545ppous
0.379
0.540
0.369Accom
0.432aass
0,482
0.541lo.mm
0.191a.c0e
0.2288
0.273
0.3488
7.7
MiscTSM
0.246ams 0,283
0.301amm
0.353.com
0.2277.00
0.249
0.282am
0.3377.ccmm
0.2 0.212a.com
0. 2565us
0.313
0.379
4.1
SinMTTMP
0.373.122 0.489
0.496
0.510
0.192c.m
0.247am09
0.301
0.300
0.1822.004
0. 242488008
0.291amm
0.3444.000
4.7
Periodic Masking
SinMTMaa
0m 0.686
0.681a
g.667
am
0.409pa9t
0.436ium
0.503
0.178
4.21
0,261amm
4.35
6.0
PatchTSTTp
0.328
0,389
0.381am
0.426
0.231amm
0.28%
0.381o
0.18Lo.0o
0. 227220
0,267am0
0.346
2.4
PatchTSTsA
0,581am 0.620
0.592
0.644
0,423
0.457am1z
0.4933um
0.527occ
0.17
0.2122mm
0,2633m0
0.33Haua1
5.2
Autnformergp
0.482 0.686
0.621
0.546
0.329%.000
0.315mmm
0.398
0.4566
0.333a
0.387o
0.406
0.458
7.5
AutoformergA
1.415amr 0,810
1.364mm
mm (.8
1.303,
0.933
1.7888
0.8009
0.335
0.3877mmr
0.4355ma
0.467
10.8
DLinearge
0.346
0.4455,,00 0.477anm
0.649
0.327c.ss
0
0.230huma
0.305
0.4733.co
0.215xuuus
0. 0.244-ums
0.284amm
0.339
5.0
DLincursA
0.605 ws 0.674 0.725511s
1.91
D.447oom
0.475m013
0.5233noz
0.626
0.190k..00
0. 0.2333000
0.276amm
D.3333
7.4
Traasforrerrg
0.358amm 0. 03135 0.1885ms
0.197
0.180
0.215um0
0,296ium
0.384
0.197a.00
0.233
0,288
0.351paam
4.2
TransformersA
0.691a sa mm 0.763n
vsa 0,773
0.512.am
0.378uma2
0,6622us
0.680
0.194
O. mm
0.2744m00
0.3
8.2
35


Reviews:
[{"review_id": "LUYn1G7gDq", "reviewer": "Reviewer_AS76", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes MissTSM, which is a model-agnostic and imputation-free layer for handling irregularly sampled multivariate time series. It combines Time-Feature Independent (TFI) embeddings, which treat each observed scalar as a token with 2D positional encoding, and a Missing Feature-Aware Attention (MFAA) mechanism that aggregates observed features at each time step via masked cross-attention. Integrated into transformer backbones, MissTSM achieves strong performance across forecasting and classification tasks, especially under high missingness (>70%).\n\nKey strengths: elegant and practical design, broad empirical validation, strong robustness to missing data, model-agnostic applicability\n\nKey weaknesses: potential information bottleneck from feature aggregation, limited scalability analysis, modest novelty of the attention mechanism", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims are well-supported through experiments across multiple datasets, masking schemes, and backbone models. They consistently demonstrate MissTSM’s robustness and competitiveness against both imputation-based and specialized IMTS methods. Results and ablations are clearly presented and reproducible.", "audience_yn": "Yes", "audience_explanation": "Researchers in time-series modeling, healthcare analytics, and machine learning for irregular data would find this work valuable, for a model-agnostic way to handle missing or irregularly sampled data.", "requested_changes": "1. Include an analysis of runtime and memory usage as $N$ increases, highlighting when the $T \\times N$ tokenization becomes computationally limiting.\n\n2. Expand the discussion on how compressing all variates into a single latent vector $L_t$ may constrain feature interaction modeling and suggest potential extensions.\n\n3. It seems that the contribution lies in applying masked cross-attention to irregular time-series data rather than introducing a new attention mechanism. This should be emphasized in the paper.", "broader_impact_concerns": "No major ethical or societal concerns are apparent.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "XXmwODtgmL", "reviewer": "Reviewer_aMoN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes MissTSM, a model-agnostic and imputation-free framework for Irregularly-Sampled Multivariate Time-Series (IMTS) modeling.\nMissTSM introduces two main components: (1) a Time-Feature Independent (TFI) embedding that independently encodes each observed (time, feature) pair without imputing missing values, and (2) a Missing Feature-Aware Attention (MFAA) mechanism that computes masked attention over observed features to obtain latent representations robust to missingness.\nThis design enables existing time-series models—such as masked autoencoders or Transformers—to handle irregularly sampled data directly.\nEmpirical results demonstrate competitive performance across diverse datasets and tasks, highlighting the framework’s generality and adaptability.\n\nKey strengths:\n* Provides a clear, modular solution to imputation dependency and model-specific design issues in IMTS.\n* Achieves competitive results on both forecasting and classification benchmarks.\n* The design is conceptually simple and compatible with existing Transformer-based models.\n\nKey weaknesses:\n* The approach does not fully resolve long-term temporal dependency modeling, since MFAA focuses on feature-level aggregation rather than temporal reasoning.\n* The computational and memory cost remains significant due to dense embedding and attention operations.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The empirical evidence is generally convincing and supports the authors’ main claims. In comparisons with multiple time-series baselines imputed using SAITS (Figure 4), MissTSM consistently achieves the best or near-best performance. For classification tasks (Figure 5), MissTSM attains strong F1 scores and maintains stable performance across varying levels of missing data. The IMTS classification results on the P12 and P19 datasets further underscore its effectiveness, with MissTSM consistently ranking among the top two models. These results collectively demonstrate the framework’s robustness and model-agnostic adaptability across diverse tasks.", "audience_yn": "Yes", "audience_explanation": "The paper tackles a timely problem in irregularly sampled multivariate time-series modeling and imputation. Its proposed MissTSM framework provides a model-agnostic, imputation-free alternative that integrates smoothly with existing Transformer-based architectures. These findings would interest TMLR readers focused on time-series modeling, imputation methods, and representation learning.", "requested_changes": "* Include a runtime and memory analysis comparing MissTSM with key baselines.\n* Provide a clearer explanation or empirical evidence of how MissTSM handles long-term temporal dependencies, especially in forecasting tasks.\n* Improve figure clarity: for bar charts, explicitly indicate whether lower or higher values are better, particularly for metrics other than F1 scores.", "broader_impact_concerns": "No major ethical or societal concerns are apparent. The work focuses on methodological advances in irregular time-series modeling and does not involve sensitive data, human subjects, or deployment scenarios.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "sA0FjDiiPL", "reviewer": "Reviewer_bYD2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the problem of modelling irregularly sampled Multivariate Time Series (IMTS) where different variables may be missing at different time steps. The authors propose MissTSM, a model-agnostic and imputation-free approach that can be integrated as a plug-and-play layer with existing multivariate time series models. The key contributions include:\n1. Time-Feature-Independent (TFI) embedding: A novel embedding scheme that treats each (time-step, variate) combination as an independent token, allowing the model to handle missing values without explicit imputation.\n2. Missing Feature-Aware Attention (MFAA): a masked cross-attention that aggregates only observed features at each time step using a learnable query, then projects back to the original shape before passing to any backbone model.\nThe authors benchmark MissTSM on both forecasting and classification with two synthetic masking schemes, and five real-world datasets. They conclude that MissTSM is competitive, and tend to be more robust when the amount of missingness is large, and when simple periodic structure is absent.\n\nStrengths:\n- The paper clearly explains the limitations of the existing imputation-based approaches, and give strong motivation of the imputation-free methods. The design of TFI + MFAA is clear, simple and well-articulated. The model-agnostic design makes it easy for potential applications on existing frameworks.\n- The algorithm designs and results are supported by a broad range of experiments, covering multiple datasets, various missing value scenarios of both synthetic and real-word, different missing value fractions, and comparisons with both imputation-based and specialised IMTS methods.\nWeakness:\n- Computational cost of TFI may be high - it makes a token per (time, features). The authors acknowledged  this as a limitation but it is not quantified. There is no comparison of training/inference time with baselines. \n- Baseline tuning fairness is discussed but not exhaustively audited (e.g. SAITS appears unable for periodic patterns)\n- On many datasets, MissTSM shows comparable but not significantly better performance than imputation-based methods. The advantages appear mainly when missing fractions are very high or data lacks periodicity. Some real-word results also show mixed performance. This marginal improvement and the potentially high computational cost give raise to concerns on the practicality of the approach.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims are mostly supported by accurate, convincing and clear evidence. Multiple datasets and experimental conditions are used in the experiments, and the trend evidence is consistent in the results, on both synthetic and real-world scenarios. There are also ablation studies supporting the value of the proposed components. \nThe evidence could however be stronger by including appropriate baseline tuning, studying the computation cost, and further experiments with non-Transformer backbone to support the \"model-agnostic\" claim.", "audience_yn": "Yes", "audience_explanation": "In real-word, IMTS is an important practical problem across many domains. The authors thoroughly address the issue, and therefore motivated the design of MissTSM. The design is elegant, and the model-agnostic nature makes it a useful ML tool rather than single-task bespoke architectures, potentially being helpful on broader scenarios.", "requested_changes": "1. Report compute & Scaling: Add a small section quantifying the computational costs, e.g. peak GPU memory, tokens/sec, inference time, comparing MissTSM with other baselines. The manuscript would benefit from better evaluation on the scalability of MissTSM. (Critical)\n2. Baseline tuning: Add tuning experiments for imputation methods under both MCAR and periodic. This would strengthen the claim one the performance gain of MissTSM with fairer comparisons. (Strengthening) \n3. Broader \"model-agnostic\" demonstrations: add experiments with non-Transformer backbone. A small table showing \"backbone X + MissTSM\" vs \"backbone X + SAITS/Spline\" will make the claim more concrete. (Strengthening)\n4. Further technical explanations: For example, we could add more details on the design choice of MFAA (why single-query? How is it initialised? Why is it time-independent?) And more detailed comparisons with existing frameworks like mTAN. (Strengthening)", "broader_impact_concerns": "This paper does not include a Broader Impact Statement. Given the potential application in sectors like healthcare, environmental monitoring, the manuscript would benefit from a short paragraph addressing:\n- Validation requirements and safety considerations when used in healthcare sectors. \n- Guidance on when the method is reliable vs when it may fail could prevent inappropriate deployment in critical applications.\n- The potential issue of over-trust in imputation-free outputs. What would the prediction based on imputation-free dataset, and potentially with high missingnessm, impact the downstream decisions?", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)