Still Competitive: Revisiting Recurrent Models for Irregular Time Series Prediction
Authors: Ankitkumar Joshi
Milos Hauskrecht
Venue: Accepted by TMLR ()

Abstract: Modeling irregularly sampled multivariate time series is a persistent challenge in domains like healthcare and sensor networks. While recent works have explored a variety of complex learning architectures to solve the prediction problems for irregularly sampled time series, it remains unclear what the true benefits of some of these architectures are, and whether clever modifications of simpler and more efficient RNN-based algorithms are still competitive, i.e. they are on par with or even superior to these methods. In this work, we propose and study GRUwE: Gated Recurrent Unit with Exponential basis functions, that builds upon RNN-based architectures for observations made at irregular times. GRUwE supports both regression-based and event-based predictions in continuous time. GRUwE works by maintaining a Markov state representation of the time series that updates with the arrival of irregular observations. The Markov state update relies on two reset mechanisms: (i) observation-triggered reset to account for the new observation, and (ii) time-triggered reset that relies on learnable exponential decays, to support the predictions in continuous time. Our empirical evaluations across several real-world benchmarks on next-observation and next-event prediction tasks demonstrate that GRUwE can indeed achieve competitive or superior performance compared to the recent state-of-the-art  (SOTA) methods. Thanks to its simplicity, GRUwE offers compelling advantages: it is easy to implement, requires minimal hyper-parameter tuning efforts, and significantly reduces the computational overhead in the online deployment.

Published in Transactions on Machine Learning Research (01/2026)
Still Competitive: Revisiting Recurrent Models for Irregular Time Series Prediction
Ankitkumar Joshi
ahjoshi@pitt.co edu
Deportment of Computer Science University of Pitlsburgh
Milos Hauskrecht
milos @pitt edu
Department Computer Science University of Pittsburgh
Reviewed on OpenReview: openrevveee....ew.....m. net/ net/forum? ? id= YLoZA7702R
Abstract
Modeling irregularly sampled multivariate time series is a persistent. challenge in domains like healthcare and sensor networks. While recent works have explored a variety of complex learning architectures to solve the prediction problems for irregularly sampled time series, remains unclear what the true benefits of some of these architectures are, and whether clever modifications of simpler and more efficient RNN-based algorithms are still competi- live, i.e. they are on par with or even superior to these methods. In this work, we propose and study GRUwE: Gated Recurrent Unit with Exponential basis functions, that builds upon RNN-based architectures for observations made at irregular times. GRUwE supports both regression-based and event-based predictions in continuous time. GRUwE works by maintaining a Markov state representation of the time series that updates with the arrival of irregular observations. The Markov state update relies on two reset mechanisms: (i) observation-triggerd resct to account for the new observation, and (ii) time-triggered reset that relies on learnable exponential decays, support the predictions continuous time. Our empirical evaluations across several real-world benchmarks on next-observation and next-event prediction tasks demonstrate that GRUwE can indeed achieve competitive superior performance compared to the recent state-of-the-art (SOTA) methods. Thanks to its simplicity, GRUwE offers compelling advantages: it is easy to implement, requires min- imal ryper-parameter tuning efforts, and significantly reduces the computational overhead in the online deployment.
1 Introduction
Multivariate time serics and their models are central to understanding and analyzing real-world dynamical systems. While traditional sequence models typically assume regularly spaced observations, real-world data such healthcare records sensor measurements, measurements may occur at uneven time intervals and under varying contextual conditions (e.g., specialized laboratory tests in clinical environments may be performed only during acute episodes). The key challenge is to develop time-series models that can accurately capture dependencies among multiple variables and events occurring at irregular timos. In this work, we study this problem in the context of both time-scries forecasting and event prediction.
often irregularly sampled. Tn such settings, events
A variety of models ranging from classic statistical and modern deep learning frameworks have been devel- oped to support time-series prediction task for irregular settings. Early approaches replaced the irregularly observed data with regularly spaced observations by inferring their values at the regular time points. Classic statistical auto-rogressive models (AR, ARMA, ARIMA) Shhmmway Stoffer (2017) or latent space models, such as, Linear Dynamical Systems (LDS) Kalman (1663) models could then be applied to support predic- tion at future regular times. To support prediction arbitrary future prediction times, various smoothing

---

Published in Transactions on Machine Learning Research (01/2026)
and interpolation methods Liu & Hauskrecht 2015) were developed. More recently, classic time-series mod- els have been gradually replaced with various types of modern neural architectures capable of handling
irregularly sampled observations
et al. (2018). The existing methods include extensions of
RNN approaches Che et al. 2018]; Mei & Eisner (2017). 1 and continue with differential equation approaches
Rubanova et al. (2019); De Brouwer el al. (2019) ; Chen el al. (2018): : |Schirmer et al. (2022); Becker el al.
(2019h, affention-bused sppronches(Shukkl & Marlin (2021]; | Chen et al. (2023), graph-based approaches Zhang
et al. 2022 2024]; Yalavarthi al. (2024) and state-space modeling approaches Smithh et al. (2023); Gu &
Dao
2023
Recent work on irregular multivariate time scries has largely gravitated toward incrcasingly complex archi- tectures such as continuous- time Neural ODEs Rubanova et al. (2019); Kidger et al. (2020); Chen et al. (202), transformers Zuo et al. (2020g Zhang et al. (2020); Shukla & Marlin (021]; Yang et al (2022), and
graph-based models Zhang et al. 2024)) |; Yalavarthi et al. (2024): Li et al. (2025); Luo et al. (2025) While
these approaches introduce powerful modeling mechanisms, they often require intricate training pipelines, exlensive hyperparameler tuning, and incur substantial computational overhead. Moreover, inductive biases folded in the complex architectures may be difficult to interpret or analyze. These factors may limit the benefits of such architectures in many realistic scttings particularly, when training data are limited or deploy- ment enviromments are resource constrained (e.g., bedside monitoring systems or embedded climical devices), where the efficiency of inference is critical for their deployment. This raises a natural question: could corefully designed, time-series prediction models that are rooted in simpler architectures, still achieve comparable or even superior predictive performance ot significantly lower computational cost? We investigate this question by proposing GRUwE (Gated Recurrent Unit with Exponential basis functions), pronounced as 'groovy'; relatively simple yet effective GRU-based architecture that maintains the Markov state reprosentation of the multivariate process that is updated at irregular observation times. GRUwE models the effect of time with the help of learnable exponential basis functions. State updates are affected by both: (i) the arrival of new observations, and (ii) the clapsed time since the previous observation. This dual mechanism allows the model to naturally accommodate influence of past observations and irregular time intervals between them. To make predictions at an arbitrary future time, GRUwE reuses the same exponential basis functions that are used for state updates.
As noted above, GRUWE supports both time series forecasting and event prediction tasks. For event pre- diction, we define a Temporal Point Process (TPP) model with the help of a Conditional Intensity Function (CIF), which characterizes the instantaneous rate of event occurrence. Following prior works by Du et al. 20166: Mei & Eisner (2017), the CIF is modeled by applying a Softplus transformation to the output of a neural network regression model. This formulation allows GRUwE's predictions at arbitrary future times to directly define the event intensity. We compare the GRUwF-bbacdd event prediction model against state- of-the-art CIF-based approaches, including methods based on RNN models Du et al. (2016); Mei & Eisuer
(2017) and attention-bassd models Zhang et al. 2020 Zuo et al. 2020 Yang et al. 2022
The main contributions of this work are:
We propose GRUwE, an irregular time series model, that maintains a compact Markov state representation, defines two reset state update mechanisms to capture dependencies inherent in multivariate time series and supports continuous-time inference.
We perform a comprehensive evaluation of GRUwE against slate-of-the-art baselines on next-observation and next-event prediction tasks across multiple real-world datasets. Our results show that GRUwE per- forms competitively on both tasks, achieves new stato-of-tho-art performance on two datascts for next- observation prediction, and attains the highest overall rank for next-event prediction.
We demonstrate that GRUwE's compact Markov state representation leads to substantial improvements in computational efficiency during ouline deployment.
2 Related Work
In this section, we review existing work on time series prodiction for irregularly sampled multivariate time se- ries data and contrast them to the proposed GRUwE model. Our review is structured along key architectural designs that have been proposed for time-series and event prediction tasks.
2

---

Published in Transactions on Machine Learning Research (01/2026)
Recurrent neural network (RNN) approaches exploit efficient sequential hidden-state updates for prediction, with the hidden state serving as compact, Markovian summary of all past observations. Classical RNN models assume regularly sampled time series and therefore require interpolation schemes to handle irregularly sampled data. Several extensions adapt recurrent models to continuous time by updating the hidden state only at observation times, including Recurrent Marked Temporal Point Processes (RMTPP) Du et al. (2016), GRU-D Che et al. (2018), FullyNN Omi et al. (2019), and the Neural Hawkes Process (NHP) Mei & Eisner (2017). RMTPP extends classic RNN architecture by adding a parametrie conditional intensity function that predicts the distribution of the next event time directly from the RNN's hidden state. NHP, based on LSTM HHocheiter & Schmidhuber (1997) architecture, uses exponential decays to a predicted target cell state to derive a continuous- cell state representation, which is eventually used to represent the intensity function to model Lime-vayying intensities and complex temporal point-process dynamics. GRU-D extends GRU by explicitly modeling missingness and irregular sampling using learnable docay mochanisms that impute both inputs and hidden states based on the time since each variable was last observed. Missing observations are inferred in GRU-D using decay mechanisins that revert to a larget value, typically determined by the mean value of the variable. The GRU-D and GRUwE models are similar in their use of the exponential decay functions for modeling the effect of elapsed time the hidden state. However, GRU-D uses decay functions to infer missing observation values. As a result, it may propagate estimation errors for missing observations in time. In contrast, GRUwE does not interpolate missing observations and uses exponential decay functions to directly model latent variable trajectories both for the state update and prediction module.
Differential equation approaches offer another widely used framework for modeling multivariate time series in continuous time. These methods define latent dynamics with the help of differential equations. Neural ODEs Chen et al. (2018) model temporal evolution using ordinary differential equations parameterized by neural networks. Extensions such as Latent ODE and ODE-RNN Rubanova et al. (2019) combine these dynamics with variational and recurrent architectures to handle irregular observations. A key limitation of Neural ODEs is that their solutions depend on fixed initial conditions that cannot easily adapt to observed data. Neural controlled differential equations Kidger et al. (2020); De Brouwer et al. (2099) address this limitation by allowing continuous modulation based on the input observations. However, most differential equation based approaches require external numerical solvers, significantly increasing training and inference costs. Schirmer et al. (2022); Becker et al. (2019) avoid the use of numerical solvers for contimuous-time dynamics by modeling latent state transitions using linear stochastic differential equations that admit closed- form solutions. In contrast, GRUWE captures latent dynamics through learnable exponential decay functions, roquiring no numerical solvers and imposing no lincarity assumptions on the underlying state cvolution.
Temporal attention methods climinate the need to approximate a latent Markov state representation and instead learn prediction functions lirectly from the observed inputs. Models such as m mnTNND [hukkla & Marlin (2021), THP Zuo et al. (2020), SAHP Zhang al.] (2020), A-NHP Yang et al. (2022) embeds time into fixed-size learnable vectors and learn relationship between irregular observations and prediction timepoints in
continuous time. ContiFormer Chen et al. (2023) combines Neural ODEs with the tartinuuustiiee attention mechanism to model irregular observations.
Graph Neural Networks (GNN) approaches focus on modeling interactions and dependencies among individual time series. T-PatchGNN Zhang et al. (2024) uses a patch-based mechanism on univariate time series to enhance local feature extraction and use graph neural network to learn relationships among dif- ferent time series. GraFITi Yalavarthi et al. (2024) first converts irregular samples to a special bipartite graph structure and cast the prediction problem as an edge weight prediction. HyperLMTSLi et al. (2025) improves the GraFITi method by replacing its fixed bipartite graph with a learnable, heterogeneous hypergraph that jointly models variable time relations, enables richer message passing, and produces more expressive irregular time series representations. Other graph-based temporal [Zhang et al. (2022), spatiotem- poral Marisca et al.] (2022) and diflusion-based TTashro et al. (2021) models have been proposed to address irregularly spaced observations. Overall, the main difference between GRUwE and existing temporal atten- tion and GNN approaches is that GRUwE maintains a Markov state representation that enables efficient update and deployment in real-time settings supporting soquential prediction tasks. Typically the above transformer and graph-based models require one to buffer sequences of past observations to encode a newly
3

---

Published in Transactions on Machine Learning Research (01/2026)
F (IrSr)
Four (9t+1,A7)
X₁
Prediction Module
Prediction
AT
AT
variable value
future obeervations
Module
f--1
State Transition Module
State Transition Module
time
h
hgg
Inter-arrival time
prediction
horizon
X2 m, AT
At-1 Ar
(a) Abstract diagram of the next-observation predic- tion problem for irregular time series. The true contin- hous dynamics function of the variables (vi, 02) (dot- ted line) are unknown; the model only sees their values at irregular time points formulate future predic- tions.
(b) GRUwE's overall architecture. At each ob- servation time point, fx,, m., Mr, Ar} f to update the previous hidden state (he-1 - h.). The prediction module make prediction using the hidden state h, and prediction horizon AT.
are processed
Figure 1: Description of the problem and overview of the model architecture.
arrived observation and reprocess all the past observations for every inference step from scratch which may become computational bottleneck when sequences of past observation become very long. In contrast, GRUwE updates its stale using only the previous state and the newly arrived observation, making it simple, efficient, and practical for real-time deployment.
3 Problem Setting
Our goal is to model D-dimensional multivariate time series with irregularly spaced observations. For clarity, consider the case D with variables ej and U2, whose latent temporal dynamics are illustrated as dotted trajectories in Figure Ia The underlying generative process governing these variables unknown. Observa- tions occur at non-uniform lime points (shown dots), and may not be syuchronized across variables; for instance, may be missing when v2 is observed. At each time step E Z (corresponding to continuous time Ti E RR, we define an input vector E IR. where observed entrics contain values and unobsorved entries are masked. The time elapsed between successive observation times, also known as inter-arrival time, is denoted by Ar = (T, - Tt-1) e R,. To support prediction at any future time, we define a prediction horizon AT E R4, representing how far ahead (relative to Tτ) the prediction should be made. the nezt observation prediction task, the model generates a predictod observation vector &t,TT time Ti + AT, conditioned on the history of all observations up to time step l. In the neal evenl prediction setting, the TPP model oulputs event intensities A(T. + AT), given the event history up to time step t.
4 GRUWE: Gated Recurrent Unit with Exponential Basis Functions
Our goal to define and learn a prediction model f(x1a Tı: AT) that takes a history a D-dimensional multivariate time series Xıæ made times Tia respectively, and predicts future values of time series at time Ti + AT.
Markov State. Since the observation history grows over time, we approximate it using fixed-size state representation H, that summarizes what is known about the process until time step 4. If He accurately summarizes the history of observations, the process becomes Markovian, rendering past observations condi- tionally independent of the future given the current state. This property allows the state to be updated using only the previous state and the newly arrived observation, which greatly simplifies real-time deployment. The state H. of our GRUwE model consists of latent vector h, representing dependencies among different

---

Published in Transactions on Machine Learning Research (01/2026)
time series variables and their values. Thus, He {he} at time vy summarizes all the observations made until time I.
State Transition Module. As new observations arrive in time, the state H, representing the information seen so far must be updated. The transition between the two consecutive GRUwE's states is implemented using two complementary update mechanisms. The first accounts for the time elapsed since the previous observation. We refer to it the time-triggered reset mechanism. The second mechanism incorporates the influence of newly observed input. We refer to it to the dbservation-lriggeeel reset. mechanism. In the following, describe the design of these mechanisms in more detail.
Time-Triee Triggered Reset Mechanism models the effect of clapsed time between two consccutive observa- tions using learnable exponential decay functions. The decay functions lake the previous hidden state and the time difference between previous and new observations to update the state. The decay functions are defined as:
(
max(O, W, AT b,)).
where AT (Te Tc-) denotes the elapsed time vector, with 1 representing vector of ones that broadcasts the scalar time difference across all state dimensions. The learnable parameters W,, and b, control the rate at which each component of the hidden state decays as function of elapsed time. The time-decayed hidden state is then computed as:
1.- = 7(AT) O h,-1,
(1)
where @ denotes element-wise multiplication operation.
Observation--Tigeerdd Reset Mechanism models the influence of a new observation on the state. The input (new observation) at each time-step is defined by an input vector x2 and a mask vector m, indicating if an individual variable i is observed al lime-step l:
{ 1, if Fy,s is observed,
(2)
Ma,i { o. otherwisc.
(3)
Since observations for multivariate times series may arrive at different times, the values missing the specific input vector are masked. That is, at each time-step, the recurrent unit takes two vectors the input: 1) a mask vector (m.) and 2) input vector (x,) where missing values are substituted with zeros:
x
m, ④ Xt.
Computing the hidden state (h,) at Te, given gi m, and x, involves set of updates similar to the ones found in the standard GRU unit:
2: oW + U U + V,m; + bb),
F: = xW,, + U, + V,m, + b,).
h -tanh W x + (A @ g- + Vam, + b),
h: (1 z gg-1A + Z, h
Note that Ws, Us, Vs and bs are learnable parameters of the model. The parameters let us fit the hidden state component and observations that the dependencies among time series most important for the prediction are captured.
Prediction module. Our main goal is to predict future observations in contimuous time given the past observation sequence. At each time slep i, we rely only on the information encoded in the model's state H, to support prediction. To enable flexible prediction across arbitrary future time points, we introduce a prediction horizon AT E R4, which specifies how far ahcad in time (from the current time 7,) the prediction should be made. The prediction module in GRUWE estimates the future output at time т AT by conditioning on the Markov state He at time Tr- It operates in two stages by: (i) projecting the Markov
5

---

Published in Transactions on Machine Learning Research (01/2026)
state to the target time T, + AT, and (fi) generating predicted values at the future time using task-specific output function. For projecting the Markov state to the prediction time one may, in principle, choose arbitrary functions of continuous-time different from the exponential decay functions used to model the effect of elapsed time in the state update (time-triggering reset mechanism). However, for simplicity sake, GRUwE shares the exponential basis functions to model the dynamics of Markov state the prediction module:
(AT) = exp( - max(0, W, +b,)},
(5)
Sr,T = (AT) O h,
(6)
After projecting the state to the prediction time, the prediction module generates the output by applying task-specific output function (-) to the projected state at prediction horizon (AT):
R Fout (B,,
(7)
For the next observation prediction, the output function can be defined by using linear combination of the components of the projected hidden state or a more complex nonlinear combination defined by a Multi- linear combination specified using
Layer Perceptron (MI.P). For simplicity sake, in our experiments,
parameters [ out, bout} J the (:):
X,AT W Wow grAT + bout-
For the next event prediction, we use GRUwE's decayed hidden state (g,, to formulate the Conditional Intensity Function (CIF) for event type at time Tr + AT as:
A(T. + AT he k) = Softplus(w) ST.T + b2).
Here, the linear projection parameters w and by are the parameters of the CIF model specific to event type k. We apply the Softplus non-linearity ensure that the output is positive, as intensity represents the instantaneous rate of event arrivals.
Summary of GRUwE. Our proposed model GRUWE, is latent state transition model that maintains Markov state of the process in continuous time lo summarize the historical events. GRUwE primarily consists of a state transition module and a prediction module. To update its Markov state, the state transition module defines: i) time-triggered reset mochanism to account for the time-clapsed, and ii) dbsorvation-triggere reset mechanism to incorporate the influence the observations. GRUwE's prediction module reuses the exponential basis function (used to model inter-arrival times) to support contimuous-time prediction module. For more clarity, Figure Ib illustrates the process of sequentially feeding observation and mask vectors along with the time elapses for each time step, followed by a state transition module, to update the previous state function of current inputs. Ultimately, prediction module generates prediction at a future time AT with the help of the updated state.
5 Analysis of Exponential Basis Functions
We thcoretically analyze the properties and inductive biases introduced by the proposed exponential basis functions for modeling the temporal dynamics.
Asymptotic Behavior. We analyze the asymptotic behavior of the exponential basis function Y(AT) expí m0 W,AT + b,)} and its effect on the decayed state grAT x(TT) @ h, Consider the component-wise behavior for dimension i as AT - oo:
Case 1: , > 0 (State Reset)
Behavior: ItATA + 0
Explanation: When WyS > O, the argument W AT - by.5 + as AT increases. As result:
max(0, AT + by.i) exp(-co) + 0.
6

---

Published in Transactions on Machine Learning Research (01/2026)
Case 2: W,, 0 (Constant Decay)
Behavior: 9t,T, - exp(- max(0, by,))) h4
Explanation: With zero weight, the decay term simplifies to a constant:
x(AT) = exp( max(0, b,,))-
This results in a constant decay of the hidden state component. The final value depends on the bias term by.
-If S 0: exp(e max(0, by,:)) = exp(0) 1 (no decay).
-If
-If by 0: exp(- max(0, by,i)) = exp(-by,) < (constant decay).
Case 3: W < 0 (No Decay)
Behavior: 9t,AT, + h4.
Explanation: For negative weights, the decay rate AT + b,, -00:
max(0, AT + by.))
exp(O) =
Asymptotic analysis reveals that the proposed exponential basis functions have three operating modes to express the latent temporal dynamics for each hidden component. Based on the learned value of the pa- rameter W,, the decay mechanism can either decay zeros, perform constant decay, or no decay at all. The no decay mode be used to remember the hidden state components indefinitely as time passes. This mechanism can be used to capture very long-term dependencies. The constant-decay mode is useful to cap- ture dependencies that relies counts, perhaps when the inter-arrival times are approximately regularly spaced (since the decay is independent of time-elapsed AT). When W, > O, the exponential basis functions docay with increasing AT, causing the state to contract toward the zero vector (or, oquivalently to its initial state). Intuilively, it means that as more time passes (since the last. observation), the impact of historical arrivals begin to diminish. In the limit, time-triggered reset mechanism forces the state representation reset to its initial state. This leads to an interesting alternative interpretation of the exponential basis func- tions: we are effectively learning a dynamic forgetting process, in which the forget rates are regulated with the help of exponential basis functions.
Lipschitz Continuity. The mapping 7 R4 + 10, 1] is the exponential decay function defined as 7(TT) exp {- max (0, WAT + b)}, where W > and are fixed scalar parameters. Then, y is Lipschitz continuous on R4, with Lipschitz constant:
exp(-b).
Refer to the detailed proof Appendix 1.1 This result establishes that the exponential basis function (AT) used GRUwE Lipschitz continuous with a closed-form constant that depends directly on the decay weight W.y and by. The decay function is guaranteed not change faster than W exp(-6), enabling GRUWE dynamically adjust the smoothness of the exponential decay function and ensuring that the function evolves smoothly over time.
6 Empirical Evaluation
We evaluate the performance of GRUWE both next observation and next event prediction tasks.
Datasets. For next observation prediction, use four irregularly sampled multivariate time series datasets: the United States Historical Climatology Network (USHCN), which contains weather-rclated attributes, and three clinical datasels: PhysioNet, MIMIC-IIISSallI, and MIMIC-III-Laree Among the four datasets, USHCN made irregular by randomly masking 50% of the observed time-points as proposed in Schirmer
ct al. (2022), whereas Physionct, MIMIC-III-Smal,, and MIMIC-II-Large are irregular in nature. For the
event prediction, we use Retweet Zhou et al. (2013) consisting of retweet event sequences; Taxi Whong (2014) consisting of taxi pick-up and drop-off events in five New York city boroughs; Amazon Ni et al. 2019 which
7

---

Published in Transactions on Machine Learning Research (01/2026)
consists of sequences of user provided product reviews; StackOverflow Leskovec & Sosić 2016 consisting of
sequence of awards received by users on a Q&A website, and Taobao Xue et al. (2022) consisting of user
activities on Taobao platform. We provide the description of the datasets in Appendix
Models. For the next observation prediction, we compare the proposed GRUwE models with the SOTA
baseline models: GRU-A, Che et al. 20188), RKN-A, Becker et al. (2019), GRU-D Chung et al.] (2014).
ITAND Shukla & Marlin (2021), CRU Schirmer et al. (2022), C-CRU Schirmer et al. (2022). ODE-
RNN Rubanova et al. (2019). Latent ODE Rubanova et al. (0019) ContiFormer Chen et al. (0023) T-
PatchGNNZhang et al. (022), GraFITi Yalavarthi et al. (2024). In the next event prediction task,
pare GRUwE with Recurrent CIF models: RMTPP Du et al. (2016). NHPMci & Eisner (2017): Attention-
based CIF approaches including THP Zuo et al. (2020), SAHP Zhang et al. (2020), ATTNHP Yang et al.
(002)) and Cumulative CIF approach FullyNN Omi et al. (0019) We include more details on the implemen-
tation of the baselines in the Appendix
Model training. For the next observation prediction task, the models processes a. masked multivariate time series imput, where the future observation are omitted. Models are optimized to output the entire imput sequence after observing partially masked sequence. We use Mean Squared Error (MSE) loss function to optimize the model parameters. Concretely, for sequence with N masked time steps, prediction horizon AT, observation mask vector MT+AT, prediction values ,T and observed values X1,T,, , we optimize the model parameters by minimizing the masked MSE loss:
Lobx
Σ
(0) m HAT
(x.0)
(5) ) x 1,AT) (())
(10)
E HTA
For the event prediction task, conditional intensity function parameterized with the help of GRUwE is optimized by maximizing the log-likelihood of observing sequence s ((Tj, at observation times (TI, 12, T2,,... ...: ) E [0;
L
1 A(L Hi) dl.
Levent (8) log A(ly (Hy
(11)
1=0
where the total intensity function is given by: A(t H4,) = E A(t H).
Evaluation criteria. For the next observation prediction, similar to prior works Schirmer et al. (0022) Yalavarthi et al. (2024); Zhang et al. (2024) we evaluate models on MSE and Mean Absolute Error (MAE) on the masked future values on the sequences in the test split. Using standard evaluation metrics in event prediction literature Yang et al. (2022) Zuo et al. (2020); Zhang et al. (2020), we compare the model quality the test set using: RMSE: Root Mcan Squared Error in predicting the next event arrival time; ER: short for Error Rate, is the percentage of incorrectly predicted next event lype and the log-likelihood of the next observed event. To ensure robust evaluation, we conduct experiments using 5 distinct random seeds and report the mean and standard deviation of the performance metrics.
Hyperparameter search. We conduct model-specific hyperparameter tuning by training various model configurations on the training set and evaluating their performance on the validation set. rrper-parameter ranges for the methods were adjusted to include the optimal hyper-parameter ranges reported by the original authors. For each model, we select the configuration that achieves the lowest validation MSE (for forecasting) and lowest validation LL (for event prediction) and report its test performance. We include more details on the hyperparameter tuning in Appendices to
7 Results and Discussion
We evaluate the proposed GRUwE model on next observation prediction and next event prediction tasks.
Next observation prediction. We evaluate the performance of GRUwE and baselines by predicting the next observation for each variable defining the multivariate time series given the history of past observations. For the Physionct datasot, the model observes the first 24 hours of the total 48-hour period to predict the next observations for all (37) variables. Similarly, for the USHCN dataset, with daily samples over four years, we use the first half to predict the next observation for all 5 variables. For the MIMIC-III-aarge
8

---

Published in Transactions on Machine Learning Research (01/2026)
Model
USHCN
Plysionet
MIMIC-III- Large
Largge
MIMIC-II-Swal
MSE x10 2
MAB x10 2
MSE x10
MAB x 2
MSE x10 2
MAE x10 2
MSE x10 1
MAB x10
&CRU
0.020 0.007
0.455a
1.085
5. 3.255 6:130
1.1911
6.535=
7.5082 0.021
5.9632
TT ND
0.007&
0.194g
0.330z
3.4112
1 1.260s
6.8852
5.9602
5.087s
GRU--1
b:0152 n:01
0.386g
0.6722
0.984s
5.853=
6.621
5.186g
Latent ODE
0.007+
0.127s
0.6762
6.5022
1:2099
6.490=
6.6382
5-547s
ConsiFormer
ODE-RNS
0.0192
0,220=
0,7701
5.519
1 1,229
6.401
5.302:
CRU
0.0302
0.519g
0.507=
5.233g
1 2363
6.7852
6.927s
5.512s
RKN-A,
0.0151 DIBLE
0.367- 077
0,0801
4.554 0.197
1.2921 0.922
6.820
8.2061 6.025
6.185 0.13
CRU-A,
0.0851 0,002
0.717: 0016
0,449: 0.0Ls
4.160: 0.154
1.414: 0.05
7.226
6.563: 6.019
5.617: 0.11
T-PateliNNN
0,0651 0,031
0,9091 0-455
0,3381 0.012
8.269: 0.166
1.2261 0.015
0.989 0.153
6.0641 0.041
4.7771 0.16
GraFITy
0.074:
0.673s u01z
0.233 com
2.781
1.4191
7.4482
4.6922 6.0
4.287 1022
HyperMMTS
0,052: 0.0L3
0.539: 0:22
0,289
2.952:
0,200
00M
00M
4.734: 6.00
4,564
0.055
GRUWE
0.015: 0.013
0.377 0.177
0.201 0.005
2.967: 0.030
0.510 : 0,003
5.278
0.036
4.051 1 0.0L5
4,200 0.022
Table I: Next observation prediction. Comparison of models on next observation prediction on USHCN, Physionet MIMIC-III-Sman and MIMIC-III-Lare datasets. We report the mean and standard deviation of MSF and MAE on five distinct random seeds. 'OOM' refers to out-of-memory GPU error triggered during training.
dataset, models consider values of 506 variables over past 48 hours from a randomly sampled time point in the patient record. The models are compared on predieted values for the next observations made on 363 numerical variables representing vital signs and labs. For the MIMIC-IIISmall dataset, the model observes the first half of the 96 numerical variables to predict their next observation. Table ① presents the predictive performance of all models on the next observation prediction task across the four datasets, evaluated over multiple random seeds. Next we briefly summarize the results on individual datascts below. On the USHCN dataset, ContiFormer, Latent ODE, mTAND are the top performing models in terms of MSE and MAE. On the Physionet dataset, GraFITi, GRUwE and HyperIMTS are the top performing methods. On the MTMIC- Ill-Large dataset, GRUWE achieves the lowest MSE and MAE, outperforming all competing models, with GRU-D ranking second among the baselines. GRU-D uses mean-reverting interpolation scheme for missing inputs, an assumption that likely holds for this setting, which we believe is contributing to its competitive predictive accuracy. We note that MIMIC-II--arge is the most challenging dataset of all the datascts used: it consists of high-dimensional input observations and highly varied observation sequence lengths as reported in Table 1 On the MMMIC-II--Small dataset, GRUwE outperforms all the baseline models. GraFITi and HyperIMTS remain competitive.
Missing data perspective. Prior work Singh 1997); Ramoni & Sebastiani 1997 categorizes missingness into three types: (i) Missing Completely at Random (MCAR), (ii) Missing at Random (MAR), and (iii) Not Missing at Random (NMAR). In our experiments, the USHCN datasct corresponds to the MCAR sctting, where missinguess is synthetically introduced by randomly removing observations. In contrast, the Physionet and MIMIC-III datasets are instances of NMAR, where missingness is contextual, i.e., dependent on observed and unobserved values. For example, certain laboratory tests may only be ordered after abnormal vital signs are detected. Our results indicate that while GRUwE remains competitive under MCAR, it performs particularly well under NMAR conditions, outperforming all compared models on two out of three datasets. This suggests that GRUwE more effectively captures dependencies inherent in the data- gentratnng process.
Next event prediction. We report the comparison of the TPP models in the Table 2 and plot the log- likelihoods in Figure 2 Notably, the FullyNN model Omi et al. (2019) is designed for single event prediction. While RMSE and LL metrics can be computed, the ER metric cannot be evaluated for this model in our comparison. As demonstrated in Figure 2 GRUwE outperforms all intensity- models in terms of log- likelihood comparison, except on the Amazon dataset, where it achieves the second- highest performance. For the Taxi datasot, which has the least number of training sequences across all datascts, GRUwE ranks first in terms of the RMSE and ER. If we compare among all RNN-based intensity models (i.e. GRUwE, NHP, RMTPP), GRUwF demonstrates the best performance on all metrics and datasets showing the effectiveness of the proposed approach.
Rank analysis. Overall, our results in the Table 2 indicate that GRUwE is consistently among the top performing models across all the datascts and evaluation metrics. To assess its benefits more rigorously, we
Original NNllyNN model does not support multi-type event.
9

---

Published in Transactions on Machine Learning Research (01/2026)
Texi
Relweet ER(L)
EverkOverllow
Amazon
AvR:
Model
RMSB(L)
BRC4)
RMBE(J)
RMSE(()
ER(4)
RMSE(()
BRCC)
Runk (L)
FullyXK
0.3733
N.A.
21.92 14.15a
N.A.
1.378
N.A.
0.81825.00
N.A.
4.75
NHP
0.369
0.301
0. 22 10.005
22, 52 0,001
40.25
0,002
1.300 0.001
56. 18.00o
0.912 5.00.
68. 00.3004a0
2.85
A-NHP
0.3770: wot
11. 11.422c.mm um
22. 28.
41 41.05gamm
1.370
55. $1.51
0.6122=
85.65
20.000
3:38
THP
0.369
5.85
22, 29.001
40.25
1,388
55. 8040.ou
0.6122g
68. 72.6..0
2:13
8ARP
0.372-0005
9.76 0,0m0
22. 40.40.001
41. 41.000
1.373
58. 1049.00
0. 0.819:3.005
67. 27..00s..
6.
RMTPP
0.37020.00 nco
0. 88:. ne0
22. 31 200
46. 1020 one
1.370gn oon
07. 50g0
0. 0.6334gom or
73.06ga
5.
CRUwE
0.369
10.000
88.88
10.000
22.21 10:234
40.81 10,004
1.360 c.000
(66.
10.004 0.011
10.00
67. 07.24:10.00
1.76
Table 2: Next event prediction. Model performance comparison for the next event prediction on the Taxi, Retweet, StackOverflow and Amazon datasets. We report the mean and standard deviation of the metrics on five distinct random seeds. For RMSE and ER metrics, lower is better.
-3.0
Retweet
StackOverflow
0.25
-2.3
Log Likelhhood
-0.25
0.00
-4.0
-4.5
-2.4
-2.5
0.50
0.75
-3.5
-2.6
-27
-1.00
-6.0
-2.
-125
-6.5
-2.9
Fullyne
wi
A-NHP
THP
savre
RNTPP
GRUWB
-7.0
Fullysti
hnp
A-HiI
TNP
SAMP
aNTPP
GRUMt
Futlystt
A-HNP
TNP
SAMP
RNTPP
GRUWE
-3.0
FutlyRii
A-M0P
THP
SAMP
ANTPP
GRUSE
Figure 2: Comparison of Log-Litee Likelliol (LL) for all the TPP models on next event prediction task. A
higher value of LL indicates a better fit.
summarize the results by adding new rank-based metric that calculates the average rank of each model across all datasets for both RMSE and ER metrics. The ranking results show that GRUwE achieves the highest rank score of 1.75, while the second-best THP model has the score 2.13. More details on this rank analysis is provided in the Appendix 10
Differences to GRU-D Che et al. (2018). GRUWE is most closely related to the previously proposed GRU-D model since it also combines GRU and exponential docay functions. However, the following kcy differences remain. While GRU-D imputes the input missing values by learning exponential schedules to revert to its empirical mean, GRUwE lifts this mean-reverting assumption by applying zero imputation to handle missingness. This reduces cascading errors in the final prediction caused by imputation errors. GRU-D was proposed for classification, and it is unclear how it can be adapted to perform contimuous-time predictions. GRUwE addresses this by introducing a flexible prediction module. GRUwF simplifies GRU-D's Markov state representation significantly by removing the need to store and update: i) the last observed variable values, fi) their respective time clapsed since observation, and iii) empirical mcan for each variable. It is important to highlight that all of these differences make GRUwE simpler model compared to GRU-D, while significantly boosting the predictive accuracy across all datasets in the next observation prediction benchmark (Table 1.
Computational cost analysis. Beyond evaluating predictive performance, we assess the computational requirements of each model. Figure summarizes key metrics, including training time, peak memory usage during training, and inference time on retrospective test data. For a focused analysis of deployment effi- ciency, Figure Ma| compares the average inference time and memory consumption of top-performing models in online (sequential) setting. A comprehensive discussion computational complexity is provided in Appendix II
Regarding training time on retrospective data (Figure 3al, methods parallelizable over the time dimen- sion (T-PatchGNN, mTAND, GraFITi, HyperIMTS) are lastest. These are followed by RNN-based models (GRUwE, GRU-D, GRU-A:), then models with linear dynamics (RKN-Ar, CRU, f-CRU). Lastly, models that require invoking numerical solvers (ContiFormer, Latent ODE, ODE-RNN) consume the most amount of train time. As Figure 3b indicates, this speed often entails a trade-off: paraleeallel models achieve shorter training times at the cost of substantially higher memory consumption. For inference retrospec-
10

---

Published in Transactions on Machine Learning Research (01/2026)
T-PatchGNN 0 ..54
GRU-A.
7322
T-PatchONN
RKN-A:
000
22.24
T-PathhNN
GRUwE
GRU-A.
00.0
0.80 0.98
Latent 00E
ODE-RNN
CRU
58.10
159.56
6520.00
Latert DDE
ODE DDE-KNN
1:10
1.52
3.75
6.58
ContiForeer
ODE-RNN
31.91
mTAND
F-CRU
10:27
46.20
GraFll HyperIMTS
ITTAN
8.12
53
m
GRU-6
1.04
1666
GraFIT
HyperIMTS
12.23
HyperIMIS
13'
memory (GBc) usage during log
time (slepoch)
Arg. (mallistaace)) log
(a)
(b)
(c)
Figure 3: Computational cost analysis on retrospective data. models; (3b) compares the peak GPU memory usage during training; retrospective data. Lower is better for all plots.
3a
compares the train times for all
33c
compares the inference times on
tive data where models process the full history at once, most approaches exhibit comparable inference times, with ODE-based methods being notable exceptions (Figure 3c The computational profile changes signif- icantly in an online setting, where data arrive sequentially. Here, models that require the full observation history must buffer and reprocess all past data with each new prediction. This leads to growing memory and computational demands over time, as sequence length increases, creating computational bottleneck. In contrast, GRUwE maintains a compact Markov state updated incrementally, enabling highly efficient sequential inference. As a result, as shown in Figure 4a GRUwE incurs significantly lower computational overhead during online deployment.
5,569
cobre merory comumee (Ctb)
30
enttee g
Step
Code
20
mTAND
b.382 0.045
GRUWE
T.PatchONN
GraFmT
HyperIMTS
0.769
0.972
mTAND
10
6.435
GRUWE
0.061
GraPm
HyperIMTS
8.420
1..736
10.736
Time decay
Concat input
Decay state
g_t gamma h prprev
gamma exp (-relu (WRT))
x_mask [x, mask]
GRU update
h_t CRU (aak, g_t)
(b)
Figure 4: (a) Comparison of inference time (left) and memory consumplion (right) in online deployment. Lower is better in both cases. (b) GRUwE's step function pseudocode.
Easy to implement. As shown in Figure 4b GRUwE's reset state mechanisms can be implemented in just a few lines of code as it builds on the components that are natively supported by most modeling packages. An easy model implementation preferred not only because it improves readability, but also reduces the scope for introducing bugs, making the model casier to port, reproduce, debug, and maintain.
GRUwE ablation analysis. To better assess the contribution of the proposed GRUwE model, we conduct series of architectural ablation studies. Specifically, we investigate: (i) how GRUwE compares to variants that employ alternative time basis functions, such as learnable linear (GRU-Lin) or sinusoidal (GRU-Sin) basis; (ii) how the predictive performance changes when model complexity increased by using seporate learnable exponential function for the transition and prediction functions (GRUwE-DualExp), and (iii) how performance affected when model complexity is reduced by freezing the randomly initialized exponential basis (GRUwE-Frorzel FrozzeExp) during model training. Across all datasets (as presented in Table 3) GRUwE consistently matches outperforms its ablated variants, highlighting the importance of shared, learnable exponential basis. Replacing the exponential basis with lincar or sinusoidal functions degrades performance significantly. This supports our hypothesis that exponentials provide a better inductive bias, as their mem- oryless, multiplicative form makes temporal effects naturally composable over arbitrary time gaps. Learning soparate exponential basis for the transition and prediction functions helps in one but does not outperform GRUWE in most cases, suggesting that sharing a single temporal basis regularizes the training and yields more robust. temporal representation. Tastly, reducing the complexity by freezing the exponential basis
11

---

Published in Transactions on Machine Learning Research (01/2026)
USHCN (MSE x10-2)
Physionet (MSE x10-
MIMIC-III- Large (MSE x10-2)
MIMIC-III-Sman
Model
(MSE x10-²)
GRU-Lin
0.019
0.925
1.332
7.573
GRU-Sin
0.016
0.359
1.216
7.324
GRUwE-DualExp
0.013
0.269
0.854
4.670
GRUwE-ProzeuExp
0.016
0.256
0.848
4.665
GRUwE
0.014
0.253
0.831
4.650
Table 3: GRUWE ablation comparison. GRUwE variants modify the time basis function: using linear or sinusoidal functions, learning separate basis for transition and prediction, and freezing the randomly initialized exponential basis.
yields competitive but worse performance in all cases, indicating that adapting decay rates to the task is important.
Limitations. Unlike compared baselines such as mTAND and CRU, our proposed architecture, GRUwE doesn't 't have a built-in notion of uncertainty, and we consider incorporating it in the future work will be an exciting extension of our current work.
8 Conclusion
In summary, this work revisits the challenge of modeling irregularly sampled multivariate time series through the lens of simplicity and computational efficiency. By introducing GRUwE, we demonstrate that principled extension of the GRU architecture, augmented with learnable exponential basis functions, can achieve state- of-the-art predictive performance across both continuous-time forecasting and event prediction tasks, while remaining significantly computationally efficient during inference in the online deployment. These results demonstrate that carefully designed recurrent architectures grounded in classical principles, can compete or outperform more complex neural and differential equation based models. We believe this work provides useful guidance for future rescarch on practical and scalable modcling of irregular time scries.
9 Acknowledgement
The rescarch work in this paper was supported in part by NIH grant Ro1 EB032752, and by the University of Pittsburgh Center for Research Computing and Data resource RRID:SCR 022735 using HTC cluster, which is supported by NIH award $100D028483. The content of this paper is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
References
Philipp Becker, Harit Pandya, Gregor Gebhardt, Cheng Zhao, C. James Taylor, and Gerhard Neumann. Recurrent kalman networks: Factorizod inferonce high-dimensional deep fcature spaces. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 544-552. PMLR, 09 Jun 2019. html
URL httpp:://proceediigs mlr
Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific Reports, 8(1), April 2018. ISSN 2045-2322. doi: (0.1088//41500--11-222222
URL http://dxx doi oooi.org/11... 100.000100//4119--1115111 24271-9
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Yuqi Chen, Kan Ren, Yansen Wang, Yuchen Fang, Weiwei Sun, and Dongsheng Li. Contiformer: Continuous- time transformer for irregular time series modeling. In Thirty-seventth Conference on Neural Information Processing Systems, 2023.
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling, 2014. URL httts:///aatti:i http://aaavvv.ogaaar/sss// hwwp./ww.wrgga////223335555
12

---

Published in Transactions on Machine Learning Research (01/2026)
Edward De Brouwer, Jaak Simm, Adam Arany, and Yves Moreau. Gru-ode-bayes: Continuous model- ing of sporadicclly--otsevved time series. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' A'Alché-Buc E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019. URL htttps://proceeding neurips neurips.cc/papert httpppssccppppprffaaaeeeee /tlee//ppper/209////1ee
pdf
Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguee Rodriguez, and Le Song. Recur- rent marked temporal point processes: Embedding event history to vector. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1555-1564, 2016.
Albert Gu and Tri Dao. Mamba: Lincar-time sequence modcling with selective state spaces. arXiv preprint arXiv:231..00752, 2023.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 988):1335-178) November 1997. ISSN 1530-888X. doi: 10.1162/neco 199.9..9...988177777 URL http://dx. doi 10o.oi..rg/11111111
paco..19777 11..9799.8... 9.7....8.773
Alistair Johnson, Tom Pollard, and Roger Mark. Mimic-iii clinical database, 2023.
URL https:
//physionet. org/ content/mimiiiiii/
Rudolf Emil Kalman. Mathematical description of linear dynamical systems. Journal of the Sociely for Industrial and Applied Mathematics, Series A: Control, 1(2):152-192) 1963.
Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural controlled differential equations for irregular time series. Advances in Newral Information Processing Systems, 33:6696 070066677777777 2020.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), San Diega, CA, USA, 2015.
Jure Leskovec and Rok Sosič. Snap: A general-purpose network analysis and graph-mining library. ACM Transactions on Intelligent Systems and Technology (TIST), 8(1):1 20, 2016.
Boyuan Li, Yicheng Luo, Zhen Liu, Junhao Zheng, Jianming Lv, and Qianli Ma. Hyporimts: Hypergraph neural network for irregular multivariate time series forecasting, ar preprint orXiv:2505. 174:2155..7731, 2025.
Zitao Liu and Milos Hauskrecht. Clinical time series prediction: Toward hierarchical dynamical system framework. Artificial intelligence in medicine, 65(1):5-18, 2015.
Yicheng Luo, Bowen Zhang, Zhen Liu, and Qianli Ma. Hi-patch: Hierarchical patch gnn for irregular multivariate time scries. In Forty-second International Conference on Machine Learning, 2025.
Ivan Marisca, Andrea Cini, and Cesare Alippi. Learning to reconstruct missing data from spatiotemporal graphs with sparse observations. Advances in Neural Information Processing Systems, 35:32069-32082, 2022.
Hongyuan Mei and Jason M Eisner. The neural hawkes process: A neurally self self-modulatin multivariate point process. Advances in neural information processing systems, 30, 2017.
MJ Menne, CN Williams Jr, and RS Vose. Long-term daily and monthly climate records from stations across the contiguous united states (us historical climatology network). Technical report, Environmental System Science Data Infrastrructure for a Virtual Ecosystem.... 2016.
Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 conference on empirical methods in natural language processing and the Ith international joint conference on natural language processing (EMNLP-IJCNLP), pp. 188 197, 2019.
Takahiro Omi, Kazuyuki Aihara, et al. Fully neural network based model for general temporal point pro- cesses. Advances in neural informalion processing systems, 32, 2019.
13

---

Published in Transactions on Machine Learning Research (01/2026)
Marco Ramoni and Paola Sebastiani. Robust parameter learning in bayesian networks with missing data. In Sirth International Workshop on Artificial Intelligence. and Statistics, pp- 399-406. PMLR, 1997.
Yulia Rubanova, Ricky T. Q- Chen, and David K Duvenaud. Latent ordinary differential equations for srregularly-sampled time series. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-But Bue, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019. URL https://proceedin
Paper....d
Mona Schirmer, Mazin Eltayeb, Stefan Lessmann, and Maja Rudolph. Modeling irregular time series with continuous recurrent units. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (ods.), Proccedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 19388 19405. PMLR, 17 23 Jul 2022. URL html
https://proceeddings mlr wttpppessseeessppieeereep
Satya Narayan Shukla and Benjamin Marlin. Multi-time attention networks for irregularly sampled time series. In International Conference on Learning Representations, 2021. URL htttp::/ net/
forum21d==c0J61w004
Robert H. Shumway and David S. Stoffer. Time Series Analysis and Its Applications (4th edition). Springer- Verlag, Berlin, Heidelberg, 2017.
Sima
Neda Tavakoli, and Akbar Siami Namin. A comparison of arima and Istm in forecasting time series. In 2018 1 7th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 1394 1401, 2018. doi: 10.1109/ICMLA 2018.00227.
Ikaro Silva, George Moody, Daniel J Scott, Leo A Celi, and Roger G Mark. Predicting in-hospital mortality of icu patients: The physionet/ /computing in cardiology challenge 2012. In 2012 Computing in Cardiology, pp. 245 248. IEEE, 2012.
Moninder Singh. Learning bayesian networks from incomplete data. AAAI/IAA, 1001:534-000 539, 1997.
Jimmy TH Smith, Andrew Warrington, and Scott W Linderman. Simplified state space layers for sequence modeling. In TCLR, 2023.
Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffusiio models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34: 24804-24816, 2021.
Chris Whong. Foiling nyc's taxi trip data. FOILing NYCs Taxi Trip Data. Np, 18:14, 2014.
Siqiao Xue, Xiaoming Shi, James Zhang, and Hongyuan Mei. Hypro: hybridly normalized probabilistic model for horizon prediction of event sequences. Advances in Newral Information Processing Systems,
35:34641-34650, 2022.
Vijaya Krishna Yalavarthi, Kiran Madhusudhaann, Randolf Scholz, Nourhan Ahmed, Johannes Burchert, Shayan Jawed, Stefan Born, and Lars Schmidt-- Schmidt-Thieme. Thieme. Grafiti: Graphs for forecasting irregularly sampled time series. In Proceedings of the AAAT Conference on Artificial Indelligence, volume 38, pp. 16255- 16263, 2024.
Chenghao Yang, Hongyuan Mei, and Jason Eisner. Transformer embeddings of irregularly spaced events and their participants. In Proceedings of the tenth international conference learning representations (ICLR), 2022.
Qiang Zhang, Aldo Lipani, Omer Kirnap, and Emine Yilmaz. Self-attentive hawkes process. In International conference on machine learning, pp. 11183 11193. PMLR, 2020.
Weijia. Zhang, Chenlong Yin, Hao Liu, Xiaofang Zhou, and Hui Xiong. Irrogular multivariate time se- ries forecasting: A transformable patching graph neural networks approach. In Forty-first International Conference Machine Learning, 2024.
14

---

Published in Transactions on Machine Learning Research (01/2026)
Xiang Zhang, Marko Zeman, Theodoros Tsllikkaidis,, and Marinka Zitnik. Graph-guided network for irreg- ularly sampled multivariate time series. In International Conference on Learning Representations, TCLR, 2022.
Ke Zhou, Hongyuan Zha, and Le Song. Learning triggering kernels for multi-dinensional hawkes processes. In International conference machine learning, pp. 1301-1309. PMLR, 2013.
Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao, and Hongyuan Zha. Transformer hawkes process. International conference on machine learning, pp. 11692- 11702. PMLR, 2020.
15

---

Published in Transactions on Machine Learning Research (01/2026)
Appendix
Table of Contents
Theory
16
Lipschitz Continuity Exponential Decay Function
16
1.2
Monotonic Latent Dynamics
17
Datasets
18
USHCN
18
2.2
Physionet
................................
2.3
MIMIC-MI--aa Large
................................
18
2.4
MINIC-III-Sa Small
19
2.5
Dataset Attributes
19
Event Prediction Dalasels
19
3
Baselines
20
Baselines Ohservation Prediction Task
20
1.2
Baselines for Event Prediction Task
21
Generic hyperparameters for Next Observation Prediction Task
Hyperparameters for Next Observation Prediction Task USHCN
Hyperparameters for Next Observation Prediction Task Physionet
23
Hyperparameters for Next Observation Prediction Task on MIMIC-III-Smal
25
Hyperparameters for Next Observation Prediction Task on MIMIC-III- Large
26
Hyperparameters for Event Prediction task
Rank-based analysis for next event prediction.
28
11 Computational Cost Analysis
29
12 Computing Infrastructure
30
Theory
1.1 Lipschitz Continuity of Exponential Decay Function Theorem 1.1 (Lipschitz Continuity of Exponential Decay Function). Let y : R, + R be the erponential decay function defined as
(AT) exp (- max (0, WAT + b)),
where W and E R are fized scalar parameters. Then, & is Lipschitz continuous R,, with Lipschitz
constant
L W exx((-))
16

---

Published in Transactions on Machine Learning Research (01/2026)
Proof. We first compute the derivative of y with respect to AT in piecewise form:
0, W exp -OPAT-W-T-- b), if AT
if
dAT dAT (AT)
W b
W
Observe that y continuous everywhere and differentiable almost everywhere (it is non-differentiable only at the point AT - - The function y is Lipschitz continuous its derivative is bounded almost everywhere, and the essential supremum of the absolute value of the derivative gives the Lipschitz constant.
For WAT + b > 0, the magnitude of the derivative is:
dy
W expexxppWWW - b).
dAT
The maximum of this expression over AT R4 occurs at AT 0, giving:
d-
S W exp(-bb.
dAT
For AT < W: the derivative is wero. Therefore, the global Lipschitz constant is:
dy
L = sup AT'0
W eex((x))
dAT
1.2 Monotonic Latent Dynamics
Remark (Exponential Basis Functions Induces Growth and Decay). Exponential basis functions can induce exponentially growing and deenyjng frends the state as a function of time elapsed.
AT
Const.
Exp.
Figure An abstract diagram of latent dynamics in the case of GRU (top) and GRUwE (bottom) high- lighting that GRUwE can recover monotonic latent dynamics between two consecutive event observation arrivals.
We note that 7(TT) E 10, the th component of state is bounded, i.c., h4 E [-, Since g,,T r (AT))) if hr E (-1, then ATAAT 2 h Basically, the transformed state value is an upper-bound of the initial value if it is negative. If the initial value is positive, then the transformed value is a lower-bound: i.c., hys E (0, 1], then 1444, I h Therefore, depending on the state h, the exponential basis function update can cause the state value to grow or decay exponentially. This property makes exponential basis functions highly expressive for time-series modeling.
17

---

Published in Transactions on Machine Learning Research (01/2026)
2 Datasets
2.1 USHCN
United States Historical Climatology Network (USHCN) Menne et al. (2016) is a publicly available dataset
11w.3//00//11111//.....1.1 NDP019) consisting of daily measurements of meteorological variables including min temperature, max temperature, precipi- tation, snowfall, and snow depth from 1218 observing stations across the United States. It is worth noting that there are multiple versions of datasets extracted from USHCN that have been used to evaluate irregu- larly sampled tim series. We extract the dataset in our analysis using the steps mentioned in [Shhrmerr et al.) (2022). We mimic their data pre-processing pipeline by: i) sub-selecting 1168 stations over 4-year period ranging from 1990 to 1993 ii) subsample 50% of time-points to increase irregularities in time dimension; and setting unobserved rate to 20% to increase sparsity of observations, iii) 20% of the entire dataset is used for testing; we train and validate on the remaining 80% data; 25% of that is used for validation.
1b1.
To be able compare methods listed in the Schirmer et al. 2022) for the prodiction task, we replicate their: (1) pre-processing logic; (2) splitting of dataset into train, validation and test sets by using the same seed; (3) 20% partial observability in feature dimension and 50% in time dimension in GRUwE run; (4)
masking logic for prediction task.
2.2 Physionet
Predicting Mortality of ICU Patients: The PhysioNet/Compputng in Cardiology Challenge 2012 Silva et al.) (202)) made publicly available tient stays that span 48 hours reporting 37 clinical real-valued time series variables observed at irregular time- intervals. The dataset includes varions variables, including Non-Invasive Mean Arterial Pressure, Platelets, Sodium, and several others. We follow the pre-processing of this dataset described by [Schirmer et al. (2022). Observations in time are rounded by 6 minutes. Similar to USHCN, test split consists of 20% of the data, the rest is used for training and validation; validation set consists of 25% of this split.
0/) 8000 TCU pa-
For fair comparison to methods listed in |Schrmer et al. 2022 ensure (1) 6-minute quantization (as done in Rubanova et al. (2019); Schirmer et al. (2022) (2) splits are created using the same seeds; (3) same masking logic is applied for prediction task.
2.3 MIMIC-III-Large
In the following, we outline the steps to extract MIMIC-III-Lare dataset from the MIMIC-II Johnson et al. (2023 htttps://physoonet: httpy///iinet..org//teteee//t/ee http://cmmttttmmmiiciii////////
A population cohort of 10,265 hospital admissions (on 8,799 patients) are extracted from MIMIC-11I based on the following criteria: i) patient record is recorded in Meta Vision critical care information system, ii) the length of the palient record is between 2 and 20 days, iii) the age of the patient is between 18 and 90. From all EHR tables available in MIMIC-III, we extract (irregularly sampled) time series that include vital signs (such as Hcart Rate and Mcan Arterial Pressure), lab results (such as Glucose and Hemoglobin), administered medications (such as Propofol and Norepinephrine), and procedures (such as Intubation). The vital signs and lab results are numerical time series, while the rest are indicator time series, indicating and when the event occurred.
We filter out any univariate time series that occurs less than 500 times across all patients in the cohort resulting in total of 506 time series: 393 numerical (vitals and labs), 77 medications and 36 procedures event time series. We define an A-point, abbreviated for sachhor-piint as a temporal moment at which a decision- making system can formulate a prediction based on the past sequence of events. We extract A-points from the filtered patient records regularly with frequency of 24 hours i.e. one sample is extracted every 24 hours from the patient record. We standardize all real-valued univariate time series (i.e. vital signs and labs) data using min- max scaling; and encode all other indicator time series as binary value 0/1. Value is 1 if event occurs; 0 otherwisc. 80% of the patient hospital admission are used for training and validation (20% of train split); the rest is used for lesting. Splits are constructed on disjoint patients.
Additional pre-processing is required to remove missing values encoded as 9999999 in the numerical time series. To remove outliers from univariate time series (for example, very large values of the order of 1e5),
18

---

Published in Transactions on Machine Learning Research (01/2026)
filter out observations that fall either in < 0.1 or 99.9 percentile ranges. Subsampling of A-points is performed as follows:
1. For each split:
a. For each patient admission record, filter out A-points with less than 50 events in history and prediction window.
b. Next, randomly sample one A-point from the filtered A-points i.e. one sample per patient admission record.
Subsample 1000 A-points from train, 250 from validation and 200 from test set to be used for experimentation.
2.4
MIMIC-III-Smal
We adopt the data extraction methodology described in De Brouwer al. (2019) to derive the MIMIC- III-Small dataset from MIMIC-III Johnson et al. (2023) Each sample comprises 96 time series variables, irregularly observed over 48-hour long patient visit. For comprehensive overview of the dataset pre- processing and extraction steps, we refer readers to Appendix K in De Brouwer et al. (2019)
2.5 Dataset Attributes
Table f summarizes the mean, standard deviation, minimum and maximum of all the sequences per dataset. Importantly the MIMIC-III dataset does not undergo any time discrctization, resulting in sequence lengths spanning from 105 to 4299. In terms of average length, the order is follows: USHCN MMMIC-III-arree Physionet MIMIC-II-Small.
Table 4: Sequence length statistics and number of target variables across datasets
Sequence length statistics
Dataset
# Target Variables
Mean S.D.
Min
Max
USHCN
730.0 + 0.00
730
730
5
Physionet
72.16 + 20.93
185
37
MIMIC-II1SSall
48.15 + 69.92
614
96
MIMIC-III-Laree
244.30 + 288.85
105
4299
393
2.6 Event Prediction Datasets
We provide descriptions of the four real-world publicly available datasets used in the next even prediction benchmark of the paper. We summarize the properties these datasets after providing the descriptions:
Amazon: This dataset includes time-stamped user product reviews behavior from January, 2008 to October, 2018. Each user has sequence of produce review events with each event containing the timestamp and category of the reviewed product, with each category corresponding to an event type. We work on a subset of 5200 most active users with an average sequence length of 70 and then end up with K- 16 event types
Retweet: This dataset contains timo-stamped user retweet event sequences. The events are cate- gorized into K = 3 types: retweets by "small," "medium" and "large" users. Small users have fewer than 120 followers, medium users have fewer than 1363, and the rest are large users. We work on a subset of 5200 most active users with an average sequence length of 70.
This dataset tracks the time-stamped taxi pick-up and drop-off events across the five boroughs of the New York City; each (borough, pick-up or drop-off) combination defines an event type, so there are K 10 event types in total. We work on a randomly sampled subset of 2000 drivers and each driver has a sequence. We randomly sampled disjoint train, dev and test sets with 1400, 200 and 400 sequences.
Taxi:
19

---

Published in Transactions on Machine Learning Research (01/2026)
StackOverflow: This dataset has two years of user awards question-answering website: each user received a sequence of badges and there are K = 22 different kinds of badges in total. We randomly sampled disjoint train, dev and test sets with 1400, 400 and 400 sequences from the dataset.
Table 5 provides description of number of unique event types (K), the number of events and sequence length statistics across all four datasets used in our experimental evaluation.
Table 5: Statistics of the used datasets for evaluation
Dataset
K
# of Event Tokens
Sequence Length
Train
Dev
Test
Min
Mean
Max
Retweet
3
369000
62000
61000
10
41
97
Amazon
16
288000
12000
30000
14
44
94
Taxi
10
51000
7000
14000
36
37
38
OverlOv
90000
25000
26000
41
65
101
3 Baselines
3.1
Baselines for Next Observation Prediction Task
We include the following bascline methods to compare against proposed model GRUwE on the next observation prediction lask.
We consider recurrent models GRU-A, as our baselines in the comparison. Gated Recurrent Unit Mhit/Chung al. (2014) have been proposed to model sequences using set of parametric update equations. Since GRU is not time-aware, variant of GRU that also feeds in time elapsed (since the last time-step) along with the input GRU-A, is used in the comparison.
GRU-A;:
RKN-A;: Recurrent Kalman Networks (RKN) Becker al. 2019 have been proposed to incorporate uncertainty in time series modeling. Similar to GRU-A, we include the baseline RKN-A, that includes time elapsed as an additional imput to the model.
GRU-D: We compare method to Gated Recurrent Unit-Decay (GRU-D) Che et al. 2018 that uses mean-reverting imputation function for missing variables; applies learnable exponential decays in the input and latent dimension to account for irregular observed times.
TTAND: We evaluate our performance against the Gncoder-Decoder generative model, Multi-Tine At- tention Network (mTAND-Full) Shukla & Marlin (2021). which defines reference points and represents continuous time-points with learnable embeddings to encode their relationship and generate predictions.
We consider Continuous Recurrent Units (CRU) and fast-Continuous Recurrent Units (f-CRU) Schirmer et al. (2022) our baselines for prediction tasks. f-CRU is a fast implementation of jointly proposed CRU method. CRU consists of an encoder-decoder framework where the hidden state progression is governed by linear stochastic difforential equation that allow incorporating arbitrary time-intervals between observations.
CRU and f-CRU:
ODE-RNN and Latent ODE: In our analysis, we consider Latent ODE and ODE-RNN proposed in Rubanova et al. (2019) as comparative baselines. ODE-RNN model consist latent states that adhere to ODE between observations and are updated at observations using standard RNN update equations. Latent ODE model adopts variational auto-encoder framework wherein the hidden state posterior modeled by an ODE-RNN model. We used the configuration of Latent ODE with ODE-RNN encoder.
We incorporate ContiFormer Chen et al. (023) as one of our basclines for the prediction tasks. It builds upon the original transformer architecture by first extending the input irregular data to the contimuous-time latent representation by assuming that the underlying dynamics governed by the ODEs.
ContiFormer:
20

---

Published in Transactions on Machine Learning Research (01/2026)
one of our baselines for the prediction tasks. T-PatchGNN first segments each time series into patches of uniform temporal resolution followed by the transformer and time-adaptive GNNs to capture dependencies in the multivariate time series. We use the official T-PatchGNN code made publicly available hcre: petts://giteen www.usaail-uusttttaaaaaaaaaaaaee PatcchhN in our experiment pipeline.
T-PatchGNN:
We include T-PatchGNN Zhang et al. (2024
GraFITi: We add GraFITi Yalavarthi et al. (2024) as one of our basclines for comparisons on the next
observation prediction task. GraFiTi casts the time series prediction task in terms of edge weight prediction problem after converting the time series to a sparse graph structure. To incorporate this method, we make use of the official implerentaatin available here:
HyperIMTS: We include HyperLMTSS MipeerTTSLLee al. 2025 as one of baselines for the next observation prediction lask. HyperIMTS models the irregular time series by constructing a hypergraph from observations and learning dependencies with the help of hyperedges connecting the observation nodes. We use the official implementation made available by the authors from this sitc: htttps://githui http://aaaabbbbb//ymmm.11.1
3.2 Baselines for Event Prediction Task
We include the following temporal point process baselines for event prediction benchmark.
FullyNN: We add Fully Neural Network (FullyNNN Omi et al.] (2019) to our comparison on the next event prediction task. FullyNN learns a cumulative hazard function for modeling the point process intensities. We use the publicly available implementation of this method from here: https://github:
NHP: We include Neural Hawkes Process (NHP) Mei & Eiisner (0117) as one of our basclines for the next event prediction task. NHP develops a contimuous-time intensity model using the LSTM architecture. haatt-reaarra//ss//pprraooaaaaa https://github.
We use the publicly available implementation of this method from here:
A-NHP: We include Attention-Neural Hawkes Process (A-NHP) Yang et al. (2022) a baseline model for next event prediction task. A-NHP with the help of attention mechanism further refines the NHP architecture to define novel intensity functions. Official code for this method is publicly available at https://githu/ com/yangalann23/ anhp- http:/aaareeeeereer////////arpar
We used the adapted versions from:
THP: Transformer Hakwes Process (THP) Zuo et al. (2020) is included in the comparison for next event prediction task. THP defines the intensity function using the transformer architecture. We use the publicly available implementation of this model from: https://github.ce com/ ant-rreeerchh/ EasylemporallooitProoess
SAHP: We add Self-Attention Hawkes Process (SAHP) Zhang al. 2020 to the next event predic- tion comparison. SAHP uses self-attention mechanism in tranformers to define intensity function. We use the publicly available implementation of this model from: https: how.a/u..oom//aaaaa wwwtooottteeeeeeeeeeeeeeee roto/antt-eseerccl EasyTenporalPPottrroeess
Recurrent Marked Temporal Point Process (RMTPP) Du et al. 2016) defines a con- ditional intensity model based on RNN architecture. We include it the next event pre- diction benchmark. We use the code available for this method in the public repository:
RMTPP:
4
Generic hyperparameters for Next Observation Prediction Task
The following hyperparameters are applicable broadly across forecasting experiments:
For all models, we use Adam optimizer Kingma & Ba 2015
For all models, we apply exponential learning rate decay of 0.99 and perform gradient clipping using max 1
For all GRUWE configurations, we use the same architecture for the hidden to observation function Fout (g. Linear(hidden dim, target target_dim).
21

---

Published in Transactions on Machine Learning Research (01/2026)
5 Hyperparameters for Next Observation Prediction Task on USHCN
We keep batch size fixed to 50, the number of training epochs to 100, learning rate decay to 0.99, with gradient clipping and perform a hyperparameter search for each model as follows:
5.0.1 mTAND
For TAND, we perform a grid search over time embedding dimension {32, 64, 128), latent state dimension ={8, 10, 16, 20), number of reference points= 64, 128} and learning rate =(0.1, 0.05, 0.01, 0.001). Of which, latent state dimension=8, number of reference points=32 and learning rate of 0.01 performs the bost the validation dala.
5.0.2 GRU-D
For GRU-D, we perform a search over latent state dimension = {8, 10, 16, 20} and learning rate = {0.1, 0.05, 0.01, 0.001). We find that configuration with latent state dimension=20 and learning rate=0.01 performs best on the validation set.
5.0.3 CRU
Fixed hyperparameters for CRU are: variance activation for encoder='square; decoder='exp' transi- tion='relu' encoder variance activation='square, decoder variance activation='exx exp', number of basis ma- trices-20, and the same encoder and decoder network architecture as used in Schirmer et al. (2022). We perform a search on latent state dimension=(8, 10, 16, 20] and learning rate {0.1, 0.05, 0.01, 0.001). We report that the latent dimension=10 and learning rate=0.05 performs the best on validation set.
5.0.4 f-CRU
Fixed hyperparameters for f-CRU include: variance activation for encoder='square' dlecoder='exp'. tran- sition='relu' encoder variance ootivation="square decoder variance activation='exp,, number of basis ma- trices=20, and the same encoder and decoder network architecture as used in SShhrmer et al. (2022). We perform a search on latent state dimension = {8, 10, 16, 20} and learning rate {0.1, 0.05, 0.01, 0.001). Wo report that the latent dimension=10 and learning rale=0.05 performs the best on the validation set.
5.0.5 Latent ODE
We use Latent ODE model with ODE-RNN encoder. We perform a grid search on la- tent state dimcnsion=(8. 10, 16, 20}, recognition network dimension= 32, 64), number of GRU (16, 64), number of generation (2, number of recognilion layers= 3} and learning rate= (0.1, 0.05, 0.01, 0.0013. The configuration that performs the best on validation split with latent state dimension, recognition network dimension, number of GRU units set to 20; learning rate of 0.01 and gener- ation and recognition network layers set to 3.
5.0.6 ODE-RNN
For ODE-RNN, we use GRU as the RNN model and use the adjoint solver method implemented in the library: https://github comm/rtgichenttorcdiiiii RNN, we search over latent state dimension= 10, 16, 20) and the learning rates=(0.1,, (0.1, 0.05, 0.01, 0.001). We find that the configuration of latent state dimension=20 and learning rate 0.01 works the best. on the validation set.
for solving the ODEs in differentiable manner. For ODE-
5.0.7 ContiFormer
We use the ContiFormer implementation released by the authors https: http://bwbccammmmcoooootttttaaia Tree/maiin/ContiFrree in our implementation. Note that since USHCN has highest average sequence length, and ContiFormer memory intensive, we can only fit a batch size of 4 samples in our GPU memory. Keeping other parameters fixed, vary the latent state dimension {8, 10, 16, 20) and learning rate {0.1, 0.05, 0.01, 0.001}. Our experiments show that latent state dimension=16 16 and learning rate=0.001 achieves the best performance on validation set.
5.0.8 GRU-A,
For GRU-A,, we scarch over latent state dimensions of the GRU =(8, 10, (6,20) and lcarning rates=(0.1, 0.05, 0.01, 0.005, 0.0013. We find latent state dimension=16 and learning rate=0.005 to be the best performing configuration.
22

---

Published in Transactions on Machine Learning Research (01/2026)
5.0.9 RKN-A,
We use the RKN-A, implementation made available by the authors https://github. http://lthuu..comAaaat com/ALRhub/ria share.git git Our RKN-A, implementation uses the same encoders and decoders architecture as the CRU model. Keeping other parameters fixed, we search over latent state dimensions = {8, 10, 16, 20} and learning rates = {0.1, 0.05, 0.01, 0.005, 0.001}. Latent state dimension=20 and learning ratc=0.001 results the best performing model.
5.0.10 T-PatchGNN
We perform a grid search over learning rates = {0.1, 0.05, 0.01, 0.005, 0.001), time and node embedding dimensions = {4,8 8, 16), number of patches=(2,-4 4] (more number of patches results in GPU OOM issue), and latent state dimension {4,8, 8, 10, 12, 16), while fixing the number of heads in one transformer layer number of transformer layers = 1. We find that the configuration with learning rate=0.001, time and node embedding dimeusion-8, number of patches-2, latent stale dimension-16 results in the best validation
MSE.
5.0.11 GraFITi
We perform grid search over learning rates {0.1, 0.01, 0.005, 0.001}, latent state dimension
{4,8, 16, 20, 32, 64), number of layers {1, 2, 4] and number of attention heads {1,2, 4]. We report that the configuration with 1r=0.01, latent state dimension=8, number of layer=2 and number of attention heads=1 results in the best validation MSE.
5.0.12 GRUWE
We perform grid scarch over latent state dimensions=(8, 10, 16, 20} (to be comparable to other basclines considered) with learning rates {0.1, 0.05, 0.01, 0.005, 0.001}. For USHCN, the best model configuration that maximizes validation prediction MSE uses learning rate=0.1 and latent state dimension=20.
5.0.13 HyperIMTS
We perform grid search over learning rates {0.1, 0.05, 0.01, 0.005, 0.001}, latent state dimension {8, 16, 22, 64, 128}, number of layers = 2, that the configuration with 1r=0.001, latent state dimension=32, number of layer=2 and number of attention heads=2 results in the best validation MSE.
2, 4] and number of attention heads = {1, 4]. We report
6 Hyperparameters for Next Observation Prediction Task on Physionet
We keep the following hyperparameters constant across all methods: batch size= number of training epochs=100, learning rate dccay=0.99 and gradient clipping enabled. Below are the model specific experi- ments we carried out.
6.0.1 mTAND
For mTAND, we perform a grid search over time embedding dimension = {32, 64, 128), latent state dimension ={8, 10, 16, 20, 22, 24), number reference points=(32, 64, 128) and learning rate = (0.1,0.05, 0.01, 0.001). Of which, time embedding dim=32, latent state dimension=22 number of reference points-64 and learning rate=0.01 performs the best on the validation data.
6.0.2 GRU-D
For GRU-D, we perform search over latent state dimension = {8, 10, 16, 20, 24, 64) and learning rate = {0.11, 0.05, 0.01, 0.001). We find that configuration with latent state dimension=16 and learning rate=0.01 performs best. on the validation set.
6.0.3 F-CRU
Fixed hyperparameters for [-CUU include: variance activation for encoder='square', dleoder='eexp' tran- sition='rehtt encoder variance "stivationnssquaree decoder variance activation='exp' number of basis ma- trices=20, and the same encoder and decoder network architecture as used in Schirmer et al. (2022). We perform search on latent state dimension = (8, 10, 16, 20, 32, 64} and learning rate {0.1, 0.05, 0.01, 0.001). We report that the latent 1imension=16 and learning rate=0.001 performs the best the validation set.
23

---

Published in Transactions on Machine Learning Research (01/2026)
6.0.4 CRU
Fixed hyperparameters for CRU are: variance activation for encodor='square"; decoder='oxp', transi- tion='relu' encoder variance iniivatioosssuarre decoder variance activation= exp', number of basis matri- ces=20, and the same encoder and decoder network architecture as used in Schirmer al. 2002)). We perform scarch latent state dimension= (8, 10, 16,20 20, 32, 64) and learning rate {0.1, 0.05, 0.01, 0.005, 0.001). We report that the latent dimension=32 and learning rate=0.005 performs the best validation set.
6.0.5 Latent ODE
We use Latent ODE model with ODE-RNN encoder. We perform a grid search on latent state dimension=(8, 10, 16, 20, 32, 64), recognition network dimension= 32, 64), number of GRU units=(16, 32, 64), mumber of generation la,crs=(2,33 number of recognition layers= 3) and learning rale= {0.1, 0.05, 0.01, 0.005, 0.001}. The configuration that performs the best on validation split with latent state dimension, recognition network dimension, number of GRU units set to 32; learning rate=0.005 and generation and recognition network layers set to 3.
6.0.6 ODE-RNN
For ODE-RNN, we scarch over latent state dimension=(8, 10, 16, 20, 32, 64) and the learning rales- (0.1, 0.05, 0.01, 0.005, 0.001}. We find that the configuration of latent state dimension=16 and learning rate = 0.001 works the best on the validation set.
6.0.7 ContiFormer
For ContiFormer, we perform a search over the latent state dimension {8, 16, 20, 64) and learning rate {0.1, 0.05, 0.01, 0.001}. Our experiments show that latent state dimension=16 and learning rate=0.001 achieves the best performance on the validation set.
6.0.8 GRU-A:
For GRU-A, we search over latent state dimensions of the GRU =(8, 10, 16, 20, 64) and learning (0.1,- 0.05, 0.01, 0.005, 0.0013. We find latent state dimension-32 and learning rate=0.005 to be the best performing configuration.
6.0.9 RKN-A,
Our RKN-A, implementation uses the same encoders and decoders architecture as the CRU model. Keeping other parameters fixed, we search over latent state dimensions {8, 10, 16, 20, 22, 64) and learning rates (0.1, 0.05, 0.01, 0.005, 0.001}. Latent state dimension=32 and learning rate=0.001 results in the best performing model.
6.0.10 T-PatchGNN
We perform a grid search over learning rates (0.1, 0.05, 0.01, 0.005, 0.001), time and node embedding
dimensions {4, 8, 16), number of patches= 10, 203. and latent state dimension {4, 16, 32, 64), while fixing the number of heads in one transformer layer number of transformer layers = 1. We find that the configuration with learning rate=0.001, time and node embedding dimension=8, number of patches=10, latent state dimension=8 results in the best validation MSE.
6.0.11 GraFITi
We perform grid search over learning rates 10.1, 0.05, 0.01, 0.005, 0.001}, latent state dimension {4,8, 10, 16, 20, 32, 64], number of layers 21 4} and number of attention heads {1, 4). We re- port that the configuration with Ir=0.005, latent state dimension=64, number of layer=2 and number of attention heads=1 results in the best validation MSE.
6.0.12 GRUWE
We perform search over learning rate=(0.1, 0.025, 0.05, 0,01, 0.005), the hidden state dimensions= {10, 16, 20, 24, 32, 64} (to be comparable to other models). For Physionet, we use learning rate=0.025 and latent state dimension-64.
6.0.13 HyperIMTS
We perform grid search over learning rates {0.1, 0.05, 0.01, 0.005, 0.001), latent state dimension
{8, 16, 20, 32, 64, 128), number of layers 21, and number of attention heads {1, 4). We report
24

---

Published in Transactions on Machine Learning Research (01/2026)
that the configuration with 1r=0.005, latent state dimension=16, number of layer=2 and number of attention heads=4 results in the best validation MSE.
7 Hyperparameters for Next Observation Prediction Task on MIMIC-III-SmalI
We keep the following hyperparameters constant across all methods: batch size=100 (we reduce it for models if we face OOM issuc), number training epochs=100, lcarning rate decay=0.99 and gradient clipping enabled. Below are the model specific experiments we carried out.
7.0.1 mTAND
For mTAND, we perform a grid scarch over time embodding dimension {32, 64, 128), latent state dimension ={8. 16, 32, 64, 128), number of reference points= 32, 64, 128) and learning rate {0.1, 0.05, 0.01, 0.001, 0.001,0.0001]. Of which, time embedding dim=32, latent state dimension=128, number of reference points=8 and learning rate=0.0001 performs the best on the validation data.
7.0.2 GRU-D
For GRU-D, we perform search over latent state dimension {8, 10, 16, 32, 64, 128) and learning rate = {0.1, 0.05, 0.01, 0.001). We find that configuration with latent state dimension=64 and learning rate=0.001 performs best on the validation set.
7.0.3 F-CRU
Fixed hyperparameters for f-CRU include: variance activation for encoder='square', decoder= transi- tion='relu' encoder variance "ssrratioon'quaare, decoder variance activation= 'exp', number of basis matri- ces=20, and the same encoder and decoder net work archilecture as used in Schirmer el al. (022). We perform search on latent state dimension {8, 10, 16, 20, 32, 64) and learning rate {0.1, 0.05, 0.01, 0.001, 0.0001). We report that the latent d and learning rate=0.0001 performs the best on the validation set.
7.0.4 CRU
Fixed hyperparameters for CRU are: variance activation for encoder='square', decoder='exp', transi- tion='relu'' encoder variance iittaatioo=ssuuarre decoder variance activation= 'exp', number of basis matri- ces-20, and the same encoder and decoder network architecture as used in Schirmer et al. 2022). We perform search on latent state dimension= 10, 16, 20, 32, 64) and learning rate {0.1, 0.05, 0.01, 0.005, 0.001}. We report that the latent dimension=32 and learning rate=0.005 performs the best on validation set.
7.0.5 Latent ODE
We use Latent ODE model with ODE-RNN encoder. We perform a grid search on latent state dimension=(8, 10, 16, 20, 32, 64), recognition network dimension= (16, 32, 64), number of GRU units=(16, 64), number of gencration (2, number of recognition layers= and learning rale= (0.1, 0.01, 0.005, 0.001}. The configuration that performs the best on validation split with latent state dimension, recognition network dimension, number of GRU units set to 32; learning rate=0.001 and generation and recognition network layers set to 3.
7.0.6 ODE-RNN
For ODE-RNN, we search over latent state dimension= 10, 16, 20, 64) and the learning (0.1, 0.05, 0.01, 0.005, 0.001, 0.00013. We find that the configuration of latent state dimension=64 and learning rate 0.0001 works the best on the validation set.
7.0.7 ContiFormer
For ContiFormer, we perform a search over the latent state dimension = 18,16,20, 20,32, = {0.1, 0.05, 0.01, 0.011. Our experiments show that latent state dimension=16 and learning rate=0.001 achieves the best performance on the validation set.
32,664.) 64) and learning rate
7.0.8 GRU-A,
For GRU-A, we search over latent state dimensions of the GRU =(8, 10, 16, 24, 64) and learning rates=(0.1, 0.05, 0.01, 0.005, 0.0013. We find latent state dimension=332 and learning rate=0.005 to be the best. performing configuration.
25

---

Published in Transactions on Machine Learning Research (01/2026)
7.0.9 RKN-A,
Our RKN-A, implementation uses the same encoders and decoders architecture as the CRU model. Keeping other parameters fixed, we search over latent state dimensions = {8, 10, 16, 20, 24, 64) and learning rates {0.1, 0.05, 0.01, 0.005, 0.001, 0.0001). Latent state dimension=64 and learning rate=0.0001 results in the best performing model.
7.0.10 T-PatchGNN
We perform grid search over learning rates {0.11 0.05, 0.01, 0.005, 0.001, 0.0001), time and node embedding
dimensions (4. 8, 16), number of patches= 10, 20). and latent state dimension {4, 8, 16, 32, 64), while fixing the number of heads in one transformer layer number of transformer layers = 1. We find that the configuration with learning rale=0.0001, time and node embedding dimension=16, number of patches=10, latent state dimension=16 results the best validation MSE.
7.0.11 GraFITi
We perform grid search over learning rates {0.1, 0.05, 0.01, 0.005, 0.001), latent state dimension {4,8, 10, 16, 20, 32, 64]; number of layers 21 4] and number of attention heads = (1, 4]. We re- port that the configuration with 1r=0.005, latent state dimension=64, mumber of layer=2 and number of attention heads-1 I results in the best validation MSE.
7.0.12 GRUwE
We perform search over learning rate= 0.025, 0.05, 0.01, 0.005). the hidden state dimensions= {10, 16, 20, 24, 32, 64) (to be comparable to other models). For MIMIC-III-Small, we use the configuration that results in the best validation MSE: learning ratc=0.001 and latent state dimension-32.
7.0.13 HyperIMTS
We perform grid search over learning rates 10.1, 0.05, 0.01, 0.005, 0.001), latent state dimension
{8, 16, 20, 32, 64, 128), number of layers {1, 2, 4] and number of attention heads {1, 4]. We report that the configuration with 1r=0.005, latent state dimension=32, number of layer=2 and number of attention heads=4 results in the best validation MSF.
8 Hyperparameters for Next Observation Prediction Task on MIMIC-III-Large
We keep the following hyperparameters constant across all methods: batch size=1 (to be able to handle the longer sequences in the MIMIC-III dataset within GPU memory constraints), number of training epochs-20, learning rate decay=0.99 and gradient clipping enabled. Below are the model specific experiments we carried out.
8.0.1 m TAND
Based on MIMIC-III experiments in Shukla & Marlin 22021p, keep the following hyperparameters fixed: time embedding dimension=122 128. We perform grid search on the hidden state dimension = {16, 32, 64). encoder hidden dimension = {16, 32, 64), the number of reference points = {64,95) and the learning rates=(0.01, 0.005, 0.001}. Note that for both hiddon state dimensions as 64 and number of reference points as 95, we hit the memory limit on our GPU for batch size=1. Nonetheless, the resulting number of param- eters (=174K) for this configuration higher than that of our proposed model. Our validation results show that latent state dimension=64, number of reference points=95 and learning ratc=0.001 performs the best across all combinations.
8.0.2 GRU-D
We perform grid search over hidden state dimension=(16,33 32,64) and learning rates=(0.01, 0.005, 0.001), We report that hidden state=32 and learning rate=0.001 performs the best on validation set and use it to report the final results.
8.0.3 Latent ODE
We use Latent ODE model with ODE-RNN encoder. We perform a grid search on latent state dimension=(16, 32, 64), recognition network dimension=(16. 32, 64), number of GRU units=(16,32 64), number of generation (2,3),=(2,3), number of recognition layers=(2,3) and learning rate=
26

---

Published in Transactions on Machine Learning Research (01/2026)
{0.01, 0.005, 0.001}. The configuration that performs the best on validation split with latent state dimension, recognition network dimension, number of GRU units set to 32; learning rate of 0.001 and generation and recognition network layers set to 3. Other hyperparameters that were kept fixed are: batch size= 1, learning rate decay=0.99 with gradient clipping.
8.0.4 F-CRU
We perform a grid search over the latent state dimensions= (16,32, 64) and learning rates=(0.01, 0.001). We set latent observation dimension as half the size of latent state di- mension. Number of basis matrices = 20, and Gradient clipping enabled. Encoder consists of 3 x( FullyConnected(50) + ReLU + Layer normalization) followed by linear output for latent observation and output; square activation for latent observation variance. Decoder consists of 3 x (FullyConnected(55) + ReLU + Layer normalization) followed by linear output. Decoder output variance consists of [FullyConnected(55) + ReLU + Layer normalization) followed by linear output and square activation. Activation function for transition function is ReLU, After performing the grid search, the best configuration of hyperparamoters are: latent state dimension=64, and learning rate=0.001.
8.0.5 CRU
Fixed hyperparameters for CRU are: variance activation for "oooder='square"" decoder='square", tran- sition='relut" number of basis matrices=20, and the same encoder and decoder network architecture used in f-CRU (above). We perform search on latent state dimension=-16. 32, 64) and learning rate = {0.1, 0.01, 0.001, 0.0001). We report that the latent dimension=32 and learning rate=0.0001 performs the best on validation set.
8.0.6 ODE-RNN
For ODE-RNN, we search over latent state dimension=(16,.33 12,1,,32000 64} and the learning rates= (0.1, 0.05, 0.01, 0.005, 0.001). We find that the configuration of latent state dimension=32 and learning rate = 0.005 works the best on the validation set.
8.0.7 ContiFormer
For ContiFormer, perform a search over the latent state dimension = (16,32, 643 and learning rate = {0.1, 0.05, 0.01, 0.001}. Our experiments show that latent state dimension=64 and learning rate=0.001 achicves the best performance on the validation set.
8.0.8 GRU-A;
For GRU-A,; we search over latent state dimensions of the GRU =(16, 64) and learning rates= (0.1, 0.05, 0.01, 0.005, 0.0013. We find latent state dimension=16 and learning rate=0.001 be the best performing configuration.
8.0.9 RKN-A,
Our RKN-A, implementation uses the same encoders and decoders architecture the CRU model. Keep- ing other parameters fixed, we search over latent state dimensions = {16, 32, 64] and learning rates = {0.001, 0.0005, 0.0001, 0.00005). Note that we search over smaller values of the learning rate because, the model would not converge for higher ones (we get "NaN' during optimization for higher rate). Latent state dimension=32 and learning ratc=0.00005 results the best performing model.
8.0.10 T-PatchGNN
We perform grid search over learning rates (0.1, 0.05, 0.01, 0.005, 0.001), time and node embedding dimensions {4, 16), number of patches= 4} (more number of patches results in GPU OOM issue). and latent state dimension = {4, 8, 10, 12), while fixing the number of heads in one transformer layer number of transformer layers = 1. We find that the configuration with learning rate=0.001, time and node embedding dimension=20, number of patches=2, latent state dimension=12 results in the best validation MSE.
8.0.11 GraFITi
We perform grid search over learning rates {0.1, 0.05, 0.01, 0.005, 0.001), latent state dimension
(16, 32,622 64), number of layers (1) (more number of layers MIMIC-IIT causes 0OM) and number of
27

---

Published in Transactions on Machine Learning Research (01/2026)
attention heads {1}. We report that the configuration with 1r=0.005, latent state dimension=64, number of layer=1 and number of attention heads=1 results in the best validation MSF.
8.0.12 GRUWE
After searching over hidden state=(10, 16, 20} and learning rates= 0.05, 0.01, 0.005, 0.001]. For the prediction tasks, we find the combination of 16 and 0.001 perform the best.
8.0.13 HyperIMTS
For HyperIMTSS we unable to experiments the MIMIC-III Large dataset because even the smallest tested configuration (batch size = 1, latent dimension 8, number of layers = 1, number of attention heads = 2) consistently triggered GPU memory errors during training. It is very likely due to long sequences in the MIMIC-III Large dataset as reported Table
9 Hyperparameters for Event Prediction task
We perform a grid search to tune the hyperparameters for the event prediction models: GRUwE, NHP, ATTNHP, THP and RMTPP. For RNN-based models (i.e., GRUwE, NHP, RMTPP), hidden size, learning rate and batch size are the relevant hyperparameters. For Attention-based models (i.e., ATTNHP, THP, SAHP), we optimize hidden size, learning rate, batch size, number of attention layers and embedding size. For all our experiments, we use Adam optimizer during training.
Table 6: Grid search configurations for hyperparameter tuning
Model Type
Hyperparameter
Values
hidden size learning tate
{16, 32, 643 (1c-1, 5e-2, 1e-2, 5e-3, 1o-3)
RNN-based
(NHP, RMTPP, GRUwE)
batch size
{32, 64, 128, 256}
hidden Size
[16, 32, 64)
embedding size
{4, 8, 16)
Attention-based (ATTNHP, THP)
mmm. layers
{1, 2, 3]
learning rate batch size
{1e-1, 5e-2, 1e-2, 5e-3, 1e-3)
(32, 64, 128, 256)
The grid search values for each parameter reported in Table 6 For each dataset, we perform indepen- dent hyperparameter scarch and train the model for up to 30 epochs with early stopping based validation log- logelioel We select the model configuration that achieves the highest validation log- liellioel
10 Rank- based analysis for next event prediction.
We use the rank metrics as described in Table to report the average rank statistic in the main results of paper.
Table 7: Model performance comparison for the next event prediction on the Taxi, Retweet, and Amazon datasets. Lower rank means better model for RMSE and ER metrics.
Taxi
Retweet
StackOverflow
Amazon
Model
RMSE (Rank)
ER (Rank)
RMSE (Tank)
ER (Itak)
RMSE (Rank)
ER (Rank)
RMSE (Rank)
ER (Rank)
FullyNN
0.373 (7)
N.A.
21.92 (1)
N.A.
1.375 (6)
N.A.
0.615 (5)
N.A.
NHP
0.369 (1)
9.22 (3)
22.32 (5)
40.25 (1)
1.369 (2)
55.78 (4)
0.612 (2)
68.30 (5)
A-NHP
0.370
11.42 (6)
22.28 (3)
41.05 (4)
1.370
55.51
0.612 (2)
65.65 (1)
(1)
(2)
(5)
(1)
(1)
55.60 (3)
(2)
66.72 (2)
SAHP
0.372 (6)
9.75 (4)
22.40 (7)
41.60 (5)
1.375 (6)
56.10 (3)
0.619 (6)
67.70 (4)
RMTPP
0.370 (4)
9.86 (5)
22.31 (4)
44.10 (6)
1.370 (4)
57.50 (6)
0.631 (7)
73.66 (6)
GRUWE
0.369 (1)
8.58 (1)
22.21 (2)
40.81 (3)
1.360 (2)
555.34 (1)
0.611 (1)
67.24 (3)
28

---

Published in Transactions on Machine Learning Research (01/2026)
11
Computational Cost Analysis
GRUWE 13.46
GRUWE T-PatchGNN
1.89
T-PatchGNN
8 9.54
22.24
GRU-A+
32
GRU-A
1.61
RKN-A
CRU
20.45
58.18
RKN-A
CRU
2.08
3.75
ODE-RNN ContiFormer
209.90
ContiFormer
ODE-RNN
1.52
6.58
620.60
Latent ODE
Latent ODE
159.56
GRU-D mTAND
1.31
GRU-D
14.31
8.12
mTAND
10.27
f-CRU
2.29
f-CRU
46.28
GraFITI
14.53
GraFITi
11.32
HyperIMTS
16.78
HyperIMTS
12.23
10"
10"
GPU memory (GBs) usage during training in log scale
Avg. train time (s/epoch) in log scale
(b)
GRUWE 1.18
T-PatchGNN
0.89
GRU-A,
0.84
5.669
39.685
RKN-A
0.98
CRU
3.18
ontrr Inferenca time (ne)
ODE-RNN ContiFormer
9.11
150
31.91
Latent ODE
10.01
28
GRU-D
1.04
2J
8.420 10.736
8.42010.736
mTAND
GraFITi
1-CRU
1311
1.49
1.66
0.382
0.045
0.769
0.972
entre
6.435
0.061
HyperIMTS
1.43
mTANDD
GRUWE
T-PatchGNN
GraFIT
HyperiMTS
mTAND
GRUWE
T-PatchGNN
GraFiTi
HyperIMTS
10'
Avg. inference time (ms/instance) log scale
(c)
(d)
Figure 6: Computational cost analysis on the Physionet dataset. (a) Compares the average train time per epoch in seconds. (b) Peak GPU memory usage in GBs during training. (c) Average inference time milliseconds per instance. (d) Comparison of the inference time (left) and memory consumption (right) in an online deployment.
We provide comprehensive comparison of the computational complexity across all models investigated in our study in the Figure D We evaluate and contrast the training times, inference times, and peak GPU memory usage during training for each model when performing next observation prediction task on the Physionet dataset, The average train and inference times are computed by taking the average across multiple epochs and samples. Our analysis reveals a clear trend in terms of the train time. We note that methods (ContiFormer, ODE-RNN, Latent ODE) that rely on numerical solvers exhibit the longest training times, followed by models (CRU, I-CRU, RKN-A,) that assume linear dynamics. Recurrent models (GRUwF, GRU-D, GRU-A,) demonstrate moderate training times. Lastly, models that can be parallelized in the time dimension (mTAND, T-PatchGNN, GraFITi, HyperLMTS) are the fastest to train. However, it is crucial to note that the models with the fastest training times (mTAND, T-PatchGNN, GraFITi, HyperIMTS) come with a significantly higher memory requirement as illustrated Figure 6b
Having trained the time series model on retrospective EHR data, our primary goal to deploy it as an early warning system to prevent adverse patient conditions in the ICU. This model operates in an inherently online context, characterized by continuous stream of (near) real-time data. This data stream includes critical patient information such vital signs and medication administration records, which the deployed model processes continuously. To simulate this online environment, obscrvations from 100 randomly selected patients in an online manner. The models are evaluated based on total inference time and peak GPU memory usage during inference. In our analysis, we compare GRUwE with models that offer the best train times and are among the top performing models in next observation prediction task: mTAND, GraFITi and T-PatchGNN. We observe that since mTAND, GraFITi, T-PatchGNN and HyperIMTS do have a Markovian state representation, it needs to buffer the past observations to make the inference. This results in higher inference cost both in terms of time and memory. Moreover, these
use the Physionet dataset, streaming
29

---

Published in Transactions on Machine Learning Research (01/2026)
costs grows over time. In contrast, models that consists of Markov state representation such GRUwE, are independent of the past observations thus, maintaining a constant, low time and memory requirement.
12 Computing Infrastructure
We used server machine to deploy the experiments reported in the paper. This machine is equipped with 100GB memory, one NVIDIA L40S GPU, with Intel Xeon Platinum 8462Y- + @ 2.80 GHz processor and 16 CPU cores.
30


Reviews:
[{"review_id": "lOYaRKM5fM", "reviewer": "Reviewer_xkyY", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper challenges the trend of using increasingly complex (thus computationally expensive) architectures for modeling irregularly sampled time series. The paper further hypothesizes that classic architectures, if carefully modified, can just as good if not better. To support the claim, this paper modifies GRU and proposes GRUwE that maintains Markov state representation and uses a dual reset mechanism to handle the irregular time gaps. Empirical study shows the proposed approach is competitive comparing to strong baselines on various dataset. Runtime comparison also shows the much better efficiency of the proposed approach. \n\n__Strengths__\n- The proposed approach is technically sound.\n- The empirical study provide evidence that supports the effectiveness and efficiency claim about the “carefully modified” “classic architecture”.\n- The paper is easy to follow.\n\n__Suggestions__\n- Lack of Ablation Studies. The paper proposes an architecture that modifies the standard GRU with several new design choices, most notably the time-triggered exponential decay mechanism. However, the experiments only compare the final GRUwE model against other baselines. The paper does not provide an empirical breakdown (an ablation study) to demonstrate which of these new components are responsible for the performance gains. It's unclear if the entire design is necessary, or if one specific part is doing all the heavy lifting.\n- The paper's motivation hinges on the strong claim that \"the benefits of...modern architectures...remain poorly understood,\" but it doesn't provide citations or evidence to support this assertion. Concurrently, while the paper provides a theoretical analysis of GRUwE, it doesn't sufficiently connect this theory back to the empirical results to convincingly argue that GRUwE is, in contrast, \"well understood.”\n\n__Note__: as I am not familiar with this line of research (irregular Multivariate time series prediction), I cannot assess the novelty/significance of the proposed approach.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper are in general convincing and evidences for supporting the claims are clear. The only concern is listed above in \"suggestions\" section.", "audience_yn": "Yes", "audience_explanation": "The research topic addressed by the paper is well within the scope of the machine learning community.", "requested_changes": "Please see suggestions section above.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "THSwmkWmex", "reviewer": "Reviewer_x2HC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces GRUwE, a novel yet intentionally simple recurrent model for irregularly sampled multivariate time series. The proposed architecture is evaluated through a comprehensive empirical study on two distinct tasks, demonstrating superior performance over existing methods. Additionally, the paper analyzes computational efficiency, showing that GRUwE is orders of magnitude faster and more memory-efficient in real-world online inference settings, compared to complex baselines that must re-process entire histories.\n\n### Strengths:\n- The model's simplicity and ease of implementation\n- The computational efficiency in the online setting is a massive and well-proven strength, making it one of the few models tested that appears genuinely suitable for real-time deployment.\n- The strong empirical performance \n- The model is shown to be particularly effective on NMAR (Not Missing at Random) data, which is common in real-world scenarios like healthcare and a known challenge for many models.\n\n### Weaknesses:\n- The authors explicitly state that the model lacks a built-in mechanism for uncertainty quantification, which is a feature present in some of the more complex baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "- Accuracy Claim: The claim of competitive to superior performance is supported by Table 1 and Table 2. \n\n- Efficiency Claim: The claim of significantly lower computational overhead for online deployment is strongly supported by Figure 4a. It shows GRUwE is orders of magnitude faster (0.045 ms vs. 5.669 ms for T-PatchGNN) and uses dramatically less memory (0.061 GB vs. 39.685 GB) than competitors in an online setting.\n\n- Simplicity Claim: The claim that GRUwE is easy to implement is visually confirmed by the pseudocode in Figure 4b, which shows the core mechanism in just four lines of code", "audience_yn": "Yes", "audience_explanation": "It addresses the persistent challenge of modeling irregularly sampled multivariate time series , which is crucial in domains like healthcare. The paper presents a counter-intuitive finding that a simple, revisited RNN model can achieve competitive to superior performance compared to complex SOTA methods. The model is proven to have significantly lower computational overhead in real-time scenarios, a finding of high practical value to the ML community", "requested_changes": "- The paper's motivation hinges on the claim that the benefits of modern, complex architectures are 'poorly understood.' Could the authors please be more specific about which aspects of their benefits they believe are poorly understood?\n\n- The learnable exponential decay is the core of the model's novelty. A small ablation study, even on one dataset, would be very insightful. For instance, how does the full GRUwE model compare to a version with a fixed, non-learnable exponential decay? This would more concretely demonstrate the value of learning the decay parameters.\n\n- Specify the Number of Random Seeds: The paper states results are averaged over \"multiple distinct random seeds\". For clarity, reproducibility, and to assess statistical robustness, the authors should specify the exact number of seeds used for these experiments.\n\n- Clarify Table Highlighting: The highlighting of top results in the tables (e.g., Table 1) appears inconsistent (how many). Please use a clear, uniform rule.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "XGEcGNspYy", "reviewer": "Reviewer_b7Hx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper revisits recurrent neural network architectures for irregularly sampled time series, arguing that recent complex models may not justify their added complexity. The authors introduce GRU with exponential basis functions, which supports both regression and event prediction in continuous time through a Markov state representation with two reset mechanisms. The method demonstrates strong empirical performance while offering practical advantages in simplicity, ease of implementation, and fewer hyperparameters.\nThe abstract is clear and well-written, making the contribution easy to understand. The problem definition and analysis of exponential basis functions are technically sound, and results appear promising. However, the paper has a critical structural flaw: the related work is relegated to the appendix, leaving only a vague list of methods in the main text without adequate explanation of why this work matters or what issues exist with current approaches. This is particularly problematic given the paper's central claim that existing architectures are poorly understood.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The technical content and experimental results appear sound, but the support for the paper's core motivation is missing.", "audience_yn": "Yes", "audience_explanation": "This work explores a timely question in machine learning: Can simpler models achieve performance on par with their more complex counterparts? The answer holds significant practical value. By simplifying implementation and tuning without sacrificing performance, these methods could have broad appeal. Methodologically, the integration of exponential basis functions with GRU architecture and a dual reset mechanism introduces a concrete, actionable innovation that practitioners can readily adopt.", "requested_changes": "- Substantially expand related work in the main paper. \nThis is the paper's most significant flaw and directly contradicts its thesis. Move key content from the appendix and replace the current list of methods without explanations with critical analysis explaining what problems exist with current architectures, why they are \"poorly understood\" with evidence, and how the proposed approach addresses these gaps. This section must make a compelling case for why revisiting RNNs matters.\n- Provide evidence for the \"poorly understood\" claim. This is the central motivating claim and requires concrete examples or systematic literature review demonstrating that modern architectures lack clear understanding. If unsupportable, reframe the motivation.\n\n- Add complete citations to Table 1.\nInclude missing recent baselines. Add HyperIMTS (ICML 2025) and other recent methods from 2024-2025. Provide explicit justification for any exclusions.\n- Clarify Figure 1(a). Add clear y-axis label.", "broader_impact_concerns": "No significant concerns are identified. The paper addresses a methodological question without obvious negative ethical implications and does not involve sensitive applications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)