HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
Authors: Boyuan Li
Yicheng Luo
Zhen Liu
Junhao Zheng
Jianming Lv
Qianli Ma
Venue: ICML 2025 poster (2025)
Keywords: Irregular Multivariate Time Series
Time Series Forecasting
Hypergraph Neural Networks
Primary area: applications->time_series
TL;DR: We propose HyperIMTS, an efficient hypergraph neural network achieving state-of-the-art performance in irregular multivariate time series forecasting task.

Abstract: Irregular multivariate time series (IMTS) are characterized by irregular time intervals within variables and unaligned observations across variables, posing challenges in learning temporal and variable dependencies. 
Many existing IMTS models either require padded samples to learn separately from temporal and variable dimensions, or represent original samples via bipartite graphs or sets.
However, the former approaches often need to handle extra padding values affecting efficiency and disrupting original sampling patterns, while the latter ones have limitations in capturing dependencies among unaligned observations.
To represent and learn both dependencies from original observations in a unified form, we propose HyperIMTS, a **Hyper**graph neural network for **I**rregular **M**ultivariate **T**ime **S**eries forecasting.
Observed values are converted as nodes in the hypergraph, interconnected by temporal and variable hyperedges to enable message passing among all observations.
Through irregularity-aware message passing, HyperIMTS captures variable dependencies in a time-adaptive way to achieve accurate forecasting. 
Experiments demonstrate HyperIMTS's competitive performance among state-of-the-art models in IMTS forecasting with low computational cost.
Our code is available at [https://github.com/qianlima-lab/PyOmniTS](https://github.com/qianlima-lab/PyOmniTS).

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
Boyuan Li Yicheng Luo'
Zhen Liu'
Junhao Zheng'
Jianming Lv! Qianli Ma
Abstract
ing has been extensively studied (Nie et al., 2022; Zhang & Yan, 2022; Zhou et al., 2021; Yu et al., 2024; Yi et al., 2023), these methods typically assume the input to be fully observed and pay less attention to potential sensor malfune- tions, varying sampling sources, or human factors in reality, These factors can result in Irregular Multivariate Time Series (IMTS), which are characterized by irregular time intervals within each variable and unaligned observations across vari- ables. These characteristics make it challenging to provide accurate forecasting on IMTS for informed decision-making and planning.
Irregular multivariate time series (IMTS) are char- acterized by irregular time intervals within vari- ables and unaligned observations across variables, posing challenges learning temporal and vari- able dependencies. Many existing IMTS mod- els either require padded samples learn sepa- rately from temporal and variable dimensions, represent original samples via bipartite graphs or sets. However, the former approaches of- ten need to handle extra padding values affect- ing efficiency and disrupting original sampling patterns, while the latter ones have limitations in capturing dependencies among unaligned ob- servations. To represent and learn both depen- dencies from original observations in unified form, we propose HyperIMTS, a Hypergraph neural network for Irregular Multivariate Time Scries forecasting. Observed values are converted as nodes in the hypergraph, interconnected by temporal and variable hyperedges enable mes- sage passing among all observations. Through irregularity aware message passing, HyperIMTS captures variable dependencies in a adaetide way to achieve accurate forecasting. Experiments demonstrate HyperIMTS's competitive perfor- mance among state-of-the-ard models in IMTS forecasting with low computational cost. Our code is available at https: / / hhub.ccomm qi anl ima 1ab/ Pyommi Ts.
In studies for IMTS, one category of methods pads the series in the sample space have same length across vari- ables (Rubanova et al., 2019; Zhang et al., 2021; Tashiro et al., 2021; Zhang et al., 2023a; 2024), as illustrated in Fig- ure 1 (a) and (b). Padded series are either time-aligned or patch-aligned, allowing for effective capturing of dependen- cies along both temporal and variable dimensions. However, such padding scheme can increase the amount of data to be processed, primarily because observation timestamps for different variables can span widely separated time periods. Furthermore, some methods rely on relative positions of observations instead of timestamps to represent temporal information, which might disrupt the original sampling pat- tern. It should be noted that padding refers to adding zeros or predicted values in the sample space before inputting IMTS into the neural network, differing from the continu- ous latent space used in ODE-based methods (Rubanova et al., 2019; Biloš al., 2021; Mercatali et al., 2024).
Instead of relying padded series, another category of IMTS analysis methods proposes using sets or bipartite graphs to represent original IMTS samples, as illustrated in Figure (c) and (d). Set-based methods view obser- vations as unordered tuples in set (Horn et al., 2020), while bipartite graph approaches represent channels and timepoints disjoint nodes connected by edges (Yalavarthi et al., 2024), both of which only express observed values for high efficiency. However, sets typically do not account for correlations among observations, and bipartite graphs are unable to propagate messages between variables without shared timestamps. These methods still have limitations in capturing dependencies on original IMTS.
1. Introduction
Multivariate Time Series (MTS) are prevalent in various fields such as healthcare, weather, and biomechanics (Zhang et al., 2023b; Shukla & Marlin, 2021). While MTS forecast-
"Scoool of Computer Science and Engineering, South China University Technology, Guangzhou, China. Correspondence to: Qianli Ma <qianlima @ scut.edu.cn".
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
tion preserved in the graph to adaptively capture both time-aware and overall variable dependencies, enabling irregularit--wware learning and accurate forecasting.
Oeservation
O Pudding
Foresasting quary
(oX(s)
0 HG
We build a unified, extensible, and highly flexible code pipeline for fair IMTS forecasting benchmark- ing across time series models from various fields and tasks, covering twenty-seven
a
[G) ftg)
tfae-oof-thearr models and five IMTS datasets. Extensive empirical results demonstrate the low forecast error and high efficiency of HyperIMTT.
G 5
Paich
Fatch 2
(5))
(5)
2. Related Work
(1)
(6)
2.1. Irregular Multivariate Time Series Modeling
Set
Existing efforts on IMTS mainly focus on classifica- tion (Che et al., 2018; Shukla & Marlin, 2020; Zhang et al., 2021: 2023a; Horn et al., 2020; Shukla & Marlin, 2018) and imputation tasks (Che et al., 2018; Rubanova et al., 2019; Shukla & Marlin, 2020; Tashiro et al., 2021). In recent years, an increasing number of studies have paid attention to IMTS forecasting (Tashiro et al., 2021: Schirmer et al., 2022; Yalavarthi et al., 2024; Zhang et al., 2024; Mercatali et al., 2024). From a data preprocessing perspective, exist- ing works on IMTS can be broadly categorized into padding and non-padding methods. The former ones typically repre- sent input time series as matrices with temporal and variable dimensions, and they design model components to learn de- pendencies along both dimensions (Shukla & Marlin, 2020; Tashiro et al., 2021; Schirmer et al., 2022). Models based on RNNs (Che et al., 2018). ordinary differential equations (ODEs) (Rubanova et al., 2019; Biloš et al., 2021; Mer- catali et al., 2024), transformers (Zhang et al., 2023a), and graph neural networks (Zhang et al., 2021; Luo et al., 2024; Zhang et al., 2024) are commonly used. While these models can achieve outstanding performance, they often require handling more input data, which can affect the efficiency during preprocessing and training. The other non-padding approaches use bipartite graph (You et al., 2020; Yalavarthi et al., 2024) or set (Horn et al., 2020) to represent IMTS. Their sparse representations can handle IMTS samples with- out padding, but their model architectures restrict the ability to capture dependencies in the original IMTS.
Figure 1. Existing methods for processing IMTS sample. (a) Canonical padding approach, which significantly increases the amount of data. (b) Patch-aligned padding approach, which also increase the amount of data. (c) Set views all observations as its unordered items. (d) Bipartite graph uses observation edges to connect variable and time nodes. It cannot model dependencies between variables without aligned observations, like V/2 and Vs, which require shared timestamps.
To represent original observations and comprehensively cap- ture their dependencies, we propose HyperIMTS, hyper- graph neural network for IMTS forecasting. HyperIMTS represents observations and their dependencies in a unified hypergraph, transforming the IMTS forecasting task into the node prediction task. views observations as nodes in the hypergraph, each associated with its corresponding timestamp and variable. Observations belonging to the same variable are connected by a multi-connected hyperedge, just as observations sharing the same timestamp. Temporal de- pendencies are captured through node-to-hy hhyeeeeee and hyperedge-to-nee -odde message passing, while variable depen- dencies further benefit from trrredee-tto-yypredege con- nections. Moreover, given the difficulty in capturing vari- able correlations under different time alignment situations, we calculate time-aware similarities using nodes and overall similarities using hyperedges to realize irregularity-wwar learning of variable dependencies.
Our major contributions are summarized as follows:
2.2. Using Graphs for MTS
Graphs are increasingly popular in time series analysis (Yi et al., 2023; Wen al., 2023; Wu et al., 2020; Cini et al., 2021; Han et al., 2024), which usually represent variables as nodes and their dependencies as edges. Although this type of graph demonstrates great potential for learning multivari- ate dependencies, it requires time-aligned or patch-aligned samples for input encoding, which increases the amount of data for IMTS samples. The hypergraph is a variant where
We propose a new hypergraph modeling approach to represent both observed values and their dependencies in IMTS, which does not require padding and remains extensible for dependency learning.
Based on the hypergraph representation, we propose HyperIMTS, a hypergraph neural network for the IMTS forecasting task. It leverages timestamp informa-
2

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
4. Methodology
Observation Node
My Temporal Variable Hyoweedge
We first detail the efficient hypergraph representation for IMTS samples in Section 4.1. Subsequently, we explain how to leverage it for the IMTS forecasting task in Section 4.2.
V
VA
1/2
4.1. Efficient Hypergraph Representation for IMTS
11 1. to
Instead of using padding or patching, we convert original observations into node embeddings within the hypergraph. thereby improving efficiency in both data preprocessing and model training. As shown in Figure 2, the hyper- graph is defined as G := (V.E). The observation node embeddings are expressed as V == {ıylů = 1,..., M), and two types of hyperedge embeddings E := Ey n Ey are defined, including temporal hyperedge embeddings Eime := {Eelt = 1, .... T) and variable hyperedge embed- dings Evar == fealu 1,.... U}. The topology of hy- pergraph can be represented using two incidence matrices: H E TR M xT for temporal hyperedges, and HU E Mx for variable hyperedges. Entries H5e in H are defined as:
Figure 2. Illustration of the proposed efficient hypergraph repre- sentation for IMTS. (a) Original IMTS sample, where V1, V/2 and V/3 represent three different variables. (b) The corresponding hy- pergraph representation. From top to bottom, variable hyperedges, observation nodes, and temporal hyperedges are displayed. Each observation node connects to the associated variable hyperedge above and the temporal hyperedge below. Gradient color lines between hyperedges indicate hyperregeeto--hypeeedd message passing.
1, V/ E H 1
{1, 1000 0. uy $ e
hyperedges can connect multiple nodes, facilitating more types of message passing. Its ability to represent data at different granularities and model their complex interactions has attracted researchers in various fields (Zhou et al., 2006; Yan et al., 2020: Gao et al., 2023; Yu et al., 2012). Hyper- graph neural networks are still in their early stages for time series analysis (Shang et al., 2024; Luo et al., 2022; Li et al., 2022; Sawhney et al., 2021). They usually assume inputs to be fully observed, which makes them not well-suited for IMTS.
And entries Hjs in H are defined in the same way as Eq. (2). Unlike the edge in traditional graph structures which can only connect two nodes, a hyperedge can connect an arbitrary number of nodes, enabling hypergraphs to cap- ture more complex relationships among nodes. Using the above notations, the IMTS forecasting problem is framed as the node prediction task in hypergraphs, where nodes representing forecast targets are the ones to predict.
We initialize node embeddings by encoding the observed values 2, of the i-th sample through non-linear mapping
3. Problem Definition
(FFF.((F (-)) into Pabx-dimension embedding V E
We consider an IMTS dataset D : {S = 1,.., n con- sisting of n samples, where S, is the i-th sample. With a total of T timestamps and U variables, each IMTS sample can be denoted as a set containing M observation tuples S, := {(t,, Zj. ū5 = M), where t, {1,..., T),. By E R. and uj E {1....U] U/} represents the timestamp, observed value, and variable indicator respectively. For the IMTS forecasting task, given a split timestamp l.s, an IMTS sample is divided into a lookback window X; := {(ty, 25,22,,,,) = 1.... M,t, S ts} and a forecast window )1= (1)),,,),
R R Pax:
== =eRLLUFFFuaa obbs (b (Z.)).
(3)
where the values of forecast targets are initialized 0. For temporal hyperedge embeddings from timestamp set Ti we initialize them using sinusoidal encoding after linear mapping FFume (-) to obtain the P Pame- dimensiooa temporal embedding Erime E RR T Pere:
Egiee
sin(FFrinn (Ti)).
(4)
For variable hyperedge embeddings from variable set Us. we use learnable parameters Waar of shape U x Pva for initializations, where Pear denotes the variable embedding dimension:
1, ...,, M, 1s]. The forecast query 9 Q is derived by combining (ty. Uy) of the j-th observation tuple within the forecast window. We aim to learn a forecasting model F(-), such that given the lookback window X; and forecast query Qi as input, it accurately predicts the corresponding observed values Z,:
E ReLUU( war).
(5)
4.2. Forecasting with HyperIMTS
The overview of our proposed model, HyperIMTS, is il- lustrated in Figure 3. It leverages the hypergraph represen-
F(X, 2,) + 2i.
3

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
Forecasted IMTS
Temporal Node et Ady Temporall Varieble Hyperedge
Message Passing
Iiregularity- aware Similarity
V5
0.78
5
5
4 ls
Xa
X
Output Decoding
Hypergraph Attention Network
query
Similartt 0.9
Similenity
0.6
Concat
Node Hyperedge Encoding
Concat
Multi-heed Attention
Scaled Produuct
Product
Flattening
query
(a) mroee-o-hhypeedde
V22
V1
Vi V:
Time-aligned Observations Only
Variable Hyperedge
Concat
Weights
5 5
G
fille
Input IMTS Sample
(c) syperedge-to-noodd
(b) hyperedge-to-hyperedee
Figure 3. The architecture of HyperIMTS. It first converts input IMTS samples with empty forecast targets into the proposed efficient hypergraph representation. Three types of message passing are used soquentially: Temporal and variable hyperedge embeddings are updated via mode-to-hypereee hyperedge message passing, where time-aware and overall variable similarities are merged based time alignment; (c) Both temporal and variable hyperedge embeddings are used for unified hyperedge-to-nod updates.
message passing; (b) Inter-variable correlations are modeled during iregularity-awaa hyperedge-to-
tation detailed in Section to represent IMTS samples. Learning for temporal and variable dependencies is realized using three types of message passing in the hypergraph, where hodeett--y is introduced in Section 4.2.1, hyperedgeeto--yypeeedge is discussed in Section 4.2.2, and hyperedge-to-node is presented in Section 4.2.3. It should be noted that temporal dependencics are learned through message passings among observations of the same variable, and variable dependencies learned through message passings between different variables.
beddings Eiime can be expressed as:
q q'kaT
O = || H Softmax
Vd/H )ra
(6)
E time
O (e)LU(FF((O)),
(7)
where || denotes the concatenation of results from H attention heads, and d denotes the embedding dimension of Küme- We note that variable hyperedge embeddings are in- cluded in both keys and values identify the corresponding variable for each observation.
4.2. 1. TEMPORAL AND VARIABLE HYPEREDGE UPDATE
In this section, we introduce hode-to-hyperede message passing for updating temporal and variable hyperedge em- beddings, as illustrated in Figure (a). HyperIMTS ini- tializes temporal hyperedge embeddings with sinusoidal encoding, making them learnable account for the vary- ing number of observations at each timestamp. With tem- poral hyperedge embeddings used as multi-head attention FF,(Eun (Eime) and concatenating observation node embeddings with variable hyperedge embeddings as keys kd FF& [ and values / FF,( (F Evar). the message passing for updated temporal hyperedge em-
Besides updating temporal hyperedge embeddings, vari- able hyperedge embeddings are also updated using multi- head attention. The query is defined as q = FF, (Evar), and timestamp information included in the keys k FFy (V] Etie)) and values vhr FF, [ guish relative positions. Similar Eq. (6) and (7), we obtain the updated variable hyperedge embeddings E
queries q² FF,
distin-
By updating the embeddings of temporal and variable hy- peredges, messages from observations are now aggregated onto hyperedges, and we describe their further propagation in the following sections.

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
4.2.2. INTER- VARRABE MESSAGE PASSING
where ò is initialized to 0.5. The logic behind a is to pri- oritize Sab. over S if there are more aligned observa- tions than unaligned ones, and the variables are similar. If Sabs O, it indicates that all observations among the vari- ables are unaligned, such as V/2 and V/3 in Figure 3. In this case, we maintain their connection for message passing using the overall variable similarity Svar-
In this section, we introduce the irregularity- eyperedge-t--hyperede message passing among variables, as illustrated in Figure (b). It begins by determining cor- relations among variables, which are realized through the calculation of variable similarities. We first calculate the overall variable similarities among all connected observa- tions. For two different variables Uo and up taken from L.., U, the dot-product similarity between their corre- sponding variable hyperedge embeddings eu. be expressed as:
By gathering all the similaritics SIMTS for each pair of vari- ables as entries in the attention map Avar. hyperedge-to- hyperedge message passing is implemented using scaled dot-product attention:
and Eug can
(8) where FF, and F are ferd-forward layers for queries and keys. However, overall variable similarity is series-level comparison, which does not leverage temporal information within the series, and may not perform optimally across all alignment scenarios. For example, similar variables Vi and 1/2 in Figure (b) have shared timestamps and 2 unaligned timestamps, which can lead to a positive but relatively small overall variable similarity due to the unbalanced number of observations. Based on the fact that if two time-aligned series are identical, then their time-aligned subseries are also identical, we can compare only the time-aligned ob- servations, leading to higher similarity scores that better promote message passing. Therefore, we refine the similar- ity calculation by calculating time-aware similarities using aime-aligned observations. Specifically, they Sabs are calcu- lated using the dot products of connected observation nodes and can be expressed as:
S = FF,((e.)) (FFF..))FF((Ce.)") FF..)C
E var
Softmax
FF (E
(12)
where E represents the updated variable hyperedge em- beddings after message passing, and d denotes the embed- ding dimension.
Through irregularity message passing among variable hyperedges, correlated messages are exchanged adaptive to the time alignment across variables. We discuss their further propagation to nodes in the next section.
4.2.3. NODE UPDATE
In this section, we introduce hyperedge-to-node message passing for node updates, as illustrated in Figure 3 (c). For the total L residual layers, we use variable hyperedge em- beddings Ex without inter-variable message passing in the first L 1 layers, which helps in learning temporal dependencies, also known as intra-variable dependencies. Variable hyperedge embeddings E propagate messages back to connected nodes, along with timestamp information from temporal hyperedge embeddings:
Sabs = [0"" V/
[M Mb
Torra
where 8h and th denote observation nodes for two dif- ferent variables, and only Tahaou number of observations with shared timestamps across variables are selected for comparison. It should be noted that while our method and padding-based methods shown in Figure I (a) and (b) all leverages alignment information, we employ distinct ap- proaches for better efficiency. Padding-based methods align IMTS by padding them before feeding them into neural net- works, leading to larger data volumes to be processed across all network modules. In contrast, our method only selects time-aligned observations during hyperedge-too sereodgg-t--hyppeee to-hyperedge message passing, thereby eliminating the need to handle large padded data in other network modules.
V SelfAtten(
(13)
V = ReLUUV + FFFndde (V E time EErr))
(14)
where SelfAlten(V) (V) performs a self-attention update on nodes V, and FFFode is the linear mapping layer for nodes. At the last layer of the residual structure, we use variable hyperedge embeddings E after inter-variable message passing during calculation, which helps in learning variable dependencies. It is implemented by replacing E with E in Eq. (14).
We then combine two similarities based on Tahardd the total number of timestamps Tioall in two variables, and a learnable threshold parameter 8 to obtain the irregular-aware variable similarity SIMTS:
Through the node update process, observations receive cor- related messages via temporal and variable dependencies, which also update the nodes to be predicted.
4.2.4. TRAINING OF HYPERIMTSS
S SIMTS aSobs + (1-1))) (1-a))Svar
(10)
Tawel Tira"
for Swrr 8 and Subs #
In this section, we introduce the process of obtaining fore- (11) cast values and training HyperIMTS. For the i-th sample where i n, we convert its updated node embeddings back
a=
0,
otherwise
5

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
Table 1. Experimental results on five irregular multivariate time series datasets evaluated by MSE (mean = std). The best and second-best results are indicated in bold and underlined, respectively, We employ four decimal places to better demonstrate standard deviations in results.
Algorithm
MIMIC-III
MIMIC-IV
PhysioNet'12 12
Human Activity
USHCN
higp
0,9726 + 0.0001
0.6616 + 0.0001
0.5025 +: 0.0179
0.2670 + 0.0198
0.2330 = 0.0003
MOIRAI
0,8655 +: 0.0000
0.4293 1 0.0000
0.4924 + 0.0000
0.1079 + 0.0000
1.2320 = 0.0000
FEDformer
0.7675 +: 0.0019
0.8008 0.0270
0.4150 0.0002
0.1619 + 0.0006
0.3237 0.0150
Ada-MSHyper
0.6680 + 0.0038
0,4217 + 0.0012
0,4143 + 0.0008
0.1480 + 0.0039
0.2406
0.0189
Autoformer
0.7082 + 0.0077
0.5963 + 0.0158
0.4137 + 0.0040
0.0983 + 0.0065
0.4367
0.0452
TimesNet
0.6515 + 0.0052
0.4355 0.0015
0.4076 0.0006
0.1206 +: 0.0025
0.2402
0.0116
iTranformer
0.7160 + 0.0081
0.5083 + 0.0095
0,3975 + 0.0023
0.0906 + 0.0020
0.4288
0.0634
Rogular
PourierGINN
0,7066 + 0.0069
0.4580 +: 0.0024
0.3946 + 0.0027
0.2822 + 0.0100
0,4244
0.0655
Mamba
0.6853 + 0.0014
0.6102
0.3906 0.0012
0.1253 0.0006
0.2100
0.0026
TSMixer
0.6127 + 0.0011
0.3566 + 0.0021
0,3817 0.000
0.1635 + 0.0028
0.2152
0.0144
PatchTST
0,6403 + 0.0019
0.2939 + 0.0009
0.3781 + 0.0009
0.0763 + 0.0001
0.2830 = 0.0754
Leddam
0.5935 + 0.0054
0.3697 0.0019
0,3754 0.0025
0.0913 0.0006
0.2887 0.0259
BigST
0,5855 + 0.0008
0.3579 = 0.0023
0,3425 + 0.0010
0.1671 + 0.0147
0.2019 0.0014
Reformer
0,5677 + 0.0007
0,3894 + 0.0023
0,3573 + 0.0003
0.0982 + 0.0013
0,2058 = 0.0042
Informer
0.5657 + 0.0037
0.4477 0.0067
0,3441 0.0006
0.0708 + 0.0003
0,2000 0.0014
Crossformer
0.6136 + 0.0128
0.3696 + 0.0025
0,3385 + 0.0053
0.1410 + 0.0208
0.2121
0.0035
PrimeNet
0.9900 + 0.0000
0.6650 + 0.0000
0.8035 + 0.0000
4.3697 + 0.0005
0,4925 = 0,0012
SeFT
0,9916 + 0.0001
0.6713 + 0.0002
0.7777 0.0003
14414 + 0.0023
0.3345 = 0.0010
mTAN
0.8963 + 0.0227
0.5346 0.0127
0,3889
0.0026
0.0925 + 0.0020
0.1929 0.0020
NeuralFlows
0.7167 + 0.0025
0,4737 + 0.0018
0.4196 + 0.0016
0,1680 + 0.0033
0.2007
0.0043
Irregular
CRU
0.7065 + 0.0028
0.4346 + 0.0022
0.6189
0.0012
0.1374 + 0.0040
0,2255 0.0086
GNeuralFlow
0.6950 + 0.0046
0,5005 0.0021
0,3881
0.0032
0.1734 + 0.0012
0.1832 0.0025
GRU-D
0.6125 + 0.0281
0.6622 + 0.0018
0,3462 + 0.0004
0.1761 + 0.0228
0. 0.1639 0.0026
Raindrop
0.5924 + 0.0013
0,3413 + 0.0046
0.3918 0.0014
0.0958 + 0.0039
0,2131
0.0067
(PatchGNN
0.5173 + 0.0037
0.2744 + 0.0022
0.3221
0.0017
0.0443 + 0.0005
0,2010
0.0191
Warpformer
0.4866 + 0.0007
0.2742 + 0.0023
0.3084 + 0.0007
0.0539 + 0.0007
0.1565
0.0012
GraFITi
0.4534 +: 0.0015
0,2454 0.0006
0,3060
0.0009
0.0435 + 0.0001
0,2026
0.0107
HyperIMTS (Ours)
0.4259 1 0.0021
0.2174 + 0.0009
0.2996 0.0003
0.0421 : 0.0021
0.1738 = 0.0078
to IMTS values via output linear mapping F.).
Table 2. Summary of five datasets. Canonical padding approach significantly increases number of observations, and patch-aligned padding can either be better or worse than canonical one. The patch lengths for the five datasets are 12 hours for MIMIC-III, MIMIC- IV. and PhysioNet' 12, 300 milliseconds for Human Activity, and
2 FT [V" E |E(rr)
(15)
where each node is decoded together with the information from its connected hyperedges. The model is trained by minimizing the Mean Squared Error (MSE) loss between the prediction 2, for forecast queries and the corresponding ground truth Z,
0.2 year for USHCN, respectively.
Deseription
MIMIC-I11
MIMIC-IV
PhysieNat
Human Activity
USHCN
Max length
131
337
Vanabll
12
e Sample
21,250
17,874
11,981
1,359
1,114
Avg abs.
144.6
301.8
308.6
362.2
313.5
Avg ubs. (pudding)
9.216.0
92,000.0
1,692.0
1,573.2
1.685.0
4.2.5. COMPUTATIONAL COMPLEXITY
Avg at abs. (patching)
9,2100
33,761.3
1,8000
1,803.
1,112.9
The computational complexity of HyperIMTS mainly arises from three attention calculations, including multi-head at- tention in Eq. (6), scaled dot-product attention in Eq. (12), and self-attention in Eq. (14). For a query matrix of shape (Nz, P) and key matrix of shape (Na. P), the linear map- ping complexity is P2), for the query and P2) for the key. For the dot-product operation, the computa- tional complexity is NN,, and multiplying resultant with a value matrix of shape (Nu, P) also has complexity of ON,,,NN, During node-to- message pass- ing as defined in Eq. (6), N₂ is U for variable hyperedge embeddings and 7 for temporal ones, and N₂ is M. In the hyperedge-to-hyperedee message passing described in Eq. (12), both N, and N₁ are U for variable hyperedges. For self attention in Eq. (14), both Ng and N₂ are M for observation nodes. Although attention is not computa-
tionally efficient operation theoretically, HyperIMTS only handles observed values, resulting in smaller data volumn compared to padding approaches. The statisties of actual datasets can be found in Table 2.
5. Experiments
5.1. Experimental Setup
1.1.1. DATASETS
Five widely studied irregular multivariate time series datasets, covering healthcare, biomechanics, and climate, are used in the experiments, and their statistics are sum- marized in Table 2. MIMIC-III (Johnson et al., 2016) is a
6

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
Table Ablation results of HyperIMTS and its five variants on five irregular multivariate time series datasets.
Ablation
MIMIC-III
MIMIC-IV
PhysioNet's 12
Human Activity
USHCN
Complete
0,4259 + 0.0021
0.2174 + 0.0009
0.2996
0.0003
0.0421 + 0.0021
0.1738
0.0078
w/o VE
0,9556 + 0.0029
0.6293 + 0.0053
0.6945 = 0.0002
1,3855 + 0.0006
0.3299 = 0.0054
w/o LAVD
0,4466 = 0.0007
0.2358 + 0.0012
0.3050
0.0006
0.0481 = 0.0004
0.1983
0,0105
p IAVD
0.4317 0.0030
0.2189 + 0.0024
0.3014
0.0003
0.0463 + 0.0008
0.1794
0.0062
w/o TE
0.4954 + 0.0043
0,2652 + 0.0005
0.3029
0.0003
0.0745 = 0.0001
0.1757
0.0088
rp TE
0.4403 0.0024
0.2333 + 0.0004
0.3029 0.0003
0.0473 + 0.0007
0.1894 0.0118
clinical database collected from ICU patients during the first 48 hours of admission, which is rounded for 30 minutes. MIMIC-IV (Johnson et al., 2023) is built upon MIMIC-III, which has higher sampling frequency and rounded for minute. PhysioNet' 12 (Silva et al., 2012) is also a clinical database collected from the first 48 hours of ICU stay, which is rounded for I hour. Human Activity contains biomechan- ies data describing 3D positional variables, which is rounded for I millisecond. USHCN (Menne et al., 2016) includes climate data over 150 years collected by meteorological stations scattered over the United States, and we focus on a subset of 4 years between 1996 and 2000. For PhysioNet" 12, MIMIC-III, MIMIC-IV, and USHCN, we follow the pre- processing setup in previous work (Yalavarthi et al., 2024). For Human Activity, we follow the preprocessing set up in (Zhang et al., 2024), All five datasets are split into training, validation, and test sets adhering to ratios of 80%, 10,, and 10%, respectively. Training and validation sets are shuffled, while test sets are not.
2024). Further details on introductions and hyperparameter settings of these haselines can be found in Appendix A.4.
5.1.3. IMPLEMENTATION DETAILS
All models are trained on a Linux server with PyTorch ver- sion 2.4.1 using two NVIDIA GeForce RTX 3090 GPUs, and efficiency analysis is conducted on another Linux server with PyTorch version 2.2.2+cul 18 using one NVIDIA GeForce RTX 2080Ti GPU. The learning rate Cn for the n-th epoch is kept unchanged when n <= 3, and adjusted according to Cn = Lo x 0.8 when n > 3. where the ini- tial learning rate Co is specific to each model and dataset and can be found in Appendix A.4. All experiments run with a maximum epoch number of 300 and early stopping patience of 10 epochs. To mitigate randomness, we conduct each experiment using five different random seeds ranging from 2024 to 2028 and calculate the mean and standard deviation of the results. MSE is used as the training loss function for models, unless a custom loss function proposed in the origi- nal paper is used. When adapting regular time series models for IMTS, masks indicating observed values are included in MSE calculations during training. The detailed settings for the hyperparameters are provided in Appendix A.4. We note that our experimental results may differ from those in existing papers, mainly due to differences in normalization methods, random seeds, and learning rate schedulers. We eliminate these differences to ensure fair comparisons.
5.1.2. BASELINES
To conduct comprehensive and fair comparisons, we design a unified and extensible pipeline to evaluate models across various domains and tasks. Twenty-seven baselines are in- cluded in the benchmark, covering SOTA methods from (1) Multivariate time series forecasting: FEDformer (Zhou et al., 2022), Ada-MSHyper (Shang et al., 2024), Auto- former (Wu et al., 2021), TimesNet (Wu al., 2022), iTrans- former (Liu et al., 2023), FourierGNN (Yi et al., 2023), Mamba (Gu & Dao, 2024), TSMixer (Chen et al., 2023), PatchTST (Nie et al., 2022), Leddam (Yu et al., 2024), Re- former (Kitaev et al., 2019), Informer (Zhou et al., 2021), Crossformer (Zhang & Yan, 2022), (2) Time series pre- training: MOIRAI (Woo et al., 2024), PrimeNet (Chowd- hury al., 2023), (3) Traffic forecasting: higp (Cini al., 2024), BigST (Han et al., 2024), (4) IMTS classification, imputation, and forecasting: SeFT (Horn et al., 2020), mTAN (Shukla & Marlin, 2020), NeuralFlows (Biloš et al., 2021), CRU (Schirmer et al., 2022), GNeuralFlow (Mer- catali et al., 2024), GRU-D (Che et al., 2018), Rain- drop (Zhang al., 2021), tPatchGNN (Zhang et al., 2024), Warpformer (Zhang al., 2023a), GraFITi (Yalavarthi et al.,
5.2. Main Results
Table shows the models' forecasting performance, eval- uated using MSE on five datasets, where the best results are highlighted in bold and the next best shown in under- line. The lookback time periods are 36 hours for MIMIC-III, MIMIC-VI, and PhysioNet" 12, 3000 milliseconds for Hu- man Activity, and 3 years for USHCN. Human Activity use 300 milliseconds as forecast length, and the rest datasets use the next 3 timestamps as forecast targets, following the settings in existing works (Biloš et al., 2021; De Brouwer et al., 2019). Results evaluated using MAE are detailed in Appendix A. and an analysis of varying lookback and forecast horizons can be found in Appendix A.2. As can be
7

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
The ablation results are summarized in Table 3. As can be seen, all model components are necessary. Results from w/o LAVD and rp LAVD show the necessity of accounting for variable dependencies when dealing with irregular ob- servation timestamps. rp TE demonstrates the importance of learnable temporal representations in effectively leverag- ing timestamp information within IMTS. w/o TE and w/o VE show the effectiveness of both temporal and variable hyperedges for feature extraction and message passing.
GPU Memory
1.0GB
3.0GB
GNeuralFlow 1oréra cuence
5.65
PatchTST D03ra 2514C0
Crossformer 5,0008
MSE
IPathhNNN
5.50
Eire 50460B
Warpformer
GraFITI
54806B
5.4. Efficiency Analysis
HyperIMTS
We select the most competitive baselines as well as represen- tative methods for the efficiency comparison. From data preprocessing perspective, these methods can be categorized as follows: (1) Non- Von-padding methods: HyperIMTS and
55505
Training Time (ms/iter)
GraFITi; (2) Patch-aligned padding method: (PatchGNN;
Figure 4. Model efficiency comparison on MIMIC-III, with 36 hours of lookback length, 3 forecast timestamps, 96 variables, and a batch size of 32. Our proposed model, HyperIMTS, achieves the lowest MSF while maintaining high computational efficiency, as measured by training time and memory footprints.
(3) Canonical padding methods: Warpformer and GNeu- ralffow; (4) Fully observed MTS methods: Crossformer and PStchTST. Models are evaluated based on their MSE, training time, and GPU memory footprints. The training time for one epoch with a batch size of 32 is recorded, then divided by the number of batches to determine the train- ing time per iteration. Memory footprints only encompass the model's usage instead of representing the entire pro- cess. The results on MIMIC-III are shown in Figure 4. As can be seen, our model HyperIMTSS achieves the lowest MSE while keeping computational costs relatively low. can also be observed that non woon-pading models like Hyper- IMTS and GraFITi achieve faster training speeds compared to other models. tPatchGNN uses patch-aligned padding, which can reduces the average number of padding values in MIMIC-III and runs faster than models using canonical padding. However, it still consumes a considerable amount of GPU memory, and may result in more padding values than canonical approaches for samples with numerous asyn- chronous observations. Transformer-based models, includ- ing Warpformer, Crossformer, and PatchTST, also exhibit large memory usage. This is primarily due to the attention calculations on padded samples, which involve a greater amount of data compared to original samples. Although using a small amount of memory, the ODE-based model GNeuralFlow takes significantly longer training time com- pared to other models, indicating the potential inefficiency of ODE solvers. Further efficiency analysis for varying lookback lengths as well as results on other datasets are available in Appendix A.3
seen, HyperIMTS excels as the best-performing model in four out of the five datasets. The results on USHCN exhibit high variance, as noted in previous work (Yalavarthi et al., 2024). However, these results are also provided for com- prehensive comparisons. By leveraging irregularity-awer message passing hypergraphs for dependency learning on all observations, HyperIMTSS provides up to 11.4% im- provement in comparison with the overall next best model GraFITi. On average, HyperIMTS exhibits significant re- duction in MSE compared to the best-performing regular time series model, Crossformer. We observed that some models designed for regular time series outperform a few IMTS models, highlighting the necessity of evaluating both regular and irregular time series models for more compre- hensive comparisons. Also, the time series pretraining mod- els MOIRAl and PrimeNet do not perform particularly well in general. This suggests potential discrepancies among different datasets, particularly between regular and irreg- ular time series datasets, which requires further efforts in designing the pretraining process.
5.3. A Ablation Study
We evaluate the performance of HyperIMTS and five of its variants on all five datasets. (1) rp TE replaces learnable temporal hyperedges with non-learnable sinusoidal embed- dings; (2) w/o TE removes all temporal hyperedges; (3) variable dependencies with overall variable dependencies only; (4) w/o LAVD re- moves irregular- aware variable dependencies by disabling syperedge-to-hypeee soreddge-tt-yyyeee hyperedge message passing among variable hyperedges; (5) w/o VE: removes all variable hyperedges;
6. Conclusion
rp IAVD replaces irregularity-
This paper introduces hypergraph neural network ap- proach, HyperIMTS, to address the IMTS forecasting prob- lem. HyperIMTS represents observed values as nodes in the hypergraph without padding, and connects them with
8

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
hyperedges denoting timestamps and variables. Through three types of message passing in hypergraphs, HyperIMTS effectively captures temporal and variable dependencies in a unified and efficient manner. Moreover, by leveraging time-aware similarity from observation nodes and overall similarity from variable hyperedges, HyperIMTS adaptively choose the optimal way to model variable correlations based on time alignment. HyperIMTS demonstrates competitive performance the IMTS forecasting task across twenty- seven tfaee-oftthe-ar time series models in our unified code pipeline. Nevertheless, there are still limitations for Hyper- IMTS. Our model currently does not support multi-modal data, such as text notes or images, which are found in med- ical IMTS datasets and may enhance forecasting. Addi- tionally, attention calculations are more resource-intensivy compared other recent methods, such as state space ap- proaches. We will address these limitations in future work.
Series Forecast-ing. Transactions on Machine Learning Research, 2023. ISSN 2835-8856.
Chowdhury, R. R.,, Li, Zhang, X., Hong, D., Gupta, R. K., and Shang, J. PrimeNet: Pre-training for Irregular Multi- variate Time Series. Proceedings of the AAAI Conference on Artificial Intelligence, (116)))788479922. 2023. ISSN 2374-3468. doi: 10.1609/amai v376..25877
Cini, A., Marisca, I., and Alippi, C. Filling the G.ap.s: Multivariate Time Series Imputation by Graph Neural Networks. In International Conference on Learning Rep- resentations, 2021.
Cini, A., Mandic, D., and Alippi, C. Graph-based Time Se- ries Clustering for End-to-End Hierarchical Forecasting, In International Conference on Machine Learning, pp. 8985-8999. PMLR, 2024.
De Brouwer, E., Simm, J., Arany, A., and Moreau, GRU-ODE MEL---DE--BssSs Continuous Modeling of Sporadically- Observed Time Series. In Advances in Neural Informa- tion Processing Systems, volume 32. Curran Associates, Inc., 2019.
Acknowledgements
We thank the anonymous reviewers for their help- ful feedbacks, and all the donors of the original datasets. The work described in this paper was par- tially funded by the National Natural Science Founda- tion of China (Grant Nos. 62272173), the Natural Sci- ence Foundation of Guangdong Province (Grant Nos. 1524 1089, 2022A 1222A151155 551511117)) the Science and Technology Planning Project of Guangdong Province (Grant No. 2023A0505050106) and the National Key R&D Pro- gram of China (Grant No. 2023YFA1011601).
Gao, Y., Feng, Y., Ji, S., and Ji, R. HGNN+: General Hypergraph Neural Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 3583) 3199, 2023. ISSN 1939-3539. doi: 10. 10.1109/TPMM..202. 3182052.
Gu, A. and Dao. T. Mamba: Linear-Time Sequence Model- ing with Selective State Spaces. In First Conference on Language Modeling, 2024.
Impact Statement
Han, J., Zhang, W., Liu, H., Tao, T., Tan, N., and Xiong, H. BigST: Linear Complexity Spatio-Temporal Graph Neu- ral Network for Traffic Forecasting on Large-Scale Road Networks. Proc. VLDB Endow., (7(5):1081-1000 2024.
This paper details efforts to advance irregular multivariate time series forecasting across various scientific domains. While our research may have broader societal impacts, we do not consider it necessary to single out any particular consequences for emphasis here.
ISSN 2150-8097. doi:
Horn, M., Moor, M., Bock, C., Rieck, B., and Borgwardt, K. Set Functions for Time Series. In International Con- ference Machine Learning, pp. 4353-4363, PMLR, 2020.
References
Biloš, M., Sommer, J., Rangapuram, S. S., Januschowski, T., and Günnemann, S. Neural Flows: Efficient Alternative to Neural ODEs. In Advances in Neural Information Pro- cessing Systems, volume 34, pp. 21325-21333 Curran Associates, Inc., 2021.
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.- W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., and Mark, R. G. MIMIC-III, a freely accessible critical care database. Scientific Data, 3(1): 160035. 2016. ISSN 2052-4463. doi: 2011 35.
Che. Z., Purushotham. S., Cho, K., Sontag, D., and Liu, Y. Recurrent Neural Networks for Multivariate Time Series with Missing Values. Scientific Reports, 8 8(1):6085, 2018. ISSN 2045-2322. doi: 10.1038/441598--18-44277/1
Johnson, A. E. W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., Pollard, T. J., Hao, S., Moody, B., Gow, B., Lehman, L.-w. H., Celi, L. A., and Mark, R. G. MIMIC-IV, a freely accessible electronic health record dataset. Scientific Data 10(1):1, 2023. ISSN 2052-4463. doi: 15920/44197-022-011885
Chen, S.-A., Li, C.-L., Arik, S. O., Yoder, N. C., and Pfis- ter, T. TSMixer: An AlI-MLP Architecture for Time

---

HyperIMTS:: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
Kitaev, N., Kaiser, L.,, and Levskaya, A. Reformer: The Efficient Transformer. In International Conference on Learning Representations, 2019.
Shang, Z., Chen, L.,, Wu, B., and Cui, D. Ada-MSHyper: Adaptive Multi-Scale Hypergraph Transformer for Time Series Forecasting. In The Thirty-eighth Annual Confer- ence on Neural Information Processing Systems, 2024.
Li, 7., Huang, C.,, Xia, I.,, Xu, Y., and Pei, J. Spatial- Temporal Hypergraph Self-Supervised Learning for Crime Prediction. In 2022 IEEE 38th International Con- ference on Data Engineering (ICDE). pp. 2984-2996. IEEE Computer Society, 2022. ISBN 1-64--0--8--88--- 7. doi: 10.1 10.1000/0DE333452222.0000
Shukla, S. N. and Marlin, B. Cirrpolation-Prediicte Net- works for Irregularly Sampled Time Series. In Interna- tional Conference Learning Representations, 2018.
Shukla, S. N. and Marlin, B. Multi-Time Attention Net-
works for Irregularly Sampled Time Series. In Interna- tional Conference on Learning Representations, 2020.
Liu, Y., Hu, T., Zhang. H., Wu, H., Wang, S., Ma, L.., and Long, M. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting, In The Twelfth International Conference on Learning Representations, 2023.
Shukla, S. N. and Marlin, B. M. A Survey on Principles, Models and Methods for Learning from Irregllally Sam- pled Time Series. 2021. doi: 10. 20.4
Luo, X., Peng, J., and Liang, J. Directed hypergraph at- tention network for traffic forecasting. IET Intelligent Transport Systems, 16(1):85-98, 2022. doi: 10.1049/tt2. 12130.
Silva, I., Moody, G. Scott, D. J., Celi, L. A., and Mark, R. G. Predicting In- In-Hospital Mortality of ICU Patients: The PhysioNet/Computten in Cardiology Challenge 2012. Computing in cardiology, 39:245-244 248, 2012.
Luo, Y., Liu, Z., Wang, L., Wu, B., Zheng, J., and Ma, Q. Knowledge-Empowered Dynamic Graph Network for Irregularly Sampled Medical Time Series. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Tashiro, Y., Song, J., Song, Y., and Ermon, S. CSDI: Con- ditional Score-based Diffusion Models for Probabilistic Time Series Imputation. In Advances in Neural Informa- tion Processing Systems, volume 34, pp. 24804-24816. Curran Associates, Inc., 2021.
Menne, M., Williams, Jr., C., and Vose, R. Long-term daily and monthly climate records from stations across the contiguous united states (u.s. historical climatology network). 1 2016. doi: 10.333/CDIAC/CLINDDDD
Wen, H. Lin, Y., Xia, Y., Wan, H., Zimmermann, R., and Liang, Y. DifISTG: Probabilistic Temtorl Graph Forecasting with Denoising Diffusion Models, 2023.
Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., and Sahoo, D. Unified Training of Universal Time Series Forecasting Transformers. In International Conference on Machine Learming, pp. 53140-53164. PMLR, 2024.
Mercatali, G., Freitas, A., and Chen, J. Graph Neural Flows for Unveiling Systemic Interactions Among Irreuularly Sampled Time Series. In The lhiitt--ightt Anmual Con- ference on Neural Information Processing Systems, 2024.
Wu, H., J., Wang, J., and Long, M. Autoformer: Decom- position Transformers with Auto Auto-Correlation for Long- Term Series Forecasting. In Advances in Neural Informa- tion Processing Systems, volume 34, pp. 22419-22430. Curran Associates, Inc., 2021.
Nic, Y., Nguyen. N. H., Sinthong, P., and Kalagnanam, J. A Time Series is Worth 64 Words: Long-tern Forecast- ing with Transformers. In The Eleventh International Conference on Learning Representations, 2022.
Rubanova, Y., Chen, R. T. Q., and Duvenaud, D. K. Latent Ordinary Differential Equations for Pregularly-Samplee Time Series. In Advances in Neural Information Process- ing Systems, volume 32. Curran Associates, Inc., 2019.
Wu, H., Hu, T., Liu, Y., Zhou, H., Wang. J., and Long, M. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. In The Eleventh International Conference on Learning Representations, 2022.
Sawhney, R., Agarwal, S., Wadhwa, A., Derr, T., and Shah, R. R. Stock Selection via Spatiotemporal Hypergraph Attention Network: A Learning Rank Approach. Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, 35(1):497-504, 2021. ISSN 2374-3468. doi: 0t..0/9/aaai.3...1....
Wu, Z., Pan, S., Long, G., Jiang, J., Chang, X., and Zhang, C. Connecting the Dots: Multivariate Time Series Fore- casting with Graph Neural Networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, KDD '20, pp. 753- 763. Association for Computing Machinery, 2020. ISBN 978-1-4503-799
Schirmer, M., Eltayeb, M., Lessmann, S., and Rudolph, M. Modeling Irregular Time Series with Continuous Recurrent Units. In International Conference on Machine Learning, pp. 19388-19405. PMLR, 2022.
doi: 10. 1e11114//334486.440000 18.
Yalavarthi, V. K., Madhusudhanan, K., Scholz, R., Ahmed, N., Burchert, J., Jawed, S., Born, S., and Schmidt- Thieme,
10

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
L. GraFITi: Graphs for Forecasting Irregularly Sampled Time Series. Proceedings of the AAAI Conference Artificial Intelligence, 38(15) (66555):6625511666 2024. ISSN 2374-3468. doi: 10.1609/aami, wo..:.......1111.55.99
Zhang, Y. and Yan, J. Crossformer: Transformer Utiliz- ing Cross-Dimension Dependency for Multivariate Time Series Forecasting. In The Eleventh International Confer- ence on Learning Representations, 2022.
Yan, Y., Qin, J., Chen, J.. Liu, L., Zhu, F., Tai, Y., and Shao, L. Learning Multi-Granular Hypergraphs for Video- Based Person Re-Identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2899-2908, 2020.
Zhou, D., Huang, J., and Schölkopf, B. Learning with Hypergraphs: Clustering, Classification, and Embedding. In Advances in Neural Information Processing Systems, volume 19. MIT Press, 2006.
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting, Proceedings of the AAAI Conference on Artificial Intelligence, 35(12): 11106-11115, 2021. ISSN 2374-3468. doi: 10.1609/aaai. :35i12.17325,
Yi, K. Zhang, Q.. Fan, W., He, H., Hu, L., Wang, P., An, N., Cao, L., and Niu, Z. FourierGNN: Rethinking Multivari- ate Time Series Forecasting from Pure Graph Perspec- tive. In Thirty-Seventh Conference on Neural Information Processing Systems, 2023.
You, J., Ma, X., Ding, Y., Kochenderfer, M. J., and Leskovec, J. Handling Missing Data with Graph Repre- sentation Learning. In Advances in Neural Information Processing Systems, volume 33, pp. 19075-19087. Cur- ran Associates, Inc., 2020.
Zhou, T., Ma, Z., Wen, Q.. Wang, X., Sun, L., and Jin, R. FEDformer: Frequency Enhanced Decomposed Trans- former for Long-term Series Forecasting. In International Conference on Machine Learning, pp. 77268-27286. PMLR, 2022.
Yu, G., Zou, J., Hu, X., Aviles-Rivero, A. I., Qin, J., and Wang, S. Revitalizing Multivariate Time Series Fore- casting: Learnable Decomposition with Inter-Series De- pendencies and Intra-Series Variations Modeling. In Forty-First International Conference on Machine Learn- ing, 2024.
Yu, J., Tao, D., and Wang, M. Adaptive Hypergraph Learn- ing and its Application in Image Classification. IEEE Transactions Image Processing. 21(7):3222-3272, 2012. ISSN 1941-0043 doi: 10. 10.1109/TTP221122299088
Zhang, J., Zheng, S., Cao, W., Bian, J.. and Li, J. Warp- former: A Multi-scale Modeling Approach for Irregular Clinical Time Series. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3273-3285, 2023a. doi: 14533885080 3599543.
Zhang, K., Wen, Q. Zhang, C., Cai, R., Jin, M., Liu, Y., Zhang, J.. Liang, Y., Pang, G., Song, D., and Pan, S. Self-Supervised Learning for Time Series Analy- sis: Taxonomy, Progress, and Prospects. 2023b. doi: 10.48550Warxiv.2306.1112
Zhang, W., Yin, C., Liu, H., Zhou, X., and Xiong, H. Ir- regular Multivariate Time Series Forecasting: A Trans- formable Patching Graph Neural Networks Approach. In Forst International Conference on Machine Learn- ing, 2024.
Zhang, X.. Zeman, M., Tsiligkaridis, T., and Zitnik, M. Graph-Guided Network for Irregularly Sampled Mul- tivariate Time Series. In International Conference on Learning Representations, 2021.

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
Table 4. Experimental results on five irregular multivariate time series datasets, evaluated using MAF The experimental setup is the same
as Table 1.
Algorithm
MIMIC-11
MIMIC-IV
PhysioNet"11 12
Human Activity
USHCN
higp
0.6529 +: 0.0003
0,5804 1: 0.0001
0,5183 0.0140
0.3870 +: 0.0115
0.3411 = 0.0003
MOIRAI
0.5790 + 0.0000
0.3998 + 0.0000
0,4937 + 0.0000
0.1914 + 0.0000
0.8274
0.0000
FEDformer
0,5631 + 0.0006
0.6469 + 0.0157
0.4549 + 0.0003
0.2897 + 0,0009
0.3704 = 0.0060
Ada-MSHyper
0.5354 +: 0.0024
0,4423 : 0.0008
0.4579 0.0002
0.2610 + 0.0037
0.3186
0,0074
Autoformer
0.5475 + 0.0033
0.5477 0.0096
0,4534
0.0031
0.2178 +: 0.0062
0.4102 0.0047
TimesNet
0.5232 + 0.0012
0,4511 + 0.0005
0.4538 + 0.0005
0.2258 + 0.0023
0,3378 = 0.0021
iTransformer
0.5581 +: 0.0052
0,4875 1 0.0068
0.4443 4 0.0019
0.2018 + 0.0024
0.3365 = 0,0065
Rogular
FourierGNN
0.5381 +: 0.0021
0,4499 0.0016
0,4386 0.0019
0.3764 1: 0.0069
0,4013 0.0206
Mamha
0,5423 + 0.0010
0,5612 + 0.0034
0.4390 + 0.0008
0.2330 + 0.0006
0,3245 = 0.0025
TSMixer
0,4949 + 0.0003
0,3977 + 0.0011
0.4351 0.0004
0.2836 + 0.0037
0,3010 = 0.0053
PutchTST
0.5153 + 0.0011
0.3246 0.0006
0,4325 + 0.0005
0.1723 1: 0.0014
0.3502 0.0424
Leddam
0.4828 + 0.0033
0.3985 + 0.0018
0.4281 + 0.0022
0.2003 + 0.0008
0.3134 0.0073
BigST
0.4734 + 0.0004
0
0,4012 + 0.0005
0.2695 + 0.0108
0.3322 = 0.0029
Reformer
0.4724 + 0.0004
0.4253 + 0.0014
0,4117 + 0.0002
0,2282 + 0.0022
0.3136 0.0042
Informer
0.4682 + 0.0026
0,4592 + 0.0044
0.4059 + 0.0002
0.1833 + 0,0010
0,3186 = 0.0024
Crossformer
0,4961 + 0.0092
0.4021 + 0.0022
0,3954 0.0054
0.2498 + 0.0170
0.3322 = 0.0058
PrimeNet
0.6690 + 0.0000
0.5879 0.0001
0.6887
0.0000
1.7268 + 0.0001
0.4945 = 0.0007
SeFT
0.6661 + 0.0004
0.5895 + 0.0005
0,6742 + 0.0003
0.9923 1 0.0006
0.4079 0.0038
mTAN
0.6437 + 0.0088
0,5184 +: 0.0057
0.4990 + 0.0019
0.2181 + 0.0032
0.3335 0.0019
NeuralFlows
0.5490 + 0.0013
0.4794 0.0010
0.4604 + 0.0014
0.3088 + 0.0035
0.3144
0.0032
Itrogular
CRU
0,5369 + 0.0014
0.4558 + 0.0011
0,5815 + 0.0008
0.2572 + 0.0040
0.3371 = 0.0074
GNeuralFlow
0,5352 + 0.0027
0,4899 1 0.0003
0.4777 + 0.0026
0.3146 + 0.0016
0,2980 0.0034
GRU-D
0.4891 + 0.0138
0.5816 0.0012
0,4065 0.0003
0.3155 0.0205
0,2923 0.0019
Raindrop
0,4850 + 0.0008
0.3879 + 0.0032
0.2174 + 0.0046
0.3128
0.0076
(PatchGNN
0.4293 + 0.0039
0.3096 1 0.0016
0.3825 + 0.0024
0.1238 + 0.0003
0.2928
0.0068
Warpformer
0.4039 0.0009
0.3142 0.0022
0.3667 0.0009
0.1296 0.0013
0,2704
0.0020
GraFITi
0.3923 + 0.0010
0.3004 + 0.0004
0.3621 + 0.0007
0.1204 + 0.0006
0.3029
0.0145
HyPerIMTS (Ours)
0.3800 + 0.0009
0,2837 + 0.0007
0.3598 + 0.0002
0.1199 + 0.0059
0.2773 + 0.0064
A. Additional Experiments
A.1. Different Metrics
We present the results measured using MAE in Table 4, which follows the same experimental setup as Table 1. As can be seen, HyperIMTS outperforms twenty-seven baselines on four of the five datasets and comes in second place on the remaining one. The variance in results for the USHCN dataset remains high, similar to the findings from MSE evaluations, as discussed in Section 5.2.
A.2. Varying Lookback Lengths and Forecast Horizons
We also assess performance across varying lookback and forecast horizons. For varying forecast horizons, we follow the same settings in previous work (Zhang et al., 2024) and keep the lookback length settings the same as in Table 1. For MIMIC-III, MIMIC-IV, and PhysioNet' 12, the forecast horizon is set to 12 hours. For Human Activity, the forecast horizon is 1000 milliseconds. For USHCN, the forecast horizon is set to I year. The results are summarized in Table 5, where we select the most competitive baselines from Table 1 and include some well-known models for comparison, including five MTS forecasting models and five IMTS models. It can be seen that HyperIMTSS remains the overall best-performing model evaluated based on ranking. We note that, unlike fully observed MTS, IMTS samples are typically split based on time periods rather than the number of observations. The forecast settings here view 75% of the time period as the lookback window, while the remaining 25% as the forecast window.
For varying lookback lengths, the results shown in Figure 5. We keep the forecast horizons the same in Table 5, and varying the lookback lengths to: (1) MIMIC-III: 12, 24, and 36 hours; (2) MIMIC-IV: 12, 24, and 36 hours; (3) PhysioNet' 12: 12, 24, and 36 hours; (4) Human Activity: 1000, 2000, and 3000 milliseconds; (5) USHCN: 1, 2, and 3 years. As can be seen in the result, the forecasting performance of HyperIMTS generally improves with increasing lookback length, except for MIMIC-III, III, where all models perform worse at 36 hours than at 24 hours. It is possible that the change in temporal patterns, observed after a 24-hour stay for patients, may be responsible.
12

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
Table 5. Experimental results on five irregular multivariate time series datasets evaluated using MSF, with the lookback length following Table and forecast horizons to the rest length of the whole series, which are 12 hours for MIMIC-III, MIMIC-VV, and PhysioNet' 12, 1000 milliseconds for Human Activity, and year for USHCN.
Algorithm
MIMIC-III
MIMIC-IV
PhysioNet'12 12
Human Activity
USHCN
Ada-MSHyper
0.7437 + 0.0098
0.4305 + 0.0035
0,4593 + 0.0006
0.1627 + 0.0027
0.5001 = 0.0047
Regular
iTransforntrr
0,7825 +: 0.0061
0.4815 +: 0.0014
0.4545 +: 0.0020
0.0997 + 0.0025
0.5624 = 0.0051
PStchTST
0.6976 + 0.0008
0.3704 0.0010
0.4374 0.0005
0.0868 + 0.0006
0.6816 0.1005
Informer
0.6606 + 0.0063
0.3752 + 0.0025
0,4050
+ 0.0004
0.0837 + 0.0003
0.4337 0,0013
Crossformer
0.6412 + 0.0018
0.3624 +: 0.0022
0,4080 + 0.0022
0.1656 + 0.0289
0.4672 = 0.0092
GNouralFlow
0,7434 + 0.0068
0,4826 + 0.0020
0.4390 + 0.0022
0.2074 + 0.0281
0.4988 = 0.0013
Irregular
GRU-D
0.6367 + 0.0020
0,4446 0.0005
0,4059 0.0006
0.1829 + 0.0229
0.5320 0.0056
(PatchGiNN
0,5896 + 0.0055
0.2881 + 0.0008
0.3806
+ 0.0009
0.0601 + 0.0005
0,5709 = 0.0438
Warpformer
0.5664 + 0.0020
0.2991 + 0.0020
0.3730 + 0.0008
0.0611 + 0.0010
0.4531 = 0.0005
GraFITi
0.5348 + 0.0008
0.2722 0.0008
0.3772 + 0.0001
0.0596 + 0.0007
0.4419 0.0092
HyperIMTS (Ours)
0,5111 + 0.0015
0.2328 + 0.0005
0.3683
0.0002
0.0589 + 0.0020
0,4400
0.0049
MIMIC-III
MIMIC-I
Physionet'l 12
0.45
0.44
6.40-
MSE
MSE
0.42
Gleurs thocth
8:30
0.25
12
24
36
12
24
36
24 Lookback Length (hours)
Lnokhack Length (hours)
Lookback Length (hours)
Human Activity
USHCN
0.20
Pothoww
0.72
Hypertert
0.18
0.72
MSE
Cheurdhhoo
0.65
PyportMis
Crufti
0.82
455
0.10
0.00
0.2
0.5
3050
Lookback Length (milliseconds)
Lookback Length (years)
Figure 5. Forecasting performance with varying lookback lengths and fixed forecast horizons.
13

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
Lookback Length
Lookback Length
HyperIMTS
HyperIMTS
12 hours
12 hours
24 hours
24 hours
36
hours
GraFITi
GraFITi
tPatchGNN
tPatchGNN
Warpformer
Warpformer
GPU Memory (GB)
Training Time (ms/iter)
Figure Efficiency comparison varying lookback lengths fixed forecast horizons on MIMIC-III. The efficiency of non- non-padding methods is relatively insensitive the increase in lookback length.
A.3. Additional Efficiency Analysis
We further analyze the efficiency of non-padding, patch-aligned padding, and canonical padding approaches for different lookback lengths. We fix the forecast horizons to 12 hours in MIMIC-III, and vary the lookback lengths from 12 to 36 hours. The results are summarized in Figure 6, where HyperIMTS and GraFITi are non-padding approaches, (PatchGNN uses patch-aligned padding. and Warpformer as well as GNeuralFlow are canonical padding approaches. As can be seen, efficiency for non-padding methods is relatively insensitive to increases in lookback length. Patch-aligned padding method IPatchGNN also achieves relatively fast speed, but deteriorates quickly with the increase of lookback length. Non-padding method Warpformer spends the highest amount of GPU memory as well training time compared to others.
We also provide four additional efficiency comparisons on MIMIC-IV, PhysioNet') 12, Human Activity, and USHCN using the same settings Table 1, and the results are summarized in Figure 7. They confirm the finding that efficiency of non- non-padding methods is relatively insensitive to increases in lookback length, as MIMIC-IV and USHCN have larger max time length than PhysioNet' 12 and Human Activity. When the data volumn is small, non- non-padding methods have a similar base cost to other approaches, explaining their slower performance on PhysioNet's 12 and Human Activity. We also notice the unexpected efficiency of Crossformer on MIMIC-IV. This may be attributed the router mechanism in the cross-dimensional stage of Crossformer, MIMIC-IV has the largest number of variables among the five IMTS datasets.
In summary, HyperLMTS demonstrates superior overall performance while maintaining low computational costs across all datasets and input settings. Non- Kon-padding approaches exhibit similar base cost to other padding methods when the lookback length relatively small. However, their efficiency is insensitive to the increase in lookback length. Therefore, they can generally use less memory and training time for longer inputs. Further improvements on non- padding methods can focus on more efficient dependency modeling mechanisms, such as the router mechanism used in Crossformer.
A.4. Baseline Details
We briefly introduce each baseline model along with their key hyperparameter settings here. Unless otherwise specified, we use batch size of 16 for USHCN, and 32 for others. We try to use the same hyperparameter settings for learning rate, hidden dimension, special loss functions, and number of layers from their original papers and codes, if available. Number of epochs, early stopping patience, random seeds, and learning rates have been described in Section 5.1.3. For all classification models, we replace the final softmax layer with a linear layer to enable forecasting.
1. METHODS FOR MTS
higp (Cini et al., 2024) uses hierarchical structure along variable dimension for traffic forecasting. The hidden dimension 512. The learning rate is x 10-3, Since the model requires pre-defined variable adjacency matrix in dataset, which not available for IMTS datasets, we borrow the idea from graph neural networks and replace with graph learner the model.
14

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
0.50
0.30
PatchTST
GNeuralFlow Se2rr 0,02708
GNeuralFlow
1,00008
Crossformer
0.35
110ma 1212
0.36
0.30
PatchTST 275m 4550
0.34
Crossformer
MSE
MSE
GraFITI t0ms 1 031GB
1PttchGNN 22me o MOGGE
0.25
0.32
Gra
Warpformer 60me 0:82nGB
HyperIMTS ms 2.92938
GPU Memory
GPU Memory
0.20
0.50
HyperIMTS dāme 0,58008
1.0GB
4.0GB
1.0GB
0.15
0.28
10020
100
150
230
550
Training Time (ms/iter)
Training Time (ms/iter)
(a) MIMIC-IV
(b) PhysioNet's 12
GPU Memory
0.17
GNeuralFlow 1642mm a DesGB
0.29
PatchTST 108me 1,37768
1,0GB
3.0GB
Crossformer 0.383G8
0.14
0.22
Crossformer
GPU Memory
108me 2 2222GB
PatchTST 40ma 0.141128
0.5GB
0.06
0.20
MSE
MSE
GraFITI
1PatchGNN timee 1,000G0
otme D. 6.5GG
Warpformer Affms O. M05G3
0.05
0.10
GNeuralFlow 4090ms 12269
GraFITI 57ms 0,305GB
HyperIMTS 67ne 033830
0.04
0.16
Warpformer 141ms 13888B
HyperIMTS TS 72mz 0470GB
(PatchGNN 73ms 0,88838
0.02
0.14
75
100
1450
50
100
150
200
4000
Training Time (ms/iter)
Training Time (ms/iter)
(c) Human Activity
(d) USHCN
Figure 7. Efficiency comparisons on MIMIC-IV, PhysioNet' 12, Human Activity, and USHCN. tPatchGNN and Warpformer run of GPU memory on MIMIC-IV and are not plotted. Efficiency for non- padding methods is similar to other approaches on shorter lookback length like PhysioNet' 12 and Human Activity, but they are less sensitive the grouth of length, thus faster than most other methods on MIMIC-IV and USHCN.
15

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
(Woo al., 2024) is pretraining model for time series forecasting. We the small version of the provided pretrained configurations and weights, comprising 6 layers with hidden dimension 384. We finetune the model on IMTS datasets with learning rate of 1 x 10-4.
MOIRAI
FEDformer (Zhou et al., 2022) is frequency-enhancee decomposed transformer for MTS forecasting. The attention factor is 3. The hidden dimension is 512. The learning rate is 1 x 10-4.
Ada-MSHyper (Shang et al., 2024) uses hypergraphs for temporal multiscale learning in MTS forecasting. The window size for multiscale 4. The learning rate is x 10-3. We also use its node and hyperedge constrainted loss function for training.
(Wu et al., 2021) is a transformer variant with auto-correlation decomposition for MTS forecasting. The attention factor is 3. The hidden dimension is 512. The learning rate is 1 10
Autoformer
TimesNet (Wu et al., 2022) uses 2-D variantion modeling for MTS analysis. The attention factor is The hidden dimension is 512. The dimension for FCN is 32. The learning rate is x 10-3
iTransformer (Liu al., 2023) exchanges the temporal and variable dimension for MTS forecasting, The hidden dimension is 512. The learning rate is 1 10-3
FourierGNN (Yi al., 2023) views all observated values nodes in a graph, and perform graph learning in frequency domain to forecast MTS. The hidden dimension is 512. The learning rate is 1 x 10-3
Mamba (Gu & Dao, 2024) is a selective state-space model for sequence modeling. The hidden dimension is 128. The dimension for FCN is 16. The learning rate is x 10 10-4
TSMixer (Chen et al., 2023) is an alI-MLP model for MTS forecasting. The number of encoder layers is 2. The number of decoder layers is The attention factor The hidden dimension is 256. The dimension for FCN is 512. The learning rate is 1 10-4
PStchTST (Nie et al., 2022) leverages patching in transformer for MTS forecasting. The patch lengths 12, 90, 6, 300 and 10 for MIMIC-III, MIMIC-IV, PhysioNet" 12, Human Activity and USHCN, respectively. The number of encoder layers is 3. The attention factor is 3. The number of heads in attention is 16. The learning rate is 1 10-1.
Leddam (Yu et al., 2024) uses learnable seasonal-trent decomposition for MTS forecasting. The hidden dimension is 512. The learning rate is x 10-3
BigST (Han et al., 2024) uses a spatio-temporal graph neural network for traffic forecasting. The hidden dimension is 32. The dropout rate is 0.3. The learning rate is 1 x 10-3
Reformer (Kitaev et al., 2019) is an efficient implementation the Transformer. The number of encoder layers The number of decoder layers is 1. The attention factor 3. The learning rate is 10-4
Informer (Zhou al., 2021) another efficient implementation of the Transformer for MTS forecasting. The hidden dimension is 512. The learning rate is 1 10-4
Crossformer (Zhang & Yan, 2022) learns cross-dimensional dependencies for MTS forecasting. The segment lengths 12, 360, 6, 300, and 3 for MIMIC-III, MIMIC-IV, PhysioNet" 12, Human Activity, and USHCN respectively. The hidden dimension is 512. The learning rate is 1 x 10-3
A.4.2. METHODS FOR IMTS
PrimeNet (Chowdhury et al., 2023) an IMTS pretraining model. Since the provided weights are specific to datasets with 41 variables, we retrain the model on all datasets. The patch lengths are 12, 180, 6, 300, and 10 for MIMIC-III,
16

---

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
MIMIC-IV, PhysioNet'12 12, Human Activity, and USHCN respectively. The number of heads in attention is 1. The learning rate is 1 10-
SeFT (Hom et al., 2020) is set-based method for IMTS classification. The number of layers is 2. The dropout rate is 0.1. The learning rate is 1 x 10-3.
mTTAN (Shukla & Marlin, 2020) converts IMTS to reference points for IMTS classification, The number of reference points are 32 on MIMIC-III and 8 on the rest datasets. The learning rate is x 10-3.
NeuralFlows (Biloš et al., 2021) is an efficient alternative to Neural ODE in IMTS analysis. The number of flow layers is The latent dimension is 20. The hidden dimension for time is 8. The number of hidden layers is 3. The learning rate
1 x 10-3
CRU
(Schirmer et al., 2022) uses continuous recurrent units for IMTS analysis. The hidden dimension is 20. The learning
rate is 1 10-3
GNeuralFlow (Mercatali et al., 2024) enhances NeuralFlows with graph neural networks for IMTS analysis. The flow model uses ResNet. The number of flow layers is 2. The latent dimension for input is 20. The latent dimension for time is 8. The number of hidden layers is 3. The learning rate is 1 x 10-3
GRU-D (Che et al., 2018) adapts GRUs for IMTS classification. The hidden dimension is 512. The learning rate is 1 10-3
Raindrop (Zhang et al., 2021) models time-varying variable dependencies for IMTS classification. The hidden dimension 512. The learning rate is x 10-4
1PatchGNN (Zhang et al., 2024) processes IMTS into patches and use graph neural networks for IMTS forecasting. The patch lengths are 12, 360, 6, 300, and 10 for MIMIC-III, MIMIC-IV, PhysioNet' 12, Human Activity, and USHCN, respectively. The number of heads in attention is 1. The learning rate is 1 10-3
Warpformer (Zhang al., 2023a) uses warping for multiscale modeling in IMTS classification. The hidden dimension is 64. The number of heads is 1. The dropout rate is O. The number of layers is 3. The learning rate is 1 10-3
GraFITi (Yalavarthi et al., 2024) uses bipartite graphs for IMTS forecasting. The latent dimension is 128. The number of heads in attention is 4. The numeber of layers is 2. The learning rate is 1 x 10-3
17


Reviews:
[{"review_id": "ddIQMHVzRT", "reviewer": "Reviewer_VDUB", "summary": "The paper introduces a hypergraph neural network designed for forecasting irregular multivariate time series, which are characterized by irregular time intervals within variables and unaligned observations across variables.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review and valuable advice. We address the concerns as follows:\n## W1. Efficiency reporting and performance analysis\n**A1:** Additional efficiency analyses can be found in **Appendix A.3**, and performance analyses on varying lookback lengths and forecast horizons are detailed in **Appendix A.2**, which can be summarized as:\n### 1. **Insensitive to time length**\nNon-padding methods can handle longer time lengths with **smaller efficiency degradation**, compared to padding-based methods.\n- In Figure 6, non-padding methods **maintain high efficiency** for longer time lengths.   \n- In Figure 7, although non-padding methods may not be the fastest on datasets with short lengths (47 timestamps for PhysioNet'12, 131 timestamps for Human Activity), they are **among the fastest for long input lengths** (971 timestamps for MIMIC-IV, 337 timestamps for USHCN).\n### 2. **Best-performing on various lengths**\n- In Table 5, our method remains the **best-performing** at longer forecast horizons compared to the strongest baselines, where 5 out of 10 models are within 1 year [1-5].\n- In Figure 5, our method has the **lowest MSE** in 13 out of 15 lookback length settings.\n## W2. Codes and hyperparameter settings\n**A2:** We apologize for any clarity issues in presenting our code and hyperparameter settings.\n### 1. **Code snippets**\nAn anonymized repository link is available **at the end of the abstract, line 40**. An overview of the codes can be found in answer **A3** below.\n### 2. **Hyperparameter settings**\nThey are discussed in **Appendix A.4** and provided in the 'scripts' directory within the anonymized code repository.\nWe follow the hyperparameter settings **from original papers or codes** for baselines, except for batch size (32), max epoch (300), and early stopping patience (10).\nThe differences in critical hyperparameter settings for top-performing models on MIMIC-III are shown here:\n||Ours|GraFITi|Warpformer|Crossformer|\n|---|---|---|---|---|\n|seg length|-|-|-|12|\n|hidden dimension|128|128|64|128\n|attention heads|4|4|1|8|\n|drop out|-|-|0|0.05\n## W3. Challenging to implement and optimize practically\n**A3:** Code implementation of our model is provided **in `models/HyperIMTS.py` file within the anonymized code repository**.\nA summary of the code implementation is provided here:\n### 1. **IMTS to hypergraph**\n`HypergraphEncoder` class, which outputs:\n- observation nodes: Eq.3, linear layers + ReLU.\n- temporal hyperedges: Eq.4, sinusoidal encoding.\n- variable hyperedges: Eq.5, learnable parameters.\n- incidence matrices:\n    Eq.2, indicate for every hyperedge, which observation nodes are connected to it.\n### 2. **Hypergraph learning**\n`HypergraphLearner` class, which is a **hypergraph attention network** similar to the backbone used in previous irregular time series models such as GraFITi [3] and Raindrop [6]:\n- node-to-hyperedge: Figure 3 (a), `MultiHeadAttentionBlock` class.\n- hyperedge-to-hyperedge: Figure 3 (b), `IrregularityAwareAttention` class.\n- hyperedge-to-node: Figure 3 (c), mainly includes self attention and linear layers with activations.\n### 3. **Hypergraph to IMTS**\nEq.15, A linear layer that maps from the hidden space to the sample space.\n### 4. **Optimization**\nThe core module `HypergraphLearner` consists of **only 2 layers with residual connections** around attention operations, achieving its best performance  **within 50 epochs** across all datasets in our experiments.\n## W4. Cannot handle text or images; Resource-intensive compared to SSMs\nThese possible improvements are not included for the following reasons:\n### 1. **Unfair comparisons** (text/images)\nMost SOTA time series models only accept time series as input.\nIf our model takes more input data than others, it would be unfair for these baselines.\n### 2. **Untested with HGNNs** (SSMs)\nGraph attention networks have been shown to be reliable in existing works [3][6].\nHowever, SSMs have not yet been tested with HGNNs for time series yet, which might lead to unexpected issues like performance degradation.\n\n[1] Mercatali, Giangiacomo, et al; \"Graph Neural Flows for Unveiling Systemic Interactions Among Irregularly Sampled Time Series\"; NeurIPS 2024\n\n[2] Shang, Zongjiang, et al; \"Ada-MSHyper: Adaptive Multi-Scale Hypergraph Transformer for Time Series Forecasting\"; NeurIPS 2024\n\n[3] Yalavarthi, Vijaya Krishna, et al; \"GraFITi: Graphs for Forecasting Irregularly Sampled Time Series\"; AAAI 2024\n\n[4] Zhang, Weijia, et al; \"Irregular Multivariate Time Series Forecasting: A Transformable Patching Graph Neural Networks Approach\"; ICML 2024\n\n[5] Liu, Yong, et al; \"iTransformer: Inverted Transformers Are Effective for Time Series Forecasting\"; ICLR 2024\n\n[6] Zhang, Xiang, et al; \"Graph-Guided Network for Irregularly Sampled Multivariate Time Series\"; ICLR 2022\n\n---\n**We hope the above response can help solve your questions. Thanks again for your thorough review and looking forward to your reply!**", "claims_and_evidence": "While the claims are generally supported, more comprehensive efficiency reporting and statistical analysis of performance differences would strengthen the evidence.", "methods_and_evaluation_criteria": "The proposed methods align well with the IMTS forecasting problem.", "theoretical_claims": "The paper does not present formal theorems or proofs. Its theoretical foundation lies in the hypergraph structure and message-passing mechanisms, drawing from graph theory and neural network principles.", "experimental_designs_or_analyses": "The experimental design is robust and sound.", "relation_to_broader_scientific_literature": "HyperIMTS relates to prior work such as IMTS modeling, GNNs for time series, and HGNNs.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n(1) Novel application of hypergraphs to IMTS forecasting, creatively combining existing ideas.\n(2) Avoids padding, reducing computational overhead, as shown in Figure 4.\n(3) Extensive comparison with 27 baselines across five datasets is a significant effort.\n\nWeaknesses:\n(1) Compared to the existing works, the hypergraph structure modeling may be challenging to implement and optimize practically.\n(2) As noted in Section 6, it cannot handle text or images, restricting applicability in some domains.\n(3) Resource-intensive compared to state-space models like Mamba.", "other_comments_or_suggestions": "None.", "questions_for_authors": "see weakness.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MLWDtyEAPd", "reviewer": "Reviewer_Wd2p", "summary": "Irregular Multivariate Time Series (IMTS) are challenging due to irregular time intervals within variables and unaligned observations across variables, making it difficult to model temporal and variable dependencies. Existing IMTS models either use padded samples (which can introduce inefficiencies and distort sampling patterns) or represent data as bipartite graphs or sets (which struggle to capture dependencies among unaligned observations).\n\nTo address these limitations, the authors propose HyperIMTS, a Hypergraph Neural Network for IMTS forecasting. In this approach, observed values are treated as nodes in a hypergraph, with temporal and variable hyperedges facilitating message passing among all observations. This irregularity-aware message passing allows the model to capture variable dependencies in a time-adaptive manner, leading to improved forecasting accuracy. Experimental results show that HyperIMTS outperforms state-of-the-art models while maintaining low computational costs.\n\n## update after rebuttal\nThank you for your response.\n\nAfter reviewing the rebuttal, my original concerns remain largely unaddressed. First, I believe that set-based methods are still capable of capturing correlations, as the embeddings of independently processed observations are ultimately aggregated to produce the final output. Second, as acknowledged by the authors in the rebuttal, the proposed model relies on a shared feature medium (hyperedge), like a bipartite graph that depends on a shared temporal medium (time nodes).\n\nThat said, I recognize the novelty and potential impact of this work on the time-series literature, particularly in its approach to handling irregular time series. Therefore, I maintain my positive score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review and encouraging feedback. We address the main concerns as follows:\n## Q1. Why set-based method fails to capture correlations among observations?\n**A1:** We summarize the reasons from three aspects, using SeFT [1] as an example:\n### 1. **Experimental aspect**\n- **Complexity**\n\n    As mentioned **in Section 3.3 of the SeFT paper**:\n    \n    > \"By contrast, our approach computes the embeddings of set elements **independently**, leading to lower runtime and memory complexity of $\\mathcal{O}(n)$\".\n\n    SeFT is compared with vanilla Transformers that have an $\\mathcal{O}(n^2)$ complexity.\n\n- **Classification accuracy**\n\n    In the same section, the paper mentioned:\n    > \"Furthermore, we observed that computing embeddings with information from other set elements (as the Transformer does) actually **decreases generalization performance** in several scenarios\".\n\n    We note that this conclusion only applies to **classification** task, not **forecasting** task.\n### 2. **Task aspect**\nSeFT was originally designed for classification rather than forecasting.\nThe **task assumptions** used for time series classification and forecasting have minor differences:\n||Classification|Forecasting|\n|---|---|---|\n|dependency *between* observations|Not necessary|Yes\n|*when* observations happened|Yes|Yes\n\nIn classification, a few critical observations can be sufficient to determine the class label without considering their event order (e.g., a low heartbeat can signify potential health risks for patients); As for forecasting, future predictions depends on past observations, so their correlations are important.\n\nFrom experimental results, including dependencies between observations has been found to lower classification accuracy for SeFT, as mentioned previously.\n### 3. **Modeling aspect**\nSet-based methods view observations as set elements, which are **invariant to their order**.\nThe output of set functions doesn't change if observations are shuffled, which contrasts with models like RNNs or Transformers.\n## Q2. Proposed method has similar limitation as bipartite graphs\n**A2:** We provide a detailed explanation of how our model learns variable dependencies without relying on shared timestamps.\n### 1. **Preliminary**\nIn our model:\n- **temporal dependencies**: learned via message propagations among observations *within the same variable*\n- **variable dependencies**: learned via message propagations among observations *between different variables*.\n### 2. **Our model**\nVariable dependency learning is conducted through a **three-step** process.\n\nIn Figure 2 of our work, variables $V_1$ and $V_3$ do not have shared timestamps.\nWhen propagating messages from observation $(t_1, V_1)$ (i.e., the blue \"1\" node) to $(t_2, V_3)$ (i.e., the orange \"2\" node), the process is:\n1. **node to hyperedge**: node $(t_1, V_1)$ $\\rightarrow$ variable hyperedge $V_1$\n2. **hyperedge to hyperedge**: variable hyperedge $V_1$ $\\rightarrow$ variable hyperedge $V_3$\n3. **hyperedge to node**: variable hyperedge $V_3$ $\\rightarrow$ node $(t_2, V_3)$\n### 3. **Bipartite graph**\nIn contrast, the bipartite graph method propagates messages in a **two-step** manner. \nAs depicted in Figure 1 (d), for variable dependencies, messages can only be propagated between $V_1$ and $V_2$ (*with* shared timestamps $t_1, t_2, t_3$), but not $V_1$ and $V_3$ (*without* shared timestamps).\nWe describe the process of message propagation between $V_1$ and $V_2$ (*with* shared timestamps) for additional clarity:\n1. **variable node to time node**: variable node $V_1$ $\\rightarrow$ time node $t_1$ (or $t_2, t_3$)\n1. **time node to variable node**: time node $t_1$ (or $t_2, t_3$) $\\rightarrow$ variable node $V_2$\n\nTo summarize, bipartite graph relies on shared time nodes to propagate messages among variables, while our method HyperIMTS uses variable hyperedge interactions instead, which bypasses the requirement of shared timestamps.\n\n[1] Horn, Max, et al; \"Set Functions for Time Series\"; ICML 2020\n\n---\n**Lastly, thanks again for your careful review and appreciation! Feel free to let us know if you have any further questions or concerns :-).**", "claims_and_evidence": "The authors state that \"However, sets (Set-based methods) typically do not account for correlations among observations.\" Can you provide more explanation why the set-based method fails to capture correlations among observations?\n\nThe authors state that \"bipartite graphs are unable to propagate messages between variables without shared timestamps.\". However, I think the proposed method has a similar limitation because hyperedge (feature information) is required to capture temporal dependencies.", "methods_and_evaluation_criteria": "I think the proposed method makes sense for the suggested purpose.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "I reviewed the experimental results in this paper. The inclusion of numerous baselines enhances the robustness of the findings, and the proposed method demonstrates strong performance.", "relation_to_broader_scientific_literature": "I believe this method has a significant impact on the time-series literature, as it introduces a novel approach for handling irregular time series models and establishes a unified benchmark for irregular time series forecasting.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "MiMdimyXwf", "reviewer": "Reviewer_Fd4V", "summary": "The paper introduces HyperIMTS, a Hypergraph neural network for forecasting Irregular Multivariate Time Series (IMTS), which are challenged by irregular time intervals and unaligned observations. Existing models either use padded samples, disrupting sampling patterns, or set function and bipartite graphs, which struggle with unaligned dependencies. HyperIMTS addresses these issues by treating observed values as nodes in a hypergraph, with temporal and variable hyperedges enabling message passing. This irregularity-aware message passing captures both temporal and variable dependencies in a time-adaptive manner, improving forecasting accuracy. Experimental results show that HyperIMTS has good performance and low computational cost.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful reading and in-depth thinking. We address the concerns as follows:\n## W1. Regarding highly accurate padding\n**A1.** We acknowledge that padding has pros and cons, which should be weighed up in IMTS forecasting.\n### 1. **Error accumulation**\nImputation errors can be amplified during subsequent predictions, as noted in previous works [3][4].\nWe conduct additional experiments to verify it:\n- Pretrain each model for imputation.\n- Pad IMTS samples into fully observed via the imputation model.\n- Train the forecasting model on imputed samples.\n\nBetter results are in **bold**:\n||MIMIC III|PhysioNet'12\n|---|---|---\n|GraFITi|0.535(**33ms**)|**0.377(34ms)**\n|GraFITi(Impute)|**↑0.520**(245ms)|↓0.465(75ms)\n|Raindrop|**0.671(75ms)**|**0.438(52ms)**\n|Raindrop(Impute)|↓0.962(211ms)|↓0.820(85ms)\n|Crossformer|**0.641(184ms)**|0.408(**80ms**)\n|Crossformer(Impute)|↓0.962(292ms)|**↑0.265**(118ms)\n\nOver half of the results show **performance degradation** after imputation, while consuming **longer training time.**\n\nAlthough the experiment uses only a subset of models and datasets, we can still conclude that **existing models are not robust enough to provide highly accurate padding values**.\n### 2. **Efficiency**\nThe goal of our work is to improve forecasting accuracy **while maintaining high efficiency**.\nHowever, the above experiments demonstrate significant efficiency degradation.\n### 3. **Informative missingness**\nAs mentioned in Section 1 of Raindrop [2]:\n> the **absence of observations can be informative** on its own and thus imputing missing observations is not necessarily beneficial\n\nAbsence actually reflects the **sampling pattern** of variables.\n\nE.g., in dataset PhysioNet'12, ALP and ALT, both liver function markers, can be observed to have nearly identical timestamps across all samples, indicating a *scheduled* sampling pattern.\n## W2. Alignment in padding-based methods V.S. Our approach\n**A2.** We apologize for any confusion with respect to alignment. Revisions are made to clarify differences:\n||Padding-based|Ours\n|---|---|---\n|*where* to align|data preprocessing & subsequent|only in time-aware similarity\n|*how* to align|padding|select shared timestamps only, or padding\n\nAlignment itself is not a problem, but *where* and *how* to align can **affect efficiency**.\nOur method aligns only in calculation of time-aware similarity by selecting shared timestamps, thereby eliminating the need to handle large padded data in other network modules.\n\n## W3. Definition of padding-based methods\n**A3.** We apologize for not fully clarifying the definition and scope of padding.\nIn this work, padding involves adding values, either zeros or predicted values (imputation), **in the sample space** before input time series are fed into the neural network, rather than within the neural network's latent space. E.g.:\n||RNN-based|ODE-based\n|---|---|---\n|*latent* space|discrete|continuous\n|*sample* space (input)|zero-padded|zero-padded\n\nFurther details on zero-padded input can be found in models' `forward` functions within the anonymized code repository provided at the end of abstract.\n## W4. More comprehensive classification of existing methods\n**A4.** Thanks for your helpful advice!\nWe revise them with the following improvements:\n1. **Definition and scope of padding-based**: Clarify the definition scope is in sample space for input time series, instead of latent space.\n2. **Classify methods from different perspectives**:\n    - **model architecture**: (1) RNN-based; (2) ODE-based; (3) GNN-based; (4) Set-based; (5) Transformer-based.\n    - **sample space padding methods**: (1) time-aligned padding; (2) patch-aligned padding; (3) non-padding.\n3. **Compare with the standard in existing works**: Clarify the differences in classification standard with existing researches [5].\n## Q1. Regarding model difference\n**A5.**\n- Our method is implemented as a **hypergraph attention network**, summarized in A3 within rebuttal to Reviewer VDUB above.\n- Main difference:\n    1. efficiency: light-weight architecture\n    2. performance: (1) aware of sample-specific alignment; (2) learn both relationships in parallel instead of sequentially, better capturing time-variable correlations.\n\n[1] Silva, Ikaro, et al; \"Predicting In-Hospital Mortality of ICU Patients: The PhysioNet/Computing in Cardiology Challenge 2012\"; Comput Cardiol (2010)\n\n[2] Zhang, Xiang, et al; \"Graph-Guided Network for Irregularly Sampled Multivariate Time Series\"; ICLR 2022\n\n[3] Wu, Sifan, et al; \"Adversarial Sparse Transformer for Time Series Forecasting\"; NeurIPS 2020\n\n[4] Ma, Qianli, et al; \"Adversarial Joint-Learning Recurrent Neural Network for Incomplete Time Series Classification\"; TPAMI 2020\n\n[5] Shukla, Satya Narayan, et al; \"A Survey on Principles, Models and Methods for Learning from Irregularly Sampled Time Series\"; arXiv:2012.00168\n\n---\n**Thank you so much for helping us improve the paper! Please let us know if you have any further questions:-).**", "claims_and_evidence": "There are no problematic main claims.", "methods_and_evaluation_criteria": "The model designed in the paper is both reasonable and effective for addressing the IMTS modeling problem. The dataset used is a commonly employed real-world dataset in IMTS modeling, which also provides practical guidance for solving real-world problems.", "theoretical_claims": "There are no theoretical claims in this paper.", "experimental_designs_or_analyses": "The experiments in the paper are sufficient, with appropriate comparison methods and thorough analysis. The use of commonly employed real-world datasets enhances the credibility of the results.", "relation_to_broader_scientific_literature": "This paper details efforts to advance IMTS forecasting across various scientific domains. The paper provides a new perspective on the forecasting problem of ISMTS to some extent and effectively improves forecasting performance.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "1. **The motivation of the paper is not persuasive enough**: The analysis of existing methods’ issues is not very convincing. For example, the authors claim that while padding increases the amount of data the model must process, this is not a critical issue, especially if the padded data is highly accurate, as it could provide more useful information for IMTS with high missing rates. Additionally, the authors emphasize the requirement for alignment in existing methods but then focus on the relationships between observations at the same time point $ t $ as an improvement, but this approach also requires aligned observation points between sequences.\n2. **Some expressions in the paper are vague or incomplete**:\n    \n    a) The definition of padding-based methods is unclear. According to the authors, padding includes directly filling in real values,  RNN-based models that learn discrete hidden states and ODE-based models that learn continuous hidden states. However, there are significant differences between these approaches, and classifying them together is somewhat oversimplified.\n    \n    b) The classification of existing methods before the related work section is not comprehensive.  I recommend revising this description to avoid misleading researchers who are not familiar with this specific field.", "other_comments_or_suggestions": "No.", "questions_for_authors": "The method discussed in the paper is non-graph-based, but it simultaneously considers inter- and intra-series relationships. What is the main difference, aside from the update of correlations, compared to models that focus on these relationships?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The idea of using hypergraphs to directly model irregular multivariate time series is both refreshing and practically useful. Despite some concerns about limited generalizability to other modalities and the complexity of implementation, the reviewers consistently agree the method is well-motivated and shows strong empirical results across five diverse datasets. The unified design, thoughtful ablations, and comprehensive comparisons make a compelling case. I believe this work makes a meaningful contribution and is ready for acceptance.
Decision: Accept (poster)