Spatiotemporal Modeling of Multivariate Signals with Graph Neural Networks and Structured State Space Models
Authors: Siyi Tang
Jared Dunnmon
Liangqiong Qu
Khaled Kamal Saab
Christopher Lee-Messer
Daniel Rubin
Venue: Submitted to ICLR 2023 (2023)
Keywords: Multivariate Signals
Graph Neural Network
Graph Structure Learning
Structured State Spaces
Time Series

Abstract: Multivariate signals are prevalent in various domains, such as healthcare, transportation systems, and space sciences. Modeling spatiotemporal dependencies in multivariate signals is challenging due to (1) long-range temporal dependencies and (2) complex spatial correlations between sensors. To address these challenges, we propose representing multivariate signals as graphs and introduce GraphS4mer, a general graph neural network (GNN) architecture that captures both spatial and temporal dependencies in multivariate signals. Specifically, (1) we leverage Structured State Spaces model (S4), a state-of-the-art sequence model, to capture long-term temporal dependencies and (2) we propose a graph structure learning layer in GraphS4mer to automatically learn the underlying graph structures in the data.  We evaluate our proposed model on three distinct tasks and show that GraphS4mer consistently improves over existing models, including (1) seizure detection from electroencephalography signals, outperforming a previous GNN with self-supervised pretraining by 3.1 points in AUROC; (2) sleep staging from polysomnography signals, a 4.1 points improvement in macro-F1 score compared to existing sleep staging models; and (3) traffic forecasting, reducing MAE by 8.8% compared to existing GNNs and by 1.4% compared to transformer-based models.

Under review as a conference paper at ICLR 2023
SPATIOTEMPORAL MODELING OF MULTIVARIATE SIG- NALS WITH GRAPH NEURAL NETWORKS AND STRUC- TURED STATE SPACE MODELS
Anonymous authors Paper under double-bline review
ABSTRACT
Multivariate signals prevalent in various domains, such healthcare, trans- portation systems, and space sciences. Modeling spatiotemporal dependencies multivariate signals is challenging due (1) long-range temporal dependen- cies and (2) complex spatial correlations between sensors. To address these challenges, we propose representing multivariate signals as graphs and introduce GRAPHS4MER, a general graph neural network (GNN) architecture that cap- tures both spatial and temporal dependencies in multivariate signals. Specifically, (1) we leverage Structured State Spaces model (S4), a state-of-the- sequence model, to capture long-term temporal dependencies and (2) we propose a graph structure learning layer in GRAPHS4MER to Iearn dynamically evolving graph structures the data. We evaluate our proposed model on three distinct tasks and show that GRAPHS4MER consistently improves (1) seizure detection from electroenccepaalogaahh signals, outperforming a pre- vious GNN with self-supervised pretraining by 3.1 points in AUROC; (2) sleep staging from polysomnography signals, a 4.1 points improvement in macro-F1 score compared to existing sleep staging models; and (3) traffic forecasting, re- ducing MAE by 8.8% compared to existing GNNs and by 1.4% compared to Transformer-bast models.
existing models, including
INTRODUCTION
Multivariate signals are time series data measured by multiple sensors and are prevalent in many real- world applications, including healthcare (Mincholé al. 2019), transportation systems (Ermagun & Levinson 2018), power systems (Negnevitsky et al. 2009), and space sciences |Camporeale et al, 2018). An example multivariate signal is scalp alecrreencephalograns (EEGs), which measure brain electrical activities using sensors placed on an individual's scalp.
Several challenges exist in modeling spatiotemporal dependencies in multivariate signals. First, many types of signals are sampled at high sampling rate, which results in long sequences that can be up to tens of thousands of time steps. Moreover, multivariate signals often involve long-range temporal correlations (Berthouze et al., 2010). Prior studies on modeling long signals often prepro-
cess the raw signals using frequency transtormations (Tang et al., 2022b; Asif al., 2020 (hhoeibi et al. 2021] Covert et al. 2019; Guillot et al. 2020 Guillot & Thorey (221] divide the signals into short windows and aggregate model predictions post-hoc (Phan & Mikkelsen [022 Pradhan
et al. 2022). However, such preprocessing steps may discard important information encoded in raw signals, as well as neglect long-range temporal dependencies the signals. Therefore, a model that capable of modeling long-range temporal correlations in raw signals is needed.
Deep sequence models, including recurrent neural networks (RNNs), convolutional neural networks (CNNs), and Transformers, have specialized variants for handling long sequences (Arjovsky et al.) 2016 Erichson et al. 2021: Katharopoulos et al. 2020 Choromanski et al.] 2021). However, they struggle to scale to long sequences of tens of thousands of time steps (Ta al. 2020). Recently, the Structured State Space sequence model (S4) (Gu et al.) (022). a deep sequence model based the classic state space model, has achieved state-of-the-art performance challenging long

---

Under review as a conference paper at ICLR 2023
Tima
Predictions
Multivariate Signals
Poaling
Linear
Class
Cheo
S4 Layers
Graph Structure Learning
Prune
GNN Layers
Classification
Time
Graph
Structure
6o)
Scale
Sultmax
A4.
Leaming
Mokilu
RoLLy
Bo0max
Forecasting
Figure 1: Architecture of GRAPHS4MER. The model has three main components: (1) stacked S4 layers to learn temporal dependencies in each sensor independently; (2) graph structure learning (GSL) layer to learn dynamically evolving graph structures; (3) GNN layers to learn spatial depen- dencies based on S4 embeddings and learned graph structures. For GSL, adopt (a) self-attention and (b) a learnable embedding for inductive and transductive settings, respectively.
sequence modeling tasks, such as the Long Range Arena benchmark (Tay et al. 2020), raw speech classification (Gu et al., 2022), and audio generation (Goel et al. (022)).
Second, sensors have complex, Euclidenn spatial correlations. For example, EEG sensors mea- sure highly correlated yet unique electrical activities from different brain regions (Michel & Murray) 2012]; traffic speeds are correlated not only based on physical distances between traffic sensors, but also dependent on the traffic flows (Li et al. [018b). Graphs are data structures that can model com- plex, non-Euciie Eucllduld correlations in the data (Chami et al. 2022, Bronstein et al., 2017). Previous works have adopted temporal graph neural networks (GNNs) in modeling multivariate time series, such as EEG-based seizure detection (Covert et al. 2019) and classification (Tang et al. (0226),
traffic forecasting (Li al., 2018b) Wu et al. 2019 Zheng al. [0020b] Tiang & Luo| [022]] Tian
& Chan 202I). and pandemic forecasting (Panagopoulos et al. [021]] Kapoor et al. (020). Never- theless, most of these studies use sequences up hundreds of time steps and require predefined,
static graph structure. However, the graph structure of multivariate signals may not be easily defined due to unknown sensor locations. For instance, while EEG sensors are typically placed according
to the 10-20 standard placement (Jasper 1958), the exact locations of sensors vary in each indi- vidual's recordings due to the variability in individual head size. Moreover, the underlying graph connectivity can evolve over time due to temporal dynamics in the data. Hence, when graph struc- tures cannot be casily predefined, the ability to dynamically learn the underlying graph structures is highly desired.
Graph structure learning (GSL) aims to jointly learn an optimized graph structure and its node and graph representations (Zhu et al. 2021 GSL techniques have been used in non-temporal graph applications, such as natural language processing (Xu et al.] 2022]. molecular optimization (Fu al., 2021), learning on point clouds (Wang et al.) 2019), and improving GNN robustness against adversarial attacks (Zhang & Zitnik 2020) Jin et al. (222). GSL. has also been employed for spa- tiotemporal modeling of traffic flows (Zhang et al. 2020 Tang et al., 2022a| (hhan et al. 202T Wu et al. 2019; Bai et al. (2020), irregularly sampled multivariate time series (Zhang et al. 2022). functional magnetic resonance imaging (IMRI) (El-Gazzar et al. 202 Gazzar et al.) 2022b), and sleep staging (Jia et al., 2020), but they are limited to sequences of less than 1k time steps and do not capture dynamic graph structures evolving over time.
In this study, we address the foregoing challenges by (1) leveraging S4 to enable long-range temporal modeling and (2) proposing a graph structure learning layer to learn dynamically evolving graph structures in multivariate signals. Our main contributions are:
general end-to-end GNN architecture for spa- tiotemporal modeling of multivariate signals. Our model has two major advantages: (1) leverages S4 to capture long-range temporal dependencies in signals and (2) it is able to dynamically learn the underlying graph structures in the data without a predefined graph.
We propose GRAPHS4MER (Figure
We evaluate GRAPHS 4MER on three datasets with distinct data types and tasks. Our model consistently outperforms existing methods on (1) seizure detection from EEG signals, out- performing a previous GNN with self-supervised pretraining by 3.1 points in AUROC; (2) sleep staging from polysomnography signals, outpeforming existing sleep staging models
2

---

Under review as a conference paper at ICLR 2023
by 4.1 points in macro-F1 score; (3) traffic forecasting on PEMS-BA BAY benchmark, reduc- ing MAE by 8.8% compared to existing GNNs and by 1.4% compared to Transformers.
Qualitative interpretability analysis suggests that our GSL method learns meaningful graph structures that reflect the underlying seizure classes and sleep stages.
2 RELATED WORKS
Temporal graph neural networks. Temporal GNNs have been widely used for multivariate time series. A recent work by Gao & Ribeiro (022) shows that existing temporal GNNs can be grouped into two categories: time-and-graph and time-then-graph. The time-and-graph framework treats a temporal graph sequence of graph snapshots, constructs a graph representation for each snap- shot, and embeds the temporal evolution using sequence models. For example, GCRN (Seo et al.) 2018) uses spectral graph convolution (Defferrard et al. (2016) for node representations and Long Short-Term Memory network (LSTM) (Hochreiter & Schmidhuber 1997) for temporal dynamics modeling. DCRNN (Li et al.] 2018b) integrates diffussion convolution with Gated Recurrent Units (GRU) (Cho et al., 2014) for traffic forecasting. Li et al. (2019) combines relational graph convo- lution with LSTM for path failure prediction. EvolveGCN (Pareja et al. 2020) treats graph convo- lutional network (GCN) parameters recurrent states and uses recurrent neural network (RNN) to evolve the GCN parameters. In contrast, tmmetthen-graph graph framework sequentially represents the temporal evolution of node and edge attributes, and uses these temporal representations to build static graph representation (da Xu et al. 2020; Rossi et al. 2020). Gao & Ribeiro (2022) proposes general framework for time-then-grapp graph, which represents the evolution of node and edge features using two RNNs independently, constructs a static graph from these representations, and encodes them using GNN. However, most of the existing temporal GNNs are limited short sequences and adopt RNNs or self-attention for temporal modeling, which may be suboptimal when applied to long sequences. To our knowledge, only one study has leveraged S4 for temporal graphs (Gazzar it al. 2022a). However, the learned graph is static and shared across all data points, and it is limited to fMRI data.
Graph structure learning. GSL aims to jointly learn graph structure and its corresponding node (or graph) representations. Briefly, GSL methods fall into the following three categories (Zhu al.] 2021]: (1) metric-based approaches that derive edge weights from node embeddings based on metric function (Li et al. 2018a| Chen et al. 2020 Zhang & Zitnik 2020). (2) neural-based ap- proaches that use neural networks to learn the graph structure from node embeddings (Luo et al.) 2021 Zheng et al., 2020aa] Jia et al.) 2020), and (3) direct approaches that treat the adjacency matrix or node embedding dictionary learnable parameters (Jin et al. 2020) Gao et al. (020). GSL has been applied to temporal graph data. Zhang et al. (2020) proposes learning graph adjacency matrix from traffic data through learnable parameters. [Tng al. (2022a) uses multi- layer perceptron and a similarity measure to learn the graph structure for traffic forecasting, |Shang et al.) (202I) leverages link predictor (two fully connected layers) obtain the edge weights. Several works adaptively learn the adjacency matrix from data using learnable node embedding dictionaries (Wu et al. 2019 El-Gazzar al. 202T]: Gazzar et al., (022a). However, these GSL approaches are limited to short sequences of less than 1k time steps and do not capture dynamically evolving graph structures.
3 BACKGROUND
3.1 STRUCTURED STATE SPACE MODEL (S4)
The Structured State Space sequence model (S4) (Gu al.,
2022) is deep sequence model that is capable of capturing long-range dependencies. S4 has achieved state-of-the-art on pixel-level 1-D image classification, the challenging Long Range Arena benchmark for long sequence modeling,
raw speech classification, and audio generation (Gu et al. 2022 Goel et al. 2022).
S4 is based on the state space model (SSM). A recent line of work (Gu et al., 2020 2021 Gu
et al. 2022) has shown that SSMs can be viewed as both convolutional neural networks (CNNs) and RNNs. The continuous-time SSM defined as follows, which maps an 1-D signal u(l) to a high dimensional latent state 2r() before projecting to a 1-D output signal y()):
x' (1) Az(!) + Bu((), (() Ca(() + Du(()
3

---

Under review as a conference paper at ICLR 2023
Equation can be used black-box sequence representation in deep learning model, where A, B, C, and are parameters learned by gradient descent. the rest of this section, D omitted for exposition because Du(t) can be viewed skip connection and can be easily computed (Gu et al. 2022).
Gu al. (2021) shows that discrete-time SSM is equivalent to the following convolution:
K == (CB, CAB ..... CA L B),
y
K
where A, B, and C are the discretized matrices of A, B, and C, respectively; K is called the SSM convolution kernel; and L is the sequence length.
S4 (Gu al. 2022 is special instantiation of SSMs that parameterizes A diagonal plus low-rank (DPLR) matrix. This parameterization has two major advantages. First, it enables fast
computation of the SSM kernel K. Second, the parameterization includes the HipPO matrices (Gu
et al. 2020), which are class of matrices that is capable of capturing long-range dependencies.
However, naively applying S4 for multivariate signals mixes the signal channels with a position- wise linear layer (Gu et al. 2022), which does not take into account the underlying graph structure of multivariate signals. This motivates our development of GRAPHS4MER described Section
3.2 GRAPH REGULARIZATION
Graph regularization encourages a learned graph have several desirable properties, such as smoothness. sparsity, and connectivity (Chen et al.) 2020 Kalofolias 2016 Zhu et al.) (02T). Let X E Nx be graph data with N nodes and D features. First, a common assumption
graph signal processing is that features change smoothly between adjacent nodes (Ortega et al. 2018). Given an undirected graph with adjacency matrix W, the smoothness of the graph can be measured by the Dirichlet energy (Belkin& & Niyogi 2001
Smoothh X, W) = 2N2
W I,, - x,,13= N2 tr (x(xxLX)
(3)
W is the graph Laplacian, D is the degree matrix of W. Minimizing Equation therefore encourages the learned graph to be smooth. In practice, the normalized graph Laplacian L D- D- 1/2 is used so that it is independent of node degrees.
where tr(.) denotes the trace a matrix,
However, simply minimizing the smoothness may result in a trivial solution W = (Chen et al. 2020). To avoid this trivial solution and encourage sparsity of the learned graph, additional con- straints can be added (Chen et al. 2020 Kalofolias 2016):
- 1" (o1( Exparse ()) N
Caegre
2
where ||-1|= / is the Frobenius norm of a matrix. Intuitively, Ldegree penalizes disconnected graphs and Lsparse discourages nodes with high degrees (i.e., encourages the learned graph be sparse).
METHODS
4.1 PROBLEM SETUP
In this study, we propose general representation of multivariate signals using graphs. Let X e NxTx M be a multivariate signal, where N is the number of sensors, T is the sequence length, and M is the input dimension of the signal (typically M = 1). We represent the multivariate signal as a graph G E, W), where the set of nodes V corresponds to the sensors, and W is the adjacency matrix. Here, & and W are unknown and will be learned by our model.
the set of edges,
While our formulation general to any type of node-level or graph-level classification and regres- sion tasks, we focus graph-level classification (e.g., seizure detection) and node-level forecasting problems (e.g., traffic forecasting) in this work. For graph classification, the goal is to learn a func- tion which maps input signal to prediction, i.e., f X y {1,2, ..., C}, where C' is the number of classes. For node-level forecasting, the goal to learn function which predicts the next S' time step signals given the previous S time step signals, i.e., ʃ X (1-S3 X (.(+(())))
4

---

Under review as a conference paper at ICLR 2023
4.2 TEMPORAL MODELING WITH S4
In this work, we leverage S4 to capture long-range temporal dependencies in signals. Naively ap- plying S4 for multivariate signals mixes the N signal channels with position-wise linear layer (Gu et al. 2022), which may be suboptimal because it neglects the underlying graph structure of multi- variate signals. Instead, we use stacked S4 layers with residual connection to embed signals in each channel (sensor) independently, resulting in an embedding H E IR NxXxX for each input signal X, where is the embedding dimension.
4.3 DYNAMIC GRAPH STRUCTURE LEARNING
Graph structure learning layer. To model spatial dependencies in signals (i.c., graph adjacency matrix W), we develop a graph structure learning (GSL) layer to learn the similarities between nodes. To capture the dynamics of signals that evolve over time, GSL layer learns unique graph structure within a short time interval r, where r is a pre-specified resolution. Instead of learning unique graph each time step, we choose to learn a graph over r time interval because (1) aggregating information across a longer time interval can result in less noisy graphs and (2) it more computationally efficient. For convenience, we let the predefined resolution r be an integer and assume that the sequence length T is divisible by r without overlaps. In the remainder of this paper, we denote n, - the number of dynamic graphs.
For inductive settings where the task is to predict on unseen multivariate signals (c.g., scizure detec- tion), the goal of GSL is to learn a unique graph for each multivariate signal within time interval. To obtain the edge weight between two nodes, we adopt self-attention (Vaswani et al. (017]) to allow each node to attend to all other nodes and use the attention weight as the edge weight:
QKT vD ) for 1,2, nd
Q =? K h M W
softmax
(5)
Here, (4) RN NDD is mean-pooled S4 embeddings within time ((t 1) x x M e DXD
and M E DDD are weights projecting (t to query Q and key K, respectively; w RNN is the learned adjacency matrix within time ((t - 1) x r,t x r). Equation 5 can be easily extended to multihead self-attention (Vaswani et al. 2017).
For transductive settings where all sensors form a graph (e.g.. traffic forecasting), we use a learnable embedding ((() E RR xdd in GSL to learn graph for all multivariate signals within time interval
Bai et al. 2020
Wu et al.
2019)
softmax (aannnxLLLLyyReeUUU (*) B )()))), for t 1, 1,2,,..,,u. 7ld
Here, de is the embedding dimension, and E(t) is initialized randomly and trained end-to-end.
There are several advantages of our GSL layer compared to prior works (Wu et al. 2019 El-Gazzar jtt al., 2021; Gazzar et al. 2022a). First, our GSL layer is operated on S4 embeddings rather than raw signals, which leverages the long-range dependencies learned by S4 layers. Second, in inductive settings, our GSL layer learns a dynamic graph within a short time interval r for each multivariate signal rather than universal, static graph for all multivariate signals. Third, our GSL layer is able to learn dynamically evolving graph structures over time.
To guide the graph structure learning process, we add k-nearest neighbor (KNN) graph (((t)
KNN to the learned adjacency matrix ((() where each node's k-nearest neighbors are defined by cosine similarity between their respective values in ((),, and (t) is the mean-pooled S4 embeddings within time ((1- 1) r, x r]:
( ( (t)
KNN + (1 e) w
where e E [0, 1) is hyperparameter for the weight of the KNN graph.
To introduce graph sparsity, we prune the adjacency matrix ((() by removing edges whose weights smaller than certain threshold, i.e., w 0 if w!( () K, where is hyperparameter. For undirected graphs, we make the learned adjacency matrix symmetric by taking the average of the edge weights between two nodes.
5

---

Under review as a conference paper at ICLR 2023
Graph regularization. To encourage the learned graphs to have desirable graph properties, we in- clude three regularization terms as detailed in Equations 3 Let h(() R be the mean-pooled S4 embeddings within ((--1) xr, |xr]- The regularization loss the weighted sum of the three reg- ularization terms and averaged across all dynamic graphs: Creg - فe Ex=1 C (h()) W(!)) +
Geerre (W (W(+)) + Thpame (W( where a, B, and ry are hyperparameters.
4.4 MODEL ARCHITECTURE
The overall architecture of our model (Figure T and Algorithm T in Appendix A GRAPHS4MER, consists of three main components: (1) stacked S4 layers with residual connection model temporal dependencies in signals within each sensor independently, which maps raw signals X E IN x to S4 embedding H E R x (2) a GSL layer to learn dynamically evolving adjacency matri- ces W W(ma), and (3) GNN layers to learn spatial dependencies between sensors given the ..... W(n)) and node features H. While our model is general to any kind of GNN layers, we use an expressive GNN architecture, Graph Isomorphism Network (GIN) (Xu et al. 2019 Hu et al. 2020), in our experiments.
learned graph structures W(1,
For graph classification tasks, a temporal pooling layer and a graph pooling layer are added to aggregate temporal and spatial representations, respectively, which is followed by a fully connected layer to produce prediction for each multivariate signal. Whereas for node forecasting tasks, a fully connected layer added to project the outputs of the GNN layers to the output dimension. The total loss is the sum of the regularization loss and the prediction loss. Figure [ architecture, and Algorithm [ in Appendix A shows the pseudocode for GRAPHS4MER, Source code is included as supplemental material.
shows the overall model
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUP
In this section, we briefly introduce the datasets and experimental setup in our study. For each exper- iment, we ran three runs with different random seeds and report mean and standard deviation of the results. See Appendix B for details on datasets, model training procedures, and hyperparameters.
Seizure detection from EEGs. We first evaluate our model on BEG-based seizure detection. We use the publicly available Temple University Hospital Seizure Detection Corpus (TUSZ) v1.5.2 (Shah
et al. 2018). and follow the experimental setup of seizure detection 60-s EEG clips as in Tang ct al. 2022b). The number of EEG sensors is 19. The sampling rate of each 60-s EEG clip is 200 Hz, resulting in sequence length of 12k time steps. The task is binary classification of detecting whether or not a 60-s EEG clip contains seizure. The resolution r in GSL layer is chosen as 10-s (i.e., 2,000 time steps), which is inspired by how trained BEG readers analyze EEGs. As this is an inductive graph classification task, we adopt Equation 5 for GSL. The EEG graphs are undirected.
We compare our model performance existing models for seizure detection, including (1) LSTM (Hochreiter & Schmidhuber 1997). variant of RNN with gating mechanisms; (2) Dense-CNN (Sab et al. 2020). a densely connected CNN specialized in seizure detection; (3) CNN-LSTM -Ahmedt-Arisizzaall et al., 2020); and (4) Dist- and CoR--DCRNN without and with self-supervised pre-training (Tang et al. 2022b). Following prior studies, we use AUROC as the evaluation metric.
Sleep staging from polysomnography signals. Next, we evaluate our model on sleep staging from polysomnography (PSG) signals. PSG is used in the diagnosis of sleep disorders such as obstructive sleep apnea. We use the publicly available Dreem Open Dataset-Healthy (DOD-H) (Guillot et al., (020).. We randomly split the data by 60/20/20 into train/validation/nest sets, where each split consists of data from distinct subjects. The number of PSG sensors is 16. The sampling rate is 250 Hz, and thus each 30-s signal has 7,500 time steps. The task is to classify each 30-s PSG signal
of the five sleep stages: wake, rapid eye movement (REM), non-REM sleep stages N1, N2, and N3. The resolution r is 10-s (i.e., 2.5k time steps). This also an inductive graph classification task, and thus adopt Equation S for GSL. The PSG graphs are undirected.
Baseline models include existing sleep staging models that achieved state-of-the-ar on DOD-H dataset, SimpleSleepNet (Guillot al. 2020) and RobustSleepNet (Guillot & Thorey 2021). both

---

Under review as a conference paper at ICLR 2023
of which are based on GRUs (Cho et al. (0144) and attention mechanisms. We also include the tradi- tional sequence model LSTM (Hochreiter & Schmidhuber 1997) as a baseline. For fair comparisons between the baselines and GRAPHS4MER, we trained SimpleSleepNet and RobustSleepNet using their open sourced code and setting the temporal context to be one 30-s PSG signal. Similar to Guillot & Thorey| (2021) we use macro-F1 score and Cohen's Kappa as the evaluation metrics.
Traffic forecasting. Finally, we evaluate our model on forecasting of traffic flows on the public PEMS-BAY dataset (Li al., 2018b). The traffic dataset is collected by California Transportation Agencies (Cal'Trans) Performance Measurement System (PeMS) from January 1st 2017 to May 3sst 2017. We follow the same experimental setup as that in Li et al. (2018b). The number of traffic sensors is 325. The traffic speed readings are aggregated into 5-min windows and the data are split by 70/10/20 into train/validationneest sets. The task is to predict the future hour traffic flow (i.e., prediction horizon 12 time steps) given the previous 1 hour traffic flow (12 time steps). The resolution r is set to be 12 time steps. This is a transductive node-level forecasting task, therefore we adopt Equation 6 for graph structure learning. The traffic graph is directed.
We compare our model performance to existing traffic forecasting models: (1) DCRNN (Li et al., 2018b), (2) Graph WaveNet (Wu et al.) 2019), (3) Graph Multi-Attention Network (GMAN) (Zheng et al. (020b), (4) Traffic Transformer (Cai et al. 2020), (5) Spacetimeformer (Grigsby et al. 2022
and (6) Adaptive Graph Oppatil-Tempprrs Transformer Network (ASTTN) (Feng & Tassiulas
2022
DCRNN, Graph WaveNet, and GMAN are temporal GNNs, whereas the rest are Transformer- models. Following previous works,
mean absolute error (MAE), root mean squared error (RMSE), and mean absolute percentage error (MAPE) as the evaluation metrics.
5.2 EXPERIMENTAL RESULTS
Seizure detection from EEG signals. Table Ishlos our model performance on EEG-based seizure detection and its comparison to existing models. GRAPHS4MER outperforms the previous state- of-the-art, Dist-DCRNN with pretraining, by 3.1 points in AUROC. Notably, Dist-DCRNN was pretrained using self-supervised task (Tang et al. 2022b), whereas GRAPHS4MER was trained from scratch.
Table 1: Results of EEG-based seizure detection TUSZ dataset. Baseline model results cited from Tang et al. (202b)). Best and second best results are bolded and underlined, respectively.
Model
AUROC
LSTM (Hochreiter & Schmidhuber
1997
0.715 0.016
Dense- (Siah el nl. 2020)
0.796 0.014
CNN-LSTM (Ahmedi- Arsttzzaall
2020
0.682 0.003
Dist-DCRNN Pretraming [ang
2h
0.793 01022
Con-DDCNNN w/o Pretraining
[ang et al
2022b
0.804 = 0.015
Dist-DCRNN w/ Pretraining
[Tang et al.
20226
0.875 0.016
Con-DCRNN w/ Pretraining
Tang ct
20226
0,850 0.014
GRAPHS4MER (ours)
0.906 0.012
Sleep staging from PSG signals. Table 2 compares our model performance to existing sleep stag- ing models on DOD-H dataset. GRAPHS4MER outperforms RobustSleepNet, specialized sleep staging model, by 4.1 points in macro-F1. It is important to note that both SimpleSleepNet and RobustSleepNet preprocess the PSG signals using short-time Fourier transform, whereas our model directly operates raw PSG signals without the need of preprocessing.
Traffic forecasting. Table 3 shows the performance of our model and existing traffic forecasting models PEMS-BAY. We observe that GRAPHS4MER outperforms temporal GNNs (DCRNN, Graph WaveNet, GMAN) by a large margin and performs better than Transformer-bait models (ASTTN and Spacetimeformer) in MAE.
Notably, the three tasks cover a wide range of sequence lengths ranging from 12 to 12k time steps. GRAPHS4MER outperforms previous state-of-the-arr all of the three tasks, suggesting that it is capable of capturing long-range temporal dependencies across a wide range of sequence lengths.
1earnin og-en https///oohhhttteee///////..o//a htte http//ooobutosoee NepNeet

---

Under review as a conference paper at ICLR 2023
Table Results of sleep staging on DOD-H dataset. Best and second best results are bolded and underlined, respectively.
Model
Macro-F1
Kappa
L.STM (Hochreiter & Schmidhuber, 1997
0.609 0.034
0.539 0.046
SimpleSIeepNet (unllor al 2020
0.720 0.001
0.703 0.013
RobustSleeppet (Guiloo & Thorcy 2021
0.777 0.007
0.758 + 0.008
GRAPHS4MER (ours)
0.818 0.008
0.802 0.014
Table 3: Results of traffic forecasting PEMS-BAY dataset. Baseline model results are cited
from the literature. Best and second best results are bolded and underlined, respectively.
Model
MAE
RMSE
MAPE
DCRNN (Li et al
80
2.07
4.74
4.90%
Graph WaveNet
al. 2019
1.95
4.52
4.63%
GMAN Zheng el
20206
1.86
4.32
4.31%
Traftic Transtormer (Cai et 2020
1.77
4.36
4.29%
Spacetimeformer (Giysby el 2022
1.73
3,79
3.85 %
ASTTN (Feng & Taassulas, 2022
1.72
4.02
3.98%
GRAPHS4MER (ours)
1.696
0.008
3.989 + 0.023 3.86 9.
0.02%
Effect of GSL. As GSL is an important component of GRAPHS4MER, we investigate the effect of GSL on the model performance. Table 4 shows the model performance when we remove the GSL layer in GRAPHS4MER and use predefined graphs. For seizure detection on TUSZ dataset, we use the predefined distance-basee based EEG graph in Tang et al. (2022b); for traffic forecasting on PEMS-BAY, we use the predefined traffic sensor graph as in Li et al. 20). DOD-H dataset has no predefined graph available, and thus we use KNN graph obtained based on cosine similarity be- tween nodes using S4 embeddings (K = 3; same as GRAPHS4MER) We observe that GRAPHS4M using GSL outperforms GRAPHS4MER with predefined graph on the TUSZ and DOD-H datasets, but not on the PEMS- BAY dataset. This suggests that sensor locations used for constructing the predefined traffic sensor graph (Li et al. 2018b) are critical for traffic forecasting. Nevertheless, a major advantage of our GSL method is that it does not require prior knowledge about the graph structure, which would be particularly useful when sensor locations are not available.
Table 4: Comparison of predefined graphs vs GSL. Best results are bolded.
Model
TUSZ (AUROC)
DOD-H (Macro-FI)
PEMS-BAS BAY (MAE)
GRAPHS4MER w/ predefined graph (w/ GSL)
0.899 +: 0.010
0.765 + 0.016
1.629 + 0.003
GRAPHS4MER (w/ GSL)
0.012
0.818
1,696
Ablations. We perform ablation studies to investigate the importance of (1) graph-based representa- tion, where remove the GSL and GNN layers our model and (2) S4 encoder, where replace the S4 layers with GRUs (Cho et al.) 2014). Table 5 shows the ablation results. There are several important observations. First, GRAPHS4MER consistently outperforms S4, suggesting the effec- tiveness of representing multivariate signals as graphs. Second, when S4 layers are replaced with GRUs, the model performance drops by a large margin, particularly on TUH and DOD-H datasets with long sequences (12k and 7.5k time steps, respectively). This indicates the effectiveness of S4 modeling long-term temporal dependencies.
Effect of temporal resolution. We examine the effect of temporal resolution in Figure in Appendix D We observe that dynamically varying graph structures are more useful for inductive tasks (i.c.. seizure detection and sleep staging) than the transductive traffic forecasting task.
Visualization of graphs. To investigate if the learned graphs are interpretable, we visualize mean adjacency matrices for EEG and PSG signals in the correctly predicted test samples in Figure 2 grouped by seizure classes and sleep stages. For EEG, we observe that the differences between generalized seizure and non-seizure adjacency matrices (Figure 2b) larger than the differences between focal seizure and non non-seizure adjacency matrices (Figure (t). This suggests that general- ized seizures disrupt normal brain connectivity more than focal seizures, which is consistent with
8

---

Under review as a conference paper at ICLR 2023
Table 5: Ablation results. For S4, we remove the GSL and GNN layers. For GRAPHS4MER w/o $4, we replace S4 with GRUs. Best and second best results are bolded and underlined, respectively.
Model
TUSZ (AUROC)
DOD-H (Macro-F1)
PEMS-BAY (MAE)
S4
0.824 + 0.011
0,778 + 0.009
2.262 + 0.004
GRAPHS4MER w/o $4
0,705 + 0.095
0.634 + 0.061
1.723 0.006
GRAPHS4MER
0,906 + 0.012
0.818 + 0.008
1.696 0.008
(o) Focal Sequres
(b) Genneskizedd Seizures
[6] Nor-Swoure
(d) Focal Non Seizure
(e) Generalized Non zure
EEG
EEST
22 EE FEUT
E2 DOR BOCOPOCR
0.0
0.
10 WAKE
N1
WAYE
WAKKE NCI
704
0.0
2114
PSG
202227 33
0.0
85
Figure 2: Top: Mean adjacency matrix for EEG in correctly predicted test samples for (a) focal seizures, (b) generalized seizures, and (c) non-seizure EEG clips. (d) Difference between focal seizure and non-seizure adjacency matrices. (e) Difference between generalized seizure and non- seizure adjacency matrices. Bottom: Mean adjacency matrix for PSG in correctly predicted test samples for sleep stage (f) WAKE, (g) NI, and (h) N3. (i) Difference between WAKE and NI adjacency matrices. (j) Difference between WAKE and N3 adjacency matrices. Self-edges (i.c.,
diagonal) not shown here.
the literature that brain functional connectivity changes significantly in generalized seizures (Sun) et al. (02T). For PSG, we observe that the differences between N3 and wake (Figure are larger than the differences between NI and wake (Figure 2) This pattern is expected given that N1 the earliest sleep stage, whereas N3 is the deep sleep stage and is associated with slow brain waves that are not present in other sleep stages (Academy Sleep Medicine 2007). Mean adjacency matrices for sleep stages N2 and REM shown in Appendix and Figure 4 In addition, visualize the
learned adjacency matrix for traffic forecasting in Appendix F and Figure 5 We also show example
dynamic graphs for EEG clip in Appendix G and Figure
6 CONCLUSION AND FUTURE WORK
In conclusion, we propose a novel graph structure learning method to learn dynamically evolving graph structures in multivariate signals, and present GRAPHS4MER, general GNN architecture integrating Structured State Spaces model and graph structure learning for spatiotemporal modeling of multivariate signals. Our method sets new otate-of-the-art on seizure detection, sleep staging, and traffic forecasting, and learns meaningful graph structures that reflect seizure classes and sleep stages. Importantly, our method does not require prior knowledge about the graph structure, and thus would be particularly useful for applications where graph structures cannot be easily predefined. Our study opens exciting future directions for graph-based spatiotemporal modeling of multivari- ate signals, including (1) leveraging domain knowledge for improved graph structure learning, (2) investigating other ways combining S4 and GSL to further improve long-range graph structure learning, and (3) applying to other types of time series.

---

Under review as a conference paper at ICLR 2023
ETHICS STATEMENT
The Temple University Hospital EEG Seizure Corpus (TUSZ) is publicty available and de- identified with full IRB approval (Sahh et al. 2018 The DOD-H dataset is publicly available deidentified, and was approved by the French Committees of Protection of Persons (Guillot al. 2020). The PEMS-BAY dataset is publicaly available and no human or animal subject is involved (Li et al. 2018b). No conflict of interest is reported from the authors. No harmful insights are provided by the models. While we show that our methods could provide improved performance for seizure detection and sleep staging, additional model validations are needed before they can be used in real-world clinical settings, such (a) external validation on datasets from multiple institutions and (b) validation on subjects with different demographics.
REPRODUCIRILITT STATEMENT
All the datasets used in our study are publicly available. Details about datasets, model training proce- dures, and hyperparameters are provided in the Appendix. Source code is included in supplemental materials.
REFERENCES
Academy Sleep Medicine. The AASM manual for the scoring sleep and associated events: rules, terminology and technical specifications. IL: American Academy of Sleep Medicine, 2007.
David Ahmedt-Aristizabal, Tharindu Fernando, Simon Denman, Lars Petersson, Matthew J Aburn, and Clinton Fookes. Neural memory networks for seizure type classification. Conf. Proc. IEEE Eng. Med. Biol. Soc., 2020:569-575, July 2020.
Martin Arjovsky, Amar Shah, and Yoshua Bengio. Unitary evolution recurrent neural networks. In Maria Florina Balcan and Kilian Q Weinberger (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume of Proceedings of Muchine Leurning Research, pp- 1720-1128, New York, New York, USA, 2016.
Umar Asif, Subhrajit Roy, Jianbin Tang, and Stefan Harrer. SeizureNet: Multi-Spectral deep fea- ture learning for seizure type classification. In Machine Learning in Clinical Neuroimaging and Radiogenomics Neuro-oncology, pp. 77-87. Springer International Publishing. 2020.
LEI Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang- Adaptive graph convolutional recurrent network for traffic forecasting. In H Larochelle, M Ranzato, R Hadsell, M F Balcan, and H Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 17804-17815, Curran Associates, Inc., 2020.
Belkin and Niyogi. Laplacian eigenmaps and spectral techniques for embedding and clustering. Adv. Neural Inf. Process. Syst., 2001.
Luc Berthouze, Leon M James, and Simon F Farmer. Human EEG shows long-range temporal correlations of oscillation amplitude in theta, alpha and beta bands across a wide age range. Clin. Neurophysiol,, 121(8):1 187-1197, 197, August 2010.
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geomet- ric deep learning: Going beyond euclidean data. IEEE Signal Process. Mag., 34(4):18-42 July 2017.
Ling Cai, Krzysztof Janowicz, Gengehen Mai, Bo Yan, and Rui Zhu. Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting. Trans. GIS, 24(3):736-755, June 2020.
Enrico Camporeale, Simon Wing, and Jay Johnson. Weather. Elsevier, May 2018.
Machine Learning Techniques for Space
10

---

Under review as a conference paper at ICLR 2023
Chami, Abu- Abu-El-Haija, Perozzi, Ré, and others. Machine learning on graphs: A model and compre- hensive taxonomy. Journal of Machine Learning Research, 2022.
Yu Chen, Lingfei Wu, and Mohammed Zaki. Iterative deep graph learning for graph neural net-
works: Better and robust node embeddings. Adv. Neural Inf. Process. Syst. 33:19314-19322 2020.
Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-Decoder approaches. September 2014.
Krzysztof Marcin Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreca Gane, Tamas Sarlos, Peter Hawkins, Jared Quincy Davis, Afroz Mohiuddin, Lukasz Kaiser, David Benjamin Belanger, Lucy J Colwell, and Adrian Weller. Rethinking attention with per- formers. In International Conference on Learning Representations, 2021. URL https: //openzeview. www. wwtp://eneetooooomeeeeeeeee http://ooormmmddaaaaaaaaa---
Ian C Covert, Balu Krishnan, Imad Najm, Jiening Zhan, Matthew Shore, John Hixson, and Ming Jack Po. Temporal graph convolutional networks for automatic seizure detection. In Finale Doshi-Velee Jim Fackler, Ken Jung, David Kale, Rajesh Ranganath, Byron Wallace, and Jenna Wiens (eds.), Proceedings of the 4th Machine Learning for Healthcare Conference, volume 106 of Proceedings of Machine Learning Research, pp. 160-180. PMLR, 2019.
da Xu, chuanwei ruan, evren korpeoglu, sushant kumar, and kannan achan. Inductive representation learning on temporal graphs. In International Conference on Learning Representations, 2020. URL /att:tss//ppeereeeee
net/ fo Forumm rJdee
Defferrard, Bresson, and others. Convolutional neural networks on graphs with fast localized spec- tral filtering. Adv. Neural Inf. Process. Syst., 2016.
Ahmed El-Gazzar, Rajat Mani Thomas, and Guido van Wingen. Dynamic adaptive graph convolution for fMRI modelling. In Ahmed Abdulkadir, Seyed Mostafa Kia, Mohamad Habes, Vinod Kumar, Jane Maryam Rondina, Chantal Tax, and Thomas Wolfers (eds.). Machine Learning in Clinical Neuroimaging, pp. 125-134, Cham, 2021. Springer International Publishing.
N. Benjamin Erichson, Omri Azencot, Alejandro Queiruga, Liam Hodgkinson, and Michael W. Mahoney. Lipschitz recurrent neural networks. In International Conference on Learning Repre- sentations, 2021.
ftr.ee//oooummiddeoooee N7PBXqOUJZ
Alireza Ermagun and David Levinson. Spatiotemporal traffic forecasting: review and proposed directions. Transp. Rev., 38(6):786-814 November 2018.
Aosong Feng and Leandros Tassiulas. Adaptive graph Spatial-Tempone transformer network for traffic flow forecasting. July 2022.
Tianfan Fu, Wenhao Gao, Cao Xiao, Jacob Yasonik, Connor W. Coley, and Jimeng Sun. Differen- tiable scaffolding tree for molecular optimization, 2021. URL httpss://axxi...ot wrw. /bs/
2109.10463
Jianfei Gao and Bruno Ribeiro. On the equivalence between temporal and static equivariant graph representations. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 7052-7076. PMLR, 2022.
Xiang Gao, Wei Hu, and Zongming Guo. Exploring Structure-Adaptive graph learning for robust Semi-Supervised classification. In 2020 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6. ieeexploreiiee..og July 2020.
Ahmed El Gazzar, Rajat Mani Thomas, and Guido Van Wingen. Improving the diagnosis of psychiatric disorders with self-super graph state space models, 2022a. URL https:
//arxiv.org/ abs /200 03331
Ahmed El Gazzar, Rajat Mani Thomas, and Guido Van Wingen. Improving the diagnosis of psychiatric disorders with supf-supe graph state space models, 2022b. URL https:
7/arxiv. warpiww..rr//bb//2200e 2331//32262120333111
11

---

Under review as a conference paper at ICLR 2023
Karan Goel, Albert Gu, Chris Donahue, and Christopher Re. It's raw! Audio generation with state- space models. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song. Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 6633.763. PMLR, 17-23 Jul 2022. URL http:htttp:/pooo.oooo
ngs .a..nngss....................... wtt.mm.pp...////////e..eee./
httll
Jake Grigsby, Zhe Wang, and Yanjun Qi. Long-range transformers for dynamic spatiotemporal forecasting, 2022. URL /ttp///wxvvvvv/.a///vv//aa. org 0200..
Gu, Dao, Ermon, Rudra, and others. Hippo: Recurrent memory with optimal polynomial projec- tions. Adv. Neural Inf. Process. Syst, 2020.
Gu, Johnson, Goel, Saab, and others. Combining recurrent, convolutional, and continuous-ting time models with linear state space layers. Adv. Neural Inf. Process. Syst., 2021.
Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations, 2022.
Antoine Guillot and Valentin Thorey. RobustSleepNet: Transfer learning for automated sleep stag- ing at scale. IEEE Trans. Neural Syst. Rehabil. Eng., 29:1441-1451, July 2021.
Antoine Guillot, Fabien Sauvet, Emmanuel H During, and Valentin Thorey. Dreem open datasets: Multi-Scored sleep datasets to compare human and automated sleep staging. IEEE Trans. Neural Syst. Rehabil. Eng., 28(9)) September 2020.
S Hochreiter and J Schmidhuber. Long short-term memory. Neural Comput., 98)) November 1997.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In International Confer- Learning Representations, 2020. URL https:: hattps://openrevieet wwtpeeiwwwwwwwweeeeeeeeeeeeeeeee net/forummid HJ1WAJSFDH
H H Jasper. The ten-twenty electrode system of the international federation. Ellectroencephalogr Clin. Neurophysiol., 10:370-375, 1958.
Ziyu Jia, Youfang Lin, Jing Wang, Ronghao Zhou, Xiaojun Ning, Yuanlai He, and Yaoshuai Zhao. Graphsleepnet: Adaptive spatial-temporal graph convolutional networks for sleep stage classification. In Christian Bessiere (ed.), Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 1324-1330. International Joint Confer- ences on Artificial Intelligence Organization, 2020. doi: 00.24963/jjeai.2220//38 URL Main track.
htttp::/doi.oro wttpoooogg/.... wttp://.....49.... 241.3/2223333138/ 2020/188
Weiwei Jiang and Jiayun Luo. Graph neural network for traffic forecasting: A survey. Expert Syst. Appl. 207: 17921, November 2022.
Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. Graph structure learning for robust graph neural networks. In Proceedings of the 26th ACM SIGKDD Inter- national Conference
Knowledge Discovery and Data Mining, KDD '20, pp. 66-74, New York, NY, USA, 2020, Association for Computing Machinery. ISBN 9781450379984. doi: 10. N0.114573394466344030444 URL https: https://doii.. http://oo..ooooo org/10 1 33/433944 94 486 3403049
Vassilis Kalofolias. How to learn a graph from smooth signals. In Proceedings of the 19th Interna- lional Conference on Artificial Intelligence and Statisties, volume 51 of Proceedingy of Machine Learning Research, pp. 920-929, Cadiz, Spain, 2016. PMLR.
Amol Kapoor, Xue Ben, Luyang Liu, Bryan Perozzi, Matt Barnes, Martin Blais, and Shawn O' Banion.. Examining covid-19 forecasting using spatio-temporal graph neural networks, 2020. URL hRl.hhtts:/:/aattt http///xxvvvrrrgaaa/ http:///s/2000..0.0 03113
Katharopoulos, Vyas, Pappas, and others. Transformers are mns: Fast autoregressive transformers with linear attention. on Machine Learning, 2020.
12

---

Under review as a conference paper at ICLR 2023
Jia Li, Zhichao Han, Hong Cheng, Jiao Su, Pengyun Wang, Jianfeng Zhang, and Lujia Pan. Predict- ing path failure in Lime-Evolving graphs. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '19, pp. 1279-1289, New York, NY, USA, July 2019. Association for Computing Machinery.
Ruoyu Li, Sheng Wang, Feiyun Zhu, and Junzhou Huang. Adaptive graph convolutional neural networks. AAAI, 32(1), April 2018a
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural net- work: Data-driven traffic forecasting. In International Conference on Learning Representations, 2018b. URL
lyya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In In- ternational Conference on Learning Representations, 2017. URL https::/
net/ htte://oommiiimkkkeee
Iyy Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- http://wwwwwweeeeeo..eeee www.:rrmmtffoummmmmmmmmmm ddd
ence on Learning Representations, 2019. URL
Dongsheng Luo, Wei Cheng, Wenchao Yu, Bo Zong, Jingchao Ni, Haifeng Chen, and Xiang Zhang- Learning to drop: Robust graph neural network via topological denoising. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, WSDM '21, pp. 779-787, New York, NY, USA, March 2021. Association for Computing Machinery.
Christoph M Michel and Micah M Murray, Towards the utilization of EEG brain imaging tool. Neuroimage, 61(2):711-385 June 2012.
Ana Mincholé, Julià Camps, Aurore Lyon, and Blanca Rodríguez. Machine learning in the electro- cardiogram. J. Electrocardiol, 57S:561-564, November 2019.
Michael Negnevitsky, Paras Mandal, and Anurag K Srivastava. Machine learning applications for load, price and wind power prediction in power systems. In 2009 15th International Conference on Intelligent System Applications to Power Systems, pp. 1-6. iecexplore.iece.org November 2009.
Antonio Ortega, Pascal Frossard, Jelena Kovačević, José M Moura, and Pierre Vandergheynst. Graph signal processing: Overview, challenges, and applications. Proc. IEEE, 10665))808-828, May 2018.
George Panagopoulos, Giannis Nikolentzos, and Michalis Vazirgiannis. Transfer graph neural net- works for pandemic forecasting. AAAI, 35()):483844855, May 2021.
Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kaneza- shi, Tim Kaler, Tao Schardl, and Charles Leiserson. EvolveGCN: Evolving graph convolutional networks for dynamic graphs. AAAI, 34(04))5333-5370, April 2020.
Huy Phan and Kaare Mikkelsen. Automatic sleep staging of EEG signals: recent development, challenges, and future directions. Physiol. Meas., 43(4), April 2022.
Ashirbad Pradhan, Jiayuan He, and Ning Jiang. Hand gesture recognition and biometric authenti- cation using multi-day dataset. In Intelligent Robotics and Applications, pp. 375-385, Springer International Publishing, 2022.
llne M Rosen, Douglas B Kirsch, Kelly A Carden, Raman K Malhotra, Kannan Ramar, R Nisha Aurora, David A Kristo, Jennifer L Martin, Eric J Olson, Carol L Rosen, James A Rowley, Anita V Shelgikar, and American Academy of Sleep Medicine Board of Directors. Clinical use of a home sleep apnea test: An updated american academy of sleep medicine position statement. J. Clin. Sleep Med., (4(12):207552077 December 2018.
Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning on dynamic graphs, 2020. h
7/arxiv.oo Tlarxiw..rrg/bb//000000 10637
13

---

Under review as a conference paper at ICLR 2023
Khaled Saab, Jared Dunnmon, Christopher Ré, Daniel Rubin, and Christopher Lee-Messer. Weak supervision as an efficient approach for automated seizure detection in electreencephalographyy NPJ Digit Med, 3:59, April 2020.
Youngjoo Seo, Michaël Defferrard, Pierre Vandergheynst, and Xavier Bresson. Structured sequence modeling with graph convolutional recurrent networks. In Neural Information Processing, pp- 362-373. Springer International Publishing. 2018.
Vinit Shah, Eva von Weltin, Silvia Lopez, James Riley McHugh, Lillian Veloso, Meysam Gol- mohammadi, lyad Obeid, and Joseph Picone. The temple university hospital seizure detection corpus. Front. Neuroinform., 12:83, November 2018.
Chao Shang, Jie Chen, and Jinbo Bi. Discrete graph structure learning for forecasting multiple time series. In International Conference on Learning Representations, 2021. URL https: //openreview. www.:netotmddddEEEeee....
Afshin Shoeibi, Marjane Khodatars, Navid Ghassemi, Mahboobeh Jafari, Parisa Moridian, Roohal- lah Alizadehsani, Maryam Panahiazar, Fahime Khozeimeh, Assef Zare, Hossein Hosseini-Nejad, Abbas Khosravi, Amir F Atiya, Diba Aminshahidi, Sadiq Hussain, Modjtaba Rouhani, Saeid Nahavandi, and Udyavara Rajendra Acharya. Epileptic seizures detection using deep learning techniques: A review. Int. J. Environ. Res. Public Health, 18(11), May 2021.
Jintao Sun, Yihan Li, Ke Zhang, Yulei Sun, Yingfan Wang, Ailiang Miao, Jing Xiang, and Xiaoshan dynamics of functional connectivity networks during seizure ter- mination in childhood absence epilepsy: A magnetuenceephalogrpphy study. Front. Neurol., 12, 2021.
Wang.
Jiabin Tang, Tang Qian, Shijing Liu, Shengdong Du, Jie Hu, and Tianrui Li. Spatio-temporal latent graph structure learning for traffic forecasting, 2022a. URL https://arxiv.onet
2202. 12586
Siyi Tang, Jared Dunnmon, Khaled Kamal Saab, Xuan Zhang, Qianying Huang, Florian Dubost, Daniel Rubin, and Christopher Lee-Messer. Self-Supervised graph neural networks for improved electreencephalograpiic seizure analysis. In International Conference on Learning Representa- tions, April 2022b.
Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler. Long range arena: benchmark for efficient transformers, 2020. URL /ttptttsss////awii.rrrrr/a/////s
04006
Chenyu Tian and Wai Kin (victor) Chan. Spatial-temporal attention wavenet: A deep learning framework for traffic prediction considering spatial-temporal dependencies. IET Intel. Transport Syst., 15(4):549 561, April 2021.
Vaswani, Shazeer, Parmar, and others. Attention is all you need. Adv. Neural Inf. Process. Syst., 2017.
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph CNN for learning on point clouds. ACM Trans. Graph., 38(5):1-12, October 2019.
Zonghan Wu, Shirui Pan. Guodong Long, Jing Jiang, and Chenggi Zhang. Graph wavenet for deep spatial-temporal graph modeling. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, 19, pp. 1907-1913. International Joint Conferences on Artificial Intelligence Organization, 7 2019. doi: 10.24963/ijcai.2119766. URL https: 2tail/or//..24933//33 (t1.:aaa...1122255
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2019. URL https:
//openreview.
Weizhi Xu, Junfei Wu, Qiang Liu, Shu Wu, and Liang Wang. Evidence-aware fake news detection with graph neural networks. January 2022.
14

---

Under review as a conference paper at ICLR 2023
Qi Zhang, Jianlong Chang, Gaofeng Meng. Shiming Xiang, and Chunhong Pan. graph structure learning for traffic forecasting. AAAI, 3401)):177--1185 April 2020.
Xiang Zhang and Marinka Zitnik. GNNGuard: Defending graph neural networks against adversarial attacks. Adv. Neural Inf. Process. Syst., 33:9263-9275, 2020.
Xiang Zhang, Marko Zeman, Theodoros Tsiligkaridis, and Marinka Zitnik. Graph-guided network for irregularly sampled multivariate time series. In International Conference on Learning Repre- sentations, 2022. URL httpp:s///peeeeeeieeeweeee///e/p wtwpeeerreeeeeeeeeeeeeeeeeeeeeee Leeeee//
Cheng Zheng, Bo Zong, Wei Cheng, Dongjin Song, Jingchao Ni, Wenchao Yu, Haifeng Chen, and Wei Wang. Robust graph representation learning via neural sparsification. In Hal Daumé lii and Aarti Singh (eds.). Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 11458-11468 PMLR, 2020a.
Chuampan Zheng. Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. GMAN: A graph Multi-Attentic network for traffic prediction. AAAI, 34(01):1234-1411 April 2020b.
Yanqiao Zhu, Weizhi Xu, Jinghao Zhang, Yuanqi Du, Jieyu Zhang, Qiang Liu, Carl Yang, and Shu Wu. survey on graph structure learning: Progress and opportunities. March 2021.
15

---

Under review as a conference paper at ICLR 2023
APPENDIX
A PSEUDOCODE FOR GRAPHS4MER
Algorithm shows the psuedocode for GraphS4mer. Note that n2 = I is the number of dynamic graphs, where T is the sequence length and the pre-specified resolution. While there is for loop over = 1,..., Rd, the code implementation can be done batches to eliminate the need of for loop.
Algorithm 1: Pseudocode for GraphS4mer.
Input multivariate signal X, label y
Parameter: model weights 0, including weights for S4, GSL, GNN, and linear layers
Output :prediction y, learned graph adjacency matrices W(!) W(ma)
Randomly initialize model weights 0
while converged do
Liog t- 0
// step
get S4 embeddings
H <- S4(X)
for = 11,,,, nd do
htt) + meen-pool((H,, ((- // mean-pool along temporal dim
// step
graph structure learning (GSL)
Construct KNN graph, ((()) KNN> based on cosine similarity between nodes using h() inductive setting then
10
W x- GSL(h(), wil)) KNN) using Eqn.
5 and Eqn.
GSL in inductive
setting
else 12
W + GSL(WWM) KNN) using Eqn. 6 and Equ. // GSL in transductive setting
end if
15
Crep
Lrgg + aLCmmoth (t), W()) + & (W 7. (W())) using Eqn.
graph regulari zatiin loss
16
!/ step 3. get node embeddings from GNN
if graph classification task then z GN w() // 200 has shape (N, 1, Dridden)
19
else
(0) + GNN(H ((t-(H (((-)))))(((x))) W(O) // 20)) has shape (N,r, Dridden)
20
21
end if
22
end for
+ (2 (2) ( (m)) // concatenate along temporal dim
23
// step 4. pooling and fully connected layers graph classification task then
25
26
graph-pool
(teppoooll (temporal-pool
ý Linear(Z) (Z)
27
28
Eped + Ceross-ontropy (f.v) // loss for classification
29
else
30
ÿ + Linear(Z) Lpeed & LMAE($..V // loss for forecasting
31
32
end if
33
L Epeed + Creg // total loss
34
Back-propagate L to update model weights 0
35 end while
16

---

Under review as a conference paper at ICLR 2023
B DETAILS OF DATASETS
Temple University Hospital Seizure Detection Corpus (TUSZ). We use the publicly available Temporal University Hospital Seizure Detection Corpus (TUSZ) v1.5.2. for seizure detection (Shah) et al. 2018). We follow the same experimental setup as in Tang et al. (2022b). The TUSZ train set is divided into train and validation splits with distinct patients, and the TUSZ test set is held-out for model evaluation. The following 19 EEG channels are included: FPI, FP2, F3, F4, C3, C4, P3, P4, O1, O2, F7, F8, T3, T4, T5, T6, FZ. CZ. and PZ. Because the EEG signals are sampled at different sampling rate, we resample all the EEG signals to 200 Hz. Following prior study (Tang et al, 20226), we also exclude patients in the TUSZ test set who appear both the TUSZ train and test sets. The EEG signals are divided into 60-s EEG clips without overlaps, and the task is to predict whether or not an EEG clip contains seizure. Table shows the number of EEG clips and patients in the train, validation, and test splits.
Table Number of EEG clips and patients in train, validation, and test splits of TUSZ dataset.
Train
Validation Set
EEG Clips (% Seizure) 38,613 (9.3%)
Patients (% Seizure) 530 (34.0%)
EEG Clips (% Seizure) 5,503 (11.45)
Patients (% Seizure) 136.1%)
EEG Clips (% Seizure) 8,848 (14.7%)
Patients (% Seizure) (77.8%)
Dreem Open Dataset-Health, (DOD-H). We use the publicly available Dreem Open Dataset- Healthy (DOD-H) for sleep staging (Guillot et al., (020). The DOD-H dataset consists of overnight PSG sleep recordings from 25 volunteers. The PSG signals are measured from 12 EEG channels, electromyographic (EMG) channel, 2 electrooculography (EOG) channels, and electrocardiogram channel using a Siesta PSG device (Compumedics)) All the signals are sampled at 250 Hz. Fol- lowing the standard AASM Scoring Manual and Recommendations (Rosen et al. 2018), each 30-s PSG signal is annotated by a consensus of 5 experienced sleep technologists as one of the five sleep stages: wake (W), rapid eye movement (REM), non-REM sleep stages N1, N2, and N3. We ran- domly split the PSG signals by 60/20/20 into train/validationttes splits, where each split has distinct subjects. Table 7 shows the number of 30-s PSG clips, the number of subjects, and the five sleep stage distributions.
Table 7: Number of subjects and 30-s PSG clips in the train, validation, and test splits of DOD-H dataset.
30-s PSG Clips
Subjects
Total
Wake
N1
N2
N3
REM
Train Set
15
14,823
1,839
925
6,965
2,015
3,079
Validation Set
5
5,114
480
254
2,480
990
910
Test Set
5
4,725
718
326
2,434
509
738
PEMS-BAY traffic dataset. The PEMS-BAY dataset is collected by California Transportation Agencies (CallTrans) Performance Measurement System (PeMS). We follow the same experimental setup in Li et al. (2018b). Specifically, 325 sensors in the Bay Area are selected with 6 months of data ranging from January 1st to May 31st, 2017. The traffic speed readings are aggregated into 5-min windows. The total number of observed traffic data points is 16,937,179. 70% of the data are used for training, 20% are used for testing, and the remaining 10% are used for validation.
C DETAILS OF MODEL TRAINING PROCEDURES AND HYPERPARAMETERS
Model training was accomplished using the AdamW optimizer (Loshchilov & Hutter 2019) in Py- Torch on a single NVIDIA A100 GPU. All experiments were run for three runs with different ran- dom seeds. Cosine learning rate scheduler with 5-epoch warm start was used (Loshchilov & Hutter, 2017]. Model training was early stopped when the validation loss did not decrease for 20 consec- utive epochs. We performed the following hyperparameter search on the validation set: (1) initial learning rate within range [lc-4, 1e-2]: (2) dropout rate in S4 and GNN layers within range 10.1, 0.5]; (3) hidden dimension of $4 and GNN layers within range (64, 128, 256); (4) number of S4
17

---

Under review as a conference paper at ICLR 2023
layers within range {2, 3, 4/; (5) S4 bidirectionality; (6) number of GNN layers within range {1, 2}: (7) graph pooling within range (mean-powd pool, max-pool, sum-pool] (only for graph classification
tasks); (8) value of threshold for graph pruming by keeping the top x% edges (within range {1,
2, 3. 4, 5, 6, 7, 8, 9, 10)) or keeping edges whose weights > K (within range (0.1, 0.2, 0.3, 0.4,
0.5)); (9) KNN graph with K 3): (10) weight of KNN graph E [0.3, 0.6]; (11) a, 3, and
weights in graph regularization within range [O, I]: (12) learnable embedding (E in Equation 6 dimension within range {10, 12, 16) (only for traffic forecasting).
Model training and hyperparameters for seizure detection on TUSZ. dataset. As there are many more negative samples in the dataset, we undersampled the negative examples in the train set during training. We used binary cross-entropy loss the loss function. The models were trained for 100 epochs with an initial learning rate of 8c-4. The batch size was 4; dropout rate was 0.1; hidden dimension was 128; number of stacked S4 layers was 4; S4 layers were unidirectional; number of GNN layers was 1; graph pooling was max-pool and temporal pooling was mean-pool; graph pruning was done by setting a threshold
0.1, where edges whose edge weights <= 0.1 were removed; K- = 2 for KNN graph and the KNN graph weight was 0.6; B, and weights were all set to 0.05. This results in 265k trainable parameters in GRAPHS4MER. Best model was picked based on the highest AUROC on the validation set.
Model training and hyperparameters for sleep staging on DOD-H dataset. As the DOD-H dataset is highly imbalanced, we undersampled the majority classes in the train set during training. We used cross-entropy loss as the loss function. The models were trained for 100 epochs with an initial learning rate of 1e-3. The batch size was 4; dropout rate was 0.4; hidden dimension
128; number of stacked S4 layers was 4 and S4 layers were unidirectional; number of GNN layers was I; graph pooling was sum-pool and temporal pooling was mean-pool; graph pruning done by setting a threshold of 0.1; K = 3 for KNN graph and the weight for KNN graph was 0.6; a. B, and y weights were all set to 0.2. This results in 266k trainable parameters in GRAPHS4MER Best model was picked based on the highest macro-F1 score on the validation set.
Model training and hyperparameters for traffic forecasting on PEMS-BAY. Following Li et al. (0188)), we used mean absolute error (MAE) as the loss function. The models were trained for 100 epochs with an initial learning rate of le-3. The batch size was 32; dropout rate was 0.1; hidden dimension was 256; number of stacked $4 layers was 4 and $4 layers were bidirectional; number of GNN layers was 2; graph pruning was done by keeping the top 2% edges in the graph; K 3 for KNN graph and the weight for KNN graph was 0.5; 8, and ? weights were 0.1, 0.2, and 0.2, respectively: the learnable embedding dimension was 16. This results in 865k trainable parameters in GRAPHS4MER. Best model was picked based the lowest MAE the validation set.
D EFFECT OF TEMPORAL RESOLUTION
To examine the effect of temporal resolution 1' model performance, show the performance versus different values of r in Figure 3 We observe that for inductive tasks (i.e., seizure detection and sleep staging), smaller value of tends result in higher performance, which suggests that capturing dynamically varying graph structures is useful for inductive tasks. In contrast, for trans- ductive traffic forecasting task, larger value of tends give better performance, which suggests that learning a static graph is sufficient for the traffic forecasting task.
E VISUALIZATION OF PSG GRAPHS FOR FIVE SLEEP STAGES
Figure shows mean adjacency matrices for PSG signals in correctly predicted test samples grouped by five sleep stages, We observe that N3 differs from wake stage more than NI (also see Figure] 2 Moreover, in REM stage, the EMG channel has very weak connection to all other channels, which is expected given that one experiences muscle paralysis in REM stage.
F VISUALIZATION OF TRAFFIC SENSOR GRAPH
Figure 5 shows the mean adjacency matrix for traffic forecasting on PEMS- -BY dataset inferred by our GST layer, as well as the adjacency matrix in Li et al. (0018)) predefined by sensor locations.
18

---

Under review as a conference paper at ICLR 2023
Seizure Detection
Sleep Staging
Traffic Forecasting
1.0
2.0
1.9
0.5
SUROC
3.6
2000
4000
12090
0g
1sep
2300
3750
7500
1.5
6
Readiution t5me stops)
Acosuuioon (tmee stops)
Roostation (time stoos)
Figure 3: Model performance versus temporal resolution using the run with median performance. For convenience, we assume that temporal resolution chosen so that the sequence length di- visible by the resolution. Asterisk indicates the temporal resolution used to report results for GRAPHS4MER Table 5 Error bars indicate 95% confidence intervals obtained using boot- strapping with 5,000 replicates with replacement.
WAKE
(6) REM
N1
(6)
(o)
0301
02:0
93.00
85.00
19
6634
m10
100
FID
D
2C CCCREY 20
2
Figure 4: Mean adjacency matrix for PSG signals for five sleep stages in correctly predicted test samples. Self-edges (i.e., diagonal) not shown here.
The learned adjacency matrix (Figure 5) does not exhibit block structures as in the predefined adjacency matrix (5igure(5p), which could be the reason why learning the traffic sensor graph results in lower performance than the predefined sensor graph (]. This also suggests that future work should explore integrating prior knowledge of the graph the GSL layer.
1.0
(a)
(b)
0.8
0.6
0.4
0.2
0.0
Figure 5: (a) Mean adjacency matrix for traffic forecasting inferred by GSL layer. (b) Adjacency matrix in Li et al. (b) constructed based on traffic sensor locations. Self-edges (i.e., diagonal) not shown here.
19

---

Under review as a conference paper at ICLR 2023
G VISUALIZATION OF DYNAMIC EEG GRAPHS
In Figure 6 we visualize dynamic EEG graphs in an example EEG clip in the TUSZ test set, inferred by the GSL layer. In the beginning of the EEG clip (Figures 60). the seizure starts as focal seizure and the corresponding EEG graphs are sparse. Starting at the last 2-s of Figure 66. the seizure starts spread. In the last 10-s of this EEG clip (Figure 6 the seizure spreads all areas of the brain and the EEG graph becomes denser with more edges than the earlier EEG graphs. This change graph connectivity reflects the dynamics of the seizure.
(g) Distanca-based graph
(o)
11 Seizure sproads
2064
1000
1000
1973
Figure 6: Example EEG graphs of 60-s EEG clip that has seizure in the test set. In each subfigure (a)-(f), top panel shows the inferred graph from the GSL layer and bottom panel shows the raw EEG signals. (g) Distance-based graph in Tang et al. (022b), defined based on physical distances between EEG sensors in the standard 10-20 EEG placement system.
20


Reviews:
[{"review_id": "nH3kA7yZk0", "reviewer": "Reviewer_FDcr", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors present a GNN architecture combining Structured State Spaces model and graph structure learning for spatiotemporal modeling of multivariate signals. \n\nThe proposed model has two major advantages: (1) it leverages S4 to capture long-range temporal dependencies in signals and (2) it is able to dynamically learn the underlying graph structures in the data without a predefined graph.", "summary_of_the_review": "The performance of the model is evaluated on several datasets. The model performs better than multiple baseline methods. However, the noveltyof the manuscript is very limited, which causes my concern.", "strength_and_weaknesses": "Strength \n\n*The authors have carefully designed a model for mining space-time relations.\n\n*The authors evaluated the model performance on several datasets.\n\nWeaknesses\n\n*The innovation of the paper is insufficient. The proposed model is just a simple combination of some existing technologies. For example, the S4 and the attention mechanism in transformer are both existing technologies.\n\n*The idea of graph structure learning exists widely in many literatures. What is the difference between the proposed method and the existing graph structure learning[1-2]?\n\n*Explainability oriented visualization is not comprehensive. For example, there seems to be no visualization of sleep stages and traffic data. In addition, there are many edges in Figure 2 (a). But this stage does not seem to be the spread stage of epilepsy.\n\n[1] A unified structure learning framework for graph attention networks. Neurocomputing.2022\n[2] GraphSleepNet: Adaptive Spatial-Temporal Graph Convolutional Networks for Sleep Stage Classification. IJCAI.2020\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "lA0v01DyaM", "reviewer": "Reviewer_aQfe", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The manuscript proposed a time series embedding through structured state spaces + dynamic graph structure learning + GNN model to learn the representations of multi-variate time series data. The model was evaluated on three datasets (two graph-level classification tasks and one node-level forecasting task). Experiment results indicate superior performance of the proposed model over commonly-applied methods. ", "summary_of_the_review": "An interesting model for multivariate time series analysis by leveraging dynamic graph structure yet lacks sufficient algorithm details to understand the reproduce. ", "strength_and_weaknesses": "Strength: Capturing and utilizing dynamic graph structures over time is an important yet largely overlooked topic in graph-based representation learning. Model performance is also quite good. \n\nWeakness: Most importantly, the author did not provide sufficient details for the reader to understand how “Dynamic Graph Structure Learning” is performed (section 4.3). Specifically, the author mentioned that “each signal forms a graph,” which is confusing: does the statement indicate each variate forms a single graph; or does the multi-variate signal at a specific time point/short resolution form a single graph? If it is the latter case, it is unclear how the adjacency matrix W is learned. Furthermore, it is unclear how the KNN graph is constructed, especially if the length of the “short resolution” is small or, in the extreme case, a single time point. Secondly, it is unclear how the singles on multiple graphs W^(1)…W^(nd) were fed to the GNN layer. Based on the current description in 4.4 and Figure 1, it seems that the GNN layer is accepting signals defined on a single graph rather than multiple graphs. In addition, the ablation study performance shown in Table 4 indicates only a marginal benefit of using the dynamic graph structure. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "hc_BUFLm0rW", "reviewer": "Reviewer_dUxW", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper combines the Structured State Space Model (S4) (to learn time behaviors of multivariate series) and Graph Structual Learning (by coding as an attention layer in inductive settings, and a learnable embedding for transductive settings, regularized with a k-NN graph and smoothness of (representation) of the series of the learnt graphs). This leads to a general proposed architecture which is used in three examples where one needs  to follow both some specific time dynamics (encoded in S4) and  a network structure (encoded in the proposed GSL).  ", "summary_of_the_review": "My apppreciation is that the work has good potentials yet that there are several things that the authors should revise or anwer to. \nMy recommendation is that, as it stands, the work is \"marginally below the acceptance threshold\" but I would be happy to engage discussion and see improvement in the presentation of the work to have a more positive evaluation.", "strength_and_weaknesses": "Strength\n\n1- the global architecture is sound with some novelties in the combination of elements.\n\n2- numerical experiments are well conducted, on relevant and diverse tasks and datasets \n\n3- even if there is no new theoretical content, there are many interesting methodological insights in Sections 3 and 4 explaining the rationales for the proposed method.\n\n4- Excepted as mentioned below (about 4.4 and lack of details on the full pipeline), the article is well written and clear.\n\nWeaknesses\n\n1- The work combines S4 model (from Gu et al, 2022) and ideas of Graph Structural Learning (present in many works in the literature of these past years), yet it does not really offer a major new insight. For that, it is more an incremental work. \n\n2- The work is not precise enough in the description of the exact model / architecture proposed. Section 4.4 sends back ton Figure 1 yet the figure is not, for me, evocative enough. I would prefer to have a details model (with clearly precised input and trainable parameters) in 4.4.\n\n3- It seems that there is some limitation in that the work assumes that the graph comes necessarily from the multivariate series X, while in many settings data consists of separate graphs and multivariate series. And it is a question in itself to fathom or asses if the graph is well related to X. Here, it seems that one assumes to whe have a pipeline going from $X$ to $h$ (embedding from S4) then $W$ (learned graph). Maybe I didn't catch one point in eq. (8): is the learning of $E$ a way to incoporate pre-existing information on $G$ ? Or is $G$ always unknown and to be derived from $X$ ? \n\n4- There is not that much variations or explorations of the parameters. \n\nSome (more minor) points to improve are:\n\n5- The S4 models could be presented with either more insight (how to choose $L$ ? Where goes $D*$ in (2) and  (3) ?), or taken as given by the original work of Gu and al. yet with the explanation that the model of eq. (4) has a rationale coming from state  space model of time series. The current presentation is too much in-between.\n\n6- The pipeline from $X$ to the GNN layers should be clarified\n\n7-  Some of the dimensions are not clear. For instance, are the T time slots cut in intervals of size r ? Or is there some superposition in sliding windows ?\n\n8- As questioned above: is it opossible to have the situation where the input is $X$ and $G$ ?\n\n9- For eq. (9), please use a different notation for $W$ on the left and $W$ on the right. (Also, the weights in eq (7) could use another letter than W... or Use A for the learned weighted adjacency matrix if you prefer to keep W for weights).\n\n10- The effect of GSLk, as discussed in 5.2 on page 8 (and Table 4) doesn't appear to be major. Why that ? For DOD-H, why not simply use as baseline the K-NN graphs from the S4 embedding (without learning another graph) as comparison. This ablation study, remplacing GSP by pre-defined graph, looks to reduce the significance of the work so more comments and discussion is expected. \n\n11- the graphs of figure 2 are hard to interpret because it is difficult, from the plotted time-series, to assess the state of the seizure in each clip. As it stands currently, it would be clearer to compare the obtained graphs to what is obtained in other studies (and remove the time series to save space).  \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "6YGF0XDVLHd", "reviewer": "Reviewer_HRnf", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a method for long-range spatial-temporal multivariate signal forecasting, by combining S4 and graph structure learning to automatically learn the dynamic graph structures.", "summary_of_the_review": "While the motivation and writing are clear, some concept descriptions and experiment settings need improvement, and the technical contribution is not significant.", "strength_and_weaknesses": "Strength:\n1. The motivation is clear and straightforward.\n\nWeaknesses:\n1. This paper proposed to taking the output of S4 as the input of GSL to overcome the limitation of GSL on capturing long-range dependencies. Two concerns: (1) Is only taking the output of S4 as input of GSL enough? More interactions are expected between S4 and GSL to guarantee long-range structure learning. (2) As the authors claimed \"long range\", how the performance of GRAPHS4MER varies for different \"ranges\" are not clear. Does it robust? It should be discussed in the experiment.\n2. In 4.3, the authors first explained why they learn a graph for a time interval, then they claimed \"our GSL layer learns a unique graph for each data point rather than a universal graph for all data points.\" Interval or a data point? It is confusing.\n3. The interval r and the threshold k are two important hyperparameters. But the authors did not discuss how the performance varies to different r and k in the experiment.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject