Time-Conditioned Dances with Simplicial Complexes: Zigzag Filtration Curve based Supra-Hodge Convolution Networks for Time-series Forecasting
Authors: Yuzhou Chen
Yulia Gel
H. Vincent Poor
Venue: NeurIPS 2022 Accept (2022)
Keywords: Multivariate time series
Graph neural networks
Zigzag persistent homology

Abstract: Graph neural networks (GNNs) offer a new powerful alternative for multivariate time series forecasting, demonstrating remarkable success in a variety of spatio-temporal applications, from urban flow monitoring systems to health care informatics to financial analytics. Yet, such GNN models pre-dominantly capture only lower order interactions, that is, pairwise relations among nodes, and also largely ignore intrinsic time-conditioned information on the underlying topology of multivariate time series. To address these limitations, we propose a new time-aware GNN architecture which amplifies the power of the recently emerged simplicial neural networks with a time-conditioned topological knowledge representation in a form of zigzag persistence. That is, our new approach, Zigzag Filtration Curve based Supra-Hodge Convolution Networks (ZFC-SHCN) is built upon the two main components: (i) a new highly computationally efficient
zigzag persistence curve which allows us to systematically encode time-conditioned topological information, and (ii) a new temporal multiplex graph representation module for learning higher-order network interactions. We discuss theoretical properties of the proposed time-conditioned topological knowledge representation and extensively validate the new time-aware ZFC-SHCN model 
in conjunction with time series forecasting on a broad range of synthetic and real-world datasets: traffic flows, COVID-19 biosurveillance, Ethereum blockchain, surface air temperature, wind energy, and vector autoregressions. Our experiments demonstrate that the ZFC-SHCN achieves the state-of-the-art performance with lower requirements on computational costs.

Time-Conditioned Dances with Simplicial Complexes: Zigzag Filtration Curve based Supra-Hodge Convolution Networks for Time-series Forecasting
Yuzhou Chen
Yulia R. Gel Department of Mathematical Sciences University of Texas at Dallas National Science Foundation ygl @al@uudallls... edu
Department of Computer and Information Sciences Temple University yuzhou. wwweehhepeeeeenoooooomom edu
H. Vincent Poor Department of Electrical and Computer Engineering Princeton University poor@princeton.eer edu
Abstract
Graph neural networks (GNNs) offer a new powerful alternative for multivariate time series forecasting, demonstrating remarkable success variety of spatio- temporal applications. from urban flow monitoring systems to health care informat- ies financial analytics. Yet, such GNN models pre-dominantly capture only lower order interactions, that is, pairwise relations among nodes, and also largely ignore intrinsic condiionit information on the underlying topology of multivariate time series. To address these limitations, we propose new time-awer aware GNN architecture which amplifies the power of the recently emerged simplicial neural networks with time-conditioned topological knowledge representation in a form of zigzag persistence. That is, our new approach, Zigzag Filtration Curve based Supra-Hodge Convolution Networks (ZFC-SHCN) is built upon the two main components: (i) new highly computationally efficient zigzag persistence curve which allows to systematically encode time-conditioned topological information, and (ii) a new temporal multiplex graph representation module for learning higher- order network interactions. We discuss theoretical properties of the proposed cone-condttooned topological knowledge representation and extensively validate the time-aware ZFC-SHCN model in conjunction with time series forecasting on a broad range of synthetic and real-world datasets: traffic flows, COVID-19 biosurveillance, Ethereum blockchain, surface air temperature, wind energy, and vector autoregressions. Our experiments demonstrate that the ZFC-SHCN achieves the state-of-the-arr performance with lower requirements on computational costs.
Introduction
Over the last few years, graph neural networks (GNNs) have emerged as a new powerful alternative to traditional statistical and machine learning models in conjunction with univariate and multivariate time series forecasting tasks 27
40 28 Such successful applications of GNNs range from
urban traffic analytics to forecasting COVID-19 hospitalizations to electrocardiogram monitoring [3] 36 [66 10 20). However, most GNNs remain inherently static and not explicitly incorporate the
inherent time characteristics of the encoded knowledge [59 42 In turn, limitations in capturing the
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

time dimension in the knowledge representation and learning mechanisms for time-evolving data results in GNNs becoming less relevant over time and, hence, requiring frequent updates.
Furthermore, GNNs tend to pre-dominantly focus only on information propagation among nodes and also be limited in their ability describe polyadic relationships among multiple substructures of multivariate time series or multi-node interactions in dynamics graphs. However, as recently shown by 16 (1], such higher-order interactions might be the key toward better understanding of the underlying mechanisms of many real-world graph-structured phenomena. This challenge on polyadic graph interactions has been recently addressed by 124 R. who propose to model higher order substructures as simplices. Then, by borrowing the concepts of the Hodge theory, these approaches allow for generalization of the ideas of the combinatorial graph Laplacian which describes a diffusion from node to node via edges to case of diffusion over simplices. Such Hodge Laplacian construction allows for extending the notion of convolution operation to simplicial convolution, and the resulting simplicial neural networks (SNNs) are arguably one of the frontlines in graph learning today. However, these ideas have never been yet applied in conjunction with knowledge representation and learning of time-evolving objects.
Our goal here to bridge the emerging concept of time-aware learning with the recent notions of simplicial convolution, with a particular focus on explicitly integrating the core time-conditioned topological characteristics. In particular, we amplify the power of SNNs with a condicondt topological knowledge representation in form of zigzag persistence for time-indexed data and, more specifically, its new highly computationally efficient summary, Zigzag Filtration Curve. As result, our new approach, Zigzag Filtration Curve based Supra- Hodge Convolution Networks (ZFC-SHCN) enables us to systematically learn the most intrinsic time-conditioned information both on the underlying topology of the time-evolving data and higher-order interactions among various substructures.
Significance of our contributions can be summarized as follows:
ZFC-SHCN is the first approach bringing the concepts of simplicial convolution and SNNs to time-aware learning.
We propose a new highly computationally efficient summary of persistence for time-indexed data, Zigzag Filtration Curve, and derive its theoretical stability guarantees.
We validate the utility of ZFC-SHCN in conjunction with forecasting multivariate time series from diverse application domains such as traffic networks, COVID-19 biosurveillance, surface air temperature, token prices on Ethereum blockchain, wind energy, and vector au- toregressions. Our findings indicate that ZFC-SHCN delivers the state-of-the-art forecasting performance, with a significant margin and demonstrates higher computational efficiency.
2
Related Work
Time-series Forecasting and Spatio-temporal Graph Convolutional Networks Time-series fore- casting one of the core subfields in statistical sciences [15] 9. Most recently, there have appeared a number of unconventional machine learning approaches to time-series forecasting. In particular, graph convolutional network (GCN)-based models for spatio-temporal network data have emerged as a promising forecasting tool. For instance, DCRNN 42) introduces spectral graph convolu- tion into spatio-temporal network data prediction, which can capture spatio-temporal dependencies. STGCN (59 uses convolutional neural networks (CNNs) to model temporal correlations. Moreover, to infer hidden inter-dependencies between different traffic variables, [57]
3 I0
conduct a convolu-
tion operation in spatial dimension through adaptive adjacency matrices. Recent Z-GCNETs [20] develops a zigzag topological layer equipped with zigzag persistence image into GCN framework to model temporal correlations. Another promising recent direction for time series forecasting beyond GCN is a fractional-orde dynamical model proposed by 27 This approach offers an alternating scheme to determine the best estimate of the model parameters and unknown stimuli. In turn, [28 proposes a Padé approximation based exponential neural operator (Padé Exp), aiming to improve time-series forecasting with exponential operators in neural operator learning schemes. However, all of the above methods only focus on node-level representations. contrast, in this paper, we focus on both higher-order structure representation and topological information learning.
2

---

Topological Data Analysis for Graph Learning Persistent homology 25 62 is suite of tools within topological data analysis (TDA) that provides way for measuring topoloiial features of shapes and functions. The extracted topological features have been recently shown to provide invaluable insights into hidden mechanisms behind the organization and functionality of graph structured data. In particular, topological features have been actively used for node classification [61 171. link prediction [58], and graph classification [31 32 [4 30J. For instance, [31] is one of the first approaches to integrate topological features into neural networks for graph classification, while [14] proposes a versatile framework for learning multiple vectorizations of persistent diagrams on graphs. In turn, [61 17 588 apply topologiaal features to GNNs to understand and improve the message passing between nodes. Finally, [33] proposes a topological graph layer with learnable filtration functions for graph and node classification tasks, while [3] advances the ideas of multipersistenee to graph learning.
Zigzag Persistent Homology Despite its promise, regular persistent homology does not explicitly model the geometric and topological information from a sequence of topological spaces. To address this limitation, a generalization of ordinary persistence, i.e., zigzag persistent homology, based on the theory of quiver representation, has been proposed by (12). Zigzag persistence allows us to systematically describe how the homology changes over sequence of spaces. Despite its high potential, especially in conjunction with analysis of time-evolving data, zigzag persistence still remains largely theoretical concept, and its applications are yet scarce. The recent results for iime-dependent data studies include, for example, zigzag-based clustering (37,, bifurcation analysis of dynamic systems [55], and time series forecasting (2).. The memory and computational efficiency of zigzag persistence is one of the daunting challenges. Inspired by [44), we propose a novel highly computationally efficient representation of zigzag persistence for learning time-evolving data, that is, zigzag filtration curve.
Simplicial Neural Networks Modeling higher-order interactions on graphs is an emerging direction in graph representation learning. While the role of higher-order structures for graph learning has been documented for number of years 0 3 and involves such diverse applications as graph signal processing in image recognition [23). dynamics of disease transmission and biological networks, integration of higher-order graph substructures into deep learning on graphs has emerged only in 2020. As shown by [6, 55), higher-order network structures can be leveraged to boost graph learning performance. Indeed, several recent approaches (24
18) propose to leverage simplicial information to perform neural networks on graphs. However, neither of these Simplicial Neural Networks (SNNs) are integrated with a topology-basee graph convolution layer allowing to learn both time-aware persistent topological features and simplicial geometry of graphs. In this paper, we propose ZFC-SHCN to address this limitation.
49 X
3 Time-Aware Topological Learning with Zigzag Curves
Spatio-temporal Graph Construction A spatio-temporal graph is a collection of snapshots at different time steps, denoted by g (Gi, 92, GT where T is the maximum timestamp. Here G. = (V,, Ee, A.. X,) is the graph observed at time step t E T]. where V; is a finite set of [V] = N nodes, & is a set of edges, A, N is the adjacency matrix, and X, E RRdx is the node feature matrix Specifically, each row of X, is a d-dimensional feature vector of the corresponding node. For sake of notations, wherever applicable below, we omit the subscript and denote graph G. at time as G.
Background on Ordinary Persistence Tools of ordinary persistence, persistent homology (PH), allow us to study salient data shape patterns along various dimensions. By shape here we broadly understand data properties that are invariant under continuous transformations, that is, transformations that do not alter "holes" in the data, for example. bending, twisting, and stretching, The key idea is to choose some suitable scale parameter and then study a graph & not as single object but as a nested sequence of graphs, or graph filtration G1 C... C Gn = G. which is induced by monotonic changes of scale v. For example, if G is an edge-heighted graph (V, w) with weight function yielding the
w. & - R, then for each Vj. j = 1,..., ,, we set Ggu, (V. w w- (-00,, induced edge-weighted filtration. We can also consider only induced subgraphs of G with maximal degree of by for each j = 1, ....., resulting in the degree sublevel set filtration. (For more discussion on graph filtrations see (3).) Armed with this construction, we can track which shape patterns, for example, independent components, loops, and voids, emerge as the scale varies. To make the
3

---

process of pattern counting more systematic and efficient, we build an abstract simplicial complex X (g,) on each 9j. We also record complex indices ja (birth) and ja (death) at which we first or last observe each shape feature. Topological features with longer lifespans are said to persist and are likelier to yield important information on the structural organization of g.
Learning Shapes of Cine-Conditioned Data with Zigzag Persistence This construction enables us to extract the key topological descriptors from single graph G. However, in our case, we observe not a single graph but a sequence of time-evolving graphs {g) :.... g" J. How can we track shape signatures which are not just individualistic for each time stamp but characterize intrinsic properties of the observed object over time? One approach to how we can bring PH tools to analysis of time- conditioned objects is zigzag persistence. Based on the theory of quiver representations, zigzag persistence generalizes ordinary persistence to track characteristics of graphs (or other topological spaces) with inclusions going in different directions sequence of graphs (g',...,, .",, we first form a set of graph inclusions over time
[12 [1]. In particular, given time-indexed
91 U92
92 U93
93 U4
91
G2
G3
94
and then assess the compatibility of persistent topological features across unions of graphs. That is, we record indices at which topological features (dis)appear, for some given scale v.. If for a given topological feature (i.e., p..,p-e p-dimensional hole, 0 S S K, where I. is the dimension of the simplicial complex (G)) first recorded in (())), we say that the feature's birth is j. and if p first appears in (()) U G we record its birth as j 1/22. In turn, if is last seen in K(G'),
). we record its death as j. while if last seen in (g U G)+1), ). we say that death at j j+ 1/2. Let 3 be the set of all observed topological features for given 14. Collecting then births and deaths over J.
summarize all extracted information as a multiset Dr, = {(bp. do) E 1216, < dp. P E 3}. called a zigzag persistent diagram (ZPD) (where 6, and dp are the birth and death of the topological feature p respectively).
This makes zigzag persistence particularly attractive for the analysis of dynamic objects which are naturally indexed by time. However, the idea of zigzag persistence applicable far beyond learning time-evolving objects. Nevertheless, zigzag persistence still remains largely theoretical concept, with yet only a handful of applications, and one of the roadblocks hindering a broader proliferation of zigzag-based based methods in practice is their computational costs. Here we take a step toward bringing a more computationally efficient summary of zigzag persistence to real-world applications.
Time-Aware Zigzag Filtration Curves Consider a sequence of time intervals associated with a zigzag filtration over time period [41, tvl
1 + (1 (1)+) (22//2
-2) 1...........++++) (.
(1,4 +
Let Dgm77 v.. be the resulting ZPD for a given v. and M be the number of off-diagonal topological features in ZPD, i.c., DgmZZ,,- Inspired by the recent results stabilized Betti sequences by [35] and filtration curves by (44) for ordinary persistence, we propose a new simple and computationally efficient summary of zigzag persistence, called Zigzag Filtration Curve.
Definition 3.1 (Zigzag Filtration Curve (ZFC)). The zigzag filtration curve evaluated at At, = (f,-1 + 1. 14, i (1, .N., for a given v., is defined as
M
ZFC (At
Σ Ettsy (iitbyttd))) Jay)
where (to, tay) E R² is vector containing the birth and death of the j-th off off-diagonal p-dimensional topological feature in DgmZZ,,, (as such, ta < t,,,),j {1, is some suitable Lipschitz continuous function with Lipschitz constant L,, for example, a Gaussian density; and w; > 0, = 21, .N are weights such that >, W, 1. Zigzag filtration curve at ALT (1i, 1 is defined analogously. (For the sake of notational simplicity, wherever applicable in the further exposition we suppress the index p in ZFC.)
0 I p S K:; & IR2 - R
Motivated by 355 here the Lipschitz continuous function & for intervals At,, we use Gaussian density f with mean (ti-1 + 1/2, ts), while for intervals At; set the mean of ff to (t,, t4 + 1/2).

---

i= 1.2......N For both At, and At, we choose the 2 x 2-variance-covarinn matrix Σ to be the identity matrix. (See Appendix ?? for more discussion on sensitivity analysis.) Another suitable choice of & is the arctan function.
As show below, the proposed ZPC also enjoys important theoretical stability guarantees in terms of Wasserstein-1 I distance.
Proposition 3.2 (Stability of Zigzag Filtration Curve), Let DgmiZz.,, be a zigzag persistence dia- gram and DgmZZ. be its perturbed copy such that Wi (DgmZZ,, DgmZZ..) 1 < G, where W2 is Wasserstein-I distance. Then, ZFC is stable with respect to Wasserstein-1 distance.
In practice topological features various dimensions p. p = 0, o......,KK may play different roles in the learning task performance, and these roles are not known a-priori. Hence, to harness time- conditioned information encoded ZFC corresponding to different dimensions p. we propose Multi- Zigzag Filtration Curves (M-ZFCs) M-ZFCs,, E R - by stacking ZFC ZFC! "FC Figure ?? in Appendix ?? shows the both 0- and 1-dimensional ZFCs obtained from the proposed ZFC. In the following section, we demonstrate how ZFC can be integrated into neural network architectures for graph learning tasks.
ZFC
Zigzag Filtration Curve Based Supra-Hodge Convolution Networks
Given graph 9 and its historical w step graph signals X = [X, (1 ....
Xr} E R (F is the node feature dimensionality), the time-series forecasting problem is to learn a mapping function f that maps the historical data (X ixi=+1.... Xx) into the next h step data (X++1,.... Xetn). The
mapping relation is represented as follows: (Xi-wtl,... X.) L {X+th.... X.4a).
4.1 Graph convolution in the spatial dimension
Given the node embedding dictionary d (wf w x E Nxde (where x E Rd and de is the dimension of node embedding), we aim to seek non-negative function S..v = 9 (w w which represents the pairwise similarity between any two nodes u and v. Concretely, the multipiccation between T and (Wo)T can (i) give sum pooling of second-order features from the outer product of all the embedding vector pairs (wy, wy) and (ii) infer the hidden spatial dependencies of nodes
exp (ReLU (W (wx))
Suv = 9 (w w
E exp (ReLUU (w()T)²
where ReLU(-) max (0, -) is nonlinear activation function, which is used to eliminate weak connections proactively, and the role the softmax function applied to normalize the learned graph S. Inspired by the recent advancements in random walk- based graph embedding learning [47] 26). we make a graph convolution in spatial dimension, feeding a power series of the learned graph S with varying random walk steps 21,2, r} (r Z4), as follows:
H H = o (Stack(I,,S,
HS") ...
where (-) stands for a nonlinear activation function, Stack(-) the function which stacks r powered learned graphs, t,GC and H (8+1)
1.GC are the input and output activations for layer (where H:[)
GC E R x is the f-th layer's trainable weights. Next, we introduce representation learning of the higher-order graph (sub)structures using the supra-Hodge Laplacian which allows to systematically leverage the underlying topological information.
X, E RN and (e)
4.2 Supra-Hodge convolution in temporal dimension
lime-evolving data such as multivariate time series, spatio-temporal processes, and dynamic networks, often exhibit highly complex dependency among its substructures that goes far beyond what can be described by dyadic (or pairwise) interactions among nodes. Instead, such higher-order polyadic interactions can be systematically addressed using the Hodge theory. In particular, the discrete Hodge theory allows to generalize the notion of standard combinatorial graph Laplacian which describes diffusion graph G from node node via edges diffusion over higher-order
5

---

substructures of G [43 6 In turn, higher-order substructures can be modeled as k-simplices of G. (See Appendix ?? for background information on Hodge Laplacians.) Convolutional architectures on simplicial complexes based on the associated concepts of the Hodge theory have emerged as a recent direction in graph neural networks but have not yet been applied to learning time-evolving data. Our goal here is to introduce the notion of simplicial convolution and the ideas of Lopge--app to time-aware learning.
In particular, to capture time-conditioned higher-order interactions on 9 and to describe diffusion of information over simplices along the temporal dimension, we build supra-Hodge convolution operation, based on the multiplex network representation learning. (In the following for simplicity. notation without sub/superscript k stands for node-level quantities and in our experiments we always consider & E Z.). First, given the historical spatio-temports network series G 1--++1:t = (Gr=++1; 9r 9i-w+2,..- , 9t),, we consider directed connected node-aligned multiplex network, which is made up of w layers with N nodes on each layer. That is, the adjacency matrix A (a I N N (where a E (t w + ,,...,tf) defines the intra-connection between nodes u and layer a and distance matrix Dag = { {dgp) NNN quantifies the transition probability of moving from node u of layer to node u of layer 3. (Here B > a, since consider information diffusion procedures only along the temporal dimension). Next, based on the discrete Hodge theory, we propose a new Hodge Sup
k-Laplacian for multiplex graphs, called the supra-Hodge k-Laplacian L E R 1802wx4e xee
(C
D (E
D D
Sup
(2)
(Ci"))
where L8 is the Hodge k-Laplacian in layer a, Da+1 is the diagonal matrix of degrees of each k-simplex, i.e., Dx+1 = max (diag( Bg+1 [1, incidence matrix, and the r-th power of L represents r-step random walk on the Hodge k-Lappaccaan of layer which will allow every k-simplex to accumulate information from its neighbors. Hence, when k = I, we can infer the spatial dependencies between each pair of edges and capture meaningful edge information in both spatial and temporal dimensions through the lens of the supra-Hodge I-Laplacian. For instance, molecule networks, each node represents an atom and each edge is bond connecting two atoms; the bond (i.e., edge) features include bond type, ring status, and molecular charge which are closely related to atom (i.e., node) features (such as atomic total and partial charges).
and Bx-1 1 is the k-simplex-to-(4 + 1)-simplex
Since the goal of the forecasting task is predict node (i.e., 0-simplex) attribute(s) in the next few time steps, we propose a novel diffusion supra-Hodge convolution on the sliding window G I:t- We then update nodes' representations by transforming the multiplex k-simplex embedding to nodes via incidence matrices
H (((1))
CKK.H (
Sup H
S. K.SH)
(E)
(3)
H (f+1)
H
(B B
H (B+1)
(4)
where (i) in Equation e (8)
E R xde is learnable filter matrix for layer (here
K.SH
HSH and USH 4gp+1 are the intermediate and output dimensions to the f-th layer), H and
(f++)) are the input and output activations for layer (where (())) H.SH = Xxa-w+1 E Rowexdf and the historical k-simplex features of the spatio-temporal networks X
(Xxo-w+1, X Xk:t-w+2....... Xaxt E Re is reshaped as matrix Xxst-+11 with shape
6rw x d and (ii) in Equation 4
we transform the k-simplex embedding ((+1) to node embed- t,k.S1
ding H (e+1)
E N through incidence matrices.
4.3 ZFC convolution: a bridge between spatial and time dimensions
Armed with the representation learning of graph (sub)structures at each timestamp, we now discuss the ZFC convolution which allows us preserve and propagate both spatial and time-aware topological information simultaneously. The intuition behind ZFC convolution is that learns a strong connec- tion between two dimensions via two ID convolution layers, i.e., time-wise and node-wise. ZFC

---

convolution consists of three key components: (i) a linear embedding on M-ZFCs, which can learn the importance of time-aware topologiaa features for each node form saaneddimenioo--peeeeee node embedding; (ii) a time-wise ID convolution layer, where it gathers time-aware topological features from the entire space into a compact set; (iii) a node-wise 1D convolution layer, which can capture relations between different nodes. The resulted ZFC convolution operation over a M- defined as
M.-FFFC = FF (O ( M-ZFC.M-222 M=FF")
(5)
where w is the size of the window for sequence learning, M-ZFCs" denotes the M-ZFCs feature extracted from the time window with size w. OM-zFe E RVxd, is weight matrix to be learned, Fo and F are ID convolutional layers, and H,M.ZFC E R. xd is the -dimeeniini output. We then combine the embeddings from graph convolution, M-ZFCs convolution, and supra-Hodge convolution get the final embedding (((t))
H
H(e+1) t,out
[H 1,00 H.M.ZFFS, H (SSH
(1+1)
(6)
where [] denotes the concatenation of the outputs from three convolution operations, and H, (E+1) t.uu E R Xddm (where doat = d+t + det + d
GC
4.4
Gate Recurrent Unit with ZFC-SHCN
To describe the complex apatio-tempont dependencies among time series and assess a hidden state of nodes at future timestamp, we feed the final embedding ((+1) into Gated Recurrent Units (GRUs). Formally, we set the forward propagation equations of the GRUs as
four
R. 11 WR W
(
bye
= 1 W3
(Et.1) t,out
+ b3
t,ou
P tanh (w
3
1-1
H
(E+1) t.ou
+ by
L R, 2 1 + (1 - Rd C Le:
where 17(-) is activation function (e.g., ReLU, LeakyReLU), O is the elementwise product, R is the update gate and 3; is the reset gate. Here by: ba. by. WR. Wa and Wi are learnable parameters, while -- (((tt) and E are the input and output of GRU model, respectively. We then obtain W which contains both the spatio-temporal and time-aware information.
5 Experiments
5.1 Datasets
We validate our ZFC-SHCN model on six diverse data types: (i) COVID-19 datasets [51] CA, PA, and TX represent the number of COVID-19 hospitalizations in California (CA), Pennsylvania (PA), and Texas (TX) respectively; (ii) traffic datasets [6]] PeMSD4 and PeMSD8 are two real-time traffic datasets from California; (iii) synthetic multivariate time-series (MTS) datasets based on vector autoregression (VAR) 229 45 (where the VAR model is a generalization of the univariate AR process with more than time-evolving component); (iv) daily surface air temperature in CA. PA, and TX over 02/01/0220-1/33/22222 (v) Bytom token prices of Ethereum blockchain over 20///00170050//0000
the East Coast. The results on (i) (iii) are presented in the main body, and the analysis of (iv) and (v) in Appendix ?? and ??. The detailed description of each dataset is in Appendix ??. We also report results the wind speed dataset in Appendix ??.
41 53); and (vi) wind speed data of 57 stations
5.2 Baselines
We compare our proposed ZFC-SHCN with types of state-of-the-art baselines (SOAs), including
FC-LSTM [54] SFM 660) N-BEATS [6). DCRNN 42 LSTNet (38). STGCN [59]
TCN 4
DeepState 48 GavpeWaveNet
57 DeepGI.O
LRGCN (39 AGCRN 3 StemGNN 10
and Z-GCNETs 220].

---

5.3 Experimental settings
We implement ZFC-SHCN within a Pytorch framework on NVIDIA GeForee RTX 3090 GPU. We optimize all the models using an Adam optimizer for maximum of 200 epochs. The learning rate searched in {0.001, 0.003, 0.005, 0.01, ).05) and the embedding dimension is searched in (1,2,3, 5, 10). Our ZFC-SHCN is trained with batch sizes of 64 and 8 on PeMSD4 and PeMSD8, respectively. On both COVID-19 and surface air temperature datasets (i.e., CA, PA, and TX), we set the batch size to be 8. We train two ID convolutional layers for ZFC representation learning with the same hidden layer dimension nhid where nhid E {8, 16, 32, 64, 128). For PeMSD4 and PeMSD8, we consider the window size = 12 and the horizon h 3; for both COVID-19 and surface air temperature datasets, we consider a window size w = and horizon h = 15; for two simulated VAR datasets VARz, and VAR,, we set the window size and set the batch size as 8; for Bytom, we consider the window size w = and horizon h = 7, and set the batch size 8; for the wind speed dataset, we consider the window size w = 12 and horizon = 12, and set the batch size as 8. All models are evaluated in terms of Mean Absolute Error (MAE), Root Mean Square Error (RMSH), and Mean Absolute Percentage Error (MAPE). The best results are shown in bold font and the results shown with dotted underlines are the second-best results. We also perform a one-sided two-sample t-test between the best result and the best performance achieved by the runner-up, where *, ** *** denote p-value < 0.1, 0.05, 0.01 (i.e., denote significant, statistically significant, and highly statistically significant results, respectively. Code is available at httph:tt //githb com zf www.n/ccceeeee /FC- SHCN. git
10 and horizon as h 5,
5.4 Experimental results
Real datasets The experimental results on PeMSD4 and PeMSD8 traffic data are reported in Table 2 As Table [ shows, ZFC-SHCN achieves the best MAE, RMSE, and MAPE compared with SOAs on both PeMSD4 and PeMSDS. Compared to the RNN-based methods such as FC- LSTM, SFM, N-BEATS, LSTNet, and TCN, ZFC-SHCN achieves relative gains in RMSE over the runner-ups, ranging from 17.68% to 65.41% for both PeMSD4 and PeMSD8. In turn, DCRNN, STGCN, GraphWaveNer AGCRN, and StemGNN only focus on learning node-level representa- tions. Compared to them, ZFC-SHCN captures interactions and encodes higher-order structure correlations beyond pairwise relations among nodes and yields relative gain from 2.06% to 5.63% in RMSE on the traffic datasets. In addition, we compare ZFC-SHCN with the method based on the zigzag persistence image, i.e., Z-GCNETs, and find that ZFC-SHCN outperforms Z-GCNETs by 1.75% on PeMSD4 and 5.36% on PeMSD8 in terms of RMSE. Table B presents COVID-19 hospitalization prediction results (RMSE) in CA, PA, and TX, and we observe the following findings. First, our proposed ZFC-SHCN achieves state-of- the-art performance on all three datasets. For in- stance, ZFC-SHCN yields 3.61 %, 1.47%, 65.55% rel- ative gains in RMSE over the runner-ups (including both GCN-based and zigzag persistence image- methods) on three biosurveillance datasets. These results indicate that the ZFC mechanism and higher- order representation learning module play significant roles in capturing both topological information and higher-order structures. Second, as shown in Fig- ure ?? in Appendix ??, we find that, compared to the runner-up (i.e., Z-GCNETs), the predicted value of COVID-19 hospitalizations is more consistent with the ground-truth. Finally, Tables ?? and ?? in Appendix ?? present the overall prediction performances of ZFC-SHCN and representative baselines on surface air temperature and Ethereum blockchain datascts. We find that our proposed ZFC-SHCN consistently outperforms all baselines with either significant or (highly) statistically significant margin across all data, except surface air temperature in TX, where ZFC-SHCN still yiclds the best performance across all models.
Table I: Forecasting performance (RMSE) of ZFC-SHCN and top three baselines on synthetic time series following vector
autoregressions (VAR).
Model
VART.
VART,
AGCRN 3
0.50+0.03
0.44++0.02
StemGNN 10
0.51-10.02
0.42-00.22
Z-GCNETs 200
0,49:00.2
0.4410001
ZFC-SHCN (ours)
** 0.45:10..01
"0..8810.00
Synthetic datasets The evaluation results on two VAR datasets are summarized in Table Compared to the three strongest baselines (i.e., AGCRN, StemGNN, and Z-GCNETss, we observe that our pro- posed ZFC-SHCN consistently yields the best performance for all synthetic datasets. More precisely, ZFC-SHCN outperforms the runner-ups from 8.89% to 10.52% for VAR,, and VART,- Furthermore, to assess the time-wise and high network interactions, we use the global clustering coefficient (GCC) and
characteristic (EPC) as measures of higher order substructures [5). We find that
8

---

Table 2: Performance comparison of all methods on PEMSD4 and PEMSD8 traffic data.
PeMSD4
PeMSD8
Model
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
FC-LSTM
27.14
41.59
18.20
22.20
34.06
14.20
SFM 60
24.36
37.10
17.20
16.01
27.41
10,40
N-BEATS 46
25.56
39.90
17.18
19.48
28.32
13.50
DCRNN 42
24.70
38.12
17.12
17.86
27.83
11.45
LSTNet
38
24.04
37.38
17.01
20.26
31.96
11.30
STGCN
22.70
35.50
14.59
18.02
27.83
11.40
TCN
26.31
36.11
15.62
15.93
25.69
16,50
DeepState
48
26.50
33.00
15,40
19.34
27.18
16.00
GraphWaveNe [57
26.85
39.70
17.29
19.13
28.16
12.68
DeepGLO
52
25.45
35.90
12.20
15.12
25.22
13.20
AGCRN
17.78
29.17
11.79
14.59
23.06
9.29
StemGNNN
10
20.20
31.83
12.00
15.83
24.93
9.26
Z-GCNETs
20
18.05
29.08
11.79
14.52
23.00
9.28
ZFC-SHCN (ours) "*" 17.58
28..5
11.68
13.86
21.83
*.9.2
Table 3: Performance on COVID-19 hospitalizations in CA, PA, and TX.
CA
TX
Model
MAE
RMSE
MAPE (%)
MAE
RMSE
MАРE (%)
MAE
RMSE
MAPE (%)
FC-LSTM
14
167 077.86+333
502.29-14.16
90.71+-.117 47 2.36
74
69.37-22.8
20.75++0.5
71.66-22.15
89.16-48.6
SPM 60
103.984 1.46
475.66:t:123
66.91
45.91=2.42
106
12:187
70.15-5.0
17.49:0.50 58
70.18=1991
85.8548.25
N N-BEATS
66
10% 99 1.51
476. 1.9
67.55+t5116
46.91-282
108
86 1.95
15.82=44.0
21.25++0.56
69. 1..66-11.61
85.11+&777
DCRNN
192.10:22.96
69.83:5557
47.49_2.13
67.15
88.95-8.01
LSTNet
59
105.631 1.82
480.61:13.11
67.7242.117
46.01=2.70
1.76 72.33-14211
13.06.10.53
72.93.12.20
84.23-19.11
STGCN
30
102.88 +-1--11111 1.11
470.52++3.06
69.73+5569
52.69-2.40
106
78+ 1.87
10.36-44.59
12.69+0.49
63, 23.26-+-2.25
61.7-18377
13.70.10.40
1.89
DeepState
4K
100.57+:11 1.58
489 3577
68 24---44
48 46-2.00
107
61--2.31
67 69:44.211
13.24++046
59.47=11.9
62.38-
89.64-1:1 1.04 80
994.33323335
67.61
48.08=2,18
1.9
69.39:24.27
12.66:20.55
58. 1.22222222
60.33-88.77
DeepGLO
9..58.1..94
AGCRN
87.24:1:.77
118.2742.78
66. 66.30:5.11
2.69-2.49
103.79 79.7993.00
63.45=41.00
10.93:2.:4 49
52. 32.96:22.92
$9 $9.92228.99
StemGNN
10
82.36:1.2.33
3.9
67.90.1394
2.89
103,15.111
63.47-4.11
9.57.1..80
2.
59 80.18.18.81
7.GONKT's
156.35-+3.20
9.37-+0.50
ZFC-SHCN (ours) 79.33:1.70
3.46
61,25:5580 42.74=2.224 101.65:1.53 61.90:44.07 "99220=00399
27.81
55.34:18.60
for GCC for VARr, and VART, are 4.96 and 5.87, respectively; while the average EPC for VARí, and VARr, are 7.47 and 6.91, respectively. Interestingly (although it could be expected), higher GCC and lower EPC tend be associated with higher relative gains delivered by ZFC-SHCN. Finally, in Appendix ??, we present the sensitivity analysis for ZFC as a function of the covariance matrix in VAR models.
5.5 Ablation studies
To evaluate the performance of different components our ZFC-SHCN model, we per- form an expansive ablation study. The ablation study is conducted with three se- tups: (i) ZFC-SHCN without graph convolution in spatial dimension (W/o Graph convo- lution in spatial dimension), (ii) ZFC-SHCN without ZFC convolution (W/o ZFC convolu- tion), and (iii) ZFC-SHCN without supra-Hodge convolution (W/o Supra-Hodge convolution). The experimental results are shown in
Table 4: Ablation study PeMSD4
Table 4 and prove the validity of each component. As Table indicates, com- pared to ZFC-SHCN w/o ZFC convolu- tion, the zigzag homological feature vital for capturing the topological structure of spatio-temporal graph and our proposed graph convolution operation on ZFC signif- icantly improves forecasting performance. By comparing to ZFC-SHCN w/o supra-
and COVID-19 hospitalizations in TX.
Archilecture
RMSE
ZFC-SHCN
*** 10.15
PeMISD4
Wio Graph comvlution in spotial dimension
7FC comolution
28 83:10 28
Supra-llodge
28.69100.19
ZFC-SHCN
"27.81==1.72
1X-COVID--19
W/o 7PC comvolution
W/e Graph comvolution in spatial dimeasion
29. 1726::73
70.75 76:7761115
Supez-Hodge
30.11:::1777

---

Hodge convolution, we illustrate the significance of higher-order structure representation learning for guiding the model how to capture information on higher-order interactions. Also, ZFC-SHCN w/o graph convolution in spatial dimension demonstrates that the learned graph obtained from trainable weights can learn hidden information and enhance (multivariate) time-series representation learning.
5.6 Computational complexity
For higher-order simplices, the incidence matrices B1 and B₃ can be calculated efficiently with complexity O(N + M) and OMM + Q) respectively, where N is the number of O-simplices (i.e.. nodes), .M is the number of 1-simplices (i.e., edges), and Q is the number of 2-simplices (i.e., filled triangles). The computational complexity of ZFC is O(T)) 22). where T represents the number of points in time interval and ō E [2, 2.373). The computational complexity of the overall approach O(N2 + T + FouFo E dout/ + dout Σii- =k+1 + WGRU), including (i) graph
convolution in spatial dimension: ((N2), (ii) zigzag filtration curve: O (iii) supra-Hodge
convolution in temporal dimension:
+ + dlact Li=t =()) k+1) (where
FR is the number of k-simplex attribute features, w is the sliding window size, dout is the output dimension of the supra-Hodee convolution layer, and = is the number of (k++1)-simpp the L-th layer), and (iv) GRU: O(WGRU). We also compare our ZFC-SHCN with the most recent approach based on multipersistence- GNN [9] (i.e., TAMP- S2GCNets).. We find that ZFC-SHCN yields either on-par or more competitive performance than CAMP-S2GCNets, while proposed ZFC-SHCN significantly improves the computational efficiency (see Appendix ?? for more details). More details about running time comparison can be found in Appendix ??
E
6 Conclusion
We have proposed novel framework for time-aware deep learning of time-evolving objects which takes advantages of both the higher-order interactions among the data substructures. described as simplices, and the most intrinsic coneiconnitooned topological information exhibited by the object, characterized via zigzag persistent homology. By leveraging the power of simplicial convolution operation and zigzag persistence for time-indexed data, ZFC-SHCN has been shown to demonstrate capabilities to yield the most competitive forecasting performance, while requiring fewer computa- tional resources than its closest competitors. Still, computational complexity and limited theoretical results on statistical inference for zigzag persistence remain one of the major existing limitations of ZFC and, more generally, all topological methods for time dependent processes. In the future, we plan to investigate these theoretical and methodological challenges and will extend the ZFC-SHCN idea to anomaly detection in streaming time-dependent processes.
Acknowledgments
This work was partially supported by the National Science Foundation (NSF) under awards # ECCS 2039701 and ECCS-2039716, the Department of the Navy, Office of Naval Research (ONR) under ONR award # -00014-21-1-2538 C3.ai Digital Transformation Institute, and NASA AIST grant 21-AIST21_2-00S3
Part of this material is also based upon work supported by (while serving at) the NSF, Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF ONR, C3.ai DTI, NASA.
10

---

References
[1] S. Agarwal, K. Branson, and S. Belongie. Higher order learning with graphs. In Proceedings of the International Conference on Machine Learning, pages 17-24, 2006.
[2] Alman and V. V. Williams. A refined laser method and faster matrix multiplication. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 522-539. SIAM, 2021.
[3] L. Bai, L. Yao, C. Li, X. Wang, and C. Wang. Adaptive graph convolutional recurrent network for traffic forecasting. In Advances in Neural Information Processing Systems, volume 33, 2020.
[4] S. Bai, J. Z. Kolter, and V. Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling, arXiv preprint arXiv:1803.0122 2018.
[5] F. Battiston, G. Cencetti, I. lacopini, V. Latora, M. Lucas, A. Patania, J. Young, and G. Petri. Networks beyond pairwise interactions: structure and dynamics. Physics Reports, 874:1-92, 2020.
[6] A. R. Benson, R. Abebe, M. Schaub, A. Jadbabaie, and J. Kleinberg. Simplicial closure and higher-order link prediction. Proceedings of the National Academy of Sciences, 1 115:E1 1221-30, 2018.
[7] C. Bodnar, F. Frasca, N. Otter, Y. G. Wang, P. Lio, G. F. Montufar, and M. Bronstein. Weisfeiler and Lehman go cellular: Cw networks. In Advances in Neural Information Processing Systems, volume 34, pages 2625-2640, 2021.
[8] C. Bodnar, F. Frasca, Y. G. Wang, N. Otter, G. Montúfar, P. Lio, and M. Bronstein. Weisfeiler and Lehman go topological: Message passing simplicial networks. In Proceedings of the International Conference on Machine Learning, 2021.
[9] G. E. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung. Time Series Analysis: Forecasting and Control. John Wiley Sons, 2015.
[10] D. Cao, Y. Wang, J. Duan, C. Zhang, X. Zhu, C. Huang, Y. Tong, B. Xu, J. Bai, J. Tong, and
Q Zhang. Spectral temporal graph neural network for multivariate time-series forecasting. In Advances in Neural Information Processing Systems, volume 33, pages 17766-17778, 2020.
[11] G. Carlsson, V. De Silva, S. Kališnik, and D. Morozov. Parametrized homology via zigzag persistence. Algebraic Geometric Topology, 19(2):657-700, 2019.
[12] G. Carlsson, V. De Silva, and D. Morozov. Zigzag persistent homology and real-valued functions. In Proceedings of the Twenty-fifth Annual Symposium Computational Geometry, pages 247-256, 2009.
[13] M. Carrière and A. Blumberg. Multiparameter persistence image for topological machine learning. In Advances in Neural Information Processing Systems, volume 33, pages 22432- 22444, 2020.
[14] M. Carrière, F. Chazal, Y. Ike, T. Lacombe, M. Royer, and Y. Umeda Perslay: A neural network layer for persistence diagrams and new graph topological signatures. In Proceedings of the International Conference on Artificial Intelligence and Staristics, pages 2786 2766, 2020.
[15] C. Chatfield. The Analysis Time Series: An Introduction. Chapman and hall/CRC, 2003.
[16] C. Chen, K. Petty, A. Skabardonis, P. Varaiya, and Z. Jia. Freeway performance measurement
system: mining loop detector data. Transportation Research Record, 1748(1):961002, 2001.
[17] Y. Chen, B. Coskunuzer, and Y. R. Gel. Topological relational learning on graphs. In Advances in Neural Information Processing Systems, volume 34, pages 27029-27042, 2021.
[18] Y. Chen, Y. R. Gel, and H. V. Poor. BScNets: Block simplicial complex neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 6333-6341, 2022.
11

---

[19] Y. Chen, I. Dogoviaa-Dminguez,, B. Coskunuzer, and Y. R. Gel. TAMP-S2GCNeN coupling time-aware multipersistence knowledge representation with spatio-supra graph convolutional networks for time-series forecasting. In Proceedings of the International Conference on Learning Representations, 2022.
[20] Y. Chen, I. Segovia-Domingueze and Y. Gel. Z-GCNETs: Time zigzags at graph convolutional networks for time series forecasting. In Proceedings of the International Conference on Machine Learning, pages 1684-1694, 2021.
[21] A. K. Dey, Y. R. Gel, and V. Poor. What network motifs tell us about resilience and reliability of complex networks. Proceedings of the National Academy of Sciences, 116(39)113368-19977 2019.
[22] K. Dey and Hou, Fast computation of zigzag persistence. arXiv preprint 10X0v:2204.11080, 2022.
[23] X. Dong, D. Thanou, M. Rabbat, and P. Frossard. Learning graphs from data: A signal representation perspective. IEEE Signal Processing Magazine, 36(3):44 63, 2019.
[24] Ebli, M. Defferrard, and G. Spreemann. Simplicial neural networks. In Advances Neural Information Processing Systems, Workshop TDA and Beyond, 2020.
[25] H. Edelsbrunner, D. Letscher, and A. Zomorodian. Topological persistence and simplification. In Proceedings 4/ Annual Symposium on Foundations of Computer Science, pages 454 463, 2000.
[26] A. Grover and J. Leskovec. node2vec: Scalable feature learning for networks. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 855-864- 2016.
[27] G. Gupta, S. Pequito, and P. Bogdan. Dealing with unknown unknowns: Identification and selection of minimal sensing for fractional dynamies with unknown inputs. In Proceedings of the Annual American Control Conference, pages 28 14-2820. IEEE, 2018.
[28] G. Gupta, X. Xiao, R. Balan, and P. Bogdan. Non-linear operator approximations for initial value problems. In Proceedings of the International Conference on Learning Representations, 2021.
[29] J. D. Hamilton. Time Series Analysis. Princeton University Press, 2020.
[30] C. Hofer, F. Graf, B. Rieck, M. Niethammer, and R. Kwitt. Graph filtration learning. In Proceedings of International Conference on Machine Learning, pages 4314-4323, 2020.
[31] C. Hofer, R. Kwitt, M. Niethammer, and A. Uhl. Deep learning with topological signatures. In Advances in Neural Information Processing Systems, volume 30, 2017.
[32] C. D. Hofer, R. Kwitt, and M. Niethammer. Learning representations of persistence barcodes. Journal of Machine Learning Research, 20(126):1455 2019.
[33] M. Horn, E. De Brouwer, M. Moor, Y. Moreau, Rieck, and K. Borgwardt. Topological graph neural networks. In Proceedings of International Conference on Learning Representations, 2021.
[34] L.. Johnson and Goldring. Discrete hodge theory on graphs: a tutorial. IEEE Computing in Science & Engineering, 15(5):2-55, 2012.
[35] M. Johnson and J. Jung, Instability of the betti sequence for persistent homology and a stabilized version of the betti sequence. arXiv preprint arXiv:2109.099 2021.
[36] A. Kapoor, Ben, I. Liu, B. Perozzi, M. Barnes, M. Blais, and S. O'Banion. Examining covid-
19 forecasting using spatio-temporal graph neural networks. arXiv preprint arXiv:2007.03113. 2020.
[37] W. Kim and F. Memoli. Stable signatures for dynamic graphs and dynamic metric spaces via zigzag persistence. arXiv preprint arXiv: 2017.
12

---

[38] G. Lai, W. C. Chang. Y. Yang, and H. Liu. Modeling long-and and short-term temporal patterns with deep neural networks. In Proceedings of the International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 95-104, 2018.
[39] J. Li, Z. Han, H. Cheng, J. Su, P. Wang. J. Zhang, and L. Pan. Predicting path failure in time- evolving graphs. In Pruceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1279-1289, 2019.
[40] J. Li, F. Yang. M. Tomizuka, and C. Choi. Evolvegraph: Multi-agent trajectory prediction with dynamic relational reasoning. Advances in Neural Information Processing Systems, 33:19783- 19794, 2020.
[41] Y. Li, U. Islambekov, C. Akcora, E. Smirnova, Y. R. Gel, and M. Kantarcioglu. Dissecting ethereum blockchain analytics: What we learn from topology and geometry of the ethereum graph? In Proceedings of the 2020 SIAM International Conference on Data Mining, pages 523-531, 2020.
[42] Y. Li, R. Yu, C. Shahabi, and Y. Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In Proceedings of the International Conference on Learning Representations, 2018.
[43] I. H. Lim. Hodge laplacians on graphs. SIAM Review, 2020.
[44] L. O'Bray, B. Rieck, and K. Borgwardt. Filtration curves for graph representation. In Pro- ceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1267-1275, 2021.
[45] D. Ofori-Boateng Y. R. Gel, and I. Cribben. Nonparametric anomaly detection on time series of graphs. Journal of Computational and Graphical Statistics, 30(3):556777, 2021.
[46] B. N. Oreshkin, D. Carpov, N. Chapados, and Y. Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. In Proceedings of the International Conference on Learning Representations, 2019.
[47] B. Perozzi, R. Al-Rfou, and S. Skiena. Deepwalk: Online learning of social representations, In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 701-710, -70, 2014.
[48] S. S. Rangapuram, M. W. Seeger, J. Gasthaus, L. Stella, Y. Wang, and T. Januschowski. Deep state space models for time series forecasting. In Advances in Neural Information Processing Systems, volume 31, pages 7785-7794, 2018.
[49] T. M. Roddenberry, N. Glaze, and S. Segarra. Principled simplicial neural networks for trajectory prediction. In Proceedings of the International Conference on Machine Learning, pages 9029, 2021.
[50] M. T. Schaub, A. R. Benson, P. Horn, G. Lippner, and A. Jadbabaie. Random walks simplicial complexes and the normalized hodge I-laplacian. SIAM Review, 62(2):353-33 -311 2020.
[51] I. Segovia Dominguez, H. Lee, Y. Chen, M. Garay, K. M. Gorski, and Y. R. Gel. Does air quality really impact 1ovid-19 clinical severity: Coupling nasa satellite datasets with geometric deep learning. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3540-3548, 2021.
[52] R. Sen, H. F. Yu, and S. Dhillon. Think globally, act locally: deep neural network approach to high-dimensional time series forecasting. In Advances in Neural Information Processing Systems, volume 32, 2019.
[53] K Shamsi, F. Victoo, M. Kantarcioglu, Y. R. Gel, and C. G. Akcora. Chartalist: Labeled graph datasets for UTXO and account-based blockchains. In Advances in Neural Information Processing Systems.
[54] I. Sutskever, O. Vinyals, and Q. Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, volume 27, pages 3104-3112, 2014,
13

---

[55] S. Tymochko, E. Munch, and F. Khasawneh. Bifurcation analysis using zigzag persistence. In Advances in Neural Information Processing Systems, Workshop on Topological Data Analysis and Beyond, 2020.
[56] L. Wang.
X. Ben, A. Adiga, A. Sadilek, A. Tendulkar, S. Venkatramanan, A. Vullikanti, G. Aggarwal, A. Talekar, J. Chen, et al. Using mobility data to understand and forecast covid19 dynamics. medRxiv, 2020.
[57] Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang. Graph wavenet for deep spatial-temporal graph modeling. In Proceedings of the International Joint Conference on Artificial Intelligence, 2019.
[58] Z. Yan, T. Ma, L. Gao, Z. Tang, and C. Chen. Link prediction with persistent homology: An interactive view. In Proceedings of International Conference on Machine Learning, pages 1 11659-1 11699, 2021.
[59] B. Yu, H. Yin, and 7. Zhu. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 3634-3640, 2018.
[60] L. Zhang, C. Aggarwal, and G. J. Qi. Stock price prediction via discovering multi-frequency trading patterns. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 2141-21499 2017.
[61] Q. Zhao, Z. Ye, C. Chen, and Y. Wang. Persistence enhanced graph neural network. In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 2896-2906, 2020.
[62] A. Zomorodian and G. Carlsson. Computing persistent homology. Discrete & Computational Geomelry, 33(2):249-277 274, 2005.
14


Reviews:
[{"review_id": "aAY1wB_A0Cc", "reviewer": "Reviewer_m8mW", "summary": "The paper proposes a new time-aware GNN networks called Zigzag Filtration Curve based Supra-Hodge Convolution Networks(ZFC-SHCN). Specifically, the networks composes 1) the simplicial neural networks with an efficient zigzag persistence curve and 2) a new temporal multiplex graph representation module. In practice, the experimental results on several synthetic and real world datasets demonstrate the effectiveness and efficiency of the proposed ZFC-SHCN.", "questions": "Same as weaknesses.", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n+ The paper is well-written and well-organized.\n+ The idea, amplifying the power of SNNs with a time-conditioned topological knowledge representation in a form of zigzag persistence, is interesting and novel. \n+ The experimental results show ZFC-SHCN outperforms other baselines.\n+ Codes are provided.\n\nWeaknesses\n\n+ It worth noting that the gains from ZFC convolution and Supra-Hodge convolution in Tabel 4 are not significant as expected, It would be fairer if the authors could provide more comparison experiments. E.g.  Replacing ZFC with zigzag persistence image\n(ZPI) to show the gain from ZFC, replacing Supra-Hodge convolution with $GCN_{temp}$ to show the gain from Supra-Hodge convolution, etc. Or theoretical analysis of the differences is given.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "KWR4_D9ZH83", "reviewer": "Reviewer_u6Ln", "summary": "This paper addresses the problem of  capturing information other than pairwise relations in GNN. A time-conditioned topological knowledge representation is proposed, and related theoretical properties are presented. The topic is very interesting and influential.", "questions": "The complexity should be discussed and some other type of tasks should be done.", "limitations": "No suggestions.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- good idea.\n- there is theoretical stability guarantees.\n\nWeaknesses\n- the complexity should be discussed.\n- it is not very convincing that the expression is enough for various of tasks.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ESZ0wZVJNSM", "reviewer": "Reviewer_isoH", "summary": "In this paper, the authors propose a method for predicting time series of graphs with the addition of the concept of TDA. There is no discussion of graph construction methods here, but rather the general setting of the existence of time-varying graphs. The proposed method is basically a conventional graph convolution -recurrent neural network with an additional mechanism to consider topological information. The zigzag filtration curve and supra-Hodge convolution are used as topological information. The authors also prove the stability of ZFC and demonstrate its effectiveness in comparison to conventional methods in their experiments.", "questions": "1. The proposed method introduces zigzag filtration features and supra-Hodge convolution features in addition to the conventional graph convolution features. Although the overall framework has been evaluated, it is not clear which features have what effect on each of them. Since it is not possible to determine whether all of the elements are necessary, I wonder if a comparison with a system that excludes any of the information would make the observation possible. Are such comparisons being made?\n\n2. Features using Zigzag filtration Creation is a natural concept, but I am sure there are other possibilities. Are there any comparisons with other methods? Of course, since the effectiveness of this system as a whole has been demonstrated, it is only a proposal as a first step, and if improvements using other TDA methods is a future issue, that is fine.\n\n3. The experimental results, especially in Table 3, show that the proposed method seems to create different models for different evaluation indicators for a single problem. The same model should be used to compare different indicators in the evaluation. If you want to show that each indicator can be set appropriately, you should compare them separately.\n", "limitations": "The authors have clearly defined the application, clearly described the performance, and adequately addressed the limitation of their work.\n", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength\n-  The authors provide a framework for time series prediction of graphs that includes topological information that has not been previously taken into account.\n-  Introduced zigzag filtration as topological information\n- The authors have organized the mathematical background of the algorithm before introducing the algorithm.\n- The authors have tested the effectiveness of this method compared to many conventional methods from multiple perspectives on multiple data sets\n\nWeakness\n- Although the authors have compared the proposed method with conventional methods that do not include topology information, they have not examined whether the proposed method is appropriate as a method that takes topology information into account.\n- There are doubts about some of the experimental setups.\n\nOverall, the prediction of graph time series is a very important issue, including from a practical viewpoint, and it is worthwhile to have provided an effective framework for this problem with the addition of TDA-based information. Since I have some doubts, I would like to make a final decision based on the answers to the following questions.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "84Hmxkf1urQ", "reviewer": "Reviewer_vJan", "summary": "The authors propose the Zigzag Filtration Curve based Supra-Hodge Convolution Networks (ZFC-SHCN) which is able to learn time-aware persistent topological features and simplicial geometry of graphs. The theoretical stability of the zigzag filtration curve is studied in the paper. Experiment results show that ZFC-SHCN achieves the best performance of forecasting with less computational requirements.\n", "questions": "Please see weakness.", "limitations": " Limitations and potential negative societal impact have been addressed.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed novel framework connects the dynamical behavior of complex systems and persistence homology analysis.\n- The authors claim that this paper is the first work that brings the concepts of simplicial convolution to time-aware learning.\n- The experiments on varying datasets including covid case and traffic flow show the superiority of the proposed method in terms of the prediction tasks. ZFC-SHCN is also faster compared with runner-ups.\n\nWeakness:\n- I recommend adding some references in the introduction section to support the statements. \n- I suggest the authors to check these related paper of time series prediction:\nhttps://ieeexplore.ieee.org/abstract/document/8430866\nhttps://openreview.net/forum?id=d2TT6gK9qZn\n- The writing quality of this paper needs to be improved, please see below some questions that need to be clarified in the paper main body.\n- The contribution is clear as stated in the introduction section. However, the connection among/inside section 3 and 4 (and the subsections) is not very clear.\n- Are the tasks to predict the further covid cases, traffic flow etc. as both time series and node attribute of graph?\n- How are the graphs or the simplicial complexes constructed on each dataset? In the appendix it says the graph represents the border connection, but it is unclear about other datasets.\n- How is the simplex constructed in the experiment section? Does it only consider nodes and edges, or also involve actual higher-order information like a 2-simplex, 3-simplex or more? If not, where does the higher-order information appear in the graph structure?\n- Is the embedding dimension mentioned in line 281 the same as the dimension of feature of each node (d_e) or something else? Is it predefined or computed following any rules? The number here looks rather small, any reason for such a choice?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept