TAMP-S2GCNets: Coupling Time-Aware Multipersistence Knowledge Representation with Spatio-Supra Graph Convolutional Networks for Time-Series Forecasting
Authors: Yuzhou Chen
Ignacio Segovia-Dominguez
Baris Coskunuzer
Yulia Gel
Venue: ICLR 2022 Spotlight (2022)
Keywords: topological data analysis
multipersistence
graph convolutional networks
supragraph diffusion
multivariate time series forecasting

Abstract: Graph Neural Networks (GNNs) are proven to be a powerful machinery for learning complex dependencies in multivariate spatio-temporal processes. However, most existing GNNs have inherently static architectures, and as a result, do not explicitly account for time dependencies of the encoded knowledge and are limited in their ability to simultaneously infer latent time-conditioned relations among entities. We postulate that such hidden time-conditioned properties may be captured by the tools of multipersistence, i.e, a emerging machinery in topological data analysis which allows us to quantify dynamics of the data shape along multiple geometric dimensions. 
 We make the first step toward integrating the two rising research directions, that is, time-aware deep learning and multipersistence, and propose a new model, Time-Aware Multipersistence Spatio-Supra Graph Convolutional Network (TAMP-S2GCNets). We summarize inherent time-conditioned topological properties of the data as time-aware multipersistence Euler-Poincar\'e surface and prove its stability. We then construct a supragraph convolution module which simultaneously accounts for the extracted intra- and inter- spatio-temporal dependencies in the data. Our extensive experiments on highway traffic flow, Ethereum token prices, and COVID-19 hospitalizations demonstrate that TAMP-S2GCNets outperforms the state-of-the-art tools in multivariate time series forecasting tasks.

Published as a conference paper at ICLR 2022
TAMP-S2GCNETS: COUPLING TIME-AWARE MUL- TIPERSISTENCE KNOWLEDGE REPRESENTATION WITH SPATIO-SUPRA GRAPH CONVOLUTIONAL NETWORKS FOR TIME-SERIES FORECASTING
Yuzhou 1.3 Ignacio Segovia- Don-inuumnn 2.4 Baris Coskunuzer Yulia R. Gel2.5
1 Depaartment of Electrical and Computer Engineering, Princeton University
*Department of Mathematical Sciences, University of Texas at Dallas Lawrence Berkeley National Laboratory
'Jet Propulsion Laboratory, Caltech " National Science Foundation 000774@princet.o on on edu [Ignacio. Sew.llo.eeegviiaamingggee coskunuz, yglJeutdallas.ed edu
ABSTRACT
Graph Neural Networks (GNNs) are proven to be powerful machinery for learn- ing complex dependencies in multivariate spatio-temporal processes. However. most existing GNNs have inherently static architectures, and as a result, do not ex- plicitly account for time dependencies of the encoded knowledge and are limited in their ability to simultaneously infer latent time-conde conditiond relations among en- tities. We postulate that such hidden cone-conditioned properties may be captured by the tools of multipersistence, i.e., a emerging machinery in topological data analysis which allows to quantify dynamics of the data shape along multiple geometric dimensions. We make the first step toward integrating the two rising research directions, that is, time-aware deep learning and multipersistence, and propose a new model, Time-Aware Aware Multipersistence Spatio-Supi Graph Convo- lutional Network (.MMP--2GGNNtss., We summarize inherent conditond doincc-P
topological properties of the data as time-aware multipersistence surface and prove its stability. We then construct a supragraph convolution mod- ule which simultaneously accounts for the extracted intra- and inter-dependencies in the data. Our extensive experiments prices, and COVID-19 hospitalizations demonstrate that HAMP-$2GCNer out- performs the state-of-the of-the--rt tools in multivariate time series forecasting tasks.
highway traffic flow, Ethereum token
INTRODUCTION
Multivariate time series forecasting plays an integral role in virtually every aspect of societal func- tioning, from biosurveillance to financial analytics intelligent transportation solutions. In the last few years, Graph Convolutional Networks (GCNs) have emerged as a powerful alternative to more conventional time series predictive models. Despite their proven success, GCNs tend to be limited in their ability to simultancously infer latent temporal relations among entities (such as correlations both within time series and in-between time series or the joint spatio-temporas dependencies). More generally, most existing GCNs architectures are inherently static and as such, do not explic- itly account for time-conditioned properties of the encoded knowledge about the complex dynamic phenomena.
At the same time, recent studies on integrating shape properties of the complex data into deep learn- ing (DL) models indicate that topological representations, obtained using the tools of single param- eter persistence, can bring an invaluable insight into the system organization and enhance the result- ing graph learning mechanisms (Hofer et al., 2019; Carrière et al., 2020; Carlsson & Gabrielsson, 2020; Horn et al., 2021). (By shape here we broadly understand data properties which are invariant under continuous deformations, e.g., stretching, bending, and twisting). However, in many appli-

---

Published as a conference paper at ICLR 2022
cations, particularly, involving spatio-temporal processes, the data exhibit richer structures which cannot well encoded with a single parameter persistence. We postulate that many critical hid- den time-conditioned interrelations which are inaccessible with other methods can be captured by the emerging machinery of multipersistence. Multipersistence, or multiparameter persistence (MP) generalizes the notion of single parameter persistence to case when dynamics of the inherent data shape is discerned along multiple geometric dimensions (Carlsson & Zomorodian, 2009). Despite its premise, applications of MP in any discipline remain nascent at best (Riess & Hansen, 2020; Kerber, 2020).
We make the first step on path of bridging the two emerging directions, namely, time-aware DL with time-conditioned MP representations of complex dynamic phenomena. By time-conditioned MP representations, we mean the most salient topological properties the data that manifest them selves over time. To summarize such time-conditione topological properties, we first introduce dynamic Muler-Poincare surface as a new MP summary. We then propose a directed multilayer supra graph abstraction represent sequence of time- varying objects and develop supragraph convolu- tion module which allows us to simultancously learn co-evolving intra- and interddepeneencies (i.c., spatial and temporal correlations) in the complex nigh-dimensional data.
The key novelty of this paper are summarized as follows:
This is the first work to bridge the concepts of MP with the time-aware learning paradigm. Appli- cations of MP in any field of study are currently nascent.
We introduce new time-aware multipersistence invariant. dynamic Euler-Poincaré surface. We prove its stability and show its substantial computational gains and high utility for encoding the time-conditioned knowledge.
We propose a mathematical abstraction of directed multilayer supra graph for time-conditioned knowledge represenntaton and construct a new Time-Aware Multipersistence Spatio-Supra Graph Convolutional Network (TAMP-S2GCNets)) which simultaneously learns latent temporal inter- and intra-relations among entities in the complex high-dimensional data.
We perform expansive forecasting experiments. in application to highway traffic flow, Ethereum token prices, and COVID-19 hospitalizations. Our findings demonstrate superior predictive per- formance, versatility and computational efficiency of TAMP-S2GCNets, compared to the state- of-thc-art methods in multivariate time series forecasting.
2 RELATED WORKS
Multipersistence Despite that MP demonstrates very promising results in terms of improving ac- curacy, tractability and robustness, applications of MP in ML are virtually non-existent (Wright & Zheng, 2020; Riess& & Hansen, 2020: Kerber, 2020). Some notable efforts in the direction to develop MP summaries which are suitable for integration with ML models include Multiparameter Persis- tence Kernel of Corbet et al. (2019), Multiparameter Persistence Landscapes (MP-L) of Vipond (2020), and Multiparameter Persistence Images (MP-I) of Carrière & Blumberg (2020) which are based on the concept of slicing, that is, restricting the MP module to an affine line (or single parame- ter persistence) (Cerri et al., 2013; Landi, 2014). Such slicing methods enjoy number of important stability guarantees but tend to be computationally expensive even in static scenarios, which makes them infeasible for time series forecasting tasks. In turn, the most recent results of Beltramo et al. (2021) (i.c., Euler Characteristic Surfaces) and Coskunuzer et al. (2021) 1) (i.e.. Multiparameter Persis- tence Grids) introduce pointwise MP representations, in application to static point clouds and graphs Such pointwise representations are weaker invariants but are substantially more computationally ef- ficient. Integration of pointwise representation with ML models has not been yet investigated. Here we propose the first time-aware pointwise MP representation, a dynamic Euler- Poincaré surface, derive its theoretical properties and integrate it with GCN in time series forecasting tasks.
Spatial-Temporal Graph Models and Forecasting Recent studies (Li et al., 2018: Yu al., 2018; Yao et al., 2018) introduce graph convolution methods into -patio-temporal networks for multivari- ate time series forecasting which, as result, allows for better modeling of dependencies among entities (Wu et al., 2019; Bai et al., 2020; Cao et al., 2020) and handling data heterogeneity. Despite the GCN successes, designs of the existing spatial-temporal GCNs largely rely on the pre-defined graph structures. As such, GCNs are restricted in their ability to explicitly integrate time dimen- sion into the knowledge representation and learning mechanisms, thereby, limiting model adaptivity
2

---

Published as a conference paper at ICLR 2022
to the dynamic environments and requiring more frequent retraining. Most recently, Chen et al. (2021) propose a time-aware GCN, Z-GCNETs, for time series forecasting which integrates zigzag persistence images based on a single filtration, as the primary time-conditioned topological repre- sentation. In general, the zigzag concept can be combined with MP, but requires more fundamental advances in the theory of algebraic topology. As such, our time-aware MP learning approach may be viewed as complementary to zigzag persistence, while considering time-changing connections of graph structures in dynamic networks.
TIME-AWARE MULTIPERSISTENCE EULER CHARACTERISTIC SURFACES
Spatio-Temport Graph Construction We define spatial network at time step t as Gi (V,, A,, Xi), where V4 is a set of nodes and & is a set of edges. We let [iel = N and [Eg] = Mr. E R.NxFw is the node
The adjacency matrix At RNN,,, and X, feature matrix with feature dimension FN. To construct the spatial network G. we can build the adjacency matrix A, based on (i) the prior knowledge of graph structure: first-order neighhours, i.e., AL.uv = 1 if the node u and node v have a connection in the dynamic graph at the time step i; and (ii) the Radial Basis Function (RBF): degrees of similarity between instances (i.e., nodes) in X,, i... Ar. 1exp (-Ila,,, - (ll/) where y denotes the length scale parameter and denotes the threshold parameter filters noisy edges. Let I" be the total number of time steps. Given sequence of observations on a multivariate variable, X' = 1 X XT} E RNx XXxxxxxx xT with T timestamps and Fx node attributes, we construct spatio-temporal networks g = {Gi, 92...., Gr} via either prior knowledge of network structure or applying RBF to the node feature matrix.
Single Parameter Persistence Persistent homology (PH) based on one parameter discerns shape of the complex data along a single geometric dimension. The goal is to select some suitable pa- rameter of interest and then to study graph G. not as a single object, but as a sequence of nested subgraphs. or graph filtration G S G ... S 9.Th G. induced by this evolving scale pa- rameter. Armed with such filtration, we can then assess which structural patterns (e.g.. loops and cavities) appear/disappear and record their lifespans. To make the counting process more efficient and systematic, we build a simplicial complex C; from each subgraph GJ. resulting in a filtration C S C ... S C. (e.g.. clique complexes). For example, to construct such filtration, common method is to consider a filtering function f V4 + R and the corresponding increasing set of thresh- olds {a,) Im such that C; = {á E C, max f(v) S a.). The resulting construction is called sublevel set filtration of f. and f can be selected, for instance, as degree, centrality, or eccentricity function (Hofer et al., 2020; Cai & Wang, 2020). Similarly, I can be defined on the set of edges &.. More details on a single filtration PH can be found in Appendix B.
Time-Aware Multiparameter Persistence (TAMP) Data in many applications, particularly, in- volving spatio-temporal modeling, might be naturally indexed by multiple parameters, e.g., real time traffic flow and optimal route in urban transportation analytics. Alternatively, the primary fo- cus might be on discerning shape properties of the complex data along multiple dimensions. For instance, to better predict cryptocurrency prices and manage cryptomarket investment performance, we may need to evaluate structural patterns in cryptoasset dynamics not along one dimension but simultaneously along the volume of transactions and transaction graph betweenness, as a measure of joint perception of the cryptomarket volatility among the key investors. Such multidimensional analysis of topological and geometric properties can be addressed using generalization of PH based on single filtration to multifiltration case. That is, the MP idea is to simultancously assess shape characteristics of Ge based on a multivariate filtering function F : Vi + Rd. As a result, e.g., for d 2 and set of nondecreasing thresholds {arr 1 and {,,)1 instead of single filtration of complexes, we get a bifiltration of complexes {C | < S m, 1 < j S n} such that if + C Finally, this bifiltra-
3x B/. then Ca ④ C and if a; < ag, then Cat,, tion induces a bigraded MP module { {H₂ Ca...f))) where Hş is the 4 homology group. Inspired by Beltramo et al. (2021); Coskunuzer et al. (2021), we propose a new time-aware MP summary, namely, a Dynamic Euler-Poincaré Surface.
Definition 3.1 (Dynamic Poinc--Pinccré Surface), Let (g. [9. be a series of time-varying graphs. Let F (F.g) be multivariate filtering function F V. F IR² with thresholds I = {(04, (3) 1 < i < m, < j < n]. Let C....; be the clique complex of the induced subgraph g. =
F-1 ((-00, a.] x (-oc, B,D), 2.2....77 T and x be the Poincar characteristic. Then,
3

---

Published as a conference paper at ICLR 2022
sequence of time-evolving x n-matrices {E')T= such that E; x (CC...)) for S S m, 1 j I n is called Dynamic Fuler-Poinceen Surface (DEPS). (Figure 2 in Appendix C shows toy example how DEPS is computed.)
Theoretical Guarantees of DEPS Consider two graphs G and G where time index t is sup- pressed for brevity. Let F : V 4 R² be a multivariate filtering function with thresholds I {as, Bz) 1 S S m, n). Let C# be the clique complexes of G and let
Č (C the bifiltration induced by (C#, F,T) as before. Let E= be the corresponding Euler-Poincaré Surfaces (i.e., m matrices). Then, set | - -B 111.1 = E Ei=1 [ Rgg
as the distance between E+ and E-, where || 1| is the vectorized I, matrix norm.
We now introduce an based MP metric instead of using -bsed metries like, e.g., matching or interleaving, due to the nature of our summaries E= (see Remark C. in Appendix Let D (C+) and (C=) be the wtth single parameter persistence diagrams (PDs) of C= for filtrations induced by functions f,g y² + R, respectively (see Appendix B). Let C and City be clique complexes corresponding to 9/20 f-'((( 00, a.)) and 9 g ((-o0_ 92)). Define the 7th column dis- (I (C,, Di )), where Wi is the Wasserstein-1
tance for the hthh PDs as D distance. Similarly, the j row distance for 1thh PDs is Dt (C+, C-) = Wi,
Wi (D/ (Ct,). (CT))) Definition 3.2 (Weak -metric for Multipersistence). The weak -m-meriic between C+
(O(CC, D Ō-) = max D Cc+, (0),, }
s.t. OCt - D. (c+ , and D, (c+ Ö - Ej= D OOt Õ.).
Now, we can state our stability result for Theorem 3.1. Let G= F, PT E+ be as defined above. Then, the Fuler-Poincaré Surfaces are stable with respect to the weak Li-metric, i.e., IE+ E 11,,1 S C CCC+ C-) some C > 0.
Surfaces.
The proof of the theorem is given in Appendix C. This stability result implies that the distances between multiparameter PDs control the distance between the resulting Euler-Poincaré Surfaces. By combining with the stability result for PDs (Cohen-Steiner et al., 2007), one can conclude that the small changes in the MP filtering function F : V. H Rd or the small changes in the input data can result only in a small change in DEPS surfaces. For further discussion on implications of the stability result, please refer to Remark C.2.
4 TIME-AWARE MULTIPERSISTENCE SPATIO-SUPRA GRAPH CONVOLUTIONAL NETWORKS
Given T historical observations XT= ing model 3(-) is learned to predict future observations in the next H + 1 time steps, i.e., Xx-x] 3(X,-T, X X,-1).
X the multivariate forecast-
(X,XX+1.....
Graph Learning Architecture The graph representation learning of our MPS2GCNets is build upon GCN. To learn node representation from graph topology and node features, the input of GCN- based approaches contains the adjacency matrix A, of the original input graph G. and the node feature matrix X,. However, the prior knowledge of graph structure (1) might restrict the modeling capacity (i.e., graph edges cannot encode the complex relationships between nodes) and (2) leads to the neglect of neighboring information with high diversity. To avoid these limitations, inspired by the adaptive dependency matrix (Wu et al., 2019; Chen et al., 2021), we learn the normalized self- adaptive adjacency matrix S {swe) NxN with the pre-defined "cost" staying in the same node based on the learnable node embedding E.- = C2, eN₂O) E RR Nxd, as
exp(ReLU(ev.deT...))
1 7 U,
Σ Everytuy PxppRRll.Uleee.. exp(Rol (.g))) trop (dow)'
Suv
duy Ever((u) exp(ReLU(es, ))xxpp (dou) 8 C.
4

---

Published as a conference paper at ICLR 2022
Here hyperparameter duu is the "cost" of staying in the same node and ReLU(.) max .) an activation function, which guarantees 8uv 2 Since En, Sup = and 8ue we can use S as the normalized Laplacian. Towards more effective and robust learning of both spatial and spectral characteristics, we represent the graph diffusion as matrix power series. That is, let S be an N x N x K Laplacian tensor containing the power series
SK-1) of S, where
IR N is the identity matrix and K >
Feature Transformation Note that spatial and temporal domains contain inter- dependent but highly heterogeneous types of information. Inspired by statistical factor analysis, facilitate signal extraction from these disparate informational sources, by mapping the original input feature space into the high-level latent feature space, which can be written as follows
H
((+1) L. (X'
E FT TU,
(C) FT E EdexPxXFF and U E R represent the learnable parameters.
where O
4.1 SUPRAGRAPHI CONVOLUTIONAL MODULE IN MULTILAYER SUPRA GRAPH
We propose a novel supragraph convolutional module simultaneously capture spatio- dependencies in dynamic networks. Our key approach is (i) to represent a sequence of time-varying graphs recorded over sliding time window, as a multilayer supra graph, instead of treating each graph snapshot individually, and (ii) to learn the resulting multilayer supra graph with the random walk exploration which allows to encode the key details of the conditiodd relationships among nodes and to boost graph convolutions over multiple edge sets.
Sliding Window Historical Data as Multilayer Supra Graph Network To increase expres- sive capability of spatio-temporal representation learning, we propose a novel directed mul- tilayer supra graph. Particularly, we treat each graph within a sliding window g E RNNN as layer within a directed T-llayerd network. (Here
G--1)= (G--T.--7++1.-.. N is the number of nodes in each graph.) Since information cesses can propagate only from past to present to future, we consider directed multilayer supra graph as abstraction for dynamic knowledge representation. That is, we assume that the informa- tion (e.g.. spatial features) is shared between layers ta and tg, whenever to > ta, and time stamps to (t T.....t t 1}. More specifically, we propose strategy to create such kind of infor- mation propagation channels by adding directed virtual edges (i.e., interlayer edges) between every node in the layer la and its counterparts in other "future" layers 4, (l,- l.).
real world spatio-temporal pro-
Definition 4.1. A directed multilayer supra graph (DMSG) is a tuple, defined as: DMSG (G.-T....55 G.- IM where G._ = &,,,,, la E (I-T...... 1- 1} are network layers and 1o (Identity Mapping) is an x N matrix of node mappings, with IM vf x of + [0, 1]. That is, f is the identity mapping between node in the "past" layer ta and the "future" layer to (ts > ta): IM = 1.
The corresponding (r-th power) supra-adjacency matrix W (woo) NT for DMSG then
(staf")",
tu t, and Iu 2l mod N + 0, ta to and uu vl mod N 0, la + ln and Iu - vl mod N == -0,
ab way
(diata)
Huls
where (shala)" y is the r-th power of State which encodes the r-th step of random walk among nodes u and v in layer las (d'afa )" is the "cost" of staying in the same node u and in the same layer to after random walk steps, and diats is the "cost" of jumping from the current node in layer to to node u in layer 4s Finally, the generalized (r-th power) supra-Laplacian L R NTN for DMSG then
(SS 0
D
D
L Sup
(3)
D
Supragraph Diffusion Convolutional Layer Armed with the (r-th power) supra-Laplacian in (3), now build supragraph diffusion convolutional layer to encode both the graph structure and node
5

---

Published as a conference paper at ICLR 2022
features from the temporal domain. That is, embedded node features are updated by message passing and aggregation via intralayer and interlayer diffusion. The supragraph diffusion convolutional layer is formulated as
H (1++)) Lup
Hmpp) (Csup H Sup)
E
(4)
Sup
where H 4,pup E ERTX and H uppup E R NTNNTXQO are the input and output activations for layer e H H S.Sup X", and O (8)
Sup E RdxxxQss are the learnable parameters. In this case, the diffusion on layer ta can extend to a fraction of nodes and propagate information of layer t. by interaction.
4.2 SPATIAL GRAPH CONVOLUTIONAL LAYER
To aggregate the features of each node with its multi-hops neighbourhoods to generate node embed- dings, we now turn to constructing the spatial graph convolution via
H
(E+1) (SH sppp) ) "EE + S,Spa
(E) Spat
(5)
where O RRdxxXxPx9pp is the trainable weight, $ is the Laplacian tensor, H RNX
Spa
and H (t-1)
1.Spa E R NxQu are the node representations at the f-th layer and (E + 1)-th layer, respec-
S. X,, i... the node features time step To reduce computational costs, we use weight sharing matrix factorization instead of matrix entry assignments. As such, we timely update the latest state of variables with back propagation algorithm and reduce the risk of over-fitting,
tively, and H (0)
4.3 LEARNING TIME-AWARE MULTIPERSISTENCE REPRESENTATION OF TOPOLOGICAL FEATURES
To fully utilize information delivered by the MP representation mechanism, we propose a CNN- inspired Time-Aware Multipersistence Fuler-Poincaré Surface Representation Learning (DEPSRL) module which employs the CNN base model fo (including convolutional and pooling layers) and readout layer. In summary, the proposed DEPSRL module offers the following unique innova- tions: (i) extracting features from the global information, (ii) learning the relationship from DEPS sequences pre and post (i.e., treating the fixed-size sliding window as multi-channels), and (iii) ag- gregating topological features to make a fixed size representation. The summarized output feature of DEPSRL module is given by
(f+1) ITAMP = @(fGAP(Fe, (( (f,
(6)
where faxp(.), faMp()), and So,, = {1,2), are global average pooling, global max pooling, and j-th CNN based model, respectively, and @ denotes concatenation, where the fGAP(-) generates summarized feature for each channel. We find that faMP(-) strengthens the representation learning of the time-conditioned multipersistence features. The matrix & AMMP E R is the output of the DEPSRL. module. To further learn and fuse multiple latent representations form different views (i.e., spatial information, spatio-temporal correlations, and persistent topological features), we combine these embeddings to obtain the final embedding
(f+1)
(C-1)
3 (H (f+1) S.Spa
((+1) H (((+)),(((L))) ((HH))
Here Y (8+1) = (1/TE A
(1-1) $ (++1)
k,p STAMP integrates global topological and global spatio-temporal (++1) NxT and then average 1,Sup
information (i.e., we reshape H i.Sup NRNTTQu to H
(1+1)
(r((tt)) over temporal dimension), $ (., is dimension-winn wise concatenation function embed- dings along the output dimension, and (ie(Sp..up. Q: Qout- DEPSRL also allows for input of E from different bifiltrations. For more details, please refer to Appendix D.2.
4.4 MODELING SPATIO-TEMPORIA TEMPORAL DYNAMICS
To encode spatio-temporal correlations among time series and get hidden state of nodes at a future timestamp, we put the final embedding Z jout into Gated Recurrent Units (GRU). The forward

---

Published as a conference paper at ICLR 2022
propagation equations of GRU are follows
R. = (W [ out) + bR),
3: = < (Wo [1- Zz + b3), E. R - (1 R2 E,,
(8)
E, tanh ( [ 1, ood) bg):
where s(-) is activation function which is ReLU in our case, + is the elementwise product, R2 is the update gate and S; is the reset gate. bg, b3. bg. WR. Ws. and W are learnable parameters. 1 out) and E, are the input and output of GRU model, respectively. Now get E, containing the structure, spatio-temporal, and topological information.
DEPSI
+++
CNN
CNN
Teemeriing
Cencatenation
Bot
Fanction
Sensor
Attrlbuttte
Prediction
Stay at the Same Nodo
Ne Naveet
01)
Intralayer Randon Walk
Interlayer Random Walk
Figure 1: extracts spatial information time (see Eq. 5). (II) Feature transformation (FT) learns representa- tion of the ppatio-temporal data X over sliding window of size T (see Eq. 1). (III) Supragraph convolutional module captures joint spatio-temporal dependencies X (see Eq. 4). (IV) Detailed architecture of the DEPSRL module (see Eqs. and 7), where the DEPS with different types of mul- tifiltrations be learned by CNN base model and global pooling mechanism. Here DEPS allows us to learn the intrinsic graph structure both across multiple geometric dimensions and across time. (V) The learned spatio-temporal dependencies are passed into GRU layer (see Eq. 8) for multi-step forecasting. Note that the whole model can be trained in an end-to-end fashion.
consists of 5 components: (I) Spatial graph convolutional layer on G
5 EXPERIMENTS
Data Description We use three different types of datasets to examine the performance of the pro- posed TAMP-S2GCNets on dynamic networks: The transportation and traffic data of California state from the freeway Performance Measurement System (PeMS). We use three well-studied traffic networks from literature: PeMSD3, PeMSD4 and PeMSD8 (Chen et al., 2001), (ii) Digital transac- tions between users in the Ethereum blockehain network. We extract dynamic networks from three token assets: Bytom, Decentraland and Golem (Li al., 2020), and (iii) The spread of coronavirus disease COVID-19 at county-level in states of California (CA) and Texas (TX). More details on each dataset and parameter settings are provided in Appendix D.I.
Experimental Settings We compare the presented TAMP-S2GCNets with 13 state-of-the-art base- lines (sec more details of baselines in Appendix D.2) on three evaluation metries, i.c, Mean Absolute Error (MAE), Root Mcan Squared Error (RMSE), and Mcan Absolute Percentage Error (MAPE). For transportation networks, following Cao et al. (2020), we (i) split PeMSD3, PeMSD4, and PeMSD8 datasets into a training set (60%), validation set (20%). and test set (20%) in chronologi- cal order and (ii) use the one hour historical data to the next 15 minutes data. Following Chen et al. (2021), we (i) split Bytom, Decentraland, and Golem token networks into training set (80%) and test set (20%) in chronological order and (ii) use 7 days historical data to predict future 7 days data. For COVID-19 19 datasets, our experiments, we use daily data of 11 months of 2020, from February to December 31, and split the graph signals into training set, first 80% of days (268 days), and test set, last 20% of days (67 days). Further specifies on experimental setup are provided in Appendix D.2. The best results are in bold font and the results with dolted underline are the best performance achieved by the runner-ups. We also perform one-sided two-sample 1-test between the best result
7

---

Published as a conference paper at ICLR 2022
and the best performance achieved by the runner-up, where ** *** denote significant, statisti- cally significant, highly statistically significant results, respectively. More detailed experiments on all datasets are in Appendix E. The data and code implementation are available at https: //wp:: dropbox. com/ sh/n0a jd51 Otdeyb80 CAABGG jVvi LOww ?d1=0.
Computational Complexity Although, in general, MP is computationally costly, in our case, to get the DEPS summary, we do not need compute PDs in other MP approaches, but only Betti numbers of each filtration cell. In particular, while a computational cost for obtaining h**h PD for graph is ((M)), where Mg the number k-simplices (Otter et al., 2017), obtaining Euler Characteristics by sparse matrix methods has computational complexity of O(Mo +M + M2) (Edelsbrunner & Parsa, 2014).
5.1 EXPERIMENTAL RESULTS
Transportation Traffic Flow Table I shows the performance comparison among 13 state-of-the-art baselines on PeMSD3, PeMSD4, and PeMSD8 for multi-step traffic flow forecasting. Our TAMP- S2GCNets consistently outperforms baseline models on all 3 datasets except for PeMSD3, which underscore effectiveness of TAMP-S2GCNets achieves the best MAE and MAPE, and has an average 2.75% relative gain, com- pared the runner-up. The average relative gains of 1AMP--S2CNNT over the runner-ups in MAE and MAPE are 2.78% and 19% on the 3 datasets, respectively.
in time series forecasting tasks. On PeMSD3, our
Ethereum Blockchain Prices Table 3 (left) summarizes the comparison results in MAPE on Ethereum token networks (i.e., Bytom, Decentrland, and Golen). Table 3 indicates that TAMP- S2GCNets is always better than baselines for all dynamic Ethereum networks. We find that, compared to the baseline which also integrates topological information (i.e., Z-GCNETs), TAMP- S2GCNets is highly competitive, implying that the time-aware MP representation with DEPS is capable of better encoding time- tune-conditioee information than the zigzag idea based on one filtration.
COVID-19 Hospitalizations Table 2 shows results on I 15-day ahead forecasting of COVID-19 hos- pitalizations the U.S. states of California (CA) and Texas (TX) at a county level basis (RMSE is aggregated over each state). For the sake of room, here we only display the top runner models. DACNeS2 yields significantly better forecasting performance, with relative gains of 1.5% 24.5% (in CA) and 5.5%-46.7%% (in TX).
Table 1: Forecasting performance on PeMSD3, PeMSD4 and PeMSD8, along with results sta- tistical significance. Standard deviations are suppressed for the sake of room and are reported in Table 8 in Appendix E.
PeMSD3
PeMSD4
PeMSD8
Model
MAE RMSE MAPE
MAE
RMSE MAPE
MAE
RMSE MAPE (%)
FC-LSTM (Sutskever ul.,
21.33
35.11
23.33
27.14
41.59
18.20
22.20
34.06
14.20
SFM (Zhang el ul., 2017)
17.67
30.01
18.33
24.36
37.10
17.20
16.01
27.41
10.40
N-BEATS (Oreshkin et al., 2019)
18.45
31.23
18.35
25.56
39.90
17.18
19.48
28.32
13.50
DCRNN et al., 2018)
18.18
30.31
24.70
38.12
17.86 17.01
27.83
11.45
LSTNet (Lai ct al., 2018)
19.07
29.67
17.73
24.04
37.38
20.26
31.96
11.30
STGCN (Yu et al., 2018)
17.49
30.12
17.15
22.70
35.50
14.59
18:02
27.83
11.40
TCN (Bai al., 2018)
25.04
26.31
36.11
15.93
DeepState (Rangapuura al., 2018)
15.59
*20.21
18.69
26.50
33.00
15.40
19.34
27.18
16.00
GraphWaveNet (Wu et al., 2019)
19.85
32.94
19.31
26.85
39.70
17.29
19.13
28.16
12.68
DecpGLO (Sen al., 2019)
17.25
23.25
19.27
25.45
35.90
12.20
15.12
25.22
13.20
AGCRN (Bai
2020)
14.22
24.03
17.78
29.17
14.59
23.06
9.29
Z-GCNET's (Chen et al., 2021)
14.20
25.29
13,88
18.05
29.08
11.79
14,52
23,00
9.28
SlemGNN (Cao el al., 2020)
14.32
21.64
16.24
20.20
31.83
12.00
15.83
24.93
9.26
TAMP-S2GCNets (ours)
13.91
23.77
13.40
17.58
2828.66
*11.01
13.77
2221.70
8.99
5.2 ABLATION STUDIES
Contribution of TAMP-S2GCNets components We now conduct the ablation stud-
ies on PeMSD4 and PeMSD8 to evaluate contribution of different components of our framework. (Ablation results on Golem and CA are Table Appendix E.)
8

---

Published as a conference paper at ICLR 2022
Table 2: 15-day ahead forecasting results (RMSE) on COVID-19 hospitalizations in CA and TX.
We compare CAMPP-2GGNets. with 4 ablated variants, i.e. (i) TAMP-S2GCNets without DEPSRL module (w/o DEPSRL module), (ii) TAMP-S2GCNets without spatial graph convolutional layer (w/o spN) (iii) TAMP-S2GCNets with- out supragraph convolutional module (w/o GCNSup). and (iv) TAMP-S2GCNets with- out FT (w/o FT). As Table 3 (right panel) shows, ablating each of above causes the performance drops sharply in comparison with our full TAMP-S2GCNets model. Especially, on PeMSD4, the supragraph convolutional module significantly improves the results it simultane- ously captures the spatial and temporal information. In addition. it is evident that DEPSRL mod- ule enhances the topological information learning ability in spatio-temporal domain, i... TAMP- S2GCNets outperforms
Model
CA
TX
DCRNN (Li et al., 2018)
292.1
90.47-12.28
AGCRN (Bai ct al., 2020)
448.27=2.78
52.96++3.22
StemGNN (Cao et al., 2020)
377.2541.911
$1.00.12.60
TAMP-S2GCNets (ours)
"371.60 2.68 "48.2113.17
SAMP-S2 w/o DEPSRL module with an average relative gain 3.92% on RMSE over PeMSD4 and PeMSD8. Besides, expected, taking the advantages of GCN and FT, we improve the performance via capturing the graph structure in the spatial di- mension and consolidating the processing of spatio-temporal correlations between node attributes, respectively.
Table 3: Forecasting performance (MAPE in %) on Ethereum networks (left panel) and the TAMP- S2GCNets ablation study PeMSD4 and PeMSD8 (right panel).
Model
Bytom
Decentraland
Golem
Architecture
MAE
RMSE
MAPE
TAMP-S2GCNets
17.58
28.56
11.01
DCRNN (L. 2018)
35.36+-1.18
1.769=11.7
23.15:+1911
W/ DEPSRL
17.89
**29.99
*11.07
STGCN (Yu al., 2018)
37.333+1.06
28.22
23.68+22.11
PoMSI4 Wo GCNoq
*19.41
** 33'90.0
*** 1'111
GraphWaveNet (Wu al., 2019)
39.18:00.9
37.6721.76
28.89:2.334
W/o GCNop
"131
..* 12.64
W/o FT
*18.30
2*269.5
12.29
AGCRN (Bai et al., 2020)
3..46:1.337
26.7521.51
22.83:1.91
TAMP-S2GCNets
13.77
21.70
8.99
Z-GCNETs (Chea et al., 2021)
31,04:00.88
23,8122,33
22,32:11:42
Wi DEPSRL
1*.22
*222.39
9.32
StemGNN (Cao al., 2020)
34.911:1004
28.57
22.5042.01
PeMSDS W/ GCNq
14.16
2-2..29
9.40
W/ GCNsp
13.99
-21.91
9.07
TAMP-S2GCNtts (ours)
1.06
1.9999911.999 "20.10::2330
W/o FT
**14.36
2*22.60
9.26
Table 4: MAPE (%) (standard deviation) of TAMP-S2GCNets with single- and multifiltrations (left panel) and MAPE (%) and computation time for TAMP-S2GCNets with our DEPS, MP-J of Carrière & Blumberg (2020) and MP-L of Vipond (2020) (right panel).
TAMP-S2GCNes Bytom (Left)
TAMP-S2GCNets on Bytom (Right)
Single filtration
Multifitration
MP summary
MAPE (%)
Running time (s)
Deg & Rtwns: 30.02:1..05
MP-I [Carrièee & Blumberg, 2020)
33.13
401.80
Deg: 30.56 1.08
Bowns & ?owertus 29.96:1.0.6
MP-1. (Vipond, 2020)
32.19
517.11
Btwas: 30. 10
Bewna& & Powerwa: 29,27=00,78
DEPS 3exna
30.02
47.67
Power 31. 11.044+1.90
Deg & i 29.86=t 1.05
DEPS & ZoxarTm
29.86
39,46
Powerwa: 1.6
Deg Fowerw:: 1.06
srons PowerTie
29,26
29.84
Contribution of Single vs. Multifiltration Persistence We consider sublevel filtrations based on node degree (Deg), betweenness (Btwns), and graph diameter (power filtration) with edge weights induced by number of transactions (Powernus) ) and volume of transactions (Powerwa). As Table (left) shows, MP always outperforms single filtrations by significant margin, both in terms of prediction accuracy and variability. The results confirm our premise that MP is able (i) to capture hidden time dependencies of high dimensional time-varying objects which are inaccessible with one parameeerr PH and (ii) to yield substantially more stable feature maps in dynamic scenarios.
Contribution of DEPS vs. the Existing MP Summaries Finally, we evaluate performance of the new time-aware MP summary, i.e., DEPS, in comparison to the MP summaries based on the slic- ing argument, namely, MP-1 of Carrière & Blumberg (2020) and MP-L of Vipond (2020) as rep- resentation input
HAMP-S2GCNets As Table 4 (right panel) shows, comparing MP-I and MP-L, DEPS yields 9%-12% improvement in MAPE, and at least 10 times decrease in com- putational time (29.84s for DEPS vs. 401.80s for MP-I and 517.11s for MP-L). Such substantial differences in computational costs are explained by the need of the slicing- based MP summaries to search for the most suitable one-parameter PH representation of MP. In turn, DEPS is based

---

Published as a conference paper at ICLR 2022
wise representation argument in linear algebra, and its high computational efficiency makes DEPS the preferred choice for time-conditioned topological representation learning (see the running time in Table 4). Since computational costs remain the major roadblock for MP applications, our DEPS approach shows that MP tools based on scalable sparse matrix algorithms appear to be the most promising direction for integration of MP into machine learning tasks.
CONCLUSION
We have explored utility of MP to enhance knowledge representation mechanisms within the time- aware DL paradigm. The developed TAMP-S2GCNets model is shown yield highly competitive forecasting performance on a wide range of datasets, with much lower computational costs. In the future plan explore combination of MP with the zigzag persistence and to investigate asymptotic distributional properties of point-wise MP invariants.
10

---

Published as a conference paper at ICLR 2022
ETHICS STATEMENT
We do not anticipate any negative ethical implications of the proposed methodology. In turn, we believe that introduction of GCNs tools, coupled with multiparameter topological approaches, into biosurveillance opens a new path for more accurate, timely, and robust tracking infectious dis- eases with high virulence such COVID-19. In particular, multiparameter persistence allows for extracting the most salient topological features along multiple geometric dimensions and, as such, be especially valuable to address disease dynamics a function of multiple highly heteroge- neous variables, e.g., vecoo-demog,,phic, vocio-envirommentall and socio-economic factors. In turn, GCNs and, more generally, geometric deep learning allow for capturing sophisticated nonlinear spatio-temporal dependencies among factors which contribute to the disease spread. Hence, we pos- tulate that in the next few years we can expect to see new set of spatio-temporal biosurveillance artificial intelligence algorithms, based on a combination of geometric deep learning, multiparame- ter persistence, and more generally, tools of topological data analysis.
REPRODUCIBILITY STATEMENT
The source code for the experiments can be accessed under https: com/ tamps2genets/ TTMPP LLR///AAAASPSSSSSSE Saaplpsssceessssee git.
ACKNOWLEDGEMENTS
This work sponsored by National Science Foundation under award numbers ECCS 2039701, INTERN supplement for ECCS 1824716 16, DMS 1925346 and the Department of the Navy, Office of Naval Research under ONR award number N00014-21-1-22
Part of this material is also based upon work supported by (while serving at) the National Science Foundation. Any opinions, findings, and conclusions or recommendations expressed this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation and/or the Office of Naval Research. The authors are also grateful to the ICLR anonymous reviewers for many insightful suggestions and engaging discussion which improved clarity of the manuscript and highlighted new open research directions.
REFERENCES
U.S. Census Bureau: county adjacency file record layout. httpa//awwaaa..epopooooo/eeccaaa html, a. 2021-05-26.
https:// htops:://www.c.o census.
Accessed:
U.S. Census Bureau: county population totals and components of change. /opess/
https:
//www census........o wwt.e/suss..o//aaaaaaaaattttttte
2010==counttessttttllltooo b. Accessed: 2021-05-26.
CovidActNow:
u.s. covid risk vaccine tracker. https:
c. Accessed:
2021-05-26.
Etherscan: block explorer and analytics platform for ethereum, decentralized smart contracts plat- form. https: hete:://ethhree /tteeeeheernn...e io, d. Accessed: 2021-05-26.
Ethereum: community-run technology powering the cryptocurrency ether (ETH) and thousands of decentralized applications. https://ethereum. org, e. Accessed: 2021-05-26.
JHU: covid-19 data from the johns hopkins university. CSSEGTSandDDta//Co
https://giie thw.b com/
Accessed: 2021-05-26.
Henry Adams, Tegan Emerson, Michael Kirby, Rachel Neville, Chris Peterson, Patrick Shipman, Sofya Chepushtanova, Eric Hanson, Francis Motta, and Lori Ziegelmeier. Persistence images: A stable vector representation of persistent homology. Journal of Machine Learning Research, 18 (8):1-35, 2017. URL http://jmmli worel.oorrgg/oooooooooo http///ppppss////spp////ss 6- 337 .htll.
11

---

Published as a conference paper at ICLR 2022
Robert J Adler, Omer Bobrowski, Matthew S Borman, Eliran Subag, Shmuel Weinberger, et al. Persistent homology for random fields and complexes. Borrowing strengih: theory powering applications-a Festschrift for Lawrence D. Brown, pp. 124-143. 2010.
Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting, Advances in Neural Information Processing Systems, 33, 2020.
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv: 1803.01271, 2018.
Yuliy Baryshnikov and Robert Ghrist. Euler integration over definable functions. Proceedings of the National Academy of Sciences, 107(21):9525
2010.
Gabriele Beltramo, Rayna Andreeva, Ylenia Giarratano, Miguel O Bernabeu, Rik Sarkar, and Pri- moz Skraba. Euler characteristic surfaces. arXiv:2102.08260, 2021.
Chen Cai and Yusu Wang. Understanding the power of persistence pairing via permutation test. arXiv preprint arXiv:2001.0600 2020.
Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bix- iong Xu, Jing Bai, Jie Tong, and Qi Zhang, Spectral temporal graph neural network for multivari- ate time-series forecasting. Advances Neural Information Processing Systems, volume 33, pp. 17766-17778, 2020.
Gunnar Carlsson and Rickard Brüel Gabrielsson. Topological approaches to deep learning. In Topological Data Analysis. pp. 119-146. Springer, 2020.
Gunnar Carlsson and Afra Zomorodian. The theory of multidimensional persistence. Discrete & Computational Geometry, 42(1):71-93, 2009.
Mathieu Carrière and Andrew Blumberg. Multiparameter persistence image for topological machine learning. Advances in Neural Information Processing Systems, 33, 2020.
Mathieu Carrière, Frédéric Chazal, Yuichi Ike, Théo Lacombe, Martin Royer, and Yuhei Umeda Perslay: A neural network layer for persistence diagrams and new graph topological signatures. In AISTATS, pp. 2786-2796, 2020.
Andrea Cerri, Barbara Di Fabio, Massimo Ferri, Patrizio Frosini, and Claudia Landi. Betti num- bers in multidimensional persistent homology stable functions. Mathematical Methods in the Applied Sciences, 36(12):1543-1557 2013.
Chao Chen, Karl Petty, Alexander Skabardonis, Pravin Varaiya, and Zhanfeng Jia. Freeway perfor- mance measurement system: mining loop detector data. Transportation Research Record, 1748 (1):96-102, 2001.
Yuzhou Chen, Ignacio Segovia- and Yulia R Gel. Z-genets: Time zigzags graph con- volutional networks for time series forecasting. International Conference on Machine Learning, 2021.
David Cohen-Steiner, Herbert Edelsbrunner, and John Harer. Stability of persistence diagrams. Discrete computational geometry, 37(1):103-120, 2007.
René Corbet, Ulderico Fugacci. Michael Kerber. Claudia Landi, and Bei Wang. A kernel for multi- parameter persistent homology. Computers & graphics: X, 2:000055, 2019.
Baris Coskunuzer, Cuneyt Gurean Akcora, Ignacio Segovia Dominguez, Zhiwei Zhen, Murat Kantarcioglu, and Yulia R Gel. Smart vectorizations for single and multiparameter persistence. arXiv:2104.04789 2021.
M. di Angelo and G. Salzer. Tokens, types, and standards: Identification and utilization in ethereum. In 2020 IEEE International Conference on Decentralized Applications and Infras- tructures (DAPPS). pp. 1-10, 2020. doi: 10. 109/DAPSS49028 2020.00001.
Ensheng Dong, Hongru Du, and Lauren Gardner. An interactive web-based dashboard track covid-19 in real time. The Lancet Infectious Diseases, 20(5), May 2020. doi: 10. 10110 11473--009920331120-111
12

---

Published as a conference paper at ICLR 2022
Herbert Edelsbrunner and John Harer. Computational topology: introduction. American Mathe- matical Soc., 2010.
Herbert Edelsbrunner and Salman Parsa. On the computational complexity of betti numbers: re- ductions from matrix rank. In Proceedings of the twenty-fitth annual ACM-SIAM symposium on discrete algorithms, pp. 152-160. SIAM, 2014.
Robert Ghrist. Homological algebra and data. Math. Data, 25:273, 2018.
Allen Hatcher. Algebraic Topology. Cambridge University Press, 2002.
Christoph Hofer, Florian Graf, Bastian Rieck, Mare Niethammer, and Roland Kwitt. Graph filtration learning. In International Conference Machine Learning, pp. 4314-43445 4314-4323, PMLR, 2020.
Christoph D Hofer, Roland Kwitt, and Mare Niethammer. Learning representations of persistence barcodes. JMLR, (0(126):1445 2019.
Max Horn, Edward De Brouwer, Michael Moor, Yves Moreau, Bastian Rieck, and Karsten Borg- wardt. Topological graph neural networks. arXiv:2102.07835, 2021.
Michael Kerber. Multi-paramete persistent homology is practical. 2020.
Michael Kerber and Alexander Rolle. Fast minimal presentations of bi-graded persistence modules. In 202/ Proceedings of the Workshop on Algorithm Engineering and Experiments (ALENEX), pp- 207-220. SIAM, 2021.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st International ACM SIGIR Conference on Research Development Information Retrieval, pp. 95-104, 2018.
Claudia Landi. The rank invariant stability via interleavings. arXiv:1412.3371 2014.
Michael Lesnick and Matthew Wright. Computing minimal presentations and bigraded betti num-
bers of 2-parameter persistent homology. arXiv preprint arXiv:1902.05708, 2019.
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural net- work: Data-driven traffic forecasting. In International Conference Learning Representations, 2018.
Yitao Li, Umar Islambekov, Cuneyt Akcora, Ekaterina Smirova, Yulia R. Gel, and Murat Kantar- cioglu. Dissecting ethereum blockchain analyties: What we learn from topology and geometry of the ethereum graph? In Proceedings of the 2020 SIAM International Conference on Data Mining, SDM 2020, pp. 523-531, 2020.
Boris Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural ba- sis expansion analysis for interpretable time series forecasting. In International Conference Learning Representations, 2019.
Nina Otter, Mason A Porter, Ulrike Tillmann, Peter Grindrod, and Heather A Harrington. A roadmap for the computation of persistent homology. EPJ Data Science, 6:1-38, 2017.
Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting. Advances in neural information processing systems, 31:7785-7794, 2018.
Hans Riess and Jakob Hansen. Multidimensional persistence module classification via lattice- theoretic convolutions. 2020.
Rajat Sen, -siang-Fu Yu, Inderjit S Dhillon. Think globally, act locally: A deep neural net- work approach to high-dimensional time series forecasting. In Advances in Neural Information Processing Systems, volume 32, 2019.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, volume 27, 2014.
13

---

Published as a conference paper at ICLR 2022
Ashleigh Linnea Thomas. Invariants and Metrics for Multiparametter Persistent Homology. PhD thesis, Duke University, 2019.
Gugan C Thoppe, Yogeshwaran, Robert J Adler, et al. On the evolution of topology in dynamic clique complexes. Advances in Applied Probability, 48(4):989-1014, 2016.
Virginia Vassilevska, Ryan Williams, and Raphael Yuster. Finding the smallest h-subgraph in real weighted graphs and related problems. Automata, Languages and Programming, pp. 262-273. Springer Berlin Heidelberg, 2006.
Oliver Vipond. Multiparameter persistence landscapes. Journal of Machine Learning Research, 21 (61):1138, 2020.
Matthew Wright and Xiaojun Zheng. Topological data analysis on simple english wikipedia articles. arXiv:2007.0000 00000000.003,,..0 2020.
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, 2019.
Huaxiu Yao, Fei Wu, Jintao Kc, Xianfeng Tang, Yitian Jia, Siyu Lu, Pinghua Gong, Jieping Ye, and Zhenhui Li. Deep multi-view spatial-tempont temporal network for taxi demand prediction. Proceed- ings of the AAAI Conference Artificial Intelligence, volume 32, 2018.
Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temport graph convolutional networks: deep learning framework for traffic forecasting. In Proceedings of the 27th International Joint Confer- ence on Artificial Intelligence, pp. 3634-3640, 2018.
Liheng Zhang, Charu Aggarwal, and Guo-Jun Qi. Stock price prediction via discovering multi- frequency trading patterns. Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 2141-2149, 2017.
14

---

Published as a conference paper at ICLR 2022
A
NOTATION
Frequently used notation is summarized in Table 5.
Table The main symbols and definitions in this paper.
Notation
Definition
St
the spatial network at timestamp the number of nodes
N
M
the number of edges at timestamp
X
sequence of observations
multivariate variable
X,
node feature matrix at timestamp
C
simplicial complex
f
filtering function for sublevel filtration
Mr
number of k-simplices
(C(C)
k-dimensional persistence diagram
((C)
Fuler-Poincaré Characteristics
(c(C)
hthh homology group multivariate filtering function
F(;)
E
Euler-Poincaré Surfaces
(E')
Dynamic Painccr Surface probability space a filtration of o-fields
(,))
W
Wasserstein-1 distance the distance function for MP the Betti function the number k-simplices
D(.))
B()
M.
FN
the feature dimension the sliding window size the multivariate forecasting model
T
3()
S
the normalized self- eelf-adaptive adjacency matrix the learnable node embedding
E.
5
Laplacian tensor
K
the length of Laplacian tensor $ supra-Laplacian
C
the output DEPSRL module
F (.).))
-iiss the dimension wise concatenation function
B SINGLE PARAMETER PERSISTENT HOMOLOGY AND ITS SUMMARIES
All extracted topological features can be then summarized as a multiset in IR?, called persistence diagram (PD): (C) {(b,,,) E g2 b; < d,). where birth b; and death d; mark the filtration indexes at which the k-dimensional topological feature 0; appears and disappears, respectively. The farther (b, d,) from the diagonal is (that is, the more persistent p; is), the likelier p; is to contain salient information on G,- Multiplicity of (Ia, Ma) E the number p-dimensional topological features (p-holes) that are born and die at Tb and yb, respectively. Features on the diagonal of D have infinite multiplicity.
Another summary which may be particularly suitable in our context of modeling noisy time-evolving objects is the Fuler-Poincaré Characteristies (see Adler et al. (2010); Baryshnikov & Ghrist (2010) on the Fuler calculus and its applications, particular, in conjunction with sensor networks).
Definition B.1 (Fuler-Poincaré Characteristics). For a given simplicial complex C, Euler-Poincaré Characteristics x (C) is defined as the alternating sum the number of k-simplices of C. That is, ny is the number of k-simplices in C, then (C) = M ( (-1)"nn, where M is the dimension of highest dimensional simplex in C. Note that Fuler-Poincare Characteristics is homotopy invariant,
15

---

Published as a conference paper at ICLR 2022
and alternative definition is X(C) -- where (C( is the 4th Betti number of C, i.. rank of the 4thh homology group H (C) (Ghrist, 2018; Hatcher, 2002).
C PROOF OF THEOREM 3.1
In this section, we provide the proof of the stability theorem (Theorem 3.1). First, let us recall the key notations we use. Let G and G be two graphs and let C# be the clique complexes of G (Ghrist, 2018). Let F (S.g) be a multivariate filter function, i.e, F: = + R², where V the set of nodes of g+, and let I {(o,. 53) < I m, 1 S S n) be the corresponding thresholds for F (f.
Next, we define Wasserstein-p distance among PDs, i.c., the important concept we borrow from the theory of single parameter persistence (Edelshrunner & Harer, 2010).
Definition C.1 (Wasserstein-p distance). Let (C#, f*,I= be two single parameter filtrations, and DC be the corresponding PDs for k-cycles (i.e, k-dimensional topological features). Let 93' (b)) E (C+) be the birth and death times of k-dimensional topological feature Py- Then, Wasserstein-p distance between Dx (C+)) and D ) is defined as
Wg(Da(Ct) inf Σ
l0 -(g )1 ) P
F
JEDIC!! jUA
a bijection (matching) and A is the diagonal set, i.e., A = ((6, E RJ, which contains k-cycles of infinite multiplicity. With A in both sides, we ensure the existence of these bijections even if the cardinalities Ky, } and Ku H are different. When p 0o, (9) corresponds to the bottleneck distance, i.c. Wo. In the proof below, we use p p= 1, i.. Wi-distance.
where o: -
Armed the PD and Wasserstein-p distance concepts, new MP summary. Let Č {Cij) be the bifiltration associated with clique complex of the induced subgraph 9 ( au) x (-o0, 3z]). (Figure 2 illustrates such bifiltration on graphs with degree and eccentricity functions.) Let E be the the corresponding Surface, i.e. E= is an m n-matrix with entries F = x (C where x(-) is the Euler Characteristics (See Figure 2), and DCC*, C-) be the weak L₃ disiitt as defined in Section 3 of the paper (see also Remark C.1 below for the motivation use this new metric).
turn the main stability result for the
A.F.)..e. i.e. C is the
Proof of Theorem 3.1: We start from providing the outline of the proof. Notice that Coler-Poinncar Surfaces is the alternating sum of Betti numbers [Ba]. Hence, the distance ||E - || 1 bounded by the difference of the corresponding Betti numbers for Č+, i.. Eqs. (10) and (11). By fixing i and k and by taking column distance D. into consideration, our main assertion
-E
|E+ - E 1111 S C- CCC Ō-). C>0
then reduces to Eqs. (12) and (13). To relate the difference |B(C)) B(C)) to Wa(D(C*), D(C we observe that the Betti number B(C₁) is the sum of the number of bar- codes in ) containing 83, i.e. # B, E [b, d (Eqs. (13) and (14)). Yet, the differences of the number of barcodes containing B, for C+ and C can be bounded by the Wasserstein- 1 dis- tances of DCC+) ) and D(C-), i.e. Eqs. (15) and (16). Aggregating these terms on indices and finishes the proof.
Now, turn to derivations in details. Recall that x(C) = o-1) where (C) the th Betti number of C and M is the maximum homological dimension of C. Then, since E = x(C))) we have E Dkan o Ba (Ct)). Hence, for any i,j.
M
M
E E = Σ
Be < Σ [Ba - B (C (-1)* (A
(10)
k=0
16

---

Published as a conference paper at ICLR 2022
Since | - E
11,,1
Σi ii=1
2j=1
Bij
Biyl. we have
m
11,,1 |E -
E
S
ΣΣΣ B& CC (C
(11)
Now, complete the proof, need to bound the latter sum by the MP distance D(C+, Ö-). Recall that by Definition 3.3
D. , -0)-E> m
(IP - (((C)) 10Ct,O7
Then, by Eq. 11, to prove the theorem, it is enough to show that for any 0 S ko S M and for any 1 S S m,
Σ Bio (Ct,)) Bio (Crr)) s C Wi (C D (Cr,))),
C 0.
(12)
For simplicity, without loss of generality, we fix 1 S to < m and 0 < kg < N. Also. for sake
of notations, we drop the subscripts and superscripts, i. C# C and C cf c D(C+)=
iu+ D (C igo) and B(.) BRO With these simplified notations, Eq. 12 is equivalent to
B(C)) B(C)) S C. (I(D(CC))
C>0.
(13)
To show (13), we need to evaluate the relation between the Betti numbers B(C and the persistence diagrams DCC=). Consider D(C) {(b,, d,) 5 Q), where b, represents the birth time of a ko-cycle Pr in C, while d, represents the death time of pr. In particular, Pr induces a barcode [by. d..). For each such pr. define indicator function (L) (., i.e. (1) = for t E [br,, dx)) and 4/r (t) = 0 otherwise. Then, the Betti function of C is defined as B [8,,00 - N such that B(()= Σ - (() for [81, (Here, 31 is the lowest threshold for the filtering function in the set of thresholds = {(a..
S m, 1 S j S n). Similarly, if we consider row sum D, instead of column sum De. then would use the other filtering function f. and the Betti functions would be defined as B [a1, 00) + N.) Hence,
q+ (3)),, jEZ.
B(C)==
As result, we get
Q+
q (B,)
Σ B(C)) - -B(7))
#7 (3,)
(14)
1=1
j=1
1 (3,)
Jeer (8,)),
where : D(C+) D(C the best matching in the definition of (D(Ct)) ((C))) and Q = max ).
Notice that the contribution of each |nbt (B,) 2b (3,) to the sum at Bg less than the number (#r) of intervals the symmetric difference of the sets. Let K = min - 3,). By dividing the length b1 dt Abbt d(r)' bo(r))l by K, we ensure that every such interval contributes at least to the Betti number count. Hence, we arrive at the following key inequality:
b )r())) containing the interval (3j. where A is
1 K
(5tt von (B))
b,
boG,) + ld, dornl--
(15)
17

---

Published as a conference paper at ICLR 2022
Finally, since
Σ 10, bōcy + la, - -do(n) s <2W((D(C)))
obtain
n
2 -W((D(Ct),(CC))
Σ (
(16)
K
Since (16) holds for any 1 S in I m and 0 v ko v N. for C = 2/ K we get
m
n M
Σ
CC Do(Ot, Ö-). B.
By Eq. (11), this implies
E - 1111 SC SC-0" Ö-),
which completes the proof.
Remark C.1 (On the Choice of the Metric). While interleaving and matching distances are very common and useful for multipersistence in various cases (Thomas, 2019), they are not suitable to study the Fuler-Poincaré Surfaces. The reason for this is that matching and interleaving distances are based on Tox metrics, and do not see the quantity of generators of small interleaving distances. However, Euler-Poincaré Surfaces by definition depends the quantity of the generators (Betti numbers) are crucial in their dcfinition even they exhibit a very short lifespan. In particular, Loo -bsed metrics find the maximum distance between the essen- tial generators, but such metrics fail to capture the quantity of small generators. That is, while the interleaving or matching distance between the multipersistence modules of the bifiltrations {C+) and { is small, there might be very large distance between the induced Euler-Poincaré Sur- faces E and E- In turn, L.p -bsed distances are successful to capture the rank even if they are small (noise) generators, while interleaving and matching distances are not very sensitive to these. (See Thomas (2019, Section 4.2) for further discussion.) This is why we introduce the weak L1 metric above which naturally suits to study the Euler-Poincare Surfaces and similar vectorizations in the theory. Furthermore, such weak L1 metric can be considered a straightforward generalization of based norm for multiparameter persistence.
the rank of the homology groups, and
Remark C.2 (Implications of the Stability Result). Note that our stability result (Theorem 3.1) directly implies that the distances between multiparameter PDs control the distance between the resulting Fuler-Poincaré Surfaces. By combining with the stability result for PDs (Cohen-Steiner al., 2007), one can conclude that the small changes in the MP filtering function F V2 + I the small changes in the input data can result only in a small change in DEPS surfaces. In particular, by Cohen-Steine Steiner et al. (2007), for single parameter persistence,
have
(CD' DO (C)) S Cd(f.g),
where DJ is the persistence diagram for filtering function f, and d(f. g) is the distance between the functions f and g. The way we define our weak -metric enables us to use this single persistence result in this context as follows. The column distance D. the sum of single PD for each column. Each column, (say jth column) is represented by a single variable filtering function Ftt, Bj). Then, the changes in F F(t, B3) reflects in the column distance for PDs corresponding to jhh coluum.. This means if have small change in FI,, B,) for each j. then the column distance for MP will be small. So is the row distance induced by F(a;,))
Similarly, if have two MP filtering functions F+ V - R2, they will induce filtrations C+ When d(Ftt F) is small, then by Cohen-Steiner et al. (2007). D(C+, Č-) is small. When we apply our stability theorem to these filtrations, we see that the distance between the induced DEPS surfaces ||E E- 111, will be small, too. Since small change in the data can be converted small change in the filtering functions on the same data, the result can also be interpreted as the small change in the data results in small change in the induced DEPS surfaces.
18

---

Published as a conference paper at ICLR 2022
Remark C.3 (DEPS through the Probabilistic Perspective). Furthermore, the new time-awe aware MP summary can also be assessed from a probabilistic perspective, we view a multivariate time se- ries X' as real-valued random process indexed by time and defined
probability space (E, A,P), J is a o-field generated by x,, XJ11...... Xt, Then, E' is real-valued / / --measurrble random matrix and DEPS {EE' ] is IF-measurable random matrix, where FF = (F (F)= is a filtration of a-fields indexed
For S J S L I oc, define the o-field f C A such that / 1 = (X < S L (k E 2)), i.c.
time. Given the results of Thoppe et al. (2016) weak convergence of the Euler characteristic X(C) for dynamic Erdös- Erdös-Rényi graphs to the ConsteinUUhlenbeed process, this opens potential path to establish asymptotic distribution of EE".
Remark C.4 (DEPS for Higher Dimensions). DEPS can be naturally expanded a case when we consider filtrations along more than two geometric dimensions. In such case, instead of m n- matrix, DEPS will be tensor. Our stability result (i.e., Theorem 3.1) verbatim extends to such case.
3
3
0
0
1
1
2
G
E =
0
0
2
1
1
2
2
E
eccentricity
3
A
6
8,
6g
8,
B:
8
4
3
8
0
5
15
8:
B
8,
3
0
By
8
Bg
2
1
2
3
4
degree
Figure 2: Illustration of multidimensional persistence (i.e., multipersistence) In the original graph (the top left panel), red numbers represent the eccentricity of the node, while blue numbers are the node degrees. The top right panel depicts the corresponding Fuler-Poincaré Surface. Shape prop- erties of the top graph are evaluated along two geometric dimensions (i.e., degree and eccentricity). In particular, for each row, the rightmost graph is filtered by degree function. For each column, the top graph filtered by eccentricity function. Each cell includes Betti numbers Bd and B, Since B and B, are 0 for each cell, we get E, By
D REPRODUCIBILITY
D.1 DATASETS
We use three different datasets, from transportation, Ethereum blockchain, and biosurveillance do- mains, to examine the performance of the proposed TAMP-S2GCNets methodology:
19

---

Published as a conference paper at ICLR 2022
1. Transportation: The transportation and traffic data for the U.S. State of California, from the freeway Performance Measurement System (PeMS) data sources (i.e., PeMSD3, PeMSD4, PeMSD8) (Chen et al., 2001), naturally produces a dynamic network where each node is a loop detector and each edge represents a freeway between two nodes. We extract aggregated data to 5 minutes, thus generating dynamic networks with 26,208, 16,992 and 17,856 nets in PeMSD3, PeMSD4 and PeMSD8 datasets, respectively. To capture spatio-temporal dependencies, we re- build the traffic graph structure, at time i, and compute edge weights Wt, between pair nodes (u, v). via the Radial Basis Function W/, we e =,,,,==,,,=l" //, />; where 7 1.0 and each node Tt,- has features speed, occupancy and flow rate. To investigate the dynamic of the traffic graph topology, restrict the final graphs only keep edges whose are below equal to threshold -==00 0.01.
2. Blockchain: Ethereum blockchain data contain token assets which naturally represents network layers of the Ethereum network (di Angelo & Salzer, 2020; Web, e), in which nodes and edges addresses of users and digital transactions, respectively. Using the publicly available blockchain, extract dynamic networks from three tokens with market value above $ $100,000,000 Bytom, Decentraland and Golem. In our experiments, we extract dynamic networks by daily transactions and use daily closed prices (Web, d), hence, creating graph/net for each day. Since each token is created at different date, in all cases last day is 05/07/2018, the dynamic networks vary in the number of nets: Bytom (285 nets), Decentraland (206 nets) and Golem (443 nets). For the sake of reasonable computation time, we reduce the original net size using the maximum weight subgraph approximation method of Vassilevska et al. (2006) from more than 400,000 nodes, average of 120,000 edges, to 100 nodes using number of transactions and price-volume as edge weights.
3. Biosurveillance: SARS-CoV-2, which is the virus that causes COVID-19, has spread around every place in the world. To analyze the progression of coronavirus disease we create a dynamic network based on daily records on COVID-19 cases and hospitalizations from the CovidActNow project (Web, c) and Johns Hopkins University (Web, f. Dong et al., 2020), and population num- bers from the U.S. Census Bureau (Web, b). We choose data at county level from California and Texas states as our cases of study, and focus on forecasting the number of hospitalized patients. Similarly to (i) we rebuild the coronavirus-spread graph structure, at day-time f, based on the county adjacency graph (Web, a) and Radial Basis Function between pair of nodes. We build dynamic network for each state, each dynamic network contains 335 nets. The number of con- firmed COVID-19 cases and county population size serves as node features, and final graphs only keep edges whose are below or equal to the threshold a = 0.01.
More details on experimental settings and filtering functions are provided in Section D.2. Table 6 summarize the characteristics of all datasets used in our experiments.
Table 6: Summary of datasets used in time series forecasting task.
Type
Dataset
No. Nodes
No. avg Edges
Time interval
Traffic
PcMSD3
358
55.91
09/01/2118 11/30/2018
Traffic
PeMSD4
307
26.32
01/01/2018
28/02/2018
Traffic
PeMSD8
170
18.51
01/07/2016
31/08/2016
Ethereum
Bytom
100
9.98
27/07/2017
07/05/2018
Ethereum
Decentraland
100
16.94
14/10/2017
07/05/2018
Ethereum
Golem
100
20.58
18/02/2017-
07/05/2018
COVID-19
CA
55
56.15
01/02/2020-
31/12/2020
COVID-19 19
TX
251
929.80
01/02/2020- 31/12/2020
D.2 EXPERIMENTAL SETTINGS
with Pytorch framework on NVIDIA GeForee RTX 3090 GPU. Further, for all datasets, TAMP-S2GCNers is trained end-to-end by using the Adam opti- mizer with L1 loss function. The tuning of our proposed TAMP-S2GCNess done via grid search over a fixed set of choices and the same cross validation setup is used to tune baselines. We compare TAMP-S2GCNets with 13 types of state-of-the-ar methods, including
We implement our
each dataset is
20

---

Published as a conference paper at ICLR 2022
FC-LSTM (Sutskever et al., 2014). SFM (Zhang et al., 2017), N-BEATS (Oreshkin et al., 2019), DCRNN (Li et al., 2018), LSTNet (Lai et al., 2018), STGCN (Yu et al., 2018), TCN (Bai et al., 2018) 8), DeepState (Rangapuram et al., 2018), GraphWaveNer (Wu et al., 2019), DeepGLP (Sen et al., 2019), AGCRN (Bai et al., 2020), Z--CNNT's (Chen al., 2021), and StemGNN (Cao al., 2020).
MODEL SPECIFICATIONS For PeMSD3, TAMP-S2GCNets consists of 2 layers whose hidden feature dimension is 64. The learning rate is 5e-3 and the batch size is set 64. The "cost" das of staying in the same node u (in the layer (n) and the "cost" d'at of jumping from the current node in layer la to node a in layer ia are set to be 0.1 and 0.2, respectively.
For PeMSD4, AAMC-e2 consists of 2 layers whose hidden feature dimension is 128. The learning rate is 5e-3 and the batch size is set 64. The "cost" d'af" of staying in the same node u (in the layer ta) and the "cost" diato of jumping from the current node u in layer to to node u in layer to are set to be 0.1 and 0.2, respectively.
consists of 2 layers whose hidden feature dimension 64. The learning rate is 2e-3 and the batch size is set to 16. The "cost" d' of staying in the same node (in the layer ta) and the "cost" d'ato jumping from the current node in layer to to node u in layer to are set to be 0.1 and O, respectively. The dimension of node embedding de (in Eg)) are 10 (PeMSD3), 10 (PeMSD4), and 20 (PeMSD8), respectively. We set the dimension K of Laplacian tensor S for all transportation networks.
For PeMSD8, HAMP-S2GCNCR
For all Ethereum token networks (i.e., Bytom, Decentraland, and Golem), TAMP-S2GCNets consists of 2 layers whose hidden feature dimension is 32. The final hyperparameter setting is learning rate of 1e-2 and batch size of 8. The "cost" d'at= of staying in the same node u (in the layer un ta) and the "cost" d'='= of jumping from the current node in layer t, to node in layer to are set to be 0.01 and 0.01, respectively. In addition, we set the embedding dimension de in Es as I for all Ethereum token networks. We set the dimension K of Laplacian tensor S as 2, 3, 3 for Bytom, Decentraland, and Golem, respectively.
For COVID-19 hospitalizations in CA and TX, TAMP-S2GCNers consists of 2 layers whose hid- den feature dimension is 128. The learning rate is set le-1 and 1e-2 for CA and TX, respectively. The batch size is set to 8, and the embedding dimension d, in E, is 10. The "cost" d' of staying in the same node u (in the layer la) and the "cost" d'ats of jumping from the current node u in layer to to node u in layer is are set to be 0.1 and 0.2, respectively. We set the dimension K of Laplacian tensor S as 2 for both CA and TX.
ALTERNATIVE OPTIONS FOR DEPS WITHIN DEPSRL MODULE In our experiments,
for all datasets, the CNN based model (i.e.. fe, where j {1, 2} in Twae-Aware Multipersistence Euler-Poincaré Surface Representation Learning (DEPSRL) module) consists of 2 CNN layers. For transportation networks and COVID, The filter size, kernel size, and stride is set to be 8, 2, 2 re- spectively. For Ethereum token networks (i.e., Bytom, Decentraland, and Golem), the filter size, kernel size, and stride is set to be 8, 1, respectively. We set the size of global average pooling and global max pooling as 3 x 3. Table 7 shows the optimal output dimensions of spatial graph convo- lutional layer (QSpa), supragraph diffusion convolutional layer (Qsup), and pptio-ttmppral feature transformation (QFT) on all datasets.
Table 7: The output dimensions of spatial graph convolutional layer (Qspa), supragraph diffusion convolutional layer (Qsup), and spatio-temport feature transformation (QFT) on different datasets.
Output dimension PeMSD3 PeMSD4 PeMSD8 Bytom Decentraland Golem
CA
TX
Qspa
Qui
Qui
3Qmt
Qui
Qui
3Qus
Qui
Qui
Qsup
Qun
Qui
Qun
Qai
Que
Qui
Qun
Qui
QFT
Qan
Qut
8
Qu
Que
Qai
Qan
A
2
2
21

---

Published as a conference paper at ICLR 2022
Furthermore, in addition to the Time-Awarr Multipersistence Euler-Poincare Surface Representation Learning (DEPSRL) module (see Section 4.3 in our main paper):
p (++1) 1TAMP @ ((E)) ((1)), foMP (Se, ( (EE)))))
we also have considered passing the aggregated DEPS as input to DEPSRL, that is, fo, (B), = 1,2, where E = 1/T - E 1=
As expected, DEPS (E') tends to yield better performance within the DEPSRL. module than its aggregated counterpart E, which can be explained by lower loss of cine-conditioned topological information within DEPS. However, in the case of PeMSD4 dataset, we find that E slightly improves the performance of S2MNess (i.e., the MAE of -etCeesseets based on the aggregated DEPS 17.58 and the MAE of TAMP-S2GCNets based on DEPS is 17.69), which is likely due to higher data heterogeneity of PeMSD4 vs. other transportation datasets. In all other datasets, report results based on DEPS.
COMPARISON TO OTHER MP SUMMARIES For fair comparison, computation of all MP representations (as shown Table 11 and Table 4 (right panel) in our main paper) have been run on an 8-cores DO droplet machine with Intel Xeon Scalable processors at base frequency of 2.5 Ghz. Our source codes are available for revision'. Source codes for computation of MP-I and MP-L are freely available in Github?. We follow the filtering function recommendations for graphs, on both MP-I and MP-L, from supplementary material of Carrière & Blumberg (2020). Further technical details on the 8-core machine are publicly available".
ON TYPES OF MULTIFILTRATIONS Good filtrations should have rich range get fine resolution, and it should reflect an important property relevant to the question at hand. For mul- tipersistence, for good combination of two such filtering functions, one should choose functions uncorrelated. One approach is to choose one function from the domain of the question, and the other from strictly graph properties (c.g. degree, betweenness, etc.). We assess shape characteristics of G. via filtering functions based on node degree (Deg), betweenness centrality (Btw,s), and edge weights (P)wer). Edge weights for each dataset are computed as described in D.I; i.e using transac- tions between addresses (Trns), volume of transactions (Vol), traffic measurements (Traffic) and COVID-19 incidence (Incidence). Our analysis focuses on multivariate filtering function F V4 + Idd with 2. For Deg and Btwns the set of nondecreasing thresholds runs on values over nodes, whilst for filtering via power filtration (P (ower) of G, nondecreasing thresholds runs on weighted paths between nodes. Notice that each cobinaambinntoon of filtering functions produces a specific MP representation in form of the Euler-Poincar Poincaré surface, which impacts the algorithm performance, shown in our multiple experiments. Table 12 presents the optimal multifiltration for each dataset.
E ADDITIONAL EXPERIMENTS AND RUNNING TIME
Table 8: Forecasting performance and standard deviations on PeMSD3, PeMSD4 and PeMSD8 datasets, All models are re-run using the original authors' code.
PeMSD3
PeMSD4
PeMSDS
Model
RMSE
MAPE
RMSE
MAPE
RMSB
MAPE (%)
DCRNN (Li al., 2018)
18.1810.15
30.3110:22
0..9
24.7010.22
18.12.
07.1
17.8610.03
27.83.10.05
11.45:10.03
STGCN (Yu et 2018)
17.4910.16
30.12:10.70
17.15=0.15
22.70110.44
35.5010.75
11.59-0.21
18.0220.144
27.83.10.00
11.1010.10
DecpState (Rangapuram all, 2018)
15.5920.43
20.21:0.00
18.6900.22
26.
25
33.00:06.7
15.40=0.23
19.3420.11
27 27.1840022
16.00:30100
Graph (Wu
2019)
19,85+003
32.94-0.118
19.31-0.49
26.85-003
70-000044
17.29-0.24
19.13-008
28 16+00.77
12.68-0.55
AGCRN (Bai et al., 2020)
14.22:10.11
21.03:10.36
13.89-0.29
17,78:0.15
29.17.:0009
11.79-0.111
14.9910.40
23.06.0.333
9.2911.111
Z-GCNETss (Chen et al., 2021)
14,20::9.58
25 29:50.3
13,88-00.23
18:05=00200
29.08:99119
11,79.00.0
14,52:00155
33,00-99.00
9,28:49.31
TAMP-S2GCNes (us)
13.91::0.16
23.77.0,32 13.40=0.39 17.58:0.20 28.56:20.28
11.01=0.10
13:77:20.08 21.70:10.25
8.99::0.15
Why MP Helps? More Experiments To validate gains (if any) delivered by multifiltration com-
pared the single filtration persistence and ensemble of multiple stacked together single parameter
LOAOaNa?
d1-0
hhttpp://gita ://gi htt.h//wwwoo..tttheeaarrrrrrreee
chttps://wer digitaliglllaooo /et.gaaaoeeeeccm//bb///////eeee
intel -itel-ccacadee lake- lake-amd-epyo-romm
22

---

Published as a conference paper at ICLR 2022
Table The TAMP-S2GCNets ablation study on Golem (MAPE) and CA. Here *, $*, *** denote
significant, statistically significant, highly statistically significant results.
Dataset
Architecture
Golem (MAPE)
CA (RMSE)
TAMP-S22 FAMP-S2GCNets
20.10%
371.60
W/o DEPSRL.
20.37%
*380.18
W/o GCN Spa
"1.551%
*378.87
W/o GCN Sup
*20.72%
**382.63
W/o FT
**21.08%
***383.37
filtrations, we also report the results of single filtration, filtration ensemble, and multifiltration on Decentraland (see Table 10). (In case of filtration ensemble, aggregate two single filtrations and feed the ensemble representation into the model.) We find that MP substantially outperforms both the single filtration persistence and the filtration ensemble. In addition, the MP results are generally more stable than ones based on the single filtration and filtration ensemble. This result can be expected, as by construction of DEPS, each individual single parameter persistence corre- sponds either to a row or column of DEPS, that is, we have only mie-dimensional fingerprint of the data. In contrast, in MP and its DEPS summary, we obtain a two-dimensional fingerprint of the data, meaning any cell in the output simultaneously related to the all 8 neighboring cells (East, West, Northeast, etc.). As such, MP provides 2D resolution of the data space and delivers much finer information on the underlyng data topology.
Table 10: MAPE (in% (standard deviation) of TAMP-S2GCNets with single filtration, filtration ensemble, and multifiltration persistence.
TAMP-S2GCNets on Decentraland
Single filtration
Filtration ensemble
Multifittratio
[eg, Btwns]: 24.35 [Btwns, Powerrmel: 23.00
Deg & Btwns: 11.13:1.55
Deg: 25.18
Btwns & Powerrms 20.00=11.51
Btwns: 24.93+1.88
[Btwns, Powerval: 22.70+2.99
Btwns Powervar 21.59-11.76
Powe 23.81+1.66
[Deg, Powerrms] 1.65
Deg & Powernss 19.89:1.669
Powervaj 22.15+:2.70
[Deg, Powervo]] 21.58+3.03
Deg & Powerval 2.88
Table shows the computational costs and performance of different MP summaries on Decen- traland and Golem datasets. We observe that our new time-aware MP summary DEPS not only outperforms the existing MP summaries (i.e., MP-I and MP-L) but also induces substantially lower computational costs. These findings demonstrate that our DEPS can effectively capture salient time- aware topological information with lower time and memory cost.
Table 11: MAPE (%) and computational time (in second) on Decentraland and Golem for TAMP- S2GCNets with our DEPS and MP-I Carrière & Blumberg (2020) and MP-L of Vipond (2020), as MP summaries.
MP summary
Decentraland
Golem
MP-I (Carrière & Blumberg, 2020)
22.58 (366.71s)
22.00 (850.20g)
MP-L (Vipond, 2020)
20.10 (489.50s)
22.05 (1137.09s)
DEPSpen & BLwas
21.13 (3.2
21.05 (126. 15s)
DEPS Poxerim
19.89 36.90s)
20.10 (112.61s)
DEPS Sewnsesns&c & Powertts
20.00 (25.34s)
21.00 (85.87s)
COVID-19 Biosurveillance: More Experiments Finally, Figure 3 depicts the results on 15-day ahead forecasting of hospitalizations in California county level basis, using the top 3 models (i.e., AGCRN (Bai et al., 2020), StemGNN (Cao et al., 2020), and our TAMP-S2GCNets). (Here, to differentiate among RMSES county level basis, omit the map for DCRNN (Li et al.,
23

---

Published as a conference paper at ICLR 2022
13
RMSE
23
0.28
3079.63
11
40
17
20
110
50
62
11
212
57
102
22
97
38
52
35
13
53
29
106
475
67
494
1642
2,6666
261
285
221
37
510
30
(a) TAMP-S2GCNcts:
(b) StemGNN.
RMSE
Hebolt
0.28
3079.63
31
13
1.3
50
59
195
1.0
157
98
49
19
17
677
3,080
482
499
62
AGCRN.
(d) California counties.
Figure 3: Maps of RMSE values for 15-day ahead forecasts of COVID-19 19 hospitalizations on county level in Califoria and counties in California. (a) TAMP-S2GCNets. (b) StemGNN. (c)
AGCRN. (d) California counties (from California Department Education).
2018) due to its lower predictive performance, relative to RMSE scale of AGCRN, StemGNN, and our (AMP-S2GCNett.) Overall, TAMP-S2GCNG and StemGNN tend to perform similarly for less populated counties in Northern California, with TAMP-S2GCNets being slightly better than StemGNN over the Pacific Coast and StemGNN yeilding some edge in the North East. In turn, AGCRN tends to deliver more competitive performance in North North-Eastern California, except of Shasta county.
However, performance among models starts differ drastically in more populated counties and, in particular, in Southern California where it is much harder to forecast COVID-19 hospitalizations due to higher population density and more disparate socio-economic factors. Here, TAMP-S2GCNes
24

---

features and graph structures can be obtained from Published as a conference paper at ICLR 2022
Table 12: Types of multifiltration for all datasets.
Dataset
Multifiltration
PeMSD3
Deg & Btwns
PeMSD4
Deg & Btwns
PeMSD8
Btwns & Powe I'rallic
Bytom
Btwns & Powerтmm
Decentraland
Deg & PowerTms
Golem
Deg & Powe
COVID-19 CA
Btwns & Power
COVID-19 TX
Deg & Powe I Incidence
substantially outperforms both StemGNN and AGCRN almost in all counties of Southern Califor- nia, yielding predictive gains from 3.8% vs. StemGNN and 29. 29.8% vs. AGCRN in San Bernardino to 57.7% vs. StemGNN and 55.7% vs. AGCRN in San Diego. While all models perform very poorly in AAMP-S2GCNets still yields the low-
Los Angeles county (i.e., one of the most populated counties), est RMSE. One important phenomenon we observe is that TAAMP-S2GNNR systematically tends to substantially outperform StemGNN and AGCRN in more highly populated counties, including both Southern and Northern California. For instance, the predictive gains of TAMP-S2GCNets in Santa Clara are 39.2% vs. StemGNN and 67.9% vs. AGCRN. These findings indicate that time- aware MP topological information contained in TAMP-S2GCNets can address hidden relationships among spatio-temporal factors contributing to COVID-19 dynamics which other models cannot.
F ADDITIONAL DETAILS OF TAMP-S2GCNETS ARCHITECTURE AND EXPERIMENTS
Remark F.1 (Spatio-Tempore Feature Transformation). We utilize and share the learnable node embedding Eg to (i) construct the normalized self-adaptive adjacency matrix S and (ii) project the target node from the input feature space (in spatial and temporal dimensions) into a high-level latent space (together with @FT). ). Hence, this -pdaae weights can not only mitigate the risk of over- fitting but allow to capture shared patterns among over the time axis. The learnable node embedding Eg in the backward propagation through time.
We use Einstein summation convention in Eq. for the multiplication of XT E RRxxTxNxP (where b is the batch size) and Eg θ (2) (i) we first perform the matrix multiplication between Eg Nxde and o (() E Rddxx to get an intermediate weight term denoted by + ( Xxxxr (ii) then compute the matrix mul- tiplication between X and (() through Einstein summation convention, i.e., in Pytorch, output = torch.einsum( btnni, nio + btno', X, intermediate, weaptlewweiihtttteeeeeeee term) (where X is X and intermedi- att_.wiihttterm is θ (
FT The implementation of matrix multiplication in Eq. 1 is as follows:
Remark F.2 (Normalized Self-Adaptive Adjacency Matrix). We utilize the node embedding dic- tionaries Eg E IR xd. to construct the normalized self-adaptive adjacency matrix S via multiplying Eg and E As such, construction of the normalized self-adaptive adjacency matrix S does not im- pose any requirements on density of the input graph. That is, our graph learning architecture (i.c., the self-adaptive adjacency matrix construction mechanism) overcomes this problem of the dense input graph met by GAT.
To explore more computationally efficient mechanisms of learning the normalized self-adaptive ad- jacency matrix, in the future work, we will (i) apply the sparse sampling strategies the graph learning architecture; (ii) learn the graph based on the original graph structure information, i.e., we only infer the dependencies between the adjacent nodes (i.e., there exists an edge between them in the original graph) via multiplying eu and e (iii) construct hypergraph representation instead of graph which not only alleviate the computational complexity but capture heterogeneous higher-order structures information.
25

---

Published as a conference paper at ICLR 2022
For S. in our experiments, performed an extensive grid search for the hyperparameter K (i.e., truncation order). We find the optimal K of for the transportation network, the optimal of for COVID-19 network dataset, and the optimal K of 3 for Ethereum blockchain network. That is, for larger networks the truncation order K' tends to be lower so that the computational costs are mitigated.
Furthermore, we have performed ablation studies on PeMSD4 and PeMSD8 assess utility of the normalized self-adaptive adjacency matrix S in TAMP-S2GCNec The results in Tables 13 and 14 indicate that stead of the normalized self-adaptive adjacency matrix) for both PeMSD4 and PeMSD8 across all the evaluation metries. The results illustrate the importance of learning the node-specific pat- terns/embeddings over the
outperforms TAMP-S2GCNet. with the plain adjacency matrix (in-
time series dataset.
Table 13: Ablation study of the normalized self-adaptive adjacency matrix on PeMSD4.
Architecture
MAE
RMSE
MAPE (%)
17.58
28.56 28.66
11.01
TAMP-S2GCNer with plain adjaceney matrix
17.69
11.74
Table 14: Ablation study of the normalized self adaptivee adjacency matrix on PeMSD8.
Architecture
MAE
RMSE
MAPE (%)
TAMP-S2GCNer
13.77
21.70
8.99
TAMP-S2GCNets with plain adjacency matrix
13.94
21.98
9.09
Remark F.3 (Global Average Pooling and Global Max Pooling). We utilize the global average pooling (GAP) to preserve information about the Euler-Poincaré surfaces fixed-size representa- tion through averaging of the learnt topological embeddings. In our experiments, we find that adding global max pooling (GMP) tends strengthen the Euler-Poincaré surfaces representation and to im- prove the overall performance. Before GAP and GMP, we first use CNN base models (i.e., fo, (-) and Sa, (.)) to learn the topological features of Suler-Poincan surfaces (i.e., fo, (( where i i= )); then we employ GAP and GMP to the outputs of CNN base models, respectively. We have conducted the ablation study to evaluate the performance of TAMP-S2GCNets without (w/o) GAP or GMP on Golem. From Table 15, we find that applying both GAP and GMP enhances prediction performance.
Table 15: Ablation study on global average pooling and global max pooling. Here denotes signif- icant results.
Architecture
TAMP-S2GCNets
TAMP-S2GCNets w/o GAP
TAMP-S2GCNets w/o GMP
MAPE (%)
200.10
20.83
20.57
Remark F.4 (Supra-Laplacian) In our experiments, for large-scale networks, we apply sparse sampling strategy for the supra-Laplacian construction. For example, on transportation network such as PeMSD3, we randomly select 7 = 3 timestamps out of the T = 12 timestamps in the slid- ing window; result, feed the sparse supra- Supra-Laplacian (i.e., 3NN 3N: where N is the number of nodes) into the supragraph diffusion convolutional layer. For smaller networks such as Ethereum token networks, we utilize the T timestamps (i.e., the whole sliding window information) to struet the supra-Laplacian. In the future we will explore various graph sparsification techniques, e.g., subsampling, to reduce computational costs for larger networks.
Remark F.5 (More Details on the Framework of TAMP-S2GCNets in Figure 1). In Figure 1, the input is the target network over past time slices (f9r-2. 1. and our HAMP-S2GCNEC consists of 5 components. That is, (i) equipped with node feature matrix X₁, we apply spatial graph convolutional layer (i.e., GCN Layer in Figure 1) on G. extracts spatial information time (ii) feature transformation (i.e., FT in Figure 1) learns representation of the spatio-temporal data T over sliding window of size T (in Figure 1, for better understanding the operation procedure,
26

---

Published as a conference paper at ICLR 2022
set T 3, ie.. X X,-,,,,,,1,,8 (iii) we create multiplex network based on the in- put networks (i.e., (9--2, 1,,, and construct the corresponding supra-Laplacian; lastly, we feed the supra-Laplacian into supragraph convolutional module (i.e., Supra GCN Layer in Figure 1); (iv) in Figure 1, we consider two types of Euler-Poincaré Surface Representation (DEPS), i.e., j f.g and J.PS are Fuler-Poincaré surface representations under different multifiltrations, where J, h. 9 represent different types of filtrations; in (iv), we feed Fuler-Poincaré surface repre- sentations into CNN base models and employ global average/max pooling to the outputs of CNN base models. Finally, we combine these embeddings obtain the final embedding, where is fed into GRU module for multi-step forecasting.
G ADDITIONAL COMPUTATIONAL COMPLEXITY AND TRAINING TIME COMPARISON
Remark G.1 (Training Time Comparison). We the average training time (per epoch in sec- onds) for our TAMP-S2GCNets and 4 baselines on PeMSID4 and Decentraland datasets in Tables 16 and 17. In terms of training time, MMMP-S2GCNets runs slightly slower than the baselines but the demonstrated running time comparable the baselines. Although the time per epoch is slightly higher for T'MP--2GCNeess, we find that the performance of TAMP-S2GCNets is significantly better than the ofate-of-the-art baselines. As such, this extra time appears to be worth
Table 16: Average training time (per epoch in seconds) for our TAMP-S2GCNets and 4 baselines on PeMSD4 dataset, where ** denotes statistically significant results.
TAMP-S2GCNets StemGNN
Z-GCNett
AGCRN
DCRNN
Average time RMSE
40.30 s
30.12
37.53 s 29.08
28.05 s 29.17
28.79 s
**28.56
31.83
38.12
Table 17: Average training time (per epoch in seconds) for our TAMP-S2GCNC and 4 baselines on Decentraland dataset, where *** denotes highly statistically significant results.
TAMP-S2GCNets
StemGNN
Z-GGNNets
AGCRN
DCRNN
Average time
3.10 *** 19.89
2.55 s
2.09 s
2.03 26.75
2.21
MAPE (%)
28.37
23.81
27.69
Remark G.2 (More Details on Computational Complexity of Topological Summaries). Com- putational complexity of Dynamic Fuler-Poincaré Surfaces (DEPS) much lower than that of per- sistence diagrams. This is because Fuler Characteristics an alternating sum of the Betti numbers, and one does not need to compute the persistence diagrams compute the Betti numbers. The com- putational complexity of a single persistence diagram PD; is O O(N3), where A is the number of k-simplices (Otter et al., 2017). On the other hand, by using sparse matrix methods, computational complexity for the Euler Characteristics
a simplicial complex with M simplices is ((M) (Edels- brunner & Parsa, 2014). If the resolution size of the multipersistence grid, the resulting complex- ity for DEPS is Opp²M) However, since one does not need to compute the persistence diagrams to obtain the Betti numbers, faster algorithms are possible. For instance, Lesnick & Wright (2019) developed a much faster algorithm (RIVET software) to compute Betti numbers in multipersistence setting. This reduction approach leads to computational complexity of O(R²), where R is the size of the filtered complex giving the minimal representation for the bipersistence module and R is far less than the original number of simplices. Furthermore, by utilizing sparse matrix reductions, re- cent results of Kerber & Rolle (2021) sigmificantly improved the RIVET approach. In particular, computational complexity for multipersistence landscapes with RIVET would be O(R5), where R again the size of the reduced filtered complex (Vipond, 2020).
We have provided both computational complexity and running time of 8 topological summaries on Decentraland and Golem (see Table 18).
The computational complexity of the overall approach is: O(N + NTFN + NT* /2 + Qsui 1 (( R² Wanr). That is, we have: (i) spatial graph convolution module: ((N2)
27

---

Published as a conference paper at ICLR 2022
Table 18: Computational complexity and running time (in seconds) comparison on Decentraland dataset.
Persistence summary"
Running time
MAPE
Computational complexity
MP-I (Carrière & Blumberg, 2020)
366.71 s
22.58
Q)P 6
MP-L (Vipond, 2020)
489.50 s
23.10
(
DEPSueg & stwns
43.25
21.13
Opp
DEPS Powernm.
36.90
19.89
or ((R))
DEPS Bewns BowBETm
25.34
20.00
Plrag (Adams al., 2017)
11.43
23.81
Pletwna (Adams et al., 2017)
10.01
24.93
O
PIp Ploowertos (Adams al., 2017)
12.79
25.18
"Deg: node degree; Btwns: node betweenness centrality; Power: edge weight; Trns: transaction be- tween addresses. Please refer on rypes of filtrations (Appendix D.2) more detailed description types of multifiltraiio..
*Where p is the resolution of MP-1, Q is the number of lines, B is the maximum number of bars of the fibered barcodes.
'Where R the size of the reduced filtered complex (sce previous paragraph). M the number of simplices the simplicial complex.
"The reported time for DEPS is given with the O(p² x M) algorithm.
"This computational complexity for the time after the computation of persistence diagrams.
(where is the number nodes); (ii) supragraph diffusion convolution module: ONNT FN + NT Quup - (f) (where Fx is the number of node attribute features, T denotes the sliding window size, Quup is the output dimension of supragraph diffusion convolutional layer, and (2) the number of edges the f-th layer); (iii DEPS: ( where R is the size of the reduced filtered complex; (iv) GRU: OWWRR)), where WGRU is the total number of parameters the GRU.
28


Reviews:
[{"review_id": "X-BVcg3sIt2", "reviewer": "Reviewer_rDH8", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- This paper addresses a very relevant problem and one of broad impact on the community.\n- Although at the very core, this paper is an attempt towards bringing together two emerging research directions, that combination is novel and unique. Furthermore, methodologically, there is a good degree of novelty in combining these two research directions (e.g., by introducing a new time-aware multi-parameter persistence invariant). Also, the authors make it clear how this work differs from previous contributions.\n- The paper is technically sound.\n- This is a well-written paper and generally well structured, making good use of appendices. \n- The empirical evaluation is extensive and covers 3 different application domains. The empirical results are also very promising and supportive of the claimed advantages of the proposed approach. \n\nWeaknesses:\n- A discussion of the computational complexity of the proposed approach with respect to other time-series forecasting baselines is missing. The authors provide runtimes for the different multi-parameter persistence summary approaches but, a discussion of the computational complexity of the overall proposed approach would also be important. \n- The supra-Laplacian is an NT x NT matrix, which raises questions about scalability that are not discussed in the paper.\n- Some statements are not adequately supported by empirical evidence. For example: \"The developed TAMP-S2GCNets model is shown to yield highly competitive forecasting performance on a wide range of datasets, with much lower computational costs.\" - this may be true across multi-parameter persistence summaries, but there are no empirical results to support such a claim across baselines. \n- There are some choices that seem arbitrary or poorly justified. Perhaps the most obvious is in Eq. 6. Why are global average pooling and global max pooling concatenated together? What are both types of pooling needed?\n- The text in Figure 1 is nearly impossible to read. ", "summary_of_the_paper": "This paper proposes the use of multi-parameter persistence (an emergent research topic in topological data analysis) to capture latent time-conditioned relations among nodes in a GNN. To do so, the authors introduce a dynamic Euler-Poincaré surface as a new multi-parameter persistence summary, and prove its stability and empirically show its computational efficiency. A \"supra\" graph convolution module is then proposed in order to allow for simultaneously learning co-evolving spatial and temporal correlations in the complex multivariate time-series data. The superior predictive performance of the proposed approach is shown empirically based on 3 different multi-variate time-series datasets from 3 different domains: highway traffic flow, Ethereum token prices, and COVID-19 hospitalizations. ", "summary_of_the_review": "Overall, this paper takes the important first steps towards a very promising research direction of bringing novel ideas from topological data analysis into GNN approaches to multi-variate time-series forecasting, and therefore could inspire and spawn several follow-up works. The paper is generally well-written. The experiments are extensive, cover 3 different application domains, and clearly demonstrate the potential of the proposed approach in particular, and this research direction in general. I have a few concerns regarding e.g. computational complexity that I would like to see more discussion on, but those do not prevent me from giving this paper a recommendation for acceptance. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "BN0Wf2DM7K_", "reviewer": "Reviewer_T4LP", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper proposes introduction of the multi-parameter persistence (as defined in the topological analysis)\nto capture the time-dependent properties of spatio-temporal data and be a part of time-aware learning of \na convolutional network. The fusion of the two presents a novel research direction with improved performance over \nstate-of-the-art approaches. \n\nStrong points:\n- novel research direction (as explained above)\n- the topic is of interest for the community. An effective approach of using multipersistence to allow \nfor capturing time dependencies and enhancing the learning of graphs.\n- technically sound. The reviewer did their best to check the soundness of the technical backbone.\n- Thorough experimental part. Code is provided and reproducability seems possible and well detailed in \nthe supplementary as well.   \n- The writing is clear. The motivation and is well stated and supported by also a very well presented \nmethodology. \n\nWeak points:\nNot many here. \n\n- I have some questions regarding implementation choices that seem arbitrary- but probably \nresult of experimentation. In Eq(6) global average pooling and global max pooling are used. Some \ndetails here would be nice.\n- I would also add more analysis on the computation time compared to other methods. This is provided in \nthe supplementary material but I would consider moving the results in the main paper.\n\n\n", "summary_of_the_paper": "The authors suggest the use of multiparameter persistence for \nexplicitly capturing he latent time dependencies in spatio-temporal data. \nThey propose the Time-Aware Multipersistence Spatio-Supra Graph \nConvolutional Network that uses the mutlipersistence as extracted by the Euler-Poincare \nsurface and allows for time-aware learning. \n", "summary_of_the_review": "A very interesting paper that proposes an approach that I expect to be impactful for future research. \nI would like this paper to be part of the ICLR accepted works.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "9h8ZxntsO4L", "reviewer": "Reviewer_8UH6", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper bridges time-aware deep learning with time-conditioned multiparameter persistence and this leads to the development of a new model TAMP-S2GCNets, which includes five components: (1) spatial graph convolutional layer, (2) feature transformation module, (3) supragraph convolutional module, (4) the DEPSRL module, and (5) GRU layer. The paper contributes non-trivial advancements over state-of-the-arts. The main concerns from the review’s end are as follows:\n(1) How the learnable node embedding E_\\phi is learned is not clear and the multiplication of X by E_\\phi in Eqn. (1) is confusing.\n(2) The mechanism to learn the normalized self-adaptive adjacency matrix S is similar to the self-attention mechanism used in GAT, which needs the input graph to be dense and which leads to inefficiency. In addition, \\tilde{S} is a power series of S. The computation will be inefficiency. So the inefficiency is a big problem of this paper.\n(3) In the ablation study, the paper does not compare using the normalized self-adaptive adjacency matrix S with using the pain adjacency matrix.\n", "summary_of_the_paper": "The paper introduces multipersistence into graph neural networks (GNNs) to render GNNs capable of finding hidden time-conditioned patterns in spatio-temporal graph data. The proposed model, Time-Aware Multipersistence Spatio-Supra Graph Convolutional Network (TAMP-S2GCNets), outperforms other state-of-the-arts methods on various spatio-temporal graph data.", "summary_of_the_review": "The paper’s contribution is non-trivial. Although the inefficiency may be a problem, I recommend accept.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors introduce the Time-Aware Multiperistence Spatio-Supra Graph CN that uses 
multiparameter persistence to capture the latent time dependencies in spatio-temporal data. 

This is a novel and experimentally well-supported work. The novelty is achieved by combining research in topological analysis (multipersistence) and neural networks. Technically sound. Clear presentation and extensive experimental section.
Reviewers were uniformly positive, agreeing that the approach was interesting and well-motivated, and the experiments convincing. Some concerns that were raised were successfully addressed by the authors and revised in the manuscript. 

Happy to recommend acceptance. A veru nice paper!
Decision: Accept (Spotlight)