Topological Graph Neural Networks
Authors: Max Horn
Edward De Brouwer
Michael Moor
Yves Moreau
Bastian Rieck
Karsten Borgwardt
Venue: ICLR 2022 Poster (2022)
Keywords: topology
persistent homology
gnn
graph neural networks
graph classification
node classification
filtrations
topological data analysis
tda

Abstract: Graph neural networks (GNNs) are a powerful architecture for tackling graph learning tasks, yet have been shown to be oblivious to eminent substructures such as cycles. We present TOGL, a novel layer that incorporates global topological information of a graph using persistent homology. TOGL can be easily integrated into any type of GNN and is strictly more expressive (in terms the Weisfeiler–Lehman graph isomorphism test) than message-passing GNNs. Augmenting GNNs with TOGL leads to improved predictive performance for graph and node classification tasks, both on synthetic data sets, which can be classified by humans using their topology but not by ordinary GNNs, and on real-world data.

Published as a conference paper at ICLR 2022
TOPOLOGICAL GRAPH NEURAL NETWORKS
Max Horn 1.2 Edward De Brouwer Michael Moor!.
Yves Moreau
Bastian Rieck!. 1 Karsten Borgwardt 1.2
Department of Biosystems Science and Engincering, ETH Zurich, 4058 Basel, Switzerland
SIB Swiss Institute of Bioinformatics, Switzerland
"ESAT-STADE DAUUS, KU Leuven, 3001 Leuven, Belgium
"Instiuute of AI for Health, Helmholtz Munich, 85764 Neuherberg, Germany Techhical University of Munich, 80333 Munich, Germany
"These authors contributed equally.
These authors jointly supervised this work.
ABSTRACT
Graph neural networks (GNNs) are a powerful architecture for tackling graph learn- ing tasks, yet have been shown to be oblivious eminent substructures such cycles. We present TOGL, novel layer that incorporates global topological infor- mation of a graph using persistent homology. TOGL can be easily integrated into any type of GNN and is strictly more expressive (in terms the Weisfeiler-Lehme graph isomorphism test) than message- passing GNNs. Augmenting GNNs with TOGL leads to improved predictive performance for graph and node classification tasks, both on synthetic data sets, which can be classified by humans using their topology but not by ordinary GNNs, and on real-world data.
INTRODUCTION
Graphs are natural representation of structured data sets in many domains, including bioinformatics, image processing, and social network analysis. Numerous methods address the two dominant graph learning tasks of graph classification networks (GNNs) describe flexible set of architectures for such tasks and have seen many successful applications over recent years (Wu et al. 2021). At their core, many GNNs are based on iterative message passing schemes (see Shervashidze and Borgwardi (2009) for an introduction to iterative message passing in graphs and [Sanchez-LLenglling et al. (2021) for an introduction to GNNs). Since these schemes are collating information over the neighbours of every node, GNNs cannot necessarily capture certain topological structures in graphs, such cycles (Bouritsas et al.) (02T). These structures are highly relevant for applications that require connectivity information, such as the analysis of molecular graphs (Hofer et al., 2020) Swenson et al.) 2020).
node classification. In particular, graph neural
We address this issue by proposing Topological Graph Layer (TOGL) that can be easily integrated into any GNN to make it 'topology-award" Our method is rooted in the emerging field of topological data analysis (TDA), which focuses describing coarse structures that can be used to study the shape of complex structured and unstructured data sets at multiple seales. We thus obtain a generic way to augment existing GNNs and increase their expressivity in graph learning tasks. FFiuure 1 provides motivational example that showcases the potential benefits of using topological information: (i) high predictive performance is reached earlier for a smaller number of layers, and (ii) learnable topological representations outperform fixed ones if more complex topological structures are present in a data set.
Our contributions. We propose TOGL, a novel layer based on TDA concepts that can be inte- grated into any GNN. Our layer is differentiable and capable of learning contrasting topological representations of graph. We prove that TOGL enhances expressivity of a GNN since it incorporates the ability to work with multi-scale topological information in graph. Moreover, show that TOGL improves predictive performance of several GNN architectures when topological information is relevant for the respective task.

---

Published as a conference paper at ICLR 2022
100.0
100.0
Tea socuracy (in %)
Test socurary (in %)
75.0
50.0
50.0
GCN-TOGL
GCN
GCN-TOGL
WL
25.0
25.0
Number of leyensiterations
Nastber of leyersinerations
(a) CYCLES
(b) NECKLACES
Figure I: As motivating example, we introduce two topology-based data sets whose graphs can be easily distinguished by humans; the left data set can be trivially classified by all topology- based methods, while the right data set necessitates learnable topological features. We show the performance of (i) a GCN with & layers, (ii) our layer TOGL (integrated into a GCN with & - 1 layers): GCN-TOGL, (iii) the Weiseeiler-Lehman (WL) graph kernel using vertex degrees as the node features, and (iv) a method based on static topological features (PH). Next to the performance charts, we display examples of graphs of each class for each of the data sets.
BACKGROUND: COMPUTATIONAL TOPOLOGY
We consider undirected graphs of the form (V, F)) with a set of vertices and a set of edges E C V x V. The basic topological features of such a graph G are the number of connected components 3n and the number of cycles 31. These counts are also known as the O-dimensional and 1-dimensional Betti numbers, respectively; they are invariant under graph isomorphism (Hatcher) 2002, pp. 103-133) and can be computed efficiently. The expressivity of Betti numbers can be increased using graph filtration, i.e. sequence nested subgraphs G such that (( C Q()) C ((2) C into the graph by 'monitoring' topological features of each G1 and calculating their topological relevance, also referred to as their persistence. If topological feature appears for the first time in (*)
Gtn-1) (()) G. filtration makes possible obtain more insights
and disappears in GU), we assign this feature persistence of i. Equivalently, we can represent the feature as tuple (i,j). which we collect in a persistence diagram D. If a feature never disappears, the ones that are counted for the respective Betti
represent by a tuple (ł, oo); such features numbers. This process was formalised and extended to a wider class of structured data sets, namely simplicial complexes, and is known under the name of persistent homology. One of its core concepts is the use of filtration function f: V + Rd, with d = 1 typically, to accentuate certain structural features of graph. This replaces the aforementioned tuples of the form (i,j) by tuples based on the values of f, i.e. Jy) Persistent homology has shown excellent promise in different areas of machine learning research (see Hensel al. (202I) for a recent survey and Appendix for a more technical description of persistent homology), ), with existing work stressing that choosing or learning an appropriate filtration function crucial for high predictive performance (Hofer et al. 2020
Zhao and Wang 2019).
Notation. We denote the calculation of persistence diagrams of a graph G under some filtration by G). This will result two persistence diagrams DD0), D(L), containing information about topological features in dimension 0 (connected components) and dimension (cycles). The cardinality of D()) is equal to the number of nodes n in the graphs and each tuple in the O-dimensional diagram is associated with the verfex that created it. The cardinality of DD(1) is the number of cycles; we pair each tuple in ((L) with the edge that created it. Unpaired edges-edges that are not used to create cycle- assigned 'dummy' tuple value, such as (0, 0). All other edges will be paired with the maximum value of the filtration, following previous work by Hofer et al. (2017).
3 RELATED WORK
Graph representation learning has received a large amount of attention by the machine learning community. Graph kernel methods address graph classification via (implicit explicit) embeddings in Reproducing Kernel Hilbert Spaces (Borgwardt et al., 2020 Kriege et al. 2020; Nikoo,ii Nikolentzos et al., 2019). While powerful and expressive, they cannot capture partial similarities between neighhour-
2

---

Published as a conference paper at ICLR 2022
hoods. This can be achieved by graph neural networks, which typically employ message passing on graphs to learn hidden representations of graph structures (Kipf and Welling 2017 Wu et al.
1021
Recent work in this domain is abundant and includes attempts to utilise additional substruc-
tures
Bouritsas et al. 2021) as well as defining higher-order message passing schemes (Morris et al.
2019)
or algorithms that generalise message passing to more complex domains (Bodnar et al. 2021
Our approach falls into the realm of topological data analysis (Edelsbrunner and Harer 2010 and employs persistent homology, a technique for calculating topological features-sucl as connected components and cycles-of structured data sets. These features are known to be highly characteristic, leading to successful topology-driven graph machine learning approaches (Hofer al.. 2017 2020) Rieck et al. 2019; Zhao and Wang (2119). At their core is the notion of a filrration, i.c. a sequence of nested subgraphs (or simplicial complexes in higher-dimennional setting). Choosing the right filtration is known to be crucial for obtaining good performance (Zhao and Wang 2019). This used to be a daunting task because persistent homology calculations are inherently discrete. Recent advances in proving differentiability enable proper end-to-end training of persistent homology (Carrière et al. 202T), thus opening the door for hybrid methods of increased expressivity by integrating the somewhat complementary view of topological features. Our method TOGL builds the theoretical framework by HHoee et al. (2020), who (i) demonstrated that the output of a GNN can be used to 'leam' one task-specific filtration function, and (ii) described the conditions under which filtration function is differentiable. This work culminated in GFL, a topological readout function that exhibited improved predictive performance for graph classification tasks. We substantially extend the utility of topological features by making existing GNNs 'topology-awand generic layer that makes topological information available to all downstream GNN layers: TOGL can be integrated into any GNN architecture, enabling the creation of hybrid models whose expressivity provably more powerful than that of GNN alone. Moreover, while GFL only uses the output of GNN to drive the calculation of topological features by means of single filtration (thus limiting the applicability of the approach, as the topological features cannot inform the remainder of a network), TOGL learns multiple filtrations of graph in end-to-end manner. More preciscly, TOGL includes topological information in the hidden representations of nodes, enabling networks to change the importance of the topological signal. Closest the scope of TOGL is Zhao al. 2020 who enhanced GNNs using topological information for node classification. In their framework, however, topology is only used to provide additional scalar-valuce weights for the message passing scheme, and topological features are only calculated over small neighbourhoods, making use of static vectorisation technique of persistence diagrams. Similarly, WWong and Vong (202T) use static, i.e. non-learaalle, topological features for 3D shape segmentation. By contrast, TOGL, being end-to-end differentiable, more general and permits the calculation of topological features at all dales--ncludiigg graph-level features as well as an integration into arbitrary GNN architectures.
through the development of
4 TOGL: A TOPOLOGICAL GRAPH LAYER
TOGL is a new type of graph neural network layer that is capable of utilising multi-scale topological information of input graphs. In this section, we give a brief overview of the components of this layer before discussing algorithmic details, theoretical expressivity, computational complexity, and limitations. Figure presents an overview of our method (we show only a single graph being encoded, but in practice, the layer operates on batches of graphs).
The layer takes as input a graph G = (V, E) equipped with a set n vertices V and a set of edges E, along with set of d-dimensional node attribute vectors f(()) E Rd for e E V. These node attributes can either be node features of a data set or hidden representations learnt by some GNN. We employ a family of & vertex filtration functions of the form : Rd + IR for i = I,...,A, k. Each filtration function fi can focus on different properties of the graph. The image of ft is finite and results in
a set of node values a: < ... < ag (n) such that the graph G is filtered according to 0 = G S
a(") .... C G(")) G. where G(O) (vv.) (v.",,(()) with v,0) := ( queV V fo (z( a (
and D : (u W E max { ()) Ss (I(()) S a (1) }. Given this filtration, we calculate a set of persistence diagrams, i.c. ph(G, fi) = ( ,...., !!. We fix / (i.c. we are capturing connected components and cycles) to simplify our current implementation, but our layer can be
3

---

Published as a conference paper at ICLR 2022
Crlo R
a
(al node attbuute g!1)
(b) mode mup
(6) graphs G
filtrations f.
(6) Penistence diagram
(1 agggnegation
(g) ourpor a
E R of graph G serve as input. b network & maps (x) to node values {al ...... dg c R. Applying 4x to the attributes of each vertex results in & views of G. A vertex filtration is computed for the üth view of G. [ A set of k persistence diagrams is encoded via the embedding function V, where T [e] denotes the embedding of vertex v. f The embedded topological features are combined with the input attributes 2(() Finally, this yields g(0) R which acts a new node representation augmented with multi-scale topological information.
Figure 2: Overview of TOGL, topological graph layer.
a
The node attributes 2x())
extended to arbitrary values of (see Appendix Coorr discussion). In order to benefit from representa- n for embedding persistence diagrams into nigh-dimensionaa space that will be used to obtain the vertex representations, where n' is the number of verlices n if = 0 and the number of edges if = 1. This step is crucial it enables us use the resulting topological features as node features, making TOGL layer that can be integrated into arbitrary GNNs. We later explain the precise mapping of (1) from set of diagrams to the elements of a graph.
tions that are trainable end-to-end, we use an embedding function ((()) { {DIT
D. (
R"
Details on filtration computation and output generation. We compute our family of k vertex-based filtrations using Φ: Rd + R& an MLP with a single hidden layer, such that fa:= Ty o &. i.e. the projection & to the ith dimension. We apply & to the hidden representations x(1) of all vertices in the graph. Morcover, we treat the resulting persistence diagrams in dimension 0 and 1 differently. For dimension 0, we have a bijective mapping of tuples in the persistence diagram to the vertices of the graph, which was previously exploited in topological representation learning (Moor al.) 2020). Therefore, aggregate (0) with the original node attribute vector xr(2) of the graph in [u], where ()()) [v] denotes taking oth
residual fashion, i.e. g(v) = z!4) + (0) (D!")... ..... D( "
row of (0) (i.e the topological embedding of vertex v). The output of our layer for dimension 0 therefore results in a new representation (²()) E IRd for each vertex making it compatible with any subsequent (GNN) layers. By contrast, (1) is pooled into a graph-level representation, to be used in the final classification layer of GNN. This is necessary because there is no bijective mapping the vertices, but rather to edges. For stability reasons (Bendich et al. (020), we consider it more useful to have this information available only on the graph level. For further details on the computational aspects, please refer to /Section A.4
Complexity and limitations. Persistent homology can calculated efficiently for dimensions and 1, having worst-case complexity of O (ma (m)) for a graph with m sorted edges, where ax-) is the extremely slow-growing inverse Ackermann function, which can be treated as constant for all intents and purposes. The calculation of ph is therefore dominated by the complexity of sorting all edges, i.e. (m log m). making our approach efficient and scalable. Higher- dimensional persistent homology calculations unfortunately do not scale well, having a worst-case complexity of O (md) for calculating dddiimensional topological features, which is why we restrict ourselves to / = 1 here. Our approach is therefore limited to connected components and cycles. Plus, our filtrations are incapable of assessing topological feature interactions; this would require learning multifiltrations (Carlsson et al. 2009), which do not afford a concise, efficient representation as the scalar-valued filtrations discussed in this paper. We therefore leave their treatment to future work.
4.1 CHOOSING AN EMBEDDING FUNCTION
The embedding function V influences the resulting representation of the persistence diagrams calculated by our approach. It is therefore crucial to pick class of functions that are sufficiently
4

---

Published as a conference paper at ICLR 2022
powerful to result in expressive representations of persistence diagram D. We consider multiple types of embedding functions V, namely (i) novel approach based on DeepSets (Zaheer et al. 2017), (ii) the rational hat function introduced by Hoter et al. (2019). as well as (ii) the triangle point transformation, (iv) the Gaussian point transformation, and (v) the line point transformation, with the last three transformations being introduced in Carrière et al. (220). Except for the deep sets approach, all of these transformations are local in that they apply to single point in a persistence diagram without taking the other points into account. These functions can therefore be decomposed as (r()) (D("...., .....,.y D()) [c] (T()) [c] DK [ei) for index v. By contrast, our novel deep sets approach uses all tuples in the persistence diagrams to compute embeddings. In practice, we did not find a significant advantage of using any of the functions defined above; thus treat them as a hyperparameter and refer to tolAppeddix I for a detailed analysis.
4.2 DIFFERENTIABILIT & EXPRESSIVE POWER
The right choice of V will lead to a differentiable downstream representation. The map ph(.) was shown to be differentiable (Gameiro et al.. 2016 Hofer et al. 20200] Moor et al. 20200 Poulenard ctt al. 2018), provided the filtration satisfies injectivity at the vertices. We have the following theorem, whose proof is due to Hofer et al. 2020 Lemma 1).
Theorem 1. Let fo be vertex filtration function fo: Vo - R with continuous parameters 6, and let V be a differentiable embedding function of unspecified dimensions. the vertex function values of fo are distinct for specific set of parameters O', i.e. solv) + so(w) for + w. then the map 0 + v (ph(G,fo) is differentiable at 0'.
This theorem is the basis for TOGL, as it states that the filtrations, and thus the resulting 'views' on graph G. can be trained end-to-end. While Hofer et al. (2020) describe this for single filtration, their proof can be directly extended to multiple filtrations as used our approach.
The expressive power of graph neural networks is well-studied (Chen et al. [2200)] Xu et al. 2019) and typically assessed via the iterative Leimann label refinement scheme, denoted as WL[1]. Given a graph with an initial set of vertex labels, WL[[]] collects the labels of neighbouring vertices for each vertex in multiset and 'hashes' them into a new label, using a perfect hashing scheme so that vertices/neighhourroooe procedure is repeated and stops either when a maximum number of iterations has been reached or no more label updates happen. The result of each iteration h of the algorithm for graph G is a feature vector d (h) that contains individual label counts. Originally conceived as a test for graph isomorphism (Weisfeiler and Lehman 1968), WL[1] has been successfully used for graph classification (Shervashidze and Borgwardt 2009). The test runs in polynomial time, but is known to fail to distinguish between certain graphs, L.c. there the same labelling by WL[1] Fürer (017). Surprisingly, Xu et al. (019) showed that standard graph neural networks based message passing are no more powerful than WL.[1]. Higher-order refinement schemes, which pass information over tuples of nodes, for instance, can be defined (Maron et al.) 2019; Morris et al. 2019). Some of these variants are strictly more powerful (they can distinguish between more classes of graphs) than WL[1] but also computationally more expensive.
with the same labels are hashed to the same value. This
non-isomorphic graphs G and G' that obtain
To prove the expressivity of our method, we will show that (i) TOGL can distinguish all the graphs WL[1] can distinguish, and (ii) that there are graphs that WL[1] cannot distinguish but TOGL can. The higher expressivity of TOGL does not necessarily imply that our approach will perform generally better. In fact, [L[[1] and, by extension, GNNs, are capable of identifying almost all non-isomorphic graphs (Babai et al. 1980). However, the difference in expressivity implies that TOGL can capture features that cannot be captured by GNNs, which can improve predictive performance if those features cannot be obtained otherwise. Since persistent homology is an isomorphism invariant, we first show that we distinguish the same graphs that WL[1] distinguishes. We do this by showing the existence of an injective filtration function f thus ensuring differentiability according to Theorem 1.
Theorem Persistent homology is at least as expressive as WL[1], i.e. the WL[1] label sequences for two graphs G and G' diverge, there exists an injective filtration J such that the corresponding O-dimensional persistence diagrams Do and Di are not equal.
We drop 0 in this notation since we do not require f to have set of continuous parameters here; the subsequent theorem is thus covering a more general case than Theorem 1
5

---

Published as a conference paper at ICLR 2022
Proof sketch. We first assume the existence of sequence of WL[1] labels and show how to construct a filtration function ff from this. While f will result in persistence diagrams that are different, thus serving to distinguish G and G', it does not necessarily satisfy injectivity. We therefore show that there is an injective function f that is arbitrarily close to f and whose corresponding persistence diagrams Do. Di do not coincide. Please refer to Appendix B for the detailed version of the proof.
The preceding theorem proves the existence of such a filtration function. Due to the capability of GNNs approximate the Weisfeiler-Lehman test (Xu et al. 2019) and the link between graph isomorphism testing and universal function approximation capabilities (Chen et al.. [2019] Theorem 4), we can deduce that they are also able to approximate f and f. respectively. Yet, this does not mean that we always end up learning f or f. This result merely demonstrates that our layer can theoretically perform al least as well as WL[[]] when it comes to distinguishing non-isomorphic graphs; this does not generally translate into better predictive performance, though. In practice, TOGL. may learn other filtration functions; injective filtrations based WL[[]] are not necessarily optimal for specific task (we depict some of the learnt filtrations in Appendix G).
To prove that our layer is more expressive than a GCN, we show that there are pairs of graphs G, G" that cannot be distinguished by WL[1] but that can be distinguished by ph(-) and by TOGL, respectively: let G be graph consisting of the disjoint union of two triangles, i.c. <D, and let G' be a graph consisting of hexagon, i.e. <>. WL[1] will be unable to distinguish these two graphs because all multisets every iteration will be the same. Persistent homology, by contrast, can distinguish G from G' using their Betti numbers. We have Bo(G) = (G) = 2, because G consists of two connected components and two cycles, whereas (o')) = Bi (G') = 1 as G' only consists of one connected component and one cycle. The characteristics captured by persistent homology are therefore different from the ones captured by WL[1]. Together with Theorem 2 this example implies that persistent homology is strictly more powerful than WL[1] (see Appendix for an extended expressivity analysis using higher-order topological features). The expressive power of TOGL hinges on the expressive power of the filtration-making it erucial that we can learn it.
EXPERIMENTS
We showcase the empirical performance of TOGL on set of synthetic and real-world data sets, with a primary focus on assessing in which scenarios topology can enhance and improve learning on graph. Next to demonstrating improved predictive performance for synthetic and structure- based data sets (Section 5.2 and /Section 5.3), we also compare TOGL with existing topology- based algorithms (Section 5.5).
5.1 EXPERIMENTAL SETUP
Following the setup of Dwivedi et al. (2020), we ran all experiments according to a consistent training setup and a limited parameter budget to encourage comparability between architectures. For further details, please refer to AAppendix D In all tables and graphs, we report the mean test accuracy along with the standard deviation computed over the different folds. All experiments were tracked (Biewald 2020]; experimental logs, reports, and code will be made publicly available.
Baselines and comparison partners. We compare our method to several GNN architectures from the literature, namely (i) Graph Convolutional Networks (Kipf and Welling 2017 GCN), (ii) Graph
Attention Networks (Veličković et al.) 2018 GAT), (iii) Gated-GCN (Bresson and Laurent 2017).
(iv) Graph Isomorphism Networks (Xu et al. 2019 GIN), (v) the Weisfeiler-Lehman kernel (Sher-
vashidze and Borgwardi] 2009, WL), and (vi) WL-OA (Kriege et al., 2016). The performance of
these methods has been assessed in benchmarking papers (Borgwardt et al. 2020 Dwivedi et al. et al.) 2020), whose experimental conditions are comparable. We use the same folds and hyperparameters as in the corresponding benchmarking papers to ensure a fair comparison.
TOGL setup. We add our layer to existing GNN architectures, replacing the second layer by TOGL, respectively 2 For instance, GCN-4 refers to a GCN with four layers, while GCN-3-TOGL- refers
We investigate the implications of different layer placements in Ampppdxx and find that the best placement. depends on the data set; placing the layer second a compromise choice.

---

Published as a conference paper at ICLR 2022
to method that has been made 'topoloyyawaar using TOGL. original and the modified architecture have approximately the same number of parameters. We treat the choice of the embedding function V/ hyperparameter in training for all subsequent experiments. [Appendix provides a comprehensive assessment of the difference between the DeepSet. approach (which is capable of capturing interactions between tuples in persistence diagram) and decomposed embedding functions, which do not account for interactions.
This setup ensures that both the
5.2 PERFORMANCE ON SYNTHETIC DATA SETS
As an illustrative example depicted in Figure we use two synthetic balanced 2-class data sets of 1000 graphs each. In the CYCLES data set (Figure la right), we generate either one large cycle (class 0) or multiple small ones (class 1). These graphs can be easily distinguished by any topological approach because they differ in the number of connected components and the number of cycles. For the NECKLACES data set (Figure lb. right), we generate 'chains' of vertices with either two individual cycles, or 'merged" one. Both classes have the same number of cycles, but the incorporation of additional connectivity information along their neighbourhoods makes them distinguishable for TOGL, since appropriate filtrations for the data be learnt. All synthetic data sets use node features consisting of 3-dimensional vectors sampled from a Normal distribution. As PH and WL would consider all instances of graphs being distinct and thus remove any potential signal, we used the node degrees as features instead as commonly done (Borgwardt et al. 2020).
For this illustrative example, we integrate TOGL with GCN. We find that TOGL performs well even without any GCN layers-thus providing another empirical example of improved expressivity. Moreover, we observe that standard GCNs require at least four layers (for CYCLES) or more (for NECKLACES) to approach the performance of TOGL. WL[1], by contrast, fails to classify CYCLES and still exhibits a performance gap to the GCN for NECKLACES, thus showcasing the benefits of having access to a learnable node representation. It also underscores the observations in /Section 4.2 the higher expressive power of WL.[1] as compared to a standard GCN does not necessarily translate to higher predictive performance. Appendix F provides an extended analysis with more configurations; we find that (i) both cycles and connected components are crucial for reaching high predictive performance, and (ii) the static variant is performing slightly better than the standard GCN, but significantly worse than TOGL. due to its very limited access to topological information. A simple static filtration (based node degrees), which we denote by PH, only works for the CYCLES data set (this is consequence of the simpler structure of that data set, which can distinguished by Betti number information already), whereas the more complex structure of the NECKLACES data necessitates learning task-based filtration.
5.3 STRUCTURF-BASED GRAPH AND NODE CLASSIFICATION PERFORMANCE
recent survey (Borgwardt et al. 2020) showed that the node features of benchmark graphs already carry substantial information, thus suppressing the signal carried by the graph structure itself to some extent. This motivated us to prepare set of experiments in which we classify graphs solely based on graph structure. We achieve this by replacing all node labels and node features by random ones, thus leaving only structural/oppoloiica information for the classification task. In the case of the PATTERN dataset we use the original node features which are random by construction (Dwivedi et al. 2020p.
Table depicts the results for graph and node classification tasks on such graphs; we observe a clear advantage of TOGL over its comparison partners: making an existing GNN topology-aware via TOGL improves predictive performance in virtually all instances. In some cases, for instance when comparing GCN-4 to GCN-3-TOGL- on MNIST, the gains substantial with an increase of more than 8%. This also transfers to other model architectures, where in most cases TOGL improves the performance across datasets. Solely the GIN model on PROTEINS and the GAT model PATTERN decrease in performance when incorporating topological features. The deterioration of TOGL on PATTERNS is especially significant. Nevertheless, the inferior performance is in line with the low performance of GAT in general compared to the other methods we considered. In this context the lower performance of TOGL is not surprising as relies on the backbone model for the construction of a filtration. This demonstrates the utility of TOGL in making additional structural information available to improve classification performance.
7

---

Published as a conference paper at ICLR 2022
Table Results for the structure- bauee experiments. We depict the test accuracy obtained on various benchmark data sets when only considering structural information (i.e. the network has access to uninformative node features). For MOLHIV, the ROC-AUC is reported. Graph classification results are shown on the left, while node classification results are shown on the right. We compare three architectures (GCN-4, GIN-4, GAT-4) with corresponding models where one layer is replaced with TOGL and highlight the winner of each comparison in bold.
Graph classification
Node classificatio
METHOD
DD
ENZYMES
MNIST
PROTEINS
MOLHIV
PATTERN
GCN-4
68.0+3.6
22.0+3.3
76,2+0.5
68.842.8
66.4+1.8
85.5+0.4
GCN-3-TOGL-1
75.1+2.1
30.326.5
84.8+0.4
73.8+4.3
69,4+1.8
86.6+0.1
GIN-4 75.6+2.8
21.346.5
83,4=0,9
74.6#3.1
68.7=0.9
84.8+0.0
GIN-3-TOGI.-1
76.2+2.4
23.7+6.9
84.4=1.1
73.9+4.9
65.146.2
86.7+0.1
GAT-4
63.3+3.7
21.7+2.9
63.2+10.4
67.5+2.6
1.855.66
73.1+1.9
GAT-3-TOGL-1
75.7+2.1
23.5+6.1
77.2=10.5
72.424.6 68.6+1.7
59.6+3.3
Table 2: Test accuracy on benchmark data sets (following standard practice, we report weighted accuracy on CLUSTER and PATTERN). Methods printed black have been run in our setup, while methods printed in grey are cited from the literature, i.e. Dwivedi et al. (2020), Morris et al. (2020) for
IMDB-B B and REDDIT-B. and Borgwwardd et al. (2020) for WL/WL-OA results. Graph classification results are shown the left; node classification results are shown on the right. Following Table I we take existing architectures and replace their second layer by TOGL; we use italics to denote the
winner of each comparison. A bold value indicates the overall winner of column, i.c. data set.
Graph classtffcution
Node ciavsificution
METHOD
CIFAR-10
ENZYMES
MNIST PROTEINS-FILA
IMDB-R
REDDIT-R
CLUSTER
GATED-GCN-4
67.3+0.3
72.9+2.1
65.7+4,9
97.3+0.1
76.4+2.9
60.4
WL
77.7+2.0
54,3:0.9
73.1
71.2+0.5
78.0+0.6
WL-OA
77.8+1.2
58.9+0.9
73.5+0.9
74.0+0.7
87.620.3
GCN4
54.2+1.5
72.8+4.1
65,8+4.6
90.040.3
76.1
68.624.9
92.8+1.7
57.020.9
GON---TOGI-1
61.7+1.
73.2+4.7
53.0+9.2
95.5+0.2
76.0+3.9
72.0+2.3
89.4+2.2
60.4+0.2
GIN-4
54.8+1.4
70,8+3.8
10.0+12.396.11 140.3 43.847.9
72.3+3.3
72.8+2.5
81.726.9
58.520.1
GIN---TOGL-1
61.3+0.4
75.2+4.2
96.1.0.1
73,624.8
74.2+4.2
89.7+2.5
60.4+0.2
GAT4
57.420.6
71.143.1
26.8+4.1 140.3
71.345.4
73.2+4.1
44.246.6
56.620.4
GAT-3-TOGL-1
63.9+1.2
73.742.9
51.5+7.3
95.940.3
75.2+3.9
70.8+8.0
89.548.7
58.4+3.7
Varying the number of layers in this experiment un- derscores the benefits of including TOGL in a stan- dard GCN architecture. Specifically, an architec- ture that incorporates TOGL reaches high predic- tive performance earlier, i.e. with fewer layers, than
Text (in%)
76.0
GCN GCN TOG-
GIN
GAT
other methods, thus reducing the risk of oversmooth- ing (Chen et al.) 2020a).. Figurre 3 depicts this for MNIST; we observe similar effects on the other data sets. Appendix I presents detailed performance comparison ,add extended analysis, and a discussion of the effects of different choices for an embedding function V.
Number of Layars
Figure 3: Classification performance when analysing the structural variant of MNIST.
5.4 PERFORMANCE ON BENCHMARK DATA SETS
Having ascertained the utility of TOGL for classifying data sets with topological information, we now analyse the effects of TOGL in standard graph and node classification tasks. Table 2 depicts the results on well-known benchmark data sets for graph and node classification. We see that TOGL performs better than its comparison partners (i.e. GCN-4, GIN-4, GAT-4) most of the data sets, showcasing the benefits of substituting a layer of GNN with TOGL. Concerning ENZ YEE performance, experienced severe degree of overfitting during training. This exacerbated
8

---

Published as a conference paper at ICLR 2022
Table 3: Test accuracy when comparing TOGL (integrated into simplified architecture) with existing topology-basee based embedding functions or readout functions. Results shown in grey are cited from the respective papers (Carrière et al. 2020 Hofer et al. 2020). For GFL, we cite degree-based results so that its performance values pertain to the same scenario.
METHOD
REDDIT-SK
IMDB-MULT
NCII
REDDIT-B
IMDB B
GFL.
55.7+2.1
49.7+2.9
71.242.1 73.5
90.242.8
74,5=4.6
PersLay
55.6
48.8
71.2
GCN-1-TOGL-1 56.1+1.8
52.0=4.0
75.8+1.8
90.1+0.8
74,3+3.6
GCN-1-TOGL-1 (static) 58.5+1.8
48.3+4.9
75.1+1.2
90.4+1.4 72.2+2.1
by the fact that ENZYMES is the smallest of the compared data sets and we eschewed the tuning of regularisation hyperparameters such as 'dropout' for the sake of being comparable with the benchmark results (Dwivedi et al. 2020). With the GAT-based GNN, generally observe an high degree of variance, as already reported in previous studies (Dwivedi et al. 2020). In particular, we experienced issues in training it on the REDDIT-B B dataset. The addition of TOGL seems to address this issue, which we consider to underline the overall potential of topological features (at least for data sets with distinct topological information).
5.5 COMPARISON TO OTHER TOPOLOGY- GAAOLGGG--BSEDD ALGORITHMS
In light of TOGL containing existing embedding functions Y Carriere et al. 2020), compare its performance to other topology-baad algorithms that have been used for graph classification, [Tblle 3 summarises the performance (for comparison purposes, of our layer; see Apppeddix foror more details). In order to permit a fair comparison, we integrate TOGL into simpler GNN architecture, consisting of a single GCN message passing layer.
also show the results for 'static" variant
We observe that our filtration learning approach outperforms the fixed filtrations used by Carrière et al. (2020), highlighting the utility of layer specifically designed to be integrated into GNNs. TOGL also fares well in comparison to the r readout function by Hofer et al. (2020). While large standard deviations (due the small size of the data sets) preclude an assessment of significant differences. we hypothesise that the benefits of having access to multiple filtrations will be more pronounced for large data sets containing more pronounced topological features, such as molecular graphs. Notice that in contrast to Hofer et al. (2020), our layer can be included at different stages of the overal GNN architecture. We investigate the impact of different positions in Appendix H and show that different positions can lead significantly different performance. These results further motivate the importance of a flexible topological layer as opposed to readout functions or static filtrations.
CONCLUSION
We presented TOGL, a GNN architecture. We proved that TOGL, due to its filtration functions (i.e. input functions) being learnable, is more expressive than WL[[1] the Weisfeiler-Lehman test for graph isomorphism, and therefore also increases expressivity of GNNs. On data sets with pronounced topological structures, we found that our method helps GNNs obtain substantial gains in predictive performance. We also saw that the choice of function for embedding topological descriptors crucial, with embedding functionn that can handle interactions between individual topological features typically performing better than those that cannot. On benchmark data sets, we observed that our topology-awe awwre approach can help improve predictive performance while overall exhibiting favourable performance in comparison to the literature. We also observed that topological information may sometimes lead to overfitting issues on smaller data sets, and leave the investigation of additional regularisation strategies for our method for future work. Furthermore, we hypothesise that the use different filtration types (Milosavljević et al. 2011), together with advanced persistent homology algorithms, such extended persistence (Cohen Steiner et al. 2009), will prove beneficial for predictive performance.
layer that incorporates topological information into any

---

Published as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
We have provided the code for our experiments along with the seeds used for training. All experiments run single GPUs, which avoids additional sources of randomness. Details are provided in infSection 5.1] The parameters and performance metrics during training were tracked and will be made publicly available (Biewald, 2020), Our code is released under BSD-3-Clause License and can be accessed under https: /ttl//gitttuu gightb.thhuu.ggggg com www.///oowwaaa...oooooooooooo..e cowm//orgwardt.aab///O00
ACKNOWLEDGEMENTS
The authors would like to thank "Weights and Biases, Inc." for providing us with free academic team account. This project was supported by the grant #2017-110 of the Strategic Focal Area "Personalized Health and Related Technologies (PHRT)" of the ETH Domain for the SPHN/PHRT Driver Project "Personalized Swiss Sepsis Study" and the Alfried Krupp Prize for Young University Teachers of the Alfried Krupp von Bohlen und Halbach-Stifture (K.B.) Edward De Brouwer gratefully acknowledges support by FWO-SB grant and support from NVIDIA for GPUs. Yves Moreau is funded by (I) Research Council KU Leuven: C14/18/092 SymBioSys3; CELSA-HIDUCTION (2) Innovative Medicines Initiative: MELLODY, (3) Flemish Government (ELIXIR Belgium, IWT, FWO 06260)
and (4) Impulsfonds AI: VR 2019 2203 DOC.0318 QUQATER Kenniscentrum Data en Maatschappij. Some computational resources and services used in this work were provided by the VSC (Flemish Supercomputer Center), funded by the Research Foundation Flanders (FWO) and the Flemish Govermment department EWI
REFERENCES
L. Babai, P. Erdös, and S. M. Selkow, Random graph isomorphism. SIAM Journal Computing, (3):628-635, 1980.
S. A. Barannikov. The framed Morse complex and its invariants. Advances in Soviet Mathematics, 21:93-115, 1994.
Bendich, P. Bubenik, and A. Wagner. Stabilizing the unstable output of persistent homology computations. Journal of Applied and Computational Topology, 4(2):309-338, 2020.
L. Biewald. Experiment tracking with weights and biases, 2020. URL https:://wwtt //ww:///wwww//.. wandb com/
C. Bodnar, F. Frasca, Y. Wang, N. Otter, G. F. Montufar, P. Lió, and M. Bronstein. Weisfeiler and Lehman go topological: Message passing simplicial networks. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 1026-1037. PMLR, 2021.
K. Borgwardt, E. Ghisu, F. Llinares-López, L. O'Bay, and B. Rieck. Graph kernels: State-of-the-art and future challenges. Foundations and Trends@ in Machine Learning, (3(56))531-712, 2020.
G. Bouritsas, F. Frasca, S. Zafeiriou, and M. M. Bronstein. Improving graph neural network expressivity via subgraph isomorphism counting. arXiv preprint 2006.09252, 2021.
X. Bresson and T. Laurent. Residual gated graph ConvNets. arXiv preprint 1rXiv:1711.077 2017.
G. Carlsson, G. Singh, and A. Zomorodian. Computing multidimensional persistence. In International Symposium Algorithms and Computation (ISAAC), pages 730-739. Springer, 2009.
M. Carrière, F. Chazal, Y. Ike, T. Lacombe, M. Royer, and Y. Umeda. PersLay: A neural network layer for persistence diagrams and new graph topological signatures. In Proceedings of the 23rd International Conference Artificial Intelligence and Statistics (AISTATS), volume 108 of Proceedings of Machine Learning Research, pages 2786-2796 PMLR, 2020.
M. Carrière, F. Chazal, M. Glisse, Y. Ike, H. Kannan, and Y. Umeda. Optimizing persistent homology based functions. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 1294-1303, PMLR, 2021.
10

---

Published as a conference paper at ICLR 2022
N. J. Cavanna, M. Jahanseir, and D. R. Sheehy. Visualizing sparse filtrations. In Proceedings the 31 International Symposium on Computational Geometry, 2015a.
N. J. Cavanna, M. Jahanseir, and D. R. Sheehy, A geometric perspective sparse filtrations. In Proceedings of the Canadian Conference on Computational Geometry, 2015b.
D. Chen, Y. Lin, W. Li, P. Li, Zhou, and X. Sun. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04))3438-344
2020a.
Z. Chen, S. Villar, L. Chen, and J. Bruna. On the equivalence between graph isomorphism testing and function approximation with GNNs. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alché-Bucc E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
7. Chen, L. Chen, S. Villar, and J. Bruna. Can graph neural networks count substructures? In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 13383-10395 Curran Associates, Inc., 2020b.
D. Cohen-Steiner, Edelsbrunner, and J. Harer. Extending persistence using Poincaré and Lefschetz duality. Foundations of Computational Mathematics, 01):79-1033 2009.
V. P. Dwivedi, C. K. Joshi, T. Laurent, Bengio, and X. Bresson, Benchmarking graph neural networks. arXiv preprint arXiv:2003.00982, 2020.
H. Edelsbrunner and J. Harer. Computational topology: An introduction. American Mathematical Society, 2010.
H. Edelsbrunner, D. Letscher, and A. J. Zomorodian. Topological persistence and simplification. Discrete Computational Geomerry, 28(4):511-533, 2002.
W. Falcon et al. PyTorch Lightning, 2019.
URL https://github.com
M. Fey and J. E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learming on Graphs and Manifolds, 2019.
M. Fürer. On the combinatorial power of the Weisfeiler-Lehman algorithm. In Algorithms and Complexity, pages 260-271. 2prnner,,,,, 2017.
M. Gameiro, Y. Hiraoka, and I. Obayashi. Continuation of point clouds via persistence diagrams. Physica D: Nonlinear Phenomena, 334:1 132-132, 2016.
A. Hatcher. Algebraic topology. Cambridge University Press, 2002.
F. Hensel, M. Moor, and B. Rieck. A survey of topological machine learning methods. Frontiers in Artificial Intelligence, 2021.
C. Hofer, R. Kwitt, M. Niethammer, and A. Uhl. Deep learning with topological signatures. In Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30, pages 1634-1644, Curran Associates, Inc., 2017.
C. D. Hofer, R. Kwitt, and M. Niethammer. Learning representations of persistence barcodes. Journal of Machine Learning Research, 20(126): 145, 2019.
C. D. Hofer, F. Graf, B. Rieck, M. Niethammer, and R. Kwitt. Graph filtration learning. In Proceedings of the 37th International Conference on Machine Learning (ICML), volume 119 of Proceedings of Machine Learning Research, pages 4314 4323. PMLR, 2020.
T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference Learning Representations (ICLR), 2017.
11

---

Published as a conference paper at ICLR 2022
N. M. Kriege, P.-L. Giscard, and R. Wilson. On valid optimal assignment kernels and applications to graph classification. In D. Lee, M. Sugiyama, U. Luxburg. I. Guyon, and R. Gamnett, edi- tors, Advances in Neural Information Processing Systems, volume 29, pages 1623-1631. Curran Associates, Inc., 2016.
N. M. Kriege, F. D. Johansson, and C. Morris. A survey graph kernels. Applied Network Science, 5(1); 2020.
H. Maron, H. Ben-Hamu, H. Serviansky, and Y. Lipman. Provably powerful graph networks. In H. Wallach, H. Larochelle, A. Beygelzimer, d' A'Ahhé--BBu Buc, E. Fox, and R. Garnett, editors, Ad- vances in Neural Information Processing Systems, volume 32, pages Inc., 2019.
Curran Associates,
B. D. McKay and E. Spence. Classification of regular two-graphs on 36 and 38 vertices. Australasian Journal of Combinatorics, 24:293-300, 2001.
N. Milosavljević, D. Morozov, and Skraba. Zigzag persistent homology in matrix multiplication time. In Proceedings of the 27th Annual Symposium Computational Geometry (SoCG), pages 216-225, 2011.
M. Moor, M. Horn, B. Rieck, and K. Borgwardt. Topological autoencoders. In Proceedings of the 37th International Conference on Machine Learning (ICML), volume 119 of Proceedings of Machine Learning Research, pages 7045-7054, PMLR, 2020.
C. Morris, M. Ritzert, M. Fey. W. L. Hamilton, J. E. Lenssen, G. Rattan, and M. Grohe. Weisfeiler and Leman neural: Higher-orded graph neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):4602-4669, 2019.
C. Morris, N. M. Kriege, F. Bause, K. Kersting, P. Mutzel, and M. Neumann, TUDataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663 2020.
G. Nikolentzos, G. Siglidis, and M. Vazirgiannis. Graph kernels: survey, arXiv preprint arXiv: 122188 art. 1uXiv:1904.12113 2019.
A. Poulenard, P. Skraba, and M. Ovsjanikov. Topological function optimization for continuous shape matching. Computer Graphics Forum, 37( 2018.
B. Rieck, C. Bock, and K. Borgwardt. A persistent Weisfeiler-Lehman procedure for graph classifi- cation. In Proceedings of the 36th International Conference on Machine Learning, volume 97, pages 5448-5458. PMLR, 2019.
B. Sanchez-Lengeling, E. Reif, A. Pearce, and A. B. Wiltschko. A gentle introduction to graph neural networks. Distill, 6(9)ee33, 2021.
D. R. Sheehy. Linear-size approximations to the Vietoris-Rips Rips filtration. Discrete & Computational Geometry, 49(4):778-796, 2013.
N. Shervashidze and K. Borgwardt. Fast subtree kernels graphs. In Y. Bengio, D. Schuurmans,
J. Lafferty, C. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 22, pages 1660-1668. Curran Associates Inc., 2009.
N. Swenson, A. S. Krishnapriyan. A. Buluc, D. Morozov, and K. Yelick. PersGNN: Applying
topological data analysis and geometric deep learning structure-based protein function prediction. arXiv preprint a0 /6027, 2020.
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
A. Verri, C. Uras, P. Frosini, and M. Ferri. On the use of size functions for shape analysis. Biological Cybernetics, 70(2)99--107 1993.
B. Weisfeiler and A. A. Lehman. The reduction of a graph to canonical form and the algebra which appears therein.
Informatsia, 9:12-16, 1968.
12

---

Published as a conference paper at ICLR 2022
C.-C. Wong and C.-M. Vong. Persistent homology based graph convolution network for fine- fine-grained 3d shape segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7098-7101 2021.
Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu. comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1):4-24 2021.
K. Xu, W. Hu, J. Leskovec, and S. Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2019.
M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. Smola. Deep sets. In Advances in Neural Information Processing Systems, volume 30, pages 3391-3401. Curran Associates, Inc., 2017.
Q. Zhao and Y. Wang. Learning metrics for persistence-basee summaries and applications for graph classification. In Advances in Neural Information Processing Systems 32 (NeurIPS), pages 9855-9866. Curran Associates, Inc., 2019.
Q Zhao, Z. Ye, C. Chen, and Y. Wang. Persistence enhanced graph neural network. In Proceedings of the 23rd International Conference on Artificill Intelligence and Statistics (AISTATS), volume 108 of Proceedings of Machine Learning Research, pages 2896-2906. PMLR, 2020.
A. J. Zomorodian. Fast construction of the Vietoris-Rie Rips complex. Computers & Graphics, 34(3): 263-271, 2010.
13

---

Published as a conference paper at ICLR 2022
A TOPOLOGICAL DATA ANALYSIS
We provide more formal introduction to persistent homology, the technique on which TOGL is fundamentally based. Persistent homology arose as one of the flagship approaches in the field of computational topology, which aims to make methods from this highly abstract branch of mathematics available for data analysis purposes.
To gain a better understanding, will briefly take a panoramic tour through algebraic topology, starting from simplicial homology, an algebraic technique for 'calculating' the connectivity of topological spaces, represented in the form of simplicial complexes, i.e. generalised graphs. Simplicial homology is said to assess the connectivity of topological space by 'counting its high-dimensional holes'. We will see how make this description more precise.
A.I SIMPLICIAL HOMOLOGY
Simplicial complexes are the central concept in algebraic topology. A simplicial complex K consists of set of simplices of certain dimensions, such as vertices (dimension 0), edges (dimension 1), and triangles (dimension 2). Each simplex o E K has a set of faces, and each face T has to satisfy T E K. Moreover, if a * for a, o' E K, then n K. Thus, K is 'closed under calculating the faces of a simplex'. graph can be seen as a low-dimensional simplicial complex that only contains O-simplices (vertices) and 1-simplices (edges), everything we say applies, mutatis mutandis, also to graphs.
Chain groups. For a simplicial complex K, we denote by Ca(K) the vector space generated over Zn (the field with two elements). The elements of Ca(K) are the d-simplices in K, or rather their formal sums with coefficients in Zg. For example, + T is an element of the chain group, also called a simplicial chain. Addition is well-defined and easy to implement as an algorithm since a simplex can only be present or absent over Zo coefficients The use of chain groups lies in providing the underlying vector space to formalise boundary calculations over a simplicial complex. The boundary calculations, in turn, necessary to quantify the connectivity!
Boundary homomorphism. Given d-simplex (Va...., .ua) E K, we can formalise its face or 'boundary' calculation by defining the boundary operator 0g: Ca(K) - Ca-1(K) as a sum of the form
Oa(a)=
co. U4 0/41, va).
(I)
Le. we leave out every vertex v of the simplex once. Since only sums are involved, this operator is seen to be homomorphism between the chain groups; the caleulation extends to Ca(K) by linearity. The boundary homomorphism gives us a way to precisely define what we understand by connectivity. To this end, note that its kernel and image are well-defined. The kernel ker Oa contains all d-dimensional simplicial chains that do not have a boundary. We can make this more precise by using a construction from group theory.
Homology groups. The last ingredient for the connectivity analysis involves calculating a special group. the homology group. The dth homology group Ha(K) of K is defined as the quotient group Ha(K) == ker Oa/ im 0.+1- The quotient operation can be thought of as calculating a special subset- the kernel of the boundary operator-ant and then removing another subset, namely the image of the boundary operator with an increased dimension. This behoves short explanation. The main reason behind this operation is that just the kernel calculation is insufficient to properly count a hole. For example, if we take the three edges of any triangle, their boundary will always be empty, i.c. they are part of ker Øz However, if the interior of the triangle is also part of the simplicial complex- in other words, if we have the corresponding 2-simplex as well-we we should not count the edges as hole. This is why we need remove all elements in the image of O2. Coincidentally, this also explains why cycles in graph never "vanish" -heree are simply no 2-simplices available since the graph only a 1-dimensional simplicial complex.
Different choices of coefficient fields would he possible, but are rarely used for data analysis purposes.
14

---

Published as a conference paper at ICLR 2022
Betti numbers. To fully close the loop. it were, it turns out that can calculate Betti numbers from homology groups. Specifically, the rank of the dth homology group in the group-theorettial sense-is is precisely the dth Betti number Sa, i.c. Ba(K) rank HK(K). The sequence of Betti numbers Bo....; Ba of a d-dimensional simplicial complex is commonly used as a complexity measure, and they he used to discriminate manifolds. For example, 2-sphere has Betti numbers (1, 0, 1). while 2-torus has Betti numbers (1, 2, 1). As we outlined in the main text, Betti numbers of limited use when dealing with complex graphs, however, because they are very coarse counts of features. It was this limited expressivity that prompted the development of persistent homology.
A.2 PERSISTENT HOMOLOGY
Persistent homology is an extension of simplicial homology, which employs filtrations to imbue simplicial complex K with scale information. Let us assume the existence of function f: K R. which only attains a finite number of function values f(l) < f()) S ... f(m-1) 1) < f f(m), We
may now, as in the main text, sort K according to f. leading again to nested sequence of simplicial complexes
= (()) C (K() C ... k(m-1) C K(m) K, E in which K(0 (o function value less than or equal to the threshold. contrast to simplicial homology, the filtration holds potentially more information because it can track changes! For example, a topological feature might be created (a new connected component might arise) or destroyed (two connected components might merge into one), as we go from some KO) K(++1), At its core, persistent homology is "just' way of tracking topological features, representing each one by a creation and destruction value (f(1) f()) R², with i S j. In case a topological feature is still present in K(m) = K, such as a cycle in a graph, it can also be assigned tuple of the form (f(1), 0o). Such tuples constitute essential features simplicial complex and are usually assigned large destruction value or treated separately in an algorithm (Hofer et al. (017). While it is also possible to obtain only tuples with finite persistence values, a process known as extended persistence "Cohen-Steines et al. 22009), we focus only on 'ordinary' persistence in this paper because of the lower computational complexity.
S ( I(o) i.e. each subset contains only those simplices whose
Persistent homology groups. The filtration above is connected by the inclusion homomorphism between K(t) C K(++)) The respective boundary homomorphisms induce a homomorphism between corresponding homology groups of the filtration, i.e. map id of homomorphisms gives rise to a sequence of homology groups
(i.)), Ha(K,) Ha(K)). This family
Ha (K(O)
(201))
Ha (K(2)
2m
(3)
(m (mm-1m))
Ha ( (k(m-l))
Hd (K(m)) = Ha (K)
for every dimension d. For i S j. the dth persistent homology group is defined as
H == ker 0d (K(0) /im (K()) O ker Oa (K())
(4)
Intuitively, this group contains all homology classes created in K( that are also present in K() Similar to the "ordinary' Betti number, we may now define the dth persistent Betti number as the rank of this group, i.e.
3 == rank H((:.)
Noting the set of indices i.j. we can see persistent homology as generating a sequence of Betti numbers, opposed to just calculating single number. This makes it possible for us to describe topological features in more detail, and summarise them in a persistence diagram.
Persistence diagrams. Given a filtration induced by a function f: - IR as described above, we store each tuple (r(),()) with multiplicity
(d) M4 = (8 -g -3(4,)
(3( (101-11111
- ((1111))
(6)
15

---

Published as a conference paper at ICLR 2022
in the dth persistence diagram Da- Notice that for most pairs of indices, Pij 0. Given a point
(d)
(z, y) Da. we refer the quantity pers(r.)) y) := ly as its persistence. The idea of persistence arose in multiple contexts (Barannikov] 1994; Edelsbrunner et al., 2002 Verri et al. 1993), but it nowadays commonly used to analyse functions on manifolds, where high persistence is seen to correspond to features of the function, while low persistence is typically considered noise.
A.3 EXAMPLES OF GRAPH FILTRATIONS
Before discussing the computational complexity of persistent homology, we briefly provide some worked examples that highlight the impact of choosing different filtrations for analysing graphs. Filtrations are most conveniently thought of as arising from a function f: G - R, which assigns vallud function value to each node and edge of the graph by means of setting f(u, v) := (()x, ](w)] for an edge (u, v). In this context, f is often picked to measure certain salient vertex features of G, such as the degree (Hofer al. 2017), or its structural role in terms of heat kernel (Carrière et al. 2020). The resulting topological features are then assigned the respective function values, i.c. (i. j)+ + (f(u,), (u,)). As a brief example, consider a degree-based filtration of simple graph. The filtration values are shown as numbers next a vertex; we use a to denote the filtration value at step (this notation will be useful later when dealing with multiple filtrations). In each filtration step, the new nodes and edges added to the graph are shown in red, while black elements indicate the structure of the graph that already exists at this step.
3
G
G(2)
G
Since all edges are inserted at (3) = 3. we obtain the following O-dimensional persistence diagram Do = {(1, x)), (1,3), (2, 3), (3,3), (3, The existence of single tuple the form (: indi- cates that 3o = 1. Similarly, there is only one cycle in the data set, which is created at a ( leading to D, {(3, } and 31 1. If we change the filtration such that each vertex has unique filtration value, we obtain a different ordering and different persistence tuples, as well as more filtration steps:
3
(1)
(
(9)
((4)
G
Here, connected components are not all created 'at once', but more gradually, leading to Do- {(1, oo), (3,3), (2, 4), 4), (5, 5)}. Of particular interest is the tuple (2, 4). It was created by the vertex with filtration value 2. In G(3) it merges with another connected component, namely the one created by the vertex with function value 3. This leads to the tuple (3.3). because in each merge, we merge from the 'younger' component (the one arising later the filtration) to the 'older' component (the arising earlier in the filtration), Again, there is only single cycle in the data set, which we detect at (6) 5, leading to Dj ((5, oo)].
A.4 COMPUTATIONAL DETAILS
The cardinality of the persistence diagram of dimension-O, Do is equal to the number of vertices, n in the graph. A natural pairing of persistence tuples then consists in assigning each tuple to the node that generated it. As for dimension-I. 1, D2 contains as many tuples as cycles in the graph. However, there is bijective mapping between dimension-1 1 persistence tuples and vertices. Rather, we link each dimension-1 1 tuple to the edge that created that particular cycle. To account for multiple distinct filtrations and because the same cycle can be assigned to different edges depending on the specific filtration function, we define a dummy tuple for edges that are not linked to any cycle for particular
filtration. The set of persistence diagrams (D(") ..... D(2) can then be concatenated as a matrix in R with di [V| (the number of vertices in graph G) if [ 0 and dj |F] (the number of
16

---

Published as a conference paper at ICLR 2022
edges in / = 1. Remarkably, that leads to w (Df)..... ".... D(")
being an operator on a matrix,
significantly facilitating computations.
Because of its natural indexing to the vertices, u (0) (D( "D......, D( can be mapped back to the graph as explained in Section 4. For I = 1, we pool ( (D(") (( to graph-level
embedding, and mask out the edge indices that are not assigned a persistence pair in any of the D
B THEORETICAL EXPRESSIVITY OF TOGL: PROOFS
This section provides more details concerning the proofs about the expressivity of our method. We first state a 'weak' variant of our theorem, in which injectivity of the filtration function is not guaranteed. Next, we show that the filtration function f constructed in this theorem can be used to prove the existence of an injective function f that is arbitrarily close to f and provides the same capabilities of distinguishing graphs.
Theorem 3 (Expressivity, weak version). Persistent homology is at least as expressive as WL/1], i.e. if the WL[1] label sequences for two graphs G and G' diverge, there is a filtration f such that the corresponding O-dimensional persistence diagrams Do and D/2 are not equal.
Proof. Assume that the label sequences of G and G' diverge iteration h. Thus, OG + o and
(b)
(N)
there exists at least one label whose count is different. Let (th) := {4, 12.... be an enumeration of the finitely many hashed labels at iteration h. We can build filtration function f by assigning a vertex y with label I, to its index, i.e. fv) := i, and setting fv. w) == max {f()), fww] for an edge (u, w). The resulting O-dimensional persistence diagram Do (and Dú for G') will contain tuples of the form (i, and each vertex is guaranteed to give rise to exactly one such pair. Letting
(4d) (Do) refer to the multiplicity of tuple in Do. we know that, since the label count is different, Mo
there is at least one tuple (k, l) with Mo
(k..) (D)). Hence, Do * D/-- (I..) (Do) # Ho
While Theorem 3 proves the existence of filtration, the proof is constructive and relies on the existence of the WL[1] labels. Moreover, the resulting filtration is typically not injective, thus precluding the applicability of Theorem The following Lemma discusses potential fix for filtration functions of arbitrary dimensionality; our context is special case of this.
Lemma 1. For all E > 0 and V + Rd there is an injective function f such that |lf If- <e
Proof. Let = = {UI..... J the vertices of a graph and im f {ur,.... um C Rd their images under f. Since f is not injective, we have < n. We resolve non-injective vertex pairs iteratively. For u E im let V : {u E V f(w) = w). If V only contains a single element, we do not have to do anything. Otherwise, for each v' E V pick new value from B, (u) \ im where (x) C Rd refers to the open ball of radius r around a point æ (for our case, i.c d = 1, this becomes an open interval in R). Since we only ever remove finite number of points, such new value always exists, and we can modify im I accordingly. The number of vertex pairs for which f is non-injective decreases by at least one in every iteration, hence after finite number of iterations, we have modified f to obtain f. an injective approximation f. By always picking new values from balls of radius e, we ensure that |lf flloo e, as required.
Using this Lemma, we can ensure that the function f in Theorem 3 is injective, thus ensuring differentiability according to [hhooemm I] However, we need to make sure that the main message of Theorem 2 still holds, i.e. the two graphs G and G' must lead to different persistence diagrams even under the injective filtration function f. This is addressed by the following lemma, which essentially states that moving from ff to injective f does not result in coinciding persistence diagrams.
Lemma 2. Let G and G' be two graphs whose O-dimensional persistence diagrams Dy and D/ are calculated using a filtration function f as described in Theorem 3) Moreover, given > 0, let î be an injective filtration function with |f flloo and corresponding O-dimensional persistence diagrams Do and D/- Do + D/y, we also have Do # Do=
17

---

Published as a conference paper at ICLR 2022
Proof. Since f is injective, each tuple in Do and D/ has multiplicity 1. But under f. there were differences in multiplicity for at least one tuple (k: I). Hence, given f. there exists at least one tuple () Do U D/ with (ks 1) & Do n D. As a consequence, Do # D/--
For readers familiar with TDA, this lemma can also be proved in a simpler way by noting that if the bottleneck distance between Dn and D/ is non-zero, it will remain if one picks perturbation that is sufficiently small (this fact can also be proved more rigorously). In any case, the preceding proofs enable us finally state a stronger variant of the expressivity theorem, initially stated as Theorem 2 in the main paper. For consistency reasons with the nomenclature in this section, we slightly renamed the filtration function.
Theorem 4 (Expressivity, strong version). Persistent homology is at least expressive as WL[1], i.e. if the WL[1] label sequences for two graphs G and G' diverge, there exists an injective filtration f such that the corresponding O-dimensional persistence diagrams Do and D/ are not equal.
Proof. We obtain non-injective filtration function f from Theorem 3] under which Do and D/
not equal. By Lemma for every e > 0, we can find an injective function f with ||f flloo e. According to Lemma 2. the persistence diagrams calculated by this function do not coincide, i.c. Do + Di- Hence, f is filtration function and, according to Theorem differentiability is ensured.
C EMPIRICAL EXPRESSIVITY, ANALYSIS OF (STRONGLY) REGULAR GRAPHS
k-regular graph is a graph G = -V, F:) in which all vertices have degree k. For h: = (3, 4), such graphs are also known cubic and quartic graphs, respectively. The Weisfeiler-Lehman test is capable of distinguishing between certain variants of these graphs (even though we observe that WL[]] is not sufficient do so). Similarly, a strongly regular graph is graph G (V, E) with two integers A, f IN such that each pair of adjacent vertices has À common neighbours, whereas every pair of non- adjaeennnntt vertices has μ common neighbours.
Persistent homology can make use of higher-order connectivity information to distinguish between these data sets. To demonstrate this, we use standard degree filtration and compute persistent homology of the graph, including all higher-order cliques. We then calculate the total persistence of each persistence diagram D, and use it to assign feature vector to the graph. This is in some sense the simplest way of employing persistent homology; notice that we are not learning new filtration but keep a fixed one. Even in this scenario, we find that there is always a significant number of pairs of graphs whose feature vectors do not coincide- or, conversely speaking, as Table S4 shows, there are only between 14%-22% of pairs of graphs that we cannot distinguish by this simple scheme. This illustrates the general expressivity that topology-based perspective can yield. For the strongly-regular graphs, we observe even lower error rates: we only fail to distinguish about 1.2% of all pairs (specifically, 8236 out of 7424731 pairs) of the 3854 strongly graphs on 35 vertices with x= = μ = 9 (McKay and Spence 2001..
the same time, this approach also not without its disadvantages. Since we perform a clique counting operation, the complexity of the calculation in- creases considerably, and we would not suggest to use persistent homology of arbitrary dimensions in practice. While there are some algorithmic improvements in topological constructions (Zomorodian 2010), naive persistent homology calculations of arbitrary order may quickly become inleasible for larger data sets. This can be alleviated to certain extent by approximating persistent homology (Ca- vanna et al.] 20155 b) [Sheehy] 2013). but the practical benefits of this are unclear. Nevertheless, this experiment should therefore be considered an indication of the utility of topological features in general to complement and enhance existing architectures.
note on computational complexity.
D EXPERIMENTAL SETUP & COMPUTATIONAL RESOURCES
Following the setup of Dwivedi et al. 22200) we implemented the following training procedure: All models are initialised with an initial learning rate Irint, which varies between the data sets. During
18

---

Published as a conference paper at ICLR 2022
Table S4: Error rates when using persistent homology with degree filtration to classify pairs of k-regular on n vertices. R3-N12 denotes 3-regular graphs on 12 vertices, for instance. This list is by no means exhaustive, but indicates the general utility of persistent homology and its iltration-based analysis.
Data set
Graphs
Pairs
Error
Error rate
R3-N12 12
85
3570
712
19.94%
R3-N14
509
129286
26745
20.69%
R3-N16
4060
8239770
1757385
21.33%
R4-N10
59
1711
229
13.38%
R4-N11
265
34980
4832
13.81% %
R4-N12
1544
1191196
170814
14.34%
Table S5: Parameters of learning rate scheduling (including 'patience' parameters) for training of models in this work.
NAME
MNIST, CIFAR-10, PATTERNS, PROTEINS, ENZYMES, DD CLUSTER
Urinst
1 10-3
7 10
Irmin
1 10-5
1 x 10-6
Iy pattience
10
25
training the loss the validation split is monitored and the learning rate is halved if the validation loss does not improve over a period of pattennce-- Runs are stopped when the learning rate gets reduced to value of lower than Irmin. The parameters for the different data sets shown in Table S5
Our method is implemented in Python, making heavy use of the gytorch-geometric li- brary (Fey and Lenssen 2019), licensed under the MIT License, and the pyytorhh-lgghtnin
library (Falcon al.] 2019)). licensed under the Apache 2.0 L. License. The training and hyperparameter selection was performed using "Weights and Biases' (Biewald 2020), resulting in additional reports, tables, and log information, which will simplify the reprodueibility of this work. We will make our own code available, using either the MIT License or a BSD 3-Clause license, which precludes endorsing/promotiin our work without prior approval. All licenses used for the code are compatible with this licensing choice.
As for the data sets, we use data sets that are available in Seltsorcc-gemmette for graph learning tasks. Some of the benchmark data sets have been originally provided by Morris et al. (2020). others (CIFAR-10. 10, CLUSTER, MNIST, PATTERN) have been provided by Dwivedi et al. (2020) in the context of large-scale graph neural network benchmarking effort.
Ablated static variant of TOGL. Next to all the experiments presented in the main paper, we also developed a static variant of our layer, serving as an additional ablation method (this nomenclature will be used in all supplementary tables). The static variant mimics the calculation of filtration terms of the number of parameters but without taking topological features into account. The based on persistent homology) of vertices
layer uses static mapping (instead of dynamic to themselves (for dimension 0), and employs a random edge selection process (for dimension 1). This has the effect of learning graph-level information that is not strietly based topology but node feature values. The static variant of TOGL reduces to an MLP that is applied per vertex in case interactions between tuples are not considered, and to the application of DeepSet Zaheer et al.) (2017) to the vertex representations interactions between tuples are incorporated. Generally, if the static variant performs well data set, assume that performance is driven much more by the availability of any graph-level type of information, such as the existence of certain nodes or groups of nodes, as opposed to topological information.
Compute resources. Most of the jobs were cores (Intel (R) Xeon (R) CPU B5-2620 v4 2. 20CHC)2) with 8 GeForce GTX 1080 GPUs. A smaller fraction has been another cluster, containing 40 physical cores (Intel (R)
internal cluster, comprising 64 physical
19

---

Published as a conference paper at ICLR 2022
Table S6: The hyperparameters that we use to train TOGL, along with their respective value ranges. Notice that "dropout" could be made configurable, but this would make our setup incomparable the setup proposed by Dwivedi al. 2020) for benchmarking GNNs.
NAME
VALUE(S)
DeepSet
(True, False]
Depth
{3,4] 4]
Diml
True (by default, we always use eycle information)
Dropout
0.
Early Topo
(True, False)
Static
(True, False} (to
variant)
Filtration Hidden Dimension
32
Hidden Dimension
138-146
No. coordinate functions
3
No. filtrations
8
Residual and Batch Norm
True True
Share filtration parameters
Xeon (R)
CPU E5-2630L. v4 @ 1.80GHz) with 2 Quadro GV100 GPUs and 1 Titan XP
GPU.
E HYPERPARAMETERS
Table contains a listing of all hyperparameters used to train TOGL. For the Weisfeller-Lehman subtree kernel, which we employed as comparison partner on the synthetic data sets, we used support vector machine classifier with linear kernel, whose regularisation parameter C' {10-1,10 10-3 ..... 10*) } is trained via 5-fold cross validation, which is repeated 10 times to obtain standard deviations. This follows closely the setup in graph classification literature.
F EXTENDED RESULTS FOR THE SYNTHETIC DATA SETS
In the main paper, we depicted concise analysis of synthetic data sets (Figure 1 Here, we provide more detailed ablation of this data set, highlighting the differences between TOGL and its static baseline, but also disentangling the performance of O-dimensional and 1-dimensional topological features, i.e. connected components and cycles, respectively. Figure S4 depicts the results. As stated in the main paper. we observe that (i) both cycles and connected components are crucial for reaching high predictive performance, and (ii) the static variant is performing slightly better than the standard GCN, but significantly worse than TOGL, due to its very limited access to topological information.
G VISUALISATION OF THE LEARNT FILTRATIONS
TOGL can learn arbitrary number of filtrations on the graphs. In Figure S7 display 3 different filtrations learnt on an randomly picked graph from the DD data set and compare it against the classical node degree filtration. The width of the nodes is plotted proportional to the node degree filtration while the colour saturation proportional the learnt filtration. Filtration appears to be correlated with the degree filtration, while other filtrations learnt by TOGL seem to focus on different local properties the graph.
H EXPERIMENTS ON LAYER PLACEMENT
A priori, it is not clear where place TOGL. We therefore investigated the impact of putting TOGL first (thus imbuing the graph with topological features arising from the neighbours of nodes) or at another position in the network. In Table Table S7 investigate the performance of the layer when TOGL placed at different positions GNN architecture composed 3 graph convolution layers. It appears that for this dataset, positioning TOGL before the last GNN layer leads best performance
20

---

Published as a conference paper at ICLR 2022
100.0
Test accuracy (in %)
75.0
50.0
25.0
TOGL (00L
WistderLaiman TOOL Rund
0.0
Number of GCN layers
(a) Performance CYCLES
Test accuracy (in %
100.0
75.0
50.0
25.0
T00I 18l. Oed 1iel
Weidde Telenr
Wil..pes Bimil)
THNE Siak des Orwg)
0.0
2
3
4
5
Number of GCN layers
(b) Performance for NECKLACES
Figure S4: Performance comparison on synthetic data as function of the number of GCN layers or Weisfeiler-Lehmee iterations. This is an extended version of Figure For TOGL, we show the performance with respect to the dimensionality of topological features that are being used. Since the standard deviations are negligible, we refrain from showing them here.
(a) DD
(b) ENZYMES
(c) PROTEINS
Figure S5: Example graphs of the benchmark data sets that describe molecular structures. These graphs give rise to complex topological structures that can be exploited.
21

---

Published as a conference paper at ICLR 2022
(a) IMDE-BINARY
(b) REDDIT-BINARY
Figure S6: Example graphs of the benchmark data sets that correspond extracted social networks. These graphs are best described in terms clique connectivity; the existence single 'hub' node does not give rise to complex topological structures that can be exploited a priori.
(a) Filtration
(b) Filtration
(c) Filtration
Figure S7: Examples of different filtrations jointly learnt example graph randomly picked from the DD data set. The width each node dot proportional to node degree, the colour saturation is proportional to the filtration value.
22

---

Published as a conference paper at ICLR 2022
Table S7: Test accuracies for different layer positions. TOGL can be placed at different positions. When placed at position 0, all downstream layers incorporate topological information, but only as much can be gleaned from the nodes of graph. By contrast, placing TOGL last makes readout function that incorporates information from multiple filtrations.
LAYER POSITION
METHOD
NECKLACES
3
GCN-3-TOGL-1 (no interaction)
97.17 + 0.4 94.5 + 2.9
GCN-3-TOGL-I (static interaction)
2
GCN-3-TOGL-1 (no interaction)
98.8 + 0.8
GCN-3-TOGL-1 (static no interaction)
77.2 + 4.5
GCN-3-TOGL-1 (no interaction)
97.4 + 1.8
GCN-3-TOGL-1 (static no interaction)
66.3 5.8
0
GCN-3-TOGL-I (no interaction) GCN-3-TOGL-1 (static interaction)
98.6 + 2.1
61.8 + 5.3
Table S8:
Test accuracies for different layer positions. TOGL can be placed at different positions. When placed position 0, all downstream layers incorporate topological information. but only as much as can be gleaned from the nodes of graph. By contrast, placing TOGL last makes a readout function that incorporates information from multiple filtrations.
LAYER METпоо
IMDB-BINARY
REDDIT-BINARY
PROTEINS-FULL
GCOLL-TOGL..
(interaction)
74.9 1 4.0
1..
74.9
1.39
71.5 1 4.5
GON-1-TOGI..
(no interaction)
75.9
GCN-3-TOGL--
7.4
9..
75.6=40
72.3 4.6
GON-1-TOGI.. (static interaction)
74.2
91.9++1.8
75.2
GCN-3-TOGL-- (static interaction)
73.8
89.4
76.8
70.9 2.6
GCN-3-TOGL-1
(static)
71.2
923
75.2
70.3 = 5.0
GCN-3 (interaction)
70.6 5.6
92.2
75.5 3.1
73.0 4 2.8
GON-3-TOGL-I (no interaction)
76.1
90.4 1.6
76.7 3.1
75.5
GCN-3-TOGL-I
7.7
3.1
OCN-3-10GL-1 (static interaction)
72.0
12.9 1.2
7..0
71.
GCN-3-TOGL-I (static interaction)
71.0
3.8
0CN-3-10GL-1 (static)
73.4 6.2
12.2 1.3
2.7
7L6 4.5
GCN-3-TOGL-1 (interaction)
73.2
91. 0.5
2.9
74.2 2.7
GCN-3-TOGL-1 (no interaction)
74.9 2.3
87.6
76.6 4.3
74.7 + 4.3
GCN-3-T0GL-1
71. 1.9
89.6 2.2
75.1
74.9 3.5
GCN-3-TOGL:1 (static interaction)
72.2 4.4
91.6 2.9
76.01... 4.1
72.7 3.9
GCN-3-T0GL-1 (static interaction)
72.1
87.6 4.1
74.1 4.1
70.7 3.4
GCN-3-TOGL-1 (static)
73.6 1 4.3
92.40.0
74.5 3.8
71.5 4.1
GCN-3-10GL-1 (interaction)
68.6 4.3
91.1 0.7
75.7 2.6
74.1
GCN-3-TOGL-I interaction)
71.
OCN-3-10GL-1
70.6 1
80.9
1.6
75.2 - 3.9
74.7 3.0
3-NGL
(static interaction)
73.4
90.5
76.3
73
CCN-3-TOGL-1
(static no interaction)
72.4 3.1
80.6
1.7
75.2
3.4
72.
3.8
3-G...
(static)
when the non-static version is considered. Importantly, this contrasts with the approach of Hofer et al.) (2019), where topological information is only available at the read-out level, which would lead to the worst performance on this dataset.
We complemented our experiments on layer placement with an exhaustive assessment of the perfor- mance of our model on the IMDB-Bing Binary, Reddit- Biniry, Proteins and DD datasets. Please refer to Table S8 for these results.
I EXTENDED EXPERIMENTS FOR STRUCTURED-BAST GRAPH CLASSIFICATION
This section contains more results for the structured based classification of graph data sets shown in Section 5.3 and Figure 3, Table Table contains detailed listing of performance values under
23

---

Published as a conference paper at ICLR 2022
Test accuracy (in %
80.0
Test accuracy (in %
40.0
75.0
30.0
70.0
20.0
65.0
60.-
10.0
GCN
TOGL (otaaaz
TOGL (unk)
TOGL (beenaate
FOGL DeERMIZE
TOGL
0.0
2
Number GCN layers
Number of GCN layers
(a) Performance for CT-STRUCT
(b) Performance for SANZYMES-STIEE STRUCT
Test accuracy (in %)
80.0
Test aceuracy (in %)
17.0
75.0
70.0
16.5
65.0
OCN
TOGL (acturt)
TOGLL HotK
OC
TOCL DocLateeeeee
TOGL metamannn
TOGL (1X))
60.0
16.0
2
3
1
Number of GCN layers
Number of GCN layers
(c) Performance for PROTEINS SURUCTT
(d) Performance for CLUSTER STRUCT
Test accuracy (in %)
87.0
Test accuracy (in %)
75.0
85.0
65.0
55.0
(16))
45.
500L Garaten (ecitmatio)) (168))
83.0
2
3
2
3 4
5
Number of GCN layers
Number of GCN layers
(e) Performance for PATTERN-STRUCT
(l) Performance for MNIST-STRUCT
Figure S8: Comparison of test accuracy for the structure-based variants of the benchmark data sets while varying network depth, i.e. the number of GCN layers. Error bars denote the standard deviation of test accuracy over 10 cross-vale volssavaiioiionn folds. Interaction refers to using DeepSet approach for embedding persistence diagrams, whereas interaction uses persistence diagrams coordinate functions that do not account for pairwise interactions.
certain additional ablation scenarios, such as disentangled comparison of the performance with respect to picking a certain type of embedding scheme for topological features, i.e. deep sets versus known embedding functions for persistence diagrams, the latter of which cannot handle interactions diagram. The version labelled General consist of non-static versions where the type of feature embedding scheme is considered as hyper-parameter. We then report the test accuracy of the version whose validation accuracy was highest. The General -Static variant is similar but for the static version of TOGL, Figure S8 depicts the performance on different data sets, focusing on a comparison between the different embedding types and the static variant of our layer.
between tuples
The behaviour of our static variant indicates that predictive performance is not driven by having access to topological information on the benchmark data sets. To some extent, this is surprising, as some of the data sets turn out to contain salient topological features even visual inspection (Figure S5). whereas the social networks exhibit more community-lice behaviour (Figure S6). Togetherr with the results from [Section 5.3) this demonstrates that existing data sets often "leak' structural information through their labels/features. Yet, for the node classification data sets CLUSTERR and PATTERN, GCN-3-TOGL-I or its static variant perform best among all comparison partners
*On these data sets, our results GCN-4 differ slightly due to known misalignment in the implementation Dwivedi 2220). as compared the original GCN architecture.
24

---

Published as a conference paper at ICLR 2022
Table S9: Test Accuracies for different structural data sets.
10
040
0853 23
11
00
050 89
194 24
0321
111
110
0903 81
6601
TLE:35
112..0
7E5:11
718
Duska
607
11
115-23
110 425
115 133
315679
120 80
652 83
013-00 444
10.1-00
111
13
103 8.00
TEST
330-20
130-08 83
49 m10
131-19
03-07
233
1981
120-11
6.401
300+13
m11
5.1+13
525 415
331-55
Table S10: Results for the structure-based experiments. We depict the test accuracy obtained various benchmark data sets when only considering structural information (i.e. the network has access uninformative node features). Graph classification results are shown the left, while node classification results are shown on the right.
Graph classificution
Node clavsification
METHOD
ENZYMES MNIST
PROTEINS
CLUSTER
PATTERN
GAT-4
63.3.13.7
21.7
63.2:10.44 83.4
67.5
266
16.71 0.0
58 58.348.8
GIN-4
75.6=228
21.32
74.6
1644 0.1
84.8+0..0
(haseline)
68.0-13.6
76.24
68.8
16:74 0.0
GCN-3-TOGL-1
75.1=2.11
30.3
6.5
84.8 0.4
73.8
4.3
16.8 0.0
86.7=0000
GCN.1-TOG- (static)
68.0:124
23.71
82.9
0.0
71.24
5.1
16.8 0.0
85
GIN-2-TOGL-1
76.2-224
23.72
84.4
73.9
16.6 0.3
86.610.1
GIN-2-TOGL-1 (static)
76.3:1228
25.2
7.0
83.9
0.1
74.2
4.2
16.4 0.1
85.4+0.1
GAT-3-TOGL-1
23.55
77.2
72.4
16.5
GAT GAT.3-TOGI. TOGL.11 (static)
68.4113.
22.7
3.9
81.91
1.1
68.91
4.0
16.7.
0.0
60.51:3.0
Table S11: Test accuracy on benchmark data sets (following standard practice, we report weighted accuracy on CLUSTER and PATTERN). Methods printed in black have been run in our setup, while methods printed grey are cited from the literature, i.e. Dwivedi et al. (2020), Morris et al. (2020) for IMDB-B B and REDDIT-B and Borgwardt et al. (2020) for WL/WL-OX results GIN-4 results printed in italics are 1-layer GIN-e, as reported in Morris et al. (2020). Graph classification results shown on the left, while node classification results are shown on the right.
Grapt clusikcttt
Mudic dlsutication
CIBAR-10
MNIST
PROTEINS.ELL
IMBI-B
REDDIT-B
CLUSTER
PATTERN
GAT4
64.2 0.4
75.9 +
68.5 5.2
85.5 0.2
76.3 2.4
57.7 + 0.3
75.8 1.8
GATED-GON-4
72.9 0.3
65.7 2
97.3
76.4
60.4 4 0.4
0..1
GIN-4
06.5+
71.9
65.3
96.5
0.3
74.1
11.9 4-7
89.8
0.2
WL
77.7 2.0
0.9
78.1 0.5
71.2=00
78.0
WL-OA
77.8
0.9
73.5 03
7.
ST.6
4CN-4 (huxeline)
54.2 1.5
72.8+441
63.8
9008
0.3
76.1
68.6
92.8 1.7
57.0
N5.5
GCN-2-TOGI-1
6.0
73.2444.7 4.7
53.0
95.5
0.2
3.9
72.0 233
88.4 09.4+22
60.4
0.2
86.6 6.1
GCN-3-TOGL (static)
0.5
71.0228
49.
9641:01
7.
728
12.
60.5 022
0.
GIN-3-TOGL-
06.1
GINL3- (static)
0.6
72.2 5.3
23.
96.1
71.11.1
73.8=24
89.11:11
60.6
86.5
GAT-3-TOGL-I
3.4
73.7
4 2.9
51. 133
0.0
3.9
7.8
8..7
59.4 1.7
59.6 4 33
GAT-3-T0GL-1 (static)
3..
63.2
9..2+-03
70.1
58.7
25

---

Published as a conference paper at ICLR 2022
Table S12: Test accuracy of the different methods on the Spheres vs. Torus classification task. We compare two architectures (GCN-4, GIN-4) with corresponding models where one layer is replaced with TOGL and highlight the winner of each comparison in bold.
METHOD
TORUS
GCN-4
77.8
GCN-3-TOGL-1
98.5+1.3
GIN-4
76.2+3.5
GIN-3-TOGL-1 99.6+0.99
Table S13: Graph classification results for the structure-based experiments. We depict the test accuracy obtained on various benchmark data sets when only considering structural information (i.e. the network has access to uninformative node features). We compare two architectures (GCN-4, GIN-4) with corresponding models where one layer is replaced with TOGL and highlight the winner of each comparison in bold.
Graph classification
METHOD
DD
PROTEINS
CIBAR 10
GCN-6
70.5+4.9
70.144.0
45.4+0.5
GCN-S-TOGL-I
75.9+2.5
73.5
54,420.7
GCN-8
68.5+33.6
72,0+3.3
45.6+0.5
GON-7-TOGL-1
75.6+2.9
73.5+3.3
54.1=1.1
GIN-6
16.1+2..4
72.943.4
53.6+1
GIN-S-TOGL-1
76.5+2.0
12.16
54.1=1.2
76.5+2.44
51.4+2.4
GIN-7-TOGL-I
76.0#3.6
72.6b4.1
55.3+0.7
J GEOMETRIC DATASET
In this section, we showcase the potential of TOGL for improving performance on tasks of geometric data sets. Tasks such as 3D object recognition gathering increasing attention in the machine learning community and our approach represents a promising direction to explore that area. We consider simple synthetic data set consisting of graphs linking points geometrical objects. In particular, we create data set with two classes where one consists of graphs lying on a sphere and the other of graphs lying on torus. The nodes of these graphs are composed of points sampled on the geometrical objects and embedded in higher dimensional space. The graph is then constructed as fully connected graphs between all sampled points. Table ST2 presents the classification accuracy of different methods on this experiment. We observe that substituting GNN layer with a TOGL layer allows to obtain nearly perfect accuracy, showcasing the crucial importance of a topology-informed layer in geometric data sets.
K LARGER NUMBER OF LAYERS THE GNN ARCHITECTURE
In this section, we investigate the performance of our approach. architecture with a larger number of layers. For computational reasons, we limit our analysis to 3 data sets, namely DD, PROTEINS and CIFAR- 10. We re-use the experimental setup as detailed in Section [.1 Tables S13 and S14 show the classification accuracy for the structural and the benchmark setup respectively. What we observe is that the advantage of including TOGL is generally conserved, and similar to the one observed in the 4 layers case. In absolute terms, for the structural dataset, the performance on the DD dataset increases with more layers (for both the baseline and the TOGL version) but decreases for the PROTEINS dataset. On the benchmark dataset (including node features), the performance tends to increase for all three datasets compared the 4-layers version.
26

---

Published as a conference paper at ICLR 2022
Table S14: Test accuracy on benchmark data sets. Graph classification results are shown on the left; node classification results are shown on the right. Following Table 1 we take existing architectures and replace their second layer by TOGL; we use italics to denote the winner of each comparison. A bold value indicates the overall winner of column, i.c. data set.
Graph classification
METHOD
CIFAR-10
DD
PROTEINS-FULL.
GCN-6
53.6+0.6
72.7+2.8
76.3+3.6
GCN-5-TOGL-1
63.0+0.8
76.0+3.2
76.0+3.2
GCN-8
52.940.9
70.744.5
75.442.4
GCN-7-TOGI-1
61.6+0.5
72.1
75.6+3.5
GIN-6
55.7+2.8
73.1
74.1+3.1
GIN-5-TOGL-1
6..5
72.744.2
73.8+3.3
GIN-8
56.0#3.5
71.6+5.8
74.8+3.2
GIN-7-TOGL-1
61.140.8
72.4+4.6
74.3+2.5
27


Reviews:
[{"review_id": "YGOShV0jdY", "reviewer": "Reviewer_PtXE", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well-motivated, well-written, and builds on a good theoretical framework. The authors also explicitly mention the limitations that they had to impose due to computational constraints (such as l1 constraints). They also show that in practice even using limited topological features such as connected components and cycles, helps GNNs perform better. There are three questions that I would like to ask the authors:\n\n1- Diffusion-based models also capture some global topological information about the graphs and it has been shown that using them in both supervised and contrastive manner significantly help GNNs. How diffusion-based models are compared to your model and what if there are any theoretical links between them and your work.\n\n2- I am curious to see how your model can contribute to contrastive learning with linear evaluation protocol as it seems it can introduce a good amount of signal. \n\n3- I was hoping to see this method to be applied to large-scale benchmarks on OGB rather than small (and almost overfitted) TUdataets. Benchmarking the model on those datasets can show the scalability and the performance of the proposed model.\n\n4- Finally, expanding more on the theoretical aspects of the work and maybe providing examples would make the paper more eadible.", "summary_of_the_paper": "Authors propose a topology-aware layer that is compatible with GNNs and can encode connected components and cycles to let the GNN learn better representations based on such global topological features.", "summary_of_the_review": "The paper roots in a nice theoretical framework but shorts fall on experimental side.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "LTZN6jVTyCN", "reviewer": "Reviewer_CsQm", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper provides new insights into topological aware GNN and I summarize the pros as follows:\n1. The work is well motivated. First of all, Figure 1 gives a motivational toy example to demonstrate the necessity to capture the topological structure. I really like the example, which is clear and easy to understand. Besides, I appreciate that the authors provide a comprehensive introduction and research roadmap in the related work section.\n\n2. The work is technically sound and relatively novel as it introduces persistent homology, which is more powerful than the WL-test from the view of topological awareness (Theorem 2), and provides a reasonable framework to integrate it. \n\nComments:\n1. I am confused about the topological structure awareness power of GCN illustrated in Figure 1. From both two subfigures, Vanilla GCN overperforms WL-test. The results are a conflict with the analysis of GIN~[1], where GCN should be less powerful. From my perspective, GCN first aggregates the neighbourhood features and then pool features with a mean operator. Such a message passing mechanism should be less powerful than WL-test.\n\n2. There may be a gap between the Betti numbers and the learned graph signal from the vertex-based filtrations, as no specific training loss was used to guarantee the alignment between the two terms.\n\n3. The author used the DeepSets embedding function as in section 4.1 to embed different graph signals into high-dimension space. However, the paper contains no empirical study to validate the choice.\n\n4. Though TOGL is demonstrated strictly powerful than WL-test, the experiment results from Table 2 showing that WL-test overperforms in several datasets.\n\n[1] Xu, K., Hu, W., Leskovec, J. & Jegelka, S. How Powerful are Graph Neural Networks? Arxiv (2018).", "summary_of_the_paper": "Motivated from the field of topological data analysis (TDA), the paper proposed a Topological Graph Layer (TOGL) plugin for Graph Neural Networks to boost the ability of topological structure detection.", "summary_of_the_review": "Motivated by topological data analysis, the paper proposed the TOGL plugin for GNN to boost the ability of topological structure detection. The idea is insightful and is analyzed theoretically and empirically. Though I have a few concerns as listed in Main Review, I would like to weakly accept the work. Surely, I will appreciate it if any of my concerns can be addressed.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "CuHxZUmugo5", "reviewer": "Reviewer_vj7C", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Strengths \n- TOGL may be a useful tool for the GNN community, while also opening the door for some further research in Topological Data Analysis (TDA). \n- The paper is well-written. \n\n# Weaknesses\n- The flow of the paper may be hard to follow for the non-GNN-expert ready (i.e. TDA-side reader) due to the lack of a short background section dedicated to the topic. \n- Numerical results, though fairly solids, are not striking either. \n\n# Minor comments : \n- I think that writing $f_\\theta$ instead of $f$ would improve clarity of some claims throughout the paper (it feels weird to me to write say \"the map $\\theta \\mapsto \\Psi(\\mathrm{ph}(G,f))$ is differentiable\", as the right-hand-side does not (visually) depends on $\\theta$).", "summary_of_the_paper": "This paper introduces TOGL, a new layer for Graph Neural Networks (GNN), making the GNN \"aware\" of topological information during this training phase. It differs from the closely related work *Graph Filtration Learning (GFL)* (although taking inspiration from it) as GFL is mostly a readout function (roughly, final layer in a GNN) while TOGL is a more general type of GNN-layer. Numerical experiments show how, when topological information is relevant, TOGL helps to leverage it.", "summary_of_the_review": "This is a competent paper that builds upon existing works and provides an interesting application case of TDA and may be a good addition to GNN toolboxes.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "C8UKnkIyxMM", "reviewer": "Reviewer_2oqu", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well motivated and mostly clear and easy to follow.\nThe method is well formulated and the experiments details are clear.\n\nIn terms of weaknesses of the paper, I have two main concerns:\n\n1. Indeed, as discussed, the aspect of oversmoothing is important in the case of GNNs. However, for a better verification of the existence/absence of oversmoothing with the proposed method, much layers need to be stacked, and preferably more datasets (like Cora, Citeseer and Pubmed) ,should be compared against to other recent methods like GCNII [Chen et al. 2020] .\n\n2.In the context of topology, which is the main contribution and scope of this work, it would be interesting to see more geometrically oriented data sets like ABC or Thingi10k (this is just an example), where objects with different topologies are given. Also, it would be interesting if the authors can discuss a recent ICCV paper that deals with geometrical data using persistent homology :\n\"Persistent Homology based Graph Convolution Network for Fine-grained 3D\nShape Segmentation\"\n", "summary_of_the_paper": "The authors present a topology analysis improvement to GCN, using persistent homology, to capture global information regarding the topology of the graph. \nThe authors conduct several experiments, from graph to node classification, and also introduce two novel data sets to exemplify the importance of topology. \nIn most cases, the proposed method outperforms other baseline methods, as well as other topology aware methods.\n\n", "summary_of_the_review": "The paper is clearly written and adds a contribution to the GNNs society. Also, the numerical experiments suggest improvement over other data sets. Some of the experiments can be improved, and since topology is of high interest geometrical datasets and application, I think that some discussion should be added, and will strengthen the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a new graph neural network layer that is sensitive to topological structure in the graph. Reviewers all believe the work is technically sound, and the experiments (particularly after author revisions) show clear benefits in cases where topological structure is important. The main questions are about whether the experimental evaluation is sufficient. While there are always more experiments that could be run, I tend to agree with the authors that the chosen experiments support the key claims in the paper, so it seems ok. The other question about the experiments is if they sufficiently convince the reader that topological structure is useful in practice. This seems more mixed. The paper would certainly be improved if there was a motivating application where there was a clear win. For example, molecular structures are used as motivation in the intro, but the best performing method on proteins doesn’t use the topological layer. All-in-all, though, there does appear to be clear improvements on carefully constructed cases, and there appear to be some benefits in real-world datasets.
Decision: Accept (Poster)