Title: Boosting Graph Pooling with Persistent Homology
Venue: NeurIPS 2024
Accepted: yes
Keywords: graph pooling

persistent homology

graph learning
Abstract: Recently, there has been an emerging trend to integrate persistent homology (PH) into graph neural networks (GNNs) to enrich expressive power. However, naively plugging PH features into GNN layers always results in marginal improvement with low interpretability. In this paper, we investigate a novel mechanism for injecting global topological invariance into pooling layers using PH, motivated by the observation that filtration operation in PH naturally aligns graph pooling in a cut-off manner. In this fashion, message passing in the coarsened graph acts along persistent pooled topology, leading to improved performance. Experimentally, we apply our mechanism to a collection of graph pooling methods and observe consistent and substantial performance gain over several popular datasets, demonstrating its wide applicability and flexibility.

Boosting Graph Pooling with Persistent Homology

Abstract

Recently, there has been an emerging trend to integrate persistent homology (PH) into graph neural networks (GNNs) to enrich expressive power. However, naively plugging PH features into GNN layers always results in marginal improvement with low interpretability. In this paper, we investigate a novel mechanism for injecting global topological invariance into pooling layers using PH, motivated by the observation that filtration operation in PH naturally aligns graph pooling in a cut-off manner. In this fashion, message passing in the coarsened graph acts along persistent pooled topology, leading to improved performance. Experimentally, we apply our mechanism to a collection of graph pooling methods and observe consistent and substantial performance gain over several popular datasets, demonstrating its wide applicability and flexibility.

1 Introduction

Persistent homology (PH) is a powerful tool in the field of topological data analysis, which is capable of evaluating stable topological invariant properties from unstructured data in a multi-resolution fashion [8]. Concretely, PH derives an increasing sequence of simplicial complex subsets by applying a filtration function (see Fig. 1(a)). According to the fact that PH is at least as expressive as WeisfeilerLehman (WL) hierarchy [19], there recently emerged a series of works seeking to merge PH into graph neural networks (GNNs), delivering competitive performance on specific tasks [52, 19]. Standard schemes of existing works achieve this by employing pre-calculated topological features [52] or placing learnable filtration functions in the neural architectures [16, 19]. Such integration of PH features is claimed to enable GNNs to emphasize persistent topological sub-structures. However, it is still unclear to what extent the feature-level integration of PH is appropriate and how to empower GNNs with PH other than utilizing features.

Graph pooling (GP) in parallel plays an important role in a series of graph learning methods [14], which hierarchically aggregates an upper-level graph into a more compact lower-level graph. Typically, GP relies on calculating an assignment matrix taking into account local structural properties such as community [34] and cuts [2]. Though the pooling paradigm in convolutional neural networks (CNNs) is quite successful [27], some researchers raise concerns about its effectiveness and applicability in graphs. For example, [30] challenges the local-preserving usage of GP by demonstrating that random pooling even leads to similar performance. Till now, it remains opaque what property should be preserved for pooled topology to better facilitate the downstream tasks.

From Fig. 1(a), it is readily observed that PH and GP both seek to coarsen/sparsify a given graph in a hierarchical fashion: while PH gradually derives persistent sub-topology (substructures such as cycles) by adjusting the filtering parameter, GP obtains a sub-graph by performing a more aggressive cut-off. In a sense of understanding a graph through a hierarchical lens, PH and GP turn out to align with each other well.

Persistence

filtering

Persistence

filtering

Persistent

homology

Persistent

homology

(a) Hierarchical view of GP and PH

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Pooling ratio

0.4

0.5

0.6

0.7

0.8

0.9

1

Persistence ratio

NCI1

NCI109

PROTEINS

ENZYMES

IMDB-BINARY

(b) Alignment of GP and PH

(c) Persistence diagrams

Driven by this observation, in this paper, we investigate the mechanism of aligning PH and GP so as to mutually reinforce each other. To this end, we conduct experiments by running a pioneer GP method DiffPool [50] to conduct graph classification on several datasets and at the same time use the technique in [16] to compute PH information. We manually change the pooling ratio and see what proportion of meaningful topological information (characterized by the ratio of non-zero persistence) is naturally preserved at the final training stage. Surprisingly, the correspondence is quite stable regardless of different datasets (see Fig. 1(b)), which implies the monotone trend between the pooling ratio and non-zero persistence is commonly shared by a large range of graph data. As a consequence, we develop a natural way to integrate PH and GP in both feature and topology levels. Concretely, in addition to concatenating vectorized PH diagram as supplementary features, we further enforce the coarsened graph to preserve topological information as much as possible with a specially designed PH-inspired loss function. Hence we term our method Topology-Invariant Pooling (TIP). TIP can be flexibly injected into a variety of existing GP methods, and demonstrates a consistent ability to provide substantial improvement over them. We summarize our contributions as follows:

We for the first time investigate the way of aligning PH with GP, by investigating the monotone relationship in between.

We further design an effective mechanism to inject PH information into GP at both feature and topology levels, with a novel topology-preserving loss function.

Our mechanism can be flexibly integrated with a variety of GP methods, achieving consistent and substantial improvement over multiple datasets.

2 Related work

Graph pooling. Graph pooling has been used in various applications, which can reduce the graph size while preserving its structural information. Early methods are based on clustering to coarsen graphs, such as the greedy clustering method Graclus [7], non-negative matrix factorization of the adjacency matrix [1], and spectral clustering [29]. Recently, learnable graph pooling methods have gained popularity, which learn to select important nodes in an end-to-end manner. DiffPool [50] follows a hierarchical learning structure by utilizing GNNs to learn clusters and gradually aggregate nodes into a coarser graph. MinCutPool [2] optimizes a normalized cut objective to partition graphs into clusters. DMoNPool [34] optimizes the modularity of graphs to ensure high-quality clusters. SEP [46] generates clusters in different hierarchies simultaneously without compromising local structures. These methods are classified as dense pooling due to the space complexity they incur. Despite their effectiveness, dense pooling methods have been criticized for high memory cost and complexity [5]. Therefore, various sparse pooling methods have been proposed, such as Top-K [11], ASAPool [38], and SAGPool [28]. These methods coarsen graphs by selecting a subset of nodes based on a ranking score. As they drop some nodes in the pooling process, these methods are criticized for their limited capacity to retain essential information, with potential effects on the expressiveness of preceding GNN layers [3].

Persistent homology in GNNs. PH is a technique to calculate topological features of structured data, and many approaches have been proposed to use PH in graph machine learning due to the high expressiveness of topological features on graphs [18]. Since non-isomorphic graphs may exhibit different topological features, the combination of PH and the Weisfeiler-Lehman (WL) algorithm leads to stronger expressive power [40]. This encourages further exploration on equipping GNNs with topological features. [52] propose that message passing in GNNs can be effectively reweighted using topological features. [16] and [19] provide theoretical and practical insights that filtrations in PH can be purely learnable, enabling flexible usage of topological features in GNNs. However, existing methods tend to view PH merely as a tool for providing supplementary information to GNNs, resulting in unsatisfactory improvements and limited interpretability.

3 Background

We briefly review the background of this topic in this section, as well as elaborate on the notations.

Let G = ( V, E ) be an undirected graph with n nodes and m edges, where V and E are the node and the edge sets, respectively. Nodes in attributed graphs are associated with features, and we denote by V = { ( v, x v ) } v ∈ 1: n the set of nodes v with d dimensional attribute x v . It is also practical to represent the graph with an adjacency matrix A ∈ { 0 , 1 } n × n and the node feature matrix X ∈ R n × d .

Graph Neural Networks. We focus on the general message-passing GNN framework that updates node representations by iteratively aggregating information from neighbors [12]. Concretely, the k -th layer of such GNNs can be expressed as:

where θ ( k ) is the trainable parameter, and M is the message propagation function. Numbers of M have been proposed in previous research [25, 15]. A complete GNN is typically instantiated by stacking multiple layers of Eq. 1. Hereafter we denote by GNN( · ) an arbitrary such multi-layer GNN for brevity.

Dense Graph Pooling. GP in GNNs is a special layer designated to produce a coarsened or sparsified sub-graph. Formally, GP can be formulated as G ↦→ G P = ( V P , E P ) such that the number of nodes | V P | ≤ n . GP layers can be placed into GNNs in a hierarchical fashion to persistently coarsen the graph. Typical GP approaches [50, 2, 34] rely on learning a soft cluster assignment matrix S ( l ) ∈ R n l -1 × n l :

Subsequently, the coarsened adjacency matrix at the l -th pooling layer is calculated as

and the corresponding node representations are calculated as

These approaches differ from each other in the way to produce S , which is used to inject a bias in the formation of clusters. In our work, we select three GP methods, i.e., DiffPool [50], MinCutPool [2], and DMoNPool [34], to cope with. Details of the pooling layers in these methods are summarized in Appendix A.

Topological Features of Graphs. A simplicial complex K consists of a set of simplices of certain dimensions. Each simplex γ ∈ K has a set of faces, and each face τ ∈ γ has to satisfy τ ∈ K . An element γ ∈ K with | γ | = k + 1 is called a k -simplex, which we denote by writing dim γ = k . Furthermore, if k is maximal among all simplices in K , then K is referred to as a k -dimensional simplicial complex. A graph can be seen as a low-dimensional simplicial complex that only contains 0simplices (vertices) and 1-simplices (edges) [19]. The simplest kind of topological features describing graphs are Betti numbers, formally denoted as β 0 for the number of connected components and β 1 for the number of cycles.

<latexi sh

1_b

64="

2

V0ZIN

zoL8EJ

5u

c

vrw

>A

C

H

j

S

FD3G

qk

K

U

X

f

B

WP

p

7

y

/

+

9

Q

m

M

Y

d

n

T

O

g

R

<latexi sh

1_b

64="ZLDmC

jQU8NWp

GOo

/

>A

3

c

VH

S

F

2

r

f

dugk

w

y

q

M

0

YR

I

B

X

E

z59

P

n

7

K

+

J

v

T

<latexi sh

1_b

64="c5Ky

GTCo2

q8n

O

>A

H

jV

LS

N

FD

r

fU

dugkW

RX

J

3

9w

/

+

Q

m

P

I

M

z

Y

E

<latexi sh

1_b

64="Sr

Hf

vMz

Q

3TG

7DA

>

C

X

cjV

N

F

2

/qOun

d

R

Y

0

Jp

5o

y

9w

k

+

m

P

Z

W

K

U

g

B

<latexi sh

1_b

64="FqB/

7Luf5GS

jZQ

P

JdW

U

>A

C

H

c

V

N

D3

2v

k

w

X

8

z

p

E

o

y

r

+

9

m

M

Y

n

T

O

g

R

<latexi sh

1_b

64="Nk2dY8/3

+y

BA7

I

Hg

>

C

n

cj

LS

FD

r

pf

Z

u

QU

v

RO

W

T

K

0G

E

9

P

M

m

Jz

5

w

X

<latexi sh

1_b

64="FqB/

7Luf5GS

jZQ

P

JdW

U

>A

C

H

c

V

N

D3

2v

k

w

X

8

z

p

E

o

y

r

+

9

m

M

Y

n

T

O

g

R

GNN

layers

(

A

(0)

,

X

(0)

)

(

A

(

l

-

1)

,

X

(

l

-

1)

)

(

A

′

(

l

)

,

X

(

l

)

)

Pooling

layers

S

(

l

)

(

A

(

l

)

,

X

(

l

)

)

Resample

Eq. (5)

PD

PD

Reweight

L

topo

Topology-Invariant Pooling

(

A

(

l

)

,

X

(

l

)

)

Eq. (6)

Add

Self-loops

Despite the limited expressive power of these two numbers, it can be improved by evaluating them alongside a filtration. Filtrations are scalar-valued functions of the form f : V ∪ E → R . Changes in the Betti numbers, named as persistent Betti numbers, can subsequently be monitored throughout the progress of the filtration: by considering a threshold ( a ∈ R ) , we can analyze the subgraph originating from the pre-image of (( -∞ , a ]) of f , denoted as ( f -1 (( -∞ , a ])) . The image of f leads to a finite set of values a 1 < · · · < a n and generates a sequence of nested subgraphs of the form ∅ ⊆ G 0 ⊆ . . . G k . . . ⊆ G n = G , where G k = ( V k , E k ) is a subgraph of G with V k := { v ∈ V | f ( x v ) ≤ a k } and E k := { ( v, w ) ∈ E | max { f ( x v ) , f ( x w ) } ≤ a k } . This process is also known as persistent homology (denoted as ph ( · ) ) on graphs. Typically, persistent Betti numbers are summarized in a persistence diagram (PD) as ph( G , f )[ i ] = D i , where i ∈ [0 , 1 , ... ] is the dimension of topological features. PD is made up of tuples ( a i , a j ) ∈ R 2 , with a i and a j representing the creation and destruction of a topological feature respectively (see Fig. 1(c)). The absolute difference in function values | a j -a i | is called the persistence of a topological feature, where high persistence corresponds to features of the function, while low persistence is typically considered as noise [19, 39].

4 Methodology

4.1 Overview

An overview of our method is shown in Fig. 2, where the shaded part corresponds to one layer of Topology-Invariant Pooling. The upper part is the GP process and the lower part is the injection of PH. Let ( A (0) , X (0) ) be the input graph. We consider to perform a GP at the ( l -1) -th layer. After obtaining a coarsened (densely connected) graph ( A ( l ) , X l ) with a standard GP method, we resample the coarsened graph using Gumbel-softmax trick as A ′ ( l ) in order to make it adapt to PH. Then, this coarsened graph is further reweighted injecting persistence, and is optimized by minimizing the topological gap L topo from the original graph, yielding ( A ( l ) , X l ) . By stacking multiple TIP layers, hierarchical pooling emphasizing topological information can be achieved. In the following sections, we elaborate on the detailed design of our mechanism.

4.2 Topology-Invariant Pooling

In many real-world applications, the topology of graphs are of utmost importance [44, 49, 16]. However, typical GNNs fail to capture certain topological structures in graphs, such as cycles [4, 51, 21]. Moreover, in dense graph pooling, graphs are pooled without preserving any topology. Even if we manage to make GNN topology-aware, the coarsened graph is nearly fully connected and has no meaningful topology at all, impairing the use of GNNs in these tasks. To overcome these limitations, we propose to inject topological information into GP. We resort to PH to characterize the importance of edges.

The core of PH is the notion of filtration, the selection of which presents a challenging task. As the coarsened graph evolves in each training step, integrating PH into GP demands multiple computations

of filtrations. To address this, we place recently proposed learnable filtration (LF) functions [16] to incorporate PH information for flexibility and efficiency. LF relies on node features and graph topology, which are readily available in GP. Consequently, LF can be seamlessly integrated into GP with minimal computational overhead. Specifically, we employ an MLP network Φ( · ) as the filtration function together with sigmoid( · ) to map node features X ∈ R n × d into n scalar values. Recently, an increasing amount of attention has been devoted to cycles [4, 51, 21] due to their significant relevance to downstream tasks in various domains such as biology [26], chemistry [35], and social network analysis [23]. Recognizing that cycles offer an intuitive representation of graph structure [31, 17], and preliminary experiments, shown in Appendix E.5, indicate that the additional inclusion of zero-dimensional topological features merely increases runtime, thus we instead focus on the one-dimensional PDs associated with cycles. For those edges do not form cycles, their creation and destruction are the same, leading to zero persistence. Following the standard way in GP (Eq. 2 3 4), we additionally propose the subsequent modules to inject PH into GP at both feature and topology levels.

Resampling. One major limitation of utilizing LF proposed in [16] is that the computation process is unaware of edge weights, i.e. edges with non-zero weights will be treated equally, so PH cannot directly extract meaningful topology from A ( l ) . Besides, rethinking GP in Eq. 3, the coarsened adjacency matrix has limited expressive power for two reasons. First, although S ( l ) is a soft assignment matrix obtained by softmax( · ) , each element still has nonzero values, i.e. A ( l ) is always densely connected. Second, the edge weights may span a wide range by multiplication (refer to Appendix D for empirical evidence). These drawbacks hinder the stability and generalization power of the subsequent message passing layers [13]. None of the existing GP methods can handle these problems properly.

Therefore, we resample the coarsened adjacency A ( l ) obtained from a normal GP layer (Eq. 3) as:

where A ( l ) is first normalized in the range of [0 , 1] , and resample( · ) is performed independently for each matrix entry using the Gumbel-softmax trick [22]. In practice, only the upper triangular matrix is resampled to make it symmetric and we add self-loops to the graph.

Persistence Injection. Now A ′ ( l ) ∈ { 0 , 1 } n l × n l is a sparse matrix without edge features so we can easily inject topological information into it. For a resampled graph with A ′ ( l ) and X ( l ) , we formulate the persistence injection as:

where ⊙ is the Hadamard product, to dense() means transforming sparse representations in terms of edges to dense matrix representations, ˜ D 1 is the augmented 1-dimensional PDs by placing the tuples correspond to self-loop edges on the diagonal part of original PDs D 1 , ˜ D 1 [ i ] is the i -th value in each tuple of ˜ D 1 , and we denote the updated adjacency matrix after persistence injection still as A ( l ) for notation consistency. Persistence injection can actually be regarded as a reweighting process. Since the filtration values are within [0 , 1] , A ( l ) after persistence injection is guaranteed to have edge weights in the range of [0 , 1] and is passed to the next pooling layer.

Topological Loss Function. The aforementioned mechanism can explicitly inject topological information into graphs, but it relies on the condition that the coarsened graph retains certain essential sub-topology. To this end, we propose an additional loss function to guide the GP process.

Intuitively, the coarsened graph should exhibit similarity to the original graph in terms of topology. Since the computation of PH is differentiable, one possible approach is to directly minimize the differences between the PDs of the original graph and the coarsened graph. However, this implementation would require computing the Wasserstein distance between two PDs through optimal transport [48], which is intractable in training due to its complexity. Considering that our objective is to estimate the difference, we instead propose vectorizing the PDs and minimizing their high-order statistical

features [36]. Specifically, we use several transformations (denoted as transform( · ) ) and concatenate the output, including triangle point transformation, Gaussian point transformation and line point transformation introduced in [6] to convert the tuples in PD into vector h t ( t ∈ [1 , m ]) . We calculate the mean vector µ as well as the second-order statistics as the standard deviation vector σ as:

In this manner, the difference between PDs can be estimated through the comparison of their statistics in the features, which is the concatenation of the mean and variance vectors. To further regularize the topological difference between layers, we introduce a topological loss term defined as:

where ( ·||· ) stands for the concatenation operation, L is the number of pooling layers, and d is the feature dimension. Note that the intuition behind L topo is different from the loss functions in existing graph pooling methods: the coarsened graph after pooling should be topologically similar to the original graph rather than having exact cluster structures.

4.3 Analysis

In this section, we examine the validity of our proposed method, and in particular, analyze its expressive power and complexity.

Theorem 1. The self-loop augmented 1-dimensional topological features computed by PH is sufficient enough to be at least as expressive as 1-WL in terms of distinguishing non-isomorphic graphs, i.e. if the 1-WL label sequences for two graphs G and G ′ diverge, there exists an injective filtration f such that the corresponding 1-dimensional persistence diagrams ˜ D 1 and ˜ D ′ 1 are not equal.

Proof Sketch. We first assume the existence of a sequence of WL labels and show how to construct a filtration function f from this. Consider nodes u and u ′ are nodes with unique label count in G and G ′ , then our filtration is constructed such that their filtration values f ( u ) and f ( u ′ ) are unique and different. Consider all three cases: (1) u and u ′ are both in cycles; (2) u and u ′ are both not in cycles; (3) one of u and u ′ is in cycles and the other is not. For all the cases, f ( u ) and f ( u ′ ) will be revealed in their respective persistence diagrams. Since f ( u ) and f ( u ′ ) are unique and different, we can use the augmented persistence diagrams to distinguish the two graphs.

This result demonstrates that the self-loop augmented 1-dimensional topological features contain sufficient information to potentially perform at least as well as 1-WL when it comes to distinguishing non-isomorphic graphs. We can then obtain the concluding remark that TIP is more expressive than other dense pooling methods by showing that there are pairs of graphs that cannot be distinguished by 1-WL but can be distinguished by TIP. Besides, our proposed simple yet effective self-loop augmentation eliminates the necessity of computing 0-dimensional topological features, thus reducing computational burdens.

Proposition 1. TIP is invariant under isomorphism.

Detailed proof and illustrations of the theorem and proposition can be found in Appendix C.

Complexity. PH can be efficiently computed for dimensions 0 and 1, with a worst-case time complexity of O ( mα ( m )) , where m represents the number of sorted edges in a graph. Here, α ( · ) represents the inverse Ackermann function, which is extremely slow-growing and can essentially be considered as a constant for practical purposes. Therefore, the primary factor that affects the calculation of PH is the complexity of sorting all the edges, which is O ( m log m ) . Our resampling and persistence injection mechanism ensures that the coarsened graphs are sparse rather than dense, making our approach both efficient and scalable. We provide running time comparisons in Appendix E.2, which indicates that the inclusion of TIP does not impose a significant computational burden.

5 Experiments

In the experiments, we evaluate the benefits of persistent homology on several state-of-the-art graph pooling methods, with the goal of answering the following questions:

Q1. Is PH capable of preserving topological information during pooling?

Q2. How does PH affect graph pooling in preserving task-specific information?

To this end, we showcase the empirical performance of TIP on two tasks, namely, topological similarity (Section 5.2) and graph classification (5.3). Our primary focus is to assess in which scenarios topology can enhance GP.

5.1 Experimental Setup

Models. To investigate the effectiveness of PH in GP, we integrate TIP with DiffPool, MinCutPool, and DMoNPool, which are the pioneering approaches that have inspired many other pooling methods. Additionally, as most pooling methods rely on GNNs as their backbone, we compare the widely used GNN models GCN [25], GIN [47], and GraphSAGE [15]. We also look into another two related and State-of-the-Art GNN models, namely TOGL [19] and GSN [4], which incorporate topological information and graph substructures into GNNs to enhance the expressive power. Several other GP methods, namely Graclus [7] and TopK [11] are also compared. For model selection, we follow the guidelines provided by the original authors or benchmarking papers. Our method acts as an additional plug-in to existing pooling methods (referred to as -TIP) without modifying the remaining model structure and hyperparameters. Appendix B.1 provides detailed configurations of these models.

Datasets. To evaluate the capabilities of our model across diverse domains, we assess its performance on a variety of graph datasets commonly used in graph related tasks. We select several benchmarks from TU datasets [32], OGB datasets [20] and ZINC dataset [43]. Specifically, we adopt molecular datasets NCI1, NCI109, and OGBG-MOLHIV, bioinformatics datasets ENZYMES, PROTEINS, and DD, as well as social network datasets IMDB-BINARY and IMDB-MULTI. Furthermore, to investigate the topology-preserving ability of our method, we conduct experiments on several highly structured datasets (ring, torus, grid2d) obtained from the PyGSP library. Appendix B.2 provides detailed statistics of the datasets.

Evaluation. In the graph classification task, all datasets are splitted into train (80%), validation (10%), and test (10%) data. Following the evaluation protocol in [50, 30], we train all models using the Adam optimizer [24] and implement a learning rate decay mechanism, reducing the learning rate from 10 -3 to 10 -5 with a decay ratio of 0.5 and a patience of 10 epochs. Additionally, we use early stopping based on the validation accuracy with patience of 50 epochs. We report statistics of the performance metrics over 20 runs with different seeds.

5.2 Preserving Topological Structure

In this experiment, we study Q1 about the ability of PH to preserve topological structure during pooling. Specifically, we assess the topological similarity between the original and coarsened graphs G and G ′ , by comparing the Wasserstein distance associated with their respective PDs ˜ D 1 and ˜ D ′ 1 . This evaluation criterion is widely used to compare the topological similarity of graphs [48, 41]. We utilize Forman curvature on each edge of the graph as the filtration, which incorporates edge weights and graph clusters to better capture the topological features of the coarsened graphs [42, 45].

We consider the 1-Wasserstein distance

W

1

,

′

1

= inf

δ

Π

(

˜

1

,

˜

′

1

)

(

x,y

)

δ

[

x

y

]

evaluation metric, where Π( · ) is the set of joint distributions δ ( x, y ) whose marginals are ˜ D 1 and ˜ D ′ 1 , respectively. Note that we are not learning a new filtration but keep a fixed one. Rather, we use learnable filtrations in training to enhance flexibility, and solely optimize L topo as the main objective.

D

D

∈

D

D

∼

∥

-

∥

We compare TIP with other pooling methods. Table 1 reports the average W values on three datasets, demonstrating that TIP can improve dense pooling methods to a large margin and have the best topological similarity. We visualize the pooling results in Fig. 3 for better interpretation, where isolated nodes with no links are omitted for clarity. It is evident that DiffPool, MinCutPool, and

(

˜

˜

)

as the

E

±

±

±

Original

DiffPool

DiffPool-TIP

MinCutPool

MinCutPool-TIP

DMoNPool

DMoNPool-TIP

TopK

Graclus

ring

torus

grid2d

DMoNPool tend to generate dense graphs and fail to preserve any topological structures. Conversely, our method, which incorporates topological features using PH, sparsifies the coarsened graphs and reveals certain essential topological structures. Notably, in the ring and torus datasets, large cycles are clearly preserved by our method. Besides, the grid2d dataset, despite having a different spatial layout, exhibits similar topology to torus (with four adjacent nodes forming a small cycle), resulting in similar shapes of their corresponding coarsened graphs. This indicates that the objective function indeed contributes to preserving topological similarity to some extent. Sparse pooling methods, which tend to preserve local topology, perform slightly better than the original dense pooling methods.

±

±

±

±

±

±

±

±

Original

DiffPool

DiffPool-TIP

MinCutPool

MinCutPool-TIP

DMoNPool

DMoNPool-TIP

TopK

Graclus

ENZYMES

NCI1

PROTEINS

5.3 Preserving Task-Specific Information

In this experiment, we examine the impact of PH on GP in downstream tasks to answer Q2 . We have observed in the former experiment that PH can preserve essential topological information during pooling. However, two additional concerns arise: (1) Does TIP continue to generate invariant sub-topology in the downstream task? (2) If so, does this sub-topology contribute to the performance of the downstream task? To address these concerns, we evaluate TIP using various graph classification benchmarks, where the accuracy achieved on these benchmarks serves as a measure of a method's ability to selectively preserve crucial information based on the task at hand.

We begin by visualizing the coarsened graphs in this task, where edges are cut-off by a small value. From Fig. 4, we can clearly observe that our method manage to preserve the essential sub-topology similar to the original graphs, while dense pooling methods cannot preserve any topology. As discussed in [30], dense pooling methods achieve comparable performance when the assignment matrix S is replaced by a random matrix. Here our visualization reveals that regardless of the value of S , the coarsened graph always approaches to a fully connected one. Sparse pooling methods, on the other hand, manage to preserve some local structures through clustering or dropping, but the essential global topological structures are destroyed.

Table 2 presents the average and standard deviation of the graph classification accuracy on benchmark datasets, where the results of GP and several baseline GNNs are provided. Experimental results demonstrate that TIP can consistently enhance the performance of the three dense pooling methods. While the original dense pooling methods sometimes underperform compared to the baselines, they are able to surpass them after integrating TIP.

Moreover, an intriguing observation can be found on ENZYMES dataset, where TOGL surpasses the baseline GNNs. TOGL in practice, incorporates PH into GNNs (GraphSAGE in our implementation), so this results underscores the significance of incorporating topological information for improved performance on ENZYMES. Further, our method demonstrates more significant improvements by augmenting the three dense pooling methods on the ENZYMES dataset. One possible explanation for the observed phenomenon is that the coarsened graphs generated by our methods bear a striking resemblance to numerous frequent subgraphs present in this dataset [10]. Such substructures may serve as indicators of unique characteristics within the graph, rendering them valuable for subsequent tasks. However, it is also worth noting that TOGL only exhibits marginal improvements or even underperforms on the other datasets. This suggests that simply integrating PH features into GNN layers does not fully exploit topological information. Conversely, injecting global topological invariance into pooling layers in our method yields superior performance.

To demonstrate the effectiveness of preserving the invariant sub-topology, we compared DiffPool-TIP with its variant counterpart, DiffPool-TIP-NL (no topological loss), by replacing L topo with the original L r in DiffPool (see Table 4 in Appendix A). The training objective curve and the Wasserstein distance curve are presented in Figure 5, both based on the ENZYMES dataset and a fixed filtration (the same as in Section 5.2). From the figures, it is evident that the objective value decreases as the coarsened graphs become more similar in topology to the original graphs when using DiffPool-TIP. However, when training without L topo , the performance is inferior. Additionally, even when the

0

20

40

60

80

100

120

Step

0.25

0.50

0.75

1.00

1.25

1.50

1.75

objective

DiffPool-TIP

DiffPool-TIP-NL

0

20

40

60

80

100

120

Step

50

100

150

200

250

300

DiffPool-TIP

DiffPool-TIP-NL

objective value converges, DiffPool-TIP-NL still exhibits changing topology, whereas DiffPool-TIP maintains a stable topology, possibly benefiting from the stability of PH [41]. This also suggests that multiple suboptimal topologies may contribute equally to the objective. Our topology invariant pooling strategy consistently selects topologies similar to the original graph, which leads to better performance. Additional visualization results and analysis about the coarsened graphs obtained by DiffPool-TIP-NL can be found in Appendix E.3.

Aside from the graph classification task, Table 3 presents the mean and standard deviation of prediction accuracy for the constrained solubility of molecules in the ZINC dataset, where mean square error is used as performance metric. We can observe that TIP can still boost the three pooling methods on regression task, which demonstrates that our proposed method can retain task-related information. Besides, we design an additional set of experiments in Appendix E.4, where the topological structure of the graph is highly task-relevant. Ablation study about the contributions of different modules are shown in Appendix E.5. Finally, to empirically demonstrate the expressive power of our proposed method, we provide an experiment on distinguishing non-isomorphic graphs in Appendix E.6.

6 Conclusion

In this paper, we developed a method named Topology-Invariant Pooling (TIP) that effectively integrates global topological invariance into graph pooling layers. This approach is inspired by the observation that the filtration operation in PH naturally aligns with the GP process. We theoretically showed that PH is at least as expressive as WL-test, with evident examples demonstrating TIP's expressivity beyond dense pooling methods. Empirically, TIP indeed preserved persistent global topology information, and achieved substantial performance improvement on top of several pooling methods on various datasets, demonstrating strong flexibility and applicability.

The potential limitation of our study is the heavy reliance of the proposed method on circular structures within graphs, potentially hindering its efficacy on tree-like graphs. Besides, our method lacks the ability to discriminate between graphs when the number of connected components is the only distinguishing factor. Our method can be extended to address this limitation by explicitly incorporating this information into the node features during the pooling process.

References

Davide Bacciu and Luigi Di Sotto. A non-negative factorization approach to node pooling in graph convolutional neural networks. In AI* IA 2019-Advances in Artificial Intelligence: XVIIIth International Conference of the Italian Association for Artificial Intelligence, Rende, Italy, November 19-22, 2019, Proceedings 18 , pages 294-306. Springer, 2019.

Filippo Maria Bianchi, Daniele Grattarola, and Cesare Alippi. Spectral clustering with graph neural networks for graph pooling. In International conference on machine learning , pages 874-883. PMLR, 2020.

Filippo Maria Bianchi and Veronica Lachi. The expressive power of pooling in graph neural networks. Advances in neural information processing systems , 36, 2023.

Giorgos Bouritsas, Fabrizio Frasca, Stefanos Zafeiriou, and Michael M Bronstein. Improving graph neural network expressivity via subgraph isomorphism counting. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(1):657-668, 2022.

C˘ at˘ alina Cangea, Petar Veliˇ ckovi´ c, Nikola Jovanovi´ c, Thomas Kipf, and Pietro Li` o. Towards sparse hierarchical graph classifiers. arXiv preprint arXiv:1811.01287 , 2018.

Mathieu Carri` ere, Fr´ ed´ eric Chazal, Yuichi Ike, Th´ eo Lacombe, Martin Royer, and Yuhei Umeda. Perslay: A neural network layer for persistence diagrams and new graph topological signatures. In International Conference on Artificial Intelligence and Statistics , pages 2786-2796. PMLR, 2020.

Inderjit S Dhillon, Yuqiang Guan, and Brian Kulis. Weighted graph cuts without eigenvectors a multilevel approach. IEEE transactions on pattern analysis and machine intelligence , 29(11):1944-1957, 2007.

Herbert Edelsbrunner and John L Harer. Computational topology: an introduction . American Mathematical Society, 2022.

Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428 , 2019.

Tianyu Fu, Chiyue Wei, Yu Wang, and Rex Ying. Desco: Towards generalizable and scalable deep subgraph counting. arXiv preprint arXiv:2308.08198 , 2023.

Hongyang Gao and Shuiwang Ji. Graph u-nets. In international conference on machine learning , pages 2083-2092. PMLR, 2019.

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning , pages 1263-1272. PMLR, 2017.

Liyu Gong and Qiang Cheng. Exploiting edge features for graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 9211-9219, 2019.

Daniele Grattarola, Daniele Zambon, Filippo Maria Bianchi, and Cesare Alippi. Understanding pooling in graph neural networks. IEEE Transactions on Neural Networks and Learning Systems , 2022.

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems , 30, 2017.

Christoph Hofer, Florian Graf, Bastian Rieck, Marc Niethammer, and Roland Kwitt. Graph filtration learning. In International Conference on Machine Learning , pages 4314-4323. PMLR, 2020.

Christoph Hofer, Roland Kwitt, Marc Niethammer, and Mandar Dixit. Connectivity-optimized representation learning via persistent homology. In International conference on machine learning , pages 2751-2760. PMLR, 2019.

Christoph Hofer, Roland Kwitt, Marc Niethammer, and Andreas Uhl. Deep learning with topological signatures. Advances in neural information processing systems , 30, 2017.

Max Horn, Edward De Brouwer, Michael Moor, Yves Moreau, Bastian Rieck, and Karsten Borgwardt. Topological graph neural networks. arXiv preprint arXiv:2102.07835 , 2021.

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems , 33:22118-22133, 2020.

Yinan Huang, Xingang Peng, Jianzhu Ma, and Muhan Zhang. Boosting the cycle counting power of graph neural networks with I 2 -gnns. In The Eleventh International Conference on Learning Representations , 2022.

Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 , 2016.

Chuntao Jiang, Frans Coenen, and Michele Zito. Finding frequent subgraphs in longitudinal social network data using a weighted graph mining approach. In Advanced Data Mining and Applications: 6th International Conference, ADMA 2010, Chongqing, China, November 19-21, 2010, Proceedings, Part I 6 , pages 405-416. Springer, 2010.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 , 2016.

Mehmet Koyuturk, Ananth Grama, and Wojciech Szpankowski. An efficient algorithm for detecting frequent subgraphs in biological networks. In ISMB/ECCB (Supplement of Bioinformatics) , pages 200-207, 2004.

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems , 25, 2012.

Junhyun Lee, Inyeop Lee, and Jaewoo Kang. Self-attention graph pooling. In International conference on machine learning , pages 3734-3743. PMLR, 2019.

Yao Ma, Suhang Wang, Charu C Aggarwal, and Jiliang Tang. Graph convolutional networks with eigenpooling. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining , pages 723-731, 2019.

Diego Mesquita, Amauri Souza, and Samuel Kaski. Rethinking pooling in graph neural networks. Advances in Neural Information Processing Systems , 33:2220-2231, 2020.

Michael Moor, Max Horn, Bastian Rieck, and Karsten Borgwardt. Topological autoencoders. In International conference on machine learning , pages 7045-7054. PMLR, 2020.

Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663 , 2020.

Christopher Morris, Martin Ritzert, Matthias Fey, William L Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and leman go neural: Higher-order graph neural networks. In Proceedings of the AAAI conference on artificial intelligence , volume 33, pages 4602-4609, 2019.

Emmanuel M¨ uller. Graph clustering with graph neural networks. Journal of Machine Learning Research , 24:1-21, 2023.

Christopher W Murray and David C Rees. The rise of fragment-based drug discovery. Nature chemistry , 1(3):187-192, 2009.

Koji Okabe, Takafumi Koshinaka, and Koichi Shinoda. Attentive statistics pooling for deep speaker embedding. arXiv preprint arXiv:1803.10963 , 2018.

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.

Ekagra Ranjan, Soumya Sanyal, and Partha Talukdar. Asap: Adaptive structure aware pooling for learning hierarchical graph representations. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 5470-5477, 2020.

Bastian Rieck. On the expressivity of persistent homology in graph learning. arXiv preprint arXiv:2302.09826 , 2023.

Bastian Rieck, Christian Bock, and Karsten Borgwardt. A persistent weisfeiler-lehman procedure for graph classification. In International Conference on Machine Learning , pages 5448-5458. PMLR, 2019.

Joshua Southern, Jeremy Wayland, Michael M. Bronstein, and Bastian Rieck. Curvature filtrations for graph generative model evaluation. In Thirty-seventh Conference on Neural Information Processing Systems , 2023.

RP Sreejith, Karthikeyan Mohanraj, J¨ urgen Jost, Emil Saucan, and Areejit Samal. Forman curvature for complex networks. Journal of Statistical Mechanics: Theory and Experiment , 2016(6):063206, 2016.

Teague Sterling and John J Irwin. Zinc 15-ligand discovery for everyone. Journal of chemical information and modeling , 55(11):2324-2337, 2015.

Nicolas Swenson, Aditi S Krishnapriyan, Aydin Buluc, Dmitriy Morozov, and Katherine Yelick. Persgnn: applying topological data analysis and geometric deep learning to structure-based protein function prediction. arXiv preprint arXiv:2010.16027 , 2020.

JunJie Wee and Kelin Xia. Forman persistent ricci curvature (fprc)-based machine learning models for protein-ligand binding affinity prediction. Briefings in Bioinformatics , 22(6):bbab136, 2021.

Junran Wu, Xueyuan Chen, Ke Xu, and Shangzhe Li. Structural entropy guided graph hierarchical pooling. In International conference on machine learning , pages 24017-24030. PMLR, 2022.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 , 2018.

Zuoyu Yan, Tengfei Ma, Liangcai Gao, Zhi Tang, Yusu Wang, and Chao Chen. Neural approximation of graph topological features. Advances in Neural Information Processing Systems , 35:33357-33370, 2022.

Chaolong Ying, Jing Liu, Kai Wu, and Chao Wang. A multiobjective evolutionary approach for solving large-scale network reconstruction problems via logistic principal component analysis. IEEE Transactions on Cybernetics , 53(4):2137-2150, 2021.

Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. Hierarchical graph representation learning with differentiable pooling. Advances in neural information processing systems , 31, 2018.

Jiaxuan You, Jonathan M Gomes-Selman, Rex Ying, and Jure Leskovec. Identity-aware graph neural networks. In Proceedings of the AAAI conference on artificial intelligence , volume 35, pages 10737-10745, 2021.

Qi Zhao, Ze Ye, Chao Chen, and Yusu Wang. Persistence enhanced graph neural network. In Silvia Chiappa and Roberto Calandra, editors, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , volume 108 of Proceedings of Machine Learning Research , pages 2896-2906. PMLR, 26-28 Aug 2020.

A Dense Graph Pooling Methods

Generally, dense graph pooling methods follow a hierarchical architecture, but their motivations differ. DiffPool suggests that nearby nodes should be pooled together, drawing on insights from link prediction and the assignment matrix S should be approximate to a one-hot vector so that the clusters are less overlapped with each other. MinCutPool, on the other hand, adapts the normalized cut as a regularizer for pooling. This encourages strongly connected nodes to be pooled together, ensures orthogonal cluster assignments, and promotes clusters of similar size. Moreover, DMoNPool additionally proposes a regularization to optimize the modularity quality of clusters so that the pooling can generate high quality clusters approach to ground truth. In summary, each of these methods introduces two types of unsupervised loss functions: the reconstruction loss L r , which regulates how the coarsened graph is reconstructed to retain some cluster structure, and the other is the cluster loss L c , which prevents convergence to local minima. The detailed formulations of these loss functions are provided in Table 4, where || · || F denotes the Frobenius norm, H denotes the entropy function, S i is the i -th row of S , D is the degree matrix, C is the number of clusters, B = A -DD T 2 m is the modularity matrix, respectively.

B Experimental Setup

B.1 Implementation detail

Hyperparameters. For dense pooling methods, the pooling ratio ranges from [0 . 1 , 0 . 5] , the number of pooling layers is 2, and the hidden dimension is selected from { 32 , 64 } . For the Graclus method we use 2 pooling layers, while for TopK we use 3 pooling layers with a pooling ratio of 0.8. The batch size for all models is uniformly set to 20, and the maximum number of training epochs is 1000. For the graphs obtained from the PyGSP library (ring, torus, grid2d), the number of nodes in each graph is fixed at 64.

Model configuration. All the methods are implemented using PyTorch and PyG [37, 9]. The compared methods are implemented following the implementations provided in the PyG library 2 . In the case of DiffPool, it uses a 3-layer GraphSAGE in each pooling layer, while MinCutPool and DMoNPool use a 1-layer GCN before pooling and a 1-layer GNN [33] in each pooling layer. Note that in DiffPool, the GNNs in Eqs. 2 and 4 are different, while in MinCut and DNoNPool they are the same one, as what their do in the original papers. TopK and Graclus are based on a 1-layer GNN [33]. TOGL is implemented using a 3-layer GraphSAGE as it has demonstrated superior performance on graph classification tasks (see Table 2). For the baseline GNN models (GCN, GIN, and GraphSAGE), we use 3 layers with mean/max pooling. In our model, TIP is incorporated as a plugin to existing pooling methods, without modifying the remaining model structure and hyperparameters. We replace the reconstruction loss L r with L topo while keeping the cluster loss L c unchanged. In the case of MinCutPool and DMoNPool, our resampling strategy is added after their original normalization of the coarsened graphs. In preserving topological structure experiments, we initialize node features as the concatenation of the first ten eigenvectors of graph Laplacian matrices. Moreover, we follow the settings in previous works [19, 18] to extend ˜ D 1 as follows: (1) each cycle is paired with the edge that created it; (2) edges e that do not create a cycle (still in this circle) are assigned a 'dummy' tuple value, such as ( f ( e ) , f ( e )) ; (3) all other edges will be paired with the maximum value of the filtration f max . In practice we set f max plus a constant as infinity

of destruction time. Therefore, ˜ D 1 consists of as many tuples as the number of edges m . Code is open-sourced at https://github.com/LOGO-CUHKSZ/TIP.git .

B.2 Dataset Statistics

The statistics of datasets used in this paper are summarized in Table 5, where we show the number of graphs, average number of nodes, average number of edges, number of features, and number of classes. We use the default dataset settings from PyG library 3 . Highly structured datasets (ring, torus, grid2d) are obtained from the PyGSP library 4 .

C Theoretical Expressivity of TIP

Theorem 1. The self-loop augmented 1-dimensional topological features computed by PH is sufficient enough to be at least as expressive as 1-WL in terms of distinguishing non-isomorphic graphs, i.e. if the 1-WL label sequences for two graphs G and G ′ diverge, there exists an injective filtration f such that the corresponding 1-dimensional persistence diagrams ˜ D 1 and ˜ D ′ 1 are not equal.

Proof. Assume that G and G ′ have n and n ′ nodes, and the label sequences of them diverge at some iteration h , which means there exists at least one label whose count is unique. Let nodes u and u ′ be the nodes with unique count in G and G ′ , respectively. Denote La ( h ) := { l 1 , l 2 , ... } as an enumeration of the finitely many hashed labels at iteration h . We can build a filtration function f by assigning a vertex v with label l i to its index, i.e. f ( v ) := i except that f ( u ) = n + n ′ +1 and f ( u ′ ) = n + n ′ +2 . The filtration of edge ( u, w ) is defined as f ( v, w ) := max { f ( v ) , f ( w ) } , and for isolated nodes v , the filtration of self-loop edges is f ( v, v ) = f ( v ) . Therefore, node with unique label count and its connected edges always correspond to the largest filtration value. Note that the 1-dimensional PD has been extended to have the same cardinality as the number of edges. If node u or u ′ forms a circle, the creation of this circle is related to the edge with the largest filtration; if node u or u ′ does not form a circle, the corresponding edges lie on the diagonal of ˜ D 1 with unique coordinates; otherwise node u constitute a circle while u ′ does not, then the corresponding edges lie in different parts in ˜ D 1 and ˜ D ′ 1 . Hence, ˜ D 1 = ˜ D ′ 1 .

̸

To demonstrate that TIP is more expressive than other dense pooling methods, we provide examples of graph pairs that cannot be distinguished by 1-WL but can be by TIP. We present an example of such non-isomorphic graphs in Fig. 6, where in the second graph the edge connecting two triangles does not form a circle. This edge corresponds to zero persistence and is eliminated in TIP. Consequently, the two originally non-isomorphic graphs can be easily distinguished. Provided that the three sufficient conditions proposed in [3] are satisfied, the pooling layers retain the same level of expressive power as GNN. In TIP, the reduction of node features remains unaltered, thereby fulfilling the three conditions. Additionally, TIP is capable of distinguishing certain non-isomorphic graphs,

indistinguishable

TIP

distinguishable

indicating its superior expressive power compared to conventional dense pooling methods such as DiffPool, MinCutPool, and DMoNPool.

Proposition 1. TIP is invariant under isomorphism .

To prove this statement, we adopt the following lemma [39] to show the isomorphic property of PH.

Lemma 1. Let G 1 and G 2 be two isomorphic graphs. For any equivariant filtration f , the corresponding persistence diagrams are equal.

In TIP, the filtration f is implemented using MLP, ensuring the equivariant property of filtration. Moreover, our resampling operations in Section 4.2 are equivariant. Therefore, the two isomorphic graphs after the resampling and persistence injection operations are still isomorphic to each other. Now we are able to prove Proposition 1.

Proof. For feature-level invariance , let X ∈ R n × d be the node features, P ∈ { 0 , 1 } n × n be the permutation matrix, S ∈ R n × n ′ be the assignment matrix, and PX be the permutated node features. The node feature map after pooling is denoted as X ′ ∈ R n ′ × d , then we have X ′ = S ⊤ X .

If we permute G using a permutation matrix P , the permutated node features after pooling are which proves the isomorphism invariant property of pooling at feature level.

For connectivity-level invariance , the connectivity after pooling is denoted as A ′ ∈ R n ′ × n ′ , then we have A ′ = S ⊤ AS . If we permute G using a permutation matrix P , the permutated connectivity after pooling is

This completes the proof.

D Empirical Evidence

We conduct experiments on the NCI1 dataset and plot the heatmap of the coarsened adjacency matrix in Fig. 7, where we can observe that the edge weights in DiffPool may span a wide range due to the involvement of multiple multiplications in their generation. For MinCutPool and DMoNPool, the edge weights are normalized by degree to mitigate numerical explosion. However, this normalization leads to the edge weights becoming excessively smooth and lacking sparsity. Learnable filtration based PH performs effectively on unweighted graphs; however, none of the existing GP methods are capable of appropriately handling the adjacency matrix.

(a) DiffPool-layer-1

(b) DiffPool-layer-2

(c) MinCutPool-layer-1

(d) MinCutPool-layer-2

(e) DMoNPool-layer-1

(f) DMoNPool-layer-2

35

60

30

50

25

40

20

30

15

20

10

10

0.5

0.25

0.4

0.20

0.15

0.3

0.10

0.2

0.05

0.1

0.00

0.0

0.16

0.04

0.14

0.12

0.03

0.10

0.08

0.02

0.06

0.01

0.04

0.02

0.00

0.00

E Additional Experiments

E.1 Visualization of persistence diagrams

We visually represent the 1-dimensional PD of graphs before and after applying TIP in terms of ring and grid2d datasets, as shown in Fig. 8. As described in Appendix B.1, in the original graphs we initialize node features with the eigenvectors of the graph Laplacian matrices. Consequently, the features of different edges exhibit slight variations, resulting in multiple nonoverlapping points in the PDs. Upon applying TIP, we can clearly observe that the one-dimensional topological features related to cycles remain similar to those in the original graphs. This demonstrates TIP's ability to preserve cycles.

E.2 Running time comparison

Wecompare the running time (in seconds) of TIP on different datasets. The experiments are conducted using an AMD EPYC 7542 CPU and a single NVIDIA 3090 GPU. We utilize the default settings from the graph classification experiments. We report the average running time of 50 epoches training in Table 6. It is worth noting that TIP is performed L times for L pooling layers, thus the inclusion of TIP does not impose a significant computational burden.

Original

DiffPool-TIP

MinCutPool-TIP

DMoNPool-TIP

ring

grid2d

Persistence diagram

Persistence diagram

Persistence diagram

Persistence diagram

0.5

0.6

50

0.5

0.6

0.5

0.4

0.3

0.4

Death

0.3

0.4

Death

0.3

0.4

Death

0.3

02

0.2

0.2

02

0.1

0.1

0.1

0.1

0.0

0.0

0.0

0.0

0.0

0.1

0.2

Birth

0.3

0.4

0.5

0.0

0.1

0.2

Birth

0.3

0.4

0.5

0.0

0.1

0.2

Birth

0.3

0.4

0.5

0.0

0.1

0.2

Birth

0.3

0.4

Persistence diagram

Persistence diagram

Persistence diagram

Persistence diagram

0.6

0.5

5 0

0.5

0.5

0.4

0.4

0.4

0.4

Death

0.3

Death

0.3

Death

03

Death

0.3

0.2

0.2

0.2

02

01

0.1

0.1

0.1

0.0

0.0

0.0

0.0

0.0

0.1

0.2

Birth

0.3

0.4

0.5

0.0

0.1

0.2

Birth

0.3

0.4

0.5

0.0

0.1

0.2

Birth

0.3

0.4

0.5

0.0

0.1

0.2

Birth

0.3

0.4

0.5

E.3 Visualization of coarsened graphs without preserving topology

We present some coarsened graphs that do not preserve topology (DiffPool-TIP-NL) in Fig. 9. These graphs contribute equally to the objective in the graph classification task, but their topologies are different. A similar observation was made by [30], who found that randomly generated graphs show equivalent performance. In DiffPool-TIP-NL, other topology-related modules in TIP are preserved, allowing some topological information to be injected into the three results shown in Fig. 9. Guided by the L topo , DiffPool-TIP tends to select the results that are most similar to the original graph among all the options. Experimental results in Fig. 5 demonstrate that this type of topology is superior and leads to better performance on downstream tasks.

E.4 Topology relevant experiments

To further demonstrate that our proposed method can effectively capture the topological features in graphs, we design an experiment where the topological structure of the graph is highly relevant. We generate a synthetic dataset named Cycles, comprising two balanced 2-class sets of 1000 graphs each. This dataset consists of either a single large cycle (class 0) or two connected large cycles (class 1),

Original

Result 1

Result 2

Result 3

ENZYMES

resembling digital numbers '0' and '8', respectively. The distinguishing factor between the classes lies in the presence of cycles, highlighting the significance of the graph's topological structure in classification. The node numbers range from 10 to 20, with 3-dimensional random node features generated. For model configuration, we uniformly use 1-GCN plus 1-pooling layer. The evaluation criteria remain consistent with those outlined in our paper. The experimental results in Table 7 demonstrate the effectiveness of TIP in leveraging topological features to significantly outperform the comparable pooling methods.

Additionally, to evaluate our method's performance on graphs with different number of connected components, we generated a synthetic dataset named 2-Cycles, comprising two balanced two-class sets of 1,000 graphs each. This dataset consists of either two disconnected large cycles (class 0) or two large cycles connected by a single edge (class 1). The distinguishing factor between the classes is the number of connected components. The node numbers range from 10 to 20, with three-dimensional random node features generated. For the model configuration, we uniformly employed one GCN layer plus one pooling layer. Experimental results in Table 7 indicate that our method is not effective in distinguishing similar graphs with different connected components. This aligns with our expectations, as our method does not explicitly incorporate such information, given that most graphs in real-world datasets are connected.

E.5 Ablation study

To assess the contributions of different modules in our TIP model, we conduct comprehensive ablation studies on NCI1, PROTEINS, ENZYMES, and IMDB-BINARY datasets. We utilize examine five ablated variants of TIP: (i) with no resampling (TIP-NR), (ii) with no persistence injection (TIP-NP), (iii) with no topological loss function (TIP-NL), (iv) with 0-dimensional topological features (TIP-0), (v) with fixed filtration (TIP-F). All these variants are applied on three baseline pooling methods.

As depicted in Table 8, ablating any of the above modules resulte in performance degradation compared to the full model, thus indicating the importance of each designed module in the success of TIP. Additionally, on all three datasets, the resampling module significantly enhance the classification outcomes, while its removal lead to a substantial performance drop. Without resampling, the learnable filtration will treat edges equally, resulting in the inclusion of nonsensical topological information. In some cases, this even impede the model's performance, as observed in the no injection variants which perform worse than their counterparts on the PROTEINS dataset.

Another noteworthy observation is that even in the absence of the topological loss function L topo , GP can still benefit from incorporating PH. This could be attributed to the fact that the learnable filtration can inherently capture certain essential topological information to some extent. Furthermore, our model can still reap the benefits of the topological loss function, which indirectly guides the pooling process, even without explicitly injecting topological information using persistence.

Further, we provide an ablation study of our topological loss term by replacing it with the Wasserstein distance. While the Wasserstein distance is a powerful metric for comparing persistence diagrams, its computation can be computationally intensive, particularly when dealing with high-dimensional vectorized representations. Therefore, it significantly increases our training time in practice. We denote the variant of using Wasserstein distance as 'TIP-W'. Here we present the ablation study results on two datasets. We can observe that TIP-W has competitive performance compared with the full version TIP (with our proposed loss term), and outperforms the variant TIP-NL (no loss term). Initially, we design our L topo to avoid the high computational complexity of Wasserstein distance, but

we are suprised to find that TIP also marginally outperforms TIP-W in numerous instances, potentially attributed to the efficacy of feature transformation and high-order statistical features. These elements serve as a feature augmentation mechanism to enhance the persistence diagrams.

In Section 4.3, we provide theoretical analysis that 1-dimensional topological features are powerful enough to distinguish non-isomorphic graphs, thus eliminating the necessity of incorporating 0dimensional features. In this section, we provide empirical evidence about incorporating additional 0-dimensional features to support our claim. The results of variant TIP-0 indicates that the inclusion of 0-dimensional topological features merely increases runtime and has no benefits for the overall performance. This explains why we merely consider 1-dimensional topological features in our method.

As for the ablation of filtration functions, we employ an MLP with randomly initialized and fixed parameters as the filtration function. Using learnable filtrations leads to significant gains over random filtration functions in more than half of the cases. In some cases, randomly initialized filtrations may happen to be close to the learned filtrations, but this does not consistently occur.

Overall, our ablation study supports the indispensability and effectiveness of each module in the TIP model, further underscoring their contributions to its success.

E.6 Evaluation of expressive power

The growing interest in the expressive capability of graph pooling has been prominent in recent studies [3]. A graph pooling model based on GNNs is deemed more effective as it can differentiate a larger set of non-isomorphic graphs by producing unique representations for each. Graph pooling integrated with appropriately designed message-passing layers proves to be as competent as the WL test in distinguishing graphs. Understanding the expressive capacity of graph pooling aids in selecting between existing pooling operators or crafting novel ones. Furthermore, to empirically assess the expressive capacity of our proposed approach, TIP, we conduct experiments on the EXPWL1 dataset

following the experimental setup detailed in [3]. Each graph pair ( G i , H i ) in EXPWL1 consists of two non-isomorphic graphs distinguishable by a WL test, which encode formulas with opposite SAT outcomes. Therefore, any GNN that has an expressive power equal to the WL test can distinguish them and achieve approximately 100% classification accuracy on the dataset. The classification outcomes on the EXPWL1 dataset are shown in Table 9, which reveal the notable improvement in the expressive capacity of graph pooling achieved through our proposed method in empirical evaluations.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our focus aims towards boosting graph pooling with persistent homology, motivated by the observation that they two align very well.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: In Sec. 6, we mentioned that the proposed method relies on circular structures within graphs, potentially hinders its efficacy on tree-like graphs.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate 'Limitations' section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All assumptions and a complete proof are provided in the Appendix C.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide implementation details and hyperparameters in Appendix B.1. We also submit codes to click and run.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All the datasets are obtained from open source libraries, and relevant links are provided in Sec. B.2. We also submit codes to click and run.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Experimental settings are provided in 5.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The error bars are provided in each table related to experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer 'Yes' if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide sufficient information on the computer resources in Sec. E.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Yes.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: No use of pretained models.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Properly credited.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes] ,

Justification:

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces topological graph pooling, a novel method that leverages persistent homology to enhance general graph pooling techniques. The use of persistent homology serves two primary purposes, preserving important topological features in the coarsened graphs and generating edge weights for them. For the first purpose, a topological regularization term comparing the topology of the original and coarsened graphs is developed. For the second purpose, edge weights for the coarsened graph are generated based on an importance score derived from the persistence of cycles containing these edges in the filtration. The authors conducted two sets of experiments to assess the utility of PH in graph pooling. The first set demonstrates that coarsened graphs computed using the proposed PH graph pooling method effectively preserve topological information, even better than other dense and sparse pooling methods. The second set compares and benchmarks the topological pooling enhancement applied to some standard pooling techniques against other pooling techniques and standard graph neural networks. The results, obtained through a comprehensive experimental setup, clearly show the superiority of topological pooling over the other benchmarked methods.\n\nSTRENGTHS:\nThis paper is one that evokes a sense of \"why didn't I think of that?\" due to its natural (but powerful) approach. In my opinion, after addressing a few questions, this paper deserves at least a spotlight at the conference.\n\n**Originality**: As far as I know, the connection between graph pooling and graph filtrations, while conceptually straightforward, has not been explored before.\n\n**Significance**: The persistent homology method addresses a significant problem in graph pooling, as highlighted by the authors. Previously, it was unclear which properties should be preserved in the coarsened graphs after pooling. It was even argued that random pooling could yield performances similar to previous state-of-the-art (SOTA) methods, making unclear why graph pooling was effective and how to design graph pooling techniques appropiately. This work clarifies that preserving the topology of the initial graph is crucial, consistently outperforming other SOTA graph pooling methods across various benchmarks. This makes intuitive sense, as topology concerns global structures, and destroying this information when getting a subgraph seems harmful for the network. This method is not only a clear win for persistent homology in the graph learning field, but also opens new avenues for effective pooling design in GNNs. \n\n**Clarity**: The paper is well-written, with pertinent and informative figures. In particular, Figure 3 effectively illustrates the importance of the contribution.\n\n**Quality**: The experiments are of high quality, encompassing various datasets and configurations. Additionally, the ablation study addressing the effectiveness of topological pooling in preserving input topology is enlightening and supports the use of persistent homology in graph pooling to preserve topology.\n\nWEAKNESSES:\n- Figure 1 (a), second row, is strange to me. Usually (and in the pipeline you describe), persistent homology goes from smaller to bigger graphs, and not the inverse path.\n- Line 80: Typographical error: \"filtraions\" should be corrected to \"filtrations\".\n- Equation 1: Missing comma after the formula.\n- In lines 114 and 115, you state that filtrations assign each vertex and edge a value. However, this is not always accurate. To have proper simplicial complexes through the filtration, the filtration function must assign lower values to the endpoints of edges than to the values of the edges themselves. Thus, the filtration needs to satisfy $f(\\{v_1, v_2\\})\\geq \\max(f(v_1), f(v_2))$.\n- In the paragraph starting at line 113, you mention that edges have a filtration value given by $f(\\{v_1, v_2\\}) = \\max(f(v_1), f(v_2))$, but, as stated in the previous comment, you do not restrict edge filtration values to these values. I suggest choosing one approach and modifying this paragraph slightly to ensure consistency.\n- In line 123 you state that the persistence diagram $ph(G, f)$ is a set of persistence diagrams $\\{D_1, D_2, ...\\}$.  However, in other parts, such as Equation 6, you imply that $ph(G, f)$ is only one of these persistence diagrams. I believe this is inconsistent.\n- Line 147 uses the word \"filtration\" twice, which seems redundant. I propose rephrasing it to \"The core of PH is the notion of filtration, the selection of which presents a challenging task.\"\n\n**Important points**: The score I give is subjected to solve/discuss at least the following points:\n\n- Lines 157 and 158 are unclear to me. How do you map the persistence diagrams to edge weights given by births and deaths of persistence diagrams points exactly? I think this should be described formally for several reasons. How do you choose the cycle representatives for the points of persistence diagrams? What does it happen if one edge belongs to more than one cycle? I think this is very relevant, even in the proofs.\n- In the proof of Theorem 4.1, I do not understand what are the nodes $u$ and $u'$. What are the nodes with different count? Are they unique? Also, the proof relies on understanding the specific function mapping persistence diagrams to edges. Should the proof end at line 537?\n- In the proof of proposition 1, what does it mean to be isomorphic for two graphs with features? I have not seen this definition before.\n\nQUESTIONS:\n- Why do you use the Gumbel-softmax trick? I understand that you obtain a full graph using usual graph pooling, but, precisely, as you are learning filtrations and using persistent homology, you are also capturing in the persistence diagrams the topology of subgraphs that may not need to be fully connected. Did you try to avoid the Gumbel-softmax trick?\n- In lines 534, 535, 546, and 541 you write \"circles\", and not \"cycles\". Is this intentional? I believe there are more \"circles\" distributed across the test.\n- I have not seen any mention of more general pooling methods, such as cell complex pooling or simplicial complex pooling. Do you think your work can be extended to encompass general pooling on topological deep learning data?\n- Have you tried using some fixed filtrations insteaf of learnable filtrations for the graphs benchmark? If so, was there a significant difference compared to learnable filtrations?\n- In the complexity section, you mention that persistence diagrams can be computed very efficiently for dimension one. Do you have a reference for this claim? I was aware of this for dimension zero, but I have not encountered it for dimension one before.\n- In the Q1 experiments, the filtration is learned during training or is it also fixed as for benchmarking?\n\nLIMITATIONS:\nThe limitations addressed are too simple. I think more effort is needed in this aspect.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-26"}

comments: SUMMARY:
This paper proposes a TDA-based mechanism for storing phase information in the pooling layer of a GNN. The proposed approach resamples the connections in the graph and scales the edge weights using persistence information from a one-dimensional persistence diagram. Experiments on artificial data for concept evaluation and on several benchmark datasets were also carried out.

STRENGTHS:
- The retention of phase information in the pooling layer is a new and interesting approach.
- Experiments have shown that performance can be improved.

WEAKNESSES:
- The proposed method uses a 1-dim betti number, but its validity is not known. All 1-dim death times seem to be the same (maximum time for filtration) and seem to be only information about the last connection of the cycle. Only limited information on the cycle seems to be stored and is less convincing.
- Learning to preserve topology information in persistent diagrams is proposed in [1]. (It would be inappropriate not to mention this.)  This paper vectorises PDs for training, but [1] and [2] show the possibility of learning in a more data-loss-free form, which lacks consideration. The novelty of the method also seems to be limited to the introduction of phase conservation in the pooling layer.
- The paper only considers TU Datasets, ZINC and OGB, which have become mainstream in recent evaluations, should also be included in this evaluation.

[1] Topological Autoencoders, ICML 2020

[2] Optimizing persistent homology based functions, ICML 2021

QUESTIONS:
See Weakness

LIMITATIONS:
The authors adequately addressed the limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a topology-based graph pooling layer, TIP. TIP fits easily with the current graph pooling frameworks. Once the pooled graph is obtained from any existing graph pooling techniques, the authors make this pooled graph adaptable to persistent homology. They consider the PH of this graph and optimize it by minimizing the topological loss function between the current persistence diagram and  original persistence diagram. The authors show experimental results on some synthetic and real-world datasets.\n\nSTRENGTHS:\n1. It is, indeed, a new idea to combine PH for graph pooling to maintain topological information in the graph during pooling. \n\n2. Experiment to show that the proposed method preserves important topological features is well-chosen.\n\n3. The paper provides experimental evidence that addition of TIP into a standard GNN framework improves the performance on most datasets for graph classification.\n\nWEAKNESSES:\n1. Theorem 4.1 is incorrect as stated. Only the 1-dimensional topological features computed by PH cannot be as expressive as 1-WL test in distinguishing non-isomorphic graphs. Consider two graphs with no loops, with different numbers of vertices. 1-WL test will be able to distinguish these graphs while 1-dimensional topological features computed by PH will not be able to. One would need to use the 0-dimensional topological features computed by PH. Refer to [1]. Adding self-loops is just a proxy for counting the number of vertices, which can be done using 0-dimensional features. \n\n2. Moreover, the authors, in [1], have already shown that PH is at least as expressive as 1-WL test. Hence, the theoretical contribution of the paper does not seem significant.\n\n[1]: Topological Graph Neural Networks, Horn et.al\n\nQUESTIONS:\n1. What happens when you consider graphs with more than one component? How does TIP perform in that scenario compared to other pooling mechanisms? \n\n2. Have you tried other ways to incorporate $L_{topo}$? For e.g., by choosing a different vectorization of PDs such as persistence images or rational hats?\n\nLIMITATIONS:\nYes, the authors have discussed limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-12"}

comments: SUMMARY:
This paper proposes a new and systematic way to integrate persistent homology into GNNs for improved performance.  Rather than applying PH in a brute force manner as much existing work does, this work adapts the method based on the coincidence between the graph pooling mechanism and the filtration of PH.  The work takes advantage of the dynamic nature of PH via the filtration and results in improved performance in message passing.

STRENGTHS:
This was a very nice paper that studies an important problem and proposes a novel approach using persistent homology.  In my mind, this is perhaps among the first most convincing applications and demonstrations of the relevance of persistent homology to deep learning theory and implementations.

WEAKNESSES:
The writing style of the paper could be improved, benefitting from a spell and linguistics check.

QUESTIONS:
GNNs are known to encompass other NN models and architectures, such as CNNs and transformers.  Is the approach adaptable to these special cases of GNNs?  Does the performance also hold up in machine learning tasks, such as image classification?  Would the theoretical guarantees also need to be adapted?

LIMITATIONS:
Limitations were considered in the check list, however, I would like to see these concerns integrated into the paper.  In addition, it would be helpful to have examples and demonstrations of the limitations of the method.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces TIP (topology-invariant pooling), a PH-based pooling layer. The proposed approach involves resampling graph connections from soft-cluster assignment matrices and adjusting edge weights using persistence information derived from 1-dimensional diagrams. TIP leverages a loss function designed to maintain topology by leveraging vectorizations of the 1-dimensional diagrams. Experimental evaluations conducted on synthetic and real classification datasets illustrate the performance of the proposed method.\n\nSTRENGTHS:\n- Flexibility: TIP is agnostic to the GNN method and can be easily combined with GNN layers\n- Novelty: This combination of PH for graph pooling tasks is novel.\n\nWEAKNESSES:\n- The theoretical part of the paper does not seem very relevant and novel. Theorem 4.1 only states the **existence** of a filtration that is at least as expressive as the 1-WL test, which has limited impact in practice. Similar results are well-known in the literature (e.g., [1]). Also, in the analysis, we must allow filtration functions f to leverage graph structure, not only node features as defined in the paper. Finally, it seems that Theorem 4.1 can be easily turned into a strictly more expressive statement (see question below). \n- Another main concern revolves around the motivation for preserving cycles in general-purpose graph coarsening methods. Overall, I found the discussion/motivation for this design choice hand-wavy. \n- While the proposed method improves over the base pooling layers (e.g., DiffPool), it does not lead to significant gains over some (pooling-free) models in Table 2. Also, the experiments include only 1 OGB dataset --- I recommend adding ZINC or at least another OGB dataset.\n\nQUESTIONS:\n1. For the non-zero persistence tuples, are the death times always equal $\\infty$ (or f_max + constant)? If so, isn't TIP exploiting only (non-persistent) homology?\n2. Figure 2 does not show self-loops after the resampling procedure. \n3. Including the definition of simplicial complexes (SCs) would be helpful. Are graphs (with self-loops) 1-dimensional SCs? Do self-loops count as independent cycles?\n4. How can the hierarchical view of PH in Figure 1 be obtained from sublevel filtrations as defined in Section 3? In other words, how can we achieve a decreasing sequence of subgraphs using sublevel filtrations?\n5. Where does the proposed method's expressivity stand compared to other graph pooling layers (see [2])? The Appendix provides a brief discussion. I think that discussion could be added to the main paper.\n6. Consider the same setting as Theorem 4. Isn't PH based on $D_1$ **strictly** more expressive than 1-WL? To prove this additional gain, it suffices to show a pair of graphs $G, G'$ with $D_1(G) \\neq D_1(G')$ that 1-WL cannot distinguish. Wouldn't G=two triangles and G'=hexagon (where all nodes have the same features) be an example of such a pair?\n7. How does the proposed model perform using a fixed (non-trainable) random filtration function (e.g., an MLP where the parameters are fixed)?\n\n[1] - Topological GNNs, ICLR 2022.\n\n[2] - The expressive power of pooling in Graph Neural Networks, NeurIPS 2023.\n\nLIMITATIONS:\nThe paper mentions as limitation the \"heavy reliance of the proposed method on circular structures within graphs\" in Appendix F.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-15"}