Neural Approximation of Graph Topological Features
Abstract
Topological features based on persistent homology capture high-order structural information so as to augment graph neural network methods.
However, computing extended persistent homology summaries remains slow for large and dense graphs and can be a serious bottleneck for the learning pipeline.
Inspired by recent success in neural algorithmic reasoning, we propose a novel graph neural network to estimate extended persistence diagrams (EPDs) on graphs efficiently.
Our model is built on algorithmic insights, and benefits from better supervision and closer alignment with the EPD computation algorithm.
We validate our method with convincing empirical results on approximating EPDs and downstream graph representation learning tasks.
Our method is also efficient; on large and dense graphs, we accelerate the computation by nearly 100 times.
Introduction
Graph neural networks (GNNs) have been widely used in various domains with graph-structured data [55, 35, 29, 52, 7].
Much effort has been made to understand and to improve graph representation power [57, 39, 3, 36].
An intuitive solution is to explicitly inject high order information, such as graph topological/structural information, into the GNN models [62, 34].
To this end, persistent homology [19, 18], which captures topological structures (e.g., connected components and loops) and encodes them in a summary called persistence diagram (PD) , have attracted the attention of researchers.
Indeed, persistence has already been injected to machine learning pipelines for various graph learning tasks [66, 67, 21, 5, 9, 61].
In particular, it has been found helpful to use the so-called extended persistence diagrams (EPDs) [11], which contain richer information than the standard PDs.
Despite the strong learning power of PDs and EPDs, their computation remains a bottleneck in graph learning.
In situations such as node classification [67] or link prediction [61], one has to compute
EPDs on vicinity graphs generated around all the nodes or all possible edges in the input graph.
This can be prohibitive especially for large and dense graphs.
Take the Amazon Computers dataset [47] [Figure] as an example.
To compute EPDs on vicinity graphs take several seconds on average, and there are 13381 nodes.
So to compute all EPDs with a single CPU can take up to a day.
This is not surprising as a practical algorithm for computing EPDs is quadratic to the graph size [61].
These computational difficulties raise the question: can we approximate the expensive computation of EPDs using an efficient learning-based approach?
This is a very challenging question due to the highly complex and unusual mathematical machinery behind the original algorithm.
First, the algorithm involves a reduction algorithm of the graph incidence matrix.
Each step of the algorithm is a modulo-2 addition of columns that can involve edges and nodes far apart.
Such algorithm can hardly be approximated by a direct application of the black-box deep neural networks.
The second challenge comes from the supervision.
The output EPD is a point set with an unknown cardinality.
The distance between EPDs, called the Wasserstein distance [10, 12], involves a complex point matching algorithm.
It is highly nontrivial to design a deep neural network with variable output and to support supervision via such Wassserstein distance.
Previous attempts [48, 37] directly use black-box neural networks to generate fixed-length vectorization of the PDs/EPDs and use mean squared error or cross-entropy loss for supervision.
The compromise in supervision and the lack of control make it hard to achieve high-quality approximation of PDs/EPDs.
In this paper, we propose a novel learning approach to approximate EPDs on graphs.
Unlike previous attempts, we address the aforementioned challenges through a carefully designed learning framework guided by several insights into the EPD computation algorithm.
In terms of model output and supervision, we observe that the computation of EPDs can be treated as an edge-wise prediction instead of a whole-graph prediction.
Each edge in the graph is paired with another graph element (either vertex or edge), and the function values of the pair are the coordinates of a persistence point in the EPD.
This observation allows us to compute EPDs by predicting the paired element for every edge of the graph.
The Wasserstein distance can be naturally decomposed into supervision loss for each edge.
This element-wise supervision can significantly improve learning efficiency compared with previous solutions, which treat PDs/EPDs as a whole-graph representation and have to use whole-graph representation pooling.
Another concern is whether and how a deep neural network can approximate the sophisticated EPD algorithm.
To this end, we redesign the algorithm so that it is better aligned with algorithms that are known to be learnable by neural networks.
Recall we observe that computing EPDs can be decomposed into finding pairing for each edge.
We show that the decomposition is not only at the output level, but also at the algorithm level.
The complex standard EPD computation algorithm can indeed be decomposed into independent pairing problems, each of which can be solved exactly using a classic Union-Find algorithm [13].
To this end, we draw inspiration from recent observations that neural networks can imitate certain categories of sequential algorithms on graphs [53, 56].
We propose a carefully designed graph neural network with specific message passing and aggregation mechanism to imitate the Union-Find algorithm.
Decomposing the algorithm into Union-Find subroutines and approximating them with a customized
GNN provide better alignment between our neural network and the EPD algorithm.
A better alignment guarantees better performance [58].
Empirically, we validate our method by quantifying its approximation quality of the EPDs.
On two downstream graph learning tasks, node classification and link prediction, we also show that our neural approximations are as effective as the original
EPDs.
Meanwhile, on large and dense graphs, our method is much faster than direct computation.
In other words, the approximated EPDs do not lose accuracy and learning power, but can be computed much more efficiently.
Finally, we observe that our model can be easily transferred to unseen graphs, perhaps due to the close imitation of the Union-Find subroutine.
This is encouraging as we may generalize topological computation to various challenging real-world graphs without extra effort.
In summary, we propose an effective learning approach to approximate EPDs with better supervision and better transparency.
The technical contributions are as follows
• We reformulate the EPD computation as an edge-wise prediction problem, allowing better supervision and more efficient representation learning.
• We show that the EPD computation can be decomposed into independent pairing problems, each of which can be solved by the Union-Find algorithm.
• Inspired by recent neural algorithm approximation works [53, 56], we design a novel graph neural network architecture to learn the Union-Find algorithm.
The closer algorithmic alignment ensures high approximation quality and transferability.
Background: Extended Persistent Homology
We briefly introduce extended persistent homology and refer the readers to [11, 18] for more details.
Ordinary Persistent Homology.
Persistent homology captures 0-dimensional (connected components) and 1-dimensional (loops) topological structures and measures their saliency via a scalar function called filter function .
Given a input graph G =
( V, E ) , with node set V and edge set E , we call all the nodes and edges simplices .
Denote by X = V ∪ E the set of all simplices.
We define a filter function on all simpices, f : X → R .
Often, f is induced by a node-valued function (e.g., node degrees), and further defined on edges as f ( uv ) =
max ( f ( u ) , f ( v )) .
Denote by X a the sublevel set of X , consisting of simplices whose filter function values ≤ a ,
X a = { x ∈ X | f ( x ) ≤ a } .
As the threshold value a increases from −∞ to ∞ , we obtain a sequence of growing spaces, called an ascending filtration of X : ∅ = X −∞ ⊂ ...
⊂ X ∞ =
X. As X a increases from ∅ to X , new topological structures gradually appear (born) and disappear (die).
For instance, the blue square persistence point at ( t 2 , t 3 ) in Figure 1 (b) indicates that the connected component u 2 appears at X t 2 and is merged with the whole connected component at X t 3 .
Applying the homology functor to the filtration, we can more precisely quantify the birth and death of topological features (as captured by homology groups) throughout the filtration, and the output is the so-called persistence diagram (PD) , which is a planar multiset of points, each of which ( b, d ) corresponds to the birth and death time of some homological feature (i.e., components, loops, and their higher dimensional analogs).
The lifetime | d − b | is called the persistence of this feature and intuitively measures its importance w.r.t.
the input filtration.
Extended Persistent Homology.
In the ordinary persistent homology, topology of the domain (e.g., the graph) will be created at some time (has a birth time), but never dies (i.e., with death time being equal to + ∞ ).
Hence we cannot capture their importance.
In the context of graphs, the importance of loops are not captured via the ordinary persistence.
To this end, extended persistence [11]introduces a descending filtration : ∅ = X ∞ ⊂ ...
⊂ X −∞ = X , where X a = { x ∈ X | f ( x ) ≥ a } is a superlevel set .
This induces a sequence of homology groups of the form H( X ) =
H( X, X ∞ ) → · · · → H( X, X a ) → · · · → H( X, X −∞ ) = ∅ .
When the input domain is a graph, the 1D EPD consists of β 1 number of persistence points, capturing the birth and death of independent loop features.
Here β 1 is the rank of the first homology group; for a connected graph, it is simply the number of independent loops, β 1 =
| E | − | V |
+ 1 .
A loop will be created during the ascending filtration, but killed during the descending filtration.
The birth and death times of the feature correspond to the threshold value a ’s when these events happen.
In general, the death time for such loop feature is smaller than the birth time.
For example, the red triangle persistence point in Figure 1 (b) denotes that the red cycle in Figure 1 (a) appears at X t 5 in the ascending filtration and appears again at X t 1 in the descending filtration.
Finally, PDs live in an infinite-dimensional space equipped with an appropriate metric structure, such as the so-called p -th Wasserstein distance [12] or the bottleneck distance [10].
They have been combined with various deep learning methods including kernel machines [44, 32, 6], convolutional neural networks [23, 27, 54, 68], transformers [64], connectivity loss [8, 22], and GNNs [67, 9, 61, 66, 21, 5].
During learning, there have been many works in the literature to vectorize persistence diagrams for downstream analysis.
Among these works a popular choice is the persistence image [1].
Algorithm Revision: Decomposing EPD into Edge-Wise Paring Predictions
In this section, we provide algorithmic insights into how the expensive and complex computation of
EPDs can be decomposed into pairing problems for edges.
And each pairing problem can be solved exactly using a Union-Find algorithm.
The benefit is two-folds.
First, the decomposition makes it possible to train the neural network through edge-wise supervision and use the original Wasserstein distance of EPDs.
This allows us to adopt the popular and effective edge-prediction GNN for the goal.
Second, we observe the similarity between the Union-Find and sequential algorithms which are known to be learnable by neural networks.
This gives us the opportunity to design a special graph neural network to imitate the algorithm accurately, and to approximate EPDs accurately.
Decompose the EPD Computation into Pairing Computations.
Recall that our goal is to compute the 0D and 1D EPDs PD 0 and PD 1 .
We observe that on these EPDs, each point corresponds to a unique pairing of graph elements (vertex-edge pair for 0D EPD, edge-edge pair for 1D EPD).
Each pair of elements are essentially the “creator” and “destroyer” of the corresponding topological feature during the filtration.
And their filtration values are the birth and death times of the topological feature.
For example, the persistence point located at ( t 2 , t 3 ) in Figure 1 (b) denotes that the edge u 2 u 3 is paired with u 2 .
Indeed, these pairings cover all elements in the graph; every edge/node belongs to a unique pairing.
We note that this is not in conflict with the fact that the PDs/EPDs are often sparse.
Many pairings are local and only pair adjacent elements.
They correspond to zero-persistence points living in the diagonal of the EPDs.
This pairing view gives us the opportunity to transform the computation of EPDs into a pairing prediction problem: for every edge in the graph, we predict its pairing element.
This will be the foundation of our design of the GNN in Sec. 4.
Meanwhile, we observe that the decomposition is not only at the output level .
The original algorithm of EPD, a sequential modulo-2 matrix reduction algorithm, can indeed be rewritten into a set of independent algorithm subroutines, each for the computation of one pairing.
Each subroutine is a Union-Find algorithm.
This new decomposed
EPD algorithm has not been reported before, although the idea follows from existing work [2].
For completeness, we will provide a proof of correctness of the algorithm.
Description of Algorithm 2.
The pseudocode for 1D EPD computation is shown in Algorithm 2.
We leave the algorithm for 0D to the supplementary material 3 .
For simplicity of presentation, we assume that all vertices have distinct function values f : V → R 4 .
Therefore finding the persistence value equals to finding the pairing.
To compute the EPD, we traverse all nodes in the vertex set and find their extended persistence pairing.
Combining the persistence pair from all nodes, we can obtain the final EPD.
The algorithm complexity analysis is provided in the supplementary material.
Finding persistence pairing for nodes.
For node u i ∈ V , we can call Algorithm 3 to identify the corresponding persistence pair.
In particular, the algorithm first sorts the graph elements according to an input scalar function, then does the edge operation by finding the roots of the corresponding nodes and merging these nodes.
See Figure 1(c) for a simple illustration.
For node u 1 , there are three upper edges: u 1 u 3 , u 1 u 4 , and u 1 u 6 .
We put each such edge u i u j in a different component C i j , – we call this upper-edge splitting operation – and start to sweep the graph in increasing values starting at f ( u i ) . Then, the first time any two such components merge will give rise to a new persistence point in the 1D EPD. For instance, C 1 4 and C 1 3 first merge at u 4 , and this will give rise to the brown
1: Input: graph G =
( V, E ) , filter function f .
2: Initialise-Nodes( V, f ) 3: Q = Sort-Queue ( V ) 4: while Q is not empty do 5: u =
Q. pop-min () 6: for v ∈ G. neighbors ( u ) do 7:
Relax-Edge( u, v, f ) 8: end for 9: end while
1: Input: filter function f , input graph G =
( V, E ) 2: V, E = sorted ( V, E, f ) 3: PD 0 =
Union-Find ( V, E, f ) , PD 1 = {} 4: for i ∈ V do 5:
C i =
{ C i j | ( i, j ) ∈ E, f ( j ) >
f ( i ) } ,
E i =
E 6: for C
i j ∈ C
i do 7: f ( C i j ) =
f ( i ) , E i =
E i − { ( i, j ) } + { ( C i j , j ) } 8: end for 9:
PD i= Union-Find-step ( V + C i − { i } , E i , f, C i ) 10:
PD 1 + = PD i11: end for 12: Output: PD 0 , PD
1: Input: V , E , f , C i 2: PD i 1 = {} 3: for v ∈ V do 4: v.value = f
( v ) , v.root = v 5: end for 6: Q =
Sort ( V ) , Q = Q − { v | f ( v ) <
f ( i ) } 7: while Q is not empty do 8: u = Q. pop-min () 9: for v ∈ G. neighbors ( u ) do 10: pu, pv =
Find-Root ( u ) , Find-Root ( v ) 11:
if pu ̸ = pv then 12: s = argmin ( pu.value, pv.value ) 13: l = argmax ( pu.value, pv.value ) 14: l.root
=
s 15: if pu ∈ V i and pv ∈ V
i then 16:
PD i 1 + { ( u.value, l.value ) } 17: end
if 18: end if 19: end for 20: end while 21: Function: Find-Root ( u ) 22: pu = u 23: while pu ̸ = pu.root do 24: pu.root = ( pu.root ) .root , pu
= pu.root 25: end while 26: Return: pu loop in Figure 1(a) with ( t 4 , t 1 ) as its persistence point.
While in Figure 1 (d), the two connected components, C 2 3 and C 2 5 (originated from u 2 ) will not be united.
Therefore, node u 2 will not lead to any persistence point in the EPD.
Correctness.
The following theorem states that the decomposed algorithm is correct.
We provide a sketch of the proof, leaving the complete proof to the supplementary material.
Theorem 3.1.
Algorithm 2 outputs the same 1D EPDs as the standard EPD computation algorithm.
Proof sketch.
To compute the 1D EPDs, we simply need to find the pairing partner for all edges.
Therefore, to prove that the two algorithms output the same 1D EPDs, we need to prove that the output pairing partners are the same (or share the same filter value).
We prove this by showing that both the standard EPD computation algorithm and Algorithm 2 find the “thinnest pair", i.e., the paired saddle points are with the minimum distance in terms of filter value, for all edges.
Neural Approximation of Union-Find.
In the previous paragraph, we showed that the computation of EPDs can be decomposed into the parallel execution of Union-Find algorithms, which share a similar sequential behavior.
This gives us the opportunity to approximate these Union-Find algorithms well, and consequently approximate EPDs well.
Approximating algorithms with neural networks is a very active research direction [63, 28, 31, 43, 45, 59].
Within the context of graph, GNNs have been proposed to approximate parallel algorithms (e.g.,
Breadth-First-Search) and sequential algorithms (e.g., Dijkstra) [53, 51, 56].
Particularly relevant to us is the success in approximating the category of sequential algorithms such as Dijkstra.
These sequential algorithms, as generally defined in Algorithm 1, sort graph elements (vertices and edges) according to certain function, and perform algorithmic operations according to the order.
As described in previous paragraphs, the Union-Find algorithm also contains these steps, and can be expressed in a sequential-like form (Algorithm 3).
Therefore we propose a framework to simulate the algorithm.
A Graph Neural Network for EPD Approximation
Previous section establishes the algorithm foundation by showing that we can decompose EPD computation into edge pairing prediction problems, each of which can be solved using a Union-Find algorithm.
Based on such algorithmic insights, we next introduce our neural network architecture to approximate the EPDs on graphs.
Our main contributions are: (1) we transform the EPD computation into an edge-wise prediction problem, and solve it using a GNN framework, inspired by the GNN for link prediction; (2) we design a new backbone GNN model PDGNN to approximate the Union-Find algorithm, with specially designed pooling and message passing operations.
4.1 EPD computation as a edge-wise prediction problem
We have established that computing PD 0 and PD 1 can be reduced into finding the pairing partners for all edges.
We transfer the problem into an edge-wise prediction problem.
We predict the persistence pairing for all edges.
This is very similar to a standard link prediction problem [7, 61], in which one predicts for each node pair of interest whether it is a real edge of the graph or not.
Inspired by standard link-prediction GNN architectures [7, 61], we propose our model (see Figure 2) as follows.
(1) For an input graph G =
( V, E ) and a filter function f , we first obtain the initial filter value for all the nodes: X = f ( V ) ∈ R | V |∗ 1 , and then use a specially designed GNN model which later we call PDGNN G to obtain the node embedding for all these vertices: H = G ( X ) ∈ R | V |∗ d H .
(2) Subsequently, a MLP (Multi-layer perceptron) W is applied to the node embeddings to obtain a two dimensional output for each edge ( u, v ) ∈ E , corresponding to its persistence pairing.
Formally, we use PP uv = W
([ h u (cid:76) h v ]) ∈ R 2 as the persistence pair.
Here, h u and h v denote the node embedding for node u and v , and (cid:76) represents the concatenation of vectors.
Given that the Union-Find-step algorithm should be implemented on all edges to obtain the EPDs.
Ideally we need a large GNN model to simulate all Union-Finds at the same time.
However, these
Union-Find-step algorithms are running in parallel on the same graph.
There are many overlapping or similar computational steps between computations on different nodes.
Besides, the node ordering are exactly the same.
Therefore we expect a limited-size single GNN model can simultaneously approximate all EPD computations.
This is confirmed by empirical evidence in Section 5.
4.2 PDGNN
In this section, we explain how to design the backbone GNN to approximate the Union-Find algorithm.
Note the Union-Find is similar to known sequential algorithms but with a few exceptions.
We design specific pooling and message passing operations to imitate these special changes.
These design choices will be shown to be necessary in the experiment section.
Recall a typical GNN learns the node embedding via an iterative aggregation of local graph neighbors.
Following [57], we write the k -th iteration (the k -th GNN layer) as:
[Equation]
where h ku is the node features for node u after k -th iterations, and N ( u ) is the neighborhood of node u .
In our setting, h 0 u
= x u is initialized to be the filter value of node u .
Different GNNs have different
MSG and AGG functions, e.g., in GIN [57], the message function MSG is a MLP followed by an activation function, and the aggregation function AGG is a sum aggregation function.
[Table]
[Table]
We now describe our specially designed GNN, called PDGNN (Persistence Diagram Graph Neural
Network).
Compared with the Sequential algorithms (Algorithm 1) [56], our Union-Find algorithm (Algorithm 3) differs in: (1) the Find-Root algorithm which needs to return the minimum of the component, (2) additional edge operations such as upper-edge splitting.
To handle these special algorithmic needs, our PDGNN modifies standard GNNs with the following modules.
A new aggregation due to the Find-Root function.
Finding the minimum intuitively suggests using a combination of several local min-aggregations.
Considering that the sum aggregation can bring the best expressiveness to GNNs [57], we implement the root-finding process by a concatenation of sum aggregation and min aggregation as our aggregation function.
To be specific:
[Equation]
Improved edge operations.
As shown in [53, 56], classic GNNs are not effective in “executing”
Relax-Edge subroutines.
Furthermore, in Algorithm 2, we also need the upper-edge splitting operation for each vertex.
In other words, the information of the separated components C
i j are formed by the information from both nodes u i and u j .
To this end, we use edge features and attention to provide bias using edges.
Specifically, we propose the following message function in the k -th iteration:
[Equation]
where σ k is an activation function, W k is a MLP module, and α kuv is the edge weight for uv .
We adopt PRELU as our activation function [Table] and the edge weight proposed in [52] as our edge weight.
Training PDGNN.
We use the 2-Wasserstein distance between the predicted diagram and the ground truth EPD as the loss function.
Through optimal matching, the gradient is passed to each predicted persistence pair.
Since we have established the one-to-one correspondence between pairs and edges, the gradient is then passed to the corresponding edge, and contributes to the representation learning. [Table] [Table]
Experiments
In this section, we thoroughly evaluate the proposed model from 3 different perspectives.
In Section 5.1, we evaluate the approximation error between the predicted diagram and the original diagram and show that the prediction is very close to the ground truth.
Even with a small approximation error, we still need to know how much does the error influence downstream tasks.
Therefore, in Section 5.2, we evaluate the learning power of the predicted diagrams through 2 downstream graph representation learning tasks: node classification and link prediction.
We observe that the model using the predicted diagrams performs comparably with the model using the ground truth diagrams.
In Section 5.3, we evaluate the efficiency of the proposed algorithm.
Experiments demonstrate that the proposed method is much faster than the original algorithm, especially on large and dense graphs.
Source code is available at https://github.com/pkuyzy/TLC-GNN.
Datasets.
To compute EPDs, we need to set the input graphs and the filter functions.
Existing state-of-the-art models on node classification [67] and link prediction [61] mainly focus on the local topological information of the target node(s).
Following their settings, for a given graph G =
( V, E ) , we extract the k -hop neighborhoods of all the vertices, and extract | V | vicinity graphs.
In terms of filter functions, we use Ollivier-Ricci curvature [41], heat kernel signature with two temprature values [49, 25] and the node degree 5 .
For an input vicinity graph, we compute 4 EPDs based on the 4 filter functions, and then vectorize them to get 4 peristence images [1].
Therefore, we can get 4 | V | EPDs in total.
The input graphs include (1) citation networks including Cora, Citeseer, and PubMed [46]; (2) Amazon shopping datasets including Photo and Computers [47]; (3) coauthor datasets including CS and Physics [47].
Details are available in the supplementary material.
We note that in the literature, EPDs often contain the extended persistence point of the whole connected component.
In our setting, we remove the point because that (1) no edge is paired with the whole connected component; (2) the value is easy to obtain, and does not need an extra prediction.
5.1 Approximation Quality
In this section, we evaluate the approximation error between the prediction and the original EPDs.
Evaluation metrics.
Recall that the input of our model is a graph and a filter function, and the output is the predicted EPD.
After obtaining the predicted EPD, we vectorize it with persistence image [1] and evaluate (1) the 2-Wasserstein ( W 2 ) distance between the predicted diagram and the ground truth
EPD; (2) the total square error between the predicted persistence image and the ground truth image (persistence image error, denoted as PIE).
Considering that our aim is to estimate EPDs on graphs rather than roughly approximating persistence images, we use the W 2 distance as the training loss, while the PIE is only used as an evaluation metric.
Given an input graph (e.g., Cora, Citeseer, etc.) and a filter function, we extract the k -hop neighborhoods of all the vertices and separate these vicinity graphs randomly into 80%/20% as training/test sets.
We report the mean W 2 distance between diagrams and PIE on different vicinity graphs and 4 different filter functions.
Baseline settings.
PDGNN denotes our proposed method, that is, the GNN framework with the proposed AGG function and MSG function.
Its strategy is to first predict the EPD, and then convert it to the persistence image.
To show its superiority, we compare with the strategy from [48, 37], i.e., directly approximate the persistence image of the input graph, as a baseline strategy.
GIN_PI and
GAT_PI denote the baseline strategy with GIN [57] and GAT [52] as the backbone GNNs.
To show the effectiveness of the modules proposed in Section 4, we add other baselines with our proposed strategy.
GAT denotes GAT as the backbone GNN.
GAT (+MIN) denotes GAT with the new AGG function.
Compared with PDGNN, it exploits the original node feature rather than the new edge feature in the MSG function.
PDGNN (w / o ew) denotes PDGNN without edge weight.
Further experimental settings can be found in the supplementary material.
Results.
Table 1 reports the approximation error, we observe that PDGNN outperforms all the baseline methods among all the datasets.
The comparison between GAT and GAT_PI shows the benefit of predicting EPDs instead of predicting the persistence image.
Comparing GAT and GAT (+MIN), we observe the advantage of the new AGG function, which shows the necessity of using min aggregation to approximate the Find-Root algorithm; Comparing GAT (+MIN) and PDGNN, we observe the effectiveness of using the new MSG function to help the model capture information of the separated connected components.
The comparison between PDGNN (w / o ew) and PDGNN shows that edge weights help the model focus on the individual Relax-Edge sub-algorithm operated on every edge.
5.2 Downstream Tasks
In this section, we evaluate the performance of the predicted diagrams on 2 graph representation learning tasks: node classification and link prediction.
We replace the ground truth EPDs in state-ofthe-art models based on persistence [67, 61] with our predicted diagrams and report the results.
Baselines.
We compare our method with various state-of-the-art methods.
We compare with popular
GNN models including GCN [29], GAT [52] and HGCN [7].
For link prediction, we compare with several state-of-the-art methods such as SEAL [65] and P-GNN [62].
Notice that GCN and
GAT are not originally designed for link prediction, therefore we follow the settings in [7, 61], that is, to get the node embedding through these models, and use the Fermi-Dirac decoder [30, 42] to predict whether there is a link between the two target nodes.
In comparison with the original EPD, we also add PEGN [67] and TLC-GNN [61] as baseline methods.
Furthermore, to show the benefit of directly predicting EPDs, we also add the baseline methods PEGN (GIN_PI) and TLC-GNN (GIN_PI) , which replace the original persistent homology feature with the output from GIN_PI.
Evaluation metrics.
For node classification, our setting is the same as [29, 52, 67].
To be specific, we train the GNNs with 20 nodes from each class and validate (resp.
test)
the GNN on 500 (resp. 1000) nodes.
We run the GNNs on these datasets 10 times and report the average classification accuracy and standard deviation.
For link prediction, our setting is the same as [7, 61].
To be precise, we randomly split existing edges into 85/5/10% for training, validation, and test sets.
An equal number of non-existent edges are sampled as negative samples in the training process.
We fix the negative validation and test sets, and randomly select the negative training sets in every epoch.
We run the
GNNs on these datasets 10 times and report the mean average area under the ROC curve (ROCAUC) scores and the standard deviation.
Results.
Table 2 and Table 3 summarize the performance of all methods on node classification and link prediction.
We observe that PEGN (PDGNN) and TLC-GNN (PDGNN) consistently perform comparably with PEGN and TLC-GNN, showing that the EPDs approximated by PDGNN have the same learning power as the true EPDs.
Furthermore, PEGN using the approximated EPDs achieve better or comparable performance with different SOTA methods.
We also discover that PEGN (GIN_PI) and TLC-GNN (GIN_PI) perform much inferior to the original models using the true EPDs.
It demonstrates that the large approximation error from GIN_PI lose much of the crucial information which is preserved in PDGNN.
Transferability.
One appealing feature of our method is its transferability.
Training on one graph, our algorithm can estimate EPDs well on another graph.
This makes it possible to apply the computationally expensive topological features to a wide spectrum of real-world graphs; we can potentially apply a pre-trained model to large and dense graphs, on which direct EPD computation is infeasible.
The experiments are provided in the supplementary material.
5.3 Algorithm Efficiency
In this section, we evaluate the efficiency of our proposed model.
For a fair and complete comparison, we compare with algorithms from Gudhi [50] and from [61].
We select the first 1000 nodes from Cora,
Citeseer, PubMed, Photo, Computers, CS, Physics, and then extract their 2-hop neighborhoods.
With
Ollivier-Ricci curvature as the filter function, we compute the EPDs and report the time (seconds) used to infer these diagrams.
Results.
We list the average nodes and edges of these vicinity graphs in the first line of Table 4.
As shown in Table 4, although our model is slower on small datasets like Cora or Citeseer, it is much faster on large and dense datasets.
Therefore we can simply use the original algorithm to compute the EPDs on small graphs, and use our model to estimate EPDs on large graphs.
The model can be applied to various graph representation learning works based on persistent homology.
Conclusion
Inspired by recent success on neural algorithm execution, we propose a novel GNN with different technical contributions to simulate the computation of EPDs on graphs.
The network is built on algorithmic insights, and benefits from better supervision and closer alignment with the EPD computation algorithm.
Experiments show that our method achieves satisfying approximation quality and learning power while being significantly faster than the original algorithm on large and dense graphs.
Another strength of our method is the transferability: training on one graph, our algorithm can still approximate EPDs well on another graph.
This makes it possible to apply the computationally expensive topological features to a wide spectrum of real-world graphs.
Acknowledgements.
We thank all anonymous reviewers for their constructive feedback very much.
This work of Zuoyu Yan, Liangcai Gao, and Zhi Tang is supported by the projects of National Key R&D Program of China (2019YFB1406303) and National Natural Science Foundation of China (No. 61876003), which is also a research achievement of Key Laboratory of Science, Technology and
Standard in Press Industry (Key Laboratory of Intelligent Press Media Technology).
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] It is discussed in the supplementary material.
(c) Did you discuss any potential negative societal impacts of your work?
[No] The work does not seem to have any negative societal impact.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[No] The data and the experimental details are provided in Section 5 and the supplementary material.
We will release the code once accepted.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] It is illustrated in Section 5 in the paper and in the supplementary material.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] Table 2 and Table 3 are examples.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] It is provided in the supplementary material.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] It is mentioned in the supplementary material.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No] We do not release new assets.
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Supplementary material
In the supplementary material, we provide (1) the related work; (2) the complexity and the correctness of the introduced algorithm; (3) the Union-Find algorithm in a sequential format; (4) additional experimental details, including the introduction of the datasets, and the experimental settings; (5) further experiments, including the evaluation on transferability, the influence of training samples, experiments on other datasets, experiments on other attributes of the model and the limitation of the paper.
A.1 Related Works
Learning with Persistent Homology.
Based on the theory of algebraic topology [40], persistent homology [19, 18] extends the classical notion of homology, and can capture the topological structures (e.g., loops, connected components) of the input data in a robust [10] manner.
It has already been used in various deep learning domains including kernel machines [44, 32, 6], convolutional neural networks [23, 27, 54, 68], transformers [64], connectivity loss [8, 22], and graph representation learning [67, 9, 61, 66, 21, 5, 33].
Some following works propose persistence-inspired frameworks on other tasks such as knowledge graph completion [60].
Neural Algorithm Execution.
Many works have studied neural execution in different domains before [63, 28, 31, 43, 45, 59].
With the rapid development of GNNs in graph representation learning, learning graph algorithms with GNNs has attracted researchers’ attention [53, 51, 56].
These works exploit GNNs to approximate certain classes of graph algorithms, such as parallel algorithms (e.g., Breadth-First-Search) and sequential algorithms (e.g., Dijkstra).
Although the computation of extended persistence diagrams can be written in a sequential-like form, it needs extra steps and considerations.
In our framework, we propose different modules to approximate these steps and achieve satisfying practical performance.
Accelerating Extended Persistent Homology.
In general, computing extended persistent homology relies on the well-known matrix reduction algorithm [11].
Much effort has been made to accelerate the computation, but it still takes matrix multiplication time [50, 16, 14].
For the specific case where the input is a function on a graph G =
( V, E ) , it turns out that one can compute it in O ( | E | log | V | ) time [2, 20].
Nevertheless, this algorithm remains theoretical, and in practice, often a quadratic
O ( | V || E | ) time algorithm is used for its simplicity [61].
Recently, some works have been proposed to accelerate the computation in a data-driven manner [48, 37, 69, 15].
However, these works try to estimate the persistence image [1], a coarsened topological feature rather than the persistence diagram itself, leading to much worse performance in both approximation error and downstream tasks.
Compared with previous works, we propose a novel framework that directly predicts extended persistence diagrams on graphs.
As shown in the experiment, the proposed model has achieved a satisfying approximation error while remaining a high efficiency as well.
A.2 Complexity and Correctness of Algorithm
In this section, we show the complexity and the correctness of Algorithm 2 (which is restated as
Algorithm 5 in the supplementary material.).
A.2.1 Complexity
The computational complexity of the Union-Find algorithm is O ( | E | α ( | E | ))
[13], where α ( · ) is the inverse Ackermann function.
Therefore, we need O ( | V || E | α ( | E | )) time to compute an EPD using
Algorithm 5.
Note this sequential algorithm is not necessarily the most efficient one.
In practice, one may use the quadratic algorithm ( O ( | V || E | ) ) as in [61]. We also note that although not formally published, the best known algorithm for EPD computation is quasilinear, O ( | E | log | V | ) , using the data structure of mergeable trees [2, 20]. But this algorithm remains theoretical so far.
1: Input: graph G =
( V, E ) , filter function f .
2: Initialise-Nodes( V, f ) 3: Q = Sort-Queue ( V ) 4: while Q is not empty do 5: u =
Q. pop-min () 6: for v ∈ G. neighbors ( u ) do 7:
Relax-Edge( u, v, f ) 8: end for 9: end while
1: Input: filter function f , input graph G =
( V, E ) 2: V, E = sorted ( V, E, f ) 3: PD 0 =
Union-Find ( V, E, f ) , PD 1 = {} 4: for i ∈ V do 5:
C i =
{ C i j | ( i, j ) ∈ E, f ( j ) >
f ( i ) } ,
E i =
E 6: for C
i j ∈ C
i do 7: f ( C i j ) =
f ( i ) , E i =
E i − { ( i, j ) } + { ( C i j , j ) } 8: end for 9:
PD i= Union-Find-step ( V + C i − { i } , E i , f, C i ) 10:
PD 1 + = PD i11: end for 12: Output: PD 0 , PD
1: Input: V , E , f , C
2: PD i 1 = {} 3: for v ∈ V do 4: v.value = f
( v ) , v.root = v 5: end for 6: Q =
Sort ( V ) , Q = Q − { v | f ( v ) <
f ( i ) } 7: while Q is not empty do 8: u = Q. pop-min () 9: for v ∈ G. neighbors ( u ) do 10: pu, pv =
Find-Root ( u ) , Find-Root ( v ) 11:
if pu ̸ = pv then 12: s = argmin ( pu.value, pv.value ) 13: l = argmax ( pu.value, pv.value ) 14: l.root
=
s 15: if pu ∈ V i and pv ∈ V
i then 16:
PD i 1 + { ( u.value, l.value ) } 17: end
if 18: end if 19: end for 20: end while 21: Function: Find-Root ( u ) 22: pu = u 23: while pu ̸ = pu.root do 24: pu.root = ( pu.root ) .root , pu
= pu.root 25: end while 26: Return: pu
A.2.2 Correctness
Formally, we restate the theorem below (The theorem is named Theorem 3.1 in the main paper).
For a clear statement, we present the standard EPD computation algorithm in Algorithm 7.
The detailed description of Algorithm 7 is beyond the scope of the paper.
We only introduce the needed information, and refer the readers to [11, 18] for details.
Theorem A.1.
Algorithm 5 outputs the same 1D EPDs as Algorithm 7.
As stated in Section 2 and Section 3 in the paper, for an edge (1-simplex) e ∈ E , it is either paired with a vertex or an edge.
In the former case, the edge, defined as a negative edge, kills a connected component, and gives rise to a 0D persistence point.
In the latter case, the edge, defined as a positive edge (in the ascending filtration), creates a loop during the ascending filtration.
The loop will ultimately be killed by another edge during the descending filtration (defined as a positive edge in the descending filtration).
Hence the positive edge in the ascending filtration is paired with a positive edge in the descending filtration, and gives rise to a 1D extended persistence point.
For simplicity, we will call the positive edges in the ascending filtration as ascending positive edges , and the positive edges in the descending filtration as descending positive edges .
In other words, to compute the 1D EPDs, we can simply find the pairing partner for all positive edges.
In the following paragraphs, we show that Algorithm 5 produces the same extended persistence pair as the standard EPD computation algorithm.
We first present a definition of the “thinnest pair":
Thinnest pair.
Given a filter function f : X → R , the pair of edges ( e 1 , e 2 ) with f ( e 1 ) < f
( e 2 ) is defined as the thinnest pair if the following condition is satisfied: (1) there is a cycle C having eas the lowest edge, and e 2 as the highest edge; (2) for any other cycle with e 1 as the lowest edge, if its highest edge e 2 satisfies that f ( e 3 ) ̸
= f ( e 2 ) , then f ( e 3 ) >
f ( e 2 ) .
Symmetrically, among all cycles having e 2 as the highest edge, e 1 is the lowest edge in a cycle such that this lowest value is the highest possible.
Lemma A.2.
For every ascending positive edge, Algorithm 5 finds its “thinnest pair".
Proof.
Algorithm 5 decomposes the 1D extended persistence pair finding for all edges into pairfinding among all nodes.
In particular, for a given node u , it uses Algorithm 6 to find the pair for its upper edges.
There are two cases:
Case 1.
If the upper edge is an ascending negative edge, then it will kill a connected component, and will not influence the 1D extended persistence pairing.
Case 2.
If the upper edge is an ascending positive edge, it will be paired with the loop once the loop is created in the union-find process.
The edge, called e , is the lowest edge in the loop, called C .
Recall that C is also the first loop that appears in the union-find process with e as the lowest edge.
Therefore C is guaranteed to contain the highest value which is the lowest possible.
According to the definition, this will lead to the “thinnest pair" 6 .
Algorithm
The standard EPD computation algorithm
Lemma A.3.
In the descending filtration of Algorithm 7, an edge e is paired if a loop C has already appeared, with e as its lowest edge.
Proof.
Every column/row of the binary matrix M shown in Algorithm 7 corresponds to a simplex (node/edge) in the input graph G .
For simplicity, we replace the index in M with the simplex it represents in the rest of the paper.
For an edge e , low M ( e ) denotes its lowest row e 1 , with
M [ e, e 1 ] = 1 .
After the matrix reduction process, e and e 1 will form an extended persistence pair.
It has been shown in [11, 61] that for a loop, its highest edge and lowest edge form its extended persistence pair.
In other words, e and e 1 are the lowest and highest edges of the loop they form.
Assume that a loop C has already aroused with e as its lowest edge, then there are two cases for the highest edge e 1 in C :
Case 1.
If there does not exist an edge e 2 , that appears before e 1 in the descending filtration, with low M ( e 2 ) =
e = low M ( e 1 ) , then e will be paired with e 1 in Algorithm 7.
Case 2.
If there exists an edge e 2 , that appears before e 1 in the descending filtration, with low M ( e 2 ) =
e = low M ( e 1 ) , then e will be paired with e 2 or even other edges that appears earlier than e 2 .
Among all possibilities, e is paired before C appears in Algorithm 7.
In other words, e will be paired with e 1 or before e 1 in Algorithm 7.
Lemma A.4.
Algorithm 7 finds the “thinnest pair" for all positive edges.
Proof.
During the descending filtration, when an edge e = u i u
j appears, there are two cases:
Case 1.
e is a negative edge, and will kill a connected component.
This will not influence the 1D extended persistence pair.
Case 2.
e is a positive edge, and will be paired with an ascending positive edge e 1 = u a u b .
Assume that e 1 does not construct the “thinnest pair" with e , then there exists an edge e 2 = u c u d , that forms the “thinnest pair" with e .
We can observe a loop C =
u a → u c u d → u b , in which all edges born earlier than e in the descending filtration, and e 1 is the latest edge in the ascending filtration.
A toy example is shown in Figure 3, where e = u 1 u 3 , e 1 = u 4 u 5 , e 2 = u 3 u 4 , and C is the red loop.
Then according to Lemma A.3, e 1 will be paired no later than C appears.
In other words, it has already been paired before e appears.
Therefore, the assumption is wrong, and Algorithm 7 will find the “thinnest pair" for all positive edges.
According to Lemma A.2 and Lemma A.4, Algorithm 5 will produce the same 1D extended persistence pair as Algorithm 7.
Therefore, they output the same 1D EPD.
A.3 Union-Find Algorithm
In this section, we rewrite the well-known Union-Find algorithm [13] in a sequential format.
The algorithm is listed in Algorithm 8.
Therefore we can use the proposed framework to estimate the 0D
EPDs.
Algorithm 8 Union-Find (Sequential) 1: Input: f 2: PD do 4: 5: end
6: Q Sort 7: while do 8: neighbors do 10: pu pv u
Find
Root pv then 12: argmax pu value pv value ) 13: l.root = s 14:
PD 0 + { ( l.value, u.value ) } 15: end if 16: end for 17: end while 18: Function: Find − Root ( u ) 19: pu
= u 20: while pu ̸ = pu.root do 21: pu.root = ( pu.root ) .root , pu
= pu.root 22: end while 23: Return: pu
A.4 Experimental details A.4.1 Datasets.
In this paper, we exploit real-world datasets including:
[Table] Citation networks: Cora, Citeseer, and PubMed [46] are standard citation networks where nodes denote scientific documents and edges denote citation links.
Amazon shopping records: In Photo and Computers [47], nodes represent goods, edges represent that two goods are frequently brought together, and the node features are bag-ofwords vectors.
Coauthor datasets: In CS and Physics [47], nodes denote authors and edges denote that the two authors co-author a paper.
The detailed statistics are available in Table 5.
A.4.2 Experimental Details
In this section, we mainly present the experimental settings on neural estimation, as for the setting in downstream graph representation learning tasks, we are consistent with [67, 61].
Following the settings in [67, 61], we extract 2-hop neighborhoods of all the nodes in Cora, Citeseer,
PubMed and 1-hop neighborhoods of all the nodes in Photo, Computers, Physics, and CS.
In the training process, we only adopt the W 2 distance between the predicted diagram and the ground truth diagram as the loss function, while the PIE between the predicted persistence image and the ground truth persistence image only serves as an evaluation metric.
We adopt Adam as the optimizer with the learning rate set to 0.002 and weight decay set to 0.01.
We build a 4-layer GNN framework with dropout set to 0.
In the training process, we set the batch size to 10, and the training epoch to 20.
In this paper, we also exploit a 2-layer MLP to transform the node embedding obtained by the GNN to the persistence points on edges.
In the framework, PRELU is adopted as the activation function, the dimension of hidden layers is set to 32, and the dimension of the output persistence image is 25.
All the experiments are implemented with two Intel Xeon Gold 5128 processors,192GB RAM, and 10 NVIDIA 2080TI graphics cards.
Notice that in the normal computation of Wasserstein distance between PDs, the persistence points can be paired to the diagonal or the persistence points in the other diagram.
However, in the experiments, we observe that with this loss function as the supervision, the model may converge to local minima, e.g., all the predicted persistence points are paired to diagonal.
Therefore, the predicted points all converge to the diagonal and contain no topological information.
To avoid such situations, we force the predicted points to pair with the persistence points in the ground truth diagram rather than the diagonal in the training stage.
In the reference stage, we report the normal W 2 distance between persistence diagrams, that is, to let the predicted points pair with the diagonal.
A.4.3 About the assets we used
Our model is experimented on benchmarks from [38, 26, 17, 46, 47] provided under MIT license.
A.5 Additional Experiments A.5.1 Experiments on transferability
In this section, we design experiments to evaluate the transferability of PDGNN in terms of different graph structures.
Our aim is to evaluate whether the pre-trained model can estimate EPDs on totally unseen graphs.
Therefore, we evaluate the models pre-trained on Photo on other datasets, and report [Table] [Table] the W 2 distance between the predicted diagrams and ground truth EPDs.
Notice that we only use
Ollivier-Ricci curvature [41] as the filter function.
The results are shown in Table 6.
In Table 6, “Pre-train" is to directly predict the EPDs with the pre-trained model, and “Fine-tune" is to fine-tune an epoch on the new datasets, and then predict the EPDs.
As shown in Table 6, directly predicting the EPDs with the pre-trained model perform comparably with the standard settings among datasets.
We also observe that with only a one-epoch fine-tuning, the pre-trained model can achieve almost an equal performance compared with the standard setting.
It justifies the fine transferability of
PDGNN.
Therefore, in a totally new environment, instead of training the uninitialized models for many epochs, we can simply fine-tune or even directly use the pre-trained model to estimate EPDs on new graph structures.
A.5.2 Evaluation on the influence of training samples
In this section, we evaluate the influence of training samples on PDGNN.
We aim to show that the model can reach an acceptable performance with only a small number of training samples.
Recall that for a given graph, we extract the k -hop neighborhoods of all the nodes and randomly select 80% of these vicinity graphs to train PDGNN.
For a thorough evaluation, we train PDGNN with 5/10/20/40% vicinity graphs in this experiment and report the W 2 distance of persistence diagrams, the PIE of persistence images, and the node classification accuracy (NCA) in Table 7.
We also visualize the influence in Figure 4 and Figure 5.
As shown in Figure 4, the training error tends to converge as the training samples gradually increase.
Considering that the W 2 distance and PIE cannot directly reflect the learning power as NCA does, we select a vicinity graph in Cora which is hard for PDGNN to learn and visualize in Figure 5.
As shown in the figure, as the number of training samples increases, we find that PDGNN can gradually capture the ground truth persistence points in the up y-axis and the up-right diagonal with much less noise.
The number of training samples may help the model learn the hard samples better.
We also observe that in Table 7, PDGNN reaches a comparable performance on NCA with much fewer training samples.
The observation shows that a little perturbation on the persistence image will not influence its structural information very much.
Combining the observation in Section A.5.1 and Section A.5.2, we can safely conclude that our model can be easily generalized to other frameworks.
PDGNN does not need many training samples to reach an acceptable performance, and it can be easily transferred to totally unseen graphs.
A.5.3 Experiments on graph classification datasets.
In the experiment part, we only consider predicting EPDs of the k -hop neighborhoods of the original graphs.
Even if these vicinity graphs can be large and dense, there can be structural differences between these vicinity graphs and other real-world graphs.
In this section, we do further experiments on graph classification datasets, in which we approximate the EPDs of the [Figure] [Table] [Figure]
real-world graphs rather than the vicinity graphs.
We exploit various datasets from the TU Dortmund
University [38], benchmarking-GNN [17], and OGB [26].
The detailed information of these datasets and the approximation error are all available in Table 8.
Notice that we do not add Ollivier-Ricci curvature as the filter function here, because computing the filter function on all the graphs will bring too much computational cost.
Comparing the results from
Table 8 and the results on vicinity graphs, we observe that the performance on graph classification datasets is slightly worse than the performance on vicinity graphs.
This may be due to the fact that in graph classification datasets, the training samples can be very small, e.g., there are only 188 graphs in
MUTAG, therefore the training is under-fit.
On the contrary, the satisfying approximation quality on
OGBG-MolHIV and ZINC can be due to their large number of training samples.
To evaluate the results more clearly, we also visualize some selected examples in Figure 6.
As shown in the figure, in most situations, PDGNN can well estimate the EPDs on these graphs, and the Wdistance around 0.3 is generally an acceptable result. [Figure] [Table]
A.5.4 Why not directly approximating PIs
We believe directly estimated PIs will lose important structural information that can be crucial for downstream tasks.
PI is only an approximation of the persistence diagram.
The L2 distance between
PIs does not accurately reflect the true Wasserstein distance between diagrams.
Therefore, using an
L2-distance-based loss to directly learn the PI may lead to the loss of important structural information carried by a diagram.
An example is provided in Figure 7.
For a sample vicinity graph from Cora, we compare the ground truth PI (computed from the ground truth diagram), the PI computed from the diagram estimated by our method PDGNN, and the PI directly estimated by GIN.
Both estimated PIs have similar L2 distances from the ground truth PI.
But we observe that the PI estimated by PDGNN has a very similar spatial distribution to the ground truth PI.
This structural property, however, is not preserved by the directly estimated PI.
Such loss of structural information of directly estimated PIs, although not captured by the L2 error, partially explains their worse representation power.
In Tableand Table 3 in the main paper, the directly estimated PIs (PEGN(GIN_PI) and TLC-GNN(GIN_PI)) perform worse in the downstream task.
This choice of estimating diagrams instead of PIs is a part of the overarching theme of our paper.
Note that the main contribution of our paper is to transfer a complicated and uncontrollable learning process to a controllable process with algorithmic insight.
This general principle also applies to our learning algorithm.
We decompose the diagram computation algorithm into sub-algorithms, which can be approximated well by a GNN.
This is the reason PDGNN approximates the diagrams much better than other baselines in Table 1 in the main paper.
A.5.5 Experiments on large and sparse datasets.
In the experiments in the main paper, the input is the k -hop vicinity graphs.
On citation graphs, the vicinity graph remains small.
On these small graphs, the exact sequential algorithm like Gudhi has less overhead, and thus is unsurprisingly faster.
Indeed, on large and sparse graphs, our method outperforms strong baselines like Gudhi significantly.
In Table 9, we compare the running time (in seconds) on popular citation networks including Cora,
Citeceer, and PubMed.
For each graph, we run experiments on the largest connected subgraph.
We also report the number of nodes/edges of the selected subgraph.
[Table]
[Table]
A.5.6 Experiment on the choice of filter functions/other graph metrics.
In Table 10, we set degree centrality and clustering coefficient as the filter function, follow the settings in Table 1 and report the approximation error on Cora, Citeseer, and PubMed.
We also report computation time following the setting in Table 4.
The only difference is that below we report the time to generate all vicinity graphs (rather than 1000 graphs as in Table 4).
We observe that (1) the filter function only has a minor influence on inference/computation speed, for both the sequential algorithm and ours; (2) the filter function does influence the approximation error.
The reason is that different filter functions have different ranges; functions with larger ranges tend to have larger approximation errors, especially on PIE.
This is another evidence that the distance function on PIs is not very robust for learning.
A.5.7 Experiments on the threshold value of average node/edge to decide which method is the fastest to compute/estimate EPDs.
To find the threshold, we use the well-known Stochastic Block Model (SBM)
[24] to generate synthetic graphs.
We set the number of nodes in these synthetic graphs from 200 to 300, with 10 as the step.
In these graphs, we randomly generate 5 different clusters, and set the probability of edges intra-cluster to 0.4, and the probability of edges inter-cluster to 0.1.
In this way, we can obtaingraphs with different nodes and edges.
We set node degree as the filter function, and add experiments on the largest connected components of these 11 graphs.
The information of the selected connected graphs and the running time (second) are listed in Table 11.
As shown in the Table, the threshold is around 100 nodes / 820 edges.
We also evaluate the influence of density.
We fix the node number of the SBM model to 250, and set the probability of edges intra-cluster from 0.5 to 0.7, and the probability of edges inter-cluster from 0.05 to 0.15.
The steps for intra-cluster and inter-cluster are 0.02 and 0.01, respectively.
In this [Table] way, we can obtain 11 graphs with the same nodes and different edges.
We set node degree as the filter function, and add experiments on the largest connected components of these 11 graphs.
The information of the selected connected graphs and the running time (second) are also listed in Table 12.
As shown in the Table, the threshold is around 100 nodes / 766 edges.
A.5.8 Limitation of the paper.
First, in certain cases like Figure 6 (d), the model only captures a tendency of the EPD.
This can be because that the distribution of the EPD of the selected graph is seldom in the training samples.
Therefore, it is hard for the model to estimate these EPDs correctly.
Second, topological features are just one side of the data.
In many cases, only using topological features such as EPDs to represent the information of graphs is not enough.
A better way is to introduce other information such as the semantic information of graphs as complementary.