Tree Mover’s Distance: Bridging Graph Metrics and Stability of Graph Neural Networks
Abstract
Understanding generalization and robustness of machine learning models funda- mentally relies on assuming an appropriate metric on the data space.
Identifying such a metric is particularly challenging for non-Euclidean data such as graphs.
Here, we propose a pseudometric for attributed graphs, the Tree Mover’s Distance (TMD), and study its relation to generalization.
Via a hierarchical optimal transport problem, TMD reflects the local distribution of node attributes as well as the distribution of local computation trees, which are known to be decisive for the learning behavior of graph neural networks (GNNs).
First, we show that TMD captures properties relevant to graph classification: a simple TMD-SVM performs competi- tively with standard GNNs.
Second, we relate TMD to generalization of GNNs under distribution shifts, and show that it correlates well with performance drop under such shifts. The code is available at https://github.com/chingyaoc/TMD .
Introduction
Understanding generalization under distribution shifts – theoretically and empirically – relies on an appropriate measure of divergence between data distributions.
This, in turn, typically demands a metric on the data space that indicates what kinds of data points are “close” to the training data.
While such metrics may be readily available for Euclidean spaces, they are more challenging to determine for non-Euclidean data spaces such as graphs with attributes in R p , which underlie graph learning methods.
In this work, we study the question of a suitable metric for message passing graph neural networks (GNNs).
An “ideal” metric for studying input perturbations captures the invariances and inductive biases of the model we are examining.
For instance, since graph isomorphism is a difficult problem [18], this metric is expected to be a pseudometric, i e., will fail to distinguish certain graphs.
These “failures” should be aligned with the GNNs’ invariances.
Moreover, several recent works highlight the importance of local structures – computation trees resulting from unrolling the message passing process – for the approximation power of GNNs and their inability to distinguish certain pairs of graphs [2, 20, 34, 55].
Works on out-of-distribution generalization of GNNs mostly focus on specific types of instances where a trained model may fail miserably [58, 54], without specifying the behavior for gradual shifts in distributions.
Yehudai et al.
[58] show that a sufficiently large, unrestricted GNN may predict arbitrarily on previously unseen computation trees.
In practice, we may observe a more gradual change, depending on the magnitude of change of the tree, in terms of both structure and node attributes, and the capacity of the aggregation function.
In summary, we desire a metric that reflects the structure of computation trees and the distribution of node attributes within trees.
Many distances and kernels between graphs have been proposed [7], several based on local structures [21], and some using structure and attributes [44].
Closest to our ideal is the Wasserstein Weisfeiler-Leman pseudometric [46], which computes an optimal transport distance between node embeddings of two graphs.
The embeddings are computed via message passing, which aligns with the computation of
GNNs, but loses structural information within trees.
Hence, we propose the Tree Mover’s Distance (TMD) , a pseudometric on attributed graphs that considers both the tree structure and local distribution of attributes.
It achieves this via a hierarchical optimal transport problem that defines distances between trees.
First, we observe that the TMD captures properties that capture relationships between graphs and common labels: a simple SVM based on TMD performs competitively with standard GNNs and graph kernels on graph classification benchmarks.
Second, we relate TMD to the performance of GNNs under input perturbations.
We determine a Lipschitz constant of GNNs with respect to TMD, which enables a bound on their target risk under domain shifts, i.e., distribution shifts between training and test data.
This bound uses the metric in two ways: to measure the distribution shift, and to measure perturbation robustness via the
Lipschitz constant.
Empirically, we observe that the TMD correlates well with the performance of
GNNs under distribution shifts, also when compared to other distances.
We hence hope that this work inspires further empirical and theoretical work on tightening the understanding of the performance of
GNNs under distribution shifts.
In short, this work makes the following contributions:
• We propose a new graph metric via hierarchical optimal transport between computation trees of graphs, which, in an SVM, leads to a competitive graph learning method; • We bound the Lipschitz constant of message-passing GNNs with respect to TMD, which allows to quantify stability and generalization; • We develop a generalization bound for GNNs under distribution shifts that correlates well with empirical behavior.
Related Works
Graph Metrics and Kernels Measuring distances between graphs has been a long-standing goal in data analysis.
However, proper metrics that distinguish non-isomorphic graphs in polynomial time are not known.
For instance, Vayer et al.
[49] propose a graph metric by fusing Wassestein distance [50] and Gromov-Wasserstein distance [32].
Similar to classic graph metrics [9, 40], the proposed metric requires approximation.
Closely related, graph kernels [52, 7] have gained attention.
Most graph kernels lie in the framework of R -convolutional [21], and measure similarity by comparing substructures.
Many R -convolutional kernels have limited expressive power and sometimes struggle to handle continuously attributed graphs [46].
In comparison, TMD is as powerful as the WL graph isomorphism test [53] while accommodating graphs with high dimensional node attributes.
Importantly, TMD captures the stability and generalization of message-passing GNNs [27, 55].
Stability and Generalization of Graph Neural Networks
A number of existing works study stability of GNNs to input perturbations.
Spectral GNNs are known to be stable to certain perturbations, e.g., size, if the overall structure is preserved [19, 25, 26, 19].
For message passing
GNNs, Yehudai et al.
[57] study perturbations of graph size, and demonstrate the importance of local computation trees.
Xu et al.
[54] study how the out-of-distribution behavior of aggregation functions may affect the GNN’s prediction.
These studies motivate to include both computation trees and inputs to aggregation functions in the TMD.
Finally, a number of works study within-distribution generalization [15, 20, 28, 42].
Background on Optimal Transport
We begin with a brief introduction to Optimal Transport (OT) and earth mover’s distance.
The earth mover’s distance, also known as Wasserstein distance, is a distance function defined via the transportation cost between two distributions.
Let X
= { x i } mi =1 and Y =
{ y i } mj
=1 be two multisets of m elements each.
Let C ∈ R m × m be the transportation cost for each pair: C ij = d ( x i , y j ) where d is the distance between x i and y j .
The earth mover’s distance solves the following OT problem:
[Equation]
where Γ is the set of transportation plans that satisfies the flow constrain γ 1 m = γ ⊤ 1 m = 1 m .
In this work, we adopt the unnormalized version of the earth’s mover distance:
[Equation]
Comparing to classic OT, unnormalized OT preserves the size information of multisets X and Y . [Figure] BlankTree BlankTree
Tree Mover’s Distance: Optimal Transport on Graphs
Next, we introduce tree mover’s distance , a new distance for graphs.
Let G =
( V, E ) denote a graph with sets of nodes V and edges E .
The graph may have node features x v ∈ R p for v ∈ V .
If not, we simply set the node feature to a scalar x v = 1 for all v ∈ V .
Local structures of graphs are characterized by computation trees [23].
In particular, computation trees are constructed by connecting adjacent nodes recursively.
Definition 1 (Computation Trees) .
Given a graph G =
( V, E ) , let T 1 v = v , and let T Lv be the depth- L computation tree of node v constructed by connecting the neighbors of the leaf nodes of T L − 1 v to the tree.
The multiset of depth- L computation trees defined by G is denoted by T LG : = { T Lv } v ∈ V .
Figure 2 illustrates an example of constructing computation trees.
Computation trees, also referred to as subtree patterns, have been central in graph analysis [38, 53] and graph kernels [39, 44].
Intuitively, the computation tree of a node encodes local structure by appending the neighbors to the tree in each level.
If depth- L computation trees are the same for two nodes, they share similar neighborhoods up to L steps away.
Therefore, an intuitive way to compare two graphs is by measuring the difference of their nodes’ computation trees [44, 53].
In this work, we adopt optimal transport, a natural way to compute the distance between two sets of objects with, importantly, an underlying geometry.
We will begin by defining the transportation cost between two computation trees. The cost then gives rise to the tree mover’s
distance, an extension of earth mover’s distance to multisets of trees.
4.1 Distance between Trees via Hierarchical OT
Let T =
( V, E, r ) denote a rooted tree.
We further let T v be the multiset of computation trees of the node v which consists of trees that root at the descendants of v .
Determining whether two trees are similar requires iteratively examining whether the subtrees in each level are similar.
For instance, two trees T a and T b with roots r a and r b are the same if x r a = x r b and T r a = T r b .
This motivates us to define the distance between two trees by recursively computing the optimal transportation cost between their subtrees.
Nevertheless, the number of subtrees could be different for r a and r b , i.e., |T r a | ̸ = |T
r b | (see Figure 1 left).
To compute the OT between sets with different sizes, unbalanced
OT [11, 41] or partial OT
[10] are usually adopted.
Inspired by [10], we augment the smaller set with blank trees .
Definition 2 (Blank Tree) .
A blank tree T 0 is a tree (graph) that contains a single node and no edge, where the node feature is the zero vector [-
-]0 p ∈ R p , and T n 0 denotes a multiset of n blank trees.
Definition 3 (Blank Tree Augmentation) .
Given two multisets of trees T u , T v , define ρ to be a function that augments a pair of trees with blank trees as follows:
[Equation]
If |T v | < |T u | , ρ augments T v by |T u | − |T v | blank trees to make the two multisets contain the same number of trees, and hence allows to define a transportation cost between two multisets of trees with different sizes.
In particular, the transportation costs of additional trees are simply the distance to the blank trees.
The distance between a tree and a blank tree can be interpreted as the norm of the tree .
In our case, the blank tree can be viewed as the origin, as it is a tree with the simplest structure and zero feature vector.
Equipped with ρ , we define the distance between two rooted trees as follows.
Definition 4 (Tree Distance) .
The distance between two trees T a , T b is defined recursively as
[Equation]
where L = max( Depth ( T a ) , Depth ( T b )) and w : N → R + is a depth-dependent weighting function. Here, OT TD w is the OT distance defined in (2) with TD w as the metric.
Figure 1 gives an illustration of computing tree distances.
The tree distance TD w ( T a , T b ) aims to optimally align two trees T a and T b by recursively comparing their roots and subtrees.
Calculating TD w ( T a , T b ) requires calculating the OT between augmented subtrees ρ ( T r a , T r b ) , where the cost function of OT is TD w again.
This formulates a hierarchical optimal transport problem: the distance of two trees is defined via the distances of subtrees, where the importance of each level is determined by the weight w ( · ) .
Increasing w ( · ) upweights the effect of nodes in the subtrees.
While the weights may be arbitrary, we found that for many applications, using a single weight for all depths yields good empirical performance, as section 4.3 shows.
The role of weights will be more significant when we use TMD to bound the stability of GNNs.
4.2 From Tree Distance to Graph Distance
Next, we extend the distance between trees to a distance between graphs.
By leveraging the tree distance TD w ( · , · ) , we introduce the tree mover’s distance (TMD) , a distance for graphs, by calculating the optimal transportation cost between the graphs’ computation trees.
Definition 5 ( Tree Mover’s Distance ) .
Given two graphs G a , G b and w, L ≥ 0 , the tree mover’s distance between G a and G b is defined as
[Equation]
where T LG a and T LG b are multisets of the depth- L computation trees of graphs G a
and G b , respectively.
Figure 2 illustrates the computation of TMD.
Intuitively, TMD is the minimum cost required to transport node-wise computation trees from one graph to another.
The blank tree augmentation ρ is again adopted to handle graphs with different numbers of nodes.
The next theorem shows that TMD is a pseudometric on attributed graphs.
Theorem 6 (Pseudometric) .
The tree mover’s distance TMD Lw is a pseudometric for finite L > 0 .
In particular, the tree mover’s distance satisfies (1)
TMD Lw ( G a , G a ) = 0 , (2)
TMD Lw ( G a , G b ) =
TMD Lw ( G b , G a ) , and (3) TMD Lw ( G a , G b ) ≤ TMD Lw ( G a , G c ) + TMD Lw ( G c , G b ) for any graphs
G a , G b , G c .
However, in some cases, the distance TMD Lw ( G a , G b ) can be zero even if G a ̸ = G b .
This is reasonable, as computing graph isomorphism is not known to be solvable in polynomial time.
Nevertheless, TMD can provably distinguish graphs that are identifiable by the (1-dimensional)
Weisfeiler-Leman graph isomorphism test [53].
Theorem 7 (Discriminative Power of TMD) .
If two graphs G a , G b are determined to be nonisomorphic in WL iteration L and w ( l ) > 0 for all 0 < l ≤ L + 1 , then TMD L +1 w ( G a , G b ) > 0 .
The unnormalized OT and blank tree augmentation are essential to prove Theorem 16.
The tree mover’s distance can be exactly computed by solving optimal transport.
In addition, TMD remains highly expressive on graphs with high dimensional continuous attributes, where most R -convolutional graph kernels struggle [46].
The discriminative power of TMD can be further strengthened by augmenting node attributes e.g. with positional encodings [16, 29].
The OT cost between node representations in different graphs is reminiscent of the recently proposed
Wasserstein WL (WWL) [46].
WWL uses the distance of node embeddings as a ground metric, where the node embeddings are computed via L iterations of message passing with average-aggregation; in contrast, TMD computes a distance that aligns the trees, retaining more structural information.
Even though an aggregation with nonlinearities can retain tree isomorphism information [55], similar to the hashing applied in the WL test (WWL uses only linear aggregations), the hierarchical OT is a more direct graded distance measure of trees.
Vayer et al.
[49] define a metric that uses a
Gromov-Wasserstein distance [32] between nodes, but need to approximate the GW computation. [Figure] MUTAG [Table]
Numerical Computation with Dynamic Programming One can numerically compute TMD with dynamic programming.
Starting with pair-wise distances (TMD L =1 w ) between node features across the two graphs, we then iteratively compute TMD L = k w between depth- k computation trees from k =to L according to Definition 4.
Let D be the maximum degree of a node in the two graphs and τ ( m ) be the complexity of computing OT between sets of cardinality m .
In each level, we have to perform
OT of sets contain at most D elements for N nodes.
Including the last OT between nodes of graph, the overall time complexity of computing TMD Lw is O ( τ ( N ) + LNτ ( D )) .
The time complexity for exact OT by solving linear programming is τ ( m ) =
O ( m 3 log( m ))
One can use faster approximation of OT, e.g., near linear time complexity [1], but we use exact OT throughout all the experiments, implemented with the POT library [17].
4.3 Experiments
We verify whether the tree mover’s distance aligns with the labels of graphs in graph classification tasks: the TUDatasets [35], which contain graphs with discrete node attributes (MUTAG, PTC-MR, PROTEINS, NCI1, NCI109) and graphs with continuous node attributes (BZR, COX2). Specifically, we run a support vector classifier (C = 1) with indefinite kernel e − γ × TMD ( · , · ) , which can be viewed as a noisy observation of the true positive semidefinite kernel [31]. The γ is selected via cross-validation from { 0 . 01 , 0 . 05 , 0 . 1 } and the weights w ( · ) are set to 0 .
5 for all depths.
For comparison, we use graph kernels based on graph subtrees: Ramon & Gärtner kernel [39], WL subtree kernel [44]; two widely-adopted GNNs: graph isomorphism network (GIN) [55], graph convolutional networks (GCN) [27]; and the recently proposed graph metrics FGW [49] and WWL [46].
Table 1 reports the mean and standard deviation over 10 independent trials with 90%/10% train-test split.
The performances of the baselines are taken from the original papers.
TMD outperforms or matches the performances of state-of-the-art GNNs, graph kernels, and metrics, implying that it captures meaningful structural properties of graphs. Appendix C shows further graph clustering and t-SNE visualization [48] results. WWL TMD
TMD Parallel
DD 7.
24.44 NCI 0. Computation Complexity To examine the computation complexity, we compare the runtime between WeisfeilerLehman (WWL) kernels, TMD, and a parallel version of TMD on DD and NCI1 datasets [35], where DD contains large graphs (avg. #node: 284.32, avg.
#edge: 715.66) and NCI1 contains small graphs (avg.
#node: 29.87, avg.
#rdge: 32.30).
Here, we additionally consider a parallel version of TMD, where the tree OTs in each level are executed simultaneously with 3 processes.
The average runtime over 200 pairs is shown in Table 3. Note that the time complexity of WWL is τ ( m ) = O ( m 3 log( m )) .
The runtime of parallelized TMD is roughly three times larger than WWL on DD, which is much faster than the (worst case) theoretical Big-O analysis. In datasets contain small graphs such as NCI1, TMD without parallelization works well.  TMD and Stability of Graph Neural Networks Next, we relate TMD to the perturbation stability of message passing GNNs.
In particular, we use the
Lipschitz constant, which relies on an underlying metric – here, a metric over graphs.
We observe that TMD is a meaningful pseudometric in this case.
The resulting Lipschitz allows to analyze the stability of GNNs under perturbations and generalization bounds under distribution shifts. 5.1 Lipschitz Constant of Message Passing Graph Neural Networks For simplicity, we consider graph binary classification with the Graph Isomorphim Network (GIN) [55], one of the most widely applied and powerful GNNs. In particular, we consider the following message passing rules of a L -layer GIN:
[Equation] [Equation]  [Equation]
GraphReadout [Equation]   where ϕ ( l ) : R d → R d and ϕ ( L +1) : R d → R are learnable functions with Lipschitz constant K ( l ) ϕ and the initial state is set to the node feature z (0) v = x v .
The ϵ > 0 is a weighting term between the center node and the neighbors. Note that the original formulation of GIN [55] weights the center node with layer dependent ϵ instead of neighbors. We adopt the form above with the purpose to simplify the notation of TMD. One can easily derive an equivalent form by changing the weight function w ( · ) of TMD to recover the original formulation of GIN.
For simplicity, we set ϵ = 1 in all experiments. This does not affect the empirically performance of GIN as the original paper shows [55].
A graph level binary classifier is constructed based on the logits
h ( G ) . The next theorem bounds the Lipschitz constant of {+GIN with respect to TMD. Although TMD is a pseudometric, it satisfies that TMD Lw ( G a , G b ) > 
+}
if G a , G b are distinguished by L iterations of the WL test. Since GIN(G a ) = GIN(G b ) for all graphs where WL fails, it holds that, if GIN(G a ) ̸ = GIN(G b ) then TMD Lw ( G a , G b ) > 0 .
That is, for all triples of graphs that GIN distinguishes pairwise, TMD is a metric. Theorem 8 ( Lipschitz Constant of GIN ) . Given an L -layer graph neural network h : X → R and two graph G a , G b ∈ G , we have
[Equation]
where w ( l ) = ϵ · P l − 1 L +1 /P lL +1 for all l ≤ L and P lL is the l -th number at level L of Pascal’s triangle.
2   1 41       1 weights L = The result can be extended to other GNNs (Appendix B). Interestingly, the weights in each level follow a simple rule determined by Pascal’s triangle.
Figure 4 illustrates the weights when
L = 4 , where the weights w ( l ) gradually decrease from 4 ϵ to ϵ/ 4 as l becomes smaller.
Note that the hyperparameter w of TMD is independent of the parameters of the
GNN.
Moreover, Theorem 8 shows that the Lipschitz constant of GNNs under TMD admits a simple form (cid:81)
L +1 l =1 K ( l ) ϕ : the product of Lipschitz constants across layers, similar to fully connected networks [33, 37]. 0 Layer r = 0. r = 0. r = 0.
r = 0.
r = 0.
r = 0.
r = 0.
r = 0.
[Figure] 5.2 Stability of GNNs under Graph Perturbation The Lipschitz constant is a common criterion to assess the stability of the neural networks to small perturbations [51]. Theorem 8 implies that the output variation of GNNs under graph perturbation can be bounded via the TMD between the original graph and the perturbed one.
In this section, we analyze the stability of GNNs in more detail by dissecting TMD under three types of graph perturbation: (1) node drop; (2) edge drop; and (3) node feature perturbations. Proposition 9 (Node Drop) . Given a graph G = ( V, E ) , let G ′ be the graph where node v ∈ V is dropped.
Then the tree mover’s distance between G and G ′ can be bounded by
[Equation]
where Width l ( T ) is the width of l -th level of tree T and λ 1 = 1 , λ l = (cid:81) l − 1 j =1 w ( L + 1 − j ) . The bound is controlled by two factors: (1) tree size and (2) tree norm (distance from the blank tree) of the computation tree T Lv . A node v with a large computation tree size implies that many nodes can be reached from v .
Deleting v from the graph then significantly changes the computation trees of those reachable nodes, while the magnitude of the variation is controlled by the tree norm. Proposition 10 (Edge Drop) . Given a graph G = ( V, E ) , let G ′ be the graph where edge ( u, v ) ∈
E is dropped.
The tree mover’s distance between G and G ′ can be bounded by
[Equation]
The bound takes a similar form as with the node drop, but includes the effects of both node v and u .
In particular, the tree size of v ( u ) will control how many computation trees of u ( v ) will be dropped.
By using Proposition 9 and 10, one can derive bounds for dropping multiple nodes and edges, or even edge rewiring .
Note that adding nodes or edges is equivalent to the analysis above.
Proposition 11 (Node Perturbation) .
Given a graph G =
( V, E ) , let G ′ be the graph where node feature x v is perturbed to x ′ v .
The tree mover’s distance between G and G ′ is equal to
[Equation]
Different from Proposition 9, the magnitude of the perturbation is controlled by the norm of the perturbation instead of the norm of the computation tree.
[Figure]
[Figure]
5.3 Experiments
Correlation between TMD and GNN output perturbation We now empirically examine the theoretical analysis with experiments on the MUTAG dataset [14].
Results for other datasets can be found in Appendix C. In particular, we measure to what extent candidate graph distances capture input perturbations that lead to output perturbations in the GNN.
We train graph isomorphism networks [55] with varying numbers of message passing layers and plot the relation between input variation TMD L +1 w ( G a , G b ) and output variation ∥ h ( G a ) − h ( G b ) ∥ for randomly sampled pairs ( G a , G b ) in Figure 5. For comparison, we also plot the input variations measured by the recently proposed graph metric WWL [46].
We can see that TMD strongly correlates with the output variation with large Pearson correlation coefficient, supporting the approach of defining the Lipschitz constant with respect to TMD, as in Theorem 8.
In contrast, WWL barely captures the input graph perturbations that lead to output variation of GNNs.
Stability under small Perturbations Next, we plot the output variation of 3-layer GNNs and the TMD under random graph perturbations in Figure 6.
For node perturbations, we change the discrete node attribute for randomly sampled nodes. We additionally plot the Lipschitz bound with empirical estimated Lipschitz constant max G a ,G b ∈ S ∥ h ( G a , G b ) ∥ / TMD L +1 w ( G a , G b ) , where S is a set of samples. We refer reader to [36, 47] for analyses on approximation error of estimating Lipschitz constants from finite samples. We can see that the bound is reasonably tight and estimates the effect of perturbations across different degrees.
Generalization of GNNs under Distribution Shifts
Finally, we relate the Lipschitz condition of GNNs to the generalization error under distribution shifts by extending the results from [43]. Consider a binary classification task with input space X and output space Y . In domain adaptation [13], a pair of source and target distributions µ S , µ T over X × Y are given. Let p S and p T denote the respective marginals on the input space X .
In unsupervised domain adaptation, the learning algorithm obtains labelled source samples from µ S and unlabelled target samples from p T . To estimate the adaptability of a hypothesis h , i.e., its generalization to the target distribution, we aim to bound the target risk R T ( h ) = E x,y ∼ µ T [ 1 h ( x ) ̸ = y ] relative to the source risk
R S ( h ) = E x,y ∼ µ S [ 1 h ( x ) ̸ = y ] [5, 13, 60]. For instance, Shen et al.
[43] bound the target risk via the source risk and the Wasserstein-1 distance W 1 between source and target distributions.
Theorem 12 (Shen et al. For all hypotheses h ∈ H , the target risk is bounded as [Equation] where K is the Lipschitz constant of h and λ H is the best joint risk
λ H :=
inf h ′ ∈H [ R S ( h ′ ) [-> 
-]+ R T ( h ′ )] . This bound relies on being able to measure Wasserstein distance between the two data distributions, which demands a ground metric on the data space, and an associated Lipschitz constant of the model h . The TMD and the resulting Lipschitz constant in Section 5.1 make this bound applicable to message passing GNNs, too.
In particular, for GNNs that satisfy the Lipschitz constant in Theorem 8, the domain discrepancy W 1 ( p S , p T ) is defined as [Equation] F M MM F R R
[Figure] MM [Figure] R
[Figure] R [Figure] fails to predict where L is the number of message-passing layers in h .
Since W 1 ( p S , p T ) can be estimated without labels, Theorem 12 applies to unsupervised domain adaptation. Assuming that there is a model that performs well in both source and target domain, i.e., λ H is small, we may empirically estimate the discrepancy between source and target risk via the GNN Lipschitz constant and W 1 ( p S , p T ) .
Domain Shifts We first verify our analysis on the PTC dataset [22], which contains carcinogenicity labels of chemical structures for four groups of rodents: male mice (MM), male rats (MR), female mice (FM) and female rats (FR). We train 3-layer GINs [55] on one group and examine the empirical performance drop on the remaining groups.
Figure 7 shows the Wasserstein distance between groups and the corresponding performance drops. As a baseline, we compute the Wasserstein distance with WWL transportation cost [46]. The W 1 distance based on TMD highly correlates with the accuracy drop (Pearson correlation r = 0.712), while WWL- W 1 only achieves r = 0.489.
Size Generalization A known challenge for GNNs is generalizing to graphs of different size [58, 56].
To evaluate TMD on this problem, we sort the PROTEINS dataset [8] based on the number of nodes and bin it into 8 subsets, each containing 125 graphs.
We again train 3-layer GINs on the smallest and the largest bins and examine the accuracy drops on the remaining ones.
Figure 8 plots the accuracy drops with respect to different subsets and the corresponding Wasserstein distance based on TMD and WWL.
TMD correlates with the accuracy drop surprisingly well when the models are trained on large graphs and tested on smaller graphs ( r = 0.97 ).
The correlation is slightly weaker when the models are trained on small graphs and tested on large graphs ( r = 0.83).
Yet, TMD shows a much more gradual change in distance, in agreement with the gradual change in drop, than WWL, and hence reflects the overall behavior better.
[Figure]
Index 1 denotes the bin of smallest graphs and index 8 the largest graphs.
For better visualization, the Wasserstein distances are normalized to make the maximal distance equal to the greatest performance drops, as the absolute scales of distances are less important.
The models in (a) / (b) are trained on subset 1 / 8, respectively.
Conclusion
In this work, we introduce Tree Mover’s Distance (TMD), a new graph distance based on optimal transport between computation trees.
First, TMD captures the structural and attribute properties that are important for many graph classification tasks.
Second, it reflects the patterns that determine the generalization behavior of message passing graph neural networks, and, hence, offers a suitable tool to predict the perturbation stability and out-of-domain generalization capability of such GNNs.
Hence, it bears promise in applications, both for graph learning tasks and predicting reliability of graph learning models, and, in theory, as a tool for new tighter analyses of generalization in GNNs.
Acknowledgements This work was in part supported by NSF BIGDATA IIS-1741341, NSF AI
Institute TILOS, NSF CAREER 1553284.
CC is supported by a IBM PhD Fellowship.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information on how to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .
You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data are proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] See the Appendix 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] In the zip file (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See the Appendix (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
Broader Impact
Graph Neural Networks are used in many applications with potential societal implications: predictions on social networks, drug design, computational chemistry and materials science, traffic predictions, etc.
In many of these applications, the model may be faced with distribution shifts that may lead to a decline in performance.
This work is a step towards understanding and estimating such behavior, by understanding and formalizing what kinds of distribution shifts may impact the model.
Indeed, robustness is closely related to fairness, when the distribution shifts are associated with different demographic groups [12, 4, 59].
Among the above applications, this may be particularly an issue with social network analysis, and possibly drug design and traffic prediction.
In that case, the results in this paper may provide a basis for diagnostic tools be enabling to quantify the amount of potentially risky distribution shifts.
A Proof A.1 Preliminaries
Equivalence between OT and Wasserstein Distance Throughout the proof, we will frequently use the equivalence between optimal transport and Wasserstein distance to simplify the notation.
Following the setting in section 3, let X =
{ x i } mi =1 and Y =
{ y i } mj
=1 be two multisets, both containing m elements, and let U ( X ) denote the uniform distribution over a multiset X .
We first note the following equivalence:
[Equation]
where W d ( P, Q ) is the Wasserstein distance between P, Q with cost function d defined as follows:
[Equation]
The Π denotes the set of measure couplings whose marginals are P and Q , respectively.
Invariance to Additional Augmentation Here we show that unnormalized OT is invariant to “blank” augmentation, which is an important property for proving the main theorem.
Lemma 13.
Assume we are given two multisets X
= { x i } mi =1 , Y = {
y j } mj
=1 with the same cardinality m , where x i , y j ∈ X for all i, j .
Let d be a metric on X and 0 ∈ X .
Then we have
[Equation]
Proof.
First, by construction, we already have
[Equation]
as we can always keep the original coupling padded with an identity matrix and get the same cost.
Now suppose we adopt a new coupling for OT d ( X ∪ ( 0 ) n , Y ∪ ( 0 ) n ) which might lead to smaller cost.
Note that any change to the coupling can be decomposed into two steps: (1) permute, change the permutation in the original coupling and (2) decouple, change the coupling ( x − y ) to ( x − 0 ) , ( y − 0 ) for some pairing ( x, y ) .
In the permutation step, the cost is always increasing as we are destroy the original optimal coupling.
In the second step, the cost will also increase due to the triangle inequality of cost function d :
[Equation]
Therefore, we prove that the equality case must hold by contradiction.
Note that the proof holds for any augmentation, not only zero vectors.
Unnormalized OT is a Metric for Multisets Finally, we show that if the transportation cost d is a pseudometric, then the unnormalized OT with blank augmentation ρ is also a pseudometric for multisets.
Lemma 14.
If d is a metric, then OT d ( ρ ( · , · )) is a metric over multisets that do not contain 0 .
If 0 is included, OT d ( · , · ) is a pseudometric.
Proof.
The first two axioms, OT d
{+( ρ ( X, X )) = 
+}
if and only if X = X
and OT d ( ρ ( X, Y ))
=
OT d ( ρ ( Y, X )) immediately hold via the property of optimal transport and Wasserstein distance if 0 / ∈ X, Y .
The constraint on 0 is due to the fact OT d ( ρ ( X, X ∪ 0 ))
= 0 , which violates the first axiom.
But practically the augmentations are specified to be different from the elements in the sets.
Next, we prove the Triangle inequality of unnormalized OT.
In particular, we will leverage the equivalence of unnormalized OT and Wasserstein distance as follows:
[Equation]
(Lemma 13)
[Equation]
of W )
[Equation]
[Equation]
The inequality holds as Wasserstein distance is a pseudometric if the transportation cost is a pseudometric [45, 3].
A.2 Tree Mover’s Distance is a Pseudometric
Via Lemma 14, to show TMD is a pseudometric, we will focus on proving that the transportation cost
TD is a pseudometric between two trees.
Lemma 15.
The distance TD w ( ρ ( · , · )) is a metric for two rooted trees if w ( · ) > 0 and node features do not contain zero vectors, where ρ is the blank tree augmentation.
Otherwise, it is a pseudometric.
Proof.
{+We first prove that TD w ( T a , T b ) = 
+}
if and only if T a = T b by induction.
In particular, we will focus on the case Depth ( T a ) =
Depth ( T b ) , otherwise the statement trivially holds.
When depth = 1 , the TD w reduces to Euclidean distance, which is a metric.
Suppose the statement holds for depth- k trees.
Given two tree T a , T b with depth equals to k + 1 , their distance is
[Equation]
Since TD w is a metric for depth-k tree by assumption, and OT TD w ( ρ ( · , · )) is a metric if TD w is a metric via Lemma 14, TD w ( T a , T b )
{+= 
+}
if and only if r T a = r T b and T r Ta = T r
Tb , which completes the proof of induction.
Nevertheless, similar to Lemma 14, this does not hold if the nodes contains 0 , which should not be the case practically.
Otherwise, one can simply add a small values to node features to distinguish them from the zero vector.
For the triangle inequality, we give a proof by induction on depth.
For the base case depth = 1 , the tree mover’s distance is simply the Wasserstein-1 distance between the distribution of augmented node feature vectors and the transportation cost TD w ( · , · ) reduces to the Euclidean distance, which is a metric and satisfies the Triangle inequality.
Next, we show that if TD w ( · , · ) satisfies the Triangle ineuqality for depth-k trees, it is also a metric for depth-(k+1) trees.
For trees T a and T b with depth k + 1 , introduced the third tree T c with depth k + 1 , we have
[Equation]
The second inequality holds since OT is a pseudometric for multisets over depth- k trees as TD w ( · , · ) satisfies the Triangle ineuqality for depth-k trees via induction hypothesis.
Since TD w satisfies the triangle inequality for depth- k + 1 trees, via Lemma 14, TMD also satisfies the triangle inequality, which completes the proof by mathematical induction.
A.3 WL Proof
Theorem 16 (Discriminative Power of TMD) .
If two graphs G a =
( V a , E a ) , G b =
( V b , E b ) are determined to be non-isomorphic in WL iteration L and w ( l ) > 0 for all 0 < l ≤ L + 1 and 0 / ∈ V a , V b , then TMD L +1 w ( G a , G b ) > 0 .
Proof.
We will show that if two nodes have the same subtree, then their WL labels will be the same by induction.
The statement holds when depth is 1, as all the WL labels are the same and all the subtrees have the same single node.
Suppose the statement holds for depth- k tree.
If WL identifies two graphs are non-isomorphic at iteration k + 1 , this means there are at least a pair of nodes whose neighbors have different WL label at k -th iteration.
This implies that the neightbors have different subtree via induction hypothesis.
Therefore, the depth- k + 1 subtree of the node, which is constructed by appending the subtrees of neightbors to a new node, will also be different.
Therefore, the TMD between the subtrees will be greater than zero via Lemma 15, implying that TMD k
+1 λ also determine two graphs are non-isomorphic for all λ > 1 .
Again, we add the zero vector constraint for the same reason as Lemma 14 and 15.
A.4 Lipschitz Bound of GNN proof
Proof.
For simplicity, we set ϵ = 1 through out the proof.
We first bound the difference in prediction based on the embedding in the last layer:
[Equation]
(Definition of Lipschitz Condition)
Let V ρa and V ρb be two multisets of nodes after blank tree augmentation: ( V ρa , V ρb )
=
ρ ( V a , V b ) , we have
[Equation]
where T ( L,V ρa V ρb ) is a transportation plan between V ρa and V ρb .
Note that the inequality holds for any valid transportation plan.
Let ∆ ( L ) T ( L,VρaVρb )
= (cid:80) i ∈ V ρa ,j ∈ V ρb T ( L,V ρa V ρb ) i,j (cid:13)(cid:13)(cid:13)
z
( L ) i − z
( L ) j (cid:13)(cid:13)(cid:13) , we have ∆
[Equation]
L )
[Equation]
≤ (cid:88)
[Equation]

[Equation]
(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
[Equation]
( L )
[Equation]
[Equation]
[Equation]
] 
 (cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
[Equation]
 
[Equation]
(cid:13)(cid:13)(cid:13)
[Equation]
(cid:13)(cid:13)(cid:13)
[Equation]
(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13) (cid:88)
i [Equation] i ) z ( [Table]
[Equation]
[Equation]
(cid:88)
[Equation]
( i ) z (
[Table]
[Equation]
[Table]
) j (cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table] [Equation] Here we introduce another transportation plan T ( L − 1 , N ( i ) ρ N ( j ) ρ ) i ′ ,j ′ to pair augmented N ( i ) ρ and N ( j ) ρ for all i ∈ V ρa , j ∈ V ρb at layer L − 1 .
We then further bound first term using similar strategy:
[Equation]
(cid:13)(cid:13)(cid:13)(cid:21) =
(cid:88) i ∈ V ρa ,j ∈ V ρb T ( L,V a V b ) i,j   (cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13) ϕ ( L − 1)   z ( L − 2) i + (cid:88) i ′ ∈N ( i )
z ( L − 2) i ′   − ϕ ( L − 1)   z ( L − 2)
j + (cid:88)
j ′ ∈N ( j ) z
( L − 2)
j ′ 
 (cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13) ≤ K ( L − 1) ϕ (cid:32) (cid:88) i ∈ V ρa ,j ∈ V ρb
T ( L,V a V b ) i,j (cid:32) (cid:13)(cid:13) (cid:13) z
( L − 2)
i − z
( L − 2)
j (cid:13)(cid:13) (cid:13) + (cid:88) i ′ ∈N ( i ) ρ ,j ′ ∈N ( j ) ρ
T ( L − 1 , N ( i ) ρ N ( j ) ρ ) i ′ ,j ′ (cid:13)(cid:13) (cid:13) z
( L − 2) i ′ − z
( L − 2) j ′ (cid:13)(cid:13) (cid:13) (cid:33)(cid:33) .
=
K ( L − 1) ϕ (cid:32) (cid:88) i ∈ V ρa ,j ∈ V ρb
T ( L,V a V b ) i,j (cid:32) (cid:13)(cid:13)(cid:13)
z ( L − 2) i − z
( L − 2) j (cid:13)(cid:13)(cid:13) [Equation] Note that we still use the same transportation plan T ( L − 1 , N ( i ) ρ N ( j ) ρ ) i ′ ,j ′ to bound the sum difference.
Apply this step recursively, the first term will eventually become ∥ x i − x j ∥ as K [Equation]
L
[Equation]
ϕ
[Equation]
L
[Equation]
i
[Equation] and the bound will become [Equation] [Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table] [Equation] [Table] [Equation]
[Table]
[Equation] [Table] [Equation] [Table]
[Equation] Next [Equation] we will bound 1 , 2 , · · · to l using a similar way as described above.
To start, we introduce the second coupling π l − 2 N ( i ′ ) , N ( j ′ ) for all i ′ , j ′ ∼ π l − 1 N ( i ) , N ( j ) : [Equation] [Table] [Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation] [Table] [Equation] [Table]
[Equation]
[Table] [Equation] [Table] [Equation]
[Table]
[Equation]
:= bound ( 2 )
We can see that the bounds for and only differ in the term
K
[Equation]
Moreover, one can show that bound ( k ) and bound( k+1 ) only differ in
K
[Equation]
Therefore
[Equation]
we can merge bound 1 to bound l and get l (cid:88)
i
[Equation] [Equation]
(cid:35)
[Equation]
[Table]
Now we see something familiar: 1’ to ( L − 1) ′
can all be bounded in the same way, e
[Table]
g., 1’ ≤ K ( L − 1) ϕ
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
K
[Table]
[Equation]
K
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table] [Equation] [Table] [Equation]
Similarly
[Equation]
plugging the bound gives =
[Equation] [Table] [Equation] [Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table] [Equation] We can see that the weight of the center nodes gradually changes as: L → 1 + 2 + · · · ( L − 1) → 1 + (1 + 2) + · (1 + 2 + · · · + ( L − 2)) → · · · . These are exactly the elements at level L + 1 of the
Pascal’s triangle.
See Figure 4 for an illustration.
For instance, if L = 4 , the L + 1 level of Pascal’s triangle is (1 , 4 , 6 , 4 , 1) =
(1 , 1 + (1 + 2) , (1 + 2 + 3) , 4 , 1) , which matches the weight of center nodes in each level.
Let P lL is the l -th number at level L of Pascal’s triangle. Applying the bound recursively and extracting (cid:81) Lm =1 K ( m ) ϕ gives the following bound:
[Equation]

[Equation]
By setting each transportation plan to the optimal plan acquired from the TMD OT problem, the bound is equivalent the tree mover distance with depth- L + 1 computation tree, and w ( l ) =
P l − 1 L +1 /P lL +1 .
The one with ϵ ̸ = 1 can be trivially extended from the current proof.
A.5 Tree Mover’s Distance Stability Proof A.5.1 Node Drop
Proof.
We use a illustration to provide the conceptual idea of the proof.
Given a graph G =
( V, E ) , let G ′ be the graph where node v ∈ V is dropped. The computation tree of node v will play an important role here. Firstly, the nodes in level l of the computation tree are the nodes that can reach v within l − 1 steps. In particular, the number of nodes in level l determines how
[Figure]
many depth- ( L − l + 1) computation tree of v exists in T G .
Therefore, deleting v from the graph will also delete all the subtrees in T G that are rooted at v , and the number of deleted trees are determined by the width of T v . See Figure 9 for a illustration. Deleting a subtree T will introduce an additional transportation cost TD w ( T, T 0 ) , which is the tree norm of subtree. By aggregating all the tree norms and consider the effect of weights w , we arrive at
[Figure] where Width l ( T ) is the width of l -th level of tree T and λ 1 = 1 , λ l = (cid:81) l − 1 j =1 w ( L + 1 − j ) . This is an upper bound instead of equality as some deleted subtrees are repeated counted in the bound.
A.5.2 Edge Drop
Proof.
We again use a illustration to provide the conceptual idea of the proof.
[Figure] Given a graph G =
( V, E ) , let G ′ be the graph where edge ( u, v ) [-
-]∈
E is dropped.
Different from node drop, deleting an edge will affect both nodes.
In particular, the number of nodes in level l of computation tree T v determines how many depth- L − l computation tree of u exists in T G . Therefore, deleting edge u − v from the graph will also delete all the subtrees in T G that are rooted at v ( u ) where the roots have ancestor u ( v ). See Figure 11 for a illustration. Similarly, by aggregating all the tree norms and consider the effect of weights w , we arrive at
[Equation]
where Width l ( T ) is the width of l -th level of tree T and λ 1 = 1 , λ l = (cid:81) l − 1 j =1 w
( L + 1 − j ) .
This is an upper bound instead of equality as some deleted subtrees are repeated counted in the bound. A.5.3 Node Perturbation
Proof. The node perturbation is the simplified case of node drop, where only the node features are perturbed.
Given a graph G =
( V, E ) , let G ′ be the graph where node feature x v is perturbed to x ′ v .
The tree mover’s distance between G and G ′ is equal to Width = Width = Width =
Width =
Width =
Width = [Equation]
where Width l ( T ) is the width of l -th level of tree T and λ 1 = 1 , λ l = (cid:81) l − 1 j =1 w ( L + 1 − j ) . This is an upper bound instead of equality as changing node features would affect the optimal coupling of OT.
We might not have the optimal transportation plan after node perturbation
[Equation]
B Lipschitz Condition for Other GNNs B.1 Graph Convolutional Network
Here [Equation]
we consider Graph Convolutional Network (GCN) [27] with the following message passing rules:
[Equation]
In particular, the SUM is replaced with MEAN in the message passing and graph readout. It is easy to show that the same bound holds by replacing all the unnormalized OT with normalized OT ∗ in tree distance and tree mover’s distance, i.e., [Equation]
otherwise ,
[Equation] This gives the bound: [Equation] B.2 Other Message Passing GNNs
Sometimes, the center nodes are treated differently compared to the neighbors, e.g., MessagePassing z
[Equation]
l
[Equation] v = ϕ [Equation]
The proof can be easily modified as follows:
[Equation] [Table] [Equation] [Table]
[Equation] [Table] [Equation] [Table]
[Equation]
[Table] [Equation] [Table]
[Equation] [Table]
[Equation] [Table] [Equation]
[Table] [Equation]
[Table] [Equation] [Table]
[Equation]
[Table]
[Equation] [Table] [Equation]
[Table] [Equation] [Table] [Equation]
[-y ] relative to the sourcerisk R S ( h ) = E x,y ⇠ µ S [ 1 h ( x ) 
-](cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13) [Equation] (cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
[Equation] [Equation]
[Table]
[Equation]
(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13) Therefore, simply replacing the ϵ with K ( l ) φ leads to a Lipschitz bound for this variant:
[Equation]
( G
[Equation]
G
[Equation]
where w ( l ) =
K ( l − 1) φ · P l − 1 L +1 /P lL +1 for all l ≤ L and P lL is the l -th number at level L of Pascal’s triangle.
C Additional Experiments C.1 Graph Clustering
One advantage of graph metric over GNNs is that we can perform geometric analysis of graph datasets such as graph clustering. We adopt k-medoids [24], a variant of k-means [30], to perform unsupervised clustering with TMD.
Figure 12 provides an qualitative example of clustering, and Table 2 measures the quallity of clusters with Normalized Mutual Information (NMI) and Completeness
Score (CS) [6].
We can see that k-medoids with tree mover’s distance generates meaningful clusters that aligns with labels.
C l u s t e r
C l u s t e r C l u s t e r [Figure] C.2 t-SNE Visualization of Graphs
Equipped with TMD, we can extend t-SNE [48] from Euclidean space to graphs.
Specifically, t-SNE constructs a probability distribution based on pairwise distance and minimizes the divergence between
MUTAG
[Table]
the distribution of low dimensional and original data.
We simply replace the Euclidean distance in conventional t-SNE with tree mover’s distance and show the t-SNE visualization of various graph datasets in Figure 13 .
Although not perfectly, we can observe the separation between points with different labels.
[Table]
R a w
F ea t (
T M D )
GNN F ea t ( L 2 ) 3 Additional
3 Layers 2 Layers 1 Layers 0 Layer r = 0.
r = 0.
r = 0.56 r =
[Figure]
We repeat the experiments in section 5 with PROTEINS dataset and show the results in Figure 14 and 15.
We can see that the GNNs are less sensitive to small graph perturbation as Figure 15 shows, as the number of nodes and edges is much larger than the one in MUTAG.
[Figure]