How Powerful are K -hop Message Passing Graph Neural Networks
Abstract
The most popular design paradigm for Graph Neural Networks (GNNs) is 1-hopmessage passing—aggregating features from 1-hop neighbors repeatedly.
However,the expressive power of 1-hop message passing is bounded by the WeisfeilerLehman (1-WL) test.
Recently, researchers extended 1-hop message passing to
K -hop message passing by aggregating information from K -hop neighbors ofnodes simultaneously.
However, there is no work on analyzing the expressivepower of K -hop message passing.
In this work, we theoretically characterizethe expressive power of K -hop message passing.
Specifically, we first formallydifferentiate two kinds of kernels of K -hop message passing which are oftenmisused in previous works.
We then characterize the expressive power of K -hopmessage passing by showing that it is more powerful than 1-hop message passing.
Despite the higher expressive power, we show that K -hop message passing
stillcannot distinguish some simple regular graphs.
To further enhance its expressivepower, we introduce a KP-GNN framework, which improves K -hop messagepassing by leveraging the peripheral subgraph information in each hop.
We provethat KP-GNN can distinguish almost all regular graphs including some distanceregular graphs which could not be distinguished by previous distance encodingmethods.
Experimental results verify the expressive power and effectiveness of
KP-GNN.
KP-GNN achieves competitive results across all benchmark datasets.
Introduction
Currently, most existing graph neural networks (GNNs) follow the message passing framework,which iteratively aggregates information from the neighbors and updates the representations of nodes.
It has shown superior performance on graph-related tasks [1, 2, 3, 4, 5, 6, 7] comparing to traditionalgraph embedding techniques [8, 9].
However, as the procedure of message passing is similar to the1-dimensional Weisfeiler-Lehman(1-WL) test [10], the expressive power of message passing GNNsis also bounded by the 1-WL test [7].
Namely, GNNs cannot distinguish two non-isomorphic graphstructures if the 1-WL test would fail.
In normal message passing GNNs, the node representation is updated by the direct neighbors of node,which are called 1-hop neighbors.
Recently, some works extend the notion of message passing into
K -hop message passing [11, 12, 13, 14, 15].
K -hop message passing is a type of message passingwhere the node representation is updated by aggregating information from not only 1-hop, but allthe neighbors within K hops of the node.
However, there is no work on theoretically characterizingthe expressive power of GNNs with K -hop message passing, e.g., whether it can improve the 1-hopmessage passing or not, and to what extent it can.
In this work, we theoretically characterize the expressive power of K -hop message passing GNNs.
Specifically, 1) we formally distinguish two different kernels of the K -hop neighbors, which are often misused in previous works.
One is based on graph diffusion and the other is based on shortest pathdistance.
For example, MixHop [11], GINE+ [15], GPR-GNN [14] all claim they use informationfrom multi-hop neighbors.
However, they actually use different kernels.
Further, we show thatdifferent kernels of K -hop neighbors will result in different expressive power of K -hop messagepassing.
We theoretically characterize the expressive power of K -hop message passing GNNs andgeneralize the proposed theorem to most existing K -hop models.
We show that K -hop messagepassing is strictly more powerful than 1-hop message passing but bounded by 3-WL and it failedin distinguishing some simple regular graphs, no matter which kernel is used.
This motivates us tofurther improve K -hop message passing.
Specifically, we introduce KP-GNN , a new GNN framework with K -hop message passing, whichsignificantly improves the expressive power of standard K -hop message passing GNNs.
In particular,during the aggregation of neighbors in each hop, KP-GNN not only aggregates neighboring nodes inthat hop
but also aggregates the peripheral subgraph (subgraph induced by the neighbors in thathop).
This additional information helps the GNN to learn more expressive local structural featuresaround the node.
We further prove that KP-GNN is able to distinguish almost all regular graphs andeven some distance regular graphs.
The proposed KP-GNN has several additional advantages.
First, itcan be applied to most existing message passing GNNs with little modification.
Second, it adds littlecomputational complexity to standard K -hop message passing.
We demonstrate the effectiveness ofthe KP-GNN framework through extensive experiments on both classification and regression tasks.
K -hop message passing and its representation power2.1 Notations
Denote a graph as G =
( V, E ) , where V = { 1 , 2 , ..., n } is the node set and E
⊆ V × V is the edgeset.
Meanwhile, denote A ∈ { 0 , 1 } n × n as the adjacency matrix of graph G .
Denote x v as the featurevector of node v and denote e uv as the feature vector of the edge from u to v .
Finally, we denote
Q 1 v,G as the set of 1-hop neighbors of node v in graph G and N 1 v,G = Q 1 v,G ∪ { v } .
Note that when wesay K -hop neighbors of node v , we mean all the neighbors that have distance from node v less thanor equal to K .
In contrast, k -th hop neighbors mean the neighbors with exactly distance k from nodev .
The definition of distance will be discussed in section 2.3.
2.2 1-hop message passing framework
Currently, most existing GNNs are designed based on 1-hop message passing framework [16].
Denoteh lv as the output representation of node v at layer l and h 0 v = x v .
Briefly, given a graph G and a 1-hopmassage passing GNN, at layer l of the GNN, h lv is computed by h l − 1 v and {{ h l − 1 u | u ∈ Q 1 v,G }} :
[Equation]
where m lv is the message to node v at layer l , MES l and UPD l are message and update functions atlayer l respectively.
After L layers of message passing, h L v is used as the final node representation ofnode v .
Such a representation can be used to conduct node-level tasks like node classification andnode regression.
To get the graph representation, a readout function is used:
[Equation]
where READOUT is the readout function for computing final graph representation.
Then h G can beused to conduct graph-level tasks like graph classification and graph regression.
2.3 K -hop message passing framework
The 1-hop message passing framework can be directly generalized to K -hop message passing, as itshares the same message and update mechanism.
The difference is that an independent message andupdate function can be employed for each hop.
Meanwhile, a combine function is needed to combinethe results from different hops into the final node representation at this layer.
First, we differentiatetwo different kernels of K -hop neighbors, which are interchanged and misused in previous research. The first kernel of K -hop neighbors is shortest path distance (spd) kernel . Namely, the k -th hopneighbors of node v in graph G is the set of nodes that have shortest path distance of k from v . 𝒗      𝐺 (1) 𝐺 (2) 𝒗      1st hop
[Figure]
Shortest path distance
Example
[Figure] graphs input         1st hop 2nd hop       𝒗      𝐺 (1) 𝐺 (2) 𝒗
Example
However, it is indistinguishable if we switch the kernel.
Finally, both two examples can be distinguished by adding peripheral edge
Definition 1.
For a node v in graph G , the K -hop neighbors N K,spdv,G of v based on shortest pathdistance kernel is the set of nodes that have the shortest path distance from node v less than or equalto K .
We further denote Q k,spdv,G as the set of nodes in G that are exactly the k -th hop neighbors (withshortest path distance of exactly k ) and N 0 ,spdv,G = Q 0 ,spdv,G = { v } is the node itself.
The second kernel of the K -hop neighbors is based on graph diffusion (gd) .
Definition 2.
For a node v in graph G , the K -hop neighbors N K,gdv,G of v based on graph diffusionkernel is the set of nodes that can diffuse information to node v within the number of random walkdiffusion steps K with the diffusion kernel A .
We further denote Q k,gdv,G as the set of nodes in G thatare exactly the k -th hop neighbors (nodes that can diffuse information to node v with k diffusionsteps) and N 0 ,gdv,G = Q 0 ,gdv,G = { v } is the node itself.
Note that a node can be a k -th hop neighbor of v for multiple k based on the graph diffusion kernel.
We include more discussions of K -hop kernels in Appendix A. Next, we formally define the K -hopmessage passing framework as follows:
[Equation]
where t = { spd, gd } indicates the kernel of K -hop neighbors.
Here, for each hop, we can applyunique MES and UPD functions.
Note that for k > 1 , there may not exist the edge feature e uv asedges are not directly connected.
But we leave it here since we can use another type of feature toreplace it, which is described in Appendix H. Compared to the 1-hop message passing frameworkdescribed in Equation (1), the COMBINE function is introduced to combine the representations ofnode v at different hops.
It is easy to see that the L layer 1-WL GNNs are actually L layer K -hopmessage passing GNNs with K = 1 .
We include more discussion of K -hop message passing
GNNsin
Appendix A.
2.4 Expressive power of K -hop message passing framework
In this section, we theoretically analyze the expressive power of K -hop message passing.
We assumethere is no edge feature and all nodes in the graph have the same feature, which means that GNNscan only distinguish two nodes using the local structure of nodes.
Note that including node featuresonly increases the expressive power of GNNs as nodes/graphs are more easily discriminated.
It hasbeen proved that the expressive power of 1-hop message passing is bounded by the 1-WL test ondiscriminating non-isomorphic graphs [7, 17].
In this section, We show that the K -hop message passing is strictly more powerful than 1-WL test when K > 1 .
Across the analysis, we utilize regulargraphs as examples to illustrate our theorems since they cannot be distinguished using either 1-hopmessage passing or the 1-WL test.
Note that our analysis is not limited to regular graphs but iscapable of describing any graphs.
To begin the analysis, we first define proper K -hop message passing GNNs .
Definition 3.
A proper K -hop message passing GNN is a class of GNN models where the message,update and combine functions are all injective given the input from a countable space.
A proper K -hop message passing GNN is easy to find due to the universal approximation theorem [18]of neural network and the Deep Set for set operation [19].
In the latter sections, by default allmentioned K -hop message passing GNNs are proper.
Next, we introduce node configuration:
Definition 4.
The node configuration of node v in graph G within k hops under t kernel is a list
A k,tv,G = ( a 1 ,tv,G , a 2 ,tv,G , ..., a k,tv,G ) , where a i,tv,G = | Q
i,tv,G | is the number of i -th hop neighbors of node v .
When we say two node configurations A k,tv 1 ,G (1) and A k,tv 2 ,G (2) are equal, we mean that these two listsare component-wise equal to each other.
Now, we state the first proposition:
Proposition 1.
A proper K -hop message passing GNN is strictly more powerful than 1-hop messagepassing GNNs when K > 1 .
To see why this is true, we first discuss how node configuration is related to the first layer of messagepassing.
In the first layer of K -hop message passing, each node aggregates neighbors from up to
K hops.
As each node has identical label, an injective message function can only know how manyneighbors at each hop, which is exactly the node configuration.
In other words, The first layer of
K -hop message passing is equivalent to inject node configuration for each node label .
Next, wecan characterize the expressive power of 1-hop message passing GNNs using node configuration.
When K = 1 , the node configuration of v 1 and v 2 are d v 1 ,G (1) and d v 2 ,G (2) , where d v,G is the node 133 degree of v .
After L layers, GNN can get node configurations of each node within L hops.
Giventhe statement, it is straightforward to see why these GNNs cannot distinguish any n -sized r -regulargraph, as each node in the regular graph have same degree.
From another prospective, the expressivepower of 1-hop message passing GNN is limited because it only has the degree information of eachnode in the graph within the receptive field of GNN.
Next, when K > 1 , the K -hop message passing is at least equally powerful as 1-hop message passingsince node configuration up to k hops includes all the information the 1 hop has.
To see why it is morepowerful, we use two examples to illustrate it.
The first example is shown in the left part of Figure 1.
Suppose here we use graph diffusion kernel and we want to learn the representation of node v 1 andnode v 2 in the two graphs, we know that the 1-hop message passing framework produces the samerepresentation for two nodes as they are both nodes in 6-sized 3-regular graphs.
However, it is easyto see that v 1 and v 2 have different local structures and should have different representations.
Instead,if we use the 2-hop message passing with the graph diffusion kernel, we can easily distinguish thetwo nodes by checking the 2nd hop neighbors of the node, as node v 1 has four 2nd hop neighbors butnode v 2 only has two 2nd hop neighbors.
The second example is shown in the right part of Figure 1.
Two graphs in the example are still regular graphs and the 1-hop message passing continue to fail indistinguishing node v 1 and node v 2 .
In contrast, suppose here we use shortest path distance kernel,node v 1 and v 2 have different numbers of 2nd hop neighbors thus will have different representationsby performing 2-hop message passing.
These two examples convincingly demonstrate that the K -hopmessage passing with K > 1 can have better expressive power than K = 1 .
The above discussion can be applied to any existing K -hop GNNs using shortest path distance kernellike GINE [15], Graphormer [20].
However, for K -hop GNNs using graph diffusion kernel like
MixHop [11], GPR-GNN [14], MAGNA [13], we find they actually use a weak version of K -hop thanthe definition of us.
We leave the detailed discussion in Appendix B. Further, Distance Encoding [21]also uses shortest path distance information to augment the 1-hop message passing, which is similarto K -hop GNNs with shortest path distance kernel.
However, we find the expressive power of the twoframeworks are actually different from each other.
We leave the detailed discussion in Appendix C.
2.5 Limitation of K -hop message passing framework
Although we show that K -hop message passing with K > 1 is better at distinguishing non-isomorphicstructures than 1-hop message passing, there are still limitations.
In this section, we discuss thelimitation of K -hop message passing.
Specifically, we show that the choice of the kernel can affectthe power of K -hop message passing .
Furthermore, even with K -hop message passing, we cannotdistinguish some simple non-isomorphic structures and the expressive power of K -hop messagepassing is bounded by 3-WL.
Continue looking at the provided examples.
In example 1, we know that node v 1 and v 2 have differentnumbers of 2nd hop neighbors with the graph diffusion kernel.
However, if we use the shortest pathdistance kernel, the two nodes have the same number of neighbors in the 2nd hop, which means thatwe cannot distinguish two nodes using 2-hop message passing with the shortest path distance kernel.
Similarly, in example 2, two nodes have the same number of neighbors in both 1st hop and 2nd hopswith graph diffusion kernel.
These results highlight that the choice of kernel can affect the expressivepower of K -hop message passing.
Furthermore, none of them can distinguish both two exampleswith 2-hop message passing.
Recently, Frasca et al.
[22] show that any subgraph-based GNNs can be implemented by 3-IGN [23,24] and thus their power is bounded by 3-WL.
Here, we show that the K -hop message passing canalso be implemented by 3-IGN and thus:
Theorem 1.
The expressive power of a proper K -hop message passing GNN of any kernel is boundedby 3-WL.
We include the proof in Appendix D. Given all these observations, we may wonder if there is a wayto further improve the expressive power of K -hop message passing?
KP-GNN: improving the power of K -hop message passing by peripheral
subgraph
In this section, we describe how to improve the expressive power of K -hop message passing byadding additional features to message passing.
Specifically, by adding the peripheral subgraphinformation, we can improve the representation power of K -hop message passing by a large margin.
3.1 Peripheral edge and peripheral subgraph
First, we define peripheral edge and peripheral subgraph .
Definition 5.
The peripheral edges E ( Q k,tv,G ) are defined as the set of edges that connect nodes withinset Q k,tv,G .
We further denote | E ( Q k,tv,G ) | as the number of peripheral edges.
The peripheral subgraph
G k,tv,G =
( Q k,tv,G , E
( Q k,tv,G )) is defined as the subgraph induced by Q k,tv,G from the whole graph G .
Briefly speaking, the peripheral edges E ( Q k,tv,G ) record all the edges whose two ends are both from
Q k,tv,G and the peripheral subgraph is a graph constituted by peripheral edges.
It is easy to seethat the peripheral subgraph G k,tv,G automatically contains all the information of peripheral edges
E ( Q k,tv,G ) .
Next, we show that the power of K -hop message passing can be improved by leveragingthe information of peripheral edges and peripheral subgraph.
We again refer to the examples in
Figure 1.
Here we only consider the peripheral edge information.
In example 1, we notice thatat the 1st hop, there is an edge between node 3 and node 4 in the left graph.
More specifically,
E ( Q 1 ,t v 1 ,G (1) )
= { (3 , 4) } .
In contrast, we have E ( Q 1 ,t v 2 ,G (2) )
= {} in the right graph, which meansthere is no edge between the 1st hop neighbors of v 2 .
Therefore, by adding this information tothe message passing, we can successfully distinguish these two nodes.
Similarly, in example 2,there is one edge between the 1st hop neighbors of node v
but no such edge exists for node v 1 .
By leveraging peripheral edge information, we can distinguish the two nodes as well.
The aboveexamples demonstrate the effectiveness of the peripheral edge and peripheral subgraph information.
3.2 K -hop peripheral-subgraph-enhanced graph neural network
In this section, we propose the K -hop P eripheral-subgraph-enhanced G raph N eural N etwork (KPGNN), which equipes the K -hop message passing with peripheral subgraph information for morepowerful GNN design.
Recall the K -hop message passing defined in Equation (3).
The onlydifference of KP-GNN is that we revise the message function as follows:
[Equation]
Briefly speaking, in the message step at the k -th hop, we not only aggregate information of theneighbors but also the peripheral subgraph at the k -th hop.
The implementation of KP-GNN can bevery flexible as any graph encoding function can be used.
To maximize the information the modelcan encode while keeping it simple, we implement the message function as:
[Equation]
where MES l,normalk denotes the message function in the original GNN model, C is the set ofconnected components in G k,t v,G , E ( Q k,t v,G ) c is the edge set of the c -th connected component in G k,t v,G .
Such implementation helps the KP-GNN to not only encode the E ( Q k,tv,G ) but also partial informationof G k,tv,G (number of components).
With this implementation, any GNN model can be incorporatedinto and be enhanced by the KP-GNN framework by replacing MES l,normalk and UPD lk with thecorresponding functions for each hop k .
We leave the detailed implementation in Appendix H.
3.3 The expressive power of KP-GNN
In this section, we theoretically characterize the expressive power of KP-GNN and compare it withthe original K -hop message passing framework.
The key insight is that, according to Equation (4),the message function at the k -th hop additionally encodes G k,tv,G compared to normal K -hop messagepassing.
Next, we show that even with very simple information, the KP-GNN can become powerfulenough for distinguishing regular graphs.
Theorem 2.
Consider all pair of n -sized r -regular graphs, where 3 ≤ r < √ 2 logn .
For any smallconstant ϵ > 0 , there exists a KP-GNN using shortest path distance as kernel and only peripheraledge information with at most K = ⌈ ( 12 + ϵ logn log ( r − 1 − ϵ ) )
⌉ , which distinguishes almost all (1 − o (1))such pair of graphs with only 1-layer message passing.
We include the proof in
Appendix E.
The above theorem proves that a simple implementation of
KP-GNN leveraging only peripheral edge information can distinguish almost all regular graphs withsome K and 1 layer.
Moreover, K -hop message passing with shortest path distance kernel cannot distinguish any distanceregular graphs with the same intersection array according to the Theorem 3.7 in Distance Encoding [21].
Here we show that KP-GNN is more powerful than Distance Encoding on distinguishingdistance regular graphs .
Theorem 3.
For two non-isomorphic distance regular graphs G (1) =
( V (1) , E (1) ) and G (2) =( V (2) , E (2) ) with the same intersection array ( b 0 , b 1 , ...,
b d − 1 ; c 1 , c 2 , ..., c d ) , we pick two nodes v 1and v 2 from two graphs respectively.
Given a proper 1-layer K -hop KP-GNN with message functionsdefined in Equation (5), it can distinguish v 1 and v 2 if b 0 − b j − c j = 2 for some j ≤ K and G j,tv 1 ,G (1)and G j,tv 2 ,G (2) are non-isomorphic.
We include the proof in Appendix F. Theorem 3 shows that the KP-GNN with a simple implementationcan distinguish some distance regular graphs, which further distinguishes KP-GNN from Distance
Encoding [21] and subgraph-based GNNs [25, 26, 27, 28] as they cannot distinguish distance regulargraphs.
We include further discussion in Appendix F.
However, with the current implementation,
KP-GNN cannot distinguish all distance regular graphs.
3.4 Time, space complexity, and limitation
In this section, we discuss the time and space complexity of K -hop message passing GNN and KPGNN.
Suppose a graph has n nodes and m edges.
Then, the K -hop message passing and KP-GNNhave both the space complexity of O ( n ) and time complexity of O ( n 2 ) for shortest path distancekernel.
Note that the complexity of graph diffusion is no less than the shortest path distance kernel.
We can see that KP-GNN only require same space complexity as vanilla GNNs and much less timecomplexity than O ( nm ) of subgraph-based GNNs.
Further, as in practical graphs are sparse and weonly need some K , the real cost is far less than theoretical results.
We leave the detailed discussionon complexity and limitation of KP-GNN in Appendix G
Related Work
Expressive power of GNN.
Analyzing the expressive power of GNN is a crucial problem as it canserve as a guidance on how to improve GNNs.
Xu et al.
[7] and Morris et al.
[17] first proved thatthe power of 1-hop message passing is bounded by the 1-WL test.
In other words, 1-hop messagepassing cannot distinguish any non-isomorphic graphs that the 1-WL test fails to.
In recent years,many efforts have been put into increasing the expressive power of 1-hop messaging passing.
Thefirst line of research tries to mimic the higher-order WL tests, like 1-2-3 GNN [17], PPGN [23],ring-GNN [29].
However, they require exponentially increasing space and time complexity w.r.t.node number and cannot be generalized to large-scale graphs.
The second line of research tries toenhance the rooted subtree of 1-WL with additional features.
Some works [30, 31, 32] add one-hot orrandom features into nodes.
Although they achieve good results in some setting, they deteriorate thegeneralization ability as such features produce different representations for nodes even with the samelocal graph structure.
Instead, ID-GNN [33] enhances the rooted subtree by labeling the root nodewith a different color.
However, the additional expressive power of ID-GNN only comes from therooted node’s labeling, which may not be powerful enough to distinguish more complex structures.
Some works like Distance Encoding [21], SEAL [34], labeling trick [35] and GLASS [36] introducenode labeling based on either distance or distinguishing target node set.
Specifically, Distance
Encoding can be regarded as K -hop message passing with the shortest path distance kernel.
On theother hand, GraphSNN [37] introduces a hierarchy of local isomorphism and proposes structuralcoefficients as additional features to identify such local isomorphism.
However, the function designedto approximate the structural coefficient cannot fully achieve its theoretical power.
The third line ofresearch attends to revising the rooted subtree.
Specifically, NGNN [25] encodes a rooted subgraphinstead of a rooted subtree thus achieving superior expressive power on distinguishing regular graphs.
However, it needs to run an inner GNN on every node of the graph thus introducing much morecomputation overhead.
GNN-AK [26] applies a similar idea as NGNN.
The only difference lies inhow to compute the node representation from the local subgraph.
K -hop message passing GNN.
There are some existing works that instantiate the K -hop messagepassing framework.
For example, MixHop [11] performs messing passing on each hop with graphdiffusion kernel and concatenates the representation on each hop as the final representation.
Khop [12] sequentially performs the message passing from hop K to hop 1 to compute the representationof the center node.
However, it is not parallelizable due to its computational procedure.
MAGNA [13]introduces an attention mechanism to K -hop message passing.
GPR-GNN [14] use graph diffusionkernel to perform graph convolution on K -hop and aggregate them with learnable parameters.
However, none of them give a formal definition of K -hop message passing and theoretically analyzeits representation power and limitations.
Experiments
In this section, we conduct extensive experiments to evaluate the performance of KP-GNN.
Specifically, we 1) empirically verify the expressive power of KP-GNN on 3 simulation datasets anddemonstrate the benefits of KP-GNN compared to normal K -hop message passing GNNs and existing models; 2) show that the KP-GNN can achieve state-of-the-art performance on 5 TU datasets; 3)demonstrate that the KP-GNN achieves comparable performance on 3 molecular prediction datasets;4) analyze the running time of KP-GNN.
The detail of each variant of KP-GNN is described in Appendix H and the detailed experimental setting is described in Appendix I.
[Table]
Datasets :To evaluate the expressive power of KP-GNN, we choose: 1) EXP dataset [38], which contains 600 pairs of 1-WL-indistinguishable but non-isomorphic graphs. 2) Graph property regression(connectedness, diameter, radius) and node property regression (single source shortest path, eccentricity, Laplacian feature) task on graph random dataset [39].
3) Graph substructure counting (triangle,tailed triangle, star and 4-cycle) tasks on random graph dataset [40].
For TU datasets evaluation,we choose MUTAG [41],D&D [42],PROTEINS [42],PTC-MR [43], and IMDB-B [44] from TUdatabase.
For molecule prediction datasets, we pick QM9 [45, 46], ZINC [47], and MolHIV [48].
The detailed statistics of the datasets are described in Appendix J.Without further highlight, all errorbars in the result tables are standard deviation of multiple runs (more details are in Appendix I).
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
Empirical verification of the expressive power : To evaluate the power of KP-GNN, we compare it with several existing models.
For the baseline model, we use
GIN [7], which has the same expressive power as 1-WLtest.
For more powerful baselines, we use GIN-AK+ [26],
PNA [39] and PPGN [23].
For KP-GNN, we implementthe KP-GIN+.
To evaluate the effectiveness of peripheralsubgraph, we also implement the normal K -hop versionof GINE+, denote as K-GIN+.
The results are shown in
Table 1.
Baseline results are taken from [26] and [39].
We also compare results of GIN-AK+ for bothwith and without D2C encoding and KP-GIN+ both with and without path encoding.
We can seeboth K-GIN+ and KP-GIN+ achieve perfect performance on EXP dataset.
Further, we conduct anablation study on KP-GIN+ and K-GIN+ using EXP dataset.
Table 2 presents the results.
We cansee that KP-GNN achieves perfect results with only K ≥ 2 for both two kernels.
However, K-GINrequires K ≥ 3 and K ≥ 4 to get perfect results for shortest path distance kernel and graph diffusionkernel respectively.
Results on various property regression tasks further demonstrate the advantage of
KP-GNN over existing models and normal K -hop message passing GNNs.
[Table]
Evaluation on TU datasets : For baseline models, we select: 1) graph kernel based method WLsubtree kernel [49]; 2) 1-hop message passing based GNN methods: GIN [7] and DGCNN [6];3) advanced GNN methods: GraphSNN [37] and GIN-AK+ [26].
For the proposed KP-GNN, weimplement GCN [1], GraphSAGE [3]
[Table]
and GIN [7] using the KP-GNN framework, denoted as KP-
[Table]
GCN, KP-GraphSAGE, and KP-GIN respectively.
The results are shown in Table 3.
For a detailedcomparison, we report the results of two different settings.
The first setting follows Xu et al.
[7] andthe second setting follows Wijesinghe and Wang [37].
We denote the second setting with ∗ in the table.
We can see that under the setting of Wijesinghe and Wang [37], KP-GNN achieves state-of-the-artperformance across all datasets, which demonstrates the performance of KP-GNN on real-worlddatasets.
Under the setting of Xu et al.
[7], KP-GNN still achieves comparable performance.
Evaluation on molecular prediction tasks : For QM9 dataset, we report results of DTNN, MPNNfrom [46].
We further select Deep LRP [40], PPGN [23], and Nested 1-2-3-GNN [25].
For ZINCdataset, we report results of MPNN [16] and PNA [39] from [47].
We further pick Graphormer [20],
GSN [50], GIN-AK+ [26], and CIN [51] .
For MolHIV dataset, we report results of PNA [39],
DeepLRP [40], GINE [15], NGNN [25], GIN-AK+[26] and GraphSNN [37].
The results of QM9dataset are shown in Table 4.
We can see KP-GNN achieves state-of-the-art performance on mostof targets.
The results of MolHIV and ZINC dataset are shown in Table 5 and Table 6.
Although
KP-GNN does not achieve the best result, it is still comparable to other methods.
[Table]
[Table] [Table]
Running time comparison :
In this section, we [Table] compare the running time of KP-GNN to 1-hopmessage passing GNN and K -hop message passing GNN.
We use GIN [7] as the base model.
Wealso include the KP-GIN+.
All models use thesame number of layers and hidden dimensions fora fair comparison.
The results are shown in Table 7.
For D&D, ZINC, Graph property dataset,we set K = 5 , K = 4 , K = 6 respectively.
We can see the computational overhead grows nearlylinearly with K , which is far less than the theoretical analysis.
Conclusion
In this paper, we theoretically characterize the power of K -hop message passing GNNs and proposethe KP-GNN to improve the expressive power by leveraging the peripheral subgraph information ateach hop.
Theoretically, we prove that KP-GNN can distinguish almost all regular graphs includingsome distance regular graphs.
Empirically, KP-GNN achieves competitive results across all simulationand real-world datasets.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex-
perimental results (either in the supplemental material or as a URL)?
[Yes]
The
implementation of KP-GNN can be found at https://github.com/anoncodesub/KP-GNN.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[No] The license can be found in theirgithub.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A] All datasets we used are all open-sourced.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
There is no personal information in the
datasets.
If you used crowdsourcing or conducted research with human subjects..
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]