$p$-Laplacian Based Graph Neural Networks

1 INTRODUCTION . In this paper , we explore the usage of graph neural networks ( GNNs ) for semi-supervised node classification on graphs , especially when the graphs admit strong heterophily or noisy edges . Semisupervised learning problems on graphs are ubiquitous in a lot of real-world scenarios , such as user classification in social media ( Kipf & Welling , 2017 ) , protein classification in biology ( Velickovic et al. , 2018 ) , molecular property prediction in chemistry ( Duvenaud et al. , 2015 ) , and many others ( Marcheggiani & Titov , 2017 ; Satorras & Estrach , 2018 ) . Recently , GNNs are becoming the de facto choice for processing graph structured data . They can exploit the node features and the graph topology by propagating and transforming the features over the topology in each layer and thereby learn refined node representations . A series of GNN architectures have been proposed , including graph convolutional networks ( Bruna et al. , 2014 ; Henaff et al. , 2015 ; Defferrard et al. , 2016 ; Kipf & Welling , 2017 ; Wu et al. , 2019 ) , graph attention networks ( Velickovic et al. , 2018 ; Thekumparampil et al. , 2018 ) , and other representatives ( Hamilton et al. , 2017 ; Xu et al. , 2018 ; Pei et al. , 2020 ) . Most of the existing GNN architectures work under the homophily assumption , i.e . the labels of nodes and their neighbors in a graph are the same or consistent , which is also commonly used in graph clustering ( Bach & Jordan , 2004 ; von Luxburg , 2007 ; Liu & Han , 2013 ) and semi-supervised learning on graphs ( Belkin et al. , 2004 ; Hein , 2006 ; Nadler et al. , 2009 ) . However , recent studies ( Zhu et al. , 2020 ; 2021 ; Chien et al. , 2021 ) show that in contrast to their success on homophilic graphs , most GNNs fail to work well on heterophilic graphs , in which linked nodes are more likely to have distinct labels . Moreover , GNNs could even fail on graphs where their topology is not helpful for label prediction . In these cases , propagating and transforming node features over the graph topology could lead to worse performance than simply applying multi-layer perceptrons ( MLPs ) on each of the nodes independently . Several recent works were proposed to deal with the heterophily issues of GNNs . Zhu et al . ( 2020 ) finds that heuristically combining ego- , neighbor , and higher-order embeddings improves GNN performance on heterophilic graphs . Zhu et al . ( 2021 ) uses a compatibility matrix to model the graph homophily or heterphily level . Chien et al . ( 2021 ) incorporates the generalized PageRank algorithm with graph convolutions so as to jointly optimize node feature and topological information extraction for both homophilic and heterophilic graphs . However , the problem of GNNs on graphs with non-informative topologies ( or noisy edges ) remains open . Unlike previous works , we tackle the above issues of GNNs by proposing the discrete p-Laplacian based message passing scheme , termed as p-Laplacian message passing . It is derived from a discrete regularization framework and is theoretically verified as an approximation of a polynomial graph filter defined on the spectral domain of the p-Laplacian . Spectral analysis of p-Laplacian message passing shows that it works simultaneously as low-pass and high-pass filters 1 and thus is applicable to both homophilic and heterophilic graphs . Moreover , when p ̸= 2 , our theoretical results indicate that it can adaptively learn aggregation weights in terms of the variation of node embeddings on edges ( measured by the graph gradient ( Amghibech , 2003 ; Zhou & Schölkopf , 2005 ; Luo et al. , 2010 ) ) , and work as low-pass or both low-pass and high-pass filters on a node according to the local variation of node embeddings around the node ( measured by the norm of graph gradients ) . Based on p-Laplacian message passing , we propose a new GNN architecture , called pGNN , to enable GNNs to work with heterophilic graphs and graphs with non-informative topologies . Several existing GNN architectures , including SGC ( Wu et al. , 2019 ) , APPNP ( Klicpera et al. , 2019 ) and GPRGNN ( Chien et al. , 2021 ) , can be shown to be analogical to pGNN with p = 2 . Our empirical studies on real-world benchmark datasets ( homophilic and heterophilic datasets ) and synthetic datasets ( cSBM ( Deshpande et al. , 2018 ) ) demonstrate that pGNNs obtain the best performance on heterophilic graphs and competitive performance on homophilic graphs against state-of-the-art GNNs . Moreover , experimental results on graphs with different levels of noisy edges show that pGNNs work much more robustly than GNN baselines and even as well as MLPs on graphs with completely random edges . Additional experiments ( reported in Appendix F.5 ) illustrate that intergrating pGNNs with existing GNN architectures ( i.e . GCN ( Kipf & Welling , 2017 ) , JKNet ( Xu et al. , 2018 ) ) can significantly improve their performance on heterophilic graphs . In conclusion , our contributions can be summarized as below : ( 1 ) New methodologies . We propose p-Laplacian message passing and pGNN to adapt GNNs to heterophilic graphs and graphs where the topology is non-informative for label prediction . ( 2 ) Superior performance . We empirically demonstrate that pGNNs is superior on heterophilic graphs and competitive on homophilic graphs against state-of-the-art GNNs . Moreover , pGNNs work robustly on graphs with noisy edges or non-informative topologies . ( 3 ) Theoretical justification . We theoretically demonstrate that p-Laplacian message passing works as both low-pass and high-pass filters and the message passing iteration is guarantee to converge with proper settings . ( 4 ) New paradigm of designing GNN architectures . We bridge the gap between discrete regularization framework and GNNs , which could further inspire researchers to develop new graph convolutions or message passing schemes using other regularization techniques with explicit assumptions on graphs . Due to space limit , we defer the discussions on related work and future work and all proofs to the Appendix . 2 PRELIMINARIES AND BACKGROUND . Notation . Let G = ( V , E , W ) be an undirected graph , where V = { 1 , 2 , . . . , N } is the set of nodes , E ⊆ V × V is the set of edges , W ∈ RN×N is the adjacency matrix and Wi , j = Wj , i , Wi , j > 0 for [ i , j ] ∈ E , Wi , j = 0 , otherwise . Ni = { j } [ i , j ] ∈E denotes the set of neighbors of node i , D ∈ RN×N = diag ( D1,1 , . . . DN , N ) denotes the diagonal degree matrix with Di , i = ∑N j=1 Wi , j , for i = 1 , . . . , N . f : V → R and g : E → R are functions defined on the vertices and edges of G , respectively . FV denotes the Hilbert space of functions endowed with the inner product ⟨f , f̃⟩FV : =∑ i∈V f ( i ) f̃ ( i ) . Similarly define FE . We also denote by [ K ] = { 1 , 2 , . . . , K } , ∀K ∈ N and we use ∥x∥ = ∥x∥2 = ( ∑d i=1 x 2 i ) 1/2 , ∀x ∈ Rd to denote the Frobenius norm of a vector . Problem Formulation . Given a graph G = ( V , E , W ) , each node i ∈ V has a feature vector Xi , : which is the i-th row of X and a subset of nodes in G have labels from a label set L = { 1 , . . . , L } . The goal of semi-supervised node classification on G is to learn a mapping M : V → L and predict the labels of unlabeled nodes . 1Note that if the low frequencies and high frequencies dominate the middle frequencies ( the frequencies that are around the cutoff frequency ) , we say that the filter works both as low-pass and high-pass filters . Homophily and Heterophily . The homophily or heterophily of a graph is used to describe the relation of labels between linked nodes in the graphs . The level of homophily of a graph can be measured by H ( G ) = Ei∈V [ ∣∣ { j } j∈Ni , yi=yj ∣∣ /|Ni| ] ( Pei et al. , 2020 ; Chien et al. , 2021 ) , where∣∣ { j } j∈Ni , yi=yj ∣∣ denotes the number of neighbors of i ∈ V that share the same label as i and H ( G ) → 1 corresponds to strong homophily while H ( G ) → 0 indicates strong heterophily . We say that a graph is a homophilic ( heterophilic ) graph if it has strong homophily ( heterophily ) . Graph Gradient . The graph gradient of an edge [ i , j ] , i , j ∈ V is defined to be a measurement of the variation of a function f 2 : V → R on the edge [ i , j ] . Definition 1 ( Graph Gradient ) . Given a graph G = ( V , E ) and a function f : V → R , the graph gradient is an operator ∇ : FV → FE defined by ( ∇f ) ( [ i , j ] ) : = √ Wi , j Dj , j f ( j ) − √ Wi , j Di , i f ( i ) , for all [ i , j ] ∈ E . ( 1 ) For [ i , j ] /∈ E , ( ∇f ) ( [ i , j ] ) : = 0 . The graph gradient of a function f at vertex i is defined to be ∇f ( i ) : = ( ( ∇f ) ( [ i , 1 ] ) , . . . , ( ∇f ) ( [ i , N ] ) ) and its Frobenius norm is given by ∥∇f ( i ) ∥2 : = ( ∑N j=1 ( ∇f ) 2 ( [ i , j ] ) ) 1/2 , which measures the variation of f around node i . We measure the variation of f over the whole graph G by Sp ( f ) where it is defined to be Sp ( f ) : = 1 2 N∑ i=1 N∑ j=1 ∥ ( ∇f ) ( [ i , j ] ) ∥p = 1 2 N∑ i=1 N∑ j=1 ∥∥∥∥∥ √ Wi , j Dj , j f ( j ) − √ Wi , j Di , i f ( i ) ∥∥∥∥∥ p , for p ≥ 1 , ( 2 ) Note that the definition of Sp here is different with the p-Dirichlet form in Zhou & Schölkopf ( 2005 ) . Graph Divergence . The graph divergence is defined to be the adjoint of the graph gradient : Definition 2 ( Graph Divergence ) . Given a graph G = ( V , E ) , and functions f : V → R , g : E → R , the graph divergence is an operator div : FE → FV which satisfies ⟨∇f , g⟩ = ⟨f , −divg⟩ . ( 3 ) The graph divergence can be computed by ( divg ) ( i ) = N∑ j=1 √ Wi , j Di , i ( g ( [ i , j ] ) − g ( [ j , i ] ) ) . ( 4 ) Fig . 4 in Appendix E.1 gives a tiny example of illustration of graph gradient and graph divergence . Graph p-Laplacian Operator . By the definitions of graph gradient and graph divergence , we reach the definition of graph p-Laplacian operator as below . Definition 3 ( Graph p-Laplacian3 ) . Given a graph G = ( V , E ) and a function f : V → R , the graph p-Laplacian is an operator ∆p : FV → FV defined by ∆pf : = − 1 2 div ( ∥∇f∥p−2∇f ) , for p ≥ 1 . ( 5 ) where ∥ · ∥p−2 is element-wise , i.e . ∥∇f ( i ) ∥p−2 = ( ∥ ( ∇f ) ( [ i , 1 ] ) ∥p−2 , . . . , ∥ ( ∇f ) ( [ i , N ] ) ∥p−2 ) . Substituting Eq . ( 1 ) and Eq . ( 4 ) into Eq . ( 5 ) , we obtain ( ∆pf ) ( i ) = N∑ j=1 √ Wi , j Di , i ∥ ( ∇f ) ( [ j , i ] ) ∥p−2 ( √ Wi , j Di , i f ( i ) − √ Wi , j Dj , j f ( j ) ) ( 6 ) The graph p-Laplacian is semi-definite : ⟨f , ∆pf⟩ = Sp ( f ) ≥ 0 and we have ∂Sp ( f ) ∂f ∣∣∣∣ i = p ( ∆pf ) ( i ) . ( 7 ) When p = 2 , ∆2 is refered as the ordinary Laplacian operator and ∆2 = I − D−1/2WD−1/2 and when p = 1 , ∆1 is refered as the Curvature operator and ∆1f : = − 12div ( ∥∇f∥ −1∇f ) . Note that Laplacian ∆2 is a linear operator , while in general for p ̸= 2 , p-Laplacian is nonlinear since ∆p ( af ) ̸= a∆p ( f ) for a ∈ R. 2f can be a vector function : f : V → Rc for some c ∈ N and here we use f : V → R for better illustration . 3Note that the definition adopted is slightly different with the one used in Zhou & Schölkopf ( 2005 ) where ∥ · ∥p−2 is not element-wise and the one used in some literature such as Amghibech ( 2003 ) ; Bühler & Hein ( 2009 ) , where ( ∆pf ) ( i ) = ∑N j=1 Wi , j Di , i |f ( i ) − f ( j ) |p−2 ( f ( i ) − f ( j ) ) for p > 1 and p = 1 is not allowed . 3 p-LAPLACIAN BASED GRAPH NEURAL NETWORKS In this section , we derive the p-Laplacian message passing scheme from a p-Laplacian regularization framework and present pGNN , a new GNN architecture developed upon the new message passing scheme . We theoretically characterize how p-Laplacian message passing adaptively learns aggregation weights and profits pGNN for being effective on both homophilic and heterophilic graphs . 3.1 p-LAPLACIAN REGULARIZATION FRAMEWORK Given an undirected graph G = ( V , E ) and a signal function with c ( c ∈ N ) channels f : V → Rc , let X = ( X⊤1 , : , . . . , X ⊤ N , : ) ⊤ ∈ RN×c with Xi , : ∈ R1×c , i ∈ [ N ] denoting the node features of G and F = ( F⊤1 , : , . . . , F ⊤ N , : ) ⊤ ∈ RN×c be a matrix whose ith row vector Fi , : ∈ R1×c , i ∈ [ N ] represents the function value of f at the i-th vertex in G. We present a p-Laplacian regularization problem whose cost function is defined to be F∗ = argmin F L ( F ) : = argmin F Sp ( F ) + µ N∑ i=1 ∥Fi , : −Xi , :∥2 , ( 8 ) where µ ∈ ( 0 , ∞ ) . The first term of the right-hand side in Eq . ( 8 ) is a measurement of variation of the signal over the graph based on p-Laplacian . As we will discuss later , different choices of p result in different smoothness constraint on the signals . The second term is the constraint that the optimal signals F∗ should not change too much from the input signal X , and µ provides a trade-off between these two constraints . Regularization with p = 2 . When p = 2 , the solution of Eq . ( 8 ) satisfies ∆2F∗ + µ ( F∗ −X ) = 0 and we can obtain the closed form ( Zhou et al. , 2003 ; Zhou & Schölkopf , 2005 ) F∗ = µ ( ∆2 + µIN ) −1X . ( 9 ) Then , we could use the following iteration algorithm to get an approximation of Eq . ( 9 ) : F ( k+1 ) = αD−1/2WD−1/2F ( k ) + βX , ( 10 ) where k represents the iteration index , α = 11+µ and β = µ 1+µ = 1− α . The iteration converges to a closed-form solution as k goes to infinity ( Zhou et al. , 2003 ; Zhou & Schölkopf , 2005 ) . We could relate the the result here with the personalized PageRank ( PPR ) ( Page et al. , 1999 ; Klicpera et al. , 2019 ) algorithm ( proof defered to Appendix D.1 ) : Theorem 1 ( Relation to personalized PageRank ( Klicpera et al. , 2019 ) ) . µ ( ∆2 + µIN ) −1 in the closed form solution of Eq . ( 9 ) is equivalent to the personalized PageRank matrix . Regularization with p > 1 . For p > 1 , the solution of Eq . ( 8 ) satisfies p∆pF∗ +2µ ( F∗ −X ) = 0 . By Eq . ( 6 ) we have that , for all i ∈ [ N ] , N∑ j=1 Wi , j√ Di , i ∥ ( ∇f∗ ) ( [ j , i ] ) ∥p−2 ( 1√ Di , i F∗i , : − 1√ Dj , j F∗j , : ) + 2µ p ( F∗i , : −Xi , : ) = 0 . Based on which we can construct a similar iterative algorithm to obtain a solution ( Zhou & Schölkopf , 2005 ) : F ( k+1 ) i , : = α ( k ) i , i N∑ j=1 M ( k ) i , j√ Di , iDj , j F ( k ) j , : + β ( k ) i , i Xi , : , for all i ∈ [ N ] , ( 11 ) with M ( k ) ∈ RN×N , α ( k ) = diag ( α ( k ) 1,1 , . . . , α ( k ) N , N ) , β ( k ) = diag ( β ( k ) 1,1 , . . . , β ( k ) N , N ) updated by M ( k ) i , j = Wi , j ∥∥∥∥∥ √ Wi , j Di , i F ( k ) i , : − √ Wi , j Dj , j F ( k ) j , : ∥∥∥∥∥ p−2 , for all i , j ∈ [ N ] , ( 12 ) α ( k ) i , i = 1 / N∑ j=1 M ( k ) i , j Di , i + 2µ p , β ( k ) i , i = 2µp αi , i , for all i ∈ [ N ] , ( 13 ) Note that in Eq . ( 12 ) , when ∥∥∥√Wi , jDi , i F ( k ) i , : −√Wi , jDj , j F ( k ) j , : ∥∥∥ = 0 , we set M ( k ) i , j = 0 . It is easy to see that Eq . ( 10 ) is the special cases of Eq . ( 14 ) with p = 2 . Remark 1 ( Discussion on p = 1 ) . For p = 1 , when f is a real-valued function ( c = 1 ) , ∆1f is a step function , which could make the stationary condition of the objective function Eq . ( 8 ) become problematic . Additionally , ∆1f is not continuous at ∥ ( ∇f ) ( [ i , j ] ) ∥ = 0 . Therefore , p = 1 is not allowed when f is a real value function . On the other hand , note that there is a Frobenius norm in ∆pf . When f is a vector-valued function ( c > 1 ) , the step function in ∆1f only exists on the axes . The stationary condition will be fine if the node embeddings F are not a matrix of vectors that has only one non-zero element , which is true for many graphs . p = 1 may work for these graphs . Overall , we suggest to use p > 1 in practice but p = 1 may work for graphs with multiple channel signals as well . We conduct experiments for p > 1 ( e.g. , p = 1.5 , 2 , 2.5 ) and p = 1 in Sec . 5 . 3.2 p-LAPLACIAN MESSAGE PASSING AND pGNN ARCHITECTURE p-Laplacian Message Passing . Rewrite Eq . ( 11 ) in a matrix form we obtain F ( k+1 ) = α ( k ) D−1/2M ( k ) D−1/2F ( k ) + β ( k ) X . ( 14 ) Eq . ( 14 ) provides a new message passing mechanism , named p-Laplacian message passing . Remark 2. αD−1/2MD−1/2 in Eq . ( 14 ) can be regarded as the learned aggregation weights at each step for message passing . It suggests that p-Laplacian message passing could adaptively tune the aggregation weights during the course of learning , which will be demonstrated theoretically and empirically in the sequel of this paper . βX in Eq . ( 14 ) can be regarded as a residual unit , which helps the model escape from the oversmoothing issue ( Chien et al. , 2021 ) . We present the following theorem to show the shrinking property of p-Laplacian message passing . Theorem 2 ( Shrinking Property of p-Laplacian Message Passing ) . Given a graph G = ( V , E , W ) with node features X , β ( k ) , F ( k ) , M ( k ) , α ( k ) are updated accordingly to Equations ( 11 ) to ( 13 ) for k = 0 , 1 , . . . , K and F ( 0 ) = X . Then there exist some positive real value µ > 0 which depends on X , G , p and p > 1 such that Lp ( F ( k+1 ) ) ≤ Lp ( F ( k ) ) . Proof see Appendix D.2 . Thm . 2 shows that with some proper positive real value µ and p > 1 , the loss of the objective function Eq . ( 8 ) is guaranteed to decline after taking one step p-Laplacian message passing . Thm . 2 also demonstrates that the iteration Equations ( 11 ) to ( 13 ) is guaranteed to converge for p > 1 with some proper µ which is chosen depends on the input graph and p. pGNN Architecture . We design the architecture of pGNNs using p-Laplacian message passing . Given node features X ∈ RN×c , the number of node labels L , the number of hidden units h , the maximum number of iterations K , and M , α , and β updated by Equations ( 12 ) and ( 13 ) respectively , we give the pGNN architecture as following : F ( 0 ) = ReLU ( XΘ ( 1 ) ) , ( 15 ) F ( k+1 ) = α ( k ) D−1/2M ( k ) D−1/2F ( k ) + β ( k ) F ( 0 ) , k = 0 , 1 , . . . , K − 1 , ( 16 ) Z = softmax ( F ( K ) Θ ( 2 ) ) , ( 17 ) where Z ∈ RN×L is the output propbability matrix with Zi , j is the estimated probability that the label at node i ∈ [ N ] is j ∈ [ L ] given the features X and the graph G , Θ ( 1 ) ∈ Rc×h and Θ ( 2 ) ∈ Rh×L are the first- and the second-layer parameters of the neural network , respectively . Remark 3 ( Connection to existing GNN variants ) . The message passing scheme of pGNNs is different from that of several GNN variants ( say , GCN , GAT , and GraphSage ) , which repeatedly stack message passing layers . In contrast , it is similar with SGC ( Wu et al. , 2019 ) , APPNP ( Klicpera et al. , 2019 ) , and GPRGNN ( Chien et al. , 2021 ) . SGC is an approximation to the closed-form in Eq . ( 9 ) ( Fu et al. , 2020 ) . By Thm . 1 , it is easy to see that APPNP , which uses PPR to propagate the node embeddings , is analogical to pGNN with p = 2 , termed as 2.0GNN . APPNP and 2.0GNN work analogically and effectively on homophilic graphs . 2.0GNN can also work effectively on heterophilic graphs by letting Θ ( 2 ) be negative . However , APPNP fails on heterophilic graphs as its PPR weights are fixed ( Chien et al. , 2021 ) . Unlike APPNP , GPRGNN , which adaptively learn the generalized PageRank ( GPR ) weights , works similarly to 2.0GNN on both homophilic and heterophilic graphs . However , GPRGNN needs more supervised information in order to learn optimal GPR weights . On the contrary , pGNNs need less supervised information to obtain similar results because Θ ( 2 ) acts like a hyperplane for classification . pGNNs could work better under weak supervised information . Our analysis is also verified by the experimental results in Sec . 5 . We also provide an upper-bounding risk of pGNNs by Thm . 4 in Appendix C.1 to study the effect of the hyperparameter µ on the performance of pGNNs . Thm . 4 shows that the risk of pGNNs is upperbounded by the sum of three terms : the risk of label prediction using only the original node features X , the norm of p-Laplacian diffusion on X , and the magnitude of the noise in X. µ controls the trade-off between these three terms . The smaller µ , the more weights on the p-Laplacian diffusion term and the noise term and the less weights on the the other term and vice versa . 4 SPECTRAL VIEWS OF p-LAPLACIAN MESSAGE PASSING In this section , we theoretically demonstrate that p-Laplacian message passing is an approximation of a polynomial graph filter defined on the spectral domain of p-Laplacian . We show by spectral analysis that p-Laplacian message passing works simultaneously as low-pass and high-pass filters . p-Eigenvalues and p-Eigenvectors of the Graph p-Laplacian . We first introduce the definitions of p-eigenvalues and p-eigenvectors of p-Laplacian . Let ϕp : R → R defined as ϕp ( u ) = ∥u∥p−2u , for u ∈ R , u ̸= 0 . Note that ϕ2 ( u ) = u . For notational simplicity , we denote by ϕp ( u ) = ( ϕp ( u1 ) , . . . , ϕp ( uN ) ) ⊤ for u ∈ RN and Φp ( U ) = ( ϕp ( U : ,1 ) , . . . , ϕp ( U : ,N ) ) for U ∈ RN×N and U : ,i ∈ RN is the ith column vector of U . Definition 4 ( p-Eigenvector and p-Eigenvalue ) . A vector u ∈ RN is a p-eigenvector of ∆p if it satisfies the equation ( ∆pu ) i = λϕp ( ui ) , for all i ∈ [ N ] , where λ ∈ R is a real value referred as a p-eigenvalue of ∆p associated with the p-eigenvector u . Definition 5 ( p-Orthogonal ( Luo , Huang , Ding , and Nie , 2010 ) ) . Given two vectors u , v ∈ RN with u , v ̸= 0 , we call that u and v is p-orthogonal if ϕp ( u ) ⊤ϕp ( v ) = ∑N i=1 ϕp ( ui ) ϕp ( vi ) = 0 . Luo et al . ( 2010 ) demonstrated that the p-eigenvectors of ∆p are p-orthogonal to each other ( see Thm . 5 in Appendix C.2 for details ) . Therefore , the space spanned by the multiple p-eigenvectors of ∆p is p-orthogonal . Additionally , we demonstrate that the p-eigen-decomposition of ∆p is given by : ∆p = Φp ( U ) ΛΦp ( U ) ⊤ ( see Thm . 6 in Appendix C.3 for details ) , where U is a matrix of p-eigenvectors of ∆p and Λ is a diagonal matrix in which the diagonal is the p-eigenvalues of ∆p . Graph Convolutions based on p-Laplacian . Based on Thm . 5 , the graph Fourier Transform f̂ of any function f on the vertices of G can be defined as the expansion of f in terms of Φ ( U ) where U is the matrix of p-eigenvectors of ∆p : f̂ = Φp ( U ) ⊤f . Similarly , the inverse graph Fourier transform is then given by : f = Φp ( U ) f̂ . Therefore , a signal X ∈ RN×c being filtered by a spectral filter gθ can be expressed formally as : gθ ⋆ X = Φp ( U ) ĝθ ( Λ ) Φp ( U ) ⊤X , where Λ denotes a diagonal matrix in which the diagonal corresponds to the p-eigenvalues { λl } l=0 , ... , N−1 of ∆p and ĝθ ( Λ ) denotes a diagonal matrix in which the diagonal corresponds to spectral filter coefficients . Let ĝθ be a polynomial filter defined as ĝθ = ∑K−1 k=0 θkλ k l , where the parameter θ = [ θ0 , . . . , θK−1 ] ⊤ ∈ RK is a vector of polynomial coefficients . By the p-eigen-decomposition of p-Laplacian , we have gθ ⋆X ≈ K−1∑ k=0 θkΦp ( U ) Λ kΦp ( U ) ⊤X = K−1∑ k=0 θk∆ k pX . ( 18 ) Theorem 3 . The K-step p-Laplacian message passing is a K-order polynomial approximation to the graph filter given by Eq . ( 18 ) . Proof see Appendix D.3 . Thm . 3 indicates that p-Laplacian message passing mechanism is implicitly a polynomial spectral filter defined on the spectral domain of p-Laplacian . Spectral Analysis of p-Laplacian Message Passing . Here , we analyze the spectral propecties of p-Laplacian message passing . We can approximately view p-Laplacian message pasing as a filter of a linear combination of K spectral filters g ( Λ ) ( 0 ) , g ( Λ ) ( 1 ) , . . . , g ( Λ ) ( K−1 ) with each spectral filter defined to be g ( Λ ) ( k ) : = ( αD−1/2MD−1/2 ) k where Mi , j = Wi , j∥ √ Wi , j Di , i Fi , : − √ Wi , j Dj , j Fj , :∥p−2 for i , j ∈ [ N ] and F is the matrix of node embeddings . We can study the properties of p-Laplacian message passing by studying the spectral properties of αD−1/2MD−1/2 as given below . Proposition 1 . Given a connected graph G = ( V , E , W ) with node embeddings F and the pLaplacian ∆p with its p-eigenvectors { u ( l ) } l=0,1 , ... , N−1 and the p-eigenvalues { λl } l=0,1 , ... , N−1 . Let gp ( λi−1 ) : = αi , i ∑ j D −1/2 i , i Mi , jD −1/2 j , j for i ∈ [ N ] be the filters defined on the spectral domain of ∆p , where Mi , j = Wi , j∥∇f ( [ i , j ] ) ∥p−2 , ( ∇f ) ( [ i , j ] ) is the graph gradient of the edge between node i and j and ∥∇f ( i ) ∥ is the norm of graph gradient at i. Ni denotes the number of edges connected to i , Nmin = min { Nj } j∈ [ N ] , and k = argmaxj ( { |u ( l ) j |/ √ Dl , l } j∈ [ N ] ; l=0 , ... , N−1 ) , then 1 . When p = 2 , gp ( λi−1 ) works as both low-pass and high-pass filters . 2 . When p > 2 , if ∥∇f ( i ) ∥ ≤ 2 ( p−1 ) / ( p−2 ) , gp ( λi−1 ) works as both low-pass and high-pass filters on node i and gp ( λi−1 ) works as low-pass filters on i when ∥∇f ( i ) ∥ ≥ 2 ( p−1 ) / ( p−2 ) . 3 . When 1 ≤ p < 2 , if 0 ≤ ∥∇f ( i ) ∥ ≤ 2 ( 2 √ Nk ) 1/ ( p−2 ) , gp ( λi−1 ) works as low-pass filters on node i and gp ( λi−1 ) works as both low-pass and high-pass filters on i when ∥∇f ( i ) ∥ ≥ 2 ( 2 √ Nk ) 1/ ( p−2 ) . Specifically , when p = 1 , Nk can be replaced by Nmin . Proof see Appendix D.7 . Proposition 1 shows that when p ̸= 2 , p-Laplacian message passing adaptively works as low-pass or both low-pass and high-pass filters on node i in terms of the degree of local node embedding variation around i , i.e . the norm of the graph gradient ∥∇f ( i ) ∥ at node i . When p = 2 , p-Laplacian message passing works as both low-pass and high-pass filters on node i regardless of the value of ∥∇f ( i ) ∥ . When p > 2 , p-Laplacian message passing works as lowpass filters on node i for large ∥∇f ( i ) ∥ and works as both low-pass and high-pass filters for small ∥∇f ( i ) ∥ . Therefore , pGNNs with p > 2 can work very effectively on graphs with strong homophily . When 1 ≤ p < 2 , p-Laplacian message passing works as low-pass filters for small ∥∇f ( i ) ∥ and works as both low-pass and high-pass filters for large ∥∇f ( i ) ∥ . Thus , pGNNs with 1 ≤ p < 2 can work effectively on graphs with low homophily , i.e . heterophilic graphs . The results here confirms our analysis of the aggregation weights of p-Laplacian message passing presented in Thm . 2 .

Summary: The paper derives the p-Laplacian message passing formula under the p-Laplacian based regularization framework and further proposes p-GNN architecture. Authors further justify the relations of p-Laplacian message passing with low and high-pass filters and the upper bound of one layer risk of p-GNNs. Experiments show the superiority of p-GNNs on both heterophilic and homophilic settings. However, we still have some concerns for the paper before further evaluation.