On the Robustness of Graph Neural Diffusion to Topology Perturbations
Abstract
Neural diffusion on graphs is a novel class of graph neural networks that has attracted increasing attention recently.
The capability of graph neural partial differential equations (PDEs) in addressing common hurdles of graph neural networks (GNNs), such as the problems of over-smoothing and bottlenecks, has been investigated but not their robustness to adversarial attacks.
In this work, we explore the robustness properties of graph neural PDEs.
We empirically demonstrate that graph neural PDEs are intrinsically more robust against topology perturbation as compared to other GNNs.
We provide insights into this phenomenon by exploiting the stability of the heat semigroup under graph topology perturbations.
We discuss various graph diffusion operators and relate them to existing graph neural PDEs.
Furthermore, we propose a general graph neural PDE framework based on which a new class of robust GNNs can be defined.
We verify that the new model achieves comparable state-of-the-art performance on several benchmark datasets.
Introduction
Deep learning on graphs and Graph Neural Networks (GNNs), in particular, have achieved remarkable success in a variety of application areas such as those related to chemistry (molecules), finance (trading networks) and the social media (the Facebook friend network)
GNNs have been applied to various tasks including node classification [3], link prediction [6] and recommender systems [7].
The key to the success of GNNs is the neural message passing scheme [8] where messages are propagated along edges and optimized toward a downstream task.
While aggregating neighboring nodes’ information is a powerful principle of representation learning, the way that GNNs exchange information between nodes makes them vulnerable to adversarial attacks [9].
Adversaries can perturb a graph’s topology by adding or removing edges [10–12] or by injecting malicious nodes to the original graph [13–15].
Another common attack is to perturb node attributes [9, 16–18].
Our paper will mainly tackle graph topology perturbation.
Adversaries who can inject nodes to the original graph while not modifying the original graph directly are called injection attacks [13–15].
Adversaries who can directly modify the original graph including edges and node features are called modification attacks [10–12, 9, 16–18].
To defend against adversarial attacks, several robust GNN models have been proposed.
Examples include RobustGCN [19], GRAND [20], and ProGNN [21].
In addition, pre-processing based defenders include GNN-SVD [22] and
GNNGuard [23].
Recent studies [24–27] have applied neural Ordinary Differential Equations (ODEs) [28] to defend against adversarial attacks.
Some works like [24, 27] have revealed interesting intrinsic properties of
ODEs that make them more stable than conventional convolutional neural networks (CNNs).
Neural
Partial Differential Equations (PDEs) have also been applied to graph-structured data [29, 30].
Some papers like [29] (GRAND) and [30] (BLEND) approach deep learning on graphs as a continuous diffusion process and treat GNNs as spatial discretizations of an underlying PDE.
However, robustness to adversarial attack has not been studied on such graph neural PDEs.
In this work, we demonstrate that graph neural PDEs are intrinsically more robust against adversarial topological perturbations compared to other GNNs.
We investigate the heat diffusion on a general Riemannian manifold and show that the diffusion process is essentially stable under small perturbations of the manifold metric.
In doing so, we provide insights into the underlining reasons why graph neural PDEs are stable under graph topological perturbations.
Such insights indicate that further improvements to the design of graph neural PDEs are possible.
We develop several such improved models under a general graph neural PDE framework and show that these models are also robust to node attribute perturbations.
Main contributions .
In this paper, our objective is to develop a general diffusion framework on graphs and study the robustness properties of the induced graph neural PDEs.
Our main contributions are summarized as follows:
• We review the notion of heat diffusion on Riemannian manifolds and the stability of its semigroup.
We present analogous concepts of gradient, divergence, and Laplacian operators for heat diffusion on a graph.
• We generalize heat flow to more general flow schemes, including mean curvature flow and Beltrami flow, which are able to preserve inter-class edges in diffusion.
A novel class of graph neural PDEs is thereby induced.
• We show that the proposed graph neural PDEs are intrinsically robust to graph topology perturbations.
We verify that the new model achieves comparable state-of-the-art performance on several benchmark datasets.
The rest of this paper is organized as follows.
In Section 2, we start with preliminaries on continuous diffusion over a Riemannian manifold and the discrete graph.
In Section 3, we present our main results on the stability properties of heat flow on graphs and generalize heat flow to edge-preserving flows from which a new class of graph neural PDEs is proposed.
Our proposed model architecture is detailed in Section 4.
We summarize experimental results in Section 5 and conclude the paper in
Section 6.
The proofs for all theoretical results in this paper are given in the supplementary material, where more experiments are also presented.
Preliminaries
Similar to [29, 30], we consider a graph as a discretization of a Riemannian manifold.
We introduce concepts and notations for flows diffused over a general manifold and a discrete graph.
Readers are referred to [31] for more details.
The stability of the heat kernel and heat semigroup of the heat diffusion equation under perturbations of the manifold metric is considered.
This discussion sheds light on the diffusion stability of graphs under perturbation of the graph topology in the next section.
2.1 Heat Equation and Solution Stability on Manifold
A Riemannian manifold ( M, g ) is a smooth manifold M endowed with a Riemannian metric g , where the norms of tangent vectors and the angles between them are defined by an inner product.
It is the natural generalization of Euclidean space and correspondingly the divergence, gradient, and
Laplace operators are well-defined, analogous to their corresponding concepts in Euclidean space.
Let C ∞ 0 ( M ) denote the space of smooth functions on M with compact support and ◦ the composition operation.
Definition 1.
The Laplace operator ∆ : C ∞ 0 ( M ) (cid:55)→ C ∞ 0 ( M ) on a d -dimensional Riemannian manifold ( M, g ) is defined as
[Equation]
where div is the divergence defined for the C ∞ 0 vector fields on M and ∇ is the gradient operator.
In a local chart U with coordinates ( x 1 , x 2 , . . . , x n ) , for function f and vector field ψ , we have
[Equation]
where ψ j is the j -th component of ψ , g ij the components of the inverse metric of g , and det is the determinant operator of a matrix.
We also have
[Equation]
The classical Cauchy problem associated with the heat diffusion equation is to find a function φ ( t, x ) ∈ C ∞
( R + × M ) such that
[Equation]
where we only consider f ∈ C ∞ 0 and R + is the space of positive real numbers.
We can extend the Laplace operator to the generalized Dirichlet Laplace operator , denoted as
L = − ∆ and defined on a larger domain 2 [31, section 4.2].
This operator is self-adjoint and nonnegative definite on L 2 ( M ) , and it can be shown that the above problem (3) is solved by means of the following family { P t } t ≥ 0 of operators:
[Equation]
where { E λ } λ ∈ spec L in L 2 ( M ) is the unique spectral resolution of the Dirichlet Laplace operator L and spec L is the spectrum set of L .
The family { P t } t ≥ 0 is called the heat semigroup associated with ∆ , and the solution is given by P t f .
It is also well-known that the solution of the above problem has an integration form via the (minimal) heat kernel function k t ( x, y ) :
R + × M × M
(cid:55)→ R [31,
Theorem 7.13]:
[Equation]
where d µ is the Riemannian measure [31, Section 3.4] on M .
For a Riemannian manifold M , different metrics g give rise to different structures.
Let (cid:101) ∆ be the
Laplacian associated with another metric ˜ g on M such that, for some α ≥ 1 ,
[Equation]
The new (cid:101) ∆ is said to be quasi-isometric [32] to g , and can be viewed as a perturbation of the metric g .
(cid:101) ∆ can be also viewed as a uniformly elliptic operator [32] with respect to (w.r.t.) to g .
The stability of the heat semigroup and the heat kernel under perturbations of the Laplace operator (i.e., changes of g ) is well studied in [33].
In the special case of uniformly elliptic operators on Euclidean manifold R d , the uniformly elliptic operator can be written as div( A ∇ ) with symmetric measurable coefficients
A ( x ) =
( a ij ( x )) such that (cid:80) i,j a ij ( x ) ξ i ξ j ≥ λ 0 | ξ | 2 for all ξ ∈ R d and for some constant λ >independent of x .
The reference [33] shows the following theorem:
Theorem 1.
On the Euclidean manifold R n , let { P t } t ≥ 0 and { (cid:101) P t } t ≥ 0 be two diffusion semigroups on R d ( d ≥ 2) associated with uniformly elliptic operators div( A ∇ ) and div( ˜ A ∇ ) with symmetric measurable coefficients
A ( x ) =
( a ij ( x )) and ˜ A ( x ) =
(˜ a ij ( x )) , respectively.
The corresponding heat kernels are denoted by p t ( x, y ) and ˜ p t ( x, y ) .
We then have that
i. There is a bounded, piecewise continuous function F 1 ( t, z ) on R + × R + with lim
z
→ 0 F 1 ( t, z ) =
for each t > 0 and a constant c > 0 , both of which depend only on d and α , such that
[Equation]
for any ( t, x, y ) ∈ R + × R d × R d , ∥ · ∥ 2 is the Euclidean norm, and ∥ · ∥ L 2 loc is the local L 2 -norm distance defined in [33]. ii.
Furthermore, we also have an L p -operator norm bound for P t − (cid:101) P t in terms of the local L 2 -norm distance between a ij and ˜ a ij : There is a bounded, piecewise continuous function F 2 ( t, z ) on R + × R + with lim
z → 0 F 2 ( t, z )
=
for each t > 0 that depends only on d and α , such that for any p ∈ [1 , ∞ ] , we have
[Equation]
The above theorem shows the pointwise convergence of the heat kernel under perturbations of the matrix A .
It proves the stability of the solution under small perturbations of the Laplace operator.
These results can also be extended to general Riemannian manifolds [34] since every Riemannian manifold can be isometrically embedded into some Euclidean space [35] and the kernel is isometrically invariant [31, theorem 9.12].
We therefore know that if the difference between ˜ A and A (or more generally ˜ g and g ) are small, the final solutions to (3) have small difference.
An intuitive explanation of the stability of the solution comes from the fact that the heat kernel k t ( x, y ) is the transition density function of a Brownian motion on the manifold [36] since Brownian motion can be viewed as a diffusion generated by half of the Laplace operator.
This means that k t ( x, y ) is a weighted average over all possible paths between x and y at time t , which does not change dramatically under small perturbations of g .
More specifically, for a Brownian motion on a domain M , if we perturb the g over a subset P ⊂ M then only the paths passing through P will be affected.
If we go one step further from the uniformly elliptic operators div( A ∇ ) defined in Theorem 1 by extending A ( x ) φ to A ( x, t, φ, ∇ φ ) , we get the following general quasilinear parabolic equation :
[Equation]
where the principle part A ( x, t, φ, ∇ φ ) is equipped with additional structure conditions as shown in [37].
When A ( x, t, φ, ∇ φ )
=
A ( x, t ) is a time-dependent diffusion process, analogous bounds to
Theorem 1 are shown in [38].
We leave the analysis of general A ( x, t, φ, ∇ φ ) for future work.
2.2 Gradient and Divergence Operators on Graphs
We consider a graph as a discretization of a Riemannian manifold.
More specifically, consider an undirected graph G =
( V , E ) consisting of a finite set V of vertices, together with a subset E ⊂ V × V of edges.
A graph is weighted when it is associated with a function w : E (cid:55)→ R + which is symmetric, i.e., w ([ u, v ])
=
w ([ v, u ]) , for all [ u, v ] ∈ E .
Let H ( V ) denote the Hilbert space on V of real-valued functions with the inner product defined as ⟨ a, b ⟩ H ( V )
= (cid:80) v ∈V a ( v ) b ( v ) , for all a, b ∈ H ( V ) .
Similarly, we define a Hilbert space H ( E ) with inner product ⟨ c, d ⟩ H ( E ) =
(cid:80) [ u,v ] ∈E c ([ u, v ])
d ([ u, v ]) , for all c, d ∈ H ( E ) .
We next define the gradient and divergence operators on graphs analogue to the general continuous manifold defined in
Definition 1 as follows [39].
Definition 2.
Given an undirected graph G =
( V , E ) , we define the following:
i. The graph gradient is an operator ∇ : H ( V ) (cid:55)→ H ( E ) defined by
[Equation]
where h ( v ) =
(cid:80) [ u,v ] ∈E w ([ u, v ]) is the degree of node v .
ii.
The graph divergence is an operator div : H ( E ) (cid:55)→ H ( V ) defined by
[Equation]
iii.
The graph Laplacian is an operator ∆ : H ( V ) (cid:55)→
H
[Equation]
V ) defined by
[Equation]
The graph Laplacian defined above is identical to the normalized Laplacian matrix, i.e.,
[Equation]
where D is a diagonal matrix with D ( v, v )
=
h ( v ) , and W is the adjacency matrix satisfying
W ( u, v ) =
w ([ u, v ]) if [ u, v ] ∈ E and W ( u, v ) = 0 otherwise.
Note that in (8) we have included the negative sign.
The analogue of the graph Laplacian ∆ in Section 2.1 is the Dirichlet Laplace operator L .
Note the use of the same notation ∆ for graphs.
The manifold Laplace operator ∆ and graph Laplacian ∆ will be apparent from the context.
Neural Diffusion and Stability on Graphs
We now consider neural diffusion on graphs by making use of concepts from Section 2.
Various parabolic-type equations on graphs are studied in this section.
The solution stability against adversarial attacks is linked to the stability of the solution for the heat diffusion equation under perturbation of the manifold metric, introduced in Section 2.1.
The general framework we consider is based on parabolic-type equations on graphs:
[Equation]
with φ ( u, 0) being the initial node attribute at node u and where A ( u, t, φ, ∇ φ ) can take different forms.
We next provide some examples.
3.1 Continuous Diffusion on Graphs
Definition 3 (Heat diffusion) .
The heat diffusion on graphs is defined by
[Equation]
Definition 4 (GRAND/BLEND [29, 30]) .
According to [29, eq (1)] and [30, eq (7) and (9)], the
GRAND/BLEND flow is defined by
[Equation]
where ∇ t is an adaptive Laplace operator (with possible graph rewiring) depending on the evolved node feature φ ( · , t ) .
The gradient operator defined in GRAND/BLEND assumes constant edge weight.
In Definition 4, the weight function w ([ u, v ]) defined in (6) is incorporated into the gradient definition, so it can be absorbed in the time-dependent term ∇ t φ .
Note that BLEND degenerates to GRAND [29] when there is no positional encoding.
In this paper, for a fair comparison, we do not use positional encoding for all GNNs.
Analogous to [40], we can define mean curvature flow and Beltrami flow as follows.
Definition 5.
The mean curvature diffusion on graphs is defined by
[Equation]
[Equation]
where − 1 2div (cid:16) ∇ φ ∥∇ φ ∥ (cid:17) is a discrete analogue of the mean curvature operator, ∥∇ φ ∥ = ⟨∇ φ, ∇ φ ⟩ 1 / 2 H ( E ) and ∥∇ φ ( u, t ) ∥
= (cid:16)(cid:80)
[ v,u ] ∈E ( ∇ φ ([ u, v ] , t )) 2 (cid:17) 1 / 2 .
Definition 6.
The Beltrami diffusion on graphs is defined by
[Equation]
[Equation] Intuitively, the term ∥∇ φ ( u, t ) ∥ , which appears in (13) and (14) but not in (11), measures the smoothness of the signals in the neighborhood around vertex u at time t .
The diffusion using (13) or (14) at vertex u is small when ∥∇ φ ( u, t ) ∥ is large, i.e., signals are less smooth around vertex u .
Hence, mean curvature flow and Beltrami flow are able to preserve the non-smooth graph signals.
This phenomenon is visualized in Fig.
1, where mean curvature flow and Beltrami flow are capable of preserving inter-class edges whose weights are much larger than those in heat flow.
3.2 Stability against Graph Topology Perturbation
Attackers can modify the original graph by adding or removing edges [10–12] and perturbing node attributes.
Adding or removing edges leads to a different graph Laplacian ∆ defined in (8).
Consequently, the solution to (11) is altered.
Using the language of Section 2.1, the edge perturbations correspond to the perturbation of the metric g (or equivalently A in Theorem 1).
According to
Theorem 1, the stability of the heat semigroup and the solutions under small perturbations of the
Laplace operator is guaranteed.
Consequently, the solution is not affected significantly by the edge perturbations.
This is different from the notion of stability in [20], where the output φ ( u, t ) is shown to be stable after perturbation of node attributes φ ( u, 0) with fixed graph edges.
The stability studied in [20] is called Lyapunov stability , which is addressed in Proposition 2.
These two notions of stability combined lead to a graph neural PDE in Section 4 that is robust against both edge and node attacks.
The following result shows the stability of the solution of (11), i.e., ∂φ ( u,t ) ∂t = − ∆ φ ( u, t ) , under graph topology or Laplacian perturbation.
If the change in the graph topology with respect to any matrix norm is small, the semigroup perturbation can be bounded similarly as the result in Theorem 1.
This result considers only the case where the Laplace operator is time-invariant.
The time-variant analogy of [38] as discussed after (5) is provided in the supplementary material.
Proposition 1. Consider ∆ =
D − 1 / 2 ( D − W ) D − 1 / 2 in (9) and suppose ˜∆ = ˜ D − 1 / 2 ( ˜ D − ˜ W )
˜ D − 1 / 2 with ˜ D being the diagonal degree matrix defined analogously to (9) for ˜ W = W + E .
Suppose ε := ~ E ~ = o (1) for a matrix norm ~ · ~ , and D and ˜ D are non-singular.
Then, ∥ φ ( u, t ) − ˜ φ ( u, t ) ∥
=
O ( ε ) .
3.3 Neural Flows on Graphs
Substituting (6) and (7) into (14), and ignoring the degree variable h ( u ) for simplicity, we propose the neural Beltrami flow as
[Equation]
where ∥∇ φ ( u ) ∥
= (cid:113)(cid:80) [ v,u ] ∈E ( ∇ φ ([ u, v ])) 2 .
Here, we further assume φ ( u ) , for all nodes u are time-independent for the sake of simplicity.
Suppose |V| = n .
The function φ : V (cid:55)→ R d maps each node to a feature vector.
Stacking all the feature vectors together, we obtain Z ∈ R n × d .
Let the weight function w ([ u, v ]) be the scaled dot product attention function [41] given by
[Equation]
where W K and W Q are the key and query learnable matrices, respectively, and d K denotes the number of rows W K has.
If multi-head attention is applied and denote A h as the attention matrix associated with head h , then A ( Z )
= 1 h (cid:80) h
A h ( Z ) with A h ( u, v ) =
w h ([ u, v ]) where w h is the weight function for head h .
The diffusion equation (15) can be compactly written in matrix form as
[Equation]
where ⊙ denotes element-wise multiplication, B ( u, v ) = softmax (cid:16)
[Equation]
and Ψ ( Z ( t )) is a diagonal matrix with Ψ( u, u )
=
(cid:80) v ( A ⊙ B ) ( u, v ) .
Similarly, the diffusion equations using mean curvature flow (13) and heat flow (11) can be written as:
[Equation]
(18) where B ( u, v ) = softmax
[Equation]
and ∂ Z (
[Equation]
(19) respectively.
Although the BLEND model in [30] is inspired from Beltrami flow, its final formulation (equation (8) in [30] where Z ( t ) contains positional and node feature embeddings) is indeed (19), i.e., heat flow using attention weight function.
Proposition 2.
Diffusion equations (17) , (18) and (19) are all Lyapunov stable [42].
Model Architecture with Lipschitz Constraint
In this section, we propose a general graph neural PDE network based on the discussion in Section 3.
A layer of graph neural PDE is illustrated in Fig.
Since we may stack up multiple such layers, the diffusion is performed in a hierarchical manner, where the high-dimensional features are diffused over the graph at the front layers and the low-dimensional features are diffused at the back layers.
One important ingredient in our model is that we perform spectral normalization on W K and W Q in (16).
This is motivated by the fact that attention models have poor performance when the depth increases.
In other words, attention weights tend to be uniformly distributed for excessive message exchanges.
This phenomenon becomes even more obvious when the nodes’ features are diffused according to (17), (18) or (19) as solving such a PDE normally requires many discrete steps.
To overcome this over-smoothing problem, we utilize the strategy proposed in [43] to enforce Lipschitz continuity by normalizing the attention scores.
Besides, enforcing Lipschitz continuity can also help improve the robustness of the model because the Lipschitz constant controls the perturbation of the output given a bounded input perturbation.
Fig.
3 illustrates that attention weights become overly smooth if no spectral normalization is applied.
[Figure]
Experiments
In this section, we compare graph neural PDEs under different flows to popular GNN architectures:
GAT [41], GraphSAGE [45], GIN [46], APPNP [47], and the state-of-the-art GNN defenders: RobustGCN [
GNNGuard [23]
GCNSVD [22], on standard node classification benchmarks.
In our ex-
[48] and PubMed (biomedical literature) [49].
We use a refined version of these datasets provided by [50].
We refer the readers to the supplementary material for more details.
Our experiment codes are provided in https://github.com/zknus/Robustness-of-Graph-Neural-Diffusion .
[Table]
5.1 Attack Setup
We apply the setup introduced in the graph robustness benchmark (GRB) [50].
Based on the assumption that nodes with lower degrees are easier to attack, GRB constructs three test subsets of nodes with different degree distributions.
According to the average degrees, GRB defines these subsets of nodes as Easy, Medium, Hard, or Full.
In the Easy subset, attacks are easy to succeed and hence the worst performance is expected.
The remaining nodes are divided into a train set (60%) and val set (10%), for training and validation respectively.
In our experiments, we choose the Easy subset, i
[Table]
e., the most challenging mode.
Following GRB, in this paper, we mainly consider the following real-world adversarial attack settings:
Black
-box : The attacker does not know the defender’s method and vice versa.
Attackers firstly attack a pre-trained GCN [3] and then transfer the perturbed graphs to the target model.
Evasion : Attacks will only happen during the inference phase.
Inductive : GNNs are used to classify unseen data (e.g., new users), i.e., validation or test data are unseen during training.
Injection : Attackers can only inject new nodes but not modify the target nodes directly.
This reflects applications like in online social networks where it is usually hard to hack into users’ accounts and modify their profiles.
However, it is easier to create fake accounts and connect them to existing users.
For other attack settings such as white-box attacks and modification attacks, relevant experiments and discussions are provided in the supplementary material.
[Table]
We apply two state-of-the-art injection attack methods: SPEIT [14] and Topological Defective Graph
Injection Attack (TDGIA) [15].
These two attacks are the two strongest attacks reported in the GRB.
For both attacks, both the maximum allowable injected nodes and the maximum allowable injected edges are set to 50 for Cora and Citeseer, and 300 for PubMed.
We provide more details about how these two attacks work under evasion, black-box, and injection setting in the supplemental material.
Table 1 indicates that graph neural PDE induced from Beltrami flow is more robust than all other
GNNs except for GNNGuard, which is specifically designed to remove malicious edges and is thus robust against injection attacks.
This suggests that the output features from a graph PDE are stable under topology perturbations, as suggested by Proposition 1.
Furthermore, the heat diffusion process on a manifold tends to diffuse differently under different geometries.
For example, it diffuses slower at points with positive curvature, and faster at points with negative curvature [34, 51].
In [52] and [53], the authors show different datasets have different geometric properties like hyperbolicity distribution or Balanced Forman curvature.
Our theoretical analysis only shows a loose uniform bound in terms of the adjacency matrix, while the performance is very much dataset-dependent.
More advanced theoretical analysis for different datasets is highly non-trivial and needs further investigations.
Our graph neural PDE can be combined with GNNGuard.
We denote this model as BeltramiGuard.
Table 2 demonstrates that BeltramiGuard renders attacks in vain and exceeds GNNGuard on the
Citeseer and PubMed datasets.
Note from Table 1 that the vanilla Beltrami model already surpasses
GNNGuard on the large PubMed dataset.
More details are provided in the supplementary material.
5.2 Ablation Studies Diffusion Schemes
We compare different diffusion equations: heat flow (19) where w ([ u, v ])
= 1 for all [ u, v ] ∈ E ,
GRAND/BLEND [29, 30] which is another heat flow where w ([ u, v ]) in (19) is the attention function defined in (16), mean curvature flow (18), and Beltrami flow (17).
For GRAND/BLEND, we stack three neural PDE layers using the architecture proposed in Fig.
2 since the original GRAND/BLEND in [29, 30], which has only one PDE layer, does not perform well.
[Table]
From Table
[Table]
we observe that even the vanilla time-invariant heat flow preserves some robustness as compared to non-PDE GNNs in Table 1.
This further validates our theoretical analysis in Proposition
[Table]
which suggests that if the topology perturbation is bounded, the learned representations are close to those under the “clean” scenario.
We can observe that 1) the flows which are capable of preserving non-smooth features are generally more robust than heat flow, 2) the proposed mean curvature flow and Beltrami flow are more robust than GRAND/BLEND, and 3) these two flows generally suffer less performance variance than the other methods.
More Experiments
Due to space constraint, we refer the reader to the supplementary material for more ablation studies including the impact of the Lipschitz constraint, PDE solvers, the number of layers, and time complexity.
The model performance under other attacks is also presented in the supplementary material.
Conclusion
In this paper we have introduced a general graph neural PDE framework from which several graph neural PDEs are proposed.
We analyzed the robustness of the graph neural PDEs and showed that graph neural PDEs are inherently robust against topology perturbations and are also Lyapunov stable.
We provided theoretical evidence showing that the robustness of graph neural PDEs stems from the stability of the heat kernel and semigroup during the diffusion process.
Moreover, we conducted extensive experiments that empirically verify the robustness of the proposed graph neural PDEs when compared with an existing graph neural PDE induced from approximated heat flow, popular GNNs and the state-of-the-art GNN defenders.
Acknowledgement
This research is supported by the Singapore Ministry of Education Academic Research Fund Tier 2 grant MOE-T2EP20220-0002 and A*STAR under its RIE2020 Advanced Manufacturing and
Engineering (AME) Industry Alignment Fund – Pre Positioning (IAF-PP) (Grant No. A19D6a0053)