EvenNet: Ignoring Odd-Hop Neighbors Improves Robustness of Graph Neural Networks
Abstract
Graph Neural Networks (GNNs) have received extensive research attention for their promising performance in graph machine learning.
Despite their extraordinary predictive accuracy, existing approaches, such as GCN and GPRGNN, are not robust in the face of homophily changes on test graphs, rendering these models vulnerable to graph structural attacks and with limited capacity in generalizing to graphs of varied homophily levels.
Although many methods have been proposed to improve the robustness of GNN models, the majority of these techniques are restricted to the spatial domain and employ complicated defense mechanisms, such as learning new graph structures or calculating edge attentions.
In this paper, we study the problem of designing simple and robust GNN models in the spectral domain.
We propose EvenNet, a spectral GNN corresponding to an even-polynomial graph filter.
Based on our theoretical analysis in both spatial and spectral domains, we demonstrate that EvenNet outperforms full-order models in generalizing across homophilic and heterophilic graphs, implying that ignoring odd-hop neighbors improves the robustness of GNNs.
We conduct experiments on both synthetic and real-world datasets to demonstrate the effectiveness of EvenNet.
Notably, EvenNet outperforms existing defense models against structural attacks without introducing additional computational costs and maintains competitiveness in traditional node classification tasks on homophilic and heterophilic graphs.
Our code is available in https://github.com/Leirunlin/EvenNet .
Introduction
Graph Neural Networks (GNNs) have gained widespread interest for their excellent performance in graph representation learning tasks [11, 17, 22, 32, 34].
GCN is known to be equivalent to a low-pass filter [2, 27], which leverages the homophily assumption that “connected nodes are more likely to have the same label” as the inductive bias.
Such assumptions fail in heterophilic settings [44], where connected nodes tend to have different labels, encouraging research into heterophilic GNNs [1, 28, 44].
Among them, spectral GNNs with learnable polynomial filters [3, 10, 18] adaptively learn suitable graph filters from training graphs and achieve promising performance on both homophilic and heterophilic graphs.
If the training graph is heterophilic, a high-pass or composite-shaped graph filter is empirically obtained.
While GNNs are powerful in graph representation learning, recent studies suggest that they are vulnerable to adversarial attacks, where graph structures are perturbed by inserting and removing edges on victim graphs to lower the predictive accuracy of GNNs [45, 37].
Zhu et al.
[43] first established the relationship between graph homophily and structural attacks.
They claimed that existing attack mechanisms tend to introduce heterophily to homophilic graphs, which significantly degrade the performance of GNNs with low-pass filters.
On the one hand, several attempts are made to improve the robustness of GNNs against the injected heterophily from the spatial domain [13, 20, 35, 40, 42].
These methods either compute edge attention or learn new graph structures with node features, requiring high computational costs in the spatial domain.
On the other hand, while spectral GNNs hold superiority on heterophilic graphs, their performance under structural perturbation is unsatisfactory as well, which arouses our interest in exploring the robustness of current spectral methods.
In this study, we consider homophily-heterophily inductive learning tasks, which naturally model non-targeted structural attacks.
We observe that structure attacks enlarge the homophily gap between training and test graphs besides introducing heterophily, challenging spectral GNNs to generalize across different homophily levels.
Consequently, despite their outstanding performance on heterophilic graphs, spectral GNNs such as
GPRGNN have poor generalization ability when the training and test graphs have different homophily.
For example, suppose we now have two friend-enemy networks like the ones in
Figure 1.
If friends are more likely to become neighbors, representing the relationship “like”, the network is homophilic.
If enemies form more links corresponding to the relationship “hate”, the network becomes heterophilic.
If we apply spectral GNNs trained on “like” networks (where a low-pass filter is obtained) to “hate” networks, we will mistake enemies for friends on “hate” networks.
Despite the strength of spectral GNNs in approximating
optimal
graph filters
of
arbitrary
shapes, the lack
of constraints on learned filters
makes it difficult
for them to generalize.
To improve the performance of current spectral methods against non-targeted structural adversarial attacks, we design a novel spectral GNN that realizes generalization across homophily.
Our contributions are:
• We proposed EvenNet, a simple yet effective spectral GNN that can be generalized to graphs of different homophily.
EvenNet discards messages from odd-order neighbors inspired by balance theory, deriving a graph filter with only even-order terms.
We provide a detailed theoretical analysis in the spatial domain to illustrate the advantages of EvenNet in generalizing to graphs of different homophily.
• We propose Spectral Regression Loss (SRL) to evaluate the performance of graph filters on specific graphs in the spectral domain.
We theoretically analyze the relationship between graph filters and graph homophily, confirming that EvenNet with symmetric constraints is more robust in homophily-heterophily inductive learning tasks.
• We conduct comprehensive experiments on both synthetic and real-world datasets.
The empirical results validate the superiority of EvenNet in generalizing to test graphs of different homophily without introducing additional computational complexity while remaining competitive in traditional node classification tasks.
Preliminaries
Notations.
Let G =
( V , E ) denote an undirected graph, where N = |V| is the number of nodes.
Let A ∈ { 0 , 1 } N × N denote the adjacency matrix.
Concretely, A ij = 1 indicates an edge between nodes v i and v j .
Graph Laplacian is defined as L = D − A , along with a normalized version ˜ L
= I − D − 1 / 2 AD − 1 / 2 , where I is the identity matrix and D is a diagonal degree matrix with each diagonal element D ii = (cid:80)
Ni =
1 A ij .
It is known that ˜ L is a symmetric positive semidefinite matrix that can be decomposed as ˜ L = U Λ U T , where Λ = diag { λ 0 , . . .
, λ N − 1 } is a diagonal eigenvalue matrix with
= λ 0 ≤ λ 1 ≤ . . .
≤ λ N − 1 ≤ 2 , and U is a unitary matrix consisting of eigenvectors.
For multi-class node classification tasks, nodes in G are divided into K classes {C 0 , . .
, C K − 1 } .
Each node v i is attached with an F dimension feature and a one-hot class label.
Let X ∈ R N × F be the input feature matrix and Y ∈ R N × K
= ( y 0 , . .
, y K − 1 ) be the label matrix, where y i is the indicator vector of class C i .
Let R =
Y ⊤ Y and the size of class C
k be R k .
Graph filtering.
The graph filtering operation on graph signal X is defined as Z = σ ( Ug (Λ) U T X ) , where g (Λ) is the so-called graph filter, and σ is the normalization function.
Directly learning g (Λ) requires eigendecomposition (EVD) of time complexity O ( N 3 ) .
Recent studies suggest using polynomials to approximate g (Λ) instead, which is:
[Equation]
where { w k } are polynomial coefficients.
We can also denote a K -order polynomial graph filter as a filter function g ( λ ) =
(cid:80) Kk =
w k λ k that maps eigenvalue λ ∈ [0 , 2] to g ( λ ) .
Homophily.
Homophily reflects nodes’ preferences for choosing neighbors.
For a graph of strong homophily, nodes show a tendency to form connections with nodes of the same labels.
The ratio of homophily h measures the level of overall homophily in a graph.
Several homophily metrics have been proposed with different focuses [26, 28].
We adopt edge homophily following [44], defined by
[Equation]
By definition, h ∈ [0 , 1] is the fraction of intra-class edges in the graph.
The closer h is to 1 , the more homophilic a graph is.
Our Proposed Method: EvenNet
In this section, we first introduce our motivation and the methodology of EvenNet.
We then explain how EvenNet enhances the robustness of spectral GNNs from the perspective of both spatial and spectral domains.
3.1 Motivations
Reconsider the toy example in Figure 1.
Relationships between nodes are opposite on homophilic and heterophilic graphs, being straightforward but erratic under changes in the graph structure.
Unconstrained spectral GNNs tend to overuse such unstable relationships and fail to generalize across homophily.
In contrast, a robust model should rely on more general topological information beyond homophily.
Balance theory [6], which arose from signed networks, offers a good perspective: “The enemy of my enemy is my friend, and the friend of my friend is also my friend.”
Balance theory always holds as a more general law, regardless of how structural information is revealed on the graph.
As a result, we can obtain a more robust spectral GNN under homophily change by incorporating balance theory into graph filter design.
3.2 EvenNet
Denote propagation matrix as P =
I − ˜ L
= D − 12 AD − 12 .
A K -order polynomial graph filer is defined as g (˜ L ) =
(cid:80) Kk =0 w k ˜
L k , where w k , k = 0 , . .
, K are learnable parameters.
We can rewrite the filter as g (˜ L ) =
(cid:80) Kk =0 w k
( I − ˜ L ) k
= (cid:80)
Kk =0 w k P
k
since w k is learnable.
Then, we discard the monomials in g (˜ L ) containing odd-order P , obtaining:
[Equation]
In practice, we decouple the transformation of input features and graph filtering process following [10, 23].
Our model then takes the simple form:
[Equation]
where t is an input transformation function (e.g. MLP), and Z is the output node representation that can be fed into a softmax activation function for node classification tasks.
From the perspective of the spectral domain, g even keeps both low and high frequencies components and suppresses medium-frequency components, which is a band-reject filter with the filter function symmetric about λ = 1 .
We provide a theoretical analysis to demonstrate further the advantages of g even in Section 3.3 and 3.4.
3.3 Analysis from the Spatial Domain
Recently, Chen et al.
[9] analyzed the performance of graph filters under certain homophily.
They concluded that graph filters operate as a potential reconstruction mechanism of the graph structure.
A graph filter g ( ˜ L ) achieves better performance in a binary node classification task when the homophily of the transformed graph is high .
The transformed homophily can therefore be seen as an indicator of the performance of graph filters on specific tasks.
We now provide the well-defined transformed homophily adopted from [9].
Definition 1.
( k -step interaction probability)
For propagation matrix P = D − 12 AD − 12 , the k -step interaction probability matrix is
[Equation]
Definition 2.
( k -homophily degree)
For a graph G with the k -step interaction probability ˜Π k , its k -homophily degree H k (˜Π) is defined as
[Equation]
The transformed 1-homophily degree with filter g (˜ L ) is H 1 ( g ( I − ˜Π)) .
By definition, the k -homophily degree reflects the average possibility of deriving a node’s label from its k -hop neighbors.
In Theorem 1, we show that even-order filters achieve more robust performance under homophily change by enjoying a lower variance of transformed homophily degree without losing average performance.
The detailed proof is provided in Appendix A.1, including discussions about multi-class cases.
Theorem 1.
In a binary node classification task, assume the edge homophily h ∈ [0 , 1] is a random variable that belongs to a uniform distribution.
An even-order graph filter achieves no less
E H (cid:104) H 1 (cid:16) g
( I − ˜Π) (cid:17)(cid:105) with lower variation than the full-order version.
3.4 Analysis from Spectral Domain
Similar to Section 3.3, we proposed Spectral Regression Loss (SRL) as an evaluation metric of graph filters in the spectral domain.
In a binary node classification task, suppose the dimension of inputs
F = 1 .
Denote the difference of labels as ∆ y =
y 0 − y 1 .
A graph filtering operation is defined as
Z = σ ( Ug (Λ) U T X ) .
Desirable filtering produces distinguishable node representations correlated to ∆ y to identify node labels.
Let α =
U ⊤ ∆ y and β = U ⊤ X .
The classification task in the spectral domain is then a regression problem in the form of σ ( α ) =
σ ( g (Λ) β ) .
We adopt Mean Squared Error (MSE) as the objective function of the regression problem and vector normalization as σ .
Then SRL is defined as follows:
Definition 3.
(Spectral regression loss.)
Denote α
=
( α 0 , . .
, α N − 1 ) ⊤ , β = ( β 0 , . . .
, β N − 1 ) ⊤ .
In a binary node classification task, Spectral Regression Loss (SRL) of filter g (Λ) on graph G is:
[Equation]
The constant √ N comes from the fact (cid:80) N − 1 i =
α 2 i
= N .
A detailed illustration is included in
Appendix A.2.
A graph filter that achieves lower SRL is of higher performance in the task.
Filters that Minimize SRL.
Suppose α i
=
wβ i
+ ϵ , where w > 0 reflects the correlation between labels and features in the spectral domain and ϵ be the noise term.
If ϵ is close to 0, indicating features are free of noise and highly predictive, an all-pass filter (for example, MLP) with g ( λ i ) =already minimizes SRL.
If the noise becomes dominant, SRL approximately equals to (cid:80) N − 1 i =
( α i √ N − g
( λ i
[Equation]
√
[Equation]
j g ( λ j ) 2 ) 2 .
In this noise-dominant case, an ideal filter is linearly correlated to α and structurebased to achieve a lower SRL.
Most real-world situations lie between these two opposite settings.
As a result, the shape of an ideal graph filter lies between an all-pass filter and an α -dependent filter.
From the discussion above, we have shown that the performance of graph filters is related to the correlation between g ( λ ) and α .
By connecting α and h in Theorem 2, we establish the relationship between graph homophily and the performance of graph filters.
Theorem 2.
For a binary node classification task on a k -regular graph G , let h be edge homophily
and λ i be the i -th smallest eigenvalue of ˜ L , then
[Equation]
The above equation can be extended to general graphs by replacing the normalized Laplacian ˜ L with the unnormalized L .
Notice that (cid:80) N − 1 i =
α
i
λ
i is a convex combination of non-decreasing { λ i } with weights
α 2 i .
On a homophilic graph where h is close to 1, the right-hand side of Equation 6 is close to 0, implying larger weights for smaller λ i .
A low-pass filter that suppresses high-frequency components is more correlated with such α and therefore achieves lower SRL.
From previous works, we have known that low-pass filters hold superiority on homophilic graphs, which is consistent with our analysis.
In the case of generalization, the distribution of { α i } is not fixed.
A graph filter that minimizes
the
SRL on training graphs could achieve poor results on a test graph of different homophily.
Remember that vanilla GCN could be worse than MLP on many heterophilic graphs.
The same conclusion can be applied to learnable filters without any constraints, as they only tried to minimize the SRL of training graphs.
In Theorem 3, we prove that even-order design helps spectral GNNs better generalize between homophilic and heterophilic graphs as a practical constraint to current filters.
Theorem 3.
Suppose λ N − 1 = 2 for a homophilic graph G 1 with non-increasing { α i } , and a heterophilic graph G 2 with non-decreasing { α i } .
Then an even-order filter g even achieves a lower
SRL gap
| L ( G 1 ) − L ( G 2 ) | than full-order filters when trained on one of the graphs and test on the other.
A discussion about the case where λ N − 1 < 2 is given in the Appendix C. Theorem 3 reveals a trade-off in filter design between fitting the training graph and generalizing across graphs of different homophily.
While naive low-pass filters and high-pass filters work better on graphs with certain homophily, EvenNet tolerates imperfect filter learning and becomes more robust under homophily changes.
A specific example on ring graphs is given in the following corollary.
We see that EvenNet intrinsically satisfies the necessary condition for perfect generalization.
Corollary 1.
Consider two ring graphs G 1 and G 2 of 2 n nodes, n ∈ N + .
Suppose h ( G 1 ) =
and h ( G 2 ) = 1 .
Assume the spectrum of input difference β
= c 1 , where c > 0 is a constant.
Then the necessary condition for a graph filter g ( λ ) to achieve L ( G 1 ) =
L ( G 2 ) is g (0) =
g (2) .
3.5 Complexity
Denote N the number of nodes, d the size of hidden channels (we assume it is of the same order as the size of input features), | E | the number of edges, L the number of MLP layers used in feature transformation and K the order of the propagation layer.
Compared with structural learning methods which usually have a space complexity of O ( N 2 ) ,
EvenNet takes up O ( | E | ) space complexity, as it only needs to store the input sparse adjacency matrix during training.
For the time complexity of EvenNet, the transform process has a time complexity of
O ( Nd 2 L ) , and the propagation process has a complexity
O ( Kd | E | ) during each forward pass.
In practice, H2GCN [44] and ProGNN [20] require O ( N 2 ) space complexity and are thus not scalable to large graphs.
GCNII [7] achieves its best performance with multiple stacked layers which is slow to train.
FAGCN [4], GAT [32] and GNNGuard [40] with attention calculations are also inefficient during training.
Notice that the space and time complexity of EvenNet are both linear to N and | E | , which is highly efficient.
We report the computational time and an experiment on a larger dataset ogbn-arxiv [19] in E to further verify the efficiency of EvenNet.
Related Work
Spectral GNNs.
GNNs have become prevalent in graph representation learning tasks.
Among them,
Spectral GNNs focus on designing graph filters with filter functions that operate on eigenvalues of graph Laplacian [5].
Graph filters could be fixed [22, 23, 34] or approximated with polynomials.
ChebNet
[11] adopts Chebyshev polynomials to realize faster localized spectral convolution.
ARMA [3] achieves a more flexible filter approximation with Auto-Regressive Moving Average filters.
GPRGNN [10] connects graph filtering with graph diffusion and learns coefficients of polynomial filters directly.
BernNet [18] utilizes Bernstein approximation to learn arbitrary filtering functions.
Although learnable graph filters perform well on heterophilic graphs, they have difficulties generalizing if a homophily gap exists between training and test graphs.
GNNs for Heterophily.
Previous works pointed out the weakness of vanilla GCN on graphs with heterophily.
Recently, various GNNs have been proposed to tackle this problem.
Geom-GCN [28] uses a novel neighborhood aggregation scheme to capture long-distance information.
Zhu et al.
[44] introduces several designs that are helpful for GNNs to learn representations beyond homophily.
FAGCN [4] adaptively combines signals of different frequencies in message passing via a selfgating mechanism.
While these methods can handle heterophilic graphs, they are not guaranteed to generalize across graphs of different homophily.
Robust GNNs.
In the field of designing robust GNNs, existing methods can be divided into two main categories: 1) Models utilizing new graph structures.
GNN-Jaccard [35] and GNNSVD [13] preprocess the input graph before applying vanilla GCN.
ProGNN [20] jointly learns a better graph structure and a robust model.
2) Attention-based models.
RGCN [42] uses variancebased attention to evaluate the credibility of nodes’ neighbors.
GNNGuard [40] adopts neighbor importance estimation, aligning higher scores to trustworthy neighbors.
TWIRLS [38] applies an attention mechanism inspired by classical iterative methods PGD and IRLS.
These methods are effective against structural attacks.
However, the learned graph structure cannot be applied to inductive learning settings and requires additional memory.
At the same time, attention-based models are limited in the spatial domain and need high computational costs.
On the contrary, EvenNet improves the robustness of spectral GNNs without introducing additional computational costs.
Experiment
We conduct three experiments to test the ability of EvenNet in (1) generalizing across homophily on synthetic datasets, (2) defending against non-targeted structural attacks, and (3) supervised node classification on real-world datasets.
5.1 Baselines
We compare our EvenNet with the following methods.
(1) Method only using node features: A 2-layer MLP.
(2) Methods achieving promising results on homophilic graphs: GCN [22], GAT [32],
GCNII [7].
(3) Methods handling heterophilic settings: H2GCN [44], FAGCN [4], GPRGNN [10].
We also include five advanced defense models in the experiment about adversarial attacks, including
RobustGCN [42], GNN-SVD [13], GNN-Jaccard [35], GNNGuard [40], and ProGNN [20].
We implement the above models with the help of PyTorch Geometric [14] and DeepRobust libraries [25].
Details about hyperparameters and network architectures are deferred to Appendix C.
5.2 Evaluation on synthetic datasets
Datasets.
In the first experiment testing generalization ability, we use cSBM model to generate graphs with arbitrary homophily levels following [10].
Specifically, we divide nodes into two classes of equal size.
Each node is attached with a feature vector randomly sampled from a class-specific
Gaussian distribution.
The homophily level of a graph is controlled by parameter ϕ ∈ [ − 1 , 1] .
A larger | ϕ | indicates that the generated graph provides stronger topological information, while ϕ =means only node features are helpful for prediction.
Note that if ϕ > 0 , the graph is more homophilic and vice versa.
Details about cSBM dataset are included in Appendix B.1.
Settings.
We set up node classification tasks in the inductive setting.
We generate three graphs of the same size for each sub-experiment, one graph each for training, validation, and testing.
Graphs for validation and testing share the same ϕ test , while training graphs either take ϕ train =
ϕ test or ϕ train = − ϕ test .
If ϕ train = − ϕ test , the training and test graphs are of opposite homophily but provide the same amount of topological information.
A model manages to generalize across homophily when it realizes high prediction accuracy in both scenarios.
In practice, we choose ( ϕ train , ϕ test ) ∈ { ( ± 0 .
5 , ± 0 .
5) , ( ± 0 .
75 , ± 0 .
Results.
The results are presented in Table 1.
When ϕ train = ϕ test , GPRGNN achieves the highest predictive accuracy as it best fits the desired graph filter.
However, when ϕ train = − ϕ test , all methods except EvenNet suffer from a huge performance drop.
Vanilla GCN, which corresponds to a low-pass filter, achieves desirable performance only when the test graph is homophilic.
GPRGNN overfits training graphs most, resulting in more severe performance degradation on test graphs of opposite homophily.
EvenNet is the only method that achieves more than 75% accuracy on all datasets among all the models, which is robust in generalization across homophily
[Table] [Table]
5.3 Performance under non-targeted structural adversarial attacks
Datasets.
For adversarial attacks, we use four public graphs, Cora, Citeseer, PubMed [30, 39] and
ACM [36] available in DeepRobust Library [25].
We use the same preprocessing method and splits as [45], where the node set is split into 10% for training, 10% for validation, and 80% for testing, and the largest connected component of each graph for attacks are selected.
We include the experiment against non-targeted attacks on heterophilic datasets in Appendix G, in which we use the same preprocessing methods and dense splits following [10]
     3HUWXUE5DWLR  
        + R P RSK LOO \  O H Y H O ',&( 7UDLQ 7HVW      3HUWXUE5DWLR     
  0HWD
7UDLQ 7HVW      3HUWXUE5DWLR     
  0LQ0D[ 7UDLQ 7HVW
Attack methods.
Graph structural attacks can be categorized into poison attacks and evasion attacks.
In poison attacks, attack models are trained to lower the performance of a surrogate GNN model.
The training graph and the test graph are both allowed to be perturbed but only with a limited amount of modifications, which are referred to as perturb ratios.
Evasion attacks only happen during inference, meaning GNNs are trained on clean graphs.
In our study, we include two poison attacks, Metattack (Meta) [45] and MinMax attack [37] with GCN the surrogate model, and an evasion variant of DICE attack [33].
Notice that we mainly focus on modification attacks, which are strictly structural attacks.
A discussion of GNNs under graph injection attacks is included in Appendix F.
For poison attacks, we use the same setting in [40] and set the perturb ratio for poison attacks to be 20%.
For the evasion DICE, we randomly remove intra-class edges and add inter-class edges on the test graph while keeping the graph structure between labeled nodes unchanged.
We set the perturb ratio of DICE attack in { 0 . 4 , 0 . 8 , 1 . 2 , 1 . 6 } .
From Figure 2, it can be seen that all attacks result in homophily gaps between the training graphs and the test graphs.
Besides the 1-hop homophily gap, in
Table 2, we present the change in two-hop homophily for learnable attacks with perturb ratio of 0.2.
As can be seen, the two-hop homophily gap is relatively smaller than the one-hop homophily gap, which is in accordance with our analysis that homophily between even-hop neighbors is more robust.
[Table]
Results.
Defense results are presented in Table 3 and Figure 3.
For the DICE attack, the performance of all methods significantly decreases along with the increase of the homophily gap except EvenNet.
Interestingly, when the homophily gap is enormous, EvenNet enjoys a performance rebound, consistent with our topological information theory (strong homo.
and strong hetero.
are both helpful for prediction).
For poison attacks, EvenNet achieves SOTA compared with advanced defense models.
Unlike spatial defense models, EvenNet is free of introducing extra time or space complexity.
5.4 Performance on real-world graph datasets
We evaluate EvenNet on real-world datasets to examine the performance of EvenNet on clean graphs.
Besides of the datasets used in Section 5.3, we additionally include four public heterophilic datasets:
Actor, Cornell, Squirrel and Texas [28, 29, 31].
The statistics of real-world Datasets are included in Table 4.
In the node classification task, we transform heterophilic datasets into undirected ones following [10].
[Table]
              $ FF X U D
F\ &RUD             
&LWHVHHU          3HUWXUE5DWLR    
  $ FF X U D F\ $&          3HUWXUE5DWLR    3XE0HG 0/3*&1*$7*&1B69'*&1B-DFFDUG5*&1)$*&1+*&1*&1B*XDUG*&1,,*35*11(YHQ1HW
For all datasets, we adopt dense splits the same as [28] to perform full-supervised node classification tasks, where the node set is split into 60% for training, 20% for validation, and 20% for testing.
[Table]
The results are shown in Table 5.
While EvenNet sacrifices its performance for robustness, it is still competitive on most datasets.
5.5 Ablation study
To analyze the effect of introducing odd-order components into graph filters, we develop a regularized variant of EvenNet named EvenReg.
EvenReg adopts a full-order learnable graph filter, with the coefficients of odd-order monomials being punished as a regularization term.
The training loss of
EvenReg then takes the form:
L =
L pred + η (cid:80) ⌊ K/ 2 ⌋ k =
|
w 2 k +1 | , where L pred is the classification loss and η is a hyper-parameter controlling the degree of regularization.
[Table]
We set η = 0 .
and repeat experiments in Section 5.2.
The results are presented in Table 6.
The performance of EvenReg lies between full-order GPRGNN and EvenNet, indicating the introduced odd orders impede spectral GNNs to generalize across homophily
[Table]
[Table]
Conclusion
In this study, we investigate the ability of current GNNs to generalize across homophily.
We observe that all existing methods experience severe performance degradation if a large homophily gap exists between training and test graphs.
To overcome this difficulty, we proposed EvenNet, a simple yet effective spectral GNN which is robust under homophily change of graphs.
We provide a detailed theoretical analysis to illustrate the advantages of EvenNet in generalization between graphs with homophily gaps.
We conduct experiments on both synthetic and real-world datasets.
The empirical results verify the superiority of EvenNet in inductive learning across homophily and defense under non-targeted structural attacks by sacrificing only a tiny amount of predictive accuracy on clean graphs.
Acknowledgement
This research was supported in part by National Natural Science Foundation of China (No. 61972401,
No. 61932001, No. 61832017), by Beijing Natural Science Foundation (No. 4222028), by the major key project of PCL (PCL2021A12), by Beijing Outstanding Young Scientist Program No.
BJJWZYJH012019100020098, by Alibaba Group through Alibaba Innovative Research Program, by
CCF-Baidu Open Fund (NO.2021PP15002000) and by Huawei-Renmin University joint program on
Information Retrieval.
We also wish to acknowledge the support provided by Intelligent Social Governance Interdisciplinary Platform, Major Innovation & Planning Interdisciplinary Platform for the “Double-First Class” Initiative, Public Policy and Decision-making Research Lab, Public Computing
Cloud, Engineering Research Center of Next-Generation Intelligent Search and Recommendation,
Ministry of Education, Renmin University of China. Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] In Section 5.4 and in Section(c) Did you discuss any potential negative societal impacts of your work?
[No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] Included in Appendix due to space limitation.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[No] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] Splits are specified in the experiment chapter, and hyperparameters are specified in supplementary materials.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] We only report mean accuracy.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] Included in supplementary materials.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] We cite the creators of used datasets.
(b) Did you mention the license of the assets?
[No] (c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[Yes] We mention that all used datasets are public in our experiments.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[Yes] We mention that all used datasets are public in our experiments without personally identifiable information or offensive content.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Additional proofs A.1 Proof of Theorem
Before proofing Theorem 1, We first demonstrate the superiority of even-hop neighbors over odd-hop neighbors from the perspective of random walks.
In a binary node classification task, denote the probability of a random walk of length k that starts and ends with nodes of the same label as p k , k > 0 .
Suppose the edge homophily level h is a random variable that belongs to a uniform distribution in [0 , 1] and p 1 = h , then:
Lemma 1.
If k is odd, E h [ p k ] = 12 .
If k is even, E h [ p k ] ≥ 12 .
Proof.
If k = 1 , p 1 = h, E h
[ p 1 ] =
E h
[ h ] = 12 .
If k = 2 , p 2 =
h 2 + (1 − h )
= 2( h − 12 )
+ 12 ≥ 12 ,
E h
[ p 2 ] = (cid:82) 1 0 p 2 dh ≥ 12 .
For k > 2 ,
[Equation]
That is, for E h
[ p k − 2 ] = 12 , E h
[ p k ] = 12 ; for E
h
[ p k − 2 ] ≥ 12 , E h
[ p k ] ≥ 12 .
Therefore, Lemma 1 is proved.
Multi-class Cases.
We now provide a brief discussion of the superiority of even-hop neighbors in multi-class node classification tasks following [44].
Definition 4.
The matrix Q ∈ R K × K is an independent between-class random walk matrix if it holds the following properties:
• Q is a random walk matrix.
• ∀ i ̸
= j , Q ii = Q jj .
• ∀ i ̸
= j, m ̸ = n, Q ij = Q mn .
Suppose there are K classes of nodes in the graph, where the number of nodes of each class is the same, and node labels are assigned independently.
Denote h as the edge homophily level, the 1-step between-class random walk matrix P is in the form of:
[Equation]
where P ij denotes the probability of a 1-step random walk that start with a node of label i and ends with a node of label j .
By definition, P is an independent between-class random walk matrix.
Lemma 2.
If M ∈ R K × K is an independent between-class random walk matrix, P ′ =
MP is an independent between-class random walk matrix as well.
Proof.
It can be verified that P ′ is still a random walk matrix, and for all i ̸
=
j, m ̸
= n :
[Equation]
By definition, P ′ is an independent between-class random walk matrix.
Denote the k -step between-class random walk matrix as P k .
From Lemma 2, we can conclude that
P k is an independent between-class random walk matrix for all k ∈ N + .
In Lemma 3, we illustrate the advantages of even-order propagation by comparing the interaction probability between classes.
Lemma 3.
If k is even, the intra-class interaction probability P kii is no less than inter-class interaction probability P kij , i ̸
= j .
Proof.
For k = 2 :
[Equation]
The inequality is tight when h
= 1 − h K − 1 .
For k = 2 m, m > 1 , m ∈ N + , P m is an independent between-class random walk matrix that can be written as:
[Equation]
The above proof for k = 2 can be generalized to all h ∈ [0 , 1] .
Therefore, P mii ≥ P mij is satisfied as well.
Note that Lemma 3 is not satisfied for odd k if h is relatively smaller than 1 − h K − 1 .
Proof of Theorem
Proof.
According to Definition 1, for a graph G with the k -step interaction probability ˜Π k , its k -homophily degree H k (˜Π) is defined as
[Equation]
The transformed 1-homophily degree with filter g (˜ L ) is H 1 ( g ( I − ˜Π)) .
Specifically, in a binary node classification problem, the k -homophily degree is:
[Equation]
Denote a k -order polynomial graph filter as g k (˜ L ) =
w 0 + w 1 ˜ L + . . .
+ w k ˜ L k ,
Let ˜ Pi
The transformed 1-homophily degree of g k (˜ L ) is:
[Equation]
where c is a constant and each θ i is a linear sum of { w i } that for arbitrary x ∈ R :
[Equation]
Following Lemma 1, the average possibility of deriving a node’s label from its odd-hop neighbors is 12 , which means E h
[ H i (˜Π)]
=
and Var h
[ H i (˜Π)] ≥ 0 for odd i .
By removing the odd-order terms, the transformed 1-homophily degree does not decrease on average but enjoys a lower variation.
Rewrite filter g k (˜ L ) as g k (˜ L )
=
θ
+ θ 1 ( I − ˜ L )
+ . . .
+ θ
k
( I − ˜ L ) k and set θ i = 0 for odd i , the graph filter is then in the form of :
[Equation]
which is exactly the graph filter of EvenNet.
Therefore, EvenNet has a lower variation on the transformed 1-homophily degree without sacrificing the average performance.
A.2 Proof of Theorem
Lemma 4.
Given normalized graph Laplacian ˜ L = U Λ U ⊤ and label difference as ∆ y =
y 0 − y 1 , unnormalized α = U
⊤ ∆ y
=
( α 0 , · · · , α N − 1 )
⊤ satisfies (cid:80) N − 1 i =
α 2 i
= N .
Proof.
Since ˜ L is a real symmetric matrix, U T can be chosen to be an orthogonal matrix.
Denote the element of the i -th row and j -th column of U T as u ik .
[Equation]
i
[Equation]
N
[Equation]
N
[Equation]
k
[Equation]
(cid:32)
[Equation]
( k )
N
[Equation]
i
[Equation]
u
[Equation]
u
[Equation]
{ j ̸
[Equation]
k } (cid:33) =
N
[Equation]
(cid:88) i =
N − 1 (cid:88)
j = u 2 ij (Orthogonality) =
N
Lemma 4 provides the relationship between normalized and unnormalized α , which is also helpful in defining the SRL loss.
Lemma 5.
Given unnormalized graph Laplacian L = U L Λ L U ⊤ L and its eigenvalues { λ ′ i } , denote the number of edges on the graph is m , and label difference as ∆ y =
y 0 − y 1 ∈ R N × 1 .
For unnormalized spectrum of label difference on L is α ′ =
U ⊤ L ∆ y
=
( α ′ 0 , α ′ 1 , . . .
, α ′ N − 1 ) ⊤ , then
[Equation]
[Equation]
Proof.
Denote the trace of a matrix M as tr ( M ) , the degree of node v i as d i .
[Equation] The Dirichlet energy of the label difference is defined as:
[Equation]
Using L = U L Λ L U TL , the Dirichlet energy of the label difference can also be expressed as:
[Equation]
By integrating equations 7 8 9, we get:
[Equation]
Proof of Theorem
Proof.
For a k -regular graph, denote normalized graph Laplacian as ˜ L =
U Λ U ⊤ , then
[Equation]
From equation 10, we get U =
U L , λ i = λ
′ i /k .
Denote the spectrum of label difference on ˜ L as α , then α = α ′ .
By substituting λ ′ i with kλ i in Lemma 5, we acquire the equation in Theorem 2.
If α is normalized as in the SRL that satisfies (cid:80) N − 1 i =
α 2 i = 1 , following Lemma 4, Theorem 2 is in the form of:
[Equation]
A.3 Proof of Theorem
Proof.
Denote filter g ( λ ) as g ( λ ) =
(cid:80) Ki =
w i (1 − λ ) i , g even ( λ ) = 12 ( g ( λ ) + g (2 − λ )) and g odd = g ( λ ) − g even ( λ ) .
The filter g even ( λ ) is free of odd order terms.
The odd filter g odd ( λ ) can be seen as the gap between the full-order filter and the even-order filter.
[Equation]
We now consider the SRL gap of the odd-order filter to illustrate the effect of removing oddorder terms.
The regression problem in the spectral domain with normalized α : α = σ ( g (Λ) β ) , (cid:80) N −i =
α 2 i = 1 .
Suppose α and β is positive correlated in the form of E [ α ] = w β , w > 0 , and λ N − 1 = 2 for both G 1 and G 2 (to ensure both graphs can achieve h = 0 ).
The SRL of filter g odd and g even between normalized α and normalized g ( λ ) β is:
[Equation]
where T type = (cid:113)(cid:80) N − 1 i =
g type ( λ i )
α 2 i .
Compare equations 11 and 12.
If g ( λ i ) is of different monotonicity against { α i } , which happens when a trained odd-filter is generalized to graphs of opposite homophily, L o becomes negative.
In contrast, L e is always positive and benefits from reducing SRL.
Suppose L o is the approximate SRL gap between g ( λ ) and g even ( λ ) .
The instability of L o implies
L g ( G train ) <
L g even ( G train ) and L g ( G test ) >
L g even ( G test ) , reflecting a larger SRL gap of fullorder filters than the even-order filters.
More generally, for the cases where λ N − 1 < 2 , we can still adopt the idea of discarding odd-order terms.
Rewrite g ( λ ) as g ′ ( λ ) =
(cid:80) Ki =
( λ mid − λ )
i , where λ mid is the median of { λ i } .
By applying the same analysis above, we can see that removing odd-order terms from g ′ ( λ ) is still beneficial to narrow the SRL gap.
A.4 Proof of Corollary
Proof.
In the case where h ( G 1 ) = 0 and h ( G 2 )
= 1 , denote the label difference of G 1 and G 2 as ∆ yand ∆ y 2 , where ∆ y 1 = (1 , − 1 , 1 , − 1 , . .
, 1 , − 1) ⊤ , ∆ y
= (1 , . .
. , 1) ⊤ .
Denote U =
( u 0 , u 1 , . . .
, u N − 1 ) ⊤ , where u i is the i -th eigenvector of U and u k ( n ) is the n -th element of u k .
On ring graphs, denote a kn = sin( π ( k + 1) n/N, b kn = cos( πkn/N ) , then the normalized u k ( n ) satisfies:
[Equation]
for
[Equation]
Let the normalized spectrum of G 1 be δ
= U T ∆ y 1 = ( δ 0 , . . .
, δ N − 1 ) ⊤ , the normalized spectrum of
G 2 be δ ′ =
U T ∆ y 2 = ( δ ′ 0 , . . .
, δ ′ N − 1 ) ⊤ .
Suppose 1 ≤ i < N − 1 is odd, δ i = 1 √
[Equation]
[Equation]
N − 1 i =
[Equation]
) i a ki , δ ′ i = 1 √ N/ 2 (cid:80) N − 1 i =
 a ki and θ =
π ( i +1) N , then:
[Equation]
Therefore, for odd i and 1 ≤ i ≤ N − 2 , δ i = δ ′ i .
The conclusion can be generalized to even i and 1 ≤ i ≤ N − 2 using the same method.
For i =
and i = N − 1 , we have δ 0 − δ ′
= −√ N , δ N − 1 − δ N − 1 = −√ N .
The spectral gap between G 1 and G 2 is:
[Equation]
Therefore, the necessary condition for the spectral gap to be 0 is g (0) =
g (2) .
B Dataset Details B.1 Synthetic Datasets.
We conduct cSBM datasets following [10] in the inductive setting.
Denote a cSBM graph G as
G ∼ cSBM ( n, f, λ, µ ) , where n is the number of nodes, f is the dimension of features, and λ and µ are hyperparameters respectively controlling the proportion of contributions from the graph structure and node features.
We assume the number of classes is 2, and each class is of the same size n/ 2 .
Each node v i is assigned with a label y i ∈ {− 1 , +1 } and an f -dimensional Gaussian vector x i = (cid:112)
µn y i u
+ Z i √ f , where u ∼ N (0 , I/f ) and Z is a random noise term.
Assume the generated graph is of average degree d , and denote the adjacency matrix as A .
The graph structure of the cSBM graph is:
[Equation]
The parameter Φ discussed in the experiments is in the form of Φ = arctan( λmµ (cid:113) nf ) ∗ 2 π , where m > 0 is a constant.
A larger | Φ | reflects a larger λ over µ , that is the proportion of information from the graph structure is larger.
In practice, we choose n = 3000 , f = 2000 , d = 5 , m = 3 √ 32 for all graphs.
The choices of λ and µ and the resulting homophily ratio are listed in Table 7.
As discussed in [12], only the hyperparameters µ and λ that satisfy λ 2 + µ 2 f 2 n 2 > 1 are guaranteed to generate informative cSBM graphs.
As presented in Table 7, all our settings satisfy the need.
[Table]
C Experiment Details C.1 Experimental Device
Experiments are conducted on a device with an NVIDIA TITAN V GPU (12GB memory), Intel(R)
Xeon(R)
Silver 4114 CPU (2.20GHz), and 1TB of RAM.
C.2 Model Architectures
For GPRGNN, GCNII, GNNGuard and ProGNN, we rely on the officially released code.
For FAGCN, we implement the method with Pytorch Geometric(PyG) based on the released code.
For H2GCN, we rely on the PyG version implemented by [26].
Defense models are based on the DeepRobust
Library implemented versions[25].
Other methods are based on the PyG implemented versions [14].
The URL and commit number are presented in Table 8)
[Table]
[Table]
C.3 Hyperparameter settings
Node classification on cSBM Datasets & Common datasets.
For all models, we use early stopping 200 with a maximum of 1000 epochs. All hidden size of layers is set to 64.
We use the Adam optimizer and search the optimal leaning rate over {0.001, 0.005, 0.01, 0.05} and weight decay {0.0,
For all models, the linear dropout is searched over {0.1, 0.3, 0.5, 0.7, 0.9}.
For the modelspecific hyperparameters, we refer to the optimal hyperparameters reported in corresponding papers.
For MLP, we include 2 linear layers.
For GCN and H2GCN, we set the number of convolutional layers to be 2.
For GAT, we use 8 attention heads with 8 hidden units each in the first convolutional layer, and 1 attention head and 64 hidden units in the second convolutional layer.
For FAGCN, we search the number of layers over {2, 4, 8}, ϵ over {0.3, 0.4, 0.5}.
For GCNII, we set λ = 0 .
5 and search the number of layers over {8, 16, 32}, α over {0.1 0.3 0.5}.
For GPRGNN and EvenNet, we set the number of linear layers to be 2, the learning rate for the propagation layer to be 0.01, and α = 0 .
1 for initialization.
For both models, we search the dropout rate for the propagation layer over {0.1, 0.3, 0.5, 0.7} and the order of graph filter over {4, 6, 8, 10}.
Against adversarial attacks.
For the poisson attacks, we use a 2-layer GCN as the surrogate model.
We use the strongest variant of Metattack, which is “Meta-Self” as the attack strategy.
For the defense models, we carefully follow their provided guidelines of hyperparameter settings and use the optimal hyperparameters as they reported.
For GNNGuard, we use the official implementation GCNGuard with a threshold of 0.1.
For other models, we use the Adam optimizer with a learning rate of 0.01, weight decay of 0.0005, and a dropout rate of 0.5.
For FAGCN, we use 8 convolutional layers and a fixed ϵ = 0 .
For GCNII, we use 16 convolutional layers and a fixed α = 0 .
For GPRGNN and
EvenNet, we set the order of graph filter K to be 4 in the DICE attack.
In the poison attacks, the PPR initialization α for GPRGNN and EvenNet is searched over {0.1, 0.2, 0.5, 0.9} and K is set to be 10.
For all models, we use early stopping 30 with a maximum of 200 epochs.
Other hyperparameters are kept the same as the ones in the node-classification experiments.
C.4 Additional Defense Results
Homophily gap We include the homophily gap between training and test graph for Citeseer and
ACM datasets in Fig 4 and 5.
The homophily gaps of all attacks on all datasets grow larger as the perturb ratio increases.
     3HUWXUE5DWLR  
        + R P RSK LOO \  O H Y H O ',&( 7UDLQ 7HVW      3HUWXUE5DWLR     
  0HWD
[Figure]
     3HUWXUE5DWLR     
  0LQ0D
[Figure]
     3HUWXUE5DWLR  
        + R P RSK LOO \  O H Y H O ',&( 7UDLQ 7HVW      3HUWXUE5DWLR     
  0HWD
[Figure]
    
[Figure]
    
  0LQ0D
[Figure]
Additional Experiments about Defense against Poison Attacks.
Similar to DICE attacks, we provide the performance of GNN models under poison attacks of different perturb ratios.
The results are presented in Figure 6 and 7.
In most cases, EvenNet achieves SOTA with fewer introduced parameters.
    3HUWXUE5DWLR  
   $ FF X U D
F\ &RUD     3HUWXUE5DWLR        &LWHVHHU     3HUWXUE5DWLR     

 $& 0/3*&1*$7)$*&1+*&1*&1,,*&1B69'*&1B-DFFDUG*&1B*XDUG5*&1*35*11(YHQ1HW3UR*1
    3HUWXUE5DWLR  
    $ FF X U D
F\ &RUD     3HUWXUE5DWLR       &LWHVHHU     3HUWXUE5DWLR     

 $& 0/3*&1*$ )$*&1+*&1*&1,,*&1B69'*&1B-DFFDUG*&1B*XDUG5*&1*35*11(YHQ1HW3UR*1
D Spectral Methods under Perturbations D.1 More Graph filters
Spectral methods have gained plenty of attention these years.
Besides GPRGNN, we additionally include BernNet [18] and pGNN [15] which are advanced spectral GNNs for comparison.
We run the experiments of spectral GNNs against Meta, MinMax attacks, and the evasion DICE on dataset
ACM.
We tune the hyperparameters of these methods using the same search space in corresponding papers.
The results are summarized below:
[Table]
[Table]
Compared with spatial methods
[Table]
spectral methods which handle both homophily and heterophily are generally more robust.
Nevertheless
[Table]
EvenNet still holds superiority against other spectral methods
[Table]
when faced with large homophily changes.
In the evasion DICE attack, where the homophily gap is directly injected between training and test graphs, the superiority of EvenNet is apparent. D.2 Graph Filters under Random Attacks
We analyze the performance of graph filters under homophily change in the main body of the paper and show that graph filters suffer from performance degradation if there is a large homophily gap between training and test graphs.
We now discuss cases where a homophily gap is not huge after the graph structure is perturbed, for example, when the graph is under random attacks for both training and test sets.
We conduct Random Attacks on datasets Cora and Citeseer for the spectral methods.
In Random attacks, we randomly delete and add edges from/to the graph (we choose to delete or add with equal probability)
[Table]
and train graph filters on the perturbed graph.
A large homophily gap does not exist since the deleted/added edges are randomly chosen on the whole graph.
We also include a spatial method EGCNGuard from [8], which is an efficient version of GNNGuard for comparison.
The results are summarized in Table 12 and Table 13:
[Table]
[Table]
Although EvenNet is designed based on homophily generalization
[Table]
we could all spectral methods including EvenNet are quite robust under random attacks
[Table]
reflecting good stability under random perturbations of graph structures.
For a more comprehensive analysis of the performance of graph filters under random perturbations, we refer the readers to stability theory
[Table]
where the bound of change in output of graph filters is discussed [16, 21, 24].
EvenNet as a spectral method holds the stability property as well. E Scability to large graphs
In this section
[Table]
we try to run EvenNet on a larger dataset to verify its efficiency.
For comparison, we include vanilla GCN and the efficient implementation of GNNGuard EGCNGuard from [8].
Notice that we could not run GNNGuard and ProGNN for their O ( N 2 ) space complexity.
In Table E and Table E, we present the performance and running time of EvenNet on dataset ogbnarxiv against Random attacks.
We could see that EvenNet uses almost the same time as a 2-layer
GCN with the same hidden size.
And EGCNGuard of O ( | E | ) space complexity is still 3x slower than EvenNet in practice.
[Table]
[Table]
F Defense against Graph Injection Attacks
In the main body of the paper, we mainly focus on graph modification attacks, which are graph structural attacks that add/remove edges to/from the existing graph.
Another line of graph structural attack is graph injection attacks (GIAs), where new nodes are injected into the graph with generated features and form connections with existing nodes on the graph.
According to [8], GIAs significantly degrade the performance of GNNs by injecting only a few nodes with limited budgets.
The authors also state that injecting nodes with suspicious features that result in homophily inconsistency helps in enhancing the ability to attack.
Following [8], we apply non-targeted GIAs with Harmonious Adversarial Objective(HAO) including
PGD+HAO, AGIA+HAO, TDGIA+HAO as the attack methods.
The authors claim that GIAs with
HAO cause larger homophily gaps between training and test graphs.
We test the defense performance of EvenNet on dataset grb-cora, grb-citeseer [41] and Arxiv [19].
We use the same budgets as in [8], which is reported in Table F.
We compare EvenNet including Layernorm in MLP layers with different combinations of EGCNGuard with Layernorm and LNi operation.
(Layernorm is shown to be effective against GIAs.)
We set the threshold for EGCNGuard as 0.1.
We set the order of EvenNet
K as 2 and tune the PPR-like initialization of EvenNet α over {0.1, 0.2, 0.5, 0.9}
The results are summarized below in Table F and Table F.
We are not able to run AGIA+HAO on
Arxiv dataset due to resource limitations.
[Table]
[Table]
Notice that although GIAs include structural modifications by adding edges between injected nodes and existing nodes, GIAs are not pure structural attacks.
The injected node features are usually
[Table]
learnable, and therefore suspicious node features together with structural perturbations are included, which is beyond the scope of the paper.
We choose to leave designing robust spectral methods under feature perturbations to future works.
Notwithstanding GIAs are somehow out of the scope of
EvenNet, we can see that EvenNet is still competitive against strong spatial baselines in Table F and
Table F, which verifies the ability of EvenNet under homophily change.
G Defense on Heterophilic Graphs
In the main body of the paper, we conduct attacks mainly on homophilic graphs.
In this section, we add experiments about the MinMax attack on heterophilic datasets chameleon and squirrel with
GCN being the surrogate model.
We set the perturb ratio to be 20%.
For the hyperparameters, we search the learning rate over {0.01, 0.05} and set weight decay to be 0 for all models.
We set the threshold in EGCNGuard to 0.1.
The number of layers used in H2GCN and FAGCN is set to be 2, and ϵ is searched over {0.3, 0.4, 0.5} for FAGCN.
For GPRGNN and EvenNet, we set α = 0 .
1 for
PPR initialization and K = 10 .
The results are summarized below:
[Table]
We can see from the results that EvenNet is still effective in defense against attacks on heterophilic graphs.
Yet, we did not focus on heterophilic datasets as GNNs already perform badly on them, which is also a reason why current attacks mainly focus on homophilic graphs.
On the Squirrel dataset, the performance of GNNs is only slightly higher than MLP, reflecting an almost useless graph structure.