A General Unified Graph Neural Network Framework Against Adversarial Attacks
Authors: Yujie Gu
Yangkun Cao
Qiang Huang
Huiyan Sun
Venue: ICLR 2022 Submitted (2022)
Keywords: Graph Neural Networks
general unified framework
against adversarial attacks
robust model
graph reconstruction operation

Abstract: Graph Neural Networks (GNNs) are powerful tools in representation learning for graphs. However, they are reported to be vulnerable to adversarial attacks, raising numerous concerns for applying it in some risk-sensitive domains. Therefore, it is essential to develop a robust GNN model to defend against adversarial attacks. Existing studies address this issue only considering cleaning perturbed graph structure, and almost none of them simultaneously consider denoising features. As the graph and features are interrelated and influence each other, we propose a General Unified Graph Neural Network (GUGNN) framework to jointly clean the graph and denoise features of data. On this basis, we further extend it by introducing two operations and develop a robust GNN model(R-GUGNN) to defend against adversarial attacks. One operation is reconstructing the graph with its intrinsic properties, including similarity of two adjacent nodes’ features, sparsity of real-world graphs and many slight noises having small eigenvalues in perturbed graphs. The other is the convolution operation for features to find the optimal solution adopting the Laplacian smoothness and the prior knowledge that nodes with many neighbors are difficult to attack. Experiments on four real-world datasets demonstrate that R-GUGNN has greatly improved the overall robustness over the state-of-the-art baselines. 

Under review as a conference paper at ICLR 2022
A GENERAL UNIFIED GRAPH NEURAL NETWORK FRAMEWORK AGAINST ADVERSARIAL ATTACKS
Anonymous authors
Paper under double-blind review
ABSTRACT
Graph Neural Networks (GNNs) are powerful tools in representation learning for graphs. However, they are reported to be vulnerable to adversarial attacks, rais- ing numerous concerns for applying it some risk-sensitive domains. There- fore, it is essential to develop a robust GNN model to defend against adversarial attacks. Existing studies address this issue only considering cleaning perturbed graph structure, and almost none of them simultaneously consider denoising fea- tures. As the graph and features are interrelated and influence each other, we pro- pose General Unified Graph Neural Network (GUGNN) framework to jointly clean the graph and denoise features of data. On this basis, we further extend operations and develop a robust GNN model(R-GUGNN)
it by introducing to defend against adversarial attacks. One operation is reconstructing the graph with its intrinsic properties, including similarity of two adjacent nodes' features, sparsity of real-world graphs and many slight noises having small eigenvalues in perturbed graphs. The other is the convolution operation for features find the optimal solution adopting the Laplacian smoothness and the prior knowledge that nodes with many neighbors are difficult to attack. Experiments on four real-world datasets demonstrate that R- R-GUGNN has greatly improved the overall robustness over the state-of-the-ar baselines.
INTRODUCTION
Graph Neural Networks(GNNss) have drawn great attention graphs can represent complex re- lationships among nodes. Graphs are ubiquitous in different domains, which are usually applied in recommender systems(Ying al. 2018a), Chemistry(Duvvenald
al.) 2015). social media
et al. 2018) and so on. Utilizing the strong representation capacity of graphs. we can enhance performance of down-stream tasks such as node classificatonn(Kpt & Welling 2017 [0177/Velickove
et al., 2018 Klicpera et al.) 2019), link predietionGrrover & Leskovee 2016 Bojchevski et al.) 2018 and graph classification Defferrard
D et al., 2016 et al. 2018b). A GNN model often consists of several graph convolution layers. A common practice of convolution layers is utilizing a feed-forwart network to transform features and then aggregating transformed features. A series of convolution layers have been proposed and achieved great success such GCN(Kipf & Welling 2017]. GAT (Velickvoo et al. 2018) and PPNP(Klicpera et al. 2019).
However, GNN models composed of these convolution layers are vulnerable adversarial attacks. Attacks can be conducted on either node features or the graph structure, while most existing adver- sarial attacks on graph data focus modifying the graph structure(Xu et al. 2020). They always try to add, delete, or rewire edges to change the graph structure. Although these perturbations are unnoticeable, they can easily degrade the performance of GNN models, which may cause bad con- sequences. For example, spammers may create virtual followers to increase the chance of false messages being recommended and spread. The lack of GNNs' robustness raises increasing con- cerns for applying it in some risk-sensitive domains. Therefore, it is necessary to develop graph defense techniques. Many existing defense methods focus on cleaning perturbed graphs by detect- ing properties of clean graphs and effects of specific attacks on graphs(Entezari et al.) 2020; Jin et al. 2020b). Prior knowledge according to these researches can help GNN models defend against adversarial attacks to certain extent. The study(Jin al.) 2020b) has proved that adversarial at- tacks could lead perturbed graphs to violate some properties of real graphs. For example, the rank of attacked graph increases and adversarial attacks often connect nodes with large feature differences.

---

Under review as a conference paper at ICLR 2022
Step Ø
Convolation Aawr
Convolation layer
log
RaLU
Drapout
Output
Poisoned
Clemn greah
graph reconstruction operations Stop !))
Figure 1: Concrete design of R-GUGNN. We clean the attacked graph and denoise features.
However, existing studies only focus cleaning the perturbed graph structure, and almost none of them simultancously consider denoising features.
As the graph and features closely tied and contain perturbations and noises, in this paper. pro- pose a General Unified Graph Neural GUGGN))(GUGNN) framework to jointly clean the graph and denoise features. Based on the GUGNN framework, we further introduce two kinds of operations. One operation is reconstructing the graph with its properties that real-world graphs are sparse (Zhou) al. 2013), the features of two adjacent nodes tend to be MicilarnMcPPerson et al., (000T) and per- turbed graphs have many slight noises with small eigenvalues. Nodes with more neighbors are hard to attack(Zügner et al. 2018). Though we cannot change the number of nodes' first-order neigh- bors, we can adjust the size of nodes' neighborhood to add some high-order neighbors to nodes. According to this principle, from the denoising perspective, we design the convolution operation for features find the optimal solution. Utilizing the two kinds of operations above, develop robust model(R-GUGNN) which can be applied for defending against different adversarial attacks.
The contributions of this paper summarized follows:
We propose GUGNN framework to jointly clean the graph and denoise features for pertur- bations and noises existing the graph and features.
We introduce kinds of operations to clean attacked graphs and denoise features respec- tively based on the GUGNN framework.
For defending against adversarial attacks, we develop a concrete model R-GUGNN to re- alize the goal of the GUGNN framework utilizing the two kinds of operations.
Experiments show that R-GUGNN has strong capacity for defending against different adversarial attacks and stably outperforms the state-of-the-art defense models.
2 RELATED WORK
This section has two parts, including graph neural networks, and adversarial attacks and defenses for GNNs.
2.1 GRAPH NEURAL NETWORKS
In this subsection, we review some famous graph neural network models, including GCN, GAT, PPNP, and a unified GNN framework UGNN. For more knowledge about GNNs, you can refer to some reviews al. 2020 Zhang al. 2020).
The convolution of Kipf & Welling, 2017) is defined in the graph spectral domain. Avoiding computing the full eigenvectors of the graph Laplacian matrix, based on Chebyshev polynomials, GCN only uses the first-order polynomial to simplify the graph convolution, which has been an accepted good graph convolution layer for its performance. The convolution of GAT GAT(Velickovic jet al. 2018) is defined in the spatial domain. The difference between GAT and GCN is that GAT learns different attention scores for neighbors when aggregating features. PPNP(Kliepera al.
2

---

Under review as a conference paper at ICLR 2022
2019) derives propagation scheme based personalized PageRank. It propagates information from a large and adjustable neighborhood instead the first-order neighborhood directly. The neighborhood can be adjusted via myper-paramette
MG et al. 2020) is a unified GNN framework available for different feature propagation processes from denoising perspective. It proposes denoising optimization problem with the Laplacian regularization term and tries to solve it in different ways utilizing the first derivative or the optimal solution. Original Laplacian matrix can be also replaced with different normalized forms. Different solutions to the optimization problem are corresponding to various convolution layers with different feature aggregation processes such as GCN, GAT, PPNP and so on.
2.2 ADVERSARIAL ATTACKS AND DEFENSES FOR GNNS
We recommend repository DeepRobust Li et al. 2020) for readers. It contains many adversarial attacks and defenses on the graph, which is quite useful for researchers. For more knowledge about adversarial attacks and defenses for GNNs, you refer to the review(Jin et al. 2020a)..
Some adversarial attack methods have been proposed to show the vulnerability of GNNs with some unnoticeable perturbations added the graph structure node attributes. In the field of node classification, the aim adversarial attacks is fooling GNNs into classifying nodes incorrectly. Poi- soning attacks change the graph structure before we train GNN models, which is one of the most common settings of adversarial attacks on graph data. Poisoning attacks have various types, includ- ing global attack, targeted attack and random attack. Based on the whole graph, the goal of global attack is to degrade the overall performance of GNNs. One of the state-of-the-and global attacks is melallackkZigner
Günnemann 2019), which generates the poisoning attacks based on meta- learning. Targeted attack generates attacks on some specific nodes and aims to fool GNNs on these target nodes. The nettack (Züiger et al., (018)) is one of the state-of-the-an targeted attacks, which aims to change the graph structure and features of target nodes nearby nodes with perturbations remaining unnoticeable. Random attack adds random noises to the clean graph whose concrete practice adding, removing or flipping edges randomly.
Methods about preventing GNNs from adversarial attacks are also developed to improve the ro- bustness of GNNs recently. To mitigate the effects of adversarial attacks on the graph, RGCN (Zhu et al. 2019) uses Gaussian distributions as hidden representations of nodes instead of plain vectors in other GNNs. Considering that nettack is high-rank attack, GCN-SVDDEEteea proposed to reconstruct the perturbed graph with only the top-k largest singular components. Using such a low-rank approximation, GCN-SVD can reduce the effects of nettack. In fact, the practice of ensuring low-rank is removing noises with small singular values of the graph. Pro-GNN et al., 2020b) jointly optimizes a structural graph and robust GNN model from the perturbed graph with some properties of clean graphs. Pro-GNN has big improvement over other defense models with these properties. However, like other defense models, Pro-GNN doesn't t take denoising features into account jointly.
et al.) 2020) is
THE PROPOSED FRAMEWORK
In this section, we first present GUGNN framework, and then we introduce a novel graph recon- struction operation. At last, show the convolution operation for features and our concrete design of R RUGGNN model, which is used to realize the goal of GUGNN framework.
3.1 NOTIONS
We denote some notations here. Denote X E Nxx as the feature matrix, where N, d represent the number of samples and the dimension of features respectively. Denote G (V,E} as the graph, where represents node sets and E represents edge sets. We also use the adjacency matrix A E RNNN to represent G.
3

---

Under review as a conference paper at ICLR 2022
Algorithm 1: R-GUGNN
Input: Adjacency matrix Feature matrix X, Labels y, m, c,8, Learning
rate 1)-
Output: GNN parameters 0 = A + Initialize A;
for i= 1 to do
Calculating Z in formula(5);
S S 2 57
S ,
S prox, (S);
Initialize F = X:Randomly initialize 0; while Stopping condition is not met do Forward propagation using two convolution layers: Using feature transformation formula: FW, 10
and feature aggregation formula: F XL) Getting output y':
12
Calculating gradient g according and y/: Backward propagation: 19g;
14
15 return 0;
3.2 THE GENERAL UNIFIED GNN FRAMEWORK
To discard perturbations and noises in the graph and features, considering the tight connection be- tween them, we propose our general united graph neural network(GUGNN) framework to solve such problem, which is shown follows:
argmin L S A 7 S,F
XII : + LF) 3 S(S)
(1)
where S and F are the learned adjacency and feature matrix. L is the Laplacian matrix of S. L D S. where D is diagonal matrix and D Σx= Sty. L can be also replaced with different normalized forms. tr LF) Laplacian regularization term for both denoising features and cleaning the graph. S(S) flexible regularization term enforce some prior over S. 7, and 3 are hyper-parameters to balance different components.
From united perspective, view that both features and the graph contain noises and our goal is jointly optimizing F and tr(F' LF) can be rewritten as 1 Σie= S.y 1 f, where f, is the 2
i-th row of F. This term represents that features of two adjacent nodes should be similar, which is the guidance for both learning F and S. Although X and A have some noises, they can represent the real features and the graph to a large extent. So, the learned and should be similar to and A respectively, which are the meanings of ||F OF--Xy X|| and SS Ali In addition, we add some prior to the graph in f(S) to make it more accurate.
3.3 THE NOVEL GRAPH RECONSTRUCTION OPERATION
We focus cleaning the perturbed graph supposing F-X. Formula(1) of GUGNN can be rewritten as follows:
argmin C = || A + secetr (X" LX) 3 f(S)
S
Considering that the graph contains noises, we rewrite formula(2) as follows:
argmin
Á + c- tr(xxxx) + S||S||
(3)
+ L3
The adjacency matrix with self-loop Á and the normalized Laplacian matrix L are adopted. =
D LD i, tr(xTix) is equal to } E S.
ززند 2 Since degrees of the perturbed
4

---

Under review as a conference paper at ICLR 2022
Table 1: Description of datasets
NLCc
ELCC
Classes
Features
Cora
2485
5069
7
1433
Citeseer
2110
3668
6
3703
Cora-ML
2810
7981
7
2879
Polblogs 1222
16714
2
graph are approximately equal to those of the real graph, for the convenience of calculation, we let
DR = Σ |S||. = E, Erank(S) os, where a; is the i-th singular value of S.
To solve formula(3), we let 0C1+ = 0 to get the closed form solution.
VNN )2
V.1
DDi
VD1
0C1 C2
2(S A) +
0
(4)
0S
XN
X1
X.V
NN
)2
VDN
VD11
VDN
VNN
DDi
DD1
VD.1
VDN
c
S
(5)
)
VDNN )
VDNN
VD
VDN
We denote formula(5) S À Z for convenience. A proximal operator of nuclear norm is adopted to remove noises and reserve main gnntettissEnnezaai et al. 2020).
prax (S)
(max 3, 0.0
where S Udiag( ... ON VT is the singular value decomposition of S. Let S = p (S) to represent this step. For the constraint S.y [0, 1], we let S S to enhance self-loop, and set Siy00 to and S to We denote this step as S = prox, (S), which can make the graph sparse at the same time.
3.4 THE CONVOLUTION OPERATION FOR FEATURES
After getting the cleaned graph through several graph reconstruction operations above, we fix it and focus on denoising features, formula(1) of GUGNN can be rewritten follows:
argmin L = | X + A tr(F" LF)
(7)
where = 4. In this case, formula(7) is equal to that of UGNN(Ma et al. 2020). We use the normalized Laplacian matrix and Øc = 0 to find the optimal solution.
OF
OC OF
2(F X) 2XLF =
(8)
F == (
(9)
Formula(9) is the process of feature aggregation. Before it, we let X XW transform features, where W is the parameter of a single GNN convolution layer. This is our convolution operation for features, which is ( et al. 2020) equal PPNPpKlicpera al. 2019). So, our convolution operation for features can also adjust nodes' neighborhood to enhance the model's robustness.
3.5 THE DESIGN OF R-GUGNN MODEL
Utilizing the two kinds operations, we design R-GUGNN model and show Figure where is the number of graph reconstruction operations we can set. In step ① we fix features and clean the graph with m graph reconstruction operations. In step @, we fix the cleaned graph and denoise features with two graph convolution layers. We train GNN parameters 0 using the two graph convolution layers and classify nodes finally. Concrete steps of R-GUGNN are shown in Algorithm

---

Under review as a conference paper at ICLR 2022
Table Node classification performance (Aceuracy=Std) under metattack
Datasets
Ptb Rate(%)
GCN
GAT
RGCN
GCN-SVD
Pro-GNN
R-GUGNN
83.06+0.52
84.09=0.66 66
83.83=0.59
77.69+0.52
85,49=0.38
82.97=0.33
5
77.08+1.05
79.96*1.01
79.21#0.42
77.54+0.91
79.03=1.31
82.25+0.51
10
10.46#1.14
74.82=1.33
73.11=0.76
72.73+0.90
74.11=0.77
80.91+0.30
Cora
15
1..21
70.17+1.34
68.68+0.80
69.1140.67
70.34=0.50
80.60:0.37
14.69+1.37
58.33+1.49
58.35+0.39
57.46+2.04
67.78=0.48
77.96*1.15
25
49.53+1.47
52.22+2.76
53.27+0.56
54.46+1.60
66.19:0.85
76.05=1.37
72.22+0.49
12.87+1.38
72.98+0.29
67.89+0.68
72.68=0.78
73.59+0.33
69.76+1.91
71.87+1.60
71.66+0.43
68.4440.44
72.17=1.67
73.15+0.71
10
13.25+1130
70.02+1.18
69.17=0.57
69.73+0.88
73.06=0.50
73.96+0042
Citeseer
15
63.87+1147
67.30+2.03
65.9340.37
68.0640.45
71.24=0.54
73.25+0.80
20
16.00#11.6
60.08+1.09
56.83+0.54
68.7140.65
29.
71.64+0.59
57.1042.45
61.00+1.99
58.6940.47
65,43+1.01
57.23=1.22
71.74+0.99
0
85.77+0.32
85,46+0.51
85.97=0.42
78,78+0.17
83.30=0.66
85,29+0.24
80.01#0.42
81.20=0.81
80.68+0.39
77.92+0.22
83.92=0.45
84.70+0.35
74.51+0.56
75.97=0.90
74.70=0.76
77.61+0.39
81,69=0.42
84.12:0.16
Cora-ML
15
54.36+0.66
57.80+1.22 .44
55.86+1.06
74.92+0.33
53.88=0.45
82.13+0.46
20
45.6420.71
42.02+2.36
48 08.08
51,01
46.99=2.82
70.74+0.90
25
48.20&1.45
26.6812.51
50.58+0.42
66.5740.51
50.82=0.45
74.30=0.47
95.83+0.40
94.99=0.41
95.30=0.25
91.93+0.37
95.25=0. 14
95.68+0.31
5
72.91
76.69=0.96
72.0440.54
89.1040.35
93.53=0.47
95.30+0.57
10
72.71
12.56+1.33
718944.51
81.25+0.50
87.5320.83
94.37=0.85
Polblogs
68.3540.42
54.7348.66
68.6640.66
70.2742.42
85, 85.8811.79
91.44+5.74
20
59.34+2.45
50.07+4.35
62.14+0.80
58.73+3.64
77.05=3.35
84.13+5.08
25
58.39#1.61
50.91
59.89=0.96
53.03+2.32
70.34=2.05
70.72+7.27
EXPERIMENTS
In this section, we evaluate the effectiveness R-GUGNN model compared with the state-of-the art GNN models against different attacks. We first introduce the experimental settings and then present results of a series of experiments. last, we conduct the ablation study and analyze hyper- parameters of R-GUGNN.
4.1
EXPERIMENTAL SETTINGS
4.1.1 DATASETS
We compare different models on four benchmark datasets, including three citation graphs, i.c., Cora(McCallum al., (2000), Citeseer(Giles et al. 1998) and Cora-ML & Günnemann 2017). and one blog graph, T.e., Polblogs(Jin et al.] 20206). Cora-MI. the subset of machine learn- ing papers from Cora dataset, which is also well-known dataset in GNN field. Since Polblogs dataset has no node features, a N x N identity matrix is used to act as the feature matrix. We only consider the largest connected COMponen(LCCC) in each dataset(Jin
2020b Zügner al.
2018). Table contains detailed information about the dataset.
4.1.2 BASELINES
R-GUGNN model are compared with the state-of-the-an GNN and defense models in repository DeepRobust(Li et al.) 2020), i.e. GCN(Kipf & Welling 2017), CAT/Velickovis et al. 2018 RGCN (Zhu et al. 2019). GCN-SVD(Entezar et al.] 2020) and Pro-GNNUin et al. 2020b). We adopt the default parameter settings in GCN and GAT. The number of hidden units of RGCN tuned from {16, 32, 64, 128). The reduced rank the perturbed graph in GCN-SVD is tuned from (5, 10, 15, 50, 100, 200). For Pro-GNN, we use the tuned hypee-pprameten the author gives online.

---

Under review as a conference paper at ICLR 2022
077
Teat
-1-00
SA7
875
631
0.0
Number Perturbations Nade
Namber Perturbetion Per Node
Perturaations
Number Pettrtottoons
(a) Cora
(b) Citeseer
(c) Cora-ML
(d) Polblogs
Figure 2: Node classification performance (Accuracy) under nettack
304
046
023
0.9
6:00
0.77
L-CUGES
Text
6.22
0.11
0.75
047
(a) Cora
(b) Citeseer
(c) Cora-ML
(d) Polblogs
Figure 3: Node classification performance (Accuracy) under random attack
4.1.3 PARAMETER SETTINGS
Just JsiJin et al. 2020b), for each dataset, we choose 10% of nodes for training, 10% nodes for validation and the remaining 80% of nodes for testing. The average performance of 10 runs is reported for all experiments below. The hyper-parameters of the models are tuned based on the loss and accuracy on the validation set. Note that the same hyper-parameterd are used under the same attack for the same dataset no matter what perturbation rate is. If there are no special instructions, all models adopt two graph convolution layers with 16 hidden units. Learning of Adam optimizer y is fixed as 0.01 and negative log likelihood loss is adopted for a fair Comparison(RGCN has its own loss function).
4.2 PERFORMANCE AGAINST DIFFERENT ATTACKS
The node classification performance of R-GUGNN is evaluated against three types of poisoning attacks, i.e., global attack, targeted attack and random attack. Since Ploblogs dataset has no real node features, parammee c of R-GUGNN
0 on Ploblogs dataset.
4.2.1 AGAINST GLOBAL ATTACK
The famous metattack metattackZZigner & Günnemann) 2019) is used as the global attack to conduct exper- iments and all the default parameter settings in the authors' original implementation adopted. Concretely, the strongest variant Meta-Self is applied for all datasets. The perturbation rate of metattack the graph is from 0% 25% with step of 5%, since too heavy attacks are no- ticeable and make no sense. We report the average accuracy of node classification with standard deviation test and highlight the optimal results in bold. Concrete results are shown in Table and we draw some conclusions:
R-GUGNN has great improvement compared to others on four datasets. The average im- provement of accuracy under different perturbation rates over GCN on four datasets is about 16%, 10%, 19% and 21% respectively. When the graph is heavily perturbed, the improvement is larger. For example, when the perturbation rate is 20%, the improvement over GCN is about 23%, 16%, 25% and 25% on four datasets respectively. When compared with different second best models, improvement can reach 10%, 6%. 19% and 7% on four datasets. These results prove that R-GUGNN can defend against metatlack very well.

---

Under review as a conference paper at ICLR 2022
0.80
0.2
Standrad Deviation
0.02
Acquracy
-8 -----UGAN 1996
0.00
0.7
0.2
0.4
01.
0.6
Porturbation Rate
Perturbation Rate
(a) Accuracy
(b) Standard Deviation
Figure 4: Performance variants of R-GUGNN
0.8
0.88
Test Aearacy
0.79
mm
-0
636
3300:
00001
0.301
601
1.5
25
05
hype-parametermm
Pyper-prrameteer
Hyper- hyprr-parametenp
hyper-purameterá
(a) m
(b) c
(c)
(d) A
Figure 5: Parameter analysis Cora-ML dataset under 15% metattack
Accuracy of R-GUGNN is stably high under different perturbation rates on all datasets. The gap between accuracy of R-GUGNN example, gaps of the accuracy of R-GUGNN under 25% and 0% metattack on Cora and Citeseer datasets are only about 7% and 2%. Besides, the overall standard deviations of R-GUGNN are small. However, the lack of real node features on Polblogs dataset causes big standard deviations when the graph heavily attacked.
clean and perturbed graphs small. For
4.2.2 AGAINST TARGETED ATTACK
The typical netttack Zügner et al. 2018) is employed as the targeted attack to conduct experiments and all the default parameter settings the authors' original implementation nodes with degree > 10 as targeted nodes from the test set. The number of perturbations of the graph on each targeted node from nodes results, which are shown in Figure 2. R-GUGNN suffers less effects of nettack and also performs greatly and stably. For example, compared to the second best method Pro-GNN, R-GUGNN achieves 10% and 5% improvement on Citeseer and Cora-MI. datasets. These results prove that R-GUGNN can defend against netttack very well.
adopted. We select
with a step of 1. We report accuracy of these targeted
4.2.3 AGAINST RANDOM ATTACK
Performance of R-GUGNN under random attack is evaluated here. We add random perturbations on the graph from 0% 100% with a step of 20%. Concrete results are shown in Figure 3. R-GUGNN outperforms other models again and the improvement is distinct. For example, compared to different second best models, R- R-GUGNN achieves 2.5% and 3.5% improvement on Citeseer and Cora-MI. datasets. These results prove that R R-GUGNN can defend against random attack very well.
From the overall performance, we observe that the advantage of R-GUGNN obvious compared with others and its performance is stably great. In conclusion, R-GUGNN is robust enough to defend against different attacks.
4.3 ABLATION STUDY
R-G R-GUGNN contains m graph reconstruction operations. we discard these operations and only use two convolution layers for features, R-GUGNN is equal to PPNP(Klicpera et al. 2019). So, in this
8

---

Under review as a conference paper at ICLR 2022
subsection, we compare PPNP with GCN and R-GUGNN Cora dataset under random attack as an example to illustrate. In addition, we set 3 and c to to understand the impact of each component in the graph reconstruction operation. Furthermore, observe standard deviations of R-GUGNN when c=0.
In (ig)re(4)(a), we can see that performance of R-GUGNN is better than PPNP and performance of GCN is the worst. It shows graph reconstruction operations are significant(R-GGGNY vs PPNP), and adjusting neighborhood beneficial to defending against attacks(PPNP vs GCN), PPNP curve and the no-3 curve overlap very well, which indicates that removing noises with small singular values plays quite important role cleaning the graph. What's more, c=0, not only model's performance is poor, but also the standard deviation rises lot especially when the graph is heavily attacked in (i)) shows the Laplacian regularization term is significant improving stability of R-GUGNN, which explains why the standard deviation is big on Polblogs dataset under heavy attack.
4.4 PARAMETER ANALYSIS
In this subsection, we show performance of R-GUGNN with different values of hyper-parameters i.e, m, c, 8, and A. We use Cora-MI. dataset under 15% metaltack as an example to illus- trate. The value range of m is from to 5 with the step of 1. The value of is selected in {10-5, 10-4, 1003, 1,-2, 10010-1 10-1). We select 3 from 1 to 3.5 and À from 0.5 to 3 with the step of 0.5. In the process of tuning one hyper-parameter, other hyper-parameters are fixed as the optimal. Figure shows effects of different values of hyper-parameters.
m is the number of graph reconstruction operations. Our novel operations of R- R-GUGNN are impor- tant for defending against attacks, and even one such operation improves model's robustness(766 76.87% accuracy). However, proper m can boost the accuracy and too many such operations cannot bene- fit R-GUGNN. .9 is also key affecting the performance of R-GUGNN, which controls how many noises with small singular values to remove. When 5 too small, noises cannot be removed entirely. While when 3 is too big, the main properties of the graph can be hurt. À is used to adjust the size of nodes' neighborhood when propagating features and choosing proper À is also important, e used to control the Laplacian smoothness of the graph. We find the big value of hurts the performance of R- R-GUGNN, but when c is small, accuracy doesn't 't decrease a lot. From a whole performance, all hyper-parameters have an interval of values where the performance of R-GUGNN is stably great.
5 CONCLUSION
In this paper, we propose GUGNN, a novel general unified framework to effectively enhance the robustness of GNNs against adversarial attacks by jointly cleaning the perturbed graph and denoising the features of data. Furthermore, we extend this framework by reconstructing the graph and making convolution operations of features with intrinsic properties, and propose a robust GNN model R- GUGNN. Experiment results show that R-GUGNN stably outperforms the of-te-of-the-ar baselines under different adversarial attacks. In the future, we aim to extend this framework to other models on graphs, even more complicated graph structures for mining the rich value underlying graph data of various domains.
REFERENCES
Aleksandar Bojchevski and Stephan Günnemann. Deep gaussian embedding of graphs: Unsuper- 2017.
vised inductive learning via ranking. arXiv preprint
Aleksandar Bojchevski, Oleksandr Shehur, Daniel Zügner, and Stephan Günnemann. Netgan: Gen- erating graphs via random walks. In JCML, 2018.
M. Defferrard, X. Bresson, and P. Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In NIPS, 2016.
D. Duvenaud, D. Maclaurin, J. Aguilera waw.terrraaaaarrrrreeeeeee R. G6ómez-Bombaree Timothy D. Hirzel, Alán Aspuru-Guzik, and Ryan P. Adams. Convolutional networks on graphs for learning molec- ular fingerprints. ArXiv, abs/1509.00 09209, 2015.

---

Under review as a conference paper at ICLR 2022
Negin Entezari, Saba A Al-Sayouri, Amirali Darvishzadeh, and Evangelos E Papalexakis. All you need is low (rank) defending against adversarial attacks on graphs. In Proceedings of the 13th International Conference Web Search and Data Mining, pp. 169-177, 2020.
Lee Giles, Kurt D Bollacker, and Steve Lawrence. Citeseer: An automatic citation indexing system. In Proceedings of the third ACM conference on Digital libraries, pp. 89-98, 1998.
Aditya Grover and J. Leskovec. node2vec: Scalable feature learning for networks. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016.
Wei Jin, Yaxin Li, Han Xu, Yiqi Wang, Shuiwang Ji, Charu Aggarwal, and Jiliang Tang. Adver- sarial attacks and defenses graphs: A review, tool and empirical studies. arXiv preprint arXiv:2003.00653 2020a.
Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. Graph structure learning for robust graph neural networks. Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, 2020b.
Thomas Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. ArXiv, abs//609.0200000 2017.
Johannes Klicpera, Aleksandar Bojehevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank. In ICLR, 2019.
Yaxin Li, Wei Jin, Han Xu, and Jiliang Tang. Deeprobust: A pytorch library for adversarial attacks and defenses. arXiv preprint arXiv:2005.06149 2020.
Yao Ma, Xiaorui Liu, Tong Zhao, Yozen Liu, Jiliang Tang, and Neil Shah. A unified view on graph neural networks as graph signal denoising. arXiv preprint arXiv:2010.01777. 2020.
Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3(2):127-163, 2000.
M. McPherson, L. Smith-Lovin, and J. Cook. Birds of a feather: Homophily in social networks Review of Sociology, 27:415 2001.
J. Qiu, Jian Tang, Hao Ma, Yuxiao Dong, Kuansan Wang, and Jie Tang. Deepinf: Social influence prediction with deep learning. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018.
Petar Velickovic, Guillem Cucurull, A. Casanova, Adriana Romero, P. Lio', and Yoshua Bengio. 2018.
Graph attention networks. ArXiv,
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 32(1): 24, 2020.
Han Xu, Yao Ma, Haochen Liu, Debayan Deb, H. Liu, Jiliang Tang, and Anil K. Jain. Adversarial attacks and defenses in images, graphs and text: A review. International Journal of Automation and Computing, 17:151-178, 2020.
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and J. Leskovec. Graph convolutional neural networks for web-scale recommender systems. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 201 8a.
Rex Ying, Jiaxuan You, Christopher Morris, Xiang Ren, William L. Hamilton, and J. Leskovec. Hierarchical graph representation learning with differenniable pooling. ArXiv, abs/1806.08840 2018b.
Ziwei Zhang, Peng Cui, and Wenwu Zhu. Deep learning on graphs: A survey. IEEE Transactions Knowledge and Data Engineering, 2020.
10

---

Under review as a conference paper at ICLR 2022
Ke Zhou, H. Zha, and Le Song. Learning social infectivity in sparse low-rank networks using multi- dimensional hawkes processes. In AISTATS, 2013.
Dingyuan Zhu, Ziwei Zhang, Peng Cui, and Wenwu Zhu. Robust graph convolutional networks against adversarial attacks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1399-1407, 2019.
Daniel Zügner and Stephan Günnemann. Adversarial attacks on graph neural networks via meta learning. arXiv preprint arXiv: 2019.
Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. Adversarial attacks on neural networks for graph data. In Proceedings of the 24th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 2847-28566 2018.


Reviews:
[{"review_id": "U0CzhZKCffC", "reviewer": "Reviewer_4H1G", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Positives:\n\n+ The paper is well-written. \n+ The problem is relevant. In particular, the idea of learning how to fix the perturbed structure as well as the node features is interesting.\n+ The results section is also well designed with three attack models, ablation studies, and hyperparameter results.\n\n Negatives:\n\n- The novelty of the paper is not clear. Part of the objective (Eq 1) is taken from UGNN (Ma et al. 2020). Also, the approach taken in the paper is step-wise. After denoising the graph through reconstruction operations, it is solving Eq. 7 (same as in UGNN).  As a backup clarification, why isn't UGNN being used as a baseline (or in the ablation study)?\n\n - It would be good to have a comparison with other defense architectures. There are already methods that defend against poisoning attacks (e.g., GNNGuard, NeurIPS, 2020). Pro-GNN does not seem to be the state of the art.\n\n- About the reconstruction: It is not clear what should be an optimal number of rounds for reconstruction. It feels ad hoc as of now. The results on m (number of rounds) are also not convincing. Why should the results be worse for a higher value of m? It will be good to have some discussion around all of these.\n\nMinor:\n\n- It will be good to have a discussion about the efficiency of the proposed method. \n\n- The model seems to produce a high standard deviation for polblogs data. I am not sure if one can project R-GUGNN as the best model for that dataset. Is there any intuition/explanation for why it is happening?", "summary_of_the_paper": "The paper proposes a robust model for graph neural networks (GNNs) to defend against adversarial attacks. While most of the defense mechanisms in previous work focus on identifying and fixing the perturbed structure of the underlying graph, this paper proposes to jointly fix the perturbed structure as well as the node features. The proposed objective function captures these components along with weighting by some hyper-parameters. The experiments include several baselines and show results on different attack models: targeted attack, untargeted or global attack, and random attack. The experiments also have results on ablation study as well as hyper-parameters. ", "summary_of_the_review": "Please refer to the negative points in the detailed review.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "OLA9Z-pa-nN", "reviewer": "Reviewer_z4si", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Problem and motivation**\n\nThe robustness of graph-based machine learning models is an important problem worthy of study. The authors do a good job explaining the problem and motivating its importance. \n\nThey also highlight the lack of research jointly considering feature and structural robustness, rather than just structural robustness. Although it should be noted that the Pro-GNN framework does consider both the graph and the features [2].\n\nOn adversarial attacks, the authors state \"They always try to add, delete, or rewire edges to change the graph structure\". This is not true, some do modify features too (such as nettack) and some also change the graph structure in other ways (i.e. through node injection [1])\n\n**Adversarial attack taxonomy**\n\nThroughout the work, there seems to be some confusion about adversarial attack taxonomy. For example, the authors write “Poisoning attacks have various types, including global attack, targeted attack and random attacks”. Whilst true, having a global or targeted setting is not unique to poisoning attacks, the other common setting of evasion attacks can also be global or targeted. Meanwhile, a random attack is a strategy of attack and can be applied in any adversarial setting. In 4.2 the authors write “.. R-GUGNN is evaluated against three types of poisoning attacks, i.e., global attack, targeted attack and random attack.”. Again, global and targeted attacks are an attack goal and random attack is a strategy. The authors may find Section 3 of [3] helpful. Related to this, the sentence “Random attack adds random noises to the clean graph whose concrete practice is adding, removing or flipping edges randomly”, this sentence is not easy to parse and it’s unclear what the authors are trying to say. \n\n**Framework**\n\nThe GUGCN framework provides a cleaned version of the adjacency matrix and the feature matrix. The terms promote the following properties:\n+ Provide a cleaned adjacency that is not too far from the original\n+ Provide a feature matrix that is not too far from the original\n+ Provide a feature matrix that is smooth with respect to the learned cleaned matrix \n+ Provide a cleaned adjacency according to some prior f(S)\n\nThe authors claim the framework is general and unified, however, I have some concerns about this claim:\n+ The term which is concerned with the feature matrix promotes a specific property (smoothness) whereas the term concerned with the adjacency is general. It’s unclear why one is specific and the other is general. \n+ Despite having a general term for the adjacency matrix, the authors only consider a single realisation (low-rank regularisation), without suggestions for other possibilities.\n+ I believe that if the authors want to propose a general and unified framework they should show that existing works are a special case of the framework or provide more than one instantiation of the framework. The authors do show that PPNP is a are special case, but this is given late in the paper in the results section (Section 4.3).\n+ The authors promote the smoothness of features, which is obviously preferable for homophilic datasets. I doubt this method would work well on non-homophilous datasets. Since most of the current benchmark semi-supervised node classification datasets are homophilous this isn’t a major issue,  but the authors could maybe acknowledge this point in their manuscript.\n\nI believe this work could be greatly improved if they demonstrated exactly how previous works fit into this framework. Currently, it is unclear what their contribution is over other works, as it seems similar to some existing works. Section 3 which derives steps 1-5 of Algorithm 1 look like it’s the same as equation (17) of the Pro-GNN paper [2]. However, I don’t think this connection is made in the manuscript. The final objective of Pro-GNN (Eq. 9) is also similar to the GUGCN objective.  \n\nThe naming is quite confusing, as the GUGCN by itself is not a graph convolutional network. \n\nI’m unsure of the meaning of the following sentence “Although X and A have some noises, they can represent the real features and the graph to a large extent”.\n\n**Derivation**\nMoving from Eq (2) to Eq (3) the authors switch the framework to operating on an adjacency matrix and the combinatorial Laplacian to an adjacency matrix with self-loops and normalised Laplacian matrix. This change is not discussed or justified, and Eq (3) no longer fits into the GUGNN framework as defined in Eq (1)\n\nI don’t think the equation at the bottom of page 4 should have the (1/2) term at the front.\n\n\"Since degrees of the perturbed graph are approximately equal to those of the real graph, for the convenience of calculation, we let\". The authors here seem to have made an assumption on the perturbation they are considering and should be more clear about this. Furthermore, the convenience they make is stronger than what they claim, the degrees are not approximately equal they are exactly equal.\n\nSection 3.3 in general, particularly the last paragraph, is quite confusing and lacks details and references. It is not clear to me where the final term in Eq. (4) comes from. \n\nThe authors should comment on the scalability of Eq. (9) which is equivalent to PPNP. Due to the need for inverting a matrix, this method is unlikely to scale to networks with 100,000+ nodes. The authors could discuss using APPNP instead. \n\n**Experiments**\n\nThe experiments section is the strongest part of this paper. It is very thorough and includes key analyses such as ablation studies and sensitivity to hyper-parameters. The results are convincingly improved compared to baseline methods.  \n\nThe standard deviations in Figure 4 (a) are small, could they be added to Figure 4 (b) as error bars?\n\nIn Section 4.4 the ranges for each hyper-parameter are a bit inconsistent and the authors do not discuss why this is. Why is c chosen on a log-scale but the other parameters aren't? Why are the other parameters (beta and lambda) not chosen on the same linear scale (e.g. 0.5, ..., 3.5 in steps of 0.5)?\n\nThe layout of results is similar to Pro-GNN (table 2, figure 2 and figure 3 in this paper vs table 2, figure 3 and figure 4 in Pro-GNN), yet the results in some cases for Pro-GNN are quite different. For example at 25% Ptb rate on Cora Jin et al. report 69.72 ± 1.69 whereas this paper reports 66.19 ± 0.85. For Citeseer Jin et al. report 68.95 ± 2.78 whereas this paper reports 57.23 ± 1.22. Do the authors know why these are so different?\n\nFrom the code, it looks like although the model training is done over many random seeds, the same poisoned graph is used each time. The variance estimates would be more reliable if a different perturbed graph was used for each seed. I believe authors should change this, or at least mention it in the manuscript. \n\n**Minor points and formatting suggestions**\nThe formatting could be improved in places. For example:\n+ No space between acronyms “Graph Neural Networks(GNNS)”\n+ Multiple spaces after full stops “among nodes.   Graphs”\n+ No space between citations “systems(Ying et al. 2018a)”\n+ R -> \\mathbf{R} for set of reals\n+ Netttack -> nettack\n\n**Code (not factored into my scores)**\nThe code in the supplementary does not run. Trying to run api.py will give an import error. It looks like “from attacked_data import PrePtbDataset” should be changed to “from deeprobust.graph.attacked_data import PrePtbDataset”. The api.py script could be improved by using argparse.\n\n[1]: Tao, Shuchang, et al. \"Single Node Injection Attack against Graph Neural Networks.\" arXiv preprint arXiv:2108.13049 (2021).\n\n[2]: Jin, Wei, et al. \"Graph structure learning for robust graph neural networks.\" Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020.\n\n[3]: Sun, Lichao, et al. \"Adversarial attack and defense on graph data: A survey.\" arXiv preprint arXiv:1812.10528 (2018).\n\n\n\n\n", "summary_of_the_paper": "The authors propose a General Unified Graph Neural Network (GUGNN) framework to address graph and feature denoising. They use this framework to design a graph neural network (R-GUGNN) which is Robust against poison adversarial attacks in a semi-supervised node classification setting. The authors demonstrate on four common datasets (Cora, Citeseer, Cora-ML & Polblogs) that their method is generally more robust against perturbation compared to commonly used graph neural networks (GCN, GAT) and models which are designed to be robust in the same setting (RGCN, GCN-SVD, Pro-GNN).", "summary_of_the_review": "The authors propose an interesting idea, to jointly sanitise the graph structure and the features as a defence against adversarial attack. The experiments are extensive and thorough and the method appears to perform well. However, the paper is let down by two major factors. The first is that it is confusing how their framework is general and how it relates to prior works. It’s also not obvious specifically how they are innovating compared to prior works as parts of their method are similar to some previous studies. The second is that the derivation is lacking details and references and, as it is written, is not at all easy to follow. I believe this paper has potential but in its current state, is not yet ready to be published. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "FS0q-XL9U50", "reviewer": "Reviewer_V2ZH", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n1.\tThis paper focuses on an important problem of learning robust GNN against attacks on both structure and node features.\n2.\tThe proposed method shows good results on four small graphs and perform better than most of the robust GNN methods.\n3.\tAblation study and hyperparameter analysis are conducted, which makes it easier to reproduce the results of the proposed method\n\nCons:\n1.\tThe proposed method lacks novelty. This method proposes to discard the perturbations on graph structure by learning a adjacency matrix S that link nodes with similar node features. And a constraint in Eq(2) is added to S to make sure it will not differ a lot from the poisoned adjacency matrix A. However, these constraints have been proposed by ProGNN[1]. The proposed method simply extends it to graph whose node features are also attacked by adding one more regularizer to ProGNN, which lacks novelty. In addition, this straightforward extension seems problematic. Because the node features are attacked, nodes with similar attacked features may not indicate clean edges anymore. In other words, the graph denoising relies on the node features. Attacked node features may lead to poor graph learning and further feature denoising. It is suggested to have more discussions and justification about the proposed method.\n\n2.\tSince the proposed method focuses on adversarial attacks on both features and structures, more details about the experimental settings on attacks such as the levels of perturbations on node features are required.\n\n3.\tExperiments are only conducted on four small datasets. It is suggested to conduct experiments on large datasets to show the scalability of the proposed method. It is also suggested to compare with some methods that also cares the attacks on node features such as [2] to make the comparisons more convincing.\n\n[1] Jin, Wei, et al. \"Graph structure learning for robust graph neural networks.\" Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020.\n\n[2] Feng, Fuli, et al. \"Graph adversarial training: Dynamically regularizing based on graph structure.\" IEEE Transactions on Knowledge and Data Engineering (2019).\n", "summary_of_the_paper": "This paper focuses on developing a robust GNN model to defend against adversarial attacks on both graph structures and node features. More specifically, a general unified graph neural network is proposed to learn a graph structure and node features to correct the adversarial attacks. This a two-step approach. First, one operation is applied to reconstruct the graph with the node features. Then, the node features are updated by another operation with the learned graph structure. Experiments are conducted on four small datasets for evaluation.", "summary_of_the_review": "In summary, this paper studies a robust GNN against adversarial attacks on both graph structure and node features. However, the methodology lacks novelty and requires further justification. Some important technical details are also not covered. Though the experimental results on the small datasets are good, some more experiments on large datasets and the details of the experimental settings are required. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "AeVC2M1al0Z", "reviewer": "Reviewer_Hj7t", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The paper tackles an important problem\n- The paper is easy to follow\n- Ablation studies and various experiments show the potential benefit of the technique (though, the datasets are rather small and further competitors could be used) \n\nWeaknesses:\n- Technical novelty and model definition: I am not fully convinced by the approach. In particular, the authors first perform \"graph cleaning\" and only subsequently denoise the features. Not only, leads this to a low technical novelty (since graph cleaning has been proposed already; likely the feature denoising -- as mentioned by the authors -- is actually PPNP) but also to some questionable choices. \nIn particular, in Section 3.3. the authors assume F=X (i.e. the features are NOT perturbed). This is a strong contraction to the core idea that the features are actually perturbed. Accordingly, the graph cleaning step clearly is based on some \"wrong\" features.\nMoreover, the claim in the abstract of \"simultaneously consider denoising features\" is also not really solved by the proposed technique, since it is only sequentially considered.\n- Model definition (II): In general, the model definition is unclear and/or not well presented. For example, Eq. (1) does not contain any learnable parameters and only considers the \"cleaning\". Only later it is added that X is actually replaced by some learnable transformation. This also leads to the situation, that learning actually only happens for the \"feature part\" (second part in Algo 1) but not the graph part. I find this very unsatisfying. Or put differently: The graph cleaning could then be used in combination with ANY standard GNN. It would be interesting to include such a comparison.\n- Model definition (III): The assumption that the degrees of the perturbed and cleaned graphs are the same seems very rough. Also, it is unclear to me why one considers the derivative of L1+L2, even though the full objective is L1+L2+L3.\n- Experiments: A comparison on more complex datasets and with more competitors would be helpful. In particular, as mentioned above, the graph cleaning could be used with any GNN. Is the actual benefit of the proposed work the graph cleaning? Since the feature part corresponds to PPNP, it seems to be the case. It would be helpful is the authors discuss this point more carefully.\n\nFurther (minor) points:\n- The abstract/introduction reads a bit like \"no work has considered feature perturbation\". This is, however, incorrect. E.g., as mentioned by the authors, Netattack actually consideres features. Also works like \"Certifiable Robustness and Robust Training for Graph Convolutional Networks, KDD'19\" do consider robustness regarding feature perturbations. I would recommend to rephrase the corresponding paragraphs.\n\n", "summary_of_the_paper": "Considering the problem of adversarial manipulations of graphs, the authors propose a framework for \"cleaning\" the graph structure and its features to obtain more robust predictions. The core contribution, compared to existing works, is the additional considering of the feature cleaning. Technically, the authors first perform the graph cleaning (phrased as some kind of sparse, trace minimization problem) followed by some feature cleaning (phrased as some kind of feature diffusion/propagation). Experiments on standard (small) benchmark datasets shows some improvements.", "summary_of_the_review": "The model definition of the paper is not convincing and the experimental results needs a more careful investigation. In particular, the novelty seems to be limited to the graph cleaning since the remaining parts are an existing model (PPNP).", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper studies a robust GNN against adversarial attacks on both graph structure and node features. 
The reviewers agree that the paper need to improve in terms of novelty and more technical details to meet ICLR standard.
Decision: Reject