BiGCN: A Bi-directional Low-Pass Filtering Graph Neural Network
Authors: Zhixian Chen
Tengfei Ma
Zhihua Jin
Yangqiu Song
Yang Wang
Venue:  (2021)
Keywords: Graph convolutional networks
graph filtering
Laplacian smooth
ADMM

Abstract: Graph convolutional networks have achieved great success on graph-structured data. Many graph convolutional networks can be regarded as low-pass filters for graph signals. In this paper, we propose a new model, BiGCN, which represents a graph neural network as a bi-directional low-pass filter. Specifically, we not only consider the original graph structure information but also the latent correlation between features, thus BiGCN can filter the signals along with both the original graph and a latent feature-connection graph. Our model outperforms previous graph neural networks in the tasks of node classification and link prediction on benchmark datasets, especially when we add noise to the node features.

Under review as a conference paper at ICLR 2021
BIGCN: A BI-DIRECTIONAL LOw-PASS FILTERING GRAPH NEURAL NETWORK
Anonymous authors Paper under double-blind review
ABSTRACT
Graph convolutional networks have achieved great success graph- graph-structured data. Many graph convolutional networks can be regarded as low-pass filters for graph signals. In this paper, we propose graph neural network as a bi-directional low-pass filter. Specifically, we not only consider the original graph structure information but also the latent correlation between features, thus BiGCN can filter the signals along with both the original graph and a latent centture-connction graph. Our model outperforms previous graph neural networks in the tasks of node classification and link prediction on most of the benchmark datasets, especially when we add noise to the node features.
model, BiGCN, which represents
INTRODUCTION
Graphs are important research objects in the field of machine learning as they are good carriers for structural data such as social networks and citation networks. Recently, graph neural networks (GNNs) received extensive attention due to their great performances in graph representation learning. A graph neural network takes node features and graph structure (e.g. adjacency matrix) as imput, and embeds the graph into a lower-dimensional space. With the success of GNNs (Kipf & Welling) 2017 Veličković et al. 2017 Hamilton et al. 2017, et al.) 2018) in various domains, more and more efforts are focused on the reasons why GNNs are so powerful (Xu et al. 2019).
Li et al (Li et al. 2018) re-examined graph convolutional networks (GCNs) and connected it with Laplacian smoothing. NT and Maehara el (NT Maehara 2019) revisited GCNs in terms of graph signal processing and explained that many graph convolutions can be considered as low-pass filters (e.g. (Kipf & Welling 2017; Wu et al. 2019)) which can capture low-frequency components and remove some feature noise by making connective nodes more similar. In fact, these findings are not new. Since its first appearance in Bruna et al. (2014), spectral GCNs have been closely related to graph signal processing and denoising. The spectral graph convolutional operation is derived from Graph Fourier Transform, and the filter can be formulated as a function with respect the graph Laplacian matrix, denoted as gL).. In general spectral GCNs, the forward function is: H(+1) (g(((())))) ()))))))
Kipf and Welling (Kipf & Welling 2017) approximated (T)) using first-order Chebyshev polynomi- als, which can be simplified muluplying the augmented normalized adjacency matrix to the feature matrix. Despite the efficiency, this first-order graph filter is found sensitive to changes in the graph signals and the underlying graph structure (lsufi al. 2016 Bianchi al. 2019). For instance, on isolated nodes or small single components of the graph, their denoising effect is quite limited due to the lack of reliable neighbors. The potential incorrect structure information will also constrain the power of GCNs and cause more negative impacts with deeper layers. As noisy/incorrect information inevitable in real-world graph data, more powerful and robust GCNs are needed to solve this problem. In this work, propose new graph neural network with more powerful denoising effectts from the perspective of graph signal processing and higher fault tolerance to the graph structure.
Different from image data, graph data usually has high dimensional features, and there may be some latent comnection/correlttion between each dimensions. Noting this, we take this connection information into account to offset the efforts of certain unreliable structure information, and remove extra noise by applying a smoothness assumption on such a "feature graph". Derived from the additional Laplacian smoothing regularization in this feature graph, we obtain novel variant of

---

Under review as a conference paper at ICLR 2021
spectral GCNs, named BiGCN, which contains low-pass graph filters for both the original graph and a latent feature connection graph in each convolution layer. Our model can extract ww-frequency components from both the graphs, so it is more expressive than the original spectral GCN; and it removes the noise from two directions, so it is also more robust.
We evaluate our model two tasks: node classification and link prediction. In addition to the original graph data, in order to demonstrate the effectiveness of our model with respect to graph signal denoising and fault tolerance, we design three cases with noise/structure mistakes: randomly adding Gaussian noise with different variances to certain percentage of nodes; adding different levels of Gaussian noise to the whole graph feature; and changing certain percentage of connections. The remarkable performances of our model in these experiments verify our power and robustness on both clean data and noisy data.
The main contributions of this work are summarized below.
We propose a new framework for the representation learning of graphs with node features. Instead of only considering the signals in the original graph, we take into account the feature correlations and make the model more robust.
We formulate our graph neural network based on Laplacian smoothing and derive bi- directional low-pass graph filter using the Alternating Direction Method of Multipliers (ADMM) algorithm.
We set three cases to demonstrate the powerful denoising capacity and high fault tolerance of our model in tasks of node classification and link prediction.
2 RELATED WORK
We summarize the related work in the field of graph signal processing and denoising and recent work on spectral graph convolutional networks follows.
2.1 GRAPH SIGNAL PROCESSING AND DENOISING
Graph-structured data is ubiquitous in the world. Graph signal processing (GSP) (Ortega al. 2018) is intended for analyzing and processing the graph signals whose values are defined on the set of graph vertices. can be seen as a bridge between classical signal processing and spectral graph theory. One line of the research in this area is the generalization of the Fourier transform to the graph domain and the development of powerful graph filters (Zhu Rabbat 2012 Ssufi al. 2016). can be applied to various tasks, such as representation learning and denoising (Chen et al. 2014). More recently, the tools of GSP have been successfully used for the definition of spectral graph neural networks, making a strong connection between GSP and deep learning. In this work, restart with the concepts from graph signal processing and define a new smoothing model for deep graph learning and graph denoising. It is worth mentioning that the concept of denoising/robustnes: in GSP is different from the defense/rrbustness against adversarial attacks (e.g. (Zügner & Günnemann 2019). so do not make comparisons with those models.
2.2 SPECTRAL GRAPH CONVOLUTIONAL NETWORKS
Inspired by the success of convolutional neural networks in images and other Euclidean domains, the researcher also started to extend the power of deep learning to graphs. One of the earliest trends for defining the convolutional operation on graphs is the use of the Graph Fourier Transform and its definition in the spectral domain instead of the original spatial domain (Bruna et al.) [014). Defferrard et al (Defferrard et al. 2016) proposed ChebyNet which defines a filter as Chebyshev polynomials of the diagonal matrix of eigenvalues, which can be exactly localized in the k-hop neighborhood. Later on. Kipf and Welling (Kipf & Welling 2017 simplified the Chebyshev filters using the first-order polynomial filter, which led to the well-known graph convolutional network. Recently, many new spectral graph filters have been developed. For example, the rational auto nuto-regressive moving average graph filters (ARMA) [ssui et al., 2016; Bianchi et al., 2019)) proposed to enhance the modeling capacity of GNNs. Compared to the polynomial ones, ARMA filters are more robust and provide a more flexible graph frequency response. Feedhack-looped filters (Wijesinghe & Wang 2019) further
2

---

Under review as a conference paper at ICLR 2021
improved localization and computational efficiency. There is also another type of graph convolutional networks that defines convolutional operations in the spatial domain by aggregating information from neighbors. The spatial types are not closely related to our work, so it is beyond the scope of our discussion. As we will discuss later, our model is closely related to spectral graph convolutional networks. We define not only to the original graph but also to a latent feature graph in order to improve the capacity and robustness of the model.
graph filter from the perspective of Laplacian smoothing. and then extend it
BACKGROUND: GRAPH SIGNAL PROCESSING
In this section, we will briefly introduce some concepts of graph signal processing (GSP), including graphs smoothness, graph Fourier Transform and graph filters, which will be used in later sections.
Graph Laplacian and Smoothness. A graph can be represented as G = (V, E). which consists of a set of n nodes V = {1.....7 , and a set of edges E S V x V. In this paper. we only consider undirected attributed graphs. We denote the adjacency matrix of G A = (atz) Raxn and the degree matrix of G as D diag(d(1)..... d(n)) E RR In the degree matrix, d(i) represents the degree of vertex
V. We consider that each vertex i E V associates a scalar r(i) E IR which is also called graph signal. All graph signals can be represented by x E IR". Some variants of graph Laplacian can be defined on graph G. We denote the graph Laplacian of G' as L = D A E Raxx It should be noted that the sum of rows of graph Laplacian L is zero. The smoothness of graph signal can be measure through the quadratic form of graph Laplacian: Azz) = x Lx Ei (z(() =(i))² Due to the fact that x Lx 2 0, L is a semi-positive definite and symmetric matrix.
Graph Fourier Transform and Graph Filters. Decomposing the Laplacian matrix with UAUT, we can get the orthogonal eigenvectors U as Fourier basis and eigenvalues A graph frequencies. The Graph Fourier Transform F R IR" defined by Fr - U z. The inverse Graph Fourier Transform is defined by F--i = Uš. It enables to transfer the graph signal to the spectral domain, and then define graph filter y in the spectral domain for filtering the graph signal 2:
g(L)x Ug(A)U" (g(A))((>)
where (AA diag(o(())....g(1)) controls how the graph frequencies can be altered.
4 BIGCN
The Graph Fourier Transform has been successfully used to define various low-pass filters on graph signals (column vectors of feature matrix) and derive spectral graph convolutional networks (Deffer rard et al. 2016 Bianchi et al., 2019 Wijesinghe & Wang 2019). A spectral graph convolutional operation can be formulated as function g with respect to the Laplacian matrix L. Although it can smooth the graph and remove certain feature-wise wise noise by assimilating neighbor nodes, it is sensitive to node-wise noise and unreliable structure information. Notice that when the node features contain rich information, there may exist correlations between different dimensions of features which can be used to figure out the low-tolerance problem. Therefore, it is natural to define filters on "feature signals" (row vectors of graph feature matrix) based on the feature correlation. Inspired by this, propose a bi-directional spectral GCN, named BiGCN, with column filters and row filters derived from the Laplacian smoothness assumption, shown in Fig In this way, denoising capacity and fault tolerance to graph structure of spectral graph convolutions. To explain better, we start with the following simple case.
enhance the
4.1 FROM LAPLACIAN SMOOTHING TO GRAPH CONVOLUTION
Assuming that f yo + r an observation with noise 7), recover the true graph signal yo. a natural optimization problem is given by:
min || y f 113 +xy Ly, y
3

---

Under review as a conference paper at ICLR 2021
fastures
faatur graph
x
ADMM
H1+1
Casture conrelstion
Row Alters
dg
W/
O
O 4
4
Column
Figure 1:
Illustration of BiGCN layer. In the feature graph, d indicates each dimension of features with a row vector the input feature matrix as its "feature vector". We use a learnable matrix to capture feature correlations.
where À is hyper-parameter, T. is the (normalized) Laplacian matrix. The optimal solution this problem is the true graph signal given by
y = -I+
we generalize the noisy graph signal f to noisy feature matrix F Yo + then the true graph feature matrix Yo can be estimated as follows:
Yo arg min
F [| +Xrrace(Y" (T ( FF.
YTLY. the Laplacian regularization, achieves a smoothness assumption on the feature matrix. -L) is equivalent to a low-pass filters in graph spectral domain which can remove feature- wise/column-wiis noise and can used to defined a new graph convolutional operation. Specifically, by multiplying a learnable matrix W (i.e. adding linear layer for node feature transformation beforchand, which is similar to (Wu et al. 2019 NT Machara, 2019)), we obtain a new graph convolutional layer follows:
H a(II AL) -
(3)
In order to reduce the computational complexity, can simplify the propagation formulation by approximating +I+ AL) with its first-order Taylor expansion I -LA. AL.
4.2 BI-DIRECTIONAL SMOOTHINN AND FILTERING
Considering the latent correlation between different dimensions of features, similar the graph adjacency matrix, can define "feature adjacency matrix" A' to indicate such feature connections. For instance, i th,j th, k th dimension feature refer to "height", "weight" and "age" respectively, then "weiigh" may have very strong correlation with "height" but weak correlation with "age", so is reasonable to assign A = 1 while A', = (if we assume is a 0 matrix). With a given "feature adjacency matrix", we can construct corresponding "feature graph" in which nodes indicate each dimension of features and edges indicate the correlation relationship. In addition, if Yaxd is the feature matrix of graph G, then Y would the "feature matrix of the feature graph". That is, the column vectors of Y xux are the feature vectors of those original nodes while the row vectors are exactly the feature vectors of "feature nodes". Analogously, we can derive the Laplacian matrix L' of this feature graph.
When noise not only feature-wist but also node-wise. or when graph structure information is not completely reliable, is beneficial consider feature correlation information in order to recover the clean feature matrix better. Thus we add a Laplacian smoothness regularization on feature graph to the optimization problem indicated above:
L min Y F 1 +Nutrace(Y" LYY + >yatraceYYL2 L22
Here L, and L2 are the normalized Laplacian matrix of the original graph and feature graph, A and X2 hyper- parametssss of the two Laplacian regularization. Y//Y is the Laplacian regularization
4

---

Under review as a conference paper at ICLR 2021
on feature graph vectors of the original feature matrix. The solution of this optimization problem is equal to the solution of differential equation:
dc OY 2Y - L + 212
0.
(5)
This equation, equivalent to \1 - is Sylvester equation. The numerical solu- tion of Sylvester equations can be calculated using some classical algorithm such as Bartels-Stewar Stewart algorithm (Bartels 1972), Hessenberg-See
method (Golub al. 1979) and LAPACK algorithm (Anderson et al. 1999). However, all them require Schur decomposition which including House- holder transforms and QR iteration with (nn²) computational cost. Consequently, transform the original problem to bi-criteria optimization problem with equality constraint instead of solving the Sylvester equation directly:
C = (())) + min g (Ye) 8.1 Y2 Y1 =0, 0,
1
S(Y,) 2
Y1
F [1 + trace Y,, 12)1),
1 || Y/2 - F 1 -2 Y2 )2 2
g(Y2)
(6)
We adopt the ADMM algorithm (Boyd et al.
2011) to solve this constrain convex optimization
problem. The augmented Lagrangian function of Lis:
CY, Y2, (). =(()) (\2) + trace ( Yi)) + P Y22 Yi 11
The update iteration form of ADMM algorithm is:
y(t (k++ = arg min (V1,
(())
Y1
= arg min- Yi
Yı F 1/ +i1 trace Lx L2Y1) + trace(Z()) (r -Y1)) +
Y Yi 11
2
(k+1) = arg min Cp
Y2, (x,( (b)
(k+1)
Y2
Y/2
Y22 1| +X2trace + trace(Z(1)) (82 - 1V2-Y(htii) r(btt))))
= ary min Y:
+ 2/ Y22 y (++1) 1|2
((++)) = ((k) + p( p(y (k+1) y (1++1)).
(8)
We obtain Yi and Y2 iteration formulation by computing the stationary points of CY1 (k) 2(0))
and (y-(((+++)) Y2, z ())):
y (k+1)
1 (I+ 2 L1) + px
+ =z)k)
1 1+1 I+p
(k+1)
1+p (F + +px 21 +
1+ L2) -1
To decrease the complexity of computation,
first-order Taylor approximation simplify the iteration formulations by choosing appropriate hyper-parameters p and X1 A2 such that the eigenvalues 1+p
2λ, L1 and 2da Lg all fall into [-1
1+p
(((+1)_
1
(I
2\1 L1)(F + (xx + +Z(1)),
1 p
1+ +pp
Y, (+++1)
1
(F (k+1) z(())(1 -
I+p L2),
2\₂
I
z = (H) + +(Y - -r(h-11)
(10)
5

---

Under review as a conference paper at ICLR 2021
In each iteration, shown in Fig we update Y1 by appling the column low-pass filter I
2 L1 to
1+p
T+p L2 to the new Y1. To some extent, the new Y1 is the low-frequency column components of the original Y2 and the new Y2 is the low-frequency row components of the new Y1. After k iteration (in our experiments, k = 2), we take
the previous Y2, then update Y2 by appling the row low-pass filter / - 2dz
the mean of y11h and y>(k) as the approximate solution Y, denote it as Y = ADMM(FF L2). In this way, the output of ADMM contains two kinds of low- fww-freuency components. Moreover, we can generalize L2 to a learnable symmetric matrix based on the original feature matrix F (or some prior knowledge), since is hard to give a quantitative description on feature correlations.
In (1+1)tt propagation layer, F = H is the output of 4th layer, L2 is learnable symmetric matrix depending on H(,, for this we denote T,2 as I The entire formulation is:
H(+1) = or (ADMM L, L2 W
(11)
Discussion about over-smoothing Since our algorithm is derived from a bidirectional smoothing, some may worry about the over-smoothing problem. The over-smoothing issue of GCN is explored in (Li al. 2018; Oono & Suzuki 2020), where the main claim is that when the GCN model goes very deep, it will encounter over-smoothing problem and lose its expressive power. From this perspective, our model will also be faced with the same problem when we stack many layers. However, single BiGCN layer is just more expressive and robust filter than a normal GCN layer. Actually, compared with the single-direction low-pass filtering GCN with general forward function: (I++))
(g (O(L))HWW (w)),, AADMM((HO)) LI, (,,") _,"),, combining wwwffreuenncy components of both column and row vectors of H is more informative than ((L)) (L))HH
(( since the latter can be regarded one part of the former to some extent. It also explains that BiGCN is more expressive that single-direction low-pass filtering GCNs. Furthermore, when take T.2 an identity matrix (in equation 5). BiGCN degenerates to a single-directional GCN with low-pass wwwpass filter: ([1 + 12) + A L1) It also illustrates that BiGCN has more general model capacity. More technical details are added Appendix.
-
In practice, we can also mix the BiGCN layer with original GCN layers or use jumping knowledge (u jt al., 2018) to alleviate the over smoothigg problem: for example, we can use BiGCN at the bottom and then stack other GCN layers above. As we will show in experiments, the adding smoothing term in the BiGCN layers does not lead to over-smoothing: instead, it improves the performance on various datasets.
5 EXPERIMENT
We test BiGCN on two graph-based tasks: semi semi-supervised node classification and link prediction on several benchmarks. As these datasets are usually observed and carefully collected through a rigid screening, noise can be negligible. However, in many real-world data, noise is everywhere and cannot be ignored. To highlight the denoising capacity of the bi-directional filters, we design three cases and conduct extensive experiments artificial noisy data. In noise level case, levels of noise to the whole graph. In noise rate case, we randomly add noise to a part of nodes. Considering the potential unreliable connection the graph, to fully verify the fault tolerance to structure information, we set structure mistakes case in which we will change graph structure. We compare our performance with several baselines including original GCN (Kipf & Welling (017). GraphSAGE (Hamilton et al. 2017). GAT (Veličković et al. 2017). GIN (Xu et al. 2019), and GDC (Klicpera al. 2019).
add different
5.1 BENCHMARK DATASETS
We conduct link prediction experiments on Citation networks and node classification experiments both on Citation networks and Co-purchase networks.
Citation. A citation network dataset consists of documents as nodes and citation links as directed edges. We use three undirected citation graph datasets: Cora (Sen et al. 2008). CiteSeer (Rossi & Ahmed 2015). and PubMed (Namata et al. (002)) for both node classification and link prediction tasks as they are common in all baseline approaches. In addition, we add another citation network DBLP (Pang al. 2015) link prediction tasks.

---

Under review as a conference paper at ICLR 2021
Co-purchase. We also use two Co-purchase networks Amazon Computers (McAuley al. 2015) and Amazon Photus (Shchur et al., 2018). which take goods as nodes, to predict the respective product category of goods. The features are bag-of-words node features and the edges represent that two goods are frequently bought together.
5.2 EXPERIMENTAL SETUP
We train two-layer BiGCN the same other baselines. Details of the hyperparameters setting and noise cases setting are contained in the appendix.
Learnable L2. We introduce completely learnable L. in our experiments. In detail, define L2 = 1 - D2 DD- A22, A2 1/2 A2 = W2 + W where W2 = sigmoid(W) and W is an uppertriangle matrix parameter be optimized. To make it sparse, also add LI regularization to L2. For each layer, L2 is defined differently. Note that our framework general and in practice there may be other reasonable choices for L2 (e.g. discussed in Appendix).
5.3 BASELINE MODELS
We compare our BiGCN with several state-of-the-art GNN models: GCN (Kipf & Welling 2017)
GraphSAGE (Hamilton al. 2017). GAT (Veličković al. 2017). GIN (Xu et al. 2019h: Graph Isomorphism Network, GDC Klicpera et al. (019): Graph diffusion convolution based on gen- eralized graph diffusion. We compare one of the variants of GDC which leverages personalized PageRank graph diffusion to improve the original GCN and adapt GCN into link prediction tasks is consistent with the implementation P-GNN.
5.4 RESULTS
We set three types of noise cases in terms of noise level, noise rate and structure mistake to evaluate each model on node classification and link prediction tasks (excluding structure mistakes). "Noise level" and "noise rate" add different types of noise to node features; "structure mistake" indicates we randomly remove or add edges in the original graph. For noise on node features, we expect our BiGCN show its ability graph filters. For structural errors, expect the latent feature graph can help with the correction of structural errors in original graphs. The detailed settings of these cases as well as some additional experimental results be found in the Appendix.
GCN
SAGE
GAT
GIN
GDC
BIGCN
Coza
Giteseer
PubMed
Photo
0.50
0.80
Aceuniy
0.60
0.80
8.70
0.50
0.60
0.50
0.40
0.40
0.40
0.30
0.75
0,50
0.75
0.50
0.75
0.25
0.50
leed
lesel
GCN
SAGE
GAT
GIN
GDC
BGCNN
Cora
Cireseer
PobiMed
DRLPP
0.90
0.90
0.93
0.85
0.90
0.50
0.88
0.80
0.00
0.30
0.25
0.00
0.25
0.50
0.25
0.25
0.50
0.75
0.25
0.50
0.75
level
Nosse Jevil
Noše level
Noise lovel
Figure 2: Node classification (Top) and link prediction (Bottom) results of models in noise level case.
Noise level case, this case, we add Gaussian noise with a fixed variance (from 0.1 to 0.9, called the noise level) to the feature matrix. As Fig 2 shows, BiGCN outperforms other baselines and
7

---

Under review as a conference paper at ICLR 2021
shows flatter declines with increasing noise levels, demonstrating better robustness in both node classification and link prediction tasks.
Noise rate case. Here, randomly choose part of nodes fixed percentage (from 0.1 to 0.9, called the noise rate) to add different Gaussian noise. From Fig 3 we can see that, on the two tasks, BiGCN performs much better than baselines all benchmarks apart from Cora. Especially on the PubMed dataset, BiGCN improves node classification accuracy by more than 10%.
GCN
SAGE
GAT
GIN
GDC
BGON
PuhMed
Photo
0.80
0.80
Aecunsyy
0.60
0.00
0.70
0.40
0.40
0.50
0.40
0.0
1.0
0.0
0.
0.0
0.5
1.0
0.0
0.5
1.0
Nolie sat
Noss sate
Nolic sare
Noise tule
GCN
SAGE
GAT
GIN
GDC
BGGNN
DBLP
0.50
0.90
0.50
0.93
0.85
AUC
0.85
0.00
0.00
0.50
0.88
0.75
0.85
Noise rite
Notre mite
Noise yare
Noise mi
Figure 3: Node classification (Top) and link prediction (Bottom) results of models in noise rate case.
Structure mistakes case. Structure mistakes refer to the incorrect interaction relationship among nodes. In this setting, we artificially remove or add certain percentage of edges of graphs at random and conduct experiments on node classification. Fig illustrats the outstanding robustness of BiGCN that superior to all baselines, demonstrating that our bi-directional filters can effectively utilize information from the latent feature graph and drastically reduce the negative impact of the incorrect structural information. At last, we would like to mention that our model also outperform other models in most cases clean data without noise. This attribute to BiGCN's 's ability to efficiently extract graph features through its bidirectional filters. The detailed values in the figures are listed in Appendix.
GCN
SAGB Citescer
GAT
GIN
GDC
BGGN
Cora
PuhMed
Phoso
0.80
0.80
(.60
0.80
Accuncy
0.70
a.50
0.60
3.40
0.60
3.40
0.40
0.20
0.50
500
1500
5.0
1000
1500
75
5.0
1000
1500
(X(0-")
mite (x10-")
Figure 4: Node classification accuracy of models in structure mistakes case.
6 CONCLUSION
We proposed bidirectional low-pass filtering GCN, more powerful and robust network than general spectral GCNs. The bidirectional filter of BiGCN can capture more informative graph signal components than the single-dires dineeeeoietti one. With the help of latent feature correlation. BiGCN also enhances the network's 's tolerance to noisy graph signals and unreliable edge connections. Extensive experiments show that model achieves remarkable performance improvement noisy graphs
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Edward Anderson, Zhaojun Bai, Christian Bischof, Susan Blackford, Jack Dongarra, Jeremy Du Croz, Anne Greenbaum, Sven Hammarling, Alan McKenney, and Danny Sorensen. LAPACK Users' guide, volume 9. Siam, 1999.
R Bartels. Algorithm 432, solution of the matrix equation ax+ xb= Comm, Ass, Computer Machinery, 15:820-826 826, 1972.
Filippo Maria Bianchi, Daniele Grattarola, Cesare Alippi, and Lorenzo Livi. Graph neural networks with convolutional arma filters. arXiv preprint arXiv: 1e0:::001100300 2019.
Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends()) in Machine learning, 3(1):1-122, 2011.
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann Lecun. Spectral networks and locally connected networks on graphs. In International Conference on Learning Representations, 2014.
Jie Chen, Tengfei Ma, and Cao Xiao. Fastgen: fast learning with graph convolutional networks via importance sampling. arXiv preprint 11f1: 2018.
Siheng Chen, Aliaksei Sandryhaila, José MF Moura, and Jelena Kovacevie. Signal denoising on graphs via graph filtering. In 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 872-876. IEEE, 2014.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems, pp. 3844-3852, 2016.
Gene Golub, Stephen Nash, and Charles Van Loan. A hessenberg-schur method for the problem ax+ xb= c. IEEE Transactions on Automatic Control, 24(6):909-913, 1979.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems. pp. 1024-1034, 2017.
Elvin Isufi, Andreas Loukas, Andrea Simonetto, and Geert Leus. Autoregressive moving average graph filtering. IEEE Transactions Signal Processing, (5(2):274-288) 2016.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
Johannes Klicpera, Stefan Weißenberger, and Stephan Günnemann. Diffusion improves graph learning. Advances in Neural Information Processing Systems, pp. 13333-1335,, 2019.
Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for
semi-supervised learning. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network:
Data-driven traffic forecasting. arXiv preprint arXiv:100.001926, 2017.
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image- recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development Information Retrieval, pp. 43-52. ACM, 2015.
Galileo Namata, Ben London, Lise Getoor, Bert Huang, and UMD EDU. Query-driven active surveying for collective classification. In 10th International Workshop on Mining and Learning with Graphs, pp. 8, 2012.
Hoang NT and Takanori Maehara. Revisiting graph neural networks: All we have is low-pass filters. arXiv preprint /005:/9995005555 2019.
Kenta Oono and Taiji Suzuki. Graph neural networks exponentially lose expressive power for node classification. In International Conference Learning Representations, 2020.

---

Under review as a conference paper at ICLR 2021
Antonio Ortega, Pascal Frossard, Jelena Kovačević, José MF Moura, and Pierre Vandergheynst. Graph signal processing: Overview, challenges, and applications. Proceedings of the IEEE, 106 (5):808 828, 2018.
Jiahao Pang, Gene Cheung, Antonio Ortega, and Oscar C Au. Optimal graph laplacian regularization for natural image denoising. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2294-2298, IEEE, 2015.
Ryan Rossi and Nesreen Ahmed. The network data repository with interactive graph analytics and visualization. In Twenty-Ninth AAAI Conference Artificial Intelligence, 2015.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification network data. Al magazine, 29(3):93 2008
Oleksandr Shehur. Maximilian Mumme, Alcksandar Bojchevski, and Stephan Günneemann. Pitfalls of graph neural network evaluation. arXiv preprint
2018.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv: 10903, 2017.
WOK Asiri Suranga Wijesinghe and Qing Wang. Dfnets: Spectral cnns for graphs with feedback- looped filters. In Advances in Neural Information Processing Systems, pp. 6007-6018, 2019.
Felix Wu, Tianyi Zhang, Amauri Holanda de Souza Jr, Christopher Fifty, Tao Yu, and Kilian Q Weinberger. Simplifying graph convolutional networks. arXiv preprint lrXiv:190..07153, 2019.
Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. In International Conference on Machine Learning, pp. 5453-5462, 2018.
Keyulu Xu, Weihua Hu, Jure Leskovee, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference Learning Representations, 2019.
Xiaofan Zhu and Michael Rabbat. Approximating signals supported graphs. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3921-3924. IEEE, 2012.
Daniel Zügner and Stephan Günnemann. Certifiable robustness and robust training for graph convolutional networks. In Proceedings of the 25th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 246 256, 2019.
10

---

Under review as a conference paper at ICLR 2021
A MODEL EXPRESSIVENESS
In this section, we add more details about the our discussion of over-smoothing in Section 4.
As bi-directional low-pass filter, our model extract more informative features from the spectral domain. To simplify the analysis, let us take just one step of ADMM (k=1). Since 2u = 0, Y1 Y2 F, we have the final solution from Equation (10) follows
2 L,)F, I+P
Y1 (I
Y/2 = (T - (1+ p)²
2pd1
2 FT) 2pd1 T 2A2 I2) = F(I
12) 11-1+p I+p 11-1
(1 (1+ (1)
1 1+p
From this solution, we can see that Y1 a low-pass filter which extracts low-frequency features from the original graph via I1; Y/ is low-pass filter which extracts low-frequency features from the feature graph via L2 and then do some transformation. Since we take the average of Yı and Y2 as the output of ADM ( (H, L1, L2). the BiGCN layer will extract -ee-freuynncy features from both the graphs. That means, our model adds new information from the latent feature graph while not losing any features in the original graph. Compared the original single- directiaaat GCN, our model has more informative features and is more powerful in representation.
When we take more than one step of ADMM, from Equation (10) we know that the additive component 2ha
( 1+p L1 F) is always in Y1 (with scaling coefficient), and the component F(1 always in Y2. So, the output of the BiGCN layer will always contain the oow-freuenccy features from the original graph and the feature graph with some additional features with transformation, which can give us the same conclusion as the one step case.
I+p L2) is
B SENSITIVITY ANALYSIS
To demonstrate how peperpparameters (iterations of ADMM, X2, p and A) influence BiGCN, we take Cora as example and present the results on node classification under certain settings of artificial noise.
First, investigate the influence of iteration and A2 clean data and three noise cases with 0.2 noise rate, 0.2 noise level and 0.1% structure mistakes respectively. Fig 5[a]sshww that ADMM with 2 iterations is good enough and the choice of Xa has very little impact results since can be absorbed into the learnable L22 Then we take a particular case in which noise rate equals to 0.2 as an example illustrate how much the performance of BiGCN depends on p and X. Fig ((b) shows that p guarantees relatively stable performance over a wide range values and only À has comparable larger impact.
C FLEXIBLE SELECTION OF L2
our paper, we assume the latent feature graph Le as a learnable matrix and automatically optimize it. However, in practice it can also be defined as other fixed forms. For example, a common way to deal with the latent correlation is to use a correlation graph Li et al. (2017). Another special case model will degenerate to normal diretti-iir
is if we define L2 identity matrix, low-pass filtering GCN. When we take T_2 = / in Equation (5), the solution becomes
Y ((1 22)1 + L1)
which is similar to the iinle-direecional low pass filter (Equation (2)). Then the BiGCN layer will degenerate to the GCN layer as follows:
((+))) o(((1 + X111 -
To show the difference between different definitions of L2, design simple approach using a thresholded correlation matrix for L2 to compare with the method used in our main paper. In particular, we define an edge weight Aij follows.
11

---

Under review as a conference paper at ICLR 2021
clean
noise_r 0.2
noise J002
struc struc_mis--0 mi000
Cora
Cora
Accurac
0.80
0.80
0.75
2
2
4
Iteration
(a) Iteration and 22-
A=0.5
A=1
1 1.5
Cora
Cora (noise r=0.2
0.80
0.80
Accuracy
0.70
0.70
0.60
0.60
0.50
0
4
P
P
(b) p and A.
Figure 5: Sensitivity analysis of iteration, A2, and node classification. For iteration and Az, we conduct experiments on clean data and three noise cases with 0.2 noise rate, 0.2 noise level and 0. % structure mistakes respectively. For and X, we provide the performance of BiGCN on Cora with 0.2 noise rate.
xfaz Is IIIl Øz beNw)))
(Po))sEN(iju softmar(1
[0
Psj S mean( Ps mean(P)
Ag
[1.
Then we compute L2 the normalized Laplacian obtained from A, i.. Lg = For simple demonstration, we only compare the two models on Cora with node feature noises. From Table and Table we can see that our learnable L2 is overall better. However, a fixed L2 can still give us decent results. When the node feature dimension is large, fixing L2 may be more efficient.
Table 1: Node classification accuracy in noise rate case on Cora dataset of types of L2, Noise_rate
0.200
0.400
0.600
0.800
1,000
Fixed_L2
0.807
0.774
0.756
0.733
0.726
Learnable_Ly
0.802
0.785
0.770
0.745
0.734
Table 2: Node classification accuracy in noise level case on Cora dataset two types of L2.
Noise level
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
Fixed_Ia Learnable_L,
0.823
0.804
0.777
0.753
0.725
0.713
0.702
0.696
0.691
0.825
0.804
0.785
0.768
0.749
0.732
0.725
0.714
0.709
12

---

Under review as a conference paper at ICLR 2021
D EXPERIMENTAL DETAILS
We train a two-layer BiGCN as the same as other baselines using Adam as the optimization method with 0.01 learning rate, 5 x 10-1 weight decay, and 0.5 dropout rate for all benchmarks and baselines. In the node classification task, we use early stopping with patience 100 to early stop the model training process and select the best performing models based validation set accuracy. In the link prediction task, we use the maximum 100 epochs to train each classifier and report the test ROCAUC selected based the best validation set ROCAUC every 10 epochs. In addition, we follow the experimental setting from P-GNN (position-aware GNN) and the approach that we adapt GCN into link prediction tasks is consistent with the implementation in P-GNN. We set the random seed for each run and we take mean test results for 10 runs to report the performances.
All the experimental datasets taken from PyTorch Geometric and we test BiGCN and other baselines on the whole graph while in GDC, only the largest connected component of the graph is selected. Thus, the experimental results we reported of GDC maybe not completely consistent with that reported by GDC. We found that the Citation datasets in PyTorcch Geometric are little different from those used in GCN, GraphSAGE, and GAT. It may be the reason why their accuracy results on Citeseer and Pubmed in node classification tasks are little lower than the original papers reported.
To highlight the denoising capacity of the bi-directional filters, we design the following three cases and conduct extensive experiments on artificial noisy data. The noise level case and noise rate cases are adding noise on node features and the structure mistake case adds noise to graph structures.
Noise level case. In this case, we add different Gaussian noise with zero mean to all the node features in the graph, i.c. to the feature matrix and use the variance of Gaussian (from 0.1 to 0.9) as the quantitative indexes of noise level.
Noise rate case. this case, we add Gaussian noise with the same distribution to different proportions of nodes, i.c. some rows of the feature matrix, a random and quantitatively study how the percentage (from 10% to 100%) of nodes with noisy features impacts the model performances.
Structure mistakes case. In practice, it is common and inevitable to observe wrong or interference link information in real-world data, especially a large-scale network, such as a social network. Therefore, we artificially make random changes in the graph structure, such as removing edges or adding false edges by directly reversing the value of the original adjacency matrix (from 0 to 1 or from to symmetrically to obtain error adjacency matrix. We choose different scales of errors to decide how many values would be reversed randomly. For example. assigming a 0.01 % егтоr rate to a graph consisting of 300 vertices means that 0.01 10 3002 values symmetrically distributed in the adjacency matrix will be changed.
We conduct all of the above cases on five benchmarks in node classification tasks and the two previous cases on four benchmarks in link prediction tasks.
For more experimental details please refer our codes: sci wwweee///feeeeeeeeeeeeeeeeeeeeee
/ttps::/anonymous..u 421 a24 422-4344444-33-22224000ee- 322.4444422ee111e555555
D.I DATASETS
We use three Citation networks (Cora, Citeseer, and Pubmed) and two Co-purchase networks for node classification tasks and all the Citation datasets for link prediction.
Table 3: Bechmark Dataset. Nodes
Dataset
Type
Edges
Features Classes Label Rate
Cora
Citation
2,708
5,278
1,433
0.052
Citeseer
Citation
3,327
4,552
3,703
6
0.036
Pubmed
Citation
19,717
44,324
500
0.003
DBLP
Citation
17716
105734
1639
4
/
AMZ Comp
Co-purchase
13,752
245,861
767
10
0,015
AMZ Photos
Co-purchase
7,650
119,081
745
8
0.021
13

---

Under review as a conference paper at ICLR 2021
D.2 EXPERIMENTAL RESULTS ON CLEAN DATA
The performances of models on clean benchmarks in node classification and link prediction are shown in Table and 5 respectively. These results correspond to the values with noise level in the figures of Section
Table 4: BiGCN compared to GNNs on node classification tasks, measured in accuracy (%). Standard deviation errors are given.
Cora
Citeseer
PubMed
Comp
Photo
GCN
81.8 + 0.6
71.0 0.6
78.9 + 0.6
82.7 + 4,6
90.8 + 1.3
SAGE
82.3 + 0.5
7..5
78.5
83.1
4.2
90.8 + 1.1
GAT
$3.1
71.7
78.5
76.3
3.5
88.2 1 1.3
GIN
7..4
62.7
77.7 77.70.11
1:
37.1
12.0
GDC
83.0 0.6
70.7
84.5
1 0.8
89.7 0.4
BiGCN
83.1 1: 0.7
71.0 0.6
80.0 1: 0.3
87.0
0.6
92.6 0.3
Table BiGCN compared to GNNs on link prediction tasks, measured in ROC AUC (%). Standard
deviation errors given.
Cora
Citeseer
PubMed
DBLP
GCN
89.2 1 0.8
87.3 1 1.7
91.7 1 0.8
92.9 1 0.4
SAGE
9.7
89.7 = 0.7
91.8 0.3
92.6 0.2
GAT
8.8
87.3 =
92.6
93.1
GIN
0.7
90.1 1.3
0.67
9.
GDC
89.5 + 0.4
88.5 + 1.1
9.76
9
BiGCN
91.5 + 0.5
90.5 + 0.7
91.6 0.3
93.1
D.3 EXPERIMENTAL RESULTS ON AMZ CоMP
The node classification performances models on AMZ Comp dataset are shown in Fig
GCN
SAGE
GAT
GIN
GDC
BiGCN
Comp
Comp
Comp
0.60
0.60
0.40
0.40
0.40
0.00
0.25
0.50
0.75
0.0
0.5
1.0
0
500
1000
1500
Noise level
Noise
Structure mistake rate
Figure 6: Node classification accuracy of models on AMZ. Comp dataset.
E NUMERICAL RESULTS AND HYPERPARAMETERS
In order to facilitate future research to compare with our results, we share the accurate numeric results here in addition to the curves shown in the pictures of the Experimental section. We also share the experimental environment and the optimal hyperparameters we used to get the results in B.2.
E.
NUMERICAL RESULTS
E. NOISE RATE (NR)
Node Classification (NC)
14

---

Under review as a conference paper at ICLR 2021
Table 6: Cora
NR
NC
Table 7: Citeseer NR
NC
0.200
0,400
0.600
0.800
1.000
0.200
0.400
0.600
0.800
1.000
GCN
0.751
0.706
0.662
0.631
0.606
GCN
0.597
0.553
0.483
0.442
0,404
SAGE
0.768
0.717
0.685
0.656
0.645
SAGE
0.612
0.543
0.497
0.450
0.427
GAT
0.713
0.668
0.626
0.605
0.603
GAT
0.564
0.457
0.405
0.371
0.346
GIN
0.712
0,654
0.621
0.607
0.601
GIN
0.535
0.468
0.432
0.405
0.401
GDC
0,814
0.806
0.799
0.784
0.783
GDC
0.617
0.575
0.548
0,520
0.511
BiGCN
0.802
0.785
0.770
0.745
0.734
BiGCN
0.626
0.580
0.561
0.531
0.516
Table 8: PubMed NR
NC
Table 9: Computers NR
NC
0.200
0.400
0.600
0.800
1.000
0.200
0.400
0.600
0.800
1.000
GCN
0.550
0.466
0.434
0.422
0.385
GCN
0.837
0,832
0.810
0.839
0.830
SAGE
0.579
0.489
0.439
0.438
0.420
SAGE
0.846
0.831
0.836
0.840
0.838
GAT
0.491
0.477
0.467
0.465
0.449
GAT
0.770
0.800
0.769
0.766
0.750
GIN
0.568
0,505
0.482
0.490
0.478
GIN
0.420
0.402
0.402
0.394
0.406
GDC
0.560
0.474
0.427
0.412
0.404
GDC
0.840
0.837
0.832
0.838
0.832
BiGCN
0.665
0.619
0.604
0.567
0.547
BiGCN
0.856
0.855
0.855
0.853
0.853
Table 10: Photos NR
NC
0.200
0.400
0.600
0.800
1.000
GCN
0.913
0.908
0.907
0.905
0.894
SAGE
0.910
0.903
0,900
0.901
0.904
GAT
0.873
0.874
0.867
0.848
0.855
GIN
0.342
0.315
0.333
0.304
0.306
GDC
0.901
0.896
0.890
0.883
0.881
BiGCN
0.922
0.921
0.920
0.917
0.916
Link Prediction (LP)
Table 11: Cora NR
L.P
Table
Citeseer NR
L.P
0.200
0.400
0.600
0.800
1.000
0.200
0.400
0.600
0.800
1.000
GCN
0.850
0.817
0.795
0.792
0.785
GCN
0.812
0.773
0.754
0.739
0.726
SAGE
0.846
0.826
0.786
0.785
0.774
SAGE
0.824
0.787
0.749
0.740
0.732
GAT
0.848
0.817
0.781
0.785
0.767
GAT
0.807
0.765
0.747
0.738
0.741
GIN
0.827
0.799
0.799
0.785
0.780
GIN
0.819
0.772
0.758
0.757
0.747
GDC
0.872
0.860
0.853
0.847
0.840
DGC
0.808
0.779
0.758
0.764
0.756
BiGCN
0.887
0.875
0.851
0.845
0.843
BiGCN
0.867
0.836
0.812
0.800
0.804
Table 13: Pubmed - NR L.P
Table 14: DBLP NR
L.P
0.200
0.400
0.600
0.800
1.000
0.200
0.400
0.600
0.800
1.000
GCN
0.838
0.767
0.745
0.743
0.741
GCN
0.901
0.879
0.868
0.860
0.854
SAGE
0.844
0.797
0.770
0.763
0.755
SAGE
0.899
0.879
0.868
0.857
0.856
GAT
0.840
0.789
0.775
0.777
0.778
GAT
0.897
0.877
0.865
0.862
0.857
GIN
0.802
0.771
0.766
0.769
0.771
GIN
0.890
0.879
0.875
0.872
0.872
GDC
0.839
0.801
0.780
0.769
0.760
GDC
0.898
0.885
0.873
0.866
0.862
BiGCN
0.875
0.846
0.825
0.811
0.803
BiGCN
0.914
0.902
0.895
0.890
0.884
15

---

Under review as a conference paper at ICLR 2021
E.1.2
NOISE LEVEL (NL)
Node Classification (NC)
Table 15: Cora NL
NC
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
GCN
0.792
0.749
0.704
0.643
0.606
0.585
0.572
0.558
0.530
SAGE
0.791
0.758
0.720
0.677
0.646
0.628
0.616
0.603
0.599
GAT
0.782
0.737
0.688
0.644
0.622
0.608
0.601
0.584
0.579
GIN
0.734
0.686
0.659
0.617
0.610
0.580
0.595
0.581
0.577
GDC
0.828
0.819
0.805
0.792
0.784
0.772
0.763
0.754
0.758
BiGCN
0.825
0.804
0.785
0.768
0.749
0.732
0.725
0.714
0.709
Table 16: Citeseer NL
NC
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
GCN
0.671
0.598
0.500
0.443
0.418
0.403
0.382
0.382
0.367
SAGE
0.670
0.605
0.492
0.464
0.429
0.423
0,408
0.408
0.398
GAT
0.620
0.534
0.476
0.436
0.405
0.374
0.333
0.309
0.293
GIN
0.557
0.497
0.430
0.410
0.392
0.392
0.381
0.385
0.376
GDC
0.656
0.613
0.555
0.523
0.515
0.506
0.497
0.489
0.486
BiGCN
0.677
0.619
0.552
0.524
0.521
0.514
0.482
0.489
0.476
Table 17: PubMed NI. NC
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
GCN
0.530
0.433
0.406
0.379
0.373
0.382
0.381
0.377
0.383
SAGE
0.565
0.455
0.432
0.422
0.431
0.414
0.411
0.412
0.405
GAT
0.537
0.445
0.434
0.430
0.435
0.436
0.426
0.434
0.433
GIN
0.643
0.542
0.498
0.493
0.476
0.477
0.473
0.477
0.484
GDC
0.550
0.440
0.415
0.409
0.388
0.389
0.400
0.393
0.389
BiGCN
0.688
0.593
0.554
0.530
0.526
0.512
0.517
0.505
0.506
Table 18: Computers
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
GCN
0.844
0.846
0.845
0.846
0.825
0.805
0.836
0.831
0.829
SAGE
0.838
0.845
0.837
0.837
0.827
0.826
0.836
0.825
0.825
GAT
0.776
0.757
0.760
0.771
0.747
0.763
0.770
0.754
0.769
GIN
0.409
0.388
0.411
0.399
0.400
0.404
0.413
0.424
0.410
GDC
0.847
0.838
0.841
0.833
0.832
0.827
0.816
0.810
0.799
BiGCN
0.856
0.856
0.856
0.853
0.855
0.852
0.851
0.847
0.851
16

---

Under review as a conference paper at ICLR 2021
Table 19: Photos
NL
NC
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
GCN
0.915
0.907
0.904
0.903
0.906
0.901
0.899
0.894
0.893
SAGE
0.903
0,904
0.909
0.904
0.905
0.901
0.902
0.901
0.896
GAT
0.879
0.881
0.871
0.868
0.849
0.845
0.840
0.770
0.740
GIN
0.336
0.337
0.327
0.332
0.317
0.345
0.351
0.314
0.330
GDC
0,901
0.896
0.896
0.887
0.885
0.877
0.876
0.863
0.859
BiGCN
0.925
0.923
0.920
0.919
0.919
0.916
0.908
0.906
0.902
Link Prediction (LP)
Table 20:
Cora NL- L.P
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
GCN
0.857
0,814
0.795
0.779
0.780
0.774
0.764
0.770
0.763
SAGE
0.864
0.819
0.792
0.774
0.769
0.770
0.770
0.770
0.761
GAT
0.850
0.814
0.790
0.785
0.775
0.763
0.768
0.765
0.767
GIN
0.854
0.815
0.796
0.786
0.782
0.787
0.775
0.772
0.770
GDC
0.877
0.862
0.845
0.844
0.833
0.839
0.839
0.832
0.833
BiGCN
0.892
0.865
0.860
0.844
0.836
0.837
0.830
0.827
0.833
Table 21: Citeseer NL LP
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
GCN
0.807
0.760
0.743
0.736
0.725
0.726
0.713
0.714
0.713
SAGE
0.815
0.762
0.730
0.737
0.734
0.723
0.715
0.719
0.718
GAT
0.799
0.760
0.746
0.736
0.726
0.740
0.725
0.724
0.722
GIN
0.838
0.780
0.748
0.744
0.744
0.731
0.730
0.734
0.736
GDC
0.820
0.777
0.760
0.743
0.747
0.744
0.745
0.738
0.743
BIGCN
0.874
0.836
0.811
0.799
0.797
0.795
0.792
0.791
0.780
Table 22:
Pubmed NL L.
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
GCN
0.821
0.774
0.753
0.749
0.741
0.739
0.735
0.738
0.738
SAGE
0.813
0.774
0.761
0.753
0.751
0.747
0.749
0.750
0.752
GAT
0.820
0.779
0.773
0.770
0.770
0.770
0.770
0.770
0.772
GIN
0.806
0.782
0.772
0.768
0.769
0.770
0.773
0.775
0.771
GDC
0.828
0.785
0.758
0.755
0.753
0.750
0.745
0.745
0.748
BIGCN
0.841
0.814
0.806
0.804
0.801
0.799
0.800
0.800
0.796
Table 23: DBLP
NL- L.P
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
GCN
0.902
0.881
0.866
0.857
0.851
0.847
0.843
0.842
0.841
SAGE
0.900
0.876
0.863
0.859
0.858
0.856
0.851
0.850
0.849
GAT
0.899
0.880
0.868
0.863
0.860
0.856
0.854
0.853
0.850
GIN
0.896
0.885
0.878
0.874
0.872
0.869
0.865
0.867
0.865
GDC
0.904
0.883
0.871
0.865
0.859
0.854
0.853
0.850
0.847
BiGCN
0.916
0,900
0.894
0.890
0.884
0.883
0.883
0.880
0.881
17

---

Under review as a conference paper at ICLR 2021
E.1.3
STRUCTURE MISTAKES (SM)
Node Classification (NC)
Table 24:
Cora SM -NC
0.001
0.003
0.005
0.007
0.009
0.011
0.013
0.015
GCN
0.693
0.611
0.564
0.527
0.507
0.486
0.468
0.468
SAGE
0.697
0.606
0.558
0.519
0.515
0.500
0.483
0.473
GAT
0.678
0.565
0.509
0.458
0.435
0.400
0.356
0.342
GIN
0.529
0.302
0.261
0.247
0.242
0.244
0.275
0.261
GDC
0.723
0.659
0.634
0.619
0.611
0.604
0.598
0.599
BiGCN
0.720
0.682
0.659
0.645
0.643
0.637
0.631
0.635
Table 25:
Citeseer
SM NC
0.001
0.003
0.005
0.007
0.009
0.011
0.013
0.015
GCN
0.521
0.446
0.409
0.378
0.362
0.351
0.347
0.329
SAGE
0.520
0,446
0.411
0.375
0.369
0.363
0.336
0.337
GAT
0.498
0.397
0.344
0.299
0.282
0.272
0.254
0.231
GIN
0.397
0.221
0.211
0.210
0.201
0,204
0.206
0.205
GDC
0.529
0.515
0.515
0.511
0.516
0,514
0.518
0.526
BiGCN
0.601
0.577
0.565
0.567
0.565
0.565
0.562
0.567
Table 26:
PubMed SM
NC
le-5
2.5e-5
5e-5
7.5e-5
le-4
2.5e-4
5e-4
7.5e-4
le-3
GCN
0.770
0.760
0.745
0.730
0.719
0.683
0.659
0.647
0.640
SAGE
0.770
0.760
0.739
0.731
0.721
0.679
0.659
0.645
0.638
GAT
0.772
0.755
0.734
0.721
0.706
0.665
0.641
0.633
0.610
GIN
0.774
0.768
0.758
0.754
0.742
0.700
0.577
0.497
0,442
GDC
0.766
0.755
0.742
0.730
0.723
0.690
0.692
0.707
0.713
BiGCN
0.778
0.770
0.761
0.751
0.746
0.728
0.721
0.724
0.725
Table 27: Computers SM NC
0.001
0.003
0.005
0.007
0.009
0.011
0.013
0.015
GCN
0.782
0.723
0.632
0.548
0.519
0.472
0.471
0.457
SAGE
0.779
0.708
0.660
0.562
0.531
0.496
0.471
0.456
GAT
0.663
0.597
0.524
0.530
0.499
0.432
0.443
0.429
GIN
0.417
0.395
0.385
0.389
0.386
0.385
0.385
0.385
GDC
0.776
0.744
0.711
0.690
0.678
0.684
0.677
0.682
BiGCN
0.822
0.795
0.775
0.764
0.756
0.756
0.749
0.749
18

---

Under review as a conference paper at ICLR 2021
Table 28:
Photos
SM- NC
0.001
0.003
0.005
0.007
0.009
0.011
0.013
0.015
GCN
0.838
0.667
0.668
0.554
0.561
0.525
0.520
0.449
SAGE
0.855
0.815
0.762
0.754
0.710
0.638
0.592
0.478
GAT
0.820
0.681
0.512
0.432
0.376
0.372
0.356
0.319
GIN
0.266
0.283
0.261
0.262
0.261
0.260
0.267
0.255
GDC
0,871
0.843
0.818
0.787
0.764
0.748
0.748
0.734
BiGCN
0.902
0.880
0.867
0.856
0.851
0.844
0.838
0.833
E.2 ADDITIONAL IMPLEMENTATION DETAILS AND
SETTING
All implementations for both node classification and link prediction are based on PyTorch 1.2.0 and Pytorch Geometrid All experiments based PyTorch running RTX 2080 Ti GPU using CUDA. The experimental datasets are taken from the PyTorrch Geometric platform. We tune our hyperparameters for each model using validation data and listed the final optimal setting in the following tables. To accelerate the tedious process of hyper-parameter tuning, and choose different hyper- hyper-parameter p for different datasets.
NVIDIA GeForce
we set 24, 2Az
1+p
I+p
E.2.1
NODE CLASSIHICATION
Table 29: Hyper-parameters of BiGCN in Node Classification
Cases
Dataset
À
k
Hidden dimension
Layer
Dropout
Ir
Noise rate
Cora
1.8
Citeseer
1.8
PubMed
2
2
0.5
0.01
Comp
2.5
1.0
Photos
1.5
0.8
Noise level
Cora
1.8
Citeseer
1.8
PubMed
1.8
2
16
2
0.5
0.01
Comp
2.5
1.0
Photos
1.5
0.8
Structure mistakes
Cora
0.1
0.8
Citeseer
0.05
0.8
PubMed
0.1
0.8
2
16
2
0.5
0.01
Comp
0.1
1.0
Photos
0.1
1.0
19

---

Under review as a conference paper at ICLR 2021
E.2.2
LINK PREDICTION
Table 30: Hapeee-as of BiGCN Link Prediction
Cases
Dataset
p
X
k
Hidden dimension
Layer
Dropout
Ir
Noise rate
Cora
8.5
1.2
Citeseer
8.5
1.2
PubMed
8.5
1.2
2
32
2
0.5
0.01
DBLP
8.5
1.2
Noise level
Cora
8.5
1.2
Citeseer
8.5
1.2
PubMed
8.5
1.2
2
32
2
0.5
0.01
DBLP
8.5
1.2
20


Reviews:
[{"review_id": "mSc0lRXx5kb", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: \n\nThe authors proposed a to apply “low pass filtering” on both node and feature domain.\n\nPros:\n1.\tInteresting idea on trying to define and apply low pass filter on feature domain\n2.\tThe experiment results sort of validates the proposed idea.\n\nCons: \n1.\tThe definition of “feature graph” seems questionable. Moreover, the authors propose to learn the feature graph $L_2$ in the actual implementation. Compare to node domain where the graph topology is given by data, this asymmetric is very weird to me.\n2.\tThe motivation of applying additional filtering on feature domain seems redundant. Doesn’t the weight matrix $W^{(l)}$ in each layer work as the feature transformation already?\n\nDetailed comments:\n\nThe main weakness of the paper would be its motivation. Indeed, as the author claim, there might be some correlations among features. However, the weight matrices $W^{(l)}$ already serve as feature transformation for the $l^{\\text{th}}$ GCN layer. Why do we need additional “low pass filtering” in feature domain? If the methodology of this paper is correct, then why we still need the additional weighted matrix $W^{(l)}$ as in equation (11)? Also, even if we are given with some feature graph, why the low pass filtering defined with respect to this feature graph is reasonable? Note that the low pass filtering on graph is strongly related to the “Homophily principle”[1] which has been verified through extensive studies in network science community. However, it is not clear that the same conclusion will hold for the newly defined feature graph. Lastly, even in node domain the homophily principle is known to be invalid for some real graphs [2]. I think the authors should explain and discuss in more detail on this.\n\nConsider the case were we only have $1$ node with $2$ features $Y = [y_1,y_2]$. Furthermore, consider the node label function $f(y_1,y_2) = sign(y_2-y_1-1)$. Also, let $y_2 = 11y_1$ and $y_1$ is generated uniformly in range $[0,1]$. In this case, we can see that graph topology in node domain is irrelevant and we can focus on the feature domain. Also, the label function is positive with high probability (0.9) and the correlation coefficient between $y_1,y_2$ is $1$. According to the definition of feature graph (Section 4.2, page 4), the node $y_1$ and $y_2$ are linked and the corresponding low pass filter is just averaging $\\frac{y_1+y_2}{2}$ (up to a constant factor). Hence, the output feature matrix becomes closer to $[\\frac{y_1+y_2}{2},\\frac{y_1+y_2}{2}]$ as $\\lambda_2$ larger (equation (4)). Note that even if we know the optimal label function $f$, now we are more likely to predict negative label (with probability 1 when $\\lambda_2\\rightarrow\\infty$). In this example, it seems like applying low pass filtering in feature domain actually hurt the label prediction performance. What I try to convey here is that merely considering the correlation in node feature is insufficient. We have to also take the label correlation into account. Maybe I misunderstand something here but I hope the authors can elaborate on this.\n\nEven if we accept the idea that we really need some low pass filtering in feature domain, the definition of “low pass filtering” given by authors seems problematic to me. In node domain, since we are given the graph topology directly from data, we are able to define “frequency” based on the spectrum of graph Laplacian. In contrast, we are not given “feature graph” that characterizes the correlation between features. The author proposed to learn the graph Laplacian of the underlying feature graph $L_2$ but I doubt the correct $L_2$ can be learnt. Furthermore, the authors use different $L_2$ for each layer which is a bit counterintuitive to me. It is weird that the underlying “feature graph” topology will change across layers. Note that the graph topology remains the same for all layers in GNNs. Finally, instead of using this convoluted design, why not just apply multilayer perceptron (MLP) to do the job? It is known that MLP can approximate arbitrary function and thus should be able to approximate the underlying “low pass filter” for feature graph. I think the authors need to explain better on why the proposed design is necessary under their motivation.\n\nAs a final comment, I think the experimental results seems interesting. Indeed, the noise is injected artificially but it kind of verifies the idea of low pass filtering in feature domain for Gaussian noise case. The paper would be greatly improved if the concerns above can be addressed.\n\nReference:\n\n[1] “Birds of a feather: Homophily in social networks.,” McPherson et al., Annual review of sociology, 2001. \n\n[2] “Geom-GCN: Geometric Graph Convolutional Networks,” Pei et al., ICLR 2020.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "lPdDxyUlEAI", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The manuscript introduces a graph convolutional layer based on the optimal solution of the minimization problem of recovering the true graph signal given a noisy observation. Moreover, the authors propose to consider the original graph structure information together from the latent correlation between nodes features. The proposed solution is interesting and the paper is well written. Only section 4.2 turns out to be a bit difficult to read. My suggestion is to explain a bit more deeply the concept of latent features connection graph already in the introduction section.  Another point that it is not clear to me, is related to the fact that the authors highlight several times the connection between the proposed convolution and the low pass filter concept. In (NT & Maehara, 2019) the authors state that many papers on graph neural networks iteratively multiply the adjacency matrix and this operation corresponds to a low-pass filter. Therefore it is not clear to me the reason why the proposed convolution differs from the convolutions in the literature on this aspect.\n\nFor what concerns the experimental results it is interesting to notice that the BiGCN is significantly more robust than the models considered in the comparison it is not clear how the proposed model performs on clean data. The results reported in appendix D.2 show comparable performance with the other considered models. The problem is that methodology used in the comparison in my opinion is not completely fair. Indeed the authors in Appendix D state that they used the same hyperparameters (lr,  weight decay, and dropout) for all benchmarks and baselines, while the epoch was chosen based on the validation set. Honestly, I think that in this way the authors introduce a bias on the comparison that could affect the results. In this regards in appendix E2 the authors state “We tune our hyperparameters for each model using validation data” (to me it is not clear what the authors meaning with “our hyperparameters”) but the information about how the validation process is performed is missing (grid-search/random search?). It is also not clear if this procedure is used also for the baseline models (for instance, how the hidden dimension had been chosen?).\n\nIn the last part of section 5, the authors discuss the “Structure mistake case”. Since the authors consider also the semi-supervised node classification task, where dataset like Cora, Pubmed, and Citeser basically use a very small sub-graph as a training set, it is interesting to know how much the introduced incorrect interaction relationships among the nodes, interest the training/test/validation sets. In the paragraph about the “Noise rate case” a more deep discussion about the results in the Cora dataset should be inserted. In particular to me, it is not clear why on Cora the GCN outperforms the BiGCN (in Citeseer the performance is very similar), while in Pubmed GCN archives one of the lower accuracy. Is there any difference in the data that justify this behavior?\n\nIn my opinion, in the set of models considered in the comparison, the authors should also consider ARMA, which has the advantage to be more robust than the other models in the literature (as reported in section 2.2). I suggest also considering in the comparison some more novel models proposed in the literature e.g. the ones defined in “Break the Ceiling: Stronger Multi-scale Deep Graph Convolutional Networks” by Luan et al. (2019),  that obtained very interesting results in the considered datasets on semi-supervised nodes classification.\n\nMinor comments:\n-In section 3 the symbol \\lambda_i is used without been introduced it in advance;\n-in sections 3/4 the authors use the same symbol (L) for the  Laplacian Matrix and for the normalized Laplacian matrix.\n-please insert the complete appendix name when it is referred. For instance, in the last part of section 4, the authors state “ More technical details are added in Appendix”, without specifying which is the appendix referred to in this context.", "extra_scores": {}, "extra_text": {}}, {"review_id": "kTyHp3k0HE", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Pros:\n- In addition to encouraging the node embeddings to be smooth over the graph space, the paper further regularize the embeddings to be smooth among different features.\n- The method outperforms comparison methods when data is polluted with three types of random noise.\n- The paper is well-organized and clearly written. To my best knowledge, the method is technically sound.\n\nCons:\n- Compared with previous methods, the proposed method only achieves comparable accuracy performance on the real-world datasets.\n- The Gaussian noise is too weak to evaluate the robustness of the method.\n- It is better to give an analysis of the computational cost of Equ.(11) and provide an empirical speed comparison.", "extra_scores": {}, "extra_text": {}}, {"review_id": "gGSNR0ao6ss", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposed a new graph convolutional network. It considers not only the original graph structure information but also the latent correlations between features, resulting in a graph neural network as a bi-directional low-pass filter. The new filter is derived using the alternating direction method of multipliers (ADMM) algorithm. Experiments show the new model's denoising performance is better than previous models.\n\nPros:\n\n- The idea of using feature correlations in graph neural networks is interesting. This technique seems to improve the model’s denoising performance.\n\n- The new model outperforms previous graph neural networks on most of the benchmark datasets in different noisy settings.\n\nCons:\n\n- The ADMM algorithm is applied to solve the optimization problem (4). However, the convergence of this algorithm with Taylor approximations is not provided in this paper. In fact, there are simpler algorithms with convergence guarantees that can be applied to solve the Sylvester equation (5), e.g., the gradient based iterative algorithm in \"Gradient based iterative algorithms for solving a class of matrix equations\" by Feng Ding and Tongwen Chen.\n\n- All graph neural networks are compared when they have two-layers. However, in the new model, $L_2$ is a learnable symmetric matrix, which makes the new model more complex compared with other models. This comparison might be unfair to other models.\n\nBecause of the above reasons, I am leaning towards rejection. Below are some additional comments.\n\n1. On page 3, \"the smoothness of a graph signal $x$ can be measure through...\" should be \"... measured ...\".\n\n2. The notations in Figure 1 are not consistent with the notations in the paragraphs. It is better to make them consistent to avoid confusion.\n\n3. On page 4, \"$L$ is the (normalized) Laplacian matrix.\" It is better to give the definition of the normalized Laplacian matrix for completeness.\n\n4. On page 4, section 4.2, $L_2$ and $L'$ are used interchangeably. It is better to make them consistent.\n\n5. On page 5, \"all of them require Schur decomposition which including Householder transforms...\" should be \"... includes...\"\n\n6. On page 5, equation (6), why is the L2 term evenly split into $f(Y_1)$ and $g(Y_2)$? What will happen if we set $f(Y_1) = ||Y_1-F||_F^2 + \\lambda_1 trace(Y_1^TL_1Y_1)$ and $g(Y_2) = \\lambda_2 trace(Y_2L_2Y_2^T)$?\n\n7. On page 5, \"... by choosing appropriate hyper-parameters $p$...\" The matrix $L_2$ is learnable in the model, which makes it difficult to guarantee that the eigenvalues of $2\\lambda_2L_2/(1+p)$ all fall into $[-1,1]$.\n\n8. Please include the initializations of the ADMM algorithm.\n\n9. On page 6, \"we update $Y_1$ by appling the... then update $Y_2$ by appling the...\" should be \"... applying... applying...\"\n\n10. On page 6, \"It also explains that BiGCN is more expressive that single-direction los-pass filtering GCNs\" should be \"... than...\"", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: From the positive side the problem addressed by the paper could be of potential interest in the case there is noise in the features associated to each node of the graph. The paper is mostly well written and clear. The proposed approach is based on solid mathematical grounds.

On the other hand there are concerns about:

i) motivation: it is not clear how significant the proposed approach is since the authors were not able to clearly highlight the advantages with respect to the standard approach where already the weight matrix (via learning) can play the role of a low-pass filter for node’s features. Maybe the main advantage is given by the fact that the network does not have to learn a low-pass filter, however this needs a better clarification;

ii) suggested approach: the authors are using an approach that seems to be more complex with respect to simpler ones already proposed in literature and not mentioned in the paper. In addition to that, the simpler approaches have convergence guarantees that have not been proved for the proposed approach;

iii) significance of the experimental results: the obtained experimental results are obtained by using a model with more parameters with respect to the baselines. Comparisons versus baselines with a similar number of parameters are necessary to have a fair assessment of the merits of the proposed approach.

Decision: Reject