OPEN: Orthogonal Propagation with Ego-Network Modeling
Authors: Liang Yang
Lina Kang
Qiuliang Zhang
Mengzhe Li
Bingxin Niu
Dongxiao He
Zhen Wang
Chuan Wang
Xiaochun Cao
Yuanfang Guo
Venue: NeurIPS 2022 Accept (2022)
Keywords: Graph Neural Network
ego-network
propagation scheme

Abstract: To alleviate the unfavorable effect of noisy topology in Graph Neural networks (GNNs), some efforts perform the local topology refinement through the pairwise propagation weight learning and the multi-channel extension. Unfortunately, most of them suffer a common and fatal drawback: irrelevant propagation to one node and in multi-channels. These two kinds of irrelevances make propagation weights in multi-channels free to be determined by the labeled data, and thus the GNNs are exposed to overfitting. To tackle this issue, a novel Orthogonal Propagation with Ego-Network modeling (OPEN) is proposed by modeling relevances between propagations. Specifically, the relevance between propagations to one node is modeled by whole ego-network modeling, while the relevance between propagations in multi-channels is modeled via diversity requirement. By interpreting the propagations to one node from the perspective of dimension reduction, propagation weights are inferred from principal components of the ego-network, which are orthogonal to each other. Theoretical analysis and experimental evaluations reveal four attractive characteristics of OPEN as modeling high-order relationships beyond pairwise one, preventing overfitting, robustness, and high efficiency. 

OPEN: Orthogonal Propagation with Ego-Network Modeling
Liang Yang 1,", Lina Kang 1.", Qiuliang Zhang 1,*, Mengzhe Li Bingxin Niu 1
Dongxiao He 2.1, Zhen Wang 3.4 Chuan Wang Xiaochun Cao 9, Yuanfang Guo 7,8
School of Artificial Intelligence, Hebei University of Technology, Tianjin, China
"College of Intelligence and Computing, Tianjin University, Tianjin, China School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi'an, China
Schooo of Cybersecurity, Northwestern Polytechnical University, Xi'an, China State Key Laboratory of Information Security, IIE, CAS, Beijing, China
School of Cyber Science and Technology, Sun Yat-sen University, Shenzhen, China Schooo of Computer Science and Engineering. Beihang University, Beijing, China ZZhonguuaneun Laboratory, Beijing, China
yangliangevip.c qq com, itkanglina@163 com, 34631947840qq com Lmnengahefrre@@rroalllll com, miubingxin6666116.. com, hedongxiao@tjj. du. w w-zhen@nwpu. edu. cn, cat.://annnnccoiiiiccccuuaaaimaa ac. en, andyguo@buaa edu, cn
Abstract
To alleviate the unfavorable effect of noisy topology Graph Neural networks (GNNs), some efforts perform the local topology refinement through the pairwise propagation weight learning and the multi-channel extension. Unfortunately, most of them suffer common and fatal drawback: irrelevant propagation to one node and multi-channels. These two kinds of irrelevances make propagation weights in multi-channels free to be determined by the labeled data, and thus the GNNs are exposed to overfitting. To tackle this issue, a novel Orthogonal Propagation with Ego-Network modeling (OPEN) is proposed by modeling relevances between propagations. Specifically, the relevance between propagations to one node is modeled by whole ego-network modeling, while the relevance between propaga- tions multi-channels is modeled via diversity requirement. By interpreting the propagations to node from the perspective of dimension reduction, propaga- tion weights are inferred from principal components of the ego-network, which are orthogonal each other. Theoretical analysis and experimental evaluations reveal four attractive characteristics of OPEN as modeling high-order relationships beyond pairwise one, preventing overfitting, robustness, and high efficiency.
Introduction
Graph Neural Networks (GNNs) have been proven be powerful tool explore irregular graph data by seamlessly combining graph topology and node attributes for node representation learning 222. From the perspective of spectral graph theory, GNNs, such as GCN and ChebyNet [4] proposed from graph signal filtering, whose filters are derived from the topology of whole graph, and their success is attributed to low-passing filtering. From the perspective of spatial propagation, GNNs, such as GraphSage [5] and MPNN [6], are presented by following the aggregation and combination scheme for node attributes smoothing Recent progress demonstrates the equivalence between
"Equal contribution. 1 Corresponding author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

these two perspectives g However, both the filters based on graph topology and node attribute propagation between neighbourhoods take the implicit assumption that the graph topology perfect and creditable [
Consequently, the oversmoothing issue [TO, I1 and the expressive power loss 12 which are caused by overusing topology via stacking multiple layers, are identified.
Actually, the graph topology exists large amount of noises. Thus, graph topology refinement is critical to the accuracy and robustness of GNNs [ 14 5 [5). The local refinement, especially through pairwise propagation weight learning, is widely investigated and employed due to the high accuracy and low computation load. As a representative local refinement, Graph Attention Network (GAT) [[ formulates the propagation weight on each edge as the attention between the two connected nodes, and introduces multi-channel propagation via multi-head attention stabilize the learning process. Afterward, a number of approaches are proposed by following the scheme of propagation weight learning and multi-channel extension. For example, GaAN [7], PGCN [88, Masked-GGN [19] and DMP 20 extend GAT via different pairwise propagation weight learning functions. FAGCN [2] relaxes the non-negative constraint in GAT to a real number. ADSF
22 extends GAT to incorporate
topology structure into the attention mechanism. GATv2 X in GAT.
removes the ranking irrelevance limit
Unfortunately, most GNNs, especially those with local refinement, suffer a common but fatal drawback: irrelevant propagations. Firstly, the propagations to each node are irrelevant, since propagation weights are either predefined according to the topology or learned based on the contents of the two connected nodes. Actually, the propagations to one node should reflect global characteristics of its ego-network. Secondly, propagations in multi-channels are irrelevant, since parameters to channels are free to be learned without any specific constraints. Actually, to obtain stable and complementary representations from different channels, propagation schemes in different channels should be diverse [20] These two kinds of irrelevances make propagation weights free to be determined by the labelled data for the specific task, thus resulting GNNs being susceptible to overfitting
To tackle this issue, novel Orthogonal Propagation with Ego-Network modeling (OPEN) is proposed by modeling these two kinds of relationships between propagations. Firstly, the relevance between propagations to one node is modeled by whole ego-network modeling. Specifically, by interpreting the propagations to one node from the perspective of dimension reduction, propagation weights are inferred from the principal component of ego-network, which corresponds to the mapping function maximizing the variance and captures the characteristies of the whole ego-network. Principal compo- nent can be obtained via the eigenvalue decomposition of the covariance matrix, which is efficient for the ego-network. Secondly, the propagations multi-channels are implemented by employing all principal components, each of which corresponds to one channel. Since principal components are orthogonal to each other, the propagations in multi-channel are diverse. The proposed OPEN possesses some attractive characteristics: modeling high-orde relationships beyond pairwise one, robustness, and high efficiency. Finally, theoretical analysis demonstrates that the two components of OPEN, i.e., Ego-Network modeling and Orthogonal Propagation in multi-channel, can prevent the over-smoothing issue.
The main contributions of this paper are summarized as follows:
We investigate the common but fatal issue in GNNs, i.e., irrelevant propagation.
We propose a novel Orthogonal Propagation with Ego-Network modeling (OPEN) by interpreting propagations in the ego-network from the perspective of dimension reduction.
We provide a theoretical analysis of OPEN's capability on preventing oversmoothing issue.
We experimentally verify the effectiveness and robustness of the proposed OPEN.
2
Preliminaries and Analysis
Notations: Let G (V.E) denote graph with node set V = {U1, 82, uNn and edge set E, where N is the number of nodes. The topology of graph G can be represented by its adjacency matrix A [a:y] E {0, x where aij- = 1 if and only if there exists an edge Eij- = (v;, v2) between nodes i and tiy. The degree matrix is diagonal matrix with diagonal element d, = =N aij as the degree of node v. N(ee)) = (uyl(v,, v7) E E} stands for the neighbourhoods of node 24
2

---

X R F and H E R F denote the collection of node attribute and representation with the tt row, i.e., E R and h; IR,,, corresponding to node 11 where F and F' stand for the dimensions of attribute and representation.
Preliminaries: Most Graph Neural Networks (GNNs) follow an aggregation-combinatio strategy [6. where each node representation is iteratively updated by aggregating node representations of neighbourhoods and combining the aggregated representation with the node representation itself as follows
AGGGEGGTEE ((h. i E (()))) h COMBINATE" (h.
(I)
where h stands for the aggregated representation from neighbourhoods. The aggregation operation is the most critical part of the message passing framework, and most GNNs utilize summarization or average function as the implementation of AGGREGATE< Therefore, most GNNs can be unified under the following formula
h
Σ BEN(c)
k h ue u
W lk
where W is the learnable parameter and the ar-) is the nonlinear mapping function. The scalar Cuv is the averaging weight, which determines the scheme of aggregation. GNNs can be divided into two categories according to the design of Cuv. The methods in the first category fix Cuv by regarding topology information as perfect.For example, GCN B and SGC [25]
set ck =
1/(ydd 1) + 1). while GIN [26] sets e = for u * v and cy I1 ck The methods in the second category tend to learn Cue by considering topology as noisy. For example, GPRGNN p7 sets d > //v//V// (dm, + Idd, + 1) with 7* learnable real number. Graph Attention Network (GAT) [66. Gated Attention Network (GaAN) TV and Probabilistic GCN [18] model the propagation weights as the function of the attributes of connecting nodes via normalized attention mechanism as
Cuv softmar(ewe) exp(exe)/
Σ kEN(u)
exp(euk).
where Eue denotes the similarity between nodes v and u. The similarity function of attributes can be specified as eu LeakyReLU(b and e
Wh ( "D, euv 5h 1 E(" k- Wh where b.. O* and * are learnable parameters. Some efforts have been paid simplify them, such setting 0 identity matrix, i.c. 0 = I or constraining * as diagonal matrix. FAGCN (21] relaxes the non-negative constraint in GAT to real number by employing tanh(-) instead of softmax.
PGCN = (Wh,, k-1
GAT [T6] proposes to improve the stability by employing multi-heat attentions as Transformer (28) The multi-head attention essentially performs multi-channel propagation. DMP 20 formates multi- channel propagation as diverse message passing by enhancing Eq. ? via feature-wise propagation weights
h = a
Σ MEN(M)
h u (
where @ denotes the element-wise product of vectors and the learnable propagation weight vector et has the same length as the node representation h To reduce the model complexity, DMP presents two schemes to efficiently learn ct..'s 's.
Analysis: By analyzing the single-channel propagation in Eq. 4 and the multi-channel propagation in Eq. 4 can be observed that existing methods possess two serious drawbacks as shown in the blue dashed box in Fig.
Propagations to each node are irrelevant. As shown in Eq. B. the propagation weight, which is based on the contents of the two connected nodes, models pairwise relationship. Thus, the different propagations to each node are irrelevant to each other. However, the node representation, which is obtained from aggregation over neighbourhood, should reflect global characteristics of its neighbourhood, such high-order relationship.

---

Original Graph
Propagation Weicht
Node Eibbeddiig
Inter-channel Irrelevant
Irre Invéevent Propapations
Ego-network Extraction
Irrele
X
Propagation Perspective
Pairnise Learsins
Learnalle
X
Dimension Reduction Perspective
Mepping en Whele F Neiwert
Whele Ego-network
Orilogaaa
Prppgstton with
Rgo-Ncoooo
Pappse
Mapping
Data
Multi-chaman
Function
Point
Figure 1: Comparisons between existing GNNs with irrelevant propagations and the proposed OPEN (Orthogonal Propagation with Ego-Network modeling). The essential difference between them is the different perspectives on ego-network modeling. Upper Part: Existing GNNs are from the perspective of propagation with pairwise propagation weights learning, and thus both inter- channel and intra-channel propagations are irrelevant. Lower Part: The proposed OPEN is from the perspective of dimension reduction of data points, and thus mapping functions are from whole ego-network modeling and orthogonal between different channels.
Propagations in multi-channels are irrelevant. As shown in Eq. (4), propagations in different channels are free to learn without any specific constraints. Thus, the propagation weights in different channels may be similar and redundant. However, to obtain stable and complementary representations from different channels, propagation schemes in different channels should be diverse enough.
In summary, neither the relationship between the propagations to each node nor the relationship between propagations in different channels is considered. Thus, propagation weights free to be determined by the labelled and the specific task, and thus the models tend to be overfitting 24]
3 Methodology
To tackle the issues mentioned above, Orthogonal Propagation with Ego-Network modeling (OPEN) is proposed by modeling these two kinds of relationships between propagations in subsections 3.] and 3.2 respectively. Section 3.3 provides implementation details. Finally, section 3.4 theoretically analyzes its capability preventing over-smoothing issuc.
3.1 Ego-Network Modeling
Firstly, the relationship between propagations to one node modeled. By ignoring the layer l and considering the first layer, the propagation component in Eq. 2 can be reformulated as
h Σ
b, = c., H.,
(5)
(EN(w)
where H, R. stands for the matrix containing the representations of nodes in No and C, (g represents the propagation weights from nodes in Nr to node v. Eq. 5 can be interpreted from two perspectives as shown in Fig. [ From the perspective of message passing, the representation of node v, i.e.. h, the aggregation of embeddings its neighbourhood, which are the rows of H., and propagation weights often model pairwise relationship as shown in the blue solid box in Fig. M This perspective induces the propagations to each node irrelevant as shown in the blue dashed box in Fig. 1
From another perspective, h, I can be regarded as the one-dimensional representations of F data points, each of which corresponds to a -idddinnnsion column vector of H And, c, E R(W can be seen the mapping function, which reduces the dimension from IR lWal to R as shown in the orange solid box in Fig. D From the perspective of dimension reduction, the representations h, should preserve the discriminative information in original [N, |dimensiooal space of H,.. Therefore,

---

learning the propagation weights in ego-network of node v is converted to the problem of seeking dimension reduction mapping function.
Among the existing dimension reduction methods, principal component analysis (PCA) is the widely- adopted unsupervised one. The principal components correspond to the mapping functions. To facilitate the description, the subscripts of H, and Cv are removed, and H is represented as the collection of column vectors
{h.,1, 2, h.,F,, each of which is an [N -dmmmnniiooall vector. By denoting the mean and covariance matrix of data matrix H as h = + Eÿ= h. and
= i hi - h)", respectively, the principal components are the cigenvectors of covariance matrix S, i.c
Su, Xyu j 1- [.Cl
where u, is the cigenvector corresponding to the eigenvalue Xy shown in the orange dashed box in Fig. By sorting AÚ in descending order, the eigenvector u1 corresponding to largest eigenvalue λ is the first principal component. By using the u the mapping function c. the obtained 1-dimensional representation possesses the largest variance, and thus preserves the discriminative ability.
Ego-network of Hub Nodes:
The computational complexity, which consists of construction of S and EVD, ( F). Thus, it is efficient most nodes with small neigbourhood. However, it may be inefficient for nodes with large neighbourhood, such as hub nodes. Thus, we present the treatment for them. By denoting H (h,1 h, h.2 h..., h.FF h). Eq (6) can be written as HH' u, uy By multiplying both sides by H", it holds
H HH u = AT N
By denoting z; H uj Eq.
and uj can be respectively reformulated as
H 2j Azzj, uj
(FX2)² Hz,.
1
Since + t H E FxF, its construction and EVD are efficient. Thus, the complexity of ego-network modeling for hub nodes, which possess large number of neighbourhoods, i.e., WW,l reduced from O(W, O(NG Therefore, ego-network for hub node can be efficiently modeled.
This new perspective and corresponding ego-network modeling provide following three attractive characteristics:
Beyond pairwise modeling: Since the covariance matrix S E [& essentially models the correlation between nodes in the ego-network Nr, its eigenvector represents the high-order characteristics of ego-network, which is beyond the pairwise relationship between two connected nodes. Therefore, the propagations to each node are jointly modeled.
Preventing overfitting issue: The mapping function, i.e., the propagation weights, is inferred from the ego-network instead of learning from labelled nodes for specific task. The model parameters of propagation weight learning are omitted, and thus it prevent overfitting.
Hight efficiency: The eigenvalue decomposition (EVD) is performed over the ego-networks whose sizes are much smaller than that of whole network, and thus the EVD is efficient.
3.2 Orthogonal Propagations in Multi-channels
Since multi-channel introduces multiple groups of trainable parameters, it tends to be more serious overfitting than the single-channel case. To tackle this issue and enhance the robustness, intuitive strategy is to boost the diversities of different channels to provide complementary effects. GNNs may implement this requirement by designing diverse propagation scheme. However, difficult to constrain pairwise weight learning to be diverse in an efficient manner, since diversity often requires impose computationally expensive orthogonality constraints.
Fortunately, the employed PCA in ego-network modeling in section 3.1 simultaneously produces [&l orthogonal components (u,, uz..... u u,A,). }. which corresponds different mapping functions. Note that the PCA is efficient ego-networks with small number of nodes. Thus, these [.
5

---

orthogonal components {ui, ug,... 4|N can be employed as the propagation weights in [ different channels to realize the diversity requirement. By letting c uj as the propagation weights over the ego-network of node e on jth channel, Eq. 5 can be extended to multi-channel form as h! H. j 1,2, [Vv.
Discussion: Ortho-GConv also introduces the orthogonality in GNNs. However, it is very different from the proposed OPEN both motivation and methodology. From the perspective of motivation, Ortho-GCom tends to alleviate oversmoothing issue, while OPEN aims at overfitting issue. From the perspective of methodology, Ortho-GConv imposes orthogonality on feature transformation W*, while OPEN presents orthogonal propagation, which is more challenging and essential in GNNs.
3.3 Model Details
Subsections and provide the basic ideas of ego- ego-network modeling and extension to multi- channels with orthogonal propagations. This section presents the details on implementation.
Firstly, the number of channels is set as J in advance. Since the number of principal components for each ego-network the same as the number of neighbourhood of the center node, i.e., [W lus these numbers varies over nodes. To make representations of nodes comparable, the number of channels set as a hyper-parameter For the nodes, whose number of neighboood less than J, the lacking components set as zero vector, i.e., c/. = 0 for j = [ e J. The tuning experiments on J are shown subsection 4.5
J.
Secondly, the final representation of node can be obtained by combing representations in multi- channels from Eq. such as summarization or concatenation. To boost the expressive capability, attention mechanism is employed to combine different representations with shared learnable parameter b, i.c.
cxp T h))
h, = a af
(10)
EL cxp(b
Similar to existing GNNs, OPEN is trained by feeding the final representation h into the cross- entropy between predicted and ground-truth labels on labelled nodes for semi-supervised task.
Thirdly, propagation weights inferred from the original node attributes are fixed for all layers. Although it can obtain optimal solution by inferring propagation weights for the next layer from the representations obtained in last layer, the inference process and the back-propagation process can't 't be seamlessly combined. For the consideration of computational complexity, propagation weights inferred from the original node attributes are employed for all layers.
3.4 Theory Analysis on Preventing Over-smoothing
wver-smothhing issue 10 i.c., representations of all nodes converge to points independent from their original attributes, has been identified as the most serious issue to prevent GNNs from being deep. The over-smoothing also leads to the exponential loss of expressive power for node classification 12 According to [II. over-smoothing issues is caused by that the eigenvector corresponding to the largest eigenvalue of normalized adjacency matrix (A₃ 1) is determined by degree, i.e., u1 De for asymmetric normalization or u, Dže for symmetric normalization, where is the vector of ones. Thus, the node representations after infinite layers are
lim A'x = /n x = un (u = in u.w".
(1I)
+00
where w = T u is the weighted combination of attributes from all nodes, and thus is shared by all nodes. Therefore, the final representation H is determined by node degree D. To alleviate this issue, many efforts have been paid. Some of them modify the topology, such as DropEdge (30 and GRAND 31 while others revise the outputs from layers, such as PairNorm
32 and Inflation [
Different from existing methods, the proposed OPEN essentially refines the topology according to the attributes of nodes in ego-network X., i.e., A X). Therefore, it is intuitive that the ú, corresponding Å the composition D and X. According to Eq. Q11), the final representation relevant to X, i.c. preventing moot-img issue. The following theorem is proved in Appendix.

---

Table 1: Statistics of datasets used for node-level tasks.
Dataset
#Nodes
#Edges
#features
#Classes
Cora
2,708
5,429
1,433
7
Citeseer
3,327
4,732
3,703
6
Pubmed
19,717
44,338
500
3
Amazon-Computerr
13,752
245,861
767
10
Amazon-Photo 7,650
119,081
745
8
Coauthor-CS 18,333
81,894
6,805
15
Coauthor-Physiie 34,493
247,962
8,415
5
Theorem 1. The representation of each node from OPEN relevant the principal components of its corresponding ego-network's 's attribute.
4 Evaluations
Firstly, this section provides experimental setups, including dataset, baselines and implementation details. Then, the node classification results are analyzed followed by dyper-parameet tuning. Finally, the capabilities preventing over-smoothing and overfitting verified.
4.1 Datasets
To comprehensively evaluate the proposed OPEN, 7 widely used datasets are employed. Statisties of datasets are shown in Table These datasets can be divided into three categories.
Citation Networks. Cora, Citeseer, and Pubmed, which are widely used to verify GNNs, are standard citation network benchmark datasets [34] 35]. In these networks, nodes and edge represent papers and citations between them, respectively. Words in the paper are employed to represent the node feature in lag-of-word oord form. The academic topic of paper is taken as the label of node.
Co-purchase Networks. Amazon-Computeri (Computers) and Amazon-Phote (Photo) are two networks of co-purchase relationships B3 In these networks nodes represent goods and edges stand for the connected two goods being frequently bought together. Each node owns bag-of- soee-of-ordds feature extracted from product reviews. The categories of the goods are employed as the label of node.
Coauthor Networks. Coauthor-CS (CS) and Coauthor-Physti (Physics) are two co-author networks based on the Microsoft Academic Graph from the KDD Cup 2016 challenge [36] In these networks nodes represent authors and edges stand for that the connected two authors have co-authored paper. Each node owns ofg-of-words feature based on paper keywords. The most active research fields of the authors are employed the label.
4.2 Baselines
To verify the superiority the proposed OPEN, 15 baseline methods are employed for performance comparison. These methods are divided into two categories. The first category consists of basic methods for graph data, including the multiple layer perception (MLP), Logistic Regression (LogReg)
and Label Propagation (LP) [7], Chebyshev
Graph Convolutional Network (GCN) 3 Graph
Attention Network (GAT) [1.. GraphSAGE (5) Simple Graph Convolution (SGC) 225 MoNet (3).
The methods in the second category possess some attractive characteristics, such as preventing over-
smoothing issue and high accuracy, etc This category contains Personalized Propagation of Neural Predictions (APPNP) [39] Jumping Knowledge Networks (JKNet) 44), GCN with Initial residual and Identity mapping (GCNII) 41,, Generalized PageRank GNN (GPRGNN) 27,, Diverse Message Passing (DMP) 2 and Constrained GAT (C-GAT) (24. We employ the authors' implementations for all baseline methods with the default
7

---

Table 2: Node classification performance in terms of micro-fl scores.
dataset
Cora
Pubmed
Citeseer
Computer
Photo
CS
Physics
MLP
74.82+2222
63.76=00.78
74.05=2.10
70.48+0.28
78.69=0.30
88.30==0.00
88.90=11.10
LogReg
70.10:12.30
61.00:12.20
71.10=3.10
76.80:1.700
79.20=6..00
86,40-10.00
86.70=11.50
LP
78.00=:0.20
75.30=0.20
69.00=0.50
70.80=0.00
67.80=0.00
74.30=0.00
90.20=0.50
Chebyshev
82.20.10.50
81.8010.50
73,40.10.30
72.6010.00
84.30_10.00
91.50.10.00
92.10.10.30
MoNet
82.30.11.30
80.20:1..00
74,60-12.30
88.60112.20
91.20-11.30
90.80-10.60
92.50-10.90
GCN
85.77+00.25
18.13-00.28
73.68+0031
86,51+0.554
92.42-0.22
94,55+0022
95.58+
GAT
87.37.10.30
0..62.10.26
74.32.10.27
86.93.10.39
92.56.10.38
93.98
95.63.10.18
GraphSAGE
87.77=1.04
88.42-00.50
1..09+1..00
83.11=0.23
90.51=0.25
OOM
95.52-=0.54
SGC
86.20+0.12
87.50++0.25
78. 10.113
00.09
88.25=0.52
89.62++0.22
90.05+0.550
GONII
88.49.12.78
89.57
77.08 11.21
86. 86.13:10.51
90,98 1.99
93.88.10.44
96.02.10.15
APPNP
87.87=0.885
89.40=0.661
76.53+11.33
81.99+0266
91.11=0.26
94.92==0.00
95.84-0.22
JKNet
88.93++1.35
87.68+0.33
14.37+1.53
77.80+0097
87.70=0.70
92.32++0.72
95.71+0.21
C-GAT
88.40=1..30
87 0..30
79,90=0.30
OOM
0OM
OOM
OOM
GPRGNN
18.37=1133
89.05=0.552
18.8
89,43=0.86
94,34=0.35
94.76=0.20
96.39=0.20
DMP
86.6811.13
89.67110.58
16.48
87.6211081
94.06=00:45
95.02:10.29
94,74.10.20
OPEN
89.31=10.53
90.05
80.42:10.72
90.93-10.29
95.43=0.27
94.9920.13
96.92=0005
Cora
Citeseer
Pubmed
92
OPEN
GAT
GAT
89
Accuracy
Accuracy
78
Accuracy
OPEN GAT
88
76
84
K/H
K/H
Figure 2: The impact of the number of channels the performance Cora, Citeseer and Pubmed.
4.3
Implementation Details
The proposed OPEN employs 2-layers network with K = 5 channels except for the hyper-parameter tuning (Sec. 4.5) and over-smoothing investigation 4.6). The whole network is trained in an end-to-end manner using the Adam optimizer with an initial learning rate of 0.001. The maximum number of epochs is set up to 1000. Besides, early stopping with a patience of 50 is also utilized. For all datasets, randomly split nodes of each class in to 60%, 20% and 20% for training validation and testing, and run on test sets over 10 random splits, as suggested 12 For fair comparison, the performance of all the methods, including baseline methods, are obtained on the same splits.
4.4 Result Analysis
The node classification results are shown in Tab. 2 The proposed OPEN achieves the new SOTA on 6 networks in all 7 networks. Note that the proposed OPEN not only outperforms all basic GNNs, such as GCN and GraphSAGE, but also significantly beats almost all SOTA methods, such as GCNII, GPRGNN and DMP. These demonstrate the superiority of the proposed OPEN on classification accuracy. This can be attributed to both the effectiveness of ego-network modeling on capturing high-order semantic information and the high diversity representations obtained from orthogonal propagation in multi-channels. Besides, the proposed OPEN also outperforms other extensions to GAT, such C-GAT and DMP. This indicates that compared other enhancements to the propagation weights learning and multi- colti-channel learning, ego-network modeling and orthogonal propagation are more effective. summary, these effectivenesses show that the modeling the irrelevance between propagations is important and necessary for GNNs.
4.5
Tuning and Ablation Study
The only
in the proposed OPEN is the number principal components, K, which also corresponds to the number of channels. This section investigates the impact of K on classification

---

Cora
Citeseer
Pubmed
100
80
70
Accuracy
Accuracy
60
70
80
30
60
50
OPEN
OPEN
OPEN
40
GAT
30
GAT
GAT
GCN
GCN
GCN
layers
lavers
layers
Figure 3: Node classification results with various model depths on Cora, Citeseer and Pubmed.
Cora
Citeseer
Pubmed
100
100
OPEN GAT
OPEN
Accuracy(
95
90
Accuracy(
GAT
Accuracy(%
95
GAT
85
70
85
65
80
60
75
Adding Ratio(%)
20
50
Adding Ratio(%)
10
20
50
70
Adding Ratio(%)
10
20
50
Figure 4: Node classification performance on graphs with randomly adding noisy edges.
accuracy. To exclude the influences from different datasets, the impact of the number of channels in GAT, i... the number of attention heads, also investigated. Shows that OPEN consistently outperforms GAT all the number of channels. However, the trends with different number of channels are different on different datasets. In most cases, the superiority OPEN is more remarkable with large K compared GAT. Thus, the K is set 5 for other experiments by balancing the accuracy and running time.
The hyper-pase seper-paraeeter tuning experiments also enable the ablation study. 1) When K 1, only the ego-network modeling component works. The significant outperformance compared to GAT demonstrates that the relevant propagation in OPEN is more superior than irrelevant propa- gations in GAT. 2) The consistent outperformances on K's reveal the importance of orthogonal propagations in multi-channels, which also relevant. The ablation study illustrates that both relevance modelings are critical and necessary.
Ablation Study:
4.6 Preventing Over-smoothing Issue
Section 3.4 theoretically analyzes the capability of OPEN on preventing oversmoothing issue. This section provides experimental evaluations. The node classification performance changes of GCN, GAT and OPEN with various model depths are shown in Fig. GCN tends to be over-smoothing via only few layers, since it employs topology-induced single-channel propagation. GAT alleviates the overmoothing issue by introducing irrelevant multi-channel propagations. Unfortunately. also be oversmoothing after few layers. By enhancing the GAT with relevant multi-channel propagations, OPEN significantly prevents the over-smoothing issue. After many layers, the performances of OPEN do not remarkably drop. This matches the theoretical results in Section 3.4 and verifies the capability of relevant propagation on preventing oversmoothing issue.
4.7 Robustness
This section investigates the robustness proposed OPEN, which indicates whether the OPEN is able to overcome the overfitting problem, as in 24 To this end, experiments are conducted by randomly perturbing the edges the testing graph. Specifically, a set of nodes are randomly selected according to given sampling ratio, and then one edge is randomly added these nodes.

---

Fig. reports the classification performance on testing graphs with different ratios of nodes. The performances of GAT significantly drop the ratios of nodes increase, which indicate tends to be overfitring to the given data. In contrary, the performance drops of the proposed OPEN are very slight. It indicates the robustness of OPEN. This robustness may be attributed to two aspects. Firstly, the propagation weights of OPEN are inferred from neighbourhoods instead of learning from labelled data. Secondly, the orthogonal propagations multi-channels promote the diversity, and thus enhance the robustness. This investigation shows that relevant propagation modeling can promote the robustness of the GNNs.
5 Conclusions
This paper identifies the irrelevant propagations issue in Graph Neural Networks (GNNs), which makes models exposed to overfitting to the labelled data. The propagation irrelevance includes 1) the propagations to node irrelevant, 2) propagations in multi-channels are irrelevant. This paper presents a novel Orthogonal Propagation with Ego-Network modeling (OPEN) to model these two kinds of relevances between propagations. By interpreting the propagations to node from the perspective of dimension reduction, propagation weights are inferred from the principal components of ego-network, which are orthogonal each other. Theoretical analysis and experimental evaluations reveal four attractive characteristics of the proposed OPEN as modeling high-order relationship beyond pairwise one, preventing overfitting issue, robustness, and high efficiency. These attractive characteristics reveal the importance of modeling propagation relevance.
Acknowledgments and Disclosure of Funding
This work was supported part by the National Science Fund for Distinguished Young Scholars under Grant 62025602, in part by the National Natural Science Foundation of China under Grant 61972442, Grant 62102413, Grant U1936210, Grants U1803263, Grant 11931015 and Grant 62276187. in part by the Key Research and Development Project of Hebei Province of China under Grant 20350802D and 2031 1802D; in part by the Natural Science Foundation of Hebei Province of China under Grant F2020202040, in part by the Natural Science Foundation of Tianjin of China under Grant 20ICYBJC00650, part by the China Postdoctoral Science Foundation under Grant 2021 M703472, in part by the XPLORER PRIZE, and in part by the Fundamental Research Funds for Central Universities.
References
[1] Zonghan Wu. Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip Yu. comprehensive survey on graph neural networks. IEEE Trans. Neural Networks Learn. Syst., 32(1):4-24, 2021.
[2] Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. Al Open, 1:57-81, 2020.
[3] Thomas N. Kipf and Max Welling. semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017.
[4] Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems 29; Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 3837-384 3845, 2016.
[5] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 1024-1034, 2017.
10

---

[6] Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning, ICMI. 2017, Sydney, NSW, Australia, 6-11 August 2017, pages 1263-1272, 2017.
[7] Liang Yang. Chuan Wang, Junhua Gu, Xiaochun Cao, and Bingxin Niu. Why do attributes propagate in graph convolutional neural networks? In AAAI, pages 4590-4598. AAAI Press, 2021.
[8] Muhammet Balcilar, Guillaume Renton, Pierre Héroux, Benoit Gaüzère, Sébastien Adam, and Paul Honeine. Analyzing the expressive power of graph neural networks in spectral perspective. In International Conference on Learning Representations, 2021.
[9] Liang Yang, Zesheng Kang, Xiaochun Cao, Di Jin, Bo Yang, and Yuanfang Guo. Topology optimization based graph convolutional network. In Proceedings of the Twenty-Eighth Interna- tional Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 4054 4061, 2019.
[10] Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018. pages 3538 3545,, 2018.
[11] Meng Liu, Hongyang Gao, and Shuiwang Ji. Towards deeper graph neural networks. In KDD 20: The 26th ACM SIGKDD Conference Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 338-343 2020.
[12] Kenta Oono and Taiji Suzuki. Graph neural networks exponentially lose expressive power for node classification. In 8th International Conference Learning Representations, ICL. 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
[13] Ruijia Wang, Shuai Mou, Xiao Wang, Wanpeng Xiao, Qi Ju, Chuan Shi, and Xing Xie. Graph structure estimation neural networks. In WWW, pages 342-353, 2021.
[14] Yixin Liu, Yu Zheng, Daokun Zhang, Hongxu Chen, Hao Peng, and Shirui Pan. Towards unsupervised deep graph structure learning. In WWW, pages 1392-1403, 2022.
[15] Liang Yang, Fan Wu, Junhua Gu, Chuan Wang, Xiaochun Cao, Di Jin, and Yuanfang Guo. Graph attention topic modeling network. In Yennun Huang, Irwin King, Tie- Yan Liu, and Maarten van Steen, editors, WWW, pages 144-154, ACM/ IW3C2, 2020.
[16] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings, 2018.
[17] Jiani Zhang. Xingjian Shi, Junyuan Xie, Hao Ma, Irwin King, and Rit-Yan Yeung. Gaan: Gated attention networks for learning on large and spatiotemporal graphs. In UAI, pages 339-349, 2018.
[18] Liang Yang, Yuanfang Guo, Junhua Gu, Di Jin, Bo Yang, and Xiaochun Cao. Probabilistic graph convolutional network via topology constay-c latent space model. IEEE Trans. Cybern., 52(4):2123-2136, 2022.
[19] Liang Yang, Fan Wu, Yingkui Wang, Junhua Gu, and Yuanfang Guo. Masked graph convolu- tional network. In Sarit Kraus, editor, IJCAI, pages 4070-4077 ijcai.org, 2019.
[20] Liang Yang, Mengzhe Li, Liyang Liu, Bingxin Niu, Chuan Wang, Xiaochun Cao, and Yuanfang Guo. Diverse message passing for attribute with heterophily. In NeurIPS, pages 4751-4763, 2021.
[21] Deyu Bo, Xiao Wang, Chuan Shi, and Huawei Shen. Beyond low-frequency information in graph convolutional networks. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Event, February 2-9, 2021, pages 3950-3957, 2021.
11

---

[22] Kai Zhang, Yaokang Zhu, Jun Wang, and Jie Zhang. Adaptive structural fingerprints for graph attention networks. In ICLR, 2020.
[23] Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks? In ICLR, 2022.
[24] Guangtao Wang, Rex Ying, Jing Huang, and Jure Leskovec. Improving graph attention networks with large margin-based constraints. arXiv preprint vrXiv:1900.19945 2019.
[25] Felix Wu, Amauri H. Souza Jr., Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Q. Wein- herger. Simplifying graph convolutional networks. In Proceedings of the 36th International Conference on Machine Learning, ICML. 2019, 9-15 June 2019, Long Beach, California, USA, 2019.
pages
[26] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
[27] Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized pagerank graph neural network. In International Conference on Learning Representations, 2021.
[28] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pages 5998-6000 2017.
[29] Kai Guo, Kaixiong Zhou, Xia Hu, Yu Li, Yi Chang, and Xin Wang. Orthogonal graph neural networks. In AAAI, volume abs/21 14438, 2022.
[30] Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. Dropedge: Towards deep graph convolutional networks on node classification. In Sth International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
[31] Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Kharlamov, and Jie Tang. Graph random neural networks for semi- cumi-supereised learning on graphs. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 12, 2020, virtual, 2020.
[32] Lingxiao Zhao and Leman Akoglu. Pairnorm: Tackling oversmoothing in gnns. 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
[33] Dongxiao He, Rui Guo, Xiaobao Wang, Di Jin, Yuxiao Huang, and Wenjun Wang. Inflation improves graph neural networks. In WWW '22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25 29, 2022, pages 1466-1474, 2022.
[34] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi- Rad. Collective classification in network data. Al magazine, 29(3):93 93, 2008.
[35] Galileo Namata, Ben London, Lise Getoor, and Bert Huang. Query-driven active surveying for collective classification. In International Workshop on Mining and Learning with Graphs, 2012.
[36] Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation, 2019.
[37] Xiaojin Zhu, Zoubin Ghahramani, and John D. Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Machine Learning, Proceedings of the Twentieth International Conference (ICMI. 2003), August 21-24, 2003, Washington, DC, USA, pages 912-919, 2003.
[38] Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Jan Svoboda, and Michael M. Bronstein. Geometric deep learning on graphs and manifolds using mixture model cmns. In CVPR, pages 5425-5434, 2017.
12

---

[39] Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
[40] Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. In Proceedings of the 35th International Conference Machine Learning. ICML 2018, Stock- holmsmässan, Stockholm, Sweden, July 10-15, 2018, pages 5449-5458, 2018
[41] Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, pages 1725-1735, 2020.
[42] Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom geom-gen: Geometric graph convolutional networks. In 8th International Conference on Learning Repre- sentations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [No]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [No]
4. If you using existing assets (e.g., code, data, models) curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, applicable? [N/A]
(b) Did you deseribe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "uASakr7i2-R", "reviewer": "Reviewer_GFKV", "summary": "The paper tackles the problem of irrelevant propagation to one node and in multi-channels in GNNs via modeling relevances between propagations. The proposed method is called OPEN for Orthogonal Propagation with Ego-Network modeling. The basic idea relies on dimension reduction through principal component analysis to infer the propagation weights. Theoretical and empirical evidence suggests the effectiveness of the proposed method. ", "questions": "The proposed method relies mainly on principal component analysis, while many other dimension reduction techniques exist in the literature (e.g., kernel PCA). The authors should elaborate more on this aspect by comparing different dimension reduction strategies and comparing them to the proposed scheme.", "limitations": "The theoretical analysis (section 3.4) is not consistent. Indeed, the intuitions behind Theorem 1 are not clear and should be clarified more precisely by the authors. It would be more interesting to provide a theoretical quantitative evaluation of the proposed method to prove its effectiveness in theory. ", "rating": 5, "confidence": 1, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- The paper studies one of the main issues in GNNs; irrelevant propagation and proposes a simple yet effective method to tackle this issue.\n- Extensive empirical evaluations are carried out and show the effectiveness of the proposed method.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "nc7xo7neaF", "reviewer": "Reviewer_aMxP", "summary": "In this paper, the authors propose a new GNN model (OPEN) to tackle the irrelevant propagations issues.\nThe authors first made observations on two limitations of traditional GNNS: \n * The propagation weights to each node are computed in a pairwise fashion, which ignores certain global information for that node's neighborhood.\n* There are no constraint on the channel specific propagation weights, which could potentially lead to overfitting.\n\nIn OPEN, the propagation weights to a node is computed by eigenvalue decomposition (EVD) of the covariance matrix of node representations in ego-network. It is claimed that this would encode global information from ego-network and prevent overfitting is supervised learning.\n\nSince most of the ego-networks are much smaller in size, the EVD can be computed efficiently (with special treatments for Hub nodes).\n\nAlso, by design, the propagation weights from different channels are orthogonal in OPEN, therefore it encourages diversity and reduces overfitting risk.\n\nEmpirical results show that the proposed architecture outperforms other GNNs on several node classification tasks.", "questions": "\n* Can the authors provide more context on how the propagation weights are incorporated in back-propagation? \n* Why the inference process and the back propagation process can't \"be seamlessly combined\" for layer specific propagation weights?\n* I'd suggest to re-phrase the narrates in Theorem 1 to be more specific. Even over-smoothed representations are \"relevant to node attributes\", just not related to the node itself.", "limitations": "The authors discussed about connections and differences between OPEN and other GNNs with orthogonal constraint. Limitations such as inability to use layer specific propagation weights are discussed.\n\nTo my knowledge the work poses no potential negative societal impact.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper is well presented and easy to follow. The authors first unifies the expression for different GNNs and formulate the problem as finding the right propagation weights. Then they introduce their proposed method via the dimension reduction point of view.\n\nThe empirical results shows the proposed model outperforms other baseline GNNs on various node classification tasks. Ablation study also shows that it can indeed prevent over-smoothing issue. \n\nEven though the method itself is not complicated, I think this could give us a good insight on the limitations of existing GNN architectures.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "G3J05gZm_MQ", "reviewer": "Reviewer_NBUs", "summary": "\nThis submission studies local refinement in GNNs via orthogonal propagation with ego-net modeling. The proposed methods are designed to address the problem of irrelevant propagation.  Specifically, the relevance between propagations to one node is modeled by whole ego-network modeling and eigenvector decomposition. Meanwhile, the propagations in multi-channels are implemented by employing all principal components, each of which corresponds to one channel. Experimental results on multiple real-world datasets were provided.", "questions": "1. it is claimed that the algorithm has high efficiency because evd with respect to the neighborhood is less expensive than evd of the entire network. However, the original propagation in GNN does not require any evd, and such evd needs to be computed for egonets of each node for OPEN. Hence, the efficiency claim is not justified. More explanation or experiments in terms of run time need to be provided\n\n2. the algorithm flow is hard to follow. An algorithm table would be helpful to improve clarify\n\n3. The difference between this work and Ortho-GConv [26] is not clear. In the discussion it is mentioned that \"Ortho-GConv tends to alleviate oversmoothing issue, while OPEN aims at overfitting issue.\" but the entire paper as well as the theoretical analysis shows OPEN sovles overfitting issue. This needs to be clarified.\n\n4. why orthogonal propagation is more challenging than orthorgonizing features as claimed? It seems to the reviewer they are similar.\n\n", "limitations": "societal impact not discussed", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1. the design motivation of viewing propagation as dimensionality reduction is new and very interesting. \n2. experiments on real datasets show the effectiveness of the algorithms compared with multiple existing works\n\nWeakness:\n1. the clarify and organization of the paper can be improved\n2. there are typos and grammar mistakes in the paper.\n3. the discussion of novelty compared with previous works is not clear", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept