Noise-robust Graph Learning by Estimating and Leveraging Pairwise Interactions
Authors: Xuefeng Du
Tian Bian
Yu Rong
Bo Han
Tongliang Liu
Tingyang Xu
Wenbing Huang
Yixuan Li
Junzhou Huang
Venue: Accepted by TMLR ()

Abstract: Teaching Graph Neural Networks (GNNs) to accurately classify nodes under severely noisy labels is an important problem in real-world graph learning applications, but is currently underexplored. Although pairwise training methods have demonstrated promise in supervised metric learning and unsupervised contrastive learning, they remain less studied on noisy graphs, where the structural pairwise interactions (PI) between nodes are abundant and thus might benefit label noise learning rather than the pointwise methods. This paper bridges the gap by proposing a pairwise framework for noisy node classification on graphs, which relies on the PI as a primary learning proxy in addition to the pointwise learning from the noisy node class labels. Our proposed framework PI-GNN contributes two novel components: (1) a confidence-aware PI estimation model that adaptively estimates the PI labels, which are defined as whether the two nodes share the same node labels, and (2) a decoupled training approach that leverages the estimated PI labels to regularize a node classification model for robust node classification. Extensive experiments on different datasets and GNN architectures demonstrate the effectiveness of PI-GNN, yielding a promising improvement over the state-of-the-art methods. Code is publicly available at https://github.com/TianBian95/pi-gnn.

Xuefeng Du Published in Transactions on Machine Learning Research (06/2023)
Noise-robust Graph Learning by Estimating and Leveraging Pairwise Interactions
afdu wisc. edu
University of Wisconsin- Madison
Tian Bian"
thion@se.. cuhk. edu.hk
The Chinese Universiby of Hong Kong
Yu Rong Tencent
golrongghhtmaail.om
Bo Han
Mhanml@@comp.bbbuu..de
Hong Kong Baptist University
Tongliang Liu
tongliang l6u www.:iiuuddddedddddeeeeooe edw.
Mohamed bin Zayed University of Artificial Intelligence The University of Sydney
Tingyang Xu Tencend AI Lab
ww.rong@hhttmail.com
Wenbing Huang
hwenbing@:122 com
Renmin University of China
Yixuan Li University of Wisconsine-Madionn
sharonki@cs..o www.s....ssc................. edu
Junzhou Huang
jehuang@euta. edu
University of Tezas at Arlington
Reviewed on OpenReview: https: // openr/viewwww... net/ forum? id= 76kkFEAAQ
Abstract
Teaching Graph Neural Networks (GNNs) to accurately classify nodes under severely noisy labels is an important problem in real-world graph learning applications, but is currently underexplored. Although pairwise training methods have demonstrated promise in supervised metric learning and unsupervised contrastive learning, they remain less studied on noisy graphs, where the structural pairwise interactions (PT) between nodes are abundant and thus might benefit label noise lcarning rather than the pointwise methods. This paper bridges the gap by proposing pairwise framework for noisy node classification on graphs, which relies the PI primary learning proxy in addition to the pointwise learning from the noisy node class labels. Our proposed framework PI-GNN contributes two novel components: (1) a confideence-wware PI estimation model that adaptively estimates the PI labels, which are defined as whether the two nodes share the same node labels, and (2) decoupled training approach that leverages the estimated P1 labels to regularize a node classification model for robust node classification. Extensive experiments on different datascts and GNN architectures demonstrate the effectiveness of PI-GNN, yielding promising improvement over the state-of- the-art methods. Code is publicly available at https: http//ggttttubb hom.:/ianiiannn//piiiiaaa
Xuefenn Tian contributed equally this work. Work is done while interning Tencent.

---

Published in Transactions on Machine Learning Research (06/2023)
1 Introduction
Graphs are ubiquitously used to represent data in different fields, including social networks, bioinformatics, recommendation systems, and computer network security. Accordingly, graph analysis tasks, such node classification, have significant impact in reality (Xu et al. 2019 The success of machine learning models, such graph neural networks (GNNs) node classification relies heavily on the collection of large datasets with human-annotated labels (Zhou et al. 2019). However, it is extremely expensive and time-consuming to label millions of nodes with high-quality annotations. Therefore, when dealing with large graphs, usually a subset of nodes labeled, and a wide spectrum of semi-supervised learning techniques have emerged for
improving node classification performance (Zhu et al. 2003 Zhou et al. 2003 Kipf & Welling 2017).
Although achieving promising results, these techniques overlook the existence of noisy node labels. For instance, practitioners often leverage inexpensive alternatives for annotation, such combining human and machine-generaled label (Hu et al. 2020), which inevitably yields samples with noisy labels. Since neural networks (including GNNSs) are able to memorize any given (random) labels (NT et al., 2019 Zhang et al, 2017), these noisy labels would casily prevent them from generalizing well. Therefore, training robust GNNs for semiisuperrised node classification againsl noisy labels becomes increasingly crucial bul less studied for safety-critical graph analysis, such as predicting the identity groups of users social networks or the function of proteins to facilitate wet laboratory experiments, etc.
Noise ratio=50% Nodes with noisy class labels
Nodes with clean class labels
x
x1
X2
X3
X4
X5
X6
X7
X8
x
X1
X2
X3
X4
X5
X6
X7
X8
y
1 1
2
2
3
3
4 4
y 1 2 2
3
3
4
4
1
Nodes with clea PI labels
Nodes with noisy PI labels
X1
X2
X3
X4
X5
X6
X7
Xg 4
X1
X2
X3
X4
X5
X6
X7
X8
1
1
2
2
3
3
4
1
2
2
3
3
4
4
1
X1
1
1
1
0
0
0
0
0
0
X1 1
1
0
0
0
0
0
0
1
X2
1
1
1
0
0
0
0
0
0
X2
2
0
1
1
0
0
0
0
0
X3
2
0
0
1
1
0
0
0
0
X3
2
0
1
1
0
0
0
0
0
X4
2
0
0
1
1
0
0
0
0
x4 3
0
0
1
1
0
0
0
X5
3
0
0
0
0
1
1
0
0
X5
3
0
0
0
1
1
0
0
0
X6
3
0
0
0
0
1
1
0
0
X6
4
0
0
0
0
0
1
1
0
X7
4
0
0
9
0
0
0
1
1
X7
4
0
0
0
0
1
1
0
Xg
4
0
0
0
0
0
0
1
1
Xg
1
1
0
0
0
0
0
0
1
Noise ratio-2%%%
Figure 1: Noise ratio comparison with noisy node labels. The noise ratio of PI labels is much smaller than that of node labels. Number in red denotes noisy label.
In this paper, we pioneer pairwise framework for noise-robust node classification on graphs, where relationships between data points are exploited. Currently, although the pairwisc approachos are prevailing
and made great progress in supervised metric learning and unsupervised contrastive learning [Qi et al. 2020:
Boudiaf et al. 2020 Chen et al.) 2020a He et al., 2020). they remain largely unexplored in noise-robust graph learning. In particular, existing pointwise noiso-robust learning algorithms (NT et al.] 2019 Xia et al. 2021
Li et al. 2021a Dong et al. 2020 Li et al.) (2000a) are mainly designed for image inputs and strictly rely on the class label that shows the class that node belongs to for learning. In contrast, the pairwise framework is
2

---

Published in Transactions on Machine Learning Research (06/2023)
able to utilize the pairwise interactions (PI) between nodes, which indicate whether or not two nodes belong to the same class, as a learning proxy. As result, it reduces the multi-class classification problem into a binary classification problem, which is easier to handle Patrini et al. (2017) and provides helpful learning signals apart from the noisy pointwise supervision. For example, Figure shows the transformation from the class labels to the PT labels. We can easily observe that the noise rate for the PI labels is much lower than that of the pointwise noisy class labels. Considering two nodes from the same class have the same noisy labels, their PI label still remains positive, which is helpful for the model to learn.
Although learning with PI intuitively demonstrates promise, it does not trivially transfer to label noise learning on graphs. For example, previous pairwise learning frameworks (Qi et al. 2020 Chen et al. 2020a) can easily calculate the PI labels either through class label comparison (same class label-xpositive PI label) or data augmentation (augmented views from the same image-
PI label). However, PI labels can still contain unneglectable noise (ef. Figure D if we directly compare their noisy node class labels. As a result, the pairwise learning algorithm relying on such suboptimal PI labels can misbehave.
We propose novel framework dubbed PI-GNN, tackling two highly dependent problems PI estimation and learning in one synergistic framework. Concretely, PI-GNN contributes two novel components: (1) We introduce an end-to-end confidence-awarg PI label estimation branch that dynamically estimates PI labels with the help of graph structure (Section 3.1 In particular, we learn a graph neural network that is trained to predict node connectivity, where the connected nodes have a ground truth of 1 and vice versa. Compared to using node counectivity as the PI label directly, i.e. connected nodes transform to a positive PI label, we derive PI labels with the predictive confidence from a PI label estimation network to quantify the reliability of such graph structure. (2) We explore novel decoupled training approach by leveraging the estimated PI labels for learning a node classification model to perform noise-robust node classification (Section 3.2 We propose to decouple the PI label estimation procedure from training with noisy node labels to prevent corruption on the estimated PI labels. Meanwhile, different from previous works Li et al. (2021b), our PI-GNN does not require a clean set of node and label pairs as extra supervision and can simultancously utilize both the labeled and unlabeled nodes for training, which works well for semi-supervised node classification.
Our main contributions are summarized as follows:
We propose to train robust GNNs against noisy labels for node classification, which serve as a crucial step towards the reliable deployment of GNNs in complex real-world applications.
We introduce a novel learning framework to simultaneously estimate and leverage the pairwise interactions, which can be applied on both labeled and unlabeled nodes without extra supervision of clean node labels.
We demonstrate PI-GNN can be effectively used on different datasets, GNN architecturos and different noise lypes and rales, e.g., improving the test accuracy by 5.4% on CiteSeer under severe label noise.
2 Preliminaries
Graph Neural Networks. Let G (V, A) be a graph with node fcature vectors X,, for v E V and edge set E, where A denoles the adjacency matrix. GNNs use the graph structure and node features X,, to learn a representation vector of node hy or the entire graph ha, which usually follow a neighborhood aggregation stratogy and iteratively update the representation of node by aggregating representations of its neighbors. After le iterations of aggregation, a node's representation captures the structural information within its k-hop network neighborhood. Formally, the k-th layer of a GNN is
(k) AGCREGATE (4) ((1l--) E ((w))) An(%) (OMBINE( (1!--);a!")
(1)
where h(()) is the feature vector of node u at the k-th layer. N X,, NV(1) denotes the neighboring nodes of V. The choices of AGGREGATE (R) (-) and COMBINE (.) can be diverse among different GNNs. For
There are some additional cases where the conclusion might hold true, please see Appendix Section
3

---

Published in Transactions on Machine Learning Research (06/2023)
he
T
G= (A,
he
PI label
X
Le Che hes Ustructure)
estimation
f
PI label ypi
ht
T
ht X
Lrog (hi x his UPI) Lets (ht, y
Node classification ft
The framework of proposed PI-GNN, which consists of different branches, i.c., PI label estimation branch and a node classification branch for noise-robust semi-supervised node classification. The two branches fe, fo first estimate the pairwise interactions between each node pair by the graph structure, and then leverage the estimated PI labels for joint training with the node classification task. x denotes the dot product operation.
Figure 2:
example, in GCN Kipf & Welling (2017), the element-wise mean pooling is used, and the AGGREGATE and COMBINE steps are integrated as follows:
" h (K) = ReLU (W- MEAN [ (k-1), Yu E())
(2)
where W learnable matrix. For node classification, each node o V has an associated label yo, the node representation (K) of the final layer is used for prediction.
Label-noise representation learning for GNNs. Let X., be the feature and ye be the label for node v, we deal with dataset D = = Dte} which consists of training set D" = ((A F.) that is drawn from a corrupted distribution 7 = p(,, X,) where denotes the corrupted label. Let p(A, X, Y) be the non-corrupted joint probability distribution of features X and labels y, and f be the (Baycs) optimal hepothesis from to y. To approximate f", the objective requires a hypothesis space H of hypotheses robust algorithm against noisy labels contains the optimization policy to search through H. in order to find 6* that corresponds to the optimal function in the hypothesis for D" ff-- E H, and meanwhile is able to assign correct labels for Dtc.
fo(-) parametrized by
3 Proposed Approach
In this section, we introduce our proposed PI-GNN, which performs noise-robust. semi-supervised node classification by explicitly estimating and leveraging the pairwise interactions on graphs. In what follows, we will first provide method overview and then illustrate the confidencc-aware estimation of the pairwise interactions in PI-GNN (Section 3.1 We introduce the decoupled training strategy for leveraging the estimated pairwise interactions for model regularization (Section 3.2
Overview. Figure demonstrates the overview of PI-GNN, which is composed of two different branches. The confidence-aware PI label estimation branch takes in the graph structure and generates the estimated PI labels for node pair. We denote it as fe- The node classification branch takes the estimated PT labels and trains a node classification model jointly with an additional regularization objective, which leverages the labels regularize the node embeddings. We denote as ft.
3.1 Confidence-aware Label Estimation
Let us suppose certain node class labels in the training set D" labels. Since, ultimately, we are interested in finding GNN model f parametrized by that minimizes the generalization error on a clean test set D natural solution exploit additional information during training for the learning algorithm to find robust parameter 0* in the hypothesis space H. One
{(A, v.)lvev are corrupted with noisy
4

---

Published in Transactions on Machine Learning Research (06/2023)
straightforward candidate for such information is leveraging the pairwise interactions between two nodes to perform extra regularization, whose learning objective is shown to hold much smaller noise rate than that. with the noisy class labels (Figure D
Train the PI label estimation model. In this paper, we propose to estimate the PI labels yPl E V|x[M] between node pairs by learning from the graph structure (Here [V] is the cardinality of the vertex set on the input graph G). While reasonable choice of ypt is by comparing whether two nodes have the same class label y and assigning those with the same class label positive PI label, it is impossible to obtain such PI labels with noisy class labels y.
Therefore, we propose to learn from the graph structure A for estimating the PI labels using a PI label imiiaaioo model fe with paprameter Oe, such as a graph neural network. Specifically, given node embeddings h which is calculated by = fe(A, X, 0e), assume i.j E V, let h hy be the dot product of two node embeddings, the training objective for the PI label estimation model Le E RIVIxIVI is formulatod as follows:
Le Ch; = A B [- log 1 + expx ff-
1
log expx des
+ E(s)e
(3)
+ exp A
where B+, denote the node pairs that are connected and disconnected, respectively. À is applied for the disconnected node pairs deal with the sample imbalance problem, which can be calculated according to Kipf & Welling (2016 2017).
Estimate the PI labels. Given well-trained PI label estimation model fe, derive the PI labels by taking the predictive confidence as the smoothed PT label, which is calculated as follows:
exp* AxpATA h,
yri(i,5)
(4)
1 exp Af
Algorithm 1 PI-GNN: Noisc-robust Graph Learning by Estimating and Leveraging Pairwise Interactions
Input: Input graph G= XV) with noisy training data D" {(A, X ge)lyev randomly initialized
GNNs Je and fi with parameter Og and O,, weight for regularization loss B, pretraining epoch K for fe- Total
training epoch N.
Output: Robust GNN ft.
for epoch = 0: epoch < NN; + + do if epoch < K then
Update the parameter Og of the P1 label estimation model fe by Equation 3
Set B = 0 in Equation 5 update the parameter 0, of the node classification model ft-
elsc
Update the parameter 0g of the PI label estimation model fe by Equation 3 Estimate the PI label yPI by Equation | with fe. Update the parameter 0g of the node classification model ft by Equation
end
end
return The node classification model fc.
where (ir(i,) is proportional to the value of the dot product hi hy. The estimated PI label measures the predictive confidence by looking at the closeness between the prediction and the graph structure (i.e., node connectivity). If the predictive confidence becomes far away from the binary node connectivity (0 or 1), then the reliability of such predictions is relatively low, which results in a smoothed PI label and vice versa. We show in Section 4.5 the ablations of training with different kinds of PI labels, including training with the PT labels by comparing the clean node labels.
The PI label estimation procedure allows for explicitly exploiting the pairwise interactions between two nodes, resulting in GNN that is affected less by the noisy class labels. Wu et al. (2021) employed similarity labels
5

---

Published in Transactions on Machine Learning Research (06/2023)
for regularization. However, it transforms the noise transition matrix estimated from the noisy class labels y to correct. the similarity labels, which is sensitive to the matrix estimation quality. Meanwhile, their approach did not deal with the noise-robust graph learning problem.
3.2 Decoupled Noise-Roban Robust Training
Given the well-estimated PI labels, now we discuss how to leverage the Pf labels for noise-robust training. One simple solution is to train a single GNN to estimate the PI labels, leverage the estimated PT labels by replacing the binary labels Ystructure with the estimated smoothed PI labels and then perform regularization by joint training with the node classification task on that single GNN, However, since the GNN is exposed to the noisy node class labels y, the PT label yrs cannot be estimated well if we entangle both the node label prediction and P1 label estimation èn single GNN.
Decoupling with two branches. In this paper, as shown in Figure 2 we propose to decouple the PI label estimation and node classification by using two separate GNNs, which are referred as a PT label estimation model fe and a node classification model ft- The PI label cstimation model generates the predictive confidence yPT by only learning with the PI estimation objective (Equation B). The node classification model uses the generated PI label yPI from the PI label estimation model at the same time for model regularization.
Overall training. Put them together, introduce new noisc-robust training objective for node clas- sification against noisy labels on GNNs, leveraging the estimated PT label in Section 3.1 The key idea is to perform the node classification task by the node classification model ft while regularizing ft to produce similar emboddings for nodes that have a larger PI label and vice versa. Different from Equation 3 that uses the discrete labels and 1, we use the smoothed PI label as the learning target. The overall noise-robust training objective for the node classification branch ft. is formulated as:
L, = Lels L
+ -reg;
(5)
where B is a hyperparameter to balance the node classification loss Lots and the regularization loss Lreg using the estimated PI labels. Coneretely, frez defined as follows:
- (1-11-pp) vPI(i,])) log
expx1
Creg A -(E
+ hN yPI(i.j) log
1 + exppi--y
(6)
1
exp ty
+ ((s))t
- - (--((,,(,))) log
1 + +exp A, vPr(i.)) log
1 + exp A bn
where ypr (i, is defined Equation
Besides, the Pl label estimation model trained only by the binary classification loss C. (Equation 3) and provides the estimated PI labels for model regularization on the node classification model, which is not allected by the noisy class labels. During inference, we discard the PI label est.mmation model fa and only use the node classification model ft for evaluation, which does not affect the inference speed.
Practically, the learning procedure relies heavily on the quality of the PI label estimation by fe. Therefore, in the implementation, we pretrain the PT label estimation model fe for K epochs (meanwhile we set. B Equation 5 to train the node classification model well) and then jointly train the two models together by Equations and 5 respectively. Our algorithm is summarized in Algorithm 1
Time complexity analysis. Assume the GNN architecture for the node classification model and the P1 label estimation model is GCN, for the node classification model, the time complexity is O (TL V according to Wang et al. (2021) where dl is the dimension of the node embedding, [E,, [V] are the number of edges and nodes of the graph and T, L are the number iterations and the layers. For the PI label estimation model, the complexity O [TL + TL|V + TV)).)). For big graph datasets with large |E] and [V], we perform subgraph sampling Hamilton al. 2017 the implementation (Section 4.2 to reduce the time complexity.

---

Published in Transactions on Machine Learning Research (06/2023)
Noise type
No Noise
Symmetric Noise
Asymmetric Noise
Cora
Noise ratio
0.0
0.2
0.4
0.6
0.8
0.2
0.4
0.8
0.8
GCN
0. 0.722(0.03) (.801(0.01)
0.615(0:07)
0-446[0.06]
0.285(0.07)
0.70310.04]
0.514((006)
0.291(0001)
0.16110.02
PI-GNG w/o pe PT-CNN
0.781(0.01)
0.731(0.02)
0.654(0.05)
0.510(0.04)
0.267
0.217/0.04)
0.563
0.332(0.06)
0.20910.00 06
0.780(0.01)
(0.0))
00.04)
(..15
0.296(0.05)
(0.28)
0.687
0.350(0.07)
(.282
Chreseer
GCN
0.683(0.01
0.003
0.32000044)
0.35210.04
0.03
0.585(0:03)
(.605)
0.281(0.05
0.17110.03
PI-GHM w/o pc P1-GNW
0.656(0. 03)
0. 0.08)
0. 0.526(0.55
0 05
22074.044
0.58%(0.04)
(0.05)
0.328 03)
0.23510 03)
0.684(0.03)
0.602
(0.0))
0 (0.27)
(0.05
0.628)
0.531(00.0)
0.353(0.06)
0.238(0.06)
PuhMled
GCN
(0.6)
(.702)
0.8 0.8.000.08
0.462(0.07)
0 0.36700077
(.682
O. 0.399.6.08 0.004(0.008)
0.38770.07
PT-GHM w/o pe P1-GNN
0.774(0.00) D0)
0. (.72300008
0.628(0.05)
0.458(0.07)
0.66
0.722
0.674(
0.412(0.05
0.40110 03
0.774(0.00) 00)
0.724
0.638
0.470(0.08)
08)
0.379(0.07)
0.723
0.58)
0.425(0.07)
0.406)
WIRCS
GON
03
0.358(0.01)
05
0.60810.051
0,27220.05
079
PI-GNB w/o pc PI-GKM
0.676/0.01)
0.624(0.02)
0.39610.07
0.00)
0.470 0.607(0.03)
2.290
0.125/0 05
0.676(0.01
0.60)
(0.0)
1.398(0.07
00.08)
0.610(0.04)
0.483 (0.05)
(.303(0.04)
(0.36)
-GB-arxiv
GCN
0. 01
0.39310.03]
0.278(0.02)
0:435(0:01]
(.300(60011
0.039(0.01
0.019(0.00)
PI-GHB w/o pe
0.042
0. 0.00)
0. 0455000005
40.10.055
0. 0.110
0.445)0.08)
0. (2.06)
0. 0.060(0.02)
(.021(0.00)
PI-ONI
0.4
0.467 0.476(0.04)
(0.44)
0.36)
(0.0))
0.461 (0.61)
(0.0)) 0.022)0.00)
Table 1: Test accuracy on datasets for PI-GNN with GCN as the backbone. Bold numbers are superior results. Std. is shown in the bracket. w/o pe means that the Pl labels are not estimated using the predictive confidence but just the node connectivity.
Underlying assumption and limitation. We note that the proposed method PI-GNN has a limitation during deployment, which relies on the assumption about the underlying graph structure and their label distribution. Namely, homophilous graphs (ie., if two nodes are connected, then with high probability, they should have the saine node label) are more suited to PI-GNN compared to heterophilous graphs (Zhu et al.) 2020; Lim et al.] 2021; Chien et al. 2022). As a quick verification, we test the performance of PI-GNN on several heterophilous datasots Appendix Section where the improvement of PI-GNN is less obvious compared to PI-GNN on homophilous graphs.
Experiments and Results 4
In this section, we present empirical evidence to validate the effectiveness of PI-GNN on different datasets with different noise types and ratios.
4.1 Experimental setting
Datasets. We used five datasets to evaluate PI-GNN, including Cora, CiteSeer and PubMed with the default dataset split as in [Kipf & Welling 2017) and WikiCS dataset (Mernyei & Cangea 2020) as well as OGB-arxiv dataset (Hu et al. 2020). For WikiCS, we used the first 20 nodes from each class for training and the next 20 nodes for validation. The remaining nodes for cach class are used as the test sct. For OGB-arxiv, we use the default split. The dalasel statistics are summarized in Appendir Section C
corrupted these datasets manually by the noise transition matrix Qi = Pr(j= jj i) given that noisy y is flipped from clean y. Assume the matrix Q has two representative structures: (1) Symmetry flipping Qvan Rooyen et al.) 2015 (2) Asymmetric pair flipping: a simulation of fine-grained classification with noisy labels, where labelers may make mistakes only within very similar classes. Note the asymmetric case much harder than the symmetry case. Their precise definition in Appendiz Section A We tested four different noise rates E (0.2, 0.6, 0.8) in this paper for two different noise types, which cover lightly and extremely noisy supervision. Note that in the most extreme case, the noise rate 80% for pair flipping means 80% training data have wrong labels that cannot be learned without additionn additional assumptions.
Since all datasets are clean, following Patrini et al. 2017
Implementation details. We used three different GNN architectures, i.e., GCN, GAT and GraphSAGE, which are implemented by torch- wrrcch-geomerricc Fey Lenssen 2019). All of them have two layers. Specifically, the hidden dimension of GCN, GAT and GraphSAGE is set to 16, and 64. GAT has attention heads in the first layer and head in the second layer. The mean aggregator is used for GraphSAGE. We applied Adam optimizer (Kingma & Ba 2015 with a learning rate of 0.01 for GCN and GraphSAGE and 0.005 for
https://githhbb hymm/pyg-tue httpr/tttammtttheeeeeeeee httaa//yyorrr..eeeeerrrc/a//a//e
7

---

Published in Transactions on Machine Learning Research (06/2023)
GAT on Cora (Sym.)
GAT on CiteSeer (Sym.)
GAT on PubMed (Sym.)
GAT on WIkiCS (Sym.)
0.6
0.7
Vanilla
0.6-
0.6
That AcC.
Test AsLACc
0.6
PI-GNN w/o pC
Test
0.4
Vanilta
PI-GNN w/o pc
0.4
Vanilla
PI-GNN w/o pC
0.5
Test
Vanilla
PI-GNN w/o pc
PI-GNN
PL-NNN
0.4
PI-GNN
0.2
0.2
0.4
0.6
0.8
0.6
Noise ratio
Noise ratio
Noise rallo
Noise ratio
GAT Cora (Asym.)
GAT CiteSeer (Asym.)
GAT Pub Mub (Asym.)
GAT WikiCS (Asym.)
Vanilla
Vanilla
0.7
Vanilla
0.6
Vanilla
0.6
PV-GNN w/o pc
Test Acc.
PI-GNN w/o pC
Test Acc.
0.6
PI-GNN w/o pc
PI-GNN w/o pc PI-GNN
PI-GNN
PI-GNN
Pl GNN
0.4
Test
0.4
0.5
Test
0.2
0.4
0.2
0.2
0.4
0.2
0.4
0.6
0.8
0.6
0.8
0.2
0.4
Noise ratio
Noise ratio
Noise
Noise ratio
GraphSage on Cora (Sym.)
GraphSage on CiteSeer (Sym.)
GraphSage PubMed (Sym.)
GraphSage on WikiCS (Sym.)
0.6
Vanilla
0.6
0.6
Thst
Test Acc.
0.6
GNN pc
Test Acc.
Test
Vanilla
Vanilla
Pt- GNN
Vanilla
0.4
Pl. GNN
0.4
PNN
0.5
Pi GNN
PI-GAN
GNN
0.4
0.2
PI-GAN
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
Noise ratio
Naise ratio
Noise ratio
Noise ratio
GraphSage on Cora (Asym.)
GraphSage on CiteSeer (Asym.)
GraphSage on PubMed (Asym.)
GraphSage on WikiCS (Asym.)
Vanilla
0.6-
Vanilla
0.7
Vanilia
Vanilla
0.6
PI-GNN w/o
PI-GNN weo
TesLAcc
0.6
PNNGNN w/o pc
TestAcc
PI-GNN w/o
0.4
PI-GNN
Test
0.4
PI-GNN
0.5
PI-GNN
0.4
PI-GNN
0.2
0.2
0.4
0.2
0.2
0.4
0.8
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
0.2
0.4
Noise ratio
Noise ratio
Noise ratio
Noise ratio
Figure 3: Test accuracy of PI-GNN and comparison with PI-GNN w/o pc and vanilla GNN on two additional model architectures under different noisy settings.
GAT. The weight, decay is set to 5e-4. We trained for 400 epochs on a Tesla P40. The loss weight ,3 is set to /V// - Q)², where [V] is the number of nodes and Q is the sum of all elements of the preprocessed adjacency matrix. The number of pretraining epochs K is set to 50 and the total epoch N is set to 400. For subgraph sampling, we sampled 15 and 10 neighbors for each node in the 1st and 2nd layer of the GNN and set the batch size to be 1024. We tuned all the hyperparameters on the validation set and reported the node classification accuracy on the clean test set. Delails about the ablation studies on these factors are shown in Section 4-5 Each experiment is repeated for 10 times with random seeds from 1 to 10. The training time for our PI-GNN and the vanilla GNN model is compared in the Appendir Section !
4.2 Effectiveness on different datasets
We evaluated the effectiveness of PI-GNN on five datasets with different noisy labels and noise rates, which is shown in Table ! with GCN as the backbone. Specifically, we are interested to observe 1) whether the introduced regularization objective between nodes can improve a vanilla GNN against noisy labels and 2) whether the predictive confidence from the PI label cstimation model is beneficial for the test accuracy. Therefore, we compared the accuracy of a vanilla GNN, PI-GNN trained with the node connectivity as the PI label (PI-GNN w/o pc) and PI-GNN.
From Table 1 we made several observations: Firstly, the GNN trained with the PI regularization objective is more robust to noisy labels, where both PI-GNN w/o pc and PI-GNN perform much better than a vanilla GNN. Secondly, by estimating the PI labels using the PI label estimation model, the test accuracy is further improved compared to dircetly using the node connectivity as the PI label. For instance, PI-GNN improves the accuracy by 2.3% with the asymmetric noise (noise ratio e = 0.8) on Cora compared to PI-GNN w/o pc, which justifies the effectiveness of our design. Thirdly, the PI-GNN does not help the GNN with the clean
8

---

Published in Transactions on Machine Learning Research (06/2023)
Noise type
Symmetric Noise
Asymmetric Noise
Noise
0.4
0.4
0.6
Test dataset:
Cora / CiteSter
Deroupling GCE
0.581(0.08)
0.518(0.03)
0.425(0.08)
(.390
(..4
0.474[0.04)
0.336(0.03)
0.323(0.06)
0.330(0.03) 0.627(0.07) /
0.447(0.06)
(0.03
0.511((005)
0.46680.05)
0.284(0.03)
(.205
API.
0.624(0.08)
/ 0.522(0.04)
0.446(0.06)
0.376
0.507(0.06)
0.456(0.06)
0.281
(..04
Co-teaching
0.577(0.11)
0.573(0.07)
0.376(0.07)
0.404(0.06)
0.457(0.10)
0..66200.0)
0.237/0.09)
0.256(0.08))
LPM-1
0.542(0.09) / 0.467(0.06)
0.447(0.07)
0.395(0.08)
0.481(0.077)
0.506(0.08)
0.318(0.04)
/ 0.341
T-Revision
0.596(0.06)
0.518/0.03)
0.425.0.06
(.380(0.04)
0.512(0.06)
0.457(0.06)
0.281(0.05)
0.263(0.05)
DivideMix
0.628(0.06)
0.515/0.05)
0.463(0.09)
0.356(0.05)
0.428(0.01)
0.396(0.03)
0.313(0.03)
(.282(0.02)
PI-GRE (ours)
0.661(0.03) (0.9) 0.515(0.03)
0.432(0.07)
0.587(0.07)
(.531(0.06)
0.350(0.07) (.353
Table Comparative results with baselines. Bold numbers are superior results. LPM-1 means one extra clean label
is used for each class. The result the left and right of each cell is the classification accuracy of the Cora dataset. and CiteSeer dataset, respectively.
node labels, e.g., 80.4% of vanilla GCN vs. 78.0% of PI-GNN on Cora, which illustrates the PI-GNN helps to combat noisy supervision rather than inherently improve the node classification with purcly clean node labels. Additional resulls on helerophilous dalasels and with lower noise ralios are in Appendix Sections and
4.3 Performance on different GNN architectures
We evaluated PI-GNN on different GNN architecturos, i.e., GAT and GraphSAGE. The experiments are conducted on Cora, CiteSeer, PubMed and WikiCS datasets, which are shown in Figure 3 As can be observed, PI-GNN performs similarly on GAT and GraphSAGE compared the results on GCN, where the regularization of PI and the predictive confidence are both beneficial for model generalization even with extremely noisy supervision. Moreover, using the predictive confidence as PI labels is more effective on GraphSAGE. For example, in the Cora. dataset, PI-GNN improves PI-GNN w/o pe by 4.2% and 3.1% on average under symmetric and asymmetric noise, respectively, which is larger than that for GAT and GCN, It may suggest the mean aggregator in GraphSAGE is more suscoptible to the sub-optimal PI labels. We provide significance test on the results of GAT in Appendir Section
4.4 Comparison with baselines
In order to further demonstrate the competitive performance PI-GNN, we compared with several powerful baselines for combating noisy labels literature. For fair comparison, we used the same GNN archilect ure, i.e., GCN, and the same overlapping hyperparameters during implementation. The other method-specific hyperparameters are tuned according to the original paper on the validation set. Specifically, we compared with noise-transition cattrix-based method, T-revision (Xia et al. 2019), robust loss functions, such Generalized Cross Entropy (GCE) loss (Zhang & Sabuncu, 2018) and Active Passive Loss (APT.) (Ma et al. 2020), optimization-basee approaches, such as Co-teaching (Han et al. 2018b), Decoupling (Malach & Shalev-Shwartz 2017) and DivideMix (Li et al. 2020b). We also compared with Label Propagation and Meta learning (LPM) Xiaa et al.) 2021), a method that is specifically designed for solving label noise for node classification but uses a small set of clean labels. For illustration, we reportod the classification accuracy on Cora and CiteSeer with symmetric and asymmetric noise (noise rate e 0.4, 0.6) in Table 2
From Table 2 PI-GNN outperforms different baselines with considerable margin, e.g., improving the classification accuracy by 5.2% on Cora under the symmetric noise (c 0.6) compared to the bost bascline. Moreover, PI-GNN is able to outperform LPM-1, which relieves the strong assumption that auxiliary clean node labels are available. We compare with traditional graph semi-superuised learning approaches and on more datasets in Appendir Section D and Section J respectively.
4.5 Ablation studies
The effect of PI labels. To show the importance of informative PI labels, we tested PI-GNN under different PI labels in addition to the one that estimated by the PI label estimation model (ours): 1) random labels, 2) noisy class label comparison and 3) estimated PI w/o fe where only one GNN is used for both node classification and PT label estimation. We also compared with the PT labels by clean
9

---

Published in Transactions on Machine Learning Research (06/2023)
(a) Cara
(c) Cora
(d)
0.32
Symnettic
1.44
Synnerk
Synmerk
Symmetic
Nommmtiic G.
Asyrmaarc
Asymmaakk
Asymmezicc
0.42
Test Ace
0.42
0.43
5.20
0.20
0.42
6.36
6.30
0:33
0.22
100
152
200
250
302
102
230 250
300
5.01
0.55
0.1
0.2
1.0
5.01
0.05
0.3
6.2
18
Figure 4: (a)-(b) Performance of PI-GNN w.r.f. different pretraining epochs on Cora and CiteSeer. x axis denotes the value the pretraining epochs for the PI label estimation model. (c)-(d) Performance of PI-GNN w.r.t. the regularization loss weight 3. axis denotes the value of the loss weight and 3 is the weight that is aware of the sparsity of the input graph (ef. Section 1.1
class label comparison, which expected to be the oracle. We used GCN as the backbone and tested on two datascts, i.e., Cora and CiteScer and different noise types.
The results are shown in Table
From Table 3 using PI labels based on the PI label estima- tion model achieves better performance than using noisy node label comparison. Employing randomly generaled Pl labels incurs the worst performance because it completely ignores the importance of the pairwise interactions. More- over, removing the PI label estimation model decreases the test accuracy because the node embeddings opti- mized by the noisy class labels well, which might not be effective for the PI label estimation (32.9% vs. 35.0% for Cora with 60% asymmetric noise). Finally, our PI-GNN achieves a similar performance compared to training by clean label comparison (the oracle case), showcasing the effectiveness of our estimated PI labels. Note in our noisy setting, it is impossible to reach the oracle. More results are in Appendir Section
Noise Type
Synametric Nois:
Asymmmetric Noise
Corn
Nota: Raido
0.6
B.6
Random PI Jabel
(.009(0.055
0,295(0.04)
Nolay labeel compurison
(..01)
0.300(0.04)
Estinaledd PI w/o Je
0.511(0.03)
0.329(0.05)
Clean lubel comparison (oraele)
(.4447(1077
0.355(0.05)
Estiaated Pl (ours)
0.515(0.08)
0.350(0.07)
Citeseer
Noise Racio
0.6
Random PI label
(.351(0.04)
0.298(0.04)
Noisy label comparison Estinated PI w/o fa
0.378(0.04)
0.312(0.04)
0.130(0.07)
0.310(0.06)
Clean label comparison (aracle)
0.447(0.07)
0.355/0.06)
Escimated (ours)
0.432(0.07)
0.353(0.06)
Table 3: Performance of PI-GNN with different PI labels.
Sensitivity to the pretraining epoch of the PI label estimation model. We investigated whether the performance of PI-GNN is sensitive to the number of pretraining epochs for the PI label estimation model. The experimental results on Cora and CiteSeer with GCN under symmetric and asymmetric noise are shown in Figure (a) and (b). For illustrration, we set the noise ralio to be 0.6. As can be observed, pretraining the Pl label estimation model for K epochs is effective for improving the generalization on the clean test set. Given small K, the confidence mask is not estimated well which is not helpful apply it the node classification model for regularization. Meanwhile, K should not be too large in order to sufficiently regularize the node classification model using Equation
5
K is set to 50 for all the experiments.
Application of PI-GNN on label- noise baselines. To observe whether PI-GNN is able improve the generalization ability for differ- ent label-noise baseline models, we extended three representative ap- proaches, i.e., T-revision (Xia et al., 2019), APL (Ma et al. 2020) and Di- videMix (Li et al. 2020b) by adding the PI regularization objective dur- ing training. Specifically, we used the sum of their original loss and the PI regularization loss (Cregs of Equation 5] to optimize the GNN. The weight for PI regularization loss set to the same value
(8ym.)
(Asym.)
GleSeer
CleSser (Asym.)
3350
2.45
Titooo on Di
2320
Thoosion
taneliee- Q6N
Dalsetil
0.50
Sonel T0
5.500
Toool 11-
Datsetia Thoebton
5275
0.200
Figure Performance of the PI-GNN applied on different label-noise baselines on Cora and CiteSeer. Noise ratio is set to 0.6.
10

---

Published in Transactions on Machine Learning Research (06/2023)
approach (cf. Section 4.1 We chose GCN the backbone and reported the test accuracy on Cora and CiteSeer with both symmetric and asymmetric noise (< = 0.6) in Figure] As the result shows, PI-GNN is orthogonal to those nobse-robust baseline models, which is potentially useful for improving their performance without bells and whistles. For instance, The test set accuracy is improved by 4.3% on DivideMix under the asymmetric noise for the CiteSeer dataset and thus demonstrates the universality of our proposed PI-GNN.
Different architectures for two branches. PI-GNN allows for a flexible choice of the architectures for the PI label es- timation model and the node classification model, where light-weight PI label estimation model can help large node clas- sification model for node classification during training. In what follows, we used three different PI label estimation-noke node clas- sification model pairs, namely GCN-GAT, GCN-GraphSAGH and GAT Grap-GGaE.SGGE. The number of parancters for GCN, GAT, GraphSAGE is 0.02, 0.09 and 0.18 M, respectively. The comparison with using the same architectures are shown in Table g From Table 4 using a light-weight GNN for the PI label estimation model is able to further improve the clean test accuracy, which is promising for efficient deployment. of PI-GNN on real-world graph datasets.
Noise Type
Sym. Noise
Asym. Noise
Noise Ratio
0.6
0.6
GAT only
(.395(0.05)
(.330)
GCN-GAT
0.412(0.06)
(0.333)
GraphSAGE only
0.503(0.08)
0.376(0.08)
GCN- GraphSAGE
0.512(0.09)
0.383(0.06)
GraphSAGE only
(.503(0.08)
0.376(0.08)
GAT- NAT--GrphSAGE.
0.516(0.08)
0.381 (0.06)
Table 4: Performance of PI-GNN with different architectures for two branches on CiteSeer.
The effect of regularization weight. To observe whether the regularization loss weight 3 matters to the model performance, we trained PI-GNN with different values of B, í.e., 0.01, 0.05, 0.1, 0.2, 0.5, 1.0 and compared with the value 3' /V/2/([V/2 Q)² which is aware of the sparsity of the graph in Figure | (c) and (d). We conducted experiments on Cora and CiteSeer with GCN and showed the results with symmetric and asymmetric noise (c 0.6). From the figure, PI-GNN is sensitive to the choice of regularization loss weight B. On both datasets with different noise types, PI-GNN trained with & achieves the best test accuracy, and simultancously avoids heavy tuning procedure on the validation set.
5
Related work
Graph Neural Networks. Graph neural networks have been widely used to model the graph-structured
data with various architectures, such as graph convolutional network (GCN) (Kipf & Welling 2017], graph
attention network (GAT) (Velickovic et al. 2018). GraphSAGE (Hamilton et al. 2017), Graph Isomorphism
Network (GIN) (Xu et al. 2019), Simple Graph Convolution (SGC) (Wu et al., 2019 etc. Common graph
analysis tasks, including node classification (Lan et al., 2020). link prediction (Zhang & Chen 2018), graph
classification (Bacciu et al. 2018), graph generation (Liao ct al.) 2019; Shi et al.] 2020). have been widcly studied in literature. However, only a few works focused on training robust GNNs against noisy labels, such as by loss correction (NT et al. 2019] for graph classification, sample reightgh (Xia et al. 2021 Li et al.,
2021a) for node classification. None of them exploited explicit PI, which are compared with our PI PI-GNN in
Section 4.4 Bui et al. (2017): Streteu et al. (2019 Ng et al. (2018 Qu et al. (2019 Ma et al.
2019 utilized
graph structures for semissupervised learning but with clean labels. Moreover, they did not further process
the graph structure while PI-GNN utilizes the graph structure and introduces a new P1 label estimation
procedure during training. Jiang et al. (2019); Yu et al. (2020); Zheng et al. (2020); Chen et al. (2020b);
Kim & Oh- 202T)); Fatemi et al. (2021) iteratively refined graph structure during training for missing edge
prodiction or error edge detoction while PI-GNN does not change the input graph. [uo ct al., (2021) proposod
a parameterized topological denoising network to improve the robustness and generalization performance of
GNNs by learning to drop insk-irreleaant edges. The main difference is that PI-GNN deals with the situation where the node labels are corrupted while they deal with the noisy edges.
Zhao et al. (2020); Stadler et al. (2021); Wu et al. (2023) proposed to use uncertainly estimation and out-of difdiistribttion detection techniques (uu et al.)
2022c b a Tao et al. 2023 Yang et al. 2021 Bai et al.,
2023 Sun et al., [2021] [Sun & Li 2022 SSn et al.
2022 Ming et al., 2022a b 2023 Wei et al.] 2022 Yang
et al. 2022) for detecting out-of-distribution or noisy node samples on graphs, where our goal is correctly classify the noisy nodes.
11

---

Published in Transactions on Machine Learning Research (06/2023)
Neural networks with noisy labels. Methods for neural networks against noisy labels can be roughly categorized into three types (Han al. 2020b), Lee. approaches from the perspective of data (van Rooyen & Williamson, 2017), learning objective (Reed et al. 2015 Miyato et al.
2019) and optimization (Arpit
ct al. 2017). Mothods based on data mainly built the noise transition matrix to explore the data relationship between clean and noisy label by an adaptation layer (Sukhbaatar et al. 2015), loss correct.ion (Patrini
ct al. 2017) and prior knowlodge (Han al. 2018a). Methods based on objective modified the learning objective by regularization (Han al., 20200a) reweighting (Liu & Tao 2016; Wang et al. 2017) and loss redesign Thulasidasan et al. 2019). Methods based on optimization mainly changed the optimization policy, such as by memorization effect (Jiang et al.) 2018), self-training (Ren et al. 2018) and co-training (Yu et al., 2019). Wu et al. (2021) proposed to use the similarity loss for noisy labels of images but it relied on the noisy transition matrix, which is sensitive to the matrix estimation quality and cannol use the graph structure for regularization. In this paper, we extend several approaches from each category to compare with PI-GNN in Scction 4.4
6
Conclusion
In this paper, we proposed PI-GNN, simple but effective learning paradigm for helping the GNN to generalize well with noisy supervision. Our key idea is to leverage the pairwise interactions wet nodes to explicitly regularize the similarity of those node embeddings during training. In order to perform noise-robust node classification, we introduce a new learning framework to adaptively estimate and leverage the pairwise interactions for model regularization. We conducted extensive experiments to demonstrate that PI-GNN can train GNNs robustly under extremely noisy supervision, which serves as a crucial step towards the reliable deployment of GNNs in complex real-world applications. We hope our work inspires future research on noise-robust graph learning, such as proposing novel pairwise approaches from the algorithmic perspective and constructing real-world noisy graph datasets for more comprehensive and practical empirical evaluation.
7 Acknowledgement
BH was supported by Tencent AI Lab Rhino-Bire Gift Fund. The authors would also like to thank TMLR reviewers for the helpful suggestions and feedback.
References
Devansh Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj. Asja Fischer, Aaron C. Courville, Yoshua Bengio, and Simon Lacoste-Julien. A closcr look at memorization in deep networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, volume 70, pp. 223-242, 2017.
Davide Bacciu, Federico Errica, and Alessio Micheli. Contextual graph markov model: A deep and generative approach to graph processing. In Proceedings of the 35th International Conference Machine Learning, ICML 2018, volume 80, pp. 304-313, 2018.
Haoyue Bai, Gregory Canal, Xuefeng Du, Jeongyeol Kwon, Robert D) Nowak, and Yixuan Li. Feed two birds with one scone: Exploiting wild data for both out-of-distribution generalization and detection. In International Conference on Machine Learning, 2023.
Malik Boudiaf, Jérôme Rony, Imtiaz Masud Ziko, Eric Granger, Marco Pedersoli, Pablo Piantanida, and Ismail Ben Ayed. A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses. In European conference on computer vision, pp. 548 564. Springer, 2020.
Thang D. Bui, Sujith Ravi, and Vivek Ramavajjala. Neural graph machines: Learning neural networks using
graphs. CoRR, abs/ abs/1703.04818, 2017.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020a.
12

---

Published in Transactions on Machine Learning Research (06/2023)
Yu Chen, Lingfei Wu, and Mohammed J. Zaki. Iterative deep graph learning for graph neural networks: Better and robust node embeddings. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NearIPS 2020, December 6-12, 2020, virtual, 2020b.
Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, and Inderjil. S Dhillon. Node feature extraction by self-supervised multi-scale neighborhood prediction. In International Conference on Learning Representations, 2022. URL https: /ttep:://ppnreeveu Jgt. iHbs8
Hande Dong, Jiawei Chen, Fuli Feng, Xianguan He, Shuxian Bi, Zhaolin Ding, and Peng Cui. On the equivalence of decoupled graph convolution network and label propagation. CoRR, abs/ hts/2010.124008 2020.
Xuefeng Du, Gabriel Gozum, Yifei Ming, and Yixuan Li. Siren: Shaping representations for detecting out-of-distribution objects. In Advances in Neurol Information Processing Systems, 2022a.
Xuefeng Du, Xin Wang, Gabriel Gozum, and Yixuan Li. Unknown-aware object detection: Learning what you don't know from videos in the wild. In Proceedings of the TERE/CVF Conference on Computer Vision and Pattern Recognition, 2022b.
Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. Vos: Learning what you don't know by virtual outlier synthesis. In Proceedings of the International Conference on Learning Represenlations, 2022c.
Bahare Fatemi, Layla El Asri, and Seyed Mehran Kazemi. SLAPS: sell-supervision improves structure learning for graph neural networks. CoRR, abs/2102.05034,, 2021.
Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorrc Geometric. In ICLR Workshop on Represenlation Learning on Graphs and Manifolds, 2019.
William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs In Advances in Neural Information Processing Systems 30, pp. 1024-1034, 2017.
Bo Han, Jiangchao Yao, Gang Niu, Mingyuan Zhou, Ivor W. Tsang, Ya Zhang, and Masashi Sugiyama. Masking: A new perspective of noisy supervision. In Advances in Neural Information Processing Systems 31, pp. 5841-5851, 2018a.
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor W. Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in Neural Information Processing Systems 31, NewrIPS 2018, pp. 8536 8536-8546, 2018b.
Bo Han, Gang Niu, Xingrui Yu, Quanming Yao, Miao Xu, Ivor W. Tsang, and Masashi Sugiyama. SIGUA: forgetting may make learning with noisy labels more robust. In Proceedings of the 37th International Conference on Machine Learning, ICML, 2020, volume 119, pp. 4006-4016, 2020a.
Bo Han, Quamming Yao, Tongliang Liu, Gang Niu, Ivor W. Tsang James T. Kwok, and Masashi Sugiyama. A survey of label-noise representation learning: Past, present and future. CoRR, Abs/2201... 1..1400.666 2020b.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729-9733, 2020.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ron, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In Advances in Neural Information Processing Systems 33, NewrIPS 2020, 2020.
Bo Jiang, Ziyan Zhang, Doudon Lin, Jin Tang, and Bin Luo. Semi-supervised lcarning with graph learning- convolutional networks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June. 16-20, 2019, pp. 11313 11320, 2019.
13

---

Published in Transactions on Machine Learning Research (06/2023)
Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In Proceedings of the 95th International Conference on Machine Learning, JCML 2018, volume 80, pp. 2309 2318, 2018.
Dongkwan Kim and Alice Oh. How to find your friendly neighborhood: Graph attention design with solf- supervision. In International Conference on Learning Regresentations, 2021. URL https:///penrrevier
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, 2015.
Thomas N Kipf and Max Welling. Variational graph auto-cncoders. arXiv preprint arXiv:1611.07906 2016.
Thomas N. Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, 2017.
Lin Lan, Pinghui Wang, Xuefeng Du, Kaikai Song, Jing Tao, and Xiaohong Guan. Node classification on graphs with few-shot novel labels via meta transformed network embedding. In Advances in Newral Information Processing Systems 33, NewrIPS 2020, 2020.
Jingling Li, Mozhi Zhang, Keyulu Xu, John P. Dickerson, and Jimmy Ba. Noisy labels can induce good 2020a.
representations. CoRR,
Junnan Li, Richard Socher, and Steven C. H. Hoi. Dividemix: Learning with noisy labels as semi-supervised learning. In 8th International Conference on Learning Representations, ICLR 2020, 2020b.
Yayong Li, Jie Yin, and Ling Chen. Unified robust training for graph neural networks against label noise. CoRR, abs/2103.034 000013 2021a.
Yayong Li, Jie Yin, and Ling Chen. Unified robust training for graph neural networks against label noise. Tn Pacific-Asio Conference Knowledge Discovery and Data Mining, pp. 528 540, 2021b.
Renjie Liao, Yujia Li, Yang Song, Shenlong Wang, William L. Hamilton, David Duvenaud, Raquel Urtasun, and Richard S. Zemel. Efficient graph generation with graph recurrent attention networks. In Advances in Neural Information Processing Systems 32, NewrIPS 2019, pp. 4257-4267 2019.
Derek Lim, Felix Matthew Hohne, Xiuyu Li, Sijia Linda Huang, Vaishnavi Gupta, Omkar Prasad Bhalerao, and Ser-Nam Lim. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL httpps://pperer//// wttp://wwwe..orrrreeeeeeeedeeeee
Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. IEEF Trans. Pottern Anal. Mach. Intell., 38(3):447 461, 2016. doi:
Qing Lu and Lise Getoor. Link-based classification. In Tom Faweett and Nina Mishra (ods.), Machine Learning, Proceedings of the Twentieth International Conference (ICML 2003), August 21-24, 2003, Washinglon, DC, USA, pp. 496 503. AAAI Press, 2003. URL htLp:///://wwe aaai, wttp:///brarry//////////////////
Dongsheng Luo, Wei Cheng, Wenchao Yu, Bo Zong, Jingchao Ni, Haifeng Chen, and Xiang Zhang. Learning to drop: Robust graph neural network via topological denoising. In Proceedings of the 14th ACM International Conference on Web Search and Dala Mining. ACM, 2021.
Tenglei Ma, Cao Xiao, Junyuan Shang, and Jimeng Sun. CGNF: Conditional graph neural fields, 2019. URL https:///penreevev. net http:///paarrreeeeeeaotorrr/a//a htt////oruuu//drrywywxx
Xingjun Ma, Hanxun Huang, Yisen Wang, Simone Romano, Sarah M. Erfani, and James Bailey. Normalizod loss functions for deep learning with noisy labels. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, volume 119, pp. 6543 6553, 2020.
14

---

Published in Transactions on Machine Learning Research (06/2023)
Yao Ma, Xiaorui Liu, Neil Shah, and Jiliang Tang. Is homophily necessity for graph neural networks? CoRR, abs/21006.6114,, 2021. URL https://arxiv.. org/abs/ wwwp////....
Eran Malach and Shai Shalev-Shwariz. Decoupling "when to update" from 'how to update". In Aduances in Neural Information Processing Systems 30, pp. 960 970, 2017.
Péter Mernyei and Catalina Cangea, Wiki-cs: A wikipedia- based benchmark for graph neural networks. CoRli abs/2007.02900 2020.
Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li. Delving into out-of-distribution detection with vision-language representations. In Advances in Neural Information Processing Systems, 2022a.
Yifei Ming, Ying Fan, and Yixuan Li. Poem: Out-of-distributiion delection with posterior sampling. In International Conference on Machine Learning, 2022b.
Yifei Ming, Yiyou Sun, Ousmane Dia, and Yixuan Ti. How to exploit hyperspherical embeddings for out-of-distribution detection? In Proceedings of the International Conference on Learning Representations, 2023.
Takeru Miyato, Shin-ichi Macda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: A regular- isation method for supervised and semi-supervised learning. IEEE Trans. Pallern Anal. Mach. Intell., 41 (8):1979-1993, 2019. doi: 20000//TT.....18.2222522
Yin Cheng Ng, Nicolò Colombo, and Ricardo Silva. Bayesian semi-supervised learning with graph gaussian processes. In Advances in Neural Information Processing Systems 31: Anreual Conference on Newral Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp. 1690- 1701, 2018.
Hoang NT, Choong Jun Jin, and Tsuyoshi Murata. Learning graph neural networks with noisy labels. CoRR, (bss/19900111,, 2019.
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In 2017 IEEE Conference on Computer Vision and Pottern Recognition, CVPR 2017, pp. 1233-22413 2017.
Qi Qi, Yan Yan, Zixuan Wu, Xiaoyu Wang, and Tianbao Yang. A simple and effective framework for pairwise deep metric learning. In European Conference on Computer Vision, pp. 375-391. Springer, 2020.
Meng Qu, Yoshua Bengio, and Jian Tang. GMNN: graph markov neural networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 5241 5250. PMLR, 2019.
Scott E. Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. Training deep neural networks on noisy labels with bootstrapping, In 3rd International Conference on Learning Representations, ICLR 2015, 2015.
Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, volume 80, pp. 4331-4340, 2018.
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow- based autoregressive model for molecular graph generation. In 8th International Conference on Learning Representations, ICLR 2020, 2020.
Maximilian Stadler, Bertrand Charpentier, Simon Geisler, Daniel Zügner, and Stephan Günneraan. Graph posterior network: Bayesian prodictive uncertainty for node classification. In A. Beygekzmmer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL http://pweeereeee.eee/////r////a
15

---

Published in Transactions on Machine Learning Research (06/2023)
Otilia Streteu, Krishnamurthy Viswanathan, Dana Movshovitz-Attias, Emmanouil A. Platanios, Sujith Ravi, and Andrew Tomkins. Graph agreement models for semi-supervised learning. In Advances in Newral Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019,
NewrIPS 2019, December 8-14, 2019, Vancowver, BC, Canada, pp. 8710-8720, 2019.
Sainbayar Sukhbaatar, Joan Bruna, Manohar Paluri, Lubomir Bourdev, and Rob Fergus. Training convolu- tional networks with noisy labels, 2015.
Yiyou Sun and Yixuan Li. Dice: Leveraging sparsification for out-of-distribation detection. In Proceedings of European Conference on Computer Vision, 2022.
Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of-distribution detection with rectified activations. In Advances in Newral Information Processing Systems, 2021.
Yiyou Sun, Yifci Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distributio detection with deep ncarest neighbors. In International Conference on Machine Learning, 2022.
Leitian Tao, Xuefeng Du, Xiaojin Zhu, and Yixuan Li. Non-paramotric outlier synthesis. In Proceedings of the International Conference on Learning Representations, 2023.
Sunil Thulasidasan, Tammoy Bhattacharya, Jeff A. Bilmes, Gopinath Chenmupati, and Jamal Mohd-Yusor Combating label noise in deep learning using abstention. In Proceedings of the 36th International Conference on Machine Learning, ICML, 2019, volume 97, pp. 6234-6222220 6243, 2019.
Brendan van Rooyen and Robert C. Williamson. A theory of learning with corrupted labels. J. Mach. Learn. Res.,
2017.
Brendan van Rooyen, Aditya Krishna Menon, and Robert C. Williamson. Learning with symmetric label noise: The importance of being unhinged. In Advances in Neural Information Processing Systems 28, pp. 10 18, 2015.
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana. Romero, Pietro Liô, and Yoshua Bengio. Graph attention networks. In 6th International Conference Learning Representations, ICLR 2018, 2018.
YEWEN Wang, Jian Tang, Yizhou Sun, and Guy Wolf. Decoupled greedy learning of graph neural networks, 2021. URL
Yixin Wang, Alp Kucukelbir, and David M. Blei. Robust probabilistic modeling with bayesian data reweighting. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, volume 70, pp. 3646-360 1646-3655, 2017.
Hongxin Wei, Renchunzi Xic, Hao Cheng, Lei Feng, Bo An, and Yixuan Li. Mitigating neural network overconfidence with logit normalization. In International Conference on Machine Learning, 2022.
Felix Wu, Amauri H. Souza Jr., Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Q. Weinberger. Simplifying graph convolutional networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, volume 97, pp. 6861-6871, 2019.
Qitian Wu, Yiting Chen, Chenxiao Yang, and Junchi Yan. Energy-based ont-of-distribution detection for graph neural networks. arXiv preprint arx::2230.002: 02914, 2023.
Songhua Wu, Xiaobo Xia, Tongliang Liu, Bo Han, Mingming Gong, Nannan Wang, Haifeng Liu, and Gang Niu. Class2simi: A noise reduction perspective on learning with noisy labels. In International Conference Machine Learning, pp. 11285-11295. PMLR, 2021.
Jun Xia, Haitao Lin, Yongjic Xu, Lirong Wu, Zhangyang Gao, Siyuan Li, and Stan Z. Li. Towards robust graph neural networks against label noise. In OpenReview, 2021.
Xiaobo Xia, Tongliang Liu, Nannan Wang, Bo Han, Chen Gong, Gang Niu, and Masashi Sugiyama. Are anchor points really indispensable in label-noise learning? In Advances in Neural Information Processing Systems 32, Ne 2019, pp. 6835 0846, 2019.
16

---

Published in Transactions on Machine Learning Research (06/2023)
Keyulu Xu, Weihua Hu, Jure Leskovee, and Stefanic Jegelka. How powerful are graph neural networks? In 7th International Conference on Learning Representations, ICLR 2019, 2019.
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. arXiv preprint ar 0rX:c:1110.11334 2021.
Jingkang Yang, Pengyun Wang, Defian Zou, Zitang Zhou, Kunyuan Ding, WENXUAN PENG, Haoqi Wang, Guangyao Chen, Bo Tí, Yiyou Sun, Xuefeng Du, Kaiyang Zhou, Wayne Zhang, Dan Hendrycks, Yixuan Li, and Ziwei Liu. OpenOOD: Benchmarking generalized out-of-distribution detection. In Thirty-sirth Conference on Neural Informalion Processing Systems Dalasels and Benchmarks Track, 2022.
Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learning with graph embeddings. In Proceedings of the 33nd International Conference on Machine Learndng, ICML 2016, volume 48, pp. 40-48, 2016.
Donghan Yu, Ruohong Zhang, Zhengbao Jiang, Yuexin Wu, and Yiming Yang. Graph-revised convolutional network. In Machine Learning and Knowledge Discovery in Databases European Conference, ECML PKDD 2020, Ghent, Belgium, September 14-18, 2020, Proceedings, Part III, volume 12459 of Lecture Notes in Computer Science, pp. 378 393. Springer, 2020.
Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor W. Tsang, and Masashi Sugiyama. How does disagreement help generalization against label corruption? In Proceedings of the 36th Internalional Conference on Machine Learning, ICML 2019, volume 97, pp. 1164-7173, 2019.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In 5th International Conference on Learning Representations, ICLR 2017,22 2017.
Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. In Advances in Newral Information Processing Systems 31, NewrIPS 2018, pp. 5171-5181, 2018.
Zhilu Zhang and Mert R. Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In Advances in Neurol Information Processing Systems 31, pp.
2018.
Xujiang Zhao, Feng Chen, Shu Hu, and Jin-llee Cho. Uncertainty aware semi-supervised learning on graph data. Advances in Neural Information Processing Systems, 33, 2020.
Cheng Zheng, Bo Zong, Wei Cheng, Dongjin Song, Jingchao Ni, Wenchao Yu, Haifeng Chen, and Wei Wang. Robust graph representation learning via neural sparsification. In Proceedings of the 37th International Conference on Machine Learning, TCML, 2020, 19-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 11458 11468. PMLR, 2020.
Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Schölkopf. Learning with local and global consistency. In Advances in Neural Information Processing Systems 16, NIPS 2003, pp. 321-328, 2003.
Fan Zhou, Chengtai Cao, Kunpeng Zhang, Goce Trajcovski, Ting Zhong, and Ji Geng. Meta-gnn: On few-shot node classification in graph meta-learning. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, pp. 2357 2360, 2019.
Joong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra, Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in Neural Information Processing Systems, 33:7793 7804, 2020.
Xiaojin Zhu and Zoubin Ghahramani. Learning from labeled and unlabeled data with label propagation. Technical Report, 2002.
Xiaojin Zhu, Zoubin Ghahramani, and John D. Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the Twentieth International Conference on Machine Learning, ICML 2003, pp. 912-91129991 919, 2003.
17

---

Published in Transactions on Machine Learning Research (06/2023)
Noise-robust Graph Learning by Estimating and Leveraging Pairwise Interactions (Appendix)
A
Definition of noise
The definition of transition matrix Q is as follows. n is number the class.
Asymmetric pair flipping:
1-c
0
0
1 -
é
0
Q
(7)
1-c
c
1 -
Symmetry flipping:
1
n-1
R-1
n-1
Q
=
(8)
n-1
n-1
n=1
B Software and hardware
We run all experiments with Python 3.8.5 and PyTorch 1.7.0, using NVIDIA TESLA P40 GPUs
C Dataset Details
Here provide the details of graph datasets for node classification.
Dataset
#Nodes
# Fxgees
# Classes
Cora
2,485
5,069
7
CiteSeer
2,110
3,668
6
PubMed
19,717
11,321
3
WikiCS
11,701
216,123
10
OGB-arxiv
169,343
1,166,243
40
Table 5: Statistics of the datasets.
D Comparison with traditional graph semi-supervised learning based approaches.
For the comparison with the traditional semi-supervised graph embedding methods, we follow the same experimental setting and compare with ICA (Lu & Getoor 2003), Planetoid (Yng et al. 2016) and Label
Propagation (LP) (Zhu & Ghahramani 2002 on Cora as follows in Table 6 The result shows the advantage
of PI-GNN across different noise ratios.
E Experimental results on heterophilous datasets
We perform extra experiments on heterophilous datasets (Ma al. 2021 The results are demonstrated in the Table It. shows that PI-GNN is still able to outperform the vanilla except for case in Chameleon
18

---

Published in Transactions on Machine Learning Research (06/2023)
Noise type
Symmetric Noise
Asymmetric Noise
Noise ratlo
0.0
0.2
0.4
0.6
U.S
0.2
0.4
0.6
U.S
ICA
0.729(0.01)
0.609(0.01)
0.523(0.04)
0.394(0.00)
0.139(0.00)
0.549(0.00)
0.453(0.00)
0.284(0.01)
0.127(0.01)
LP
0.603(0.00)
0..02)0.02)
(..07
0.003
0.170(0.03)
0.513(0.03)
(.394
0.238(0.03)
0.141(0.02)
Planetoid
0.739(0.01)
(0.89
(.004)
0.379
(0265(0.06)
0.627(0.08)
(.441(0.04)
(0.06)
0.210/0.09)
PI-ONN
0.780(0.01)
0.702
(0.64)
0.515
0.723 (0.05)
0.587(0.07)
0.350)0.07)
(.236)
Table 6: Comparison with more baselines on Cora Dataset.
dataset. Meanwhile, the improvement is somewhat smaller, which implies PI-GNN may be more effective on homophilous datascts.
Noise type
Symmetric
Asymmetric
Actor
Noise Ratio
0.4
0.6
0.4
0.6
GCN
0.209(0.02)
0.208(0.02)
0.198(0.03)
0. 0..00)
PI-GNI w /o pe PI-GNN
0.216(0.01)
0.210(0.02)
0.201(0.02)
0.202(0.02)
0.218(0.02)
0.213(0.02)
(.204)0.02)
0.200(0.02)
Chameleon
GCN
0.251(0.03)
0.245 0.246(0.03)
0. 0.228 (0.03)
PI-GND w/o pe PI-GNN
0.264
0.249 (0.03)
0.242(0.05)
0.229
(0.02)).02)
0.251 (0.03)
0.239(0.05)
(0.04)
Table 7: Experimental results on heterophilous datasets.
F
Significance test results
We perform significance tests to verily whether PI-GNN outperforms the vanilla GNN model significantly using double-sided T-test in Table S We use python package "scipy.stats.ttestt Itest and report the average results over 10 different runs follows. PI-GNN is better than GAT because the absolute value of the t-statistic is relatively large and the p-value is small.
Method
Setting
T -statisticc
p-value
Cora
Symmetric Noise-0.8
4.78
0.001
GAT vs. PI-GNN
Asymmetric Noise-0.8
3.42
0.008
CiteSeer
Symmetric Noise-0.8
2.09
0.060
GAT vs. PI-GNN Asymmetric Noise-0.8
4.63
0.001
Table 8: Statistical significance tests.
G
Experimental results using clean label comparison
To observe the node classification results by training with PI labels from clean label comparison (which are obtained by comparing the clean class labels for two nodes), we did experiments on Cora, CiteSecr and PubMed with GCN and a noise ratio of 0.4 and 0.6. The results are shown in the Table g In most cases, clean label comparison can help the PI-GNN to combat noisy labels except for some challenging cases with asymmetric noise. One reason may be the inherent noise exists in clean node labels for Cora, where we cannot obtain perfectly clean PI label.
Experimental results with lower noise ratios
For the PI-GNN under lower noise ratios, we empirically verify its effectiveness on Cora and CiteSoer with the noise ratio of 0.1, which is shown in Table 10
19

---

Published in Transactions on Machine Learning Research (06/2023)
Noise type
Symmetric
Asymmetric
Cora
Noise Ratio
0.4
0.6
0.4
0.6
PI-GNN
0.664(0.00)
0.515(0.03)
(.587(0.07)
0.350(0.07)
Clean PI-GNN
0.671(0.03)
(..87 (0.20)
0.341(0.07)
CiteSeer
PI-GNN
0.59T(0.03)
0.432(0.07)
(0.06)
0.353(0.06)
Clean PI-GNN
0.605(0.04)
0.536(0.05) 0.447(0.07)
0.355(0.05)
PubMed
PI-GNN
0.638(0.04)
0.470(0.088
0.583(0.07)
0. (..05)
Clean PI-GNN
0.640(0.02)
0.485(0.07)
0.590
0.429(0.07)
Table 9: Experimental results using clean label comparison. Clean PI-GNN means the PI-GNN is trained with the PI labels from clean label comparison.
Noise type
Symmetric
Asymmetric
Cora
Noise Ratio
0.1
0,1
GCN
0.766(0.08)
0.762
PI-GNN w/o pc PI-GNN
0.769(0.03)
0.763(0.03)
0.772(0.02)
0.768 (0.0))
CiteSeer
Noise Ratio
0.1
0.1
GCN
0.642(0.03)
0.618(0.05)
PI-GNN w/o pc
0.648(0.04)
(0.02))
PI-GNN
0.659
0. 0.658 (0.05)
Table 10: Experimental results with lower noise ratios.
Comparison of the training time
We compare the training time of PI-GNN and the vanilla GNN as follows. We observe that using dual GNNss does not incur much higher computational cost because the two GNNs run in parallel rather than in soquence on the GPU and the major tino-consuming part is for data. loading and transforms rather than cewaadd/backwarr pass. Additionally, PI-GMN does not incur more inference cost than a vanilla model.
Dataset
Cora
CiteSeer
PubMed
WikiCS
OGB
Time- (CNns)
25.79
26.98
140.53
71.12
3930.
Time-PI PI-N- (s)
26.46
30.33
160.29
81.95
4117.00
J Additional comparison with baselines
In addition to Table 2 in the main paper, we provide the comparison on PubMed and WikiCS with baselines to further demonstrate the effectiveness of our PI-GNN, which shown as follows.
20

---

Published in Transactions on Machine Learning Research (06/2023)
Noise type
Symmetric (0i4
Asymmetric Noiso(0.4)
Test dataset:
PubMed / WikiCS
Decoupling GCE
0.627(0.05)
(.555
0.578(0.06)
0.465(0.05)
0.604(0.06) / 0.536(0.06)
0.524(0.08)
0.456(0.06)
APL
0.606(0.06) / 0.526(0.00)
0.524(0.08)
0.336(0.11))
Co-teaching
0.523(0.06) / 0..33(00.10)
0.433(0.11)
0.262
LPM-1
0.634(0.06) / 0..54
0.570(0.09)
(.404
T-Revision
0.603(0.04) / 0.542(0000)
0.554(0.08))
0.478(0010)
DivideMix
0.543(0.08) / 0.419(0.08)
0.566(0.07)
0.169(0.08)
PI-GNN (ours)
0. 0.638(0.04) / 0.562(0.04)
0.587
0..83(0.05)
Table 11: Additional comparison with baselines on two different datasets. We use the GCN the graph neural network backbone.
K Sensitivity to the model initialization
We provide the sensitivity analysis for PI-GNN on different model initializations in Table 12
The results
demonstrate that the performance is not sensitive to different model initializations, where the biggest gap among all the model initializations is smaller than 3%.
Noise type
Symmetric Noise
Asymmetric Noise
Noise ratio
0.6
0.6
Test dataset: Cora CiteSeer
Unilorm Initialization
0.514(0.05) / 0.443(0.05)
0.364(0.02)
0.324
Normal Initialization
0.548(0.03) / 0.426(0.05)
0.361(0.05)
0.345(0.01))
Constant Initialization
0.504(0.01) / 0.403(0.04)
0.339(0.03)
0.341
Kaiming Initialization
0.503(0.04) / 0.429(0.03)
0.333(0.02)
0.372(0.09)
Glorot Initialization (Ours)
0.515(0.03) / 0.432(0.00))
0.350(0.07) (..06)
Table 12: Sensitivity to the model initialization. Model architecture is the GCN.
L Results on a larger graph dataset
We evaluate our proposed PI-GNN on an even larger OGB-productss dataset, which has 2,449,029 nodes with 61,859, 140 edges and thus is much larger than the OGB-arxiv dataset (169,343 nodes and 1.166,243 edges) used in Table 1 The results are shown in Table 13 where our PI-GNN can still demonstrate promise compared to the vanilla GCN model and PI-GNN w/o predictive confidence.
Noise type
No Noise
Symmetric Noise
Asymmetric Noise
OGB products
Noise ratio
0.0
0.2
0.4
0.6
0.S
0.2
0.4
0.6
0.3
GCN
0.742(0.04)
0.709
0.861(0.09)
0.81210.01
0.471[0.08)
0.639[0.03]
0.635(6.03)
0.102(0.03)
0.053/0.61
PI-CWN w/o pe
0.711/0.03)
00.04(
0.669/0.03)
(0.06)
0.487(0.09)
0.700(0.09)
0.641 (..41(0.04)
0.136/0.01)
0.110(0.04)
0.727(0.08
0.706
0.677(0.06)
0.658(0.08)
0.506(0.05)
0.719(0.06)
0.667
0.196(0.06)
0.153(0.04)
Table 13: Test accuracy on the OGB BrB-uurouucs dataset for PI-GNN with GCN as the backbone. Bold numbers are superior results. Std. is shown in the bracket. w/o pe means that the PI labels not estimated using the predictive confidence but just the node connectivity.
M Results on using low-rank approximation for PI estimation
We estimate the PI labels by performing SVD on the input graph and using the low-rank representations as the estimation results. Suppose the rank of the representation is r, we tested the node classification performance under different values of (ie.., 1, 50, 100, 200, 400, 600, 1000). Note that use GCN and
21

---

Published in Transactions on Machine Learning Research (06/2023)
Cora as the GNN architecture and the dataset, respectively. During the low-rank approximation, we force the values of the smoothed reconstruction to be greater than 0 and less than 1 by value clipping. The results are updated in Table 14
Noise
Symmeetei Noize
Asyimeetiic
Noize
Noise ralio
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
GON
(.722(0.03)
0.613(0.07)
(.446(0.00))
0.285(0.07)
0.703(0.04)
0.514(0.06)
0.291
0.161(0.02)
Low-rank approximation-1
0.693(0.01)
0.587(0.03
0.418(0.05)
0.272/0.08 03)
0.688(0.06)
0.485(0.02
04]
0.269(0.02
0.110:0.05)
Low-rank approctmation-56
(.608)
0.593(0.03]
0.436(0.83)
0.269/0.01)
0.679(0.04)
0.502(0.06)
0.271(0.08)
0.132(0.08)
Low-rank approximation- 100
0.711(00095
0.603|0.05]
0.427)
0.217(0.05)
0.673(0.01]
0.515(0.06)
0.286(0.04)
Low-rank approximation-2U)
0.732(00044
0.607(0.04)
(.474(0.84)
0.282(0-033
0.694[0.05]
(.547(0.06)
0.294(0.01)
0.173/0.08)
Low-rank approximation-100 100
0.731(0.01)
0.611(0.09
0.196(
0.300(0.07)
0.686(0.01]
0.563(0.02
0.319(0.00)
0.183(0.05)
Low-rank approximation-600
(0.0))
0.626(0.02)
0.476(0.03)
0.298(0.00)
(.695(0.04)
0.534(0.07)
0.321(0.00)
0.201/0.00
Low-rank apprextmation-100 1000
0.704
0.610(0.08)
0.455)
0.256(0.07)
0.668(0.02)
0.508(0.05)
0.346(0.07)
0.188(0.09)
PT-GWE
0.739 (0.02)
(0.64
0.615(0.05)
0.29660.005
(.728(0.08)
0.687(0.07) (.300 (.292)0.06)
Table 14: Test accuracy on using low-rank approximation as the PI labels.
400 roughly achieves the best performance but still cannot outperform our PI-GNN. The reason might be that low-rank approximation is mainly designed for matrix compression, denoising and completion (https:/ wreb.stanford.edug www.fforrddddd/laassssseeeeeeeee (dtaas/ss66///111//////
which cannot capture the uncertainty of the PI labels in essence as in PI-GNN. Moreover, we observe that during low-rank approximation, the values of each node pair will quickly goes from negative values to values larger than 1, which is not suitable to be the PI labels for training. Although we dirrecly adopt the clipping approach to force the values to be in the range of 0 and 1, additional curated designs might be more beneficial. Finally, the rank r is an important hyperparameter tune in the low-rank approximation approaches, which requires extra manual tuning compared to PI-GNN.
N Discusssion on a different training strategy
In Table 15 and 16 we test. the performance of PI-GNN using a different training scheme, where we firstly pretrain the PI label estimation network fe for 400 epochs. Then we directly apply the pretrained fe to output the estimated PI labels for training ft. We show the results on Cora and CiteSeer datasets as follows:
Noise type
No Noise
Symmectic Noise
Asymmetri Noise
Urod-produeets 0.0
Noise rabin
D.0
0.2
0.4
U.S
1-22
D.4
0.6
0.8
P1-689 (Preirain)
0. 0.77200.1)
O. 0.300.00
0.872(0.04)
0.508(0.066
0.319(0.03)
0.728(0.04)
O. 0.56910.01)
O 33991.0066
0.245(0.06)
PI-DMM (Ours)
0.780(0.01)
(0.02)
0.664(0.03)
0.515(0.03)
0.20)
0.723/0.03)
0.587 (0.87)
0.350(0.07)
0.23210.061
Table 15: Test accuracy on Cora using different training scheme.
Notso type
No Noise
Symmetiic Noise
Asymmesiic Noise
OCIE-prodicts
Voise padio
0.0
0.2
0.4
0.6
0.8
D22
0.4
D&6
0.8
PI-DUN ( (Pretrain)
(0.61
0.63
0.606 0.606(0.04)
0. 0.220 (0.04)
O. (.804)(.04)
0.242(0.06)
PI-CUN (Ours)
0.604)
0.64)
(.591())033
0.442(00.07)
D. 2.245(0.055
(..2)
(.63)(
(0.66) 0.259
Table 16: Test accuracy on CiteSeer using a different training scheme.
where we can observe that firstly pretraining the PI label estimation network and freezing it during training the node classification model achieves similar classification performance compared to training scheme (c.f. Algorithm
O Additional cases of the teaser example
We note that there are additional cases that the conclusion of the leaser example might not hold. For instance, four nodes have the clean labels as x1 - 0, x2 - 0, x3 - 0, x4 - 0, after noise corruption, the node labels are changed to x1 1, 0, 0, x4 0. Therefore, the noise ratio of the node labels is 25% while that of the PI labels is 37.5%. We provide the comparison of noise ratios between PI labels and node labels on real-world datasets in Section in order to verify the validity of our proposed PI-GNN.
22

---

Published in Transactions on Machine Learning Research (06/2023)
P Noise ratio comparison between PI labels and node labels on real-world datasets
For evaluating the noise ratio of the PI labels on real-world graph datasets, we compare the noisc ratio of the PI labels and node labels on Cora and CiteSeer datasets. The results are shown as follows:
Cora
Noise ratio of the node labels (Symmetric)
0.2
0.4
0.6
0.8
Noise ratio of the PI labels
0.008
0.021
0.031
0.036
Noise ratio of the node labels (Asymmetrie)
0.2
0.4
0.6
0.8
Noise ratio of the PT labels
0.012
0.021
0.029
0.035
CiteSeer
Noise ratio of the node labels (Symmetric)
0.2
0.4
0.6
0.8
Noise ratio of the P1 labels
0.009
0.017
0.024
0.034
Noise ratio of the node labels (Asymmetrie)
0.2
0.4
0.6
0.8
Noise ratio of the PI labels
0.009
0.016
0.025
0.031
Table 17: Noise ratio comparison between P1 labels and node labels real-world datasets
From the above table, we can observe the noise ratio of the PI labels is indeed small on the real graph datasets, which able to justify the intuition of our PI-GNN.
Q Subgraph sampling on smaller graphs
We test the performance of PI-GNN (GCN as the backbone) after using subgraph sampling on smaller graphs, i.c., Cora and CiteSeer datascts. Specifically, we sample 15 and 10 neighbors for cach node in the 1st and 2nd layer of the GNN and set the batch size to 128. The results are shown as follows:
Noise type
No Nois
Symmetric Noise
Asymmettic Naian
OGD products
Naise
P1-GAM (8mbgraph sumplng)
(3. (.30) 0.780 0.739(0.02) 0.664(0.08)
0.728
0170810:33) 0.573 0.349/0.01)
0.246
(Ours)
0.518 0.296(0.00)
0.723 0.587(0.07) 0.350(0.07) 0.222
Table 18: Test accuracy on Cora using subgraph sampling.
Noise
OUT
Noien ratio
0.0
022
0.4
0.6
0.8
0.2
0.4
0.6
0.8
PT-GGN (Sulgraph sumpling)
0.67400L01
0.57510.07)
O. 0,268(0.08
0.528
0.315(0.33)
0.260
PI-UM (Ours)
0.684(0.03)
0.642(0.03)
(050)(
(..200)
0.245(0.08)
(.628(
0.353(0.06) (..31)
0.206/0.06)
Table 19; Test accuracy CiteSeer using subgraph sampling.
where we observe that applying subgraph sampling is less effective than using the entire graph as the input on smaller noise ratios.
R Comparison with baselines on OGB-arxiv
In Table 20] implement all the basclines on OGB-arxiv dataset and compare them with PI-GNN using subgraph sampling.
where we observe PI-GNN can still outperform all the baselines in most cases.
23

---

Published in Transactions on Machine Learning Research (06/2023)
Noise type
Symmetric Noise
Asymmetric Noise
Noise ratio
0.4
0.6
0.4
0.6
Test dataset:
OGB-arxiv
Decoupling
0.385(0.09)
0.347(0.05)
0.401(0.04)
0.029(0.01)
GCE
0..05
0.407(0.02)
0.391(0.06)
0.057(0.03)
APL
0.412(0.05)
0.375(0.05)
0.399(0.06)
0.062(0.01)
Co-teaching
0.461(0.04)
0.403(0.04)
0.410(0.05)
0.038(0.01)
LPM-1
0.450(0.00)
0.397(0.03)
0.439(0.06)
0.056(0.01)
T-Revision
0.417(0.04)
0.409(0.05)
0.427(0.05)
0.071(0.04)
DivideMix
0.448(0.01)
0.403(0.05)
0.438(0.05)
0.041
PI-GNN (ours)
0.447(0.033
0.461(0.01) 0.448)0.04)
0.069
Table 20: Comparative results with baselines on OGB-arxiv.
24


Reviews:
[{"review_id": "oWXMATk4D4", "reviewer": "Reviewer_AoRd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n- The proposed pairwise training method leverages structural pairwise interactions (PI) between nodes to improve label noise learning, which is a novel approach to address an important problem in graph learning applications.\n- The experimental results presented in the paper demonstrate the effectiveness of PI-GNN compared to several baselines on different datasets and GNN architectures.\n- The proposed confidence-aware PI estimation model adaptively estimates the PI labels, which are defined as whether the two nodes share the same node labels. This approach can help improve the accuracy of noisy node classification on graphs.\n- The decoupled training approach leverages the estimated PI labels to regularize a node classification model for robust node classification. This approach can help improve the robustness of GNNs to noisy labels.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a novel method for teaching Graph Neural Networks (GNNs) to accurately classify nodes under severely noisy labels. The authors propose a pairwise training method that leverages structural pairwise interactions (PI) between nodes to improve label noise learning.\n\nThe paper is well-structured and provides a clear introduction to the problem of noisy labels in graph learning applications. The authors highlight the importance of this problem and note that it is currently underexplored. They then introduce their proposed method, which builds on previous work in supervised metric learning and unsupervised contrastive learning.\n\nThe experimental results presented in the paper are comprehensive and provide strong evidence for the effectiveness of the proposed method. The authors compare their method with several baselines on test datasets Cora and CiteSeer with symmetric and asymmetric noise (noise rate ε = 0.4, 0.6). From Table 2, it is evident that PI-GNN outperforms different baselines with a considerable margin, improving the classification accuracy by 5.2% on Cora under symmetric noise (ε = 0.6) compared to the best baseline.\n\nMoreover, PI-GNN is able to outperform LPM-1, which relieves the strong assumption that auxiliary clean node labels are available. The authors also compare their method with traditional graph semi-supervised learning approaches and on more datasets in Appendix Section D and Section J, respectively.\n\nOverall, this paper presents a novel approach to address an important problem in graph learning applications - noisy labels. The proposed pairwise training method leverages structural pairwise interactions between nodes to improve label noise learning and provides strong experimental evidence for its effectiveness compared to several baselines.\n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Cons:\n- The proposed method may require more computational resources compared to traditional graph semi-supervised learning approaches due to its reliance on pairwise interactions between nodes. Thus, please provide complexity analysis and experiments.\n\n- The paper does not provide a detailed analysis of how sensitive the proposed method is to different hyperparameters or initialization methods. Further research may be needed to explore these aspects.\n\n- While the experimental results presented in the paper are promising, it is unclear how well PI-GNN would perform on larger or more complex datasets. Further research may be needed to explore this aspect as well.\n\n- Some most relevant papers are not cited and compared. Such as:\nLearning to Drop: Robust Graph Neural Network via Topological Denoising.\nThe 14th ACM International WSDM Conference (WSDM'21), 2021", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "HOjYqERgk4", "reviewer": "Reviewer_8ntc", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- The propose framework, PI-GNN is effective compared to the vanilla GCN and other noisy-robust GNN method.\n- The experiment results are comprehensive, covering different 3 GNN architectures, 2 noisy-label schemes with different nosy levels, etc. \n\n\n**Weaknesses**\n\n- The writing of Section 3 is not clear and some definitions are missing. For example,\n(1) what's the definition of $y_{\\text{structure}}$? It seems to me its equivalent to the graph adjacency matrix A_{i,j}?\n(2) what's the exact regularization loss used in Equation (5)? It seems to be missing in the paper.  \n\n- The effectiveness of the proposed method PI-GNN relies on a critical assumption about the underlying graph structure and their label distribution. Namely, the graph need to be *homophilous* (i.e., if two nodes are connected, then with high probability, they should have same node label). However, there are real-world graph datasets that violate the homophily assumption. See more examples in [1,2,3].\nIn other words, if the graph is non-homophily, then encouraging two connected nodes to have high similarity scores may not help the regularization, but even harm the predictive power of GNN.\n\n- The necessity of smoothly-estimated PI label matrix using another GNN seems questionable. For example, from Table 1 and Figure 3, PI-GNN w/o pc seems to have very similar performance compared to the PI-GNN, where the former is using binary PI label matrix, while the latter learned another GNN to estimate the PI label matrix, and produced an smooth prediction of the binary PI. The author should also consider other conventional methods that generate smooth estimation of the graph adjacency matrix. For example, if we just conduct SVD on the graph adjacency matrix, and obtain the low rank presentation for each node, and use that to generate the smooth estimation, how that compare with the PI-GNN framework that used GNN to do the PI estimation problem?\n\n- Note that learning the GNN $f_e$ for PI estimation in Equation (3) itself is an independent process to $f_t$. Why performing alternative update between $f_e$ and $f_t$ in Algorithm 1? Why not consider first solving $f_e$ up to converge, and use that optimal $f_e^*$ to output the final regularized loss used in Equation (5)?\n\n\n**Reference**\n- [1] Zhu et al. Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs. NeurIPS 2020\n- [2] Lim et al. Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods. NeurIPS 2021\n- [3] Chien et al. Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction. ICLR 2022\n\n \n ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper consider graph learning problems (i.e., node classification) where its node labels may be noisy and corrupted. Standard training objective on noisy labels may lead to overfitting and poor generalization. Thus, the author introduced an additional regularized loss function that encouraged two node embeddings to have higher similarity scores if that two nodes are connected and vice versa. In addition to the binary pairwise interactions (PI) label matrix used in the regularization loss, the author also proposed a smoothly estimated PI matrix obtained by via another standalone GNN and shown marginal improvement over the binary PI version.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I can not recommend acceptance of this paper, unless the following changes are made:\n- (1) Discuss the underlying assumption on graph/label distribution, namely graph homophily, exploited by the PI-GNN. The author need to be aware of the limitation of their PI-GNN framework on certain non-homophily graphs.\n- (2) More justification on why using another GNN to estimate the adjacency matrix of graph is a good choice, and compare with other baselines which can also generate smooth estimation of the graph adjacency matrix, such as other matrix decomposition methods.\n  ", "broader_impact_concerns": "No concerns on the ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nU4IstG1gl", "reviewer": "Reviewer_kfuH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength\nS1. To the best of my knowledge, estimating pair-wise interactions for noise-robust node classification is novel.\nS2. The proposed approach is easy to understand and follow.\n\nWeakness\nW1. (Major) The motivation of considering PI is unclear. On the first place, contrary to authors’ claims, noise rate for the PI labels actually might be larger than that of the pointwise noisy class labels (a simple example: x1-0,x2-0,x3-0,x4-0 -> x1-1,x2-0,x3-0, x4-0). While it might be that noise rates for the PI labels are usually small in practice, the authors did not verify it in real-world datasets. Additionally, it is also confusing why smaller PI label noise rate implies that it is easier to estimate PI. In fact, there are more possible pairwise interactions (|V|^2) than pointwise labels (|V|), which might indicate accurately predicting them requires larger model capacity.\n\nW2. (Major) The quadratic computational cost w.r.t. number of nodes is very high. While the authors suggest using subgraph sampling for large graphs, whether such approach will decrease the testing accuracy is unknown. I would suggest the authors to conduct additional experiments on smaller datasets to verify this point and also compare its scalability with other baselines. In addition, table 2 only reports comparison with baselines on two small datasets. I wonder if PI-GNN can still outperform baselines on large datasets (e.g., ogb datasets), after using subgraph sampling.\n\nW3. In experiments, noise is manually generated, which might not be able to reflect the true noise distribution in real-world scenarios. \n\nW4. In figure 1, ’Noise ratio’ is incorrectly placed: 50% should be placed in the pointwise noisy ratio part (uppper panel), and 25% should be placed in the PI noisy ratio part (bottom panel).\n\nW5. Lack of discussions on some closely-related topics such as out-of-distribution node detection and uncertainty prediction [1-3] whose target is to detect out-of-distribution or noisy samples. \n\n[1] Uncertainty aware semi-supervised learning on graph data. in NeurIPS 2020\n[2] Graph posterior network: Bayesian predictive uncertainty for node classification. in NeurIPS 2021\n[3] Energy-based Out-of-Distribution Detection for Graph Neural Networks. in ICLR 2023", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes to use pairwise interactions for semi-supervised node classification with noisy node labels. Experimental results show its effectiveness on some graph datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See my comments above.", "broader_impact_concerns": "So far as I can tell, there is no broader impact concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a graph learning method for the case where node labels can be noisy, and shows its good performance.  The reviewers' requests, including improving clarity, additional experiments, and discussion on missing related work, have been well addressed.  

In the revision, there are wrong uses of citet and citep for example in the end of Section 3.  Please correct them along with typos for the final version.
Decision: Accept (poster)