Year: 2021

Abstract:
Semi-supervised node classification, as a fundamental problem in graph learning, leverages unlabeled nodes along with a small portion of labeled nodes for training. Existing methods rely heavily on high-quality labels, which, however, are expensive to obtain in real-world applications since certain noises are inevitably involved during the labeling process. It hence poses an unavoidable challenge for the learning algorithm to generalize well. In this paper, we propose a novel robust learning objective dubbed pairwise interactions (PI) for the model, such as Graph Neural Network (GNN) to combat noisy labels. Unlike classic robust training approaches that operate on the pointwise interactions between node and class label pairs, PI explicitly forces the embeddings for node pairs that hold a positive PI label to be close to each other, which can be applied to both labeled and unlabeled nodes. We design several instantiations for PI labels based on the graph structure and the node class labels, and further propose a new uncertainty-aware training technique to mitigate the negative effect of the sub-optimal PI labels. Extensive experiments on different datasets and GNN architectures demonstrate the effectiveness of PI, yielding a promising improvement over the state-of-the-art methods.

heading: 1 INTRODUCTION
text: Graphs are ubiquitously used to represent data in different fields, including social networks, bioinformatics, recommendation systems, and computer network security. Accordingly, graph analysis tasks, such as node classification, have a significant impact in reality (Lan et al., 2020). The success of machine learning models, such as graph neural networks (GNNs) on node classification relies heavily on the collection of large datasets with human-annotated labels (Zhou et al., 2019). However, it is extremely expensive and time-consuming to label millions of nodes with high-quality annotations. Therefore, when dealing with large graphs, usually a subset of nodes is labeled, and a wide spectrum of semi-supervised learning techniques have emerged for improving node classification performance (Zhu et al., 2003; Zhou et al., 2003; Kipf & Welling, 2017).
Although achieving promising results, these techniques overlook the existence of noisy node labels. For instance, practitioners often leverage inexpensive alternatives for annotation, such as combining human and machine-generated label (Hu et al., 2020), which inevitably yields samples with noisy labels. Since neural networks (including GNNs) are able to memorize any given (random) labels (NT et al., 2019; Zhang et al., 2017), these noisy labels would easily prevent them from generalizing well. Therefore, training robust GNNs for semi-supervised node classification against noisy labels becomes increasingly crucial for safety-critical graph analysis, such as predicting the identity groups of users in social networks or the function of proteins to facilitate wet laboratory experiments, etc.
A natural solution is extending the robust training techniques specifically designed for image data into graphs, such as by estimating noise transition matrix (NT et al., 2019), re-weighting training samples (Xia et al., 2021; Li et al., 2021; Dong et al., 2020) and modifying model structure (Li et al., 2020a), which achieve promising results. However, there are two limitations. Firstly, some techniques require a clean set of node and label pairs for training, which are not practical to obtain under extremely noisy settings (Li et al., 2021). Secondly, all of them solely rely on the noisy pointwise input-label interactions during training. Although some of them implicitly exploit other attributes (the clean graph structure, etc.) that encodes the pairwise information between nodes, such as aggregating features by GNNs on the input graphs or launching label propagation to get the pseudo
labels for unlabeled nodes, they still operate on the noisy embeddings for inference, which may lead to limited performance. Thus, designing a robust GNN training technique for node classification against noisy labels is still a challenging problem.
In this paper, we propose a novel framework, which explicitly leverages the Pairwise Interactions in Graph Neural Network (PI-GNN) to perform robust training without auxiliary clean labels. The PI, which is based on the similarity of two nodes, is meaningful for GNN against noisy labels because the noise rate for the PI labels can be much lower than that of the pointwise noisy class labels (Figure 1(a)). Consider two nodes from the same class have the same noisy labels, their similarity is still high w.r.t. the PI, which is helpful for the model to learn. Meanwhile, the pointwise information will inject both of these noisy class labels into GNN, which hurts its generalization performance.
To ensure an effective PI learning procedure, two important components are highlighted. The first one is how to design informative PI labels. We propose two instantiations: 1) we use the adjacency matrix as PI labels so that if two nodes are connected, their PI label is positive. 2) We compare the noisy labels, i.e., two nodes that have the same class labels are given a positive PI label. During training, PI-GNN explicitly forces the embedding similarity of two nodes to be close to the PI labels. The second one is how to deal with the sub-optimal PI labels during regularization. It is critical to notice that the two instantiations are not optimal. Firstly, connected nodes do not guarantee to have the same (clean) class label, so their PI label is not always positive. Secondly, the class labels are noisy, and comparing them to obtain the PI labels inevitably introduces noise. To address this problem, we introduce uncertainty estimation, which quantifies the reliability of each PI label (Figure 1(b)). Specifically, we propose to exploit a separate GNN only trained with the PI learning objective and calculate a confidence mask for each node pair from its PI prediction. Then, we filter out uncertain pairs by weighting the PI loss from another GNN jointly trained with the PI loss and node classification loss. Figure 1(c) shows the accuracy of GNNs trained on the node pairs with higher and lower confidence on CiteSeer (Yang et al., 2016) and the former one performs better, which justifies the intuition of PI-GNN. Our main contributions are summarized as follows:
• We propose robust GNNs against noisy labels for node classification, which serve as a crucial step towards the reliable deployment of GNNs in complex real-world applications. • We introduce a promising strategy to explicitly model the pairwise interactions and an uncertainty estimation approach to mitigate the negative effects of sub-optimal PI labels. • We demonstrate PI-GNN can be effectively applied on different datasets, GNN architectures and different noise types and rates, e.g., improving the test accuracy by 6.7% on CiteSeer with 80% asymmetric noise.

heading: 2 RELATED WORK
text: Graph neural networks. Graph neural networks have been widely used to model the graphstructured data with various architectures, such as graph convolutional network (GCN) (Kipf & Welling, 2017), graph attention network (GAT) (Velickovic et al., 2018), GraphSAGE (Hamilton
et al., 2017), Graph Isomorphism Network (GIN) (Xu et al., 2019), Simple Graph Convolution (SGC) (Wu et al., 2019), etc. Common graph analysis tasks, including node classification (Park & Neville, 2019; Oono & Suzuki, 2020), link prediction (Baek et al., 2020; Zhang & Chen, 2018), graph classification (Bacciu et al., 2018; Errica et al., 2020), graph generation (Liao et al., 2019; Shi et al., 2020), have been widely studied in literature. However, only a few works focused on training robust GNNs against noisy labels, such as by loss correction (NT et al., 2019) for graph classification, sample re-weighting (Xia et al., 2021; Li et al., 2021) for node classification. None of them exploited explicit pairwise interactions, which are compared with our PI-GNN in Section 5.4. Bui et al. (2017); Stretcu et al. (2019) utilized graph structures for semi-supervised learning but with clean labels.
Neural networks with noisy labels. Methods for neural networks against noisy labels can be roughly categorized into three types (Han et al., 2020b), i.e., approaches from the perspective of data (van Rooyen & Williamson, 2017), objective (Reed et al., 2015; Miyato et al., 2019) and optimization (Arpit et al., 2017). Methods based on data mainly built the noise transition matrix to explore the data relationship between clean and noisy label by an adaptation layer (Sukhbaatar et al., 2015), loss correction (Patrini et al., 2017) and prior knowledge (Han et al., 2018a). Methods based on objective modified the learning objective by regularization (Han et al., 2020a), reweighting (Liu & Tao, 2016; Wang et al., 2017) and loss redesign (Thulasidasan et al., 2019). Methods based on optimization mainly changed the optimization policy, such as by memorization effect (Jiang et al., 2018), self-training (Ren et al., 2018) and co-training (Yu et al., 2019). Wu et al. (2020) proposed to use the similarity loss for noisy labels on image data but it relied on the noisy transition matrix, which is sensitive to the matrix estimation quality and cannot use the graph structure for regularization. In this paper, we extend several approaches from each category to compare with PI-GNN in Section 5.4.

heading: 3 PRELIMINARY
text: Graph Neural Networks. Let G = (V,E) be a graph with node feature vectors Xv for v ∈ V and edge set E. GNNs use the graph structure and node features Xv to learn a representation vector of a node hv, or the entire graph hG, which usually follow a neighborhood aggregation strategy and iteratively update the representation of a node by aggregating representations of its neighbors. After k iterations of aggregation, a node’s representation captures the structural information within its k-hop network neighborhood. Formally, the k-th layer of a GNN is
a(k)v = AGGREGATE (k) ({ h(k−1)u : u ∈N (v) }) , h(k)v = COMBINE(k) ( h(k−1)v ,a (k) v ) , (1)
where h(k)v is the feature vector of node v at the k-th layer. h (0) v = Xv. N (v) denotes the neighboring nodes of v. The choices of AGGREGATE (k)(·) and COMBINE (k)(·) can be diverse among different GNNs. For example, in GCN, the element-wise mean pooling is used, and the AGGREGATE and COMBINE steps are integrated as follows:
h(k)v = ReLU ( W ·MEAN { h(k−1)u ,∀u ∈N (v)∪{v} }) , (2)
where W is a learnable matrix. For node classification, each node v ∈V has an associated label yv, the node representation h(K)v of the final layer is used for prediction.
Label-noise representation learning for GNNs. Let Xv be the feature and yv be the label for node v, we deal with a dataset D = {Dtr,Dte} which consists of training set Dtr = {(A,Xv,yv)}v∈V that is drawn from a corrupted distribution D = p(A,X ,Y ) where Y denotes the corrupted label. Let p(A,X ,Y ) be the non-corrupted joint probability distribution of features X and labels y, and f ∗ be the (Bayes) optimal hypothesis from X to y. To approximate f ∗, the objective requires a hypothesis spaceH of hypotheses fθ (·) parametrized by θ . A robust algorithm against noisy labels contains the optimization policy to search throughH in order to find θ ∗ that corresponds to the optimal function in the hypothesis for Dtr : fθ∗ ∈H, and meanwhile is able to assign correct labels for Dte.

heading: 4 PROPOSED APPROACH
text: In this section, we introduce our proposed PI-GNN, which mitigates the negative effects of noisy labels for semi-supervised node classification by explicitly exploiting the pairwise interactions in
GNNs. In what follows, we will first provide a method overview and then illustrate the learning objective to enhance the pairwise interactions in PI-GNN (Section 4.1). We introduce the uncertaintyaware robust training approach by using PI for regularization in Section 4.2.
Overview. Figure 2 demonstrates the overview of PI-GNN, which is composed of two different GNNs. The first one is only trained with the PI learning objective, whose outputs are used to generate the confidence mask. We denote it as the mask generator fm. The second GNN is trained with both the PI learning objective and the noisy node classification objective, which is used to perform the node classification task. We denote it as the task executor ft . The PI loss for each node pair in the task executor is multiplied by the confidence mask from the mask generator in order to reduce the uncertainty caused by the collected sub-optimal PI labels.

heading: 4.1 LEARNING FROM PAIRWISE INTERACTIONS
text: Let us suppose the semantic class labels for certain nodes in the training set Dtr = {(A,Xv,yv)}v∈V are corrupted. Since, ultimately, we are interested in finding a GNN model f parametrized by θ that minimizes the generalization error on a clean test set Dte, a natural solution is to exploit additional information for the learning algorithm to find a robust parameter θ ∗ in the hypothesis spaceH. One straightforward candidate for such information is leveraging the pairwise interactions between two nodes to perform extra regularization, whose learning objective is shown to hold a much smaller noise rate than that with the noisy class labels (Figure 1(a)).
Construct informative PI labels. In order to perform PI learning, the first step is to construct reliable PI labels yPI ∈ R|V |×|V | to alleviate the negative effect of noisy labels for the GNNs. Here |V | is the cardinality of the vertex set on the input graph G. While a reasonable choice of yPI is by comparing whether two nodes have the same class label y and assign those with the same class label a positive PI label, it is impossible to obtain such PI labels with noisy class labels y. Therefore, assume i, j ∈V and ∧ as the “and” operator, we relieve such condition by proposing two relaxed instantiations:
y1PI = A or y 2 PI = yi∧ y j, (3)
where the first one models the PI between two nodes by their connectivity in the input adjacency matrix A. The second one firstly performs label propagation (Zhu & Ghahramani, 2002) to get the class labels for each node in the input graph. Then it assigns the PI label of node pair < i, j > to 1 if their noisy class labels are the same and 0 otherwise. Both of them are not optimal because firstly, the fact that two nodes are connected does not mean their clean class labels are the same. Secondly, comparing the noisy class labels also creates noisy PI labels if the same clean class labels are corrupted to two different noisy labels despite the noise rate is greatly reduced. We provide an uncertainty-aware training approach in Section 4.2 to alleviate this problem.
Learning objective for enhancing pairwise interactions. Given node embeddings h which is calculated by h = f (A,X ,θ), let hTi ·h j be the dot product between two embeddings, and P(hTi ·h j)
be the estimated probability of the node pair < i, j > has the positive pairwise interaction, the PI learning objective `PI ∈ R|V |×|V | is formulated as follows:
Algorithm 1 PI-GNN: Learning Pairwise Interactions for GNNs against noisy labels
Input: Input graph G= (V,E,X) with noisy training dataDtr = {(A,Xv, ȳv)}v∈V , randomly initialized GNNs fm and ft with parameter θm and θt , weight for PI loss β , pretraining epoch K for fm. Total training epoch N. Output: Robust GNN ft against noisy labels. for epoch = 0;epoch < N;epoch++ do
if epoch ≤ K then Set M = 1, update the parameter θm of the mask generator fm by Equation 4 and the parameter θt of the task executor ft by Equation 7. else Estimate the confidence mask M by Equation 5 with the mask generator fm. Calculate the PI loss `tPI for the task executor ft by Equation 6. Update the parameter θm of the mask generator fm by Equation 4 and the parameter θt of the task executor ft by Equation 7.
end end return The model ft .
`PI (h;yPI) = ∑ <i, j>∈B+PI − logP(hTi ·h j)+ ∑ <i, j>∈B−PI − log(1−P(hTi ·h j)), (4)
where B+PI ,B − PI denote the node pairs that hold the positive and negative PI labels, respectively.

heading: 4.2 UNCERTAINTY-AWARE ROBUST TRAINING
text: Uncertainty estimation. In order to train a robust GNN that is not sensitive to the sub-optimal PI labels, we resort to uncertainty estimation and reduce the negative effect of the node pairs that the model is uncertain about during regularization (Equation 4). Specifically, we measure the uncertainty by calculating the confidence map of the PI predictions as follows:
M(i, j) = {
σ(hTi ·h j), yPI(i, j) = 1 1−σ(hTi ·h j), yPI(i, j) = 0
(5)
where yPI(i, j) denotes the PI label between node i and j and σ(·) is the sigmoid function. The confidence map M measures the uncertainty by looking at the closeness between the prediction and the given PI label. If the prediction becomes close to the given labels more easily, then the reliability of the PI labels is higher and more attention should be paid for such node pairs.
Therefore, we introduce a re-weighting mechanism for the PI loss `PI = `PI⊗M where the re-weighted PI loss is obtained by multiplying the confidence mask and its original PI loss in an element-wise way. However, since the GNN is trained with the noisy class labels y at the same time, the confidence mask M cannot be estimated well.
Decoupling with two branches. In this paper, as shown in Figure 2, we propose to decouple the confidence mask estimation and node classification by using two separate GNNs, which are referred as a mask generator fm and a task executor ft . The mask generator generates the confidence mask M by only learning with the PI objective. The task executor uses the mask M from the mask generator to re-weight different node pairs in order to combat the noisy PI labels as follows:
`tPI = ` t PI⊗M, (6)
where `tPI denotes the PI loss for the task executor ft and ⊗ means element-wise multiplication. The PI learning procedure allows for explicitly exploiting the pairwise interactions between two nodes, resulting in a GNN that is affected less by the noisy class labels. Wu et al. (2020) employed similarity labels y2PI for regularization. However, it transforms the noise transition matrix estimated for noisy class labels y to correct the similarity labels, which is sensitive to the matrix estimation quality. Meanwhile, the pairwise interactions from the input adjacency matrix cannot be explored.
Overall training. Put them together, we introduce a new robust training objective for node classification against noisy labels on GNNs, leveraging the pairwise interactions in Section 4.1. The key idea is to perform the node classification task by the task executor ft while regularizing ft to produce similar embeddings for nodes that have a closer PI and vice versa. The overall uncertainty-aware training objective for the task executor ft is formulated as:
`t = ` t cls( ft(A,X ,θt),y)+β · `tPI , (7)
where β is a hyperparameter to balance the node classification loss `tcls and the PI loss ` t PI . Besides, the mask generator is trained only by the PI loss `mPI and provides confidence mask following Equation 5, which does not touch noisy class labels for learning. During the inference stage, we discard the mask generator fm and only use the task executor for evaluation, which does not affect the inference speed.
Practically, the learning procedure relies heavily on the quality of the uncertainty estimation by fm. Therefore, we pretrain the mask generator fm for K epochs and re-weight the PI loss of the task executor `tPI by Equation 6. Note that the task executor is still optimized by the un-reweighted PI loss and the noisy node classification loss during pretraining the mask generator. The training outline is presented in Algorithm 1.

heading: 5 EXPERIMENTS AND RESULTS
text: In this section, we present empirical evidence to validate the effectiveness of PI-GNN on different datasets with different noise types and ratios.

heading: 5.1 EXPERIMENTAL SETTING
text: Datasets. We used five datasets to evaluate PI-GNN, including Cora, CiteSeer and PubMed with the default dataset split as in (Kipf & Welling, 2017) and DBLP (Pan et al., 2016) as well as WikiCS dataset (Mernyei & Cangea, 2020). For the latter two datasets, we used the first 20 nodes from each class for training and the next 20 nodes for validation. The remaining nodes for each class are used as the test set. The statistics of these datasets are summarized in Appendix Section B.
Since all datasets are clean, following Patrini et al. (2017), we corrupted these datasets manually by the noise transition matrix Qi j = Pr(y = j | y = i) given that noisy y is flipped from clean y. Assume the matrix Q has two representative structures: (1) Symmetry flipping (van Rooyen et al., 2015); (2) Asymmetric pair flipping: a simulation of fine-grained classification with noisy labels, where labelers may make mistakes only within very similar classes. Note the asymmetric case is much harder than the symmetry case. Their precise definition is in Appendix Section A.
We tested four different noise rates ε ∈ {0.2,0.4,0.6,0.8} in this paper for two different noise types, which cover lightly and extremely noisy supervision. Note that in the most extreme case, the noise rate 80% for pair flipping means 80% training data have wrong labels that cannot be learned without additional assumptions.
Implementation details. We used three different GNN architectures for evaluation, i.e., GCN, GAT and GraphSAGE, which are implemented following the package torch-geometric (Fey & Lenssen, 2019). All of them have two layers. Specifically, the hidden dimension of GCN, GAT and GraphSAGE is set to 16, 8 and 64, respectively. GAT has 8 heads for attention calculation in the first layer and 1 head in the second layer. The mean aggregator is used for GraphSAGE. We applied Adam optimizer (Kingma & Ba, 2015) with a learning rate of 0.01 for GCN and GraphSAGE and 0.005 for GAT. The weight decay is set to 5e-4. We conducted training for 400 epochs on a Tesla P40. The regularization loss weight β is set to |V |2/(|V |2−M)2, where |V | is the number of nodes and M is the number of edges in the input graph G. The number of pretraining epochs K is set to 50 and the total epoch K is 400. The PI label is constructed based on the adjacency matrix by default. We tuned all the hyperparameters on the validation set and reported the node classification accuracy on the clean test set. Details about the ablation studies on these factors are shown in Section 5.5. Each experiment is repeated for 10 times with random seeds from 1 to 10.

heading: 5.2 EFFECTIVENESS ON DIFFERENT DATASETS
text: We evaluated the effectiveness of PI-GNN on five datasets with different noisy labels and noise rates, which is shown in Table 1 with GCN as the backbone. Specifically, we are interested to observe 1) whether the introduced pairwise interactions between nodes can improve a vanilla GNN against noisy
labels and 2) whether the uncertainty estimation to combat the sub-optimal PI labels is beneficial for the test set accuracy. Therefore, we compared the accuracy of a vanilla GNN, PI-GNN trained without the mask generator for uncertainty estimation (PI-GNN wo/ ue) and PI-GNN.
From Table 1, we made several observations: Firstly, the GNN trained with the PI learning objective is more robust to noisy labels, where both PI-GNN wo/ ue and PI-GNN perform much better than a vanilla GNN. Secondly, by performing uncertainty estimation by an additional mask generator and weighting the PI loss with the confidence mask, the negative effect of noisy labels is further reduced. For instance, PI-GNN improves the accuracy by 1.1% with the symmetric noise (noise ratio ε = 0.8) on Cora compared to PI-GNN wo/ ue, which justifies the effectiveness of our design. Thirdly, the PI learning objective does not help the GNN with the clean node labels, e.g., 80.4% of a vanilla GCN vs. 78.0% of PI-GNN on Cora, which illustrates the PI-GNN helps to combat noisy supervision rather than inherently improve the node classification with purely clean node labels. Additional results on heterophilous datasets and with lower noise ratios are presented in Appendix Sections D and G.

heading: 5.3 PERFORMANCE ON DIFFERENT GNN ARCHITECTURES
text: We evaluated our proposed PI-GNN on different GNN architectures, i.e., GAT and GraphSAGE. The experiments are conducted on Cora and CiteSeer dataset, which are shown in Table 2. As can be observed, our proposed approach performs similarly on GAT and GraphSAGE compared to the results on GCN, where the regularization of the pairwise interactions and the uncertainty estimation are both beneficial for model generalization even with extremely noisy supervision. Moreover, the uncertainty estimation is more effective on GraphSAGE. For example, in the Cora dataset, PI-GNN improves PI-GNN wo/ ue by 4.2% and 3.1% on average under symmetric noise and asymmetric noise, respectively, which is larger than that for GAT and GCN. It may suggest the mean aggregator in GraphSAGE is more susceptible to the sub-optimal PI labels. We provide significance test on the results of GAT in Appendix Section E.

heading: 5.4 COMPARISON WITH BASELINES
text: In order to further demonstrate the competitive performance of PI-GNN, we compared with several powerful baselines for combating noisy labels in literature. For a fair comparison, we used the same
GNN architecture, i.e., GCN, and the same overlapping hyperparameters during implementation. The other method-specific hyperparameters are tuned according to the original paper on the validation set. Specifically, we compared with noise-transition matrix-based method, T-revision (Xia et al., 2019), robust loss functions, such as Generalized Cross Entropy (GCE) loss (Zhang & Sabuncu, 2018) and Active Passive Loss (APL) (Ma et al., 2020), optimization-based approaches, such as Co-teaching (Han et al., 2018b), Decoupling (Malach & Shalev-Shwartz, 2017) and DivideMix (Li et al., 2020b). We also compared with Label Propagation and Meta learning (LPM) (Xia et al., 2021), a method that is specifically designed for solving label noise for node classification but uses a small set of clean nodes for assistance. We reported the classification accuracy on Cora and CiteSeer with symmetric noise (noise rate ε = 0.4,0.6) and asymmetric noise (noise rate ε = 0.2,0.4) in Table 3.
From Table 3, PI-GNN outperforms different baselines with a considerable margin, especially under extremely noisy supervisions, e.g., improving the classification accuracy by 3.7% on CiteSeer under the symmetric noise (noise rate ε = 0.6). Moreover, PI-GNN is able to outperform LPM-1, which relieves the strong assumption that auxiliary clean node labels are used for training. We present comparison with traditional graph semi-supervised learning approaches in Appendix Section C.

heading: 5.5 ABLATION STUDIES
text: Sensitivity to the pretraining epoch of the mask generator. We investigated whether the performance of PI-GNN is sensitive to the number of pretraining epochs for the mask generator. The experimental results on Cora and CiteSeer with GCN under symmetric and asymmetric noise (noise rate ε = 0.6) are shown in Figure 3 (a) and (b). As can be observed, pretraining the mask generator for K epochs is effective for improving the generalization on the clean test set. Given a small K, the confidence mask is not estimated well which is not helpful to apply it on the task executor for regularization. Meanwhile, K should not be too large in order to sufficiently regularize the task executor by the uncertainty-aware training objective. K is set to 50 for all the experiments.
The effect of regularization weight. To observe whether the regularization loss weight β matters to the model performance, we trained PI-GNN with different values of β , i.e., 0.005, 0.01, 0.05, 0.1, 0.2, 0.5 and compared with the value β ′ = |V |2/(|V |2−M)2 which is aware of the sparsity of the graph in Figure 3 (c) and (d). We conducted experiments on Cora and CiteSeer with GCN and showed the results with symmetric and asymmetric noise (noise rate ε = 0.4). From the figure, PI-GNN is sensitive to the choice of regularization loss weight β . On both datasets with different noise types, PI-GNN trained with β ′ achieves the best clean test accuracy, and simultaneously avoids heavy tuning procedure on the validation set.
The effect of pairwise interaction labels. To demonstrate the importance of informative PI labels for PI-GNN, we tested PI-GNN under different PI labels in addition to those based on adjacency matrix,
namely by 1) noisy class label comparison and 2) random labels. We also compared with using the task executor ft to generate the confidence mask without introducing the mask generator. We used GCN as the backbone and tested on two datasets, i.e., Cora and CiteSeer and different noise types. The results are shown in Table 4. From Table 4, using PI labels based on the graph structure obtains the best performance. Directly comparing the similarity of the noisy class labels to get the PI labels or using randomly generated PI labels incurred the worst performance because of the heavy noise during the PI label generation. Moreover, removing the mask generator for confidence
mask generation decreases the test accuracy because the node embeddings are optimized by the noisy class labels as well, which is ineffective for the task executor to generalize even with uncertaintyaware training by itself (32.9% vs. 34.7% for Cora with asymmetric noise and a noise rate of 60%). We also compare with training with the clean PI labels, which can be regarded as an upper bound of our method in Appendix Section F.
Application of PI-GNN on label-noise baselines. To observe whether PI-GNN is able to improve the generalization ability for different label-noise baseline models, we extended three representative approaches, i.e., T-revision (Xia et al., 2019), APL (Ma et al., 2020) and DivideMix (Li et al., 2020b) by adding the PI learning objective during training. Specifically, we used the sum of the original loss and the PI loss to optimize the GNN. The weight for PI loss is set to β ′. We chose GCN as the backbone and reported the test accuracy on Cora with both symmetric noise (noise rate ε = 0.4) and asymmetric noise (noise rate ε = 0.4) in Table 5 Left. As the result shows, PI-GNN is orthogonal to those robust baseline models, which is potentially useful for improving their performance without bells and whistles. For instance, The test set accuracy is improved by 4.6% on DivideMix under the symmetric noise and thus demonstrates the universality of our proposed PI-GNN.
Different architectures for two branches. PI-GNN allows for a flexible choice of the architectures for the mask generator and the task executor, where a light-weight mask generator can help a large task executor for node classification during the uncertainty-aware training. In what follows, we used three different mask generator-task executor pairs, namely GCN-GAT, GCN-GraphSAGE and GAT-GraphSAGE. The number of parameters for GCN, GAT, GraphSAGE is 0.02, 0.09 and 0.18 M, respectively. The comparison with using the same architectures are shown in Table 5 Right. From Table 5 Right, using a light-weight GNN for the mask generator is able to further improve the clean test accuracy, which is promising for efficient deployment of PI-GNN on real-world graph datasets.

heading: 6 CONCLUSION
text: In this paper, we proposed PI-GNN, a simple but effective learning paradigm for helping the graph neural networks to generalize well with noisy supervision. Our key idea is to leverage the pairwise interactions between nodes to explicitly adjust the similarity of those node embeddings during training. In order to alleviate the negative effect of the collected sub-optimal PI labels, we introduce a new uncertainty-aware training approach to reweight the PI learning objective by its prediction confidence. We conducted extensive experiments to demonstrate that PI-GNN can train GNNs robustly under extremely noisy supervision, which serves as a crucial step towards the reliable deployment of GNNs in complex real-world applications.
ETHICS STATEMENT
This paper doesn’t raise any ethics concerns. This study doesn’t involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potentially conflicts of interest and sponsorship, discrimination/bias/fairness concerns, privacy and security issues, legal compliance, and research integrity issues.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of experimental results, we will provide source codes of this paper using an anonymous repository link in the discussion phase.

heading: A DEFINITION OF NOISE
text: The definition of transition matrix Q is as follows. n is number of the class.
Asymmetric pair flipping:
Q =  1− ε ε 0 . . . 0 0 1− ε ε 0 ... . . . . . . ...
0 1− ε ε ε 0 . . . 0 1− ε
 , (8)
Symmetry flipping:
Q =  1− ε εn−1 · · ·
ε n−1 ε n−1ε
n−1 1− ε ε n−1 · · · ε
n−1 ... . . . ...
ε n−1 · · ·
ε n−1 1− ε ε n−1ε
n−1 ε n−1 · · · ε n−1 1− ε
 . (9)

heading: B DATASET DETAILS
text: Here we provide the details of graph datasets for node classification.

heading: C COMPARISON WITH TRADITIONAL GRAPH SEMI-SUPERVISED LEARNING BASED APPROACHES.
text: For the comparison with the traditional semi-supervised graph embedding methods, we follow the same experimental setting and compare with ICA (Lu & Getoor, 2003), Planetoid (Yang et al., 2016) and Label Propagation (LP) (Zhu & Ghahramani, 2002) on Cora as follows. The result shows the advantage of PI-GNN across different noise ratios.

heading: D EXPERIMENTAL RESULTS ON HETEROPHILOUS DATASETS
text: We perform extra experiments on heterophilous datasets (Ma et al., 2021). The results are in the following table. It shows that PI-GNN is still able to outperform the vanilla one except for one case
in Chameleon dataset. Meanwhile, the improvement is somewhat smaller, which implies PI-GNN may be more effective on homophilous datasets.

heading: E SIGNIFICANCE TEST RESULTS
text: We perform significance tests to verify whether PI-GNN outperforms the vanilla GNN model significantly using double-sided T-test. We use python package “scipy.stats.ttest1samp” and report the average results over 10 different runs as follows. PI-GNN is better than GAT because the absolute value of the t-statistic is relatively large and the p-value is small.

heading: F EXPERIMENTAL RESULTS ON USING CLEAN PI LABELS
text: To observe the node classification results with clean PI labels, we did experiments on Cora, CiteSeer and PubMed with GCN and a noise ratio of 0.4 and 0.6. The results are shown in the Table 10. In most cases, clean PI labels can help the PI-GNN to combat noisy labels except for some challenging cases with asymmetric noise. One reason may be the inherent noise exists in clean node labels for Cora, where we cannot obtain perfectly clean PI label.

heading: G EXPERIMENTAL RESULTS WITH LOWER NOISE RATIOS
text: For the PI-GNN under lower noise ratios, we empirically verify its effectiveness on Cora and CiteSeer with the noise ratio of 0.1, which is shown in the following table.

Review summaries:
This paper focuses on semi-supervised node classification with noisy node labels. The authors propose a novel learning objective called the pairwise interactions, which encourages node pairs holding positive PI labels to have close node embeddings. Extensive experiments show promising results.

This paper is mainly dedicated to developing a robust model against noisy labels. Since it is observed from Fig. 1(a) that the noise ratio for the PI labels is lower than that of the point-wise labels, the authors introduce positive and negative sample pairs (Pair Interactions) to enhance model capabilities, which is likely contrastive learning. Moreover, the labels of positive and negative sample pairs may not be confident, e.g., sub-optimal PI labels with noises. To address this problem, the authors introduce another branch (Mask Generator) trained on pair interaction learning objective to generate a confidence map to reweight the loss of PI learning objective in Task Executor. Experimentally, the accuracies of the proposed method are higher than baselines in the case of noisy labels.

This paper proposes the PI-GNN (Pairwise Interactions in Graph Neural Network) model for semi-supervised tasks for nodes with noisy labels. It employs a new auxiliary task to predict PI (pairwise interactions) labels using a pair of nodes. Since it is costly to get clean PI labels, the authors use (1) two instantiations of PI labels: adjacency matrix and class agreement after label propagation and (2) uncertainty-aware (pre-)training with two model branches. The experiments on five real-world benchmarks demonstrate that PI-GNN outperforms GNN models (GCN, GraphSAGE, and GAT) and methods for noisy label settings (Decoupling, GCE, DivideMix, and so on.) on various noise levels and settings. PI label learning and uncertainty-aware training both contribute to performance improvement.