Title: Deep Insights into Noisy Pseudo Labeling on Graph Data
Venue: NeurIPS 2023
Accepted: yes
Keywords: Pseudo labeling

Graph data

Error analysis

Cautious
Abstract: Pseudo labeling (PL) is a wide-applied strategy to enlarge the labeled dataset by self-annotating the potential samples during the training process. Several works have shown that it can improve the graph learning model performance in general. However, we notice that the incorrect labels can be fatal to the graph training process. Inappropriate PL may result in the performance degrading, especially on graph data where the noise can propagate. Surprisingly, the corresponding error is seldom theoretically analyzed in the literature. In this paper, we aim to give deep insights of PL on graph learning models. We first present the error analysis of PL strategy by showing that the error is bounded by the confidence of PL threshold and consistency of multi-view prediction. Then, we theoretically illustrate the effect of PL on convergence property. Based on the analysis, we propose a cautious pseudo labeling methodology in which we pseudo label the samples with highest confidence and multi-view consistency. Finally, extensive experiments demonstrate that the proposed strategy improves graph learning process and outperforms other PL strategies on link prediction and node classification tasks.

Deep Insights into Noisy Pseudo Labeling on Graph Data

Abstract

Pseudo labeling (PL) is a wide-applied strategy to enlarge the labeled dataset by self-annotating the potential samples during the training process. Several works have shown that it can improve the graph learning model performance in general. However, we notice that the incorrect labels can be fatal to the graph training process. Inappropriate PL may result in the performance degrading, especially on graph data where the noise can propagate. Surprisingly, the corresponding error is seldom theoretically analyzed in the literature. In this paper, we aim to give deep insights of PL on graph learning models. We first present the error analysis of PL strategy by showing that the error is bounded by the confidence of PL threshold and consistency of multi-view prediction. Then, we theoretically illustrate the effect of PL on convergence property. Based on the analysis, we propose a cautious pseudo labeling methodology in which we pseudo label the samples with highest confidence and multi-view consistency. Finally, extensive experiments demonstrate that the proposed strategy improves graph learning process and outperforms other PL strategies on link prediction and node classification tasks.

1 Introduction

Pseudo Labeling (PL) [27, 13] is one of the most popular self-supervised learning approaches and has been widely used to tackle the label sparsity problem. Its core idea is to enlarge the training set by self-labeling. As most self-labeled samples should be consistent with ground truth, the enlarged dataset has a larger sample capacity to improve the model generalization. Plenty of studies have shown the effectiveness of PL [7, 26, 14, 5]. However, there is a trade-off between the benefit of PL and the effect of mislabeled samples. When the benefit of PL outweighs the impact of introduced noise, the performance of the base model (i.e., without PL) can be improved. But for non-i.i.d. condition such as graph data, the introduced noisy labels may transfer among the samples and be amplified, which may degrade the performance of base model. Although several methods have been proposed to alleviate this noisy phenomenon [31, 32, 15], there is still neither a clear explanation nor a quantification of how pseudo labeling affects the graph learning models.

In this study, we attempt to answer questions above. Specifically, we evaluate PL's effect on the prediction error of the base model and the convergence of the empirical loss function. For a graph learning model, the message aggregation process would amplify the noises of incorrect labels introduced by PL. These noises can even accumulate to damage the base model's performance. For

65.0

75

GAE

94

70

GAE+PL

62.5

AUC (%)

(%)

65

GAE+CPL

AU

C

60.0

92

AUC

60

57.5

55

55.0

90

50

Ok

5k

10k

15k

20k

Ok

20k

40k

60k

80k

100k

Ok

10k

20k

30k

40k

50k

No. PL samples(Actor)

No. PL samples(WikiCS)

No. PL samples (Amazon_Photo)

example, in a two-layer graph neural network (GNN) for node classification, the mislabeled nodes can inappropriately reduce the predicted confidence of their 2-hop neighbors. Moreover, in some circumstances, such as link prediction task, the pseudo labeled sample would not only serve as the label but also as the model inputs in the consequent iterations. This characteristic further aggravates the loss function's convergence due to the noises added to adjacency matrix.

To visualize the side effect of the PL on graph data, we conduct a toy experiment as shown in Figure 1, where the benefit of PL to a popular link prediction model GAE [12] depends on the choice of graph dataset, i.e., PL improves the performance of GAE on Actor, but degrades that on WikiCS. Even worse, due to the incorrect label introduced by PL, PL leads to the model's collapse on Amazon_Photo.

To obtain a stable and consistent result by PL strategy, it is necessary to quantify the impact of introduced noisy labels and theoretically analyze how it affects the graph training procedure compared to the procedure without PL. In this paper, we build a theoretical connection between PL strategy and graph learning models with multi-view augmentations. We prove that the error bound of the PL predictor is jointly bounded by a confidence threshold and the prediction consistency over different augmentations. Moreover, we theoretically analyze that PL strategy affects convergence property by the covariance term in the optimization function. Accordingly, we propose the Cautious Pseudo Labeling (CPL) strategy for the graph learning process that maximizes the confidence threshold by committing the PL samples with high prediction probability. We evaluate the CPL on different graph learning tasks, including link prediction and node classification models, where we observe remarkable and consistent improvements over multiple datasets and base models. As shown in Figure 1, compared with the base model GAE, the average AUC improvement of CPL over three datasets is 7.79%, which clearly validates the superiority of our model.

2 Model

In this section, we define the general problem of graph learning, including link prediction and node classification tasks. We give the error bound and convergence property analysis associated with the application of PL strategy. Lastly, we propose cautious PL strategy accordingly.

2.1 Problem Definition

In graph learning, given graph G = ( V , E ) , V = { v i } is the node set, E = { ( i, j ) } is the edge set, and |V| = N is the node number. The feature matrix and adjacent matrix are denoted by X = [ x ij ] N × F and A = [ a ij ] N × N , respectively, where ( i, j ) ∈ E if and only if a ij = 0 . A base GNN model g is trained on the graph G and the observed labels. It outputs the probability of prediction target. In this work, we adopt the PL scheme which involves a teacher model g ϕ and a student model g ψ . The teacher model calculates confidence of the unlabeled samples and PL a subset of samples using the strategy T . Subsequently, student model utilizes the enlarged set to fine-tune the base model and becomes the teacher model in the next iteration.

̸

Node classification task. The objective of the node classification task is to predict the probabilities of an unlabeled node belonging to different classes g : G → R N × M , where M is the number of classes. The original training set comprises the labeled nodes V o and their labels Y o . And the model aims to predict the classes of the unlabeled nodes V u . In the PL scheme, the teacher model calculates the confidence of the unlabeled nodes ˆ Y u and assigns pseudo labels to a selected subset of nodes

{V p , Y p |V p ⊂ V u , Y p ⊂ ˆ Y u } . Then the student model undergoes fine-tuning on the enlarged label set g ψ ( G ) → ˆ Y o , ˆ Y o = Y o ∪ Y p , ˆ Y o is the enlarged label set.

Link prediction task. The prediction targets are the probabilities of edges between unlabeled node pairs. In our scheme, the link prediction model g ( g emb , s ) consists of an embedding network g emb : G → R N × D and a score function s : R D × R D → R that estimates the probability of the link, where D is the dimension of the node embeddings. A portion of edges is observed as the training graph G = ( V , E o ) , E o ⊂ E . The teacher model predicts the confidence of the unobserved edges E u and enlarge the observed edge set ˆ E o = E p ∪ E o for fine-tuning g ψ : G ( V , ˆ E o ) → R N × N .

It is worth to mention that in the link prediction task, the node embeddings undergo a change once pseudo labels are added, even before fine-tuning. Because the enlarged edge set is also utilized as input for the GNN.

̸

The optimization target is formulated as minimizing L = CE ( g ψ , Y o ) / | Y o | , where CE ( · ) denotes cross entropy, Y o is the ground truth label of the observed set and represents E o for the link prediction task. The overall performance is evaluated by the 0-1 loss: Err ( g ) = E [ argmax ( g ψ ) = Y ] .

2.2 Pseudo Labeling Error Analysis

We here present the error bound analysis of the graph learning under the PL strategy. To facilitate mathematical treatment, we introduce several assumptions.

2.2.1 Graph perturbation invariant

We assume graph perturbation invariant (GPI) property in GNN, which states that the prediction variance is linearly bounded by the difference between the augmented and original inputs.

Definition 2.1 : Given a graph G and its perturbation ˆ G = G ( X ⊙ M x , A ⊙ M a ) by the random feature masks M x ∈ { 1 , 0 } N × F and adjacent matrix mask M a ∈ { 1 , 0 } N × N satisfying

the GNN g ( · ) has GPI property if there exists a constant C > 0 such that the perturbed prediction confidence satisfies ∥ g ( ˆ G ) -g ( G ) ∥ 2 2 < Cϵ . Here, ⊙ is element-wise product, and ∥·∥ 2 is the 2-norm of the vector or matrix.

The GPI property guarantees the variation of the output confidence is linearly bounded by the degree of graph perturbation, which is similar to the C -Lipschitz condition applied to GNN.

2.2.2 Additive expansion property

With Definition 2.1, if we have a convex measurable function f ( y ) that satisfies the C -Lipchitz condition, we can find a probability measure on the output prediction p f ( y ) that satisfies the additive expansion property as stated in Proposition 2.2.

Proposition 2.2 Define a local optimal subset as U ⊂ Y , whose probability is higher than a threshold p f ( y ) > 1 -q, y ∈ U , and its perturbation set U ϵ = { ˆ y = g ( G ) : ∥ ˆ y -y ∥ 2 ≤ Cϵ, y ∈ U } , where G ∈ { ˆ G } is the space of the perturbed graph. Then, there exists α > 1 , η > 0 , s.t. the probability measure p f satisfying following additive expansion property: p αf ( U ˆ ϵ \ U ) ≥ p αf ( U ) + η · α .

The proposition guarantees the continuity of the measurable function in the neighborhood of the local optimal subset U . In practice, U represents the PL samples, which ideally should be close to the ground truth labels in ideal. However, the correctness of the PL samples is discrete where the continuity condition is hard to be satisfied. To address this issue, we can leverage GPI. By applying multi-view augmentations and calculating average confidence, we canreparameterize the probability measure to be continuous. In this condition, the Proposition 2.2 implies that the probability of the neighborhood under the amplified measure is greater than the original local optimum. This observation opens up potential opportunities for optimization. For a detailed proof of Proposition 2.2, please refer to Appendix A.

2.2.3 Prediction error measurement

Then, we use the additive expansion property to calculate the error bound of the multi-view GNN model under PL strategy. According to Theorem B.2 in [25], when we use 0-1 loss to measure the error of predictor, the error bound can be evaluated by the following theorem.

Theorem 2.3 Let q > 0 be a given threshold. For the GNN in the teacher model g ϕ , if its corresponding density measure satisfies additive expansion, the error of the student predictor g ψ is bounded by

where A ( g ψ ) = E Y test [ 1 ( ∃ g ψ ( ˆ G ) = g ψ ( G ))] measures the inconsistency over differently augmented inputs, Y test is the test set for evaluation.

̸

The brief derivation is provided in Appendix.B. From to Eq.2, we observe that the prediction error (0-1 loss) is bounded by the confidence threshold q and the expectation of the prediction inconsistency A . Intuitively, if the predictions of multi-view augmented inputs tend to be consistent and the threshold is smaller, the error bound will be smaller, which implies a more accurate predictor.

If q is a small value, the threshold probability of PL 1 -q approaches 1. This cautious approach in self-training leads to a smaller lower bound in the error estimate. When applying random PL, setting the confidence threshold as q = 0 . 5 , then the maximum theoretical error rate is 1. It means that there is no guarantee on the performance after applying PL.

The inconsistency term A is the expectation of the probability that predictions are inconsistent when input graphs are augmented differently. A small value of A indicates consistent prediction across different views. In such cases, we have more confidence in the predictions, leading to a smaller error bound. On the other hand, if the predictions from different views are inconsistent with each other, the model lacks robustness, resulting in a larger error bound.

2.3 Convergence Analysis

In this section, we analyze the influence of the PL strategy on the empirical loss to illustrate its convergence property. First, we assume that optimizing teacher model will not influence the convergence property of PL strategy. This assumption stems from the understanding that the teacher model can converge independently without the incorporation of PL.

Assumption 2.4 The loss function defined in Algorithm 1 is non-increasing during the optimization: CE ( g ( t ) ψ , ˆ Y ( t +1) o ) ≤ CE ( g ( t ) ϕ , ˆ Y ( t ) o ) .

Then, we show that the PL sample selection strategy T influences the covariance term derived from the empirical loss, then affects the convergence property.

where β = | ˆ Y u | / ( | ˆ Y o | + k ) , ce ( · ) is the element-wise cross entropy. The equality is achieved when the model reaches the global optimal under given training set.

The detailed derivation of the Eq.3 is shown in Appendix.C. From Eq.3, we observe that the effect of PL strategy is decoupled and encapsulated in the covariance term. The covariance sign determines whether the empirical loss will increase or decrease in the next iteration of optimization. For instance, when PL samples are randomly selected, T would be independent with g ψ . The covariance becomes 0, indicating that the PL does not influence the loss function. However, a carefully designed PL strategy can accelerate the optimization of empirical loss and yield improved convergence properties.

2.4 Cautious Pseudo Labeling

According to Theorem 2.3, setting a higher confidence threshold for pseudo labeling (PL) can lead to a lower prediction error. This is reflected in a positive covariance term, which accelerates the optimization of the empirical loss and improves the convergence property. In order to satisfy these requirements, we propose the iterative Cautious Pseudo Labeling (CPL) strategy. CPL involves carefully and iteratively pseudo labeling the most confident samples to maximize these improvements in prediction accuracy and convergence.

Iterationunit

Adjacent matrix

1

Enlarge

Enlarge

ObservedSet

Observed Set

Train

Train

Train

Teacher

Parameter

Student

Parameter

Student

sharing

sharirg

Link Predictor

Link Predictor2

Prediction

Prediction

Prediction

0.9

0.9

0.2

0.9

Cautious

0.7

0.9

0.4

0.8

Cautious

0.9

0.9

0.2

0.8

0.9

0.7

.7

0.3

PseudoLabels

0.7

0.9

0.7

0.8

PseudoLabels

0.9

0.9

0.8

0.9

0.9

0.7

0.8

0.9

0.9

0.8

0.1

0.9

0.9

0.8

0.1

0.2

0.7

0.8

0.1

q(0)=0.9

CELoss

0.4

0.7

0.8

01

q(1)=0.8

CELoss

0.2

0.8

0.8

0.1

Minimization

Minimization

0.9

0.30.40.1

0.8

0.8

0.1

0.1

0.8

0.9

0.1

0.1

Confidence of:

Observed links

Unobserved links

There are a teacher model and a student model in each iteration. First, we calculate multi-view prediction by the teacher model g ϕ as the confidence measure. Although biased, it is a simple and effective measure in many PL strategies. In node classification, the GNN directly outputs the probability as confidence g ( G ) . For link prediction, the confidence is determined by the inner product similarity of node embedding g ( G ) = σ ( E T E ) , where E = ( e 1 , ..., e N ) T = g emb ( G ) is the node embedding, and σ ( · ) is the sigmoid function.

Then we select PL samples in unobserved set with the highest confidence, enlarging the training set. We iteratively select topk confident samples for PL: Y ( t ) p = T ( ˆ Y t u , g ϕ , k ) , where T ∈ { 0 , 1 } | ˆ Y ( t ) u | , ∑ T = k . g ( t ) ϕ is the averaged confidence of the multi-view augmentation and is equal to g ( t ) ϕ for the single input. At the t -th iteration, the selected PL samples are Y ( t ) p .

Then we update the observed and the unobserved set. The student model is fine-tuned by the enlarged set and becomes the new teacher model in the next iteration. We take the link prediction task as the example, whose main scheme is in Fig.2. The complete algorithm of CPL is shown in Algorithm 1.

The confidence threshold q ( t ) recorded in Algorithm 1 is the lowest confidence among the PL samples Y ( t ) p . At each iteration, we record the lowest confidence in these k PL samples as q ( t ) . We update q to be q ( t ) if q ( t ) is smaller. Finally, q serves as the confidence threshold in Eq.2 for error analysis in Theorem 2.3.

The following theorem states the improvement on convergence property under CPL.

Theorem 2.5 Let L ( t ) T denote the optimization target at the t -th iteration. The risk is monotonically decreasing under pseudo labeling strategy, i.e.,

We have demonstrated the first inequality in Eq.3. The second inequality holds because the covariance between the cross-entropy loss and PL strategy is non-positive. On one hand, each view of g ψ is positively correlated to the ground-truth label Y due to the consistency regularization. Then, g ψ exhibits negative correlation with the cross-entropy loss ce ( g ψ , Y ) . On the other hand, the PL strategy T can be seen as a discrete sign function. In CPL, higher confidence samples are more likely to be pseudo-labeled. Therefore, T is positively correlated with g ψ . Consequently, T is negatively correlated with the cross-entropy term, resulting in a non-positive covariance term Cov ( · ) . This guarantees the monotone decreasing behavior of the loss function, as stated in Theorem 2.5. Furthermore, since the loss function is lower bounded, Theorem 2.5 ensures the convergence of Algorithm 1.

Algorithm 1: Iterative cautious pseudo labeling.

Time complexity The computational complexity of CPL depends on the complexity of specific GNN models used. The key operation in CPL is the selection of topk values. For node classification, it takes O ( N log k ) , while for link prediction, it takes O ( N 2 log k ) . However, since GNN models typically compute probabilities for all samples, the overhead introduced by CPL does not increase the complexity of existing methods significantly. For example, the complexity of GAE [12] for link prediction is O ( |E o | D 2 + N 2 D ) (i.e., the complexity of graph convolution operations and innerproducts). Integrating CPL into GAE does not increase the complexity since both involve O ( N 2 ) operations. In practice, the additional time consumption mainly results from the fine-tuning process.

3 Experiments

In this section, we conduct an evaluation to assess the effectiveness of the CPL strategy on both link prediction and node classification tasks. We compare CPL with raw base models as well as other PL strategies. Then, we analyze the impact of CPL capacity, training data ratio, PL strategy, and augmentation methods. Finally, a case study is bridged to the theoretical analysis on convergence and error bound. The implementation is open-sourced at https://github.com/AcEbt/CPL.

3.1 Datasets and Benchmarks

We adopt five public available datasets to evaluate CPL strategy for link prediction, i.e. CiteSeer, Actor, WikiCS, TwitchPT, and Amazon_Photo, and five datasets for node classification, i.e. Cora, CiteSeer, PubMed, Amazon_Photo, and LastFMAsia. Detailed statistics are reported in Table 1.

In link prediction task, as there are few PL-based methods, we apply the CPL strategy on three popular models: GAE [12], node2vec [4], SEAL [29] . To reserve sufficient candidate unobserved samples for PL, the dataset is randomly split into 10%,40%,50% for training, validation, and testing.

In node classification task, we employ CPL on 4 popular base models: GCN , GraphSAGE , GAT , APPNP . CPL is compared with two other PL strategies, namely M3S [22] and DR-GST [18], using the implementation and parameters provided by the authors 2 . We adopt the official split for the citation datasets and 5%,15%,80% split for other datasets in node classification.

We run experiments with 5 random seeds and report the mean and standard deviations of the metrics.

3.2 Performance Comparison

3.2.1 Overall performance comparison

For link prediction, Table 2 lists the AUC and AP of raw baselines and ones with CPL employed. We observe that CPL distinctively increases the performance of baseline models in nearly all cases. Note that the CPL strategy can achieve performance gain under the circumstances of both high and low performance (e.g., the AUC of GAE on Actor improves from 55.34 to 65.58 and the AP of SEAL on CiteSeer improves from 64.38 to 64.94).

For node classification, Table 3 shows the improvement on the base models and comparison with other PL strategies. The CPL strategy can consistently improve the performance of the base models. However, other PL strategy may be ineffective or degrade the base model (e.g., DR-GST on Cora with APPNP, M3S for PubMed with GraphSAGE). CPL also consistently outperforms other PL strategies.

The time consumption is also reported in AppendixD.

3.2.2 Impact of pseudo labeling capacity

In the experiment, the number of PL samples in each iteration k is set from 100 to 1000. We provide a reference interval for the selection of k . Intuitively, a small k can lead to a reliable result. But a too small k will unnecessarily increase the training time. And it does not prominently influence the overall performance as we test on different datasets. In the experiment, the predicted confidence distribution is around 1, as there are usually plenty of potential PL samples. k is much smaller than the total number of unobserved samples. Then the k -th highest confidence in the unobserved set will not have much difference, as the cases in Table 4.

3.2.3 Impact of training data ratio

PL enlarges the observed dataset and introduces extra information for the training. This effect has a different degree of contribution on different training data. When the training set is partially applied for the training, the ratio of the observed set ranges from 0.1 to 0.9. The variation of AUC and AP on the Amazon_Photo dataset is shown in Fig.3. The CPL method can consistently improve the performance of the raw model even starting from a small training set. It is also worth to mention that CPL is more likely to have a more significant contribution when the training set is small, as there is more introduced information.

85

80

85

80

Raw

PL

64

66

65

Random

NodeView

(%)

AUC

75

(%)

AP

75

(%)

AUC

62

60

(%)

60

Multiview

FeatureView

StructureView

58

70

70

56

55

54

0.1

Ratio of training data (Amazon)

0.3

0.5

0.7

0.9

0.1

Ratio of training data (Amazon)

0.3

0.5

0.7

0.9

No. PL samples (Actor)

No. PL samples (Actor)

3.2.4 Impact of noisy pseudo labeling strategy

As the noise introduced by PL has a large effect in the graph, we compare the CPL with the PL strategy. In PL, the samples are selected randomly in the unobserved set, whose labels are estimated by the teacher model. The comparison of AUC's variation on different datasets is shown in Fig.1. We discover that PL has different effects on the baseline models. On Actor, the PL consistently improves the link predictor, but not as much as the CPL. On WikiCS however, PL keeps weakening the AUC, as the introduced noise outweighs. On Amazon_Photo, PL can improve the performance at the first few iterations. But the AUC suddenly drops from 70% to around 50% after iterations, which illustrates that introducing too much noise may degrade the model. As for the CPL, although it may drop after reaching its best, i.e. for WikiCS, it can distinctively improve the performance compared to the base model and will not lead to failure.

3.2.5 Impact of multi-view augmentation

During the training, we apply multi-view augmented graphs as the input of the teacher model, so that the continuity condition in Theorem 2.3 is easier to be satisfied. We here conduct the ablation experiments that compares multi-view augmentation with different single-view augmentation methods, including drop node (Node view), feature mask (Feature view), and DropEdge (Structure view). In each experiment, a single augmentation method was applied three times. And "Random" refers to the random selection of samples during PL. As shown in Fig.4, all of the single-view augmentation methods have better performance than the base model and their improvements are almost the same. When we use multi-view augmentation, the AUC and AP are further improved. It illustrates that multi-view augmentation contributes to a more robust graph learning and tends to obtain a consistent result, which echoes with our theoretical analysis.

3.3 Case Study

The case study on node classification is conducted on LastFMAisa on different baselines. The detailed intermediate variables about the error analysis are listed in Table 5. We record the inconsistency A and confidence 1 -q during the CPL as Algorithm.1. Then the theoretical error bound Err th ( g ) is calculated by Theorem 2.3. As Err th ( g ) > Err exp ( g ) , the experimental error is bounded by the theoretical error, implying that it is an effective bound. We also report the accuracy of PL samples. The CPL has smaller error than other PL strategies and is consistently better than Err exp ( g ) .

The case study on link prediction is conducted based on GAE on WikiCS. The relationship between consistency 1 -A ( g ) and the number of PL samples is shown in Fig.5. The result shows that more PL samples help to increase the consistency of the prediction. We also compare the optimization target of PL L ( t ) R and CPL L ( t ) T in Fig.5. We discover that the optimization target of CPL converges faster than PL and is consistently smaller. It

implies the covariance term Cov [ ce , T ] is negative as illustrated in Theorem 2.5. Thus, the CPL strategy can improve the convergence property of graph learning. A detailed illustration of error bound verification and Knowledge discovery is shown in Appendix E.

4 Related Works

Pseudo labeling is a popular approach in self-training (ST), aiming to enlarge the training set by self-annotation. Most studies focus on learning a more accurate PL algorithms to avoid noisy samples. Confidence thresholding is a simple yet effective method in ST, where the similarity with ground truth or consistency is used to measure the confidence, ensuring that flawed samples are excluded from the enlarged dataset [9, 20]. [6] uses the perturbation on the hidden states to yield close predictions for similar unlabeled inputs. [7, 16, 26] rely on the consistency of the unsupervised clustering. [30] utilizes adversarial learning to acquire domain-uninformative representations and train a PL classifier. [3, 17] use a contrastive loss to improve the representation learning before PL. Directly modeling noisy labels can also enhance noise tolerance [31], such as using soft pseudo labels [2], or distilling correct information to mitigate overfitting on noise [32].

The confidence measure plays a crucial role in avoiding the overconfident results [11]. [33] constructs confidence regularizers, expecting a smooth prediction with soft-label and preventing infinite entropy minimization. [10] learns a confidence metric based on the generality and unity of its distribution of pseudo loss. [1] uses the distance between the distributions of positive and negative samples as a confidence measure. [24] applies data augmentation on node features, where consistency is used as a confidence measure for classification. The co-training method constructs multi-view classifiers. It adds pseudo labels in each view to provide complementary information for each other, showing better performance than single-view [5].

Some studies on graph data apply the PL strategy to node classification tasks. M3S [21], IFC-GCN[8] utilize the clustering to PL the unlabeled samples. CaGCN-st [23] is a confidence-calibrated model

that utilizes low-confidence but high-accuracy samples. DR-GST [18] employs dropout and edge-drop augmentation to conduct information gain inference for selecting PL samples.

5 Conclusion

In this study, we provide deep insights into PL strategy by analyzing its impact on prediction error and the convergence properties. We offer theoretical explanations for the effect of PL strategies on graph learning processes, particularly addressing degradation issues. Based on the theoretical analysis, we introduce the CPL strategy, a plug-in and practical technique that can be generally applied to various baseline models. The experiments demonstrate effectiveness and superiority of CPL in link prediction and node classification tasks.

In future work, we plan to explore a more reliable confidence measures as the PL criteria, such as informativeness in the multi-view network and prediction uncertainty.

Acknowledgments and Disclosure of Funding

The research of Tsung was supported in part by the Industrial Informatics and Intelligence (Triple-i) Institute at HKUST(GZ).

The research of Jia Li was supported by NSFC Grant No. 62206067, Tencent AI Lab Rhino-Bird Focused Research Program and Guangzhou-HKUST(GZ) Joint Funding Scheme 2023A03J0673.

References

Charles Corbière, Nicolas Thome, Avner Bar-Hen, Matthieu Cord, and Patrick Pérez. 'Addressing failure prediction by learning model confidence'. In: NIPS 32 (2019).

Yixiao Ge, Dapeng Chen, and Hongsheng Li. 'Mutual mean-teaching: Pseudo label refinery for unsupervised domain adaptation on person re-identification'. In: arXiv preprint arXiv:2001.01526 (2020).

Yixiao Ge, Feng Zhu, Dapeng Chen, Rui Zhao, et al. 'Self-paced contrastive learning with hybrid memory for domain adaptive object re-id'. In: NIPS 33 (2020), pp. 11309-11321.

Aditya Grover and Jure Leskovec. 'node2vec: Scalable feature learning for networks'. In: Proceedings of the 22nd ACM SIGKDD . 2016, pp. 855-864.

Tengda Han, Weidi Xie, and Andrew Zisserman. 'Self-supervised co-training for video representation learning'. In: NIPS 33 (2020), pp. 5679-5690.

Junxian He, Jiatao Gu, Jiajun Shen, and Marc'Aurelio Ranzato. 'Revisiting Self-Training for Neural Sequence Generation'. In: ICLR . 2020.

Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 'Hubert: Self-supervised speech representation learning by masked prediction of hidden units'. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing 29 (2021), pp. 3451-3460.

Zhihui Hu, Guang Kou, Haoyu Zhang, Na Li, Ke Yang, and Lin Liu. 'Rectifying pseudo labels: Iterative feature clustering for graph representation learning'. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management . 2021, pp. 720-729.

Zijian Hu, Zhengyu Yang, Xuefeng Hu, and Ram Nevatia. 'Simple: Similar pseudo label exploitation for semi-supervised classification'. In: CVPR . 2021, pp. 15099-15108.

Kai Huang, Jie Geng, Wen Jiang, Xinyang Deng, and Zhe Xu. 'Pseudo-Loss Confidence Metric for Semi-Supervised Few-Shot Learning'. In: ICCV . 2021, pp. 8671-8680.

Susmit Jha, Sunny Raj, Steven Fernandes, Sumit K Jha, Somesh Jha, Brian Jalaian, Gunjan Verma, and Ananthram Swami. 'Attribution-based confidence metric for deep neural networks'. In: NIPS 32 (2019).

Thomas N Kipf and Max Welling. 'Variational Graph Auto-Encoders'. In: NIPS Workshop on Bayesian Deep Learning (2016).

Dong-Hyun Lee. 'Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks'. In: Workshop on challenges in representation learning, ICML . Vol. 3. 2. 2013, p. 896.

Jia Li, Yongfeng Huang, Heng Chang, and Yu Rong. 'Semi-Supervised Hierarchical Graph Classification'. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2022).

Jia Li, Jiajin Li, Yang Liu, Jianwei Yu, Yueting Li, and Hong Cheng. 'Deconvolutional Networks on Graph Data'. In: NeurIPS . 2021.

Jia Li, Yu Rong, Hong Cheng, Helen Meng, Wenbing Huang, and Junzhou Huang. 'Semisupervised graph classification: A hierarchical graph perspective'. In: The World Wide Web Conference . 2019, pp. 972-982.

Junnan Li, Caiming Xiong, and Steven CH Hoi. 'Comatch: Semi-supervised learning with contrastive graph regularization'. In: ICCV . 2021, pp. 9475-9484.

Hongrui Liu, Binbin Hu, Xiao Wang, Chuan Shi, Zhiqiang Zhang, and Jun Zhou. 'Confidence may cheat: Self-training on graph neural networks under distribution shift'. In: Proceedings of the ACM Web Conference 2022 . 2022, pp. 1248-1258.

László Lovász and Miklós Simonovits. 'Random walks in a convex body and an improved volume algorithm'. In: Random structures & algorithms 4.4 (1993), pp. 359-412.

Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. 'Fixmatch: Simplifying semisupervised learning with consistency and confidence'. In: NIPS 33 (2020), pp. 596-608.

Ke Sun, Zhouchen Lin, and Zhanxing Zhu. 'Multi-stage self-supervised learning for graph convolutional networks on graphs with few labeled nodes'. In: Proceedings of the AAAI conference on artificial intelligence . Vol. 34. 04. 2020, pp. 5892-5899.

Xiao Wang, Deyu Bo, Chuan Shi, Shaohua Fan, Yanfang Ye, and S Yu Philip. 'A survey on heterogeneous graph embedding: methods, techniques, applications and sources'. In: IEEE Transactions on Big Data (2022).

Xiao Wang, Hongrui Liu, Chuan Shi, and Cheng Yang. 'Be confident! towards trustworthy graph neural networks via confidence calibration'. In: Advances in Neural Information Processing Systems 34 (2021), pp. 23768-23779.

Yiwei Wang, Wei Wang, Yuxuan Liang, Yujun Cai, Juncheng Liu, and Bryan Hooi. 'Nodeaug: Semi-supervised node classification with data augmentation'. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 2020, pp. 207-217.

Colin Wei, Kendrick Shen, Yining Chen, and Tengyu Ma. 'Theoretical Analysis of SelfTraining with Deep Networks on Unlabeled Data'. In: ICRL . 2021.

Shaoan Xie, Zibin Zheng, Liang Chen, and Chuan Chen. 'Learning semantic representations for unsupervised domain adaptation'. In: ICML . PMLR. 2018, pp. 5423-5432.

David Yarowsky. 'Unsupervised word sense disambiguation rivaling supervised methods'. In: 33rd annual meeting of the association for computational linguistics . 1995, pp. 189-196.

Percy Liang Yuchen Zhang and Moses Charikar. 'A hitting time analysis of stochastic gradient Langevin dynamics'. In: Proceedings of the 2017 Conference on Learning Theory . Vol. 65. Proceedings of Machine Learning Research. PMLR, 2017, pp. 1980-2022.

Muhan Zhang and Yixin Chen. 'Link prediction based on graph neural networks'. In: NIPS 31 (2018), pp. 5165-5175.

Weichen Zhang, Wanli Ouyang, Wen Li, and Dong Xu. 'Collaborative and adversarial network for unsupervised domain adaptation'. In: CVPR . 2018, pp. 3801-3809.

Zhilu Zhang and Mert Sabuncu. 'Generalized cross entropy loss for training deep neural networks with noisy labels'. In: NIPS 31 (2018).

Zizhao Zhang, Han Zhang, Sercan O Arik, Honglak Lee, and Tomas Pfister. 'Distilling effective supervision from severe label noise'. In: CVPR . 2020, pp. 9294-9303.

Yang Zou, Zhiding Yu, Xiaofeng Liu, BVK Kumar, and Jinsong Wang. 'Confidence regularized self-training'. In: ICCV . 2019, pp. 5982-5991.

A Proof of Proposition 2.2: additive expansion proposition

We denote the embedding vector of a node v i by e i ≜ g i ( ˆ G ) = GNN ( ˆ G )[ i ] . Without loss of generality, we drop the subscript for short. We can also define the density measure d f as:

For any subset of the embedding space E ⊂ g ( X ) , the local probability can be measured by p f ( E ) = ∫ E p f ( e ) de . If we have a local optimal subset U ⊂ E with a confidence threshold of 1 -q , and its perturbation U ϵ , then the consistency at the boundary of the subset separation problem E → U, E \ U can be quantified by Cheeger constant. We introduce the continuous Cheeger inequality to elaborate the lower bound of the Cheeger constant under an amplified measure αf , where α > 1 is a constant.

We first define the restricted Cheeger constant in the link prediction task. Given the function f and any subset E , Cheeger constant is calculated by

According to the definition, the Cheeger constant is a lower bound of probability density in the neighborhood of the given set. It quantifies the chance of escaping the subset A under the probability measure f and reveals the consistency over the set cutting boundary.

Then we prove that for the any subset E ⊂ g ( G ) with its local optimal subset U : { e ∈ E : p f ( e ) > 1 -q } , there exists α > 1 s.t. C αf ( E \ U ) ≥ 1 .

As the measurable function for the link prediction is defined as f ( e ) = -e T e a . When e ∗ = e a , f reaches the global minimal. For the embedding vectors outside the local minimal subset e y ∈ g ( G ) \ U , there exsits ϵ > 0 s.t.

where ˆ ϵ = Cϵ . If we define E ∗ ϵ = { e ∗ ˆ ϵ } ∩ g ( G ) , where e ∗ ˆ ϵ is the ˆ ϵ neighbor of e ∗ , according to the Lipchitz condition of f , for e ∈ E ∗ ϵ , we have:

Combining Eq.8 and Eq.7 leads to f ( e y ) -f ( e x ) ≥ ˆ ϵ . Thus, for the amplified probability measure p αf , we have

According to the inequality property from [28] (formula 63), we have

As p αf ( g ( G ) \ U ) + p αf ( U ) = 1 . If we select α large enough s.t. the RHS of Eq.10 is larger than 1, p αf ( g ( G ) \ U ) ≤ 1 2 . Thus, according to [19] (Theorem 2.6), we have C αf ≥ 1 . It guarantees consistency around the perimeter of the U . As α > 1 and p f , p αf are bounded on any subsets of embedding space. It implies a probability margin η > 0 at the neighborhood of the local optimal between two measurable functions f, αf , where

which, according to [25], implies additive expansion property of the probability measure in the link prediction, as Proposition 2.2.

B Proof of Theorem 2.3: error analysis

̸

In [25], M ( g ϕ ) is also assumed to satisfy additive-expansion ( q, ϵ ) , where M ( g ) ≜ { y ∈ Y : g ( y ) = y } is the set of mis-classified samples, and they give the error bound of the trained classifier s (Theorem B.2):

Here in link prediction task, M ( g ϕ ) is mis-classified samples by the pseudo labeler (teacher model). It can be written by { y i : Y p [ i ] = E T [ i ] } , which is intractable during the training. The probability threshold is 1 -q and a local optimal subset U for PL is constructed accordingly. We aim to let M ( g ϕ ) ∩ U be close to ∅ , so that g ( G ) \ U can cover M ( g ϕ ) as much as possible. So we define the robust set S ( g ) as

̸

where y ϵ is the ϵ neighborhood of sample y . Then, according to Proposition 2.1, we have:

̸

̸

which has similar form with [25] Lemma B.3 for link prediction task. Besides, the analysis of p f ( { y ∈ Y : g ϕ ( y ) = y, g ψ ( y ) = y, y ∈ S ( g ψ ) } ) and p f ( S ( g ψ )) are the same. Thus, the assumption on M ( g ϕ ) is satisfied. Then, we can draw the same conclusion with Eq.12, and the classifier is the student model g ψ . The theorem is proofed.

C Proof of convergence inequality

The PL strategy T for the unlabeled data provides a Bayesian prior, from which we formalize the empirical loss defined in Eq.1 as

We can decompose the cross-entropy loss of the pseudo labeled samples by:

.

Thus, Eq.16 can be simplified to:

Note that T is the indicator-like function, where we have

Based on the Eq.15 and Eq.17, we can rewrite L ( t +1) T as

where β = | ˆ Y u | / ( | ˆ Y o | + k . The inequality holds due to the assumption.

D Time consumption of CPL

The time consumption of CPL on node classification and link prediction are shown in Table 6 and Table 7 respectively.

E Case study of CPL on link prediction

Error bound: In the case study, the recorded confidence threshold is 1 -q = 0 . 98 for WikiCS. We adopt 5 views of dropout with the augmentation drop rate 0.05. And according to the error bound given by Theorem 2.3, given the confidence threshold, Eq.2 suggests that the higher prediction consistency should lead to a smaller error bound. The final prediction consistency is A ( g ) = 0 . 0358 , thus, we can calculate error bound Err ( g ) = 0 . 1116 . The AUC and AP are 95 . 56 ± 0 . 24% , 95 . 58 ± 0 . 29% which are bounded within Err ( g ) .

Knowledge discovery: In the 5 random experiments, we add 500 pseudo links in each iteration. Here we focus on the common PL links in the first iteration, which are considered the most confident samples. We look for the metadata of WikiCS whose node, feature, link and node label represent paper, token, reference relation and topic of the paper respectively. There are These 7 most confident links categorized into 2 groups. We take 3 out of 5 nodes in group1 and the 2 nodes in group2 for analysis, whose detailed information of these nodes is shown in AppendixE.

For group 1, 3 nodes are connected by the pseudo links, and they are all linked to a central node whose degree is 321. The metadata information of the nodes are all strongly relevant to "Linux" in the "operating systems" topic. Thus, the PL linked nodes are likely to have common neighbors discovered triangle relationship. In group2, node 3489 has no in/out degree and is pseudo linked to node 7680. Both papers focus on the "malware"/"phishing" under the topic "Computer security". Although they only have one common token, the CPL strategy successfully discovers the correlation and consistently add it to the training set. The detailed result of the case study is shown in Table 8.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to provide in-depth insights into pseudo labeling (PL) in the context of graph learning models. The authors first present an error analysis of the PL strategy, demonstrating that the error is bounded by the confidence threshold of PL and the consistency of multi-view predictions. Furthermore, they theoretically illustrate the impact of PL on convergence properties. Building upon this analysis, they propose a careful pseudo labeling methodology that involves assigning pseudo labels to samples with the highest confidence and multi-view consistency. Finally, extensive experiments demonstrate that the proposed strategy enhances the graph learning process and outperforms alternative PL strategies in link prediction and node classification tasks.\n\nSTRENGTHS:\n1. This paper theoretically analyzes how the PL strategy affects convergence properties in GNN.\n2. Based on the analysis, this paper proposes a cautious pseudo labeling methodology, and extensive experiments demonstrate the effectiveness of the proposed strategy.\n\nWEAKNESSES:\n1.There might be some mistake in Theorem 2.5. since the covariance between the cross-entropy loss and PL strategy and $\\beta$ is non-negative, the inequality $\\beta Cov + L \\leq L$ holds only if $\\beta Cov = 0$.\n2.It might be better to clearly illustrate $q(t)$ in Figure 2, which demonstrates the framework of the algorithm.\n3.It is suggested to remove the first picture in Figure 1 and add two more pictures to show the performance on node classification.\n4.The word ‘noisy’ in the title is redundant for pseudo labeling. Furthermore, the whole paper rarely mentions ‘noisy PL’.\n\nQUESTIONS:\nHow much time does the proposed method consume on the dataset like PubMed?\n\nLIMITATIONS:\nSome figures could be further polished up, and check the correctness of the theorem.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nPseudo labeling is significant for GNN. This paper first theoretically analyzed the effect of pseudo labeling by showing the error bound and the convergence property. Then, accordingly, the paper proposes a cautious pseudo labeling based on confidence and multi-view consistency. The experimental results demonstrate the effectiveness of the proposed cautious pseudo labeling strategy.\n\nSTRENGTHS:\nThe paper is well written. The analysis of Pseudo labeling for GNN is valuable.\n\nWEAKNESSES:\nHowever, the analysis or the conclusion is general to all fields instead of highly related to the graph. Besides, the solutions, including high confidence and multi-view consistency, are not that novel. \n\nMore experiments are required. The time complexity should be provided when comparing with previous methods. Performance comparison on link prediction should also contain the previous PL methods.\n\nQUESTIONS:\n1) The paper is well written. The analysis of Pseudo labeling for GNN is valuable. However, the analysis or the conclusion is general to all fields instead of highly related to the graph. Besides, the solutions, including high confidence and multi-view consistency, are not that novel. \n\n2) Some contents may contain errors. For example, the results of Figure 1 may contain some errors. When the samples of PL and CPL are equal to/similar to 0, the GAE/GAE+PL/GAE+CPL should have equal/similar performance.\n\n3) More details need to be provided. What are the details of multi-view teachers? Does it refer to multi teachers or one teacher with multi-data-augmentation inputs? If the multi-view is obtained based on the data augmentation, what types of data augmentation are used? What is the data augmentation in Figure 4?\n\n4) More experiments are required. The time complexity should be provided when comparing with previous methods. Performance comparison on link prediction should also contain the previous PL methods.\n\n5) Other questions. In Algorithm 1, the student was fine-tuned based on the teacher model. I wonder whether there exists overfitting since the observed data set has been put into the model again and again. In 3.1, I wonder whether the data from the testing would be selected by CPL to be labeled for training. I wonder whether CPL may reduce the performance of nodes of low-frequently labeled classes.\n\nLIMITATIONS:\nThere is no potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper provides an error bound for pseudo labeling on graphs. Moreover, the authors propose a cautious pseudo labeling method and validate it through experiments.\n\nSTRENGTHS:\n1. The paper presents good experimental results, demonstrating the effectiveness of the proposed cautious pseudo labeling method.\n2. The authors have conducted comprehensive experiments\n3.  The error bound proposed in the paper is deemed useful and practical\n\nWEAKNESSES:\nThe paper suffers from unclear notation and contains numerous typos. These issues hinder the reader's understanding and make it challenging to follow the presented ideas. Clarifying the notation and addressing the typos is necessary to improve the clarity of the paper.\n\nQUESTIONS:\n1. Line 86: For \"err(g)\", does it mean the expectation is taken over the test points? Please clarify this point.\n\n2. Line 91: Is the GPI property introduced in this paper, or has it been introduced previously? Please provide the necessary context for understanding this property.\n\n3. Line 104: When stating \"whose probability is higher than a threshold,\" does it mean this condition must hold for each \"y ∈ U\"? Please clarify this point.\n\n4. Line 105: Since \"g(G) ∈ ℝ^(N×M)\", what do you mean by \"ŷ ∈ g(G)\"? Please explain this notation.\n\n5. Line 102: Do you mean we can find \"p_f(⋅)\" or we can find \"α, η\"? Please clarify this statement.\n\n6. Line 108: In \"p_{αf}\", there should be no \"α\" here. Please correct this notation.\n\n7. Line 122: When stating \"the teacher predictor satisfies additive expansion,\" the additive expansion is defined for a probability density, not a classifier. Please provide clarification and ensure consistency in terminology.\n\n8. Line 124: Could you explain what is meant by \"E_{Y_test}\"?\n\n9. Line 145: The first term should change \"t\" to \"t+1\". \n\n10. Line 147: What is the definition of the covariance term in this context? Please provide clarification.\n\n11. Lines 163-164: There seems to be a typo in the sentence, \"we calculate multi-view prediction of the by.\" Please clarify this phrase.\n\n12. Do you have any formal proof of Theorem 2.5? \n\nI am willing to increase my score if these concerns are resolved.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe article discusses noisy pseudo labeling (PL) on graph data and proposes a new cautious PL methodology (CPL) to improve the graph learning process. The authors conduct experiments to evaluate CPL strategy for link prediction on various datasets and apply it on popular models in node classification task. The result shows that the proposed strategy outperforms other PL strategies. The paper also provides a theoretical analysis of the impact of noisy labels introduced by PL on the graph training procedure. 1. The error introduced by PL is bounded by the confidence of PL threshold and consistency of multi-view prediction. 2. PL can be designed to contribute to the convergence property.\n\nSTRENGTHS:\nThe paper makes a highly original and significant contribution to graph learning. It proposes a new cautious Pseudo Labeling (PL) methodology that addresses limitations in prior works by introducing a confidence threshold and a consistency criterion for selecting high-confidence PL samples. This methodology, combined with a new consistency-based PL (CPL) strategy, improves the convergence property of graph learning and outperforms other PL strategies in link prediction and node classification tasks. The research methodology is rigorous, and the paper is well-structured, clear, and provides practical solutions to the challenges of limited and noisy labeled data.\n\nWEAKNESSES:\nThe paper could be strengthened by addressing several weaknesses. Firstly, in Table 2, the authors can also compare with other PL methods in the link prediction task. Secondly, conducting experiments with larger sample sizes or exploring different configurations would provide a more comprehensive evaluation. Thirdly, clarifying the methodology by providing implementation details and explaining data preprocessing steps would enhance replicability and understanding. Finally, considering the applicability of the proposed approach to different domains with highly imbalanced class distributions or datasets with different types of noise would broaden its practical relevance.\n\nQUESTIONS:\n1. Were there any specific assumptions made when applying the proposed approach to the benchmark datasets? It would be helpful to understand the compatibility of the approach with different dataset characteristics, such as class imbalance or noise types. Insights into these considerations would shed light on the generalizability of the approach.\n\n2. Could you provide additional information on the hyperparameters used in the experiments? Specifically, how were the hyperparameters set for the proposed cautious Pseudo Labeling (PL) methodology and the baseline models? Sharing these details would aid in replicating and fine-tuning the approach in future research.\n\n3. In the discussion of results, could you provide further insights into the potential limitations or failure cases of the proposed approach? Understanding the scenarios where the approach may not perform optimally would help in setting realistic expectations and identifying areas for further improvement.\n\nLIMITATIONS:\nThe authors did not mention their limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}