SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \sum_ {i = 1} ^ {c} \mathbb {E} _ {x} \left[ \mathrm {V a r} (x) \right] = \frac {1}{N} \sum_ {x \in G} \sum_ {i = 1} ^ {c} \left(h (x) ^ {T} \frac {1}{\sqrt {2 n _ {i}}} \left(h _ {1} ^ {i} - h _ {2} ^ {i}\right)\right) ^ {2} \\ = \frac {1}{N} \sum_ {k = 1} ^ {n _ {j}} \sum_ {j = 1} ^ {c} \sum_ {i = 1} ^ {c} \left(\left(\mu_ {k} ^ {j} + \epsilon_ {k} ^ {j}\right) ^ {T} \frac {1}{\sqrt {2 n _ {i}}} \left(h _ {1} ^ {i} - h _ {2} ^ {i}\right)\right) ^ {2} \\ = \frac {1}{N} \sum_ {k = 1} ^ {n _ {j}} \sum_ {j = 1} ^ {c} \sum_ {i = 1} ^ {c} \left(\left(\mu_ {k} ^ {j}\right) ^ {T} \frac {1}{\sqrt {2 n _ {i}}} \left(\epsilon_ {1} ^ {i} - \epsilon_ {2} ^ {i}\right) + \left(\epsilon_ {k} ^ {j}\right) ^ {T} \frac {1}{\sqrt {2 n _ {i}}} \left(\epsilon_ {1} ^ {i} - \epsilon_ {2} ^ {i}\right)\right) ^ {2} \\ = \frac {1}{2 N} \sum_ {k = 1} ^ {n _ {j}} \sum_ {j = 1} ^ {c} \sum_ {i = 1} ^ {c} \left((\mu_ {k} ^ {j}) ^ {T} \frac {1}{\sqrt {n _ {i}}} \left(\epsilon_ {1} ^ {i} - \epsilon_ {2} ^ {i}\right) + (\epsilon_ {k} ^ {j}) ^ {T} \frac {1}{\sqrt {n _ {i}}} \left(\epsilon_ {1} ^ {i} - \epsilon_ {2} ^ {i}\right)\right) ^ {2} \\ = \underbrace {\frac {1}{2 N} \sum_ {k = 1} ^ {n _ {j}} \sum_ {j = 1} ^ {c} \sum_ {i = 1} ^ {c} \left(\frac {1}{n _ {i}} \left[ \left(\mu_ {k} ^ {j}\right) ^ {T} \left(\epsilon_ {1} ^ {i} - \epsilon_ {2} ^ {i}\right) \right] ^ {2}\right)} _ {\mathcal {T} _ {1}} + \underbrace {\frac {1}{2 N} \sum_ {k = 1} ^ {n _ {j}} \sum_ {j = 1} ^ {c} \sum_ {i = 1} ^ {c} \left(\frac {1}{n _ {i}} \left[ \left(\epsilon_ {k} ^ {j}\right) ^ {T} \left(\epsilon_ {1} ^ {i} - \epsilon_ {2} ^ {i}\right) \right] ^ {2}\right)} _ {\mathcal {T} _ {2}} \\ + \underbrace {\frac {1}{N} \sum_ {k = 1} ^ {n _ {j}} \sum_ {j = 1} ^ {c} \sum_ {i = 1} ^ {c} \left(\frac {1}{n _ {i}} \left[ \left(\mu_ {k} ^ {j}\right) ^ {T} \left(\epsilon_ {1} ^ {i} - \epsilon_ {2} ^ {i}\right) \left(\epsilon_ {k} ^ {j}\right) ^ {T} \left(\epsilon_ {1} ^ {i} - \epsilon_ {2} ^ {i}\right) \right]\right)} _ {\mathcal {T} _ {3}} \tag {24} \\ \end{array}
$$
The above Equation can be decomposed into three parts, namely  $\mathcal{T}_1$ ,  $\mathcal{T}_2$ , and  $\mathcal{T}_3$ . Notably, each of these parts is associated with weight  $\frac{1}{n_i}$ . This observation supports Theorem 1 and Lemma 1, which suggest that the variance of a model is highly dependent on the distribution of the dataset's sample size, and that the extent of sample imbalance can significantly increase the model's variance.
Note that  $\epsilon^i\sim N(0,\Lambda^i)$ ,  $e^{i}\sim N\left(0,\frac{1}{n_{i}}\Lambda^{i}\right)$ , and so  $\frac{1}{\sqrt{n_i}}\epsilon^i\sim N\left(0,\frac{1}{n_i}\Lambda^i\right)$ . So, if we use the class center  $C^i$  to replace  $h$  in Equation 3, then we can get the following expression,
$$
\begin{array}{l} \frac {1}{N} \sum_ {x \in G} \sum_ {i = 1} ^ {c} \left(\left(C ^ {i}\right) ^ {T} h (x) - \left(C ^ {i ^ {\prime}}\right) ^ {T} h ^ {\prime} (x)\right) ^ {2} \\ = \frac {1}{N} \sum_ {k = 1} ^ {n _ {j}} \sum_ {j = 1} ^ {c} \sum_ {i = 1} ^ {c} \left(\left(\mu^ {i}\right) ^ {T} \left(\epsilon_ {k} ^ {j} - \epsilon_ {k} ^ {j ^ {\prime}}\right) + \left(e ^ {i}\right) ^ {T} \epsilon_ {k} ^ {j} - \left(e ^ {i ^ {\prime}}\right) ^ {T} \epsilon_ {k} ^ {j ^ {\prime}}\right) ^ {2} \\ = \frac {1}{N} \sum_ {k = 1} ^ {n _ {j}} \sum_ {j = 1} ^ {c} \sum_ {i = 1} ^ {c} \left((\mu^ {i}) ^ {T} (\epsilon_ {k} ^ {j} - \epsilon_ {k} ^ {j ^ {\prime}}) + (\frac {1}{\sqrt {n _ {i}}} \epsilon^ {i}) ^ {T} \epsilon_ {k} ^ {j} - (\frac {1}{\sqrt {n _ {i}}} \epsilon^ {i}) ^ {T} \epsilon_ {k} ^ {j ^ {\prime}}\right) ^ {2} \\ = \frac {1}{N} \sum_ {k = 1} ^ {n _ {i}} \sum_ {i = 1} ^ {c} \sum_ {j = 1} ^ {c} \left(\left(\mu^ {j}\right) ^ {T} \left(\epsilon_ {k} ^ {i} - \epsilon_ {k} ^ {i ^ {\prime}}\right) + \left(\frac {1}{\sqrt {n _ {j}}} \epsilon^ {j}\right) ^ {T} \epsilon_ {k} ^ {i} - \left(\frac {1}{\sqrt {n _ {j}}} \epsilon^ {j}\right) ^ {T} \epsilon_ {k} ^ {i ^ {\prime}}\right) ^ {2} \tag {25} \\ = \frac {1}{N} \sum_ {k = 1} ^ {n _ {i}} \sum_ {i = 1} ^ {c} \sum_ {j = 1} ^ {c} \left(\left(\mu^ {j}\right) ^ {T} \left(\epsilon_ {k} ^ {i} - \epsilon_ {k} ^ {i ^ {\prime}}\right) + \left(\frac {1}{\sqrt {n _ {j}}} \epsilon^ {j}\right) ^ {T} \left(\epsilon_ {k} ^ {i} - \epsilon_ {k} ^ {i ^ {\prime}}\right)\right) ^ {2} \\ = \underbrace {\frac {1}{N} \sum_ {k = 1} ^ {n _ {i}} \sum_ {i = 1} ^ {c} \sum_ {j = 1} ^ {c} \left(\left[ (\mu^ {j}) ^ {T} \left(\epsilon_ {k} ^ {i} - \epsilon_ {k} ^ {i ^ {\prime}}\right) \right] ^ {2}\right)} _ {\mathcal {L} _ {1}} + \underbrace {\frac {1}{N} \sum_ {k = 1} ^ {n _ {i}} \sum_ {i = 1} ^ {c} \sum_ {j = 1} ^ {c} \left(\frac {1}{n _ {j}} \left[ (\epsilon^ {j}) ^ {T} \left(\epsilon_ {k} ^ {i} - \epsilon_ {k} ^ {i ^ {\prime}}\right) \right] ^ {2}\right)} _ {\mathcal {L} _ {2}} \\ + \underbrace {\frac {2}{N} \sum_ {k = 1} ^ {n _ {i}} \sum_ {i = 1} ^ {c} \sum_ {j = 1} ^ {c} \left(\frac {1}{\sqrt {n _ {j}}} \left[ (\mu^ {j}) ^ {T} (\epsilon_ {k} ^ {i} - \epsilon_ {k} ^ {i ^ {\prime}}) (\epsilon^ {j}) ^ {T} (\epsilon_ {k} ^ {i} - \epsilon_ {k} ^ {i ^ {\prime}}) \right]\right)} _ {\mathcal {L} _ {3}} \\ \end{array}
$$
Like the previous Equation 24, Equation 25 can be decomposed into three parts:  $\mathcal{L}_1$ ,  $\mathcal{L}_2$ , and  $\mathcal{L}_3$ . If we substitute the class center  $C^i$  for  $h$ , we can observe that even though  $\mathcal{L}_1$  is insensitive to class imbalance, variables  $\mathcal{L}_2$  and  $\mathcal{L}_3$ , which respectively incorporate weights  $\frac{1}{n_j}$  and  $\frac{1}{\sqrt{n_j}}$ , can still provide the following support: when optimizing the variance of the model, more attention can be given to the variance introduced by the minority classes, which also provides an innovative perspective for understanding class imbalance on graphs.
C.1 More results Under Traditional Semi-supervised Settings
In Table 3, we present the results of ReVar and other algorithms on Cora-Semi  $(\rho = 10)$ . The experimental results demonstrate that TAM has a significant gain effect on all models (BalancedSoftmax, Renode, GraphENS), while GraphENS achieved the best performance among all current baselines on various models. By comparing with GraphSMOTE, we can conclude that addressing the node classification imbalance issue should focus on the topological characteristics of the graph. Finally, and most importantly, ReVar achieved state-of-the-art results under all basic models, which also verifies the superiority of our model in traditional imbalanced segmentation.
Table 3: Experimental results of our method ReVar and other baselines on Cora-Semi. We report averaged balanced accuracy (bAcc.,%) and F1-score (%) with the standard errors over 5 repetitions on three representative GNN architectures. Highlighted are the top first and second.  $\pmb{\Delta}$  is the margin by which our method leads state-of-the-art method.
Dataset(Cora-Semi)	GCN	GAT	SAGE
Imbalance Ratio (ρ = 10)	bAcc.	F1	bAcc.	F1	bAcc.	F1
Vanilla	62.82 ± 1.43	61.67 ± 1.59	62.33 ± 1.56	61.82 ± 1.84	61.82 ± 0.97	60.97 ± 1.07
Re-Weight	65.36 ± 1.15	64.97 ± 1.39	66.87 ± 0.97	66.62 ± 1.13	63.94 ± 1.07	63.82 ± 1.30
PC Softmax	68.04 ± 0.82	67.84 ± 0.81	66.69 ± 0.79	66.04 ± 1.10	65.79 ± 0.70	66.04 ± 0.92
GraphSMOTE	66.39 ± 0.56	65.49 ± 0.93	66.71 ± 0.32	65.01 ± 1.21	61.65 ± 0.34	60.97 ± 0.98
BalancedSoftmax	69.98 ± 0.58	68.68 ± 0.55	67.89 ± 0.36	67.96 ± 0.41	67.43 ± 0.61	67.66 ± 0.69
+ TAM	69.94 ± 0.45	69.54 ± 0.47	69.16 ± 0.27	69.39 ± 0.37	69.03 ± 0.92	69.03 ± 0.97
Renode	67.03 ± 1.41	67.16 ± 1.67	67.33 ± 0.79	68.08 ± 1.16	66.84 ± 1.78	67.08 ± 1.75
+ TAM	68.26 ± 1.84	68.11 ± 1.97	67.50 ± 0.67	68.06 ± 0.96	67.28 ± 1.11	67.15 ± 1.11
GraphENS	70.89 ± 0.71	70.90 ± 0.81	70.45 ± 1.25	69.87 ± 1.32	68.74 ± 0.46	68.34 ± 0.33
+ TAM	71.69 ± 0.36	72.14 ± 0.51	70.15 ± 0.18	70.00 ± 0.40	70.45 ± 0.74	70.40 ± 0.75
ReVar	72.92 ± 2.27	72.60 ± 2.26	74.56 ± 0.96	74.61 ± 0.96	73.32 ± 3.02	68.91 ± 3.13
Δ	+ 1.23(1.72%)	+ 0.46(0.64%)	+ 4.11(5.83%)	+ 4.61(6.59%)	+ 2.87(4.07%)	- 1.49(2.12%)
Table 4: Experimental results of our method ReVar and other baselines on Computers-Random. We report averaged balanced accuracy (bAcc.,%) and F1-score (%) with the standard errors over 5 repetitions on three representative GNN architectures. Highlighted are the top first and second.  $\Delta$  is the margin by which our method leads state-of-the-art method.
Dataset(Computers-Random)	GCN	GAT	SAGE
Imbalance Ratio (ρ = 25.50)	bAcc.	F1	bAcc.	F1	bAcc.	F1
Vanilla	78.43 ± 0.41	77.14 ± 0.39	71.35 ± 1.18	69.60 ± 1.11	65.30 ± 1.07	64.77 ± 1.19
Re-Weight	80.49 ± 0.44	75.07 ± 0.58	71.95 ± 0.80	70.67 ± 0.51	66.50 ± 1.47	66.10 ± 1.46
PC Softmax	81.34 ± 0.55	75.17 ± 0.57	70.56 ± 1.46	67.26 ± 1.48	69.73 ± 0.53	67.03 ± 0.6
GraphSMOTE	80.50 ± 1.11	73.79 ± 0.14	71.98 ± 0.21	67.98 ± 0.31	72.69 ± 0.82	68.73 ± 1.01
BalancedSoftmax	81.39 ± 0.25	74.54 ± 0.64	72.09 ± 0.31	68.38 ± 0.69	73.80 ± 1.06	69.74 ± 0.60
+ TAM	81.64 ± 0.48	75.59 ± 0.83	74.00 ± 0.77	70.72 ± 0.50	73.77 ± 1.26	71.03 ± 0.69
Reniode	81.64 ± 0.34	76.87 ± 0.32	72.80 ± 0.94	71.40 ± 0.97	70.94 ± 1.50	70.04 ± 1.16
+ TAM	80.50 ± 1.11	75.79 ± 0.14	71.98 ± 0.21	70.98 ± 0.31	72.69 ± 0.82	70.73 ± 1.01
GraphENS	82.66 ± 0.61	76.55 ± 0.17	75.25 ± 0.85	71.49 ± 0.54	77.64 ± 0.52	72.65 ± 0.53
+ TAM	82.83 ± 0.68	76.76 ± 0.39	75.81 ± 0.72	72.62 ± 0.57	78.98 ± 0.60	73.59 ± 0.55
ReVar	85.00 ± 0.07	82.35 ± 0.08	81.94 ± 0.54	80.94 ± 0.25	80.61 ± 0.11	77.49 ± 0.09
Δ	+ 2.17(2.62%)	+ 5.48(7.13%)	+ 6.13(8.09%)	+ 8.32(11.46%)	+ 1.63(2.06%)	+ 3.90(5.30%)
C.2 More results On Naturally Imbalanced Datasets
One of our major highlights is testing our algorithms on naturally imbalanced datasets, which is more representative of real-world scenarios. This means that in a semi-supervised setting, our training set and unlabeled data are both imbalanced, which poses a significant challenge for oversampling methods. In Table 4, we present the results of ReVar and all baselines on Computers-Random,
demonstrating that our model still achieves state-of-the-art performance. In contrast, oversampling methods such as GraphSMOTE perform poorly on this dataset due to overfitting limitations. We believe that our model's ability to regularize variance from a bottom-up perspective contributes to its superior performance on naturally imbalanced graphs, while also demonstrating its powerful generalization capabilities.
D More Analysis
D.1 More Ablation Analysis for the Loss Function
(a) CiteSeer-Semi
(b) PubMed-Semi
Figure 4: More Ablation Analysis for the Loss Function.
Details of Experimental Setup. We conduct more experiments on imbalance ratio  $10(\rho = 10)$  for the datasets CiteSeer and PubMed. For each GNN network, we add one loss regularization at a time, i.e. for GCN, we gradually add  $\mathcal{L}_{\mathrm{IR}}$ ,  $\mathcal{L}_{\mathrm{VR}}$  from the initial  $\mathcal{L}_{\mathrm{sup}}$ . The architecture we employed consisted of a 2-layers graph neural network (GNN) with 128 hidden dimensions, using GCN [16], GAT [35], and GraphSAGE [9]. The models were trained for 2000 epochs.
Analysis. Figure 4 reports more experiments for each component in the loss function. As we have seen, each component of the loss function can bring an improvement in the training effect. It is worth noting that in most cases  $\mathcal{L}_{\mathrm{VR}}$  has a larger effect boost than  $\mathcal{L}_{\mathrm{IR}}$ . We believe that  $\mathcal{L}_{\mathrm{VR}}$  plays a more important role in the loss function.
D.2 More Experiments for Variance and Imbalance Ratio Correlation in Theorem 1
(a) PubMed-GCN
Figure 5: More experiments for variance and imbalance ratio correlation in Theorem 1.
(b) PubMed-GAT
(c) PubMed-SAGE
Details of Experimental Setup. In this experiment, the classifier is not MLP, which means that the probability of a node  $v$  being classified into class  $i$  depends on the distance of this node from the center of class  $i$ . We classify half of the classes as majority classes and the other half as minority classes. Initially, each class in majority classes and minority classes has 200 labeled nodes. To generate different imbalance scenarios  $(\rho)$ , we reduce the number of labeled nodes in the minority class by 1 and add 1 to the number of labeled nodes in the majority class each time, so as to keep the number of samples in the training set consistent and eliminate the effect of the sample size variance in the training set. To calculate the variance of the model under a certain imbalance ratio  $(\rho)$ , we repeat 20 times to randomly select different but the same number of training sets, and train 2000 epochs for each fixed training set. The architecture we employed consisted of a 2-layers graph neural network (GNN) with 128 hidden dimensions, using GCN [16], GAT [35], and GraphSAGE [9].
Hypothesis testing. To test the correlation between the variance and imbalance ratio, we computed the Pearson correlation coefficient  $(\rho)$  between variance and imbalance ratio (log), as presented in the table below. The Pearson correlation coefficient, denoted as  $r$ , is a prevalent metric for gauging linear correlations. This coefficient lies between (-1) and (1), and reflects both the magnitude and direction of the correlation between two variables. An  $(r)$  value greater than (0.5) indicates a strong positive correlation. Furthermore, the p-value results from a hypothesis test with the null hypothesis  $(H_0: \rho = 0)$  and the alternative hypothesis  $(H_1: \rho \neq 0)$ , where represents the population correlation coefficient.
CiteSeer-GCN	CiteSeer-GAT	CiteSeer-SAGE	PubMed-GCN	PubMed-GAT	PubMed-SAGE
r	0.751	0.786	0.642	0.694	0.760	0.747
p-value	3.203×10-14	1.516×10-14	5.107×10-07	2.233×10-09	2.344×10-14	5.634×10-13
Given that the Pearson correlation coefficient between variance and imbalance ratio exceeds (0.5), and the p-value is below (0.01), we deduce that there is a robust correlation between variance and imbalance ratio. This relationship is statistically significant at the (0.01) significance level.
Analysis. Figure 5 reveals an intriguing pattern, as the majority of data points closely align along a regression curve exhibiting a positive slope. This observation provides substantial evidence that establishes a strong and direct linear relationship between the imbalance ratio  $(\rho)$  and the associated variance. Consequently, our findings provide compelling support for the hypothesis postulated in Theorem 1.
D.3 More Experiments for Hyperparameter Sensitivity Analysis of ReVar
(a)CiteSeer-  $\lambda_{1}$
(b)CiteSeer-  $\lambda_{2}$
(c) Computers-  $\lambda_{1}$
Figure 6: More Experiments of Hyperparameter Sensitivity Analysis for ReVar.
(d)Computers-  $\lambda_{2}$
Details of Experimental Setup. We implemented experiments on the two datasets, CiteSeer and Amazon-Computers. In this experiment, we evaluate the F1 score when we fix one  $\lambda$  while the other changes. The architecture we employed consisted of a 2-layer graph neural network (GNN) with 128 hidden dimensions, using GCN [16], GAT [35], and GraphSAGE [9]. The models were trained for 2000 epochs.
Analysis. In Figure 6, we present the sensitivity analysis of ReVar to two weight hyperparameters  $(\lambda_{1},\lambda_{2})$  in the loss function on the CiteSeer and Amazon-Computers. It is evident that ReVar exhibits varying degrees of sensitivity to hyperparameters on different datasets. Specifically, the model demonstrates lower sensitivity to a and b on CiteSeer, while the opposite is observed on Amazon-Computers. We postulate that the number of nodes (i.e., dataset size) may be a crucial factor in this discrepancy. Nevertheless, the optimal range of hyperparameter selection appears to be relatively narrow, indicating that significant efforts are not be required for model fine-tuning.
D.4 More Results of loss curve and F1 score
Details of Experimental Setup. We implemented experiments on three datasets PubMed, CiteSeer and Amazon-Computers. We compare our method ReVar and other vanilla models (only using cross-entropy and not using graph augmentation). The architecture we employed for ReVar and
(a) PubMed-Loss
(b) Computers-Loss
(d) Computers-F1 score
Figure 7: More analysis of loss curve and F1 score.
(c) PubMed-F1 Score
vanilla model consisted of a 2-layer graph neural network (GNN) with 128 hidden dimensions, using GCN [16], GAT [35], and GraphSAGE [9]. The models were trained for 2000 epochs.
Analysis. We plot the loss curve and F1 score with epoch on the PubMed and Computers datasets. By analyzing the plotted loss curves, we observe that our model demonstrates notable advantages, including faster convergence of the loss curves and improved performance across multiple datasets. For the F1 score, our model consistently outperforms alternative approaches, showcasing its superior capability in effectively capturing the complex relationships within the data and making accurate predictions.
E Elaboration of the experimental setup
In this section, we present our approach for constructing imbalanced datasets, describe our evaluation protocol, and provide comprehensive details on our algorithm as well as the baseline methods utilized in our study. To achieve this, we leverage sophisticated techniques and utilize advanced metrics to ensure the reliability and relevance of our results.
E.1 Construction for Imbalanced datasets
The detailed descriptions of the datasets are shown in Table 5. The details of label distribution in the training set of the five imbalanced benchmark datasets are in Table 7, and the label distribution of the full graph is provided in Table 7.
Table 5: Summary of the datasets used in this work.
Dataset	Nodes	Edges	Features	Classes
Cora	2,708	5,429	1,433	7
CiteSeer	3,327	4,732	3,703	6
Pubmed	19,717	44,338	500	3
Amazon-Computers	13,752	491,722	767	10
Coauthor-CS	18,333	163,788	6,805	15
Imbalanced datasets construction for Traditional Semi-supervised Settings. For each citation dataset, we adopt the "public" split and apply a random undersampling technique to make the class distribution imbalanced until the target imbalance ratio  $\rho$  is achieved. Specifically, we convert the minority class nodes to unlabeled nodes in a random manner. Regarding the co-purchased networks Amazon-Computers, we conduct replicated experiments by randomly selecting nodes as the training set in each trial. We create a random validation set that contains 30 nodes in each class, and the remaining nodes are used as the testing set.
Imbalanced datasets construction for Naturally Imbalanced Datasets. For Computers-Random and CS-Random, we constructed a training set with equal proportions based on the label distribution of the complete graph (Amazon-Computers). The label distributions for the training sets of Computers-Random and CS-Random are presented in Table 7. To achieve this, we employed a stratified sampling approach that ensured an unbiased representation of all labels in the training set.
Table 6: Label distributions in the training sets
Dataset	C0	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12	C13	C14
Cora-Semi (ρ=10)	23.26% (20)	23.26% (20)	23.26% (20)	23.26% (20)	2.32% (2)	2.32% (2)	2.32% (2)	-	-	-	-	-	-	-	-
CitSeer-Semi (ρ=10)	30.30% (20)	30.30% (20)	30.30% (20)	3.03% (2)	3.03% (2)	3.03% (2)	-	-	-	-	-	-	-	-	-
PubMed-Semi (ρ=10)	47.62% (20)	47.62% (20)	4.76% (2)	-	-	-	-	-	-	-	-	-	-	-	-
Computers-Semi (ρ=10)	18.18% (20)	18.18% (20)	18.18% (20)	18.18% (20)	18.18% (20)	1.82% (2)	1.82% (2)	1.82% (2)	1.82% (2)	-	-	-	-	-	-
Computers-Random (ρ=25.50)	3.01% (4)	15.79% (21)	10.53% (14)	3.76% (5)	38.35% (51)	2.26% (3)	3.01% (4)	6.02% (8)	15.79% (21)	1.50% (2)	-	-	-	-	-
CS-Random (ρ=41.00)	3.98% (7)	2.27% (4)	11.36% (20)	2.27% (4)	7.39% (13)	11.93% (21)	1.70% (3)	5.11% (9)	3.98% (7)	0.57% (1)	7.95% (14)	11.36% (20)	2.27% (4)	23.30% (41)	4.55% (8)
Table 7: Label distributions on the whole graphs
Dataset	C0	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12	C13	C14
Cora (ρ≈4.54)	351	217	418	818	426	298	180	-	-	-	-	-	-	-	-
CiteSeer (ρ≈2.66)	264	590	668	701	696	508	-	-	-	-	-	-	-	-	-
PubMed (ρ≈1.91)	4103	7739	7835	-	-	-	-	-	-	-	-	-	-	-	-
Amazon-Computers (ρ≈17.73)	436	2142	1414	542	5158	308	487	818	2156	291	-	-	-	-	-
Coauthor-CS (ρ≈35.05)	708	462	2050	429	1394	2193	371	924	775	118	1444	2033	420	4136	876
E.2 Architecture of GNNs
We conducted evaluations using three classic GNN architectures, namely GCN [16], GAT [35], and GraphSAGE [9]. The GNN models were constructed with different numbers of layers, namely  $L = 1,2,3$ . To enhance the learning process, each GNN layer was accompanied by a BatchNorm layer with a momentum of 0.99, followed by a PReLU activation function [11]. For the GAT architecture, we employed multi-head attention with 8 heads. We search for the best model on the validation set. The available choices for the hidden unit sizes were 64, 128, and 256.
E.3 Evaluation Protocol
We utilized the Adam optimizer [15] with an initial learning rate of 0.01 or 0.005. To manage the learning rate, we employed a scheduler based on the approach outlined in [31], which reduced the learning rate by half when there was no decrease in validation loss for 100 consecutive epochs. Weight decay with a rate of 0.0005 was applied to all learnable parameters in the model. In the initial training iteration, we trained the model for 200 epochs using the original training set for Cora, CiteSeer, PubMed, or Amazon-Computers. However, for Flickr, the training was extended to 2000 epochs in the first iteration. Subsequently, in the remaining iterations, we trained the models for 2000 epochs using the aforementioned optimizer and scheduler. The best models were selected based on validation accuracy, and we employed early stopping with a patience of 300 epochs.
E.4 Technical Details of ReVar
For all datasets Cora-Semi, CiteSeer-Semi, PubMed-Semi, Computers-Semi, Computers-Random and CS-Random, the learning rate  $\eta$  is chosen from  $\{0.0001, 0.0005, 0.001, 0.005, 0.01, 0.1\}$ . The temperature hyperparameter  $\tau$  is chosen from  $\{0.05, 0.08, 0.13, 0.16, 0.21, 0.23, 0.26\}$ . The threshold  $v$  is chosen from  $\{0.6, 0.63, 0.66, 0.7, 0.8, 0.83, 0.9, 0.93, 0.96, 0.99\}$ . The factor  $\lambda_1$  of  $\mathcal{L}_{\mathrm{VR}}$  is chosen from  $\{0.25, 0.35, 0.5, 0.85, 1, 1.5, 2, 2.15, 2.65, 3\}$ . The factor  $\lambda_2$  of  $\mathcal{L}_{\mathrm{IR}}$  is chosen from  $\{0.35, 0.5, 1, 1.25, 1.5, 2.85, 3\}$ . The factors of mask node properties and edges in  $\tilde{G}$  are chosen from  $\{0.4, 0.45, 0.5, 0.6, 0.65, 0.7\}$  and  $\{0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7\}$ . The factors of mask node properties and edges in  $\tilde{G}'$  are chosen from  $\{0.1, 0.15, 0.2, 0.3, 0.4, 0.45\}$  and  $\{0.1, 0.15, 0.2, 0.3, 0.35, 0.4, 0.45\}$ .
E.5 Technical Details of Baselines
For the GraphSMOTE method [50], we employed the branched algorithms whose edge predictions are discrete-valued, which have achieved superior performance over other variants in most experiments. Regarding the ReNode method [6], we search hyperparameters within the lower bound of cosine annealing,  $w_{\mathrm{min}} \in 0.25, 0.5, 0.75$ , and the upper bound range,  $w_{\mathrm{max}} \in 1.25, 1.5, 1.75$ , as suggested in [6]. The PageRank teleport probability was fixed at  $\alpha = 0.15$ , which is the default setting in the released codes. As for TAM [31], we performed a hyperparameter search for the coefficient of the ACM term,  $\alpha \in 1.25, 1.5, 1.75$ , the coefficient of the ADM term,  $\beta \in 0.125, 0.25, 0.5$ , and the minimum temperature of the class-wise temperature,  $\phi \in 0.8, 1.2$ , following the approach described in [31]. The sensitivity to the imbalance ratio of the class-wise temperature,  $\delta$ , was fixed at 0.4 for all the main experiments. Additionally, consistent with [31], we implemented a warmup phase for 5 iterations, as we utilized model predictions for unlabeled nodes.
E.6 Configuration
All the algorithms and models are implemented in Python and PyTorch Geometric. Experiments are conducted on a server with an NVIDIA 3090 GPU (24 GB memory) and an Intel(R) Xeon(R) Silver 4210R CPU @ 2.40GHz.
F Algorithm