SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this section, we propose an extension to the concept of graph contrastive learning that emphasizes the invariance of node representations in semi-supervised scenarios. To achieve this, we partition the nodes into two groups: labeled and unlabeled. Specifically, for the unlabeled nodes, we learn node representation invariance through the use of positive examples exclusively. For each labeled node in both views, its positive sample no longer includes itself in the other view, but instead encompasses all labeled nodes in both views belonging to the same class. Through this approach, labeled nodes belonging to the same class can be aggregated, enabling the model to learn that representation invariance now applies not to a single node, but to the complete class representation.
$$
\mathcal {L} _ {\mathrm {I R}} = - \frac {1}{| V _ {U} |} \sum_ {\mathrm {h} _ {i}, \mathrm {h} _ {i} ^ {\prime} \in V _ {U}} \operatorname {s i m} \left(\mathrm {h} _ {i} \cdot \mathrm {h} _ {i} ^ {\prime}\right) - \frac {1}{N _ {\text {a l l}}} \left(\sum_ {l = 1} ^ {k} \sum_ {\mathrm {h} _ {i}, \mathrm {h} _ {j} ^ {\prime} \in \mathbf {C} _ {l}} \operatorname {s i m} \left(\mathrm {h} _ {i} \cdot \mathrm {h} _ {j} ^ {\prime}\right) + \sum_ {l = 1} ^ {k} \sum_ {\substack {\mathrm {h} _ {i}, \mathrm {h} _ {j} \in \mathbf {C} _ {l} \\ i \neq j}} \operatorname {s i m} \left(\mathrm {h} _ {i} \cdot \mathrm {h} _ {j}\right)\right) \tag{8}
$$
where  $N_{all} = \sum_{i=1}^{k} |\mathbf{C}_i|(|\mathbf{C}_i| - 1)$ ,  $\mathrm{h}_i$  and  $\mathrm{h}_i'$  represent feature embedding in  $\tilde{G}$  and  $\tilde{G}'$  respectively for node  $i$ . To the best of our knowledge, we are the first to introduce label information into the contrastive learning paradigm to obtain invariant representations within the context of intra-class variation and address the challenge of semi-supervised imbalanced node classification. Notably, our approach only utilizes positive examples, which significantly reduces the model's training complexity and enhances its scalability and generalization capabilities.
4.3 Objective Function
To derive our ultimate objective function, we incorporate both  $\mathcal{L}_{\mathrm{VR}}$  and  $\mathcal{L}_{\mathrm{IR}}$ . These are weighted by coefficients  $\lambda_{1}$  and  $\lambda_{2}$ , respectively. Formally, the amalgamation can be represented as:
$$
\mathcal {L} _ {\text {c o m p o s i t e}} = \lambda_ {1} \mathcal {L} _ {\mathrm {V R}} + \lambda_ {2} \mathcal {L} _ {\mathrm {I R}} + \mathcal {L} _ {\sup } \tag {9}
$$
Additionally, we introduce the cross-entropy loss denoted by  $\mathcal{L}_{\mathrm{sup}}$ , which is established over a collection of labeled nodes, denominated as  $V_{L}$ .
Datasets and Baselines. We have demonstrated the efficacy of our method on five commonly used benchmark datasets across various imbalance scenarios. For the conventional setting  $(\rho = 10)$  of imbalanced node classification in [50, 24, 31], we conducted experiments on Cora, CiteSeer, Pubmed, and Amazon-Computers. More precisely, we select half of the classes as the minority classes and randomly convert labeled nodes into unlabeled ones until the training set reaches an imbalance ratio of  $\rho$ . To be more precise, for the three citation networks, we adopt the standard splits proposed by [44] as our initial splits, with an initial imbalance ratio of  $\rho$ . In order to better reflect real-world scenarios, we conducted representative experiments on the naturally imbalanced datasets Amazon-Computers and Coauthor-CS. For this setting, we utilize random sampling to construct a training set that adheres to the true label distribution of the entire graph. The comprehensive experimental settings, including the evaluation protocol and implementation details of our algorithm, are explicated in Appendix E. For baselines, we evaluate our method against classic techniques, including cross-entropy loss with re-weighting [13], PC Softmax [12], and Balanced Softmax [26], as well as state-of-the-art methods for imbalanced node classification, such as GraphSMOTE [50], GraphENS [24], ReNode [6], and TAM [31]. See Appendix E for implementation details of the baselines.
GNN Encoder Selection. As the foundational structures for our GNN encoder, we harness architectures such as GCN [16], GAT [35], and GraphSAGE [9].
Experimental Results Under Traditional Semi-supervised Settings. Table 1 presents the averaged balanced accuracy (bAcc.) and F1 score, along with their standard errors, obtained by the baselines and ReVar on three class-imbalanced node classification benchmark datasets, where  $\rho = 10$ . The outcomes showcase the superiority of ReVar in comparison to the existing methods. Our proposed technique exhibits a consistent and significant outperformance of the state-of-the-art approaches across all three datasets and three base models. In all cases, ReVar achieves a decisive advantage that underscores its efficacy in addressing the challenge of class imbalance in node classification. Due to space limitations, the experimental results and analysis on Cora are presented in the Appendix C.
Experimental Results On Naturally Imbalanced Datasets. Our model was evaluated on two naturally imbalanced datasets, Coauthor-CS ( $\rho \approx 41.0$ ) and Amazon-Computers ( $\rho \approx 17.7$ ), both of which exhibit imbalanced unlabeled data (as indicated in Table 7). The construction of the training, validation, and testing sets is detailed in Appendix E. Experimental results are presented in Table
Table 2: Comparison of our method ReVar and other baselines on CS-Random. Highlighted are the top first and second.  $\Delta$  is the margin by which our method leads state-of-the-art method.
Dataset(CS-Random)	GCN	GAT	SAGE
Imbalance Ratio(ρ=41.00)	bAcc.	F1	bAcc.	F1	bAcc.	F1
Vanilla	84.85 ± 0.16	87.12 ± 0.14	82.47 ± 0.36	84.21 ± 0.31	83.76 ± 0.27	86.22 ± 0.19
Re-Weight	87.42 ± 0.17	88.70 ± 0.10	83.55 ± 0.39	84.73 ± 0.32	85.76 ± 0.24	87.32 ± 0.16
PC Softmax	88.36 ± 0.12	88.94 ± 0.04	85.22 ± 0.31	85.54 ± 0.33	87.18 ± 0.14	88.00 ± 0.19
GraphSMOTE	85.76 ± 1.73	87.31 ± 1.32	84.65 ± 1.32	85.63 ± 1.01	85.76 ± 1.98	87.34 ± 0.98
BalancedSoftmax	87.72 ± 0.07	88.67 ± 0.07	84.38 ± 0.20	84.53 ± 0.41	86.78 ± 0.10	88.05 ± 0.09
+ TAM	88.22 ± 0.11	89.22 ± 0.08	85.48 ± 0.24	85.77 ± 0.50	87.83 ± 0.13	88.77 ± 0.07
Renode	87.53 ± 0.11	88.91 ± 0.06	85.98 ± 0.19	86.97 ± 0.09	86.13 ± 0.10	87.89 ± 0.09
+ TAM	87.55 ± 0.06	89.03 ± 0.05	86.61 ± 0.30	87.42 ± 0.24	85.21 ± 0.33	87.01 ± 0.31
GraphENS	85.97 ± 0.29	86.68 ± 0.20	85.86 ± 0.19	86.51 ± 0.32	85.39 ± 0.26	86.41 ± 0.24
+ TAM	86.34 ± 0.12	87.36 ± 0.08	86.29 ± 0.20	87.28 ± 0.13	85.99 ± 0.13	87.25 ± 0.07
ReVar	88.44 ± 0.16	89.54 ± 0.11	87.33 ± 0.04	88.33 ± 0.06	90.11 ± 0.11	91.18 ± 0.11
Δ	+ 0.08(0.09%)	+ 0.32(0.36%)	+ 0.72(0.83%)	+ 0.91(1.04%)	+ 2.28(2.60%)	+ 2.41(2.71%)
4 and Table 2. Notably, ReVar outperformed other methods consistently on these two datasets, a significant finding given the challenging nature of the imbalanced data in these datasets.
5.3 Further Analysis of ReVar
Analysis of Each Component in the Loss Function. As shown in Figure 3(a), each component of our loss function can bring performance improvements. In particular, in all three settings in the figure, our full loss achieves the best F1 scores. In all cases, VR loss improves the performance of the model, proving our hypothesis that can approximate the variance of the model to improve the performance of the model. More analysis is presented in Appendix D.
(a) Computers-Semi
(b) Pubmed-  $\lambda_{1}$
(c) PubMed-  $\lambda_{2}$
(d) CiteSeer-Loss
Figure 3: Analysis of ReVar.
Sensitive to Hyperparameters  $\lambda_{1}$  and  $\lambda_{2}$ . The two intensity terms,  $\lambda_{1}$  and  $\lambda_{2}$ , corresponding to the  $\mathcal{L}_{\mathrm{VR}}$  and  $\mathcal{L}_{\mathrm{IR}}$ , respectively, is depicted in Figure 3(b), 3(c). We explore the impact of perturbations on model performance when one of the two intensity terms is held constant while the other is varied. We believe that a smaller value of  $\lambda$  is inadequate to reflect the regularization of ReVar. Instead, suitable values of  $\lambda$  can significantly enhance the performance of ReVar. More analysis is presented in Appendix D.
Analysis of Test Loss and F1 Score Curves. The present study includes a comparison of ReVar and Vanilla models(only use cross-entropy) in terms of their loss and F1 score. As shown in Figure 3(d), various models were evaluated. The ReVar model, which incorporates additional  $\mathcal{L}_{\mathrm{VR}}$  and  $\mathcal{L}_{\mathrm{IR}}$  components, demonstrated a faster convergence of loss, as evidenced by the larger slope in Figure 3(d). More analysis is presented in Appendix D.
This paper presents a novel approach to address imbalanced node classification. Our method integrates imbalanced node classification and the Bias-Variance Decomposition framework, establishing a theoretical foundation that closely links data imbalance to model variance. Additionally, we employ graph augmentation techniques to estimate model variance and design a regularization term to mitigate the impact of class imbalance. We conduct exhaustive testing to demonstrate superior performance compared to state-of-the-art methods in various imbalanced scenarios. Future work includes extending ReVar and its theories to the fields of computer vision and natural language processing. Moreover, we anticipate the emergence of more effective theories and algorithms to be developed and flourish.
This work is supported by National Natural Science Foundation of China No.U2241212, No.62276066.
Reproducibility Statements
The model implementation and data is released at https://github.com/yanliang3612/ReVar.
[1] Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine-learning practice and the classical bias-variance trade-off. Proceedings of the National Academy of Sciences, 116(32):15849-15854, 2019.
[2] Erica Briscoe and Jacob Feldman. Conceptual complexity and the bias/variance tradeoff. Cognition, 118(1):2-16, 2011.
[3] Jiarui Cai, Yizhou Wang, and Jenq-Neng Hwang. Ace: Ally complementary experts for solving long-tailed recognition in one-shot. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 112-121, 2021.
[4] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with label-distribution-aware margin loss. Advances in neural information processing systems, 32, 2019.
[5] Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-357, 2002.
[6] Deli Chen, Yankai Lin, Guangxiang Zhao, Xuancheng Ren, Peng Li, Jie Zhou, and Xu Sun. Topology-imbalance learning for semi-supervised node classification. Advances in Neural Information Processing Systems, 34:29885-29897, 2021.
[7] Yoav Freund and Robert E Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 55(1):119-139, 1997.
[8] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
[9] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
[10] Hui Han, Wen-Yuan Wang, and Bing-Huan Mao. Borderline-smote: a new over-sampling method in imbalanced data sets learning. In International conference on intelligent computing, pages 878-887. Springer, 2005.
[11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026-1034, 2015.
[46] Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 974–983, 2018.
[47] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in neural information processing systems, 33:5812-5823, 2020.
[48] Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. Advances in Neural Information Processing Systems, 34:18408-18419, 2021.
[49] Hengrui Zhang, Qitian Wu, Junchi Yan, David Wipf, and Philip S. Yu. From canonical correlation analysis to self-supervised graph neural networks. In Advances in Neural Information Processing Systems, 2021.
[50] Tianxiang Zhao, Xiang Zhang, and Suhang Wang. Graphsmote: Imbalanced node classification on graphs with graph neural networks. In Proceedings of the 14th ACM international conference on web search and data mining, pages 833-841, 2021.
[51] Yizhen Zheng, Shirui Pan, Vincent C. S. Lee, Yu Zheng, and Philip S. Yu. Rethinking and scaling up graph contrastive learning: An extremely efficient approach with group discrimination. In Advances in Neural Information Processing Systems, 2022.
[52] Boyan Zhou, Quan Cui, Xiu-Shen Wei, and Zhao-Min Chen. Bbn: Bilateral-branch network with cumulative learning for long-tailed visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9719–9728, 2020.
[53] Zhi-Hua Zhou and Xu-Ying Liu. Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Transactions on knowledge and data engineering, 18(1):63-77, 2005.
[54] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep graph contrastive representation learning. In International Conference on Machine Learning Workshop on Graph Representation Learning and Beyond, 2020.
[55] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph contrastive learning with adaptive augmentation. In The Web Conference, 2021.
Imbalanced Learning in Traditional ML. Most real-world data is naturally imbalanced, presenting a significant challenge in training fair models that are not biased towards majority classes. To address this problem, various approaches have been commonly utilized. Ensemble learning [7, 18, 52, 39, 20, 3] combines the outputs of multiple weak classifiers. Data re-sampling methods [5, 10, 29, 27, 14, 37] aim to adjust the label distribution in the training set by synthesizing or duplicating samples from the minority class. Another approach tackles the imbalance issue by modifying the loss function, assigning larger weights to minority classes or adjusting the margins between different classes [53, 33, 4, 32, 42, 26, 38]. Post-hoc correction methods compensate for the imbalanced classes during the inference step, after completing the model training [14, 34, 21, 12]. Although these techniques have been extensively applied to i.i.d. data, extending them to graph-structured data poses non-trivial challenges.
Graph Contrastive Learning. Contrastive methods, which have proven effective for unsupervised learning in vision, have also been adapted for graph data. One notable approach is DGI [36], which presents a framework for unsupervised node-level representation learning that maximizes global mutual information. Other approaches, such as GRACE [54], GCA [55], and GraphCL [47], utilize augmented graphs to optimize the similarity between positive node pairs and minimize negative pairs. CCA-SSG [49] introduces an efficient loss function based on canonical correlation analysis, eliminating the need for negative samples. Incorporating community information, gCooL [17] enhances node representations and downstream task performance. GGD [51] simplifies the mutual information loss function by directly discriminating between two sets of node samples, resulting in faster computation and lower memory usage. These contrastive methods exhibit potential for improving unsupervised learning on graph data. However, it is important to note that our model operates within the context of semi-supervised learning, which significantly differs from the mechanisms employed by these models.
B.1 Proofs of Theorem 1
Theorem 1 Under the condition that  $\sum_{i=1}^{c} n_i$  is a constant, the variance  $\sum_{i=1}^{c} \mathbb{E}_x\left[\frac{1}{n_i} h^T(x) \Lambda^i h(x)\right]$  reach its minimum when all  $n_i$  equal.
Proof The expression  $\sum_{i=1}^{c} \mathbb{E}_x\left[\frac{1}{n_i} h^T(x) \Lambda^i h(x)\right]$  can be equivalently expressed as  $\sum_{i=1}^{c} \frac{1}{n_i} \mathbb{E}_x\left[h^T(x) \Lambda^i h(x)\right]$ . As previously assumed, the  $\mathbb{E}_x[h^T(x) \Lambda^i h(x)]$  is the same for different  $i$ , which implies that our goal is to demonstrate that  $\sum_{i=1}^{c} \frac{1}{n_i}$  is minimized when all  $n_i$  are equal.
Let  $m$  be  $\sum_{i=1}^{c} n_i$ . We wish to find the extremum of the sum of their reciprocals, which is given by
$$
S = \frac {1}{n _ {1}} + \frac {1}{n _ {2}} + \dots + \frac {1}{n _ {c}}. \tag {10}
$$
Using the inequality of arithmetic and harmonic means, we have
$$
\frac {c}{\frac {1}{n _ {1}} + \frac {1}{n _ {2}} + \cdots + \frac {1}{n _ {c}}} \leq \frac {n _ {1} + n _ {2} + \cdots + n _ {c}}{c}, \tag {11}
$$
with equality if and only if  $n_1 = n_2 = \dots = n_c$ . Rearranging, we get
$$
\frac {c ^ {2}}{n _ {1} + n _ {2} + \cdots + n _ {c}} \leq S, \tag {12}
$$
with equality if and only if  $n_1 = n_2 = \dots = n_c$ . Since  $m = n_1 + n_2 + \dots + n_c$ , we have
$$
\frac {c ^ {2}}{m} \leq S, \tag {13}
$$
with equality if and only if  $n_1 = n_2 = \dots = n_c = \frac{m}{c}$ . Therefore, when all the  $c$  numbers are equal, the sum of their reciprocals is minimized and given by  $S = \frac{e^2}{m}$ .
We can also use the method of Lagrange multipliers. Let  $f(n_{1},n_{2},\ldots ,a_{c}) = \frac{1}{n_{1}} +\frac{1}{n_{2}} +\dots +\frac{1}{n_{c}}$  be the function that we want to extremize. Then, the Lagrangian is:
$$
\mathcal {L} \left(n _ {1}, n _ {2}, \dots , n _ {c}, \lambda\right) = f \left(n _ {1}, n _ {2}, \dots , n _ {c}\right) + \lambda \left(n _ {1} + n _ {2} + \dots + n _ {c} - m\right). \tag {14}
$$
Taking the partial derivatives of  $\mathcal{L}$  with respect to  $a_{i}$  and  $\lambda$ , we get:
$$
\frac {\partial \mathcal {L}}{\partial n _ {i}} = - \frac {1}{n _ {i} ^ {2}} + \lambda
$$
$$
\frac {\partial \mathcal {L}}{\partial \lambda} = n _ {1} + n _ {2} + \dots + n _ {c} - m. \tag {15}
$$
Setting these partial derivatives to zero, we get:
$$
n _ {1} = n _ {2} = \dots = n _ {c} = \frac {m}{c}, \lambda = \frac {c ^ {2}}{m ^ {2}}. \tag {16}
$$
Thus, when the  $c$  numbers are equal, their reciprocal sum is minimized and is equal to  $\frac{c^2}{m}$ . Moreover, since  $S$  is a continuously differentiable function of  $n_1, n_2, \ldots, n_c$ , this extremum is a minimum.
Therefore, we have shown that the reciprocal sum of  $c$  numbers with a sum of  $m$  is minimized and equal to  $\frac{c^2}{m}$  when the  $c$  numbers are equal.
B.2 Proofs of Lemma 1 and More Details for Estimating the Expectation with Unlabeled Nodes (Section 3.2)
B.2.1 Proofs of Lemma 1
Lemma 1 Under the above assumption for  $h^i \sim N(\mu^i, \Lambda^i)$ ,  $C^i \sim N\left(\mu^i, \frac{1}{n_i} \Lambda^i\right)$ , minimizing the  $\sum_{i=1}^{c} \mathbb{E}_x[\operatorname{Var}(x)]$  is equivalent to minimizing Equation 17:
$$
\frac {1}{N} \sum_ {x \in G} \sum_ {i = 1} ^ {c} \left(h (x) ^ {T} \frac {1}{\sqrt {2 n _ {i}}} \left(h _ {1} ^ {i} - h _ {2} ^ {i}\right)\right) ^ {2} = \frac {1}{N} \sum_ {k = 1} ^ {n _ {j}} \sum_ {j = 1} ^ {c} \sum_ {i = 1} ^ {c} \left(\left(\mu_ {k} ^ {j} + \epsilon_ {k} ^ {j}\right) ^ {T} \frac {1}{\sqrt {2 n _ {i}}} \left(\epsilon_ {1} ^ {i} - \epsilon_ {2} ^ {i}\right)\right) ^ {2}. \tag {17}
$$
Proof Let  $\epsilon^i$  denote  $h^i -\mu^i$ , which follows the distribution  $\epsilon^i\sim N(0,\Lambda^i)$ . Similarly, we denote  $e^{i} = C^{i} - \mu^{i}$  and it follows that  $e^i\sim N\left(0,\frac{1}{n_i}\Lambda^i\right)$ .
$$
\begin{array}{l} \operatorname {V a r} (x) = \sum_ {i = 1} ^ {c} \mathbb {E} _ {D} \left[ \left(h ^ {T} (x) C ^ {i} - \mathbb {E} _ {D} \left[ h ^ {T} (x) C ^ {i} \right]\right) ^ {2} \right] \\ = \sum_ {i = 1} ^ {c} \mathbb {E} _ {e ^ {i}} \left[ \left(h ^ {T} (x) \left(\mu^ {i} + e ^ {i}\right) - h ^ {T} (x) \mu^ {i}\right) ^ {2} \right] \tag {18} \\ = \sum_ {i = 1} ^ {c} \mathbb {E} _ {e ^ {i}} \left[ \left(h ^ {T} (x) e ^ {i}\right) ^ {2} \right] = \sum_ {i = 1} ^ {c} \frac {1}{n _ {i}} h ^ {T} (x) \Lambda^ {i} h (x), \\ \end{array}
$$
$$
\sum_ {i = 1} ^ {c} \mathbb {E} _ {x} [ \operatorname {V a r} (x) ] = \sum_ {i = 1} ^ {c} \mathbb {E} _ {x} \left[ \frac {1}{n _ {i}} h ^ {T} (x) \Lambda^ {i} h (x) \right]. \tag {19}
$$
Motivated by stochastic gradient descent, we opt to sample an  $e^i$  on each occasion and compute, as opposed to directly calculating the expectation  $\mathbb{E}_{D\subset G}$ .
At present, we remain uncertain about how to sample the noise term  $e^i$  associated with the class center  $C_i$ . We put forth a proposition to estimate the class center noise  $e$  utilizing the feature noise  $\epsilon$ . Under the supposition that when  $v \in C_k$ , the  $j_{th}$  element of the feature  $f(v)$  adheres to a Gaussian distribution:
$$
\epsilon^ {i} \sim N \left(0, \Lambda^ {i}\right), e ^ {i} \sim N \left(0, \frac {1}{n _ {i}} \Lambda^ {i}\right). \tag {20}
$$
Consequently, multiplying the noise term  $\epsilon^i$  in Equation 20 by  $\frac{1}{\sqrt{n_i}}$  yields a random variable that exhibits an identical distribution to  $e^i$ :
$$
\frac {1}{\sqrt {n _ {i}}} \epsilon^ {i} \sim N \left(0, \frac {1}{n _ {i}} \Lambda^ {i}\right) \tag {21}
$$
Moreover, how might we compute  $\varepsilon_{i}$ ? In practical scenarios, we merely possess the feature  $h$ . However, we are able to calculate the disparity between two features,  $h_1^i$  and  $h_2^i$ , originating from the same class  $i$ :
$$
\frac {1}{\sqrt {2 n _ {i}}} \left(h _ {1} ^ {i} - h _ {2} ^ {i}\right) = \frac {1}{\sqrt {2 n _ {i}}} \left(\epsilon_ {1} - \epsilon_ {2}\right) \sim N \left(0, \frac {1}{n _ {i}} \Lambda^ {i}\right). \tag {22}
$$
Incorporating this into Equation 19, the loss is expressed as:
$$
\frac {1}{N} \sum_ {x \in G} \sum_ {i = 1} ^ {c} \left(h (x) ^ {T} \frac {1}{\sqrt {2 n _ {i}}} \left(h _ {1} ^ {i} - h _ {2} ^ {i}\right)\right) ^ {2}. \tag {23}
$$
B.2.2 More Details for Estimating the Expectation with Unlabeled Nodes (Section 3.2)
Lemma 1 suggests that  $\sum_{i=1}^{c} \mathbb{E}_x[\mathrm{Var}(x)]$  can be estimated by sampling labeled node pairs  $(h_1^i, h_2^i)$  from the same class  $i$ . However, due to the scarcity of data in the minority class, this can often be challenging. In Section 3.2, we address this issue by utilizing random data augmentation and defining nodes from different views as members of the same class, enabling us to sample node pairs for Equation 3 and perform embedding subtraction.
However, as we mentioned, the pseudo node pairs lack information regarding their class membership, making it impossible to assign appropriate  $\frac{1}{\sqrt{2n_i}}$  values for different  $i$  in Equation 3. These coefficients play a crucial role in compensating for the minority class, making it imperative to reintroduce information about the class number when constructing variance regularization. Therefore, in the second step, we replace  $h$  in Equation 3 with the class center  $C^i$ , as demonstrated in Equation 4. As  $C^i = \mu^i + e^i$  and the variance of  $e^i$  is proportional to  $\frac{1}{n_i}$ , using Equation 4 to estimate Lemma 1 can provide similar compensatory effects for the minority class. We present the proof for the aforementioned propositions below.
Proof Firstly, the  $\sum_{i=1}^{c} \mathbb{E}_{x}\left[\operatorname{Var}(x)\right]$  can be decomposed as follows: