SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 1 ReVar
Input: Imbalanced Graph  $\mathbf{G} = (\mathbf{V},\mathbf{E},\mathbf{V}_{\mathbf{L}})$ , feature matrix  $\mathbf{X}$ , adjacency matrix  $\mathbf{A}$ , unlabeled set  $\mathbf{V}_{\mathbf{U}} = \mathbf{V} - \mathbf{V}_{\mathbf{L}}$ , label matrix  $\mathbf{Y}$ , feature matrix of labeled nodes and unlabeled nodes  $\mathbf{X}_{\mathbf{L}}$  and  $\mathbf{X}_{\mathbf{U}}$ , the number of classes  $\mathbf{k}$ , the threshold  $v$  for determining whether a node has confident prediction, temperature hyperparameter  $\tau$ , GNN model  $f_{\theta}$ , the classifier  $h_{\theta}$ , learning rate  $\eta$ , The total number  $T$  of epochs for model training. The  $\sin(\cdot, \cdot)$  computes the cosine similarity between two vectors,  $CE$  represents the cross-entropy loss function, and  $\mathbb{I}(\cdot)$  is an indicator function.
1: for  $t = 0,1,\ldots ,T$  do
2: Generate two differently augmented views  $\tilde{G} = (\tilde{\mathrm{A}},\tilde{\mathrm{X}})$  and  $\tilde{G}^{\prime} = (\tilde{\mathrm{A}}^{\prime},\tilde{\mathrm{X}}^{\prime})$  from original graph  $\mathbf{G}$ .
3:  $\bar{\mathbf{O}}_L\gets f_\theta (\bar{\mathrm{A}},\bar{\mathrm{X}}_L),\bar{\mathbf{O}}_U\gets f_\theta (\bar{\mathrm{A}},\bar{\mathrm{X}}_U)$
4:  $\tilde{\mathbf{O}}_L^{\prime}\gets f_{\theta}(\tilde{\mathrm{A}}^{\prime},\tilde{\mathrm{X}}_{L}^{\prime}),\tilde{\mathbf{O}}_{U}^{\prime}\gets f_{\theta}(\tilde{\mathrm{A}}^{\prime},\tilde{\mathrm{X}}_{U}^{\prime})$
5: % Component 1: Supervised Loss
6:  $\mathcal{L}_{sup}\gets \frac{1}{2} CE(h_{\theta}(\tilde{\mathbf{O}}_L),\mathrm{Y}) + \frac{1}{2} CE(h_{\theta}(\tilde{\mathbf{O}}_U'),\mathrm{Y})$
7: % Component 2: Intra-Class Aggregation Regularization
8:  $\mathcal{L}_{\mathrm{IR}}\gets -\frac{1}{N_U}\sum_{i = 1}^{N_U}\sin (\tilde{\mathbf{O}}_i,\tilde{\mathbf{O}}_i')$ $1 / N_{all}(\sum_{i = 1}^{N_L}\sum_{\substack{(i,j)\in same}}^{N_L})\sin (\tilde{\mathbf{O}}_i,\tilde{\mathbf{O}}_j')$  +  $\sum_{i = 1}^{N_L}\sum_{\substack{j = 1,j\neq i\\ (i,j)\in same}}^{N_L}\sin (\tilde{\mathbf{O}}_i,\tilde{\mathbf{O}}_j)$
9: % Component 3: Variance-constrained Optimization with Adaptive Regularization
0: for  $i = 1, 2, \ldots, k$  do
1: Compute the class centers  $\tilde{\mathbf{C}}_i$  and  $\tilde{\mathbf{C}}^{\prime}_i$  for class  $i$
2: end for
3:  $\triangleright$  Obtain the Label Probability for Each Node
4: for  $i = 1,2,\ldots ,|\mathbf{V}|$  do
5:  $\tilde{\mathbf{p}}_i\gets \mathrm{Softmax}(\tilde{\mathbf{O}}_i\cdot \left[\tilde{\mathbf{C}}_1,\ldots ,\tilde{\mathbf{C}}_k\right]^T)$
6:  $\tilde{\mathbf{p}}_i^{\prime}\gets \mathrm{Softmax}\left(\tilde{\mathbf{O}}_i^{\prime}\cdot \left[\tilde{\mathbf{C}}^{\prime}_{1},\ldots ,\tilde{\mathbf{C}}^{\prime}_{k}\right]^T\right)$
17: end for
8:  $\triangleright$  Eliminate Nodes with Low Confidence.
9:  $\mathbf{V}_{\mathrm{conf}}\gets \{i\mid \mathbb{I}(\max (\tilde{\mathbf{p}}_i^{\prime}) > v) = 1,\forall i\in V_U\}$
20: Replace Labeled Node Prediction
21: for  $i = 1,2,\ldots ,|\mathbf{V}_{\mathbf{L}}|$  do
22:  $\tilde{\mathbf{p}}_i^{\prime}\gets \mathbf{Y}_i$
23: end for
24:  $\mathcal{L}_{\mathbf{VR}}\gets \frac{1}{|V_{\mathrm{conf}}|}\sum_{i\in V_{\mathrm{conf}}}CE(\tilde{\mathbf{p}}_i',\tilde{\mathbf{p}}_i) + \frac{1}{|V_L|}\sum_{v_i\in V_L}CE(\mathbf{Y}_i,\tilde{\mathbf{p}}_i)$
25:  $\mathcal{L}_{\text {Training }} \gets \mathcal{L}_{\text {sup }} + \lambda_1 \mathcal{L}_{VR} + \lambda_2 \mathcal{L}_{IR}$
26:  $\theta \gets \theta -\eta (\nabla \mathcal{L}_{Training})$
27: end for
G Notations
Table 8: Notation table.
Indices
n	The number of nodes, n = |V|.
f	The node feature dimension.
k	The number of different classes.
D	The dimension of the embedding space, or the dimension of the last layer of GNNs.
Parameters
G	An undirected and unweighted graph.
V	The node set of G.
E	The edge set of G.
X	The feature matrix of G, X ∈ Rn×f.
VL	The set of labeled nodes of G.
A	The adjacency matrix of G, A ∈ {0,1}n×n.
N(v)	The set of 1-hop neighbors for node v.
VU	The set of unlabeled nodes, VU := V \ VL.
Ci	The labeled set for class i.
ρ	Imbalance ratio of a dataset, ρ = maxi|Ci| / mini|Ci|.
G	A view from the original graph G, G' = (A', X').
G'	Another view from the original graph G, G' = (A', X').
h	Output of GNN network, h = fθ(A, X).
S	The probability distribution reference matrix, S := [C1; C2; ...; Ck].
C	The collection of class centers, C := (C1; C2; ...; Ck).
πi	The probability distribution for node i.
yi	The one-hot label vector for node i.
Lsup	Supervised loss.
LIR	Intra-class aggregation regularization.
LVR	Variance-constrained optimization with adaptive regularization.
Lfinal	The sum of all losses.
Vconf	The set of nodes with confident predictions.
λ1	The weight for LVR.
λ2	The weight for LIR.
v	The threshold for determining whether a node has confident prediction.
τ	The temperature hyperparameter.
α	The size threshold of nodes being added in each class per round.
η	Learning rate of GNN model.
Functions
I(·)	The indicator function.
sim(·)	The function that computes the cosine similarity between two vectors.
CE(·)	The cross-entropy function.
fθ	GNN model.