SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The primary problem defined in Eq. (2) without constraints can be represented as:
$$
\underset {\mathbf {B}} {\arg \min } \mathcal {L} = \| \mathbf {I} - \mathbf {B} \| _ {F} ^ {2} + \lambda \operatorname {t r} \left(\mathbf {B} ^ {\top} \tilde {\mathbf {L}} \mathbf {B}\right). \tag {9}
$$
Comparing Eq. (7) with Eq. (9), the only difference lies in the first term, where the feature matrix  $\mathbf{X}$  is set to be identity matrix  $\mathbf{I}$ . Then, we can follow the same steps to solve the Eq. (9).
If we use the multiple-step gradient descent with the stepsize  $\gamma = \frac{1}{2(1 + \lambda)}$ , then we have the following update steps:
$$
\mathbf {B} ^ {k} = \frac {\lambda}{1 + \lambda} \tilde {\mathbf {A}} \mathbf {B} ^ {k - 1} + \frac {1}{1 + \lambda} \mathbf {I}.
$$
Then, for  $K$  steps iteration,  $B^K$  will be:
$$
\mathbf {B} ^ {K} = \left(\left(\frac {\lambda}{1 + \lambda}\right) ^ {K} \tilde {\mathbf {A}} ^ {K} + \sum_ {i = 0} ^ {K - 1} \frac {1}{1 + \lambda} \left(\frac {\lambda}{1 + \lambda}\right) ^ {i} \tilde {\mathbf {A}} ^ {i}\right), \tag {10}
$$
which is the propagation matrix of APPNP in Eq. (8). As a result, the message passing of APPNP can be written as  $\mathbf{F} = \mathbf{B}\mathbf{X}$
If we use one-step gradient descent to solve Eq. (9), then  $\mathbf{B}$  can be represented as:
$$
\mathbf {B} = \mathbf {I} - \left. \gamma \frac {\partial \mathcal {L}}{\partial \mathbf {B}} \right| _ {\mathbf {B} = \mathbf {I}} = (1 - 2 \gamma \lambda) \mathbf {I} + 2 \gamma \lambda \tilde {\mathbf {A}}.
$$
If we set the step size  $\gamma = \frac{1}{2\lambda}$ , then the  $\mathbf{B} = \tilde{\mathbf{A}}$ . As a result, the aggregation in GCN can also be represented by  $\mathbf{F} = \mathbf{BX}$ .
Proposition B.1. The influence scores from all labeled nodes to any unlabeled node  $i$  will be the equal, i.e.,  $\sum_{j \in \mathcal{V}_L} I_i(j) = c$ , when using the unbiased graph structure  $\mathbf{B}$  obtained from the optimization problem in Eq. (2) as the propagation matrix in GNNs.
Proof. Following the definition in [23], the influence of node  $j$  on node  $i$  can be represented by  $I_{i}(j) = \text{sum}\left[\frac{\partial\mathbf{h}_{i}}{\partial\mathbf{x}_{j}}\right]$ , where  $\mathbf{h}_i$  is the representation of node  $i$ ,  $\mathbf{x}_j$  is the input feature of node  $j$ , and  $\left[\frac{\partial\mathbf{h}_i}{\partial\mathbf{x}_j}\right]$  represents the Jacobian matrix.
If we use the unbiased graph  $\mathbf{B}$  as the propagation matrix, then  $\mathbf{H} = \mathbf{B}\mathbf{X}$ . Thus,  $\mathbf{h}_{ij} = \sum_{k=0}^{n} \mathbf{B}_{ik} \mathbf{x}_{kj}$ . The Jacobian matrix  $\left[\frac{\partial \mathbf{h}_i}{\partial \mathbf{x}_j}\right]$  can be written as:
$$
\left[ \frac {\partial \mathbf {h} _ {i}}{\partial \mathbf {x} _ {j}} \right] = \operatorname {D i a g} \left(\left[ \mathbf {B} _ {i j}, \mathbf {B} _ {i j}, \dots , \mathbf {B} _ {i j} \right]\right), \tag {11}
$$
where  $\mathrm{Diag}$  represents the diagonal matrix. As a result,  $I_{i}(j) = sum\left[\frac{\partial\mathbf{h}_{i}}{\partial\mathbf{x}_{j}}\right] = n\mathbf{B}_{ij}$ .
Suppose the constraint  $\mathbf{BT1}_n = \frac{c}{n}$  is satisfied, then the influence scores from all labeled nodes to the unlabeled node  $i$  can be represented as:
$$
\sum_ {j \in \mathcal {V} _ {L}} I _ {i} (j) = \sum_ {j \in \mathcal {V} _ {L}} n \mathbf {B} _ {i j} = n \mathbf {B T 1} _ {n} = c. \tag {12}
$$
Finally, the influence scores from all labeled nodes to any unlabeled node  $i$  are equal.
C Datasets Statistics
In the experiments, the data statistics used in Section 4 are summarized in Table 6. For Cora, CiteSeer and PubMed dataset, we adopt different label rates, i.e., 5, 10, 20, and  $60\%$  labeled nodes per class, to get a more comprehensive comparison. For label rates 5, 10, and 20, we use 500 nodes for validation and 1000 nodes for testing. For label rates of  $60\%$  labeled node per class, we use half of the rest nodes for validation and the remaining half for the test. For each labeling rate and dataset, we adopt 10 random splits for each dataset. For the ogbn-arxiv dataset, we follow the original data split.
Table 6: Dataset Statistics.
Dataset	Nodes	Edges	Features	Classes
Cora	2,708	5,278	1,433	7
CiteSeer	3,327	4,552	3,703	6
PubMed	19,717	44,324	500	3
Coauthor CS	18,333	81,894	6,805	15
Coauthor Physics	34,493	247,962	8,415	5
Amazon Computer	13,381	245,778	767	10
Amazon Photo	7,487	119,043	745	8
Ogbn-Arxiv	169,343	1,166,243	128	40
D Hyperparameters Setting
In this section, we describe in detail the search space for parameters of different experiments.
For all deep models, we use 3 transformation layers with 256 hidden units for the ogbn-arxiv dataset and 2 transformation layers with 64 hidden units for other datasets. For all methods, the following common hyperparameters are tuned based on the loss and validation accuracy from the following search space:
- Learning Rate:  $\{0.01, 0.05\}$
- Dropout Rate:  $\{0, 0.5, 0.8\}$
Weight Decay:  $\{0,5\mathrm{e} - 5,5\mathrm{e} - 4\}$
For APPNP and Label Propagation, we tune the teleport probability  $\alpha$  in  $\{0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9\}$ . For  $\mathrm{GRADE}^2$ , we set the hidden dimension 256, the temperature in  $\{0.2, 0.5, 0.8, 1, 1.1, 1.5, 1.7, 2\}$ , which covers all the best values in their original paper. For  $\mathrm{SRGNN}^3$ , we set the weight of CMD loss in  $\{0.1, 0.5, 1, 1.5, 2\}$ .
For the proposed LPSL, we set the  $c$  in the range [0.7, 1.3],  $\rho$  in  $\{0.01, 0.001\}$ ,  $\gamma$  in  $\{0.01, 0.001\}$ ,  $\beta$  in  $\{1e-4, 1e-5, 5e-5, 1e-6, 5e-6, 1e-7\}$ . For the LPSLAPPNP, we set  $\lambda$  in  $\{8, 9, 10, 11, 12, 13, 14, 15\}$ . For the LPSLGCN, we set  $\lambda$  in  $\{1, 2, 3, 4, 5, 6, 7, 8\}$ .
E Node Classification Results
For the semi-supervised node classification task, we choose nine common used datasets including three citation datasets, i.e., Cora, Citeseer and Pubmed [27], two coauthors datasets, i.e., CS and Physics, two Amazon datasets, i.e., Computers and Photo [28], and one OGB datasets, i.e., ogbn-arxiv [29].
To the best of our knowledge, there are no previous works that aim to address the label position bias. In this work, we select three GNNs, namely, GCN [17], GAT [31], and APPNP [18], two Non-GNNs, MLP and Label Propagation [19], as baselines. Furthermore, we also include GRADE [32], a method designed to mitigate degree bias. Notably, SRGNN [33] demonstrates that if labeled nodes are gathered locally, it could lead to an issue of feature distribution shift. SRGNN aims to mitigate the feature distribution shift issue and is also included as a baseline. The overall performance are shown in Table 7.
In this subsection, we explore the impact of different hyperparameters, specifically the smoothing term  $\lambda$  and the constraint  $c$ , on our model. We conducted experiments on the Cora and CiteSeer datasets using ten random data splits with 20 labels per class. The accuracy of different  $\lambda$  and  $c$  values for  $\mathrm{LPSL_{APPNP}}$  and  $\mathrm{LPSL_{GCN}}$  on the Cora and CiteSeer datasets are illustrated in Figure F.
$^{2}$ https://github.com/BUPT-GAMMA/Uncovering-the-Structural-Fairness-in-Graph-Contrastive-Learning/  $^{3}$ https://github.com/GentleZhu/Shift-Robust-GNNs
Table 7: The overall results of the node classification task.
Dataset	Label Rate	MLP	LP	GCN	APPNP	GAT	GRADE	SRGNN	LPSL_GCN	LPSL(APPNP
Cora	5	42.34 ± 3.31	57.60 ± 5.71	70.68 ± 2.17	75.86 ± 2.34	72.97 ± 2.23	69.51 ± 6.79	70.77 ± 1.82	76.58 ± 2.37	77.24 ± 2.18
10	51.34 ± 3.37	63.76 ± 3.60	76.50 ± 1.42	80.29 ± 1.00	78.03 ± 1.17	74.95 ± 2.46	75.42 ± 1.57	80.39 ± 1.17	81.59 ± 0.98
20	59.23 ± 2.52	67.87 ± 1.43	79.41 ± 1.30	82.34 ± 0.67	81.39 ± 1.41	77.41 ± 1.49	78.42 ± 1.75	82.74 ± 1.01	83.24 ± 0.75
60%	76.49 ± 1.13	86.05 ± 1.35	88.60 ± 1.19	88.49 ± 1.28	88.68 ± 1.13	86.84 ± 0.99	87.17 ± 0.95	88.75 ± 1.21	88.62 ± 1.69
CiteSeer	5	41.05 ± 2.84	39.06 ± 3.53	61.27 ± 3.85	63.92 ± 3.39	62.60 ± 3.34	63.03 ± 3.61	64.84 ± 3.41	65.65 ± 2.47	65.70 ± 2.18
10	47.99 ± 2.71	42.29 ± 3.26	66.28 ± 2.14	67.57 ± 2.05	66.81 ± 2.10	64.20 ± 3.23	67.83 ± 2.19	67.73 ± 2.57	68.76 ± 1.77
20	56.96 ± 1.80	46.15 ± 2.31	69.60 ± 1.67	70.85 ± 1.45	69.66 ± 1.47	67.50 ± 1.76	69.13 ± 1.99	70.73 ± 1.32	71.25 ± 1.14
60%	73.15 ± 1.36	69.39 ± 2.01	76.88 ± 1.78	77.42 ± 1.47	76.70 ± 1.81	74.00 ± 1.87	74.57 ± 1.57	77.18 ± 1.64	77.56 ± 1.44
PubMed	5	58.48 ± 4.06	65.52 ± 6.42	69.76 ± 6.46	72.68 ± 5.68	70.42 ± 5.36	66.90 ± 6.49	69.38 ± 6.48	73.46 ± 4.64	73.57 ± 5.30
10	65.36 ± 2.08	68.39 ± 4.88	72.79 ± 3.58	75.53 ± 3.85	73.35 ± 3.83	73.31 ± 3.75	72.69 ± 3.49	75.67 ± 4.42	76.18 ± 4.05
20	69.07 ± 2.10	71.88 ± 1.72	77.43 ± 2.66	78.93 ± 2.11	77.43 ± 2.66	75.12 ± 2.37	77.09 ± 1.68	78.75 ± 2.45	79.26 ± 2.32
60%	86.14 ± 0.64	83.38 ± 0.64	88.48 ± 0.46	87.56 ± 0.52	86.52 ± 0.56	86.90 ± 0.46	88.32 ± 0.55	87.75 ± 0.57	87.96 ± 0.57
CS	20	88.12 ± 0.78	77.45 ± 1.80	91.73 ± 0.49	92.38 ± 0.38	90.96 ± 0.46	89.43 ± 0.67	89.43 ± 0.67	91.94 ± 0.54	92.44 ± 0.36
Physics	20	88.30 ± 1.59	86.70 ± 1.03	93.29 ± 0.80	93.49 ± 0.67	92.81 ± 1.03	91.44 ± 1.41	93.16 ± 0.64	93.56 ± 0.51	93.65 ± 0.50
Computers	20	60.66 ± 2.98	72.44 ± 2.87	79.17 ± 1.92	79.07 ± 2.34	78.38 ± 2.27	79.01 ± 2.36	78.54 ± 2.15	80.05 ± 2.92	79.58 ± 2.31
Photo	20	75.33 ± 1.91	81.58 ± 4.69	89.94 ± 1.22	90.87 ± 1.14	89.24 ± 1.42	90.17 ± 0.93	89.36 ± 1.02	90.85 ± 1.16	90.93 ± 1.40
ogbn-arxiv	54%	61.17 ± 0.20	74.08 ± 0.00	71.91 ± 0.15	71.61 ± 0.30	OOM	OOM	68.01 ± 0.35	72.04 ± 0.12	69.20 ± 0.26
and Figure F, respectively. From the results, we can find both  $\mathrm{LPSL}_{\mathrm{APPNP}}$  and  $\mathrm{LPSL}_{\mathrm{GCN}}$  are not very sensitive to  $\lambda$  and  $c$  at the chosen regions.
(b) LPSLGCN
Figure 15: The accuracy of different  $\lambda$  for LPSLAPPNP and LPSLGCN on Cora and CiteSeer datasets.
(a) LPSLAPPNP
(b) LPSLGCN
Figure 16: The accuracy of different  $c$  for LPSLAPPNP and LPSLGCN on Cora and CiteSeer datasets.