SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The proposed LPSL learns an unbiased graph structure with respect to the labeled nodes. Therefore, the learned graph structure can be applied to various GNN models to mitigate the Label Position bias. In this work, we test LPSL on two widely used GNN models, i.e., GCN [17] and APPNP [18]. For the GCN model, each layer can be represented by:
$$
\mathbf {H} ^ {l + 1} = \sigma \left(\mathbf {B} _ {\lambda} \mathbf {H} ^ {l} \mathbf {W} ^ {l}\right),
$$
where  $\mathbf{H}^0 = \mathbf{X}$ ,  $\sigma$  is the non-linear activation function,  $\mathbf{B}_{\lambda}$  is the unbiased structure with a ppermeter  $\lambda$ , and  $\mathbf{W}^l$  is the weight matrix in the  $l$ -th layer. We refer to this model as LPSL<sub>GCN</sub>. For the APPNP model, we directly use the learned  $\mathbf{B}_{\lambda}$  as the propagation matrix, and the prediction can be written as:
$$
\mathbf {Y} _ {\text {p r e d}} = \mathbf {B} _ {\lambda} f _ {\theta} (\mathbf {X}),
$$
Algorithm 1 Algorithm of LPSL
1: Input: Laplacian matrix  $\tilde{\mathbf{L}}$ , Label mask matrix  $\mathbf{T}$ , Hyperparameters  $\lambda, c, \beta, \rho$ , learning rate  $\gamma$
2: Output: Label position unbiased graph structure B
3: Initialization:  $\mathbf{B}^0 = \mathbf{I}$  and  $\mathbf{y}^0 = \mathbf{0}$
4: while Not converge do
5: for each block  $j$  do
6: for  $i = 0$  to update steps  $t$  do
7:  $\mathbf{B}_{\cdot ,j} = \mathbf{B}_{\cdot ,j} - \gamma *\frac{\partial L_{\rho}}{\partial\mathbf{B}_{\cdot,j}}$
8: end for
9:  $\mathbf{B}_{:,j} = S_{\beta /\rho}(\mathbf{B}_{:,j})$
10: end for
1:  $\mathbf{y} = \mathbf{y} + \rho (\mathbf{B}\mathbf{T}\mathbf{1}_n - c\mathbf{1}_n)$
12: end while
13: return B
where  $f_{\theta}(\cdot)$  is any machine learning model parameterized by the learnable parameters  $\theta$ . We name this model as LPSL<sub>APPNP</sub>. The parameter  $\lambda$  provides a high flexibility when applying  $\mathbf{B}_{\lambda}$  to different
GNN architectures. For decoupled GNNs such as APPNP, which only propagates once, a large  $\lambda$  is necessary to encode the global graph structure with a denser sparsity. In contrast, for coupled GNNs, such as GCN, which apply propagation multiple times, a smaller  $\lambda$  can be used to encode a more local structure with a higher sparsity. Our code is available at: https://github.com/haoyuhan1/LPSL.
In this section, we conduct comprehensive experiments to verify the effectiveness of the proposed LPSL. In particular, we try to answer the following questions:
- Q1: Can the proposed LPSL improve the performance of different GNNs? (Section 4.2)
- Q2: Can the proposed LPSL mitigate the label position bias? (Section 4.3)
- Q3: How do different hyperparameters affect the proposed LPSL? (Section 4.4)
Datasets. We conduct experiments on 8 real-world graph datasets for the semi-supervised node classification task, including three citation datasets, i.e., Cora, Citeseer, and Pubmed [27], two co-authorship datasets, i.e., Coauthor CS and Coauthor Physics, two co-purchase datasets, i.e., Amazon Computers and Amazon Photo [28], and one OGB dataset, i.e., ogbn-arxiv [29]. The details about these datasets are shown in Appendix C.
We employ the fixed data split for the ogbn-arxiv dataset, while using ten random data splits for all other datasets to ensure more reliable results [30]. Additionally, for the Cora, CiteSeer, and PubMed datasets, we experiment with various labeling rates: low labeling rates with 5, 10, and 20 labeled nodes per class, and high labeling rates with  $60\%$  labeled nodes per class. Each model is run three times for every data split, and we report the average performance along with the standard deviation.
Baselines. To the best of our knowledge, there are no previous works that aim to address the label position bias. In this work, we select three GNNs, namely, GCN [17], GAT [31], and APPNP [18], two Non-GNNs, MLP and Label Propagation [19], as baselines. Furthermore, we also include GRADE [32], a method designed to mitigate degree bias. Notably, SRGNN [33] demonstrates that if labeled nodes are gathered locally, it could lead to an issue of feature distribution shift. SRGNN aims to mitigate the feature distribution shift issue and is also included as a baseline.
Hyperparameters Setting. We follow the best hyperparameter settings in their original papers for all baselines. For the proposed  $\mathrm{LPSL_{GCN}}$ , we set the  $\lambda$  in range [1,8]. For  $\mathrm{LPSL_{APPNP}}$ , we set the  $\lambda$  in the range [8, 15]. For both methods,  $c$  is set in the range [0.5, 1.5]. We fix the learning rate 0.01, dropout 0.5 or 0.8, hidden dimension size 64, and weight decay 0.0005, except for the ogbn-arxiv dataset. Adam optimizer [34] is used in all experiments. More details about the hyperparameters setting for all methods can be found in Appendix D.
4.2 Performance Comparison on Benchmark Datasets
In this subsection, we test the learned unbiased graph structure by the proposed LPSL on both GCN and APPNP models. We then compare these results with seven baseline methods across all eight datasets. The primary results are presented in Table 1. Due to space limitations, we have included the results from other baselines in Appendix E. From these results, we can make several key observations:
- The integration of our proposed LPSL to both GCN and APPNP models consistently improves their performance on almost all datasets. This indicates that a label position unbiased graph structure can significantly aid semi-supervised node classification tasks.
- Concerning the different labeling rates for the first three datasets, our proposed LPSL shows greater performance improvement with a low labeling rate. This aligns with our preliminary study that label position bias is more pronounced when the labeling rate is low.
- SRGNN, designed to address the feature distribution shift issue, does not perform well on most datasets with random splits instead of locally distributed labels. Only when the labeling rate is very low, SRGNN can outperform GCN. Hence, the label position bias cannot be solely solved by addressing the feature distribution shift.
- The GRADE method, aimed at mitigating the degree-bias issue, also fails to improve overall performance with randomly split datasets.
Table 1: Semi-supervised node classification accuracy (%) on benchmark datasets.
Dataset	Label Rate	GCN	APPNP	GRADE	SRGNN	LPSLGCN	LPSLAPPNP
Cora	5	70.68 ± 2.17	75.86 ± 2.34	69.51 ± 6.79	70.77 ± 1.82	76.58 ± 2.37	77.24 ± 2.18
10	76.50 ± 1.42	80.29 ± 1.00	74.95 ± 2.46	75.42 ± 1.57	80.39 ± 1.17	81.59 ± 0.98
20	79.41 ± 1.30	82.34 ± 0.67	77.41 ± 1.49	78.42 ± 1.75	82.74 ± 1.01	83.24 ± 0.75
60%	88.60 ± 1.19	88.49 ± 1.28	86.84 ± 0.99	87.17 ± 0.95	88.75 ± 1.21	88.62 ± 1.69
CiteSeer	5	61.27 ± 3.85	63.92 ± 3.39	63.03 ± 3.61	64.84 ± 3.41	65.65 ± 2.47	65.70 ± 2.18
10	66.28 ± 2.14	67.57 ± 2.05	64.20 ± 3.23	67.83 ± 2.19	67.73 ± 2.57	68.76 ± 1.77
20	69.60 ± 1.67	70.85 ± 1.45	67.50 ± 1.76	69.13 ± 1.99	70.73 ± 1.32	71.25 ± 1.14
60%	76.88 ± 1.78	77.42 ± 1.47	74.00 ± 1.87	74.57 ± 1.57	77.18 ± 1.64	77.56 ± 1.44
PubMed	5	69.76 ± 6.46	72.68 ± 5.68	66.90 ± 6.49	69.38 ± 6.48	73.46 ± 4.64	73.57 ± 5.30
10	72.79 ± 3.58	75.53 ± 3.85	73.31 ± 3.75	72.69 ± 3.49	75.67 ± 4.42	76.18 ± 4.05
20	77.43 ± 2.66	78.93 ± 2.11	75.12 ± 2.37	77.09 ± 1.68	78.75 ± 2.45	79.26 ± 2.32
60%	88.48 ± 0.46	87.56 ± 0.52	86.90 ± 0.46	88.32 ± 0.55	87.75 ± 0.57	87.96 ± 0.57
CS	20	91.73 ± 0.49	92.38 ± 0.38	89.43 ± 0.67	89.43 ± 0.67	91.94 ± 0.54	92.44 ± 0.36
Physics	20	93.29 ± 0.80	93.49 ± 0.67	91.44 ± 1.41	93.16 ± 0.64	93.56 ± 0.51	93.65 ± 0.50
Computers	20	79.17 ± 1.92	79.07 ± 2.34	79.01 ± 2.36	78.54 ± 2.15	80.05 ± 2.92	79.58 ± 2.31
Photo	20	89.94 ± 1.22	90.87 ± 1.14	90.17 ± 0.93	89.36 ± 1.02	90.85 ± 1.16	90.93 ± 1.40
ogbn-arxiv	54%	71.91 ± 0.15	71.61 ± 0.30	OOM	68.01 ± 0.35	72.04 ± 0.12	69.20 ± 0.26
4.3 Evaluating Bias Mitigation Performance
In this subsection, we aim to investigate whether the proposed LPSL can mitigate the label position bias. We employ all three aforementioned bias metrics, namely label proximity score, degree, and shortest path distance, on Cora and CiteSeer datasets. We first group test nodes into different sensitive groups according to the metrics, and then use three representative group bias measurements - Weighted Demographic Parity (WDP), Weighted Standard Deviation (WSD), and Weighted Coefficient of Variation (WCV) - to quantify the bias. These are defined as follows:
$$
\mathrm {W D P} = \frac {\sum_ {i = 1} ^ {D} N _ {i} \cdot | A _ {i} - A _ {\mathrm {a v g}} |}{N _ {\mathrm {t o t a l}}}, \mathrm {W S D} = \sqrt {\frac {1}{N _ {\mathrm {t o t a l}}} \sum_ {i = 1} ^ {D} N _ {i} \cdot (A _ {i} - A _ {\mathrm {a v g}}) ^ {2}}, \mathrm {W C V} = \frac {\mathrm {W S D}}{A _ {\mathrm {a v g}}},
$$
where  $D$  is the number of groups,  $N_{i}$  is the node number of group  $i$ ,  $A_{i}$  is the accuracy of group  $i$ ,  $A_{\mathrm{avg}}$  is the weighted average accuracy of all groups, i.e., the overall accuracy, and  $N_{\mathrm{total}}$  is the total number of nodes. We choose six representative models, i.e., Label Propagation (LP), GRADE, GCN, APPNP, LPSL<sub>GCN</sub>, and LPSL<sub>APPNP</sub>, in this experiment. The results of the label proximity score, degree, and shortest path on the Cora and CiteSeer datasets are shown in Tabel 2, 3, and 4, respectively. It can be observed from the tables:
- The Label Propagation method, which solely utilizes the graph structure information, exhibits the most significant label position bias across all measurements and datasets. This evidence suggests that label position bias primarily stems from the biased graph structure, thereby validating our strategy of learning an unbiased graph structure with LPSL.
- The proposed LPSL not only enhances the classification accuracy of the backbone models, but also alleviates the bias concerning Label Proximity Score, degree, and Shortest distance.
- The GRADE method, designed to mitigate degree bias, does exhibit a lesser degree bias than GCN and APPNP. However, it still falls short when compared to the proposed LPSL. Furthermore, GRADE may inadvertently heighten the bias evaluated by other metrics. For instance, it significantly increases the label proximity score bias on the CiteSeer dataset.
Table 2: Comparison of Methods in Addressing Label Proximity Score Bias.
Dataset	Cora	CiteSeer
Method	WDP ↓	WSD ↓	WCV ↓	WDP ↓	WSD ↓	WCV ↓
LP	0.1079	0.1378	0.1941	0.2282	0.2336	0.4692
GRADE	0.0372	0.0489	0.0615	0.0376	0.0467	0.0658
GCN	0.0494	0.0618	0.0758	0.0233	0.0376	0.0524
LPSLGCN	0.0361	0.0438	0.0518	0.0229	0.0346	0.0476
APPNP	0.0497	0.0616	0.0732	0.0344	0.0426	0.0594
LPSLAPPNP	0.0390	0.0476	0.0562	0.0275	0.0349	0.0474
Table 3: Comparison of Methods in Addressing Degree Bias.
Dataset	Cora	CiteSeer
Method	WDP ↓	WSD ↓	WCV ↓	WDP ↓	WSD ↓	WCV ↓
LP	0.0893	0.1019	0.1447	0.1202	0.1367	0.2773
GRADE	0.0386	0.0471	0.0594	0.0342	0.0529	0.0744
GCN	0.0503	0.0566	0.0696	0.0466	0.0643	0.0901
LPSLGCN	0.0407	0.0468	0.0554	0.0378	0.0538	0.0742
APPNP	0.0408	0.0442	0.0527	0.0499	0.0688	0.0964
LPSLAPPNP	0.0349	0.0395	0.0467	0.0316	0.0487	0.0665
Table 4: Comparison of Methods in Addressing Shortest Path Distance Bias.
DataSet	Cora	CiteSeer
Method	WDP ↓	WSD ↓	WCV ↓	WDP ↓	WSD ↓	WCV ↓
LP	0.0562	0.0632	0.0841	0.0508	0.0735	0.109
GRADE	0.0292	0.0369	0.0459	0.0282	0.0517	0.0707
GCN	0.0237	0.0444	0.0533	0.0296	0.0553	0.0752
LPSLGCN	0.0150	0.0248	0.0289	0.0246	0.0526	0.0714
APPNP	0.0218	0.0316	0.0369	0.0321	0.0495	0.0668
LPSLAPPNP	0.0166	0.0253	0.0295	0.0265	0.0490	0.0654
In this subsection, we first explore the impact of different hyperparameters, specifically the smoothing term  $\lambda$  and the constraint  $c$ , on our model. We conducted experiments on the Cora and CiteSeer datasets using ten random data splits with 20 labels per class. The accuracy of different  $\lambda$  values for  $\mathrm{LPSL}_{\mathrm{APPNP}}$  and  $\mathrm{LPSL}_{\mathrm{GCN}}$  on the Cora and CiteSeer datasets are illustrated in Figure 4.
From the results, we note that the proposed LPSL is not highly sensitive to the  $\lambda$  within the selected regions. Moreover, for the APPNP model, the best  $\lambda$  is higher than that for the GCN model, which aligns with our discussion in Section 3 that the decoupled GNNs require a larger  $\lambda$  to encode the global graph structure. The results for hyperparameter  $c$  can be found in Appendix F with similar observations.
(a) LPSLAPPNP
(b) LPSLGCN
Figure 4: The accuracy of different  $\lambda$  for LPSLAPPNP and LPSLGCN on Cora and CiteSeer datasets.
In addition, we investigate the impact of the sparse graph structure matrix  $\mathbf{B}$  generated using  $l_{1}$  regularization, as described in Eq. 5, on the performance of different models. For this purpose, we utilize the Cora dataset and select an appropriate  $\beta$  value to produce the sparse graph structure matrix  $\mathbf{B}$  at sparsity levels of  $80\%$ ,  $90\%$ , and  $95\%$ . This means that  $80\%$ ,  $90\%$ , and  $95\%$  of the entries in the matrix  $\mathbf{B}$  are zero, respectively.
The results are presented in Table 5. We observe that the accuracy at  $80\%$  and  $90\%$  sparsity is closely aligned with that of the dense matrix. However, when sparsity reaches  $95\%$ , there is a slight drop in accuracy, mirroring the findings of PPRGo [35]. Notably, fairness appears to be relatively unaffected by the varying levels of sparsity.
Table 5: Performance comparison using different sparsity levels of  $\mathbf{B}$  on the Cora dataset.
LPSLAPPNP	LPSLGCN
Sparsity	15%(β=0)	80%	90%	95%	15%(β=0)	80%	90%	95%
ACC	83.24	83.20	82.67	81.90	82.74	82.66	82.70	81.78
WDP	0.033	0.0345	0.0323	0.0322	0.0347	0.0346	0.0347	0.0337
WSD	0.042	0.0444	0.0417	0.0411	0.0443	0.0439	0.0425	0.0433
WCV	0.0505	0.0534	0.0504	0.0503	0.0536	0.0532	0.0515	0.053
Graph Neural Networks (GNNs) serve as an effective framework for representing graph-structured data, primarily employing two operators: feature transformation and propagation. The ordering of these operators classifies most GNNs into two categories: Coupled and Decoupled GNNs. Coupled GNNs, such as GCN [17], GraphSAGE [36], and GAT [31], entwine feature transformation and propagation within each layer. In contrast, recent models like APPNP [18] represent Decoupled GNNs [30, 37, 38] that separate transformation and propagation. While Graph Neural Networks (GNNs) have achieved notable success across a range of domains [1], they often harbor various biases tied to node features and graph topology [39]. For example, GNNs may generate predictions skewed by sensitive node features [8, 6], leading to potential unfairness in diverse tasks such as recommendations [40] and loan fraud detection [41]. Numerous studies have proposed different methods to address feature bias, including adversarial training [8, 42, 43], and fairness constraints [6, 44, 45]. Structural bias is another significant concern, where low-degree nodes are more likely to be falsely predicted by GNNs [9]. Recently, there are several works aimed to mitigate the degree bias issue [10, 11, 12]. Distinct from these previous studies, our work identifies a new form of bias - label position bias, which is prevalent in GNNs. To address this, we propose a novel method, LPSL, specifically designed to alleviate the label position bias.
In this study, we shed light on a previously unexplored bias in GNNs, the label position bias, which suggests that nodes closer to labeled nodes typically yield superior performance. To quantify this bias, we introduce a new metric, the Label Proximity Score, which proves to be a more intrinsic measure. To combat this prevalent issue, we propose a novel optimization framework, LPSL, to learn an unbiased graph structure. Our extensive experimental evaluation shows that LPSL not only outperforms standard methods but also significantly alleviates the label position bias in GNNs. In our current work, we address the label position bias only from a structure learning perspective. Future research could incorporate feature information, which might lead to improved performance. Besides, we have primarily examined homophily graphs. It would be interesting to investigate how label position bias affects heterophily graphs. We hope this work will stimulate further research and development of methods aimed at enhancing label position fairness in GNNs.
[1] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 32(1):4-24, 2020.
[2] Yao Ma and Jiliang Tang. Deep learning on graphs. Cambridge University Press, 2021.
A Preliminary Study
In this section, we first present a comprehensive set of experimental results, showcasing the performance disparity across various GNNs concerning three metrics related to Label Distance Bias: Degree, Shortest Path Distance, and Label Proximity Score. Subsequently, we extend our examination of label position bias to additional GNNs, including ReNode [46], GCNII [47], JKNet [23], MAGNA [48], and NodeFormer [49]. We begin with an introduction to the first study.
A.1 The performance disparity of GNNs across three metrics
Datasets. We selected three representative datasets for our experiments: Cora, CiteSeer, and PubMed. For each of these datasets, we worked with three different labeling rates: 5 labels per class, 20 labels per class, and  $60\%$  labels per class. For the data splits consisting of 5 and 20 labels per class, we adopted a commonly used setting [50] that randomly selects 500 nodes for validation and 1000 labels for testing. When dealing with a labeling rate of  $60\%$ , we randomly selected  $20\%$  of nodes for validation and another  $20\%$  for testing.
Models. Our study also incorporates three representative models: APPNP [18], GCN [17], and Label Propagation [19]. APPNP, a decoupled GNN, directly leverages the PPR matrix for feature propagation. On the other hand, GCN, a coupled GNN, uses the original adjacency matrix for feature propagation across each layer. Label Propagation relies solely on graph structure and labeled nodes for prediction. For all the models, we select their best hyperparameters based on the search space in their original papers.
Experimental Setup. For both Degree and Shortest Path Distance metrics, we employ their actual values, [1, 2, 3, 4, 5, 6, 7], to segregate nodes into separate sensitive groups, considering only a handful of nodes possess a degree or shortest path Distance greater than seven. We delete the groups with only a few nodes. For the Label Proximity Score (LPS), we divided the test nodes evenly into seven sensitive groups based on their LPS, each group possessing a uniform range. It's important to note that we also filtered out outliers, particularly those with significantly larger LPS than the rest. Additionally, if a group contained only a few nodes, we merged it with adjacent groups.
(a) Degree
Figure 6: GCN with 20 labeled nodes per class.
Figure 5: APPNP with 20 labeled nodes per class.
(b) Shortest Path Distance
Figure 7: LP with 20 labeled nodes per class.
(b) Shortest Path Distance
Figure 8: APPNP with 5 labeled nodes per class.
(a) Degree
Figure 9: GCN with 5 labeled nodes per class.
(a) Degree
Figure 10: LP with 5 labeled nodes per class.
(a) Degree
Figure 11: APPNP with  $60\%$  labeled nodes per class.
(c) Label Proximity Score
Figure 12: GCN with  $60\%$  labeled nodes per class.
(c) Label Proximity Score
Figure 13: LP with  $60\%$  labeled nodes per class.
Our analysis of the results above leads us to the following key observations:
- Label Position Bias is a widespread phenomenon across all GNN models and datasets. Classification accuracy exhibits substantial variation between different sensitive groups, with discernible patterns.
- When contrasted with Degree and Shortest Path Distance, the proposed Label Proximity Score consistently shows a robust correlation with performance disparity across all datasets and models. This underscores its efficacy as a measure of Label Position Bias.
- The severity of Label Position Bias is more prominent when the labeling rate is low, such as with 5 or 20 labeled nodes per class. However, with a labeling rate of  $60\%$  labeled nodes per class, the bias becomes less noticeable. This is evident from the fact that the shortest path distance is either 1 or 2 for all datasets, implying that all test nodes have at least one labeled node within their two-hop neighbors.
A.2 Label Position Bias of different GNNs
We further select 4 more GNNs, i.e., ReNode [46], GCNII [47], JKNet [23], and MAGNA [48], and 1 transformer-based model, i.e., and NodeFormer [49], to further verify the Label Position Bias.
(a) Cora
Figure 14: The label position bias of different GNNs.
From the above results, we can find all the selected GNNs suffer from the label position bias issue. However, the graph transformer model, i.e., NodeFormer, demonstrates small label position bias on the CiteSeer dataset.
These findings highlight the importance of further exploring Label Position Bias and developing strategies to mitigate its impact on GNN performance.
B Understandings
Remark 3.1 The feature aggregation using the learned graph structure  $\mathbf{B}$  directly as a propagation matrix, i.e.,  $\mathbf{F} = \mathbf{BX}$ , is equivalent to applying the message passing in GNNs using the original graph if  $\mathbf{B}$  is the approximate or exact solution to the primary problem defined in Eq. (2) without constraints.
Proof. There are several recent studies [51, 52, 53] unified the message passing in different GNNs under an optimization framework. For instance, Ma et al. [52] demonstrated that the message-passing scheme of GNNs, such as GCN [17], GAT [31], PPNP and APPNP [18], can be viewed as optimizing the following graph signal denoising problem:
$$
\underset {\mathbf {F} \in \mathbb {R} ^ {n \times d}} {\arg \min } \mathcal {L} = \| \mathbf {X} - \mathbf {F} \| _ {F} ^ {2} + \lambda \operatorname {t r} \left(\mathbf {F} ^ {\top} \tilde {\mathbf {L}} \mathbf {F}\right), \tag {7}
$$
where  $\mathbf{X} \in \mathbb{R}^{n \times d}$  is the node features,  $\mathbf{F}$  is the optimal node representations after applying GNNs, and  $\lambda$  are used to control the feature smoothness. The gradient of  $\frac{\partial \mathcal{L}}{\partial \mathbf{F}}$  can be represented as:
$$
\frac {\partial \mathcal {L}}{\partial \mathbf {F}} = 2 (\mathbf {F} - \mathbf {X}) + 2 \lambda \tilde {\mathbf {L}} \mathbf {F}.
$$
Here, we provide two examples of using the Eq. (7) to derive APPNP and GCN. For APPNP, we can adopt multiple-step gradient descent to solve the Eq. (7):
$$
\mathbf {F} ^ {k} = \mathbf {F} ^ {k - 1} - \gamma \frac {\partial \mathcal {L}}{\partial \mathbf {F}} = (1 - 2 \lambda - 2 \lambda \gamma) \mathbf {F} ^ {k - 1} + 2 \lambda \gamma \tilde {\mathbf {A}} \mathbf {F} ^ {k - 1} + 2 \gamma \mathbf {X}.
$$
If we set the stepsize  $\gamma = \frac{1}{2(1 + \lambda)}$ , then we have the following update steps:
$$
\mathbf {F} ^ {k} = \frac {\lambda}{1 + \lambda} \tilde {\mathbf {A}} \mathbf {F} ^ {k - 1} + \frac {1}{1 + \lambda} \mathbf {X}
$$
which is the message passing scheme of APPNP. Then, if we propagate  $K$  layers, then
$$
\begin{array}{l} \mathbf {F} ^ {K} = \frac {\lambda}{1 + \lambda} \tilde {\mathbf {A}} \mathbf {F} ^ {K - 1} + \frac {1}{1 + \lambda} \mathbf {X} \\ = \frac {\lambda}{1 + \lambda} \tilde {\mathbf {A}} \left(\frac {\lambda}{1 + \lambda} \tilde {\mathbf {A}} \mathbf {F} ^ {K - 2} + \frac {1}{1 + \lambda} \mathbf {X}\right) + \frac {1}{1 + \lambda} \mathbf {X} \tag {8} \\ = \left(\left(\frac {\lambda}{1 + \lambda}\right) ^ {K} \tilde {\mathbf {A}} ^ {K} + \sum_ {i = 0} ^ {K - 1} \frac {1}{1 + \lambda} \left(\frac {\lambda}{1 + \lambda}\right) ^ {i} \tilde {\mathbf {A}} ^ {i}\right) \mathbf {X}. \\ \end{array}
$$
For GCN, we can use one step gradient to solve the Eq. (7):
$$
\mathbf {F} = \mathbf {X} - \left. \gamma \frac {\partial \mathcal {L}}{\partial \mathbf {F}} \right| _ {\mathbf {F} = \mathbf {X}} = (1 - 2 \gamma \lambda) \mathbf {X} + 2 \gamma \lambda \tilde {\mathbf {A}} \mathbf {X}.
$$
If we set the step size  $\gamma = \frac{1}{2\lambda}$ , then the  $\mathbf{F} = \tilde{\mathbf{A}}\mathbf{X}$ , which is the message passing of GCN.