SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- Experimental results and analysis. In Figure C1 we show the CH index scores with node embeddings at different propagation iteration  $t$ , and we can observe that, as the information propagation weight  $w$  for cross-links decreases, the CH index score increases consistently and far exceeds that in normal settings ( $w = 1$ ), which indicates that the final node embeddings present more serious polarization problems among communities. Since the information in a single community is relatively limited as shown in Figure 2, the information cocoon problem actually becomes more severe with the lack of cross-links.
A.2 The relationship between cross-links and graph connectivity
By borrowing the concept of network diffusion, we try to explore the role of cross-links in graph connectivity in this part. Specifically, we apply a classic model in network diffusion: the SI model [18], to simulate the process of information propagation. In this model, each node is randomly initialized with a status called susceptible or infected at the beginning. During the diffusion iteration process, SI assumes that each infected node could infect its susceptible neighbors with probability  $p$ , and once a node becomes infected, it stays infected until the end of network diffusion, i.e. there are no more new infected nodes in a new iteration.
In order to provide a clear and vivid illustration, we take one of the most representative social networks – Zachary's karate club<sup>7</sup> as an example. All nodes are divided into four non-overlapping
<sup>7</sup>https://en.wikipedia.org/wiki/Zachary%27s_karate_club
(a) Original graph
(b) Drop links randomly
Figure C2: The visualization of diffusion simulation with SI model. Red nodes denote infected nodes, and green nodes represent susceptible nodes. (a) The original graph of the karate club. Nodes with the same color denote a community. (b) Infected graph after dropping some random edges. (c) Infected graphs after dropping some cross-links.
(c) Drop cross-links
communities by Louvain algorithm [2] in advance, and node #0, which stands for the instructor in this club, is initialized as the only infected node at iteration 0. We further randomly remove  $80\%$  cross-links in the graph before starting the simulation. For getting a more convincing conclusion, the network diffusion process on a graph with the same number of random edges removed is also simulated for comparison.
The final visualization results are shown in Figure C2. It can be seen that although we remove some edges randomly from the whole graph, the infected node #0 still propagates its information to almost all nodes in the club (red nodes in the figure) successfully. In contrast, after removing the same number of cross-links, there appears to be an obvious information isolation phenomenon, and nearly half of the nodes remain susceptible. In other words, cross-links plays a role in bridging two different communities during network diffusion, and it would be hard for a node to send or receive messages from other communities without enough cross-links. In this way, the existence of cross-links plays a key role in preserving graph connectivity.
B Additional Experimental Settings
B.1 Base Models and Baselines
Here we introduce additional details for the base models and baselines used in our experiments.
(1) Base models
- GraphSAGE [13]: GraphSAGE is an inductive learning framework for generating node embeddings, which samples a fixed number of neighbors during aggregation to alleviate the "neighborhood explosion" issues.
- GIN [41]: GIN is a graph neural network that is theoretically as powerful as the Weisfeiler-Lehman test with injective aggregation, combination, and readout functions.
- GAT [37]: GAT deploys the attention mechanism during aggregation to capture the neighborhood information with different weights.
- PPRGo[3]: By utilizing a Personalized Page Rank matrix to approximate the propagation and aggregation steps with multi-layer graph convolution, PPRGo greatly improves the efficiency and effectiveness on large graphs.
- LightGCN [14]: LightGCN empirically finds the redundancy of feature transformation and non-linear activation functions, and greatly simplifies the model architecture with even higher performance on the recommendation tasks.
- UltraGCN [27]: Based on LightGCN, UltraGCN further theoretically simplifies the model architecture with approximating the infinite-layer information propagation and aggregation.
(2)Baselines
- FairWalk [30]: Instead of a random walk in node2vec, FairWalk chooses its next hop by considering the sensitive attributes in the neighborhood, which successfully mitigates the unfairness related to the sensitive attribute.
- CFC [4]: To ensure that the learned embeddings are not correlated with sensitive attributes, such as age or gender, CFC introduces an adversarial framework to enforce fairness constraints.
- FairAdj [20]: With learning to assign each edge with different weights, FairAdj generates a fair adjacency matrix and greatly improves the dyadic fairness with comparable utility in link prediction tasks.
- FLIP [28]: Concentrating on bursting the filter bubbles in social networks from a dyadic fairness perspective, FLIP also utilizes an adversarial learning framework to generate non-sensitive node embeddings for further link prediction.
- UGE [38]: UGE aims at learning unbiased graph embeddings from an unobserved graph, which involves no sensitive information, and further derives three kinds of variants namely UGE-w, UGE-r, and UGE-c.
B.2 Evaluation Metric
In this work, we apply Hits@50, which is widely adopted in other research [43, 45] and OGB leaderboard [15], as our main evaluation metric to measure the link prediction performance of different GNN models. The Hits@50 can be computed by:
$$
H i t s @ 5 0 = \frac {1}{N _ {t e s t}} \sum_ {i = 1} ^ {N _ {t e s t}} \mathbb {I} \left(\operatorname {r a n k} _ {i} <   5 0\right) \tag {11}
$$
where  $N_{test}$  represents the sample size of test set, and  $\mathbb{I}$  represents an indicator function. rank $_i$  denotes the similarity ranking of the  $i$ th sample.
B.3 Reproducibility
Dataset. For each dataset, we randomly sample and remove  $5\%$  of links in the original graph to construct the validation set and test set, and the remaining links are treated as the training set. Each true sample will be ranked among a set of 100000 randomly sampled negative links for evaluation<sup>8</sup>. Note that, there is no side information, such as node features or link attributes, involved during our experiments, and we assign each node on the graph with a learnable embedding vector for training.
Hyper-parameters. As a model-agnostic framework, we deploy six kinds of GNN models as backbones, including GraphSAGE [13], GIN [41], GAT [37], PPRGo [3], LightGCN [14], and UltraGCN [27]. For all these models, we set the output embedding dimension as 64. The layer of the embedding fusion module is set to 1. The learning rates for twin GNNs are both set as 0.001 after grid search. As for the hyper-parameters in Eq.(8),  $\alpha$  is set to be 0.005, and  $T$  is selected from  $\{10, 25, 50\}$  depending on the datasets and base models, and  $\beta$  is set to be 20. Augmentation ratio  $k$  is searched from  $\{0.75, 1.0, 1.25\}$  for each dataset. Both weight decay and dropout rate are set to 0.
In particular, for GraphSAGE, we adopt a mean-pooling during aggregation; for GIN, we apply a linear layer to update node features and use max-pooling during aggregation; for GAT, we use 4 attention heads in each layer; for PPRGo, we set  $\alpha$  as 0.3, the walk length as 100; for LightGCN, we set the layer number as 2 and use the final layer's output as embeddings; for UltraGCN, we set the number of negative samples as 64,  $\lambda$  as 0.8,  $\gamma$  as 3.5.
For Fairwalk, we follow the settings in the original paper and set the walk number to 20, and the window length to 80; for CFC, we set the training steps of the discriminator as 5; for FairAdj, we set  $T_{2}$  to 15 and  $\lambda$  to 10; for FLIP, we take the suggestions in the original implementation, and the settings are  $\alpha(0.1), \beta(0.2)$ ; for UGE, we deploy the weighting-based variant as our baseline given that there is no non-sensitive attribute in our settings.
B.4 Details on LastFM Dataset
Due to the heterogeneity of the recommendation datasets, it is hard to directly deploy community detection algorithms on the original networks and define the corresponding internal-links and cross-links.
Here we follow the evaluation protocol in OGB [15], which is widely used in research.
To this end, inspired by ItemCF [22, 31], we first generate an item-item graph according to the co-occurrence relationship. For example, given a pair of items  $< v_{1}, v_{2} >$ , if they both have interactions with user  $u$  at least  $O$  times, where  $O$  is a hyperparameter to control the confidence of generated item-item graph, there will form a link between  $v_{1}$  and  $v_{2}$ . Next, we can deploy our framework on the generated item-item graph for learning debiased item embeddings  $\mathbf{I} \in \mathbb{R}^{|V| \times D}$ , where  $|V|$  represents the number of items and  $D$  represents the embedding dimension. After that, item similarity matrix
$\mathbf{I}^2 \in \mathbb{R}^{|V| \times |V|}$  is calculated, which is further used for the final recommendation:
$$
\mathbf {P} = \mathbf {A} \times \mathbf {I} ^ {2} \tag {12}
$$
where  $\mathbf{P} \in \mathbb{R}^{|U| \times |V|}$  denotes the predicted confidence matrix between users and items, and  $\mathbf{A} \in \mathbb{R}^{|U| \times |V|}$  represents the adjacency matrix of the original user-item graph.
In order to prove that our implementation will not affect the performance of the original GNN models, we compare our implementation (denoted as "Ours"), where  $O$  is set to be 1, with GNNs trained on the user-item graph normally (denoted as "Original") in Table C1. The results indicate that our implementation does not sacrifice the capability of base GNN models severely to adapt our framework.
C Further Experiments and Analysis
C.1 Hyper-parameter Analysis
As the core component in our framework, supervision augmentation plays a key role in mitigating the bias between internal-links and cross-links. To explore its impact in a finer granularity, we vary the augmentation ratio  $k$  and see how the performance of our method changes. Specifically, we take LightGCN as the base model and investigate the performance on internal-links (Internal.), cross-links (Cross.), and the whole link set (Overall) by varying  $k$  in  $\{0, 0.25, 0.5, 0.75, 1, 1.25\}$ . Without losing generality,
(a) Epinions
Figure C3: The impact of augmentation size
(b) DBLP
here we take Jaccard-based supervision augmentation. As shown in Figure C3, the performance of the two kinds of links increasingly improves as  $k$  grows, accompanied by a steady decrease in the difference between them. This is expected because we introduce a large amount of augmented cross-links signals to mitigate the bias. And when  $k$  reaches 1, which means  $|\mathcal{E}_{in}| \approx |\mathcal{E}_{cr}|$ , the framework gradually converges to a stable status. Empirically, a setting  $k = 1$  would be a near-optimal option.
C.2 Alternative Community Detection Algorithm
In this part, we aim to conduct an ablation study on different community detection methods to prove the usefulness of our proposed framework. Since we emphasize the bias from a topological perspective, we prefer to use the community detection algorithm based on graph structure. Specifically, to illustrate the universality of our framework, we conduct experiments based on the METIS [17] algorithm as an ablation study, and the results on three GNNs are shown in Table C2. Specifically, the number of communities is set to be 50 in advance for METIS, and all other hyper-parameters are set to be the same as that in Louvain-based experiments, which can be found in Section B.3. All results are based on Jaccard-based augmentation.
The results indicate that, although we change the community detection algorithm, our framework still successfully mitigates the bias between internal-links and cross-links, and achieves competitive results compared with the Louvain-based results, which verifies our work's compatibility.
Table C2: Ablation study with METIS community detection algorithm on two real-world datasets (Hits@50)
Epinions	DBLP
Internal.↑	Cross.†	Overall↑	Bias↓	Internal.↑	Cross.↑	Overall↑	Bias↓
SAGE	Orig.	36.97	19.48	30.75	17.49	69.80	19.00	54.28	50.80
Debias	39.06	28.93	35.89	10.13	78.67	34.65	65.22	44.02
GAT	Orig.	38.33	34.77	37.15	3.56	68.62	28.15	56.26	40.47
Debias	39.96	36.88	38.86	3.08	75.94	42.77	65.81	33.17
UltraGCN	Orig.	27.27	11.62	20.77	15.65	97.34	70.57	89.16	26.77
Debias	46.92	38.18	44.04	8.74	97.47	73.67	90.20	23.80
C.3 Supervision Augmentation Analysis
In Section 3.3, we design two kinds of data augmentation methods for generating pseudo cross-links supervision signals. Intuitively, if the pseudo supervision signals have a high confidence level, they can provide significant benefits to our framework. To this end, we aim to verify our hypothesis and analyze the impact of different supervision augmentation methods on our framework.
We first statistic the average hop distance between the node pairs generated with different supervision augmentation methods. As shown in Table C3, since we only choose node pairs with the most common neighbors in Jaccard-based augmentation, the hop distance is fixed to 2. When we use random walk-based augmentation, the average distance increases consistently on two datasets, which verifies its effectiveness in covering nodes that are not located in the boundary of communities.
Table C4 further presents the performance of our framework with random walk-based augmentation, which is literally described in Section 3.3. Specifically, for providing fair comparison, all hyper-parameters in random walk-based experiments, including augmentation ratio  $k$  and others are set to be the same as that in Jaccard-based experiments. Compared to Table 2, it can be shown that the random walk-based framework
Table C3: The average hop distance between node pairs generated by different supervision augmentation methods
Epinions	DBLP
Jaccard based	2.00	2.00
Random walk based	2.69	3.14
Table C4: Link prediction performance (Hits@50) of internal-links, cross-links, and the whole link set of our methods with random walk based augmentation and corresponding base models on two real-world datasets. The results are reported in percentage (\%). We **bold** the results when our framework improves the base GNN model.
Epinions	DBLP
Internal↑	Cross↑	Overall↑	Bias↓	Internal↑	Cross↑	Overall↑	Bias↓
SAGE	Orig.	31.68	28.91	30.69	2.77	69.27	14.62	56.41	54.65
Fair.	31.72	29.17	31.28	2.55	80.28	28.63	68.12	51.65
GIN	Orig.	33.49	30.97	32.59	2.52	56.66	16.86	47.29	39.80
Fair.	38.35	36.89	37.12	1.46	68.12	32.07	59.64	36.05
GAT	Orig.	39.30	34.90	37.73	4.40	66.25	22.47	55.94	43.78
Fair.	40.02	36.29	37.98	3.73	75.03	32.18	64.94	42.85
PPRGo	Orig.	42.86	28.75	37.83	14.11	85.71	41.14	75.28	44.58
Fair.	45.36	40.12	43.49	5.24	90.48	49.51	80.47	42.08
LightGCN	Orig.	46.43	37.11	43.11	9.32	85.95	47.41	76.88	38.54
Fair.	48.15	40.45	45.41	7.65	92.16	57.55	84.01	34.61
UltratGAON	Orig.	30.62	5.81	21.78	24.81	95.74	63.82	88.22	31.92
Fair.	52.16	51.99	52.10	0.17	96.47	66.25	89.35	30.22
shows less improvement on cross-links, which results in worse debias results. This observation can be explained by the hop distance in Table C3, which implies that the random walk-based augmentation may have lower confidence due to the longer topological distance between node pairs. However, compared with the base GNNs, the random walk-based framework can still consistently reduce the bias between internal-links and cross-links with improved overall performance.