Paper: A Topological Perspective on Demystifying GNN-Based Link Prediction Performance (ICLR-2024)

Reviewer comment (Theory):
The theoretical analysis of the relationship between TC and link prediction performance could be further strengthened with additional mathematical proofs or rigorous analysis. Providing a more solid theoretical foundation would increase the credibility and impact of the proposed methods.

Author response:
**=====Question 1=====**

Thank you for the question. As listed in Eq. (2), we calculate ATC in the following steps:
- Firstly, we initialize node embeddings by sampling from a d-dimensional multivariate normal distribution with zero mean
 
- Secondly, we perform message-passing K times, summarize the adjacency matrices from the power of 1 to K, and obtain the propagated embeddings for each node.

- Thirdly, we compute each node's embedding similarity to each neighbor and average them to calculate the ATC.

**Our paper has justified the theoretical connection between ATC and TC by proving the linear relationship between TC and the mean value of ATC in Theorem 1 in Section 3.4**

**=====================**

Reviewer comment (Evaluation):
The paper could provide a more in-depth analysis of the cold-start problem and its relationship with the proposed metrics, as well as discuss potential strategies for addressing this issue. This would help to better understand the practical implications of the proposed methods and their potential to improve link prediction performance in real-world scenarios.

Author response:
**=====Question 3=====**

From the above summary, we can see that:

(1) conventional link prediction papers work on cold-start problems at the link level and define the cold-start link as the newly joined one with little or no topological information available.

(2) some node-level works such as recommender systems define the cold-start nodes/users as the newly joined ones with sparse interactions.


Based on the above literature review, we answer the question 3 as follows:

**Following the above second paradigm, we study the cold-start link prediction at the node level since our paper targets demystifying the varying link prediction performance**. Therefore, we follow some conventional literature [1-3] in that paradigm and generally deem the nodes with few degrees are cold-start ones. Concretely, in Figure 4(b)-(e), we change the degree threshold from 1 to 10, divide nodes into two groups at each degree threshold, and further visualize the average performance for each group. We can see that nodes in the low-degree groups generally have higher performance than nodes in the high-degree groups.

The above observation has two promising insights compared with conventional literature:

**(1) Many existing recommendation-based papers define cold-start users/nodes as the ones with few/little interactions/topological signals. However, our paper empirically justifies such a group of nodes even exhibiting higher performance in LP**

We justify the above observation by relating to real-world scenarios where users with high degrees usually tend to possess diverse interests (nodes with higher degrees may tend to belong to diverse communities) and therefore, using the equal capacity of embedding cannot equally characterize all of their interests [4].

**(2) Many existing node classification papers [5-7] find nodes with low degrees have lower performance. However, our work sheds new insights on the link prediction where nodes with lower degrees actually possess higher performance.**

We justify the above observation by relating to the inherent difference between the mechanism of node classification and the mechanism of link prediction. For node classification, high-degree nodes are more likely to obtain the supervised signals from labeled nodes in the same class [8]. For link prediction, the ground-truth class for each node is actually its testing neighbors and hence there is no implicit biasing assumption that high-degree nodes would obtain more supervised signals. 


We include the above systematical literature review with a discussion about our observation on the cold-start problem in the Appendix B.

**Note that the above analysis makes another significant contribution by summarizing the cold-start definition and sheds new insights on future works addressing cold-start/sparsity issues in link prediction.**


[1] Dong, Manqing, et al. "Mamo: Memory-augmented meta-optimization for cold-start recommendation." Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 2020.

[2] Li, Yunqi, et al. "User-oriented fairness in recommendation." Proceedings of the Web Conference 2021. 2021.

[3] Hao, Bowen, et al. "Pre-training graph neural networks for cold-start users and items representation." Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 2021.

[4] Zhaok, Xiangyu, et al. "Autoemb: Automated embedding dimensionality search in streaming recommendations." 2021 IEEE International Conference on Data Mining (ICDM). IEEE, 2021.

[5] Tang, Xianfeng, et al. "Investigating and mitigating degree-related biases in graph convoltuional networks." Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020.

[6] Liu, Zemin, Trung-Kien Nguyen, and Yuan Fang. "Tail-gnn: Tail-node graph neural networks." Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021.

[7] Wang, Ruijia, et al. "Uncovering the Structural Fairness in Graph Contrastive Learning." Advances in Neural Information Processing Systems 35 (2022): 32465-32473.

[8] Chen, Deli, et al. "Topology-imbalance learning for semi-supervised node classification." Advances in Neural Information Processing Systems 34 (2021): 29885-29897.

**=====Question 3=====**

Reviewer comment (Theory):
Could you please provide more details on the implementation of the Approximated Topological Concentration (ATC) and its theoretical justification for approximating TC? This would help to better understand the rationale behind the proposed method and its potential advantages over other approaches.

Author response:
**=====Question 1=====**

Thank you for the question. As listed in Eq. (2), we calculate ATC in the following steps:
- Firstly, we initialize node embeddings by sampling from a d-dimensional multivariate normal distribution with zero mean
 
- Secondly, we perform message-passing K times, summarize the adjacency matrices from the power of 1 to K, and obtain the propagated embeddings for each node.

- Thirdly, we compute each node's embedding similarity to each neighbor and average them to calculate the ATC.

**Our paper has justified the theoretical connection between ATC and TC by proving the linear relationship between TC and the mean value of ATC in Theorem 1 in Section 3.4**

**=====================**

Reviewer comment (Experiments):
How do the proposed methods compare to other state-of-the-art link prediction techniques in terms of performance and computational efficiency?

Author response:
We really thank the reviewer for asking about the effectiveness and efficiency of our proposed method. Our proposed strategy is a model-agnostic strategy that can be applied to existing LP models. Therefore, we compare the link prediction performance of many existing baselines (including state-of-the-art ones) with their augmented versions by our method. We visualize their performance as follows:

| Baseline | Cora Hits@100  | Citeseer Hits@100 | Pubmed Hits@100 | Collab Hits@50 | Citation2 MRR | Reptile Hits@100 | Vole Hits@100 |
|----------|----------------|-------------------|-----------------|----------------|---------------|------------------|---------------|
| GCN      | 70.63±0.67     | 65.96±2.12        | 69.35±1.02      | 49.52±0.52     | 84.42±0.05    | 65.52±2.73       | 73.84±0.98    |
| GCN$_{rw}$    | 75.98±1.28     | 74.40±1.13        | 68.87±0.99      | 52.85±0.14     | 83.54±0.30    | 70.79±2.00       | 74.50±0.84    |
| SAGE     | 74.27±2.08     | 61.57±3.28        | 66.25±1.08      | 50.01±0.50     | 80.44±0.10    | 72.59±3.19       | 80.55±1.59    |
| SAGE$_{rw}$   | 74.62±2.30     | 69.89±1.66        | 66.77±0.69      | 52.96±0.37     | 80.61±0.10    | 74.35±3.20       | 81.27±0.96    |
| NCN      | 87.73±1.41     | 90.93±0.83        | 76.20±1.55      | 54.43±0.17     | 88.64±0.14    | 68.37±3.57       | 66.10±1.13    |
| NCN$_{rw}$    | 87.95±1.30     | 91.86±0.82        | 76.51±1.41      | 54.16±0.24     | OOM             | 73.81±4.71       | 67.32±0.81    |
| BUDDY    | 84.82±1.93     | 91.44±1.14        | 74.46±3.32      | 55.73±0.16     | More than two days    | 74.69±7.55                | 83.37±1.05             
| BUDDY$_{rw}$    | 85.06±1.81     | 91.86±1.00        | 75.74±2.03      | 56.17±0.42     | More than two days    | 77.20±4.10                | 84.64±1.29             |

We can see the performance after equipping our reweighting strategy increases for most cases. 



Moreover, we compare the efficiency (in running time (s)) of the baselines before/after equipping our strategies as follows:


| Baseline | Cora    | Citeseer | Pubmed | Collab | Reptile | Vole   |
|----------|---------|----------|--------|--------|---------|--------|
| GCN      | 19.5    | 15.5     | 158.4  | 2906   | 18.3    | 53.8   |
| GCN$_{rw}$    | 21.1    | 17.2     | 158.5  | 2915   | 17.0    | 53.0   |
| SAGE     | 22.3    | 17.0     | 189.8  | 2970   | 20.9    | 61.2   |
| SAGE$_{rw}$  | 23.8    | 20.5     | 192.4  | 2982   | 21.7    | 61.9   |
| BUDDY    | 3.41    | 4.51     | 15.51  | 906.18 | 2.50    | 4.99   |
| BUDDY$_{rw}$  | 3.98    | 4.92     | 14.56  | 907.52 | 2.62    | 5.06   |

We can find that equipping our strategy would only marginally increase the running time. This is because we only periodically update the edge weight and when calculating the weight using the node embeddings, we directly use the cached one obtained from the message-passing during the training stage. 

**=====================**

Reviewer comment (Writing):
What does the "cold-start" nodes mean? Please clarify the definition and implications of cold-start nodes in the context of link prediction. This would help to better understand the relevance and importance of addressing the cold-start problem in the proposed methods.

Author response:
**=====Question 3=====**

We really appreciate the reviewer's observation about the definition of cold-start nodes in our paper. Following the suggestion, we conduct a systematic literature review about the definition of cold-start nodes in the context of link prediction. We first present the summary of the definition of cold-start nodes in existing literature as follows:

| Literature | Definition  | Solution | 
|----------|----------------|-------------------|
|  [1]    |  Little or no topological information (isolated user)    |   Augment the cold-start nodes by auxiliary information (user profile/rich text information)      |
|  [2]   |   New node, new user   |   Augment the link prediction for new users using their social features      |
|  [3]   |   Users with no interactions | Augment the link prediction for new users using their interactions in another platform based on the bidirectional relation.
|  [4]   |   Vertices without edges | Consider the node community information in metric learning|
|  [5]   |   No network information is observed | Leverage user group membership information to bootstrap probabilistic graph and then refine the graph probability by considering the transitivity of the contract relationship.|
|  [6]   |   Pseudo cold start link prediction (predicting the structure of a social network when only a small subgraph is observed) | Augment the link prediction by features from multiple auxiliary networks with feature selection and regularization|
|  [7]/[8]   |   Low activity users/Disadvantaged users | Reranking to reduce the performance gap between low-activity users and high-activity users|
|  [9]   |   Fresh users with limited interactions | Meta-optimization by learning to learn the user preference based on his/her only a few past interactions|
|  [10]  |  When no time is given, cold-start items are defined based on items with fewer interactions | Learning the local parameters for each user in meta-optimization|
|  [11]  |  Users with sparse interactions | Training GNNs to adapt to cold-start scenarios by mimicking the cold-start scenario for warm users |
| [12]  | Users with sparse interactions | Use knowledge graph embedding task to assist with recommendation task |


From the above summary, we can see that:

 **(1) conventional link prediction papers work on cold-start problems at the link level and define the cold-start link as the newly joined one with little or no topological information available.** 

**(2) some node-level works such as recommender systems define the cold-start nodes/users as the newly joined ones with sparse interactions.**

[1] Wang, Zhiqiang, et al. "An approach to cold-start link prediction: Establishing connections between non-topological and topological information." IEEE Transactions on Knowledge and Data Engineering 28.11 (2016): 2857-2870.

[2] Han, Xiao, et al. "Link prediction for new users in social networks." 2015 IEEE International Conference on Communications (ICC). IEEE, 2015.

[3] Yan, Ming, et al. "Friend transfer: Cold-start friend recommendation with cross-platform transfer learning of social knowledge." 2013 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2013.

[4] Xu, Linchuan, et al. "On learning mixed community-specific similarity metrics for cold-start link prediction." Proceedings of the 26th International Conference on World Wide Web Companion. 2017.

[5] Leroy, Vincent, B. Barla Cambazoglu, and Francesco Bonchi. "Cold start link prediction." Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. 2010.

[6] Ge, Liang, and Aidong Zhang. "Pseudo cold start link prediction with multiple sources in social networks." Proceedings of the 2012 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics, 2012.

[7] Li, Yunqi, et al. "User-oriented fairness in recommendation." Proceedings of the Web Conference 2021. 2021.

[8] Rahmani, Hossein A., et al. "Experiments on generalizability of user-oriented fairness in recommender systems." Proceedings of the 
45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2022.

[9] Wei, Tianxin, and Jingrui He. "Comprehensive fair meta-learned recommender system." Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2022.

[10] Dong, Manqing, et al. "Mamo: Memory-augmented meta-optimization for cold-start recommendation." Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 2020.

[11] Hao, Bowen, et al. "Pre-training graph neural networks for cold-start users and items representation." Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 2021.

[12] Wang, Hongwei, et al. "Multi-task feature learning for knowledge graph enhanced recommendation." The world wide web conference. 2019.

**=====Question 3=====**

Reviewer comment (Experiments):
No comparison with state-of-the-art subgraph-based methods (e.g. subgraph sketching).

Author response:
Thanks for sharing another SOTA baseline Subgraph sketch: ELPH and BUDDY, we have run their code, and here are the updated results for Table 1:

| Baseline | Cora Hits@100  | Citeseer Hits@100 | Pubmed Hits@100 | Collab Hits@50 | Citation2 MRR | Reptile Hits@100 | Vole Hits@100 |
|----------|----------------|-------------------|-----------------|----------------|---------------|------------------|---------------|
| GCN      | 70.63±0.67     | 65.96±2.12        | 69.35±1.02      | 49.52±0.52     | 84.42±0.05    | 65.52±2.73       | 73.84±0.98    |
| GCN$_{rw}$    | 75.98±1.28     | 74.40±1.13        | 68.87±0.99      | 52.85±0.14     | 83.54±0.30    | 70.79±2.00       | 74.50±0.84    |
| SAGE     | 74.27±2.08     | 61.57±3.28        | 66.25±1.08      | 50.01±0.50     | 80.44±0.10    | 72.59±3.19       | 80.55±1.59    |
| SAGE$_{rw}$   | 74.62±2.30     | 69.89±1.66        | 66.77±0.69      | 52.96±0.37     | 80.61±0.10    | 74.35±3.20       | 81.27±0.96    |
| NCN      | 87.73±1.41     | 90.93±0.83        | 76.20±1.55      | 54.43±0.17     | 88.64±0.14    | 68.37±3.57       | 66.10±1.13    |
| NCN$_{rw}$    | 87.95±1.30     | 91.86±0.82        | 76.51±1.41      | 54.16±0.24     | OOM             | 73.81±4.71       | 67.32±0.81    |
| BUDDY    | 84.82±1.93     | 91.44±1.14        | 74.46±3.32      | 55.73±0.16     | More than two days    | 74.69±7.55                | 83.37±1.05             
| BUDDY$_{rw}$    | 85.06±1.81     | 91.86±1.00        | 75.74±2.03      | 56.17±0.42     | More than two days    | 77.20±4.10                | 84.64±1.29             |

We download the most latest version of the BUDDY code and also modify it based on our reweighting strategy. We can see that equipping BUDDY with the reweighting could still enhance its link prediction performance. As running BUDDY on Citation2 cannot fit on our machine with 128 G RAM, we are currently working on getting another machine to run it. We will update the results once we have on Citation2.

As you can see, equipping our strategy achieves higher performance than SOTA LP methods such as BUDDY.

Reviewer comment (Experiments):
Discussion of the time complexity is not so good. The adjacent matrix are based on each layer's embedding, so the time consumption will be larger. Experiments on this are needed.

Author response:
**=====Question 1=====**

We sincerely thank the reviewer for the sharp observation of the issue in our time complexity analysis. We first clarify this complexity analysis in two cases. The first one is the time complexity of calculating approximated topological concentration. The second one is the time complexity of applying TC-inspired reweight to the message-passing of GNNs. We discuss them in the following respectively:

For the time complexity of calculating approximated topological concentration, we:
- Firstly, we initialized each node embedding sampled from d-dimensional standard multi-normal Gaussian distribution, taking $O(|V|d)$ time
- Secondly, we perform message-passing K times, taking $O(K(|E| + |V|)d)$ time if using sparse operator [1]
- Thirdly, we summerize the propagated embeddings from $k = 1$ to $k = K$, taking $O(k|V|d)$.
- Lastly, we calculate the ATC by calculating the embedding similarity of each node to each of its neighbors, taking $O(|V|d)$.

It takes $O(K(|E| + |V|)d)$ time, which is far more efficient than the original TC computation. And we further verify it empirically in Figure 6.


For the time complexity of the TC-based edge reweighting method, we have an additional step to periodically update the edge weight based on the node embeddings obtained during the training process. However, the node embedding used for updating the edge weight can be directly borrowed from the most recent epoch as long as we cache it. Then, the only additional computation is the group-based softmax-nomralization. However, as shown in Algorithm 1 in Appendix G, we only periodically updated the edge weight; hence, the extra computational is pretty affordable.

**=====================**

Reviewer comment (Experiments):
Can you measure the time complexity through experiments? I guess the time consumption will not be so close to the original method.

Author response:
We really thank the reviewer for taking care of the efficiency of our method. We have addressed this partially from the analysis point of view in our reply to question 1 above. Here we further empirically verify the efficiency by conducting running time comparison (s) among multiple baselines as below:

| Baseline | Cora    | Citeseer | Pubmed | Collab | Reptile | Vole   |
|----------|---------|----------|--------|--------|---------|--------|
| GCN      | 19.5    | 15.5     | 158.4  | 2906   | 18.3    | 53.8   |
| GCN$_{rw}$    | 21.1    | 17.2     | 158.5  | 2915   | 17.0    | 53.0   |
| SAGE     | 22.3    | 17.0     | 189.8  | 2970   | 20.9    | 61.2   |
| SAGE$_{rw}$  | 23.8    | 20.5     | 192.4  | 2982   | 21.7    | 61.9   |
| BUDDY    | 3.41    | 4.51     | 15.51  | 906.18 | 2.50    | 4.99   |
| BUDDY$_{rw}$  | 3.98    | 4.92     | 14.56  | 907.52 | 2.62    | 5.06   |

We can find that equipping our reweighting strategy only induces marginal time complexity compared with its corresponding baseline. This is because, as we justified in the response above, 

(1) we only update the weight for each existing edge, which is linear to the size of the graph

(2) we only periodically update rather than every epoch.

(3) we directly use the cached node embeddings rather than computing from scratch.

Reviewer comment (Experiments):
NCN already explicitly accounts for this common neighbor signal in its decoder, the performance gains from our strategy are relatively modest. So why don't you choose another subgraph-based method because I'm curious what's your strategy's effect on subgraph-based methods that don't explicitly use common neighbor signals.

Author response:
**=====Question2=====**

Thanks for asking about the effect of our method on subgraph-based methods. Here we choose the SOTA subgraph-based one BUDDY and augment it with our proposed reweighting strategy in Appendix H. We present the updated results in Table 1:

| Baseline | Cora Hits@100  | Citeseer Hits@100 | Pubmed Hits@100 | Collab Hits@50 | Citation2 MRR | Reptile Hits@100 | Vole Hits@100 |
|----------|----------------|-------------------|-----------------|----------------|---------------|------------------|---------------|
| GCN      | 70.63±0.67     | 65.96±2.12        | 69.35±1.02      | 49.52±0.52     | 84.42±0.05    | 65.52±2.73       | 73.84±0.98    |
| GCN$_{rw}$    | 75.98±1.28     | 74.40±1.13        | 68.87±0.99      | 52.85±0.14     | 83.54±0.30    | 70.79±2.00       | 74.50±0.84    |
| SAGE     | 74.27±2.08     | 61.57±3.28        | 66.25±1.08      | 50.01±0.50     | 80.44±0.10    | 72.59±3.19       | 80.55±1.59    |
| SAGE$_{rw}$   | 74.62±2.30     | 69.89±1.66        | 66.77±0.69      | 52.96±0.37     | 80.61±0.10    | 74.35±3.20       | 81.27±0.96    |
| NCN      | 87.73±1.41     | 90.93±0.83        | 76.20±1.55      | 54.43±0.17     | 88.64±0.14    | 68.37±3.57       | 66.10±1.13    |
| NCN$_{rw}$    | 87.95±1.30     | 91.86±0.82        | 76.51±1.41      | 54.16±0.24     | OOM            | 73.81±4.71       | 67.32±0.81    |
| BUDDY    | 84.82±1.93     | 91.44±1.14        | 74.46±3.32      | 55.73±0.16     | More than two days    | 74.69±7.55                | 83.37±1.05             
| BUDDY$_{rw}$    | 85.06±1.81     | 91.86±1.00        | 75.74±2.03      | 56.17±0.42     | More than two days    | 77.20±4.10                | 84.64±1.29             |

We download the most latest version of the BUDDY code and also modify it based on our reweighting strategy. We can see that equipping BUDDY with the reweighting could still enhance its link prediction performance. As running BUDDY on Citation2 cannot fit on our machine with 128 G RAM, we are currently working on getting another machine to run it. We will update the results once we have them on Citation2. But based on the results on most of the datasets, we can still see the positive impact of our reweighting strategy on most of the datasets.

**=====================**

Reviewer comment (Evaluation):
How to get the performances of link prediction of nodes with different TC? I mean one link has two nodes. For example for the results for nodes' TC in \[0,0.2), does every link includes two nodes in that domain or just one node is enough?

Author response:
**=====Question 3=====**

Thanks for the question. We measure the link prediction performance of each node following existing works in recommender systems [1]. The high-level idea is for each node, we compute its pairwise link prediction score with all other nodes and rank them. Then we check the overlap of the top-K of the ranked other nodes with the top-K of the ground-truth neighbors of this node. In this way, we define Recall/Precision/F1/NDCG/MRR/Hits@K for each node and group this performance based on nodes in different TC groups. **More details can be seen in Appendix B.**

[1] He, Xiangnan, et al. "Lightgcn: Simplifying and powering graph convolution network for recommendation." Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. 2020.

As another benefit, evaluating in this way avoids any sampling bias. Previously in evaluating each link prediction performance, we randomly sampled negative node pairs and compared their link prediction score with the ground one, which might suffer from bias by only sampling a tiny portion from all negative pairs, the number of which is quadratic to the total number of nodes in the network. However, our node-centric evaluation metric avoids this issue since for each node, we rank its ground-truth neighbors against all other nodes. We provide this discussion in Appendix B.

**=====================**

Reviewer comment (Experiments):
Have you tried to make TC^{Tr} closer to TC^{VAL}, if wanting to make model more generalizing to prediciting test links. Because we may assume validation set has the same distribution of test set.

Author response:
**=====Question 4=====**

We thank the reviewer for the excellent suggestion and experiment by reweighting the neighbors based on their connection level to the training neighbors or the validation neighbors. We indeed find that the performance improves more if we reweight based on validation neighbors closer to the training neighbors. We put the results as follows:

| Performance | GCN No | GCN Train | GCN Valid | SAGE No | SAGE Train | SAGE Valid |
|-------------|--------|-----------|-----------|---------|------------|------------|
| Hits@5      | 18.94±1.20 | 19.48±0.75 | 22.36±0.32 | 11.25±1.24 | 20.52±2.35 | 24.34±0.07 |
| Hits@10     | 31.24±3.44 | 32.69±1.00 | 35.15±2.42 | 26.41±1.88 | 31.23±3.52 | 37.15±2.44 |
| Hits@50     | 50.12±0.22 | 52.77±1.00 | 53.24±0.22 | 49.68±0.25 | 51.87±0.10 | 52.69±0.26 |
| Hits@100    | 54.44±0.49 | 56.89±0.17 | 57.28±0.10 | 54.69±0.18 | 56.59±0.19 | 57.27±0.25 |

In the above Table, we can see that reweighting by training edges and validation edges can both lead to higher performance. Even more, reweighting by validation edges leads to more considerable performance gain, which aligns with our expectation that the distribution of validation edges is closer to testing edges due to the shorter time interval.

**=====================**

Reviewer comment (Writing):
The writing is unclear and hard to follow. The notations are confusing. The paper uses similar symbols for some concepts, such as ${TC}^{TR}$, ${TC}^{Tr}$, ${Tc}^{Tr}$, which are hard to distinguish. The paper also uses uncommon abbreviations for training and testing sets, such as Tr and Te.

Author response:
We really appreciate the reviewer for pointing out the potential notation confusion. Since this work evaluates the link prediction performance on training/validation/testing links and also studies the distribution shift issue in node-link prediction, we need to separately denote them as Training (TR), Validation (Val), Testing (Te). These notations have also been frequently used in other works such as the following ones:

[5] Jin, Wei, et al. "Empowering Graph Representation Learning with Test-Time Graph Transformation." The Eleventh International Conference on Learning Representations. 2022.

[6] Bai Y, Chen M, Zhou P, et al. How important is the train-validation split in meta-learning?[C]//International Conference on Machine Learning. PMLR, 2021: 543-553.

Reviewer comment (Theory):
The motivation of probing the node characteristic for LP is questionable. The paper does not explain why this is a meaningful problem, given that [1] has proven that LP cannot be reduced to two node problems.

Author response:
Moreover, we appreciate the reviewer sharing this excellent literature [1]. We have thoroughly gone through it. The claimed “two-node problem” here refers to only aggregating two-node representations directly learned by basic GNNs like GAE. **We acknowledge that this would inform the problem of missing topological-relational information. However, we want to emphasize that, at any place in our work, we did not claim that we characterize the link prediction performance of each node only by itself and some other nodes.** Conversely, our proposed node-level topological metric, Topological Concentration (TC), indicates the intersection between the subgraph around that node and the subgraph around the neighbors of that node, which explicitly considers the topological-relational information.

Furthermore, we want to highlight some connections between [1] and ours, which further justifies the motivation for using TC in characterizing node link prediction performance. Both works study  LP tasks and investigate the underlying reason for successfully performing LP. However, [1] focuses on the performance at each node pair (link level), while ours focuses on the performance at the node level. Therefore, the labeling trick proposed in [1] captures the topological characteristics of each node pair, while our TC essentially captures the topological attributes of each node.  Even more, a level of subgraph overlap between the node and its neighbors would equal the high TC of that node.

Reviewer comment (Theory):
The definition of Topological Concentration is complicated and the rationality is not obvious. The paper uses a complex formula to find the intersection of subgraphs at different hops, but does not justify its choice, such as the exponential decaying coefficients. Why don't you define $\mathcal{H}_i^k$ as $k$-hop neighbors, which is more clear and straightforward?

Author response:
Sorry for the confusion. Previous studies [1]-[2] have found that the structural features of the enclosed subgraph of the node pair are essential in determining the link prediction between that node pair. Following this same logic, we hypothesize that each node's link prediction heavily depends on the enclosed subgraph between the node and its neighbors. This exactly corresponds to the formula we propose in Eq. (1). For each neighbor of a specific node $v_j \sim \mathcal{N}_i^t$, we consider the interaction between the computational tree $\mathcal{S}_i^K$ and $\mathcal{S}_j^K$. We discussed the intuition and rationality of TC at the beginning of Section 3.2.


[1] Zhang, Muhan, and Yixin Chen. "Link prediction based on graph neural networks." Advances in neural information processing systems 31 (2018).

[2] Chamberlain, Benjamin Paul, et al. "Graph neural networks for link prediction with subgraph sketching." arXiv preprint arXiv:2209.15486 (2022).

Reviewer comment (Evaluation):
Obs.2 and Obs. 3 in section 3.3 seem irrelevant for the proposed model and discussion. The paper does not explain how these observations inform the design or evaluation of the edge reweighting strategy.

Author response:
Sorry for causing this confusion. Instead of contributing by solely proposing an edge reweighing strategy to augment GNN and enhance its LP performance, our work is to delve deeper into the relation between the node local topology and its link prediction performance and demonstrate the importance of studying this issue. To quantify such a relation, we propose topological concentration and verify its higher correlation to node LP performance in Obs 1. section 3.3. Then, we demonstrate other real-world applications of TC by Obs. 2 (better identifying low-performing nodes compared to a degree in cold-start issues) and Obs. 3 (discovering a new topological distribution shift issue by TC). Therefore, as our main story focuses on TC and its real-world application but not on proposing a specific method for enhancing LP, Obs.2 and Obs.3 are still crucial in our story.

Reviewer comment (Experiments):
The experiment in this paper is weak. You only show the relative gain of GCN/SAGE/NCN with reweighting, but you do not compare with the SOTA LP methods like BUDDY [4]. Your result is not competitive, as you only achieve 54% Hits@50 on Collab. Why do you not include experiments on other OGB datasets like ogbl-ddi and ogbl-ppa?

Author response:
Thanks for sharing another SOTA baseline ELPH and BUDDY, we have run their code, and here are the updated results for Table 1:

| Baseline | Cora Hits@100  | Citeseer Hits@100 | Pubmed Hits@100 | Collab Hits@50 | Citation2 MRR | Reptile Hits@100 | Vole Hits@100 |
|----------|----------------|-------------------|-----------------|----------------|---------------|------------------|---------------|
| GCN      | 70.63±0.67     | 65.96±2.12        | 69.35±1.02      | 49.52±0.52     | 84.42±0.05    | 65.52±2.73       | 73.84±0.98    |
| GCN$_{rw}$    | 75.98±1.28     | 74.40±1.13        | 68.87±0.99      | 52.85±0.14     | 83.54±0.30    | 70.79±2.00       | 74.50±0.84    |
| SAGE     | 74.27±2.08     | 61.57±3.28        | 66.25±1.08      | 50.01±0.50     | 80.44±0.10    | 72.59±3.19       | 80.55±1.59    |
| SAGE$_{rw}$   | 74.62±2.30     | 69.89±1.66        | 66.77±0.69      | 52.96±0.37     | 80.61±0.10    | 74.35±3.20       | 81.27±0.96    |
| NCN      | 87.73±1.41     | 90.93±0.83        | 76.20±1.55      | 54.43±0.17     | 88.64±0.14    | 68.37±3.57       | 66.10±1.13    |
| NCN$_{rw}$    | 87.95±1.30     | 91.86±0.82        | 76.51±1.41      | 54.16±0.24     | OOM             | 73.81±4.71       | 67.32±0.81    |
| BUDDY    | 84.82±1.93     | 91.44±1.14        | 74.46±3.32      | 55.73±0.16     | More than two days    | 74.69±7.55                | 83.37±1.05             
| BUDDY$_{rw}$    | 85.06±1.81     | 91.86±1.00        | 75.74±2.03      | 56.17±0.42     | More than two days    | 77.20±4.10                | 84.64±1.29             |

We download the most latest version of the BUDDY code and also modify it based on our reweighting strategy. We can see that equipping BUDDY with the reweighting could still enhance its link prediction performance. As running BUDDY on Citation2 cannot fit on our machine with 128 G RAM, we are currently working on getting another machine to run it. We will update the results once we have them on Citation2

Moreover, **our result on Collab is still competitive** since different from subgraph sketching allowing validation edges used in message-passing when evaluating edges in the testing set, we strictly follow the conventional setting[1] avoiding using validation edges during testing time.

[1] Yun, Seongjun, et al. "Neo-gnns: Neighborhood overlap-aware graph neural networks for link prediction." Advances in Neural Information Processing Systems 34 (2021): 13683-13694.

Reviewer comment (Experiments):
The paper lacks ablation studies or case studies to demonstrate the effectiveness of the reweighting strategy. You could also provide some qualitative analysis or visualization to show how the reweighting strategy affects the prediction results.

Author response:
Based on our observation that TC consistently exhibits a stronger correlation with GNNs’ LP performance, we aim to boost TC by aggregating more information from neighbors who are more connected to the whole neighborhood and demonstrating the performance improvement in Table 1. We indeed provided case studies to illustrate the effectiveness of the proposed reweighting strategy. As shown in Figure 7(b), we visualize the relationship between the improvement of Hits@10 before and after equipping with our proposed TC-based reweighting strategy. We can see nodes with a more significant gain in their TC would also have a larger improvement in their link prediction performance.

Reviewer comment (Presentation):
The font in every figure is too small to read, and the figures are not well-designed.

Author response:
We really thank reviewer 1hhc for taking care of the readability of our work and we have constructed a comprehensive update to our paper throughout the discussion phase based on the suggestions from other reviewers. Now that all of them have joined the discussion, we have provided the updated version of our revised pdf file including all modifications. We apologize for not uploading a new PDF earlier, but did not want to cause confusion with multiple updates and just maintained all the changes in the comments until the end of this phase with the single updated PDF. 

**For the notations, we indeed admit that TC$^{\text{Tr}}$, TC$^{\text{Te}}$ are very confusion and have updated them to TC$^{\text{Train}}$ and TC$^{\text{Test}}$.**

**Moreover, we have increased the font size from 15 to 25 for all figure x/y tick labels**. We really hope with such an larger font size, our work will be user-friendly for others to read.