Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper is trying to tackle the area of graph self-supervised learning, which aims to derive useful node representations without labeled data. The paper proposes T-BGRL, a novel non-constrastive model to solve the link prediction problem. 
strength_and_weaknesses: Strength:
1. The problem the paper is trying to solve is important research problem.
2. The method proposed is easy to implement and the experiments show that the performance did improve on most datasets.

Weakness:
1. I do not quite understand why we want to apply non-contrastive learning methods for link prediction. What is wrong with those contrastive learning methods?
2. Moreover, I do not understand why the proposed method falls in the category of non-contrastive learning methods. So it is very hard for me to judge the significance of the method.
clarity,_quality,_novelty_and_reproducibility: The paper is clear in terms of the algorithm and experiments but the importance and originality of the work needs to be addressed more clearly.
summary_of_the_review: Overall, the paper needs some improvement on explaining the significance of the work for it to be accepted.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper follows BGRL, and notices that it generalizes poorly due to a lack of negative examples.
They propose a non-contrastive framework for link prediction named T-BGRL that uses cheap “negative” samples to improve generalization. 
Experiments show that T-BGRL improves BGRL’s inductive performance in 5/6 datasets and is more efficient than contrastive methods.
strength_and_weaknesses: Strength:

[1] This paper finds that BGRL tends to overfit the training graph, and thus can only perform well in the transductive setting. They introduce cheap “negative” samples to improve generalization on the indictive setting.

Weaknesses:

[1] They said it's the first work to explore link prediction with non-contrastive SSL methods. However, BGRL is a non-contrastive model and can be applied to link prediction tasks easily.

[2] These papers didn't compare with subgraph-based methods due to they are slow during inference. however, these subgraph-based methods perform well on large datasets. How about the performance of T-RGRL on large datasets (i.e. ogbl-ppa and ogbl-collab)?
clarity,_quality,_novelty_and_reproducibility: [1] This paper is clearly written and well organized.

[2] They have proof of the advantage of T-BGRL on small datasets but haven't evaluation on large datasets.

[3] This paper is an extension of BGRL and the implication of cheap “negative” samples is novel.

[4] The method is easy to reproducible.
summary_of_the_review: The analysis of BGRL makes sense and the proposed T-BGRL performs well in the inductive setting.
I am open to raising my score if the authors can conduct experiments on ogbl-collab or ogbl-ppa.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper focuses on investigating the performance of existing 
non-contrastive methods for link prediction in both transductive and inductive 
settings. In their experiments, they find that BGRL generally performs well in transductive settings, but poorly in the more realistic inductive settings, which motivates them to develop an extension of BGRL named T-BGRL to alleviate its overfitting phenomenon.
strength_and_weaknesses: Strength

1.The writing of this paper is good and easy to follow.
2.The experimental results on non-contrastive learning for link prediction is convinced and comprehensive.

Weakness
1.The novelty and theoretical analysis of the developed method is not enough
clarity,_quality,_novelty_and_reproducibility: I appreciate the effect of the authors to provide comprehensive experimental results of non-contrastive learning methods for link prediction, and also the intuitive analysis of their performance. However, this paper is more like a technical report of these non-contrastive learning methods to me, rather than a paper focusing on a new developed method.

1.The motivation is mainly derived from the experimental results rather than the theoretical analysis of shortcomings of existing non-contrastive learning methods for link prediction. In chapter 3.1.1, the author did not provide a convinced explanation for the reason why BGRL can work well in the transductive setting.

2.I can understand why the loss of ML-GCN can work well on link prediction, but what’s the connection between the loss of BGRL and ML-GCN? Why the loss of BGRL can also work well on link prediction?

3.The extension of BGRL is based on the hypothesis that ``One possible reason for the poor 
performance of BGRL in the inductive setting is that it is unable to correctly differentiate unseen 
positive from unseen negatives, i.e., it is overfitting on the training graph.’’ But there is no experiment to support this hypothesis.

4.Additionally, I am not sure whether other non-contrastive learning methods also perform poorly on link prediction. The best solution could be provide a theoretical analysis on why these methods tend to fail on link prediction.

5.Some suggestions. 
It will be better to release the full name of BGRL when you first cite it.

6.Typo
Page 5. which relative to
summary_of_the_review: This paper is more like a technical report of these non-contrastive learning methods to me, rather than a paper focusing on a new developed method.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper first compares the performance of contrastive and non-contrastive methods on the graph link prediction task. Second, a novel model i.e. Triplet-BGRL (T-BGRL) is proposed based on Bootstrapped Graph Latents (BGRL). The proposed T-BGRL is categorised as a non-contrastive method and it relies on an efficient corruption method to generate cheap negative samples. Extensive experiments demonstrate that T-BGRL can achieve competitive performance for both the transductive and inductive settings with a good efficiency. 
strength_and_weaknesses: ### Strength:

1. The discussion of contrastive and non-contrastive methods for the link prediction task is comprehensive. 

### Weakness:

1. In Section 3.1, it’s unclear why the experiments only focus on the performance of the encoder and adapt the same decoder for all evaluated models. Moreover, it’s unclear why the encoder and decoder are not trained together. If this is following some existing works, please cite them and discuss the reason(s).
2. ML-GCN is only loosely defined but used as an important baseline. Is ML-GCN just a GCN model using the margin-loss? Also, what is ‘the supervised GCN’ (Section 3.1.1)? Is it just an end-to-end GCN?
3. Figure 1 does not have an x-axis. 
4. ‘collapse’ is used without only once in Section 3.1.1 without definition and further discussion or claim. 
5. Why ML-GCN and BGRL are both claimed as ‘our’ best-performing contrastive/non-contrastive models?
6. Is the SHUFFLEFEATRANDEDGE the same as the SHUFFLEFEATRANDOMEDGE method described in A.6? It’s unclear why a randomly generated adjacency matrix can help the effectiveness of BGRL. In fact, if the proposed method only generates a random matrix, then there is no corruption. 
7. It’s unclear why some arguments/conclusions are stated in the caption of the figures e.g. figures 1&3. 
8. It’s unclear why the early-stop is only used by ML-GCN.
9. This paper is poorly organised. Usually, there is no connection between two sections. Furthermore, many sections (e.g. the last subsection of Section 3.1.1) end without a solid conclusion or constructive advice. 

### Typos:
1. On page 3, ‘none evaluate or target link prediction tasks.’  ’none evaluates or targets the link prediction task.’
2. On page 5, ‘which relative to an anchor node u is expressed as follows’. This sentence needs to be rewritten.
3. On page 5, ‘wen & Li (2022) looks at xxx’ ‘wen & Li (2022) looked at xxx’

clarity,_quality,_novelty_and_reproducibility: The quality of this paper needs to be improved since it is not organised very well and some claims are not fully supported. The novelty is somewhat limited since the main content is focusing on the discussion of existing works while the proposed model is based on BGRL. 
summary_of_the_review: This paper needs to be polished, and some experimental setup needs to be justified to support the proposed claims.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The authors propose an empirical study for non-contrastive graph self-supervised learning (SSL) methods, in the context of link prediction. The arguments for this study are that:
* non-contrastive methods are significantly faster due to no need for generating negative samples, therefore making them more industrially relevant;
* link prediction tasks have significant downstream importance, but have been comparatively neglected by existing studies into non-contrastive SSL.

Therein, the authors first demonstrate that, despite opposite expectations, BGRL (a current state-of-the-art non-contrastive SSL method) appears to perform quite well in this regime, at least when the task is transductive. Qualitative analyses demonstrate one possible failure mode when moving into inductive tasks (overfitting to the training graph), and a modification, T-BGRL, is proposed. T-BGRL also includes a "negative" sample obtained by cheaply corrupting the target node's neighbourhood. The proposed T-BGRL method appears to significantly improve on BGRL on these tasks.

After the rebuttals and clarifications by the authors, the reviewers are split on whether to accept. The key remaining concerns appear to be:

* The lack of theoretical insight for the utility of (T-)BGRL in this context;
* The lack of motivation for the random corruption utilised in T-BGRL.

Having carefully analysed the paper and the reviewers' points, I have decided to discard both of these arguments and recommend (weak) acceptance. I fully agree that the paper would have benefitted from more rigorous analysis of these effects. However, especially given the complete lack of prior work in this area, the comprehensive empirical study conducted by the authors, and the potential immediate utility of the work, I am of the opinion the work in its current form will be quite interesting, and likely useful, to the ICLR audience. That being said I do hope that the authors will continue working on these problems in the future, and deepening the theoretical links!

Lastly, I was a bit surprised not to see VGAE [Kipf and Welling, NeurIPS BDL'16] cited or compared against when relevant. It's one of the simplest (and earliest) methods for self-supervised link prediction, albeit contrastive.
justification_for_why_not_higher_score: The work is, in its present form, a borderline accept. Giving it a Spotlight would have required significantly stronger empirical and theoretical investigation.
justification_for_why_not_lower_score: As discussed above, I have decided to recommend acceptance because of the complete lack of prior work in this area, the comprehensive empirical study conducted by the authors, and the potential immediate utility of the work, I am of the opinion the work in its current form will be quite interesting, and likely useful, to the ICLR audience.
Decision: Accept: poster