Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The paper provides a systematic approach to building graph contrastive learning models for
any particular graph learning taks and dataset. The suggested approach builds on the Information Bottleneck principle and unifies all recent graph representation learning methods based on contrastive learning.

main_review: Overall I think this is a good paper, trying to present a unifying view on graph contrastive learning. A typical graph contrastive learning model is decoupled into three sequential modules: View augmentation, view encoding and representation contrasting. Based on the principle of the Information Bottleneck, the paper suggest feasible objectives to identify optimal modules for each of these three parts:
* The augmented views should contain as much task-relevant information as possible, while they shoul share as little information as possible.
* The view encoder should be task-relevant and simple as much as possible.
* Thhe contrastive mode should keep task-relevant information as much as possible70after contrasting

The paper also argues that negative samples are not necessarily required for graph contrastive learning.

Originality: The paper unifies existing approaches, and provides a novel systemic perspective on graph contrastive learning.

Quality: The paper is technically sound, corollaries are proven in the the appendix, the work is complete as such.

Clarity: The paper is written well. 

Significance: The results are important to researches in the area of graph representation learning. It's holistic approach to graph contrastive learning through the lens of the Information Bottleneck principle provides novel insights for this area of research. The presented empirical results look convincing.

A minor detail is Eq.9: I think it would be nice to explain with a few sentences what the principle behind SimSiam is.
limitations_and_societal_impact: Limitations are mentioned in the final section, but not explicitly explained.
Societal Impact is missing.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes InfoGCL, a general framework for graph contrastive learning framework design. More specifically, InfoGCL investigates the optimal (and practically optimal) option for three modules in graph contrastive learning, i.e. view augmentation, view encoding and representation contrasting. Experimental results have proven effective over various benchmarks compared with groups of the kernel, supervised and unsupervised. 
main_review: Strengths: 
1. Good formulation and clear identification of the three-stage graph contrastive learning process. 
2. Technically sound through the angle of information bottleneck of optimal graph contrastive learning options, with improved explainability.
3. Extensive experimental on InfoGCN over a large group of baseline through fair comparison with important observations.

Weaknesses:
1. A few terms and concepts that may not be generally recognized are not formally defined or fully discussed.
2. IB-based InfoGL seems to be only applicable for a single (given) task.
3. Some experimental results are less convincing and somewhat lack sufficient explanation. 

Detailed comments and questions:
1. As pointed out in the weaknesses, it is assumed that all the proposition 1-3 are applied when only one task T is pre-given, while many GNN frameworks are intended for multiple downstream tasks while graph contrastive learning is served for pre-training. One drawback is that, once the views are optimized over “task-related” information, the InfoGCL would have to perform independently (or train from scratch) for multiple downstream tasks. 
2. Regarding experiment: 
2.1  The variances of Node2vec and sub2vec on the MUTAG dataset are extremely high which seems strange. 
2.2 The performance drop without negative samples on the Cora dataset and others is insufficient and not convincing (even though the authors only hypothesize the sparsity is one factor). 
3. Many terms that are not universally recognized or self-explained are not formally defined and may need clarification. Examples are: 
3.1 The term “view” i.e. generated variations (subgraphs) from the original graph which may have confusing meanings in other graph communities. 
3.2 The Venn figures in Figures 1 and 2 need revision for better visibility and explanation. 
3.3 The detailed definition of 5 modes in Section 4.3
3.4 “Negative pairs” in InfoNCE loss vs “Negative samples” in Section 4.5

Minor corrections or suggestions:
1. The Algorithm 1 block in the appendix is relatively important to understand the practical implementation of InfoGCL, which is recommended to be added in the main section. Otherwise, reviewers may be lost about how InfoGCL is working in practice.
2. Line 172: Typo “gprah”


limitations_and_societal_impact: Not applicable
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: In this paper, the authors study the graph contrastive learning problem and propose an information-aware approach called InfoGCL. The motivation for this work is that all recent graph contrastive learning approaches are customized, and it is an open question how to build the graph contrastive learning method for particular tasks and datasets. The authors' method decouples the typical graph contrastive learning into three modules and formalizes how to find the optimal of the three modules into three optimization problems. Experimental results on both node-level and graph-level tasks demonstrate the advantages of the proposed method.
main_review: This paper is pretty interesting overall. Different from existing work, this work tries to unify the various contrastive learning approaches for graph-structure data. It extends the information bottleneck theory to optimize the representation learning in the graph contrastive learning framework.

One main contribution of this work is to understand graph contrastive learning from the optimization view. From the observations in section 5.3, it seems that the proposed InfoGCL is general and well supported by existing work. Personally, I like this work because instead of simply proposing a method to achieve better results, it digs into the hidden reasoning behind graph contrastive learning.

This paper contains solid theoretical content, which is well organized and easy to follow. The motivation, contribution, and methodology are well written. The experimental results support the claims. Overall, this paper is with good writing and clarity.

However, there are some specific questions/concerns. 

1. In the Introduction, “Second, graph data has unique properties that distinguish it from other types of data, such as rich structural information and highly diverse distribution”. I see the proposed method considers graph structure, but not clear how it could generalize to cover various graph properties.

2. In the Introduction, “However, different from the typical representation learning, there are two information flows involved in the two augmented views in contrastive learning.”. I am not an expert in the information theory area. Could the authors elaborate more on the difference/significance in terms of the two flows?

3.  Corollary 1 (Section 4.1). Are you sure to achieve optimal conditions in real data scenarios? It looks very constrained.

4. In Section 4.3, “The main motivation is: the optimal contrastive model keeps the most task-relevant information after the representations are aggregated”. This problem appears to be a discrete optimization problem, and the authors have used the method of obtaining the optimum by traversal. Can there be a more optimal solution here?

5. In Section 5.4, “Therefore, we make the hypothesis: negative samples benefit graph modeling, especially when i) network topology, and ii) node features are extremely sparse.”. I am interested in the study of negative samples. I have not seen these conclusions in the cv domain. Can the authors provide more analytical or experimental supports?

6. In Section 5 (Experiments). The datasets of Cora Citeseer Pubmed are very small. I suggest the authors experiment with larger datasets.

7. The propositions in Section 4.4 are interesting and seem reasonable. More evidence/support is preferred.


limitations_and_societal_impact: Yes
review_title: null

Meta-review: The paper proposes a systematic approach to contrastive learning on graphs which builds on ideas from information bottleneck theory and unifies multiple graph representation learning methods in a single framework. The paper is written well and relevant to the NeurIPS community. All reviewers support its acceptance, especially due to the novel and promising approach which unifies existing methods and provides new perspectives for graph learning. Reviewers highlighted also that the approach is technically sound, that claims in the paper are theoretically supported, and convincing experimental results. Please take the feedback on concerns and limitations from reviewers into account when preparing the camera ready version of the manuscript.
Decision: Accept (Poster)