Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper focuses on the augmentation of graph contrastive learning. Authors make further explorations on graph contrastive learning based on the previous CV contrastive work (HaoChen et al. (2021) ) and hope to design a good benchmark by analyzing the relationship between GCL/auto-encoding methods and augmentations. In the analysis part, authors mainly use the perspective of population augmentation graph work to find connections with existing graph augmentations (mainly You et al., 2020a) and analyze the relationship between GED and task performance/downstream labels. In the benchmark part, authors proposes a benchmark based on the idea of "STYLE VS. CONTENT" which is inherited from CV contrastive work. The analysis of the experimental results is okay.

main_review: Strengths: The motivation of this paper is relatively new. Data augmentation in graph contrastive learning is an important part which is often overlooked by researchers. It is a successful transfer of work from CV and can show that the author has a deep understanding of the latest work on CV contrastive learning. The benchmark design based on motif and the design of "STYLE VS. CONTENT" on the graph are both highlights.

Weaknesses: The analysis of the relationship between graph contrastive learning effective conditions and data augmentation is not theoretical enough. Some sentences confused me when reading such as "To define a min-cut partition, the number of edges crossing a subset of nodes should be minimized while the number of edges within a subset should be high". It would be better if the theoretical analysis part was supplemented in more detail. The problem and process of analysis in Section 3.2 is too hand-waving.
Secondly, I think your experiment with GNN without training is controversial in Table 1. If you follow the training mode of graph contrastive learning (first defined in DGI), the untrained GNN is combined with a trained downstream linear layer. I think the existence of this linear layer is not enough to support your conclusion that untrained GNN is competitive. If you insist on proving this statement, please use the results of the clustering experiment which can be found in MVGRL. 
Finally, I think the contribution of the paper is not enough. Although the analysis and motivation are new, they are still used from CV. The design of benchmark and "STYLE VS. CONTENT" is novel but simple. I think it is a relatively complete work to propose a better data augmentation or sampling method on this benchmark based on the riched theory.

summary_of_the_review: The motivation of the article is good, but I have some problems in terms of contribution and theoretical analysis. Experiment and analysis are okay.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper studies the success conditions for graph contractive learning (GCL) with generic graph augmentation (GGA) and asserts that these conditions depend on the graph edit distance of samples within and across classes. Using empirical evidence, the authors demonstrate that reconstruction-based approaches (GAE and AAGAE) perform well at low graph style regimes, while GCL with GGA benefits from moderate graph style regimes.
main_review: This paper studies the success conditions of graph contractive learning (GCL) with generic graph augmentation (GGA) and claims the success depends on the graph edit distance between classes. It empirically show that reconstruction-based approaches (GAE and AAGAE) perform well in low graph style regimes and GCL with GGA benefits from moderate style. 

The problems investigated in the paper is interesting and is significant to the unsupervised graph representation learning community. However, their claims lack rigorous mathematical and empirical demonstrations. In addition, the analysis is restricted to spectral contractive loss functions, and it is not clear how to extend the analysis to general GCL approaches. Further, the experiments were conducted on relatively limited methods, and it is not clear whether the results are applicable to more general approaches.


## Strength


1.	The presentation is easy to follow.
2.	The problem of identifying success conditions of graph contractive learning investigated in the paper is interesting.
3.	The experimental results, i.e. the impacts of the ratio of style and content on the performance of GCL, GAE, and AAGAE, are interesting and may provide some insights to the unsupervised graph representation learning community.

## Weakness  



1. The claims of the paper mainly build  upon the theoretical results of a recent paper [1], whose details are omitted in the paper. Would be great to carefully discuss the relationship to the results of [1]
2.  The analysis in Section 3 is restricted to PAG and the spectral contrastive loss and there are no illustrations to show how to extend the analysis to general GCL approaches.
3. Additionally, the claim, that the success of GCL with GGA depends on the graph edit distance between classes, lacks rigorous mathematical and empirical evidence.
4.  The experiments were conducted for only three methods (i.e., GCL, GAE, AAGAE) and the analysis of paper is restricted to GCL with GGA. There is no indication whether their results can be applied to general GCL approaches or other reconstruction-based approaches.

summary_of_the_review: This paper studies an important open problem,  but is lack of rigorous mathematical demonstrations and has limited empirical studies.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper attempts to study the connection between graph edit distance and the reason behind the success of self-supervised graph learning. 
main_review: Recently, we have witnessed an explosive development of self-supervised graph learning methods. As one of the most import parts, data augmentation technology provides a flexible way to generate training set without the high-cost labels. Many works draw lessons from the computer vision community and design a bunch of methods to generate the augmented views for contrastive learning graph representations. However, it's still not clear why and to what extent the data-augmentation methods have positive benefits for directing the graph neural models to the right way. There're some works trying to uncover the mystery by studying the contrastive loss, but rare attention being paid to the impact of the graph augmentation. While this work seems to be a pilot study about the condition when contrastive graph learning can work well. From the manuscript, we can see that authors present the relation between graph edit distance and augmentation operations (e.g. dropping edges/nodes, adding edges/nodes), and claim that the performance of contrastive graph learning could be bounded by the graph edit distance (GED) between different classes. 

Concerns:
1. The studied problem in this work is very important for the research community. However, the quality of the presentation make it difficult to understand the motivation behind. After checking the draft several times, it's still unclear why the GED can be a general criterion to judge the success of GraphCL with both node attributes and graph structure as the input data. As we know that GED is defined on the graph structure. I just wonder the generalizability of the claim raised by this work. Suppose that the GED has positive impact, the writing is difficult to follow. There're many claims which are copied from the previous work proposed by HaoChen et al.. While they are not organized logically, and the missing background knowledge makes some claims difficult to capture the logic behind. For example, what is the definition of spectral decomposition, which is presented in HaoChen et al. But it shows up in Section 3 without any explanation. 

2. Besides the discussion about the connection GED and GraphCL, the contribution of this work seems to be applied the augmentation method to reconstruction-based task like edge reconstruction. However, it seems not to be discussed in the main content until the section for presenting the experiment setting.


summary_of_the_review: Good research problem, but the presentation is not good enough to make the key idea clear.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper targets on investigating the role of dataset properties and augmentation strategies on the success of graph contrastive learning and reconstruction-based approaches.
main_review: One of the claimed contributions is the “theoretical analysis of when contrastive learning is expected to work well”. However, in section 3.2, there is no link of the “fixed error” of the contrastive learning performance to the graph properties.  

A conclusion is made from Fig 1 (a), “the learning invariance was not necessary for downstream task performance”.  However, there does exist performance (accuracy) improvement on dataset of IMDB-BINARY and NCI1.  It would be more valuable to study the necessarity of learning invariance on different types of graphs, and how this necessarity depends on what kinds of graph properties. 

Another difficulty is found in understanding the claimed contribution to “identify a style vs. content trade-off in graphs and introduce an extensive benchmark setup that can carefully control this trade-off”.  Section 5 did analysis of style (irrelevant information) vs. content (task relevant information). However, it is still unclear about how to identify a trade-off parameter in a given application problem. 

More details should be given about datasets PROTEINS in evaluation, same for  MUTAG, IMDB-BINARY and IMDB-BINARY.


summary_of_the_review: The paper works on an interesting and valuable problem. However, the analysis results are not strong enough to support the claimed contributions.
review_title: null

Meta-review: In this paper, data augmentation for graph contrastive learning (GraphCL) is studied. Most reviewers agree that the problems addressed in this paper are interesting and important for unsupervised graph representation learning literature. However, many reviewers were not fully satisfied with the novelty and the claim of the main contribution of this paper, a theoretical analysis of the conditions under which data augmentation works in GraphCL, due to the lack of clear explanation and evidence. Unfortunately, no reviewer has suggested acceptance of this paper at this time.
Decision: Reject