Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper investigates the unsupervised representation of graph data and proposes an adversarial self-supervised learning (GASSL) framework. The paper adopts the self-supervised framework in reference [12] for graph embedding representation, and apply adversarial learning to improve the robustness of the model, following the framework in [26]. A number of experimental results are given to validate the performance of the proposed model. The contribution of the paper is incremental.
main_review: The problem addressed in the paper is significant and interesting to the readers of the conference. The main contribution of the paper is to extend the self-supervised learning framework to graph data and to adopt adversarial learning to improve the robustness of the model. Both self-supervised learning framework and adversarial learning model used in the paper are not original and available in the reference. The novelty of the paper is to combine these two models to learn a robust graph representation. 
The paper provides a number of experimental results to support that good performance of the proposed approach and ablations experimental results are also given, but it lacks of sufficient evidence that why the proposed method performs well. The reviewer did not see implications from experimental results. In particular, the paper claims that “We use adversarial training to automatically generate challenging views for self-supervised learning in place of handcrafted views, which yield performance gains on multiple datasets.”  From the procedure of adversarial generator, the model is to generate an adversarial attack in the vicinity of the input sample that maximizes its loss function, how to guarantee that the generated sample should be the different view of the same instance? What is the definition of "challenging views".
English writing is clear and easy to understand. 
limitations_and_societal_impact: I would like to suggest that the authors provide to visualize the feature distribution of graph representation to validate results. 
Some minor comments: 
L84-87. Some paragraphs need to be rewritten. The paper states " For a thorough review, we refer the reader to the recent survey [50] ", and then gives two references later. 
L104.  "[26] proposed " ==> "Authors [26]  proposed "
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes to utilize the methodology of adversarial training to help the SSL problem in GNN. It establishes some basic formulations for the problem and validate the effectiveness on benchmark datasets.
main_review: ## Pros
1) This paper is well-written. The idea is presented in a clear and straightforward manner.
2) The idea aligns with the common understandings about adversarial training. For instance, adversarial training is also applied as an data augmentation method in NLP [1].
3) The proposed method is evaluated on many benchmark datasets.

## Cons
1) Although the topic is interesting and insightful, the proposed method is somewhat primitive. 
2) The improvements is relatively marginal. Sometimes the improvement is less than 0.5%, which may well be noise. The variance shown in Table 3 also suggests that the improvement is not very stable.
3) If I was right, this paper only provide evaluations on the graph classification task but omit the commonly used node classification task. Please explain.
4) The proposed method should be compared with deep graph infomax. The [44] cited paper in this work.
5) I am curious about the robustness of the proposed method. Can the author provide corresponding results [2][3]?


I may raise my score if the authors can properly answer my questions.

[1] Adversarial Mutual Information for Text Generation

[2] N. Entezari, S. A. Al-Sayouri, A. Darvishzadeh, and E. E. Papalex-akis,  “All  you  need  is  low  (rank)  defending  against  adversarialattacks on graphs,” inWSDM, 2020, pp. 169–177

[3] H.  Wu,  C.  Wang,  Y.  Tyshetskiy,  A.  Docherty,  K.  Lu,  and  L.  Zhu,“Adversarial  examples  on  graph  data:  Deep  insights  into  attackand defense,” inIJCAI, 2019

limitations_and_societal_impact: None
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper introduces a novel way to improve unsupervised graph representation learning. The proposed method utilizes the adversarial attacking techniques with a teacher and a student network to refine the generalization of graph representation.
main_review: + This paper proposes an interesting solution of unsupervised representation learning of the graphs. The work is solid, and it contains some novelty.
+ The experiments show that their methods were evaluated widely on 10 popular datasets and the results of the experiments showed significant improvement with the existing benchmark results.
 
 
This paper contains some uncleared descriptions and concerns. There are a few questions as listed below:
 
	1. Why do you add the perturbations to hidden layer 1 instead of other layers or the generated representation? Is there any discussion or comparison about this choice? 
	2. Eq3. is probably confused to readers. Could you provide more details? 
	3. In L 125, the definition of G and G' are unclear. What is G and G' exactly? 
	4. For the experiments in Table 3, why do you compare your unsupervised learning result with supervised and kernel results? 
	5. Table 4 shows some inferior results on some datasets. Is there any possible analysis and reasons? 

limitations_and_societal_impact: The authors solved some limitations but it still needs more explorations in the related area.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper presents a self-supervised learning method GASSL for graph representation learning. The authors have designed 2 key points:  a) Utilizing BYOL framework to learn graph representation; b) Using adversarial training to automatically generate challenging views for self-supervised learning. The idea seems to be simple but effective. Sufficient experimental analyses are conducted.
main_review: Strengths:

1.	To the best of my knowledge, it is the first time that BYOL has been applied for graph self-supervised learning. While this point seems minor, its benefit is greatly supported by the experiments in Table 2 compared to SimCLR.

2.	Different from Euclidean data like images, how to generate valuable positive samples for graphs is non-trivial and central in the domain. Traditional methods like GCC and GraphCL resort to hand-crafted sampling which easily incurs bias during training. This paper proposes to apply adversarial noise to the input features or hidden layers to generate worse-cast positive samples, with the goal of enhancing robustness for BYOL training. The idea is simple yet interesting.

3.	Experimental evaluations are good on the conducted benchmarks and sufficiently support the claims the authors have made. 


Weaknesses

1.	This paper is almost well written. Yet there still remains certain unclarity. In line 123-124, the authors mentioned that “Our method relies on the positive view of the input graph, which commonly contains: node dropping, edge perturbation, subgraph, and these need to be designed manually.” This statement is quite confusing. Does this mean that the proposed adversarial technique is not self-contained and is performed on the perturbed samples generated by existing data augmentation methods? If so, why has the claim “We use adversarial training to automatically generate challenging views for self-supervised learning in place of handcrafted views, which yield performance gains on multiple datasets.”

2.	The authors mentioned that the experiments are conducted on large-scale OGB, which is over-exaggerated. It is true that OGB contains large datasets for benchmarking, but the datasets that applied in this paper are small. Please correct the sentence to make it more professional. 

3.	It is well known that involving adversarial objective will hinder the training dynamics. Have the authors met this issue and how to address it if so. It is suggested to provide the convergence curves for better clarification.

4.	Regarding the comparison between initial Perturbation and hidden Perturbation, what about the ablation by considering them both? And what about performing perturbation on other layers (not just the first hidden layer)?

5.	Several important references on graph self-supervised learning are not discussed and cited. See below.


[A] Strategies for pre training graph neural networks, ICLR 2020.
[B] Self-Supervised Graph Transformer on Large-Scale Molecular Data, NIPS 2020.
[C] Graph Representation Learning via Graphical Mutual Information Maximization, WWW 2020.


##########Post Rebuttal###########
After reading the authors' responses carefully (including their replies to my further questions), my concerns are well addressed. I believe the contribution by this paper is novel and valuable, and it does address a crucial issue in graph self-supervised learning. In terms of the writing, I strongly suggest the authors to keep their words and revise their paper according to their responses. 

Overall, I decide to increase my score.
limitations_and_societal_impact: Please see the weakness part above.
review_title: null

Meta-review: This paper investigates the problem of self-supervised learning in the context of graph representation learning. It proposes to adopt the technique of adversarial training to automatically augment a training set, and then devises the corresponding adaptation scheme to make adversarial training viable. The authors conducted thorough and insightful experiments (also supplementing some experimental results in the rebuttal) on several benchmark datasets. 

Although adversarial training was originally proposed as a defensive algorithm aiming at increasing the robustness of a certain learning model, it has also been recognized as a variant of data augmentation or hard example mining in many other works. Moreover, employing the philosophy of adversarial training to help generalization has also been discussed in other fields like NLP and style transformation. Thus, it is reasonable to see that the same theory can be verified in the domain of graph learning. Of course, verifying this theory in a new domain like graph learning requires huge efforts and insightful designs, which holds as the main contribution of this work. The proposed method is relatively simple but very effective on the evaluated tasks. In the rebuttal phase, the authors adequately answered the questions from the reviewers, including addressing the issues of writing clarity and evaluations in extra experimental settings. Eventually, the four reviewers reached the consensus of accepting the paper. Therefore, the AC recommends acceptance as poster regarding this submission. 
Decision: Accept (Poster)