Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors of the paper identify and demonstrate a negative augmentation problem in graph augmentation methods. They propose to use a knowledge distillation method to solve this problem by training a teacher model on the augmented graphs and a student model on the original graphs. Experiments are conducted on two homogeneous graph datasets and six heterogeneous graph datasets. Improved performance are observed when the knowledge distillation method is applied to three learning-based graph augmentation methods and three different GNNs.
strengths_and_weaknesses: Strength
1. The authors demonstrated sound statistics and analysis of the negative augmentation problem.
2. Extensive experimental results are presented on the node classification task, using three representative GNNs and three learning-based graph augmentation methods.
3. The paper is well written.

Weakness
1. In Figure 3, are the inputs of the teacher model and the student model swapped?
2. Evaluations are only conducted on the node classification task. Would like to see experiments on more tasks, such as edge classification, graph contrastive learning.
questions: 1. Can other techniques achieve the same effect, such as increasing regularization, increasing the diversity of the sampled augmented graphs? Could you provide a comparison with these methods?
2. Has anyone proposed using knowledge distillation (in any way) to solve the negative augmentation problem?
3. Can KGDA be easily applied to graph contrastive learning or any other graph application?
limitations: 1. Evaluations are only conducted on the node classification task. Would like to see experiments on more tasks, such as edge classification, graph contrastive learning.
2. The proposed method does not seem to address the problem that graph augmentation methods tend to increase the homophily ratio of the graphs. It is more of a regularization that guides the student model on downstream tasks.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper claims that the existing graph structure augmentation methods may cause the distribution shift issue. In this paper, the authors design a knowledge distillation based method to improve the generalization capability for graph structure augmentation.
strengths_and_weaknesses: Strengths:
1. Authors find that the existing graph augmentation method will lead to the distribution shift problem.
2. This paper is well-written and easy to follow
3. The idea behind is straightforward.

Weaknesses:
1. The authors claim that Negative Augmentation will lead to poor generalization. While It does not make sense based on your experiments.  The experiments show that the existing methods do perform well than baselines. This means that if those baselines are negative augmentation, they should perform worse. Or authors may need to conduct the experiment specifically to test the generalization. 
2. While the existing methods have a big gap in homopgily ratio from the augmented to original data, but no experiments show that the proposed method does not have it. Moreover, why big gap in homopgily atio represents worse generalization capability? intuitively, a big gap in homopogily ratio can help model to ''see'' more different graphs and increase the generalization.
3 . Typos in Table 2: GKDA -> KDGA? 
4. The novelty is limited. As the proposed KDGA is just a simple knowledge Distillation method. The authors just apply it to existing augmentation methods to achieve better performance. 
questions: See the weaknesses.
limitations: N.A.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper claims that they are the first in considering the distributional shift between an original and augmented graph. They propose Knowledge Distillation for Graph Augmentation (KDGA) to address the potentially negative impact of this distributional shift. The model consists of a teacher and a student model. The teacher is a GNN model that extracts information from the augmented graph and injects the knowledge into the student model which has only seen the original graph by minimizing the KL divergence of the output logits of these models. 
strengths_and_weaknesses: **Strength**

-- The paper is well-written and the authors are trying to tackle an important aspect of graph contrastive learning. 

-- The approach is simple and can be easily implemented in an efficient way.

**Weaknesses**

-- The authors should make it clear from the begining of the manuscript that by 'distribution' they mean the homophily ratios of the graph structures.

-- The claim in line 63 that little work studies structure augmentation is not accurate just by considering the already cited literature. 

-- The claim in lines 130 and 131 that graph structures for training and testing in graph augmentations are completely different isn't necessarily true. This is a property you can control in a few graph contrastive methods and automated GCL methods might limit this issue automatically. 

-- typo: Figure 5 caption, "homopgily"
questions: -- Have the authors studied the sensitivity of the final performance on the GKD loss weight in equation 9?

-- Figure 5 is showing homophily ratios in the original and augmented graphs during training w/o the GKD loss, right? If that's true, what would the trajectory look like with the GKD loss?
limitations: there is not social impact section in the manuscript.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors investigate on an interesting problem for graph augmentation: distribution shift between the original and the augmented graphs. The authors leveraged knowledge distillation to narrow the distribution gaps between the original and the augmented graphs. Experiment results show that the proposed framework (KDGA) based on the existing graph augmentation algorithms achieve superior results. 
strengths_and_weaknesses: Strengths: 
1. The authors pointed out the observation that there is a distribution shift between the original graph and the augmented graph. They also proposed to use the graph homophily ratio as the metric to quantify the distribution shift. 
2. The authors illustrated the concepts and methods clearly with well presented figures. The overall presentation is good and easy to follow

Weakness:
The proposed the KDGA framework is based on the existing graph augmentation algorithms, and the student-teacher distillation framework. Would like to see more novelty in this work.

questions: See above
limitations: See above
review_title: null

Meta-review: metareview: The paper identifies the problem of negative augmentation in graph augmentation methods as it may cause the distribution shift issue. The paper thus proposes a knowledge distillation method that trains a teacher model on the augmented graphs and a student model on the original one. Reviewers had concerns on the novelty of the approach and experiments. The discussion between the reviewers and the authors were effective, and two of the reviewers have raised the scores to accept/borderline accept. I'd recommend acceptance.
confidence: Certain
Decision: Accept