Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents a data augmentation methods for training graph neural networks in general. They firstly fit a generative model that learns the conditional probability of input node features. The training data are then augmented using the generative model using the proposed importance sampling method and used for training GNNs. The proposed methods were evaluated on 7 public graph benchmark datasets. Authors attempted to show that the proposed augmentation methods could improve the benchmark classification accuracy over the baseline methods.
main_review: Strengths:
The idea of using generative model to augment GNN training data is interesting.

Weakness:
1. The writing and organization of this paper need to be polished before submission. The presentation is chaotic. There are also grammatical errors and typos through out the paper. eg. 'data argumentation' in Abstract. The experiment numbers across the seven benchmark datasets are hard to read. It is unclear why they are not presented fully in a single table.
2. The description of the proposed methods are unnecessarily notion heavy making it time consuming to decipher what the main ideas really were. Authors may consider elaborating the intuitions of the main ideas clearly before formulating them with notions.
3. Given the complexity of implementing the proposed augmentation methods, the accuracy gain is marginal comparing to the baselines with / without the complex data augmentations. It is unclear if the proposed methods can be useful for real-world applications. 

summary_of_the_review: Though the idea of using generative model to augment GNN training data sounds interesting, I would not recommend to accept this paper given its current presentation quality.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper studies the problem of feature augmentation for training the graph neural networks. Specifically, given the central nodes features and local structure, the proposed approach estimates the distribution of the node features of neighbors. Experimental results show that the proposed approach can further improve the baseline backbones.
main_review: Generally, the idea is clear and easy to follow. The proposed local augmentation is feature-level augmentation, which uses CVAE to generate the neighboring features based on the local structure and the center node's features. The augmented features can 1) improve the generalization and 2) recover the noisy/missing features. However, several issues require addressing as listed below.

1. The unclear comparison with baselines. GraphVAE can actually generate the structure and node features by VAE (\tilde{A} and \tilde{F} in the paper). It is suggested to discuss the main difference. Moreover, in Section 2.2, it states that the feature-level augmentation \bar{X} is from other papers [Kong et al., 2020]. It is unclear that this paper modifies [Kong et al., 2020] or directly applies it. Last, the idea of using Skip-gram in graph embedding has been developed before (e.g., Deepwalk). The difference should be highlighted and the papers should be cited.
2. The theoretical contribution is minor. The optimization of VAE is mature, while there are also several papers using importance sampling for estimating marginal likelihood in VAEs.
3. The improvement to GRAND is relatively small in Table 3, e.g., 0.47% on Cora dataset. Moreover, the experiments on the robustness to missing information should contain other baselines to demonstrate the superiority of the proposed approach.
summary_of_the_review: The idea is interesting but the difference and theoretical contributions are unclear. It may be the presentation issue but seems to borrow several parts from other papers. Moreover, the improvement is minor (0.47% on Cora dataset).
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a local augmentation strategy to enhance the graph representation and benefits corresponding downstream tasks, where the augmentation is achieved with a generation conditioned on the local neighbor structure. The authors justify the proposed method with both quantitative and qualitative results on benchmark datasets.
main_review: This paper presents an interesting method to enhance the graph representations with the proposed local augmentation techniques. The authors propose to leveraging a generator to vary the node features conditioned on the neighbor information in the subgraph.  The generated samples are further re-selected with an importance sampling technique to ensure the generated features are informative to enhance representation learning.

From the side of technique in the paper, the proposed method well differs the current augmentation methods and shows the improvement on different backbones. The presentation of the technical part is generally well written and clear. However, I still have some small confusion about the notations in section 2.2. For example: Is $p_\psi(z|X_i)$ a parameterized by $\psi$ or not? In the text I found $p_\psi(z|X_i) = N(0,I)$. 


The experiments show the results of classification with different backbones on different datasets and we can see the improvement of local augmentations. Some expected ablations are not shown; thus I have some suggestions for this part to improve this paper:

- I guess it is non-trivial to re-organize Table 3, as it would be easier to compare if the authors could put methods on the same backbone together (e.g. GCN, G-GCN, DE-GCN, GAUG-OGCN and LA-GCN).  

- If it is possible, it will be better to show the results of GAT, GCNII and GRAND with the other augmentation methods. 

- I am curious how the proposed local augmentation performs when combined with other augmentation techniques. For example, will it further improve the GNNs by putting LA and DropEdge together? 

- How does the importance sampling module improve the generated feature quality? Probably it could be shown in Fig 3 to justify the importance of the proposed module. 

------- update after rebuttal ---------------


After carefully reading all the reviews, the authors' responses, and corresponding revisions, I think the quality of this paper is better than the original submission. However, as their presentation is still not clear enough to me and some points raised by other reviewers, e.g. computation complexity vs performance improvement are not well explained, I am inclined to keep my current rating.
summary_of_the_review: This paper presents an interesting local augmentation method in a way of generation to enhance the graph representations and show the results of classification with different backbones on different datasets and we can see the improvement of local augmentations, but can be further improved by providing more ablation studies.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: Data augmentation remains under-explored on graph-structured data. In this work, the authors introduce local augmentation that enhances the feature of each node by its local subgraph structure. Specifically, they model the data augmentation as a feature generation process: given a node’s feature, they learn the conditional distribution of its neighbors’ features to make a generative augmentation model. Experiments show that the proposed local augmentation makes performance improvement on real-world data.
main_review: Strengths
- This paper solves an important problem, graph augmentation, which has a large impact on many graph-related tasks such as node classification.
- Eq. (2) to (4) provide a good motivation for why augmentation is able to improve the performance of a classifier, in a probabilistic view.
- The proposed approach improves various GNN models in benchmark datasets.

Weaknesses
- The node index i just appears at Eq. (5) without any background information. Does i iterate over all nodes or only the training nodes? Do we assume the independence between different i values? Clarification is needed.
- The discussion before the “Optimization of the MLE” paragraph seems redundant. The local augmentation is not very different from previous approaches, since X_v works as the condition variable of the variational inference to infer the X_u1 and X_u2; X_v is expected to be similar to X_u1 and X_u2 for better estimation.
- Eq. (6) is incomplete and needs much clarification. a) What are the structures of networks f and g? b) Do the authors use the reparameterization trick for generating Z? c) Why do the authors use the same f function for both the encoder and decoder functions? d) What is the meaning of running variational inference if all neighbors of each node have Z sampled from the same distribution N(0, I)?
- The name of Section 2.3 should be active learning rather than importance sampling, since a) the goal is not to estimate an unknown probability distribution, and b) the augmentation is done for improving only the current status of the classifier; there is no guarantee that the quality of \bar{X} improves globally.
- The proposed LA-GNN structure contains much more parameters, which result in a higher risk of overfitting as a consequence. An additional study is needed for the number of parameters, running time, and the sensitivity to hyperparameters.
- It is unclear what the authors want to say with Figure 3; what does it mean?
- The experiment of Section 4.3 seems meaningless, since the approaches with partial ideas such as “+width” or “+concatenation” are not reasonable at all. For example, “+concatenation” just repeats the original features multiple times. Such results do not give any insights on the proposed approach.

summary_of_the_review: This paper proposes a new graph augmentation method that enhances the performance of various GNN models by augmenting input features. However, the technical quality of this paper is insufficient, and the ideas, writings, and experiments are unclear and overwrapping. The idea of running variational inference for graph augmentation is intriguing, so I suggest the authors further develop their approach.
review_title: null

Meta-review: The paper presents a new algorithm for data augmentation in graph neural networks. The algorithm works by learning a conditional model of a node's neighbor features, and augment the neighborhood representation using the generative model.

 In response to the reviews, the authors provided long answers and clarified much of the text. Nonetheless, after the discussion, two main concerns remained. First, the presention still felt subpar, too notationally heavy for what was presented. Second, the gains with respect to the baselines were assessed as not sufficiently significant to justify the approach which is substantially more complex than a baseline such as GRAND.
Decision: Reject