Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper deals with building robust graph learning methods not susceptible to adversarial attacks. Existing methods learn the structure of graph either by preprocessing the graph structure or by parametrically learning the graph adjacency. In this paper, the authors propose to simultaneously learn the graph structure as well as to use it for message passing iteratively. They call it message passing with adaptive structure (ASMP), which is shown as a way of doing proximal gradient descent for simultaneous denoising of graph signal and graph structure. Experimental results are provided to show the usefulness of the approach.
strength_and_weaknesses: ### Strengths:
1. The paper is well-motivated to handle the noise in graph structure with simultaneous learning of graph structure with features.
1. The idea of adaptively learning the graph structure iteratively is appealing.
1. Experimental results show that the method is promising. However, improvements on most datasets is only marginal.

### Weaknesses:
1. I do not understand the usefulness of having ‘different structure’ in each layer. More plausible to me is that the with adaptive graph structure, it gets progressively better and not just different. 
1. Why do authors use $L_{rw}$ and not $L_{sym}$. Is there any reason which makes it better? Experimental analysis to study the difference would be helpful as well.
1. The paper can also be strengthened with further ablations. What are the learnt values of ASGNN coefficients which give the best results? This would be helpful to see the terms affecting the performance.

clarity,_quality,_novelty_and_reproducibility: The paper is well written. I do not know much of the related work. However, it seems the main idea is novel and well-developed.
summary_of_the_review: The overall idea of adaptively learning graph structure with message passing seems nice and well-developed. However, I am not fully aware of the related works and hence, not clear on the degree of novelty and significance of the results. Hence, I'm less confident. Therefore, I'm going with weak acceptance.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The major contribution is the message passing scheme with the adaptive structure that enables learning different propagation structures for different GNN layers. The proposed method ASGNN is one of the graph structure learning methods and the improvement of this work in terms of motivation is marginal. 

strength_and_weaknesses: Strength:
1. It’s a generalized framework, achieving conduct message passing over different graph structures at different layers.
2. In most experiments, it achieves the best results. 
3. The proposed update scheme has good theoretical support.

Weaknesses:
1. The authors claim that the message passing scheme of ASGNN is ‘interpretable’. But relevant experimental results are not given. 
2. This paper lacks a complexity analysis of the proposed model, considering the proposed ASGNN is trained to learn different propagation structures on each GNN layer.
3. The challenges and innovation of introducing the adaptive structure are not clear.


clarity,_quality,_novelty_and_reproducibility: 1. Limited innovation.
2. The code is not open, and some setting of hyperparemeters like $\mu_1$, $\mu_2$ is unknown.

summary_of_the_review: The motivation of this work is not quite novel. And the adaptive structure assumption increases the complexity of the proposed model. In a word, the proposed ASGNN brings performance improvements, but the complexity of this model also makes it hard to apply to large-scale graph data.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose ASGNN, a robust GNN architecture. ASGNN learns a "cleaned" adjacency matrix as well as feature matrix per message passing layer with the goal of removing adversarial perturbations from the graph. The message passing procedure is framed as an optimization problem, and the individual message passing steps are proximal gradient steps. The authors present experimental results on well-known datasets from the literature.
strength_and_weaknesses: Strengths:
* Framing the structure learning as an optimization approximately solved via the message-passing procedure itself appears novel and is interesting
* The presented results suggest that the method is effective at defending against existing attack methods.
* The presentation of the work is clear and easy to follow.


Weaknesses:
* The authors do not evaluate their defense against adaptive attacks designed to circumvent their defense
* The comparison is missing an important baseline with GNNGuard [Zhang and Zitnik, 2020].
* The authors only evaluate their method on small datasets.
clarity,_quality,_novelty_and_reproducibility: **Clarity**: very good and easy to follow.

**Quality**: There are some important weaknesses of the paper and the experiments:
* The authors do not compare to GNNGuard [Zhang and Zitnik, 2020], which is an important baseline.
* The authors do not evaluate their defense on adaptive attacks, i.e., attacks designed to overcome the specific defense the authors propose. This is very important, as it is always easier to defend against existing attacks, but often an attacker can break novel defenses by adapting the strategy. This is, of course, different from certifiable defenses where we get a mathematical guarantee of robustness.
* There is no information about the computational complexity of the method and/ or runtime analysis. Given that the authors only report on small datasets, I expect the method to be quite computationally expensive.
* From the description in the text it is unclear whether the authors repeated all experiments ten times on one single split of the data or whether they evaluate on ten different splits. If they only evaluate on a single split, this is a problem because GNN performance has been shown to be highly variable depending on the individual split [Shchur et al., 2018].
* It would be interesting to see some qualitative insight into what kind of graphs the method learns. Does it remove adversarial edges? How sparse are the resulting graphs?
* There are no hyperparameter sensitivity of ablation study results. There are five terms in the loss function: are all of them required, and how sensitive is the model to different choices of hyperparameters?

**Novelty**: While the loss function is very similar to the one in Pro GNN [Jin et al. 2020] and the idea of framing message passing as gradient steps is not new, their combination is novel, interesting and non-trivial.

**Reproducibility**: Okay; while the description of the method is sufficiently clear to implement it, in order to exactly reproduce the results some information is missing: the authors did not provide an implementation of their method, did not provide details which of the hyperparameters in their search space was selected in the end, it is unclear which exact split of the data was used, and the hyperparameters of the MLP of the method are not provided.


References:
Shchur, O., Mumme, M., Bojchevski, A., & Günnemann, S. (2018). Pitfalls of graph neural network evaluation. Relational Representation Learning Workshop, NeurIPS 2018.
summary_of_the_review: The paper is a solid piece of work developing a robust GNN architecture. The presentation is clear and easy to follow. There are some key limitations of the work that prevent me from recommending acceptance (see details above):
* Missing the GNNGuard baseline
* Lack of adaptive attacks.
* Only reports results on small datasets, no runtime/ complexity information or ablation study results provided.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The authors tackle robustness of graph neural networks. They propose a novel model which optimizes the adjacency/feature matrix with the aim to remove adversarial changes. From a technical site, the message passing scheme is formulated as an optimization problem which allows the authors to introduce the adaptive graph structure in a principles way. This, indeed, is a nice technical contribution of the paper. The experiments show good performance of the method.

As also highlighted by some other reviewers, the high complexity (quadratic in the number of nodes) might not be practical for real applications -- accordingly, the authors test also only rather small datasets. Moreover, adapting the graph structure to improve robustness has also been explored by multiple other works, thus, limiting a bit the novelty.

Overall, the paper presents a nice solution to an important problem. Weighing pros and cons, it remains a borderline case. Considering the overall set of papers in my batch, I provide 'reject' at the moment, but the decision can be improved to accept as poster as well.
justification_for_why_not_higher_score: As described above, an accept as poster would be justifiable as well.
Decision: Reject