Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Combining Graph Convolutional Networks (GCNs) with Graph Attention Trees (GATs), the authors propose a novel architecture for Graph Neural Networks, Learnable Convolutional Attention Layer (L-CAT).


In order to bridge the gap between the graph convolutional layer, the graph attention layer, and the graph convolutional attention layer, this paper introduces a new layer type for graph neural networks.


They add two scalar parameters per layer and learn them along with the other parameters in an end-to-end fashion. Inspiring theoretical conclusions based on prior work are presented, and the authors' suggested strategy is experimentally validated. 
strength_and_weaknesses: - The overall quality of the writing and organization of the work is quite high. 

- The background of the study and the significance of the contribution are laid out in great detail. 

- The second and third parts (preliminaries and limitations of GCN and GAT) offer an insightful summary of GNNs. 

- Findings from theory are addressed, and they are also confirmed by empirical research.

- In general, improving graph neural networks by interpolating across different layer types using parameters that can be learned in an end-to-end manner seems promising.

clarity,_quality,_novelty_and_reproducibility: As described in the above section, the motivation and the problem definition are clear.

Their approach is logical and seems to have sufficient applicability.
summary_of_the_review: I recommend to accept the paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes graph Convolutional Attention Networks (CAT) to exploit the advantages of Graph Convolutional Networks (GCNs) and Graph Attention Networks (GATs). Further, the authors design Learnable graph Convolutional Attention Networks (L-CAT), which softly selects GCNs, GATs, and CAT. The paper presents theoretical analysis of CAT and conduct node classification tasks using ten datasets.
strength_and_weaknesses: **Strengths**

(1) This paper shows the effectiveness of proposed methods with theoretical analysis. 

(2) The proposed CAT shows consistently good performance across datasets.

(3) Experiments on synthetic datasets prove the CAT’s effectiveness compared to GCNs and GATs. 

**Weakness** 

(1) The formulation GCNs is different from the formulation proposed in [1] on undirected graphs. Since the datasets used in experiments are undirected graph, it is more appropriate to use $\tilde{D}^{-1/2}\tilde{A}\tilde{D}^{-1/2}$.

************************Additional Questions************************

(1) I wonder the performance of GCN, GAT, and CAT when using more than 1 layer on the synthetic dataset.

---

[1] Fountoulakis, Kimon, et al. "Graph Attention Retrospective." arXiv 2022.
clarity,_quality,_novelty_and_reproducibility: **************Clarity**************

The paper is clearly written to understand overall method.

**************Quality**************

Some experiments for baselines are not properly conducted. 

**********Novelty**********

The proposed method has a fair novelty. 

******************************Reproducibility******************************

The paper has good reproducibility.
summary_of_the_review: Overall, I think this paper is marginally over the acceptance threshold. The paper demonstrates the effectiveness of their methods through theoretical analysis.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This manuscript focuses on the problem of learning better aggregation attention. The motivation comes from the observation that there is no clear winner of the existing three aggregation methods, including native GCN and GAT. The authors propose a new CAT aggregation method and merge it with GCN and GAT. The proposed method is evaluated on several standard datasets to show its effectiveness.
strength_and_weaknesses: Strength

1, The motivation that different data may need different aggregation makes sense.

2, The analysis of the evaluation of $\lambda_1$ and $\lambda_2$ is interesting.

Weaknesses

1, The comparison is insufficient. Only two methods are compared on most of the datasets: GCN and GAT. The weak comparison makes it hard to support the effectiveness of the proposed method.

2, The novelty is limited. The core idea of the proposed method is to integrate several different aggregation strategies. There are already several research works along this line like the sampling method proposed in [1]. 

3, In terms of technical detail, it seems that the proposed method is designed only for native GCN and GAT. It is hard to generalize the proposed method to other kinds of aggregations. It is more like another new aggregation scheme, which limits the contribution.

Other issues:

The highlights of Table.2 are improper.  E.g. the GCN performs the best on proteins while the L-CATv2 is highlighted.

[1] Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networks, ICCV 2021 
clarity,_quality,_novelty_and_reproducibility: Please see the above reviews. 
summary_of_the_review: In short, the main concerns of this manuscript are two parts: the insufficient evaluation and the limited novetly.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: __Summary.__ This paper investigates efficient aggregation/attention layers in GNN architectures. More specifically, to go beyond the usual dichotomy between aggregation (GCNs) and attention (GAT), they begin by introducing a  new type of layer, CAT (convolution attention layer), which computes an attention score after the usual aggregation step. They show the performance of this new type of "pooling" layer, showing its benefits on several datasets and highlight its theoretical competitive advantage in a Contextual SBM model. To forego the need to then "tune" the architecture  and select the "right" pooling mechanism, the authors propose to use a learnable convolution attention network that interpolates between GCN, GAT and their  layer, CAT. The weights ($\lambda_1$ and $\lambda_2$) are learned jointly with the parameters of the networks. They show that such a layer provides better performance of the GNN under higher levels of edge and feature noise.


__Feedback from the reviewers.__ Overall, the reviewers seem to agree that this is a good paper and that the approach has merit. The proposed mechanism seems to work, as highlighted by the extensive experiments proposed by the authors. The paper is nicely written --- with, we note, a nice theoretical justification of their proposed layer---, and the experiments are very complete (nice investigation of the robustness). Initial reviews raised objections regarding the limited evaluation methods (the authors had just used GCN as an architecture), but the authors have extended their evaluation and used l-CAT with various type of GNN architectures during the rebuttal period.  The contribution is therefore significant enough to warrant admission. 

justification_for_why_not_higher_score: The approach is nice and has significant merit to be accepted.
However, the approach is a simple (albeit noteworthy) extension of existing approaches.
justification_for_why_not_lower_score: The theoretical justification of l-CAT is rather elegant. The evaluation of the method on various benchmarks clearly highlight its merit.
Decision: Accept: poster