Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The work developed by the authors in the paper consists in a design of a novel convolutional operation to achieve learning on graph data. This convolutional operation lies at the crossroad of "isotropic" GCN and "anisotropic"/ attentional GNN to take the advantages of both operations without increasing the complexity of the operation.
In this study, appropriate datasets and baselines are introduced to support the theoretical motivation and the robustness of the proposed operation. 
strengths_and_weaknesses: This work is a novel combination of two well known convolutional operations on unstructured data.  The combination is well motivated / studied methodologically and empirically. It can bring a value for a better understanding of learning on graphs and might give insight for the design of more expressive convolutional operations on graphs. 
The authors studied theoretically the expressive powers of GCN and GAT while identifying their failures. Based on the latter, their combination is proposed in a straightforward manner while showing a gain in a performance without substantial increase of the complexity of GNN architectures. Moreover, their architecture is less sensitive to noisy data compared to GCN and GAT while being quasi-agnostic to the weights initialization which add significant value to the work. This direction could be interesting to explore further towards invariant architectures w.r.t weight initializations to make the networks more robust and get rid-off careful choice of initialization which is a tedious engineering task. 

The related work is well studied and the experiments are compared with appropriate GNN models. Moreover, the paper is well structured and easy to follow, including the consistency of math variables. 

I argue that the work is technically correct where the claims are well motivated theoretically and validated empirically with an appropriate experimental protocol. However, l am not sure to find some  mathematical parts developed  in the supplementary necessary to understand the contribution of the paper. Namely line 477 and 519.

One may wonder if the proposed framework can be generalized to any convolutional operations on graphs and to what extent it is possible or it is restricted to combining only GAT and GCN.  As a consequence, it's not clear if the proposed combination doesn't interfere with recent convolutional operations, including the seminal works on the design of convolutional operations from dynamical systems, partial differential equations and differential geometry standpoints. 
questions: 1/ Is the idea of combining GAT and GCN could be generalized to any pairs of convolutional operations on graphs ? or the proposed framework is restricted to these two convolutional operations

2/ Can you shed light on the fact that the proposed combination doesn't interfere with any recent convolutional operations design ?

3/ Could you compare (table 1, table 2) with recent convolutional operations ? it's a good starting that L-CAT outperforms GAT and GCN, however, one may wonder if it the case w.r.t to other convolutional operations.
limitations: The limitations are fairly discussed and the work does't have any societal concern.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this paper, the authors first investigate the expressiveness of convolutional-based and attention-based GNN architectures under the CSBM setting, and propose graph convolutional attention (CAT) which hinges the attention with convolution. Knowing there is no clear winner between attention and convolution, the authors propose a technique to interpolate in the middle of all the three mechanisms, called L-CAT.
strengths_and_weaknesses: Strengths:

+ The paper is clearly written, well formulated, and easy to follow.

+ Both theoretical and empirical analysis on the separate power of attention using CSBM seems interesting, though largely followed from the “Graph Attention Retrospective” paper.

+ The ideas of combining and interpolating between convolution and attention are novel proposals and well-motivated.

+ The empirical evaluation justifies the theory on CSBM, demonstrates performance on both real data and benchmark, and presents many insightful visualization and ablation studies e.g., the learning curve and distributions of tunable parameters, robustness to initialization and noises.

Weaknesses:

- The theoretical analysis is based on toy settings. And the theoretical significance is not claimed very clearly given the “Graph Attention Retrospective” paper. Also the separation power of convolution-based architecture still remains obscure without a formal argument.

- The performance on the benchmark is not as good as expected. GCN and GAT seem to be a special case of L-CAT. However, L-CAT fails to outperform GCN on proteins and mag data. Also, lots of strong baselines are missing.

questions: - In “Graph Attention Retrospective”, the authors considered two-class CSBM. Is there any special reason that this paper considers three classes?

- CAT computes the attention score based on the neighborhood information of the two nodes. Does this mean CAT actually cheats using second-hop information? And naturally improve the expressiveness? Would it be more fair to compare CAT with two-layer GAT or GCN?

limitations: The analysis in this paper is fundamental and provides good insights. However, the major limitation lies in the performance on the benchmark datasets. According to the theoretical results in this paper, L-CAT generalizes GCN and GAT and learns to trade off between these two architectures. However, the final performance on OGB shows even GCN outperforms L-CAT on two datasets. Moreover, current SOTA models faraway outperform GCN and GAT. Trading off between GCN and GAT seems not a key to improve the performance.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The authors propose a new graph neural network by combining existing graph convolution networks: GCNs and GAT. The authors proposed a new learnable convolutional attention layer (L-CAT), which subsumes the graph convolution in GCNs, the graph attention layer in GAT, and a normalized attention layer (CAT).
strengths_and_weaknesses: 1. A new layer that subsumes a couple of popular graph convolution layers has been proposed. Without changing layers in GNNs, by optimizing parameters in the proposed layer, the proposed graph neural network can change its behavior. 
2. This paper theoretically discussed the expressive power of GAT and the proposed layer CAT with a contextual stochastic block model (CSBM). 
questions: 1. The experiments are somewhat obvious. The proposed method subsumes GCN, GAT, CAT. Then L-CAT must be on par with the models or outperform them.
2. In Table 1, L-CAT and CAT are comparable. The performance gain is marginal compared to GAT. Unless I missed something, both GCN and GAT are baselines of the proposed method.
3. In the literature, more powerful GNNs have been recently studied. Why do the authors compare their method with GCN?
limitations: 1.  The proposed method is a simple generalized graph neural network that subsumes GCN and GAT layers. The simplicity of the proposed method is great as long as it has practical merits. However, the performance gain by the proposed method is marginal. In addition, the authors did not compare the proposed method with more recent works that exhibit stronger performance.
2. The proposed method is incremental, and the lack of generality of the proposed method makes the impact of this paper questionable.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This submissions has two parts of contribution. 

First, it proposes a novel attention mechanism for graph neural networks. The proposed technique first aggregates neighborhood features (as done in GCN) before computing the attention weights. This new method is named CAT. By a theoretical, analysis, the authors show that CAT can be more effective than GAT but their comparison depends on the graph topology. 

Second, for a given layer, the submission proposes to learn two additional scalar weights that interpolates between GCN , GAT, the newly proposed CAT. The intuition here is that the network can discover the correct network type to use depends on the input data. 

Overall, I think the proposed method is novel and the theoretical analysis provides an interesting perspective. The empirical analysis shows that the proposed methods are more robust to data noise and weight initialization scheme. However, in the standard setting, the proposed method is mostly on-par with GCN (the comparison is not consistent across datasets). To take this empirical weakness into account, I am giving a weak accept.
strengths_and_weaknesses: Strength

1. The proposed method is novel
2. The proposed method is motivated by a theoretical analysis. I appreciate the effort to understand the proposed method.
3. On the empirical side, the proposed method is more robust to data noise and different initialize schemes
4. The writing is very clear

Weakness

1. For the standard benchmarks, the proposed method is only on par with GCN, which is a simpler method. 
2. I provide more requests and suggestions for improving the current draft in the questions section below.
questions: Questions:

1. As the authors acknowledged, many new variants of graph neural networks have been proposed. Can the authors include more baselines into comparison to contextualize the results better? I am especially interested in knowing how other baselines would react to the added feature noises. 
2. Can the theretical analysis  also be applied to L-cat? Similarly, how would L-CAT behave on the synthetic data in Figure 1?
3. Another form of data noise is corruption to the graph adjacency matrix. Does L-CAT also handle this form of corruption better?
limitations: I do not see any negative societal impact.
review_title: null

Meta-review: metareview: The paper explores the advantages of both GCN and GAT by proposing a learnable network that can interpolate between GCN, GAT and CAT for each layer automatically. The proposed research idea is novel and discovers an interesting perspective by combining and interpolating between convolution and attention networks. The paper is theoretically sound, and extensive experiments are conducted over 15 datasets with a comprehensive analysis. 

However, all the reviewers consistently raise concerns regarding incremental improvement compared with baselines, and another common concern is that the authors do not extend the proposed method with more advanced convolutional and attention networks. The authors argue that their intuition is to design a more robust replacement to GCN/GAT, not a SOTA. However, the author should be aware that L-CAT is able to extend to other networks does not guarantee it will work, and it is possible that the convolution and attention network may conflict during the training. Since the proposed method is a novel and general paradigm, solid experiments are needed to thoroughly evaluate its performance. The motivation is promising, but more experiments should be conducted to sufficiently prove the superiority of the proposed method. 
confidence: Less certain
Decision: Reject