Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The main objectives of this paper are two-fold. 
First, the authors introduced a linearization framework for transforming various GNNs into a linearized form. Removing non-linearities should improve interpretability.
Second, the authors applied what they have learned about different GNNs (pain points and factors that distinguish different GNN variants) in designing of their own GNN-variant which is optimized for interpretability, robust to various labeling scenarios in graphs and has fewer parameters.
The authors presented their results together with sanity checks where they identified three groups of graph scenarios composed of combination of Edge types, Labels types and Feature types.
 
strength_and_weaknesses: This paper follows a strict logical path. The authors first describe their linearization framework. They proceed to describe lessons learned when applying linearization to previous GNN variants. The chief problem they listed have to do with the lack of explainability of GNN models with many layers. 
I was particularly impressed with the sanity checks proposed in this paper for evaluating robustness of the proposed method and for comparing it with previous methods.

clarity,_quality,_novelty_and_reproducibility: This paper may benefit from a discussion on why replacing nonlinearities with linear activation functions led to the improvement in accuracy. How the loss in model expressiveness due to removing nonlinearities can lead to the improved performance?
Can the authors mention any improvements upon their presented method that they can address in the future work?

summary_of_the_review: Overall, I found this paper to be well-written and well-motivated. I believe this paper addresses an important problem and provides an innovating solution.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper reviews the desiderata of GNN and proposes SlenderGNN, a simple linear GNN, to meet the needs. The proposed SlenderGNN comes from the linearization framework, which can resemble various GNN designs. Towards these desiderata, the authors present several sanity check, and the proposed SlenderGNN pass all of them. Moreover, SlenderGNN shows superiority over various GNNs on several real-world node-classification tasks.
strength_and_weaknesses: Strength:
1. This paper is well written, where all the definitions, declarations, and tables are self-contained.
2. The discussion of linearizing GNN is comprehensive and valuable for the community, which unifies many SOTA GNN works.
3. The performance of SlenderGNN is amazing, especially considering that outperforming those SOTA GNNs on such a variety of node classification tasks is very difficulty in nowadays.

Weaknesses:
1. The SlenderGNN is not a sophisticated instance among the linearization framework, that is to say, it has obvious limitations such as a restricted receptive field. In another word, it is not that novel to me.
2. As a linear GNN, the transformation $P$ could be interpreted from the perspective of feature engineering. Thus, it might be necessary to include several works in that line as baselines (e.g., graph kernel methods) in the experiments.
clarity,_quality,_novelty_and_reproducibility: Clarity: In all, the flow is good to me, where I can easily follow the whole story quickly. However, it seems that there is a lack of the explanation about the linear property of SlenderGNN.

Quality: The definitions are consistent with existing literature. The claims are correct imo, including the concrete linearization of existing GNNs elaborated on in the supplementary materials. The sanity checking is designed to accurately reflect the desired properties of a GNN. The experiments are conducted on a comprehensive collection of node classification tasks with various kinds of SOTA GNNs.

Novelty: The discussions of linearization is novel to me. However, the concrete design of the proposed SlenderGNN is straightforward and sacrifices several advantages of existing SOTA spectral GNNs, which raises my concerns about its novelty.

Reproducibility: It is convincing as the authors have provided the repository.
summary_of_the_review: I appreciate the discussion of what an ideal GNN is supposed to do, which makes the design of SlenderGNN well-motivated. Although the discussed linearization is somewhat straightforward, it well resembles various kinds of GNNs. I think it is helpful for the community to view existing GNNs from such a perspective. However, I have concerns about the novelty of SlenderGNN, which seems to be a trivial linearization. The consistent advantages achieved by SlenderGNN makes me eventually give a relatively positive score.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Authors propose the framework that tackles crucial aspects of modeling, which include accuracy, robustness, slenderness, and interpretability. To tackle this, authors propose the generic framework, named GNNLIN, that consists of the propagation layer and the logistic regression layer. Through the generically structured propagation layer, the proposed method can provide the interpretability. Furthermore, authors propose the sanity checking framework that verifies whether a given model algorithms works broadly for various scenarios of graphs and features. Finally, authors use the proposed sanity checks and real-world datasets to show the proposed algorithm outperforming the baseline methods.
strength_and_weaknesses: * Strengths
- A simple but generic interpretable model is proposed.
- The proposed model achieves many aspects all together.
- Sanity check framework is proposed and it would be great for other researchers to analyze their own models.

* Weaknesses
- The non-linear component is cleared for simplicity, but it is not clear whether the activation is not necessary or it can play a role to improve the performance more.
- In particular, the proposed method is just a logistic regression model that uses neighbor information, connectivity information, and node attributes as features. Eventually, the proposed model solves the iid logistic regression regardless of any latent space. Apparently, the proposed model does not seem to be a GNN. Authors need to modify the definition or the explanation of modeling, or regard the proposed model as the linear model using graph features. Robustness and interpretability can be achieved typically through such simple models.
- Many GNN models rely on sampling methods, while the proposed method is based on more matrix operation. It would be great if the proposed method can work with some sampled data or scale enough.

clarity,_quality,_novelty_and_reproducibility: - The manuscript is well-written for authors to follow easily.
- The model is well-organized, but it is actually a logistic regression with features. In that sense, the proposed model is not novel enough to this literature.
- Sanity checking framework is great to assess the model performance in various scenarios.
summary_of_the_review: The proposed framework is very intuitive and passes the sanity checking framework as well as outperforms the baseline methods. However, the model is essentially the logistic regression model using graph and node features, which are already being used commonly by industry. The novelty of the proposed model is limited in that sense.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposes SlenderGNN, a new GNN method that is accurate, robust, and interpretable. Authors report experimental results comparing SlenderGNN's performance with a set of alternative methods. They also conduct an ablated study to assess the importance of individual modules of the proposed method in its performance.
strength_and_weaknesses: Strength
- The paper addresses an important topic (i.e. interpretability and robustness of GNN).

Weaknesses
- Scattered and not well supported claims. The contributions from GNNLin and SlenderGNN are mixed in the paper and empirical results only provide partial evidence about the claims. Maybe it would have been more beneficial if authors had fully focused on SlenderGNN rather than including SlenderGNN related materials as well.
- Experimental results are not clearly discussed and it is not clear why LR shows comparable to superior results in most data sets vs. more state of the art and GNN-based approaches.
- Also refer to reviewer's comment on clarity, quality, and novelty in the box below.
clarity,_quality,_novelty_and_reproducibility: - Clarity:
In general, the way the proposed method works (section 4 of the paper) is not very clear. e.g. The connection between the components of the propagation function and GNNLin (Table 1) is not clear and the choices of 2-step neighbors, neighbors, etc. is not well justified in equation 3 (i.e. propagation function). 

- Quality
Interpretability and robustness are not well explained in the paper and they are not backed by experimental results either. e.g. there are quantification methods for interpretability/explainability in the literature that authors could have reported for their work. For example, one metric is to have a data set with explanation ground truth and measure the overlap between the decisioning factor identified by the proposed method vs. the ground truth explanation. Same point holds about robustness where authors claims are not backed by strong empirical evidence.

- Novelty
Novelty contributions are marginal.

- Reproducibility
Although authors have provided the source code for their method, due to lack of clarity explained earlier in this box, it's hard to reproduce authors work.
summary_of_the_review: The paper addresses an important question/concern with GNNs, however there are major limitations in connecting the experimental results with the paper's claims. Also theoretical grounds of the paper are not very clear.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a linearization framework for transforming various GNNs into a linearized form, and proposes a simple logistic regression model, SlenderGNN, for node classification.  The strong points include analyzing some pain points of existing GNNs, proposing a simple model combining interpretability, robustness to various labeling scenarios, and a small number of parameters. However, the weaknesses below outweigh the strengths:

Firstly, the proposed model boils down to a logistic regression over the concatenation of 4 handcrafted features, which is of limited technical depth. I believe in some specific scenarios it can outperform advanced GNNs but in general GNNs with multi-layer graph convolutions and nonlinearity will be much more powerful, both theoretically and empirically as shown in many other papers. The results in Table 3 are under a extremely sparse setting where 2.5%/2.5%/95% train/val/test split is used. This is different from the normal 60%/20%/20% setting widely adopted in the GPRGNN and other papers focusing on heterophily, but is never discussed. This raises the question whether the proposed model only works better on the sparse label setting. The setting of the two OGB datasets seem also different from the official one. If the setting is different from standard, it definitely needs a note.

Secondly, the paper claims it can automatically filter out those useless node/structure features, which explains why in the synthetic datasets the model works very well. However, those advanced GNN models can also handle such settings, given you simply use all-one node features as input (when node features are useless), or use fully-connected/node-isolated graph as input (when graph structure is useless). Both of these settings are easy to determine in practice, and being able to automatically adjust or adapt to these (rare) settings is not a significant contribution.

Thirdly, the paper emphasizes its interpretability, yet the only experiment presented in Figure 2 is an illustration of the logistic regression weights which not suprisingly, aligns with the data setting. It is expected to see more than this, e.g., an interpretability evaluation over human-annotated datasets like other interpretability papers do. Those papers can provide explanations for which part of the graph or which feature dimensions determine the prediction. 

Overall, I feel this paper still lacks a real solid contribution to the field. It is known that GNNs will fail if graph structure does not make sense, or node features are just noise. However, modern spectral GNNs could universally approximate any filter functions on a graph, thus theoretically can handle all different settings. The paper need to compare with those advanced spectral GNNs (besides GPRGNN) too.
justification_for_why_not_higher_score: This is a borderline paper. With reviewer-AC discussion, I am convinced that the paper does not meet the bar.
Decision: Reject