Kati8/ICLR_2023_2025 huggingface dataset:
authors: Salvatore Calderaro
authors: Domenico Amato
authors: Giosue' Lo Bosco
authors: Riccardo Rizzo
authors: Filippo Vella
reviews: 
Comments: This paper introduces a post-hoc explainability method for Graph Neural Networks (GNNs), specifically for the task of graph classification. The authors propose that communities within a graph represent the most crucial subgraphs for understanding the GNN's predictions. By identifying each community and assigning it a score, they aim to pinpoint the most significant subgraph influencing the GNN's decision. The method leverages a well-known community detection algorithm from graph theory to identify these communities. The authors demonstrate the effectiveness of their approach through experiments on various synthetic and real-world datasets.
Strengths: - The problem of explainability in Machine Learning is important and highly relevant.
- The proposed method is simple and straightforward.
- The paper is  easy to read.
Weaknesses: - The authors did not mention or compare their work with the most recent studies in this area (e.g., [1], [2]).
- A significant drawback of this method is its focus on communities. It is possible that only certain nodes within a community are important, but this approach may fail to identify those specific nodes.
- The authors state, "The algorithm is based on the hypothesis that a GNN learns to recognize specific structures in the input graph" and claim that communities are these specific structures. However, they did not formally prove that this is indeed how GNNs learn, nor did they elaborate on or provide evidence for their claims.
- An ablation study using different community detection algorithms is necessary to justify the choice of the one used in their method.
- In Section 3.3, the explanation of datasets is too lengthy and could be partially moved to the appendix. Conversely, the method section is too brief and lacks formal mathematical details.
- The font size in Figure 1 is too small, making it difficult to read.
- The authors should consider incorporating sparsity as a metric and demonstrate how fidelity changes with different subgraph sizes, as the size of the subgraph plays an important role in the quality of the explanation.

[1] Yaochen Xie, Sumeet Katariya, Xianfeng Tang, Edward Huang, Nikhil Rao, Karthik Subbian, and Shuiwang Ji. Task-agnostic graph explanations.
[2] Jialin Chen, Rex Ying. TempME: Towards the Explainability of Temporal Graph Neural Networks via Motif Discovery
Recommendation: 3
Reviewer confidence: 5
Soundness/correctness: 2
Clarity: 2
reviews: 
Comments: The paper introduces a method to improve the explainability of GNNs by identifying key communities that contribute to model predictions. GECo uses community detection to isolate important subgraphs, evaluates their impact on the GNN's output, and selects those with high predictive value for explanations.
Strengths: 1. The paper is well-organized and clearly written.

2. This paper introduces a novel community-based method for explaining GNNs, focusing on identifying key subgraphs rather than just individual nodes or edges.
Weaknesses: 1. GECo uses Blondel et al.'s modularity optimization algorithm for community detection, which performs well on large sparse matrices. However, it does not discuss how different community detection algorithms might impact the explanation results, leading to a lack of robustness verification.

2. GECo determines the threshold 𝜏 by calculating the probability values of communities, using the mean or median as the threshold. However, this method may not be suitable for all cases, especially when the graph structure is uneven or community sizes vary. It is recommended to add experiments exploring adaptive adjustments of 𝜏 in different situations.

3. The baselines compared with GECo are not the latest methods. It would be useful to compare with some instance-level explanation models from the past two years.

4. The paper uses fidelity-based metrics for evaluation. However, these metrics have limitations due to the OOD problem. Therefore, new metrics Fid_{α1,+} and Fid_{α2,-} [1] could be added to assess model fidelity.
[1] Zheng, X., Shirani, F., Wang, T., Cheng, W., Chen, Z.,Chen, H., Wei, H., and Luo, D. Towards robust fidelity for evaluating explainability of graph neural networks. In ICLR, 2024.
Recommendation: 3
Reviewer confidence: 4
Soundness/correctness: 2
Clarity: 3
reviews: 
Comments: This paper presents GECo, a novel explainability method for Graph Neural Networks (GNNs). GECo focuses on identifying key subgraphs, or communities, that significantly contribute to the model’s classification outcomes. It operates by detecting communities within a graph, then evaluating each subgraph’s individual contribution to the prediction. By setting a probability threshold, GECo isolates the most impactful communities, offering a clear interpretation of the classification results. Tested on synthetic and real datasets, GECo demonstrates superior performance compared to other explainability methods like PGExplainer and GNNExplainer.
Strengths: 1. GECo enhances the interpretability of GNNs by identifying the most influential subgraphs, providing a more intuitive and targeted explanation for classification results.

2. GECo achieves significant explainability performance demonstrating effectiveness across both synthetic and real-world datasets.
Weaknesses: 1. This paper does not adequately highlight the advantages it has over other explainability models. Specifically, it should analyze the limitations of existing explainability models mentioned in the Related Work section (such as GNNExplainer, PGExplainer, SubgraphX, and PGMExplainer) and convincingly argue the advantages and necessity of a community-based approach for explainability, based on these limitations.

2. The approach in this study is straightforward and lacks novelty. In particular, using communities to generate explanations is an already known method [1]. Additionally, the overall methodology is very similar to studies that recognize motifs and predict their importance to produce explanations [2].

- [1] Martínez Mora, Andrés, et al. "Community-aware explanations in knowledge graphs with XP-GNN." bioRxiv (2024): 2024-01.
- [2] Chen, Jialin, and Rex Ying. "Tempme: Towards the explainability of temporal graph neural networks via motif discovery." Advances in Neural Information Processing Systems 36 (2023): 29005-29028.

3. The proposed model explanation (Section 3.2) is less than one page in length, while the experimental section occupies most of the paper. However, the experimental settings largely replicate those of other studies, leaving little in terms of new insights.

4. Experiments related to runtime performance are necessary.
Recommendation: 1
Reviewer confidence: 5
Soundness/correctness: 1
Clarity: 2