Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an approach to applying the Entropy-based Logic Explained Network (E-LEN) to generate global explanations for a GNN from computed local explanations of the GNN, where both local and global explanations are expressed by sub-graphs. Experimental results on three datasets demonstrate good performance in terms of fidelity, accuracy and concept purity.
strength_and_weaknesses: Strengths:

(1) The work may have a significant impact since generating global explanations for GNNs is seldom studied by now. There seems only one notable approach to generating global explanations for GNNs, namely XGNN (Yuan et al., 2020), but this approach is based on reinforcement learning and thus is unstable and time consuming in course of training.

(2) Out of three experimental datasets, two are proposed by the paper for the first time. The authors promise to make them publicly available upon acceptance.

Weaknesses:

(1) The novelty is limited since the proposed approach seems to be only an application of E-LEN. I suggest the authors to strengthen the novelty for clarifying which enhancements are made beyond the framework of E-LEN.

(2) There is no comparison with XGNN in terms of the main metrics namely fidelity, accuracy and concept purity. In particular, the proposed approach seems inferior to XGNN on Mutagenicity in terms accuracy according to the accuracy reported in (Yuan et al., 2020).

(3) The presentation is generally good but can be improved by clarifying the following two questions.

Q1: How to convert a prototype vector (obtained in the concept projection step) to a subgraph (learnt concept) as shown in Figure 3?

Q2: Why not to use the same set of metrics provided in (Yuan et al., 2020) but introduce a new metric concept purity? Besides, concept purity has not been clearly defined in the paper; it should be defined in a more formal way, e.g., by using formulas.

clarity,_quality,_novelty_and_reproducibility: The quality of the paper is fine except a lack of comparison with XGNN in terms of main metrics and some presentation problems. The clarity is fair considering that two important questions mentioned above have not been answered in the paper. The originality is marginal since the main idea is merely an application of the E-LEN framework to generate global explanations for GNNs. There is no code or data provided in the supplemental material.
summary_of_the_review: The paper addresses an important yet insufficiently-studied problem by applying the E-LEN framework. New datasets are proposed to accelerate the on-going studies. However, the proposed approach lacks comparison with the state-of-the-arts in main metrics. Besides, the paper is unclear in some important aspects and need to be improved by clarifying them.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes GLGExplainer (Global Logic-based GNN Explainer), which is aimed at capturing the behavior of the model as a whole, abstracting individual noisy local explanations in favor of a single robust overview of the GNN model by generating explanations as arbitrary Boolean combinations of learned human-understandable graphical concepts. GLGExplainer uses PGExplainer to extract local explanations (as subgraphs) from graphs and learns an embedding for each local explanation that allows to cluster together functionally similar local explanations, and then projects each local explanation embedding into a set of m prototypes, which are initialized randomly from a uniform distribution and are learned along with the other parameters. The projection for each embedding is thus a vector containing a probabilistic assignment of local explanation embedding. Finally, a Logic Explainable Network learns to map a concept activation vector to a class while encouraging a sparse use of concepts that allows to reliably extract Boolean formulas emulating the network behavior. The authors validated GLGExplainer on both synthetic and real-world datasets, and claims that GLGExplainer is able to accurately summarize the behavior of the model to explain in terms of
concise logic formulas.
strength_and_weaknesses: - Strength
    - The paper is well written and organized, and readers can easily follow the content.
    - The paper organically aggregates a series of established methods, such as PGExplainer for extracting local explanation subgraphs, and concept or prototype based learning for human-understandable graphical concepts, and E-LEN for generating explanations with compositionality in form of Boolean combinations, in order to address an interesting and useful problem of capturing the behavior of the model as a whole.
- Weaknesses
    - There is only single baseline XGNN in the experiments. This is understandable, after all, the previous work on Global Explainer is limited. The table 2 only shows the performance of GLGExplainer, but does not show that of the baseline. 
    - Although this paper mainly aggregates a series of established methods, it would be helpful if some specific details were explained more clearly in self-contained way.
        - For example, the Equation 2 and E-LEN

clarity,_quality,_novelty_and_reproducibility: - Generally, Clarity, Quality and Novelty of the paper is good.
- Probably there will be better reproducibility when the source codes are available.

summary_of_the_review: GLGExplainer is the first Global Explainer for GNNs in terms of logic formulas, and it offers a helpful and human-understandable diagnostic tool for learned blackbox GNNs.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This practical paper proposes a method for computing global explanations for the predictions made by graph neural networks (GNNs). The approach hinges on the use of a local GNN explainer invoked for computing local explanations for the GNN's predictions in the form of sub-graphs of the inputs. Given a number of local explanations, the proposed approach then combines them into a single propositional formula serving as a global summary of the GNN's behavior. The experimental results obtained on three datasets demonstrate the faithfulness and accuracy as well as high interpretability of the proposed approach.
strength_and_weaknesses: First of all, I am not an expert on GNNs and their explainability and so I am afraid I am not the right person to judge the merits of this concrete work. For the same reason, I find this paper quite hard to follow.

This is a purely practical paper that describes how one can aggregate local explanations, provided in the form of sub-graphs of the input graphs, into a joint propositional formula that summarizes the behavior of the GNN model from the global perspective.

One of the upsides of the paper is that it is rigorous in terms of citations and seems to cover the prior work in the area quite extensively. It offers a large number of nicely drawn images illustrating the general flow and also examples in the experimental results, which is also a plus as they help a reader a lot.

Having said that, I have to mention that the proposed solution does not look like a sufficient contribution to me. Granted that this paper hinges on a significant amount of engineering work but the insight is a bit shallow. The fact that the approach is described in essentially a single page seems to confirm this issue. What is worse, the authors give no rationale behind the choices they propose to make. It is just a sequence of steps (1), (2), ..., followed by (n) and a non-expert reader like me is left wondering why these steps are better than the possible alternatives (if any). The steps themselves can be seen as a bunch of technicalities applied widely in the ML community and so the novelty seems a bit on the edge here (again, for a non-expert).

Experimental results are thoroughly presented and discuss the accuracy and fidelity of the computed explanations. However, these concepts do not look sufficient for measuring the "quality" of explanations, especially from the perspective of logic and formal correctness of those. I believe in this case a user study is necessary to confirm to what extend the explanations offered by this approach actually help a human understand the behavior of the model.
clarity,_quality,_novelty_and_reproducibility: Language-wise and quality-wise the paper is well-written. The experimental results are described up to a sufficient level of detail. The experimental setup and the tool are unavailable and are promised to be released upon paper acceptance. As mentioned above, the novelty of the paper seems limited while the proposed approach represents a recipe applying the known and existing technology with no clear rationale provided.
summary_of_the_review: In my view, the paper is under-delivering on the novelty side and represents purely practical/engineering effort. I doubt that this should meet the quality standards of a flagship AI/ML conference like ICLR. However, given the lack of my expertise, I admit that I may overlook some important merits of the work, hence my score is not entirely negative.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents a new method for explaining graph neural network predictions, where first local subgraph explanations are computed, then they are combined into a logical formula summarizing the GNN behavior. Reviewers appreciate how the paper brings together ideas to present a compelling approach to a less-studied regime of GNN explanation. Most of the reviewer concerns were addressed in the author responses. While the authors do address the issue in their response and it is reasonable, reviewers still wished that evaluation against SOTA could be made on the metrics from previous work. 
justification_for_why_not_higher_score: Would like to see higher originality and/or slightly cleaner experimental evaluation to recommend a spotlight.
justification_for_why_not_lower_score: Solid contribution with support for acceptance from all reviewers.
Decision: Accept: poster