On Logic-based Self-Explainable Graph Neural Networks
Authors: Alessio Ragno
Marc Plantevit
Céline Robardet
Venue: NeurIPS 2025 poster ()
Keywords: graph neural networks
explainable ai
self-explainable gnns
interpretability
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Graphs are complex, non-Euclidean structures that require specialized models, such as Graph Neural Networks (GNNs), Graph Transformers, or kernel-based approaches, to effectively capture their relational patterns. This inherent complexity makes explaining GNNs decisions particularly challenging. Most existing explainable AI (XAI) methods for GNNs focus on identifying influential nodes or extracting subgraphs that highlight relevant motifs. However, these approaches often fall short of clarifying how such elements contribute to the final prediction. To overcome this limitation, logic-based explanations aim to derive explicit logical rules that reflect the model's decision-making process. Current logic-based methods are limited to post-hoc analyzes and are predominantly applied to graph classification, leaving a significant gap in intrinsically explainable GNN architectures. In this paper, we explore the potential of integrating logic reasoning directly into graph learning. We introduce LogiX-GIN, a novel, self-explainable GNN architecture that incorporates logic layers to produce interpretable logical rules as part of the learning process. Unlike post-hoc methods, LogiX-GIN provides faithful, transparent, and inherently interpretable explanations aligned with the model's internal computations. We evaluate LogiX-GIN across several graph-based tasks and show that it achieves competitive predictive performance while delivering clear, logic-based insights into its decision-making process.

On Logic-based Self-Explainable Graph Neural Networks
Alessio Ragno INSA Lyon, CNRS, LIRIS UMR 5205, F-69621 Villeurbanne, France alessio. ragnorinaa- fr
Mare Plantevit EPITA Research Laboratory (LRE), F-94276, Le Kremlin- France marc. teeatevit@epiiie fa.
Céline Robardet INSA Lyon, CNRS, LIRIS UMR 5205, F-69621 Villeurbanne, France celine ror..rrroordeetins--yyyoo....
Abstract
Graphs are complex, non-Euclidean structures that require specialized models, such Graph Neural Networks (GNNs), Graph Transformers, or kernel-based approaches, to effectively capture their relational patterns. This inherent com- plexity makes explaining GNNs decisions particularly challenging. Most existing explainable AI (XAI) methods for GNNs focus on identifying influential nodes extracting subgraphs that highlight relevant motifs. However, these approaches often fall short clarifying how such elements contribute to the final prediction. To overcome this limitation, logic-based explanations aim derive explicit log- ical rules that reflect the model's s decision-making process. Current logic-bases methods are limited to post-hoc analyzes and are predominantly applied graph classification, leaving significant gap in intrinsically explainable GNN architec- tures. In this paper, explore the potential of integrating logic reasoning directly into graph learning. We introduce LogiX-GIN, a novel, self- self-explainable GNN architecture that incorporates logic layers to produce interpretable logical rules part of the learning process. Unlike post-hoc methods, LogiX- --GN provides faithful, transparent, and inherently interpretable explanations aligned with the model's 's internal computations. We evaluate LogiX-GIN across several graph-based tasks and show that it achieves competitive predictive performance while delivering clear, logic-basee based insights into its Decision-making process.
Introduction
Graph Neural Networks (GNNs) are powerful models for learning from graph-structured data, excelling in tasks like node classification and graph classification (25 29]. However, their complex bassage-passing mechanisms make it difficult interpret how individual graph components influence predictions, limiting their use in critical domains that require transparency.
Several Explainable AI (XAI) techniques have been proposed to address the opacity of GNNs, typically falling into post-hoc and ante-hoc approaches. Post-hoc methods aim to explain an already trained black-box GNN by identifying influential graph components (e.g., nodes, edges, sub- graphs) that contribute to a specific decision [31 20 15] These methods include perturbation- approaches, gradient-based attribution, and concept-based methods that aim extract meaningful graph patterns. Although effective, they often lack fidelity to the internal reasoning of the model, as they hardly capture the internal process of the model and can generate inconsistent or misleading
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

explanations [24]. Ante-hoc, or self self-xxplainable models, on the other hand, integrate interpretability into the learning process. Self-explainable GNNss (SE-GNNss) aim to ensure that the learned represen- tations are inherently interpretable, often by enforcing constraints on the structure of explanations or by explicitly modeling feature importance. However, existing SE-GNNs present some limitations: copology-based approaches [17] & [2 (3, 21 identify the most relevant subgraphs while providing little insight into the logical rules underlying how the model represents graphs and makes its decisions
18 [9] ; symbolic approaches propose distillation of the models into simpler more interpretable architectures such as decision trees; and other approaches restrict the focus on node features, without using information about substructures [ B
To address these limitations, logic-based XAI methods provide an alternative approach by extracting numan-interpretable logical rules that govern the model's 's decision decision-making process (6). Unlike gradient-based or perturbation-baser explanations, logic rules explicitly define how features interact to produce given outcome. When applied to GNNs, these approaches aim to map high-dimensional feature spaces Boolean expressions that describe the presence of specific subgraph motifs or node interactions 22] [. However, currently existing logic-based GNN explainers are exclusively post-hoc or based on distillation of the GNNs using decision trees [9].
Recent advances in logic-explaiedd networks for tabular data provide potential directions. Logic Explained Networks (LENs) 6 and Transparent Explainable Logic Layers (TELL) 23 propose architectures convertible into logical rules. Particularly relevant is TELL's approach, where non- negative weight constraints allow direct extraction of first-order logic rules from the model parameters. However, these architectures were designed specifically for tabular data and cannot handle the complex relational structure of graphs. To address the lack of self-explainable logic GNN, in this work, we design novel architecture that is intrinsically convertible into logic rules.
The theoretical foundation for connecting GNNs with formal logic is established by Barceló et al. [4. who demonstrate that some GNN architectures can represent statements in graded modal logic (GML) [T]. GML extends standard modal logic with numerical constraints, allowing expressions like "at least 3 neighboring nodes satisfy property P". a natural fit for graph-based reasoning, While Barceló et al. [4] prove that GNNs can express GML propositions, does not address the reverse problem: architecting GNNs to explicitly yield interpretable GML rules.
Constraining GNNs to adhere to logic comes with different challenges. To implement our model, LogiX-GIN, we modify the GIN architecture [30], such that it is possible to extract explanations in the form of GML. The choice of the GIN layer is dictated by its aggregation scheme, which uses the sum over neighboring nodes. This aggregation allows us to respect the counting functionality of GML. However, in order count over input nodes, we cannot use simple threshold as is done in TELL. For this reason, we introduce a preprocessing function that binarizes the input features depending on learnable intervals. An additional challenge comes from the use of sigmoid activations directly on the weights, as proposed in TELL. These tend to hinder gradient flow in multi-layer settings. This limits the effective training of models with many layers, as early layers receive little to no gradient updates. For this reason, we propose to first train black-box model, GIN, and use it to pretrain our LogiX- model. This allows for pretraining each layer separately and therefore avoids problems such as vanishing gradients.
Overall, the main contributions of our work are as follows: we propose LogiX-GIN, the first graph neural network that integrates logical reasoning directly into its architecture; we introduce a novel preprocessing function for automatic binarization of input features using intervals; we propose the first implementation of multi layer logic-based neural architecture by applying distillation based pretraining strategy to address vanishing gradient issues; we conduct extensive experiments assess the interpretability, predictive performance, and limitations of LogiX- we provide an open-source implementation of our proposed approach and the conducted experiments!
The remainder of this work is structured as follows: we begin by surveying the current literature on XAI for GNNs, self--xplainable models and logic-based XAI; subsequently, detail the mathemati- cal foundations needed to comprehend our approach; we then present LogiX-GIN; we follow with an experimental evaluation on several datasets; finally, we summarize our findings and suggest potential directions for future research.
PPublic GitHub repository:
2

---

2 Related Work
In this section, we present the related literature to our work. We start by introducing self-explainable models for GNNs, and we then continue focusing on approaches that deliver explanations the form of logic. Finally, we focus approaches aimed at delivering logic explanations for GNNs.
GNNs. SE-GNNs are designed to generate intrinsic explanations during infer- ence, removing the need for post-hoe interpretation methods. central approach in this paradigm involves the use of prototypes, intended as representative training instances, to construct inherently interpretable models. Prototype-based SE-GNNs adapt prototypical mechanisms to the graph domain, where the output class is determined using the similarity between the input and prototypes [36]]]]]]]]]]]] 22 21]. Despite the interpretability of the case-based approach, these models rely on embedding similarity, which isn't ' guaranteed to actually reflect structural similarity. Other approaches leverage information bottleneck methods to identify maximally informative subgraphs, ensuring that the extracted explana- tions retain only the most relevant structural components. For example, GIB [22 and VGIB [33 utilize informationntheorrele predictive performance. Kernel-based approaches such KerGNNs [3] take different route by employing graph kernels to provide explanations through kernel activations, allowing for a structured decomposition of the learned representations.
principles to compress node and edge representations while preserving
Logic-based XAI Logic-based explainability methods [9] aim to improve model interpretability by incorporating logical constraints or symbolic reasoning directly into the learning process. Traditional machine learning models, like decision trees and rule-based classifiers, inherently provide logical explanations by explicitly representing decision boundaries. Although these models are transparent, they typically struggle with complex, high-dimensional data and cannot be easily integrated into deep learning frameworks due to their non-differentiabbiity. which prevents the use of gradient-based optimization. This limitation has led to the development of neural architectures that incorporate logical reasoning while remaining compatible with backpropagation. A prominent example is represented by LENS (). class of neural networks that can be interpreted through logical rules. LENs are built on binary inputs and employ strong regularization, allowing the construction of truth tables from which logical rules can be derived. However, due to the post-hoc truth table procedure, these rules may not always faithfully represent the model's 's true behavior (23). To overcome this issue, Ragno et al. (23 introduce TELL. TELL is an architecture with resttie-weight constraints that ensure direct and faithful translation into logical rules. Furthermore, TELL is capable of handling continuous input data through a preprocessing function that automatically learns thresholds over features.
Logic-based Explanations for GNNs. Among the first logic-based approaches for GNNs, GLGEx- plainer 22 proposes to extract logical rules over explanations generated by another post-hoe method, such as PGExplainer [15]. This is achieved through the application of LENs over concept activation vector that encode the presence of specific subgraphs. While innovative, this approach relies on the application of an instance-level post-hoc procedure and on LENs, that were demonstrated to not be properly faithful [3 D. With GraphTrail, Armgaan et al. 1 propose to overcome these issues by using computation trees to identify relevant subgraphs and avoiding the need of post-hoc methods. These approaches primarily aim to identify rules over subgraphs in order to gain insight into the model's is behavior. Taking different direction, Pluska et al. 19 propose distilling symbolic model from a GNN using iterative decision trees. Despite this difference, their method is also post-hoc, as the rules are extracted from graph embeddings. Additionally, the final model is no longer GNN and cannot be reused in other neural network settings, thus losing the advantages of neural representations.
Although we share with the aforementioned works the goal of incorporating logic- reasoning, our approach differs from current literature a key aspect: rather than extracting rules in a post-hoc fashion, we aim to directly design GNN model that is self- reff-interpretable by construction. For this reason, our case, distillation is used solely as a means to pre-initialize the self- reff-interpretable model, not as end in itself.
3

---

3 Background
We now introduce the key concepts and notations underlying our approach, focusing how each element supports our logic-based self-explainable GNN. We start by formalizing GNNs, the base model we build upon. We then show how architectural constraints can enable direct extraction of logic rules from neural layers, introducing the core ideas of TELL. Finally, we present the mathematical foundations of GML, the formal system use express the logic rules produced by our model.
Graph Neural Networks. GNNs are neural networks designed for graph-structun data. They generally operate via message passing, where at each layer a node aggregates messages from its neighbors and updates its representation. Formally, this process can be written as:
h = (k(k) (hg 1=) (,())) (I(t"")=EN(o))))
(1)
1, Here, (/ denotes the updated representation of node v at layer kt using the ones at layer 2((=") In the first layer, f ((==1) directly correspond to node features. The message function (a aggregates information from the neighborhood, and the update function σi(k) combines it with the
node's current state. Since nodes in a graph have inherent ordering, the aggregation function must be permutation-invariian.
We use GIN 30 model, where the aggregation function is the sum, and the update function is multi-layer perceptron (MLP) as this combination has been theoretically shown to achieve maximum discriminative power among message- message-passing GNNs. The aggregation can distinguish between different neighborhood structures and the MIP update function provides the necessary expressive capacity obtain meaningful representations, enabling GIN to approximate the 1-WL (Weisfeiler- Lehman) graph isomorphism test, which is a strong baseline for distinguishing non-isomorphic graphs. In this architecture, each layer computes:
1 MIP(2) (1+ (1+600)100
Σ (EN(2)
h((t1))
with c(k) learnable parameter. Multiple layers are stacked capture increasingly abstract features. For node-level tasks, the resulting node embeddings are directly used. For graph-level tasks, readout function (e.g., sum, mean, or max) aggregates node embeddings into a global representation for classification.
Transparent Explainable Logic Layer. TELL is a neural component designed to be interpretable through a direct conversion into first-order logic formulas. It constrains a standard linear layer, with T input neurons and output ones, by enforcing non-negative weights and applying a threshold-based activation. Given an input X R it computes the output y = (XW* + b) E RO, W+ x is a weight matrix constrained to have only non-negative entries, b E o is bias term, and a is the sigmoid activation function.
key feature of TELL is that allows to be interpreted through logic rules. Considering the j-th binarized output y
[i 1o0])-0.5 it then be associated with rule in disjunctive normal form + Exes W+ [ > -bi]. S denotes the minimal subsets w.n.t inclusion
(DNF): y hin [j] of the input sufficient activate the neuron yiu [23). Following this, E, VsEs, NEs Hy can be defined as the logic rule for y[i], where S, contains all the minimal subsets of inputs that satisfy the threshold -b]j]. This ensures a direct and interpretable mapping from network parameters to logical expressions. To extend TELL to handle continuous inputs, a preprocessing binarization step introduces an adaptive thresholding mechanism. This is done using X o @ exp(W) + δ), with @ representing element-wise multiplication, and W E R and E / being learnable parameters. Each transformed feature can then be interpreted as binary predicate where 22[i] 1
= kg
These binary features can be incorporated into logical expressions, preserving TELL's explainability while enabling it to process real-valued data effectively.

---

The set of Graded Modal Logic. GML is a fragment of first-order logic (FO) designed to express local properties of nodes in relational structures such as graphs, with support for counting. It restricts quantification to the immediate neighborhood of node and ensures that formulas are evaluated locally, which makes it well-suited for graph-structured data.
Let us consider a first-order language that includes symbol (xx y). for a binary relation between representing adjacency between nodes æ and y of an undirected graph, and finite collection of unary predicate symbols (z). Pr(x) of size I, which represent properties, features or types associated with individual nodes.
graph structure = (V, (Piher)) consists of: a finite set nodes V: symmetric edge relation E S V X V; and, a collection of unary relations P. S V interpreting each predicate symbol.
Formulas of GMI. are a syntactic restriction of FO with exactly one free variable a. formulas (xx) is defined inductively by:
(ax)) : P(x)) (+))) ti (x) A (2(1) (xr) V v2(21) 132 1320 A)
where E No and (xr), v/ (Ir), (2())) the (y) are GMI. formulas.
Given graph G and node the relation G, } ((() that indicates that E G satisfies generic GML formula (px) is defined as follows:
G.v = P(z): node v satisfies the atomic predicate P (i.c., e belongs to the unary predicate
P).
G,u == (z)): node does not satisfy the formula (xx) (negation),
G,v -= (x) A (z)): node satisfies both subformulas (1() and (12( (logical
conjunction),
G,v F= (x) V (12()):: node v satisfies least one of the subformulas & (z) (2) (logical disjunction),
G, 92N y (F(a,y) v) A (1)(:)): node o has at least N neighbors a such that (v, u) E E and each и satisfies tha (1) (quantified neighborhood condition).
distinctive feature of GML that all quantification is conditioned by the adjacency predicate F(x, y). This ensures that the evaluation of any formula at node depends only on and its neighborhood, not on the entire graph. Consequently, the logic is strietly less expressive than full FO, but computationally more tractable and naturally suited for expressing local patterns in graphs. For concrete examples of GML formulas, we refer the reader to Appendix B The correspondence between GML and local properties of graphs plays central role in recent work by Barceló et al. 44, where the authors show that precisely characterizes the class of node properties computable by certain class of graph neural networks.
4 LogiX-GIN
In this section, we formalize LogiX-GIN. We start by defining the different components of the architecture and then present implementation and training details.
LogiX-GIN components. Like the original GIN architecture, LogiX-GIN performs aggregates embedding values through the sum, then learnable parameterized function binarizes the aggregated values order to apply a logic transformation À that responsible to learn logic rules (as the one in TELL). The overall operation of the je-th LogiX-Gil --GI layer be formulated with the notation of Equation [ as follows:
h (1) 8(h)
Σ
1((k-1)
(EN(e)U(e)
The first step involves aggregating the information of node e E and its neighbors by summing (k) Emev(ou(u)) ( (k-1) This aggregation is equivalent to the one of GIN when setting 0, which is necessary represent counting operations in GML, After summing binary
their features: de
5

---

embeddings from neighboring nodes, the aggregated features become integer-valued. To apply logic-based rules, we binarize them using a novel thresholding mechanism. Specifically, we need a function capable to activate if its input falls within certain intervals. For this reason, we design a parametric Fourier step function, which flexibly and differentiably activates (outputs one) when counts fall within learnable value ranges:
Σ
0 = (a) clamp
+
00,1
(5)
sin (2
21+1
where @ denotes element-wise multiplication (Hadamard product), is temperature hyperparameter (we use 10), (()) E RR and ((4) IR are learnable parameters, and d the hidden dimension. is a periodic step function with respect to (k), where (*(h) and 8(0) modulate the width and the displacement of the counting intervals corresponding be obtained as follows:
activation. Specifically, the intervals can
a a|d h = - afa
( E
U nEZ
( WB
(2mn+ (oo++))-bie -- Iptt
(6)
(
This enables the extraction of literals that are active only under precise counting conditions. To better clarify the functioning of B3 we provide in Appendix A an example of the function varying the parameters (A) and ((A).
The final transformation (*) consists of the constrained linear transformation followed by a sigmoid activation function, proposed in TELL:
A (b) (k)
(((k)
(( t ( )
)
with T = 10
Using the results of Ragno et al. [23]. model be monotonic. This allows for retrieving logic rules in FO over the activations ((k) which correspond to formulas in GML over the literals h (k-1) of each node u E V. Such logic rules are identified by the sets of weights of W+(K), which sum up to a value greater than -)((k, For a more
non-negative weights (++)) in order to constrain the
detailed and formal demonstration, the reader can refer Appendix D
LogiX-GIN overview. Here, we describe the practical implementation details of the LogiX-GIN architecture used in our experiments. The model is composed of five sequential LogiX- -INN layers, whose outputs are concatenated and passed through three global readout functions: sum, mean, and max. These aggregation functions extract global graph-level features by computing the total, average, and maximum activations of the rules learned by the five graph layers. The resulting features are then processed by final TELL layer, which learns first-order logic rules over these activations to perform the prediction.
As previously discussed, the presence of sigmoid activation functions within each LogiX-GIN layer introduces significant challenges when training deep architectures. To address this, we adopt a knowledge distillation strategy for pretraining. Specifically, we initialize auxiliary model with the same architecture, but replace the LogiX-GIN layers with standard GIN layers. To encourage binarized representations, this GIN model uses Gumbel- activations. After training the GIN model, we use it as teacher to guide the LogiX-GIN model for the first half of the training process: minimize the cross-entropy loss between the hidden states of the GIN and LogiX- X-NN models at each layer. This yields a layer-wise pretraining of the LogiX-GIN model. In the second half of training, finetune the full LogiX-GIN model using standard supervised learning. Throughout the entire training procedure, optimization is carried out using gradient descent to minimize the cross-entropy loss combined with an L1 regularization term on the weights W+
Since there is no built-in constraint on the size of the learned logic rules, the resulting model can sometimes produce overly complex and less interpretable explanations. To enhance interpretability. we apply post-training pruning procedure. This step consists of a final optimization stage which each layer is trained individually in reverse order (from the last the first) using an additional Hoyer | | 1 where ò is a small constant to avoid division by zero. This
regularization term: LHoyer =
regularization promotes sparsity in structured way: unlike standard L, regularization, the Hoyer

---

loss encourages the use of few large weights while pushing the remaining ones toward zero. Since the size of a logic rule determined by how many weights need to be summed reach the threshold -b, this sparsity directly reduces rule complexity. By applying this pruning procedure gradually and layer-wise, we minimize any potential impact on model performance while significantly improving the readability and compactness of the learned rules.
LogiX-GIN can provide explanations in different forms: logic rules can be extracted for each single layer to obtain a fine-grained set of rules that exactly reflect the model behavior; for a simpler overview, global logic rules can be obtained by identifying motifs that when present activate class- specific rules of the last layer; node attributions visualize node level contributions towards a prediction. Further details on the LogiX-GIN GINN pipeline are detailed in Appendix C
5 Experiments
In this section, we perform extensive experiments evaluate our proposal. We begin by comparing the classification performances of LogiX-GIN with respect to its black-box counterpart. Successively, we perform analysis of the interpretability of LogiX-GIN also in comparison with state-of-the-an approaches. We mainly focus on a logic-based analysis by identifying the global logic rules and layer-wise rules. For an additional analysis, we also report an experiment on node attributions in the Appendix E comparing with other self reff-interpretable models and post-hoc approaches.
As the GIN layer, which inspires this work, is mainly developed for graph classification, focus on this task for the experimental section. However, to offer a broader overview of the capabilities of LogiX-GIN, we also test the accuracy performances graph classification, the black-box GNNs are composed of 5 GIN layers, followed by max, sum and mean readouts. The readouts are concatenated and finally fed into an 2-layer MLP. We perform our experiments on the following graph and 3 node classification datasets spanning synthetic graphs (BA2Motifs (15), BAMultiShapes 22. BAShapes. BACommunity and TreeGrid (3I),, molecular graphs (MUTAG [II]. Mutagenicity [14), NCII (28). and BBBP (16). and protein graphs (PROTEINS [2].. We use grid-search find the optimal hyperparameter combinations and report the test set results of the best performing hyperparameters on the validation set. Specifically, to
some node classification datasets. For
ensure statistical validity, we report mean and standard deviation values over 10 different seeds.
In T we report the accuracy of the LogiX-GIl -IIN model compared to that of the black-box box model across all datasets. In all cases, LogiX-GIN achieves accuracy levels within 3% of the black-box model, demonstrating strong performance despite the addition of logic constraints. These results indicate that incorporating logic constraints has limited impact predictive accuracy overall. While some trade-off in performance is expected, the primary goal of this work is to explore the integration of logic for enhanced model transparency. Table [ instead, reports the accuracy scores of several SE-GNNs on graph classification. We observe that, compared to other self-interpretable approaches, our method reaches state-of-the-an performances 5 datasets out of In the remainder
Table 1: Accuracy on Graph and Node Classification. Values are reported as + over 10 seeds. We report the accuracy of the black-box model and LogiX-GIM, For an easier comparison, column "Acc. Decay" indicates the difference between the mean accuracy of the black-box and the LogiX-GINS
Dataset
Black-box GIN
LogiX-GIN
Acc. Decay
Class. Obj-
BAMuldShaapes
100.00 0.00
100.00 0.00
0.00
Graph
BA2Motifs
100.00 # 0.00
100.00 0.00
0.00
Graph
BBBP
87.95
85.9
2.05
Graph
MUTAG
84.74
82.63
2.11
Graph
NCII
81.87 1.39
7.59
2.94
Graph
PROTEINS
72.59 + 3.04
7.70
0.54
Graph
Mutagenicity
82.21 + 1.87
79.31
2.90
Graph
BaShapes
9..1
94.29 2.52
2.85
Node
BaCommunity
81.00 + 1.59
84.14
-3.14
Node
TreeGrid
100.00 0.00
98.71 0.87
1.29
Node
7

---

MUTAG
Mutagenicity -Pojy (PB)
BA2Motifs
BAMultiShapes
GraphTrail
N/A
Aighment 0.896
Algnment: 0.757
Nighment: 1.000
IHA Giy (WIA
LogiX-GIN
Algnment:
Alggmeet: 1,000
Aligmmen: 1.000
Aligmment 1 10000
Legend
Oaygon
Nitrognn
(not patter)
Carbon
Hydrogon
General Node
Figure I: Visual comparison between rules extracted with GraphTrail and LogiX-GIN, Alignment is measured as the accuracy between the predictions of the model and the logic activations of the rules following Armgaan al. [ Additional details about the explanations extraction provided in Appendix C
of this section, we focus on the interpretability of LogiX-GIN, while in Appendix detail hyperparameters settings and perform ablation studies.
| and H
we
Obtaining faithful global logic rules. Here, we compare the explanations produced by LogiX-GIN with those of GraphTrail []. As discussed in previous sections, despite sharing a logical foundation, the two methods diffier substantially in their explanatory procedures: GraphTrail is post-hoc method that generates rules over computation trees, whereas LogiX-GIn embeds logical reasoning directly into its architecture. LogiX-GIN, in fact, employs GML its reasoning logic, which goes beyond simply detecting the presence specific patterns via first-order logic. Despite these significant differences, if narrow focus to LogiX-GIN' final layer, the activations of the literals after the readout function can be interpreted as indicative patterns whose presence directly affects the final prediction. This observation forms the basis for comparison between LogiX-GIN and GraphTrail.
For this experiment, we employ the four datasets used the study by Armgaan et [ Due the layer architecture of the GIN blackbox, Graph Trail's SHAP-bace based computation becomes particularly resource-intensive. For this reason, on the BAMultiShapes dataset, Graph fails to generate explanations within feasible time frame (2 days). In Figure D illustrate the patterns learned by both models. The LogiX-GIN patterns are extracted by analyzing the activations over the validation set. Full activation data is provided in Appendix
Notable differences emerge, especially on MUTAG and Mutagenicity, For MUTAG, GraphTrail identifies the NO2 group, a well-known mutagenic substructure, while LogiX-GIN emphasizes carbon-based structures, especially aromatic atoms, which also play a key role in mutagenicity 11. However, we also anticipate that deeper analysis reveals that LogiX-GIN does recognize
Table 2: Comparison of SE-GNNs across graph classification datasets.
Dataset
LogiX-GIN
PiGNN
GIB
KerGNN
GNAN
Ba2Motifs
100.00 + 0.00
99.89 + 0.31
100.00 + 0.00
98.80 + 0.75
49.10 + 0.66
BaMultiShapes
1.0000
85.40
97.60 2.06
83.20
49.10 0.66
MUTAG
32.63
82.51 10.48
90.53
82.11
55.79 16.43
Mutagenicity
79.31 1.25
82.39
0.99
2.992
55.36 + 0.52
NCI1
78.93 1.51
7.75
78.15 1.32
69,00 1.38
50.80 + 1.15
PROTEINS
72.05
70.00
68.47 4.000
72.97
57.67 + 2.62
BBBP
85.90
83.54 0.37
84.78 + 2.57
8..02
22.80 0.86

---

LogoX-aIN Layer
Ogegreph
mook ad aeees
MAT KE
bost Creph
LogiX-GIN Layer
Ouyger
Masger
Canoo
TRI
Figure 2: Layer-wise explanation of LogiX-GIN model trained on the MUTAG dataset.
the .O2 group, although this occurs within its inner layers. This illustrates that our pattern-based analysis, when limited to the last layer, may offer only a partial view of the model's reasoning. On the Mutagenicity dataset, LogiX-GIN identifies simpler patterns such as successive nitrogen atoms and the NO2 group. Graph Trail similarly detects these patterns in the black-box model, while also uncovering additional carbon- carbon-related related structures. On BA2Motifs, both identify the house motif, while on BAMultiShapes, only LogiX- LogiX-GIN recovers the ground-trus rule.
Overall, although both methods highlight chemically and structural meaningful patterns, Logix-GNN exhibits less complexity and higher versatility than GraphTrail More importantly, the key difference lies in the "alignment" between the model's behavior and the explanations. Alignment defined as the accuracy between the model's predictions and the rule activations. GraphTrail, being post-hoc, may misalign with model behavior, while LogiX- ' explanations are fully aligned by design.
Explaining the single layers of LON.X-GIN. In this section, we analyze the core functionality of our approach, which enables inspection of the model at each individual layer. In Figure 2 we present a real-world example showing the actual rules learned by LogiX-GIN on the MUTAG dataset. This example provides insight into how the model learns predict molecular mutagenicity through interpretable rule-based reasoning.
As first observation, although the architecture consists of 5 convolutional layers, only 2 are actively used the decision process. Notably, the first layer captures low-level features, such as atom type and connectivity, which are then leveraged to identify functional groups. The first rule (green), in fact, detects the NO₂ group by counting the number of nitrogens, oxygens, and carbons. Interestingly, to enhance specificity, the model also checks on the number of carbons to detect the NO2, The second rule (violet) is instead more complex, likely related to aromatic rings: the model counts the number of carbon atoms in each atom's 's neighborhood. Additionally, it also focuses on non-carbon atoms connected to only one carbon, which may help in identifying cyclic functional groups. The third feature (orange) activates across all nodes. While this might initially seem trivial. it serves an important role in the subsequent layer. In the second layer, in fact, features from the first layer are combined, and this feature is used for counting node connections. This offers valuable insight into the model's internal logic, showing how features that appear unrelated to the final prediction can still play critical role.
Ultimately, the model's decision is determined by counting the number of nodes that activate a specific feature (pink) in the second convolutional layer. This mechanism sheds light on an important phenomenon: "removing graph parts may not affect predictions key activations stay within the learned range". Consequently, even when model appears to have lower fidelity w.r.t. node removal, this may be due not to reduced explainability, but to its reliance on counting- canting-basse decision logic.
Analyzing rules' complexity. We now analyze how the models use the layers to learn logic rules. With this aim, we analyze the models after the pruning procedure and study how layers learn rules. Figure 3 reports the amount of rules learned by each layer of the models. For each dataset, we select only one seed and one class and plot the number of rules extracted by each layer. the cases of multiple classes, we use class 0 as the target. From this analysis, we exclude rules that provide

---

Ra2Motiis
BaMultiShapps
MUTTAG
Mutagenicity
PROTFINS
60
20
Rules
Rulox
vailes
20
2.
Layer NC11
Layer BOBP
Layer BaCommunity
Layer BaShapes
TreeGrid
60
20
1.-
Bules
40
20
Figure 3: Amount of rules extracted by each convolutional layer on the 10 datasets in exam.
activations activate on all the nodes, as they mainly used to count node degrees. This analysis shows how models extract information from the graphs. Specifically, we observe that the first layers generally contain more rules, thus confirming the findings of the previous section. Indeed, the first layers tend to identify low-level patterns, which are then combined in the successive layers. We also see that in most cases the model does not use all the layers as the information extracted in the first layers is enough to perform the prediction. Contrarily, Mutagenicity and NCI1 are the datasets that require the highest amount of literals to describe the activity. This is most likely due to the higher complexity of the tasks and the high variability of the molecular graphs.
Overall, this analysis highlights the hierarchical nature of logic rules in the models. Early layers are densely populated with rules that extract fundamental features, while deeper layers serve a more selective role. This result is also confirmed when analysing rules activations (Appendix G). Furthermore, the redundancy of deeper layers in several datasets suggests a degree of architectural overcapacity, which pruning helps to address. Finally, the number of literals required across datasets underscores the varying complexity of the tasks and the richness of the input graphs, particularly in chemically diverse datasets. These interpretable insights directly inform model design: "no rules after layer 2 imply excess layers can be removed".
Conclusions
In this work, we introduced LogiX-GIN, a novel GNN architecture that incorporates logic-based reasoning directly into its layers to achieve self-explainability. We demonstrated that each layer of the architecture can be converted into logical formulas in GML formalism, offering transparent and faithful representation of the model's 's reasoning. Through a series of experiments on both synthetic and real-world datasets for graph and node classification, we showed that LogiX-GIN maintains competitive performance while offering a new level of interpretability.
Despite these promising results, work has some limitations. First, LogiX-GIN does not take into account edge features, limiting its potential expressiveness. Moreover, the usage of sigmoid activations and learnable binarization introduces training challenges, particularly in deeper networks or larger graphs. Although we mitigate these issues with a distillation-based pretraining strategy, the computational cost remains significant thresholding on input features might not be interpretable in cases where they are not attributed with specific semantic meanings. Future work could investigate case the integration with prototype-based or concept- based techniques that could improve interpretability thanks to their case-based reasoning. Another important limitation concerns the expressivity of the model itself: since LogiX-GIn relies on GML, its reasoning capabilities are constrained by what GML can express. As a result, the model may be suitable for tasks that require more expressive or higher-order forms of reasoning beyond the scope of GML. Future research can address these limitations by exploring more scalable architectures and optimization techniques that preserve interpretability while improving computational efficiency. Additionally, extending the model to support broader range of graph tasks or other logic formalisms might be interesting. In this case, combining logic-based reasoning with other forms of explanation, such prototypes, may further enhance performance and interpretability.
the training is composed of multiple steps. Also,
10

---

Acknowledgments and Disclosure of Funding
This work was supported by French state aid managed by the National Research Agency under the France 2030 program, with the references "WAIT4 ANR-22-PEAE-0008" "PANDORA ANR-24- CE23-0950", and "PORTRAIT
References
[1] Burouj Armgaan, Manthan Dalmia, Sourav Medya, and Sayan Ranu. Graphtrail: Translating gnn predictions into human-interpretable logical rules. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[2] Steve Azzolin, Antonio Longa, Pietro Barbiero, Pietro Lio, Andrea Passerini, al. Global explainability of gnns via logic combination of learned concepts. In ICLR 2023, pages 1-19. online, 2023.
[3] Steve Azzolin, SAGAR MALHOTRA, Andrea Passerini, and Stefano Teso. Beyond topologiaal self-explainable gnns: A formal explainability perspective. In Forty-second International Conference on Machine Learning, 2025.
[4] Pablo Barceló, Egor Kostylev, Mikael Monet, Jorge Pérez, Juan Reutter, and Juan-Pablo Silva. The logical expressiveness of graph neural networks. In 8th International Conference on Learning Representations (ICLR 2020), 2020.
[5] Maya Bechler-Speicher, Amir Globerson, and Ran Gilad-Bachrach. The intelligible and effective graph neural additive network. Advances Neural Information Processing Systems, 37:90552-90578, 2024.
[6] Gabriele Ciravegna, Pietro Barbiero, Francesco Giannini, Marco Gori, Pietro Lió, Marco Maggini, and Stefano Melacci. Logic explained networks. Artificial Intelligence, 314:103822, 2023.
[7] Luigi Pietro Cordella, Pasquale Foggia, Carlo Sansone, Mario Vento, et al. An improved algorithm for matching large graphs. In IAPR-TCI5, 2001.
[8] Enyan Dai and Suhang Wang. Towards prototype-basee based self-explainable graph neural network. ACM Transactions Knowledge Discovery from Data, 2022.
[9] Adnan Darwiche. Logic for explainable ai. In 2023 38th Annual ACM/IEEE Symposium on Logic in Computer Science (IJCS), pages 1-11. IEEE, 2023.
[10] Maarten De Rijke. A note graded modal logic. Studia Logica, 64(2):271-283, 2000.
[11] Asim Kumar Debnath, Rosa L. Lopez de Compadre, Gargi Debnath, Alan J. Shusterman, and Corwin Hansch. Structure-activnn activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of Medicinal Chemistry, 34(2):786-797, Feb 1991. ISSN 0022-2623.
[12] Paul D. Dobson and Andrew Doig. Distinguishing enzyme structures from non-enzymes without alignments. Journal of Molecular Biology, 330(4):771-783, 2003. ISSN 0022-2836.
[13] Aosong Feng, Chenyu You, Shiqiang Wang, and Leandros Tassiulas. Kergnns: Interpretable graph neural networks with graph kernels. Proceedings of the AAAI Conference on Artificial Intelligence, 36(6))6614-6622)
Jun. 2022. doi: 10. 10.1609/aaai. http//aai.v3666.2061555
URL https
//ojs. aaajs.aaaa..m wta.s/aaa...ogga/aaa.o..o forg/indder index,iiexxxi/ pwaiddee..pp///eee
[14] Jeroen Kazius, Ross McGuire, and Roberta Bursi. Derivation and validation of toxicophores for mutagenicity prediction. Journal of Medicinal Chemistry, 48(1):312-320, December 2004. ISSN 1520-4804.
[15] Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. Advances in neural information processing systems, 33:19620-19611. 2020.

---

[16] Ines Filipa Martins, Ana L. Teixeira, Luis Pinheiro, and Andre O. Falcao. A bayesian approach in silico blood-brain barrier penetration modeling. Journal of Chemical Information and Modeling, 52(6): 00116666-160 1697, jun 2012.
[17] Siqi Miao, Mia Liu, and Pan Li. Interpretable and generalizable graph learning via stochastic attention mechanism. International Conference on Muchine Learning, pages 15524-15543. PMLR, 2022.
[18] Peter Müller, Lukas Faber, Karolis Martinkus, and Roger Wattenhofer. Graphchef: Learning the recipe of your dataset. ICML 3rd Workshop on Interpretable Machine Learning in Healthcare (IMLH), 2023.
[19] Alexander Pluska, Pascal Welke, Thomas Gärtner, and SAGAR MALHOTRA. Logical distilla- tion of graph neural networks. In ICML 2024 Workshop on Mechanistic Interpretability, 2024. URL
wwwp:/neee/oounindnnnee.e YRD22yRO
[20] Phillip E Pope, Soheil Kolouri. Mohammad Rostami, Charles E Martin, and Heiko Hoff- mann. Explainability methods for graph convolutional neural networks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 10772-10781, 2019, 2019.
[21] Alessio Ragno and Roberto Capobianco. Impo: Interpretable memory- baseo prototypical pooling. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, WSDM '25, page 625-632 New York. NY. USA, 2025. Association for doi: 10. 15.1144557711551337035544 URI.ttpss:
Computing Machinery. ISBN 9798400713299
//doi.org//0. 110.00..1140 1111/1111//11111 3703553
[22] Alessio Ragno, Biagio La Rosa, and Roberto Capobianco. Prototype- interpretable graph neural networks. IEEE Transactions on Artificial Intelligence, 5(4):1486-1495, 2022.
[23] Alessio Ragno, Mare Plantevit, Celine Robardet, Roberto Capobianco, et al. Transparent explainable logic layers. In ECAI 2024-27th European Conference on Artificial Intelligence, 19 19-24 October 2024, Santiago de Compostela, Spain-Including 13th Conference on Prestigious Applications of Intelligent Systems (PAIS 2024), volume 392, pages 914-921. Ios Press, 2024.
[24] Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell., 1(5): 2019. doi: 10.1038/ S42256-019-0048-X URL https://:/// htt:///ii.oogg//// 10. 1038/ 01103////222225501-0-4444
[25] Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE transactions on neural networks, 20(1))61-80, 2008.
[26] Thomas Schnake, Farnoush Rezaei Jafari, Jonas Lederer, Ping Xiong, Shinichi Nakajima, Stefan Gugler, Grégoire Montavon, and Klaus-Robers Müller. Towards symbolic xai aai-exllanttion through human understandable logical relationships between features. Information Fusion, 1 118: 102923, 2025.
[27] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pages 3319-3328. PMLR, 2017.
[28] Nikil Wale, Ian A. Watson, and George Karypis. Comparison of descriptor spaces for chemical compound retrieval and classification. Knowledge and Information Systems, 14(3):3473375, August 2007.
[29] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 32(1 (21)): (::4:::2444 24, 2020.
[30] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2019. URL https //openrevies nett/oorum2d-rryG66...55
[31] Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32, 2019.
12

---

[32] Junchi Yu, Tingyang Xu, Yu Rong, Yatao Bian, Junzhou Huang, and Ran He. Graph information bottleneck for subgraph recognition. In International Conference on Learning Representations. 2021. URL http:///ppeneeeeeeeeeeeeefrrrrra
[33] Junchi Yu, Jie Cao, and Ran He. Improving subgraph recognition with variational graph information bottleneck. In Proceedings of the IEEE/CVF Conference on Computer Vision and Partern Recognition (CVPR), pages 19396-19405, June 2022. URL https://oppeaaccess thecvf. com http://coneettcc/cccc ortpa:/w..........oooooooooooooo
Vathaaariati R
pdf
[34] Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In International conference on machine learning, pages 12241-12252. PMLR, 2021.
[35] Shichang Zhang, Yozen Liu, Neil Shah, and Yizhou Sun. Gstarx: Explaining graph neural networks with sructure-awan cooperative games. Advances in Neural Information Processing Systems,
2022.
[36] Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Cheekong Lee. Protgnn: Towards self-explaining graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 9127-9135, 2022.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our abstract and introduction clearly outline the contributions of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are addressed in Section 6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors shouldindeed if u look at the rule in discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
We provide proof of our theoretical result in Supplementary Section
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibiliyy
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide in the Supplementary Material the code and all the necessary information to reproduce our results.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification: We do provide open access to the code. The data is already publicly available and cited the proper sources.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https /nipss.cc/pulli//uuui guwde//ddee/ee/ Cot//ooodeeesiseeeeeeeeee hwtpbbmmsssioooooooooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The Experimental Settings are detailed in Supplementary Section [and in the code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We do replicate experiments on different seeds and report mean and standard deviation of the results for statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
16

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We do report such information in the Supplementary Section] !
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We follow the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All the datasets and techniques are cited in the references.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The background, proposal and experimental sections document the setting used. The code and the assets is also available supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
19

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non non sta-standard components.
Please refer to our LLM policy aatt:a///narrips.cco///offreeeee for what should or should not be described.
20

---

A Fourier Step Function
In this section, we analyze the functionality of the function of LogiX-GIN as defined in Equation Figure S shows three examples illustrating how the parameters w (width) and b (bias or displacement) affect the shape and position the intervals. The first case (w 1, b shows baseline case where, in this interval of input values (-5 < < 5), the output corresponds to a if z < -3 or x The second case (w 2, 0) highlights how w can be used modify the width of the intervals. Finally, the third case (w 1, - 2) shows how the parameter is instead responsible to apply a displacement of the thresholds (in this case -2).
1, =
10
08
06
0.4
Inpe
www.w. M
10
08
Volue
000000oll
0.0
Input Volue
bb
Binanzed
0.4
0.2
00
Input Value
Figure 4: Examples of the binarization using the Fourier step function Equation
B Examples of Graded Modal Logic Formulas
Following Barceló et al. 4 we use colors as unary predicates: Blue(x), Red(x), and Green(x) denote that node æ is respectively blue, red, or green.
Example 1: Atomic predicate.
<pxx) = Bluc(x)
This formula holds if is blue.
Example 2: At least two red neighbors.
(<x) == 3 (E(x, y) A Red(y))
This states that has at least two neighbors that red.
21

---

Example 3: Blue node without red neighbors.
(xx)) := Blue(x) A -32²y (E(x,))
(E(r)) v) A Red(y))
This expresses that x is blue and has no red neighbors.
Example 4: Disjunction of neighborhood conditions.
((x) = (= y (E(x,y) v) Red(y))) v (=² y (E(z, 9) A Green(y)))
This means that x has either at least three red neighbors, or at least one green neighbor. Example 5: Nested neighborhood property.
(xx) 32 (E(r, y) 3 (E(y, z) A Green(z)))
This states that has a neighbor which itself has at least one green neighbor z.
These examples illustrate how formulas, written with colored predicates, capture local structural patterns in graphs using counting quantifiers and logical connectives.
C LogiX-GIN Explanation Procedure
Sum Aggregation
Fourier Thresholding
Symbolic Prediction
y a(2x + +2 2.5)
X1 A Vx2
Σ 23 ( VENMM
S4v Σ
$2) VEN
VENMUM
Figure 5: Diagram of LogiX-GIN layer information flow and explanation procedure. At the first step. the aggregated node features of node v are obtained by summing up over its neighbors. Successively, Fourier thresholding allows to activate only when features lie in specific intervals by generating binary representation. Finally, a TELL-like classifier outputs a value whose activation can directly converted into DNF formula. Finally overall rule is obtained by merging the thresholds into the DNF extracted by the last step. In The final rule, the output feature of for node v will be one if and only if the neighboorhod contains least 3 nodes with the green feature activated and either there are at most two nodes with the red feature activated or between and 4 nodes with the blue features activated.
Overview and pipeline. Figure 5 shows the LogiX-GIN pipeline. Node features from a target node and its neighborhood are first aggregated by summation. These aggregated values are then binarized by a learnable Fourier step function, which converts counts into discrete on/off literals. The binarized signals are passed through a logic layer with non-negative weights, producing interpretable logical activations. During training, a teacher GIN model is used for distillation, and Hoyer-style pruning can be applied to remove redundant rules. From this pipeline, we can extract explanations at different levels of granularity:
Layer-by-layer explanations provide fine-grained analysis representing the exact model behavior. Each hidden layer corresponds to a set of logic rules that are directly readable from the parameters of its logic transformation. For every output unit, the threshold conditions
22

---

on its binary inputs define minimal logical clauses. Presenting rules per layer allows us to reconstruct how evidence is accumulated step by step, without needing collapse the entire network into single formula.
Global logic rules identify logic formulas over subgraphs that globally approximate the model's behavior. To obtain them, at the final classifier, we extract global rules that summarize the decision process for the whole graph. The extraction procedure follows the following step:
1. Apply rule extraction to the final logic layer, obtaining global rules associated with each class.
Identify the nodes in given graph that activate these rules.
3. Collect the induced neighborhoods of these nodes and compute canonical subgraphs. merge isomorphic subgraphs and
Use a standard graph isomorphism algorithm identify recurring motifs.
The resulting set of global rules provides numan-readable summaries such as graphs contain- ing motif M are classified as class c". To evaluate these rules, we measure their alignment, that is, the agreement between the rule activations and the model's predictions on held-out data.
Node attributions from global rules provide scores for nodes of a graph which measure their contribution towards a prediction. When global rule is activated for graph, attribute responsibility to the specific nodes that support its conditions. A node is considered relevant if it appears in least explanation highlights the exact part of the graph that triggered the decision. This yields node-level attributions that are directly derived from the logic rules, avoiding the need for post-hoc gradient or perturbation methods.
minimal conjunct that satisfies the rule. In this way, the
D Demonstration
(K) denote the output of the k-th layer of the LogiX-GIN architecture, defined as Equation 4 where ( a monotonic logic transformation as in Equation 7 and ( is differentiable binarization function over learned activation intervals as in Equation Then, for each
Proposition 1. Let lee
there exists a set of formulas {1p (x)} in the syntax of GML such that
(K)
[ =1 = (G. (G.0) F 4j (x),
where G the input graph and is the free variable corresponding to node v.
Proof. We proceed by structural correspondence between the computational components of (a) and the semantic constructs of GML:
Aggregation. Given a node E V, the expression
ag
(k-1)
(8)
(o)(()
computes, for each feature index {l...., d}. with d the hidden dimension, the number of neighbors such that h (k-1) [ 1. This results in a count vector a, N4, where each entry encodes the multiplicity property among the neighbors.
Binarization. The function g(k) applies learnable thresholding to each coordinate [,] producing:
c = (a.) E 10.
(A) Each output de [i] encodes the satisfaction of a threshold condition:
a0 = + [il E θ + U()) (v) ( - 1}1 Igce) (10)
23

---

where IgcN ; is learnable interval determined by g(h) for the i-th dimension, defined in Equation 6 By inductive hypothesis, for each i, there exists a formula v/ (((-11) (x) such that hu
- (G, T ( Therefore:
= - F Ge
(80
V
((12h1 (B(r,y)
(B(t)(((()))) A 1/ (v))
(11)
^ (35S.y
(E(r (
vs
( (w))
Hence, each dge [1] corresponds to a valid graded modality GMI.
Monotonic Logic Layer As shown by Ragno et al. [23] due to non-negativity of weights
and the monotonicity of a, each output neuron hhe [j] from Equation corresponds to logic formula disjunctive normal form (DNF):
[ -1
V SES,
A iES
a [ =1,
(12)
wher Sj is the set of subsets of features with corresponding weights that sum up to a value greater than ( Since each d [i] corresponds to a GML formula as shown above, follows that:
(1) [1] = + (G, F pj (K)
(z),
where
/ (x)
V A V
((32h1 E((r,y) A&g (
Ses, iES (t..T.)EZZ(0)
(13)
(3STLy (E(c,)) (k- (v)
By induction on k; and given that the base case (input binarization) is definable by unary predicates, we conclude that all h [i] are convertible into GML formulas. Thus, the output of the LogiX-GIN architecture is fully convertible in GML.
E
Node Attribution Evaluation
Here, we perform an analysis to evaluate the explanations proposed by different approaches when simply focus on node attributions. We perform our comparison with post-hoc methods (GNNEx- plainer [31]. PGExplainer [15), Integrated Gradients (27). SubgraphX [34), and GStarX (35) applied
on the black-box GIN model and self-interpretable models (PiGNN [22] and GIB (32 We perform
the evaluation in terms of the Fidelity of the attributions. Fidelity is defined as the ratio of graphs
whose prediction is shifted when removing the most important nodes.
In literature, node attribution evaluation is mainly designed of post-hoc instance-level approaches, which focus finding the most important nodes that lead to specific prediction. On the contrary, self- interpretable models are designed to provide visual explanations thanks to architectural constraints. For these reasons, node attributions represent only approximation of the explanatory power of self-interpretable models and this evaluation setting can end up being unfair to them. This is primarily due to lack of evaluation methods for self aifepntterrettble models in literature for several reasons. First, these type of models are still highly unexplored, second, there is often a high diversity between the various approaches. For instance, prototype- based approaches use the similarity between prototypes and subgraphs of the input graph to determine the class. Node attributions for these methods are obtained by extracting such similarities. However, is not guaranteed that removing the portions of the input graphs that actually led to specific prediction cause a shift in the prediction. On the contrary, optimization based methods such as GStarX that specifically aims to find subgraphs that when removed provoke shifts in the prediction. Additionally, while with post-hoe methods are generally compared when applied on the same model, in the case of self-explainable models, we have different models under analysis.
24

---

NC11
Mutagenicity
MUTAG
0.4
Feelity
Ficelity
0.6
0.3
0.4
0.4
0.2
0.2
Sparsity
Sparsity
Sparsity
BA2Mobifs
BAMultiShapes
BBBP
0.5
0.6
0.4
Edelity
0.4
0.3
0.3
0.2
0.5
0.9
0.6
0.7
0.8
0.9
0.5
0.9
Sparsity
Sparsity
Sparsity
PROTEINS
0.6
0.5
Fideiity
0.3
0.2
0.1
0.5
0.6
0.7
0.8
0.9
Sparsity
GNNExpl.
PGExpl.
SubGraphx
CStarX
GI8
PIGNN
LogIX-GIN
Figure 6: Fidelity.
Figure 6 shows the Fidelity score on the graph classification datasets. Since the different methods report continuous attribution score, the fidelity is calculated the hard masks different sparsity level. In particular, we vary the sparsity between 0.5 and 0.95.
We observe that on four cases, namely NCII, MUTAG, BA2Motifs and BAMultiShapes, LogiX-GIN produces Fidelity scores that are comparable or better that the state-of-the-ar post-hoc approaches, specifically with optimization-batteee based ones. On BBBP, Mutagenicity and PROTEINS. instead observe better values from GStarX and PiGNN. However, as already underlined, the case of LogiX-GIN, explanations
obtained by simply looking the rules activated on the last layer, which might not truly indicate its true behavior. For this reason, we believe that a more thorough examination of the model through inspection of its complete logic rules can actually provide better transparency of the model.
F
Last Layer Rule Activations
This section reports the activations of the logic rules of the last layers the datasets of the global rules experiment. F graphs
validation set highlighting the nodes that activate rules of the last layer. In order to provide a compact representations, activations are aggregated using isomorphism. Therefore, for each dataset, only non-isomorphic activations are shown. We observe that the rules capture patterns that generally known to be related with the predicted class.
25

---

(a) BA2Motifs
(b) MUTAG
(c) Mutagenicity
(d) BAMuldShapes
Figure 7: Last layer rules activations for obtaining global explanations. each dataset, we show validation set graphs that activate for the rules and highlight in red the nodes corresponding to such activations.
G Logic Rules Activations
This section reports the activations of the logic rules on the datasets. For each model we extract the rules of each layer and plot in Figure 8 the distribution the percentage of the nodes that
26

---

Leyer
Layer 2
Layer
Layer 4
Layer 5
Bo2Mottr
Rules
Da2Moott
Ba2Monll
Ba2Mootf
100
120
50
100
0.3
300
00
50
Activated hodes
Activated Nodes Layer
Activoted Nodes
Activated Nodes
%6 Activated Nodes
Layer
Soh S0 Stapphpp Rules
Do Shapes
Boh AU Shapes
Rutes
Rules
100
50
100
300
Activated hodees
75 Activated Noces
Activated Nodes
76 Activated Nodes
56 Activated Nodes Layer
Leyer
Layer
0,5
0.05
HUTRG Rules
HUTBG Riless
HUFFAG
Rules
-0.05
-0.03
-0.55
Activsted Nodes
Activated Nodes Layer
Activated Nodes Layer
Activated Nodes Layer
Layer
Layer 2
Mubagenccty Riles
15
Mutegenccyy
Mutageniccy
Iultspencctyy
Riics
93
0.00
-0.55
100
50
100
30
Nodes
Layer
Layer 2
Layer
Layer 4
Layer 5
1.2
PROTEINS
MOTEINS
MOTEISS BLice
PROTEINS
0.00
0.0
160
50
100
0.0
300
Activated hodes Layer
99 Activeted Nocies
Activated Nodes
36 Activated Nodes Layer
56 Activated Nodes
Layer
NO1
NC1 Rules
NOT Rule
300
Activated Nodes
55 Activsted Nodes Layer
Activated Nodes Layer
36
Activated Nodes Layer
56 Activated Modes
Layer
Layer
0.05
B88P R/ies
RURS
BBBP Rules
-0.5
Activated
Activated Nodes Layer 2
Activated Nodes Layer
Activated Layer
Activated Nodes Layer 5
Layer 1
Tredardd RJles
TreeSind
Threeaad Rules
Treesaid Pules
TreeGrid Ihlls
100
50
100
300
80
Layer
Layer 2
Layer
Layer 4
Layer 5
ExCommunty
CaCmmnnn
ScCammuntty
SyCommun
Riles
5.0
5.0
25
100
cn
50
na
300
Activated Kodes Layer
%6 Activated Nodes Layer
Activated Nodes Layer
56 Activated Nodes Layer
%6 Activated Modes
Layer
1.0
0.05
0.09
0.55
DeSPoopes
ReSPapps
Ruiles
-0.05
-0.05
-0.03
100
50
300
Activated Rodes
56 Activsted
Noces
Activated Nodes
36 Activated Nodes
56 Activated Modes
Figure 8: Analysis on percentage rules activations for each layer in LogiX-GIN.
activated. We observe general pattern that early layers have high variations the activations, most likely due to the fact that they tend to recognise more general patterns. On the contrary later layers focus on more detailed patterns, drastically reducing the percentage of nodes that are activated.
27

---

H Ablation Study on Pruning
We performe an ablation study on the BBBP and BA2MOTIFS datasets to evaluate the effect of pruning strategies. We compare (i) layer-by-laye pruning, where the Hoyer regularization is applied sequentially starting from the last layer, and (ii) full pruning, where it is applied jointly to all layers. T reports the percentage decrease of non-zero weights (from the unpruned to the pruned model) for both strategies.
Table 3: Ablation study on pruning. We report the percentage decrease of non-zero weights after pruning. layer-by-layer pruning consistently yields stronger reductions compared to full pruning.
Dataset
Layere
Full
Ba2Motif'
83.13
78.84
BBBP
30.39
0.00
The results show that the layer- strategy better reduces the number non-zero weights. In contrast, jointly optimizing all layers once leads to weaker pruning due to interdependencies among rules: if rule at layer depends on features from layer & 1, then pruning must respect this order. For this reason, pruning all layers together may incorrectly remove weights that are still needed in subsequent rules.
Hyperparameters, Resources and Reproducibility
All the experiments, including those on GraphTrail, post-hoc methods and other self-explainable architectures, are documented in the supplementary material. The experiments were performed on machine equipped with an Intel(R) Core(TM) 19- 10900K CPU @ 3.70GHz and an NVIDIA GeForce RTX 4090. The implementation is done in Py PyTorch and Py TyToch-Geometne Geometric and all the library versions are detailed in the environment specifics available in the supplementary material. In Table Table we detail the hyperparameters of the models.
In Table we report the average training times of GIN and LogiX-GIN. Although the LogiX-GIN training times are much higher compared to the ones of the black-box, it is always important to keep mind that they allow for full inspection of the single layers of the model. Before LogiX-GIN, no model or technique allowed for such level of explainability. Additionally, it is also important to keep in mind that, in the case of the GIN model, order have an explanation only of the last layer, we would still need to execute GraphTrail which might take days to generate an explanation.
Table 4: GIN Models Hyperparameters
Dataset
batch
size
dropout
epochs
hidden dim
12
lr
BaShapes
32
3000
32
0.0001
0,001
BaMultiShapes
128
0..5
3000
32
0.0001
0.001
BaCommunity
128
0.5
3000
16
0.0001
0.01
Ba2Motifs
128
3000
16
0.0001
0.001
BBBP
32
0.5
3000
16
0.0001
0.001
MUTAG
128
0.5
3000
32
0,0001
0.001
NCII
128
3000
32
0.0001
0.001
PROTEINS
32
3000
32
0.0001
0.001
Mutagenicity
32
0.5
3000
32
0.0001
0.001
TreeGrid
128
3000
32
0.0001
0.001
Rule Pruning Statistics
Here, we report an analysis of the effects of our pruning strategy on the LogiX-GIN model. Table 7 shows the test accuracy and the number of non-zero weights before and after pruning on all the datasets in exam. We observe that the strategy does not impact the performances drastically. On the
28

---

Table 5: LogiX-GIN Models Hyperparameters.
Dataset
batch size
conv conv_reg
epochs
fe reg
12
Ir
BaShapes
128
0.001
5000
0.01
0.01
BaMultiShapes
32
0.001
5000
0.01
0.0
0.001
BaCommunity
32
0.001
5000
0.1
0.0001
0.01
Ba2Motifs
32
0.001
5000
0.01
0.0
0.001
BBBP
32
0.001
5000
0.01
0.0
0.001
MUTAG
32
0.001
5000
0.01
0.0
0.01
NC11
32
0.001
5000
0.1
0.0001
0.0001
PROTEINS
32
0.001
5000
0.01
0.0
0.001
Mutagenicity
128
0.001
5000
0.01
0.0001
0.001
TreeGrid
32
0.001
5000
0.01
0.0
0.001
Table Average and standard deviation of training times (in milliseconds) for GIN and LogiX-GIN across datasets (updated values).
Dataset
GIN (avg + std)
LogiX-GIN (avg + std)
Ba2Motifs
2727.00 + 207.80
6671.67 + 177.83
BaMultiShapes
1193.67 35.37
6354.00 201.92
MUTAG
279.00 = 122.06
797.00 26.87
Mutagenicity
3656.00 40.90
5748.33 27.38
PROTEINS
1872.33 = 13.89
11077.00 4.97 4.97
NCII
1695.33 48.80
49646.67 = 636.76
BBBP
2044.00 + 77.94
19855.00 + 46.50
TreeGrid
265.33 + 19.94
4361.00 12.03
BaCommunity
497.33 # 11.09
21057.67 = 3.77
BaShapes
1072.67 73.19
16680.67 # 4.50
Table 7: Statistics of classification performances and number of non-zero weights the LogiX-GIN model before and after pruning. The experiments are performed over 5 seeds and mean and standard deviation are reported for statistical significance.
Datusel
Stage
Accuracy
Total Nom-zeros
Com-Layer
Com--Laee
Com-Layer aaee33
Com-Layer
Con--Layer
FC Layer
BaZMonifs
Original
1.00 0.00
4332
252 48
957 + 112
958 56
845 51
1007 127
137 70
Pruned
1.00 0.00
880 58)
31
201 i 211
334 360
243 2.399.399
2
BaMuluSSapes
Original
1.00 0.00
4546
280
1080 57
1013
908
1
Original
Pruned
711
133
218
107
8
Mutagenicity
Original
0.82 0.02
4643 250
474 26
967 41
1000 30
999 111
976 184
205 15
Pruned
0.51 1 0.02
4133 54,5
474 26
967
1000 2: 30
957 161
651 6.30
61 61
NCT1
Original
1:800=0..00
4443 + 139
295 32
827 46
942 142
952
1861 68
165 +
Pruned
0.80
208
25
PROTEINS
Original
BBBP
Original 0.88 0.01
2775
68
587
155
116
Pruned 0.86 0.00
1732 250
272 91
740
481 327
154 300
0
8.5 37
TreeGrrdd
Original
0.98 0.00
4014
180 +:00010
860
965 74
970 54
1008 52
701
Pruned
1.00+0.01
1007 4-4/4
99
333 107
317
205 101
46 35
5
BaShapes
Original
1.93..00.05
1016 101
68
14111477
220 2.29
220
195
69 2.
Pruned
0.97
58.1
50
BaCommunity
Original
99.2
contrary, in some cases we also record improvement of the performances as the pruning encourages generalization in the model. The amount of non-zero weights reflects the number rules identified in the experimental section. Indeed, we observe that the number of non-zero weights is directly linked with the number rules that the model uses.
29


Reviews:
[{"review_id": "WBrJO41Ge8", "reviewer": "Reviewer_Dowp", "summary": "The authors introduce an inherently explainable GNN method. They use transparent explainable logic layer and graded modal logic to make an interpretable version of a GIN layer. This interpretable model is shown to have similar performance to the black-box GIN while extracting sensible logic rules for real-world datasets commonly used to evaluate explainable GNNs.", "questions": "In Node Attribution section, you state that attributions of LogiX-GIN are taken from the rules activated in the last layer. Is it instead possible to provide an end-to-end node attribution in your method, taking all layers into account? Something akin to running integrated gradients on the method. While I appreciate the value of having inspectable model rules, in practice, having good node attribution can be very useful for the end-user when inspecting a given graph.\n\nMaybe to get this 'full' explanation one could also just expand the logic rules of all the layers into one? Or does that in practice become unreasonably large and is unsimplyfiable?\n\nOne missing reference is GraphChef: https://openreview.net/pdf?id=IjMUGuUmBI which in some ways is quite similar: inherently explainable architecture based on GIN, with a transfer learning step from a black-box GIN, but uses decision trees instead of logic rules (thus not differentiable).", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "I think work on self-explainable GNNs is super important, as it greatly increases the trustworthiness of the methods (which post-hoc analysis can't necessarily provide). The proposed method in my opinion is a valuable contribution to this sub-field. As far as I know this is first method providing a self-explainable GNN of this type that uses logic rules. The fact that the final self-explainable method uses TELL layers is quite handy, as the model still is GPU accelerated and in principle differentiable (even though the gradients might be bad due to sigmoids). Which contrasts with some other self-explainable GNNs that exist.\nThe writing and presentation are clear. The benchmarks used are quite standard and the results look good.\n\nThe authors themselves acknowledge the two main weaknesses that I also see: 1) needing to start with a black-box GIN, which I think is a very minor issue and 2) that to extract explanations they rely on logic rules in the last layer. Here I think the method would be noticeably improved if authors provided a way to easily extract the full explanation that takes into account all layers. Of course this information is still in the model, so persistent user could pull it out manually, but it would be quite cumbersome.", "was_revised": true, "final_justification": "After rebuttal and reading other reviews I still like the paper and recommend acceptance.", "rebuttal": "We thank the reviewer for the comments and suggestions. Regarding the second weakness, only the \"global\" explanations of Figure 1 are w.r.t. the last layer. Instead, the innovation of our proposal is exactly that, with LogiX-GIN, we can extract full rules for all the layers, as shown in Figure 2.\n\nFor the first question, although the node attributions use the last layer, they are still generated using the activations that come from the first layers.\n\nFor the second question, it is in theory possible. However, we believe that merging the explanations would make it even less interpretable, given that other reviewers have already raised concerns about the interpretability of logic and the amount of rules. \n\nRegarding question 3, we have included GraphChef as suggested by the reviewer and also other reviewers."}, {"review_id": "C4T7tfcF6y", "reviewer": "Reviewer_LV2N", "summary": "The paper tackles the challenge of providing explanations for GNNs by proposing a new ante-hoc method called LogicX-GIN. The proposed model incorporates some constraints in the way information is processed by the architecture, making the model amenable to rule extraction. Claims are tested on several datasets, showing that the model retains the predictive power of black-box GNNs while allowing for extracting graded model logic formulas.", "questions": "- **Q1**: What is the role of the clamp function in Equation 5? I was expecting $\\tilde{\\alpha}$ scores to be binary in $\\{0,1\\}$, but it seems that $\\tilde{\\alpha}$ can take other values, like $0.1$. Why is it done in this way?\n\n- **Q2**: Are $\\tilde{\\alpha}$ vectors or numbers? $\\alpha$ is defined in line 209 as the sum of embedding vectors, but in Eq. 6, you write $\\tilde{\\alpha}=1$, which indicates they are simple scalars. Maybe you mean $\\tilde{\\alpha}[j]=1$, where $j$ is a single hidden dimension?\n\n- **Q3**: Why is the additional Hoyer loss applied as a post-training step, rather than being added as an additional loss during training?\n\n- **Q4**: Can you please elaborate on lines 336-337? It is not clear what the underlying message the authors are trying to convey.\n\n- **Q5**: How does the model cope with graphs where node features are complex embeddings, like textual embeddings?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n\n- Investigating novel paradigms for self-explainable GNNs is a very interesting direction, and adapting logic explanations to the ante-hoc setting is rarely studied\n\n- The paper makes a connection between the logical expressivity of GNNs and their relative logical explanations, which is a novel contribution\n\n- Albeit the description of the proposed contribution is unclear in some parts (see below), the architecture remains relatively simple and does not require articulated components or training losses, which is a pro to the proposed method\n\n\n# Weaknesses\n\n- The model is grounded on the TELL architecture, but this is only briefly explained in the Background. The self-containment of the paper would benefit from providing more context and background about TELL. For example, it is not clear how the thresholding of input features works, requiring the reader to read the original paper to fully understand it.\n\n- The architecture is presented as a sort of extension of the GIN architecture, but in practice, the aggregation function in Equation 4 shares only the sum over neighbors with the original GIN, making the connection not particularly strong. For example, GIN aggregates the self-loop by multiplying the $\\epsilon$ parameter, which is neglected in LogicX-GIN by setting $\\epsilon=1$. I believe the architecture can be presented in more general terms without grounding on GIN, but by referring to any GNN architecture with sum aggregation.\n\n- The proposed solution, in particular Equations 5 and 6, is introduced abruptly. The clarity of the proposed solution would greatly benefit from more intuition and explanation regarding it. I also checked Appendix A, but it is not enough to have a feeling of the functioning of that operation. I think that providing some toy analytical examples would be useful to understand how this counting-based step function works. \n\n- There is a mismatch between practice -- Eq. 5 in particular -- and the theory in Proposition 1. The proof is based on the fact that summing over $h_u^k$ yields, for each hidden dimension $j$, the count of neighbors where $h_u^k[j]=1$. Note that this is true as long as each $h_u^k[j]$ is binary (taking values either 1 or 0). However, according to Eq. 7, each $h_u^k[j]$ is not necessarily binary, as the sigmoid function can yield any value in between 0 and 1.\n\n- There is a conceptual contradiction regarding the pretraining stage. The authors claim that the sigmoid function of LogicX-GIN can create issues with model training, and therefore propose to train a standard GIN model to \"initialize\" model weights. Nonetheless, this standard GIN is still implemented with a sigmoid activation function, which supposedly inherits the same problem of LogicX-GIN (since they both use sigmoid activations). Even more, the standard GNN employs a Gumbel-sigmoid activation, which arguably makes the training even more unstable. Then, is it really the sigmoid activation function the problem when training LogicX-GIN from scratch?\n\n- The paper lacks the fundamental part of explaining how explanations are extracted from LogicX-GIN. The authors broadly refer to *the activations of the literals after the readout function can be interpreted as indicative patterns whose presence directly affects the final prediction* in line 292. Nonetheless, this is not trivial, and being part of the key contribution of the paper, it is definitely not explained well enough.\n\n\n\n# Minors\n\n- In line 1: Graphs do not necessarily *require* GNNs. There exist many alternatives to GNNs, like Graph Transformers and kernel-based approaches. Please update this line.\n\n- Given that the paper proposes to move beyond the usual paradigm of topological explanations, it might be worth including SE-GNNs that are not intrinsically topological, like [1,2,3]. This would make the related work section more complete.\n\n- Line 119: Please add some references to support the claim that post-hoc methods are not properly faithful.\n\n- Line 148: The statement: *The sum aggregation can distinguish between different neighborhood structures without ambiguity* is not correct. In fact, albeit expressive, GIN still has not reached full discrimination power as it is still upper bounded by 1-WL (see GIN's paper).\n\n- Line 157: *TELL is designed to be interpretable in terms of logic* sounds a bit vague. Please reformulate this statement to make it more precise.\n\n- Line 157: What do you mean by *feed-forward operation*? Is this a linear layer?\n\n- Equation 3: What is the difference between any $\\psi_i(x)$ and $\\theta(y)$? If they are simply different GML formulas, I would rather make the notation uniform and replace $\\theta$ with $\\psi_3$.\n\n- It is not clear whether the statement *which is necessary to represent counting operations in GML* in line 210 refers to the sum aggregation or to the fact that $\\epsilon=1$.\n\n- The caption of Figure 1 is not self-contained. Details should be added, like what is *alignment* (a proper formal definition is lacking even in Sec. 5), and whether those explanations are global or local.\n\n\n\n[1] Graphchef: Learning the recipe of your dataset. IMLH 2023\n\n\n[2] The intelligible and effective graph neural additive network. NeurIPS 2024\n\n\n[3] Beyond Topological Self-Explainable GNNs: A Formal Explainability Perspective. ICML 2025", "was_revised": true, "final_justification": "The authors did make a substantial attempt at addressing my concerns, which made me increase my score to weak accept.\n\nBelow is a justification of why I'm not in support of full acceptance.\n\nDespite the clarity of the proposed contribution being increased after the authors' clarifications, some major concerns still apply. As also observed by other reviewers, the paper struggles in providing unambigous evidence that the final formulas are indeed more interpretable (see Qw5B), and the fact that the proposed approach requires both a pre-training (otherwise the model does not train properly) and a post-training (otherwise explanations are overly complex and less interpretable) makes the overall model unstable and arguably not easy for usage in new downstream tasks.", "rebuttal": "We thank the reviewer for the thorough work. We in the following we address the raised concerns raised by the reviewer, hoping to improve their concerns regarding quality and clarity.\n\n1. **TELL presentation and thresholding function.**\n    Due to space limits, we opted for a concise presentation of background work like TELL. However, we acknowledge the need for clarity, especially for the thresholding. Thanks to a comment from another reviewer, we found a typo in the thresholding eq. that impacted readability. We have corrected it in 169 to match the correct formulation, which the same from Ragno et al. [19]:\n$$\n\\tilde{x} = \\sigma(X \\odot \\exp(\\tilde{W}) + \\tilde{b}).\n$$\n    That said, we use a different thresholding to operate the counting semantics of GML. This is described in Eq. 5 using a novel learnable Fourier step function. To improve clarity in the revised paper, we rewrite lines 211–214 as follows: “To apply logic-based rules, we binarize them using a novel thresholding mechanism. Specifically, we need a function capable to activate if its input falls within certain intervals. For this reason, we design a parametric Fourier step function, which flexibly and differentiably activates (outputs one) when counts fall within learnable value ranges.”\n2. **GIN Extension.**\n    We acknowledge that we use a \"restricted\" form of GIN. However, referring to our approach as a GNN architecture with sum aggregation would be too broad. We believe the link to GIN to be meaningful, especially since in Xu et al. [26], the authors show that  $\\epsilon = 0$ can in fact improve the performances. As pointed out by another reviewer, there was a typo: we actually use $\\epsilon = 0$. This is also consistent with the default configuration in PyG, with $\\epsilon=0$ and not trainable.\n3. **Sigmoid binary values.**\n    The observation is accurate: Eq. 7, as currently written, applies a standard sigmoid function, which yields continuous outputs in  $[0,1]$, and this would invalidate the counting assumption used in Prop. 1 if left unaddressed. In practice, however, we overcome this issue by using a temperature term: we divide the input of the sigmoid by a small $\\tau$ (set to $\\tau = 10^{-4}$) to have outputs very close to 0 or 1. This was not explicitly mentioned in the paper. We have now corrected Eq. 7 to make this explicit:\n    $$h^{(k)}_v = \\lambda^{(k)}\\left( \\tilde{a}^{(k)}_v \\right) = \\sigma\\left( \\frac{ \\tilde{a}^{(k)}_v W^{+(k)} + b^{(k)} }{\\tau} \\right), \\quad \\text{with } \\tau = 10^{-4}    $$\n4. **Problems of the sigmoid.** We realize our explanation may have been unclear. The issues related to the sigmoid activation arise in TELL due to its specific usage: the sigmoid is applied at multiple points, to binarize inputs, to constrain weights to be non-negative, and to produce the final output. These uses hinder gradient flow in multi-layer settings. In contrast, the sigmoid activation in a standard GIN, even when replaced with a Gumbel-sigmoid as in our pretraining phase, does not introduce the same optimization difficulties. Indeed, our black-box GIN model with sigmoid is stable and achieves performances aligned with prior works. Without pretraining, LogiX-GIN does not learn at all. \n5. **Global Explanation Procedure.**\n   We understand that the procedure to obtain the patterns in Figure 1 was not properly explained, we then propose to add a new Appendix section. In line 293, we insert the following sentence: \"Further details on the procedure to extract global rules are available in Appendix G\". Additionally, we will introduce the following explanation in Appendix G. For spaces constraints in the rebuttal we cannot include the full section. However, to give an idea, we use the TELL rule extraction system on the last layer to find the nodes that activate logic rules. Then, we extract isomorphic subgraphs using Cordella, L. et al. (2001) to determine visual rules.\n6. **Minors:**\n    - **Graphs do not need GNNs.** We agree with the reviewer and we will change to \"Graphs are complex, non-Euclidean structures that require specialized models, such as Graph Neural Networks, Graph Transformers, or kernel-based approaches, to effectively capture their relational patterns.\"\n    - **Non-topological SEGNNs are not cited.** We propose the following two edits: (1) Lines 35-37 \"However, existing SE-GNNs present some limitations: topology-based approaches [14, 5, 18, 32, 17] identify the most relevant subgraphs while providing little insight into the logical rules underlying how the model represents graphs and makes its decisions [15, a]; symbolic approaches propose distillation of the models into simpler or more interpretable architectures such as decision trees; and other approaches restrict the focus on node features, without using information about substructures [b,c]\"; (2) In line 122-125, \"Taking a different direction, Pluska et al. [15] and Muller et al. [a] propose distilling symbolic models from a GNN using decision trees. Despite this difference, their methods are either post-hoc, as the rules are extracted from graph embeddings, or the final models is are longer a GNN and cannot be reused in other neural network settings, thus losing the advantages of neural representations\"; \n    [a] Graphchef: Learning the recipe of your dataset. IMLH 2023\n    [b] The intelligible and effective graph neural additive network. NeurIPS 2024\n    [c] Beyond Topological Self-Explainable GNNs: A Formal Explainability Perspective. ICML 2025\n    - **Missing Citations.** We acknowledge the lack of citations pointing to evidence and we propose to put [1,15] after the assertion in line 119.\n    - **WL test.** We propose the following edits on lines 147-151: \"The sum aggregation can distinguish between different neighborhood structures and the MLP update function provides the necessary expressive capacity to obtain meaningful representations, enabling GIN to approximate the 1-WL (Weisfeiler-Lehman) graph isomorphism test, which is a strong baseline for distinguishing non-isomorphic graphs.\"\n    - **TELL is designed to be interpretable in terms of logic.** We understand that the sentence might not be clear, for line 115, we propose the following edit: \"TELL is designed to be interpretable through a direct conversion into first-order logic formulas.\"\n    - **Feedforward operation.** The reviewer is indeed right, we can say it is a linear layer with a non linear activation function. Thanks for pointing it out. Indeed converting it to \"a standard linear layer\" is more appropriate.\n    - **GML formula.** We opted for $\\theta(y)$ as it is applied to neighboring nodes. However the reviewer stands corrected, changing it to $\\psi_3(y)$ does not impact the correctness of the definition. We will follow the suggestion of the reviewer and change it to avoid reading ambiguities.\n    - **Sentence unclarity.** The outlined sentence is referred to the aggregation scheme. However, to increase readability we propose the following change in line 209: \"This aggregation is equivalent to the one of GIN when setting...\"\n    - **Figure 1 Caption.** We agree with the reviewer that Figure 1 deserves a more detailed caption. For this reason, we propose the following new caption:\n   \n   \"Visual comparison between rules extracted with GraphTrail and LogiX-GIN. For LogiX-GIN, the global rules on subgraphs are obtained from the last classification layer following the same procedure of Ragno et al. [19]. Alignment is measured as the accuracy between the predictions of the model and the logic activations of the rules following  Armgaan et al. [1]. Additional details about the explanations extraction are provided in Appendix G.\"\n   \n   \n7. **Questions:**\n    1. **Role of the clamp.** As visible in Figure 4, the fourier step function, when applying low $\\tau$ values, might have oscillating values around one (very close to 1). While we can afford having values that are slightly lower than 1, in case they are slightly bigger than one we would get out of the [0,1] interval. However this does not impact the interpretability as we use a very high $\\tau$ values.\n    2. **$\\tilde{\\alpha}$ are vectors?** They are actually vectors. Indeed, as suggested by the reviewer, we do mean $\\tilde{\\alpha}[j]=1$, thanks for the correction. We will edit the sentence as suggested. \n    3. **Hoyer loss only in pruning?** During training we follow a classical approach using L2 regularization to ensure smooth learning. During pruning, instead, the Hoyer Loss applies a strong regularization that we only want to apply once the model is trained, in order to avoid training instabilities.\n    4. **XAI Evaluation** Given the counting operation, GNNs might learn that a certain class is determined if there exist at least n nodes that have a certain embedding activated, as in the case of Figure 2. Therefore, if we use soft input attributions and we remove nodes one by one, as long as we stay within the range, the prediction does not change. This is actually important to keep in mind because most of the evaluation mechanisms for post-hoc approaches remove the most important nodes up to certain sparsity levels and evaluate prediction changes.\n    5. **Complex embeddings, like textual embeddings?** Following the suggestion of another reviewer we have decided to mark this as a limitation as it would be impossible to interpret the logic rules on \"non-interpretable embeddings\". The model would be still convertible into rules, however we the input literals wouldn't be interpretable. We leave to future work the exploration of other techniques to augment such interpretability through concepts or prototypes."}, {"review_id": "OArB7zqL3i", "reviewer": "Reviewer_Vtmq", "summary": "This paper combines logical explanation methods with self-explaining models by proposing a novel self-explaining graph neural network, **LogiX-GIN**. The model aims to automatically generate interpretable rules that adhere to *Graded Modal Logic (GML)*, while maintaining predictive performance and offering faithful, transparent explanations for its decisions.", "questions": "1. Consider including a diagram of the LogiX-GIN pipeline to illustrate data flow, binarization, distillation, and rule generation.\n2. Please reorganize the notation system and add a comprehensive notation table.\n3. Provide concrete examples of GML formulas to improve accessibility.\n4. The logic extraction seems to focus on input feature–output feature relationships. Yet, the experiment on “faithful global logic rules” presents node-level rules. How is this abstraction from feature-level to node-level achieved?", "limitations": "Yes", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 1, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n* While most prior work on logical explanation methods has focused on post-hoc techniques, this paper explores their integration into self-explaining models.\n* The proposed LogiX-GIN model provides interpretable explanations across multiple datasets without sacrificing predictive accuracy.\n\n## Weaknesses\n\n1. **Unsupported Claims**:\n   Several claims lack proper justification or references.\n\n   * Lines 117–119 state that LENs and instance-level post-hoc methods are not faithful, but no evidence or citations are provided.\n   * Lines 300–302 assert that LogiX-GIN highlights aromatic carbon structures as mutagenic, in contrast to GraphTrail’s NO₂ detection. However, the mutagenicity of simple aromatic rings is not well-established without specific functional groups like NO₂ or NH₂. These claims require citations or qualification.\n\n2. **Inconsistent and Unclear Notation**:\n   The paper lacks a notation table, and many symbols are ambiguous.\n\n   * In Line 159, the equation \\$y = \\sigma(XW^+ + b)\\$ uses \\$X\\$ and \\$y\\$ as vectors but with inconsistent capitalization. Since vectors are typically column vectors, the multiplication \\$XW^+\\$ is dimensionally incorrect—it should be \\$y = \\sigma(W^+x + b)\\$ if \\$x\\$ is a column vector.\n   * Similar notation issues occur in Equation 7 and Line 170, where \\$\\tilde{W} \\in \\mathbb{R}^I\\$ should be lowercase to avoid confusion with matrices.\n   * The semantics of \\$E(x, y)\\$ and the \\$\\models\\$ symbol are not defined.\n\n3. **Logical and Mathematical Errors**:\n\n   * The statement \\$y^{\\text{bin}}\\[j] = 1 \\Leftrightarrow \\sum\\_{i \\in S} W^+\\[i, j] > -b\\[j]\\$ omits the dependence on input \\$x\\$, contradicting the earlier expression \\$y = \\sigma(XW^+ + b)\\$.\n   * Line 169’s \\$\\tilde{x} = \\sigma(X \\odot \\exp(\\tilde{W}^+ \\tilde{b}))\\$ does not logically lead to the binarization formula in Line 171, and \\$\\tilde{W}^+\\$ is undefined.\n   * Lines 209–210 incorrectly state that the aggregated nodes \\$N(v) \\cup {v}\\$ correspond to \\$\\epsilon=1\\$ in Equation 2, when they actually match \\$\\epsilon=0\\$.\n\n4. **Unclear Formalism**:\n\n   * Lines 183–185 introduce GML formulas but use recursive expressions like \\$\\psi(x), \\psi\\_1(x), \\psi\\_2(x), \\theta(y)\\$ without explaining what constitutes a GML formula.\n   * Similarly, Lines 186–192 use the \\$\\models\\$ symbol in both definitions and explanations without clarification.\n   * The training procedure in Section 4 is under-specified, including a lack of clarity on the loss function and the distillation mechanism.\n\n5. **Limited Experimental Validation**:\n\n   * The experimental comparison only includes standard GIN and LogiX-GIN. Other self-explaining GNNs—such as VGIB, KerGNN, and Pi-GNN, all mentioned in the paper—should also be included for a more comprehensive evaluation.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the time and the thorough evaluation. We will address the concerns hoping to clarify and further improve the quality of our method\n1. **Unsupported Claims:**\n    - **GLGExplainer faithfulness.** The unfaithfulness of GLGExplainer (which demands for a post-hoc explanation to be run first and on LENs) was demonstrated by Armgaan et al. [1] while the unfaithfulness of LENs was demonstrated by Ragno et al. [19]. We acknowledge the lack of citations pointing to evidence and we propose to put [1,15] after the assertion in line 119.\n    - **Aromatic carbon structures as mutagenic.** We apologize for the lacking citation. Aromatic structures are well known to be involved with mutagenicity from Debnath et al. [8] and was also reported in Armgaan et al. [1]. To fix the lack of citations we therefore propose to put [1,8] after the assertion in line 302.\n2. **Inconsistent and Unclear Notation:**\n    - We propose to the modify the defintion of $X$ in line 159 as follows \"$X\\in\\mathbb{R}^{1\\times I}$ to remove the doubt of whether the input is a row or a column vector. We indeed referred to it as row vector, given that generally in ML and DL, they are row vectors.\n    - We propose to correct the definition of $\\tilde{W} \\in \\mathbb{R}^{1\\times I}$ in Equation 7 and 170, following the same idea.  \n    - We propose to insert the notation for $E(x,y)$ by modifying the sentence in line 178 with \"... a symbol $E(x,y)$ for a binary relation between representing adjacency between nodes $x$ and $y$ of an undirected graph...\" and the definition of the symbol $\\models$ with with the edit in lines 186: \n\n\"Given a graph $G$ and a node $v$, the relation $G, v \\models \\varphi(x)$ that indicates that $v \\in G$ satisfies a generic GML formula $\\varphi(x)$ is defined as follows:\n- $G, v \\models P(x)$:\nnode $v$ satisfies the atomic predicate $P$ (i.e., $v$ belongs to the unary predicate $P$),\n- $G, v \\models \\neg \\psi(x)$:\nnode $v$ does not satisfy the formula $\\psi(x)$ (negation),\n- $G, v \\models \\psi_1(x) \\land \\psi_2(x)$:\nnode $v$ satisfies both subformulas $\\psi_1(x)$ and $\\psi_2(x)$ (logical conjunction),\n- $G, v \\models \\psi_1(x) \\lor \\psi_2(x)$:\nnode $v$ satisfies at least one of the subformulas $\\psi_1(x)$ or $\\psi_2(x)$ (logical disjunction),\n- $G, v \\models \\exists^{\\geq N} y\\, (E(x, y) \\land \\theta(y))$:\nnode $v$ has at least $N$ neighbors $u$ such that $(v, u) \\in E$ and each $u$ satisfies $\\theta(y)$ (quantified neighborhood condition).\"\n\n3. **Logical and Mathematical Errors:**\n    - The statement $y^{\\text{bin}}[j] = 1 \\Leftrightarrow \\sum_{i \\in S} W^+[i, j] > -b[j]$ is a conclusion that was already mathematically demonstrated by Ragno et al. [19] (TELL) and does not constitute our contribution. Indeed we report it in the background section of our paper, being a contribution of a previous work. Nonetheless, Ragno et al. [19] show that the equation is true thanks to the fact that inputs are binary and weights are non-negative, giving the possibility of obtaining rules without depending on the input $x$.\n    - The expression in 169 is reported with a typo (indeed in the question, the reviewer has tried to fix the typo by adding a final parenthesis). The correct form is $\\tilde{x} = \\sigma(X \\odot \\exp(\\tilde{W})+ \\tilde{b})$. Also this is a result of Ragno et al. [19] and is presented in the background as it does not constitute our contribution.\n    - We thank the reviewer for the correction about $\\epsilon$. The reviewer is correct and our work corresponds to $\\epsilon$ = 0, we will correct it.\n\n4. **Unclear Formalism:**\n    - **Inductive definition of GML.** We apologize if the presentation was not immediately clear. The use of recursive expressions reflects the inductive nature of the definition of GML formulas. This recursive style is standard in formal logic, where formulas can be nested and composed using logical operators. Each formula on the right-hand side of the grammar rule is itself a GML formula, defined by the same inductive process.\n    - **Undefined symbol.** Regarding the use of the satisfaction symbol $\\models$, this follows standard logical notation: $a \\models b$ means that the structure or element $a$ satisfies the formula or condition $b$. Nonetheless, to improve clarity for readers who may be less familiar with this convention, we propose the edit in lines 186 as already specified.\n\n5. **Limited Experimental Validation:** We agree with the reviewer with the necessity to include the comparison in classification performances with other SE-GNNs. We didn't include them in the paper as we were more concerned about the loss of performance of LogiX-GIN w.r.t GIN. However, we will add in the appendix the following comparison table where we also include GNAN, as asked by another reviewer. However we want to highlight that the official code of GNAN does not allow for reproduction of the results provided in the paper (as also pointed out in an issue in their public github repository). For this reason we provide here also the results from the paper to show that our model provides performances that are in line with other state of the art models, with the additional capacity of providing full logic rules for its behavior.\n\n| Model     | Ba2Motifs          | BaMultiShapes        | MUTAG               | Mutagenicity        | NCI1                | PROTEINS            | BBBP                |\n|-----------|--------------------|----------------------|---------------------|---------------------|---------------------|---------------------|---------------------|\n| LogiX-GIN |  100.00 ± 0.00     |  100.00 ± 0.00       |   82.63 ± 5.58      |  79.31 ± 1.25       |  78.93 ± 1.51       |   72.05 ± 5.77      |  85.90 ± 0.99       |\n| PiGNN     |  99.89 ± 0.31      |  85.40 ± 5.08        |   82.51 ± 10.48     |  82.39 ± 1.68       |  78.54 ± 2.74       |   70.00 ± 2.44      |  83.54 ± 0.37       |\n| GIB       | 100.00 ± 0.00      |  97.60 ± 2.06        |   90.53 ± 6.14      |  80.14 ± 0.98       |  78.15 ± 1.32       |   68.47 ± 5.00      |  84.78 ± 2.57       |\n| KerGNN    | 98.80 ± 0.75       |  83.20 ± 1.94        |   82.11 ± 9.18      |  73.32 ± 2.93       |  69.00 ± 1.38       |   72.97 ± 4.87      |  84.12 ± 2.09       |\n| GNAN      | 49.10 ± 0.66       |  49.10 ± 0.66        |  55.79 ± 16.43      |  56.05 ± 1.49       |  50.80 ± 1.15    |   57.67 ± 2.62      |  41.22 ± 26.05       |\n|GNAN (From original paper) |       -          |         -            |        -            |  72.20 ± 1.00       |  76.90 ± 1.2        |   73.20 ± 3.10      |        -            |\n\n6. **Questions**\n    - **Diagram of the LogiX-GIN pipeline to illustrate data flow, binarization, distillation, and rule generation.** Unfortunately we cannot include any image in the rebuttal, however we have created a diagram that we put in the appendix and in the final repository.\n    - **Please reorganize the notation system and add a comprehensive notation table.** We thank the reviewer for the suggestion. Space constraints in the main text do not allow us for a table, however, we will include such Table in the appendix.\n    - **Provide concrete examples of GML formulas to improve accessibility.** We have improved our presentation of the GML as already replied. However, this is an interesting idea, we will use the appendix to provide such examples together with the notation table.\n    - **How is this abstraction from feature-level to node-level achieved?** We have realized, also thanks to other reviewers, that the global patterns extraction was not well detailed. We have therefore decided to add an appendix section for that. However, since we do not have space here to present it, we will give you an intuition. The visual rules are extracted from the last tell layers, which uses activations of the convolutional layers to predict the output class. Since this last layer is a TELL, we use TELL extraction procedure for rules. Successively, for each rule, we extract from the training set all the isomorphic subgraphs (using using Cordella, L. et al. (2001)) that activate for such rules."}, {"review_id": "a8fnY7pFrP", "reviewer": "Reviewer_Qw5B", "summary": "The paper presents an interpretable graph neural network. Its interpretability stems from binarizing each layer’s input and constraining the layer weights to be non-negative. These weights are translated into graded modal logic expressions that are more suitable for GNNs than binary expressions. The pipeline goes like this: Node features are binarized using TELL’s preprocessing step, which learns weights to threshold the features. Aggregating the binary features generates an integer-valued vector. A parametric Fourier step function is used to binarize it. The step function’s weights learn the integer ranges in which dimensions activate, acting as a proxy for counting. The ability to count facilitates the use of graded modal logic. The authors mention that their use of the sigmoid function hinders initial training and suggest pre-training the interpretable layers against a black-box GIN, where the interpretable layers emulate the GIN layers’ weights binarized using Gumbel-sigmoid activations. The proposed method’s novelty lies in using the Fourier step function as a proxy for counting facilitating the use of TELL layers on graphs.", "questions": "1. How is the non-negative constraint on weights preserved during gradient descent? Methods like non-negative matrix factorization (NMF) use hand-crafted updates to ensure such behavior.\n\n2. I have assumed the input node features are binarized using TELL’s preprocessing as mentioned in line 168. Is this correct?\n\n3. Line 223: Is the output of the sigmoid function binarized? If yes, what threshold is used?\n\n4. Why are the layers pruned individually, and why do it in reverse?\n\n5. How is the black-box GIN modified for pre-training so that the interpretable layer can mimic it? For example, are the hidden weights binarized actively during training or at the end of each epoch? Please provide these details.\n\n6. Is “alignment” the rule’s accuracy against the model’s predictions? And in LogiX-GIN, the model and rules are one and the same resulting in 100\\% alignment. Is my understanding correct?\n\n7. Please provide a figure similar to figure 2 for the non-mutagenic class of MUTAG and the negative class in BA2Motifs. I’m curious to see the explanations of the negative classes.\n\n8. Line 311: “exhibits less complexity and higher versatility than GraphTrail” What does versatile mean here? I contest the complexity part. Table 2 shows 651 active rules for Mutagenicity at layer 5 compared to 1 rule given by GraphTrail.\n\n9. Given the 243 active rules for BA2Motifs, how did the authors single out the explanation shown in figure 1. Same question for 829 active rules for BAMultiShapes, and Mutagenicity with 651 rules.", "limitations": "The authors have mentioned most of their limitations. There are two that should also be included:\n1. The inability to utilize edge features should be stated in the limitations.\n\n2. Input binarization is appropriate when dimensions are independent and semantically meaningful—such as one-hot encoded atom types, categorical indicators, or continuous scalar features with clear interpretations (e.g., atomic radius). However, it is not suitable for dense embeddings like GloVe, where individual dimensions lack interpretability and meaning is distributed across the entire vector. Thresholding assumes each component is interpretable, which is not the case with such embeddings. The presence of specific values in certain dimensions may not correspond to anything meaningful, as the semantics are not localized. This limits the datasets on which the proposed method remains interpretable and should be clearly stated as a limitation.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 1, "strengths_and_weaknesses": "# Strengths\n1. Graded Modal Logic is a good fit for GNNs and is likely to motivate future research on global explanations.\n2. The Fourier Step function is a neat way to binarize integer inputs.\n3. The interpretable model matches black-box performance on molecular and synthetic data.\n4. The authors have highlighted some of their model’s limitations.\n\n# Weaknesses\n1. **Limited originality:** The proposed method draws heavily from TELL (ECAI 24), with much of the architecture and pipeline inherited directly. The authors' contribution lies in using the Fourier step function as a proxy for counting facilitating the use of TELL layers on graphs. Other core aspects of the pipeline like input binarization, constrained positive weights and their conversion to graded modal logic are reused from TELL.\n\n2. **Missing baseline**: GNAN (NeurIPS '24) is a natural baseline for this work, yet it has not been compared against. Here’s the citation for your reference: Bechler-Speicher, Maya, Amir Globerson, and Ran Gilad-Bachrach. \"The intelligible and effective graph neural additive network.\" Advances in Neural Information Processing Systems 37 (2024): 90552-90578.\n\n3. Line 118: “demonstrated to not be properly faithful” This statement needs supporting evidence.\n\n4. **Binarizing vectors holding a semantic meaning**: Input binarization is appropriate when dimensions are independent and semantically meaningful—such as one-hot encoded atom types, categorical indicators, or continuous scalar features with clear interpretations (e.g., atomic radius). However, it is not suitable for dense embeddings like GloVe, where individual dimensions lack interpretability and meaning is distributed across the entire vector. Thresholding assumes each component is interpretable, which is not the case with such embeddings. The presence of specific values in certain dimensions may not correspond to anything meaningful, as the semantics are not localized. This limits the datasets on which the proposed method remains interpretable and should be clearly stated as a limitation.\n\n5. **Sigmoid struggling on 5 layers**: The paper mentions that the model struggles with initial learning due to its use of sigmoid functions. GNN’s are typically 2-5 layers deep. The authors have also used 5 layers. That’s not deep. Depth isn’t the reason the model is struggling. An empirical proof needs to show that the detrimental effect on training at such a shallow depth is caused by the sigmoid function.\n\n6. **Invalid pruning assumption**: Pruning layers individually seems problematic. Suppose we prune layer $L$, selecting a few prominent weights based on its current input. If we then prune layer $L−1$, its output changes, thus altering the input to layer $L$. This is problematic because layer $L$ was pruned based on the original input, which is no longer valid.\n\n7. **Dataset diversity**: All the datasets used in the paper are synthetic or molecular. Experiments on continuous valued features, especially the ones with dense features, e.g., GloVe embeddings, are missing.  A case study similar to MUTAG is needed on such datasets.\n\n8. **Spurious agreement between explanation and performance**: BAMultiShapes explanation is incorrect even though it exactly matches the one reported in GLGExplainer and PyTorch Geometric. It doesn't exclude all three motifs appearing together. Let me explain as there’s a bigger problem at hand. GLGExplainer’s authors have used one set of formulae to create the dataset (elaborated in their section 4.1) but visualized an incorrect version which has been picked up by PyG. Please see this [anonymous repo](https://anonymous.4open.science/r/bamultishapes_proof-BFD9) for the experiment proof. The formula presented in figure 1 and the corresponding results in table 1 makes me skeptical of the validity of the other results as well. That formula should not achieve 100% accuracy as the experiment should make clear.\n\n9. **Too many rules to be considered interpretable**: Even in simple, synthetic datasets like BA2Motifs, BAShapes, and TreeGrids, the last layer has 243, 46 and 26 active rules post-pruning. In these datasets, the class is identified by a single motif. Why do we need 243 rules for it? This number goes to 651 and 956 for Mutagenicity and NCI1. These many rules, even though individually interpretable, cannot be considered human interpretable in any meaningful sense. The cognitive load of understanding all 500+ rules together is too high for a human to grasp. In contrast, post-hoc global explainers generate a single concise formula for each of these datasets offering significantly greater human interpretability.\n\n10. **Instability:** The model is unstable. Table 2: BAMultiShapes has 829 active rules with an std of 360; Mutagencity has 651 active rules with an std of 530.\n\n11. **Ablations:** The authors should conduct the following ablation studies:\n    - Validate the need for pre-training by presenting results with and without pre-training.\n    - Pruning layers individually vs. pruning the model as a whole.\n\n12. The inability to utilize edge features should be stated in the limitations.\n\n13. Figure 6 is poorly visualized: nodes are very small; they should be colored instead of their outline being highlighted (which is barely visible); node IDs are irrelevant to the figure and should be replaced by node types.\n\n14. Minor corrections:\n    - Line 140: “using from” a word seems to be missing.\n    - Figure 2’s font is too small.\n    - In figure 10, $v’ \\in N(v)$ should be replaced with $v’ \\in N(v) \\cup {v}$\n    - Include the number of available cores in section G.\n    - Start the checklist on a new page.", "was_revised": true, "final_justification": "In light of my discussion with the authors, I am raising my score from **reject** to a **borderline accept**.\n\nI still find the novelty limited by NeurIPS standards. However, there is precedent for similar extensions. For instance, one of the works cited by the authors, LEN [4], has been employed similarly for GNN explanations in GLGExplainer (ICLR 2023).\n\nI found the experimental section in the paper unconvincing. However, the additional experiments, ablations, and clarifications provided in the rebuttal have positively swayed my assessment.\n\nThat said, I still find the alignment experiment quite odd and misleading, and I don’t think it adds value to the paper.\n\nI appreciate the authors’ acknowledgment of certain limitations, which helps clarify the scope of the work.\n\nAbove all, the paper most certainly needs a detailed methodology subsection which demonstrates extraction of explanations in the form of graded modal logic, as that is THE selling point of the paper. While the authors have indicated that this is implemented in code, it is essential that it be presented explicitly in the paper.\n\nIn its current form, which parts of the pipeline are intended for interpretability and how remains a bit ambiguous to me. There are multiple potential candidates and their interplay: weights, GML equivalents, activations, and isomorphic subgraphs. The paper would benefit from describing the intended use of each these aspects of the model’s output from an end-user's perspective. The authors have partially addressed this in the rebuttal, and incorporating that clarification into the paper would aid future readers.", "rebuttal": "We thank the reviewer for the suggestions and comments.\n1. **Limited originality.** We understand the concerns of the reviewer. However, we would like to point out that integrating such constraints in the graph domain is not trivial and has required several changes in the architecture. In TELL, all the experiments were conducted with 1-layer model and no pruning strategies were presented.  \n2. **Missing baseline.** We thank the reviewer for the comment. We agree on the need to add the comparison with GNAN as it is also pointed out by other reviewers. Also, as highlighted by reviewer Vtmq, a comparison of the classification performances with the other SEGNNs was missing. We have also GNAN among them. We report the results in the table in the replies to reviewer Vtmq. However we want to highlight that the official code of GNAN does not allow for reproduction of the results provided in the paper. For this reason we provide  also the results from the GNAN paper to show that our model provides performances that are in line with other state of the art models.\n\n3. **Missing citations.**   We will include proper support for the statement of line 118 by proposing to insert [1,15] as citations.\n4. **Binarizing vectors holding a semantic meaning.** We understand the limitation in case of continuous input node features. However, the limitation is only present when input features do not have a semantic meaning. While in GloVe we agree that thresholding on input features is difficult to interpret, in chemistry this is not true: thresholding on the sum of atom radius in a certain neighborhood can actually be interpreted as whether the model is recognizing areas with either high or low \"steric hindrance\", a property that plays an important role in chemistry. The same applies for instance with the atom charge. However, we propose the following addition in the conclusions in line 366, to highlight it as a limitation: \"... composed of multiple steps. Also, thresholding on input features might not be interpretable in cases where they are not attributed with specific semantic meanings. Future work could investigate case the integration with prototype-based or concept-based techniques  that could improve interpretability thanks to their case-based reasoning.\"\n5. **Sigmoid Problems.** We realize our explanation may have been unclear. The issues related to the sigmoid activation arise in TELL due to its specific usage: the sigmoid is applied at multiple points, to binarize inputs, to constrain weights to be non-negative, and to produce the final output. These uses hinder gradient flow in multi-layer settings. In contrast, the sigmoid activation in a standard GIN, even when replaced with a Gumbel-sigmoid as in our pretraining phase, does not introduce the same optimization difficulties. Indeed, our black-box GIN model with sigmoid is stable and achieves performances aligned with prior works. Without pretraining, LogiX-GIN does not learn at all.\n6. **Invalid pruning assumption.** When pruning, we do not look at layers as single entities. Layers are pruned one-by-one, in the sense that when pruning a layer, the parameters of the others are freezed. However, during pruning we keep cross-entropy to ensure that there is not degradation in the performances. In Table 2, indeed, we show that in some cases, performance even improve.\n7. **Dataset diversity.** We perform evaluation on 10 datasets that are often used in prior work. Relating to the study of the network, we have already agreed that obtaining sematically valid rules on non-semantic continuous inputs is a limitation as would provide non-sematic interpretabtion. However, the model would still report rules that are perfectly matching its behavior, thus keeping its self-explainable nature. \n8. **Spurious agreement between explanation and performance.**  We thank the reviewer for this comment. The reviewer's intuition would is correct. However alignment here is presented following Armgaan et al. [1] to ensure a fair comparison. In their work Armgaan et al. [1] consider alignment as the accuracy between the rules and the model predictions. They consider rules activations not using subgraph presence, but using activations of the symbolic regression over computation trees. With the same method we use rules activation on the logic extracted from the last TELL. As we understand that this is not properly explained we propose to improve its explanation and Figure 1 caption. Additional details are provided in replies to other questions of the reviewers and other reviewers.\n9. **Too many rules to be considered interpretable.** We understand the concerns of the reviewer. Although there might be some problems regarding the semantic and subjective interpretability of the rules, in this work, we are more concerned on objective mathematical explainability given by the equivalence of the logic rules with LogiX-GIN's behavior.\n10. **Instability.**  The high difference on seed variations highly depend on the pre-training strategy. Additionally, the number mentioned identify the amount of non-zero weights, and not the amount of rules. There could be weights that are not zero but still with low values that are not used in the rule. \n11. **Ablations.** We did not include such ablations as they lead to inability of the model to learn. Regarding the removal of pre-training, the model does not learn at all. Regarding pruning layers individually, our strategy is justified by the fact that we do it from the last to the first. If we pruned the model as a whole we would actually end up in the  \"invalid pruning assumption\", as the removal of a weight at layer $L-1$ would critically impact layer $L$.\n12. **Edge-features limitation.** We will include the impossibility of using edge features as a limitations. \n13. **Figure 6 quality.** Although we cannot upload the figure for proof due to rebuttal regulations, we will modifiy the image according to the reviewer's comments.\n14. **Minor corrections:**\n - We thank the reviewer for the correction in line 140.\n - We will increase the font of Figure 2. However, we cannot upload figures/links in the rebuttal.\n - We thank the reviewer for the correction in Figure 10.\n - The mentioned CPU only comes with 10 cores. However we will include the number of cores in section G.\n - We agree with the reviewer and we will start the checklist on a new page.\n15.  **Questions:**\n -  **Non-negative constraint.** Following the results of Ragno et al. [19], we use a combination of the sigmoid and exponential functions: $W^+ = \\exp{W_1} \\odot \\sigma{W_2}$ where $\\odot$ is the Hadamard product. They find that this solution helps because the sigmoid acts as a switch for input selection while the exponential function scales the values to compensate the fact that the component of the sigmoid outputs only values between 0 and 1.\n - **Input node features binarization.** Input node features are not binarized, we directly use LogiX-GIN, which using the Fourier threshold function learns intervals. In case of non-binary features, what happens is that the first layer of the network thresholds over the sum of the features in a certain neighbor. \n - **Sigmoid binarization.** We use 0.5 as threshold. Also, as we replied in response to a comment of Reviewer LV2N, we failed to mention that we use a temperature scalar to enforce binary output values from the sigmoid and therefore proposed to update Eq. 7.\n\n - **Pruning order.** Layers are pruned individually  and in reverse order because of the dependencies between the rules. Assuming layer $L$ uses some information from layer $L-1$ that are not necessary, we have to first prune them from $L$ and only then we can prune $L-1$, otherwise we will impact on the performances.\n- **Pretraining using GIN.** Only for pretraining, we use a GIN model that is trained with Gumbel-sigmoid activation function, thus having binary latent spaces.\n- **Alignment.** The reviewer stands corrected: the alignment it represents the accuracy between model's predictions and rules activations. We will specify this in Figure 1 caption.\n- **Other Layer-wise Rules and Negative Classes.** Unfortunately we cannot provide the figure in the rebuttal phase. We have only included the positive class of MUTAG for space issues, the image wouldn't enter in a paper space without affecting visibility and we thought about putting it in a notebook after publication. Despite this, to answer to your interest, in Ba2Motifs we have interestingly found that the models either had these two behaviors: (1) using rules to find nodes that belong to the loop and counting them (for the opposite class, it would count that it is under 5; (2) using rules to find nodes belonging to the BA distribution, then negating them to select the loops and finally count the nodes in the loop. This is the same in the case of BAMultiShapes, but it checks that the nodes belonging to the shapes are within well defined intervals to avoid returning 1 when all the three are present.\n- **Complexity and versatility of high level logic explanations.** By versatility we mean that it can work also when the model is bigger, whereas GraphTrail does not work in BAMultiShapes in our case. Also, 651 corresponds to non-zero weights, not to the rules. The amount of rules are instead shown in Figure 3. \n- **Figure 1 rules.** Figure 1 shows a different analysis and global rules on patterns, only using the last TELL layer. We acknowledge the limited clarity of the procedure used to extract the rules and for this reason we add an explantion of the procedure in the appendix. To give an intuition, we exploit the last TELL layer to extract rules that activate over aggregated outputs from the convolutional layers. For each rule we extract all the activating isomorphic subgraphs using Cordella et al. (2001).  Additionally, also in this case, it is important to remind that 829 and 651 are not the number of rules, but instead the number of non-zero weights."}]

Author rebuttal:

Meta-review: Most reviewers appreciated the aims and technical contributions of the paper. The author-reviewer discussion was mostly productive, and the authors managed to clarify many of the more serious issues raised by most the reviewers. Residual concerns about incrementality, interpretability and usability are comparatively modest. This makes me lean towards acceptance. (One reviewer was very critical of the work and raised sensible points, but was completely unresponsive to the author's on-point rebuttal.  I have taken this into consideration.)

I warmly recommend the authors to integrate the paper with all the useful suggestions received during the review stage.

(Concerning the "interpretability" debate: my take is that the authors seem to be thinking of faithfulness, while the reviewers are thinking of understandability for actual practitioners. If the authors want to prevent further confusion, they'd better clarify what they mean by "interpretability" in the paper, or to avoid the term altogether.)
Decision: Accept (poster)