SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this case, we can interpret every single layer of our model with concepts of increasing complexity. The concepts extracted from the first layer represent subgraphs corresponding to the 1-hop neighborhood of a node, those extracted at the second layer will correspond to 2-hop neighborhoods, and so on. These hierarchical iGNNs can be useful to get insights into concepts with different granularities. By analyzing the concepts extracted at each layer, we gain a better understanding of the GNN inference and of the importance of different (sub)graph structures for the classification task.
3.2.3 Training
For the classification layer, the choice of the activation and loss functions for iGNNs depends on the nature of the task at hand and does not affect their interpretability. For classification tasks, we use standard activation functions such as softmax or sigmoid, along with standard loss functions like cross-entropy. For hierarchical iGNNs (HiGNNs), we apply the same loss function at each layer of the concept hierarchy, as their layered architecture enables intermediate supervisions. This ensures that each layer is doing its best to extract the most relevant concepts to solve the task. Internal losses can also be weighted differently to prioritize the formation of optimal concepts of a specific size, allowing the HiGNN to learn in a progressive and efficient way.
4.1 Research questions
In this section we analyze the following research questions:
- Generalization - Can GNNs generalize when trained to predict universal algebra's properties? Can interpretable GNNs generalize as well?
- Interpretability - Do interpretable GNNs concepts empirically validate universal algebra's conjectures? How can concept-based explanations suggest novel conjectures?
Baselines. For our comparative study, we evaluate the performance of iGNNs and their hierarchical version against equivalent GNN models (i.e., having the same hyperparameters such as number layers, training epochs, and learning rate). For vanilla GNNs we resort to common practice replacing the Gumbel-Softmax with a standard leaky ReLU activation. We exclude from our main baselines prototype or concept-based GNNs pre-defining graph structures for explanations, as for most datasets these structures are unknown. Appendix C covers implementation details. We show more extensive results including local and post-hoc explanations in Appendix G.
Evaluation. We employ three quantitative metrics to assess a model's generalization and interpretability. We use the Area Under the Receiver Operating Characteristic (AUC ROC) curve to assess task generalization. We evaluate generalization under two different conditions: with independently and identically distributed train/test splits, and out-of-distribution by training on graphs up to eight nodes, while testing on graphs with more than eight nodes ("strong generalization" (41)). We further assess generalization under binary and multilabel settings (classifying 5 properties of a lattice at the same time). To evaluate interpretability, we use standard metrics such as completeness (44) and fidelity (37). Completeness<sup>4</sup> assesses the quality of the concept space on a global scale using an interpretable model to map concepts to tasks, while fidelity measures the difference in predictions obtained with an interpretable surrogate model and the original model. Finally, we evaluate the meaningfulness of our concept-based explanations by visualizing and comparing the generated concepts with ground truth lattices like e.g.  $\mathbf{M}_3$  and  $\mathbf{N}_5$ , whose omission is known to be significant for modular and distributive properties. All metrics in our evaluation, across all experiments, are computed on test sets using 5 random seeds, and reported using the mean and  $95\%$  confidence interval.
5 Key Findings
5.1 Generalization
iGNNs improve interpretability without sacrificing task accuracy (Figure 4). Our experimental evaluation reveals that interpretable GNNs are able to strike a balance between completeness and fidelity, two crucial metrics that are used to assess generalization-interpretability trade-offs (37). We observe that the multilabel classification scenario, which requires models to learn a more varied and diverse set of concepts, is the most challenging and results in the lowest completeness scores on average. We also notice that the more challenging out-of-distribution scenario results in the lowest completeness and fidelity scores across all datasets. More importantly, our findings indicate that iGNNs achieve optimal fidelity scores, as their classification layer consists of a simple linear function of the learnt concepts which is intrinsically interpretable (38). On the contrary, interpretable surrogate models of black-box GNNs exhibit, as expected, lower fidelity scores, confirming analogous observations in the explainable AI literature (37; 38). In practice, this discrepancy between the original black-box predictions and the predictions obtained with an interpretable surrogate model questions the actual usefulness of black-boxes when interpretable alternatives achieve similar results in solving the problem at hand, as extensively discussed by Rudin (38). Overall, these results demonstrate how concept spaces are highly informative to solve universal algebra's tasks and how the interpretable graph layer may improve GNNs' interpretability without sacrificing task accuracy. We refer the reader to Appendix E for detailed discussion on quantitative analysis of concept space obtained by iGNNs under different generalization settings with comparisons to their black-box counterparts.
GNNs strongly generalize on universal algebra's tasks (Figure 5). Our experimental findings demonstrate the strong generalization capabilities of GNNs across the universal algebra tasks we designed. Indeed, we stress GNNs test generalization abilities by training the models on graphs of size up to  $n$  (with  $n$  ranging from 5 to 8), and evaluating their performance on much larger graphs of size up to 50. We designed this challenging experiment in order to understand the limits and robustness of interpretable GNNs when facing a significant data distribution shift from training to test data. Remarkably, iGNNs exhibit robust generalization abilities (similar to their black-box counterparts) when trained on graphs up to size 8 and tested on larger graphs. This evidence confirms the hypothesis that interpretable models can deliver reliable and interpretable predictions, as suggested by Rudin
<sup>4</sup>We assess the recall of the completeness as the datasets are very unbalanced towards the negative label.
Figure 4: Accuracy-interpretability trade-off in terms of concept completeness (accuracy) and model fidelity (interpretability). iGNNs attain optimal fidelity as model inference is inherently interpretable, outmatching equivalent black-box GNNs. All models attain similar results in terms of completeness.
(38). However, we observe that black-box GNNs slightly outperform iGNNs when trained on even smaller lattices. We hypothesize that this is due to the more constrained architecture of iGNNs, which imposes tighter bounds on their expressiveness when compared to standard black-box GNNs.
Notably, training with graphs of size up to 5 or 6 significantly diminishes GNNs generalization in the tasks we designed. We hypothesize that this is due to the scarcity of non-distributive and non-modular lattices during training, but it may also suggest that some patterns of size 7 and 8 might be quite relevant to generalize
Figure 5: Strong generalization performance with respect to the maximum number of nodes used in training.
to larger graphs. Unfortunately, running generalization experiments with  $n \leq 4$  was not possible since all such lattices trivially omitted  $\mathbf{N}_5$  and  $\mathbf{M}_3$ . It is worth mentioning that GNNs performed well even in the challenging multilabel case, where they had to learn a wider and more diverse set of concepts and tasks. In all experiments, we observe a plateau of the AUC ROC scores for  $n = 8$ , thus suggesting that a training set including graphs of this size might be sufficient to learn the relevant patterns allowing the generalization to larger lattice varieties. For detailed numerical results across all tasks, we refer the reader to Table 1 in Appendix D. Overall, these results emphasize the potential of GNNs in addressing complex problems in universal algebra, providing an effective tool to handle lattices that are difficult to analyze manually with pen and paper.
5.2 Interpretability
Concept-based explanations empirically validate universal algebra's conjectures (Figure 6). We present empirical evidence to support the validity of theorems 2.3 and 2.4 by examining the concepts generated for modular and distributive tasks. For this investigation we leverage the interpretable structure of iGNNs.
Similarly to Ribeiro et al. (37), we visualize in Figure 6 the weights of our trained linear classifier representing the relevance of each concept. We remark that the visualization is limited to the (top-5) most negative weights, as we are interested in those concepts that negatively affect the prediction of a property. In the same plot, we also show the prototype of each concept represented by the 2-hop neighbor
Figure 6: Ranking of relevant clusters of lattices (x-axis) according to the interpretable GNN linear classifier weights (y-axis, the lower the more relevant the cluster).  $\mathbf{N}_5$  is always the most important sub-lattice to omit for modularity, while both  $\mathbf{M}_3$  and  $\mathbf{N}_5$  are relevant for distributivity, thus validating theorems 2.3 and 2.4.
hood of a node activating the concept, following a similar procedure as Ghorbani et al. (10); Magister et al. (28); Azzolin et al. (2). Using this visualization, we investigate the presence of certain concepts when classifying modular and distributive lattices. For the modularity task, our results show that the lattice  $\mathbf{N}_5$  appears among non-modular concepts, but is never found in modular lattices, while the lattice  $\mathbf{M}_3$  appears among both modular and non-modular concepts, which is consistent with Theorem 2.3. In the case of distributivity, we observe that both  $\mathbf{M}_3$  and  $\mathbf{N}_5$  are present among non-distributive concepts, and are never found in distributive lattices, which is also in line with Theorem 2.4. These findings provide a large-scale empirical evidence for the validity of theorems 2.3 and 2.4, and further demonstrate the effectiveness of graph neural networks in learning and analyzing lattice properties. Overall, these results highlight how interpretable GNNs can not only learn the properties of universal algebra but also identify structures that are unique to one type of lattice (e.g., non-modular) and absent from another (e.g., modular), thus providing human-interpretable explanations for what the models learn.
Contrastive explanations highlight topological differences between properties of lattice varieties (Figure 7). We leverage interpretable GNNs to analyze the key topological differences of classical lattice properties such as join and meet semi-distributivity characterized by relevant quasi-equations (cf. Appendix A.5). To this end, we visualize specific concept prototypes corresponding to lattices that are not meet semi-distributive against lattices that are meet semi-distributive.
We observe  $\mathbf{N}_5$  but not  $\mathbf{M}_3$  among the concepts of meet semi-distributive lattices, while we observe both  $\mathbf{N}_5$  and  $\mathbf{M}_3$  only in concepts that are not meet semi-distributive. This observation suggests that  $\mathbf{N}_5$  is not a key lattice for meet semi-distributive
Figure 7: Contrastive explanations showing lattice varieties with a pair of discording labels to highlight the key difference between join and meet semi-distributivity.
lattices, unlike distributive lattices. Furthermore, we find that the lattice pattern  $\mathcal{V}$  is relevant for non meet semidistributivity, while its dual  $\mathcal{A}$  is relevant for non join semidistributivity, thus empirically confirming the hypotheses of Jónsson and Rival (19). These findings are significant because they demonstrate how analyzing concepts in interpretable GNNs can provide universal algebraists with a powerful and automatic tool to formulate new conjectures based on identifying simple lattices that play a role in specific properties. By leveraging the power of interpretable GNNs, we may uncover previously unknown connections between different properties and identify new patterns and structures that could lead to the development of new conjectures and theorems in universal algebra, providing exciting opportunities for future research in universal algebra.
Relations with Graph Neural Network explainability. Graph Neural Networks (GNNs,(39)) process relational data generating node representations by combining the information of a node with that of its neighbors, thanks to a general learning paradigm known as message passing (11). A number of post-hoc explainability techniques have been proposed to explain the reasoning of GNNs. Inspired by vision approaches (40; 37; 10), early explainability techniques focused on feature importance (35), while subsequent works aimed to extract local explanations (45; 27; 42) or global explanations using conceptual subgraphs by clustering the activation space (28; 46; 29; 22). However, all these techniques either rely on pre-defined subgraphs for explanations (which are often unknown in UA) or provide post-hoc explanations which may be brittle and unfaithful as extensively demonstrated by Rudin (38). On the contrary, our experiments show that iGNNs generate interpretable predictions according to Rudin (38) notion of interpretability via linear classifiers applied on sparse human-understandable concept representations.
Limitations. The approach proposed in this paper focuses on universal algebra conjectures characterized both algebraically and topologically. Our methodology is limited to finite lattices, which may not capture all relevant information about infinite algebraic structures. However, the insights gained from finite-lattice explanations can still provide valuable information regarding a given problem (albeit with potentially limited generalization). Moreover, our approach is restricted to topological
properties on graphs, while non-structural properties may require the adoption of other kinds of (interpretable) models.
Broader impact and perspectives. AI techniques are becoming increasingly popular for solving previously intractable mathematical problems and proposing new conjectures (23; 26; 7; 12). However, the use of modern AI methods in universal algebra was a novel and unexplored field until the development of the approach presented in this paper. To this end, our method uses interpretable graph networks to suggest graph structures that characterize relevant algebraic properties of lattices. With our approach, we empirically validated Dedekind (9) and Birkhoff (5) theorems on distributive and modular lattices, by recovering relevant lattices. This approach can be readily extended—beyond equational properties determined by the omission of a sublattice in a variety (43)—to any structural property of lattices, including the characterization of congruence lattices of algebraic varieties (1; 21; 31; 43). Our methodology can also be applied (beyond universal algebra) to investigate (almost) any mathematical property that can be topologically characterized on a graph, such as the classes of graphs/diagrams with a fixed set of polymorphisms (25; 3; 32).
Conclusion. This paper presents the first-ever AI-assisted approach to investigate equational and topological conjectures in the field of universal algebra. To this end, we present a novel algorithm to generate datasets suitable for AI models to study equational properties of lattice varieties. While topological representations would enable the use of graph neural networks, the limited transparency and brittle explainability of these models hinder their use in validating existing conjectures or proposing new ones. For this reason, we introduce a novel neural layer to build fully interpretable graph networks to analyze the generated datasets. The results of our experiments demonstrate that interpretable graph networks: enhance interpretability without sacrificing task accuracy, strongly generalize when predicting universal algebra's properties, generate simple explanations that empirically validate existing conjectures, and identify subgraphs suggesting the formulation of novel conjectures. These promising results demonstrate the potential of our methodology, opening the doors of universal algebra to AI with far-reaching impact across all mathematical disciplines.
This paper was supported by TAILOR and by HumanE-AI-Net projects funded by EU Horizon 2020 research and innovation programme under GA No 952215 and No 952026, respectively. This paper has been also supported by the Austrian Science Fund FWF project P33878 "Equations in Universal Algebra" and the European Union's Horizon 2020 research and innovation programme under grant agreement No 848077. This project has also received funding from the European Union's Horizon-MSCA-2021 research and innovation program under grant agreement No 101073307.
[1] Paolo Agliano, Stefano Bartali, and Stefano Fioravanti. On Freese's technique. International Journal of Algebra and Computation, 2023. doi: https://doi.org/10.1142/S0218196723500601. URL https://arxiv.org/abs/2302.11452.
[2] Steve Azzolin, Antonio Longa, Pietro Barbiero, Pietro Lio, and Andrea Passerini. Global explainability of gnns via logic combination of learned concepts. arXiv preprint arXiv:2210.07147, 2022.
[3] Libor Barto, Marcin Kozik, and Todd Niven. The csp dichotomy holds for digraphs with no sources and no sinks (a positive answer to a conjecture of bang-jensen and hell). SIAM Journal on Computing, 38(5):1782-1802, 2009.
[4] Joel Berman and Paweł Idziak. Counting finite algebras in the post varieties. International Journal of Algebra and Computation, 10(03):323-337, 2000.
[5] Garrett Birkhoff. On the structure of abstract algebras. In Mathematical proceedings of the Cambridge philosophical society, volume 31, pages 433-454. Cambridge University Press, 1935.
[6] Stanley Burris and H. P. Sankappanavar. A Course in Universal Algebra. Springer, 1981.
[44] Chih-Kuan Yeh, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar. On completeness-aware concept-based explanations in deep neural networks. Advances in Neural Information Processing Systems, 33:20554-20565, 2020.
[45] Rex Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks, 2019.
[46] Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Cheekong Lee. Protgnn: Towards self-explaining graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 9127-9135, 2022.
A Algebra definitions
A.1 Formal definitions for Universal Algebra
Universal algebra is the field of mathematics that studies algebraic structures, which are defined as a set  $A$  along with its own collection of operations. In this section, we recall some basic definitions and theorems from Burris and Sankappanavar (6); Day (8); Jonsson (17), about elements of universal algebra and lattice theory.
Definition A.1. N-ary function For a non-empty set  $A$  and  $n$  non-negative integer we define  $A^0 = \{\emptyset\}$  and, for  $n > 0$ ,  $A^n$  is the set of n-tuples of elements from  $A$ . An  $n$ -ary operation on  $A$  is any function  $f$  from  $A^n$  to  $A$ ;  $n$  is the arity of  $f$ . An operation  $f$  on  $A$  is called an  $n$ -ary operation if its arity is  $n$ .
Definition A.2. Algebraic Structure An algebra  $\mathbf{A}$  is a pair  $(A,F)$  where  $A$  is a non-empty set called universe and  $F$  is a set of finitary operations on  $A$ .
Apart from the operations on  $A$ , an algebra is further defined by axioms, that in the particular case of universal algebras are in the form of identities.
Definition A.3. A lattice  $\mathbf{L}$  is an algebraic structure composed by a non-empty set  $L$  and two binary operations  $\vee$  and  $\wedge$  satisfying the following axioms and their duals obtained exchanging  $\vee$  and  $\wedge$ :
$$
x \vee y \approx y \vee x
$$
$$
(\text {c o m m u t a t i v i t y})
$$
$$
x \vee (y \vee z) \approx (x \vee y)
$$
$$
(\text {a s s o c i a t i v i t y})
$$
$$
x \vee x \approx x
$$
$$
(\text {i d e m p o t e n c y})
$$
$$
x \approx x \vee (x \wedge y)
$$
$$
(a b s o r p t i o n)
$$
Theorem A.4 ((6)). A partially ordered set  $L$  is a lattice if and only if for every  $a, b \in L$  both supremum and infimum of  $\{a, b\}$  exist (in  $L$ ) with  $a \lor b$  being the supremum and  $a \land b$  the infimum.
Definition A.5. Let  $\mathbf{L}$  be a lattice. Then  $\mathbf{L}$  is modular (distributive,  $\vee$ -semi-distributive,  $\wedge$ -semidistributive) if it satisfies the following:
$$
x \leq y \rightarrow x \vee (y \wedge z) \approx y \wedge (x \vee z)
$$
$$
\mathrm {(m o d u l a r i t y)}
$$
$$
x \vee (y \wedge z) \approx (x \vee y) \wedge (x \vee z)
$$
$$
\text {(d i s t r i b u t i v i t y)}
$$
$$
x \vee y \approx x \vee z \rightarrow x \vee (y \wedge z) \approx x \vee y
$$
$$
(\vee - \text {s e m i - d i s t r i b u t i v i t y})
$$
$$
x \wedge y \approx x \wedge z \rightarrow x \wedge (y \vee z) \approx x \wedge y
$$
$$
(\wedge \text {- s e m i - d i s t r i b u t i v i t y}).
$$
Furthermore a lattice  $\mathbf{L}$  is semi-distributive if is both  $\vee$ -semi-distributive and  $\wedge$ -semi-distributive
Figure 8:  $\mathbf{N}_5$ , a non-modular non-distributive and  $\mathbf{M}_3$ , a modular non-distributive lattice.
Theorem A.6. If a lattice  $\mathbf{L}$  is distributive, then  $\mathbf{L}$  is also modular.
Proof. By assuming  $x \leq y$ , we have  $x \vee y = y$ . Hence, from the distributive property we get:
$$
x \vee (y \wedge z) \approx (x \vee y) \wedge (x \vee z) \approx y \wedge (x \vee z)
$$
Definition A.7. Congruence Lattice
An equivalence relation on a set  $A$  is a binary relation  $\sim$  that satisfies three properties: reflexivity, symmetry, and transitivity.
Reflexivity: For every element  $a$  in  $A$ ,  $a$  is related to itself, denoted as  $a \sim a$ ;
Symmetry: For any elements  $a$  and  $b$  in  $A$ , if  $a \sim b$ , then  $b \sim a$ ;
Transitivity: For any elements  $a, b$ , and  $c$  in  $A$ , if  $a \sim b$  and  $b \sim c$ , then  $a \sim c$ .
In other words, an equivalence relation partitions the set  $A$  into subsets, called equivalence classes, such that elements within the same class are equivalent to each other under the relation  $\sim$ .