Title: Logical Expressiveness of Graph Neural Networks with Hierarchical Node Individualization
Venue: NeurIPS 2025
Accepted: yes
Keywords: Graph Neural Networks

Hybrid Logic

Individualization-Refinement
Abstract: We propose and study Hierarchical Ego Graph Neural Networks (HE-GNNs), an expressive extension of graph neural networks (GNNs) with hierarchical node individualization, inspired by the Individualization-Refinement paradigm for isomorphism testing. HE-GNNs generalize subgraph-GNNs and form a hierarchy of increasingly expressive models that, in the limit, distinguish graphs up to isomorphism. We show that, over graphs of bounded degree, the separating power of HE-GNN node classifiers equals that of graded hybrid logic. This characterization enables us to relate the separating power of HE-GNNs to that of higher-order GNNs, GNNs enriched with local homomorphism count features, and color refinement algorithms based on Individualization-Refinement. Our experimental results confirm the practical feasibility of HE-GNNs and show benefits in comparison with traditional GNN architectures, both with and without local homomorphism count features.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Hierarchical Ego Graph Neural Networks (HE-GNNs), a novel class of GNNs that enhance expressive power through a mechanism inspired by the Individualization-Refinement (IR) paradigm from graph isomorphism testing. HE-GNNs iteratively individualize nodes and apply message-passing, resulting in a hierarchy of models where increasing the nesting depth increases expressive power. The authors provide a tight logical characterization of HE-GNNs using graded hybrid logic (GML($\\downarrow$)) and its subgraph-restricted variant (GML($\\downarrow$,W)). They show that HE-GNNs match or surpass existing expressive models such as higher-order GNNs, subgraph-GNNs, and homomorphism-based GNNs. The theoretical results are backed by empirical experiments showing HE-GNNs can distinguish graphs that standard GNNs fail on, including strongly regular graphs and graphs in datasets like ZINC.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. HE-GNNs is a compelling and novel approach that generalizes and unifies various expressive GNN architectures under the lens of logic. The integration of hybrid logic to describe the expressive capabilities of GNNs is particularly innovative.\n2. The paper is rigorous in its formalism. The equivalence theorems connecting HE-GNNs with graded hybrid logic (Theorems 3.3, 4.1) are carefully proved and grounded in prior foundational results (e.g., Barcelo et al., Morris et al.). The use of logical tools for characterization, such as nesting-depth correspondence, is both insightful and technically strong.\n3. This work is foundational, it provides a formal framework to measure and compare the expressive power of advanced GNN architectures. The logical perspective opens new research directions for designing more powerful and interpretable models. Also, HE-GNNs offer a scalable alternative to higher-order GNNs with bounded memory overhead.\n\n**Weaknesses**\n1. The experimental validation is relatively limited in breadth. While the ZINC and SRG evaluations support the theory, inclusion of synthetic benchmarks like CSL, EXP, and SR25 could offer a clearer picture of the model's capabilities in distinguishing structurally similar yet non-isomorphic graphs.\n2. The empirical evaluation lacks diversity in real-world benchmarks. The model is only evaluated on a single real-world dataset (ZINC), with no experiments on large-scale and widely used benchmarks such as the OGB datasets. This limits the ability to assess the practical effectiveness, scalability, and generalizability of HE-GNNs in more complex and varied real-world scenarios.\n3. Although HE-GNNs are presented as more efficient than higher-order GNNs, no detailed comparison of runtime, memory, or practical training complexity is provided.\n4. The impact of depth $d$ and radius $r$ in HES-GNNs is not fully analyzed in experiments (missing ablation study). A study on how performance and runtime vary with $d$ and $r$ would strengthen the claims.\n\nQUESTIONS:\n1. Have you considered evaluating HE-GNNs on synthetic isomorphism benchmarks like SR25, CSL, or EXP to better demonstrate expressiveness in challenging graph distinction tasks?\n2. Can you provide empirical evidence on how performance and computational cost scale with increased nesting depth $d$ and radius $r$?\n3. Theorem 4.1: Could you elaborate on how the quantifier radius in GML($\\downarrow$, W) corresponds exactly to the radius-$r$ subgraphs in HES-GNNs? Are there cases where the logical quantification covers more nodes than the subgraph accessible by radius-$r$ message passing?\n4. Theorem 4.3: Is the strict increase in expressiveness dependent on the existence of graphs where node distinctions require radius > $r$ neighborhoods? Could you provide such a minimal pair of graphs or clarify how often such cases occur in real-world datasets?\n5. While you show that 2-WL and Nested GNNs are subsumed by HES-GNNs under certain conditions, could you clarify whether this inclusion is strict in all cases or only under specific configurations of depth and radius?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors propose a graph neural network (GNN) architecture HE-GNN that goes beyond the expressivity limitations (i.e. 1-WL) of vanilla GNNs. This is then extended by allowing the model to consider only ego-nets of radius r, which improves the computational complexity. In particular, the proposed methods exploit the paradigm of Individualization-Refinement used by many subgraph isomorphism solvers. Most of the theoretical results on the graph separating power of the proposed architecture are obtained via logical characterizations, and the results are compared to some of the existing literature on higher-order GNNs and subgraph GNNs. Finally, some proof-of-concept experimental results are provided.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n- the theoretical results seem to be correct. Moreover, the use of logic, while it has already been used in some works on GNNs, is an interesting approach, and the graph learning community can benefit from it. \n- The introduction of bounded radius ego-nets is an interesting and effective approach to reduce the model complexity.\n- The comparison with higher-order and subgraph GNNs is well written and provides interesting insights. \n- Code is provided and will be made public. \n\nWeaknesses:\n- The literature review is largely insufficient, as the work is not situated within the quite rich literature of individualization-refinement approaches in graph learning. I name a few here:\n    - [1] provides a very general framework (IRNI) for endowing GNNs with individualization-refinement features, allowing to obtain full graph separating power. This work (and [2]) should be discussed thoroughly, as it is the closest to what the present work proposes.\n    - [2] provides a learning-theoretic analysis of methods that individualize nodes as a way of improving expressivity, and proposes the Tinhofer encoding (based on the individualization-refinement framework) as a compromise between expressivity and generalization. Moreover, the work proposed a bounded-radius ego-net architecture EGO-NN. What are the differences with HES-GNNs?\n    - [Haroon Dupty and Lee] (cited but discussed too briefly) and [3] provide methods to integrate the individualization refinement framework into GNN architectures. \n    - [4] studies the expressive power of GNNs endowed with individualization methods from the prespective of logic, like the present work. The differences in the two approaches should be highlighted. \n    - Several other works propose methods to individualize nodes to improve expressivity. This technique was first introduced in [5], and for other works check related work sections of [1] and [2].\n- The use of logic to prove graph separation results, while interesting, is not properly justified. I suspect that easier proofs with tools that are more accessible to the machine learning and graph learning community could lead to the same results. Therefore, the authors should argue why a logic-based approach is to be preferred in this case. \n- The high memory costs (quadratic) and time complexity (exponential in the depth d) make this architecture, while inteesting from the theory perspective, unlikely to be adopted in practice. Moreover, these time and space complexities should be discussed directly in the methods sections, rather than only in the limitations. \n- The experimental validation is quite unsatisfactory. \n    - The experimental framework is not properly defined neither in the main paper nor in the appendix. For example: does the test set include the same isomorphism classes of the training set in the strongly regular experiment? If so, why is there is a gap between train and test performance? \n    - On the strongly regular dataset, it would be interesting to compare HES-GNNs to other individualization-refinement methods such as the Tinhofer encoding [2] or IRNI [1]. \n    - Only one real-world dataset (ZINC) is used, and the proposed baselines are quite weak. Comparing to the aforementioned methods or to node-marking-based subgraph GNNs would be interesting. \n- The definition of HE-GNNs and HES-GNNs by bullet points is somewhat imprecise. Saying that a HE-GNN of nesting depth $d>0$ is a tuple $(B, C)$ with $B$ a HE-GNN of nesting depth $d-1$ and $C$ a GNN is not really enough. If the model just ignored $B$, we would get a normal GNN. I think run_A should be included in the definition (something like “a HE GNN of nesting depth $d>0$ is a tuple $(B,C)$ […] such that embeddings are updated according to run_A“)\n\n---\n\n- [1] Franks et al. A Systematic Approach to Universal Random Features in Graph Neural Networks. TMLR 2023.\n- [2] Pellizzoni et al. On the Expressivity and Sample Complexity of Node-Individualized Graph Neural Networks. NeurIPS 2024. \n- [3] Guo et al. Improving the Expressive Power of Graph Neural Network with Tinhofer Algorithm. 2021. (preprint)\n- [4] Abboud et al. The surprising power of graph neural networks with random node initialization. IJCAI 2021.\n- [5] Sato et al. Random features strengthen graph neural networks. SDM 2021.\n\nQUESTIONS:\n- the authors should argue why a logic-based approach was chosen to describe the expressivity of the proposed architecture. \n- can you discuss the memory and time complexity of the proposed method?\n\n---\nFinally, I wish to remark that if the authors address my concerns (missing related work, more justification for logic-based approach, and more thorough experimental evaluation), I would be willing to increase my score.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Hierarchical Ego GNNs (HE-GNNs), which augment standard message-passing GNNs by recursively “individualiing” a chosen node, marking it uniquely, and re-running a base GNN to capture increasingly fine-grained local structure. By nesting this individuation process to depth $d$, HE-GNNs form a strict hierarchy: at depth $d$, they match the distinguishing power of $d$-nested graded hybrid logic with variable binders, and as $d\\to\\infty$, they can separate any pair of non-isomorphic graphs. The authors provide rigorous logical characterizations showing that depth-1 HE-GNNs subsume common subgraph-based GNNs, that $d$ rounds of HE-GNN individuation lower-bound $d$ steps of the Individualization-Refinement paradigm, and that with sufficient nesting they can simulate homomorphism-count-augmented GNNs. An illustrative experiment on ZINC-12k and graph isomorphism testing capabilities concludes the paper.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The paper is very well written, with precise definitions and a coherent, logical progression.\n2. Prior approache are surveyed thoroughly and compared clearly.\n3. Adopting graded hybrid modal logic's variable binders to model node individualization is elegant and conceptually well motivated.\n4. The paper delivers an extensive number of expressiveness results—characterizing HE-GNNs by depth $d$ and radius $r$ and situating them relative to competing paradigms.\n\n**Weaknesses**\n1. The modal-logic characterization focuses on graph distinguishability; proving uniform expressiveness (i.e., the ability to compute arbitrary logical queries) would significantly strengthen the contribution.\n2. Theorem 5.10 (relating HE-GNNs to higher-order GNNs) may also be derived via the Geerts & Reutter ICLR 2022 analysis. It remains unclear whether the logic perspective offers insights beyond serving as an alternative proof technique.\n3. While Section 5.4 sketches the relationship, a tighter, perhaps quantitative, comparison to state-of-the-art subgraph-augmented GNNs would be valuable.\n4. The empirical evaluation, though illustrative, remains limited. More diverse benchmarks or ablations (e.g., varying $d$, $r$) would bolster confidence in practical gains.\n\nQUESTIONS:\n1. How does the recursive individualization in HE-GNNs compare to the Individualization–Refinement framework used in “A Systematic Approach to Random Data Augmentation on Graph”?  https://arxiv.org/abs/2112.04314 \n2. Are there known expressiveness or complexity results for graded hybrid modal logics with variable binders that you can leverage? Grounding HE-GNN’s logical characterization in a well-studied fragment would broaden its impact.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study the expressivity of graph neural networks (GNNs) with hierarchical node individualisation. For that they propose a new model of GNNs that they name Hierarchical Ego Graph Neural Networks (HE-GNNs) and show how its expressivity can be captured by graded hybrid modal logic. The expressivity characterisation follows the line of work initiated by Barcelo et at. [4] where the expressivity of GNNs was related to graded modal logic. Hence the main contribution of the authors here is to extend this connection to GNNs with node initialisation, which is in the logical side represented by the binder operator of hybrid logic. They also study a variant of HE-GNNs, where message passing related to node initialisations is restricted to some k-hop neighbourhood of the initialised node and obtain a related logical characterisation to that model, and an expressivity hierarchy related to k and initialisation depth d.\n\nIn addition the authors introduce a variant of the 1-WL algorithm with node initialisation (WL-IR) and show that the separating power of depth d HE-GNNs is limited below by the separating power of WL-IR with d node initialisations (WL-IR-d). Note that, over graphs of size at most d, WL-IR-d is able to separate all non-isomorphic graphs.\n\nThey empirically confirm that depth 2 HE-GNNs have improved performance to distinguish strongly regular graphs in comparison to common subgraph architectures.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The topic of the paper is timely and the results are interesting. The paper is very well written. I did not find any major technical issues, though I did not check all the details from the appendix. \n\n\nWeaknesses:\n- The presentation of HE-GNNs is quite compact. A worked out example of a simple HE-GNNs  would have been nice to have.\n- There is not much discussion whether their GNN model is useful in practice. \n- In isolation the results are not that surprising, but the authors have done a comprehensive job in developing matching models of GNNs, logic, and WL-algorithm.\n\n\nMinor comments:\n\n- l. 151. Typo: \"in in\"\n- l. 161. Technically, in GML formulas with free nominals, you cannot eliminate the \"@\" operator. \n- l. 186. Typo: Period missing after the syntax of the logic.\n\nQUESTIONS:\n1. What is the benefit or difference of your HE-GNNs of depth d compared to a simpler variant that would simply run a GNN on the |G|^d many copies of the graph with all possible and then have some aggregation via a GNN. That is, what is the benefit of the d-layer recursive definition.\n\n2. Could you say something on the topic of training  HE-GNNs of depth d. Can this be done in practise?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Hierarchical Ego Graph Neural Networks (HE-GNNs), a class of GNNs inspired by the individualization-refinement (IR) paradigm used in graph isomorphism testing. The authors provide a logical characterization of HE-GNNs (and a subgraph-restricted variant), relate their expressive power to higher-order GNNs, subgraph-GNNs, and GNNs with homomorphism counts, and demonstrate empirically that shallow HE-GNNs offer improved separating power over traditional GNNs. The authors conducted experiments on ZINC-12k and synthetic regular graphs.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- Theoretical framework is rigorous and unifies connections to higher-order GNNs, subgraph-GNNs, and homomorphism-count GNNs.\n- Logical analysis is novel and deepens understanding of GNN expressiveness.\n\n**Weaknesses**\n\n- Experimental validation is quite limited, focusing only on ZINC-12k and small synthetic datasets. There is no evaluation on larger or more diverse real-world benchmarks, nor ablations on computational overhead or scalability.\n- There are limited comparisons with related literature and several omitted works. Please see next section for more details\n\nQUESTIONS:\n- Could you please conduct experiments on a broader set of real-world datasets and compare HE-GNNs with additional, recently proposed expressive GNN baselines? I leave the choice of datasets and baselines to the authors’ discretion, but a more comprehensive empirical evaluation would greatly strengthen the paper.\n\n- The reference list is very weak. Please conduct a thorough literature review. Some related works can be found below:\n\nHuang, Yinan, William Lu, Joshua Robinson, Yu Yang, Muhan Zhang, Stefanie Jegelka, and Pan Li. \"On the stability of expressive positional encodings for graphs.\" arXiv preprint arXiv:2310.02579 (2023).\n\nKanatsoulis, Charilaos I., Evelyn Choi, Stephanie Jegelka, Jure Leskovec, and Alejandro Ribeiro. \"Learning Efficient Positional Encodings with Graph Neural Networks.\" arXiv preprint arXiv:2502.01122 (2025).\n\nBlack, Mitchell, Zhengchao Wan, Gal Mishne, Amir Nayyeri, and Yusu Wang. \"Comparing graph transformers via positional encodings.\" arXiv preprint arXiv:2402.14202 (2024).\n\nKanatsoulis, Charilaos, and Alejandro Ribeiro. \"Counting graph substructures with graph neural networks.\" In The twelfth international conference on learning representations. 2024.\n\nLim, Derek, Joshua Robinson, Lingxiao Zhao, Tess Smidt, Suvrit Sra, Haggai Maron, and Stefanie Jegelka. \"Sign and basis invariant networks for spectral graph representation learning.\" arXiv preprint arXiv:2202.13013 (2022).\n\nNerem, Robert R., Samantha Chen, Sanjoy Dasgupta, and Yusu Wang. \"Graph neural networks extrapolate out-of-distribution for shortest paths.\" arXiv preprint arXiv:2503.19173 (2025).\n\nLIMITATIONS:\nPlease see previous sections", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-08"}