Title: Higher-Order Learning with Graph Neural Networks via Hypergraph Encodings
Venue: NeurIPS 2025
Accepted: yes
Keywords: Graph Neural Networks

Relational Learning

Hypergraph Neural Networks

Discrete Curvature
Abstract: Higher-order information is crucial for relational learning in many domains where relationships extend beyond pairwise interactions. Hypergraphs provide a natural framework for modeling such relationships, which has motivated recent extensions of graph neural network (GNN) architectures to hypergraphs. Most of these architectures rely on message-passing to encode higher-order information. In this paper, we propose to instead use hypergraph-level encodings based on characteristics such as hypergraph Laplacians and discrete curvature notions. These encodings can be used on datasets that are naturally parametrized as hypergraphs and on graph-level datasets, which we reparametrize as hypergraphs to compute encodings. In both settings, performance increases significantly, on social networks by more than 10 percent. Our theoretical analysis shows that hypergraph-level encodings provably increase the representational power of message-passing graph neural networks beyond that of their graph-level counterparts. For complete reproducibility, we release our codebase: https://github.com/Weber-GeoML/Hypergraph_Encodings.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a data augmentation approach for graph neural networks (GNNs) by introducing hypergraph-level encodings from hypergraph Laplacians and discrete curvature to capture higher-order structural information.\n\nThe authors theoretically demonstrate that these encodings enhance the representational power of GNNs.\n\nExperiments demonstrate improvements on various datasets, particularly when used with graph-level architectures applied to hypergraphs or lifted graphs.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n1. The paper provides theoretical analysis in the form of Theorems 3.2, 3.5, 3.11, 3.13, demonstrating that the proposed hypergraph-level encodings provably increase the representational power of GNNs. \n2. The authors provide an anonymized codebase and extensive appendices detailing architectures (Appendix B), datasets (Appendix C), hyperparameters (Appendix C.2), and further experimental results (Appendix D, G) and expressivity analysis (Appendix E, F).\n\n$~$\n\n### Weaknesses\n\n1. The experiments primarily focus on performance improvements over old models (e.g., GCN, UniGCN, GPS) and lack a comparison against state-of-the-art baselines, see surveys [1, 2]. Moreover, it is unclear from the paper how the demonstrated improvements compare against the current state-of-the art results on the respective datasets.\n2.   The observation that dedicated hypergraph neural networks (like UniGCN) do not benefit significantly from these hypergraph-level encodings is intriguing and counter-intuitive (lines 276-278, Appendix G). While the paper suggests it as an area for future work (lines 340-343), a more elaborated discussion or hypothesis in the main text about why this might be the case (e.g., are HNNs already implicitly capturing this, or are current message-passing schemes in HNNs not well-suited to integrate such explicit features?) would add depth.\n3. A more in-depth discussion of the practical computational overhead, especially for very large graphs leading to dense hypergraph expansions would be beneficial. An accuracy - run time trade-off analysis, including the pre-processing times, would significantly strengthen the paper's empirical contributions. \n\n$~$\n\nReferences\n1. A Survey on Hypergraph Neural Networks: An In-Depth and Step-By-Step Guide, In KDD 2024.\n2. A Survey of Graph Transformers: Architectures, Theories and Applications, In arXiv 2502.16533.\n\nQUESTIONS:\n1. Is it possible that current hypergraph message-passing schemes (e.g., the two-stage node-to-hyperedge-to-node employed by UniGCN) inherently struggle to integrate such explicit, pre-computed node-level features effectively compared to how standard GNNs integrate them?\n2. Could the normalization or aggregation steps within these HNNs be inadvertently diminishing the unique information provided by the unnormalized encodings (as hinted at in Remark 3.14 regarding normalization obscuring differences)?\n3. What was the approximate pre-processing time for the encodings on representative datasets?\n4. How much did the size of the graph (nodes/edges) increase when lifted to a hypergraph (nodes/hyperedges/total incidence entries), and how did this impact subsequent GNN training time/memory compared to using only graph-level encodings or no encodings?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper works on hypergraphs and neural networks for them. The main contribution is the lifting of common positional and structural encodings from graphs to hypergraphs for which they claim that they are always superior to their graph counterpart, even though all datasets are lifted graph datasets (mostly because there are no true hypergraph datasets available).\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\nThe main strength is a clear and lifting procedure for the structural and positional embeddings, allowing them to make sense of hyperedges. The experiments indicate that each hypergraph-lifted encoding is always better than the corresponding graph-level encoding. The paper is well-written and easy to follow.\n\n**Weaknesses:**\nmostly the experimental evaluation. While the authors (roughly) claim that \"holding the parameter constant makes everything more fair\", I strongly disagree with this claim. In the experiments, the absolute performance of the models is also really low and for example a simple HGNN from 2019[1] achieves significantly higher performance than any of the models in this paper. I would have expected this model to be a baseline. \n\nMy main problem with the experiments is that the provided results are not representative of the model for each of the datasets and thus the performance gains through additional encodings could behave entirely different when performing a small hyperparameter search based on a generally good architecture. Instead of reporting 50 trained models for each dataset, reporting the average of 5 while using the other 45 for hyperparameter tuning would be much preferred by me.\n\nI would also like to see a plain graph transformer and a simple MLP as baselines as both architectures rely only on encodings to learn about the graph/hypergraph structure. \n\n\n** further detailed comments**\n- please use booktabs.\n- In general, the tables are hard to read and it is not directly clear what they show - maybe it is possible to make a plot that directly compares each PE with its lifted variant?\n- having only the \"best\" value in bold but not everything within $1\\sigma$ or with overlapping standard deviations is problematic as it indicates that there is a single best model while this is not significant and actually multiple models are equally good.\n- it could be made more clear that there are settings where both graphs and hypergraphs are used \"together\"\n- 306: with encodings (e.g. LapPE) there is no difference in expressiveness between any of the models (at least in terms of non-uniform expressivity). GPS internally uses GIN, so it should not be worse-off. It would be nice that all those expressiveness results only hold in the regime without encodings.\n- Leman is written without h.\n- 65ff: message passing only sometimes add self-loops while the formula assumes that this is always the case.\n- 74: maybe also cite a second one here such as LGI-GT [2] that follows up on GPS\n- 91: as encodings are crucial for graph transformers, I would have expected this to be much more prominent here. I was also surprised to find RWSE later but not in this paragraph.\n\n\n[1] Hypergraph Neural Networks, Feng et al. AAAI 2019\n[2] LGI-GT: Graph Transformers with Local and Global Operators Interleaving, Yin and Zhong IJCAI-23\n\nQUESTIONS:\n1) How do a pure GT and MLP react to the hypergraph embeddings? \n\n2) Do the main results (i.e. hypergraph variant is always better than \"normal\" one) carry over when getting closer to SOTA results?  E.g. there seems to be diminishing returns for GIN which starts with a higher performance.\n\n3) What is the main difference between UniGCNII and a model such as HGNN which reports a lot better results on cora and pubmed? (on pubmed a simple GCN is stronger than any of the hypergraph networks)\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes to use hypergraph encodings in order to enhance performance of graph neural networks. These encodings are designed to capture higher-order structural and positional information, which cannot be captured by standard MPNNs. Expressivity is also partly investigated theoretically.\nThe experimental evaluation shows that the encodings outperform graph-level methods, especially on social networks.\n\nSTRENGTHS AND WEAKNESSES:\n**S1** The idea of the paper is nice, using encodings for improving MPNN-performance is an interesting research direction. Since the source code is available online, and the experimental setup is described in great detail, one should be able to reproduce the results easily. \n\n**S2** The discussion at the end is really extensive. Most limitations are discussed in detail.\n\n**W1** The presentation of the paper can be improved significantly. In the background section several definitions are incomplete or missing. For the graph definition, V should be included (G=(V,X,E)) and F should be defined formally. The neighborhood of a vertex is not formally defined. The abbreviations GNN and MPGNN are used but never introduced. A short intuition of how a graph is transformed into a hypergraph should be given within the paper (currently it is only in the appendix). D, d_k, A, RW (and maybe other variables) are never defined. Some variables (like d) also seem to be used multiple times in different ways. H-LDP is also not formally defined. In the references, there are multiple duplicate entries.\n\n**W2** From the experimental evaluation, it seems as though no encoding consistently outperforms the others. If there are differences in performance, these could be highlighted by computing a ranking of the methods over all datasets.\n\n**W3** The runtime needed to compute **all** the different encodings is not (neither theoretically nor empirically) shown.\n\nOther comments:\n\n- Many (not all) paragraph titles are missing the .\n- For some sections/subsections the title is not capitalized consistently.\n- A figure showing an overview of the different encodings would be nice.\n---\n- l17ff maybe rephrase the first sentence or split it in two.\n- l171 (see F.2) maybe Appendix missing?\n- l212 Both graphs were not introduced before, so \"again\" is not fitting (and maybe the graphs should also be defined first).\n- l230 Why does the LDP not get an equation number?\n- l260 When referring to tables in the appendix, please make clear that they are in the appendix.\n- l302 Lehman -> Leman\n- l303 hypergrpah -> hypergraph\n\nQUESTIONS:\nPlease improve the quality of the paper by making sure that all definitions are correct and included. Please define all variables used. I like the idea of the paper, but in the current state, I am against accepting it.\n\n**Q1** Which of the encodings should be used? Can you say anything about which one might be better depending on the dataset?\n\n**Q2** What is the computational complexity of all the proposed encodings (there are some missing an analysis)? What was the runtime for computing them in practice?\n\n**Q3** Regarding the significance of the proposed methods.\n\na) For hypergraphs: As stated in the discussion section, it seems there is not really data available that is naturally represented as hypergraphs. At first glance, it does not seem to be useful then, or am I missing something?\n\nb) For traditional graphs: In the experimental evaluation, the encodings outperform the models without encoding. However, they are not compared to state-of-the-art GNNs, so this seems more like an ablation study. Can you comment on that?\n\n**Q4** Why are the last three entries in Equation 8 (and in LDP) repeated?\n\nLIMITATIONS:\nComputational complexity of the proposed encodings was not discussed as a limitation.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces hypergraph-level encodings to enhance higher-order relational learning. The authors propose structural and positional encodings (e.g., hypergraph Laplacian eigenvectors, random walk transition probabilities, curvature profiles) that capture higher-order information beyond pairwise interactions. Theoretical analysis proves that the encodings increase the expressivity of message-passing GNNs beyond the 1-WL test. Experiments show performance gains when combining hypergraph encodings with graph-level architectures, though hypergraph-specific architectures benefit minimally.\n\nSTRENGTHS AND WEAKNESSES:\n##### Strengths\n\n- High-order information is crucial for complex tasks and is a promising research topic. The authors propose a novel shift from architectural extensions (e.g., message-passing hypergraph NNs) to data-centric hypergraph encodings.\n- Well-structured with clear statements. Appendices provide the necessary background.\n- Theoretical proofs (Theorems 3.2, 3.5, 3.11) demonstrate enhanced expressivity. Extensive experiments validate claims, with ablation studies and scalability analysis.\n\n##### Weaknesses\n\n- It is still unclear why the covariance matrix of the incidence matrix, as shown in Definition 3.1, helps to capture higher-order information. Theorem 3.2 shows that this way has higher discrimination power for nonisomorphic graphs. Does this mean all models with higher discrimination power potentially capture higher-order information?\n- Encodings improve graph-level models but not hypergraph architectures. This discrepancy is noted but not analyzed.\n- All proofs of expressive power are conducted within BREC datasets. Given so, the theoretical contributions can be limited.\n\nQUESTIONS:\n- It would be better to provide a formal point of view of the term higher-order information and hypergraph-level encoding.\n- Why do hypergraph architectures (e.g., UniGCN) not benefit from hypergraph encodings?\n- The incidence matrix given in Definition 3.1 is also used in [1]. Also, the proposed covariance matrix $B_1B_1^T$ seems to be a simple form of $HWD^{-1}H^T$ in [1]. Please discuss their differences.\n\n[1] Feng, Y., You, H., Zhang, Z., Ji, R., & Gao, Y. (2019). Hypergraph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33 (pp. 3558-3565).\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}