Hypergraph Neural Networks through the Lens of Message Passing: A Common Perspective to Homophily and Architecture Design
Authors: Lev Telyatnikov
Maria Sofia Bucarelli
Guillermo Bernardez
Olga Zaghen
Simone Scardapane
Pietro Lio
Venue: Accepted by TMLR ()

Abstract: Most of the current learning methodologies and benchmarking datasets in the hypergraph realm are obtained by \emph{lifting} procedures from their graph analogs, leading to overshadowing specific characteristics of hypergraphs. This paper attempts to confront some pending questions in that regard:  Q1 Can the concept of homophily play a crucial role in Hypergraph Neural Networks (HNNs)? Q2 How do models that employ unique characteristics of higher-order networks perform compared to lifted models? Q3 Do well-established hypergraph datasets provide a meaningful benchmark for HNNs? To address them, we first introduce a novel conceptualization of homophily in higher-order networks based on a Message Passing (MP) scheme, unifying both the analytical examination and the modeling of higher-order networks. Further, we investigate some natural strategies for processing higher-order structures within HNNs (such as keeping hyperedge-dependent node representations or performing node/hyperedge stochastic samplings), leading us to the most general MP formulation up to date --MultiSet. Finally, we conduct an extensive set of experiments that contextualize our proposals.

Published in Transactions on Machine Learning Research (02/2025)
Hypergraph Neural Networks through the Lens of Message Passing: A Common Perspective to Homophily and Archi- tecture Design
Lev Telyatnikov Sapienza University of Rome
lew.telgaanikoo/@nnnooaa.
Maria Sofia Bucarelli
mariasofia. hatissofow..ccaarlliiaannnnn....
Sapienza University of Rome
Guillermo Bernardez
guillermo__beraandeeeuua bernardez@uuch edu
University of California, Santa Barbara
Olga Zaghen University of Amsterdam
o. zaghen Grggenn@uvo..l
Simone Scardapane
Sapienza University of Rome
Pietro Liò
pl2190cum.ae.
University of Cambridge
Reviewed on OpenReview: https: //
// openreview. net/ forum? id= BratL0k2nx
Abstract
Most of the current learning methodologies and benchmarking datasets in the hypergraph realm are obtained by lifling procedures from their graph analogs, leading to overshadow- ing specific characteristies of hypergraphs. This paper attempts to confront some pending questions in that regard: Q1 Can the concept of homophily play a crucial role in Hyper- graph Neural Networks (HNNs)? Q2 How do models that employ unique characteristics of higher-order networks perform compared to lifted models? Q3 Do well-established hy- pergraph datasets provide meaningful benchmark for HNNs? To address them, we first introduce novel conceptualization of homophily in higher-order networks based Mes- sage Passing (MP) scheme, unifying both the analytical examination and the modeling of higher-order networks. Further, we investigate some natural strategies for processing higher- ordor structures within HNNs (such as keeping hyperodgo-dependent node representations or performing node hade/hhperedge stochastic samplings), leading us to the most general MP formulation up date -MultiSet. Finally, we conduct an extensive set of experiments that contextualize our proposals.
1 Introduction
Hypergraph learning techniques have multiplied in recent years, demonstrating their effectiveness pro- cessing higher-order interactions in numerous fields, spanning from recommender systems (Yu al.] 2021
La Gatta 2022). to bioinformatics (Zhang et al. 2018 Yadati al., (2020) and computer vision (Li
et al. 2022 Xu al. 2022 However, far, the development of Hypergraph Neural Networks (HNNs) has

---

Published in Transactions on Machine Learning Research (02/2025)
Hommppha
0..7
0.9
0.6
118
134
262
119
194
202
-19
720
262
(a)
0.5
0.4
196
0.3
0.2
0.1
0.0
Class
1.0
0.8
Homonpii
0.6
6685
0.5
1519
1096
(b)
0.4
0.3
6811
3519
0.1
0.0
Class 0
Class
Class 2
Class
Figure Node Homophily Distribution for CORA-CA (a) and 20Newsgroups (b). The plots depict node homophily scores computed using Equation / at 0, 1, and t 10. For each dataset class, points are sorted in ascending order of homophily and visualized sequentially along the x-axis. Horizontal lines represent the mean homophily score for each class, with the numbers above indicating the total number of points in each class.
been largely influenced by the well-established Graph Neural Network (GNN) field. In fact, most of the cur- rent methodologies and benchmarking datasets in the hypergraph realm are obtained by lifting procedures from their graph counterparts.
The advancement of hypergraph research has been significantly propelled by drawing inspiration from graph- based models (Chien et al.) 2022 Feng et al.) 2019 Yadati et al., 2019). but it has simultancously led overshadowing hypergraph network foundations. We argue that is now the time to address fundamental questions in order to pave the way for further innovative ideas in the field. In that regard, this study explores some of these open questions to understand beller current HNN architectures and benchmarking datasets along three axes. Q1 Can the concept of homophily play crucial role in HNNs, similar to its significance in graph-based rescarch? Q2 Given that current HNNs are predominantly extensions of GNN architectures adapted to the hypergraph domain, are these extended methodologies suitable, or should we explore new strategies tailored specifically for handling hypergraph- data? Q3 Are existing hypergraph benchmarking datasets truly meaningful and representative to draw robust conclusions?
To begin with, explore how the concept of homophily can be characterized in complex, higher-order networks. Notably, there are many ways of characterizing homophily in hypergraphs such as the distribution of node features, the analogous distribution of the labels, or the group connectivity similarity (as already discussed in Veldt et al. (202))). ) In particular, this work places the node class distribution at the core of the analysis, and introduces a novel definition of homophily that relies on Message Passing (MP) scheme. This enables us to analyze both hypergraph datascts and architecture designs from the same perspective, as well to successfully describe model performances (see Section and 5.2
Next, shift our focus towards the design of bypergraph-specific methodologies that HNNs could benefit from, no longer relying on lifting strategics. To this end, after examining state-of-the-art HNN architectures. we first. describe the most versatile MP framework up to date, called MultiSet (see Section 4.2 Our formulation, which enables hyperedge--ependent node representations and residual connections, inherently gencralizes most existing HNN frameworks and models, including AllSet (Chicn et al. (2022). UniGCNII (Huang Yang 2021), EDHNN (Wang et al. 2023) and recently proposed model Net that allows for hypereegee-eependent node representations CChoe al. 2023).
2

---

Published in Transactions on Machine Learning Research (02/2025)
Subsequently, to facilitate comparison between standard lifted hypergraph models and new class of hypergraph-specifcc architectures, in Section 4.3 we present the realization of hypergraph Muxee that accommodates multiple hyperedge-basee node hidden representations H By the same node is repre-
allowing multiple node ropresentations based on hyperedge memberships- sented differently depending on the hyperedges it belongs to to-thi new class of archilectures results in an exponential increase in potential node representations as the number of connections within the hypergraph network grows. To address this complexity, we propose and evaluate novel connectivity-bused mini-batching strategies tailored to the specific characteristics of hypergraph networks, detailed in Section 4.4 These sampling procedures not only facilitate processing large hy hyperedges but also give rise to an interesting be- havior which we term connectivity-based distribution shift thoroughly discussed in the paper (see Section 5.3
Last but not least, we provide an extensive set of experiments that, driven by the general questions stated above, aim to gain a better understanding on fundamental aspocts of hypergraph representation learning. In fact, the obtained results not only help us contextualize the proposals introduced in this work but open new challenges in hypergraph modeling, such as signal oversquashing caused by large hyperedges, which impacts the performance of all HNNs (see Scction 5.1 or the tendency of HNNs to ignore connectivity for common benchmark datasets (see Section 5.4.1
Summary of contributions:
We introduce a novel definition of homophily for hypergraphs, capable of effectively describing HNN model performances (Q1 and Q3, Sections and 5.2)
We present the novel MultiSet framework, which incorporates prperedge-dopendart node repre- sentations and generalizes most of the existing hypergraph models in the literature (Q2, Section
4.2
We explore novel mini-batch sampling strategics for architectures with and without hyperedge- dependent node classification (Q2, Section 4.3
We perform a large set of experiments assessing benchmarking both datasets and HNN architectures, as well as connecting the proposed MP homophily with models' performance (Q1, Q2, Q3, Sections and
2 Related Works
Homophily in hypergraphs. Homophily measures are typically defined only for pairwise relationships. In the context of Graph Neural Networks (GNNs), many of the current models implicitly use the homophily assumption, which is shown to be crucial for achieving robust performance with relational data (Zhou ct al. 2020 Chien et al. 2020 Halcrow et al. 2020 Nevertheless, despite the pivotal role that homophily plays in graph representation learning, its hypergraph counterpart mainly remains unexplored. In fact, to the best of our knowledge, Veldt et al. (2023) is the only work that faces the challenge of defining homophily for higher-order networks. This work introduces a framework in which homophily is quantified through group interactions, measuring the distribution of classes among hyperedges (see Appendix G for a detailed description). However, their definition of homophily is restricted to uniform hypergraphs -where all hyperedges have exactly the same size which hinders its practical application to a great extent. Clique- expanded (CE) homophily, as used in Wang el. al. (2023). is calculated by determining node homophily Pei hypergraph. Thus, CE homophily is not directly defined on the hypergraph but requires a clique expansion. It is important to note that this expansion is a non-invertible transformation, which leads a loss of information. Refer Appendix G for a detailed
et al. (2020) on the graph derived from the
explanation of these homophily measures and a comparison with the proposed method.
Note that MultiSetMixer simple yet powerful model architecture that serves primarily as baseline, providing example of model that incorporates and employs the unique characteristies of hypergraph networks.
3

---

Published in Transactions on Machine Learning Research (02/2025)
Hypergraph Neural Networks. Numerous machine-learning techniques have been developed for hy- pergraph data processing. A common early approach is transforming hypergraphs into graphs via clique expansion (CE), where each hyperedge is replaced with edges between all pairs of vertices in the hyperedge, enabling graph- algorithm analysis (Agarwal et al.) 2006; Zhou et al. 2006 Zhang et al. 2018 Li & Milenkovic 2017
Hypergraph Neural Networks (HNNs) have also been applied to semi-supervised learning. One early method extends graph convolution by incorporating the normalized hypergraph Laplacian (Feng et al. 2019 with weighted CE spectral convolution (Dong et al. 2020). HyperGCN (Yadati et al., 2019) uses mediators for reduced CE, lowering the number of edges required to represent a hyperedge, and applying spectral convolution for information diffusion. Hypergraph Convolution and Hypergraph Attention (HCHA) (Bai
et al. 2021 introduces modified normalizations and attention weights dependent on node and hyperedge features.
However, CE can result. in the loss of structural information, leading to suboptimal performance (Hein al., 2013 Chien et al. 2022). These models often perform best with shallow architectures, while deeper layers can cause oversmoothing (Huang & Yang 2021). Recent work has tried to mitigate oversmoothing with residual connections but still relics on hypergraph Laplacians and clique expansion (Chen & Zhang 2022). Another method, line expansion (L.E), treats vertices and hyperedges equally and models vertex-hyperent pairs to induce a homogeneous structure, though both CE and LE demand significant computational resources Yang
et al. 2020
Another research line focuses on two-stage message passing: nodes communicate with hyperedges, which then relay information back to the nodes (Wei et al. 2021 Yi & Park 2020 Dong et al. 2020 Arya et al.,
2020 Huang & Yang 2021 Yadati et al. 2020 HyperSAGE (Arya et al. 2020 exemplifies this approach, offering improvements over spectral methods but limited by a single learnable transformation and inefficient
computation due to nested loops (Chien et al. 2022).
The work of Chien et al. (022)) introduced AllSet, a general framework that describes HNNs through the composition of two learnable permutation invariant functions, defining two-step message passing based mechanism -from nodes to hyperedges, then back from hyperodges to nodes. In particular, AllSet is shown
to generalize most commonly used HNNs, including all clique expansion based (CE) methods, HNN (Feng
et al. 2019h. HNHN (Dong et al. 2020). HCHA (Bai et al.) 2021), HyperSAGR (Arya et al. 2020) and
HyperGCN Yadati et al. 2019) Chien et al. (2022) also proposes two novel AllSet-like learnable layers: the first one -AlDeeeSet-- exploits Deep Set (Zahcer et al., 2017), and the second one -AlSSet Transformer- Set Transformer (Tee al.
2019 both of them achieving stale-of-the-art results in the most common hypergraph benchmarking datasots. Concurrent to AllSet, Huang & Yang (2021) also aimed at designing a common framework for graph and hypergraph NNs, and its more advanced UniGCNII method leverages initial residual connections and identity mappings in the hyperedge-to-nost
propagation to address over-smoothing issues; notably, UNi does not fall under the AllSet framework due to these residual connections. Likewise, the more recent EDHNN model (Wang et al.) 2023) also goes beyond this framework by incorporating lypereegeedepeeddet messages from hyperedges to nodes, a step closer to the lypereege-dependenu node representations that we propose in this work.
Recently, Choe et al.) (2023) introduced WHAT&Net, the first model capable of producing hyperedge- dependent node representations. In particular, WHATsNel model is particular instance of the MultiSet framework introduced in our work (see Section 4.2 but its architecture is specifically designed to perform hyperodge-dependeet node classification tasks -where each node has as many labels as the number of hyper- edges it belongs to. This prevents benchmarking this model in our evaluation, where we focus on standard node-level classification tasks.
Please refer to Appendix A for an extended literature review.
4

---

Published in Transactions on Machine Learning Research (02/2025)
Notation. A hypergraph is an ordered pair of sets (V, where V is the set of nodes and the set of hyperedges. Each hyperedge & is subset of V, i.e., e S V. A hypergraph is generalization of the concept of a graph where (hyper)edges can connect more than two nodes. A vertex and hyperedge are said to be incident if v E For each node v, we denote its class by yo, and we denote by & {e E & E e) the subset of hyperedges in which it is contained. We represent by de = |,,l the node degree. The set of classes is represented by C = {<4)==
3 Homophily Metrics in Hypergraphs
In this section, we present a new definition of homophily that employs a two-step message passing scheme applicable to general, non-uniform hypergraphs, in contrast to the definition by Vy/Vt et al. (2023 In essence, our definition focuses on capturing hyperedge interconnections by the exchange of information following the message passing scheme. Following that, we illustrate its applicability in examining higher-order networks through qualitative analysis. Finally, we demonstrate the applicability of the proposed concept by deriving A homophily measure. In Section 5.2 we show its capability to describe HNNs performance. These play pivotal role in our attempt to answer the fundamental question Q1 raised in the Introduction.
Message passing homophily. Given a hyperedge e E E, we define the O-level hyperedge homophily h2(c) as the fraction of nodes within e that belong to class c, i.e.
1 Σ 1y-=e. UEE
[C
(1)
This score describes how homophilic the initial connectivity is with respect to class c. Computing the score for each class & C generates a categorical distribution for each e E, i.e ha = [h_(co).... k[(ccc)) Using this information as starting point, we calculate higher-level homophily measurements for both nodes and hyperedges through the two-step message passing approach. Formally, we define the t-level homophily score as
Ao- AGGE (4ht-1(w) ((v) (((c) AGGy (th--) JuEe,
(2)
(3)
where AGGg and AGGy are functions aggregating edge and node homophily scores, respectively (we consider the mean operator in our implementation). We note that our homophily measure enables the definition of a score for each node and hyperedge for any neighborhood resolution.
Qualitative analysis. One straightforward way to make use of the message passing homophily measure is visualize how the node homophily score dynamically changes, as described in Eq. 2 Figure [ depicts this process for non-isolated nodes on CORA-CA and 20NewsGroup datasets (Appendix G shows the plots
of the rest of considered datasets, which in turn are described in Section 5). Looking at CORA-CA (Figure
(a)), we note that there are a significant number of nodes with high O-level homophily at each class (except
number 6), and this homophily distribution is kept mostly unchanged as we move to the 1-hop neighborhood
(t= Interestingly, the same trend holds even when shifting to the 10-level node homophily -only classes and show a relevant drop in highly homophilic nodes. This suggests the presence of isolated homophilic subnetworks within the hypergraph. In contrast, 20Newsgroups dataset (Figure ! (b)) displays relatively low node homophily scores from the O-level (specifically for class 2, with mcan value around 0.3). Moving to t there is significant decrease in the homophily scores for every class. Finally, at time step t 10, we can observe that all the classes converge to approximately the same homophily values within each class. This convergence and low homophily scores suggest that the network is highly interconnected.
Rather than measuring homophily in individual discrete timestamps, we next derive homophily measure, which offers a dynamic perspective to explore hypernetworks. This measure is based on the assumption that if the 1-hop neighborhood of a node u E is predominantly homophilic, then the change in homophily score between two consecutive timestamps will be small. Conversely, substantial change in
A homophily.
5

---

Published in Transactions on Machine Learning Research (02/202)
Fe
I
I Readout
Readout
Readout
I+1
2+11
1-1
ce ei
2 f+1 U2IE1
++1 ac U2uda
x+1
4++11
Fej
Zel
2
x 21
x
x= 6z
F2
(a) AllSet layout.
(b) MultiSet layout.
Figure Visual representation of the AllSet and MultiSet frameworks.
u's homophily implies that the node resides in a neighborhood characterized as heterophilic. Specifically, for each node, we quantify the homophily change after t-th step of message passing by computing the difference between the node homophily at that step from its value at previous one, t-1. Subsequently, we look at the proportion of nodes whose homophily difference below a certain threshold 1 E IR+, i.e.
4
Σ 1 NEV
(4)
V
As show in Section 5.2 this dynamic measure becomes helpful tool to analyze the performance of HNNs.
4 Methods
Current HNNs aim to generalize GNN concepts to the hypergraph domain, and are specifically focused on redefining graph-basee based propagation rules to accommodate higher-order structures. In this regard, the work of Chien et al. (2022) introduced a general notation framework, called AllSet, that encompasses most of currently available HNN layers, including CEGCN/CEGAN
HNN (ongg et al.) 2019). HNHN (Dong) let al. ) HCHA (Bai el al.] (001)), HyperGCN (Yadati et al. 2019), as well as AllDeepSet and AllSei- Transformer presented in the same work (Chien et al.) (2022). This section first revisits the original AllSet formulation, and then introduces new framework -MultiSct- which extends AllSet by allowing multiple lypereeggedepeendent representations of nodes. Finally, we present particular realization of archilecture that takes into account hyperedge dependent-node representations. Then, open a discussion ability issues of hypergraph architectures and propose a novel mini-batching scheme. In contrast to previous formulations, our proposed framework and implementation are inspired by hypergraph needs and features, and motivated by the fundamental question Q2
the scal-
4.1 AllSet Propagation Setting
For given node v E V and hyperedge E E in hypergraph G = (V,E), let a E R/ and z! E Rd denote their vector representations at propagation step t. We say that function f is a multiset function if it is permutation invariant w.r.t. each of its arguments in turn. Typically, (0)) and x! are initialized based
the corresponding node and hyperedge original features, if available. The vectors { and (0 represent the
initial node and hyperedge features, respectively. In this context, the AllSet framework (Chien al., 2022) consists in the following two-step update rule:

---

Published in Transactions on Machine Learning Research (02/2025)
[[+1) fv--e((0 Jumees ) ((+1) fe-v((z((+]] JeEE, )
Ze
(5)
(6)
where fy-+g and fE-Y are two permutation invariant functions with respect to their first imput. Equations 5
and 0
describe the propagation from nodes to hyperedges and vice versa, respectively. We extend the original AllSet formulation to accommodate UniGCNII (Huang & Yang 2021) by modifying the node update rule
(Eq. 6 as follows
The a:((1)) = J.- -E-->((=((t)) JeEE, {ar )==)
(7)
i.e. allowing residual connections. Again, the only requirement is to be invariant w.r.t. the first input. Proposition 4.1. UniGCNII (Huang 8 Yong. 2021 is special case of AllSet considering 5 and
In the practical implementation of model, fy-g and fE-VV are parametrized and learnt for each dataset and task; particular choices of these functions give rise to most of the HNN architectures considered in this paper (see Appendix B
4.2 MultiSet Framework
In this Section, we introduce our proposed MultiSet framework, which can be seen as an extension of AllSet where nodes can have multiple co-existing hyperedge based representations. For a given hyperedge e & in a hypergraph G = (V,E), we denote by x(() E I2 its vector representation at step For a node v E V. MultiSet allows for as many representations of the node as the number of hyperedges belongs to. We denote by Øe.e E /R/ the vector representation of node v in hyperedge e E & at propagation time 1, and by xx(*) = ( the set of all dy hidden states of that node in the specified time-step. Accordingly, the hyperedge and node update rules of MultiSet are formulated to accommodate hyperedge-dependent node representations:
= =(+1) = (v=e((x()) Jumeei )()),
(8)
(++1) fs-v( ((t(((()))) JeEe, (x(( (1 }k=o))
(9)
where fr-+ and fE-v are two multiset functions with respect to their first input. After T iterations of message passing, MultiSet also considers last readout-based step to obtain a unique final representation
a.o E R8" for each node from the set of its hyperedge based representations:
x ( fv-v((%)) HLL)
(10)
where Jv-vy is also a iult function.
As we show in the following propositions (proofs can be found in Appendices C.2 C.3 and C.4. both AllSet
framework and the more recent EDHNN and WHATsNet architectures (Wang et al. 2023 Choe et al. 2023) can be expressed in terms of the general MultiSet notation.
Proposition 4.2. AllSet 5
as well as its extension
5 7 are special cases of MultiSel g 70
Proposition 4.3. EDHNN WWang et al.
2023 is special case of MultiSet 8 10
Proposition 4.4. WHATsNet Chooe et al. 2023 a special case of MultiSet 10
Figure 2 represents the AllSet and MultiSet layouts. Please notice that to avoid clutter notation at step
for the MultiSet layout, we omit, populating nodes at step L. As shown at the top of the figure, we obtain
s41 and I o2,e ++,, which can subsequently be used to update the corresponding hyperedges and nodes.
4.3 Training MultiSet Networks
This section describes possible realization of a MultiSet layer implementation -which we refer to as Mul- Mixet model.

---

Published in Transactions on Machine Learning Research (02/2025)
Learning MultiSet layers. Following the mixer-style block designs practice, we use the following MultiSet layer implementation:
Tolstikkhin al. 2021 and standard
((+1) fV-+v ( ((ar WaMEnj 2
(11)
))
=
Σ
10 uic
+ +MLP
LN
1
Σ
x010
|el
ee]
PEe
x
((+1)) fe-v(zz( (1+1), (2(11)
(12)
0,0
= = + MLP (LN(x!!ee)
(((11)
x = Sv-v((x!)) = du EE
(t)
(13)
where MLPs are composed of two fully-connected layers, and LN stands for layer normalization. This architecture, which we call MultiSetMixer, is based on a mixer-based pooling operation for (i) updating hy- peredges from its node's representations, and (ii) generate and update hyperedge-dependent representations of the nodes.
Proposition 4.5. The functions fv-E, fE-V and fv-v defined MultiSetMizer 11 I
are permatation
invariant. Farthermore, these functions are universal approzimators of mullisel functions when the size of the input multiset is finite.
4.4 Mini-batching
The motivation for introducing new strategy to iterate over hypergraph datasets is twofold. First, current HNN pipelines face scalability issues when processing datasets with large nunber of nodes and hyper- edges. This problem is particularly pronounced in architectures that allow for pepperegg-dependee representations, as each node must. be represented multiple times. This scaling challenge becomes significant as the number of nodes, hyperedges, and hyperedge sizes grow (see Table g in the Appendix for relevant statistics). Second, pooling operations over large sets can lead to signal oversquashing, negatively impacting the performance of HNNs that do not account for this issue (see Table D as demonstrated by the 20News- groups dataset, where hyperedges have median of 537 nodes per hyperedge (sce Table 7 in the Appendix for detailed statistics).
node
To address these issues, we propose sampling X mini batches of a certain size B at each iteration in two steps. At step 1, we sample B hyperedges from E. The hyperedge sampling over & can be either uniform or weighted (e.g. by taking into account hyperedge cardinalities). Then in slep 2 L nodes are turn sampled from each sampled hyperedge e, padding the hyperedge with I, - lel special padding tokens if lel < T. consssstin of o vectors that can be easily discarded in some computations. Overall, the shape of the obtained mini-batch X is B x L.
mini-batches
Step 1 is particularly beneficial for hypergraph datasets with a large number of hyperedges, but it can be skipped when the entire network fits into memory. In Sections 5.1 and 5.3 we demonstrate that step 2 (node mini-batching within a hyperedge) is useful for two key reasons: (i) pooling operations over large sets may lead to signal oversquashing, while pooling over smaller sets helps mitigate this issue, and (ii) node batching introduces an intriguing offect-what we refer to as connectivity- based distribution shift-whif offers novel way to leverage connectivity rebalance the training distribution. Therefore, it can still be beneficial to use node mini- hini-batchhng even when the entire hyperedge fits into memory.
When both step 1 and step 2 are employed, the memory required during the forward pass is B x L x d. where d is the hidden dimension size. If only slep 2 is used, the batch size becomes |&] x L x d, where |8| is the total number of hyperedges in the hypernetwork. Finally, if no mini-batehin mini-batching is applied, the batch size is El x max,es |e| x d, where max,es |e| denotes the size of the largest hyperedge. For a detailed empirical analysis of memory utilization across different datasets and sampling schemes, see Section 5.3 For the theoretical analysis of the proposed sampling scheme, refer to Appendix F
8

---

Published in Transactions on Machine Learning Research (02/2025)
5 Experimental Results
The questions that raised in Scction [ have shaped our rescarch, leading to new definition of higher-order homophily and unexplored architectural designs that can potentially fit better the properties of hypergraph networks. In subsequent subsections, we set four questions that follow up from these fundamental inquiries and can help contextualize the technical contributions of this paper.
haaee
Datasets and models. We use the same datasets used in Chien et al. 2022 which includes Cora, Citeseer, Pubmed, ModelNet40, NTU2012, 20Newsgroups, Mushroom, ZOO, CORA-CA, and DBLP-CA. More information about datasets and corresponding statisties are in Appendix D.2 We also utilize the benchmark implementation provided by Chien et al. (2022) to conduct the experiments with several models, including AllDeepSets AllSet Transformer, EDHNN, UnIGCNII, CEGAT, CEGCN, HCHA, HNN, HNHN, HyperGCN, HAN, and HAN mini- batchng. Since the design of these architectures is primarily inspired by the GNN literature, we refer to them as lifted architectures. Additionally, we consider vanilla MLP applied to node features and transformer architecture, as well as MultiSetMixxer representing a baseline that employs lyperedge-dependeett node represental ions and a new MLP baseline leveraging Connectivity Batching (MLP CB). We refer to Section 4.3 for more details about all these architectures. All models are optimized using 15 splits with 2 model initializations, resulting in a total of 30 runs; see Appendix D.I for further dctails.
5.1 How do the lifted models perform in comparison to hypergraph-speccff, models?
Our first experiment directly targets our fundamental Q2 by assessing the performance of lifted architectures in comparison with those that leverage unique characteristies of hypergraph networks, such as hyperedge- dependent node representations and basede mini-batching.
Figure 3 shows the average rankings across all models and datasets- of the top-5 best-performing models for different training splits, exhibiting that those splits can impact the relative performance among models. In the main body of this work, we focus our analysis on the 50% split results presented in Table 1 while the corresponding tables for other scenarios are provided in Appendix E.4 Table ! emphasizes MultiSet- Mixer solid performance, obtaining the highest test accuracy on NTU2012, ModeINet40,, and 20Newsgroups datasets. Notably, MultiSetMixer and MLP CB share similar patterns, and both significantly outperform all the other architectures on 20Newsgroups, which we further discuss in Section
In fact, the comparable performance among the rest of HNN models on this datasct suggests that existing architectures can not account for the dataset connectivity. According to what we observed in the qualita- tive homophily analysis performed in Section g 20Newsgroup is densely interconnected, making it highly heterophilic as the MP evolves; we argue this presents a challenge for most of current HNNs architectures. In contrast, CORA-CA exhibits a high degroe of homophily within its hyperodges and shows the most sig- nificant performance gap between HNNs and the baselines. A similar trend is observed for DBLP-CA (see
Unless otherwise specified, all tables in the main body of the paper use 50%/25% /25% split between training and testing. The results are shown Mean Accuracy Standard Deviation, with the best result highlighted in bold and shaded in grey, and results within one standard deviation are displayed in blue-shaded boxes.
Table 1: Hypergraph model performance benchmarks. Test accuracy in % averaged over 15 splits.
Moded
Cora
Clicser
Palarsl
QURA-AIA
DBLP-CA
Mushmonn
NYUU012
MadcNNet0)
S0Nevegnauee
200
cooking
AlDDwgsSets
77.51
1.01
1 1-4
REDI c:15
82-33 1 1.16
91.31 11:07
$0.06 0005
S6 1. 1.45
14.20
0.25
81.15
1 0.00
40.10 10:10:1
7:00
6.81
AllSet Trandomee
79.34
102
72.2211
0.28
0.1
81:43
1:15
91.03
11:11
59.55 0.15
22222
50.01
0.12
81.30
11:00
51
1: 7.31
3::25
VURMNNI
25.46 = 1.14
78000
1.45
07
6.47
53.02
= 1.10
01.56
0.18
$0.50
2 0.07
58 = 2.56
07:81
0.36
81.16
0.00
59.61
4.7G
KDMINN
68.74
1.00
78.09
1.14
80.12
0:47
65.17
1.09
01.04
3: 0.33
59.04
0.11
58
65
07.20
0.10
01.61
6.99
2.00
CEGAT
28.33
1.08
11.58
6,60
12.00
20.24
0.51
98.51 1.41
02.77
0.44
9.15
12.11
CECCN
77.03
1.8L
70.7
6.62
12.65
0.22
94.01
50,00
50004
0.47
0.8L
12.50
HGNN
HSBN
100
0,47
T8.53
01.00
00:97 0.04
0
0.36
81.53 0.05
7.65
Hy Hypee BAN
1.11
66.06
0.68
17
3.22
68:51
45.06
7.56
00.54
0.85
14.30
9025
1.30
T3.R 9.95
90.56 0.81
81.09 0 11.11
91..0 11:20
91.35 0.51
55.17- 1.75
50.85 11.55
19.87 11:00
50.28 0.42
1.500
BAN minilonch
96.24
4 2.12
75:66 3: L.13
86.41 4: 2.32
K204 0 2.56
98.32 (1.00
93.57 1/04
SIL12 2 2,50
02.05 0.55
73.76 01.0
T11200 = 11.29
10.50
Mairs-MMitree
26.66-1
1:51
71.55 + 1.39
19 0.53
71 à 1.53
93.05 11111
98:16 4 0.16
A8.00 1:30
$8.08 4: 0.21
W5.57 4: 1:86
05.06 0 3.01
6.90
SELP CB
24066 2 1:20
71.58 + 1.63
50,83 2: 6,51
14.21 = 1.40
64.3 0.46
1908 0008
$6.65 :: 1.51
50.41 = 0.52
26.13 = 2
51.6 = 10.56
16.30
MIP
23.27
1.00
72.07 + 1/65
57.18 = 6.40
TA.77 = 1.00
64.77
0.41
98.51
2 0.05
TO.
6: 1.56
0G.L 0.28
20.35 = 0.20
65.18-2:000 1115-21000
13.50
Trarsfermee
24.35
1:17
71,22.1 1.31
N7.37 c.1b
T361 1.58
25.36 00:38
9938 1000
52.85 1:53
W:25 0.28)
81.17 0.54
16.72 10:25
4.85
9

---

Published in Transactions on Machine Learning Research (02/2025)
13
EDHNN
Welch
AllSaTTransoormr
UniGCNTI
MultiSetMixer
HNHN
12
11
Average ranking
10
8
7
6
5
4
0 O 1 T
3
2
111
10%
20%
30%
40%
50%
avg
Figure 3: Average rankings at different training percentages, with the overall average performance shown on the right. The error bars indicate the standard deviation of the average rankings across multiple datasets.
node homophily plot in Appendix G connectivity.
Please refer to Scction 5.4 for more experiments the impact of
Finally, we highlight the strong overall performance of the non-inductive baselines (MLP, MLP CB) across most datasots. Notably, HNN architectures significantly outperform them in only 3 out of 10 cases (Cora, CORA-CA, DBLP-CA). This fact showcases that, so far, features are being more representative than con- nectivity in most considered hypergraph datasets -a relevant insight for Q3.
5.2 When are HNNs exploiting the connectivity?
ModelNett0
DBLP-CA
CORA-CA
Citeseer
Cora
NTU2012
20Newsgroups
Pubmed
Normalized Accuracy (rank)
EDHNN
MultiSetMixer
AllSetTransformer
homophily
CB homophily
Homophily (rank)
Homophily (rank)
Homophily (rank)
Figure 4: Joint visualization of rank dependencies, showing Norm. Aec. versus A Homophily (Eq. at step t= and f 0.1) and CE Homophily (Wang et al., 2023). Norm. Acc. (Eq. 14] is assossed for various instances of model (specified in column litles), with model B being MLP CB. Both axes represent rank values, with lower values indicating better metrics. Arrows denote the rank shift in homophily between CE homophily and A homophily for cach dataset.
Motivated by the previous observation of the general results, in this section, we investigate when HNN models actually take advantage of the inductive bias provided by the message-passigg scheme. To do so, we first propose a way to capture the influence of the bias in the downstream task performance -in an attempt decouple it from the impact of just the dataset features-, and then investigate if the datascts' homophily scores are able to account for the resulting observations. We argue this study provides valuable contribution to questions Q2 and Q3.
10

---

Published in Transactions on Machine Learning Research (02/2025)
In order to quantitatively assess the impact of inductive bias, compare the results of HNNS -ie., model A-, with those of another architecture that does not leverage the connectivity ii.e. non-inductive baseline, model B. Specifically, we measure the difference between the accuracy of model A (acca) and B (aceB) (Table 2 shows the computed differences considering MLP and MLP CB as baselines). The real-world datasets employed in this shudy span diverse domains and, as depicted in Table [ this implies considerable variations in performance values across datasets. In order to mitigate such variability, we introduce the following normalized accuracy relative to acep:
Norm. Acc. (acea (uccaaooo))(00aaccaa)) accp).
(14)
Table 2: Difference in Accuracy between Model A and Model B; they represent, respectively, models with and without inductive bias.
Model A
Model B
Cora
Clseseer
Pubmed
CORA-CA
DBLP-CA
Mushroom
NTU2012
ModelNet
20Newsgroupe
ZOO
AllDeepSets
MLP
3.84
-1.40
1.91
8.96
6.57
0.05
6.79
1.39
0.26
3.97
MLP CB
3.05
-1:26
3.21
7.81
6.43
3.13
1.06
0.29
-1.91
7.49
MLP
6.27
0.45
1.61
11.16
6.84
0.04
8.52
2.60
0.66
5.90
MLP CB
5.48
0.59
2.91
10.04
6.70
3.12
2.79
1.59
-4.54
9.42
MLP
7.17
1.15
1.99
11.90
7.17
0.03
8,34
2.39
0.71
4.36
EDHNN
MLP CB
6.68
1.29
3.29
10.78
7.03
2.61
1.29
-4.49
7.88
MLP
4.79
-0.22
0.06
9.47
5.91
-0.33
9.20
3.07
7.64
2.95
MultiSetMixee
MLP CB
4.00
-0.08
1.36
8:35
3.77
-0.35
3.47
1.97
2.44
6.47
Next, are interested in assessing whether the homophily of the dataset can shed some light on the resulting normalized accuracy measurements. To that end, we consider two different homophily mcasures: on the one hand, our proposed A homophily between the two first steps of the MP (Eq 4 with t t= and μ == 0.1). On the other hand, Clique Expansion (CE) homophily, calculated over the clique expansion of the hypergraph following the approach of Wang et al. (2023). .. Tigure[fillustinnn
the rank dependency of normalized accuracy against these two homophily measures, with MLP CB serving the strongest non-inductive baseline (as show the ideal correlation -performance directly proportional to
indicated by Table D. Additionally, homophily- with the dashed diagonal, well as the rank shift in homophily between the homophily measures through the arrows. The difference between EDHNN, MultiSetMixer, and Transforaaerr lies in the way message passing propagates information (see Section 4.2). Mushroom and Zoo datasets were excluded due to Mushroom's 's discriminatory node features and Zoo's small hypernetwork size.
The comparison between CE homophily and A homophily reveals a notable trend, with A homophily consis- tently aligning closer, on average, to the middle dashed line -indicating higher positive correlation between performance and A homophily level in comparison to CE homophily. Remarkably, across all architectures, the highest normalized accuracy is consistently distributed across ModelNet40, DBLP- C-C, and CORA-CA, with A homophily ranking them as the top three accordingly. A striking shift in rankings is observed for the Pubmed dataset, transitioning from the most homophilic under the CE homophily measure to the least homophilic under A homophily. We associate this to the high percentage of isolated nodes (80.52%, see Table 8 while CE homophily scores are largely influenced by them, our proposed measure ignores self- connections. Additionally, the 20Newsgroup dataset occupies the last positions in both homophily ranks, aligning with our quantitative analysis findings.
In summary, we show the applicability of A homophily by showing that it exhibits positive correlation with respect to the ability of exploiting the connectivity by HNN architectures, significantly stronger than the CE homophily that is commonly used nowadays. Our findings underscore the crucial role of accurately expressing homophily in hypergraph networks, entangling with the complexity in capturing higher-order dependencies. Additionally, please see Appendix E.3 for the analysis on how parameter μ impacts homophily.
5.3 What is the impact of the mini-batch sampling?
Next, we examine the role of our proposed mini-batching sampling in explaining the general results shown in Table D. and investigate how it influences other models' performance. These experiments provide valu-
11

---

Published in Transactions on Machine Learning Research (02/2025)
20newsgroups
Pubmed
0.4
0.6
Node
Step and
Step
VIlla
Node
Step and 2
Step 1
0.5
0.3
0.4
0.2
0.3
0.2
0.1
0.1
0.0
0.0
Class 0
Class 1
Class 2
Class 3
Class 0
Class
Class 2
Figure 5: Class distribution shift induced by mini-batchinn 'Node' represents the original node class distri- bution, SSeep 1 and 2' the resulting one after sampling both hyperedges and nodes, and SSee 1 when only sampling hyperedges.
able insights Q2. Please refer to Section 5.5 for deeper analysis of how hyperedge-depennentt node representations impact HNNs.
Table 3: Mini-batching experiment. Test accuracy in % averaged over 15 splits.
Fibural
DORACCA
BALPAA
N7V500
Mock(Ncc))
AlSertmaaatooee (tetcbed)
TL.34
87.25 6.30
0.22
9131 2116
67.00 1:
1.17
122
10.20 6.38
2
2DEDNN
77.55 - 3.09
16:31 0.87
66.32 0:33
88.18 0.89
80455 0.39
88.5 0.01
8761 0.00
97:55 0.17
81:33 0.1
90000
2:40
MulSSeffie
T8:06
5715 0:5
65.4
0008 0.19
93.34
88.00 1.30
08.38 0.21
88.77 1:90
1:40
nistleca
16.38
Class distribution analysis. To evaluate and motivate the potential of the proposed mini- mini-batching sam- pling, we investigate the reason behind both (i) the superior performance of MultiSetMixer and MI.P CB on 20NewsGroup, NTU2012, ModelNet400, and (ti their poor performance on Pubmed. Framing mini-batching from the connectivity perspective presents challenge that conceals significant. potential for improvement (Teney et al., 2023). It is important to note that connectivity, by definition, describes relationships among the nodes, implying that some parts of the dataset might interconnect more densely, creating some sort of hubs within the network. Thus, mini-batching might introduce unexpected skew in training distribution. In particular, Figure 5 depicts the original class distribution of the dataset, and compares to the skewed distributions resulting from employing the correspouding steps of mini- batching (see Section 4.3). Note that, when sampling both hyperedges and nodes ('Step 1 and 2¹), dominant classos and 3 undergo undersampled, contributing to more balanced distribution in the case of 20Newsgroup. Conversely, for Pubmed, class 2 is undersampled, while the predominant 1st class experiences oversampling, further skewing the distribution in this dataset. This observation leads to the hypothesis that, in some cases, the sampling procedure produces natural shift that rebalances the class distributions, which in turn helps to improve the performance.
Application to other models. Furthermore, we explore the proposed mini-batch sampling procedure with the AllSe\Transformer and EDHNN models by just sampling hyperedges without additional hyperpa- rameter optimization. From Table we can observe drop in performance for most of the datasets both for AllSet Transfrrane and for EDHNN, but overall they in turn outperform the HAN (nini--atching) model. This suggests the substantial potential of the proposed sampling procedure.
Scalability. To evaluate the scalability of the proposed mini-batching approach, we compare memory utilization across the mid-sized datasels used in this study, as well as the large-scale Amazon dalaset, which consists of 100k hyperedges and 547k nodes. This comparison highlights the efficiency of our method when applied to larger datascts. On Amazon, MultiSetMixer achieves substantial improvement in memory efficiency, reducing utilization by an order magnitude compared Transforrerrrrer (from 19020MiB 1232MiB) while achieving comparable performance (92.22 0.84 vs 93.22 + 0.089). In contrast, the batched
12

---

Published in Transactions on Machine Learning Research (02/2025)
version of AllSetTrranfommer performs worse the same dataset, achieving 0.0234 while consuming around eight times more memory compared to MultiSetMixer.
Table 4: Memory usage comparison actoss datascts.
Dataset
Pubmed
DBLP CA
NTU2012
ModelNet40
20news
Amazon
TlsSeeTrensoomer
2536MLB
2306MBB
1206MBB
2126MBB
2352MBB
19020MIB
AllSet Transformer (batched)
2652MiB
1940M1B
1138MiB
2038MB1
4224MBB
8022MBB
Mixer (no batching)
124S0MBB
17850MBB
1576MIB
2728MIB
3039MBB
OOM
Muxer (batched)
1290MBB
2364MIB
1174MBB
1280MBB
1112MBB
1232M:B
5.4 How do connectivity changes affect performance?
In this section, we extensively explore the structure of datasets and assess model performance by manipulating the original connectivity of the datascts. The extent which the performance of the models is affected by changes in connectivity provides valuable information both on the properties of the datasets and the considered architectures.
We design two different experimental approaches, aiming to systematically modify the original connectiv- ity of datasets. The first experiment tests the performance when some hyperedges are removed following different drop connectivity strategies. The second one examines the models performance by introducing two preprocessing strategies on the hypergraph connectivity. Our findings below shed some light on the fundamental questions Q1, Q2 and Q3.
5.4.1 Reducing connectivity
This experiment aims to investigate the significance of connectivity information in datasets and the extent which influences the performance of the models. We divide this experiment into two parts: (i) drop connectivity and (ii) connectivity rewiring. In the first part of the experiment, we employ three strategies to introduce variations in the initial dataset's 's connectivity. The first two strategics involve ordering hyperedges based their lengths in ascending order. In the first approach, referred to as brimming initial 2% of ordered hyperedges, for a certain fixed fraction x. The second approach, referred to as retention, involves keeping the first x% of hyperedges and discarding the remaining 100- 100-x% x%. The last strategy instead involves randomly dropping a% of hyperedges from the dataset, and is referred to as random drop.
remove the
Results. The results are shown Table 5 and they indicate that connectivity minimally impacts CEGCN and AllSecTransformrr for the Citeseer and Pubmed datasets. On the other hand, MultiSetMixer performs better at the trimming 25% setting, although the achieved performance is on par with MLP, as reported in Table [ This indicates that the proposed model is negatively affected by the distribution shift. Conversely, for the Pubmed datasct, we observe that MultiSetMixer's performance improves, likely duc to the reduced impact of the distribution shift in this case. Notably, CEGCN shows improvements on 7 out of 9 datasets, with the most significant gains seen the ZOO dataset, where performance nearly doubles. Another interesting pattern is observed in the Cora, CORA-CA, and DBLP-CA datasets, where retaining only 25% of the highest relationships (retention 25%) consistently results in better performance compared to retaining 50% 75%. This outcome is surprising, as at. the 25% level, only small fraction of the higher-order relationships is preserved. In contrast, the trimming strategy shows the opposite trend, with performance declining as more higher-order relationships are removed. This phenomenon remains consistent across all models. Finally, when hyperedges are removed randomly, the performance consistently degrades as the percentage of removed hyperedges increases.
13

---

Published in Transactions on Machine Learning Research (02/2025)
Table 5: Drop connectivity. Test accuracy in % averaged over 15 splits.
Model
Taye
Cor
Closee
Pelused
DORA-C5
DBLF-CA
Mtelecon
NTUN35
Mod-INeeaa
200
Oriiiaal
P7:11 4: 1.00
71487 1.15
A6J54 0.45
R2:25
1:46
91.36 4 8:27
1,46
9670-1 0:35
11:10
0:10
T:00
2.85
Banakn 20%
76.8 1,06
70:53 5 1.70
65.57 0.47
50.35 1.51
96.35
0.25
95:1 0.5
55:45
3.74
96.55
0.26
63.35
2 0.52
56.72 = 6.97
4.35
Allbwybbets
Banden 316
Baoke 106
14.36
15.08
1,06
1:15
70:45-1
70:79 1.77
= 1.66
55.76
55.55
2 0.45
0:11
76.08 = 1.27
77.87
1.36
50:35
57.6
1.35
0.30
51.70
51.13
= 1.57
1.75
96.88
96.44
0.27
51.31
61.21
4 0.30
+ 0.50
51.03 = 6.55
54.15
6.66
7:30
1.30
Reteskon 2005
7687 0.95
70.90 + 1.53
55.34
0.45
5165 1:36
90.93
0.35
99.53
0.09
55.05
8.05
96.85
0.26
$1.00
0.46
56.67
= 7.86
3.63
Retratzon
73.80
1.36
73:44
1.3
1.40
50330 1.36
90.33
1421
19:11
52.43 0 2.23
95-42
1.25
03.04
0.45
50:31 0.25
6.73
0.04
Trinoinng 1075
15.34
0.50
02 148
0.16
1.53 77,50
0.31 96.64
69.78
Traumii 10%
76.08
1.30
70:88 8 1.45
88.53 0.45
1.52
0.27
18:38 s 0.09
54.74
3 1:81
96.82
0.20
63.20 8 0.45
56.08 = 8.45
6.20
Original
78.54 + 1.00
75:15
86.74 0.51
84.45 1:14
91.41
0.10
59:35 0:06
86.23 1.42
06.00
0.13
51:78 0.10
11:28 = 7:31
1,46
Razales 25%
76633
72.76
1.14
01.1
12.38 1.35
1120
19:04 2
1:36
11.1
81.70
0.32
50:37 =
3.50
AlS All
Bazaon 15%
meteation 35%
76.39
0.96
55.73 0.10
67:91
97.63
0.3T
$1.33
0.16
56.90 7.36
4.30
Reieaton 505
75.16
0.56
72.55 0 1.35
85.70 2 0.57
55.30 1.15
90.50
0.25
10.89 = 0.15
56.57 = 1.64
97.35 0.21
53.53 = 0.40
55.05 = 7.61
5.30
lonteatinn 75%
17.38
1.35
72.13
165%%
55.73
0.39
$1,07 1,30
14.55
MUJT 1: 0.04
55.18 1 1.70
07.27
1032
$1.36
0.15
50.97 6.90
5.90
Trauniz 55%
76.53
1.51
72:39
x 1.50
55.40 + 0.45
76.31
+ 1.30
$6.35
0.33
$9.24
0.35
56.35
1:66
97.30
0.24
53.55 0.55
05.06 = 7.70
8.03
72.37
1.37
72:32 1.30
15:40
11:00
77.41 1.75
1L.7
19:11
B-22
18.95 1.85
97.05
14:21
n1.45 0.50
15:74 8:30
7:30
78.35
3.31
72467
2 1.60
$5.46 4. 0.39
75:31
4 1.34
95.35
1135
16:12 B.9
51.86
1.86
91:30
1435
$3.41
0.53
90,00 8: 7.37
5.35
Orighal
78:16
1:34
13.05
1.44
85.0 01:17
K3.92
91.36
19.99
H6.26
1.56
1616
$9.16 06:1
66.61 8:00
2:25
Banke 20%
76.31
1.20
72:53 8: 1.34
85.29 + 0.47
55.42
1.36
90.55
0:25
18.55 + 0.30
57.65
1.46
07.87
0.10
53.38 + 0.58
56.10 = 7.76
3.55
Bandon 10%
77.36
1.34
72.54 5: 1.40
57.34 : 0.58
50.17 + 1.36
55:95
0.34
58.55 5 0.12
$7.43
1.44
S7.77
0:15
53:36
+ 0.55
57.33 = 6.21
6.00
Randen 7556
76.20
1.35
72:33
2 1.53
57.01 2 0.50
77.73 2 1.31
55.34
1025
13.53 2 1U50
57.56 2 1.56
97.40
= 1.55
53:06:
1154
57.05 2 8.50
7.30
Reteation 2505
78.00
0.82
79:67-2
1.94
86.10 0.51
1696
1.20
59-57 0.06
$7.50
1.45
51:01
0.15
7.56
4.00
Roteatoon
11.15
1.32
72.53
55.22 0.47
12.1 1.32
96:90
19:53 4 1U56
57.15
122
01:20 WAY
00:52
0.40
56.15
7.45
1.26
1.15
0.30
1.85
02.96
8.16
10%
3.30
72.30
1.61
1.42 78.65
ILTT
98.50
0.30
1.56
$1.32
90.35 7.00
6.35
Triuning 70%
77.27
1.06
72:69 6 1,31
87.38 0.58
0.96
$6.35
0.25
18.54
5 0030
57.85
1.60
97.86
0.36
$3.28 e 0.45
90.64 = 6.80
4.55
Origital
80.76
1.00
13.22
1:14
N6.12 0:47
85.17
1.29
81.38
0.28
5P:1 8:11
1:85
87.70
0.18
R1.84
0.18
56:23 = 6.90
1.30
Bazkes 35%
76,85
72.03
0.55
30.54
1.37
0.35
1964 0.36
56.37 =
37.15
0.27
75.70
0.49
Bunke 75%
1,60
1.25
0.24
93.45
0.25
0.51
50.13
Nettraton 25%
16.79
71.87
0.14
1.54
1.22
56.33 0.00
51.45
1.51
$1.03
1132
25.45
1..6
4.00
Reteaton 3055
75.71
1.44
73:26 5 1:14
55,37 + 0.36
50.26 : 1.55
56.91
0.37
18:57 = 0006
55.12 = 1.91
90.45 0.55
73:57
0.70
56.00 7.56
5.90
Reteatoo 75%
78:37
1.45
72400-2
1.35
25.34 2 0.41
78.56 2 1.24
$5.92
1415
90.99 :: 0.00
$3.00 2 1.80
B535 = 1424
07.15
11.76
13.97
6.64
6.35
Trammiu 25%
19,03
3.37
72.71
2 1.45
15:33 = 1.12
27.34 2 1.46
61.0
1625
15:15 1.46
$2.00
= 1.86
95.87 = 1430
75.15
16.44
56:15
=2 SUOT
5.00
Trloningg 10%
78.03
1.33
1.30
1.16
1176
19-11-
8:11
63:30
1:35
91.83
1.25
27.76
0.55
103
10:00
6.40
Traumiu
75.00
1.30
72.15 1.34
85.52 a 0.45
75.85 = 1.30
$5.44
0:25
0.31 90.56
55:34 = 1.50
96.95 = 0.25
75,70
0.54
99.74 =: 1.79
4.55
Originnl
76.55
1.58
71.15
67:11
17.50 1.51
$6.51
50:37
92:79 0.44
NA
41.53 9.15
5.17
Banden 201
75,35
1.55
71.53
1.65
57.53 5 1.47
78:00 1.48
57.55
1:35
94.35
2:00
$2.00 : 1.47
95.34
1134
NA
45.33 65.00
5.34
Rarden 50%
71.31
1.50
71.55
1.99
65:51 4 0.46
78:35 1.00
0.32 45.79
$1.11 - 9:00
51.73
4 1.41
$1.63
0.35
NA
17:00
6.75
6.45
CECAT
Reteation
Baskea 1055
75.90
75,25
1.45
72:37 2: 1.55
71:39 1.35
87/02 2 0.47
67:13 0.50
17.33
16.37
1.25
1.58
1.35
16300
95:30 = 1.40
95-45
1:19
$2.96
60.35
= 1.35
1:53
$4.54
81.30
= 1445
1.15
NA
NAA
19.57
0.50
8.35
6.35
6:23
Reteston
75.36
67.16
77:38 1.89
55.14 0.31
96-75 1:59
50.56
2.16
64)
NA
45.35 9,97
6:00
76,40
72
1.70
87.46
1.30
0:48
59.75
84.94
1.57
0.15
NA
3: T.38
3,67
trmant s4f
15,90
72:35
1.85
1:D 56.15
98.80
0.32
52.24
1136
NA
$7.25
Trinningg 70%
76.19
1.65
71.52 4 1.38
$7.3 = 0.16
76.04 : 0.90
0.35
$9.31
= 0.34
$1.71 : 1.45
90.79 = 0.35
NA
$3.33
8 6.60
6:23
Origiaa
17.05
1.31
77.35 1.45
MK.12
0:25
$1:51
0:41
50.90
BAT
NA
15:33
0.53
Bunkes 50%
76.06
0.59
1.58
65.61
0.39
93:11
0.46
50:55
90.86
0.66
NX
40.74 6.29
1434
Rankos 705
1.65
1.50
1.56
0:66
$3.01
50.56
NA
53.20 5.50
cac
Reteaton 25%
16.33
73.57 1.42
15.34 0.56
1.35
9634 0.45
79.20
1.42
NA
43.57 7.30
5.30
Roteatio 50%
18,43
3.25
70.53 5 1.52
66.16 2 0.54
16.37
1.40
st.6 0:25
94.17
0.40
18.33 = 1.90
9000 = 0.56
NA
45.77 0.56
6.55
Reteetin 15%
13-33
1.35
71.79-1
1.12
67.11 0.53
78.38
1:48
0.36
$1.71
0.39
79.47
1.11
02.38
NA
10:36
5.12
5:11
Tranmiz 26%
16.58
1.86
73.20 + 1.33
87.30 = 0.51
14.34 = 1.31
54.97
(.31
08:60
:: 0.11
85.10
2: 1.60
91.85
NA
87.60 2: 7.31
3:44
Demonny
76:59
71:83
57.05 1.35
74.06
1.8A
1033
50.73
1.54
NA
71.38 6.50
4.00
Trauning
76.33=2
73.45
86.76 =
74.36 1.30
55.55
0.35
$8.44 0.53
50.35 =
90005
0:37
NA
35.46 7.17
6:25
Origiaal
79.53
1.33
12.57
1.26
W6.3T 0.35
K3.53
91.21
$5.51 0.51
REA0
1.36
91.50
0.35
R0.75 0.53
K5.75
2,40
Banden 20%
76,74
1,30
72.35
8 1.25
66.54 0.56
$1.36
1.34
9668
0.35
15.55 = 0.55
55.34
1.76
94.75
025
60.6
0.46
58.10 = 6.71
4.30
Barden 306
17.03
1.16
72:11
1.12
$5.37
0.15
70.13
1.11
1.35
55-1
1.15
65.33
1.75
1.76
0.50
67.56 1 6.97
6.00
BCBA
Nettrttion
Buokn 70%
76.56
1,60
72:23
72:39 1.17
= 1.35
26.72 2 0.56
0.52
77.11
1.26
1.00
ST
0.34
1025
15-2 1.30
$5.59 0.76
54.55
60.35
1.44
1.05
06-87
21:33
1421
0.34
79,49
0.45
58.15
16:00
= 6.56
3:65
7.30
Rotestion
1.37
90.17
0:39
55.22 0.36
$4.80
1.70
94.34
0.29
0.44 76:56
75-77
Trimming 55%
15.58
1.31
1.75
56.53
1.35
0.10
06.87 + 0.11
$5.00
95.00
0.56
58.66
01.92 3: 7.10
5:50
Tramiu 10%
17.76
1.25
72.10 5 1.49
55.16 0.46
13.28
1.17
36.57
0.35
18:4 3
1.37 54.35
94.73
0125
0.50 80.02
$3.36 6.75
6.45
Termning 75%
78.35
1.30
72.32
4 1.2
$6.41
0.53
78.55
0.90
65.09
0.3T
$0.13
4 D.16
40.00
1.60
91.61
0.37
0.57
66:30 - 1:10
5.30
Original
79.35
1.335
72.34 3.6%
#EJUT 0.35
83.45
1-22
91.26
1126
15:1 0.54
H5.71
1.4M
$4.30 = 1435
HU.76
0.32
10.23
2.50
50%
90.11
0.3
$5.15
1.65
0.55
1.25
0.50
BCNS
Retealin 2055
Retratio 35%
Randon 75%
76.56
17.77
16:99
1.36
1.50
75.16 1.56
72:20 0: 1.32
56.52 2: 0.45
0.53
0.56
$5.30 = 1.25
77.03 1.37
90.21
0.27
1.55
0.31
$430
58.23 0.36
$4.33 = 1.77
$5.19 1.37
1.58
00.57
$4.54
91.33
0.55
0.51
79.50
53.13
79:18
0.11
0.15
0.46
16.77
63.1 6.56
6.56
6.45
7:30
4.35
lentrabin 75%
17,05
1.53
72.35 1. 1.40
15.79 1 0.47
SU.S 0.95
K0.03
1035
11 1.30
11.15
1.10
90.31
1130
79.15
0.54
.1.36
6.77
6:00
Traunizz 55%
76.68
1.51
71.51 = 1.63
26.53 : 0.47
76,71 + 1.44
55.75
0.41
00:37
# 0:1
$5.85
0 1.67
95.02
= 026
88.56
0.56
01.99
=: 7.10
5.53
Drmuntur
77.28
1.25
73.03
2
15.26
0.
21.49
1.28
ab.45
1434
54.57
: 1.45
94.73
1626
53.09
6.30
Trinning
72:07:2
+ 1.55
0.53
76.78 1.34
0.34
90.73
0.3
1.55
91.6
56:30
5.40
Orighaal
76.76
1.31
06.86
1.56
K2.32 11.22
77.48
1.14
MEAT
3.32
8851
17.55
06:00
1:55
$1.54
3.40
Bankon 20%
15.50
1.76
34.77
1.68
65.50 = 0.62
36.11
1.55
$1.18
0.30
0001 = 4.77
$7.42
= 1.55
47.75
1.85
17.50
0.4
$1.4] = 6.55
6.05
Randen 316
33.75
2.3m
36:81
1.72
09-17 2 1119
80.11
1.87
67:35
231
67.99
19:38
2 3.12
= 5.5%
71.24
51.07 12 $4.00
6.90
PiperE2N
Barkees 70%
45.48
3.51
16:31
1.85
73:59 = 0.65
17.35
1.94
50.31
074
16-21
= 1
15.11
= 1-01
4505
2.08
43:76
$1.90 6.06
6.90
Reteatio
37.38
1.80
35,87
1,80
05:73 1.55
53:44 1.22
133
00.61 + 411
51.13
= 2.50
$1.54 9.54
4.55
1.5
1.86
2.55
0.51
Trmoni 30%6
50.35
1.72
55.35
1.57
21:6
1.50
52.17
2.74
K7.51
1.51
0.59
16.07
1.50
Trammiz 20%
16.30
2.74
44.4
= 1.35
71.15 = 0.55
19.4 2 2.35
$2.55
0.45
$4.77 = 7.40
19.12
1.51
65.29
5.55
45.07
1.36
G1
10.20
6.70
Trinming 10%
14.70
1.58
36.60 = 1.59
69.39 5 0.54
37.53 2: 1.56
$1.69
1.52
61.73 3:19
$5.00 0 2.77
73:66
= 1.75
73.12
0.68
00.00
11.14
7,00
Origins
75:00
1:24
73.85
$7.29 0L3S
82.74
90.08 0.10
15:55 0.36
85:00 1.30
08:38
0.11
85.8
1.90
55:05 12
2.65
0.31
$9.17
0.30
30%
Bunka 10%
1.35
72.31
0.51
0.35
38.12
0.26
0.65
Metration 25%
17.88
23.33 5 1.45
55.35
3.00
1125
19-07
1..6
$7.14 1.7%
$7.32 1137
18.77
4.40
Roteaton 105
76.56 = 1.50
71.49 6 1.7%
87.08 9 0.99
$1.13 + 1.36
$6.90
0.34
99.36 0.30
56.11 = 1.96
91.5 = 0.27
55.15
1.42
89.12 +: 7.50
5.00
Reteatin 75%
76.23 3.52
23.23 2 2500
67.38 2 0.77
79.39 2 1.26
15:73
1425
15.34 1:00
15.15 2 1.35
15.75 = 1632
53.97
1.27
58:30 02 5.97
7200
Trinning 30%
75.07
1.41
72-41 4: 1.61
87:10 0.56
73.51
1.35
45.45
0.31
08355 3: 0.07
51.11 + 1.47
91.5 0.25
78:29
0.15
55.57 5.75
6.53
Trammiu
76.34 1.35
72:17 2
8T.42 0.30
76.77 1.50
95.05 0U37
15:15 0.15
55.23 = 2.94
= 6125
71422
0.56
56.32
6,70
14

---

Published in Transactions on Machine Learning Research (02/2025)
Table 6: Rewiring connectivity, Test accuracy in % averaged over 15 splits.
Made
Trye
Own
Chowee
Palrred
Me
200
Zalel Baxd
#2.2 + 1:12
10.00 EMY
0.40
11.13
0.00
16.00
0.11
1036
0.04
05.13
1:39
10.30 0.11
1078 0.15
01.08 a: 7:34
1,06
AlDayySos
Jarenn
18:00
1:11
Latel Bend
85.03 = 1:98
18.40 # 1,45
10.10 =: 0:43
91:78 2 0.80
96.76 # 0.30
29.36 0.05
94.85 2: 1.04
50.68 0,00
10.35 2 0.05
14.10 #: 8.91
1,00
fourrr
27.44 1.41
72.5 =
58.09
0.41
01.10:4
1.31
0.06 90.96
81:50-22
1:24
91.39
51.35
16.53
2:15
Orgiral
1/02
1/42
9:31
Latel Bund
02.13 2: 3:31
13.23
4: 1,84
89.18 8: 0.30
80.99 28 0.95
14.77
0.38
90.33 = 0.07
92.87 2: 3.02
10.31 0.30
19.70 2 0.30
94.74 2: 6.35
1,00
>-neree
76-99
1.1
29788-1
1.50
66.03
15
16:55 0:67
26-05-11
1.28
16 10 0.31
61:05 + 0.44
83 13
7.89
3.01
Orgiral
Zatel Beed
94.0 = 1:23
76.70
1.M
10:88
0:35
190.09
07:39 9: 0.11
90.96 0.09
99.06 =:: 1:12
19:39 2: 0:00
10.85 0.05
92.44 #: 5.72
1,00
EDINX
Aaxxi
15,25
188
13:31
1.36
55.36
= 0.43
1.33
45.45 1.36
10194 BLE5
05.31
1.41
(6.58 0.16
51:34 9 0.35
05.08.0 7168
2:35
Orgical
1.00
60.13
1/02
1/63
00.70:0.10
0.41
1.99
5:05
Latel Boad
89.05 =: 1.05
17.82
1.80
50.28
0.33
931.42
38 0.88
99.20
0:35
99.91
0:07
94.23
2:: 0:77
99.20
0:34
0031
98.85 $ 7.89
1,00
CREATT
>-nsene
15-15
1.51
72.55
1:10
67.33
L 0:47
07:01
1.51
07
01.9
07.85 L. 0.73
65-08
1.00
16:10 01.6
00M
04:00
8:28
2:31
Orgiral
4..5
71.66
1:11
67.15
0.49
77.50
3.551
14.T4 11.0
16.51
$3.05
1.991
00:79
11.44
00M
41:00 5.18
9.67
Tatel Bood
88.90
95.00
1,58
10.06
0.445
10.85 0.05
1:31
0:0.1
000
05.00
7.08
OROON
Janee
75.29
1:53
73.00
18
67:33
= 0.31
76-03:44
1:41
04.51 0.0.7
65:05:1
1.71
00-17 0.51
00M
73.79
7.90
2:41
Orgival
37.0
1:31
20.55 16
67:01 - 0.03
27.05
1.65
19
04.91 0.0.1
03:00-11
1.24
14104 0.45
00M
05.90
1.41
5.54
Zated Boad
0.06
BCRA
17.11
1.41
7214
1.58
50,29 = 0.45
$1.10
1.51
$0.42 0.20
06:35 =: 0.27
6/.02
1.55
06.56 0.16
50.35 8 0.37
$4.10 1,85
2.00
Orgiral
79.31
1.31
7457 0 1.06
14:37 a 11.35
03.03
3.12
11.23 0 11.28
16:31 a 1.54
63.001
0: 3.44
14.70 2 0.25
10.75 à 16.3
99335 1.01
2.41
Label Boad
IINN
17.51
1:11
1.56
56.99
00.10
1.35
16.55 0.37
07.50-1 1.51
50.35 0.57
54.30
5.80
5.01
Orgiral
7953
1.30
73.54 L 1.06
54-97
L 0.33
63-45:1
1- 3.55
91.56 L 01.96
16:91 1 0.51
65.71
1./48
94.70 0.25
50.75 6 0.53
09/05) 31 8181
3:91
Basd
78.20-
0.45
53.07
45.20
1.97
1.35
3.00
5.20
Orginal
76.98
+ 1:11
09.06 1:56
60:33
77:08-:3 - 9:144
89.00 3.30
06:51 4.95
47.05 + 5.01
06. 10 7
50.84 + 0.43
51 04 + 9:88
1.50
Lale Baxd
95.01
1.09
1:46
00:43 =: 0.36
93.94
1.09
0T 0.30
90.95 =: 0.05
05.99
10.87 =: 0.06
91.229 7.85
Mal.B:Maasz
Orgiral
76,06
1.24
1.50
K7.19 a
43.74
7.41
18:34 a 0:33
04.00 3.30
54.18: 5:00
2:45
Label Bued
76.04
12.41 1.47
86.02 0.30
74.77 3:36
54.8 0.28
10.37 0.06
03.34
53,03 (6.1
07.00 7.33
MIP-
Orgiral
7006 1:90
21.00
0:51
0.44
$3//3.111
0.33
A6.13 2.88
5:31
5.4.2 Rewiring connectivity
In this experiment, we preserve the original connectivity while investigating the influence of homophilic hyperedges on performance. To do so, we adjust the given connectivity in two different ways.
Label-based Rewiring. Our first strategy aims to unveil the full potential of the homophily measure for cach dataset. To that end, we split hyperedges into fully homophilic ones based on their true node labels. While this approach is not applicable to practical evaluations, it represents meaningfiin meaningful baseline to assess the meaningfulnees of pure homophilic connections.
Feature-based Rewiring. In contrast, the second strategy explores the possibility of partition- ing hyperedges based on their initial node features. This approach, supported by the hypothesis that nodes of the same class might have similar features, represents an attempt of casily dividing hyperedges into more homophilic connections -its practicality assured given that only features are taken into account. In particular, the hyperedge splitting results from applying multiple times the k-means algorithm for each hyperedge e E E. At each iteration, the number of centroids m is varied from 2 to min(C, |e(), where we recall C is the number of classes; the elbow method is then used to determine the m value for the optimal hyperedge partitioning.
Results. As shown in Table 0 the fabel-based strategy enhances performance for all datasets and models; as expected, pure homophilic connections enhance performance. Notably, the graph-based method CEGON achieves similar results to HNNs with this strategy. Additionally, on average, only CEGCN performs better with the k-means stralegy, and this method also miligates distribution shifts for MultiSetMixer. These findings collectively suggest the crucial role of connectivity preprocessing, especially for graph-based models.
5.5 Benefits and challenges of dypereegeedependeent node representations in hypergraphs
In this section, we provide a deeper comparison between lifted models, such as AllSet AllDeepSets, and EDHNN, and the MultiSetMixxr, which allows for hyperedge-dependdnt node representa- tion and thus leverages specific characteristics of hypergraph networks.
Table D highlights MultiSetMixer's superior performance on three datasets: NTU2012, ModelNet40, and 20 20Nesggrups.. The improved results on 20Newsgroups can be attributed to two aspects (i) the pooling
15

---

Published in Transactions on Machine Learning Research (02/2025)
operations over the hypeeredgedepeeneent node representations, and (ii) distribution shift (see Section Regarding (i), Table [ highlights that the distribution shift, affects the MLP CB model, resulting in performance of 86.13 - 2.82, which already outperforms all HNN baselines. However, the MultiSet.Mixer achioves an even higher performance of 88.57 = 1.96. Notably, the best-performing HNN baseline achieves 81.64 + 0.49, while the MLP baseline achieves 80.93 + 0.59, showing an absolute difference of approximately 0.5%. In contrast, the MultiSetMixer leverages the introduced MultiSet messago-passingg mechanism to achieve = 1.96, resulting in an improvement of 2%. Our hypothesis is that, in this case, MultiSetMixer works by giving each document separate representation for every group (hyperedge) to which it belongs. This means the model can adjust how it sees a document depending on the context of each group. By doing this, it captures how a document is similar to or different from others in each group; which is a strong clue for classification. This flexibility makes it much better al handling situations where important relationships aren't limited to one group but spread across many. Traditional methods, with unique representations, struggle in these cases because they miss those cross-group connections. Regarding (ii), the uncontrolled distribution shift might negalively impact MultiSetMMxer's performance on the Zoo, Mushroom, and Pubmed datasets. As shown in Sections [5.4 and 5.4.2 this distribution shift can be mitigated by modifying the initial connectivity. Specifically, Table 0 demonstrates that applying k-means clustering to adjust the initial connectivity allows MultiSetMixer to achieve better performance on Zoo, Mushroom, and Pubmed. In the case of Mushroom, this approach enables performance comparable to models like AllSet Transforrer,, AllDeepSets, and EDHNN, At the same time, modifying the connectivity on 20Newsgroups mitigates the distribution shift but results in a performance decline, bringing MultiSetMixxe's results closer to those of Transoorr,,,,r AllDeepSets, and EDHNN.
5.3
Additionally, we observed that MultiSetMixee excels in processing Computer Vision Vision/Graphies datasets such as NTU2012 and 4et This success is due to the initial graph construction, which involves lifting the k-uniform graph by constructing hyperedges based on the one-hop neighborhood of each node, On the Cora and Citeseer datasets, MultiSetMMixe outperforms AllDeepSets and performs comparably to AllSet- Transformer, all without requiring an attention mechanism. However, on CORA-CA, AllSet Transformer outperforms Mulri, which instead produces results similar to AllDeepSets. The strongest con- nectivity effects appear in naturally-ocouring hypergraph datasets like CORA-CA and DBLP-CA, where hyperedges are derived from metadata rather than constructed from existing graphs. In these cases, Mul- LiSelMixer's underperformance suggests the need to explore alternative MultiSet framework architectures, including attention- based approaches and permutationn-equvariin tors similar to EDHNN. Moreover, MultiSetMMxxr's lower performance on Pubmed and DBLP-CA can be attributed to the difficulty of processing the entire hypergraph in a single forward pass due to memory constraints when storing all hyperelgg-dependentt node representations. However, employing the proposed mini-batching scheme still allows MultiSetMixer to achieve strong performance on these datasets.
continuous hyperedge diffusion opera-
At the conclusion of this section, we summarize the main benefits and challenges of sxperedge-dependees node representations in hypergraphs. The key advantage lies in their ability to model and capture com- plex interactions within hypergraph structures, as demonstrated by their superior performance on certain datasets, especially those involving computer vision and large-scale hyperedges. However, challenges arise in managing the scalability of these models, particularly in scenarios with large hyperedges or datasets, where memory constraints and distribution shifts can negatively impact performance. Despite these issucs, careful modifications to the connectivity and the use of mini-batching strategies can miligate some of the limitations, offering a promising path forward for further optimization.
6 Conclusion and Discussion
This section summarizes the key findings of our extensive evaluation and the proposed frameworks, relating them to the fundamental questions that motivated our work.
Q1: Can the concept of homophily play a crucial role in HNNs, similar to its significance in grsph-based rescarch? We show that the concept of homophily in higher-order networks considerably more complicated compared to networks that exhibit only pairwise connections. To address the issuc, we introduce novel message passing homophily framework that is capable of characterizing homophily
16

---

Published in Transactions on Machine Learning Research (02/2025)
in hypergraphs through the distribution of node features well as node class distribution. In Section we present A homophily, based on the dynamic nature of the proposed message passing homophily, showing that it correlates better with HNN models' performance than classical homophily measures over the clique-cxpanded hypergraph. Our findings underscore the crucial role of accurately expressing homophily in HNNs, emphasizing its complexity and the potential in capturing higher-order dynamics. Moreover, in our experiments (see Section 5.4.2), we demonstrate that rewiring hyperedges for perfect homophily leads to similar results for graph-baser methods (CEGCN, CEGAT) and HNN models. In conclusion, the proposed framework offers robust foundation for exploring homophily in higher-order networks and can be seamlessly adapted to incorporate other characteristies (such as the distribution of node features, which parallel the node label homophily examined in this study). While we presented this framework in the context of hypergraphs, its adaptability allows for slrwightforward extensions to other topological domains.
Q2: Given that current HNNS are predominantly extensions of GNN architectures adapted to the hypergraph domain, are these extended methodologies suitable, or should we explore new strategies tailored specifically for handling hypergraph-basedd data? The three main contributioos presented in this paper Messagee Passing Homophily, the MultiSet framework (integrating existing HNNs within unified framework), and the formulation of a novel mini-batch sampling scheme to address scalability issues- are directly inspired by the inherent properties of hypernetworks and their higher-order dynamics. Based on our experimental results and analysis, the proposed methodologies open an interesting discussion about the impact of ways of processing hypergraph data and defining HNNs. For instance, our muni-batching sampling strategy -which helps address scalability issues of current solutions- allowed us to realize the implicit introduction of node class distribution shifts in the process. This study could potentially lead to the definition of mcaningful connectivity rewiring techniques, as we alrcady explore in Section 5.4 Furthermore, we show that the introduced message passing and A homophily measures allow for deeper understanding of the hypernetwork topology and its correlation to the HNN models' performances. Overall, this study focuses on comparing graph-based extensions for modeling hypergraph networks and methodologies that incorporate hypergraphi-specffic characteristics. This approach allows for identifying common failure modes in current hypergraph modeling techniques (Section 5). We argue that these contributions offer a new perspective on processing higher-order networks that extend beyond the graph domain.
Q3: Are the existing hypergraph benchmarking datasets meaningful and representative enough to draw robust and valid conclusions? In Sections 5.4 and 5.4.2 we demonstrate that the significant per- formance gap of HNNs models on Cora, CORA-CA, and DBLP-CA CA is primarily attributed to the hyperedges with the largest cardinalitics. Further analysis using A homophily reveals that their notable improvement is strongly tied to the homophilic nature of the one-hop neighborhood. Additionally, the experimental results in Scction 5.1 and 5.4 highlight challenges for current HNNs with certain benchmark hypergraph datasets. Specifically, we find that lifted HNN models ignore connectivity for Citeseer, Pubmed, and 20Newsgroups, well as for the Mushroom dataset, due to highly discriminative features. Furthermore, observe that models that do not rely on inductive bias (i.e. do not use connectivity in the architecture), consistently exhibit good performance across the majority of datascts. This suggests that the expressive power of node features alone is sufficient. for efficient task execution. Addressing this gap presents an open challenge for future research endeavors, and we posit the necessity for additional benchmark datasets where connectivity plays a pivotal role. In addition to this, we believe would be also interesting to analyze datascts involving higher-order relationships where node classes explicitly depend on hyperedges, as introduced in Choe et al.) 2023). This represents an insightful line of research to further exploit hyperedge-based node representations.
17

---

Published in Transactions on Machine Learning Research (02/2025)
References
Sameer Agarwal, Kristin Branson, and Serge Belongie. Higher order learning with graphs. In Proceedings of the 23rd international conference on Machine learning, pp. 17 24, 2006.
Ryan Aponte, Ryan A Rossi, Shunan Gui, Jane Hoffswell, Nedim Lipka, Chang Xiao, Gromit Chan, Eunyce Koh, and Nesreen Ahmed. A hypergraph neural network framework for learning hyperedgeedepeendent node embeddings. arXiv preprint arXiw:2212.14077 2022.
Devanshu Arya, Deepak K Gupla, Stevan Rudinac, and Marcel Worring. Hypersage: Generalizing inductive representation learning on hypergraphs. orXiv preprint arXiv:2010.04550 2020.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey arXiv:1607.06.4 2016.
Hinton.
Layer normalization.
arXiv preprint
Song Bai, Feihu Zhang, and Philip HS Torr. Hypergraph convolution and hypergraph attention. Pattern Recognition, 110:107637, 2021.
Muhammet Balcilar, Pierre Héroux, Benoit Gauzere, Pascal Vasseur, Sébastien Adam, and Paul Honeine. Breaking the limits of message passing graph neural networks. In International Conference on Machine Learning, pp. 599 608. PMLR, 2021.
Ding- Yun Chen, Xiao-Pei Tian, Yu-Te Shen, and Ming Ouhyoung. On visual similarity based 3d model retrieval. In Computer graphics forum, volume 22, pp. 223-232. Wiley Online Library, 2003.
Guanzi Chen and Jiying Zhang, Preventing over-smoothing for hypergraph neural networks. arXiv preprint arXiv:2203. 2022.
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In International conference machine learning, pp. 1725-1735. PMLR, 2020.
Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized pagerank graph
neural network. Xii preprint
2020.
Eli Chien, Chao Pan, Jianhao Peng, and Olgica Milenkovic. You are allset: A multiset function framework for hypergraph neural networks. Tn International Conference on Learning Representations, 2022. URL __
Minyoung Choe, Sunwoo Kim, Jaemin Yoo, and Kijung Shin. Classification of edge-dependent labels of nodes in hypergraphs. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 298-309, 2023.
Yihe Dong, Will Sawin, and Yoshua Bengio. Hnhn: Hyporgraph networks with hyperodge neurons. arXiv preprinl ar Xiv:2006. 12278, 2020.
Dheeru Dua, Casey Graff, et al. Uci machine learning repository, 2017. URL http://archive. ics. uci, edu/mi. /ml 7(1), 2017.
Yilan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. In Proceedings of the AAAT conference artificial intelligence, volume 33, pp. 3558-3565, 2019.
Fangda Gu, Heng Chang, Wenwu Zhu, Somayeh Sojoudi, and Laurent El Ghaoui. Implicit graph neural networks. Advances in Neural Information Processing Systems, 33:11984 11995, 2020.
Jonathan Halcrow, Alexandru Mosoi, Sam Ruth, and Bryan Perozzi. Grale: Designing networks for graph learning. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2523 2532, 2020.
Matthias Hein, Simon Setzer, Leonardo Jost, and Syama Sundar Rangapuram. The total variation on hypargraphs-leamiing on hypergraphs revisited. Advances in Neural Information Processing Systems, 26, 2013.
18

---

Published in Transactions on Machine Learning Research (02/2025)
Jing Huang and Jie Yang. Unignn: unified framework for graph and hypergraph neural networks. In Zhi-Hua Zhou (ed.), Proceedings of the Thirlieth International Joint. Conference on Artificial Intelligence, IJCAI-21, pp. 2563 2569. International Joint Conferences on Artificial Intelligence Organization, 8 2021. doi: 10.24963 2t//249///jjai.....
URL https:://oi...... org/10. 14963/ijcai.. 2021/353
Main Track.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
Valerio La Gatta, Vincenzo Moscato, Mirko Pennone, Marco Postiglione, and Giancarlo Sperlí. Music rec- ommendation via hypergraph embedding. IEEE Transactions Neural Networks and Learning Systems, 2022.
Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutation--nvarint neural networks. In International conference machine learning, pp. 3744 3753. PMLR, 2019.
Jiachen Li, Chuanbo Hua, Jinkyoo Park, Hengbo Ma, Victoria Dax, and Mykel J Kochenderfer. Evolve- hypergraph: Group-aware dynamic relational reasoning for trajectory prediction. arXiv preprinl ar arXiv:2208.00 054700 2022.
Pan Li and Olgica Milenkovic. Inhomogeneous hypergraph clustering with applications. Advances in neural information processing systems, 30, 2017.
Tom Mitchell. Twenty Newsgroups. bttps://doi.ogg (00.24432/C5C335
UCT
Machine
Learning Repository, 1997.
DOL:
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-gon: Geometric graph convolutional networks. arXiv preprint arXic:2002. 05287, 2020.
Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. Multí-view convolutional neural networks for 3d shape recognition. In Proceedings of the IEEE international conference computer vision, pp. 2015.
Xiangguo Sun, Hong Cheng, Bo Liu, Jia Li, Hongyang Chen, Guandong Xu, and Hongzhi Yin. Self-supervised hypergraph representation learning for sociological analysis. IEEE Transactions on Knowledge and Data Engineering, 2023.
Damien Teney, Jindong Wang, and Ehsan Abbasnejad. Selective mixup helps with distribution shifts, but not (only) because of mixup. arXiv preprint ar 1ii::2305.16817, 2023.
llya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mip-mixer: An all-mlp architecture for vision. Advances in neural information processing systems, 34:24261-222 -242222 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Nate Veldt, Austin R. Benson, and Jon Kleinberg. Combinatorial characterizations and impossibilities for higher-order homophily. Science Advances, ((1)cabb,3000, 2023.
Petar Veličković, Guillem Cueurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1110.10003 2017.
Peihao Wang, Shenghao Yang, Yunyu Liu, Zhangrang Wang, and Pan Li. Equivariant hypergraph diffusion neural operators. In The Eleventh International Conference Learning Representations, 2023. URL
https:://oeenreview
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip Yu. Heterogencous graph attention network. In The world wide web conference, pp. 2022 2032,002, 2019.
19

---

Published in Transactions on Machine Learning Research (02/2025)
Jinfeng Wei, Yunxin Wang, Mengli Guo, Pei Lv, Xiaoshan Yang, and Mingliang Xu. Dynamic hypergraph convolutional networks for skeleton-based action recognition. arxiv preprint arXiv:21 12. 10570, 2021.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEFE conference computer vision and pattern recognition, pp. 1912-1920, 2015.
Chenxin Xu, Maosen Li, Zhenyang Ni, Ya Zhang, and Siheng Chen. Groupnet: Multiscale hypergraph neural networks for trajectory prediction with relational reasoning. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pallern Recognition, pp. 6498 6507, 2022.
Naganand Yadati, Madhay Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. Hypergen: A new method for training graph convolutional networks on hypergraphs. Advances in neural information processing systems, 32, 2019.
Naganand Yadati, Vikram Nitin, Madhav Nimishakavi, Prateek Yadav, Anand Louis, and Partha Talukdar. Nhp: Neural hypergraph link prediction. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM '20, pp. 1703-1714, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781 9781450368599
doi: 10.1145/3340531.BB 3411870. URL https:://oi.....
10. N0.1145/3340531 3411870
Chaoqi Yang, Ruijie Wang, Shuochao Yao, and Tarek Abdelzaher. Hypergraph learning with line expansion. arXiv preprint arxix:2005. 04843, 2020.
Jaehyuk Yi and Jinkyoo Park. Hypergraph convolutional recurrent neural network. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 3366-3376, 2020.
Junliang Yu, Hongzhi Yin, Jundong Li, Qinyong Wang, Nguyen Quoe Viet Hung, and Xiangliang Zhang, Self-supervised multi-channel hypergraph convolutional network for social recommendation. In Proceedings of the web conference 2021, pp. 413 424, 2021.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexan- der Smola. Deep scts. Advances neural information processing systems, 30, 2017.
Muhan Zhang, Zhicheng Cui, Shali Jiang, and Yixin Chen. Beyond link prediction: Predicting hyperlinks in adjacency space. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
Dengyong Zhou, Jiayuan Huang, and Bernhard Schölkopf. Learning with hypergraphs: Clustering, classifi- cation, and embedding. Advances in newral information processing systems, 19, 2006.
Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. AT open, 1:57-81 81, 2020.
20

---

Published in Transactions on Machine Learning Research (02/2025)
Supplementary Materials
A Extended Related Works on Hypergraph Neural Networks
Numerous machinc-loarning techniques have been developed for processing hypergraph data. One commonly used approach in early literature is to transform the hypergraph into a graph through clique expansion (CE). This technique involves substituting each hyperedge with an edge for every pair of vertices within the
hyperedge, creating a graph that can be analyzed using graph-baser algorithms (Agarwal et al., 2006 Zhoul
ct al. 2006 Zhang et al.. 2018 Li & Milenkovic 2017).
Several techniques have been proposed that use Hypergraph Neural Networks (HNNs) for semi-supervised learning. One of the earliest methods extends the graph convolution operator by incorporating the normalized hypergraph Laplacian (Feng et al.) 2019). As pointed out in Dong et al. 2020), spectral convolution with the normalized Laplacian corresponds to performing a weighted CE of the hypergraph. HyperGCN (Yadati ct al. 2019) employs mediators for incomplete CE on the hypergraph, which reduces the number of edges required to represent a hyperedge from a quadratic to a linear nunber of edges. The information diffuusion is then carried out using spectral convolution for hypergraph- based semi-supervised learning. Hypergraph Convolution and Hypergraph Attention (HCHA) (Bai et al., 2021) employs modified degree normalizations and attention weights, with the attention weights depending on node and hyperedge features.
CE may cause the loss of important structural information and result in suboptimal learning performance (Hein et al.] 2013 Chien et al. (022). Furthermore, these models typically obtaín the best performance with shallow 2-layer architectures. Adding more layers can lead to reduced performance due to oversmoothing (Huang & Yang 2021). Tn the recent study Chen & Zhang (2022), , an attempt was made to address oversmoothing in this type of network by incorporating residual connections; however, the method still relies on using hypergraph Laplacians to build weighted graph through clique expansion. Another method presented in Yang el al. (2020) introduces a new hypergraph expansion called line expansion (LE) that treats vertices and hyperedges equally. The LE bijectively induces a homogencous structure from the hypergraph by modeling vertex-lypperedge pairs. In addition, the LE and CE techniques require significant computational resources to transform the original hypergraph into a graph and perform subsequent computations, hence making the methods unpractical for large hypergraphs.
Another line of research explores hypergraph modeling involving two-stage procedure: information is transmitted from nodes to hyperedges and then back from hyperedges to nodes (Wei et al. 2021 Yi & Park,
2020 Dong et al.) 2020 Arya et al.) 2020 Huang & Yang 2021 Yadati et al. 2020). This procedure can be viewed as a two-step message passing mechanism. HyperSAGE (Arya et al. 2020) is a prominent carly example of this line of research allowing transductive and inductive learning over hypergraphs. Although HyperSAGE has shown improvement in capturing information from hypergraph structures compared to spectral-based methods, it involves only one learnable linear transformation and cannot model arbitrary multiset function (Chien et al. 2022 Moreover, the algorithm utilizes nested loops resulting in inefficient computation and poor parallelism.
UniGNN (Haan & Yang 2021 addresses some of these limitations by using a permuutationinnvaiian function to aggregate vertex features within each hyperedge in the first stage and using learnable weights only during the sccond stage to update each vertex with its incident hyperedges. One of the variations of UniGNN, called UniGCNII addresses the oversmoothing problem, which common for most of the methods described above. It accomplishes this by adapting GCNII (Chen et al., 2020) to hypergraphs. The AllSet method, proposed in Chien et al. 20022). employs composition of two learnable multiset functions to model hypergraphs. It presents two model variations: the first one exploits Deep Set (Zahcer et al.] 2017) and the second one Set Transformer (Lee el al. 2019). The AllSet method can be seen as a general lisation of the most commonly
used hypergraph HNNs (Yadati et al. 2019 Feng et al., 2019 Bai et al.) 2021 Dong et al. 2020 Arya et al. 2020). More implementation details and detailed drawbacks discussion can be found in Section 4.1 Although
AllSet achieves state-of thethhtttattttt results, it suffers from the drawbacks of the message passing mechanism,
including the local receptive field, resulting in limited ability to model long-range interactions (Gu et. al.
21

---

Published in Transactions on Machine Learning Research (02/2025)
2020
Balcilar et al. 2021 Two additional issues poor scalability to large hypergraph structures and oversmoothing that occurs when multiple layers are stacked.
Finally, we would like to mention two related papers that put the focus on beperedgeedepenndet compu- tations. On the one hand, EDHNN (Wang et al,, 2023) incorporates the option of byperedge-dependem messages from hyperedges these messages to generate a unique node hidden representation, and thus it doesn't enable to keep different hyperedge-based node representations across the whole procedure our MultiSetMixer docs. On the other hand, the work Aponte et al. (2022) does allow multiple hyperedge- based representations across the message passing, but the theorctical formulation of this unpublished paper is not clear and rigorous, and the evalua- tion is neither reproducible nor comparable to other hypergraph models. Hence, we argue that our MultiSet framework represents a step forward by rigorously formulating a simple but general MP framework for hy- pergraph modeling that is flexible enough to deal with hyperedge- beaddd node representations and residual connections, demonstrating as well that it generalizes previous hypergraph and graph models.
nodes; however, at each iteration of the message passing it aggregates all
B Details of the Implemented Methods
We provide a detailed overview of the models analyzed and tested in this work. In order to make their similarities and differences more evident, we express their update steps through standard and unified notation.
Notation. A hypergraph with nodes and m hyperedges can be represented by an incidence matrix H E Raxm If the hyperedge ej is incident to a node U; (meaning e E ej), the entry in the incidence matrix Hi, is set to 1. Instead, if 24 & ej, then H. =0
We denote with W and b learnable weight matrix and bias of a neural network, respectively. Generally, Fe and Ze used denote features for node v and a hyperedge e respectively. Stacking all node features together we obtain the node feature matrix X, while Z is instead the hyperedge feature matrix. (-) indicates nonlinear activation function (such as ReLU, ELU or LeakyReLU) that depends on the model used. Finally, we use to denote concatenation.
B.1 AllSet-like models
This Section addresses the models that are covered in the AllSet unified framework introduced in 4.1 and that can potentially be expressed as particular instances of equations and 7 For a detailed proof of the claim for most of the following models, refer to Theorem 3.4 in Chien et al. (002))
CEGCN / CEGAT. As introduced in the previous Sections, the CE of hypergraph g (V, E) is weighted graph obtained from G with the same set of nodes. In terms of incidence matrix, it can be described (CE) = H (Chien et al. 2022 A one-step update of the node feature matrix X E R can be expressed both in compact way as H (CE) X or directly as a node-level update rule, as
a:! (t+1)
Σ EEC. WiEE
Σ
x
(15)
Some types of hypergraph convolutional layers in the literature adopt CE-based propagation, for example generalizing popular graph-targeting models such Graph Convolutional Networks KKipf & Welling 2017)
and Graph Attention Networks /Veličković al. 2017
HNN. Before describing how HNN Feng et al. 2019 works, it is necessary to define some notation. Let H be the hypergraph's incidence matrix. Suppose that cach hyperodge is assigned a fixed positive weight že, and let Rm now denote the matrix stacking all these weights in the diagonal entries. Additionally, the vertex degree is defined as
22

---

Published in Transactions on Machine Learning Research (02/2025)
d, = Σ Eez
(16)
3EEu
while the hyperedge degree, instcad, is
b. = Σ I.
(17)
ViDEe
The degree values can be used to define two diagonal matrices, E Raxs and B E R
The core of the hypergraph convolution introduced in Feng al. 2019 can be expressed
((+1) x
Σ NEEE MiMEe
E
_e W(t)
(18)
where a is non-linear activation function like LeakyReLU and ELU, and W R is weight matrix between the (4)-th and (l + 1)-th layer, to be learnt during training. Note that in this case the dimensionality of the node feature vectors f(9 can be layer-dependent.
The update step can be rewritten also in matrix form as
( = xHZ2
(19)
where (++1) E Raxf and ( e R
In practice, a normalized version of this update procedure proposed. The matrix-based formulation allows clearly express the symmetric normalization that is actually put in place through the vertex and hyperedge degree matrices D and B defined above:
- 1-122 x (1) w())).
(20)
With respect to the previously described models, HCHA (Bai et al. 2021 uses a different kind of weights that depend on the node and hyperedge features. Specifically, starting from the same convolutional model proposed by Feng et al. (2019) and described in Equation 20 they explore the idea of introducing an attention learning model on H.
HCHA.
Their starting point is the intuition that hypergraph convolution as implemented in Equation 20 implicitly puts in place some allention mechanism, which derives from the fact that the afferent and efferent information flow to vertexes may be assigned different importance levels, which are statically encoded in the incidence matrix H, hence depend only on the graph structure. In order to allow for such information on magnitude of importance to be determined dynamically and possibly vary from layer to layer, they introduce an attention learning module on the incidence matrix H: instead of maintaining H as binary matrix with predefined and fixed entrics depending on the hypergraph connectivity, they suggest that its entries could be learnt during the training process. The entries of the matrix should express probability distribution describing the degree of node- hyperedge connectivity, through non-binary and real values.
Nevertheless, the proposed hypergraph attention is only feasible when the hyperedge and vertex sets share the same homogeneous domain, otherwise, their similarities would not be compatible. In case the comparison is feasible, the computation attention scores is inspired by /Velččkovi et al. 2017): for a given vertex v and a hyperedge e, the score computed
explo(sim(z, W, 2. W)))
H Egee, (xm(o W, 2g W)))
(21)
23

---

Published in Transactions on Machine Learning Research (02/2025)
where a is non-linear activation function, and sim is a similarity function defined as
aa ze) - a _ [ || øe],
(22)
in which is a weight vector, and the resulting similarity value is scalar.
HyperGCN. The method proposed by Yadati et al. (2019) can be described as performing two steps sequentially: first, a graph structure is defined starting from the input hypergraph, through a particular procedure, and then the well known CGN model (Kipf & Welling 2017) for standard graph structures is executed on it. Depending on the approach followed in the first step, three slight variations of the same model can be identified: 1-HyperGCN, HyperGCN (enhancing 1-HyperGCN with so-called mediators) and FastHyperGCCN
Before analyzing the differences among the three techniques, we introduce some notation and express how the GCN-update step is performed. Suppose that the input hypergraph G == (V,E) is equipped with initial edge weights {zeJeee and node features hev (if missing, suppose to initialize them randomly or with constant values). Let A denote the normalized adjacency matrix associated to the graph structure at time-step (I). The node-level one-step update for a specific node v can be formalized as:
((wri))
2 (++1) =a
Σ MEE.
A a=!!)
in which we what concerns All) it refers to the element at index u, v ( which can be defined in the following ways according to the method:
((+1) is the (1+ 1) step hidden representation of node v and & is the set of neighbors of v. For
I-HyperGCN: starting from the hypergraph g = (V,E), a simple graph is defined by considering exactly one representative simple edge for each hyperedge E E, and it is defined as (Ve, ue) such that
)22 This implies that each hyperedge is represented by just one pairwise edge (Ve, ue), and this may also change from one step to the other, which leads to the graph adjacency matrix ( being layer-dependent, too.
(Ve, ue) argmax,.ueel (W T (a!!) xx 1|2
2. HyperGCN: the model extends the graph construction procedure of 1-HyperGCN by also considering medialor nodes, that for each hyperedge e consist in K, := {k E k + ve, k + ue). Once the representative edge (ve, u.) is determined and added to the newly defined graph, two edges for each mediator are also introduced, connecting the mediator to both Ue and ue- Because there are 2/el -ql-3 3 odges for each hyperedge e, cach weight chosen to be 2/e|-3 in order for the weights in cach hyperedge to sum to 1. The generalized Laplacian obtained this way satisfies all the properties of the HyperGCN's Laplacian (Yadati et al.) 2019).
3. FastHyperGCN: in order to save training time, in this case the adjacency matrix Á is computed only once before training, by using only the initial node features of the input hypergraph.
UnIGCNII.
This model aims to extend to hypergraph structures the GCNII model proposed by Chen 2020) for simple graph structures, that is a deep graph convolutional network that puts in place an
et al.
initial residual connection and identity mapping as a way to reduce the oversmoothing problem (Huang &
Yang
2021
Let d, denote the degree of vertex v, while d, = म Eien de for each hyperedge e E E. single node-level update step performed by UniGCNII can be expressed as:
a(()
1
Σ cEE.
(24)
V
The
Vit'
24

---

Published in Transactions on Machine Learning Research (02/2025)
(+1) ((1 (11-8)2 B)I (11 + )
(25)
in which a and / are hyperparameters, I is identity matrix and zy is the initial feature of vertex i.
HNHN. For the HNHN model by Dong et al. (2020), hypernode and hyperedge features are supposed to share the same dimensionality d, hence in this case X E Raad and Z E Rmxd The update rule this case can be easily expressed using the incidence matrix as
z(+1)= = - X Wz + bz),
(26)
x(+1) ooH Wx x + bx).
(27)
in which a is nonlinear activation function, Wx. Wz Raxd are weight matrices, and bx bz R are bias terms.
AllSet. The general formulation for the propagation setting of AllSet (Chien et al. 2022 is introduced in Subsection 4.1 and, starting from that, we now analyze the different instances of the model obtained by imposing specific design choices in the general framework.
In the practical implementation of the model, the update functions fa-E and JE-V, that are required to be permutation invariant with respect to their first imput, are parametrized and learnt for each dataset and task. Furthermore, the information of their second argument not utilized in practice, hence their input can be more generally denoted set S.
The two architectures AllDeepSets and AllSet Transforree are obtained in the following way, depending on whether the update functions are defined either as MLPs or Transformers:
AllDeepSets Chien et al.
2022
fV (S) = fe--v(S) = MMP ES MLP(s));
2. AllSet Transforrer CChien et al.
2022). in which the update functions are defined iteratively through multiple steps as they were first designed by Vaswani et al. (2017).
The first set of operations corresponds to the self-attention module. Suppose that attention heads are considered: first of all, pairs of matricos K; (keys) and V, (values) with (1,.... h) are computed from the input set through different MLPs. Additionally, h weights 0,, E {1,..., h) are also learned and together with the keys and values they allow for the computation of each head- specific attention value O using an activation function w (Vaswani et al.] (017]). The h attention heads are processed in parallel and they are then concatenated, leading to unique vector being the result of the multi-head attention module MH After that, sum operation and a Layer Normalization (LN) (Ba et al. 2016) are applied:
K, = MI.P (S), V, MLP) (S), where i (l,,......
(28)
(29)
O w(O,(K,) )") >',, where iE (1,....,
(30)
MH (0, S, = h (0)
(31)
Y= LN + MH (,),
(32)
A ford-forward module follows the self-attention compulations, in which a MLP is applied to the feature matrix and then sum and LN are performed again, corresponding to the last operations to be performed:
= LN(Y MLP(Y)).
(33)
B.2 Other models
This Section describes the models that are considered for the experiments but that don't fall directly under the AllSet unified framework defined in Section 4.1
25

---

Published in Transactions on Machine Learning Research (02/2025)
The Equivariant Diffusion-based HNN model, shortened EDHNN Wang al. 2023 rep- resents the first attempt to draw connection between the class of hypergraph diffusion algorithms and the design of Hypergraph Neural Networks. The underlying motivation is that, by enabling the model to approximate any continuous equivariant hypergraph diffusion operator, a broad spectrum of higher-order relations can be encoded.
EDHNN.
In EDHNN, the hypergraph diffusion operators are learned directly from data, harnessing the expressive power of Neural Networks. This leads to the development of a novel Hypergraph Neural Network inspired by hypergraph diffusion solvers, with the subsequent operations executed at each layer described in the following.
The hyperedge-level feature update is performed as
(1+1) Σ )(x!!1),
(34)
MEE
and starting from that, the node-level update is defined as
=(+11)),, x d.)
x
(t+1)
&6
(1)) Σ
p(z!
ze
(35)
cEE.
In the equations above, & â and و are three MLPs shared across layers.
The Heterogeneous Graph Attention Network model (Wang al. 2019 is specifically designed for processing and performing inference heterogeneous graphs. Heterogeneous graphs have various types of nodes and/or edges, and standard GNN models that treat all of them equally are not able to properly handle such complex information.
HAN.
In order to apply this model on hypergraphs, (Chien et al. 2022 define a preprocessing step to derive heterogencous graph from a hypergraph. Specifically, a bipartite graph defined such that there is a bijection between its nodes and the set of nodes and hyperedges in the original hypergraph. The nodes obtained in this way belong to one of two distinct types, that the sets V and E (if they correspond to either a node hyperedge in the original hypergraph, respectively). Edges only connect nodes of two different types, and one edge exists between node u, E V and a node E E if and only if v E in the input hypergraph. We consider two types of so-called meta-paths (in this case, paths of length 2) in the heterogeneous graph, that are + E + V and E + V + E. We denote the sets of such meta-paths as Py and & respectively. Furthermore, let M denote the neighbors of node u, V through paths To E dy, and vice-versa let N denote the neighbors of node u. E through paths 7 4y.
At each step, the model updates separately and sequentially the node features of nodes in V and E. Consider for example the case of nodes in V (for nodes in E the process is the same, except. that &g is considered instead of Φy). The node-level update is performed as follows, for a certain u E V:
a(1
((()) dy x
(36)
x (1+1) = o
By u,
(t)
(37)
WEN
In the equations above, (( is meta-path dependent weight matrix while cv is an attention score computed between neighboring nodes in the same way proposed in Veličković al. (2017), through similar equations as 21 and 22 More generally, h attention heads may be considered, that give rise to different attention scores for each head and consequently multiple results for the node feature update, that then concatenated to obtain unique feature vector x ((+1)
by
26

---

Published in Transactions on Machine Learning Research (02/2025)
MLP. We also add the MLP model baseline; this model doesn't use connectivity at all and only relies the initial node features to predict their class. The node feature matrix X is obtained as
X (LP((((-))
(38)
MLP CB. This model employs a sampling procedure as outlined in Section 4.2 in which we straight- forwardly apply Multilayer Perceptron to the initial features of nodes. During the training phase, we incorporate dropout by applying an MLP with distinct weights dropped out for each hyperedge, resulting in slightly different representations for nodes for each hyperedge they belong to. Furthermore, we execute the mini-batching procedure in accordance with the guidelines presented in Section 4.2 Importantly, that these two choices affect the training approach significantly so that the results of this model are very different from MLP's performances: see, for example, Table
During the validation phase dropout is not utilized, ensuring that the representations used for each hyperedge remain exactly the same. Consequently, there is no need for the readout operation in this context. The node- level update is described by:
(t+1) 2 0,0
(()
(z!(L
(39)
x
Σ
x (T)
(40)
d
EE
Transformer. Along with MLP, we consider another simple baseline that the basic Transformer model (Vaswani et al.) 2017
Also in this case, let S denote the set of input vectors, and define as S MLP(S) the matrix of input embeddings, obtained from the input set through MLP. The operations performed S generalize the ones described for the Transformer module adopted in AllSetTransformes and they can be split in two main modules, that are the self-attention module and the feed-forward module:
1. Suppose that h attention heads are considered in the self attention module. First of all, h triples of matrices K; (keys), V, (values) and Q; (queries) with E {1...., h} are obtained from S through linear matrix multiplications with weight matrices W.K. W.V. and w that are learned during training. The result for each attention module is computed through the key, query and key matrices using an activation function w (Vaswani et al., 2017) and a normalixation factor dk, that corresponds to the dimension of the key and query vectors associated to each input element. The outputs of the different attention heads are then concatenated, leading to unique result matrix. After that, sum operation and Layer Normalization (LN) (Ba et al. 2016) applied:
K, = SW V = SW. Qi = SW.. = where iE (1,...,h)
(41)
O Q((K))
Vi, where ie f1.... h],
(42)
Vdhe
MH (Q, (Q,K, K, V) = 000
(43)
Y LN(S MH,,
(44)
Since it will be useful in the following we introduce, we denote by MAB the output of the multihead self-attention block, namely
MAB(S) L MH,,
( (45)
As described for Transfrrrrrrrr Chien et al. 2022 in the feed-forward module a MLP is applied to the feature matrix, followed by a sum operation and Layer Normalization. After that, the output of the overall Transformer architecture is obtained:
Yout IN(Y MLP(()).
(46)
27

---

Published in Transactions on Machine Learning Research (02/2025)
WHAT&Net In Choe et al. 20223 authors introduce classification of edge-dependent node labels. In the following section, we will describe their method and define all the components. The author introduced WithinATT, inspired by the Transformer model Vaswani et al. (2017), which adjusts node embedding by focusing on interactions with other nodes within the same hyperodge. Specifically, it applics a set of node embeddings as queries, keys, and values in the attention mechanism. This approach effectively captures relationships between nodes by computing the dot-product for each node pair within the hyperedge. However, caleulating the dot-product for all pairs results in a computational complexity that is quadratic relative to the hyperedge size, posing challenges for large-scale, real-world hypergraphs. To address this, they utilize the inducing point method from Set Transformer Lee et al. (2019), which achieves performance comparable to all-pair attention but with significantly greater efficiency. More precisely, Lee et al. (2019) introduce a matrix of trainable inducing points of shape way smaller than the shape of Q and K the MAB(Q, K) is approximated by MAB(Q, MAB(I, K)).
Let us now describe the WHATsNet model described in Choe et al. (2023). first we need to define the WithinATT module. Let _ be the set of embeddings of nodes in hyperedge in the t-th layer. Along with x!" Rl+xxd ATT uses number of trainable inducing points I E R where k is typically much smaller than maxege [e], in particula in their experiments they fix k 4. Then, WithinATT is formally expressed as follows:
WithinATT(XX (X) I.)) MAB ( (. x())).
(47)
They also define WithinOrderPE, positional encodings used edge- dependent attention between nodes within each hyperedge. To introduce WithinOrderPEE, the authors first define the order of each element in set A as follows:
where MAB is defined 450-
Order (a, A)
Ell'sa
(48)
a'EA
Then, given node centralities F Nxd, where dj represents the number of centrality measures, which corresponds to the dimensionality of the positional encodings, they define the WithinOrderPE of node within hyperedge as follows:
WithinOrddePE((t.eeFF = 11 [e]
1 Order(Fu.s (Fuji
(49)
This WithinOrderPE is then added to the node embeddings, which are subsequently fed into WithinATT. Four centrality measures are used: degree, eigenvector centrality, PageRank, and coreness. The notation c refers to the node embedding that incorporates the positional encodings from WithinOrderPE. To integrate these positional encodings, they first align the dimensionality of the positional encodings with that of the node features at layer t by using a learnable weight matrix. We are now prepared to outline the hyperedge and node update embeddings as presented in Choe et al. (023).. Specifically, the a hyperedge e's embedding (t) are obtained as
(t)
X (1) C
(a! II] WichinOrddePP((eee): uee))
(50)
x(t) E WithinATT(XX" ..
(51)
(0 MAB(
(52)
Similarly, node embeddings are updated by aggregating node-dependent hyperedge embeddings from Withi- nATT and WithinOrderPE. Specifically, node v embedding (() is updated as follows,
z (=( E WithinOOrdePP((uee)) e e Ex}.
(53)
(1)
5, = (ithinATT((
(54)
( MAB(x!'-)) 22())
(55)
28

---

Published in Transactions on Machine Learning Research (02/2025)
C Proof of Propositions
C.1 Proof of Proposition 4.1
UniGCNII inhorits the same hyperedge update rule of other hypergraph models (e.g. HNN Feng et al. 2019p, HyperGCN (Yadati et al. 2019 ) so it directly follows from Theorem 3.4 of (Chien et al. 2022) that can be expressed through 5 By looking at the definition of the node update rule of UniGCNIT (Eq. 24
and 25 we can re-express it
x (1(+1) = ((1 - x 3W()) (1-a) Σ vd
(t+1) 2e
car ) = -e--v((>(+)) JeEE,; !
(56)
vde
3EE.
Note that this is particular instance of the extended AllSet node update rule where only residual connection to the initial node features is considered. Lastly, it is straightforward to see that fe-vv is permutation invariant w.r.t the set (=((+1))cee. as it processes the set through a weighted mean.
C.2 Proof of Proposition 4.2
We prove this proposition by showing that we can obtain AllSet update rules 6 and 7 from our proposed MultiSet framework 10 This can easily follow by not distinguishing node representations among hy- perodges, so x JesE, =x(²) With this particular choice, we directly get 5 from [ and can be obtained from 9 by further disregarding the hyperedge subscript there is only single node representation to update. Analogously, we can get from 9 if we additionally do not consider node residual connections, so (x(*))) - simply bocomes x!!() Finally, the readout 10 can be defined the identity function applied the node representations at the last message passing step T.
C.3 Proof of Proposition 4.3
We also prove this proposition by showing that we can obtain EDHNN WWag et al., 2023 update rules from proposed MultiSet. framework
1 EDHNN hyperedge update rule can be expressed
((+1) )()(() fv-o(t=l Juev),
26
uEc
JUEN C xx(t), As for the node update rule, we have which is a particular instance of given that {a(h)) {
1++1) = & 2
x
Σ eEE.
p(x (1)
x 2o ((+11) ze
do) 202 do
= fe-v ((x JeEE, Jeee, (2! (e((0)))
(57)
where we recall that d, E. By disregarding the hyperedge superscript in Eq. 9 essentially making all hyperedge-based node representations the same one-, it is straightforward to see that the previous expression
57 is particular case of the MultiSet node update rule. Lastly, the readout step T0 can be just defined
the identity function, given that all involved hyperedge-based messages to nodes (i.e. p(x:)) =z ((+1))) are already being aggregated each iteration while updating the node state.
(1)
C.4 Proof of Proposition 4.4
The node updates in Wet output single node feature, while in MultiSet, the output of node updates results in multiple features for each node, i.e., one for each hyperedge to which it belongs. MultiSet framework preserves a higher level of generality by not enforcing pooling for edge-wise node representations at each layer, but a pooling operation could be intrinsically defined as part of the function in our Equation g leading to a single feature for each node as output.
We denote by x!!!) the set of the embeddings of nodes belonging to edge e. The functions are
(0) MAB 1-1 With m WithinOrderPE(uee) :0 e))) (o-ee(2)))eee))
(58)
29

---

Published in Transactions on Machine Learning Research (02/2025)
with WithinOrderPE defined as in equation 49 Similarly,
MAB(x WithinATT((( ([2! B WithinOrderPE(t.e))
(e- of o!--1).
: fe-v
(59)
where MAB is defined according to Equation 45 In the work of Choe al. 2023 the centrality measures used to compute the vectors in WithinOrderPE((r, rely solely the topology of the graph. However, centrality measures based on node and edge features could also be considered, and our framework is capable of capturing these as well.
C.5 Proof of Proposition 4.5
Tt is straightforward that functions fV-EE JE-vV and fv-v defined in MultiSetMiixer (Equations 11 are permutation invariant w.r.t their first argument: hyperedge update rule and readout 13 process it through a mean operation, whereas the node update rule only receives a singlo-clement set. The rest the proof follows from the proof of Proposition 4.1 of Chien al. 2022).
Experiments
D.1 Hyperparameters optimization
In order implement the benchmark models, we followed the procodure described in Chien al. (2022 :
particular, the maximum epochs were set 200 for all the models. The models were trained with categorical cross-entropy loss, and the best architecture
selected at the end of training depending
validation (Chien et al.) 2022] UniGCNII
accuracy. For the AllDeepSets (Chien et al. (2022),
(Huang & Yang 2021). CEGAT, CEGCN, HCHA (Bai et al. 2021). HNN (Feng et al. 2019). HNHN (Dong
et al. 2020 HyperGCN (aadai et al. 2019). HAN ( el al. 2019 and HAN (mini-batching)
Wang
ct al. 2019) and MLP, we performed the same hyperparameter optimization proposed in Chien et al. (0222
For both the proposed model and the introduced baseline, we conducted thorough hyperparameter search
across the following values:
learning rate within the range of 0.001, 0.01;
weight decay values from the set 0.0, - 5, 1;
MLP hidden layer sizes of 64, 128, 256, 512;
mini-batch sizes set 256, 512, with full-batch utilization when memory resources allow;
the number of sampled neighbors per hyperedge ranged from 2,3,5, 10.
It's important to note that the limitation of the number of sampled neighbors per hyperedge this small range intentional. This limitation showcases that even for datasets with large hyperedges, effective processing be achieved by considering only subset of neighbors.
The models' hyperparameters were optimized for 50% split and subsequently applied to all the other splits.
Reproducibility. We are committed to providing a comprehensive overview of our experimental setup, encompassing machine specifications, environmental details, and the optimal hyperparameters selected for each model. Additionally, the source code, including training/valle htt///vaavdaioontteee /estt splits, will be supplied with both the initial release and the camera-ready version, ensuring the reprodueibility of our results.
D.2 Further information about the datasets
For our experiments we utilized various benchmark datascts from existing literature on hypergraph neural networks, the statistical properties of which are in Table 7 For what concerns co-authorship networks (Cora-CA and DPBL-CA) and co-citation networks (Cora, Citeseer, and Pubmed), relied the datasets
30

---

Published in Transactions on Machine Learning Research (02/2025)
provided in Yadati et al. (2019) Additionally, employed the Princeton ModelNet40 (Wu et al 2015 and the National Taiwan University (Chen et al. 2003) dataset introduced for 3D object classification. For these
two datasets, we complied with what Feng et al. (2019) and Yang et al. (2020) proposed for the construction of the hypergraphs, using both MVCNN (Su et al. 2015 and GVCNN (Feng et al., 2019) features. Additionally,
we lested our model on three datasets with categorical attributes, namely 20. 20Newggrupps, Mushroom, and
Z0O, obtained from the UCI Categorical Machine Learning Repository (Dua et al. 2017). In order construct hypergraphs for these datasets, we followed the approach described in Yadati et al. (2019), where hyperedge is defined for all data points sharing the same categorical feature value.
The 20Newsgroups dataset consists of 100-dimensional attributes, where each attribute corresponds to a TF- IDF value. Hyperedges are formed by grouping all data points that share the same value for a categorical feature, with each hyperedge assigned a uniform weight of 1. For our analysis, we utilized a preprocessed version of the 20Newsgroups dataset (Zhou et al. 2006), where data samples (nodes) are represented by binary occurrence values for 100 words across 16,242 articles. These articles are categorized into four topics, aligned with the highest-level groupings of the original 20Newsgroups dataset. available al. the UCI ML Repository (Mitchell 1997p. The topic-specific group sizes in the preprocessed dataset are 4, 605, 3, 519, 2, 657, and 5, 461, respectively. However, the exact filtering criteria used in Zhou et al. (2006) to reduce the sample count from 20, 000 (in the original 20Newsgroups dataset (Mitchell 1997j) to 16, 242 or to determine the number of classes remain unclear. We hypothesize that the filtering process may have removed documents with insufficient content or relevance to the top 100 words. Furthermore, we observed that the original version of the dataset (from the UCI ML Repository) is balanced, with approximately 0.000 samples per class. In contrast, the preprocessed dataset (Zhou et al. 2006) shows skewed distribution with uneven group sizes of 4, 605, 519, 2, 657, and 5, 461. Based statistical analysis of the group distributions in the UCI dataset, we hypothesize that some original groups (e.g., all, comp, misc, rec, sci, soc, talk) were merged during preprocessing. As a result, we were unable to reproduce the dataset from scratch and relied on the preprocessed version for our experiments. While we acknowledge that the preprocessed version has certain limitations, such as its restricted 100-term vocabulary, the construction of hyperedges directly from node features, and challenges with reproducibility, limitations of HNN models. Despite its artificial nature, the dataset exhibits interesting characteristics, such as large hyperedges, which make it valuable for highlighting these challenges.
nonetheless provides an illustrative case for studying the
Table 7: Statistics of hypergraph datasets: |e| denotes the size of hyperedges while d, denotes the node degree.
Cora
Citeseer
Pubmed
CORACA
DBLP-CA
ZOO
20Newsgroups
Mushroom
NTU2012
ModelNet
1579
1079
7963
1072
22363
43
100
298
2012
12311
clusses
7
2
67
40
min le]
1
1
5
5
med [e]
40
537
72
5
5
max d,
145
88
99
23
18
17
44
5
19
30
min d,
17
1
1
avg dy
1.77
1.04
1.76
1.69
2.41
17
4.03
5
5
med de
17
5
4
CE Homophily
89.74
89.32
95.24
80.26
86.88
82.88
75.25
85.33
46.07
24.07
I Evey Ag
84.10
78.25
82.05
80.81
88.86
91.13
81.26
88.05
53.24
42.16
M Eeep 78.08
74.18
75.73
76.51
86.01
85.79
74.78
84.41
41.95
29.42
We downloaded co-citation and co-authoring networks from Yadati et al. 2019 Below are the details how the hypergraph was constructed. Cora-CA, DBLP: all documents co-authored by an author are in one hyperedge, following what was done in Yadati et al. (2019). Co-citation data Citeseer, Pubmed, Cora: all documents citod by a document are connected by a hyperedge. Each hypernode (document abstract) is represented by bag-of-words features (feature matrix X).
Citation and co-authorship datasets. In the co-citation and co-authorship networks datasets, the node features are the woodds representations of the corresponding documents. In co-citation datasets (Cora,
31

---

Published in Transactions on Machine Learning Research (02/2025)
Citeseer, PubMed) all documents cited by document are connected by hyperedge. In co-authored datasets (CORA-CA, DBIP--CA,, all documents co-authored by an author belong to the same hyperedge.
Computer Uision/graphin which == =.
The hyperedges are constructed using the k-nearest neighbor algorithm in
Categorical datasets. There are instances with categorical altributes within the datasets. To construct the hypergraph, each attribute value is treated as hyperedge, meaning that all instances (hypernodes) with the same attribute value contained in hyperedge. The node features of 20Newsgroups IDF representations of news messages. The node features of mushrooms (in Mushroom dataset) represent categorical descriptions of 23 species. The node features of a zoo (in ZOO dataset) are combination of categorical and numeric measurements describing various animals.
the TF-
Table 8: Node Connectivity Statistics. For brevity we use the following notation in this table: under the columns labeled |&,l = k, we report the amount of nodes that belong to A: hyperedges. This value can expressed in a more formal way as |o V [ = kl. Moreover, |,,]] = 0, denotes the number of isolated nodes. In addition, the columns labeled "% | k" indicate the percentage of nodes belonging k hyperedges relatively to the total number of nodes. Finally, Eeer |el corresponds to the number of bepereegeedepeendet node representations.
M
E.
E.
[T: E.1
27
E.
E.
E.
[2.1
26 E.
2 E.
2.
575
327
376
Citsseer
3312
1854
798
307
144
209
55.98
24.09
9.27
4.35
6.31
5307
Puhmed
330
313
2896
CORA-CA
2708
320
951
287
165
11.82
36, 76.74
35.12
10.60
6.72
4905
DBLP-CA
41302
$998
16721
92/9
6331
0.00
21.79
10.19
22.39
15.33
99561
Mushroom
8124
0
0
8124
0.00
0.00
0.00
0.00
100.00
40620
NTU2012
2012
173
256
296
1287
0.00
8.60
12,72
14.71
63.97
10080
ModelNet.0
12311
1491
1755
1394
7471
0.00
12.11
14,26
12.95
60.09
61555
20newsW100
16242
3053
3149
2720
7320
0.00
18.90
19.39
16.75
45.07
65451
700
E Experiment results
E.1 Additional experiments with heterophilic datasets
In this section, we broaden our experimental scope by including set of datasets that were previously used in Wang et al. (202)). These datasets, namely Senate, Congress, House, and Walmart, are classified as heterophilic based on the CE homophily measure. Notably, they present an interesting challenge as they do not have inherent node features. Therefore, generating artificial node attributes is necessary before applying hypergraph models. Duc to this constraint, we have docided to postpone the exploration these datasets future work, as we acknowledge the significance of addressing such limitations for a more thorough analysis. Table g displays the performance results of MultiSet EDHNN, and AllSet-like architectures the mentioned datasets. Notably, EDHNN consistently outperforms other models across all datasets, demonstrating superior results. MultiSetMixen ranks as the second-best model, deviating by one standard deviation from EDHNN on Senate and House datasets, and performing similarly to Transformee and AddDeepSess on Congress and Walmart.
Table 9: Additional hypergraph model performance benchmarks on heterophilic datasets (test accuracy in %). Results for AllDeepSets and Allseerrransfommrr are taken from Wang et al. (2233
Senate
Congress
House
Walmart
AllDeepSets
48.17 5.67
91.80
67.82 + 2.40
64.55 4 0.33
Transtrranrr
51.53 5.22
92.16
69.33 2.20
65.46 4 0.25
EDHNN
64.79 + 5.14
95.00 + 0.99
72.45 = 2.28
66.91 0.41
MultiSetMixer
61.34 = 3.45
92.13 1.30
70.77 2.03
64.23 = 0.41
32

---

Published in Transactions on Machine Learning Research (02/2025)
E.2 Rewiring experiment
Figure 0 visualizes the impact of connectivity modifications across different architectures and datasets. We organize results in matrix format where each column represents distinct rewiring strategy, and each row corresponds to a dataset. Model variants are color-coded, with dashed lines showing their original performance as baselines. The HNN models show little deviation from the corresponding baselines, regardless of the rewiring strategy. In contrast, CRGCN demonstrates significant. improvement, particularly with the trimming and random strategies.
Random
Retention
Trimming
100
90
80
ZOO
70
70
60
50
40
50
98
98
ModelNetdg
96
96
96
94
94
92
92
90
90
90
88
88
88
NTU2012
86
86
84
84
84
82
82
80
78
80
78
25%
50%
75%
25%
50%
75%
25%
50%
73%
AllSetTranstomer
CEGCN
MultiSetMixer
Figure 6: Visualization of model performance under different rewiring strategies. Columns show connectivity modifications, rows show datasots. Solid lines indicate modified architectures while dashed lines show bascline performance. Each color represents different model variant.
E.3
Analysis of μ parameter in A Homophily
This Section sheds more light on the impact of parameter y in our proposed A homophily. To that end, we show in Figure how A-homophily evolves as varies across multiple datasets.
We first observe that A A-homophily generally increases with μ. However, conceived as an homophily frame- work, the relevant aspect here the relative homophily values across datasets -and not the specific absolute values. In this regard, we can see that the relative homophilic ranking quite stable for different values of n,, and in particular around the 0.1 value (our choice for the homophily analysis in Section 5.2
E.4 Benchmarking models across multiple training proportions splits
33

---

Published in Transactions on Machine Learning Research (02/2025)
1.0
Cora
Citeseer
Pubmed
0.8
CORA-CA
4-homophily
DBLP-CA
20Newsgroups
0.6
NTU2012
ModelNet40
0.4
0.2
0.0
0.01
0.05
0.1
0.15
0.2
Figure A-homophily scores across several models for different values of μ parameter (Eq.
Table 10: Hypergraph model performance benchmarks. Test accuracy in % averaged over 15 splits. Training
split: 50%.
Palest
PHIP
NYUSI
70X0
27.11
WTT
0115
10.00
1:25
5B 16:1011:700
Allse Prandemee
72.50
0.84
86
0,51
81:0
01:10
8632
06.00
0.12
81.55
$1.00
UGRNND
78:46
7800
5..07
6.47
52.02
1.12
91.38 0.18
90.50 0.07
58.24 0 3.56
07.84
0.36
81.15
50.01 =
4.7G
EDRNN
90.74
1.00
73.59
1.14
80.12
0:47
55.17 0: 1.00
01:94 3 0.25
58.84 2:
58.04 3.65
07.70
0.30
81.61
20.10 0.09
2,00
CEGAT
28.33
1.00
21.08 3 1.11
ST.11
6.10
T0.30 = 1.54
20.24 11:01
11.51 2:
52.21
1.50
02.29
0.64
NA
41.32 9.15
12.01
CBCCN
07203
1.51
70L5T 2 1.19
ST00
6.82
12.85 = 1.56
25.12 0.25
94.01 2: 0.44
50.00
1.24
MUB
0.47
NA
40.23 0.81
12.96
79:33
72:37
96.7
6.50
8151
08.35
$5.91
50
61:00
0.33
7.85
2933
1:23
72.31
6.35
13-15
00:35
95.01
0.23
0.50
sustluath
71.55
6.53
60.74
58.55
1.30
98.35
68.57 1.08
55.16
MEP
71.88 4
0.51
T4.39
55.
16.11
06.13
51.61
16.30
MLP
23,27
72.00 +: 1.5
$7.18 2 6.40
70.27
1.00
84.77 0.41
98.31
70.70
1.56
06.8
80.31 0.20
85:15 0.90
13.50
Tousfoemer
24.15 =:
71.52 = 1.51
$7.87 # 0.40
78.1 =
85.25 0.30
99.00
0.08
$2.88 1.95
00.20 =
81.17 = 0.54
68.72 10.35
0.55
Table 11: Hypergraph model performance benchmarks. Test accuracy in % averaged over 15 splits. Training
split: 40%.
AlB Taaakunn
71.16
VGRNNNI
72:00
$2.23
98.55
EDJENN
80.45
12.12
88.70
0.34
54.33
10.07
U1.00 1.20
9836 0019
97.85 11.01
8E.4 19:55
50.75 0.34
2.00
CEGAT
23.06
110
10.19
T1BL
11.1
98.72
02.95 0.20
NA
CRRCON
76.15
16
70.06 4 1:05
66
0.50
76.17 6 1.44
#7.61 0.90
$5.00 4 0.25
78
1.30
09.79 0 0.30
NA
51.00
21.10-4770
7.01
12.72
IKCHA
78.87
11
71.75 à 0:91
46
6.13
03103
0300
91.01
$9.00 0.45
65.53 1.43
04.33 0.28
80.77
0.31
50.58
5.70
7.40
HCNN
23.37
1.04
71.44 + 1.00
50.29
6,45
82.00
= 1.00
01.06 0.24
35.00 : 0.45
50.71 1.37
04.03 0.28
20.77
0.31
50.54 5.29
7.40
BSHN
25.47 0.00
72:50
57.17
0.40
77.27 = 3.11
86.63 0..0
00:90 0.08
$7.10 2:25
07.82
0.17
01.25
0.00
50.40
2.03
6.30
HCpeeGCNN
73.38
0151
61.35
0.67
16.34
2.03
69:10
7.54
1078
75.38
1091
91.30
00.81 0.227
79.02
0.40
8.75
10.15
MLP
70.06
6.40
00.00
0.28
Trarsferuer
1.15
70.70 1.99
6.31
71.95
1.0G
10:52
SU.
Is.15 0.28
80.95
58.06 7.33
34

---

Published in Transactions on Machine Learning Research (02/2025)
Table 12: Hypergraph model performance benchmarks. Test accuracy in % averaged over 15 splits. Training split: 30%.
Cora
Cinweer
Pulmat
CIGIEA-CAA
DREP-CA
NYUSI
Kinde Nee-1
200
20g: ranitieg
1112
BKDE
11:31
WILSS 0.06
SCD
11.25
8038
57.00
1:622
7:25
AllSe Teneeemeee
72.82
01:22
21:46:11
STR2 0:34
R2100 11:66
M1.00 01.18
16-12
07.23 01.21
81.05
00.10
66.1
71.38
6:30
00.97
0.16
2.35
OEGAT
CBCON
1:35
69.17
6.43
T4.83
96.08
18.18
0.40
5.08
ICTHA
77.53
176
71.10
1:11
56.6 6.41
81.81 1.06
01.18
59:00 0.47
58.51 1.51
04.38 0.28
80.75
M006S 5.58
7-00
IGNN
77.81 116
70.86
58.
6:11
81.78
1.15
1..36
98:00
1.26
08.35
0..26
86.75 11:03
MIL.S 5.58
7.00
ENHN
74.55
1.14
71.34 + 1/03
56.34
# 6.30
T0.45 0 1.00
86.33 0.55
00:96 0.07
50.00 0
07.73
0.20
81.00
0.48
55.37
7.95
0.20
HyperCLUN
71.54 + 1.35
63.50 + 1.31
51.57 6.30
T4:04 + 1.55
85:663 2.80
60:44 + 3.02
16. 6 1.00
63.28
5.15
80.61 8
57.46 6.78
14.20
BAN
75:80
1:24
T2.44 3: 1.05
58.80 2: 6.44
R2J80 3: 1..Y
190.5 01.13
91.47 2: 0.29
53.64
2: 1.44
90.29 = 0.20
73.5 = 11.08
78.51 0.111
7.50
BAN sutzibutch
78:54 1.10
72:20 3 0.03
55,70
2: 6.51
TALAL = 1.68
N.A
95.50 2 0,84
70.50 + 1.65
NA
73,46 2: 1.30
46.74 = 15.85
0.88
26.03
1.37
70:10 1:15
16
6:30
90.05 01:00
9P-16 0.15
85.06
58.20 0:37
K6.20
1:21
N6.11 1116.90
5.80
MLD
71:04
1/
70-21
6.83
T1.14 1.61
84.37 10:23
$9.92 10AX
51
1:79
10:10 01:22
83:34
5.83
79-36 6.22
16.10
MLP
06.34
33.87
69.02
6:20
66.14
28.96
50.50
T0.08
0.31
20.33
7.47
11.70
Table 13: Hypergraph model performance benchmarks. Test accuracy in % averaged over 15 splits. Training
split: 20%.
Housed
Uiteest
Puluae
CUIGE-CA
DULFCCA
NEUSI1
20,0
All
72.06
01.40
LA7
R7 E. C330
11.24 $ 1.12
29:3: 11:31
09.52 0.0T
HLVI 2 1.35
PN.0 1 1.24
8I.11 = 0.33
54.12 7.35
7.I6
AlS AlBeTDaaknnes
79:88
1.0
60.50
+ 1.00
S0.6 6,40
90554: 0.94
98.72 0..17
00.9 00.52-0 0.07
92.8 = 1.33
07.48 0.24
80.82 = 0.22
55.01
7.22
2.5G
URIGUNI
24.31
1:28
10S1
1/48
N6-7 6.11
T9-31 1 1.23
90.-7 0.11
59.00 9.05
82.54
..0
M033 (1.15
80.05 11:22
56.21
1:355
6:21
EDDENN
77.00 +: 1.30
70.76
1.29
87.02
0.40
$1.06
1.27
00.01 + 0.30
59.90 0.10
68.45
1.34
07.37
0.17
20.92 = 0.35
55.54
6.335
2.10
ORGINT
71.86
1:42
65.11
1.34
$4.0
6.51
78.18 = 1.30
05.95 0.93
96.10 1.35
76.11
1.20
01.31
0.31
41.59 6.16
13.67
CBGCN
11.25
1.35
67-28
53.47
6.52
12.99 1.44
N9.20 0.21
94.08
15.06
50.00
0.41
0.31
12.44
ICHA
06:04
69:00
1:25
6.37
0.57
0.11
99.08
94.81 0.20
90.80
89:30
5.80
HNNN
1.20
62.08
0.45
T1.42
2.31
53.85 8.74
14.10
BAN
1:18
71.21 1:18
0.44
90:33 0.50
0.15 NA
91.50
0.59 NA
79.33
75.54 6.50
BAN susibutch
1.51
XA
15.00
6.72
18.30
93.22 1.34
152799 1.34
79.30 11:02
07.12
14.36
10.34
MultSSrblinrr
1.14
68.56 2: 1.37
55,01
6.82
18:32 0..6
20,50 0..1
99:42 0.23
12:00
98.01 0.39
87.85 1:53
81:00 7.04
ME. 23
08.07
1:50
65.64 4 1:31
53.06
6.35
08-37 6 1.53
83.11 0.55
9R.54 1:35
17.01
+ 1.69
60.85
0.27
85.00
- 5.70
75.01
1.7.3
16.15
MLP
11.73 1 17:31
06:30
1 1.21
15 6:31
51.75
1 17:31
83.23 (0:1
9R.1 0.13
09.25
1 9.19
61.00 0:54
00.56
01:31
75.54
8.55
13:70
07.34
65.06 +: 1.89
50.81 + 6.40
06.01 = 1.50
41.33 0.27
59.50 0:11
74.17
+ 1.65
00.95 = 0.29
20.5:
= 0.28
33.45 12 0.41
16.70
Table 14: Hypergraph model performance benchmarks. Test accuracy in % averaged over 15 splits. Training
split: 10%.
Houee
Ultcatt
Puluaet
DULF-CA
Z000
05.01
2 114
64.00 2 1.43
TEXI = 1.70
25,32 11:21
00:53
0.08
15.44 1.3)
10.9 0.21
79.0. = (1.0
76.31 7.05
T.I0
AllSe Troeekmmee
71.2
1:38
65.16
1: 1-48
SLT1 6.55
76.16 2 1.36
90.05 01.8
$9.96
0.05
75-78 1:55
16:0 0.21
80.15 11:11
T5.23 11.77
4.711
URRGNNI
09.36
1.455
05.41
1 1.39
8531 630
T0.81 2: 1.155
29.20
11:25
59.96
71.95
2.30
97.58
0.38
20.41
11:25
T9.36 2: 7.97
6.101
KDMENN
78.78
1.64
87.00
1.21
86,00 0.51
77:20 9: 1.31
06.30 4: 0.17
$9.50
0.10
70.00 = 1.70
66.01 0.55
00.47 =
78.06 7.85
2:30
CRRIAT
69.08
1.65
64.15-1
1.99
$1.83 6.43
00:11 = 1.00
65.93
0.53
$6.01
1.31
10.26 2.37
11.17 0.37
39.50 0::00=100 6.19
13.00
CECCN
70.23
1.12
42:58
52 6.44
67.45
85.4
0.20
94.55
58.91
57.25
0.50
5,09
ICHA
65.15
52
6.36
70:07 0.50
90.00
95.08
5104 0.21
90.23
79.75
HNNIN
Hopee
1.00
$7.51
6,46
65.58
1.75
67.56
7.00
79.07
14.10
HAN
1:47
67.56
10 6,45
77.45 1.29
0.17 NA
91.67 0.855
0.50 NA
76.77
T0.M4 14.54
BAN ausilunch
0.30
08.25
3.15
SLUG 0.65
16.27 1.54
SA
55.30 2.85
NA
45.02 9.44
8.17
MaltSSibiinr
09.68
1.30
05.71 2: 1.45
52.01 6.66
14385
1.00
20.12 0.2
99.08 2: 0.33
14:92 2.39
90.82 0.20
88.92 3: 1.05
73.55 7.99
0110
MEPP 231
07.17
1:37
61.55 4 1:33
$1.40 0.55
07.83 6 1.50
87.00
1136
9R.61 0.15
86 1 1.51
60.16 0.26
85.7 - 3.81
67.40 9.14
10.70
MLP
38.61
12.37
61:21
1 1:53
S336 6-19
37.45
1 11:79
81.45 (0:11
99.72 1 0.15
15:20 3.31
K171 0:36
79,63 0.:02
72.17 19.21
13.80
Tonofoener
G1.45
1.05
68.76 1.89
58.80 4: 6,50
00065 = 1.57
22.40=00.0 0:47
06:74:2:0:00
66.14 4: 1.63
04.60 =: 0.40
29.6.
= 0.20
65.50 8.03
13.15
F Sampling analysis
As it has been discussed in Section 4.3 the proposed mini- batchng procedure consists of two steps. At step 1, samples B hyperedges from E. The hyperedge sampling over E can be either uniform or weighted (e.g. by taking into account hyperedge cardinalitics). Then in step L nodes are in turn sampled from cach sampled hyperedge e, padding the hyperedge with L - |el special padding tokens if lel < L consisting of O computatiie
vectors that can be easily discarded in some computations. Overall, the shape of the obtained mini-batch X has fixed size B x L.
Step 0 (hyperedge mini-batching) is particularly beneficial for large datasets; however, it can be skipped when the network fits fully into memory. Empirically, we found step 1 (node mini- batching within a hyperedge) to be useful for two reasons: (i) pooling operations over a large set may over-squash the signal, and (ii) node batching leads to the training distribution shift, hence it can be useful to keep even when the full hyperedge can be stored in memory.
35

---

Published in Transactions on Machine Learning Research (02/2025)
When both step and step 2 are employed, considering the hidden dimension size, the batch size required to be stored in memory during the forward pass is B x T. x d, where d represents the hidden dimension. If only step 2 is employed, the batch size is |8] x L x d, where |E] is the number of hyperedges within the hypernetwork. Finally, when no mini-batching steps are used, the batch size |8| x max.eg |e] x d, where maxeeg le| the size of the longest hyperedge.
Theoretical analysis. In this Section, provide an analysis regarding the uniform sampling of the hyperedges in Step 1. We propose sampling X mini-batches of certain size B at each iteration. At step 1, we sample B hyperedges from E; in step 2, for each hyperedge we sample a fixed mumber nodes, that are randomly chosen among the ones belonging to that specific hyperedge. If the hyperedge does not contain enough samples, we use padding so that the size of the set of sampled nodes increased the desired value. By choosing to sample the nodes uniformly at random from the hyperedge, there is no guarantee that will eventually sample all the nodes of each hyperedge. Indeed, sampling uniformly random iteems from set of size the probability of not sampling our desired one is 1- s. The probability of having to wait for T independent trials before finding node among the sampled nodes is described by the geometric distribution
Namely, let x E e and le| = n, and assume the size of the considered mini-batch is c:
C T-1 n
P ( Sample node from hyperedge e for the first time at epoch T) (1
(60)
It follows that
E # of epochs to wait before sampling node z]
Assume now that a node æ belongs to k hypergraphs e ck of respective sizes n....., R2. The events (Node x is sampled from hyperodge e, } and (Node x sampled from hyperedge e; are independent if * j. It follows that the random variable { # of epochs to wait until we sample node z from all the hyperedges e ) is the maximum of k independent, non-identically distributed geometric distributions. Denote by Ti the random variable that corresponds to the number of epochs have to wait before sampling sample from edge e, The exact distribution for the random variable T, that is, the number of epochs we have to wait until we sample node x from all hyperedges e1,...ep eg at least once, is
b I S S II pi)*
(T S =P
max T, S h i=l,,.,
It follows that
E ( # of epochs to wait until we sample node from all the hyperedges eI,...,
(61)
= 1
1-
E
max il.....k
Tj
1-
(62)
i=0
==1
This can't be expressed in closed form: can use the Moment Generating Function to bound the expected value of the maximum. Alternatively, can also try to use the inequality due to Aven (1985), that
E
max Tj k....k
(63)
k
< max k...k
E
(Ti)
v
(T.)
k
1
)
(64)
k
C
1=1
12 ))
+
k
(65)
36

---

Published in Transactions on Machine Learning Research (02/2025)
Probability that specific node is not sampled in epoch. Let be node and let d, be its degree. In one epoch, we "see" all hyperedges but, of course, not necessarily all their nodes. It holds that
P( node is sampled in epoch T' P node is not sampled in epoch T
(66)
We can write the event
{ node is not sampled in epoch T' } a.t.ve node u is not sampled in e}.
It follows that
du
P node sampled in epoch T) = max 1
eij min
laise
(67)
Indeed, if any on the edges e belongs to has a size smaller than the batch size for nodes (c), the node is for sure seen in the first epoch.
G Node Homophily
In this Section, we report the node homophily plots for the datasets not illustrated Figure ! For each dataset, we choose illustrate 3 different levels of node homophily, respectively 0, and 10- level homophily, using Equation 2 at 1 and 10 (left, middle, and right plots respectively). Horizontal lines depict class mcan homophily, with numbers above indicating the number of visualized points per class.
G.1 Figure node homophily
1.0
0.9
135
125
155
0.7
284
0.6
0.5
0.4
125
0.3
0.2
0.0
Class
Class
Figure 8: Node Homophily Distribution Scores for Cora.
1.0
Homophal
0.9
0.8
316
205
245
316
245
385
516
185
0.6
164
245
104
0.5
104
0.3
0.2
0.1
Class
Class
Class
Class
Class
Class 5
Figure 9: Node Homophily Distribution Scores for Citeseer.
37

---

Published in Transactions on Machine Learning Research (02/2025)
1.0
0.8
282
107
720
262
117
Homopoe
199
319
362
167
0.6
199
0.5
0.4
199
0.2
0.0
Class
Class
Class
Class
Class
Class
Class
Figure 10: Node Homophily Distribution Scores for CORA-CA.
1.0
6700
1105
211
7778
1818
6600
0105
12114
1378
3916
Homophoe
0.7
1105
11211
0.6
0.4
0.3
0.1
0.0
Class
Class
Class
Class
Class
Class 5
Figure 11: Node Homophily Distribution Scores for DBLP-CA.
1.0
0.9
Homophan
0.7
0.6
0.4
0.3
0.2
0.0
Figure 12: Node Homophily Distribution Scores for NTU2012.
38

---

Published in Transactions on Machine Learning Research (02/2025)
1.0
0.9
Homoppo
0.7
0.6
04
0.3
0.2
0.0
Figure 13: Node Homophily Distribution Scores for ModelNet40.
1.0
0.9
3016
1200
Homophiyg
0.6
0.5
0.3
0.2
0.0
Figure 14: Node Homophily Distribution Scores for Mushroom.
1.0
0.9
0.8
Homopplly
0.6
4685
0.5
3319
30133
3535
0.3
2626
2836
4605
0.2
2676
0.1
Figure 15: Node Homophily Distribution Scores for 20News W100.
39

---

Published in Transactions on Machine Learning Research (02/2025)
1.0
0.9
Homophal
0.7
0.6
04
20
0.3
0.2
0.0
Class
Class
Class
Class
Class
Cless
Class
Figure 16: Node Homophily Distribution Scores for ZOO.
H
Comparisons with others Homophily measure in literature
K-uniform homophily measure
Hyperedge homophily. Veldt al. (2023) defines the group homophily measure for k-uniform hyper- graphs G2 (V,E). The type t-affinity score for each t E fl....,ky .),, indicates the likelihood of node belonging to class participating in groups in which exactly t group members also belong to class and defined as in equation 68 (v) is the number of hyperedges that e belongs to with exactly members from class c. The hors also consider a standard baseline score (c), equation 69 that measures the probability that a class-c node is in the group where t members are from class c, given that the other k nodes were chosen uniformly at random.
ne(c) = Evy.=c de(u)
(fle=1 (o r-1 k-t
(68)
be(c)
(69)
Evmi=o do
Te is the number of nodes in class e and n the total number of nodes the hypergraph. The k-uniform hypergraph homophily measure can be expressed as a ratio of affinity and baseline scores, with ratio value of indicating that the group is formed uniformly at random, while any other number indicates that group interactions either overexpressed underexpressed for class c.
They suggest three possible ways for extending the concept of homophily to the hypergraph context. The first one is the simple homophily and it means that (e(c) > (c() for t = k and check whether a class has a higher-than-baseee
affinity for group interactions that only involve members their class. The second one is order-j majority homophily that is obtained when the top j affinity scores for one class are higher than the baseline, i.e. [k-j+1(c) bk-s+1(e),.... 7 (c) bx(c). The last one they consider is order-j monolonic homophily, which corresponds to the case when top ratio scores are increasing monotonic, i.e. mc(c)/bu(e) TR-1(c)/bu-il
nk-s+1(c) > bx-)+1(c).
Finally, considering that the value (c))) bz(c) is the bias of class for type t, they introduce type-t normalized bias score that normalizes the maxinum possible bias, hence the obtained metric is bounded in (o, 1] and it is computed as:
bu(c) I-b.le)
if mm)( 2 b((c) if mf < bi(c)
f((c)
(70)
brlc)-be(0 b((()
Comparisons to our mcasure Unlike Veldt et al. 2023 our mcasure of homophily does not assumo a k-uniform hypergraph structure and can be defined for any hypergraph. Furthermore, the proposed measure enables the definition a score for each node and hyperedge for any neighborhood resolution, i.e., the connectivity of the hypergraph can be explicitly investigated. It gives definition of homophily that puts more emphasis on the connections following the two-step message passing mechanism starting from the hyperedges of the hypergraph.
40

---

Published in Transactions on Machine Learning Research (02/2025)
H.1 Figure k-uniform homophily
Some of the hypergraphs have lot of different size for hyperedges, here, report the plots only for some k for each dataset for brevity, All additional plots can be found in the supplementary material's zip files.
2-uniform hypergraph, number of hyperedges: 623
1.00
1.0
AffinitylBaseline BBaeelin
Normal Issed bias
0.75
Affinity/
0.5
Affiiiy
0.50
0.0
10
0.25
-0.5
0.00
-1.0
Class
Closs
Class 2
Cless 3
Closs
Cless
Closs
Figure 17: k-uniform homophily Cora.
3-uniform hypergraph, number of hyperedges: 464
1.00
1.0
Affinty/Baslline
Normolised bias
0.75
0.5
Affinty
0.50
0.0
0.25
-0.5
0.00
-1.0
Class
Class
Class 2
Class 3
Class
Class
Class
Figure 18: k-uniform homophily Cora.
4 4-uniform hypergraph, number of hyperedges: 312
1.00
1.0
BBaslii/
Normalised lised bios
0.75
0.5
Affnity
0.50
0.0
0.25
005
10a
0.00
-1.0
Class
Class
Class 2
Class 3
-
Class
Class
-
Class
Figure 19: k-uniform homophily Cora.
41

---

Published in Transactions on Machine Learning Research (02/2025)
2 2-uniform hypergraph, number of hyperedges; 486
1.00
1.0
Affinity/Baseline
Normalised leel bias
0.75
0.5
0.50
0.0
10
0.25
-0.5
-1.0
2
Closs
Cless
Closs
Class
Closs
Figure 20: k-uniform homophily CORA-CA.
3-uniform hypergraph, number of hyperedges: 205
1.00
1.0
Affiiiy/Basoliin
Normalised bias
0.75
0.5
Affintty
0.50
0.0
0.25
10"
0.00
-1.0
Class
Class
Class
Class
Class
Class
Figure 21: k-uniform homophily CORA-CA.
4-uniform hypergraph, number of hyperedges: 106
1.00
1.0
Affinit/BBaselnn
Normelised bias
0.75
0.5
Affinity
0.50
0.0
0.25
-0.5
10a
0.00
-1.0
2
Class
Cless
Class 2
Cless
Class
Cless
Class
Figure 22: k-uniform homophily CORA-CA.
42

---

Published in Transactions on Machine Learning Research (02/2025)
2-uniform hypergraph, number of hyperedges: 9976
1.00
1.0
Lffinity//asellnn
Normalised bias
0.75
0.5
0.50
0.0
10a
0.25
-0.5
-1.0
Cless
Class
Class
Closs
Figure 23: k-uniform homophily DBLP-CA.
3-uniform hypergraph, number of hyperedges: 4339
1.00
1.0
Afinity/aaselino
Normalised bias
0.75
0.5
Affintty
0.50
0.0
0.25
109
0.00
-1.0
Class
Class
Class
Class
Class
Class
Figure 24: k-uniform homophily DBLP-CA.
4-uniform hypergraph, number of hyperedges: 2312
1.00
1.0
fainlly/Baslline
Normelised bias
0.75
0.5
Affinity
0.50
0.0
0.25
-0.5
10a
0.00
-1.0
2
Cless
Class
Cless
Closs
Cless
Class
Figure 25: k-uniform homophily DBLP-CA-
43

---

Published in Transactions on Machine Learning Research (02/2025)
5-uniform hypergraph, number of hyperedges: 12311
1.00
1.0
Normalised bias
0.75
Affinity/Baselinnn
0.5
0.50
0.0
00.5
100
3
2
Figure 26: k-uniform homophily ModelNet40.
5-uniform hypergraph, number of hyperedges: 2012
1.00
1.0
Affnity/Baseline
Normalised bias
0.75
0.5
Affintty
0.50
0.0
0.25
-0.5
0.00
100
-1.0
2
Figure 27: k-uniform homophily NTU2012.
2 2-uniform hypergraph, number of hyperedges: 3522
1.0
lised blos
0.75
Affinity/Baselina
0.5
Aftnity
0.50
10a
0.0
0.25
0.5
0.00
-1.0
Class
Cleas
Classs 2
Figure 28: k-uniform homophily Pubmed.
44

---

Published in Transactions on Machine Learning Research (02/2025)
3-uniform hypergraph, number of hyperedges: 1626
1.00
1.0
Affinity/Baselino
Normalised bias
0.75
0.5
Affiity
0.50
0.0
109
0.25
00.5
-1.0
Class
Clisss
Figure 29: k-uniform homophily Pubmed.
4 4-unfform hypergraph, number of hyperedges: 845
1.00
1.0
AffinityBBasllne
Normalised bias
0.75
0.5
Affintty
0.50
0.0
0.25
10"
0.00
-1.0
3
2
Class
Class
Figure 30: k-uniform homophily Pubmod.
41-uniform hypergraph, number of hyperedges: 2
1.00
1.0
AffinityBaallln
Normelised bias
0.75
0.5
Affinity
0.50
0.0
0.25
-0.5
0.00
108
-1.0
41
Class
Cless
Class 2
Cless 3
Class
Cless
Class
Figure 31: k-uniform homophily ZOO.
45

---

Published in Transactions on Machine Learning Research (02/2025)
8-uniform hypergraph, number of hyperedges:
1.00
1.0
/ffinity/Baselinn Bineline
Normalised bias
0.75
0.5
Affiity
0.50
0.0
0.25
-0.5
10
-1.0
3
8
2
5
7
Class
Closs
Class
Cless
Class
Class
Classs
Figure 32: k-uniform homophily ZOO.
10-uniform hypergraph, number hyperedges:
1.00
1.0
Aftniit/Baseline
Normalised bias
0.75
0.5
Affnity
0.50
0.0
0.25
00.5
0.00
10
-1.0
2
4
5
6
9
10
2
3
4
6
7
8
9
10
3
4
5
7
6
9
10
Figure 33: k-uniform homophily ZOO.
Social equivalence
The social cquivalence Sun al. (2023) is calculated through an expectation taken over pairs of users sampled from probability distributions. Specifically, E(u, P(Ep)) represents the expectation over positive user pairs sampled from the distribution of the hyperedges of the hypergraph P(Ep). In contrast, E(up, vo P(V VV Ep)) represents the expectation over negative user pairs sampled from P(V x V Ep). The measure is fraction between the numerator that involves the expected Jaccard index of environments for positive user pairs and the denominator, which comprises a similar calculation for negative user pairs. The Jaccard index is used to mcasure the similarity between two sets, and this context, it assesses the similarity of hyperedges associated with positive user pairs. If the expected Jaccard index for positive user pairs is higher than that for negative pairs, the measure exceeds 1, indicating a significant level of observed social equivalence among users.
Comparisons to our measure The concept of Social Equivalence, as introduced by Sun et. al. (0023) differs significantly from our definition of homophily. In our approach, the measure is initially defined at both the node and hyperedge levels. Our primary objective is to address the question: 'How similar a node is to its neighbors?" Following a message passing scheme, our definition allows us to examine different time points, attempting to answer how similar a node to the nodes can reach with steps of message passing. This consideration also extends edges, where we seek to understand the coherence or uniformity of an edge within itself. The guiding notion of similarity in our work is belonging the same class. Specifically, in level-O homophily, evaluate whether node is more connected with nodes of the same classes. We then aggregate the node-level heteee//hhperedge-eeeee homophily to provide definition for hypergraph homophily, In contrast, in the work of Sun el. al. (2023), the concept of social equivalence yields single result. for the entire hypergraph. The approach involves comparing the set of similar nodes that are connected with those that are not connected. The key question is whether the set of non-connected nodes is, on average, more similar or if the set of connected pairs exhibits greater similarity. It's important to note that this definition makes sense only for the entire hypergraph and captures different. notion of similarity.
46

---

Published in Transactions on Machine Learning Research (02/2025)
Social conformity
Social conformity, described in Sun el al. (2023) involves leveraging learned representations of users and hyperedges within model to understand and quantify the level of conformity among users in a social network.
Node homophily computed the clique expansion of the hypergraph
Clique-cxpanded (CE) homophily, employed in Wang et al. 2023 is determined by calculating node ho- mophily Pei el al. (020 on the graph derived from the clique-expanded hypergraphs.
Comparisons to our measure The node homophily for a graph computes the fraction of neighbors with the same class for all nodes and then averages these values across the nodes. In contrast our metric, CE homophily is not defined directly on the hypergraph but necessitates an expanded clique representation. While sharing some similaritics with node-wise measure ho; it lacks the dynamic aspect inherent in the MP homophily measure, consequently failing to capture the dynamic information within connections. Our analysis in Section 3 underscores the significance of this dynamic element in understanding the correlation between homophily measures and the observed patterns in Hypergraph Neural Networks (HNNs).
Further consideration on the concepts of simulated social environment evolving and group entropy from Sun et al. (2023)
Further exploration of the concepts of simulated social environment evolving and group entropy is presented by Sun et al. {Sun et al. 2023). In their study, a dynamic analysis of specific hypergraph characteristics is conducted through message passing. They specifically focus on the evolving proportion of "significant nodes' within the hyperedge relative to the original nodes across different epochs. These 'significant nodes' are identified those with a probability of belonging to the hyperedge greater than 0.5, initially determined by multiplying the representation of the node with that of the hyperedge (averaged over its constituent nodes).
A noteworthy distinction from our methodology lies in their reliance on representations provided by a model, in contrast to our representation-indeeende approach. Despile the shared use of message passing in both approaches, we underscore these methodological differences.
It's important highlight that the concept of group entropy introduced by Sun et al. is also noteworthy, representing evolving model concept; however, its computation requires node representations provided by model.
We posit that our measure and the metrics employed in Sun et al. (022) 's paper can complement each other effectively.
Class Distribution Shift
We now report the results for the class distribution shift obtained by applying the mini-batch sampling procedure described in Section B For each dataset, we choose to illustrate 3 different distributions: the one corresponding to the original labels ("Node"), the one obtained by applying both Step 1 and Step described the mini-batch paragraph of Section 4.2 and the one obtained by only applying Step 2.
1.1 Figure - node homophily
47

---

Published in Transactions on Machine Learning Research (02/2025)
Cora
CORA-CA
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0.0
0.0
Class 0 Class
Class Class 3 Class
Class Class 6
Class Class Class 2 Class 3 Class Class 5 Class 6
Node
Node
Class distribution for Cora.
(b) Class distribution for CORA-CA.
20newsgroups
DBLP-CA
0.4
0.6
0.5
0.3
0.4
0.3
0.2
0.2
0.1
0.1
0.0
0.0
Class 0
Class
Class Class 3
Class
Class
Class 0
Class 1
Class 2
Class 3
277
Node
and
Stop
(c) Class distribution for 20Newsgroup.
(d) Class distribution for DBLP-CA.
Mushroom
20newsgroups
0.6
0.4
0.5
0.3
0.4
0.3
0.2
0.2
0.1
0.1
0.0
Class
Class
0.0
Node
Step and 2
Step
Class 0
Class
Class
Class 3
Class distribution for Mushroom.
Class distribution for 20News
Figure 34: Class distribution shifts.
48


Reviews:
[{"review_id": "XMMeaQuN0O", "reviewer": "Reviewer_iCjf", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The proposed measure of $\\Delta$ Homophily seems to characterize datasets effectively and is also quite intuitive. \n2. The experiments are thorough and insightful.\n3. The discussion clearly links the numerous experimental results to specific research questions.\n\nWeaknesses:\n1. I think there is a discrepancy in Equation 8. It states that $\\mathbb{X}_u^{(t)}$ is used to update $\\mathbf{z}_e$. Based on Equation 11 it seems to me like this should say $\\mathbf{x}^{(t)} _{u,e}$. Figure 2b on the other hand suggests that $\\mathbf{x}_v^{(t)}$ is used to update $\\mathbf{z}_e$. Overall, this seems somewhat confusing. Is this a notation error or did I misunderstand the MultiSet framework?\n2. In Section 4.4 the second to last paragraph refers to \"step  0\" and \"step 1\" while other paragraphs use \"step 1-2\".\n3. The manuscript also has some typos that should be fixed (\"a a hyperedge\"  (Page 4), \"We represents by\" (Page 4))", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a novel definition of homophily for general hypergraphs ($\\Delta Homophily$) and demonstrates that this measure correlates to HNN model performances. It further proposes MultiSet as a unified HNN framework that encompasses most prior architectures for hypergraphs. The paper also proposes specific mini-batch sampling strategies that improve the computational efficiency of MultiSet models. Finally, an experimental evaluation is conducted to compare the performance of HNNs in the context of homophily and investigate the efficacy of existing hypergraph benchmark datasets to assess model performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Clarify the definition of the MultiSet Framework (see Weakness 1, critical)\n2. Improve the presentation of the work (see Weakness 2+3)", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Fgulseez0V", "reviewer": "Reviewer_nGeB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths:\n* Overall: The paper clearly identifies limitations with existing methods for learning on hypergraphs and outlines a proposal to address these limitations.\n* Claims: the proposed methods, model architecture, and experiments are reasonable and well justified (see Weaknesses for concerns about the claims _based_ on the experiments).\n* Proofs: formal results appear to be correct, with the connection between MultiSetMixer and existing models being well defined. The proof that MultiSetMixer is a generalization of existing methods is correct and identifies weaknesses of existing approaches.\n* Experiments: the authors conduct experiments on 11 datasets and 16 modelling approaches, plus several perturbation methods to evaluate the predictive performance, memory cost, and susceptibility to connectivity changes of different HNN models.\n* Readability and form: the paper is clear, well written and is structured in a coherent, readable manner.\n* Broader interest: the authors highlight the need to identify when and where hypergraph-specific aspects are required for modelling, and provide theoretical and practical examples where ‘lifted’ graph neural network architectures applied to hyper graphs are limited in their modelling power.\n\n### Weaknesses:\n* In its current state, the paper's claims about the advantages of hypergraph-specific architectures are not fully supported by the empirical results. The proposed method outperforms baselines only on datasets where the hyper graphs are constructed from non-relational data (NTU2012, ModelNet40, and 20Newsgroups), where two computer vision and one text classification datasets are cast as hyper graph problems. In contrast, in relational datasets that naturally contain nodes and hyper-edges like citation and co-authorship networks, the proposed method does not outperform the baselines. The authors claim that “features are being more representative than connectivity in most considered hypergraph datasets”, but this can be turned on its head to ask whether the approach to measure connectivity is meaningful given performance in these datasets.\n* Certain parameters seem to have been picked without an explanation (e.g. \\mu, see Additional Question 1.) but control key aspects of how the homophily measure functions and the results that follow from it, i.e. Figure 4.\n* The sensitivity of the model to various hyperparameters (beyond \\mu) is not thoroughly explored, particularly the impact of the mixer-block design choices and aggregation functions on model performance and memory usage (see Additional Question 2.).\n* The evaluation of scalability for MultiSetMixer is tangled with the proposed batching approach. These two contributions should be studied separately through:\n    * In Section 5.3, the authors compare MultiSetMixer with their proposed batching approach versus AllSetTransformer with and without hyper edge batching. \n    * It would be valuable to understand what is the performance of MultiSetMixer with ‘vanilla’ batching to further motivate the need for hypergraph-aware batching and training procedures, and to isolate the memory impact of each component versus the baseline.\n* While the technical foundations are sound, the paper does not conclusively demonstrate that this more complex approach provides meaningful advantages over simpler methods for practical hypergraph learning problems, especially given concerns regarding dataset construction (see Additional Questions 3 and 4).\n\n### Additional questions:\n1. How is the value of \\mu chosen? The authors rely on this threshold to determine whether their proposed homophily value changes between propagation timestamp. However:\n    * It is not clear how sensitive the reported results are to different values of \\mu (e.g. what about \\mu \\in {0.01, 0.05, 0.15, 0.2}?).\n    * The reported results in Figure 4. show the rank shift between CE Homophily and \\Delta-homophily. The relationship between those metrics is hard to understand, especially as there is no table or figure summarising the values of \\Delta-homophily (including Table 7. in the appendix, which reports _average_ homophily values at t=0 and t=1 for each graph).\n\n2. How sensitive is MultiSetMixer to the choice of aggregation function? It is a well known result in the GNN literature that certain aggregation functions are more expressive than others for e.g. isomorphism detection. Exploring this question is important, as the mean is particularly known to be less expressive than the sum (see Xu et al. (2019) [1]).\n\n3. How is the 20 Newsgroups dataset modelled as a hyper graph dataset? The study highlights MultiSetMixer’s comparably superior performance against the baselines, but the task being modelled and dataset being used are both not clear.\n    * To my knowledge, [20 Newsgroups](https://archive.ics.uci.edu/dataset/113/twenty+newsgroups) is a standard 20-label multi-class text dataset where documents are represented by their contents, or, frequently, by TF-IDF representations of the terms in the collection. The task is to classify each document on one out of 20 disjoint classes.\n    * Table 7 indicates that there are only 4 classes, and authors refer to Yadati et al. (2019)’s [2] approach to generate hyper graphs by defining a hyper-edge for all data points sharing a categorical feature. \n    * Furthermore, in Section 5., authors report reusing the dataset and benchmarking approach provided by Chien et al. (2022) [3], which also refers to the UCI Categorical ML Repository without indicating the procedure, processing, or task adjustments made on the dataset. Chien et al.’s work also indicates that the cardinality of hyper node features is 100, which seems unexpectedly low for TF-IDF document representations (and implies a vocabulary of 100 terms).\n    * Can the authors clarify what the dataset, features, classes and task are in this case? Otherwise, it is not possible to understand whether their results are applicable in a practical dataset, or instead a by-product of how the dataset was constructed. \n    * Understanding the precise dataset construction is crucial because it would help clarify whether the performance gains come from the model architecture itself or from how the hypergraph structure was imposed on what is fundamentally a text classification task.\n\n4. The direction of the work presented is valuable, and my concerns about performance per dataset and the provenance of data, indicate that existing benchmarks are not appropriate to assess the presented work. Thus, what properties should we look for in benchmark datasets? Have the authors identified any connectivity patterns that their method would perform well on, and that could be mapped to practical applications and datasets? In other words, it would be valuable to see if authors can identify or construct a dataset for which their method can perform well, but existing baselines cannot.\n\n### Minor comments:\n* In table 5, the avg. rank for CEGAT and CEGCN seem to be populated with data from a different column — across the whole column, avg. ranks are rounded in 0.05 intervals and have the best performing value per row on the first original, unperturbed value.\n\n### References:\n[1] Keyulu Xu, Weihua Hu, Jure Leskovec, & Stefanie Jegelka (2019). How Powerful are Graph Neural Networks?. In International Conference on Learning Representations.\n\n[2] Yadati, N., Nimishakavi, M., Yadav, P., Nitin, V., Louis, A., & Talukdar, P. (2019). HyperGCN: A New Method For Training Graph Convolutional Networks on Hypergraphs. Advances in Neural Information Processing Systems (NeurIPS).\n\n[3] Eli Chien, Chao Pan, Jianhao Peng, & Olgica Milenkovic (2022). You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks. In International Conference on Learning Representations.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes three main contributions (1) a measure of homophily in hyper graphs that can be used to evaluate the degree to which hyper node connectivity is given by hyper node labels, (2) MultiSetMixer — a hyper graph neural network architecture that explicitly considers hyper graph connectivity by representing nodes according to their learnable representations in all the hyper edges that contain them, and (3) a batching approach to reduce memory consumption when training HNNs. \n\nRelying on the three components, the authors highlight that existing approaches to hypergraph representation learning do not fully consider hypergraph attributes. Based on their experimental results, the authors suggest that the proposed message-passing homophily measure can help identify properties of hyper graphs where existing models are limited and not expected to perform well, which can motivate future work.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weaknesses and additional questions. In order of importance, the main adjustments I would suggest for the submission are:\n* Explain how the datasets where MultiSetMixer perform favourably are constructed (particularly 20Newsgroups) and either adjust the claims or provide additional empirical justification for the method being broadly applicable compared to 'lifted' approaches. This could come in the form of results in additional datasets (incl. a synthetic dataset that baseline models can't beat, see Additional Question 4), extended analyses to explain the disparity in MultiSetMixer performance across dataset types, or an understanding on why existing baselines perform well where MultiSetMixer performs stat. sig. worse than baselines.\n* Explain how parameters were chosen and provide, if possible, additional information and results about the impact of changing key parameters such as \\mu or the choice of aggregation function.\n* Evaluate the impact of training MultiSetMixer without the proposed batching approach. Practically, this would introduce a new row in Table 4 presenting the memory consumption of MultiSetMixer as a drop-in replacement of AllSetTransformer. This would help understand the impact of the proposed batching on its own, and would let practitioners assess whether the batching approach alone can be helpful in other hypergraph representation learning works.", "broader_impact_concerns": "N/A, no ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "kXlYmJLVsn", "reviewer": "Reviewer_CN4Y", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n* The empirical study is extensive, considering both a large variety of hypergraph neural networks, simple baselines such as learning only on the individual features, as well as a large variety of datasets.\n* In addition, the empirical analysis contains many ablations aimed at answering the three main questions posed in the abstract.\n* The writing is clear and the entire paper is written along the narrative of answering the three main questions, which allows readers to quickly understand the importance of different sections. This is particularly the case in the experimental section.\n\n**Weaknesses:**\n\n* Compared to the detailed descriptions of the other experiments, the rewiring experiments receive little attention. Here, I would suggest to expand the description of the two rewiring approaches, perhaps even formally defining them. While I can roughly follow what the authors propose here, I am not too sure about the details after reading 5.4.2. \n* Table 5 is a bit difficult to parse. In particular, since the different drop connectivity variants are evaluated at multiple levels, line plots would additionally improve the presentation of these results (with the x-axis showing the level of drop connectivity and the y-axis showing the performance). While of course performing such line plots for all methods and datasets would take up a lot of space, I would still recommend to show plots for the most important baselines and datasets (or a selection thereof). This, together with the raw numerical data in Table 5, would likely improve the presentation of these results.\n* I had a hard time understanding Figure 1, in particular, because the Figure lacks an x-axis. I assume the authors sort every node in a dataset according to class and then plot the distribution of homophily scores for each? I think this should be described more clearly, for example in the figure caption.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this work, the authors explore and advance the broader field of hypergraph neural networks. In particular, the authors seek to answer three questions, regarding (a) the role that homophily (which is commonly considered in graph learning) can play in hypergraph tasks, (b) how previous attempts at hypergraph neural networks can be unified into a new architectural design that is more specific to the hypergraph setting, and (c) the quality of existing hypergraph benchmark datasets in terms of how well they are suited for measuring progress on hypergraph neural networks. The authors provide extensive empirical analysis to answer these questions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "My weaknesses above largely reflect improvements that I would appreciate (while none are critical to my recommendation). \n\nIn addition, I want to point the authors to some minor points I noticed while reading the paper:\n\n* At the beginning of 4.1, the authors write \"We say that a function $f$ is a multiset function if it is permutation invariant w.r.t. each of its arguments in turn.\" From what I can see, this definition does not characterize a multiset function (which to me is a function which receives a multiset as input). Perhaps I misunderstood what the authors mean here, but this statement could likely be made more precise.\n* 4.4, in the third paragraph, you write \"Step 0\". Do you mean Step 1? I was not able to find where a Step 0 was described.\n* Last sentence of 5.3: the word \"memory\" appears twice.", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers initially raised several concerns about the original submission, including unclear presentation of certain technical aspects and experiments, insufficient experimental support for the claims, a lack of analysis or justification for hyperparameters, and the absence of important ablation studies in model learning. In their response, the authors addressed these concerns effectively by improving the overall clarity, providing additional experiments and thorough analysis, and including details on dataset construction. 

Following the discussion phase, the paper received a consensus of positive recommendations from all three reviewers (2 Accept and 1 Leaning Accept), reflecting its key strengths:
1. The work is well-motivated, identifying and addressing limitations in existing methods for hypergraph learning.
2. The proposed methods, model architecture, theoretical results, and experiments are well-justified and reasonable.
3. The experimental evaluations are comprehensive and convincing, and the analysis provides valuable insights.
4. The paper is clearly and effectively written.

The AE agrees with the reviewers' recommendation to accept the paper, noting that the authors have effectively addressed the initial concerns. The work presents valuable contributions to the ML community, supported by clear evidence that substantiates its claims, thereby meeting the standards of TMLR.
Decision: Accept (poster)