Training-Free Message Passing for Learning on Hypergraphs
Authors: Bohan Tang
Zexi Liu
Keyue Jiang
Siheng Chen
Xiaowen Dong
Venue: ICLR 2025 Poster (2025)
Keywords: Hypergraphs
Hypergraph Neural Networks
Graph Neural Networks
Primary area: learning on graphs and other geometries & topologies
TL;DR: We propose a novel model called TF-HNN, which addresses the challenge of efficient training of hypergraph neural networks while retaining effectiveness.

Abstract: Hypergraphs are crucial for modelling higher-order interactions in real-world data. Hypergraph neural networks (HNNs) effectively utilise these structures by message passing to generate informative node features for various downstream tasks like node classification. However, the message passing module in existing HNNs typically requires a computationally intensive training process, which limits their practical use. To tackle this challenge, we propose an alternative approach by decoupling the usage of hypergraph structural information from the model learning stage. This leads to a novel training-free message passing module, named TF-MP-Module, which can be precomputed in the data preprocessing stage, thereby reducing the computational burden. We refer to the hypergraph neural network equipped with our TF-MP-Module as TF-HNN. We theoretically support the efficiency and effectiveness of TF-HNN by showing that: 1) It is more training-efficient compared to existing HNNs; 2) It utilises as much information as existing HNNs for node feature generation; and 3) It is robust against the oversmoothing issue while using long-range interactions. Experiments based on seven real-world hypergraph benchmarks in node classification and hyperlink prediction show that, compared to state-of-the-art HNNs, TF-HNN exhibits both competitive performance and superior training efficiency. Specifically, on the large-scale benchmark, Trivago, TF-HNN outperforms the node classification accuracy of the best baseline by 10% with just 1% of the training time of that baseline.

Published as a conference paper at ICLR 2025
TRAINING-FREE MESSAGE PASSING FOR LEARNING ON HYPERGRAPHS
Bohan Tang'
Zexi Liu
Keyue Jiang
Siheng Chen Xiaowen Dong'
Universiity of Oxford
Shanghai Jiao Tong University *University College London
*Shanghai AI Laboratory
Oahan.tan@@eng. ax.ac.uk
ABSTRACT
Hypergraphs are crucial for modelling higher-order interactions in real-world data. Hypergraph neural networks (HNNs) effectively utilise these structures by mes- sage passing to generate informative node features for various downstream tasks like node classification. However, the message passing module in existing HNNs typically requires a computationally intensive training process, which limits their practical use. To tackle this challenge, propose alternative approach by decou- pling the usage of hypergraph structural information from the model training stage. This leads novel training-free message passing module, named TF-MP-Modul which can be precomputed in the data preprocessing stage, thereby reducing the computational burden. We refer the hypergraph neural network equipped with our TF-MP-Module as TF-HNN, We theoretically support the efficiency and ef- fectiveness of TF-HNN by showing that: 1) It more training-efficient compared to existing HNNs; 2) It utilises as much information as existing HNNs for node feature generation; and 3) It is robust against the oversmoothing issue while us- ing long-range interactions. Experiments based on seven real-world hypergraph benchmarks in node classification and hyperlink prediction show that, compared state-of-the-ard HNNs, TF-HNN exhibits both competitive performance and superior training efficiency. Specifically, on the large-scale benchmark, Trivago, TF-HNN outperforms the node classification accuracy of the best baseline by 10% with just 1% of the training time of that baseline.
INTRODUCTION
Higher-order interactions involving more than two entities exist in various domains, such as co- authorships in social science (Han et al. 2009). Hypergraphs extend graphs by allowing hyperedges to connect more than two nodes, making them suited to capture these complex relationships (Bick) al., 2023; [hun] 202T]: Tang et al., 2023b a). To utilise such structures for downstream tasks, learning algorithms on hypergraphs have garnered increasing attention.
Inspired by the success of graph neural networks (GNNs) (Wu et al.. 2020), current research focuses on developing hypergraph neural networks (HNNs) with message passing module (MP-Module) that can be compatible with various task-specific modules. The MP-Module enables information exchange between connected nodes to generate informative node features for specific tasks (Feng al. 2019
Wang et al.] 2023b Telyatnikov et al. (202)). However, similar to other message passing neural networks (Frasca et al. 2020 Wu et al. 2002),, training the MP-Module makes loss computation interdependent for connected nodes, resulting in a computationally intensive training process for HNNs. This limits their practical applications, especially in the process of large-scale hypergraphs.
To address this challenge, our key solution is to develop a training-free MP-Module that shifts the processing of hypergraph structural information from the training stage to the data pre-processing phase. This approach is inspired by recent advancements in efficient GNNs (Wu et al.) 2019a Gasteiger al. 2201 Frasca al. (202)), where training-free MP-Modules are typically implemented as graph filters (Ortega et al. 2018). However, directly using existing hypergraph filters (Zhang et al.
Equal contribution.

---

Published as a conference paper at ICLR 2025
Training
MP-Modul
HNN:
the H. !
MP Layor
(H x
MP Layer
Classifier
8XX, Y)
Xy & H
Fix Layers
X
Training
TF-HNN:
TF-MP-Module
Xy SXy
Classifier
Xy H
Figure 1: Training pipeline of HNN vs. TF-HNN for node classification. Top row: HNN uses hypergraph structure to generate node features by a learnable MP-Module, which are then used by classifier, with the MP-Module and the classifier being trained together. For brevity, we omit the MLP in HNN for the input node features. Bottom row: TF-HNN comprises only the classifier trained for node classification and the TF-MP-Module that can be recomputed before the classifier training.
2019: Qu et al. 2022 presents two major obstacles. Firstly, these filters are usually designed for k-uniform hypergraphs, where all hyperedges must have the same size k. limiting their applicability to more general hypergraph structures with varying hyperedge sizes. Secondly, the reliance on an incidence tensor to represent the hypergraph presents significant challenges in practical usage. As the tensor dimension grows exponentially with n², existing computational resources struggle to perform multiplications involving such high-dimensional tensors. Therefore, instead of adopting existing hypergraph filters, we introduce a novel training-free free MP-Module for hypergraphs.
In this work, we develop a novel, training- free message passing module, hypergraphs to decouple the usage of hypergraph structure information from the model training stage. To achieve this, we first construct a theoretical framework that provides a unified view of existing HNNs (Huang & Yang. 2021 Chen et al. 2022, Chien et al. 2022, Wang et al. (023a)). Specifically, this framework identifies the feature aggregation function as the core operator MP-Modules in existing HNNs. Based on these insights, we design the TF-MP-M MMdulee in two stages: 1) We make the feature aggregation functions within the MP-Modules of four state-of-the-ar HNNs (Huang & Yang 2021 Chen al., 2022 Chien et al. 2022, Wang et al. 2023a) teining-free by removing their learnable parameters; and 2) To further enhance efficiency, we remove the non-linear activation functions and consolidate the feature aggregation across L layers into single propagation step. Remarkably, this two-stage process unifies the chosen MP-Modules into a single formulation, despite their different design philosophies. We refer to this unified formulation as the TF-MP-Module and denote the hypergraph neural network equipped with it as TF-HNN. To demonstrate the efficiency and effectiveness of TF-HNN, we provide a theoretical analysis showing three key advantages: 1) TF-HNN is more training-efficient efficient compared to existing HNNs; 2) TF-HNN can utilise as much information as existing HNNs for generating node features; and 3) TF-HNN is robust against the oversmoothness issue while taking into account the long-range information.
for
The main contributions of this work are summarised as follows:
We present an original theoretical framework that identifies the feature aggregation function as the core component of MP-Modules in existing HNNs, which processes the hypergraph structural information. This insight provides a deeper understanding of existing HNNs.
We develop TF-HNN, an efficient and effective model for hypergraph-strucct knowledge, TF-HNN is the first model that decouples the processing of hypergraph structural information from model training, significantly enhancing training efficiency.
data. To our
We thcoretically support the efficiency and effectiveness of TF-HNN by showing that: 1) It leads to
remarkably low training complexity when solving hypergraph-related downstream tasks; 2) It utilises the same amount of information existing HNNs for generating node features; and 3) It is robust against the oversmoothing issue when utilising long-range information.
We conduct extensive experiments in both node classification and hyperedge prediciton tasks to compare TF-HNN with nine ofttee-o--hh-arr HNNs. The empirical results show that the proposed TF-HNN exhibits both competitive learning performance and superior training efficiency.
2

---

Published as a conference paper at ICLR 2025
2 NOTATION
Hypergraph. Let (V,&,H) be a hypergraph, where V (v1, "n, is the node set, E (ei,e,,"",,, is the hyperedge set, and H E {0, is incidence matrix in which H means that Eg contains node v, and H = otherwise. Define DHv e 1850 as a diagonal matrix of node degrees and DHE= Rg
diagonal matrix of hyperedge degrees, where D ny and Duc, are the number of hyperedges with e and the number of nodes in eg. respectively. We assume hypergraphs do not have isolated nodes or empty hyperedges; details of this assumption are in Appendix Q Moreover, we define: 1) The distance between two nodes on a hypergraph is the number of hyperedges in the shortest path between them, e-g., the distance is one if they are directly connected by a hyperedge; and 2) The h-hop neighbours of node e; are all nodes with a distance of & or less.
Graph and elique expansion. Let G {V. W) be a graph, where V {ur, "2,"" Up) is the node set, and W R is the adjacency matrix of G in which Wi O means that v2 and fj are connected and W ky otherwise. We set D e Rgx" as diagonal node degree matrix for g, where Da: the sum of the i-th row of W. Moreover, we denote the graph Laplacian of G as L D W. Given hypergraph H = {V,E E, H), its clique expansion is defined as graph G = {V, W). where remains unchanged, and Wij > 0 if and only if u, and Vy connected on H. and W, sj otherwise.
Other notations. We set node features Xy= [x!,, X x I _ which is matrix that contains d-dimensional features. We denote functions or variables at the 1-th layer of a model using the superscript (1) and use @ for concatenation. We use @ to represent the learnable weight matrix in a model, MLP() (.) for a multilayer perceptron (MLP), and (-) for non-linear functions (e.g., ReLU).
METHODOLOGY
In this section, first show overview of the proposed TF-HNN approach (Subsec. 33. Moreover, we present a theoretical framework that offers a unified view on existing HNNs (..bsec.(3.2) Finally, we use the insights gained from the theoretical framework to design our TF-MP- (Subsec.
3.1 OVERVIEW
To enhance the GNN efficiency, Gasteiger et al. (0119) design APPNP by using the connection between GCN Kippf & Welling (017) and PageRank (Page et al. 1998), shifting learnable parameters to an MLP prior to the message passing. Wu et al. (2019) and Frasca et al. (2020) further improve training efficiency by removing the MLP before the message passing, completing message passing in preprocessing and making it training- training-free. Inspired by them, we design the TF-HNI that removes the reliance on message passing at training. Generally, TF-HNN described as the following framework:
Y = (xvv)), Xp= = S(x,, H),
(1)
where (p)()) is a task-specific module, Y denotes the task-specific output whose dimension is task-dependen!" | S(-) denotes an MP-Module that contains only pre-defined parameters allowing be precomputed in the pre- reepproessing phase, and X, é Rund denote the features generated by S()) In the GNN literature, S..) is implemented as the graph filter (Ortega et al. 2018). However, we cannot directly use existing hypergraph filters (Zhang et al., 2019: Qu et al. al. 2022) for two main reasons. Firstly, existing hypergraph filters are usually designed for k-uniform hypergraphs, where all hyperedges have size Secondly, these filters require the use of an incidence tensor, which presents challenges in handling high-dimensional matrix multiplication. Hence, rather than relying on existing methods, we design a novel function S(-) for hypergraphs, the TF-MP-Module. Since our design is based existing MP-Modules for hypergraphs, the next section introduces an original framework that offers a unified perspective on current HNNs.
3.2 REVISITING HYPERGRAPII NEURAL NETWORKS
A traditional HNN for certain downstream task is formulated as the following equation: Y= (==o(()) (X)), X = =
For instance, in the node classification task, Y E Buxe contains the logits for e categorics; and the hyperlink prediction task, Y Th contains the probability of mn potential hyperedges.
3

---

Published as a conference paper at ICLR 2025
Table I: Overview for four state-of-the-art HNNS and our TF-HNN. In this table, Y0 YE, TD. 7/ 10,1) are hyperparameters and I Iddd denotes an identity matrix. Moreover. M is the training computational complexity of the task-specific module, n is the node count, m is the hyperedge count, || is the number of non-zero values H, T
m' is the number of edges in the clique expansion, the number of training epochs, L is the number of layers, and d is the feature dimension.
Hracur
Corpotatioal Corplesiy
xf
(a (a-soip Prab Pi HUX Pa Jam)
Ofd 43 [[i 21nd))
Dtpp HILNN
2002
Litect
x (1 (a-na Iiu uogee th xox ((1 1602 "i
DEN + Amml--indd
AlDoayees
Chicuit d.
3022
Inlistt
X MLP
Dii HSUP (D.. MIP(X)""))
OUN TCJBi,d) TM/5 wad')
ID-INN
Dhang aagg
ait1a
Intrnct
x
(1- Dell manr (D MUrCK!"))) Xu
Distt
Xu
OLAB)
where ())() is a task-specific module, Y denotes the task-specific output whose dimension is desendepnn, (-) is a learnable message passing module, and Xy R denote the features generated by ( The modules (.).) and ( are trained together with the supervision given by the downstream tasks. See Figure I for the comparison between the training pipelines of traditional HNNs and TF-HNN in the context node classification. Existing HNNs use the hypergraph structural information by the message passing module (.) (MP-Module). The MP-Module in existing HNNs
is typically based on one of four mechanisms [2] clique-exxie sxuuue-epansion, star-exanssion line-expansion, or
incidence-tensee (Huang& & Yang, 202T; Wang al.
[0236] CCen et al.) [002]] Chien et al.) [202]
Wang et al.) 2023a|] Yang et al. 2022 Antelmi et al. 2023 Kim et al. (024). We summarize the
MP-Module using these mechanisms in Proposition 3.1, with the proof provided in Appendix C
Proposition 3.1. Let X be features of node i in the 1-th message passing layer of a HNN based on patppansiionsta--eppannnnniiiiii
diqque-expansion/starr expansion//ine-eeee sxppasiin/ne-xxppsooonniiieeeiii wise feature aggregation function, and NH.,, is the set of neighbours of Vs the hypergraph. For
tensee, @e(-) denotes a learnable node-
(1) i O, we have (0) ML.(xx.). For i 2t, we have Xu: Φ0(xx, Xx; @.,eNNu.. X03 21-1)).
(0) (1-1)
Specifically, (.) defined in Proposition 3.1 can be categorized into two types: direct and indirect
feature aggregation functions. The direct approaches aggregate features of neighbouring nodes to that of the target node directly (Huang & Yang, (201]] Wang et al., (2033)), while the indirect methods aggregate features of neighbouring nodes to that of the target node via some virtual nodes (Chien
et al. 2022
Wang et al.
2023a Yang et al. 2022). These approaches can be formulated as:
Ja fo(x + fi (-1))) +
Σ
f2(x(111)).
(3a)
ByENNo,
0 90(90(x(!)) (x + +p 91 (x(( BuneNun. ))
(3b)
where fo()), fi(). fe(.). fa(:) go()). (t), 92(() represent seven learnable functions, and p e
R" is a vector used to aggregate the features of virtual nodes generated by (-)... Here, n. is hyperparameter corresponding to the number of the virtual nodes, and Eq. (3a) and Eq. (3b) represent the direct and indirect approaches, respectively. Based on Propostion 3. the hypergraph structure is only used to construct the neighbourhood set Nr, within the MP-Module of HNNs. Consequently, the feature aggregation function (e(() is the core component of the MP-Module in processing hypergraph structural information. Moreover, based on Eq. (3a) and Eq. (35). the usage of the hypergraph structural information is not directly related to the learnable parameters and the non-linear function. Therefore, in the next subsection, we design our (F-MP-Module by first removing the learnable parameters for the feature aggregation function from existing HNNs to make it training-fres To further improve its efficiency, we eliminate the non-linear function from the feature aggregation process. Specifically, we design our model based on HNNs shown in Table
3.3 TRAINING- MESSAGE PASSING
We introduce the TF-MP-Module by removing the learnable parameters and the non non-linear function from four ofthe-att HNNs shown in Table which includes two methods with the direct feature aggregation function and two methods with the indirect feature aggregation function.
Details on these mechanisms can be found in Appendix B
4

---

Published as a conference paper at ICLR 2025
Learnable parameters removal. Based on the insights from Subsec. 3.2 to develop a training- free MP-Module, we first remove learnable parameters from the feature aggregation functions of the selected HNNs. Specifically, we replace the learnable matrices in these functions with identity matrices. This removal allows us to reformulate the feature aggregation functions as follows:
x - -w
1/2 HV
HD RE
DHe
H'
x
x
(4a)
x of -DD) D
HD
He
D H
1/2 ((-))) xDxx) Hv
(4b)
x D Dau Ho D h
H o (xxx11))
(4c)
x a (1 - 78)0 Duu Ho (D H' (xx + x (0)
(4d)
where Eq.
4a Eq. Eq. (4c) and Eq. 4d represent the simplified formulation of UniGCNIII
Deep-HGNN, AllDeepSets and ED-HNN, respectively. Notably, (-) only denotes a general non-
linear function without parameters, not implying that these formulas use the same function.
Linearisation. Inspired by the insights from Subsec. 3.2 we further remove the non-linearity from the functions represented by Eqs. (4a-4d) thereby transforming an MP-Module with I, feature aggregation functions into a more efficient single propagation step. For the sake of brevity, we present the simplification results as Proposition 3.2 and provide its mathematical proof in Appendix D
Proposition 3.2. Let Xy be input node features, Xy be the output of the MP-Module of an Setp://AADDDeeSeee//E-H---DDD
with L MP layers, and arE [0, 1). Assume that learn- able parameters and the non-linearity are removed from the module. For each model, given hypergraph H = -V-VE,,H, there exists a clique expansion G = (V, W) to unify its output as the following formula Xy= ((1-a)'"'+ ta (1-a)
This proposition shows that, via our simplification, the selected models can be unified into a single formula. Based on this observation, our -Fodull is designed as the following equation:
X= SXy.
where S (1-a) " - +a El=o (1-o)' W Intuitively, in this formula, larger value emphasizes the retention of the node's inherent information, while a smaller a value increases the influence of information from neighbouring nodes. We define that the L used to compute S is the number of layers of an MF-MleModule. To understand the behaviour of an L-layer TF-MP-Module, we present the Proposition 3.3 For conciseness, we prove it in Appendix E
Proposition 3.3. Let H = (V, H) be a hypergraph and G (V, W) be its clique expansion with self-loops. Assume that a [0, 1). and S= (1 + a I--1 (1 a) WW Then, for i * j. we have Siz > 0 if and only if vi is an L-hop neighbour of Uj on H. and S:j otherwise.
enables the information exchange between any node 2 and its h-hop neighbours on the hypergraph. the next paragraph, we detail the generation of S.
By Proposition 3.3 an L-layer
Operator design. To use the TF-MP-Module in our TF-HNN, the key is designing W to generate S. We achieve this using a hyperedge-sizze size-baseee edge weight that is defined as:
8(v,, ex)
W
(5)
DHL,
1=11
where 5(-) is a function that returns if eg connects v; and v, and returns 0 otherwise. We discuss the connection between the clique expansion mentioned in Proposition 3.2 and the one defined in Eq (5) in Appendix T The main intuition behind the design of this edge weight is that, under certain conditions, the value of W H., is positively correlated with the probability of nodes and uj having the same label. Consequently, this edge weight design can make the weighted clique expansion the homophily assumption, which is that connected nodes tend to be similar to each other (McPherson jet al., (001). Appendix(Flelaboruii graph domain (Wu et al. 2020), we generate S based symmetrically normalised W n with the self-loop. Specifically, we generate the S by:
eelabbrrsss on this intuition. Furthermore, inspired by previous works in the
I- s= <= 1/' WW 1/231 +a Σ (D=
(6)
10
5

---

Published as a conference paper at ICLR 2025
where WH WH + I, and DH R is the diagonal node degree matrix of W H. In the next section, we present the theoretical analysis to support the efficiency and effectiveness of our TF-HNN.
4 THEORETICAL ANALYSIS
In this section, we begin by showing the efficiency of the proposed TF-HNN by comparing its training complexity with the traditional HNNs. Additionally, we demonstrate the effectiveness of our TF-HNN by analyzing its information utilisation and its robustness against the oversmoothing issue.
Training complexity. As illustrated by Eq. D the TF-HNN in downstream tasks requires training only the task-specific module. Additionally, since the TF-HNN relies on pre-defined propagation operator, its computations can be handled during data pre- processing. Consequently, the training complexity of the TF-HNN is determined solely by the task- module. In contrast, the basis of Eq. 2 the HNNs involve training both the MP-Module and the task-specific module. The MP-Module in HNNs includes complex learnable operations that facilitate message passing between connected nodes on a hypergraph. inherently making the training of the HNNs more complex than the TF-HNN, See Figure [ for an example that qualitatively compares the training pipelines of the TF-HNN and HNN in node classification. To provide a quantitative comparison, we summarize the training computational complexity of the TF-HNN and the four state-of-the HNNs in Table [ This table shows that the training complexity of the TF-HNN is consistently lower than that of the HNN, theoretically confirming the training efficiency of the proposed TF-HNN.
Information utilisation. The core strength of existing HNNs lies in their ability to utilise the informa- tion embedded within hypergraph structures to generate powerful node features for downstream tasks. Consequently, analyse the effectiveness of our proposed TF-HNN by comparing its information utilisation capabilities with those existing HNNs and conclude that our methods utilise the same amount of information as exising HNNs without any loss, which is summarised in Proposition 4.1 For conciseness, we show the detailed proof for Proposition 4.1 in Appendix G
Proposition 4.1. Let H² be the entropy of information used by an HNN with L feature aggregation functions in generating the features of a node v. H" be the entropy of information used by TF-HNN with an L-layer (F-MM--Mddul for the same purpose, and H² denote the entropy of information within node vi and its L-hop neighbours on the hypergraph. Then, Hy =H H²
The proposition above theoretically demonstrates that TF-HNN is as effective as existing HNNs in utilising information from pre-defined hypergraphs to generate node features. Specifically, both TF-HNN and existing HNNs achieve this by aggregating neighbourhood information.
Robustness against the oversmoothing issue. According to Proposition 3.3 TF-HNN can use global interactions within the given hypergraph to generate node features by deepening the TF-MP-MMdule. However, as shown in a recent study (Chen et al., 2022). the message- message-passinn-- pass-ag--assseeeee based models on hypergraphs may suffer from the oversmoothing issue. This issue refers to the tendency of model to produce indistinguishable features for nodes in different classes as the model depth increases, which might degrade the performance of the framework in downstream tasks. Consequently, to further support the effectiveness of TF-HNN, we analyse its robustness against the oversmoothing issue. Inspired by works in the graph domain (Yang et al. 202T. Zhang et al. 2020, Singh et al. 2025). we
show Proposition 4.2 with the proof provided in Appendix H
Proposition 4.2. Let H. (V,E, H) denote a hypergraph, G (V.WH) be its clique expansion with edge weights computed by Eq. (5). be the graph Laplacian matrix of g computed by symmetrically normalised and self-loops added W Xy represent the input node features, and
a E [0, 1) be a hyperparameter. We define that F F(X) tt(X" T-a tr((X (X-))xXXv)] and Fmin the global minimal value of F(X) for X & gand, Assume that: 1) Xv SXy; 2) is computed by Eq. (6) 3) 0; and 4) L +00. Then, F(X,)= = Fmin-
a
Notably, minimising the first term of F(X) enhances feature similarity among connected nodes on H. and minimising the second term of F (X) encourages the generated features of each node to retain
We use "entropy of information" in a conceptual way- A detailed discussion is in Appendix U

---

Published as a conference paper at ICLR 2025
Table 2: Dataset statistics, More details of these datasets are in Appendix
Name
Nodes
# Hyperodges
Features
Classes
Node Definition
Hyperedge Definition
Con-CA
1072
1433
Paper
Co-authorhhp
DBLP-CA
41302
22363
1425
6
Paper
Co-sullorship
Citeseer
3312
1079
3703
Puper
Co-citaion
Congress
83105
100
Congressperson
Legishaiiv--Bill--S
Spols--Soo
House
1290
340
100
2
Represcntative
Committee
Senate
282
315
100
2
Congressperson
Legislatie SeiislavveBBillsSSpee -plls-Sponooshhip
Trivago
233202
300
160
Ascommodation
Browsing Session
the distinct information from its input features. Therefore. node features that can minimise F(X) capture two key properties: 1) neighbouring nodes on H have similar features; and 2) each node contains unique information reflecting its individual input characteristics.
tends to assign similar features to connected nodes on hypergraph while maintaining the unique input information of each node. Hence, even with an extremely large number of layers, the TF-MP-Module can preserve the unique input information of each node, making robust against the oversmoothing issue. This property further ensures the effectiveness of our TF-HNN. Sec. empirically supports the analysis presented in this section.
Based on Proposition 4.2 deep
5 RELATED WORK
Recent efforts have leveraged hypergraphs to improve node feature learning for downstream tasks (Bick al., 2023 Liu et al. [0044] Xu et al. 2024). The predominant models in this area are hypergraph neural networks (HNNs) with message passing module that enables information exchange between connected nodes (Antelmi et al. (023). However, as noted for their graph coun- terparts (Wu et al. 2019a Frasca al., (020), HNNs suffer from low learning efficiency. Two recent works attempt to mitigate this limitation (Feng et al., 2024 Tang et al. 2024). In (Feng al. 2024), HNN named HGNN (Feng et al. 2019) is distilled into an MLP during training; while in Taag et al. 2024). hypergraph structural information is integrated into an MIP via a loss function computed by the sum of the maximum node feature distance in each hyperedge. These approaches primarily focus on reducing model inference complexity via dedicated training process. However, these designs still lead to hypergraph machine learning models with computationally intensive training procedures. For instance, the distillation process in (Feng et al.) 2024) necessitates training HNN with a computational complexity O(M TT''d + where with M the training computational complexity of the task-specific module, is the node count, m' is the number of edges in the clique expansion, the number layers, and dl the feature dimension. Given that many existing models applied semi semi-supervised hypergraph node classification settings, which involve the simultancous use of training and test data (Huang & Yang) 2021; [Chien et al.) 2022 Wang et al. 2023a Joachims al. 1999 Chapelle et al. 2009), reducing the training complexity of hypergraph-based models remains a critical yet underexplored challenge. We address this challenge with novel model named TF-HNN, which is the first to decouple the processing of hypergraph structural information from model training.
6 EXPERIMENTS
6.1 EXPERIMENTAL SETUP
Task description. We conduct experiments in two tasks: hypergraph node classification and hyperlink prediction. In hypergraph node classification (Wang et al. 2023a b Duta et al. (023). we are given the hypergraph structure H. node features Xy. and set labelled nodes with ground truth labels
Yias >eVinn where yo; (0, 1) be one-hot label. Our objective is to classify the unlabeled nodes within the hypergraph. hyperlink prediction (Chen & Liu 2023), we are given node features Xv. set of observed real and fake hyperedges Eos, and a set of potential hyperedges Ep. The task requires the model to distinguish between real and fake hyperedges in E, using Xy and E
Dataset and baseline. We conduct experiments on seven real-world hypergraphs: Cora-CA, DBLP- CA, Citeseer, Congress, House, Senate, which are from (Chien et al. (2022), and Trivago from Kim al. 2023). We use nine HNNs as baselines. Five of these baselines utilize direct feature aggregation, including HGNN (Feng et al.) 2019), HCHA (Bai et al., 202T). UniGCNII (Huang & Yang 2021).
PhenomNN (Wang al. 2023b), and Deep-HGNN (Chen et al. 2022). The remaining four baselines
7

---

Published as a conference paper at ICLR 2025
Table 3: Node classification accuracy (%) for HNNs and TF-HNN. The best result on each dataset is highlighted in bold font. The second and third highest accuracies are marked with an underline.
Com--Ca
DBLP.CC
Clteseer
Congress
Hruse
Senate
Avg. Mean
HGNN
82.64
1.65
91.03
12,45 : 1.16
26,26.3
1.15
39
2.96
48.59
4.52
74.56
HCHA
82,55 = 0.97
90.92 1 0.22
72.42
1.42
90.43 1 1.20
61.36 2.53
48.62 1 4.41
74.38
HNHN
77.19 1.49
86.78 0.229
72.64
1.57
53.35
+ 1.45
67.80
2.59
50.93
6,35
68.
UniGCNII
83.60
0.
73.05
2.21
94.81
0.81
67.25
49.30
76.62
AllDees
81.97 1.50
91.27
70.83
1.63
91.80
1.53
67.82
48.17
75.31
1.47
73.08
=: 1.20
1.05
5.22
76.93
PhenomNN
05.8
91.91 + 0.21
75.10
+ 1.59
88.24
1.47
70.71
2.35
67.70
+.224
79.91
ED-HNN
83,97 1.55
91.90
73.70
1.38
95.00+00.9
72.45
2.28
64.
5.14
80.30
Deep- HGNN
84,89 + 0.88
91.76 + 0.28
74.07
+ 1.64
93.91 + 1.18
75.26
1.76
68.39 4.79
81.38
TF-HNN (ours)
86,54 1.32
74.82 + 1.67
95.09
0.89
76.29
1.99
70.42
2.74
82.50
T7-48 Dumo
Abbesssea
ED-WY
HOHA
Phoremi
10a
Relative Traiiing
101
Giteseer
DBLP
Senatz
Figure 2: The relative training time required for HNNs and TF-HNN to achieve optimal performance.
employ indirect feature aggregation, consisting of HNHN (Dong et al. 2020). AllDeepSets (Chien et al. 2022), AllSetTransformer (Chien et al. 2022j. and ED-HNN ( (Wang et al. 2023a). The dataset statisties are summarized in Table with detailed descriptions of datasets available in Appendix [|
Metric. Following previous works (Wang et al., 2023a Chen & Liu 2023 we evaluate models for node classification and hyperlink prediction with classification accuracy and the Area Under the ROC Curve (AUC), respectively. Similar to Wu et al., we employ the relative training time ry, defined as ry 1//4, to evaluate the efficiency of the models, where L is the training time to achieve optimal performance for the evaluated model, and t. is the training time for optimal performance of TF-HNN.
Implementation. For node classification, we follow previous works (Wang et al. 2023a Duta et al., 2023) to use 50%/25%//25% train/validation/test data split and adapt the baseline classification accuracy from them" [] Additionally, similar to these works, we implement the classifier based on MLP for our TF-HNN and report the results from ten runs. For hyperlink prediction, existing works (Chen) & Liu 2023) primarily focus on the design of the prediction head, with no reported results for applying our baseline HNNs this task, we report all results from five runs conducted by ourselves. We employ the deep set function implemented by (Chien et al. 2022) as the prediction head for our method and bascline methods due its simplicity. We use a 20%%25%/2544 train//allddaiionness data split and ensure each split contains five times as many fake hyperedges as real hyperedges. TF-HNN and HNNs only use real hyperedges in training and validation sets for message passing. Experiments were on RTX 3090 GPUs by PoTor... See our code here. More details are in Appendix and S
6.2 COMPARISON WITH BASELINES
Node classification. We summarize the classification accuracy and relative training time of the TF-HNN and HNNs in Table and Figure 2 respectively. Table 3 demonstrates that TN-HNN not only leads to the best results across four datasets (Cora-CA. Congress, House, and Senate) but also results in the highest average mean accuracy overall. These findings confirm the effectiveness of TF-HNN in generating powerful node features for node classification. As discussed in Sec. 4 we attribute this effectiveness to the ability of TF-HNN to match the power of existing HNNs in utilising neighbourhood information to generate node features, while requiring the optimization of fewer parameters. This makes the TF-HNN more efficient in utilising training data and reduces the risk of overfitting, thereby outperforming more complex HNN counterparts. On the other hand, as shown in Figure 2, the TF-HNN consistently needs less training time to achieve superior performance compared to the HNNS. This highlights the high training efficiency of TF-HNN downstream tasks.
Hyperlink prediction. Figure shows the results for hyperlink prediction, which exhibit a pattern similar to that observed in the hypergraph node classification task: the TF-HNN outperforms the
Sinne there are no reported results for Deep- under the chosen data split, we use our own results.
8

---

Published as a conference paper at ICLR 2025
100
AUI (96
AUC
SF
AUC 196)
80-
95
Deep ION4
HGW ABSahhnnfamme
HGNN
AEathurl'mme
20-1RI
90
10
10
10
Relative Training Time
Relative Training Time
Relative Training Time
(a) Cora-CA.
(b) Citeseer.
(c) House.
Figure 3: Hyperlink prediction AUC (%) and relative training time for HNNs and TF-HNN.
Table 4: Node classification accuracy (%) and training time for models on Trivago.
Table 5: Hyperlink prediction AUC (%) and training time for models on Trivago.
Metric Methods
Deqp-HONN
EISHNN
TF-HNN (Chrs)
Metri Methods
Deep HGNN
ED HNN
TF' HNN (Oun)
Accuracy (58)
84,06 1.70 70
48.38 1.35
94,03 0.51
AUC (5)
$4.26 + 1.62
69.44 7.09
95.71 + 1.00
Training Time (9)
3352.55 1089.3
14392 34.64
38.099 6.65
Training Time (9)
1873.67 #:559.55
1311.01 027.99
48400 = 110.99
75
Accunicy 190
Accunccy (9)
Accuracy 18)
Accuracy
15
4m00
6-0.6
6-0 2.
6m0 6-
2
2-
16
32
Number
The Number
(a) Cora-CA.
(b) Citeseer.
(c) House.
(d) Senate.
Figure The impact from the hyperparameters of MF-MMle in node classification.
HNNs while requiring less training time. An interesting observation is that the training efficiency improvement from TF-HNN is smaller in the hyperlink prediction task compared to the node classification task. The intuition is that the task-specific module used for hyperlink prediction is more computationally expensive than the one used for hypergraph node classification. Specifically, the hyperlink predictor needs to generate prediction for each hyperedge by aggregating the features of the nodes connected by the hyperedge, whereas the node classifier only needs perform forward propagation for each individual node. Appendix Midisusses that the efficiency improvement provided by TF-HNN is inversely correlated with the complexity of the task-specific module.
Scalability of state-of-the-art models. We compare the scalability of our TF-HNN against the top two baseline methods from previous experiments, Deep-HGNN and ED-HNN. Deep-HGNN uses direct feature aggregation function, while ED-HNN uses an indirect feature aggregation function. We evaluate the performance of all methods node classification and hyperlink prediction using the large-scale hypergraph benchmark, Trivago, with the results detailed in Table 4 and Table 5 respectively. The results show that TF-HNN not only surpasses its competitors in performance but also dramatically reduces training time. Remarkably, in node classification, TF-HNN improves the accuracy by around 10% over the best baseline, training time. These findings underscore the value of TF-HNN for large-scale hypergraphs Due to the space limit, additional discussions and experiments are in Appendix N
while using only about 1% of its
R and V
6.3 ANALYSIS
We conduct series of analyses on the components within the proposed TF-HNN model in node classification on the Cora-CA, Citeseer, House, and Senate datasets.
Impacts from the TF-MP-Module In this study, we evaluate the effectiveness of our proposed TFdMleModule within the TF-HNN framework by comparing two training-free message passing methods developed for graphs: SGC (Wu et al. 2019a) and SIGN (Frasca et al. 2020). To adapt
OOn Trivago, Deep-HGNN's training time is much higher for node classification than hyperlink prediction. This is because 64-layer model largely improves node classification accuracy, while 2-layer model suffices for hyperlink prediction, as deeper models offer no henefit. The extra layers account for the longer training time.

---

Published as a conference paper at ICLR 2025
Table 6: The impact from the TF-MP-Module in node classification. The best results are bold font.
Cora-CA
Citeseer
Senate
House
Avg. Mean
SGC with CE
81.00 1.68
71,42
47.18 6.48
64.09 74.59 76.29 1.99
56.16 3.21
SIGN with TF-MP Module (ours)
86.04 = 1.53
73.62 + 1.79
64.93
73.75 1.91
86.54 + 1.32
74.82 1.67
2.74
77.02
Table 7: The impact from the weighted S in node classification. The best results are in bold font.
Cora-CA
Cieeseer
Senste
House
Avg. Mean
TF-IINN without Weighted HF-HNN with Weighted $
K4.90 1.55
74.75 1.62
68.87 4.57
15.57 L.11
76.02 77.02
86.54 1.32
74.82 + 1.67
70.42 2.74
76.29 + 1.99
these methods to hypergraphs, we first project the given hypergraph into its clique expansion (CE), as introduced Sec. [ and then apply SGC and SIGN to this projected structure. Notably, we only replace the TF-MP-Modun with SGC using CE and SIGN with CF, while keeping all other components of the TF-HNN unchanged. Table 6 show that our TF-MP-Module consistently out- performs the alternatives across all datasets. The superior performance of our method over SGC with CE is due to SGC with CE being a special case of our discussed in Proposition 4.2 can cause node features become overly similar to their neighbours, potentially reducing distinctiveness. For SIGN with CE, the method concatenates features from different neighbor hops with the initial node features and processes them through a classifier, forcing it to handle both feature merging and classification, which increases complexity and can hinder convergence. contrast, our "F-MM-MModul directly merges neighbouring node information using the operator in Eq. (o) allowing the classifier to focus solely on classification, thereby simplifying the learning process and improving performance. These findings highlight the greater capacity of our TF-MP-Module compared to existing graph-based training-free modules in processing hypergraph structures, further underscoring its unique value for hypergraph machine learning.
with a = O, which, as
Impacts from hyperparameters of WT-MP-Module, We focus on two key hyperparameters of the TF-MP-Module: the number of layers and the a. As illustrated in Figure 4 models with > exhibit more consistent accuracy as the number of layers increases compared to models with O. We attribute this consistency to the robustness discussed in Proposition 4.2] where positive a helps retain the distinct information of each node, thereby mitigating the oversmoothing issue. Additionally, we observe that a positive a significantly enhances performance in the House and Senate datasets, which are more heterophilic, compared the Cora-CA and Citeseer datasets. This suggests that retaining the distinct information of each node is more beneficial for heterophilic datasets.
Impacts from the weighted S. We define S based on Eq. 5 as weighted S. In contrast, in this experiment, for the framework without weighted S, S is generated using an adjacency matrix W of a clique expansion with all positive values set one. As shown in Table adding weighted S further enhances performance, highlighting its informativeness.
7 CONCLUSION
In this paper, we propose novel model called TF-HNN. The key innovation of TF-HNN is an original training-frea message passing module (TF-MP-Module) tailored for data on hypergraphs. We present both theoretical and empirical evidence demonstrating that TF-HNN can efficiently and effectively address hypergraph-related downstream tasks. To our knowledge, TF-HNN is the first model shift the integration of hypergraph structural information from the model training stage to the data pre- rre-processing stage, significantly enhancing training efficiency. The proposed TF-HNI can advance the field of hypergraph machine learning research by not only improving the efficiency of using observed hypergraph structures to solve downstream tasks, but also serving as a simple starting point for the development of future hypergraph machine learning models.
Limitation and future work. Learning tasks on hypergraphs can be defined at the node, hyperedge, and hypergraph levels. In this paper, we test our TF-HNN on node-level and hyperedge-level tasks, with the main limitation being the lack of results on hypergraph-level tasks. Designing task-specific features based on existing node features remains a challenging
modules to generate hypergraph-Ieww topic in the literature. Therefore, consider the application of our TF-HNN to hypergraph-lewe leveel tasks as future work. Finally, developing an automated and efficient hyperparameter search method for hypergraph neural networks presents promising avenue for further enhancing model efficiency.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
Keyue Jiang was supported by the UKRI Engincering and Physical Sciences Research Council (EPSRC) Igrant number EP/R513143/1) Siheng Chen gratefully acknowledges support from the National Natural Science Foundation of China under Grant 62171276. Xiaowen Dong acknowledges support from the Oxford-Man Institute of Quantitative Finance, the EPSRC (EP/T023333/1) and the Royal Society (IEC ANSFC 1211188).
REFERENCES
Sameer Agarwal, Kristin Branson, and Serge Belongie. Higher order learning with graphs. In Proceedings of the 23rd international conference on Machine learning, pp. 17-24, 2006.
Alessia Antelmi, Gennaro Cordasco, Mirko Polato, Vittorio Scarano, Carmine Spagnuolo, and Dingqi Yang. A survey on hypergraph representation learning. ACM Computing Surveys, (6(1):1-38, 2023.
Song Bai, Feihu Zhang, and Philip HS Torr. Hypergraph convolution and hypergraph attention. Pattern Recognition, 110:107633 2021.
Christian Bick, Elizabeth Gross, Heather Harrington, and Michael T Schaub. What are higher-order networks? SIAM Review, 65(3):686-77) 731, 2023.
O. Chapelle, B. Scholkopf, and A. Zien, Eds. Semi-supervised learning (chapelle. o. et al., eds.; 2006) [book reviews]. IEEE Transactions on Neural Networks, 20(3):542-542, 2009, doi: 10.1 1,.11999NN.2009.200 20155994
Can Chen and Yang- Yu Liu. A survey on hyperlink prediction. IEEE Transactions on Neural Networks and Learning Systems, 2023.
Guanzi Chen, Jiying Zhang, Xi Xiao, and Yang Li. Preventing over-smoothing for hypergraph neural networks. arXiv preprini arXiv: 17159, 2022.
Eli Chien, Chao Pan, Jianhao Peng. and Olgica Milenkovic. You are allset: multiset function frame- work for hypergraph neural networks. In International Conference Learning Representations, 2022. URL http::::////ppneeeeeeeee
Yihe Dong, Will Sawin, and Yoshua Bengio. Hnhn: Hypergraph networks with hyperedge neurons. ICML Graph Representation Learning and Beyond Workshop, 2020. URL https: :
orq/ hogg/abs/0000 2006. 12278
Tulia Duta, Giulia Cassarà, Fabrizio Silvestri, and Pietro Liò. Sheaf hypergraph networks. Advances in Neural Information Processing Systems, 36, 2023.
Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 3558-3565, 2019.
Yifan Feng. Yihe Luo, Shihui Ying, and Yue Gao. LightHGNN: Distilling hypergraph neural networks into MIPss for 100x faster inference. In The Twelfth International Conference on Learning Representations, 2024. URL
net. forum? wid.eaase...ssssS
Fabrizio Frasca, Emanuele Rossi, Davide Eynard, Ben Chamberlain, Michael Bronstein, and Federico Monti. Sign: Scalable inception graph neural networks. arXiv preprint 10Xx::0004.11990 2020.
Johannes Gasteiger, Aleksandar Bojchevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank. In International Conference on Learning Representations (ICLR), 2019.
Dmitriy Genzel and Eugene Chamnak. Entropy rate constancy in text. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 199-206, 2002.
Yi Han, Bin Zhou, Jian Pei, and Yan Jia. Understanding importance of collaborations in co-authorship networks: A supportiveness analysis approach. In Proceedings of the 2009 SIAM International Conference Data Mining, pp. 1112-1123. SIAM, 2009.
11

---

Published as a conference paper at ICLR 2025
Jing Huang and Jie Yang. Unignn: unified framework for graph and hypergraph neural networks. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, 2021.
Bukyoung Jhun. Effective epidemic containment strategy in hypergraphs. Physical Review Research, 3(3):033282, 2021.
Thorsten Joachims et al. Transductive inference for text classification using support vector machines. In leml, volume 99, pp. 200-209. Citeseer, 1999.
Jinwoo Kim, Saeyoon Oh, Sungjun Cho, and Seunghoon Hong. Equivariant hypergraph neural networks. In European Conference Computer Vision, pp. 86-103. Springer, 2022.
Sunwoo Kim, Dongjin Lee, Yul Kim, Jungho Park, Tacho Hwang, and Kijung Shin. Datasets, tasks, and training methods for large-scale hypergraph learning. Data Mining and Knowledge Discovery, 37(6))2216 2254, 2023.
Sunwoo Kim, Soo Yong Lee, Yue Gao, Alessia Antelmi, Mirko Polato, and Kijung Shin. A survey hypergraph neural networks: an in-depth and step-by-step guide. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6534-6544, 2024.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017. URL /ttppppps////eeerrvieee
net / Eorum ? Si-
Solomon Kullback. Information theory and stutistics. Courier Corporation, 1997.
Zexi Liu, Bohan Tang, Ziyuan Ye, Xiaowen Dong, Siheng Chen, and Yanfeng Wang. Hyper- graph transformer for cenissupereieed classification. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7515-7519. IEEE, 2024.
Miller McPherson, Lynn Smith-Lovin, and James Cook. Birds of a feather: Homophily in social networks. Annual review of sociology, 27(1):115444 444, 2001.
Carl D Meyer and lan Stewart. Matrix analysis and applied linear algebra. SIAM, 2023.
Vu Nguyen. Bayesian optimization for accelerating hyper-parameter tuning. In 2019 IEEE second international conference on artificial intelligence and knowledge engineering (AIKE), pp. 302-305.
IEEE, 2019.
Antonio Ortega, Pascal Frossard, Jelena Kovačević, José MF Moura, and Pierre Vandergheynst. Graph signal processing: Overview, challenges, and applications. Proceedings of the IEEE, 106 (5):808-828, 2018
Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The PageRank Citation Ranking: Bringing Order to the Web. Technical report, Stanford Digital Library Technologies Project, 1998. URL
ist
31 1768
Ruyuan Qu, Hui Feng, Chongbin Xu, and Bo Hu. Analysis of hypergraph signals via high-order total variation. Symmetry, 14(3):543, 2022.
Claude E Shannon. Prediction and entropy of printed english. Bell system technical journal,, 30(1): 50-64, 1951.
Jasraj Singh, Keyue Jiang, Brooks Paige, and Laura Toni. Effects of random edge-dropping on over-squashing in graph neural networks. arXiv preprint arXiv:2502.07364, 2025.
Bohan Tang, Siheng Chen, and Xiaowen Dong. Hypergraph structure inference from data under smoothness prior. arXiv preprint arXiv:2308. 14172, 2023a.
Bohan Tang, Siheng Chen, and Xiaowen Dong. Learning hypergraphs from signals with dual smoothness prior. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEF, 2023b.
12

---

Published as a conference paper at ICLR 2025
Bohan Tang, Siheng Chen, and Xiaowen Dong. Hypergraph-mip: learning hypergraphs without message passing. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 13476-13480. IEEE, 2024.
Lev Telyatnikov, Maria Sofia Bucarelli, Guillermo Bernardez, Olga Zaghen, Simone Scardapane, and Pietro Lio. Hypergraph neural networks through the lens of message passing: a common perspective to homophily and architecture design. arXiv preprint arXiv:2310.07684, 2023.
Helen Victoria and Ganesh Maragatham. Automatic tuning of hyperparameters using bayesian optimization. Evolving Systems, 12(1):217-223, 2021.
Peihao Wang, Shenghao Yang, Yunyu Liu, Zhangyang Wang. and Pan Li. Equivariant hypergraph diffusion neural operators. In The Eleventh International Conference on Learning Representations, 2023a.
net Fooummid-RRii T
Yuxin Wang, Quan Gan, Xipeng Qiu, Xuanjing Huang, and David Wipf. From hypergraph energy functions to hypergraph neural networks. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 35605 300633 PMLR, 23-29 Jul 2023b.
Douglas Brent West et al. Introduction to graph theory, volume 2. Prentice hall Upper Saddle River, 2001.
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Sim- plifying graph convolutional networks. In International conference machine learning, pp. 6861-6871. PMLR, 2019a.
Jia Wu, Xiu-Yur Chen, Hao Zhang, Li-Dong Xiong, Hang Lei, and Si-Hao Deng. Hyperparameter optimization for machine learning models based on bayesian optimization. Journal of Electronic Science and Technology, 17(1):66-40, 2019b.
Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui. Graph neural networks in recommender systems: survey. ACM Computing Surveys, 55(5):1-37, 2022.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 32(1):4 24, 2020.
Chenxin Xu, Yuxi Wei, Bohan Tang, Sheng Yin, Ya Zhang, Siheng Chen, and Yanfeng Wang. Dynamic-groupe group-aware networks for multi-agent trajectory prediction with relational reasoning. Neural Networks, 170:564-577, 2024.
Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. Hypergen: A new method for training graph convolutional networks hyper- graphs. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alché-Buc, E. Fox, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Asso- ciates, Inc., 2019. URL httpps://procsst 1eff39bcaec6f3 www.eecc633900000 0ET100049666999996999993 3OFF333aaaeF13311111111333333333
wwwiiigg...euurpse neurips cc wwtppp/pp/p11pppp 9/file/
Chaoqi Yang, Ruijie Wang, Shuochao Yao, and Tarek Abdelzaher. Semi-supervised hypergraph the 31st ACM International Conference Information & Knowledge Management, pp. 2352-2361, 2022.
node classification on hypergraph line expansion. In Proceedings the
Yongyi Yang, Tang Liu. Yangkun Wang, Jinjing Zhou, Quan Gan, Zhewei Wei, Zheng Zhang, Zengfeng Huang, and David Wipf. Graph neural networks inspired by classical iterative algorithms. In International Conference on Machine Learning, pp. 11773-11771 11783. PMLR, 2021.
Hongwei Zhang, Tijin Yan, Zenjun Xie, Yuanqing Xia, and Yuan Zhang. Revisiting graph convolu- tional network on semi-supervite supervised node classification from an optimization perspective. arXiv preprint arXiv:2009. 1469, 2020.
Songyang Zhang, Zhi Ding, and Shuguang Cui. Introducing hypergraph signal processing: The- oretical foundation and practical applications. IEEE Internet of Things Journal, 7(1):39-6660, 2019.
13

---

Published as a conference paper at ICLR 2025
A IMPACT STATEMENTS
This paper presents work that aims to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted.
B BACKGROUND OF HYPERGRAPH MESSAGE PASSING
Clique expansion. Given a hypergraph H = (V,& & H), its clique expansion defined as a graph (V, W.. where V remains unchanged. and Way > if and only if v; and V are connected by hyperedge on H and Wij 0 otherwise. Hence, each hyperedge in H. is a clique in G.
Star expansion. Given hypergraph H (V,,,, HJ, its star expansion is defined G -V(VVV W), where V is node in H. hypernode Vee E VV corresponds to the hyperedge ej in H. and there is an edge between U, and Eee, if and only if e, contains U, in H.
bipartite graph
Line expansion. Given hypergraph H. (V,E, H), its line expansion is defined g= (Vi, W). The node set Vi of 9 is defined by vertex parte-hhyperedgr pair ((v,e) e u v.ce E} from the original hypergraph. The adjacency matrix W ε (0, 1) if either v v' or e' for i = (v,e). v (v',e') V/. In the following sections, we refer the nodes on a line expansion as line-nodes. Notably, node U, on the original hypergraph corresponds multiple line-nodes, each representing a hhyee-h pair involving Us These line-nodes are interconnected and also connect to line-nodes corresponding to the 1-hop neighbors of 2 on the given hypergraph.
graph
1 M is defined by pairwise relation
with W(u, vt)
Incidence tensor. A k-uniform hypergraph can be represented by a k-dimensional supersymmetric tensor such that for distinct node sets (v1,..., rk] e V, T, Ti..... (k-2)! if hyperedge {ui,..., } E, and T in.....aa = 0 otherwise.
C PROOF OF PROPOSITION 3.1
Proof. In this section, we provide proofs for the clique-based, star-based, tensor-based and line-based methods, respectively.
For the elique-based approaches, there are two specific designs for this type of approaches. The first design is based the hypergraph convolution operator (Feng et al. 2019 Yadati et al. 2019 Bai ct al.
2021 Duta et al. 2023 Chen et al.] 2022), which is derived from the hypergraph Laplacian (Agarwal optimization algorithm that minimizes a handcrafted hypergraph energy function. Despite the differences in their designs, these methods start by using an MLP project the given node features to the latent space, namely, for i
al. 2006 The second design (Wang et al. (023b) relies
we have X = Mx,,(x,,) ). Moreover, these methods are analogous to using matrix multiplication between the adjacency matrix of a clique expansion, denoted as W, and the node features Xy to facilitate information exchange among connected nodes on hypergraphs. Therefore, for i Z+, the node-wise form of this type of methods can be summarized as:
x()) fa (x((xx + (((((-))
Σ fa(x((=1)))
VyENGu,
where fo(.), fi(-), fa).. fa(-) represent three learnable functions, and NH.,, is the set of 1-hop neighbours of ii on the hypergraph. Here the right-hand side Eq. is a special case of (1-1)
de (x(" x!! @ujeNH.. ).
For the star- approaches, three primary designs exist for this type of approach. Despite the differences in their designs, these methods start by using an MLP to project the given node features to the latent space, namely, for O. we have x MLP(xx.). Moreover, the first type of methods, as presented in (Huang & Yang 2021), directly applies existing GNNs to a star expansion graph, initializing the features of hypernode Ue, the mean of the features of nodes connected by the hyperedge ey. Due to the linear generation of hypernode features, for i E Z the node-wise wise form of this method can be expressed as (7. Additionally, (Chien et al. (022) integrates set functions with star expansion to create HNNs, while (Wang et 22.23) develops a model by studying the information diffusion process on hypergraph using star expansion. The node-wise forms of the
14

---

Published as a conference paper at ICLR 2025
Table Overview for four state-of-the the- HNNs. In this table, TU 7D. (0, are hyperpa- rameters and I Raxd denotes an identity matrix.
Nane
Typs
Message Pasiing Fanciinn
Hanng Ying
Message Passing
X se (1-10 20 Hi Pi
Doep-HON
Chever al.
Diinct Message Passing
(d-sw)) Hin HiD Ky to (2
SllDeepSes Chien 2023
Indirect Message Passing
x MLP
D Dille (pil ut MiLex)
ED-HNN 2002s
Indinest Missage Pascng
0h
MIP (1-7))
HMLP (Di) MEP(XU -xxx
models in (Chien et al. 2022) and (Wang et al. 2023a) can be summarized as:
x 92 go (x..) + p" 91 (x!. (1-11, @e,eNu.. x, ))-"))
(8)
go(+), (-). 92(+) represent three learnable functions, NH.,, is the set of 1-hop neighbours of o on the hypergraph, and p R is vector used to aggregate the features of hypernodes generated by 9i (+). Here, me, is the number of hyperedges containing et., Here the right-hand side of Eq. (8) is
(1-1) special case of th (xp X
@ eyeNN.n XX1 ((--)).
For tensor-based approaches, these approaches facilitate message passing among connected nodes using the incidence tensor. These methods start by using MLP project the given node features to the latent space, namely, for i= O, we have x2: = Mx.((x.. ). Moreover, as noted by the Theorem 3.3 of (Chien al. 2022), practically, they are implemented by combining set functions with star expansion. Consequently, for Z the node-wise forms of these methods are specific instances of Eq. (8), where the right-hand side is special case of Φe Xx= Xx; OrjENN., Xn; (1-1)).
(0) (l-1)
For based approaches, these approaches Yang al. 2022) generate node features using hyper- graph structural information four steps. Firstly, these methods start by using an MLP to project (0) MLP(x..). Then, the given node features to the latent space, namely, for i == O, we have X the hypergraph is projected onto its corresponding line expansion graph, where the features of each line-node
initialised the features of its corresponding node the original hypergraph. Next, a GNN is applied to this line expansion graph to facilitate message passing among connected line-nodes. Finally, the features of each node e; the hypergraph are generated by aggregating the features of the line-nodes corresponding to v. This method can be summarized as Eq, (8). with three key differences from star-based approaches. Firstly, Nyz,, contains the m-hop neighbors of us which includes all nodes with path to Secondly, the line-based method utilises only one layer for generating node features, with the number of feature aggregation functions usually referring to the number of GNNs used to implement (-). As result, for i E Z we still have (I-1)).
xe, ée Xe,
(!) (0) (1-1)
BryeNno, X
D PROOF OF PROPOSITION 3.2
To recap, the overview of UniGCNII, Deep-HGNN, AllDeepSets, and ED-HNN is in Table g
Before proving Proposition 3.2 we first prove the following Lemmas.
Lemma D.1. Let H {0, 1 be an incidence matrix of a hypergraph, Day e Raxn diagonal matrix with node degrees, Dus pmmm is diagonal matrix with hyperedge degrees,
Σi-1 H,, H,,
D DNS, E Rmmm is a diagonal matrix with D Dug
TU (0, and Wo = (1
DNg,
nu D HD D/2 HT. We have: Wo is the adjacency matrix of a clique expansion.
RV
HE H
Proof. We have:
H H
WU, = =01,=11-0))
Σ Am
1/2 Ryy
D
1/2 Hee
15

---

Published as a conference paper at ICLR 2025
Here Wu;; > if and only if v and by are connected by hyperedge eg H. otherwise W Therefore, Wy is the adjacency matrix of a clique expansion.
Lemma D.2. Let H {0, [0,1) 1) muxm be an incidence matrix of hypergraph, Dxy Ruxa is diagonal matrix with node degrees. DHE Ruxx is a diagonal matrix with hyperedge degrees, "DD (0, 1) and WD (L- (L-7D)D 1/2 expansion.
HD H'D We have: WD is the adjacency matrix of clique HV H u6
Proof. We have:
H. jk 1/2 D D PC
W D.y (1 (1-7D)
Dxg.
Here W D., O if and only if o; and Uy are connected by hyperedge en on H. otherwise W D. Therefore, W is the adjacency matrix of a clique expansion.
Lemma D.3. Let H {0,1) be an incidence matrix of a hypergraph, Dav E R be a diagonal matrix with node degrees, and DHE e mux be diagonal matrix with hyperedge degrees. We have: -1 H is the adjacency matrix of a clique expansion.
Ws D HD 4/
Proof. We have:
Σ H3 DuY Duc,
W S.y
Here Ws., > O and only if i and fy are connected by hyperedge on H. otherwise Ws., Therefore, Ws is the adjacency matrix of a clique expansion.
With the Lemmas above, we present the proof of Proposition 3.2 as follows.
Proof. For an UniGNN (Huang & Yang 2021) with L feature aggregation functions, devoid non- linear activation and by setting the learnable transformation matrix to the identity matrix, its MP. Module can be reformulated as:
L-1 [(-
( ((11 (1-
(L)
1/2 Hy
HD D HE
v0 HJ)
HD
1/2 He
Di H Xv.
1-0
Based Lemma D.I this equation is special of Xv ( (1-a) ta (1-a)
with Xy (1)) W Di HD -1/2
D H
H and a 7V
For a Deep-HGNN (Chen activation and by setting the learnable transformation matrix to the identity matrix, its MP-Module can be reformulated as:
2022) with L feature aggregation functions, devoid non-linear
( (1-7D))
1/2 HV
HD HE
HE
H D HV
ND
1/2
HD H He
D
1/3) |v.
Based Lemma D.2 this equation is special of Xy ( 'W' (1-))
with Xy (L)
D
1/2
HDDH!
D1/2 and 7/-
Re
For an AllDeepSets (Chien et al., 2022) with feature aggregation functions, devoid non-lineat activation and by setting the learnable transformation matrix to the identity matrix, its MP-Module can be reformulated as:
x (L) (D HD H
Based Lemma D.3 this equation is special case of Xy (11-a) EI
with Xy x W = D HV
HD-1
H He
and a O.
For an ED-HNN (Wang et al. activation and by setting the learnable transformation matrix to the identity matrix, its MP-Module can be reformulated as:
2023a) with feature aggregation functions, devoid non-linear
x (1177)) HD H
N Σ (1
TE) (D HD H )
16

---

Published as a conference paper at ICLR 2025
Based on Lemma D.3
this equation is a special case of Xy= ( (L-a)
with Xy ( W D. HD H" and a TE.
(L)
He
E PROOF OF PROPOSITION 3.3
We first introduce some concepts about neighbours on hypergraphs and graphs: 1) The k-th hop neighbours of a node V/ on a hypergraph are exactly hyperedges away from uy and 2) The k-hop neighbours of node 2 neighbours are exactly k edges away from Us. We first introduce some Lemmas.
graph are all nodes with distance of & or less, while the k-th hop
Based on the definition of clique expansion, we present the following Lemma without proof:
Lemma E.1. Let H = EV,E, H) be a hypergraph and g = (V, W) be its clique expansion with self-loops. For any node vi, let Nt., be the node set containing the 1-hop neighbours of Us on H. and Ng., be the node set containing the 1-hop neighbours of U; on G. Then, Ng., =Nit. U(e)).
Moreover, we present the following Lemma about S:
Lemma E.2. Let H = (V,EHD H) be a hypergraph and G = (V, W) be its clique expansion with self-loops. Assume that a [0, 1). and S (1 W + a Ei-o (1 ) /W' Then, for + j. we
have Siz² 0 if and only if U; is L-hop neighhour of 0i on G, and St 0 otherwise.
Proof. According to previous works in graph theory (West et al. 2001), the element W i/ represents the number of walks of length / from node v; to node vy on the graph G. Since W includes self- loops, Wi also accounts for paths where nodes revisit themselves. Hence, for
have W ij > 0 if and only if U; is a l-hop neighbour of vy on 9, and Sij O otherwise. As a result, for = (1- = a El (1 -aa'a'' ''''' and + j. we have Si > 0 if and only if os L-hop neighbour of Uj on G, and Sij- otherwise.
On the basis of Lemmas E. and E.2 the proof of Proposition 3.3 can be transformed into the proof of the following Lemma:
Lemma E.3. Let H = (V, H) be hypergraph and (V, w) be its clique expansion with self- loops. For any node Us let Net, be the node set containing the l-hop neighbours of us on H. and and Ng.. be the node set containing the l-hop neighbours of 1; on G. Then, for Z., Ng., = =Nu.. Ulu).
Proof. We prove this Lemma based on the mathematical induction.
Firstly, for the base case with = according to Lemma E.I this case can be proved by the definition of the clique expansion.
Secondly, for the inductive step, we assume that Nt.,, N U(1}}. Let G.. be the 1-hop neighbours of the k-hop neighbours of v on the graph 9, and Ne, be the 1-hop neighbours of the k-hop neighbours v; on the hypergraph H. Then, we have N Né. UNV and N = Ni.,, U.W... According to the result in the base case, we have N = -Nar! U(v)). By induction, for I Z,, N, =N1 Hug U(v.).
F DISCUSSION ABOUT EQ. (5)
To recap, for a given hypergraph = EV,E, E), the edge weight between is and vy in our clique expansion is defined as:
m 5(1 ex)
W H,, DHC
17

---

Published as a conference paper at ICLR 2025
where 6() is function that returns 1 if ej connects 2 and oy and returns 0 otherwise. In this section, we discuss the relationship between this edge weight and the probability of nodes o and by being in the same category.
We summarise the relationship between this edge weight and the probability of nodes o; and vj being in the same category as the following Lemma:
Lemma F.1. Let Pi,j denote the probability of vs and v; having the same label, P4. represent the probability of vi and v; being connected by a hyperedge that contains only nodes with the same label, and De,, be the probability of containing nodes with different labels. Then, Pig is positively correlated with W WH;, if the following conditions hold:
a) Hyperedges with higher degrees are more likely to connect nodes with different labels, and 1 D EE
Per 91
where 91 (-) is function, for any æ, y > O. with 91 (z) .91 (1 y).
dei and g1 (x) E (0,1), dæ
b) Pig is positively correlated with pl.g. namely, Pcs P(5=92(1(s) where 92(-) is a function for any > with dg2 and 92(x)) (0,1).
dæ
Proof. According the pre-defined condition a), have: 8i( --1-)-1---------- gi Σ )-11)-11--
-1--11 n Be, -1-
)- -s (W Way.). (9)
D
Duee
where Ex, is set containing hyperedges connecting vs and Vi. Based on Eq. condition b) we can have:
and the pre-defined
P = g2(P(,) 92((1 91 (WH,)).
(10)
The derivative of Eq. I0 with respect to W H., is:
d dp..j dg dg dp
dg = = dg2 dgi
W H. d aWn, n. d d dg d 1W
Haj
d
W Hツ
On the basis of our pre-defined conditions a) and b), have have:
dez
0 and
d a aWag
0, thereby
art.s
dp dph,
dg2
dg
0.
W Hiy
dW Hij
As result, Pı,j is positively correlated with WH.g-
G PROOF OF PROPOSITION
Before proving Proposition we first present two Lemmas.
Lemma G.1. Let Ho denote the entropy of the information that HNN utilises by L feature aggregation function Φe introduced in Propostion for generating features of and H denote the entropy of information within node U; and its L-hop neighbours on the hypergraph. Then, 11" H
Proof. Based on the mathematical induction, we start by proving this Using mathematical induction, we first prove this lemma for the case where Nn., includes only the 1-hop neighbours of v, which applies to all models except the line-based models discussed in Appendix
Firstly, for the base case with L = 1, the message passing layer for generating the features of o can be reformulated as:
x R1 (1) = de (x (0)
x
@ Bu,eNRa,
(0)) By
(11)
where NH., is set containing the 1-hop neighbours of on the hypergraph. Hence, the entropy of the input information utilised to generate (1) H namely, we have H H
Secondly, for the inductive step, we assume that H¹ 11t Moreover, we have the feature generation function as:
(k+1) de (x(() (x) , @ojeNnnn x (k)). x
(12)
18

---

Published as a conference paper at ICLR 2025
(k) Based on the assumption about L = k. for L k 1, DryeNn., X1y contain the information from the (k+ + 1)-hop neighbours of v; and x contain the information from the k-hop neighbours uy
Let "o, denote a set containing I-hop neighbours of U,. Based on information theory (Kullback
1997.. we have the information entropy N is zero with the information of W is given, as
Not S
1++1_ As result, based on the feature aggregation function introduced in Eq. 12 we have
Heg
Hoy
H H
By induction, for L E Z,, Hij H2 .Nw, conduct the proof for the line-based models.
Let H denote the entropy of the information contained by the T.-hop neighbours of the line-nodes corresponding to vi. According to previous papers the GNN literature (Wu et al. (2020), the entropy of information used by an L-layer GNN to generate the features of line-nodes corresponding U;
is H" Based the discussion in Appendix B and Proposition 3.1 we have H H By definition, the line-nodes corresponding to e are only connected to each other and the line-nodes
corresponding to the 1-hop neighbours of vs Therefore, we have H -12 Accordingly, we have 11" 112
Lemma G.2. Let H1 denote the entropy of the information that TF-HNN with an L-layer TF-MP- Module utilises to generate features of vi and H denote the entropy of information within node Ui and its L-hop neighbours the hypergraph. Then, H -H2
Proof. For the feature generation of U4 L-layer TF-Ml-Module can be directly reformulated
Se, fto (Xx:) fi, (1, (#,,eN Xuy):
(13)
where Ji, and fi are two linear functions, and No. is a set containing L-hop neighbours of w; on the hypergraph. Hence, have the entropy of the input information utilised by an L-layer TF-MP-Module for generating features of is H namely, 11 =1,
After having the Lemmas above, we prove Proposition 4.1 follows:
Proof. Based on Lemma G.1 we have Ho H Moreover, according to Lemma G.2 we have L H H. As result, Ho H H.
H PROOF OF PROPOSITION 4.2
Before proving Proposition we first prove the following Lemma.
Lemma H.1. Let H (V, EV,E, H denote hypergraph, G (V, WH} be its clique expansion with edge weights computed by Eq. ()) L be the graph Laplacian matrix of computed by symmetrically normalised and self-loops added W H, Xv represent the input node features, and E (0, 1). We
set F(X)= = tr(X LX)+
I-a tr[(X- -xv) (X-(XXXX)) Xx)], and X as the global minimal point for
F(X). Then, we have:
X X'= (I= WWD -1 where WH WH In and DH Eaxn is the diagonal node degree matrix of W H.
axv,
(14)
Proof. Taking the partial derivative of F with the respective of X, we have: aF
2a
2LX
1-a (X- -Xx).
0X
By setting ar Xx
0. have:
X" = (L +
)-1 Xy
In, - (1-a)D
a 1/2 1/2 Wn 11111 axv,
1 1-a a
1 1-a
19

---

Published as a conference paper at ICLR 2025
where WH WH I, and Dy B is the diagonal node degree matrix of WHr
Since the second-order derivative F with the respect X is:
8 2L
20
1-a In-
(15)
0X
Given L IR= is graph Laplacian matrix, which is positive semi-definite, and, for (0, 1). 2a: I IR is diagonal matrix, which is positive definite. Accordingly, we have the righthand
I-a
side of Eq. 15) is positive definite matrix for a (0, 1). As result, for é (0, 1), X' in Eq. 14
the global minimal point for F.
With the Lemma above, we present the proof of Proposition 4.2 follows.
Proof. Let H EV,E, H} denote a hypergraph and G (V, W W be its clique expansion with edge the output an L-layer TF-MP-Module can be
weights computed by Eq. 5 According to Eq. 66. reformulated as:
I-1 (D 1/2 // (PWDD '',,
Xy ((L (1--a)
1/2 W/ W 133
where WH WH + I, and D и R is the diagonal node degree matrix of W H- When L +00, the left term tends to and the right term becomes a convergent geometric series (Meyer & Stewart 2023 Hence, for L +00, this equation can be further represented as:
Xy
(In (1- (1-a)D 1/2 W WD
aXv.
Based on Lemma H.l we have F F(Xv)= Fmin-
I DETAILS OF BENCHMARKING DATASETS
Our benchmark datasets consist of existing six datasets (Cora-CA, DBLP-CA, Citeseer and House from (Chien et al. 2022), Congress and Senate from (Wang et al. 2023a)). co-citation networks (Citeseer), all documents cited by particular document are connected by hyperedge. For co- authorship networks (Cora-CA: DBLP-CA), all documents co-authored by a single author are grouped into one hyperedge. The node features in these co-citation and co-authorship networks using bag-of-we ofooosswords models of the corresponding documents, with node labels corresponding to the paper classes. In the House dataset, each node represents member of the US House of state-oftthe- arts, with hyperedges grouping members of the same committee. Node labels denote the political party of the state-of-the-arts. In the Congress dataset, nodes represent US Congresspersons, with hyperedges linking the sponsor and co-sponsors of bills introduced in either the House or the Senate. In the Senate dataset, nodes also represent US Congresspersons, but hyperedges link the sponsor and co- oo-sponsors of Senate bills only. Nodes in both datasets are labeled by political party affiliation.
represented
J ADDITIONAL EXPERIMENTAL RESULTS
Table 9: The hyperlink prediction AUC (%). The best result is in bold font.
CA
DBL
Congress
Avg.
AllSet
95.31 0.44
93.77 2 1.01
93.98
71.56 0.07
80.2% 1 3.33
63.56 1.15
83.78
PhenomnNN
0.92
0.34
13022
1.98
82.26
ED-HNN
97.61 0.83
95.71
#2110
96.04 0.85
7
81.09
+.47
68.96
1.79
84.64
Deep-HGNN
0.50
0.118
96.45
7
78.84
62.06
3.13
83.88
HINN (ouns)
97.87 a 0.65
97.96
17.9622.. 2.43
96.95
72.33 0.14
82.18 33.70
65.04 0.97
85.39
Table 10: The training time (s) in hyperlink prediction. The best result is in bold font.
Cora-CA
DBLF-CA 64.07 x 5.75
Citesee
Coegress
House $1.76
Senate
Avg. Mean
AllSet Trandommer
2088 5.27
V.18
2086
244.37
13.03
33.15
8:26
377.58
PhenomNN
365
144.07
190.22 18.67
30.99
266.19
Deep HGNN
12.31 2.72
107.97 37.50
8.49 1.68
795 25 173.54
36.33 7.74
7.82 1.32
161.36
TF-HNN (ours)
3.22 1.99
29.36 6.31
1.75 1.26
501.66 111.99
17.83 4.98
3.19 1,31
93.00
20

---

Published as a conference paper at ICLR 2025
Table 11: The node classification accuracy (%). The best result is in bold font.
Coma-CA
D.A.P.C
Clleseer
House
Senate
Vean
HyperGCL HDS
83.02 # 1,36
91.12 = 0.28
70.70 = 1.77
94.30 0.75
65.25 + 7.3
48.76 4.73
75.53
85.60 1.07
91.55 0.33
73.68 = 1.99
90.58 + 1.72
71.58 = 1.60
59.72 =
78.79
TF-HNN (ours)
86.54 1.32
91.80 0.30
74.82 * 1.67
95.09 + 0.89
76.29 1.99
70.42 + 2.74
82.50
Table 12: The training time (s) in node classification. The best result is in bold font.
Cora-CA
DBLF-CA
Chicser
Congress
House 108 + 6.38
Scnale
Avg. Mean
HyperCICL
33.94 + 3.86
722.25 137.08
7.94 1,72
1384.02 + 160.35
14.67 + 1.25
378.48
HDS
+
TE-HNN (ours)
0.22 a 0.12
4,39 0.45
1.12 a 0.30
0.98 0.35
1.01 # 0.51
0.19 = 0.10
1.32
K
NEGATIVE SAMPLING FOR HYPERLINK PREDICTION
We adapt the algorithm from (Chen & Liu 2023) for this purpose. For each (positive) hyperedge e F,, we generate a corresponding negative hyperedge f. where a x 100% of the nodes in / are from and the remaining genuineness of the negative hyperlinks, i.e., higher values of a indicate that the negative hyperlinks closer to the true ones. Additionally, define be the number of times of negative sampling which controls the ratio between positive and negative hyperlinks. In practice, we set a = 0.5 and 3 5.
from Ve. Denote the negative hyperlink set F. The number controls the
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our results, provide detailed hyperparameter settings used in experiments. The configurations for the TF-HNN model on the hypergraph node classification task and the hyperlink prediction task listed below.
Table 13: Hyperparameter settings for TF-HNN in Table 3 and Table
Datasel
TF-HNN Layers
MLP Layers MLP Hiddee Dimcessoo
Leaming Rote
Alpha
Dropout
Weight Devay
ConyCA
2
1024
0.001
0..
0.7
0.0
P-C.
1024
D.7
Citeseer
0.9
House
16
128
0.7
0.9
0.0
Cregress
1024
0.001
0.05
08
D.D
Senate
512
0.005
0.5
Trivago
In practice, we conduct the hyperparameter search based on grid search. For the hypergraph node classification task listed in Table we apply grid search to the following hyperparameters:
The number of layers of ModuleModule: (1,2,4,8, 16).
The of TF-MP-Module: {0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35 0.45, 0.5, 0.55, 0.6, 0.65,.666..0 0.7).
The number of layers of the node classifier: (3)
The hidden dimension of the node classifier: (128, 256, 512, 1024).
The learning rate for node classifier: {1 14-4, x 10 10 4,4x 10 1x 10
10-4.8 888x x 10-3.9 .9 10-4, 10-3, xx 10-3).
The dropout rate for node classifier: {0.5, 0.6, 0.7, 0.8, 0.9).
For node classification listed in Table 4 we apply grid search to the following hyperparameters:
The number of layers of TF-MP-Module: {1,4, 16, 64).
The of TF-MP- {0.0, 0.1. 0.5).
The number of layers of the node classifier: (3).
The hidden dimension of the node classifier: {64).
The learning rate for node classifier: {1 1 1 10-4).
The dropout rate for node classifier: (0.0,0.2,0.4).
For hyperlink prediction listed in Figure 3 we apply grid search to the following hyperparameters:
21

---

Published as a conference paper at ICLR 2025
Table 14: Hyperparameter settings for TF-HNN in Figure
and Table
Dataset
TE-HNN
Predictor Layers
Predictor Hidden Dimension
Learning Rate
Alpha
Dropont
Cora-CA
64
3
256
0.0005
0.5
0.0
Cincseor
256
0.0005
House
128
0.0002
0.0
Trivago
0.0001
0.5
The number of layers of TT-MP-Module: {1,2, 16, 32, 64).
The of TF-MP-Module: 0.00, 0.05,
The number of layers the hyperedge predictor: (3).
The hidden dimension of the hyperedge predictor: {128, 256).
The learning rate for node classifier: {1 x 10 2 10-4, 3 10 4x 10 10-4).
The dropout rate for node classifier: 0.20, 0.4, 0.6, 0.8).
For hyperlink prediction listed Table 5 we apply grid search to the following hyperparameters:
The number of layers of TF-MP-Module: (1,2,4)
The a of TF-MP-Module: (0.0,0.1, 0.3,
0.1,
The number of layers of the hyperedge predictor: (3).
The hidden dimension of the hyperedge predictor: {64].
The learning rate for node classifier: {1 10 1 10-4).
The dropout rate for node classifier: (0.0,0.2).
The experiments on Trivago are highly time consumcnn and need large GPU memory. We used relatively small search space for hyperparameters ours and the baselines, which are provided below.
Node classification on Trivago of TF-HNN:
The number of layers of TF-MP-Module: (1,4, 16, 64).
The of TF-MM-MMddul {0.0, 0.1,0.....3,000 0.5..
The number of layers of the node classifier: {3).
The hidden dimension of the node classifier: {64].
The learning rate for node classifier: {1 10-1 10-3).
The dropout rate for node classifier: 0..0, 0.4).
Node classification Trivago of ED-HNN:
The number of layers of ED-HNN: (1,2,4) 4) (larger values show the out-of-memory (OOM) issue server).
The number of layers of è. and MLPs within message passing: 10, (larger values show the out-of-memory (OOM) issue on our server).
The number of layers of the classifier: (3).
The hidden dimension of è. A. and G:: (64, 128) (larger values show the out-of-memory (OOM)
issue on our server).
The hidden dimension of the classifier: {64).
The learning rate for the model: {1 10-4 1,1 10-3).
The dropout rate for for the model: {0.0, 0.2,
Node classification on Trivago of Deep-HGNN:
The number of message passing layers: (1,4, 16, 64).
22

---

Published as a conference paper at ICLR 2025
The a: (030.1,033,000 0.5).
The A: {0.1}.
The message passing hidden dimension: {64, 256, (larger values show the out-of-memooy (OOM) issue on our server).
The number of layers of the classifier: (3).
The hidden dimension of the classifier: (64).
The learning rate for the model: {1 10-4 x 10-3).
The dropout rate for for the model: {0.0, 0.2, 0.4).
Hyperlink prediction on Trivago of ours:
The number of layers of ToMMe-Moull: (1,2,
The of TF-MP-Module: {0.0, 0.1,,0.,,,33, 0.5).
The number of layers of the predictor: (3).
The hidden dimension of the predictor: (64].
The learning rate for the predictor: {1 10-1 1 10-33.
The dropout rate for the predictor: {0.0,0z 0.2).
Hyperlink prediction on Trivago of ED-HNN:
The number of layers of ED-HNN: (1,2,4)
The number of layers of Φ. A. and Φ, MLPs within message passing: {0,1,2)
The number of layers of the predictor: (3).
The hidden dimension of Φ. p. and Φ: (64, 128).
The hidden dimension of the predictor: (64).
The learning rate for the model: {1 10-4 x 10-3).
The dropout rate for for the model: 0.2)..
Hyperlink prediction on Trivago of Deep-HGNN:
The number of message passing layers: (1,2,
The a: (0,0.1, 0...).
The A: {0.1).
The message passing hidden dimension: [64, 128, 256).
The number of layers the classifier: (3).
The hidden dimension of the classifier: (64).
The learning rate for the model: {1 10-4 1 x 10-3).
The dropout rate for for the model: {0.0, 0.2).
M EFFICIENCY IMPROVEMENT FROM TF-HNN
In this section, we theoretically discuss that the efficiency improvement provided by TF-HNN is inversely correlated with the complexity of the task-specific module. Specifically, we prove a Lemma.
Lemma M.1. Let M denote the training complexity of the task-specific module and, J denote the training complexity module, we approximate its training complexity with the function t.(M) = M. Similarly, we
HNN. Since an TF-HNN only requires the training of the task-specific
approximate the training complexity of an HNN with the function M, J) M + J. Furthermore,
23

---

Published as a conference paper at ICLR 2025
We quantify the efficiency improvement brought by TF-HNN using the ratio r(M,J) (a(M))
(,(M)
where a larger J) indicates a greater efficiency improvement provided by the TF-HNN. Then we have Or < O.
3M
Proof. We can reformulate the J),
M+J
r(M,J)
M
Taking the partial derivative of with the respective of M, we have:
Ör
J
< O.
0m
M
N ADDITIONAL DISCUSSION AND RESULTS ABOUT APPNP
The APPNP (Gasteiger et al. 2019) has contributed to the development of more efficient GNN models by shifting learnable parameters to an MLP prior to message passing. In this section, we aim clarify the differences between the APPNP method proposed in [1] and our TF-HNN in two different perspectives.
Methodological Design: The key difference in the model design is that APPNP requires running the message passing process during model training, whereas our TF-HNN performs message passing only during data preprocessing. According to Eq. (4) in [1], APPNP employs a message-passing block after multi-layer perceptron (MLP). The MLP uses the given original node features to generate latent features for the nodes, which then serve as inputs for message passing. This design requires the message- passiggggggingggg block to be executed during each forward propagation in the training phase, making it incompatible with preprocessing. In contrast, the training-fres module in TF-HNN directly takes the given original node features as inputs, enabling it to be fully precomputed during the data preprocessing stage, thereby eliminating the need for computation during training, which further enhances the training efficiency. Notably, Table 2 below demonstrates that TF-HNN is more training efficient than an APPNP with clique expansion. As a result, we argue the APPNP in [1] does not diminish the core novelty of our TF-HNN, which is the first model to decouple the message- saaagggg-pssinggggee operation from the training process for hypergraphs.
pesssae--assingg
Practical differences: In Tables 15 and 16 empirically demonstrate that, for hypergraph- structured data, TF-HNN is both more training efficient and more effective than APPNP with the clique expansion defined in our Eq. (5). For training efficiency, since APPNP requires message- passing computations during each forward propagation step in training, its training time is at least times longer than that of TF-HNN. For effectivenesss our results show that TF-HNN, which performs message passing in the original feature space, achieves better performance compared to APPNP, which projects features into a latent space by an MLP before message passing. We hypothesize that this improvement may be attributed to the MLP in APPNP, which may not be able to fully retain information from the original node features during the projection to the latent space.
Besides the two key differences between APPNP and our TF-HNN above, we also hope to emphasize the unique theoretical contribution of our paper for hypergraph machine learning- While the mechanism of graph neural networks (GNNs) is well-studied, the foundational component of HNNs is not clearly identified until our work. In Sec. 3.2, we identify the feature aggregation function as the core component of HNNs, and show that existing HNNs primarily enhance node features by aggregating features from neighbouring nodes. This insight encompasses various HNN models and provides researchers with deeper understanding of the behaviour of existing HNNs.
As a result, while APPNP represents a significant contribution to the field, it does not diminish the unique value and contributions of TE-HNN for hypergraphs.
24

---

Published as a conference paper at ICLR 2025
Table 15: The node classification accuracy (%) for TF-HNN, and APPNP+CE. The best result is in bold font.
Table 16: The training time (s) for TF-HNN and APPNP*CE corresponding to results in The best result in bold font.
Table T5
Methods Datasets
Cora-CA
Citeseer
Methods Datasets
Cora-CA
Citeseer
TF-HNN
86.54 1.32
74.82 + 1.67
TF-HNN APPNP+CE
0.22 = 0.12
1.12
APPNP+CE
86.01 = 1.35
74.32 + 1.57
0.73 + 0.16
4.61 0.88
Table 17: Results for baselines with/without learnable parameters in node classification. The best result on each dataset is in bold font (+LPI-LP means the model
learnable parameters).
Datuset
+LP
AllDeepSets
-LP
+LP
UnKINN
-LP
+LP
BD-HNN
-LP
+LP
Deep-HGNNN
-LP
Cora-CA
81.97 1,50
82.53 0.70
1.14 84.82 1.56
83 23,70 1.38
1.55 85.69 74,08
84-89
86.22 34.07
Citeseer
70.83 1.63
73,05 + 2,21
73.96 1.67
74.64
O RESULTS ABOUT REMOVING LEARNABLE PARAMETERS FROM BASELINES
In Table 77 present experiments that remove only the learnable parameters from the node feature generation process in the baselines. In this setup, all the baselines studied in Eq. (4a) to Eq. (4d) (AllDeepSets, UniGNN, ED-HNN, Deep-HGNN) directly apply non-learnable message passing to the original node features. These results confirm that removing the learnable parameters and directly applying message passing on the original given node features can enhance the model performance. These results, together with the ablation study on the weight design for S presented in Table 7 of our submission, demonstrate that both the removal of learnable parameters and the design of our S, contribute to the effectiveness of our TF-HNN.
P ADDITONAL RESULTS ON YELP
In this section, we conduct experiments on a dataset named Yelp from (Chien et al. 2022). and the results are summarized in T8ble For HGNN, HCHA, HNHN, UniGCNII, DeepesS and ArnSerTransformer, we directly use the accuracy reported in (Chien et al. 2022) and record the training time by running the models with the hyperparameters provided in (Chien al. 2022) on our RTX 3090 GPUs. For ED-HNN and Deep-HGNN, we report both accuracy and training time based on runs using the optimal hyperparameters we identified. Due to out-of-memory issues with PhenomNN on our RTX 3090 GPUs, we did not include it in the table. The results demonstrate that model is both effective and efficient on this dataset. Specifically, our model achieves a general second-best performance among the baselines, with accuracy only 0.83% lower than AllSetTransformer, while being 24 times faster. These findings highlight the effectiveness and efficiency of our TF-HNN.
Q THE ASSUMPTION ABOUT THE STRUCTURE OF HYPERGRAPH
In Sec. we assume that the hypergraph does not contain isolated nodes or empty hyperedges to maintain consistency with prior works Huang & Yang (2021): Chien et al. (2022); Kim et al. (2022); Chen et al. (2022) in the literature. This assumption Is based on the observation that most existing hypergraph neural networks (HNNs) use node degree or hyperedge degree as denominators during forward propagation. This inherently presumes these values are nonzero-ia i.e., there are no isolated nodes or empty hyperedges- since division by zero is undefined. Moreover, we emphasize that this assumption does not compromise the practical applicability of our method. Consistent with prior works, address scenarios where zero degrees occur by assigning default value of 1 to such cases.
R ADDITIONAL DISCUSSION ABOUT THE SUPERIOR PERFORMANCE, ESPECIALLY ON THE LARGE-SCALE HYPERGRAPH
Generaly, we attribute the superior performance of TF-HNN primarily to its more efficient utilization of node information from the training data compared baseline models.
Baseline models with training- required message- paaagggg modules process node information by first using trainable module project node features into latent space, and then performing trainable
25

---

Published as a conference paper at ICLR 2025
Table 18: Results on Yelp. The best and second-best results are marked with bold font and underlined. respectively.
HONN
HCHA
HNHHN
UnIOCNII -
AllDeopSets
AllSeTTranffmmr n.51
ED-HNN 35.00 0.53
Deep-HONN
TI-HNN
Accurey
35.04 2.64
32615
147.54
00.25
5087
2
2 28.52
message passing within that space. Learning a latent space that preserves the unique characteristics of individual nodes is extremely challenging, especially for large-scale hypergraphs. This difficulty is compounded by computational constraints: both the projection and pesssage-passing operations require computing gradients for backpropagation. which are extremely resource-intensive To prevent out-of-memory (OOM) errors during experiments on the large-scale hypergraph Trivago, the hidden dimensions of these baseline models need to be limited to maximum of 256. This dimension constraint, introduced by GPU memory limitations, makes it difficult for the model to fully preserve information that is helpful for classifying a large number of nodes.
In contrast, TF-HNN performs message passing directly on the original node features during the data pre-processing stage without requiring any trainable parameters. This allows TF-HNN to preserve node-specific information without incurring the expensive memory requirements associated with training-requiinn models. Notably, our experimental results align with previous observations in the Graph Neural Network (GNN) literature. For example, training-free models like SIGN (Frasca et al., 2020) have been shown to outperform training-required models like GCN (Kipf & Welling (017)) on large-scale graphs such the Protein-Protedd Interaction (PPI) network by significant margin.
Table 19; Hyperparameter search time for ED-HNN and TF-HNN in node classification on Cora-CA.
ED-HNN
TF-HNN
Ins/tep
Runtime for Completing the Grid Search (hours)
48.95
2.74
0.06
Table 20: Hyperparameter search time for Deep- ED-HNN and our TF-HNN in the node classification task on Trivago (The search time of ED-HNN and TF-HNN on Trivago is shorter than the one on Cora, as the search space used on Trivago is much smaller than the one used on Cora).
Desp-HGNN
ED-HNN TF-HNN
Hyperparameter Searching Time (hours)
59.61
30.42
S HYPERPARAMETER SEARCH
Our hyperparameter tuning process strictly adheres to standard practices using the validation set to determine optimal settings. Following the codes of prior works (Chien et al. 2022] Wang et al. a), our code records the best performance on the validation set, the training epoch that achieved the best performance on the validation set and the performance of this recorded training epoch on the test set. The hyperparameters yielding the best results on the validation set were adopted as the final settings for our model. Furthermore, using the node classification task DBLP-CA we conducted a detailed hyperparameter sensitivity analysis. The heatmaps in Figure 5
case study, illustrate validation and test performance across various hyperparameter configurations. The hyperparameters yielding the best results on the validation set highlighted with a green rectangle were adopted as the final settings for our model. Based on these results, we derive three key insights for hyperparameter search in TN-HNN: 1) The model exhibits minimal sensitivity (performance variations within 1%) to the following combinations, suggesting they can be tuned independently: MLP Hidden Dimension & Dropout, MLP Hidden Dimension & TF-HNN Layers, Learning Rate & Dropout, Learning Rate & MLP Hidden Dimension, Learning Rate& & TF-HNN Layers, TF-HNN Layers & MLP Layers, and TF-HNN Layers& & Dropout; 2) The MLP performs well with relatively few layers, reducing the need for deeper architectures; and 3) The following combinations require more careful tuning due to their greater impact on performance: Alpha & MLP Layers, Alpha & Dropout, Alpha & MLP Hidden Dimension, Alpha & Learning Rate, and Alpha & TF-HNN Layers.
Generally, the efficient search for the optimal hyperparameters is challenging within the hypergraph machine learning literature. However, with the fast training speed of TF-HNN, our hyperparameter searching process is still efficient compared with previous methods. Table 19 shows the hyperpa- rameter searching time used for TF-HNN and ED-HNN (Wang et al.) (023a) in node classification on the Cora-CA dataset in our 8 RTX 3090 GPUs server. For TN-
the hyperparameter
26

---

Published as a conference paper at ICLR 2025
(a) Alpha & MLP Layers.
(b) Alpha & Dropout.
(c) Alpha & MLP Hidden Dimen- sion.
(d) Alpha & Learning Rate.
Alpha & TF-HNN Layers.
(f) MLP Layers & Dropout.
MLP Hidden Dimension & MLP (h) MLP Hidden Dimension & (i) MLP Hidden Dimension & TT- HNN Layers.
Layers.
Dropout.
ii) Learning Rate& & MLP Layers. Leamnnng Rate Dropout.
(1) Learning Rate & MLP Hidden
Dimension.
(m) Learning Rate & TF-HNN Lay-(n) TF-HNN Layers & MLP Layers. (0) TF-HNN Layers & Dropout. ers.
Figure 5: Heatmaps of validation and test performance under varying hyperparameter combinations.
combinations mentioned in Appendix L and for ED-HNN, we run the hyperparameter combination mentioned in Appendix F.3 of their paper. Specifically, according to the discussion and table 6 in Appendix F.3 of Wang et al. (2023a), ED-HNN does grid search for the following hyperparameters:
The number of layers of ED-HNN, whose search space is 4,,2,4,6,8)
The number of layers of MLP projector, à. whose search space is 40,
The number of layers of an MLP projector, é. whose search space is (0,1,2,4,6,p
The number of layers of an MLP projector, + whose search space is 20,1,2,4,6,8),
The number of layers of the node classifier, whose search space is (1,2,4,6,8).
The hidden dimension of J. c. and م. whose search space is (96, 128, 256, 512).
The hidden dimension of the node classifier, whose search space is {96, 128, 256, 512).
Based on the results presented in Table 19, our hyperparameter search process requires only about 6% of the time required by ED-HNN, demonstrating its efficiency. We treat develop more efficient
27

---

Published as a conference paper at ICLR 2025
hyperparameter search method for hypergraph neural networks based on the methods like Bayesian
Optimisation (Victoria Maragatham 2021 Wu et al. 2019b Nguyen 2019) as a future work.
T DISCUSSION ABOUT PROPOSITION 3.2
We summarise the connection between the clique expansion mentioned in Proposition 3.2 and the clique expansion defined in Eq (5) as the following Lemmas:
Lemma T.1. For both the W of the clique expansion mentioned in Proposition 3.2 and the one
defined in Eq (3). we have Waz 0 if and only if uz and Vy are connected on the given hypergraph
and W ij = otherwise.
Proof. Here we first prove this lemma for the clique expansion mentioned in Proposition 3.2
Based on the definition of clique expansion and Lemmas D.I D.2
D.3
we can directly conclude
that, for the weight matrix W of the clique expansion mentioned in Proposition 3.2 Wiy > 0 if and only if U, and fj are connected on the given hypergraph and W 0 otherwise.
For the W defined in Eq 50. we have:
m. (01,,,,,
Wiy = Σ
DzC
where 5(-) is function that returns 1 if connects es and i and returns otherwise. Since m], (15,, Vj ex) O, namely, and By are not connected on the hypergraph. Further, we can have Wi, > if and only there exist ke [1,2,3, "., m] to make 0(v;, 6x) 1, namely, vi and uj are connected the given hypergraph.
Duc. > 0, we have Wi 0 if and only if, AA
Lemma T.2. Let F. = ",,",, denote the set of hyperedges connecting v; and Vj- Then, for both the clique expansion mentioned in Proposition 3.2 and the clique expansion defined in Eq (5)
we have d W.2 0, Vea E, where |ex] represents the size of ex-
d
Proof. Here we first prove this lemma for the clique expansion mentioned in Proposition 3.2
Based on the discussion in Sec. D for UniGNN (Huang Yang 2021),
we have
Wiy = (1-70)
H H jk
(1 )V) Ex=1 H
Duy
Hig Hik
Σ
Wk
112
D 1/2
D 1/2 H/
|/2
HE
k-1
HY
E
where W/ (1-70)se and 8q is the sum of degrees of nodes with eg. Since w/ > and the design of 1/2
xy
UniGNN only considers the positive square root, we have:
di
Wh
0.
dlei
2/ex 3/2
Based on the discussion in Sec.
D for Deep-HGNN (Chnn et al.
2022).
we have:
m
H jk
(1-
TD)
m
H H
Wij = (1 7D)
1/2 Hx
1/2 Hy
Dug
1/2 Hy #y
D 1/2
k=
DHg.
=
Σ
|ex
where w (1-9D)
Since 0, have:
D//2 D 1/2-
W
<..
d d
le 2
Based on the discussion Sec. 2023a), we have:
D for AllDeepSets CChien al. 2022 and ED-HNN (Wang et al..
W
H
jk
1
H H
Σ
D Day
D DMc.
D
DaY
D
Duc
leal
exeE
28

---

Published as a conference paper at ICLR 2025
where
Since O, we have:
Duy
d Waj <0. d del
leal2
Finally, we demonstrate the proof for the clique expansion defined in Eq (5
o(v;, En)
We
DuC.
EEE
Ck
Then, we have:
We 1
O.
d d|ex
|ex|
U DISCUSSION ABOUT PROPOSITION 4.1
In Proposition 4.1, the "information" refers to the contextual information embedded in the node features, while "information entropy" denotes the entropy of this contextual information. Typically, node features are generated based on specific contextual information. For example, in co-citation datasets where nodes represent research papers, the features of node are often derived from keywords or sentences in the paper abstract. Consequently, the contextual information embedded in such node features corresponds to these keywords or sentences. According to prior works in Linguistics (Shannon [951] Genzel & Charniak 2002). the information entropy of these keywords or sentences can be computed using the Shannon entropy formula: H (X) = Σxex ((x) log p()), represents the keywords sentences, denotes token within X, and p(x)) can be defined using various methods, such as an n-gram probabilistic model mentioned (Genzel & Charniak 2002).
While did not compute the exact Shannon entropy here, refer to the amount of information helpful for the downstream task. From this perspective, the key takeaway from the proposition is to highlight that both HNN with an I.-layer feature aggregation function and TF-HNN with an L-layer TF-MP-Module can leverage the hypergraph structure to enhance the features of each node by aggregating features containing contextual information from its L-hop neighbouring nodes. For instance, in a co-citation hypergraph. if within L-hop node is connected several nodes with features containing contextual information about machine learning and others with features related to biology, the feature aggregation process will incorporate both machine learning and biology related features into the node's 's features.
"entropy" in conceptual way to
V ADDITONAL RESULTS ABOUT THE PREPROCESSING TIME
Table 21: The training time (s) and preprocessing time (s) for TF-HNN in node classification.
Cora CA 0.12
DBLP-CA
Ciesseer
Congress
House
Senate
Avg. Mean
Traiing
1
1 1.30
0.98 0.35
1.0 0.51
0,19 1 U.
1.32
Preprocesing
00002
0,0080 0.0001
Let be the number of edges of the clique expansion used in TF-MP-Module then the theoretical complexity of this module is ((m). Moreover, unlike the training-required message- passing operators used in existing HNNs, which must be computed during forward propagation in each training epoch and require gradient descent computations for backpropagation, our training free gressage-passingg operator only needs be computed once during single forward propagation in preprocessing. Thus, in practice, the training-free message-passinn operator is quite fast. We summarise the runtime Table 2T These results indicate that the average preprocessing time of our training-for free message-passing operator is about 1 % of the average training time for our TF-HNN.
29


Reviews:
[{"review_id": "LLtcHv9jqw", "reviewer": "Reviewer_uQFu", "summary": "The paper proposes TF-HNN, a training-free hypergraph neural network that removes the need for computationally intensive message passing during training. By shifting hypergraph structural processing to the preprocessing stage, TF-HNN reduces computational complexity. The model achieves efficient, robust node feature generation without oversmoothing, utilizing as much structural information as traditional HNNs. Experiments show that TF-HNN outperforms state-of-the-art HNNs in accuracy and training speed, especially on large-scale benchmarks.", "questions": "See weaknesses.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "1. The paper is well-structured and easy to follow.\n2. The summary of hypergraph neural networks is comprehensive, particularly with the insights provided in Table 1 and the related analysis.", "weaknesses": "Major weaknesses:\n1  Proposition 4.2 shows the similarity between the proposed method and APPNP [1], yet the paper does not cite APPNP. Specifically, Eq (6) and Eq (14) appear to apply APPNP after performing clique expansion on the hypergraph. This connection should be discussed more thoroughly, referencing APPNP's Eq (7) and Eq (8) to clarify the relationship and implications of this similarity in the context of hypergraph learning. While the experimental results demonstrate superiority, this oversight is a significant limitation on the paper's originality.\n\n2  Equations (4a) to (4d) result from removing learnable parameters from several baselines, corresponding to the different scenarios in the authors' proposed framework. While these modifications reasonably reduce training time, there is insufficient ablation study to explain the performance improvements. The authors' comparison of the impact from the weighted $S$ of TF-HNN in node classification is commendable. However, they should also present the performance of these modified baselines to clearly demonstrate the impact of removing the learnable parameters.\n\nMinor weakness:\n\nChien et al.'s work [2] provides the challenging YELP dataset, where many baseline methods yield unsatisfactory performance. Including results on this dataset would enhance the paper's quality and offer a more comprehensive evaluation of the proposed method.\n[1] You are allset: A multiset function framework for hypergraph neural networks. Chien  et al. ICLR 2022 \n\n[2] Predict then Propagate: Graph Neural Networks meet Personalized PageRank\nGasteiger et al. ICLR 2019", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "t0aMKyLCvE", "reviewer": "Reviewer_GqRw", "summary": "The paper proposes a novel approach called TF-HNN (Training-Free Hypergraph Neural Network) to address the high computational complexity during training in existing hypergraph neural networks (HNNs). The key innovation is a training-free message passing module (TF-MP-Module) that decouples the processing of hypergraph structural information from the model learning stage. The authors first derive a theoretical framework that provides a unified view of existing HNN approaches, identifying the feature aggregation function as the core component processing hypergraph structure. Based on this insight, they remove the learnable parameters and non-linear activations from the feature aggregation functions of four state-of-the-art HNNs to make them training-free. Further, they consolidate the feature aggregation across layers into a single propagation step, resulting in the proposed TF-MP-Module. Extensive experiments on seven real-world datasets for the tasks of node classification and hyper-link prediction demonstrate the competitive performance of TF-HNN, with very less training time. TF-HNN is the first approach to shift the processing of structure to pre-processing stage, which significantly enhances training efficiency.", "questions": "1. An assumption is made about the structure of hypergraph i.e., absence of isolated nodes or empty hyperedges, for the theoretical results. What happens if isolated nodes or empty hyperedges are present? I am not able to see why this assumption is required, and what breaks if it is violated?\n2. It is commendable that the proposed TF-HNN performs significantly better than the baselines, but it is also a bit strange to see the baselines performing so poor, particularly on trivago. I understand the boost in training time, but not able to fully understand why there is a 10% improvement, it seems to me  that the learning ability of any SOTA HNN should be similar to TF-HNN. I may have missed something, but curios to hear what the authors have to say on this.", "limitations": "", "rating": 10, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The paper makes a significant contribution by addressing the issue of high computational complexity of Hypergraph learning algorithms. \n2. The proposed solution, TF-HNN is novel and elegant, which decouples the processing of structural information from the model training stage. \n3. Authors provide a strong theoretical foundation for TF-HNN, the unified framework presented in the paper links all the popular HNN approaches, which shows that TF-HNN is designed by keeping many existing methodologies in mind, and hence provides a comprehensive mechanism for efficient training.\n4. Extensive experiments on diverse real-world datasets for node classification and hyperedge prediction tasks demonstrate the competitive performance of TF-HNN against state-of-the-art HNN baselines while requiring significantly less training time.\n5. The paper is well-written, with a clear motivation, rigorous theoretical analysis, and thorough empirical evaluation supporting the proposed method's effectiveness and efficiency.", "weaknesses": "I do not see any weak points in this paper. This is a very well written paper, with significant contributions. Please refer to the questions sections for the questions I have.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MwXX5hAbUz", "reviewer": "Reviewer_bexR", "summary": "This paper introduces an efficient hypergraph learning scheme that performs message passing prior to learning neural network parameters. The proposed framework, called TF-HNN, includes a TF-MP-Module in which training-free message passing is performed. Using the updated features from the TF-MP-Module, TF-HNN learns an MLP model without a heavy computational burden. Despite its high learning efficiency, TF-HNN demonstrates either superior or competitive performance in hypergraph learning tasks. Theoretical analysis supports the design of the proposed method.", "questions": "Isn’t too much time being spent on hyperparameter search due to the extensive hyperparameter search range?\n\nIf the hyperparameters were indeed selected based on the validation set, could you demonstrate this by providing heatmaps of the hyperparameters across the various validation and test sets used in the experiments?\n\n**minor comments**\n- line 209: shonw -> shown\n- line 144: Ortega et al. (2018) -> (Ortega et al. (2018))", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The design of the proposed method is very interesting.\n2. The proposed method is both highly efficient and effective.\n3. The paper is clearly written and well-organized, making the research easy to follow.", "weaknesses": "It appears that there is an issue regarding the hyperparameters. The combinations of hyperparameters used in the experiments shown in Table 13 and Table 14 are quite diverse. For example, the value of alpha ranges from 0.05, 0.15, 0.3, 0.6, 0.65, to 0.7. The learning rate also varies, with values like 0.0006, 0.0001, 0.005, 0.001, and 0.0002. What method was used for hyperparameter search? Additionally, upon reviewing the attached anonymous GitHub link, it appears that the optimal hyperparameters were selected based on performance on the test set rather than the validation set. Were the hyperparameters selected in a fair manner? An analysis of hyperparameter sensitivity should be added.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nzaomzq2L9", "reviewer": "Reviewer_9Q7r", "summary": "The authors propose a scalable hypergraph neural network (HNN) that reduces the computational cost associated with message passing at each training epoch by decoupling the message passing step.\n\nTo achieve this, they (1) establish a general framework that explains popular HNN models, and (2) simplify this framework by removing learnable components and non-linearity, resulting in a single linear operator.\n\nThey demonstrate the effectiveness of their approach on both small- and large-scale hypergraphs, showing improvements over several existing HNN models.", "questions": "Refer to the Weakness section.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- S1. Given that real-world hypergraphs are often large, scalable HNNs are necessary in practice. \n- S2. Although the model design is quite trivial, the theoretical motivation (i.e., building a general unified framework and simplifying it) of the model design is systematic and interesting.\n- S3. Experiments are comprehensive.", "weaknesses": "***Major comments.***\n\n- ***W1. Regarding Proposition 3.2.*** My understanding is that the key idea is the existence of a clique expansion that satisfies the property outlined in Proposition 3.2. \nIn practice, however, the authors use a fixed clique expansion as described in Equation 5. \nTo what extent does this chosen clique expansion align with the one referenced in Proposition 3.2? \nWhile the exact formulations may differ, it would be helpful to know if the high-level characteristics of these clique-expanded graphs are similar. \nThis is essential, in my view, as it clarifies whether the theoretical analysis effectively supports the proposed method.\n\n- ***W2. Regarding Proposition 4.1.*** Could you clarify what is meant by the \"entropy of information\"? Does this refer to mutual information between features and node labels? Further elaboration on this point would help in understanding the key takeaway from this proposition.\n\n- ***W3. Regarding the Initial Message Passing Operator Computation Complexity*** Although the message passing operator incurs a one-time computation cost, the time required for this process should be reported. If this initial computation is substantial and exceeds the typical training time of existing HNNs, it could limit the practical efficiency of the proposed method.\n\n***Minor comments.***\n- In Lines 52-53, the text mentions that $n^{k}$ memory is required. While this is accurate for dense tensor formats, typical tensor representations are stored in a sparse format, and sparse operations are well-supported in modern deep-learning libraries. Thus, storing a dense incidence tensor is generally not necessary in practice. It may be helpful to revise this part to reflect real-world scenarios.\n- In Lines 79-80, the period \".\" is missing.\n- Please provide clarification on the source of the datasets used.\n\nPlease let me know if I have misunderstood any parts. Thank you.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)