Disentangling Hyperedges through the Lens of Category Theory
Authors: Yoonho Lee
Junseok Lee
Sangwoo Seo
Sungwon Kim
Yeongmin Kim
Chanyoung Park
Venue: NeurIPS 2025 poster ()
Keywords: Graph Neural Network
Category Theory
Disentangled Representation Learning
Categorical Deep Learning
Hypergraph
Primary area: deep_learning
TL;DR: We propose a novel criterion for hyperedge disentanglement. We analyze hypergraph MPNN and disentanglement through the lens of category theory, and derive the criterion.

Abstract: Despite the promising results of disentangled representation learning in discovering latent patterns in graph-structured data, few studies have explored disentanglement for hypergraph-structured data.
Integrating hyperedge disentanglement into hypergraph neural networks enables models to leverage hidden hyperedge semantics, such as unannotated relations between nodes, that are associated with labels.
This paper presents an analysis of hyperedge disentanglement from a category-theoretical perspective and proposes a novel criterion for disentanglement derived from the naturality condition.
Our proof-of-concept model experimentally showed the potential of the proposed criterion by successfully capturing functional relations of genes (nodes) in genetic pathways (hyperedges).

Disentangling Hyperedges through the Lens of Category Theory
Yoonho Lee
Junseok Lee KAIST
Sangwoo Seo KAIST sangwoosee@kkisst.ackkk
KAIST
sml bmnleggbbenbmmkaiist.ackkkk
junseoklee@kaist.acckk
Sungwon Kim KAIST
Yeongmin Kim KAIST
Chanyoung Park KAIST
swkimkkaist.ac.kt
cytotoxicity@@kaitttackkk
cy gy.parkkkist.acckk
Abstract
Despite the promising results of disentangled representation learning in discovering latent patterns in mraph-structure data, few studies have explored disentangle- ment for hypergraph-struutuue hypergraph neural networks enables models to leverage hidden hyperedge seman- ties, such as unannotated relations between nodes, that are associated with labels. This paper presents analysis of hyperedge disentanglement from category- theoretical perspective and proposes a novel criterion for disentanglement derived from the naturality condition. Our proof-of-come conceptp model experimentally showed the potential of the proposed criterion by successfully capturing functional relations of genes (nodes) in genetic pathways (hyperedges). Our implementation is available at https::/ /ttpthu.///hu......///g
data. Integrating hyperedge disentanglement into
Lee AILScience/Natttte
Introduction
Disentangled representation learning, which aims to identify underlying factors behind observed data, has been applied graph neural networks (GNNs) to capture hidden semantics mechanisms in graph-structured data. In molecular graphs, for example, molecular properties are determined by underlying graph-level mechanisms, where specific substructures play distinct roles in shaping these properties. To reflect such graph-level mechanisms, graph-level disentanglement can be designed capture multiple substructures, each corresponding to different molecular properties. As another example, in opinion dynamics [54, 29, 28), which studies how individuals' opinions evolve through interactions within social network. individual's 's opinion change after engaging in discussions with neighbors. These discussions act as edge-level mechanisms that influence opinion updates. To reflect edge-level mechanisms, edge-level disentanglement can be designed capture multiple topics underlying discussions, each affecting different aspects individual opinions. Depending on the type of mechanism, several types of disentanglement, including node-level [47], edge-level [85], and graph-structure
[77] approaches, have been proposed.
fundamental challenge lies in designing criterion for disentanglement, which determines how relevant each factor is to each mechanism (e.g., each node, edge, or subgraph). Since the represen- tation reflects each factor in proportion to its relevance determined by the criterion, the criterion should be designed in accordance with the type of disentanglement to ensure that the intended type of mechanism properly captured in the representation. Thus, many disentanglement models strive to identify fundamental characteristics associated with the type of disentanglement and incorporate them into the design of the criterion.
Corresponding author.
39th Conference on Neural Information Processing Systems (NeurIPS 2025).

---

2
Representation (vector)
Factor 1
(i) Factor representation for nodes
(ii) Node features
(III) Factor representation for nodes
Factor 2
Entangled
Message passing
High relevance
Low relevance
Projection
(Iv) Factor representation for hyperadge
(v) Hyperedge representation (Entangied)
(vi) Factor representation for a hyperedge
Figure I: The factor representation consistency criterion assigns a high relevance score when the factor representation learned by two different routes similar (i.e., consistent factor representation).
Despite numerous studies on disentanglement conducted far [47, 85, 77, 31, 51], hyperedge disen- tanglement remains largely unexplored. Hyperedge disentanglement assumes that group interactions (i.e., hyperedges) have mechanisms that determine the labels, and aims capture the factors (i.e., context or condition) that influence these group interactions. A representative example is the genetic pathway, which is set of genes (i.e., hyperedge) that interact to perform specific biological function. When a pathway becomes dysregulated, its associated biological function can be impaired, potentially leading to diseases such as cancer. The functional context of pathway can thus be re- garded as an underlying factor that governs how group interaction among genes influences high-level labels, such as disease types [65, 74]. Therefore, we aim to design criterion for hyperedge- disentanglement that enables a model to capture hyperedge-lewe factors, such as the functional context of genetic pathways.
To the best of our knowledge, entanglement. To identify characteristics that can be derived from the definition of hyperedge disentanglement, rather than from any data-specific assumptions, analyzed the hypergraph mes- sage passing neural network (MPNN) and hyperedge disentanglement from a category-theoretion perspective, as this viewpoint provides global structural understanding of 'how the system works.' We discovered that the naturality condition holds between entangled and disentangled representations, and we used this as a characteristic associated with hyperedge disentanglement. Based on this characteristic, we defined factor representation consistency as the criterion. Figure briefly illustrates our criterion for hyperedge disentanglement. As shown in Figure I, there are two ways to obtain hyperedge factor representations: one by disentangling first and then performing message passing (i.e., (ii) - (i) (iv)), and the other by performing message passing first and then disentangling (i.e., (a) (v) (iv)). Our criterion suggests that the hyperedge factor representation learned by both methods should be similar (i.c., consistent representation) when the factor is relevant to hyperedge disentanglement. To validate whether novel criterion can disentangle hyperedges, we created a proof-of offof-ffooee offconcept model, Watural-HNN Neural Network), and performed a cancer subtype classification task with hypergraphs of genetic pathways. Our model outperformed the baselines by successfully capturing the functional context of pathways, which are the underlying factors influencing group interactions.
the first propose a criterion designed for hyperedge dis-
cntturality-uuidee disentangled Hypergraph
Our main contributions
summarized as follows:
This paper, for the first time, provides an analysis of hypergraph message passing neural networks and hyperedge disentanglement through the lens of category theory.
derive novel criterion for hyperedge disentanglement. To the best of our knowledge, this is the first paper to propose a criterion for hyperedge disentanglement.
Based the analysis,
We create simple yet effective sroof-of-concepy model, Natural-HNN- and performed cancer subtype classification task. Experimental results showed that the model could capture the functional context of pathways, which are factors associated with hyperedge disentanglement.
2 Related Work
In Section 2.1, we briefly describe a criterion widely used in disentangled representation learning and discuss why it may not be suitable for hyperedge disentanglement. In Section 2.2, we discuss how category theory has been applied in the field deep learning and explain how we adopt the theory to problem at hand. In Section 2.3, we briefly summarize several hypergraph neural networks.

---

2.1 Disentangled Representation Learning (DRL)
Disentangled representation learning consists of three components: factor encoder, factor discrimina- tion loss, and criterion. A factor encoder projects the entangled representation into factor-specific representations. These factor encoders are implemented as K MI.Ps, where K denotes the number of factors, given as a hyperparameter. To encourage each factor representation to contain distinct information, factor discrimination losses, including factor classifier- loss [85], factor-wise contrastive learning loss [41], and the Hilbert-Schmidt Independence Criterion (HSIC) [47], are used.
The disentanglement criterion is the most crucial component of DRL, as it determines the relevance of each factor and consequently how much it is reflected in the representation. This criterion is designed based on the characteristics that the intended disentangled factors should ideally possess. Although defining the criterion based on such ideal properties does not theoretically guarantee successful disentanglement, numerous studies have empirically confirmed that these criteria indeed enable effective disentanglement. For example, in the early image generative models that pioneered DRL, disentanglement was guided by adopting the equivariant property as the disentanglement criterion [30], since an ideal generative factor should cause the image to vary equivariantly with changes in the generative factor. Therefore, many studies have strived to identify suitable ideal characteristics for the type of disentanglent lemett they pursue, under the assumption that such ideal properties of factors facilitate effective disentanglement.
In the field of graph and hypergraph representation learning, disentanglement has been used to exploit hidden semantics behind subgraphs or interactions with neighbors. Since such hidden semantics are highly abstract concepts, it is difficult identify generally applicable properties. Consequently, as these semantics exhibit different characteristics depending on the data, the design of criteria has often relied heavily on assumptions about the data. The most widely used criterion for disentanglement is the factor representation similarity-basee approach. For example, DisenGCN [49] assumes that the k-th factor is likely the reason behind the existence of an edge in a graph if the k-th factor representations of the two connected nodes are similar. A similar criterion is also used by HSDN [31], which performs syppeggapph-structureleeee disentanglement aimed at identifying substructures that contribute to hypergraph properties. The authors of the paper assume that important hyperedges would share commonalities and therefore need to have similarity in the factor representations of nodes.
However, factor representation similarity-based criterion may not be suitable for hyperedge dis- entanglement because the way group interactions influence labels are not necessarily related to the similarity or commonalities between participants. For instance, consider the case of opinion dynamics involving a group engaged in a discussion; the topic of such discourse need not necessarily pertain to the commonalities shared among its participants. One can easily imagine situation where researchers from diverse fields gather to discuss and solve complex and challenging problems. As another example, in genetic pathways, the similarity of gene expression values (i.e., gene features) of the constituent genes bears no relation to the functional context. Therefore, since the existing criteria based on data-specifie assumptions are not suitable for hyperedge disentanglement, we aim to develop a broadly applicable hyperedge disentanglement criterion that does not rely on heuristics.
To develop a universally applicable criterion that does not rely on heuristics or data-specific as- sumptions, we first need to analyze how hidden semantics are involved in the mechanisms through which group interactions contribute to labels, and to derive the corresponding characteristics or properties from this analysis. However, since the hidden semanties underlying group interactions are highly abstract concepts, conducting such an analysis is inherently challenging. To address this challenge, we employ category theory, which is well-suited for representing and analyzing systems as compositional structures. By formulating hyperedge disentanglement and investigating how factors contribute to the mapping mechanism through the lens of category theory, discovered that naturality condition must hold between the entangled and disentangled representations of nodes and hyperedges. Based on this observation, we derive a novel criterion based on hyperedge representation consistency. Finally, we conclude this section with a formal defnnition of hyperedge disentanglement.
Hyperedge Disentanglement. A hypergraph with N nodes and M hyperedges can be represented by incidence matrix I E (0, 1) MxM which indicates whether a node belongs to a hyperedge or not. Hyperedge disentanglement assumes the existence of multiple hidden factors underlying group interactions, which influence how labels are determined, and aims to capture these factors while predicting the labels. In other words, it is assumed that there exists a set of disentangled incidence
3

---

Ixl which are not explicitly provided in the data, where I, denotes incidence matrix of subhypergraph for factor i. The objective of hyperedge disentanglement to learn a hypergraph neural network funw (I that approximates the ground-trute label mapping function faaia (I X) by learning approximation of I Note that hyperedge-level disentanglement differs from hypergraph structure-lewe disentanglement, as the latter assumes that the presence of certain substructures determines the labels fi.e.. ()
matrices
= I T
2.2 Category Theory for Deep Learning
Category theory is abstract language of mathematics that focuses on the compositional structure of a system. One of the applications in the field of deep learning that uses category theory is neural algorithmic reasoning [68] which aims to train neural network that can execute algorithmic computation in latent space. Several studies [14, 15] have attempted align the computational structure of an algorithm with that of the model to effectively approximate computer algorithms. The motivation for aligning the structures comes from the theoretical conclusion [75] that structurally aligned models generalize better due to lower sample complexity (i.e., they require fewer samples in training ensure low test error). Motivated from the works above, we analyze a hyperedge disentanglement model using category theory from the perspective that the computational structure of the model should be structurally aligned with the factor-related mechanism. Through this formulation, we identify a characteristic that can serve as a criterion. Note that the basic concepts in category theory used are described in Appendix A.
2.3 Hypergraph Neural Networks (HNNs)
Several HNN models have been recently proposed to leverage information contained in multiway interaction. HGNN [20] and HCHA [3] use a normalized hypergraph Laplacian, which is math- ematically equivalent to clique expansion (CE) [67]. and apply the traditional graph convolution mechanism. HNHN [12] additionally adopts nonlinearity when calculating hyperedge representations to differentiate a hypergraph from a clique expanded graph, while UniGNN [32] unifies HNNs and GNNs into the same framework. Moreover, HyperGAT [11] adopts the attention mechanism to HNN for text classification, and SHINE [48] proposes dual attention mechanism for the disease classifica- tion task. ED-HNN [70] proposes equivariant message passing HNN, which allows hyperedges to propagate different messages to its incident nodes. AllDeepSets and AlSSeCTransformer [6] consider a hyperedge as a set and apply DeepSets [83] and Set Transformer [37], respectively, to increase expressive power of HNN.
Efforts apply disentanglement to ippergraph-structuree data have been relatively limited. HIDE [42] and DCNenHCN [43] applied hypergraph disentanglement in the context of recommender systems. However, in these works, the hyperedge semantics were explicitly provided as hyperedge types in the data, and their approaches focused on disentangling node features corresponding to each hyperedge type, rather than capturing the underlying hyperedge semantics. HSDN [31] proposed hypergraph structure-level disentanglement, rather than hyperedge-level disentanglement.
3 Categorical Interpretation of Message Passing HNN and disentanglement
Before addressing hyperedge disentanglement, we first analyze the relationship between the mecha- nism by which group interactions influence labels and hypergraph MPNNs from the perspective of category theory, which will be discussed in Section 3.1. In Section 3.2, we further concretize this analysis by examining how factors relate to the mechanism and describe the process of deriving the characteristic (i.c., naturality condition) from it.
Notation. Let 9 = (V, E) denote a hypergraph. where == {vi, V2...., Vy indicates set of nodes and E (e1, E22 ..., ) indicates set of hyperedges, where N = =1 and M = |1] are the number of nodes and the number of hyperedges in hypergraph G, respectively. A set of node features given of hyperedge representations
input each layer of the model is denoted X = 1 (calculated in each layer of the model) is denoted as H = the, .... hey) and set of representations obtained after message passsing is denoted as Y (Ye, ... Fry 'en' denotes entangled object

---

eepersentatt parsing
Wt
Air
Furcor
Flaul
Hypuredge
Hpawiedp
lwperolge rxpresse
personng
bathoodadi
bet rode repersental
(a) Hypergraph
(b) Hypergraph mechanism as PiSet category
[c) Hypergraph representation (d) Hypergraph MPNN learning
Figure 2: Compositional structure in hypergraph representation learning.
morphism and is written in superscript subscript, while 'dis' denotes disentangled object morphism. The symbol '3' is used to denote the composition of morphisms.
3.1 Compasitionality in Hypergraph Representation Learning
Most hypergraph representation learning methods produce the representation of node by integrating its own representation and its neighbors' representations defined by a hypergraph topology. The fundamental assumption underlying these models is that group interactions with neighbors contribute, some manner, to the labels. To further elucidate this assumption, consider the hypergraph example depicted in Figure 2 (a). Although not given by the hypergraph topology, we introduced the set Ne. which includes the center node Ve and its neighbors, in order to represent the information that Ve possess after message passing. Then, the assumption can be illustrated as in Figure 2 (b). Each group interaction, given hyperedge. can produce new meanings or information (e.g., new meaning for e1) through some interaction mechanisms (e.g.fii-fel) Subsequently, the assumption posits that this newly generated information influences the participants (e.g., v.)) of the group interaction via some mechanism (e.g.. 81e, 82c), thereby resulting in new information (e.g., N.) for the participants (e.g., V.) that may be associated with the label.
The abstract description above can be formalized through the lens of category theory. Specifically, we consider each node as a set, since a hyperedge contains nodes, there are morphisms (inclusion) between nodes and hyperedges induced by the poset structure. We defined this as PISet, the category with poset structure where morphisms are inclusions and objects are sets. Thus, we can see nodes (V1, Ves V2, V3) and hyperedges (e1, e2) constitute PISet as shown in Figure (b), where gray-colored nodes and hyperedges are set objects, and inclusions are morphisms (blue arrow) between sets. The same mechanism holds between hyperedges (e1, 62) and set N₂ that includes node v. and its neighbors. In Figure 2 (b), for instance, we can see hyperedges (e1, e2) and N, constitute PISet as they have morphisms (green arrow) induced by the poset structure.
In order to learn and predict with computers, such objects and morphisms must be expressed in numerical values and their transformations. Hence, we define category of deep learning repre- sentations, DLRep, where objects are vector representations and morphisms are transformations between them. Figure (c) shows the result of applying functor F PISet - DLRep, which can be simplified to a diagram in Figure 2 (d). Thus, any kind of hypergraph MPNNS can seen as way of learning representations and their transformations respecting compositional structure of entities. In other words, hypergraph MPNNs can be seen structurally aligned, to some extent, with the mechanisms by which group interactions present in hypergraph data influence the labels.
However, the degree to which model is structurally aligned depends on implementation details. For example, convolution-based models are structurally well-aligned with mechanisms which all nodes contribute equally during group interactions. Conversely, when node contributions vary within the group interaction, attention-based methods are more structurally aligned with the mechanism than convolution-baul ones. Therefore, to perform hyperedge disentanglement, we structurally analyze how factors involved in the mechanism and, in Section 3.2, investigate the characteristies of hyperedge disentanglement model that is well structurally aligned with the mechanism.
3.2 Guiding Disentanglement with Naturality Condition
Since entangled and disentangled representations are different ways of representing the same compositional structure, can regard them the result of applying two different functors PlSet - DLRep (for entangled representations) and G PlSet DLRep (for disentan- gled representations) shown in Figure (a). Thus, we have the naturality condition between
'Two notations f gg and of have the same meaning "applying/ / first, and then applying g". We use the notation 'y' following [23].
5

---

E
Ros
Diemtangee
V
E
Hln
Hills
Hen 00
HRS
[x
X
R1c
X
Xole Disentangled
Entangled
PiSet
Natural Transformation HNN
Natural Transformation HNN, factor perspectire
Figure 3: Naturality condition in disentangled representation learning to capture group interaction mechanism related factors. X denotes a set of node representations and H denotes hyperedge representation. V and E denote nodes and hyperedge in PISet. 'c" and 'd' denotes factors.
entangled and disentangled representations. Figure 3 (b) is equivalent to Figure 3 (a), but only the components related to the factor "c' are shown (explanations are in Appendix A.6). Note that
Ox,e = Ox Pe where P. xdir - x If factor "c" is relevant to the morphism between node set and hyperedge E, the naturality condition must hold for the perspective of factor 'c'. Thus, factor "c' representation of hyperedge (i.e., H!=")) must be the same (or similar) regardless of applying 1m an,, (i.e., message passing on entangled representation first, and then disentangling factors) or
0x,e .ef (i.e., disentangling factors first, and then message passing disentangled representation). In other words, the factor representation must be consistent regardless of the sequence of operations if that factor is relevant to the interaction context of hyperedge. We use this property guidance for disentanglement, since must hold for any kind of hypergraph message passing neural networks, and must work regardless of data characteristics.
4 Proof-of-concept model Natural-HNN
To validate whether our criterion can effectively capture factors relevant to hyperedge disentanglement, we implemented simple yet effective model, Natural-HNN, Each layer the model is consisted with 3 components shown in Figure 4: 1) Node-to- hh-pe-ee-eee propagation step that learns hyperedge factor representations and relevance scores, which is calculated by our criterion. 2) Hyperedge-to- Node propagation step that propagates factor representations of hyperedges to nodes with weights proportional to relevance scores. 3) The last component concatenates factor representations and produces final outputs by interpolating with the node representations given as input to the layer. Note that each layer of Natural-HNe has K factors where is hyperparamter.
4.1 Hode-to-Hyperedee Factor Propagation
Obtaining Two Disentangled Hyperedge Representations. To validate whether the naturality condition (Figure (a)) holds, we need get two disentangled hyperedge factor representations for
every factor (i.e., Hr for every factor & [1, K]). The two disentangled representations are obtained through 1) Aggregation-firs Branch and 2) Disentalgle-first Branch. In the following, we describe how morphisms Figure (a) are implemented as operations in the two branches shown in Figure 4 (b).
Aggregation- Branch, The first disentangled representation obtained from the aggregation- first branch performing fdh ş &'H for each factor k. This process is implemented performing aggregation agg.2, (i.e., in Figure 4 (a)) first, and then disentangling into hyperedge factor representations using factor encoder Oh, The factor representations hyperedge e, obtained from this branch are denoted as ,,....,, Disentangle-first Branch. The other one is obtained from the disentangle- first branch performing ax, ä for each factor k. This process is implemented as disentangling into node factor represen- tations with factor encoder ax, first, and then performing aggregation agga2e (i.e.. - in Figure 4 (a)). Factor representations of hyperedge e, obtained from this branch are denoted h h A
For both branches, we used mean aggregation as aggw2e and MLPs as factor encoders for disentan- gling factors. Factor representations vectors with size d/ K (i.c., R Hi R²), when the desired size for node representations after message passing is d. In summary, operations of the two branches regarding factor & can be written follows:
7t MLP; (Mea (xx, (x,/Mx |M: E e,))),
ni mean( (MLP, ej)
(1)

---

(a) Naturality Condition
(b) Node-to-Hyperedge propagation
(c) Hyperedge-to- propagation
Factor
Aggregation-fiss crrtttttoo---ttttttt Branch
Encoder
Sau
(d) Concatenation
Aggregation-frst Branch
Ny concot(A)
Similarity scorer
comcect
Disentangle-fiie firstt Branch
(e) Output
Disentangle first Branch
Factor
agihh (mean)
Loyer Un 1z,
Encoder
concatt at
Figure 4: Architecture of proof-of-concept model Natural-HNN, It calculates the relevance of factor k (a.) and performs weighted message passing for each factor.
Deciding Factors with Consistency. The extent to which the naturality condition is satisfied can be measured by caleulating the similarity between the disentangled hyperedge factor representations Hy and Kf,- In other words, we can consider that the naturality condition holds when the two representations are similar (i.e., consistent), and does not hold when the two representations are largely different. We introduce a similarity scorer that calculates the similarity of two Lg-normalized vectors. Specifically, we calcualte the relevance or importance of factor for hyperedge e, as af - We 1K,,
where W₂
a learnable parameter matrix for factor k, and a is the
sigmoid function. Lastly, we obtain the final hyperedge factor representations by multiplying of to the corresponding hyperedge factor representations obtained from the disentangle-iixt branch".. i... ofke that reflects the relevance of the factor for the hyperedge e;-
4.2 Hyperedge-to-Node Factor Propagation
When aggregating hyperedge representations (i.c., a hh,) to update node representations, the sum of neighboring hyperedge representations with respect to factor k must be divided by the sum of aj so that hyperedge relevance scores (i.e., af) are normalized during aggregation. Thus, the updated factor representation of node v,, i.e., yr,, can be written as y E,,0,, 0 ay 0j K
4.3 Final Output of each Layer of Natural-HNN
To allow model to determine its focus between information from neighbors (i.e., yú,) and information from the node itself (i.c., x one can introduce hyperparameter that determines the interpolation ratio between them (i.e., interpolate in 3 1 3 ratio). However, for simplicity, we set 3 0.5. so that the two pieces of information are interpolated in a 1:1 ratio. To make sure that interpolation is performed on disentangled representations, we used the factor encoder used in the message passing step (i.e., hr (x.)) Specifically, 2, LayerNorm(D.5y, + 0.5h,), where yv, Concat(y,, y,), h,, = Chncat(h!.......
4.4 Optional: Factor Discrimination Loss
Existing disentangled representation learning methods [47, 77] have widely adopted a factor discrim- ination loss aiming promoting factors to contain different information. Following [85], we added factor discrimination loss Las to the final loss, i.e., C Cuss A 3 where is a factor discrimi- nation loss weight given as a hyperparameter. Details can be found in the Appendix C.2. Using the factor discrinination loss increases the performance of our model (Table 7) and helps each factor to contain different information (Figure 6). However, introducing this loss requires additional hyperpa- rameter tuning for X, which often involves large search space and increases experimental runtime. Considering that this loss is not closely related to our primary experimental serecctvee-vaiiidatii whether proposed criterion captures factors relevant to hyperedge disentanglement- consider an optional component of the proof-of-comee model.
Although we choose the disentangle- first branch here, we can instead use the output of the aggregation-first branch. Both choices give similar results. Please refer Appendix E.I.
"non denotes the task related loss calculated from cross- enosopy loss with labels and predictions. Details are available Appendix C.3
7

---

Table I: Model performance on subtype classification task (Macro FI). Top two models
colored by First, Second. f the variant of the model using multihead attention. + Las is not used.
Method
BRCA
STAD
SARC
LGG
HNSC
CLSC
HONN
0.736 a 02055
0.565 = 0.1400
11.684
= 11.06
0.694 x 03155
0.799 0./94=010 01053
11K33 11052
HCHA
0.704 +: 0.051
0.558 + 0.044
0.675 : -0.068
0.682
0.041
0.783
0.844
1.44400.055 0.054
HNHN
0.697 0.046
0.573 + 0.072
0.688 + 00.075
0.674 0.038
0.791=0.035
0.059
UniGCNII
0.697 0.052
0.617 + 0.059
0.728 a: 0.066
0.663 a 0.039
0.841 0.046
AllDeepSets
0.056
0.557 = 0.044
0.599 + 0.058
0.665
0.801
00058
1.870=0.004
TransSrras
0.743
0.553 + 0.046
0.719 + 0.052
0.653
0.81400.066
0.847 0,046
HyperGAT
0.637 0.121
0.534 0.063
0.574 0.153
0.665
0.832 0.769
HynerGArT
0.641 0.115
0.502 0.087
0.584
0.04
0.791
0.041
SHINE
0.446 0.155
0,371 = 0.135
0.529 # 0.160
0.628
0.718=0.055
0.745 + 0.159
SHINE"
0.651 +: 0.053
0.532 + 0.064
01.773 +: 0.059
0.64
0.070
0.837 + 0.061
HSDN
0.757 + 0.044
0,629 + 0.045
0.726 + 0.063
0.811
0. 0.867 0.033
ED-HNN
0.735 0.047
0.615 0.050
0.718 0.071
0.700
0.030
0.835
0.047
0.875 0.053
ED-HNNII
0.722 + 0.045
0,536 + 0.057
0.650
+ 0.087
0.695 # 0.039
0.845 = 0.025
0.895 + 0.044
Natural-NNN (Ours)
0.804 + 0.036
0.659
0.049
0,745
0.045
0.707
0.035
0.862 0.045
0,881 0.042
5
Experiment
To evaluate our criterion, we performed a cancer subtype classification task from genetic pathways using our proof-of-concee model, Natural-HNN. Genetic pathways possess unannotated or hidden functional contexts (i.e., factors) underlying group interactions. Since these are closely linked to cancer and disease, they serve as appropriate data for validating the criterion. Through experiments, we aim to answer the following questions:
RQI Does Natural-HNN perform well on data where factors, such as functional context, underlying the mechanism are present? (Section 5.2)
RQ2 Are the factors captured by Natural-HNN related to hyperedge disentanglement? other
words, are they related to the functional context? (Section 5.3)
RQ3 Can Natural-HH HNN generalize well? And how much is Natural-HNi affected by hyperparam-
eters? (Section 5.4)
5.1 Experimental Setup
Dataset. For the cancer subtype classification task, we downloaded clinical data for 6 cancer types (BRCA, STAD, SARC. LGG, CESC, HNSC) and preprocessed data following Pathformer [46] (Details in Appendix B.2). Every patient (i.e., hypergraph) has the same genes (i.e., nodes) and pathways (i.e., hyperedges), but the clinical data (i.e., gene features) are different. The data statistic of each cancer data is provided in Appendix B.
Compared Methods. We compared Eitural-HNN with HNNs introduced in Section 2.3. Specif- ically, HGNN[20J, HCHA [3], HNHN [12], UniGCNII [32], AllDeepSets [6], AllSerTransformer [6]. HyperGAT [11], SHINE [48], ED-HNN [70], ED-HNNII [70] and a hypergraph disentangling method HSDN [31] are used as baselines. Implementation details of some baselines and their variants
are described in Appendix C.I.
Evaluation. We randomly split the data into 50% for training/validdationnesss set. We measured average and standard deviation of the performances for 10 different data splits. The hyperparameter search space is provided in Appendix C.5.
5.2 Results for Cancer Subtype Classification (RQ1)
The cancer subtype classification task can be considered hypergraph classification task, since every patient (i.e., a hypergraph) has the same genes (i.c., nodes) and pathways (i.e., hyperedges). Specifically, we generated the representation of a hyperedge by simply concatenating representations of hyperedges in a hypergraph following Pathformer [46], due to the lack of an effective pooling method reflecting the hypergraph topology developed to date. Then, we applied one layer MLP as the classifier. We have the following observations in Table 1. 1) Natural-HNN shows superior performance in most of the cancers with large performance gap compared with most of the models. Especially in the case of BRCA, we achieve approximately 5% performance improvement compared to the second-best model. It can be concluded that incorporating the functional context (i.e., factors) of pathways has contributed to improved performance. 2) Natural-HNN outperforms the hypergraph- structure-Iewe disentanglement model, HSDN, with a significant performance gap. HSDN uses factor similarity- based criterion to determine the relevance of factors. However, the superior performance of Natural-HNN validates that naturality-guider disentanglement is more effective at integrating the context behind group interactions.
8

---

with pathways selected by Natural-iNNi
161
Figure 6: Pearson correlation between hyperedge factors.
(e)
Nette 1999
A Sat Tortttatorer
Figure 5: Captured interaction context. Captured patterns shown in red boxes and not captured patterns are shown with orange boxes. Weakly captured cases are marked as Figure 7: Marcro Fl scores with differ- dotted red block.
Deeep modals
modal:
ent training ratio.
5.3 Capturing the Interaction Context of Hyperedges (RQ2)
To validate that Natural-HNN captured factors relevant to hyperedge disentanglement, we checked whether our model captures the functional semantics of genetic pathways. Because the models rely solely on cancer subtype labels during training", we expect the interaction contexts of informative hyperedges (such as cancer-related pathways) be captured by the models, while non-informative hyperedges (such as pathways not relevant to cancer) are not. For this experiment, we first selected top-15 pathways" based on the SHAP value for each model (Natural-HNN in Figure top and HSDN in Figure bottom). Note that we rely on the SHAP value since information regarding which pathways are relevant to cancers is not given. Then, after clustering these 15 pathways with CliXO algorithm [34], calculate the similarity between clusters based on the average similarity of pathways that belong to each cluster. Our goal is to check how well Natural-HNN preserves the functional semantic similarity between pathway clusters compared with the cluster similarity calculated with Lin's s method [44] (BMA), which we consider as the ground-trute For HSDN and Natural-HNN, cluster similarity is calculated based on the relevance score vector of each hyperedge
across all factors, i.e., Og [a] ..., o, which can be calculated 1/(1 ||a - agll2)) As the experiment setting is somewhat complicated, we described the detailed procedure Appendix B.3.
The result on the BRCA datset is shown Figure 5. The row and column of each heatmap is the index of the pathway clusters and color represents similarity between clusters. Figure 5 (a). (b) and (c) shows the measured similarity between clusters with pathways selected by Natural-HNN. Comparing (b) and (c) with (a), observe that Natural-HNN preserves the functional similarity (red box) better than HSDN, which fails do so (orange box). Moreover, Figure 5 (d), (e) and (f) shows the measured similarity between clusters with pathways selected by HSDN. An interesting observation is that even with the pathways that were informative to the HSDN, HSDN fails (orange box) to preserve the functional similarity between clusters while Natural-HND could capture them. The results imply that the naturality condition in category theory is effective in capturing the interaction context of hyperedge.
Finally, checked whether each factor captures different context by calculating Pearson correlation coefficients among hyperedges captured by each factor, following [85]. As shown Figure (b), factors tend to exhibit only weak correlations. Note that even when factors are completely disentangled, a small degree of correlation can naturally exist between factors, as described [59]. We observe that the factor discrimination loss decreases correlation between factors when comparing Figures 6 (b) and (a).
"This means that models do not external data related pathway types or pre-trained models. 'Ouly a few pathways are related to each type of cancer. We can also observe this with the SHAP value distribution in Figure 15 of Appendix B.4.

---

[ Gruudd muth
nown 50%
01 Broundrrsh
[i] Drturujaaann feas
[R] lucter Macro 0.797
Ovi3 lactee Macro 0.795
trll
LM] traln 30%
M.A 0.08L Macro SL: 0.8SS (M2 0.01, Macoo F2: 01:6.86
140 16 dim, Nacio F1:0.77
[vi) G4 dim, Macio F1-0.77
(a) Traliing
(b) Hyperparameters
Figure 8: Captured functional context with different (a) Training set ratio and (b) Hyperparameters. Patterns that are well-captured are shown in red and those that are not captured are shown in orange.
5.4 Generalizability and hyperparameter sensitivity of Natural-HNN (RQ3)
Generalizability. To validate the generalizability of Natural-HNe HNNN we measured performance while gradually reducing the training set ratio from 50% to 10% in 10% decrements. Figure 7 shows the performance of model (blue) and baselines. Figure 7 (a) shows comparison with convolution- or DeepSet-based models. These baselines rely on a strong inductive bias that nodes contribute equally hyperedges during the group interaction process. Models with such strong inductive biases typically exhibit strong generalizability. Observing the extent performance degradation as the training ratio decreases, Figure (b) compares Natural-HNN with attention- models, which are known for their strong expressivity. As shown in the figure, Natural-HNN consistently outperforms these models. This indicates that NNN possesses both strong generalizability and sufficient expressivity. Figure 8 (a) presents experimental results evaluating whether the functional context (i.c., factors) is well captured even the training ratio decreases. As can be seen, a significant portion of the functional context is well captured despite the reduced training data, demonstrating that our proposed criterion effectively captures the factors.
that Natural-HNN also demonstrates good generalizability.
Hyperparameter Sensitivity. We conducted experiments to evaluate the impact of hyperparameters, such as the number of factors, on -atursl-HNN's ability to capture factors. Figure (b) reveals the following insights: 1) When the number of factors is 2 or the overall similarity tends to be slightly higher than the ground truth; however, the core strong similarities are still well captured. 2) Regardless of the value of the factor discrimination loss weight X, the functional context (factors) is consistently well captured. 3) When the dimensionality is too large, the core strong similarities are well captured, but the overall similarity tends to be slightly higher than the ground truth. Conversely, too small, some functional similarities are missed. These observations
when the dimensionality suggest that, except when the dimensionality is too small, Natural-HNN can generally capture the functional context well, regardless of hyperparameter settings.
Additional Experiments. In the Appendix, we provide ablation studies (Appendix E), time complex- ity analysis (Appendix F.1) and results hypergraph benchmark datasets (Appendix D).
Conclusion
In this work, we propose criterion for hyperedge disentanglement by discovering characteristic called factor representation consistency. To uncover this characteristic, we analyzed the compositional structure in hypergraph message passing and focused on the naturality condition that satisfied between entangled and disentangled representations. The characteristic derived from a hyperedge disentanglementt model that structurally aligns with the underlying mechanism demonstrated effec- tiveness in capturing the functional context (i.c., factors) of genetic pathways (i.e., group interactions). Experiments showed that this simple criterion generalizes well and consistently captures factors regardless of hyperaarameter choices.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by the Institute of Information & Communications Technology Planning Evaluation(IITP) grant funded by the Korea govemment((SIIT) (RS-2025-02304967,, Al Star Fellow- ship(KAIST)). Additionally, this work was supported by Institute of Information & communications Technology Planning & Evaluation (ITTP) grant funded by the Korea governmen((MSIT) 0-00004) Finally, this work was supported by National Research Foundation of Korea funded by Ministry of Science and ICT based upon data generated by the TCGA Research Network: https:/ : :/ww:////www///// cancer, gov/ /tgga.
N020-
The results shown here are in whole or part
References
[1] S. A. Aleksander, J. Balhoff, S. Carbon, J. M. Cherry, H. J. Drabkin, D. Ebert, M. Feuermann, P. Gaudet, N. L. Harris, et al. The gene ontology knowledgebase in 2023. Genetics, 224(1): iyad031, 2023.
[2] M. Ashbumer,, C. A. Ball, J. A. Blake, D. Botstein, H. Butler, J. M. Cherry, A. P. Davis, K. Dolinski, S. S. Dwight, J. T. Eppig, et al. Gene ontology: tool for the unification of biology. Nature genetics, 25(1):25-29, 2000.
[3] S. Bai, F. Zhang, and P. H. Torr. Hypergraph convolution and hypergraph attention. Pullern Recognition, 110:107637, 2021.
[4] P. Barbiero, S. Fioravanti, F. Giannini, A. Tonda, P. Lio, and E. Di Lavore. Categorical foundations of explainable ai: A unifying formalism of structures and semantics. arXiv preprint arXiv:2304. 14094, 2023.
[5] M. G. Bergomi and P. Vertechi. Neural network layers parametric spans. arXiv preprint ArXiv:2008. 00800, 2022.
[6] E. Chien, C. Pan, J. Peng, and O. Milenkovic. You allset: A multiset function framework for hypergraph neural networks. arXiv preprint arXiv:2106. 132444 2021.
[7] A. Colaprico, T. C. Silva, C. Olsen, L. Garofano, C. Cava, D. Garolini, T. S. Sabedot, T. M. Malta, S. M. Pagnotta, Castiglioni, et al. Tegabiolinks: an r/bioconductor package for integrative analysis of tega data. Nucleic acids research, (8)))e71-e71 2016.
[8] D. Croft, G. O'kelly, G. Wu, Haw, M. Gillespie, L. Matthews, M. Caudy, Garapati, G. Gopinath, B. Jassal, et al. Reactome: a database of reactions, pathways and biological processes. Nucleic acids research, (9(suppL_)::D61111>))
2010.
[9] G. S. Cruttwell, B. Gavranović, N. Ghani, P. Wilson, and F. Zanasi. Categorical foundations of gradient-based based learning. In European Symposium Programming, pages 1-28. Springer International Publishing Cham, 2022.
[10] P. de Haan, T. S. Cohen, and M. Welling, Natural graph networks. Advances in neural information processing systems, 33:3636-3646, 2020.
[11] K. Ding, J. Wang, J. Li, D. Li, and H. Liu. Be more with less: Hypergraph attention networks for inductive text classification. arXiv preprint 1xXi::2011.00385 2020.
[12] Y. Dong, W. Sawin, and Y. Bengio. Hnhn: Hypergraph networks with hyperedge neurons. arXiv preprint arXiv:2006.1223 12278, 2020.
[13] A. Dudzik, т. von Glehn, R. Pascanu, and P. Veličković. Asynchronous algorithmic alignment with cocycles. arXiv preprint arXiv:2306. /5632, 2023.
[14] A. J. Dudzik and P. Veličković. Graph neural networks are dynamic programmers. Advances in neural information processing systems, 35:20635-20647. 2022.
[15] A. J. Dudzik, T. von Glehn, R. Pascanu, and Veličković. Asynchronous algorithmic alignment with cocycles. In Learning Graphs Conference, pages 3-1. PMLR, 2024.
11

---

[16] S. Duriinck, Y. Moreau, A. Kasprzyk, S. Davis, B. De Moor, A. Brazma, and W. Huber. Biomart and bioconductor: powerful link between biological databases and microarray data analysis. Bioinformatics, 21(16):3439-33 3440, 2005.
[17] S. Durinck, P. T. Spellman, E. Bimey, and W. Huber. Mapping identifiers for the integration of genomic datasets with the r/bioconductor package biomart. Nature protocols, 4(8): 184-1191, 2009.
[18] I. Duta, G. Cassara, F. Silvestri, and P. Liò. Sheaf hypergraph networks, Advances in Neural Information Processing Systems, 36, 2024.
[19] Y. Feng, Z. Zhang, X. Zhao, R. Ji, and Y. Gao. Gvenn: Group-view convolutional neural networks for 3d shape recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 264-272, 2018.
[20] Y. Feng, H. You, Z. Zhang, Ji, and Gao. Hypergraph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 3558-3565, 2019.
[21] Fiorini, G. M. Bovolenta, S. Coniglio, M. Ciavotta, P. Morerio, M. Parrinello, and A. Del Bue. Dignet: Hyperedge classification through directed line graphs for chemical reactions. arXiv preprint arXiv:2410.0690 2024.
[22] B. Fong and M. Johnson. Lenses and learners. arXiv preprint arXiv:1903.0367/ 2019.
[23] B. Fong and D. Spivak. Seven sketches in compositionality: An invitation to applied category theory. arXiv preprini arXiv: /803:058003333 2018.
[24] B. Fong, D. Spivak, and R. Tuyéras. Backprop functor: A compositional perspective on supervised learning. In 2019 34th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS), pages 1-13. IEEE, 2019.
[25] B. Gavranović. Compositional deep learning. arXiv preprint arXiv: 2019.
[26] J. Hansen and T. Gebhart. Sheaf neural networks. arXiv preprint arXiv:2012.0633: 2020.
[27] J. Hansen and R. Ghrist. Toward spectral theory of cellular sheaves. Journal of Applied and Computational Topology, 3:315-358, 2019.
[28] J. Hansen and R. Ghrist. Opinion dynamics on discourse sheaves. SIAM Journal on Applied Mathematics, ((5):2033-2066) 2021.
[29] A. Hickok, Y. Kureh, H. Z. Brooks, M. Feng, and M. A. Porter. A bounded--coffidnnc model of opinion dynamics on hypergraphs. SIAM Journal Applied Dynamical Systems, 21(1): 1-32, 2022.
[30] I. Higgins, D. Amos, D. Pfau, S. Racaniere, L. Matthey, D. Rezende, and A. Lerchner, Towards a definition of disentangled representations. arxiv. arXiv preprint wrXiv:181..022.00. 2018.
[31] B. Hu, X. Wang, Z. Feng, J. Song. J. Zhao, M. Song, and X. Wang. Hsdn: A high-order structural semantic disentangled neural network. IEEE Transactions Knowledge and Data Engineering, 35(9):8742-8756, 2022.
[32] J. Huang and J. Yang. Unignn: unified framework for graph and hypergraph neural networks. arXiv preprint arXiv:2105.000 009560 2021.
[33] M. Kanehisa and S. Goto. Kegg: kyoto encyclopedia of genes and genomes. Nucleic acids research, 28(1):27-30, 2000.
[34] M. Kramer, J. Dutkowski, M. Yu, V. Bafna, and T. Ideker. Inferring gene ontologies from pairwise similarity data. Bioinformatics, 30(12):134-14,, 2014.
[35] A. Kratz, M. Kim, M. R. Kelly, F. Zheng, C. A. Koczor, J. Li, K. Ono, Y. Qin, C. Churas, J. Chen, et al. A multi-scale map of protein assemblies in the dna damage response. Cell Systems, 14(6):447-463, 2023.
12

---

[36] H. Kvinge, B. Jefferson, C. Joslyn, and E. Purvine. Sheaves framework for understanding and interpreting model fit. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4222-4230, 2021.
[37] J. Lee, Y. Lee, J. Kim, A. R. Kosiorek, S. Choi, and Y. W. Teh. Set transformer. 2018.
[38] Y. Lee, Lee. S. Sco, Kim, Kim, and C. Park. Capturing functional context of genetic pathways through hyperedge disentanglement. In ICLR 2025 Workshop on Machine Learning for Genomics Explorations, 2025.
[39] T. Leinster. Basic category theory. arXiv preprint arXiv:1612.09377 2016.
[40] M. Lewis. Compositionality for recursive neural networks. arXiv preprint 1rXiv:190.11072. 2019.
[41] H. Li, Wang, Zhang, Z. Yuan, H. Li, and Zhu. Disentangled contrastive learning on graphs. Advances in Neural Information Processing Systems, 34:21 144:1172--21844 2021.
[42] Y. Li, C. Gao, H. Luo, D. Jin, and Y. Li. Enhancing hypergraph neural networks with intent disentanglement for session- recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1997- 2002, 2022.
[43] Y. Li, C. Gao, Q. Yao, T. Li, D. Jin, and Li. Disenhen: Disentangled hypergraph convolutional networks for spatiotemporal activity prediction. arXiv preprint arXiv:2208.065 06794, 2022.
[44] D. Lin et al. An information- definition similarity. Icml, volume 98, pages 296-304, 1998.
[45] M. Liu and P. Thomas. Go functional similarity clustering depends on similarity measure, clustering method, and annotation completeness. BMC bioinformatics, 20(1):1-15, 2019.
[46] X. Liu, Y. Tao, Z. Cai, P. Bao, H. Ma, K. Li. M. Li, Y. Zhu, and Z. J. Lu. Pathformer: biological pathway informed transformer integrating multi-omics data for disease diagnosis and prognosis. bioRxiv, pages 2023 05, 2023.
[47] Y. Liu, X. Wang, S. Wu, and 7. Xiao. Independence promoted graph disentangled networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 4916-4923, 2020.
[48] Y. Luo. Shine: Subhypergraph inductive neural network. Advances in Neural Information Processing Systems,
2022.
[49] J. Ma, P. Cui, K. Kuang, X. Wang, and W. Zhu. Disentangled graph convolutional networks. In PMLR, 2019.
International conference on machine learning, pages
[50] S. M. Mansourbeigi. Sheaf Theory as a Foundation for Heterogeneous Data Fusion. PhD thesis, Utah State University, 2018.
[51] G. Mercatali, A. Freitas, and V. Garg. Symmetry-induced disentanglement on graphs. Advances in neural information processing systems, 35:31497-31511 2022.
[52] C. H. Mermel, S. E. Schumacher, B. Hill, M. L. Meyerson, R. Beroukhim, and G. Getz. Gistic2. O facilitates sensitive and confident localization of the targets of focal somatic copy-number alteration in human cancers. Genome biology, 12:1-14, 14, 2011.
[53] M. Mounir, M. Lucchetta, C. Silva, C. Olsen, G. Bontempi, Chen, H. Noushmehr, A. Colaprico, and E. Papaleo. New functionalities in the tegabiolinks package for the study and integration of cancer data from gdc and gtex. PLoS computational biology, 100670. 2019.
[54] L. Neuhäuser, M. T. Schaub, A. Mellor, and R. Lambiotte. Opinion dynamics with aulti-body interactions. In International Conference on Network Games, Control and Optimization, pages 261-271. Springer, 2021.
13

---

[55] D. Nishimura. Biocarta. Biotech Software & Internet Report: The Computer Software Journal for Scient, 2(3):117-120, 2001.
[56] J.H. Oh, W. Choi, F. Ko, M. Kang, A. Tannenbaum, and J. O. Deasy. Pathenn: interpretable con- volutional neural networks for survival prediction and pathway analysis applied to glioblastoma. Bioinformatics, 37(Supplement
2021.
[57] Y. Qin, C. F. Winsnes, E. L. Huttlin, F. Zheng, W. Ouyang, J. Park, A. Pitea, J. F. Kreisberg, S. P. Gygi, J. W. Harper, et al. Mapping cell structure across scales by fusing protein images and interactions. bioRxiv, pages 2020-06, 2020.
[58] J. Reimand, R. Isserlin, V. Voisin, M. Kucera, C. Tannus-Lopes, A. Rostamianfar, L. Wadi, M. Meyer, Wong, C. Xu, et al. Pathway enrichment analysis and visualization of omics data using& g: Profiler, gsea, cytoscape and enrichmentmap. Nature protocols, 14(2):482-517, 2019,
[59] K. Roth, M. Ibrahim, Z. Akata, P. Vincent, and D. Bouchacourt, Disentanglement of correlated factors via hausdorff' factorized support. arXiv preprint arX1iv:211000334, 2022.
[60] F. Sanchez- Vega, M. Mina, J. Armenia, W. K. Chatila, A. Luna, K. C. La, S. Dimitriadoy, D. L. Liu, H. S. Kantheti, S. Saghafinia, et al. Oncogenic signaling pathways in the cancer genome atlas. Cell, 173(2):321-33) 2018.
[61] C. F. Schaefer, K. Anthony, S. Krupa, J. Buchoff, M. Day. T. Hannay, and K. H. Buetow, Pid: the pathway interaction database. Nucleic acids research, (
2009. [62] A. Sheshmani and Y.-Z. You. Categorical representation learning: morphism is all you need. Machine Learning: Science and Technology, 3(1):015016, 2021.
[63] D. Shiebler, B. Gavranović, and Wilson. Category theory in machine learning. arXiv preprint arXiv:2106.07032, 2021.
[64] T. C. Silva, A. Colaprico, C. Olsen, F. D' Angelo, G. Bontempi, M. Ceccarelli, and H. Noushmehr. Tega workflow: Analyze cancer genomics and epigenomics data using bioconductor packages. F1000Research, 5, 2016.
[65] R. Stoney, D. L. Robertson, G. Nenadic, and J.-M. Schwartz. Mapping biological process relationships and disease perturbations within a pathway network. NPJ systems biology and applications, 4(1):22 2018.
[66] H. Su, S. Maji, E. Kalogerakis, and Lealeed-Miller. Multi-view convolutional neural networks for 3d shape recognition. In Proceedings of the IEEE international conference on computer vision, pages 945-953, 2015.
[67] L. Sun, S. Ji, and J. Ye. Hypergraph spectral learning for multi-label classification. In Pro- ceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 668-676, 2008.
[68] P. Veličković and C. Blundell. Neural algorithmic reasoning. Putterns, 2(7), 2021.
[69] L. Vepstas. Sheaves: a topological approach to big data. arXiv preprint urXiv:1901.0134/
2019.
[70] P. Wang, S. Yang, Y. Liu, Z. Wang, and P. Li. Equivariant hypergraph diffusion neural operators. arXiv preprint arXiv:2207.06680 2022.
[71] T. Wang, W. Shao, Z. Huang, H. Tang, Zhang, Z. Ding, and K. Huang. Mogonet inte- grates multi-omics data using graph convolutional networks allowing patient classification and biomarker identification. Nature Communicationss 12(1):3445, 2021.
[72] Y. Wang, Q. Gan, X. Qiu, X. Huang, and D. Wipf. From hypergraph energy functions to hypergraph neural networks. In International Conference on Machine Learning, pages 35605- 35623. PMLR, 2023.
[73] J. N. Weinstein, E. A. Collisson, G. B. Mills, K. R. Shaw, B. A. Ozenberger, K. Ellrott, I. Shmulevich, C. Sander, and J. M. Stuart. The cancer genome atlas pan-cancer analysis project Nature genetics, 111-1001 2013.
14

---

[74] S. F. Windels, N. Malod- Malod-Dognin, and N. Pržulj. Identifying cellular cancer mechanisms through pathway-driven data integration. Bioinformatics, 38(18):4344 43511, 2022.
[75] K. Xu, J. Li, M. Zhang, S. S. Du, K.-i. Kawarabayashi, and S. Jegelka. What can neural networks reason about? arXiv preprint arXiv: 13211, 2019.
[76] N. Yadati, M. Nimishakavi. P. Yadav, V. Nitin, A. Louis, and P. Talukdar. Hypergen: A new method for training graph convolutional networks on hypergraphs. Advances in neural information processing systems, 32, 2019,
[77] Y. Yang, Z. Feng, M. Song, and X. Wang. Factorizable graph convolutional networks. Advances in Neural Information Processing Systems, 33:20286-20296, 2020.
[78] G. Yu. Gene ontology semantic similarity analysis using gosemsim. Stem Cell Transcriptional Networks: Methods and Protocols, pages 207-215, 2020.
[79] G. Yu, F. Li, Y. Qin, X. Bo, Y. Wu, and S. Wang. Gosemsim: an package for measuring semantic similarity among go terms and gene products. Bioinformatics, 26(7):976-978, 2010.
[80] G. Yu, L.-G. Wang, Y. Han, and Q.-Y. He. clusterprofiler: an package for comparing biological themes among gene clusters. Omics: journal of integrative biology, 16(5):284-287, 2012.
[81] Y. Yuan. A categorical framework of general intelligence. arXiv preprint arXiv:2303.04571 2023.
[82] Y. Yuan. On the power of foundation models. In International Conference Machine Learning, pages 40519-40530. PMLR, 2023.
[83] M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. J. Smola. Deep sets. Advances in neural information processing systems, 30, 2017.
[84] Y. Zhang and M. Sugiyama. A category-theoreticn meta-analysis of definitions of disentan- glement. In International Conference on Machine Learning, pages 41596-41612. PMLR, 2023.
[85] T. Zhao, X. Zhang, and S. Wang. Exploring edge disentanglement for node classification. In Proceedings of the ACM Web Conference 2022, pages 1028-1036, 2022.
[86] Zheng, M. R. Kelly, D. J. Ramms, M. L. Heintschel, K. Tao, B. Tutuncuoglu, J. J. Lee, K. Ono, H. Foussard, M. Chen, et al. Interpretation of cancer mutations using multiscale map of protein systems. Science, xebbf56)) 2021.
15

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transpareccy, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant
information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because would be too computationally expensive" or "we were unable to find the license for the dataset we used"). ))" In general, answering binary way,
"[No] " or "[NA] is not grounds for rejection. While the questions are phrased acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification elaborate. All supporting evidence can appear either in the main paper the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have clearly mentioned our scope(disentangle, hypergraph, interaction context) and summarized contributions
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
Justification: Appendix G
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specification,
16

---

asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g. if the approach was only tested few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't I acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: Although our paper uses concepts in category theory, it not about theoretical result. We used existing concepts to create our model.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-eeferenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by
formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experi- mental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: Appendix B Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others replicate the model with the same dataset, access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access hosted model (e.g., in the case of a large language model), releasing of model checkpoint, or other means that are appropriate to the research performed.
provide
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
17

---

the case of closed- (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
cc) the contribution is a new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results way reproduce the model (e.g.. with an open-source dataset or instructions for how to construct the datasct).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome to describe the particular way they provide for reproducibility. source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, described in supplemental material? Answer: [Yes]
Justification: We provide instructions for downloading original data, preprocessing codes at ht https://anonymmrn
40pen. wwwteeeeceeceeceeaeetteeeeee.eet httpe//r///aaaaaaaaaaeeaaaaaaaea HNN well as preprocessed data at
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nip) cc/ cc/public
/guides/Ce "guides/CodeSubee /oodeSubmi is ionPol l Licy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed run to reproduce the results. See the NeurIPS code and data submission guidelines (https: :
phttpss//nipp: /nips..p...../// cc/ pu sicc/guides /tt////uuuiiisiioooo woooooiiiccy licy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why. At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Appendix C.5 Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material,
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: For the most of the tables, provide standarad deviations. Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main
claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
18

---

The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors). It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experi- ments?
Answer: [Yes]
Justification: Appendix C.6
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimen- tal runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neuri: Answer: [Yes]
cc/ public/EthicsGull
inns?
Justification:
We have read Code of Ethics. Our work does not violate the contents in the Code of
Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics. If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethies.
The authors should make sure to preserve anonymity (e.g., there a special consideration
due laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Appendix G
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g.. deploy- ment technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied par- ticular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used
19

---

as intended but gives incorrect results, and harms following from (intentional or unintentional)
misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: This paper does not contain such risks Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary
safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes]
Justification: For biological datasets, we provided links(URL), their paper in Appendix B.2 and acknowledgement the main text.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets. paperswi paperswithcode, com/ datasets has curated licenses for some datasets. Their licensing guide help determine the license of a dataset. For existing datasets that are re-packaged, both the original license and the license the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Yes, we have well described how to use our code the repository. Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodll as part of their sub-
missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects Question: For crowdsoureing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
20

---

Justification:
This paper does not contain crowdsourcing or human subject research.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human
subjects.
Including this information in the supplemental material is fine, but the main contribution of the paper involves human subjects, then as much detail possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper only uses publicly available datasets.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human
subjects.
Depending on the country which research conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do include any information that would break anonymity (if applica- ble), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standess standdadd component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: Our model does not use LLM.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https://neurip) /ttp:///ppccccooceee//e///////// LLLM) for what should should not be described.
21

---

Appendix
Category Theory
24
A.I Category Theory
24
A.2 Category
24
A.3 Functor
25
A.4 Natural Transformation
25
A.5 Product
26
A.6 Derivation of Figure (b) from Figure 3 (a).
26
Dataset and Experiment Details
27
B.1 Statistics Cancer Subtype Classification Dataset
27
B.2 Preprocessing Cancer Subtype Classification Dataset
Del.............................
27
B.3 Experiment Details of Capturing Context Types
28
B.4 Selecting Pathways with SHAP values
29
B.5 Calculating Functional Similarity between Pathways
30
B.6 Assigning Pathway Type with CliXO
30
B.7
Calculating Functional Similarity between clusters
31
Implementation Details
32
C.I Baselines and their variants
32
C.2 Factor Discrimination Loss
32
C.3 Loss used for training Leask
32
C.4 Factor Encoder
32
C,5 Hyperparameter search space
33
C.6 Environment for experiment
34
Standard Hypergraph Benchmark dataset
35
D.1 Statistics Standard Hypergraph Benchmark Dataset
35
D.2 Node Classification Benchmark Datasets
36
D.3 Training with only 5% of data
36
E Ablation studies and Hyperparameter sensitivity
37
E.1 Selecting Alternative Branch
E.2
Natural-HNN without naturality constraint
37
E.3
Hyperparameter Analysis
38
Additional Experiment Result
F11
Computational Complexity
39
F22
Scalability Analysis (training time)
39
22

---

F33
Generalization power of Natural-HNN
39
F.4
Captured Context in CESC
41
F55
Cancer Subtype Classification (Micro F1)
F6
Chemical Reaction Classification (Hyperedge Classification)
42
F.7
Reliability of Natural-HNN in Biology
42
G
Limitations, impacts and Future Work
45
G.I Broader Impacts
45
G.2 Limitation
45
G.3 Future Work Model for Graph Neural Network
45
G.4 Future Work Hyperedge-Nee Node co-disentanglement
45
23

---

A Category Theory
A.1 Category Theory
Category theory [23, 39] is widely used to represent and analyze the structure or relation of a system. Instead of focusing on the details, category theory takes bird's eye view to global structure and patterns, Recently, category theory is used to explain learning mechanism of machine learning methods [5, 40, 25, 22, 24, 9, 63, 10, 4, 82, 13, 14, 81J. In this paper, we only use simple, fundamental concepts of category theory: category, functor, natural transformation and product.
A.2 Category
f
ida
F(A)
F(ida)
ida
B
idB
P(
9
fig
F(B)
F().F
fig
F(J)
C
idc
FUda
idc
F(C)
Fldd
Category D
Category E
(a) Category
(b) Functor
Figure 9: Category and Funetor
category C is contains four components: collection of objects, morphisms, composition rule and identities.
Collection of objects Ob(C) (ex [A, B, C} in Figure 9 (a)) For every pair objects A, E Ob(C), there exists set Home(A, B). Element of the is morphism and is denoted as: AA - B.
For every three objects A, B, C Ob(C), morphisms Home B) (i.e. f A B) and C), composition rule holds f g gof f Homc(A, C)
g E Home(B, C) (i.e. g
For every object A E Ob(C). there exists an identity morphism id, E Home(A,A) satisfying the following id, a. = f g idg for morphism
Fig. (a) shows an example of a category with three objects (A, C). For each object, there an identity morphism (id₁, idg, idc). For every object pair, there is morphism (f, &. g) with composition rules.
One of the most important categories Set. In Set, the objects are sets and morphisms are functions mapping two sets. The composition rule is satisfied since composition of two functions becomes a function. Another important category is category of relations, which is denoted as Rel. The objects of Rel are sets and relations R = A x B are morphisms between objects A and B. Partially ordered set or poset can considered as a category where objects are sets and morphisms are partial orders <. Since partial order is a kind of relation, we can consider this category is kind of Rel.
In Section 3, analyzed hypergraph message passing framework, and found that, nodes (con- sidering node as set) are included in hyperedges, hypergraph message passing framework has poset structure with inclusion maps between them. We will define it PISet, category for poset with inclusion morphisms (object is a set, morphisms are inclusions). Since inclusions are partial orders, which is also relation, we can consider PISet as kind of Rel category.
We define representations and their (linear or non-linear) transformations are morphisms. We will call this 'category of Deep Learning Representations' and denote DLRep.
category, similar to the in prior work [62], such that objects vector
Two notations f'g 8g and g of have the same meaning "applying/ first, and then applying g"

---

G
A
F(A)
G(A)
aA
F
|GG
F()
F
B
F(B)
G(
CategoryE Figure 10: Natural transformation. Identity morphisms are omitted in the figure for simplicity,
Category
G
A.3 Functor
Functor structure preserving map between categories. Objects and morphisms in one category are mapped to objects and morphisms in different category, respectively. Figure (b) shows an category Each object category (i.e., A, C)
example of functor mapping from category mapped to objects category E (i.e., F(A), F(B), F(C)). The morphisms, including identity morphism, and their compositions in category ID (i.e., id,, idg. idc..ff.fff g) also mapped to morphisms in category (i.e., F(id)), F(idg) F(idc), F(f), F(g), F(f) [ F(g)). In metaphorical sense, functors serve bridges that connect two distinct realms while maintaining identical compositional structure?.
One example can be a functor mapping from Set to DLRep. Each set (object) in Set is mapped to vector representation (object) in DeR Functions (morphisms) in Set are mapped to transformations (morphism) between vector representations in DLRep. This functor is related representation learning, since entities (i.e. concept or set) are mapped to their vector representations preserving their compositional structure (relation).
A.4 Natural Transformation
Given two functors mapping from one category to another category. i.c., F and DD E. natural transformation is way of relating these two functors using morphisms in target category E. Specifically, for each object ID there exists a morphism as F(A) G(A) in E. The natural transformation must satisfy the following condition. For every morphism f A - in D,
F(f)sa== 0 3
must hold. This condition is called the naturality condition. Figure 10 shows example of natural transformation. Functors and map objects and morphisms category to category E. Natural transformation F G maps F(A) and F(B) with ag and maps G(A) and G(B) with 0g. The objects and morphisms mapped by two functors as well as natural transformation a all belong to the category E. Thus, natural transformation can be seen as a way of relating different views using morphisms in E..
'The typical example of deep learning method using this concept is sheaf neural network [26], motivated from cellular sheaf [27]. There also numerous studies in data science with similar perspective [50, 69, 36). One typical example of deep learning method using this concept is Natural Graph Networks [10].
25

---

Y
Y
X1
P1
X1
x
X2
P2
X2
f1
f2
f
A
f2
f2)
Fi
f1
f1
f2
coproduct is the dual a product, which can be obtained by reversing the direction of the arrows. Let C be category. For two objects X1, X2 Ob(C), one can define coproduct of two objects X, 11X, with morphisms i : X1 X1 x X2 and i2 X2 X, x X2 which are called injections. Then, the composition of objects in Figure 12 must be satisfied. Given object Y Ob(C) with two morphisms that satisfies the X1 x X
X1
P2
2
X1
X1
X2
Y1
Y1 Y2
42
Y2
Figure 11: Product
Figure 12: Coproduct
Figure 13: Product of morphisms.
A.5 Product
Product (Objects) Let C be category. For two objects X, E Ob(C), one can define product of two objects X X₂ with morphisms pi X, X2 X₁ and P2 X1 X2 X2 which called projections. Then, the composition of objects in Figure 11 must be satisfied. Given object Y Ob(C) with two morphisms fi Y X, and Y X2, there exists unique morphism called 'paring' [84] (fi.f)) - X1 x X2 that satisfies the composition f1 (fi.fa) P1 and f2 (f1 P2
Coproduct (Objects)
fi :X Y and f2 X2 Y. there exists a unique morphism f2 X1 uX2
composition fi i ff and f2 = 12 [f.ffll.
Product of Morphisms
Let C be category. For objects X,, X2, Y, Y2 E ob(C) and morphisms fi : X, Y and f2 X2 Y2, we can define product of morphisms fi x f2 X2 x X2 - Y1 x Y2 = (pi 3/1, 8.f2) satisfying
the compositional structure shown in Figure 13.
A.6 Derivation of Figure 3 (b) from Figure (a).
Since we are dealing with the commutative diagram between entangled and disentangled representa- x x" 11"", I1dis in Figure 3 (a). Since the morphism
tions, we focus on the morphisms between X"
(i..., pdlis xfd pdii in the disentangled representation is the product of factor-specific morphisms glia and ydix we apply the diagram at Figure 13 to f x fdix Then we can get the morphisms between Hdiss Hdis .I - xdir xdi in the Figure 14 where his H x H and x x x x Note that morphisms between H Hels Hdi Hdl are products shown in the Figure 11. If we extract components Figure 14 that are related to factor 'c' and entangled representation, have the diagram in Figure 3 (b).
H
0h
X
Hdis
Hdiss
Hdis
Og
Oxy,c
ox
14
X
ydis
d.
dis
4a
X
ydis
Figure 14: Derivation of Figure 3 (b) from Figure 3 (a).
26

---

B Dataset and Experiment Details
Note that KIPAN and NSCLC are known to be cancers where subtypes can be easily classified based on features alone [71, 56]. Because these datasets offer limited value for evaluating model performance, we excluded them in this study. The results of Natural- -NNN and baselines for these datasets (i.e., KIPAN and NSCLC) are reported [38].
B.1 Statisties: Cancer Subtype Classification Dataset
The statistics of cancer datasets shown in the Table 2. Note that every hypergraphs in all 6 cancers have 1497 pathways (hyperedges) and 11552 genes (nodes) with feature dimension. The degree statistics of cancer dataset is shown in the Table 3. When converted to graph with star-expansion, the graph contains 98013 edges. When converted to a graph with clique-expansion, the graph contains 101 14890 edges. Thus, converting the hypergraph into a graph with clique-expant expanssionn requires large computation during message passing. The downloading and preprocessing details are provided in Appendix B.2.
Table 2: Statistics of 6 cancer datasets used for cancer subtype classification task.
datasct
summary
class distribution(county
BRCA
5 class, 769 hypergraphs
Normal-like 33, Her2 44, Basal-like 134, LumB 143, LumA 415
STAD
5 class, 341 hypergraphs
CIN 200, EBV 29, GS 46, MSI 59, HMM-NV
SARC
4 class, 257 hypergraphs
LMS 104, MFS/UPS 75, DDLPS 57, Other
LGG
2 class, 503 hypergraphs
G2 242, G3 261
HNSC
2 class, 507 hypergraphs
HPV- 411, HPV+ 96
CESC
2 class, 280 hypergraphs
AdenoCarcinoma 46, SquamousCarcinoma 234
Table 3: statistics of hypergraphs in cancer subtype classification task
min
median
mean
max
std
node degree
2
5
8.485
239
13.301
hyperedge degree
13
35
57
84.720
B.2 Preprocessing: Cancer Subtype Classification Dataset
The overall procedure was adopted from Pathformer [46]. However, statistics of the data can be slightly different due the difference of time which the data was downloaded.
Creating Hypergraph
We downloaded pathways from several pathway databases including KEGG [33], PID [61], Reactome [8] and Biocarta.[55]. The pathways were selected based on their size and overlap ratio with other pathways. These two conditions must be considered as extremely large pathways do not represent specific functions but rather general functions, 2) small pathways complicate interpretations 3) overlapping pathways cause redundancies. The more detailed explanations can be found in [58]. Pathways with too small or too big size or large overlaps are excluded. A specific threshold was chosen following the Pathformer.
Generating Hypergraph Labels
For BRCA and STAD, we gathered cancer subtypes from TCGA [73] using TCGAbiolinks, [7, 64, 53] library. For the rest of 4 cancer datasets we downloaded cancer subtypes from Broad GDAC
Firehose (https:/ http::::///.c...rrdiissstttttttt
'Pathformer used labels from pan cancer atlas study [60] for HNSC, CESC and SARC. However, we decided the in Broad GDAC Firehose since it easier process the same data
27

---

Generating Node Features
We gathered mRNA/miRNA expression, DNA methylation" 10 DNA copy number variation (CNV)" using TCGAbiolinks. Gene lengths were acquired from biomaRt R package [17. 16]. The procedure of processing each data with Gistic2 [52], normalization by TPM are adopted from Pathformer. At the end of the processing step, calculate statisties (mean, min, max, count) of modalities as values for each feature dimension.
B.3 Experiment Details of Capturing Context Types
To check whether HNNs could capture functional semantics of pathways (i.e. interaction context of hyperedges), we need functional context annotations for each hyperedge. However, there is no data that annotates the functional semanties of genetic pathways. Instead, to assign canction-related hyperedge types or labels, we clustered pathways based on the functional similarity between pathways, which can be calculated with computational biology method.
Now that we have obtained the hyperedge types, one might think can simply check whether there is a one-to-one correspondence between hyperedge types and factors. However, there is another issue: hyperedge types themselves can be similar to each other. In other words, due to functional correlations between hyperedge types, a single factor may appear not in just one hyperedge type but across multiple hyperedge types. Therefore, examining the relationship between factors and hyperedge types alone makes it difficult to determine whether disentanglemer we can indirectly verify that factors are related to the functional context by checking whether the functional similarity between hyperedge types aligns with the functional similarity inferred from the model's 'ss factor relevance. Thus, we evaluated whether the model effectively captured the functional context by comparing the ground truth functional similarity between hyperedge types (i.e., clusters) with the similarity inferred from the model. If the functional similarity predicted by the model shows some correlation with the functional similarity defined as ground truth, we can say that the model has captured the functional context. We do not directly compare the exact values of prediction and the ground truth since the way of calculating the value is different in prediction (calculation based on relevance scores ax,)) and ground truth (algorithm used in computational biology). Therefore, instead of comparing exact values, we assessed it based on the similarity of patterns observable in a heatmap, as shown in Figure 5.
has captured the functional context. Instead,
In summary, our experiment involves selecting pathways to be analyzed, collecting imnctio-rrlatte information for each pathway. measuring the functional similarity between pathways based on the collected information, and performing clustering based on this similarity. Afterward, we compute the similarity between clusters to derive the ground truth similarity, which is then compared with the model's predictions. Thus, in order to perform the experiment, we need consider the followings: 1) Which pathways need to be analyzed? 2) How to get ground truth pathway functions? (i.c. How to get function related information?) 3) How to calculate ground truth functional similarity between pathways 4) How to cluster functionally similar pathways in a reliable manner 5) How to measure ground truth cluster similarity and how to predict cluster similarity with model outputs.
Which pathways need to be analyzed? There are two reasons behind selecting pathways: : 1) Since CliXO algorithm (Appendix B.6) used for clustering pathways takes a lot of time, the number of pathways to be analyzed must be reduced. 2) The ground truth functional similarity (Appendix B.5) contains vast biological context derived from biological domain knowledge or researches, which might not be present in our dataset. Since our dataset contains only cancer-specific information, there is no way to capture non-existing context (contexts that are not related to cancer) without external supervision. Thus direct comparison between the ground truth and our result is impossible. The most ideal way for fair comparison would be selecting the ground truth that is only relevant to our dataset or task. However, it is impossible since there are no databases with annotated context (cancer or environment) specific pathway functionalities. An alternative way was selecting the pathways that were informative or important in the decision of the model. If a model can correctly capture functional context of pathways, since pathway functions are highly related the cancers [74, 65), informative pathways (for the model prediction) are the pathways that contain cancer contexts. Since only need to check whether functional context correctly captured under the cancer specific
10 we do not use promoter methylation
but we do not use gene level CNV
28

---

circumstances or condition, by selecting those pathways, we can compare functional similarities that are specific to our data or cancer' 'he details for selecting pathways are described in Appendix B.4.
How to get ground truth pathway functions. Since there is no database that annotates functional similarity scores between pathways, we rely on methods used in computational biology. Hence, we need to get pathway function information. Similarity calculations and clusterings are based on the annotation of pathway functions. The details are described in Appendix B.5.
How to calculate ground truth functional similarity between pathways. Based on the functions of pathways, pathway functional similarity can be calculated. The calculated similarity will be used in clustering and generating ground truth functional similarity between clusters. The details are dealt in Appendix B.5.
How to cluster ffunctionalyy similar pathways in a reliable manner. With functional similarity between pathways, we can cluster functionally similar pathways with CliXO algorithm. The details and example results are shown in Appendix B.6.
How to measure ground truth cluster similarity and how to predict cluster similarity with model outputs. Finally, we need to devise way to measure the similarity between clusters based on the model outputs. Also, we need to measure ground truth functional similarity between clusters. The details are described in Appendix B.7.
In summary, the procedure of experiments can be described as follows. First, we get functional annotation of pathways (hyperedges). Second, we calculate functional similarity between pathways based on annotations. Third, we select pathways to be analyzed based on the model output. Fourth, we cluster the selected pathways with pathway similarity. Finally, we calculate the predicted functional similarity between clusters from model prediction and compare that with the ground truth cluster similarity.
B.4 Selecting Pathways with SHAP values
To select pathways that were the most informative for prediction, we provide the final representation of pathways generated by model. layer classifier (MLP) that predicts labels from final representation as well as labels to the DeepExplainer to get SHAP values. Then we select top-k pathways based on the SHAP value. Note that only small number of pathways are relevant to the task as shown in Figure 15. This is due to the fact that not all pathways are related to very specific type of cancer. Although Natural-Ha HNN and HSDN both use the same number of pathways (top-k), the pathways selected by each model can be different. This also leads to different number of clusters in Figure 5 and 18.
250
250
200
200
150
175
SHAP voloo
SHAP vell
150
100
125
75
0
200
400
600
800
1000
1200
1400
10
15 ranidng
20
25
30
ranking
(a) SHAP value for all pathways
(b) SHAP value for top-30 pathways
Figure 15: SHAP value distribution Natural-HNN on BRCA dataset. We sorted pathways with SHAP value. X axis represents ranking of pathways and Y axis represents SHAP value for pathways with corresponding ranking.
On the other hand, if the model could not correctly capture pathway functionalities, cancer irrelevant pathways will be selected and will have differrnt result from the ground truth in section 5.3
29

---

Pathway cluster G
Fathway P
Fune
P530
Puu
Past
Faz
F121
Ezz
G
Pirt
66
4
Praw
Pyrs
Paiz
Pirt
Fo
Pizse
P1332
For
Plane
(a) Clustering result for (SHAP value) top 15 pathways Natural-HNN @
(b) Clustering result for (SHAP value) top 15 pathways of HSDN @ BRCA
Fuz
C2
Frest
Ploc7
Pist
Er
Pirst
Press
Plara
C
Pixas
Pazy
Pias
51
Puoz
Pasa
Er
Py14
Fizzs
(c) Clustering result for (SHAP value) top 15 pathways of Natural-HNN @ CESC
(d) Clustering result for (SHAP value) top 15 pathways HSDN @ CESC
Figure 16: The result of applying CliXO algorithm to 10 pathways of Natural- and HSDN on BRCA and CESC. The pathway number denotes the index of pathway in our dataset (hyperedge index).
B.5 Calculating Functional Similarity between Pathways
This process consists of two steps: 1) assigning pathway level function to pathways and 2) calculating functional semantic similarities between pathways. For both two steps, we adopted the most frequently used and verified methods through several studies. For the assignment pathway functions, we use GO enrichment analysis. Gene ontology (GO) [2, 1] is functional annotation of genes that has hierarchical structure. Note that, however, the hierarchical structure of functional annotations is close to directed acyclic graph (DAG) rather than a tree-like hicrarchical structure. As an example, we can see DAG structure in the result of CliXO algorithm in the Figure 16. We can computationally annotate pathway functions with GO terms using GO enrichment analysis. We use 'enrichGO' function provided by R package clusterProfiler [80], with pvalue of 0.01 followig the paper [65]. Then we selected the most specific GO terms with set cover algorithm proposed in [65] to assign pathways precise representation of their functions.
The next step is calculating functional semantic similarities between pathways. We used Lin's method [44] with best matching average (BMA) as the combination was proven to perform well with CliXO and proven be robust in incomplete annotation cases in [45]. We used mgoSim function in R package GOSemSim [79, 78] for the calculation of Lin's method.
B.6 Assigning Pathway Type with CliXO
To cluster functionally similar pathways, adopted CliXO [34]. It originally designed to cluster gene function annotations (GO) and has been used in multiple biological studies[35, 57]. However, it can also be effectively applied to higher functional semantics such pathways in [86]. We used official implementation of CliXO 1.0 our research. We used the following 4 values as 0.1, b 0.6. = 0.005, 0.2.
hyperparameter of CliXo
Since CliXO can cluster functionally similar pathways, we can assign interaction types pathways by assigning them to the cluster. Figure 16 shows the result of applying CliXO for top-15 pathways selected by Natural-HNV or HSDN for BRCA as well CESC. Unlike other hierarchical clustering based methods, CliXO created clusters having DAG structure. Considering that GO also has DAG structure, CliXO can be seen natural way of reflecting complex structure relations in biology.
30

---

B.7 Calculating Functional Similarity between clusters
Ground Truth Given pair of clusters, calculating functional similarity between them is simple. We average the similarity of all possible pathway pairs belonging to different clusters to get functional similarity between clusters.
Model's prediction If model correctly captures functional context of pathways, then the relevance scores (a.) of two similar pathways must similar for all factors. Thus we define the similarity between pathways as T+]|,,-,īTl where a; = [at ..... oz is factor vector pathway (hyperedge)
e;. The cluster similarity can be calculated in the same way as in the ground truth case. We average the similarity all possible pathway pairs belonging to different clusters to get functional similarity between clusters.
31

---

C Implementation Details
In Appendix C.1, we describe some implementation details baselines and their variants, which can be different from official implementations. From Appendix C.2 to C.5, we describe implementation details for the components of Natural-HNN.
C.1 Baselines and their variants
We implemented HyperGAT based on the paper as its official implementation different from what explained in the paper. Moreover, the original version of SHINE and HyperGAT do not involve multihead attention, we implement it for fair comparisons. For SHINE, we also implemented two versions, without using Lug and the other with Lwy which is loss introduced by the paper for the purpose of making node representations to similar if the nodes are included in the same hyperedge. However, we did not use the version with Lay in cancer subtype classification task since the loss converts a hypergraph to a graph using clique expansion, which causes tremendous computational cost.
C.2 Factor Discrimination Loss
We defined a factor discrimination loss Las similar to the one used in [85]. In order to promote factors to contain different information, we use factor classifier implemented with one layer MLP. Each factor representation of every hyperedge will be given as input to the factor classifier. The classifier needs to identify which factor the factor representation belongs. If the classifier can correctly identify the factor with factor representation, i.e. if factor representations of two different factors of a hyperedge are distinguishable, is highly likely that factors contain different information.
Specifically, we can calculate the loss by creating pseudo labels. For each factor representation of each hyperedge ),,,), we assign pscudo label Y = k. Then the loss can be defined as follows:
K
K
Las = Σ CEEE
1 = (0logssofmax(MMP()))
This loss is applied to each layer of Natural-HNN, As described in Section 4.4, the final loss would be L. Linst + X Lais- As mentioned before, Lass an optional part of our model. The hyperparameter search space for X is provided in Appendix C.5
C.3
Loss used for training feask
After the final message passing layer of Natural-HNN, we get the final node embeddings Z..- The classifier Natural-HNe will predict labels P. e R where denotes the number of classes. In other words, Pv..c denotes the probability that node v, has class e as answer. If we denote 4, as the label (one-hot vector) for node V/, the task loss can be calculated with cross-entropy loss.
M C
Cusk = - Σ
b log(P,,c)
i=c=
Note that, hyperedge embedding of the final layer instead of node embeddings for cancer subtype classification task.
C.4 Factor Encoder
In Section 4, we explained that we use K number of MLPs to get K factor representations. The resulting factor representation layer is given as d. When implementing the factor encoder as code, we use single MLP that outputs vector with size d. Note that applying K different MLPs (with output vector size d/ K) is the same as applying one MLP (with output vector size d) and chunking the vector to smaller ones with size d/ K. (i.e. First d K values corresponds to the 1" factor representation, and following d/ K values
vector with size d/K when desired output representation size of
32

---

Table 4: Hyperparameter search space in standard benchmark dataset. f: MLP layers used in AltDeepSets, AlSSe(Transfoer, ED-HNN, ED-HNNII
models
1cl
classäfier dim
bead (factor)
i MLP Jayer
Mor Eae
Total
HGNN
HCHA
32
HNHN
UniGCNII
AlDDeepSets
1.2
14,1222566512
120
AllSeTTunsformer
1,2
64,128,566,512
1,2,488
1,2
1280
HyperGATT
1,2.4.8
SHINE
1,2,188
128
HSDNN
1,2,4.8
0.0001, 0.0005, 0.001, 0.005, 0.01, 0.05, 0.1
896
ED-HNN
1.2
64,128,256.512
10.1.2]
ED-HNNII
1.2
61.128,256.512
[0,1.2]
[1.2] [0.1.2]
2880
Nitural-HNN
2A8
96
Natural-ENNN- Ea-
2,4.8
0.0001. 0.0005, 0.001, 0.005, 0.01, 0.05,
672
Table 5: Hyperparameter search space in cancer subtype classification task. f: MLP layers used in AltDeepSets, AlSSetTransforer ED-HNN, ED-HNNII
models
head (factor)
ML. layer"
for Cake
Total
HGNN
HCHA
HNHN
UniGCNII
AllDeepSets
48
AllSeTTransformer
1,2,4,8
1,2
192
HyperGAT
1,2,4,8
96
SHINE
1,2,4,8
96
ED-HNN 1,2,4,8
HSDN
0.0001, 0.0005, 0.001, 0.005, 0.01, 0.05, 0.
672
[0.1] x [1] x [0.1]
96
ED-HNNII 2,4.8
[0.1] x [1] [0.1]
96
Natural-HNN
72
corresponds the nd factor representation and so on.) The nonlinear activation function we used for factor encoder is hyperbolic tangent (tanh).
C.5 Hyperparameter search space
We report the hyperparameter search space of each model in standard benchmark dataset as well as cancer subtype classification task. We used Adam optimizer for Natural-HNN, For the baselines, we closely followed optimizers or schedulers they used in their paper. Table 4 and Table 5 shows the hyperparameter search space in the standard benchmark dataset and cancer subtype datasets respectively. Total' denotes the number of all possible hyperparameter combinations that each model needs search. 'cl' denotes the number classifier layers. When the number of classifiers larger than 1, those models have an additional hyperparameter that decides the hidden dimension of the classifier. & MI.P layer denotes the number of layers in MLP that was used in SelDeepSets, Tanseerrrn,sorrrr, FD-HNN, ED-HNNII, In the case ED-HNN and ED-HNNII, there were three types of MLPs and each MLP could have different number of layers. À for Cai is hyperparameter that changes the reflection ratio of the factor discrimination loss.
For standard hypergraph benchmark datasets, we used [64, 128, 256, 512] as hidden dimension and [0.1, 0.01, 0.001, 0.0001] as learing rate. For weight decay, used [0. le-5]. We fixed the number of layers to 2, except for HSDN, because HSDN uses only a single layer. Generally, we used 0.5 as dropout. (If the paper of model specified dropout to specifie value, used the value following the paper.) As we can see, our model generally has a small hyperparameter search space comparable to GAT (when not using L. Although ED-HNN and ED-HNNII had good performance on standard hypergraph benchmark datasets, they had to rely on very large hyperparameter search space.
For cancer subtype classification tasks, we used [16, 32, 64] as the hidden dimension and [0.1, 0.01, 0.001, 0.0001] learning rate. For weight decay, we used [0, 1e-5]. We fixed the number of layers to 2, except for HSDN, because HSDN uses only a single layer. During training, we set 50 as the batch size. Generally,
used 0.5 as dropout. (If the paper of model specified dropout to specific value, we used the value following the paper.) Since we fixed the number classifiers to I, the hyperparameter search space of some models are largely reduced when compared to the node
33

---

classification task. For ED-HNN and ED-HNNII, we reduced the search space of the number of MLPs since took too much time get the results.
C.6 Environment for experiment
We used 48GB NVIDIA RTX A6000 GPU. We created a anaconda environment with python
3.7.16, pytorch 1.11.0 and pytorch geometric with version 2.0.4. Details can also be found at https://githute htttss///ggtubb..cmooooooooeeeee
34

---

Table 6: Dataset statistics of standard hypergraph benchmark dataset
Cora
Citeseer
Puhmed
Cora-CAA
DBL.CAC
NTU2012
ModelNerd0
20Newsgroups
# nodes
2708
3312
19717
2708
41302
2012
12311
16242
edge
1579
1079
7963
1072
22363
2012
12311
16242
# feature
1433
3703
500
1433
1425
100
100
100
# classes
7
3
7
67
40
avg. |o|
3,03
3,200
4.349
4.277
4,452
5
654.51
CE Homophily
0.897
0.893
0.952
0.803
0.869
0.753
0.853
0.461
Table Model performance on standard hypergraph benchmark datasets (Accuracy). The last row is the result with extreme hyperparameter search space that includes hyperparmeter searching for dropout and interpolation ratio 3 (introduced in Section 4.3). Top three models (excluding the last row) are colored by First, Second, Third. t the variant of the model using multihead attention. the variant of the model using Ligg defined in SHINE[48].
Cooa
DILP CA
NTU2012
MosalNeusO
HGNN
19.101 1O01
$7.356
11.43
33.333
1105
91410200
35 1152
20.00E 11411
81,246 20A3
HCHA
70.236 1.158
73,93 1.687
$7.250 = 0.511
83.911 10.368
91.358 0.377
68.270 1.304
94. 20.088
0.285
81.189 x 0..997
HNHH
76.765 1.500
72.524 +: 1.570
87.237 0.523
77,490
$6.927 00.340
58,859 01377
97.81
0.231 0.231
81,059 00.05
UNOCNNI
19. 895
1.505
75.514 2. 10T
19.124
= 11.77
K3.500 11
91.728 0:2225
59 09.255 2 21.88
97.24 07.243=1.335
81.007 + 0.4.53
AllDaepSeti
70.366
1.627
72.959 = 1.705
$9.418 0 0.366
$4.594200.77
91.594 2..308
58.847 2 0.995
07.532
0.185
81.721 0.665
AllSer Framfforee
71.14 141
1.994
58.067
64.766
01 1915911 61 10
59
11014
98.217
111
81.785
HypesCITT
4.128
41.751 1.814
48.191
73.560
1.529
90.292
83:35
92.455
0.387
HyperGAT
S8.183 2,070
1.874 42:246
48 48.350 0.426
73.752
1.508
90.394 0.366
45467 1.876
02:451
0.665
81.083 a 0.574
SHINE
57.755 4 3.195
11.413=0.000
48.576
0.455
75/057
1.912
90,759 0.2991
57.256 1.395
11395 93.903
81.001 0.03
SHINE!
56. 4 4.452
41.765
0.693
48.576 0145
75
1.500
90.AF7 00.129
67.157 -1 1.420
91.878 1.512
M1.239
+ 0.489
SHINE"
S6.1S 1.591
41.41%+
1.563
46.622
1200--117
74,6214
1.444
61.507 6 100
41 + 999
40 11.718
492 166
SHINE
S8.065 1.616
41.123 1.707
43.19
1.400
73:067
1.077
15.215:1767
70,835 23, 358
75.055 : 23.686
45.452 13,043
HSEN
76,632 1.509
71,824
1.779
$7.193 0.322
81.595
1.011
90.229 0.242
59.722 : 1.196
53.439 1.204
81.372 a 0.435
ED- ED HNNI
WLA35 1,670
78,09 =: 19M2
98.11 00410
K5.000
92:151
11:291
57.594 = 1111
97 999 1.19
81 008 + 0.055
78,951 1.445
72:524
=: 1.682
79 00.355-35 0.953
$3,693
0.539
91,702
0..325
56.223 : 0.958
95 749 0.335
80 10.150 2 7.3
Natara HNN (oMr))
86.709 1,635
73.285
E 1.742
37.136
= 0.450
$4,393
0.499
50,561
0..37
$9.300 1.017
98.558 = 1.295
81.734 2
Nomml-HNN (ours Caa)
1.520 90.729
73,581
58
88.081
00-581
91.057
0.1.7
1.565
98.564 01244
N0,0011
74105K
1,355
38.746 0311
85.381
10.77
91.30
10.19
30
0099
96 11221
D Standard Hypergraph Benchmark dataset
We performed experiments with standard hypergraph benchmark dataset to check whether Natural- HNN can be applied to the datasets that are not verified to have multiple factors behind group interactions. Considering how hyperedges were created for benehmark datasets, it is not likely that those datasets contain meaningful or task related interaction contexts. In co-citation and co-authorship networks, for example, hyperedges are created by simply connecting all documents cited by a paper or written by an author. Citations between a pair of papers might have context that is related to a reason for citation, however, is hard expect that a group of documents (papers) cited by a paper creates a special meaning or have a special context. Even if we assume that hyperedges in co-citation networks contain interaction context, is still not clear how these interaction contexts related to the labels of nodes. is also hard to expect interaction context in co-authorship networks for similar reason. Thus, the benchmark dataset experiment will verify whether Natural-HNN can be applied to the datasets where the existence of factors behind group interactions is not known.
For the node classification task with standard hypergraph benchmark datasets, randomly split the data into 50%/25%%/25% for vaainnngvaidattionnestt set. We measured average and standard deviation of the performances for 10 different data splits. The hyperparameter search space is provided in Appendix C.5.
D.1 Statistics : Standard Hypergraph Benchmark Dataset
Cocitaion networks and coauthor networks are adopted from [76]. The node features are bag-of- words representation of each documents. NTU2012 and ModelNet40 dataset is computer vision and graphics datasets where features are generated by applying GVCNN[19] and MVCNN[66]. Node feature of 20Newsgroups are generated by TF-IDF representations of news. The statisties of standard benchmark dataset is given in Table 6. Homophily ratio was calculated after converting hypergraph into a graph with clique expansion (CE) following the method described in the other work [70].
35

---

Table 8: Model performance standard hypergraph benchmark datasets (Accuracy) trained with only 5% of data
NTU201
20Nowogroopp
2..35
x
50115
HCHA
9.379
31.379
59,32
79
HNHN
1,000
5.206
2..009
58.753
0.382
Q.,00
UsiGCNT
68.212
1.205
0.822
60.25
96.584
0.334
79 79.061 0.50
4,012
O.647
71.319 2.964
59.892 4.833
0.534
AlS:tTraistrmer
2..35
1.720
82,942
+0,491
71.249 :
60.444 5.204
79.409 0.500
HSDN
5
57.812 1.308
64.345
4.025
87.656 0.233
17.016
0.179
79.406 0.599
ED HNN
66.435
69.809 2.559
90.039 0.342
57.954 6.477
96,698 0.336
78.3%6
Natural HENN
1.837
70.009 2.779
60.11
98.031 0.119
79.329
Natural HINN Ca.)
67.353 1,908
62,894 2.316
82,838 0,009
70.909
4 3,639
38.906 0.200
61.384 4,570
98.141
0.116
79.431
D.2 Node Classification on Benchmark Datasets
Table 7 summarizes the node classification performance in standard hypergraph benchmark datasets. We have the following observations: 1) Our model generally performs well various datasets by taking the first or second place in terms of accuracy. In the case of Citeseer and Cora-CA the performance of our model is comparable to the best performing model. The results indicate that our model can be applied to various circumstances, even when the context variety of hyperedges is not guaranteed. 2) Attention-basse models (i.c., A.ISerTransformer, SHINE, and HyperGAT) and disentangle- model (i.c., HSDN) generally perform similar to or worse than convolution-bas models (i.e., HGNN, HCHA, HNHN, UniGCNII) and AllDeepSets (which also does not have heads or factors) on Citeseer, Pubmed and DBLP-CA. Through the results, we can guess that those datasets do not contain various interaction contexts that is helpful for the model performance. This can also be a reason why our model does not perform well on those datasets as much as on other datasets.
We consider a model that achieves sufficiently good performance without relying excessively on hyperparameter tuning to be reliable. However, there has been an increasing number of papers, such as Sheaf Hypergraph Networks [18], PhenomNN [72], and ED-HNN [70], that report performance obtained through extreme level of hyperparameter tuning. Therefore, to enable a fair comparison with these works, we also included dropout and the interpolation ratio 3 (introduced in Section 4.3) in the hyperparameter tuning and conducted additional experiments. For both dropout and the interpolation ratio 3, we set the hyperparameter search space from 0.1 to 0.9 with an interval of 0.1. The results are reported in the last row of Table 7. Comparing the results of Natural-HNN with the official performance results of the papers mentioned earlier that rely on extreme hyperparameter tuning, we can see that NaNural-HNN outperforms them despite having a much simpler model architecture.
D.3 Training with only 5% of data
To check the generalization power of our model, we performed an experiment of training with only 5% of data. Following the split ratio of HGNN for Cora dataset, we trained with 5% of data, validated with 18.5% and tested with 37% of data. Table 8 shows the result. We have the following observations: 1) The performance of Natural-HNN tends to be similar or slightly better than convolution- This shows that Ratural-HNN has good generalization power that is comparable to convolution- based methods. 2) Our model performs better than recently introduced model, ED-HNN. Even ED-HNN has much larger hyperparameter search space, Natural-Hi HNNN performs better due to generalization power.
models.
36

---

E
Ablation studies and Hyperparameter sensitivity
E.1 Selecting Alternative Branch
In Section 4, we used the representation earned from 'Disentangle-free Branch' (K() when creating final hyperedge factor representations (a, The experiment results below shows the result when using the other branch, 'Aggregation-fine Branch' for creating final hyperedge factor representations (a (a/h%,) ). Table shows the result for standard hypergraph benchmark dataset and Table 10 shows the result for cancer subtype classification task.
Table 9: Comparison of our model (first two rows) with alternative model that uses the other type of hyperedge factor representation (last two rows)
Cura
Pulnnel
Can-CA
DBLI-CA
NTU20112
MackINes40
20Nevaggougg
Natunl HINN
80.709 L635
73.255
1.742
37.163 0.40
34 995 0.491
50.961
0.157
69,900
1.017
0.295
81.734 0.745
Namrra-NNN (+C.e)
1,570
73.51
+ 1.94
40466
85 + 0.85
91 011
$0.000
165
96. 96.164
0.254
K1.820 + 00AMS
NItraaTINNN mrarch)
WEARO
7X:31+ 1.575
3T. cass
U1137
R0321
-0347
31:7777410:00 11.75
Natrra-FNN (oterr hounch fay)
80.22 1.157
73,595 x 1,700 8KS21 0.434
85.01 0.563
91 070 0.178
90.060 0.205
98.577 0.222
81,837 0.554
As we can see in Table 9, there is no big difference in the performance between using 'Disentangle-first Branch' and 'Aggregation-firs Branch'.
Table 10: Comparison of our model (first row) with alternative model that uses the other type of
hyperedge factor representation (last row).
Method
BRCA
STAD
SARC
LGG
HNSC
CESC
Natural-HNN
0.804 2 0.036
0.659 2 0.049
0.745
2 0.045
0,707 20033
0.860 2 0.042
0.881 = 2.0042 0.875 0.051
Natural-HNe (other branch) 0.797 0.028
0.654
0.041
0.747
0.063
0.707
0.033
0.863 0.022
As we can see in Table 10, there is no big difference in the performance between using 'Disentangle- first Branch' and 'Aggregation-firs first Branch'.. The reason for this phenomenon is quite simple. We can consider the two cases: 1) when K,, and it are similar and 2) when they are largely different. 1) When It, and it, are similar, the result will not differ a lot between using It, or nt, as similar representations will be used. 2) When Ke, and it are largely different, the result will not be different a lot since relevance score of will be very small. In other words, of af kh. = of ( - ke,) will be very small for very small a,. This case means that the factor representation will not be reflected a lot during message passing since the representation is inconsistent (different result for two branches).
E.2 Natural-HNN without naturality constraint
We performed another ablation study to check whether naturality condition proposed in the paper important part that contributes to the model. We created an ablation model that do not satisfies naturality condition by not reflecting relevance score of during message passing. The results for standard hypergraph benchmark dataset is provided in Table 11. The results for the cancer subtype classification task are provided in Table 12.
Table 11: Model performance on standard hypergraph benchmark datasets (Accuracy). The ablation
model does not satisfy the naturality condition.
Mathod
Cama
Cheazrr
Puhmal
Con-CA
DRUPCCA
NTU22012
ModeINes40
Natural HISS (ours)
8000 = TA35
732855 : 1.742
87.15 0.350
0.101
50.6110017
89300 1331
98.358 0.295
BL.754 s 1.35
Nitara-Hhh fours & Eas)
80.739
= 1.570
73.551
: 1.664
R8.475
= 0446
85.81
2 2.588
91.022 0.179
90.060 = 1.565
98.584
0.254
81,827 a 0.633
Natrra--NN HNN (silatio))
20.200 1 1.53
75237
+ 1.248
821121
+ 0.100
86.874
0.424
90 896
+ 165
89 2012112
= 0.118
98 4
B1 A8S noms
Naturl HINN (ablation + (...)
11555
73.392
+ 1.832
88,448 = B.407
45.022
0.505
90.968 17668:000 0.169
89.679 = 1.129
98.177 0.216
81.783 a U1771
In Table 11, we can see that there a slight to moderate level of performance gap between Natural- HNN and its ablation model. It is not surprising result that there is not big difference between them since standard benchmark datasets do not seem to have informative interaction contexts related the task (Appendix D).
In Table 12, model. Since interaction context matters in cancer subtype classification task, naturality condition seems to boost the performance by capturing interaction context.
observe that there is a big difference between Natural-HNN and its ablation
37

---

Table 12: Model performance cancer subtype classification task (Macro F1). The ablation model does not satisfy the naturality condition.
Method
BRCA
STAD
SARC
LGG
HNSC:
CESC
Natural-HNN' (ours)
0.804 0.036
0.659 = 0.049
0.745 # 0.045
0,707 0.035
0.862 + 0.045
0.881 0.042
Natural-NNN' (ablation)
0.756 = 0.031
0.605 = 0.039
0.713 # 0.071
0.692 # 0.034
0.814 + 0.037
0.852 0.032
E.3
Hyperparameter Analysis
Since Natural-HNN does not have many hyperparameters, we analyzed how performance changes by the number of factors. Table 13 shows the result for the standard hypergraph benchmark dataset. Table 14 shows the result for cancer subtype classification task. Note that the tables below show the result of Natural-HNN without Cais-
Table 13: Performance of Natural-HNN with different number of factors. The best performances (reported in Table 7) are marked in red.
sumber of factors
Com
Citeseer
Puhmed
Cura CA
DBLP CA
NTU2012
ModelNest0
20Newwarougs
00.3847 1320
75.133=
84034 1:488
MI.351
0133
US 0.310
XI
1668
73.255
87,055
0.137
98.513
1.622
73.188
87.083
0450
0.491
90.939 = 0.151
89.821
1.070
98.559 0.295
80.591 1,673
73,237 1.783
83.136 0.450
0.385
90.955 0.131
80.900 1.017
98.513
81,660
We have interesting observations when we analyze the result in Table 7 with Table 13. 1) In Table we observe that Natural-HNN does not perform well on the Citeseer, Pubmed, and DBLP-CA datasets. Except for Pubmed, Table 13 shows that Natural-HNN used two or fewer factors on these datasets.
2) Natural-HNN demonstrated good performance on the remaining five datasets in Table 7. Except for the 20Newsgroups dataset, Natural-HNN used four or more factors to achieve its best performance, as shown in Table 13. These observations suggest that Natural-HNN generally performs well when capturing multiple factors. Furthermore, since the model did not benefit from using more than two factors on Citeseer and DBLP-CA, we suspect that these datasets lack diverse interaction contexts that would enhance performance. A similar trend is observed for other attention-based (AllS((Tranffommer) and disentangllement-bsseee (HSDN) models Table 7. Although these models are capable of capturing relational information, they showed poor performance performance-SSmetiinng sometimes even worse than some convolution- based models.
Table 14: Performance of Natural-HNN with different number of factors. The best performance (reported in Table 1) are marked in red.
number of factors BRCA
STAD
SARC
LGG
HNSC
CESC
0.789 0.036
0.630
0.729
0.055
0.695 0.030
0.853 0.047
0.869
0.048
0.787 0.038
0.642 0.043
0.745
0.045
0.707 # 0.035
0.858 0.031
0.867 0.043
0.804 # 0.036
0.049
0.725 # 0.048
0.689 + 0.047
0.858 0.036
0.881 = 0.042
0.785 + 0.027
0.637 + 0.032
0.729 a 0.058
0.691 + 0.044
0.860 0.042
0.878 + 0.034
We have similar observations when comparing the result Table and Table 1) For the SARC and LGG datasets in Table 14, Natural-HNN achieved its best performance when using two factors. 2) For the remaining datasets, Natural-HNN achieved its best performance with four or more factors. Except for CESC, these cases showed a meaningful increase in performance. Therefore, we can draw similar conclusion the one derived from the comparison of Table 7 and Table 13.
38

---

F
Additional Experiment Result
F1
Computational Complexity
Let d, be the input embedding dimension, d, be the output embedding dimension, K number of factors. N denotes number of nodes and M denotes number of hyperedges, E denotes the number of hhypeeeeee)))) pair (v, e) satisfying v e. We will assume that d, > d, d K, E M and E > N.
The computational complexity of one layer of Natural-HNN can be calculated by the following:
Aggregation-fing Branch (aggregation + MLP): O + (Md,d,) Disentangle-fiss Branch (MLP + aggregation): (Nd,d, + O(Ed))
Similarity (a) calculation : O(K( ) = 0 K
propagation back to nodes O(KE Ed,) O(E,,)
other calculations (concat, interpolation by 5) : O(Nd,) Thus, total computational complexity
becomes (((M + Nyd,d E(d; + d, + 1) + Nd, + 4. O((M + N)d,d, + E(d,+4)
For HGNN with dimension d; > d, > d, (d, denotes dimension of hyperedge embedding), com- putational complexity becomes (EE(d, + d.) (Md, + Nd,)dd).. The computational complexity of HGNN and Natural-HNN differs only by constant times. is not surprising since Natural-HNN is quite similar to HGNN but instead use two branches (only) during Vode-to-Hyperedge propagation and use factor similarity calculation. Thus, Natural-HNN is as scalable as HGNN.
F22 Scalability Analysis (training time)
We measured the time it takes for the model to train for 10 epochs. We averaged the values after measuring 5 times each. Also, we conducted the experiment in two settings: one with 2 heads and 16-dimensional vector as hidden representation and the other with 8 heads and 64-dimensional vector as hidden representation. Note that convolution-basee models, AllDeepSets and ED-HNN (II) use 1 head they do not have multi-head attention mechanism. The table 15 shows the result of our model's scalability. We have the following observations: 1) Our model is slower than convolution-basel models and HSDN. Since convolution-base models use strong inductive bias with simple computations, they are naturally scalable than our model. HSDN took less time since they only one message passing layer. 2) Our model is much faster than all attention- based models. Thus, we can conclude that our model scales well with hypergraph and parameter size next to the invoolutionbbsee models.
Table 15: Time took for training 10 epochs for BRCA. We tested with two cases by differing hidden dimension size and number of headst: multihead attention version
(dimension. head
(16,2)
(64, 8))
HGNN
8,492 2.00
HCHA
0.003
8.322 0.011
HNHN
0.005
4.362 + 0.005
UniGCNII
9.166 0.004
AllDeepSets
0.026
18.679 0.040
AllSeCTransformer
27.004 11.213
HyperGAT
0.024
24.579 + 0,047
SHINE
9,099 1.419
22,253 0.162
HSDN
2.944
10.130 0.006
ED-HNNN
11.937
22.738 0.026
ED-HNNII
21.621 0.029
36.418 + 0.026
Natural -INN (ours)
$.409
18.974
F33 Generalization power of Natural-HNN
To check the generalization power of our model, we experimented with different training set split ratio, while maintaining the validation and test set ratio to 25%. From 50% we gradually reduced training set proportion 10% shown in Figure 17. Figure 17(a) and (b) are the result of measuring performance with accuracy or Macro-F1 scores and (c) and (d) are the result of measuring relative degradation of performance dataset experiment, which is measured with Macro-Fl score, the relative degradation of performance is caculated by (Fso FF)/Fso 100% where F, denotes the Macro-Fl score when trained with x x%. The same applics to Cora-CA. which is measured with accuracy. Figure 17 (a) and (c) are the result in Cora-CA dataset, which is standard hypergraph benchmark, (b) and (d) are the result
the performance when trained with 50% For example, for BRCA
39

---

19.06
(a) Accuracy,
(b) F1,
LUCON
(c) Relative performance degradation, Cora-CA
(d) performance degradation,
Figure 17: The performance of models when reducing training set proportion. First row shows Macro Fl score and the second row shows relative performance degradation compared to the performance when using 50% dataset as training set. -NNN (ours, colored in blue) maintains best Macro Fl score and small relative performance degradation on both Cora-Cl CA and BRCA dataset.
for BRCA dataset, which is dataset used for cancer subtype classification task. The left figure in each Figure 17 (a,b,c,d) is the result of comparing ours (blue) and convolution of deepset based models. These baselines cannot perform context-dependent message passing. The right figure in each Figure 17 (a.b..,d) is the result of comparing ours (bluc) and other baselines that have potential for context-dependent message passing
We have the following observations 1) The degradation of performance for Natural-HNN was smaller when compared with most of the baselines in both Cora-CA and BRCA. Specifically, we can see that Natural-HH
has comparable result with convoluation based models in left figures of Figure 17 (c) and (d). Considering that convolutions based models have strong generalization performance due their strong inductive bias, can say that our model has good generalization power comparable to convolution based models. When compared with other baselinese in Figure observe that Natural- -NN had very small degradation in performance. In 17 (b) and (d), HNN had nearly the smallest degradation when compared with models that other words, Natural-HNe have more expressive power than convolution based methods. We consider model had good generalization among baselines with more expressive powers. Specifically, Figure 17 (d), Natural-HNN showed outstanding result in cancer dataset which has various context of interactions. This might be due to the fact that the inductive bias (context interaction) that Natural-HNe used matched the actual data characteristics.
2) Natural-HNN had the best Ma score for all different training ratio. Our model always had the best performance compared to convolution or deepset based models in left figures Figure 17 (a) and (b). Specifically, we can that Natural-HNN had outstanding performance in BRCA cancer dataset in the left figure of Figure 17 (b). Thus, we can conclude that Natural-HNN is more expressive compared to convolution based models. Also, when inductive bias (interaction context) matches the data characteristics (BRCA), Natural-Hue the result, we could verify that Natural- -NNN can utilize context information to get good performance. When compared with other baselines. in the right figures of Figure 17 (a) and (b), can that our model could achieve better, or at least comparable performance when compared with baselines. We conclude that our model has expressive power comparable other attention (including Set Transformer) or equivariance based models. Again, we can observe that Natural- -NNN achieved outstanding performance in BRCA dataset by capturing context types. Considering that Natural HNN had good generalization and expressivity, we argue that our model made a proper trade-off between expressive power and generalization.
provides outstanding performances. From
40

---

F4
Captured Context in CESC
(9 With pathways selected by Natural-HiNN
(a) AuT
(b) Natural-HMN
[c]
(i) With pathways by
(d) Ground-Truth
(c) Natural-HNN
Figure 18: Captured interaction context. Pathways are selected by SHAP value. Captured patterns are shown red box and not captured patterns are shown with orange box. Weakly captured case is marked as dotted red block.
Figure 18 shows the captured context result in CESC. The evaluation and interpretation method is identical that of Section 5.3. As can see the figure, for pathways selected by Natural-HNN, Hotural-HNN correctly captures context similarities between clusters (red box) while HSDN does not (orange box). For the pathways selected by HSDN, Natural-HNN and HSDN partially captures cluster similarity. However, when comparing orange box in (d) and (f), we can observe that Natural-HNN captures interaction context slightly better than HSDN even with the pathways selected by HSDN.
F.5 Cancer Subtype Classification (Micro F1)
We briefly provide Micro Fl scores each model in cancer subtype classification task. The Table 16 also shows that our model generally performs well on most of cancer datasets.
Table 16: Micro Fl score of each model with parameter and hyperparameter of the best Macro FI score. Top two models are colored by First, Second. f: the variant of the model using multihead attention. we did not use Lais-
Method
BRCA
STAD
SARC
LGG
HNSC
CESC
HGNN
0.877 0.027
0.727 + 0.026
0.739
0.057
0.696 0.034
0.888 0.031
0.903 0.0.34
HCHA
0.808
0.725 0.036
0.731 0.058
0.685
0.039
0.876
HNHN
0.806 0.027
0.729 0.067
0.733 2 0.046
0.676 0.037
0.884 a: 0.018
1.910
UniGCNII
0.791 0.027
0.797 + 0.038
0.761 0.046
0.665 0.038
0.910
0.911 + 0.018
AllDeepSets
0.823 0.025
0.748 0.039
0.657 0.035
0.669 0.045
0.895 1 0.025
0.927 0.024
Transforeeer
0.827
0.710
0.656
0,908 0.025
HyperGAT
0.
0.725 0.050
0.645 = 106
0.669 0.051
0.889 0.030
0.900
HyperGAT!
0.002
0.676 001108
0.098
0.665 0.042
0
0.022
SHINE
0.659 0.090
0.590 0.127
0.618 106
0.649 0.058
0.846 1.46000.000 0.032
SHINE"
0.027
0.045
0.654
0.873 0.027
0.907 0.031
HSDN
0.838
0.758 + 0.047
0.694 0.036
0.892 0.025
0.925 + 0.024
ED-HNN
0.826 1 0.024
0.793 1 0.047
0.761 0.039
0.703 + 0.028
0.913 2 0.021
0.925 = 0.035
ED-HNNII
0.815 0.027
0.0024
0.694
0.696 0.038
0.916 0.013
0.942 0.024
Natural HNN"" (ours)
0.024
0.027
0.932 0.044
41

---

Table 17: Hyperedge classification result (accuracy). Top two models colored by First, Second.
Ditaset
HONN
HCHA
HNHN
UnIGCNU
AlDesppess
AlSeeThansfomes
HSON
Naural-HNN (ours)
Chemical Rection
0.499 OOBS
04KT 0.010
0,27
0672 0704
0.93 F0022
(.727 0002
0APT #002
0,773 0.08
F.6
Chemical Reaction Classification (Hyperedge Classification)
To validate whether Natural-HNN performs well not only on cancer subtype classification but also on other hypergraph datasets that contains meaningful hyperedge semantics, we performed hyperedge classification task on a chemical reaction dataset [21]. Among the three datasets proposed in that paper, we used the first dataset for validation, the other two datasets have relatively small number of samples and the prediction tasks are too easy to serve as a meaningful evaluation of the model. The hyperparameter search space was kept the same that used for the standard hypergraph benchmark datasets, and Natural-HNN was evaluated without Lav. As shown in Table 17, Natural-HNY demonstrates overwhelmingly superior performance compared to other models, including HSDN. Therefore, Natural- proves to be highly effective not only for cancer subtype classification but also for datasets in which hyperedges contain hidden semantics related to labels.
E.7 Reliability of Natural-HNN in Biology
In order for model be reliable, the model should provide consistent output regardless of the choice of hyperparameters. So we conducted an experiment to check whether models consistently rely on the same pathways. If model consistently rely on the same pathways for prediction regardless of the hyperparameter, biologists might consider the model to be reliable since it potentially captured and used what can be explained with biological domain knowledge. On the other hand, if the model relies on different pathways for different hyperparameters, biologists might not trust the model.
To check whether model relies on the same pathways, we ranked the pathways with SHAP value and selected top-k pathways. These pathways the ones that models relied most for their prediction. Then, we calculated Jaccard similarity of top-k pathways for different hyperparameters. pathways earned from each hyperparameter combination is similar, then we can conclude that model always rely on the same pathways regardless of the hyperparameters.
top-k
Figure 19 and Figure 20 are the result of calculating Jaccard similarity between different hyperparam- eter combinations BRCA dataset. The hyperparameters we changed was the hidden dimension size and the number of factors. Values each tick of row and column is the pair of the two hyper- parameters (i.e., the value in the ticks represent (hidden dimesion, number of factors) pair). Each heatmap shows Jaceard similarity when selecting top 10, 15, 20, 50, 100 and 500 pathways. Figure 19 is the results for Natural- and Figure 20 is the result for HSDN. We also calculated average Jaceard similarity for each heatmap.
The ideal result would show dark blue colors (high similarity) to all cells in the heatmap. It means that top-k pathways that a model relied on always the same regardless of the hyperparameter. When comparing Figures 19 and 20, we can see that Natural-HNN tends to rely on the same pathway regardless of the hyperparameter while HSDN does not. When comparing average Jaccard similarity scores, we can quantitatively observe that Natural-HNN has better consistency when compared to HSDN. For example. Jaccard similarity with top 15 pathways of Natural -NNN (19 (b)) has average similarity of 0.759 while that of HSDN (20 (b)) has average similarity of 0.555.
From this experiment, we can conclude that Natural-HNN reliable since it consistently focuses on the same pathways regardless of the choice of hyperparameters. Also, we could again verify that our model captures the functionality of pathways (interaction context of hyperedge) and expect that our model will work reliably in different dataset or different biological applications. Note that similar analysis for Figure 21 and Figure 22 provides similar conclusion.
42

---

brow 30, axg
broe, lop 15, sog
bii 8.187
382
$63
18.
18.
15.8
16.5
32
32.2
322
12A
$2.
$2.b
32.8
52.
$4.2
642
64.
18.
64.
623
15.4
(a) Top 0.787
(b) Top 0.759
(c) Top 0.787
braa Iop 50, axg
brra, top 180, suy
brra Inp 500, wvg
182
162
18.
Mt
15.8
383
153
10.2
312
5L..
$2.4
32.8
52..
5t3.
$4.2
64.
642.
64.
65.
$3.4
44.
68.5
64,3
164
22.2
15.4
16.2
1666
(d) Top avg 0.758
(e) Top 100, avg 0.717
(f) Top 500,
0.755
Figure 19: Jaccard similarity calculation result for Natural-HNN on BRCA. We can observe that Houral-HNNN generally relies on similar pathways regardless of hyperparameters by showing high Jaceard similarity value.
0522
brca, avg: 0.535
trca, top, 20, avg 645
16
16.
1...
16.
322
SLA
32A
SLA
32.
72.
65.3
6A.5
88.3
65.
16.
362
16.
160
(a) Top avg: 0.562
(b) Top avg
(c)
20, avg
brea, top 30, avg 055
brca, top 100, avg 0,600
trra bop 506, avg 0.638
16.
18.
18.2
38
38.
16.8
16.5
22A
SAA
36
22.5
242
643
55.5
16.2
182
16.4
(d) Top 50,
631
(0) Top 100,
0.600
(0) Top 500,
0.638
Figure 20: Jaccard similarity calculation result for HSDN on BRCA. We can observe that HSDN relies on different pathways for different hyperparameters by showing strong diagonal pattern. This inconsistency makes HSDN an unreliable model for biology.
43

---

Insc top 10. oug 6.716
Inse, top auy
hroc, too 20. avg 3.684
1833
18.4
15.2
183
183
32.
SL3
33.3
52.5
663
s4.4
ck
s8.
543
(45
16.2
16.0
(a) Top 10, avg : 0.716
(b) Top 15, avg 0.761
(c) Top 20, avg 0.684
Insc:
Insc.
708
1622
19.2
164
9
16.5
16
$2.2
53.2
122
$1.4
52.5
$1.3
32.5
6A.2
643
64.
64.
643
66.
162
184
18x
184
322
162
18.4
36.
(d) Top 50, avg
(e) Top 100, avg
(1) Top 500, avg
Figure 21: Jaccard similarity calculation result for Natural-HNN on HNSC. We can observe that Natural-HNN generally relies similar pathways regardless of hyperparameters by showing high Jaccard similarity value.
hndc too, 10, avg 0.357
hrisc, 15p,15, 20g 00442
bnoc,
182
18.3
18.
18-4
183
19
16.9
$22
9.2
et $2.4
M3
32.8.
862
ch
66.
66.
64.
643
645
645.
162
1844
18.8
32.
18.5
18
32.5
182
16.4
16.
(a) Top 10, avg 0.357
(b) Top 15, avg
(c) Top 20, avg 0.488
besc, top, 50, Nu 582
Inst, tapp1oppp...... 160, avg 619
besc, top, 500, aup D.
162
164
184
15.
523
52.2
32A
$2.
32.
5L.
51..
6A2
61.
61.2
5
6
343
18.
165
162
1844
10.6
162
184
10.
(d) Top 50, avg 0.582
(e) Top 100, avg 0.619
Top 500, avg 0.685
Figure 22: Jaccard similarity calculation result for HSDN on HNSC. We observe that HSDN
relies on different pathways for different hyperparameters by showing strong diagonal pattern. This inconsistency makes HSDN unreliable model for biology.
44

---

G Limitations, impacts and Future Work
G.1 Broader Impacts
Potential Positive Societal Impacts. As demonstrated through various experiments, Natural-HNN has the potential to capture the inherent heterogeneity of interactions and diverse interaction contexts. In complex systems such as biological organisms, many interactions have unknown functionalitics. Natural-HNN's ability to capture these latent interaction contexts can contribute to the development of more reliable models for wide range of real-world problems.
Potential Negative Societal Impacts. Our proposed method is designed to automatically identify and incorporate the factors underlying interactions. The relevance scores indicate which factors most relevant to each interaction. However, this method is applied to data where privacy is critical, it could potentially lead to indirect leakage of sensitive information through those relevance scores.
G.2 Limitation
Natural-HNN, uses hyperparameter K decide number of factors instead of automatically discovering the number of factors within data. In real world problems, it might require lot time to get optimal number of factors. This is kind of problem that all disentangle-base methods need to solve in the future.
G.3 Future Work Model for Graph Neural Network
Since Natural-HNN is designed for hypergraph neural network, we can apply our model to graphs. However, computationally inefficient since Natural-HNN performs two step message passing (node-to-hyperedge byperedge-to-node) while most of the gnns perform one step message pass- ing. Thus, need to devise novel criterion for disentangling edge types in graphs without using edge representations. Since there are many interaction types in graphs, developing reliable edge disentangling model in the perspective of category theory will be useful for many real world applications.
G.4 Future Work 2 Hyperedge-Node co-disentanglemmnt
Our goal was to disentangle the factors behind group interactions, and thus we assumed that the nodes participating in interaction share the same context (factor). However, is also possible that individual nodes have their own distinct contexts (factors) when participating in an interaction. For example, consider group discussion involving multiple individuals. In Natural-HNN, the disentanglement focused on hyperedge-level factors, such as the discussion topic. However, node- level disentanglement could also be applied in this scenario. Each participant might have specific role in the discussion. Separately from the discussion topic, factors such as the context or role of each participant in the discussion could also be disentangled. Performing hyperedge-node co-disentanglemmntt which is disentangling both hyperedge-leve and node-level factors, would allow for more nuanced approximation of diverse underlying mechanisms.
45


Reviews:
[{"review_id": "AgvZtttRVc", "reviewer": "Reviewer_8x4v", "summary": "The authors propose a novel method for hyperedge disentanglement through the lens of category theory, where a criterion called factor representation consistency derived from the naturality condition. Hyperedge disentanglement assumes that group interactions have mechanisms and semantics. So this consistency criterion and the leraned representations should remain consistent no matter when the message passing through the direct message passing or disentanglement. To validate the theory and assumption, they create a proof-of-concept model, Natural-HNN, and test on a cancer subtype classification task using genetic pathway data, where genes are nodes and pathways are hyperedges.", "questions": "Q1. The current evaluation focuses on hypergraph classification within the bioinformatics domain. Could the authors comment on the model's performance and applicability in different domains? While Appendix D provides results on other datasets, a discussion on why the performance gains are most pronounced in the bioinformatics setting would strengthen the paper's claims about where this criterion is most effective.\n\nQ2. Figure 4 illustrates how the model generates two pathway representations, i.e., $h_{v_i}$ and $y_{v_i}$. Could the difference between these representations like $h_{v_i}-y_{v_i}$ (or difference before the concatenation) be leveraged to directly estimate the disentanglement or consistency of a hyperedge? Such a metric might offer a more straightforward validation of the core mechanism than the more complex analysis presented in the Appendix B.5.\n\nQ3. Can we estiamte the efficiency of learning by providing different numbers of training graphs or training graph sizes? Since graph neural networks can be prone to overfitting, demonstrating strong performance with limited data (e.g., by varying the number or size of training graphs) would further highlight the effectiveness and robustness of the proposed criterion.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n\nS1. **Well-structured**: The paper is well-written and logically structued. Figure 1 provides an intuitive explanation of the core factor representation consistency criterion, but there is no limitation but ask the neural networks to learn the structural semantics automatically. Many hyper-graph based method is struggle to determine the hyper-parameters to enforce the topology captured, but the proposed method believe that the neural network can learn it automatically.\n\nS2. **Novel Contribution**: The most contribution is to prosose the criterion specifically designed for hyperedge disentanglement. This is a significant step forward for the field of hypergraph representation learning in neural networks. The core idea is that if a hyperedge can be disentangled, then the representations learned through two different pathways (one entangled, one disentangled) should be similar. This concept is simple, straightforward, and allows for learning to be easily integrated from these two channels.\n\nS3. **Strong Empirical Results**: Given the simplicity of the learning process, the detailed analysis provided is essential for supporting the underlying idea. The authors conduct thorough qualitative analyses, using visualizations to verify that their model, Natural-HNN, can indeed capture meaningful biological context and patterns from the data.\n\n\n**Weaknesses**:\n\nW1. **Missing Analysis on Graph Structures**: The datasets used exhibit considerable variance in node degrees and hyperedge degrees (as shown in Table 3 in the Appendix). However, the paper lacks a discussion on how graph density might correlate with the model's performance improvement across the six different datasets. Such an analysis would provide deeper insight into the model's behavior under varying topological conditions.\n\nW2. **Limited Data Scope**: The evaluation is confined to a single data domain (bioinformatics) and task (hypergraph classification). There is only one domain of data for evaluation. Many general graph representation learning methods can outperform hypergraph-specific ones on standard node classification benchmarks (as data used in Appendix D), which may not be the ideal task for validating the unique advantages of hypergraph learning. The inclusion of synthetic datasets, where the \"factor representation consistency\" can be explicitly controlled, would also be beneficial for a more rigorous validation of the core criterion.\n\nW3. **Gap Between Theory and Representation Analysis**: The paper provides a theoretical analysis to justify the consistency criterion. However, the evaluation does not fully bridge the gap between this theory and the learned distributed representations. The assessment relies on overall performance and visual patterns, but it is difficult to directly verify the theoretical claims from the representations themselves. An evaluation focused on link-level or hyperedge-level prediction tasks could offer more direct and convincing evidence than the current node-based and graph-based evaluations.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive review.\n# Q1 & W3 & W2\nWhile our current evaluation focuses on bioinformatics, our model could be broadly applicable to other domains. We acknowledge the reviewer’s concern that the performance improvements on benchmark datasets in Appendix D are relatively modest compared to those observed in bioinformatics tasks. However, we believe this discrepancy primarily stems from differences in the informativeness of the factors (contexts) encoded in hyperedges across datasets. The more strongly a factor is related to the label, the more critical it becomes to capture that factor, which in turn can have a significant impact on performance. While the degree of informativeness is difficult to quantify, it can be reasonably inferred.\n\nSpecifically, in co-citation and co-authorship networks used in Appendix D, hyperedges are formed by simply grouping all documents cited by a single paper or all papers authored by a single researcher. While citations between a pair of papers may carry meaningful context (e.g., a specific reason for citation), it is hard to argue that a group of cited documents shares a well-defined or coherent context. Even if we assume some form of interaction context exists in such hyperedges, the connection between this context and the node labels is often unclear. In such cases, even if the model successfully captures contextual information, it may not significantly contribute to improving label prediction performance.\n\nIn contrast, in the cancer subtype classification task, pathways are known to carry functional context [2,3] and are closely associated with the labels (i.e., cancer types) [2,4]. To summarize briefly, pathways are associated with specific biological functions (i.e., factors), and whether these functions operate normally—potentially reflected in the factor representations—can determine the type of disease. This indicates that, in this case, the factors are highly relevant to the labels.\n\nTo support our claim that the model is much effective in domains where hyperedges contain rich, label-relevant contextual factors, and to demonstrate its broader applicability, we conducted an additional experiment in the chemical domain—another setting where hyperedges encode meaningful interactions. Specifically, we evaluated our model on the Molecule Reaction Dataset-1 [1], where nodes represent molecules and hyperedges represent chemical reactions annotated with reaction types. The task is to perform hyperedge classification by identifying the reaction type corresponding to each set of participating molecules (i.e., each hyperedge). \n\n|                                        | HGNN        | HNHN        | AllDeepSets   | AllSetTransformer | HSDN        | **Natural-HNN (ours)** |\n|----------------------------------------|-------------|-------------|---------------|-------------------|-------------|--------------------|\n| Molecule Reaction Dataset 1 (Macro-F1) | 0.449±0.005 | 0.257±0.008 |  0.493±0.023 |  0.727±0.026     | 0.491±0.023 | **0.773±0.008**        |\n\n\nWhile time constraints prevented us from evaluating all baselines during the rebuttal period, we highlight that Natural-HNN substantially outperforms HSDN (a model that also performs structure-level disentanglement) and achieves a 5-point improvement over AllSetTransformer. These results reinforce our claim that our method is particularly effective when hyperedges encode rich, informative contexts, even outside the bioinformatics domain.\n\nWe also believe that this response directly addresses the reviewer’s concern regarding the limited data scope (W2) —specifically, the critique that the evaluation was confined to a single domain (bioinformatics) and a single task (hypergraph classification). By presenting results on a new domain (chemistry) and a different task (hyperedge classification), we provide concrete evidence of our model’s broader applicability and generalizability.\n\n# Q2\nTwo pathway representations are $h_{e_{i}}^k$ and $\\tilde{h}_{e_{i}}^k$. The difference between the two representations can be used to assess the degree of consistency. However, this approach is not applicable to the cancer subtype classification task, as there are no explicit ground-truth labels for the functional roles of individual pathways.\n\n\n# Q3\nRegarding the cancer subtype classification task, we evaluated the performance of our model and the baselines while gradually reducing the number of hypergraphs used during training. The results are presented in Figure 7 of Section 5.4. As shown in the figure, our model maintains strong performance even when the training ratio is reduced.\n\nAdditionally, in Figure 8(a), we examined whether our model continues to capture the functional context effectively when the number of training hypergraphs is decreased. The figure shows that our model still captures the functional context well, despite having fewer hypergraphs during training.\n\nFor the standard benchmark datasets, please refer to Figure 17 in the Appendix F.3. This figure demonstrates that our model performs well even with a small portion of the training set. Furthermore, in Appendix D.3 (Table 8), we evaluated performance with the training set ratio reduced to as low as 5%, and as shown, our model achieved competitive or superior performance under these conditions.\n\nTherefore, these results demonstrate that the proposed naturality-based criterion enhances the model’s robustness by effectively capturing the functional context or underlying factors, even under limited data conditions.\n\n# W1\nIn the cancer subtype dataset, the goal is to predict each patient’s cancer subtype. All patients share the same set of genes and pathways, but their gene features differ. As a result, although this is a hypergraph classification task, it is not affected by topological differences in the hypergraph.\n\n\n[1] DLGNet: Hyperedge Classification through Directed Line Graphs for Chemical Reactions (OpenReview)\n\n[2] Mapping biological process relationships and disease perturbations within a pathway network.\n\n[3] Disentangling the multigenic and pleiotropic nature of molecular function\n\n[4] Identifying cellular cancer mechanisms through pathway-driven data integration"}, {"review_id": "VlrSIEX0F4", "reviewer": "Reviewer_gAsC", "summary": "This paper addresses hyperedge disentanglement problem in hypergraph neural networks. While previous work has focused on disentangling node or edge-level representations, the authors argue that group-level interactions (hyperedges) encode rich semantic factors,especially in domains like biology, adn drug discovery, where hyperedges represent pathways or functional modules.\nThe authors propose a novel disentanglement criterion based on category theory, which ensures that the representation of a factor is consistent regardless of whether disentanglement is applied before or after message passing. They introduce Natural-HNN, which computes two versions of hyperedge factor representations and measures their agreement to assess factor relevance.", "questions": "- Can you clarify why category theory is necessary for defining the disentanglement criterion, and why alternative heuristic or empirical approaches would fall short?\n\n- Have you considered evaluating disentanglement using quantitative metrics such as mutual information, total correlation, or latent factor separation?\n\n- Could the naturality-based criterion be integrated into simpler or more conventional GNN frameworks, such as bipartite models?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strenghts\n\n- Hyperedge disentanglement is an unexplored and novel problem. \n- Natural-HNN consistently outperforms other baselines across multiple datasets.\n- The model’s disentangled factors offer interpretable insights into biological pathway functions\n\n## Weaknesses\n- It is not clear why this problem must be addressed using category theory; the motivation for this choice needs to be made stronger in the Introduction. As it stands, it is unclear whether simpler methods could achieve similar outcomes without the added abstraction.\n\n- All experiments focus on biomedical hypergraphs. The common hypergraph datasets used in literature has not been explored.\n- The evaluation of disentanglement relies on SHAP and functional similarity metrics rather than direct ground truth or quantitative disentanglement measures. Given that disentanglement is the paper’s core contribution, a more rigorous and targeted evaluation is necessary.", "was_revised": false, "final_justification": "", "rebuttal": "# Q1 & W1\nHeuristic-based methods typically rely on assumptions about the data. While such methods may perform well when those assumptions hold, they often struggle when the assumptions are violated. A representative example is the similarity-based criterion, which assumes that instances belonging to the same factor will have similar factor representations. However, this assumption does not always hold in practice. For instance, in genetic pathways, genes within the same pathway or context do not necessarily share similar characteristics or gene expression values.\n\nOne of the baseline models, HSDN, adopts a similarity-based criterion. As shown in Figure 5, it failed to disentangle hyperedges effectively, likely due to the limitations of its heuristic assumptions.\n\nIn contrast, our goal was to develop a criterion for hyperedge disentanglement that does not rely on heuristics. To achieve this, we sought to identify properties that can be directly derived from the definition of hyperedge-level disentanglement itself. As an analytical tool, we employed category theory, as it provides a mathematically rigorous framework for representing abstract concepts and complex systems.\n\nBy formulating hyperedge disentanglement within the language of category theory, we were able to formally express its underlying structure and derive a principled criterion that facilitates effective disentanglement.\nWe will incorporate this motivation into the final version of the paper.\n\n\n# Q2 & W3\nSince there is no ground truth for pathway functions, it is not possible to directly compare the learned factors with ground-truth labels. Even if such labels existed, meaningful evaluation would still be difficult without a guarantee that the pathway functions are independent—because disentanglement fundamentally aims to uncover independent factors.\n\nHowever, evaluating hyperedge-level disentanglement is feasible. Although explicit function labels are unavailable, functional similarity between pathways can be calculated using widely adopted methods in computational biology. By comparing functional similarity from computational biology method and that of ours, we were able to verify that our model captures hyperedge-level factors, rather than hypergraph-structure-level or other types of disentanglement.\n\nFurthermore, as shown in Figure 6, we analyzed the correlation between hyperedge-level factors. This analysis provides supporting evidence that each factor captures distinct information or mechanisms.\n\n\n# Q3\nWe believe that our approach can also be applied to bipartite GNNs. Our model essentially consists of two phases: node-to-hyperedge message passing and hyperedge-to-node message passing. If nodes and hyperedges are regarded as two distinct node types, the structure can be interpreted as a bipartite graph. Therefore, the proposed naturality-based criterion is also applicable in the context of bipartite GNNs.\n\n# W2\nThe results on the standard hypergraph benchmark datasets (including citation networks) are provided in Appendix D. Although we report performance on the benchmark datasets, it is difficult to conduct in-depth analysis comparable to that of the cancer subtype classification task, as the underlying factors present in these benchmark datasets are not clearly defined or known.\n\nAdditionally, during the rebuttal period, we conducted a hyperedge classification experiment on the Chemical Reaction Dataset 1 [1]. In this task, the goal is to classify the reaction type (i.e., the hyperedge label) given a set of molecules (nodes) participating in the reaction.\n\nDue to time constraints, we selected a subset of representative baseline models to evaluate their performance in terms of Macro-F1 score. The results are summarized in the table below. We will evaluate the remaining baseline models and include their performance results in the final version of the paper.\n|                                        | HGNN        | HNHN        | AllDeepSets   | AllSetTransformer | HSDN        | **Natural-HNN (ours)** |\n|----------------------------------------|-------------|-------------|---------------|-------------------|-------------|--------------------|\n| Molecule Reaction Dataset 1 (Macro-F1) | 0.449±0.005 | 0.257±0.008 |  0.493±0.023 |  0.727±0.026     | 0.491±0.023 | **0.773±0.008**        |\n\nIn the chemical reaction dataset, Natural-HNN shows a large performance gap compared to HSDN, which performs hypergraph-structure level disentanglement, and also outperforms AllSetTransformer by approximately 5 percentage points. These results suggest that naturality-based disentanglement can be effectively applied not only in biomedical domain but also in other domains.\n\n\n\n\n[1] DLGNet: Hyperedge Classification through Directed Line Graphs for Chemical Reactions (OpenReview)"}, {"review_id": "SYjFyalS3y", "reviewer": "Reviewer_28h3", "summary": "This paper addresses the issue of hyperedge disentanglement in hypergraph neural networks (HNNs). The authors propose a new criterion for hyperedge disentanglement based on category theory, with a particular focus on the naturality condition. Their approach dissects hyperedges within hypergraphs, paying special attention to meaningful factors such as functional context. To validate the proposed criterion, the authors implement a proof-of-concept model—Natural-HNN—and apply it to a cancer subtype classification task using genetic pathway data. The experimental results demonstrate that the model outperforms traditional baseline models, thereby validating the effectiveness of the proposed disentanglement criterion.", "questions": "1. Could the authors provide further clarification on the premise mentioned in Weakness 1?\n\n2. Could the authors provide a more detailed explanation of how the method performs hyperedge disentanglement?\n\n3. The improvements in the method do not seem very significant, but the visualization experiments appear to be very effective. Could the authors explain why this is the case?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. This work introduces a new category-theoretical perspective on hyperedge disentanglement, which is rather novel in the context of hypergraph learning.\n2. The paper clearly identifies its contributions, including the novel criterion for disentangling hyperedges, the Natural-HNN model, and experimental validation on cancer subtype classification.\n\nweakness\n1. The proposed method in the paper is based on the premise: \"The factor representation must be consistent regardless of the sequence of operations if that factor is relevant to the interaction context of a hyperedge.\" However, this premise has not been convincingly proven theoretically, and the explanation provided is not sufficiently detailed. Is it possible that the representations could differ? Personally, I believe this is indeed a possibility. The representation of the same target does not necessarily have to be fixed; perhaps new information emerges after hyperedge disentanglement, or the neural network has modeled different relationships. This phenomenon is quite likely to occur in connectionist models like hypergraph neural networks. I think this point should at least be discussed in more detail, and more space should be devoted to this argument.\n2. The specific process of hyperedge disentanglement is not clearly elaborated, which should be one of the key focuses of the paper.", "was_revised": true, "final_justification": "The authors' rebuttal has addressed my concerns. I believe this paper is worthy of acceptance.", "rebuttal": "# Q1 & W1\nDisentanglement is a methodology based on the assumption that data can be decomposed into independent factors, and it aims to identify and isolate these underlying factors. Importantly, the disentanglement process does not introduce new information; rather, it extracts specific factor-related information from the existing data.\n\nConsider the example of people (nodes) participating in group discussions (hyperedges). A node representation corresponds to an individual's opinion. Suppose each factor represents an independent discussion topic (e.g., Factor 1: \"Is Hawaiian pizza tasty?\" Factor 2: \"Do you support a particular political party?\"). Then, the factor representation of a node would reflect that individual’s stance on the corresponding topic. In this context, disentanglement refers to the process of extracting the relevant opinions from each person’s overall set of beliefs, focusing only on a specific topic. This extraction does not create new meanings but selectively reveals components of existing information.\n\nNow, consider that a group discussion eventually leads to a conclusion—this can be seen as the hyperedge representation. An entangled hyperedge representation may be thought of as a comprehensive outcome resulting from a discussion that simultaneously covers all possible topics. However, if the topics (factors) are independent of one another, extracting the conclusion related to a specific topic from an entangled hyperedge representation (i.e., message passing and then disentangle) yields the same result as conducting a discussion focused solely on that topic (i.e., disentangle and then message passing).\n\nTo make this more concrete, suppose a group discusses two independent topics: Factor 1, \"Is Hawaiian pizza tasty?\" and Factor 2, \"Do you support a particular political party?\" These topics are assumed to be independent—opinions on one do not influence opinions on the other. Therefore, the conclusion reached about political party preference after a discussion covering both topics (message passing followed by disentanglement) will be equivalent to the conclusion reached after a discussion focused solely on political preference (disentanglement followed by message passing).\n\nIn fact, this is not merely a premise but a fundamental property that must always hold.\nLet us begin the proof by analyzing the definition of hyperedge disentanglement in category theoretical viewpoint. Let $V$ be a set of nodes in a hyperedge and let $E$ be the result of group interaction (intuitively, conclusion of group discussion). The transition from $V$ to $E$ is the group interaction mechanism. In category theory, we can represent this as morphism $f$ : $V \\rightarrow E$.\n\nBy the definition of hyperedge disentanglement, mechanism can be decomposed into several independent mechanisms. (i.e., morphism $f$ can be decomposed as the product of independent morphisms $f_{c}$ and $f_{d}$. $f = f_{c} \\times f_{d}$) \nAs can be seen in Figure 13 at the Appendix A.5, objects (nodes $V$ and conclusion $E$) can be decomposed correspondingly : $V = V_{c} \\times V_{d}$ and $E = E_{c} \\times E_{d}$ such that $f_{c} : V_{c} \\rightarrow E_{c}$ and $f_{d} : V_{d} \\rightarrow E_{d}$.\n\nNow, let’s examine this from the perspective of factor $c$. We have the commutative diagram for factor $c$. Disentangle first ($V \\rightarrow V_{c}$) and then performing interaction $ f_{c} : V_{c} \\rightarrow E_{c} $ results $E_{c}$.\nPerforming interaction $f : V \\rightarrow E$ first, and then disentangling $E \\rightarrow E_{c}$ results $E_{c}$.\nThis is category-theoretical representation of hyperedge disentanglement.\n \nNow, let’s bring this abstract concept into the representation space. In category theory, these mappings are carried out through a functor. Functor $F$ will map entities in the definition into representation spaces.\nSince functors are structure-preserving maps between different domains, the commutative diagram will also hold in the representation space.\n\nDisentangle and then message passing : $ F(V) \\rightarrow F(V_{c}) \\rightarrow F(E_{c})$\n\nMessage passing and then Disentangling : $ F(V) \\rightarrow F(E) \\rightarrow F(E_{c})$\n\nThus, regardless of the sequence of action, we get the same hyperedge factor representation $F(E_{c})$.\n\n# Q2 & W2\nThe criterion we derived from category theory establishes that performing message passing followed by disentanglement (disentanglement is implemented as MLP), or disentanglement followed by message passing, should yield consistent results. As demonstrated in our proof-of-concept model, when the hyperedge factor representations obtained through these two paths are similar, we interpret this as the corresponding factor being highly relevant to the hyperedge. Consequently, we assign a higher weight ($\\alpha_{i}^{k}$) to that factor, allowing it to contribute more significantly to the final representation.\n\n# Q3\nThe naturality-based criterion is derived directly from the definition of hyperedge disentanglement. By guiding the disentanglement process based on this property, our model was able to capture factors that are closely related to the functional context (i.e., hyperedge context), as clearly demonstrated in the visualizations. Because the proposed criterion effectively captures contextual information, our model achieved the best performance on all datasets except CESC. Notably, on four of the datasets, it outperformed the second-best model by a margin of more than 1.5 point (up to 4.7 point), further highlighting its effectiveness."}, {"review_id": "HA3rYQWMMB", "reviewer": "Reviewer_DgXL", "summary": "The paper introduce factor-representation consistency, a new criterion to find hidden factors inside hyperedges. This criterion comes from category theory: if we first disentangle nodes then aggregate, result should be same as first aggregate then disentangle. Authors build a model named Natural-HNN that implements this rule as an extra loss. They evaluate the peformance of the model on six cancer hypergraph datasets and show that the method performs better that baselines. They also show that the learned factors match known gene-pathway functions.", "questions": "Q1 – Could you add, in Section 1 or a new “Problem Definition” part, formal explanations of (a) what exactly you call disentanglement in hypergraphs and (b) what a factor is (latent variable, dimension in the embedding, etc.)?\n\nQ2 – Beyond empirical evidence, can you give a theoretical or synthetic argument showing why the equality between “aggregate → disentangle” and “disentangle → aggregate” is expected when factors are truly causal for a hyperedge? Even a toy proof or a controlled synthetic experiment would strengthen the idea.\n\nQ3 – Domain generalisation evidence\nAll experiments are biomedical, while e.g. HSDN was tested on several domains (social, recommendation, citation). This makes it hard to judge whether your approach really generalises beyond genomics. Could you add at least one public non-biomedical hypergraph benchmark, e.g., DBLP-coauthorship, MovieLens, Yelp, or explain why such transfer is not possible? Showing competitive performance outside the cancer datasets would greatly increase the perceived significance of the method.\n\nQ4 – Table 7 hints at a variant of Natural-HNN trained without the consistency loss. Could you move this comparison to the main results, report exact numbers, and discuss the gap? This will clarify how much improvement comes from the new criterion itself.", "limitations": "The paper does include a dedicated limitations section in Appendix, but it largely reiterates compute cost and future work; it should explicitly discuss domain specificity and absence of theoretical guarantees.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths  \nS1: Good technical quality. The naturality-based loss is clearly derived, and experiments use 10 random splits plus useful ablation studies.  \nS2: Originality. To the best of my knowledge, this is the first work that applies the naturality concept from category theory to hyperedge disentanglement.  \nS3: On six TCGA cancer datasets the method improves macro-F1 by about 4–8 pp over twelve strong baselines.  \nS4: Implementation details, hyper-parameters, and anonymous code are provided.  \n\nWeaknesses.  \nW1: Key concepts are not defined. Section 1 never explains what “disentanglement” means in the hypergraph setting. The term “factor” appears many times but a formal definition is missing, so the reader can only guess that it refers to some latent variables explaining hyperedges.  \nW2: Naturality lacks theoretical support. The paper shows that the naturality criterion works in experiments, but it first needs a higher-level justification—why should consistency between “aggregate-then-disentangle” and “disentangle-then-aggregate” reveal true factors?  \nW3: Domain scope is narrow. All experiments come from one biomedical domain; there is no evidence that the approach generalises to social, recommendation, or synthetic hypergraphs.", "was_revised": true, "final_justification": "While I am satisfied with some of the responses, I believe the clarity issues raised in W1/Q1 are not easily addressed without a more substantial restructuring or rewriting of parts of the paper. I consider this a good paper overall, but given the outstanding concerns, I will maintain my current rating for now and await further discussion and the conclusion of the discussion period.", "rebuttal": "# Q1 & W1\nDisentanglement refers to the process of decomposing an entity into multiple independent latent components, or factors. For example, a feature vector can be projected into several distinct latent spaces, each capturing a different aspect of the underlying information, resulting in a set of factor representations.\n\nIn the context of hyperedge-level disentanglement, this involves decomposing a hyperedge into multiple types (factors), each corresponding to a distinct interaction mechanism or condition. We will add the explanation in the final version of the paper.\n\n# Q2 & W2\nLet us begin by analyzing the definition of hyperedge disentanglement in category theoretical viewpoint. Let $V$ be a set of nodes in a hyperedge and let $E$ be the result of group interaction (intuitively, conclusion of group discussion). The transition from $V$ to $E$ is the group interaction mechanism. In category theory, we can represent this as morphism $f$ : $V \\rightarrow E$.\n\nBy the definition of hyperedge disentanglement, mechanism can be decomposed into several independent mechanisms. (i.e., morphism $f$ can be decomposed as the product of independent morphisms $f_{c}$ and $f_{d}$. $f = f_{c} \\times f_{d}$) \nAs can be seen in Figure 13 at the Appendix A.5, objects (nodes $V$ and conclusion $E$) can be decomposed correspondingly : $V = V_{c} \\times V_{d}$ and $E = E_{c} \\times E_{d}$ such that $f_{c} : V_{c} \\rightarrow E_{c}$ and $f_{d} : V_{d} \\rightarrow E_{d}$.\n\nNow, let’s examine this from the perspective of factor $c$. We have the commutative diagram for factor $c$. Disentangle first ($V \\rightarrow V_{c}$) and then performing interaction $ f_{c} : V_{c} \\rightarrow E_{c} $ results $E_{c}$.\nPerforming interaction $f : V \\rightarrow E$ first, and then disentangling $E \\rightarrow E_{c}$ results $E_{c}$.\nThis is category-theoretical representation of hyperedge disentanglement.\n \nNow, let’s bring this abstract concept into the representation space. In category theory, these mappings are carried out through a functor. Functor $F$ will map entities in the definition into representation spaces.\nSince functors are structure-preserving maps between different domains, the commutative diagram will also hold in the representation space.\n\nDisentangle and then message passing : $ F(V) \\rightarrow F(V_{c}) \\rightarrow F(E_{c})$\n\nMessage passing and then Disentangling : $ F(V) \\rightarrow F(E) \\rightarrow F(E_{c})$\n\nThus, regardless of the sequence of action, we get the same hyperedge factor representation $F(E_{c})$.\n \nLet’s now explain this concept through a more intuitive example. Consider a group discussion involving several people. In this scenario, each factor represents a discussion topic. Suppose there are two independent discussion topics, $c$ and $d$, that do not influence each other. Whether we discuss both topics and then extract the conclusion for $c$, or we focus on topic $c$ alone to derive the conclusion, the conclusion of discussion for $c$ will be the same. This is because $c$ and $d$ are independent, and thus do not affect one another.\n \n\n\n# Q3 & W3\nThe results on the hypergraph benchmark datasets in Appendix D include citation networks such as DBLP-coauthorship. For disentanglement to lead to substantial performance improvement, the underlying context or factors must be strongly associated with the labels. If the context exists but is only weakly related to the labels, even a well-disentangled representation may not yield a significant performance gain.\n\nFor example, in co-citation and co-authorship networks, hyperedges are formed by grouping all documents cited by a single paper or all papers authored by a single researcher. While citations between a pair of papers may carry meaningful context (e.g., a specific reason for citation), it is difficult to argue that a group of cited documents shares a well-defined or coherent context. Even if we assume that some interaction context exists in such hyperedges, the connection between this context and node labels is often unclear. In such cases, even if contextual information is captured, it may not contribute meaningfully to label prediction performance.\n\nIn contrast, in the cancer subtype classification task, pathways are known to carry functional context [2,3] and are closely associated with the labels (i.e., cancer types) [2,4]. To summarize briefly, pathways are associated with specific biological functions (i.e., factors), and whether these functions operate normally—potentially reflected in the factor representations—can determine the type of disease. This indicates that, in this case, the factors are highly relevant to the labels.\n\n$\\textbf{To assess the applicability of our model beyond genomics}$, we conducted a hyperedge classification experiment on the Chemical Reaction Dataset 1 [1]. In this task, the goal is to classify the reaction type (i.e., the hyperedge label) given a set of molecules (nodes) participating in the reaction.\n \nDue to time constraints, we selected a subset of representative baseline models to evaluate their performance in terms of Macro-F1 score. The results are summarized in the table below. We will evaluate the remaining baseline models and include their performance results in the final version of the paper.\n|                                        | HGNN        | HNHN        | AllDeepSets   | AllSetTransformer | HSDN        | **Natural-HNN (ours)** |\n|----------------------------------------|-------------|-------------|---------------|-------------------|-------------|--------------------|\n| Molecule Reaction Dataset 1 (Macro-F1) | 0.449±0.005 | 0.257±0.008 |  0.493±0.023 |  0.727±0.026     | 0.491±0.023 | **0.773±0.008**        |\n \nIn the chemical reaction dataset, Natural-HNN shows a large performance gap compared to HSDN, which performs hypergraph-structure level disentanglement, and also outperforms AllSetTransformer by approximately 5 percentage points. These results suggest that naturality-based disentanglement can be effectively applied not only in biomedical domain but also in other domains.\n\n# Q4\nTable 7 presents the performance with and without the factor discrimination loss introduced in Section 4.4. The comparison based on the presence or absence of the naturality constraint (consistency) was shown in Table 11. We will incorporate these results into the main table and add corresponding explanations in the final version of the paper.\n\n\n\n[1] DLGNet: Hyperedge Classification through Directed Line Graphs for Chemical Reactions (OpenReview)\n\n[2] Mapping biological process relationships and disease perturbations within a pathway network.\n\n[3] Disentangling the multigenic and pleiotropic nature of molecular function\n\n[4] Identifying cellular cancer mechanisms through pathway-driven data integration"}]

Author rebuttal:

Meta-review: This paper's key strengths include a well-structured derivation of a theoretically grounded loss function, strong empirical performance (4-8% macro-F1 gains over 12 baselines on bioinformatics tasks), and interpretable factor visualizations that align with biological knowledge. However, the work requires further development in several areas: 1) Broaden evaluation beyond biomedical data to include social or synthetic hypergraphs where factor ground truth may be controlled; 2) Strengthen theoretical justification for the naturality criterion and formally define core concepts like "disentanglement" and "factors"; 3) Include structural analysis and task-specific evaluations to better validate the theory; 4) Clarify whether category theory is essential or if simpler methods could achieve similar results. With these improvements, this promising framework could become a cornerstone for hypergraph representation learning.
Decision: Accept (poster)