Hypergraph Dynamic System
Authors: Jielong Yan
Yifan Feng
Shihui Ying
Yue Gao
Venue: ICLR 2024 poster (2024)
Keywords: Hypergraph
Ordinary Differential Equations
Dynamic System
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Recently, hypergraph neural networks (HGNNs) exhibit the potential to tackle tasks with high-order correlations and have achieved success in many tasks. However, existing evolution on the hypergraph has poor controllability and lacks sufficient theoretical support (like dynamic systems), thus yielding sub-optimal performance. One typical scenario is that only one or two layers of HGNNs can achieve good results and more layers lead to degeneration of performance. Under such circumstances, it is important to increase the controllability of HGNNs. In this paper, we first introduce hypergraph dynamic systems (HDS), which bridge hypergraphs and dynamic systems and characterize the continuous dynamics of representations. We then propose a control-diffusion hypergraph dynamic system by an ordinary differential equation (ODE). We design a multi-layer HDS$^{ode}$ as a neural implementation, which contains control steps and diffusion steps. HDS$^{ode}$ has the properties of controllability and stabilization and is allowed to capture long-range correlations among vertices. Experiments on $9$ datasets demonstrate HDS$^{ode}$ beat all compared methods. HDS$^{ode}$ achieves stable performance with increased layers and solves the poor controllability of HGNNs. We also provide the feature visualization of the evolutionary process to demonstrate the controllability and stabilization of HDS$^{ode}$.

Published as a conference paper at ICLR 2024
HYPERGRAPH DYNAMIC SYSTEM
Jielong Yan', Yifan Feng', Shihui Ying" & Yue Gao School of Software, BNRist, THUIBCS, BLBCI, Tsinghua University *Department of Mathematics, School of Science, Shanghai University fyanjl. jason, @vanfeng97 email. com, shying shu, edu. cn ganyue@tsinghua edu.cr
ABSTRACT
Recently, hypergraph neural networks (HGNNs) exhibit the potential to tackle tasks with high-order correlations and have achieved success in many tasks. How- ever, existing evolution the hypergraph has poor controllability and lacks suf- ficient theoretical support (like dynamic systems), thus yielding sub-optimal per- formance. One typical scenario is that only
two layers of HGNNs achieve good results and more layers lead to degeneration of performance. Under such circumstances, it is important increase the controllability of HGNNs. In this paper, we first introduce hypergraph dynamic systems (HDS), which bridge hypergraphs and dynamic systems and characterize the continuous dynamics of representations. We then propose control-diffusion hypergraph dynamic system by an ordinary differential equation (ODE). We design a multi-layer HDSode as a neural implementation, which contains control steps and diffusion steps. HDS has the properties of controllability and stabilization and is allowed to capture long-range correlations among vertices. Experiments 9 datasets demonstrate HDS beat all compared methods. HDSode achieves stable performance with increased layers and solves the poor controllability of HGNNs. We also provide the feature visualization of the evolutionary process to demonstrate the controlla- bility and stabilization of oDD
INTRODUCTION
Real-world correlation data inherently includes high- order correlations that graphs cannot fully depict, such
as group ties social networks (Bu al. (0100) and
co-actor relationships in movies (Fan al. 2021I. The
60
hyperedge in hypergraphs can connect two or more ver- tices, allowing the hypergraph to perform high-order relation modeling compared to the graph. Recently, hy-
Accuracy(%)
50
pergraph neural networks have gained interest due to
their ability to handle high-order correlation tasks such drug-targed target interactions (Ruan al.) 202T), social ommendation (Xia et al. (2021). and gene expression im- putation (Viñas al. 2023). The information propaga- tion from layer to layer in traditional convolutional neu- ral networks can be regarded as discrete information dif- fusion (Lin et al.) 20177, [Saharia et al., 20222; Rombach ct al. 2022). and as the layer goes deeper (more diffusion steps), the expressive ability of features also increases
HGNN
HGNN"
HDS
16
32
64
#Layer
Figure 1: Performance comparison of different methods with respect to var- ied numbers of neural network layers on Cora-CA CA dataset.
RRlnick & Tegmark 2017 Li et al. 2021).
However, we found that existing hypergraph neural networks tolerate only small diffusion steps (e.g., HGNN (Feng et al.) 2019) contains only 2 layers), while the performance drops significantly by raising layer numbers, as shown in Figure The reason is that diffusion in hypergraph neural networks is merely simple message-smoothing within neighbors, resulting in poor controllability
"Corresponding author: Yue Gao

---

Published as a conference paper at ICLR 2024
and stabilization with controllability referring to the capability fine-tune and adjust the diffusion process. Graph ODE- based methods bridge graph evolution and dynamic systems to the smoother representation of diffusion dynamics, enabling deeper networks (Poli et al., 2019 Xhonneux et al. 2020 Rusch et al., 2022). However, is challenging to directly apply these methods to high- order structures since diffusion in pair-wise correlation structures (graphs) and beyond pair-wise correlation structures (hypergraphs) follow different paradigms. In this paper, we aim to propose a hypergraph dynamic system improve the controllability and stabilization of information diffusion on the hypergraph, thereby improving the expressive ability of features as the number of diffusion steps increases.
In this paper, we theoretically introduce hypergraph dynamic systems, which bridge hypergraphs and dynamic systems. We propose a specific hypergraph dynamic system based on a control-diffusion ODE. Based on this, we propose neural network implementation HD$ode that achieves control- lable and stable long-distance diffusion on hypergraphs. Our HDS method exhibits steady perfor- mance as the diffusion steps (layers) increase, shown in Figure I We also present the properties of HDSode, including stability analysis and the connection to hypergraph neural networks. In our ex- periments, we employ 9 real-world hypergraph benchmarks and thoroughly evaluate HDS in an inductive setting and a production setting with 8 compared methods to validate the effectiveness of HDS Furthermore, we provide feature visualizations of the evolutionary process demonstrate the controllability and stability of HDSode, We summarize our contributions as follows:
We introduce hypergraph dynamic systems to establish the connection between hypergraph and dynamic systems. This dynamic system characterizes dynamic continuous representa- tions. We then propose a control-diffusse hypergraph dynamic system based on an ODE.
We design a multi-layer framework HDSode as a neural implementation of the hypergraph dynamic system to generate accurate vertex representations and prove the properties of HDSode including stability analysis which indicates that HDS can capture long-range relations among vertices.
We perform an extensive empirical evaluation of HDSode on 9 datasets, indicating that HDS can achieve best performance compared with all methods. Moreover, HDS can achieve stable performance with respect to the increase of 16 or more layers, which solves the poor controllability issue of HGNNs.
2 RELATED WORKS
Hypergraph neural networks. Hypergraph neural networks have been proposed for convolution operations on hypergraphs to handle non-Euclidean hypergraph data, which is first introduced from the spectral perspective by HGNN (Feng et al.) 2019). Hyper-Atten (Bai et al., (02T) additionally focuses the hypergraph attention module based on HGNN. Besides, HyPerGCN (aadati et al. 2019) is proposed for training GCN on hypergraphs by converting hypergraphs into graphs with intermediaries to represent hyperedges. In addition to the spectral- methods mentioned above, HGNN (Gao et al. 2022) provides a spatial-based method for propagating messages from vertices hyperedges and then vertices. UniGNN (Huang & Yang 2021) presents unified structure for message passing in graph and hypergraph neural networks, allowing common graph neural network models (e.g.. GCN (Kipf & Welling 2017), GAT (Veličković et al. 2018), GIN (Xu et al. 2018), and GraphSAGE (Hamilton et al. 2017)) to be generalized to hypergraphs.
Neural ordinary differential equations (Neural ODEs). Neural ODE is first proposed by Chen et al. (2018) to represent the continuous dynamics of the hidden representations. ODEs parame- terized by neural networks have been utilized in recent years to analyze structured graph data and develop connections between dynamic systems and correlation structures. GDE (Poli et al.) 2019) is a continuous deep correspondence formal extension of graph neural networks. CGNN (Xhonneux et al. 2020) characterizes the continuous dynamics of vertex representations in terms of solutions to linear graph diffusion differential equations. In addition, GREAD (Choi et al. [022] adds a reaction term to the graph diffusion ODE to obtain sharpening of the vertex representation, and GraphCON (Rusch et al.) 2022) models control and damping oscillators and couples them based on the graph structure. Further, there are also implementations based on partial differential equations to model
2

---

Published as a conference paper at ICLR 2024
deep learning on the correlation structure continuous diffusion process (Chamberlain et al. 2021 Thorpe et al. 2021 Bodnar et al. 2022 Eliasof et al. 20211.
PRELIMINARY
Notations and problem statement. Compared to the simple graph, each hyperedge in the hyper- graph is a subset of the vertex set. Generally, a hypergraph is defined as g = (V, E) with V, & representing the vertex set and hyperedge set, respectively. The hyperedges is denoted by an cidence matrix H {0, 1} VIx|éi, whose entities are defined as He. 1(v
with indicator function 1().. The degree of vertex v E V is defined as d(v) = Enge Ho.e. Similarly, the degree of hyperedge e E & is defined as ō(e) = Evey He,e- The diagonal degree matrices of vertex and hyperedge are denoted by D = diag( d)) and D diag(o), respectively. Given a hypergraph 9, a corresponding vertex feature matrix Z, E R|v|xe, and a corresponding hyperedge feature matrix Ze R [xxx, our goal is to learn vertex representation Y. and hyperedge representation Y..
Hypergraph neural networks. Most current hypergraph neural networks follow the message- passing framework. In each layer, the input vertex representations are first aggregated into hyper- edges, and then the output vertex representations are obtained from the corresponding hyperedges. Formally, in the k-th layer, the output vertex representations ax!*) are obtained from the previous representations using aggregation function AGG and update function UPD as:
(A) UPD(x!!-11 (G e (c)))))), where the (v) and (e) are the vertex and hyperedge neighbor function, respectively.
a!(e)
AGG(( v E (G(e)))) (I)
Neural ordinary differential equations. For certain types of models, including residual networks (He et al. 2016), the conversion of the hidden feature is regarded as the following discrete system: + xtt) f(x(tt) ((t)). can be considered as the forward Euler discretization form of the following first-order ODE equation with time step At
f(x(t), 0), where is a parameterized function defined a dynamic system. In the remainder of the paper, at will be abbreviated à for the sake of brevity.
4 METHOD
In this section, we first theoretically introduce hypergraph dynamic systems. Then, we provide specific hypergraph dynamic system form based on an ODE. Next, we divide the ODE into a control step and diffusion step by an ODE discretization for neural implementation. Furthermore, we describe the detailed neural implementation of the HDDode framework, and also the time complexity of the control step and the diffusion step.
4.1 HYPERGRAPH DYNAMIC SYSTEM.
We first propose the definition of hypergraph dynamic systems based on the following equation:
= f (
X (0) and
X xx x Z.
[ 2,
(t)
(2)
where X!.()) and Xt(t) represent the vertex representation matrix and the hyperedge representa- tion matrix at time t, respectively. The function represents the velocity of representation in the dynamic system, Z, and Z. represent the initial conditions of vertex features and hyperedge fea- tures, respectively. Due to the continuous existence of timestamp t, the above hypergraph dynamic systems can produce the representation status at any moment.
ODE-based hypergraph dynamic system. The velocity function in equation can be described in different ways. We consider the velocity function as union of a control function and diffusion function to propose an based hypergraph dynamic system as follows:
X
go X,
X
ge
+
X (1)
(3)
3

---

Published as a conference paper at ICLR 2024
HDS ode Layer
1
(t
Ln
Time Line
(x(
Xolter1)
Control Step
Diffusion Steps
to
Hypergraph Sequence
[k
fx+1
Figure 2: Illustration of our HDS framework.
Here, the first term, 9N and 9e are the control functions, acting as the control velocity each ver- tex representation and hyperedge representation, respectively. The second term is the diffusion term, where denotes the diffusion velocity effect between the vertex representation and the hyperedge representation in the dynamic system by the correlation of the hypergraph. The diffusion term de- scribes the process by which features or representations teleport across the vertices and hyperedges of the hypergraph. The control term specifically refers fine-tuning step that complements the primary diffusion process and acts an auxiliary function, adjusting and controlling the diffusion term aligm more precisely with the downstream goals.
X
(0) (0)
I' ((x(((
x.)) xx di.
(4)
Given vertex features X. (0) and hyperedge features X.(O) input, the vertex representations (T) and hyperedge representations (T) of time T are generated using the integral for the learning tasks in the hypergraph equation 4 Not only can obtain accurate final represen- tations, but more importantly. we can also acquire the dynamic changes of representations from (0), (0) to (T), (T) by changing the upper bound of the integral.
ODE discretization with Lie-Troi Tie-Trotter splitting. We expect to propose multi-layer neural net- work framework oDS related to the above ODE-based hypergraph dynamic system to obtain accurate representations. We first employ the Lie-Trotter (Geiser 2009) splitting method for the discretization of equation
3 which is as follows:
(t,
} = X (
(t) (t)
(x. (t)) 9e (x. (t))
X.
X, (t X (t 2
X (t+ X (t
(t+++
+
(5)
where the time step is integrated into control functions Jv, Je, and diffusion matrix A. We notice that each iteration representations the dynamic system has a time interval of and time iteration is separated into control step and diffusion step. Similar to the residual network, the control step modifies the representation of vertices and hyperedges by the control functions. The diffusion step propagates representation messages between vertices and hyperedges according to matrix A. It worth mentioning that the diffusion step is parameter-fie free. the diffusion matrix A is suitably designed, representations stable specific value in the diffusion step with proof in Section D
4.2 HDSode; NEURAL IMPLEMENTATION OF ODE-BASES HYPERGRAPH DYNAMIC SYSTEM
In the following, we present the implementation of HDS framework to the previous analysis by introducing the neural implementation of the control step and diffusion step in the HDDSode layer, respectively. Furthermore, will analyze the time complexity of the steps, respectively. The illustration of the HDS framework can be found in Figure
Neural implementation of control step. Our HDS layer allows any function with the same input and output dimensions control functions. In this paper, take simple one-layer fully
4

---

Published as a conference paper at ICLR 2024
connected networks modifications of vertex representations and hyperedge representations, spectively. Specifically, can be expressed by the following formula:
Xx(1 X
xx((t) x
+o ( x.,.X.t)) b. (
(.. +
(6)
where is the activate function, W.. We E Rexe are the learnable weight matrices of vertex representations and hypergraph representations, respectively, and b,, b. E R are learnable biases.
Neural implementation of diffusion step. the diffussin process. If not suitably created, vertex representations and hyperedge representations will diverge and become uncontrollable. In this study, we provide design that holds both stability and interpretability, as follows:
The design of the diffusion matrix A is essential during
X. + 1)] (t(++))
(t xx
X (t + } and A = (t
- a a.D. H -a (7)
a, H
+
where Oy and aa are hyperparameters representing the teleport probabilities of vertices and hy- peredges, respectively. We furthermore expand the matrix multiplication term obtain the vertex (1 ag (t
representation as (t+ that the vertex representations stay unmodified with a keep-rate 1 ay in the diffusion process. The second term denotes that the representation of the hyperedges directly connected to each vertex is aggregated by average with contribution proportion of agg, where H + ()) represents vertex- level aggregation, and D represents an average normalization matrix. Similarly, the hyperedge representation can be calculated as X, 11+1) a. D. H X. 5) +)++((( - X (!++ 5), where the first and the second term represent the aggregation from the vertex representation at ae rate and the original representation retained at 1 ag rate, respectively.
) (t + () D. (t. + 1). The first term denotes
We obtain the vertex and hyperedge representations of any non-negative integer timestamp & using the implementation described above. The hypergraphs from time to to time f, constitute hy- pergraph sequence as the left part in Figure [ which corresponds to the evolution process of the hypergraph dynamic system. Once the time T of the hypergraph dynamic system is selected, the final vertex representation is Y, = (T)) and the hyperedge representation is Y. = =x( The whole algorithm of HDS is shown Appendix A
Time complexity analysis. Here, we analyze the time complexity of the control step and the diffusion step in each HDS layer. In the control step, the running time is limited by multiplying the weight matrices and the representations, so the time complexity of the control step is O([V) + [El)c2) where denotes the dimension of representations. In the diffusion step. the running time is (1+
limited by the matrix multiplication operations of representations aggregation (i.e., H X ++ 2) and H x, + 1)). Considering that the incidence matrix H is sparse matrix, the time complexity is (((tt(D.)) tr (r)))) It should be noticed that the time complexity of the control term is quadratic concerning the representation dimension, where the diffusion term is linear. Therefore, in the implementation, we mask the control function in most time iterations to lower the total running duration of the framework (i.e., control step is conducted every certain number of layers).
PROPERTIES OF HDS
In this section, we introduce the properties of our proposed HDSoode First, we provide the eigenvalue propositions of the diffusion matrix A HDDode and then explore the stability of diffusion steps. Then, we discuss the relationship between HDDood and hypergraph neural networks.
5.1 STABILITY ANALYSIS.
Since the diffusion step in the ODE reflects the time iteration of the vertex and hyperedge repre- sentations, the stability analysis of diffusion is essential. We first analyze the proposition of the cigenvalue of the diffusion matrix A. Then, we perform a stability analysis on the diffusion process.
Proposition 5.1. Assume that the diffusion matrix's eigendecomposition is A UAU with eigenvalue matrix A = d)ag(X;) and eigenvectors u, in equation 7 the eigenvalue X; lies in the left half-plane of the complex plane is O.
5

---

Published as a conference paper at ICLR 2024
Table 1: Test accuracy (%) and standard deviation of semi-supervised vertex classification on transductive setting. "OOM" and "Avg. rank" represent "out of memory" and "Average rank", respectively. The best results are shown bold.
Model
Cora-CA
DBLP-CA
News20
IMDB4K-CA
IMDB4K-CD
DBLP4K-CO
DBLP4K-CP
Avg. rank
GCN
65.99
22.2221..5
67.57 an.0
48.47.2..
41,02=222
90.18_122
64.47_m0
7.6
GraphSAGE
66.44g282
81.072150
69.59
42.061g1.0
41.07=211
92. 12.18_na8
64.34_158
8.0
GDE
e1.0
69.95
43.9G
41.80=095
92.45-m44
67.71-2.4
4.9
GraphCON
66.72
22.06g1.1
OOM
43.94.22.
41.922229
0OM
67.94_1.0z
444
HGNN
67.58e188
83
76.58=0.44
43.21=2.50
41.08=2.45
03.46
67.99-2.12
4.0
HGNN
66.85+221
62.40+1.27
76.49e1.30
43. 1..12
41.49--241
93.46-100
68.76-275
UniGCN
66.472204
82.3621.m
76.5621.1
43.34aa..
41.33g250
03.28_na7
67.68_1.90
UniSAGE
a..59
$2. 16
75.52
42.82.22.0
41.62g305
93.64_aas
07.81=2122
HDS
68.99 28
83. 2..55
76. 15
44.20 44.26g2.11
42.30,2.2
93.8520.50
69.52.110
1.0
The proof is provided in Appendix [ The system is stable when the real part of the eigenvalue is less than 0. the eigenvalues have additional non-zero imaginary parts, the system will oscillate and the oscillation will decrease with time. The number of eigenvalues is then determined to further investigate the characteristics of the diffusion matrix.
Proposition 5.2. The multiplicity of 0 eigenvalues after eigendecomposition of the diffusion matrix A is equal to the number of connected components in the hypergraph.
The proof is provided in Appendix C
This property is the same the property of our commonly used graph Laplacian matrix. Once all of the hypergraph's 's vertices are reachable from one other, the hypergraph has only one connected component and the multiplicity of eigenvalue in A is 1. For ODE containing only diffusion terms as follows:
|V++18 Σ
x X
X (1)
X X
A
with solution
ATA
uuu
(8)
If Re(A:) < O, there lim Ax O, while lim ex for A O. This indicates that the representations are stable to the state corresponding to the 0 eigenvalue by the diffusion. When each class of vertices the hypergraph connects to vertices within the class, vertices in distinct classes are stabilized to various representations. Control terms are required to stabilize distinct categories of vertices to different representations there are inter- suer-hyperedges of classes. For the global including the diffusion step and the control step, give stability condition in the Appendix D
5.2 COMPARISON WITH HYPERGRAPH NEURAL NETWORKS.
We formalize the relationship between HDS and hypergraph neural networks. Consider sit- uation where the control term is masked and the teleport probabilities a, and a₂ both 1, the vertex representations between every two layers include the relationship as X,,t + 2) HD ' X., whose form is consistent with linear HGNN layer (Gao et al. 2022) with- out learning parameters. Given that the teleport probabilities in HDS are susceptible to modi- fication by the hypergraph structure and the control term finetunes the representation of diffusion, oD has better chance of producing an accurate representation than HGNN*
EXPERIMENTS
In this section, we conduct experiments and compare HDS graph neural networks, graph ordi- nary differential equations, and hypergraph neural networks on various benchmarks.
6.1 SEMI-SUPERVISSDD VERTEX CLASSIFICATION
Our following experiments concentrate on the semi-supervised vertex classification task, which aims to predict the labels of unlabeled vertices in hypergraph given known partial vertex labels and all vertex features. The output layer Y. o o(Y.), RDixe + | acts the final vertex representation to obtain the probability that the vertex belongs to each category, where o represents the number of categories.

---

Published as a conference paper at ICLR 2024
Table 2: Test accuracy (%) and standard deviation of semi-supervised vertex classification on a production setting with inductive and transductive predictions. "OOM", "prod.", "ind.", "trans." denote "out of memory", "production", "inductive", "transductive", respectively. The best results shown in bold.
Model
Cora-CA
DBLP-CAA
News20
IMDB4K-CA
IMDB4K-CD
DBL.44-CC
DBLP4K-CT
prod.
65. 13g487
51.62+1.42
67.13
42 22.00
41.12
90. 30.1641.8 a2
63.48e1.7
GCN
ind.
64.45e455
81.344171
67. 67.33-0.05
41 41.40+102
42 12.05
90,80+1 32
62.81+2.2
trans.
65.4614.83
11.22
07.13_0.83
42.50
40.8412.00
10.45
63,461144
prod.
00.522238
80.67
68.9 18.71
42.98
41.28
61.53
GraphSAGE
ind.
67 37.33
50.91+1.0
68. 51-1.95
45. 45.04=2247
41.58
01.58+e0. 50
62.11+4.8
trans.
66 66.08e.2.5
79.62+1114
68.99-1.85
42.67 42.67
41.15+2225
91.74+070
60.7d+3587
prod.
16.74
12.53
OOM
42.63
41.51 12.76
OOM
67.2211.8
GraphCON
ind.
64. 163.07
81.37 12
OOM
42.25
41 A1.255
OOM
67.53+2.13
trans.
06.52+3.44
81.38++2.0
OOM
42.79 2.22
41.58
OOM
67.14+2.2
prod.
11.96 06.72.3.08
77.09 .0.00
41.70.3.02
41.23
83.76.088
67.2511.7
HGNN
ind.
60.472211
81. 11.63.+122 28
77.02-0.55
11.18
42.27
93. 19.73-077
65.08+366
trans.
66.
06.66es.55 53
81.46+1008 08
77.08-0.e
42. 12.10+22))
40.72+3.01
73
67.42+1.55
prod.
07.401a.18
11.44
76.76_o.83
41.79
41.23
93,49 21
66.97
HGNN
ind. 07.0514.12
81.33
70.56
1451
42.27 12.50
93.38 09
65.7912.70
trans.
07.26+3.41
80.93 25
76.72-0.78
42.33
40.72+3.01
93.57+1.0
66.86+1.5
prod.
66.7712.28
$1.67 11.44
76. 6..44_1.
42,43
41.39 12.00
93.34
66.6311.80
UniGCN
ind.
66. 14..8
81.41
76.50 1.53
43.02.2294
41.65
93.30 00
66.3412.52
trans.
21.00
81.09
76. 7..6
42.58
41.45g2.75
93.61 93.61 20.36
66.31
prod.
68 (13+2.
82.01+1.1
76. 15.10
43.
43.14+3..0
41 11.224a.m0
93.37 +0
68.05+1.8
UniSAGE
ind.
68. 22.85
$1.97 11.45
15.97_1.26
42.1614.47
42.01 10.58
B3.30 48
66.92199.8
trans.
07.88
81.38
76. 10_1.00
43.3222.87
40.83
93.70 49
68. 08.13.1.447
prod.
69. 19.17=ss.1
82. 32.99=0.s
77.16+1.04
43.05
41.98 42.49
93.90 $3.96-.. 42
68. 13.66522..
HDS
ind.
71. 13.04
83. 13..222
77. 77.26
43.711422
42.3412.0
93.26 1126011001
67. 35.35
trans. 68. 29.3.
82.23_0065
77.11
42.74
41.8712.42
94.10_073
68. 61 22.43
Datasets. As explained below, we employ 9 publicly accessible hypergraph benchmark datasets from existing research on hypergraph neural networks, including Cora- Cora-CA and DBLP-CA from Yadati et al. (2019), News20 from Assncion & Newman (2007). IMDB4K-CA and IMDB4k CCD from Fu et al. (2020), DBLP4K-CC and DBLP4K-CP from [Snn et al. (201T). Cooking from Gao ct al. 2022), and NTU from Chen et al. (2003). The details of datasets are shown in Appendix
Experiment settings and details. the following experiments, we evaluate HDSode and the compared methods in an inductive and production setting. In both settings, fix the total number of known label vertices the training set and the validation set, which contains a total of 1, 500 vertices including 10 vertices per class for training. Vertices not in the training set and validation set are for test. The training, validation, and test data for each experiment are divided five times at random, and the average performance and standard deviation of each method are reported for fair comparisons, Other experiment settings, details, and implementations are in the Appendix F
Compared methods. We compare oDS to comprehensive set of baselines divided into three groups. The first group is the graph neural network (GNN) group, in which we select two popular architectures, namely Graph Convolutional Network (GCN) (Kipf & Welling (017)) and GraphSage (Hamilton et al., 2017).
addition, we compare two ODE-based GNN models, Graph Neural Ordinary Differential Equations (GDE) (Poli et al. 2019) and Graph-Coupled Oscillator Networks (GraphCON) (Rusch et al.) 2022). In the last group, we focus on methods that compute directly on the hypergraph, including Hypergraph Neural Networks (HGNN) (Feng al. 2019), General Hypergraph Neural Networks (HGNN+) (Gao et al. 2022). UniGCN (Huang & Yang 2021 and UniSAGE (HASAGa/uaang & Yang (0011.
6.1.1 VERTEX CLASSIFICATION UNDER THE TRANSDUCTIVE SETTING.
Table T shows the accuracy and average ranking among different methods on public datasets of the transductive vertex classification task. HIDSode ranks first in all datasets, surpassing hypergraph
7

---

Published as a conference paper at ICLR 2024
Rocurocy
Acc-
HGNN
HONY
DELF
22
aTran
(a) On control steps and diffusion steps.
(b) On different vertex (c) On different vertex (d) On different vertex
numbers of DBLP4K-CP numbers of Cora-CA in numbers of IMDB4K-CA in training set.
training set.
in training
Figure 3: Influence of different parts of HDS
neural network, graph ODE, and graph neural network methods. Two hypergraph neural network techniques, HGNN+ and HGNN, with average rankings of 3.7 and 4.0 respectively, come second and third place, respectively. In the following ablation experiments, we only compare HDS with these two methods. GraphCON the graph ODE approach that ranks after HGNN+ and HGNN but outperforms the two hypergraph neural network methods (UniGCN and UniSAGE), even though relational structures can only use graphs that are less expressive than hypergraphs. Among the various comparison methods, two graph neural network methods (GCN and GraphSAGE) place last, with average rankings of just 7.6 and 8.0, respectively. We first fix the correlation structure and conduct comparisons, namely HDD graph neural network. It's important to emphasize that ODE-based methods show a clear advantage, which suggests that the continuity of neural networks strengthens vertex representations. Then we engage in another two sets of comparisons, namely HDS vs. graph ODE and hypergraph neural network vs. graph neural network. In both comparisons, the hypergraph-basee their graph- based counterparts, indicating that hypergraphs have richer ability to represent the correlation relationship than graphs. essence, binary correlation cannot adequately represent high- order correlation. Furthermore, even though graph ODE methods employ continuous processing in graph structures, they still fall below the best hypergraph neural network methods, demonstrating the significance of hypergraph structure.
hypergraph neural network and graph ODE
methods outperform
6.1.2 VERTEX CLASSIFICATION UNDER THE PRODUCTION SETTING.
The transductive setting excludes predictions for unseen vertices. For more comprehensive eval- uation in real-world production scenarios, we set up a production setting experiment that includes production, transductive, and inductive predictions, where the detailed description is presented in Appendix F22] Table delineates the performance of various methods on the vertex classification task under the production setting. HDS achieves the best results across the datasets in most cases, except for production and transductive results in IMDB4K-CA, and the inductive result in DBLP4K-CC. HDS still demonstrates advantages over hypergraph neural networks and graph ODE methods in inductive prediction. In DBLP4K-CC, the inductive result of HDS is marginally below HGNN. We hypothesize this is due to the larger hyperedge in DBLP4K-CC -CC, implying that removing inductive vertices has greater impact on HDSode diffusion process than HGNN.
6.2 ABLATION STUDIES
How does the number of layers affect HDS and hypergraph neural networks? We found that HDS has the ability to obtain long- distanee neighbor knowledge. We experiment on oDe and the two best comparison methods HGNN and HGNN+ in Cora-CA, with different numbers of layers. The results are shown Figure It is worth mentioning that the number of layers represents the farthest distance for each vertex to obtain neighbor information. Additionally, the number of lay- ers in HDS also corresponds to the termination time T. The hypergraph neural network methods HGNN and +GNN+ only receive competitive results in the shadow neural networks, which perform best at 2 layers and rapidly drop when more than layers. This means that each vertex only benefits from its local neighbors and not the full hypergraph. In contrast, HDS has achieved competitive outcomes in shallow layers although not stabilized. As the number of layers increases, it acquires long-distance neighbors and brings additional benefits. Finally, when the number of layers exceeds
8

---

Published as a conference paper at ICLR 2024
(a) 1.
(b) Att = 1/3-T.
(c) Att - 2/3-T.
(d) Att = T.
Figure 4: T-SNE visualization of vertex representation at different timestamp /.
16, HDS produces reliable results. In this situation, each vertex collects knowledge from all of the vertex in the hypergraph.
How are the contributions of the control steps and the diffusion steps? We found that both the control step and the diffusion step are indispensable in HDSode, Figure (3a) depicts the results of HDS using only the control steps and the diffusion steps in different benchmarks, denoted by (Con))" and "HDSode (Diff.))" respectively. It has been observed that removing either step reduces the results. Comparing the contributions of two steps, the result is lower if just the control step is included, since does not employ the hypergraph structure but simply initial vertex features.
How does the label rate affect HDSode? When varying the HDSode training set label rate from to 20 labels per class, we observe that the performance gradually increases (see Figure 3(b) ((c) 3(d). We notice that HDS tends to improve performance more for 1, 5 or 10 labels in each class. This is because when the number of labeled vertices is relatively limited, the distance between each unlabeled vertex and the nearest labeled vertex grows. If vertices wish to profit from label information, the layer number is required to be increased, which is challenging for HGNNs. Detailed results can be found in the Appendix G.2
6.3 FEATURE VISUALIZATION.
We found the vertex representation evolution of the hypergraph dynamic system based T-SNE Van der Maaten & Hinton 2008) visualization. Figure A vertex representations in DBLP- CA at various timestamps. DBLP-CA chosen since it contains the most vertices to provide more trustworthy visualization results. It has been discovered that as time goes from 1 to T, various types of vertices increasingly congregate. Figure [4(a] shows points of various colors interlaced, making difficult to distinguish between various kinds of vertices at first. After a time interval of about T T/3, vertices of the same subclass gradually are moved closer, wherein the yellow region only contains few vertices that are accidentally entered. When the timestamp rises close to 2T/3 boundaries between the categories become more acute and the regions of different types of vertices interact with relatively few vertices from other categories. The vertex representation is accurate at the moment, and the vertex classification effectiveness is at its peak.
T. the
CONCLUSION
In this paper. we first introduce hypergraph dynamic systems, which bridge hypergraphs and dy- namic systems. We then provide specific hypergraph dynamic system based on a control-diffusion ODE. Based on this, we propose a neural implementation framework HDS which has the ability to capture long-range correlations among vertices. We introduce the properties of HDSode includ- ing stability analysis and relationship with hypergraph neural networks, which are essential to un- derstanding oDS HDS has been evaluated vertex classification tasks in different settings. aDe has been demonstrated to obtain stable performance with increased layers while HGNNs are not controllable and stable with more layers. The evolutionary process is demonstrated by feature visualization. Future directions include extending HDS to other classes of differential equations (e.g., partial differential equations) and to the time-varying setting of hyperedges.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This work was supported by National Natural Science Funds of China (No. 62021002, 62088102),
Beijing Natural Science Foundation (No. 4222025).
REFERENCES
Arthur Asuncion and David Newman. Uci machine learning repository, 2007.
Song Bai, Feihu Zhang, and Philip HS Torr. Hypergraph convolution and hypergraph attention. Pattern Recognition, 110:107637, 2021.
Cristian Bodnar, Francesco Di Giovanni, Benjamin Chamberlain, Pietro Liò, and Michael Bronstein. Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in gnns. Advances in Neural Information Processing Systems, 35: 35:18527-18541 2022.
Jiajun Bu, Shulong Tan, Chun Chen, Can Wang, Hao Wu, Lijun Zhang, and Xiaofei He. Music recommendation by unified hypergraph: combining social media information and music content. In Proceedings of the 18th ACM international conference on Multimedia, pp. 391-400, 2010.
Ben Chamberlain, James Rowbottom, Maria I Gorinova, Michael Bronstein, Stefan Webb, and Emanuele Rossi. Grand: Graph neural diffusion. In International Conference Machine Learn- ing. pp. 1407-1418. PMLR, 2021.
Ding-Yun Yun Chen, Xiao-Pei Tian, Yu-Te Shen, and Ming Ouhyoung. On visual similarity based 3d model retrieval. In Computer graphics forum, volume 22, pp. 223-232. Wiley Online Library, 2003.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Jeongwhan Choi, Seoyoung Hong, Noseong Park, and Sung-Bae Cho. Gread: Graph neural reaction-diffusion equations. arXiv preprint arXiv:2211.14200 2022.
Moshe Eliasof, Eldad Haber, and Eran Treister. Pde-gen:
Novel architectures for graph neural networks motivated by partial differential equations. Advances in neural information processing systems, 14:3836-38899 2021.
Haoyi Fan, Fengbin Zhang, Yuxuan Wei, Zuoyong Li. Changqing Zou, Yue Gao. and Qionghai Dai. Heterogeneous hypergraph variational autocncoder for link prediction. IEEE Transactions Pattern Analysis and Machine Intelligence, 44(8):4125 4130-8883-418.. 2021.
Yifan Feng, Zizhao Zhang, Xibin Zhao, Rongrong Ji, and Yue Gao. Gvenn: Group-view convo- lutional neural networks for 3d shape recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 264 272, 2018.
Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. In Proceedings of the AAAI conference artificial intelligence, volume 33, pp. 5558-3565, 2019.
Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding. In Proceedings of The Web Conference 2020, pp- 2331-2341, 2020.
Yue Gao, Yifan Feng, Shuyi Ji, and Rongrong Hgmn+: General hypergraph neural networks.
IEEE Transactions on Pattern Analysis and Machine Intelligence, ((3):3181-3199 2022.
Juergen Geiser. Decomposition methods for differential equations: theory and applications. CRC
Press. 2009.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs.
Advances in neural information processing systems, 30, 2017.
10

---

Published as a conference paper at ICLR 2024
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Jing Huang and Jie Yang. Unigmn: a unified framework for graph and hypergraph neural networks. arXiv preprint (rXiv:2105000956, 2021.
KVV Kanuri, Bhagavathula, and K Murty. Stability analysis of linear sylvester system of first order differential equations. International Journal of Engineering and Computer Science, 9(11), I), 2020.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In International Conference on Learning Representations, 2017
Guohao Li, Matthias Müller, Guocheng Qian, Itzel Carolina Delgadillo Perez, Abdulellah Abual- shour, Ali Kassem Thabet, and Bemard Ghanem. Deepgens: Making gens go as deep as cnns. IEEE transactions on pattern analysis and machine intelligence, 2021.
Henry W Lin, Max Tegmark, and David Rolnick. Why does deep and cheap learning work so well?
Journal of Statistical Physics, (68:1223-1247 2017.
Michael Poli, Stefano Massaroli, Junyoung Park, Atsushi Yamashita, Hajime Asama, and Jinkyoo
Park. Graph neural ordinary differential equations. arXiv preprint arX152191..07532, 2019.
David Rolnick and Max Tegmark. The power of deeper networks for expressing natural functions. arXiv preprint arXiv: 17X5::100500502.. 2017.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Ding Ruan, Shuyi Ji, Chenggang Yan, Junjie Zhu, Xibin Zhao, Yuedong Yang. Yue Gao, Changqing Zou, and Qionghai Dai. Exploring complex and heterogeneous correlations on hypergraph for the prediction of target interactions. Patterns, 2(12), 2021.
T Konstantin Rusch, Ben Chamberlain, James Rowbottom, Siddhartha Mishra, and Michael Bron- stein. Graph-coupled oscillator networks. International Conference on Machine Learning, pp- 18888-18909, PMLR, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Informa- tion Processing Systems, 35:36479-36494, 2022.
Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. Multi-view convo- lutional neural networks for 3d shape recognition. In Proceedings of the IEEE international conference on computer vision, pp. 945-953, 2015.
Yizhou Sun, Jiawei Han, Xifeng Yan, Philip S Yu, and Tianyi Wu. Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. Proceedings of the VLDB Endowment, 4(11):992-1033, 2011.
Matthew Thorpe, Tan Minh Nguyen, Hedi Xia, Thomas Strohmer, Andrea Bertozzi, Stanley Osher, and Bao Wang. Grand++: Graph neural diffusion with source term. In International Conference on Learning Representations, 2021.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
11

---

Published as a conference paper at ICLR 2024
Ramon Viñas, Chaitanya K Joshi, Dobrik Georgiev, Phillip Lin, Bianca Dumitrascu, Eric R Gama- zon, and Pietro Liò. Hypergraph factorization for multi-tissue gene expression imputation. Nature Machine Intelligence, pp. 1-15, 2023.
Louis-Pascal Xhonneux, Meng Qu, and Jian Tang. Continuous graph neural networks. In Interna- tional Conference on Machine Learning, pp. 10432-10441. PMLR, 2020.
Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, and Xiangliang Zhang. Self- supervised hypergraph convolutional networks for session- based recommendation. In Proceed- ings of the AAAI conference on artificial intelligence, volume 35, pp. 4503-4511, 2021.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful graph neural networks? In International Conference on Learning Representations, 2018.
Naganand Yadati, Madhav Nimishakavi, Prateck Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. Hypergen: A new method for training graph convolutional networks on hypergraphs. Advances in neural information processing systems, 32, 2019.
12

---

Published as a conference paper at ICLR 2024
A
THE ALGORITHM OF HDS oode
Algorithm The algorithm of HDSode framework
Input: A Hypergraph G (V, E) with an incidence matrix H, a vertex feature matrix z.. hyperedge features matrix Ze degree matrix of vertices D., and degree matrix of hyperedges De. The stop time of the dynamic system T. The teleport probabilities a, Qee The control step interval 8.
Output: The vertex representations Y. The hyperedge representations Y.. I: Initialize vertex and hyperedge representations by (, <- Z. (0) <- Ze.
2: for = to T do
3:
if Current is the initial time or time has elapsed since the last control step then The control step performed according to equation 6
4:
5: else
6:
Skip the control step by (t + 2 + X Xe ()) (t)..
7: end if
8: The diffusion step is performed according to equation 7
9:
end for
10: The representaions at time 'I' are the final results by Y, + (T))), + X,(T).
return The vertex representations Y, and the hyperedge representations Y.
B PROOF OF PROPOSITION 5.
Proof. For the diffusion matrix in HDSode it in the following form:
Ctp I
D.
A
0g D
H
-a,1
Consider the sum of all rows except the elements on the diagonal. For all i e (1,2...., [V]].
Σ
Aigl = On.
since
d(u) = Σ Hn,.
(10)
330
Similarly, For all ([VI 1, | ....... 11 18])
Σ |A1gl ae,
since
5(e) > Ho,e-
(11)
jihh
VEV
According to the Gershgorin circle theorem, the eigenvalues of the matrix A satisfies ) Ry
Rg. Here
Ry (2 1: C: Ar,l S Σ
AAglAiC {1,2, [1}} (a C: ||2 a av), (12)
jf1
R3 =
Σ Jtf
M ( = (a e C s ae).
(13)
It is noticed that the sets By and Rg have similar forms, therefore they are merged further
A E Ry U Rg {= E C : ||= + max(o., 0:e)ll2 S max(o., (t)).
(14)
Therefore, the eigenvalue X; of is or distributed in the left half-plane of the complex plane (i.e., the real part of X 0).
C PROOF PROPOSITION 5.2
Proof. In the proof of this proposition, prove it from two directions, namely the multiplicity of cigenvalues is greater or equal to the connected components of the hypergraph, and the multiplicity of () eigenvalues is less equal to the connected components of the hypergraph.
13

---

Published as a conference paper at ICLR 2024
We first prove the multiplicity of 0 eigenvalues is greater equal to the connected components of the hypergraph. Considering that the hypergraph g = (V, E) contains connected components
91 = (),, 92 (V2, (2) .a).......... Gh = =V&, Ex), can define & cigenvectors U1 U2..... Ug as
follows:
1
(1) V + E&
Va V4 and u,(e) VIJ.
Ve E E
(15)
&
According to the definition of A, Au; = indicates that u, is the corresponding cigenvector of zero eigenvalue. According the size of the vertices and hyperedges in the connected component i, it can be known that
||ll|
Σ (2(()) BEV
Σ (e) = CEE
Σ DEV
1.
(16)
+++
Vi
+
E,
cEE;
Since any two connected components don't 't include the same vertices hyperedges, any two eigen- vectors don't contain non-zero values at the same element, that is
u ; 0, Vi +
(17)
Each connected component of the hypergraph can construct an eigenvector corresponding to a eigenvalue, the multiplicity of 0 eigenvalues is greater or equal to the connected components of the hypergraph.
Then we prove the multiplicity hypergraph. We multiply the rows of matrix A by different coefficient to obtain the auxiliary proof matrix A' with the same eigenvalue multiplicity, follows:
eigenvalues is less or equal to the connected components of the
D A'
H
(18)
H
De
A' is positive semidefinite matrix because it satisfies
A' ='E= ((E()))()) 0,
(19)
where the condition for the equality is that the element of the vector & corresponding the vertex and hyperedge in each connected component is a constant. It can be noted that the eigenvectors u1, u2..... U4 of A are also the eigenvectors of A'. Assume that the multiplicity of eigenvalues in matrix A' is greater than the number of connected components in the hypergraph, then
E + O,
A'E
=0 A L u1 U2,..... Ubj-
(20)
As E * O, there exists {(e) o or E(e) + O. Let's assume that there are non-zero elements of the vertices or hyperedges in the i-th connected component in &. Since the element of the eigenvectors & corresponding to the vertex and hyperedge in each connected component is a constant, & and u, satisfy E u, A O, which contradicts the hypothesis. So the multiplicity of 0 eigenvalues is less or equal to the connected components of the hypergraph.
According to the proofs in the above two directions, there only one possibility that the multiplicity of eigenvalues of matrix equal to the number of connected components of the hypergraph. [
D A STABILITY CONDITION OF THE DYNAMIC SYSTEM
We consider the overall ODE system as represented by X AX + (X),, where X encapsulates both the vertex and hyperedge rate of change Xx,, x_]" JT and g = (XW W b) denotes the control function modulating vertex and hyperedge representations. Here, the control function employs an activation function defined by a(.) = max(0..). Notably, in instances where the control term nullifies, system dynamics revert being just diffusion-driven, thus reverting stability condition ruled only by the diffusion term.
For scenarios divergent from this particular case, we front a Sylvester system defined by AX + XW, promoting further analysis based on the prior work of Kanuri al. (0020) We embark solving the constituent subsystems X AX and X W' respectively, delineating
14

---

Published as a conference paper at ICLR 2024
(t) and (t) as their respective solutions, with W standing for the transpose of the complex conjugate of W. We introduce the notion that matrix Y1 is deemed V-bounded on if there exists bound for (t) (t) over I. Specifically, a scalar M > such that suP,er 1 ( M. Similarly, a matrix function (t) V) characterized as V)-bounded if | |/ remains bounded on R.
The stability of the system is predicated on two key conditions. Firstly, the continuity of matrices A and W I, which comports with our definitions. Secondly, the existence of positive scalar K such that for all 2 O, the integral J (I) PY2 (L)V*(I))) dl, remains bounded by K. Here, (t) and (t) are the bounded solutions for (t) and (2))), respectively.
E DATASETS
Table S1: Real-world hypergraph benchmark dataset statistics.
Dataset
Cora-CA
DBIPPCA
News20
IMDB4K-CA
IMDB4K-CD DBLP4K-CC
DBLP4k-CC
Cooking
NTU
#Vertices
2, 708
41,302
16,312
4,278
4,278
4,057
1,057
7,304
2. 010
#Hyperedges
1,072
22,363
100
5,257
2, D81
20
14,328
2,755
3. 343
#Features
1,433
1,425
1,333
3,066
066
334
334
7,304
6. 144
#Classes
7
6
4
3
20
67
Table S2: The train/val/test statistics of datasets in the transductive setting.
Dataset
#Vertices
#Hyperedges
#Train/Val/Teen Vertices
Split Ratio (%)
Cora-CA
2, 708
1, 072
70/1, 430/1, 208
2.6/52.8/44.6
DBLP-CA
41, 302
22,
60/1,4 440/39, 802
0.1/3.5/96.4
News20
16, 342
100
40/1, 842
0.3/8.9/90.8
IMDB4K-CA
4, 4,278
5, 257
30/1, 470/2, 778
0.7/34.4/64.
IMDB4K-CD
4,278
2, 0808
3701,470/2, 778
0.7/34.4/64.9
DBLP4K-CC
4,057
20
1.0/36.0/66 0/6660
DBLP4K-CP
4,057
14,328
460/2,,55
1.0/36.0/63.0
Cora-CA and DBLP-CA are obtained from Yadati et al. (019)) targeting paper classification, in which each paper is regarded as vertex of the hypergraph, the hyperedge represents the co-author relationship of the paper, and each vertex's 's feature is the bag-of-words representation of the related paper. The News20 from Asuncion & Newman (2077) with the goal being to classify each news, in which each vertex represents news message, the hyperedge represents the occurrence of 100 popular words among news, and the vertex features are from the TF-IDF representations of the news content. The IMDB4K-CA and IMDB4k-CD datasets have been developed by Fu et al. 22020) whose aim is to categorize each film into action, comedy, and drama. Both datasets consider movies vertices with vertex features bag-of- words representations of movie narrative keywords, and hyperedges are conducted based on common actors and common directors, respectively. DBLP4K- CC and DBLP4K-CP are academic networks classify authors into 4 areas, namely databases. data mining, machine learning, and information retrieval (Sun et al., (20TT) The datasets treat each vertex as an author, and hyperedges are provided based on co-conferences attended and co-publications among authors in two datasets, respectively. The Cooking dataset consists of vertices representing dishes, with hyperedges indicating dishes that use the same ingredients. Each dish also associated with categorical information indicating its cuisine type, such French, Japanese. This dataset poses a unique challenge as it lacks initial vertex features. The NTU dataset (Chen et al, 2003) includes 3D shapes categorized into various classes like chairs, doors, etc. The vertex features are extracted using Multi View Convolutional Neural Networks (MVCNN) (Su et al.) 2015) and Group- View Convolutional Neural Networks (GVCNN) (Feng et al. 2018) for 3D shapes. Since the NTU dataset does not come with an initial hypergraph structure, we constructed it by treating each 3D shape as a vertex and using k-nearest neighbors method to build hyperedges based MVCNN features and
15

---

Published as a conference paper at ICLR 2024
GVCNN features, respectively, thereby establishing the hypergraph structure. The detailed statistics of 9 benchmarks are in ST
F EXPERIMENT SETTINGS AND DETAILS
F.1 TRANSDUCTIVE SETTING
The transductive setting one of the most common experimental settings in graph/hypergraph vertex classification. In the transductive setting, the graph is fixed at training and testing. We know the features of every node in the graph, but we only know the labels portion of the vertices (i.e., the vertices in the training set, and the validation set). The goal is to predict the labels of the remaining unlabeled vertices in the hypergraph.
In experiment, 1, 500 vertices considered vertices with known labels for each dataset, which represent the union of the training and validation sets. We select 10 vertices for each category the training set among the known labeled vertices. The test set composed of vertices not included in the training set or validation set. Table S2 provides the statistics of datasets in the transductive setting.
F.2 PRODUCTION SETTING
Table S3: The train/val/test statistics of datasets the production setting.
Dataset
#Vertices
#Hyperedges
#Train/ (Traain/al/Trras/Inn Vertices
Split Ratio (%)
Cora-Cl CA
708
1, 072
70/1, 430/966/242
2.6/52.8/35.788 7/8.9
DBLP-CA
41,
22, 22,363 363
60/1, 440/31, 842/7, 960
0.1/3.5/77.1/19.3
News20
16, 342
100
40/1, 872/2, 968
0.3/8.9/72.6/18/
IMDB4K-CA
4,278
5, 257
30/1, 470/2, 222/556
0.7/34.4/51, 19/13.0
IMDB4K-CD
4,278
2, 2,081
30/1, 222/556
0.7/34 0/1/344/5119/13.1
DBLP4K-CC 4,057
20
40/1, 460/2, 046/511
..0/36.0/50.44
DBLP4K-CP 4,057
14,328
40/1, 046/511
1.0/36.0/50.4/13
Despite being a common research setting for vertex classification, the transductive setting excludes predictions for hidden vertices. As a result, we also take into account how well HDSode is actual production settings, including transductive and inductive forecasts.
To evaluate the model inductively, we retain a few test vertices from training to form an inductive set, and the remaining test vertices treated as the observed transductive set (i.e., Vy Virans LJ Vind where Vv, Virans: Vind represent unlabeled test vertex set, transductive vertex set, and inductive vertex set, respectively). Since the model is retrained periodically in production, Vind is regarded as the fresh unobserved set of vertices that enter the hypergraph between two pieces of training. For hypergraph, we need remove the inductive vertex and its associated hyperedges obtain the observable hypergraph Gou. = (Votes Eobs).
Here, we formally present training and testing methods in production settings. For the vertices. features of the whole hypergraph dataset, we can divide it into V = V2 LJ Virans LJ Vind- X. XL Xtrans Xinds where the observable vertices Vode V2 Verane. the observable features Xohs
We use yr to represent the labels of the vertices of the training and validation sets. obtain transductive results, we use hypergraph Gotss feature X Xohss and labels yr. to train the model, and then use the hypergraph Gobe. and feature Xobe to predict the labels of the vertex set Virans. To obtain indutive results, we use hypergraph Gobe. feature Xober and labels yr to train the model, and then the hypergraph G, and feature X, predict the labels of the vertex set Vind- To obtain production results, we use hypergraph Gobes feature X and labels yr to train the model, and then the hypergraph G, and feature X, to predict the labels of the vertex set Virans L Vind-
In a manner akin to the transductive setting, 1, 500 vertices in the production setting embody the combined training and validation sets, with each vertex treated having known label within
16

---

Published as a conference paper at ICLR 2024
datasets. Vertices outside the scope of the training and validation sets are designated the test set, wherein 20% serves as targets for inductive predictions. This arrangement derives from the observa- tion that, average, fewer vertices in inductive examination compared to transductive examination in production setting. Table S3 provides the statistics of datasets the production setting.
F.3 HYPERGRAPH CLIQUE EXPANSION TO GRAPH
Given that the graph-base methods in our comparison methods are unable to execute directly on hypergraph datasets, describe method here that expands cliques to transform hypergraph G (V, E) into graph Geg (Veg. Eeg). The goal of a clique expansion is to transform the hyperedges in hypergraph into graph edges. Clique expansion will add edges any two of the hyperedge's vertices for each hyperedge. Formally, the vertex set and edge set after clique expansion can be defined
Vey = V and Eog [(vi, 3e E (ui Sej.
(21)
F.4 TRAINING DETAILS
Table S4: The hyperparameters of HDSode
Dataset
Cora-CA
DBLP-CA
News20
IMDB4K-CA
IMDB4k-CD
DBLP4K-CC
DBLP4K-CP
O₂
0.05
0.05
0.1
0.05
Oe
0.9
0.9
0.9
learning rate
10-2
10-2
10-2
10-2
10-2
10-2
10-2
weight decay
x 10
10-4
10
10
5 10
10
5 10
dropout
0.15
0.15
0.15
0.1
0.15
0.15
0.15
100
(Dn
(Dff HDg
Accuracy(%)
40
Cora-CA
DBLP-CA
News20
IMDB4K-CA
IMDB4k-CD
DBLP4K-CC
DBLP4K-CP
Datasets
Figure S1: Comparison of the contribution of control steps and diffusion steps across all datasets.
For graph neural networks and hypergraph neural networks, the number of network layers set 2 to prevent over-smoothing, and the dimension of hidden layers is 64. In GDE, we up the same network structure as in the paper with hidden layer dimension 64. In the experiment of the GraphCON, the graph convolution operator is selected number of layers is 16, the hyperparameter search range in ODE is in {0, 0.05, .,,,)..,,................ In oDS the control term time interval set to 20, the termination time 7 is to 40, and the search range for the hyperparameters ae, ag in ODE is set to (0.05, 0.1..... 0.95). For datasets lacking hyperedge features, the initial value the hyperedge feature the aggregation vertices (i.e., Z. D
an autonomous coupling function, the
features from its internal Z.). The random seed is the same for all experiments of different methods and is set to 2, 022. All models are trained for 200 epochs using Adam optimizer with learning rate in {10 10 10-3,, weight decay in {5 10 1 10 5 10 10-5), and dropout in (0.05, 0. 0.1,...0.95). The loss function is cross-entres cross-entropy loss. We randomly partition the dataset five times, select hyperparameters based the average accuracy under the validation and report
17

---

Published as a conference paper at ICLR 2024
the average test accuracy and standard deviation under the hyperparameters. Table provides the hyperparameter details of HDSode in the transductive setting.
G ADDITIONAL ABLATION STUDY RESULTS
G.1 ABLATION STUDY ON CONTROL STEP AND DIFFUSION STEP
We further analyze the contribution of the control step and the diffusion step in 7 benchmarks, with results shown in Figure S1 In all datasets, both steps are essential. HDSode (Con.) results are worst since any hypergraph structure not exploited. In IMDB4K-CA and IMDB4K- along with DBLP4k-CC and DBLP4k-CP, can be noticed that HDS (Con.) achieves the same effect since these two groups of datasets share the same vertex features, respectively.
G.2 ABLATION STUDY ON THE NUMBER OF VERTICES IN THE TRAINING SET
Table S5: Test accuracy (%) and standard deviation of semi-supervised vertex classification on transductive setting with training vertex each class.
Model
Cora-CA
DBLP-CA
News20
IMDB4K-CA
IMDB4K-CD
DBLP4K-CC
DBLP4K-CP
GCN
25. 25.1417.72
46.2416.78
40.85-11.39
36.65_2.11
36.3011.r
83.46
36.6519.90
GraphSAGE
25.21
16.92=10.31
42.44=12.20
36.23_o.83
36. 10.1.1111
77.55
37.13
HIGNN
27.35
47 68.44
46 16.63
37 37.28-1.97
37. 37.12+1.78
03.32.0..6
42.15
HGNN
34.9813.03
52.731..90
49.50
34.70
93.37
41.92
UniGCN
27.94
50.91 26.86
47.08
37.02_1.70
36.83
92.73
39.83
UniSAGE
38 38.80=2.04
53 53.33=3.14
49. 49.23.29.24
37.05_1168
36. 36.93
92.64 02.64
42.39+7.18
oD
30.58+3.68
60 00.22=2.17
52.70
37. 37.12=2..2
37. 37.38+1.29
93. 43.5
43.46.7.51
Table S6: Test accuracy (%) and standard deviation of semi-supervised vertex classification on transductive setting with training vertex each class.
Model
Cora-CA
DBLP-CA
News20
IMDB4K-CA
IMDB4K-CD
DBLP4K-CC
DBLP44-CP
GCN
34.61+444
73.65
62.81+2.45
39.1442.97
38.54=2.44
89.46-1.17
56 56.96.25.
GraphSGGE
53.11
73.58
62.664a.7a
39.72
38.51
91.08=100
55 55.95
HGNN
60.52+4.12
75.55+4427
75.05+1.67
40.63+1.76
30 30.42=2.58
93. 93.35
58 58.07=6.00
HGNN
58.17+5.10
77.34+2
75.43
40.92
39.4 39.41
93.03-21.44
60.83
UniGCN
38.3645.11
74.544-1.5
74.9841.10
41.51+2.50
39.94=2.31
93.04g:128
59.91244.2
UniSAGE
62.564a.27
77.9422
73.90
#2.50
39.56=2.83
93.28
59.74+a.22
HDSode
64.73+2.27
78,89+2
75.42+2.00
42.19+1.00
41.10=2.88
03.41=0.55
61.62+1.83
Table and Table show the result of semi-supervised vertex classification a transductive set- ting with and 5 training vertex for each class, respectively. Under conditions with limited effective supervisory information, information cannot be sufficiently propugated through the network, result- ing in limitations of general hypergraph neural networks. However, our ODE-based model, through the use of more layers, allows for the more extensive propagation of this limited supervisory infor- mation. This more comprehensive message passing enables our model to overcome the performance bottlenecks with the propagation capabilities of general hypergraph neural networks (with an aver- age 1.73 and 0.82 accuracy enhancement in 1 and 5 training vertex in each class, respectively). In our experiments, we have explored scenarios with significantly fewer training samples each class, specifically, datasets with only 5 and even training vertex in each class. These few-shot conditions present a more challenging environment for learning accurate vertex representations, as the available information considerably limited.
18

---

Published as a conference paper at ICLR 2024
G.3 ABLATION STUDY ON CHALLENGING DATASET
Table S7: Results on Cooking and NTU datasets.
Model
Cooking
NTU
#Train Vertices
1
GCN
26.90
32.87+3.99
61.48+4.07
79.6441.85
GraphSAGE
26.68
32.73+5.11
60.83+6.53
16.78.1.13
HGNN
28.83+7.05
45.35
67.3842.07
85.3511162
HGNN
32.94
47.87
67.34
85.27
UniGCN
28.25
45.98
66.60
84.10.10.86
UniSAGE
35.2615.00
47.26
67-9612.96
84.261
HDS
36.15+4.18
49.37 73.7+3.00 38
+3.337
71.36+1.36
86.64+1.28
We further show the performance of our model in diverse scenarios. Specifically, we have included tests on datasets with unique characteristics, without initial vertex features (Cooking dataset), and without an initial hypergraph structure (NTU dataset). The Results are shown in Table [S7 In our experimental evaluation, we observe significant performance improvements in both the Cooking and NTU datasets when using our model. Specifically, when training with only samples per class, our model achieves an enhancement of 3. 3.13% on the Cooking dataset and 1.51% on the NTU dataset. With only sample per class, the performance gains are even more pronounced, 2.52% on the Cook- ing dataset and 5.00% on the NTU dataset. This indicates notable enhancement in model's ability to capture and propagate limited supervisory information effectively across the hypergraph structure,
in scenarios with minimal training data. Moreover, in more challenging few-shot scenario, these results highlight our model's proficiency in leveraging the overall hypergraph struc- ture extract and utilize latent relational information, especially when dealing with limited known vertex labels. In the case of the NTU dataset, where we construct the hypergraph structure from 3D shape features using MVCNN and GVCNN, the performance improvement emphasizes our model's capacity to exploit complex relational patterns from visually extracted features. Overall, these re- sults not only validate the effectiveness of our dynamical system- based hypergraph neural network in diverse settings but also illustrate its potential to address challenges in hypergraph learning tasks where general hypergraph models may be limited.
G.4 ABLATION STUDY ON THE TELEPORT PROBABILITY
-0
109
28
0.5
07
06
C5
0. 06
23
(a) aa of DBLP4K-CP.
(b) Qa of DBLP4K-CP.
(c) az of IMDB4K-CA.
(d) a. of IMDB4K-CA.
Figure S2: Additional performance comparison with teleport rates on DBLP4k-CP and IMDB4k-
CA datasets.
Figure $2 shows the result of teleport probabilities a and a₂ of HDS fall inside a certain range of ar E 10.05. 0.5] and E [0.5, 0.95) in DBLP4K-CP and IMDB4K-CA, respectively. It should be highlighted that oDS outperforms the second-best result across a wide range of teleport prob- abilities. We also notice that the teleport probability from hyperedge to vertex a, is optimal when small, and conversely, the teleport probability from the vertex to the hyperedge ae achieves the best results when large. This is because the initial hyperedge features missing in DBLP4k-CP and IMDB4k-CA and are calculated using the corresponding vertex features, which is slightly incor- rect. To get the precise representations, it is evident to lower the transfer probability from inaccurate
19

---

Published as a conference paper at ICLR 2024
hyperedge representation to vertices while increasing the transfer probability from accurate vertex representation to hyperedges. In general, as the hyperparameters increase, performance rises and subsequently drops. When using HDS fresh dataset, we can alter a, to about 0.1 and a, to approximately 0.9 by summarizing the results of IMDB4K-CA and DBLP4K-CP.
G.5 ABLATION STUDY ON THE INDUCTIVE RATIO OF PRODUCTION SETTING
Table S8: The ablation study on inductive vertex ratio of production setting. We report test accuracy (%) and standard deviation semi-supervised vertex classification on production setting with in- ductive and transductive predictions. "prod.", ",, "ind.", and "trans." denote "production", "inductive", "transductive", respectively. The best results are shown in bold.
Inductive venex rabi
0.7
pcod.
67.31
96.72.3.0s
06.6
G7 71
06.90as.19
67:21
67.71:2.9
68.21
67 27.226
HGNN
ind.
65.89
06.47g2.5
66 70.79922
66 66665
66 36.68
66.98
67.64-3.37
68.0923.35
67. 37. 07.37=2244
trans.
27.50
30
00 34.14
06.70a1
06.7241
66.36
66.00
66.94.2.3
64.9022241
pood.
g..5
67 as
67 an
67 pa
67 27
06.60.a.6
07.28
06. 2..4
06. 08.93
HGNNN
ind.
67. 44.0s
05
66 aa
65.90
07.21
06.52.4.40
trans
07.64gx44
67. 67.20g
66.54g9 0n
66 26.29
66. 12g5 16
66 06.86
66.39 7.
65.05ga.0 1
65.62.2
pcod.
70.49 4a
69.17,9 sd
70.46_2.4
20.09_202 69.33_1a7
G9 14
GB 6:3
G8. 49
HDS
1nd.
72. 66:40..
71.28:5.04
11.82:115
69.85-151
70.19:194
70 70.52:3.00
70 20.05:1.23
68. 82:2..1
08.68 18:77
trans
69. 80
68. at
69. 69.26_xx29
68 68.52-2.
68 68.67
67. 67.19
66. 23.99
65. 20
67.10.a.4
In order to verify the impact of different inductive vertex ratios on the results under production settings, conduct ablation study of it on dataset Cora-CA with HGNN, HGNN+ and HDS Detailed results are shown in Table [S8] The table shows that, in various inductive vertex ratio settings, our HDS method outperforms both HGNN and HGNN among production, inductive, and transductive results. Additionally, as the ratio rises, the number of the observation graph vertices |Vonsl gradually decreases, causing the three results to essentially exhibit downward trend.
G.6 ABLATION STUDY ON HYPERGRAPH OPERATORS AND GRAPH OPERATORS
Table S9: Test accuracy (%) and standard deviation of semi-supervised vertex classification of the ablation study on hypergraph operators and graph operators. "OOM" and "HDSode (graph op.)" represent "out of memory" and "HDSode using graph operators", respectively. The best results are shown in bold.
Model
Cora-CA
DBLP-CA
News20
IMDB4K-CA
IMDB4K-CD
DBLP4K-CC
DBLP4K-CP
GCN
65.99
82,22 00
67.57 20.70
43.4722.9
41.0222.12
90.18
64.47 20.000000
HGNN
67.5821.8
82.83 09
76.58
43.2129.9
41.08
93.4620.77
67.99
Hgrahe op.)
67.48
82.78
00M
43.81
41.53
90.67
66.75gu.m
#D
68.9 68.92
83.05
76. 76.75-1.0r
16
22.30=2.22
93.85.0055
69.52:1.19
To verify the necessity of using hypergraph operators, we present an ablation study on whether the model uses hypergraph operators or graph operators in different hypergraph datasets. The ablation study includes four methods, namely GCN, HGNN, HDSode using graph operators, and oD
According to the results in Table we found that in the hypergraph dataset, the hypergraph method achieves better results than the corresponding graph method (i.e., HGNN beats GCN, and HDS beats ode (graph op.), respectively). which indicates that it is necessary to use hypergraph oper- ators in hypergraph correlation structures.
As a generalization of graphs, hypergraphs have richer correlation expression capabilities than graphs. Such as in film background, hypergraphs adeptly capture the multifaceted connections among actors. Each actor can be represented as vertex, and hyperedge encompasses various correlations, such as actors co-starring in the same movie or originating from the same country. This hypergraph approach allows for comprehensive representation of connections beyond pairwise in- teractions. Traditional graph models fall short since edges link only two nodes. They struggle to directly represent these group- relationships without becoming complex or losing clarity. In
20

---

Published as a conference paper at ICLR 2024
the graph operator, each vertex interacts with its adjacent vertices based on the edge connecting two vertices, while the hypergraph operator generates pair-wise interaction or group interaction based on the hyperedge that can connect two or more vertices.
H DETAILED VISUALIZATION OF REPRESENTATION
(a) Att =tt=1. 1.
(b) At 1/8- T.
At 2/8-
(d) A T.
(c) Att 4/8-T. T.
(f) At 5/8- T.
(g) Att 6/8- T'
(b) Att 7/8-T.
Att=
Figure S3: More specific T-SNE visualization of vertex representation at different timestamp
Taking T/8 as the time interval, Figure S3 gives a more detailed representation evolution process in DBLP-CA. The dynamic variations in the vertex representation distribution more clearly depicted as the time period is shortened. When the timestamp / is larger, the representation is closer stability, the change in the same time interval less obvious. By reducing the visualization time interval, we get more accurate evolution rules of representations. Through the accurate dynamic evolution, we provide circumstantial evidence for the evolution process of representation from raw to final.
21


Reviews:
[{"review_id": "sx7LOBJ0SW", "reviewer": "Reviewer_3YqS", "summary": "This paper introduces hypergraph dynamic systems (HDS) to characerize the continuous dynamics of representations, and then proposes a control-diffusion HDS by an ODE. A multi-layer HDS-ODE is designed as a neural implementation, having the properties of controllability and stabilization, and   can capture long-range correlations among vertices. The paper performs evaluation experiments on 7 datsets to show its dominant performance.", "questions": "1. Why do you call these two terms in eq.(3) as the \"control\" term and the \"diffusion\" term. We are not familiar with the terminology in the \"small\" field (that is consisted of these ~5 papers in introduction). However, in the mature fields of \"dynamical systems\" in math, \"stochastic analysis\" in probability or mathematical finance  and \"control theory/engineering\" or \"cybernetics\" in engineering, neither the name of \"control\" nor \"diffusion\" may be  properly defined. The control term usually refers to external signals or extrogeneous variables (in econometrics) that can be designed or modified. Indeed, if you assume, eq.(3) is not an autonomous system in nature (if purely looking at (3) itself, it is), but the feedback system by using the nonlinear state feedback law g(X) as the control \"u\" for the linear dynamical system X_dot = A X + u. It somehow legitimate the name \"control\". However, it is still not recommended in this way since it is confusing. Referring to the diffusion term, we cannot see why it can be call as this name, since the diffusion term is h(X)dW in the SDE dX(t) = f(X, t) dt + h(X,t) dW, where W is the Brownian motion. The AX(t) term in eq.(3) is actually the term of f(X,t), so the drift term. \n2. See Weakness point 2 on Sec.5.1. Please explain, in particular, the first sentence of Sec.5.1, which seems not correct.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The pape present the implementation of HDS-ODE framework by posing the neural implementation of the control step and diffusion step in sequence  via Lie-Trotter splitting method.\n2. The time complexity and relation to HGNN+ are analyzed. And the experiments show its very good performance.\n3. The properties of HDS-ODE are discussed and we appreciate such an effort on analysis in theory, although we do have concerns on these contents (please see comments below).", "weaknesses": "1. To the terminology (and the preliminary math tools) of dynamical systems used in this paper, it seems that the authors does not soundly cook its article based on the strict math that has been widely accepted in math and engineering. See Queation 1, Weakness point 2 and 3.\n\n2. The discussion of statibility seems wrong. That's why we say the authors may not well pick up knowledge of (linear/nonlinear) dynamical systems. Sec. 5.1 discussed the stability of HDS-ODE, where the statement of the first sentence in this section is basically wrong. The so-called \"control\" term also iterate over time. It cannot be simplified the stability analysis of HDS-ODE as simply a dissusion of linear system X_dot = A X. Let us use the eq.(3) to clarify the point simply. Supposing we accept the split of the general state-space equation X_dot = f(X, t) as eq.(3), it is obviously that the stability analysis is a general stability analysis of nonlinear system, besides AX(t) there also exists g(X(t))! If you said your discussion of HDS-ODE refer to the neural implementation, we can see that in eq. (6) the nonlinear term is still there, except it is in NN form, and then embeds into eq.(7) to complete the iteration t+1. Furthermore, these propositions on stability for your \"simplified\" linear systems are well-known in the field of electrical engineering.\n\n3. Your abstract and contribution summary tell that yours studies the \"controllability\", which we could find anything related to it. And regarding the starting point of the most general eq.(2), there cannot be any controllablity-related problem can be formulated. Maybe you refer to something different? We strongly recommend the authors to learn essentials of dynamical systems, it helps to avoid conceptual misunderstandings and misuse for better communications.\n\n   To help you with essential knowledge on dynamical systems, you may refer to the following classic textbooks (basics on linear, nonlinear systems):\n\n   - Zhou, K., Doyle, J. C., & Glover, K. (1995). Robust and Optimal Control. Pearson.\n   - H. K. Khalil, “Nonlinear Systems,” 3rd Edition, Prentice Hall, Upper Saddle River, 2002.\n\nYour idea may be valuable and appreciated, considering your sound performance in experiments. However, you really have to first carefully deal with theory and fix any possible mistakes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8DtlAEWlsL", "reviewer": "Reviewer_4j9N", "summary": "In this paper, the authors target on the task of representation learning with high-order correlations on hypergraph, in which the challenge is the sub-optimal problem during the process the neural network. Existing hypergraph neural networks cannot be deeper than 2 layers and is unstable. This problem is a common but challenged issue in this field. The authors introduce the framework of hypergraph dynamic systems, which connects hypergraph learning and dynamic systems to achieve continuous dynamics of representation using high-order correlations. The authors further propose an implementation of hypergraph dynamic systems based on ordinary differential equation and experiments have shown stable and satisfied performance. This control-diffusion process introduced in this paper have been demonstrated effectiveness through the results and theoretical discussions.", "questions": "1. For the framework figure (Fig. 2), a more detailed and clearer introduction should be helpful.\n2. As shown in Fig. 1, the performance of HDS-ode increases during the first 8 layers and becomes stable then. Can the authors further explain how to control the diffusion speed during this procedure?\n3. There are a few typos. Please find and correct them.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This paper targets on an important but challenged task in representation learning, i.e., how to achieve stable representation learning in hypergraph neural network, which is also a common issue in the general graph neural networks. Usually, HGNNs cannot be more than 2 layers, which leads to performance degradation significantly. The introduced hypergraph dynamic systems framework in this paper bridges hypergraph learning and dynamic systems, which can take the advantages of hypergraph on high-order correlation modeling and dynamic systems on controllable diffusion process. The idea of hypergraph dynamic systems is novel. It is a good attempt towards better representation learning and could be helpful to a broad field.\n\nThe authors also propose an implementation of HDS using ODE, and a multi-layer HDS-ode is given. The stability analysis has also been detailed analyzed. The difference between HDS-ode and traditional HGNNs has discussed.\n\nExperiments are sufficient. Experiments on semi-supervised vertex classification with two different settings have been conducted on 7 datasets. Experimental results have clearly shown the superior performance of HDS-ode compared with recent state-of-the-art GNN/HGNN methods. From the results, we can observe the control-diffusion process of HDS-ode is stable, which solves the limitations of existing HGNNs, i.e. only 1 or 2 layers can be used.\n\nIn general, this paper is well organized and writing. The related works are sufficient and the motivation is clear. The method has been detailed introduced. This paper brings in a new aspect of representation learning of taking both high-order correlation modeling and dynamic systems into consideration simultaneously, which has the potential to have broad impact.", "weaknesses": "For the framework figure (Fig. 2), a more detailed and clearer introduction should be helpful.\n\nThere are a few typos. Please find and correct them.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Val6hT4y6H", "reviewer": "Reviewer_KkAS", "summary": "This paper theoretically introduces hypergraph dynamical systems based on a control-diffusion\nODE, which bridge hypergraphs and dynamical systems. It then proposes a neural implementation $HDS^{ode}$ and presents stability analysis of it and the connection to hypergraph neural networks. Finally, the paper empirically evaluates $HDS^{ode}$ using benchmark datasets and show its effectiveness to some extent. Some ablation studies are also included for more thorough investigation of the proposed method.", "questions": "\"Dynamic systems\" reads weird, and it is used kind of interchangeably with \"dynamical systems\" in the paper. Is there any specific reason to sometimes use \"dynamic systems\" instead of \"dynamical systems\" in the paper? If not, sticking with \"dynamical systems\" and \"hypergraph dynamical systems\" might be more appropriate. See discussion on Mathoverflow https://mathoverflow.net/questions/366856/why-is-a-dynamical-system-not-a-dynamic-system\" about why the right mathematical jargon is \"dynamical systems\".", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1.\tGiven the graph counterpart, it is a natural and interesting idea to develop a hypergraph neural ODE to improve the controllability and stabilization of information diffusion on hypergraphs. On the other hand, given that graphs and hypergraphs are different in nature, it is also a challenging problem how the system should be designed. \n2.\tThe paper is well-written and easy to follow.\n3.\tThe paper is very complete, with clear presentation of the method, some theoretical analysis and thorough empirical evaluation.", "weaknesses": "The main weakness of the paper is the weak empirical results supporting the effectiveness of the proposed method. It is unconvincing why small diffusion steps themselves constitute a real problem for hyper graph neural networks. Although $HDS^{ode}$’s performance does not suffer from more layers, $HDS^{ode}$ has very marginal improvement in terms of optimal performance. This is evident from all the experimental results (Figure 1, Table 1 and Table 2), where the improvement over baseline methods is barely noticeable and almost never statistically significant.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Xq8wUrrJtV", "reviewer": "Reviewer_HBiR", "summary": "This paper extends hypergraph neural networks (HGNNs) to model hypergraph dynamic systems. Specifically, the authors integrate the graph propagation scheme of HGNN into a control-diffusion ODE form to capture dynamics. Theoretical analysis highlights the controllability and stabilization properties of the proposed HDS$^{ode}$, which allows it to capture long-range correlations among vertices. Experimental results demonstrate the effectiveness of HDS$^{ode}$.", "questions": "1. What is the rationale for applying ODEs and continuous methods to static graphs? How does the model benefit from the system dynamics brought by its structure?\n2. Could you provide insights into the model's performance when the hypergraph convolution is replaced with simpler graph operators like GCN, as part of ablation studies? The functionality of hypergraph convolution in the model is unclear.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The combination of Neural ODEs with hypergraphs is an interesting idea, bringing together two distinct approaches to modeling dynamic systems.\n+ The authors introduce a Lie-Trotter splitting method as the ODE solver, which is a notable contribution.\n+ The theoretical analysis on stability and eigenvalue properties of hypergraphs is solid and persuasive.", "weaknesses": "- The proposed model appears to be a straightforward combination of existing efforts on HGNN and neural graph ODEs, essentially replacing the message-passing scheme of GNNs with a diffusive hypergraph adjacency. It would be beneficial to clarify how this combination advances the field beyond existing approaches.\n- The application of hypergraph ODEs is not sufficiently motivated. Although the proposed model claims to capture system dynamics, it is evaluated on node classification tasks using static graphs, leaving the potential benefits on dynamical systems unclear.\n- The experimental results show limited improvement over baseline methods, and from the comprehensive comparison it seems that variations in model structures have little influence on performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)