Hyperbolic-PDE GNN: Spectral Graph Neural Networks in the Perspective of A System of Hyperbolic Partial Differential Equations
Authors: Juwei Yue
Haikuo Li
Jiawei Sheng
Xiaodong Li
Taoyu Su
Tingwen Liu
Li Guo
Venue: ICML 2025 poster (2025)
Keywords: Graph Neural Networks
Hyperbolic Partial Differential Equations
Primary area: deep_learning->graph_neural_networks

Abstract: Graph neural networks (GNNs) leverage message passing mechanisms to learn the topological features of graph data. Traditional GNNs learns node features in a spatial domain unrelated to the topology, which can hardly ensure topological features. In this paper, we formulates message passing as a system of hyperbolic partial differential equations (hyperbolic PDEs), constituting a dynamical system that explicitly maps node representations into a particular solution space. This solution space is spanned by a set of eigenvectors describing the topological structure of graphs. Within this system, for any moment in time, a node features can be decomposed into a superposition of the basis of eigenvectors. This not only enhances the interpretability of message passing but also enables the explicit extraction of fundamental characteristics about the topological structure. Furthermore, by solving this system of hyperbolic partial differential equations, we establish a connection with spectral graph neural networks (spectral GNNs), serving as a message passing enhancement paradigm for spectral GNNs.We further introduce polynomials to approximate arbitrary filter functions. Extensive experiments demonstrate that the paradigm of hyperbolic PDEs not only exhibits strong flexibility but also significantly enhances the performance of various spectral GNNs across diverse graph tasks.

Hyperbolic-PDE GNN: Spectral Graph Neural Networks in the Perspective of A System of Hyperbolic Partial Differential Equations
Juwei Yue 12 Haikuo L
Jiawei Sheng 12 Xiaodong Li Taoyu Su 12 Tingwen Liu 12 Li Guo 12
Abstract
Table 1. Comparison between traditional based (Line 1) and hyperbolic PDE-based (Line 2) paradigm, where e is the unit vector, À and are the eigenvalue and eigenvector of Laplacian.
Graph neural networks (GNNs) leverage message passing mechanisms learn the topological fea- tures of graph data. Traditional GNNs learns node features in spatial domain unrelated to the topol- ogy, which can hardly ensure topological features. In this paper, we formulates message passing as a system of hyperbolic partial differential equations (hyperbolic PDEs), constituting a dynamical sys- tem that explicitly maps node representations into particular solution space. This solution space is spanned by set of eigenvectors describing the topological structure of graphs. Within this sys- tem, for any moment in time, a node features can decomposed into a superposition of the basis of cigenvectors. This not only enhances the in- terpretability of message passing but also enables the explicit extraction of fundamental character- istics about the topological structure. Further- more, by solving this system of hyperbolic par- tial differential equations, we establish a connec- tion with spectral graph neural networks (spectral GNNs), serving as message passing enhance- ment paradigm for spectral GNNs. We further in- troduce polynomials approximate arbitrary fil- ter functions. Extensive experiments demonstrate that the paradigm of hyperbolic PDEs not only exhibits strong flexibility but also significantly enhances the performance
Parndigm
Bases Solution Space
(1) hi 1. (n." (M,,
(2) Lx
T.
W:
1. Introduction
Currently, graph neural networks (GNNs) (Wu et al., 2021) have experienced rapid development due to the widespread usage of graph data. They have not only achieved significant advancements in artificial intelligence fields such as recom- mendation systems (Wu et al., 2023; Liao al., 2025), social networks (Lin et al., 2021: Zhu al., 2023), and anomaly detection (Ma et al., 2023; Gao et al., 2023), but have also promoted progress in various other domains such as particle physics (Shlomi et al., 2021) and drug discov- cry (Gaudelet al., 2021).
Mainstream methods are spectral GNNs represented by GCN (Kipf & Welling, 2017). The key idea of these meth- ods is to utilize message passing (MP) (Gilmer al., 2017) to explore the topological features of the graphs. Specifi- cally, MP aggregates information from neighbors based on the topology of graphs. The essence of MP involves map- ping node features into the spatial domain to the spectral domain through Fourier transformation, performing graph convolutions in the spectral domain, and then inversely trans- forming them back into the spatial domain. However, the issue lies in the fact that, even if node features learn the topology of the graph, they are still mapped back to a spatial domain unrelated to the topology. As shown in Table 1, traditional MP-based GNNs obtain node feature vectors that are actually situated the Euclidean space with unit vector bases, which can hardly ensure topological feature therein, potentially learning redundant information. (In contrast, our hyperbolic PDE-based model derives node feature vec- tors with bases stemmed from Laplacian, incorporating the underlying topological information. Detailed later.)
various spectral GNNs across diverse graph tasks. Our code is re- leased https: / //g/ttttbbbb.. com/ YueAWu/ Hyperbol CNN..
'Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China "School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China. Correspondence to: Jiawei Sheng esshengiawei@iic.... @ii..ac.on>.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Current researches (Rusch et al., 2022; Chamberlain et al., 2021; Li al., 2024) model MP using differential equa-

---

Hyperbolic-PDE GNN
tions (DEs), formulating MP dynamical system that captures the evolution of messages over time. It provides interpretable way to embed node feature vectors into spaces with particular properties. We utilize the solution of DEs as the node embedding space, and propose a graph neural network named Hyperbolic-PDDS GNN based on a system of hyperbolic partial differential equations (PDEs). We the- oretically demonstrate that by modeling MP as the paradigm of the system of hyperbolic PDEs, the solution space (i.e., the space where node features reside) is spanned by a set of orthogonal bases describing topological features, with these bases fundamentally stemming from the eigenvectors of the Laplacian matrix. This proof of the solution space is derived by converting the system of hyperbolic PDEs into system of first-order constant-coefficient homogeneous linear differential equations. Therefore, the node features obtained from this paradigm can be expanded into a linear combination of these eigenvector basis, where node features inherently contain the fundamental characteristies describ- ing the topological structure, and messages are propagated along the directions of the cigenvectors. Benefiting from the property, our Hyperbolic-PDBB GNN enjoys an advantage in generating better node features with topology of graphs.
hyperbolic partial differential equations can enhance the performance of traditional GNNs.
2. Preliminaries
2.1. Notations of Graph
A simple undirected graph 9 = (V.E) consists of a set of nodes and a set of edges E. A feature matrix X E Rnxd and a adjacency matrix A represent respectively the features and structure of the graph, where n [V] is the number of nodes and d is the dimension of features. In the graph signal processing (GSP) (Shuman et al., 2013), the sym- metric normalized Laplacian is L = I - 1-1 AD -1/2, where D denotes the diagonal degree matrix of A. The Laplacian can be decomposed into L = UAUT, where A = diag {À1, A2,.... An) a diagonal matrix of eigen- values and U [n, .2...., un] E 1axx the matrix of corresponding eigenvectors.
2.2. Spectral Graph Neural Networks
Spectral Graph Neural Networks achieve Spectral Graph Convolution
Lastly, we discover that solving this system of equations requires eigen decomposition, which incurs significant com- putational costs in practice. Besides, the original system of equations with vanilla Laplacian struggles to model complex nonlinear relationships, resulting in unsatisfactory model flexibility and robustness. Therefore, we further introduce polynomials to approximate the solution space. Some GNNs based on polynomial spectral filters (He et al., 2021; Wang & Zhang, 2022; He et al., 2022) have demonstrated the advan- tage of polynomials in approximating arbitrary filter func- tions. Leveraging this excellent property, we can ensure that the approximate solution space does not deviate too far from the original solution space and can describe more complex nonlinear relationships, greatly enhancing the flexibility of model. Furthermore, owing the utilization of polyno- mials, we establish connection between JyyeebboiccPPDIt GNNs and traditional spectral GNNs which can further enhance the performance of these methods. In summary, the contributions can be summarized as follows:
Z =
by designing Graph Spectral Filter g(A), where g(A) = (Ai)........... g),))]. To fit an ideal filter, g(A) is often approximated using a K-th order polynomial:
0(A)
Σ 0
(2)
R=
where On is the filter coefficient. Then, the Spectral Graph Convolution is given by:
K Z 0 QUAUX D
K 0LXX
(3)
Table 2 presents some classic examples of spectral GNNs.
2.3. Hyperbolic Partial Differential Equation
A hyperbolic partial differential equation is often used in the real world to describe phenomena involving vibrations: if point in space is disturbed by an initial value, this dis- turbance will propagate through space at a finite speed, affecting certain points in space. The propagation pattern of the disturbance is wave-like and propagates along the characteristic directions of the equation. The most common form of hyperbolic partial differential equations is given by:
We propose novel Hyperbolic-PDE GNN based on system of hyperbolic PDEs, and theoretically demon- strate that node features encapsulate fundamental char- acteristics of the topological structure.
By utilizing polynomials to the solution space, we establish connections between Hyperbolic-PDE GNNs and traditional GNNs to further improve performance.
Extensive experiments indicate that Hyperbolic-PDD GNN is effective and the paradigm of the system of
pºu
2
d
82n u
(4)
0t2
Ow?
1=1
2

---

Hyperbolic-PDE GNN
Table 2. Spectral Graph Neural Networks.
Methods (Polynomial Basis)
Graph Spectral Filter
Spectral Graph Convolution
GCN (Monomial)
s(A) (1 X). de (0.2)
Z (I- LJXW
SGC (Monomial)
9(A) = (1 -)) X (0,
2 = (I --)
APPNP (Monomial)
9(A) = 2R=0 I-a (1-11)1)) X) (0,2)
at
Z a (1- a] -))
GPR-GNN (Monomial)
0(A) = > -K-9 (1 [ x
2 = Σ 0x 0 -x(ITLL)56()
k
ChebNet (Chebyshev Polynomial)
9(A) I
IR-D OcTa(A - A [,
Z Σ Lg-D Ta(L- -I)XW,
BernNet (Bernstein Polynomial)
9(A) 2n-0 2x (A/2), [0.2]
Z ΣR 2t (x) L) "
JacobiNet (Jacobi Polynomial)
((A) I
2N-0 (1 -)), A [0.2]
Z, L Oag pa.0 L))
ChebNetlI (Chebyshev Polynomial)
9(A) CK- - 1), A [0,2]
Z K+I Ek-o Ee (2 I))(X)
where u(w1, w. wa, 1) represents the amplitude, / E [0, oo) denotes time, (Wi, W2,,,, .Wa) stands for the coor- dinates in d-dimensional space Q2 and a signifies the propagation velocity.
When we combine the hyperbolic partial differential equa- tions of all nodes on all dimensions, we obtain system of hyperbolic partial differential equations. For simplicity, we can rewrite it in matrix form:
02X
3. Methodology
L'L
(9)
3.1. A System of Hyperbolic PDEs on Graph
To distinguish from matrix L, hereafter will denote the eigenvalues and eigenvectors of matrix L as X and û. De- tailed derivation of the system of partial differential equa- tions can be found in Appendix A.I.
Considering the right-hand side of Equation (4), it is essen- tially expressed by a Laplacian operator A = V. V, which can then written in the form of the divergence of the gradient of u:
Su 012
3.2. Derivation of Solution Space
2 Vu,
For the system of partial differential equations (9), re- quire the existence of solutions for this system to ensure the meaningful extraction of node features. Next, we will introduce the theorem regarding the existence of solutions for this system of partial differential equations.
where V and V. denote the operator of gradient and diver- gence, respectively. Here, the gradient and divergence are defined on a manifold. In graphs, nodes are set of dis- crete points, thus we first discretize the amplitude u on the manifold into features X(t) in the spatial domain, where each element væ (1) denotes the feature of each node V each dimensions = 1, d at time t. Next. we express the gradient on the graph as the feature difference on any dimension / between a node and its neighbors vy E (wx)):
Theorem 3.1. For given system of differential equations (9) and a graph signal on any dimension 1, there exists a solu- tion space determined by a ffundamental matrix of solution:
Texple It
0 a exp Lt
C (t)
(10)
Vzu = Fzt IJl.
(6)
with the direction pointing from the neighbor towards the node, indicating the direction of information flow. Finally, define the divergence the graph as the total feature difference any dimension between node and all its neighbors:
Therefore, graph signal any dimension can be ex- pressed as linear combination 2n linearly independent column vectors (t), (t. .2n. ....)..............
2n
from the funda-
mental matrix of solution (t).
By performing a variable substitution on Equation (9), we transform it into a system of first-order constant- coofffiin homogeneous linear differential equations, and successfully complete the proof using relevant theory. See Appendix A.2 for proof.
VE
Σ
(za-11z))
()E)((e))
since divergence essentially involves a summation operation. Therefore, for node i; any dimension 1, its hyperbolic partial differential equation is:
Remark 3.2. Theorem 3.1 ensures the existence of a solution when modeling message passing as a system of hyperbolic partial differential equation. Here, the solution corresponds to the node features. In other words, we can definitely obtain precise node representation through Equation (9).
0. 012
a²
Σ (vE)(()))
(wa 2T2).
3

---

Hyperbolic-PDE GNN
Although we obtain a fundamental matrix of solution of Equation (9), in order to determine it, we need to calcu- late exp Lt. However, exp Lt is defined by matrix series, it is generally difficult to obtain directly. Therefore, we introduce the following theorem to indirectly find the funda- mental matrix of solution.
matrix of solution of Equation (13) still requires the compu- tation of the eigenvalues and eigenvectors of the Laplacian matrix. Its computational complexity is O(n²), which can significantly impact the efficiency of the model when deal- ing with large-scale graphs. Therefore, in this section, we introduce polynomial approximation theory (Stone, 1937) to address the aforementioned issue.
Theorem 3.3. For given system of differential equations (9) and a graph signal on any dimension I, matrix
Theorem 3.5. For a function f(a) that is continuous on [a, for any E O, there always exists an algebraic poly- nomial P(x)) such that
C=
(11)
0
a
[I(()) - P(z)ll> <E
(14)
= 1, 1, = a² a1. Xn+2 a² .i.....,., adxn and corresponding 2n linearly independent eigenvectors
has eigenvalues X/ = N/2 =
X
holds uniformly on [a, b].
The Bernstein polynomials have been demonstrated to hold uniformly on [0, 1] and have been practically applied in He et al. (2021). Orthogonal polynomials are crucial tools for function approximation, where any polynomial can be expressed linear combination of orthogonal polyno- mials. Therefore, we approximate the right-hand side of Equation (9) using polynomial with respect to Laplacian:
u1 =
V1
112 =
V2
Vn 0
0
0
(12)
0
0
0 ün
u'++1 =
u2
ū1
=
01
where Vi 11.0.....0 O]T, V2 = [0, 10.
O]T, ,...,,,, Vn
[0, O,......... o] E IR" are corresponding eigenvectors of I, then matrix
0- = = P(L,)X ōt2
K
OL.Pe())X, (15)
R=0
(()) = [c c u'. Aon unnl
(13)
where pa(-)) is the k-th order orthogonal polynomial. Here, we replace the basic Laplacian matrix L with the more commonly used symmetric normalized Laplacian matrix L, which differs only in the nonzero elements.
is a fundamental matrix of solution of Equation (9). Partic- ularly, exp Ct = -
See Appendix A.3 for proof.
As introduced in Section Preliminaries, it is due to the achievements of pioneers in the field of graphs who intro- duced Laplacian polynomials into graph data that we have laid the foundation for establishing the connection between the system of hyperbolic partial differential equations and spectral GNNs. As shown in Table 2, Chebyshev polynomi- als serve as an example of utilizing orthogonal polynomials to approximate graph spectral filter. The distinction lies in the fact that in this paper, we employ polynomial to ap- proximate the solution space of the system of hyperbolic partial differential equations. Then, based on Chebyshev polynomials, Equation (15) can be rewritten as:
Remark 3.4. Theorem 3.3 reveals that the solutions of Equa- tion (9) lie within a particular space, which is spanned by a basis of eigenvectors deseribing the topological structure of the graph. In other words, in the modeling of a sys- tem of hyperbolic partial differential equations, the node features propagate messages along specific directions of cigenvectors. The theorem not only enhances the inter- pretability of the paradigm of system of hyperbolic partial differential equation but also ensures that the node features imply the fundamental characteristics of the topological structure of the graph.
3.3. Polynomial Approximation of Solution
0x
K-1 Σ k=
- -
(16)
Note that Theorem 3.1 and Theorem 3.3 explicitly expound that for any graph signal in Equation (9), there exists a so- lution space determined by the topological structure of the graph and the propagation coefficient a. Among these, L is constant matrix, thus the only parameter that can be uniquely determined by the model is the propagation coeffi cient a. If we solely rely on a to regulate message passing, it is not only hard to depict intricate nonlinear relationships but also inevitably leads to models with significantly re- duced flexibility and robustness. Moreover, the fundamental
012
3.4. Hyperbolic-PDE GNNs
For given PDE, solving an analytical solution is often challenging, numerical methods are commonly employed to approximate its solution in mathematics. This technique is currently extensively utilized within the realm of graph studies, with the forward Euler method standing out as the predominant choice (Eliasof et al., 2021; Chamberlain
4

---

Hyperbolic-PDE GNN
Table 3. Dataset statisties.
Table 4. The results of spectral GNNs: mean accuracy (%) stan- dard deviation 60%/20%/20% data splits on 10 runs. Bold and underline indicate optimal and suboptimal results.
Datesets
#Nodes
#Edges
#Features
#Classes
Cora
2708
5278
1433
7
CiteSeer
3327
4552
3703
6
Metbods
Cora
CiteSeer
PubMed
Actor
PubMed
19717
44324
500
GCN
87.1 87.14
79.86
86. 20.744
83.23 21.30
Computers
13752
245861
767
10
APPNP
88. 28.14
80.47
88. 12..31
39.66 29.66
Photo
7650
119081
745
8
ARMA
87.13 87.13
20.04
86, 86,93
37.67 20.54
CS
18333
81894
6805
15
GPR GNN
88.57
80.1220.53
88. 08.44
39.9220.07
ChebNet
86.67 20.87
79.11 10
87, 07.90go.0s
37. 1.
Actor
7600
30019
932
5
BerNNet
88. 28.995....5
80.00 20
88, 88.48
41.79
Texas
183
325
1703
JacobiConv
0.
80.78 20.79
89, g.04..
11.17
Cornell 183
298
1703
ChebNellI
88. 28.71
80.53
88, 2..22
41.75=11.7
DeezerEurope
28281
92752
128
EvenNet
87.77=0.07
78.51 +0.63
90.87 4:0.4
40.36
OptBasisGNNN
87. 87.00g1.65
80,58
90.30
42.39
PyGNN
88.9440.a1
79.49
80.52+00.24
SpecFormer
88. 18.5721.01
81,49
87. 23.55
41,903
et al., 2021; Thorpe et al., 2022; Li et al., 2024; Yue et al., 2025). Specifically, the forward Euler method discretizes continuous time, dividing time into m equidistant segments with time step T: Im = mt, m = 0, ..... The initial and terminal time are denoted as tg and represents the m-th time moment in time.
NEGNN
89.822n.a2
80.56
89.8920.0s
40.62
UniFilter
39.19
81.39
91. 14.0.0
10.81g1.21
Our methods
90. 90.82.0.as
81.88
91.36g0.58
42.03
ty, and tm
network, Computers and Photo (Yang et al., 2016), and co- author network CS (Shchur et al., 2018); b) heterophilic datasets include co-occurrence graph Actor, two WebKB datasets, Texas and Cornell (Pei et al., 2020), and social network DeezerEurope (Rozemberezki Sarkar, 2020). Note that do not utilize the Chameleon and Squirrel datasets (Rozemberczki et al., 2019) because they have se- rious issues of node duplication (Platonov et al., 2023). Table shows the dataset statistics in our paper.
After discretizing time, Equation (15) can be reformulated into the following difference scheme:
X(tm) X(1m-1) P(L, tm (tm),
(17)
T
where (tm) and (1m-11) are the first derivative w.r.t time: X(t,.) X((m+1)-XX -X((m) (tmm-1)
(0m)-X((m-1)
Furthermore, to solve Equation (17), we initialize the initial values of node features and its first derivatives as:
We choose classical spectral GNNs as baselines: GCN (Kipf & Welling, 2017), SGC (Wu al., 2019), APPNP (Klicpera et al., 2019a), ARMA (Bianchi et al., 2021), GPR- GNN (Chien et al., 2021), ChebNet (Defferrard et al., 2016), BernNet (He et al., 2021), EvenNet (Lei et al., 2022), Jaco- biConv (Wang & Zhang, 2022), ChebNetl (He et 2022), OptBasisGNN (Guo & Wei, 2023), PyGNN (Geng et al., 2023), SpecFormer (Bo et al., 2023), NEGNN (Zheng et al., 2024), and UniFilter (Huang et al., 2024b).
(14))=o000 X (lo) ((t)) X(t))_((t_) -6 (X), (18)
where (-)-) and Φ1 (-) can be obtained from nonlinear layers. Substituting Equation (18) into Equation (17), we obtain the initial value 1st time step as:
72 2 2 P(I, X(to)- X(t))= = - (I+
(19)
We split the datasets into training/validatiioneess sets in 20%/20%/ /20% and report the mean accuracy and stan- dard deviation of the models over 10 random splits. For our method, we utilize cross-entropy as the loss function and optimize parameters using the Adam optimizer (Kingma & Ba, 2015). We conduct experiments on a NVIDIA Tesla V100 GPU with 16GB memory and Intel Xeon E5-2660 v4 CPUs. The experimental code is implemented based on Py- Torch and PyTorch Geometric (Fey & Lenssen, 2019), and we employ wandb for parameter search. See Appendix C for specific experimental settings.
Given X(1o) and X(1)), we can derive the updating iterative formula for the node features at m-th time step:
X(tm+1) (2I + P(L,tt)) X(tmm) X(tm-1). (20)
4. Experiments
4.1. Node Classification on Graphs
We evaluate the performance of Hyperbolic-PDD GNN the classic node classification task (Shchur et al., 2018). Fol- lowing the practice of Wang & Zhang (2022), we conduct experiments on following real-world datasets: a) homophilic datasets include three citation networks, Cora, CiteSeer, and PubMed (Shchur et al., 2018), two Amazon co-purchase
Table 4 illustrates the performance of Hyperbolic- GNN on four prominent benchmarks. can be observed that Hyperbolic- GNN achieves optimal performance on Cora and CiteSeer, with a 1% improvement on Cora. Ad-
'httpa://github com/ wwwdw//nnwwwnnn
5

---

Hyperbolic-PDE GNN
Table 5. The results of hase models and Hyperholic--Pe GNN: mean accuracy standard deviation on 60%/20%/20% data splits on
10 runs. Bold indicates optimal results for all methods, and Bold indicates optimal results between Hyperbolic-PDE GNN and base model. * indicates reproduced results.
Methods
Cora
CiteSeer
PubMed
Computers
Phuto
CS
Actor
Texas
Cocnell
DurzerEurupe
SGC
86.48_11.s
80 15
86.362a82
88 19 as
50.00amw**
96, 13
28. 81
S1. 81.31ga
72 22
63. 15,1
Hyperbolie-SGC
86 22,1.2
18.70.1.s L 2.02
88 25
87. 67.964111
94.09
96, 06.46-.22
41.49.1.37
93.61=2.1
91.28-22.5
69.00 09.02:0.72
(m))
0.74
2.89
L. 0.23
? 00:49
1.33
13..8
1 112.00
1
1 55.77
APPNP
88.14
80.07
68, 12
85, 65,32
68.51 at
96, 83
39 16.66...5
90.98
91.81
68.53
Hoperbolic-APPTT
90.00 20.07
11:1.11
30.79. o7
88_. op
58 188.... 16
40. 10.19:2am ov
91.31:5.40
45
68.78.0.39
(A)
11.98
7 1.22
2.07
7. 4.56
7 6..21
1 0.76
? 10.5
1 10.33
4 4.366
1 0.25
GPR-GNN
80.12
88, 16
25
03.85
47 us
39 92guur
92 15
Hyperbolle-GPP
90 20.82
81 88.3..5
88 so
24.4020.45
40 40.68820.4 as.
98 28.-2823.34
92.77 30
68.62200z
Emggron(()) (A)
2.22
1.76
2:90
11.36
1 1.064
1.111
1.776
0.33
1..40
1.0.7
ChebNet
=0.67
79.11
87.96 as
57.542n.4
93.77a
20
37, 37.6120m0
22:2.45
$3.9322.13
08.88.0.81
Hyperholic-Cheh
89.38,
81, 28 on
90. 20.50
80. 89.16_00
94. 44.83g0.44
96 10.50_.u
40.63_1 as
93.988_
91.06 91.06.2a.4
68. 08.7
(mgrron((A)
2.71
2.17
2.65
1 1.66
1 1.06
1 0.84
1 13.02
7.771
1 7711
L..13
BernNet
68. 52
$0.09:0079
88, 48
44
93, 13:6
24
41. 11.7921.. on
93. 93.12:u.s
92.13:1.84
08 27.077000
Hyperbolii-Ber
88. 88.34.
80 27 27.0.r
69. 52:52:20
88 61 os
94 24:0.a
96 64
30 961nas
93 11
69. 70,
68 74:07s
(mpraa.(A)
1.118
0.18
11.04
0.97
? 00:63
7 0.100
1.88
0.001
4 2.34
1 0.07
JacobiCome
88, 88.98
80.78.mm
89, 41
90 39
95. 23
96. 24
41.1 41.17
93. 93.44:2
92 95
68, 18.66
Hyperbolio-Jacoii
80.60 20.65
81. 11...2..
90. 17 as
88 60 79
94.53
96. 20.66_-
30 733024
91. 11.97 2411
90.64 20.64:2..3
68 76.76.0
(m)ran.(A)
0.71
" +0.39
-0.55
L 1.70
0..00
0.04
1.44
1.47
2.31
0.30
ChebNeell
80.53..0.7
88, n8.90:.n.2
91 06
95 no
96. 19
41 11.75:100 or
93 18.2781147
92. 30 22.30:1144
68 95.0.s
Hyperbolic-Chehl
50.39 20.us
81. 60_ 1.38. 1.38
90.38 34g877
80. 80.822u0
94.53
96. 67 -
40 40.5821.0
933.30
91. 1..0Gg
68.68
(mprron(AA)
1.67
1.07
1..1
L 1.24
1.35
1 70.88
11:17
1 10.02
1.24
[ 0..27
Cleseer
Gompstur
Acter
Wacoman
GFR-CR
2.0
GPA-GNN Berrkit
Bemilet
2.5
IacobiCorv ony
Chebketll
2.0
Hlypethol CPS
Hispertel lypertelleGGa
1.0
0-
60-
65-
Gm CRN
GPA-GMN
Barritet
Bernie
60-
JacesiBer
-55
Neab/erll
Hyperta
Hyyperbelee
-10
Figure 1. The graph spectral filter learned by the methods on graph datasets.
ditionally, our method also demonstrates competitive per- formance on PubMed and Actor These results attest to the effectiveness of Hyperbolic-PDE GNN.
high error (further experiment can be found in Section 4.2, Our approach explicitly constrains the solution space of all aforementioned base models, ensuring that they are solely expressed by eigenvectors representing the topological struc- ture and do not introduce additional features.
To further validate the performance improvement of model- ing message passing as a system of hyperbolic partial differ- ential equations, we choose several base models and modify them accordingly. Specifically, we let the polynomial P(x) in Equation (15) to be the spectral graph convolution of the base model. Initially, we choose base models such as SGC, APPNP, and GPR-GNN based on monomials. Subsequently, we choose ChebNet, BernNet, JacobiCony, and ChebNedl as convolution forms, which are based on polynomials as basis. Table 5 indicate that methods based on monomials un- derperformed due to the insufficient ability of monomials to approximate filtering functions. In contrast, methods based on polynomials exhibit lower errors in function approxi- mation, thus demonstrating superior performance. Upon enhancing all these methods to the system of hyperbolic PDEs, they all show varying degrees of performance im- provement. Particularly for methods based on monomials, the introduction of the mechanism help to mitigate their
Filter analysis. Figure visualizes the filters learned by some methods. We can observe that all methods exhibit pow- erful low-pass characteristics on CiteSeer and Computers. Particularly, due to the adoption of a Laplacian with self- loop in GPR-GNN, the absence of scaling coefficients in the higher frequency range results in a certain degree of weight explosion in the lower frequency range. Hyperbolic effectively mitigates this issue. On Actor and Wisconsin, the methods show weaker high-pass characteristies. Only the curve of Hyperbolic GPR displays trend of being high on the left and low on the right, indicating its amplification of high-frequeng signals and suppression of low-frequency signals. Furthermore, due to the issues in GPR-GN GNNN when dealing with heterophilic graphs, it can only suppress low- frequency signals by assigning negative scaling coefficients
6

---

Hyperbolic-PDE GNN
Table 6. Average of squared error (R² score) of methods over 50 images.
Low-pass
High-pass
Band-pass
Band-rejection
Comb
Low-band-pers band-pass
Runge
e -110-1
|sin(xA))
1/(1 + 26A")
GCN
3.5364( 1871))
67. 1805; 2440)
25. 99.9 1061)
20. (9440)
51.0312 20.03) 20133)
5343( 9686)
3.7037( 9860)
Hyperboli-GGNN
(39562
(.787)
1.4976/.7133
3.0469(.9925)
86577)
4.6308 .869))
(597))997))
Improv.(%)
65.6%%%
97.38%
82 65%
85.43%
82.7%
70.19%
13.43%%
GAT
2.6614 9899))
22, 12.1274(,743 7457)
13.7478( 4942)
12.8713(.9650)
22. 22.1316/ 6985)
(2.155)(60885
13. 13.2557(19633
2.8153 (8850)
HyperbolieGGAT
1.2671
(592(.9822)
2.1492i 9152)
2.5862(.9933
0.03655 (955))
4.7609( 9866)
(.6192(9999))
(1ngroc.(%)
52.399%
93.86%
84.37%
797.71%
72.72%
64.0%%
78.001%
ARMA
18344 9931)
L8304( 9793)
7,8931(7099)
8,1731 1785)
15.2385(
57457 9677)
2,1851( (9133)
Hyperbolic-ARMA
0. 0.9699/.9909 .9964)
(2063 .9854)
(T174_9212)
1.0044,9950
1.98(19100
4,9021 .8858)
(637(.9988
Imggroc((%)
13.%%
43.93%
17.299%
80.38%
61.03%
57.65%
78.78%
ChebNet
0.8024( 9078))
0.7877( 9003)
2,32321 2.32321.9000)
2.5446, 9934)
4.0797( 9443)
(.9788
0.4048(.008)
Hyperbolie-Cheb
0.00771.0000 000)
(.999)
(05677.9778
(.777799997)
(0119(9960)
0.2276 9904)
01699
Improu.(%)
19.904
98.88%
57.56%
95.777
J. 97.500%
94.88%
95.83%
BerNNet
0.030B( (9999)
0 (1105.9998))
(1477((9968)
(99733)
1,1073 9854)
2.9633.99177
0.0348( (9989)
Hyperbolie-Bern
1..004551.0000
0.0138( 9998)
0.0154i, 99922)
0.1532 (996)
1921 .9733)
0.6738( 9982)
02.0256.99000
Imppror((%)
1.85.444 L.85444%
33.98%
67.7. 71%
83.76%
82.65%
77. 26%
26.01%
ChebNedll
0.0068(1.000)
0.0028( (1000))
0. 0.0980[,900 9760))
0.0162(1
2832 996-1)
(9941)
0.0089 9399)
HCperbllii-ChehlI
0. (0039(1.000
0 0..0077 .9999)
0.0099i 9991
0.0 0.0125(1 1.000)
0.2918( 9962)
1.3800 .962)
0 0059 1.000)
Improu.(%))
42.65%
39.29%
89.90%
22.22%
1 3.04%
4 1.04%
33.71%
a) Lowspass
b) High-pass
Band pass
d) Band-rejection
e) Comb
Low-band pass
g) Runge
Figure The image signals processed by the real filter (first row), Hyperbolic-Gee GCN (second row), and GCN (third row).
to them. Hyperbolic-GPe also circumvents this problem.
in (Balcilar et al., 2021; He et al., 2021), where each pixel denotes a node. Each image associated with 100 x 100 adjacency matrix and a 10000-dimensional signal vector that records pixel intensity.
4.2. Signal Filtering on Images
In this section, we investigate the fitting capability of polyno- mial spectral filters under the influence Hyperbolic-Pe PDE GNN. We conduct experiment on 50 real images with a resolution of 100 x 100 from the Image Processing Toolbox in Matlab. Here, we transform the images into 2DD regular neeighooroood grid graph following the experimental setup
For each image, we filter its original signals using 7 follow- ing spectral filters: low-pass filter, high-pass filter, band- pass filter, band-rejection filter, comb filter, low-band-pass filter, and runge filter. The formula and its curve of each filter can be found in Table For example, applying low-

---

Hyperbolic-PDE GNN
pass filter to an image is equivalent to performing operation y = Udiag(e- -10x e 1013 JUTx x on graph signal, where the filtered graph signal y used as the ground truth.
alized PageRank. These early works focus on exploring features of homophilic graphs but show poor performance heterophilic graphs. GPR-GNN (Chien et al., 2021) de- signs filter parameters based on PageRank to adaptively learn low-pass or high-pass filters. GNN-LF/HF (Zhu et al., 2021) uses adjustable graph kernels to design low-pass and high-pass filters. ARMA (Bianchi et al., 2021) employs auto-regressive moving average filters to design graph con- volutions, which can better capture global graph features. While these methods can adapt to different types of graphs, their performance suffers when fitting complex filters due to their design of graph filters based on monomials. Cheb- Net (Defferrard et al., 2016), as an early attempt of polyno- mial filters, first approximates graph filters using Chebyshev polynomials as a basis. BernNet (He et al., 2021) utilizes basis to simulate complex filters such as band-pass and comb filters. JacobiConv (Wang & Zhang, 2022) unifies some methods based on orthogonal polynomials, like ChebNet, by employing more general Ja- cobi polynomials. ChebNetll (He et al., 2022) addresses the issue of learning Illegal filter parameters in ChebNet by in- troducing Chebyshev interpolation to reduce approximation errors in Chebyshev polynomials. Most of these methods construct graph filters based on mature polynomial basis in mathematics, theoretically capable of approximating any filter function. Another class of methods designs more tar- geted filters to address various graph problems. G2CN (Li et al., 2022) introduces Gaussian graph filters to enable models to flexibly adapt to different types of graphs. Even- Net (Lei et al., 2022) proposes even-polynone ven-polynomial graph filters to enhance the model's robustness on heterophilic graphs. OptBasisGNN (Guo & Wei, 2023) learn the optimal basis in the orthogonal polynomial space. UniFilter (Huang et al., 2024b) specifically designs adaptive heterophily basis for heterophilic graphs. NEGNN (Zheng et al., 2024) addresses the challenge of local patterns in graphs by proposing local spectral filters. AdaptKry (Huang et al., 2024a) optimizes graph filters using basis from adaptively Krylov subspaces. All these methods are defined in the original spatial domain, meaning the node features they obtain lack an interpretable solution space to explain their relationship with topological features, thus lacking necessary interpretability.
The objective of this experiment is to minimize the average of of squared error between the model output and ground truth across 50 images. Therefore, this experiment is consid- ered as a regression task. We utilize the coefficient of deter- mination (i.e., HP2 score, closer to 1 is better) to reflect the degree of in regression. We select GCN (Kipf & Welling, 2017), GAT (Velickovic et al., 2018), ARMA (Bianchi et al., 2021), GPR-GNN (Chien al., 2021), ChehNet (Defferrard et al. 2016), BernNet (He et al., 2021). JacobiConv (Wang & Zhang, 2022), and ChebNetll (He et al.. 2022) as baselines and evaluate their fitting capabilities under the enhancement of Hyperbolic-PDD GNN. To ensure fairness, we follow the setup of He et al. (2021) to reproduce the results of each baseline. For our method, we conduct parameter search using wandb. See appendix for specific experimental configurations.
Bernstein polynomials
As shown in Table 6, GCN and GAT exhibit good prop- erties as low-pass filters, but struggle to fit more complex filters. On the other hand, polynomial spectral filters have the ability to fit complex filters. All the aforementioned base models demonstrate enhanced fitting capabilities un- der the augmentation of Hyperbolic-PDD GNN. Particularly notable is the significant improvement observed in GCN and GAT, indicating that Hyperbolic-PDD GNN not only enhances the expressive power of spectral filters but also endows them with the ability to fit complex filters.
Filter analysis. Figure 2 illustrates graph signals obtained through real filters, Hyperbolic-GCN and GCN. We can observe that the graph signals filtered by Hyperbolic-GGC closely resemble the original graph signals, while GCN struggles to extract useful features when fitting complex filters. Moreover, the most notable improvement is seen in ChebNet, whose inferior performance is attributed to learn- ing illegal filter coefficients (He et al., 2022). Hyperbolic- Cheb can limit the solution space by defining initial values, thereby reducing the risk of learning illegal coefficients.
5. Related Works
Differential Equation on Graph. In recent years, re- search has gradually delved into the dynamical systems of graph neural networks. Neural ODE (Chen et al., 2018) initially models embeddings as a continuous dynamic model based on ordinary differential equations (ODE). GDE (Poli et al., 2019) employs forward Euler method to solve ODE on graphs. CGNN (Xhonneux et al., 2020) directly com- putes the analytical solution of ODE. GraphCON (Rusch et al., 2022) establishes a connection with conventional graph convolutions based on second-order ODE. Kuramo- toGNN (Nguyen et al., 2024) specifically utilizes the Ku-
Spectral Graph Neural Networks. Spectral CNN (Bruna et al., 2014), as a pioneering work in spectral graph neural networks. first apply spectral graph theory to the field of graphs. GCN (Kipf& & Welling, 2017), as the most classic subsequent work, behaves like low-pass filter. SGC (Wu et al., 2019) builds upon GCN by removing non-linear ac- tivation functions and simplifying the parameter matrices in each layer, demonstrating that linear GNNs also exhibit strong performance. APPNP (Klicpera et al., 2019a) pro- poses an improved propagation scheme based on Person-
8

---

Hyperbolic-PDE GNN
ramoto model to address over-smoothing issues. MGKN (Li et al., 2020) introduces partial differential equations (PDE) into graph problems. PDE-GCN (Eliasof et al., 2021) con- structs graph convolutions using diffusion and hyperbolic equations, respectively. DGC (Wang et al., 2021) decouples terminal time from propagation steps. GDC (Kliepera et al., 2019b) proposes graph diffusion convolution to extract in- formation from indirectly connected neighbors. ADC (Zhao al., 2021) further introduces adaptively diffusion convolu- tion to aggregate neighbors within the optimal neighborhood. Subsequent models like GRAND (Chamberlain et al., 2021), GRAND++ (Thorpe et al., 2022), and HiD-Net (Li et al., 2024) treat message passing as a heat diffusion process and model based on the heat diffusion equation. Most of the aforementioned methods heuristically integrate differential equations into graph neural networks and demonstrate em- pirically their effectiveness. However, these methods do not systematically explore why models based on differential equations perform well, and do not delve into the intrinsic properties of node features obtained these methods.
Innovation Promotion Association of CAS (Grant No. 2021153), and the Postdoctoral Fellowship Program of CPSF (No.GZC20232269).
Impact Statement
The work presented in this paper aims to advance the field of graph deep learning. We do not expect any immediate, negative societal impact of the present work. It delves into the theory of graph neural networks, and thus may benefit the applications that involve graphs by making them more effective and reliable.
References
Balcilar, M., Renton, G., Héroux, P., Gaüzère, B., Adam, S., and Honeine, P. Analyzing the expressive power of graph neural networks in a spectral perspective. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. Open- Review.net, 2021.
Discussion with Hyperbolic Geometry GNNs The hy- perbolic PDEs and hyperbolic geometry (Liu et al., 2019; Chami et al., 2019) exhibit fundamentally distinct charac- teristics. Specifically, hyperbolic geometry focuses on the hierarchical structure of graphs, which typically involves mapping nodes to hyperbolic space using models such as Poincaré Ball models or Lorentz models to extract hier- archical relationships between nodes, On the other hand, our focus of hyperbolic PDEs lies in constructing the mes- sage passing mechanism of GNNs based on the theory of PDE. This design naturally maps nodes into a solution space spanned by a set of eigenvectors of the graph, which can improve the ability to learn complex graph filters.
Bianchi, F. M., Grattarola, D., Livi, L., and Alippi, C. Graph neural networks with convolutional ARMA filters. IEEE Trans. Pattern Anal. Mach. Intell., 2021. doi: 10.1109/ TPAMI.2021.305444.
Bo, D., Shi, C., Wang, L., and Liao, R. Specformer: Spec- tral graph neural networks meet transformers. In The Eleventh International Conference on Learning Repre- sentations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
Bruna, J., Zaremba, W., Szlam, A., and LeCun, Y. Spectral networks and locally connected networks on graphs. In Bengio, Y. and LeCun, Y. (eds.), 2nd International Con- ference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Pro- ceedings, 2014.
6. Conclusion
To resolve limitation of the spatial space of MP in rep- resenting node semantic features, this paper introduces a Hyperbolic-PDD GNN based on a system of hyperbolic PDEs. We theoretically demonstrate that the solution space is spanned by orthogonal bases that describe the topological structure of graphs. enabling to represent the topological features, thereby significantly increasing the interpretability of the model. Moreover, we enhance the flexibility and ro- bustness of the model using polynomial spectral filters and establish connection between hyperbolic PDEs and spec- tral GNNs. Extensive experiments show that our approach exhibits effective and robust results with traditional GNNs.
Chamberlain, B., Rowbottom, J., Gorinova, M. I., Bronstein, M. M., Webb, S., and Rossi, E. GRAND; graph neural diffusion. Meila, M. and Zhang, T. (eds.), Proceedings of the 38th International Conference on Machine Learn- ing, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 1407-1418. PMLR, 2021.
Chami, I., Ying. Z., Ré, C., and Leskovee, J. Hyperbolic graph convolutional neural networks. In Wallach, H. M., Larochelle, H., Beygelzimer, A., d' A'Alché-Bunn Buce F., Fox, E. B., and Garnett, R. (eds.), Advances in Neural Infor- mation Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp- 4869-4880, 2019. URL. https: http:///ooooogeeeeeee/ee
Acknowledgments
This work is supported by the National Natural Sci- ence Foundation of China (No.62406319), the Youth

---

Hyperbolic-PDE GNN
pttrripsscc/pappr2/119//////// html.
Geng, H., Chen, C., He, Y., Zeng, G., Han, Z., Chai, H., and Yan, J. Pyramid graph neural network: A graph sam- pling and filtering approach for multi-scale disentangled representations. In Singh, A. K., Sun, Y., Akoglu, L., Gunopulos, D., Yan, X., Kumar, R., Ozcan, F,, and Ye, J. (eds.), Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2023, Long Beach, CA, USA, August 6-10. 2023, pp. 518-530. ACM, 2023. doi: 1444555830553559999
Chen, T. Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D. Neural ordinary differential equations. In Bengio, S., Wallach, H. M., Larochelle, H.,, Grauman, K., Cesa- Bianchi, N., and Garnett, R. (eds.), dads),,Avvvan Advances in Neural In- formation Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pp. 6572- 6583, 2018.
Gilmer, J., Schoenholz, S. S., Riley, F., Vinyals, O., and Dahl, G. E. Neural message passing for quantum chem- istry. In Precup, D. and Teh, Y. W. (eds.), Proceedings of the 34th International Conference on Machine Learning. ICMI. 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 1263-1272. PMLR, 2017.
Chien, E., Peng, J., Li, P., and Milenkovic, O. Adaptive universal generalized pagerank graph neural network. In 9th International Conference on Learning Representa- tions, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
Guo, Y. and Wei, Z. Graph neural networks with learnable and optimal polynomial bases. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), International Conference on Machine Learning. ICMI. 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Re- search, pp. 12077-12097. PMLR, 2023.
Defferrard, M., Bresson, X., and Vandergheynst, P. Con- volutional neural networks on graphs with fast localized spectral filtering. In Lee, D. D., Sugiyama, M., von Luxburg, U., Guyon, I., and Garnett, R. Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 3837- 3845, 2016.
Advances
He, M., Wei, Z.,, Huang, Z.,, and Xu, H. Bernnet: Learning arbitrary graph spectral filters via bernstein approxima- tion. In Ranzato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W. (eds.), Advances in Neu- ral Information Processing Systems 34: Annual Confer- ence on Neural Information Processing Systems 2021, NeurlPS 2021, I, December 6- 64, 2021, virtual, pp. 14239- 14251, 2021.
Eliasof, M., Haber, E., and Treister, E. PDE-GCN: novel architectures for graph neural networks motivated by par- tial differential equations. In Ranzato, M., Beygelzimer, A., Dauphin, N., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, vir- tual, pp. 2836-3849 2021.
He, M., Wei, Z., and Wen, J. Convolutional neural networks on graphs with chebyshev approximation, revisited. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Infor- mation Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022.
Fey, M. and Lenssen, J. E. Fast graph representation learning with Py Torch Geometric. In ICLR Workshop on Repre- sentation Learning on Graphs and Manifolds, 2019.
Gao, Y., Wang, X., He, X., Liu, Z., Feng, H., and Zhang, Y. Addressing heterophily in graph anomaly detection: A perspective of graph spectrum. In Ding, Y., Tang, J., Sequeda, J. F., Aroyo, L., Castillo, C., and Houben, G. (eds.). Proceedings of the ACM Web Conference 2023, WWW 2023, Austin, TX, USA, 30 April 2023 4 May 2023. pp. 1528-1538. ACM, 2023. doi: 10. 10.1145/3543507. 3583268.
Huang, K., Cao. W., Ta, H., Xiao, X., and Liò, P. Opti- mizing polynomial graph filters: A novel adaptive krylov subspace approach. In Chua, T., Ngo. C., Kumar, R., Lauw, H. W., and Lee, R. K. (cds.). Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 13-17, 2024, pp. 1057-1068. ACM, 2024a. doi: 00.1455/58933433455755
Gaudelet, T., Day, B., Jamasb, A. R., Soman, J., Regep, C., Liu, G., Hayter, J. B. R., Vickers, R., Roberts, C., Tang, J., Roblin, D., Blundell, T. L., Bronstein, M. M., and Taylor- King, J. P. Utilizing graph machine learning within drug discovery and development. Briefings Bioinform., 22(6),
Huang, K., Wang. Y. G., Li, M., and Lio, How universal polynomial bases enhance spectral graph neural networks: Heterophily, over-smoo and over-squas In
2021. doi:
10

---

Hyperbolic-PDE GNN
Forty-first International Conference on Machine Learn- ing, ICML. 2024, Vienna, Austria, July 21 21-27, 2024. Open- Review.net, 2024b.
Li, Z.., Kovachki, N. B., Azizzadenesheli, K., Liu, B., Stuart,
A. M., Bhattacharya, K., and Anandkumar, A. Multipole graph neural operator for parametric partial differential equations. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural In- formation Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Kingma, D. and Ba, J. Adam: A method for stochastic optimization. In 3rd International Conference Learn- ing Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Truck Proceedings. OpenReview.net, 2017.
Liao, W., Zhu, Y., Li, Y., Zhang, Q., Ou, Z., and Li, X. Revgnn: Negative sampling enhanced con- trastive graph learning for academic reviewer recom- mendation. ACM Trans. Inf. Syst., 43()::1:11111 2025. doi: 10. 10.1145/3679200 URL https: //tp:///oii/// org/10. 1145/367922
Klicpera, J., Bojchevski, A., and Günnemann, S. Predict then propagate: Graph neural networks meet personalized pagerank. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019a.
Lin, J., Liu, W., Dai, X., Zhang, W., Li, S., Tang, R., Hc, X., Hao, J., and Yu. Y. A grraph-enhancee click model for web search. In Diaz, F., Shah, C., Suel, T. Castells, P., Jones, R., and Sakai, T. (eds.), SIGIR '21: The 44th International ACM SIGIR Conference on Re- search and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, pp. 1259-1268 1268. ACM, 2021. doi: 10.1 0111444444485534662685 / //doi. org/ 10. 11 /5/4///4444 3404835443333 34628955
Klicpera, J., Weißenberger, S., and Günnemann, S. Dif- fusion improves graph learning. Wallach, H. M., Larochelle, H., Beygelzimer, A., d' Alché-Buu, F., Fox, E. B., and Garnett, R. (eds.), Advances in Neural Infor- mation Processing Systems 32: Annual Conference Neural Information Processing Systems 2019, NeurlPS 2019, December 8- 14, 2019, Vancouver, BC, Canada, pp. 13333-13345, 2019b.
URL https:
Liu, Q., Nickel, M., and Kiela, D. Hyperbolic graph neural networks. In Wallach, H. M., Larochelle, H., Beygelzimer, A., d'Alché-Buu F., Fox, E. B., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 8228- 8239, 2019. URL https: //tpc/e/pooceeeooo igg.. (1330ddd555333137766116662 Absstraat.. html.
Lei, R., Wang. Z., Li, Y., Ding, B., and Wei, Z. Evennet: Ignoring odd-hop neighbors improves robustness of graph neural networks. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022.
Li, M., Guo, X., Wang, Y., Wang, Y., and Lin, Z. G2en: Graph gaussian convolution networks with concentrated graph filters. In Chaudhuri, K., Jegelka, S., Song, L.., Szepesvári, C., Niu, G., and Sabato, S. (eds.), Interna- tional Conference on Machine Learning, ICML. 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp- 12782-12796. PMLR, 2022.
Ma, X., Wu, J., Xue, S., Yang. J., Zhou, C., Sheng, Q Z.,
Xiong, H., and Akoglu, L. A comprehensive survey graph anomaly detection with deep learning. IEEE Trans. Knowl. Data Eng., (2(12) 12038, 2023. doi: 10.1109/TKDE 311 8815. URL https:/ / doi. wotg/1o.110000 TKDE 2021. 3118815.
Nguyen, T., Honda, H., Sano, T., Nguyen, V., Nakamura, S., and Nguyen, T. M. From coupled oscillators to graph neu- ral networks: Reducing over-smoothing via kuramoto model-based approach. In Dasgupta, S., Mandt, S.,, and Li, Y. (eds.), International Conference on Artificial Intelli- gence and Statistics, 2-4 May 2024, Palau de Congressos, Valencia, Spain, volume 238 of Proceedings of Machine Learning Research, pp. 27 2710-2718. 18. PMLR, 2024.
Li, Y., Wang, X., Liu, H., and Shi, C. A generalized neural diffusion framework on graphs. In Wooldridge, M. J.. Dy, J. G., and Natarajan, S. (eds.), Thirty-Eightt AAAI Con- ference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelli- gence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pp. 8707-8715. AAAI Press, 2024. doi: 10.1609/AAAI, 3881 28716.
Page, L., Brin, S., Motwani, R., and Winograd, T. The pagerank citation ranking: Bringing order to the web. Technical report, Stanford infolab, 1999.

---

Hyperbolic-PDE GNN
Pei, H., Wei, B., Chang, K. C., Lei, Y., and Yang, B. Geom- gen: Geometric graph convolutional networks. In 8th International Conference Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
neural diffusion with A source term. In The Tenth Inter- national Conference Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
Platonov, O., Kuznedelev, D., Diskin, M., Babenko, A., and Prokhorenkova, L. A critical look the evaluation of gnns under heterophily: Are we really making progress? In The Eleventh International Conference Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
Wang, X. and Zhang, M. How powerful are spectral graph neural networks. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvári, C., Niu, G., and Sabato, S. (eds.), Inter- national Conference on Machine Learning, 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. PMLR, 2022.
Poli, M., Massaroli, S., Park, J.. Yamashita, A., Asama, H., and Park, Graph neural ordinary differential equations. CoRR, abs/1911 91.07532, 2019.
Rozemberezki, B. and Sarkar, R. Characteristic functions on graphs: Birds of a feather, from statistical descriptors to parametric models. In CIKM '20: The 29th ACM Interna- tional Conference on Information and Knowledge Man- agement, Virtual Event, Ireland, October 19-23, 2020. ACM, 2020. doi: 10. 155553340531134118883
Wang. Y., Wang. Y., Yang. J., and Lin, Z. Dissecting the dif- fusion process in linear graph convolutional networks. In Ranzato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Informa- tion Processing Systems 34: Anmual Conference on Neu- ral Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pp. 7758-5769, 2021.
Rozemberczki, B., Allen, C., and Sarkar, R. Multi-scale at- tributed node embedding. CoRR, abs/1909.13021, 2019.
Rusch, K., Chamberlain, B., Rowbottom, J., Mishra, S., and Bronstein, M. M. Graph-coupled oscillator networks. In Chaudhuri, K., Jegelka, S., Song, L.., Szepesvári, C.,, Niu, G., and Sabato, S. (eds.), International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Bal- timore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 18888-18999 18909. PMLR, 2022.
Wu, F., Jr., A. H. S., Zhang, T., Fifty, C., Yu, T., and Wein- berger, K. Q. Simplifying graph convolutional networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA. PMLR, 2019.
Wu, S., Sun, F., Zhang, W., Xie, X., and Cui, B. Graph neural networks in recommender systems: A survey, ACM Comput. Surv., 55(5):97:199733) 2023. doi: 10. 10.114550 145/333331...
Shchur, O., Mumme, M., Bojchevski, A., and Günnemann, S. Pitfalls of graph neural network evaluation. CoRR, abs/1811.08 1.05868, 2018.
Wu, 7.., Pan, S., Chen, F., Long, G., Zhang, C., and Yu, P. S. A comprehensive survey graph neural networks. IEEE Trans. Neural Networks Learn. Syst., 32(1):4 244,:4224,2 2021. doi: 1LL110//NNNLS22220299758855
Shlomi, J., Battaglia, P. W., and Vlimant, J. Graph neural networks in particle physics. Mach. Learn, Sci. Technol., 2(2):21001, 2021. doi: 10. 2225:266222113/BBBBB3
Shuman, D. I., Narang, S. K., Frossard, P., Ortega, A., and Vandergheynst, P. The emerging field of signal processing on graphs: Extending high-dimensions to networks and other irregular domains. IEEE Signal Process. Mag., 30(3):83 2013. doi: 10.1109/MSP. 2012.2235192.
Xhonneux, L. A. C.,, Qu, M., and Tang. J. Continuous graph neural networks. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 10432 10441. PMLR, 2020.
data analysis
Stone, M. H. Applications of the theory of boolean rings to general topology. Transactions of the American Mathe- matical Sociery, 41(3):375-483 1937.
Yang. Z., Cohen, W. W., and Salakhutdinov, R. Revisiting semi-supervised learning with graph embeddings. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016. JMLR.org, 2016
Thorpe, M., Nguyen, T. M., Xia, H., Strohmer, T., Bertozzi, A. L.., Osher, S. J., and Wang, B. GRAND++: graph
12

---

Hyperbolic-PDE GNN
Yue, J., Li, H., Sheng, J., Guo, Y., Zhang, X., Zhou, C., networks. In Long. G., Blumestein, M., Chang, Y., Lewin-Eytan, L.,, Huang, Z. H., and Yom- Tov, E. (eds.), Proceedings of the ACM on Web Conference 2025, WWW 2025, Sydney, NSW, Aus- tralia, 28 April 2025- May 2025, pp. 1365-1379. ACM, 2025. doi: 10. 000114////96411.371465... URL https: //doi. org/ 10. wtt.e//.........11211 wtt..111///1...113111
Liu, T., and Guo, L. Graph
Zhao, J., Dong, Y., Ding, M., Kharlamov, E., and Tang, Adaptive diffusson in graph neural networks. In Ran- zato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Neural Processing Systems 34: Annual Conference
Information Processing Systems 2021, NeurlPS 2021, De- cember 6-14, 2021, virtual, pp. 23321-23333, 2021.
Zheng, S., Zhu, Z., Liu, Z., Li, Y., and Zhao, Y. Node- oriented spectral filtering for graph neural networks. IEEE Trans. Pattern Anal. Mach. Intell., 46(1):388-402, 2024. doi: N0.10997PAM.222333249999
Zhu, M., Wang, X., Shi, C.,, Ji, H., and Cui, Interpreting and unifying graph neural networks with optimization framework. In Leskovec, J.. Grobelnik, M., Najork, M., Tang, J., and Zia, L. (eds.), WWW '21: The Web Con- ference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, pp. 1215-1226. ACM/ IW3C2, 2021. doi: 00.11457344238134499933
Zhu, Y., Cong, F., Zhang, D., Gong, W., Lin, Q. Feng. W., Dong, Y., and Tang, J. Wingnn: Dynamic graph neural networks with random gradient aggregation window. In Singh, A. K., Sun, Y., Akoglu, L., Gunopulos, D., Yan, X., Kumar, R., Ozean, F., and Ye, J. (eds.), Proceedings Knowledge
of the 29th ACM SIGKDD Conference Discovery and Data Mining, KDD) 2023, Long Beach, CA, USA, August 6-10, 2023, pp. 3650-3662. ACM, 2023. doi: 10. 000,1145//58000035559551511 URL https: //tttps::/oo... /doi. htt//0..115/3580000 35995511
13

---

Hyperbolic-PDE GNN
A. Theoretical Analysis
A.1. Detailed Derivation of the System of Partial Differential Equations
Given Equation (8), we combine the equations all nodes on any dimension to obtain a system of sccond-order linear partial differential equations:
Exu 0e2
a²
Σ
(x1 kitt)
vs (vs)
0x1211
a
(22) Izi),
012
Uy (v2))
(21)
0 0x 0t
a
(ont - xzz.)
My (o,))
This system of equations represents the evolution pattern of a graph signal on any dimension I over time. Next, we extend the dimension dimensions:
02x1
8 8xn
Øxid
012
012
012
012
02x2
82121
02m
8z2d
012
012
01
01
(22)
8xxn
Pan2 8xI11
8x
012
01
012
012
Then, the right-hand side of Equation (21) can be extended as
EnjeNton (e1 - -21)
E (111 - -5
E Fza)
E EnseN(ma) (221- xz1)) EneeN(re) (122 -zz2)
EnjeN(ra) ( (I2d Ija)
a
(23)
Σ EeyEN(uw) 2z1) ExjeN(on (2n2 xz22)
ExyeNtio IzJa)
By combining Equation (22) and Equation (23), we obtain:
02x 01
a²l Σ xx)x Σ
(x2-x))- (x)xx)).....
Σ
xx--x)11 x,)] = a²Tx
(24)
()EN(e))
1yEN(Um)
where x; [Ta, T2,,..., x.]
A.2. Proof of Theorem 3.1
Given Equation (9), we consider a graph signal on any dimension
I (i.e., Equation (21)).
Let UEu 01 Vit, 01 V21 ..... Grat de
Orai
Ynt, then Syn a² EnjeN(on (Fn I.1), Oya a² (nye
I.J1).... ...,
0yml
a EngEN(r..) (Int Izt).
Subsequently, let w, = Iy., x,] E 222, Equation (21) can be rewritten in the following form:
dwi Cw:
T, 0
a'L
Wi.
(25)
where İ. D A E RRax is a the Laplacian matrix.
Note that since spatial derivatives w.r.t space have been transformed into gradients and divergences of graph, w, is now only differentiated w.r.t time. Therefore, Equation (25) essentially a system of first-order constant-coefficiett homogeneous linear differential equations.
14

---

Hyperbolic-PDE GNN
To obtain the fundamental matrix of solution of Equation (25), it is necessary introduce the following definition and theorem from the mathematical domain.
Given system of first-order homogeneous linear differential equations
dx
C(t)x,
(26)
dl
we have:
Definition A.1. If each column of matrix (() E R is solution Equation (26), then the matrix (t)) is referred
as the solution matrix of Equation (26). If the columns of the solution matrix are linearly independent over the interval S t S 6, it is referred to as the fundamental matrix of solution of Equation (26) on the interval S S 6.
Here, we represent linearly independent column vectors from the matrix (t) as 4 (1) (2)( (f). According to Definition A.1, we can deduce the following theorems:
Theorem A.2. The fundamental matrix of solution defines the solution space of Equation (26), enabling us to express any solution of Equation (26) as a linear combination of linearly independent column vectors ( (P) P (1) from the fundamental matrix of solution (t)).
Now consider system of first-order constant-coefficient homogeneous linear differential equations
dx -C,,
(27)
dt
where C E Raxa is constant matrix.
Theorem A.3. The matrix
((1) = exp Ct is a fandamental matrix of solution of Equation (27), and (0) = I.
(28)
Proof. is easily inferred from the definition that 2 I. Taking the derivative of Equation (28) w.r.t t, obtain:
(1)(1) d
exp Ct
dt
dt
C21
C
+ A (k 1)! +
= C+
+
(29)
1!
2!
Cexp Ct
(t)
This indicates that (() is the solution matrix of Equation (27). Furthermore, because det. +b det. I 1, (t)) is the
fundamental matrix of solution of Equation (27).
Therefore, for Equation (25), its fundamental matrix of solution is
(()
exp Ct.
I
2
I
A R!
++
0
a
L
1!
0
a
21
0
a
(30)
Texp Il
O
a² exp Lt
where sp (t), (t))..... P2n (t) are the column vectors from (t).
Ultimately, the solution space of all dimension can be represented as the Cartesian product the solution spaces of each dimension:
x $ x Φ
(31)
where is the operator of Cartesian product.
15

---

Hyperbolic-PDE GNN
A.3. Proof of Theorem 3.3
Proof. Let (p()) uu o O, and substituting it into Equation (25), we obtain:
xe u = Ceau
(32)
Because e # 0, Equation (32) becomes:
(33)
We can observe that Equation (33) is the characteristic equation of matrix C. Therefore, e constitutes a solution to Equation (25) if and only if is the eigenvalue of C and u is the corresponding eigenvector.
For the matrix C in Equation (25), its characteristic equation
(AI - C) = -T- -
AI -I a'L = [AT A - =0.
(34)
0
Therefore, the eigenvalues of and L are both eigenvalues of C, that is X1 = X/2 =... X₂ = 1,, = a²x1, Nn42
a² a²x2. X/2n 2
The corresponding eigenvectors of I are vi [1,0, ,,. 0_, V2 [0, 1, ,,.,,
10,0, 11. and the
Vn
corresponding eigenvectors of L are i u2 Un thus the eigenvectors of are
V1
11
V2
u/
Vr
0
12
0
(35)
i1
i1,
Because u1, ub...., uhn are linearly independently. matrix
(()) = [e^''u', u
Ann' ubn):
(36)
a fundamental matrix of solution of Equation (25).
Because both exp Cl and (() are solutions of Equation (25), there exists a non-singular constant matrix B such that
exp Ci = (()
(37)
Let t 0, we have B (0) Therefore,
exp Ct -())
(38)
B. Details of Polynomial-based Spectral Graph Neural Networks
SGC. Wu et al. (2019) directly utilizes monomials as filter
((²) = (1 X)K Ā (0,2)
(39)
to obtain the spectral graph convolution
Z = (I L.* XW.
(40)
APPNP. Klicpera et al. (2019a) leverage concept of personalized PageRank (Page al., 1999) to design graph convolutions, where its filter is equivalent a polynomial
(1- R)*, λe (0
(41)
with monomial basis (1 - -)).. Then the spectral graph convolution of APPNP is given by:
Z a - (1 a) aI-I - (XX)
(42)
where a is hyperparameter, so the filter cannot be learned during the training process.
16

---

Hyperbolic-PDE GNN
GPR-GNN. Chien et al. (2021) employ learnable parameters og coefficients to learn a polynomial filter
K 02 x)* x (0,2)
((a)
(43)
k=0
with monomial basis like APPNP, can be viewed as Generalized PageRank. Then the spectral graph convolution of GPR-GNN is given by:
Z
Σ 02 0a(IIL)*((X)
(44)
k=
ChebNet. Defferrard et al. (2016) approximate graph spectral filter using Chebyshev polynomial approximation
k-1
.(A) = Σ
(xTM(A - 1), A E [0.
(45)
k=
Here, for O..... K. To(A) 1,))(A) X, (A) 2XTx-1(A) T2- are the Chebyshev base. Then the
spectral graph convolution of ChebNet is given by:
K-1 Z = T
(46)
k=0
BernNet.
He et al. (2021) utilize Bernstein polynomial approximation approximate graph spectral filters:
K
s(A)
Σ K=
Ga (A/2). A [0,
(47)
Here, for k = O...., K, (A) (1)
[0, 1] are the Bernstein base. The Bernstein polynomial
approximation can learn arbitrary spectral filters. Then the spectral graph convolution of BernNet is given by:
K
Z
21
L)K
(48)
2K
k=
JacobiConv. Wang & Zhang (2022) approximate spectral graph convolutions using Jacobi polynomial approximation:
K
g(A) = Σ
0sp (1 A), A AE E
(49)
k=0
Here, for k O..... K, P(( 1, p(a.)) (A) 2
a+b+2 X pla (2) 2
(a&d plaab(())= a dk. (A) a pf (A)
are Jacobi base, where a, are hyperparameters and 3x [2++a+ o/. = 10--aab-1)(a2--)
2k(k+a+b)
1) 11t The Jacobi polynomial is a general form of Chebyshev polynomials (i.e. = = 0.5). Then the K[e+t= spectral graph convolution of JaocibConv is given by:
K O pa.b
Z: =
()),
(50)
1=0
ChebNetII. He et al. (2022) discover that the poor performance of Chebyshev polynomials is due to the learned illegal parameters triggering the Runge phenomenon (where the fitted function deviates from the original function as the order increases). Subsequently, they introduce Chebyshev interpolation into Chebyshev polynomials to enhance graph spectral filters:
K
9(A)
Σ K=O
( 1), A A [0.
(51)
17

---

Hyperbolic-PDE GNN
Here, the coefficients co( रमन Ej=o ay, مー= 0,TR (x,)) are obtained from Chebyshev nodes 3; cos((i 1 1)). Then the spectral graph convolution of ChebNetll given by:
2 Z
(TTR(Z))T&(L (X)
(52)
K +1
A=
C. Detailed Experimental Setup
Parameter search. We utilize wandb library for parameter search in node classification task. The search strategy adopt the Bayes strategy, with parameter ranges shown in Table 7.
Table 7. The range of hyperparameters in node classification task.
Hyperparameters
Range
Distribution
Hidden-dimension
(32, 64, 128)
values
Polynomial order
{1, 4, 6, 9,
values
Time size 7
{0.2, 0.5, 1.0,
values
Terminal time T
[1
uniform
Dropout
0.,0.8
uniform
Learning rate
0.001, 0.25)
log.uniform_vee values
Weight decay
[0, 0.10
uniform
Furthermore, we conduct parameter search in signal filtering task according to the schemes outlined in Table 8. For ElyperbolicGGCN, Hyperbolic-GANe and Hyperboli-AARMA, only utilize scheme 1. For Hyperbolic- Chebb, Hyperbolic- Bern, and HyperbolicCChebll, to minimize the number of parameter combinations, we respectively employ scheme I and scheme 2 for search and select the optimal parameter combination from them.
Table 8. The range of hyperparameters in signal filtering task.
Scheme 1
Scheme
Hyperparameters
Range
Distribution
Range
Distribution
Polynomial order K
(1, 10]
values
10 {0.5. 1}
value
Time size
(0.5, 0.75,...,
values
values
Terminal time Learning rate
[0]
uniform
(0.5,
values
[0.001, 0.25)
log uniform values
[0.001, 0.25)
log uniform values
Training process. In the node classification task, we employ cross-entropy as the loss function and utilize the Adam optimizer (Kingma& Ba, 2015) for optimization. The models are trained for 200 epochs with early stopping set to 10. In the signal filtering task, we optimize the squared error between the model output and ground truth using the Adam optimizer. The models trained for 2000 epochs with early stopping set 100.
18


Reviews:
[{"review_id": "P5K6xxRHjB", "reviewer": "Reviewer_1BNk", "summary": "This paper proposes to formulate message passing in spectral GNNs as a system of hyperbolic PDEs by extending the concepts of\ngradient and divergence on manifolds to graphs. Based on this formulation, node features are shown to propagate messages along specific directions of eigenvectors and therefore better capture the topology of graphs. To improve the efficiency of the model, polynomials are used to approximate the solution. Experimental results on some graph tasks demonstrate the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[Cons 1/2]**: *Somewhat limited experimental evaluation, and some marginal improvements on a few datasets.*\n\n**[Answer]**: The proposed method is essentially a general framework that enhances the capability of spectral GNNs, as shown in Tables 4 and 5. The results on both homophilic and heterophilic graphs, and the extensive experimets on the low-pass filter and arbitrary filter methods, consistently achieve competitive results. All the analyses comprehensively demonstrate the effectiveness of our method.\n\nOn the other hand, our method also demonstrates high interpretability to connect spectral GNNs. It provide high flexibility to implement different spectral graph filters for further improvements, and reflect a new direction of GNN studies.\n\nThanks for the comments. We will conduct further exploration in our future works.\n\n**[Cons 3/Q1]**: The lack of the proof of Theorem 3.5.\n\n**[Answer]**: Theorem 3.5 is a well-known mathematical theory as Weierstrass approximation theorem [1,2], and we will introduce more details and references in the next version.\n\n[1] Stone, M. H. (1937), \"Applications of the Theory of Boolean Rings to General Topology\", Transactions of the American Mathematical Society, 41 (3): 375–481, doi:10.2307/1989788, JSTOR 1989788.\n\n[2] https://en.wikipedia.org/wiki/Stone%E2%80%93Weierstrass_theorem\n\n**[Q2]**: *Evaluation without graph classification or link prediction.*\n\n**[Answer]**: Actually, our method is a foundational and general framework, which is a **task-agnostic** graph learning method. The node classification task is a widely-adopted evaluation setting for evaluating the effectivness of GNNs [1,2,3], and thus we follow this setting. The extensive results also reflect the superiority.\nThanks for the suggestion, we would like conduct more explorations on link prediction and graph classification in our future works.\n\n[1] Wang, X. and Zhang, M. How powerful are spectral graph neural networks. ICML 2022.\n[2] Geng, H., Chen, C., He, Y., Zeng, G., Han, Z., Chai, H., and Yan, J. Pyramid graph neural network: A graph sampling and filtering approach for multi-scale disentangled representations. KDD 2023.\n[3] Zheng, S., Zhu, Z., Liu, Z., Li, Y., and Zhao, Y. Node-oriented spectral filtering for graph neural networks. IEEE Trans. Pattern Anal. Mach. Intell., 46(1):388–402.\n\n**[Q3]**: *Comparison with methods for hierarchical structures.*\n\n**[Answer]**: Actually, our hyperbolic PDE-based GNNs and traditional hyperbolic GNNs are quite different paradigms.\n- Traditional hyperbolic GNNs map nodes to hyperbolic embedding space, which can be capable to capture hierarchical structures.\n- This paper proposes a new paradigm (Equation 9 and 24) of message passing through **a system of hyperbolic PDEs**. This paradigm can naturally map nodes into a solution space spanned by a set of eigenvectors of the graph, which improves the capability of learning complex graph filters, such as handling the challenging heterophilic graphs.\n\nTherefore, the motivations, modeling paradigms and properties are different. Please refer to #R3 for more details. Thanks for your suggestions. We will make more discussion in the next version.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "I read all the proofs to get the general ideas but could not follow all the details", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "The proposed idea seems novel. It generally improves existing spectral GNNs and the interpretability of message passing scheme", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengs:\n- The exposition is smooth and easy to follow\n- The proposed idea of formulating message passing as a system of hyperbolic PDEs seems novel\n- Various spectral GNNs are considered to demonstrate the practical benefits of the proposed method\n\n\nWeaknesses:\n- Experimental evaluation for a variety of graph tasks is somewhat limited\n- The proposed method only shows marginal improvements over state-of-the-art methods on the node classification task\n- The mathematical proof of Theorem 3.5 is not provided", "other_comments_or_suggestions": "Please see the questions.", "questions_for_authors": "- The proof of Theorem 3.5 is not provided or I am missing something ?\n- Why other popular graph learning tasks like graph classification or link prediction are not considered ? \n- Since the message passing scheme is formulated as a system of hyperbolic PDEs, I am wondering if the proposed model is advantagous over existing ones on graph datasets with strong hierarchical structures (e.g., Disease, Airport) ? If that is not the case, could the authors elaborate ?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cfHSCTWQZ8", "reviewer": "Reviewer_TsKB", "summary": "This paper proposes a Hyperbolic Graph Neural Network (GNN) framework based on a system of hyperbolic partial differential equations (PDEs), establishing a novel message-passing paradigm that derives topology-aware node representations by solving these equations. Supported by a solid theoretical foundation and comprehensive empirical validation, the method demonstrates both effectiveness and flexibility. This work makes a significant contribution to redefining GNN architectures and advancing the field.\n\nThanks to the author for the reply, this solved most of my problems, I will keep my score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[Cons 1]**: *Slightly lower performance on some datasets.*\n\n**[Answer]**: In this paper, we aim to propose a general framework for GNNs, with the advantage of being applicable to spectral GNNs. The hyperbolic PDE-based paradigm allows the model to learn complex graph filters, generating better results on heterophilic graphs. Extensive results demonstrate the superiority, and reflect a new research direction of GNNs. We leave further improvements in future works. \n\n**[Cons 2/Q3]**: *Complexity and parameter efficiency.*\n\n**[Answer]**: Converting spectral GNNs into a dynamical system may increase the the computational costs, but can also be controlled by hyperparameters (termination time $T$ and time step $\\tau$). For instance, when $T=1$ and $\\tau=1$, the efficiency is comparable to the original method while performing better. Furthermore, this method introduces no additional parameters apart from the MLP parameters for initializing $\\varphi(\\cdot)$. Therefore, the cost and efficiency is acceptable and worthwhile.\n\n**[Q1]**: *The theoretical distinctions between different polynomial families.*\n\n**[Answer]**: Different polynomials, such as Chebyshev and Bernstein polynomials, would fit filters **based on their mathematical properties**. For example, Chebyshev polynomials exhibit Runge's phenomenon, while Bernstein polynomials can fit any function. Experimental results indicate that in graph tasks, learnable GPR performs the best, whereas ChebNetII may excel in image tasks.\n\n**[Q2]**: *Applicability of hyperbolic framework on non-spectral GNNs.*\n\n**[Answer]**: Our method is applicable to non-spectral GNNs. We can simply replace the function $P(\\cdot)$ in Equation 17 with any feasible convolution operation such as attention mechanisms, as demonstrated in our enhancement of GAT on image tasks. The effectiveness of this framework depends on whether the base model can effectively adapt to the task. Attention mechanisms can effectively capture the importance levels between pixels, thus Hyperbolic-GAT further improves the performance of GAT.", "claims_and_evidence": "The authors provide thorough theoretical proofs and experimental evidence to support their claims.", "methods_and_evaluation_criteria": "Extensive experiments on node classification and image signal filtering tasks validate the effectiveness and robustness.", "theoretical_claims": "The appendix includes informative theoretical proofs. The existence of solutions to the hyperbolic PDE system is formally established (Appendix A.2), and the simplified solution space linking node features to topology is derived and analyzed (Appendix A.3). The mathematical foundations are robust and align with established principles.", "experimental_designs_or_analyses": "The experiments on node classification and image filtering tasks are well-designed and adhere to established practices in spectral GNN research. The selection of datasets, baselines, and evaluation protocols is appropriate, ensuring reliability and reproducibility.", "relation_to_broader_scientific_literature": "This work leverages mathematical-physical methods to address AI challenges. While existing graph studies often employ diffusion equations empirically, this paper particularly indicates the effectiveness of PDE-based approaches through topology-aware solution spaces, providing theoretical improvements to existing works.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n1.\tNovelty: The framework introduces a principled connection between node representations and topology via hyperbolic PDEs, providing a theoretical and novel message-passing paradigm.\n2.\tTheoretical Interpretability: The derivation of solution spaces and their decomposition into topology-dependent components is mathematically sound and clearly articulated.\n3.\tSignificance: Beyond proposing a practical GNN, the work provides theoretical insights into PDE-based methods, facilitating deeper understanding of related approaches.\n\nWeaknesses:\n1.\tSome of results on the node classification task remains slightly below state-of-the-art benchmarks (in Table 3). \n2.\tA limited discussion of complexity and parameter efficiency may hinder practical performance. It would be better to conduct more analysis.", "other_comments_or_suggestions": "The theoretical derivations (e.g., Appendices A.2–A.3) enhance interdisciplinary accessibility. While the framework excels in spectral GNNs, its applicability to non-spectral architectures (e.g., spatial GNNs) warrants further exploration.", "questions_for_authors": "1.\tWhat are the practical and theoretical distinctions between different polynomial families (e.g., Chebyshev vs. Bernstein) in approximating the solution space? Which family yields optimal empirical results?\n2.\tCould the hyperbolic framework be adapted to non-spectral GNNs (e.g., attention-based models)? What challenges might arise?\n3.\tThe framework introduces additional hyperparameters (e.g., polynomial order). How should practitioners balance expressivity and computational overhead?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1U137QNK0Y", "reviewer": "Reviewer_fiaR", "summary": "This paper introduces \"Hyperbolic GNNs\" (Hyperbolic Graph Neural Networks), a type of spectral graph neural network where the message passing is implemented through hyperbolic partial differential equations. The network can thus be viewed as a kind of dynamical system. The authors provide several experiments demonstrating the superior performance of the hyperbolic GNNs compared to other spectral graph networks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[Cons]**: *The question about the concept of hyperbolic PDE.*\n\n**[Answer]**: We appreciate your theoretical endorsement for our approach. As you pointed out, the hyperbolic PDE in this paper differs fundamentally from the hyperbolic geometry mentioned in the literature [1,2].\n\nThe focus of hyperbolic PDEs lies in constructing the **message passing process** of GNNs ($i.e.$, $\\frac{\\partial^2 \\mathbf{X}}{\\partial t^2} = a^2 \\widehat{\\mathbf{L}} \\mathbf{X}$) based on the theory of partial differential equations. This design can naturally map nodes into a solution space spanned by a set of eigenvectors of the graph, which improves the capability of learning complex graph filters, such as handling the challenging heterophilic graphs.\n\nOn the other hand, hyperbolic geometry focuses more on the **hierarchical structure** of graphs. It typically involves mapping nodes to hyperbolic space using models such as the Poincaré Ball Model ($i.e.$, $d(\\mathbf{x}, \\mathbf{y})=\\mathrm{arcosh}(1+2\\frac{||\\mathbf{x}-\\mathbf{y}||^2}{(1-||\\mathbf{x}||^2)(1-||\\mathbf{y}||^2)})$) or the Lorentz Model ($i.e.$, $<\\mathbf{x}, \\mathbf{y}>=-x_0y_0+\\sum^n_{i=1}x_ny_n$) to extract hierarchical relationships between nodes.\n\nThe essence of the two concepts mentioned above is fundamentally different, with their focus being distinct. To avoid ambiguity for the readers, we will incorporate your valuable suggestions and modify the title as well as the method names in the next version. We will also discuss the distinctions from hyperbolic geometry in related works to avoiding misunderstanding. \n\nBelow are our new title and method name for modification:\n***\"Hyperbolic-PDE GNN: Spectral Graph Neural Networks in the Perspective of A System of Hyperbolic Partial Differential Equations\"***\n\n[1] Qi Liu, Maximilian Nickel, Douwe Kiela: Hyperbolic Graph Neural Networks. NeurIPS 2019.\n[2] Ines Chami, Zhitao Ying, Christopher Ré, Jure Leskovec: Hyperbolic Graph Convolutional Neural Networks. NeurIPS 2019.", "claims_and_evidence": "Yes, it is my opinion that the claims are backed up by both theoretical and experimental evidence.", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "I have checked parts of the proofs of the Theorems and they seem logically sound and correct to me.", "experimental_designs_or_analyses": "No", "relation_to_broader_scientific_literature": "See below", "essential_references_not_discussed": "Here comes my main criticism of this paper. The paper introduces a type of network that they refer to as \"Hyperbolic GNN\". But the hyperbolic graph neural networks already exist in the literature. As far as I know they were first introduced in the paper\n\n\"Hyperbolic Graph Neural Networks\" by Liu, Nickel, Kiela\n(Neurips 2019), arXiv:1910.12892\n\nAfter reading the paper under review as well as the paper by Liu et al, I have come to the conclusion that these papers are not referring to the same kind of networks at al. As mentioned above, in the paper under review the term \"hyperbolic\" enters because the message passing is implemented using hyperbolic PDEs. However, in the paper by Liu et al, the term \"hyperbolic\" refers to the underlying hyperbolic geometry, i.e. the signature of the manifold on which the network is defined. \nAccording to me understanding these are fundamentally different.\n\nMoreover, there is also the paper\n\n\"Hyperbolic Graph Convolutional Neural Networks\" by Chami et al\n(Neurips 2019)\n\nwhich introduces a similar structure for convolutional networks. \n\nSo, first of all, I think it is very strange that the authors of the present paper would introduce the name \"Hyperbolic GNNs\" without any type of mentioning or comparison with the previous papers on hyperbolic graph neural networks. \n\nIf indeed my understanding is correct, and these are inherently different structures, then I think it would be appropriate for the authors of the present paper to change the name of their networks, and the title of their paper. The name \"hyperbolic graph neural networks\" is taken and they have to come up with something else. \n\nSo until this issue has been resolved I cannot recommend publication of the present paper.\n\nUpdate after rebuttal: I approve of the authors changes and I will modify my overall assessment accordingly.", "other_strengths_and_weaknesses": "See my comments above.", "other_comments_or_suggestions": "See above", "questions_for_authors": "NA", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gEnqzbnJWY", "reviewer": "Reviewer_4yEq", "summary": "The paper introduces Hyperbolic GNN, a novel method that models a graph as a system of differential equations by leveraging the good properties of hyperbolic differential equations. It incorporates the topological structure characteristics of the graph into the message passing by modeling the differential equations, and provides a complete theoretical proof for the solution space of the constructed system. To address the computational complexity caused by Laplacian, polynomial theory is introduced to improve the original Laplacian and enhance the nonlinearity of the filter. Compared to state-of-the-art methods, the approach exhibits excellent performance on a large number of graph datasets and image datasets. This paper demonstrates a well-organized logical structure and rigorous theoretical proofs, making a significant contribution to the theoretical research on GNN within the paradigm of differential equations.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[Cons 1/Q1]**: *Computational costs under different modules.*\n\n**[Answer]**: The efficiency of the proposed method primarily depends on the choice of the spectral GNN filter. A more efficient filter leads to higher efficiency. Additionally, hyperparameters ($i.e.$, termination time $T$ and time step $\\tau$) also affect the efficiency of the model. A smaller $T/\\tau$ results in fewer iteration steps and higher efficiency.\n\n**[Cons 2]**: *The impact of different hyperbolic equations.*\n\n**[Answer]**: In essence, this work proposes a spectral GNN framework applicable to a broad class of hyperbolic PDEs. The specific GNN architecture for particular equations can be derived following this framework. In dynamical system modeling, second-order hyperbolic equations are most commonly encountered, characterized by real-valued eigenvalues and wave-like solution behavior. \n\nThe computational performance of the constructed GNN primarily depends on the numerical methods employed. The explicit scheme induced by the forward Euler method, when combined with initial conditions, enables rapid iterative solving, whereas implicit backward schemes may result in slower computation speeds. Different hyperbolic equations may exhibit variations in numerical stability. We will further investigate potential subtle distinctions arising from varying forms of hyperbolic equations in future works.\n\n**[Cons 3]**: *The implicit method may work better.*\n\n**[Answer]**: The advantage of the forward Euler method lies in its **simplicity and efficiency**. The implicit method offers better performance but involves solving linear equation systems, incurring higher costs. Literature [1] demonstrates that the performance improvement brought by the implicit method is small, yet it triggers significant performance costs.\n\n[1] Ben Chamberlain, James Rowbottom, Maria I. Gorinova, Michael M. Bronstein, Stefan Webb, Emanuele Rossi: GRAND: Graph Neural Diffusion. ICML 2021: 1407-1418.\n\n**[Q2]**: *The effect of different polynomials.*\n\n**[Answer]**: As shown in Tables 4 and 5, polynomials such as Bernstein and Jacobi can fit arbitrary filters, making them superior GNNs. Chebyshev polynomials exhibit the Runge phenomenon, leading to the fitting of suboptimal filters. Since our method enhances the capability of polynomial fitting for filters, it results in significant performance improvements for Chebyshev polynomials.\n\n**[Sug]**: *Improvement of writing.*\n\n**[Answer]**: We will consider and incorporate your suggestions into the revised version. Thanks for your comments.", "claims_and_evidence": "This paper presents the whole process of the proposed Hyperbolic GNN through a specific hyperbolic equation. The graph is modeled as a system of differential equations, and the coefficient matrix is completely determined by the topological structure of the graph itself. Based on the ordinary differential equation theory, the existence and spatial structure of the solution of the system are completely proved, and the original Laplacian of the system is approximated by the polynomial theory.", "methods_and_evaluation_criteria": "The proposed approach models the graph as a system of differential equations. The coefficient matrix of this system depends entirely on the graph's properties, and then induces the corresponding solution space. Based on the solution space, the method offers a clear description of node feature changes or feature directions during message passing, significantly enhancing the interpretability of graph neural networks. Compared to traditional GNNs, it offers a more accurate depiction of graph node embedding spaces during message passing. Additionally, the incorporation of polynomial approximation enhances flexibility, reduces computational complexity, and ensures the provision of more adaptive nonlinear filters in practical applications.", "theoretical_claims": "In this paper, Theorems 3.1 and 3.3 are the core parts to prove the existence of solutions and the structure of the solution space. For the process of proof, the key is to transform the original partial differential equations into a system of homogeneous linear ordinary differential equations with constant coefficients by variable substitution, and then utilize the solution theory of ordinary differential equations. The proof is relatively clear and rigorous.", "experimental_designs_or_analyses": "The experiments are mostly convincible, which mainly consist of three parts: \n1. The performance of Hyperbolic GNN is compared with common spectral graph networks on graph datasets. \n2. Based on hyperbolic partial differential equations, it is analyzed that the base enhances the performance of traditional spectral GNNs, and the effectiveness of polynomial approximation is verified. \n3. Interestingly, the designed Hyperbolic GNN is applied to image filtering experiment, and the effect is clear. The results also seem to verify that the polynomial approximation can achieve more flexible filters.", "relation_to_broader_scientific_literature": "Compared to traditional spectral GNNs, this paper proposes a PDE-based model to achieve message passing mechanism. Besides, compared with the current differential equation-based paradigm, it provides better scalability and interpretability.", "essential_references_not_discussed": "The work highly related to this paper is mainly spectral GNNs based on the differential equation (ODEs, PDEs) paradigm. The necessary articles have been cited in this paper.", "other_strengths_and_weaknesses": "Strengths: \n- This paper is mostly clear with a well-organized structure and mathematical proofs.\n- The proposed method models the graph as a system of hyperbolic partial differential equations, constructing an embedding variation space for graph nodes. This effectively captures the direction of node feature changes or significant feature directions during message passing, offering enhanced scalability and interpretability.\n- The designed approach incorporates polynomial approximation into the Laplacian within the framework of differential equations, achieving significantly enhanced flexibility.\n- The experimental results across diverse graph datasets demonstrate the effectiveness of the proposed method. Furthermore, extensive experiments conducted on image datasets confirm its operational flexibility and practical applicability.\n\nWeaknesses:\n- The possible computational costs under different modules are not analyzed.\n- The performance of the constructed GNN may vary across different hyperbolic equations.\n- Although the forward Euler method is simple, the implicit method may work better.", "other_comments_or_suggestions": "Please see pros & cons. Besides, there are suggestions:\n- In Table 1, the unit basis vectors in Euclidean space are typically formatted as bold italics (e.g., $\\bm{e}_i$). The current writing may require further consideration.\n- In Appendix A.2, the notation C in Equation (27) of Theorem A.2 denotes the equation system's coefficient matrix. This differs mathematically from the C in Equation (28), suggesting appropriate notational distinction.", "questions_for_authors": "- Considering the computational cost of the different components of the analysis approach may further strengthen the paper's point of view.\n- Will different polynomials affect the performance of the proposed GNN?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces Hyperbolic GNNs, a type of spectral graph neural network where the message passing is implemented through hyperbolic partial differential equations. Several theoretical advantages of hyperbolic dynamic systems are highlighted to be beneficial in the scope of GNNs. Some nice experimental results are also given to support the proposed method. Here, all reviewers support the publication of the work. In the discussion period, the reviewers agree that the method is not entirely novel, as hyperbolic GNNs were already suggested in the papers (Eliasof et al., 2021) and (Rusch et al., 2022). In light of the positive reviews, and after discussion, I vote for the acceptance of the paper. In the camera-ready version, the authors should revise their paper to give (Eliasof et al., 2021) and (Rusch et al., 2022) credit for being the first to propose hyperbolic-type GNN architectures.
Decision: Accept (poster)