Spiking Graph Neural Network on Riemannian Manifolds
Authors: Li Sun
Zhenhao Huang
Qiqi Wan
Hao Peng
Philip S. Yu
Venue: NeurIPS 2024 poster (2024)
Keywords: Graph Neural Network
Spiking Neural Network
Riemannian Geometry
Primary area: graph_neural_networks
TL;DR: We propose a new spiking neuron on Riemannian manifold so that the high-latency BPTT algorithm is replaced by our Differentiation via Manifold.

Abstract: Graph neural networks (GNNs) have become the dominant solution for learning on graphs, the typical non-Euclidean structures. Conventional GNNs, constructed with the Artificial Neuron Network (ANN), have achieved impressive performance at the cost of high computation and energy consumption. In parallel, spiking GNNs with brain-like spiking neurons are drawing increasing research attention owing to the energy efficiency. So far, existing spiking GNNs consider graphs in Euclidean space, ignoring the structural geometry, and suffer from the high latency issue due to Back-Propagation-Through-Time (BPTT) with the surrogate gradient. In light of the aforementioned issues, we are devoted to exploring spiking GNN on Riemannian manifolds, and present a Manifold-valued Spiking GNN (MSG). In particular, we design a new spiking neuron on geodesically complete manifolds with the diffeomorphism, so that BPTT regarding the spikes is replaced by the proposed differentiation via manifold. Theoretically, we show that MSG  approximates a solver of the manifold ordinary differential equation. Extensive experiments on common graphs show the proposed MSG achieves superior performance to previous spiking GNNs and energy efficiency to conventional GNNs.

Spiking Graph Neural Network on Riemannian Manifolds
Li Sun'
Zhenhao Huang
North China Electric Power University Beijing 102206, China ccesunli @ncepu. edu. cn
North China Electric Power University Beijing 102206, China
huangzhenhao@necpu. edu. cn
Qiqi Wan
Hao Peng
North China Electric Power University Beijing 102206, China wanqiqi@ncepu. edu. cn
Beihang University
Beijing 100191, China penghao@buaa edu. cn
Philip S. Yu
University of Illinois Chicago
IL, USA psyu@uic.
Abstract
Graph neural networks (GNNs) have become the dominant solution for learning graphs, the typical non-Fuclidean structures. Conventional GNNs, constructed with the Artificial Neuron Network (ANN), have achieved impressive performance at the cost of high computation and energy consumption. In parallel, spiking GNNs with brain-like spiking neurons are drawing increasing research attention owing to the energy efficiency. So far, existing spiking GNNs consider graphs in Euclidean space, ignoring the structural geometry, and suffer from the high latency issue due to Back- -riok--orpaaaioonTTa Tarooonn--hhhhh-TTT light of the aforementioned issues, we are devoted to exploring spiking GNN on Riemannian manifolds, and present a Manifold-valle
(BPTT) with the surrogate gradient. In
Spiking GNN (MSG). particular, we design a new spiking neuron on geodesically complete manifolds with the diffeoomrphism, so that BPTT regarding the spikes is replaced by the pro- posed differentiation via manifold. Theoretically, we show that MSG approximates solver of the manifold ordinary differential equation. Extensive experiments on common graphs show the proposed MSG achieves superior performance to previous spiking GNNs and energy efficiency to conventional GNNs.
Introduction
Graphs are the ubiquitous, non- non-Euclidean structures that describe the relationship among objects. Graph neural networks (GNNs), constructed with the floating-point Artificial Neuron Network (ANN), have achieved state-of-thh-arr accuracy for learning on graphs I
However, they raise the concerns about computation and energy consumption, particularly when dealing with real-world graphs of considerable scale (5) In contrast, Spiking Neuron Networks (SNNs), inspired by the biological mechanism of brains, utilize neurons that communicate using sparse and discrete spikes, showcasing their superiority in energy efficiency 87 Attempting to bring the best of both worlds, spiking GNNs are drawing increasing research attention.
2 3 4
Corresponding Author: Li Sun, ccesunli@ncepu, www.ennccppcuu....mmmmm edu.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

In the literature of spiking GNNs, recent efforts have been made to design different architectures with spiking neurons, e.g., graph convolution [5], attention mechanism 9.. variational autoencoder [10] and continuous GNN [I. While achieving encouraging results, existing spiking GNNs still face several fundamental issues: (1) Representation Space. Spiking GNNs consider the graph in Euclidean space, ignoring the inherent geometry of graph structures. Unlike the Euclidean structures (e.g., pixel matrix and grid structures), graphs cannot be embedded Euclidean space with bounded distortion [12]. Instead, Riemannian manifolds have been shown the promising spaces to model graphs in recent years [3 4 (e.g., hyperbolic space, a type of Riemannian manifolds, is well aligned with the graphs dominated by hierarchical structures). However, none of the existing works study SNN on Riemannian manifolds, to the best of our knowledge. It is thus an interesting and urgent problem to consider how to endow the spiking GNN with a Riemannian manifold. (2) Training Algorithm. Training spiking GNN is challenging, since the spikes are non-differenniaale. Existing studies consider the spiking GNN as recurrent neural network and apply Backward- Passing- Through-Tiin Time (BPTT) with the surrogate gradient [5] 9 10: 11]. They recurrently compute the backward gradient at each time step, and thus suffer from the high latency issue [14] 15. 16 especially when the spike trains are long.
Present work. Deviating from pre- vious spiking GNNs in Euclidean space, in this paper, open new direction to explore spiking GNNs on Riemannian manifolds, and propose novel Manifold-valued Spiking GNN (MSG) sketched in Fig. [ istic to place spike trains in a manifold such hyperbolic or hyperspherical space, given the fact that spike trains cannot align with the defining domain. Instead, we design a Manifold Spik- ing Layer that conducts parallel for- warding of spike trains and manifold representations. Specifically, we first incorporate the structural information into spike trains by graph convolution. Then, a new manifold spiking neuron is proposed to emit spike trains and relate them to manifold representations with diffeomorphism, where the spike train generates a momentum that forwards manifold representation along the geodesic. Instead of applying BPTT in spike domain, the proposed neuron provides us with an alternative of Disferentiation via Manifold (DvM). (The red dashed line in Fig. I Yet, differentiation in Riemannian manifold is nontrivial. We leverage the properties of pullback and derive the closed-fome backward gradient (Theorem 4. DvM enables the recurrence- free gradient backpropagation, which no longer needs to perform recurrent computation of time steps in BPTT. Theoretically, MSG is essentially related to manifold Ordinary Differential Equation (ODE). Each layer creates a chart of the manifold, and MSG approximates the dynamic chart solver 17 of manifold ODE (Theorem 5.2).
9
1-th MS Layer
T- th MS Layer
L-th MS Layer
It is not real-
Parward Dariovent
Palls
x E.M
2' e' M
Figure 1: MSG conducts parallel forwarding and enables new training algorithm alleviating the high latency issue.
Contributions. Overall, the key contributions are summarized as follows: (1) To the best of our knowledge, we propose the first spiking neural network on Riemannian manifolds (MSG F and show its connection to manifold ODE theoretically. (2) We design a new training algorithm of differentiation via manifold, which avoids the high latency of BPTT methods. (3) Extensive experiments show the superior effectiveness and energy efficiency of the proposed MSG.
2 Related Work
We briefly overview the ANN-based GNNs (i.c.. the conventional. floating GNNs living in either Fuclidean space or Riemannian manifolds) and baN-based GNNs (i.e., spiking GNNs).
ANN-based GNNs (Euclidean and Riemannian). The majority of GNNs are built with floating-
point ANN, conducting message passing on the graphs [8 2 19] The Euclidean space has been the
Codes are available at httpts
com/ http://ttuubcccm////hhnalaaagg

---

workhorse for graph representation learning for decades, and the popular GCN 18 GAT 2 and SGC [9] are also designed in the Euclidean space. In recent years, Riemannian manifolds have emerged as an exciting alternative considering the geometry of graph structures 20 21 Among Riemannian manifolds, hyperholic space is recognized for its alignment with the graphs of hierarchical structures, and a series of hyperbolic GNNs (e.g., HGNN (22). HGCN [31 show superior performance to their Euclidean versions. Beyond hyperbolic space, hyperspherical space is well suited for cyclical structures [23], and recent studies further investigate the constant curvature spaces [3], product 25 26 27], quotient spaces [28] SPD manifolds [29] 30), etc. Riemannian manifolds
spaces 24
achieve remarkable success in graph clustering B1 32], structural learning [33, graph dynamics
34 35 36 37], information diffusion (88 and graph generation [39 but have rarely been
touched yet in the SNN counterpart.
Spiking Neural Networks (SNNs) & Spiking GNNs. Mimicking the biological neural networks, SNNs [7] 8 utilize the spiking neuron to process spike trains, and offer the advantage of energy efficiency. Despite the wide application of SNN in computer vision [41 42). SNNs are still at an early stage in the graph domain. The basic idea of spiking GNNs is adapting ANN-based GNNs to the SNN framework by substituting the activation functions with spiking neurons. Pioneering works study the graph convolution [43 51, and efforts have also been made to the graph attention [9] variational graph autoencoder [10], graph differential equations [44], etc. SpikeGCL [6] is a recent endeavor to conduct graph contrastive learning with SNN. In parallel, spiking GNNs are extended to model the dynamic graphs [45] 46 47.. We focus on the static graph in this work. In both dynamic and static cases, previous spiking GNNs are trained with the surrogate gradient, leading to high latency, and consider the graphs in the Euclidean space.
3 Preliminaries
Different from aforementioned spiking GNNs, we study the spiking GNN on Riemannian manifolds. Thus, we formally introduce the basic concepts of Riemannian geometry and SNN. Throughout this paper, the lowercase boldfaced æ and uppercase X denote vector and matrix, respectively. Important notations are summarized in Appendix A
Riemannian Geometry & Riemannian Manifold. Riemannian geometry provides elegant frame- work to study structures and manifolds. A Riemannian manifold is described as a smooth and real manifold M endowed with Riemannian metrie. Each point in the manifold associated with the tangent space M that "looks Euclidean", and the Riemannian metric is given by the inner product in the tangent space, so that geometric properties (e.g, angle, length) can be defined. A geodesic be- tween two points on the manifold is the smooth path connecting them with the minimal length. There exist three types of isotropic manifold, namely, the Constant Curvature Space (CCS): hyperbolic space H. hyperspherical space S and the special case of Euclidean space with "flat" geometry E.
Graph & Riemannian Graph Representation Learning. A graph g on the node set V and edge set & C V x V, and A E R|VXX|V is the adjacency matrix describing the structure information. Each node U; is associated with a feature fü, and node features are summarized in F E R In this paper, we resolve the problem of Riemannian Graph Representation Learning with SNN. Specifically, we seek graph encoder Fo H where E M is point on the manifold, instead of Fuclidean space, and Fg is defined with an energy-efficient SNN.
A) is defined
Spiking Neural Network. SNNs are constructed by spiking neurons that communicate with each other by spike trains. Concretely, a spiking neuron is conceptualized as "a capacitor of the membrane potential", and processes the spike trains by the following 3 phases [48]. First, the incoming current [[t] is accumulated in the capacitor, leading to the potential buildup (integrate). When the membrane potential [i( reaches or exceeds a specific threshold ViA, the neuron emits spike (fire). After that, the membrane potential is reset to the resting potential Vreset (reset). There are two popular spiking neurons: IF model and LIF model [49]. In particular, the three phases of IF model are formalized as
Integrate: V[t] 9 (V 1], 7(t) = Vit 1] + 7[t]
Fire: S(t] = HV[(t] - Vin.)
(2)
[(1 - ((())[(++ + Viees
Fixed-reset,
Reset: : V[t]
{ [[((t - Stt..
(3)
Subtraction-reset.
3

---

where the incoming current [[t] is related to the input spike train, and Vreset is lower than Vin- t denotes the time index of the spike. The Heaviside function H-) is non-differentiable, H 1 if x 2 0, and O otherwise. There are two options for reset, and fixed-reset is adopted in this paper. Overall, an IF model is given as S't] TF [FModee and the only difference between IF model and LIF model lies in the definition of g(-) in Eq. 1 In this paper, we are interested in designing new spiking neuron on Riemannian manifold.
4 Methodology: Manifold-valued Spiking GNN
In this section, we present simple yet effective Manifold-valued Spiking GNN (MSG), which can be applied to any geodesically complete manifolds, e.g., the Constant Curvature Space (CCS), including hyperbolic space and hyperspherical space, or the product of CCS. In particular, we design spiking neuron on Riemannian manifolds (named Manifold Spiking Neuron) that allows for the differentiation via manifold. It provides new perspective of training spiking GNN, so that we avoid the high latency of typical backward passsag-throuughhe (BPTT) training.
4.1 Manifold Spiking Layer
We elaborate on the sole building block of the proposed model Manifold Spiking Layer. Note that, the spike train or spiking represen- tation in existing spiking GNNs 43 g T0
6 45
46; 47] cannot align with the defining do- main of Riemannian manifolds (e.g., hyperbolic space and hyperspherical space), thus posing fundamental challenge. Our solution is to gen- erate node representation on the manifold (re- ferred to as manifold representation) in parallel, and leverage the notion of Diffeomorphism to create the alignment between the two domains. We formulate the parallel forwarding of spike trains and manifold representations as follows.
Fowwadd Backward
GCN
de
(z-((1)
1-1
aw
IFModel
Pooling
f(.
D ($
P2(C)
Unified Formulation. The forward pass of the spiking layer consists of a graph convolu- tion and one proposed manifold spiking neuron. Without loss of generality, for each node v; E G. the (-th spiking layer is formulated as follows,
Figure 2: Manifold Spiking Layer. It conducts par- allel forwarding of spike trains and manifold rep- resentations, and creates an alternative backward pass (red dashed line). The backward gradient with By/-1
x 1 GCN(s) [4]; W
(4)
S 2 MSNeuron ((-1) 2 (111)),
(5)
where x is the incoming current generate spike trains. s and z denote the spike trains and manifold representation, respectively. GCN(-) is GNN in the Euclidean space, and W is the learnable parameter in the layer. Different from the neuron of previous spiking GNNs, design a novel manifold spiking neuron (MSNeuron) as shown in Fig. 2 It emits spike trains and relates them to manifold representations simultaneously, which formulated follows,
De: 1 and Vxd C will be introduced in
PWT
Sec.
4.2 2
s = IFMMdel( (-1 [1]] [1]]
(6)
v I Pooling x (
(7)
z = f(z -v/-,///=-)) M
(8)
where is the time step of spike trains. The IF model can replaced by LIF model, and we utilize IF model by default for simplicity. Pooling is defined as the average pooling of the current over I, and v is given as the Fuclidean vector. denotes the diffeomorphism to the Riemannian manifold in which is the step size.

---

Incorporating structural information. We inject the structural information when the received spikes transform into the incoming current of the neuron. A GNN is leveraged to define the current, conducting passage-passing over the graph. Each node's is representation is derived recursively by neighborhood aggregation [18 2 I. Accordingly, GCN in the proposed neuron is given as follows,
(9) where the neighborhood Ω2 is the set of immediate neighbors centering at node e4. The aggregate function aggregates the messages from neighborhood f2, where we create the message of node by W's, [4] combine(.) denotes the combination of the center node's message and aggregated message. We utilize GCN 18 as the backbone define aggregate and combine.
GCN(s (t):: w') = combine(s; [t]] (eggregate((sstt j E 94 W
Diffeomorphism between manifolds. In the proposed neuron, we bridge the spikes and manifold representation with the notion of diffeomorphism in differential geometry. A diffeomorphism connects two smooth manifolds, saying M and N. Formally, a map f M + N is a diffeomorphism between M and N if the smooth f' is bijective and its inverse f-1 is also smooth.
Recall that the tangent space locally Euclidean. We propose to place the Euclidean v, a representa- tion of the spikes, in the tangent space T&M of the point z. In MSG, we choose the exponential map to act as the diffeomorphism between the tangent space and manifold. With a step size e, we have
f(z) (1)( .ev, (e) = Exp_(-n (ev((-))) E EM
(10)
Concretely, given z E M and v E Mr.,, the exponential D
of v at point z, Expz(v): T&M -
M. maps tangent vector onto the manifold M. The map pushes along the geodesic (1) [0, + M starting at (0) = z and ending at y = (1.) (t) denotes the velocity of (t), and the direction of geodesic at the beginning is given (0) v. That is, the tangent vector v. derived from the spikes, pushes the manifold representation along the geodesic via the exponential map. The advantage of our choice is that we are able to define the diffeomorphism in arbitrary geodesically complete manifold (detailed in Appendix D
Note that, our idea is inherently different from the exponential/lgaritmmi based Riemannian GNNs 3 4 (2), which leverage the tangent space of the origin for neighborhood aggregation. In contrast, we consider the successive process over the tangent spaces of manifold representations, which will be further studied in Sec. 5.
Model Initialization In MSG, we need to simultaneously initialize the spiking input S and manifold representation Z", which is a collection of points on the given manifold. Given graph G(V E, A, the node features are first encoded by one graph convolution layer H = GCN A, F; W and we generate T copies of the node encodings H. where T is the number of time steps spike trains. Then, we complete model initialization with the proposed manifold neuron [S z MSNeuron (H, O), where the encoding H is regarded as the incoming current that charges the neuron in each time step. O consists of the original points of the manifold, e.g., in the sphere model of hyperspherical space, the original point is given as the south pole o = 1,,, ..... 0) and = [o ...,.., '1" Note that, the exponential map in the proposed neuron guarantees that z lives in the manifold.
4.2 Learning Approach: Differentiation via Manifold
Optimizing SNNs is challenging, as the Heaviside step function is non-differentiable. In the literature, existing spiking GNNs typically regard SNN as the recurrent neural network, and leverage backward- bussing-through (BPTT) to train the model [50] 5T [5.. Coneretely, given real loss function C, the gradient backpropagation conducts Differentiation via Spikes (DvS) s as follows,
1 0s' owi
C >
VolgC
(11)
where Wi is the parameter, and denotes the time step. The surrogate gradient [51] is required for Dwis' [f], where Heaviside step function is replaced by differentiable surrogate, e.g., sigmoid function. The differentiation via spikes presents high latency in the backward pass 14 15 I6]. as it needs to recur all the time steps in BPTT. We notice that, in the computer vision domain, the sub- -abdgradient method [15] is proposed to address such issues in Euclidean space. However, it cannot be generalized to the Riemannian manifold since the linearity does not hold in Riemannian geometry.
'Te inverse map from M to T is the logarithmic map.
5

---

In MSG, we decouple the forward pass and backward pass, and propose Differentiation via Manifold (DuM) to avoid the high latency in differentiation via spikes. The overall pro- cedure of training MSG by the proposed learn- ing approach summarized in Algorithm|| [ Thanks to the parallel forwarding of spikes and manifold representation, the proposed neu- ron provides with alternative of studying Ww through the forwarding pass on the manifold (i.e., differentiation via manifold). Nevertheless, it is nontrivial and it requires to derive the pullback between different dual spaces.
Algorithm Training MSG by the proposed Differ- entiation via Manifold
Input: Graph 9(V, F,F, A),, Manifold M, Loss function over the manifold L()) Number of spiking layers L, Original points O. Parameters {W") 11=
Output: I: while not converging do
D forward pass
3: Input current X GCN(A, F; W Initialize [S - MS X OO;;
5:
for each spiking layer / = 1 to I. do
6:
((-1) = GCN(A, ((1-1), W
SS z MSNeuron (X (111) ((-1));
7:
end for
Pushforward, Pullback and Dual Space.
9:
> backward pass
10: Compute VgcL from C(Z²)).
We first introduce the differentiation in Rie- mannian geometry which is essentially dif- ferent from that in Euclidean space. In Rie- mannian geometry, a pushforward refers to a derivative of map connecting two manifolds M and N. Concretely, given f M + N and a point E M, the pushforward Dzf maps a tangent vector v E T.M to the tan- gent vector (v)(v) E N. On the nota- tion, for manifold-valle p E N. 0p/Oz is equivalent to f. For a scalar function f. Dzf is interchangeable with Vzff.
11:
for layer 1 - T. 1 to do
Compute Dzres D dv Compute V x C, Vw L as Eq. 12
12:
13:
14:
Update W
15:
end for
16: end while
function f(z)
In the proposed MSG, we consider scalar loss function on the manifold L : + R. The pushfor- ward D&C at point z E M maps tangent vector v E T..M to scalar value and, correspondingly. D&C belongs to the dual space of the tangent space TM.M which is a vector space consisting all linear functional F T..M > R. As the tangent spaces at different points of the manifold are different, it requires pullback that maps the dual space T.MM M to the dual space T M.
We derive the backward gradient with properties of differential form (Lemma B.T communication (Lemma B.2). and pullback of a sum and a product (Lemma B.3) detailed in Appendix B.I]
Theorem 4.1 (Backward Gradient). Let L be the scalar-valued function, and z' is the output of 1-th layer with parameter W which delivered by tangent vector v Then, the gradient of function C W.E. W is given as follows:
V V L = -vi-1
aw
(D
(12)
where -1-(())= = Exprr- (0) (x'r)(v"), and [.]* means the matrix form of pullback.
The detailed proof is given in Appendix B. and we derive the two Jacobian matrices D and Da in Appendix C There are three key advantages of the proposed DvM. First, every term in Equation (1) is differentiable, and thereby the surrogate gradient is no longer needed. Second, DvM enables the recurrence-free backward pass alleviating the high latency training. We specify that both DvM and the previous DyS recurrently compute every time step in the forward pass, and the difference lies in the backward pass. In particular, only conduct recurrence-free gradient backpropagation layer by layer, while the previous DvS' recurs every time step of each layer in BPTT. In addition to the differentiable and recurrence-free properties, DvM does not suffer from gradient vanishing/cxplosion. and the empirical evidence is provided in Appendix F
5 Theory: MSG as Neural ODE Solver
Next, we demonstrate the theoretical aspects of our model that MSG approximates a solver of manifold Ordinary Differential Equations (ODEs).

---

We leverage the notion of chart to study the relationship between MSG and manifold ODE. A manifold ODE defined as
dz(() ()((t), z(t) E M, dt
(13)
describes a vector,fiedd u that maps a smooth path z(t) [0. - .M to the tangent bundle TM In other words, the vector field u as- signs each point z(t) M tangent vector u(z(t), t) E Tx(t)-M. A chart at z, denoted as (Ug, o2), is a smooth bijection Φz between z's neighborhood U2 C M and subspace of Eu- clidean space. Thus, the chart relates Eq. (33) ODE in Euclidean space [17] Specifically, if
Figure 3: Charts given by the logarithmic map.
y(t): [T, IR" is the solution of
dy(t) -D (00 (v(()))
(14)
dt
then y(t) = (z(())) is a valid solution of Eq. (13 on E [T, +
Definition 5.1 (Dynamic Chart Solver (17). The manifold ODE in Eq. 13 with initial condition
z(0) z can be solved with a finite collection of successive charts {(U,, ii) Ifode; is the
numerical solver Euclidean ODE corresponding the i-th chart, y() = ole,(t) on [To T3 + ei).
then zt) in Eq. 13 is given as
(Φ odez olD o $7-1) o... (02 o TT o ode; (t)
(15)
That is, manifold ODE can be solved in Euclidean subspaces given by series of successive charts.
In MSG, we consider the charts given by the logarithmic map as illustrated in 3 and we prove that MSG approximates dynamic chart solver of manifold ODE (Theorem 5.2)
Theorem 5.2 (MSG as Dynamic Chart Solver). yf) [T, T + + R" is the solution of
dy (t) (Dexp, Logg (xEx),((()))) dt
(16)
then ztt) = (y()))(t) is a valid solution to the manifold ODE of Eq. (13) on t E [T, + c), where z z(T). (t) given by the order approximation with the e small enough,
y(T + = c (D₂ (ozg))(((().))) T).
(17)
then the update process of Eqs. and in MSG is equivalent to Dynamic Chart Solver in Eq. [5
Proof. The proof utilizes some facts of Riemannian manifolds and is detailed in Appendix B.2
In other words, in MSG, the transformation of manifold input and output is described as some manifold ODE, whose vector field is governed by a spiking-related neural network in the tangent bundle. To solve the manifold ODE, MSG leverages the Dynamic Chart Solver (Definition 5.1). Specifically, each manifold spiking layer corresponds to a chart, and thus the number of spiking layers equals the number of charts. Each layer solves the ODE of a smooth path yt) : [T, 7 + cj + R in the tangent space that centered at the manifold layer input. With the first-order approximation in Theorem 5.2 given a step size c, the endpoint y(T + e) of the path is parameterized by a GNN related to the spikes. Layer-by-laye forwarding solves the manifold ODE from the current chart the successive chart. Consequently, the manifold output of MSG approximates the solution to the manifold ODE.
We notice that recent work [11 connects spiking GNN to ODE in Euclidean space. In contrast, the proposed MSG is essentially related to the manifold ODE.
The Appendix contains the proofs, the derivation of Jacobian, necessary facts on Riemannian geometry (i.e., Lorentz/Sphere model, stereographic projection and k-stereographic model, and Cartesian product and product space), empirical details and additional results.
'The tangent bundle I'M is the disjoint union of all the tangent spaces of the manifold T'M LLeM T.M
7

---

Table 1: Node Classification (NC) in terms of classification accuracy (%) and Link Prediction in terms of AUC (%) on Computers. Photo, CS and Physics datasets. The best results are boldfaced, and the runner-ups are underlined. The standard derivations are given in the subscripts.
Computers
Photo
Physies
NC
LP
NC
LP
NC
LP
NC
LP
GCN [8]
03.35
92.07=0040
86.01
88.34
91.68=044
93.68=0.84
95.03
93.46=0.39
ANNLR
GAT 2
86.82=0004
=1.01
88.45=00.7
91.74=022
94.06=0.70
95.11
98.44=0.70
SGC
12.17=125
90.46=0.50
87.91
89.84:0.40
92.09
943
94.77
95.93-0070
SAGE [T
81. 11.691003
90.56..0.4
89.86
92.71 2.77
95,62.1026 95.22=.014
95.75
HGCN 3
88, 7 00.24
28.58
89.18 18 18.
04.54.0.a0
90.72=0016
93.02=0.26
94, 94.46:10.0
94, 10=004
ANN.R R
<-GCN
13
89, 20.20=.0..
95. o0
92.22.20aa
14.89
91. 18
94.86_0.18 18
94.58_0.22
Q-GCN
4
85.94
96.98
92.50
18.18
93.39 93.39_00.0
94.84 94.84
OOM
HyboNet
S4
86.29=2.30
96.80=e.os
92.67=009
97.70=a.07
92.34:0033
95.65-0.36
95.56+0014
98.46=9.99
SpikeNel
45
08.00
92. 12.90
92. 15
92. 92.66.2030
SNN
SpikeGCN
86,90=0.30
91.12=11.7
92. 02.60=0.0
93.84
90.86=0011
95.07=1.22
94.53+0.15
92.88-0.80
GpikeGCL
o
89.0420.s
92.72
92.50
95.58
91.77=0.11
95.13=0.24
95.21
94.15=0.2
SpikeGTT [55
11.00
90.661008
91. 91,86
94.38
MSG (Ours)
89.27 00.19
94.65
93.11
96.75
92.65=004
93.19
95.93
93.43=0.16
Experiments
We conduct extensive experiments with 12 strong baselines to evaluate the proposed MSG in terms of (1) the representation effectiveness, (2) the energy efficiency, and (3) the advantages of the proposed components. Additional results are presented in Appendix F
6.1 Experimental Setups
Datasets.
Our experiments are conducted on 4 commonly used benchmark datasets including two popular co-purchase graphs: Computers and Photo [53], and two co-author graphs: CS and Physics [53]. Datasets are detailed in Appendix B
Baselines. We compare the proposed MSG with 12 strong baselines of three categories: (1) ANN-
based Euclidean GNNs: the popular GCN 18) GAT 22.. GraphSAGE ( and SGC [19, (2)
ANN-based Riemannian GNNs: HGCN [3] and HyboNet (54) of hyperbolic spaces, K- K-GCN [3] of the constant curvature space, and the recent Q GCN 4 of the quotient space. (3) The previous
Euclidean Spiking GNNs: SpikeNet [45. SpikeGCN [.. SipkeGraphormer (55] (termed as SpikeGT for short) and the recent SpikeGCL 6 Note that, we focus the graph representation learning on static graphs, and thereby graph models for the dynamic ones are out of the scope of this paper. SpikeNet [45] was originally designed for dynamic graphs, and utilize its variant for static graphs according to 6 So far, spiking GNN has not yet been connected to Riemannian manifolds, and we devoted to bridging this gap.
Evaluation Protocol. All models are evaluated by node classification and link prediction tasks. The evaluation metrics of node classification is classification accuracy; we employ the popular Area Under Curve (AUC) for link prediction. The hyperparameter setting is the same as the original papers. We perform 10 independent runs for each case, and report the mean with standard derivations. Experiments are conducted on the hardware of NVIDIA GeForce RTX 4090 GPU 24GB memory, and AMD EPYC 9654 CPU with 96-Core Processor. Our model is built upon GeoOpt (56), SpikingJelly and PyTorch [57]
56
Model Instantiation & Configuration. Note that, the proposed MSG applies to any Constant Curvature Space (CCS) or the product of CCS. We instantiate MSG in the Lorentz model of hyperbolic space by default (whose Riemannian metric, exponential map, and the derived Jacobian is given in Appendix c). and study the impact of representation space in the Ablation Study. The dimension of the representation space is set as 32. The manifold spiking neuron based on the IF model (49 by default, and it is ready to switch to the LIF model 44 whose results are given in Appendix F The
time steps T for neurons is set to 5 or 15. The step size in Eq. g is set to 0. The hyperparameters are tuned with grid search, in which the learning rate is {0.01. 0.003) for node classification and (0.003, 0.001} for link prediction, and the dropout rate is in (0.1, 0.5). We provide the source
code of MSG at the anonymous link https: / http//gtttttbttt hwtp:////eehhhuuugg///S...
8

---

6.2 Results & Discussion
Effectiveness. We evaluate the effectiveness of MSG in both node classification and link prediction tasks. Specifically, for node classification, we cannot directly feed the manifold representations of Riemannian baselines to a softmax layer with Euclidean measure. We bridge the manifold representation and Euclidean softmax with the logarithmic map of respective manifold. For link prediction, we utilize the generalized sigmoid for all the baselines, i.c., the Fermi-Dire decoder 3 in which the distance function is defined under the respective geometry. In the proposed MSG, the model inference does not need the expensive successive exponential maps, and only limited float-point operations (i.e., addition) are involved. Accordingly, we leverage the tangent vectors for the downstream tasks. The performance of both learning tasks on Computer, Photo, CS and Physics datasets are collected in Table ! since they are designed for node classification and do not offer spiking representation. The proposed MSG consistently achieves the best results among SNN-based models. In addition, MSG generally outperforms the best ANN-based baselines in node classification, and has competitive results to the recent ANN-based Riemannian baselines in link prediction.
Note that, SpikeNet and SpikeGT cannot do link prediction,
Ablation Study. Here, we examine the impact of representation space and the effectiveness of the proposed Dif- ferentiation via Manifold (DvM). For the former goal, we instantiate geo- metric variants of MSG in hyperbolic space HH hyperspherical space g32 Euclidean space 1232 and the products of ' x H H x 16 and S 1916
Table 2: Ablation study of geometric variants. Results of node classification in terms of ACC (%).
Computers
Photo
CS
Physics
H
=9.27
93.11+a.1
92.65
95.93=aum
17.84
92.03
92.72.10.66
95.85.0.0
E32
88 14.244
92,931021
92.82.10.4
95.81
H
x T6
89.18
92.061014
92.67.0.10
95.901001
g
88. 0.00
91.97+008
92.33++021
95.73=a.11
S16 g16
12.4
92.31
92.18
95.81
The superscript denotes the dimension of representation space, and we leverage DuM for optimization. Manifold variants generally achieve superior results to the Euclidean one, thus verifying our motivation. On CS dataset, the performance of geometric variants is aligned with that of Euclidean and Riemannian baselines in Table The proposed MSG is ready to switch among HI, S, E, and their products, matching the geometry of graphs.
Loweii
Spterm
ofvn
olvin
1.4
Surregviee
Surmpate
1.0
0.6
100005
1:00000
10
imc Stops
Time Stoce
(a) Backward times in model training.
(b) Gradient of regarding langent vector
Figure 4: Backward time and gradient norm for node classification on Computer.
To examine the effectiveness of DvM, we design the optimization variant (named as Surrogate) for a given representation space. In the variant, we conduct differentiation via spikes and leverage BPTT for optimization, same as previous spiking GNNs. The training time of the optimization variants in different representation spaces are given in Fig. 4aa Backward time of DvM is significantly less than that of BPTT algorithm. The reason is that DvM no longer needs recurrent gradient calculation of each time step (recurrence-free) while BPTT leads to high training time especially when the time step large. In addition, we examine the backward gradient of DvM, and plot the gradient norm of each layer in Fig. 4(b) It demonstrates that DvM does not suffer from gradient vanishing/explosion.
Energy Cost. We investigate the energy cost of the graph models in terms of theoretical energy consumption (mJ) 5 61. whose formula is specified in Appendix B We summarize the results for node classification in Ta which the number of parameters at the running time listed as a reference. It shows that baN-based models generally enjoy less energy cost than ANN-based ones.

---

Table 3: Energy cost. The number of parameters at the running time (KB) and theoretical energy consumption (mJ) on Computers, Photo, CS and Physies datasets. The best results are boldfaced, and the runner ups are underlined.
Computers
Photo
Physies
#(para.)
energy
#(para.)
energy
(para.)
energy
#(para.)
energy
GCN
18
24.91
1.671
24.14
0.893
218.29
18.444
269.48
42.842
SNN- ANN-R ANN-E
GAT
2
24.99
2.477
24.22
1.273
218.38
28.782
269.55
81.466
SGC 19
7.68
0.508
5.97
0.219
102.09
8.621
42,08
6.688
SAGE
49.77
1.671
48.23
0.893
436.53
18.444
538.92
42,842
HGCN 5
24.94
1.614
24.96
0.869
217.79
18.390
269.31
42,800
k-GCN
13
25.89
1.647
25.12
0.889
218.24
18.440
269.44
42,836
2 GCN
24.93
1.629
24.96
0.876
217.83
18.393
269.34
42.809
HybaNet
54
27.06
1.625
26.29
0.875
219.94
18.399
271.47
42.825
SpikeNet
4
101.22
0.070
98.07
0.040
438.51
0.218
540.04
0.334
SpikingGCN [5]
38.40
0,105
29.84
0.046
510.45
1.871
210.40
1.451
SpikeGCL
[6
59.26
0.121
57.85
0.067
445.69
0.128
548.
0.214
SpikeGT
55
77.07
1.090
74.46
0.584
365.28
6.985
355.77
12.524
MSG(Ours)
26.95
0.047
25.68
0.043
226.15
0.026
143.72
0.029
Note, MSG achieves the best energy efficiency among SNN-based models except Photo dataset. In addition, has least 1/20 energy cost to the Riemannian baselines.
Visualization & Discussion. We empirically study the connection between the proposed MSG and manifold ODE. In particular, visualize toy example of Zachary Karate Club dataset [58] on a S' x S! in Fig. 5, where we plot each layer output the manifold. The red curve the path connecting the layer input and layer output, and the blue one is the direction of the geodesic. As shown in Fig. 5, the red and blue curves are coincided, that is, each layer solves an ODE describing the geodesic on the manifold.
7 Conclusion
VECTOT
pein:
In this paper, we study spiking GNN from fundamentally different perspective of Riemannian geometry, and present a simple yet effective Manifold-valued Spiking GNN (MSG). Concretely, design manifold spiking neuron which
Figure 5: Visualization on S x S1
leverages the diffcomorphism to bridge spiking representations and manifold representations. With the proposed neuron, we propose new training algorithm with Differentiation via Manifold, which no longer needs to recur the backward gradient and thus alleviates the high latency of previous methods. An interesting theoretical result is that, MSG is essentially related to manifold ODE. Extensive empirical results on benchmark datasets demonstrate the superior effecttveness and energy efficiency of the proposed MSG.
Broader Impact and Limitations
Our work brings together two previously separate domains: spiking neural network and Riemannian geometry, and presents novel Manifold-valuee Spiking GNN for energy-efficiency graph learning, especially for the large graphs. Our work is mainly a theoretical exploration, and not tied to particular applications. A positive societal impact is the possibility of decreasing carbon emissions in training large models. None of negative societal impacts we feel must be specifically highlighted here.
Limitation. Our work as well as the previous spiking GNNs considers undirected, homophilous graphs, while the spiking GNN directed or heterophilous graphs may find challenging to implement the proposed method. However, we provide downloadable code and will offer casy-to-use interface.
still remains open. Also, readers
10

---

Acknowledgement
This work is supported part by NSFC under grants 62202164 and 62322202. Philip S. Yu is supported in part by NSF under grants I11-2106758, and POSE-2346158.
References
[1] Hamilton, W., Z. Ying, J. Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
[2] Veličković, P., G. Cucurull, A. Casanova, al. Graph Attention Networks. International Conference on Learning Representations, 2018. Accepted as poster.
[3] Chami, I., Z. Ying, C. Ré, et al. Hyperbolic graph convolutional neural networks. In Advances in NeurlPS, vol. 32. 2019.
[4] Xiong, B., S. Zhu, N. Potyka, et Pseudo-riemannian graph convolutional networks. In Advances in NeurlPS, vol. 32. 2022.
[5] Zhu, 7., J. Peng. J. Li, et al. Spiking graph convolutional networks. In Proceedings of the 1 IJCAI, pages 2434-2441 2440. ijeai.org, 2022.
[6] Li, J., H. Zhang, R. Wu, al. Graph is Worth 1-bit Spikes: When Graph Contrastive Learning Meets Spiking Neural Networks. arXiv preprint arXiv:2305.19330 2023.
[7] Maass, W. Networks of spiking neurons: The third generation of neural network models. Neural networks, 10(9):1659-16) 1671, 1997.
[8] Brette, R., M. Rudolph, T. Carnevale, et al. Simulation of networks spiking neurons: A review of tools and strategies. Journal of computational neuroscience, 23:349-398. 2007.
[9] Wang, B., B. Jiang. Spiking GATs: Learning graph attentions via spiking neural network. arXiv preprint arXiv:2209.1355 13539, 2022.
[10] Yang, H., R. Zhang, Q. Kong, et al. Spiking variational graph auto-encoders for efficient graph representation learning. arXiv preprint urX1922211.01952, 2022.
[11] Yin, M. Wang, Shen, abs/2404.01897, 2024.
Continuous spiking graph neural networks. CoRR,
[12] Sarkar, R. Low distortion delaunay embedding of trees in hyperbolic plane. In Proceedings of the 19th International Symposium on Graph Drawing, vol. 7034 of Lecture Notes in Computer Science, pages 355-366. Springer, 2011.
[13] Bachmann, G., G. Bécigneul, O. Ganea. Constant curvature graph convolutional networks. In Proceedings of the 37th ICML, vol. 119, pages 486-496. PMLR, 2020.
[14] Wu, H., Y. Zhang, W. Weng, et al. Training Spiking Neural Networks with Accumulated Spiking Flow. Proceedings of the AAAI Conference Artificial Intelligence, 35(12):10320-1033 2021.
[15] Meng, Q. M. Xiao, S. Yan, et al. Training high-performance low-latency spiking neural networks by differentiation spike representation. In Proceedings of CVPR, pages 12434- 12443. IEEE, 2022.
[16] J., Y. Chua, M. Zhang, et al. A Tandem Learning Rule for Effective Training and Rapid Inference of Deep Spiking Neural Networks. IEEE Transactions on Neural Networks and Learning Systems, 34(1):446 460, 2023.
[17] Lou, A., D. Lim, I. Katsman, et al. Neural manifold ordinary differential equations. In Advances NeurIPS. 2020.
[18] Kipf, T. N., M. Welling. Semi-supervised classification with graph convolutional networks, In Proceedings of the 5th ICLR. OpenReview.net, 2017.

---

[19] Wu, F., A. Souza, T. Zhang, et al. Simplifying graph convolutional networks. In International conference on machine learning, pages 6861-6871. PMLR, 2019.
[20] Feng, Y., H. You, Z. Zhang, et Hypergraph Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):3558 3565, 2019.
[21] Guo, K., Y. Hu, Y. Sun, et al. Hierarchical Graph Convolution Network for Traffic Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 35(1):151-159, 2021.
[22] Liu. Q., M. Nickel, D. Kiela. Hyperbolic Graph Neural Networks. In Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc., 2019.
[23] Coors, B., A. P. Condurache, A. Geiger. SphereNet: Learning spherical representations for detection and classification in omnidirectional images. In Proceedings of ECCV, pages 518-533. 2018.
[24] Gu, A., F. Sala, B. Gunel, et al. Learning mixed-curvature representations in product spaces. In Proceedings of the 7th ICLR. OpenReview.net, 2019.
[25] Zhang. S., Y. Tay, W. Jiang. et al. Switch Spaces: Learning product spaces with sparse gating. CoRR, abs/2102.08652 2021.
[26] Sun, L., Z. Zhang, J. Ye, et al. A self-supervised mixed-curvature graph neural network. In Proceedings of the 36th AAAI, pages 4146-4155. 2022.
[27] Sun, L., Z. Huang, Wang, al. Motif-aware riemannian graph neural network with generative- contrastive learning. In Proceedings of the 38th AAAI, pages 9044-9052. 2024.
[28] Law, M. Ultrahyperbolic neural networks. In Advances in NeurIPS. 2021.
[29] Gao, Z., Y. Wu, Y. Jia, et al. Learning to optimize on SPD manifolds. In Proceedings of CVPR, pages 7697-7706. Computer Vision Foundation IEEE, 2020.
[30] Dong. S. S. Jia, C. Zhang, et al. Deep manifold learning of symmetric positive definite matrices with application to face recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31. 2017.
[31] Sun, I.., 7. Huang, H. Peng, et al. Lsenet: Lorentz structural entropy neural network for deep graph clustering. In Proceedings of the 4/st ICMI.. OpenReview.net, 2024.
[32] Sun, L. Hu, S. Zhou, et al. Riccinet: Deep clustering via a riemannian generative model. In Proceedings of the ACM Web Conference 2024 (WWW'24) 24), pages 4071-4082. 2024.
[33] Sun, L., Z. Huang. H. Wu, et al. Deepricci: Self-supervised graph structure-feature co-refinement for alleviating over-squashing. In Proceedings of the 23rd ICDM, pages 558-567. 2023.
[34] Sun, L., J. Ye, H. Peng, et al. Self-supervised continual graph learning in adaptive riemannian spaces. In Proceedings of the 37th AAAI, pages 4633-4642. 2023.
[35] Sun, L.. Z. Zhang, J. Zhang, et al. Hyperbolic variational graph neural network for modeling dynamic graphs. In Proceedings of the 35th AAAI, pages 4375 4383. 2021.
[36] Sun, L., J. Ye, H. Peng, et al. A self-supervised riemannian gnn with time varying curvature for temporal graph learning. In Proceedings of the 31st ACM CIKM, pages 1827-1836. 2022.
[37] Sun, L., J. Ye, J. Zhang, et al. Contrastive sequential interaction network learning co-evolving riemannian spaces. Int. J. Mach. Learn. Cybern., 197441137, 2024.
[38] Sun, L., J. Hu, M. Li, et al. R-ode: Ricci curvature tells when you will be informed. In Proceedings of the ACM SIGIR. 2024.
[39] Wang, Y. S. Zhang, J. Ye, al. A mixed-curvature graph diffusion model. In Proceedings of the 33rd ACM CIKM, page 2482-2492. ACM, 2024.
[40] Fu, X., Gao, Y. Wei, et al. Hyperbolic geometric latent diffusion model for graph generation. In Proceedings of the 41st ICML. OpenReview.net, 2024.
12

---

[41] Cao, Y. Y. Chen. D. Khosla. Spiking deep convolutional neural networks for energy-efficient object recognition. International Journal of Computer Vision, 113:54-66 66, 2015.
[42] Cao, J., Z. Wang, H. Guo, et Spiking denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4912-4921. 2024.
[43] Xu, M., Y. Wu, L. Deng, et al. Exploiting spiking dynamics with spatial-temporal feature normalization in graph learning. In Proceedings of the 30th IJCAI, pages 3207-3213, ojcai.org, 2021.
[44] Poli, M., S. Massaroli, J. Park, et al. Graph neural ordinary differential equations. arXiv preprint 1rXiv:1911.07532, 2019.
[45] Li, J., Z. Yu, Z. Zhu, al. Scaling up dynamic graph representation learning via spiking neural networks. Proceedings of the 37th AAAI, vol. 37, pages 8588-8596. 2023.
[46] Zhao, H., X. Yang, C. Deng, et Dynamic reactive spiking graph neural network. In Proceedings of the 38th AAAI, vol. 38, pages 16970-16978. 2024.
[47] Yin, N., M. Wang, Z. Chen, et al. Dynamic spiking graph neural networks. In Proceedings of the 38th AAAI, vol. 38, pages 16495-16503. 2024.
[48] Salinas, E., J. Sejnowski. andegrate-and-iin neurons driven by correlated stochastic input. Neural computation, (4(9):2111-2155 2002.
[49] Burkitt, N. review of the integrate-ane and-fire neuron model: I. homogeneous synaptic input. Biological Cybernetics, 95(1):1-19, 2006.
[50] Huh, D., T. J. Sejnowski. Gradient Descent for Spiking Neural Networks. In Advances in Neural Information Processing Systems, vol. 31. 2018.
[51] Neftci, E. O., H. Mostafa, F. Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient- based optimization to spiking neural networks. IEEE Signal Process. Mag., 36(6):51 1-66,, 2019.
[52] Li, Y., Y. Guo, S. Zhang, et al. Differentiable Spike: Rethinking Gradient- for Training Spiking Neural Networks. In Advances Neural Information Processing Systems, vol. 34. 2021.
(53) Shehur, O., M. Mumme, A. Bojchevski, et al. Pitfalls of graph neural network evaluation. arXiv preprint 1rXiv:1811.055
2018.
[54] Chen, W., X. Han, Y. Lin, et al. Fully hyperbolic neural networks. In Proceedings of the 60th ACL, pages 5672-5686. Association for Computational Linguistics, 2022.
[55] Sun, Y., Zhu, Y. Wang, et SpikeGraphormer: A high-performance graph transformer with spiking graph attention. arXiv preprint arXiv:2403. /5480, 2024.
[56] Bécigneul, G., O. Ganea. Riemannian adaptive optimization methods. In Proceedings of ICLR. OpenReview.net, 2019.
[57] Paszke, A., S. Gross, F. Massa, et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems, vol. 32. 2019.
[58] Zachary, W. W. An information flow model for conflict and fission in small groups Journal of anthropological research, 33(4):452-473, 1977.
[59] Tu, L. W. An Introduction to Manifolds. Springer New York, 2011.
[60] Crouch, P. E., R. Grossman. Numerical integration of ordinary differential equations on manifolds. Journal of Nonlinear Science, 3(1):1-33, 1993.
[61] Petersen, P. Riemannian Geometry, 3rd edition. Springer-Vente 2016.
(62] Zhou, Z., Y. Zhu, C. He, al. Spikformer: When spiking neural network meets transformer. arXiv preprint arXiv:2209.155: 15425, 2022.
13

---

The appendix is organized as follows:
Notation table,
2. Proofs of Theorem 4.1 and Theorem 5.2,
3. Derivation of the Jacobian,
4. Riemannian geometry including geodesically complete manifold, stereographic projection and k-stereographic model, and Cartesian product and product manifold,
5. Empirical details, i.e., datasets/baselines description, theoretical energy consumption and implementation notes,
6. Additional results of link prediction, layer-wise gradient and the visualization.
A
Notations
We summarize the important notations of our paper in Table
Table 4: Notations.
Notation
Description
M,N
Smooth manifolds
y,
Points manifolds
o
The original point on manifold
T M
The tangent space at point
TM
The tangent bundle of the manifold M
u
The vector field over the manifold, described by an ODE
T N
The dual space of T&N
Dzf
The differential of f at point z
(df) 2
The differential f-form
at point 2
(00/021) (0/02") 21
basis of the tangent space T,
((dz) Jay .... (dz") )z)
A basis of TW
VC
The gradient of a smooth scalar function L
Exp2 (-)
The exponential map at point
Logg
The logarithmic map at point
Sd
d-dimensional Sphere model of hyperspherical space d-dimensional Euclidean space
Ed
Hd
d-dimensional Lorentz model of Hyperbolic space
V, 3, F, A
Node set V, edge set Ε, feature matrix F and adjacency matrix A A graph defined V, E, F and A Neighbourhood of node i
9 (V, E, F, A)
Ω
Fe
A graph encoder with parameters 0
Membrane potential of spiking neuron at time step
H(.
The Heaviside step function
ST
Spikes fired by spiking neuron at time step
Vih
Threshold membrane potential of spiking neuron The index of spiking layer in neural network
MSNcuron
The proposed manifold spiking neuron
B Proofs
In this section, demonstrate the proofs of Theorem 4.1 (Backward Gradient) and Theorem 5.2 (MSG as Dynamic Chart Solver).
B.1 Proof of Proposition 4.1
First, we give the formal definition of the pullback. Given a smooth map Φ M - N connecting two manifolds M and N, and a real, smooth function f : - R, the pullback of ʃ by Φ the smooth function o² f on M defined by (o* fx) f(o(x)).
14

---

Next, we introduce some properties of the pullback in the smooth manifold [59] (i.e., differential 1-form of smooth function, communication, and pullback of a sum and a product), supporting the derivation of the backward gradient (Theorem 4.1).
Lemma B.1 (Differential I-form of smooth function). For point z E N related with a coordinate chart (U, ..... z"), there is series of covectors ((dz")a,... (dz") )x} forming a basis of of dual the basis {(0/0z)) 21: 2.... (0/0z")l )2) of langent space TgN. Then, for any smooth function on N restrict to U, the differential I-form of f is
0f
df = E
1=1 Oz dz:
dz'.
(18)
Lemma B.2 (Communication). Let F :N + M be a smooth map, for any smooth function on M, we have F (dg) = g))
Lemma B.3 (Pullback of a sum and a product). Let F : N + M be a smooth map, g is a smooth scalar function on M, and w,7 7 are differential I-forms M. Then, we have
F" + 7) = F + F (gw) = (F"'))
(19)
(20)
Given the properties of the pullback, we derive the closed-fore backward gradient of the real function on the manifold, and prove Theorem 4.1.
Theorem 4.1 (Backward Gradient) Let C be the scalar-valued function, and z' is the output of I-th layer with parameter W which is delivered by tangent vector vt. Then, the gradient of function L. W.E.t W is given as follows:
:vl-1
V @wwl l' 10t VoV.,, VzLC= (D
(21)
where of d-) Expx-- (+), (-) (v"), and [:]* means the matrix of pullback.
Proof. Given z', + in M. and M + M be the smooth map such that F(z') = z
Consider scalar loss function L M - IR, if we relate z' with a chart (U.x*. ..... "!) and z with
(V,y² y.. " "), the gradients of 2/41 and zt are given by Lemma. B.
VatiL= = Σ dy
OC
dy
(22)
z241
8L o F
Vei(CoF) F) Σ
dx.
(23)
dx
Then, we apply the pullback F* on V
22'11L that
dL
F (V
F*
Σ Oy
dy'/2/+1
(24)
x1-1
0C
= Σ E
(IF"dy'a"))
from Lemma. B.
(25)
Oy
2141
OL
Σ
F)l:(d(F"'))))
from Lemma. B.2
(26)
Oy:
Σ Oy
OL
Fdly:(d(y' F))
(27)
Σ OL
ap' Ozi
from Lemma. B.1
(28)
Oy
2f+1
Σ 7L
aF
dæ
(29)
ay
Ozi
kl
2
Then, we can find that the matrix form of the pullback F* can be written as the transpose of the
Jacobian matrix of F, denoted as [
Ix or [] F)+. The derivation of V Wi: L is similar, we only
need to use an addition process like above on V C.
L
15

---

Note that, we give the closed-form expression of exponential map, logarithmic map, and parallel transport for hyperbolic and hyperspherical space, and derive the corresponding Jacobian in Sec.
B.2
Proof of Theorem 5.2
Theorem 5.2 (MSG as Dynamic Chart Solver) (1) [T, + c) R the solution of
dt(t) (DExp_(((t)) Log,
(30)
di
then ztt) (xpx(y(t)) a valid solution to the manifold ODE of Eq. (13) on [T, + c). where z(T). y() is given by the first-order approximation with the small enough,
y(T + = (Dz Logz) )u(z(r),T)
(31)
then the update process of Eqs. and in MSG is equivalent to Dynamic Chart Solver Eq.
[55
Proof.
Let & E [T, + e), then we have
dz(t)
dy(t) (Dy(t) Exp2) dt
(32)
dt
(Dy(t) Exp,) (D (xxx,((())) Log, (EEx)(((()))
(33)
= (Dy(t) Exp,) (Da(t) Loga )
(34)
= u(z(t),t).
(35)
Consider two adjacent charts Log, and (U2, Logz,). such that E U1 and Z2 E U1 0 U2.
Note that, in interval [T, + c), z(T) 21 and z(T + c) = Z2, we have y(T) Expx, (z))
With the first-order approximation, y + e) is thus given by
y(T +) y(T) +
(D (Dexp,, (y(r)) Logz, (E(pp
(36)
(Dz, Logz, )u((())
(37)
Also, Eq. 37 can be treated as a step in the Euler solver 60 for small e. Finally, we have
2(T + c) ExP2, (y(T <)) Z2, ending the process of dynamic chart solver. That is, MSG
considers the logarithmic map to define the charts, and equivalent to Dynamic Chart Solver
(Definition 5.1. completing the proof.
C Deviation of Jacobian
We instantiate the proposed MSG in the Lorentz model H of hyperbolic space, sphere model & of hyperspherical space, and the products H's or/and E's. Accordingly, we derive the Jacobian in H and 8, and introduce the construction in the products in D.3
C.1 Hyperbolic Space
Lorentz Model The d-dimensional Lorentz model Hd is defined on the (d + 1)-dimensional manifold of (z = [20, ø1. "2)" Rd+1 ( z) -1, 0} equipped with the Minkowski inner product.
(u, o = -u0 + Σ u,yt-
(38)
The tangent space point Hd is T. {v E dd v). 0}, and Projx(u) (z, u) cz is to project vector E Rd+1 into the tangent space T2 Hd. The Lorentz norm of tangent vector is defined as ||v||c V(z,z) z)
Theorem requires the Jocabian ((-) = Expx(1) and v(-)) Expey(v), and Lorentz model has the closed-form exponential map given as follows,
sinh([[v]le). |vlle v.
Exp, (v) = cosh(llvll2)z+
(39)
'We utilize the manifold of standard curvature for model instantiation, i.e., constant curvature of -1 for hyperbolic space, and 1 for hyperspherical space. Note that, hyperbolicchypershhrriecll spaces of different constant curvatures mathematically equivalent in essence.
16

---

The inverse of exponential map (i.e, the logarithmic map) is arcosh (zz, x)
Log (x) sinh(arcosh(( (z, x) (x (z, x)
(40)
Deviation of Jacobian We first calculate the Jacobian of 4, and it is given as Dzeh = cosh([Iv]le)))
(41)
Note that, Jacobian of Φ needs the Jacobian of l[vllc, which is derived as
d Dullvil= d(v.v)e (v (v, (D) v) livlle
(42)
where V = (-0 01, ..... val". Then, we compute the derivative of the first term of Eq.
39
Dy cosh( (aahh(lvvlaa [|vllc) d||v|c (cosh (cos)(((vlc))))))lllll)) sinh([lv[le). 2V";
d
(43)
Millc
and the derivative of the second term is derived as sinh([/v]le)) v D sinh (Ivlll])))))))))))>))
sinh((lvlle) D.V
Dy
vv+
(44)
lvlle
||vl|c
l|vIcc
livle cosh([[v[la2) sinh((Iv]le)) VV + sinh((lv|le)
(45)
1
l/vlc
Summing up above equations, we finally have
Dug l cosh((|v]lz)) sinb(IIvllc))
v sinh([[vlle) (I+
(46)
llv/12
l|v|le
where I is the identity matrix.
C.2 Hyperspherical Space
Sphere Model The sphere model Sd is defined on the (d + 1)-dimensional manifold of {z = [zo, =1. _d)" E Rd+1 (2, z) = 1, =0 > 0} with the standard inner product (x, y) = Σi= Tiyh
and norm ||||| = V ビi x? The tangent space at point z is TN5" {v E Rd+1 (2, v) 0}. Similar to Lorentz model, we have Projz
(z, u/z projecting a vector E Rd+1 into T,5
The exponential map in the sphere model is given as
Expx(v) = cos([|v|))z sin(lIvl))
(47)
||vl|
and the logarithmic map is
arccos( (z, x))
Logg (x) sin(arcoos( (z, x)))
(x (z, x)2)
(48)
Derivation of Jacobian
We first calculate the Jacobian of vh and is given as
Dau cI)(
(49)
Similar to that in Lorentz model, the Jacobian of áb needs the Jacobian of ||v
Do = d(v, (VV,.v))D,((v,,))
[vvl vT.
(50)
Then, we compute the derivative of the first term of Eq. 47
d
sin(([vl))
Dy cos(|lvll)= d|[
(cos([\v\)2((
ZvT.
(51)
|vll
and the derivative of the second term is D& sin((Ivl]),
D sin([[vil),
>v sin( [VI( Dev
(52)
|
|||||
|v] cos([)
[| sinh((|v]]) vv"
sin ([vll) |
1
(53)
|
Summing up above equations, we finally have ||v cos[[l]]]]]]]]] c) sin((Ivl))
vVT sin([v[]]
D.
(I ZV7).
(54)
|
|
17

---

D Riemannian Geometry
D.1 Some Notations
Here, we give the formal descriptions of the notions mentioned in the main paper, and please refer to for systematic elaborations.
61
Geodesically Complete Manifold. A manifold is said to be geodesically complete if the maximal defining interval of any geodesic is R. For any Riemannian manifold (M. g) admitting a metric structure given by the length of geodesic
d(p, q) L())l is a piecewise smooth curve connecting to 4h
(55)
the completeness of d can be described as a metric space complete if any Cauchy sequence in converges. For instance, hyperbolic space well as hyperspherical space is geodesically complete.
Tangent Bundle. Given an n-dimensional smooth manifold M, the tangent bundle TM is the disjoint union of all the tangent spaces of the manifold TM LiEE T,M, and the tangent bundle with the projection (vv) p for all TpM is vector bundle of rank n.
Chart. A chart of manifold is a pair (U, Φ) where is an open set in the manifold and Φ U RR is homeomorphism onto it image, giving local coordinate of the manifold. In other words, it provides a way of identifying the manifold locally with a Euclidean space. Given two charts (U,, Φ1) and (U2, 22), if the overlap
h o Φ Φ1 (U, n - Φ2(UU U2) and oi o 25 (2) n2),, (56)
the two charts are said to be compatible.
Curvature and Sectional Curvature. The curvature is a notion describing the extent of how manifold derivatives from being "flat". In particular, the curvature should be viewed as measure R(X, Y)Z of the extent to which the operator (X, Y) - Vx Vy y Z is symmetric, where V a connection on M (where X, Y. Z are vector fields, with Z fixed). Sectional curvature is simpler object of curvature and is defined on two independent vector unit in the tangent space. When V is the Levi-Civita connection induced by a Riemannian metric on M,, it turns out that the curvature operator R can be recovered from the sectional curvature.
Riemannian manifold M
Constant Curvature Space, Hyperbolic Space, Hyperspherical Space. A Riemannian manifold is said to be a constant curvature space (CCS) if the sectional curvature is constant scalar everywhere on the manifold. When the CCS has a negative constant curvature, it is referred to as hyperbolic space, and the CCS is hyperspherical when its constant curvature is positive.
D.2 x-stereographic model and Stereographic Projection
s-stercographic model gives unified formalism for both positive and negative constant curva- tures. For positive curvature, it is the hyperspherical model for the hyperspherical space, and for negative curvature, switches to the Poincaré ball model.
Specifically, for a curvature x and dimension d 2 2, the k-stereographic model st is defined on the manifold of (x E Rd - x x||x|| 1 <<< 1}. which is equipped with a Riemannian metric o [x++|*|]-)>
for any constant curvature K. When i 2 O, the defining domain is Rd in which the stereographic projection of the Sphere model of hyperspherical space is endowed. When x < the manifold st is represented in an open ball of radius -=x and is the stereographic projection of the Lorentz model of hyperbolic space.
The x-stercographical model is gyrovector space in which a non-associative vector operator system is defined. For x, y E G", a E IR, the k-addition (a.k.a. Möbius addition) is given as
(1- 2xx
<|yy x + (1+ (1++||x]]3)
x +
(57)
2xx
12
18

---

The distance function given by æ-addition is thus formulated
(x, y) = 2 tan (ll(-x) 0x yll)
(58)
The k-scaling for any real scalar c is defined
NN ta tan (([xl))) ||x]
(59)
The unit-speed geodesic from y
Xx-y x 0xx (t xx ((-x) 0x y))
(60)
With the unit-speed geodesic, the exponential map as well as its inverse (i.e., the logarithmic map) has the closed-form expression follows,
(vp=(v) +xx (tan,, (1x|3 X X v
(61)
2 |vll
k||
(-x) Bx
Log = X
tan - ([((-x) Bx yll)
(62)
(x-x) +x yll
where the curvature-aware trigonometric function is utilized, e.g..
VR (a)
>>>>
tan, (z)
0,
(63)
tanh(x)
Stereographic Projection ferent model spaces of Riemannian manifold. In particular, is defined as map / + st taking the form of
The stereographic projection is diffcomorphism connecting the dif-
π Ld / st.
x
1+ VInix'441 X/aa-
(64)
where x' is point the Lorentz model TLg or Sphere model L, and x, the image of the projection, the corresponding point in the gyrovector hall of w- sterreoggrpphic model. The inverse projection is given follows,
(DE 11)
st + S
x'
Mxx,
(65)
where Xg
is known the conformal factor.
|+
D.3 Cartesian Product and Product Manifold
The concept of product manifolds allows for creating a new manifold from a finite collection of existing ones. Given set of smooth manifolds as the Cartesian product of these manifolds:
Mr, the product manifold given
IP = M1 M2 x... Mg.
(66)
where @ denotes the Cartesian product. Specifically, with the Cartesian product construction, point x are represented by a concatenation of = [xI...., where x; E M,. A tangent vector T&P at point x can be given v [VI,.... vnl. where v; Mi If each manifold M; equipped with a metric tensor gi, the product metric g decomposes into the direct sum of the individual metrics g @t_g', which can be expressed as Diag(g' .... ....g).. For x and y P, the distance between them defined as y) E d.m. (x,, y:). Accordingly, the exponential map is given as
Expx(lvi,..., val) Expx, (v1), Expx, (v2).... ..... Expx, (v+)).
(67)
19

---

Table 5; Dataset statitics.
Computers
Photo
CS
Physics
#Nodes
13752
7650
18,333
34,493
#Features
767
745
6,805
8,415
#Edges
245861
119081
163,788
495,924
#Classes
10
8
15
5
E Experimental Setups
E.1
Dataset
We four common benchmark datasets to evaluate model and Table the details of the datasets. There are two co-purchase graphs including Amazon-Photo and Amazon-Computers [53 and two co-author network including CS and Physies (53)
E.2 Baselines
Table 6: The categories of the baselines
ANN-based Models
SNN-based Models
GCN 18.. SAGE [T]
SpikeNet 45 SpikeGraphormer [55],
Euclidean Space
Q GCN 4 K-GCN [13]. SpikeGCN []. SpikeGCL (6)
GAT 2 SGC [T9]
Riemannian Space
HyboNet [54]. HGCN
As shown in Table we divide the baselines into three categories: ANN-based Euclidean GNNs, ANN-based Riemannian GNNs and SNN-based Euclidean GNNs. Note that, none of the existing work studies the SNN-based GNN in Riemannian space, the best of our knowledge.
ANN-based Euclidean GNNs
GCN defines graph convolution on the spectral domain.
SAGE It gives the and-combine formulation for the message passing over the graph.
GAT 2 It introduces the attention mechanism for the learning on graphs.
SGC [9] reformulates GCN [88 with feature propagation and linear layer, acting as a
low-pass filter.
ANN-based Riemannian GNNs
generalizes GAT [2] in the Lorentz model of hyperbolic space in which the graph convolution is conducted in the tangent space.
HGCN 3
k G-GN :
It generalizes GCN 18 to the x streogrrapphl model constant curvature
spaces where several gyrovector operators are given in the unified formalism.
HyboNet [54]: It introduces parameterized Lorentz transformation for hyperbolic graph modeling without the tangent space.
GCN studies the graph convolution network in the Pseudo-Riene Riemannian manifold.
SNN-based Euclidean GNNs.
SpIkeGCN 5 integrates SNN and graph convolution network in which SNN acts as an activation function.
SpikeGraphormer [55] (termed SpikeGT for short in our main paper): It generalizes
kind of graph transformer with spiking neurons, accompanied by an ANN for improving the performance.
20

---

SpikeGCL [6]: It introduces a method to perform graph contrastive learning with the spiking GNN.
SpikeNet [5 is original designed for dynamic graph modeling, and we utilize its version for static graph following [6].
Note that, existing SNN-based GNNs work with Euclidean space, and leverage the BPTT training with surrogate gradient, suffering from high latency.
E.3 Theoretical Energy Consumption
Following the previous works [6 62 we calculate the theoretical energy consumption for each model, instead of measuring actual electricity usage, for fair comparison.
For the SNN-based models, the energy consumption involves encoding energy Exncoding and spiking process energy Espiking. The former is calculated by the number of multiply-and accumulate (MAC) operations. and the latter is given by the number of SOP operations. The energy consumption is thus defined as follows,
E Fenccdingg + Esgiking EMAC Σ Nds. ESOP
S
(68)
t=1 1=1
where the scaling constant EMAC and Esop are set to 4.6pJ and 3.7pJJ, respectively, N is the number of nodes in the graph, d is the dimension of node features, L is the number of layers the neural model. T is the time steps of the spikes, and S denotes the output spikes at time step t and layer 1.
For the ANN-based models, the energy consumption is given by embedding generation step and aggregation step, and both of them calculated by MAC operations, In the embedding generation step, the feature transformation with a weight matrix Rd... x xdoue executes N Nd,, dout multiplication and Nd,, dout addition operations. In the aggregation step, |E (din. is the number multiplication and |E dout is the number of addition operations. Supposing Eld,, E dout, the energy consumption is given as follows.
(69) where the scaling constant EMAC is set to 4.6pJ, |E| is the number of edge, din and dout are the input and output dimensions.
E EMAC (Nd + |E dout).
E.4 Implementation Notes
Model Instantiation. The proposed MSG is instantiated in the Lorentz model IH of hyperbolic space or Sphere model S of hyperspherical space as well as the products over and S. Note that, MSG can be equivalently instantiated on the x strreroogac model (i.e., Poincaré model of hyperbolic space or Hyperspherical model of hyperspherical space), given the closed form Riemannian metric and exponential map. The equivalence can be achieved by scaling the stereographical projection. We leverage the Lorentz model H by default.
Hyperparameters. The dimension of the manifold set as 32. When we instantiate MSG on the product manifold, the sum of factor manifold's dimensions is defined as 32. The manifold spiking neuron is based on the IF model [49] by default, and it is ready to switch to the LIF model [49]. The time latency T for neurons is to 5 or 15. The step size in Eq. is set to 0.1. The hyperparameters are tuned with grid search, in which the learning rate is {0.01, 0.003) for node classification and (0.003, 0.001} for link prediction, and the dropout rate is {0.1, 0.3, 0.5].
Hardware. Experiments are conducted on the NVIDIA GeForce RTX 4090 GPU 24GB memory, and AMD EPYC 9654 CPU 315 with 96-Core Processor.
F Additional Results
In this section, we show the additional results on backward gradient, comparison between IF and LIF model, link prediction and visualization.
21

---

gradient norm of in layer D
gradient norm of in layer
gradient narm of layer
M M
20
25
Norm 11.5
Norm
1.0
1.55
150
175
200
50
75
100 Rerations
125
150
175
25
50
75
100
175
202
Beeratonss
Iterations
(a) The norm of backward gradient of L with respect to in each spiking layers.
gradient norm v in layer 0
gradient norn v in layer
gradient norm of in layer
20-
25
25-
20
2.0
Norm 16.5
20
1.5
10-
100
150
375
200
25
50
105
150
25
90
200
Ber
Itenations
(b) The norm of backward gradient with respect to each spiking layers.
Training loss
4.3
Ioorations
206
(c) The loss in model training.
Figure 6: Visualizations of the training process for node classification on Computer dataset.
Backward Gradient. Previous studies compute backward gradients though the Differentiation via Spikes (DvS). Distinguishing from the previous studies, compute backward gradients though the Differentiation via Manifold (DvM). In order to examine the backward gradients, we visualize the training process for node classification on Computer dataset. Concretely, we plot the norm of backward gradients in each iteration in Figs. (a) and (b) together with the value of loss function in Figs. (c). As shown in Fig. 6 the proposed algorithm with DvM converges well, and the backward gradients do not suffer from gradient vanishing or gradient explosion.
Comparison between IF and LIF model. the main paper, the proposed MSG is built with the IF model, and it is applicable to LIF model as well. We compare the performance between IF and LIF model different algorithms (DvM and DvS) and in different manifolds (hyperbolic Hi, hyperspherical S, Euclidean E and the product spaces among Hi and S) for comprehensive evaluation.
22

---

The results of node classification on Computer, Photo, CS and Physics datasets are summarized in Table [nd T Note that, IF model and LIF model achieves competitive performance every case. We opt for IF model in the model instantiation for simplicity.
Table 7: Comparison between IF and LIF model in Node Classification, qualified by classification accuracy (%). The proposed model is trained by Differentiation via Spikes (i.e., BPTT with the surrogate gradient).
Computers
Photo
CS
Physics
H
89.65=0.18
93.46:0.12
91.73=0.34
95.16=10117
g32
89.37=0026
93.39:10.21
91.59:10.24
95.15++0.10
R
08.3
92.75+0.40
92.53+0006
96.00+00.03
H1
89.5810.34
92.8110.21
92.44:0.13
95.63:10.02
g02
89.32+0.19
92.82=#.115
92.11+0.16
95.54+0.04
g
89.1
92.93=:.233
92.56+0.15
95.97=0.055
Table Comparison between IF and LIF model in Node Classification, qualified by classification accuracy (%). The proposed model is trained by Differentiation via Manifold.
Computers
Photo
CS
Physics
H
89.27=0.19
93.11=0.11
92.65=0.044
95.9310.07
S82
87.8410.77
92.03:1.79
92.72=0.06
95.85:10.02
R
88.94+0.24
92.93+0.21
92.82+0.04
95.81+0004
H2
88.7110.13
92.74.0.077
92.43:0.11
95.6410.06
g02
88.43+0.10
92.45+0.13
92.53+0.06
95.84+10.33
E
86.3410.19
92.42:10.09
92.66+0.099
96.02=10.03
Link Prediction. The performance of link prediction in terms of AUC provided the main paper. We show the results on Computer, Photo, CS and Physics datasets in terms of AP (%) in Table 9 In particular, we feed the spiking representation of SpikingGCN and SpikeGCL into the Fermi-Dirac decoder same the proposed MSG, while SpikeNet and SpikeGT are designed for node classification specially. As shown in Table 99, the proposed spiking KSG consistently achieves the best results among the spiking GNNs on all the four datasets, and even achieves the competitive performances with the strong Riemannian baselines.
Table 9: Link Prediction in terms of AP (%) Computers, Photo, CS and Physics datasets. The best results are boldfaced, and the rumner-ur upp are underlined. The standard derivations are given in the subscripts. OOM denotes Out-Of- Of-merrry.
Computers
Photo
CS
Physics
GCN 18
92.1010.50
86.43.10.4
93.38
0.44
ANNEE
GAT
2
91.61
87.04.10.0
94.34.0.60
93.44
SGC
19
90.78=0060
+0.05
95.34+0.58
95.37=081
SAGE
00.51+0.44
88.40
94.864021
95.15
HGCN
96.46=20.74
93,86=+0.30
91.90++0.35
92,01=0.57
ANK-R
R-GCN
13
94.8010.60
93.50.10.09
94.9710.07
94.16
Q-GCN
4
96.28 10.03
96.65 10.10
92.24.10.75
OOM
HyboNet
54
95.78 10.07
96.79
96.21
98.12
SpikeNet
45
EANG
SpikingGCN
S
01.1741.64
93. 16:1044
94.79
92.19
SpikeGCL
6
92.54
95.16.10.12
95.06.10.19
91.82=0.25
SpikeGT
(55
MSG (Ours)
94.45
96.46+0.19
95.12+0.12
92.53=0.19
Visualization. Here, we visualize the forward pass of the proposed MSG and empirically demonstrate the connection between MSG and manifold ordinary differential equation (ODE).
We choose a toy example of KarateClub dataset. The proposed MSG are instantiated on the 2D manifold for the ease of visualization. Specifically, we plot the node representation of each spiking
23

---

point
(a) 1-th nodes.
point
(b) 17-th nodes.
Figure 7: Visualizations node representations on Zachary karateClub datasets (58).
layer in Fig. (aa and Fig. ( in which the curve connecting the outputs of successive layer is marked red, and blue arrow the direction of the geodesic. is shown that spiking layer forwards the node along the geodesic on the manifold. In other words, each layer, constructing chart given by the exponential map, a solver of the ODE describing the geodesic.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We propose novel Manifold-valued Spiking GNN (MSG), and design new training algorithm with Differentiation via Manifold with theoretical gaurantee. Extensive experiments show the effectiveness the proposed approach.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
We discuss the limitations in Sec. g The proposed model is applicable to any geodesically complete manifold (e.g., hyperbolic space, hyperspherical space and their products), and its generalization more generic manifold leaves the future work.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
25

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
We provide the complete proof in Appendix B
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Experimental details are given in Sec. 6.1 and further introduced in Appendix E Also, we give pseudocodes in Algorithm
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
26

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
The datasets publicly available. We properly cite and introduce the datasets Sec. 6. and Appendix E.T Our source code at the anonymous link https
Justification:
// //anonymoos 4open. scwence/r /MMG- 16E9
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr cc/
public/guides//odeeiiee sttp://ieeeeeeeeeeeaaaeaaaiiiiee SonrooeeSubmmssssooooooooy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/
guwdes/ CodeeSumieee onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
reproducible, they
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We give experimental settings in Sec. 6.I which is further detailed in Appendix Implementation details be found in the anonymous link at httt:::///aoommmuus
E.
4open sppen.sciencee//MSCC 16E9
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: In the experiment, we perform 10 independent runs for each case, and report the mean with standard derivations in Sec. 6
Guidelines:
The answer NA means that the paper does not include experiments.
27

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In Sec. 6.1 we provide the information on the computer resources; NVIDIA
GeForee RTX 4090 GPU 24GB memory, and AMD EPYC 9654 CPU with 96-Core Proces-
sor.
Guidelines:
The answer means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported didn't it make it into the paper).
paper (e.g., preliminary or failed experiments that NeurlPS Code of Ethics :tttps://neur// http:://wwrrrps..ppplleleee/e/// httpp//blccctticccccciceeiaaaaa.
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the
Answer: [Yes]
Justification:
The research conducted in the paper conforms with the NeurIPS Code of
Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
28

---

We discuss both potential positive societal impacts and negative societal impacts of the work performed in Sec. g
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We properly cite the public datasets and open source Python library (e.g..
Geoopt and SpikingJelly) in Sec. 6. and Appendix E.1
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
29

---

The authors should state which version of the asset is used and, if possible, include URL.
The name the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thoode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: We give the source code of the proposed model with documentation at https // //anonymous 4open science/r/MSG-1100 16E9
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
30

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "lD3fqggPnh", "reviewer": "Reviewer_BtYr", "summary": "This paper focuses on the theoretical aspects of spiking neural networks, which are a variant of neural networks closer to the human brain, particularly incorporating a time component. One of the hopes for this type of networks is that they are much more energy efficient. In this paper, the authors extend the classical spiking graph neural networks setting, which only handle graphs in Euclidean space, to the setting of Riemannian manifolds. This is highly non-trivial, requiring a specific of each spiking neuron on manifolds. Intriguingly, they then relate this new model to manifold ODEs, and also show even theoretically a significant improvement of energy efficiency as opposed to classical GNNs.", "questions": "see weaknesses", "limitations": "---", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "* SNNs are very timely due to the severe energy problems of classical NNs.\n* Goes significantly beyond the state of the art.\n* Very well-organized paper, intriguing to read.\n* From deep math to numerical experiments, this paper covers the full range\n* Comprehensive analysis of the newly introduced type of SNNs, and also showing their superiority", "weaknesses": "* Argument, why Riemannian manifolds are required, could be strengthened, and also what about other types of manifolds?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q: Argument, why Riemannian manifolds are required, could be strengthened, and also what about other types of manifolds?**\n\nOn the one hand, Riemannian manifolds are well aligned with graph structures (e.g., the hierarchical/tree-like and cyclical structures of graphs correspond to hyperbolic and spherical manifold, respectively), and Riemannian graph models (e.g., HGCN of hyperbolic space [1], $\\kappa$-GCN of constant curvature manifold [2]) report superior performance in recent years. \n\nNote that, besides matching the graph geometry, the proposed MSG essentially decreases the energy consumption of Riemannian GNN, which is shown in Table 3. **Thus, the proposed model is much more scalable and greener to handle large graphs.**\n\nOn the other hand, owing to the Diffeomorphism of Riemannian manifolds, the proposed MSG obtains manifold representation based on SNN. **As a result, MSG becomes a more general approach that can handle not only node classification but also link prediction.**\n\nNote that, existing SNNs do not achieve satisfactory results in link prediction, but MSG achieves competitive link prediction results to the state-of-the-art ANNs with the aid of the Riemannian manifold.\n\n**For other types of manifolds:**\n\nThe diffeomorphism and distance measure of Riemannian manifold essentially support the design of DvM optimization approach and benefit link prediction, respectively.\n\nFor pseudo-Riemannian manifolds, we can apply the idea of MSG but it lacks an elegant exponential map, given the geodesic is broken in pseudo-Riemannian manifolds [3]. Thus, we consider the Riemannian manifolds, e.g., hyperbolic space, sphere, and their products.\n\nFor the topological manifolds other than Riemannian/pseudo-Riemannian ones, the alignment between graph and the manifold is not well established.\n\n[1] Chami, I., Ying, Z., Re, C., and Leskovec, J. Hyperbolic graph convolutional neural networks. In Advances in the 32nd NeurIPS, pp. 4869–4880, 2019.\n\n[2] Bachmann, G., G. Bécigneul, O. Ganea. Constant curvature graph convolutional networks. In Proceedings of the 37th ICML, vol. 119, pages 486–496. PMLR, 2020.\n\n[3] Xiong, B., S. Zhu, N. Potyka, et al. Pseudo-riemannian graph convolutional networks. In Advances in NeurIPS, vol. 32. 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L6Vd6OU0AB", "reviewer": "Reviewer_NhQj", "summary": "This paper first analyzes the limitations of spiking GNN, representation space and model training, and then present a Riemannian model called MSG, which is connected to manifold ODE. Finally, the authors conduct experiment to show the effectiveness and energy efficiency.", "questions": "See W1, and other questions are listed as follows:\n1.I have double checked Proofs 4.1 and 5.1, and the proofs are correct. However, I would like to clarify: whether or not there is a typo in Line 275 of Thm 5.2, that z and y are swapped.\n2.I notice that, the authors mention another optimization method, sub-gradient [15], also different from the BPTT scheme. Thus, specify the sub-gradient, and the different between sub-gradient and the proposed DvM.\n3.Given that gradient exploding and vanishing are common problems in training SNN, will MSG suffer from these issues? Also, is there any treatment for the numerical stability of Riemannian operators, e.g, sinh and cosh in the Appendix?", "limitations": "No limitation (negative societal impact) needs to be specifically addressed.", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "On the significance,\n1.This paper relates two previously disparate research areas, SNN and Riemannian geometry.\n2.Energy efficiency is of importance to the wide use of Riemannian models, given that they are typically computationally expensive.\n\nOn the originality,\n1.Unlike existing Riemannian GNNs, the authors design a Riemannian GNN with the spiking neuron.\n2.It is a novel idea to parallel Riemannian and spiking domains, providing the opportunity for a faster training algorithm, DvM.\n\nOn the quality, this is a Solid paper with few technical,\n1.The theoretical results (connection to ODE) are proved, and the closed form gradient is well elaborated.\n2.The empirical results are adequate, and Codes are provided for reproducibility.\n\nOn the clarity, this paper is well-organized and easy to follow in general.", "weaknesses": "1.In the experiment, in Table 1, the authors do not provide some of link perdition results of SNNs. Thus, it needs to specify on those SNN baselines, e.g., is SpikeNet [33] able to conduct link prediction? Why not?\n2.This paper requires systematic understanding of differential geometry, and thus is not friendly to the reader not familiar to this area.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: In Table 1, some link prediction results of SNN baselines are lacked.**\n\nFor the SNNs generating spiking representations (i.e., SpikeGCN and SpikeGCL), we feed the spiking representation into Fermi-Dirac decoder, obtaining the pairwise similarity for link prediction.\n\nFor the SNNs that cannot generate spiking representations (i.e., SpikeNet and SpikeGraphormer), we do not report the results of link prediction, since there exists no well-recognized similarity for the spikes.\nAlso, SpikeNet is designed for node classification specifically.\n\n\n**Q1: Typo in Line 275.**\n\nThanks for your careful review and kind reminder. Concretely, $\\mathbf{y(t)}=Log_\\mathbf{z}(\\mathbf{z}(t))$ in the Theorem.  \n\n**Q2: The difference between the sub-gradient method in reference [15] and the proposed DvM approach.**\n\nThe sub-gradient method generates spike representations from corresponding spike trains, and conduct gradient backpropagation through the spike representations.\n\nThe sub-gradient method works with the Euclidean space of spike representations, while the proposed DvM conducts gradient backpropagation on the manifold.\n\nIn the sub-gradient method, the clamp operator may cause gradient error, while each term in DvM is differentiable, and thus DvM computes gradient without approximation.\n\n**Q3: Will proposed MSG suffer from gradient exploding/vanishing?**\n\nSNNs are exposed to the issue of gradient exploding/vanishing due to BPTT training, which requires recursive gradient backpropagation for each timestep. Thus, SNN tends to encounter gradient exploding/vanishing when the number of time steps is large (similar to the RNN going deeper).\n\nIn contrast, in MSG, we propose DvM training approach that conducts gradient backpropagation via the manifold and does NOT need recursive backpropagation. Moreover, from Eq. (39) in Appendix C, our formulation with the exponential map is similar to the residual connection, alleviating the gradient vanishing.\n\nEmpirically, as shown in Figure 4(a) and the results in Appendix F, the proposed MSG does not expose to gradient exploding/vanishing.\n\n**On the numerical stability of cosh and sinh.**\nWe address the issue of numerical stability in the division of $cosh$ and $sinh$, especially for the zero-value. \n\nSpecifically, we compute the limits in the form of $\\frac{sinh(x)}{x}$ and $\\frac{cosh(x)-1}{x^2}$ when $x\\rightarrow 0$. We substitute the limits into the formulas as $x$ is less than a threshold.\n$$\n\\lim_{x\\rightarrow 0}\\frac{sinh(x)}{x} = 1.\n$$\n\n$$\n\\lim_{x\\rightarrow 0}\\frac{cosh(x)-1}{x^2} = 0.5.\n$$", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2brW563fXr", "reviewer": "Reviewer_aPZg", "summary": "In order to improve energy efficiency and performance in graph learning, the research presents a unique Manifold-valued Spiking Graph Neural Network (MSG), which combines spiking neurons with Riemannian manifolds. Differentiation via Manifold (DvM), a unique training approach, is proposed by the authors to solve the high latency issue with spiking GNNs.", "questions": "Can the authors provide more details on the potential neuromorphic implementation? \n\nAre there any specific real-world applications where MSG could be particularly beneficial?", "limitations": "See weakness and questions", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Combining spiking neurons with Riemannian manifolds is a novel idea that addresses both energy efficiency and performance\n\nNumerous experiments show that MSG is more effective and energy-efficient than both traditional and spiking GNNs currently in use.", "weaknesses": "It is not the first time the Riemann manifold has been introduced into spiking neural networks. The authors lack proper discussion of these works. \n\nThe energy consumption is unknown for the proposed method due to the complexity of integrating spiking neurons with Riemannian manifolds.\n\n\nWhile the theoretical contributions are significant, the paper lacks discussion on practical applications and real-world scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: It is not the first time the Riemann manifold has been introduced into spiking neural networks.**\n\nIn this paper, the geometric concept of Riemannian manifold is **a smooth manifold endowed with a Riemannian metric**. To the best of our knowledge, this is the first time that Riemannian manifold is introduced into spiking neural networks.\nCould you please provide the relevant paper?\n\n**W2: The energy consumption is unknown for the proposed method.**\n\nThe result of theoretical energy consumption is given in Table 3 where both encoding process energy and spiking process energy are calculated. \n\nSpecifically, we calculate the number of multiply-and-accumulate operations (MACs), and the number of synaptic operations operations (SOPs), following the previous works [1] [2]. The formula of theoretical energy consumption is given below,\n$$\nE=E_{\\text{encoding}}+E_{\\text{spiking}}=E_{\\mathrm{MAC}}\\sum_{t=1}^{T}Nd S_t+E_{\\mathrm{SOP}}\\sum_{t=1}^{T}\\sum_{l=1}^{L}S_{t}^{l},\n$$\n- \\\\( N\\\\): the number of nodes in the graph\n- \\\\( d\\\\): the dimension of node features\n- \\\\( L\\\\): the number of layers in the neural model. \n- \\\\( T\\\\): the time steps of the spikes\n- \\\\( S_l^t\\\\): the output spikes at time step $t$ and layer $l$.\n\nMore details are given in Appendix E.3. \n\nIn addition, **in the model inference, the proposed MSG leverages the tangent vectors for the downstream tasks (i.e., we sum the $\\epsilon\\mathbf{v}^{l-1}_i$ in Eq. (7) accumulatively for each layer.), and do not need the expensive exponential maps in the manifold**. Consequently, the proposed MSG achieves low energy consumption.\n\n[1] Li, J., H. Zhang, R. Wu, et al. A Graph is Worth 1-bit Spikes: When Graph Contrastive Learning Meets Spiking Neural Networks. arXiv preprint arXiv:2305.19306, 2023.\n\n[2] Zhu, Z., J. Peng, J. Li, et al. Spiking graph convolutional networks. In Proceedings of the 31st IJCAI, pages 2434–2440. ijcai.org, 2022.\n\n**W3/Q2: On the real-world applications where MSG could be particularly beneficial.**\n\nFirst, the proposed MSG is particularly beneficial to the **Large-scale Riemannian Graph Learning**.  \n\nRiemannian graph models (e.g., HGCN of hyperbolic space, $\\kappa$-GCN of constant curvature manifold) report superior performance in recent years. Fundamentally, Riemannian models are well aligned with the graph structures, but they are computationally expensive and thus results in high energy consumption.\n\nBridging this gap, MSG essentially decreases the energy consumption of Riemannian GNN, which is shown in Table 3. We list the theoretical energy consumption (mJ) of HGCN, $\\kappa$-GCN and the proposed MSG here for reference.\n\n| Model | Computers | Photo | CS | Physics |\n| --- | --- | --- | --- | --- |\n| HGCN | 1.614 |0.869 | 18.390 | 42.800 |\n| $\\kappa$-GCN | 1.647 | 0.889 | 18.440 | 42.836|\n| MSG | 0.047 | 0.043 | 0.026 | 0.029|\n\nConsequently, the proposed model is much more scalable and greener to handle large graphs.\n\nSecond, the proposed MSG is particularly beneficial to the **SNN-based Link Prediction.**\n\nExisting SNNs do not achieve satisfactory results in link prediction, but MSG achieves competitive link prediction results to the state-of-the-art ANNs with the aid of Riemannian manifold, as shown in Table 1.\n\n**Q1: Details on the potential neuromorphic implementation.**\n\nThe neuromorphic implementation of the proposed MSG has **no difference to the previous Euclidean Spiking GNNs**, such as SpikeGCL and SpikeGCN.\n\nSpecifically, referring to the model inference (the answer of W2), the proposed MSG only needs simple operations besides SNN, and do not need the complicated manifold operations, which is similar to the previous methods, such as SpikeGCL and SpikeGCN. Also, the spikes can be directly implemented on neuromorphic hardware [1][2].\n\n[1] J. Zhao, E. Donati and G. Indiveri, \"Neuromorphic Implementation of Spiking Relational Neural Network for Motor Control,\" 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), Genova, Italy, 2020, pp. 89-93, doi: 10.1109/AICAS48895.2020.9073829.\n\n[2] Thiele J C, Bichler O, Dupret A. Spikegrad: An ann-equivalent computation model for implementing backpropagation with spikes[J]. arXiv preprint arXiv:1906.00851, 2019.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Zs0jDY2CDI", "reviewer": "Reviewer_RDnD", "summary": "This work proposes a spiking graph neural network on Riemannian manifolds, named Manifold-valued Spiking GNN (MSG). This work also develops a new training algorithm of differentiation via manifold (DvM) that avoids the high training overhead of BPTT methods and proves that the MSG is a neural ODE solver. Experimental results show that MSG outperforms existing spiking GNNs with low energy consumption.", "questions": "Please refer to the weaknesses", "limitations": "The authors have stated the limitations and broader impact.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. It is a novel idea to explore spiking GNN on Riemannian manifolds.\n2. This work is technical solid. It proves in detail the theory that the MSG is a neural ODE solver.\n3. The proposed MSG outperforms existing spiking GNNs in both node classification and link prediction tasks with low energy consumption.", "weaknesses": "1. Figure 4(a) is confusing. In line 356-357, the author state that backward time of DvM is significantly less than that of BPTT algorithm. But in figure 4(a), the bar of DvM is higher. Is this a mistake? If not, please explain. In addition, it would be helpful to list the complete training times.\n2. Figure 2 shows that the manifold representations will feed forward with the spike trains. This is acceptable during training, but during inference, since $f(\\cdot)$ is an exponential function, the computation of the manifold representations involves a lot of float-point operations, which is incompatible with the nature of spiking neural networks. Please describe in detail the computation of the manifold representations during inference, especially the float-point operations involved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: On Figure 4(a) and Training Time**\n\nSorry for the typo in Figure 4(a) where we mislabeled the legend. We list the training time as follows.\n\nThe gradient backpropagation (BP) time of DvM and Surrogate on Lorentz model is shown below.\n\n|  Time steps | *BP Times (DvM)*  | BP Times (Surrogate) |\n|--- | ----|--- |\n| 30 | 0.0405|  0.0523|\n| 50 | 0.0408 | 0.1054 |\n| 70 | 0.0418|  0.6098|\n| 100 | 0.0621|  2.3260|\n\nThe gradient backpropagation (BP) time on Sphere model is as follows.\n\n|  Time steps | *BP Times (DvM)*  | BP Times (Surrogate) |\n|--- | ----|--- |\n| 30 | 0.0075|  0.0224|\n| 50 | 0.0076 | 0.0605 |\n| 70 | 0.0082|  0.1572|\n| 100 | 0.0333|  1.4022|. \n\nAs shown in the tables, the proposed DvM achieves the significant less training time to BPTT, as DvM does not perform gradient backpropagation recursively for each time step.\n\n**W2: Details of model inference.**\n\nIn the model inference, **the proposed MSG do not need the expensive successive exponential maps, and thereby only limited float-point operations (i.e., addition) are involved.** \n\nIn MSG, we leverage the tangent vectors $\\mathbf{v}$’s for the downstream tasks, in accordance to the Charts of manifold ODE in Sec. 5. \n\nSpecifically, with the model parameters trained by DvM, \nwe first conduct the pooling operation over the spikes to obtain the tangent vector as in Equation 7, \n\nand then sum over each layer to get $\\mathbf{v}_i=\\epsilon\\sum_l \\mathbf{v}^{l-1}_i$. \n\nFinally, $\\mathbf{v}_i $ is feed in the classification head; or the Fermi-Dirac decoder for link prediction.\n\nWe will specify this point in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eJpnoCtgUk", "reviewer": "Reviewer_hcr5", "summary": "The authors generalize spiking GNN to Riemannian manifold, and design a new architecture of parallel forwarding so as to boost model training. Then, the proposed MSG is evaluated with 12 baselines on real graphs.", "questions": "Q1. The authors add a parallel Riemannian forwarding to accelerate the SNN training, leveraging DvM. However, it needs to specify the additional parameter and will it damage the energy efficiency of SNN?\n\nQ2. In the ablation study of Section 6.2, the training time of DvM is much less than that of BPTT, and the question is that, is DvM’s backward time of independent from the steps of spikes?\n\nQ3. Please specify the manifold of Fig. 5 in the Experiment, i.e., the construction of the manifold or its equation.\n\nQ4. What is the different between this paper and the recent paper, Continuous spiking graph neural networks, regarding ODE?", "limitations": "The authors specify the limitations and potential negative social impact in Sec. 8.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "S1. A technical strong paper, and No error is detected. It presents a Riemannian optimization to boost the SNN training and connects the proposed model to Riemannian ODE. \n\nS2. The studied problem of Spiking GNN training is interesting, since it is time consuming to conduct BPTT training of SGNN especially when the spiking train is long.\n\nS3. The authors give a new architecture of parallel forwarding so that graph is modeled in Riemannian manifold, different from the previous SGNN in Euclidean space.\n\nS4. The experiment and visualization are convincing, and the results show that Riemannian modeling of SNN generally achieves superior performance.", "weaknesses": "W1. The proposed model works with the backbone of GCN, and the performance of other backbones, GAT for instance, is not examined.\n\nW2. Some experimental details are not specified, e.g., the link prediction setting with SNN baselines, which is not frequently reported in graph SNN studies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: Performance on GAT backbone.**\n\nThe proposed MSG is applicable to any backbone GNN, where the incoming current $x$ is given by the corresponding backbone.\n\nWe show the results of GAT backbone in the sphere manifold on Computers, Photo, CS and Physics datasets as follows. (The mean with standard derivations of 10 independent runs are reported.)\n\n|         | Computers   | Photo       |  CS         | Physics     |   \n|---------|-------------|-------------|-------------|-------------|\n| GAT   | 86.82±0.04  | 86.68±1.32  | 91.74±0.22  | 95.11±0.29  |   \n| $\\mathbb{S}^{32}$(MSG GCN) | 87.84±0.77  | 92.03±0.79  | 92.72±0.06  | 95.85±0.02  |   \n| $\\mathbb{S}^{32}$(MSG GAT)| 88.51±0.88  | 92.63±0.45  | 92.51±0.08  | 95.82±0.03  |\n\nAs shown in the table above, MSG achieves competitive performance on GAT and GCN backbones, and consistently outperforms the original GAT methods.\n\n**W2: Experimental settings of SNN baselines for link prediction**\n\nFor the SNNs generating spiking representations (i.e., SpikeGCN and SpikeGCL), we feed the spiking representation into Fermi-Dirac decoder, obtaining the pairwise similarity for link prediction.\n\nFor the SNNs that cannot generate spiking representations (i.e., SpikeNet and SpikeGraphormer), we do not report the results of link prediction, since there exits no well-recognized similarity for the spikes.\n\n**Q1: The additional parameters and energy consumption by adding Riemannian forwarding.**\n\nThe proposed MSG does not introduce additional parameters in the Riemannian forwarding.\n\nThe proposed MSG leverages the tangent vector, rather than the manifold representations, for model inference. Thus, only limited float-point operations are involved. \n\nWe report the theoretical energy consumption (mJ) in Table 3and, in fact, MSG presents competitive and even better energy efficiency to the previous Spiking GNNs, and obtain superior accuracy.\n\n**Q2: In model training, is DvM’s backward time of independent from the steps of spikes?**\n\nThe gradient backpropagation is introduced in Theorem 4.1, and DvM relates to the step of spikes ONLY in Eq. (7), or more specifically, the derivate of $\\frac{\\partial \\mathbf{v}_i}{\\partial {\\mathbf{x}_i[t]}}$.\nNote that, the pooling of Eq. (7) is the mean of $x$ sequence, and the derivative is simple and very fast to compute.\n\nIn contrast, the traditional BPTT methods recursively compute the whole backpropagation process for each time step. \n\nConsequently, the proposed DvM achieves the significant less training time to BPTT, as shown in Fig 4.\n\n**Q3: Specify the manifold in Figure 5.**\n\nFig. 5 is a torus, generated by rotating a circle around an axis coplanar with the circle. \n\nThe equation of the torus is specified as follows,\n$$\n\\begin{array}{lll}\nx(\\theta, \\phi) = (R + r \\cos\\theta) \\cos\\phi, \\\\\\\\\ny(\\theta, \\phi) = (R + r \\cos\\theta) \\sin\\phi, \\\\\\\\\nz(\\theta, \\phi) = r \\sin\\theta,\n\\end{array}\n$$\nwhere $x(\\theta, \\phi)$, $y(\\theta, \\phi)$, and $z(\\theta, \\phi)$ represent the three-dimensional coordinates of any point on the surface of the torus.\n\n$R$ is the radius of the major circle, which is the distance from the center of the torus to the center of the minor circle.\n\n$r$ is the radius of the minor circle, which is the distance from the center of the minor circle to the surface of the torus.\n\n$\\theta$ is the angle parameter for points on the minor circle, used to describe the position of points on the minor circle.\n\n$\\phi$ is the angle parameter for the position of the minor circle's center on the major circle, used to describe the location of the minor circle center along the path of the major circle.\n\nEquivalently, a torus is constructed as the product of two circles: $S^1 \\times S^1$.\n\n**Q4: The difference between this paper and the recent paper (Continuous spiking graph neural networks) regarding ODE.**\n\n''Continuous spiking graph neural networks''' is the reference [11] of our paper. [11] study the connections between spiking neural network, continuous GNN, and ODE in Euclidean space, while we relate GNN and Manifold ODE, as stated in Sec. 5.\n\nMore specifically, [11] establishes ODE w.r.t. to the **spiking** representations, while we consider the ODE w.r.t. the **manifold** representations, and the spikes are related to the Charts as elaborated in Sec. 5.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thanks for the appreciation and detailed comments of all the reviewers! Tables in the rebuttal are also given in the PDF for better readability.



All authors of submission 6586.

Meta-review: I have read all comments and responses. Reviews appear to be consentaneous, with five scores of 8, 8, 8, 6, and 6. All concerns of reviewers have been fixed. It is recommended to accept this manuscript.
Decision: Accept (poster)