Title: Spiking Graph Neural Network on Riemannian Manifolds
Venue: NeurIPS 2024
Accepted: yes
Keywords: Graph Neural Network

Spiking Neural Network

Riemannian Geometry
Abstract: Graph neural networks (GNNs) have become the dominant solution for learning on graphs, the typical non-Euclidean structures. Conventional GNNs, constructed with the Artificial Neuron Network (ANN), have achieved impressive performance at the cost of high computation and energy consumption. In parallel, spiking GNNs with brain-like spiking neurons are drawing increasing research attention owing to the energy efficiency. So far, existing spiking GNNs consider graphs in Euclidean space, ignoring the structural geometry, and suffer from the high latency issue due to Back-Propagation-Through-Time (BPTT) with the surrogate gradient. In light of the aforementioned issues, we are devoted to exploring spiking GNN on Riemannian manifolds, and present a Manifold-valued Spiking GNN (MSG). In particular, we design a new spiking neuron on geodesically complete manifolds with the diffeomorphism, so that BPTT regarding the spikes is replaced by the proposed differentiation via manifold. Theoretically, we show that MSG  approximates a solver of the manifold ordinary differential equation. Extensive experiments on common graphs show the proposed MSG achieves superior performance to previous spiking GNNs and energy efficiency to conventional GNNs.

Spiking Graph Neural Network on Riemannian Manifolds

Abstract

Graph neural networks (GNNs) have become the dominant solution for learning on graphs, the typical non-Euclidean structures. Conventional GNNs, constructed with the Artificial Neuron Network (ANN), have achieved impressive performance at the cost of high computation and energy consumption. In parallel, spiking GNNs with brain-like spiking neurons are drawing increasing research attention owing to the energy efficiency. So far, existing spiking GNNs consider graphs in Euclidean space, ignoring the structural geometry, and suffer from the high latency issue due to Back-Propagation-Through-Time (BPTT) with the surrogate gradient. In light of the aforementioned issues, we are devoted to exploring spiking GNN on Riemannian manifolds , and present a Manifold-valued Spiking GNN ( MSG ). In particular, we design a new spiking neuron on geodesically complete manifolds with the diffeomorphism, so that BPTT regarding the spikes is replaced by the proposed differentiation via manifold. Theoretically, we show that MSG approximates a solver of the manifold ordinary differential equation. Extensive experiments on common graphs show the proposed MSG achieves superior performance to previous spiking GNNs and energy efficiency to conventional GNNs.

1 Introduction

Graphs are the ubiquitous, non-Euclidean structures that describe the relationship among objects. Graph neural networks (GNNs), constructed with the floating-point Artificial Neuron Network (ANN), have achieved state-of-the-art accuracy for learning on graphs [1; 2; 3; 4]. However, they raise the concerns about computation and energy consumption, particularly when dealing with real-world graphs of considerable scale [5; 6]. In contrast, Spiking Neuron Networks (SNNs), inspired by the biological mechanism of brains, utilize neurons that communicate using sparse and discrete spikes, showcasing their superiority in energy efficiency [7; 8]. Attempting to bring the best of both worlds, spiking GNNs are drawing increasing research attention.

In the literature of spiking GNNs, recent efforts have been made to design different architectures with spiking neurons, e.g., graph convolution [5], attention mechanism [9], variational autoencoder [10] and continuous GNN [11]. While achieving encouraging results, existing spiking GNNs still face several fundamental issues: (1) Representation Space. Spiking GNNs consider the graph in Euclidean space, ignoring the inherent geometry of graph structures. Unlike the Euclidean structures (e.g., pixel matrix and grid structures), graphs cannot be embedded in Euclidean space with bounded distortion [12]. Instead, Riemannian manifolds have been shown as the promising spaces to model graphs in recent years [3; 13; 4] (e.g., hyperbolic space, a type of Riemannian manifolds, is well aligned with the graphs dominated by hierarchical structures). However, none of the existing works study SNN on Riemannian manifolds, to the best of our knowledge. It is thus an interesting and urgent problem to consider how to endow the spiking GNN with a Riemannian manifold. (2) Training Algorithm. Training spiking GNN is challenging, since the spikes are non-differentiable. Existing studies consider the spiking GNN as a recurrent neural network and apply Backward-PassingThrough-Time (BPTT) with the surrogate gradient [5; 9; 10; 11]. They recurrently compute the backward gradient at each time step, and thus suffer from the high latency issue [14; 15; 16; 6] especially when the spike trains are long.

Present work. Deviating from previous spiking GNNs in Euclidean space, in this paper, we open a new direction to explore spiking GNNs on Riemannian manifolds, and propose a novel Manifold-valued Spiking GNN ( MSG ) sketched in Fig. 1. It is not realistic to place spike trains in a manifold such as hyperbolic or hyperspherical space, given the fact that spike trains cannot align with the defining domain. Instead, we design a Manifold Spiking Layer that conducts parallel forwarding of spike trains and manifold representations. Specifically, we first incorporate the structural information

𝒗 𝟎

𝑧 0 ∈ ℳ

𝑠 0

𝑧 𝐿 ∈ ℳ

𝛻 𝑧 𝐿(ℒ)

…

Forward

Backward

𝑠 𝐿

…

𝒢

ℒ

into spike trains by graph convolution. Then, a new manifold spiking neuron is proposed to emit spike trains and relate them to manifold representations with diffeomorphism , where the spike train generates a momentum that forwards manifold representation along the geodesic. Instead of applying BPTT in spike domain, the proposed neuron provides us with an alternative of Differentiation via Manifold ( DvM ). (The red dashed line in Fig. 1.) Yet, differentiation in Riemannian manifold is nontrivial. We leverage the properties of pullback and derive the closed-form backward gradient (Theorem 4.1). DvM enables the recurrence-free gradient backpropagation, which no longer needs to perform recurrent computation of time steps as in BPTT. Theoretically, MSG is essentially related to manifold Ordinary Differential Equation (ODE). Each layer creates a chart of the manifold, and MSG approximates the dynamic chart solver [17] of manifold ODE (Theorem 5.2).

Contributions. Overall, the key contributions are summarized as follows: (1) To the best of our knowledge, we propose the first spiking neural network on Riemannian manifolds ( MSG ) 2 , and show its connection to manifold ODE theoretically. (2) We design a new training algorithm of differentiation via manifold, which avoids the high latency of BPTT methods. (3) Extensive experiments show the superior effectiveness and energy efficiency of the proposed MSG .

2 Related Work

We briefly overview the ANN-based GNNs (i.e., the conventional, floating-point GNNs living in either Euclidean space or Riemannian manifolds) and SNN-based GNNs (i.e., spiking GNNs).

ANN-based GNNs (Euclidean and Riemannian). The majority of GNNs are built with floatingpoint ANN, conducting message passing on the graphs [18; 2; 19]. The Euclidean space has been the

workhorse for graph representation learning for decades, and the popular GCN [18], GAT [2] and SGC [19] are also designed in the Euclidean space. In recent years, Riemannian manifolds have emerged as an exciting alternative considering the geometry of graph structures [20; 21]. Among Riemannian manifolds, hyperbolic space is recognized for its alignment with the graphs of hierarchical structures, and a series of hyperbolic GNNs (e.g., HGNN [22], HGCN [3]) show superior performance to their Euclidean versions. Beyond hyperbolic space, hyperspherical space is well suited for cyclical structures [23], and recent studies further investigate the constant curvature spaces [13], product spaces [24; 25; 26; 27], quotient spaces [28], SPD manifolds [29; 30], etc. Riemannian manifolds achieve remarkable success in graph clustering [31; 32], structural learning [33], graph dynamics [34; 35; 36; 37], information diffusion [38] and graph generation [39; 40], but have rarely been touched yet in the SNN counterpart.

Spiking Neural Networks (SNNs) & Spiking GNNs. Mimicking the biological neural networks, SNNs [7; 8] utilize the spiking neuron to process spike trains, and offer the advantage of energy efficiency. Despite the wide application of SNN in computer vision [41; 42], SNNs are still at an early stage in the graph domain. The basic idea of spiking GNNs is adapting ANN-based GNNs to the SNN framework by substituting the activation functions with spiking neurons. Pioneering works study the graph convolution [43; 5], and efforts have also been made to the graph attention [9], variational graph autoencoder [10], graph differential equations [44], etc. SpikeGCL [6] is a recent endeavor to conduct graph contrastive learning with SNN. In parallel, spiking GNNs are extended to model the dynamic graphs [45; 46; 47]. We focus on the static graph in this work. In both dynamic and static cases, previous spiking GNNs are trained with the surrogate gradient, leading to high latency, and consider the graphs in the Euclidean space.

3 Preliminaries

Different from aforementioned spiking GNNs, we study the spiking GNN on Riemannian manifolds. Thus, we formally introduce the basic concepts of Riemannian geometry and SNN. Throughout this paper, the lowercase boldfaced x and uppercase X denote vector and matrix, respectively. Important notations are summarized in Appendix A.

Riemannian Geometry & Riemannian Manifold. Riemannian geometry provides elegant framework to study structures and manifolds. A Riemannian manifold is described as a smooth and real manifold M endowed with a Riemannian metric. Each point x in the manifold is associated with the tangent space T x M that 'looks Euclidean', and the Riemannian metric is given by the inner product in the tangent space, so that geometric properties (e.g, angle, length) can be defined. A geodesic between two points on the manifold is the smooth path connecting them with the minimal length. There exist three types of isotropic manifold, namely, the Constant Curvature Space (CCS): hyperbolic space H , hyperspherical space S and the special case of Euclidean space with 'flat' geometry E .

Graph & Riemannian Graph Representation Learning. A graph G = ( V , E , F , A ) is defined on the node set V and edge set E ⊂ V × V , and A ∈ R |V|×|V| is the adjacency matrix describing the structure information. Each node v i is associated with a feature f i , and node features are summarized in F ∈ R |V|× d . In this paper, we resolve the problem of Riemannian Graph Representation Learning with SNN. Specifically, we seek a graph encoder F θ : v ↦→ z where z ∈ M is a point on the manifold, instead of Euclidean space, and F θ is defined with an energy-efficient SNN.

Spiking Neural Network. SNNs are constructed by spiking neurons that communicate with each other by spike trains. Concretely, a spiking neuron is conceptualized as 'a capacitor of the membrane potential', and processes the spike trains by the following 3 phases [48]. First, the incoming current I [ t ] is accumulated in the capacitor, leading to the potential buildup ( integrate ). When the membrane potential V [ t ] reaches or exceeds a specific threshold V th , the neuron emits a spike ( fire ). After that, the membrane potential is reset to the resting potential V reset ( reset ). There are two popular spiking neurons: IF model and LIF model [49]. In particular, the three phases of IF model are formalized as

where the incoming current I [ t ] is related to the input spike train, and V reset is lower than V th . t denotes the time index of the spike. The Heaviside function H ( · ) is non-differentiable, H ( x ) = 1 if x ≥ 0 , and 0 otherwise. There are two options for reset, and fixed-reset is adopted in this paper. Overall, an IF model is given as S [ t ] = IFModel( I [ t ]) , and the only difference between IF model and LIF model lies in the definition of g ( · ) in Eq. (1). In this paper, we are interested in designing a new spiking neuron on Riemannian manifold.

4 Methodology: Manifold-valued Spiking GNN

In this section, we present a simple yet effective Manifold-valued Spiking GNN ( MSG ), which can be applied to any geodesically complete manifolds, e.g., the Constant Curvature Space (CCS), including hyperbolic space and hyperspherical space, or the product of CCS. In particular, we design a spiking neuron on Riemannian manifolds (named as Manifold Spiking Neuron ) that allows for the differentiation via manifold . It provides a new perspective of training spiking GNN, so that we avoid the high latency of typical backward-passing-through-time (BPTT) training.

4.1 Manifold Spiking Layer

GCN

𝑠 𝑙-1

IFModel

Pooling

𝑓

( ·  )

𝑠 𝑙

Forward

Backward

𝑣 𝑙-1

𝑧 𝑙

𝑧 𝑙-1

𝛻 𝑧 𝑙(ℒ)

𝛻 𝑧 𝑙-1(ℒ)

𝜕𝑣 𝑙-1

𝜕𝑊 𝑙

𝐷 𝑣 𝑙-1 (𝜙 𝑙-1

)

𝑧 𝑙-1

𝑥 𝑙-1

We elaborate on the sole building block of the proposed model - Manifold Spiking Layer. Note that, the spike train or spiking representation in existing spiking GNNs [43; 5; 9; 10; 6; 45; 46; 47] cannot align with the defining domain of Riemannian manifolds (e.g., hyperbolic space and hyperspherical space), thus posing a fundamental challenge. Our solution is to generate node representation on the manifold (referred to as manifold representation) in parallel, and leverage the notion of Diffeomorphism to create the alignment between the two domains. We formulate the parallel forwarding of spike trains and manifold representations as follows.

Unified Formulation. The forward pass of the spiking layer consists of a graph convolution and one proposed manifold spiking neuron. Without loss of generality, for each node v i ∈ G , the l -th spiking layer is formulated as follows,

where x is the incoming current to generate spike trains. s and z denote the spike trains and manifold representation, respectively. GCN( · ) is a GNN in the Euclidean space, and W l is the learnable parameter in the layer. Different from the neuron of previous spiking GNNs, we design a novel manifold spiking neuron ( MSNeuron ) as shown in Fig. 2. It emits spike trains and relates them to manifold representations simultaneously, which is formulated as follows,

where t is the time step of spike trains. The IF model can be replaced by LIF model, and we utilize IF model by default for simplicity. Pooling is defined as the average pooling of the current x over t , and v is given as the Euclidean vector. f denotes the diffeomorphism to the Riemannian manifold in which glyph[epsilon1] is the step size.

Incorporating structural information. We inject the structural information when the received spikes transform into the incoming current of the neuron. A GNN is leveraged to define the current, conducting message-passing over the graph. Each node's representation is derived recursively by neighborhood aggregation [18; 2; 1]. Accordingly, GCN in the proposed neuron is given as follows,

where the neighborhood Ω i is the set of immediate neighbors centering at node v i . The aggregate function aggregates the messages from neighborhood Ω i , where we create the message of a node by W l s j [ t ] . combine( · ) denotes the combination of the center node's message and aggregated message. We utilize GCN [18] as the backbone to define aggregate and combine .

Diffeomorphism between manifolds. In the proposed neuron, we bridge the spikes and manifold representation with the notion of diffeomorphism in differential geometry. A diffeomorphism connects two smooth manifolds, saying M and N . Formally, a map f : M→N is a diffeomorphism between M and N if the smooth f is bijective and its inverse f -1 is also smooth.

Recall that the tangent space is locally Euclidean. We propose to place the Euclidean v , a representation of the spikes, in the tangent space T z M of the point z . In MSG , we choose the exponential map to act as the diffeomorphism between the tangent space and manifold. With a step size glyph[epsilon1] , we have

Concretely, given z ∈ M and v ∈ T z M , the exponential map 3 of v at point z , Exp z ( v ) : T z M→ M , maps tangent vector v onto the manifold M . The map pushes z along the geodesic γ z , v ( t ) : [0 , 1] → M starting at γ z , v (0) = z and ending at y = γ z , v (1) . ˙ γ z , v ( t ) denotes the velocity of γ z , v ( t ) , and the direction of geodesic at the beginning is given as ˙ γ z , v (0) = v . That is, the tangent vector v , derived from the spikes, pushes the manifold representation along the geodesic via the exponential map. The advantage of our choice is that we are able to define the diffeomorphism in arbitrary geodesically complete manifold (detailed in Appendix D).

Note that, our idea is inherently different from the exponential/logarithmic based Riemannian GNNs [3; 4; 22], which leverage the tangent space of the origin for neighborhood aggregation. In contrast, we consider the successive process over the tangent spaces of manifold representations, which will be further studied in Sec. 5.

Model Initialization In MSG , we need to simultaneously initialize the spiking input S 0 and manifold representation Z 0 , which is a collection of points on the given manifold. Given a graph G ( V , E , F , A ) , the node features are first encoded by one graph convolution layer H = GCN( A , F ; W 0 ) , and we generate T copies of the node encodings H , where T is the number of time steps in spike trains. Then, we complete model initialization with the proposed manifold neuron [ S 0 , Z 0 ] = MSNeuron( H , O ) , where the encoding H is regarded as the incoming current that charges the neuron in each time step. O consists of the original points of the manifold, e.g., in the sphere model of hyperspherical space, the original point is given as the south pole o = [ -1 , 0 , ..., 0] glyph[latticetop] and O = [ o glyph[latticetop] , ..., o glyph[latticetop] ] glyph[latticetop] . Note that, the exponential map in the proposed neuron guarantees that Z 0 lives in the manifold.

4.2 Learning Approach: Differentiation via Manifold

Optimizing SNNs is challenging, as the Heaviside step function is non-differentiable. In the literature, existing spiking GNNs typically regard SNN as the recurrent neural network, and leverage backwardpassing-through-time (BPTT) to train the model [50; 51; 52]. Concretely, given a real loss function L , the gradient backpropagation conducts Differentiation via Spikes ( DvS ) s as follows,

where W is the parameter, and t denotes the time step. The surrogate gradient [51] is required for D W l s l [ t ] , where Heaviside step function is replaced by a differentiable surrogate, e.g., sigmoid function. The differentiation via spikes presents high latency in the backward pass [14; 15; 16], as it needs to recur all the time steps in BPTT. We notice that, in the computer vision domain, the sub-gradient method [15] is proposed to address such issues in Euclidean space. However, it cannot be generalized to the Riemannian manifold since the linearity does not hold in Riemannian geometry.

In MSG , we decouple the forward pass and backward pass, and propose Differentiation via Manifold ( DvM ) to avoid the high latency in differentiation via spikes. The overall procedure of training MSG by the proposed learning approach is summarized in Algorithm 1. Thanks to the parallel forwarding of spikes and manifold representation, the proposed neuron provides us with an alternative of studying ∇ W l L through the forwarding pass on the manifold (i.e., differentiation via manifold). Nevertheless, it is nontrivial and it requires to derive the pullback between different dual spaces.

Pushforward, Pullback and Dual Space. We first introduce the differentiation in Riemannian geometry which is essentially different from that in Euclidean space. In Riemannian geometry, a pushforward refers to a derivative of a map connecting two manifolds M and N . Concretely, given f : M→N and a point z ∈ M , the pushforward D z f maps a tangent vector v ∈ T z M to the tangent vector D z f ( v ) ∈ T f ( z ) N . On the notation, for a manifold-valued function f ( z ) =

Algorithm 1 Training MSG by the proposed Differentiation via Manifold

Input: Graph G ( V , E , F , A ) , Manifold M , Loss function over the manifold L ( · ) , Number of spiking layers L , Original points O .

p ∈ N , ∂ p /∂ z is equivalent to D z f . For a scalar function f , D z f is interchangeable with ∇ z f .

In the proposed MSG , we consider a scalar loss function on the manifold L : M→ R . The pushforward D z L at point z ∈ M maps tangent vector v ∈ T z M to a scalar value and, correspondingly, D z L belongs to the dual space of the tangent space T ∗ z M , which is a vector space consisting all linear functional F : T z M → R . As the tangent spaces at different points of the manifold are different, it requires a pullback that maps the dual space T ∗ z l M to the dual space T ∗ z ( l +1) M .

Wederive the backward gradient with properties of differential 1 -form (Lemma B.1), communication (Lemma B.2), and pullback of a sum and a product (Lemma B.3) detailed in Appendix B.1.

Theorem 4.1 (Backward Gradient) . Let L be the scalar-valued function, and z l is the output of l -th layer with parameter W l , which is delivered by tangent vector v l . Then, the gradient of function L w.r.t W l is given as follows:

The detailed proof is given in Appendix B.1, and we derive the two Jacobian matrices D v l -1 φ l -1 and D z l ψ l in Appendix C. There are three key advantages of the proposed DvM . First, every term in Equation (12) is differentiable , and thereby the surrogate gradient is no longer needed. Second, DvM enables the recurrence-free backward pass alleviating the high latency training. We specify that both DvM and the previous DvS recurrently compute every time step in the forward pass, and the difference lies in the backward pass. In particular, we only conduct recurrence-free gradient backpropagation layer by layer, while the previous DvS recurs every time step of each layer in BPTT. In addition to the differentiable and recurrence-free properties, DvM does not suffer from gradient vanishing/explosion, and the empirical evidence is provided in Appendix F.

5 Theory: MSG as Neural ODE Solver

Next, we demonstrate the theoretical aspects of our model that MSG approximates a solver of manifold Ordinary Differential Equations (ODEs).

𝑣 2

𝑍2

𝑣 1

𝑍1

𝑣 0

𝑍0

𝑍0

𝑍1

𝑍2

y ( t ) : [ τ, τ + glyph[epsilon1] ] → R n is the solution of

then y ( t ) = φ i ( z ( t )) is a valid solution of Eq. (13) on t ∈ [ τ, τ + glyph[epsilon1] ] .

Definition 5.1 (Dynamic Chart Solver [17]) . The manifold ODE in Eq. (13) with initial condition z (0) = z can be solved with a finite collection of successive charts { ( U i , φ i ) } i =1 ,...,L . If ode i is the numerical solver to Euclidean ODE corresponding to the i -th chart, y ( t ) = ode i ( t ) on [ τ i , τ i + glyph[epsilon1] i ] , then z ( t ) in Eq. (13) is given as

That is, a manifold ODE can be solved in Euclidean subspaces given by a series of successive charts. In MSG , we consider the charts given by the logarithmic map as illustrated in 3, and we prove that MSG approximates a dynamic chart solver of manifold ODE (Theorem 5.2).

Theorem 5.2 ( MSG as Dynamic Chart Solver) . If y ( t ) : [ τ, τ + glyph[epsilon1] ] → R n is the solution of

then z ( t ) = Exp z ( y ( t )) is a valid solution to the manifold ODE of Eq. (13) on t ∈ [ τ, τ + glyph[epsilon1] ] , where z = z ( τ ) . If y ( t ) is given by the first-order approximation with the glyph[epsilon1] small enough,

then the update process of Eqs. (4) and (8) in MSG is equivalent to Dynamic Chart Solver in Eq. (15).

Proof. The proof utilizes some facts of Riemannian manifolds and is detailed in Appendix B.2.

In other words, in MSG , the transformation of manifold input and output is described as some manifold ODE, whose vector field is governed by a spiking-related neural network in the tangent bundle. To solve the manifold ODE, MSG leverages the Dynamic Chart Solver (Definition 5.1). Specifically, each manifold spiking layer corresponds to a chart, and thus the number of spiking layers equals to the number of charts. Each layer solves the ODE of a smooth path y ( t ) : [ τ, τ + glyph[epsilon1] ] → R n in the tangent space that centered at the manifold layer input. With the first-order approximation in Theorem 5.2, given a step size glyph[epsilon1] , the endpoint y ( τ + glyph[epsilon1] ) of the path is parameterized by a GNN related to the spikes. Layer-by-layer forwarding solves the manifold ODE from the current chart to the successive chart. Consequently, the manifold output of MSG approximates the solution to the manifold ODE.

We notice that a recent work [11] connects spiking GNN to an ODE in Euclidean space. In contrast, the proposed MSG is essentially related to the manifold ODE.

The Appendix contains the proofs, the derivation of Jacobian, necessary facts on Riemannian geometry (i.e., Lorentz/Sphere model, stereographic projection and κ -stereographic model, and Cartesian product and product space), empirical details and additional results.

We leverage the notion of chart to study the relationship between MSG and manifold ODE. A manifold ODE defined as

describes a vector field u that maps a smooth path z ( t ) : [0 , 1] → M to the tangent bundle T M 4 . In other words, the vector field u assigns each point z ( t ) ∈ M to a tangent vector u ( z ( t ) , t ) ∈ T z ( t ) M . A chart at z , denoted as ( U z , φ z ) , is a smooth bijection φ z between z 's neighborhood U z ⊂ M and a subspace of Euclidean space. Thus, the chart relates Eq. (13) to ODE in Euclidean space [17]. Specifically, if

6 Experiments

We conduct extensive experiments with 12 strong baselines to evaluate the proposed MSG in terms of (1) the representation effectiveness, (2) the energy efficiency, and (3) the advantages of the proposed components. Additional results are presented in Appendix F.

6.1 Experimental Setups

Datasets. Our experiments are conducted on 4 commonly used benchmark datasets including two popular co-purchase graphs: Computers and Photo [53], and two co-author graphs: CS and Physics [53]. Datasets are detailed in Appendix E.

Baselines. We compare the proposed MSG with 12 strong baselines of three categories: (1) ANNbased Euclidean GNNs : the popular GCN [18], GAT [2], GraphSAGE [1] and SGC [19], (2) ANN-based Riemannian GNNs : HGCN [3] and HyboNet [54] of hyperbolic spaces, κ -GCN [13] of the constant curvature space, and the recent Q -GCN [4] of the quotient space, (3) The previous Euclidean Spiking GNNs : SpikeNet [45], SpikeGCN [5], SipkeGraphormer [55] (termed as SpikeGT for short) and the recent SpikeGCL [6]. Note that, we focus on the graph representation learning on static graphs, and thereby graph models for the dynamic ones are out of the scope of this paper. SpikeNet [45] was originally designed for dynamic graphs, and we utilize its variant for static graphs according to [6]. So far, spiking GNN has not yet been connected to Riemannian manifolds, and we are devoted to bridging this gap.

Evaluation Protocol. All models are evaluated by node classification and link prediction tasks. The evaluation metrics of node classification is classification accuracy; we employ the popular Area Under Curve (AUC) for link prediction. The hyperparameter setting is the same as the original papers. We perform 10 independent runs for each case, and report the mean with standard derivations. Experiments are conducted on the hardware of NVIDIA GeForce RTX 4090 GPU 24GB memory, and AMDEPYC 9654 CPU with 96-Core Processor. Our model is built upon GeoOpt [56], SpikingJelly [56] and PyTorch [57].

Model Instantiation & Configuration. Note that, the proposed MSG applies to any Constant Curvature Space (CCS) or the product of CCS. We instantiate MSG in the Lorentz model of hyperbolic space by default (whose Riemannian metric, exponential map, and the derived Jacobian is given in Appendix C), and study the impact of representation space in the Ablation Study. The dimension of the representation space is set as 32 . The manifold spiking neuron is based on the IF model [49] by default, and it is ready to switch to the LIF model [49] whose results are given in Appendix F. The time steps T for neurons is set to 5 or 15 . The step size glyph[epsilon1] in Eq. 8 is set to 0 . 1 . The hyperparameters are tuned with grid search, in which the learning rate is { 0 . 01 , 0 . 003 } for node classification and { 0 . 003 , 0 . 001 } for link prediction, and the dropout rate is in { 0 . 1 , 0 . 3 , 0 . 5 } . We provide the source code of MSG at the anonymous link https://github.com/ZhenhHuang/MSG .

6.2 Results & Discussion

Effectiveness. We evaluate the effectiveness of MSG in both node classification and link prediction tasks. Specifically, for node classification, we cannot directly feed the manifold representations of Riemannian baselines to a softmax layer with Euclidean measure. We bridge the manifold representation and Euclidean softmax with the logarithmic map of respective manifold. For link prediction, we utilize the generalized sigmoid for all the baselines, i.e., the Fermi-Dirac decoder [3] in which the distance function is defined under the respective geometry. In the proposed MSG, the model inference does not need the expensive successive exponential maps, and only limited float-point operations (i.e., addition) are involved. Accordingly, we leverage the tangent vectors for the downstream tasks. The performance of both learning tasks on Computer, Photo, CS and Physics datasets are collected in Table 1. Note that, SpikeNet and SpikeGT cannot do link prediction, since they are designed for node classification and do not offer spiking representation. The proposed MSG consistently achieves the best results among SNN-based models. In addition, MSG generally outperforms the best ANN-based baselines in node classification, and has competitive results to the recent ANN-based Riemannian baselines in link prediction.

Ablation Study. Here, we examine the impact of representation space and the effectiveness of the proposed Differentiation via Manifold (DvM) . For the former goal, we instantiate 6 geometric variants of MSG in hyperbolic space H 32 , hyperspherical space S 32 , Euclidean space E 32 and the products of H 16 × H 16 , H 16 × S 16 and S 16 × S 16 . The superscript denotes the dimension

of representation space, and we leverage DvM for optimization. Manifold variants generally achieve superior results to the Euclidean one, thus verifying our motivation. On CS dataset, the performance of geometric variants is aligned with that of Euclidean and Riemannian baselines in Table 1. The proposed MSG is ready to switch among H , S , E , and their products, matching the geometry of graphs.

30

50

70

100

Time Steps

0.0

0.5

1.0

1.5

2.0

Backward time (s)

0.04

0.04

0.04

0.06

0.05

0.11

0.61

2.33

Lorentz

DvM

Surrogate

30

50

70

100

Time Steps

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Backward time (s)

0.01

0.01

0.01

0.03

0.02

0.06

0.16

1.40

Sphere

DvM

Surrogate

0

25

50

75

100

125

150

175

200

Iterations

5

10

15

20

25

30

Norm * 1e-5

gradient norm of v in layer 0

0

25

50

75

100

125

150

175

200

Iterations

5

10

15

20

25

30

Norm * 1e-5

gradient norm of v in layer 1

(a) Backward times in model training.

(b) Gradient norm of

L

regarding tangent vector

v

.

To examine the effectiveness of DvM , we design the optimization variant (named as Surrogate) for a given representation space. In the variant, we conduct differentiation via spikes and leverage BPTT for optimization, same as previous spiking GNNs. The training time of the optimization variants in different representation spaces are given in Fig. 4(a). Backward time of DvM is significantly less than that of BPTT algorithm. The reason is that DvM no longer needs recurrent gradient calculation of each time step (recurrence-free), while BPTT leads to high training time especially when the time step is large. In addition, we examine the backward gradient of DvM , and plot the gradient norm of each layer in Fig. 4(b). It demonstrates that DvM does not suffer from gradient vanishing/explosion.

Energy Cost. We investigate the energy cost of the graph models in terms of theoretical energy consumption (mJ) [5; 6], whose formula is specified in Appendix E. We summarize the results for node classification in Table 3 in which the number of parameters at the running time is listed as a reference. It shows that SNN-based models generally enjoy less energy cost than ANN-based ones.

30

25

20

15

Norm * 1e-5

10

5

0

gradient norm of v in layer 2

25

50

75

100

125

Iterations

150

175

200

Note, MSG achieves the best energy efficiency among SNN-based models except Photo dataset. In addition, it has at least 1 / 20 energy cost to the Riemannian baselines.

Visualization & Discussion. We empirically study the connection between the proposed MSG and manifold ODE. In particular, we visualize a toy example of Zachary Karate Club dataset [58] on a S 1 × S 1 in Fig. 5, where we plot each layer output on the manifold. The red curve is the path connecting the layer input and layer output, and the blue one is the direction of the geodesic. As shown in Fig. 5, the red and blue curves are coincided, that is, each layer solves an ODE describing the geodesic on the manifold.

7 Conclusion

In this paper, we study spiking GNN from a fundamentally different perspective of Riemannian geometry, and present a simple yet effective Manifold-valued Spiking GNN ( MSG ). Concretely, we design a manifold spiking neuron which

leverages the diffeomorphism to bridge spiking representations and manifold representations. With the proposed neuron, we propose a new training algorithm with Differentiation via Manifold, which no longer needs to recur the backward gradient and thus alleviates the high latency of previous methods. An interesting theoretical result is that, MSG is essentially related to manifold ODE. Extensive empirical results on benchmark datasets demonstrate the superior effectiveness and energy efficiency of the proposed MSG .

8 Broader Impact and Limitations

Our work brings together two previously separate domains: spiking neural network and Riemannian geometry, and presents a novel Manifold-valued Spiking GNN for energy-efficiency graph learning, especially for the large graphs. Our work is mainly a theoretical exploration, and not tied to particular applications. A positive societal impact is the possibility of decreasing carbon emissions in training large models. None of negative societal impacts we feel must be specifically highlighted here.

Limitation. Our work as well as the previous spiking GNNs considers the undirected, homophilous graphs, while the spiking GNN on directed or heterophilous graphs still remains open. Also, readers may find it challenging to implement the proposed method. However, we provide downloadable code and will offer an easy-to-use interface.

References

Hamilton, W., Z. Ying, J. Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems , 30, 2017.

Veliˇ ckovi´ c, P., G. Cucurull, A. Casanova, et al. Graph Attention Networks. International Conference on Learning Representations , 2018. Accepted as poster.

Chami, I., Z. Ying, C. Ré, et al. Hyperbolic graph convolutional neural networks. In Advances in NeurIPS , vol. 32. 2019.

Xiong, B., S. Zhu, N. Potyka, et al. Pseudo-riemannian graph convolutional networks. In Advances in NeurIPS , vol. 32. 2022.

Zhu, Z., J. Peng, J. Li, et al. Spiking graph convolutional networks. In Proceedings of the 31st IJCAI , pages 2434-2440. ijcai.org, 2022.

Li, J., H. Zhang, R. Wu, et al. A Graph is Worth 1-bit Spikes: When Graph Contrastive Learning Meets Spiking Neural Networks. arXiv preprint arXiv:2305.19306 , 2023.

Maass, W. Networks of spiking neurons: The third generation of neural network models. Neural networks , 10(9):1659-1671, 1997.

Brette, R., M. Rudolph, T. Carnevale, et al. Simulation of networks of spiking neurons: A review of tools and strategies. Journal of computational neuroscience , 23:349-398, 2007.

Wang, B., B. Jiang. Spiking GATs: Learning graph attentions via spiking neural network. arXiv preprint arXiv:2209.13539 , 2022.

Yang, H., R. Zhang, Q. Kong, et al. Spiking variational graph auto-encoders for efficient graph representation learning. arXiv preprint arXiv:2211.01952 , 2022.

Yin, N., M. Wang, L. Shen, et al. Continuous spiking graph neural networks. CoRR , abs/2404.01897, 2024.

Sarkar, R. Low distortion delaunay embedding of trees in hyperbolic plane. In Proceedings of the 19th International Symposium on Graph Drawing , vol. 7034 of Lecture Notes in Computer Science , pages 355-366. Springer, 2011.

Bachmann, G., G. Bécigneul, O. Ganea. Constant curvature graph convolutional networks. In Proceedings of the 37th ICML , vol. 119, pages 486-496. PMLR, 2020.

Wu, H., Y. Zhang, W. Weng, et al. Training Spiking Neural Networks with Accumulated Spiking Flow. Proceedings of the AAAI Conference on Artificial Intelligence , 35(12):10320-10328, 2021.

Meng, Q., M. Xiao, S. Yan, et al. Training high-performance low-latency spiking neural networks by differentiation on spike representation. In Proceedings of CVPR , pages 1243412443. IEEE, 2022.

Wu, J., Y. Chua, M. Zhang, et al. A Tandem Learning Rule for Effective Training and Rapid Inference of Deep Spiking Neural Networks. IEEE Transactions on Neural Networks and Learning Systems , 34(1):446-460, 2023.

Lou, A., D. Lim, I. Katsman, et al. Neural manifold ordinary differential equations. In Advances in NeurIPS . 2020.

Kipf, T. N., M. Welling. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th ICLR . OpenReview.net, 2017.

Wu, F., A. Souza, T. Zhang, et al. Simplifying graph convolutional networks. In International conference on machine learning , pages 6861-6871. PMLR, 2019.

Feng, Y., H. You, Z. Zhang, et al. Hypergraph Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence , 33(01):3558-3565, 2019.

Guo, K., Y. Hu, Y. Sun, et al. Hierarchical Graph Convolution Network for Traffic Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence , 35(1):151-159, 2021.

Liu, Q., M. Nickel, D. Kiela. Hyperbolic Graph Neural Networks. In Advances in Neural Information Processing Systems , vol. 32. Curran Associates, Inc., 2019.

Coors, B., A. P. Condurache, A. Geiger. SphereNet: Learning spherical representations for detection and classification in omnidirectional images. In Proceedings of ECCV , pages 518-533. 2018.

Gu, A., F. Sala, B. Gunel, et al. Learning mixed-curvature representations in product spaces. In Proceedings of the 7th ICLR . OpenReview.net, 2019.

Zhang, S., Y. Tay, W. Jiang, et al. Switch Spaces: Learning product spaces with sparse gating. CoRR , abs/2102.08688, 2021.

Sun, L., Z. Zhang, J. Ye, et al. A self-supervised mixed-curvature graph neural network. In Proceedings of the 36th AAAI , pages 4146-4155. 2022.

Sun, L., Z. Huang, Z. Wang, et al. Motif-aware riemannian graph neural network with generativecontrastive learning. In Proceedings of the 38th AAAI , pages 9044-9052. 2024.

Law, M. Ultrahyperbolic neural networks. In Advances in NeurIPS . 2021.

Gao, Z., Y. Wu, Y. Jia, et al. Learning to optimize on SPD manifolds. In Proceedings of CVPR , pages 7697-7706. Computer Vision Foundation / IEEE, 2020.

Dong, Z., S. Jia, C. Zhang, et al. Deep manifold learning of symmetric positive definite matrices with application to face recognition. In Proceedings of the AAAI Conference on Artificial Intelligence , vol. 31. 2017.

Sun, L., Z. Huang, H. Peng, et al. Lsenet: Lorentz structural entropy neural network for deep graph clustering. In Proceedings of the 41st ICML . OpenReview.net, 2024.

Sun, L., J. Hu, S. Zhou, et al. Riccinet: Deep clustering via a riemannian generative model. In Proceedings of the ACM Web Conference 2024 (WWW'24) , pages 4071-4082. 2024.

Sun, L., Z. Huang, H. Wu, et al. Deepricci: Self-supervised graph structure-feature co-refinement for alleviating over-squashing. In Proceedings of the 23rd ICDM , pages 558-567. 2023.

Sun, L., J. Ye, H. Peng, et al. Self-supervised continual graph learning in adaptive riemannian spaces. In Proceedings of the 37th AAAI , pages 4633-4642. 2023.

Sun, L., Z. Zhang, J. Zhang, et al. Hyperbolic variational graph neural network for modeling dynamic graphs. In Proceedings of the 35th AAAI , pages 4375-4383. 2021.

Sun, L., J. Ye, H. Peng, et al. A self-supervised riemannian gnn with time varying curvature for temporal graph learning. In Proceedings of the 31st ACM CIKM , pages 1827-1836. 2022.

Sun, L., J. Ye, J. Zhang, et al. Contrastive sequential interaction network learning on co-evolving riemannian spaces. Int. J. Mach. Learn. Cybern. , 15(4):1397-1413, 2024.

Sun, L., J. Hu, M. Li, et al. R-ode: Ricci curvature tells when you will be informed. In Proceedings of the ACM SIGIR . 2024.

Wang, Y., S. Zhang, J. Ye, et al. A mixed-curvature graph diffusion model. In Proceedings of the 33rd ACM CIKM , page 2482-2492. ACM, 2024.

Fu, X., Y. Gao, Y. Wei, et al. Hyperbolic geometric latent diffusion model for graph generation. In Proceedings of the 41st ICML . OpenReview.net, 2024.

Cao, Y., Y. Chen, D. Khosla. Spiking deep convolutional neural networks for energy-efficient object recognition. International Journal of Computer Vision , 113:54-66, 2015.

Cao, J., Z. Wang, H. Guo, et al. Spiking denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 4912-4921. 2024.

Xu, M., Y. Wu, L. Deng, et al. Exploiting spiking dynamics with spatial-temporal feature normalization in graph learning. In Proceedings of the 30th IJCAI , pages 3207-3213. ijcai.org, 2021.

Poli, M., S. Massaroli, J. Park, et al. Graph neural ordinary differential equations. arXiv preprint arXiv:1911.07532 , 2019.

Li, J., Z. Yu, Z. Zhu, et al. Scaling up dynamic graph representation learning via spiking neural networks. In Proceedings of the 37th AAAI , vol. 37, pages 8588-8596. 2023.

Zhao, H., X. Yang, C. Deng, et al. Dynamic reactive spiking graph neural network. In Proceedings of the 38th AAAI , vol. 38, pages 16970-16978. 2024.

Yin, N., M. Wang, Z. Chen, et al. Dynamic spiking graph neural networks. In Proceedings of the 38th AAAI , vol. 38, pages 16495-16503. 2024.

Salinas, E., T. J. Sejnowski. Integrate-and-fire neurons driven by correlated stochastic input. Neural computation , 14(9):2111-2155, 2002.

Burkitt, A. N. A review of the integrate-and-fire neuron model: I. homogeneous synaptic input. Biological Cybernetics , 95(1):1-19, 2006.

Huh, D., T. J. Sejnowski. Gradient Descent for Spiking Neural Networks. In Advances in Neural Information Processing Systems , vol. 31. 2018.

Neftci, E. O., H. Mostafa, F. Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Process. Mag. , 36(6):51-63, 2019.

Li, Y., Y . Guo, S. Zhang, et al. Differentiable Spike: Rethinking Gradient-Descent for Training Spiking Neural Networks. In Advances in Neural Information Processing Systems , vol. 34. 2021.

Shchur, O., M. Mumme, A. Bojchevski, et al. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868 , 2018.

Chen, W., X. Han, Y. Lin, et al. Fully hyperbolic neural networks. In Proceedings of the 60th ACL , pages 5672-5686. Association for Computational Linguistics, 2022.

Sun, Y., D. Zhu, Y. Wang, et al. SpikeGraphormer: A high-performance graph transformer with spiking graph attention. arXiv preprint arXiv:2403.15480 , 2024.

Bécigneul, G., O. Ganea. Riemannian adaptive optimization methods. In Proceedings of ICLR . OpenReview.net, 2019.

Paszke, A., S. Gross, F. Massa, et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems , vol. 32. 2019.

Zachary, W. W. An information flow model for conflict and fission in small groups. Journal of anthropological research , 33(4):452-473, 1977.

Tu, L. W. An Introduction to Manifolds . Springer New York, 2011.

Crouch, P. E., R. Grossman. Numerical integration of ordinary differential equations on manifolds. Journal of Nonlinear Science , 3(1):1-33, 1993.

Petersen, P. Riemannian Geometry, 3rd edition . Springer-Verlag, 2016.

Zhou, Z., Y. Zhu, C. He, et al. Spikformer: When spiking neural network meets transformer. arXiv preprint arXiv:2209.15425 , 2022.

The appendix is organized as follows:

Notation table ,

Proofs of Theorem 4.1 and Theorem 5.2,

Derivation of the Jacobian,

Riemannian geometry including geodesically complete manifold, stereographic projection and κ -stereographic model, and Cartesian product and product manifold,

Empirical details , i.e., datasets/baselines description, theoretical energy consumption and implementation notes,

Additional results of link prediction, layer-wise gradient and the visualization.

A Notations

We summarize the important notations of our paper in Table 4.

B Proofs

In this section, we demonstrate the proofs of Theorem 4.1 (Backward Gradient) and Theorem 5.2 ( MSG as Dynamic Chart Solver).

B.1 Proof of Proposition 4.1

First, we give the formal definition of the pullback. Given a smooth map φ : M→N connecting two manifolds M and N , and a real, smooth function f : N → R , the pullback of f by φ is the smooth function φ ∗ f on M defined by ( φ ∗ f )( x ) = f ( φ ( x )) .

Next, we introduce some properties of the pullback in the smooth manifold [59] (i.e., differential 1-form of a smooth function, communication, and pullback of a sum and a product), supporting the derivation of the backward gradient (Theorem 4.1).

Lemma B.1 (Differential 1-form of a smooth function) . For a point z ∈ N related with a coordinate chart ( U, z 1 , ..., z n ) , there is a series of covectors { ( dz 1 ) z , ..., ( dz n ) z } forming a basis of T ∗ z N dual to the basis { ( ∂/∂z 1 ) | z , ..., ( ∂/∂z n ) | z } of tangent space T z N . Then, for any smooth function f on N restrict to U , the differential 1-form of f is

Lemma B.2 (Communication) . Let F : N → M be a smooth map, for any smooth function g on M , we have F ∗ ( dg ) = d ( F ∗ g ) .

Lemma B.3 (Pullback of a sum and a product) . Let F : N → M be a smooth map, g is a smooth scalar function on M , and ω, γ are differential 1-forms on M . Then, we have

Given the properties of the pullback, we derive the closed-form backward gradient of the real function on the manifold, and prove Theorem 4.1.

Theorem 4.1 (Backward Gradient) Let L be the scalar-valued function, and z l is the output of l -th layer with parameter W l , which is delivered by tangent vector v l . Then, the gradient of function L w.r.t. W l is given as follows:

where φ l -1 ( · ) = Exp z l -1 ( · ) , ψ l ( · ) = Exp ( · ) ( v l ) , and [ · ] ∗ means the matrix form of pullback.

Proof. Given z l , z l +1 in M , and F : M → M be the smooth map such that F ( z l ) = z l +1 . Consider scalar loss function L : M→ R , if we relate z l with a chart ( U, x 1 , ..., x m ) and z l +1 with ( V, y 1 , ..., y m ) , the gradients of L at z l +1 and z l are given by Lemma. B.1,

Then, we apply the pullback F ∗ on ∇ z l +1 L that

Then, we can find that the matrix form of the pullback F ∗ can be written as the transpose of the Jacobian matrix of F , denoted as [ ∂F i ∂x j | z l ] ∗ or [ D z l F ] ∗ . The derivation of ∇ W l L is similar, we only need to use an addition process like above on ∇ v l -1 L .

Note that, we give the closed-form expression of exponential map, logarithmic map, and parallel transport for hyperbolic and hyperspherical space, and derive the corresponding Jacobian in Sec. C.

B.2 Proof of Theorem 5.2

Theorem 5.2 ( MSG as Dynamic Chart Solver) If y ( t ) : [ τ, τ + glyph[epsilon1] ] → R n is the solution of

then z ( t ) = Exp z ( y ( t )) is a valid solution to the manifold ODE of Eq. (13) on t ∈ [ τ, τ + glyph[epsilon1] ] , where z = z ( τ ) . If y ( t ) is given by the first-order approximation with the glyph[epsilon1] small enough,

then the update process of Eqs. (4) and (8) in MSG is equivalent to Dynamic Chart Solver in Eq. (15).

Proof. Let t ∈ [ τ, τ + glyph[epsilon1] ] , then we have

Consider two adjacent charts ( U 1 , Log z 1 ) and ( U 2 , Log z 2 ) , such that z 1 ∈ U 1 and z 2 ∈ U 1 ∩ U 2 . Note that, in interval [ τ, τ + glyph[epsilon1] ] , z ( τ ) = z 1 and z ( τ + glyph[epsilon1] ) = z 2 , we have y ( τ ) = Exp z 1 ( z 1 ) = 0 . With the first-order approximation, y ( τ + glyph[epsilon1] ) is thus given by

Also, Eq. (37) can be treated as a step in the Euler solver [60] for a small glyph[epsilon1] . Finally, we have z ( τ + glyph[epsilon1] ) = Exp z 1 ( y ( τ + glyph[epsilon1] )) = z 2 , ending the process of dynamic chart solver. That is, MSG considers the logarithmic map to define the charts, and is equivalent to Dynamic Chart Solver (Definition 5.1), completing the proof.

C Deviation of Jacobian

We instantiate the proposed MSG in the Lorentz model H of hyperbolic space, sphere model S of hyperspherical space, and the products of H 's or/and S 's. Accordingly, we derive the Jacobian in H and S , and introduce the construction in the products in D.3.

C.1 Hyperbolic Space

Lorentz Model The d -dimensional Lorentz model H d is defined on the ( d + 1) -dimensional manifold of { z = [ z 0 , z 1 , · · · , z d ] T ∈ R d +1 |〈 z , z 〉 L = -1 , z 0 > 0 } 5 , equipped with the Minkowski inner product,

The tangent space at point z ∈ H d is T z H d = { v ∈ R d +1 |〈 z , v 〉 L = 0 } , and Proj z ( u ) = u + 〈 z , u 〉 L z is to project a vector u ∈ R d +1 into the tangent space T z H d . The Lorentz norm of tangent vector is defined as ‖ v ‖ L = √ 〈 z , z 〉 L .

Theorem 4.1 requires the Jocabian of φ ( · ) = Exp z ( · ) and ψ ( · ) = Exp ( · ) ( v ) , and Lorentz model has the closed-form exponential map given as follows,

The inverse of exponential map (i.e, the logarithmic map) is

Deviation of Jacobian We first calculate the Jacobian of ψ , and it is given as

Note that, Jacobian of φ needs the Jacobian of ‖ v ‖ L , which is derived as

where

ˆ

v

= [

-

v

0

, v

1

, ..., v

d

]

T

. Then, we compute the derivative of the first term of Eq. 39.

and the derivative of the second term is derived as

Summing up above equations, we finally have

where I is the identity matrix.

C.2 Hyperspherical Space

Sphere Model The sphere model S d is defined on the ( d + 1) -dimensional manifold of { z = [ z 0 , z 1 , · · · , z d ] T ∈ R d +1 |〈 z , z 〉 = 1 , z 0 > 0 } with the standard inner product 〈 x , y 〉 = ∑ d i =0 x i y i and norm ‖ x ‖ = √ ∑ d i =0 x 2 i . The tangent space at point z is T z S d = { v ∈ R d +1 |〈 z , v 〉 = 0 } . Similar to Lorentz model, we have Proj z ( u ) = u -〈 z , u 〉 z projecting a vector u ∈ R d +1 into T z S d . The exponential map in the sphere model is given as

Derivation of Jacobian We first calculate the Jacobian of ψ , and it is given as

Similar to that in Lorentz model, the Jacobian of

φ

needs the Jacobian of

‖

v

‖

,

Then, we compute the derivative of the first term of Eq. 47.

and the derivative of the second term is

Summing up above equations, we finally have

(49)

and the logarithmic map is

D Riemannian Geometry

D.1 Some Notations

Here, we give the formal descriptions of the notions mentioned in the main paper, and please refer to [61] for systematic elaborations.

Geodesically Complete Manifold. A manifold is said to be geodesically complete if the maximal defining interval of any geodesic is R . For any Riemannian manifold ( M , g ) admitting a metric structure given by the length of geodesic

the completeness of d can be described as a metric space is complete if any Cauchy sequence in it converges. For instance, hyperbolic space as well as hyperspherical space is geodesically complete.

Tangent Bundle. Given an n -dimensional smooth manifold M , the tangent bundle T M is the disjoint union of all the tangent spaces of the manifold T M = ⊔ z ∈M T z M , and the tangent bundle with the projection π ( v ) = p for all v ∈ T p M is a vector bundle of rank n .

Chart. Achart of a manifold is a pair ( U, φ ) where U is an open set in the manifold and φ : U → R n is homeomorphism onto it image, giving a local coordinate of the manifold. In other words, it provides a way of identifying the manifold locally with a Euclidean space. Given two charts ( U 1 , φ 1 ) and ( U 2 , φ 2 ) , if the overlap

the two charts are said to be compatible.

Curvature and Sectional Curvature. The curvature is a notion describing the extent of how a manifold derivatives from being 'flat'. In particular, the curvature of a Riemannian manifold M should be viewed as a measure R ( X,Y ) Z of the extent to which the operator ( X,Y ) →∇ X ∇ Y Z is symmetric, where ∇ is a connection on M (where X,Y,Z are vector fields, with Z fixed). Sectional curvature is simpler object of curvature and is defined on two independent vector unit in the tangent space. When ∇ is the Levi-Civita connection induced by a Riemannian metric on M , it turns out that the curvature operator R can be recovered from the sectional curvature.

Constant Curvature Space, Hyperbolic Space, Hyperspherical Space. ARiemannian manifold is said to be a constant curvature space (CCS) if the sectional curvature is constant scalar everywhere on the manifold. When the CCS has a negative constant curvature, it is referred to as hyperbolic space, and the CCS is hyperspherical when its constant curvature is positive.

D.2 κ -stereographic model and Stereographic Projection

κ -stereographic model It gives a unified formalism for both positive and negative constant curvatures. For a positive curvature, it is the hyperspherical model for the hyperspherical space, and for a negative curvature, it switches to the Poincaré ball model.

Specifically, for a curvature κ and a dimension d ≥ 2 , the κ -stereographic model st d κ is defined on the manifold of { x ∈ R d | -κ ‖ x ‖ 2 < 1 } , which is equipped with a Riemannian metric g κ x = 4 (1+ κ ‖ x ‖ 2 ) 2 I

for any constant curvature κ . When κ ≥ 0 , the defining domain is R d in which the stereographic projection of the Sphere model of hyperspherical space is endowed. When κ < 0 , the manifold st d κ is represented in an open ball of radius 1 √ -κ , and is the stereographic projection of the Lorentz model of hyperbolic space.

The κ -stereographical model is a gyrovector space in which a non-associative vector operator system is defined. For x , y ∈ G n , a ∈ R , the κ -addition (a.k.a. Möbius addition) is given as

The distance function given by κ -addition is thus formulated as

The κ -scaling for any real scalar c is defined as

The unit-speed geodesic from x to y is

With the unit-speed geodesic, the exponential map as well as its inverse (i.e., the logarithmic map) has the closed-form expression as follows,

where the curvature-aware trigonometric function is utilized, e.g.,

Stereographic Projection The stereographic projection is a diffeomorphism connecting the different model spaces of Riemannian manifold. In particular, it is defined as a map π : L d κ / S d κ → st d κ taking the form of

where x ′ is a point on the Lorentz model L d κ or Sphere model L d κ , and x , the image of the projection, is the corresponding point in the gyrovector ball of κ -stereographic model. The inverse projection is given as follows,

where λ κ x = 2 1+ κ ‖ x ‖ 2 is known as the conformal factor.

D.3 Cartesian Product and Product Manifold

The concept of product manifolds allows for creating a new manifold from a finite collection of existing ones. Given a set of smooth manifolds M 1 , M 2 , . . . , M k , the product manifold P is given as the Cartesian product of these manifolds:

where ⊗ denotes the Cartesian product. Specifically, with the Cartesian product construction, a point x ∈ P are represented by a concatenation of x = [ x 1 , . . . , x k ] , where x i ∈ M i . A tangent vector v ∈ T x P at a point x can be given as v = [ v 1 , . . . , v k ] , where v i ∈ T x i M i . If each manifold M i is equipped with a metric tensor g i , the product metric g decomposes into the direct sum of the individual metrics g = ⊕ k i =1 g i , which can be expressed as Diag( g 1 , ..., g k ) . For x and y ∈ P , the distance between them is defined as d P ( x , y ) = ∑ k i =1 d M i ( x i , y i ) . Accordingly, the exponential map is given as

E Experimental Setups

E.1 Dataset

We use four common benchmark datasets to evaluate our model and Table 5 show the details of the datasets. There are two co-purchase graphs including Amazon-Photo and Amazon-Computers [53] and two co-author network including CS and Physics [53].

E.2 Baselines

As shown in Table 6, we divide the baselines into three categories: ANN-based Euclidean GNNs, ANN-based Riemannian GNNs and SNN-based Euclidean GNNs. Note that, none of the existing work studies the SNN-based GNN in Riemannian space, to the best of our knowledge.

ANN-based Euclidean GNNs

GCN [18]: It defines graph convolution on the spectral domain.

SAGE [1]: It gives the aggregate-and-combine formulation for the message passing over the graph.

GAT [2]: It introduces the attention mechanism for the learning on graphs.

SGC [19]: It reformulates GCN [18] with feature propagation and linear layer, acting as a low-pass filter.

ANN-based Riemannian GNNs

HGCN [3] : It generalizes GAT [2] in the Lorentz model of hyperbolic space in which the graph convolution is conducted in the tangent space.

κ -GCN [13]: It generalizes GCN [18] to the κ -stereographical model of constant curvature spaces where several gyrovector operators are given in the unified formalism.

HyboNet [54]: It introduces a parameterized Lorentz transformation for hyperbolic graph modeling without the tangent space.

QGCN [4]: It studies the graph convolution network in the Pseudo-Riemannian manifold.

SNN-based Euclidean GNNs.

SpikeGCN [5]: It integrates SNN and graph convolution network in which SNN acts as an activation function.

SpikeGraphormer [55] (termed as SpikeGT for short in our main paper): It generalizes a kind of graph transformer with spiking neurons, accompanied by an ANN for improving the performance.

SpikeGCL [6]: It introduces a method to perform graph contrastive learning with the spiking GNN.

SpikeNet [45]: It is original designed for dynamic graph modeling, and we utilize its version for static graph following [6].

Note that, existing SNN-based GNNs work with Euclidean space, and leverage the BPTT training with surrogate gradient, suffering from high latency.

E.3 Theoretical Energy Consumption

Following the previous works [6; 5; 62], we calculate the theoretical energy consumption for each model, instead of measuring actual electricity usage, for fair comparison.

For the SNN-based models, the energy consumption involves encoding energy E encoding and spiking process energy E spiking. The former is calculated by the number of multiply-andaccumulate (MAC) operations, and the latter is given by the number of SOP operations. The energy consumption is thus defined as follows,

where the scaling constant E MAC and E SOP are set to 4 . 6 pJ and 3 . 7 pJ , respectively. N is the number of nodes in the graph, d is the dimension of node features, L is the number of layers in the neural model. T is the time steps of the spikes, and S l t denotes the output spikes at time step t and layer l .

For the ANN-based models, the energy consumption is given by embedding generation step and aggregation step, and both of them are calculated by MAC operations. In the embedding generation step, the feature transformation with a weight matrix of R d in × d out executes Nd in d out multiplication and Nd in d out addition operations. In the aggregation step, |E| d in is the number of multiplication and |E| d out is the number of addition operations. Supposing |E| d in = |E| d out , the energy consumption is given as follows,

where the scaling constant E MAC is set to 4 . 6 pJ , |E| is the number of edge, d in and d out are the input and output dimensions.

E.4 Implementation Notes

Model Instantiation. The proposed MSG is instantiated in the Lorentz model H of hyperbolic space or Sphere model S of hyperspherical space as well as the products over H and S . Note that, MSG can be equivalently instantiated on the κ -stereographic model (i.e., Poincaré model of hyperbolic space or Hyperspherical model of hyperspherical space), given the closed form Riemannian metric and exponential map. The equivalence can be achieved by scaling the stereographical projection. We leverage the Lorentz model H by default.

Hyperparameters. The dimension of the manifold is set as 32 . When we instantiate MSG on the product manifold, the sum of factor manifold's dimensions is defined as 32 . The manifold spiking neuron is based on the IF model [49] by default, and it is ready to switch to the LIF model [49]. The time latency T for neurons is set to 5 or 15 . The step size glyph[epsilon1] in Eq. 8 is set to 0 . 1 . The hyperparameters are tuned with grid search, in which the learning rate is { 0 . 01 , 0 . 003 } for node classification and { 0 . 003 , 0 . 001 } for link prediction, and the dropout rate is in { 0 . 1 , 0 . 3 , 0 . 5 } .

Hardware. Experiments are conducted on the NVIDIA GeForce RTX 4090 GPU 24GB memory, and AMD EPYC 9654 CPU 315 with 96-Core Processor.

F Additional Results

In this section, we show the additional results on backward gradient, comparison between IF and LIF model, link prediction and visualization.

0

25

50

75

100

125

150

175

200

Iterations

5

10

15

20

25

30

Norm * 1e-5

gradient norm of z in layer 0

0

25

50

75

100

125

150

175

200

Iterations

5

10

15

20

25

30

Norm * 1e-5

gradient norm of z in layer 1

0

25

50

75

100

125

150

175

200

Iterations

5

10

15

20

25

30

Norm * 1e-5

gradient norm of z in layer 2

(a) The norm of backward gradient of L with respect to z in each spiking layers.

0

25

50

75

100

125

150

175

200

Iterations

5

10

15

20

25

30

Norm * 1e-5

gradient norm of v in layer 0

0

25

50

75

100

125

150

175

200

Iterations

5

10

15

20

25

30

Norm * 1e-5

gradient norm of v in layer 1

0

25

50

75

100

125

150

175

200

Iterations

5

10

15

20

25

30

Norm * 1e-5

gradient norm of v in layer 2

0

25

50

75

100

125

150

175

200

Iterations

0.5

1.0

1.5

2.0

Loss

Training loss

(c) The loss in model training.

Backward Gradient. Previous studies compute backward gradients though the Differentiation via Spikes ( DvS ). Distinguishing from the previous studies, we compute backward gradients though the Differentiation via Manifold ( DvM ). In order to examine the backward gradients, we visualize the training process for node classification on Computer dataset. Concretely, we plot the norm of backward gradients in each iteration in Figs. 6 (a) and (b) together with the value of loss function in Figs. 6 (c). As shown in Fig. 6, the proposed algorithm with DvM converges well, and the backward gradients do not suffer from gradient vanishing or gradient explosion.

Comparison between IF and LIF model. In the main paper, the proposed MSG is built with the IF model, and it is applicable to LIF model as well. We compare the performance between IF and LIF model in different algorithms ( DvM and DvS ) and in different manifolds (hyperbolic H , hyperspherical S , Euclidean E and the product spaces among H and S ) for a comprehensive evaluation.

The results of node classification on Computer, Photo, CS and Physics datasets are summarized in Table 7 and Table 8. Note that, IF model and LIF model achieves competitive performance in every case. We opt for IF model in the model instantiation for simplicity.

Link Prediction. The performance of link prediction in terms of AUC is provided in the main paper. We show the results on Computer, Photo, CS and Physics datasets in terms of AP (%) in Table 9. In particular, we feed the spiking representation of SpikingGCN and SpikeGCL into the Fermi-Dirac decoder same as the proposed MSG , while SpikeNet and SpikeGT are designed for node classification specially. As shown in Table 9, the proposed spiking MSG consistently achieves the best results among the spiking GNNs on all the four datasets, and even achieves the competitive performances with the strong Riemannian baselines.

Visualization. Here, we visualize the forward pass of the proposed MSG and empirically demonstrate the connection between MSG and manifold ordinary differential equation (ODE).

We choose a toy example of KarateClub dataset. The proposed MSG are instantiated on the 2D manifold for the ease of visualization. Specifically, we plot the node representation of each spiking

vector

point

(a)

1

-th nodes.

vector

point

(b)

17

-th nodes.

layer in Fig. 7(a) and Fig. 7(b) in which the curve connecting the outputs of successive layer is marked in red, and blue arrow is the direction of the geodesic. It is shown that a spiking layer forwards the node along the geodesic on the manifold. In other words, each layer, constructing a chart given by the exponential map, is a solver of the ODE describing the geodesic.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We propose a novel Manifold-valued Spiking GNN ( MSG ), and design a new training algorithm with Differentiation via Manifold with theoretical gaurantee. Extensive experiments show the effectiveness of the proposed approach.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations in Sec. 8. The proposed model is applicable to any geodesically complete manifold (e.g., hyperbolic space, hyperspherical space and their products), and its generalization to more generic manifold leaves as the future work.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide the complete proof in Appendix B.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Experimental details are given in Sec. 6.1, and further introduced in Appendix E. Also, we give pseudocodes in Algorithm 1.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The datasets are publicly available. We properly cite and introduce the datasets in Sec. 6.1 and Appendix E.1. Our source code is at the anonymous link https: //anonymous.4open.science/r/MSG-16E9 .

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We give experimental settings in Sec. 6.1, which is further detailed in Appendix E. Implementation details can be found in the anonymous link at https://anonymous. 4open.science/r/MSG-16E9 .

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: In the experiment, we perform 10 independent runs for each case, and report the mean with standard derivations in Sec. 6.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: In Sec. 6.1, we provide the information on the computer resources: NVIDIA GeForce RTX 4090 GPU 24GB memory, and AMD EPYC 9654 CPU with 96-Core Processor.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conforms with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss both potential positive societal impacts and negative societal impacts of the work performed in Sec. 8.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We properly cite the public datasets and open source Python library (e.g., Geoopt and SpikingJelly ) in Sec. 6.1 and Appendix E.1.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We give the source code of the proposed model with documentation at https: //anonymous.4open.science/r/MSG-16E9 .

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors generalize spiking GNN to Riemannian manifold, and design a new architecture of parallel forwarding so as to boost model training. Then, the proposed MSG is evaluated with 12 baselines on real graphs.\n\nSTRENGTHS:\nS1. A technical strong paper, and No error is detected. It presents a Riemannian optimization to boost the SNN training and connects the proposed model to Riemannian ODE. \n\nS2. The studied problem of Spiking GNN training is interesting, since it is time consuming to conduct BPTT training of SGNN especially when the spiking train is long.\n\nS3. The authors give a new architecture of parallel forwarding so that graph is modeled in Riemannian manifold, different from the previous SGNN in Euclidean space.\n\nS4. The experiment and visualization are convincing, and the results show that Riemannian modeling of SNN generally achieves superior performance.\n\nWEAKNESSES:\nW1. The proposed model works with the backbone of GCN, and the performance of other backbones, GAT for instance, is not examined.\n\nW2. Some experimental details are not specified, e.g., the link prediction setting with SNN baselines, which is not frequently reported in graph SNN studies.\n\nQUESTIONS:\nQ1. The authors add a parallel Riemannian forwarding to accelerate the SNN training, leveraging DvM. However, it needs to specify the additional parameter and will it damage the energy efficiency of SNN?\n\nQ2. In the ablation study of Section 6.2, the training time of DvM is much less than that of BPTT, and the question is that, is DvM’s backward time of independent from the steps of spikes?\n\nQ3. Please specify the manifold of Fig. 5 in the Experiment, i.e., the construction of the manifold or its equation.\n\nQ4. What is the different between this paper and the recent paper, Continuous spiking graph neural networks, regarding ODE?\n\nLIMITATIONS:\nThe authors specify the limitations and potential negative social impact in Sec. 8.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-01"}

comments: SUMMARY:
This work proposes a spiking graph neural network on Riemannian manifolds, named Manifold-valued Spiking GNN (MSG). This work also develops a new training algorithm of differentiation via manifold (DvM) that avoids the high training overhead of BPTT methods and proves that the MSG is a neural ODE solver. Experimental results show that MSG outperforms existing spiking GNNs with low energy consumption.

STRENGTHS:
1. It is a novel idea to explore spiking GNN on Riemannian manifolds.
2. This work is technical solid. It proves in detail the theory that the MSG is a neural ODE solver.
3. The proposed MSG outperforms existing spiking GNNs in both node classification and link prediction tasks with low energy consumption.

WEAKNESSES:
1. Figure 4(a) is confusing. In line 356-357, the author state that backward time of DvM is significantly less than that of BPTT algorithm. But in figure 4(a), the bar of DvM is higher. Is this a mistake? If not, please explain. In addition, it would be helpful to list the complete training times.
2. Figure 2 shows that the manifold representations will feed forward with the spike trains. This is acceptable during training, but during inference, since $f(\cdot)$ is an exponential function, the computation of the manifold representations involves a lot of float-point operations, which is incompatible with the nature of spiking neural networks. Please describe in detail the computation of the manifold representations during inference, especially the float-point operations involved.

QUESTIONS:
Please refer to the weaknesses

LIMITATIONS:
The authors have stated the limitations and broader impact.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-09

comments: SUMMARY:
In order to improve energy efficiency and performance in graph learning, the research presents a unique Manifold-valued Spiking Graph Neural Network (MSG), which combines spiking neurons with Riemannian manifolds. Differentiation via Manifold (DvM), a unique training approach, is proposed by the authors to solve the high latency issue with spiking GNNs.

STRENGTHS:
Combining spiking neurons with Riemannian manifolds is a novel idea that addresses both energy efficiency and performance

Numerous experiments show that MSG is more effective and energy-efficient than both traditional and spiking GNNs currently in use.

WEAKNESSES:
It is not the first time the Riemann manifold has been introduced into spiking neural networks. The authors lack proper discussion of these works. 

The energy consumption is unknown for the proposed method due to the complexity of integrating spiking neurons with Riemannian manifolds.


While the theoretical contributions are significant, the paper lacks discussion on practical applications and real-world scenarios.

QUESTIONS:
Can the authors provide more details on the potential neuromorphic implementation? 

Are there any specific real-world applications where MSG could be particularly beneficial?

LIMITATIONS:
See weakness and questions
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper first analyzes the limitations of spiking GNN, representation space and model training, and then present a Riemannian model called MSG, which is connected to manifold ODE. Finally, the authors conduct experiment to show the effectiveness and energy efficiency.\n\nSTRENGTHS:\nOn the significance,\n1.This paper relates two previously disparate research areas, SNN and Riemannian geometry.\n2.Energy efficiency is of importance to the wide use of Riemannian models, given that they are typically computationally expensive.\n\nOn the originality,\n1.Unlike existing Riemannian GNNs, the authors design a Riemannian GNN with the spiking neuron.\n2.It is a novel idea to parallel Riemannian and spiking domains, providing the opportunity for a faster training algorithm, DvM.\n\nOn the quality, this is a Solid paper with few technical,\n1.The theoretical results (connection to ODE) are proved, and the closed form gradient is well elaborated.\n2.The empirical results are adequate, and Codes are provided for reproducibility.\n\nOn the clarity, this paper is well-organized and easy to follow in general.\n\nWEAKNESSES:\n1.In the experiment, in Table 1, the authors do not provide some of link perdition results of SNNs. Thus, it needs to specify on those SNN baselines, e.g., is SpikeNet [33] able to conduct link prediction? Why not?\n2.This paper requires systematic understanding of differential geometry, and thus is not friendly to the reader not familiar to this area.\n\nQUESTIONS:\nSee W1, and other questions are listed as follows:\n1.I have double checked Proofs 4.1 and 5.1, and the proofs are correct. However, I would like to clarify: whether or not there is a typo in Line 275 of Thm 5.2, that z and y are swapped.\n2.I notice that, the authors mention another optimization method, sub-gradient [15], also different from the BPTT scheme. Thus, specify the sub-gradient, and the different between sub-gradient and the proposed DvM.\n3.Given that gradient exploding and vanishing are common problems in training SNN, will MSG suffer from these issues? Also, is there any treatment for the numerical stability of Riemannian operators, e.g, sinh and cosh in the Appendix?\n\nLIMITATIONS:\nNo limitation (negative societal impact) needs to be specifically addressed.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-12"}

comments: SUMMARY:
This paper focuses on the theoretical aspects of spiking neural networks, which are a variant of neural networks closer to the human brain, particularly incorporating a time component. One of the hopes for this type of networks is that they are much more energy efficient. In this paper, the authors extend the classical spiking graph neural networks setting, which only handle graphs in Euclidean space, to the setting of Riemannian manifolds. This is highly non-trivial, requiring a specific of each spiking neuron on manifolds. Intriguingly, they then relate this new model to manifold ODEs, and also show even theoretically a significant improvement of energy efficiency as opposed to classical GNNs.

STRENGTHS:
* SNNs are very timely due to the severe energy problems of classical NNs.
* Goes significantly beyond the state of the art.
* Very well-organized paper, intriguing to read.
* From deep math to numerical experiments, this paper covers the full range
* Comprehensive analysis of the newly introduced type of SNNs, and also showing their superiority

WEAKNESSES:
* Argument, why Riemannian manifolds are required, could be strengthened, and also what about other types of manifolds?

QUESTIONS:
see weaknesses

LIMITATIONS:
---
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-16