Spiking Graph Neural Network on Riemannian Manifolds

Authors: li  sun
zhenhao  huang
qiqi  wan
hao  peng
philip s yu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 13:20:45

Abstract: Graph neural networks (GNNs) have become the dominant solution for learning on graphs, the typical non-Euclidean structures. Conventional GNNs, constructed with the Artificial Neuron Network (ANN), have achieved impressive performance at the cost of high computation and energy consumption. In parallel, spiking GNNs with brain-like spiking neurons are drawing increasing research attention owing to the energy efficiency. So far, existing spiking GNNs consider graphs in Euclidean space, ignoring the structural geometry, and suffer from the high latency issue due to Back-Propagation-Through-Time (BPTT) with the surrogate gradient. In light of the aforementioned issues, we are devoted to exploring spiking GNN on Riemannian manifolds, and present a Manifold-valued Spiking GNN (MSG). In particular, we design a new spiking neuron on geodesically complete manifolds with the diffeomorphism, so that BPTT regarding the spikes is replaced by the proposed differentiation via manifold. Theoretically, we show that MSG  approximates a solver of the manifold ordinary differential equation. Extensive experiments on common graphs show the proposed MSG achieves superior performance to previous spiking GNNs and energy efficiency to conventional GNNs.

Main paper: 
 introduction: 
 Graphs are the ubiquitous, non-Euclidean structures that describe the relationship among objects. Graph neural networks (GNNs), constructed with the floating-point Artificial Neuron Network (ANN), have achieved state-of-the-art accuracy for learning on graphs [1; 2; 3; 4] . However, they raise the concerns about computation and energy consumption, particularly when dealing with real-world graphs of considerable scale [5; 6] . In contrast, Spiking Neuron Networks (SNNs), inspired by the biological mechanism of brains, utilize neurons that communicate using sparse and discrete spikes, showcasing their superiority in energy efficiency [7; 8] . Attempting to bring the best of both worlds, spiking GNNs are drawing increasing research attention.In the literature of spiking GNNs, recent efforts have been made to design different architectures with spiking neurons, e.g., graph convolution [5] , attention mechanism [9] , variational autoencoder [10] and continuous GNN [11] . While achieving encouraging results, existing spiking GNNs still face several fundamental issues: (1) Representation Space. Spiking GNNs consider the graph in Euclidean space, ignoring the inherent geometry of graph structures. Unlike the Euclidean structures (e.g., pixel matrix and grid structures), graphs cannot be embedded in Euclidean space with bounded distortion [12] . Instead, Riemannian manifolds have been shown as the promising spaces to model graphs in recent years [3; 13; 4] (e.g., hyperbolic space, a type of Riemannian manifolds, is well aligned with the graphs dominated by hierarchical structures). However, none of the existing works study SNN on Riemannian manifolds, to the best of our knowledge. It is thus an interesting and urgent problem to consider how to endow the spiking GNN with a Riemannian manifold. (2) Training Algorithm. Training spiking GNN is challenging, since the spikes are non-differentiable. Existing studies consider the spiking GNN as a recurrent neural network and apply Backward-Passing-Through-Time (BPTT) with the surrogate gradient [5; 9; 10; 11] . They recurrently compute the backward gradient at each time step, and thus suffer from the high latency issue [14; 15; 16; 6] especially when the spike trains are long. 1 . It is not realistic to place spike trains in a manifold such as hyperbolic or hyperspherical space, given the fact that spike trains cannot align with the defining domain. Instead, we design a Manifold Spiking Layer that conducts parallel forwarding of spike trains and manifold representations. Specifically, we first incorporate the structural information into spike trains by graph convolution. Then, a new manifold spiking neuron is proposed to emit spike trains and relate them to manifold representations with diffeomorphism, where the spike train generates a momentum that forwards manifold representation along the geodesic. Instead of applying BPTT in spike domain, the proposed neuron provides us with an alternative of Differentiation via Manifold (DvM). (The red dashed line in Fig. 1 .) Yet, differentiation in Riemannian manifold is nontrivial. We leverage the properties of pullback and derive the closed-form backward gradient (Theorem 4.1). DvM enables the recurrence-free gradient backpropagation, which no longer needs to perform recurrent computation of time steps as in BPTT. Theoretically, MSG is essentially related to manifold Ordinary Differential Equation (ODE). Each layer creates a chart of the manifold, and MSG approximates the dynamic chart solver [17] of manifold ODE (Theorem 5.2).Contributions. Overall, the key contributions are summarized as follows: (1) To the best of our knowledge, we propose the first spiking neural network on Riemannian manifolds (MSG) 2 , and show its connection to manifold ODE theoretically. (2) We design a new training algorithm of differentiation via manifold, which avoids the high latency of BPTT methods. (3) Extensive experiments show the superior effectiveness and energy efficiency of the proposed MSG. 
related work: 
 We briefly overview the ANN-based GNNs (i.e., the conventional, floating-point GNNs living in either Euclidean space or Riemannian manifolds) and SNN-based GNNs (i.e., spiking GNNs).). The majority of GNNs are built with floatingpoint ANN, conducting message passing on the graphs [18; 2; 19] . The Euclidean space has been the workhorse for graph representation learning for decades, and the popular GCN [18] , GAT [2] and SGC [19] are also designed in the Euclidean space. In recent years, Riemannian manifolds have emerged as an exciting alternative considering the geometry of graph structures [20; 21] . Among Riemannian manifolds, hyperbolic space is recognized for its alignment with the graphs of hierarchical structures, and a series of hyperbolic GNNs (e.g., HGNN [22] , HGCN [3] ) show superior performance to their Euclidean versions. Beyond hyperbolic space, hyperspherical space is well suited for cyclical structures [23] , and recent studies further investigate the constant curvature spaces [13] , product spaces [24; 25; 26; 27] , quotient spaces [28] , SPD manifolds [29; 30] , etc. Riemannian manifolds achieve remarkable success in graph clustering [31; 32] , structural learning [33] , graph dynamics [34; 35; 36; 37] , information diffusion [38] and graph generation [39; 40] , but have rarely been touched yet in the SNN counterpart.Spiking Neural Networks (SNNs) & Spiking GNNs. Mimicking the biological neural networks, SNNs [7; 8] utilize the spiking neuron to process spike trains, and offer the advantage of energy efficiency. Despite the wide application of SNN in computer vision [41; 42] , SNNs are still at an early stage in the graph domain. The basic idea of spiking GNNs is adapting ANN-based GNNs to the SNN framework by substituting the activation functions with spiking neurons. Pioneering works study the graph convolution [43; 5] , and efforts have also been made to the graph attention [9] , variational graph autoencoder [10] , graph differential equations [44] , etc. SpikeGCL [6] is a recent endeavor to conduct graph contrastive learning with SNN. In parallel, spiking GNNs are extended to model the dynamic graphs [45; 46; 47] . We focus on the static graph in this work. In both dynamic and static cases, previous spiking GNNs are trained with the surrogate gradient, leading to high latency, and consider the graphs in the Euclidean space.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We discuss the limitations in Sec. 8. The proposed model is applicable to any geodesically complete manifold (e.g., hyperbolic space, hyperspherical space and their products), and its generalization to more generic manifold leaves as the future work.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
preliminaries: 
 Different from aforementioned spiking GNNs, we study the spiking GNN on Riemannian manifolds. Thus, we formally introduce the basic concepts of Riemannian geometry and SNN. Throughout this paper, the lowercase boldfaced x and uppercase X denote vector and matrix, respectively. Important notations are summarized in Appendix A.Riemannian Geometry & Riemannian Manifold. Riemannian geometry provides elegant framework to study structures and manifolds. A Riemannian manifold is described as a smooth and real manifold M endowed with a Riemannian metric. Each point x in the manifold is associated with the tangent space T x M that "looks Euclidean", and the Riemannian metric is given by the inner product in the tangent space, so that geometric properties (e.g, angle, length) can be defined. A geodesic between two points on the manifold is the smooth path connecting them with the minimal length. There exist three types of isotropic manifold, namely, the Constant Curvature Space (CCS): hyperbolic space H, hyperspherical space S and the special case of Euclidean space with "flat" geometry E .Graph & Riemannian Graph Representation Learning. A graph G = (V, E, F, A) is defined on the node set V and edge set E ⊂ V × V, and A ∈ R |V|×|V| is the adjacency matrix describing the structure information. Each node v i is associated with a feature f i , and node features are summarized in F ∈ R |V|×d . In this paper, we resolve the problem of Riemannian Graph Representation Learning with SNN. Specifically, we seek a graph encoder F θ : v → z where z ∈ M is a point on the manifold, instead of Euclidean space, and F θ is defined with an energy-efficient SNN.Spiking Neural Network. SNNs are constructed by spiking neurons that communicate with each other by spike trains. Concretely, a spiking neuron is conceptualized as "a capacitor of the membrane potential", and processes the spike trains by the following 3 phases [48] . First, the incoming current I[t] is accumulated in the capacitor, leading to the potential buildup (integrate). When the membrane potential V [t] reaches or exceeds a specific threshold V th , the neuron emits a spike (fire). After that, the membrane potential is reset to the resting potential V reset (reset). There are two popular spiking neurons: IF model and LIF model [49] . In particular, the three phases of IF model are formalized asIntegrate : V [t] = g(V [t -1], I[t]) = V [t -1] + I[t](1) Fire :EQUATIONReset :V [t] = (1 -S[t])V [t] + S[t]V rest , Fixed-reset, V [t] -V th S[t],Subtraction-reset.(EQUATIONwhere the incoming current I[t] is related to the input spike train, and V reset is lower than V th . t denotes the time index of the spike. The Heaviside function H(•) is non-differentiable, H(x) = 1 if x ≥ 0, and 0 otherwise. There are two options for reset, and fixed-reset is adopted in this paper.Overall, an IF model is given as S[t] = IFModel(I[t]), and the only difference between IF model and LIF model lies in the definition of g(•) in Eq. (1) . In this paper, we are interested in designing a new spiking neuron on Riemannian manifold.4 Methodology: Manifold-valued Spiking GNNIn this section, we present a simple yet effective Manifold-valued Spiking GNN (MSG), which can be applied to any geodesically complete manifolds, e.g., the Constant Curvature Space (CCS), including hyperbolic space and hyperspherical space, or the product of CCS. In particular, we design a spiking neuron on Riemannian manifolds (named as Manifold Spiking Neuron) that allows for the differentiation via manifold. It provides a new perspective of training spiking GNN, so that we avoid the high latency of typical backward-passing-through-time (BPTT) training. ∂W l , D v l-1 φ l-1 and ∇ z l L will be introduced in Sec. 4.2.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes]• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
manifold spiking layer: 
 GCN 𝑠 𝑙-1 IFModel Pooling 𝑓( • ) 𝑠 𝑙 Forward Backward 𝑣 𝑙-1 𝑧 𝑙 𝑧 𝑙-1 𝛻 𝑧 𝑙 (ℒ) 𝛻 𝑧 𝑙-1 (ℒ) 𝜕𝑣 𝑙-1 𝜕𝑊 𝑙 𝐷 𝑣 𝑙-1 (𝜙 𝑙-1 ) 𝑧 𝑙-1 𝑥 𝑙-1We elaborate on the sole building block of the proposed model -Manifold Spiking Layer. Note that, the spike train or spiking representation in existing spiking GNNs [43; 5; 9; 10; 6; 45; 46; 47] cannot align with the defining domain of Riemannian manifolds (e.g., hyperbolic space and hyperspherical space), thus posing a fundamental challenge. Our solution is to generate node representation on the manifold (referred to as manifold representation) in parallel, and leverage the notion of Diffeomorphism to create the alignment between the two domains. We formulate the parallel forwarding of spike trains and manifold representations as follows.Unified Formulation. The forward pass of the spiking layer consists of a graph convolution and one proposed manifold spiking neuron. Without loss of generality, for each node v i ∈ G, the l-th spiking layer is formulated as follows,EQUATIONEQUATION)where x is the incoming current to generate spike trains. s and z denote the spike trains and manifold representation, respectively. GCN(•) is a GNN in the Euclidean space, and W l is the learnable parameter in the layer. Different from the neuron of previous spiking GNNs, we design a novel manifold spiking neuron (MSNeuron) as shown in Fig. 2 . It emits spike trains and relates them to manifold representations simultaneously, which is formulated as follows,EQUATIONEQUATIONEQUATION)where t is the time step of spike trains. The IF model can be replaced by LIF model, and we utilize IF model by default for simplicity. Pooling is defined as the average pooling of the current x over t, and v is given as the Euclidean vector. f denotes the diffeomorphism to the Riemannian manifold in which is the step size.Incorporating structural information. We inject the structural information when the received spikes transform into the incoming current of the neuron. A GNN is leveraged to define the current, conducting message-passing over the graph. Each node's representation is derived recursively by neighborhood aggregation [18; 2; 1]. Accordingly, GCN in the proposed neuron is given as follows,EQUATIONwhere the neighborhood Ω i is the set of immediate neighbors centering at node v i . The aggregate function aggregates the messages from neighborhood Ω i , where we create the message of a node by W l s j [t]. combine(•) denotes the combination of the center node's message and aggregated message. We utilize GCN [18] as the backbone to define aggregate and combine.Diffeomorphism between manifolds. In the proposed neuron, we bridge the spikes and manifold representation with the notion of diffeomorphism in differential geometry. A diffeomorphism connects two smooth manifolds, saying M and N . Formally, a map f : M → N is a diffeomorphism between M and N if the smooth f is bijective and its inverse f -1 is also smooth.Recall that the tangent space is locally Euclidean. We propose to place the Euclidean v, a representation of the spikes, in the tangent space T z M of the point z. In MSG, we choose the exponential map to act as the diffeomorphism between the tangent space and manifold. With a step size , we haveEQUATIONConcretely, given z ∈ M and v ∈ T z M, the exponential map3 of v at point z, Exp z (v) : T z M → M, maps tangent vector v onto the manifold M. The map pushes z along the geodesic γ z,v (t) : [0, 1] → M starting at γ z,v (0) = z and ending at y = γ z,v (1). γz,v (t) denotes the velocity of γ z,v (t), and the direction of geodesic at the beginning is given as γz,v (0) = v. That is, the tangent vector v, derived from the spikes, pushes the manifold representation along the geodesic via the exponential map. The advantage of our choice is that we are able to define the diffeomorphism in arbitrary geodesically complete manifold (detailed in Appendix D).Note that, our idea is inherently different from the exponential/logarithmic based Riemannian GNNs [3; 4; 22] , which leverage the tangent space of the origin for neighborhood aggregation. In contrast, we consider the successive process over the tangent spaces of manifold representations, which will be further studied in Sec. 5.In MSG, we need to simultaneously initialize the spiking input S 0 and manifold representation Z 0 , which is a collection of points on the given manifold. Given a graph G(V, E, F, A), the node features are first encoded by one graph convolution layer H = GCN(A, F; W 0 ), and we generate T copies of the node encodings H, where T is the number of time steps in spike trains. Then, we complete model initialization with the proposed manifold neuron [S 0 , Z 0 ] = MSNeuron(H, O), where the encoding H is regarded as the incoming current that charges the neuron in each time step. O consists of the original points of the manifold, e.g., in the sphere model of hyperspherical space, the original point is given as the south pole o = [-1, 0, ..., 0] and O = [o , ..., o ] . Note that, the exponential map in the proposed neuron guarantees that Z 0 lives in the manifold.Optimizing SNNs is challenging, as the Heaviside step function is non-differentiable. In the literature, existing spiking GNNs typically regard SNN as the recurrent neural network, and leverage backwardpassing-through-time (BPTT) to train the model [50; 51; 52] . Concretely, given a real loss function L, the gradient backpropagation conducts Differentiation via Spikes (DvS) s as follows,EQUATION)where W is the parameter, and t denotes the time step. The surrogate gradient [51] is required forD W l s l [t],where Heaviside step function is replaced by a differentiable surrogate, e.g., sigmoid function. The differentiation via spikes presents high latency in the backward pass [14; 15; 16] , as it needs to recur all the time steps in BPTT. We notice that, in the computer vision domain, the sub-gradient method [15] is proposed to address such issues in Euclidean space. However, it cannot be generalized to the Riemannian manifold since the linearity does not hold in Riemannian geometry. forward pass3:Input current X 0 = GCN(A, F; W 0 );4:Initialize [S 0 , Z 0 ] = MSNeuron(X 0 , O);5:for each spiking layer l = 1 to L do 6:X (l-1) = GCN(A, S (l-1) ; W l );7:[S l , Z l ] = MSNeuron(X (l-1) , Z (l-1) ); Compute ∇ z L L from L(Z L ).11:for layer l = L -1 to 1 do12: Compute D z l ψ l , D v l-1 φ l-1 , ∂v l-1 ∂W l . 13:Compute ∇ z l L, ∇ W l L as Eq. 12.14:Update W l .15:end for 16: end whileIn MSG, we decouple the forward pass and backward pass, and propose Differentiation via Manifold (DvM) to avoid the high latency in differentiation via spikes. The overall procedure of training MSG by the proposed learning approach is summarized in Algorithm 1. Thanks to the parallel forwarding of spikes and manifold representation, the proposed neuron provides us with an alternative of studying ∇ W l L through the forwarding pass on the manifold (i.e., differentiation via manifold). Nevertheless, it is nontrivial and it requires to derive the pullback between different dual spaces.Pushforward, Pullback and Dual Space. We first introduce the differentiation in Riemannian geometry which is essentially different from that in Euclidean space. In Riemannian geometry, a pushforward refers to a derivative of a map connecting two manifolds M and N . Concretely, given f : M → N and a point z ∈ M, the pushforwardD z f maps a tangent vector v ∈ T z M to the tan- gent vector D z f (v) ∈ T f (z) N . On the nota- tion, for a manifold-valued function f (z) = p ∈ N , ∂p/∂z is equivalent to D z f . For a scalar function f , D z f is interchangeable with ∇ z f .In the proposed MSG, we consider a scalar loss function on the manifold L : M → R. The pushforward D z L at point z ∈ M maps tangent vector v ∈ T z M to a scalar value and, correspondingly, D z L belongs to the dual space of the tangent space T * z M, which is a vector space consisting all linear functional F : T z M → R. As the tangent spaces at different points of the manifold are different, it requires a pullback that maps the dual space T * z l M to the dual space T * z (l+1) M. We derive the backward gradient with properties of differential 1-form (Lemma B.1), communication (Lemma B.2), and pullback of a sum and a product (Lemma B.3) detailed in Appendix B.1. Theorem 4.1 (Backward Gradient). Let L be the scalar-valued function, and z l is the output of l-th layer with parameter W l , which is delivered by tangent vector v l . Then, the gradient of function L w.r.t W l is given as follows:EQUATIONwhere  and [ •] * means the matrix form of pullback.φ l-1 (•) = Exp z l-1 (•), ψ l (•) = Exp (•) (v l ),The detailed proof is given in Appendix B.1, and we derive the two Jacobian matrices D v l-1 φ l-1 and D z l ψ l in Appendix C. There are three key advantages of the proposed DvM. First, every term in Equation ( 12) is differentiable, and thereby the surrogate gradient is no longer needed. Second, DvM enables the recurrence-free backward pass alleviating the high latency training. We specify that both DvM and the previous DvS recurrently compute every time step in the forward pass, and the difference lies in the backward pass. In particular, we only conduct recurrence-free gradient backpropagation layer by layer, while the previous DvS recurs every time step of each layer in BPTT.In addition to the differentiable and recurrence-free properties, DvM does not suffer from gradient vanishing/explosion, and the empirical evidence is provided in Appendix F. We leverage the notion of chart to study the relationship between MSG and manifold ODE. A manifold ODE defined asEQUATIONdescribes a vector field u that maps a smooth path z(t) : [0, 1] → M to the tangent bundle T M4 . In other words, the vector field u assigns each point z(t) ∈ M to a tangent vector u(z(t), t) ∈ T z(t) M. A chart at z, denoted as (U z , φ z ), is a smooth bijection φ z between z's neighborhood U z ⊂ M and a subspace of Euclidean space. Thus, the chart relates Eq. ( 13) to ODE in Euclidean space [17] . Specifically, ifEQUATIONthen y(t) = φ i (z(t)) is a valid solution of Eq. ( 13) on t ∈ [τ, τ + ]. Definition 5.1 (Dynamic Chart Solver [17] ). The manifold ODE in Eq. ( 13) with initial condition z(0) = z can be solved with a finite collection of successive charts {(U i , φ i )} i=1,...,L . If ode i is the numerical solver to Euclidean ODE corresponding to the i-th chart,y(t) = ode i (t) on [τ i , τ i + i ],then z(t) in Eq. ( 13) is given asEQUATIONThat is, a manifold ODE can be solved in Euclidean subspaces given by a series of successive charts.In MSG, we consider the charts given by the logarithmic map as illustrated in 3, and we prove that MSG approximates a dynamic chart solver of manifold ODE (Theorem 5.2). Theorem 5.2 (MSG as Dynamic Chart Solver). If y(t)EQUATIONthen z(t) = Exp z (y(t)) is a valid solution to the manifold ODE of Eq. ( 13) on t ∈ [τ, τ + ], where z = z(τ ). If y(t) is given by the first-order approximation with the small enough,EQUATIONthen the update process of Eqs. ( 4) and ( 8) in MSG is equivalent to Dynamic Chart Solver in Eq. (15) .Proof. The proof utilizes some facts of Riemannian manifolds and is detailed in Appendix B.2.In other words, in MSG, the transformation of manifold input and output is described as some manifold ODE, whose vector field is governed by a spiking-related neural network in the tangent bundle. To solve the manifold ODE, MSG leverages the Dynamic Chart Solver (Definition 5.1). Specifically, each manifold spiking layer corresponds to a chart, and thus the number of spiking layers equals to the number of charts. Each layer solves the ODE of a smooth path y(t) : [τ, τ + ] → R n in the tangent space that centered at the manifold layer input. With the first-order approximation in Theorem 5.2, given a step size , the endpoint y(τ + ) of the path is parameterized by a GNN related to the spikes. Layer-by-layer forwarding solves the manifold ODE from the current chart to the successive chart. Consequently, the manifold output of MSG approximates the solution to the manifold ODE.We notice that a recent work [11] connects spiking GNN to an ODE in Euclidean space. In contrast, the proposed MSG is essentially related to the manifold ODE.The Appendix contains the proofs, the derivation of Jacobian, necessary facts on Riemannian geometry (i.e., Lorentz/Sphere model, stereographic projection and κ-stereographic model, and Cartesian product and product space), empirical details and additional results. Baselines. We compare the proposed MSG with 12 strong baselines of three categories: (1) ANNbased Euclidean GNNs: the popular GCN [18] , GAT [2] , GraphSAGE [1] and SGC [19] , (2) ANN-based Riemannian GNNs: HGCN [3] and HyboNet [54] of hyperbolic spaces, κ-GCN [13] of the constant curvature space, and the recent Q-GCN [4] of the quotient space, (3) The previous Euclidean Spiking GNNs: SpikeNet [45] , SpikeGCN [5] , SipkeGraphormer [55] (termed as SpikeGT for short) and the recent SpikeGCL [6] . Note that, we focus on the graph representation learning on static graphs, and thereby graph models for the dynamic ones are out of the scope of this paper. SpikeNet [45] was originally designed for dynamic graphs, and we utilize its variant for static graphs according to [6] . So far, spiking GNN has not yet been connected to Riemannian manifolds, and we are devoted to bridging this gap.Evaluation Protocol. All models are evaluated by node classification and link prediction tasks. The evaluation metrics of node classification is classification accuracy; we employ the popular Area Under Curve (AUC) for link prediction. The hyperparameter setting is the same as the original papers. We perform 10 independent runs for each case, and report the mean with standard derivations. Experiments are conducted on the hardware of NVIDIA GeForce RTX 4090 GPU 24GB memory, and AMD EPYC 9654 CPU with 96-Core Processor. Our model is built upon GeoOpt [56] , SpikingJelly [56] and PyTorch [57] .Model Instantiation & Configuration. Note that, the proposed MSG applies to any Constant Curvature Space (CCS) or the product of CCS. We instantiate MSG in the Lorentz model of hyperbolic space by default (whose Riemannian metric, exponential map, and the derived Jacobian is given in Appendix C), and study the impact of representation space in the Ablation Study. The dimension of the representation space is set as 32. The manifold spiking neuron is based on the IF model [49] by default, and it is ready to switch to the LIF model [49] whose results are given in Appendix F. The time steps T for neurons is set to 5 or 15. The step size in Eq. 8 is set to 0.1. The hyperparameters are tuned with grid search, in which the learning rate is {0.01, 0.003} for node classification and {0.003, 0.001} for link prediction, and the dropout rate is in {0.1, 0.3, 0.5}. We provide the source code of MSG at the anonymous link https://github.com/ZhenhHuang/MSG. 
results & discussion: 
 Effectiveness. We evaluate the effectiveness of MSG in both node classification and link prediction tasks. Specifically, for node classification, we cannot directly feed the manifold representations of Riemannian baselines to a softmax layer with Euclidean measure. We bridge the manifold representation and Euclidean softmax with the logarithmic map of respective manifold. For link prediction, we utilize the generalized sigmoid for all the baselines, i.e., the Fermi-Dirac decoder [3] in which the distance function is defined under the respective geometry. In the proposed MSG, the model inference does not need the expensive successive exponential maps, and only limited float-point operations (i.e., addition) are involved. Accordingly, we leverage the tangent vectors for the downstream tasks. Energy Cost. We investigate the energy cost of the graph models in terms of theoretical energy consumption (mJ) [5; 6] , whose formula is specified in Appendix E. We summarize the results for node classification in Table 3 in which the number of parameters at the running time is listed as a reference. It shows that SNN-based models generally enjoy less energy cost than ANN-based ones. Visualization & Discussion. We empirically study the connection between the proposed MSG and manifold ODE.In particular, we visualize a toy example of Zachary Karate Club dataset [58] on a S 1 × S 1 in Fig. 5 , where we plot each layer output on the manifold. The red curve is the path connecting the layer input and layer output, and the blue one is the direction of the geodesic. As shown in Fig. 5 , the red and blue curves are coincided, that is, each layer solves an ODE describing the geodesic on the manifold. 
conclusion: 
 In this paper, we study spiking GNN from a fundamentally different perspective of Riemannian geometry, and present a simple yet effective Manifold-valued Spiking GNN (MSG).Concretely, we design a manifold spiking neuron which leverages the diffeomorphism to bridge spiking representations and manifold representations. With the proposed neuron, we propose a new training algorithm with Differentiation via Manifold, which no longer needs to recur the backward gradient and thus alleviates the high latency of previous methods. An interesting theoretical result is that, MSG is essentially related to manifold ODE. Extensive empirical results on benchmark datasets demonstrate the superior effectiveness and energy efficiency of the proposed MSG. 
broader impact and limitations: 
 Our work brings together two previously separate domains: spiking neural network and Riemannian geometry, and presents a novel Manifold-valued Spiking GNN for energy-efficiency graph learning, especially for the large graphs. Our work is mainly a theoretical exploration, and not tied to particular applications. A positive societal impact is the possibility of decreasing carbon emissions in training large models. None of negative societal impacts we feel must be specifically highlighted here.Limitation. Our work as well as the previous spiking GNNs considers the undirected, homophilous graphs, while the spiking GNN on directed or heterophilous graphs still remains open. Also, readers may find it challenging to implement the proposed method. However, we provide downloadable code and will offer an easy-to-use interface.The appendix is organized as follows:1  
 
 Appendix: We summarize the important notations of our paper in Table 4 . The original point on manifold T z MThe tangent space at point z T MThe tangent bundle of the manifold M uThe vector field over the manifold, described by an ODET * z N The dual space of T z N D z f The differential of f at point z (df ) z The differential 1-form of f at point z {(∂/∂z 1 )| z , ..., (∂/∂z n )| z } A basis of the tangent space T z N {(dz 1 ) z , ..., (dz n ) z } A basis of T * z N ∇LThe gradient of a smooth scalar function L Exp z (•)The exponential map at point z Log z (•)The Threshold membrane potential of a spiking neuron lThe index of spiking layer in neural networkThe proposed manifold spiking neuronIn this section, we demonstrate the proofs of Theorem 4.1 (Backward Gradient) and Theorem 5.2 (MSG as Dynamic Chart Solver).First, we give the formal definition of the pullback. Given a smooth map φ : M → N connecting two manifolds M and N , and a real, smooth function f : N → R, the pullback of f by φ is the smooth function φ * f on M defined by(φ * f )(x) = f (φ(x)).Next, we introduce some properties of the pullback in the smooth manifold [59] (i.e., differential 1-form of a smooth function, communication, and pullback of a sum and a product), supporting the derivation of the backward gradient (Theorem 4.1). Lemma B.1 (Differential 1-form of a smooth function). For a point z ∈ N related with a coordinate chart (U, z 1 , ..., z n ), there is a series of covectors {(dz 1 ) z , ..., (dz n ) z } forming a basis of T * z N dual to the basis {(∂/∂z 1 )| z , ..., (∂/∂z n )| z } of tangent space T z N . Then, for any smooth function f on N restrict to U , the differential 1-form of f isEQUATION)Lemma B.2 (Communication). Let F : N → M be a smooth map, for any smooth function g on M, we have F * (dg) = d(F * g). Lemma B.3 (Pullback of a sum and a product). Let F : N → M be a smooth map, g is a smooth scalar function on M, and ω, γ are differential 1-forms on M. Then, we haveEQUATION)Given the properties of the pullback, we derive the closed-form backward gradient of the real function on the manifold, and prove Theorem 4.1.Theorem 4.1 (Backward Gradient) Let L be the scalar-valued function, and z l is the output of l-th layer with parameter W l , which is delivered by tangent vector v l . Then, the gradient of function L w.r.t. W l is given as follows:EQUATIONwhereφ l-1 (•) = Exp z l-1 (•), ψ l (•) = Exp (•) (v l ),* means the matrix form of pullback.Proof. Given z l , z l+1 in M, and F : M → M be the smooth map such that F (z l ) = z l+1 . Consider scalar loss function L : M → R, if we relate z l with a chart (U, x 1 , ..., x m ) and z l+1 with (V, y 1 , ..., y m ), the gradients of L at z l+1 and z l are given by Lemma. B.1,EQUATIONEQUATIONThen, we apply the pullback F * on ∇ z l+1 L thatEQUATIONEQUATIONEQUATIONEQUATIONThen, we can find that the matrix form of the pullback F * can be written as the transpose of the Jacobian matrix of F , denoted as[ ∂F i ∂x j | z l ] * or [D z l F ] * .The derivation of ∇ W l L is similar, we only need to use an addition process like above on ∇ v l-1 L.Note that, we give the closed-form expression of exponential map, logarithmic map, and parallel transport for hyperbolic and hyperspherical space, and derive the corresponding Jacobian in Sec. C.EQUATIONthen z(t) = Exp z (y(t)) is a valid solution to the manifold ODE of Eq. ( 13) on t ∈ [τ, τ + ], where z = z(τ ). If y(t) is given by the first-order approximation with the small enough,EQUATIONthen the update process of Eqs. ( 4) and ( 8) in MSG is equivalent to Dynamic Chart Solver in Eq. ( 15).Proof. Let t ∈ [τ, τ + ], then we haveEQUATIONEQUATION)Consider two adjacent charts (U 1 , Log z1 ) and (U 2 , Log z2 ), such thatz 1 ∈ U 1 and z 2 ∈ U 1 ∩ U 2 . Note that, in interval [τ, τ + ], z(τ ) = z 1 and z(τ + ) = z 2 , we have y(τ ) = Exp z1 (z 1 ) = 0.With the first-order approximation, y(τ + ) is thus given byEQUATIONEQUATIONAlso, Eq. ( 37) can be treated as a step in the Euler solver [60] for a small . Finally, we have z(τ + ) = Exp z1 (y(τ + )) = z 2 , ending the process of dynamic chart solver. That is, MSG considers the logarithmic map to define the charts, and is equivalent to Dynamic Chart Solver (Definition 5.1), completing the proof.We instantiate the proposed MSG in the Lorentz model H of hyperbolic space, sphere model S of hyperspherical space, and the products of H's or/and S's. Accordingly, we derive the Jacobian in H and S, and introduce the construction in the products in D.3.Lorentz Model The d-dimensional Lorentz model H d is defined on theEQUATION)The tangent space at point z ∈ H d is T z H d = {v ∈ R d+1 | z, v L = 0},EQUATION)The inverse of exponential map (i.e, the logarithmic map) isEQUATIONDeviation of Jacobian We first calculate the Jacobian of ψ, and it is given asD z ψ = cosh( v L )I.(41) Note that, Jacobian of φ needs the Jacobian of v L , which is derived asEQUATIONwhere v = [-v 0 , v 1 , ..., v d ] T . Then, we compute the derivative of the first term of Eq. 39.EQUATIONand the derivative of the second term is derived asEQUATIONSumming up above equations, we finally haveEQUATION)where I is the identity matrix.Sphere Model The sphere model S d is defined on the The exponential map in the sphere model is given as(d + 1)-dimensional manifold of {z = [z 0 , z 1 , • • • , z d ] T ∈ R d+1 | z, z = 1, z 0 > 0}EQUATIONand the logarithmic map isEQUATIONDerivation of Jacobian We first calculate the Jacobian of ψ, and it is given asEQUATION) Similar to that in Lorentz model, the Jacobian of φ needs the Jacobian of v ,EQUATION)Then, we compute the derivative of the first term of Eq. 47.EQUATIONand the derivative of the second term isEQUATIONSumming up above equations, we finally haveEQUATIONHere, we give the formal descriptions of the notions mentioned in the main paper, and please refer to [61] for systematic elaborations.Geodesically Complete Manifold. A manifold is said to be geodesically complete if the maximal defining interval of any geodesic is R. For any Riemannian manifold (M, g) admitting a metric structure given by the length of geodesic d(p, q) = inf{L(γ)|γ is a piecewise smooth curve connecting p to q},the completeness of d can be described as a metric space is complete if any Cauchy sequence in it converges. For instance, hyperbolic space as well as hyperspherical space is geodesically complete.Tangent Bundle. Given an n-dimensional smooth manifold M, the tangent bundle T M is the disjoint union of all the tangent spaces of the manifold T M = z∈M T z M, and the tangent bundle with the projection π(v) = p for all v ∈ T p M is a vector bundle of rank n.Chart. A chart of a manifold is a pair (U, φ) where U is an open set in the manifold and φ : U → R n is homeomorphism onto it image, giving a local coordinate of the manifold. In other words, it provides a way of identifying the manifold locally with a Euclidean space. Given two charts (U 1 , φ 1 ) and (U 2 , φ 2 ), if the overlapEQUATIONthe two charts are said to be compatible.Curvature and Sectional Curvature. The curvature is a notion describing the extent of how a manifold derivatives from being "flat". In particular, the curvature of a Riemannian manifold M should be viewed as a measure R(X, Y )Z of the extent to which the operator (X, Y ) → ∇ X ∇ Y Z is symmetric, where ∇ is a connection on M (where X, Y, Z are vector fields, with Z fixed). Sectional curvature is simpler object of curvature and is defined on two independent vector unit in the tangent space. When ∇ is the Levi-Civita connection induced by a Riemannian metric on M, it turns out that the curvature operator R can be recovered from the sectional curvature.Constant Curvature Space, Hyperbolic Space, Hyperspherical Space. A Riemannian manifold is said to be a constant curvature space (CCS) if the sectional curvature is constant scalar everywhere on the manifold. When the CCS has a negative constant curvature, it is referred to as hyperbolic space, and the CCS is hyperspherical when its constant curvature is positive.κ-stereographic model It gives a unified formalism for both positive and negative constant curvatures. For a positive curvature, it is the hyperspherical model for the hyperspherical space, and for a negative curvature, it switches to the Poincaré ball model. Specifically, for a curvature κ and a dimension d ≥ 2, the κ-stereographic model st d κ is defined on the manifold of {x ∈ R d | -κ x 2 < 1}, which is equipped with a Riemannian metric g κ x = 4 (1+κ x 2 ) 2 I for any constant curvature κ. When κ ≥ 0, the defining domain is R d in which the stereographic projection of the Sphere model of hyperspherical space is endowed. When κ < 0, the manifold st d κ is represented in an open ball of radius 1 √ -κ , and is the stereographic projection of the Lorentz model of hyperbolic space.The κ-stereographical model is a gyrovector space in which a non-associative vector operator system is defined. For x, y ∈ G n , a ∈ R, the κ-addition (a.k.a. Möbius addition) is given asEQUATIONThe distance function given by κ-addition is thus formulated asEQUATIONThe κ-scaling for any real scalar c is defined asEQUATIONThe unit-speed geodesic from x to y isEQUATIONWith the unit-speed geodesic, the exponential map as well as its inverse (i.e., the logarithmic map) has the closed-form expression as follows,EQUATIONEQUATIONwhere the curvature-aware trigonometric function is utilized, e.g.,EQUATION)Stereographic Projection The stereographic projection is a diffeomorphism connecting the different model spaces of Riemannian manifold. In particular, it is defined as a map π :EQUATIONwhere x is a point on the Lorentz model L d κ or Sphere model L d κ , and x, the image of the projection, is the corresponding point in the gyrovector ball of κ-stereographic model. The inverse projection is given as follows,EQUATIONwhereλ κ x = 2 1+κx 2 is known as the conformal factor.The concept of product manifolds allows for creating a new manifold from a finite collection of existing ones. Given a set of smooth manifolds M 1 , M 2 , . . . , M k , the product manifold P is given as the Cartesian product of these manifolds:EQUATIONwhere ⊗ denotes the Cartesian product. Specifically, with the Cartesian product construction, a point x ∈ P are represented by a concatenation of x = [x 1 , . . . , x k ], where x i ∈ M i . A tangent vector v ∈ T x P at a point x can be given asv = [v 1 , . . . , v k ], where v i ∈ T xi M i . If each manifold M iis equipped with a metric tensor g i , the product metric g decomposes into the direct sum of the individual metrics g = ⊕ k i=1 g i , which can be expressed as Diag(g 1 , ..., g k ). For x and y ∈ P, the distance between them is defined as d P (x, y) = k i=1 d Mi (x i , y i ). Accordingly, the exponential map is given as We use four common benchmark datasets to evaluate our model and Table 5 show the details of the datasets. There are two co-purchase graphs including Amazon-Photo and Amazon-Computers [53] and two co-author network including CS and Physics [53] .EQUATIONTable 6 : The categories of the baselinesEuclidean Space GCN [18] , SAGE [1] , GAT [2] , SGC [19] SpikeNet [45] , SpikeGraphormer [55] , SpikeGCN [5] , SpikeGCL [6] Riemannian Space Q-GCN [4] , κ-GCN [13] , HyboNet [54] , HGCN [3] As shown in Table 6 , we divide the baselines into three categories: ANN-based Euclidean GNNs, ANN-based Riemannian GNNs and SNN-based Euclidean GNNs. Note that, none of the existing work studies the SNN-based GNN in Riemannian space, to the best of our knowledge.• GCN [18] : It defines graph convolution on the spectral domain.• SAGE [1] : It gives the aggregate-and-combine formulation for the message passing over the graph. • GAT [2] : It introduces the attention mechanism for the learning on graphs.• SGC [19] : It reformulates GCN [18] with feature propagation and linear layer, acting as a low-pass filter.• HGCN [3] : It generalizes GAT [2] in the Lorentz model of hyperbolic space in which the graph convolution is conducted in the tangent space. • κ-GCN [13] : It generalizes GCN [18] to the κ-stereographical model of constant curvature spaces where several gyrovector operators are given in the unified formalism. • HyboNet [54] : It introduces a parameterized Lorentz transformation for hyperbolic graph modeling without the tangent space. • Q-GCN [4] : It studies the graph convolution network in the Pseudo-Riemannian manifold.• SpikeGCN [5] : It integrates SNN and graph convolution network in which SNN acts as an activation function.• SpikeGraphormer [55] (termed as SpikeGT for short in our main paper): It generalizes a kind of graph transformer with spiking neurons, accompanied by an ANN for improving the performance.• SpikeGCL [6] : It introduces a method to perform graph contrastive learning with the spiking GNN. • SpikeNet [45] : It is original designed for dynamic graph modeling, and we utilize its version for static graph following [6] .Note that, existing SNN-based GNNs work with Euclidean space, and leverage the BPTT training with surrogate gradient, suffering from high latency.Following the previous works [6; 5; 62] , we calculate the theoretical energy consumption for each model, instead of measuring actual electricity usage, for fair comparison.• For the SNN-based models, the energy consumption involves encoding energy E encoding and spiking process energy E spiking . The former is calculated by the number of multiply-andaccumulate (MAC) operations, and the latter is given by the number of SOP operations. The energy consumption is thus defined as follows,EQUATIONwhere the scaling constant E MAC and E SOP are set to 4.6pJ and 3.7pJ, respectively. N is the number of nodes in the graph, d is the dimension of node features, L is the number of layers in the neural model. T is the time steps of the spikes, and S l t denotes the output spikes at time step t and layer l.• For the ANN-based models, the energy consumption is given by embedding generation step and aggregation step, and both of them are calculated by MAC operations. In the embedding generation step, the feature transformation with a weight matrix of R din×dout executes N d in d out multiplication and N d in d out addition operations. In the aggregation step, |E|d in is the number of multiplication and |E|d out is the number of addition operations. Supposing |E|d in = |E|d out , the energy consumption is given as follows,EQUATION)where the scaling constant E MAC is set to 4.6pJ, |E| is the number of edge, d in and d out are the input and output dimensions.Model Instantiation. The proposed MSG is instantiated in the Lorentz model H of hyperbolic space or Sphere model S of hyperspherical space as well as the products over H and S. Note that, MSG can be equivalently instantiated on the κ-stereographic model (i.e., Poincaré model of hyperbolic space or Hyperspherical model of hyperspherical space), given the closed form Riemannian metric and exponential map. The equivalence can be achieved by scaling the stereographical projection. We leverage the Lorentz model H by default.Hyperparameters. The dimension of the manifold is set as 32. When we instantiate MSG on the product manifold, the sum of factor manifold's dimensions is defined as 32. The manifold spiking neuron is based on the IF model [49] by default, and it is ready to switch to the LIF model [49] . The time latency T for neurons is set to 5 or 15. The step size in Eq. 8 is set to 0.1. The hyperparameters are tuned with grid search, in which the learning rate is {0.01, 0.003} for node classification and {0.003, 0.001} for link prediction, and the dropout rate is in {0.1, 0.3, 0.5}.Hardware. Experiments are conducted on the NVIDIA GeForce RTX 4090 GPU 24GB memory, and AMD EPYC 9654 CPU 315 with 96-Core Processor.In this section, we show the additional results on backward gradient, comparison between IF and LIF model, link prediction and visualization. Link Prediction. The performance of link prediction in terms of AUC is provided in the main paper. We show the results on Computer, Photo, CS and Physics datasets in terms of AP (%) in Table 9 . In particular, we feed the spiking representation of SpikingGCN and SpikeGCL into the Fermi-Dirac decoder same as the proposed MSG, while SpikeNet and SpikeGT are designed for node classification specially. As shown in Table 9 , the proposed spiking MSG consistently achieves the best results among the spiking GNNs on all the four datasets, and even achieves the competitive performances with the strong Riemannian baselines. Visualization. Here, we visualize the forward pass of the proposed MSG and empirically demonstrate the connection between MSG and manifold ordinary differential equation (ODE). We choose a toy example of KarateClub dataset. The proposed MSG are instantiated on the 2D manifold for the ease of visualization. Specifically, we plot the node representation of each spiking vector point Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Codes are available at https://github.com/ZhenhHuang/MSGThe inverse map from M to TzM is the logarithmic map.Theory: MSG as Neural ODE SolverNext, we demonstrate the theoretical aspects of our model that MSG approximates a solver of manifold Ordinary Differential Equations (ODEs).The tangent bundle T M is the disjoint union of all the tangent spaces of the manifold T M = z∈M TzM.We utilize the manifold of standard curvature for model instantiation, i.e., constant curvature of -1 for hyperbolic space, and 1 for hyperspherical space. Note that, hyperbolic/hyperspherical spaces of different constant curvatures are mathematically equivalent in essence. 


Decision: I have read all comments and responses. Reviews appear to be consentaneous, with five scores of 8, 8, 8, 6, and 6. All concerns of reviewers have been fixed. It is recommended to accept this manuscript.

Reviews: review: paper_summary: paper_summary: This paper focuses on the theoretical aspects of spiking neural networks, which are a variant of neural networks closer to the human brain, particularly incorporating a time component. One of the hopes for this type of networks is that they are much more energy efficient. In this paper, the authors extend the classical spiking graph neural networks setting, which only handle graphs in Euclidean space, to the setting of Riemannian manifolds. This is highly non-trivial, requiring a specific of each spiking neuron on manifolds. Intriguingly, they then relate this new model to manifold ODEs, and also show even theoretically a significant improvement of energy efficiency as opposed to classical GNNs.
strength_weakness: strength_weakness: * Argument, why Riemannian manifolds are required, could be strengthened, and also what about other types of manifolds?
questions: questions: see weaknesses
limitations: limitations: ---
paper_summary: paper_summary: This paper first analyzes the limitations of spiking GNN, representation space and model training, and then present a Riemannian model called MSG, which is connected to manifold ODE. Finally, the authors conduct experiment to show the effectiveness and energy efficiency.
strength_weakness: strength_weakness: 1.In the experiment, in Table 1, the authors do not provide some of link perdition results of SNNs. Thus, it needs to specify on those SNN baselines, e.g., is SpikeNet [33] able to conduct link prediction? Why not?
2.This paper requires systematic understanding of differential geometry, and thus is not friendly to the reader not familiar to this area.
questions: questions: See W1, and other questions are listed as follows:
1.I have double checked Proofs 4.1 and 5.1, and the proofs are correct. However, I would like to clarify: whether or not there is a typo in Line 275 of Thm 5.2, that z and y are swapped.
2.I notice that, the authors mention another optimization method, sub-gradient [15], also different from the BPTT scheme. Thus, specify the sub-gradient, and the different between sub-gradient and the proposed DvM.
3.Given that gradient exploding and vanishing are common problems in training SNN, will MSG suffer from these issues? Also, is there any treatment for the numerical stability of Riemannian operators, e.g, sinh and cosh in the Appendix?
limitations: limitations: No limitation (negative societal impact) needs to be specifically addressed.
paper_summary: paper_summary: In order to improve energy efficiency and performance in graph learning, the research presents a unique Manifold-valued Spiking Graph Neural Network (MSG), which combines spiking neurons with Riemannian manifolds. Differentiation via Manifold (DvM), a unique training approach, is proposed by the authors to solve the high latency issue with spiking GNNs.
strength_weakness: strength_weakness: It is not the first time the Riemann manifold has been introduced into spiking neural networks. The authors lack proper discussion of these works. 

The energy consumption is unknown for the proposed method due to the complexity of integrating spiking neurons with Riemannian manifolds.


While the theoretical contributions are significant, the paper lacks discussion on practical applications and real-world scenarios.
questions: questions: Can the authors provide more details on the potential neuromorphic implementation? 

Are there any specific real-world applications where MSG could be particularly beneficial?
limitations: limitations: See weakness and questions
paper_summary: paper_summary: This work proposes a spiking graph neural network on Riemannian manifolds, named Manifold-valued Spiking GNN (MSG). This work also develops a new training algorithm of differentiation via manifold (DvM) that avoids the high training overhead of BPTT methods and proves that the MSG is a neural ODE solver. Experimental results show that MSG outperforms existing spiking GNNs with low energy consumption.
strength_weakness: strength_weakness: 1. Figure 4(a) is confusing. In line 356-357, the author state that backward time of DvM is significantly less than that of BPTT algorithm. But in figure 4(a), the bar of DvM is higher. Is this a mistake? If not, please explain. In addition, it would be helpful to list the complete training times.
2. Figure 2 shows that the manifold representations will feed forward with the spike trains. This is acceptable during training, but during inference, since $f(\cdot)$ is an exponential function, the computation of the manifold representations involves a lot of float-point operations, which is incompatible with the nature of spiking neural networks. Please describe in detail the computation of the manifold representations during inference, especially the float-point operations involved.
questions: questions: Please refer to the weaknesses
limitations: limitations: The authors have stated the limitations and broader impact.
paper_summary: paper_summary: The authors generalize spiking GNN to Riemannian manifold, and design a new architecture of parallel forwarding so as to boost model training. Then, the proposed MSG is evaluated with 12 baselines on real graphs.
strength_weakness: strength_weakness: W1. The proposed model works with the backbone of GCN, and the performance of other backbones, GAT for instance, is not examined.

W2. Some experimental details are not specified, e.g., the link prediction setting with SNN baselines, which is not frequently reported in graph SNN studies.
questions: questions: Q1. The authors add a parallel Riemannian forwarding to accelerate the SNN training, leveraging DvM. However, it needs to specify the additional parameter and will it damage the energy efficiency of SNN?

Q2. In the ablation study of Section 6.2, the training time of DvM is much less than that of BPTT, and the question is that, is DvM’s backward time of independent from the steps of spikes?

Q3. Please specify the manifold of Fig. 5 in the Experiment, i.e., the construction of the manifold or its equation.

Q4. What is the different between this paper and the recent paper, Continuous spiking graph neural networks, regarding ODE?
limitations: limitations: The authors specify the limitations and potential negative social impact in Sec. 8.

Comments: comment: Thanks for your response. My concerns have been addressed.
Thanks for the authors' response, and the responses have addressed my concerns. Having reviewed the comments from other reviewers, I have decided to endorse this paper, and raise my score to Strong Accept.