Original: Theoretically Provable Spiking Neural Networks
Revised: Theoretically Provable Spiking Neural Networks

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Spiking neural networks have attracted increasing attention in recent years due totheir potential of handling time-dependent data.
Revised: Spiking neural networks have attracted increasing attention in recent years due to their potential of handling time-dependent data.

Original: Many algorithms and techniqueshave been developed; however, theoretical understandings of many aspects ofspiking neural networks are far from clear.
Revised: Many algorithms and techniques have been developed; however, theoretical understandings of many aspects of spiking neural networks are far from clear.

Original: A recent work [43] disclosed thattypical spiking neural networks could hardly work on spatio-temporal data due totheir bifurcation dynamics and suggested that  self-connection  has to be added.
Revised: A recent work [44] disclosed that typical spiking neural networks could hardly work on spatio-temporal data due to their bifurcation dynamics and suggested that the self-connection structure has to be added.

Original: Inthis paper, we theoretically investigate the approximation powers and computationalefﬁciency of spiking neural networks with self connections, and show that the selfconnection structure enables spiking neural networks to approximate continuousdynamical systems within polynomial  parameters and time complexities.
Revised: In this paper, we theoretically investigate the approximation ability and computational efficiency of spiking neural networks with self connections, and show that the self-connection structure enables spiking neural networks to approximate continuous dynamical systems using polynomial number of parameters within polynomial time complexities.

Original: Ourtheoretical results may shed some insights on developing provable and soundspiking neural networks.
Revised: Our theoretical results may shed some insight for the future studies of spiking neural networks.

Original:  Introduction
Revised:  Introduction

Original: The past decades have witnessed an increasing interest in spiking neural networks (SNNs) due totheir great potential of modeling time-dependent data [28, 32, 37].
Revised: The past decades have witnessed an increasing interest in spiking neural networks (SNNs) due to their great potential of modeling time-dependent data [28, 32, 37].

Original: The fundamental units of SNNsare usually formulated as the combination of an integrated process (e.g., some ﬁrst-order parabolicequations) and a step ﬁring function.
Revised: The fundamental units of SNNs are usually formulated as the combination of an integrated process (e g., some first-order parabolic equations) and a step firing function.

Original: There has been signiﬁcant progress on computational andimplementation techniques for SNNs in  vision [13, 33], speech recognition [30, 39], reinforcementlearning [11, 38], few-short learning [16, 22, 25], etc.
Revised: There has been significant progress on computational and implementation techniques for SNNs in computer vision [13, 33], speech recognition [30, 39], reinforcement learning [11, 38], few-short learning [16, 22, 25], etc.

Original: However, theoretical understandings of manyaspects, such as the approximation powers and computational efﬁciency on spatio-temporal systems,of SNNs are still cloudy.
Revised: However, theoretical understandings of many aspects of SNNs, such as the approximation ability and computational efficiency on spatio-temporal systems, are far from clear.

Original: Some researchers [19, 20, 21, 31] focused on the approximation universality of SNNs, in which sometypical SNNs can simulate the standard computational models such as Turing machines, randomaccess machines, threshold circuits, sequence-to-sequence mapping, etc.
Revised: Some researchers [19, 20, 21, 31] focused on the approximation universality of SNNs, in which some typical SNNs can simulate the standard computational models such as Turing machines, random access machines, threshold circuits, sequence-to-sequence mapping, etc.

Original: There are also some effortson the computational efﬁciency of SNNs forsolving some speciﬁc issues, such as the convergence inthe limit results and computational complexity of SNNs forsolving the sparse coding problem [34, 35]and temporal quadratic programming [5, 8], respectively.
Revised: There are also efforts on the computational efficiency of SNNs for some specific issues such as the convergence in the limit results and computational complexity of SNNs for the sparse coding problem [34, 35] and temporal quadratic programming [5, 8], respectively.

Original: Amazingly, a recent study [43] theoretically proved that, contrary to previous beliefs, typical SNNscan hardly work  on spatio-temporal data, because they  are bifurcation dynamical systems with ﬁxedeigenvalues in which many patterns inherently cannot be learned.
Revised: Amazingly, a recent study [44] theoretically proved that, contrary to previous beliefs, typical SNNs can hardly work well on spatio-temporal data, because they in nature are bifurcation dynamical systems with fixed eigenvalues in which many patterns inherently cannot be learned.

Original: They also suggested that addingself-connection can enhance the representation ability of SNNs on spatio-temporal systems, whichfully connects the spiking neurons in the same layer and solves adaptive eigenvalues of continuousdynamical systems.
Revised: They also suggested that adding self-connection structure can enhance the representation ability of SNNs on spatio-temporal systems that fully connect the spiking neurons in the same layer and solves adaptive eigenvalues of continuous dynamical systems.

Original: Thus, it is intriguing to develop an in-depth analysis of the effects led byself-connection.

Original: In this paper, we theoretically investigate the approximation power and computational efﬁciencyof the self-connection spike neural networks (scSNNs).
Revised: In this paper, we theoretically investigate the approximation ability and computational efficiency of the self-connection spike neural networks (scSNNs).

Original: Our theoretical results show that equippedwith self connections, scSNNs can approximate continuous dynamical systems  within polynomialparameters and time complexities.
Revised: Our theoretical results show that equipped with self connections, scSNNs can approximate continuous dynamical systems using polynomial number of parameters within polynomial time complexities.

Original: Our main contributions are summarized as follows:
Revised: Our main contributions are summarized as follows:

Original: • We prove that the proposed scSNNs are universal approximators in Theorem 1.•
Revised: • We prove that the proposed scSNNs are universal approximators in Theorem 1.

Original:  As for spatial approximation, we prove that a broad scope of radial functions can be wellapproximated by scSNNs within polynomial spiking neurons in Theorem 2.•
Revised: • As for spatial approximation, we prove that a broad range of radial functions can be well approximated by scSNNs with polynomial spiking neurons in Theorem 2.

Original:  As for temporal approximation, we prove thatthe multivariate spike ﬂows can be approximated by scSNNs within polynomial time in Theorem 3 and verify this conclusion insimulation experiments.
Revised: • As for temporal approximation, we prove that multivariate spike flows can be approximated by scSNNs within polynomial time in Theorem 3 and verify this conclusion in simulation experiments.

Original: The rest of this paper is organized as follows.
Revised: The rest of this paper is organized as follows.

Original: Section 2 introducessome useful notations.
Revised: Section 2 introduces notations.

Original: Section 3presents the scSNNs and some related concepts.
Revised: Section 3 presents the scSNNs and some related concepts.

Original: Section 4 provides three key theorems to show theuniversal power and (parameters and time) complexity of scSNNs for approximating the continuousdynamical systems.
Revised: Section 4 provides three key theorems to show the universal approximation ability and (parameters and time) complexity of scSNNs for approximating the continuous dynamical systems.

Original: Section 5  concludes this work with discussions and prospects.
Revised: Section 5 develops in-depth discussions of the effects led by selfconnection. Section 6 concludes this work.

Original:  Notations
Revised:  Notations

Original: Here, we provide some useful notations, which are detailed in Appendix.
Revised: Here, we provide some useful notations, which are detailed in Appendix.

Original: Let [ N ] = { 1 , 2 , . . 
Revised: Let [ N ] = { 1 , 2 , . . .

Original: , N } bean integer set for N ∈ N + , and |·| # denotes the number of elements in a collection, e.g., | [ N ] | # = N .
Revised: , N } be an integer set for N ∈ N + , and |·| # denotes the number of elements in a collection, e.g., | [ N ] | # = N .

Original: Let i =
Revised: Let i =

Original: √− 1 be the imaginary unit, and x (cid:52) 0 means that every element x i ≤ 0 for any i .
Revised: √− 1 be the imaginary unit, and x ≼ 0 means that every element x i ≤ 0 for any i .

Original: Let thesphere S ( r ) and globe B ( r ) be S ( r )
Revised: Let the sphere S ( r ) and globe B ( r ) be S ( r )

Original: = { x | (cid:107) x (cid:107) = r } and B ( r ) = { x | (cid:107) x (cid:107) ≤ r } for any r ∈ R ,respectively.
Revised: = { x | ∥ x ∥ = r } and B ( r ) = { x | ∥ x ∥ ≤ r } for any r ∈ R , respectively.

Original: Given a function g ( n ) , we denote by h 1 ( n ) =
Revised: Given a function g ( n ) , we denote by h 1 ( n ) =

Original: Θ( g ( n )) if there exist positive constantsc 1 , c 2 and n 0 such that c 1 g ( n ) ≤ h 1 ( n ) ≤ c 2 g ( n ) for every n ≥ n 0 ; h 2 ( n ) =
Revised: Θ( g ( n )) if there exist positive constants c 1 , c 2 and n 0 such that c 1 g ( n ) ≤ h 1 ( n ) ≤ c 2 g ( n ) for every n ≥ n 0 ; h 2 ( n ) =

Original: O ( g ( n )) if there existpositive constants c and n 0 such that h 2 ( n ) ≤ cg ( n ) for every n ≥ n 0 ; h 3 ( n ) =
Revised: O ( g ( n )) if there exist positive constants c and n 0 such that h 2 ( n ) ≤ cg ( n ) for every n ≥ n 0 ; h 3 ( n ) =

Original: Ω( g ( n )) if thereexist positive constants c and n 0 such that h 3 ( n ) ≥ cg ( n ) for every n ≥ n 0 ; h 4 ( n ) =
Revised: Ω( g ( n )) if there exist positive constants c and n 0 such that h 3 ( n ) ≥ cg ( n ) for every n ≥ n 0 ; h 4 ( n ) =

Original: o ( g ( n )) ifthere exist positive constants c and n 0 such 
Revised: o ( g ( n )) if there exist positive constants c and n 0 such that

Original: that h 4 ( n ) < cg ( n ) for every n ≥ n 0 .
Revised: h 4 ( n ) < cg ( n ) for every n ≥ n 0 .

Original: Let C ( K, R ) be the set of all scalar functions f : K
Revised: Let C ( K, R ) be the set of all scalar functions f : K

Original: → R continuous on K ⊂ R m .
Revised: → R continuous on K ⊂ R n .

Original: Givenα = ( α 1 , α 2 , . . .
Revised: Given α = ( α 1 , α 2 , . . .

Original: , α m ) (cid:62) ,  α i ≥ 0 for i ∈ [ m ] , | α | = (cid:80) i ∈ [ m ] α i, and x = ( x 1 , x 2 , . . . , x m ) ∈ K , wedeﬁne an n -tuple partial derivative
Revised: , α m ) ⊤ , satisfying α i ≥ 0 for i ∈ [ m ] and | α | ≜ (cid:80) i ∈ [ m ] α i ∈ N + , we define a | α | -tuple partial derivative

Original: [Equation]
Revised: [Equation]

Revised: where x = ( x 1 , x 2 , . . .

Revised: , x n ) ∈ K .

Original: Further, we deﬁne
Revised: Further, we define

Original: [Equation] R 
Revised: C l ( K, R ) =

Original: [Equation] and D r f ∈ C ( K, R ) , for r ∈
Revised: { f | f ∈ C ( K, R ) , and D r f ∈ C ( K, R ) , for r ∈

Original: [ l ] } .
Revised: [ l ] } .

Original: For 1 ≤ p < ∞ , we deﬁne
Revised: For 1 ≤ p < ∞ , we define

Original: L

Original: [Equation]

Original: (cid:12) f

Original: [Equation]

Original: [Equation]

Original: , R

Original: [Equation]
Revised: [Equation]

Original: and (cid:107) f (cid:107) p,K (cid:44) (cid:18)(cid:90)

Original: K

Original: [Equation]

Original: f ( x [Equation] | p d x (cid:19) 1 /p < ∞ (cid:41) . This work considers the Sobolev space W l,pµ ( K, R ) , deﬁned as the collection of all functions f ∈
Revised: This work considers the Sobolev space W l,pµ ( K, R ) , defined as the collection of all functions f ∈

Original: C l ( K, R ) and D r f ∈
Revised: C l ( K, R ) and D r f ∈

Original: L p ( K, R ) for all | r | ∈ [ l ] , that is,(cid:107) D [Equation] f (cid:107) p,K = (cid:18)(cid:90) [Equation]
Revised: L p ( K, R ) for all | r | ∈ [ l ] , that is,

Original: (cid:19)

Original: [Equation]
Revised: [Equation]

Original:  < ∞ .
Revised: 1 /p < ∞ .

Original: For inﬁnite spaces, we deﬁne the following collection of seriesl
Revised: For infinite spaces, we define the following collection of series

Original: [Equation]
Revised: [Equation]

Original: and [Equation]   [Equation]
Revised:   , 1 ≤ p < ∞ .

Original: Especially, l ∞ = { y | y
Revised: Especially, l ∞ = { y | y

Original: =
Revised: =

Original: { x n } n ∈ N + , and | x n | < ∞} .
Revised: { x n } n ∈ N + , and | x n | < ∞} .

Original: This paper employs E n to denote the n × n unit matrix and det( · ) to indicate the determinantoperation on the matrix.
Revised: This paper employs E n to denote the n × n unit matrix and det( · ) to indicate the determinant operation on the matrix.

Original: Two n -by- n matrices A and B are called similar , denoted as A ∼ B , if there exists an invertible n × n matrix P such that B = P − 1 AP .
Revised: Two n -by- n matrices A and B are called similar , denoted as A ∼ B , if there exists an invertible n × n matrix P such that B = P − 1 AP .

Original: The general linear group overﬁeld F , denoted as GL ( n, F ) , is the group of n × n invertible matrices with entries in F .
Revised: The general linear group over field F , denoted as GL ( n, F ) , is the group of n × n invertible matrices with entries in F .

Original: Especially,we deﬁne that a special linear group SL ( n, F ) is the subgroup of GL ( n, F ) and consists of matriceswith a determinant of 1.
Revised: Especially, we define that a special linear group SL ( n, F ) is the subgroup of GL ( n, F ) and consists of matrices with a determinant of 1.

Original: For any ﬁeld F , the n × n orthogonal matrices form the following subgroupO [Equation] n,
Revised: For any field F , the n × n orthogonal matrices form the following subgroup

Original: [Equation]
Revised: [Equation]

Original: } of the general linear group GL ( n, F ) .
Revised: of the general linear group GL ( n, F ) .

Original: Similarly, we have the special orthogonal group, denoted as
Revised: Similarly, we have the special orthogonal group, denoted as

Original: SO ( n, F ) , which consists of all orthogonal matrices of determinant 1 and is a normal subgroup of
Revised: SO ( n, F ) , which consists of all orthogonal matrices of determinant 1 and is a normal subgroup of

Original: O ( n, F ) .
Revised: O ( n, F ) .

Original: This group is also called the rotation group, generalizing that linearly transforms geometrieswhile holding the surface orientation.
Revised: This group is also called the rotation group, generalizing that linearly transforms geometries while holding the surface orientation.

Original: Let φ 2 ( x ) be the density function of some probability measure µ , which satisﬁes
Revised: Let ϕ 2 ( x ) be the density function of some probability measure µ , which satisfies

Original: [Equation]
Revised: [Equation]

Original: For continuous functions f
Revised: For continuous functions f

Original: and g , we have the following equalities under Fourier transform
Revised: and g , we have the following equalities under Fourier transform

Original: [Equation]
Revised: [Equation]

Original: and(cid:99)
Revised: and

Original: [Equation]
Revised: [Equation]

Original: for the convolution operator ∗ .
Revised: for the convolution operator ∗ .

Original:  Self-connection Spiking Neural Networks
Revised:  Self-connection Spiking Neural Networks

Original: Let n be the number of spiking neurons, u ( t ) ∈ R n and
Revised: Let n be the number of spiking neurons, u ( t ) ∈ R n and

Original: s ( t ) ∈ R n denote the vectors of membranepotentials and spikes in which u i ( t ) and s i ( t ) are the membrane potential and spikes of neuron i ∈ [ n ]at time t ≥ 0 , respectively.
Revised: s ( t ) ∈ R n denote the vectors of membrane potentials and spikes in which u i ( t ) and s i ( t ) are the membrane potential and spikes of neuron i ∈ [ n ] at time t ≥ 0 , respectively.

Original: Inheriting the recognition in [43], we here consider the self-connection
Revised: Inheriting the recognition in [44], we here consider the self-connection

Original: SNN (scSNN) as follows:
Revised: SNN (scSNN) as follows:

Original: [Equation]
Revised: [Equation]

Original: where τ m is a positive-valued hyper-parameter with respect to membrane time, I k ( t ) indicatesthe signal from input channel k ∈ [ m ] at time t , V ∈ R n × n and W ∈ R n × m denote the selfconnection and connection weights matrices, respectively.
Revised: where τ m is a positive-valued hyper-parameter with respect to membrane time, I k ( t ) indicates the signal from input channel k ∈ [ m ] at time t , V ∈ R n × n and W ∈ R n × m denote the selfconnection and connection weights matrices, respectively.

Original: Hence, (cid:80) k ∈ [ m ] W ik I k
Revised: Hence, (cid:80) k ∈ [ m ] W ik I k

Original: ( t ) indicates thesignal received by neuron i at time t , and (cid:80) j ∈ [ n ] V ij s j ( t ) denotes the effect on the membranepotential of neuron i when neuron j ﬁres a spike, as illustrated in Figure 1.
Revised: ( t ) indicates the signal received by neuron i at time t , and (cid:80) j ∈ [ n ] V ij s j ( t ) denotes the effect on the membrane potential of neuron i when neuron j fires a spike, as illustrated in Figure 1.

Original: In general, we force theself-connection matrix V to be symmetric.
Revised: In general, we force the self-connection matrix V to be symmetric.

Original: Based on the spike response model scheme [12], Eq.
Revised: Based on the spike response model scheme [12], Eq.

Original: (1) has the following solution with the boundarycondition u rest =[-
-]
Revised: (1) has the following solution with the boundary condition u rest =[-
-]

Original: [Equation]
Revised: [Equation]

Original: where t (cid:48) denotes the last ﬁring time t(cid:48)
Revised: where t ′ denotes the last firing time t

Original:  =
Revised: ′ =

Original: max { s |
Revised: max { s |

Original: u
Revised: u

Original: i ( s ) =
Revised: i ( s ) =

Original: u firing , s < t } for a pre-given ﬁringthreshold u firing > 0 .
Revised: u firing , s < t } for a pre-given firing threshold u firing > 0 .

Original: Spiking neuron model employs the typical threshold rule, that is, neuron iﬁres spikes s i ( t ) at time t if and only if u i ( t ) ≥ u firing .
Revised: Spiking neuron model employs the typical threshold rule, that is, neuron i fires spikes s i ( t ) at time t if and only if u i ( t ) ≥ u firing .

Original: After ﬁring, the membrane potential isinstantaneously reset to a lower value u rest (rest voltage).
Revised: After firing, the membrane potential is instantaneously reset to a lower value u rest (rest voltage).

Original: Note that this work does not consider usingabsolute refractory periods [14] or refractory kernels [9].
Revised: Note that this work does not consider using absolute refractory periods [14] or refractory kernels [9].

Original: Let the timing set T i
Revised: Let the timing set T i

Original: = { t | u i ( t ) =
Revised: = { t | u i ( t ) =

Original: u firing , t ∈ [0 , T ] } record the ﬁring times of neuron i and
Revised: u firing , t ∈ [0 , T ] } record the firing times of neuron i and

Original: N i = | T i | # .
Revised: N i = | T i | # .

Original: In general, we consider using the ﬁring rate f ave i( T )
Revised: In general, we consider using the firing rate f ave i

Original:  =
Revised: ( T ) =

Original: N i /T to characterize the spikedynamics, where f ave i ( T ) indicates the average number of spikes of neuron i at time interval [0 , T ] .
Revised: N i /T to characterize the spike dynamics, where f ave i ( T ) indicates the average number of spikes of neuron i at time interval [0 , T ] .

Original: It is clearly observed that f ave i ( t ) is discontinuous since N i is a step function regarding u i ( t ) andtime t .
Revised: It is clearly observed that f ave i ( t ) is discontinuous since N i is a step function regarding u i ( t ) and time t .

Original: To ensure the well-posed characteristics of the ﬁring rate functions, we here employ the spikeexcitation function f e : u → s to smooth the ﬁring procedure, for example, 
Revised: To ensure the well-posed characteristics of the firing rate functions, we here employ the spike excitation function f e : u → s to smooth the firing procedure, for example, [Equation] u firing

Original: [Equation]
Revised: [Equation]

Original: where t ( k ) and t ( k +1) are two adjacent timings from T i in which k ∈ N + and t ( k ) <
Revised: where t ( k ) and t ( k +1) are two adjacent timings from T i in which k ∈ N + and t ( k ) <

Original: t ( k +1) .
Revised: t ( k +1) .

Original: Similar smooth treatments on spike excitation functions can refer to [13, 33].
Revised: Similar smooth treatments on spike excitation functions can refer to [13, 33].

Original: Thus, f ave i ( T ) can beapproximated by an Instantaneous Firing Rate (IFR) function
Revised: Thus, f ave i ( T ) can be approximated by an Instantaneous Firing Rate (IFR) function

Original: [Equation]
Revised: [Equation]

Original: 4) where x ∈ R m , t (cid:48) ∈ T i , and t ∈
Revised: where x ∈ R m , t ′ ∈ T i , and t ∈

Original: [ T ] .
Revised: [ T ] .

Original: It is observed that the proposed IFR function f i ( x , t ) induces acontinuous dynamical system, in which f ( · , t ) :
Revised: It is observed that the proposed IFR function f i ( x , t ) induces a continuous dynamical system, in which f ( · , t ) :

Original: R m → R is a static function and f ( x , · ) :
Revised: R m → R is a Spatial function and f ( x , · ) :

Original: R → R isa temporal ﬂow.
Revised: R → R is a temporal flow.

Original: About Neural Encoding.
Revised: About Neural Encoding.

Original: The actual input data (e.g., image or video) should usually be pre-convertedinto a spiking version before fed up to SNNs.
Revised: The actual input data (e.g., image or video) should usually be pre-converted into a spiking version before fed up to SNNs.

Original: The conversion procedure is called neural encoding , asshown in Figure 1.
Revised: The conversion procedure is called neural encoding , as shown in Figure 1.

Original: There are two main categories of neural encoding approaches: temporal encodingand rate-based encoding ; the former encodes input data by exploiting the distance between timeinstances that ﬁre spikes [40], and the latter encodes input data as a count sequence of the ﬁredspikes within temporal windows [18].
Revised: There are two main categories of neural encoding approaches: temporal encoding and rate-based encoding ; the former encodes input data by exploiting the distance between time instances that fire spikes [40], and the latter encodes input data as a count sequence of the fired spikes within temporal windows [18].

Original: The rate-based encoding is the simplest and most popularscheme in SNNs.
Revised: The rate-based encoding is the simplest and most popular scheme in SNNs.

Original: The representative techniques are usually encoded by a Poisson distribution orrecorded by a dynamic vision sensor [3, 27].
Revised: The representative techniques are usually encoded by a Poisson distribution or recorded by a dynamic vision sensor [3, 27].

Original: Recent years have witnessed a lot of efforts on theinformation capacity of neural encoding, specially rate-based encoding, from empirical [6, 17] andtheoretical [24, 35, 36] sides.
Revised: Recent years have witnessed a lot of efforts on the information capacity of neural encoding, specially rate-based encoding, from empirical [6, 17] and theoretical [24, 35, 36] sides.

Original: Throughout this paper, we adopt rate-based encoding as the default andfocus on the ﬁring rates of SNNs, generalizing the computational powers concerning the spike count.
Revised: Throughout this paper, we adopt rate-based encoding as the default and focus on the firing rates of SNNs, generalizing the computational powers concerning the spike count.

Original: About Firing Rates.
Revised: About Firing Rates.

Original: When we investigate the dynamics and neural computation of SNNs, the ﬁringrates or equally the number of ﬁring spikes are the key measure of network activities for investigatingneural computation and model dynamics because of the close relation between ﬁring rates and networkfunction (including neural input, connectivity, spiking function, and ﬁring process)
Revised: When we investigate the dynamics and neural computation of SNNs, the firing rates or equally the number of firing spikes are the key measure of network activities for investigating neural computation and model dynamics because of the close relation between firing rates and network function (including neural input, connectivity, spiking function, and firing process)

Original: There aregreat efforts to use ﬁring rates in SNNs for some real-world tasks, such as vision [13, 33, 43] andspeech recognition [30, 39].
Revised: There are great efforts to use firing rates in SNNs for some real-world tasks, such as vision [13, 33, 44] and speech recognition [30, 39].

Original: Besides, Barrett et al.
Revised: Besides, Barrett et al.

Original: [5] and Chou et al.
Revised: [5] and Chou et al.

Original: [8] showed that the averagedﬁring rate can approximate the optimal solutions of some quadratic programs within polynomialcomplexity.
Revised: [8] showed that the averaged firing rate can approximate the optimal solutions of some quadratic programs within polynomial complexity.

Original: This work employs an “instantaneous” ﬁring rate rather than the averaged ﬁring rate orthe total number of ﬁring spikes used in previous studies.
Revised: This work employs an “instantaneous” firing rate rather than the averaged firing rate or the total number of firing spikes used in previous studies.

Original: This manner provides a feasible way toconstruct the continuous dynamical systems using the IFR functions, based on which we can developin-depth understandings of SNNs from spatial and temporal aspects.
Revised: This manner provides a feasible way to construct the continuous dynamical systems using the IFR functions, based on which we can develop in-depth understandings of SNNs from spatial and temporal aspects.

Original:  Approximation to Continuous Dynamical Systems
Revised:  Approximation to Continuous Dynamical Systems

Original: In this section, we ﬁrst present the universal approximation for scSNNs, and then show the parametersand time complexities of the IFR functions led by scSNNs for approximating continuous dynamicalsystems from spatial and temporal aspects, respectively.
Revised: In this section, we first present the universal approximation for scSNNs, and then show the parameters and time complexities of the IFR functions led by scSNNs for approximating continuous dynamical systems from spatial and temporal aspects, respectively.

Original: 4.1 Universal Approximation
Revised: 4.1 Universal Approximation

Original: Now, we present our ﬁrst theorem about the universal approximation of scSNNs as follows:
Revised: Now, we present our first theorem about the universal approximation of scSNNs as follows:

Original: Deﬁnition 1 Let l ∈ N + .
Revised: Definition 1 Let l ∈ N + .

Original: The function f ( x ) is said to be l - ﬁnite if f is an l -times differentiablescalar function that satisﬁes
Revised: The function f ( x ) is said to be l - finite if f is an l -times differentiable scalar function that satisfies

Original: [Equation]
Revised: [Equation]

Original: Theorem 1 Let K ⊂ R m be a compact set.
Revised: Theorem 1 Let K ⊂ R m be a compact set.

Original: If the spike excitation function f e is l -ﬁnite and w
Revised: If the spike excitation function f e is l -finite and w

Original: i ∈
Revised: i ∈

Original: Rfor any i ∈ [ n ] , then for all r ∈
Revised: R where i ∈ [ n ] , then for all r ∈

Original: [ l ] , there exists some time t such that the set of IFR functionsf ( · , t ) :
Revised: [ l ] , there exists some time t such that the set of IFR functions f ( · , t ) :

Original: K → R of the form f ( x , t ) =
Revised: K → R of the form f ( x , t ) =

Original: (cid:80) i ∈ [ n ] w i f
Revised: (cid:80) i ∈ [ n ] w i f

Original: i ( x , t ), is dense in C r ( K, R ) .
Revised: i ( x , t ) is dense in C r ( K, R ) .

Original: Theorem 1 shows  the scSNN is a universal approximator, which provides a solid cornerstone fordeveloping SNNs’ theory.
Revised: Theorem 1 shows that the scSNN is a universal approximator, which provides a solid cornerstone for developing SNNs’ theory.

Original: The proof idea of this theorem can be summarized as follows.
Revised: The proof idea of this theorem can be summarized as follows.

Original: We utilize the invertibility of the
Revised: We utilize the invertibility of the

Original: Fourier transform on Sobolev space W l,pµ ( K, R ) ( p > 1) , to project the concerned functional space
Revised: Fourier transform on Sobolev space W l,pµ ( K, R ) ( p > 1) , to project the concerned functional space

Original: C r ( K, R ) into a characteristic space, and the corresponding objective function is transformed asa single integral over the characteristic space.
Revised: C r ( K, R ) into a characteristic space, and the corresponding objective function is transformed as a single integral over the characteristic space.

Original: According to Fubini’s theorem, the approximationproblem on C r ( K, R ) can be converted into another that uses multiple integrals to a single integralon the characteristic space.
Revised: According to Fubini’s theorem, the approximation problem on C r ( K, R ) can be converted into another that uses multiple integrals to a single integral on the characteristic space.

Original: The subsequent proof can then be completed along the thought lines ofthe technical exposition given by Carslaw and Rogosinski [7].
Revised: The subsequent proof can then be completed along the thought lines of the technical exposition given by Carslaw and Rogosinski [7].

Original: The full proof of Theorem 1 can beobtained in Appendix A.
Revised: The full proof of Theorem 1 can be obtained in Appendix A.

Original: 4.2 (cid:15) -Approximation with Parameters Complexity
Revised: 4.2 ϵ -Approximation with Parameters Complexity

Original: This subsection presents our second theorem about the parameters complexity of scSNNs as follows:
Revised: This subsection presents our second theorem about the parameters complexity of scSNNs as follows:

Original: Deﬁnition 

Revised: Definition

Original: We say that g is a radial function if g ( x (cid:48) )
Revised: We say that g is a radial function if g ( x ′ )

Original: =
Revised: =

Original: g ( x ) for any (cid:107) x (cid:48) (cid:107) =
Revised: g ( x ) for any ∥ x ′ ∥ =

Original: (cid:107) x (cid:107) .
Revised: ∥ x ∥ .

Original: Theorem 2 Given a compact set K m ⊂ R m , a probability measure µ , and a radial functiong : K m → R .
Revised: Theorem 2 Given a compact set K m ⊂ R m , a probability measure µ , and a radial function g : K m → R .

Original: For some apposite spike excitation function f e and any (cid:15) > 0 , there exists some time tsuch that the radial function g can be well approximated by a one-hidden-layer scSNN of O ( Cm 15 / 4 )spiking neurons, that is,
Revised: For some apposite spike excitation function f e and any ϵ > 0 , there exists some time t such that the radial function g can be well approximated by a one-hidden-layer scSNN of O ( Cm 15 / 4 ) spiking neurons, that is,

Original: [Equation]
Revised: [Equation]

Original: Theorem 2 shows that the scSNNs  can well approximate a broad scope of radial functions withinpolynomial spiking neurons, which may shed some insights on that scSNNs admit input rotations [36],such as the rotation-based data argument techniques [29] and invariant models [23].
Revised: Theorem 2 shows that the scSNNs with polynomial spiking neurons can well approximate a broad scope of radial functions, which may shed some insights on that scSNNs admit input rotations [36], such as the rotation-based data argument techniques [29] and invariant models [23].

Original: This paper provides two ways for proving this theorem.
Revised: This paper provides two ways for proving this theorem.

Original: Here, we only introduce the proof idea ofthe interesting one (full proof is shown in Appendix B), and another proof way can be accessedin Appendix C.
Revised: Here, we only introduce the proof idea of the interesting one (full proof is shown in Appendix B), and another proof way can be accessed in Appendix C.

Original: This proof idea can be summarized as follows.
Revised: This proof idea can be summarized as follows.

Original: The radial function is invariant torotations and dependent on the input norm, corresponding to the phase and norm (i.e., radial) of inputs,respectively.
Revised: The radial function is invariant to rotations and dependent on the input norm, corresponding to the phase and norm (i.e., radial) of inputs, respectively.

Original: It is observed that s ( t ) in Eq.
Revised: It is observed that s ( t ) in Eq.

Original: (1) induces a local recurrent function concerning the inputx ( t ) .
Revised: (1) induces a local recurrent function concerning the input x ( t ) .

Original: Thus, there exists a collection of parameters such that the IFR function is dependent on the normof inputs.
Revised: Thus, there exists a collection of parameters such that the IFR function is dependent on the norm of inputs.

Original: Besides, the IFR function admits the phase since V is a symmetric matrix.
Revised: Besides, the IFR function admits the phase since V is a symmetric matrix.

Original: Thus, thereexist some linear connections (including rotation transformations) such that the weighted aggregationof these spiking neurons is invariant to rotations.
Revised: Thus, there exist some linear connections (including rotation transformations) such that the weighted aggregation of these spiking neurons is invariant to rotations.

Original: Summing up the approximation recognition, thereis a family of radial functions that can be well approximated by one-hidden-layer scSNNs.
Revised: Summing up the approximation recognition, there is a family of radial functions that can be well approximated by one-hidden-layer scSNNs.

Original: We formally begin our proof of Theorem 2 with some useful lemmas.
Revised: We formally begin our proof of Theorem 2 with some useful lemmas.

Original: Lemma 1 Let g : [ r, R ]
Revised: Lemma 1 Let g : [ r, R ]

Original: → R be an L -Lipschitz function for r ≤ R .
Revised: → R be an L -Lipschitz function for r ≤ R .

Original: For any δ > 0 , C s ≥ 1 , andn ≤ C s R 2 Lm/ ( √ rδ ) , there exist some time t and an IFR function f ( x , t ) led by a one-hidden-layerscSNN of n spiking neurons such that
Revised: For any δ > 0 , C s ≥ 1 , and n ≤ C s R 2 Lm/ ( √ rδ ) , there exist some time t and an IFR function f ( x , t ) led by a one-hidden-layer scSNN of n spiking neurons such that

Original: [Equation]
Revised: [Equation]

Original: Lemma 

Revised: Lemma

Original: For m > C 2 > 0 , g : R m → R is an L -Lipschitz radial function supported on the set
Revised: For m > C 2 > 0 , g : R m → R is an L -Lipschitz radial function supported on the set

Original: [Equation]
Revised: [Equation]

Original: For any δ > 0 , there exist some time t and an IFR function f ( x , t ) led by scSNNs of one-hidden layerwith width at most C s ( C 2 ) 3 / 2 L ( m ) 7 / 4 /δ such that
Revised: For any δ > 0 , there exist some time t and an IFR function f ( x , t ) led by scSNNs of one-hidden layer with width at most C s ( C 2 ) 3 / 2 L ( m ) 7 / 4 /δ such that

Original: [Equation]
Revised: [Equation]

Original: Lemma 2 shows that the L -Lipschitz radial functions can be approximated by the IFR function f ( x , t )led by scSNNs of one-hidden layer with polynomial parameters.
Revised: Lemma 2 shows that the L -Lipschitz radial functions can be approximated by the IFR function f ( x , t ) led by scSNNs of one-hidden layer with polynomial parameters.

Original: Lemma 3 Deﬁne
Revised: Lemma 3 Define

Original: [Equation]
Revised: [Equation]

Original: with [Equation] where (cid:15) i ∈ {− 1 , +1 } , N is a polynomial function of m , and Ω i ’s are disjoint intervals of width
Revised: where ϵ i ∈ {− 1 , +1 } , N is a polynomial function of m , and Ω i ’s are disjoint intervals of width

Original: O (1 /N ) on values in the range Θ( √ m ) .
Revised: O (1 /N ) on values in the range Θ( √ m ) .

Original: For any (cid:15) i ∈ {− 1 , +1 } ( i ∈ [ N ]) , there exists a Lipschitzfunction h : S ∆ → [ − 1 , +1] such that
Revised: For any ϵ i ∈ {− 1 , +1 } ( i ∈ [ N ]) , there exists a Lipschitz function h : S ∆ → [ − 1 , +1] such that

Original: [Equation]
Revised: [Equation]

Original: Lemma 3 shows that any non-Lipschitz function h ( x ) can be approximated and bounded by a
Revised: Lemma 3 shows that any non-Lipschitz function h ( x ) can be approximated and bounded by a

Original: Lipschitz function with density φ 2 .
Revised: Lipschitz function with density ϕ 2 .

Original: Proof of Theorem 2.
Revised: Proof of Theorem 2.

Original: Let g ( x ) =
Revised: Let g ( x ) =

Original: (cid:80) Ni =1 (cid:15)
Revised: (cid:80) Ni =1 ϵ

Original: i g i ( x ) be deﬁned by Eq.
Revised: i g i ( x ) be defined by Eq.

Original: (5) and N ≥ 4 C 5 / 2 2 m 2 .
Revised: (5) and N ≥ 4 C 5 / 2 2 m 2 .

Original: According to Lemma 3, there exists a Lipschitz function h with range [ − 1 , +1] such that
Revised: According to Lemma 3, there exists a Lipschitz function h with range [ − 1 , +1] such that

Original: [Equation]
Revised: [Equation]

Original: Based on Lemmas 1 and 2, any Lipschitz radial function supported on S ∆ can be approximated byan IFR function f ( x , t ) led by scSNNs of one-hidden layer with width at most C 3 C s ( m ) 15 / 4 , where
Revised: Based on Lemmas 1 and 2, any Lipschitz radial function supported on S ∆ can be approximated by an IFR function f ( x , t ) led by scSNNs of one-hidden layer with width at most C 3 C s ( m ) 15 / 4 , where

Original: C 3 is a constant relative to C 2 and δ .
Revised: C 3 is a constant relative to C 2 and δ .

Original: This means that there exists some time t such that
Revised: This means that there exists some time t such that

Original: [Equation]
Revised: [Equation]

Original: Thus, we have
Revised: Thus, we have

Original: [Equation]
Revised: [Equation]

Original: Hence, the range of f ( · , t ) is in [ − 1 − δ, +

Revised: Hence, the range of f ( · , t ) is in [ − 1 − δ, +

Original: + δ ] ⊆
Revised: + δ ] ⊆

Original: [ − 2 , +2] .
Revised: [ − 2 , +2] .

Original: Provided the radial function, deﬁnedby Eq.
Revised: Provided the radial function, defined by Eq.

Original: (5), we have
Revised: (5), we have

Original: [Equation]
Revised: [Equation]

Original: This implies that given constants m > C 2 > 0 and C 3 > 0 , for any δ > 0 and (cid:15) i ∈ {− 1 , +1 }( i ∈ [ N ]) , there exists some time t , such that the target radial function g can be approximated by an
Revised: This implies that given constants m > C 2 > 0 and C 3 > 0 , for any δ > 0 and ϵ i ∈ {− 1 , +1 } ( i ∈ [ N ]) , there exists some time t , such that the target radial function g can be approximated by an

Original: IFR function f ( x , t ) led by scSNNs of one-hidden layer with range in [ − 2 , +2] and width at most
Revised: IFR function f ( x , t ) led by scSNNs of one-hidden layer with range in [ − 2 , +2] and width at most

Original: C 3 C s ( m ) 15 / 4 , that is,
Revised: C 3 C s ( m ) 15 / 4 , that is,

Original: [Equation]
Revised: [Equation]

Original: This completes the proof.
Revised: This completes the proof.

Original: [Equation]
Revised: □

Original: 4.3 (cid:15) -Approximation with Time Complexity
Revised: 4.3 ϵ -Approximation with Time Complexity

Original: The IFR function generates a continuous dynamical system, comprising a static function and atemporal ﬂow.
Revised: The IFR function generates a continuous dynamical system, comprising a Spatial function and a temporal flow.

Original: Both Theorems 1 and 2 focus on the static (e.g., spatial) approximation characteristicsof the IFR function.
Revised: Both Theorems 1 and 2 focus on the Spatial (e.g., spatial) approximation characteristics of the IFR function.

Original: This subsection investigates its temporal characteristics, especially how long ittakes for a self-connected SNN to achieve a speciﬁed task or target function.
Revised: This subsection investigates its temporal characteristics, especially how long it takes for a self-connected SNN to achieve a specified task or target function.

Original: Now, we present ourthird theorem about the time complexity of scSNNs as follows.
Revised: Now, we present our third theorem about the time complexity of scSNNs as follows.

Original: Deﬁnition 

Revised: Definition

Original: A matrix V ∈ R m × n (if m ≤ n ) is said to be non-degenerate if any m × m sub-matrixof V has full rank.
Revised: A matrix V ∈ R m × n (if m ≤ n ) is said to be non-degenerate if any m × m sub-matrix of V has full rank.

Original: Theorem 3 Let n ≥ m .
Revised: Theorem 3 Let n ≥ m .

Original: Let x k ( t ) ∈
Revised: Let x k ( t ) ∈

Original: π ( λ k ) and E ( x k ) =
Revised: π ( λ k ) and E ( x k ) =

Original: λ k for λ k > 0 , k ∈ [ m ] , and t ∈ [ T ] .
Revised: λ k for λ k > 0 , k ∈ [ m ] , and t ∈ [ T ] .

Original: If
Revised: If

Original: V is a non-degenerate matrix, then for any (cid:15) > 0 and matrix G ∈ R m × n with (cid:107) G (cid:107) 2 < ∞ , when thetime complexity satisﬁes
Revised: V is a non-degenerate matrix, then for any ϵ > 0 and matrix G ∈ R m × n with ∥ G ∥ 2 < ∞ , when the time complexity satisfies

Original: [Equation]
Revised: [Equation]

Original: there exists some one-hidden-layer scSNNs with the IFR vector f = ( f 1 , . . .
Revised: there exists some one-hidden-layer scSNNs with the IFR vector f = ( f 1 , . . .

Original: [Equation]

Original:  f n ) (cid:62) such that
Revised: , f n ) ⊤ such that

Original: [Equation]
Revised: [Equation]

Original: Theorem 3 shows that without training, the scSNNs can well approximate the multivariate spikeﬂows (MSF, i.e., linear functions of λ ) within an explicit polynomial time complexity.
Revised: Theorem 3 shows that without training, the scSNNs can well approximate the multivariate spike flows (MSF, i.e., linear functions of λ ) within an explicit polynomial time complexity.

Original: In contrast tothe conventional ANN theory and our proposed theorems above that show the spatial approximationpowers, this theorem portrays the computational efﬁciency of a ﬂow function led by scSNNs.
Revised: In contrast to the conventional ANN theory and our proposed theorems above that show the spatial approximation ability, this theorem portrays the computational efficiency of a flow function led by scSNNs.

Original: Besides,there are few studies on the computational efﬁciency of SNNs, and we summarize the comparativeresults in Table 1.
Revised: Besides, there are few studies on the computational efficiency of SNNs, and we summarize the comparative results in Table 1.

Original: Notice that approximating speciﬁc functions, such as MSF in this work, is morechallenging than solving some problems with locally competitive algorithms [34].
Revised: Notice that approximating specific functions, such as MSF in this work, is more challenging than solving some problems with locally competitive algorithms [34].

Original: Thus, it is observedthat Theorem 3 provides a rigorous guarantee for SNNs to solve some algorithmic problems.
Revised: Thus, it is observed that Theorem 3 provides a rigorous guarantee for SNNs to solve some algorithmic problems.

Revised: Besides,

Revised: Theorem 3 relaxes the dimension of the self-connection matrix V from n × n to n × m , which provides a more general result for approximating MSF with any dimension.

Original: [Table]
Revised: [Table]

Original: The proof idea of this theorem originates from the attractor theory in dynamical systems.
Revised: The proof idea of this theorem originates from the attractor theory in dynamical systems.

Original: One keylemma is as follows.
Revised: One key lemma is as follows.

Original: Lemma 

Revised: Lemma

Original: Given U = √ Λ U (cid:101) U (cid:62) , ˜ λ ∈ R n , and (cid:15) > 0 , when t ≥ Ω (cid:16) √ n (cid:107) Λ G (cid:107) 2 (cid:15) (cid:107) U (cid:107) 2 (cid:17) , it holds(cid:107) U ˜ f
Revised: Given U = √ Λ U (cid:101) U ⊤ , ˜ λ ∈ R n , and ϵ > 0 , when t ≥ Ω (cid:16) √ n ∥ Λ G ∥ 2 ϵ ∥ U ∥ 2 (cid:17) , it holds ∥ U ˜ f

Original: (˜ λ , t ) − ˜ λ
Revised: (˜ λ , t ) − ˜ λ

Original: (cid:107) ≤ (cid:15) , where the modiﬁed IFR function ˜ f
Revised: ∥ ≤ ϵ , where the modified IFR function ˜ f

Original: (˜ λ , t ) is led by a scSNN without connection matrix W and fed up to the constant spike sequence ˜ λ
Revised: (˜ λ , t ) is led by a scSNN without connection matrix W and fed up to the constant spike sequence ˜ λ

Original: at every timestamp.
Revised: at every timestamp.

Original: Lemma 4 shows that when fed up to a constant spike sequence ˜ λ at every timestamp, a weightedsum of IFR functions can converge to an attractor around the inputs within a polynomial temporalcomputation.
Revised: Lemma 4 shows that when fed up to a constant spike sequence ˜ λ at every timestamp, a weighted sum of IFR functions can converge to an attractor around the inputs within a polynomial temporal computation.

Original: Based on the results of Lemma 4, it sufﬁces to prove that the expectation of theconcerned IFR function f ( x , t ) can approximate ˜ f
Revised: Based on the results of Lemma 4, it suffices to prove that the expectation of the concerned IFR function f ( x , t ) can approximate ˜ f

Original: (˜ λ , t ∗ ) within time interval [0 , T ] where t ∗ ≤
Revised: (˜ λ , t ∗ ) within time interval [0 , T ] where t ∗ ≤

Original: T ≤ Ω( √ n (cid:107) G (cid:107) 2 (cid:15) √ (cid:107) V (cid:107) 2 ) .
Revised: T ≤ Ω( √ n ∥ G ∥ 2 ϵ √ ∥ V ∥ 2 ) .

Original: Full proof of Theorem 3 can be obtained in Appendix D.
Revised: Full proof of Theorem 3 can be obtained in Appendix D.

Original: The aforementioned results, time complexity bound in detail, can be veriﬁed by a simulated experiment.
Revised: The aforementioned results, time complexity bound in detail, can be verified by a simulated experiment.

Original: We here simulate a 4 × 10 , 000 spike sequence from the Iris data sets with a timestamp of 0.001using Poisson encoding.
Revised: We here simulate a 4 × 10 , 000 spike sequence from the Iris data sets with a timestamp of 0. using Poisson encoding.

Original: We employ the one-hidden-layer scSNN [43] that contains self-connectionstructure as the conducted SNN model.
Revised: We employ the one-hidden-layer scSNN [44] that contains self-connection structure as the conducted SNN model.

Original: The number of input channels and hidden neurons are 4 and10, respectively.
Revised: The number of input channels and hidden neurons are 4 and 10, respectively.

Original: The self-connection matrix V is randomly sampled from [0 , 1] with bias = 1 / 3 .
Revised: The self-connection matrix V is randomly sampled from [0 , 1] with bias = 1 / 3 .

Original: The above conﬁgurations meet the conditions of Theorem 3.
Revised: The above configurations meet the conditions of Theorem 3.

Original: Table 2 lists the hyper-parametervalues in the conducted scSNN.
Revised: Table 2 lists the hyper-parameter values in the conducted scSNN.

Original: For any linear matrix G ∈ R 4 × 10 with (cid:107) G (cid:107) 2 < ∞ , we deﬁne an[Table] indicator t c = √ n (cid:107) G (cid:107) 2 (cid:15) √ (cid:107) V (cid:107) 2 .
Revised: For any linear matrix G ∈ R 4 × 10 with ∥ G ∥ 2 < ∞ , we define an indicator t c = √ n ∥ G ∥ 2 ϵ √ ∥ V ∥ 2 .

Original: Thus, by exploiting the relation among (cid:15) , t , and t c , we can verify the explicitpolynomial bound, especially the order of magnitude function Ω( · ) in Theorem 3. 
Revised: Thus, by exploiting the relation among ϵ , t , and t c , we can verify the explicit polynomial bound, especially the order of magnitude function Ω( · ) in Theorem 3. [Table]

Original: Figure 2 plots the experimental results.
Revised: Figure 2 plots the experimental results.

Original: From Figure 2(a), the conducted scSNN ﬁrst approximatesthe objective function at a faster rate and then maintains a lower approximation error.
Revised: From Figure 2(a), the conducted scSNN first approximates the objective function at a faster rate and then maintains a lower approximation error.

Original: Figure 2(b)signiﬁes that log( (cid:15) ) is inversely proportional to log( t c ) , i.e., a log( t c )+ b log( (cid:15) ) =
Revised: Figure 2(b) signifies that log( ϵ ) is inversely proportional to log( t c ) , i.e., a log( t c )+ b log( ϵ ) =

Original: c where a, b, c > 0 .
Revised: c where a, b, c > 0 .

Original: Figure 2(c) shows the relation plots between log( t c ) and log( t ) .
Revised: Figure 2(c) shows the relation plots between log( t c ) and log( t ) .

Original: Notice that the conducted ScNN canachieve the same approximation error at different timestamps, we only care about the shortest one,that is, the blue solid curve below the red dashed line.
Revised: Notice that the conducted ScNN can achieve the same approximation error at different timestamps, we only care about the shortest one, that is, the blue solid curve below the red dashed line.

Original: It is observed that log( t ) is proportional tolog( t c ) , i.e., log( t ) =
Revised: It is observed that log( t ) is proportional to log( t c ) , i.e., log( t ) =

Original: a log( t c ) + b where a > 0 and b ∈ R .
Revised: a log( t c ) + b where a > 0 and b ∈ R .

Original: In summary, we have demonstrated theeffectiveness of our theoretical results.
Revised: In summary, we have demonstrated the effectiveness of our theoretical results.

Original: [Figure]
Revised: [Figure]

Original:  Discussionsand Prospects
Revised:  Discussions

Original: In this paper, we present the theoretical understandings of the approximation power and computationalefﬁciency of the self-connection SNNs, i.e., scSNNs. We provided three main theorems to showthe universal approximation properties, parameters complexity for spatial approximation, and timecomplexity for temporal approximation of the scSNNs. Our theoretical results disclose the effects

Original: of self-connection of scSNNs for approximating continuous dynamical systems within polynomialparameters and time complexities.

Original: Solid Support for SNNs Theory . Despite an increasing focus on the potential of handling spatiotemporal data, the theoretical understandings of many aspects (e.g., approximation power andcomputational efﬁciency) of SNNs are still cloudy.
Revised: Despite an increasing focus on the potential of handling spatio-temporal data, the theoretical understandings of many aspects (e.g., approximation ability and computational efficiency) of SNNs are still far from clear.

Original: Some seminal studies focused on the universalityof SNNs.
Revised: Some seminal studies focused on the universality of SNNs.

Original: Maass et al.
Revised: Maass et al.

Original: [19, 20, 21] showed that some typical SNNs can simulate the standardcomputational models such as Turing machines, random access machines, threshold circuits, etc.
Revised: [19, 20, 21] showed that some typical SNNs can simulate the standard computational models such as Turing machines, random access machines, threshold circuits, etc.

Original: She et al.
Revised: She et al.

Original: [31] showed some universal approximation properties of SNNs by exploiting the spikepropagation paths.
Revised: [31] showed some universal approximation properties of SNNs by exploiting the spike propagation paths.

Original: However, to the best of our knowledge, few theoretical guarantees on theapproximation complexity and computational efﬁciency of SNNs have been provided.
Revised: However, to the best of our knowledge, few theoretical guarantees on the approximation complexity and computational efficiency of SNNs have been provided.

Original: There are onlysome academic studies on the convergence in the limit results for SNNs solving the sparse codingproblem [34, 35] and the convergence rates for SNNs solving temporal quadratic programming [8].
Revised: There are only some academic studies on the convergence in the limit results for SNNs solving the sparse coding problem [34, 35] and the convergence rates for SNNs solving temporal quadratic programming [8].

Original: This work develops a systematic framework for analyzing the approximation power and computationalefﬁciency for SNNs representing continuous dynamical systems. The theoretical derivatives not onlyshow the universal approximation properties of scSNNs but also disclose the parameters and time complexities for scSNNs approximating continuous dynamical systems. Prospectively, our workprovides solid support for the theory development of SNNs. Towards Provable and Sound SNNs .
Revised: Recently, two studies [41] and [44] provided calculable ways for approaching the lower and upper bounds of adaptive SNN systems, respectively.

Original: Our starting point is a recent advance proposed by Zhangand Zhou [43], in which they showed that adding self-connection enables SNNs to achieve adaptivedynamical systems with spatio-temporal representation.
Revised: Our starting point is a recent advance proposed by Zhang and Zhou [44], in which they showed that adding self connections enables SNNs to achieve adaptive dynamical systems with spatio-temporal representation.

Original: This paper developed an in-depth analysis ofapproximation power and computational efﬁciency of self-connection in SNNs.
Revised: This work developed an in-depth analysis of approximation ability and computational efficiency of self connections in SNNs.

Original: It is observed thatself-connection facilitates our results; re-using ﬁring spike variables s ( t ) contributes to approximatingthe norm of input sequences x ( t ) , and the symmetric self-connection matrix coincides with rotationand dual transformations, which are crucial in the proofs of Theorems 2 and 3.
Revised: It is observed that self connections facilitate our results; re-using firing spike variables s ( t ) contributes to approximating the norm of input sequences x ( t ) , and the symmetric self-connection matrix coincides with rotation and dual transformations, which are crucial in the proofs of Theorems 2 and 3.

Revised: Our theoretical derivatives not only show the universal approximation properties of scSNNs but also disclose the parameters and time complexities for scSNNs approximating continuous dynamical systems, which prospectively provide solid support for the theory development of SNNs.

Original: Unfortunately, thispaper has not yet shown the theoretical advantages of scSNNs over non-self-connection SNNs (itis valuable to be studied in the future).
Revised: Unfortunately, this paper has not yet shown the theoretical advantages of scSNNs over non-self-connection SNNs (it is valuable to be studied in the future).

Original: But we believe that the current results have disclosed theimportance and power of self connections for enhancing the approximation and computational abilityof SNNs, which may shed some insights on developing provable and sound SNNs.
Revised: But we believe that the current results have disclosed the importance and power of self connections for enhancing the approximation and computational ability of SNNs, which may shed some insights on developing provable and sound SNNs.

Original: In light of the preceding merits, many issues are worthy of being studied in the future.
Revised: In light of the preceding merits, many issues are worthy of being studied in the future.

Original: One importantfuture issue is to explore some practical techniques for scSNNs.
Revised: One important future issue is to explore some practical techniques for scSNNs.

Original: For example, Theorem 2 shows thepower of scSNNs on representing radial or equally rotation-invariant functions.
Revised: For example, Theorem 2 shows the power of scSNNs on representing radial or equally rotation-invariant functions.

Original: So we conjecture thatadding self connections may be more compatible with some invariant models and data augmentationtechniques, such as image rotation [23, 29].
Revised: So we conjecture that adding self connections may be more compatible with some invariant models and data augmentation techniques, such as image rotation [23, 29].

Original: Besides, the current implementation roughly follows theideas of Zhang and Zhou [43], i.e., the effect from the k -th neuron to the i -th neuron equals to the lastspike of neuron k multiplied by a weighted factor as shown in Eq.
Revised: Besides, the current implementation roughly follows the ideas of Zhang and Zhou [44], i.e., the effect from the k -th neuron to the i -th neuron equals to the last spike of neuron k multiplied by a weighted factor as shown in Eq.

Original: Such a self-connection graphenable SNNs to maintain adaptive characteristics for representing continuous dynamical systems.
Revised: Such a self-connection graph enable SNNs to maintain adaptive characteristics for representing continuous dynamical systems.

Original: However, it inevitably leads to a larger memory consumption when the input spike sequences arehigh-dimensional and high-frequency.
Revised: However, it inevitably leads to a larger memory consumption when the input spike sequences are high-dimensional and high-frequency.

Original: Therefore, it is prospective to explore some more practicaltechniques or modules for scSNNs, which may open up possibilities for achieving sound SNNs.
Revised: Therefore, it is prospective to explore some more practical techniques or modules for scSNNs, which may open up possibilities for achieving sound SNNs.

Original: Another important future issue is to develop in-depth theoretical understandings of SNNs, fromaspects of approximation complexity, computational efﬁciency, representation ability [15, 26, 41],and over-parameterized architectures [4, 44].
Revised: Another important future issue is to develop in-depth theoretical understandings of SNNs, from aspects of approximation complexity, computational efficiency, representation ability [15, 26, 43], and over-parameterized architectures [4, 45].

Original: Furthermore, it is interesting to explore the theoreticaladvantages of SNNs with self connections over SNNs without ones, especially from the perspectivesof approximation, optimization, and generalization
Revised: Furthermore, it is interesting to explore the theoretical advantages of SNNs with self connections over SNNs without ones, especially from the perspectives of approximation, optimization, and generalization.

Revised:  Conclusions

Revised: In this paper, we present the theoretical understandings of the approximation ability and computational efficiency of the self-connection SNNs, i.e., scSNNs.

Revised: We provided three main theorems to show the universal approximation properties, parameters complexity for spatial approximation, and time complexity for temporal approximation of the scSNNs.

Revised: Our theoretical results disclose the effects of self connections of scSNNs for approximating continuous dynamical systems using polynomial number of parameters within time complexities.

Revised: Acknowledgments and Disclosure of Funding

Revised: This research was supported by the National Natural Science Foundation of China (NSFC 61921006) and the Collaborative Innovation Center of Novel Software Technology and Industrialization