Original: Theoretically Provable Spiking Neural Networks
Revised: Theoretically Provable Spiking Neural Networks

Original: Abstract
Revised: Abstract

Original: Spiking neural networks have attracted increasing attention in recent years due to their potential of handling time-dependent data.
Revised: Spiking neural networks have attracted increasing attention in recent years due to their potential of handling time-dependent data.

Original: Many algorithms and techniques have been developed; however, theoretical understandings of many aspects of spiking neural networks are far from clear.
Revised: Many algorithms and techniques have been developed; however, theoretical understandings of many aspects of spiking neural networks are far from clear.

Original: A recent work [44] disclosed that typical spiking neural networks could hardly work on spatio-temporal data due to their bifurcation dynamics and suggested that the self-connection structure has to be added.
Revised: A recent work [44] disclosed that typical spiking neural networks could hardly work on spatio-temporal data due to their bifurcation dynamics and suggested that the self-connection structure has to be added.

Original: In this paper, we theoretically investigate the approximation ability and computational efﬁciency of spiking neural networks with self connections, and show that the self-connection structure enables spiking neural networks to approximate discrete dynamical systems using a polynomial number of parameters within polynomial time complexities.
Revised: In this paper, we theoretically investigate the approximation ability and computational efﬁciency of spiking neural networks with self connections, and show that the self-connection structure enables spiking neural networks to approximate discrete dynamical systems using a polynomial number of parameters within polynomial time complexities.

Original: Our theoretical results may shed some insight for the future studies of spiking neural networks.
Revised: Our theoretical results may shed some insight for the future studies of spiking neural networks.

Original:  Introduction
Revised:  Introduction

Original: The past decades have witnessed an increasing interest in spiking neural networks (SNNs) due to their great potential of modeling time-dependent data [28, 32, 37].
Revised: The past decades have witnessed an increasing interest in spiking neural networks (SNNs) due to their great potential of modeling time-dependent data [28, 32, 37].

Original: The fundamental units of SNNs are usually formulated as the combination of an integrated process (e g., some ﬁrst-order parabolic equations) and a step ﬁring function.
Revised: The fundamental units of SNNs are usually formulated as the combination of an integrated process (e g., some ﬁrst-order parabolic equations) and a step ﬁring function.

Original: There has been signiﬁcant progress on computational and implementation techniques for SNNs in computer vision [13, 33], speech recognition [30, 39], reinforcement learning [11, 38], few-short learning [16, 22, 25], etc.
Revised: There has been signiﬁcant progress on computational and implementation techniques for SNNs in computer vision [13, 33], speech recognition [30, 39], reinforcement learning [11, 38], few-short learning [16, 22, 25], etc.

Original: However, theoretical understandings of many aspects of SNNs, such as the approximation ability and computational efﬁciency on spatio-temporal systems are far from clear.
Revised: However, theoretical understandings of many aspects of SNNs, such as the approximation ability and computational efﬁciency on spatio-temporal systems are far from clear.

Original: Some researchers [19, 20, 21, 31] focused on the approximation universality of SNNs in which some typical SNNs can simulate the standard computational models such as Turing machines, random access machines, threshold circuits, sequence-to-sequence mapping, etc.
Revised: Some researchers [19, 20, 21, 31] focused on the approximation universality of SNNs in which some typical SNNs can simulate the standard computational models such as Turing machines, random access machines, threshold circuits, sequence-to-sequence mapping, etc.

Original: There are also efforts on the computational efﬁciency of SNNs for some speciﬁc issues, such as the convergence in the limit results and computational complexity of SNNs for the sparse coding problem [34, 35] and temporal quadratic programming [5, 8], respectively.
Revised: There are also efforts on the computational efﬁciency of SNNs for some speciﬁc issues, such as the convergence in the limit results and computational complexity of SNNs for the sparse coding problem [34, 35] and temporal quadratic programming [5, 8], respectively.

Original: Amazingly, a recent study [44] theoretically proved that, contrary to previous beliefs, typical SNNs can hardly work well on spatio-temporal data, because they in nature are bifurcation dynamical systems with ﬁxed eigenvalues in which many patterns inherently cannot be learned.
Revised: Amazingly, a recent study [44] theoretically proved that, contrary to previous beliefs, typical SNNs can hardly work well on spatio-temporal data, because they in nature are bifurcation dynamical systems with ﬁxed eigenvalues in which many patterns inherently cannot be learned.

Original: They also suggested that adding self-connection structure can enhance the representation ability of SNNs on spatio-temporal systems that fully connect the spiking neurons in the same layer and solves adaptive eigenvalues of discrete dynamical systems.
Revised: They also suggested that adding self-connection structure can enhance the representation ability of SNNs on spatio-temporal systems that fully connect the spiking neurons in the same layer and solves adaptive eigenvalues of discrete dynamical systems.

Original: In this paper, we theoretically investigate the approximation ability and computational efﬁciency of the self-connection spike neural networks (scSNNs).
Revised: In this paper, we theoretically investigate the approximation ability and computational efﬁciency of the self-connection spike neural networks (scSNNs).

Original: Our theoretical results show that equipped with self connections, scSNNs can approximate discrete dynamical systems using polynomial number of parameters within polynomial time complexities.
Revised: Our theoretical results show that equipped with self connections, scSNNs can approximate discrete dynamical systems using polynomial number of parameters within polynomial time complexities.

Original: Our main contributions are summarized as follows:
Revised: Our main contributions are summarized as follows:

Original: • We prove that the proposed scSNNs are universal approximators in Theorem 1.
Revised: • We prove that the proposed scSNNs are universal approximators in Theorem 1.

Original: • As for spatial approximation, we prove that a broad range of radial functions can be well approximated by scSNNs with polynomial spiking neurons in Theorem 2.
Revised: • As for spatial approximation, we prove that a broad range of radial functions can be well approximated by scSNNs with polynomial spiking neurons in Theorem 2.

Original: • As for temporal approximation, we prove that multivariate spike ﬂows can be approximated by scSNNs within polynomial time in Theorem 3 and verify this conclusion in simulation experiments.
Revised: • As for temporal approximation, we prove that multivariate spike ﬂows can be approximated by scSNNs within polynomial time in Theorem 3 and verify this conclusion in simulation experiments.

Original: The rest of this paper is organized as follows.
Revised: The rest of this paper is organized as follows.

Original: Section 2 introduces notations.
Revised: Section 2 introduces notations.

Original: Section 3 presents the scSNNs and some related concepts.
Revised: Section 3 presents the scSNNs and some related concepts.

Original: Section 4 provides three key theorems to show the universal approximation ability and (parameters and time) complexity of scSNNs for approximating the discrete dynamical system.
Revised: Section 4 provides three key theorems to show the universal approximation ability and (parameters and time) complexity of scSNNs for approximating the discrete dynamical system.

Original: Section 5 develops in-depth discussions of the effects led by self-connection.
Revised: Section 5 develops in-depth discussions of the effects led by self-connection.

Original: Section 6 concludes this work.
Revised: Section 6 concludes this work.

Original:  Notations
Revised:  Notations

Original: Here, we provide some useful notations, which are detailed in Appendix.
Revised: Here, we provide some useful notations, which are detailed in Appendix.

Original: Let [ N ] = { 1 , 2 , . . .
Revised: Let [ N ] = { 1 , 2 , . . .

Original: , N } be an integer set for N ∈ N + , and | · | # denotes the number of elements in a collection, e.g., | [ N ] | # = N .
Revised: , N } be an integer set for N ∈ N + , and | · | # denotes the number of elements in a collection, e.g., | [ N ] | # = N .

Original: Let i =
Revised: Let i =

Original: √− 1 be the imaginary unit, and x ≼ 0 means that every element x i ≤ 0 for any i .
Revised: √− 1 be the imaginary unit, and x ≼ 0 means that every element x i ≤ 0 for any i .

Original: Let the sphere S ( r ) and globe B ( r ) be S ( r )
Revised: Let the sphere S ( r ) and globe B ( r ) be S ( r )

Original: = { x | ∥ x ∥ = r } and B ( r ) =
Revised: = { x | ∥ x ∥ = r } and B ( r ) =

Original: { x | ∥ x ∥ ≤ r } for any r ∈ R , respectively.
Revised: { x | ∥ x ∥ ≤ r } for any r ∈ R , respectively.

Original: Given a function g ( n ) , we denote by h 1 ( n ) =
Revised: Given a function g ( n ) , we denote by h 1 ( n ) =

Original: Θ( g ( n )) if there exist positive constants c 1 , c 2 and n 0 such that c 1 g ( n ) ≤ h 1 ( n ) ≤ c 2 g ( n ) for every n ≥ n 0 ; h 2 ( n ) =
Revised: Θ( g ( n )) if there exist positive constants c 1 , c 2 and n 0 such that c 1 g ( n ) ≤ h 1 ( n ) ≤ c 2 g ( n ) for every n ≥ n 0 ; h 2 ( n ) =

Original: O ( g ( n )) if there exist positive constants c and n 0 such that h 2 ( n ) ≤ cg ( n ) for every n ≥ n 0 ; h 3 ( n ) =
Revised: O ( g ( n )) if there exist positive constants c and n 0 such that h 2 ( n ) ≤ cg ( n ) for every n ≥ n 0 ; h 3 ( n ) =

Original: Ω( g ( n )) if there exist positive constants c and n 0 such that h 3 ( n ) ≥ cg ( n ) for every n ≥ n 0 ; h 4 ( n ) =
Revised: Ω( g ( n )) if there exist positive constants c and n 0 such that h 3 ( n ) ≥ cg ( n ) for every n ≥ n 0 ; h 4 ( n ) =

Original: o ( g ( n )) if there exists positive constant n 0 such that
Revised: o ( g ( n )) if there exists positive constant n 0 such that

Original: h 4 ( n ) < cg ( n ) for every c > 0 and n ≥ n 0 .
Revised: h 4 ( n ) < cg ( n ) for every c > 0 and n ≥ n 0 .

Original: Let C ( K, R ) be the set of all scalar functions f : K
Revised: Let C ( K, R ) be the set of all scalar functions f : K

Original: → R continuous on K ⊂ R n .
Revised: → R continuous on K ⊂ R n .

Original: Given α = ( α 1 , α 2 , . . .
Revised: Given α = ( α 1 , α 2 , . . .

Original: , α m ) ⊤ ∈ N m , we deﬁne
Revised: , α m ) ⊤ ∈ N m , we deﬁne

Original: [Equation]
Revised: [Equation]

Original: where x = ( x 1 , x 2 , . . .
Revised: where x = ( x 1 , x 2 , . . .

Original: , x n ) ∈ K .
Revised: , x n ) ∈ K .

Original: Further, we deﬁne
Revised: Further, we deﬁne

Original: C l ( K, R ) =
Revised: C l ( K, R ) =

Original: { f | f ∈ C ( K, R ) and D r f ∈ C ( K, R ) , for r ∈
Revised: { f | f ∈ C ( K, R ) and D r f ∈ C ( K, R ) , for r ∈

Original: [ l ] } .
Revised: [ l ] } .

Original: For 1 ≤ p < ∞ , we deﬁne
Revised: For 1 ≤ p < ∞ , we deﬁne

Original: [Equation]
Revised: [Equation]

Original: This work considers the Sobolev space W l,pµ ( K, R ) , deﬁned as the collection of all functions f ∈
Revised: This work considers the Sobolev space W l,pµ ( K, R ) , deﬁned as the collection of all functions f ∈

Original: C l ( K, R ) and D r f ∈
Revised: C l ( K, R ) and D r f ∈

Original: L p ( K, R ) for all | α | ∈
Revised: L p ( K, R ) for all | α | ∈

Original: [ l ] , that is,
Revised: [ l ] , that is,

Original: [Equation]
Revised: [Equation]

Original: This paper employs E n to denote the n × n unit matrix and det( · ) to indicate the determinant operation on the matrix.
Revised: This paper employs E n to denote the n × n unit matrix and det( · ) to indicate the determinant operation on the matrix.

Original: Two n -by- n matrices A and B are called similar , denoted as A ∼ B , if there exists an invertible n × n matrix P such that B = P − 1 AP .
Revised: Two n -by- n matrices A and B are called similar , denoted as A ∼ B , if there exists an invertible n × n matrix P such that B = P − 1 AP .

Original: The general linear group over ﬁeld F , denoted as GL ( n, F ) , is the set of n × n invertible matrices with entries in F .
Revised: The general linear group over ﬁeld F , denoted as GL ( n, F ) , is the set of n × n invertible matrices with entries in F .

Original: Especially, we deﬁne that a special linear group SL ( n, F ) is the subgroup of GL ( n, F ) and consists of matrices with determinant 1.
Revised: Especially, we deﬁne that a special linear group SL ( n, F ) is the subgroup of GL ( n, F ) and consists of matrices with determinant 1.

Original: For any ﬁeld F , the n × n orthogonal matrices form the following subgroup
Revised: For any ﬁeld F , the n × n orthogonal matrices form the following subgroup

Original: [Equation]
Revised: [Equation]

Original: of the general linear group GL ( n, F ) .
Revised: of the general linear group GL ( n, F ) .

Original: Similarly, we have the special orthogonal group, denoted as
Revised: Similarly, we have the special orthogonal group, denoted as

Original: SO ( n, F ) , which consists of all orthogonal matrices of determinant 1 and is a normal subgroup of
Revised: SO ( n, F ) , which consists of all orthogonal matrices of determinant 1 and is a normal subgroup of

Original: O ( n, F ) .
Revised: O ( n, F ) .

Original: This group is also called the rotation group, generalizing that linearly transforms geometries while holding the surface orientation.
Revised: This group is also called the rotation group, generalizing that linearly transforms geometries while holding the surface orientation.

Original: Let ϕ 2 ( x ) be the density function of some probability measure µ , which satisﬁes Z
Revised: Let ϕ 2 ( x ) be the density function of some probability measure µ , which satisﬁes Z

Original: [Equation]
Revised: [Equation]

Original: For continuous functions f
Revised: For continuous functions f

Original: and g , we have the following equalities under Fourier transform
Revised: and g , we have the following equalities under Fourier transform

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: for the convolution operator ∗ .
Revised: for the convolution operator ∗ .

Original:  Self-connection Spiking Neural Networks
Revised:  Self-connection Spiking Neural Networks

Original: Let n be the number of spiking neurons, u ( t ) ∈ R n and
Revised: Let n be the number of spiking neurons, u ( t ) ∈ R n and

Original: s ( t ) ∈ R n denote the vectors of membrane potentials and spikes in which u i ( t ) and s i ( t ) are the membrane potential and spikes of neuron i ∈ [ n ] at time t ≥ 0 , respectively.
Revised: s ( t ) ∈ R n denote the vectors of membrane potentials and spikes in which u i ( t ) and s i ( t ) are the membrane potential and spikes of neuron i ∈ [ n ] at time t ≥ 0 , respectively.

Original: Inheriting the recognition in [44], we here consider the selfconnection SNN (scSNN) as follows:
Revised: Inheriting the recognition in [44], we here consider the selfconnection SNN (scSNN) as follows:

Original: [Equation]
Revised: [Equation]

Original: where τ m is a positive-valued hyper-parameter with respect to membrane time, I k ( t ) indicates the signal from input channel k ∈ [ m ] at time t , V ∈ R n × n and W ∈ R n × m denote the self-connection and connection weights matrices, respectively.
Revised: where τ m is a positive-valued hyper-parameter with respect to membrane time, I k ( t ) indicates the signal from input channel k ∈ [ m ] at time t , V ∈ R n × n and W ∈ R n × m denote the self-connection and connection weights matrices, respectively.

Original: Hence, P k ∈ [ m ] W ik I k
Revised: Hence, P k ∈ [ m ] W ik I k

Original: ( t ) indicates the signal received by neuron i at time t , and P j ∈ [ n ] V ij s j ( t ) denotes the effect on the membrane potential of neuron i when neuron j ﬁres a spike, as illustrated in Figure 1.
Revised: ( t ) indicates the signal received by neuron i at time t , and P j ∈ [ n ] V ij s j ( t ) denotes the effect on the membrane potential of neuron i when neuron j ﬁres a spike, as illustrated in Figure 1.

Original: In general, we force the selfconnection matrix V to be symmetric.
Revised: In general, we force the selfconnection matrix V to be symmetric.

Original: Based on the spike response model scheme [12], Eq.
Revised: Based on the spike response model scheme [12], Eq.

Original: (1) has the following solution with the boundary condition u rest =
Revised: (1) has the following solution with the boundary condition u rest =

Original: [Equation]
Revised: [Equation]

Original: where t ′ denotes the last ﬁring time t ′
Revised: where t ′ denotes the last ﬁring time t ′

Original: =
Revised: =

Original: max { s |
Revised: max { s |

Original: u
Revised: u

Original: i ( s ) =
Revised: i ( s ) =

Original: u firing , s < t } for a pre-given ﬁring threshold u firing > 0 .
Revised: u firing , s < t } for a pre-given ﬁring threshold u firing > 0 .

Original: Spiking neuron model employs the typical threshold rule, that is, neuron i ﬁres spikes s i ( t ) at time t if and only if u i ( t ) ≥ u firing .
Revised: Spiking neuron model employs the typical threshold rule, that is, neuron i ﬁres spikes s i ( t ) at time t if and only if u i ( t ) ≥ u firing .

Original: After ﬁring, the membrane potential is instantaneously reset to a lower value u rest (rest voltage).
Revised: After ﬁring, the membrane potential is instantaneously reset to a lower value u rest (rest voltage).

Original: Note that this work does not consider using absolute refractory periods [14] or refractory kernels [9].
Revised: Note that this work does not consider using absolute refractory periods [14] or refractory kernels [9].

Original: Let the timing set T i
Revised: Let the timing set T i

Original: = { t | u i ( t ) =
Revised: = { t | u i ( t ) =

Original: u firing , t ∈ [0 , T ] } record the ﬁring times of neuron i and
Revised: u firing , t ∈ [0 , T ] } record the ﬁring times of neuron i and

Original: N i = | T i | # .
Revised: N i = | T i | # .

Original: In general, we consider using the ﬁring rate f ave i ( T )
Revised: In general, we consider using the ﬁring rate f ave i ( T )

Original: =
Revised: =

Original: N i /T to characterize the spike dynamics, where f ave i ( T ) indicates the average number of spikes of neuron i at time interval [0 , T ] .
Revised: N i /T to characterize the spike dynamics, where f ave i ( T ) indicates the average number of spikes of neuron i at time interval [0 , T ] .

Original: It is clearly observed that f ave i ( t ) is discontinuous since N i is a step function regarding u i ( t ) and time t .
Revised: It is clearly observed that f ave i ( t ) is discontinuous since N i is a step function regarding u i ( t ) and time t .

Original: To ensure the well-posed characteristics of the ﬁring rate functions, we here employ the spike excitation function f e : u → s to smooth the ﬁring procedure, for example,
Revised: To ensure the well-posed characteristics of the ﬁring rate functions, we here employ the spike excitation function f e : u → s to smooth the ﬁring procedure, for example,

Original: [Equation]
Revised: [Equation]

Original: where t ( k ) and t ( k +1) are two adjacent timings from T i in which k ∈ N + and t ( k ) <
Revised: where t ( k ) and t ( k +1) are two adjacent timings from T i in which k ∈ N + and t ( k ) <

Original: t ( k +1) .
Revised: t ( k +1) .

Original: Similar smooth treatments on spike excitation functions can refer to [13, 33].
Revised: Similar smooth treatments on spike excitation functions can refer to [13, 33].

Original: Thus, f ave i ( T ) can be approximated by an Instantaneous Firing Rate (IFR) function
Revised: Thus, f ave i ( T ) can be approximated by an Instantaneous Firing Rate (IFR) function

Original: [Equation]
Revised: [Equation]

Original: where x ∈ R m , t ′ ∈ T i , and t ∈
Revised: where x ∈ R m , t ′ ∈ T i , and t ∈

Original: [ T ] .
Revised: [ T ] .

Original: It is observed that the proposed IFR function f i ( x , t ) induces a discrete dynamical system, in which f ( · , t ) :
Revised: It is observed that the proposed IFR function f i ( x , t ) induces a discrete dynamical system, in which f ( · , t ) :

Original: R m → R is a Spatial function and f ( x , · ) :
Revised: R m → R is a Spatial function and f ( x , · ) :

Original: R → R is a temporal ﬂow.
Revised: R → R is a temporal ﬂow.

Original: About Neural Encoding.
Revised: About Neural Encoding.

Original: The actual input data (e.g., image or video) should usually be preconverted into a spiking version before fed up to SNNs.
Revised: The actual input data (e.g., image or video) should usually be preconverted into a spiking version before fed up to SNNs.

Original: The conversion procedure is called neural encoding , as shown in Figure 1.
Revised: The conversion procedure is called neural encoding , as shown in Figure 1.

Original: There are two main categories of neural encoding approaches: temporal encoding and rate-based encoding ; the former encodes input data by exploiting the distance between time instances that ﬁre spikes [40], and the latter encodes input data as a count sequence of the ﬁred spikes within temporal windows [18].
Revised: There are two main categories of neural encoding approaches: temporal encoding and rate-based encoding ; the former encodes input data by exploiting the distance between time instances that ﬁre spikes [40], and the latter encodes input data as a count sequence of the ﬁred spikes within temporal windows [18].

Original: The rate-based encoding is the simplest and most popular scheme in SNNs.
Revised: The rate-based encoding is the simplest and most popular scheme in SNNs.

Original: The representative techniques are usually encoded by a Poisson distribution or recorded by a dynamic vision sensor [3, 27].
Revised: The representative techniques are usually encoded by a Poisson distribution or recorded by a dynamic vision sensor [3, 27].

Original: Recent years have witnessed a lot of efforts on the information capacity of neural encoding, specially rate-based encoding, from empirical [6, 17] and theoretical [24, 35, 36] sides.
Revised: Recent years have witnessed a lot of efforts on the information capacity of neural encoding, specially rate-based encoding, from empirical [6, 17] and theoretical [24, 35, 36] sides.

Original: Throughout this paper, we adopt rate-based encoding as the default and focus on the ﬁring rates of SNNs, generalizing the computational powers concerning the spike count.
Revised: Throughout this paper, we adopt rate-based encoding as the default and focus on the ﬁring rates of SNNs, generalizing the computational powers concerning the spike count.

Original: About Firing Rates.
Revised: About Firing Rates.

Original: When we investigate the dynamics and neural computation of SNNs, the ﬁring rates or equally the number of ﬁring spikes are the key measure of network activities for investigating neural computation and model dynamics because of the close relation between ﬁring rates and network function (including neural input, connectivity, spiking function, and ﬁring process)
Revised: When we investigate the dynamics and neural computation of SNNs, the ﬁring rates or equally the number of ﬁring spikes are the key measure of network activities for investigating neural computation and model dynamics because of the close relation between ﬁring rates and network function (including neural input, connectivity, spiking function, and ﬁring process)

Original: There are great efforts to use ﬁring rates in SNNs for some real-world tasks, such as vision [13, 33, 44] and speech recognition [30, 39].
Revised: There are great efforts to use ﬁring rates in SNNs for some real-world tasks, such as vision [13, 33, 44] and speech recognition [30, 39].

Original: Besides, Barrett et al.
Revised: Besides, Barrett et al.

Original: [5] and Chou et al.
Revised: [5] and Chou et al.

Original: [8] showed that the averaged ﬁring rate can approximate the optimal solutions of some quadratic programs within polynomial complexity.
Revised: [8] showed that the averaged ﬁring rate can approximate the optimal solutions of some quadratic programs within polynomial complexity.

Original: This work employs an “instantaneous” ﬁring rate rather than the averaged ﬁring rate or the total number of ﬁring spikes used in previous studies.
Revised: This work employs an “instantaneous” ﬁring rate rather than the averaged ﬁring rate or the total number of ﬁring spikes used in previous studies.

Original: This manner provides a feasible way to construct the discrete dynamical systems using the IFR functions, based on which we can develop in-depth understandings of SNNs from spatial and temporal aspects.
Revised: This manner provides a feasible way to construct the discrete dynamical systems using the IFR functions, based on which we can develop in-depth understandings of SNNs from spatial and temporal aspects.

Original:  Approximation to Discrete Dynamical Systems
Revised:  Approximation to Discrete Dynamical Systems

Original: In this section, we ﬁrst present the universal approximation for scSNNs, and then show the parameters and time complexities of the IFR functions led by scSNNs for approximating discrete dynamical systems from spatial and temporal aspects, respectively.
Revised: In this section, we ﬁrst present the universal approximation for scSNNs, and then show the parameters and time complexities of the IFR functions led by scSNNs for approximating discrete dynamical systems from spatial and temporal aspects, respectively.

Original: 4.1 Universal Approximation
Revised: 4.1 Universal Approximation

Original: Now, we present our ﬁrst theorem about the universal approximation of scSNNs as follows:
Revised: Now, we present our ﬁrst theorem about the universal approximation of scSNNs as follows:

Original: Deﬁnition 1 Let l ∈ N + .
Revised: Deﬁnition 1 Let l ∈ N + .

Original: The function f ( x ) is said to be l - ﬁnite if f is an l -times differentiable scalar function that satisﬁes
Revised: The function f ( x ) is said to be l - ﬁnite if f is an l -times differentiable scalar function that satisﬁes

Original: [Equation]
Revised: [Equation]

Original: Theorem 1 Let K ⊂ R m be a compact set.
Revised: Theorem 1 Let K ⊂ R m be a compact set.

Original: If the spike excitation function f e is l -ﬁnite and w
Revised: If the spike excitation function f e is l -ﬁnite and w

Original: i ∈ R where i ∈ [ n ] , then for all r ∈
Revised: i ∈ R where i ∈ [ n ] , then for all r ∈

Original: [ l ] , there exists some time t such that the set of IFR functions f ( · , t ) :
Revised: [ l ] , there exists some time t such that the set of IFR functions f ( · , t ) :

Original: K → R of the form f ( x , t ) =
Revised: K → R of the form f ( x , t ) =

Original: P i ∈
Revised: P i ∈

Original: [ n ] w i f i ( x , t ) is dense in C r ( K, R ) .
Revised: [ n ] w i f i ( x , t ) is dense in C r ( K, R ) .

Original: Theorem 1 shows that the scSNN is a universal approximator, which provides a solid cornerstone for developing SNNs’ theory.
Revised: Theorem 1 shows that the scSNN is a universal approximator, which provides a solid cornerstone for developing SNNs’ theory.

Original: The proof idea of this theorem can be summarized as follows.
Revised: The proof idea of this theorem can be summarized as follows.

Original: We utilize the invertibility of the
Revised: We utilize the invertibility of the

Original: Fourier transform on Sobolev space W l,pµ ( K, R ) ( p > 1) , to project the concerned functional space
Revised: Fourier transform on Sobolev space W l,pµ ( K, R ) ( p > 1) , to project the concerned functional space

Original: C r ( K, R ) into a characteristic space, and the corresponding objective function is transformed as a single integral over the characteristic space.
Revised: C r ( K, R ) into a characteristic space, and the corresponding objective function is transformed as a single integral over the characteristic space.

Original: According to Fubini’s theorem, the approximation problem on C r ( K, R ) can be converted into another that uses multiple integrals to a single integral on the characteristic space.
Revised: According to Fubini’s theorem, the approximation problem on C r ( K, R ) can be converted into another that uses multiple integrals to a single integral on the characteristic space.

Original: The subsequent proof can then be completed along the thought lines of the technical exposition given by Carslaw and Rogosinski [7].
Revised: The subsequent proof can then be completed along the thought lines of the technical exposition given by Carslaw and Rogosinski [7].

Original: The full proof of Theorem 1 can be obtained in Appendix A.
Revised: The full proof of Theorem 1 can be obtained in Appendix A.

Original: 4.2 ϵ -Approximation with Parameters Complexity
Revised: 4.2 ϵ -Approximation with Parameters Complexity

Original: This subsection presents our second theorem about the parameters complexity of scSNNs as follows:
Revised: This subsection presents our second theorem about the parameters complexity of scSNNs as follows:

Original: Deﬁnition 

Revised: Deﬁnition

Original: We say that g is a radial function if g ( x ′ )
Revised: We say that g is a radial function if g ( x ′ )

Original: =
Revised: =

Original: g ( x ) for any ∥ x ′ ∥ =
Revised: g ( x ) for any ∥ x ′ ∥ =

Original: ∥ x ∥ .
Revised: ∥ x ∥ .

Original: Theorem 2 Given a compact set K m ⊂ R m , a probability measure µ , and a radial function g : K m → R .
Revised: Theorem 2 Given a compact set K m ⊂ R m , a probability measure µ , and a radial function g : K m → R .

Original: For some apposite spike excitation function f e and any ϵ > 0 , there exists some time t such that the radial function g can be well approximated by a one-hidden-layer scSNN of O ( Cm 15 / 4 ) spiking neurons, that is,
Revised: For some apposite spike excitation function f e and any ϵ > 0 , there exists some time t such that the radial function g can be well approximated by a one-hidden-layer scSNN of O ( Cm 15 / 4 ) spiking neurons, that is,

Original: [Equation]
Revised: [Equation]

Original: Theorem 2 shows that the scSNNs with polynomial spiking neurons can well approximate a broad scope of radial functions, which may shed some insights on that scSNNs admit input rotations [36], such as the rotation-based data argument techniques [29] and invariant models [23].
Revised: Theorem 2 shows that the scSNNs with polynomial spiking neurons can well approximate a broad scope of radial functions, which may shed some insights on that scSNNs admit input rotations [36], such as the rotation-based data argument techniques [29] and invariant models [23].

Original: This paper provides two ways for proving this theorem.
Revised: This paper provides two ways for proving this theorem.

Original: Here, we only introduce the proof idea of the interesting one (full proof is shown in Appendix B), and another proof way can be accessed in Appendix C.
Revised: Here, we only introduce the proof idea of the interesting one (full proof is shown in Appendix B), and another proof way can be accessed in Appendix C.

Original: This proof idea can be summarized as follows.
Revised: This proof idea can be summarized as follows.

Original: The radial function is invariant to rotations and dependent on the input norm, corresponding to the phase and norm (i.e., radial) of inputs, respectively.
Revised: The radial function is invariant to rotations and dependent on the input norm, corresponding to the phase and norm (i.e., radial) of inputs, respectively.

Original: It is observed that s ( t ) in Eq.
Revised: It is observed that s ( t ) in Eq.

Original: (1) induces a local recurrent function concerning the input x ( t ) .
Revised: (1) induces a local recurrent function concerning the input x ( t ) .

Original: Thus, there exists a collection of parameters such that the IFR function is dependent on the norm of inputs.
Revised: Thus, there exists a collection of parameters such that the IFR function is dependent on the norm of inputs.

Original: Besides, the IFR function admits the phase since V is a symmetric matrix.
Revised: Besides, the IFR function admits the phase since V is a symmetric matrix.

Original: Thus, there exist some linear connections (including rotation transformations) such that the weighted aggregation of these spiking neurons is invariant to rotations.
Revised: Thus, there exist some linear connections (including rotation transformations) such that the weighted aggregation of these spiking neurons is invariant to rotations.

Original: Summing up the approximation recognition, there is a family of radial functions that can be well approximated by one-hidden-layer scSNNs.
Revised: Summing up the approximation recognition, there is a family of radial functions that can be well approximated by one-hidden-layer scSNNs.

Original: We formally begin our proof of Theorem 2 with some useful lemmas.
Revised: We formally begin our proof of Theorem 2 with some useful lemmas.

Original: Lemma 1 Let g : [ r, R ]
Revised: Lemma 1 Let g : [ r, R ]

Original: → R be an L -Lipschitz function for r ≤ R .
Revised: → R be an L -Lipschitz function for r ≤ R .

Original: For any δ > 0 , C s ≥ 1 , and n ≤ C s R 2 Lm/ ( √ rδ ) , there exist some time t and an IFR function f ( x , t ) led by a one-hidden-layer scSNN of n spiking neurons such that
Revised: For any δ > 0 , C s ≥ 1 , and n ≤ C s R 2 Lm/ ( √ rδ ) , there exist some time t and an IFR function f ( x , t ) led by a one-hidden-layer scSNN of n spiking neurons such that

Original: [Equation]
Revised: [Equation]

Original: Lemma
Revised: Lemma

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: For any δ > 0 , there exist some time t and an IFR function f ( x , t ) led by scSNNs of one-hidden layer with width at most C s ( C 2 )
Revised: For any δ > 0 , there exist some time t and an IFR function f ( x , t ) led by scSNNs of one-hidden layer with width at most C s ( C 2 )

Original: [Equation]
Revised: [Equation]

Original: 2 L ( m )
Revised: 2 L ( m )

Original: [Equation]
Revised: [Equation]

Original: 4 /δ such that
Revised: 4 /δ such that

Original: [Equation]
Revised: [Equation]

Original: Lemma 2 shows that the L -Lipschitz radial functions can be approximated by the IFR function f ( x , t ) led by scSNNs of one-hidden layer with polynomial parameters.
Revised: Lemma 2 shows that the L -Lipschitz radial functions can be approximated by the IFR function f ( x , t ) led by scSNNs of one-hidden layer with polynomial parameters.

Original: Lemma 3 Deﬁne
Revised: Lemma 3 Deﬁne

Original: [Equation]
Revised: [Equation]

Original: where ϵ i ∈ {− 1 , +1 } , N is a polynomial function of m , and Ω i ’s are disjoint intervals of width O (1 /N ) on values in the range Θ( √ m ) .
Revised: where ϵ i ∈ {− 1 , +1 } , N is a polynomial function of m , and Ω i ’s are disjoint intervals of width O (1 /N ) on values in the range Θ( √ m ) .

Original: For any ϵ i ∈ {− 1 , +1 } ( i ∈ [ N ]) , there exists a Lipschitz function h : S ∆ → [ − 1 , +1] such that Z
Revised: For any ϵ i ∈ {− 1 , +1 } ( i ∈ [ N ]) , there exists a Lipschitz function h : S ∆ → [ − 1 , +1] such that Z

Original: [Equation]
Revised: [Equation]

Original: Lemma 3 shows that any non-Lipschitz function h ( x ) can be approximated and bounded by a Lipschitz function with density ϕ 2 .
Revised: Lemma 3 shows that any non-Lipschitz function h ( x ) can be approximated and bounded by a Lipschitz function with density ϕ 2 .

Original: Proof of Theorem 2.
Revised: Proof of Theorem 2.

Original: Let g ( x ) =
Revised: Let g ( x ) =

Original: P Ni =

Revised: P Ni =

Original: ϵ i g
Revised: ϵ i g

Original: i ( x ) be deﬁned by Eq.
Revised: i ( x ) be deﬁned by Eq.

Original: (5) and N ≥ 4 C 5 / 2 2 m 2 .
Revised: (5) and N ≥ 4 C 5 / 2 2 m 2 .

Original: According to Lemma 3, there exists a Lipschitz function h with range [ − 1 , +1] such that
Revised: According to Lemma 3, there exists a Lipschitz function h with range [ − 1 , +1] such that

Original: [Equation]
Revised: [Equation]

Original: Based on Lemmas 1 and 2, any Lipschitz radial function supported on S ∆ can be approximated by an IFR function f ( x , t ) led by scSNNs of one-hidden layer with width at most C 3 C s ( m ) 15 / 4 , where
Revised: Based on Lemmas 1 and 2, any Lipschitz radial function supported on S ∆ can be approximated by an IFR function f ( x , t ) led by scSNNs of one-hidden layer with width at most C 3 C s ( m ) 15 / 4 , where

Original: C 3 is a constant relative to C 2 and δ .
Revised: C 3 is a constant relative to C 2 and δ .

Original: This means that there exists some time t such that
Revised: This means that there exists some time t such that

Original: [Equation]
Revised: [Equation]

Original: Thus, we have
Revised: Thus, we have

Original: [Equation]
Revised: [Equation]

Original: Hence, the range of f ( · , t ) is in [ − 1 − δ, +

Revised: Hence, the range of f ( · , t ) is in [ − 1 − δ, +

Original: + δ ] ⊆
Revised: + δ ] ⊆

Original: [ − 2 , +2] .
Revised: [ − 2 , +2] .

Original: Provided the radial function, deﬁned by Eq.
Revised: Provided the radial function, deﬁned by Eq.

Original: (5), we have
Revised: (5), we have

Original: [Equation]
Revised: [Equation]

Original: This implies that given constants m > C 2 > 0 and C 3 > 0 , for any δ > 0 and ϵ i ∈ {− 1 , +1 } ( i ∈ [ N ]) , there exists some time t , such that the target radial function g can be approximated by an
Revised: This implies that given constants m > C 2 > 0 and C 3 > 0 , for any δ > 0 and ϵ i ∈ {− 1 , +1 } ( i ∈ [ N ]) , there exists some time t , such that the target radial function g can be approximated by an

Original: IFR function f ( x , t ) led by scSNNs of one-hidden layer with range in [ − 2 , +2] and width at most
Revised: IFR function f ( x , t ) led by scSNNs of one-hidden layer with range in [ − 2 , +2] and width at most

Original: C 3 C s ( m ) 15 / 4 , that is,
Revised: C 3 C s ( m ) 15 / 4 , that is,

Original: [Equation]
Revised: [Equation]

Original: This completes the proof.
Revised: This completes the proof.

Original: □
Revised: □

Original: 4.3 ϵ -Approximation with Time Complexity
Revised: 4.3 ϵ -Approximation with Time Complexity

Original: The IFR function generates a discrete dynamical system, comprising a Spatial function and a temporal ﬂow.
Revised: The IFR function generates a discrete dynamical system, comprising a Spatial function and a temporal ﬂow.

Original: Both Theorems 1 and 2 focus on the Spatial (e.g., spatial) approximation characteristics of the IFR function.
Revised: Both Theorems 1 and 2 focus on the Spatial (e.g., spatial) approximation characteristics of the IFR function.

Original: This subsection investigates its temporal characteristics, especially how long it takes for a self-connected SNN to achieve a speciﬁed task or target function.
Revised: This subsection investigates its temporal characteristics, especially how long it takes for a self-connected SNN to achieve a speciﬁed task or target function.

Original: Now, we present our third theorem about the time complexity of scSNNs as follows.
Revised: Now, we present our third theorem about the time complexity of scSNNs as follows.

Original: Deﬁnition 

Revised: Deﬁnition

Original: A matrix V ∈
Revised: A matrix V ∈

Original: R m × n (if m ≤ n ) is said to be non-degenerate if any m × m sub-matrix of V has full rank.
Revised: R m × n (if m ≤ n ) is said to be non-degenerate if any m × m sub-matrix of V has full rank.

Original: Theorem 3 Let n ≥ m .
Revised: Theorem 3 Let n ≥ m .

Original: Let x k ( t ) ∈
Revised: Let x k ( t ) ∈

Original: π ( λ k ) and E ( x k ) =
Revised: π ( λ k ) and E ( x k ) =

Original: λ k for λ k > 0 , k ∈ [ m ] , and t ∈ [ T ] .
Revised: λ k for λ k > 0 , k ∈ [ m ] , and t ∈ [ T ] .

Original: If
Revised: If

Original: V is a non-degenerate matrix, then for any ϵ > 0 and matrix G ∈
Revised: V is a non-degenerate matrix, then for any ϵ > 0 and matrix G ∈

Original: R m × n with ∥ G ∥ 2 < ∞ , when the time complexity satisﬁes
Revised: R m × n with ∥ G ∥ 2 < ∞ , when the time complexity satisﬁes

Original: [Equation]
Revised: [Equation]

Original: there exists some one-hidden-layer scSNNs with the IFR vector f = ( f 1 , . . .
Revised: there exists some one-hidden-layer scSNNs with the IFR vector f = ( f 1 , . . .

Original: , f n ) ⊤ such that
Revised: , f n ) ⊤ such that

Original: [Equation]
Revised: [Equation]

Original: Theorem 3 shows that without training, the scSNNs can well approximate the multivariate spike ﬂows (MSF, i.e., linear functions of λ ) within an explicit polynomial time complexity.
Revised: Theorem 3 shows that without training, the scSNNs can well approximate the multivariate spike ﬂows (MSF, i.e., linear functions of λ ) within an explicit polynomial time complexity.

Original: In contrast to the conventional ANN theory and our proposed theorems above that show the spatial approximation ability, this theorem portrays the computational efﬁciency of a ﬂow function led by scSNNs.
Revised: In contrast to the conventional ANN theory and our proposed theorems above that show the spatial approximation ability, this theorem portrays the computational efﬁciency of a ﬂow function led by scSNNs.

Original: Besides, there are few studies on the computational efﬁciency of SNNs, and we summarize the comparative results in Table 1.
Revised: Besides, there are few studies on the computational efﬁciency of SNNs, and we summarize the comparative results in Table 1.

Original: Notice that approximating speciﬁc functions, such as MSF in this work, is more challenging than solving some problems with locally competitive algorithms [34].
Revised: Notice that approximating speciﬁc functions, such as MSF in this work, is more challenging than solving some problems with locally competitive algorithms [34].

Original: Thus, it is observed that Theorem 3 provides a rigorous guarantee for SNNs to solve some algorithmic problems.
Revised: Thus, it is observed that Theorem 3 provides a rigorous guarantee for SNNs to solve some algorithmic problems.

Original: Besides, Theorem 3 relaxes the dimension of the self-connection matrix V from n × n to n × m , which provides a more general result for approximating MSF with any dimension.
Revised: Besides, Theorem 3 relaxes the dimension of the self-connection matrix V from n × n to n × m , which provides a more general result for approximating MSF with any dimension.

Original: [Table]
Revised: [Table]

Original: The proof idea of this theorem originates from the attractor theory in dynamical systems.
Revised: The proof idea of this theorem originates from the attractor theory in dynamical systems.

Original: One key lemma is as follows.
Revised: One key lemma is as follows.

Original: Lemma 

Revised: Lemma

Original: Given U = √ Λ U e U ⊤ , ˜ λ ∈ R n , and ϵ > 0 , when t ≥ Ω (cid:16) √ n ∥ Λ G ∥ϵ ∥
Revised: Given U = √ Λ U e U ⊤ , ˜ λ ∈ R n , and ϵ > 0 , when t ≥ Ω (cid:16) √ n ∥ Λ G ∥ϵ ∥

Original: U ∥(cid:17) , it holds ∥ U ˜ f
Revised: U ∥(cid:17) , it holds ∥ U ˜ f

Original: (˜ λ , t ) − ˜ λ
Revised: (˜ λ , t ) − ˜ λ

Original: ∥ ≤ ϵ , where the modiﬁed IFR function ˜ f
Revised: ∥ ≤ ϵ , where the modiﬁed IFR function ˜ f

Original: (˜ λ , t ) is led by a scSNN without connection matrix W and fed up to the constant spike sequence ˜ λ
Revised: (˜ λ , t ) is led by a scSNN without connection matrix W and fed up to the constant spike sequence ˜ λ

Original: at every timestamp.
Revised: at every timestamp.

Original: Lemma 4 shows that when fed up to a constant spike sequence ˜ λ at every timestamp, a weighted sum of IFR functions can converge to an attractor around the inputs within a polynomial temporal computation.
Revised: Lemma 4 shows that when fed up to a constant spike sequence ˜ λ at every timestamp, a weighted sum of IFR functions can converge to an attractor around the inputs within a polynomial temporal computation.

Original: Based on the results of Lemma 4, it sufﬁces to prove that the expectation of the concerned IFR function f ( x , t ) can approximate ˜ f
Revised: Based on the results of Lemma 4, it sufﬁces to prove that the expectation of the concerned IFR function f ( x , t ) can approximate ˜ f

Original: (˜ λ , t ∗ ) within time interval [0 , T ] where t ∗ ≤
Revised: (˜ λ , t ∗ ) within time interval [0 , T ] where t ∗ ≤

Original: T ≤ Ω( √ n ∥ G ∥ 2 ϵ √ ∥ V ∥ 2 ) .
Revised: T ≤ Ω( √ n ∥ G ∥ 2 ϵ √ ∥ V ∥ 2 ) .

Original: Full proof of Theorem 3 can be obtained in Appendix D.
Revised: Full proof of Theorem 3 can be obtained in Appendix D.

Original: The aforementioned results, time complexity bound in detail, can be veriﬁed by a simulated experiment.
Revised: The aforementioned results, time complexity bound in detail, can be veriﬁed by a simulated experiment.

Original: We here simulate a 4 × 10 , 000 spike sequence from the Iris data sets with a timestamp of 0.
Revised: We here simulate a 4 × 10 , 000 spike sequence from the Iris data sets with a timestamp of 0.

Original: using Poisson encoding.
Revised: using Poisson encoding.

Original: We employ the one-hidden-layer scSNN [44] that contains self-connection structure as the conducted SNN model.
Revised: We employ the one-hidden-layer scSNN [44] that contains self-connection structure as the conducted SNN model.

Original: The number of input channels and hidden neurons are 4 and 10, respectively.
Revised: The number of input channels and hidden neurons are 4 and 10, respectively.

Original: The self-connection matrix V is randomly sampled from [0 , 1] with bias = 1 / 3 .
Revised: The self-connection matrix V is randomly sampled from [0 , 1] with bias = 1 / 3 .

Original: The above conﬁgurations meet the conditions of Theorem 3.
Revised: The above conﬁgurations meet the conditions of Theorem 3.

Original: Table 2 lists the hyper-parameter values in the conducted scSNN.
Revised: Table 2 lists the hyper-parameter values in the conducted scSNN.

Original: For any linear matrix G ∈ R 4 × 10 with ∥ G ∥ 2 < ∞ , we deﬁne an indicator t c = √ n ∥ G ∥ϵ √ ∥ V ∥ 2 .
Revised: For any linear matrix G ∈ R 4 × 10 with ∥ G ∥ 2 < ∞ , we deﬁne an indicator t c = √ n ∥ G ∥ϵ √ ∥ V ∥ 2 .

Original: Thus, by exploiting the relation among ϵ , t , and t c , we can verify the explicit polynomial bound, especially the order of magnitude function Ω( · ) in Theorem 3.
Revised: Thus, by exploiting the relation among ϵ , t , and t c , we can verify the explicit polynomial bound, especially the order of magnitude function Ω( · ) in Theorem 3.

Original: Figure 2 plots the experimental results.
Revised: Figure 2 plots the experimental results.

Original: From Figure 2(a), the conducted scSNN ﬁrst approximates the objective function at a faster rate and then maintains a lower approximation error.
Revised: From Figure 2(a), the conducted scSNN ﬁrst approximates the objective function at a faster rate and then maintains a lower approximation error.

Original: Figure 2(b) signiﬁes that log( ϵ ) is inversely proportional to log( t c ) , i.e., a log( t c ) + b log( ϵ ) =
Revised: Figure 2(b) signiﬁes that log( ϵ ) is inversely proportional to log( t c ) , i.e., a log( t c ) + b log( ϵ ) =

Original: c where a, b, c > 0 .
Revised: c where a, b, c > 0 .

Original: Figure 2(c) shows the relation plots between log( t c ) and log( t ) .
Revised: Figure 2(c) shows the relation plots between log( t c ) and log( t ) .

Original: Notice that the conducted ScNN
Revised: Notice that the conducted ScNN

Original: [Table]
Revised: [Table]

Original: can achieve the same approximation error at different timestamps, we only care about the shortest one, that is, the blue solid curve below the red dashed line.
Revised: can achieve the same approximation error at different timestamps, we only care about the shortest one, that is, the blue solid curve below the red dashed line.

Original: It is observed that log( t ) is proportional to log( t c ) , i.e., log( t ) =
Revised: It is observed that log( t ) is proportional to log( t c ) , i.e., log( t ) =

Original: a log( t c ) + b where a > 0 and b ∈ R .
Revised: a log( t c ) + b where a > 0 and b ∈ R .

Original: In summary, we have demonstrated the effectiveness of our theoretical results.
Revised: In summary, we have demonstrated the effectiveness of our theoretical results.

Original: [Figure]
Revised: [Figure]

Original:  Discussions
Revised:  Discussions

Original: Despite an increasing focus on the potential of handling spatio-temporal data, the theoretical understandings of many aspects (e.g., approximation ability and computational efﬁciency) of SNNs are still far from clear.
Revised: Despite an increasing focus on the potential of handling spatio-temporal data, the theoretical understandings of many aspects (e.g., approximation ability and computational efﬁciency) of SNNs are still far from clear.

Original: Some seminal studies focused on the universality of SNNs.
Revised: Some seminal studies focused on the universality of SNNs.

Original: Maass et al.
Revised: Maass et al.

Original: [19, 20, 21] showed that some typical SNNs can simulate the standard computational models such as Turing machines, random access machines, threshold circuits, etc.
Revised: [19, 20, 21] showed that some typical SNNs can simulate the standard computational models such as Turing machines, random access machines, threshold circuits, etc.

Original: She et al.
Revised: She et al.

Original: [31] showed some universal approximation properties of SNNs by exploiting the spike propagation paths.
Revised: [31] showed some universal approximation properties of SNNs by exploiting the spike propagation paths.

Original: However, to the best of our knowledge, few theoretical guarantees on the approximation complexity and computational efﬁciency of SNNs have been provided.
Revised: However, to the best of our knowledge, few theoretical guarantees on the approximation complexity and computational efﬁciency of SNNs have been provided.

Original: There are only some academic studies on the convergence in the limit results for SNNs solving the sparse coding problem [34, 35] and the convergence rates for SNNs solving temporal quadratic programming [8].
Revised: There are only some academic studies on the convergence in the limit results for SNNs solving the sparse coding problem [34, 35] and the convergence rates for SNNs solving temporal quadratic programming [8].

Original: Recently, two studies [41] and [44] provided calculable ways for approaching the lower and upper bounds of adaptive SNN systems, respectively.
Revised: Recently, two studies [41] and [44] provided calculable ways for approaching the lower and upper bounds of adaptive SNN systems, respectively.

Original: Our starting point is a recent advance proposed by Zhang and Zhou [44], in which they showed that adding self connections enables SNNs to achieve adaptive dynamical systems with spatio-temporal representation.
Revised: Our starting point is a recent advance proposed by Zhang and Zhou [44], in which they showed that adding self connections enables SNNs to achieve adaptive dynamical systems with spatio-temporal representation.

Original: This work developed an in-depth analysis of approximation ability and computational efﬁciency of self connections in SNNs.
Revised: This work developed an in-depth analysis of approximation ability and computational efﬁciency of self connections in SNNs.

Original: It is observed that self connections facilitate our results; re-using ﬁring spike variables s ( t ) contributes to approximating the norm of input sequences x ( t ) , and the symmetric self-connection matrix coincides with rotation and dual transformations, which are crucial in the proofs of Theorems 2 and 3.
Revised: It is observed that self connections facilitate our results; re-using ﬁring spike variables s ( t ) contributes to approximating the norm of input sequences x ( t ) , and the symmetric self-connection matrix coincides with rotation and dual transformations, which are crucial in the proofs of Theorems 2 and 3.

Original: Our theoretical derivatives not only show the universal approximation properties of scSNNs but also disclose the parameters and time complexities for scSNNs approximating discrete dynamical systems, which prospectively provide solid support for the theory development of SNNs.
Revised: Our theoretical derivatives not only show the universal approximation properties of scSNNs but also disclose the parameters and time complexities for scSNNs approximating discrete dynamical systems, which prospectively provide solid support for the theory development of SNNs.

Original: Unfortunately, this paper has not yet shown the theoretical advantages of scSNNs over non-self-connection SNNs (it is valuable to be studied in the future).
Revised: Unfortunately, this paper has not yet shown the theoretical advantages of scSNNs over non-self-connection SNNs (it is valuable to be studied in the future).

Original: But we believe that the current results have disclosed the importance and power of self connections for enhancing the approximation and computational ability of SNNs, which may shed some insights on developing provable and sound SNNs.
Revised: But we believe that the current results have disclosed the importance and power of self connections for enhancing the approximation and computational ability of SNNs, which may shed some insights on developing provable and sound SNNs.

Original: In light of the preceding merits, many issues are worthy of being studied in the future.
Revised: In light of the preceding merits, many issues are worthy of being studied in the future.

Original: One important future issue is to explore some practical techniques for scSNNs.
Revised: One important future issue is to explore some practical techniques for scSNNs.

Original: For example, Theoremshows the power of scSNNs on representing radial or equally rotation-invariant functions.
Revised: For example, Theoremshows the power of scSNNs on representing radial or equally rotation-invariant functions.

Original: So we conjecture that adding self connections may be more compatible with some invariant models and data augmentation techniques, such as image rotation [23, 29].
Revised: So we conjecture that adding self connections may be more compatible with some invariant models and data augmentation techniques, such as image rotation [23, 29].

Original: Besides, the current implementation roughly follows the ideas of Zhang and Zhou [44], i.e., the effect from the k -th neuron to the i -th neuron equals to the last spike of neuron k multiplied by a weighted factor as shown in Eq.
Revised: Besides, the current implementation roughly follows the ideas of Zhang and Zhou [44], i.e., the effect from the k -th neuron to the i -th neuron equals to the last spike of neuron k multiplied by a weighted factor as shown in Eq.

Original: Such a self-connection graph enable SNNs to maintain adaptive characteristics for representing discrete dynamical systems.
Revised: Such a self-connection graph enable SNNs to maintain adaptive characteristics for representing discrete dynamical systems.

Original: However, it inevitably leads to a larger memory consumption when the input spike sequences are high-dimensional and high-frequency.
Revised: However, it inevitably leads to a larger memory consumption when the input spike sequences are high-dimensional and high-frequency.

Original: Therefore, it is prospective to explore some more practical techniques or modules for scSNNs, which may open up possibilities for achieving sound SNNs.
Revised: Therefore, it is prospective to explore some more practical techniques or modules for scSNNs, which may open up possibilities for achieving sound SNNs.

Original: Another important future issue is to develop in-depth theoretical understandings of SNNs, from aspects of approximation complexity, computational efﬁciency, representation ability [15, 26, 43], and over-parameterized architectures [4, 45].
Revised: Another important future issue is to develop in-depth theoretical understandings of SNNs, from aspects of approximation complexity, computational efﬁciency, representation ability [15, 26, 43], and over-parameterized architectures [4, 45].

Original: Furthermore, it is interesting to explore the theoretical advantages of SNNs with self connections over SNNs without ones, especially from the perspectives of approximation, optimization, and generalization.
Revised: Furthermore, it is interesting to explore the theoretical advantages of SNNs with self connections over SNNs without ones, especially from the perspectives of approximation, optimization, and generalization.

Original:  Conclusions
Revised:  Conclusions

Original: In this paper, we present the theoretical understandings of the approximation ability and computational efﬁciency of the self-connection SNNs, i.e., scSNNs.
Revised: In this paper, we present the theoretical understandings of the approximation ability and computational efﬁciency of the self-connection SNNs, i.e., scSNNs.

Original: We provided three main theorems to show the universal approximation properties, parameters complexity for spatial approximation, and time complexity for temporal approximation of the scSNNs.
Revised: We provided three main theorems to show the universal approximation properties, parameters complexity for spatial approximation, and time complexity for temporal approximation of the scSNNs.

Original: Our theoretical results disclose the effects of self connections of scSNNs for approximating discrete dynamical systems using polynomial number of parameters within time complexities.
Revised: Our theoretical results disclose the effects of self connections of scSNNs for approximating discrete dynamical systems using polynomial number of parameters within time complexities.

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: This research was supported by the National Natural Science Foundation of China (NSFC 61921006) and the Collaborative Innovation Center of Novel Software Technology and Industrialization.Supplementary Materials of Theoretically Provable Spiking Neural Networks (Appendix)
Revised: This research was supported by the National Natural Science Foundation of China (NSFC 61921006) and the Collaborative Innovation Center of Novel Software Technology and Industrialization.

Original: In this Appendix, we provide the supplementary materials for our work “Theoretically Provable

Original: Spiking Neural Networks", constructed according to the corresponding sections therein.

Original: Before that, we review the useful notations as follows.

Original: Let [ N ] = { 1 , 2 , . .

Original: , N } be an integer set for N ∈ N + , and | · | # denotes the number of elements in a collection, e.g., | [ N ] | # = N .

Original: Let i =

Original: √− 1 be the imaginary unit, and x ≼ 0 means that every element x i ≤ 0 for any i .

Original: Let the sphere S ( r ) and globe B ( r ) be S ( r )

Original: = { x | ∥ x ∥ = r } and B ( r ) =

Original: { x | ∥ x ∥ ≤ r } for any r ∈ R , respectively.

Original: Given a function g ( n ) , we denote by h 1 ( n ) =

Original: Θ( g ( n )) if there exist positive constants c 1 , c 2 and n 0 such that c 1 g ( n ) ≤ h 1 ( n ) ≤ c 2 g ( n ) for every n ≥ n 0 ; h 2 ( n ) =

Original: O ( g ( n )) if there exist positive constants c and n 0 such that h 2 ( n ) ≤ cg ( n ) for every n ≥ n 0 ; h 3 ( n ) =

Original: Ω( g ( n )) if there exist positive constants c and n 0 such that h 3 ( n ) ≥ cg ( n ) for every n ≥ n 0 ; h 4 ( n ) =

Original: o ( g ( n )) if there exists positive constant n 0 such that

Original: h 4 ( n ) < cg ( n ) for every c > 0 and n ≥ n 0 .

Original: Let C ( K, R ) be the set of all scalar functions f : K

Original: → R continuous on K ⊂ R n .

Original: Given α = ( α 1 , α 2 , . . .

Original: , α m ) ⊤ ∈ N m , we deﬁne

Original: [Equation]

Original: where x = ( x 1 , x 2 , . . .

Original: , x n ) ∈ K .

Original: Further, we deﬁne

Original: C l ( K, R ) =

Original: { f | f ∈ C ( K, R ) and D r f ∈ C ( K, R ) , for r ∈

Original: [ l ] } .

Original: For 1 ≤ p < ∞ , we deﬁne

Original: [Equation]

Original: This work considers the Sobolev space W l,pµ ( K, R ) , deﬁned as the collection of all functions f ∈

Original: C l ( K, R ) and D r f ∈

Original: L p ( K, R ) for all | α | ∈

Original: [ l ] , that is,

Original: [Equation]

Original: This paper employs E n to denote the n × n unit matrix and det( · ) to indicate the determinant operation on the matrix.

Original: Two n -by- n matrices A and B are called similar , denoted as A ∼ B , if there exists an invertible n × n matrix P such that B = P − 1 AP .

Original: The general linear group over ﬁeld F , denoted as GL ( n, F ) , is the set of n × n invertible matrices with entries in F .

Original: Especially, we deﬁne that a special linear group SL ( n, F ) is the subgroup of GL ( n, F ) and consists of matrices with determinant 1.

Original: For any ﬁeld F , the n × n orthogonal matrices form the following subgroup

Original: [Equation]

Original: of the general linear group GL ( n, F ) .

Original: Similarly, we have the special orthogonal group, denoted as

Original: SO ( n, F ) , which consists of all orthogonal matrices of determinant 1 and is a normal subgroup of

Original: O ( n, F ) .

Original: This group is also called the rotation group, generalizing that linearly transforms geometries while holding the surface orientation.

Original: Let ϕ 2 ( x ) be the density function of some probability measure µ , which satisﬁes Z

Original: [Equation]

Original: where B is an abbreviation of the unit ball B (1) .

Original: For continuous functions f, g , we have the following equalities under Fourier transform

Original: [Equation]

Original: and

Original: [Equation]

Original: for the convolution operator ∗

Original: [Equation]

Original: A Full Proof for Theorem

Original: For convenience, we abbreviate f ( · , t ) as f ( · ) in this proof.

Original: For r ∈ [ l ] , we have

Original: [Equation]

Original: where α, β ∈ R

Original: [Equation]

Original: and the above equations hold from the Fourier transforms and some of their properties.

Original: By taking the real part of Eq.

Original: [Equation]

Original: we have

Original: [Equation]

Original: [Equation]

Original: where

Original: [Equation]

Original: In this proof

Original: [Equation]

Original: we set

Original: [Equation]

Original: and the k -th element of vector x equals to a temporal-weighted average of I k ( t ) at time interval [ t ′ , t ]

Original: [Equation]

Original: Thus, we have

Original: [Equation]

Original: Based on Eq.

Original: [Equation]

Original: we can construct a family of approximation functions of the form

Original: [Equation]

Original: [Equation]

Original: where B 1 = { x | x ≼ κ } and B 2 = { x | x ≼ ( C x m + 1) κ } .

Original: Thus

Original: [Equation]

Original: we have

Original: [Equation]

Original: [Equation]

Original: It sufﬁces to prove that D r f κ → D r f uniformly on K , as κ → ∞ .

Original: Now

Original: [Equation]

Original: For R 1 , one has

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) d

Original: [Equation]

Original: (cid:13)(cid:13)

Original: [Equation]

Original: (cid:12)(cid:12)(cid:12)

Original: [Equation]

Original: (cid:12)(cid:12)(cid:12)

Original: Z

Original: [Equation]

Original: (cid:12)(cid:12)(cid:12)

Original: [Equation]

Original: (cid:12)(cid:12)(cid:12) d

Original: [Equation]

Original: where ˜ B 1 = { β x | β x ≼ βκ } .

Original: For R 2 , one has

Original: [Equation]

Original: Z

Original: [Equation]

Original: where µ = y ⊤ x + ~ and ˜ B 2 = { x | x ≼ κ } since | µ | ≥ κ .

Original: Summing up the inequalities above, we have

Original: [Equation]

Original: with

Original: [Equation]

Original: d

Original: [Equation]

Original: and C 2 κ = ∥

Original: D r f e ( µ ) ∥ 1 , ˜ B

Original: Z

Original: R / ˜

Original: B| D r f e

Original: [Equation]

Original: µ ) | d µ , which tends to 0 as κ → ∞ .

Original: Given κ , it sufﬁces to construct a series of approximations to f κ in Eq.

Original: Formally, we deﬁne

Original: [Equation]

Original: , where 8>>

Original: [Equation]

Original: [Equation]

Original: with µ i ∈ [ − n, n ] ∩ Z for i ∈ [ m ] ,

Original: [Equation]

Original: It is observed that ˜ f nκ belongs to the set of IFR functions, and

Original: [Equation]

Original: Next, we are going to prove that D r ˜ f

Original: nκ → D r f κ uniformly on K , as n → ∞ .

Original: For simplicity, we deﬁne the following function

Original: [Equation]

Original: Thus, Eq.

Original: (10) and Eq.

Original: (11) become

Original: [Equation]

Original: and

Original: [Equation]

Original: respectively, where ∪

Original: [Equation]

Original: Hence, one has sup

Original: [Equation]

Original: Let

Original: [Equation]

Original: Thus

Original: [Equation]

Original: we have

Original: [Equation]

Original: where the last inequality holds from Z

Original: Bd

Original: [Equation]

Original: d y =

Original: ( C x m + 1)( κ/n ) m + and |U|

Original: # = (2 n ) m +1 .

Original: Further, we can obtain sup

Original: [Equation]

Original: which tends to 0 as n → ∞ .

Original: Finally, we ﬁnish the proof by taking double limits n → ∞ before κ → ∞ .

Original: □

Original: B Full Proof for Theorem

Original: There are two proof methods for Theorem 2, one based on the rotation group action and one based on the Fourier transformation, which are detailed in this and the following sections, respectively.

Original: This proof idea can be summarized as follows.

Original: It is observed that the IFR function comprises the radius and phase since V is a symmetric matrix.

Original: Thus, there exist some linear connections (including rotation transformations) such that the combination of these spiking neurons is invariant to rotations.

Original: In other words, the IFR function led by scSNNs with one-hidden layer can easily and well approximate some radial functions, see Lemma 1, since the radial function is invariant to rotations, and is dependent on the input norm.

Original: B.1 Proof of Lemma

Original: Let f ′ :

Original: R m → R be a radial function with f ′ ( x ) =

Original: ∥ x ∥ .

Original: According to Theorem 1, for any δ > 0 , i ∈ [ n ] , and m ≥ 1 , there exists some time t such that

Original: [Equation]

Original: where f i denotes the IFR function of the i -th hidden spiking neuron, deﬁned by Eq.

Original: Further, we deﬁne a new function g ′ :

Original: R → R as follows.

Original: [Equation]

Original: where α ′ i , a ′ i ∈ R .

Original: For Lipschitz continuous function r √· and from [42, Lemma 1], we have

Original: [Equation]

Original: (13) where n ′ ≤ C ′

Original: L ( R k − r k ) /

Original: [-( k √ rδ ) for some constant C ′ > 
-]
Revised: [-( k √ rδ ) for some constant C ′ > 
-]

Original: and integer k ≥ 2 .

Original: Further, we have

Original: [Equation]

Original: ) where

Original: [Equation]

Original: in which { w ′ i } denotes another collection of weighted parameters that corresponds to f ( · , t )

Original: = P i ∈ [ n ] w i f i ( · , t ) .

Original: The ﬁrst term of Eq.

Original: (14) can be bounded by δ/ 4 from [42, Lemma 1] for any s ∈ [ r k , R k ] .

Original: The second term is at most δ/ 4 when n ≥ n ′ from Eq.

Original: This follows that

Original: [Equation]

Original: [Equation]

Original: Combining with Eqs.

Original: (13) and (15), we have

Original: [Equation]

Original: where x ∈ R m and s ∈ [ r k , R k ] .

Original: We ﬁnally obtain

Original: [Equation]

Original: provided n ≤ mn ′ and C ′ ≤ C s .

Original: Finally, we can complete the proof by setting k = 2 in the above upper bound.

Original: □

Original: B.2 Proof of Lemma

Original: Let r

Original: =

Original: C 2 √ m , R = 2 C 2 √ m , and m ≥ 1 , then we have r ≥ 1 , which satisﬁes the condition of [42, Lemma 1].

Original: Invoke Lemma 1 to construct the concerned spiking neural networks and deﬁne δ ′ ≤ δ/d .

Original: Then for any L -Lipschitz radial function g : R m → R supported on S ∆ , we have

Original: [Equation]

Original: where the width of the hidden layer is bounded by

Original: [Equation]

Original: This completes the proof.

Original: □

Original: B.3 Proof of Lemma

Original: Deﬁne a branch function

Original: [Equation]

Original: with

Original: [Equation]

Original: Let

Original: [Equation]

Original: B i = 1 , Ω i ’s are disjoint intervals, h i ( x ) is an N -Lipschitz function.

Original: Thus, h is also an N -Lipschitz function.

Original: So we have

Original: [Equation]

Original: where the last inequality holds from [10, Lemma 22].

Original: This completes the proof.

Original: □

Original: Based on the lemmas above, we can ﬁnish the proof of Theorem 2.

Original: Finishing the Proof of Theorem 2.

Original: Let g ( x ) =

Original: [-P Ni =
-]
Revised: [-P Ni =
-]

Original: ϵ i g

Original: i ( x ) be deﬁned by Eq.

Original: (5) and N ≥ 4 C 5 / 2 2 m 2 .

Original: According to Lemma 3, there exists a Lipschitz function h with range [ − 1 , +1] such that

Original: [Equation]

Original: Based on Lemmas 1 and 2, any Lipschitz radial function supported on S ∆ can be approximated by an IFR function f ( x , t ) led by scSNNs of one-hidden layer with width at most C 3 C s ( m ) 15 / 4 , where

Original: C 3 is a constant relative to C 2 and δ .

Original: This means that there exists some time t such that

Original: [Equation]

Original: Thus, we have

Original: [Equation]

Original: [-Hence, the range of f ( · , t ) is in [ − 1 − δ, +
-]
Revised: [-Hence, the range of f ( · , t ) is in [ − 1 − δ, +
-]

Original: + δ ] ⊆

Original: [ − 2 , +2] .

Original: Provided the radial function, deﬁned by Eq.

Original: (5), we have

Original: [Equation]

Original: This implies that given constants m > C 2 > 0 and C 3 > 0 , for any δ > 0 and ϵ i ∈ {− 1 , +1 } ( i ∈ [ N ]) , there exists some time t , such that the target radial function g can be approximated by an

Original: IFR function f ( x , t ) led by scSNNs of one-hidden layer with range in [ − 2 , +2] and width at most

Original: C 3 C s ( m ) 15 / 4 , that is,

Original: [Equation]

Original: This completes the proof.

Original: □

Original: C Another Proof for Theorem

Original: The another proof idea can be summarized as follows.

Original: Given any permutation operation on the input channels, there are some permutation operations on the self-connection and ﬁnal connection weights such that the output of SNNs are invariant.

Original: In other words, the IFR function led by scSNNs with one-hidden layer admits the rotation transformations.

Original: Therefore, it sufﬁces to show that before ﬁnal weighted aggregation, the component IFR functions of hidden spiking neurons can approximate any

Original: L -Lipschitz radial function, similar to the proof line of Lemma 1, and then ﬁnd a special radial function that can be well approximated by these IFR functions within the polynomial parameter complexity, similar to the proof line of Lemma 3.

Original: Lemma 5 Let f e and f ( x , t ) be an l -ﬁnite function and the IFR function led by scSNNs with onehidden layer, respectively.

Original: For any δ > 0 and A ∈ SO ( m, R ) , there exists some time t such that

Original: [Equation]

Original: Lemma 6 Let f e be an l -ﬁnite function, g : R → R is a scalar function, and f ( x , t ) is the IFR function led by scSNNs with one-hidden layer, where x ∈ S ( r ) and S ( r ) is a sphere supported with density ϕ 2 for 0 < r < ∞ .

Original: For any δ > 0 , if there exists some time t ∈ R such that it holds | f ( x , t ) − g ( ∥ x ∥ ) | < δ if and only if | f ( A x , t ) − g ( ∥ x ∥ )

Original: | < δ , for any A ∈ SO ( m, R ) .

Original: Lemma 7 Deﬁne a radial function g ′ :

Original: R → R with the form of

Original: [Equation]

Original: where N is a polynomial function of m ,

Original: ϵ =

Original: ( ϵ 1 , . .

Original: , ϵ N ) ∈ {− 1 , +1 } N , and Ω i ’s are disjoint intervals of width O (1 /N ) on values in the range Θ( √ m ) .

Original: For C 2 , C 3 > 0 with d > C 2 , any δ > 0 , and any choice of ϵ i ∈ {− 1 , +1 } ( i ∈

Original: [ N ]) , there exist some time t and A ∈ SO ( m, R ) such that

Original: [Equation]

Original: where f ( x , t ) indicates the IFR function led by scSNNs of one-hidden layer with range in [ − 2 , +2] and at most C 3 C s ( m ) 15 / 4 hidden spiking neurons.

Original: Finishing the Proof of Theorem 2.

Original: The rest proof is a straightforward combination of Lemmas 5,and 7.

Original: Deﬁne a radial function g ′ :

Original: R → R with the form of

Original: [Equation]

Original: where N is a polynomial function of m ,

Original: ϵ =

Original: ( ϵ 1 , . .

Original: , ϵ N ) ∈ {− 1 , +1 } N , and Ω i ’s are disjoint intervals of width O (1 /N ) on values in the range Θ( √ m ) .

Original: Obviously, g ′ is equivalent to another radial function deﬁned in Eq.

Original: (5) as follows

Original: [Equation]

Original: with

Original: [Equation]

Original: provided x ∈ S ( r ) .

Original: From Lemma 7, it is observed that provided the concerned radial function g ( x ) with a collection of choices ϵ , there exist some time t and n ∈

Original: O ( C 3 C s ( m ) 15 / 4 ) such that g ( x ) can be approximated a IFR function f

Original: ( x , t ) led by one-hidden-layer scSNNs of with range in [ − 2 , +2] and at most n hidden spiking neurons, such that

Original: [Equation]

Original: Let A ∈ SO ( m, R ) .

Original: If ∥ x ∥ ∈ Ω i ⊆ S ( r ) for 0 < r < ∞ , then it holds ∥ A x ∥ ∈ Ω i ⊆ S ( r ) and g ′ ( ∥ x ∥ )

Original: =

Original: g ′ ( ∥

Original: A x ∥ ) .

Original: According to Lemma 6, we have ∥

Original: [Equation]

Original: without any incremental change in the parameter complexity.

Original: This completes the proof.

Original: □

Original: D Full Proof for Theorem

Original: We begin this proof with an investigation that acting SVD on the matrix G .

Original: Let G =

Original: P Λ G Q ⊤ , where P ∈ R m × m and Q ∈ R n × n are two unitary matrices, Λ G ∈ R m × n is a diagonal matrix.

Original: Thus, we have

Original: [Equation]

Original: If the matrix G is degenerate, then Λ G contains some null diagonal elements.

Original: Then there must exists a non-degenerate sub-matrix G sub of G such that Inequality (6) degenerates to one about G sub in a lower dimensional space within at most constant time cost.

Original: Thus, we only care about the case that

Original: G is a non-degenerate matrix.

Original: When m ≤ n , the vector Λ G λ becomes a constant vector, denoted as ˜ λ , whose elements consist of λ 1 , . .

Original: λ m and n − m zeros.

Original: Notice that the n × n -dimensional matrix e V is symmetric and non-degenerate, since V is a symmetric and non-degenerate matrix and Q ∈ O ( n ) .

Original: Thus, we have e V =

Original: e U Λ

Original: U e U ⊤ , where

Original: e U ∈

Original: O

Original: ( n ) and Λ U

Original: = diag { ρ 1 , . . .

Original: , ρ n } in which ρ k ̸ = 0 for any k ∈ [ n ] .

Original: Then we have the following lemma.

Original: Lemma 8 (Lemma 4 as aforementioned) Given U = √ Λ U e U ⊤ , ˜ λ ∈ R n , and ϵ > 0 , when t ≥ Ω (cid:16) √ n ∥ Λ G ∥ϵ ∥

Original: U ∥(cid:17) , it holds ∥ U ˜ f

Original: (˜ λ , t ) − ˜ λ

Original: ∥ ≤ ϵ , where the modiﬁed IFR function ˜ f

Original: (˜ λ , t ) is led by a scSNN without connection matrix W and fed up to the constant spike sequence ˜ λ

Original: at every timestamp.

Original: This lemma is a straightforward derivation of [8, Theorem 1] due to the following two facts.

Original: (1) For any 0 < γ ≤ min { ρ k , k ∈ [ n ] } , the matrix U meets the regular conditions of [8, Deﬁnition 1].

Original: The optimal solution of U z =

Original: ˜ λ becomes z ∗ =

Original: U − 1 ˜ λ .

Original: Hence, we omit the detailed proof of this lemma.

Original: Finally, it sufﬁces to prove that the concerned IFR function f ( x , t ) can approximate ˜ f

Original: (˜ λ , t ∗ ) within time interval [0 , T ] where t ∗ ≤ T and T ≥ Ω( √ n ∥ G ∥ 2 ϵ √

Original: ∥ V ∥ 2 ) .

Original: Since f e is the linear function deﬁned by

Original: Eq.

Original: The approximation above can be converted into another one between

Original: [Equation]

Original: and

Original: [Equation]

Original: Since

Original: [Equation]

Original: (cid:19) (cid:18) X j ∈ [ n ]

Original: V ij E

Original: [ s j ( t )]

Original: +

Original: X k ∈ [ m ]

Original: W ik λ k

Original: ( t ) (cid:19)

Original: d s , we have ∥ E [ u ( T )] − ˜ u

Original: ( t ∗ )

Original: ∥ ≤ ϵ, (cid:13)(cid:13)(cid:13) e U E [ u ( T )] − ˜ u ( t ∗ ) (cid:13)(cid:13)(cid:13) ≤ ϵ, and ∥ U E [ u ( T )] − ˜ u

Original: ( t ∗ )

Original: ∥ ≤ ϵ .

Original: Thus, we have (cid:13)(cid:13)(cid:13) U E [ f ( x , T )] − ˜ f

Original: ( x , t ∗ ) (cid:13)(cid:13)(cid:13) ≤ ϵ .

Original: We ﬁnally complete the proof according to ∥ G ∥ 2 = ∥ Λ G ∥ 2 and p ∥ V ∥ 2 = ∥

Original: U

Original: ∥ 2 .

Original: □