Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
Authors: Wanjin Feng
Xingyu Gao
Wenqian Du
Hailong Shi
Peilin Zhao
Pengcheng Wu
Chunyan Miao
Venue: ICML 2025 poster (2025)
Keywords: Parallel Training
Spiking Neural Networks
Fixed-point Iteration
Primary area: deep_learning->algorithms
TL;DR: We propose Fixed-Point Parallel Training (FPT), a method that accelerates Spiking Neural Network training by reducing time complexity from \(O(T)\) to \(O(K)\) using parallel updates, without sacrificing accuracy or changing model architecture.

Abstract: Spiking Neural Networks (SNNs) often suffer from high time complexity $O(T)$ due to the sequential processing of $T$ spikes, making training computationally expensive. 
In this paper, we propose a novel Fixed-point Parallel Training (FPT) method to accelerate SNN training without modifying the network architecture or introducing additional assumptions.
FPT reduces the time complexity to $O(K)$, where $K$ is a small constant (usually $K=3$), by using a fixed-point iteration form of Leaky Integrate-and-Fire (LIF) neurons for all $T$ timesteps.
We provide a theoretical convergence analysis of FPT and demonstrate that existing parallel spiking neurons can be viewed as special cases of our approach. 
Experimental results show that FPT effectively simulates the dynamics of original LIF neurons, significantly reducing computational time without sacrificing accuracy. 
This makes FPT a scalable and efficient solution for real-world applications, particularly for long-duration simulations.

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
Wanjin Feng 12
Xingyu Gao* 12 Wenqian Du 12 Hailong Shi!
Peilin Zhao? 3 Pengcheng Wu+ Chunyan Miao 4
Abstract
spatiotemporal data effectively (Ma et al., 2023; Singh et al., 2022; Cao et al., 2015). Consequently, SNNs have emerged as a promising bridge between neuroscience and computa- tional science, gaining significant research interest in recent years (Zhang et al., 2023b; Yin et al., 2023).
Spiking Neural Networks (SNNs) often suffer from high time complexity ((T) due to the se- quential processing of T spikes, making training computationally expensive. In this paper, we pro- pose novel Fixed-point Parallel Training (FPT) method to accelerate SNN training without modi- fying the network architecture or introducing ad- ditional assumptions. FPT reduces the time com- plexity to O(K), where K is small constant (usually = 3), by using a fixed-point iteration form of Leaky Integrate-and-Fing (L.IF) neurons for all T timesteps. We provide a theoretical con- vergence analysis of FPT and demonstrate that existing parallel spiking neurons can be viewed as special cases of our proposed method. Experimen- tal results show that FPT effectively simulates the dynamics of original LIF neurons, significantly reducing computational time without sacrificing accuracy. This makes FPT a scalable and efficient solution for real-world applications, particularly for long-term tasks. Our code will be released at https:/ htt//ggtttuuu wwm/Wan wenniiiooonFPPPn
Despite their potential, SNNs often require a large number of timesteps achieve optimal performance (Ding et al., 2024). For instance, neuromorphic benchmark datasets such as HAR-DVS, DVS-CIFARIO, 10, and DVS- Westure typ- ically need 10 or more timesteps to reach satisfactory ac- curacy (Wang et al., 2024a; Zhuge et al., 2024; Jiang et al., 2024). While longer timesteps enable the network to cap- ture richer temporal information and improve accuracy, they also introduce significant computational overhead (Zhang et al., 2023a). The sequential processing of spikes across T timesteps increases simulation time, slowing down both training and inference, resulting in a time complexity of ((T) (Wang et al., 2023b; Kim et al., 2023b). Moreover, traditional SNN training using PaopPraagtttion Through Time (BPTT) struggles to fully utilize the parallel process- ing capabilities of modern hardware, such as GPUs, ex- acerbating computational inefficiencies (Stan & Rhodes, 2024; Hu et al., 2024). This limitation becomes especially pronounced in long-term tasks (Yao et al., 2023).
1. Introduction
To address these challenges, we introduce the Fixed- Fixed-point Parallel Training (FPT) method, which leverages the parallel processing capabilities of modern hardware to significantly accelerate SNN training. By employing fixed-point iter- ation framework. FPT decouples sequential dependencies, enabling simultaneous computation across all T timesteps. This reduces the time complexity from O(T) to O(K), where is the number of fixed-point iterations and typ- ically a small constant. Importantly, FPT preserves essential neural dynamics, including the reset mechanism, ensuring both accuracy and biological interpretability during training. We provide a thcoretical analysis to prove the convergence of FPT and demonstrate that existing parallel spiking neuron models can be interpreted as specific instances of our frame- work. Experimentally, FPT achieves better performance while significantly reducing computational time, making it scalable and efficient solution for real-world applications.
SNNs represent a biologically inspired evolution of artifi- cial neural networks (ANNs) (Zhu et al., 2024; Zheng al., 2024). Unlike traditional ANNs that rely continuous- value propagation, SNNs utilize discrete spikes, mimicking the way the brain processes information (Wang et al., 2024b). This unique spike-based computation offers several advan- tages, including improved energy efficiency through sparse activation, robustness to noise, and the ability to process
IIntiitte of Microelectronics, Chinese Academy of Sci- ences. Beijing. China University of Chinese Academy of Sci- ences. Beijing, China 'School of Artificial Intelligence, Shang- hai Jiao Tong University, Shanghai, China "Nanyang Techno- logical University, Singapore. Correspondence to: Xingyu Gao <gxy9910@ @exy9910@gmail.com gmail.com>.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
The main contributions of this paper follows:

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
We propose a novel Fixed-point Parallel Training (FPT) method that reduces the training time complexity of SNNs from O(T)) to O(K), enabling efficient parallel processing while preserving all neural dynamics.
reset mechanism, leading faster simulation speeds (Fang et al., 2023). Furthermore, PSU and its derivatives, IPSU and RPSU, facilitate parallel computation by decoupling integration and firing mechanisms (Li et al., 2024).
Despite their efficiency, these methods often rely on as- sumptions, such as the presence of a refractory period or the removal of the reset mechanism, which fail to fully capture the dynamic behavior of LIF neurons. Moreover, models like PSN and PSU introduce additional O(T2) complex- ity in learning parameters to maintain accuracy, making it difficult to adapt them for sequential models that handle variable-length sequences.
We provide a theoretical analysis proving the conver- gence of FPT and demonstrate that existing parallel spiking neuron models can be derived as special cases of our method.
Experimental results demonstrate that FPT retains the dynamic properties of original LIF neurons, and sig- nificantly reduces computational time, addressing the bottlenecks of long-term SNN training.
2.3. Fixed-point in Neural Networks
2. Related Work
Fixed-point is crucial in neural network models, particularly in implicit layers and recurrent networks, where it occurs when outputs stabilize after multiple iterations (Bai et al., 2019; El Ghaoui et al., 2021). Several studies have inte- grated fixed-point conditions into RNNs enhance training efficiency and network stability (Wang & Ragni, 2021; Zhu & Rosenbaum, 2024; Lim et al., 2024). In SNNs, feed- back connections cause the average firing rates to evolve toward an equilibrium state. Implicit differentiation of this equilibrium equation allows gradient computation without explicitly tracking the forward process (Xiao et al., 2021; 2023). However, this method requires a sufficient number of timesteps for the model to reach equilibrium, and the model's 's expressive power is influenced by the depth of the weight-tied block. Recent work, such as Cao et al. (2025), introduces MPIS-SNNs, which also can be viewed as a weight- tied block that leverages fixed-point theory.
2.1. Training Acceleration for SNNs
Various approaches have been proposed to reduce the com- putational complexity in SNNs. One common strategy is to reduce the number of timesteps. This can be achieved by dynamically adjusting the number of timesteps based input samples using confidence score thresholds (Li et al., 2023a;b), dividing SNNs into multiple stages with progres- sively shrinking timesteps (Ding et al., 2024), or introducing stochastic latency training (Anumasa et al., 2024). Another approach is online training, where the gradients of backprop- agation are truncated or approximated along the temporal axis (Xiao et al., 2022; Meng et al., 2022; Jiang et al., 2024). A more direct method involves reducing surrogate gradi- ents from multi-timestep to single timestep (Suetake et al., 2023). Additionally, SSF stabilizes input and output ac- tivations by averaging them over time, reducing gradient computation along the time dimension (Wang et al., 2023a). T-RevSNN accelerates training by deactivating the temporal dynamics of most spiking neurons and introducing multi- level reversible interactions (Hu al., 2024).
Despite these advancements, fxed-point iterative training methods in SNNs still heavily rely on specific network archi- tectures. In particular, the possibility of reformulating the dynamics of LIF neurons into fixed-point iteration frame- work- allowing parallel training across timesteps- has not been explored. As a result, achieving an architecture- independent fixed-point parallel training method for SNNs remains an open problem.
However, these methods often compromise accuracy (e.g., timestep compression, online training), rely on specific net- work modifications (e.g., T-RevSNN) or assumptions about stabilized spiking flow ge.g., SSF), and therefore lack gener- ality, limiting their scalability diverse tasks.
3. Motivation
2.2. Parallel Spiking Neurons
Spiking neurons are the fundamental components of SNNs. Among these, Leaky integrate-and-Fing (LIF) neurons are widely used due to their simplicity in simulating the neu- ronal behavior. The dynamics of an LIF neuron are de- scribed by:
Other approaches focus on developing spiking neuron mod- els that enable parallel computation. For instance, by lever- aging the absolute refractory period (ARP) of neurons, the adaptive LIF model achieves constant sequential complex- ity over the ARP simulation length (Taylor et al., 2023). By separating the linear integration component from the non-linear spiking function, SPSN allows parallel compu- tation across timesteps (Yarga & Wood, 2023). The PSN model further enhances parallelizability by eliminating the
U1 = = - Vih + Cc.
(I)
where u, is the membrane potential at timestep f, A < 1 is the decay factor, Vnn is the threshold potential, s, is the binary spike output, and o is the synaptic current. When the
2

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
1.4
Heerasle Peocikon
Sganed (1-1)
Sigeedd (rr))
$1
(22)
Narodd (and)
Forward Pass
Aggodd (wr)))
Backward Pass
My
(1)
(2)
(4KK)
Surrogate Gradient
0.2
Compact Representation of Vectors
-10
10
Timesteps
Iterations
(a) Heaviside vs. Sigmoid
(b) BPTT Forward and Backward
(c) FPT Forward and Backward
Figure 1. Comparison of the forward and backward procedures of BPTT and FPT. In BPTT, the neuron processes sequentially over T timesteps, computing step-by-step through time. In contrast, FPT processes timesteps simultaneously K iterations, where K < Circles represent scalars, and rounded rectangles represent vectors.
membrane potential exceeds Via, the neuron fires spike:
4. FPT: Fixed-point Parallel Training
(u: Vin),
(2)
In this section, we introduce Fixed-point Parallel Training (FPT) method, novel approach aimed improving the effi- ciency of SNN training by leveraging fixed-point iterations and parallel processing.
where H(.) is the Heaviside step function. LIF neurons process inputs sequentially over discrete timesteps, as shown Figure ((b).
To train SNNs, BPTT used backpropagate gradients through the inverse process of Eq. (1) and (2). The gradients for T timesteps are computed as:
4.1. Fixed-point Mapping
To accelerate the training process of SNNs, straightfor- ward approach to unroll the states across all timesteps into vectors matrices for parallel processing, thereby re- ducing the time complexity. Specifically, assume the initial membrane potential up 0, the recursive updates for the membrane potential over T timesteps can be approximated as follows:
Oe
T
Ol 0st, Our
Ow
Ust Ou, Ow
Jus+1
0u,+1 ds; Our
Du:
Ds:
Ou:" Ow
T<t
where represents the network weights. However, Sas Bu:
is non-differentiable due to the discontinuity of H-).. To address this, the surrogate gradient method is applied, using functions like the sigmoid:
u1 Ci U2 (ni 81 Vih + C2
Ut (n2 82Ven) +
(6)
(u) + e-oon
(4)
WT X (UT-1 ST + cr
where a controls the steepness of the approximation. As -> oo, (.) approaches H.)), as shown in Figure ((a).
The gradient of (-) can be approximated as:
Introducing vector and matrix notation, we define:
Ou ~ (s) (S)(()(1 - (.))),, ((,())
0s
(5)
Ou
f1
$1
C1
which smooths the optimization landscape, facilitating effec- tive weight updates while preserving the spiking behavior of the neuron.
112
82
C2
Consequently, training and inference in SNNs scale linearly with the number of timesteps 7', resulting in a time com- plexity of OTT). This sequential nature poses challenges for real-world applications requiring long temporal depen- dencies or rapid decision-making. maainng.
WT
ST
cr
where u represents the membrane potentials at different timesteps, s is the vector of spike outputs, and c denotes the synaptic currents over time. To capture the influence of past
3

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
inputs, we introduce the decay matrix A:
Algorithm Forward Pass of FPT
Require: Input current c, threshold potential Via. decay
o
0
0
factor X, timesteps T
A
...
0
Ensure: Membrane potentials u. spike outputs s.
A
(8)
1: Initialize Ug S(o)
AT-1 27-2
X0
2: Compute A based on Eq. (8).
3: for k = 1 to K do
4: Compute (()) using:
Here, A is lower triangular matrix, where each element in the i-th and j-th column is X for i 2 and O otherwise. This matrix models the exponential decay of the past inputs on the current membrane potential, with À being the decay factor.
u(a) = (A - ()s(k=1) + Ac
5: Compute S(k) using:
(u)) Say (u(k) - V(h)
Using these notations, the membrane potential across timesteps for parallel iteration of LIF neurons can be com- pactly represented
6: end for
7: Set equilibrium membrane potential: u, U(k)
8: Compute spike outputs s, based on Eq. (14) or (15)
(A 1)s Ac,
(9)
H(u Vnn)
propagation process described as:
The corresponding fixed-point mapping is:
U(1) Ac; (22) Van -I)s(1) + Ac.
(u) = VvA (A - () Va.) Ac
(10)
(12)
However, this cannot be directly used in optimization algo- rithms due to the discontinuity introduced by the Heaviside function (-).
(KK Vvn(A I)s(K-1) + Ac,
S(k) = S (u(k) Vo.), 1 1s
(13)
Here, S(k) represents the intermediate spike outputs during the iterative process. As shown in Figure 1(c), this process preserves the LIF neuron dynamics, particularly the reset mechanism, which is absent in models like PSN (Fang et al., 2023).
4.2. Surrogate Fixed-point Learning
To ensure smooth convergence of the fixed-point iterative equation, we replace the Heaviside function H-)) with a surrogate function, such the sigmoid approximation:
Upon convergence, the equilibrium membrane potential u, is computed, and the final spike outputs s, can be deter- mined as follows:
(u)) (u) - (A, I)S. (u Vo.) + Ac (11)
As a > 0o, (.) converges to (().. However, for large values of o, the function becomes too steep, which may hin- der training due to sharp transitions. Drawing from the con- cept of surrogate gradients, we use different approximation factors a for the forward and backward passes. Specifically, of for the forward pass and og for the backward pass, ensuring as S a.. This ensures that the forward pass provides a sufficiently approximation to (F.), while main- taining a stable gradient in the backward pass, facilitating more efficient learning.
S. = H(, - V.x)-
(14)
Alternatively, probabilistic firing mechanism that samples once can also be used, as proposed in Ma et al. (2023):
8, ~ Bernoulli(Sa, (u. - Vix)).
(15)
The forward propagation process is summarized in Algo- rithm It worth noting that this forward algorithm can be further improved several ways, such as early termination of iterations upon convergence or adapting ay dynamically during the iterative process. Furthermore, ay can also be learnable. Detailed implementations and discussions provided in the Appendix.
4.3. Parallel Training
1. FORWARD PASS
During the forward pass, assuming convergence after K iterations, the membrane potential vector ц(к) converges to an equilibrium point, denoted as u., where further iter- ations produce negligible changes. The iterative forward
4.3.2. BACKWARD PASS
Due to the favorable convergence properties of LIF neu- rons, we set for most experiments, resulting in a

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
minimal number of iterations. Unlike traditional deep equi- librium models (Bai et al., 2019; Cao et al., 2025), which rely on iterative methods to compute the inverse Jacobian at equilibrium, our approach directly employs automatic differentiation with surrogate gradients for backpropagation. This is feasible due to the small value of K, simplifying the implementation.
Algorithm 2 Backward Pass of FPT
Require: Input current c, threshold potential Via. decay factor X, iterations K
Ensure: Compute the gradient Us.
0S0, vv.)
Compute the gradient
du.
du(x)
2: Initialize the gradient
0uga DW
A
de 7W
3: for k = 2 to K do
Specifically, the loss gradient with respect network weights is:
4: Compute the gradient for the previous potential:
(s(k-1)
0S (u Vea)
ae de Üs. du. de Üs. (u(K) (16)
(u(k-1)
du(k-1)
dw Üs, du, dw ds. Ou. Ow
Here, whether the spike outputs s, computed based 5: Update the gradient for the current iteration: Eq. (14) or (15), their surrogate gradients are:
du(k) +- - Vea VA - 0s(k--) 1) Ōu(k-1)
+ A dc
Ow
(u((--))
Ow
Ow
Üs. iu.
OS., (u( Vin)
(17)
6: end for
Zu(K)
7: Compute the final gradient for the network weights:
duck it can be computed iteratively over K steps For 8w follows:
òs.
Øs. Ou(K)
+ du. Ow
Ow
@ug1)
A dc
Ow
Ow
@u(2) VAn(A -) Tu( Ow
@5(1) @u(1)
dc
5.1. Analysis of Convergence Condition and Speed
A Ow
Ow
Lemma 5.1. Assume the surrogate function Sa(.) is Lip- schitz continuous with a constant La- If the condition < 1, satisfied, then the mapping 4. (u(u) Vvn (A I)8. Vea) Ac contraction mapping under the I I-norm. Consequently, the iterative scheme
Ve X(1-X-11 1, where 1-1
@u(k)
0s (S(K-1) (u(K11) Ōu(K-1) dw
+ A Ōc
(A I)
Ow
Ow
(18)
The surrogate gradient method is applied as follows:
((k) Vva(A I)Sa(u(--) Von) Ac
(20)
Os(k)
aS.o (u()) Ven)
(19)
Tu(k)
@u(k)
converges unique fixed point u..
The detailed backward propagation process is summarized in Algorithm The time complexity for both the forward and backward passes of the FPT algorithm is O(K), com- pared to the traditional O(T) (K < T). This decoupling from 7 makes FPT particularly efficient for long-term tasks, enabling extended temporal processing at reduced computa- tional costs. Moreover, automatic differentiation eliminates the need to explicitly construct and invert Jacobian matrices, simplifying implementation and making the approach more practical.
illustrate the convergence of the algorithm, consider the typical values VeA 1, À 0.25, and the sigmoid function (-). which is Lipschitz continuous with constant L., q Substituting these values into the convergence condition:
VnnLa X11-A--3 a(1 0.25
(21)
1-
12
12
We see that for any & < 12, this expression is always less than 1, thereby satisfying the convergence condition. Moreover, the convergence proof considers the worst-case scenario. In practice, the actual convergence condition is more relaxed due to the inherent sparsity of neural activity. Additionally, this convergence condition also reflects the rate of convergence: smaller a leads to faster convergence.
5. Theoretical Analysis
In this section, we focus on three key research questions: whether convergence can be guaranteed and at what rate, how FPT relates to other parallel spiking neuron models, and what its computational complexity is.
Furthermore, the matrix - I is strictly lower triangular, with spectral radius of zero since all its eigenvalues are
5

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
Table Complexity Comparison of SNN Training Methods
Training Methods
Memory
Training Time
Inference Energy
Applicability
OTTT (Xiao et al., 2022)
O(L)
OLT)
O
Limited
SLTT-k (Meng et al., 2023)
O(L)
((Lk)
O(T)
Limited
T-RevSNN turnooff (Hu et al., 2024)
O(L)
O(L)
0(1)
Limited
T-RevSNN turn-on (Hu al., 2024)
((L)
O(T)
0(1)
Limited
BPTT (Zheng et al., 2021)
O(LT)
O(LT)
O(T) Unlimited
FPT (Ours)
OLT)) + (OLKT))
O(LK)
O(T)
Unlimited
zero. This property ensures rapid convergence, as the iter- ative process does not amplify errors through eigenvalues, thereby maintaining stability and guaranteeing efficient con- vergence. Importantly, Eq. (21) shows that the convergence rate (or condition) can be decoupled from the timesteps T by approximating the term 0.25"-1 as O. This indicates that the iteration is weakly influenced by T, ensuring rapid convergence even for longer timesteps.
where VaA - I). This formulation incorporates feedback mechanisms where the previous state influences the current potential. The term S. (Ac Vaa) represents estimate of the spike output, used to adjust the mem- brane potential. Therefore, PSN and PSU can be viewed as intermediate forms of FPT for learnable LIF neurons.
However, is important to note that convergence is not guar- anteed with just one or two iterations. In these cases, the fixed-point iteration may fail to reach an equilibrium point, potentially leading to inaccurate simulations of LIF dynam- ics and degraded performance. Additionally, PSN and PSU introduce an extra ( learnable parameter complexity, which limits their applicability to long sequences.
5.2. Comparison with Parallel Spiking Neuron
By modifying FPT slightly, we can adapt to learnable LIF neurons, with the update rule Eq. (9) incorporating learablee parameters as follows:
- VAn(A I)s + Ac
5.3. Analysis of Computational Complexity
(22)
8 = H (u - B)
Table summarizes the theoretical complexity of various SNN training approaches. Let L denote the number of net- work layers, & the truncated temporal length in SLTT-k, and the number of fixed-point iterations used FPT. Typ- ically, K is a small constant (e.g.. independent of T, making its contribution negligible for large T. Thus, space complexity OLLKT) and time com- plexity O(LK)) can be approximated as (LTT) and OLL), respectively. The coefficient À represents the fraction of memory introduced by the LIF component, which is the only part affected by FPT, while all other components of the network remain unchanged.
Here, A E RTXT is the learnable decay matrix, and B E R² is the learnable threshold vector. This model can be trained using a forward and backward propagation algorithm similar those in Eq. (12) and (18).
3) and remains
For single fixed-point iteration, the update for the mem- brane potential and spike output can be expressed as:
u Ac s ((uu-))
(23)
This formulation represents the standard Parallel Spiking Neuron (PSN) (Fang et al., 2023). Unlike the traditional LIF model, PSN removes the reset mechanism, which can affect the algorithm's 's accuracy and biological fidelity.
Notably, methods like OTTT, SLTT, and T-RevSNN acceler- atc training by truncating gradients discarding temporal dependencies. While effective, such strategies limit their ability to model fine fine-grained temporal dynamics. In contrast, FPT preserves complete neuronal dynamics and enables efficient training without altering the model architecture, thereby offering broader applicability across diverse SNN models.
By setting the threshold vector in Eq. (22) to a fixed value Vin and performing two iterations, the Parallel Spiking Unit (PSU) model can be derived (Li al., 2024):
= (A (Ac Von) Ac s H(u Vvn),
(24)
6. Experiments
This section answers three key questions: First, does the equilibrium point of FPT converge to the original LIF dy- namics? Second, can FPT improve training speed without
which further simplifies to:
s = H(Ac DS. Vih) V.n)
(25)
6

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
1-256,
T-256, K-3
10
Stea Abolgee Errerr of
Erroe
Absulute Error
1.5
Ernur State 0%
Mran Abalsic Errve
B.66-
Mean Absolate Ezror
GPU Speodap
Speedup
10
Spedig mm CPU
8,05
Speddap en
0.44
8:44
28
0.03
6.82
5
10'
16
64
256
1924
Number of Herations
Approsimation Foctor
Timesteps
(a) Error vs. Number of Iterations
(b) Error vs. Approximation Factor
(c) Error vs. Timesteps
Figure 2. Convergence and efficiency of the LIF neuron using proposed FPT method. The absolute eroo in membrane potential, spike firing error rate, and speedup factor relative to the original LIF are reported. Solid lines show the mean values across three experiments, with shaded areas representing the standard deviation.
compromising accuracy? Finally, it necessary to preserve the complete neural dynamics of LIF neurons?
100
Accuracy (%)
08.61
96.18
3.22
Time (6)
3.0
$1.57
6.1. LIF Dynamics Simulation
80
2.5
In this section, we validate whether the equilibrium point of the forward pass in FPT converges to the original LIF dynamics. We used LIF neuron with decay coefficient A = 0.25, threshold VaA 1, and input currents drawn from a Gaussian distribution with mean of 0 and a variance of 1. The experiments were conducted on an RTX 3060 laptop GPU and a 12th Gen Intel 17-12700H CPU.
Accuracy (%)
60
4.6x
Time
40-
8.13
1.0
0.70
2,3xx
0.70
2.4x
0.49
00.5
As shown in Figure 2(a), with increasing iterations. the equi- librium point of FPT quickly converges the original LIF dynamics, including both membrane potential and spike behavior. This demonstrates the high biological fidelity of FPT, retaining the key characteristics of the LIF neuron. For K 2 3, changes in membrane potential and spike activity become negligible, indicating equilibrium. Based on this observation, we chose K = for all subsequent experi- ments Figure 2(b) highlights the impact of the sigmoid approximation parameter on convergence. With fixed number of iterations, increasing reduces the discrepancy between the parallel and original LIF dynamics, approach- ing zero. This aligns with Figure (a), showing that larger a values improve approximation accuracy. Finally, Fig- ure 2(c) demonstrates the time efficiency of FPT on GPU and CPU platforms. As T increases, the GPU-based FPT achieves linear speedup, outperforming traditional methods. For T = 512 longer, FPT-based LIF models achieve over 100x speedup on GPUs, underscoring the effectiveness of FPT for long-term simulations.
LocalZO
FPT
LucalZO
FPT
LocalZO
FPT
DVS-Cifar10
DVS-Gesture
ImageNet-100
Figure 3. Comparison of accuracy and speed between FPT and the BPTT-like algorithm (LocalZO). Time indicates the average training time per batch on a single RTX 3090 GPU, with both models using the same architecture and hyperparameters.
the static ImageNet- 100 dataset, as shown in Table 2. De- tailed experimental settings are provided in the Appendix. On the DVS-CIFAR10 10 dataset, FPT achieved improve- ment of over 3% in accuracy compared the baselines. In contrast, methods such as online training and timestep shrinkage, which rely on gradient truncation or reducing timesteps, yielded inferior accuracy. On the DVS-Gesture dataset, our method achieved an accuracy comparable to LocalZO. For the static ImageNet- 100 dataset, FPT out- performed LocalZO. This improvement in performance attributed to the enhanced generalization capability provided by our surrogate fixed-point learning framework.
6.2. General Classification Performance of FPT
To evaluate the effectiveness of FPT compared to differ- ent training methods, we tested it on three datasets: the dynamic DVS- CIS-CIOAROO and DVS- DVS-Gesture datasets, and
Figure compares FPT with LocalZO in terms of both accuracy and speed across the datasets. Since FPT reduces time complexity from (T) to O(K). with K = 3, and it also avoids the multiple sampling required by LocalZO. As
7

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
Table Comparison of FPT with previous SOTA training method. The average accuracy and standard deviation across three runs are
reported, with the highest accuracy from the three runs shown in parentheses.
Dataset
Model
Method
Architecture
Training Timestep
Accuracy (%)
OTTT (Xiao et al., 2022)
Online training
VGG-II
10
76.63
NDOT (Jiang al., 2024)
Online training
VGG-11
10
77.40
SI.TT (Meng al., 2023)
Timestep Shrinkage
VGG-I1
10
77.17
SHENN (Li et al., 2023b)
Timestep Shrinkage
VGG-16
10
82.70
DVS-CIFAR
SSNN (Ding et al., 2024)
Timestep Shrinkage
VGG-9
8
78.57
SLT (Anumasa et ul., 2024)
Timestop Shrinkage
VGG-11
10
81,46
SSF (Wang et al., 2023a)
Stabilized Spiking Flow
VGG-11
20
78.00
T-RevSNN (Ilu al., 2024)
Temporal Reversible
R8
16
79.20
LocalZO (Mukhoty ct al., 2023)
Zeroth Order
VGG-11
10
81.87
FPT (Ours)
Parallel Training
VGG-11
10
3)
85.50.10.22 (85.70)
SLTT (Meng et al., 2023)
Timestep Shrinkage
VGG-I1
20
97.92
SSNN (Ding et al., 2024)
Timestep Shrinkage
VGG-9
8
94.91
DVS-Gesture
T-RevSNN (Hu al., 2024)
Temporal Reversible
ResNet-18
16
97.90
LocalZO (Mukhoty et al., 2023)
Zeroth Order
VGG-11
10
98.43
FPT (Ours)
Parallel Training
VGG-11
20 (K = 3)
98.38=0.17 (98.61)
Efticient Net (Kim et al., 2023a)
Normal BPPT
ResNet-19
5
79.44
ImageNet- 100
Local70 (Mukhoty et al., 2023)
7eroth Order
SFW-Resnet34
4
81.56
FPT (Ours)
Parallel Training
SFW-Resnet34
4 (K- 3)
83.27++0.16 (83.48)
a result, our algorithm achieved more than twice the speed of LocalZO while maintaining comparable or even superior performance.
tently greater than 0.05. Furthermore, regardless of whether BPTT or FPT is used, the firing rate remains around 12% across different timesteps. Notably, FPT exhibits a signifi- cant efficiency advantage. For T 2 32, FPT is over 10 times faster than BPTT. This demonstrates FPT's ability dra- matically accelerate training while maintaining performance comparable to that of BPTT.
6.3. Efficiency Comparison with BPTT
Table Comparison of FPT and BPTT with different timesteps T. The p-values from I-tests show that accuracy differences are statistically significant. FPT accuracy is reported across three random trials. Time indicates the average training time per batch on a single RTX 3090 GPU.
6.4. Ablation Study
As introduced in Section 5.2, PSN and IPSU can be viewed as intermediate results of our FPT with different values of K. At the same time, they diminish the impact of the reset mechanism, with PSN removing it entirely. To evaluate the impact of the number of iterations K and the reset mecha- nism, we conducted an ablation study comparing various configurations of FPT, PSN, and IPSU the sequential CIFAR10 and CIFAR100 datasets. In these tasks, images are processed column by column over 32 timesteps, and the experimental settings follow those in (Fang et al., 2023).
Method
Timesteps
Firing rate (%)
Accuracy (%)
Time (ms)
P-value
8
12.24+00.01
90.95+00.17
6.49
BPTT
32
12.29=0.07
92. 02.15200.3
21.96
128
12.5120.10
92.66120.34
74.04
8
12.33++0.09
90.77+-0.38
1.55
0.512
FPT
32
12.4110.133
92.25:10.15
2.01
0.668
128
12.61:0.04
92.55::0.30
5.58
0.696
To further evaluate the efficiency of FPT across different timesteps T, we conducted experiments comparing it with the traditional BPTT on the Amazon Photos dataset using DRSGNN, which incorporates a layer of LIF neurons. The experiments were performed on a single RTX 3090 GPU, with batch size of 64. Except for the timesteps, all experi- mental settings followed those in (Zhao al., 2024).
As shown in Table 4, when the decay matrix A is fixed and non-learnable, PSN, exhibits approximately 2% lower ac- curacy compared to LIF across both datasets, which aligns with previous results reported for PSN (Fang et al., 2023). The key distinction here is that PSN removes the reset mech- anism, which is an essential feature in the traditional LIF model. Akin to the forget gate in LSTM networks, it reg- ulates the retention of historical information by discarding past states after each spike.
As shown in Table 3, both FPT and BPTT demonstrate improved accuracy as T increases, indicating that longer timesteps allow the model to capture more temporal dynam- ics and achieve better performance. However, the t-tests reveal that the differences in accuracy between FPT and BPTT are not statistically significant, with p-values consis-
In contrast, when the decay matrix A and the firing thresh- old B are learnable, FPT consistently outperforms both PSN
8

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
tions where performance, speed, and biological fidelity critical.
Table Comparison the sequential CIFAR datasets. The sub- script and symbol indicate that the matrix A is set to A and non-learnable. yx denotes that only the lower triangular part of the matrix A is learnable to avoid using future information. The
Acknowledgements
table reports the accuracy of FPT across three random trials. Other results taken from (Fang et al., 2023) and (Li al., 2024).
This work was supported in part by Science and Technol- ogy Innovation (STI) 2030-Major Projects under Grant 20222D0208700, and National Natural Science Foundation of China under Grant 62376264.
Method
Learnable
Accuracy (%)
LIF PSN, FPT IPSU
x
81.50
79.80
x
81.54:10.34
X
87.28
Impact Statement
Sequential CIFARIO
Masked PSN
VX
$5.81
FPT w/ Masked A
VX
87,48:20.14
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
PSN FPT w/ A
88.45
89.53:10.13
LIF
x
55.45
PSNo FPT IPSU
x
53.12
55.19:10.37
Sequential CIFAR100
Masked PSN
VX
VX
59.76
60.69
References
FPT w/ Masked A
VX
62.24 +-052
PSN
62.21
Anumasa, S., Mukhoty, B., Bojkovic, V., Masi, G. D.,
FPT w/ A
64.50:10.34
Xiong, H., and Gu, B. Enhancing Training of Spiking Neural Network with Stochastic Latency. In Wooldridge, J.J., Dy, J. G., and Natarajan, S. (eds.), Proceedings of the AAAI Conference on Artificial Intelligence, pp. 10900- 10908. AAAI Press, 2024. doi: 10. AAAL V38110. 28964.
and IPSU, achieving optimal results across both datasets. This suggests that FPT with K iterations outperform the PSN and IPSU models, which can be considered as having K 1 or K 2. Additionally, for FPT without learnable parameters, the accuracies on both datasets are nearly identical to those of the original LIF, indicating that K 3 is sufficient to ensure convergence. These find- ings highlight the advantages of our approach, underscoring the importance of both the reset mechanism and sufficient iterations for maintaining model accuracy.
Bai, S., Kolter, J. Z., and Koltun, V. Deep equilibrium mod- els. Advances in neural information processing systems, 32, 2019.
Cao, Y., Chen, Y., and Khosla, D. Spiking Deep Con- volutional Neural Networks for Energy- Object Recognition. International Journal of Computer Vision, 113(1):54 66. 2015. ISSN 0920-5691, 1573-1405. doi: 10.1007/s1 22101/1-112--01--8-8---1
7. Conclusion
Cao, 7.,, Li, M., Wang, X., Wang, H., Wang, F., Li, Y., and Huang, Z.-G. Efficient Training of Spiking Neural Networks with Multi-parallel Implicit Stream Architec- ture. In European Conference on Computer Vision, vol- ume 15092, pp. 422 422-438, Cham, 2025. Springer Nature Switzerland. ISBN 978-3-031- 2. doi: 10. 1011117977---0311-22
In this paper, we introduce the Fixed-point Parallel Training (FPT) method, which significantly improves the efficiency of training SNNs. By leveraging parallel fixed-point itera- tions of LIF neurons, FPT reduces the time complexity from O(T) to O(K), where K is small constant, leading to sub- stantial speedup. We demonstrate that FPT preserves the bi- ological dynamics of LIF neurons and effectively converges to them in practical applications. This equivalence allows for seamless conversion between sequential and parallel training modes, making FPT ideal for parallel training and subsequent deployment on neuromorphic devices, where inputs can be processed sequentially, Furthermore, FPT out- performs BPTT in computational efficiency, achieving up to 100x speedup GPUs for long-term tasks. Importantly, FPT requires no modifications to the network architecture, ensuring its broad applicability to a wide range of SNN models. This makes FPT a highly scalable and adaptable solution for efficient SNN training, especially in applica-
978-3-031-72754- 24.
Cheung, W.-S. Generalizations of Hölders inequality. In-
ternational Journal of Mathematics and Mathematical Sciences, 2001.
Ding, Y. Zuo, L., Jing, M., He, P., and Xiao, Y. Shrinking Your TimeStep: Towards Low-Latency Neuromorphic Object Recognition with Spiking Neural Networks. In
Wooldridge, M. J., Dy, J. G., and Natarajan, S. (eds.), Proceedings of the AAAI Conference on Artificial In- telligence, pp. 11811 1-11819. AAAI Press, 2024. doi: 10. 10.1609/AAAI 0381100.933000

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
EI Ghaoui, L., Gu, F., Travacca, B., Askari, A., and Tsai, A. Implicit Deep Learning. SIAM Journal on Mathematics of Data Science, 3(3):930-558, 2021. ISSN 2577-0187. doi: 10. 10.1137720M115851,
Meng, Q. Xiao, M., Yan, S., Wang, Y., Lin, Z., and Luo, Z.- Q. Training nigh-perffrmance low-latency spiking neural networks by differentiation Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12444-12453, 2022.
spike representation. In
Fang. W., Yu, Z., Zhou, Z., Chen, D., Chen, Y., Ma, Z., Masquelier, T., and Tian, Y. Parallel spiking neurons with high efficiency and ability learn long-term depen- dencies. In Advances in Neural Information Processing Systems, volume 36, 2023.
Meng, Q., Xiao, M., Yan, S., Wang, Y., Lin, Z., and Luo, Z.- Q. Towards memory-and time-efficient backpropagation for training spiking neural networks. In Proceedings of the IEEF/CVF International Conference Computer Vision, pp. 6166-6176, 2023.
Hu,J. J., Yao, M., Qiu, X., Chou, Y., Cai, Y., Qiao, N., Tian, Y., Xu, B., and Li, G. high-perffrmmnce temporal reversible spiking neural networks with Ol)) training memory and (() inference cost. In International Conference on Machine Learning, pp. 19516-19530. PMLR, 2024.
Mukhoty, B., Bojkovic, V., de Vazelhes, W., Zhao, X., De Masi, G., Xiong, H., and Gu, B. Direct Training of SNN using Local Zeroth Order Method. In Advances in Neural Information Processing Systems, volume 36, 2023.
Jiang, H., Masi, G. D., Xiong, H., and Gu, B. NDOT: Neuronal Dynamics-based Online Training for Spiking Neural Networks. In International Conference on Ma- chine Learning. pp. 21806-21823. PMLR, 2024.
Shukla, S., Balasubramanian, S., and Pavlović, M. A Generalized Banach Fixed Point Theorem. Bulletin of the Malaysian Mathematical Sciences Society, 39(4): 1529-1539, 2016. ISSN 0126-6705, 2180-4206. doi: 10.1007//40840-010 0100000-1---0 015-0255-5.
Kim, Y., Li, Y., Moitra, A., Yin, R., and Panda, P. Shar- ing teaky-integrate-and-fire neurons for memory-efficient spiking neural networks. Frontiers in Neuroxcience, 17: 1230002, 2023a.
Singh, A. K., Saraswat, V., Baghini, M. S., and Ganguly, U. Quantum tunneling based ultra-compact and energy efficient spiking neuron enables hardware SNN. IEEE Transactions on Circuits and Systems I: Regular Papers, 69(8):3212-322 2022.
Kim, Y., Li, Y., Park, H., Venkatesha, Y., Hambitzer, A., and Panda, Exploring temporal information dynam- ies in spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 8308 8316, 2023b.
Stan, M.-I. and Rhodes, O. Learning long sequences in spiking neural networks. Scientific Reports, 14(1):21957, 2024.
Li, C., Jones, E. G., and Furber, S. Unleashing the Po- tential of Spiking Neural Networks with Dynamic Con- fidence. In IEEE/CVF International Conference on Computer Vision, pp. 13304-13314. IEEE, 2023a. doi: 10.1 LL1009ICCV5100702222...2222
Suetake, K., Ikegawa, S.-i., Saiin, R., and Sawada, Y. S3NN: Time step reduction of spiking surrogate gradients for training energy efficient single-step spiking neural net- works. Neural Networks, 159:208-211 2023. doi: 10. 2..10166..eeunet2222222...0
Li, Y., Geller, T., Kim, Y., and Panda, P. SEENN: To- wards Temporal Spiking Early Exit Neural Networks. In Advances in Neural Information Processing Systems, volume 36. 2023b.
Taylor, L., King, A., and Harper, N. S. Addressing the speed-accuracy simulation trade-off for adaptive spiking neurons. In Advances in Neural Information Processing Systems, volume 36, 2023.
Li, Y., Sun, Y., He, X., Dong. Y., Zhao, D., and Zeng, Y. Parallel Spiking Unit for Efficient Training of Spiking Neural Networks, 2024.
Wang, J., Song, Z., Wang, Y., Xiao, J., Yang, Y., Mei, S., and Zhang, Z. SSF: Accelerating Training of Spiking Neural Networks with Stabilized Spiking Flow. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5982-5991, 2023a.
Lim, Y. H., Zhu, Q.. Selfridge, J., and Kasim, M. F. Paral- lelizing non-linear sequential models over the sequence length. In The Twelfth International Conference on Learn- ing Representations, 2024.
Wang. X., Wu, Z., Jiang, B., Bao, Z., Zhu, L., Li, G., Wang, Y., and Tian, Y. HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensors. In Proceedings of the AAAI Conference on Artificial In- telligence, volume 38, pp. 5615-5623, 2024a. doi: 10. 1609/aaai, 3816.28372
Ma, G., Yan, R., and Tang, H. Exploiting noise as a resource for computation and learning in spiking neural networks. Patterns, 4(10):100831, 2023. ISSN 26663899. doi: 10. 100831.
10

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
Wang, Z. and Ragni, A. Approximate Fixed-Points in Re- current Neural Networks, 2021.
J. G., and Natarajan, S. (eds.), Proceedings of the AAAI Conference Artificial Intelligence, pp- 16970-16978. AAAI Press, 2024, doi: 11609//AAAAVV13811222226666
Wang, Z., Zhang, Y. Lian, s., Cui, X., Yan, R., and Tang, H. Toward high-aceuracy and low-latency spiking neural net- works with two-stage optimization. IEEE Transactions Neural Networks and Learning Systems, 2023b.
Zheng, H., Wu, Y., Deng, L.,, Hu, and Li, Going deeper with directly-trained larger spiking neural net- works. In Proceedings of the AAAI Conference on Artifi- cial Intelligence, volume 35, pp. 11062-11070, 2021.
Wang, Z., Tao, P., and Chen, L. Brain-inspired Chaotic Spiking Backpropagation. National Science Review, pp- nwae037, 2024b. doi: 10. 1...0933/s//wwa0037.
Zheng, H., Zheng, Z.,, Hu, R., Xiao, B., Wu, Y., Yu, F., Liu, X., Li, G., and Deng. L. Temporal dendritic heterogeneity incorporated with spiking neural networks for learning multi-timescall dynamics. Nature Communications, 15 (1):277, 2024. doi:
Xiao, M., Meng, Q.. Zhang, Z., Wang. Y., and Lin, Z. Train- ing feedback spiking neural networks by implicit differ- entiation on the equilibrium state. In Advances in Neurul Information Processing Systems, volume 34, pp. 14516- 14528, 2021.
Zhu, V. and Rosenbaum, Learning fixed points of re- current neural networks by reparameterizing the network model. Neural Computation, 15688)8568-888
2024.
Xiao, M., Meng, Q., Zhang, Z., He, D., and Lin, Z. On- line training through time for spiking neural networks. In Advances in Neural Information Processing Systems, volume 35, pp. 20717-20730, 2022.
Zhu, Y., Ding, J., Huang, T., Xie, X., and Yu, Z. Online stabilization of spiking neural networks. International Conference on Learning Representations, 2024.
Xiao, M., Meng, Q. Zhang. Z.,, Wang, Y., and Lin, 7. SPIDE: A purely spike- method for training feed- back spiking neural networks. Neural Networks, 161: 9-24, 2023.
Zhuge, Z., Wang, P., Yao, X., and Cheng, Towards Efficient Spiking Transformer: A Token Sparsification Framework for Training and Inference Acceleration. In International Conference on Machine Learning, pp. PMLR, 2024.
Yao, M., Hu, J., Zhao, G., Wang, Y., Zhang, Z., Xu, B., and Li, G. Inherent Redundancy in Spiking Neural Networks. IEEE/CVF International Conference on Computer Vision. pp. 16878-16888. IEEE, 2023. doi: 10.1 14110//CCV511070023....555
Yarga, S. Y. A. and Wood, S. U. Accelerating snn train- ing with stochastic parallelizable spiking neurons. 2023 International Joint Conference on Neural Networks (IJCNN), pp. 1-8. IEEE, 2023.
Yin, B., Corradi, F., and Bohté, S. M. Accurate online training of dynamical spiking neural networks through Forward Propagation Through Time. Nature Machine Intelligence, pp. 1-10, 2023.
Zhang, A., Shi, J., Wu, J., Zhou, Y., and Yu, W. Low Latency and Sparse Computing Spiking Neural Networks With Self-Driven Adaptive Threshold Plasticity. IEEE Transactions on Neural Networks and Learning Systems, 2023a.
Zhang, T., Cheng, X., Jia, S., Li, C. T., Poo, M.-m., and Xu, B. A brain-inspired algorithm that mitigates catas- trophic forgetting of artificial and spiking neural networks with low computational cost. Science Advances, 9(34): cadi2947,
Zhao, H., Yang, X., Deng, C., and Yan, J. Dynamic Reactive Spiking Graph Neural Network, In Wooldridge, M. J., Dy,

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
A. Convergence Proof for FPT
Lemma A.I. Assume the substitution function S.() is Lipschitz continuous with a constant Lae If the condition
VinLa - where X is satisfied, then the mapping 4. (u) (va(A I) Vin) + Ac 1-X is contraction mapping under the I-norm. Consequently, the iterative scheme
U(ke = - Ven (A () Voa) + Ac
(26)
converges to a unique fixed point u,
Proof. Let in U2 E IR Considering the 1-norm, we have:
(ui) - (u2)ll1
= =|| (A - (1; - Vin) + Ac (-Vi ()2 - Van) + Ac)Il1
= VA - () Vin) - Sa(112 Von))l11
Since (u) is Lipschitz continuous with constant Lis, we have:
JS - Vin) - (up Vin)ll S Lallur u2ll1-
(27)
Using Hölder's inequality (Cheung, 2001), get: (ur) - @. (u2)lla S Vaal(A - 1) Lallu; - Uzll1 L|lu: - u2 -u2l1,
(28)
where L VinLa ((A l)
We use the maximum norm (infinity norm) for A - I:
T-1
A(1 XT-1
A I 00 = Σ X
(29)
1-X
According the Banach fixed-point theorem, if the condition 7. Vv I.H X(1-17-1) contraction mapping (Shukla et al., 2016). Consequently, the iterative scheme
1 is satisfied, then a. (u) is
(()) = Vin (A ()Sa(u(--1) Voa) + Ac
(30)
converges a unique fixed point u.
B. FPT Variants
In some cases, the LIF neuron does not require K iterations according to Eq. (30) to converge. Therefore, we introduce an early stopping criterion based on the change in membrane potential, which can accelerate the training process, as shown in Algorithm 3. Specifically, the change membrane potential between iterations falls below a certain threshold e, the iteration can be terminated early.
Additionally, based on the convergence theorem of FPT, observe that smaller values of a lead to faster convergence, but with lower approximation accuracy. Conversely, larger values of a provide more accurate approximations the cost of slower convergence. A natural strategy is to start the iteration with smaller a for good initial approximation, and then gradually increase a for more accurate approximations, as shown Algorithm 4, where {ayı I a S ... S &ayk Alternatiell,, a could also be treated as a learnable parameter during training to dynamically adjust the balance between convergence speed and approximation accuracy.
12

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
Algorithm 3 Forward Pass of FPT with Early Stop for
Require: Input current c, threshold potential Vin, decay factor A, timesteps T,, tolerance e
Ensure: Membrane potentials u,, spike outputs s,
1: Initialize Ug S(o) 0
2: Compute A based on Eq. (8).
3: Set k k=
4: while & & K do
5: Compute U()) using:
(() = (A 1)) + Ac
6: Compute S(k) using:
(()) Sa, (u(x) Veh)
7:
if [lugk)) u(k-1) < then
8: Break (early stop)
9:
end if
10: Increment + + +1
end while
Set equilibrium membrane potential: u, U(A:)
Compute spike outputs based on Eq. (14) or (15)
Algorithm 4 Forward Pass of FPT with Adaptive af
Require: Input current c, threshold potential Vih, decay factor X, timesteps T, list of ay values {ay1 a/2. ..,, afk
Ensure: Membrane potentials u. spike outputs s.
1: Initialize u. = S(o) = 0
2: Compute A based on Eq. (8).
Set h
4: while à: < K do
5: Set a ask (select a from the predefined list)
6: Compute U(k) using:
((k) = -V - -I)S(k-1) + Ac
7: Compute S(x) using:
S(k) (u()) V(n)
8: Increment & < k +
9:
end while
10: Set equilibrium membrane potential: u, U(k)
11: Compute spike outputs s. based on Eq. (14) or (15)
C. Experimental Setup
For the FPT experiment on the Amazon Photos dataset using DRSGNN, we set and used Eq. (14) as the firing function.
Table 5 provides the hyperparameter settings corresponding Table in the main text. The parameter a represents the incrementally increasing parameter used during each iteration. According Lemma 5.1, when a is relatively small, it facilitates convergence during the first iteration. As the number of iterations increases, a gradually approaches the step function used by the LIF neuron. In the backward pass, / which is one-third of the forward pass value, helps to obtain a smoother gradient and avoid vanishing gradients.
As described in the PSN paper, the CIFAR dataset's images are processed by the SNN one column at time, similar to how humans read from left to right. The corresponding hyperparameters are listed in Table 6.
13

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
Table Training parameters for FPT various datasets. Optimizer: Adam with betas: (0.9, 0.999), Rate Scheduler: cosine annealing.
DVS-CIFAR-10
DVS-Gesture
ImageNet-100
Number epochs
300
200
300
Mini batch size
32
32
64
T
10
20
4
0.5
0.5
0.5
40
Vih
of
3, 12, 12 ///3
3, 12, 12
2, 6, 12
06 K
ay/3
3
3
3
λTET Learning Rate
0.05
0.05
0.1
0.1
0.005
Optimizer
SGD
SGD
SGD
Firing Mechanism
Probabilistic firing
Probabilistic firing
Probabilistic firing
Table Training parameters for FPT on Sequential CIFAR datasets.
Sequential CIFAR10
Sequential CIFAR100
Number epochs
256
256
Mini batch size
128
128
T
32
32
0.5
0.5
Uth
af
1, 3, 12
1, 12
Ob
af/3
0f/3
K
3
3
Learning Rate
0.1
0.1
Optimizer
SGD
SGD
Firing Mechanism
Probabilistic firing
Probabilistic firing
Table 7. Comparison between BPTT and FPT at Different Simulation Timesteps T'
Method
Training Time (s)
Inference Time (s)
Memory (MB)
Accuracy (%)
8
0,0195
0.0042
600
97.75 = 0.16
BPTT
64
0.0835
0.0257
956
97.67 = 0.17
512
1.701
0,2092
3238
97.84 + 0.12
0,0096
0.0021
622
97.73 0.23
FPT
64
0.0109
0.0021
1162
97.71 = 0.11
512
0.0803
0.0021
4264
97.70 + 0.22
D. Training and Inference Complexity Comparison
Our experiments focus comparing the complexity of BPTT and FPT at different timesteps 7, as shown in Table 7. We conducted experiments on the MNIST dataset using a 3-layer MLP 188x116x128x10),, with a batch size of 256, learning rate 0.001, and 80 epochs.
Here, "Time" refers to the average running time per batch during training or inference on a single A100 GPU. As shown,
14

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
both training and inference times for BPTT increase approximately linearly with T. In contrast, FPT's training time increases only slightly, while its inference time remains nearly constant. At T 256, FPT achieves 21x speedup over BPTT during training.
FPT introduces only slight increase in memory usage due to the LIF activation function during training, without impacting other network components. For larger networks such as MSS-ResNet-1e 18, the relative increase in memory consumption is even smaller.
E. Network Dynamics Across Long Timesteps
Table 8. Cosine Similarity (%) between Original and FPT Outputs under Different and Simulation Timesteps T
T
8
T = 64
T == 512
5
99.89
99.55
99.53
7
99.90
99.49
99.47
We replaced the LIF neurons in pre-trained 3-layer LIF-based ML. (784x266x128xI) trained on the MNIST dataset with parallel LIF neurons based FPT. Table 8 shows the cosine similarity (%) between the original and FPT-replaced outputs for 7-8, 64, and 512.
As '' increases, the outputs of the FPT-based parallel LIF and the original LIF neurons show slight divergence due to error accumulation, reducing the similarity of the final network outputs. However, even at = 512, the similarity remains above 99.5%, demonstrating that FPT preserves a high degree of consistency in network dynamics. Furthermore, this minor discrepancy can be effectively mitigated through light fine-tuning.
F. Learnable Decay Matrix Evaluation
1.5
1.0
14 20 20
10 14
Masked
FPT
0.0
-0.5
K
Figure 4. Learned decay matrices for different parallel spiking neurons.
Figure 4 illustrates the final learned decay matrices different parallel spiking neurons on the Sequential CIFARIÓ dataset. Although PSN achieved accuracy of 88.45% this dataset, failed to learn an exponential decay time-dynamic pattern similar to A, overly relying on future time information. Both IPSU and "FPT w/ Masked A applied a lower triangular mask during training to ensure that future information was not utilized. These models focused more on the present and immediate past, neglecting older information. Notably, "FPT w/ Masked A" learned a more distinct exponential decay pattern. For "FPT w/ A", no constraints were imposed during training, resulting in a stronger focus on information from adjacent timesteps. The comparison of PSN with "FPT w/ A" suggests that the reset mechanism favors learning reasonable
15

---

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
temporal patterns rather than maintaining reliance on distant temporal information.
G. Ablation Study of K
Table 9, Effect of Different Iteration Counts K on the DVS- Dataset,
K
K 4
K =
5
DVS-Gesture 98.61
98.61
98.61
To evaluate whether increasing the number of iterations K' improves performance, conducted experiments with different iteration counts under the same settings, including random seeds. The accuracy results on the DVS-Gesture dataset are shown in Table 9. As seen, increasing the iteration count does not affect accuracy. This is because, for K 2 2, with a = 12, the membrane potential ((K) converges by the time K = 3, and no significant changes occur with additional iterations. Therefore, the accuracy remains unchanged.
H. Discussion and Limitation
Motivation: Our primary goal was to develop parallel training approach that is independent of the model architecture, ensuring that the neuronal dynamics remain unchanged throughout the training process.
The forward process in FPT is identical to the original LIF model, ensuring that the key dynamics are retained. However, the backward process does not directly correspond to traditional BPTT.
Backward Process:
The reset mechanism is essential for certain tasks, especially those that require the network focus on the current input rather than be influenced by previous states. For example, in tasks where the input remains constant over time, the reset mechanism might not be necessary. However, for tasks involving long-term dependencies or highly variable inputs, retaining the reset mechanism becomes crucial. allows the network to discard irrelevant past information and focus on new, incoming data.
Reset Mechanism:
Flexibility and Compatibility: A key advantage of our algorithm is its flexibility. imposes no restrictions on specific network architectures and can be applied broadly across various SNN models. Importantly, since FPT preserves the original neuron dynamics, models trained using FPT remain fully compatible with standard sequential SNN inference, allowing seamless deployment on existing neuromorphic hardware without modification. This compatibility, combined with its efficiency, makes FPT a practical and powerful approach for pretraining SNN models, bridging the gap between scalable training and woal-world application.
One limitation FPT, compared to traditional SNN models, is its inability to handle scenarios where there no time-dependent decay, as seen in Integrate-and -nde-Fnn--Fir models. However, more neuron models typically incorporate a decay factor or gating mechanism, which ensures that the influence of past inputs diminishes over time. This allows the model to focus more recent data and prevents the accumulation of errors from earlier timesteps. Another limitation of FPT lies in the memory usage associated with parallel processing. Since FPT processes all timesteps simultaneously, it generally requires more memory than sequential training methods, as shown in Table 7. Future work could focus on optimizing memory efficiency. For instance, leveraging the binary nature of spike events could lead to more efficient memory compression techniques, enabling faster and more memory memory-efficient processing without sacrificing model performance.
Limitations:
16


Reviews:
[{"review_id": "WKlg9fTYXC", "reviewer": "Reviewer_hARo", "summary": "This paper proposes Fixed-point Parallel Training for efficient training of SNN, which does not change the network architectures. This training mode does not affect the dynamics of LIF neurons and achieves better performance on data with time series information such as DVS.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for recognizing the novelty and contributions of our proposed FPT framework, including its ability to reduce the training complexity of SNNs and the fact that existing parallel spiking neuron models can be viewed as special cases of FPT. We also appreciate the constructive and insightful feedback. Below, we answer the reviewers' questions in detail.\n\n**Q1: Various complexities of FPT during training refer to T-RevSNN**\n\n**A1**: Thank you for your valuable suggestion to further demonstrate the various complexities of FPT. The table below compares the theoretical training and inference complexity of different algorithms:\n\n|Methods|Training||Inference|Applicable scope|\n|-|-|-|-|-|\n||Memory|Time|Energy||\n|OTTT|$O(L)$| $O(LT)$|$O(T)$|Limited|\n|SLTT-k|$O(L)$|$O(Lk)$|$O(T)$|Limited|\n|T-RevSNN turn-off|$O(L)$|$O(L)$|$O(1)$|Limited|\n|T-RevSNN turn-on|$O(L)$|$O(T)$|$O(1)$|Limited|\n|BPTT|$O(LT)$|$O(LT)$|$O(T)$|Unlimited|\n|FPT|$O(LT)+\\lambda O(LKT)$|$O(LK)$|$O(T)$|Unlimited|\n\nHere, $L$ is the number of network layers, $T$ is the timestep, and $K$ is the number of iterations in FPT, which is typically 3 and does not increase with $T$. Thus, for long $T$, $K$ can be approximated to be negligible. The space complexity $O(LKT)$ and time complexity $O(LK)$ can be approximated as $O(LT)$ and $O(L)$, respectively. The coefficient $\\lambda$ represents the proportion of memory attributed to LIF components, which is the only part FPT increases—other parts of the network remain unaffected.\n\nIt is worth noting that methods such as OTTT, SLTT, and T-RevSNN truncate gradients or discard most of the temporal connections, which can limit their applicability to tasks requiring fine-grained temporal dynamics. In contrast, FPT accelerates SNN training without modifying the network and retaining the original neuron dynamics, so it is applicable to a wider range of SNN models.\n\nDue to time constraints and the limitations of OTTT, SLTT, and T-RevSNN in capturing temporal features, our experiments mainly focus on comparing the complexity of BPTT and FPT at different timesteps $T$. The experiments are conducted on the MNIST dataset using a 3-layer MLP (784×256×128×10), a batch size of 256, a learning rate of 0.001, and 80 epochs.\n\n|Method|T|Training Time (s)|Inference Time (s)|Firing rate (%, layer 1)|Firing rate (%, layer 2)|Memory (MB)|Accuracy (%)|\n|-|-|-|-|-|-|-|-|\n|BPTT|8|0.0195|0.0042|44.47|47.03|600|$97.75\\pm0.16$|\n||64|0.0835|0.0257|45.1|46.43|956|$97.67\\pm0.17$|\n||512|1.701|0.2092|46.55|47.76|3238|$97.84\\pm0.12$|\n|FPT|8|0.0096|0.0021|40.83|44.29|622|$97.73\\pm0.23$|\n||64|0.0109|0.0021|38.24|40.36|1162|$97.71\\pm0.11$|\n||512|0.0803|0.0021|38.14|39.94|4264|$97.70\\pm0.22$|\n\nHere, \"Time\" refers to the average running time for one batch during training or inference on a single A100 GPU. As shown in the table, both training and inference time for BPTT increase with $T$. In contrast, for FPT, the training time increases slightly, while the inference time remains almost constant. At $T=512$, FPT is 21x faster than BPTT during training. FPT only slightly increases the memory usage for the LIF during training without affecting other components of the network. In larger networks, such as MS-ResNet-18 trained on ImageNet1K, BPTT occupies 22.54GB on 4 3090 GPUs, while FPT occupies 23.52GB, resulting in only about a 4% increase in memory usage.\n\n**Q2: Experiments on ImageNet1K and If possible, HAR-DVS**\n\n**A2:** For a detailed discussion of the ImageNet1K experiments, we respectfully refer you to our response to **Reviewer gXhg (A2)**, due to space constraints here.\n\nWe thank the reviewer for suggesting the HAR-DVS dataset, a promising new benchmark for DVS-based action recognition. We will cite it in the introduction: `For instance, neuromorphic benchmark datasets such as HAR-DVS, DVS-CIFAR10 and DVS-Gesture typically need 10 or more timesteps to reach satisfactory accuracy.` However, due to time and computational constraints, and because this dataset was recently released, the main algorithms we compare have not yet reported results on it, making a direct comparison difficult in the short term.\n\n**Q3: TET Loss**\n\n**A3:** The baseline accuracy and metrics in our submission are from the best results in the respective papers. As TET loss is a mainstream loss function for SNNs, most of these baselines, such as T-RevSNN and LocalZO, utilize TET loss during training.\n\n**Q4: \"Time\" in Table 2 and Figure 3**\n\n**A4:** We apologize for any confusion. The \"time\" here refers to the average time required to train one batch on a single 3090 GPU. All baseline models were implemented with the same network architecture and hyperparameter configuration as ours, differing only in training method. FPT requires significantly lower training time than these baselines.\n\nWe sincerely hope that our clarifications have addressed your concerns and helped strengthen your confidence in our work. Thank you again for your thoughtful review.", "claims_and_evidence": "The three contributions of this paper are:\n\n- It \"proposes a novel Fixed-point Parallel Training (FPT) method that reduces the training time complexity of SNNs\";\n- it \"proves the convergence of FPT and demonstrates that existing parallel spiking neuron models can be derived as special cases of our framework\";\n- it \"retains the dynamic properties of original LIF neurons, achieves better performance, and significantly reduces computational time\".\n\nI agree with the first two contributions, but I am skeptical about the last one.\n\n- This paper does not show in detail the computation, time, and space complexity of using FPT to train SNN. Please refer to T-RevSNN [1] for detailed experimental data on various complexities during training.\n- FPT is mainly experimented on time series data, lacking experiments on static data. Please add experiments on ImageNet1k to prove that FPT is still reliable on static data tasks in the rebuttal.\n\n[1] High-Performance Temporal Reversible Spiking Neural Networks with O(L) Training Memory and O(1) Inference Cost. ICML 2024.", "methods_and_evaluation_criteria": "This paper lacks experiments on large static datasets, such as ImageNet1k. Also, the DVS data selected in this article is also relatively small. If possible, please consider supplementing the experiments on HAR-DVS [1] in the rebuttal.\n\n[1] HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensors. AAAI 2024.", "theoretical_claims": "The proof in Appendix A is correct in my view.", "experimental_designs_or_analyses": "Please see `Methods And Evaluation Criteria`.", "relation_to_broader_scientific_literature": "This paper focuses on addressing parallel training methods for SNNs and does not seem to make an obvious contribution to the broader scientific literature.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "There is no description of the `Time` in Table 2 and Figure 3, which may confuse me.", "questions_for_authors": "Please see `Claims And Evidence`, `Methods And Evaluation Criteria`, and `Supplementary Material`.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6lsnRp0tGA", "reviewer": "Reviewer_rhny", "summary": "The paper proposes a new training method for SNNs called Fixed-point Parallel Training (FPT), which aims to improve efficiency by reducing time complexity from O(T) to O(K), where K is a small constant. The method leverages a fixed-point iteration framework to enable parallel computation across timesteps rather than processing them sequentially. The authors argue that this approach preserves key neural dynamics, including the reset mechanism of Leaky LIF neurons, while accelerating training significantly. Theoretical analysis is provided to prove the convergence of FPT, and the authors demonstrate that some existing parallel spiking neuron models can be viewed as special cases of this method. Experiments show that FPT achieves competitive accuracy while significantly reducing computational costs compared to traditional BPTT and other existing parallel spiking neuron training methods.\n\n## update after rebuttal\n\nThanks to the authors for the detailed and thoughtful rebuttal. I appreciate the additional clarifications on memory usage, the effect of surrogate gradients on convergence, and the broader applicability of FPT. It’s clear a lot of effort went into addressing the points I raised.\n\nThat said, my overall assessment remains the same. While the new details help, the broader concerns — like the limited experimental scope, lack of deeper analysis on the practical trade-offs, and the fairly narrow range of tested applications — are still there. I think the paper presents a promising idea and the results are solid within the datasets tested, but it doesn't quite reach the level of novelty and thorough validation I’d expect for acceptance. So I’m keeping my original score of Weak Reject.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for recognizing that FPT enhances the efficiency of SNN training without modifying the underlying network architecture and for acknowledging FPT as a novel and well-motivated approach. Below, we provide detailed responses to your questions.\n\n**Q1: How does FPT compare in memory consumption to standard BPTT?**\n\n**A1:** To compare the memory consumption of BPTT and FPT, we conducted experiments on the MNIST dataset using a 3-layer MLP (784×256×128×10), a batch size of 256, a learning rate of 0.001, and 80 epochs. The table below shows the comparison at different $T$:\n\n|Method|T|Training Time (s)|Inference Time (s)|Memory (MB)|Accuracy (%)|\n|-|-|-|-|-|-|\n|BPTT|8|0.0195|0.0042|600|$97.75\\pm0.16$|\n||64|0.0835|0.0257|956|$97.67\\pm0.17$|\n||512|1.701|0.2092|3238|$97.84\\pm0.12$|\n|FPT|8|0.0096|0.0021|622|$97.73\\pm0.23$|\n||64|0.0109|0.0021|1162|$97.71\\pm0.11$|\n||512|0.0803|0.0021|4264|$97.70\\pm0.22$|\n\nHere, \"Time\" refers to the average running time for one batch during training or inference on a single A100 GPU. At $T=512$, FPT is 21x faster than BPTT during training. \n\nFPT only slightly increases the memory usage for the LIF during training without affecting other components of the network.\nThus, the overall memory consumption is not significantly higher than BPTT. For instance, in larger networks such as MS-ResNet-18 trained on ImageNet1K, BPTT consumes 22.54 GB on 4 3090 GPUs, while FPT consumes 23.52 GB, resulting in only a 4% increase in memory usage.\n\n**Q2: How does the surrogate gradient approximation affect convergence guarantees?**\n\n**A2:** The surrogate gradient approximation does not affect the convergence guarantees because these guarantees are based on the forward pass of the network, while the surrogate gradient approximation is applied on the backward pass. In addition, the surrogate gradient approximation is necessary for backpropagation in SNNs due to the non-differentiability of the spike activation function. The effectiveness of surrogate gradient approximation has been well verified through experiments on various datasets.\n\n**Q3: Can FPT be applied to reinforcement learning tasks or other non-spiking domains?**\n\n**A3:** FPT can indeed be applied to reinforcement learning tasks as long as the task uses LIF-based SNNs for temporal processing. Our approach is particularly suitable for addressing the high latency problem of sequence processing using LIF neurons. However, for other non-spiking domains that do not rely on SNNs, this is beyond the scope of our current work.\n\n**Q4: What are the potential failure cases where FPT might not converge effectively?** \n\n**A4:** In the submission，we have discussed this issue in Appendix F \"Discussion and Limitation\". FPT works well for LIF-based SNNs with temporal decay, but may not be effective for simple Integrate-and-Fire models that lack such mechanisms. However, mainstream neuron models typically incorporate a decay factor or gating mechanism, which ensures that the influence of past inputs diminishes over time. \n\n**Other Questions**\n\n- *The claim that FPT maintains all critical neural dynamics is also somewhat oversimplified because the backward process does not exactly correspond to traditional BPTT.*\n\nNeural dynamics, including leakage, integration, firing, and reset, are part of the neuron's forward process. Neither the backward process of BPTT nor FPT belongs to the neural dynamics process; they are solely used to optimize network weights.\n\n- *Since one of the key motivations of SNNs is energy efficiency, a discussion of power consumption during training would be valuable.*\n\nThe energy efficiency of SNNs typically refers to inference on neuromorphic hardware. Both BPTT and FPT rely on floating-point operations on GPUs during training, which does not reflect deployment energy costs. FPT does not affect sequential inference behavior of SNNs. As shown in Table 2 of the submission and in additional experiments presented in **Review hARo (A1)**, its firing rate is comparable to or even lower than BPTT, ensuring that SNNs trained with FPT remain energy efficient during deployment.\n\n- *The paper primarily focuses on neuromorphic datasets like DVS-CIFAR10 and DVS-Gesture*\n\nIn the submission, we reported results on dynamic datasets such as DVS-CIFAR10 and DVS-Gesture, static datasets including ImageNet-100, and graph datasets like Amazon Photos, as well as sequential datasets such as Sequential CIFAR10 and Sequential CIFAR100.\n\n- *Additional numerical experiments demonstrating different convergence behaviors would strengthen the argument.*\n\nIn the submission, Section 6.1 \"LIF Dynamics Simulation\" already discusses the convergence behavior.\n\n- *Providing open-source code would improve reproducibility.*\n\nThe code will be publicly available on GitHub after the publication of this work.\n\nWe sincerely hope that our clarifications above have addressed your concerns and that our responses contribute positively to your understanding of our work.", "claims_and_evidence": "The main claim is that FPT allows parallel training of SNNs without modifying the network architecture while maintaining biological interpretability. The authors provide both theoretical and empirical support for this claim. The convergence proof of the fixed-point iteration is a positive contribution, though it relies on certain assumptions about the Lipschitz continuity of the surrogate function. The empirical results show that FPT achieves a significant speedup over BPTT while maintaining comparable accuracy across multiple datasets. However, the experiments lack detailed comparisons with alternative approaches such as event-based SNN training techniques or alternative parallelization strategies like recurrent SNNs with gating mechanisms. The claim that FPT generalizes well to different datasets is only partially supported since the paper primarily focuses on neuromorphic datasets like DVS-CIFAR10 and DVS-Gesture but does not test on more complex real-world applications, such as continuous control or large-scale spiking datasets.\n\nOne aspect that is not thoroughly examined is the potential trade-off between computational speed and memory usage. Since FPT processes all timesteps in parallel, it likely requires more memory than sequential training methods. The authors acknowledge this in the discussion but do not provide a quantitative breakdown of memory overhead. The claim that FPT maintains all critical neural dynamics is also somewhat oversimplified because the backward process does not exactly correspond to traditional BPTT. The experimental results support the main claims to some extent, but the paper would benefit from deeper analysis of efficiency trade-offs and broader comparisons to alternative SNN training methods.", "methods_and_evaluation_criteria": "The authors evaluate FPT on standard neuromorphic datasets such as DVS-CIFAR10, DVS-Gesture, and ImageNet-100. These datasets are commonly used for benchmarking SNNs, so the choice is reasonable. The experiments compare FPT to existing training methods such as timestep shrinkage, online training, and stabilized spiking flow, providing a fair assessment of performance gains. However, the evaluation criteria focus almost entirely on accuracy and training speed, without considering other important factors like memory usage, energy efficiency, or sensitivity to hyperparameters. Since one of the key motivations of SNNs is energy efficiency, a discussion of power consumption during training would be valuable. \n\nThe authors also conduct an ablation study to examine the impact of the number of iterations (K) and the role of the reset mechanism. This is a strong aspect of the evaluation, as it provides insight into the effectiveness of the method. However, a more detailed breakdown of how K affects convergence speed and accuracy would make the evaluation even stronger. The experimental design is generally sound but could be improved by including a broader set of tasks, particularly those requiring long-term dependencies, to further validate the generalization ability of FPT.", "theoretical_claims": "The paper provides a theoretical proof that FPT converges to a fixed point under certain conditions. This is an important contribution, as many SNN training methods lack formal convergence guarantees. The proof is based on contraction mapping and Lipschitz continuity arguments, which are reasonable assumptions for neural networks with smooth surrogate gradients. However, the proof does not establish how the convergence rate of FPT compares to BPTT or whether it guarantees optimal weight updates in all cases. The surrogate gradient approach used in the backward pass introduces an additional layer of approximation, and the impact of this approximation on convergence is not fully analyzed. The theoretical claims are generally well-supported but would benefit from additional discussion on potential failure modes, such as scenarios where the fixed-point iteration might converge too slowly or to suboptimal solutions.", "experimental_designs_or_analyses": "The experimental setup is well-structured, with comparisons across multiple datasets and an ablation study to analyze key design choices. The main advantage demonstrated is the speedup of training, particularly for long-duration simulations where traditional BPTT is inefficient. However, there are some weaknesses in the analysis. The paper does not include a detailed breakdown of computational cost beyond training time, such as memory usage and GPU utilization. While FPT reduces the number of sequential operations, it likely increases parallel memory requirements, and this trade-off is not analyzed in depth. \n\nAnother issue is the limited scope of dataset selection. Most of the experiments focus on neuromorphic benchmarks, which are useful for validating the method but do not fully demonstrate its applicability to broader machine learning tasks. It would be beneficial to see results on tasks like speech recognition, reinforcement learning, or other domains where SNNs are increasingly being applied. Finally, the statistical significance of the results is not clearly reported. While standard deviations are included, there is no discussion of whether the differences in accuracy are statistically significant across multiple runs.", "relation_to_broader_scientific_literature": "The paper is well-grounded in the existing literature on SNN training methods and parallel computing in neural networks. It references prior work on timestep reduction, online training, and surrogate gradient methods, positioning FPT as an improvement over these approaches. However, the discussion is somewhat limited to SNN-specific methods and does not draw connections to broader machine learning literature. There are similarities between FPT and parallelization techniques used in recurrent neural networks (RNNs) and deep equilibrium models, but these connections are not explored in depth. It would be useful to compare FPT to other parallel training techniques used in non-spiking networks to highlight its broader relevance.", "essential_references_not_discussed": "The paper discusses prior work on parallel SNN training and timestep reduction but does not cite some relevant research on alternative acceleration techniques, such as event-driven training methods or hybrid ANN-SNN approaches. There is also limited discussion of how FPT compares to neuromorphic hardware implementations, which are a key motivation for efficient SNN training. Some recent work on efficient surrogate gradient methods and biologically inspired training rules could also be relevant for positioning FPT within the broader field.", "other_strengths_and_weaknesses": "The main strength of the paper is its focus on improving the efficiency of SNN training without modifying the underlying network architecture. The use of fixed-point iteration for parallel training is a novel and well-motivated approach. The experimental results demonstrate clear speed improvements over BPTT while maintaining accuracy. However, the paper has several weaknesses. The evaluation lacks a detailed analysis of computational trade-offs, particularly regarding memory usage and energy efficiency. The theoretical analysis, while useful, does not fully address the impact of surrogate gradients on convergence. The experiments are mostly limited to neuromorphic datasets, which may not fully demonstrate the generalization ability of FPT.", "other_comments_or_suggestions": "The authors should include a discussion on memory efficiency and computational cost beyond training time. They should also test FPT on a broader range of tasks to demonstrate its applicability. Providing open-source code would improve reproducibility.", "questions_for_authors": "1. How does FPT compare in memory consumption to standard BPTT? \n2. How does the surrogate gradient approximation affect convergence guarantees?\n3. Can FPT be applied to reinforcement learning tasks or other non-spiking domains?\n4. What are the potential failure cases where FPT might not converge effectively?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NNCR1TCD5v", "reviewer": "Reviewer_gXhg", "summary": "This work introduces Fixed-point Parallel Training (FPT), a novel method that reduces SNN training time complexity from O(T) to O(K) (where K is a small constant, typically K = 3) by enabling efficient parallel processing across all timesteps without modifying the network architecture. A theoretical convergence analysis proves the stability of FPT and demonstrates that existing parallel spiking neuron models are special cases of this framework. Importantly, FPT preserves LIF neuron dynamics, including membrane potential updates and reset mechanisms, ensuring both biological interpretability and computational efficiency. By decoupling sequential dependencies, FPT significantly accelerates training while maintaining or even improving accuracy, making it a scalable and efficient solution for large-scale, long-duration SNN tasks on modern hardware. This advancement enhances the feasibility of deploying SNNs in real-world applications, particularly in neuromorphic computing and time-sensitive spatiotemporal processing tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your encouraging feedback and for recognizing that FPT significantly accelerates SNN training, making it a scalable and efficient solution for large-scale, long-duration tasks. We also appreciate your recognition of its potential in practical applications, especially neuromorphic computing and time-sensitive spatiotemporal processing. Below, we provide detailed responses to your questions and concerns.\n\n**Q1:  The network dynamics affacted across long T**\n\n**A1:** We replaced the LIF neurons in a pre-trained 3-layer LIF-based MLP (784×256×128×10) on the MNIST dataset with parallel LIF neurons based on FPT. The cosine similarity (%) between the original and FPT-replaced outputs for $T$=8, 64, and 512 is shown in the table below:\n\n||T = 8|T = 64|T = 512|\n|-|-|-|-|\n|$\\alpha=5$|99.89|99.55|99.53|\n|$\\alpha=7$|99.90|99.49|99.47|\n\nAs $T$ increases, the outputs of the FPT-based parallel LIF and the original LIF become less consistent due to error accumulation, resulting in a slight decrease in the similarity of the final network output. However, even for $T$=512, the similarity remains around 99.5%, indicating that FPT maintains a high degree of consistency in the network dynamics. Moreover, this minor difference can be addressed by light fine-tuning.\n\n**Q2: Experiments on ImageNet1K**\n\n**A2:** The proposed FPT mainly focuses on improving the speed for long $T$. Thus, in the submission, we reported results on dynamic datasets such as DVS-CIFAR10 ($T$=10) and DVS-Gesture ($T$=20), static datasets including ImageNet-100 ($T$=4), and graph datasets like Amazon Photos ($T$=8, 32, 128), as well as sequential datasets such as Sequential CIFAR10 ($T$=32) and Sequential CIFAR100 ($T$=32). In contrast, static datasets like ImageNet usually use smaller timesteps (e.g., $T$=5 or 6).\n\nIn addition, due to the high computational cost of training SNNs on ImageNet-1K, many previous studies (e.g., SSNN and LocalZO) have adopted alternative benchmarks such as Tiny-ImageNet or ImageNet-100. For this reason, we also chose ImageNet-100 as the representative benchmark in the main results. We are currently actively conducting experiments on ImageNet-1K, using the same experimental settings as MS-ResNet-18 [1]. Due to time constraints, we report the preliminary results we have obtained so far below.\n\n||BPTT (82 epoch, our running, lastest)|FPT (82 epoch, our running)|FPT (97 epoch, lastest)|\n|-|-|-|-|\n|Accuarcy (%)|52.27|53.57|57.55|\n|Training loss|2.21|2.18|1.98|\n|Per-batch time (s)|8.48|5.76|5.76|\n\nThe training loss refers to the cross-entropy loss of the last batch in the given epoch. As seen, with the same hyperparameters and number of epochs, FPT achieves higher accuracy and lower loss. Additionally, even with $T$=6, FPT achieves approximately 1.5× faster training speed compared to BPTT on the same 4 3090 GPUs. It is important to note that, due to time and computational limitations, we have not yet fine-tuned the hyperparameters for FPT. Further adjustments may lead to even better performance.\n\n[1] Advancing Spiking Neural Networks Toward Deep Residual Learning. TNNLS 2024\n\nWe sincerely hope that our clarifications have addressed your concerns and will help improve your opinion of our work.", "claims_and_evidence": "The claims are supported by clear evidence.", "methods_and_evaluation_criteria": "The \"LIF Dynamics Simulation\" part only include a single LIF neuron. However, the propagation of network dynamics might be quite different since the effect could accumulate. How does the network dynamics be affacted across long T.\nHow does the method perform on ImageNet-1K.", "theoretical_claims": "The proofs are correct.", "experimental_designs_or_analyses": "See \"Methods And Evaluation Criteria\"", "relation_to_broader_scientific_literature": "Yes", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "See \"Methods And Evaluation Criteria\"", "other_comments_or_suggestions": "No", "questions_for_authors": "See \"Methods And Evaluation Criteria\"", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bg7qaqazqx", "reviewer": "Reviewer_3oso", "summary": "This paper introduces a novel Fixed-point Parallel Training (FPT) method to accelerate Spiking Neural Networks (SNNs) training.\nThe method is theoretically analyzed for convergence, and the authors show that existing parallel spiking neurons are special cases of this approach.\nExperimental results demonstrate that FPT simulates the dynamics of original LIF neurons effectively, significantly reducing computational time without sacrificing accuracy", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your high evaluation of our work, seeing this paper as a milestone due to its potential to enable parallel pre-training of SNNs. We also appreciate your valuable and detailed feedback. Below, we will answer your questions.\n\n**Q1: The usage of \"\\cite\" and \"\\citet\"**\n\n**A1:** We have carefully re-checked all citations throughout the manuscript. However, since ICML does not allow manuscript modifications during the rebuttal period, we will correct any remaining formatting issues (e.g., incorrect use of `\\cite` vs `\\citet`) in the camera-ready version if the paper is accepted.\n\n**Q2: Running time comparison of the training process and inference process**\n\n**A2:** Our proposed algorithm supports parallel training, including parallel forward and backward passes. We achieve speedups in both passes. To demonstrate this, we compare the training and inference complexity of BPTT and FPT at different timesteps ($T$). The experiments are conducted on the MNIST dataset using a 3-layer MLP (784×256×128×10), batch size = 256, learning rate = 0.001, and 80 training epochs.\n\n|Method|T|Training Time (s)|Inference Time (s)|Accuracy (%)|\n|-|-|-|-|-|\n|BPTT|8|0.0195|0.0042|$97.75\\pm0.16$|\n||64|0.0835|0.0257|$97.67\\pm0.17$|\n||512|1.701|0.2092|$97.84\\pm0.12$|\n|FPT|8|0.0096|0.0021|$97.73\\pm0.23$|\n||64|0.0109|0.0021|$97.71\\pm0.11$|\n||512|0.0803|0.0021|$97.70\\pm0.22$|\n\nHere, \"time\" refers to the average time required to train or infer a single batch on a single A100 GPU. As shown in the table, both the training and inference time of BPTT increase with the number of timesteps $T$. In contrast, the training time of FPT increases only slightly, and its inference time remains almost the same regardless of $T$. Notably, at $T = 512$, FPT is 21 times faster to train compared to BPTT.\n\n**Q3: Extended to pre-training SNNs parallelly**\n\n**A3:** A key advantage of our algorithm is its flexibility - it does not impose restrictions on specific network architectures and can be applied to a wide range of SNN models. By replacing the time-consuming LIF neuron sequential computation with our proposed FPT-based parallel iterations, both the forward and backward passes during training can be significantly accelerated.\n\nImportantly, since FPT preserves the original neuron dynamics, the pre-trained model can still be deployed and inferred using standard SNN sequential processing. This makes FPT not only suitable for accelerating training, but also highly compatible with existing SNN hardware deployments. Therefore, FPT provides a practical and general solution for efficient parallel pre-training of SNNs without sacrificing biological fidelity or inference compatibility.\n\nWe again sincerely thank you for the thoughtful and encouraging comments. We hope that our response will clarify your concerns and further demonstrate the practicality and generality of FPT.", "claims_and_evidence": "Yes. The claim in this paper is clear.", "methods_and_evaluation_criteria": "Yes. The method is novel and theoretically complete.", "theoretical_claims": "Yes. The proofs about fixed points are correct.", "experimental_designs_or_analyses": "Yes. Experiments in this paper is extensive and sound.", "relation_to_broader_scientific_literature": "This paper is an extension of previous work on parallel training SNNs.\nI see this paper as a milestone because this method can be used to pre-train a SNN parallelly.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n1. correct and robust proofs;\n2. well-written and clear motivations;\n3. extensive and convincing experiments.\n\nWeaknesses:\nNo obvious weaknesses.", "other_comments_or_suggestions": "1. The usage of \"\\cite\" and \"\\citet\" in latex should be careful. There are several misuses of \"\\cite\" in section 2.3 and section 4.3.1\n2. The authors should consider the running time comparison (or operation counts) of the training process and inference process between your method and traditional surrogate methods.", "questions_for_authors": "1. I wonder whether your method can be extended to pre-training SNNs parallelly.\nThere have been several works like SpikeBERT, SpikeGPT, and SpikeLM which pre-trained LIF-based spiking Transformers non-parallelly.\nPlease discuss the possibility of your method on pre-training SNNs.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a novel fixed-point parallel training method for spiking neural networks, aiming to accelerate training by reducing temporal complexity without modifying the network architecture. The method simulates the dynamics of LIF neurons and achieves improved performance, particularly for long-duration simulations.

Reviewer hARo acknowledged the theoretical contributions. Reviewers rhny and gXhg found the experiments with speed improvements while maintaining accuracy. Reviewer 3oso further acknowledged the theoretical analysis and the extensiveness of experiments for scalable parallel SNN training.

Reviewer hARo questioned the generality of the method on large-scale datasets and insufficient discussion of training complexity. Reviewer rhny also raised concerns about comparisons with other models. Reviewer gXhg suggested a more detailed explanation of the method. Besides, Reviewer 3oso suggested minor improvements in citation formatting. The authors addressed their concerns during the rebuttal phase. Following the discussion, the paper received final scores of (4, 4, 3, 2). Overall, I think this work is technically sound and useful. Therefore, I recommend this paper be accepted.
Decision: Accept (poster)