Rethinking Spiking Neural Networks from an Ensemble Learning Perspective
Authors: Yongqi Ding
Lin Zuo
Mengmeng Jing
Pei He
Hanpu Deng
Venue: ICLR 2025 Poster (2025)
Keywords: Spiking neural network
Membrane potential smoothing
Neuromorphic object recognition
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: Rethinking spiking neural networks from an ensemble learning perspective and improving their consistency across timesteps and overall performance.

Abstract: Spiking neural networks (SNNs) exhibit superior energy efficiency but suffer from limited performance. In this paper, we consider SNNs as ensembles of temporal subnetworks that share architectures and weights, and highlight a crucial issue that affects their performance: excessive differences in initial states (neuronal membrane potentials) across timesteps lead to unstable subnetwork outputs, resulting in degraded performance. To mitigate this, we promote the consistency of the initial membrane potential distribution and output through membrane potential smoothing and temporally adjacent subnetwork guidance, respectively, to improve overall stability and performance. Moreover, membrane potential smoothing facilitates forward propagation of information and backward propagation of gradients, mitigating the notorious temporal gradient vanishing problem. Our method requires only minimal modification of the spiking neurons without adapting the network structure, making our method generalizable and showing consistent performance gains in 1D speech, 2D object, and 3D point cloud recognition tasks. In particular, on the challenging CIFAR10-DVS dataset, we achieved 83.20\% accuracy with only four timesteps. This provides valuable insights into unleashing the potential of SNNs.

Published as a conference paper at ICLR 2025
RETHINKING SPIKING NEURAL NETWORKS FROM AN ENSEMBLE LEARNING PERSPECTIVE
Yongqi Ding, Lin Zuo", Mengmeng Jing, Pei He, Hanpu Deng School of Information and Software Engineering University of Electronic Science and Technology of China
ABSTRACT
Spiking neural networks (SNNs) exhibit superior energy efficiency but suffer from limited performance. In this paper, we consider SNNs as ensembles of temporal subnetworks that share architectures and weights, and highlight a crucial issue that affects their performance: excessive differences in initial states (neuronal mem- brane potentials) across timesteps lead to unstable subnetwork outputs, resulting in degraded performance. To mitigate this, we promote the consistency the initial membrane potential distribution and output through membrane potential smooth- ing and temporally adjacent subnetwork guidance, respectively, to improve overall stability and performance. Moreover, membrane potential smoothing facilitates forward propagation of information and backward propagation of gradients, mit- igating the notorious temporal gradient vanishing problem. Our method requires only minimal modification of the spiking neurons without adapting the network structure, making our method generalizable and showing consistent performance gains in ID speech. 2D object, and 3D point cloud recognition tasks. In particu- lar, on the challenging CIFARIO-DVS dataset, we achieved 83, 33.20% accuracy with only four timesteps. This provides valuable insights into unleashing the potential of SNNs.
INTRODUCTION
As the third generation of neural networks, spiking neural networks (SNNs) transmit discrete spikes between neurons and operate over multiple timesteps (Maass, 1997). Benefiting from the low power consumption and peatiottmppora feature extraction capability, SNNs have achieved widespread ap- plications in spatio-temporal tasks (Wang & Yu, 2024; Chakraborty et al., 2024). In particular, ultra- low latency and low-power inference can be achieved when SNNs integrated with neuromorphic sensors and neuromorphic chips (Yao et al., 2023; Ding et al., 2024).
To advance the performance of SNNs, previous work has improved the training method (Wu al., 2018; Bu et al., 2022; Zuo et al., 2024b), network architecture (Yao et al., 2023; Shi et al., 2024), and neuron dynamics (Taylor al., 2023; Fang al., 2021b; Ding al., 2023) to significantly reduce the performance gap between SNNs and artificial neural networks (ANNs). Typically, these methods treat the spiking neurons in an SNN as an activation function that evolves over timesteps T, with the membrane potential expressing continuous neuronal state. In this paper, we seek to rethink the spatio-temporal dynamics of SNNs from an alternative perspective: ensemble learning, and explore the key factor influencing the ensemble to optimize its performance.
For an SNN (0). its instance f(0); produces an output O at each timestep I, we consider this instance to be temporal subnetwork. In this way, we can obtain a collection of 7 temporal subnet- works with the same architecture f()) and parameters 0: {f(0).. f(0),. ... f(0)r). In general, the final output of the SNN is the average of the outputs over T timesteps: O = 1. Σ O₁. We view this averaging operation as an ensemble strategy: averaging the different outputs of multiple models improves overall performance (Rokach, 2010; Allen-Zhu & Li, 2020). From this perspective, we can consider an SNN as an ensemble of T temporal subnetworks.
"Corresponding author: linzuo@ weeooeuuustteuuccm Emails: Yongqi Ding dodding@sdd.uestceeduu Mengmeng Jing (jigmmen11992 @maiilmmmm..mmmm Pei He (hepei wttee.ssst..dsduuu... Hanpu Deng (hp- deng @ std.ueescc.cdu.cmm

---

Published as a conference paper at ICLR 2025
T-0 Initialized to
T=1 (j.o)-10.08.0100 (0.08.0.10)
Tw2 (0.000(000....1))))
Tw3 (w.df/
Tw4 (.00)1000000222)
C3 62
-5 EX
-22 Membrane potential
+8s Membrane potential
Membrane potential
Membrane potential (pue)-10.01.0113
Membrane potential (p.00((0002.001
(1.o)-(-0.02,00
(p.00=1-0.02.0221
30 Membrane potential
2x +us TO Membrane potenbal
Membrane potential
DJ Membrane potential
Membrane potenúal
Figure I: Membrane potential distribution on CIFARIO-DVS, where l and a denotes the mean and standard deviation, respectively. Top: The membrane potential distribution of the vanilla SNN varies greatly across timesteps, which affects performance. Bottom: Our method allows for a more stable distribution with smaller differences across timesteps. See Appendix A.10 for more visualizations.
Temporal subnetworks share architecture and parameters, and their output variance arises from their different neuronal states, i.e., different membrane potentials U (t)) each timestep trigger different output spikes S(t). For example, for a leaky integrate-and- -nd-fere (LIF) neuron (Wu et al., 2018) (see Section 3.1) with an initial membrane potential of 0 and time constant and firing threshold of 2.0 and 1.0, respectively, the neuron will produce spikes {0, 1, 1.0, 0, 1} even if inputs of intensity 0.8 are repeated over timesteps. In light of this, we have identified a factor that usually overlooked, but has a major impact on the performance of this temporal subnetwork ensemble learning: if the differ- ence membrane potentials across timesteps is too large, it will lead to unstable outputs and thus affect the ensemble performance. In particular, the initial membranc potential is usually set 0 (Wu et al., 2018; Ding et al., 2024), which leads to a drastic discrepancy in the membrane potential for the first two timesteps, thus degrading the performance of the SNN. To illustrate this phenomenon, we have visualized the membrane potential distribution in Fig 1 (Top). where the distribution differ- ences
timesteps can be clearly seen. In Table trained SNN for inference with to timesteps. The results show that the output of the vanilla SNN poorly informative the first timestep and only achieves decent performance after integrating subsequent temporal subnetworks with smaller differences in the membrane potential distribution. Additionally, the output is visualized in Fig. 4, again showing that the first two timestep outputs are confusing and thus affect the overall output. Therefore, to improve overall performance, the prob- lem of excessive difference membrane potential distribution and the resulting output should be mitigated rather than simply ignored.
further explore the performance of the
To this end, we propose membrane potential smooth- ing reduce the difference in membrane potential dis- tribution and thus improve the overall ensemble per- formance. At each timestep, we adaptively smooth the membrane potentials using that of the previous timestep, pushing the initial states of these tempo- ral subnetworks more consistent and preventing them from producing outputs with excessive variances. We visualize the smoothed membrane potential in Fig. 1(Bottom), where the obtained membrane potential shown to be smoother than the vanilla SNN. Meanwhile, membrane potential smoothing creates new pathways for forward propagation of information and backward propagation of gradients (see Fig. 2), alleviating temporal gradient vanishing (Meng al., 2023; Huang et al., 2024) and boosting the performance from another side. In addition, we propose to guide temporally adjacent subnetworks through distillation, encouraging them to produce more consistent outputs, further enhancing ensemble stability and performance. Notably, membrane potential smoothing employs only one additional smoothing coefficient, and the distillation guidance only modifies the training loss without affecting the architecture, making our method superiorly generalizable. Extensive experiments in ID, 2D, and 3D scenarios demonstrate the effectiveness, generalizability, and performance advantages of our method. Our contribution can be summarized as follows:
Table I: Comparison of SNNs with to inference timesteps.
T=1
T=2
T=3
T=4
T=5
Vanilla SNN 10.00 60.10
69.50
73.30
74.10
Random MP 11.90
61.50
71.40
72.30
74.90
Ours
66.60 74.30 75.50 75.70
76.60
2

---

Published as a conference paper at ICLR 2025
We consider the SNN as an ensemble of multiple temporal subnetworks and point out that excessive differences in membrane potential distributions across timesteps, and hence output instability, is the key factor affecting performance.
We propose membrane potential smoothing and temporally adjacent subnetwork guidance to adaptively reduce membrane potential differences and enhance output consistency across timesteps, respectively, while facilitating the propagation of forward information and back- ward gradients to improve ensemble stability and overall performance.
Extensive experiments on neuromorphic ID speech/2D object recognition, and 3D point cloud classification tasks confirm the effectiveness, versatility, and performance advantages of our method. With only 4 timesteps, we achieved 83.20% accuracy on the challenging CIFARIO-DVS dataset.
2 RELATED WORK
2.1 SPIKING NEURAL NETWORK
Existing methods for training SNNs avoid the non-differentiabiiit of the spiking neurons either by converting pre-trained ANN (Rueckauer et al., 2017) by using the surrogate gradient for direct training (Wu et al., 2018). Conversion- baaed methods require large latencies and struggle with the temporal properties of SNNs (Deng & Gu, 2021; Bu et al., 2022; kang you et al., 2024), the surro- gate gradient-based methods are widely used as they can achieve decent performance with smaller latencies (Taylor et al., 2023; Zuo et al., 2024a; Hu et al., 2024). In addition to training methods, previous work has focused on improving network architectures and spiking neuron dynamics, such as the Spiking Transformer architecture (Yao et al., 2023; Shi et al., 2024), the ternary spike (Guo et al., 2024), and the attention spiking neuron (Ding et al., 2023). Compared to existing methods, rethink the spatio-temporal dynamics of the SNN from the perspective of ensemble learning, identify the key factor affecting its performance: the excessive difference in membrane potential distribution, and propose solutions. Our solutions do not modify the core philosophies of these ex- isting methods and are therefore compatible with a wide range of architectures and neuron types, and integration with existing methods can further unleash the potential of SNNs.
2.2 ENSEMBLE LEARNING
Ensemble learning aggregates the predicted outputs of multiple models to improve the performance of a deep learning model (Rokach, 2010; Allen-Zhu & Li, 2020). To reduce ensemble overhead, some methods use backbone network and multiple heads to produce multiple outputs (Tran al., 2020; Ruan et al., 2023), or use checkpoints during training for the ensemble (Furlanello et al., 2018; Lee al., 2022). In the field of SNNs, previous studies have ensembled multiple SNN models to improve performance without optimizing the ensemble overhead (Neculae et al., 2021; Elbrecht al., 2020; Panda et al., 2017: Hussaini et al., 2023). In this paper, we consider each timestep SNN instance as temporal subnetwork and treat the entire SNN as an ensemble, thus avoiding additional ensemble overhead. A previous study (Ren et al., 2023) attributed the effectiveness of SNNs static point cloud classification to the ensemble effect, without further analysis. Instead, we point out the key factor influencing the ensemble performance: excessive differences in membrane potential distributions can lead to unstable outputs of these subnetworks, and propose solutions to mitigate this problem, therehy improving the performance. Moreover, our experiments on various tasks suggest that this ensemble instability is ubiquitous and should be highlighted rather than simply ignored.
is worth noting that previous ANN ensemble methods increase diversity within reasonable limits to promote generalization (Ali al., 2021; Zhang et al., 2020). Instead, we take different phi- losophy in SNNs, reducing difference rather than increasing diversity to promote stability, because the temporal subnetworks in SNNs are already beyond the limits of effective ensemble, and exces- sive diversity will only degrade overall performance. The necessity to reduce the tross-timester differences of the SNN is discussed in detail in Appendix A.I.
2.3 TEMPORAL CONSISTENCY IN SNNS
Previous studies have shown that promoting temporal consistency can improve the performance of SNNs, such as distillation (Zuo et al., 2024a; Dong et al., 2024) and contrastive learning (Qiu et al., 2024) in the temporal dimension. However, existing methods directly promote output/feature
3

---

Published as a conference paper at ICLR 2025
consistency, similar to ANNs, without adequately considering the properties of SNNs. In contrast to existing methods, this paper highlights the negative impact of differences membrane potential distributions across timesteps from ensemble perspective and proposes improve distribution consistency. Compared to output/feature consistency, membrane potential distribution consistency offers significant performance gains and be combined with them synergistically maximize performance.
3 METHOD
In this section, we describe the temporal dynamics of the SNN from the basic LIF neuron model and interpret it as the ensemble of multiple temporal subnetworks. We then show that excessive differ- ences in membrane potentials across timesteps affect ensemble performance, and we mitigate this problem by adaptively smoothing membrane potentials. In addition, we encourage temporally ad- jacent subnetworks produce stable and consistent outputs through distillation, further facilitating ensemble stability and performance.
3.1 SNN AS AN ENSEMBLE OF TEMPORAL SUBNETWORKS
SNNs transmit information by generating binary spikes from spiking neurons. In this paper, we use the most commonly used LIF neuron model (Wu et al., 2018). LIF neurons continuously receive inputs from presynaptic neurons accumulating membrane potential H. and spike fired and resets the membrane potential when it reaches the firing threshold 0. The LIF neuron dynamics can be expressed as:
U: (1-
Hi(1 1) A(),, charge
$1() - 0, H < 0
H{ 2
fire spike
(1()) = -1(1)-
- (1)) reset
(3)
where 1, and & denote the layer, neuron, and timestep indexes, respectively, and (t)
E, 5/ (t) is the cumulative current of the previous layer's neuron outputs and weights. is the time constant that controls the decay of the membrane potential with time. (t) is the initial membrane potential at timestep t.
From Eq. 1, we can see that the output (t) of the neuron at timestep t depends on its initial mem- brane potential (L). The membrane potential evolves continuously, so the SNN output varies from timestep timestep. Established methods recklessly compute the average over T timesteps outputs O = + Σd On without considering the rationale behind it, leading to suboptimal performance, whereas we explore this from an ensemble perspective to improve the performance of SNNs.
We consider each timestep the SNN as a temporal subnetwork sharing the architecture s(() and the parameter 0, and different subnetworks produce different outputs O2 arising from distinct neuron membrane potentials U (t). When the membrane potential difference of these subnetworks is small, their outputs vary slightly, and the ensemble can promote the generalizability of the SNN. However, excessive differences in membrane potentials can lead to drastically different outputs from these subnetworks and degrade the SNN. Unfortunately, vanilla SNNs seem to suffer from this degrada- tion, especially since the initial membrane potential is usually set to 0 (Wu et al., 2018; Ding et al., 2024), the membrane potentials of the first two timesteps show drastic differences, see Fig. This membrane potential discrepancy leads to highly unstable outputs across timesteps, which increases the optimization difficulty and degrades the ensemble performance. In addition, we show the per- formance of the trained SNN with different inference timesteps in Table 1, and the results show that the output of the vanilla SNN at the first timestep not discriminative at all.
3.2 MEMBRANE POTENTIAL SMOOTHING
To mitigate the above degradation problem, natural solution is randomly initialize the membrane potential to reduce the membrane potential difference for the first two timesteps, similar to (Ren al., 2023). However, the experiments show in Table 1 indicate that this practice (Random
4

---

Published as a conference paper at ICLR 2025
Forward propagation
Spanal
Backgroopagation
Timestpp
(((()
(c)
1-
(r(()
(((+)
A:
Change
Reset
Charge
Reset
()
$1(()
(1
Fire
H0
S'O
H1O0
Tanyoral
Lenk
Look
(1++1
(1
I-aar
(c+1
(1-11
(1-1
Change
Beset
Charge
Resa
Timestop
(211)
(1+
Lesk
Leak
Timestop
Timestop
Timestop
Timestep
(a) Vanilla LIF
(b) Membrame Potential Smoothing
(c) Vanilla LIF
Membrane Potential Smoothing
Additional forward propagation pathway
backprogagation pailway
Original bickprepagation pathway
Figure 2: Illustration of (a) the vanilla LIF neuron and (b) the membrane potential smoothing. We smooth the membrane potential at timestep ł using the layer-shared coefficient a' and the smoothed membrane potential H (1 ( 1) at timestep -- 1 to reduce membrane potential differences and create additional iioommation/graieen propagation pathways.
MP) only slightly alleviates the problem and still struggles to achieve satisfactory performance. To this end, propose membrane potential smoothing, which mitigates degradation by adaptively reducing the membrane potential difference between adjacent timesteps using a learnable smoothing coefficient.
At each timestep, we consider the initial membrane potential state (t) the spiking neuron as the initial state of the corresponding subnetwork. We argue that by allowing these subnetworks have similar initial states, the output spikes generated after receiving the input current will also be similar, resulting in stable SNN outputs (The imput currents typically follow the same distribution due the normalization layer). Therefore, we weight the current initial state by the smoothed state of the previous timestep with a layer-shared smoothing coefficient a' to reduce the difference between the two. The smoothed membrane potential (t) receives the input current from the previous layer, which then generates spikes and resets the membrane potential, iterating to the next timestep. The membrane potential smoothing and the charge dynamics a spiking neuron can be expressed as:
(t) a' ( - 1) + (1 (t), smoothing (!(1) A:(0 + chhrree
(4)
(5)
The smoothing coefficient a and the parameter 0 of the SNN are co-optimized during training to achieve the optimal smoothing effect. To ensure that a' E (0. 1). in the practical implementation we train the parameter 3 and let a' sigmoid(33), By default, S is initialized to 0, i.e. the initial value of a 0.5. In Section 4.3, we will analyze the influence of the initial value of of on the performance and convergence. Since the spike activity (Fq. 2) is not differentiable, we use the rectangular function (Wu et al., 2018) to calculate the spike derivative:
(t) (h(H 9) sign H) 01 -)
(6)
(!)
(H!(1)
where is the hyperparameter that controls the shape of the rectangular function and is set to 1.0. Accordingly, the derivative a spike with respect to a' can be calculated as:
(t)(() (t) (H! oit((t) sign a
0| x -)
0 (t) (t Oa
The derivative of the loss function L with respect to a' can be calculated as:
T dc as ac = -Σ
(8)
0 (S d
5

---

Published as a conference paper at ICLR 2025
where is the number of neurons in layer 1.
worth noting that addition to mitigating the difference membrane potential distribution, membrane potential smoothing
also facilitate the propagation of the gradient in the temporal dimension. Previous studies have shown that in SNNs, the temporal gradient is a small percent- age (Meng et al., 2023) and is highly susceptible to gradient vanishing, leading performance degradation (Huang et al., 2024). As shown in Fig. 2, our method establishes a forward information transfer pathway from (1 - 1) to (())). and also propagates the error gradient in the backward direction, thus mitigating the influence of the temporal gradient vanishing (Detailed theoretical anal- ysis is provided in Appendix A.3). From another perspective, the additional pathways can be viewed as residual connections in the temporal dimension, facilitating the propagation of information and gradients.
Notably, membrane potential smoothing is integrated with LIF neurons in this paper (see Ap- pendix A.2 for complete neuron dynamics), but this smoothing method is not limited to specific neuron types and can be integrated with other pre-existing neurons to further improve performance (See Table 3).
3.3 TEMPORALLY ADIACENT SUBNETWORK GUIDANCE
Membrane potential smoothing aims to pull together the initial states of the subnetworks. addi- tion, we propose the temporally adjacent subnetwork guidance to further promote the output stability of these subnetworks and improve the ensemble performance.
Inspired by knowledge distillation (Hinton, 2015), we guide the output by identifying the "teacher" and "student" from two temporally adjacent subnetworks. Since spiking neurons need to accumulate membrane potentials before they can produce stable spiking outputs, treat the early timestep subnetwork as a weak "student" that is guided by the stable "teacher" with later timestep. Taking example, the output probabilities are first calculated based
the 1-th and + 1-th subnetworks on the logits O₂ and O++1 of the two subnetworks, respectively:
Oug/TRL
p(1) Ee_ OralTKe pll +1
E o
where C' denotes the C-way classification task, the subscript j indicates the j-th class, and TkL is the temperature hyperparameter set 2. We then KL divergence to encourage the output probability of subnetwork t to be as similar as possible to the output probability of subnetwork & + (For the regression task of predicting continuous values, we compute the output MSE loss directly, as in the object detection task in Appendix A.7):
L. TKL (T(pKK+(()))llllll TKL Ep(t+ 1),log(
p(l 1)5 p(t);
(10)
By performing guidance between each pair of temporally adjacent subnetworks, we obtain a total of T 1 guidance losses: {L1, C..... -r- Instead of accumulating all these losses directly, we keep the largest one and drop the others with probability P, as in (Sariyildiz et al., 2024). This promotes consistency while preserving diversity among subnetworks, rather than eliminating differences altogether, thus ensuring generalization of the ensemble. In this paper, P is set to 0.5.
We define the function that selects the largest loss and randomly discards the others as drop(.) (see Appendix A.4 for pseudocode). During training, drop( (Li C2, -,, and cross-entropy loss LCE synergistically train the SNN:
Liotal = Gggiddance + ndrooprpp((Ls ( LCE
+
(11)
where y is the coefficient for controlling the guidance loss, which is set to 1.0 by default.
In this way, we synergistically increase the stability of the ensemble at both the level of the initial state (membrane potential) and the output of the subnetwork, greatly improving the overall perfor- mance of the SNN. We show the pseudocode for the training process of the temporally adjacent subnetwork guidance in Algorithm

---

Published as a conference paper at ICLR 2025
Algorithm Temporally adjacent subnetwork guidance for SNNs
Input: imput data æ, label Y.
Parameter: timestep T' Guidance loss coefficient 7,
Output: Trained SNN.
1: Initialize SNN f(-) with parameters
2: // Optimization over multiple iterations
3: for 1,2, ... Atrain iterations do do O, = =f(0; xxt)) // Calculate the output at timestep
4: for 1,,2
5:
6: if/ - = KL(O,-1: O²) // Calculate the guidance loss by Eq. 10
then
7:
8:
end if
9:
end for
O Lt=1 Os Calculate the ensemble average output
11: LCe = Cross-entropy(
Calculate the cross-entropy loss 12: Lquidance = drop( [C1, L2, Cr--)) // Random drop guidance loss
13: Stotal guiidance + Loe // Calculate total loss by Eq. 11
14:
Backpropagation and optimize model parameters
15:
end for
16:
return Trained SNN.
T-o Initialized to
T-1 (0.0010.10.0.155
T-2 (gud)=(0.000 (0.03,0.18)
(p.0)=(0.05,0117
Tw4 (0.0)-00.0330.10
Membrane potential
Membrane potential
Membrane potential
Hembrane potential
Membrane potential
T-0 Initialized
Tw1 (p.o0=(0.00.0.1000)
T-2 (p.0)=(0.04,0...
(0.00.00005001133)
Tw4 (0..0-(0.04..11))))
Mombrane potantial
Mombrane potential
Membrane
Membrane potential
Membrana potantial
Figure 3: Visualization of the membrane potential distribution before (top) and after (bottom) smoothing. Smoothing reduces distribution differences, especially for 71 T2 T3.
EXPERIMENTS
We have conducted extensive experiments with various architectures (RNN, VGG, ResNet, Trans- former, PointNet+++) on neuromorphic speech (1D), object (2D) recognition, and 3D point cloud classification tasks. Please see the Appendix A.5 for detailed experimental setup.
4.1 ABLATION STUDY
Table 2: Ablation study results of the proposed method (%). SpikingResformer
We perform ablation studies using the VGG-9, ResNet-18. and Spik- ingResfiormer (Shi et al., 2024) architectures on the neuromorphic object recognition benchmark datasets CIFARIO-DVS (Li al., 2017) and DVS-Gesture (Amir al., 7). The results in Table show that excessive differences in membrane potentials across timesteps are prevalent across different architectures and datasets, and
Dataset
Method
VGG-9
ResNet-18
Baseline
73.97
66.73
77.60
+Smooth
74.80
68.07+1.34
78. 78.45+0.85
CIFARIO-DVS
+Guidance
76.23
69.33
79. 79.23
+Both
76.77_2..0
70.03
80.60
Buseline 87.85
80.56
90.63
+Smooth
89.93
82.99+2.44
91.32+0.69
DVS-Gesture
+Guidance
91.32.+4 47.32
84. 84.8444..2
48.44.28
93.06
+Both 93.23
85.30
94.44 +3.61
that solution consistently mitigates this problem. The visualization of the membrane potential distribution before and after smoothing is shown in Fig. 3, and it can be seen that smoothing does indeed reduce the distribution difference.

---

Published as a conference paper at ICLR 2025
The output of the vanilla SNN varies greatly across timesteps, and the overall output is confusing, making it
Figure 4: awo-dimensional t-SNE visualization on the CIFARIO-DV' dataset. Top:
difficult to distinguish between classes. Bottom: The output of our SNN is more stable across timesteps and more distinguishable across classes, especially for the first two timesteps. Layer
Layer
1ayer
Layer
shisa
whodu
vlock
Trastion
Serstce
10 Strecot Layer
Layer
Layer
Layer
120 2000 1000 2000 2000
200 200
300 1000 080 000 1000 000 10C 2008 200 200
200 100 140
Figure 5: The optimization trend of a during training. with different initial values gradually converge with training iterations, indicating that our method is insensitive to the initial value of a.
In addition, the generalizability experi- ments with membrane potential smooth- al., 2021b) and
Table 3:
Generalizability experiments (%) of mem- brane potential smoothing different neurons.
ing for PLIF (Fang CLIF (Huang et al., 2024) (on CIFAR10- DVS) as well as for RNN-LIF and DH- LIF (Zheng et al., 2024) (on the neuromor- phic speech dataset SHD (Cramer et al., 2022)) are shown in Table 3. In particular, we reduce rather than eliminate the differences be- tween subnetworks, preserving the temporal properties of SNNs and thus providing performance gains even in time-dependent speech recognition tasks. Additional ablation experimental results and analysis are presented in Appendix A.6.
PLIF
CLIF
RNN-LIF DH-LIF
Original
74.83
74.97
81.87
89.86
+Smooth
75. 75.10+0.27 75.97+1.00
83.04+117
90.33+0.47
4.2 OUTPUT VISUALIZATION
We have visualized the output of the SNN in Fig. 4 with 2D t-distributed stochastic neighbor embed- ding (t-SNE) show the improvement in output stability and distinguishability with our method. In Fig. 4(Top), the output of the vanilla SNN at each individual timestep varies widely, and the output of the first two timesteps in particular confusing. This results in poor distinguishability of its ensemble average output, which limits the recognition performance. Our SNN on the other hand, has more stable outputs across timesteps, and in particular the outputs generated at the first two timesteps are also well distinguished, as shown in Fig. 4(Bottom). Benefiting from the stability across timesteps, our final output is more spread across different classes of clusters and more com- pactly distributed within the clusters, yielding better performance. Please see Appendix A.10 for more visualization comparisons.
4.3 SMOOTHING COEFFICIENT ANALYSIS
By default, a is initialized to 0.5. To analyze the influence of the initialization value on a, we visualize the optimization trend of in each layer of VGG-9 trained on CIFAR10-DYY in Fig. 5,
8

---

Published as a conference paper at ICLR 2025
(Nrd 500
77:00
Vauslaa SNNN
70.33
Accuncty (3))
76,00
Nccuncy (N2
10.00
Aceunny
14.00
14.00
71.93
24.00
34.50
03 o2 000010
01050702000 10 1415
85
1.5
(a) Timestep
(b) Drop probability
(c) Coefficient 7
(d) Temperature
Figure 6: Influence of hyperparameters on performance. (a) Our method consistently outperforms
vanilla SNN across different timesteps. (b) (c) Our method is insensitive to the drop probability of
the guidance loss and the coefficient y. (d) Our method is insensitive to the temperature hyperpa-
rameter within a reasonable range (TKL 2 1).
Table 4: Comparative results on neuromorphic datasets. *: self-implemennttioon results with open- source code. 1; knowledge transfer from static data. +: Data augmentation.
Dataset
Method
Architecture
T
Accuracy (%)
Ternary Spike (Guo et al., 2024)
ResNet-20
10
78.70
SLTT (Meng et al., 2023)
VGG-11
10
77.17
NDOT (Jiang et al., 2024a)
VGG-11
10
77.50
SSNN (Ding et al., 2024)
VGG-9
73.63
SLT (Anumasa et al., 2024)
VGG-9
5
74.23*
CIFARIO-DVS
CLIF (Huang et al., 2024)
VGG-9
5
74.97'
SpikingResformer (Shi et al., 2024)
SpikingRestfomerr
5
77.60"
SDT (Yao al., 2023)
Spiking Transformerr2-256
5
72.53*
TRR (Zuo et al., 2024b)
Spiking Transformer-2-256
75.55
VGG-9
76.77
Ours
SpikingResformerIIi
5
80,60
VGGSNN
4
83,20*
SSNN (Ding et al., 2024)
VGG-9
5
90.74
TRR (Zuo et al., 2024b)
VGG-9
5
91.67
SLT (Anumasa al., 2024)
VGG-9
5
89.35'
DVS-Gesture
SpikingResformer (Shi al., 2024)
SpikingRestfomerr-ii
5
90.63*
SDT (Yao al., 2023)
Spiking Transformer-2-251
92.24 5
VGG-9
93.23
Ours
SpikingRestormer-II
94.44 5
TCIA-TET-SNN (Zhu et al., 2024)
CombinedSNN
82.50 14
EventMix (Shen et al., 2023)
ResNet-18
10
79.47
(Shen et al., 2024)
Spikformer
79.00 10
NDA (Li et al., 2022)
VGG-11
78.20 10
N-Caltech101
Knowledge-Trame Transfer (He al., 2024)
VGGSNN
91.72*t 10
SSNN (Ding et al., 2024)
VGG-9
5
77.97
TEBN (Duan et al., 2022)
VGG-9
5
81,24
VGG-9
5
82.71
Ours
VGGSNN
10
93.681
where the initial values a are taken from (0.2,0.35,005,,066500033 As can seen in Fig. 5, a gradually converges at each layer during the training iterations. In particular, in the last four layers gradually converges to almost the same optimal value, indicating that our method is robust to initial values. The first few layers of do not converge to the same optimal value, which attribute to the accumulation of errors in the surrogate gradient. The SNN uses the surrogate gradient instead of the derivative of the spike activity, which introduces gradient error in backpropagation, and this error accumulates the further ahead the layer is. Eventually, these gradient errors cause the parameters of the earlier layers be underoptimized, optimal value. However, compared to the 73.97% accuracy of the vanilla SNN, these different a initialization values achieved average accuracies 76.20%, 76.80% 76.77%, 75.75%, and 76.85%, respectively, both of which provide significant performance gains.
that a does not converge to the same
4.4 INFLUENCE OF HYPERPARAMETERS
To explore the influence of typerparameters on the performance of the proposed method, we show in Fig. 6 the performance of VGG-9
SIFARIO-DY. with different hyperparameter settings. We increased the timestep from 3 to and found that the overall performance of the model also gradually increased and then saturated, and the average accuracy reached maximum of 77.4% when the timestep was 7, as shown in Fig. 6(a). Compared to the vanilla SNN, our method consistently shows better performance.

---

Published as a conference paper at ICLR 2025
The influence of the drop probability of the guidance loss and the coefficient on the performance is shown in Fig. 6(b) and Fig. 6(c), and the results show that our method is not sensitive to these hyperparameters and consistently outperforms the vanilla SNN. This indicates that method can significantly improve model performance without intentionally adjusting the hyperparameters.
In Fig. 6(d), investigate the influence of the temperature hyperparameter Tx L in the guidance on the performance, taking values set to {0.5, 1, 1,2.3, 2, 4). The results show that the performance of our method fluctuates only slightly when TKy. > and degrades when TKy. = 0.5 (still outperforming the vanilla SNN). We argue that this is due to the fact that too small a TK causes the softened subnetwork logit to be too sharp. making it difficult to pull together subnetwork outputs that are already too dissimilar. These experiments show that our method is not sensitive to specific values as long as the hyperparameters are within reasonable ranges, and thus offers great robustness.
4.5
COMPARISON WITH EXISTING METHODS
1D neuromorphic speech recogni- tion. Time-dependent speech recog- nition tasks require models with com- plex temporal processing capabilities, so stability across timesteps is ex- tremely important. The comparative results with other SNNs the SHD dataset are shown in Table 5. Our method achieves a recognition accuracy of 91.19% with lightweight RNN architecture, outper- forming other methods. This confirms the effectiveness of our method on long time sequences.
Table 5: Comparative results on the ID SHD dataset.
Method
Architecture Param (M)
Acc. (%)
Origin (Cramer et al., 2022)
LSTM
0.43
89.20
TIM (Shen et al., 2024)
Spikformer
86.30
TC-LIF (Zhang al., 2024a)
Recurrent
0.39
88.91"
DH-LIF (Zheng et al., 2024)
Recurrent
0.39
89.86
Ours
Recurrent
0.39
91.19
2D neuromorphic object recognition. As shown in Table 4, VGG-9 achieves accuracy of 76.77% and 93.23% on CIFARIO-DVS and DVS-Gesture, respectively, with only 5 timesteps. Using the Transformer architecture, we achieved accuracies of 80.60% and 94.44%, respectively, significantly exceeding other methods. Specifically, with data augmentation, we were able to achieve 83.20% accuracy on the CIFAR on N-Caltech101 (Orchard et al., 2015) and achieved 82.71% accuracy using VGG-9. Note that we only used vanilla LIF neurons and the general training method, which can further improve the performance when combined with other methods. For example, employing the knowledge transfer strategy (He et al., 2024), we were able to increase our accuracy to 93.68%.
with 4 timesteps. In addition, we conducted experiments
3D point cloud classification. For the challenging point cloud classification, we performed ex- periments on the 1e0/0N datasets (Wu et al., 2015) us- ing the lightweight PointNet++ architecture (Qi et al., 2017),
Table 6: Comparative results (%) on point cloud classification.
Method
Type
ModelNet10 ModelNet40
PointNet++ (Qi et al., 2017)
ANN
95.50
92.16
Converted SNN (Lan et al., 2023)
SNN 16
92.75
89.45
Spiking PointNet (Ren et al., 2023)
SNN
2
92.98
88.46
P2SResL.Ntt (Wu et al., 2024)
SNN
89.20
2
94.54
91.13
Ours
SNN
and the comparative results are shown in Table With T 2, our method achieves 94.54% and 91.13% accuracy, respectively, outperforming other SNN mod- els. To evaluate the oie-timestep inference performance, we directly infer the trained -vo-timestep model with one timestep. This preserves the benefits of our method while avoiding additional train- ing overhead. The results show that even with only one timestep, we can achieve accuracies of 94.39% and 89.82%, which still outperform other SNNs.
94.39
89.82
CONCLUSION
In this paper, we highlight a key factor that is generally overlooked in SNNs: excessive differences in membrane potential distributions can lead to unstable outputs across timesteps, thereby affecting performance. To mitigate this, we propose membrane potential smoothing and temporally adjacent subnetwork guidance to facilitate consistency of initial membrane potentials and outputs, respec- tively, thereby improving stability and performance. Meanwhile, membrane potential smoothing also facilitates the propagation of forward information and backward gradients, mitigating the tem- poral gradient vanishing and providing dual gains. Extensive experiments on neuromorphic speech (1D)object (2D) recognition and 3D point cloud classification tasks confirmed the effectiveness and versatility of our method. We expect that our work will inspire the community to further analyze the spatio-temporal properties of SNNs.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China under Grant No. 62276054 and 62406060.
REFERENCES
Muhammad Ammar Ali, Yusuf Sahin, Süreyya Özögür Akyüz, Gozde Unal, and Buse Cisil Otar. Tuning dicuracy-diversity trade-off in neural network ensemble via novel entropy loss function. In 202/ 13th International Conference on Electrical and Electronics Engineering (ELECO), pp- 365-368, 2021.
Zeyuan Allen-Zhi and Yuanzhi Li. Towards understanding ensemble, knowledge distillation and self-distillation in deep learning, arXiv preprint arXiv:2012.09816 2020.
Arnon Amir et al. A low power, fully event-based gesture recognition system. In 2017 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 7388 7397,7397, 2017.
Srinivas Anumasa, Bhaskar Mukhoty, Velibor Bojkovic, Giulia De Masi, Huan Xiong, and Bin Gu. Enhancing training of spiking neural network with stochastic latency. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 10900-10908, 2024.
Tong Bu, Wei Fang, Jianhao Ding, PENGLIN DAI, Zhaofei Yu, and Tiejun Huang. Optimal ANN- SNN conversion for high-accuracy and low-latency spiking neural networks. In Interna- tional Conference on Learning Representations, 2022.
Biswadeep Chakraborty, Beomseok Kang, Harshit Kumar, and Saibal Mukhopadhyay. Sparse spik- ing neural network: Exploiting heterogeneity in timescales for pruning recurrent SNN. In The Twelfth International Conference Learning Representations, 2024.
Benjamin Cramer, Yannik Stradmann, Johannes Schemmel, and Friedemann Zenke. The heidelberg spiking data for the systematic evaluation of spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, 33(7):2744-2757, 2022.
Shikuang Deng and Shi Gu. Optimal conversion of conventional artificial neural networks to spiking neural networks. In International Conference on Learning Representations, 2021.
Yongqi Ding, Lin Zuo, Kunshan Yang, Zhongshu Chen, Jian Hu, and Tangfan Xiahou. An improved probabilistic spiking neural network with enhanced discriminative ability. Knowledge-Bosse -ased Sys- fems, 280:1 1024, 2023.
Yongqi Ding, Lin Zuo, Mengmeng Jing, Pei He, and Yongjun Xiao. Shrinking your timestep: To- wards low-latency ncuromorphic object recognition with spiking neural networks. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 11811-11819, 2024.
Yiting Dong, Dongcheng Zhao, and Yi Zeng. Temporal knowledge sharing enable spiking neural network learning from past and future. IEEE Transactions Artificial Intelligence, 5(7):3524 3534, 2024.
Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred Hamprecht. Essentially no barriers in neural network energy landscape. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 131991318. PMLR, 10-15 Jul 2018.
Chaoteng Duan, Jianhao Ding, Shiyan Chen, Zhaofei Yu, and Tiejun Huang. Temporal effective batch normalization in spiking neural networks. In Alice H. Oh, Alekh Agarwal, Danielle Bel- grave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
Daniel Elbrecht, Shruti R. Kulkarni, Maryam Parsa, J. Parker Mitchell, and Catherine D. Schu- man. Evolving ensembles of spiking neural networks for neuromorphic systems. In 2020 JEEE Symposium Series Computational Intelligence (SSCI), pp. 1989-1994 2020.

---

Published as a conference paper at ICLR 2025
Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems, 34:21056-21066 2021a.
Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian. In- corporating learnable membrane time constant to enhance learning of spiking neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2661- 2671, October 2021b.
Wei Fang. Yanqi Chen, Jianhao Ding. Zhaofei Yu, Timothée Masquelier, Ding Chen, Liwei Huang, Huihui Zhou, Guoqi Li, and Yonghong Tian. Spikingjelly: An open-source machine learning infrastructure platform for spike-based intelligence. Science Advances, (40) 1480, 2023a.
Wei Fang, Zhaofei Yu, Zhaokun Zhou, Ding Chen, Yanqi Chen, Zhengyu Ma, Timothée Masquelier, and Yonghong Tian. Parallel spiking neurons with high efficiency and ability to learn long-term dependencies. Advances in Neural Information Processing Systems, 36, 2023b.
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Linear mode con- nectivity and the lottery ticket hypothesis. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine PMLR, 13-18 Jul 2020.
Learning Research, pp.
Tommaso Furlanello, Zachary Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Born again neural networks. In International conference on machine learning, pp. 1607-1616. PMLR, 2018
Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. BesaaBBincch,, and R. Garnett (eds.), Advances in Neural Infor- mation Processing Systems, volume 31. Curran Associates, Inc., 2018
Yufei Guo, Xinyi Tong, Yuanpei Chen, Liwen Zhang, Xiaode Liu, Zhe Ma, and Xuhui Huang- Recdis-snn: Rectifying membrane potential distribution for directly training spiking neural net- works. Proceedings of the IEEE/CVV Conference on Computer Vision and Pattern Recognition (CVPR), pp. 326-335, June 2022.
Yufei Guo, Xiaode Liu, Yuanpei Chen, Liwen Zhang, Weihang Peng, Yuhan Zhang, Xuhui Huang, and Zhe Ma. Rmp-loss: Regularizing membrane potential distribution for spiking neural net- works. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 17391-17401, October 2023a.
Yufei Guo, Weihang Peng, Yuanpei Chen, Liwen Zhang, Xiaode Liu, Xuhui Huang, and Zhe Ma. Joint a-snn: Joint training of artificial and spiking neural networks via self self-distillation and weight factorization. Pattern Recognition, 142:109639 2023b.
Yufei Guo, Yuanpei Chen, Xiaode Liu, Weihang Peng, Yuhan Zhang, Xuhui Huang, and Zhe Ma. Ternary spike: Learning ternary spikes for spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 12244-12553 2024.
Xiang He, Dongcheng Zhao, Yang Li, Guobin Shen, Qingqun Kong, and Yi Zeng. An efficient knowledge transfer strategy for spiking neural networks from static to event domain. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 512-520, 2024.
Geoffrey Hinton. 2015.
Distilling the knowledge in neural network. arXiv preprint arXiv:1503.02531,
JiaKui Hu, Man Yao, Xuerui Qiu, Yuhong Chou, Yuxuan Cai, Ning Qiao, Yonghong Tian, Bo XU, and Guoqi Li. High-performance temporal reversible spiking neural networks with S mathhal(())(IS training memory and S\ mathcal (Oathaal(O))(()) (1) inference cost. In Forty-first Inter- national Conference on Machine Learning, 2024.
Yulong Huang, Xiaopeng LIN, Hongwei Ren, Haotian FU, Yue Zhou, Zunchang LIU, biao pan, and Bojun Cheng. CLIF: Complementary leaky integrate-and fnegrate-and-ffe In Forty-first International Conference Machine Learning, 2024.
and--ind--f fire neuron for spiking neural networks.
12

---

Published as a conference paper at ICLR 2025
Somaych Hussaini, Michael Milford, and Tobias Fischer. Ensembles of compact, region-specific & regularized spiking neural networks for scalable place recognition. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. +200-4201 2023.
Samyak Jain, Sravanti Addepalli, Pawan Kumar Sahu, Priyam Dey, and R. Venkatesh Babu. Dart: Diversify-agrregat--rpp training improves generalization of neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pultern Recognition (CVPR), pp. 16048- 16059, June 2023.
Haiyan Jiang, Giulia De Masi, Huan Xiong, and Bin Gu. NDOT: Neuronal dynamics-based on- line training for spiking neural networks. In Forty-first International Conference on Machine Learning. 2024a.
Haiyan Jiang, Vincent Zoonekynd, Giulia De Masi, Bin Gu, and Huan Xiong. TAB: Temporal accu- mulated batch normalization in spiking neural networks. In The Twelfth International Conference on Learning Representations, 2024b.
kang you, Zekai Xu, Chen Nie, Zhijic Deng, Qinghai Guo, Xiang Wang, and Zhezhi He. SpikeZIP- TF: Conversion is all you need for transformer-basee SNN. In Forty-first International Conference Machine Learning, 2024.
Seijoon Kim, Seongsik Park, Byunggook Na, and Sungroh Yoon. Spiking-yolo: spiking neural network for energy-efficient object detection. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 11270-11277, 2020.
Anders Krogh and Jesper Vedelsby. Neural network ensembles, cross validation, and active learning- In G. Tesauro, D. Touretzky, and T. Leen (eds.), Advances in Neural Information Processing Systems, volume 7. MIT Press, 1994.
Yuxiang Lan, Yachao Zhang, Xu Ma, Yanyun Qu, and Yun Fu. Efficient converted spiking neural network for 3d and 2d classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9211 1-9220, October 2023.
Jun Ho Lee, Jae Soon Baik, Tae Hwan Hwang, and Jun Won Choi. Learning from data with noisy labels using temporal self-ensemble. arXiv preprint arXiv:2207.10335 2022.
Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. 10-vr: An event-stream dataset for object classification. Frontiers in Neuroscience, 11, 2017.
Yuhang Li, Youngeun Kim, Hyoungseob Park, Tamar Geller, and Priyadarshini Panda. Neuro- morphic data augmentation for training spiking neural networks. In European Conference Computer Vision, pp. 631-649. Springer, 2022.
Xinhao Luo, Man Yao, Yuhong Chou, Bo Xu, and Guoqi Li. Integer-valued training and spike- driven inference spiking neural network for nigh-peeffrmmanc and energy-efficiee object detec- tion. arXiv preprint 20Xiv:240..207088 2024.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models. Neural networks, 10(9):155-11611 1997.
Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang, Zhouchen Lin, and Zhi-Quan Luo. Towards memory- and time-efficient backpropagation for training spiking neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6166 6176, October 2023.
Georgiana Neculae, Oliver Rhodes, and Gavin Brown. Ensembles of spiking neural networks, 2021.
Quynh Nguyen. On connected sublevel sets in deep learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov (cds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 4790-4799. PMLR, 09-15 Jun 2019.
Garrick Orchard, Ajinkya Jayawant, Gregory K. Cohen, and Nitish Thakor. Converting static image datasets to spiking neuromorphic datasets using saccades. Frontiers in Neuroscience, 9, 2015.
13

---

Published as a conference paper at ICLR 2025
Priyadarshini Panda, Gopalakrishnan Srinivasan, and Kaushik Roy. Ensemblesnn: Distributed as- sistive stdp learning for energy-efficient recognition in spiking neural networks. In 2017 Interna- tional Joint Conference on Neural Networks (IJCNN). pp.
2017.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical fea- ture learning on point sets in metric space. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Haonan Qiu, Zeyin Song, Yanqi Chen, Munan Ning, Wei Fang, Tao Sun, Zhengyu Ma, Li Yuan, and Yonghong Tian. Temporal contrastive learning for spiking neural networks. In Artificial Neural Networks and Machine Learning ICANN 2024, pp. 422-436, 2024.
Dayong Ren, Zhe Ma, Yuanpei Chen, Weihang Peng. Xiaode Liu, Yuhan Zhang, and Yufei Guo. Spiking pointnet: Spiking neural networks for point clouds. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Lior Rokach. Ensemble-based classifiers. Artificial intelligence review, 33:1-39, 2010.
Yangjun Ruan, Saurabh Singh, Warren Richard Morningstar, Alexander A. Alemi, Sergey Ioffe, Ian Fischer, and Joshua V. Dillon. Weighted ensemble self-supervised learning. In ICLR, 2023.
Bodo Rueckauer, Iulia-Ales Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih- Chii- Liu. Con- version of continuous- deep networks efficient event-driven networks for image classifi- cation. Frontiers in neuroscience, 1 11:682, 2017.
Mert Bulent Sariyildiz, Philippe Weinzaepfel, Thomas Lucas, Diane Larlus, and Yannis Kalantidis. Unic: Universal classification models via multi-teacher distillation, 2024.
Guobin Shen, Dongcheng Zhao, and Yi Zeng. Eventmix: An efficient data augmentation strategy for event-based learning. Information Sciences, 644: 114:7, 2023.
Sicheng Shen, Dongcheng Zhao, Guobin Shen, and Yi Zeng. Tim: An efficient temporal interaction module for spiking transformer, 2024.
Xinyu Shi, Zecheng Hao, and Zhaofei Yu. Spikingresformer: Bridging resnet and vision transformer in spiking neural networks. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Puttern Recognition (CVPR), pp. 5610-5619, June 2024.
Qiaoyi Su, Yuhong Chou, Yifan Hu, Jianing Li, Shijie Mei, Ziyang Zhang, and Guoqi Li. Deep directly-trained spiking neural networks for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.
October 2023.
Mohammad Reza Taesiri, Giang Nguyen, Sarra Habchi, Cor-Paul Bezemer, and Anh Totti Nguyen. Imagenet-hard: The hardest images remaining from study of the power of zoom and spatial biases in image classification. In sevitty-eventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
Luke Taylor, Andrew J King, and Nicol Spencer Harper. Addressing the speed-accuracy simula- tion trade-off for adaptive spiking neurons. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Linh Tran, Bastiaan S Veeling, Kevin Roth, Jakub Swiatkowski, Joshua V Dillon, Jasper Snoek, Stephan Mandt, Tim Salimans, Sebastian Nowozin, and Rodolphe Jenatton. Hydra: Preserving ensemble diversity for model distillation. arXiv preprint arXiv:2001. 044694, 2020.
Lihao Wang and Zhaofei Yu. Autaptic synaptic eircuit enhances spatio-temporal predictive learning of spiking neural networks. In Forty-first International Conference on Machine Learning, 2024.
Ziming Wang, Runhao Jiang, Shuang Lian, Rui Yan, and Huajin Tang. Adaptive smoothing gra- dient learning for spiking neural networks. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th In- ternational Conference on Machine Learning, volume 202 of Proceedings of Machine Learning PMLR, 23-29 Jul 2023.
Research, pp.
14

---

Published as a conference paper at ICLR 2025
Qiaoyun Wu, Quanxiao Zhang, Chunyu Tan, Yun Zhou, and Changyin Sun. Point-to-spie toskk residual learning for energy-efficient 3d point cloud classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 6092-6099, 2024.
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in neuroscience, 12:331, 2018.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2015.
Qu Yang, Jibin Wu, Malu Zhang, Yansong Chua, Xinchao Wang, and Haizhou Li. Training spiking neural networks with local tandem learning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
Man Yao, JiaKui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo XU, and Guoqi Li. Spike-driven transformer. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Man Yao, JiaKui Hu, Tianxiang Hu, Yifan Xu, Zhaokun Zhou, Yonghong Tian, Bo XU, and Guoqi Li. Spike-driven transformer v2: Meta spiking neural network architecture inspiring the design of next-generation neuromorphic chips. In The Twelfth International Conference Learning Representations, 2024.
Shaofeng Zhang, Meng Liu, and Junchi Yan. The diversified ensemble neural network. In
H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Infor- mation Processing Systems, volume 33, pp. 16001-16011. Curran Associates, Inc., 2020.
Shimin Zhang, Qu Yang, Chenxiang Ma, Jibin Wu, Haizhou Li, and Kay Chen Tan. Te-lif: A two- compartment spiking neuron model for long-term sequential modelling. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 16838-16847, 2024a.
Yuhan Zhang, Xiaode Liu, Yuanpei Chen, Weihang Peng, Yufei Guo. Xuhui Huang, and Zhe Ma. Enhancing representation of spiking neural networks via similarity contrastive learning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 16926-16934, 2024b.
Hanle Zheng, Zhong Zheng, Rui Hu, Bo Xiao, Yujie Wu, Fangwen Yu, Xue Liu, Guoqi Li, and Lei Deng. Temporal dendritic heterogeneity incorporated with spiking neural networks for learning multi-timescale dynamics. Nature Communications, 15(1):277, 2024.
Zhaokun Zhou, Yuesheng Zhu, Chao He, Yaowei Wang, Shuicheng YAN, Yonghong Tian, and Li Yuan. Spikformer: When spiking neural network meets transformer. In The Eleventh Interna- tional Conference on Learning Representations, 2023.
Rui-Jie Zhu, Malu Zhang, Qihang Zhao, Haoyu Deng, Yule Duan, and Liang-Jian Deng. Teja- snn: comporal-channel joint attention for spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, pp. 1-14, 2024.
Lin Zuo, Yongqi Ding, Mengmeng Jing, Kunshan Yang, and Yunqian Yu. Self-distillation learning based on temporal-spatial consistency for spiking neural networks. arXiv preprint arXiv:2406.00 07862, 2024a.
Lin Zuo, Yongqi Ding, Wenwei Luo, Mengmeng Jing, and Kunshan Yang. Temporal reversal reg- ularization for spiking neural networks: Hybrid spatio-temporal invariance for generalization. arXiv preprint 49Xiv:2408.09 09100, 2024b.
15

---

Published as a conference paper at ICLR 2025
A APPENDIX
The Appendix section provides detailed theoretical analysis, neural dynamics, and additional exper- iments described below:
Appendix A.1 details and experimentally confirms the necessity to reduce the variance across timesteps in SNNs.
Appendix A.2 shows the dynamics of LIF neurons with integrated membrane potential smoothing.
Appendix A.3 analyzes the effectiveness of membrane potential smoothing to mitigate the temporal gradient vanishing.
Appendix A.4 provides pseudo-code for randomly dropping guidance losses and tempo- rally adjacent subnetwork guidance.
Appendix A.5 provides experimental details such as the dataset and training setup.
Appendix A.6 shows the additional ablation studies and analysis of method.
Appendix A.7 demonstrates the application of the temporally adjacent subnetwork guid- ance to the object detection task.
Appendix A.8 presents the analysis of the effectiveness and power consumption our method for the spiking Transformer model with different timesteps.
Appendix A.9 provides additional experiments on the static image datasets.
Appendix A.10 shows additional visualizations to demonstrate the effectiveness our method in promoting consistency in the membrane potential distribution.
16

---

Published as a conference paper at ICLR 2025
A.1 THE NECESSITY TO REDUCE -IOSSTT DIFFERENCES IN SNNS
Previous research on ensemble learning has shown that increasing the diversity of members in an ensemble contributes to overall generalization under certain constraints (Krogh & Vedelsby, 1994; Frankle et al., 2020; Zhang et al., 2020). In contrast, we point out that the differences in the output of the SNN across timesteps should be reduced. which first glance seems to contradict the idea of ensemble learning. In this section, we will elucidate the necessity to reduce the differences in SNNs across timesteps by analyzing the constraints of ensemble learning and experimental results.
Previous studies have shown that the solutions learned by Stochastic Gradient Descent (SGD) lie on a nonlinear manifold (Draxler et al., 2018; Garipov et al., 2018; Nguyen. 2019) and that converged models with common initial optimization path are linearly connected with a low-loss barrier (Fran- kle al., 2020). Multiple optimal solutions that are linearly connected belong to common basin that is separated from other regions of the loss landscape (Jain et al., 2023). The linearly connected multiple models in a common basin can thus be ensembled for additional gains. From this we derive constraint that is important for ensemble, but typically overlooked: The members of the ensem- ble should be converged optimal solutions, or at the very least optimized feasible solutions. However, it is possible that particular temporal submodel in the SNN is not feasible solution. For example, the results in Table show that the SNN submodel is only 10% accurate at the first timestep on the ten-class CIFARIO-J RI-DDVS dataset, which is equivalent is far from an optimal or feasible solution. At this point, it has stepped outside the constraints of the ensemble member, so cannot continue to increase the diversity of its members as it would in normal practice. Instead, we can make the infeasible solution gradually converge to the feasible so- lution by decreasing the difference between this overly outlier submodel (or infeasible solution) and the other submodels (feasible solutions). Finally, these feasible solutions can be used as an effective ensemble to improve overall stability and performance.
randomly initialized network that
From another point of view, it is difficult to have both performance and diversity of tasks in an en- semble. To balance performance and diversity, existing methods use multiple losses during training to guide the optimization of the neural network ensemble (Ali et al., 2021; Zhang et al., 2020). Fol- lowing (Zhang et al., 2020), we divide the training loss of an ensemble into three parts (Take the classification task an example): i) the cross-entropy loss of ensemble members and labels; ii) the diversity of ensemble members; and iii) the aggregated loss of the ensemble.
The cross-entropy loss of ensemble members and labels can be calculated as
L. H((((T(X)), Y),
(12)
where X is the input, is the label, T; is the i-th member model, g(-) is the normalization function, and H is the cross entropy.
The diversity of ensemble members can be calculated as
f-=1-
1 N
Σ q (
(13)
ISIJIN
where N the number of members in the ensemble.
The aggregated loss of the ensemble can be calculated as
C.
H
K T(X), Y),
(14)
where n/s is the integration weight, bounded by Σ 9t =
The total loss during training is
Lensemble = Cg C. aLas
(15)
where a is the coefficient controlling the increase in diversity that can be adaptively updated in (Zhang et al., 2020) with training iterations.
From Eq. 15, we can see that ensemble learning cannot blindly increase the diversity of its members, but must strike balance between overall performance and diversity. Our method,
17

---

Published as a conference paper at ICLR 2025
Table 7: Comparative results (x10-3) of ensemble metrics on the CIFARIO-DVS dataset.
C. +
La t
La +
Lensemble
Vanilla SNN Ours
25.1526 16.2025 (4 35.58%)
60.0711
13.6560 12.2561 (4 10.25%)
-21.2622 -29.7778 (L 10.05%)
58.2364 (L 3.05%)
on the other hand, takes this into account by pointing out that excessive variance the SNN (over- diversity) can degrade overall performance, thus requiring appropriate reduction in diversity to achieve a better balance between performance and diversity.
To further confirm the effectiveness of method in promoting balance between overall perfor- mance and diversity, a comparison of the ensemble metrics on the CIFARIO-DVS dataset is pre- sented in Table The results show that although our method slightly reduces the diversity of the individual timestep submodels of the SNN, it achieves better performance in all other three metrics. In particular, our diversity degraded by only negligible 3.05% compared to the vanilla SNN, but the final integration loss is reduced by 40.05%, significantly improving the overall performance.
A.2 INTEGRATING MEMBRANE POTENTIAL. SMOOTHING IN LIF NEURONS
In this paper, membrane potential smoothing is integrated into commonly used LIF neurons. The neuron dynamics after integration can be expressed as:
Ul(t) = (1 1 (1-;) ;) H. (t - 1), leakage
(16)
(t) a' (t - 1) (1 a¹ (t), smoothing
(17)
H((t) Ht((t) - [tt), charge
(18)
S1(1) = {{ 1, H!()) 2
11(() 0 fire spike
(19)
H =(()) (())), reset
(20)
where H is the smoothed membrane potential. Note that H 1) does exist when t 0. at which point the dynamics of the neuron are the same as the original LIF dynamics. When = 1,
[t - 1) (0) is set to the initialized value of membrane potential H, which is in this paper.
18

---

Published as a conference paper at ICLR 2025
A.3 MEMBRANE POTENTIAL SMOOTHING MITIGATES TEMPORAL GRADIENT VANISHING
When training SNNs directly with the surrogate gradient, the gradient propagates backward in both spatial and temporal domains. Taking the vanilla LIF neuron as an example, we convert Eq. 1 to Eq. 3 into the following form to illustrate the temporal gradient vanishing problem:
H (1) = (1 - (H 1)
(L-
us' 1)) + w's
(21)
The gradient of the loss L with respect to the weights W over T'' timesteps is:
V -
ac s "' -= L, -,...
(22)
1=0 (H'(1)
where I. is the total number of layers in the SNN.
The derivative of the loss L with respect to the membrane potential (t) is: t-t
Oc ac (t)
T-1
Oc (t) (t)
II
c - tt",, 1- (23)
(t) as (t) OH (t)
OH
Dc ac
0H as
( as (t) (H) (s'(t) (H)
ac
Σ 1=#41
II
ce - --",t,, (<),
II,,
OH HH (t) as OH
(24)
where the blue portion on the left indicates the spatial gradient and the green portion on the right indicates the temporal gradient. (1) is defined as the sensitivity of the membrane potential H' (1 + 1) to (I) in adjacent timesteps (Meng et al., 2023; Huang et al., 2024):
c(t) & OH (1 1)
HH (L [[++1] 1) (S)
(25)
(t)
(t)
(t)
As can be seen in Eq. 25, the sensitivity e'(t) controls the percentage of the temporal gradi- ent in the total gradient. However, for typical surrogate gradient settings, the diagonal matrix
Ih ettt' - t") has only small spectral norm (Meng al., 2023). To illustrate, for the rect- angular function used in Eq. 6, when 0, the diagonal elements of (t) are
{()) = {
0, 10 < (t) < <
(26)
1
otherwise.
Typically, 1- 1 is set to values less than 1, such as 0.5 and 0.25 (Ding et al., 2024: Guo et al., 2022), which causes the diagonal values of the matrix II e'(''= (t") to become smaller f' increases, eventually causing the temporal gradient vanishing and degrading performance.
When membrane potential smoothing is used, the neuron each timestep is smoothed by the smoothed membrane potential of the previous timestep, and the modified LIF neuron dynamics are shown in Appendix A.2. When >> the sensitivity of the membrane potential timestep t + with respect to the membrane potential at timestep t can be calculated as:
0H'
(t + 1) oh (t + = (1 a))
É(A = 1) air + 1) (H)
(27)
When the timestep interval is 2. the sensitivity of the temporal gradient is:
z(At= = HH (t + 2)
at + 2) (t (t +1 of' + 2)t 1))
(t (t+ (H 0H'(++1) 1)
(28)
= =a(1
+1 (1- (1=))
By iterating over timestep according to the chain rule, the sensitivity of the temporal gradient can be obtained as
e(A1) (1 - (1) (1 -a) (1)
(29)
19

---

Published as a conference paper at ICLR 2025
Thus, the derivative of the loss L with respect to the membrane potential H' (t) from Eq. 23 and Eq. 24 becomes
ac OC as' (1) (1) (I)
ac (t'
+
(At=t -1,
(30)
(1) (H)
0c (t)
Oc 0H)+ 0S
Oc (H)) (S' (H) (s' (H
UH+ (t) (t) OH
1=H+1
(31)
It can be seen that the sensitivity of the temporal gradient in the neuron after smoothing the mem- brane potential becomes (At = - t) instead of Ih e'tt' -t"). If the time step interval is small, such then (Lt
t) is slightly lower than IT e'(t' -t", but this does not matter because the temporal gradient does not vanish at this point. If the time step interval is large, then the diagonal values in I et +") are greatly reduced, causing performance degradation. In contrast, (
can produce larger values at this point mitigate the temporal gradient vanishing. For example, if - 0.5, 0.25, and the timestep interval At is 3, the valid value of the diagonal element in I1 (t' t") is 0.125, while the valid value in (At = 1' -t) 0.146. When the timestep interval is increased to the difference between the two becomes 0.03125 versus 0.05722, at which point the sensitivity of the temporal gradient increases by 83%. This demonstrates the effectiveness membrane potential smoothing in mitigating the temporal gradient vanishing.
A.4 PYTORCH-STVE CODE
For reproducibility, we provide Pytorch-style code for the drop function drop(-), which randomly drops guidance losses the Algorithm 2 inspired by (Sariyildiz al., 2024).
Algorithm 2 PyTorch-style code for randomly dropping guidance losses
lcsses: Guidance loss between T-1 temporally adjacent sub-networks.
lossss_shaap [T-1]
ps random discard probability.
def drop (losses,
F, x.ahape
teech.ooes ((losses.
index torch. azgmax
for i in range shopseess.hhppeO00e
index:
consirue
else:
p zandom.random()
p prob:
w[1]
w.div_ w.div_(w.sum (u.
roturn W loss
20

---

Published as a conference paper at ICLR 2025
A.5 EXPERIMENTAL DETAILS
A.5.1 DATASETS
In this paper, we perform experiments on the neuromorphic datasets CIFARIO-DVS, and N-Caltech101, as well as the static image datasets CIFARIO, CIFARIO0, and the 3D point cloud datasets ModelNet10 and ModelNet4O.
CIFARIO-DV'S (Li et al., 2017) is benchmark dataset for neuromorphic object recognition, which contains 10,000 event samples of size 128 x 128. The dimension of each event sample æ is [t, p, 2, y]. where t is the time stamp. p is the polarity of the event, indicating the increase decrease of the pixel value, and [z. y] are the spatial coordinates. There are 10 classes samples in CIFARIO-DV, and we divide each class of samples into training and test sets in the ratio of 9:1 to evaluate the model performance, the same as the existing work (Zuo et al., 2024b; Shi et al., 2024; Yao et al., 2023).
DVS-Gesture (Amir al., 2017) dataset contains event samples for 11 gestures, of which 1176 used for training and 288 are used for testing. The dimension of each event sample is [4.p. c. y) and the spatial dimension size 128 x 128.
N-Caltech 101 (Orchard et al., 2015) contains event stream data for 101 objects, each sample with a spatial size of 180 x 240. There are 8709 samples in N-Caltech and divide the training set and the test set at a ratio of 9:1.
Since the event data is of high temporal resolution, we use the SpikingJelly (Fang et al.. 2023a) framework to integrate each event sample into '" event frames, where I corresponds to the timestep of the SNN. In addition, event frames are downsampled to spatial size of 48 x 48 before being input to the SNN.
The Spiking Heidelberg Digits (SHD) (Cramer et al., 2022) dataset contains 1000 spoken digits in 20 categories (from O to 9 in English and German) for the speech recognition task. For processing the SHD dataset, we followed (Zheng et al., 2024).
ModelNet10 (Wu et al., 2015) and ModelNer40 (Wu et al., 2015) are benchmark datasets for 3D point cloud classification, ModelNet10 contains point cloud data for 4899 objects in ten categories; ModelNet40 contains data for 12311 objects in 40 categories. For point cloud data preprocessing, we follow (Ren et al., 2023). We uniformly sample 1024 points on the mesh faces and input them into the SNN after normalizing them to unit sphere.
A.5.2 TRAINING SETTING
Our experiments are based on the PyTorch package, using the Nvidia RTX 4090 GPU. By default, we use the VGG-9 (Zuo et al., 2024b) architecture on the neuromorphic datasets. For R use the architecture settings described in (Ding al., 2024). We trained the model for 100 epochs using stochastic gradient descent optimizer with an initial learning rate of 0. and a tenfold decrease every 30 epochs. We trained the VGG-9 and ResNet-18 models without using any data augmentation techniques, and the weight decay value was 1e-3. The batch size during training is 64. The firing threshold / and membrane potential time constant T of spiking neurons were 1.0 and 2.0, respectively.
When our method used for the Transformer architecture SNN, we use the Reskingee (Shi et al., 2024) architecture. At this point, our training strategy is exactly the same as in the original paper, and we use the officially released training code directly.
Our method can be combined with the knowledge transfer strategy (He et al., 2024) to maximize performance gains, and we conducted experiments on N-Caltech101 using the officially released code. At this point, our training strategy is exactly the same (He al., 2024), but we set the batch size to 48 to reduce the memory overhead.
For the ID SHD speech recognition task, we follow the training strategy and architecture of (Zheng et al., 2024). We use the one-layer two-branch recurrent network architecture defined in (Zheng al., 2024) to which the proposed method is applied.
21

---

Published as a conference paper at ICLR 2025
For the 3D point cloud classification task, the spiking version of the lightweight Point- Net++ (Qi et al., 2017) architecture. We directly use the code released by (Ren et al., 2023), and thus our training strategy is exactly the same as that of (Ren al., 2023).
To reduce randomness, we report the average results of three independent experiments in our exper- iments, in addition to the experiments performed on Table (randomly selected individual models) and on the large-scale ImageNet.
A.6 ADDITIONAL ABLATION STUDY
In this section, we evaluate the proposed method on the neuromorphic speech recognition dataset SHD (Cramer et al., 2022). Taking DH-LIF (Zheng et al., 2024) as the baseline, the experimental results of the proposed method for ablation are shown in Table The results show that our proposed method still improves the performance of the baseline in speech recognition tasks, demonstrating the generality our method.
Furthermore, the experiments show that this view of SNN as ensemble of multiple subnetworks also holds for deeedeeendent tasks. Although time-dependent tasks require more time-varying in- formation than static tasks, much variance still negatively affect their performance. Our method reduces excessive variance across timesteps, improving ensemble stability and overall per- formance. It is worth noting that method does not completely eliminate variance, thus preserving the necessary dynamic information and allowing its application to time-dependent tasks.
Table 8: Ablation study results of the proposed method on neuromorphic speech recognition dataset SHD.
Method
Accuracy (%)
Baseline (DHI-LIF (Zheng et al., 2024))
89.86
+Smooth
90.33+0.47
+Guidance
90.46 90.46+0.60
+Both
91.19 91.19 L133
Ablation experiments on the 3D point cloud classification dataset ModelNet10 using the PointNet++ architecture and with timestep of effectiveness of our method. We also compare the performance when trained with T=2 but with inference T=1. Our method is able achieve 94.39% accuracy in timestep inference, surpassing the performance of the vanilla SNN by 1.78% for the same training and inference timestep.
shown Table and the results further confirm the
Table 9: Ablation study results of the proposed method on 3D point cloud classification dataset ModelNet10.
Method
Inference T=2 Acc. (%)
Method
Inference T=1 Acc. (%)
Baseline
93.75
Direct training
91.62
+Smooth
94.23+0.44
Vanilla SNN Training T=2
92.61
+Guidance
94.05
+Both
94.5440.77
Ours SNN Training T=2
94.39
22

---

Published as a conference paper at ICLR 2025
A.7 OBJECT DETECTION EXPERIMENT
We perform object detection on PASCAL VOC 2012 and COCO 2017 to evaluate whether the proposed method can provide performance gains non-classification tasks. When method is used for non-classification tasks (e.g., regression), we can compute the MSE loss between outputs instead of the KL divergence. We take Spike (Luo al., 2024) as the baseline and compute the guidance loss using MSE for its predicted coordinates, and the results for the PASCAL VOC 2012 and COCO 2017 datasets are shown in Table and Table 11, respectively (The training setup follows (Luo et al., 2024) and total of 80 epochs are trained). The results show that our method can be applied to the object detection task and improve the performance of the baseline model, and should be similarly applicable to other regression tasks.
Note that Spike (Luo et al., 2024) uses multi-bit neurons during training so that it can achieve satisfactory performance lower timesteps. However, this slightly affects the temporal correlation of the SNN. Therefore, our method may be able to achieve more significant facilitation when used in other SNNs (using vanilla single-bit multi multi-timestep neurons).
Table 10: Comparative results with existing methods PASCAL VOC 2012. T x D indicates that timesteps are set and each timestep is expanded D times. is set to by default in the comparative methods.
Method
#Param (M)
T x D
mAP@50 (%)
mAP@50:95 (%)
SpikeYOLO YOLO (Luo al., 2024)
13.2
x
48.0
30.9
Ours
13.2
2
48.7
31.5
Table 11: Comparative results with existing methods on COCO 2017 val. T D indicates that T timesteps are set and each timestep is expanded D times. D is set to by default in the comparative methods.
Method
#Param (M)
T D
mAP@50 (%)
mAP@55000 (%)
Spiking-YOOl YOLO (Kim et al., 2020)
10.2
3500
25.7
EMS-YOLO (Su al., 2023)
26.9
4
50.1
30.1
Meta-Spikeformer (YOLO) (Yao et al., 2024)
16.8
4
50.3
YOLO (Luo al., 2024)
13.2
2
$3.6
37.9
Ours
13.2
2 x 4
54.0
38,4
23

---

Published as a conference paper at ICLR 2025
A.8 TIMESTEP AND POWER CONSUMPTION ANALYSIS OF SPIKING TRANSFORMER MODEL
The timestep of the SNN is proportional to the training and inference overhead. To balance perfor- mance and overhead, the timestep in this paper on the neuromorphic dataset since the typical spiking Transformer (Shi et al., 2024) model uses larger timesteps such as 10 and 16 on the neuromorphic datasets, we explore here the performance of our method for the Transformer architecture at different timesteps. To evaluate the influence of the time-step hyperparameter on the performance, we conducted experiments on DVS-Gesture based on the SpikingResformer (Shi et al., 2024) architecture, and the results are shown in Table 12. The results show that even at larger timesteps, our method is still able to improve the performance of the baseline SpikingResformee model, although the effect gradually decreases the performance of the model saturates.
5. However,
Table 12: Comparison of performance and power consumption at different timesteps. Experiments were performed on DVS-Gesture with SpikingResformer architecture.
T=4
T=5
T=8
T=10
T=16
Baseline
89.93
90.63
92.89
93.58
96,99
Acc. (%)
+Smoohhing
91.67+1.74
94.4443.81
94.32+1.43
94.33++0.7
97.23+0.24
Baseline
0.3699
0.479
0,8119
1.0374
1.5788
Power (mJ)
+Smoothing
0,3869
0.4874
0.8079
1.0199
1.5769
In addition, Table 12 compares the power consumption of method with that of the bascline model. The calculation of the power consumption follows (Shi et al., 2024) (Power consumption positively correlated with the number of spikes). The results show that method only slightly increases the power consumption when the timestep is small, and our method with lower power consumption when the timestep large. In particular, the low number of spikes of the SNN low timesteps tends to lead to an inadequate feature representation. Our method generates slightly more spikes low timesteps to enhance the representation quality and thus the performance. In contrast, when the timestep is large, the vanilla SNN suffers from redundant spikes, and our method reduces the redundant spikes, thus improving performance and reducing power consumption. Overall, the difference in power consumption between our method and the baseline model is negligible, and therefore hardly affects the power consumption of the SNN model.
24

---

Published as a conference paper at ICLR 2025
A.9 EXPERIMENTS ON STATIC IMAGE DATASETS
In addition to neuromorphic datasets, we also conducted experiments on the static object recognition task demonstrate the generalizability of method. The comparative results for static datasets shown in Table 13, where we achieved 96. 16% and 79.22% accuracy for CIFAR10 and CIFAR 100, respectively, outperforming the other methods.
Table 13: Comparative results (%) on static datasets. denotes self-immlemennationn results.
Method
Architecture
Param (M)
T
CIFARI0 CIFAR 100
CLIF (Huang et al., 2024)
ResNet-18 18
11.21
94,89
77.00
RMP-Loss (Guo et al., 2023a)
ResNet-19
12.54
4
95.51
78.28
NDOT (Jiang et al., 2024a)
VGG-11
9.23
4
94.86
76.12
TAB (Jiang al., 2024b)
ResNet-19
12.54
4
94.76
76.81
SLT-TET (Anumasa al.. 2024)
ResNet-19
12.54
95.18
75.01
Spikformer (Zhou et al., 2023)
Spiking Tramsformer-4-384
9.28
95.19
77.86
SpikingResformer (Shi et al., 2024)
Spikinggesfforeer-T]
10.79
4
95.93"
78.23"
SDT (Yao al., 2023)
Spiking Transformer-2-33
2-512
10.21
4
95.60
78.40
Ours
SpikingRestormer-TT
10.79
96.16
79.22
To validate the scalability of our method, we performed experiments on Tiny-ImageNed, ImageNet- Hard (Taesiri al., 2023), and ImageNet.
The comparative results with existing methods on Tiny-lmageNet are shown in Table 14. Our method achieves an accuracy of 58.04% in only 4 timesteps, surpassing other comparative meth- ods.
Table 14: Comparative results on the Imageet dataset.
Method
Architecture
T
Accuracy (%)
Online LTL (Yang et al., 2022)
VGG-16
16
56.87
ASGL (Wang et al., 2023)
VGG-13
56.81
Joint A-SNN (Guo al., 2023b)
VGG-16
55.39
Ours
VGG-13
58.04
ImageNet- Hard is slightly smaller in scale than ImageNet, but more challenging. We take (Fang et al., 2023b) as the baseline and employ the same training strategy as (Fang et al., 2023b). The comparative results ImageNet-Hard are shown in Table 15. The results show that method can still be effective for this challenging dataset.
Table 15: Comparative results with (Fang et al., 2023b) on the challenging ImageNet-Hard dataset.
Method
Architecture
Top- Acc. (%)
Top-5 Acc. (%)
PSN (Fang et al., 2023b)
SEW-ResNer18 4
44.32
52.57
Ours
SEW-ResNer18
45.89
53.16
Since ImageNet requires more training resources and time, show the performance of training 250 epochs Table 16. The results show that our method is already able to achieve competitive performance even with only 250 epochs of training.
Table Comparative results with existing methods on ImageNet dataset.
Method
Architecture
T
Accuracy (%)
RecDis-SNN (Guo al., 2022)
ResNet-34
6
67.33
RMP-Loss (Guo et al., 2023a)
ResNet-34
4
65.17
SSCL (Zhang al., 2024b)
ResNet-34
4
66.78
TAB (Jiang et al., 2024b)
ResNet-34
4
67.78
NEW-ResNet (Fang al., 2021a)
NeW-ResNet34
4
63.18
Ours
SEW-ResNet34
4
69.03
25

---

Published as a conference paper at ICLR 2025
Layer 0
Layer
Layer
Layer
0.95
0.90
0.95
Cosing similarity
0.90
6311
Cosine sinilarity
0.30
Cosine similarii aritl
00.90
0.85
Cosine similarig
0.90
0.85
0.70
0.80
0.50
0.75
0.75
0.60
0.70
157
0.75
87
0.65
0.70
T1E-2
T12-3)
T(3-4)
0.50
T(L-2)
112-3)
T13-)
0.60
TU--3)
112-3)
T134)
0.65
111-2)
1(23) T13-4)
Layer
Layer
Layer
Layer
similarty
0.90
Cosine similartty
Cosine similarity
0.90
Cosine similarity
0.85
0.80
0.80
031
0.75
0.70
0.70
0.75
0.75
0.65
0.60
0.60
0.70
0.65
T01-2)
112-3)
103-4)
0.35
T(-3)
T12-)
T13-4)
0.40
m1-3)
11-3)
T13-)
0.65
111-2)
T12-3)
T13-4)
~Vanilla SNN
Ours
Figure 7: Comparison of the cosine similarity of membrane potential distributions for adjacent timesteps between the vanilla SNN and our method. T(a b) denotes the similarity of the distri- bution between timestep and timestep b. Our method significantly improves the similarity of the membrane potential distribution across timesteps, thus facilitating the ensemble performance.
T-0 Initialized to
Tw1 (100010000...111
T (0.0000003..016)
03.0
Tw3 (ue)m(co 01,0.19
Ted (wo0m1-0000.01111
36 52
43 29 63
95
-IA
Mombrane potential
Membrane potential (1.00(0.0330011)
potantial
Membrane potential
potential
Initialized
())0)=(0.022.0.1
T=4 (get=(0000.0.0000
Mombrane potential
Membrane patential
Membrane potential
Membrane
ALR Ls 12 03E
Membrane potential
Figure 8: Additional visualization of the membrane potential distribution on CIFARIO-DVS for comparison ((H, σ)) denotes the mean and standard deviation of the distribution). Top: Vanilla SNN. Bottom: Our method allows more stable distribution with smaller differences across timesteps.
A. ADDITIONAL VISUALIZATIONS
In this section, we provide additional visualizations to demonstrate the effectiveness our method in enhancing the similarity of membrane potential distributions across timesteps.
The cosine similarity of the membrane potential distribution across timesteps for the eight convolu- tional layer neurons in VGG-9 is shown in Fig. 7. Since the membrane potential is initialized to O, the similarity of the membrane potential distribution between the Oth timestep and the 1st timestep is O, and ignore this term in the figure. It can be seen that method consistently shows higher similarity compared to the vanilla SNN, which mitigates the difference in the distribution of membrane potentials across timesteps. Although method has lower similarity than the vanilla SNN for layer 7 timestep and timestep 2, this exception does not affect our overall role in reducing distributional differences.
In addition, additional visualizations of the membrane potential distribution on CIFARIO-DV'S are shown in Fig. to illustrate the effect of our method in smoothing the membrane potential distri-
26

---

Published as a conference paper at ICLR 2025
T-0 Initialized to
T=1 (1.00-(0.110..13
T=2 (0.00-00.680.22
Tw3 (6600000333))
(1.0)=10.00,000
-22
63 Membrane potential
-2
402
+0 asa 03
Membrane potential
Membrane potential
Membrane potential
Membrane potential
(0.00.0010.00117
T-2 (u.0)_(0.99.02 2.10
(quo)= (0.01.0.24)
Twa (wo))(0. (0.002000
Membrane potential
Membrane potenbal
90
05 Membrane potenbal
45 Nembrane potential
-55 Membrane potenial
Figure 9: Membrane potential distribution in spiking VGG-9 on DVS-Gesture (11, o)) denotes the mean and standard deviation of the distribution). Top: Vanilla SNN. Bottom: The overall membrane potential distribution our method is more stable.
bution. The results of the membrane potential visualization on DVS-Gesture are shown in Fig. 9, where again our method shows a more consistent distribution.
27


Reviews:
[{"review_id": "k7tYDjWKQE", "reviewer": "Reviewer_rfBg", "summary": "The authors proposed a method to improve the performance of SNN from a new perspective, ensemble learning, by membrane potential smoothing and temporally adjacent subnetwork guidance. The smoothing method solved the problem that if the differences between initial states of neuron membrane potentials is excessive, the performance of SNN in ensemble learning will be degraded. The knowledge distillation in temporally adjacent subnetworks also improves the consistency of the subnetworks’ output.", "questions": "1.The proposed method does not change the core of previous methods, the novelty compared with existing architectures and neuron models should be explained further, and how to improve the overall performance by these proposed approach.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.The proposed membrane potential smoothing not only mitigates the difference in membrane potential distribution, which degrades the performance in ensemble learning of SNN, but also reduces the influence of the temporal gradient vanishing.", "weaknesses": "1.Some spelling errors. In ABSTRACT, line 25, “Transofrmer” -> “Transformer”.\n\n2.The differences in the membrane potential distribution may not be easily observed. Adding quantitative data tables may be better.\n\n3.The proposed method does not change the core of previous methods, the novelty compared with existing architectures and neuron models should be explained further, and how to improve the overall performance by these proposed approaches.\n\n4. To be honest, the performance of the proposed model on CIFAR10-DVS and DVS-GESTURE is not higher compared to most of the existing spiking models (proposed in the last six months) based on VGG or ResNet backbone, Thus I could not determine the novelty and effectiveness of the proposed model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AOApCIUpyH", "reviewer": "Reviewer_Szjy", "summary": "The authors propose to adaptively smooth the membrane potential cross adjacent timesteps to reduce the initial state discrepancy.  They propose the temporally adjacent subnetwork guidance to improve the output consistency of subnetworks through distillation. Compared to existing methods, their method shows superior performance in neuromorphic/static object/gesture recognition and 3D\npoint cloud classification tasks.", "questions": "***1.*** The text in the figures is too small; please enlarge it to improve readability.\n\n***2.*** Why perform 3D point cloud classification experiments. How does this validate the method's characteristics and effectiveness?\n\n***3.*** The authors' method aims for the membrane potential distribution to become more consistent over time. However, at timestep 0, the method still shows a significant difference compared to other timesteps, with this difference being greater than that between other timesteps. This discrepancy makes it difficult for the method to support the argument of achieving consistent membrane potentials.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "***1.*** This paper is well-structured with high-quality figures, tables, and formulas.\n\n***2.*** The paper includes relatively adequate analysis, comparative experiments, and ablation studies.", "weaknesses": "***1.*** This paper is not the first to view SNNs as a form of ensemble learning; it should reference prior works that have held this perspective.\n\n***2.*** One of the motivations of this paper is to make instances at each time step more consistent; however, this has been demonstrated in many previous works and cannot be considered a novel contribution. But the method is novel. Author should change the focus when the write the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xLiARP9SBH", "reviewer": "Reviewer_eENu", "summary": "This work rethinks spiking neural networks (SNNs) from the perspective of an ensemble of multiple temporal subnetworks and argues that improving the stability of the membrane potential and the output of each subnetwork would benefit the performance of SNNs. Based on these motives, this work proposes an approach of membrane potential smoothing and a well-designed learning objective with temporally adjacent subnetwork guidance to improve stability. When applied to several SNNs, the proposed method achieves higher performance across multiple datasets and tasks.", "questions": "1. How does the method perform when applied to some more time-dependent datasets, such as spiking Heidelberg digits (SHD) [3]? Does membrane potential smoothing still benefit the performance in such cases?\n2. How does temporally adjacent subnetwork guidance compare to or improve upon the contrastive learning method [2]? In addition, how can the loss function be adapted for non-classification tasks, such as object detection which requires the output of coordinates?\n3. Please provide results on ImageNet or explain why ImageNet experiments are not included.\n\n[3] Benjamin Cramer, Yannik Stradmann, Johannes Schemmel, and Friedemann Zenke. The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks. IEEE Transactions on Neural Networks and Learning Systems. 2020.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Most commendable is the match between the motivation and the proposed methodology of this work. The stability of temporal subnetworks is a meaningful research focus, and both membrane potential smoothing and temporally adjacent subnetwork guidance intuitively fit well with the goal of improving stability, especially since the smoothing design is concise and sufficiently novel.\n2. The method efficiently improves several SNNs' performance across multiple datasets and tasks, demonstrating its effectiveness and generalizability.\n3. The ablation studies are very comprehensive, illustrating the contribution of the methods and the influence of the hyperparameters. The visualization results show graphically the advantages of the method for improving stability.\n4. The paper is well-written and easy to follow.", "weaknesses": "Major points:\n1. The view of SNNs as an ensemble of temporal subnetworks actually originates from Spiking PointNet [1], and they proposed this view because the point cloud data is static and does not involve time dependence. One of the great features of SNNs is their ability to handle temporal data. Therefore, it is doubted whether the ensemble view is still justified when SNNs process data with strong time dependence, and whether membrane potential smoothing is necessary, as more varied potentials seem to be more conducive to representing the dynamic information of the input.\n2. The idea of promoting the consistency of SNN instances at different timesteps has been proposed in previous work [2]. While they used contrastive learning to bring network features closer at different time steps, this work uses KL divergence to bring the probability distribution of outputs closer at different time steps. The former design has broader applicability, while the latter is limited to classification tasks.\n3. Given that all experiments in this work are performed on classification tasks, the lack of ImageNet results makes the method somewhat unconvincing.\n\nMinor point:\n1. The statement that membrane potential smoothing is effective in other neurons is reckless (lines 278-279) and there is a lack of experiments to prove this.\n\n[1] Dayong Ren, Zhe Ma, Yuanpei Chen, Weihang Peng, Xiaode Liu, Yuhan Zhang, and Yufei Guo. Spiking pointnet: Spiking neural networks for point clouds. In Conference on Neural Information Processing Systems. 2023.\n\n[2] Haonan Qiu, Zeyin Song, Yanqi Chen, Munan Ning, Wei Fang, Tao Sun, Zhengyu Ma, Li Yuan & Yonghong Tian. Temporal Contrastive Learning for Spiking Neural Networks. In Conference on Artificial Neural Networks and Machine Learning. 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m0qf4dmGK7", "reviewer": "Reviewer_yRDN", "summary": "This paper conducts a thorough investigation into the common issues faced by Spiking Neural Networks (SNNs) and identifies that SNNs primarily suffer from performance degradation due to excessive differences in membrane potential between time steps and uneven feature distribution across time steps. To address these issues, the paper proposes two approaches: **subnetwork guidance** (where each step of the model may be considered as a subnet) and **membrane potential smoothing**. Additionally, ablation experiments are presented to demonstrate the effectiveness of both methods, showing significant improvements in model performance.", "questions": "* The study lacks experiments on large-scale static datasets, such as ImageNet. Can the authors provide additional validation on these datasets to better assess the generalizability of their methods?\n\n* For the spiking transformer model, the commonly used timesteps are 4, 10, or 16. Could the authors clarify the rationale behind selecting a timestep of 5, and discuss any observed impact on performance or efficiency?\n\n* While the paper highlights the efficiency of Spiking Neural Networks (SNNs), there is no evaluation of the proposed method’s energy consumption. Could the authors provide insights into whether this method might lead to high energy consumption, potentially offsetting the efficiency benefits typically associated with SNNs? A practical energy consumption assessment on hardware platforms such as GPUs, FPGAs, or neuromorphic chips would strengthen this aspect.\n\n* In Figure 1, the paper compares the membrane potential distribution of a spiking VGG-9 model on the CIFAR10-DVS dataset, contrasting the vanilla SNN with the proposed method. The figure suggests that the proposed method stabilizes the membrane potential distribution across timesteps. However, the distinction between the two distributions is not clearly evident, making it difficult to interpret the extent of improvement. Could the authors provide further clarification or additional visualizations to better illustrate this difference?", "limitations": "", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* The paper is well-written, with a clear and easily understandable motivation. This clarity allows readers to quickly grasp the purpose and significance of the proposed methods.\n* The effectiveness of the proposed methods is thoroughly validated on both static image datasets (CIFAR10/CIFAR100) and dynamic vision sensor (DVS) datasets, demonstrating strong performance across different data types.\n* Experimental results indicate that the proposed methods are compatible with both CNN-based and Transformer architectures, highlighting their robust generalizability. This versatility suggests that the methods could serve as a widely applicable paradigm in SNN research.", "weaknesses": "* The paper mentions the efficiency of Spiking Neural Networks (SNNs) but does not provide experimental evidence on whether the proposed method retains this efficiency. It is recommended to evaluate energy consumption, preferably on hardware platforms such as GPUs, FPGAs, or neuromorphic chips, or to provide theoretical calculations as an alternative. Relevant computation methods can be referenced from prior works [1–4].\n\n* The current experiments are limited to smaller datasets and do not include evaluations on larger, static datasets such as ImageNet. This limitation raises concerns about the method's generalizability and scalability. To strengthen the paper, I suggest including experiments on larger datasets or, at minimum, providing a theoretical analysis of expected performance on large-scale data. Such an evaluation would support the paper's claims of generalizability and effectiveness.\n\n* In Figure 1, the paper shows the membrane potential distribution in a spiking VGG-9 model on the CIFAR10-DVS dataset, comparing the vanilla SNN with the proposed method. Although the figure aims to demonstrate a more stable membrane potential distribution across timesteps with the proposed approach, the visual difference between the two distributions is unclear, making it challenging to assess stability improvements. To clarify this, I recommend including quantitative measures of distribution stability across timesteps, such as mean and variance, or other relevant statistical characteristics. Additionally, a side-by-side comparison of these statistics could highlight any notable differences, making the figure’s intended message more accessible to readers.\n\n\n[1] Horowitz, M. \"1.1 Computing's energy problem (and what we can do about it).\" *IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC)*, 2014.\n\n[2] Luo, X. et al. \"Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-performance and Energy-efficient Object Detection.\" *arXiv preprint arXiv:2407.20708*, 2024.\n\n[3] Yao, M. et al. \"Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips.\" *Twelfth International Conference on Learning Representations (ICLR)*.\n\n[4] Shen, G. et al. \"Are Conventional SNNs Really Efficient? A Perspective from Network Quantization.\" *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)