QP-SNN: Quantized and Pruned Spiking Neural Networks
Authors: Wenjie Wei
Malu Zhang
Zijian Zhou
Ammar Belatreche
Yimeng Shan
Yu Liang
Honglin Cao
Jieyuan Zhang
Yang Yang
Venue: ICLR 2025 Poster (2025)
Keywords: Spiking Neural Networks
Neuromorphic Computing
Spiking Pruning
Spiking Quantization
Primary area: applications to neuroscience & cognitive science

Abstract: Brain-inspired Spiking Neural Networks (SNNs) leverage sparse spikes to encode information and operate in an asynchronous event-driven manner, offering a highly energy-efficient paradigm for machine intelligence. However, the current SNN community focuses primarily on performance improvement by developing large-scale models, which limits the applicability of SNNs in resource-limited edge devices. In this paper, we propose a hardware-friendly and lightweight SNN, aimed at effectively deploying high-performance SNN in resource-limited scenarios. Specifically, we first develop a baseline model that integrates uniform quantization and structured pruning, called QP-SNN baseline. While this baseline significantly reduces storage demands and computational costs, it suffers from performance decline. To address this, we conduct an in-depth analysis of the challenges in quantization and pruning that lead to performance degradation and propose solutions to enhance the baseline's performance. For weight quantization, we propose a weight rescaling strategy that utilizes bit width more effectively to enhance the model's representation capability. For structured pruning, we propose a novel pruning criterion using the singular value of spatiotemporal spike activities to enable more accurate removal of redundant kernels. Extensive experiments demonstrate that integrating two proposed methods into the baseline allows QP-SNN to achieve state-of-the-art performance and efficiency, underscoring its potential for enhancing SNN deployment in edge intelligence computing.

Published as a conference paper at ICLR 2025
QP-SNNS: QUANTIZED AND PRUNED SPIKING NEURAL NETWORKS
Wenjie Wei', Malu Zhang': Zijian Zhou', Ammar Belatreche", Yimeng Shan Yu Liang', Honglin Jieyuan Zhang'. Yang Yang'
University of Electronic Science and Technology of China 2 Northumbria University, Liaoning Technical University
ABSTRACT
Brain-inspired Spiking Neural Networks (SNNs) leverage sparse spikes en- code information and operate in an asynchronous event-driven manner, offering highly energy-efficient paradigm for machine intelligence. However, the current SNN community focuses primarily on performance improvement by developing large-scale models, which limits the applicability of SNNs in limureee-lmieed edge devices. In this paper, propose hardware- friendly and lightweight SNN, aimed at effectively deploying high-performance SNN in resource-limitt sce- narios. Specifically,
first develop baseline model that integrates uniform quantization and structured pruning, called QP-SNN baseline. While this baseline significantly reduces storage demands and computational costs, it suffers from performance decline. To address this, we conduct an in-depth analysis of the challenges in quantization and pruning that lead to performance degradation and propose solutions to enhance the baseline's ' performance. For weight quantization, we propose weight rescaling strategy that utilizes bit width more effectively enhance the model's representation capability. For structured pruning, we pro- pose a novel pruning criterion using the singular value of spatiotemporal spike activities to enable more accurate removal of redundant kernels. Extensive exper- iments demonstrate that integrating two proposed methods into the baseline allows QP-SNN to achieve fftte-of-the-art performance and efficiency, underscoring its potential for enhancing SNN deployment in edge intelligence computing.
INTRODUCTION
Inspired by the information processing paradigm of biological systems, Spiking Neural Networks (SNNs) encode information via binary spikes and process them in a sparse spike-driven manner (Gerstner & Kistler (2002): [zhikevich| (2003)). This paradigm simplifies the matrix computations of weight and spike activity in SNNs from computationally inten- sive multiply-accumulate (MAC) operations to com- putationally efficient accumulate (AC) operations. Therefore, SNNs are regarded as a promising energy- efficient solution for achieving next-generation ma- chine intelligence (Pfeiffer & Pfeil (2018); Roy et al. (019b)). Furthermore, the energy efficiency of SNNs has driven the development of neuromorphic hardware, such as SpiNNaker (Painkras et al. (2013))
75
[IC RINI
QRESNN
3.19
Accurasy 100 Acccuge
70
Stiveal
65
(JONNOI
TESE
144DB
Deegeral
1L/70MB
Tankal
Despital (eNtiss)
55
wachh for weight
Figure 1: Comparison of QP-SNN and re- lated work CIFAR- 100. The bubble size represents the model size. TrueNorth (Akopyan et al. (2015)) Loihi (Davies)
et al.) (2018)), Tianjic (Pei et al. (2019) etc. These neuromorphic hardware can fully exploit the energy effiiiecy potential of SNNs. Despite these advantages, the application scenarios and perfor- mance of SNNs still require improvement compared to traditional artificial neural networks (ANNs).
"Corresponding author; maluzhang@@estt..ddu..

---

Published as a conference paper at ICLR 2025
In the past few years, the SNN community has focused primarily on designing complex SNNS
chitectures Yu et al. (2020); Yao et al. (2024b a): Luo et al. (2024): Wang et al. (2024b); Zhang
eet al. (2024) and traiiin strategies (hang et al. 2018 2019 2021a Wu et al. (2021); Guo et al.
(2022) Xu et al. 2023); ; Wei et al. (024b)) to achieve impressive performance across various ap-
plication tasks. While these studies have yielded satisfactory performance, they typically come at the cost of large model parameters, high memory consumption, and increased computational com- plexity (Yan et al. (2024); Liu et al. (2024); Cao et al. (2025). This undermines the inherent energy efficiency of SNNs and restricts their applicability in resource- resource-limited scenarios. To achieve effec- tive deployment, growing number of researchers have worked on compressing large-scale SNNs. Existing methods to improve SNN energy efficiency primarily fall into two categories: (1) reducing the precision of parameter representations and (2) reducing redundant parameters within the model.
Quantization is key technique for the first category, which reduces memory storage and computa- tional complexity by storing full-precision values in low bit-width precision (Gholami et al. (2022). Based on whether an equal-size interval is used to discretize full-precision values, it can be divided into non-uniform and uniform quantization (Rokh et al. (2023). Non-uniform quantization divides discretization intervals unevenly, enabling more precise capture of critical information and leading to improved performance. However, it is challenging to deploy this approach on general computing hardware efficiently (Cai al. (2017); Kulkarni et al.] (2022). In contrast, uniform quantization maps full-precision values to equal-sized discrete intervals, offering advantages such as simplicity, low computational cost, and efficient mapping to hardware (Zhu et al. (2016); JJinn al. (2020)).
Pruning is one of the effective methods for reducing redundant parameters in a model, which reduces the model size by removing unimportant connections (Li et al. (2016)). Pruning can be classified into unstructured and structured pruning (Vadera & Ameen (2022)) Unstructured pruning removes individual nodes like single neuron of networks, resulting in unstructured sparsity. This often leads to high compression rate, but requires specialized hardware or library support for acceleration (Han jet al. 2015)). contrast, structured pruning removes entire convolutional filters, ensuring model's structure. This avoids complex sparse matrix operations, enabling acceleration with standard hard- ware by taking advantage of a highly efficient library (He & Xiao (2023); Xu et al. (2020).
Real-world deployments are typically limited by size, weight, area, and power. This makes com- bined quantization and pruning a promising approach for maximizing SNN compression. Existing research integrating quantization and pruning in SNNs faces two challenges. Firstly, they do not sufficiently account for hardware-frienae friendliness,, for example, (Rathi et al. (2018)) and (Deng et al. (202T) adopt unstructured pruning. Secondly, despite significant energy efficiency, these studies suffer from severe performance degradation, with evaluations limited to simple datasets (Chowd- hury al. (202T): [Dnn et al. (202T)) Thus, integrating both techniques for maximal compression while ensuring hardware efficiency and high performance remains challenging. In this paper, we introduce the QP-SNN, tailored for effective deployment in resource-limited environments. We first build a QP-SNN baseline that integrates uniform quantization and structured pruning. While this baseline offers notable efficiency gains, it suffers from reduced performance. To address this, we investigate the root causes of performance degradation in quantization and pruning, and propose solutions to enhance the baseline's performance. As shown in Figure [ QP-SNN utilizing the pro- posed solutions achieves excellent accuracy and model size. We summarize main contributions as,
We first develop hardware-efficient and lightweight QP-SNN baseline by integrating uni- form quantization and structured pruning. This baseline significantly reduces storage and computational demands, but suffers from performance limitations.
To improve performance through quantization, we reveal that the vanilla uniform quan- tization in the QP-SNN baseline constrains the model's representation capability due to inefficient bit-width utilization. To address this, we propose a weight rescaling strategy (ReScaW) that optimizes bit-width usage for improved representation.
To further boost performance through pruning, we introduce novel structured pruning criterion for the QP-SNN baseline that leverages the singular value of spatiotemporal spike activity (SVS). This SVS criterion provides greater robustness across varying input samples and allows more precise removal of redundant convolutional kernels.
Extensive experiments show that integrating ReScaW- quantization and the SVS- based criterion into the baseline allows QP-SNN to achieve of performance and efficiency, revealing its potential for advancing edge intelligence computing.
2

---

Published as a conference paper at ICLR 2025
2
RELATED WORK
Quantization technique Early research on quantization in SNNs is primarily based on ANN-to- SNN conversion algorithms, where quantized ANN is first trained and then converted into the corresponding quantized SNN version (Sorbaro et al. 2020); ; Roy et al. (2019))). To mitigate the performance loss associated with the conversion, researchers have proposed many strategies, such as the utilization of activation penalty term (Sorbaro et al. (2020) and the weight-threshold balanc- ing method (Wang et al. (2020). However,these quantized SNNs still experience significant perfor- mance degradation and long latency issues. To address these limitations, some studies have explored directly training quantized SNNs and introduced different strategies to enhance performance, such as alternating direction method of multipliers (Deng ct al. (202T)) accuracy loss estimator (Pei et al. (202)), ) suitable activation function (Hu al. (2024)), and weight-spike dual regulation (Wei et al. 20244); Wang et al. (2024a)). Despite performance improvement, these studies fail to effectively leverage the allocated bit-width, resulting in the limited expressive capability of models. Therefore, there still remains significant room for performance improvement.
Pruning technique Existing research on pruning SNNs can be broadly divided into two groups. The first group is unstructured pruning. For example, (Yin ct al. (2021) use magnitude-basee method to remove insignificant weights, and (Shi et al. (2024)) propose a fine-grained pruning framework that integrates unstructured weight and neuron pruning to enhance SNN energy effi- ciency. Additionally, there are some biologically inspired unstructured pruning works (Bellec et al.) 2017): Chen et al.] (2022). While these studies achieve great sparsity and performance, they lead to irregular memory access in forward propagation, requiring specialized hardware for acceleration. The second group is structured pruning that offers better hardware compatibility. (Chowdhury et al.) (222T) use principal component analysis on membrane potentials to evaluate channel correlations and eliminate redundant ones. However, it suffers from long latency and cannot handle neuromor- phic datasets. Recently. (Li et al. (2024)) evaluate the importance of kernels based on spike activity, advancing the performance of pruned SNNs to a new level. However, this evaluation criterion ex- hibits high dependency on inputs and may not accurately reflect the importance of kernels.
Compression with joint quantization and pruning Several studies have explored combining quantization and pruning to maximize the compression of SNNs. First, (Rathi al. (2018) adopt the STDP learning rule and a predefined pruning threshold to remove insignificant connections, and then quantizes retained important weights. Then, (Chowdhury et al. (202T) perform principal component analysis on membrane potentials for spatial pruning and gradually decreases the time step during training for temporal pruning. They also use post-training quantization to compress tained weights. Moreover, (Deng et al. (202T) formulate pruning and quantization as a constraint optimization problem in supervised learning, and address it with the alternating direction method of multipliers. However, these existing studies combining quantization and pruning face two main problems. Firstly, the unstructured pruning methods in (Rathi et al. (2018)) and (Deng et al.] (202I)) require specialized hardware for efficient acceleration. Secondly, (Rathi et al. (2018)) only evaluate their method on very simple datasets, and (Chowdhury et al. (2021) and (Deng et al. (202T) only extend their methods to CIFAR (88.6% and 87.84% accuracy CIFAR-I0 with 5, 3 bits respec- tively), leading to significant room for improvement in both performance and efficiency.
QUANTIZED AND PRUNED SNN BASELINE
In this section, we develop the QP-SNN baseline by combining uniform quantization and structured pruning. These two compression techniques are highly compatible with existing hardware accelera- tors, significantly improving the model's deployment efficiency.
Neuron model.
Many spiking neurons have been proposed (Hodgkin & Huxley Zhang et al.
(2021a); Wei al. (2023)) we use the Leaky Integrate- and-Fee (LIF) model in QP-SNN due to its high computational efficiency (Wu et al. (2018)) The membrane potential of LIF model is,
(1) where U'tt] is the membrane potential of neurons in layer I at time t, T is the leaky factor, and X W is input current. Neurons integrate incoming signals and generate spike when
Ü'tt] = - 1] [/]
3

---

Published as a conference paper at ICLR 2025
the membrane potential exceeds the threshold 0. The spike generation function is computed as, s' 1, if 1' 2 0,
0, otherwise.
After spike emission, we use the hard reset mechanism to update the membrane potential. This mechanism resets membrane potential zero when spike occurs and remains inactive otherwise.
U'[t] U'( st()))
Vanilla uniform quantization, Quantization can be grouped into non-uniform and uniform quan- tization. Non-uniform quantization exhibits superior performance, but requires specialized hardware support. In contrast, uniform quantization maps weights to integer grids with equal size, simplifying both computational complexity and hardware implementation (Zhu et al. (2016)). In this study, we explore the uniform quantization in QP-SNNs. The vanilla uniform quantization for weights in layer 1, i.e. W can be formulated as follows,
((b) 2
W ind
(clamp(W'; 1, + +2)
(4)
where clamp(-) is a clipping operator, [.] is a rounding operator, 2 is the zero-point, b is the bit width, and s(b) 2 -1 1 is the number of integer grids. We set to I and explore bit widths b of
8, 4, 2. Therefore, Eq. 4 maps W onto the unsigned integer grid (0. . 2 1}. To reconstruct
w through their quantized counterparts, the de-quantization is defined as,
W
w =
int
(5)
s(b)
Consequently, the general definition for the quantization used in the QP-SNN baseline is stated as,
s(b)
w W
s(b) 2 (clamp -1,1) +
The vanilla quantization greatly reduces the baseline's storage and computation demands, but suffers from the limited weight precision. This constrains the model's representation capability, reducing performance. In the next section, we resolve this issue by effectively using the assigned bit-width.
Structured pruning. Pruning can be classified unstructured and structured pruning. Unstruc- tured pruning enables high sparsity and excellent performance but requires specialized design for hardware acceleration. In contrast, structured pruning preserves the model's structure and is highly compatible with existing hardware accelerators. Currently, the most advanced structured pruning method in SNN is presented by (Li et al. (2024)). They prune convolutional kernels according to the spiking channel activity (SCA) criterion. We use this criterion in our QP-SNN baseline for the cri- terion evaluates and prunes kernels based on the magnitude of membrane potential. The importance evaluation for the /th kernel, i.e., Wi./ RR--1 xkxk is defined as,
following analysis and comparison. For the weight tensor W e Raxa xxkx the
1 B.T
Score Score(W Wh
where B is the number of samples per mini-batch, T is the time step. ||. is the Ll-norm, and Ü.. [[] is the membrane potential of the f-th feature map. As shown in Eq. the SCA-based criterion regards positive values in Ü.. as excitatory postsynaptic potentials and negative values as inhibitory postsynaptic potentials, thus removing kernels that contribute less to the membrane potential. By unifying the SCA-based criterion, the number of parameter and computation in baseline is further reduced. Noteworthy, the performance of the pruned QP-SNN baseline model relies strongly on the scoring function. Therefore, the idea Score(W)' should accurately identify the important kernels.
METHOD
To enhance the performance of the QP-SNN baseline, analyze and resolve the underlying issues in quantization and pruning that cause performance reduction. In quantization, we reveal that the baseline suffers from limited representation capability due inefficient bit-width utilization, and propose the weight rescaling strategy to use bit-width more effectively. In pruning, we propose a novel pruning criterion for the QP-SNN baseline more accurately remove redundant kernels.
4

---

Published as a conference paper at ICLR 2025
W
-a a
Po (oooW)maa mi:5
Comod
165
Conv10
PawtW"l m030
15
Conv16
Pi PiwlWimm6z
Pa -032
Pas -233
Pasl Panl (W0) .214
Vanilla
1.0
1.0
Wasted
Wasted
0.5
W inc
Nactual a x s(b)
s(b):
0.
0.5
0.0
0.5
0.0
VGG-U6
0.
0.0
0.55
VGG-16
0.5
0.0
0.5
Laver Canv!
Conv
soated
0
1.0
(WF)
ReScaW
((b)
0.5
2
W
0
N actuual 2
s(b)
0.0
RocNu
Rochaz30
Rochar20
0.5
0.0
0.3
005
0.0
0.3
0.0
0.5
(a) Uniform quantization comparison
(b) Visualization of weight distribution
Figure 2: (a) Vanilla uniform quantization exhibits inefficient bit-width utilization, while ReScaW. based quantization can fully leverage the allocated bit-width. Green dots represent normal weights within the 1st and 99th percentiles, orange dots are boundary values, and blue dots are outliers. (b) Distribution is plotted to show that weights are concentrated in a narrow range around zero.
4.1 WEIGHT RESCALING STRATEGY
Problem analysis. The vanilla uniform quantization the QP-SNN baseline minimizes resource usage, but suffers from inefficient bit-width utilization. This weakens the discrimination of the quan- tized weights in the QP-SNN baseline, limiting the model's representation capability. To evaluate the bit-width utilization efficiency, define metric as,
Ratilize = Noctual (Wl
(8)
Neotal (Wi inut
where Nactuai(.), N-otai(-) are the actual, available number of distinct values that W repre- sents. Next, analyze Rutitize for QP-SNN baseline to assess bit-width utilization efficiency. We compute Nactwal (W! ine) using the range of full precision weights, shown in Figure 2(a) We consider full precision weights between the 1st and 99th percentiles to eliminate outliers. For clarity,
we denote W E e- a). where a = max Po.01 01 (W') ) Po.99 (W')) [. Typically, a is a positive
value near zero (LeCun al. (2002): ; He et al. (2015)) ). This means that the clamp function in Eq.44)
doesn't 't alter weight values. Based on this, we can deduce W E [| =
2 +bb+ z+a
2 s(b)
Therefore, Nectuaii (W!.) (nt)) approximately (I+aa sb))), leading a utilization rate of a(0)-a+1
a(b)+1
To clearly show the low bit-width utilization of the baseline, we analyze the weight distribution and determine the value of a. We plot the weight distributions of VGG-16 and ResNer20 in Figure 2(b) and also label the 1st and 99th percentiles. 16.conv 16), and the largest is 0.32 (VGG-16.conv4). This indicates that under 8-bit quantization, the QP-SNN baseline uses less than half of the assigned bit width, with a minimum of 14.33% and maximum of 32.26%. This inefficient utilization causes large number of weights to be quantized to the same integer grid, reducing the discrimination of quantized weights. This limits the representation capacity of the QP-SNN baseline, leading decreased performance. The complete weight distributions are provided in Appendix E.I
Figure 2(b) the smallest value a is 0.14 (VGG-
ReSca W-based uniform quantization. To resolve the limited representation capacity, we pro- pose a simple yet effective weight rescaling (ReScaW) strategy for the QP-SNN baseline that uses bit-width more efficiently. Specifically, we introduce scale coefficient y to regulate the weight distribution to a wider range before quantization. The proposed ReScaW strategy is defined as,
W
Wi scaled
The scaling coefficient y can assume any positive value within the range 0 <у <1. We provide three options for 7: (1) the maximum absolute value: max{ W (2) the maximum absolute value of the
5

---

Published as a conference paper at ICLR 2025
REAR-LRR
S'e [0.1)
Squeeze
E (0.1)
W
0
Sang 129
W
h. W
W
SVS Cricerfon
1 W
W
0
h1
importance Score(
-0,0,,0
Figure 4: Proposed pruning criterion based on the singular value of spatiotemporal spike activity.
r-th and (1 (1- percentiles: Ve (w') max (1P1- (W") (w"))))): and (3) 1-norm mean
|w'll, where |W|| is the number of entries in W These three options scale weights value: T
to span the range of [-1, 1], thereby ensuring more efficient bit width utilization. The impact of
these three options performance will be explored in the experimental section. Consequently, formulate the ReScaW-based uniform quantization as,
8(b) s(b) 2
(clamp( w : -1,1)
W' z
(10)
We compare vanilla uniform quantization in the QP-SNN baseline with the ReSca W in Figure 2(a) Clearly, the ReScaW method utilizes the allocated bit width more efficiently. This efficient bit- width utilization preserves the discrimination of quantized weights, enhancing the representation capability and performance of the QP-SNN baseline.
4.2 PRUNING CRITERION BASED ON THE SINGULAR VALUE OF SPIKE ACTIVITY
Problem analysis. The structured pruning work (Li et al. )2024)) employing the SCA- based criterion can produce high-performance pruned models, but the performance is ensured through multiple iterative pruning and regrowth processes. In fact, observe that the SCA- based pruning criterion exhibits a high depen- dency inputs. Specifically. it generates vary- ing importance scores for the same convolu- tional kernel when processing different inputs. To prove this observation, we plot the impor- tance scores of different kernels under varying inputs, as shown in Figure 3 The strong in- put dependency of SCA criterion can lead bi- ases in kernel evaluation, posing a risk of erro- neously identifying crucial kernels as insignif- icant or misjudging unimportant kernels as essential. These misidentifications can affect the liability of pruning, ultimately diminishing the performance of the QP-SNN baseline. Complete importance scores available in Appendix F.I
3000
2500
Batch index
2000
1500
1000
V
RosNo20
500
Channel index
Figure 3: SCA assigns different scores to the same kernel for different input samples. The colors in the figure represent the value importance score.
bys-b pruning criterion. Several studies suggest that the number of singular values corre- lates with information richness (Sadek (2012); Baker (2005): Jaradat et al. (22TT) I)). Inspired by this, we propose a novel pruning criterion for SN- baseline using the singular value of spike activity (SVS) to remove redundant kernels precisely. As shown in Figure 4 the SVS-basee criterion applies singular value decomposition to the average spike matrix over a given time window T, defined as,
1 Σ
s (1]
PE QT,
(11)
T
1=!

---

Published as a conference paper at ICLR 2025
where h, are the height and width of the spike matrix. P [p:...... Phl
R and Q
Iq....., qw] E R are orthogonal matrices representing the left and right singular vectors. Σ E Rox" is diagonal matrix containing the singular values of S in descending order, denoted as,
Σ = diag (0i, ... ",mmn(hh,w))
(12)
with 0i 2 02 2 ....... o,- 2 0 2.. ... mmin((h 2
Here, serves threshold distinguish significant and negligible singular values, and it is gen- erally set to a positive value near zero. Based on the threshold 6, aigg can be expressed as two
components: aig = E o.P.q, + Eier'+ a.p.q'" The first term captures the core feature dictated by the significant singular values, while the second term reflects potentially noise-related information (Jaradat et al. (2021)). Based on this decomposition, we define our pruning criterion as,
Score( Score(W'/J) EB
EE Emin(H, min(H,W)
T(o; ((o...)).
(13)
where Eg denotes the average over the mini- and II(-) is the indicator function that counts only sigmificant singular values (those exceeding e) for importance evaluation. We will demonstrate in the experimental section that the proposed SVS-based pruning criterion achieves more accurate removal of unimportant convolutional kernels.
By integrating the ReScaW strategy and the SVS-based pruning criterion into the baseline, we de- velop QP-SNN, with its workflow outlined in Appendix [A] In summary, the QP-SNN is lightweight and hariware-friendlly, while also achieving high performance, making an efficient solution for applications in resource-constrained scenarios.
5 EXPERIMENT
In this section, we first present the experiment setup. Second, we evaluate the performance of QP- SNN by comparing it to existing methods. Third, we demonstrate QP-SNN's scalability to complex architectures and tasks. Finally, conduct extensive ablation studies to verify the effectiveness of the ReScaW and the SVS-based pruning criterion. Additional analyses of iitiization-prunnngg order and QP-SNN's efficiency advantages provided in Appendices B and respectively.
5.1 EXPERIMENT SETUP
We first evaluate our method on image classification tasks, including static datasets like CI- FAR (Krizhevsky al. )2009). TinylmageNet, and ImageNet-Ik (Deng et al. (2009)), alongside neuromorphic DVS-CIII CIFARIIRRIOOO (Li et al. (2017)) These datasets serve as standard benchmarks in machine learning and neuromorphic computing. For architecture, we use classical structures VG- GNet and Spiking ResNet (Zheng et al.) (2021). with details provided in Table [ We use SEW- ResNet (Fang et al. (202T)) on ImageNet- Ik for a fair comparison with (Shi et al. (2024).. As for in 13). we observe minimal variation in the singular values, so set it to a small value of 1c-6 (Jaradat et al. (202I)). For the learning of QP-SNN, we use the surrogate gradient and straight- through estimator handle the nondifferentiabiiity, We provide additional details in the appendix, with the learning algorithm described in Appendix D and experimental setups in Appendix G
5.2 PERFORMANCE COMPARISON
As shown in Table D we compare QP-SNN with related work in performance and model size to prove the effectiveness and efficiency. We use 8, 4, and 2- weight configurations in experiments across all datasets. Compared 1op-1 performance with fewer timesteps, such as 2 or 4 on static datasets. When compared to direct algorithms, QP-SNN also performs well. On CIFAR-I0 and CIFAR-100, QP-SNN outperforms previous methods (Shi et al. (2024): Li et al. (024)) with smaller models and shorter timesteps (e.g.. CIFAR-10: 1.61 MB, Acc=95.06 06%. T=2; CIFAR- 100: 1.79 MB, Acc=75.13% T=2). On TinylmageNet, using the same timesteps and architecture, QP-SNN reduces model size by 90.26% and increase accuracy by 3.71% 71% compared to (Li et al.) (2024)). On ImageNet, we are the first study to report results for structured pruning in SNNs, achieving comparable performance unstructured
ANN2SNN conversion and hybrid algorithms, QP-SNN achieves
7

---

Published as a conference paper at ICLR 2025
Table 1: Performance comparison on static and neuromorphic datasets. Note: 'H', 'D', and 'C' represent hybrid, direct, and conversion learning, respectively. 'HardF" denotes whardware hhrdwlle-friendly
Dataset
Method
Network
Train
Bits
HardF
Accuracy(%))
Timestep
Size (MB)
Chowdhury et al.
2021
H
32
90.02
100
44.52
VGG-9
DCNN21
H
5
88,60
25
12.59
Deng et al.
2021
D
32
X
90.19
8
62.16
7Con22FC
NNS221
D
3
X
87.59
8
5.84
CIFAR-10
Shi et al.
2024
D
32
X
92.63
8
50.28
6Conv2FC
JCL224
32
X
90.65
28.40
2024
32
91.67
17.32
VGG-16
[CML22
32
90.26
5.68
8,4,2
95.12, 95.41, 95.06
2
6.27, 1.61
ResNet20
Proposed QP-SNN
D
8,4,2
94.56, 94.65, 94.44
2
3.92, 1.98, 1.02
D
8,4,2
V
91.98, 91.90, 91.61
4
4.28, 2.16, 1.10
VGG-16
D 4,
/
91.30, 91.19, 90.59
1,45, 0.74, 0.39
Chowdhury et al.
2021
H
32
67.80
50
75.90
VGG-11
DCN22
H
5
/
66.20
30
25.43
Deng et al.
2021
D
3
X
57.83
8
11.75
7Con22FC
CIFAR- 100
TNN2227
1
X
55.95
5.99
Shi
2024
32
X
72.34
13.18
ResNet18
[CLR24
32
X
70.45
4
7.67
2024
32
65.53
4
14.40
VGG-16
[CMT22
D
32
64.64
4
9.48
D
8,4,2
75.29,
75.77, 75.13
2
6.45, 3.35, 1.79
ResNer20
Proposed QP-SNN
D
8,4,2
/
74.78,
74.73,
2
4.10, 2.17, 1.20
D
8,4,2
66.69,
66.21, 65.69
4
2.48, 1.35, 0.79
VGG-16
D
8,4, 2
V
64.70, 64.22, 63.08
1.85, 1.04, 0.63
Tiny TinyImageNet
Kundu al.
2021
VGG-16
32
X
52.70
150
24.21
MACV27
Li al.
2024
D
32
49.36
4
27.92
VGG-16
[CML24
D
32
49.14
4
19.76
D
8,4, 2
53.32, 53.11, 53.07
4
5,90, 3.78, 2.72
Proposed QP-SNN
VGG-16
D
8,4, 2
/
51.99, 51.78, 51.67
4
4.67, 3.17, 2.41
32
X
61.89
15.72
ImageNet
Shi 2024
ResNet18
32
X
60.00
12.40
CLL24
32
X
10.48
8
61.36
13.28
Proposed QP-SNN ResNet18
4
V
58.06
4
7.71
Shi 2024
D
32
X
81.90
10
14.08
DVS-CIFAR I0
VGGSNN
CLR24
D
32
X
78.30
10
7.24
Li et al.
2024 3ConvIFC
D
32
73.00
20
3.92
[CML24
D
32
71.90
20
0.32
D
8,4,2
82.10, 81 31,80, 81.30
10
1.61, 0.90, 0.55
Proposed QP-SNN VGGSNN
D
8,4,2
81,50, 80.90, 80.50
10
1.05, 0.62, 0.41
8,4,
/
75.90, 75.40, 74.90
10
0.40, 0.29, 0.24
pruning method (Shi et al. (024)) with 15.55% reduction in model size. On DVS-CIFAR 10, QP-
SNN achieves an 88.55% smaller model size and a 0.2% higher accuracy compared to (Shi et al. plot comparison results in Figure These results show that QP-SNN achieves superior results in both efficiency and performance, positioning it leading approach for compact and hirf-performance SNNs.
2024 ) To intuitively demonstrate SNN's improvements,
8

---

Published as a conference paper at ICLR 2025
5.3 SCALABILITY TO COMPLEX ARCHITECTURES AND TASKS
QP-SNN can be extended to complex architectures like Transformer and complex tasks like ob- ject detection. To validate this scalability, we conduct two additional experiments: (1) using the Spikingformer (Zhou et al. (2023)) architecture and (2) applying QP-SNN to object detection task.
Scalability to the Spikingformer. We use the Spikingformer-4-8888 architecture and evaluate on the CIFAR-100 dataset. The training setups are consistent with (Zhou et al. (2023)) As shown in Table 2 QP-SNN reduces model size by 8.28x, SOPs by 2.25 x, and power consumption by 2.25x while achieving competitive accuracy of 76.94%. These results demonstrate the effectiveness of QP-SNN in compressing complex Spiking Transformer architectures.
Table 2: Performance of QP-SNN on CIFAR-100 with the Spikingformer architecture.
Architecture
Method
Connection Bit Model size (MB)
SOPs (M) Power (mJ) Accuracy (%)
Spikingformer
Zhou et al. (2023
100%
16
18.64 base
292.14 14 bas
0.266 base
79.09 buse
Spikingformer
QP-SNN
44.74%
4
2.25
8
130.05 225x
0.118 x.25
76.94 -2.15
Scalability to object detection. We conduct object detection experiments datasets: SSDD (Wang et al. (2019)) and NWPU VHR-10 (Cheng et al. (2017). SSDD consists of synthetic aperture radar images for ship detection, while NWPU VHR-I0 is a high-resolution remote sensing dataset containing ten object classes. We use the YOLO-v3 detection architecture with ResNet10 as the backbone. During training, we apply pruning to the backbone and optimize the model using SGD with polynomial decay learning rate schedule, initializing at le-2 for 300 epochs. As shown in 3 QP-SNN significantly reduces model size while maintaining satisfac- tory detection performance, demonstrating its potential for more challenging tasks.
remote sensing
Table 3: Object detection results of QP-SNN on SSDD and NWPU V0R-10.
Dataset
Method
Bit
Model size (MB)
m.P@0.5 (%)
Fuee-precision QP-SNN
32
19.29
96,80
SSDD
2.15 8.97x
97.10 +0.3
Full-precision QP-SNN
32
19.29 base
89.89 base
NWPU VHR-10
2.15
8.97x
86.68 3.21
5.4
ABLATION STUDY
To prove the effectiveness of QP-SNN, we conduct extensive ablation studies. Firstly, we analyze the three options for y in the ReScaW strategy select the optimal one. Then, we perform thorough ablation experiments to validate the effectiveness of the proposed ReScaW strategy and SVS-based pruning criterion. Finally, we visualize the effect of the ReScaW strategy and the SVS-based cri- terion to demonstrate that they have effectively addressed the above mentioned issues. All ablation experiments are conducted on the CIFAR- 100 dataset using ResNet20 with 1.20 MB model size.
Analysis of three options for % We compare the performance of quantized SNNs (not involve pruning process) with different settings to determine the optimal one as the default experimental setting. As depicted in Figure 5(a)) the quantized SNN using mar([W')) achieves an accuracy of 77.85% the one using V (W") achieves an accuracy of 77.8%, and the one using [|'ll///'' achieves an accuracy of 79.16%. Clearly, the accuracy differences between them are minimal, with the 1-norm mean value performs best. This may be because W | / can effectively capture the
characteristics of the full precision distribution (Rastegari al. (2016): Qin Qin al. (2020). Therefore,
we choose the 1-norm mean value the default experimental setting.
Effectiveness of two proposed methods. As shown in Table we conduct extensive ablation ex- periments to validate the effectiveness of two proposed methods in QP-SNN, i.e., the ReScaW strat- egy and the SVS-based pruning criterion. First, we demonstrate the effectiveness of the ReScaW strategy. A and B are models that apply vanilla and ReScaW quantization for SNN respectively, without involving pruming process. Their comparison shows that ReScaW-based quantization im- proves the performance of the quantized SNN by 0.63% over vanilla quantization, highlighting its effectiveness. Moreover, the comparison between Models and indicates that merely replac- ing the quantization in the baseline also results in significant performance gain of 4.24%. further

---

Published as a conference paper at ICLR 2025
Conv4
Conv10
Convi6
78.6
77.2
71.85
79-16
Refccll sased
Vastin
4
Weaee
Vaniaa
1.0
Vanilla
Rescat-baard
75.8
74.4
0.5
ma)
v.(W)
Iw"
(W]
VOG-16
VGG-15
VGG-I6
(a) Analysis of three options for
0
-2
0.0
Jct
Lave?2
Lave3 Cony
RuSesWhhood
RodeeW
ReSeeWehaod
Butch indox
2
3000
1.0
3
0.5
2
RoaNett
RodNa20
15
22
36 44
50
64
0.0
Chamnel Indes
(b) Kernel scores with SVS critcrion
(c) Weight distribution with ReSca W-hased quantization
Figure 5: Visualization of ablation experiments.
Table 4: Ablation study on the effectiveness two proposed methods. Note: 'Increment' represents the accuracy improvement relative to the specified model; '.w/"" denotes 'replaced with'.
Model
Accuracy(%o
Compared to Increment(56)
A. only vanilla quant
78.53
B. only ReScaW quant
79.16
4.
0.63 1
C. baseline
69.16
D. Lw./ ReScaW
73,40
C.
4.24
E. r.w./ SVS
73.32
C.
4.16 f
r.w./ ReScaW & SVS
73.89
C.
4.73
validating the ReScaW strategy. Second, we demonstrate the effectiveness of the SVS-based prun- ing criterion. The comparison between Models C and E reveals that the SVS criterion enhances the baseline performance by 4. 16%, confirming ability to remove kernels accurately and preserve model performance. By integrating these two methods into baseline, the performance is significantly improved by 4.73%, underscoring their importance for preserving QP-SNN performance.
Impact of the ReScaW strategy and the SVS-based pruning criterion. To demonstrate that the ReScaW strategy and SVS-based criterion effectively address the previously mentioned issues, we present the weight distribution and importance scores of QP-SNN. We depict the weight distribu- tion of QP-SNN in Figure(5(c) [((] This indicates that ReSca W-based quantization results in a broader weight distribution compared vanilla quantization, indicating improved bit-widtt utilization effi- ciency. In addition, Figure (b) depicts the importance scores of QP-SNN, showing that the SVS- based pruning criterion produces stable scores with minimal fluctuation across different inputs. This input-insensitive characteristic enables QP-SNN to remove redundant kernels accurately. Complete visualization of weight distributions and kernel scores are provided in Appendix E.2 and
CONCLUSION
SNNs offer energy-efficient solutions for artificial intelligence. However, the current SNN commu- nity focuses mainly on building large-scale SNNs to increase performance, which limits their feasi- bility in essurce-constrained edge devices. To tackle this limitation, we first developed QP-SNN baseline using uniform quantization and structured pruning, which significantly reduces resource usage. Furthermore, we analyzed and addressed the underlying issues of the QP-SNN baseline in quantization and pruning to improve performance. For quantization, we revealed that the vanilla uniform quantization suffers from limited representation capability due to inefficient bit-width uti- lization and proposed weight rescaling strategy to resolve it. For pruning, we observed that the SCA criterion exhibits low robustness on inputs and introduced a novel criterion using the singular value of spike activity to remove unimportant kernels more accurately. By integrating the ReScaW and SVS pruning criteria, our QP-SNN achieved superior efficiency and performance, demonstrating its potential for advancing neuromorphic intelligent systems and edge computing.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported in part by the National Natural Science Foundation of China under grant U20B2063, 62220106008, and 62106038, the Sichuan Science and Technology Program under Grant 2024NSFTD0034 and 2023YFG0259, the Open Research Fund of the State Key Laboratory of Brain-Machine Intelligence, Zhejiang University (Grant No.BMI2400020).
REFERENCES
Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-Icaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, et al. Truenorth: Design and tool flow of 65 mw million neuron programmable neurosynaptic chip. IEEK transactions on computer-aidel mided design of integrated circuits and systems,
2015.
Kirk Baker. Singular value decomposition tutorial. The Ohio State University, 24:22, 2005.
Guillaume Bellec, David Kappel, Wolfgang Maass, and Robert Legenstein. Deep rewiring: Training very sparse deep networks. arXiv preprint 171105 2017.
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint
2013.
Zhaowei Cai, Xiaodong Hc, Jian Sun, and Nuno Vasconcelos. Deep learning with low precision by hale-wave gaussian quantization. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5918-5926, 2017.
Honglin Cao, Zijian Zhou, Wenjie Wei, Ammar Belatreche, Yu Liang, Dehao Zhang, Malu Zhang, Yang Yang, and Haizhou Li. Binary event-driven spiking transformer. arXiv preprint arXiv:2501.05900 059904, 2025.
Yanqi Chen, Zhaofei Yu, Wei Fang, Zhengyu Ma, Tiejun Huang, and Yonghong Tian. State tran- sition of dendritic spines improves learning of sparse spiking neural networks. In International Conference on Machine Learning, pp. 3701-3715. PMLR, 2022.
Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Bench- mark and state of the art. Proceedings of the IEEE, 185510):1865-1883, 2017.
Sayeed Shafayet Chowdhury, Isha Garg, and Kaushik Roy. Spatio-temporal pruning and quantiza- tion for low-latency spiking neural networks. In 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-9. IEEE, 2021.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint 1e0.:v::800.09955 2018
Mike Davies, Narayan Srinivasa, Tsung- Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, al. Loihi: neuromorphic manycore processor with on-chip learning. leee Micro, 38(1):82-2)) 2018.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-25825555555 255. Icve, 2009.
Lei Deng, Yujie Wu, Yifan Hu, Ling Liang, Guoqi Li, Xing Hu, Yufei Ding, Peng Li, and Yuan Xie. Comprehensive snn compression using admm optimization and activity regularization. JEEE transactions on neural networks and learning systems, 34(6))2771-288) 2021.
Shikuang Deng, Yuhang Li, Shanghang Zhang, and Shi Gu. Temporal efficient training of spiking neural network via gradient re-weighting. arXiv preprint arXiv:2202. /946, 2022.
Terrance DeVries. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv: 770. 04552, 2017.
11

---

Published as a conference paper at ICLR 2025
Yifu Ding, Haotong Qin, Qinghua Yan, Zhenhua Chai, Junjie Liu, Xiaolin Wei, and Xianglong Liu. Towards accurate post-training quantization for vision transformer. In Proceedings of the 30th ACM international conference multimedia, pp. 5380-5388, 2022.
Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems, 34:21056-21069, 2021.
Wulfram Gerstner and Werner M Kistler. Spiking neuron models: Single neurons, populations, plasticity. Cambridge university press, 2002.
Amir Gholami, Schoon Kim, Zhen Dong. Zhewei Yao. Michael W Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural network inference. Low-Power Computer Vision, pp. 291-326 Chapman and Hal//CRC, 2022.
Yufei Guo, Xinyi Tong, Yuanpei Chen, Liwen Zhang, Xiaode Liu, Zhe Ma, and Xuhui Huang. Recdis-snn: Rectifying membrane potential distribution for directly training spiking neural net- works. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 326-335, June 2022.
Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance imagenet classification. In Proceedings of the IEEE international conference on computer vision, pp. 1026-1034, 2015.
Yang Hc and Lingao Xiao. Structured pruning for deep convolutional neural networks: A survey.
IEEE transactions on pattern analysis and machine intelligence, 2023.
Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdcep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 29(6):82-97, 201122... 2012.
A. L. Hodgkin and A. F. Huxley. A quantitative description of membrane current and its ap- plication to conduction and excitation in nerve. The Journal of Physiology, pp. 500-544. doi: 10.1 11..111/ppyysiool1922.pp/pp
HtRLhttp://at
http:/xxxxoi..coo wtt.:..oo..org/1.1111//////
Sp50.760
Yangfan Hu, Qian Zheng, and Gang Pan. Bitsnns: Revisiting energy-efficient spiking neural net- works. JEEE Transactions on Cognitive and Developmental Systems, 2024.
Eugene M Izhikevich. Simple model of spiking neurons. IEEE Transactions on neural networks, 14(6):1569-155) 15722 2003.
Sambhav Jain, Albert Gural, Michael Wu, and Chris Dick. Trained quantization thresholds for accurate and efficient fixed-point inference of deep neural networks. Proceedings of Machine Learning and Systems, 2:1 12-1211 1-11822, 2020.
Yousef Jaradat, Mohammad Masoud, Ismael Jannoud, Ahmad Manasrah, and Mohammad Alia. A tutorial singular value decomposition with applications image compression and dimension- ality reduction. 202/ international conference on information technology (ICIT), pp. 769-772.
IEEE, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Uday Kulkarni, Abhishek S Hosamani, Abhishek S Masur, Shashank Hegde, Ganesh R Vernekar, and K Siri Chandana. A survey quantization methods for optimization of deep neural networks. In 2022 international conference on automation, computing and renewable systems (ICACRS), pp- 827-834. IEBE, 2022.
12

---

Published as a conference paper at ICLR 2025
Souvik Kundu, Gourav Datta, Massoud Pedram, and Peter A Beerel. Spike-thrift: Towards energy- efficient deep spiking neural networks by limiting spiking activity via attention ceantion-guidedd compres- sion. In Proceedings of the IEEE/CVF winter conference applications of computer vision, pp- 3953-3962, 2021.
Yann LeCun, Léon Bottou, Genevieve OIT, and Klaus-Robert Müller. Efficient backprop. In Neural networks: Tricks of the trade, pp. 9-50. Springer, 2002.
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. arXiv preprint arXiv: 087000 2016.
Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifarl0-dvs: an event-stream dataset for object classification. Frontiers in neuroscience, 11:244131, 2017.
Yaxin Li, Qi Xu, Jiangrong Shen, Hongming Xu, Long Chen, and Gang Pan. Towards efficient deep spiking neural networks construction with spiking activity based pruning. arXiv preprint arXiv:2406.01072, 2024.
Yuhang Li, Youngeun Kim, Hyoungseob Park, Tamar Geller, and Priyadarshini Panda. Neuromor- phic data augmentation for training spiking neural networks. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part VII, pp- 631-649. Springer, 2022.
Qianhui Liu, Jiaqi Yan, Malu Zhang, Gang Pan, and Haizhou Li. Lite-snn: and efficient spiking neural network through spatial-temporal compressive network search and joint optimization. arXiv preprint arXiv:2401. 14652, 2024.
Designing lightweight
Xinhao Luo, Man Yao, Yuhong Chou, Bo Xu, and Guoqi L.. Integer-valued training and spike- driven inference spiking neural network for high-performance and energy-efficiee object detec- tion. arXiv preprint arXiv:2407.20700 2024.
Eustace Painkras, Luis A Plana, Jim Garside, Steve Temple, Francesco Galluppi, Cameron Patter- son, David R Lester, Andrew D Brown, and Steve B Furber. Spinnaker: A 1-w 18-core system- on-chip for massively-pan paaralllll neural network simulation. IEEE Journal of Solid-State Circuits, 1993):1943-1953, 2013. 2013.
Jing Pei, Lei Deng, Sen Song, Mingguo Zhao, Youhui Zhang, Shuang Wu, Guanrui Wang, Zhe Zou, Zhenzhi Wu, Wei He, et al. Towards artificial general intelligence with hybrid tianjic chip architecture. Nature, 572(7767):06-111, 2019.
Yijian Pei, Changqing Xu, Zili Wu, and Yintang Yang. Albsnn: ultra-low latency adaptive local bi- nary spiking neural network with accuracy loss estimator. Frontiers in Neuroscience, 17:1225871, 2023.
Michael Pfeiffer and Thomas Pfeil. Deep learning with spiking neurons: opportunities and chal- lenges. Frontiers in neuroscience, 12, 2018
Haotong Qin, Ruihao Gong, Xianglong Liu, Xiao Bai, Jingkuan Song, and Nieu Sebe. Binary neural networks: A survey. Pattern Recognition, 105:107281, 2020.
Haotong Qin, Yifu Ding, Mingyuan Zhang, Qinghua Yan, Aishan Liu, Qingqing Dang, Ziwei Liu, and Xianglong Liu. Bibert: Accurate fully binarized bert. arXiv preprint arXiv:2203.06330. 2022.
Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. Xnor-net: Imagenet classification using binary convolutional neural networks. In European conference on computer vision, pp. 525-542. Springer, 2016.
Nitin Rathi, Priyadarshini Panda, and Kaushik Roy. Stdp-based pruning of connections and weight quantization in spiking neural networks for energy-efficient recognition. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 38(4):668-677 2018.
13

---

Published as a conference paper at ICLR 2025
Babak Rokh, Ali Azarpeyvand, and Alireza Khanteymoori. A comprehensive survey model quantization for deep neural networks in image classification. ACM Transactions on Intelligent Systems and Technology, 14(6):1-50, 2023.
Deboleena Roy, Indranil Chakraborty, and Kaushik Roy. Scaling deep spiking neural networks with binary stochastic activations. In 2019 IEEE International Conference on Cognilive Computing (ICCC), pp. 50-58. IEEE, 2019a.
Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575(7784):077-617, 2019b.
Rowayda A Sadek. Svd based image processing applications: state of the art, contributions and research challenges. arXiv preprint arXiv:1211.71022 2012.
Xinyu Shi, Jianhao Ding, Zecheng Hao, and Zhaofei Yu. Towards energy efficient spiking neu- ral networks: An unstructured pruning framework. In The Twelfth International Conference on Learning Representations, 2024.
Martino Sorbaro, Qian Liu, Massimo Bortone, and Sadique Sheik. Optimizing the energy consump- tion of spiking neural networks for neuromorphie applications. Frontiers in neuroscience, 14: 516916, 2020.
Sunil Vadera and Salem Ameen. Methods for pruning deep neural networks. IEEE Access, 10: 63280-63300, 2022.
Shuai Wang, Dehao Zhang, Ammar Belatreche, Yichen Xiao, Hongyu Qing, Wenjie We, Malu Zhang, and Yang Yang. Ternary spike-based neuromorphic signal processing system. arXiv preprint 4071v2200..05310, 2024a.
Shuai Wang. Dehao Zhang. Kexin Shi, Yuchen Wang, Wenjie Wei, Jibin Wu, and Malu Zhang. Global-local convolution with spiking neural networks for energy-efficient keyword spotting. arXiv preprint arXiv:2406.15 /3179, 2024b.
Yixuan Wang, Yang Xu, Rui Yan, and Huajin Tang. Deep spiking neural networks with binary weights for object recognition. IEEE Transactions Cognitive and Developmental Systems, 13 (3):514-523, 2020.
Yuanyuan Wang, Chao Wang, Hong Zhang, Yingbo Dong, and Sisi Wei. A sar dataset of ship detection for deep learning under complex backgrounds. remote sensing, 11(7):765, 2019.
Wenjie Wei, Malu Zhang, Hong Qu, Ammar Belatreche, Jian Zhang, and Hong Chen. Temporal- coded spiking neural networks with dynamic firing threshold: Learning with event-driven back- propagation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10552-10562, 2023.
Wenjie Wei, Yu Liang, Ammar Belatreche, Yichen Xiao, Honglin Cao. Zhenbang Ren, Guoqing Wang, Malu Zhang, and Yang Yang. Q-snns: Quantized spiking neural networks. arXiv preprint arXiv:2406. 13672, 2024a.
Wenjie Wei, Malu Zhang, Jilin Zhang, Ammar Belatreche, Jibin Wu, Zijing Xu, Xuerui Qiu, Hong Chen, Yang Yang, and Haizhou Li. Event-driven learning for spiking neural networks. arXiv preprint 4r3iv:2403.0027 2024b.
Jibin Wu, Chenglin Xu, Xiao Han, Daquan Zhou, Malu Zhang, Haizhou Li, and Kay Chen Tan. Progressive tandem learning for pattern recognition with deep spiking neural networks. IEEE Transactions Pattern Analysis and Machine Intelligence, 44(11):724--844, 2021.
Yujie Wu, Lei Deng, Guoqi Li. Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training regh-performance spiking neural networks. Frontiers in neuroscience. 12:331, 2018.
Qi Xu, Yaxin Li, Jiangrong Shen, Jian K Liu, Huajin Tang, and Gang Pan. Constructing deep spiking neural networks from artificial neural networks with knowledge distillation. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 7886-7895, 2023.
14

---

Published as a conference paper at ICLR 2025
Sheng Xu, Anran Huang, Lei Chen, and Baochang Zhang. Convolutional neural network pruning: A survey. In 2020 39th Chinese Control Conference (CCC). pp. 1458-7463. IEEE, 2020.
Jiaqi Yan, Qianhui Liu, Malu Zhang, Lang Feng, De Ma, Haizhou Li, and Gang Pan. Efficient spiking neural network design via neural architecture search. Neural Networks, 173:106172. 2024.
Man Yao. Jiakui Hu, Tianxiang Hu, Yifan Xu, Zhaokun Zhou, Yonghong Tian, Bo Xu, and Guoqi Li. Spike-driven transformer v2: Meta spiking neural network architecture inspiring the design of next-generation neuromorphic chips. arXiv preprint arXiv:2404.0300 03663, 2024a.
Man Yao, Jiakui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo Xu, and Guoqi Li. Spike-driven transformer. Advances in neural information processing systems, 36, 2024b.
Hang Yin, John Boaz Lee, Xiangnan Kong, Thomas Hartvigsen, and Sihong Xie. Energy-efficient models for agh-dimensiooal spike train classification using sparse spiking neural networks. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp- 2017-2025, 2021.
Zhaofei Yu, Jian Liu, Shanshan Jia, Yichen Zhang, Yajing Zheng, Yonghong Tian, and Tiejun Huang. Toward the next generation of retinal neuroprosthesis: Visual computation with spikes. Engineering, 6(4):449-461, 2020.
Dehao Zhang, Shuai Wang, Ammar Belatreche, Wenjie Wei, Yichen Xiao, Haorui Zheng, Zijian Zhou, Malu Zhang, and Yang Yang. Spike-based neuromorphie model for sound source localiza- tion. In The eiirth- Annual Conference on Neural Information Processing Systems, 2024.
Malu Zhang, Hong Qu, Ammar Belatreche, Yi Chen, and Zhang Yi. A highly effective and robust membrane potential-driven supervised learning method for spiking neurons. IEEE transactions on neural networks and learning systems, (0(1):123-137 2018.
Malu Zhang, Jibin Wu, Yansong Chua, Xiaoling Luo, Zihan Pan, Dan Liu, and Haizhou Li. Mpd-al: an efficient membrane potential driven aggregate-labed learning algorithm for spiking neurons. In Proceedings of the AAAI conference artificial intelligence, volume 33, pp. 1327-1334, 2019.
Malu Zhang, Jiadong Wang, Jibin Wu, Ammar Belatreche, Burin Amornpaisannon, Zhixuan Zhang, Venkata Pavan Kumar Miriyala. Hong Qu, Yansong Chua, Trevor E Carlson, et al. Rectified linear postsynaptic potential function for backpropagation in deep spiking neural networks. IEEE transactions on neural networks and learning systems, 33(5): 2021a.
Xiangguo Zhang, Haotong Qin, Yifu Ding, Ruihao Gong, Qinghua Yan, Renshuai Tao, Yuhang Li, Fengwei Yu, and Xianglong Liu. Diversifying sample generation for accurate data-free quanti- zation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 15658-15667, 2021b.
Yichi Zhang, Zhiru Zhang, and Lukasz Lew. Pokebnn: A binary pursuit of lightweight accuracy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 12475- 12485, 2022.
Hanle Zheng, Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li. Going deeper with directly-traines larger spiking neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11062-11070, 2021.
Chenlin Zhou, Liutao Yu, Zhaokun Zhou, Zhengyu Ma, Han Zhang, Huihui Zhou, and Yonghong Tian. Spikingformer: Spike-driven residual learning for transformer- spiking neural net- work. arXiv preprint arXiv:2304. 1954, 2023.
Chenzhuo Zhu, Song Han, Huizi Mao, and William J Dally. Trained ternary quantization. arXiv preprint arXiv: arXiv:16/2.01064 2016.
15

---

Published as a conference paper at ICLR 2025
A THE OVERALL WORKFLOW OF QP-SNN
We present the workflow of QP-SNN in Algorithm[]] which consists of two main steps: quantization and pruning. First, the initial SNN model undergoes ReScaW-basee uniform quantization, where weights are rescaled and quantized, followed by iterative training with backpropagation. Then, the quantized model is pruned using SVS-based criterion that analyzes spatiotemporal spike activity to identify and remove less important channels. Finally, the quantized and pruned model is fine- tuned restore performance.
Algorithm 1:
The overall workflow of QP-SNN.
Input: Initial SNN model: M = (W), .... W);; Bit width: b; Pruning channel ratio: r;; Number of training epoch: N Nepoct; Number of training iteration per epoch: Irrain.
Output: The trained QP-SNN Magkr
b Step 1: Get quantized SNN My by using the ReSca uniform quantization;
for epoch < to Nepoch do for + 1 to Itran do for / + 1 to L do
7 E {maxx W V. (W").
I [W]] }:
D The selection fixed during training:
W santed w!
> Rescale 32-bit weight parameters to wide range:
W
clammp
wi -= -1,1) + =)):
2(6)
2
for
L +- 1 to T do
Calculate (tt)) s' [t. and ('(t) according to Eq.(I~ 3)
end end
11
12
Perform backpropagation and update the quantized model parameters Mgi
13
end
14 end
15 b Step 2: Get the pruned QP-SNN Makp with the SVS-based pruning criterion;
16 for + to I. do
17
Initialize an array F;
18
for [ <- 1 vi
19
Perform a inference process with mini-batch data, get spatiotemporal spike activity;
Get the singular value matrix E: 1 ΣT= [4 = PE Q
20
Score(W<h) ED (Em min(H, I(0, e)):
21
Fif] Score(WWi]])
23
end
24
Iprua [r:ni]: nul; Sindex = (F :]
b Select kernels with high score;
25
Glop Kernels with index in Sindez;
end
27
function MannetumeMMaapp
D Fine-tune the pruned model optimize performance.
B ANALYSIS ON THE ORDER OF QUANTIZATION AND PRUNING
When more model lightweight techniques are employed, compatibility issues often arise, such as the order of applying these techniques and the training strategies involved. In this paper, we adpot the "quantize first, then prune' strategy based on the following two considerations. First, this strategy can better guarantee the effect of pruning technique. Specifically, if pruning is applied be- fore quantization, important convolutional kernels identified in the full-precision parameter domain may become misaligned after quantization, as the quantization reintroduces additional errors. In contrast, by quantizing first and then pruning, redundant convolutional kernels are identified directly in the target low low-precision parameter domain. This order allows for more accurate identification and preservation of critical kernels. Second, this strategy significantly reduces training overhead. Pruning before quantization requires three weight updates: "full-precisio" SNN training pruning with fine-tuning - quantization with fine-tuning,' while 'quantize first, then prune' only requires two adjustments: "quatized SNN training-> pruning with fine-tuning."
16

---

Published as a conference paper at ICLR 2025
In addition to the theoretical analysis, have also conducted experiments by reversing the order of quantization and pruning, termed PQ-SNN, to validate the effectiveness QP-SNN. Experiments performed on the CIFAR- 100 with ResNet20 under the bit-width of 4. We summarize the experi- mental results in 5 from which two conclusions can be obtained. First, the proposed ReScaW and SVS can improve both performance, regardless of the order in which they applied, leading to a 1.83% improvement in PQ-SNN and a 4.46% improvement in QP-SNN. This proves the ef- fectiveness of our ReScaW and SVS methods. Second, QP-SNN achieves the highest performance (surpassing PQ-SNN by 1.39%), demonstrating that 'quantize first, then prune' is more effective.
Table 5: Ablation study on the order of quantization and pruning.
Method
PQ-SNN baseline
PQ-SNN
QP-SNN baseline
QP-SNN
Accuracy
71.51%
73.34 (vaselinet1
70.27%
74.73 (5.73% (bscllinnnn+4444
C EFFICIENCY VALIDATION OF QP-SNN
Model compression aims optimize efficiency during the inference phase, facilitating efficient deployment on resource-constraine devices. Therefore, we present the key efficiency metrics of QP-SNN during inference, including model size, SOPs, power consumption, and accuracy, verify the efficiency advantage of QP-SNN.
We first present a comparison of our model with the full-precision uncompressed SNN counterparts. The results are summarized in Table 6 We acknowledge that our method exhibits accuracy loss compared to uncompressed SNNs. However, this performance degradation is common challenge in the field of model compression. Fortunately, QP-SNN demonstrates satisfactory performance under extreme compression ratios. For example, on the CIFAR- 10 dataset, under the extreme con- nection ratio of 9.61 %, QP-SNN reduces the model size by 98. 74%, SOPs by 78.69%, and power consumption by 77.45%, while the accuracy decreases by only 2.44%. This trade-off between per- formance degradation and resource efficiency is highly advantageous in edge computing scenarios.
Table 6: Efficiency metrics comparison of QP-SNN with full-precision uncompressed SNN.
Architecture Connection Bit Model size (MB)
SOPs (M)
Power (mJ)
Accuracy
VGG-16
100%
58.88
54.60
0.204
93.63%
CIFAR-10
VGG-16
9.61%
0.74
11.63
0.046
91.19%
ResNet20
100%
32
68.4
415.64
0.756
79.49%
CIFAR-100
ResNet20 22.69%
4
2.17
131.53
0.126
74.73%
We then add comparison of our method with related studies on C1FAR-10. Experimental results are shown in Table 7 It can be seen that QP-SNN exhibits competitive SOPs compared to compression work in the SNN domain, and exhibits extremely low model size due quantization. Moreover, it worth noting that the advanced works (Deng et al. (202T); Shi et al. (2024)) focus on unstructured pruning, which typically achieves higher sparsity and performance but requires specialized hardware support. In contrast, our work adopts uniform quantization and structured pruning, balancing the advantages of sparsity, performance, and hardware compatibility.
Table Efficiency metrics comparison of QP-SNN with related studies on the CIFAR- 10 dataset.
Method
Architecture
Time step
HardF
Model size (MB)
SOPs (M)
Deng et al.
2022
(INNLS21]
7Conv2FC
8
X
62.16
107.97
Shi et al.
(004
[ICLR24]
6Conv2FC
8
X
33.76
11.98
Li et al.
2024
[CCML24)
VGG-16
4
5.68
QP-SNN
VGG-16
4
0.74
11.63
17

---

Published as a conference paper at ICLR 2025
D LEARNING ALGORITHM FOR QP-SNN
In this section, we introduce the learning algorithm for the QP-SNN. We use the spatio-temporal backpropagation (STBP) (Wu et al. (2018) ) and the straight-through estimator (STE) (Bengio et al. (203)) methods to solve the dioddiffereniiaabiityy of the spike generation function and quantization.
Training QP-SNNs requires calculating the gradient of the loss function with respect to the synaptic weight. In this work, both spatial and temporal dimensions. By applying the chain rule, STBP computes the derivative of the loss function L with respect to synaptic weights W through the following equation,
the STBP learning algorithm, which performs gradient propagation in
OC
T
dc
Ow
0
Ow!
(14)
[++1
+ 1] 0S
[| au+
au
as
as
aU
aw
T<T E--
ac is obtained in iterative where the derivative of the loss function with respect to the spike osi+1[]]
|btt][+]] manner. The terms of (0 [0*+1[+]] and oS- can be computed from Eq. Unfortu-
aw "UU T[8]
nately, the direct training of SNNs faces a distinct challenge due to the non-differentiable nature of the spiking (i.e. firing) mechanism. Specifically, the term of au represents the gradient of the spike generation function (described in Eq. (2)). This function evaluates to infinity at the moment of spike emission and to zero elsewhere, making incompatible with the traditional error back- propagation used in ANN training. STBP addresses this non-differentiabiliy problem by employing surrogate gradients to approximate the true gradient [u et al. (2018). this paper, we use the trian- max (a - -U
ag(+1(0
gular surrogate function (Deng al. (2022)), described as a
01,
aU where a is the coefficient that controls the width of the gradient window. In this paper, we use the cross-entropy loss function to access the difference between the predicted probability distribution
and the true label, given by, C = - -N Vi log
where Nr. is the number cxp(+ ET ((414)
E, Σ!- U/:1()
of classes and y/ E (0. 1} is the label for the i-th neuron in the last layer. Moreover, to solve the non-differentiability of quantization, we use the STE method (Hinton et al. (2012); [Bengio et al. [wi By using the surrogate gradient function and STE, the proposed QP-SNN can be trained directly with gradient backpropagation.
22033) ) expressed as, aw! int
aw
E COMPLETE WEIGHT DISTRIBUTION COMPARISON
E. VANILLA UNIFORM QUANTIZATION
We present the weight distributions of models utilizing vanilla uniform quantization across multiple datasets and architectures, such as ResNet20 on CIFAR100, VGG-16 on TinylmageNet, and VG- GSNN on IVS-CIFAR1O. In addition to the weight distribution, we also label the 1st and 99th percentiles of each layer's 's weights in the figure to determine the value of a. Based on the value of
s(b)-a++1 in Sec. 4.1 calculate the bit-width utilization for each and the utilization rate equation .(b)+1 model. these calculations, we consider an 8-bit weight configuration, i.c., s(b) = 256.
The weight distribution of ResNet20 on the CIFAR-100 dataset is presented in Figure 6 It can be seen from this figure that only the weight distribution of the first layer is relatively wide, with an value of 0.7, which corresponds for the subsequent layers are predominantly around 0.2, resulting in a significantly lower bit-width utilization rate of approximately 20.31%.
bit-width utilization rate of 70.12%. In contrast, the value
18

---

Published as a conference paper at ICLR 2025
Convl
Layeel II, Convi
Layert O Conv?
L LoCov Ps We 6200 Pr (e - 4.30
Layerl L.Cov22
1.0
"uel) 1225
Poay Wy was Paat 4.15
Pusl .a.5
Paa (W) -4.4
PacliW)) - 40.9
1.0
0.5
0.5
Redero
0.0
0.0
0C3
0.0
0.5
0.3
0.3
0.5
0,3
0.3
0.3
0.5
Laverl Comy
Lavar 2 Comy?
Laver? n Canel
Lo
Laver? l Como?
Lavar2 Conv
Pa mi2 Po 422
Pr syd saiis
Pn -611
Pas (W) 4:20
Pagi
Pr (W) + ais
7.5
0.5
2.5
20
RoNe02
ReaNc.0
-1.3
0.0
-03
0.5
-035
0.
0.
0.0
-33
0.0
0.0
0.3
0.3
0.5
Laver2.1 Cony2
Laver 2.2 Convi
Jet
Layer2.2 2. Gany2
3.yer Canvl
Loyes3 O. Cony2
Powi Wi) wa1s
SewiW) 8:13
Paay WP 203
Pt Wi wiis
Pown W7 2014
7.5
Po -135
7.5
Paxi
Pg (W) 2015
5.0
5.0
5.0
2.5
2.5
2.5
RoNN:.00
20
ReaNe12
Ro
0.0
0,0
0.0
0.3
0.5
Laver5. Conyl
Lager Comy?
1st Layer? 2 Conel
1ct Lavert 2 Cone?
Paay WP
Ps W))
PW' -135
(W dile
Par W
Rg
RooNedn
Rocher2
-0.5
0.0
-0.5
0.0
-0.5
0.0
-0.5
0.5
Figure 6: Weight distribution: vanilla uniform quantization, ResNet20, CIFAR-100.
The weight distribution of VGG-16 on TinyImagenet is illustrated in Figure 7 From this figure, it can be revealed that the weight distribution of each layer is broader than ResNet20 on CIFAR- 100. However, the maximum a value is 0.64, which corresponds to a bit-width utilization rate of approximately 44%. This result indicates it is still quite far from full utilization.
Coav1
Comv3
Coned
Camo
Su
Paye Wi
1.S
1.0
VOG
0.5
VGG2
0.0
VGB 10
0.5
0.0
0.5
5400
0.0
0.0
0.5
0.0
10
Po 6/2
PW) 4:17
1.0
8:13
3.0
(W) 0.57
V00 5
V0G 5
X000-4
Y00
140 15
1.3
0.0
033
0.0
-13
010
0.5
0.5
0.1
-0.
0.0
Co
Co
Conyl6
1.0
Pw -3.35
1.0
WW 4,37
1.0
PW -8.20
0.5
0.5
0.5
0.0
0.5
0.0
0.5
0.0
0.5
0.0
0.5
0.0
3.5
0.0
0.5
Figure 7: Weight distribution: vanilla quantization, VGG-16, TinyImageNet.
The weight distribution of VGGSNN on DVS-CIFARIO is displayed in Figure X from the figure, the maximum a value is 0.44 (Convl), 1), corresponding a bit width utilization rate of 44.22%. Moreover, the a value of subsequent layers is mainly around 0.3, resulting in a significantly lower bit width utilization rate of about 30.27 20.27%
As can be seen
19

---

Published as a conference paper at ICLR 2025
Convill
Cony
Com 3
Comes
Canvé
w/W
4.30
PaxtW)
Fra)(W)
0.5
VGG-16 16
VGG-15
VGG-00 -6
VGG 16
VGG-15
0.0
-.3
0.0
0.5
0.5
0.0
0.5
0
0.0
0.3
0.0
0.0
0.5
Cony?
Conv
1o
Pa m/ 1224
Ps ProdWWe 0.25
WW) 1224
(o -0.11
Peuc WW 4133
Poul -0.00
VOG-16 49
V65-15
VCK--1 1
Figure 8: Weight distribution: vanilla quantization, VGGSNN, DVS-CIFARIO.
Clearly, these weight distributions prove the inefficient bit-width utilization of vanilla uniform quan- tization. This inefficiency leads to substantial number of floating-point weights being discretized on the same integer grid during the quantization process, thus reducing the discrimination of the quantized weights. Consequently, this reduction adversely impacts the network's 's representational capacity and overall performance.
E.2 RESCAW-BASER UNIFORM QUANTIZATION
Convl
Laver] CoCol
Laverl 0.Con22
Layerl L.Cmvl
Layerl. L.Com
Rosod
have Veailla
RodkaW Rosod Vasilla
Redunl hood
Yesill
Vaslll
Vaslls
2
Ro0
Raz2
RonNa
ReeNart0
RecNes20
2
2
2
1 2 Conel
L 2. Cony
Laver2 O Convl
2aver2 O Cony2
Laver Comvl
hoood Vanill
Reka W. hourd
Itekca baodd Veailla
Itdicaai Roard Stalll
Rekoow taed
Veaill
Vaaills
1.0
2
2
0.5
RoNe
Ro2a20
RaaNa20
RooNaako
RecNaz10
-2
0.0
Liver211 1. Comv
Laver2.2 2. Convl
Lever2 2 Conv2
Laver L. Convl
Laver 3.0. Conv2
Refiar--lasce Veilla
Rekia basse
Mekca Wi based
Rebeaw- www.
Vellla
Varills
Neatll
1.0
0.5
ResNe20
RosNa20
Ros
ResNex20
ResNe23
0.0
L.ver3.1 1. Cooavl
Lover3.1 Conm2
Lover2 2 Convi
Laver3 2 Conv2
RaSa Woood
1.0
Vonits
1.0
Vanits
1.0
Vaniil
1.0
Varitla
0.5
0.5
0.5
0.5
Ne00
Nen
RocNe220
0.0
0.0
0.0
Figure 9: Weight distribution: ReScaW-based uniform quantization, ResNet20, 100.
We also present comparison of weight distributions between vanilla uniform quantization and ReScaW-based uniform quantization across multiple datasets and architectures. The weight distri- bution of ResNet20 on CIFAR-100 is presented in Figure VGG-16 on TinyImageNet is illustrated
20

---

Published as a conference paper at ICLR 2025
in Figure IO and VGG-SNN on IVS-CIFARIO displayed in Figure 11 demonstrate that the weight distribution using ReScaW. --aaaa-bbddedde quantization is broader than that of vanilla uniform quantization, indicating the more efficient bit-width utilization of our ReScaW.
These three figures clearly
Conco
Convl
Conv3
Conee
Corms
Veuill
Maria
Vaalla
Varila
5.0
1.0
2.5
0.S
VoG+15
VOL--Te
X00-11
Vole-1
Conclo
lowe
1.0
1.0
0.5
0.5
VoG-16
V00-16
0.00
2.0
Convi4
Chael5
Convtó
adiationnd Yeulls
Redoot-bund Vedia
Mards
2
VGG
16
VGG
Figure 10: Weight distribution: ReSca ReScaW-basse W-based uniform quantization, VGG-16, TinyImageNet.
Conel
Cony 3
Cuned
Cami
a.5
VoG-1
Vou-le
VOGUs
V00-16
VGL-15
Conc1o
7.5
7.5
7.5
5.0
2.5
0.0
0.0
Figure 11: Weight distribution: ReScaW-based uniform quantization, VGGSNN, DVS-CIFARIO.
F COMPLETE IMPORTANCE SCORE COMPARISON
F. SCA-BASED PRUNING CRITERION
We present the convolutional kernel scores of models using the SCA criterion across different archi- tectures and datasets, including ResNet20 on CIFAR 100, VGG-16 on TinyImageNet, and VGGSNN on CVS-CIFARID Note that we only display the layers that perform pruning operations, and the colors in these figures represent the value of the importance score. Moreover, to intuitively reflect the robustness of the pruning criterion to input samples, we compute the average cosine similarity of kernel scores between pairs of input batches for each layer in every model. The calculation for the average cosine similarity of 1-th layer is outlined as,
Score, (.Soorree"..))) Score, Score,WW'..)
2
AvgCosS, = NN 1) KCj Σ, Scorc, Sco.r v EE, Score, (Wi.r)2
(15)
where Ng is the number of input batches and Score, is the kernel score for input batch i.
The kernel scores for ResNet20 on CIFAR- 100 are presented Figure I2 It can be seen from this figure that the SCA-based pruning criterion yields varying scores for the same kernel when process- ing different input samples. Furthermore, we calculated AvgCosS, for each layer in ResNet20, and the min; A AvgCosS, is 0.870. This indicates that the SCA criterion is not robust enough to inputs.
21

---

Published as a conference paper at ICLR 2025
Conv1
Layer1.0.Conv/
Layerl. C.Cov22
Layerl.1.Convl
Layer1.1.Com
Beteh index
V/: ResNe120
Res
ReeNee20
ReoNer20
ResNee
15 29
43
57
29
S7
85
113
29
57
85
113
29
57
85 113
29
57
85
113
Layer 1.2.ovl
2ycm2.2.Comvv2
Layer2.0.Com C.novv
Layer2.0.c
2.Cr222
Layer2.1
Convv
Botch index
3 2
3 2
RaNNe20
RoNNe20
ReaNe:20
ResNe20
29 57 85 113
29 57 85 113
50 99 148197226
50 99 197297
1 50 99 148 197246
Leyer2.1.Com's
Layer2.2. Conv1
Sgyer2.2.Conv2
L.yyrr30.Comm
Layer3 0.Con22
Batch index
3 2
2
3 2
3 2
3
ReeNee20
ResNet20
ReeNee00
50 99 148 197 246
50 99 148 197246
50 99 148 197 246
99 197 295393491 491
99 197295 393491
L
Layer3.1.Comd
Layer3 2.Cmvl
L.yer3.2.Comv2
Butch mnde
2
04
3
ResNe20
ResNec2
2c
ResNet20
197 29539349) 491
99 197295 39349]
99 197295 393491
99 197 393339445 491
Channel index
Channel index
Channel index
Channel index
Figure 12: Kernel scores: SCA-based pruning criterion, ResNet20, CIFAR-I00.
The kernel scores for VGG-16 16 on Tinylmagenet are illustrated in Figure 13 We calculate AvgCosS, for each layer in VGG-16, 16, and obtain the min, AvgCosS, is 0.879. this structure, the kernel scores' fluctuation with inputs is slightly better compared to ResNer20, but still not negligible.
Convál
Convl
Conv3
Cony4
Convi
Batch index 3
4
VOG-16
VOG-15
VOG-15
VGG-16
VG0-16
113
99 197246
Conv77
Conv8
Conv 10
Conv 11
Conv12
Butch Index
2
2
3
5
3
VOG-16
V00-16
VOG-15
VGG-16
VOG-16
99 148197246
148 197246
99 197295 393491
197395393491
197295 393491
Conv14
Conv15
Batch index
5 2
VG-
197 29229933444
99 197245 39349] Channel index
Channel index
Figure 13: Kernel scores: SCA-based pruning criterion, VGG-16, TinyImageNet.
As can be seen from the figure, The kernel score's 's fluctuation with input data is better compared to both ResNet and VGG-16, but in deeper layers, the fluctuation is higher. We also calculate AvgCosS, for each layer in VGGSNN, and the min; AvgCosS, is 0.952.
The kernel scores for VGGSNN on DVS-CIFAR-10 are displayed in Figure 14
22

---

Published as a conference paper at ICLR 2025
Convo
Conv]
Convs
Convd
Convi
VEGGSNN
15
29
57
29
S7
K5
113
50
99
148
197246
50 99 148 197246
99 195145191451
Conv?
Conv9
99
97295393449
99 99725534348 Channul index
Channol index
Figure 14: Kernel scores: SCA-based pruning criterion, VGGSNN, DVS-CIFARIO.
These results demonstrate that the SCA-based pruning criterion yields varying scores for the same kernel when processing different input sample, demonstrating low robustness to input samples. This sensitivity to inputs suggests that the criterion may fail to accurately identify critical convolutional kernels within SNNs, potentially impacting the reliability of the pruning process.
F.2 SVS-BASED PRUNING CRITERION
We also depict kernel scores using the SVS pruning criterion. The kemeel score for ResNet20 on CIFAR-100 in Figure 15 VGG-16 on Tinylmagenet in Figure 16 and VGGSNN on DVS-CIFARI0 in T7 We still only display the layers that perform pruning operation. In VGG-16, ResNer20, and VGGSNN, the min; AvgCosS, values are 0.997, 0.993, and 1.000 respectively, which exceed the corresponding min; AvgCosS, when using the SCA-Basee pruning criterion by 13.4%, 14.1%, and 5.0%, respectively. The results demonstrate that the SVS-based pruning criterion yields consis- tent evaluations, with only minor variations between different input samples. This high robustness to input samples enables QP-SNN to effectively identify and eliminate redundant kernels.
Convi
Layer! L.yyer....oovv .d.Col
Layeri.fl Co.v2
Layerl.1.Com
Layerl. 1.Conv2
Bosh index
Ro
No
ResN420
BesN--00
29
113
57
L.verl.2.Comv/
Layer2.0.oomv
Layer2.0 0.nvv2
Layor2. L.oovl
Betch
Na21
Nac00
29 ST 85 113
29 37 K5 [13
50 99 1:88 197246
30 99 148 197246
50 99 145 197246
Layer2. L.Cov2
Layer2.2 Convi
Laver2.2 Conv2
Layer3 O. Convi
Layer3 0.Conv2
Brach
3
Bes
Mea
50
14R
99
395
197 30349)
Layer3.1.Coo LoCovl
Layer3.1.Coom 1.Com22
Layer3.2 Couvl
Layer5.2 2.nnv2
Bonch
Ne100
N422
59 197295 391491
197295 393491
197293 393491
99 19/295 395491
Charnol index
Channol index
Channel index
Channel index
Figure 15: Kernel scores: baseb pruning criterion, ResNet20, 100.
23

---

Published as a conference paper at ICLR 2025
Convü
Conv1
Conv3
Conv4
Conv6
Batch index
3 2
VGG-16
VOC-I6
5
VOGH26 6
VOKH-S
VGGG-1 16
15
29
43
57
15
29
43 57
29
57
85 113
29
57
85 113
50 99 148
197 246
Conv?
Conv8
Conv
Conv
Conv12
Batch index
3
4 3
4
V0G-16
V0G-16
VOG-16
VGG-16
VOG-16
99 148
18899246
50 197246
197299 393-4999999
197 295 393491
197295393491
Conv14
Convi5
1
Batch index
2
3 2
3
4
VGG-16
VGG-16
99 197295393491
99 197295 393491
Channel index
Channel index
Figure 16: Kernel scores: SVS-based pruning criterion, VGG-16, TinyImageNet.
ConvO
Convl
Conv3
Conv4
Conv6
Batch index
2
2
3
3
3
4
4
VGGSNN
VGOOSN
VGGSNN
VGGSNN
VGGSNN
15
43
29
S7
113
50 99 148 197246
50 148 197246
197 09729339344
Conv7
Conv9
Batch indox
3
VGGSNN
VG0SNN
197295 393491
197 295 393491
Channol index
Channel index
Figure 17: Kernel scores: SVS-based pruning criterion, VGGSNN, DVS-CIFARIO.
G EXPERIMENT
Datasets We evaluate method on image classification datasets, including static datasets CIFAR- 10 (Krizhevsky et al. (2009)) CIFAR-100 (Krizhevsky et al. (2009). TinylmageNet (Deng et al. (2009), ImageNet- 1k (Deng al. (2099), and the neuromorphic dataset DVS-CIFARTOLE et al. (017).. Before introducing the experiment setups, we briefly outline each dataset. The CIFAR- 10 and CIFAR-100 are color image datasets, with each dataset containing 50,000 training images and 10,000 testing images. Each image features 3 color channels and spatial resolution of 32x pixels. CIFAR-10 10 is composed of 10 categories, whereas CIFAR-IO1 comprises 100 categories. During the preprocessing process of CIFAR datasets, we apply the commonly used data augmenta- tion techniques (Cubuk et al. 2 Vries (207)). The TinylmageNet dataset is subset of the ImageNet dataset, consisting of 200 categories, with each category containing 500 training images and 50 test images. Each image has color channels and a spatial resolution of 64 64x64 64 pixcls. The ImageNet-IK 1K dataset is large-scale dataset commonly used for computer vision tasks. It spans 1000 classes and contains around 1.3 million training images and 50,000 validation images. The DVS-CIFARI is neuromorphic dataset captured using Dynamic Vision Sensor (DVS) event cam- eras. It is the most challenging neuromorphic dataset, featuring 9,000 training samples and 1,000 testing samples, featuring a spatial resolution of 128 x 128. During the preprocessing process of the DVS-CIFARIO dataset, we apply the data augmentation technique proposed in (Li et al. 2022
24

---

Published as a conference paper at ICLR 2025
Experimental Setups We summarize the training hyperparameters for each dataset in Table g including time step, image resolution, optimizer, and other factors. Additionally, we present the network architectures and the corresponding pruning rates for each module in Table IT In our experiments, we directly utilize the classification head after completing the convolution operations. Therefore, we do not prune the output channels of the last convolutional layer preserve the in- legrity of the classification head. Note that the pruning rates used in our experiments are manually selected, without rigorous design or the application of parameter search methods.
Table 8: Experimental setups.
Hyper-parameter
CIFAR-10/100
TinyImageNet
ImageNet
DVS-CIFARIO
Timestep
4
10
Resolution
32x32
64x64
x24x224
48x48
Batch size
256
256
256
64
Epoch (Train/Fine-tume)
300 150
300 / 150
320 / 200
300 / 150
Optimizer (Train/Fine-tune))
SGD / Adam
SGD / Adam
SGD / SGD
SGD Adam
Initial Ir (Train/Fine-tune)
0.1 / 0.001
0.1 / 0.001
0.1 / 0.05
0.1 / 0.001
Learning rate decay
Cosine
Cosine
Cosine
Cosine
Table 9: Detailed network architecture and the channel pruning ratio for VGG-16.
Channel Pruning Ratio
Layer
Resolution
Channel
Module
CIFAR-10
CIFAR- 100
TinylmageNet
4.25M
1.42M
2.3
1.68M
4.65M
3.43M
Conv
H W
64
0.45
0.49
0.45
0.45
0.45
0.45
-NN-LIF
QConv
2
H W
64
0.45
0.49
0.45
0.45
0.45
0.45
-BN-LIF
3
MaxPool
QConv
H
128
0.45
0.49
0.45
0.45
0.45
0.45
2
2
-BN-LIF
QCony
H X 2
128
0.45
0.49
0.45
0.45
0.45
0.45
2
-BN-LIF
6
MaxPool
7
H x
W
256
QConv
0.45
0.49
0.45
0.45
0.45
0.45
4
-BN-LIF
8
H
4
W
256
-BN-LIF
QCony
0.45
0.49
0.45
0.45
0.45
0.45
QConv
W
256
0.45
0.49
0.45
0.45
0.45
0.45
-BN-LIF
10
MaxPool
QConv
512
0.51
0.8
0.7
0.78
0.51
0.62
8
8
-BN-LIF
QConv
12
H
8
X
512
BN-LIF
0.51
0.8
0.7
0.78
0.51
0.62
13
H
W
512
QConv
0.51
0.8
0.7
0.78
0.51
0.62
-NN-LIF
14
MaxPool
15
W
512
QConv
0.51
0.8
0.7
0.78
0.51
0.62
-BN-LIF
QCony
16
H
512
0.51
0.8
0.7
0.78
0.51
0.62
16
-BN-LIF
QCony
17
16
X
16
5.12
-BN-LIF
25

---

Published as a conference paper at ICLR 2025
Table 10: Detailed network architecture and the channel pruning ratio for ResNet20.
Channel Pruning Ratio
Layer
Resolution
Channel
Module
CIFAR-10 100
6.22M
6.27M
3.87M / 33.2MM
conv0
H X W
64
Conv-BN-LIF
0.1
0.1
QConv-BN-LIF
0.3
0.35
Layer1.0
H X W
128
QConv-BN-Lin
0.3
0.35
QConv-BN-
0.6
0.75
Layer1.1
H X W
128
QConv-BN-LIF
0.3
0.35
QConv-BN-LIF
0.6
0.75
Layer1.2
X W
128
QConv-BN-LIF
0.3
0.35
QConv-BN-LIF
0.6
0.75
Layer2.0
H x W 2 2
256
QConv-NN-LIF
0.6
0.75
QConv-BN-LLP
0.6
0.75
Layer2.1
x 2 2
H W
256
QConv-BN-LIF
0.6
0.75
256 QConv-BN-LIF
0.6
0.75
Layer2.2
H x 2 2
QConv-BN-LIF
0.6
0.75
QConv-BN-LIIF
0.6
0.75
Layer3.0
512
T
QConv-BN-LIF
QConv-BN-LIF
0.6
0.75
Layer3.1
512
4
QConv-BN-LIF
QConv-BN-LIF
0.6
0.75
Layer3.2
512
QConv-BN-LIIF
Table 11: Detailed network architecture and the channel pruning ratio for VGGSNN.
Channel Pruning Ratio
Layer
Resolution
Channel
Module
DVS-CIFARIO
1.46M
0.9M
0.25M
x
W
Conv-BN-LIF
0.5
0.5
0.82
2
W
128
QConv-BN-LIF
0.5
0.5
0.82
3
MaxPool
4
x
256
QCCnv-BN-LIF
0.5
0.5
0.82
d
5
7
x
256
QConv-BN-LIF
0.7
0.8
0.93
6
MaxPool
7
H
x
W
512
QConv-BN-LIF
0.7
0.8
0.93
8
M
x
W
512
OConv-BN-LIE
0.7
0.8
0.93
9
MaxPool
10
H
x
W
512
QConv-BN-LII
0.7
0.8
0.93
11
512
CConv-BN-LIF
Model size calculation
The model size is computed by aggregating the storage requirements of both quantized and full precision parameters, as expressed by the following equation (Qin et al. 2022 Zhang et al. (2022)) )
M = Params x Bitwidth
P P x Bjps
(16)
where Py and Pyp denote the quantized parameters and full precision parameters, respectively, while B, and Byp represent their corresponding bit widths. is important to note that, in our experiments, full-precision weights are employed both the initial convolutional layer and the final fully con-
nected layer to ensure optimal performance (Zhang al. (2021b);; Ding et al. (022))). We also take this configuration into account when calculating our model size.
26


Reviews:
[{"review_id": "fRJHu42u3u", "reviewer": "Reviewer_rh3C", "summary": "This work focuses on the applicability of SNNs on edge devices, integrating weights quantization and structured pruning techniques, and providing an in-depth analysis of the main challenges in each respective area. Specifically, for weights quantization, the work proposes weight scaling; for structured pruning, it introduces a method to remove redundant convolutional kernels by analyzing the temporal-spatial spike activity singular values in the spike feature matrix. This is an effective joint exploration of weight quantization and pruning in SNNs. I appreciate the idea of leveraging quantization, pruning, and event-driven techniques—three approaches that accelerate, conserve energy, and enhance the efficiency of neural networks.", "questions": "1.To my knowledge, the highest-performing architecture in the SNN field is the Spiking Transformer [1-5]. Please discuss whether the proposed method can be effectively applied to the Spiking Transformer. If feasible, please provide some preliminary experimental results.\n\n[1] Zhou, Z., Zhu, Y., He, C., Wang, Y., Shuicheng, Y. A. N., Tian, Y., & Yuan, L. Spikformer: When Spiking Neural Network Meets Transformer. In The Eleventh International Conference on Learning Representations.\n\n[2] Yao, M., Hu, J., Zhou, Z., Yuan, L., Tian, Y., Xu, B., & Li, G. (2024). Spike-driven transformer. Advances in neural information processing systems, 36.\n\n[3] Zhou, C., Yu, L., Zhou, Z., Ma, Z., Zhang, H., Zhou, H., & Tian, Y. (2023). Spikingformer: Spike-driven residual learning for transformer-based spiking neural network. arXiv preprint arXiv:2304.11954.\n\n[4] Zhou, Z., Che, K., Fang, W., Tian, K., Zhu, Y., Yan, S., ... & Yuan, L. (2024). Spikformer v2: Join the high accuracy club on imagenet with an snn ticket. arXiv preprint arXiv:2401.02020.\n\n[5] Yao, M., Hu, J., Hu, T., Xu, Y., Zhou, Z., Tian, Y., ... & Li, G. Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips. In The Twelfth International Conference on Learning Representations.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The illustrations are thoughtfully crafted.\n\n2. The manuscript presents a clear structure, and maintains logical consistency.\n\n3. A detailed analysis of the weight distribution in SNNs provides the motivation for the re-scaling method.\n\n4. The experiments demonstrate the effectiveness of this method on both static and dynamic datasets.", "weaknesses": "1. Regarding the selection of the quantization scaling factor $\\gamma$, this work employs three different sampling methods. Although experiments indicate that scaling with the 1-norm mean yields the best results, I believe there is still room for improvement, such as using a piecewise $\\gamma$. I encourage the authors to provide further explanation.\n\n2. To my knowledge, when two or more model lightweigh techniques are employed, compatibility issues often arise, such as the order of applying these techniques and the training strategies involved. Additional clarification is needed on these aspects.\n\n3. At the end of Algorithm. 1, fine-tuning is required after pruning. The specific details of this fine-tuning process should be provided.\n\n4. In Table. 2 of the experimental section, the baselines for A-F are not consistent. I suggest the authors provide a unified ablation study. Additionally, regarding the compatibility issues of interest, I encourage the authors to conduct an ablation study on the compatibility of quantization and pruning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cnFctnGMdi", "reviewer": "Reviewer_LMLx", "summary": "This paper presents QP-SNN, which integrates quantization and structure pruning methods. The quantization approach is based on hardware-friendly uniform quantization, while the pruning method employs SVD techniques. QP-SNN has been validated on classification datasets, achieving state-of-the-art performance.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The writing of this paper is good and easy to read.\n\n2. The experiments in the paper demonstrate the effectiveness of the proposed ReScaW algorithm and SVD-based pruning.", "weaknesses": "1. The ReScaW-based uniform quantization lacks innovation, as it is merely a combination of uniform quantization and weight scaling [1].\n\n2. The SVD-based pruning appears to be an application of SVD pruning in SNNs. There are many methods utilizing SVD decomposition or selecting important singular values for pruning [2][3]. Please clarify the specific advantages of applying this in SNNs.\n\n3. The SVD decomposition method introduces additional time complexity. How much does it specifically affect the training time?\n\n4. The experiments are insufficient from several perspectives. Firstly, why use the 2021 SEW-ResNet and Spiking ResNet instead of the latest SNN models? Secondly, this method has only been tested on simple classification tasks; without testing on more complex downstream tasks. Thus, it is difficult to prove the potential applicability of this work.\n\n5. As an efficiency-focused study, why does the paper not present sufficiently robust efficiency metrics, such as OPs/SOPs, training time, inference speed, and energy consumption?\n\n6. Why there is no presentation of pruning ratio statistics in the main text, which obscures the specific effectiveness of this aspect?\n\n**Reference**\n\n[1] AWQ: ACTIVATION-AWARE WEIGHT QUANTIZATION FOR ON-DEVICE LLM COMPRESSION AND ACCELERATION.\n\n[2] Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsification.\n\n[3] HRank: Filter Pruning using High-Rank Feature Map.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VpSigpgmCd", "reviewer": "Reviewer_Likb", "summary": "The paper proposes two technologies including a weight-rescaling strategy and an SVS-based pruning criterion for weight quantization and structured pruning of SNN respectively. These technologies reduce the model size significantly while maintains the SNN accuracy.", "questions": "1. What is the effectiveness of this work in saving synaptic operation in SNN inference? The author can use the code in https://github.com/zhouchenlin2096/Spikingformer/tree/master/energy_consumption_calculation to provide the detailed number of synaptic operations of the compressed SNN in this work.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "1. The paper develops a hardware-efficient and lightweight QP-SNN baseline by integrating uniform quantization and structured pruning.\n2. The weight-rescaling strategy and the SVS-based pruning criterion works well on SNN benchmarks.", "weaknesses": "1. The proposed technologies including the weight-rescaling strategy and the SVS-based pruning criterion do not fully utilize the unique characteristics of SNN, such as the sparsity to reduce synaptic operation and temporal information to maintain the SNN accuracy. The technologies can also be used in ANN models.\n2. The comparison with SNNs that are not compressed is missing, such as SEW-ResNet[1] and MS-ResNet[2]. Compared to these works, the accuracy degradation is still large.\n3. The results that the saving of synaptic operation is missing.\n\n[1] Fang W, Yu Z, Chen Y, et al. Deep residual learning in spiking neural networks[J]. Advances in Neural Information Processing Systems, 2021, 34: 21056-21069.\n[2] Hu Y, Deng L, Wu Y, et al. Advancing spiking neural networks toward deep residual learning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kNZ0Qjvjcv", "reviewer": "Reviewer_AehT", "summary": "This paper presents QP-SNN, a novel approach to creating efficient and hardware-friendly Spiking Neural Networks (SNNs) by combining uniform quantization and structured pruning. The authors first develop a baseline model, then address its performance limitations through two key innovations: (1) a weight rescaling (ReScaW) strategy for more effective bit-width utilization in quantization, and (2) a novel pruning criterion based on the singular value of spatiotemporal spike activities (SVS). The experimental results demonstrate that QP-SNN achieves state-of-the-art performance while significantly reducing model size.\nStrengths", "questions": "1. How does the computational cost of calculating singular values in the SVS criterion compare to existing pruning criteria? Is there a significant overhead during training?\n\n2.What is the impact of the proposed methods on the network's spike sparsity?\n\n3.How sensitive is the method to the choice of ε in the SVS-based pruning criterion?\n\nHow does the approach perform when applied to other types of neural networks beyond classification tasks, such as object detection?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Theoretical Foundation:\n- The analysis of bit-width utilization provides clear theoretical justification for the ReScaW strategy.\n- The proposed SVS criterion is well-grounded in mathematical principles.\n\n2. Method Innovation:\n- The paper presents two novel techniques (ReScaW and SVS) that effectively address specific limitations in existing quantization and pruning approaches.\n- The solutions are well-motivated by careful analysis of the underlying problems.\n\n\n3. Experimental Validation:\n- Comprehensive experiments across multiple datasets and architectures.\n- Detailed ablation studies that validate each component's contribution.", "weaknesses": "1.The selection process for pruning rates appears arbitrary and lacks systematic justification. \n2.This paper is only verified on the simple computer vision classification task.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)