MI-TRQR: Mutual Information-Based Temporal Redundancy Quantification and Reduction for Energy-Efficient Spiking Neural Networks
Authors: Dengfeng Xue
Wenjuan Li
Yifan Lu
Chunfeng Yuan
Yufan Liu
Wei Liu
Man Yao
Li Yang
Guoqi Li
Bing Li
Stephen Maybank
Weiming Hu
Zhetao Li
Venue: NeurIPS 2025 poster ()
Keywords: spiking neural networks
redundancy quantification
redundancy reduction
mutual information
attention mechanism
Primary area: neuroscience_and_cognitive_science
TL;DR: To mitigate temporal redundancy for SNNs, we propose a module named Mutual Information-based Temporal Redundancy Quantification and Reduction (MI-TRQR), constructing energy-efficient SNNs.

Abstract: Brain-inspired spiking neural networks (SNNs) provide energy-efficient computation through event-driven processing. However, the shared weights across multiple timesteps lead to serious temporal feature redundancy, limiting both efficiency and performance. This issue is further aggravated when processing static images due to the duplicated input. To mitigate this problem, we propose a parameter-free and plug-and-play module named Mutual Information-based Temporal Redundancy Quantification and Reduction (MI-TRQR), constructing energy-efficient SNNs. Specifically, Mutual Information (MI) is properly introduced to quantify redundancy between discrete spike features at different timesteps on two spatial scales: pixel (local) and the entire spatial features (global). Based on the multi-scale redundancy quantification, we apply a probabilistic masking strategy to remove redundant spikes. The final representation is subsequently recalibrated to account for the spike removal. Extensive experimental results demonstrate that our MI-TRQR achieves sparser spiking firing, higher energy efficiency, and better performance concurrently with different SNN architectures in tasks of neuromorphic data classification, static data classification, and time-series forecasting. Notably, MI-TRQR increases accuracy by \textbf{1.7\%} on CIFAR10-DVS with 4 timesteps while reducing energy cost by \textbf{37.5\%}. Our codes are available at https://github.com/dfxue/MI-TRQR.

MI-TRQR: Mutual Information-Based Temporal Redundancy Quantification and Reduction for Energy-Efficient Spiking Neural Networks
Dengfeng Xue Wenjuan Li--/ Yifan Lu Chunfeng 234 Yufan Liu Wei Liu
Man Yao² Li Yang Guoqi Li Bing Li2.3.55 Stephen Maybank Weiming Hu2 Zhetao Li
School of Computer Science and Technology, Xidian University State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information
*School of Artificial Intelligence, University of Chinese Academy of Sciences PeooleAl Inc.
DDepartment of Computer Science and Information Systems, Birkbeck College, University of London "Scooo of Information Science and Technology, ShanghaiTech University s College of Information Science and Technology, Jinan University
Abstract
Brain-inspired spiking neural networks (SNNs) provide energy-efficient computa- tion through event-driven processing. However, the shared weights across multiple timesteps lead serious temporal feature redundancy, limiting both efficiency and performance. This issue is further aggravated when processing static images due to the duplicated imput. To mitigate this problem. we propose parameter-free and plug-and-play module named Mutual Information-base Temporal Redundancy Quantification and Reduction (MI-TRQR), constructing energy-efficient SNNs. Specifically, Mutual Information (MI) is properly introduced to quantify redun- dancy between discrete spike features at different timesteps on two spatial scales: pixel (local) and the entire spatial features (global). Based on the multi scale redun- dancy quantification, we apply probabilistic masking strategy to remove redundant spikes. The final representation is subsequently recalibrated to account for the spike removal. Extensive experimental results demonstrate that our MI-TRQR achieves sparser spiking firing, higher energy efficiency, and better performance concur- rently with different SNN architectures in tasks neuromorphic data classification, static data classification, and time-series forecasting, Notably, MI-TRQR increases accuracy by 1.7% on CIFARIO-DY with 4 timesteps while reducing energy cost by 37.5% %. Our codes available at htthtts: /tttps://githube com xwe//
1 Introduction
Spiking Neural Networks (SNNs), inspired by the processing mechanisms biological neural networks (18), offer an energy-efficient approach to neuromorphic hardware by performing spike- based accumulation and avoiding the computation of zero-value inputs (i.e., event-driven) [36] 48 [0]. With the release of neuromorphic chips [5] 37 and the proposal of algorithms for various tasks
56
11 87
79|
50 63 73
64 55 neuromorphic computing systems are progressing toward practical
deployment in world applications
33)
SNNs rely on sequential timesteps to transmit temporal information, and their outputs are typically averaged across time to improve accuracy 8 771. The temporally shared weights induce spatio-
intern CASIA.
CCorrespondigg author: wenjuan. li@ia.ac.cn
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

3.118
0.1247
Tinastps
0:0311
2055
0.1372
0.200
0.0
Firing 78.
Firing rale- 43%
Fining rate 05%
Firing rate-67.068 67.08%
Timesteps
(a) PSN VGG Average firing rate=67 .07%
(b) PSN MI matrix
0:115
3:309
Tinastips
0.1134
8,5000
33300
0.355
1.000
3.0000
03003
3
0.0
Firing 2100%
Firing rate-20.333%
Firing rate-20,269
Firing rate- 28%
Timesteps
(c) MI-TRQR VGG Average firing rate=20.47%
(d) MI-TRQR MI matrix
Figure 1: Visualization of spike features on CIFARIO-DVS: (a) Spike features in PSN 14 and (b) their MI matrix; (c) Spike features in MI-TRQR and (d) their MI matrix. MI-TRQR reduces the spiking firing rate by 69.48% (67.07% + 20.47%) and temporal redundancy by about 24% (computed from the redundancy reduction between the first timestep and other timesteps, e.g., 0. 0.1507 -0.1144,
introducing serious feature redundancy and resulting in a high spiking firing rate (68 25], as shown in Fig. [a). This phenomenon degrades energy efficiency and compromises the compactness of learned features. When processing static images, existing methods often duplicate the same input across all timesteps [0)] further exacerbating temporal redundancy in SNNs [68]
temporal invariance
22
This practice has been shown to impair both the energy efficiency and overall performance of SNNs
48 38 69].
Energy efficiency has been a longstanding concern in recent research [54 49 46
motivating
the research and development of diverse strategies aimed at reducing redundancy. Kim et al. [25] found that features at later timesteps had minimal impact on the final predictions, highlighting considerable temporal redundancy. Yao et al. (68) conducted systematic yet qualitative analysis on redundancy. Qin al. [44) provided a quantitative yet indirect strategy by defining similar spikes through the cosine similarity of their membrane potential. We argue that it is insufficient to quantify the redundancy between the spike features due to two intrinsic faults: (1) the inability of a linear tool to capture correlations between floating-poin membrane potentials, and (2) the quantization error caused by the discontinuous and nonlinear property of the spiking activation function. The absence of direct redundancy quantification significantly hinders progress toward highly energy-efficits SNNs.
To directly and accurately quantify redundancy between discrete spike features, we propose to use Mutual Information (MI), which is a principled and widely used metric [43 81 82 57). Existing metrics, such as Pearson correlation and Euclidean distance, measure basic similarity but fall short in capturing the complex and non- linear similarity between high nigh-dimensional spike features [40]. In contrast, Mutual Information (MI) captures complex statistical dependencies through probability distribution analysis, making it well-suited for quantifying redundancy between the high-dimensional discrete spike features. this work, we calculate MI between spike features at different timesteps, as shown in Fig. (b) and (d).
Previous studies analyzing neural recordings from various monkey brain regions have shown that spike features are significantly less redundant with activity-depender depression [15). Predictive coding has also demonstrated the ability to learn efficient visual representations by removing pixel-wise redundant spikes 22 39]. Inspired by these works, we propose parameter-free, ffweeeellggaanneeeeeee plug-and-pla MI- based Temporal Redundancy Quantification and Reduction (MI-TRQR) module, which is seamlessly integrated into the SNNs. Specifically, MI is used to quantify temporal redundancy between high- level spike features on pixel-level (local) and the entire spatial scale (global). These multi-scale redundancy quantifications are aggregated to compute pixel-wise probability for removing spikes, which achieved using a binary mask. Existing SNNs typically produce final representations
by averaging outputs across timesteps, whether for classification [9 [2 20 forecasting (35),

---

implicitly assuming a uniform temporal contribution or information distribution. However, due to the non-uniform information distribution after spike removal, we recalibrate the weights for the final representation. As illustrated in Fig. T MI-TRQR reduces spiking firing rate and temporal redundancy. The proposed MI-TRQR is validated on neuromorphic data classification, static image classification, and time-series forecasting tasks. Experimental results demonstrate that our MI-TRQR obtains more compact representations, improving both performance and energy efficiency. Our contributions are as follows:
We use MI to directly and accurately quantify temporal redundancy between high- dimensional discrete spike features at multiple spatial scales in SNNs.
We propose parameter-free, plug-and-play MI-TRQR module, which removes pixel-wise redundant spikes based the multi-scale redundancy quantification. The weight of the final representation is recalibrated to balance the information distribution.
We demonstrate the significant advantages of our approach by comparing the energy con- sumption of MI-TRQR with baseline methods, showing clear improvement. Extensive experiments across a range of tasks confirm that our method achieves higher accuracy and enhanced energy efficiency.
2 Related Work
Redundancy is a critical factor that must addressed when designing efficient SNNs. In the following, we briefly review some representative studies that focus on analyzing and/or reducing redundancy in SNNs.
2.1
Redundancy Amalysis
Kim al. [25] investigated the temporal information distribution and identified the Temporal
Information Concentration (TIC) phenomenon, wherein information is highly concentrated in the early timesteps after training. This observation highlights the presence of serious temporal redundancy. Yao et al. [68] conducted a systematic analysis, particularly for event- based vision tasks. They discussed three key questions ('which', 'why'. and 'how') and concluded that redundancy primarily stems from the spatio-temporal invariance caused by temporally shared weights in SNNs (22]. Qin et al. 44 introduced the concept of spike cluster, defined based on the cosine similarity of membrane potentials across timesteps. Despite substantial progress in redundancy years, direct quantification of redundancy between spike features across timesteps remains largely unexplored. In this paper, we address this gap by using MI to directly quantify the redundancy between spike features different timesteps.
research in recent
2.2 Redundancy Reduction
Many methods have been proposed to mitigate redundancy in different ways. Perez et al. 42 introduced an early sparse backpropagation algorithm tailored for SNNs. Subsequent works further enhance the energy efficiency of SNNs applied sparsity regularization during backpropagation presented a cohesive exploration of atten-
(66] 75 A recent series of studies [72 71
67
68
65
tion mechanisms in SNNs. These studies used diverse attention designs strategically modulate membrane potential distributions and spiking responses across various dimensions, such as spatial, temporal, channel-wise, and/or their combinations. They underscored the transformative impact of attention mechanisms in SNNs, leading to more efficient and accurate models. However, such attention modules inevitably increase network complexity and introduce additional multiplications, which may raise hardware costs and hinder deployment on mnouurce-limited edge devices. The temporal self-erasing method dynamically adjusted the regions of interest for different timesteps 34. Another widely adopted approach is to apply penalty functions to encourage the deep net- work to learn sparser spike representations [41]
78 These methods often require careful tuning of many hyperparameters, which may increase the training complexity. In this paper, we develop a parameter-free free and plug-and-play module to enable SNNs to learn compact and powerful representations.
3

---

Temporal Redundancy Reduction
Weight Recalibration
8 ( 1)
S1
FC
1=1 Input
Mask Generation
2000
M3
p
FC
class
S
82
t-2 Input
Mask
23
:
Generation
My
000
OO
FC
Sr
ET Impul
Feature $1
p.
I(;;; (;;
M.sw B(C, PrAm))
$1..1
Global reoundancy
xt (a ecaler)
PLia Pa1 Paip
MiNs
0
PuLi Plaa Plaa
Pla
0
SA1.1
Local redundancy
Pla. Mlaa plaa
PLal P1aa PMaa
000
Hf
1
Feature S,
PlA1 viaa Plaa
pd e/CE
Piha Plhoo
M, (
P
(Sr ( SLAN)
Mask Generation via MI Quantification
Figure Our MI-TRQR module preserves the original spike features the first timestep. At subsequent timesteps, local and global redundancy quantification are first aggregated with a training factor. This aggregation is then used compute a probability that guides the mask generation. The generated mask is responsible for removing pixel-wise redundant spikes, aiming to achieve temporal redundancy reduction. Weight recalibration balances the con-uniform information distribution.
3 Methodology
In this section, we introduce the proposed MI-TRQR module, which is integrated after the final stage to process the high-level features [83]. as shown Fig. 2 Section 3.1 details how MI is used to quantify temporal redundancy between spike features. Section introduus the temporal redundancy reduction strategy, removing redundant spikes using the multi-scale redundancy quantification.
Section 3.3 recalibrates the weight of the final representation based the information density.
3.1 Temporal Redundancy Quantification
MI is particularly effective for evaluating similarity between discrete variables, as elaborated in Appendix[A] Therefore, MI is suitable metric to quantify temporal redundancy between binary spike features. Given the dour-dimeniional binary spike feature S E {0, 1) Tx where T, C, H. W indicate the timestep, channel, height, and width respectively, we define two terms to quantify temporal redundancy.
Definition 1. Global redundancy quantifies the overall temporal redundancy between spike features at two different timesteps and is denoted as R?, It is computed as follows:
p(())8j)
Rd
Σ p(ss, 85),,8))og log ,,E,,
Σ
(1)
(p(s))((s)))
where I(S,; S,) denote MI between spiking features S, E [0,1)cxHxW at timestep i and and sy represent specific values of S; and Sj. p(s;) and p(s,) the marginal probability distributions of s; and 8j. respectively. p(s,, 8;) is their joint probability distribution. R (i.j) 2 0 is float scalar that quantifies the degree of global redundancy between spike features S, and Sj-
Definition 2. Local redundancy quantifies temporal redundancy the pixel scale. For each spatial location, compute the MI between the local spike vectors two timesteps:

---

p(st, 85) ((s))p(s))
(2.3)) = I(S (phww)= =
Σ
Σ
p(s,, log log
(2)
where vh E H, Vw [1,W], Sih,,w, S.h E 10 are the raw pixel (spatial location (h, w)) timestep and j. respectively. R (i, 2 0 also float scalar, indicating the pixel- wise redundancy between St,.w and S.A,... It corresponds the (h, wi-th element of the local redundancy matrix R (i,j) B2W
3.2 Temporal Redundancy Reduction
Based on the redundancy quantification between spike features at multiple spatial scales, we derive a probability that estimates the likelihood of each pixel-wise spike being redundant. This probability is then used to generate a pixel-wise temporal redundancy mask, which selectively removes redundant spikes.
Temporal redundancy-guided probability derivation. The global redundancy between spike features S and S; (t E [2) is served the global probability factor for spikes removal, denoted by pf t). We observe that the MI of a feature with itself I(S1, S1) is not equal to one. This is evident from the varying diagonal elements in the MI matrix shown in Fig. Therefore, normalize the global probability factor pi as follows:
ri (1, I1)
(1,
T(S1:S1)
where p e [0, 1), since S,) ((S;:S) S1) when + 1, as shown the Fig
Similarly to the computation of pł in Eq. the local probability factor pl is defined follows:
Ry, (1, 1(S1, St,,,w) Peh, R (1,1) 1(S1, S1,h,w)
(4)
where p E [0, 1), similar to the p!.
In addition, the TIC phenomenon demonstrates that information gradually concentrates on the first timestep as training goes on [25]. Accordingly, we retain the spikes at the first timestep, as illustrated in Fig. 2 However, when removing spikes, it is crucial consider the training epoch, the informative spikes are still distributed across the later timesteps during the early training stages. Thus, introduce waaiing-deppndder factor pd into the pixel-wise probability Pt.h..
Pt,b, pe x Pi.h.u D., D² ccC
(5)
where and denote the current training epoch and the total number of epochs, respectively. Temporal redundancy-basee mask. We use the pixel-wise probability Pt,h,w to generate a binary mask at timestep t:
M. = [1]G if
Me ~ B(C, - Pt.h,,):
(6)
otherwise
where M, E {0, 1xx denotes the mask at timestep t. M.A. E (0. 1) represents the pixel
at spatial location (h, w). B(:,)) denotes Binomial distribution. The spike removal operation using the mask M, is expressed as:
5, S 3 M.
where S, denotes the output spike features at timestep I, and @ the element-wis wise multiplication. The mask at the first timestep. M,, consists entirely of one, thereby preserving the original spikes.
5

---

Algorithm The implementation of removing pixel-wise redundant spikes
1: Input: Spike feature S E [0, (0,1)TxCxxxX the training factor pd
2: Output: Spike feature 5 E ( X
3: Obtain the number of spatial pixels: N = x W
4: Flatten spatial dimension: S' reshape (T, CJ)
5: Define a matrix filled with for redundancy quantification: R' <- ones (T,
6: for = 1 + T
7: Compute globul redundoncy: Rº [(
8: for - N do
9: Compute local redundancy: R (1, 1 S S,,))
10: Get the pixel-wise combined redundancy metric: R2 R2 n] (1, x (1,0)
11: end for
12: end for
13: Unfold spatial dimension: R reshape (R', (T, H, W))
14: Obtain T x N probability matrix: P R J. : 1//R x p
15: Set the probability at the first timestep into zeros: P(
16: Generate mask: M = [ M M. ~ B Ptt, w)).
17: Get the output spike feature with clement-wise multiplication: 5 = S
For E [2, T), zero values in M, convert active spikes (1) into inactive states (0). Consequently, proportion (pt.h.w)) of spikes at the pixel St.h. is removed. The pseudo-code for spike removal is given in Alg. Spike removal also reduces the spiking firing rate earlier layers, which is analyzed in Appendix B
3.3 Weight Recalibration
The spike removal later timesteps introduces a significant divergence in information distribution across timesteps. Consequently, it is unsuitable to simply average temporal outputs for the final
representation, as done in existing works [12 20 35). To address this issue, we adaptively recalibrate the weight of the final representation using the normalized network spiking firing rate:
freet
E N E NL.
0g
ΣT fremt
Tract
(8)
where a, and Srnet denote the weight and network spiking firing rate timestep respectively. Nř, indicates the number of spikes, and Ni, indicates the total number of elements, both timestep and layer I. L is the total number of layers. In this way, the final representation Q is obtained as follows:
ce pp)
where Q denotes the representation at timestep t.
4
Experiments
In this section, we report the experimental results for MI-TRQR on both classification and time-series forecasting tasks. The operation of removing spikes is only used during training. The experimental setup is described Appendix C.I
4.1 Comparison with Other Methods
Neuromorphic data classification. On CIFAR1O-DV? MI-TRQR consistently surpasses PSN [4] in both accuracy and energy efficiency across different timesteps, as shown in Tab. [ When T=4, PSN vs. MI-TRQR: Accuracy, 82.3% vs. 83.9% (+1.6%); Power, 0.72mJ vs. 0.45mJ (-0.27mJ,

---

Table 1: Comparison with other methods neuromorphic datasets. Power denotes the energy cost of inference one test sample. 1 and + indicate desired metric direction. Results denoted by were obtained through our reimplementation.
Dataset
Model
Network
T
Accuracy (1%)
Power (fmJ)
Deep 33
VGG-11
10
81.2
TIM 53
Spikformer
10
81.6
SEMM 85
Spikformen-2-256
82.9
RM-SNN 71
VGG
10
82.9
CIFAR 1O-DVS
STAtten
SpikingReformen-4-3884
83.9
QKFormer
84
HST-2-256
16
84.0
CLIF 21
VGG
86.1
4
82.3
0.72
PSN
VGG
85.3
1.28
10
85.9
1.20
4
83.9
0.45--0.06
MI-TRQR (ours)
VGG
8
86.2-0.12
0.74++0.07
86.5=0.13 10
0.84
ASA
68
3- SNN
83.2
3D GCN 660
86.0
Gait
DSNN 70
Layer SNN
90.2
PSN 14
3B-Net
88.8*
0.19
MI-TRQR (ours)
3B-Net
9.16=0.13
0.13:10.08
Table 2: Comparison with other methods on ImageNet.
Method
Model
Network
Accuracy (1%)
Power ((mJ)
Hybird 47
ResNet34
250
61.48
Tandem
VGG-16
16
65.08
ANN2SNN
Two-stage 6
ResNet34
16
67.77
Optimal
ResNet34
32
69.37
OSR+OTS
ResNet34
67.54
EnOF-SNN
17
ResNet34
4
67.40
rate
76
MS-ResNet33
70.01
Direct
GAC-SNN 45
MS- Ne-Ree
6
70.42
3.38
Training
IMP+LTS 52
SEW-ResNet50
71.83
3.11
SEW-ResNer18
67.63
2.42
PSN 14
SEW-ResNer34
70.54
3.70
SEW-ResNet50
72.01
4.11
SEW-ResNet18
68.28+00.12
1.92-+0.07
Direct
MI-TRQR (ours)
NEW-ResNer34
71.06
3.06:10.09
Training
SEW-ResNet50
4
73.23+013
3.09-10.08
(37.50%). Similar advantages of our MI-TRQR are observed when 10= MI-TRQR shows 1.8%
accuracy improvement and 31.6% energy reduction over PSN on Gait (T=10)..
Static data classification. On ImageNet, MI-TRQR consistently outperforms PSN across different backbones, as shown in Tab, 2 For example, with ResNet50, PSN vs. MI-TRQR: Accuracy, 72.01 %
73.23% (+1.22% %% Power, 1.1
3.09mJJ (-1.02mJ, J. 26). More experiments
CIFAR 10/100 are provided in Appendix C.2. On CIFARIO, PSN vs. MI-TRQR: Accuracy 95.32%
vs. 95.83% (+0.51 %)); Power, 1.32mJ 0.35mJ (-0.97mJ, 73.48%)
Time-series forecasting, In Tab. 3 we show the superiority of our MI-TRQR over CPG-PE 355 in two metries (R2 and RSE) the Electricity dataset with various prediction lengths 6.24,48.96.

---

Table 3: Results Electricity. Our results averaged across random seeds.
Eletricity
Method
Metric
Avg.
6
24
48
96
R²1
.956
.955
.953
.943
.952
Spikformer 86
RSE
.371
.375
.386
.450
.396
2²1
.971
.971
.968
.962
.968
CPG-PE
RSE
.304
.308
.311
.439
.341
R R²T
.973
.972
.972
.967
.971
MI-TRQR (ours)
RSE
.292
.307
.297
.368
.316
Table 4: Impact of local redundancy (LR) and global redundancy (GR) on CIFARIO-DVS.
Method
LR
GR
T
Accuracy (1%)
Firing rate (4%
Training time
PSN
14
4
82.3
14.47
24.1s
83.2 (10.9)
9.73 (L332.6)
51.1s
MI-TRQR (ours)
/
4
83.4 (+1.1)
10.91 (124.60)
24.8s
84.0 (11.7)
8.35 (144.229)
51.9s
Table 5: Impact of MI-TRQR placement on CIFARIO-DVS.
Method
Placement
T Accuracy(1%)
Firing rate(4%)
Training time
PSN 14
82.3
14.47
24.1s
After layer
82.8(10.5)
11.52((20.399)
597.3s
After layer 6
4
83.4(11.1)
10.18([29.65%%)
171.6s
MI-TRQR(ours)
After layer 6+8
83.5($1.2)
9.74(132.69%
222.6s
After layer 8(last)
4
84.0(11.7)
8.35(142.29%)
51.9s
4.2 Ablation Study
conducted ablation studies on CIFAR IO-DVS to verify the impact of weight recalibration and the effectiveness, efficiency, and optimal placement of the Mi-TRQR module. Additional ablation results on CIFARIO 10 are provided in Appendix C.2.2 More ablation studies are provided in Appendix C.3
Effectiveness. In Tab. we quantitatively show the gains of incorporating local and/or global redundancy for learning more compact representations. By leveraging multi-scale redundancy, MI- TRQR obtains the best accuracy of 84.0%(+1.7%%) We also report the training time per epoch to provide a comprehensive comparison.
Efficiency. In Tab, 4 we can see that using multi-scale redundancy reduces the spiking firing rate by 42.29% (from 14.47% 8.35%). The detailed spike counts and firing rates are presented in Fig. Key observations are: (1) MI-TRQR consistently fires fewer spikes than PSN at each timestep; (2) its spike removal propagates from deeper layers to shallower layers; (3) consistently reduces the spiking firing rate across various datasets (CIFARIO in Appendix C.2.2p.
MI-TRQR placement. In Tab. 5 we report the results when MI-TRQR is integrated after different convolutional layers. Results show that positioning MI-TRQR after the last stage (layer 8) yields the biggest advantages in both accuracy and firing rate compared to PSN, confirming the benefit of targeting MI-TRQR to high-level features. In addition, the computation probability Pt, in Eq. 5 is spatiotemporally dependent, with computational complexity of (T)) for global redundancy and (T x x W) for local redundancy. Inserting the module into earlier layers leads to a higher computational cost. We provide the corresponding time of training epoch.
Weight recalibration. Tab. 6 presents the impact of weight recalibration on CIFAR10-DVS MI- TRQR with reweighting yields the highest accuracy while consuming the least energy.

---

Number of spikes
200000
MI-TRQR (ours)
Number spikes
MI-TRQR (ours)
180000-
PSN
200000
PSN
160000
L of
180000
140000
1
140000
120000
120000
100000-
100000
80000
80000
25
50
75
100 Epochs
125
150
175
200
25
50
75
100
125
150
175
200
Epochs
(a) Number of spikes during training at t=1.
(b) Number of spikes during training at t=2.
250000
MI-TRQR (ours)
250000
MI-TRQR (ours)
Number of spikes
225000
PSN
spikes
225000
PSN
200000
200000
175000
175000
150000
150000
Mo
125000
125000
100000
100000
75000
50
100
125
175
200
75000
25
100
125
150
175
200
Epochs
Epochs
(c) Number of spikes during training at t=3.
(d) Number of spikes during training t=4.
0.67
0.22
MI-TRQR (ours)
0.66
MI-TRQR (ours)
PSN
0.65
0.20
PSN
0.20
20.16
0.15
0.14
0.12
0.10
0.05
50
100
125
175
200
0.00
Layer Layerz Layers Layord Layers Layers Layer7 Layers
Epochs
VGG
(e) Network spiking firing rate during training (T==)).
(f) Layer-wise wise spiking firing rate of VGG.
Figure 3: Comparison of the number of spikes and firing rate in different methods CIFARIO-DVS.
Table Impact of weight recalibration on CIFARIO-DVS.
Method
Reweighting
T
Accuracy (1%)
Firing rate (1%)
PSN 14
4
82.3
14.47
4
1314(+1.1)
9.22( 288)
MI-TRQR (ours)
V
4
84.0(11.7)
8.35([42.29))
4.3 Visualization
CIFARIO-DVS. In Fig. I we can observe that MI-TRQR reduces the spiking firing rate by 69.48% (67.07% - 20.47%) and temporal redundancy by approximately 24% (e.g., 0.1507 -0.1134) even shallow layer.
ImageNet. In Fig. we visualize spike features and their MI matrices. We can see that MI-TRQR significantly decreases the temporal redundancy between spike features by 79% (e.g..) and reduces the average spiking firing rate by 18.39% (23.34% vs. 28.60%). These results demonstrate that MI-TRQR can effectively remove redundant spikes and enhance representation compactness.
Conclusion and Limitations
In this paper, we propose effective and efficient module, named MI-TRQR. Based on the temporal redundancy quantification using MI, MI-TRQR identifies and removes pixel-wise redundant spikes, enabling SNNs to learn more compact and powerful representations. Experimental results various

---

0.700
0.887
0.1991
C/000
Timestops
02
0:8233
6.7264
Firing rate=30.188
Fining ralo-27 96%
Firing rate=27. 75%
Firing rate-28.51%
(a) PSN ResNet18 Average firing rate=28.60%
(b) PSN MI matrix
3:1063
0,249S
C.0006
Timestepi
1/90
01:941
9.3543
0.3739
6.1071
0.0
Firing rate=28.13%
Finin ralo=27-19%
Firing rate=21.39%
Firing rale=16.67%
Timesteps
(c) MI-TRQR ResNet18 Average firing rate=23.34%
(d) MI-TRQR matrix
Figure 4: Visualization of spike features after stage ResNet18 on ImageNet: (a) Spike features in PSN and (b) their MI matrix; (c) Spike features in MI-TRQR and (d) their MI matrix. For a single sample, the spike features has shape [T, C, H, W] (timestep, channel, height, width). Since T 4, the left). We plot each channel in small grid. We report the average spiking firing rates under the spike features. MI-TRQR reduces the spiking firing rate by 18.39% (28.60% - 23.34%) and temporal redundancy by about 79% (computed from the redundancy reduction between the first timestep and other timesteps, c.g., 15837-001108
we first split into four [C, H, W] features (shown
tasks demonstrate that MI-TRQR consistently outperforms baseline methods in both accuracy and energy efficiency. To best of our knowledge, this the first study to use MI to quantify redundancy between spike features in SNNs, providing foundation for subsequent related research.
Limitations Although our method does not introduce extra cost during inference, it requires comput- ing mutual information between spike features during training, resulting in increased computational cost and longer training time.
Acknowledgements
This work is supported by the Natural Science Foundation of China (62036011, 62192782. 1051, 62032020, U2441241, U24A20331, 62372451, 62192785, 62372082, 62202469, 62403462, 62202470, 62473363), Beijing Natural Science Foundation (L223003, L243015, L251005, JQ24022), the Key R&D Program of Xinjiang Uyghur Autonomous Region (2023801005), CAAI- Ant Group Research Fund (CAAI-MYJJ 2024-02), and Young Elite Scientists Sponsorship Program by CAST (2024QNRC00)).
References
[1] Srinivas Anumasa, Bhaskar Mukhoty, Velibor Bojkovic, Giulia De Masi, Huan Xiong, and Bin Gu. Enhancing training of spiking neural network with stochastic latency. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 10900-10908. 2024.
[2] Rafal Bogacz. A tutorial on the free-energy framework for modelling perception and learning, Journal of Mathematical Psychology, 76:198-211, 2017
[3] Tong Bu, Wei Fang, Jianhao Ding, PengLin Dai, Zhaofei Yu, and Tiejun Huang. Optimal ann-snn conversion for high-accuracy and ultra-low-latency spiking neural networks. In The Tenth International Conference Learning Representations, 2022.
[4] Simon Davidson and Steve B Furber. Comparison of artificial and spiking neural networks digital hardware. Frontiers in Neuroscience, 1411 2021.
10

---

[5] Mike Davies, Narayan Srinivasa, Tsung- Han Lin, Gautham Chinya. Yongqiang Cao, Sri Harsha Choday. Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, al. Loihi: neuromorphic manycore processor with on-chip learning, IEEE Micro, 38(1):82-99, 2018.
[6] Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. Imagenet: large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Puttern Recognition, pages 248-255. IEEE, 2009.
[7] Shikuang Deng, Yuhang Wu, Kangrui Du, and Shi Gu. Spiking token mixer: An event-driven friendly former structure for spiking neural networks. In Advances in Neural Information Processing Systems, volume 37, pages 128825-128846, 2024.
[8] Yongqi Ding, Lin Zuo, Mengmeng Jing. Pei He, and Hanpu Deng. Rethinking spiking neural networks from an ensemble learning perspective. In The Thirteenth International Conference on Learning Representations. 2025.
[9] Chaoteng Duan, Jianhao Ding, Shiyan Chen, Zhaofei Yu, and Tiejun Huang. Temporal effective batch normalization spiking neural networks. Advances in Neural Information Processing Systems, 35:34377- 34390, 2022.
[10] Jason K Eshraghian, Max Ward, Emre Neftci, Xinxin Wang, Gregor Lenz, Girish Dwivedi, Mohammed Bennamoun, Doo Seok Jeong, and Wei D Lu. Training spiking neural networks using lessons from deep learning. Proceedings of the IEEE, 111(9):001--100) 1054, 2023.
[11] Bin Fan, Jiaoyang Yin, Yuchao Dai, Chao Xu, Tiejun Huang, and Boxin Shi. Spatio-temporal interactive learning for efficient image reconstruction of spiking cameras. Advances in Neural Information Processing Systems, 37:21401-21427, 2024.
[12] Wei Fang. Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems, 34:21056-21065 2021.
[13] Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian. Incorporating learnable membrane time constant enhance learning of spiking neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2661-2671, 2021.
[14] Wei Fang, Zhaofei Yu, Zhaokun Zhou, Ding Chen, Yanqi Chen, Zhengyu Ma, Timothée Masquelier, and Yonghong Tian. Parallel spiking neurons with high efficiency and ability to learn long-term dependencies. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[15] Mark S. Goldman, Pedro Maldonado, and L. F. Abbott. Redundancy reduction and sustained firing with stochastic depressing synapses. Journal of Neuroscience, 22(2):584-591, 2002.
[16] Yufei Guo, Xiaode Liu, Yuanpei Chen, Weihang Peng, Yuhan Zhang, and Zhe Ma. Spiking transformer: Introducing accurate addition-only spiking self-attention for transformer. In Proceddings of the Computer Vision and Pattern Recognition Conference, pages 11398-24408, 2025.
[17] Yufci Guo, Weihang Peng, Xiaode Liu, Yuanpei Chen, Yuhan Zhang, Xin Tong, Zhou Jic, and Zhe Ma. Enof-snn: Training accurate spiking neural networks via enhancing the oulput feature. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[18] Andreas VM Herz, Tim Gollisch, Christian K Machens, and Dieter Jaeger. Modeling single-neuron dynamics and computations: balance of detail and abstraction. Science, 314(5796):80-855 2006.
[19] Mark Horowitz. 1.1 computing's 's energy problem (and what we can do about it). In 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), pages 10-14. IEEE, 2014.
[20] Yifan Hu, Lei Deng, Yujie Wu, Man Yao, and Guoqi Li. Advancing spiking neural networks toward deep residual learning. IEFF Transactions on Neural Networks and Learning Systems, 2024.
[21] Yulong Huang, Xiaopeng Lin, Hongwei Ren, Haotian Fu, Yue Zhou, Zunchang Liu, Biao Pan, and Bojun Cheng. Clif: Complementary Icaky integrate-and- fire neuron for spiking neural networks. In Forty-first International Conference on Machine Learning, 2024.
[22] Ziyuan Huang. Shiwei Zhang, Liang Pan, Zhiwu Qing. Mingqian Tang. Ziwei Liu, and Marcelo H Ang Jr. Tada! temporally anmporall-adappivee convolutions for video understanding, In The Tenth International Conference Learning Representations, ICLR, 2022.
11

---

[23] Haiyan Jiang, Giulia De Masi, Huan Xiong, and Bin Gu. Ndot: neuronal dynamics-based online training for spiking neural networks. In Forty-first International Conference Machine Learning, 2024.
[24] Haiyan Jiang. Vincent Zoonekynd, Giulia De Masi, Bin Gu, and Huan Xiong. TAB: Temporal accumulated batch normalization spiking neural networks. In The Twelfth International Conference on Learning Representations, 2024.
[25] Youngeun Kim, Yuhang Li, Hyoungscob Park, Yeshwanth Venkatesha, Anna Hambitzer, and Priyadarshini Panda. Exploring temporal information dynamies in spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8308-8316, 2023.
[26] Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
[27] Souvik Kundu, Gourav Datta, Massoud Pedram, and Peter A Beerel. Spike-thrift: Towards energy-efficient deep spiking neural networks by limiting spiking activity via attention-guides compression. Proceedings of the IEEE/CVF Winter Conference on Aplications of Computer Vision, pages 3953-3962, 2021.
[28] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long- and short-term temporal patterns with deep neural networks. In The s1 International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 95-104, 2018
[29] Donghyun Lee, Yuhang Li, Youngeun Kim, Shiting Xiao, and Priyadarshini Panda. Spiking transformer with spatial-temporal attention. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 13948-13958, 2025.
[30] Guoqi Li, Lei Deng, Huajin Tang, Gang Pan, Yonghong Tian, Kaushik Roy, and Wolfgang Maass, Brain- inspired computing: A systematic survey and future trends. Proceedings of the IEEE, 2024.
[31] Hongmin Li, Hanchao Liu, Xiangyang Ji. Guoqi Li, and Luping Shi. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in Neuroscience, 11:244131, 2017.
[32] Qianhui Liu, Jiaqi Yan, Malu Zhang, Gang Pan, and Haizhou Li. Lite-snn: designing lightweight and efficient spiking neural network through optimization. In Proceedings of the Thirty-Thine Third International Joint Conference on Artificial Intelligence, pages 3097-3105, 2024.
compressive network search and joint
[33] Wei Liu, Li Yang. Mingxuan Zhao, Shuxun Wang, Jin Gao, Wenjuan Li, Bing Li, and Weiming Hu. Deeptage: Deep cemporal-aligma gradient enhancement for optimizing spiking neural networks. In The Thirteenth International Conference on Learning Representations, 2025.
[34] Wei Liu, Li Yang. Mingxuan Zhao, Dengfeng Xue, Shuxun Wang. Boyu Cai, Jin Gao, Wenjuan Li, Bing Li, and Weiming Hu. Towards more discriminative feature learning in snns with temporal-sele self--rasing supervision. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 1420-1428, 2025.
[35] Changze Lv, Dongqi Han, Yansen Wang, Xiaoqing Zheng, Xuanjing Huang, and Dongsheng Li. Advancing spiking neural networks for sequential modeling with central pattern generators. Advances in Neural Information Processing Systems, 37:26915 269400, 2024.
[36] Wolfgang Maass, Networks of spiking neurons: the third generation of neural network models. Neural networks, 1659-166,,, 1997.
[37] Paul A Merolla, John V Arthur, Rodrigo Alvarez-Icaza, Andrew S Cassidy, Jun Sawada, Filipp Akopyan. Bryan L Jackson, Nabil Imam, Chen Guo, Yutaka Nakamura, al. A million spiking-neurge integrated circuit with scalable communication network and interface. Science, 345(6197):668 673, 2014.
[38] Surya Narayanan, Karl Taht, Rajeev Balasubramonian, Edouard Giacomin, and Pierre-Emmann Emmanuel Gaillar- don. Spinalflow: An architecture and dataflow tailored for spiking neural networks. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA), pages 349-362. IEAF, 2020.
[39] Antony W d'dri, William Gebhardt, Céline Teulière, Fleur Zeldenrust, Rajesh PN Rao, Jochen Triesch, and Alexander Ororbia. Predictive coding with spiking neural networks: survey. arXiv preprint arXiv:2409.00 05666 2024.
[40] Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-basse based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6):51 63, 2019.
12

---

[41] Daniel Neil, Michael Pfeiffer. and Shih-Chii Liu. Learning to be efficient: Algorithms for training low- latency, low-compute deep spiking neural networks. In Proceedings of the 31st Annual ACM Symposium on Applied Computing, pages 293-298, 2016.
[42] N. Perez-Nieves and D. Goodman. Sparse spiking gradient descent. Advances in Neural Information Processing Systems, 34:1 1975-1 1808, 2021.
[43] Haina Qin, Wenyang Luo, Libia Wang, Dandan Zheng, Jingdong Chen, Ming Yang, Bing Li, and Weiming Hu. Reversing flow for image restoration. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages
2025.
[44] Lang Qin, Ziming Wang, Rui Yan, and Huajin Tang. Attention-based deep spiking neural networks for temporal credit assignment problems. IEFE Transactions on Neural Networks and Learning Systems, 1538):10301-1011, 2023.
[45] Xuerui Qiu, Rui-Jic Zhu, Yuhong Chou, Zhaorui Wang, Liang Deng, and Guoqi Li. Gated attention coding for training high performane and efficient spiking neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 601-610, 2024.
[46] Arjun Rao, Philipp Plank. Andreas Wild, and Wolfgang Maass. A long short-term memory for ai applications in spike-based neuromorphic hardware. Nature Machine Intelligence, 4(5):467-479, 2022.
[47] Nitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, and Kaushik Roy. Enabling deep spiking neural networks with hyhrid conversion and spike timing dependent backpropagation. In 8th International Conference on Learning Representations, 2020.
[48] Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575(7784))607-617, 2019.
[49] Catherine D Schuman, Shruti R Kulkarni, Maryam Parsa, J Parker Mitchell, Prasanna Date, and Bill Kay. Opportunities for neuromorphic computing algorithms and applications. Nature Computational Science, 2(1):10-19, 2022.
[50] Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, and Kaushik Roy. Going deeper in spiking neural networks: Vgg and residual architectures. Frontiers in Neuroscience, 13:95, 2019.
[51] Claude Elwood Shannon. mathematical theory of communication. The Bell System Technical Journal, 27(3):379-423, 1948.
[52] Hangchi Shen, Qian Zheng, Huamin Wang, and Gang Pan. Rethinking the membrane dynamics and optimization objectives of spiking neural networks. Advances in Neural Information Processing Systems, 37:92697- 92720, 2024.
[53] Sicheng Shen, Dongeheng Zhao, Guobin Shen, and Yi Zeng. Tim: efficient temporal interaction module for spiking transformer. In Proceedings of the Thirty-Thire International Joint Conference on Artificial Intelligence, pages 3133-3141, 2024.
[54] Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for modern deep learning research. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 13693-13696, 2020.
[55] Qiaoyi Su, Yuhong Chou, Yifan Hu, Jianing Li, Shijie Mei, Ziyang Zhang, and Guoqi Li. Deep directly- trained spiking neural networks for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6555-6565, 2023.
[56] Qiaoyi Su, Weihua He, Xiaobao Wei, Bo Xu, and Guoqi Li. Multi-scale full spike pattern for semantic segmentation. Neural Networks, 176: 106330 2024.
[57] Hanshi Wang, Jin Gao, Weiming Hu, and Zhipeng Zhang. Mambafusion: Height-fidelity dense global fusion for multi-modal object detection. arXiv preprint arXiv:2507.04369, 2025.
[58] Shuai Wang, Malu Zhang, Dehao Zhang, Ammar Belatreche, Yichen Xiao, Yu Liang, Yimeng Shan, Qian Sun, Enqi Zhang, and Yang Yang. Spiking vision transformer with saccadic attention. In The Thirteenth International Conference on Learning Representations, 2025.
[59] Yanxiang Wang. Bowen Du, Yiran Shen, Kai Wu, Guangrong Zhao, Jianguo Sun, and Hongkai Wen. Ev- gait: Event- based robust gait recognition using dynamic vision sensors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6358-6367, 2019.
13

---

[60] Yanxiang Wang. Xian Zhang, Yiran Shen. Bowen Du, Guangrong Zhao, Lizhen Cui, and Hongkai Wen. Event-stream representation for human gails identification using deep neural networks. IEEE Transactions on Puttern Analysis and Machine Intelligence, 44(7):3436-344) 3449, 2021.
[61] Ziming Wang. Yuhao Zhang, Shuang Lian, Xiaoxin Cui, Rui Yan, and Huajin Tang. Toward high-accuracy and low-latency spiking neural networks with two-stage optimization. JEEE Transactions on Neural Networks and Learning Systems, 2023.
[62] Jibin Wu, Chenglin Xu, Xiao Han, Daquan Zhou, Malu Zhang, Haizhou Li, and Kay Chen Tan. Progressive tandem learning for pattern recognition with deep spiking neural networks. IEEE Trunsactions on Pattern Analysis and Machine Intelligence, (4(11):884--8440, 2021.
[63] Keming Wu, Man Yao, Yuhong Chou, Xuerui Qiu, Rui Yang, Bo XU, and Guoqi Li. RSC-SNN: Exploring the trade-off between adversarial robustness and accuracy in spiking neural networks via randomized smoothing coding. In ACM Multimedia 2024, 2024.
[64] Xiaoshan Wu, Weihua He. Man Yao, Ziyang Zhang. Yaoyuan Wang, Bo Xu, and Guoqi Li. Event-based depth prediction with deep spiking neural network. IEEE Transactions Cognitive and Developmental Systems, (666)2200-2016 2024.
[65] Dengfeng Xue, Wenjuan Li, Chunfeng Yuan, Haowei Liu, Man Yao, Wei Liu, Li Yang, Bing Li, Weiming Hu, Haoliang Sun, et al. Msfi: Multi-timescale spatio-temporal features integration in spiking neural networks. Neural Networks, page 108024, 2025.
[66] Yulong Yan, Haoming Chu. Yi Jin, Yuxiang Huan, Zhuo Zou, and Lirong Zheng, Backpropagation with sparsity regularization for spiking neural network learning. Frontiers in Neuroscience, 16:760298, 2022.
[67] Man Yao, Huanhuan Gao, Guangshe Zhao, Dingheng Wang, Yihan Lin, Zhaoxu Yang, and Guoqi Li. Temporal- attention spiking neural networks for event streams classification. Proceedings of the IEEÉ/CVF International Conference on Computer Vision, pages 10221-102341 2021.
[68] Man Yao, Jiakui Hu, Guangshe Zhao, Yaoyuan Wang, Ziyang Zhang, Bo Xu, and Guoqi Li. Inherent redundancy in spiking neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16924-16934, 2023.
[69] Man Yao, Xuerui Qiu, Tianxiang Hu, Jiakui Hu, Yuhong Chou, Keyu Tian, Jianxing Liao, Luziwei Leng, Bo Xu, and Guoqi Li. Scaling spike-driven transformer with efficient spike firing approximation training. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4):2973-29) 2025.
[70] Man Yao, Ole Richter, Guangshe Zhao, Ning Qiao, Yannan Xing, Dingheng Wang, Tianxiang Hu, Wei Fang, Tugba Demirci, Michele De Marchi, et al. Spike-based dynamic computing with asynchronous sensing-computing neuromorphic chip. Nature Communications, 15(1):4464, 2024.
[71] Man Yao, Hengyu Zhang, Guangshe Zhao, Xiyu Zhang, Dingheng Wang, Gang Cao, and Guoqi Li. Sparser spiking activity can he better: Feature refine-and-man spiking neural network for event-based visual recognition. Neural Networks, 166:410-443 423,00--2222222 2023.
[72] Man Yao, Guangshe Zhao, Hengyu Zhang, Yifan Hu, Lei Deng, Yonghong Tian, Bo Xu, and Guogi Li. Attention spiking neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[73] Zhiwei Yao, Shaobing Gao, and Wenjuan Li. Snn using color-opponent and attention mechanisms for object recognition. Pattern Recognition, 158: 1070, 2025.
[74] Bojian Yin, Federico Corradi, and Sander M Bohté. Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks. Nature Machine Intelligence, 3(10):905 913, 2021.
[75] Hang Yin, John Boaz Lee, Xiangnan Kong. Thomas Hartvigsen, and Sihong Xie. Energy- efficient models for dinee-iiinsional spike train classification using sparse spiking neural networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 2017-2025, 2021.
[76] Chengting Yu, Lei Liu, Gaoang Wang, Erping Li. and Aili Wang. Advancing training efficiency of deep spiking neural networks through rate-based backpropagation. Advances in Neural Information Processing Systems, 2024.
[77] Chengting Yu, Xiaochen Zhao, Lei Liu, Shu Yang. Gaoang Wang. Erping Li, and Aili Wang. Efficient lagit-based knowledge distillation of deep spiking neural networks for full-range timestep deployment. In second International Conference on Machine Learning, 2025.
14

---

[78] Friedemann Zenke and Tim P Vogels. The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks. Neural Computation, 33(())899-92) 925599999922999 2021.
[79] Dehao Zhang, Shuai Wang. Ammar Belatreche, Wenjie Wei, Yichen Xiao, Haorui Zheng. Zijian Zhou, Malu Zhang, and Yang Yang. Spike- based neuromorphic model for sound source localization. In The eigtt Annual Conference on Neural Information Processing Systems, 2024.
[80] Hong Zhang and Yu Zhang. Memorry-fficient reversible spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16759 16767. 2024.
[81] Yuhan Zhang, Xiaode Liu, Yuanpei Chen, Weihang Peng, Yufei Guo. Xuhui Huang, and Zhe Ma. Enhancing representation spiking neural networks via similartty-sensitive contrastive learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16926-16934, 2024.
[82] Zhirun Zheng, Zhetao Li, Hongbo Jiang. Leo Yu Zhang, and Dengbiao Tu. Semantic-award aware privacy- preserving online location trajectory data sharing. IEEE Transactions on Information Forensics and Security, 17:2256-223 2271,, 2022.
[83] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2921 2929, 2016.
[84] Chenlin Zhou, Han Zhang, Zhaokun Zhou, Liutao Yu, Liwei Huang, Xiaopeng Fan, Li Yuan, Zhengyu Ma, Huihui Zhou, and Yonghong Tian. Qkformer: Hicrarchical spiking transformer using qk attention. Advances in Neural Information Processing Systems, 37:13074-1 13098, 2024.
[85] Zhaokun Zhou, Yijie Lu, Yanhao Jia, Kaiwei Che, Jun Niu, Liwei Huang. Xinyu Shi, Yuesheng Zhu, Guoqi Zhaofei Yu, and Li Yuan. Spiking transformer with experts mixture. In Advances Neural Information Processing Systems, volume 37, pages 10036-10059, 2024.
[86] Zhaokun Zhou, Yuesheng Zhu, He Chao, Yaowei Wang, Shuicheng Yan, Yonghong Tian, and Li Yuan. Spikformer: When spiking neural network meets transformer. In The Eleventh International Conference on Learning Representations, 2023.
[87] Rui-Jic Zhu, Ziqing Wang, Leilani Gilpin, and Jason Eshraghian. Autonomous driving with spiking neural networks. Advances in Neural Information Processing Systems, 77:136782-136808 2024.
[88] Rui-Jie Zhu, Malu Zhang. Qihang Zhao, Haoyu Deng, Yule Duan, and Liang- Jian Deng. Teja-snn: Temporal-channel joint attention for spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2024.
[89] Yaoyu Zhu, Jianhao Ding, Tiejun Huang, Xiaodong Xie, and Zhaofei Yu, Online stabilization of spiking neural networks. In The Twelfth International Conference on Learning Representations, 2024.
[90] Zhengyang Zhuge, Peisong Wang, Xingting Yao, and Jian Cheng. Towards efficient spiking transformer: token sparsification framework for training and inference acceleration. In Forty-first International Conference on Machine Learning, 2024.
15

---

Forward palh
S
Hea(U" Vaa)
U
Wa Was
Membrane poteniial
U
Vanila spike feature $
Mambrane potential U/H+1
Un U12 U1 Us
511
0
1
0
Weight W
U1 U1 Ul
Un Um Uy U2i
0
0
0
Wu Wiz Wis
U WS
U21 Us2 Uss U/
Usi Us
0
Wa War Wa
Us Us. UsA Usi
Un Un Un Uni
0
0
0
Wa Wa Way
Un Up Us Us
Partial derivalive of loss membrane potential
Backward path
8L
8L
Hea (U!)
8L
61 618 83 du
85
oW2
81 612 6ix 61s f2: dig dgg dec
BL
Wn + Wn.n1 Wn + Wz
n
8g: das åss dsc
des
Vanilla forward and backward propagation
Forward path
5
Hea(U Vin)
U
Wis Ws
Membrane patential U
Un U12 Una Usi
S11
Masked spike feature
0
0
1
0
Wu Wiz Wis
Weight W
Membrane potential 171+1
Un U1 Un Ui
Un Um Ug Upi
0
0
0
Wa
U WS
Un
Looa
Usi Us Us Usi
0
0
0
Usi U12 Us Usi
Un Un Un Uni
0
Un Up Ugy Usi
Backward path
8L
0L
0L
61s + 852 + 84s
80
8S
aW 28
MI-TRQR forward and backward propagation
Figure 5: Forward and backward process in vanilla and MI-TRQR method. We detail the forward and backward propagation processes of a layer of the SNN network, only considering the calculation in the spatial domain. When performing convolution calculations, the padding (without drawing) and stride are set to 1. A masked spike Sn is marked in red. The differences in the computing formulations between different methods are enclosed by boxes of various colors.
A
Mutual Information
Mutual information (MI), first formalized in Shannon's information theory [51] provides general- ized dependence measure between random variables. Unlike measures limited to linear relationships, MI captures complex statistical dependencies through probability distribution analysis. This property makes it suitable for quantifying similarity between discrete spike features in SNNs. The MI between two spike features S and S' is expressed as:
p(s. p(s)p(s')
I(S:S)= = Σ Σ p(s, s')log S'S
(10)
where s and s' indicate the values in feature S and S", respectively. p(s, is the joint probability distribution, and p(s) and p(s') are the marginal probability distributions. MI satisfies two fundamen- tal properties: (1) Non-negativity (I(S; > ensures meaningful information measures, and higher MI value indicates greater similarity. I(S 5') = 0, if and only if S and S' are independent random variables. (2) Symmetry (I(S; S') I(S"; S)) reflects bidirectional dependence.
B Analysis of MI-TRQR
B.1 Analysis of Forward Propagation
SNNs become promising alternative to ANNs, contributing to their advantages in energy efficiency. Unlike ANNs, SNNs achieve inter-layer communication with binary signals (0-nothing or 1-spike) 36 As kind of neuromorphic computing algorithm, SNNs only compute sparse spike-based accumulation (AC) and avoid handling zero-value inputs (48,, such as the calculation of membrane potential U = W as shown in Fig. 5 The energy consumption of SNNs Eçonv is influenced by the spiking firing rates of input features. Referring to the equation in consumption of network is computed as follows:
45 the inference energy
16

---

N
M Σ
Enet EMAC Fon + EAC T
F Fo fro +
FLFC frFe) m=1
(11)
where FLCcoue FLFC. frem and fr the FLOPs and spiking firing rate of the n-th convolution and m-th FC layer. and M are the total number of convolution and FC layers. EMAC and EAC are the energy consumption of MAC and AC operation. Following the works 774 19 assume that AC = 4.6pJ and F:AC = 0.9pJ.
In forward propagation, our MI-TRQR removes some spikes. We can see the effect on the calculation of membrane potential U
Vanilla: U U 22 = W + W3
W 11 + W 13 - W32
(12)
MI-TRQR:
(13)
where indicates the layer number. We that MI-TRQR requires less accumulation than the vanilla method.
B.2 Analysis of Backward Propagation
In the backward propagation, discuss the partial derivative of loss on different variables with different methods in the following three cases. Firstly, we denote the partial derivative loss on the 1 1-th layer membrane potential U++1 as:
ØL
(14)
oUl+1
where 8 partial derivative, as shown Fig.
5
Case 1: The partial derivative of loss on weight. Similar to ANNs, the partial derivative is:
0L
ΣΣ i=1==
d.g S S4tm 2
(15)
AW
where H and W indicate the height and width of features. The difference is that the spike feature S a sparse binary tensor, meaning the computing of Eq. 15 spike-based accumulation. For example, with different methods, the partial derivative of loss Wa 22 is:
aL
Vanilla:
0 + 832 844 811
(16)
W
W22
22
MI-TRQR:
513 + 832 + 844
(17)
W 22
We can observe that the gradient calculation of MI-TRQR requires less accumulation than the vanilla method, meaning the parameter update in MI-TRQR removes the redundant/invalid gradient in the vanilla method.
Case 2: The partial derivative of loss on spike features is computed with:
0L
W i.j W m
(18)
as m.
==1==1
where indicates the size of the convolution kernel. However, the calculation of this partial derivative has precondition: The input feature needs to involve the forward calculation, which means that the input feature must non-zero In other words, the gradient of the zero input zero. Thus, the partial derivative of loss the spike S11 is:
17

---

AL
Vanilla: S
W 111222 + W W20021 + W W21012 + W W22011
(19)
AL S
MI-TRQR:
(20)
We that the partial derivative of loss the removed spike in the two methods is completely different. Thus, we perceive that removing a spike means removing its derivattie/gradient.
Case 3: The partial derivative of loss the 1-th layer membrane potential U' easily obtained based on case 2. This partial derivative can be represented as:
OL OL os au as au
aL Hea' (Uea
(21)
as
where Hea'(U') indicates the derivative of Hea(U"). Specifically, with different methods, the partial derivative of loss on the membrane potential U is:
aL
OL
Vanilla:
Hea' Hea
(22)
Un usn
aL
MI-TRQR:
Hea' 0
(23)
U
We can see that the removed spike can influence the gradient of the membrane potential at the previous layer.
Summary. Through the above formula derivation and analysis, we can conclude that removing the spikes after the last convolutional layer affect the derivative of the previous membrane potential, and further affect the weight gradient and parameter updates at the previous layers. this way, MI-TRQR can learn compact and powerful representations in SNNs.
C Experiments
C.1 Experimental Setup
Datasets.
CIFAR10/100 26), two small datasets, contain 50,000 training and 10,000 test samples. ImageNet 6 is large-scale dataset of about 1.3 million images (1.25 million for training and 0.5 million for testing) across 1,000 classes. CIFAR10-DVS [11, an event-stream dataset converted from CIFAR10 with dynamic vision sensors, includes 10,000 event streams in 10 classes. DVS128 Gait (59) contains 4200 samples in 20 classes from 21 volunteers. Electricity consumption measured in kilowatt-hours (kWh).
28 captures hourly electricity
Implementation. We adopt the PSN 14 and STMixer 7 as baselines of the classification task. We adopt the CPG-PE [35] the baseline of the time-series forecasting task. We follow their experimental setup, such as network architecture, training methods, data preprocessing, etc. All experiments were conducted on Ubuntu 20.04.6 LTS server with 8 NVIDIA GeForce RTX 3090. The MI dependence is calculated with the TorchMetrics package.
For DVS128 Gait, we designed small 3B-Net. Its structure is C128k33s1-N--PLIF-SSE Block- MPK252#3-FC20 Here, c12833 denotes convolution layer with channels 128, kernel size 3, and stride 1. PLIF refers to the PLIF spiking neuron [3]. MPk2s2 represents the max pooling with kernel size 2 and stride SEW Block is custom-designed SEW block, expressed -32
c128k3s1 1---3-s----PPA- -NN-LIF. The symbol #3 indicates that the structure in is repeated three times.
18

---

Table 7: Comparison with other methods CIFAR10/100.
Dataset
Method
Backbone
T
Accuracy (1%)
Power (mmJ)
TAB 24
ResNet19
4
94.76
RevSFormer
80
RevSFormer-4-33
4
95,34
TCJA-SNN
88
ResNet18
4
95.60
CIFAR10
PSN 14
Modified PLIF Net
4
95.32
1.32
MI-TRQR (Ours)
Modified PLIF Net
4
95.83+10.10
0.35
STATA 90
Spikingformer
4
95.8
STAute 29
SDTT22551
4
96.03
STMixer
STMixer-4-884-22
96.01 4
0.95
MI-TRQR (Ours)
STM4xer-4-884-2
4
96.64+1.11
0.51+10.08
SLT-TET ||
ResNet19
4
75.01
A 23
VGG11
4
76.18
LietE-SNN B2
6
77.10
ResNet18
4
75.75
0.43
CIFAR 100
PSN 14
ResNet19
4
76.14
1.78
ResNet18
4
76.70
0.31
MI-TRQR (Ours)
ResNet19
77.70 4
1.24:10.07
ST [6
ST-4-384
4
79.69
SNN-ViT VIT 58
SDT
4
80.1
SDT+SEMM 85
SDT
4
80.26
STMixer 7
STMixer-4-384-32
4
81.87
1.08
MI-TRQR (Ours)
STMixer-4-34--22
4
83.06+10.11
0.77+10.88
Table 8: Impact of local redundancy (LR) and global redundancy (GR) on CIFAR10.
Method
LR
GR
T
Accuracy (1%)
Firing rate (1%)
PSN 14
4
95.32
16.39
V
4
95.53 (10.21)
10.59 (135.39)
MI-TRQR (ours)
/
4
95.77 (10.45)
8.98 (J45.21)
/
/
4
95.83 (10.51)
5.36 (J67.30)
C.2 Experimental Results on CIFAR10/100
C.2.1 Comparison with other methods
CIFARIO. Tab. 7 presents the classification performance and energy consumption of various methods on CIFAR-I0. MI-TRQR achieves 95.83 % accuracy, outperforming PSN (95.32%) by +0.51% while reducing energy consumption by 73.48 % (0.35mJ vs. 1.32mJ). With the STMixer- 4-384-32 backbone, MI-TRQR attains 96.64% % accuracy, surpassing STMixer (96.01 %) by 0.63% while reducing energy consumption by 46.3% (0.51 mJ vs. 0.95 mJ).
CIFAR100. As shown in Tab. 7 MI-TRQR consistently outperforms PSN across both ResNet18 and ResNet19 backbones, achieving 76.70% and 77.70% accuracy respectively, compared 75.75% and 76.14% of PSN, while also reducing power consumption from 0..3 mJ and 1.78 mJ to 0.31 mJ and 1.24 mJ. With the STMixer-4-384-39 backbone, MI-TRQR attains 83.06 % accuracy, outperforming the original STMixer (81.87%) by 1.19% % while reducing energy consumption by 28.70 % (0.77 mJ vs. 1.08 mJ).
C.2.2 Ablation Study on CIFAI10
We conduct ablation studies on CIFARIO to validate the effectiveness our redundancy designs in MI-TRQR. As summarized in Tab. g both local and global redundancy contribute to performance
19

---

Number of spikes
120000
Number spikes
160000
MI-TRQR (ours)
140000
PSN
100000
MI-TRQR (ours)
120000
80000
PSN
100000
60000
80000
60000
20000
200
400
600
800
1000
200
400
600
800
1000
Epochs
Epochs
(a) Number of spikes during training at t == 1.
(b) Number of spikes during training at 2.
200000
200000
Number of spikes
175000
Number spikes
150000-
150000
125000
MI-TRQQ (ours)
125000
MI-TRQR (ours)
100000
PSN
100000
PSN
75000
50000
50000
25000
600
800
1000
200
400
600
1000
Epochs
Epochs
(c)
Number of spikes during training t 3.
(d) Number of spikes during training at 4.
0.16
MI-TRQR (ours)
0.14
PSN
Firing rate
Firing rate
0.12
MI-TRQR (ours)
0.110
PSN
0.08
0.06
0.04
200
600
1000
Layer
Layer
Layer 4 Modified PUFF Net
Layer
Layer
Layer
Epochs
(e) Network spiking firing rate during training.
(I) Layer-wise spiking firing rate the network.
Figure 6: Comparison of the number of spikes and firing rate in different methods CIFARIO.
improvements over the PSN [4]. Using local redundancy alone improves accuracy by 0.21 % while reducing the firing rate by 35.39%. Feature redundancy demonstrates stronger effects, achieving 0.45% accuracy gain with 45.21% % fewer spikes. Notably, combining both mechanisms yields the best results our full model attains 95, 53.8%% accuracy (+0.51%) with only 5.36% firing rate, representing 67.30% reduction in spike activity. The number of spikes and firing rates in various methods are shown in Fig. 6 Specifically, the number of spikes four timesteps during training is shown from Fig. [a to 6d It can be seen that our MI-TRQR consistently fired fewer spikes than PSN at each timestep. The overall spiking firing rates during training are plotted Fig. [6e| The firing rate of our MI-TRQR is reduced by 67.29% compared to that of PSN (5.36% vs. 16.39%). The layer-wise firing rates of the Modified PLIF Net are presented Fig. 6f Our MI-TRQR has lower firing rate than PSN at every layer of the Modified PLIF Net.
C.3 Extended Ablation Studies
C.3.1 Spike normalization
To verify the effect of spike normalization in Eq. 5 we apply identical removal ratios to all spikes. Specifically, after obtaining the mask M, with Pt.,h.w. we shuffle its elements to generate random mask for spike removal. Results Tab. indicate that both the MI-based ratio and the removed spikes play essential roles.
In Tab. 10 we evaluate different statie penalty factors that increase with timesteps and serve as regularizer on the number of spikes. The results show that these static penalties effectively suppress redundant spikes and improve overall performance. Notably, MI-TRQR achieves higher accuracy with lower firing rate.
20

---

Table 9: Effect of a random mask CIFARIO-DVS when 7=4.
Method
Shuffle
T
Accuracy(1%)
Firing rate([%%
PSN
4
82.3
14.47
83.4 (11.1)
8.51 (J41.19)
MI-TRQR (ours)
4
84.0 (f1.7)
8.35 (L42.29)
Table 10: Effect of static penalty factors CIFARIO-DVS when 7=4.
Method
1=1 1
2
1=
= 4
Accuracy(1%)
Firing rate(
PSN 14
82.3
14.47
0.1
0.2
0.3
83.0 (10.7)
10.83 (125.11)
Static penalty
0
0.15
0.3
0.45
82.9 (10.6)
9.55 (L.34.00)
0.2
0.4
0.6
82.8 (10.5)
10.61 (126.68)
MI-TRQR (ours)
84.0 (11.7)
8.35 (U42.29)
Table 11: Effect of different techniques on CIFARIO-DVS when T=4.
Method
Technique
T
Accuracy(1%)
Firing rate([%)
PSN
82.3
14.47
Absolute distance
Euclidean
4
82.4 (f0.1)
11.74 ((18.87)
Linear similarity
Pearson
82.5 (f0.2)
10.92 (124.53)
Feature direction
Cosine
4
82.9 (f0.6)
9.59 (133.72)
MI-TRQR (ours)
MI
84.0 (t1.7)
8.35 (142.229)
Table 12: Effect of different factors on CIFARIO-DVS when T=4.
Method
Factor a
T
Accuracy(1%)
Firing rate(] (6)
PSN 14
4
82.3
14.47
0.5
4
84.5 (12.2)
10.66 (J26.33)
MI-TRQR (ours)
4
84.0 (11.7)
8.35 ((42.29)
2
4
82.7
(10.4)
8.16 (143.61)
C.3.2 Other techniques
We replace MI with simpler similarity measures, including Fuclidean similarity (via 1/(++Euclidean distance)), Pearson correlation, and Cosine similarity. As shown in Tab. [1] spiking firing rates still decrease with these alternatives, while MI-TRQR achieves higher accuracy and a lower firing rate.
C.3.3 Scaling factor
We multiply Pt,,, (Eq. 5) with an additional factor a, and evaluate the results under different setting As shown, the factor influences the trade-off between accuracy and firing rate.
in Tab. 12]
C.3.4 Mask generation strategies
In method, adopt Bernoulli based masks remove redundant spikes in data-dependem manner, without introducing additional hyperparameters. This design also admits clear theoretical in-
terpretation. To further validate its effectiveness, compare with a global redundancy based thresh- olding approach for pixel-wise spike removal. Specifically, for the feature S, E {0. 1) 1) x (1
1), we compute the global redundancy p (Eq. 3) and the local redundancy Pi (Eq. 4
Ph.... > rg. the spike feature Sy,, E 1) is set into O. As shown in Tab. 13 MI-TRQU with Hirnoulli-asse masks achieves better performance.
21

---

Table 13: Effect of mask generation strategies CIFARIO-DVS when 7=4.
Method
Mask type
T
Accuracy(t%)
Firing rate(L%)
PSN 14
82.3
14.47
Global threshold
82.6 (10.3)
8.97 (L38.01)
MI-TRQR (ours)
Bernoulli
84.0 (11.7)
8.35 (142.29)
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly state our contributions in the field of SNNs.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in the final section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
23

---

Justification:
The paper provides a complete proof of the proposed method in Appendix
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We will upload our code and data to Github upon acceptance, We have shown our experiment results in the Experiment Section, which can be reproduced by referring to the submitted code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
24

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The dataset used in this article publicly available, and the source code will be uploaded to ensure that others can reproduce the experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.cc/pblle pwwppppguuuiiieeeee http:////uiiees//eeSuuuuiiis
(CodeSubiisss wwtauummmsssooni ssiiiioiiccy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have shown experimental settings and implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We report the mean as well as the standard deviation accuracy in experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
25

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The energy consumption is provided the experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http://///eerree.cccc///////bbia
Answer: [Yes]
Justification: We have read and confirmed that the research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
26

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The original paper for datasets we used are all cited.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcone com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
27

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[NA]
Justification:
We adopt public datasets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
28

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The paper does not involve LLMs as any important, original, or non-standary components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standart components.
Please refer to our LLM policy ftttps://neuriiee /t/CConffreeces//2222 for what should or should be described.
29


Reviews:
[{"review_id": "ych1dWLfCa", "reviewer": "Reviewer_5G1g", "summary": "Based on earlier work that identifies redundancy among the learned spike features, the paper proposes a new nonlinear measure of distributional similarity - that of mutual information between the spike features across times. They measure the mutual information locally - between the same spike feature across timesteps, and globally across different features.  Based on these techniques, the authors propose a pruning (masking) technique for spikes and show that they can get higher accuracy and efficiency, along with lower spike rates on both event driven and static image data.", "questions": "I am happy to revise my score based on the answers to these questions, ordered by importance:    \n1. Reliability of the mutual information measuring technique\n    1. Measuring mutual information is hard with sparse high dimensional data, since we just cannot have enough samples.  To the best of my understanding, how mutual information is measured is not mentioned anywhere in the equations, algorithms, main text or appendix. More details, including equations, and number of samples are required to complete this section.\n    2. There need to be  some study done to verify that mutual information, as captured in this manuscript, is actually performing as expected.  For instance, the compelling reason for redundancy is for static images as they are duplicated across timesteps. However, we see similar impacts of MI-TRQR for both event driven and static datasets. Would we not have expected to see MI-TRQR be much more successful at reducing spikes for data with more redundancy (static duplicated images)\n    3. What is the computational cost of calculating this mutual information? Since we have to do this during training, how much longer does training take now?\n2. Ablation of the role of the spike normalization\n    1. The authors do a good job of ablating the effect of local and global redundancy, and show that both help. However, I am curious about the role of spike normalization as shown in equation 5. This aims to suppress only the spikes occurring at the later layer, and if we applied the same ratios without masks to random spikes or to all spikes (without the mask, what would the accuracy be?\n    2. If we had a static penalty of the number of spikes that increased with time steps as a regularizer, what would be the accuracy then at iso-firing rate?\n3. Did we need to do MI based pruning? What would have happened if we swapped out MI with simpler techniques (perhaps even linear). Would spiking rate still have gone down? The ablations are done only against the whole network, without any spike pruning? \n4. Did the resulting network actually have lesser redundant spike features? If you did the qualitative analysis and/or measured MI for the original and new network, would they be lesser correlated, perhaps even linearly?   \n\nGood to have, but lower impact on rating.    \n    1. Why Bernoulli? Why not threshold against a value?     \n    2. I don’t understand the placement experiment (line 208-211) Can the authors explain why if the reduction in spikes is placed after only at the last layer (layer 8 IIUC) the firing rate is suppressed more? Why not across all layers? I would have intuitively expected more redundancy in lower level features than high level features.    \n    3. [Minor] Algorithm 1 is not referenced at any point in the main text.   \n    4. [Minor] Tables should occur on the same page as they are referenced, ideally", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The experiment settings are comprehensive - they cover direct training and conversion, and both event driven and static image datasets.\n2. The paper is really clearly written, with a sensible flow.\n3. The metric of mutual information makes intuitive sense, as information centric techniques can reveal correlations that simple techniques such as Pearson correlation cannot\n4. The results show good improvement in efficiency, and it is measured both by firing rate and power\n\nWeaknesses\nSee Questions", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for taking the time to review our paper. Our responses to your questions are as follows:\n\n**1. Reliability of the mutual information measuring technique**\n\n**1.1. Question: Details of mutual information.**\n\nResponse:\n\n(1). We provide the equation of mutual information (MI) in Appendix A (line 452). \n\n(2). The MI calculation is implemented using the **TorchMetrics** package (line 525). The implementation code is: \n\n```python\nMI_value = torchmetrics.functional.clustering.mutual_info_score(S1, S2)\n```\n\nwhere S1 and S2 are two spiking features. $S1,S2\\in\\lbrace0,1\\rbrace^{C×H×W}$ and $S1,S2\\in\\lbrace 0,1\\rbrace ^C$ when computing global and local redundancy. MI_value is a scalar, indicating the MI between $S1$ and $S2$. \n\n(3) It does need many samples for the computation of MI. The MI of two random variables is a measure of the mutual dependence between the two variables. Therefore, the calculation of MI only needs two spike features.\n\n**1.2. Question: Impacts of MI-TRQR for both event-driven and static datasets.**\n\nResponse: The reviewer points out the similar impacts for both event-driven and static datasets. We believe the reviewer is referring to the reduction in energy consumption on CIFAR10-DVS (**37.50%** with T=4) and ImageNet (**24.82%** on ResNet50). We argue that the CIFAR10-DVS is a small dataset (10,000 samples in 10 classes), while ImageNet is a large-scale dataset (about 1.3 million images in 1,000 classes). Therefore, they are not directly comparable. We also show the impact on CIFAR10 (60,000 samples in 10 classes) in Appendix C.2. MI-TRQR reduces energy consumption by **73.48%**. Therefore, our MI-TRQR achieves the expected results, in that it is much more successful at reducing spikes for data with more redundancy (static duplicated images).\n\n**1.3. Question: The computational cost of calculating mutual information.**\n\nResponse: We show the time of training an epoch using different methods with a single NVIDIA RTX 3090 GPU:\n\n|Dataset|Method|Network|Local|Global|Time|\n|---|---|---|---|---|---|\n|CIFAR10-DVS|PSN|VGG|-|-|24.1s|\n||MI-TRQR|VGG|-|$\\checkmark$|24.8s|\n|||VGG|$\\checkmark$|-|51.1s|\n|||VGG|$\\checkmark$|$\\checkmark$|51.9s|\n|ImageNet|PSN|VGG|-|-|68.9min|\n||MI-TRQR|VGG|-|$\\checkmark$|70.4min|\n|||VGG|$\\checkmark$|-|78.5min|\n|||VGG|$\\checkmark$|$\\checkmark$|79.0min|\n\n**2. Ablation of the role of the spike normalization**\n\n**2.1. Question: The same ratios without masks to random spikes or to all spikes**\n\nResponse: Thanks for your profound thinking.\n\n(1) The mask is one concrete strategy to remove redundant spikes; other mechanisms could also be used. The reason why we use the mask is that matrix calculation is more efficient in current machines. \n\n(2) When only global redundancy is used to remove redundant spikes, the removal within $S_t\\in\\lbrace0,1\\rbrace^{C×H×W}$ at timestep $t$ is essentially random. \n\n(3) We apply the same ratios to all spikes. Specifically, after obtaining the mask with $p_{t,h,w}$ (Eq. 5), we shuffle all elements to generate a random mask for spike removal. \n\n|Method|Shuffle|Accuracy|Firing rate|\n|---|---|---|---|\n|PSN|-|82.3|14.47|\n|MI-TRQR|$\\checkmark$|83.4|8.51|\n||-|84.0|8.35|\n\nWe observe that both the MI-based ratio and the removed spikes are crucial.\n\n**2.2 Question: Static penalty of the number of spikes.**\n\nResponse: We use different static penalty factors:\n\n|Method|$t$=1|$t$=2|$t$=3|$t$=4|Accuracy|Firing rate|\n|---|---|---|---|---|---|---|\n|PSN|-|-|-|-|82.3|14.47|\n|Static penalty|0|0.1|0.2|0.3|83.0|10.83| \n||0|0.15|0.3|0.45|82.9|9.55| \n||0|0.2|0.4|0.6|82.8|10.61| \n|MI-TRQR|-|-|-|-|**84.0**|**8.35**|\n\nStatic penalties are indeed effective at reducing redundant spikes. Notably, MI-TRQR achieves better results on accuracy and firing rate. \n\n**3. Question: Do MI based pruning? Swap out MI with simpler techniques (perhaps even linear). Would spiking rate still have gone down? The ablations are done only against the whole network, without any spike pruning?**\n\nResponse:\n\n(1) Spike removal and pruning serve distinct purposes: pruning discards low-impact weights, whereas our method removes redundant spikes from features. This distinction allows the two to be applied together in a complementary manner.\n\n(2) We swap out MI with simpler techniques, such as Euclidean similarity (via 1/(1+Euclidean distance)), Pearson correlation, and Cosine similarity. \n\n|Method|Technique|Accuracy|Firing rate|\n|---|---|---|---|\n|PSN|-|82.3|14.47| \n|Absolute distance|Euclidean|82.4|11.74|\n|Linear similarity|Pearson|82.5|10.92|\n|Feature direction|Cosine|82.9|9.59| \n|MI-TRQR|MI|**84.0**|**8.35**|\n\nWe can see that the spiking rate still goes down using simpler techniques. Notably, MI-TRQR performs better.\n\n(3) Ablation with spike pruning.\n\nWe multiply $p_{t,h,w}$ (Eq. 5) by an additional factor $a$, and the results of different factors are:\n\n|Method|a|Accuracy|Firing rate|\n|---|---|---|---|\n|PSN|-|82.3|14.47|\n|MI-TRQR|0.5|84.5|10.66|\n||1|84.0|8.35|\n||2|82.7|8.16|\n\nWe can see that factor $a$ affects the balance between accuracy and firing rate.\n\n**4. Question: Lesser redundant spike features? Lesser correlated, perhaps even linearly?**\n\nResponse:\n\n(1) Yes. The resulting network actually has fewer redundant spike features. We show the MI for the original and new networks on CIFAR10-DVS and ImageNet in Fig. 1 and Fig. 4, respectively. Specifically, the spike features of the original network at four timesteps on CIFAR10-DVS are placed in Fig. 1(a), and their MI matrix is placed in Fig. 1(b). The spike features of the new network at four timesteps on CIFAR10-DVS are placed in Fig. 1(c), and the measured MI is placed in Fig. 1(d). We show the results on ImageNet using the same layout in Fig. 4. As shown in Fig. 1, MI-TRQR reduces the MI on CIFAR10-DVS by about **24%** (from 0.1507 to 0.1134) and on ImageNet by about **79%** (from 0.5837 to 0.1081). The related statements about Fig. 1 are in lines 28, 50, and 62.\n\n(2) Using the results in Question 3(3), we compute the MI of spike features (Fig. 1) using different $a$.\n\n$a$=0.5:\n\n|||||\n|---|---|---|---|\n|0.6167|0.1378|0.1201|0.1122|\n|0.1378|0.6056|0.2203|0.1901|\n|0.1201|0.2203|0.6033|0.2081|\n|0.1122|0.1901|0.2081|0.6003|\n\n$a$=2:\n\n|||||\n|---|---|---|---|\n|0.5097|0.1129|0.0993|0.0922|\n|0.1129|0.5064|0.1875|0.1801|\n|0.0993|0.1857|0.5009|0.2010|\n|0.0922|0.1801|0.2010|0.4987|\n\nWe can see that it is not linear.\n\n**5. Question: Why Bernoulli? Why not threshold against a value?**\n\nResponse: \n\n(1) Bernoulli-based masks admit a clearer theoretical interpretation. In contrast, a threshold lacks such grounding. \n\n(2) Bernoulli-based masks remove redundant spikes in a data-dependent manner and avoid introducing extra hyperparameters. The specific threshold is a hyperparameter that requires extensive studies to find the optimal configuration on different datasets. \n\n(3) To further verify the effectiveness of our method, we define a global redundancy-based threshold for pixel-wise spike removal. Specifically, for the feature $S_t\\in\\lbrace0,1\\rbrace^{C\\times H\\times W} (t>1)$, we compute the global redundancy $p_t^g$ (Eq. 3) and the local redundancy $p_{t,h,w}^l$ (Eq. 4). If $p_{t,h,w}^l>p_t^g$, the spike feature $S_{t,h,w}\\in\\lbrace0,1\\rbrace^C$ is set to 0. \n\n|Method|Accuracy|Firing rate|\n|---|---|---|\n|PSN|82.3|14.47|\n|MI-TRQR with threshold|82.6|8.97|\n|MI-TRQR with Bernoulli| **84.0**|**8.35**|\n\nThese results demonstrate that our method with Bernoulli performs better.\n\n**6. Question: The placement experiment (line 208-211). Why not across all layers?**\n\nResponse: Thanks for your consideration.\n\n(1)\tWhy is our MI-TRQR placed after only the last layer? \n\nPrior studies [1, 2] show the final convolutional layer features are most critical for correct classification, while earlier layers mainly learn reusable general representations and deeper layers become highly specific. Placing MI-TRQR after the last layer therefore helps the classifier directly filter out redundant spikes, and this decision can in turn dynamically guide the removal of redundancy in shallower layers. Its rigorous analysis and derivation are provided in Appendix B, a similar statement in line 160.\n\n(2)\tWhy not across all layers? \n\nFirst, redundancies at shallow and deep layers are not equivalent. Specifically, the redundancy identified in shallow layers may be useful in deep layers. Features that are considered useful in shallow layers may be useless in deep layers. \n\n(3) The computation of probability $p_{t,h,w}$ (Eq. 5) is spatiotemporally dependent, with a computational complexity of $\\mathcal{O}(T)$ for global redundancy and $\\mathcal{O}(T×H×W)$ for local redundancy. Therefore, inserting the module into earlier layers leads to higher computational cost. We provide the corresponding time of training an epoch based on Tab. 5. Due to time constraints, we only obtain the results of the module inserted after layers 6 and 8. \n\n|Method|Placement|Accuracy|Firing rate|Training time|\n|---|---|---|---|---|\n|PSN|-|82.3|14.47|24.1s|\n|MI-TRQR|After layer 4|82.8|11.52|597.3s|\n||After layer 6|83.4|10.18|171.6s|\n||After layer 6+8|83.5|9.74|222.6s|\n||After layer 8|**84.0**|**8.35**|**51.9s**|\n\nWe can see that adding the module to earlier layers introduces substantial computational cost, yet yields no gains in accuracy or firing rate reduction.\n\n**7. Question: Algorithm 1 is not referenced.**\n\nResponse: Algorithm 1 is referenced at line 159.\n\n**8. Question: Tables should occur on the same page as they are referenced, ideally.**\n\nResponse: Thanks for your advice. We will try our best to improve the layout.\n\n**References**\n\n[1] Squeeze-and-excitation networks. CVPR 2018\n\n[2] Learning deep features for discriminative localization. CVPR 2016"}, {"review_id": "e3FlePhKUf", "reviewer": "Reviewer_CMhp", "summary": "This paper proposes a novel Mutual Information-based Temporal Redundancy Quantification and Reduction approach (MI-TRQR) for energy-efficient SNNs. Extensive experimental results demonstrate that MI-TRQR achieves superior performance across neuromorphic data classification, static data classification, and time-series forecasting tasks.", "questions": "See in weakness.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The topic is quite interesting and offers a novel perspective for improving SNN efficiency. The paper is well-written and well-structured. \n\nHowever, I have a few suggestions and questions for the authors:\n\nThe ablation study shows that applying the MI-TRQR module after the last convolutional layer yields the best results. Could the authors provide further insight into this observation? This result is somewhat puzzling, as temporal redundancy likely exists in intermediate layers as well. In addition, have the authors considered applying MI-TRQR at multiple stages throughout the network?\n\nRegarding the reported power consumption: how is the power measured? Is it obtained from real hardware or through simulation? If the latter, please clarify the simulation method and include the formulation used to estimate power consumption.", "was_revised": true, "final_justification": "The authors have addressed my questions. I will keep the positive review for this paper.", "rebuttal": "Thank you very much for taking the time to review our paper.  Our responses to your questions are as follows:\n\n**1. Question: The ablation study shows that applying the MI-TRQR module after the last convolutional layer yields the best results. Could the authors provide further insight into this observation? This result is somewhat puzzling, as temporal redundancy likely exists in intermediate layers as well. In addition, have the authors considered applying MI-TRQR at multiple stages throughout the network?**\n\nResponse: Thanks for your consideration. We address this concern in the following aspects:\n\n(1)\tWhy do we apply the MI-TRQR module after the last convolutional layer?\n \nPrior studies [1, 2] show the final convolutional layer features are most critical for correct classification, while earlier layers mainly learn reusable general representations and deeper layers become highly specific. Placing MI-TRQR after the last layer therefore helps the classifier directly filter out redundant spikes, and this decision can in turn dynamically guide the removal of redundancy in shallower layers. Its rigorous analysis and derivation are provided in Appendix B, a similar statement in line 160.\n\n(2)\tWhy we did not apply MI-TRQR at multiple stages? \n\nRedundancies at shallow and deep layers are not equivalent. Specifically, the redundancy identified in shallow layers may be useful in deep layers. Features that are considered useful in shallow layers may be useless in deep layers. \n\n(3) The computation of probability $p_{t,h,w}$ (Eq. 5) is spatiotemporally dependent, with a computational complexity of $\\mathcal{O}(T)$ for global redundancy and $\\mathcal{O}(T×H×W)$ for local redundancy. Therefore, inserting the module into earlier layers leads to higher computational cost. We provide the corresponding time of training an epoch based on Tab. 5. Due to time constraints, we only obtain the results of the module inserted after layers 6 and 8. \n\n|Method|Placement|Accuracy|Firing rate|Training time|\n|---|---|---|---|---|\n|PSN|-|82.3|14.47|24.1s|\n|MI-TRQR|After layer 4|82.8|11.52|597.3s|\n||After layer 6|83.4|10.18|171.6s|\n||After layer 6+8|83.5|9.74|222.6s|\n||After layer 8|**84.0**|**8.35**|**51.9s**|\n\nWe can see that adding the module to earlier layers introduces substantial computational cost, yet yields no gains in accuracy or firing rate reduction.\n\n**2. Question: Regarding the reported power consumption: how is the power measured? Is it obtained from real hardware or through simulation? If the latter, please clarify the simulation method and include the formulation used to estimate power consumption.**\n\nResponse: The power consumption is obtained through simulation. The simulation method follows prior works [3,4], and the estimation formulation is provided in Eq. 11 (page 15). We show it as follows:\n\n$$\nE_{net}=E_{MAC} \\cdot FL_{Conv}^{1} + E_{AC} \\cdot T \\cdot (\\sum_{n=2}^{N}FL_{Conv}^{n}\\cdot fr_{Conv}^{n}+ \\sum_{m=1}^{M}FL_{FC}^{m}\\cdot fr_{FC}^{m})\n$$\n\nwhere $FL_{Conv}^{1}$ is the FLOPs of first convolution layer, $FL_{Conv}^{n}$ and $FL_{FC}^{m}$ are the FLOPs of the $n$-th convolution layer and the $m$-th FC layer, $fr_{Conv}^{n}$, and $fr_{FC}^{m}$ are the firing rate of the $n$-th convolution layer and the $m$-th FC layer, respectively. $N$ and $M$ are the total number of convolution and FC layers. Following prior works [5,6], $E_{MAC}$=4.6pJ, $E_{AC}$=0.9pJ.\n\n\n**References**\n\n[1] Jie Hu, Li Shen, and Gang Sun, Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7132–7141, 2018.\n\n[2] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2921–2929, 2016.\n\n[3] Hangchi Shen, Qian Zheng, Huamin Wang, and Gang Pan. Rethinking the membrane dynamics and optimization objectives of spiking neural networks. Advances in Neural Information Processing Systems, 37:92697–92720, 2024.\n\n[4] Xuerui Qiu, Rui-Jie Zhu, Yuhong Chou, Zhaorui Wang, Liang-jian Deng, and Guoqi Li. Gated attention coding for training high-performance and efficient spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 601–610, 2024.\n\n[5] Mark Horowitz. 1.1 computing’s energy problem (and what we can do about it). In 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), pages 10–14. IEEE, 2014.\n\n[6] Bojian Yin, Federico Corradi, and Sander M Bohté. Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks. Nature Machine Intelligence, 3(10):905–913, 2021."}, {"review_id": "NIwLU652UJ", "reviewer": "Reviewer_nHVz", "summary": "This paper proposes a novel approach named “MI-TRQR” that learns sparse visual representations in a spiking neural network (SNN). This is motivated by the fact that in the context of neuromorphic processing, spikes are the main source of energy consumption. The authors characterize redundancy using the concept of mutual information and develop a framework that adaptively keeps only the less redundant spikes to learn efficient (and therefore inexpensive) visual representations. Crucially, the paper was tested on different datasets (Time-series data, CIFAR10-DVS, DVS Gait and ImageNet) with different network backbones. It achieves a competitive performance with other methods while significantly reducing energy consumption.", "questions": "1) I would kindly ask the authors to please clarify further how the weight recalibration happens in the network.\n\n2) A relevant body of work which has not been discussed in the paper is the one of predictive coding [1, 2]. In predictive coding, pixel-wise redundant spikes (and internal layer spikes) are removed, usually following some sort of minimization of the so-called free-energy [1]. I believe that the authors should at the very least cite this body of work and discuss differences between their work and some of the spiking counterparts of predictive coding [2], as both types of models aim at learning efficient visual representations through the removal of redundant spikes.\n\n3) I would kindly ask the authors to analyze the weaknesses/limitations of their work. \n\n4) I encourage the authors to modify the formatting of Sec. 4.1, as the text is not properly justified.\n\n[1] Bogacz, Rafal. \"A tutorial on the free-energy framework for modelling perception and learning.\" Journal of mathematical psychology 76 (2017): 198-211.\n\n[2] N'dri, Antony W., et al. \"Predictive Coding with Spiking Neural Networks: a Survey.\" arXiv preprint arXiv:2409.05386 (2024).", "limitations": "The authors do not address the weaknesses/limitations of their work. I suggest to the authors to add a paragraph describing such limitations, and provided some examples in the \"Strengths and Weaknesses\" section of this review.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n\n1) A new direct training model that learns efficient visual representations for spiking neural networks. The principle of the model is sound and derived from mathematical principles. \n2) An evaluation on multiple datasets with convincing results that are in the same range as other methods while significantly reducing the energy consumption of the network (see Tab. 1 with MI-TRQR achieving a higher accuracy with less energy consumptions than a PSN model that uses the same network). Results are averaged over multiple simulations which highlight their statistical significance.\n3) The study of the effect of local redundancy and global redundancy on accuracy (Table 4) gives an interesting perspective on how optimizing efficiency on local and global information affects information coding.\n\nWeaknesses:\n\n1) The addition of the MI-TRQR module, while reducing energy consumption leads to additional parameters, but the number of parameters (memory) could be a bottleneck depending on the application considered.\n2) I find it unclear how the weight recalibration takes place. It appears that the input spikes are still projected through some sort of residual connections (see Fig. 2, dashed red line). Equations 8 and 9 however appear to specify otherwise, as the spiking rate (which should be, I reckon the one after spike removal) of the network is used to calculate α. \n3) While the MI-TRQR module allows for the removal of input spikes, the module in itself is not spike-based as it calculates and optimizes continuous quantities, which I find unfortunate for the deployment of fully spiking-based neural network processes.", "was_revised": true, "final_justification": "The author's response addressed my concerns in terms of weaknesses and strengths of the work and also clarified the methodology of the module described in their work. I will therefore be maintaining my score.", "rebuttal": "Thank you very much for taking the time to review our paper. Our responses to your questions are as follows:\n\n**1. Question: The addition of the MI-TRQR module, while reducing energy consumption leads to additional parameters, but the number of parameters (memory) could be a bottleneck depending on the application considered.**\n\nResponse: Our MI-TRQR requires extra computational and memory overhead but no additional parameters. Our MI-TRQR is only applied during training, which is stated in line 525. At the inference stage, our method **behaves identically** to the baseline model. Therefore, our method **does not pose a memory bottleneck** in various applications. We will include the related statement (our module is only used during training) in the main text. \n\n**2. Question: I find it unclear how the weight recalibration takes place. It appears that the input spikes are still projected through some sort of residual connections (see Fig. 2, dashed red line). Equations 8 and 9 however appear to specify otherwise, as the spiking rate (which should be, I reckon the one after spike removal) of the network is used to calculate α.**\n\nResponse: \n\n(1) Yes. Your understanding is correct. The weight $a_t$ is computed based on the spiking firing rate of the network at the timestep $t$. Specifically, we count the number of spikes and total elements after each spiking neuron at every timestep and use this to derive the firing rate used for weight recalibration. \n\n(2) We apologize for the discontinuity of the dashed red line in Fig. 2. It should be connected to the same line in the upper left corner, indicating the network spiking firing rate. The break was introduced to avoid visual clutter in the middle part.\n\n**3. Question: While the MI-TRQR module allows for the removal of input spikes, the module in itself is not spike-based as it calculates and optimizes continuous quantities, which I find unfortunate for the deployment of fully spiking-based neural network processes.**\n\nResponse: Your question is very insightful. Our MI-TRQR calculates continuous quantities. Since current neuromorphic chips still support operations on continuous values. The chips are energy-efficient as long as the outputs of spiking neurons are binary (0 or 1). Therefore, our module remains compatible with the deployment of fully spiking-based neural network processes.\n\n**4. Question: I would kindly ask the authors to please clarify further how the weight recalibration happens in the network.**\n\nResponse: During training, at the $l$-th layer of spiking neuron, we count the number of spikes $N_{l,t}^{s}$ and total number of elements $N^e_{l,t}$ at each timestep. These values are used to calculate the timestep-wise firing rate $fr^{net}_t$:\n\n$$\nfr_{t}^{net} = \\frac{\\sum_{l=1}^{L} N_{l,t}^{s}}{\\sum_{l=1}^{L} N_{l,t}^{e}}\n$$\n\nwhere $t$ indicates the timestep. $L$ is the total number of spiking neuron layers. $fr_{t}^{net}$ is then used to calculate the weight in Eq. 8. We will provide a detailed demonstration in the final version.\n\n**5. Question: A relevant body of work which has not been discussed in the paper is the one of predictive coding [1, 2]. In predictive coding, pixel-wise redundant spikes (and internal layer spikes) are removed, usually following some sort of minimization of the so-called free-energy [1]. I believe that the authors should at the very least cite this body of work and discuss differences between their work and some of the spiking counterparts of predictive coding [2], as both types of models aim at learning efficient visual representations through the removal of redundant spikes.**\n\nResponse: Thank you very much for providing these excellent works. They indeed provide additional theoretical support for spike removal in our MI-TRQR. We will cite these references and revise the motivation (line 51) as follows in the final version:\n\n>Previous studies analyzing neural recordings from various monkey brain regions have shown that spike features are significantly less redundant with activity-dependent depression. **Predictive coding has also demonstrated the ability to learn efficient visual representations by removing pixel-wise redundant spikes [1,2].** Inspired by these works, we propose a parameter-free, plug-and-play MI-based Temporal Redundancy Quantification and Reduction (MI-TRQR) module, which is seamlessly integrated into the SNNs.\n\n**6. Question: I would kindly ask the authors to analyze the weaknesses/limitations of their work.**\n\nResponse: Thank you for your suggestion. We will include a limitation section in the final version.\n\n>Although our method does not introduce extra cost during inference, it requires computing mutual information between spike features during training, resulting in increased computational cost and longer training time. \n\n**7. Question: I encourage the authors to modify the formatting of Sec. 4.1, as the text is not properly justified.**\n\nResponse: Thank you for your suggestion. Due to space constraints, we arrange the text of Sec. 4.1 and Tab. 3 side by side. We will revise the formatting and improve layout quality in the final version.\n\n**References**\n\n[1] Bogacz, Rafal. \"A tutorial on the free-energy framework for modelling perception and learning.\" Journal of mathematical psychology 76 (2017): 198-211.\n\n[2] N'dri, Antony W., et al. \"Predictive Coding with Spiking Neural Networks: a Survey.\" arXiv preprint arXiv:2409.05386 (2024)."}, {"review_id": "l2U8IIfHjz", "reviewer": "Reviewer_wE4j", "summary": "This paper proposes a Mutual Information-based Temporal Redundancy Quantification and Reduction (MI-TRQR) module that reduces the computational overhead imposed by spikes. This method computes global and local mutual information in spiking neural networks (SNNs) to mask redundant spikes. The authors demonstrate through experimentation that the proposed method effectively reduces the number of spikes and improves performance on object classification and time-series forecasting tasks.", "questions": "See weaknesses.", "limitations": "The authors claim that their method has no limitations.", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. The proposed MI-TRQR module does not require additional attention mechanisms or other learnable parameters.\n2. Inserting the proposed module into the existing SNN reduces the number of spikes and improves performance.\n\nWeakness:\n1. **Theoretical analysis**. Although the authors claim that mutual information is an effective way to evaluate the similarity between discrete variables, there is a lack of theoretical analysis regarding the effectiveness of binary spikes. Does the limited range of representations of binary spikes affect the precision of mutual information?\n2. **Complexity and energy efficiency**. Reducing the number of 1-value spikes in the SNN can reduce deployment power consumption by avoiding addition operations caused by those spikes. However, computing mutual information requires a large number of multiplications and logarithmic operations (Equations 1 and 2), which consume more power than addition. In fact, this increases the overall inference power and complexity rather than decreasing them. Therefore, the authors should not rely solely on the number of spikes to calculate the theoretical power consumption.\n3. **Method rationality**. The authors retained all spikes from the initial time step and used them as references to calculate probabilities for subsequent time steps. However, these calculations did not consider interactions between other time steps. Furthermore, recent work [1,2] has shown that the performance of SNNs degrades dramatically when relying only on early time steps. Does this suggest that the limited amount of information in the first time step cannot be used as a reference benchmark?\n4. **Method effectiveness**. The authors point out that removing spikes results in an imbalanced distribution across time steps. Therefore, they reweight the weights for each time step. However, the authors calculated the weights using firing rates only and did not adjust them for the unbalanced distribution. Even more importantly, no ablation studies have verified the effectiveness of this reweighting.\n5. **Insufficient experimental comparisons**. Although the related work is described, the authors do not compare the proposed method with other lightweighting or redundancy reduction methods in the experimental section. This makes it difficult to verify whether the proposed method has any significant advantages. It is recommended that the authors add a detailed comparison with other redundancy reduction methods such as [3,4].\n6. **Insignificant performance gains**. The performance improvement on the time-series forecasting task is too minimal to determine whether the method is effective. More experimental evidence is recommended. Additionally, are the results reported in the paper the best results obtained from multiple experiments, such as CIFAR10-DVS? If so, please provide the average results.\n\n```\n[1] Rethinking Spiking Neural Networks from an Ensemble Learning Perspective. ICLR. 2025.\n[2] Efficient Logit-based Knowledge Distillation of Deep Spiking Neural Networks for Full-Range Timestep Deployment. ICML. 2025.\n[3] Inherent Redundancy in Spiking Neural Networks. ICCV. 2023.\n[4] Sparser spiking activity can be better: Feature Refine-and-Mask spiking neural network for event-based visual recognition. 2023.\n```", "was_revised": true, "final_justification": "Although the author responded to my concerns, these responses did not convince me. I do not believe that this paper has reached the standard for acceptance, and therefore I am maintaining my score.", "rebuttal": "Thank you very much for taking the time to review our paper. Our responses to your questions are as follows:\n\n**1. Question: Theoretical analysis. Although the authors claim that mutual information is an effective way to evaluate the similarity between discrete variables, there is a lack of theoretical analysis regarding the effectiveness of binary spikes. Does the limited range of representations of binary spikes affect the precision of mutual information?**\n\nResponse: Thanks for the meticulous comment. \n\n(1) Mutual information (MI) is a widely used non-linear metric for quantifying statistical dependence between discrete variables. In the context of binary spike features, where each element is assigned a value of 0 (no spike) or 1 (spike), MI remains theoretically valid and interpretable. \n\n(2) The state space of binary variables ({0,1}) does not limit the theoretical precision of MI. As established in classical information theory [1], MI measures the reduction in uncertainty of one variable given the knowledge of another. For binary variables, MI is bounded between 0 and 1, with 1 indicating identical variables and 0 indicating completely independent variables. Empirical evidence also supports the utility of MI: for example, MI increases monotonically with similarity intensity, even when applied to discrete or event-based representations [2]. In a word, the limited range of representations of binary spikes **does not affect** the precision of MI.\n\n**2. Question: Complexity and energy efficiency. Reducing the number of 1-value spikes in the SNN can reduce deployment power consumption by avoiding addition operations caused by those spikes. However, computing mutual information requires a large number of multiplications and logarithmic operations (Equations 1 and 2), which consume more power than addition. In fact, this increases the overall inference power and complexity rather than decreasing them. Therefore, the authors should not rely solely on the number of spikes to calculate the theoretical power consumption.**\n\nResponse: Yes. You are right. Computing mutual information requires a large number of multiplications and logarithmic operations. Nevertheless, as stated in line 525, our module is applied solely during training and has no impact on the inference stage. Therefore, it is reasonable to rely solely on the number of spikes to calculate the theoretical energy consumption. We will include the related statement (our module is only used during training) in the main text.\n\n**3. Question: Method rationality. The authors retained all spikes from the initial time step and used them as references to calculate probabilities for subsequent time steps. However, these calculations did not consider interactions between other time steps. Furthermore, recent work [3.4] has shown that the performance of SNNs degrades dramatically when relying only on early time steps. Does this suggest that the limited amount of information in the first time step cannot be used as a reference benchmark?**\n\nResponse: Thanks for your insightful and valuable thinking. \n\n(1) The interactions between other time steps.\n\nThe temporal information concentration (TIC) phenomenon indicates that, as training goes on, the amount of information at the first timestep increases while that of the other timesteps decreases. Therefore, we argue that especially in the later stages of training, the first time step carries the most critical information and can be used as a reference, whereas the others become negligible and unsuitable for reference. Accordingly, we measure the mutual information between the features at each subsequent timestep and those at the first timestep. Even when mutual information is measured between features from other timesteps, the resulting values are of limited significance, since both features convey little information on their own.\n\n(2) The amount of information in the first time step.\n\nWe study these two works [3.4] and find the related demonstration as follows:\n \n>We view this averaging operation as an ensemble strategy: averaging the different outputs of multiple models improves overall performance. [3]\n\n>Inspired by ensemble learning, viewing the mean output of SNNs as an ensemble aggregated through voting over time, it becomes apparent that the overall outcome across these temporal dimensions tends to improve as the accuracy at each individual timestep increases. [4]\n\nWe also agree that averaging the different outputs of multiple models improves overall performance. We argue that this is not in conflict with the TIC phenomenon [5], which demonstrates that information gradually concentrates on the first timestep as training goes on. In other words, the TIC phenomenon means that the improvement extent will gradually decrease as training goes on. Moreover, to avoid misunderstanding, we will modify the description on line 26 in the final version as follows:\n\n>SNNs rely on sequential timesteps to transmit temporal information, **and their outputs are typically averaged across time to improve accuracy [3,4]**. The temporally shared weights induce spatio-temporal invariance, introducing substantial feature redundancy and resulting in a high spiking firing rate, as shown in Fig. 1(a).\n\n**4. Question: Method effectiveness. The authors point out that removing spikes results in an imbalanced distribution across time steps. Therefore, they reweight the weights for each time step. However, the authors calculated the weights using firing rates only and did not adjust them for the unbalanced distribution. Even more importantly, no ablation studies have verified the effectiveness of this reweighting.**\n\nResponse: Thanks for your careful review. We conduct the ablation study to verify the effectiveness of this reweighting.\n\n| Method | Reweighting | Accuracy | Firing rate |\n| --- | --- | --- | --- |\n| PSN | - | 82.3 | 14.47 |\n| MI-TRQR | - | 83.4 | 9.22 |\n| MI-TRQR | $\\checkmark$ | 84.0 | 8.35 |\n\nWe can observe that the reweighting is effective. We will include the ablation study in the final version.\n\n**5. Question: Insufficient experimental comparisons. Although the related work is described, the authors do not compare the proposed method with other lightweighting or redundancy reduction methods in the experimental section. This makes it difficult to verify whether the proposed method has any significant advantages. It is recommended that the authors add a detailed comparison with other redundancy reduction methods such as ASA [6] and RM-SNN [7].**\n\nResponse: I have compared our method to ASA [6] on Gait. For CIFAR10-DVS, RM-SNN obtained 82.9% with $T$=10. Our method achieved 86.5%. We will include the comparison in the final version.\n\n**6. Question: Insignificant performance gains. The performance improvement on the time-series forecasting task is too minimal to determine whether the method is effective. More experimental evidence is recommended. Additionally, are the results reported in the paper the best results obtained from multiple experiments, such as CIFAR10-DVS? If so, please provide the average results.**\n\nResponse: Thanks for your careful consideration.\n\n(1) Although the performance improvement is not overwhelming, it is sufficient to demonstrate the effectiveness of our method to learn more compact and powerful representations. Because our MI-TRQR not only increases the accuracy but also **reduces the energy consumption** compared to multiple SNN baselines for various tasks. We provide much evidence in various ways. \n\n(2) We follow your advice and run three experiments. The result on CIFAR10-DVS with T=4 is 83.9$\\pm$0.08.\n\n**References**\n\n[1] Thomas M. Cover, and Joy A. Thomas. Elements of information theory {(2.} ed. 2006\n\n[2] Thomas Kreuz, Florian Mormann, Ralph G. Andrzejak, Alexander Kraskov, Klaus Lehnertz, and Peter Grassberger. Measuring synchronization in coupled model systems: A comparison of different approaches. Physica D, Nonlinear Phenomena, 2007, 225(1): 29-42.\n\n[3] Yongqi Ding, Lin Zuo, Mengmeng Jing, Pei He, and Hanpu Deng. Rethinking Spiking Neural Networks from an Ensemble Learning Perspective. ICLR. 2025.\n\n[4] Chengting Yu, Xiaochen Zhao, Lei Liu, Shu Yang, Gaoang Wang, Erping Li, and Aili Wang. Efficient Logit-based Knowledge Distillation of Deep Spiking Neural Networks for Full-Range Timestep Deployment. ICML. 2025.\n\n[5] Youngeun Kim, Yuhang Li, Hyoungseob Park, Yeshwanth Venkatesha, Anna Hambitzer, and Priyadarshini Panda. Exploring temporal information dynamics in spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8308–8316, 2023.\n\n[6] Man Yao, Jiakui Hu, Guangshe Zhao, Yaoyuan Wang, Ziyang Zhang, Bo Xu, and Guoqi Li. Inherent Redundancy in Spiking Neural Networks. ICCV. 2023.\n\n[7] Man Yao, Hengyu Zhang, Guangshe Zhao, Xiyu Zhang, Dingheng Wang, Gang Cao, and Guoqi Li. Sparser spiking activity can be better: Feature Refine-and-Mask spiking neural network for event-based visual recognition. 2023."}]

Author rebuttal:

Meta-review: ##  Summary
This paper proposes MI-TRQR, a novel, parameter-free, plug-and-play module designed to quantify and reduce temporal redundancy in Spiking Neural Networks (SNNs) to improve energy efficiency and accuracy. The method uses mutual information (MI) to measure redundancy between discrete spike features across timesteps on both local (pixel-level) and global (spatial feature-level) scales. Based on the quantified redundancy, MI-TRQR applies probabilistic masking to remove redundant spikes and recalibrates the final representation. Extensive experiments demonstrate that MI-TRQR achieves sparser spike firing, reduced energy consumption, and improved performance across diverse SNN architectures and tasks including neuromorphic data classification, static image classification, and time-series forecasting.  

## Strengths    
- **Novelty:** Introduces a mutual information based approach to quantify temporal redundancy in spike features, a fresh perspective in SNN efficiency optimization.    
- **Effectiveness:** Demonstrates simultaneous gains in accuracy and substantial reductions in spike rates and energy consumption across multiple datasets and architectures.    
- **Comprehensive evaluation:** Covers event-driven and static image datasets, including large-scale ImageNet, with ablation studies on local/global redundancy and spike normalization.    
- **Clarity:** Well-written and logically structured manuscript with clear explanations and detailed ablations.    
- **Compatibility:** The MI-TRQR module is parameter-free and only used during training, preserving inference efficiency and compatibility with existing SNN frameworks.  
  

  
## Weaknesses and Missing Elements    
- **Mutual information estimation concerns:** Questions raised regarding the validity of the MI calculation method on sparse binary spike data, requiring careful justification and validation.    
- **Computational cost:** MI calculation introduces additional training overhead; while inference cost is unaffected, training time increase may be significant.    
- **Theoretical analysis:** Limited theoretical discussion on why MI is especially suitable for binary spike features and how weight recalibration compensates for spike removal.    
- **Module placement:** Ablation shows best results when MI-TRQR is applied after the last convolutional layer, but deeper analysis on redundancy across layers and multi-stage application is limited.    
- **Comparison to related methods:** Lack of direct experimental comparison to other redundancy reduction or pruning methods in SNN literature reduces clarity on relative advantage.    
- **Limited discussion of limitations:** Initial submission lacks a dedicated section on method limitations and potential bottlenecks.  
 
  
## Discussion and Rebuttal Summary    
  
### Reviewers and Their Key Concerns:    
- **Reviewer 5G1g:**    
  - Raised a critical question on the correctness of the mutual information estimation approach for sparse high-dimensional binary spike vectors.    
  - Requested more detailed explanation of MI calculation, its reliability, and sample requirements.    
  - Asked about differences in effectiveness between static and event-driven datasets.    
  - Inquired about computational cost and efficiency impact during training.    
  - Questioned the role of spike normalization and whether simpler similarity measures would suffice.    
  - Requested ablations on MI-based pruning and evidence that redundancy is actually reduced.  
  
- **Reviewer CMhp:**    
  - Asked for clarification on why MI-TRQR module is applied only after the last convolutional layer and if multi-stage application was considered.    
  - Requested details on power consumption measurement methodology (simulation vs. hardware).    
  - Appreciated clarity and structure but desired more insight on module placement and efficiency metrics.  
  
- **Reviewer nHVz:**    
  - Requested clarification on weight recalibration mechanics and its theoretical basis.    
  - Raised concern that MI-TRQR operates on continuous quantities, potentially limiting fully spike-based deployment.    
  - Suggested citing and discussing related predictive coding literature.    
  - Requested analysis of weaknesses/limitations.    
  - Asked for formatting improvements.  
  
- **Reviewer wE4j:**    
  - Expressed concern about the theoretical analysis of MI effectiveness for binary spikes.    
  - Pointed out potential increase in training complexity and energy cost due to MI calculations.    
  - Questioned the rationality of using first timestep as reference in MI calculation.    
  - Requested ablation studies on reweighting effectiveness and comparison to other redundancy reduction methods.    
  - Noted limited performance gains on time-series forecasting and asked for statistical robustness of results.  
  
### Authors’ Responses:    
- Provided detailed explanation and equations for MI calculation using TorchMetrics and NPEET libraries, confirming correctness and consistency.    
- Argued MI is suitable for discrete binary variables and demonstrated effectiveness empirically.    
- Clarified computational overhead is limited to training phase; inference remains efficient.    
- Explained rationale for using first timestep as reference based on temporal information concentration (TIC) phenomenon and ensemble learning perspectives.    
- Conducted ablation studies showing spike normalization and reweighting improve accuracy and spike rate reduction.    
- Compared MI-TRQR to simpler similarity metrics and redundancy reduction methods, showing MI-based method performs best.    
- Presented module placement ablations showing best trade-off when applied after last convolutional layer due to computational cost and feature specificity.    
  
### Final Decision:    
The authors addressed major concerns with detailed explanations, additional experiments, and clarifications that validate the MI calculation and demonstrate practical benefits. Some questions about computational cost and theoretical framing remain but are reasonably mitigated by the fact that overhead is training-only. The paper presents a meaningful contribution toward energy-efficient SNNs by tackling temporal redundancy with a novel MI-based approach. The empirical results are convincing and robust across datasets and architectures. Given the borderline opinions of reviewers and the authors’ thorough rebuttal, this submission merits acceptance conditioned on addressing minor outstanding points in the final version.
Decision: Accept (poster)